# Deep Unsupervised Domain Adaptation: A Review of Recent Advances and
Perspectives

Xiaofeng Liua, Chaehwa Yoob, Fangxu Xinga, Hyejin Ohb, Georges El Fakhria, Je-
Won Kangb, Jonghye Wooa
a Gordon Center for Medical Imaging, Massachusetts General Hospital and
Harvard Medical School, Boston, MA, USA
b Dept. of Electronic and Electrical Engineering and Graduate Program in Smart
Factory, Ewha Womans University, Seoul, South Korea

###### Abstract

Deep learning has become the method of choice to tackle real-world problems in
different domains, partly because of its ability to learn from data and
achieve impressive performance on a wide range of applications. However, its
success usually relies on two assumptions: (i) vast troves of labeled datasets
are required for accurate model fitting, and (ii) training and testing data
are independent and identically distributed. Its performance on unseen target
domains, thus, is not guaranteed, especially when encountering out-of-
distribution data at the adaptation stage. The performance drop on data in a
target domain is a critical problem in deploying deep neural networks that are
successfully trained on data in a source domain. Unsupervised domain
adaptation (UDA) is proposed to counter this, by leveraging both labeled
source domain data and unlabeled target domain data to carry out various tasks
in the target domain. UDA has yielded promising results on natural image
processing, video analysis, natural language processing, time-series data
analysis, medical image analysis, etc. In this review, as a rapidly evolving
topic, we provide a systematic comparison of its methods and applications. In
addition, the connection of UDA with its closely related tasks, e.g., domain
generalization and out-of-distribution detection, has also been discussed.
Furthermore, deficiencies in current methods and possible promising directions
are highlighted.

Keywords: Deep Learning, Unsupervised Domain Adaptation, Transfer Learning,
Adversarial Training, Self Training.

##  1 Introduction

Deep learning is a subfield of machine learning, which aims at discovering
multiple levels of distributed representations of input data via hierarchical
architectures (Goodfellow et al., 2016). For the past several years, there has
been an explosion of deep learning-based approaches, where deep learning has
substantially improved state-of-the-art approaches to diverse machine learning
problems and applications (LeCun et al., 2015). In particular, deep learning
has transformed conventional signal processing approaches into simultaneously
learning both features and a prediction model in an end-to-end fashion (Bengio
et al., 2013). Although supervised deep learning is the most prevalent and
successful approach for a variety of tasks, its success hinges on (i) vast
troves of labeled training data and (ii) the assumption of independent and
identically distributed ($i.i.d.$) training and testing datasets (Huo et al.,
2022). Because reliable labeling of massive datasets for various application
domains is often expensive and prohibitive, for a task without sufficient
labeled datasets in a target domain, there is strong demand to apply trained
models, by leveraging rich labeled data from a source domain (Xu and Yan,
2022). This learning strategy, however, suffers from shifts in data
distributions, i.e., domain shift, between source and target domains (Zhang et
al., 2022). As a result, the performance of a trained model can be severely
degraded, when encountering out-of-distribution (OOD) data, i.e., a source
distribution differs from a target distribution (Che et al., 2021). For
example, the performance of a disease diagnostic system, applied to a
population in a target domain that is different from a population in a source
domain, cannot be guaranteed.

Figure 1: A taxonomy of transfer learning approaches based on the availability
of labeled data in a source or target domain.

To counter this, unsupervised domain adaptation (UDA) is proposed as a viable
solution to migrate knowledge learned from a labeled source domain to unseen,
heterogeneous, and unlabeled target domains (Liu et al., 2021i, 2022g), as
shown in Fig. 2. UDA is aimed at mitigating domain shifts between source and
target domains (Kouw, 2018). The solution to UDA is primarily classified into
statistic moment matching (e.g., maximum mean discrepancy (MMD) (Long et al.,
2018)), domain style transfer (Sankaranarayanan et al., 2018), self-training
(Zou et al., 2019; Liu et al., 2021e, l), and feature-level adversarial
learning (Ganin et al., 2016; He et al., 2020a, b; Liu et al., 2018).

Figure 2: Illustration of the UDA classification and segmentation with the
examples on the VisDA17 challenge database. The target domain data are
unlabeled, as indicated by the orange triangle.

There are several previous review papers focusing on domain adaptation
(Beijbom, 2012; Bungum and Gambäck, 2011; Betlehem et al., 2015; Sun et al.,
2015; Wang and Deng, 2018; Csurka, 2017; Zhao et al., 2018; Kouw, 2018; Kouw
and Loog, 2019; Zhao et al., 2020b; Wilson and Cook, 2020; Ramponi and Plank,
2020), and the broader problem of transfer learning (Pan and Yang, 2009; Cook
et al., 2013; Lazaric, 2012; Shao et al., 2014; Tan et al., 2018; Zhang et
al., 2019b). As shown in Fig. 1, domain adaptation can be seen as a special
case of transfer learning, with the assumption that labeled data are available
only in a source domain (Pan and Yang, 2009). In this review paper, we aim to
provide a wide coverage of models and algorithms for UDA from a theoretical
and practical point of view. This review also touches on emerging approaches,
especially those developed recently, providing a thorough comparison of
different techniques as well as a discussion of the connection of unique
components and methods with unsupervised deep domain adaptation. The coverage
of UDA, especially deep learning-based UDA, has been limited in the general
transfer learning reviews. Many prior reviews of domain adaption do not
incorporate deep learning approaches; however, deep learning-based approaches
have been the mainstream of UDA. In addition, some reviews do not touch deeply
on domain mapping (Csurka, 2017; Kouw, 2018; Kouw and Loog, 2019),
normalization statistic-based (Csurka, 2017; Kouw and Loog, 2019; Zhao et al.,
2018, 2020b), ensemble-based (Csurka, 2017; Wang and Deng, 2018; Kouw and
Loog, 2019; Zhao et al., 2018), or self-training-based methods (Wilson and
Cook, 2020). Moreover, some of them only focus on limited application areas,
such as visual data analytics (Wang and Deng, 2018; Csurka, 2017; Oza et al.,
2021) or natural language processing (NLP) (Ramponi and Plank, 2020). In this
review, we provide a holistic view of this promising technique for a wide
range of application areas, including natural image processing, video
analysis, NLP, time-series data analysis, medical image analysis, and climate
and geosciences. The topics with which other review papers dealt are
summarized in Table 1.

Table 1: Comparison with the previous UDA survey papers.

Surveys | Deep UDA | Generative Mapping | Normalization | Ensemble | Self-training | Self-supervision | Low density
---|---|---|---|---|---|---|---
Beijbom (2012) | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
Sun et al. (2015) | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
Csurka (2017) | $\surd$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
Wang and Deng (2018) | $\surd$ | $\surd$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
Zhao et al. (2018) | $\surd$ | $\surd$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
Kouw and Loog (2019) | $\surd$ | $\surd$ | $\times$ | $\times$ |  $\surd$* (no deep) | $\times$ | $\times$
Wilson and Cook (2020) | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\times$ | $\times$ | $\surd$
Ramponi and Plank (2020) | $\surd$ | $\surd$ | $\times$ | $\times$ | $\surd$ | $\times$ | $\times$
Zhang (2021) | $\surd$ | $\surd$ | $\surd$ | $\times$ | $\surd$ | $\times$ | $\surd$
Guan and Liu (2021) | $\surd$ | $\surd$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
Ours | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$

The rest of the paper is organized as follows. We first analyze possible
domain shifts in UDA in Sec. 2. Then, various recent UDA methods are discussed
and compared to each other in Sec. 3. Next, we show how UDA is applied to
multiple application areas in Sec. 4. In Sec. 5, we highlight promising future
directions. Finally, we conclude this paper in Sec. 6.

##  2 Overview

In this section, without loss of generality, we first introduce terms and
notations as well as a formal definition of UDA. In UDA, there are an
underlying source domain distribution $p_{s}(x,y)\in p_{\mathcal{S}}$ and a
different target domain distribution $p_{t}(x,y)\in p_{\mathcal{T}}$. Then, a
labeled dataset $\mathcal{D_{S}}$ is selected $i.i.d.$ from $p_{s}(x,y)$, and
an unlabeled dataset $\mathcal{D_{T}}$ is selected $i.i.d.$ from the marginal
distribution $p_{t}(x)$. The goal of UDA is to improve a generalization
ability of a trained model in a target domain, by learning on both
$\mathcal{D_{S}}$ and $\mathcal{D_{T}}$. We note that
$\mathcal{Y}=\left\\{1,2,\dots,c\right\\}$ is the set of the class labels for
discriminative tasks, e.g., classification and segmentation. In contrast,
$\mathcal{Y}$ can be continuous values, sentences, images, or languages in
generative tasks (Ge et al., 2021a). UDA (Ganin et al., 2016; Tzeng et al.,
2017) is motivated by the following theorem (Kouw, 2018):

Theorem 1 For a hypothesis $h$

$\displaystyle\mathcal{L}_{t}(h)\leq\mathcal{L}_{s}(h)+d[p_{\mathcal{S}},p_{\mathcal{T}}]+$
(1) $\displaystyle{\rm{min}}[\mathbb{E}_{x\sim
p_{s}}|p_{s}(y|x)-p_{t}(y|x)|,\mathbb{E}_{x\sim
p_{t}}|p_{s}(y|x)-p_{t}(y|x)|].$

Here, $\mathcal{L}_{s}(h)$ and $\mathcal{L}_{t}(h)$ are predefined losses with
a hypothesis $h$ in source and target domains, respectively. $d[\cdot]$
represents a divergence measure, e.g., the Jensen–Shannon (JS) divergence in
the case of conventional adversarial UDA (Salimans et al., 2016). Of note, the
third term on the right hand, ${\rm{min}}[\mathbb{E}_{x\sim
p_{s}}|p_{s}(y|x)-p_{t}(y|x)|,\mathbb{E}_{x\sim
p_{t}}|p_{s}(y|x)-p_{t}(y|x)|]$, is a negligible value, for which the error in
a source domain $\mathcal{L}_{s}(h)$ and the divergence between two domains is
considered an upper bound of the error in a target domain
$\mathcal{L}_{t}(h)$. $\mathcal{L}_{s}(h)$ can be minimized, using recent
advances in supervised learning, e.g., advanced deep feature extractor
networks. Overall, UDA methods aim at minimizing the divergence between two
domains to lower the upper bound of the generalization error in the target
domain $\mathcal{L}_{t}(h)$.

Figure 3: A summary of four possible domain shifts. The red mask indicates the
most common domain shift scenarios in UDA (Liu et al., 2021b). Note that
$p(x)$ can be aligned, if $p(x|y)$ is aligned with the law of total
probability (Zhang et al., 2013).

Domain shifts can be categorized into four types (Kouw, 2018), as shown in
Fig. 3. Existing work primarily focuses on a single shift only, by assuming
that other shifts remain invariant across domains. The covariate shift w.r.t.
$p(x)$ is to align the marginal distribution for all of the data samples. At a
more fine-grained level, the conditional shift is used to align the shift of
$p(x|y)$, which is a more realistic setting than the covariate shift only
setting, since different classes could have their own shift protocols. For
instance, some street lamps glitter, while other lamps are dim at night (Liu
et al., 2021b). However, estimating $p_{t}(x|y)$ without $p_{t}(y)$ is ill-
posed (Zhang et al., 2013). Moreover, the label shift (Chan and Ng, 2005),
a.k.a. target shift, indicates the sample proportion of involved classes is
different between two domains. Furthermore, the concept shift (Kouw, 2018) can
arise, when classifying, for example, tomato as a vegetable or fruit in
different countries; it is, however, usually not a common problem in popular
object classification or semantic segmentation tasks. As such, this review
mainly focuses on the covariate shift alignment in UDA, as is most commonly
studied. The challenges of aligning the other shifts and their combinations
are also discussed as directions for future research.

##  3 Methodology

The past few years have witnessed a proliferation of UDA methods, following
the rapid growth of neural network research. Popular approaches include domain
alignment with statistic divergence and adversarial training, generative
domain mapping, normalization statistics alignment, ensemble-based methods,
and self-training, as summarized in Fig. 4. In addition, these approaches can
be combined to further enhance performance on a variety of tasks. In this
section, we discuss each category in more detail as well as their combinations
and connections.

###  3.1 Statistic Divergence Alignment

Learning domain invariant feature representations is the most widely used
philosophy in many deep UDA methods, which hinges on minimizing domain
discrepancy in a latent feature space. To achieve this goal, choosing a proper
divergence measure is at the core of these methods. Widely used measures
include MMD (Rozantsev et al., 2018), correlation alignment (CORAL) (Sun et
al., 2016), contrastive domain discrepancy (CDD) (Kang et al., 2019),
Wasserstein distance (Liu et al., 2020b), graph matching loss (Yan et al.,
2016), etc.

Figure 4: A summary of the main stream UDA methods discussed in this paper.

Following the hypothesis of a two-sample statistical test, MMD measures the
distribution divergence with observed samples. Specifically, the mean of a
smooth function w.r.t. the samples from two domains are compared, where a
larger mean difference indicates a larger domain discrepancy. Conventionally,
the unit ball in characteristic reproducing kernel Hilbert spaces (RKHS)—as a
means of analyzing and comparing distributions—is used as the smooth function,
which provides a zero population if and only if the two distributions are
equal. In practice, the alignment component serves as another classifier akin
to a task classifier. In what follows, MMD can be calculated and minimized
between the outputs of the classifiers’ layers (Rozantsev et al., 2018), as
shown in Fig. 5(a). Following vanilla MMD, multiple kernel MMD (MK-MMD) (Long
et al., 2015) and joint MMD (JMMD) (Long et al., 2017) are further proposed to
achieve a more robust MMD estimation.

Similar to MMD, CORAL is proposed, based on a polynomial kernel (Sun et al.,
2016). CORAL is defined as the difference of the second-order statistics,
i.e., covariances, across the features of two domains. To measure the
difference of the covariances, different distances have been explored, e.g.,
squared matrix Frobenius norm (Sun and Saenko, 2016), an Euclidean distance
measure in mapped correlation alignment (Zhang et al., 2018b), log-Euclidean
distances (Wang et al., 2017), and geodesic distances (Morerio et al., 2017).
CORAL has also been generalized to possibly infinite-dimensional covariance
matrices in RKHS (Zhang et al., 2018c). The statistics beyond the first-order,
e.g., MMD, and second-order, e.g., CORAL, are further investigated for more
accurate CORAL estimation (Chen et al., 2020a).

Figure 5: Different architectures of (a) MMD-based UDA (e.g., DAN (Long et
al., 2015)), and (b) adversarial training based deep UDA (e.g., domain-
adversarial neural networks (DANN) (Ganin et al., 2016)).

To achieve class conditioned distribution alignment, CDD (Kang et al., 2019)
is proposed to incorporate the class label into MMD. By minimizing CDD, cross-
class divergence is enlarged, while within-class divergence is minimized.
Considering that the label in a target domain is missing in UDA, contrastive
adaptation networks (CAN) (Kang et al., 2019) is proposed to alternatively
estimate the target domain label with clustering, while minimizing CDD.

In addition, the Wasserstein distance (Liu et al., 2020d; Han et al., 2020;
Liu et al., 2019c; Ge et al., 2021b; Liu et al., 2019c), a.k.a. optimal
transportation distance or earth mover’s distance (Liu et al., 2020b, c),
could be another alternative to measure the distribution divergence. The joint
distribution optimal transport (JDOT) is proposed to measure the Wasserstein
distance between two feature distributions (Courty et al., 2017). As a deep
learning framework, DeepJDOT is further proposed to achieve an end-to-end UDA
(Damodaran et al., 2018).

Furthermore, graph matching has been used as a divergence measure, which aims
at finding an optimal correspondence between two graphs (Yan et al., 2016).
With a batch of samples, the feature extraction can be regarded as nodes in an
undirected graph. The distance between two nodes represents their similarity.
The domain divergence is defined as the matching cost between the graphs in
source and target domain batches (Das and Lee, 2018a, b).

###  3.2 Adversarial Learning

Instead of choosing a divergence measure, such as MMD, recent work focuses on
adaptively learning a measure of divergence. With recent advances in
generative adversarial networks (GAN), adversarial training is widely used to
achieve domain invariant feature extraction.

Following Theorem 1, to efficiently minimize the upper bound, i.e., the right-
hand side of Eq. (1), adversarial UDAs are used to minimize domain divergence
at the feature level with guidance of a discriminator as an adaptively learned
divergence measure. Specifically, as shown in Fig. 5(b), in Ganin et al.
(2016); Tzeng et al. (2017), a feature extractor $f(\cdot)$ is applied onto
$x$ to extract a feature representation $f(x)\in\mathbb{R}^{K}$. We would
expect that $d[p_{s}(f(x)),p_{t}(f(x))]$ could be a small value. Targeting
this goal, in addition to training a classifier $Cls$ to correctly classify
source data, $f(\cdot)$ is also optimized to encourage the source and target
feature distributions to be similar to each other, following the supervision
signal from a domain discriminator $Dis:\mathbb{R}^{K}\rightarrow(0,1)$. We
note that the classifier $Cls:\mathbb{R}^{K}\times\mathcal{Y}\rightarrow(0,1)$
outputs the probability of an extracted feature $f(x)$ being a class $y$ among
$c$ categories, i.e., $C(f(x),y)=p(y|f(x);Cls)$. The objective of different
modules can be

$\displaystyle~{}_{Cls}^{\rm{max}}~{}~{}^{\mathbb{~{}~{}E}}_{x\sim p_{s}}{\rm
log}{{C}}(f(x),y)$ (2)
$\displaystyle~{}_{Dis}^{\rm{max}}~{}~{}^{\mathbb{~{}~{}E}}_{x\sim p_{s}}{\rm
log}(1-Dis(f(x))+^{\mathbb{~{}~{}E}}_{x\sim p_{t}}{\rm log}Dis(f(x))$ (3)
$\displaystyle~{}_{~{}f}^{\rm{max}}~{}^{\mathbb{~{}~{}E}}_{x\sim p_{s}}{\rm
log}{{C}}(f(x),y)+\lambda^{\mathbb{~{}~{}E}}_{x\sim p_{t}}{\rm
log}(1-Dis(f(x)),$ (4)

where $\lambda\in\mathbb{R}^{+}$ is used to balance between the two loss
terms. Following the conventional adversarial UDA methods, the three $max$
strategy (Tran et al., 2019; Salimans et al., 2016) can be leveraged, and the
three objectives above are used to update the corresponding three modules,
respectively. In Eq. (3), if $f(x)$ is a source domain feature, then
$Dis(f(x))$ is trained to produce 0 and vice versa. Note that maximizing
$~{}^{\mathbb{~{}~{}E}}_{x\sim p_{t}}{\rm log}Dis(f(x))$ for $Dis$ in Eq. (3),
while maximizing $~{}^{\mathbb{~{}~{}E}}_{x\sim p_{t}}{\rm log}(1-Dis(f(x)))$
for $f$ in Eq. (4) has made this formula a $minmax$ adversarial game.

Figure 6: Different architectures for (a) adversarial UDA with Conditional and
Label Shift (CLS) (Liu et al., 2021b), and (b) generative adversarial UDA
(Bousmalis et al., 2017).

Specifically, domain adversarial neural network (DANN) (Ganin et al., 2016)
utilizes the gradient reversal layer as a domain discriminator $Dis$. In
addition, adversarial discriminative domain adaptation (ADDA) (Tzeng et al.,
2017) is proposed to initialize the target model with source domain training,
followed by adversarial adaptation, which amounts to the target domain-
specific classifier. Other than minimizing cross-entropy based domain
confusion losses, Tzeng et al. (2015) propose to enforce the prediction as a
uniform distribution of binary bins. Assuming that the samples are the same,
these two domain discriminative losses are essentially equivalent to each
other (Goodfellow et al., 2016). Similarly, Motiian et al. (2017) group the
domains and classes as four pairs, by utilizing a four-class classifier for
the domain discriminative network. The feature generator is further developed
in Volpi et al. (2018) to achieve source domain feature augmentation.

Instead of modeling the domain divergence with the JS divergence as in
conventional adversarial UDA (Salimans et al., 2016), a discriminator for
estimating the Wasserstein distance is further proposed (Shen et al., 2018).
Following the recent Wasserstein GAN (Adler and Lunz, 2018), the Wasserstein
distance can be used as a better distance measure, especially to cope with
large discrepancies. This is because the JS-divergence cannot differentiate
the distance between distributions if there is no overlap between two
distributions. In Saito et al. (2018), there are two discriminators to
maximize the discrepancy of each class in the target domain, which renders the
target domain features to have a wider class-wise boundary region to
facilitate the classification.

###  3.3 Normalization Statistics

Figure 7: Different architectures for BN-based methods, e.g., (a) AdaBN (Li et
al., 2018c) and OSUDA (Liu et al., 2021m, 2022b).

In modern deep neural networks, batch normalization (BN) layers have played an
important role in achieving faster training (Ioffe and Szegedy, 2015),
smoother optimization, and more stable convergence (Wu and He, 2018), due to
its insensitivity to initialization (Santurkar et al., 2018). In each
normalization layer, there are two low-order batch statistics, including mean
and variance, and two learnable high-order batch statistics, including scaling
and bias.

Some early work assumes that the BN statistics of mean and variance inherit
domain knowledge. As an early attempt of applying BN to domain adaptation,
AdaBN (Li et al., 2018c), as illustrated in Fig. 7(a), is proposed to achieve
UDA, by modulating BN statistics from a source domain to a target domain. In
AdaBN, once training is completed, the parameters and weights learned during
the training, except for BN layers, are fixed. As a result, BN layers can be
simply added to a target domain, without having an interaction with the source
domain (Li et al., 2018c). In addition, AutoDIAL (Carlucci et al., 2017) is
further proposed as a generalized AdaBN, which retrains the network weights
simultaneously with additional domain alignment layers.

Recent work (Chang et al., 2019; Maria Carlucci et al., 2017; Wang et al.,
2019; Mancini et al., 2018b) demonstrates that the low-order batch statistics,
including the mean and variance, are domain-specific, because of the
divergence of feature representations across two domains. Note that simply
forcing the mean and variance to be the same between source and target domains
is likely to lose expressiveness of networks (Zhang et al., 2020a). Besides,
once the low-order BN statistics discrepancy has been partially mitigated, the
high-order BN statistics can be shareable between two domains (Maria Carlucci
et al., 2017; Wang et al., 2019). Note that all of the aforementioned
approaches (Chang et al., 2019; Maria Carlucci et al., 2017; Zhang et al.,
2020a; Wang et al., 2019; Mancini et al., 2018b) need joint training on source
domain data. Recently, OSUDA (Liu et al., 2021m, 2022b), as shown in Fig.
7(b), is proposed to reduce the domain discrepancy, by means of a momentum-
based adaptive low-order batch statistics progression strategy and an explicit
high-order BN statistics consistency loss for source-free UDA segmentation.

###  3.4 Generative Domain Mapping

Rather than aligning features in a latent space, an alternative can be
directly rendering the target domain data at the data level. The classifier or
segmentation network can be trained on the generated target domain data from
source domain data alongside their labels (Shrivastava et al., 2017). In
addition, the network can be trained simultaneously with GANs (Bousmalis et
al., 2017; Hoffman et al., 2018), as shown in Fig. 6(b).

Cycle reconstruction for image style translation plays an important role in
unpaired translation tasks (Zhu et al., 2017; Kim et al., 2017; Yi et al.,
2017). However, it is challenging to efficiently constrain local structures,
thus leading to significant distortions in the translated images and their
segmentations (Yang et al., 2020a). To address this, Yang et al. (2020a)
extract a modality-independent neighborhood descriptor (MIND) feature
$M(x_{t})$ and $M(G_{TC}(x_{t}))$ of $x_{t}$ and $G_{TC}(x_{t})$ with a
manually defined extractor $M$, and minimize their reconstruction loss
$||M(x_{t})-M(G_{TC}(x_{t}))||_{1}$. Liu et al. (2022e) propose to learn a
general structure feature extractor $f$ in lieu of $M$. To achieve more fine-
grained class-wise image mapping, conditional GANs have been widely used for
generative domain mapping.

###  3.5 Self-training

Unlike approaches that reduce domain discrepancy with a divergence measure,
self-training is proposed as an alternative training scheme, by utilizing
unlabeled target domain data to achieve domain adaptation (Zou et al., 2019).
Self-training is based on a round-based alternative training scheme, which is
originally developed for semi-supervised training and has recently been
adapted for UDA. There are two steps involved in deep self-training based UDA:
(1) creating a set of pseudo-labels in the target domain, and (2) retraining
the network using the generated pseudo-labels with target domain data.

Recently, self-training-based approaches have surpassed adversarial training-
based approaches in several deep UDA tasks (Wei et al., 2021; Mei et al.,
2020b; Shin et al., 2020). Whereas self-training was initially presented as
part of semi-supervised learning (Triguero et al., 2015), recently proposed
deep self-training methods combine feature embedding with alternative learning
in a unified manner, thus yielding flexible domain adaptation (Zou et al.,
2019).

A crucial issue in self-training-based approaches, however, is that pseudo-
labels in the target domain could be noisy; and thus it is likely that a large
proportion could be unreliable. To mitigate this issue, selecting the
prediction with high confidence is essential. To this end, in classification
or segmentation tasks with softmax output unit, a possible solution would be
to gauge the confidence as the maximum value of histogram (Zou et al., 2019).
Additionally, to tackle the problem of the noisy and unreliable pseudo-labels,
Zou et al. (2019) propose to construct a more conservative pseudo-label in
order to smooth the one-hot hard label to a soft label vector. Liu et al.
(2021e) further resort to an additional supervision signal of an energy-based
model for regularization, which is independent of the pseudo-label. Mei et al.
(2020a) propose to explore instance-wise self-training for UDA.

In addition to the discriminative tasks, such as classification and
segmentation, Liu et al. (2021l) further extend self-training to a generative
task, by controlling the confident pseudo-label of continuous pixel value with
a Bayesian uncertainty mask. In learning-based tasks, two kinds of uncertainty
exist, including an aleatoric uncertainty and an epistemic uncertainty (Der
Kiureghian and Ditlevsen, 2009; Kendall and Gal, 2017; Hu et al., 2019).
Specifically, the aleatoric uncertainty is caused by the uncertainty from
noisy training data observations, whereas the epistemic uncertainty is caused
by models that are not sufficiently trained. In self-training, the pseudo-
labels are typically noisy, thus leading to the aleatoric uncertainty. In
addition, the epistemic uncertainty in self-training is caused by a limited
number of iterations for model training and a limited number of target domain
training samples. Therefore, taking both uncertainties into account is vital
to build a robust model with a holistic uncertainty calibration.

###  3.6 Self-supervision

Another solution to UDA is to incorporate auxiliary self-supervision tasks
into the network training. Self-supervised learning hinges on only unlabeled
data to prescribe a pretext learning task, such as context prediction or image
rotation, for which a target objective can be computed without supervision
(Kolesnikov et al., 2019). This group of work assumes that alignment can be
achieved by carrying out source domain classification and reconstruction of
target domain data (Ghifary et al., 2016) or both source and target domain
data (Bousmalis et al., 2016). In Ghifary et al. (2016), a deep
reconstruction-classification network is optimized with a pair-wise squared
reconstruction loss. In particular, the scale-invariant mean squared error
reconstruction loss is introduced in Bousmalis et al. (2016) to train its
domain separation networks.

In addition to the conventional reconstruction tasks (Liu et al., 2018), new
self-supervision tasks have been proposed, e.g., image rotation and jigsaw
predictions (Xu et al., 2019a). Kim et al. (2020a) propose both in-domain and
across-domain self-supervision to achieve UDA with fewer source domain labels.
Lian et al. (2019) propose a self-motivated pyramid curriculum for
segmentation.

###  3.7 Low density target boundary

Several UDA approaches based on a popular clustering assumption (Chapelle and
Zien, 2005) are proposed in the context of semi-supervised training, which
indicates target domain samples from the same class are likely to be
distributed closely as a cluster. The target domain class-wise decision
boundaries should be located in the low-density regions (He et al., 2020b). To
this end, Shu et al. (2018) propose virtual adversarial domain adaptation. In
addition, after training, a decision-boundary iterative refinement step with a
teacher is further applied to refine the decision boundary in a target domain
(Shu et al., 2018). Kumar et al. (2018) propose to combine variational
adversarial training with a conditional entropy loss to achieve a low-density
boundary and avoid overfitting in unlabeled data. Similarly, an entropy loss
has been applied to AutoDIAL (Carlucci et al., 2017). Other than the feature
level, generative methods at the image level have also been developed to make
the decision boundary lie in a lower density region (Wei and Hsu, 2018).

Saito et al. (2017) propose adversarial dropout regularization, which is seen
as the difference between two dropout networks as a discriminator to generate
target discriminative features. Lee et al. (2019b) extend the adversarial
dropout for convolutional layers with a channel drop rather than an element
drop. TDDA (Gholami et al., 2019) focuses on task-discriminative alignment for
UDA.

###  3.8 Other Methods

DEV (You et al., 2019) is proposed to achieve UDA via model selection. The
prototype with clustering is utilized in Pan et al. (2019) for class-wise
adaptation. Liu et al. (2021i, 2022g) further extend the class-wise prototype
to fine-grained subtypes. Wu et al. (2020) propose to apply dual mixup
regularization to adversarial UDA. Domain randomization is proposed in
Rodriguez and Mikolajczyk (2019); Kim et al. (2019b) to randomly generate
source domain data with a different style to achieve a decent generalization
ability in a target domain. To further utilize unlabeled data, a mean teacher
has been used in Cai et al. (2019); Deng et al. (2021). The inter/intra object
correlation is explored in a graph reasoning framework for domain adaptation
in Xu et al. (2020). Liu et al. (2022c) propose to utilize the self-semantic
contour as an intermediate feature to facilitate domain adaptation.

###  3.9 Combinations and Connections

Several aforementioned approaches can be combined with each other to exploit
complementary optimization. Both feature-level adversarial alignment and
image-level generative mapping can be combined sequentially, e.g., GraspGAN
(Bousmalis et al., 2018), or jointly, e.g., CyCADA (Judy et al., 2018).
Following AdaBN, several works have shown that the BN alignment can be added
on top of other UDA methods (Li, 2018; Bousmalis et al., 2018; French et al.,
2017; Kang et al., 2019). The BN alignment and entropy minimization for low-
density target boundary are combined for source data free UDA (Liu et al.,
2021m, 2022b). Adversarial domain-invariant feature alignment has been applied
on different levels, following an ensemble scheme (Kumar et al., 2018). Low-
density target boundary and domain-invariant feature learning are jointly
learned in Lee et al. (2019a); Saito et al. (2018). Kang et al. (2018) combine
generative image mapping with the alignment of model attention. PANDA (Hu et
al., 2020a) integrates adversarial training with prototype-based
normalization.

##  4 Applications

UDA has been successfully applied to a variety of application areas, including
perception and understanding of images, video analysis, NLP, time-series data
analysis, medical image analysis, and climate and geosciences. While some
works are based on general principles of UDA, other works are targeted to
tackle specific applications under consideration, by exploiting the
characteristics of training and testing datasets. In this section, we do not
intend to provide a comprehensive review, but rather opt to highlight examples
of trends in UDA for various application areas, given the presence of a huge
body of work and a number of excellent prior reviews.

###  4.1 Image Analysis

Natural image analysis is the most explored area in UDA, due to the
availability of large-scale visual databases. Depending on the label and
corresponding output, popular tasks include image classification, e.g., object
recognition and face recognition, object detection, semantic segmentation,
image generation, image caption, etc.

####  4.1.1 Image Classification

Classification or recognition of object categories has been a fundamental task
in computer vision. As such, numerous attempts have been made to use deep
learning and UDA for the classification. For instance, Long et al. (2017) use
AlexNet (Krizhevsky et al., 2012) backbone for the task, where the approach is
compared against the source model, the DANN method (Ganin and Lempitsky,
2015), and the variations of MMD, e.g., DDC (Tzeng et al., 2014), DAN (Long et
al., 2015), JAN (Long et al., 2017) and RTN (Long et al., 2016). Zellinger et
al. (2017) compare their CMD methods with other discrepancy-based methods,
e.g., DDC (Tzeng et al., 2014), deep CROAL (Sun and Saenko, 2016), DLID
(Chopra et al., 2013), AdaBN (Li et al., 2016)) and adversarial DANN (Ganin
and Lempitsky, 2015). In addition to the object classification, UDA of face
recognition is another hot research topic, in which the most important shifts
include pose, illumination, expression, age, ethnicity, and imaging modality
(Liu et al., 2021f, g, 2017). Among these shifts, the expression, ethnicity,
and imaging modality have discrete variations, while other attributes have
continuous variations (Liu et al., 2019b, 2021a). In Kan et al. (2015), a bi-
shifting auto-encoder framework is proposed for face identification with the
domain shifts of view, ethnicity, and sensor. Hong et al. (2017) generate
different face views for domain adaptation. Sohn et al. (2017) propose to
achieve adversarial UDA for video face recognition.

There are several widely adopted benchmarks for classification tasks. As for
databases to test the domain shift in natural images, Office-31 dataset
(Saenko et al., 2010) is widely used, which contains data from three different
sources, i.e., Amazon (A), DSLR (D), and Webcam (W). As for image synthesis to
achieve real image domain adaptation, VisDA17 (Peng et al., 2017) is a
preferred choice. DomainNet (Peng et al., 2019b) is the largest domain
adaptation dataset to date, which consists of $\sim$0.6M images with 345 sub-
classes from 24 meta-classes.

In addition to classification with discrete labels, several tasks have ordinal
class labels (Liu et al., 2019a, 2020a). In the case of medical diagnosis, it
is likely that the labels are discrete and distributed successively. As such,
UDA for ordinal classification needs to induce a non-trivial ordinal
distribution first, prior to projecting the data onto a latent space. In Liu
et al. (2021h, 2022a), a recursively conditional Gaussian distribution is
adapted to ordered constraint modeling, which admits a tractable joint
distribution prior.

####  4.1.2 Image Detection

In addition to recognizing objects, image detection has been further
investigated, by localizing objects in a wide view of field with a bounding-
box (Oza et al., 2021). Deep object detection has been an integral part of
several tasks, e.g., surveillance, augmented/mixed reality (AR/MR), autonomous
driving, and human-computer interface.

Adversarial feature alignment has been utilized for UDA object detection in
Chen et al. (2018c); Saito et al. (2019b); Sindagi et al. (2020); Hsu et al.
(2020a); VS et al. (2021). In addition, adversarial generative mapping at the
image level has been applied in Zhang et al. (2019a); Rodriguez and
Mikolajczyk (2019); Hsu et al. (2020b); Chen et al. (2020b); Yu et al. (2022).
In RoyChowdhury et al. (2019); Khodabandeh et al. (2019); Kim et al. (2019a);
Zhao et al. (2020a); Li et al. (2021b); Gu et al. (2022), the pseudo-label
based self-training is adopted for progressive adaptation.

For UDA in image detection, popular domain adaptation scenarios include
adaptation of cross weather conditions, synthetic to real imagery, etc. For
example, domain adaptation is performed from Cityscapes (Cordts et al., 2016)
to Foggy Cityscapes (Sakaridis et al., 2018), which is rendered from
Cityscapes, by adding the fog noise. In addition, several works (Xu et al.,
2019b) use SIM10k dataset as the source domain and the Cityscapes dataset as
the target domain.

####  4.1.3 Image Segmentation

Image segmentation aims at pixel-wise classification (Tajbakhsh et al., 2020;
Liu et al., 2022d). Rather than indicating the rough position of the object
like a detection task, segmentation provides fine-grained delineation to
support subsequent operations. Compared with the sample wise classification in
UDA, it is difficult to apply the low-density target region and prototype
based UDA methods. Since each pixel needs to be represented as a point in the
feature space, it is difficult to scale up to large-scale data. Instead,
adversarial training at both feature and image levels have been widely used
(Yu and Koltun, 2015; Tsai et al., 2018; Mei et al., 2020a). Self-training
based methods have also been developed for semantic segmentation (Chen et al.,
2019b; Liu et al., 2021d).

A typical task is to adapt the large-scale labeled game engine rendered data,
i.e., GTA5 (Richter et al., 2016), to the real-world data, i.e., Cityscapes
(Cordts et al., 2016), for which there are a total of 19 shared labels for
semantic segmentation. In a source domain, there are a total of 24,000 labeled
game engine rendered images from Grand Theft Auto 5. As the standard
evaluation protocol (Yang et al., 2020b), all of the samples in the GTA5
dataset are used as the source domain, while the training set of Cityscape
with a total of 2,975 images is used as the target domain training set. The
testing set of Cityscapes has a total of 500 images.

####  4.1.4 Image Generation

Generative models have been widely applied to diverse tasks, e.g.,
entertainment, image harmonization/stylization, and data completion and
augmentation (Yang et al., 2018, 2019a; Liu et al., 2021k, j; Wang et al., ;
Xing et al., 2022).

The image style synthesis task itself can be regarded as a cross-domain
translation task, when the input involves another style or domain. To address
this, GAN-based methods have been widely used for cross-domain image
generation tasks (He et al., 2021b). Self-training has also been applied to
the image synthesis task. For example, Liu et al. (2021l) propose to leverage
self-training for the image synthesis task, which also considers both
epistemic and aleatoric uncertainties (Der Kiureghian and Ditlevsen, 2009).
Specifically, Liu et al. (2021l) aim at cross modality synthesis using paired
sets of images acquired from two different sites.

###  4.2 Medical Image Analysis

Medical image analysis has been a major application ground for image analysis
methods, due to its wide usage in real-life imaging problems. In addition, a
variety of imaging modalities are used in a clinical setting, each of which
poses unique challenges. An increasing amount of deep network-based methods
have been proposed to achieve enhanced computational speed and better
algorithmic performance over traditional medical image analysis methods. UDA
has been successfully adopted in image segmentation, classification, and
generation tasks in addition to a few other varying applications.

Perone et al. (2019) use a self-ensembling technique in semantic image
segmentation, demonstrating that it can improve model generalization. Their
method is evaluated using a small number of magnetic resonance imaging (MRI)
datasets, serving as a proof-of-concept of the advantage of UDA in medical
imaging, rather than showing an actual application in real medical problems.
Ouyang et al. (2019) report UDA for multi-domain medical image segmentation
via a VAE-based feature prior matching, which features data efficiency. It is
applied to a multi-modality cardiac image dataset to achieve segmentation. Zou
et al. (2020) propose UDA with the so-called Dual-Scheme Fusion Network, where
both source-to-target and target-to-source connections are built to help
bridge the gap between domain differences for improved performance. It is
applied to the segmentation of both brain tumors and cardiac data, yielding
decent results. He et al. (2020c) propose to achieve cross-device retinal OCT
segmentation. Liu et al. (2022c) propose to facilitate cross-modality brain
tumor segmentation with self-semantic contouring. Additionally, more methods
have been proposed to improve the segmentation UDA networks from within their
structures. To enable flexibility of two-way adaptations, Ning et al. (2021)
propose a bidirectional UDA framework based on disentangled representation
learning. It achieves decent performances in both the forward adaptation
direction, from MRI to computed tomography (CT), and the backward direction,
from CT to MRI. The popular evaluation setting is to use the MMWHS challenge
dataset as the source domain and the MMAS dataset as the target domain,
respectively (Zhuang and Shen, 2016).

It often poses a challenge to share medical data for collaboration due to
sensitive patient information. To address the privacy concern of the large-
scale and well-labeled medical data in the source domain, Liu et al. (2021m,
2022h) propose to adapt a pre-trained “off-the-shelf” segmentation model
without source domain data at the adaptation stage. The test-time adaptable
segmentation networks have been developed to achieve UDA in a source-free
manner (He et al., 2021b; Karani et al., 2021). In addition, He et al. (2021b)
show that their method can be generalized to image translation UDA tasks.

Besides, recent years have seen increased usage of UDA to solve segmentation
problems using a variety of imaging modalities, such as CT, MRI, X-ray imaging
(Zhang et al. (2018a)) and optical coherence tomography imaging (Wang et al.,
2021b; Li et al., 2021a). Additionally, UDA has been used in medical image
classification (Ahn et al., 2020; Mahapatra et al., 2021) and diagnosis (Zhang
et al., 2020b). For instance, Liu et al. (2021i, 2022g) propose to explore the
subtype of congenital heart disease (Wang et al., 2021c). The disease level
has been investigated in Liu et al. (2021h, 2022a), in which the Kaggle
Diabetic Retinopathy (KDR)111https://www.kaggle.com/c/diabetic-retinopathy-
detection is used as the source domain, and the recent Indian Diabetic
Retinopathy Image Dataset (IDRiD) dataset (Porwal et al., 2018) is used as the
target domain.

The evaluation database used in He et al. (2021b) for T1-weighted to
T2-weighted MRI translation is brain MRI datasets from three IXI
centers222https://brain-development.org/ixi-dataset/. In addition, paired cine
and tagged tongue MRI data in two clinical sites datasets are used in Liu et
al. (2021l).

###  4.3 Video Analysis

Figure 8: (a) Example of image-to-video adaptation (Sohn et al., 2017) and (b)
example of UDA for video analysis (Chen et al., 2019c).

Video data contain rich spatial and temporal semantic information. However, it
is challenging to collect and annotate a large volume of video data to learn
useful spatiotemporal features. Annotation of all video frames is labor-
intensive and time-consuming for different target applications and devices (Li
et al. (2019a); Peng et al. (2019a); Saleh et al. (2019)). Accordingly, UDA
has been applied to video analysis tasks, including action recognition (Chen
et al. (2019c); Choi et al. (2020b); Pan et al. (2020); Chen et al. (2022)),
person re-identification (Mekhazni et al. (2020)), action segmentation (Chen
et al. (2020e, d)), video captioning (Chen et al. (2021b)), video quality
assessment (Chen et al., 2021a), and video artifact reduction (Ham et al.,
2021).

Because there are few well-organized video datasets in early work, an image-
to-video adaptation method is proposed to use a large-scale image dataset to
train a model for video analysis. Sohn et al. (2017) improve accuracy in face
recognition using a video through image-to-video domain adaptation as in Fig.
8(a). They attempt to overcome the difference of visual quality between still
images and video frames. Liu et al. (2019d) propose a deep image-to-video
adaptation and fusion network (DIVAFN) to enhance accuracy in video action
recognition, by transferring knowledge learned from images. In addition, UCF-
HMDBfull and Kinetics-Gameplay (Chen et al., 2019c) have been collected to
promote video domain adaptation and benchmark the performance in the presence
of large domain discrepancy.

Most pre-trained networks for video analysis tend to perform poorly, when a
pre-trained model encounters unseen temporal dynamics on the target side.
There is prior work to resolve the problems in video action recognition, by
overcoming domain discrepancies along the spatial and temporal directions.
Chen et al. (2019c) propose a temporal attentive adversarial adaptation
network (TA3N) in Fig. 8(b). They attempt to align two domains spatio-
temporally, by encoding spatio-temporal features using an attention mechanism.
Choi et al. (2020b), Pan et al. (2020), and Chen et al. (2022) improve the
attention mechanism for better alignment. Video UDA on action recognition is
extended to more realistic settings, using videos collected from surveillance
cameras (Mou et al., 2021) and drones (Choi et al., 2020a).

UDA is actively studied for video scene analysis and restoration. Chen et al.
(2020c) propose VideoGAN to focus on the translation of video-based data and
transfer the data across different domains. Guizilini et al. (2021) present a
video segmentation method using self-learning to bridge a domain gap between
simulated and real videos. UDA is also applied to face recognition (Ekladious
et al., 2020), person re-identification (Mekhazni et al., 2020), and video
captioning (Chen et al., 2021b). In addition to the analysis of high-level
semantics, there is prior work for UDA in low-level video processing. Chen et
al. (2021a) and Ham et al. (2021) present various UDA methods for video
quality assessment and video artifact reduction. They attempt to provide
reliable performance of a network, when the visual quality of a video frame is
different between source and target domains.

The typical evaluation datasets for image-to-video adaptation include UCF-
Olympic, UCF-HMDBsmall, UCF-HMDBfull, and Kinetics-Gameplay (Chen et al.,
2019c). In addition, the Kinetics and NEC-DRONE datasets are utilized for
evaluation of video UDA on action recognition (Choi et al., 2020a). For the
video quality assessment, UDA approaches are evaluated on DIV2K, BSD68, and
Set12 datasets (Ham et al., 2021).

###  4.4 Natural Language Processing

Similar to the visual data processing, the necessity of developing UDA methods
has emerged in NLP (Sharir et al., 2020), partly because it is costly and
demanding to annotate the sheer volume of language data.

Sentiment analysis is the most explored application to develop UDA methods in
NLP (Ramponi and Plank, 2020). In early attempts of UDA in NLP, Ganin et al.
(2016) propose a domain-adversarial neural network (DANN). UDA is carried out
by adding a domain classifier that is connected to a feature extractor through
a gradient reversal layer. It has motivated several studies (Li et al., 2019b;
Shen et al., 2018; Rocha and Cardoso, 2019; Ghosal et al., 2020). Shen et al.
(2018) utilize the adversarial training to minimize the estimated Wasserstein
distance between source and target samples. Rocha and Cardoso (2019) indicate
that the adversarial training method can be more effective, when the source
and target language datasets contain several content variations in addition to
the language shift. Furthermore, UDA methods are applied to perform various
NLP tasks, including dependency parsing (Sato et al., 2017; Rotman and
Reichart, 2019), POS (part-of-speech) tagging (Desai et al., 2019; Lim et al.,
2020), relation extraction (Fu et al., 2017; Rios et al., 2018; Shi et al.,
2018), trigger identification (Naik and Rose, 2020), language identification
(Li et al., 2019b), political data classification (Desai et al., 2019), etc.

Pre-training has become a key ingredient to deploy an NLP model due to the
inherent complexity of the structure of language and the nature of NLP tasks
(Sharir et al., 2020; Ramponi and Plank, 2020). In recent NLP studies, it is a
standard training strategy to fine-tune a transformer-based model with a small
amount of data for a target application. A large-scaled language dataset is
used for pre-training in the source domain, and task-specific data become the
target domain in the context of UDA. With the domain shift, adaptive pre-
training has been proposed to compensate for the classical pre-training, such
as BERT (Burstein et al., 2019). AdaptBERT (Han and Eisenstein, 2019) performs
domain-adaptive fine-tuning to adapt contextualized embedding by masked
language modeling from the target domain. Gururangan et al. (2020) propose to
use both domain-adaptive pre-training and task-specific pre-training methods.

Image captioning is an interdisciplinary area to connect computer vision and
NLP. A typical solution to UDA for image captioning would be to leverage a
convolutional encoder for extracting the necessary latent information of
visual scenes, followed by adopting a text generator, e.g., recurrent neural
networks. Similarly, Chen et al. (2017) propose to use adversarial training
for the paired source domain data and unpaired target domain data. Zhao et al.
(2017) develop a dual learning scheme to fine-tune a source domain model
trained on a limited dataset to the target domain. Because the output of an
image captioning model is a sentence, it poses a challenge to model a
conditional distribution. A possible solution would be to encode a sentence
label with an additional recurrent neural network as in Che et al. (2021).

The sentiment classification UDA, across English, Chinese, and Arabic with the
dataset in Chen et al. (2018b), is used for evaluation (Rocha and Cardoso,
2019). The English OntoNotes 5.0 and the Universal Dependencies datasets are
used for dependency parsing UDA evaluation (Rotman and Reichart, 2019). In
addition, the English portion of ACE2005 dataset is used for relation
extraction UDA evaluation, which covers a total of 6 genres and 11 relation
types.

###  4.5 Time Series Data Analysis

Figure 9: (a) Example of UDA for sleep classification (Yoo et al., 2021) and
(b) sleep signal space.

Various UDA strategies are exploited for tasks using time series data. Among
others, with time series medical data, such as electroencephalogram (EEG),
electrocardiogram (ECG), and multivariate healthcare data, UDA has been
applied to perform sleep classification (Zhao et al., 2021; Yoo et al., 2021;
Fan et al., 2022), arrhythmia classification (Niu et al., 2020; Wang et al.,
2021a), motor imagery (Raza and Samothrakis, 2019; Tang and Zhang, 2020), etc.
Especially, these methods attempt to tackle the distribution discrepancy
between different datasets and between subjects, because medical data vary
depending on demographic features such as age, sex, and illness. For example,
Yoo et al. (2021) apply both adversarial training and self-training with three
different domain discriminators, including domain, subject, and stage
discriminators, as shown in Fig. 9(a), to preserve local structures of sleep
stages as shown in Fig. 9(b).

Existing work on emotion recognition (Li et al., 2018b; Yin et al., 2020b; He
et al., 2021a, 2022), speech recognition (Wang et al., 2018; Manohar et al.,
2018; Khurana et al., 2021; Anoop et al., 2021), and imagined speech
recognition (Jimenez-Guarneros and Gomez-Gil, 2021) also brings the concept of
UDA. Moreover, the effectiveness of UDA is explored for applications that use
industrial time series data, including human action recognition (Jiang et al.,
2018; Chang et al., 2020; Du et al., 2019; Sanabria et al., 2021), inertial
tracking (Chen et al., 2019a), driving maneuver prediction (Tonutti et al.,
2019), anomaly detection (Michau and Fink, 2021), fault diagnosis (Lu et al.,
2021), and lifetime prediction (da Costa et al., 2020; Ragab et al., 2020).

Besides, time-series UDA approaches are developed to effectively capture the
temporal dependencies of time series data that may be neglected, by visual
data-based methods. For instance, based on DANN, recurrent domain adversarial
neural network (R-DANN) and variational recurrent adversarial deep domain
adaptation (VRADA) (Purushotham et al., 2017) are proposed by exploiting the
long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) and
variational RNN (Chung et al., 2015) as a feature extractor, respectively.
More models, such as a sparse associative structure alignment (SASA) model
(Cai et al., 2021) and a convolutional deep domain adaptation model for time
series data (CoDATS) (Wilson et al., 2020), are developed to improve time
series UDA performance.

For sleep signal UDA, the Montreal Archive of Sleep Studies (MASS) is used as
the source domain, while the Sleep-EDF database and Sleep-EDF-st database are
used as the target domain (Yoo et al., 2021). For emotion recognition, the
DEAP dataset and DREAMER dataset are usually used as the benchmarks (He et
al., 2022).

####  4.5.1 Climate science and Geosciences

In recent years, deep learning has been applied to numerous applications on
the Earth science, e.g., climate science and geosciences (Camps-Valls et al.,
2021). Similar to the other application areas, the perception of remote
sensing data can also have the problem of domain shift, across location and
time. In Huang et al. (2020), UDA across active and passive satellite data is
developed for cloud type detection. Notably, the active spaceborne Lidar
sensor CALIOP onboard CALIPSO satellite has better representation capability
and sensitivity to aerosol types and cloud phases, while the passive
spectroradiometer sensor VIIRS onboard Suomi-NPP satellite has wide swaths and
better spatial coverage. Mengqiu et al. (2022) propose a UDA method to bridge
the gap between the abundant labeled land fog data and the unlabeled sea fog
data for sea fog detection. Soto et al. (2020) exploit the cycleGAN-based UDA
approach (Zhu et al., 2017) for deforestation detection in the Amazon forest.

In addition, UDA has been widely explored in many applications on geoscience
research. Nasim et al. (2022) investigate a UDA approach to mitigate the
domain gap between seismic images of the F3 block 3D dataset from offshore
Netherlands and Penobscot 3D survey data from Canada, which utilizes the
EarthAdaptNet to semantically segment the seismic images, when a few classes
have data scarcity. The teacher-student network has been used in Hu et al.
(2020b) for the classification of the Sentinel-2 images across cities, e.g.,
Moscow and Munich. Lucas et al. (2020) conduct experiments on Satelite Image
Time Series (SITS) classification using existing natural image-based UDA
methods and find that those UDA methods are ineffective, due to the temporal
nature of SITS. Nyborg et al. (2022) propose an explicit UDA method that
learns the temporal shift of SITS for crop classification and introduce a
dataset for cross-region adaptation from SITS in four different regions in
Europe. Ma and Zhang (2021) introduce a UDA approach for corn-yield prediction
using time-series vegetation indices and weather observations.

##  5 Promising Directions

As stated above, advanced deep UDA methods have been widely applied to
numerous tasks and applications. In this section, we point to a number of
underexplored areas that are of great theoretical and practical importance,
which can be promising future research directions.

###  5.1 Realistic Shift Assumption

Most of the current UDA methods have focused on the alignment of covariate
shift. As analyzed in Sec. 1, however, there exist four kinds of possible
shifts in real-world applications (Kouw and Loog, 2019). While numerous works
are proposed in the literature to address conditional or covariate shifts,
label and concept shifts have not been investigated extensively. Notably,
approaches for the adversarial feature alignment of the covariate shift and
approaches without considering the conditional shift have been outperformed by
several competing approaches, e.g., self-training (Zou et al., 2019), dropout
(Saito et al., 2017), and moment matching methods (Pan et al., 2019) in most
of the benchmarks. As such, it is important to incorporate both conditional
and covariate shifts, as it is ill-posed to take one of them into
consideration (Zhang et al., 2013; Kouw, 2018). In Liu et al. (2021b),
theoretical analysis and methodology under the conditional and label shift
assumptions are discussed in adversarial learning-based UDA.

It is, therefore, necessary to incorporate more realistic assumptions of the
domain shifts, depending on real-world tasks at hand.

###  5.2 Partial/Open-set Domain Adaptation

Partial UDA can be seen as a special category of label shifts, in which some
classes have zero probability in a target domain. Due to the mismatch of
categories between source and target domains, conventional UDA approaches may
result in negative transfer (Cao et al., 2018b, a; Kim et al., 2020b).
Similarly, open-set UDA and universal UDA are presented under the assumption
that there are novel classes in a target domain; this approach thus could lead
to novel class discovery or out-of-distribution detection (Panareda Busto and
Gall, 2017).

Lipton et al. (2018) propose a test distribution estimator to detect the label
shift. Azizzadenesheli et al. (2019) introduce a regularization approach to
correct the label shift. Chen et al. (2018a) cast the problem of the label
shift as an optimal transportation-based UDA task, which is closely related to
the class imbalance problem in the MMD framework. Wu et al. (2019a) propose an
asymmetrically-relaxed alignment approach using the adversarial UDA. However,
these approaches assume that there is no conditional shift.

As noted above, partial UDA can be regarded as a special case of the label
shifts. Therefore, developing more general label shift UDA methods for both
small label distribution shift and partial UDA can be more practical for real-
world applications. In addition, novel class/subtype discovery could be
incorporated into the open-set UDA.

###  5.3 Source-free domain adaptation

Data privacy has been a critical concern over cross-center collaboration,
especially in the medical domain. Conventional UDA requires the large scale
and well-labeled source domain data to be shared, which may cause issues over
source domain data leakage and intellectual property (Bateson et al., 2020).
To address this, Liu et al. (2021m, 2022b) propose a source-free UDA approach
with white-box domain adaptation to delineate anatomic structures in medical
imaging data. Specifically, that work leverages an off-the-shelf pre-trained
segmentation model to adapt to a target domain, by migrating its batch
normalization statistics. In addition, recently, Yin et al. (2020a) propose a
deep inversion technique to demonstrate that original training data can be
recovered from knowledge used in the course of white-box domain adaptation
(Zhang et al., 2021). To address this, a recent work (Liu et al., 2022h) uses
black-box UDA segmentation, for which no prior knowledge of network weights is
needed for adaptation. Liu et al. (2022i) further propose that a target domain
network structure could be different from a trained source domain model to
achieve UDA for segmentation.

Source-free domain adaptation is also closely related to test time adaptation,
in that we encounter a single or a few test samples that are different from
source domain data (Royer and Lampert, 2015; Fredericks et al., 2014; Wang et
al., 2020; Liu et al., 2021l). To accommodate continuously changing
environments, we expect that frameworks employed would be source-free, involve
low-cost training in mobile devices, and avoid catastrophic forgetting
(Hoffman et al., 2014; Wulfmeier et al., 2018; Wu et al., 2019b; Mancini et
al., 2019).

Therefore, UDA under a more strict data sharing setting can be a promising
direction, which only shares the pre-trained white/black-box source domain
model. We note that the model sharing is also related to the federated
learning, which is another important transfer learning problem (Yang et al.,
2019b).

###  5.4 Continuous and test time adaptation

Existing work on UDA usually assumes that several stationary domains exist for
which prior work attempts to achieve domain adaptation between discrete
distributions. In real-world environments, however, the change in
distributions could be continuous. For example, when one drives from Seattle
to Boston, one will cross snow-capped mountains, deserts, plateaus, flatlands,
hilly areas, etc. There is, however, no distinct boundaries between these
environments, and thus the shift is smoothly evolving. Therefore, one needs to
consider lifelong learning to progressively adapt a trained model to new
environments (Liu et al., 2021c).

There is a need to build well organized and gradually changing UDA datasets.
In addition, the mixup or interpolation technology (Liu et al., 2018) would be
useful to hallucinate the intermediate data between two largely different
domains to facilitate UDA.

###  5.5 Adaptation in Foundation Model Era

Foundation models (Bommasani et al., 2021) are recently surged as a hot topic
to utilize super large labeled data, which can incorporate sufficiently
variant data. In addition, they are robust to the covariate shift in many
cases. Then, one can ask: if we have a sufficiently large training set with
diverse data distributions, can they generalize well on all of the
implementation scenarios? Though applying domain generalization may address
the covariate and conditional label shifts, it is challenging to alleviate the
label shift, without access to target domain data. In addition, the concept
shift can also cause a problem, even though there are sufficient training
data.

UDA methods to deal with label shift can be an important direction in the era
of foundation models. In addition, it is interesting to investigate the
generality of different foundation models.

###  5.6 Semi-supervised Domain Adaptation

While there have been great advances in UDA, due to diverse target domains,
the performance of UDA is not satisfactory in many cases (Liu et al., 2022f).
In such circumstances, labeling a small set of target domain data could be a
viable solution (Van Engelen and Hoos, 2020). Along this direction, semi-
supervised domain adaptation (SSDA) is proposed, as it can leverage both
labeled source and target data as well as unlabeled target data. Further,
several SSDA classification methods have been proposed to use instance
constraints (Donahue et al., 2013), subspace learning (Yao et al., 2015),
entropy minimax (Saito et al., 2019a), adversarial attack (Kim and Kim, 2020),
etc. These methods are based on discriminative class boundaries for image
classification, which, however, cannot be directly applied to segmentation. In
Liu et al. (2022f), the asymmetric co-training is proposed to achieve semi-
supervised domain adaptation for medical image segmentation.

The unified framework for both SSL and UDA is able to utilize both labeled and
unlabeled target domain data. The alternative training based methods, e.g.,
self-training, have been applied to these two tasks, which can have a great
potential for semi-supervised domain adaptation. Semi-supervised domain
adaptation for object detection and image generation, however, are largely
underexplored.

###  5.7 Domain Generalization

Most of prior work on UDA assumes that there is a single source domain, while
recent work has shown that network generalization can be further improved with
multiple source domain sets (Montesuma and Mboula, 2021). By observing varying
datasets, networks can learn domain invariant cues. Domain generalization
further removes the requirement of unlabeled target domain data in multi-
source UDA (Matsuura and Harada, 2020). There are two main streams for domain
generalization tasks. The first stream is to learn domain invariant features
(Ghifary et al., 2017). For example, Li et al. (2018a); Liu et al. (2021a)
utilize adversarial training to mitigate the domain divergence. The second
stream targets to fuse the domain-specific feature representations. For
instance, Mancini et al. (2018a) develop the domain-specific classifiers with
multiple independent models. Then, the domain agnostic components are fused to
form the domain-wise classification probability. Ding and Fu (2017) propose to
match the low-rank structure of domain-specific features. Liu et al. (2021c)
further propose to align the conditional distribution with a variational
inference scheme.

Domain generalization is usually considered a multi-task learning problem (Liu
et al., 2021a), by exploring multiple source domains. How to achieve good test
time adaptation for an unseen domain can be a challenging problem. For
example, the label shift can be adaptively corrected in test time
implementation as in Liu et al. (2021c).

###  5.8 Out-of-distribution Detection

OOD detection or deep OOD detection has recently been an active research topic
(Che et al., 2021). If the domain shift is too large for reliable adaptation,
a more reasonable choice would be to reject significant outliers rather than
to make adapted predictions with high uncertainty. While detecting the OOD
samples in a low-dimensional space has been well-studied (Pimentel et al.,
2014), it is still challenging to detect OOD in high-dimensional complex data,
e.g., images (Liang et al., 2018). For example, Hendrycks and Gimpel (2017)
identify that trained DNNs usually have higher maximum softmax output for in-
distribution examples than anomalous ones. A possible improvement of this
baseline would be to consider both the in-distribution and out-of-distribution
training samples during training (Hendrycks et al., 2019). However,
enumerating all possible OOD distributions before deployment is usually not
possible. Liang et al. (2018) propose that the difference between maximum
probabilities in softmax distributions on ID/OOD samples can be made more
significant, by means of adversarial perturbation pre-processing during
training.

Devries and Taylor (2018) augment the classifier with a confidence estimation
branch, and adjust the objective using the predicted confidence score for
training. Lee et al. (2018a) train a classifier simultaneously with a GAN,
with an additional objective to encourage low confidence in generated samples.
Hendrycks et al. (2019) propose to use real OOD samples instead of generated
ones to train the detector. Vyas et al. (2018) label a part of training data
as OOD samples to train a classifier, where that approach dynamically changes
the partition of ID and OOD samples. These improvements based on Hendrycks and
Gimpel (2017) typically need to retrain a classifier with modified structures
or optimization objectives. Recently, Lee et al. (2018b) propose a new
framework for anomaly detection. A number of methods (Liang et al., 2018; Vyas
et al., 2018; Lee et al., 2018b) need OOD samples for tuning hyper-parameter
selection, e.g., the threshold for verification. DVN (Che et al., 2021) aims
to verify the predictions of a trained deep model, by estimating $p(x|y)$
rather than $p(x)$.

It remains an exciting open problem of how to train good density estimators on
complex datasets, which is an important module for OOD detection. In addition,
the connection and difference between OOD sample and adversarial attack
samples also need further explorations.

##  6 CONCLUSION

In this paper, we have systematically reviewed deep learning-based UDA
approaches. Deep learning has already surpassed its predecessors in a variety
of fields, and future research in deep learning will strive toward the
seamless deployment of trained models in a source domain into unseen and new
target domains. Toward this goal, we provided a comprehensive summary of
recent deep UDA approaches along with the merits and demerits of those
approaches. Furthermore, several successful applications of deep UDA methods
were reviewed. Finally, several challenges of the current deep UDA approaches
were identified, which could serve as promising future directions.

## References

  * Adler and Lunz (2018) Jonas Adler and Sebastian Lunz.  Banach wasserstein gan.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Ahn et al. (2020) Euijoon Ahn, Ashnil Kumar, Michael Fulham, Dagan Feng, and Jinman Kim.  Unsupervised domain adaptation to classify medical images using zero-bias convolutional auto-encoders and context-based feature augmentation.  _IEEE transactions on medical imaging_ , 39(7):2385–2394, 2020.
  * Anoop et al. (2021) CS Anoop, AP Prathosh, and AG Ramakrishnan.  Unsupervised domain adaptation schemes for building asr in low-resource languages.  In _2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)_ , pages 342–349. IEEE, 2021.
  * Azizzadenesheli et al. (2019) Kamyar Azizzadenesheli, Anqi Liu, Fanny Yang, and Animashree Anandkumar.  Regularized learning for domain adaptation under label shifts.  _ICLR_ , 2019.
  * Bateson et al. (2020) Mathilde Bateson, Hoel Kervadec, Jose Dolz, Herve Lombaert, and Ismail Ben Ayed.  Source-relaxed domain adaptation for image segmentation.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 490–499. Springer, 2020.
  * Beijbom (2012) Oscar Beijbom.  Domain adaptations for computer vision applications.  _arXiv preprint arXiv:1211.4860_ , 2012.
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent.  Representation learning: A review and new perspectives.  _IEEE transactions on pattern analysis and machine intelligence_ , 35(8):1798–1828, 2013.
  * Betlehem et al. (2015) Terence Betlehem, Wen Zhang, Mark A Poletti, and Thushara D Abhayapala.  Personal sound zones: Delivering interface-free audio to multiple listeners.  _IEEE Signal Processing Magazine_ , 32(2):81–91, 2015.
  * Bommasani et al. (2021) Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al.  On the opportunities and risks of foundation models.  _arXiv preprint arXiv:2108.07258_ , 2021.
  * Bousmalis et al. (2016) Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan.  Domain separation networks.  _Advances in neural information processing systems_ , 29, 2016.
  * Bousmalis et al. (2017) Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan.  Unsupervised pixel-level domain adaptation with generative adversarial networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3722–3731, 2017.
  * Bousmalis et al. (2018) Konstantinos Bousmalis, Alex Irpan, Paul Wohlhart, Yunfei Bai, Matthew Kelcey, Mrinal Kalakrishnan, Laura Downs, Julian Ibarz, Peter Pastor, Kurt Konolige, et al.  Using simulation and domain adaptation to improve efficiency of deep robotic grasping.  In _2018 IEEE international conference on robotics and automation (ICRA)_ , pages 4243–4250. IEEE, 2018.
  * Bungum and Gambäck (2011) Lars Bungum and Björn Gambäck.  A survey of domain adaptation in machine translation: Towards a refinement of domain space.  In _Proceedings of the India-Norway Workshop on Web Concepts and Technologies_ , volume 112, 2011.
  * Burstein et al. (2019) Jill Burstein, Christy Doran, and Thamar Solorio.  Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers).  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 2019.
  * Cai et al. (2019) Qi Cai, Yingwei Pan, Chong-Wah Ngo, Xinmei Tian, Lingyu Duan, and Ting Yao.  Exploring object relation in mean teacher for cross-domain detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11457–11466, 2019.
  * Cai et al. (2021) Ruichu Cai, Jiawei Chen, Zijian Li, Wei Chen, Keli Zhang, Junjian Ye, Zhuozhang Li, Xiaoyan Yang, and Zhenjie Zhang.  Time series domain adaptation via sparse associative structure alignment.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 6859–6867, 2021.
  * Camps-Valls et al. (2021) Gustau Camps-Valls, Devis Tuia, Xiao Xiang Zhu, and Markus Reichstein.  _Deep learning for the Earth Sciences: A comprehensive approach to remote sensing, climate science and geosciences_.  John Wiley & Sons, 2021.
  * Cao et al. (2018a) Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Michael I Jordan.  Partial transfer learning with selective adversarial networks.  In _CVPR_ , pages 2724–2732, 2018a.
  * Cao et al. (2018b) Zhangjie Cao, Lijia Ma, Mingsheng Long, and Jianmin Wang.  Partial adversarial domain adaptation.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 135–150, 2018b.
  * Carlucci et al. (2017) Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo.  Autodial: Automatic domain alignment layers.  In _2017 IEEE international conference on computer vision (ICCV)_ , pages 5077–5085. IEEE, 2017.
  * Chan and Ng (2005) Yee Seng Chan and Hwee Tou Ng.  Word sense disambiguation with distribution estimation.  In _IJCAI_ , volume 5, pages 1010–5, 2005.
  * Chang et al. (2019) Woong-Gi Chang, Tackgeun You, Seonguk Seo, Suha Kwak, and Bohyung Han.  Domain-specific batch normalization for unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 7354–7362, 2019.
  * Chang et al. (2020) Youngjae Chang, Akhil Mathur, Anton Isopoussu, Junehwa Song, and Fahim Kawsar.  A systematic study of unsupervised domain adaptation for robust human-activity recognition.  _Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies_ , 4(1):1–30, 2020.
  * Chapelle and Zien (2005) Olivier Chapelle and Alexander Zien.  Semi-supervised classification by low density separation.  In _International workshop on artificial intelligence and statistics_ , pages 57–64. PMLR, 2005.
  * Che et al. (2021) Tong Che, Xiaofeng Liu, Site Li, Yubin Ge, Ruixiang Zhang, Caiming Xiong, and Yoshua Bengio.  Deep verifier networks: Verification of deep discriminative models with deep generative models, 2021.
  * Chen et al. (2019a) Changhao Chen, Yishu Miao, Chris Xiaoxuan Lu, Linhai Xie, Phil Blunsom, Andrew Markham, and Niki Trigoni.  Motiontransformer: Transferring neural inertial tracking between domains.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 8009–8016, 2019a.
  * Chen et al. (2020a) Chao Chen, Zhihang Fu, Zhihong Chen, Sheng Jin, Zhaowei Cheng, Xinyu Jin, and Xian-Sheng Hua.  Homm: Higher-order moment matching for unsupervised domain adaptation.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 34, pages 3422–3429, 2020a.
  * Chen et al. (2020b) Chaoqi Chen, Zebiao Zheng, Xinghao Ding, Yue Huang, and Qi Dou.  Harmonizing transferability and discriminability for adapting object detectors.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 8869–8878, 2020b.
  * Chen et al. (2019b) Cheng Chen, Qi Dou, Hao Chen, Jing Qin, and Pheng-Ann Heng.  Synergistic image and feature adaptation: Towards cross-modality domain adaptation for medical image segmentation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 865–872, 2019b.
  * Chen et al. (2020c) Jiawei Chen, Yuexiang Li, Kai Ma, and Yefeng Zheng.  Generative adversarial networks for video-to-video domain adaptation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 3462–3469, 2020c.
  * Chen et al. (2019c) Min-Hung Chen, Zsolt Kira, Ghassan AlRegib, Jaekwon Yoo, Ruxin Chen, and Jian Zheng.  Temporal attentive alignment for large-scale video domain adaptation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 6321–6330, 2019c.
  * Chen et al. (2020d) Min-Hung Chen, Baopu Li, Yingze Bao, and Ghassan AlRegib.  Action segmentation with mixed temporal domain adaptation.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 605–614, 2020d.
  * Chen et al. (2020e) Min-Hung Chen, Baopu Li, Yingze Bao, Ghassan AlRegib, and Zsolt Kira.  Action segmentation with joint self-supervised temporal domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 9454–9463, 2020e.
  * Chen et al. (2022) Peipeng Chen, Yuan Gao, and Andy J Ma.  Multi-level attentive adversarial learning with temporal dilation for unsupervised video domain adaptation.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 1259–1268, 2022.
  * Chen et al. (2021a) Pengfei Chen, Leida Li, Jinjian Wu, Weisheng Dong, and Guangming Shi.  Unsupervised curriculum domain adaptation for no-reference video quality assessment.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 5178–5187, 2021a.
  * Chen et al. (2018a) Qingchao Chen, Yang Liu, Zhaowen Wang, Ian Wassell, and Kevin Chetty.  Re-weighted adversarial adaptation network for unsupervised domain adaptation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 7976–7985, 2018a.
  * Chen et al. (2021b) Qingchao Chen, Yang Liu, and Samuel Albanie.  Mind-the-gap! unsupervised domain adaptation for text-video retrieval.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 1072–1080, 2021b.
  * Chen et al. (2017) Tseng-Hung Chen, Yuan-Hong Liao, Ching-Yao Chuang, Wan-Ting Hsu, Jianlong Fu, and Min Sun.  Show, adapt and tell: Adversarial training of cross-domain image captioner.  In _Proceedings of the IEEE international conference on computer vision_ , pages 521–530, 2017.
  * Chen et al. (2018b) Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, and Kilian Weinberger.  Adversarial deep averaging networks for cross-lingual sentiment classification.  _Transactions of the Association for Computational Linguistics_ , 6:557–570, 2018b.
  * Chen et al. (2018c) Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Domain adaptive faster r-cnn for object detection in the wild.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3339–3348, 2018c.
  * Choi et al. (2020a) Jinwoo Choi, Gaurav Sharma, Manmohan Chandraker, and Jia-Bin Huang.  Unsupervised and semi-supervised domain adaptation for action recognition from drones.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pages 1717–1726, 2020a.
  * Choi et al. (2020b) Jinwoo Choi, Gaurav Sharma, Samuel Schulter, and Jia-Bin Huang.  Shuffle and attend: Video domain adaptation.  In _European Conference on Computer Vision_ , pages 678–695. Springer, 2020b.
  * Chopra et al. (2013) Sumit Chopra, Suhrid Balakrishnan, and Raghuraman Gopalan.  Dlid: Deep learning for domain adaptation by interpolating between domains.  In _ICML workshop on challenges in representation learning_ , volume 2. Citeseer, 2013.
  * Chung et al. (2015) Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Bengio.  A recurrent latent variable model for sequential data.  _Advances in neural information processing systems_ , 28, 2015.
  * Cook et al. (2013) Diane Cook, Kyle D Feuz, and Narayanan C Krishnan.  Transfer learning for activity recognition: A survey.  _Knowledge and information systems_ , 36(3):537–556, 2013.
  * Cordts et al. (2016) Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele.  The cityscapes dataset for semantic urban scene understanding.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3213–3223, 2016.
  * Courty et al. (2017) Nicolas Courty, Remi Flamary, Amaury Habrard, and Alain Rakotomamonjy.  Joint distribution optimal transportation for domain adaptation.  _Advances in Neural Information Processing Systems_ , 30, 2017.
  * Csurka (2017) Gabriela Csurka.  Domain adaptation for visual applications: A comprehensive survey.  _arXiv preprint arXiv:1702.05374_ , 2017.
  * da Costa et al. (2020) Paulo Roberto de Oliveira da Costa, Alp Akçay, Yingqian Zhang, and Uzay Kaymak.  Remaining useful lifetime prediction via deep domain adaptation.  _Reliability Engineering & System Safety_, 195:106682, 2020.
  * Damodaran et al. (2018) Bharath Bhushan Damodaran, Benjamin Kellenberger, Remi Flamary, Devis Tuia, and Nicolas Courty.  Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation.  In _Proceedings of the European Conference on Computer Vision (ECCV)_ , pages 447–463, 2018.
  * Das and Lee (2018a) Debasmit Das and CS Lee.  Graph matching and pseudo-label guided deep unsupervised domain adaptation.  In _International conference on artificial neural networks_ , pages 342–352. Springer, 2018a.
  * Das and Lee (2018b) Debasmit Das and CS George Lee.  Unsupervised domain adaptation using regularized hyper-graph matching.  In _2018 25th IEEE International Conference on Image Processing (ICIP)_ , pages 3758–3762. IEEE, 2018b.
  * Deng et al. (2021) Jinhong Deng, Wen Li, Yuhua Chen, and Lixin Duan.  Unbiased mean teacher for cross-domain object detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 4091–4101, 2021.
  * Der Kiureghian and Ditlevsen (2009) Armen Der Kiureghian and Ove Ditlevsen.  Aleatory or epistemic? does it matter?  _Structural safety_ , 31(2):105–112, 2009.
  * Desai et al. (2019) Shrey Desai, Barea Sinno, Alex Rosenfeld, and Junyi Jessy Li.  Adaptive ensembling: Unsupervised domain adaptation for political document analysis.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4718–4730, 2019.
  * Devries and Taylor (2018) Terrance Devries and Graham W. Taylor.  Learning confidence for out-of-distribution detection in neural networks.  2018\.
  * Ding and Fu (2017) Zhengming Ding and Yun Fu.  Deep domain generalization with structured low-rank constraint.  _TIP_ , 2017.
  * Donahue et al. (2013) Jeff Donahue, Judy Hoffman, Erik Rodner, Kate Saenko, and Trevor Darrell.  Semi-supervised domain adaptation with instance constraints.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 668–675, 2013.
  * Du et al. (2019) Hao Du, Tian Jin, Yongping Song, and Yongpeng Dai.  Unsupervised adversarial domain adaptation for micro-doppler based human activity classification.  _IEEE geoscience and remote sensing letters_ , 17(1):62–66, 2019.
  * Ekladious et al. (2020) George Ekladious, Hugo Lemoine, Eric Granger, Kaveh Kamali, and Salim Moudache.  Dual-triplet metric learning for unsupervised domain adaptation in video face recognition.  In _2020 International Joint Conference on Neural Networks (IJCNN)_ , pages 1–9. IEEE, 2020.
  * Fan et al. (2022) Jiahao Fan, Hangyu Zhu, Xinyu Jiang, Long Meng, Chen Chen, Cong Fu, Huan Yu, Chenyun Dai, and Wei Chen.  Unsupervised domain adaptation by statistics alignment for deep sleep staging networks.  _IEEE Transactions on Neural Systems and Rehabilitation Engineering_ , 30:205–216, 2022.
  * Fredericks et al. (2014) Erik M Fredericks, Byron DeVries, and Betty HC Cheng.  Towards run-time adaptation of test cases for self-adaptive systems in the face of uncertainty.  In _Proceedings of the 9th International Symposium on Software Engineering for Adaptive and Self-Managing Systems_ , pages 17–26, 2014.
  * French et al. (2017) Geoffrey French, Michal Mackiewicz, and Mark Fisher.  Self-ensembling for visual domain adaptation.  _arXiv preprint arXiv:1706.05208_ , 2017.
  * Fu et al. (2017) Lisheng Fu, Thien Huu Nguyen, Bonan Min, and Ralph Grishman.  Domain adaptation for relation extraction with domain adversarial neural network.  In _Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 425–429, 2017.
  * Ganin and Lempitsky (2015) Yaroslav Ganin and Victor Lempitsky.  Unsupervised domain adaptation by backpropagation.  In _ICML_ , 2015.
  * Ganin et al. (2016) Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  _The Journal of Machine Learning Research_ , 17(1):2096–2030, 2016.
  * Ge et al. (2021a) Yubin Ge, Ly Dinh, Xiaofeng Liu, Jinsong Su, Ziyao Lu, Ante Wang, and Jana Diesner.  Baco: A background knowledge-and content-based framework for citing sentence generation.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1466–1478, 2021a.
  * Ge et al. (2021b) Yubin Ge, Site Li, Xuyang Li, Fangfang Fan, Wanqing Xie, Jane You, and Xiaofeng Liu.  Embedding semantic hierarchy in discrete optimal transport for risk minimization.  In _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 2835–2839. IEEE, 2021b.
  * Ghifary et al. (2016) Muhammad Ghifary, W Bastiaan Kleijn, Mengjie Zhang, David Balduzzi, and Wen Li.  Deep reconstruction-classification networks for unsupervised domain adaptation.  In _European conference on computer vision_ , pages 597–613. Springer, 2016.
  * Ghifary et al. (2017) Muhammad Ghifary, David Balduzzi, W Bastiaan Kleijn, and Mengjie Zhang.  Scatter component analysis: A unified framework for domain adaptation and domain generalization.  _IEEE T-PAMI_ , 2017.
  * Gholami et al. (2019) Behnam Gholami, Pritish Sahu, Minyoung Kim, and Vladimir Pavlovic.  Task-discriminative domain alignment for unsupervised domain adaptation.  In _ICCV_ , 2019.
  * Ghosal et al. (2020) Deepanway Ghosal, Devamanyu Hazarika, Abhinaba Roy, Navonil Majumder, Rada Mihalcea, and Soujanya Poria.  Kingdom: Knowledge-guided domain adaptation for sentiment analysis.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3198–3210, 2020.
  * Goodfellow et al. (2016) I. Goodfellow, Y. Bengio, and A. Courville.  _Deep Learning_.  MIT press, 2016.
  * Gu et al. (2022) Qiao Gu, Brian Okorn, and David Held.  Ossid: Online self-supervised instance detection by (and for) pose estimation.  _IEEE Robotics and Automation Letters_ , 2022.
  * Guan and Liu (2021) Hao Guan and Mingxia Liu.  Domain adaptation for medical image analysis: a survey.  _IEEE Transactions on Biomedical Engineering_ , 2021.
  * Guizilini et al. (2021) Vitor Guizilini, Jie Li, Rareș Ambruș, and Adrien Gaidon.  Geometric unsupervised domain adaptation for semantic segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 8537–8547, 2021.
  * Gururangan et al. (2020) Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A Smith.  Don’t stop pretraining: Adapt language models to domains and tasks.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 8342–8360, 2020.
  * Ham et al. (2021) Yu-Jin Ham, Chaehwa Yoo, and Je-Won Kang.  Training compression artifacts reduction network with domain adaptation.  In _Applications of Digital Image Processing XLIV_ , volume 11842, page 118420U. International Society for Optics and Photonics, 2021.
  * Han and Eisenstein (2019) Xiaochuang Han and Jacob Eisenstein.  Unsupervised domain adaptation of contextualized embeddings for sequence labeling.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 4238–4248, 2019.
  * Han et al. (2020) Yuzhuo Han, Xiaofeng Liu, Zhenfei Sheng, Yutao Ren, Xu Han, Jane You, Risheng Liu, and Zhongxuan Luo.  Wasserstein loss-based deep object detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pages 998–999, 2020.
  * He et al. (2020a) Gewen He, Xiaofeng Liu, Fangfang Fan, and Jane You.  Classification-aware semi-supervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pages 964–965, 2020a.
  * He et al. (2020b) Gewen He, Xiaofeng Liu, Fangfang Fan, and Jane You.  Image2audio: Facilitating semi-supervised audio emotion recognition with facial expression image.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pages 912–913, 2020b.
  * He et al. (2021a) Wenwen He, Yalan Ye, Yunxia Li, Tongjie Pan, and Li Lu.  Online cross-subject emotion recognition from ecg via unsupervised domain adaptation.  In _2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)_, pages 1001–1005. IEEE, 2021a.
  * He et al. (2020c) Yufan He, Aaron Carass, Yihao Liu, Shiv Saidha, Peter A Calabresi, and Jerry L Prince.  Adversarial domain adaptation for multi-device retinal oct segmentation.  In _Medical Imaging 2020: Image Processing_ , volume 11313, page 1131309\. International Society for Optics and Photonics, 2020c.
  * He et al. (2021b) Yufan He, Aaron Carass, Lianrui Zuo, Blake E Dewey, and Jerry L Prince.  Autoencoder based self-supervised test-time adaptation for medical image analysis.  _Medical image analysis_ , 72:102136, 2021b.
  * He et al. (2022) Zhipeng He, Yongshi Zhong, and Jiahui Pan.  An adversarial discriminative temporal convolutional network for eeg-based cross-domain emotion recognition.  _Computers in biology and medicine_ , 141:105048, 2022.
  * Hendrycks and Gimpel (2017) Dan Hendrycks and Kevin Gimpel.  A baseline for detecting misclassified and out-of-distribution examples in neural networks.  _ICLR_ , 2017.
  * Hendrycks et al. (2019) Dan Hendrycks, Mantas Mazeika, and Thomas Dietterich.  Deep anomaly detection with outlier exposure.  _ICLR_ , 2019.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  _Neural computation_ , 1997.
  * Hoffman et al. (2014) Judy Hoffman, Trevor Darrell, and Kate Saenko.  Continuous manifold based adaptation for evolving visual domains.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 867–874, 2014.
  * Hoffman et al. (2018) Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei A Efros, and Trevor Darrell.  Cycada: Cycle-consistent adversarial domain adaptation.  In _ICML_ , 2018.
  * Hong et al. (2017) Sungeun Hong, Woobin Im, Jongbin Ryu, and Hyun S Yang.  Sspp-dan: Deep domain adaptation network for face recognition with single sample per person.  In _2017 IEEE International Conference on Image Processing (ICIP)_ , pages 825–829. IEEE, 2017.
  * Hsu et al. (2020a) Cheng-Chun Hsu, Yi-Hsuan Tsai, Yen-Yu Lin, and Ming-Hsuan Yang.  Every pixel matters: Center-aware feature alignment for domain adaptive object detector.  In _European Conference on Computer Vision_ , pages 733–748. Springer, 2020a.
  * Hsu et al. (2020b) Han-Kai Hsu, Chun-Han Yao, Yi-Hsuan Tsai, Wei-Chih Hung, Hung-Yu Tseng, Maneesh Singh, and Ming-Hsuan Yang.  Progressive domain adaptation for object detection.  In _Proceedings of the IEEE/CVF winter conference on applications of computer vision_ , pages 749–757, 2020b.
  * Hu et al. (2020a) Dapeng Hu, Jian Liang, Qibin Hou, Hanshu Yan, Yunpeng Chen, Shuicheng Yan, and Jiashi Feng.  Panda: Prototypical unsupervised domain adaptation.  _ECCV_ , 2020a.
  * Hu et al. (2020b) Jingliang Hu, Lichao Mou, and Xiao Xiang Zhu.  Unsupervised domain adaptation using a teacher-student network for cross-city classification of sentinel-2 images.  _The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences_ , 43:1569–1574, 2020b.
  * Hu et al. (2019) Shi Hu, Daniel Worrall, Stefan Knegt, Bas Veeling, Henkjan Huisman, and Max Welling.  Supervised uncertainty quantification for segmentation with multiple annotations.  In _MICCAI_ , pages 137–145. Springer, 2019.
  * Huang et al. (2020) Xin Huang, Sahara Ali, Chenxi Wang, Zeyu Ning, Sanjay Purushotham, Jianwu Wang, and Zhibo Zhang.  Deep domain adaptation based cloud type detection using active and passive satellite data.  In _2020 IEEE International Conference on Big Data (Big Data)_ , pages 1330–1337. IEEE, 2020.
  * Huo et al. (2022) Xinyue Huo, Lingxi Xie, Hengtong Hu, Wengang Zhou, Houqiang Li, and Qi Tian.  Domain-agnostic prior for transfer semantic segmentation.  _arXiv preprint arXiv:2204.02684_ , 2022.
  * Ioffe and Szegedy (2015) Sergey Ioffe and Christian Szegedy.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In _International conference on machine learning_ , pages 448–456, 2015.
  * Jiang et al. (2018) Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shuochao Yao, Yaqing Wang, Ye Yuan, Hongfei Xue, Chen Song, Xin Ma, Dimitrios Koutsonikolas, et al.  Towards environment independent device free human activity recognition.  In _Proceedings of the 24th Annual International Conference on Mobile Computing and Networking_ , pages 289–304, 2018.
  * Jimenez-Guarneros and Gomez-Gil (2021) Magdiel Jimenez-Guarneros and Pilar Gomez-Gil.  Standardization-refinement domain adaptation method for cross-subject eeg-based classification in imagined speech recognition.  _Pattern Recognition Letters_ , 141:54–60, 2021.
  * Judy et al. (2018) Hoffman Judy, Tzeng Eric, Park Taesung, Zhu Jun-Yan, Isola Phillip, Saenko Kate, Efros Alexei, and Darrell Trevor.  Cycada: Cycle-consistent adversarial domain adaptation.  In _ICML_ , pages 1994–2003, 2018.
  * Kan et al. (2015) Meina Kan, Shiguang Shan, and Xilin Chen.  Bi-shifting auto-encoder for unsupervised domain adaptation.  In _Proceedings of the IEEE international conference on computer vision_ , pages 3846–3854, 2015.
  * Kang et al. (2018) Guoliang Kang, Liang Zheng, Yan Yan, and Yi Yang.  Deep adversarial attention alignment for unsupervised domain adaptation: the benefit of target expectation maximization.  In _Proceedings of the European conference on computer vision (ECCV)_ , pages 401–416, 2018.
  * Kang et al. (2019) Guoliang Kang, Lu Jiang, Yi Yang, and Alexander G Hauptmann.  Contrastive adaptation network for unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 4893–4902, 2019.
  * Karani et al. (2021) Neerav Karani, Ertunc Erdil, Krishna Chaitanya, and Ender Konukoglu.  Test-time adaptable neural networks for robust medical image segmentation.  _Medical Image Analysis_ , 68:101907, 2021.
  * Kendall and Gal (2017) Alex Kendall and Yarin Gal.  What uncertainties do we need in bayesian deep learning for computer vision?  _arXiv:1703.04977_ , 2017.
  * Khodabandeh et al. (2019) Mehran Khodabandeh, Arash Vahdat, Mani Ranjbar, and William G Macready.  A robust learning approach to domain adaptive object detection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 480–490, 2019.
  * Khurana et al. (2021) Sameer Khurana, Niko Moritz, Takaaki Hori, and Jonathan Le Roux.  Unsupervised domain adaptation for speech recognition via uncertainty driven self-training.  In _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 6553–6557. IEEE, 2021.
  * Kim et al. (2020a) Donghyun Kim, Kuniaki Saito, Tae-Hyun Oh, Bryan A Plummer, Stan Sclaroff, and Kate Saenko.  Cross-domain self-supervised learning for domain adaptation with few source labels.  _arXiv preprint arXiv:2003.08264_ , 2020a.
  * Kim et al. (2019a) Seunghyeon Kim, Jaehoon Choi, Taekyung Kim, and Changick Kim.  Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 6092–6101, 2019a.
  * Kim et al. (2017) Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim.  Learning to discover cross-domain relations with generative adversarial networks.  In _International Conference on Machine Learning_ , pages 1857–1865. PMLR, 2017.
  * Kim and Kim (2020) Taekyung Kim and Changick Kim.  Attract, perturb, and explore: Learning a feature alignment network for semi-supervised domain adaptation.  In _European Conference on Computer Vision_ , pages 591–607. Springer, 2020.
  * Kim et al. (2019b) Taekyung Kim, Minki Jeong, Seunghyeon Kim, Seokeon Choi, and Changick Kim.  Diversify and match: A domain adaptive representation learning paradigm for object detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12456–12465, 2019b.
  * Kim et al. (2020b) Youngeun Kim, Sungeun Hong, Seunghan Yang, Sungil Kang, Yunho Jeon, and Jiwon Kim.  Associative partial domain adaptation.  _arXiv preprint arXiv:2008.03111_ , 2020b.
  * Kolesnikov et al. (2019) Alexander Kolesnikov, Xiaohua Zhai, and Lucas Beyer.  Revisiting self-supervised visual representation learning.  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , pages 1920–1929, 2019.
  * Kouw (2018) Wouter M Kouw.  An introduction to domain adaptation and transfer learning.  _arXiv preprint arXiv:1812.11806_ , 2018.
  * Kouw and Loog (2019) Wouter M Kouw and Marco Loog.  A review of domain adaptation without target labels.  _IEEE transactions on pattern analysis and machine intelligence_ , 43(3):766–785, 2019.
  * Krizhevsky et al. (2012) Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  _Advances in neural information processing systems_ , 25, 2012.
  * Kumar et al. (2018) Abhishek Kumar, Prasanna Sattigeri, Kahini Wadhawan, Leonid Karlinsky, Rogerio Feris, Bill Freeman, and Gregory Wornell.  Co-regularized alignment for unsupervised domain adaptation.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * Lazaric (2012) Alessandro Lazaric.  Transfer in reinforcement learning: a framework and a survey.  In _Reinforcement Learning_ , pages 143–173. Springer, 2012.
  * LeCun et al. (2015) Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.  Deep learning.  _nature_ , 521(7553):436–444, 2015.
  * Lee et al. (2019a) Chen-Yu Lee, Tanmay Batra, Mohammad Haris Baig, and Daniel Ulbricht.  Sliced wasserstein discrepancy for unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 10285–10295, 2019a.
  * Lee et al. (2018a) Kimin Lee, Honglak Lee, Kibok Lee, and Jinwoo Shin.  Training confidence-calibrated classifiers for detecting out-of-distribution samples.  _ICLR_ , 2018a.
  * Lee et al. (2018b) Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  _NIPS_ , 2018b.
  * Lee et al. (2019b) Seungmin Lee, Dongwan Kim, Namil Kim, and Seong-Gyun Jeong.  Drop to adapt: Learning discriminative features for unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 91–100, 2019b.
  * Li et al. (2018a) Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot.  Domain generalization with adversarial feature learning.  In _CVPR_ , 2018a.
  * Li (2018) Jerry Li.  Twin-gan–unpaired cross-domain image translation with weight-sharing gans.  _arXiv preprint arXiv:1809.00946_ , 2018.
  * Li et al. (2019a) Wei Li, Wei Wei, Lei Zhang, Cong Wang, and Yanning Zhang.  Unsupervised deep domain adaptation for hyperspectral image classification.  In _IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium_ , pages 1–4. IEEE, 2019a.
  * Li et al. (2021a) Xiaohui Li, Sijie Niu, Xizhan Gao, Tingting Liu, and Jiwen Dong.  Unsupervised domain adaptation with self-selected active learning for cross-domain oct image segmentation.  In _International Conference on Neural Information Processing_ , pages 585–596. Springer, 2021a.
  * Li et al. (2018b) Yang Li, Wenming Zheng, Yuan Zong, Zhen Cui, Tong Zhang, and Xiaoyan Zhou.  A bi-hemisphere domain adversarial neural network model for eeg emotion recognition.  _IEEE Transactions on Affective Computing_ , 12(2):494–504, 2018b.
  * Li et al. (2016) Yanghao Li, Naiyan Wang, Jianping Shi, Jiaying Liu, and Xiaodi Hou.  Revisiting batch normalization for practical domain adaptation.  _arXiv preprint arXiv:1603.04779_ , 2016.
  * Li et al. (2018c) Yanghao Li, Naiyan Wang, Jianping Shi, Xiaodi Hou, and Jiaying Liu.  Adaptive batch normalization for practical domain adaptation.  _Pattern Recognition_ , 80:109–117, 2018c.
  * Li et al. (2019b) Yitong Li, Timothy Baldwin, and Trevor Cohn.  What’s in a domain? learning domain-robust text representations using adversarial training.  In _Proceedings of NAACL-HLT_ , pages 474–479, 2019b.
  * Li et al. (2021b) Yu-Jhe Li, Xiaoliang Dai, Chih-Yao Ma, Yen-Cheng Liu, Kan Chen, Bichen Wu, Zijian He, Kris Kitani, and Peter Vadja.  Cross-domain object detection via adaptive self-training.  _arXiv preprint arXiv:2111.13216_ , 2021b.
  * Lian et al. (2019) Qing Lian, Fengmao Lv, Lixin Duan, and Boqing Gong.  Constructing self-motivated pyramid curriculums for cross-domain semantic segmentation: A non-adversarial approach.  _ICCV_ , 2019.
  * Liang et al. (2018) Shiyu Liang, Yixuan Li, and R. Srikant.  Enhancing the reliability of out-of-distribution image detection in neural networks.  _ICLR_ , 2018.
  * Lim et al. (2020) KyungTae Lim, Jay Yoon Lee, Jaime Carbonell, and Thierry Poibeau.  Semi-supervised learning on meta structure: Multi-task tagging and parsing in low-resource scenarios.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 8344–8351, 2020.
  * Lipton et al. (2018) Zachary Lipton, Yu-Xiang Wang, and Alexander Smola.  Detecting and correcting for label shift with black box predictors.  In _ICML_ , 2018.
  * Liu et al. (2017) Xiaofeng Liu, BVK Vijaya Kumar, Jane You, and Ping Jia.  Adaptive deep metric learning for identity-aware facial expression recognition.  In _CVPR Workshops_ , pages 20–29, 2017.
  * Liu et al. (2018) Xiaofeng Liu, Yang Zou, Lingsheng Kong, Zhihui Diao, Junliang Yan, Jun Wang, Site Li, Ping Jia, and Jane You.  Data augmentation via latent space interpolation for image classification.  In _24th International Conference on Pattern Recognition (ICPR)_ , pages 728–733, 2018.
  * Liu et al. (2019a) Xiaofeng Liu, Xu Han, Yukai Qiao, Yi Ge, Site Li, and Jun Lu.  Unimodal-uniform constrained wasserstein training for medical diagnosis.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops_ , pages 0–0, 2019a.
  * Liu et al. (2019b) Xiaofeng Liu, Site Li, Lingsheng Kong, Wanqing Xie, Ping Jia, Jane You, and BVK Kumar.  Feature-level frankenstein: Eliminating variations for discriminative recognition.  In _CVPR_ , 2019b.
  * Liu et al. (2019c) Xiaofeng Liu, Yang Zou, Tong Che, Peng Ding, Ping Jia, Jane You, and BVK Kumar.  Conservative wasserstein training for pose estimation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 8262–8272, 2019c.
  * Liu et al. (2020a) Xiaofeng Liu, Fangfang Fan, Lingsheng Kong, Zhihui Diao, Wanqing Xie, Jun Lu, and Jane You.  Unimodal regularized neuron stick-breaking for ordinal classification.  _Neurocomputing_ , 2020a.
  * Liu et al. (2020b) Xiaofeng Liu, Yuzhuo Han, Song Bai, Yi Ge, Tianxing Wang, Xu Han, Site Li, Jane You, and Jun Lu.  Importance-aware semantic segmentation in self-driving with discrete wasserstein training.  In _AAAI_ , pages 11629–11636, 2020b.
  * Liu et al. (2020c) Xiaofeng Liu, Wenxuan Ji, Jane You, Georges El Fakhri, and Jonghye Woo.  Severity-aware semantic segmentation with reinforced wasserstein training.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12566–12575, 2020c.
  * Liu et al. (2020d) Xiaofeng Liu, Yunhong Lu, Xiongchang Liu, Song Bai, Site Li, and Jane You.  Wasserstein loss with alternative reinforcement learning for severity-aware semantic segmentation.  _IEEE Transactions on Intelligent Transportation Systems_ , 2020d.
  * Liu et al. (2021a) Xiaofeng Liu, Yang Chao, Jane J You, C-C Jay Kuo, and Bhagavatula Vijayakumar.  Mutual information regularized feature-level frankenstein for discriminative recognition.  _IEEE T-PAMI_ , 2021a.
  * Liu et al. (2021b) Xiaofeng Liu, Zhenhua Guo, Site Li, Fangxu Xing, Jane You, C-C Jay Kuo, Georges El Fakhri, and Jonghye Woo.  Adversarial unsupervised domain adaptation with conditional and label shift: Infer, align and iterate.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 10367–10376, 2021b.
  * Liu et al. (2021c) Xiaofeng Liu, Bo Hu, Linghao Jin, Xu Han, Fangxu Xing, Jinsong Ouyang, Jun Lu, Georges El Fakhri, and Jonghye Woo.  Domain generalization under conditional and label shifts via variational bayesian inference.  In _IJCAI_ , 2021c.
  * Liu et al. (2021d) Xiaofeng Liu, Bo Hu, Linghao Jin, Xu Han, Fangxu Xing, Jinsong Ouyang, Jun Lu, Georges EL Fakhri, and Jonghye Woo.  Domain generalization under conditional and label shifts via variational Bayesian inference.  _IJCAI_ , 2021d.
  * Liu et al. (2021e) Xiaofeng Liu, Bo Hu, Xiongchang Liu, Jun Lu, Jane You, and Lingsheng Kong.  Energy-constrained self-training for unsupervised domain adaptation.  In _2020 25th International Conference on Pattern Recognition (ICPR)_ , pages 7515–7520. IEEE, 2021e.
  * Liu et al. (2021f) Xiaofeng Liu, Linghao Jin, Xu Han, Jun Lu, Jane You, and Lingsheng Kong.  Identity-aware facial expression recognition in compressed video.  In _2020 25th International Conference on Pattern Recognition (ICPR)_ , pages 7508–7514. IEEE, 2021f.
  * Liu et al. (2021g) Xiaofeng Liu, Linghao Jin, Xu Han, and Jane You.  Mutual information regularized identity-aware facial expression recognition in compressed video.  _Pattern Recognition_ , 2021g.
  * Liu et al. (2021h) Xiaofeng Liu, Site Li, Yubin Ge, Pengyi Ye, Jane You, and Jun Lu.  Recursively conditional gaussian for ordinal unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 764–773, 2021h.
  * Liu et al. (2021i) Xiaofeng Liu, Xiongchang Liu, Bo Hu, Wenxuan Ji, Fangxu Xing, Jun Lu, Jane You, C-C Jay Kuo, Georges El Fakhri, and Jonghye Woo.  Subtype-aware unsupervised domain adaptation for medical diagnosis.  _AAAI_ , 2021i.
  * Liu et al. (2021j) Xiaofeng Liu, Fangxu Xing, Georges El Fakhri, and Jonghye Woo.  A unified conditional disentanglement framework for multimodal brain mr image translation.  In _2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI)_ , pages 10–14. IEEE, 2021j.
  * Liu et al. (2021k) Xiaofeng Liu, Fangxu Xing, Jerry L Prince, Aaron Carass, Maureen Stone, Georges El Fakhri, and Jonghye Woo.  Dual-cycle constrained bijective vae-gan for tagged-to-cine magnetic resonance image synthesis.  In _2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI)_ , pages 1448–1452. IEEE, 2021k.
  * Liu et al. (2021l) Xiaofeng Liu, Fangxu Xing, Maureen Stone, Jiachen Zhuo, Timothy Reese, Jerry L Prince, Georges El Fakhri, and Jonghye Woo.  Generative self-training for cross-domain unsupervised tagged-to-cine mri synthesis.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 138–148. Springer, 2021l.
  * Liu et al. (2021m) Xiaofeng Liu, Fangxu Xing, Chao Yang, Georges El Fakhri, and Jonghye Woo.  Adapting off-the-shelf source segmenter for target medical image segmentation.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 549–559. Springer, 2021m.
  * Liu et al. (2022a) Xiaofeng Liu, Site Li, Yubin Ge, Pengyi Ye, Jane You, and Jun Lu.  Ordinal unsupervised domain adaptation with recursively conditional gaussian imposed variational disentanglement.  In _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2022a.
  * Liu et al. (2022b) Xiaofeng Liu, Fangxu Xing, Georges El Fakhri, and Jonghye Woo.  Memory consistent unsupervised off-the-shelf model adaptation for source-relaxed medical image segmentation.  In _Medical Image Analysis_ , 2022b.
  * Liu et al. (2022c) Xiaofeng Liu, Fangxu Xing, Georges El Fakhri, and Jonghye Woo.  Self-semantic contour adaptation for cross modality brain tumor segmentation.  _IEEE International Symposium on Biomedical Imaging (ISBI)_ , 2022c.
  * Liu et al. (2022d) Xiaofeng Liu, Fangxu Xing, Thibault Marin, Georges El Fakhri, and Jonghye Woo.  Variational inference for quantifying inter-observer variability in segmentation of anatomical structures.  _arXiv preprint arXiv:2201.07106_ , 2022d.
  * Liu et al. (2022e) Xiaofeng Liu, Fangxu Xing, Jerry L Prince, Maureen Stone, Georges El Fakhri, and Jonghye Woo.  Structure-aware unsupervised tagged-to-cine mri synthesis with self disentanglement.  _arXiv preprint arXiv:2202.12474_ , 2022e.
  * Liu et al. (2022f) Xiaofeng Liu, Fangxu Xing, Nadya Shusharina, Ruth Lim, C-C Jay Kuo, Georges El Fakhri, and Jonghye Woo.  Act: Semi-supervised domain-adaptive medical image segmentation with asymmetric co-training.  In _MICCAI_ , 2022f.
  * Liu et al. (2022g) Xiaofeng Liu, Fangxu Xing, Jane You, Jun Lu, C-C Jay Kuo, Georges El Fakhri, and Jonghye Woo.  Subtype-aware dynamic unsupervised domain adaptation.  _IEEE Transactions on Neural Networks and Learning Systems_ , 2022g.
  * Liu et al. (2022h) Xiaofeng Liu, Chaehwa Yoo, Fangxu Xing, C.-C. Jay Kuo, Georges El Fakhri, and Jonghye Woo.  Unsupervised domain adaptation for segmentation with black-box source model.  _SPIE Medical Imaging 2022: Image Processing_ , 2022h.
  * Liu et al. (2022i) Xiaofeng Liu, Chaehwa Yoo, Fangxu Xing, C.-C. Jay Kuo, Georges El Fakhri, and Jonghye Woo.  Unsupervised black-box model domain adaptation for brain tumor segmentation.  _Frontiers in Neuroscience_ , 2022i.
  * Liu et al. (2019d) Yang Liu, Zhaoyang Lu, Jing Li, Tao Yang, and Chao Yao.  Deep image-to-video adaptation and fusion networks for action recognition.  _IEEE Transactions on Image Processing_ , 29:3168–3182, 2019d.
  * Long et al. (2015) Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I Jordan.  Learning transferable features with deep adaptation networks.  _ICML_ , 2015.
  * Long et al. (2016) Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan.  Unsupervised domain adaptation with residual transfer networks.  In _Advances in Neural Information Processing Systems_ , pages 136–144, 2016.
  * Long et al. (2017) Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan.  Deep transfer learning with joint adaptation networks.  In _Proceedings of the 34th International Conference on Machine Learning-Volume 70_ , pages 2208–2217. JMLR. org, 2017.
  * Long et al. (2018) Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan.  Conditional adversarial domain adaptation.  In _Advances in Neural Information Processing Systems_ , pages 1647–1657, 2018.
  * Lu et al. (2021) Nannan Lu, Hanhan Xiao, Yanjing Sun, Min Han, and Yanfen Wang.  A new method for intelligent fault diagnosis of machines based on unsupervised domain adaptation.  _Neurocomputing_ , 427:96–109, 2021.
  * Lucas et al. (2020) Benjamin Lucas, Charlotte Pelletier, Daniel Schmidt, Geoffrey I Webb, and François Petitjean.  Unsupervised domain adaptation techniques for classification of satellite image time series.  In _IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium_ , pages 1074–1077. IEEE, 2020.
  * Ma and Zhang (2021) Yuchi Ma and Zhou Zhang.  Multi-source unsupervised domain adaptation on corn yield prediction.  In _AI for Agriculture and Food Systems_ , 2021.
  * Mahapatra et al. (2021) Dwarikanath Mahapatra, Ruwan Tennakoon, et al.  Gcn based unsupervised domain adaptation with feature disentanglement for medical image classification.  2021\.
  * Mancini et al. (2018a) Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci.  Best sources forward: domain generalization through source-specific nets.  In _ICIP_ , 2018a.
  * Mancini et al. (2018b) Massimiliano Mancini, Lorenzo Porzi, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci.  Boosting domain adaptation by discovering latent domains.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 3771–3780, 2018b.
  * Mancini et al. (2019) Massimiliano Mancini, Samuel Rota Bulo, Barbara Caputo, and Elisa Ricci.  Adagraph: Unifying predictive and continuous domain adaptation through graphs.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6568–6577, 2019.
  * Manohar et al. (2018) Vimal Manohar, Pegah Ghahremani, Daniel Povey, and Sanjeev Khudanpur.  A teacher-student learning approach for unsupervised domain adaptation of sequence-trained asr models.  In _2018 IEEE Spoken Language Technology Workshop (SLT)_ , pages 250–257. IEEE, 2018.
  * Maria Carlucci et al. (2017) Fabio Maria Carlucci, Lorenzo Porzi, Barbara Caputo, Elisa Ricci, and Samuel Rota Bulo.  Autodial: Automatic domain alignment layers.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 5067–5075, 2017.
  * Matsuura and Harada (2020) Toshihiko Matsuura and Tatsuya Harada.  Domain generalization using a mixture of multiple latent domains.  _AAAI_ , 2020.
  * Mei et al. (2020a) Ke Mei, Chuang Zhu, Jiaqi Zou, and Shanghang Zhang.  Instance adaptive self-training for unsupervised domain adaptation.  _ECCV_ , 2020a.
  * Mei et al. (2020b) Ke Mei, Chuang Zhu, Jiaqi Zou, and Shanghang Zhang.  Instance adaptive self-training for unsupervised domain adaptation.  _ECCV_ , 2020b.
  * Mekhazni et al. (2020) Djebril Mekhazni, Amran Bhuiyan, George Ekladious, and Eric Granger.  Unsupervised domain adaptation in the dissimilarity space for person re-identification.  In _European Conference on Computer Vision_ , pages 159–174. Springer, 2020.
  * Mengqiu et al. (2022) XU Mengqiu, WU Ming, GUO Jun, Chuang Zhang, WANG Yubo, and MA Zhanyu.  Sea fog detection based on unsupervised domain adaptation.  _Chinese Journal of Aeronautics_ , 35(4):415–425, 2022.
  * Michau and Fink (2021) Gabriel Michau and Olga Fink.  Unsupervised transfer learning for anomaly detection: Application to complementary operating condition transfer.  _Knowledge-Based Systems_ , 216:106816, 2021.
  * Montesuma and Mboula (2021) Eduardo Fernandes Montesuma and Fred Maurice Ngole Mboula.  Wasserstein barycenter for multi-source domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 16785–16793, 2021.
  * Morerio et al. (2017) Pietro Morerio, Jacopo Cavazza, and Vittorio Murino.  Minimal-entropy correlation alignment for unsupervised deep domain adaptation.  _arXiv preprint arXiv:1711.10288_ , 2017.
  * Motiian et al. (2017) Saeid Motiian, Quinn Jones, Seyed Iranmanesh, and Gianfranco Doretto.  Few-shot adversarial domain adaptation.  _Advances in neural information processing systems_ , 30, 2017.
  * Mou et al. (2021) Quanzheng Mou, Longsheng Wei, Conghao Wang, Dapeng Luo, Songze He, Jing Zhang, Huimin Xu, Chen Luo, and Changxin Gao.  Unsupervised domain-adaptive scene-specific pedestrian detection for static video surveillance.  _Pattern Recognition_ , 118:108038, 2021.
  * Naik and Rose (2020) Aakanksha Naik and Carolyn Rose.  Towards open domain event trigger identification using adversarial domain adaptation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7618–7624, 2020.
  * Nasim et al. (2022) M Quamer Nasim, Tannistha Maiti, Ayush Srivastava, Tarry Singh, and Jie Mei.  Seismic facies analysis: a deep domain adaptation approach.  _IEEE Transactions on Geoscience and Remote Sensing_ , 60:1–16, 2022.
  * Ning et al. (2021) Munan Ning, Cheng Bian, Dong Wei, Shuang Yu, Chenglang Yuan, Yaohua Wang, Yang Guo, Kai Ma, and Yefeng Zheng.  A new bidirectional unsupervised domain adaptation segmentation framework.  In _International Conference on Information Processing in Medical Imaging_ , pages 492–503. Springer, 2021.
  * Niu et al. (2020) Lisha Niu, Chao Chen, Hui Liu, Shuwang Zhou, and Minglei Shu.  A deep-learning approach to ecg classification based on adversarial domain adaptation.  In _Healthcare_ , volume 8, page 437. Multidisciplinary Digital Publishing Institute, 2020.
  * Nyborg et al. (2022) Joachim Nyborg, Charlotte Pelletier, Sébastien Lefèvre, and Ira Assent.  Timematch: Unsupervised cross-region adaptation by temporal shift estimation.  _ISPRS Journal of Photogrammetry and Remote Sensing_ , 188:301–313, 2022.
  * Ouyang et al. (2019) Cheng Ouyang, Konstantinos Kamnitsas, Carlo Biffi, Jinming Duan, and Daniel Rueckert.  Data efficient unsupervised domain adaptation for cross-modality image segmentation.  In _International Conference on Medical Image Computing and Computer-Assisted Intervention_ , pages 669–677. Springer, 2019.
  * Oza et al. (2021) Poojan Oza, Vishwanath A Sindagi, Vibashan VS, and Vishal M Patel.  Unsupervised domain adaptation of object detectors: A survey.  _arXiv preprint arXiv:2105.13502_ , 2021.
  * Pan et al. (2020) Boxiao Pan, Zhangjie Cao, Ehsan Adeli, and Juan Carlos Niebles.  Adversarial cross-domain action recognition with co-attention.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 11815–11822, 2020.
  * Pan and Yang (2009) Sinno Jialin Pan and Qiang Yang.  A survey on transfer learning.  _IEEE Transactions on knowledge and data engineering_ , 22(10):1345–1359, 2009.
  * Pan et al. (2019) Yingwei Pan, Ting Yao, Yehao Li, Yu Wang, Chong-Wah Ngo, and Tao Mei.  Transferrable prototypical networks for unsupervised domain adaptation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 2239–2247, 2019.
  * Panareda Busto and Gall (2017) Pau Panareda Busto and Juergen Gall.  Open set domain adaptation.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pages 754–763, 2017.
  * Peng et al. (2019a) Jiangtao Peng, Weiwei Sun, Li Ma, and Qian Du.  Discriminative transfer joint matching for domain adaptation in hyperspectral image classification.  _IEEE Geoscience and Remote Sensing Letters_ , 16(6):972–976, 2019a.
  * Peng et al. (2017) Xingchao Peng, Ben Usman, Neela Kaushik, Judy Hoffman, Dequan Wang, and Kate Saenko.  Visda: The visual domain adaptation challenge, 2017.
  * Peng et al. (2019b) Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang.  Moment matching for multi-source domain adaptation.  _ICCV_ , 2019b.
  * Perone et al. (2019) Christian S Perone, Pedro Ballester, Rodrigo C Barros, and Julien Cohen-Adad.  Unsupervised domain adaptation for medical imaging segmentation with self-ensembling.  _NeuroImage_ , 194:1–11, 2019.
  * Pimentel et al. (2014) Marco A. F. Pimentel, David A. Clifton, Clifton Lei, and Lionel Tarassenko.  A review of novelty detection.  _Signal Processing_ , 99(6):215–249, 2014.
  * Porwal et al. (2018) Prasanna Porwal, Samiksha Pachade, Ravi Kamble, Manesh Kokare, Girish Deshmukh, Vivek Sahasrabuddhe, and Fabrice Meriaudeau.  Indian diabetic retinopathy image dataset (idrid): a database for diabetic retinopathy screening research.  _Data_ , 3(3):25, 2018.
  * Purushotham et al. (2017) S. Purushotham, Wilka Carvalho, Tanachat Nilanon, and Yan Liu.  Variational recurrent adversarial deep domain adaptation.  In _ICLR_ , 2017.
  * Ragab et al. (2020) Mohamed Ragab, Zhenghua Chen, Min Wu, Chuan Sheng Foo, Chee Keong Kwoh, Ruqiang Yan, and Xiaoli Li.  Contrastive adversarial domain adaptation for machine remaining useful life prediction.  _IEEE Transactions on Industrial Informatics_ , 17(8):5239–5249, 2020.
  * Ramponi and Plank (2020) Alan Ramponi and Barbara Plank.  Neural unsupervised domain adaptation in nlp—a survey.  _arXiv preprint arXiv:2006.00632_ , 2020.
  * Raza and Samothrakis (2019) Haider Raza and Spyridon Samothrakis.  Bagging adversarial neural networks for domain adaptation in non-stationary eeg.  In _2019 International Joint Conference on Neural Networks (IJCNN)_ , pages 1–7. IEEE, 2019.
  * Richter et al. (2016) Stephan R Richter, Vibhav Vineet, Stefan Roth, and Vladlen Koltun.  Playing for data: Ground truth from computer games.  In _ECCV_ , 2016.
  * Rios et al. (2018) Anthony Rios, Ramakanth Kavuluru, and Zhiyong Lu.  Generalizing biomedical relation classification with neural adversarial domain adaptation.  _Bioinformatics_ , 34(17):2973–2981, 2018.
  * Rocha and Cardoso (2019) Gil Rocha and Henrique Lopes Cardoso.  A comparative analysis of unsupervised language adaptation methods.  In _Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)_ , pages 11–21, 2019.
  * Rodriguez and Mikolajczyk (2019) Adrian Lopez Rodriguez and Krystian Mikolajczyk.  Domain adaptation for object detection via style consistency.  _arXiv preprint arXiv:1911.10033_ , 2019.
  * Rotman and Reichart (2019) Guy Rotman and Roi Reichart.  Deep contextualized self-training for low resource dependency parsing.  _Transactions of the Association for Computational Linguistics_ , 7:695–713, 2019.
  * RoyChowdhury et al. (2019) Aruni RoyChowdhury, Prithvijit Chakrabarty, Ashish Singh, SouYoung Jin, Huaizu Jiang, Liangliang Cao, and Erik Learned-Miller.  Automatic adaptation of object detectors to new domains using self-training.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 780–790, 2019.
  * Royer and Lampert (2015) Amelie Royer and Christoph H Lampert.  Classifier adaptation at prediction time.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 1401–1409, 2015.
  * Rozantsev et al. (2018) Artem Rozantsev, Mathieu Salzmann, and Pascal Fua.  Beyond sharing weights for deep domain adaptation.  _IEEE transactions on pattern analysis and machine intelligence_ , 41(4):801–814, 2018.
  * Saenko et al. (2010) Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell.  Adapting visual category models to new domains.  In _European conference on computer vision_ , pages 213–226. Springer, 2010.
  * Saito et al. (2017) Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko.  Adversarial dropout regularization.  _arXiv preprint arXiv:1711.01575_ , 2017.
  * Saito et al. (2018) Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada.  Maximum classifier discrepancy for unsupervised domain adaptation.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 3723–3732, 2018.
  * Saito et al. (2019a) Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko.  Semi-supervised domain adaptation via minimax entropy.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 8050–8058, 2019a.
  * Saito et al. (2019b) Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko.  Strong-weak distribution alignment for adaptive object detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 6956–6965, 2019b.
  * Sakaridis et al. (2018) Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Semantic foggy scene understanding with synthetic data.  _International Journal of Computer Vision_ , 126(9):973–992, 2018.
  * Saleh et al. (2019) Khaled Saleh, Ahmed Abobakr, Mohammed Attia, Julie Iskander, Darius Nahavandi, Mohammed Hossny, and Saeid Nahvandi.  Domain adaptation for vehicle detection from bird’s eye view lidar point cloud data.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops_ , pages 0–0, 2019.
  * Salimans et al. (2016) Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen.  Improved techniques for training gans.  In _Advances in neural information processing systems_ , pages 2234–2242, 2016.
  * Sanabria et al. (2021) Andrea Rosales Sanabria, Franco Zambonelli, and Juan Ye.  Unsupervised domain adaptation in activity recognition: A gan-based approach.  _IEEE Access_ , 9:19421–19438, 2021.
  * Sankaranarayanan et al. (2018) Swami Sankaranarayanan, Yogesh Balaji, Carlos D Castillo, and Rama Chellappa.  Generate to adapt: Aligning domains using generative adversarial networks.  In _CVPR_ , 2018.