#  Mitigating Uncertainty of Classifier for Unsupervised Domain Adaptation

Shanu Kumar
Microsoft, India
<EMAIL_ADDRESS>Vinod K Kurmi
IIT Kanpur
<EMAIL_ADDRESS>Praphul Singh
Oracle, India
<EMAIL_ADDRESS>Vinay P Namboodiri
University of Bath
<EMAIL_ADDRESS>
###### Abstract

Understanding unsupervised domain adaptation has been an important task that
has been well explored. However, the wide variety of methods have not analyzed
the role of a classifier’s performance in detail. In this paper, we thoroughly
examine the role of a classifier in terms of matching source and target
distributions. We specifically investigate the classifier ability by matching
a) the distribution of features, b) probabilistic uncertainty for samples and
c) certainty activation mappings. Our analysis suggests that using these three
distributions does result in a consistently improved performance on all the
datasets. Our work thus extends present knowledge on the role of the various
distributions obtained from the classifier towards solving unsupervised domain
adaptation.

##  1 Introduction

Efficacy of using deep neural networks for solving a variety of problems in
computer vision has been well established [18, 12, 31, 3]. These models are
trained on large datasets and work in a variety of real-world settings.
However, it has been shown [41] that even after being trained on large
datasets, these models suffer from inferior performance when they are used on
data that differ for reasons such as background clutter, distribution of
classes and illumination and pose conditions. To address this mismatch in
performance, there has been a large body of work that aims to reduce the
covariate shift by adapt features using unsupervised domain adaptation [29,
45]. However, these techniques proposed so far have mainly ignored the
response of the classifiers. They have focused on aligning the feature
representations between the source and target domains that are input to the
classifiers. In our work, we argue that such an approach under-utilizes the
use of the classifier. Through this work, we analyse the role of the classifer
and observe that carefully matching the classifier’s response for source and
target domains can result in consistent improvement in performance and lower
uncertainty in the target domain.

(a)

(b)

(c) Predictive uncertainty

Figure 1: Comparison of adversarial feature adaptation and our proposed method
( red: source domain and  blue: target domain). (a) The mean and variance of
the decision boundary are not correctly approximated for the target domain,
due to this all the samples of decision boundary do not separate most target
domain examples. (b) Our approach learns better approximation of the mean and
variance of the decision boundary and also learns more class-discriminative
feature representations for the target domain. (c) Predictive uncertainty
estimates of the target domain is relatively much higher than of the source
domain images after only adapting the distribution of features on Office-31
dataset (AlexNet). For A$\rightarrow$D, uncertainty in the source domain is
almost 0.

In order to understand the existing dominant approach adopted by most
classifiers, we need to understand the underlying assumption. Most of the
existing works in domain adaptation [24, 9, 42] have focused on learning
domain-invariant feature representations. They work on the assumption that the
classifier learned in the source domain can be used in the target domain after
feature adaptation. This theory is based on the principle that the posterior
of the classifier in the target domain may be similar to the posterior learned
in the source domain. But this may not be true always as the label-feature
distribution can differ for the source and target domain. In figure 1(a), we
illustrate an instance where the feature representations are domain-invariant
and yet, the features in the target domain are not class-discriminative for
the classifier learned in the source domain. As the classifier is only trained
on the source domain, all the decision boundaries sampled from the posterior
of the classifier entirely separates both classes in the source domain. Even
though features are domain invariant, every sampled decision boundary do not
separate both the classes in the target domain which results in lower
performance and higher uncertainty. We observe similar pattern after
performing adversarial adaptation of features for W$\rightarrow$A task using
AlexNet. The features shown in the figure 2(a) are domain-invariant. In the
figure 2(b), we can verify that features of the source domain images form
unique class clusters but all the features of the target domain images are not
class discriminative. The tools we use to analyze this performance are based
on probabilistic uncertainty measures [6, 17] and certainty activation
mappings [19]. Recently, there have been a few methods that have focused on
classifier adaptation [38, 47, 23]. [38] addressed this issue by incorporating
conditional entropy loss which refines the decision boundary, while [23]
enables the adaptation by generating transferable examples to fill in the gap
between the source and target domains. However, we believe that they have not
sufficiently addressed this problem.

In this paper, we solve the problem by learning a better approximation of the
posterior of the classifier in the target domain. This is challenging as we
cannot approximate the true posterior of the classifier without the class
labels. We propose an unsupervised method that jointly aligns three specific
distributions across domains. In our analysis, we observed that there is
higher predictive uncertainty in the target domain as compared to the source
domain. Also, there is a domain shift in the probability distribution of the
classifier, which strengthens our belief that the mean of the posterior of the
classifier is not correctly approximated in the target domain. Recently, [19]
has proposed certainty activation mappings that highlight the regions where
the classifier is certain during predictions. We observe that certainty
activation mappings in the source domain highlight class-discriminative
regions, but in the target domain due to high uncertainty, these mappings do
not highlight class-discriminative regions. This suggests that the
approximation of the variance of the posterior is also not correct in the
target domain. Based on these findings, we propose Triple Distribution
Matching for Domain Adaptation (TDMDA), which jointly matches the distribution
of feature representations, probability distributions, and certainty
activation mappings from the target domain to that of the source domain. The
main contributions of this paper are summarized as follows:

  * •

We investigate the limitations of the traditional adversarial feature
adaptation methods based on predictive uncertainty, probability distribution,
and certainty activation mappings.

  * •

We propose TDMDA for learning a better approximation of the posterior of the
classifier in the target domain by jointly matching three distributions.

  * •

We provide the evaluation of our method on various domain adaptation benchmark
datasets and its comparisons with most of the state of the art methods.

  * •

We show the effectiveness of our method by providing an empirical analysis
using the estimates of predictive uncertainty and various visualizations.

##  2 Related Work

(a) Class-wise Feature Representation

(b) Domain-wise Feature Representation

(c) Class-wise Probability Distribution

(d) Domain-wise Probability Distribution

Figure 2: Domain-wise and Class-wise t-SNE visualization of feature and
probability distribution after preforming adversarial feature adaptation on
Office31 dataset ( Source: W and  Target: A) using AlexNet. The distribution
of feature representation is domain-invariant after performing feature
adaptation but many target domain features are not class-discriminative. In
the probability distribution, we can observe domain shift between the target
and source domain data points. We also have provided the magnified version of
these plots in the supplementary material.

Domain Adaptation: There has been a wide variety of feature adaptation
techniques for solving unsupervised domain adaptation. Early approaches were
based on feature alignment through estimation of divergence between
distributions such as MMD [43, 24, 49], CORAL [39], CCD and Wasserstein metric
[37]. Ganin et al.[8] enable the model to learn both class-discriminative and
domain invariant representations by training a discriminator in an adversarial
way. Other approaches such as [36, 42, 41] have also proposed methods based on
adversarial domain adaptation. More recent methods [13, 34, 14, 2] have
performed unpaired image-to-image translation using generative adversarial
networks (GAN) [10] by generating synthetic images of one domain in other.
Attention based approaches have been also applied in domain adaptation. TADA
[46] uses entropy of the local and global discriminators for finding
transferable regions which are also used as attention for classifier. In CADA
[19], attention is obtained by discriminator’s certainty activation mappings
and are used to focus on more discriminative regions. Huang et al[15] shows
that there is domain discrepancy in the classifier by visualizing attention
mechanism. They propose to align the distributions of activations of
intermediate layers of source and target to adapt the classifier. We also
propose a similar idea except the self attention or class activation mappings,
we align certainty activation mappings.

Along with feature adaptation, few approaches have explored adaptation of
classifier. Shu et al. [38] propose decision-boundary refinement via
conditional entropy loss. They have also incorporated domain adversarial
training which penalizes violation of the cluster assumption. [23] encourages
adaptation of classifier by generating transferable adversarial examples.
Similarly, [20] uses adversarial dropout to learn discriminative features by
pushing the decision boundary away from the target domain’s features. [5] also
aligns the class conditional distribution of features of source and target
domain. These have however, not studied classifier’s distributions thoroughly
as proposed in this work. We analyze it and provide a more general framework
that considers various distributions for the classifier and jointly matches
them to enforces both class-discriminative features and lower uncertainty in
target domain.

Uncertainty Estimation: Our method relies on the various uncertainty
estimation techniques that have been investigated. We provide an overview of
the literature in this area. Bayesian neural networks (BNN) have been used in
a wide variety of tasks for estimating predictive uncertainty. But determining
the exact inference of $p(\mathbf{Y}|\mathbf{X})$ is an intractable problem
because computing the posterior expression
$p(\theta|\mathbf{X},\mathbf{Y})=p(\mathbf{Y}|\mathbf{X},\theta)p(\theta)/p(\mathbf{Y}|\mathbf{X})$
is analytically impossible. Thus the posterior
$p(\theta|\mathbf{X},\mathbf{Y})$ is approximated by a simple distribution
$q^{*}(\theta)$ parameterized by $\\{\mu,\sigma\\}$. To approximate inference
for deep learning models, [7] has proposed a Dropout variational inference
method. In [28], it has been shown that dropout variational inference has
better calibration under in-domain distribution or distributional shifts than
Temp Scaling [11], SVI[1] and LL[32]. Thus, we have employed dropout
variational inference for quantifying predictive uncertainty.

Based on [7], Kurmi et al. [19] has incorporated estimation of uncertainty for
domain adaptation. They propose a method to visualize the activation mappings
of certainty and uncertainty estimates. In their work, the discriminator’s
certainty activation mappings are used as an attention for guiding the
classifier, but the classifier uncertainty estimates, and certainty activation
mappings are not investigated. As a result, the method does not ensure the
discriminativity of the adapted features. This drawback is thoroughly
investigated and addressed in our work. [47] propose to minimize the
difference of predictive uncertainty of the source and target domains, along
with feature adaptation.

##  3 Probabilistic Justification for the Proposed Approach

In general, any adversarial feature adaptation method focuses only on reducing
the domain shift in the feature distribution, thus making
$\mathcal{P}(F(\mathbf{X}_{s}))\approx\mathcal{P}(F(\mathbf{X}_{t}))$. The
inference in target domain can be defined mathematically as
$\mathcal{P}(\mathbf{Y}_{t}|\mathbf{X}_{t})=\mathcal{P}(\mathbf{Y}_{t}|F(\mathbf{X}_{t}))\mathcal{P}(F(\mathbf{X}_{t}))$.
Adapting the features results in
$\mathcal{P}(F(\mathbf{X}_{s}))\approx\mathcal{P}(F(\mathbf{X}_{t}))$. Now,
$\mathcal{P}(\mathbf{Y}_{t}|F(\mathbf{X}_{t}))$ requires evaluation of the
posterior $p(\theta_{c}|F(\mathbf{X}_{t}),\mathbf{Y}_{t})$ for inference.
Existing methods assume that the posterior of target-domain classifier can be
approximated directly by using the posterior,
$p(\theta_{c}|F(\mathbf{X}_{s}),\mathbf{Y}_{s})$ learned from the source
domain. But the approximation $q^{*}(\theta_{c})$, learned by using the source
domain do not guarantee that $p(\theta_{c}|F(\mathbf{X}_{s}),\mathbf{Y}_{s})$
is the correct approximation of
$p(\theta_{c}|F(\mathbf{X}_{t}),\mathbf{Y}_{t})$ as the class labels for the
target domain are not used. Therefore, reducing the domain shift alone do not
guarantee that
$\mathcal{P}(\mathbf{Y}_{s}|F(\mathbf{X}_{s}))\approx\mathcal{P}(\mathbf{Y}_{t}|F(\mathbf{X}_{t}))$.

###  3.1 High Predictive Uncertainty in Target Domain

In figure 1(c), we have plotted the predictive uncertainty for source and
target domains after the feature adaptation in Office-31 dataset [33]. From
the figure, it is evident that after the feature adaptation, there is
reasonably high predictive uncertainty still present in the target domain. The
reason for the high predictive uncertainty is that the parameters {$\mu_{c}$,
$\sigma_{c}$} of the posterior $q^{*}(\theta_{c})$, learned from the source
domain are not the correct approximation of the posterior,
$p(\theta_{c}|F(\mathbf{X}_{t}),\mathbf{Y}_{t})$ of the target domain. In the
proposed method, we show that by considering the unlabeled target domain data
$\mathbf{X}_{t}$, the posterior $q^{*}(\theta_{c})$ can be approximated better
for the target domain. The result for the same is provided later in figure 6.

###  3.2 Domain Shift in Probability Distribution

Figure 2 shows the probability distribution after performing the adversarial
adaptation of features. In figures 2(d) and 2(c), we can see that all the data
points lie on curved lines with unique class clusters in the source domain,
but data points of the target domain do not follow a similar structure. Many
data points in the target domain are in the center of the plot and do not
correspond to any cluster. These are also not class-specific and causes a
domain shift in the probability distribution $\hat{y}_{c}$ of the source and
target domain. Domain shift can happen if either the mean of decision
boundaries do not separate the data well in the target domain or the margin
between the data and mean of decision boundaries is very small in the target
domain. We can assume that this is a result of incorrect approximation of the
$\mu_{c}$ of $q^{*}(\theta_{c})$, as the classifier is trained only using
source-domain labels. We can enforce better approximation of the mean,
$\mu_{c}$ of $q^{*}(\theta_{c})$ by minimizing the domain shift in the
probability distribution $\hat{y}_{c}$, as we want the mean of decision
boundaries to separate the classes in both domains very well with a large
margin.

Figure 3: Visualization of certainty and uncertainty activation mappings of
last convolutional layer in source domain. Uncertainty activation mappings are
shown in (b) and (e) and, certainty activation mappings are shown in (c) and
(f). Certainty activation mappings also highlight the class-discriminative
regions of an image.

###  3.3 Certainty Activation Mapping

Similar to class activation mappings proposed in [35], we can compute
certainty activation mappings to identify the regions where the classifier is
certain for performing predictions. Like class activation mappings, certainty
activation mappings are highly class-discriminative for images of the source
domain. Similarly, uncertainty activation mappings lead us to the uncertain
regions for the predictions. By computing both the mappings through the
approach proposed in [19], we can visualize the certain and uncertain regions
in the images for the classifier. This, in turn, provides much more
information than the class activation mappings. In figure 3, certainty and
uncertainty activation mappings are shown respectively for the source domain
for Office-31 dataset. In figure 4, certainty activation mapping in the target
domain is shown after adversarial adaptation of features. We can observe that
the class-discriminative regions are highlighted by uncertainty activation
mappings instead of certainty activation mappings. This was not the case for
the images in source domain as the posterior, $q^{*}(\theta_{c})$ is
approximated well for the source domain. For the target domain, we are using
the same posterior, which therefore results in high uncertainty and non-class
discriminative certainty activation mappings.

Figure 4: Visualization of certainty activation mappings of the last
convolutional layer in target domain after adversarial feature adaptation for
A$\rightarrow$D (AlexNet). These mappings are not highlighting any class-
discriminative region.

If the variance $\sigma_{c}$ of $q^{*}(\theta_{c})$ is not learned correctly
for the target domain, it will result in the incorrect sampling of decision
boundaries. These decision boundaries may not separate the data well and thus
will cause higher predictive uncertainty for target domain images. Variance
for the target domain cannot be directly approximated, but reducing the
uncertainty or any function proportional to uncertainty can lead to a better
approximation of the variance $\sigma_{c}$. A similar kind of approach has
been proposed in [47], where the difference between the predictive
uncertainties of source and target domain is minimized for producing
consistent predictions. One important point to note is that reduction in
predictive uncertainty do not necessarily mean a reduction in uncertainty on
the class-discriminative regions. On the contrary, it may also be due to a
higher reduction of uncertainty for the non-class discriminative regions. This
brings us to an observation that learning correct explanation is better than
learning correct answers.

We propose a mechanism to ameliorate the certainty activation mappings in the
target domain to be class discriminative by matching the distribution of
certainty activation mappings of the target domain to that of the source
domain. As certainty activation mapping is a function of uncertainty,
therefore this will lead to a reduction in uncertainty and finally to a better
approximation of variance $\sigma_{c}$ of $q^{*}(\theta_{c})$ in the target
domain.

Figure 5: The architecture of Triple Distribution Matching for Domain
Adaptation (TDMDA) consists of feature extractor $F$, Classifier $C$, Feature
domain discriminator $D_{f}$, Probability distribution domain discriminator
$D_{p}$, Certainty activation mapping domain discriminator $D_{c}$ and
Certainty activation mapping generator $G$.

##  4 Methodology

In unsupervised domain adaptation, we have a source domain $P(X_{s},Y_{s})$
with $n_{s}$ i.i.d. labeled observations
${\\{x^{s}_{i},y^{s}_{i}\\}}^{n_{s}}_{i=1}$ and a target domain
$P(X_{t},Y_{t})$ with $n_{t}$ i.i.d. unlabeled observations
${\\{x^{t}_{i}\\}}^{n_{t}}_{i=1}$. The goal is to learn transferable features
$f=F(x)$ and an adaptive classifier $C$ which minimises the generalisation
error on the target domain with lower uncertainty. For learning better
approximation of the posterior $q^{*}(\theta_{c})$ for target domain, we
propose Triple Distribution Matching for Domain Adaptation (TDMDA), which
solves the following three tasks: (a) Feature adaptation, (b) Predictive
probability adaptation and (c) Certainty Activation Mapping adaptation.

###  4.1 Feature Adaptation

Adversarial adaptation of features aims to learn both class-discriminative and
domain-invariant features. Domain-invariance is achieved by training a domain
discriminator $D_{f}$, which predicts the domain label for both the source and
target domain features, which are generated by training the feature extractor
$F(x_{i})$. The aim is to confuse the discriminator $D_{f}$ so that it is
unable to distinguish between source and target domain, thus making the
feature $f_{i}$ domain-invariant. The classifier $C$ is trained to learn class
discriminative features through the cross-entropy loss the source domain:

$\mathcal{L}_{c}=\frac{1}{n_{s}}\sum_{x_{i}\in\mathcal{D}_{s}}\mathcal{L}(C(\mathbf{f_{i}}),y_{i}^{s})$
(1)

where $y_{i}^{s}$ is true class label. The domain discriminator $D_{f}$ is
trained using the following loss:

$\mathcal{L}_{df}=\frac{1}{n_{s}+n_{t}}\sum_{x_{i}\in\mathcal{D}_{s}\cup\mathcal{D}_{t}}\mathcal{L}(D_{f}(\mathbf{f_{i}}),d_{i})$
(2)

where $d_{i}$ is the domain label. The predictive uncertainty $U_{i}$ is
estimated by using entropy of the average class probabilities of $T$ Monte
Carlo samples of the features.

$\displaystyle p({y}_{i,t}^{c})=\text{Softmax}(C(\mathbf{f_{i,t}}))$ (3)
$\displaystyle
U_{i}=-\frac{1}{\text{T}}\sum_{t=1}^{T}\sum_{c=1}^{C}p({y}_{i,t}^{c}=c)*\log
p({y}_{i,t}^{c}=c)$ (4)

###  4.2 Probability Distribution Adaptation

Earlier works used the mean of posterior $\mu_{c}$ learned from the source
domain for the posterior of the target domain. For high-confidence predictions
in the target domain, the $\mu_{c}$ for the target domain needs to be
approximated separately using target domain images. Therefore, we aim at
minimizing the domain shift in the probability distribution
$C(\mathbf{f}_{i})$ of source and target domains for learning a better
approximation of the mean $\mu_{c}$ of the posterior of the classifier.
Similar to adversarial feature adaptation, we train a domain discriminator
$D_{p}$ for predicting the domain of probability distribution
$C(\mathbf{f}_{i})$. The classifier $C$, on the other hand, is trained to fool
the discriminator by predicting high-confidence class predictions for both
source and target domains. The probability distribution adaptation also
results in learning more domain-invariant feature representation. The loss
function for probability distribution adaptation is defined in Eq. 5.

$\displaystyle\mathcal{L}_{dp}=\frac{1}{n_{s}+n_{t}}\sum_{x_{i}\in\mathcal{D}_{s}\cup\mathcal{D}_{t}}\mathcal{L}(D_{p}(C(\mathbf{f_{i}})),d_{i})$
(5)

###  4.3 Certainty Activation Mapping Adaptation

We have followed [19] for obtaining certainty activation mappings
$\mathbf{C}_{i}^{t}$ for the classification task. We perform element-wise
multiplication of feature with negative of the gradient of predictive
uncertainty $U_{i}$ with respect to feature. We are only interested in the
features that have a positive influence on the certainty estimates i.e.,
pixels whose intensity should be increased in order to improve the certainty.
Therefore negative activations are replaced by a large negative number $b$,
which will become zero after applying Softmax.

$\displaystyle
s_{i}=\begin{cases}\mathbf{f}_{i}*-\frac{\partial{U_{i}}}{\partial{{\mathbf{f}}_{i}}}&\text{if
}\mathbf{f}_{i}*-\frac{\partial{U_{i}}}{\partial{\mathbf{f}_{i}}}\geq 0\\\
b,&\text{otherwise}\end{cases}$ (6)
$\displaystyle\mathbf{C}_{i}^{t}=1+\text{Softmax}(s_{i})$ (7)

Certainty activation mapping highlights the class-discriminative regions in
the source domain. But, for many cases in the target domain, this is not
observed, and this causes a domain shift for certainty activation mappings
between source and target domains. Due to the wrong approximation of the
variance $\sigma_{c}$ of the posterior of classifier $C$, there is high
predictive uncertainty for the target domain as the classifier is uncertain on
class-discriminative regions. To fix the issue, we propose to match the
distribution of certainty activation mappings of the target domain to that of
the source domain by using a domain discriminator $D_{c}$. The classifier is
trained to fool the discriminator $D_{c}$ by learning a better approximation
of the variance $\sigma_{c}$.

The certainty activation mapping is a function of class logits
$C(\mathbf{f}_{i})$ and features $\mathbf{f}_{i}$. To fool the discriminator
$D_{c}$, we need to compute the gradients of predictive uncertainty with
respect to certainty activation map, which itself is a function of derivative
of predictive uncertainty. This will result in the computation of second
derivatives of predictive uncertainty with respect to the features. We propose
a way to deal with this complexity. As Neural Networks are universal
approximators, we can easily learn the certainty activation mapping function
using them. Therefore, we suggest using a generator $G$ for generating the
certainty activation mappings for both source and target domains. Now, since
we are generating the certainty activation mapping $\mathbf{C}_{i}^{g}$, we
only have to deal with the first derivatives, thus making the computations
simpler than before. Certainty activation mapping generator $G$ is trained by
minimizing the mean squared loss $\mathcal{L}_{g}$.

$\displaystyle\mathbf{C}_{i}^{g}=G(C(\mathbf{f}_{i}),\mathbf{f}_{i})$ (8)
$\displaystyle\mathcal{L}_{g}=||\mathbf{C}_{i}^{t}-\mathbf{C}_{i}^{g}||^{2}$
(9)

Certainty activation mapping do not provide any information without the
feature or image. Therefore the discriminator $D_{c}$ will have both certainty
activation mappings $\mathbf{C}_{i}^{g}$ and $\mathbf{f}_{i}$ as inputs. Since
we only aim to match the distribution of certainty activation mappings here,
the features will act as weights for the certainty activation mappings. The
discriminator $D_{c}$ is trained using the loss function $\mathcal{L}_{dc}$.

$\displaystyle\mathcal{L}_{dc}=\frac{1}{n_{s}+n_{t}}\sum_{x_{i}\in\mathcal{D}_{s}\cup\mathcal{D}_{t}}\mathcal{L}(D_{c}(\mathbf{C}_{i}^{g},\mathbf{f}_{i}),d_{i})$
(10)

###  4.4 Final Objective

We enable better approximation of the posterior $q^{*}(\theta_{c})$ in both
source and target domain by Triple Distribution Matching for Domain Adaptation
(TDMDA), which jointly learns class discriminative and domain invariant
features, domain invariant probability distribution, and class-discriminative
certainty activation mappings. The final objective function $J$ of TDMDA is
defined as:

$J(\theta_{f},\theta_{c},\theta_{df},{\theta}_{g},\theta_{dp},\theta_{dc})=\mathcal{L}_{c}+\mathcal{L}_{g}-\lambda_{f}*\mathcal{L}_{df}\\\
-\lambda_{p}*\mathcal{L}_{dp}-\lambda_{c}*\mathcal{L}_{dc}$ (11)

where $\theta_{f},\theta_{c},\theta_{df},{\theta}_{g},\theta_{dp},\theta_{dc}$
are the parameters of feature extractor $F$, classifier $C$, feature domain
discriminator $D_{f}$, certainty activation mapping generator $G$, probability
distribution domain discriminator $D_{p}$ and certainty activation mapping
domain discriminator $D_{c}$ respectively.
$\lambda_{f},\lambda_{p},\lambda_{c}$ are hyper-parameters that provides a
trade-off between classifier and discriminators. The optimization problem is
to find the parameters
$\hat{\theta}_{f},\hat{\theta}_{c},\hat{\theta}_{g},\hat{\theta}_{df},\hat{\theta}_{dp},\hat{\theta}_{dc}$
that jointly satisfy:

$(\hat{\theta}_{f},\hat{\theta}_{c},\hat{\theta}_{g})=\underset{\theta_{f},\theta_{c},\theta_{g}}{\text{arg
min
}}J(\theta_{f},\theta_{c},\theta_{df},{\theta}_{g},\theta_{dp},\theta_{dc})$
(12)

$(\hat{\theta}_{df},\hat{\theta}_{dp},\hat{\theta}_{dc})=\underset{\theta_{df},\theta_{dp},\theta_{dc}}{\text{arg
max
}}J(\theta_{f},\theta_{c},\theta_{df},{\theta}_{g},\theta_{dp},\theta_{dc})$
(13)

Table 1: Classification accuracy (%) on Office-31 dataset for unsupervised domain adaptation (AlexNet [18]) Method | A$\rightarrow$W | A$\rightarrow$D | D$\rightarrow$A | W$\rightarrow$A | Avg
---|---|---|---|---|---
AlexNet[18] | 60.6 | 64.2 | 45.5 | 48.3 | 54.65
DANN[8] | 73.0 | 72.3 | 52.4 | 50.4 | 62.03
JAN[26] | 75.2 | 72.8 | 57.5 | 56.3 | 65.45
MADA[30] | 78.5 | 74.1 | 56.0 | 54.5 | 65.78
CDAN[25] | 77.9 | 74.6 | 55.1 | 57.5 | 66.28
Entro[47] | 78.9 | 77.8 | 56.6 | 57.4 | 67.68
CAT[5] | 80.7 | 76.4 | 63.7 | 62.2 | 70.75
CADA[19] | 83.4 | 80.1 | 59.8 | 59.5 | 70.70
TDMDA | 82.6 | 85.2 | 66.3 | 66.3 | 75.10
Table 2: Classification accuracy (%) on Office-31 dataset for unsupervised domain adaptation (ResNet-50 [12]) Method | A$\rightarrow$W | A$\rightarrow$D | D$\rightarrow$A | W$\rightarrow$A | Avg
---|---|---|---|---|---
ResNet[12] | 68.4 | 68.9 | 62.5 | 60.7 | 65.13
DANN[8] | 82.0 | 79.7 | 68.2 | 67.4 | 74.33
MADA[30] | 90.0 | 87.8 | 70.3 | 66.4 | 78.62
DAAA [16] | 86.8 | 88.8 | 74.3 | 73.9 | 80.95
CDAN[25] | 93.1 | 93.4 | 71.0 | 70.3 | 81.95
CAT[5] | 94.4 | 90.8 | 72.2 | 70.2 | 81.90
TAT[23] | 92.5 | 93.2 | 73.1 | 72.1 | 82.73
TADA [46] | 94.2 | 92.8 | 72.6 | 73.6 | 83.30
CADA[19] | 97.0 | 95.6 | 71.5 | 73.1 | 84.30
ETD[21] | 92.1 | 88.0 | 71.0 | 67.8 | 79.73
DMRL[48] | 90.8 | 93.4 | 73.0 | 71.2 | 82.10
ALDA[4] | 95.6 | 94.0 | 72.2 | 72.5 | 83.58
DADA[40] | 92.3 | 93.9 | 74.4 | 74.2 | 83.78
PMDA | 91.9 | 93.8 | 72.5 | 74.4 | 83.12
CMDA | 93.6 | 94.1 | 73.3 | 74.5 | 83.88
TDMDA | 94.8 | 93.5 | 74.7 | 75.8 | 84.70
Table 3: Classification accuracy (%) on ImageCLEF dataset for unsupervised domain adaptation (ResNet-50 [12]) Method | I $\rightarrow$ P | P$\rightarrow$ I | I $\rightarrow$ C | C $\rightarrow$ I | C $\rightarrow$ P | P $\rightarrow$ C | Avg
---|---|---|---|---|---|---|---
ResNet[12] | 74.8 | 83.9 | 91.5 | 78.0 | 65.5 | 91.2 | 80.7
DANN[8] | 75.0 | 86.0 | 96.2 | 87.0 | 74.3 | 91.5 | 85.0
JAN[26] | 76.8 | 88.0 | 94.7 | 89.5 | 74.2 | 91.7 | 85.8
MADA[30] | 75.0 | 87.9 | 96.0 | 88.8 | 75.2 | 92.2 | 85.8
CDAN[25] | 77.2 | 88.3 | 98.3 | 90.7 | 76.7 | 94.0 | 87.5
CAT[5] | 77.2 | 91.0 | 95.5 | 91.3 | 75.3 | 93.6 | 87.3
TAT[23] | 78.8 | 92.0 | 97.5 | 92.0 | 78.2 | 94.7 | 88.9
CADA[19] | 78.0 | 90.5 | 96.7 | 92.0 | 77.2 | 95.5 | 88.3
DMRL[48] | 77.3 | 90.7 | 97.4 | 91.8 | 76.0 | 94.8 | 88.0
AADA[50] | 79.2 | 92.5 | 96.2 | 91.4 | 76.1 | 94.7 | 88.4
TDMDA | 77.8 | 92.0 | 97.2 | 92.7 | 78.7 | 96.2 | 89.1
Table 4: Classification accuracy (%) on Office Home dataset for unsupervised domain adaptation (ResNet-50 [12]) Method | Ar$\rightarrow$Cl | Ar$\rightarrow$Pr | Ar$\rightarrow$Rw | Cl$\rightarrow$Ar | Cl$\rightarrow$Pr | Cl$\rightarrow$Rw | Pr$\rightarrow$Ar | Pr$\rightarrow$Cl | Pr$\rightarrow$Rw | Rw$\rightarrow$Ar | Rw$\rightarrow$Cl | Rw$\rightarrow$Pr | Avg
---|---|---|---|---|---|---|---|---|---|---|---|---|---
ResNet[12] | 34.9 | 50.0 | 58.0 | 37.4 | 41.9 | 46.2 | 38.5 | 31.2 | 60.4 | 53.9 | 41.2 | 59.9 | 46.1
DANN[8] | 45.6 | 59.3 | 70.1 | 47.0 | 58.5 | 60.9 | 46.1 | 43.7 | 68.5 | 63.2 | 51.8 | 76.8 | 57.6
JAN[26] | 45.9 | 61.2 | 68.9 | 50.4 | 59.7 | 61.0 | 45.8 | 43.4 | 70.3 | 63.9 | 52.4 | 76.8 | 58.3
CDAN[25] | 50.6 | 65.9 | 73.4 | 55.7 | 62.7 | 64.2 | 51.8 | 49.1 | 74.5 | 68.2 | 56.9 | 80.7 | 62.8
TAT[23] | 51.6 | 69.5 | 75.4 | 59.4 | 69.5 | 68.6 | 59.5 | 50.5 | 76.8 | 70.9 | 56.6 | 81.6 | 65.8
TADA[46] | 53.1 | 72.3 | 77.2 | 59.0 | 71.2 | 72.1 | 59.7 | 53.1 | 78.4 | 72.4 | 60.0 | 82.9 | 67.6
CADA[19] | 56.9 | 76.4 | 80.7 | 61.3 | 75.2 | 75.2 | 63.2 | 54.5 | 80.7 | 73.9 | 61.5 | 84.1 | 70.2
ALDA[4] | 53.7 | 70.1 | 76.4 | 60.2 | 72.6 | 71.5 | 56.8 | 51.9 | 77.1 | 70.2 | 56.3 | 82.1 | 66.6
AADA[50] | 54.0 | 71.3 | 77.5 | 60.8 | 70.8 | 71.2 | 59.1 | 51.8 | 76.9 | 71.0 | 57.4 | 81.8 | 67.0
ETD[21] | 51.3 | 71.9 | 85.7 | 57.6 | 69.2 | 73.7 | 57.8 | 51.2 | 79.3 | 70.2 | 57.5 | 82.1 | 67.3
DCAN[22] | 54.5 | 75.7 | 81.2 | 67.4 | 74.0 | 76.3 | 67.4 | 52.7 | 80.6 | 74.1 | 59.1 | 83.5 | 70.5
TDMDA | 55.4 | 79.3 | 81.6 | 64.3 | 76.5 | 76.8 | 67.7 | 52.2 | 83.5 | 74.9 | 60.4 | 85.6 | 71.5

##  5 Experiments and Results

###  5.1 Datasets

Office-31 [33] dataset consists of images of office environment of three
domains (Amazon, Webcam, and DSLR). It consists of 4,652 images and 31
classes. We have evaluated our model on four adaptation tasks
(A$\rightarrow$W, A$\rightarrow$D, W$\rightarrow$A, D$\rightarrow$A). We
removed the other two adaptation tasks (D$\rightarrow$W, W$\rightarrow$D)
because they are very similar domains.

Office-Home [44] dataset consists of 65 classes with four domains Artistic
(Ar), Clipart (Cl), Product (Pr) and Real-World (Rw) images. These domains are
very different from each other, and adaptation is very challenging. We have
reported all 12 transfer tasks for this dataset.

ImageCLEF-2014 dataset consists of three datasets: Caltech-256 (C), ILSVRC
2012 (I), and Pascal VOC 2012 (P). There are 12 common classes, and each class
has 50 samples. So, there is a total of 600 images in each domain.

###  5.2 Results

We have reported the results of the Office-31 dataset based on AlexNet in
Table 1. Our method TDMDA has outperformed all of the mentioned methods by a
significant margin of 4.35$\%$. Entro [47] proposed to match the predictive
uncertainty for adapting the classifier in the target domain. However, our
method exceeds Entro by a large margin of 7.42$\%$ on the average accuracy. We
have also reported our model performance for Office-31 dataset based on
ResNet-50 in Table 2 with an improvement of 0.3$\%$. TDMDA outperforms methods
based on classifier adaptation such as TAT[23], CAT[5] and DAAA[16] by margin
greater than 2$\%$. The proposed method even performs better than the recent
approaches [22, 50, 21, 4] and it also incorporates complex attention
mechanisms such as [19, 46]. This shows that the adaptation of classifiers in
the target domain is necessary for effective domain adaptation.

Table 4 shows the results on Office-Home dataset. This dataset is far more
challenging than other datasets as domain shift is significantly high here.
Despite the challenge, our method achieves a substantial improvement over the
other methods across most of the tasks. TDMDA outperforms TAT[23] by a
significant margin of 5.7$\%$ on average accuracy. While CADA[19] outperforms
our method when Clipart (Cl) is the target domain, but the results are still
competitive performing on average better than CADA. Moreover, the proposed
work can be plugged with any domain adaptation framework to improve the
performance further. We have also reported the results on the ImageCLEF
dataset in Table 3. Our method exceeds the rest of the methods’ performance
with a minor improvement of 0.2$\%$ on average accuracy. The relatively small
margin in improvement is because of the smaller domain shift in the dataset.

###  5.3 Ablation Study

Figure 6: Predictive uncertainty in the target domain is significantly lower
using our method than the adversarial feature adaptation method on Office-31
dataset (AlexNet)

(a) Class-wise Feature Representation

(b) Domain-wise Feature Representation

(c) Class-wise Probability Distribution

(d) Domain-wise Probability Distribution

Figure 7: (a) and (b) shows t-SNE visualization of feature representations
learned by TDMDA which are now class-discriminative in the target domain. (c)
and (d) represents t-SNE visualization of probability distribution of TDMDA
without any domain shift for Office31 dataset ( Source: W and  Target: A)
using AlexNet.

We have also studied the contributions of both probability distribution
adaptation and certainty activation mappings adaptation. The comparison
between TDMDA and its variants is reported in Table 2 for the Office-31
dataset. PMDA refers to the probability distribution adaptation, and CMDA
refers to the certainty activation mapping adaptation. Results show that both
PMDA and CMDA outperform most of the comparison methods. PMDA performs
slightly worse than CMDA as it only encourages the classifier to learn class-
discriminative feature representation with higher margins. This can lead to
features aligned to incorrect class-cluster. By adapting both probability
distribution along with certainty activation mapping reduces this problem.
Thus, TDMDA exceeds both of them, which shows that learning both mean and
variance is essential for approximating the classifier’s posterior in the
target domain.

##  6 Empirical Analysis

###  6.1 Predictive Uncertainty

In figure 6, we have shown the predictive uncertainty of source and target
domain after feature adaptation, and target domain after using our method
(TDMDA) on the Office-31 dataset using AlexNet. We can observe that predictive
uncertainty on the target domain using our approach is much less than on
target domain after feature adaptation for all the tasks. This shows that the
posterior of the classifier is now much better approximated for the target
domain.

###  6.2 Feature Visualization

We have visualized the feature representations learned by our method TDMDA and
performing only adversarial feature adaptation using t-SNE embedding [27] in
figure 7(b) and 2(b) for W$\rightarrow$A task using AlexNet respectively. Our
method learns more class-discriminative feature representation and aligns most
of the target domain images to a class cluster compared with features learned
by feature adaptation, as observed in the figure 7(a).

###  6.3 Probability Distribution Visualization

We have visualized source and target probability distributions on
W$\rightarrow$A task on AlexNet using t-SNE embedding [27] in figures 7(c) and
7(d). Most of the target domain’s probability distributions are now aligned to
a class-cluster similar to source domain. Now there is much lesser domain
shift as compared to that shown in figure 2. This provides an evidence that
our method has learned a better approximation of mean $\mu_{c}$ of the
posterior for target domain.

###  6.4 Certainty Activation Mapping

In figure 8, we can see that after using our method, certainty activation
mapping now highlights class-discriminative regions in target domain.
Performing only feature adaptation does not provide class-discriminative
certainty activation mappings, as shown in figure 4. This depicts the
effectiveness of our method in better approximating the posterior of the
classifier in target domain.

Figure 8: Visualization of certainty activation mappings using TDMDA for
A$\rightarrow$D task (AlexNet). Certainty activation mappings are highlighting
class-discriminative regions for target domain image similar to source domain

##  7 Conclusion

In this paper, we have thoroughly analyzed the problem of matching the
classifier responses between the source and target domains for successful
adaptation. We observe that ignoring this aspect results in higher uncertainty
in the classifier and various probability distributions being quite varied
between the source and target domains. As we do not have access to the target
class labels, we propose a way to indirectly ensuring that the classifier
distributions are matched by the triple distribution matching approach. Our
thorough empirical analysis demonstrates that this does indeed result in
improved performance and lower the uncertainty for domain adaptation.
Moreover, as the eventual goal is classification, it is therefore more
pertinent to consider the classifier response in a more comprehensive way as
we show in this work.

## References

  * [1] Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra.  Weight uncertainty in neural network.  In International Conference on Machine Learning, pages 1613–1622, 2015.
  * [2] Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan.  Unsupervised pixel-level domain adaptation with generative adversarial networks.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, page 7, 2017.
  * [3] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille.  Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs.  IEEE transactions on pattern analysis and machine intelligence, 40(4):834–848, 2017.
  * [4] Minghao Chen, Shuai Zhao, Haifeng Liu, and Deng Cai.  Adversarial-learned loss for domain adaptation.  In AAAI, pages 3521–3528, 2020.
  * [5] Zhijie Deng, Yucen Luo, and Jun Zhu.  Cluster alignment with a teacher for unsupervised domain adaptation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 9944–9953, 2019.
  * [6] Yarin Gal.  Uncertainty in Deep Learning.  PhD thesis, University of Cambridge, 2016.
  * [7] Yarin Gal and Zoubin Ghahramani.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In international conference on machine learning, pages 1050–1059, 2016.
  * [8] Yaroslav Ganin and Victor Lempitsky.  Unsupervised domain adaptation by backpropagation.  In International Conference on Machine Learning, pages 1180–1189, 2015.
  * [9] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky.  Domain-adversarial training of neural networks.  The Journal of Machine Learning Research, 17(1):2096–2030, 2016\.
  * [10] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In Advances in neural information processing systems, pages 2672–2680, 2014.
  * [11] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger.  On calibration of modern neural networks.  In International Conference on Machine Learning, pages 1321–1330, 2017.
  * [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [13] Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell.  Cycada: Cycle-consistent adversarial domain adaptation.  In International Conference on Machine Learning, pages 1994–2003, 2018.
  * [14] Lanqing Hu, Meina Kan, Shiguang Shan, and Xilin Chen.  Duplex generative adversarial network for unsupervised domain adaptation.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  * [15] Haoshuo Huang, Qixing Huang, and Philipp Krahenbuhl.  Domain transfer through deep activation matching.  In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
  * [16] Guoliang Kang, Liang Zheng, Yan Yan, and Yi Yang.  Deep adversarial attention alignment for unsupervised domain adaptation: the benefit of target expectation maximization.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 401–416, 2018.
  * [17] Alex Kendall and Yarin Gal.  What uncertainties do we need in bayesian deep learning for computer vision?  In Advances in neural information processing systems, pages 5574–5584, 2017.
  * [18] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  In Advances in neural information processing systems, 2012.
  * [19] Vinod Kumar Kurmi, Shanu Kumar, and Vinay P. Namboodiri.  Attending to discriminative certainty for domain adaptation.  In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  * [20] Seungmin Lee, Dongwan Kim, Namil Kim, and Seong-Gyun Jeong.  Drop to adapt: Learning discriminative features for unsupervised domain adaptation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 91–100, 2019.
  * [21] Mengxue Li, Yi-Ming Zhai, You-Wei Luo, Peng-Fei Ge, and Chuan-Xian Ren.  Enhanced transport distance for unsupervised domain adaptation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13936–13944, 2020.
  * [22] Shuang Li, Chi Harold Liu, Qiuxia Lin, Binhui Xie, Zhengming Ding, Gao Huang, and Jian Tang.  Domain conditioned adaptation network.  In AAAI, pages 11386–11393, 2020.
  * [23] Hong Liu, Mingsheng Long, Jianmin Wang, and Michael Jordan.  Transferable adversarial training: A general approach to adapting deep classifiers.  In International Conference on Machine Learning, pages 4013–4022, 2019.
  * [24] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael Jordan.  Learning transferable features with deep adaptation networks.  In International Conference on Machine Learning, pages 97–105, 2015\.
  * [25] Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan.  Conditional adversarial domain adaptation.  In Advances in Neural Information Processing Systems, pages 1647–1657, 2018.
  * [26] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan.  Deep transfer learning with joint adaptation networks.  In International Conference on Machine Learning, pages 2208–2217, 2017.
  * [27] Laurens van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  Journal of machine learning research, 9(Nov):2579–2605, 2008.
  * [28] Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek.  Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift.  In Advances in Neural Information Processing Systems, pages 13991–14002, 2019.
  * [29] Vishal M Patel, Raghuraman Gopalan, Ruonan Li, and Rama Chellappa.  Visual domain adaptation: A survey of recent advances.  IEEE signal processing magazine, 32(3):53–69, 2015.
  * [30] Zhongyi Pei, Zhangjie Cao, Mingsheng Long, and Jianmin Wang.  Multi-adversarial domain adaptation.  In Thirty-Second AAAI Conference on Artificial Intelligence, 2018\.
  * [31] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In Advances in neural information processing systems, pages 91–99, 2015.
  * [32] Carlos Riquelme, George Tucker, and Jasper Snoek.  Deep bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling.  In International Conference on Learning Representations, 2018.
  * [33] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell.  Adapting visual category models to new domains.  In European conference on computer vision, pages 213–226. Springer, 2010.
  * [34] Swami Sankaranarayanan, Yogesh Balaji, Carlos D Castillo, and Rama Chellappa.  Generate to adapt: Aligning domains using generative adversarial networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 8503–8512, 2018.
  * [35] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra, et al.  Grad-cam: Visual explanations from deep networks via gradient-based localization.  In ICCV, pages 618–626, 2017.
  * [36] Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu.  Adversarial representation learning for domain adaptation.  arXiv preprint arXiv:1707.01217, 2017.
  * [37] Jian Shen, Yanru Qu, Weinan Zhang, and Yong Yu.  Wasserstein distance guided representation learning for domain adaptation.  In AAAI, 2018.
  * [38] Rui Shu, Hung H Bui, Hirokazu Narui, and Stefano Ermon.  A dirt-t approach to unsupervised domain adaptation.  In Proc. 6th International Conference on Learning Representations, 2018.
  * [39] Baochen Sun and Kate Saenko.  Deep coral: Correlation alignment for deep domain adaptation.  In European Conference on Computer Vision, pages 443–450. Springer, 2016.
  * [40] Hui Tang and Kui Jia.  Discriminative adversarial domain adaptation.  In AAAI, pages 5940–5947, 2020.
  * [41] Eric Tzeng, Judy Hoffman, Trevor Darrell, and Kate Saenko.  Simultaneous deep transfer across domains and tasks.  In Computer Vision (ICCV), 2015 IEEE International Conference on, pages 4068–4076. IEEE, 2015.
  * [42] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell.  Adversarial discriminative domain adaptation.  In Computer Vision and Pattern Recognition (CVPR), volume 1, page 4, 2017.
  * [43] Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell.  Deep domain confusion: Maximizing for domain invariance.  arXiv preprint arXiv:1412.3474, 2014.
  * [44] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan.  Deep hashing network for unsupervised domain adaptation.  In Proc. CVPR, pages 5018–5027, 2017.
  * [45] Mei Wang and Weihong Deng.  Deep visual domain adaptation: A survey.  Neurocomputing, 2018.
  * [46] Ximei Wang, Liang Li, Weirui Ye, Mingsheng Long, and Jianmin Wang.  Transferable attention for domain adaptation.  In Thirty-Third AAAI Conference on Artificial Intelligence, 2019\.
  * [47] Jun Wen, Nenggan Zheng, Junsong Yuan, Zhefeng Gong, and Changyou Chen.  Bayesian uncertainty matching for unsupervised domain adaptation.  In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 3849–3855. AAAI Press, 2019.
  * [48] Yuan Wu, Diana Inkpen, and Ahmed El-Roby.  Dual mixup regularized learning for adversarial domain adaptation.  ECCV, 2020.
  * [49] Hongliang Yan, Yukang Ding, Peihua Li, Qilong Wang, Yong Xu, and Wangmeng Zuo.  Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2272–2281, 2017.
  * [50] Jianfei Yang, Han Zou, Yuxun Zhou, Zhaoyang Zeng, and Lihua Xie.  Mind the discriminability: Asymmetric adversarial domain adaptation.  ECCV, 2020.