# Open-Set Multi-Source Multi-Target
Domain Adaptation

Rohit Lal 1, Arihant Gaur 2, Aadhithya Iyer 2, Muhammed Abdullah Shaikh2,
Ritik Agrawal 2, Shital Chiddarwar2
1Indian Institute of Science (IISc), Bangalore
2 Visvesvaraya National Institute of Technology (VNIT), Nagpur

###### Abstract

Single-Source Single-Target Domain Adaptation (1S1T) aims to bridge the gap
between a labelled source domain and an unlabelled target domain. Despite 1S1T
being a well-researched topic, they are typically not deployed to the real
world. Methods like Multi-Source Domain Adaptation and Multi-Target Domain
Adaptation have evolved to model real-world problems but still do not
generalise well. The fact that most of these methods assume a common label-set
between source and target is very restrictive. Recent Open-Set Domain
Adaptation methods handle unknown target labels but fail to generalise in
multiple domains. To overcome these difficulties, first, we propose a novel
generic domain adaptation (DA) setting named Open-Set Multi-Source Multi-
Target Domain Adaptation (OS-nSmT), with $n$ and $m$ being number of source
and target domains respectively. Next, we propose a graph attention based
framework named DEGAA which can capture information from multiple source and
target domains without knowing the exact label-set of the target. We argue
that our method, though offered for multiple sources and multiple targets, can
also be agnostic to various other DA settings. To check the robustness and
versatility of DEGAA, we put forward ample experiments and ablation studies.

##  1 Introduction

Recent advances in deep learning have achieved state-of-the-art (SOTA) results
in tasks like image classification [1], object detection and segmentation [2].
Most of these methods require a lot of labelled data and assumes that all data
comes from identical distributions (i.i.d.) that is, training data
distribution is the same as test data distribution. This assumption rarely
holds for real-world cases. In Unsupervised Domain Adaptation (UDA), we have
labelled source data and unlabelled target data. The source and target data
here belong to different distributions. The goal of UDA is to overcome this
distributional shift and perform well on both labelled source and unlabelled
target data.

Even though closed-set Single-Source Single-Target (1S1T), Multi-Source Single
Target (nS1T), Single Source Multi-Target (1SmT) and Multi-Source Multi-Target
(nSmT) settings show promising results, these methods share a common label set
between the source and target domains (closed-set labels). This assumption may
not hold for models deployed in the wild as the model may encounter unknown
classes as well. The aforementioned restrictions motivated us to look for a
new setting that can model real-world problems more closely and is scalable to
a vast range of problems. In this paper, we propose a novel Domain Adaptation
(DA) setting named Open-Set Multi-Source Multi-Target Unsupervised Domain
Adaptation (OS-nSmT) that closely resembles the real-world scenarios. In OS-
nSmT, we have access to multiple labelled source domain dataset with the same
label set and multiple unlabelled target domain datasets. Here, we consider
the source label set as a subset of the target label set. This is a very
difficult problem to solve as one has to overcome both the category and domain
gap. Graphs Neural Networks (GNN) based approaches [3, 4, 5, 6] have gained
traction in this field due to their ability to find relations in unstructured
data. It also helps to represent the domains in a unified subspace. We use GNN
with attentional aggregation to find optimal relationships between the samples
and perform semantic transfer accordingly. We propose an algorithm named
Domain Embedding based Graph Attentional Aggregation (DEGAA) to solve this
important yet unexplored setting. This algorithm finds potential application
in autonomous systems, where rather than adapting the model for one country at
a time, we can incorporate data for multiple countries. This work can be
potentially extended in the field of domain generalization in which we have
multiple unseen target domains. Other applications might include face
recognition, cross-lingual learning in Natural Language Processing, sentiment
classification, etc. To summarise, the work proposes the following
contributions:

  * •

Introduction of a novel DA setting (OS-nSmT) which is close to modelling real
world applications.

  * •

Development of DEGAA: A Graph Attentional Aggregation based algorithm that can
work with all kind of DA setting (1S1T, 1SmT, nSmT, etc) with open-set labels.

##  2 Related works

(a) Closed set domain adaptation (CSDA). CSDA is a setting with an assumption
of shared support between source and target domains. A common method is to
minimize the domain discrepancy using the Moment Matching technique which
utilizes various metrics like MMD [7], CMD [8] and other variations [9, 10,
11, 12, 13, 14, 15]. Alternatively, to reduce the domain shift, adversarial
networks have been used to maximize the confusion between source and target
domains [16, 17, 18, 19, 20]. With the advancement in image synthesis using
GAN [21], various generative model-based methods [22, 23, 24, 25, 26, 27, 28]
have been proposed. Though researchers have achieved good accuracy in 1S1T, it
can’t leverage the huge amount of data that is readily available across
multiple different domains. Recent progress in DA settings like nS1T [29, 30,
31, 32] and 1SmT [33, 34, 35, 36] tries to take advantage of multiple
sources/targets. nSmT [37, 38] largely remains unexplored due to its increased
complexity. While few methods [38] in nSmT handle multiple DA settings, they
still can’t handle unseen target labels.

(b) Open set domain adaptation (OSDA). In OSDA setting there aren’t any
assumptions that source and target should have common labels. Prior work [39,
40, 41, 42] trains feature generator such that the private class samples from
target domain are assigned as "unknown", [43] uses coarse-to-fine refining of
samples, while other methods [44, 45] harness the capability of GAN for
generating more "unknown" samples for training. A survey on OSDA [46] shows
that prior work in open set setting is mostly for 1S1T scenario. Rakshit et
al.[47] introduced the paradigm of nS1T in Open Set scenario and uses
adverserial training strategy, while other cases are yet to be explored. Our
work proposes a framework for a new setting i.e OS-nSmT which can further be
utilized for other scenarios as well.

(c) Graph neural networks (GNN). GNN are essentially neural network models,
directly applied to data structured as a graph, capturing relationships
between the nodes through the propagation of messages from nearby nodes [48,
49]. Recent methods [4, 5, 6] involving GNNs have gained popularity owing to
their transductive ability for semantic propagation of related samples among
multiple domains. Roy et al. [3] proposes using graph convolution networks in
1SmT setting, along with co-curriculum teaching to handle noisy pseudo-labels.
In our novel setting, we propose the use of attentional graph neural network
[50] by passing messages between the source and target domains to aggregate
them in a unified space.

(d) Domain generalization. Domain generalization involves training on multiple
observed domains, while expecting it to work well on unseen domains. In
contrast to domain adaptation, there is no bridge network to distill the
knowledge from source to target domain. We leverage the use of domain
embeddings [51], that were originally used for domain generalization, by
taking both the input and domain into account to generate pseudo - labels for
target domain. A recent method [52] used entropy minimization technique for
modulating features to learn domain-invariant representations. Other methods
include using adversarial training [53, 54, 55] inspired from [17], meta
learning [56], domain mixup [57, 58, 59], distributionally robust optimization
[60], causal matching [61] and invariant risk minimization [62].

##  3 Methodology

##### Notations

We define $F(\mathbf{x};\theta):\mathbb{R}^{3\times w\times
h}\to\mathbb{R}^{d}$ as the backbone network, parameterized by weights
$\theta$ and input $\mathbf{x}$. The domain embedding network is represented
as $G(\mathbf{x};\psi):\mathbb{R}^{3\times w\times h}\to\mathbb{R}^{d}$, that
yields the vector corresponding to each domain. Being an nSmT problem, there
are $n$ source domains ($S=\cup_{i=1}^{n}S_{i}$) and $m$ target domains
($T=\cup_{j=1}^{m}T_{j}$). They are collectively defined in the domain set
$D=S\cup T$.

In the following section, we will discuss the full architecture in detail
(Fig. 1). For algorithm, please refer to the supplementary material.

###  3.1 Domain embedding

We follow an approach similar to [51] for obtaining domain embeddings. We
propose to train the domain embedding network in a prototypical fashion [63],
over a subset of points from each domain, divided into support and query
points. They are sampled from the union of source and target domains. The
support points yield an approximate embedding for each domain. Using this
embedding, the probability of occurrence of query points in that domain is
estimated, whose negative log-likelihood is minimized in an unsupervised
fashion. In other words, the query is used to minimize the distance from the
embeddings obtained using support points. During its inference, the Kernel
Mean Embedding (KME) $\boldsymbol{\mu}$ [64] for domain is determined as,

$\boldsymbol{\mu}(D_{i})=\frac{1}{N(D_{i})}\sum_{\mathbf{x}\in
D_{i}}G(\mathbf{x};\psi)$ (1)

Where $N(D_{i})$ represents number of points in domain $D_{i}$. We store the
KME of all the domains as a dictionary, to obtain a domain embedding
prototype, $d_{e}$. We propose that the domain embedding acts as additional
information. This enables effective domain adaptation to unseen domains as it
is possible to learn domain embedding without having access to labels. The new
feature vector generated by concatenating image representation and domain
embedding, can be used to project source and target image to a common feature
space.

###  3.2 Stage 1: Warm-Up Stage

We propose to train the feature extractor, in a supervised fashion, with the
labels from source domains. The extracted feature, concatenated with the
domain embedding from Section 3.1 is used to calculate the corresponding
centroid for each class in its space (Stage 1 of Fig. 1). This is used to
generate pseudo-labels in the next stage.

Figure 1: DEGAA Architecture. In stage 1, we do supervised source training by
concatenating image features with domain embeddings. This model is initially
used in stage 2 for generating pseudo-labels. In stage 3, we utilise the
pseudo-labels generated from stage 2 for domain adaptation. For the rest of
the epochs, we toggle between stage 2 and stage 3 for iterative training.

###  3.3 Stage 2: Pseudo-Labelling stage

For the training of the target domain, pseudo-labels are generated. The inputs
to the backbone are the target images, which yield a set of feature maps. To
isolate the unknown classes, we use Local Outlier Factor (LOF) [65]. The LOF
tries to determine the deviation of the data points from local densities. If a
point is not within the reachability space in any of the points, it will be
considered an outlier. We propose an ablation in Section 4 to determine the
optimum feature map dimension for outlier detection and removal.

The outliers are assigned "unknown" class. The known classes representations
are then assigned a class by querying the nearest class centroid, obtained
from Section 3.2. Since the Open-Set label detection runs iteratively, we
eliminate the need of highly complex outlier detection algorithms. As the
representations become more discriminative, the separating ability of open-set
classes increases. Such approaches have empirically shown promising results in
"learning with noisy label" literature [66]. To improve the accuracy of
pseudo-labels, we propose re-computing them after every $K$ iterations of
adaptation stage.

###  3.4 Stage 3: Adaptation Stage

We propose the use of graph neural network with attentional aggregation. The
GNN is a fully connected undirected graph, with nodes representing image and
its domain features. There are two types of edges present in the graph. (a)
Intra-edges $E_{self}$: connecting one source domain to other source domains
and one target to another, (b) Inter-edges $E_{cross}$: connecting source and
target domains together. Consider ${}^{(l)}\mathbf{x}^{\mathcal{U}}_{i}$,
which represents image $i$ present in domain $\mathcal{U}\in\\{S,T\\}$ at an
intermediate layer $l$. The message passing step can be represented as,

${}^{(l+1)}\mathbf{x}_{i}^{\mathcal{U}}={}^{(l)}\mathbf{x}_{i}^{\mathcal{U}}+\text{MLP}([^{(l)}\mathbf{x}_{i}^{\mathcal{U}}|\mathbf{m}_{E\to
i}])\\\ $ (2)

Where $\mathbf{m}_{E\to i},E\in\\{E_{self},E_{cross}\\}$ is a message pass,
showing the aggregation of information from first order neighbours of the
node, and $[.|.]$ denotes a concatenation operation. Following [50], we define
a fixed number of layers $L$ (starting from $l=1$), with $E=E_{self}$ if the
layer is odd and $E=E_{cross}$ if the layer is even.

For attentional aggregation, self edges are used for self attention and cross
edges for cross attention [67]. Given the node
${}^{(l)}\mathbf{x}^{\mathcal{U}}_{i}$, the key $\mathbf{k}_{j}$, value
$\mathbf{v}_{j}$ and query $\mathbf{q}_{i}$ can be obtained as,

$\displaystyle\mathbf{q}_{i}=\mathbf{W}^{(l)}_{1}\mathbf{x}^{\mathcal{U}}_{i}+\mathbf{b}_{1}^{(l)}$
(3) $\displaystyle\begin{bmatrix}\mathbf{k}_{j}\\\
\mathbf{v}_{j}\end{bmatrix}=\begin{bmatrix}\mathbf{W}_{2}\\\
\mathbf{W}_{3}\end{bmatrix}^{(l)}\mathbf{x}^{\mathcal{U}}_{j}+\begin{bmatrix}\mathbf{b}_{2}\\\
\mathbf{b}_{3}\end{bmatrix}^{(l)}$ (4)

Where, $\mathbf{W}_{t}^{(l)},\mathbf{b}_{t}^{(l)};t\in\\{1,2,3\\}$ represent
weight matrix and bias at layer $l$ to determine query, key and value
respectively. This can be used to formulate the message as,

$\displaystyle\mathbf{m}_{E\to i}=\sum_{j:(i,j)\in
E}\alpha_{ij}\mathbf{v}_{j}$ (5)
$\displaystyle\alpha_{ij}=\text{Softmax}_{j}(\mathbf{q}_{i}^{T}\mathbf{k}_{j})$
(6)

To improve the model expressivity, multi - head attention [67] can be used. In
such case, the message passing expression can be shown as,

$\mathbf{m}_{E\to i}=\mathbf{W}^{(l)}(\mathbf{m}^{1}_{E\to
i}|\mathbf{m}^{2}_{E\to i}|...|\mathbf{m}^{h}_{E\to i})$ (7)

Where $h$ denotes the number of attention heads and $\mathbf{W}^{(l)}$ denotes
the linear transformation at layer $l$. Such a model is proposed to improve
the data association between the source and target domain and at the same
time, only match the essential characteristics between multiple domains.

##### Loss function

We intend to train the backbone for domain embedding and feature extractor in
an offline fashion. The adaptation stage shall be trained online and used for
inference. The source domain shall be trained using the available labels from
the dataset, whereas the target domain will utilize the pseudo-labels obtained
from Section 3.3. The two components (source and target class loss) are used
as the loss for online training and can be represented as,

$\mathcal{L}=\mathcal{L}^{S}_{CE}+\lambda\mathcal{L}^{T}_{CE}$ (8)

Where $\mathcal{L}^{\mathcal{U}}_{CE}$ is the cross-entropy loss for source
and target heads, and $\lambda$ denotes the weightage of target loss.

##  4 Experimental protocol

###  4.1 Datasets

We will be testing our performance on popular domain adaptation benchmarks
like Office31 [68], Office-Home [69], VisDA-2017 [70] and DomainNet [71]. For
more details on the dataset composition, refer the supplementary material.

###  4.2 Model architectures

For Stage 1, we propose offline training of domain embeddings as well as the
backbone parameters. We intend to vary the backbone based on the datasets used
for a fair comparison with other methods. For Office31 and Office-Home, we
plan to use ResNet50 [72]. For large-scale datasets such as VisDA and
DomainNet, we will use ResNet101 as the backbone. The networks will be pre-
trained on ImageNet, but will not be frozen. While the exact dimension of the
feature vector will be decided based on ablation study in Section 4.4, we
propose a dimension of $d=1024$ for both the domain embedding and the
bottleneck of feature extractor. The graph neural network with attentional
aggregation will require nodes equal to the number of domains $m+n$, where $n$
and $m$ are the number of source and target domains, respectively. We intend
to use $L=9$ layers in the initial experimentation setup, with $h=4$ heads in
multi-head attention. For iterative refinement of target representation, we
propose $K=10,000$ for small scale datasets (Office31 and Office-Home) and
$K=50,000$ for large scale datasets (VisDA and DomainNet). We will also be
doing a random hyperparameter search to find optimal values of
hyperparameters.

###  4.3 Experimental setting

The proposed method does not have any underlying assumptions regarding Domain
Adaptation settings. Hence, the same model should be able to tackle problems
such as 1S1T [16], 1SmT [33], nS1T [73] and nSmT [37]. We would be testing the
same model under all the mentioned settings and compare it with their
respective SOTA results as a baseline. All the mentioned experiments will be
done for open set. The accuracy would be documented for each dataset, for
different combinations of source and target domains. We would also analyse the
behaviour of the proposed framework with the ratio of unknown class. All the
experiments would be performed with 3 different seed values. The final results
would be reported by taking average of all such accuracies. For information
regarding performances of previous methods in these settings, refer the
supplementary material.

We plan to implement the model using PyTorch, trained using Stochastic
Gradient Descent (SGD) optimizer and having an initial learning rate
$10^{-3}-10^{-2}$ with cosine learning rate schedule.

###  4.4 Ablation study

####  4.4.1 Testing the effect of domain embedding

While most of the recent approaches to domain generalization focused on
learning domain invariant features, in the proposed paper, we are
investigating a domain adaptive approach [51]. To test the effect of domain
embedding on the model’s performance, we will train two similar models, one
with domain embedding attached to the feature vector and one without it. Then
we will compare their performance to quantify the effect of domain embedding
on model accuracy. Further, we will be testing different functions to add the
domain embedding to the feature vector, namely, concatenation and element-wise
multiplication.

####  4.4.2 Attentional aggregation vs affinity matrix

We used Attentional Aggregation to obtain a local representation of a node
based on its first-order neighbours in the previous layer. But we can also use
an Affinity matrix [3, 6] to represent the semantic similarity [74] between
the images. We will be comparing the effect of both representations on the
overall accuracy of the model.

####  4.4.3 Testing outlier prediction

The proposed method uses centroid space to define pseudo-labels for objects.
We are using LOF [66] to extract outliers which will be used to assign the
unknown class. We intend to check how well LOF handles outliers with varying
input dimension, for this test we plan to change the nodes in the output layer
of the network before LOF, effectively changing the dimensions of input to LOF
and measure its effect on the overall accuracy, finding out the optimal
dimension. We will also be tracking the improvement of pseudo-label accuracy
per completion of an iteration. We predict that while the initial pseudo-label
accuracy would be low, it should continually increase as training progresses.
Hence, pseudo-label vs iteration curve should be non-decreasing.

##  5 Limitations and conclusion

We address a potential issue that can arise in our approach. This involves the
use of LoF, requiring outliers to isolate them from the main data. Our model
assumes sufficient number of unknown classes to be classified as an outlier.
This might restrict the number of unknown classes our model can handle.

In this work, we propose a novel domain adaptation setting ’Open-Set Multi-
Source Multi-Target’ and a plausible solution for the same. Specifically, we
leverage graph neural networks for message passing amongst different domains,
along with domain embedding for providing higher level information to each
sample. An extension to the proposed approach is the generalization ability on
unseen domains which can be explored in future works.

## References

  * [1] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton.  Imagenet classification with deep convolutional neural networks.  Advances in neural information processing systems, 25:1097–1105, 2012.
  * [2] Ross B. Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  CoRR, abs/1311.2524, 2013.
  * [3] Subhankar Roy, Evgeny Krivosheev, Zhun Zhong, Nicu Sebe, and Elisa Ricci.  Curriculum graph co-teaching for multi-target domain adaptation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5351–5360, 2021.
  * [4] Xu Yang, Cheng Deng, Tongliang Liu, and Dacheng Tao.  Heterogeneous graph attention network for unsupervised multiple-target domain adaptation.  IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1–1, 2020.
  * [5] Xinhong Ma, Tianzhu Zhang, and Changsheng Xu.  Gcan: Graph convolutional adversarial network for unsupervised domain adaptation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8266–8276, 2019.
  * [6] Yadan Luo, Zijian Wang, Zi Huang, and Mahsa Baktashmotlagh.  Progressive graph learning for open-set domain adaptation.  In International Conference on Machine Learning, pages 6468–6478. PMLR, 2020.
  * [7] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan.  Learning transferable features with deep adaptation networks.  ArXiv, abs/1502.02791, 2015.
  * [8] W. Zellinger, Thomas Grubinger, E. Lughofer, T. Natschläger, and Susanne Saminger-Platz.  Central moment discrepancy (cmd) for domain-invariant representation learning.  ArXiv, abs/1702.08811, 2017.
  * [9] Basura Fernando, Amaury Habrard, Marc Sebban, and Tinne Tuytelaars.  Unsupervised visual domain adaptation using subspace alignment.  In 2013 IEEE International Conference on Computer Vision, pages 2960–2967, 2013.
  * [10] Lixin Duan, Ivor W. Tsang, and Dong Xu.  Domain transfer multiple kernel learning.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3):465–479, 2012.
  * [11] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I Jordan.  Unsupervised domain adaptation with residual transfer networks.  arXiv preprint arXiv:1602.04433, 2016.
  * [12] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan.  Deep transfer learning with joint adaptation networks.  In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 2208–2217. JMLR.org, 2017.
  * [13] Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada.  Maximum classifier discrepancy for unsupervised domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  * [14] Hongliang Yan, Yukang Ding, P. Li, Qilong Wang, Yong Xu, and W. Zuo.  Mind the class weight bias: Weighted maximum mean discrepancy for unsupervised domain adaptation.  2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 945–954, 2017.
  * [15] Jaemin Na, Heechul Jung, Hyung Jin Chang, and Wonjun Hwang.  Fixbi: Bridging domain spaces for unsupervised domain adaptation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1094–1103, June 2021.
  * [16] Yaroslav Ganin and Victor Lempitsky.  Unsupervised domain adaptation by backpropagation.  In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pages 1180–1189, Lille, France, 07–09 Jul 2015\. PMLR.
  * [17] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario March, and Victor Lempitsky.  Domain-adversarial training of neural networks.  Journal of Machine Learning Research, 17(59):1–35, 2016.
  * [18] Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell.  Adversarial discriminative domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  * [19] Guoliang Kang, Liang Zheng, Yan Yan, and Yi Yang.  Deep adversarial attention alignment for unsupervised domain adaptation: the benefit of target expectation maximization.  In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
  * [20] Mingsheng Long, ZHANGJIE CAO, Jianmin Wang, and Michael I Jordan.  Conditional adversarial domain adaptation.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
  * [21] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  Advances in neural information processing systems, 27, 2014.
  * [22] Judy Hoffman, Eric Tzeng, Taesung Park, Jun-Yan Zhu, Phillip Isola, Kate Saenko, Alexei Efros, and Trevor Darrell.  CyCADA: Cycle-consistent adversarial domain adaptation.  In Jennifer Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 1989–1998. PMLR, 10–15 Jul 2018\.
  * [23] Konstantinos Bousmalis, Nathan Silberman, David Dohan, Dumitru Erhan, and Dilip Krishnan.  Unsupervised pixel-level domain adaptation with generative adversarial networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), July 2017.
  * [24] Lanqing Hu, Meina Kan, Shiguang Shan, and Xilin Chen.  Duplex generative adversarial network for unsupervised domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  * [25] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  2017 IEEE International Conference on Computer Vision (ICCV), pages 2242–2251, 2017.
  * [26] Swami Sankaranarayanan, Yogesh Balaji, Carlos D. Castillo, and Rama Chellappa.  Generate to adapt: Aligning domains using generative adversarial networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018.
  * [27] Riccardo Volpi, Pietro Morerio, S. Savarese, and Vittorio Murino.  Adversarial feature augmentation for unsupervised domain adaptation.  2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5495–5504, 2018.
  * [28] Guanglei Yang, Haifeng Xia, M. Ding, and Zhengming Ding.  Bi-directional generation for unsupervised domain adaptation.  In AAAI, 2020.
  * [29] Xia Cui and Danushka Bollegala.  Multi-source attention for unsupervised domain adaptation.  In AACL, 2020.
  * [30] Han Zhao, Shanghang Zhang, Guanhang Wu, José M. F. Moura, Joao P Costeira, and Geoffrey J Gordon.  Adversarial multiple source domain adaptation.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
  * [31] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang.  Moment matching for multi-source domain adaptation.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1406–1415, 2019.
  * [32] Ruijia Xu, Ziliang Chen, Wangmeng Zuo, Junjie Yan, and Liang Lin.  Deep cocktail network: Multi-source unsupervised domain adaptation with category shift.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3964–3973, 2018.
  * [33] Takashi Isobe, Xu Jia, Shuaijun Chen, Jianzhong He, Yongjie Shi, Jianzhuang Liu, Huchuan Lu, and Shengjin Wang.  Multi-target domain adaptation with collaborative consistency learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8187–8196, June 2021.
  * [34] Xingchao Peng, Zijun Huang, Ximeng Sun, and Kate Saenko.  Domain agnostic learning with disentangled representations.  In International Conference on Machine Learning, pages 5102–5112. PMLR, 2019.
  * [35] Ziliang Chen, Jingyu Zhuang, Xiaodan Liang, and Liang Lin.  Blending-target domain adaptation by adversarial meta-adaptation networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  * [36] Le Thanh Nguyen-Meidine, Atif Belal, Madhu Kiran, Jose Dolz, Louis-Antoine Blais-Morin, and Eric Granger.  Unsupervised multi-target domain adaptation through knowledge distillation.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1339–1347, 2021.
  * [37] Yuxi Wang, Zhaoxiang Zhang, Wangli Hao, and Chunfeng Song.  Attention guided multiple source and target domain adaptation.  IEEE Transactions on Image Processing, 30:892–906, 2021.
  * [38] Ying Jin, Ximei Wang, Mingsheng Long, and Jianmin Wang.  Minimum class confusion for versatile domain adaptation.  In European Conference on Computer Vision, pages 464–480. Springer, 2020.
  * [39] Pau Panareda Busto and Juergen Gall.  Open set domain adaptation.  In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017.
  * [40] Kuniaki Saito, Shohei Yamamoto, Yoshitaka Ushiku, and Tatsuya Harada.  Open set domain adaptation by backpropagation.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 153–168, 2018.
  * [41] Abhijit Bendale and Terrance E. Boult.  Towards open set deep networks.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1563–1572, 2016.
  * [42] Renjun Xu, Pelen Liu, Yin Zhang, Fang Cai, Jindong Wang, Shuoying Liang, Heting Ying, and Jianwei Yin.  Joint partial optimal transport for open set domain adaptation.  In IJCAI, pages 2540–2546, 2020.
  * [43] Hong Liu, Zhangjie Cao, Mingsheng Long, Jianmin Wang, and Qiang Yang.  Separate to adapt: Open set domain adaptation via progressive separation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  * [44] ZongYuan Ge, Sergey Demyanov, Zetao Chen, and Rahil Garnavi.  Generative openmax for multi-class open set classification.  arXiv preprint arXiv:1707.07418, 2017.
  * [45] Inhyuk Jo, Jungtaek Kim, Hyohyeong Kang, Yong-Deok Kim, and Seungjin Choi.  Open set recognition by regularising classifier with fake data generated by generative adversarial networks.  In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2686–2690, 2018.
  * [46] Chuanxing Geng, Sheng-jun Huang, and Songcan Chen.  Recent advances in open set recognition: A survey.  IEEE transactions on pattern analysis and machine intelligence, 2020\.
  * [47] S. Rakshit, Dipesh Tamboli, Pragati Shuddhodhan Meshram, Biplab Banerjee, G. Roig, and S. Chaudhuri.  Multi-source open-set deep adversarial domain adaptation.  In ECCV, 2020.
  * [48] Marco Gori, Gabriele Monfardini, and Franco Scarselli.  A new model for learning in graph domains.  In Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., volume 2, pages 729–734. IEEE, 2005.
  * [49] Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and S Yu Philip.  A comprehensive survey on graph neural networks.  IEEE transactions on neural networks and learning systems, 32(1):4–24, 2020.
  * [50] Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich.  Superglue: Learning feature matching with graph neural networks.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4938–4947, 2020.
  * [51] Abhimanyu Dubey, Vignesh Ramanathan, Alex Pentland, and Dhruv Mahajan.  Adaptive methods for real-world domain generalization.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14340–14349, 2021.
  * [52] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell.  Tent: Fully test-time adaptation by entropy minimization.  In International Conference on Learning Representations, 2021.
  * [53] Baochen Sun and Kate Saenko.  Deep coral: Correlation alignment for deep domain adaptation.  In European conference on computer vision, pages 443–450. Springer, 2016.
  * [54] Ya Li, Mingming Gong, Xinmei Tian, Tongliang Liu, and Dacheng Tao.  Domain generalization via conditional invariant representations.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
  * [55] Ya Li, Xinmei Tian, Mingming Gong, Yajing Liu, Tongliang Liu, Kun Zhang, and Dacheng Tao.  Deep domain generalization via conditional invariant adversarial networks.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 624–639, 2018.
  * [56] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M Hospedales.  Learning to generalize: Meta-learning for domain generalization.  In Thirty-Second AAAI Conference on Artificial Intelligence, 2018\.
  * [57] Minghao Xu, Jian Zhang, Bingbing Ni, Teng Li, Chengjie Wang, Qi Tian, and Wenjun Zhang.  Adversarial domain adaptation with domain mixup.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6502–6509, 2020.
  * [58] Shen Yan, Huan Song, Nanxiang Li, Lincan Zou, and Liu Ren.  Improve unsupervised domain adaptation with mixup training.  arXiv preprint arXiv:2001.00677, 2020.
  * [59] Yufei Wang, Haoliang Li, and Alex C Kot.  Heterogeneous domain generalization via domain mixup.  In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3622–3626. IEEE, 2020.
  * [60] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang.  Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.  arXiv preprint arXiv:1911.08731, 2019.
  * [61] Divyat Mahajan, Shruti Tople, and Amit Sharma.  Domain generalization using causal matching.  In International Conference on Machine Learning, pages 7313–7324. PMLR, 2021.
  * [62] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz.  Invariant risk minimization.  arXiv preprint arXiv:1907.02893, 2019.
  * [63] Jake Snell, Kevin Swersky, and Richard S Zemel.  Prototypical networks for few-shot learning.  arXiv preprint arXiv:1703.05175, 2017.
  * [64] Krikamol Muandet, Kenji Fukumizu, Bharath Sriperumbudur, and Bernhard Schölkopf.  Kernel mean embedding of distributions: A review and beyond.  arXiv preprint arXiv:1605.09522, 2016.
  * [65] Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander.  Lof: identifying density-based local outliers.  In Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pages 93–104, 2000.
  * [66] Yisen Wang, Weiyang Liu, Xingjun Ma, James Bailey, Hongyuan Zha, Le Song, and Shu-Tao Xia.  Iterative learning with open-set noisy labels.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8688–8696, 2018.
  * [67] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  In Advances in neural information processing systems, pages 5998–6008, 2017.
  * [68] Piotr Koniusz, Yusuf Tas, and Fatih Porikli.  Domain adaptation by mixture of alignments of second- or higher-order scatter tensors.  CoRR, abs/1611.08195, 2016.
  * [69] Mohammad Mahfujur Rahman, Clinton Fookes, Mahsa Baktashmotlagh, and Sridha Sridharan.  Multi-component image translation for deep domain generalization.  CoRR, abs/1812.08974, 2018.
  * [70] Shuhao Cui, Shuhui Wang, Junbao Zhuo, Chi Su, Qingming Huang, and Qi Tian.  Gradually vanishing bridge for adversarial domain adaptation.  CoRR, abs/2003.13183, 2020.
  * [71] Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang.  What is being transferred in transfer learning?  CoRR, abs/2008.11687, 2020.
  * [72] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [73] Sicheng Zhao, Bo Li, Colorado Reed, Pengfei Xu, and Kurt Keutzer.  Multi-source domain adaptation in the deep learning era: A systematic survey.  CoRR, abs/2002.12169, 2020.
  * [74] Thomas N Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  arXiv preprint arXiv:1609.02907, 2016.