# Positive-Unlabeled Domain Adaptation

Jonas Sonntag1,2111Contact Author Gunnar Behrens2&Lars Schmidt-Thieme1
1University of Hildesheim
2Volkwagen Financial Services
<EMAIL_ADDRESS>
###### Abstract

Domain Adaptation methodologies have shown to effectively generalize from a
labeled source domain to a label scarce target domain. Previous research has
either focused on unlabeled domain adaptation without any target supervision
or semi-supervised domain adaptation with few labeled target examples per
class. On the other hand Positive-Unlabeled (PU-) Learning has attracted
increasing interest in the weakly supervised learning literature since in
quite some real world applications positive labels are much easier to obtain
than negative ones. In this work we are the first to introduce the challenge
of Positive-Unlabeled Domain Adaptation where we aim to generalise from a
fully labeled source domain to a target domain where only positive and
unlabeled data is available. We present a novel two-step learning approach to
this problem by firstly identifying reliable positive and negative pseudo-
labels in the target domain guided by source domain labels and a positive-
unlabeled risk estimator. This enables us to use a standard classifier on the
target domain in a second step. We validate our approach by running
experiments on benchmark datasets for visual object recognition. Furthermore
we propose real world examples for our setting and validate our superior
performance on parking occupancy data.

##  1 Introduction

One of the biggest challenge in applied machine learning is the absence of
labeled data in real world applications while most deep learning methods rely
on supervised training often requiring several thousands of labeled training
data. Recently this problem gets more and more acknowledged from the research
community as the direction of few shot learning Wang2020  is gaining
attention.
In this work we will unite two different approaches on few shot learning in
one single framework, namely domain adaptation and positive unlabeled learning
and we are the first to introduce the problem type and a methodology for the
combination of both. The idea of domain adaptation is to transfer knowledge
from a label rich source domain to a label scarce target domain. A rich line
of research focuses on unlabeled domain adaptation (UDA)
wang2021discriminative , where the target domain contains no labels at all
while some methods consider semi-supervised domain adaptation (SSDA) where few
labeled examples per class are available in the target domain. The biggest
challenge for all kinds of domain adaptation is to cope with the shift of data
distribution between different domains. Most approaches tackle this challenge
by learning a domain invariant latent feature representation.
Another scenario in few shot learning is Positive-Unlabeled (PU) learning, as
a special case of semi-supervised learning. In this binary classification
scenario the target domain contains a small number of positive labeled samples
and a large amount of unlabeled data containing both positive and negative
samples. PU learning arises naturally in many applications where positive
labels are much easier to obtain than negative labels. Examples include
medical MRI image classification where not yet diagnosed patients can not be
treated as negative Chen2020 .
Both PU learning and domain adaptation have been studied intensively, however
to the best of our knowledge there is no research yet to the combination of
the two, i.e. predicting in a target domain with only few positive labels
given a related, fully labeled source domain. In this work we are the first to
present a learning approach for this problem and validate it’s performance on
well known benchmark datasets. Moreover we introduce real world problem
settings where currently PU learning is used but fully labeled related domains
are available and show the superiority of our methodology for parking
availability prediction. To summarize our contributions are the following

  * •

We are the first to introduce the problem setting of Positive-Unlabeled Domain
Adaptation.

  * •

We investigate how existing state of the art models from domain adaptation and
PU learning perform in this new setting for both real world and benchmark
datasets.

  * •

We propose a two-step learning methodology designed for this new problem type
and consistently outperform the baselines given only a few positive labels in
the target domain.

##  2 Related Work

Few shot learning is a type of machine learning problem where the algorithm is
trained to perform a target task with only limited supervised training data
Wang2020 . Semi-supervised learning tackles the challenge of limited labels by
leveraging information from unlabeled examples. Positive-Unlabeled learning is
a special case of semi-supervised learning where only positive labels are
given while the unlabeled dataset contains positive as well as negative
observations Bekker2020 .
Domain Adaptation on the other hand uses training data from an other related
domain where more data is available to transfer knowledge to the label scarce
domain.
There is no research yet on the combination of the two areas. There is related
work dealing with covariate shift in PU-learning Sakai2019 ; Hammoudeh2020 .
These methods however deal with a pure positive-unlabeled setting where the
distribution of positive data shifts between training and testing. They do not
consider a fully labeled related source domain and are therefore fundamentally
different from our approach. The authors of Loghmani2020  use PU-learning
methodologies to find previously unseen classes in an unlabeled target domain.
Although this is an application of PU learning to domain adaptation they do
not consider a target domain with only positive labels.
The idea of creating pseudo-labels is frequently used in semi-supervised
learning rizve2021 ; Arazo2020  and is therefore not new in itself. These
methods construct the pseudo labels solely based on unlabeled data in the same
domain and therefore also miss the domain transfer that our methodology is
leveraging.

###  2.1 Positive-Unlabeled Learning

In many applications positive labels are easier to obtain, whereas
representative negative labels are prohibitively expensive or difficult to
sample. In the automated diagnosis of diabetes patients one has access to
diagnosed patients while non-diagnosed patients still have a high likelihood
of being undiagnosed diabetes positive Claesen2015 . Other areas of interest
include land-cover-classification Li2011 , knowledge base completion
Neelakantan2015  or outlier detection Blanchard2010 .
A common strategy for learning with PU data is to use special unbiased risk
estimators Plessis2015 ; Kiryo2017  that exploit the fact that the unlabeled
data $U$ is a weighted mixture of positive $P$ and negative $N$ data. Another
strategy is the so called two-step approach Bekker2020  where in a first step
reliable negative examples are selected and in a second step (semi-)
supervised learning methods are applied. Recently Chen2020  proposed a self
learning PU algorithm that combines both ideas by first selecting confident
positive and negative examples based on PU-risk and in a latter stage performs
self-training with student and teacher models.
A third approach to tackle PU-problems are Generative Adversarial Networks
(GANs) used to generate synthetic positive and negative examples Hou2018 .

###  2.2 Domain Adaptation

Domain Adaptation aims at utilizing data from a label-rich source domain to
solve a classification problem in a label-scarce target domain. We distinguish
homogeneous domain adaptation where the feature space in both domains is the
same (e.g. images of same resolution) and heterogeneous domain adaptation
where this assumption does not hold and more general transfers (e.g. text to
image) can be considered Yao2019a . In our work we will focus on homogeneous
domain adaptation. The biggest challenge for this task is the different
feature distributions between the domains, which is why most approaches map
the domains to a common subspace Saito2018 ; Yao2019a . Much research in this
area focuses on unsupervised domain adaptation where no labels in the target
domain are available Long2016 ; Saito2018 ; wang2021discriminative . Matching
the distributions of source and target features in a shared subspace is the
key component of these methodologies and their main differentiating element.
One line of research considers generative adversarial learning methods between
the latent feature generator and the classifier to make the latent features of
both domains indistinguishable Saito2018 ; Ganin2017 . Other methods directly
minimize some statistical distance measures between the domains like MMD
Long2015 ; Long2016 . The authors of Xu2019  found that a stepwise adaption of
feature norms of both domains results in significant transfer gains.
Semi-supervised domain adaptation on the other hand assumes that some labeled
examples of all classes are available in the target domain. The methodologies
also rely mostly on an aligned shared latent feature space but are able to do
a semantic alignment Motiian2017a  hence aligning the class conditional
distributions. Similar to the UDA case many state of the art algorithms apply
adverserial training Motiian2017 ; Saito2019 .

##  3 Methodology

Figure 1: Overview of our model architecture (best viewed in colour). In the
first step we identify a candidate set of likely positive and negative data
based on classical loss on source domain and PU loss on target domain after
aligning the distribution space between both domains. In the second step we
train a ResNet classifier on the pseudo-labeled target set. Only the
classifier from the second step is used to do inference.

Given a source domain $\mathcal{D}_{S}=\\{(x_{i},y_{i})\\}_{i=1}^{n_{S}}$ with
$n_{S}$ labeled samples and a target domain
$\mathcal{D}_{T}=\mathcal{D}_{T}^{P}\cup\mathcal{D}_{T}^{U}$ where
$\mathcal{D}_{T}^{P}=\\{(x_{i},1)\\}_{i=1}^{n_{P}}$ contains positive examples
from the target domain and $\mathcal{D}_{T}^{U}=\\{(x_{i})\\}_{i=1}^{n_{U}}$
are unlabeled examples from the target domain, containing both positive and
negative examples. Usually we have $n_{P}\ll n_{U}$ and $n_{P}\ll n_{S}$.
We train on $D_{S}$, $D_{T}^{P}$ and $D_{T}^{U}$ and evaluate on $D_{T}^{U}$.
For our setting we focus on domain transfer between images of same resolution,
so $\mathcal{D}_{S}$ and $\mathcal{D}_{T}$ live on the same feature space but
with different feature distributions.
We propose a two step learning approach where in the first step we identify
reliable negative and additional positive data in the target domain based on
labeled source examples and positive target data using a PU risk estimator
combined with an unsupervised domain adaptation algorithm. In the second step
we then apply standard supervised learning models to the pseudo-labeled target
data. Our algorithm is summarized in Algorithm 1 and visualized in Figure 1.

###  3.1 Step 1 - Constructing pseudo target labels

####  3.1.1 Warm-Up Phase

The key idea of our approach is to identify a subset of the unlabeled target
data which we can label with high confidence based on the labeled source
dataset. It is widely known that the naive transfer from a model between
different datasets performs poorly due to feature distribution shift between
the domains. In Domain Adaptation it is therefore common to learn a shared
embedding space between the two domains with some (deep) network $G$ which is
stacked together with a classifier $F$ such that the predicted class
probability is calculated as $p(x)=softmax(F(G(x))$ where $x$ is either from
$D_{S}$ or $D_{T}$. For our purpose we build upon the state-of-the art feature
alignment methodology from wang2021discriminative . The authors propose a
discriminative feature alignment (DFA) using an indirect alignment of the
shared embedding space guided by a Gaussian prior distribution together with
an encoder-decoder architecture. Given a Gaussian prior distribution
$z\sim\mathcal{N}(0,1)$ we follow their work and define two distribution
alignment losses for source and target respectively. The source domain is
aligned to the Gaussian prior directly in the shared embedding space using
Kullback-Leibler divergence $D_{KL}$.

$\mathcal{L}_{S}=D_{KL}(z\ \|\ \mathcal{D}_{S}).$ (1)

The target domain is first reconstructed using the decoder network $D$
together with the prior distribution before aligning their distributions.
According to the authors this ensures the learned embedding to be
discriminative. The alignment loss on the target is therefore

$\mathcal{L}_{T}=\frac{1}{n_{T}}\sum_{x\in\mathcal{D}_{T}}\lVert
D(G(x))-D(z)\rVert_{1},$ (2)

where $\lVert x\rVert_{1}$ is the L1-norm of $x$. The DFA algorithm is also
leveraging the stepwise adaptive feature norm (SAFN) loss from Xu2019 . The
authors found that including a regularization for the feature norm enlargement
helps in transfer tasks since any domain gap is to some extend due to
misaligned feature norm distributions. Denote for the classifier network $F$
with $l$ layers the first $l-1$ layers as $F_{l}$ and with $h(x)=\lVert
F_{l}(G(x))\rVert_{2}$ the L2-norm of the feature representation in the last
layer. The loss is defined as

$\mathcal{L}_{SAFN}=\lVert h(x,\theta_{P})+\delta-h(x,\theta_{C})\rVert_{2},$
(3)

where $\theta$ describes the network parameters of the current- ($\theta_{C}$)
and the previous ($\theta_{C}$) training iteration with $\delta$ being a
hyperparameter to control the feature norm enlargement in each step.

Putting it all together, we define the sum of the distribution alignment
losses stemming from DFA as

$\mathcal{L}_{align}=\mathcal{L}_{S}+\mathcal{L}_{T}+\mathcal{L}_{SAFN}.$ (4)

We further denote with

$\mathcal{L}_{cls}=\frac{1}{n_{S}}\sum_{(x_{i},y_{i})\in\mathcal{D}_{S}}y_{i}\log(p(x_{i}))+(1-y_{i})\log(1-p(x_{i}))$
(5)

the cross-entropy loss on the source domain. Since the DFA algorithm is
designed for unlabeled domain adaptation and hence is not leveraging the
positive labeled target data, we include an additional loss term $L_{PU}$ to
the architecture which is the non-negative unbiased PU-risk estimator from
Kiryo2017 . It leverages the fact that we can regard the unlabeled data as
mixture of positive and negative examples which a prior probability $\pi$ of
an example being positive, hence

$p(x)=\pi p(x|y=1)+(1-\pi)p(x|y=0).$ (6)

The class prior $\pi$ can be estimated from unlabeled data Jain2016 ;
Christoffel2016  and is assumed to be known throughout this paper. Applying
their unbiased PU-loss formulation for a given loss function $l$ leads to

$\displaystyle\mathcal{L}_{PU}$
$\displaystyle=\frac{\pi}{n_{P}}\sum_{x\in\mathcal{D}_{T}^{P}}l(F(G(x)),1)+\max\big{(}0,$
$\displaystyle\quad\frac{1}{n_{U}}\sum_{x\in\mathcal{D}_{T}^{U}}l(F(G(x)),0)-\frac{\pi}{n_{P}}\sum_{x\in\mathcal{D}_{T}^{P}}l(F(G(x)),0)\big{)}.$

Our final objective function is therefore

$\mathcal{L}=\mathcal{L}_{PU}+\alpha\mathcal{L}_{cls}+\beta\mathcal{L}_{align}.$
(7)

Since the PU-loss is the only measure defined on the target domain we consider
it as most important and define regularization weights $\alpha$, $\beta$ for
both other loss terms.

We train the network for 20 epochs in the warm-up phase minimizing (7) before
we start to identify likely positive and negative target data using our label
selection methodology.

####  3.1.2 Label Selector

We apply the label selector each epoch $i$ after the warm up phase until the
end of the first step. The label selector is selecting a subset
$\mathcal{D}_{L}^{i}$ of $\mathcal{D}_{T}^{U}$ and stores it into a label-
candidate set $D_{c}^{i}$. We make use of the positive labeled target examples
to compute the threshold $T_{pos}=\frac{1}{n_{P}}\sum_{x\in D_{T}^{P}}p(x)$
and select those $x\in D_{T}^{U}$ as likely positive where $p(x)>T_{pos}$
holds. For the negative data we estimate the threshold using the source domain
and define $T_{neg}=\frac{1}{n_{S}^{N}}\sum_{x\in D_{S}^{N}}p(x)$ where
$D_{S}^{N}=\\{(x,y)\in\mathcal{D}_{S}|y=0\\}$. We select those $x\in
D_{T}^{U}$ as likely negative where $p(x)<T_{neg}$ holds. We use the predicted
positive class probabilities as pseudo label and obtain

$\mathcal{D}_{c}^{i}=\\{(x,p(x))\in\mathcal{D}_{T}^{U}\ |\ p(x)>T_{pos}\text{
or }p(x)<T_{neg}\\}.$ (8)

Note that unlike other self-learning approaches like Chen2020  we do not
remove label-candidates from the unlabeled dataset. This ensures that the PU-
risk is estimated on all initially unlabeled data during the whole training
period in step 1. Another implication is that the same observation can be
added to $\mathcal{D}_{c}$ multiple times with different $p(x)$. We even
expect a confident examples to be consistently classified with high confidence
throughout the epochs. We therefore include a final label extractor logic that
computes the pseudo-target-labels for a subset of $\mathcal{D}_{c}$ after the
first step is finished, i.e. the model was trained for the specified number of
epochs . Let $D_{c}=\bigcup_{i}D_{c}^{i}$ and

$Count(x;\mathcal{D}_{c})=\sum_{(x^{\prime},p)\in D_{c}}I(x=x^{\prime}).$ (9)

Then we construct the final pseudo-labeled dataset
$\mathcal{D}_{pseudo}=\mathcal{D}_{pseudo}^{P}\cup\mathcal{D}_{pseudo}^{N}$ as

$\displaystyle\begin{split}\mathcal{D}_{pseudo}^{P}=\\{(x,\overline{p(x)})\in\mathcal{D}_{c}\
|&\ Count(x)\geq m\text{ and }\\\ &\ \min(p(x))>t_{P}\\}\end{split}$ (10)

and

$\displaystyle\begin{split}\mathcal{D}_{pseudo}^{N}=\\{(x,\overline{p(x)})\in\mathcal{D}_{c}\
|&\ Count(x)\geq m\text{ and }\\\ &\ \max(p(x))<t_{N}\\}\end{split}$ (11)

where $t_{N}$ and $t_{P}$ are thresholds for the minimum required negative and
positive confidence respectively and $m$ is the minimal required number of
occurrences of $x$ in $\mathcal{D}_{c}$. We set those parameters empirically
to $0.05$, $0.95$ and $20$, respectively, throughout all experiments.

###  3.2 Step 2 - Obtaining the final classifier

After finishing step 1 and building $\mathcal{D}_{pseudo}$ we use this pseudo
labeled dataset for training a classifier $C$ using a standard cross entropy
loss

$\displaystyle\begin{split}\mathcal{L}_{cls}=\frac{1}{|D_{pseudo}|}&\sum_{(x_{i},y_{i})\in\mathcal{D}_{pseudo}}y_{i}\log(p(x_{i})\\\
&+(1-y_{i})\log(1-p(x_{i}))\end{split}$ (12)

We initialise $C$ with a pretrained ResNet-34 He2016  architecture trained on
ImageNet, which is common technique in domain transfer Sankaranarayanan2018 ,
and train until a stopping criterion is reached.

Algorithm 1 PU-DA

Input: $\mathcal{D}_{S}$, $\mathcal{D}_{T}^{P}$, $\mathcal{D}_{T}^{U}$
Output: Trained Classifier $C$

1: Initialize $\mathcal{D}_{c}=\mathcal{D}_{pseudo}=\varnothing$

2: Initialize Networks $G$, $F$, $C$

3: while $epoch<max\\_epoch$ do

4: Compute $T_{pos}$ and $T_{neg}$

5: Minimize (7) and update $G$, $F$

6: if $epoch>warm\\_up$ then

7: $\mathcal{D}_{c}=\mathcal{D}_{c}\cup\\{(x,p(x))\in\mathcal{D}_{T}^{U}\ |\
p(x)>T_{pos}\\}$

8: $\mathcal{D}_{c}=\mathcal{D}_{c}\cup\\{(x,p(x))\in\mathcal{D}_{T}^{U}\ |\
p(x)<T_{neg}\\}$

9: end if

10: end while

11: Construct $\mathcal{D}_{pseudo}$ based on $\mathcal{D}_{c}$ using (10) and
(11)

12: while $epoch<max\\_epoch$ do

13: for $batch=1,\ldots N$ do

14: Sample batch $(x_{i},y_{i})$ from $\mathcal{D}_{pseudo}$

15: Optimize (12) and Update $C$

16: end for

17: end while

Target | c | DFA | MME | STN | FADA | NN-PU | PUbN | PU-DA(OURS)
---|---|---|---|---|---|---|---|---
Product | 1 | $93.77^{0.8}$ | $\textbf{94.74}^{0.5}$ | $87.68^{1.6}$ | $85.53^{2.2}$ | $61.12^{1.9}$ | $58.59^{1.4}$ | $\textbf{94.91}^{1.5}$
5 | $93.77^{0.8}$ | $\textit{94.02}^{0.4}$ | $92.35^{1.2}$ | $85.72^{2.3}$ | $79.99^{4.5}$ | $84.75^{0.8}$ | $\textbf{97.15}^{1.4}$
10 | $93.77^{0.8}$ | $\textit{94.16}^{0.5}$ | $92.35^{2.5}$ | $85.16^{2.0}$ | $91.57^{4.9}$ | $93.4^{1.4}$ | $\textbf{97.58}^{1.2}$
Art | 1 | $\textit{85.86}^{1.5}$ | $\textbf{87.38}^{0.8}$ | $64.37^{2.2}$ | $78.88^{2.4}$ | $56.16^{1.2}$ | $62.14^{0.2}$ | $79.04^{1.8}$
5 | $\textit{85.86}^{1.5}$ | $84.74^{0.8}$ | $73.29^{1.4}$ | $78.75^{2.9}$ | $61.31^{5.0}$ | $69.44^{1.3}$ | $\textbf{86.58}^{2.0}$
10 | $\textit{85.86}^{1.5}$ | $83.60^{1.0}$ | $77.96^{1.1}$ | $78.70^{2.9}$ | $64.14^{5.2}$ | $76.82^{4.8}$ | $\textbf{87.09}^{1.7}$
Real | 1 | $\textbf{91.98}^{1.1}$ | $\textit{91.63}^{0.7}$ | $73.61^{2.8}$ | $84.91^{2.2}$ | $60.31^{6.0}$ | $73.37^{2.3}$ | $88.65^{1.5}$
5 | $\textit{91.98}^{1.1}$ | $90.78^{0.8}$ | $86.67^{3.2}$ | $84.22^{2.5}$ | $65.61^{7.2}$ | $61.76^{3.7}$ | $\textbf{92.52}^{1.5}$
10 | $91.98^{1.1}$ | $\textbf{92.8}^{0.8}$ | $89.18^{2.0}$ | $82.69^{2.7}$ | $73.45^{7.6}$ | $90.87^{1.7}$ | $\textit{92.0}^{1.4}$
Clipart | 1 | $\textit{87.71}^{1.8}$ | $\textbf{88.3}^{0.8}$ | $86.41^{4.4}$ | $81.99^{3.1}$ | $62.06^{5.0}$ | $68.57^{3.5}$ | $80.43^{2.6}$
5 | $87.71^{1.8}$ | $86.98^{0.6}$ | $\textit{88.93}^{2.6}$ | $84.25^{1.9}$ | $70.22^{5.0}$ | $74.87^{1.7}$ | $\textbf{90.81}^{2.7}$
10 | $87.71^{1.8}$ | $86.36^{0.6}$ | $\textit{89.65}^{1.9}$ | $84.23^{1.6}$ | $77.64^{4.9}$ | $86.01^{0.3}$ | $\textbf{91.95}^{1.8}$
Summary Score | 3.5 | 5.5 | 1 | 0 | 0 | 0 | 8.5
Table 1: Accuracy and standard deviation on unlabeled target data for Monitor
vs. Laptop classification on Office-Home dataset.
All results are averaged over 20 runs and three adaptation scenarios. We
compute a summary score per methodology to allow an easy performance
comparison across different domains and label frequencies. We award one point
for the best performing model and half a point for the second best model per
scenario.

##  4 Experiments

We validate our methodology on well known benchmark object recognition and
image classification datasets for domain adaptation, namely Office-Home
venkateswara2017deep , Office-Caltech Saenko2010  and MNIST-USPS Hull1994
which are all publicly accessible. We binarize the prediction problem by
choosing two visually similar classes as positive and negative respectively
while dropping all other classes. In the target domain we remove labels
according to the label frequency $c=P(s=1|y=1)$, where $s$ is the property of
an example being labeled. Hence we keep $c$ percent of the positive labels and
consider everything else as unknown. We conduct every experiment for label
frequencies 1, 5 and 10 percent.
All results are reported as the average accuracy together with standard
deviation of twenty training runs, where each training uses a different
randomly selected labeled subset in the target domain. The standard deviation
is therefore mainly measuring the influence of different labeled positive
subsets and not so much the influence of stochasticity in the learning
optimization process itself.
We perform hyperparameter training for each (baseline-) model using baysian
search Perrone2019 . We report all hyperparameters and their chosen values in
the supplementary material.

###  4.1 Baselines

We consider state of the art baselines from unsupervised- and semi-supervised
Domain Adaptation as well as from PU learning. All baselines are implemented
based on their public available code repositories with small adaptions related
to the structure of our datasets which is described in more detail in the
supplementary material.

DFA wang2021discriminative  denotes the discriminative feature alignment
methodology by a gaussian prior which we introduced in the methodology section
as basis for the domain alignment we use. Since this algorithm performs
unsupervised domain adaptation, this is our baseline for considering only the
domain adaptation problem, ignoring the positive labels in the target domain.
From semi-supervised domain adaptation methodologies we consider MiniMax
Entropy (MME)Saito2018 , Soft Transfer Network(STN)Yao2019a  and Few-Shot
Adversarial Domain Adaptation (FADA)Motiian2017  methodologies. All of them
provide strong performance under the assumption that few labels of every class
are available in the target domain. Since these algorithms originally use
examples of all classes we had to make small adaptations to apply them to our
novel setting. STN computes a classwise loss in the target domain, so we
removed the summand of the negative class from the loss function. FADA also
requires an explicit negative class in the target domain which is used in a
more complex way than simply as summand in the loss function, so we regarded
all unlabeled data as negative. The MME algorithm uses all available target
labels without explicitly separating the classes, so we were able to use it’s
original implementation.
From PU research we consider NN-PUKiryo2017  and PUbNHsieh2019 . Both use
special risk estimators to account for the PU setting. All PU baselines were
applied to the target domain only since they are not designed to leverage
domain transfer information.

###  4.2 Office-Home Task

(a) Product Domain

(b) Clipart Domain

(c) Art Domain

(d) Real World Domain

Figure 2: Examples of positive monitor class (upper row) and negative laptop class (lower row) per domain in Office-Home  Source | c | DFA | MME | STN | FADA | NN-PU | PUbN | PU-DA(OURS)
---|---|---|---|---|---|---|---|---
Webcam | 1 | $91.57^{2.5}$ | $\textbf{94.96}^{0.5}$ | $91.10^{2.7}$ | $79.56^{6.1}$ | $66.61^{7.0}$ | $84.66^{1.7}$ | $\textit{94.35}^{3.2}$
5 | $89.47^{2.5}$ | $\textit{93.63}^{0.4}$ | $93.5^{1.4}$ | $79.41^{4.3}$ | $74.09^{4.8}$ | $85.03^{1.1}$ | $\textbf{94.76}^{2.7}$
10 | $89.47^{2.5}$ | $93.34^{0.5}$ | $94.30^{1.3}$ | $82.17^{2.5}$ | $88.88^{1.0}$ | $\textbf{94.38}^{0.5}$ | $\textbf{95.15}^{2.2}$
Caltech | 1 | $91.57^{2.0}$ | $\textit{94.27}^{0.6}$ | $84.90^{1.5}$ | $90.70^{1.7}$ | $66.61^{7.0}$ | $84.66^{1.7}$ | $\textbf{95.06}^{0.6}$
5 | $91.57^{2.0}$ | $93.53^{0.4}$ | $\textit{94.31}^{0.9}$ | $88.97^{2.0}$ | $74.09^{4.8}$ | $85.03^{1.1}$ | $\textbf{94.81}^{0.6}$
10 | $91.57^{2.0}$ | $92.81^{0.6}$ | $94.35^{0.6}$ | $89.19^{1.3}$ | $88.88^{1.0}$ | $\textit{94.38}^{0.5}$ | $\textbf{95.10}^{1.0}$
Summary Score | 0 | 2 | 0.5 | 0 | 0 | 1.5 | 5.5
Table 2: Accuracy and standard deviation on the Office-Caltech classification
task for target amazon

The Office-Home dataset contains 4 domains (Real (R), Clipart (C), Art (A),
Product (P)) with 65 classes. To make the binary prediction class challenging
we consider two visually similar classes as positive and negative, namely
computer-monitor and laptop. Figure 2 shows example images of each domain. One
special property of this dataset is that the number of images per class is
rather limited. We have approximately 100 images per class which makes the
label percentage $c$ equal to the absolute number of labeled target examples.
The four domains allow 12 adaptation scenarios in total and we evaluate our
methodology on all of them. For each adaptation scenario we investigated the
influence of the labeled prior in the target domain which leads to a total of
36 experiments. For simplicity we report the averaged result per target in
Table 1, e.g. the results for target product is the averaged result of three
adaptation scenarios (R $\rightarrow$ P, C $\rightarrow$ P, A $\rightarrow$
P). We provide the full results in the supplementary material.
We compute a summary score per method to allow a quick overall comparison of
methodologies over different adaptation scenarios and labeled priors. The
summary score is obtained by awarding one point for each scenario where the
methodology performs best and half a point for the second best methodology. If
there is no statistically significant difference (two-sided T-test, p-value
smaller 5 percent) between the best methodologies we provide one point for
each of them. We find that our approach has the best performance overall,
followed by SSDA approach MME and UDA approach DFA. We therefore can conclude
that among the SSDA baselines MME can best leverage information from a PU
subset in the target domain, while FADA is by design not performing well in
our setting. Also we find the PU-approaches to perform worse than DA which is
indicating that the source domain contains more information than the small
labeled positive subset in the target domain. The strong performance of our
methodology shows however that combining both sources of information is
beneficial.
Taking a closer look at the single scenarios we note that our methodology is
mainly outperforming all baselines for 5- and 10-shot learning of positive
labels in the target domain. However we find that for the one-shot scenario
our methodology seems to perform worse than unsupervised domain adaptation.
Since the labeled positive data is guiding our selection process of new
trusted positives, a single, unrepresentative labeled target data apparently
hurts the performance. We plan to investigate how to make the approach more
robust in the one-shot scenario in future works.
We also find that even though MME is mostly outperforming unsupervised domain
adaptation, increasing the number of labeled target data seem to even hurt
model performance of MME since it introduces a bias towards positive
predictions.
Although not surprising, it is worth mentioning that the performance gap
between DA and PU approaches is getting smaller with an increasing number of
positive labels in the target domain.
With respect to standard deviation we find that especially the NN
methodologies varies quite much in its performance. This is indicating that in
few shot scenarios pure PU-learning performance highly depends on the
representativity of the labeled subset. Methodologies that also use the source
domain are more robust in their performance.

###  4.3 Office-Caltech Task

The Office dataset contains pictures of different product categories from
three distinct sources (Webcam, DSLR and Amazon) which differ in resolution
and background noise. The Caltech-256 dataset shares ten product categories
with the office dataset and therefore they are often combined for domain
adaptation tasks Gong2012 . We follow Yao2019a  and consider amazon data as
target domain and investigate caltech and webcam as sources respectively.
While amazon and caltech are of similar size with little more than 100
examples per class, webcam is even smaller providing only 43 positive and 30
negative examples. We apply the same setting as in Office-Home and investigate
the scenario computer-monitor vs laptop.
In terms of overall performance we again find that our methodologies
outperforms all baselines from which MME again is second-best. Looking at the
single scenarios, we find that unlike in the Office-Home dataset there is no
significant difference in performance of our algorithm between different
labeled priors. Since the amazon domain contains images taken from amazon
marketplace which are usually of high quality and similar to each other within
one class, even one randomly selected image seem to be representative for the
positive domain and enables our algorithm to build a meaningful pseudo-labeled
dataset.
We also find that 10 labelled positive examples are enough for the PUbN
methodology to outperform domain adaptation, i.e. for the amazon data 10
available labels in the target domain enable a better prediction than the
fully labeled source domain. This again is an indication that the classes are
better separated in the amazon domain compared to the Office-Home domains.
This hypotheses is in line with the high performance in the product domain of
the Office-Home task since the product domain like the amazon dataset contains
pictures of products sold in some online marketplace and are therefore less
noisy than domains like art or clipart.
We therefore find that our methodology performs strongly given a
representative positive labeled subset.
We find that over both experiments only 5 labeled datapoints in the target are
sufficient for us to consistently outperform all baselines. Whether our
methodology works on a single labeled datapoint is dependent on the domain,
where we outperform baselines in domains of high picture quality and class
separability like amazon or product.

###  4.4 Digit Task

Domainc | 1 | 5 | 10 |  | Summary
---
Score
DFA | $\textit{96.09}^{2.4}$ | $96.09^{2.4}$ | $96.09^{2.4}$ | 0.5
MME | $93.41^{0.6}$ | $95.94^{0.6}$ | $\textit{96.79}^{0.7}$ | 0.5
STN | $92.1^{1.0}$ | $95.6^{1.2}$ | $96.5^{0.9}$ | 0
FADA | $49.92^{2.3}$ | $60.68^{6.9}$ | $61.73^{5.7}$ | 0
NN-PU | $52.25^{6.8}$ | $72.15^{1.3}$ | $84.22^{1.1}$ | 0
PUbN | $92.68^{2.5}$ | $\textit{96.38}^{1.7}$ | $95.37^{1.7}$ | 0.5
| PU-DA
---
(Ours)
$\textbf{97.48}^{0.9}$ | $\textbf{97.2}^{1.1}$ | $\textbf{97.46}^{0.9}$ | 3
Table 3:  Accuracy and standard deviation on the adaptation scenario from
MNIST to USPS for 3 vs 5 classification.

Another commonly used benchmark for domain adaptation is the transfer between
the digit datasets MNIST LeCun1998  and USPS Hull1994 . Both contain images of
handwritten digits and have around 5000 training points per class. We follow
Hou2018  and binarize the classification by considering the visually similar
digits 3 as positive and 5 as negative class. We consider this dataset to
investigate our methodology in a scenario where we have more labeled data in
the target domain, i.e. even a small labeled prior of 1 percent is considering
fifty labeled target images. The results are summarized in Table 3. We clearly
see that our approach is outperforming the baselines consistently over all
scenarios. Similar to Office-Caltech we do not note significant differences
between the different labeled priors which again indicates that a
representative labeled subset even of small size allows to build a high
quality pseudo-labeled set.

###  4.5 Real World Applications

In the previous experiments we showed that our methodology outperforms
baselines from PU learning and domain adaptation on well known benchmark
datasets. However for the considered data it is unrealistic to assume that
only positive lables are available in one domain while the other is completely
labeled. We therefore like to discuss real world applications of our
methodology and validate our methodology on a dataset from the mobility
domain, namely parking availability prediction.

####  4.5.1 Parking Availability

Predicting the availability of on-street parking locations is a relevant
problem for traffic management in smart cities. Parking behaviour in different
cities is related but suffers from domain shift Shao2021 , e.g. business areas
tend to be occupied during the day but available in the evening in most cities
while the distribution of those areas varies between cities. Many researchers
are aiming at predicting the binary occupancy status of parking location where
the location consists of several individual parking bays (e.g. all on-street
parking bays between two intersections). A location is considered occupied if
all individual parking bays are occupied, otherwise available Sonntag2021 .
Some cities like Melbourne in Australia have installed in-ground sensors that
detect the occupancy status of the parking
bay222https://www.melbourne.vic.gov.au/about-council/governance-
transparency/open-data/Pages/on-street-parking-data.aspx. However due to the
high costs of installing and maintaining such sensors only 5 percent of all
individual parking bays in Melbourne are equipped with such a sensor. As a
result negative availability labels can only be inferred for locations where
every single bay is equipped with a sensor. Positive labels however are much
easier to obtain since it is enough to observe just one available individual
bay to label the whole location as available. To regard parking prediction as
PU learning problem was first proposed by Sonntag2021 .
Our methodology can therefore be used to predict parking availability in
cities which are only partially covered by sensors guided by fully sensorized
cities.
We used those streets in Melbourne which are fully sensorized as source domain
and as target we consider Seattle, US which also publishes its parking
data333https://data.seattle.gov/Transportation/Annual-Parking-Study-Data/7jzm-
ucez. Details on the data processing and the considered features are provided
in the supplementary material. Since the labels are imbalanced towards the
positive class we report the balanced accuracy score Brodersen2010 .

Modelc | 1 | 5 | 10 |  | Summary
---
Score
DFA | $61.98^{1.0}$ | $63.35^{1.3}$ | $62.14^{1.0}$ | 0
MME | $62.36^{1.1}$ | $62.36^{1.2}$ | $62.37^{0.9}$ | 0
STN | $63.20^{0.7}$ | $63.21^{0.7}$ | $63.25^{0.8}$ | 0
FADA | $\textit{63.68}^{1.9}$ | $\textit{63.67}^{2.0}$ | $\textit{63.68}^{1.6}$ | 1.5
NN-PU | $61.13^{1.2}$ | $61.11^{1.4}$ | $61.13^{1.3}$ | 0
PUbN | $62.40^{1.7}$ | $62.41^{1.7}$ | $62.51^{1.4}$ | 0
| PU-DA
---
(Ours)
$\textbf{64.84}^{1.7}$ | $\textbf{65.33}^{1.5}$ | $\textbf{65.09}^{1.5}$ | 3
Table 4:  Balanced Accuracy and standard deviation on the parking availability
scenario from Melbourne to Seattle.

The results are summarized in Table 4 and we clearly see our methodology
outperforming all baselines.

####  4.5.2 Classification of Alzheimer Disease from MRI Images

Image classification for medical MRI images suffer from domain shift since
different scanners, scanning parameters and subject cohorts induce different
feature distributions Guan2021 . The classification of Alzheimer patients from
MRI images on the other hand is an accepted baseline for PU learningChen2020
since ill patients go through a transition period where they are seldom
diagnosed. In this phase they are considered ill but are often labeled wrongly
as healthy (negative). The authors in Chen2020  successfully considered all
non-diagnosed patients as unlabeled instead of healthy.
It is however in general possible to provide full labels in retrospective.
This effort is much better invested when the labeled dataset can be used
together with images on all kinds of scanners and scanner parameters. We
therefore think it would be highly interesting to apply our methodology to
medical image classification in future research.

##  5 Conclusion

In this work we introduced a novel domain adaptation scenario were we aim to
generalize from a fully labeled source domain to a target domain with only few
positive and unlabeled data. Since there are naturally no methodologies
designed for this setting yet, we proposed a relatively simple yet effective
two-step learning approach inspired by recent work from semi-supervised
learning and unsupervised domain adaptation. We thereby set the first baseline
for positive-unlabeled domain adaptation. We benchmarked our approach on
various publicly available datasets from image classification and found that
our approach outperforms the baselines from (semi-supervised)-domain
adaptation and PU-learning even in few shot scenarios with 5 and 10 positive
target examples.
We also proposed real-world applications for our novel setting and validated
our methodology on the use case of parking availability predictions.
In future works we plan to develop further methodologies in our new setting
inspired by approaches that are commonly used in DA- and PU-learning like
GANs.

## References

  * [1] Eric Arazo, Diego Ortego, Paul Albert, Noel E. O’Connor, and Kevin McGuinness.  Pseudo-labeling and confirmation bias in deep semi-supervised learning.  In 2020 International Joint Conference on Neural Networks, IJCNN 2020, Glasgow, United Kingdom, July 19-24, 2020, pages 1–8. IEEE, 2020\.
  * [2] Jessa Bekker and Jesse Davis.  Learning from positive and unlabeled data: a survey.  Mach. Learn., 109(4):719–760, 2020.
  * [3] Gilles Blanchard, Gyemin Lee, and Clayton Scott.  Semi-supervised novelty detection.  J. Mach. Learn. Res., 11:2973–3009, 2010.
  * [4] Kay Henning Brodersen, Cheng Soon Ong, Klaas Enno Stephan, and Joachim M. Buhmann.  The balanced accuracy and its posterior distribution.  In 20th International Conference on Pattern Recognition, ICPR 2010, Istanbul, Turkey, 23-26 August 2010, pages 3121–3124. IEEE Computer Society, 2010.
  * [5] Xuxi Chen, Wuyang Chen, Tianlong Chen, Ye Yuan, Chen Gong, Kewei Chen, and Zhangyang Wang.  Self-pu: Self boosted and calibrated positive-unlabeled training.  In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Research, pages 1510–1519. PMLR, 2020.
  * [6] Marthinus Christoffel, Gang Niu, and Masashi Sugiyama.  Class-prior estimation for learning from positive and unlabeled data.  In Asian Conference on Machine Learning, pages 221–236, 2016.
  * [7] Marc Claesen, Jesse Davis, Frank De Smet, and Bart De Moor.  Assessing binary classifiers using only positive and unlabeled data.  CoRR, abs/1504.06837, 2015.
  * [8] Marthinus Christoffel du Plessis, Gang Niu, and Masashi Sugiyama.  Convex formulation for learning from positive and unlabeled data.  In Francis R. Bach and David M. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 1386–1394. JMLR.org, 2015.
  * [9] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor S. Lempitsky.  Domain-adversarial training of neural networks.  In Gabriela Csurka, editor, Domain Adaptation in Computer Vision Applications, Advances in Computer Vision and Pattern Recognition, pages 189–209. Springer, 2017.
  * [10] Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman.  Geodesic flow kernel for unsupervised domain adaptation.  In 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, June 16-21, 2012, pages 2066–2073. IEEE Computer Society, 2012.
  * [11] Hao Guan and Mingxia Liu.  Domain adaptation for medical image analysis: a survey.  arXiv preprint arXiv:2102.09508, 2021.
  * [12] Zayd Hammoudeh and Daniel Lowd.  Learning from positive and unlabeled data with arbitrary positive shift.  In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
  * [13] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 770–778. IEEE Computer Society, 2016.
  * [14] Ming Hou, Brahim Chaib-draa, Chao Li, and Qibin Zhao.  Generative adversarial positive-unlabelled learning.  In Jérôme Lang, editor, Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden, pages 2255–2261. ijcai.org, 2018.
  * [15] Yu-Guan Hsieh, Gang Niu, and Masashi Sugiyama.  Classification from positive, unlabeled and biased negative data.  In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, volume 97 of Proceedings of Machine Learning Research, pages 2820–2829. PMLR, 2019.
  * [16] Jonathan J. Hull.  A database for handwritten text recognition research.  IEEE Transactions on pattern analysis and machine intelligence, 16(5):550–554, 1994.
  * [17] Shantanu Jain, Martha White, Michael W Trosset, and Predrag Radivojac.  Nonparametric semi-supervised learning of class proportions.  arXiv preprint arXiv:1601.01944, 2016.
  * [18] Ryuichi Kiryo, Gang Niu, Marthinus Christoffel du Plessis, and Masashi Sugiyama.  Positive-unlabeled learning with non-negative risk estimator.  In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017, Long Beach, CA, USA, pages 1675–1685, 2017.
  * [19] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner.  Gradient-based learning applied to document recognition.  Proceedings of the IEEE, 86(11):2278–2324, 1998.
  * [20] Wenkai Li, Qinghua Guo, and Charles Elkan.  A positive and unlabeled learning algorithm for one-class classification of remote-sensing data.  IEEE Trans. Geosci. Remote. Sens., 49(2):717–725, 2011.
  * [21] Mohammad Reza Loghmani, Markus Vincze, and Tatiana Tommasi.  Positive-unlabeled learning for open set domain adaptation.  Pattern Recognit. Lett., 136:198–204, 2020.
  * [22] Mingsheng Long, Yue Cao, Jianmin Wang, and Michael I. Jordan.  Learning transferable features with deep adaptation networks.  In Francis R. Bach and David M. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 97–105. JMLR.org, 2015.
  * [23] Mingsheng Long, Han Zhu, Jianmin Wang, and Michael I. Jordan.  Unsupervised domain adaptation with residual transfer networks.  In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett, editors, Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 136–144, 2016.
  * [24] Saeid Motiian, Quinn Jones, Seyed Mehdi Iranmanesh, and Gianfranco Doretto.  Few-shot adversarial domain adaptation.  In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6670–6680, 2017.
  * [25] Saeid Motiian, Marco Piccirilli, Donald A. Adjeroh, and Gianfranco Doretto.  Unified deep supervised domain adaptation and generalization.  In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 5716–5726. IEEE Computer Society, 2017.
  * [26] Arvind Neelakantan, Benjamin Roth, and Andrew McCallum.  Compositional vector space models for knowledge base completion.  In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 1: Long Papers, pages 156–166. The Association for Computer Linguistics, 2015.
  * [27] Valerio Perrone and Huibin Shen.  Learning search spaces for bayesian optimization: Another view of hyperparameter transfer learning.  In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 12751–12761, 2019.
  * [28] Mamshad Nayeem Rizve, Kevin Duarte, Yogesh Singh Rawat, and Mubarak Shah.  In defense of pseudo-labeling: An uncertainty-aware pseudo-label selection framework for semi-supervised learning.  In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
  * [29] Kate Saenko, Brian Kulis, Mario Fritz, and Trevor Darrell.  Adapting visual category models to new domains.  In Kostas Daniilidis, Petros Maragos, and Nikos Paragios, editors, Computer Vision - ECCV 2010, 11th European Conference on Computer Vision, Heraklion, Crete, Greece, September 5-11, 2010, Proceedings, Part IV, volume 6314 of Lecture Notes in Computer Science, pages 213–226. Springer, 2010.
  * [30] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko.  Semi-supervised domain adaptation via minimax entropy.  In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 8049–8057. IEEE, 2019.
  * [31] Kuniaki Saito, Kohei Watanabe, Yoshitaka Ushiku, and Tatsuya Harada.  Maximum classifier discrepancy for unsupervised domain adaptation.  In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 3723–3732. IEEE Computer Society, 2018.
  * [32] Tomoya Sakai and Nobuyuki Shimizu.  Covariate shift adaptation on learning from positive and unlabeled data.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 4838–4845, 2019.
  * [33] Swami Sankaranarayanan, Yogesh Balaji, Carlos Domingo Castillo, and Rama Chellappa.  Generate to adapt: Aligning domains using generative adversarial networks.  In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 8503–8512. IEEE Computer Society, 2018.
  * [34] Wei Shao, Sichen Zhao, Zhen Zhang, Shiyu Wang, Mohammad Saiedur Rahaman, Andy Song, and Flora D. Salim.  FADACS: A few-shot adversarial domain adaptation architecture for context-aware parking availability sensing.  In 19th IEEE International Conference on Pervasive Computing and Communications, PerCom 2021, Kassel, Germany, March 22-26, 2021, pages 1–10. IEEE, 2021.
  * [35] Jonas Sonntag, Michael Engel, and Lars Schmidt-Thieme.  Predicting parking availability from mobile payment transactions with positive unlabeled learning.  In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021, pages 15408–15415. AAAI Press, 2021.
  * [36] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan.  Deep hashing network for unsupervised domain adaptation.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5018–5027, 2017.
  * [37] Jing Wang, Jiahong Chen, Jianzhe Lin, Leonid Sigal, and Clarence W de Silva.  Discriminative feature alignment: Improving transferability of unsupervised domain adaptation by gaussian-guided latent alignment.  Pattern Recognition, page 107943, 2021.
  * [38] Yaqing Wang, Quanming Yao, James T. Kwok, and Lionel M. Ni.  Generalizing from a few examples: A survey on few-shot learning.  ACM Comput. Surv., 53(3):63:1–63:34, 2020.
  * [39] Ruijia Xu, Guanbin Li, Jihan Yang, and Liang Lin.  Larger norm more transferable: An adaptive feature norm approach for unsupervised domain adaptation.  In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 1426–1435. IEEE, 2019.
  * [40] Yuan Yao, Yu Zhang, Xutao Li, and Yunming Ye.  Heterogeneous domain adaptation via soft transfer network.  In Laurent Amsaleg, Benoit Huet, Martha A. Larson, Guillaume Gravier, Hayley Hung, Chong-Wah Ngo, and Wei Tsang Ooi, editors, Proceedings of the 27th ACM International Conference on Multimedia, MM 2019, Nice, France, October 21-25, 2019, pages 1578–1586. ACM, 2019.