# Uncertainty-guided Open-Set Source-Free Unsupervised Domain Adaptation with
Target-private Class Segregation

Mattia Litrico, Davide Talon, Sebastiano Battiato, Alessio Del Bue, Mario
Valerio Giuffrida, Pietro Morerio Mattia Litrico and Sebastiano Battiato are
with University of Catania, Italy.Davide Talon is with Fondazione Bruno
Kessler, Trento, Italy.Mario Valerio Giuffrida is with University of
Nottingham, UK.Pietro Morerio and Alessio Del Bue are with Pattern Analysis &
Computer Vision (PAVIS), Istituto Italiano di Tecnologia (IIT), Genova,
Italy.Primary email contact<EMAIL_ADDRESS>received
Month day, Year; revised Month Day, year.

###### Abstract

Standard Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from
a labeled source domain to an unlabeled target but usually requires
simultaneous access to both source and target data. Moreover, UDA approaches
commonly assume that source and target domains share the same labels space.
Yet, these two assumptions are hardly satisfied in real-world scenarios. This
paper considers the more challenging Source-Free Open-set Domain Adaptation
(SF-OSDA) setting, where both assumptions are dropped. We propose a novel
approach for SF-OSDA that exploits the granularity of target-private
categories by segregating their samples into multiple unknown classes.
Starting from an initial clustering-based assignment, our method progressively
improves the segregation of target-private samples by refining their pseudo-
labels with the guide of an uncertainty-based sample selection module.
Additionally, we propose a novel contrastive loss, named NL-InfoNCELoss, that,
integrating negative learning into self-supervised contrastive learning,
enhances the model robustness to noisy pseudo-labels. Extensive experiments on
benchmark datasets demonstrate the superiority of the proposed method over
existing approaches, establishing new state-of-the-art performance. Notably,
additional analyses show that our method is able to learn the underlying
semantics of novel classes, opening the possibility to perform novel class
discovery.

###### Index Terms:

Unsupervised Domain Adaptation, Source Free, Open Set, Novel class discovery,
Negative Learning, InfoNCE

##  1 Introduction

Deep learning methods achieve remarkable performance in visual tasks when
training and test data share a similar distribution. However, learning
approaches struggle to generalise in presence of _domain shift_ [1, 2], namely
data coming from different distributions, such as different illumination
conditions, imaging modalities or styles. To this end, Domain Adaptation (DA)
takes advantage of the knowledge learned on a _source domain_ to aid the
learning in a different but related _target domain_. As the considerable cost
of data collection and annotation limits the availability of large amounts of
target data for supervised adaptation of deep learning models, DA focuses on
leveraging unsupervised target data for knowledge transfer across domains. In
particular, Unsupervised Domain Adaptation (UDA) aims to transfer the
knowledge learned on a labelled source domain to an unseen target domain
without requiring any target label [3, 4, 5, 6, 7, 8, 9].

Most UDA techniques typically rely on two strong assumptions on the data. On
the one hand, they require simultaneous access to source and target data
during the adaptation, hindering the deployment in applications where data
privacy or transmission bandwidth become critical issues. To overcome this
issue, recent works focused on the setting of Source-free Domain Adaptation
(SF-DA) [10, 11, 12, 13, 14], where source data is no longer accessible during
the adaptation phase, but only unlabelled target data is available. On the
other hand, most UDA algorithms assume a _closed-set_ setting, i.e. the two
domains share the same class space. Despite the adaptation to the new domain,
the transferred source model has no capability to make predictions for
novel/unseen classes. This severely limits their applicability in real-world
scenarios [15, 16], where the target domain might contain unknown classes that
are not present in the source data.

To address this issue, Open-set Domain Adaptation (OSDA) [17, 18, 19, 15, 16]
aims to empower the adapting model with the ability to recognise novel-class
samples in the target domain as unknown. In this scenario, the target dataset
contains both classes already seen in the source domain (shared classes), as
well as novel classes that are not present in the source (private classes). In
[19, 15], the authors used adversarial learning to align the source domain
with the portion of the target domain sharing the same classes, excluding the
target private classes, leading to a poor decision boundary. To avoid this
issue, Jang et al. [18] included the target private classes in the adversarial
learning to align the source domain with the shared classes, while segregating
private samples in the unknown classes. However, all these previous works are
not source-free, as they learn from source and target data simultaneously.

The Source-Free Open-set Domain Adaptation (SF-OSDA) addresses both
challenges, assuming that the adaptation process has no access to source data
in presence of private classes in the target domain. While this joint setting
is underexplored, some recent approaches [13, 20, 21] have been proposed.
However, these approaches consider samples from target-private classes as
outliers and cluster them into a single “unknown” class. By gathering in a
single class semantically different samples, this yields a degradation of the
feature space. Additionally, to segregate samples of unknown classes, previous
approaches rely on a statistic-based separation criterion computed on the
outputs of the adapted model, often leading to the so called negative
transfer, a consequence of the well-known confirmation bias [22]. Namely, the
model relies on its noisy outputs for the adaptation process, increasing the
confidence in its wrong predictions (in essence, the model is overfitting its
own noise).

In this work, we propose a novel Source-free Open-set Domain Adaptation
approach that, leveraging the granularity of target-private classes,
segregates their samples into multiple clusters, resulting in a more effective
adaptation compared to previous works. As a by product, this strategy also
allows the discovery of the underlying semantics of novel classes. As an
extension of [10], our method produces pseudo-labels for all target samples
and progressively refines them using the consensus from neighbours samples.
However, differently from [10], the initial pseudo-labels assignment (which
also includes target private classes) is obtained by clustering the features
space spanned by the source model, rather than directly using its predictions.
A subset of the obtained clusters centroids is matched with the source model
prototypes computed from the classifier weights. The unmatched centroids are
treated as new prototypes of the target private classes and used to initialise
the columns of the classifier weights corresponding to the unknown classes.
Differently from the state-of-the-art, this strategy does not require any
separation criterion to distinguish between samples of shared and unknown
classes, limiting the negative effects of the confirmation bias.

The pseudo-labels produced during the adaptation are inevitably affected by
the so called shift noise [23], namely the noise caused by the domain shift,
here further intensified by the presence of novel classes in the target.
Moreover, the initial clustering-based assignment could also introduce some
noise, potentially leading to classify an entire shared class as unknown or
viceversa. To obtain robustness to such noise, we perform an uncertainty-based
samples selection to discard sample with unreliable (and possibly noisy)
pseudo-labels. To that end, we propose a novel uncertainty estimation
solution, which extends the one used in [10] that was not devised to handle
this scenario. The uncertainty of the pseudo-labels is measured by analysing
the consensus of neighbours samples predictions, and the relative distances of
samples with respect to the class prototypes. Such newly introduced strategy
assigns low uncertainty values when samples are close to one prototype, while
being far from the others. Contrarily, when the distance values are similar,
samples will have a high uncertainty and their pseudo-labels are considered
unreliable.

To obtain a regularised features space where neighbours samples are
semantically similar, we leverage a self-supervised contrastive learning
framework together with a negative pairs exclusion strategy. Differently from
[10], we introduce the NL-InfoNCELoss, a novel contrastive loss that
integrates the principles of negative learning in the standard contrastive
loss [24]. By empowering the standard loss with the well known benefits of the
negative learning paradigm [25, 26], this formulation increases the robustness
of the contrastive framework to the noise inevitably affecting the pseudo-
labels.

Extensive experiments demonstrate the effectiveness of our method in the
challenging setting of SF-OSDA, outperforming the state-of-the-art on two
major benchmark datasets. Specifically, on Office31 [27] and Office-Home [28],
we set the new state-of-the-art by improving the performance of $+2.1$ and
$+0.1$ HOS, respectively. Ablation studies show the effectiveness of the new
components in handling the issues introduced by the presence of novel classes
in the target domain. Notably, additional analyses show the ability of our
model to classify samples from target-private classes, meaning the model
learned the underlying semantics of novel classes without being explicitly
optimised for that task.

In summary, the main contributions of this work are the following:

  * •

We tackle the Source-Free Open-Set UDA problem by exploiting the granularity
of target-private classes via an uncertainty-guided pseudo-labelling. As a
byproduct, we explicitly learn the underlying semantics of target-private
classes, unveiling the possibility of performing novel class discovery;

  * •

We introduce NL-InfoNCELoss, a novel contrastive loss that integrates the
negative learning paradigm into self-supervised contrastive learning to
regularise the feature space, while increasing the robustness to pseudo-label
noise;

  * •

We evaluate the proposed methodology with extensive experiments proving the
superiority against existing approaches on two major benchmark datasets.

The rest of this paper is organised as follows: Section 2 introduces and
discusses the relevant literature related to Open-Set domain adaptation,
pseudo-labelling, and learning from noisy labels. Section 3 formalises the
Open-Set Source-Free UDA setting and presents the proposed methodology.
Section 4 presents experiments and additional analyses to assess our approach.
Eventually, Section 5 concludes the paper with a summary and a discussion of
major findings.

##  2 Related Work

Open-set and Source-free UDA. The assumption that both source and target
domains share the same label space may not hold in several real-world
scenarios, where the target domain can contain classes that are not included
in the source domain. In the literature, such classes are referred to as
target private (or simply private) or unknown. Open-set domain adaptation aims
to transfer knowledge from a source domain to a target domain containing
private classes. Recent works mainly focus on separating the known and unknown
samples in the target domain [17, 18, 19, 15] or aligning the known
distributions [19, 29]. Differently, [30] proposed to use word-prototypes
based on mid-level features, which are more robust to the negative transfer to
perform a subsidiary prototype-space alignment. UADAL [18] proposed to
segregate the unknown classes leveraging a 3-way adversarial learning among
the source, the shared, and unknown classes of the target. However, all these
methods assume the availability of source data during the adaptation, i.e.
they are not source-free.

More recently, [31, 32, 33] proposed source-free strategies for the open-set
domain adaptation setting. However, these approaches rely on ad-hoc pre-
training strategies on the source domain that assume the awareness of the
presence of unknown classes in the target during the pretraining. For example,
OneRing [31] pretrained a classifier on the source data with an extra class to
empower the model to better segregate target-private samples during
adaptation. In [32, 33], the authors leverage a weighting scheme and feature-
splicing to avoid negative transfer, but they generate extra training samples
of unknown categories during pretraining on the source domain, facilitating
the separation of private classes in the target domain. Differently, our
method makes no ad-hoc assumptions on the training in the source domain,
providing a more general approach to this challenging setting, widening its
application.

Other methods have been proposed to avoid such ad-hoc training strategies.
SHOT [13] and AaD [20] use a separation criterion based on the entropy of
predictions, to distinguish between shared and unknown classes. Instead, USD
[21] proposed to segregate private samples based on Jenson-Shannon divergence.
However, since these separation criteria are computed on model outputs, they
are easily affected by confirmation bias. Differently, we do not use any
explicit separation criterion, by relying on a clustering-based strategy to
initially assign samples to classes, either shared or unknown. Assuming that
such assigned labels are noisy, we refine the assignment via pseudo-labelling
and self-supervision strategies.

Self-supervised Learning and Pseudo-Labelling. Self-supervised methods are
successful in learning transferable representations of visual data [34, 35,
36, 37, 38, 39, 40, 41, 42]. Specifically, [35, 37, 39] use contrastive-based
pretext tasks to enhance the generalisation ability of deep models. Moreover,
several recent self-supervised approaches have been utilised within UDA [43,
44] and SF-UDA [11, 45, 24, 46] settings. On the other hand, pseudo-labelling
is a simple but effective technique used in semi-supervised [47, 48] and self-
supervised learning [34], as well as in domain adaptation [13, 49, 45, 24]. It
consists in using labels predicted by the model as self-supervision. Fix-Match
[48] and On-target [45] take advantage of pseudo-labels without any refinement
during training. In this paper, we perform refinement steps to progressively
improve the quality of the generated pseudo-labels.

Noisy Labels. Noisy training sets often result in poorly trained models. In
[50], the authors demonstrated that a deep neural network can easily overfit
an entire dataset with any ratio of corrupted labels, resulting in a poor
generalisation on test data. To address this problem, different approaches
have been proposed focusing on noise-robust losses [51, 52, 53, 54],
estimation of the noise-transition matrix [55], reweighting of the loss based
on the reliability of samples [56, 57, 58], as well as the selection of clean
data from noisy samples [59, 60]. In [59, 60], the authors proposed sample
selection strategies based on the loss, but they require multiple deep neural
networks to make the selection more robust, leading to an increase of the
computational cost of the method. Furthermore, the sample selection is based
on the loss, which is computed with noisy labels that are not refined during
the training. This implies that the amount of noise in the labels will not be
reduced and the selection will be highly affected by the noise. Instead, we
propose a sample selection strategy based on the uncertainty (reliability) of
pseudo-labels, which is estimated after their refinement using the consensus
among neighbours, without requiring the use of multiple deep networks.
Differently from [10], which used a reweighting strategy on target samples, we
perform a hard sample selection to account for the increased amount of noise
produced by the unknown classes. Lastly, NEL [26] combined a negative learning
loss with a pseudo-labels refinement framework based on ensembling. Negative
learning [54] refers to an indirect learning method which uses complementary
labels to combat noise. While we also use a negative learning loss, we do not
require an ensemble of networks to perform refinement, which results in a
reduced computational cost. Additionally, we leverage the paradigm of negative
learning to also propose a novel contrastive loss empowered with more
robustness to the noise.

Summary. This paper extends our previous work [10], which proposed a
methodology for source-free domain adaptation in a closed-set scenario.
Differently than [10], here we focus on the open-set setting, where the target
domain contains novel classes that are not present in the source. As in [10],
we train the target model with generated pseudo-labels that are progressively
refined during the adaptation. However, the initial pseudo-labels assignment
is obtained by performing a clustering on the target feature space, instead of
directly leveraging the source model. Moreover, we propose a novel uncertainty
measure devised to limit the impact of the additional noise in the pseudo-
labels produced by the presence of unknown classes in the target domain. In
[10], we reweighted the contribution of the pseudo-labels in the loss based on
their uncertainty. Here we introduce an uncertainty measure that is combined
with the one proposed in [10], to select (instead of reweight) samples with
reliable pseudo-labels, which improves the robustness to the noise in both
shared and private classes. Finally, we propose a novel contrastive loss that
incorporates the principles of negative learning in the contrastive framework
of [10], to further improve the robustness of the model against the noise.

##  3 Proposed Method

Figure 1: Overview of the proposed adaptation approach. (a) Target samples are
clustered based on the features extracted from the pretrained source model, to
provide an initial pseudo-labels assignment. Next, clustering allows the
identification of class centroids (coloured shapes), which are matched against
the most similar prototypes of shared classes. Such prototypes are taken as
the columns of weight matrix $W$ of the classifier (blue clusters). Since the
target domain has more classes than the source domain, some clusters are left
out from this matching, which will be treated as target-private classes
(yellow clusters). (b) After the refining of the pseudo-label, its uncertainty
is estimated using two approaches. Neighbours consensus: the uncertainty is
determined by analysing the consensus of neighbours on the refined pseudo-
label. Class separation: the model uncertainty is estimated based on the
distance w.r.t. the two closest class prototypes. A novel contrastive loss
(NL-InfoNCELoss) gathers same-class samples in the features space, while being
robust to noisy pseudo-labels.

This section presents our approach to the SF-OSDA problem in classification
tasks, which is depicted in Fig. 1. Our model is firstly trained on a source
domain without using any ad-hoc training strategies, as done in [31, 32, 33].
Then, at the beginning of the adaptation process, the pretrained source model
extracts the features for each of the unlabelled target samples. A clustering-
based strategy on target features roughly detects samples from private classes
and provides an initial pseudo-labels assignment on the target samples
(Section 3.2). Due to the domain shift between source and target domains and
the presence of unknown class samples, this initial assignment is unreliable.
Therefore, we mitigate the impact of the noise in the pseudo-labels by
refining them and selecting reliable samples for the training. On the one
hand, the nearest neighbors voting scheme in Section 3.3 progressively refines
the pseudo-labels according to a majority rule. On the other, in Section 3.4
we select reliable pseudo-labels with low uncertainty, which are then used in
the adaptation process. The contrastive scheme proposed in Section 3.6
enforces the underlying assumption of the pseudo-labels refinement process:
samples of the same class should be close in the target feature space. In
Section 3.5, we leverage the pseudo-labels to adapt the model to the target
domain through a negative learning classification loss. In Section 3.7, we
provide the overall training objective that progressively refines the noisy
pseudo-labels and segregates target-private samples, while aligning the
pretrained model to the target domain.

Problem formulation. Let $\mathcal{D}_{s}$ be the labelled source dataset
including pairs $\\{x_{s},y_{s}\\}$, where $x_{s}\in\mathcal{X}_{s}$ and
$y_{s}\in\mathcal{Y}_{s}$ are images and ground-truth labels, respectively.
Due to the source-free constraint, the source data $\mathcal{D}_{s}$ is not
available during adaptation. Similarly, let $\mathcal{D}_{t}$ be the target
data composed of images $\\{x_{t}\\}$ only, with $x_{t}\in\mathcal{X}_{t}$.
The open-set scenario assumes that $\mathcal{Y}_{s}\subset\mathcal{Y}_{t}$,
where $C_{S}:=\mathcal{Y}_{s}\cap\mathcal{Y}_{t}$ and
$C_{P}:=\mathcal{Y}_{t}\setminus\mathcal{Y}_{s}$ are the set of shared and
private classes, respectively. The set of all classes $C$ is given by
$C=C_{S}\cup C_{P}$. The source model $g_{s}(\cdot)=h_{s}(\varphi_{s}(\cdot))$
is composed of a features extractor
$\varphi_{s}:\mathcal{X}\rightarrow\mathbb{R}^{D}$ and a classifier
$h_{s}:\mathbb{R}^{D}\rightarrow\mathbb{R}^{|C_{S}|}$, where $D$ is the size
of the features space. Without loss of generality, we assume that the weights
of the classifier are characterised by a matrix
$W\in\mathbb{R}^{D\times|C_{S}|}$. We will refer to each column $i$ of the
weight matrix $W[i]$ as a shared class prototype.

###  3.1 Leveraging the granularity of target-private classes

Open-set source-free domain adaptation considers a two-steps approach. In the
first stage, a model is trained on large labelled source data. Without further
access to the source samples, the second step adapts the source model to the
unlabelled target domain. During the (pre-)training on the source dataset, the
classifier $h_{s}$ is trained to accommodate the classes in $C_{S}$, i.e. the
shared classes. The target domain contains not only the classes in $C_{S}$,
but also includes a set of private classes $C_{P}$.

Previous works usually train models to segregate samples from target-private
classes into a single “unknown” class, which is more affine to outlier-
detection. While samples from shared classes are separated with respect to
their semantics, this strategy forces the model to aggregate samples from
target-private classes into a single class, independently from their
semantics. This affects the learned feature space, leading to a suboptimal
geometry, where samples with possibly very different semantics are grouped
together. Moreover, this strategy hinders the possibility to discover the
underlying semantics of novel classes. To solve this issue, we design our
model to leverage the granularity of target-private classes by segregating
their samples into multiple unknown classes. This choice has the side effect
of learning a features space where samples from private classes are aggregated
based on their semantics, as discussed in Section 4.2. Hence, before the
adaptation process, we craft a new classifier $h_{t}$ whose weight matrix is
made of two parts, such that $W_{T}=[W_{S}|W_{P}]$. The set of weights $W_{S}$
is initialised by taking the pretrained weights from $h_{s}$, whereas $W_{P}$
is randomly initialised with values sampled from a uniform distribution. This
means that we augment the pretrained classifier by adding a certain number of
columns matching a plausible number of private classes $|\widehat{C}_{P}|$ in
the target domain. Note that $|\widehat{C}_{P}|$ is arbitrarily chosen, since
the real cardinality of the target private classes is unavailable under the
open-set setting. In Section 4.2, we analyse the performance with different
values for $|\widehat{C}_{P}|$. Using the extended classifier $h_{t}$, we use
the model $h_{t}(\varphi_{s}(\cdot))$ for an initial pseudo-label prediction
in the target dataset $\mathcal{X}_{t}$ (Fig. 2(a)).

###  3.2 Clustering-based Target Model Initialisation

Figure 2: The extended source model $h_{t}$ incorrectly classifies samples
from private classes as belonging to shared ones. Nonetheless, samples from
both shared and private classes exhibit a low intra-class variability.
Building on this observation, we cluster the feature space to perform an
initial pseudo-labels assignment on the target domain. The figure shows the
initial identification of target-private samples using clustering. In (a) and
(b) coloured samples belong to target-private classes, while gray samples
belongs to shared classes. The colour of samples represents the predicted
class. For visualisation purposes, all samples predicted in a target-private
class are represented in the blue class. The $\bullet$ symbol represents
correct predictions, whereas the $\times$ represents incorrect predictions.
While the source model cannot correctly assign target-private samples, the
clustering leverages the structure in the features space to aggregate private
samples and provides a good pseudo-labels initialisation. In fact, after the
clustering, more target-private samples are correctly classified in the blue
class. Nonetheless, pseudo-labels still contain noise, which is progressively
reduced during training.

Since the pretrained model has never seen any of the private classes $C_{P}$
and the classifier $h_{t}$ is randomly initialised for its extended portion,
most of the samples from private classes will be predicted into the shared
classes. This will produce a certain amount of incorrect pseudo-labels, as
shown in Fig. 2(a). To alleviate this problem, we propose a clustering-based
solution to discover suitable prototypes for the private classes $C_{P}$, and
thus initialize $W_{P}$ in a more convenient way. This will produce a better
initial pseudo-labels assignment. The idea of using a clustering approach
builds on the observation that the features of target samples (both from
shared and private classes) are characterised by a low intra-class variability
(c.f. Fig. 2(a)). Therefore, a standard unsupervised clustering algorithm can
, to some extent, group both shared and private target samples, providing the
model with an improved initial pseudo-labeling. Differently from other
approaches [18, 19, 20], this strategy avoids to rely on a separation
criterion that hardly distinguishes whether samples belong to shared or
target-private classes. However, a major drawback in utilising an unsupervised
clustering algorithm is the class misalignment, as there is no correspondence
between the clusters and the model classes and one needs to find the correct
permutation which maps cluster ids to class ids.111In the classic example of
cat vs. dog classification, there is no guarantee that a clustering algorithm
would assign cluster 1 to the class cat and cluster 2 to the class dog. As
they can be easily inverted, such class misalignment becomes more exasperate
in the presence of more than 2 classes. To alleviate this problem, we leverage
the pretrained source model $g_{s}$ to align the shared classes $C_{S}$ with a
subset of clusters obtained by the clustering algorithm. The remaining
unmatched clusters are treated as unknown classes and their centroids as novel
prototypes for the initialization of $W_{P}$, as depicted in Fig. 2.

More formally, we extract the features from the target samples
$z_{t}=\varphi_{s}(x_{t}),\;\forall x_{t}\in\mathcal{X}_{t}$ using the
pretrained source feature extractor $\varphi_{s}$. At this point, we perform
an unsupervised clustering over all the extracted features $z_{t}$ to obtain
$K$ centroids
$c_{k}=\frac{1}{|\mathcal{S}_{k}|}\sum_{a\in\mathcal{S}_{k}}z_{t}^{a}$, where
$k\in\\{1,2,\ldots,K\\}$, $K=|C_{S}|+|\widehat{C}_{P}|$, and $\mathcal{S}_{k}$
is the set of target samples assigned to the $k$-th cluster. Note that while
$|\widehat{C}_{S}|$ is known, $|\widehat{C}_{P}|$ is arbitrarily chosen, since
the real cardinality of the target private classes is unavailable under the
open-set setting. In Sec. 4.2, we analyse the performance with different
values for $|\widehat{C}_{P}|$ (which implies in different $K$).

To account for the class misalignment, we align the shared classes with a
subset of the obtained clusters by computing the cosine similarity between the
shared class prototypes $W_{S}[i]$ and the cluster centroids $c_{k}$, and we
match each column in $W_{S}$ with the centroid with the highest similarity.
This strategy solves the class misalignment by determining a permutation that
reorders the centroids $c_{k}$ according to their match with shared class
prototypes. Finally, centroids that have not been assigned to any shared
classes prototypes will be considered as discovered prototypes of the target-
private classes, and used to fill the columns of $W_{P}$ (here the order of
the clusters does not matter). The refined model $W_{T}=[W_{S}|W_{P}]$ is then
used to produce an initial pseudo-labels assignment, as detailed in the next
section.

###  3.3 Pseudo-Label Refinement with Neighbours Consensus

Similar to [10, 24], the refinement of the pseudo-labels is accomplished by
aggregating knowledge from nearest neighbour samples. The underlying idea is
that similar samples are likely to belong to the same class. We thus assume
that features extracted from semantically similar samples lie close to each
other in the feature space and we further enforce this assumption by means of
a contrastive learning loss, as described in Sec. 3.6.

More formally, given a target sample $x_{t}$ and a random weak augmentation
$t_{wa}$ drawn from a distribution $\mathcal{T}_{wa}$, we obtain a feature
vector $z_{t}=\varphi_{t}(t_{wa}(x_{t}))$ from the augmented image, which is
used to search the neighbours of $x_{t}$ in the target features space [10].
The pseudo-label of $x_{t}$ is refined by aggregating predictions from the
neighbours by soft-voting [61] as follows:

$\bar{p}_{t}^{(c)}=\frac{1}{|\mathcal{I}|}\sum_{i\in\mathcal{I}}p^{\prime(c)}_{i},$
(1)

where $\mathcal{I}$ is the set of indices of the selected neighbours,
$p^{\prime}$ is the softmax output, and the superscript $c$ indicates the
class index. To obtain a refined pseudo-label, we find the class $c$ with the
highest probability computed in Eq. 1, i.e.
$\bar{y}_{t}=\operatorname*{arg\,max}_{c}\;\bar{p}_{t}^{(c)}.$ The refined
pseudo-label is assigned to the sample $x_{t}$ and used as self-supervision
signal (see Sec. 3.7).

This refining process relies on a bank $\mathcal{B}$ of length
$M<|\mathcal{X}_{t}|$, storing pairs
$\\{z^{\prime}_{j},p_{j}^{\prime}\\}_{j=1}^{M}$ of features and softmax
predictions. The neighbours of a sample $x_{t}$ are then selected by computing
the cosine distances between the features stored in the bank. The
$|\mathcal{I}|$ samples with the lowest distance are selected as neighbours.
Following [40, 10], we use a slowly changing momentum model
$g^{\prime}_{t}(\cdot):=\text{EMA}[h_{t}(\varphi_{t}(\cdot))]$ to update
features $z^{\prime}$ and predictions $p^{\prime}$, in order to maintain the
information stored in the bank stable during adaptation. A bank composed of
$M$ randomly selected samples reduces the number of pairwise comparisons
needed for finding the neighbours.

Bank initialisation with discovered classes prototypes. As described above,
the refinement of the pseudo-labels is performed by averaging neighbours
probabilities stored in $\mathcal{B}$. In order to generate the initial
pseudo-labels that reflect the clusters obtained in Sec. 3.2, we need to
initialise the bank accordingly. However, the clustering strategy we use in
Sec. 3.2 produces class assignments for all samples $x_{t}$ rather than
probabilities. To overcome this issue, after the clustering initialisation, we
associate to each target sample a vector of probabilities proportional to its
similarity to the centroids $c_{k}$. Given target features $z_{t}$ and
centroids $c_{k}$, we determined the probabilities vectors $p^{\prime}$ as the
per-class similarity between $z_{t}$ and each centroid in $c_{k}$. Formally,
for each target sample, we obtain the probabilities vector $p^{\prime}$ to
initialise the bank as follows:

$p^{\prime}=1-\frac{CosDist(z_{t},c_{k})}{\underset{j\in\\{1,...,|\hat{C}_{P}|\\}}{\max}CosDist(z_{t},c_{k}^{j})}$
(2)

where $CosDist(z_{t},c_{k})=1-\frac{z_{t}\cdot c_{k}}{||z_{t}||\;||c_{k}||}$
is the cosine distance and $i$ is the class index. Finally, to obtain
probabilities, the vector $p^{\prime}$ is provided to the softmax function
$\sigma$ with a temperature parameter $\tau_{2}$,
$p^{\prime}=\sigma(p^{\prime}/\tau_{2})$. Consequently, for all samples
$x_{t}$ the probability value for class $c$ will be proportional to the
similarity between the features $z_{t}=\varphi_{t}(x_{t})$ and the centroid of
$c$. Also, the class with the highest probability will be exactly the one
assigned to the sample by the clustering strategy in Sec. 3.2.

###  3.4 Pseudo-labels Uncertainty in the Open-set Scenarios

The refined pseudo-labels $\bar{y}_{t}$ are used as a self-supervision signal
for a classification loss on target data, as detailed in Sec. 3.7. However,
such pseudo-labels are far from perfect, especially at the beginning of the
iterative refining process, and thus still contain noise. Therefore, using all
of them in the classification loss will disrupt adaptation, since the model
would be trained with incorrect information. To mitigate this, we perform a
sample selection, in order to pick only samples with low uncertainty: these
are selected stochastically via Bernoulli sampling, based on the uncertainty
of their pseudo-label, estimated in a twofold manner, as detailed below.

Uncertainty estimation via Neighbours Consensus. The first uncertainty
estimation method is as our previous work [10]. It estimates the uncertainty
of pseudo-labels (after their refinement) by measuring the consensus among
neighbours’ predictions. This strategy builds upon the intuition that if the
neighbourhood agrees on a predicted class, the derived pseudo-labels should be
considered reliable (low uncertainty). Otherwise, if the network predicts
different classes for the neighbour samples, the obtained pseudo-labels should
be considered unreliable (high uncertainty). To this end, we compute the
entropy $\mathcal{H}$ of $\bar{p}_{t}$ (Eq. 1) as an estimator of pseudo-label
uncertainty, noting that $\mathcal{H}$ yields low values in low uncertainty
cases, and high values otherwise. We define the uncertainty coefficient
$u^{nc}_{t}$ as follows:

$u^{nc}_{t}=\frac{\mathcal{H}(\bar{p}_{t})}{\log_{2}\,|C|}.$ (3)

While the uncertainty coefficient $u^{nc}_{t}$ reduces the impact of the noisy
pseudo-labels in a closed-set scenario [10], we noted that it can fail to
model uncertainty in open-set scenario. In fact, during the initial clustering
assignment, a cluster made of target-private samples can be incorrectly
assigned to any of the shared classes. This is mainly due to the fact that the
model has not been optimised to separate shared and target-private classes
during the pretraining on the source, implying that shared and target-private
clusters can lie very close in the feature space. In this scenario, all the
samples of the clusters will share the same incorrect pseudo-label, making the
uncertainty estimation unsuccessful.

Uncertainty estimation via Class Separation. To tackle this problem, we
propose a novel strategy to estimate the uncertainty of pseudo-labels that is
well-suited in a open-set scenario, by analysing the distance of samples to
class prototypes, as depicted in Figure 3. We build upon the intuition that
when two classes are close in the features space, the distance of the samples
from the two classes prototypes will be similar. Contrarily, when those
classes are clearly separated, the samples will be close to one prototype and
far from the other. Following this intuition, we consider the former and the
latter scenarios of high and low uncertainty, respectively. For instance, in a
situation where a cluster of target-private samples is so close to a shared
class that can be switched, the distances of samples to prototypes of the two
classes will be similar, resulting in high uncertainty. More formally, for a
target sample $x_{t}$ we select from $\bar{p}_{t}$ the indexes $\\{i,j\\}$ of
the two classes with the largest values as the most probable pseudo-labels for
the sample. Then we compute the cosine distance between the features of the
target sample $z_{t}=\varphi_{t}(x_{t})$ and the prototypes $W[i]$ and $W[j]$.
Finally, we obtain the uncertainty value $u^{cs}_{t}$ as follows:

$u^{cs}_{t}=\frac{CosDist(z_{t},W[i])}{CosDist(z_{t},W[i])+CosDist(z_{t},W[j])}$
(4)

Figure 3:  Pseudo-label uncertainty via Class Separation. The uncertainty of
the pseudo-labels is estimated by computing the distance of samples from the
two closest class prototypes. If a sample is similarly distant in the features
space from two classes, its pseudo-label has high uncertainty. Contrarily, if
a sample is close to one class and far from the other, its pseudo-label has
low uncertainty and we consider it as reliable.

Bernoulli Sampling. To improve the model robustness to the noisy pseudo-labels
for samples of both shared and target-private classes, we combine these two
uncertainty estimation strategies for selecting samples with the lowest
uncertainty to be used in the classification loss, as detailed in Sec. 3.7.
Specifically, we compute the probabilities $w^{nc}_{t}$ and $w^{cs}_{t}$ for a
target sample to have a correct pseudo-label with respect to both $u^{nc}_{t}$
and $u^{cs}_{t}$, respectively:

$w^{nc}_{t}=\mathcal{F}(u^{nc}_{t})$ (5) $w^{cs}_{t}=\mathcal{F}(u^{cs}_{t})$
(6)

where $\mathcal{F}$ is a monotonically decreasing function to assign high
probabilities to samples with low uncertainty and viceversa. In Sec. 4.2, we
analyse two choices for $\mathcal{F}$.

We select low uncertainty samples via Bernoulli sampling with probabilities
$w^{nc}_{t}$ and $w^{cs}_{t}$, which are combined as follows:

$M_{i}=\operatorname{B}(w^{nc}_{t})\oplus\operatorname{B}(w^{cs}_{t})$ (7)

where $\operatorname{B}(p)\in\left\\{0,1\right\\}$ is the Bernoulli sampling
with probability of success $0\leq p\leq 1$ and $\oplus$ is a logical
operator. In Sec. 4.2 we compare two choices for the operator $\oplus$.

Finally, we define $\mathcal{U}=\\{x_{t}^{i}\in\mathcal{X}_{t}|M_{i}=1\\}$ as
the set of uncertainty-based selected target samples. This set of samples is
used during the adaptation process in the classification loss, as detailed in
the next section.

Figure 4: A couple of target images would be wrongly considered as a negative
pair if only comparing the latest predictions (green box). Instead, since
$x_{t}^{i}$ and $x_{t}^{j}$ share the same pseudo-labels (at least once) in
the past T epochs, i.e. $\\{e-T,...,e-1\\}$, we exclude them from the list of
negative pairs. Figure from [10], reproduced with the permission of the
authors.

###  3.5 Adaptation with Refined Pseudo-labels

The refined pseudo-labels obtained with neighbours’ knowledge aggregation are
used to compute a classification loss on target data to adapt the model to the
new domain. We use the refined pseudo-labels $\bar{y}_{t}$ obtained from a
weakly-augmented image $t_{\text{wa}}(x_{t})$ as self-supervision for a
strongly-augmented version $t_{\text{sa}}(x_{t})$. The refining of the pseudo-
labels is an iterative process, progressively improving the pseudo-labels
accuracy during training through several refining steps.

As in our previous work [10], in addition to the proposed sample selection and
exclusion strategies, to mitigate the effect of the noisy pseudo-labels, we
use the negative learning loss [54, 25] as classification loss. Differently
from [54, 25], which use the negative learning loss concurrently or
alternating with a positive loss, here we do not use a positive loss in the
entire training. The used classification loss is the following:

$L_{t}^{cls}=-\;\mathbb{E}_{x_{t}\in\mathcal{U}}\Bigr{[}\sum_{c=1}^{C}\tilde{y}^{c}\,log\,(1-p_{\text{sa}}^{c})\Bigr{]},$
(8)

where $\tilde{y}^{c}$ is a complementary label
$\tilde{y}\in\\{1,\ldots,C\\}\>\backslash\>\\{\bar{y}_{t}\\}$ chosen randomly
from the set of labels and without the refined pseudo-labels,
$p_{\text{sa}}=\sigma(h_{t}(\varphi_{t}(t_{\text{sa}}(x_{t}))))$ is the
probabilistic output for the strongly-augmented image $t_{\text{sa}}(x_{t})$,
and $\mathcal{U}$ is the set of samples with low uncertainty, as detailed in
Section 3.4. The random selection of the complementary label $\tilde{y}$ is
coherent with the negative learning framework [54].

TABLE I: HOS score (%) on Office31. All methods use the ResNet-50 backbone.
The proposed approach outperforms the previous SF-UDA state-of-the-art by 2.1%
on average (Avg.), while being competitive with UDA baselines. The underlined
results are the best within UDA methods, while the bold one are the best SF-
UDA approaches.

Office31 Method Source-free A $\rightarrow$ D A $\rightarrow$ W D
$\rightarrow$ A D $\rightarrow$ W W $\rightarrow$ A W $\rightarrow$ D Average
OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS OS*
UNK HOS STA [19] ✗ 91.0 63.9 75.0 86.7 67.6 75.9 83.1 65.9 73.2 94.1 55.5 69.8
66.2 68.0 66.1 84.9 67.8 75.2 84.3 64.8 72.6 OSBP [15] ✗ 90.5 75.5 82.4 86.8
79.2 82.7 76.1 72.3 75.1 97.7 96.7 97.2 73.0 74.4 73.7 99.1 84.2 91.1 87.2
80.4 83.7 UAN [62] ✗ 95.6 24.4 38.9 95.5 31.0 46.8 93.5 53.4 68.0 99.8 52.5
68.8 94.1 38.8 54.9 81.5 41.4 53.0 93.4 40.3 55.1 OSLPP [29] ✗ 92.6 90.4 91.5
89.5 88.4 89.0 82.1 76.6 79.3 96.9 88.0 92.3 78.9 78.5 78.7 95.8 91.5 93.6
89.3 85.6 87.4 ROS [17] ✗ 87.5 77.8 82.4 88.4 76.7 82.1 74.8 81.2 77.9 99.3
93.0 96.0 69.7 86.6 77.2 100.0 99.4 99.7 86.6 85.8 85.9 DANCE [44] ✗ 94.3 50.7
66.9 92.8 55.9 70.7 97.0 66.8 80.0 97.6 73.7 84.8 82.4 85.3 65.8 81.6 60.6
70.2 91.0 60.2 73.1 cUADAL [18] ✗ 86.4 95.1 90.1 86.0 90.4 87.9 98.6 97.7 98.2
99.3 99.4 99.4 75.4 87.8 80.5 67.6 87.8 75.1 85.6 93.0 88.5 ANNA [63] ✗ 93.2
76.1 83.8 82.8 88.4 85.5 75.4 91.1 82.5 99.4 99.6 99.5 76.0 87.9 81.6 100.0
96.8 98.4 87.8 90.0 88.6 SHOT [13] ✓ 94.0 46.3 62.0 95.6 42.3 58.7 83.3 39.1
53.3 100.0 75.7 86.1 82.7 46.6 59.6 100.0 69.7 82.1 92.6 53.3 67.0 AaD [20] ✓
73.0 84.6 78.3 63.5 89.5 74.3 63.6 88.9 74.2 78.0 98.5 87.0 61.9 88.9 73.0
94.6 96.8 95.7 72.4 91.2 80.4 USD [21] ✓ 90.7 73.4 81.2 82.8 72.7 77.9 65.7
84.4 73.9 97.9 96.6 97.3 64.6 86.7 74.0 98.0 92.6 95.2 83.3 84.4 83.3 Ours ✓
85.7 93.0 89.2 90.9 77.8 83.9 73.3 83.4 78.0 89.9 95.7 93.2 65.0 80.0 71.8
98.6 94.6 96.6 83.9 89.6 85.4

###  3.6 Contrastive Loss for Open-set Scenarios

As described in Sec. 3.3, the process of pseudo-labels refinement through
neighbours samples assumes that features extracted from same-class samples lie
closer in the features space, compared to samples from other classes. We
encourage this assumption during adaptation via self-supervised contrastive
learning. We build on MoCo [40] to aggregate features from different
augmentations of the same image and separate representations originated from
different samples (negative pairs). Specifically, for each sample $x_{t}$, we
generate two strongly-augmented samples $t_{sa}(x_{t})$ and
$t^{\prime}_{sa}(x_{t})$, where
$t_{sa},t^{\prime}_{sa}\in\mathcal{T}_{\text{sa}}$ are two randomly sampled
transformations from the space of strong augmentations $\mathcal{T}_{sa}$. To
build the constrastive pairs in the feature space, the feature extractor
embeds augmented samples as query $q=\varphi_{t}(t_{sa}(x_{t}))$ and key
$k=\varphi_{t}(t_{sa}(x_{t}))$. While we employ representations $q$ and $k$
associated to the same sample $x_{t}$ as positive pairs, we build negative
pairs by maintaining a queue $Q_{e}$ that stores key features
$\\{k^{(i)}\\}_{i=1}^{N}$ as detailed below.

MoCo [40] uses the pairs $(q,k)$ as positive and all the pairs
$\\{(q,k^{(i)})\\}_{i=1}^{N}$ as negative pairs by minimising and maximising
their cosine distance for positive and negative pairs, respectively. Since
features are stored in $Q_{e}$ independently from the class, even features
from samples belonging to the same class will be pushed away, which is in
contrast with our objective to better aggregate same-class samples in the
feature space. In [24], authors propose a strategy to exclude some negative
pairs from the contrastive loss. For every negative pair, they just compare
the pseudo-labels of the two samples. If they share the same pseudo-label,
then the negative pair is excluded; otherwise, the pair is included in the
negative pairs list. However, the exclusion strategy in [24] does not take
into account the noise that inevitably affects the pseudo-labels during
adaptation. Indeed, such noise may still lead to wrong negative pairs, i.e.,
pairs classified differently but having the same ground-truth label. This
causes instability of standard contrastive frameworks, highly affecting the
learned features space. Hence, we follow the temporal exclusion strategy in
[10], leveraging past predictions to identify and exclude pairs composed of
same class samples, even with noisy pseudo-labels. This exclusion strategy is
based on the intuition that, rather than only looking at the current pseudo-
label, it is more reliable to look at its past history to have a higher
probability of observing, at least once, the correct label: the history will
probably reveal the correct one, improving the exclusion process. To this end,
we build a temporal queue $Q_{e}$ by storing, for each sample, also the
refined pseudo-labels $\\{\bar{y}^{(j)}\\}_{j=1}^{\tau}$ of the $\tau$ past
epochs, i.e., $\\{e-\tau,\ldots,e-1\\}$, and we filter out all the pairs that
shared the same pseudo-labels at least once in the past $\tau$ epochs from the
set of available negative pairs, as illustrated in Figure 4. Although this
process will likely filter out a lot of true negatives, it ensures most of the
false negatives are excluded.

NL-InfoNCELoss. The refining of the pseudo-labels is an iterative process,
which implies that an amount of noise is still present in the pseudo-labels.
Moreover, the presence of target-private classes in the open-set setting,
which are hard to be correctly identified, increases such noise with respect
to the closed-set case. Inspired by [54] and differently from our previous
work [10], we propose a novel contrastive loss, called NL-InfoNCELoss, that
inserts the principle of negative learning in the standard InfoNCELoss.
Although the strategies proposed in Sec. 3.4 and Sec. 3.6 aim at reducing the
impact of such noise, we posit that leveraging the negative learning paradigm
can be beneficial for a contrastive loss to reduce the impact of false
negative samples.

Accordingly, the formulation of the proposed NL-InfoNCELoss is the following:

$L_{\text{NL-
InfoNCE}}=-\,log\left(1-\,\frac{exp(q\cdot{k}_{-}/\tau)}{\sum_{j\in\mathcal{N}_{q}}exp(q\cdot
k_{j}/\tau)}\right),$ (9)
$\mathcal{N}_{q}=\\{j|\bar{y}_{j}^{(i)}\neq\bar{y}^{(i)},\;\forall
j\in\\{1,\ldots,N\\},\forall i\in\\{1,\ldots,\tau\\}\\},$

where ${k}_{-}$ is one negative sample chosen randomly from the set of
negative samples, $\mathcal{N}_{q}$ is the set of indices of samples in
$Q_{e}$ that never shared with the query sample the same pseudo-labels in the
past $\tau$ epochs. By optimising the proposed NL-InfoNCELoss, the model is
trained to increase the feature distance between the query and a randomly
selected negative sample, rather than to all the negative samples, as in the
standard InfoNCELoss. Albeit this formulation slackens the separation effect
of the contrastive loss, it reduces the impact of noisy pseudo-labels in the
negative pairs.

In Sec. 4.2, we analyse the contribution of the proposed loss compared to the
standard InfoNCELoss.

###  3.7 Overall Framework

In addition to the classification and the contrastive losses detailed in the
previous sections, following the standard state-of-the-art protocol [10], we
add the following regularisation term to prevent the posterior collapse:

$L_{t}^{div}=\mathbb{E}_{x_{t}\in\mathcal{X}_{t}}\Bigr{[}\sum_{c=1}^{C}\bar{p}_{q}^{c}\,log\,\bar{p}_{q}^{c}\Bigl{]},$
$\bar{p}_{q}=\mathbb{E}_{x_{t}\in\mathcal{X}_{t}}\left[\sigma\left(h_{t}\left(\varphi_{t}\left(t_{sa}\left(x_{t}\right)\right)\right)\right)\right].$

When trained with noisy labels, the model may learn a degenerate latent
representation that causes to predict all the samples in a single class,
especially if the noise is very skewed towards a single category. By
encouraging diversity in model predictions, this term helps to avoid such
phenomenon.

The overall objective function used to adapt the model to the target data is
the following:

$L_{t}=\gamma_{1}L_{t}^{cls}+\gamma_{2}L_{t}^{ctr}+\gamma_{3}L_{t}^{div},$
(10)

TABLE II: HOS score (%) on Office-Home. All methods use the ResNet-50
backbone. The proposed approach achieves the highest accuracy on average
(Avg.) compared with SF-UDA baselines and outperforms several UDA methods. The
underlined results are the best within standard UDA methods, while bold ones
are the best SF-UDA approaches.

Office-Home Method Source-free Ar $\rightarrow$ Cl Ar $\rightarrow$ Pr Ar
$\rightarrow$ Rw Cl $\rightarrow$ Ar Cl $\rightarrow$ Pr Cl $\rightarrow$ Rw
OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS PGL
[64] ✗ 63.3 19.1 29.3 78.9 32.1 45.6 87.7 40.9 55.8 85.9 5.3 10.0 73.9 24.5
36.8 70.2 33.8 45.6 STA [19] ✗ 50.8 63.4 56.3 68.7 59.7 63.7 81.1 50.5 62.1
53.0 63.9 57.9 61.4 63.5 62.5 69.8 63.2 66.3 OSBP [15] ✗ 50.2 61.1 55.1 71.8
59.8 65.2 79.3 67.5 72.9 59.4 70.3 64.3 67.0 62.7 64.7 72.0 69.2 70.6 DAOD
[65] ✗ 72.6 51.8 60.5 55.3 57.9 56.6 78.2 62.6 69.5 59.1 61.7 60.4 70.8 52.6
60.4 77.8 57.0 65.8 OSLPP [29] ✗ 55.9 67.1 61.0 72.5 73.1 72.8 80.1 69.4 74.3
49.6 79.0 60.9 61.6 73.3 66.9 67.2 73.9 70.4 ROS [17] ✗ 50.6 74.1 60.1 68.4
70.3 69.3 75.8 77.2 76.5 53.6 65.5 58.9 59.8 71.6 65.2 65.3 72.2 68.6 DANCE
[44] ✗ 54.4 53.7 53.1 82.2 35.4 49.8 87.4 25.3 39.4 71.2 28.4 40.9 74.6 32.8
45.9 81.3 18.4 30.2 cUADAL [18] ✗ 55.8 75.6 63.6 69.6 73.9 71.6 81.8 73.3 77.5
54.9 82.0 65.0 61.8 77.4 68.3 69.5 76.3 72.6 ANNA [63] ✗ 61.4 78.7 69.0 68.3
79.9 73.7 74.1 79.7 76.8 58.0 73.1 64.7 64.2 73.6 68.6 66.9 80.2 73.0 SHOT
[13] ✓ 67.0 28.0 39.5 81.8 26.3 39.8 87.5 32.1 47.0 66.8 46.2 54.6 77.5 27.2
40.2 80.0 25.9 39.1 AaD [20] ✓ 50.7 66.4 57.6 64.6 69.4 66.9 73.1 66.9 69.9
48.2 81.1 60.5 59.5 63.5 61.4 67.4 68.3 67.8 USD [21] ✓ 53.3 71.5 61.1 65.7
74.9 70.0 73.3 79.5 76.3 52.2 70.8 60.1 62.4 68.4 65.2 69.3 68.6 68.9 Ours ✓
46.6 72.8 56.9 61.1 80.0 69.3 74.1 84.6 79.0 44.8 62.4 52.7 62.7 69.6 66.0
69.5 76.9 73.0

Method Pr $\rightarrow$ Ar Pr $\rightarrow$ Cl Pr $\rightarrow$ Rw Rw
$\rightarrow$ Ar Rw $\rightarrow$ Cl Rw $\rightarrow$ Pr Average OS* UNK HOS
OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS OS* UNK HOS PGL
[64] 73.7 34.7 47.2 59.2 38.4 46.6 84.8 27.6 41.6 81.5 6.1 11.4 68.8 0.0 0.0
84.8 38.0 52.5 76.1 25.0 35.2 STA [19] 54.2 72.4 61.9 44.2 67.1 53.2 76.2 64.3
69.5 67.5 66.7 67.1 49.9 61.1 54.5 77.1 55.4 64.5 61.8 63.3 61.1 OSBP [15]
59.1 68.1 63.2 44.5 66.3 53.2 76.2 71.7 73.9 66.1 67.3 66.7 48.0 63.0 54.5
76.3 68.6 72.3 64.1 66.3 64.7 DAOD [65] 71.3 50.5 59.1 58.4 42.8 49.4 81.8
50.6 62.5 66.7 43.3 52.5 60.0 36.6 45.5 84.1 34.7 49.1 69.6 50.2 57.6 OSLPP
[29] 54.6 76.2 63.6 53.1 67.1 59.3 77.0 71.2 74.0 60.8 75.0 67.2 54.4 64.3
59.0 78.4 70.8 74.4 63.8 71.7 67.0 ROS [17] 57.3 64.3 60.6 46.5 71.2 56.3 70.8
78.4 74.4 67.0 70.8 68.8 51.5 73.0 60.4 72.0 80.0 75.7 61.6 72.4 66.2 DANCE
[44] 69.7 43.9 54.2 48.9 67.4 55.7 84.2 27.1 41.2 76.8 16.7 27.5 59.4 41.3
48.3 84.1 29.6 44.0 72.8 35.0 44.2 cUADAL [18] 52.1 82.4 62.9 42.7 80.7 54.6
71.7 83.4 76.8 67.3 79.6 72.6 52.5 71.1 59.9 77.7 75.6 76.7 63.1 77.6 68.5
ANNA [63] 63.0 70.3 66.5 54.6 74.8 63.1 74.3 78.9 76.6 66.1 77.3 71.3 59.7
73.1 65.7 76.4 81.0 78.7 65.6 76.7 70.7 SHOT [13] 66.3 51.1 57.7 59.3 31.0
40.8 85.8 31.6 46.2 73.5 50.6 59.9 65.3 28.9 40.1 84.4 28.2 42.3 74.6 33.9
45.6 AaD [20] 47.3 82.4 60.1 45.4 72.8 55.9 68.4 72.8 70.6 54.5 79.0 64.6 49.0
69.6 57.5 69.7 70.6 70.1 58.2 71.9 63.6 USD [21] 54.3 73.8 62.6 47.3 69.6 56.3
70.0 74.5 72.2 64.6 71.3 67.8 53.8 65.5 59.1 73.3 69.1 71.1 61.6 71.5 65.9
Ours 52.3 63.7 57.4 65.3 80.7 72.2 66.8 78.6 72.3 51.4 77.4 61.8 50.3 74.6
60.1 69.7 73.1 71.4 59.5 74.5 66.0

##  4 Experiments and Results

Datasets. To evaluate our approach, we performed experiments on Office31 [27]
and Office-Home [28] for the open-set source-free domain adaptation setting.
– Office31 contains three domains, Amazon (A), Dslr (D), and Webcam (W), with
31 classes, where the first 10 classes are shared and the last 11 private
classes. The remaining 10 classes are not used in this setting.
– Office-Home consists of 65 classes of labeled images deriving from four
specific domains, Art (Ar), Clipart (Cl), Product (Pr), and Real World (Rw).
The first 25 categories in alphabetic order are shared classes and the
remaining 40 classes are private.
Evaluation metrics. Following the literature [63, 17, 19], we use three
evaluation metrics to compare the performance, including the average class
accuracy over known classes $\text{OS}^{\ast}$, the accuracy of unknown class
UNK, and
$\text{HOS}=2\,\frac{\text{OS}^{\ast}\times\text{UKN}}{\text{OS}^{\ast}+\text{UNK}}$
which is the harmonic mean between OS∗ and UNK. HOS is the core metric in the
latest open-set domain adaptation literature, since it requires a good balance
on both shared and private class accuracy in an unbiased manner.
Implementation details. We use standard classification architectures
comprising a feature extractor followed by a classifier. For fair comparison
purposes, we choose the ResNet-50 model [66] used by competitors as a backbone
in the experiments.

For source training, we initialise the ResNet backbone with ImageNet-1K [67]
pre-trained weights available in the Pytorch model zoo. We train the source
model with only a standard cross-entropy loss. For the adaptation stage, the
target model is initialised with the source model’s parameters. For open-set
experiments, before the adaptation, we increase the classifier size to
$2\cdot|C_{S}|$ to account for the unknown classes. The initial clustering is
performed with the K-means algorithm of Scikit-learn [68]. For more details,
the code will be made available upon paper acceptance.

###  4.1 Results

Office31. Table I compares our method with state-of-the-art UDA and SFDA
methods on the Office31 dataset under the open-set setting. For the UDA
setting, even though our method does not use source data at all during
adaptation, it achieves comparable performance with other methods. Note that
the availability of source data in the UDA setting is a considerable
advantage, given that they can be used for aligning domains. For the more
challenging SF-UDA setting, we achieve the highest HOS score by a notable
margin of $+2.1\%$, $+5.0\%$ on recent baselines, such as USD [21] and AaD
[20]. In addition, our method achieves the highest HOS score in four source-
target configurations, demonstrating to obtain a better balance between
$\text{OS}*$ and UNK. On the contrary, previous methods perform well on one
but highly degrade on the other. For example, SHOT often achieves the best
result on $\text{OS}*$, but performs poorly on UNK, yielding a lower HOS
score.

Office-Home. Table II shows the performance of our method w.r.t. state-of-the-
art UDA and SF-UDA methods on Office-Home under the open-set setting. Although
our method does not use source data during the adaptation, we outperform
several UDA methods, such as STA [19], OSBP [15], DAOD [65], and DANCE [44].
In addition, our method surpasses the more recent source-free approaches such
as AaD [20] and USD [21] with $+2.4\%$ and $+0.1\%$ in HOS, and $+2.6\%$ and
$+3.0\%$ in UNK comparison, demonstrating the robustness of our approach
across the board.

TABLE III: Ablation study of the components of the proposed method measured by
HOS score (%) on three sub tasks of Office31. First row excludes all the
components. Second row includes the clustering initialisation Sec. 3.2. Third
row and fourth row include the two strategies in Sec. 3.4 for estimating the
uncertainty of pseudo-labels, respectively. Last row introduces the proposed
NL-InfoNCELoss Sec. 3.6.

Clustering init. Neighbours Consensus Uncertainty Estimation Class Separation
Uncertainty Estimation NL-InfoNCELoss Avg. Acc. ✗ ✗ ✗ ✗ 35.3 ✓ ✗ ✗ ✗ 77.9 ✓ ✓
✗ ✗ 80.6 ✓ ✓ ✓ ✗ 82.4 ✓ ✓ ✓ ✓ 85.4

###  4.2 Analysis

Ablation study. Our method comprises several components to effectively adapt
to a target domain in the context of source-free adaptation. Table III shows
the benefit of these components: clustering initialisation (c.f. Sec. 3.2),
neighbours consensus and class separation uncertainty estimations (c.f. Sec.
3.4), and the proposed NL-InfoNCELoss (c.f. Sec. 3.6). Clustering
initialisation brings a major benefit to the training performance, doubling
the average accuracy. The benefit in performance of estimating uncertainties
is approx. $+2\%$ each. The best performance is achieved when also including
the NL-InfoNCELoss, with a performance gain of approx. $+3\%$.

Cardinality of target-private classes. As detailed in Sec. 3.2, the number of
the target-private classes $|C_{P}|$ is unknown during adaptation. We set the
expected number of target-private $|\widehat{C}_{P}|$ arbitrarily and we use
this value to both extend the size of the classifier output and perform the
initial clustering. Here, we study the performance of our method with
different values of $|\widehat{C}_{P}|$. Results of this experimentation are
shown in Figure 5. Particularly, the best performance is obtained when the
number of target private classes is equal to the cardinality of source
classes, i.e. $|C_{P}|=|C_{S}|$. Nonetheless, using even higher values for
$|C_{P}|$, our method achieves good performance obtaining an HOS score close
to the one achieved with the optimal value of $C_{P}$. Contrarily, the worst
performance is obtained using $C_{P}=1$. This results strengthen our
hypothesis that predicting all the samples from target-private classes in a
unique unknown class is suboptimal, since it requires aggregating semantically
different samples together. Following this, we design our method to segregate
these samples into multiple unknown classes.

Figure 5: $HOS$ score on three sub tasks of Office31 ($\%$) with a various
number of estimated target classes $|C_{P}|$. The best performance is obtained
with $|C_{P}|=|C_{S}|$ target classes.

From uncertainty to selection probabilities. In Eqs. 5 and 6, the uncertainty
coefficient $w^{nc}_{t}$ and $w^{cs}_{t}$ are estimated with the use of a
monotonically decreasing function $\mathcal{F}$. We made two choices for this
function: (i) a linear function $\mathcal{F}_{\operatorname{lin}}(x)=1-x$; and
(ii) an exponential function $\mathcal{F}_{\operatorname{exp}}(x)=e^{-x}$. In
Table IV, we study the effect of using either or both functions to calculate
the uncertainty coefficients. Our model achieves comparable performance with
all the combinations of these functions. As reported in [10], the exponential
function works better for Eq. 5 because it avoids to penalise samples near the
boundaries. For Eq. 6, the effect of the choice of $\mathcal{F}$ is marginal,
as the performance drop is approx. $0.1\%$. A motivation of this behaviour is
that the minimum value for the coefficient $u^{cs}_{t}$ is $0.5$, which avoids
over-penalising samples when $\mathcal{F}_{\operatorname{lin}}$ is used.

Combining uncertainty estimation strategies. In Sec. 3.4, we estimate the
uncertainty coefficients $w^{nc}_{t}$ and $w^{cs}_{t}$, which are used as the
success probability to sample values from Bernoulli distributions. Sampled
values are combined together via a logical operator $\oplus$ (Eq. 7). We opted
for two choices for the $\oplus$ operator: and and or operators. We compare
the benefits of these operators in Table V. Although either operator surpasses
the state-of-the-art, our model achieves the best performance by combining the
uncertainty probabilities with the and operator. The and operator requires
that samples have low uncertainty in both measures, resulting in a more fine-
grained sample selection with respect to the or operator.

TABLE IV: HOS score (%) on three sub-tasks of Office31 using different choices
of the function $\mathcal{F}$ in Eqs. 5 and 6.

Eq. 5 Eq. 6 D $\rightarrow$ W A $\rightarrow$ W D $\rightarrow$ A Avg.
$\mathcal{F}_{\operatorname{lin}}$ $\mathcal{F}_{\operatorname{lin}}$ 91.5
79.0 76.2 82.2 $\mathcal{F}_{\operatorname{lin}}$
$\mathcal{F}_{\operatorname{exp}}$ 90.5 77.4 77.8 81.9
$\mathcal{F}_{\operatorname{exp}}$ $\mathcal{F}_{\operatorname{exp}}$ 93.1
83.6 78.0 84.9 $\mathcal{F}_{\operatorname{exp}}$
$\mathcal{F}_{\operatorname{lin}}$ 93.2 83.9 78.0 85.0

TABLE V: $HOS$ score on three sub tasks of Office31 ($\%$) comparing different
choices of the logical operator $\oplus$ in Eq. 7.

Method D $\rightarrow$ W A $\rightarrow$ W D $\rightarrow$ A Avg. Ours w/ OR
sampling 91.5 80.1 77.7 83.1 Ours w/ AND sampling 93.2 83.9 78.0 85.0

TABLE VI: Clustering accuracy ($\%$) on three sub tasks of Office31 evaluating
the ability of our model to perform novel class discovery. Note that accuracy
is calculated only for samples belonging to target-private classes.

D $\rightarrow$ W A $\rightarrow$ W D $\rightarrow$ A Avg. 71.3 54.8 66.7 64.2

Discovery of the underlying semantics of novel classes. We present here
additional experiments to asses whether our model is able, as a byproduct, to
aggregate samples from target-private classes into different clusters based on
the semantics. To achieve this objective, the model has to learn the semantics
of target-private classes, even if it cannot be directly optimised for this
scope, due to the lack of labels in the target domain. We follow the same
protocol as in state-of-the-art novel class discovery works [69, 70]. We train
the model by setting $|C_{P}|$ as the actual number of private classes: this
is necessary to evaluate the results of these experiments. Since we do not use
labels for target-private classes during adaptation, a class misalignment is
present between the predicted and the ground-truth target-private classes. To
solve this problem, at inference time, we match the predicted target-private
classes against the ground-truth classes. Specifically, we compute target-
private class prototypes by averaging features of samples belonging to each
class, according to their ground truth labels. The same process is repeated to
compute the estimated prototypes using predictions rather than labels. We then
run the Hungarian algorithm to perform the matching between the ground-truth
and the estimated prototypes, and adjust the predicted labels accordingly.
Once the predicted and ground-truth target-private classes are aligned, we
quantitatively and qualitatively assess the ability of our model to correctly
classify samples from target-private classes. As in previous works[69, 70], we
quantitatively evaluate the ability of our method to cluster samples from
target-private classes by computing the clustering accuracy:

$\operatorname{ClusterAcc}=\max_{t\in\mathcal{P}(C^{P})}\frac{1}{|C^{P}|}\sum_{j=1}^{|C^{P}|}\mathds{1}(y_{j}=t(\hat{y}_{j}))$
(11)

where $y_{j}$ and $\hat{y}_{j}$ are ground-truth labels and predictions,
respectively. For each target-private sample, $\mathcal{P}(C^{P})$ is the set
of all possible permutations of $C^{P}$ classes and $t$ is a permutation.

Table VI and Fig. 6 show that our model is able to achieve satisfying results
in the classification of target-private samples, even if it has been neither
optimised for this task nor trained with labels from these classes. This means
the model has learned the underlying semantics of novel classes, enabling the
possibility to potentially perform novel category discovery. Note that this
behaviour is not possible with previous works that classify all the private
samples in a single unknown class. Finally, Fig. 6 shows the ability of our
model to produce a well-structured features space, where samples belonging to
the same target-private class lie close in the space, allowing them to be
clustered in the same novel class.

Figure 6: Feature space visualization for target-private classes. The colour
of samples represents the predicted class. The $\bullet$ symbol represents
correct predictions, whereas the $\times$ represents incorrect predictions.
Our model produces a well-structured features space for target-private
classes, where samples belonging to the same novel classes are clustered
together. This opens the possibility of performing novel class discovery.

##  5 Conclusion

In this paper, we introduced a novel approach to address the challenges of
Source-Free Open-Set Domain Adaptation (SF-OSDA) for image classification.
Unlike existing approaches that group samples from unknown classes into a
single “unknown” class, our method leverages the granularity of target-private
classes to segregate their samples into multiple clusters, resulting in a more
effective adaptation that enables the learning of the underlying semantics of
novel classes. To that end, we introduce a novel uncertainty estimation
technique to handle noisy pseudo-labels produced during the adaptation. We
also propose a novel NL-InfoNCELoss that integrates negative learning into
self-supervised contrastive learning further increasing the robustness of the
contrastive framework to the noise in the pseudo-labels. Our approach
outperforms state-of-the-art methods on benchmark datasets, such as Office31
and Office-Home. Further analyses show that our model can aggregate samples
from target-private classes into different clusters as a byproduct of the
adaptation, as it has learned the underlying semantics of novel classes. This
behaviour also opens the possibility of performing novel class discovery.

## References

  * [1] J. Quionero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, _Dataset Shift in Machine Learning_. The MIT Press, 2009.
  * [2] A. Gretton, A. Smola, J. Huang, M. Schmittfull, K. Borgwardt, and B. Schölkopf, _Covariate shift and local learning by distribution matching_. Cambridge, MA, USA: MIT Press, 2009, pp. 131–160.
  * [3] K. Bousmalis, N. Silberman, D. Dohan, D. Erhan, and D. Krishnan, “Unsupervised pixel-level domain adaptation with generative adversarial networks,” _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 95–104, 2017.
  * [4] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” _J. Mach. Learn. Res._ , vol. 17, no. 1, p. 2096–2030, jan 2016.
  * [5] J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell, “CyCADA: Cycle-consistent adversarial domain adaptation,” in _Proceedings of the 35th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. PMLR, 10–15 Jul 2018, pp. 1989–1998. [Online]. Available: https://proceedings.mlr.press/v80/hoffman18a.html
  * [6] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarial discriminative domain adaptation,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 2962–2971.
  * [7] M. Mancini, L. Porzi, S. R. Buló, B. Caputo, and E. Ricci, “Inferring latent domains for unsupervised deep domain adaptation,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 43, no. 2, p. 485–498, feb 2021. [Online]. Available: https://doi.org/10.1109/TPAMI.2019.2933829
  * [8] W. Li, Z. Xu, D. Xu, D. Dai, and L. Van Gool, “Domain generalization and adaptation using low rank exemplar svms,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 40, no. 5, pp. 1114–1127, 2018.
  * [9] Q. Zhou, Q. Gu, J. Pang, X. Lu, and L. Ma, “Self-adversarial disentangling for specific domain adaptation,” _IEEE Trans. Pattern Anal. Mach. Intell._ , vol. 45, no. 7, p. 8954–8968, jul 2023. [Online]. Available: https://doi.org/10.1109/TPAMI.2023.3238727
  * [10] M. Litrico, A. D. Bue, and P. Morerio, “Guiding pseudo-labels with uncertainty estimation for source-free unsupervised domain adaptation,” _2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 7640–7650, 2023.
  * [11] Y. Sun, X. Wang, Z. Liu, J. Miller, A. Efros, and M. Hardt, “Test-time training with self-supervision for generalization under distribution shifts,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, H. D. III and A. Singh, Eds., vol. 119. PMLR, 13–18 Jul 2020, pp. 9229–9248. [Online]. Available: https://proceedings.mlr.press/v119/sun20b.html
  * [12] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell, “Tent: Fully test-time adaptation by entropy minimization,” in _International Conference on Learning Representations_ , 2021. [Online]. Available: https://openreview.net/forum?id=uXl3bZLkr3c
  * [13] J. Liang, D. Hu, and J. Feng, “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. ICML’20. JMLR.org, 2020.
  * [14] S. Yang, Y. Wang, J. van de Weijer, L. Herranz, and S. Jui, “Generalized source-free domain adaptation,” in _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2021, pp. 8958–8967.
  * [15] K. Saito, S. Yamamoto, Y. Ushiku, and T. Harada, “Open set domain adaptation by backpropagation,” in _Computer Vision – ECCV 2018_ , V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Cham: Springer International Publishing, 2018, pp. 156–171.
  * [16] P. P. Busto and J. Gall, “Open set domain adaptation,” in _2017 IEEE International Conference on Computer Vision (ICCV)_ , 2017, pp. 754–763.
  * [17] S. Bucci, M. R. Loghmani, and T. Tommasi, “On the effectiveness of image rotation for open set domain adaptation,” in _Computer Vision – ECCV 2020_ , A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, Eds. Cham: Springer International Publishing, 2020, pp. 422–438.
  * [18] J. Jang, B. Na, D. Shin, M. Ji, K. Song, and I.-C. Moon, “Unknown-aware domain adversarial learning for open-set domain adaptation,” 2022.
  * [19] H. Liu, Z. Cao, M. Long, J. Wang, and Q. Yang, “Separate to adapt: Open set domain adaptation via progressive separation,” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 2922–2931.
  * [20] Y. Shiqi, W. Yaxing, W. Kai, J. Shangling, and v. d. w. Joost, “Attracting and dispersing: A simple approach for source-free domain adaptation,” in _Advances in Neural Information Processing Systems_ , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds. Curran Associates, Inc., 2022.
  * [21] C. S. Jahan and A. Savakis, “Unknown sample discovery for source free open set domain adaptation,” _ArXiv_ , vol. abs/2312.03767, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:266052705
  * [22] E. Arazo, D. Ortego, P. Albert, N. E. O’Connor, and K. McGuinness, “Pseudo-labeling and confirmation bias in deep semi-supervised learning,” _2020 International Joint Conference on Neural Networks (IJCNN)_ , pp. 1–8, 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:199501839
  * [23] P. Morerio, R. Volpi, R. Ragonesi, and V. Murino, “Generative pseudo-label refinement for unsupervised domain adaptation,” in _Winter Conference on Applications of Computer Vision (WACV)_ , 2020.
  * [24] D. Chen, D. Wang, T. Darrell, and S. Ebrahimi, “Contrastive test-time adaptation,” in _CVPR_ , 2022.
  * [25] Y. Kim, J. Yun, H. Shon, and J. Kim, “Joint negative and positive learning for noisy labels,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2021, pp. 9442–9451.
  * [26] W. Ahmed, P. Morerio, and V. Murino, “Cleaning noisy labels by negative ensemble learning for source-free unsupervised domain adaptation,” in _2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)_ , 2022, pp. 356–365.
  * [27] K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual category models to new domains,” in _Computer Vision – ECCV 2010_ , K. Daniilidis, P. Maragos, and N. Paragios, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2010, pp. 213–226.
  * [28] H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. Los Alamitos, CA, USA: IEEE Computer Society, jul 2017, pp. 5385–5394. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/CVPR.2017.572
  * [29] Q. Wang, F. Meng, and T. Breckon, “Progressively select and reject pseudo-labelled samples for open-set domain adaptation,” _ArXiv_ , vol. abs/2110.12635, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:239768329
  * [30] J. N. Kundu, S. Bhambri, A. R. Kulkarni, H. Sarkar, V. Jampani, and V. B. R, “Subsidiary prototype alignment for universal domain adaptation,” in _Advances in Neural Information Processing Systems_ , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 29 649–29 662. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/bf121b033db3bac31c3193e8a0dcbf66-Paper-Conference.pdf
  * [31] S. Yang, Y. Wang, K. Wang, S. Jui, and J. van de Weijer, “One ring to bring them all: Towards open-set recognition under domain shift,” _arXiv preprint arXiv:2206.03600_ , 2022.
  * [32] J. Nath Kundu, N. Venkat, M. V. Rahul, and R. Venkatesh Babu, “Universal source-free domain adaptation,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020, pp. 4543–4552.
  * [33] J. N. Kundu, N. Venkat, A. Revanur, M. V. Rahul, and R. V. Babu, “Towards inheritable models for open-set domain adaptation,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. Los Alamitos, CA, USA: IEEE Computer Society, jun 2020, pp. 12 373–12 382. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/CVPR42600.2020.01239
  * [34] M. Caron, P. Bojanowski, A. Joulin, and M. Douze, “Deep clustering for unsupervised learning of visual features,” in _Computer Vision – ECCV 2018_ , V. Ferrari, M. Hebert, C. Sminchisescu, and Y. Weiss, Eds. Cham: Springer International Publishing, 2018, pp. 139–156.
  * [35] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. ICML’20. JMLR.org, 2020.
  * [36] T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. Hinton, “Big self-supervised models are strong semi-supervised learners,” in _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , ser. NIPS’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
  * [37] X. Chen and K. He, “Exploring simple siamese representation learning,” in _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2021, pp. 15 745–15 753.
  * [38] S. Gidaris, P. Singh, and N. Komodakis, “Unsupervised representation learning by predicting image rotations,” _ArXiv_ , vol. abs/1803.07728, 2018.
  * [39] J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko, “Bootstrap your own latent a new approach to self-supervised learning,” in _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , ser. NIPS’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
  * [40] K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020, pp. 9726–9735.
  * [41] G. Larsson, M. Maire, and G. Shakhnarovich, “Colorization as a proxy task for visual understanding,” _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 840–849, 2017.
  * [42] M. Noroozi and P. Favaro, “Unsupervised learning of visual representations by solving jigsaw puzzles,” in _ECCV_ , 2016.
  * [43] K. Saito, D. Kim, S. Sclaroff, and K. Saenko, “Universal domain adaptation through self-supervision,” in _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , ser. NIPS’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
  * [44] ——, “Universal domain adaptation through self supervision,” in _Advances in Neural Information Processing Systems_ , H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin, Eds., vol. 33. Curran Associates, Inc., 2020, pp. 16 282–16 292. [Online]. Available: https://proceedings.neurips.cc/paper/2020/file/bb7946e7d85c81a9e69fee1cea4a087c-Paper.pdf
  * [45] D. Wang, S. Liu, S. Ebrahimi, E. Shelhamer, and T. Darrell, “On-target adaptation,” 2022. [Online]. Available: https://openreview.net/forum?id=6ooiNCGZa5K
  * [46] A. Maracani, R. Camoriano, E. Maiettini, D. Talon, L. Rosasco, and L. Natale, “Key design choices in source-free unsupervised domain adaptation: An in-depth empirical analysis,” _arXiv preprint arXiv:2402.16090_ , 2024.
  * [47] D.-H. Lee, “Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks,” 2013. [Online]. Available: https://api.semanticscholar.org/CorpusID:18507866
  * [48] K. Sohn, D. Berthelot, C.-L. Li, Z. Zhang, N. Carlini, E. D. Cubuk, A. Kurakin, H. Zhang, and C. Raffel, “Fixmatch: Simplifying semi-supervised learning with consistency and confidence,” in _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , ser. NIPS’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
  * [49] J. Liang, D. Hu, and J. Feng, “Domain adaptation with auxiliary target domain-oriented classifier,” in _2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2021, pp. 16 627–16 637.
  * [50] C. Zhang, S. Bengio, M. Hardt, B. Recht, and O. Vinyals, “Understanding deep learning (still) requires rethinking generalization,” _Commun. ACM_ , vol. 64, no. 3, p. 107–115, feb 2021. [Online]. Available: https://doi.org/10.1145/3446776
  * [51] Y. Wang, X. Ma, Z. Chen, Y. Luo, J. Yi, and J. Bailey, “Symmetric cross entropy for robust learning with noisy labels,” _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 322–330, 2019.
  * [52] A. Ghosh, H. Kumar, and P. S. Sastry, “Robust loss functions under label noise for deep neural networks,” in _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence_ , ser. AAAI’17. AAAI Press, 2017, p. 1919–1925.
  * [53] X. Ma, H. Huang, Y. Wang, S. Romano, S. Erfani, and J. Bailey, “Normalized loss functions for deep learning with noisy labels,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. Proceedings of Machine Learning Research, H. D. III and A. Singh, Eds., vol. 119. PMLR, 13–18 Jul 2020, pp. 6543–6553.
  * [54] Y. Kim, J. Yim, J. Yun, and J. Kim, “Nlnl: Negative learning for noisy labels,” _2019 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 101–110, 2019.
  * [55] J. Goldberger and E. Ben-Reuven, “Training deep neural-networks using a noise adaptation layer,” in _ICLR_ , 2017.
  * [56] T. Liu and D. Tao, “Classification with noisy labels by importance reweighting,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 38, no. 3, pp. 447–461, 2016.
  * [57] L. Jiang, Z. Zhou, T. Leung, J. Li, and F.-F. Li, “Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels,” in _ICML_ , 2018.
  * [58] K.-H. Lee, X. He, L. Zhang, and L. Yang, “Cleannet: Transfer learning for scalable image classifier training with label noise,” _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 5447–5456, 2018.
  * [59] H. Wei, L. Feng, X. Chen, and B. An, “Combating noisy labels by agreement: A joint training method with co-regularization,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2020, pp. 13 723–13 732.
  * [60] J. Li, R. Socher, and S. C. H. Hoi, “Dividemix: Learning with noisy labels as semi-supervised learning,” _ArXiv_ , vol. abs/2002.07394, 2020.
  * [61] H. B. Mitchell and P. A. Schaefer, “A “soft” k‐nearest neighbor voting scheme,” _International Journal of Intelligent Systems_ , vol. 16, 2001.
  * [62] K. You, M. Long, Z. Cao, J. Wang, and M. I. Jordan, “Universal domain adaptation,” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 2715–2724.
  * [63] W. Li, J. Liu, B. Han, and Y. Yuan, “Adjustment and alignment for unbiased open set domain adaptation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2023, pp. 24 110–24 119.
  * [64] Y. Luo, Z. Wang, Z. Huang, and M. Baktashmotlagh, “Progressive graph learning for open-set domain adaptation,” in _Proceedings of the 37th International Conference on Machine Learning_ , ser. ICML’20. JMLR.org, 2020.
  * [65] Z. Fang, J. Lu, F. Liu, J. Xuan, and G. Zhang, “Open set domain adaptation: Theoretical bound and algorithm,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 32, no. 10, pp. 4309–4322, 2021.
  * [66] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 770–778.
  * [67] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , 2009, pp. 248–255.
  * [68] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, “Scikit-learn: Machine learning in Python,” _Journal of Machine Learning Research_ , vol. 12, pp. 2825–2830, 2011.
  * [69] M. Yang, L. Wang, C. Deng, and H. Zhang, “Bootstrap your own prior: Towards distribution-agnostic novel class discovery,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , June 2023, pp. 3459–3468.
  * [70] E. Fini, E. Sangineto, S. Lathuilière, Z. Zhong, M. Nabi, and E. Ricci, “A unified objective for novel class discovery,” _2021 IEEE/CVF International Conference on Computer Vision (ICCV)_ , pp. 9264–9272, 2021. [Online]. Available: https://api.semanticscholar.org/CorpusID:237213460

|  Mattia Litrico Mattia Litrico is a Ph.D. Student in Deep Learning and
Computer Vision at the University of Catania. He obtained his M. Sc. in
Computer Science at the University of Catania (Italy) in 2021. He was a
Research Fellow at the Italian Institute of Technology (IIT), Pattern Analysis
and Computer Vision (PAVIS) department in 2022. His research interests include
machine learning and computer vision, with a particular emphasis on domain
adaptation techniques.
---|---
|  Davide Talon Davide Talon is a Postdoc at Fondazione Bruno Kessler. He
obtained his M. Sc. in Computer Engineering at University of Padova (Italy).
He pursued a PhD degree in Computer Science at the Italian Institute of
Technology (IIT), Pattern Analysis and Computer Vision (PAVIS) department. His
research interests span across machine learning, deep learning, representation
learning and causality.
---|---
|  Sebastiano Battiato Sebastiano Battiato (Senior Member, IEEE) is a Full
Professor of Computer Science at the University of Catania. Throughout his
career, Battiato has held significant leadership roles such as Chairman of the
Undergraduate Program in Computer Science, Rector’s delegate for postgraduate
education, and Scientific Coordinator of the PhD Program in Computer Science.
His research interests span Computer Vision, Imaging Technology, and
Multimedia Forensics, focusing on enhancing imaging algorithms for both
acquisition and post-processing. Battiato is a prolific scholar, having edited
six books and authored around 350 papers. He has been a principal investigator
in numerous research projects, supervised many PhD students and postdocs, and
is a co-inventor of 25 international patents. His contributions to computer
science have been recognized with several awards, including the 2017 PAMI Mark
Everingham Prize and the 2011 Best Associate Editor Award of the IEEE
Transactions on Circuits and Systems for Video Technology.
---|---
|  Alessio Del Bue Alessio Del Bue (Member, IEEE) is a Tenured Senior
Researcher leading the Pattern Analyisis and computer VISion (PAVIS) Research
Line of the Italian Institute of Technology (IIT), Genoa, Italy. He is a
coauthor of more than 100 scientific publications in refereed journals and
international conferences on computer vision and machine learning topics. His
current research interests include 3D scene understanding from multi-modal
input (images, depth, and audio) to support the development of assistive
artificial intelligence systems. He is a member of the technical committees of
major computer vision conferences (CVPR, ICCV, ECCV, and BMVC). He serves as
an Associate Editor for Pattern Recognition and Computer Vision and Image
Understanding journals. He is a member of ELLIS.
---|---
|  Mario Valerio Giuffrida Mario Valerio Giuffrida is an Assistant Professor
in Computer Vision at The University of Nottingham. He pursued a PhD degree in
Image Analysis at IMT School For Advanced Studies Lucca. In the 2016, he
participated at the enrichment program of The Alan Turing Institute. His
current research interests include machine learning and computer vision for
plant phenotyping.
---|---
|  Pietro Morerio Pietro Morerio received his M. Sc. in Theoretical Physics
from the University of Milan (Italy) in 2010 (summa cum laude). He was
Research Fellow at the University of Genoa (Italy) from 2011 to 2012, working
in Video Analysis for Interactive Cognitive Environments. He pursued a PhD
degree in Computational Intelligence at the same institution in 2016.
Currently he is a Researcher at Istituto Italiano di Tecnologia (IIT), Pattern
Analysis and Computer Vision (PAVIS) department, his research including
machine learning, deep learning and computer vision, with a particular focus
on multimodal learning.
---|---