# Online Continual Domain Adaptation for Semantic Image Segmentation Using
Internal Representations

Serban Stan and Mohammad Rostami

###### Abstract

Semantic segmentation models trained on annotated data fail to generalize well
when the input data distribution changes over extended time period, leading to
requiring re-training to maintain performance. Classic Unsupervised domain
adaptation (UDA) attempts to address a similar problem when there is target
domain with no annotated data points through transferring knowledge from a
source domain with annotated data. We develop an online UDA algorithm for
semantic segmentation of images that improves model generalization on
unannotated domains in scenarios where source data access is restricted during
adaptation. We perform model adaptation is by minimizing the distributional
distance between the source latent features and the target features in a
shared embedding space. Our solution promotes a shared domain-agnostic latent
feature space between the two domains, which allows for classifier
generalization on the target dataset. To alleviate the need of access to
source samples during adaptation, we approximate the source latent feature
distribution via an appropriate surrogate distribution, in this case a Gassian
mixture model (GMM). We evaluate our approach on well established semantic
segmentation datasets and demonstrate it compares favorably against state-of-
the-art (SOTA) UDA semantic segmentation methods.111Partial results of this
work were presented in the AAAI Conference Stan and Rostami (2021b).

##  1 Introduction

Recent progress in deep learning has led to developing semantic segmentation
algorithms that are being adopted in many real world tasks. Autonomous driving
Zhang et al. (2016); Feng et al. (2020), object tracking Kalake et al. (2021)
or aerial scene parsing Sun et al. (2021) are just a few examples of these
applications. Deep neural networks (DNNs) have proven indispensable for
reaching above human performance in semantic segmentation tasks, given the
ability of large networks to approximate complex decision functions He et al.
(2015). Training such networks however requires access to large continuously
annotated datasets. Given that in semantic segmentation each image pixel
requires a label, generating new labeled data for semantic segmentation tasks
requires significant more overhead compared to regular classification
problems.

Domain Adaptation (DA) is a sub-field of AI which aims to allow model
generalization for input distributions different from those observed in the
training dataset Wang and Deng (2018). Unsupervised Domain Adaptation (UDA)
addresses this problem for instances where the deployment dataset lacks label
information Wilson and Cook (2020). This set of approaches is of especial
interest for semantic segmentation tasks, where data annotation is expensive
and time-consuming. In UDA, a model trained on a fully annotated source domain
needs to generalize on an unannotated target domain with a different data
distribution. A primary approach for achieving domain generalization is
learning a shared embedding feature space between the source and target in
which the domains become similar. If domain agnostic features are learnt, a
semantic classifier trained on source data will maintain predictive power on
target data. While this high level approach is shared among various UDA
frameworks, different methods have been proposed for achieving this goal.

A common line of research achieves domain alignment by the use of adversarial
learning Goodfellow et al. (2020). A feature extractor produces latent feature
embeddings for the source and target domains, while a domain discriminator is
tasked with differentiating the origin domain of the features. These two
networks are trained adversarially, process which leads the feature extractor
to learn a domain invariant feature representation upon training completion.
There is a large body of UDA work following this methodology Hoffman et al.
(2018a); Benjdira et al. (2019); Hoffman et al. (2016); Hong et al. (2018);
Chen et al. (2018a); Jian and Rostami (2023). A different set of approaches
attempts direct distribution alignment between the source and target domains.
Distribution alignment can then be achieved by minimizing an appropriate
distributional distance metric Wu et al. (2018); Zhang et al. (2017); Gabourie
et al. (2019); Zhang et al. (2019); Lee et al. (2019); Yang and Soatto (2020);
Rostami et al. (2023); Rostami and Galstyan (2023b), such as $l2$-distance,
$KL$-divergence or Wasserstein Distance.

While both types of approaches are able to obtain state of the art (SOTA)
results on UDA semantic segmentation tasks, most methods assume simultaneous
access to both source and target data samples. This benefits model stability
during adaptation, as source domain access ensures a gradual shift of the
decision function. However, in real world settings there are many situations
where concomitant access to both domains cannot be achieved. For instance,
datasets may need to be stored on different servers Stan and Rostami (2022)
due to latency constraints Xia et al. (2021) or data privacy requirements Li
et al. (2020b); Stan and Rostami (2021a). To adhere to these settings, UDA has
been extended to situations where the source domain data is no longer
accessible during adaptation. This class of methods is named source-free
adaptation, and provides a balance between accuracy and privacy Kundu et al.
(2020a); Kim et al. (2021). Compared to regular UDA, source-free UDA is less
explored. Our approach addresses source-free adaptation, making it a suitable
algorithm for scenarios where data privacy is an issue. Moreover, our proposed
method is based on common DNN architectures for semantic segmentation, and
requires little parameter fine-tuning compared to adversarial approaches.

Contributions: We propose a novel algorithm that performs source-free UDA for
semantic segmentation tasks. Our approach eliminates the need for access to
source data during the adaptation phase, by approximating the source domain
via a internal intermediate distribution Rostami (2021); Rostami et al.
(2020). During adaptation, our method aligns the target and intermediate
domain via a suitable distance metric to ensure classifier generalization on
target features. We demonstrate the performance of our method on two benchmark
semantic segmentation tasks, GTA5$\rightarrow$CITYSCAPES and
SYNTHIA$\rightarrow$CITYSCAPES, where the source datasets are composed of
computer generated images, and the target datasets are real world segmented
images. We offer theoretical justification for our algorithms’ performance,
proving our approach minimizes target error under our adaptation framework.

##  2 Related Work

We provide an overview of semantic segmentation algorithms, as well as
describe recent UDA and source-free UDA approaches for this setting.

###  2.1 Semantic Segmentation

Compared to image classification problems, semantic segmentation tasks are
more complicated because we require each pixel of an image to receive a label,
which is part of a set of semantic categories. As each image dimension may
have thousands of pixels, semantic segmentation models require powerful
encoder/decoder architectures capable of synthesizing large amounts of image
data and encode the spatial relationships between the pixels well. Recent SOTA
results for supervised semantic segmentation have thus been obtained by the
use deep neural networks (DNNs), and in particular convolutional neural
networks (CNNs) LeCun et al. (1995), which are specialized for image
segmentation. While different architecture variants exists Long et al. (2015);
Wang et al. (2020); Chen et al. (2018b); Tao et al. (2020), approaches often
rely on embedding images into a latent feature space via a CNN feature
extractor, followed by an up-sampling decoder which scales the latent space
back to the original input space, where a classifier is trained to offer
pixel-wise predictions. The idea is if the extracted features can reconstruct
the input image with a relatively high accuracy, then they carry an
information content similar to the input distribution, yet in a lower
dimensional space. Skip connections between different levels of the network
Ronneberger et al. (2015); Lin et al. (2017), using dilated convolutions Chen
et al. (2017a) or using transformer networks as feature extractors instead of
CNNs Strudel et al. (2021) have been shown to improve supervised baselines.

While improvements in supervised segmentation are mostly tied to architecture
choice and parameter fine-tuning, model generalization suffers when changes in
the input distribution are made. This phenomenon is commonly referred to as
domain shift Sankaranarayanan et al. (2018). Changes in the input distortion
translate into shifted extracted features that do not match the internal
distribution learned by the DNN Rostami et al. (2019c). This issue is common
in application domains where the same model needs to account for multi-modal
data, and the training set lacks a certain mode, e.g. daylight and night-time
images Romera et al. (2019), clear weather and foggy weather Sakaridis et al.
(2018), medical data obtained from different imaging devices and scanners Guan
and Liu (2021). Such differences in input data distributions between source
and target domains greatly impact the generalization power of learnt models.
When domain shift is present, source-only training may be at least three-fold
inferior compared to training the same model on the target dataset Hoffman et
al. (2016, 2018a); Lee et al. (2019). While it is possible to retrain the
model to account for distribution shifts, we will require to annotate data
again which can be time-consuming and expensive in semantic segmentation tasks
Rostami et al. (2018). Because label information is expensive to obtain,
adding a cost to using such techniques, especially on new datasets. Weakly-
supervised approaches explore the possibility of having access to limited
label information after domain shift to reduce the data annotation requirement
Wei et al. (2018); Hung et al. (2018); Wang et al. (2019). However, data
annotation is still necessary.

On the other hand, due to the limited label availability for semantic
segmentation tasks, the use of synthetic images and labels has become an
attractive alternative for training segmentation models even if domain shift
is not a primary concern. The idea is to prepare a synthetic source dataset
which can be annotated automatically. Semantic labels are easy to generate for
virtual images, and a model trained on such images could then be used on real
world data as a starting point. Overcoming domain shift becomes the primary
bottleneck for successfully applying such models to new domains.

###  2.2 Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) addresses model generalization in
scenarios where target data label information is unavailable but there a
source domain with annotated data that shares the same labels with the target
domain problem. UDA techniques primarily employ a shared feature embedding
space between the source and target domain in which the distributions of both
domains are aligned. A majority of these methods achieve this goal by either
using domain discriminators based on adversarial learning or direct source-
target feature alignment based on metric minimization.

####  2.2.1 Adversarial Adaptation for UDA

Techniques based on adversarial learning employ the idea of domain
discriminator, used in GANs Goodfellow et al. (2014), to produce a shared
source/target embedding space. A discriminator is tasked with differentiating
whether two image encodings originate from the same domain, or one is from the
source and one is from the target. A feature encoder aims to fool the
discriminator, thus producing source/target latent features more and more
similar in nature as training progressed. Over the course of training, this
leads the feature extractor producing a shared embedding space for the source
and target data.

In the context of UDA for semantic segmentation, Luc et al. (2016) employ an
image segmentation model and adversarially train a semantic map generator,
which uses a label map discriminator to penalize the segmentation network for
producing label maps more similar to the generated ones rather than the source
ones. Murez et al. (2017) use an intermediate feature distribution that
attempts to capture domain agnostic features of both source and target
datasets. To improve the domain agnostic representation, a discriminator is
trained to differentiate whether an encoded image is part of the source or
target domain. The encoder networks are then adversarially trained to fool the
discriminator, resulting in similar embeddings between source and target
samples. Bousmalis et al. (2017) develop a model for pixel level domain
adaptation by creating a pseudo-dataset by making source samples appear as
though they are extracted from the target domain. They use generative model to
adapt source samples to the style of target samples, and a domain
discriminator to differentiate between real and fake target data. Hoffman et
al. (2018b) employ the cycle consistency loss proposed in Zhu et al. (2017) to
improve the adversarial network adaptation performance. In addition to this,
Hoffman et al. (2018b) use GANs to stylistically transfer images between
source and target domains, and use a consistency loss to ensure network
predictions on the source image will be the same as in the stylistically
shifted variant. Saito et al. (2018) use an approach based on a discriminator
network without using GANs to attempt to mimic source or target data
distributions. They propose the following adversarial learning process on a
feature encoder network with two classification heads: (1) they first keep the
feature encoder fixed and optimize the classifiers to have their outputs as
different as possible, (2) they freeze the classifiers and optimize the
feature encoder such that both classifiers will have close outputs.
Sankaranarayanan et al. (2018) employs an image translation network, that is
tasked with translating input images into the target domain feature space. A
discriminator is tasked with differentiating source images from target images
passed through the network, and a similar procedure is done for target images.
A pixel-level cross entropy loss ensures the network is able to perform
semantic segmentation. Lee et al. (2019) use a similar idea to Saito et al.
(2018) in that a network with two classifiers is used for adaptation. The
feature extractor and classification heads are trained in an alternating
fashion. The work of Lee et al. (2019) differentiates itself by employing an
approximation of optimal transport to compute these discrepancy metrics,
leading to improved performance over Saito et al. (2018).

####  2.2.2 Adaptation by distribution alignment

Adaptation methods using direct distribution alignment share the same goal as
adversarial methods. However, distribution alignment is reached by directly
minimizing an appropriate distributional distance metric between the source
and target embedding feature distributions.

Wu et al. (2018) propose an image translation network that takes as input
source and target images, and outputs source images in the style of the target
domain. Their proposed architecture does not use adversarial training, rather
is based on the idea that in order for stylistic transfer to be achieved,
domain mean and variance should be similar at different levels of abstraction
throughout the translation network. They achieve this goal by minimizing
$\ell_{2}$-distance in the feature space at various levels of abstraction.
Zhang et al. (2017) develop a method for semantic segmentation adaptation by
observing that a source trained model should produce the same data statistics
on the target domain as present in the annotated source distribution. Examples
include label distribution or pixels of a certain class clustering around
specific regions in an image. Pseudo-labeling is used to estimate these
statistics Wu and Rostami (2023). To enforce similarity in the output of a
source-trained model to the estimated target statistics, KL-divergence is used
as a minimization metric. Zhang et al. (2019) employ an adaptation framework
based on the idea that source and target latent features should cluster
together in similar ways. They use a pseudo-labeling approach to produce
initial target labels, followed by minimizing the distance between class
specific latent feature centroids between source and target domains. The
minimization metric of choice is $\ell_{2}$-distance. For improved
performance, they use category anchors to align the adaptation process.
Gabourie et al. (2019) propose an adaptation method based on a shared network
encoder between source and target domains. Their model is trained by
minimizing cross entropy loss for the source samples, and is tasked with
minimizing the distance between source and target embeddings in the latent
feature space. To achieve this goal, Sliced Wasserstein Distance is minimized
between the source and target embeddings, leading to improved classifier
performance on target samples.

The expectation that continuous access to source data is guaranteed when
performing UDA is not always true, especially in the case of privacy sensitive
domains. This setting of UDA has been previously explored by methods that do
not employ DNNs Dredze and Crammer (2008); Jain and Learned-Miller (2011); Wu
(2016), and has recently become the focus of DNN based algorithms for image
classification tasks Kundu et al. (2020b); Kim et al. (2021); Saltori et al.
(2020); Yang et al. (2021). Source-free semantic segmentation has been
explored relatively less compared to joint UDA adaptation approach. Kundu et
al. (2021) employ source-domain generalization and target pseudo-labeling in
the adaptation method. Liu et al. (2021) rely on self supervision and patch
level optimization for adaptation. You et al. (2021) allow models trained on
synthetic data to generalize on real data by a mixture of positive and
negative class inference.

Our adaptation approach shares the idea of direct distribution alignment. As
described previously, several choices for latent feature alignment have been
previously explored, such as $l2$-distance Wu et al. (2018), KL-divergence
Zhang et al. (2017) or Wasserstein Distance (WD) Gabourie et al. (2019); Lee
et al. (2019). WD has been proven to leverage the geometry of the embedding
space better than other distance metrics Tolstikhin et al. (2017).
Empirically, the behavior of using the Wasserstein metric has been observed to
benefit the robustness of training deep models, such as in the case of the
Wasserstein GAN Arjovsky et al. (2017), or by improving the relevance of
discrepancy measures, as reported by Lee et al. (2019). One of the limitations
of using the WD is the difficulty of optimizing this quantity, as computing
the WD distance requires solving a linear program. Therefore, we employ an
approximation of this metric, the Sliced Wasserstein Distance (SWD), which
maintains the nice metric properties of the WD while allowing for an end-to-
end differentiation in the optimization process.

We base our source-free UDA approach on estimating the latent source
embeddings via an internal distribution Rostami (2019, 2022). This
approximation relies on the concept that a supervised model trained on $K$
classes will produce a $K$ modal distribution in its latent space. This
property of the internal distribution allows us to perform adaptation without
direct access to source samples. The idea is to approximate the internal
distribution and then sample from the $K$ modal distributional approximation
to use them as a surrogate for the source domain distribution. The
distribution approximation introduces a small number of parameters into our
model. Once we produce a pseudo-dataset from sampling the internal
distribution, we align the target feature encodings by minimizing the SWD
between the two data distributions. Our theoretical bounds demonstrate our
approach leads to minimizing an upperbound for the target domain error.

##  3 Problem Formulation

Let $\mathcal{P}_{\mathcal{S}}$ be the data distribution corresponding to a
source domain, and $\mathcal{P}_{\mathcal{T}}$ be similarly the data
distribution corresponding to a target domain, with
$\mathcal{P}_{\mathcal{S}}$ being potentially different from
$\mathcal{P}_{\mathcal{T}}$. We consider a set of multi-channel images
$\bm{X_{S}}$ is randomly sampled from $\mathcal{P}_{\mathcal{S}}$ with
corresponding pixel-wise semantic labels $\bm{Y_{S}}$. Let $\bm{X_{T}}$ be a
set of images sampled from $\mathcal{P}_{\mathcal{T}}$, where we don’t have
access to the corresponding labels $\bm{Y_{T}}$. We consider that both
$\bm{X_{S}}$ and $\bm{X_{T}}$ are represented as images with real pixel values
in $\mathbb{R}^{W\times H\times C}$, where $W$ is the image width, $H$ is the
image height and $C$ is the number of channels. The labels
$\bm{Y_{S}},\bm{Y_{T}}$ share the same input space of label maps in
$\mathbb{R}^{W\times H}$ which makes the two domain relevant.

Our goal is to learn the parameters $\theta$ of a semantic segmentation model
$\phi_{\theta}(\cdot):\mathbb{R}^{W\times H\times
C}\rightarrow\mathbb{R}^{W\times H}$ capable of accurately predicting pixel-
level labels for images sampled from the target distribution
$\mathcal{P}_{\mathcal{T}}$. We can formulate this problem as a supervised
learning problem, where our goal is to minimize the target domain empirical
risk, achieved by $\theta^{*}=\arg\min_{\theta}\\{\mathbb{E}_{\bm{x}^{t}\sim
P_{\mathcal{T}}(X^{t})}(\mathcal{L}(f_{\theta}(x^{t}),y^{t})\\}$, where
$x^{t}\in\bm{X_{T}},y^{t}\in\bm{Y_{T}}$. The difficulty of the above
optimization stems from the lack of access to the label set $\bm{Y_{T}}$. To
overcome this challenge, we instead are provided access to the labeled source
domain $(\bm{X_{S}},\bm{Y_{S}})$, and then sequentially the target domain
$\bm{X_{T}}$. Many UDA algorithms consider that both domains are accessible
simultaneously but the source-free nature of our problem requires that once
the target images $\bm{X_{T}}$ become available, access to source domain
information becomes unavailable. This assumption is a practical assumption
because domain shift is often a temporal problem that arises after the initial
training phase.

To achieve training a generalizable model for the target domain, we need to
first train a model on the provided source dataset and then adapted to
generalize well on the target domain. Let $N$ be the size of the source
dataset $\bm{X_{S}}$, and let $(x_{i}^{s},y_{i}^{s})$ be the image/label pairs
from $\bm{X_{S}},\bm{Y_{S}}$. Consider $K$ to be the number of semantic
classes and $\mathbbm{1}_{a}(b)$ denote the indicator function determining
whether $a$ and $b$ are equal. Then, we learn the parameters that minimize
empirical risk on the source domain by optimizing the standard cross entropy
loss on the labeled dataset:

$\begin{split}&\hat{\theta}=\arg\min_{\theta}\\{\frac{1}{N}\sum_{i=1}^{N}\mathcal{L}_{ce}(\phi_{\theta}(x_{i}^{s}),y_{i}^{s})\\}\\\
&\mathcal{L}_{ce}(p,y)=-\frac{1}{WH}\sum_{w=1}^{W}\sum_{h=1}^{H}\sum_{k=1}^{K}\mathbbm{1}_{y_{wh}}(k)\log(p_{wh}),\end{split}$
(1)

The optimization setup in Eq. 1 ensures model generalization on inputs sampled
from $\mathcal{P}_{\mathcal{S}}$. In cases where $\mathcal{P}_{\mathcal{T}}$
differs from $\mathcal{P}_{\mathcal{S}}$ the model will not generalize on the
target domain, due to domain shift. To account for domain shift, we need to
map data points from both domains into an invariant feature space between the
two domains, without joint access to $\bm{X_{S}}$ and $\bm{X_{T}}$. To this
end, let $f(\cdot),g(\cdot),h(\cdot)$ be three parameterized sub-networks such
that $\phi=f\circ g\circ h$. In this composition, $f:\mathbb{R}^{W\times
H\times C}\rightarrow\mathbb{R}^{L}$ is an encoder sub-network,
$g:\mathbb{R}^{L}\rightarrow\mathbb{R}^{W\times H}$ is an up-scaling decoder,
and $\mathbb{R}^{W\times H}\rightarrow\mathbb{R}^{W\times H}$ is a semantic
classifier, where $L$ represents the dimension of the latent network
embedding. In order to create a shared source-target embedding space, our goal
is that the network $(f\circ g)(\cdot)$ embeds source and target samples in a
shared domain-agnostic embedding space. Under this condition, the classifier
$h(\cdot)$ trained on source domain samples will be able to generalize on
target inputs.

We can make the shared embedding space domain-agnostic by direct distribution
alignment between the embeddings of the two domains at the decoder output. As
previously explored in literature Gabourie et al. (2019), a suitable
distributional distance metric $\mathcal{D}(\cdot,\cdot)$ can be minimized
between the source and target domain data points at the network $(f\circ
g)(\cdot)$ output. However, because the source domains are inaccessible during
model adaptation, we cannot compute the distribution distance between the two
domains. Hence, directly minimizing $\mathcal{D}(f\circ g(\bm{X_{S}}),f\circ
g(\bm{X_{T}}))$ is not feasible. We need to develop a solution that relaxes
the need for access to the source domain samples during adaptation for domain
adaptation. Our core idea is to benefit from another distirbution that can be
served as a surrogate for the source domain distirbution. We describe our
source domain approximation approach and the choice for
$\mathcal{D}(\cdot,\cdot)$ in the next section.

##  4 Proposed Algorithm

We visually describe our method in Figure 1. The first step of our approach is
to fully train a segmentation model on the labeled source domain. As training
progresses on the source domain, the latent feature space will begin to
cluster into $K$ clusters, where each of the clusters encode one of the
semantic classes. If we use the output of the softmax layer as our embedding
space, the softmax classifier will be able to learn a linear decision function
based on the decoder output which leads to high label accuracy at the end of
this pre-training stage. After the source-training stage, we approximate the
source distribution via a learnt internal distribution. We use this as a
surrogate for $f\circ g(\bm{X_{S}})$ during adaptation.

Figure 1: Diagram of the proposed model adaptation approach (best seen in
color): (a) initial model training using the source domain labeled data, (b)
estimating the internal distribution as a GMM distribution in the embedding
space, (c) domain alignment is enforced by minimizing the distance between the
internal distribution samples and the target unlabeled samples, (d) domain
adaptation is enforced for the classifier module to fit correspondingly to the
GMM distribution.

As linear separation in the latent space is reached, we can produce an
approximation of the source domain distribution in the embedding space and
thus relax the need for having access to the source samples during adaptation.
We are interested in learning a $K$-modal approximation to the latent feature
distribution at the decoder output, $f\circ g(\mathcal{P}_{\mathcal{S}})$. Let
$p_{k},1\leq k\leq K$ represent the component of $f\circ
g(\mathcal{P}_{\mathcal{S}})$ corresponding to class $k$. For approximation
purposes, we will a Gaussian mixture model (GMM), with each semantic class
approximated by $T$ high Gaussians components, i.e, the GMM would have $kT$
components in total. Our choice for this approximation method stems from the
result by Goodfellow et al. (2016) that concludes with sufficient Gaussians,
any distribution can be approximated to vanishing error. While the GMM model
is traditionally learned in an unsupervised fashion, we can leverage our
knowledge of the source labels to partially supervise the process. As we have
access to the source domain labeled data, we can directly identify which
latent feature vectors correspond to each of the $K$ classes. Once the latent
feature vectors are pooled for each class, a $T$ component GMM is learned
using expectation maximization. During the learning process, we attempt to
avoid inclusion of outlier elements in the GMMs, which may lead to decreased
class separability in the latent space. This is a detrimental outcome for us,
as an increased separability improves the performance of the classification
layer. We thus only consider data samples which have high associated
classifier confidence. Let $\tau$ be this confidence threshold, and let
$S_{k}=\\{u_{i,j}|\exists(x^{s},y^{s}),f\circ
g(x^{s})_{i,j}=u_{i,j},y^{s}_{i,j}=k,f\circ g\circ h(x^{s})_{i,j,k}>\tau\\}$
be the set of source embedding feature vectors at the decoder output which
have label $k$ and on which the classifier assigns to $k$ more than $\tau$
probability mass. Then, we use expectation maximization to learn
$\alpha_{k,t},\mu_{k,t},\Sigma_{k,t},1\leq t\leq T$ as the parameters of the
$T$ components approximating class $k$. Thus, for each semantic class $k$, we
model the latent feature distribution $p_{k}$ as:

$p_{k}(z)=\sum_{t=1}^{T}\alpha_{k,t}\mathcal{N}(z|\mu_{k,t},\Sigma_{k,t})$ (2)

Learning a GMM approximation for each semantic class $k$ alleviates the need
for source domain access during adaptation. Once adaptation stage starts, the
source domain becomes unavailable, however we use the learnt GMM approximation
distribution as a surrogate. We achieve classifier generalization on the
target domain by minimizing a distributional distance metric between then GMM
approximation and the target latent embeddings. For this purpose, consider the
dataset $\mathcal{Z}=(\bm{X_{Z}},\bm{Y_{Z}})$ produced by sampling from the
GMM distribution, with $N^{z}=|\mathcal{Z}|$. Let $(x_{i}^{z},y_{i}^{z})$ be
embedding/label pairs from this dataset. We achieve distribution alignment by
empirically minimizing an appropriate distributional distance metric $D$
between samples from $\mathcal{Z}$ and from the target embeddings. In addition
to distribution alignment, we need to account for shifts in the classifier
input space between the internal distribution and the original source
embedding distribution. We account for such shifts by fine tuning the
classifier on labeled GMM samples. Our adaptation loss can be formalized as:

$\mathcal{L}_{adapt}=\mathcal{L}_{ce}(h(\bm{X_{Z}}),\bm{Y_{Z}})+\lambda\mathcal{L}_{D}(f\circ
g(\bm{X_{T}}),\bm{X_{Z}})$ (3)

for an appropriate choice of regularizer $\lambda$.

The first loss term in Eq. 3 is the cross entropy classifier fine-tuning loss
obtained for the GMM samples across the whole sampling dataset, i.e.,

$\begin{split}\mathcal{L}_{ce}(h(\bm{X_{Z}}),\bm{Y_{Z}})=-&\frac{1}{N^{z}}\sum_{i=1}^{N^{z}}\frac{1}{WH}\sum_{w=1}^{W}\sum_{h=1}^{H}\sum_{k=1}^{K}\\\
&\mathbbm{1}_{y_{i,wh}^{z}}(k)\log(h(x_{i}^{z})_{wh})\end{split}$ (4)

where $(x_{i}^{z},y_{i}^{z})$ are the $i$’th GMM data point in the sampling
dataset $\mathcal{Z}$. This loss terms helps maintaining model
generalizability as we perform distribution alignment.

The second loss term in Eq. 3, $\mathcal{L}_{D}$, represents the
distributional distance metric between the GMM in the latent space and the
target domain data embedding vectors. We choose the Sliced Wasserstein
Distance (SWD) as our choice for the distributional distance metric $D$. In
the context of domain adaptation, several distribution alignment metrics have
been previously used. Wu et al. (2018) propose an approach where the feature
space between source and target images is made similar by directly minimizing
the $\ell_{2}$-distance between feature vectors. $KL$-divergence has been used
in domain adaptation Yu et al. (2021) to detect noisy samples or target
samples from private classes. Wasserstein Distance (WD) has been explored as a
distributional distance metric Gabourie et al. (2019) by directly minimizing
the metric on the output feature space for a source and target encoder. While
study of appropriate distributional distance metrics is still ongoing, the WD
aims to find the optimal way of moving mass between two distributions, and
thus is tied to the geometry of the data. This has lead to the WD offering
improved stability when used in a number of deep learning and domain
adaptation tasks Arjovsky et al. (2017); Kolouri et al. (2016); Solomon et al.
(2015); Bhushan Damodaran et al. (2018); Rostami et al. (2019b, a); Xu et al.
(2020); Li et al. (2020a); Rostami and Galstyan (2023a). The WD metric Kolouri
et al. (2019) between two distributions $P$ and $Q$ are defined as:

$W_{d}(P,Q)=(\inf_{L\in\mathcal{L}(P,Q)}\int\lVert
x-y\rVert^{d}dL(x,y))^{\frac{1}{d}}$ (5)

where $\mathcal{L}(P,Q)$ represents all transportation plans between $P$ and
$Q$, i.e., all joint distributions with marginals $P$ and $Q$. The $WD$ metric
offers a closed form solution only when $P$ and $Q$ are one dimensional
distributions Kolouri et al. (2019). For higher dimensions, we need to solve a
linear program. While employing the WD has desirable properties, solving a
linear program at every optimization step can lead to significant
computational costs in the adaptation phase of UDA. To alleviate this issue we
employ the SWD, an alternative for the WD that is fully differentiable, yet
has a closed form formula. Computing the SWD between to high dimensional
distributions involves repeatedly projecting them along random on dimensional
projection directions, obtaining one dimensional marginals for which
computation of the WD which has a closed form solution. This process allows
for an end-to-end differentiation via gradient based methods, such as
Stochastic Gradient Descent Bottou et al. (2018). Averaging one dimensional WD
over sufficient random projction directions will produce a closed-form
approximation to the high dimensional WD objective. We can the distributional
distance term in Eq. 3 for two distributions $p,q$ as follows:

$\begin{split}\mathcal{L}_{D}(p,q)=\mathcal{SWD}_{d}(p,q)=\frac{1}{J}(\sum_{i=1}^{J}\lVert\gamma_{i}p-\gamma_{i}q\rVert^{d})^{\frac{1}{d}}\end{split}$
(6)

where $\mathcal{SWD}_{d}$ represents the $d$ order SWD, $J$ represents the
number of random projection to be averaged, and $\gamma_{i}$ is one of the $J$
random projections. In our approach, we will choose $\mathcal{SWD}_{2}$ due to
ease of computation and comparable performance to higher order choices of $d$.
Pseudocode for our approach, named Model Adaptation for Source-Free Semantic
Segmentation (MAS3) is provided in Algorithm 1.

Algorithm 1 $\mathrm{MAS^{3}}\left(\lambda,\tau\right)$

1:Initial Training:

2: Input: source domain dataset
$\mathcal{D}_{\mathcal{S}}=(\bm{X}_{\mathcal{S}},\bm{Y}_{\mathcal{S}})$,

3: Training on Source Domain:

4:
$\hat{\theta}_{0}=(\hat{\bm{w}}_{0},\hat{\bm{v}}_{0}\hat{\bm{u}}_{0})=\arg\min_{\theta}\sum_{i}\mathcal{L}(f_{\theta}(\bm{x}_{i}^{s}),\bm{y}_{i}^{s})$

5: internal Distribution Estimation:

6: Estimate the GMM parameters

7:Model Adaptation:

8: Input: target dataset $\mathcal{D}_{\mathcal{T}}=(\bm{X}_{\mathcal{T}})$

9: Pseudo-Dataset Generation:

10:
$\mathcal{D}_{\mathcal{P}}=(\textbf{Z}_{\mathcal{P}},\textbf{Y}_{\mathcal{P}})=$

11:
$([\bm{z}_{1}^{p},\ldots,\bm{z}_{N}^{p}],[\bm{y}_{1}^{p},\ldots,\bm{y}_{N}^{p}])$,
where:

12: $\bm{z}_{i}^{p}\sim\hat{p}_{J}(\bm{z}),1\leq i\leq N_{p}$

13: $\bm{y}_{i}^{p}=\arg\max_{j}\\{h_{\hat{\bm{w}}_{0}}(\bm{z}_{i}^{p})\\}$,
$p_{ip}>\tau$

14:for $itr=1,\ldots,ITR$ do

15: draw random batches from $\mathcal{D}_{\mathcal{T}}$ and
$\mathcal{D}_{\mathcal{P}}$

16: Update the model by solving Eq. (3)

17:end for

##  5 Theoretical Analysis

We prove Algorithm 1 can lead to improving the model generalization on the
target domain by minimizing an upperbound for the empirical risk of the model
on the target domain. For such a result, we need to tie model generalization
on the source domain to the distributional distance between the source and
target domains. For this purpose, we use the framework developed by Redko and
Sebban (2017) designed for upper bounding target risk with respect to the
distance between the source and target domains in the classic joint UDA
process. We rely on the following Theorem 2 from Redko and Sebban (2017) in
our approach:

######  Theorem 1.

(Redko and Sebban (2017)) For the variables defined under Theorem 8, the
following distribution alignment inequality loss holds:

$\begin{split}\epsilon_{T}\leq&\epsilon_{S}+W(\hat{\mu}_{S},\hat{\mu}_{T})+\sqrt{\big{(}2\log(\frac{1}{\xi})/\zeta\big{)}}\big{(}\sqrt{\frac{1}{N^{s}}}+\sqrt{\frac{1}{N^{t}}}\big{)}+e_{\mathcal{C}}(h^{*})\end{split}$
(7)

The above relation characterises target error after source training, and does
not consider our specific scenario of using an intermediate distribution. We
adapt this bound for Algorithm 1 to derive the following theorem:

######  Theorem 2.

Consider the space of all possible hypotheses $\mathcal{H}$ applicable to the
proposed segmentation task. Let $\epsilon_{S}(h),\epsilon_{T}(h)$ represent
the expected source and target risk for hypothesis $h$, respectively. Let
$\hat{\mu}_{S},\hat{\mu}_{Z},\hat{\mu}_{T}$ be the empirical mean of the
embedding space for the source, intermediate and target datasets respectively.
Let $W(\cdot,\cdot)$ represented the Wasserstein distance, and let $\xi,\zeta$
be appropriately defined constants. Consider $e_{\mathcal{C}}(h)$ to be the
combined error of a hypothesis $h$ on both the source and target domains, i.e.
$\epsilon_{S}(h)+\epsilon_{T}(h)$, and let $h^{*}$ be the minimizer for this
function. Then, for a model $h$, the following results holds:

$\begin{split}\epsilon_{T}(h)\leq&\epsilon_{S}(h)+W(\hat{\mu}_{S},\hat{\mu}_{Z})+W(\hat{\mu}_{Z},\hat{\mu}_{T})+\sqrt{\big{(}2\log(\frac{1}{\xi})/\zeta\big{)}}\big{(}\sqrt{\frac{1}{N^{s}}}+\sqrt{\frac{1}{N^{t}}}\big{)}+e_{\mathcal{C}}(h^{*})\end{split}$
(8)

Proof: We expand the second term of Eq. 7. Given $W(\cdot,\cdot)$ is a convex
optimization problem, we can use the triangle inequality as follows

$W(\hat{\mu}_{S},\hat{\mu}_{T})\leq
W(\hat{\mu}_{S},\hat{\mu}_{Z})+W(\hat{\mu}_{Z}+\hat{\mu}_{T})$ (9)

Combining Eq. 9 and Eq. 7 leads to the result in Theorem 8.

The above results provides a justification Algorithm 1 is able to minimize the
right hand side of the Eq. 8. The first term is minimized during the initial
training phase on the source domain. Note that, as expected, the performance
on the target domain cannot be better than the performance on the source
domain. We conclude that the model we select, should be a good model to learn
the source domain. The second term represents the WD distance between the
source and sampling dataset. This distance will be small if the GMM
approximation of the source domain will be successful. As we explained before,
if select a large enough $T$, we can make this term negligible. The third term
is the WD distance e between the sampling dataset and the target domain
dataset. This term is directly minimized by the adaptation loss that we use to
align the distribution. The term $1-\tau$ is a constant directly dependent on
the confidence threshold $\tau$, which we choose close to $1$. The fourth term
is directly dependent on the dataset size, and becomes small when a large
number of samples is present. Finally, the last term is a constant indicating
the difficulty of the adaptation problem.

##  6 Experimental Validation

We validate the proposed algorithm using common UDA benchmarks for semantic
segmentation. Our implementation code is available as a supplement at
https://github.com/serbanstan/mas3-continual.

###  6.1 Experimental Setup

####  6.1.1 Datasets

We follow the UDA literature to evaluate our approach. We consider three
common datasets used in semantic segmentation literature: GTA5 Richter et al.
(2016), SYNTHIA Ros et al. (2016) and Cityscapes Cordts et al. (2016). Both
GTA5 and SYNTHIA are datasets consisting of artificially generated street
images, with 24966 and 9400 instances respectively. Cityscapes is composed of
real world images recorded in several European cities, consisting of 2957
training images and 500 test images. We can see that the diversity of sizes of
these datasets which demonstrates the challenge of data annotation for
semantic segmentation tasks. For all three datasets, images are processed and
resized to a standard shape of $512\times 1024$.

Following the literature, we consider two adaptation tasks designed to
evaluate model adaptation performance when the training set consists of
artificial images, and the test set consists of real world images. For both
SYNTHIA$\rightarrow$Cityscapes and GTA5$\rightarrow$Cityscapes, we evaluate
performance under two scenarios, when $13$ or $19$ semantic classes are
available.

####  6.1.2 Implementation and Training Details

We use a DeepLabV3 architecture Chen et al. (2017a) with a VGG16 encoder
Simonyan and Zisserman (2014) for our CNN architecture. The decoder is
followed by a $1\times 1$ convolution softmax classifier. We choose a batch
size of $4$ images for training, and use the Adam optimizer with learning rate
of $1e-4$ for gradient propagation. For adaptation, we keep the same optimizer
parameters as for training. We choose $100$ projections in our SWD
computation, and set the regularization parameter $\lambda$ to $.5$. We
perform training for $100k$ iterations on the source domain and then fpr
adaptation we perform $50k$ iterations.

When approximating the GMM components, we chose the confidence parameter
$\tau$ to be $0.95$. We observe higher values of $\tau$ to be correlated with
increased performance, as expected from our theorem, and conclude that a
$\tau$ setting above $0.9$ will lead to similar target performance.

We run our experiments on a NVIDIA Titan XP GPU. Given that our method relies
on distributional alignment, the label distribution between target batches may
vary significantly between different batches. As the batch distribution
approaches the target label distribution as the batch size increases, we use
the oracle label distribution per batch when sampling from the GMM, which can
be avoided if sufficient GPU memory becomes present. Experimental code is
provided with the current submission.

####  6.1.3 Baselines for Comparison

Source-free model adaptation algorithms for semantic segmentation have been
only recently explored. Thus, due most UDA algorithms being designed for joint
training, besides source-free approaches we also include both pioneer and
recent UDA image segmentation method to be representative of the literature.
We have compared our performance against the adversarial learning-based UDA
methods: GIO-Ada Chen et al. (2019), ADVENT Vu et al. (2019), AdaSegNet Tsai
et al. (2018), TGCF-DA+SE Choi et al. (2019), PCEDA Yang et al. (2020), and
CyCADA Hoffman et al. (2018b). We have also included methods that are based on
direct distributional matching which are more similar to $MAS^{3}$: FCNs in
the Wild Hoffman et al. (2016), CDA Zhang et al. (2017), DCAN Wu et al.
(2018), SWD Lee et al. (2019), Cross-City Chen et al. (2017b). Source-free
methods include GenAdapt Kundu et al. (2021) and SFDA Liu et al. (2021).

###  6.2 Comparison Results

###  6.3 SYNTHIA$\rightarrow$Cityscapes Task

We provide quantitative and qualitative results for this task in Table 1. We
report the performance our method produces on the
SYNTHIA$\rightarrow$CITYSCAPES adaptation task along with other baselines.
Notably, even when confronted with a more challenging learning setting,
$MAS^{3}$ demonstrates superior performance compared to the majority of
classic UDA methods that have access to the source domain data during model
adaptation. It is essential to highlight that some recently developed UDA
methods leveraging adversarial learning surpass our approach in performance;
however, it’s worth noting that these methods often incorporate an additional
form of regularization, aside from probability matching and are unable to
address UDA when the source domain data is missing.

In an overarching evaluation, $MAS^{3}$ exhibits commendable performance,
particularly when compared to UDA methods that rely on source samples.
Furthermore, our method excels in specific crucial categories, such as the
accurate detection of traffic lights, where it outperforms its counterparts.
These results underscore the robustness and effectiveness of $MAS^{3}$ in
handling challenging learning scenarios and achieving notable performance,
especially in key object categories. We conclude that $MAS^{3}$ can be used to
address classic UDA setting reasonably well.

Method Adv. road sdwlk bldng light sign vgttn sky person rider car bus mcycl
bcycl mIoU Source Only (VGG16) N 6.4 17.7 29.7 0.0 7.2 30.3 66.8 51.1 1.5 47.3
3.9 0.1 0.0 20.2 FCNs in the Wild N 11.5 19.6 30.8 0.1 11.7 42.3 68.7 51.2 3.8
54.0 3.2 0.2 0.6 22.9 CDA N 65.2 26.1 74.9 3.7 3.0 76.1 70.6 47.1 8.2 43.2
20.7 0.7 13.1 34.8 DCAN N 9.9 30.4 70.8 6.70 23.0 76.9 73.9 41.9 16.7 61.7
11.5 10.3 38.6 36.4 SWD N 83.3 35.4 82.1 12.2 12.6 83.8 76.5 47.4 12.0 71.5
17.9 1.6 29.7 43.5 Cross-City Y 62.7 25.6 78.3 1.2 5.4 81.3 81.0 37.4 6.4 63.5
16.1 1.2 4.6 35.7 GIO-Ada Y 78.3 29.2 76.9 10.8 17.2 81.7 81.9 45.8 15.4 68.0
15.9 7.5 30.4 43.0 ADVENT Y 67.9 29.4 71.9 0.6 2.6 74.9 74.9 35.4 9.6 67.8
21.4 4.1 15.5 36.6 AdaSegNet Y 78.9 29.2 75.5 0.1 4.8 72.6 76.7 43.4 8.8 71.1
16.0 3.6 8.4 37.6 TGCF-DA+SE Y 90.1 48.6 80.7 3.2 14.3 82.1 78.4 54.4 16.4
82.5 12.3 1.7 21.8 46.6 PCEDA Y 79.7 35.2 78.7 10.0 28.9 79.6 81.2 51.2 25.1
72.2 24.1 16.7 50.4 48.7 SFDA SF(Y) 81.9 44.9 81.7 3.3 10.7 86.3 89.4 37.9
13.4 80.6 25.6 9.6 31.3 45.89 GenAdapt SF(Y) 89.9 48.8 80.9 19.5 26.2 83.7
84.9 57.4 17.8 75.6 28.9 4.3 17.2 48.9 MAS3 (Ours) SF(N) 74.8 51.6 71.5 20.4
32.3 73.0 75.3 48.9 19.7 66.3 25.7 10.1 40.8 47.0

Table 1: Model adaptation comparison results for the
SYNTHIA$\rightarrow$Cityscapes task. We have used DeepLabV3 Chen et al.
(2017a) as the feature extractor with a VGG16 Simonyan and Zisserman (2014)
backbone. The first row presents the source-trained model performance prior to
adaptation to demonstrate effect of initial knowledge transfer from the source
domain.

###  6.4 GTA5$\rightarrow$Cityscapes Task

We present the quantitative outcomes for this particular task, detailed in
Table 2. It is noteworthy that we observe a more competitive performance in
this task, and yet the overall trend in the performance comparison remains
similar. These findings highlight the versatility of our proposed method,
$MAS^{3}$. While our primary motivation lies in achieving source-free model
adaptation, these results indicate that $MAS^{3}$ can effectively function as
a joint-training UDA algorithm. We conclude our method manages to achieve
state-of-the-art performance even in a setting involving a larger number of
semantic classes. This capability underscores the robustness and adaptability
of $MAS^{3}$ in diverse scenarios, making it a versatile solution that goes
beyond its original focus on source-free model adaptation.

Method road sdwk bldng wall fence pole light sign vgttn trrn sky person rider
car truck bus train mcycl bcycl mIoU Source (VGG16) 25.9 10.9 50.5 3.3 12.2
25.4 28.6 13.0 78.3 7.3 63.9 52.1 7.9 66.3 5.2 7.8 0.9 13.7 0.7 24.9 FCNs Wld.
70.4 32.4 62.1 14.9 5.4 10.9 14.2 2.7 79.2 21.3 64.6 44.1 4.2 70.4 8.0 7.3 0.0
3.5 0.0 27.1 CDA 74.9 22.0 71.7 6.0 11.9 8.4 16.3 11.1 75.7 13.3 66.5 38.0 9.3
55.2 18.8 18.9 0.0 16.8 14.6 28.9 DCAN 82.3 26.7 77.4 23.7 20.5 20.4 30.3 15.9
80.9 25.4 69.5 52.6 11.1 79.6 24.9 21.2 1.30 17.0 6.70 36.2 SWD 91.0 35.7 78.0
21.6 21.7 31.8 30.2 25.2 80.2 23.9 74.1 53.1 15.8 79.3 22.1 26.5 1.5 17.2 30.4
39.9 CyCADA 85.2 37.2 76.5 21.8 15.0 23.8 22.9 21.5 80.5 31.3 60.7 50.5 9.0
76.9 17.1 28.2 4.5 9.8 0.0 35.4 ADVENT 86.9 28.7 78.7 28.5 25.2 17.1 20.3 10.9
80.0 26.4 70.2 47.1 8.4 81.5 26.0 17.2 18.9 11.7 1.6 36.1 AdaSegNet 86.5 36.0
79.9 23.4 23.3 23.9 35.2 14.8 83.4 33.3 75.6 58.5 27.6 73.7 32.5 35.4 3.9 30.1
28.1 42.4 TGCF-DA+SE 90.2 51.5 81.1 15.0 10.7 37.5 35.2 28.9 84.1 32.7 75.9
62.7 19.9 82.6 22.9 28.3 0.0 23.0 25.4 42.5 PCEDA 90.2 44.7 82.0 28.4 28.4
24.4 33.7 35.6 83.7 40.5 75.1 54.4 28.2 80.3 23.8 39.4 0.0 22.8 30.8 44.6 SFDA
81.8 35.4 82.3 21.6 20.2 25.3 17.8 4.7 80.7 24.6 80.4 50.5 9.2 78.4 26.3 19.8
11.1 6.7 4.3 35.86 GenAdapt 90.1 44.2 81.7 31.6 19.2 27.5 29.6 26.4 81.3 34.7
82.6 52.5 24.9 83.2 25.3 41.9 8.6 15.7 32.2 43.4 MAS3 (Ours) 75.8 55.6 72.9
20.9 24.7 20.5 30.5 39.8 80.0 36.9 77.9 51.9 22.4 77.3 26.5 45.2 22.6 18.8
51.7 44.8

Table 2: Domain adaptation results for different methods for the
GTA5$\rightarrow$Cityscapes task.

###  6.5 Analytic Experiment

We offer additional experiments to offer a better insight about our algorithm.
In Figure 2, we have provided visualizations of representative frames from the
Cityscapes dataset for the SYNTHIA$\rightarrow$Cityscapes task. These frames
are segmented using our model both prior to and after adaptation, and are
juxtaposed with the corresponding ground-truth manual annotations for each
image. Through visual inspection, it becomes evident that our method brings
about significant improvements in image segmentation, transitioning from
source-only segmentation to post-adaptation segmentation. This improvement is
particularly notable in semantic classes such as sidewalk, road, and cars for
the model initially trained on GTA5 which are particularly important classes
in autonomous driving applications. The visual comparison highlights the
considerable enhancement achieved in performance. To further complement these
findings, examples of segmented frames for the SYNTHIA$\rightarrow$Cityscapes
task are included in Figure 3, revealing similar observations. These
visualizations collectively underscore the effectiveness of our method in
enhancing image segmentation across diverse datasets.

Figure 2: Qualitative performance: examples of the segmented frames for
SYNTHIA$\rightarrow$Cityscapes using the MAS3 method. Left to right: real
images, manually annotated images, source-trained model predictions,
predictions based on our method.  Figure 3: Qualitative performance: examples
of the segmented frames for SYNTHIA$\rightarrow$Cityscapes and
GTA5$\rightarrow$Cityscapes using the MAS3 method. From left to right column:
real images, manually annotated images, source-trained model predictions,
predictions based on our method.

We study the effect our algorithm on data distirbution in the embedding space.
To validate the alignment achieved by our solution, we employed the UMAP
McInnes et al. (2018) visualization tool to reduce the dimensionality of data
representations in the embedding space to two for 2D visualization. Figure 4
visually represents samples from the internal distribution, along with the
target domain data both before and after adaptation for the
GTA5$\rightarrow$Cityscapes task. In this figure, each point corresponds to a
single data point, and each color represents a semantic class cluster. Upon
comparing Figure 4 (b) and Figure 4 (c) with Figure 4 (a), a noticeable
observation emerges. The semantic classes in the target domain exhibit greater
separation and similarity to the internal distribution after the model
adaptation process. This signifies a substantial reduction in domain
discrepancy facilitated by $MAS^{3}$, where the source and target domain
distributions align indirectly through the intermediate internal distribution
in the embedding space, as originally anticipated.

(a) GMM samples

(b) Pre-adaptation

(c) Post-adaptation

Figure 4: Indirect distribution matching in the embedding space: (a) drawn
samples from the GMM trained on the SYNTHIA distribution, (b) representations
of the Cityscapes validation samples prior to model adaptation (c)
representation of the Cityscapes validation samples after domain alignment.

###  6.6 Sensitivity Analysis Experiments

An inherent advantage of our algorithm, in contrast to methods relying on
adversarial learning, lies in its simplicity and dependence on only a few
hyperparameters. We study the sensitivity of performance with respect to these
hyperparameters. The primary hyperparameters specific to our algorithm are
$\lambda$ and $\tau$ constants. Through experimentation, we have observed that
the performance of $MAS^{3}$ remains stable with respect to the trade-off
parameter $\lambda$. This stability is expected as the $\mathcal{L}_{ce}$ loss
term remains relatively small from the outset due to prior training on the
source domain and then optimization mostly reduces the cross-domain alignment
loss term. We further delved into the impact of the confidence hyperparameter
$\tau$. Figure 5 visually illustrates the fitted Gaussian Mixture Model (GMM)
on the source internal distribution for three different values of $\tau$.
Notably, when $\tau=0$, the fitted GMM clusters appear cluttered. However, as
we increment the threshold $\tau$ and selectively use samples for which the
classifier demonstrates confidence, the fitted GMM represents well-separated
semantic classes. This increase in interclass clusters in knowledge transfer
from the source domain is evident as semantic classes become more distinctly
defined. This empirical exploration aligns with our earlier deduction
regarding the significance of $\tau$, as outlined in our Theorem, thereby
validating the theoretical analysis. The experimental findings underscore the
robustness and effectiveness of our method across different hyperparameter
configurations.

(a) $\tau=0$, mIoU=$41.8$

(b) $\tau=0.8$, mIoU=$42.7$

(c) P$\tau=0.97$, mIoU=$44.7$

Figure 5: Ablation experiment to study effect of $\tau$ on the GMM learnt in
the embedding space: (a) all samples are used; adaptation mIoU=41.8, (b) a
portion of samples is used; adaptation mIoU=42.7, (c) samples with high model-
confidence are used; adaptation mIoU=44.7

We also extend our empirical investigation to analyze the balance between the
quantity of projections employed for computing SWD for distribution alignment
and the resulting UDA performance on the target domain. Results for this
experiment is presented in Figure 6. We observe that the performance remains
robust even with a modest number of projections, such as $5$. Moreover,
empirical evidence indicates that the performance tends to plateau after
approximately $50$ projections. It is noteworthy that the runtime complexity
scales linearly concerning the number of projections, as the only operation
involving the one-dimensional Wasserstein computations is the subsequent
averaging process. In consideration of these findings, the results presented
throughout the rest of the paper are based on utilizing $100$ projections.
This choice is made to ensure that we operate within a favorable regime
concerning adaptation performance while maintaining a balanced runtime to
compute SWD. By doing so, we aim to achieve desirable performance with a
decent computational load.

Figure 6: Performance accuracy versus the number of projections used to
compute SWD using the SYNTHIA$\rightarrow$Cityscapes task.

##  7 Conclusions

We devised an algorithm tailored for adapting an image segmentation model to
achieve generalization across new domains, a process facilitated solely
through the use of unlabeled data for the target domain during training. At
the core of our approach is the utilization of an intermediate multi-modal
internal distribution, strategically employed to minimize the distributional
cross-domain discrepancy within a shared embedding space. To estimate this
internal distribution, we employ a parametric Gaussian Mixture Model (GMM)
distribution. Through rigorous experimentation on benchmark tasks, our
algorithm has demonstrated its effectiveness, yielding competitive performance
that stands out even when compared to existing UDA algorithms rooted in joint-
domain model training strategies. The results underscore the robustness and
efficacy of our approach in achieving domain adaptation for image segmentation
tasks, particularly in scenarios where only unlabeled data is available for
training. Future exploration inlcudes partial domain adaptation settings in
which the source and the target domain do not share the same classes.

## References

  * Arjovsky et al. (2017) Arjovsky, M., Chintala, S., and Bottou, L. (2017).  Wasserstein GAN.  _arXiv preprint arXiv:1701.07875_
  * Benjdira et al. (2019) Benjdira, B., Bazi, Y., Koubaa, A., and Ouni, K. (2019).  Unsupervised domain adaptation using generative adversarial networks for semantic segmentation of aerial images.  _Remote Sensing_ 11, 1369
  * Bhushan Damodaran et al. (2018) Bhushan Damodaran, B., Kellenberger, B., Flamary, R., Tuia, D., and Courty, N. (2018).  Deepjdot: Deep joint distribution optimal transport for unsupervised domain adaptation.  In _Proceedings of the European Conference on Computer Vision (ECCV)_. 447–463
  * Bottou et al. (2018) Bottou, L., Curtis, F. E., and Nocedal, J. (2018).  Optimization methods for large-scale machine learning.  _Siam Review_ 60, 223–311
  * Bousmalis et al. (2017) Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., and Krishnan, D. (2017).  Unsupervised pixel-level domain adaptation with generative adversarial networks.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 3722–3731
  * Chen et al. (2018a) Chen, C., Dou, Q., Chen, H., and Heng, P.-A. (2018a).  Semantic-aware generative adversarial nets for unsupervised domain adaptation in chest x-ray segmentation.  In _International workshop on machine learning in medical imaging_ (Springer), 143–151
  * Chen et al. (2017a) Chen, L.-C., Papandreou, G., Schroff, F., and Adam, H. (2017a).  Rethinking atrous convolution for semantic image segmentation.  In _arXiv:1706.05587 [cs.CV]_
  * Chen et al. (2018b) Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018b).  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In _ECCV_
  * Chen et al. (2019) Chen, Y., Li, W., Chen, X., and Gool, L. V. (2019).  Learning semantic segmentation from synthetic data: A geometrically guided input-output adaptation approach.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 1841–1850
  * Chen et al. (2017b) Chen, Y.-H., Chen, W.-Y., Chen, Y.-T., Tsai, B.-C., Wang, Y.-C. F., and Sun, M. (2017b).  No more discrimination: Cross city adaptation of road scene segmenters.  In _arXiv:1704.08509 [cs.CV]_
  * Choi et al. (2019) Choi, J., Kim, T., and Kim, C. (2019).  Self-ensembling with gan-based data augmentation for domain adaptation in semantic segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_. 6830–6840
  * Cordts et al. (2016) Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., et al. (2016).  The cityscapes dataset for semantic urban scene understanding.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 3213–3223
  * Dredze and Crammer (2008) Dredze, M. and Crammer, K. (2008).  Online methods for multi-domain learning and adaptation.  In _Proceedings of the Conference on Empirical Methods in Natural Language Processing_ (Association for Computational Linguistics), 689–697
  * Feng et al. (2020) Feng, D., Haase-Schütz, C., Rosenbaum, L., Hertlein, H., Glaeser, C., Timm, F., et al. (2020).  Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges.  _IEEE Transactions on Intelligent Transportation Systems_ 22, 1341–1360
  * Gabourie et al. (2019) Gabourie, A. J., Rostami, M., Pope, P. E., Kolouri, S., and Kim, K. (2019).  Learning a domain-invariant embedding for unsupervised domain adaptation using class-conditioned distribution alignment.  In _2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)_. 352–359
  * Goodfellow et al. (2016) Goodfellow, I., Bengio, Y., and Courville, A. (2016).  _Deep Learning_ (MIT Press).  http://www.deeplearningbook.org
  * Goodfellow et al. (2014) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. (2014).  Generative adversarial nets.  In _Advances in Neural Information Processing Systems_. 2672–2680
  * Goodfellow et al. (2020) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., et al. (2020).  Generative adversarial networks.  _Communications of the ACM_ 63, 139–144
  * Guan and Liu (2021) Guan, H. and Liu, M. (2021).  Domain adaptation for medical image analysis: a survey.  _IEEE Transactions on Biomedical Engineering_ 69, 1173–1185
  * He et al. (2015) He, K., Zhang, X., Ren, S., and Sun, J. (2015).  Deep residual learning for image recognition.  _CoRR_ abs/1512.03385
  * Hoffman et al. (2018a) Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., et al. (2018a).  Cycada: Cycle-consistent adversarial domain adaptation.  In _International conference on machine learning_ (Pmlr), 1989–1998
  * Hoffman et al. (2018b) Hoffman, J., Tzeng, E., Park, T., Zhu, J.-Y., Isola, P., Saenko, K., et al. (2018b).  CyCADA: Cycle-consistent adversarial domain adaptation.  In _International Conference on Machine Learning_. 1989–1998
  * Hoffman et al. (2016) Hoffman, J., Wang, D., Yu, F., and Darrell, T. (2016).  Fcns in the wild: Pixel-level adversarial and constraint-based adaptation.  _arXiv preprint arXiv:1612.02649_
  * Hong et al. (2018) Hong, W., Wang, Z., Yang, M., and Yuan, J. (2018).  Conditional generative adversarial network for structured domain adaptation.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 1335–1344
  * Hung et al. (2018) Hung, W.-C., Tsai, Y.-H., Liou, Y.-T., Lin, Y.-Y., and Yang, M.-H. (2018).  Adversarial learning for semi-supervised semantic segmentation.  _arXiv preprint arXiv:1802.07934_
  * Jain and Learned-Miller (2011) Jain, V. and Learned-Miller, E. (2011).  Online domain adaptation of a pre-trained cascade of classifiers.  In _Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition_. 577–584
  * Jian and Rostami (2023) Jian, D. and Rostami, M. (2023).  Unsupervised domain adaptation for training event-based networks using contrastive learning and uncorrelated conditioning.  _arXiv preprint arXiv:2303.12424_
  * Kalake et al. (2021) Kalake, L., Wan, W., and Hou, L. (2021).  Analysis based on recent deep learning approaches applied in real-time multi-object tracking: a review.  _IEEE Access_ 9, 32650–32671
  * Kim et al. (2021) Kim, Y., Cho, D., Han, K., Panda, P., and Hong, S. (2021).  Domain adaptation without source data.  _IEEE Transactions on Artificial Intelligence_ 2, 508–518
  * Kolouri et al. (2019) Kolouri, S., Nadjahi, K., Simsekli, U., Badeau, R., and Rohde, G. (2019).  Generalized sliced wasserstein distances.  _Advances in neural information processing systems_ 32
  * Kolouri et al. (2016) Kolouri, S., Zou, Y., and Rohde, G. K. (2016).  Sliced wasserstein kernels for probability distributions.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 5258–5267
  * Kundu et al. (2021) Kundu, J. N., Kulkarni, A., Singh, A., Jampani, V., and Babu, R. V. (2021).  Generalize then adapt: Source-free domain adaptive semantic segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_. 7046–7056
  * Kundu et al. (2020a) Kundu, J. N., Venkat, N., Babu, R. V., et al. (2020a).  Universal source-free domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4544–4553
  * Kundu et al. (2020b) Kundu, J. N., Venkat, N., V, R. M., and Babu, R. V. (2020b).  Universal source-free domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_
  * LeCun et al. (1995) LeCun, Y., Bengio, Y., et al. (1995).  Convolutional networks for images, speech, and time series.  _The handbook of brain theory and neural networks_ 3361, 1995
  * Lee et al. (2019) Lee, C.-Y., Batra, T., Baig, M. H., and Ulbricht, D. (2019).  Sliced wasserstein discrepancy for unsupervised domain adaptation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 10285–10295
  * Li et al. (2020a) Li, M., Zhai, Y.-M., Luo, Y.-W., Ge, P.-F., and Ren, C.-X. (2020a).  Enhanced transport distance for unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 13936–13944
  * Li et al. (2020b) Li, X., Gu, Y., Dvornek, N., Staib, L. H., Ventola, P., and Duncan, J. S. (2020b).  Multi-site fmri analysis using privacy-preserving federated learning and domain adaptation: Abide results.  _Medical Image Analysis_ 65, 101765
  * Lin et al. (2017) [Dataset] Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017).  Feature pyramid networks for object detection
  * Liu et al. (2021) Liu, Y., Zhang, W., and Wang, J. (2021).  Source-free domain adaptation for semantic segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 1215–1224
  * Long et al. (2015) Long, J., Shelhamer, E., and Darrell, T. (2015).  Fully convolutional networks for semantic segmentation.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 3431–3440
  * Luc et al. (2016) Luc, P., Couprie, C., Chintala, S., and Verbeek, J. (2016).  Semantic segmentation using adversarial networks.  In _NIPS Workshop on Adversarial Training_
  * McInnes et al. (2018) McInnes, L., Healy, J., Saul, N., and Großberger, L. (2018).  UMAP: Uniform manifold approximation and projection.  _Journal of Open Source Software_ 3, 861
  * Murez et al. (2017) Murez, Z., Kolouri, S., Kriegman, D. J., Ramamoorthi, R., and Kim, K. (2017).  Image to image translation for domain adaptation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_
  * Redko and Sebban (2017) Redko, A., I.and Habrard and Sebban, M. (2017).  Theoretical analysis of domain adaptation with optimal transport.  In _Joint European Conference on Machine Learning and Knowledge Discovery in Databases_ (Springer), 737–753
  * Richter et al. (2016) Richter, S. R., Vineet, V., Roth, S., and Koltun, V. (2016).  Playing for data: Ground truth from computer games.  In _European conference on computer vision_ (Springer), 102–118
  * Romera et al. (2019) Romera, E., Bergasa, L. M., Yang, K., Alvarez, J. M., and Barea, R. (2019).  Bridging the day and night domain gap for semantic segmentation.  In _2019 IEEE Intelligent Vehicles Symposium (IV)_ (IEEE), 1312–1318
  * Ronneberger et al. (2015) [Dataset] Ronneberger, O., Fischer, P., and Brox, T. (2015).  U-net: Convolutional networks for biomedical image segmentation
  * Ros et al. (2016) Ros, G., Sellart, L., Materzynska, J., Vazquez, D., and Lopez, A. M. (2016).  The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 3234–3243
  * Rostami (2019) Rostami, M. (2019).  _Learning Transferable Knowledge Through Embedding Spaces_.  Ph.D. thesis, University of Pennsylvania
  * Rostami (2021) Rostami, M. (2021).  Lifelong domain adaptation via consolidated internal distribution.  _Advances in neural information processing systems_ 34, 11172–11183
  * Rostami (2022) Rostami, M. (2022).  Increasing model generalizability for unsupervised visual domain adaptation.  In _Conference on Lifelong Learning Agents_ (PMLR), 281–293
  * Rostami et al. (2023) Rostami, M., Bose, D., Narayanan, S., and Galstyan, A. (2023).  Domain adaptation for sentiment analysis using robust internal representations.  In _Findings of the Association for Computational Linguistics: EMNLP 2023_. 11484–11498
  * Rostami and Galstyan (2023a) Rostami, M. and Galstyan, A. (2023a).  Overcoming concept shift in domain-aware settings through consolidated internal distributions.  In _Proceedings of the AAAI conference on artificial intelligence_. vol. 37, 9623–9631
  * Rostami and Galstyan (2023b) Rostami, M. and Galstyan, A. (2023b).  Thirty-second international joint conference on artificial intelligence.  In _arXiv preprint arXiv:2110.04662_
  * Rostami et al. (2018) Rostami, M., Huber, D., and Lu, T.-C. (2018).  A crowdsourcing triage algorithm for geopolitical event forecasting.  In _Proceedings of the 12th ACM Conference on Recommender Systems_. 377–381
  * Rostami et al. (2019a) Rostami, M., Kolouri, S., Eaton, E., and Kim, K. (2019a).  Deep transfer learning for few-shot sar image classification.  _Remote Sensing_ 11, 1374
  * Rostami et al. (2019b) Rostami, M., Kolouri, S., Eaton, E., and Kim, K. (2019b).  Sar image classification using few-shot cross-domain transfer learning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_. 0–0
  * Rostami et al. (2020) Rostami, M., Kolouri, S., Pilly, P., and McClelland, J. (2020).  Generative continual concept learning.  In _Proceedings of the AAAI conference on artificial intelligence_. vol. 34, 5545–5552
  * Rostami et al. (2019c) Rostami, M., Kolouri, S., and Pilly, P. K. (2019c).  Complementary learning for overcoming catastrophic forgetting using experience replay.  In _Proceedings of the 28th International Joint Conference on Artificial Intelligence_ (AAAI Press), 3339–3345
  * Saito et al. (2018) Saito, K., Watanabe, K., Ushiku, Y., and Harada, T. (2018).  Maximum classifier discrepancy for unsupervised domain adaptation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 3723–3732
  * Sakaridis et al. (2018) Sakaridis, C., Dai, D., Hecker, S., and Van Gool, L. (2018).  Model adaptation with synthetic and real data for semantic dense foggy scene understanding.  In _Proceedings of the European Conference on Computer Vision (ECCV)_. 687–704
  * Saltori et al. (2020) Saltori, C., Lathuilière, S., Sebe, N., Ricci, E., and Galasso, F. (2020).  Sf-uda 3d: Source-free unsupervised domain adaptation for lidar-based 3d object detection.  In _2020 International Conference on 3D Vision (3DV)_ (IEEE), 771–780
  * Sankaranarayanan et al. (2018) Sankaranarayanan, S., Balaji, Y., Jain, A., Nam Lim, S., and Chellappa, R. (2018).  Learning from synthetic data: Addressing domain shift for semantic segmentation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 3752–3761
  * Simonyan and Zisserman (2014) Simonyan, K. and Zisserman, A. (2014).  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_
  * Solomon et al. (2015) Solomon, J., De Goes, F., Peyré, G., Cuturi, M., Butscher, A., Nguyen, A., et al. (2015).  Convolutional wasserstein distances: Efficient optimal transportation on geometric domains.  _ACM Transactions on Graphics (TOG)_ 34, 66
  * Stan and Rostami (2021a) Stan, S. and Rostami, M. (2021a).  Privacy preserving domain adaptation for semantic segmentation of medical images.  _arXiv preprint arXiv:2101.00522_ 1
  * Stan and Rostami (2021b) Stan, S. and Rostami, M. (2021b).  Unsupervised model adaptation for continual semantic segmentation.  In _Proceedings of the AAAI conference on artificial intelligence_. vol. 35, 2593–2601
  * Stan and Rostami (2022) Stan, S. and Rostami, M. (2022).  Secure domain adaptation with multiple sources.  _Transactions on Machine Learning Research_
  * Strudel et al. (2021) [Dataset] Strudel, R., Garcia, R., Laptev, I., and Schmid, C. (2021).  Segmenter: Transformer for semantic segmentation
  * Sun et al. (2021) Sun, L., Wang, J., Yang, K., Wu, K., Zhou, X., Wang, K., et al. (2021).  Aerial-pass: Panoramic annular scene segmentation in drone videos.  In _2021 European Conference on Mobile Robots (ECMR)_. 1–6.  10.1109/ECMR50962.2021.9568802
  * Tao et al. (2020) [Dataset] Tao, A., Sapra, K., and Catanzaro, B. (2020).  Hierarchical multi-scale attention for semantic segmentation
  * Tolstikhin et al. (2017) Tolstikhin, I., Bousquet, O., Gelly, S., and Schoelkopf, B. (2017).  Wasserstein auto-encoders.  _arXiv preprint arXiv:1711.01558_
  * Tsai et al. (2018) Tsai, Y.-H., Hung, W.-C., Schulter, S., Sohn, K., Yang, M.-H., and Chandraker, M. (2018).  Learning to adapt structured output space for semantic segmentation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 7472–7481
  * Vu et al. (2019) Vu, T.-H., Jain, H., Bucher, M., Cord, M., and Pérez, P. (2019).  Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 2517–2526
  * Wang et al. (2020) Wang, H., Zhu, Y., Green, B., Adam, H., Yuille, A., and Chen, L.-C. (2020).  Axial-deeplab: Stand-alone axial-attention for panoptic segmentation.  In _European Conference on Computer Vision (ECCV)_
  * Wang and Deng (2018) Wang, M. and Deng, W. (2018).  Deep visual domain adaptation: A survey.  _Neurocomputing_ 312, 135–153
  * Wang et al. (2019) Wang, Q., Gao, J., and Li, X. (2019).  Weakly supervised adversarial domain adaptation for semantic segmentation in urban scenes.  _IEEE Transactions on Image Processing_ 28, 4376–4386.  10.1109/TIP.2019.2910667
  * Wei et al. (2018) Wei, Y., Xiao, H., Shi, H., Jie, Z., Feng, J., and Huang, T. S. (2018).  Revisiting dilated convolution: A simple approach for weakly- and semi-supervised semantic segmentation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_
  * Wilson and Cook (2020) Wilson, G. and Cook, D. J. (2020).  A survey of unsupervised deep domain adaptation.  _ACM Transactions on Intelligent Systems and Technology (TIST)_ 11, 1–46
  * Wu (2016) Wu, D. (2016).  Online and offline domain adaptation for reducing bci calibration effort.  _IEEE Transactions on Human-Machine Systems_ 47, 550–563
  * Wu and Rostami (2023) Wu, M. and Rostami, M. (2023).  Unsupervised domain adaptation for graph-structured data using class-conditional distribution alignment.  _arXiv preprint arXiv:2301.12361_
  * Wu et al. (2018) Wu, Z., Han, X., Lin, Y.-L., Gokhan Uzunbas, M., Goldstein, T., Nam Lim, S., et al. (2018).  Dcan: Dual channel-wise alignment networks for unsupervised scene adaptation.  In _Proceedings of the European Conference on Computer Vision (ECCV)_. 518–534
  * Xia et al. (2021) Xia, W., Wen, W., Wong, K.-K., Quek, T. Q., Zhang, J., and Zhu, H. (2021).  Federated-learning-based client scheduling for low-latency wireless communications.  _IEEE Wireless Communications_ 28, 32–38.  10.1109/MWC.001.2000252
  * Xu et al. (2020) Xu, R., Liu, P., Wang, L., Chen, C., and Wang, J. (2020).  Reliable weighted optimal transport for unsupervised domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4394–4403
  * Yang et al. (2021) Yang, S., Wang, Y., van de Weijer, J., Herranz, L., and Jui, S. (2021).  Generalized source-free domain adaptation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_. 8978–8987
  * Yang et al. (2020) Yang, Y., Lao, D., Sundaramoorthi, G., and Soatto, S. (2020).  Phase consistent ecological domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9011–9020
  * Yang and Soatto (2020) Yang, Y. and Soatto, S. (2020).  Fda: Fourier domain adaptation for semantic segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 4085–4095
  * You et al. (2021) You, F., Li, J., Zhu, L., Chen, Z., and Huang, Z. (2021).  Domain adaptive semantic segmentation without source data.  In _Proceedings of the 29th ACM International Conference on Multimedia_ (New York, NY, USA: Association for Computing Machinery), MM ’21, 3293–3302.  10.1145/3474085.3475482
  * Yu et al. (2021) Yu, Q., Hashimoto, A., and Ushiku, Y. (2021).  Divergence optimization for noisy universal domain adaptation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 2515–2524
  * Zhang et al. (2019) Zhang, Q., Zhang, J., Liu, W., and Tao, D. (2019).  Category anchor-guided unsupervised domain adaptation for semantic segmentation.  In _Advances in Neural Information Processing Systems_. 435–445
  * Zhang et al. (2017) Zhang, Y., David, P., and Gong, B. (2017).  Curriculum domain adaptation for semantic segmentation of urban scenes.  In _Proceedings of the IEEE International Conference on Computer Vision_. 2020–2030
  * Zhang et al. (2016) Zhang, Z., Fidler, S., and Urtasun, R. (2016).  Instance-level segmentation for autonomous driving with deep densely connected mrfs.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 669–677
  * Zhu et al. (2017) Zhu, J., Park, T., Isola, P., and Efros, A. A. (2017).  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In _ICCV_. 2223–2232