# Constraining Pseudo-label in Self-training Unsupervised Domain Adaptation
with Energy-based Model

Lingsheng Kong  Bo Hu  Xiongchang Liu  Jun Lu  Jane You  Xiaofeng Liu
<EMAIL_ADDRESS><EMAIL_ADDRESS>Harvard University, Cambridge, MA,
02138 USA  CIOMP, Chinese Academy of Sciences, CAS, China.  National
University of Singapore, Singapore.  Dept. of Info. and EE, China University
of Mining and Technology, China.  Beth Israel Deaconess Medical Center and
Harvard Medical School, Boston, MA, USA  Dept. of Computing, The Hong Kong
Polytechnic University, Hong Kong

###### Abstract

Deep learning is usually data starved, and the unsupervised domain adaptation
(UDA) is developed to introduce the knowledge in the labeled source domain to
the unlabeled target domain. Recently, deep self-training presents a powerful
means for UDA, involving an iterative process of predicting the target domain
and then taking the confident predictions as hard pseudo-labels for
retraining. However, the pseudo-labels are usually unreliable, thus easily
leading to deviated solutions with propagated errors. In this paper, we resort
to the energy-based model and constrain the training of the unlabeled target
sample with an energy function minimization objective. It can be achieved via
a simple additional regularization or an energy-based loss. This framework
allows us to gain the benefits of the energy-based model, while retaining
strong discriminative performance following a plug-and-play fashion. The
convergence property and its connection with classification expectation
minimization are investigated. We deliver extensive experiments on the most
popular and large-scale UDA benchmarks of image classification as well as
semantic segmentation to demonstrate its generality and effectiveness.

###### keywords:

Self-training , Energy-based model , Domain adaptation

††journal: International Journal of Intelligent Systems

##  1 Introduction

Deep learning (DL) has demonstrated its effectiveness in many areas, while it
is usually data-starved and relies on the $i.i.d$ assumption of training and
testing data. However, there are usually exists a discrepancy when we deploy
the model in a new environment. The collection of a large number of target
domain data for deep learning is challenging for many applications (e.g.,
medical image analysis), thereby hindering the wide adoption of deep learning
methods [1, 2]. For example, densely annotating a Cityscapes image on average
takes about 90 minutes [3], severely hindering the generalization of an
autonomous driving system in different cities. Therefore, the unsupervised
domain adaptation (UDA) is proposed to transfer knowledge from one labeled
source domain to another target domain with the aid of unlabeled target data
[4, 5, 6].

To this end, several solutions have been explored, e.g., feature/image-level
adversarial training and maximum mean discrepancy [7, 8]. Recently, one of the
promising methods in UDA is the self-training [9, 10], which iteratively
generates a set of one-hot pseudo-labels in the target domain, and then
retrains the network according to these pseudo-labels of the unlabeled target
data. The typical choice of the recent deep self-training usually be the one-
hot pseudo-label (i.e., hard label), and demonstrating that it is essentially
an entropy minimization process [9], which enforces the output of the network
to be as sharp as the one-hot hard pseudo-label.

Figure 1: The illustration of our Energy-constrained Self-training framework
for UDA. Minimizing the pseudo label-irrelevant energy of
$E_{\mathbf{w}}(\mathbf{x}_{t})$ is introduced as additional objective for the
target sample.

However, the correctness of the generated pseudo-label is largely uncontrolled
and can be hard to guarantee. Actually, the state-of-the-art accuracy of many
UDA benchmarks is approximately 50%. Especially in the first few epochs, it is
hard to produce a reliable pseudo-label. In addition, the labels of natural
images can also be highly ambiguous. Taking a sample image from VisDA17 [11]
(see Fig. 1) for illustration. We can see that both the person and car
dominate significant portions in this image. Enforcing a model to be very
confident in only one of the classes during training can also hurt the
learning behavior [12], particularly within the context of no ground truth
label for the target samples. Encoding the pseudo-label as the one hot hard
label vector is essentially trusting all of the pseudo-label as the “ground
truth”, which can lead to overconfident mistakes and propagated errors [10].

Targeting for this issue, our previous work [10] proposes to make the network
more conservative w.r.t. the pseudo-label of target sample via the manually
defined pseudo label smoothing or entropy minimization. Specifically, [10]
proposes to modify the possibly inaccurate pseudo-label histogram distribution
to be more smooth following a fixed smoothing operation for any data sample.
For instance, the conventional self-training only transform the three-classes
prediction $[0.3,0.6,0.2]$ to the one-hot pseudo-label $[0,1,0]$ with an
argmax operation. Therefore, we can further apply the label smoothing to
revise $[0,1,0]$ to $[0.05,0.9,0.05]$ for the subsequent conservative self-
training. With the conventional cross-entropy loss, the confidence of the
label can be reduced, which aligns with the practical reliability of pseudo-
label. However, these constraints for pseudo-label can not be adaptively
adjusted for different inputs or network parameters. It is clear that the
reliability of the pseudo-label of the sample in Fig. 1 can be significantly
lower than the normal samples. Besides, the pseudo-label produced in the later
stage can be more trustable than the early stage if the training goes
smoothly. Moreover, the label smoothing is explicitly dependent on the
inaccurate pseudo-label, since it only revises the weight of histogram
distribution. Actually, if the model with the current parameters is
sufficiently confident to a specific sample, it is more reasonable to trust
its pseudo label.

The difficulties mentioned above motivate us to develop an adaptive
regularizer rather than the pre-defined pseudo label smoothing rules. We would
expect our regularization signal for the target sample can adaptively reflect
the current input and the network parameters in this training iteration rather
than depend on the inaccurate pseudo label.

Recently, the correlation between the standard discriminative classifier and
the energy-based model (EBM) is revealed [13]. Essentially, the standard
discriminative classifier can be reinterpreted as an EBM for the joint
distribution $p(\mathbf{x},\mathbf{y})$. Both the energy-based model and
supervised discriminative model can be benefited from simultaneously
optimizing both $p(\mathbf{y}|\mathbf{x})$ with cross-entropy (CE) loss and
optimizing ${\rm log}p(\mathbf{x})$ with EBM [13], respectively. Here, we
demonstrate that optimizing ${\rm log}p(\mathbf{x})$ can be even more
promising on self-training-based UDA. Since we do not have reliable labels for
C.E. loss training on the target domain, and ${\rm log}p(\mathbf{x})$ can be
an idea regularization signal. We note that ${\rm log}p(\mathbf{x})$ is
explicitly correlated with the input $\mathbf{x}$ and network parameter w.r.t.
$p(\cdot)$, and independent to pseudo label $\mathbf{y}$.

Therefore, we propose a simple and straightforward idea that configures the
energy minimization of data point $\mathbf{x}$ (i.e., ${\rm
log}p(\mathbf{x})$) as an additional regularization term. The target domain
examples are optimized with pseudo-label C.E. loss and EBM objective. With the
help of the energy-based model, our self-training is expected to be more
controllable. We give a thorough convergence analysis and theoretically
connect it with classification expectation-maximization (CEM).

In addition to the EBM-based regularization, we further propose an efficient
EBM loss for the target sample, which is able to integrate the confidence-
aware pseudo-label into the energy minimization objective. Based on the
insights that the EBMs capture dependencies by associating a scalar energy
value (i.e., a measure of compatibility) to each configuration of the
variables (e.g., $\mathbf{y}$ and $\mathbf{x}$), we should not minimize the
energy of all possible $\mathbf{y}$ and $\mathbf{x}$ configuration. It is more
reasonable to focus on minimizing the correct energy w.r.t. configuration of
$\mathbf{y}$ and $\mathbf{x}$. In the context of self-training UDA, the
unprocessed soft pseudo-label can be a good signal as the weights of energy at
each class. The histogram distribution value can also inherit the confidence
of the discriminator, which provides more information for the later round
training. Besides, the pseudo-label of Fig. 1 can be a multi-modal
distribution which is also able to alleviate the adverse effect of the
confusing images. It not only speeds up the training, removes the hyper-
parameter validation but also achieves a comparable or even better performance
than EBM regularization.

This work extends our previous paper [14] in the following significant ways:

$\bullet$ We propose a practical and scalable EBM loss for the target sample
with the confidence-aware soft pseudo-label. It can efficiently speed up the
training and remove the hyper-parameter validation, while not sacrificing the
performance.

$\bullet$ We theoretically analyze the convergence property of our energy-
constrained self-training. Besides, its connection with the classification
expectation maximization (CEM) is investigated from the probabilistic
perspective.

In addition, we further demonstrate the generality of our methods (both EBM
regularizer and EBM loss) in more UDA benchmarks (e.g., Office-31 and
SYNTHIA2Cityscapes) with more visualization and analysis.

The main contributions of this paper are summarized as

$\bullet$ We propose a novel and intuitive framework to incorporate the EBM
into the self-training UDA as a pseudo-label-irrelevant regularization signal,
which benefits from both the generative and the discriminative model.

$\bullet$ It can simply be formulated as a regularization term of hard pseudo-
label supervised training, and can also be an EBM loss for the target sample
with the confidence-aware soft pseudo-label.

$\bullet$ The convergence property of our energy-constrained self-training is
investigated. In addition, it can be explained from the probabilistic
perspective. The throughout theoretical analysis bridge its connection with
the classification expectation maximization (CEM).

We empirically validate the effectiveness and generality of the proposed
method on multiple challenging benchmarks (both classification and semantic
segmentation) and achieve state-of-the-art performance.

##  2 Related Works

Unsupervised domain adaptation (UDA). Deep learning (DL) has achieved
tremendous milestones in computer vision recently [15]. Instead of designing
features by hand and then feeding the features to a prediction model, DL
suggests learning specific features and the prediction model simultaneously
from raw image following an end-to-end fashion [16, 17]. However, DL is
usually data-starved and relies on the $i.i.d$ assumption of training and
testing data. The model trained on an annotated source domain does not
generate well on a different target domain due to the domain shift (or drift)
[18, 19]. In reality, collecting the large-scale labeled data in the new
target domain is expensive or even prohibitive. Therefore, we would expect to
migrate the knowledge from a labeled source domain to a different unlabeled
target domain [20, 21].

UDA with deep networks targets to learn domain invariant embeddings by
minimizing the cross-domain difference of feature distributions with certain
criteria. Examples of these methods include maximum mean discrepancy (MMD),
deep Correlation Alignment (CORAL), sliced Wasserstein discrepancy,
adversarial learning at input-level, feature level, output space level, etc.
[22, 23]. Despite the underlying difference among these techniques, there
exists an interesting connection between some of these methods with
conditional forms111E.g., class-wise adversarial learning, or discriminators
taking network predictions as input. and Self-training, as they can be broadly
considered as E.M. algorithms [24], and such conditional formulation has been
widely proved to benefit the adaptation [10].

Other than the self-training, there are some of the other UDA methods that
also rely on the pseudo label. For instance, [25] utilizes pseudo labels to
estimate target class centers, which are used to match source class centers.
CAN [26] utilizes target pseudo-labels to estimate contrastive domain
discrepancy. Since the pseudo labels can be noisy, [27] proposes to measure
its correctness by the Gaussian-uniform mixture model. Instead, we rely on our
energy model for regularization.

Self-training was initially proposed for the semi-supervised learning [28].
The deep self-training is different from the self-training with fixed hand-
crafted feature input in that the deep self-training involves the learning of
deep embedding, which renders greater flexibility towards domain alignment
than classifier-level adaptation only. With the development of deep learning
and UDA, there have been several deep self-training/pseudo-label-based works
that are developed for UDA [29, 30, 24].

Targeting the noisy and unreliable pseudo-label, our preliminary work [10]
proposes to construct a more conservative pseudo-label that smoothes the one-
hot hard label to a soft label vector.

The EBM regularizer proposed in this work can be orthogonal to the recent
progress of self-training [10]. We resort to the additional supervision signal
of EBM, which is independent of the pseudo-label. In contrast with the pre-
defined label smoothing as [10], our proposed energy constraint is able to
adaptively regularize the training with respect to the sample and the current
network parameters. More appealingly, the EBM-based regularization can be
simply added on state-of-the-art self-training methods following a plug-and-
play fashion.

Energy-Based Models (EBMs) is developed to capture the dependencies between
variables via associating scalar energy to each configuration of the variables
[31]. The recent studies [32, 13] propose to approximate the expectation of
the log-likelihood for a single example $\mathbf{x}$ w.r.t. the network
parameter with a sampler based on the Stochastic Gradient Langevin Dynamics
(SGLD) [33].

EBM is a typical unsupervised generative model. To combine the classifier and
EBMs, [34, 35] reinterpreted the logits (i.e., network prediction before the
softmax) to define a class-conditional EBM $p(\mathbf{x}|\mathbf{y})$, which
requires additional parameters to be learned to derive a classifier and an
unconditional model. [36] inherits a similar idea as well, while it is trained
with a GAN-like generator and targets different applications. In contrast, we
do not focus on the generative model of $p(\mathbf{x}|\mathbf{y})$, and
propose to utilize the probability density of $p(\mathbf{x})$ as regularizer.
Based on [32], the recent study [13] scales the training of EBMs to high-
dimensional data with the Contrastive Divergence and SGLD.

In this paper, we propose to regard EBMs as the pseudo-label-irrelevant
optimization constraint, and explore their potentials on unsupervised domain
adaptation (UDA). This helps realize the potential of EBMs on downstream
supervised discriminative problems. In addition, thorough convergence analysis
and theoretical connection with CEM are provided.

##  3 Methodology

In the scenario of UDA, we can access to the labeled source samples
$(\mathbf{x}_{s},\mathbf{y}_{s})$ from source domain
$\\{\mathbf{X}_{S},\mathbf{Y}_{S}\\}$, and target samples $\mathbf{x}_{t}$
from unlabeled target domain data $\mathbf{X}_{T}$. Any target label
$\hat{\mathbf{y}}_{t}=(\hat{y}_{t}^{(1)},...,\hat{y}_{t}^{(K)})$ from
$\hat{\mathbf{Y}}_{T}$ is unknown. $K$ is the total number of classes. The
parametric network $f_{\mathbf{w}}:\mathbb{R}^{D}\rightarrow\mathbb{R}^{K}$
with the weights $\mathbf{w}$ is used to process the $D$-dim input sample. The
output $K$ real-valued numbers known as logits and usually followed by a
softmax normalization to produce ${p_{\mathbf{w}}}(k|\mathbf{x})={\rm
exp}({f_{\mathbf{w}}}({\mathbf{x}})[k])/{\sum_{k\in K}}{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])$ as the classifier’s softmax probability
for class $k$. $f_{\mathbf{w}}({\mathbf{x}})[k]$ indicates the $k^{th}$ index
of $f_{\mathbf{w}}({\mathbf{x}})$, i.e., the logit corresponding the the
$k^{th}$ class label.

###  3.1 Preliminary of energy-based model

The energy based function
$E_{\mathbf{w}}(\mathbf{x}):\mathbb{R}^{D}\rightarrow\mathbb{R}$ maps each
point of an input space to a single scalar, which is called “energy”. One can
parameterize an EBM using any function that takes $\mathbf{x}$ as the input
and returns a scalar. The learning is designed in such a way that
$E_{\mathbf{w}}$ can assign low energies to observed configurations of
variables while give high energies to unobserved ones [32]. With
$E_{\mathbf{w}}$, the probability density $p(\mathbf{x})$ for
$\mathbf{x}\in\mathbb{R}^{D}$ in the EBMs is formulated as

$\displaystyle
p_{\mathbf{w}}(\mathbf{x})=\frac{\exp(-E_{\mathbf{w}}(\mathbf{x}))}{Z(\mathbf{w})}$
(1)

where $Z(\mathbf{w})=\int_{\mathbf{x}}{\rm exp}(-E_{\mathbf{w}}(\mathbf{x}))$
is the normalizing constant (with respect to $\mathbf{x}$) also known as the
partition function. Usually, the normalized densities
$p_{\mathbf{w}}(\mathbf{x})$ are intractable, since we cannot reliably
estimate $Z(\mathbf{w})$ for the most choice of $E_{\mathbf{w}}(\mathbf{x})$.
The typical solutions rely on the sophisticate Markov Chain Monte Carlo (MCMC)
sampler to train EBMs [32]. In our work, we use the deep neural networks as
$E_{\mathbf{w}}$. The gradient estimation is usually used for EBM
optimization, and sample data from it by MCMC [32].

###  3.2 EBM for Discriminative Model

The recent study [13] reveals that a standard classifier can be interpreted as
an energy based model of joint distribution
$p_{\mathbf{w}}(\mathbf{x},\mathbf{y})$, and optimizing the likelihood ${\rm
log}p_{\mathbf{w}}(\mathbf{x},\mathbf{y})={\rm
log}p_{\mathbf{w}}(\mathbf{x})+{\rm log}p_{\mathbf{w}}(\mathbf{y}|\mathbf{x})$
can be helpful for both the discrimination and generation task. Specifically,
optimizing $p(\mathbf{y}|\mathbf{x})$ is simply achieved by using the standard
cross-entropy loss as conventional classification. The additional optimization
objective ${\rm log}p_{\mathbf{w}}(\mathbf{x})$ has been proven and
demonstrating that can improve the confidence calibration and robustness for
conventional classification task [13].

Considering the target samples do not have a ground truth label, the self-
training methods [10] utilize the inaccurate pseudo label to calculate the
cross-entropy loss. Therefore, optimizing ${\rm
log}p_{\mathbf{w}}(\mathbf{x})$ can potentially be more helpful for UDA
setting. Actually, ${\rm log}p_{\mathbf{w}}(\mathbf{x})$ is adaptive w.r.t.
the input $\mathbf{x}$ and network parameter $\mathbf{w}$, and irrelevant to
the inaccurate pseudo label, which can be an ideal regularizer of self-
training based UDA.

However, how to modeling ${\rm log}p_{\mathbf{w}}(\mathbf{x})$ can be a
challenging task. Considering $\frac{\partial{\rm
log}p_{\mathbf{w}}(\mathbf{x})}{\partial{\mathbf{w}}}$ can be approximated
with $-\frac{\partial E_{\mathbf{w}}(\mathbf{x})}{\partial{\mathbf{w}}}$ [13],
it is possible to modeling the energy function $E_{\mathbf{w}}(\mathbf{x})$
instead of ${\rm log}p_{\mathbf{w}}(\mathbf{x})$.

As detailed in Eq. (1), in EBMs, the probability density
$p_{\mathbf{w}}(\mathbf{x})$ for $\mathbf{x}\in\mathbb{R}^{D}$ can be
formulated as $p_{\mathbf{w}}(\mathbf{x})=$ ${\rm
exp}(-E_{\mathbf{w}}(\mathbf{x}))/Z(\mathbf{w})$. Following [13], we can
define an EBM of the joint distribution
$p_{\mathbf{w}}(\mathbf{x},\mathbf{y})={\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])/Z({\mathbf{w}})$, by defining
$E_{\mathbf{w}}(\mathbf{x},\mathbf{y})=-f_{\mathbf{w}}({\mathbf{x}})[k]$. By
marginalizing out $\mathbf{y}$, we have
$p_{\mathbf{w}}(\mathbf{x})=\frac{\sum_{k}{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])}{Z({\mathbf{w}})}$ [13]. Considering
$p_{\mathbf{w}}(\mathbf{x})={\rm
exp}(-E_{\mathbf{w}}(\mathbf{x}))/Z(\mathbf{w})$, the energy function of
$\mathbf{x}$ can be

$\displaystyle E_{\mathbf{w}}(\mathbf{x})$ $\displaystyle=-{\rm
LogSumExp}_{k}(f_{\mathbf{w}}({\mathbf{x}})[k])=-{\rm log}\sum_{k}{\rm
exp}{(f_{\mathbf{w}}({\mathbf{x}})[k])}$ (2)

In this setting,
$p_{\mathbf{w}}(\mathbf{y}|\mathbf{x})=\frac{p_{\mathbf{w}}(\mathbf{x},\mathbf{y})}{p_{\mathbf{w}}(\mathbf{x})}=\frac{{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])/Z({\mathbf{w}})}{{\sum_{k}{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])}/{Z({\mathbf{w}})}}$. The normalization
constant $Z({\mathbf{w}})$ will be canceled out and yielding the standard
softmax function, which bridges the EBMs and conventional classifiers. Since
we can factorize the likelihood as

$\displaystyle{\rm log}p_{\mathbf{w}}(\mathbf{x},\mathbf{y})={\rm
log}p_{\mathbf{w}}(\mathbf{x})+{\rm log}p_{\mathbf{w}}(\mathbf{y}|\mathbf{x})$
(3)

we can simultaneously optimize $p(\mathbf{y}|\mathbf{x})$ with standard cross-
entropy loss and ${\rm log}p_{\mathbf{w}}(\mathbf{x})$ with SGLD (where
gradients are taken w.r.t. ${\rm
LogSumExp}_{k}(f_{\mathbf{w}}({\mathbf{x}})[k])$) in the supervised training
setting [13].

Compared with conventional classifier which will not be affected by shifting
the logits $f_{\mathbf{w}}({\mathbf{x}})$ with a scalar, the
${\log}p_{\mathbf{w}}(\mathbf{x})$ is sensitive to this kind of perturbation.
It introduces an extra degree of freedom w.r.t. logits to define the density
function over input examples and the joint density among examples and labels,
therefore further constraining the training.

###  3.3 EBM as regularization for UDA

The self-training-based UDA proposes to utilize the iterative loss
minimization scheme to gradually refine the model for the target domain. In
self-training, the pseudo-labels are regarded as the learnable latent
variables [10]. Considering the target domain samples do not have the ground
truth label, the optimization objective in self-training relies on the cross-
entropy loss with the pseudo label, which is the relatively confident
prediction in the previous round [24]. Considering the inaccuracy of pseudo
label, the smoothed soft label [10] is proposed as an alternative to the one-
hot hard pseudo label [24].

Therefore, a straightforward solution for adapting EBM to UDA is to integrate
Eq. 2 as a regularization term. For the target sample, the network is enforced
to achieve a good prediction of pseudo-label and minimize the energy of
$E_{\mathbf{w}}(\mathbf{x}_{t})$. The energy constraint is inherently
dependent on $\mathbf{x}$ and $\mathbf{w}$. Therefore, it is more flexible
than the pre-defined label smoothing or entropy regularization [10]. Given
that the pseudo-label is usually noisy, the latter objective is expected to be
even more helpful than the supervised learning task.

Following the notations in our CRST [10], the self-training with EBM
regularization (R-EBM) for target sample, i.e.,
$E_{\mathbf{w}}(\mathbf{x}_{t})$, can be formulated as

$\displaystyle\underset{\mathbf{w},{{{\hat{\mathbf{Y}}}}_{T}}}{\mathop{\min}}\,\mathcal{L}_{R-EBM}(\mathbf{w},{\hat{\mathbf{Y}}})=-\sum\limits_{{{s}}\in{{S}}}{\sum\limits_{k=1}^{K}{y_{s}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{s}})}$
$\displaystyle-\sum\limits_{{{t}}\in{{T}}}\\{{\sum\limits_{k=1}^{K}{[\hat{y}_{t}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}-\hat{y}_{t}^{(k)}{{\log\lambda}_{k}}]-\alpha
E_{\mathbf{w}}(\mathbf{x}_{t})\\}$ $\displaystyle
s.t.~{}{{{\hat{\mathbf{y}}}}_{t}}\in\Delta^{K-1}\cup\\{\mathbf{0}\\},\forall
t$ (4)

For each class $k$, $\lambda_{k}$ is determined by the confidence value
selecting the most confident $p$ portion of class $k$ predictions in the
entire target set [24]. If a sample’s predication is relatively confident with
$p_{\mathbf{w}}(k^{*}|\mathbf{x}_{t})>\lambda_{k^{*}}$, it is selected and
labeled as class
$k^{*}=\text{argmax}_{k}\\{\frac{p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}{\lambda_{k}}\\}$.
The less confident ones with
$p_{\mathbf{w}}(k^{*}|\mathbf{x}_{t})\leq\lambda_{k^{*}}$ are not selected.
The same class-balanced $\lambda_{k}$ strategy introduced in [24] is adopted
for all self-training methods in this work.

The feasible set is the union of $\\{\mathbf{0}\\}$ and a probability simplex
$\Delta^{K-1}$ [24]. $\alpha$ is a balancing hyper-parameter of the
regularization term, which does not directly relate to the label
$\hat{y}_{t}$. The self-training can be solved by an alternating optimization
scheme.

Step 1) Pseudo-label generation Fix $\mathbf{w}$ and solve:

$\displaystyle\underset{{{{\hat{\mathbf{Y}}}}_{T}}}{\mathop{\min}}-\sum\limits_{{{t}}\in{{T}}}\\{{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}[\log{p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}-\log{{{\lambda}_{k}}}]}-\alpha
E_{\mathbf{w}}(\mathbf{x}_{t})\\}$ $\displaystyle s.t.\text{
}{{{\hat{y}}}_{t}}\in\Delta^{K-1}\cup\\{\mathbf{0}\\},\forall t$ (5)

For solving step 1), there is a global optimizer for arbitrary
$\hat{\mathbf{y}}_{t}=(\hat{y}_{t}^{(1)},...,\hat{y}_{t}^{(K)})$ as [10]:

$\hat{y}_{t}^{(k)*}=\left\\{\begin{aligned} 1,&~{}\text{if}~{}k={~{}}^{\rm
argmax}_{~{}~{}~{}~{}k}~{}{\frac{p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}{\lambda_{k}}}~{}~{}~{}\text{and}~{}~{}~{}p_{\mathbf{w}}(k|\mathbf{x}_{t})>\lambda_{k}\\\
0,&~{}\mathrm{otherwise}\end{aligned}\right.$ (6)

Despite a long period of little progression, there have been some recent works
[35, 32] using the sampler based on SGLD to train the large-scale EBMs on
high-dimensional data, which is parameterized by deep neural networks.

Given that the goal of this work is to incorporate EBM training into the
standard classification setting, the classification part is the same as [10],
and we only change the regularizer. Therefore, we can follow [13] to train the
network with both cross-entropy loss and SGLD to ensure this distribution is
being optimized with an unbiased objective. Similar to [35] we also adopt the
contrastive divergence to estimate the expectation of derivative of the log-
likelihood for a single example $\mathbf{x}$ with respect to $\mathbf{w}$.
Since it gives an order of magnitude savings in computation compared to
seeding new chains at each iteration as in [32].

Step 2) Network retraining Fix $\hat{\mathbf{Y}}_{T}$ and minimize

$\displaystyle\tiny-\sum\limits_{{{s}}\in{{S}}}{\sum\limits_{k=1}^{K}{y_{s}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{s}})}-\sum\limits_{{{t}}\in{{T}}}{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}$ (7)

w.r.t. $\mathbf{w}$. Carrying out step 1) and 2) for one time is defined as
one round in self-training.

For each class $k$, $\lambda_{k}$ is determined by the confidence value
selecting the most confident $p$ portion of class $k$ predictions in the
entire target set [24]. If a sample’s predication is relatively confident with
$p(k^{*}|\mathbf{x}_{t};\mathbf{w})>\lambda_{k^{*}}$, it is selected and
labeled as class
$k^{*}=\operatorname*{arg\,max}_{k}\\{\frac{p(k|{\mathbf{x}_{t}};\mathbf{w})}{\lambda_{k}}\\}$.
The less confident ones with
$p(k^{*}|\mathbf{x}_{t};\mathbf{w})\leq\lambda_{k^{*}}$ are not selected. The
same class-balanced $\lambda_{k}$ strategy introduced in [24] is adopted for
all self-training methods in this work.

###  3.4 EBM loss for target domain

The essence of the energy-based model [31] is a data-driven process that
shapes the energy surface in such a way that the desired configurations get
assigned low energies, while the incorrect ones are given high energies.

In this framework, the conventional supervised learning [37] for each
$\mathbf{x}$ in the training set, the energy of the pair $\\{\mathbf{x,y}\\}$
takes low values when $\mathbf{y}$ is the correct label and higher values for
incorrect $\mathbf{y}$. Similarly, when modeling $\mathbf{x}$ alone within an
unsupervised learning setting, lower energy is attributed to the data
manifold. Actually, the term contrastive sample is often used to refer to a
data point causing an energy pull-up, such as the incorrect $\mathbf{y}$ in
supervised learning and points from low data density regions in the setting of
unsupervised learning. Therefore, we can construct an efficient EBM loss for
the target sample as

$\mathcal{L}_{EBM}=-{\rm log}\sum_{{k}}{\hat{\mathbf{y}}}{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[{{k}}])+{\sum_{{k}}{\hat{y}_{t}^{(k)}}\log{{{\lambda}_{k}}}}$
(8)

The soft pseudo-label ${\hat{\mathbf{y}}}$ used here is the original softmax
prediction of the previous round classifier/segmentor without the one-hot
processing. This is also essentially different from the soft-label used in
[10] which post-process the one-hot label to construct a smoothed one. Then,
Eq. 8 will be used to replace the objective in Eq. 3.3. This configuration
also dispenses with the validation of $\alpha$.

This can be regarded as using ${\hat{\mathbf{y}}}$ as the weight to indicate
which class’s energy should be minimized. For the class with larger
${\hat{\mathbf{y}}}$ probability, the larger
$f_{\mathbf{w}}({\mathbf{x}})[{{k}}]$ will be emphasised. Therefore, the
network will pron to produce the large $f_{\mathbf{w}}({\mathbf{x}})[{{k}}]$
value at the pseudo-label’s category.

With the soft pseudo-label setting, the confidence of each class prediction
can also be encoded in the histogram distribution of ${\hat{\mathbf{y}}}$.
Although the additional information can enrich the training, the signal may be
even noisier at the beginning. To avoid the model collapse, in practice, we
propose a loss annealing scheme that starts with the EBM regularization and
then smoothly changes to $\mathcal{L}_{EBM}$. The objective can be formulated
as
${\frac{1}{1+\beta}\left\\{\mathcal{L}_{EBM}+\beta\mathcal{R}_{EBM}\right\\}}$
where $\beta$ is a balance hyper-parameter and decrease $\beta$ from 10 to 0
gradually in the training, $i.e.,$ $\beta=\frac{10}{1+N^{2}}$ for $N\leq$5,
and $\beta=0$ for $N>$5, where $N$ is the epoch number.

###  3.5 Energy-based self-training as CEM

[38] proves that semi-supervised logistic regression is a Classification
Expectation-Maximization (CEM) to solve Classification Maximum Likelihood
(CML). We generalize the statement for proposed energy-constrained self-
training as follows.

######  Proposition 1.

Energy-constrained self-training can be a regularized Classification
Expectation-Maximization (CEM) for solving a generalized Regularized
Classification Maximum Likelihood (RCML) problem and is convergent with
gradient descent.

The problem of classification maximum likelihood (CML) was first proposed for
clustering tasks in [39], and can be solved through classification expectation
maximization (CEM). Compared with traditional expectation maximization (EM)
that has an Expectation (E) step and a Maximization (M) step, CEM has an
additional Classification (C) step (between E and M step) that assigns a
sample to the cluster with maximal posterior probability. In [38], the CML is
generalized for discriminant semi-supervised learning with both labeled and
unlabeled data that is defined as follows:

${\log{\mathcal{L}}_{C}}={\log\tilde{\mathcal{L}}_{C}}+\sum\limits_{{{i}}\in{{S,T}}}\log
p(\mathbf{x}_{i})$ (9)

where:

$\begin{split}{\log\tilde{\mathcal{L}}_{C}}=\sum\limits_{{{s}}\in{{S}}}{\sum\limits_{k=1}^{K}{y_{s}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{s}})}+\sum\limits_{{{t}}\in{{T}}}{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}\end{split}$ (10)

Note that ${{{\hat{y}}}_{t}}\in\\{0,1\\}^{K},\forall t$.
$p_{\mathbf{w}}(k|\mathbf{x}_{t})$ is a posterior probability modeled by
probabilistic classifier such as logistic classifier, neural network, etc.
$\mathbf{w}$ is the learnable weight. As mentioned in [38], when using a
discriminant classifier, we make no assumption about the data distribution
$p(\mathbf{x}_{t})$. Thus maximizing Eq. 9 is equal to maximizing Eq. 10. The
Classification Expectation Maximization (CEM) is used to solve Eq. 10 via
alternating E-step: estimating the posterior probability
$p_{\mathbf{w}}(k|\mathbf{x}_{t})$; C-step: assigning the pseudo-labels
according to the maximal posterior probability
$p_{\mathbf{w}}(k|\mathbf{x}_{t})$; M-step: maximizing the log-likelihood in
terms of $\mathbf{w}$.

For proposition 1, we show the inherent connections between the energy-based
self-training and Regularized Classification Maximum Likelihood (RCML). We
also show that the proposed alternating optimization is essentially a
Classification Expectation-Maximization (CEM) towards maximizing RCML. The
energy-based self-training minimization problem can be seen as the following
maximization problem:

$\begin{split}&\underset{{{{\mathbf{w},\hat{\mathbf{Y}}}}_{T}}}{\mathop{\max}}\,{\log\mathcal{\tilde{L}}_{C}}+EBM=\sum\limits_{{{s}}\in{{S}}}{\sum\limits_{k=1}^{K}{y_{s}^{(k)}}\log(p_{\mathbf{w}}(k|{\mathbf{x}_{s}}))}\\\
&~{}~{}~{}~{}~{}~{}~{}~{}~{}+\sum\limits_{{{t}}\in{{T}}}[{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}\log(p_{\mathbf{w}}(k|{\mathbf{x}_{t}}))}-\alpha
E_{\mathbf{w}}(\mathbf{x}_{t})]\\\
&\text{s.t.}~{}~{}\mathbf{\hat{y}}_{t}\in~{}\Delta^{(K-1)}\cup\\{\mathbf{0}\\},~{}\forall
t\end{split}$ (11)

where $\alpha>0$.
$\mathbf{\hat{y}}_{t}\in\Delta^{K-1}=\\{(\hat{y}_{t}^{(1)},..,\hat{y}_{t}^{(K)}):\hat{y}_{t}^{(k)}\geq
0,\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}=1\\},\forall t$. ($\Delta^{K-1}$
denotes a probability simplex) Compared with Eq. 10, the above problem can be
regarded as a generalized Classification Maximization Likelihood
($\log\tilde{\mathcal{L}}_{C}$) with a energy regularizer ($E$), relaxing the
one-hot vector feasible set to the probability simplex. The alternating
optimization can be regarded as a Classification Expectation Maximization
(CEM) for solving this regularized CML (RCML) problem:

E-Step: Given the model weight $\mathbf{w}$, estimate the posterior
probability $p_{\mathbf{w}}(\mathbf{x}_{t}),\forall t$.

C-Step: Fix $\mathbf{w}$ and solve the following problem for
$\hat{\mathbf{Y}}_{T}$.

$\begin{split}&\underset{{{{\hat{\mathbf{Y}}}}_{T}}}{\mathop{\max}}\,\sum\limits_{{{t}}\in{{T}}}[{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}-\alpha E_{\mathbf{w}}(\mathbf{x}_{t})]\\\
&\text{s.t.}~{}~{}\hat{\mathbf{y}}_{t}\in~{}\Delta^{(K-1)}\cup\\{\mathbf{0}\\},~{}\forall
t\end{split}$ (12)

M-Step: Fix $\hat{\mathbf{Y}}_{T}$ and use gradient ascent to solve the
following problem for $\mathbf{w}$.

$\begin{split}\underset{{{{\mathbf{w}}}}}{\mathop{\max}}\,&\sum\limits_{{{s}}\in{{S}}}{\sum\limits_{k=1}^{K}{y_{s}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{s}})}+\sum\limits_{{{t}}\in{{T}}}[{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}-\alpha
E_{\mathbf{w}}(\mathbf{x}_{t})]\end{split}$ (13)

Now we have shown energy-based self-training can be regarded as a CEM.

In the next subsection, we also provide the convergence analysis of energy-
constrained self-training for UDA.

###  3.6 Convergence analysis for energy-constrained self-training

Considering that our energy-constrained self-training follows an alternating
optimization scheme, which is different from the conventional single round
updating, we further explain that the energy-constrained self-training is
convergent. We emphasize the proposed energy regularizer is convex w.r.t.
either prediction distributions or pseudo-label distributions. The problem can
be solved by alternatively taking the two steps. The training is convergent
only if the loss in both of the two steps are non-increasing.

Specifically, we have the following analysis for these two steps:

Loss in step (1) is non-increasing: We prove that the solver (6) is a global
minimum of optimization problem (3.3). Specifically, the feasible set of the
problem is a union of $\Delta^{K-1}$ and $\\{\textbf{0}\\}$. Thus, two sub-
problems share the same objective with either a probability simplex and 0 as a
feasible set. For the proposed convex energy regularizer, the first sub-
problem is a convex problem with a unique solution (see Table 1 of the main
paper), while the latter one has a global optimum solver at 0. Therefore, (6)
is the solver for the combined two subproblems, making it the global optimum.
Thus step 1 is non-increasing and can be minimized with the global solver.

Loss in step (2) is non-increasing: Due to the convexity of proposed model
regularizers, with a proper learning rate, the gradient descent can
monotonically decrease the regularized objective function of (7).

Since the proposed energy regularization are lower-bounded by $0$, it’s easy
to see that energy regularized self-training loss is lower-bounded by
$\sum\limits_{t\in T}\sum\limits_{{{k=1},...,{K}}}\log\lambda_{k}$. Thus
energy-regularized self-training is convergent.

ine Method | Aero | Bike | Bus | Car | Horse | Knife | Motor | Person | Plant | Skateboard | Train | Truck | Mean
---|---|---|---|---|---|---|---|---|---|---|---|---|---
ineine Source-Res101 [40] | 55.1 | 53.3 | 61.9 | 59.1 | 80.6 | 17.9 | 79.7 | 31.2 | 81.0 | 26.5 | 73.5 | 8.5 | 52.4
ine MMD [19] | 87.1 | 63.0 | 76.5 | 42.0 | 90.3 | 42.9 | 85.9 | 53.1 | 49.7 | 36.3 | 85.8 | 20.7 | 61.1
DANN [41] | 81.9 | 77.7 | 82.8 | 44.3 | 81.2 | 29.5 | 65.1 | 28.6 | 51.9 | 54.6 | 82.8 | 7.8 | 57.4
ENT [42] | 80.3 | 75.5 | 75.8 | 48.3 | 77.9 | 27.3 | 69.7 | 40.2 | 46.5 | 46.6 | 79.3 | 16.0 | 57.0
MCD [43] | 87.0 | 60.9 | 83.7 | 64.0 | 88.9 | 79.6 | 84.7 | 76.9 | 88.6 | 40.3 | 83.0 | 25.8 | 71.9
ADR [44] | 87.8 | 79.5 | 83.7 | 65.3 | 92.3 | 61.8 | 88.9 | 73.2 | 87.8 | 60.0 | 85.5 | 32.3 | 74.8
DEV [45] | 81.8 | 53.5 | 82.9 | 71.6 | 89.2 | 72.0 | 89.4 | 75.7 | 97.0 | 55.5 | 71.2 | 29.2 | 72.4
TDDA [40] | 88.2 | 78.5 | 79.7 | 71.1 | 90.0 | 81.6 | 84.9 | 72.3 | 92.0 | 52.6 | 82.9 | 18.4 | 74.03
PANDA [46] | 90.9 | 50.5 | 72.3 | 82.7 | 88.3 | 88.3 | 90.3 | 79.8 | 89.7 | 79.2 | 88.1 | 39.4 | 78.3
DMRL [47] | - | - | - | - | - | - | - | - | - | - | - | - | 75.5
ine CBST [24] | 87.1$\pm$1.2 | 79.5$\pm$2.3 | 58.3$\pm$2.6 | 50.4$\pm$3.9 | 82.8$\pm$2.1 | 73.7$\pm$7.2 | 80.9$\pm$2.6 | 71.8$\pm$3.1 | 81.6$\pm$3.2 | 88.4$\pm$3.3 | 75.2$\pm$1.2 | 68.4$\pm$3.4 | 74.8$\pm$0.5
ine CBST+$R_{EBM}$ | 87.9$\pm$1.6 | 79.6$\pm$1.5 | 68.5$\pm$1.3 | 68.6$\pm$1.9 | 83.2$\pm$1.2 | 78.4$\pm$1.9 | 83.5$\pm$1.5 | 72.2$\pm$1.5 | 82.2$\pm$1.6 | 84.3$\pm$1.5 | 80.9$\pm$1.4 | 67.5$\pm$1.3 | 77.0$\pm$0.6
CBST+${\mathcal{L}}_{EBM}$ | 87.8$\pm$1.2 | 80.4$\pm$1.9 | 69.3$\pm$1.7 | 67.5$\pm$1.6 | 83.2$\pm$1.0 | 76.8$\pm$1.4 | 81.6$\pm$1.8 | 72.3$\pm$1.5 | 81.6$\pm$1.8 | 84.5$\pm$1.3 | 81.7$\pm$1.1 | 67.0$\pm$1.4 | 76.8$\pm$0.5
ine CRST[10] | 89.2$\pm$1.6 | 79.6$\pm$4.6 | 64.2$\pm$4.0 | 57.8$\pm$3.4 | 87.8$\pm$1.9 | 79.6$\pm$8.5 | 85.6$\pm$2.6 | 75.9$\pm$4.2 | 86.5$\pm$2.2 | 85.1$\pm$2.4 | 77.7$\pm$2.2 | 68.5$\pm$0.9 | 78.1$\pm$0.7
ine CRST+$R_{EBM}$ | 90.3$\pm$1.5 | 82.6$\pm$1.2 | 72.4$\pm$1.5 | 71.7$\pm$1.8 | 87.6$\pm$1.8 | 80.5$\pm$1.9 | 85.4$\pm$1.5 | 80.8$\pm$1.5 | 87.1$\pm$1.6 | 89.9$\pm$1.5 | 83.6$\pm$1.6 | 71.5$\pm$1.3 | 80.2$\pm$0.5
CRST+${\mathcal{L}}_{EBM}$ | 91.9$\pm$1.7 | 81.8$\pm$1.8 | 72.8$\pm$1.2 | 71.5$\pm$1.3 | 88.4$\pm$1.6 | 81.3$\pm$1.3 | 89.3$\pm$1.1 | 78.8$\pm$1.2 | 87.2$\pm$1.7 | 88.0$\pm$1.5 | 85.3$\pm$1.0 | 71.2$\pm$1.6 | 80.4$\pm$0.7
ine ine SimNet* [48] | 94.3 | 82.3 | 73.5 | 47.2 | 87.9 | 49.2 | 75.1 | 79.7 | 85.3 | 68.5 | 81.1 | 50.3 | 72.9
GTA* [49] | - | - | - | - | - | - | - | - | - | - | - | - | 77.1
ine CRST+$R_{EBM}$* | 93.2$\pm$1.3 | 85.8$\pm$1.2 | 73.7$\pm$1.0 | 74.3$\pm$1.5 | 89.5$\pm$0.6 | 87.6$\pm$1.6 | 88.2$\pm$1.6 | 82.2$\pm$1.6 | 90.9$\pm$1.3 | 91.6$\pm$1.8 | 85.1$\pm$1.4 | 79.9$\pm$1.4 | 82.8$\pm$0.5
CRST+${\mathcal{L}}_{EBM}$* | 93.0$\pm$1.6 | 84.4$\pm$1.6 | 74.4$\pm$1.0 | 73.5$\pm$1.4 | 91.2$\pm$1.5 | 82.4$\pm$1.5 | 88.8$\pm$1.6 | 81.8$\pm$1.2 | 89.8$\pm$1.2 | 90.8$\pm$1.4 | 86.5$\pm$1.2 | 77.1$\pm$0.9 | 82.5$\pm$0.5
ine |  |  |  |  |  |  |  |  |  |  |  |  |

Table 1: Experimental results for VisDA17-val setting. We use ResNet101 as
backbone except SimNet and GTA.*ResNet152 backbone.
Figure 2: Sensitive analysis of hyper-parameter $\alpha$ in VisDA17 (top) and
CTA52Sityscapes (bottom) with CRST+$R_{EBM}$.

ine Method | A$\rightarrow$W | D$\rightarrow$W | W$\rightarrow$D | A$\rightarrow$D | D$\rightarrow$A | W$\rightarrow$A | Mean Accuracy
---|---|---|---|---|---|---|---
ine ResNet-50 [50] | 68.4$\pm$0.2 | 96.7$\pm$0.1 | 99.3$\pm$0.1 | 68.9$\pm$0.2 | 62.5$\pm$0.3 | 60.7$\pm$0.3 | 76.1
ineine DAN [ICML2015] [19] | 80.5$\pm$0.4 | 97.1$\pm$0.2 | 99.6$\pm$0.1 | 78.6$\pm$0.2 | 63.6$\pm$0.3 | 62.8$\pm$0.2 | 80.4
RTN [NeurIPS2016] [51] | 84.5$\pm$0.2 | 96.8$\pm$0.1 | 99.4$\pm$0.1 | 77.5$\pm$0.3 | 66.2$\pm$0.2 | 64.8$\pm$0.3 | 81.6
DANN [JMLR2016] [41] | 82.0$\pm$0.4 | 96.9$\pm$0.2 | 99.1$\pm$0.1 | 79.7$\pm$0.4 | 68.2$\pm$0.4 | 67.4$\pm$0.5 | 82.2
ADDA [CVPR2017] [52] | 86.2$\pm$0.5 | 96.2$\pm$0.3 | 98.4$\pm$0.3 | 77.8$\pm$0.3 | 69.5$\pm$0.4 | 68.9$\pm$0.5 | 82.9
JAN [JMLR2017] [53] | 85.4$\pm$0.3 | 97.4$\pm$0.2 | 99.8$\pm$0.2 | 84.7$\pm$0.3 | 68.6$\pm$0.3 | 70.0$\pm$0.4 | 84.3
GTA [CVPR2018] [49] | 89.5$\pm$0.5 | 97.9$\pm$0.3 | 99.8$\pm$0.4 | 87.7$\pm$0.5 | 72.8$\pm$0.3 | 71.4$\pm$0.4 | 86.5
DMRL [ECCV2020] [47] | 90.8$\pm$0.3 | 99.0$\pm$0.2 | 100.0$\pm$0.0 | 93.4$\pm$0.5 | 73.0$\pm$0.3 | 71.2$\pm$0.3 | 87.9
ineine CBST [ECCV2018] [24] | 87.8$\pm$0.8 | 98.5$\pm$0.1 | 100$\pm$0.0 | 86.5$\pm$1.0 | 71.2$\pm$0.4 | 70.9$\pm$0.7 | 85.8
ine CBST+$R_{EBM}$ | 89.4$\pm$0.2 | 99.6$\pm$0.1 | 100$\pm$0.0 | 88.7$\pm$0.9 | 73.8$\pm$0.2 | 73.1$\pm$0.2 | 87.4
CBST+${\mathcal{L}}_{EBM}$ | 89.0$\pm$0.4 | 99.6$\pm$0.1 | 100$\pm$0.0 | 88.4$\pm$0.8 | 74.7$\pm$0.2 | 73.0$\pm$0.4 | 87.4
ineine CRST [ICCV2019] [10] | 89.4$\pm$0.7 | 98.9$\pm$0.4 | 100$\pm$0.0 | 88.7$\pm$0.8 | 72.6$\pm$0.7 | 70.9$\pm$0.5 | 86.8
ine CRST+$R_{EBM}$ | 90.6$\pm$0.4 | 99.7$\pm$0.1 | 100$\pm$0.0 | 91.0$\pm$0.8 | 74.0$\pm$0.6 | 74.0$\pm$0.3 | 88.6
CRST+${\mathcal{L}}_{EBM}$ | 90.4$\pm$0.9 | 99.7$\pm$0.1 | 100$\pm$0.0 | 90.0$\pm$0.9 | 73.7$\pm$0.8 | 73.9$\pm$0.4 | 88.3
ine |  |  |  |  |  |  |

Table 2: Experimental results for Office-31

##  4 Experiments

This section provides comprehensive evaluations of our proposed EBM-based
regularization and loss on several UDA benchmarks, including image
classification and semantic segmentation. We implement our methods with the
PyTorch platform [54] on a V100GPU.

###  4.1 Domain Adaptation for Image Classification

We test on two challenging benchmarks: VisDA17 [11] and Office-31 [55].

The VisDA17 [11] dataset involves 12-class for UDA classification problem.
Following the standard protocol as [10, 49], the source domain utilizes the
training set with $152,409$ synthetic 2D images, while the target domain uses
the validation set with $55,400$ real images from the COCO dataset.

For a fair comparison, we use the same backbone network for VisDA17 as the
compared works, e.g., ResNet-101 and ResNet-152. Moreover, the model was also
pre-trained with the ImageNet dataset as the previous works and fine-tuned in
source domain by SGD [10]. We use a fixed learning rate $1\times 10^{-3}$,
weight decay $5\times 10^{-4}$, momentum $0.9$ and batch size $32$. From Fig.
2, we can see that the performance is not sensitive to $\alpha$ when
$\alpha\in[0.8,1.1]$. Therefore, we simply choose $\alpha=1$ consistently for
all settings.

ine Method | Base Net | Road | SW | Build | Wall | Fence | Pole | TL | TS | Veg. | Terrain | Sky | PR | Rider | Car | Truck | Bus | Train | Motor | Bike | mIoU
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
ineine Source | DRN26 | 42.7 | 26.3 | 51.7 | 5.5 | 6.8 | 13.8 | 23.6 | 6.9 | 75.5 | 11.5 | 36.8 | 49.3 | 0.9 | 46.7 | 3.4 | 5.0 | 0.0 | 5.0 | 1.4 | 21.7
CyCADA [56] |  | 79.1 | 33.1 | 77.9 | 23.4 | 17.3 | 32.1 | 33.3 | 31.8 | 81.5 | 26.7 | 69.0 | 62.8 | 14.7 | 74.5 | 20.9 | 25.6 | 6.9 | 18.8 | 20.4 | 39.5
ine Source | DRN105 | 36.4 | 14.2 | 67.4 | 16.4 | 12.0 | 20.1 | 8.7 | 0.7 | 69.8 | 13.3 | 56.9 | 37.0 | 0.4 | 53.6 | 10.6 | 3.2 | 0.2 | 0.9 | 0.0 | 22.2
MCD [43] |  | 90.3 | 31.0 | 78.5 | 19.7 | 17.3 | 28.6 | 30.9 | 16.1 | 83.7 | 30.0 | 69.1 | 58.5 | 19.6 | 81.5 | 23.8 | 30.0 | 5.7 | 25.7 | 14.3 | 39.7
ine Source | PSPNet | 69.9 | 22.3 | 75.6 | 15.8 | 20.1 | 18.8 | 28.2 | 17.1 | 75.6 | 8.00 | 73.5 | 55.0 | 2.9 | 66.9 | 34.4 | 30.8 | 0.0 | 18.4 | 0.0 | 33.3
DCAN [57] |  | 85.0 | 30.8 | 81.3 | 25.8 | 21.2 | 22.2 | 25.4 | 26.6 | 83.4 | 36.7 | 76.2 | 58.9 | 24.9 | 80.7 | 29.5 | 42.9 | 2.50 | 26.9 | 11.6 | 41.7
ine Source | DeepLabv2 | 75.8 | 16.8 | 77.2 | 12.5 | 21.0 | 25.5 | 30.1 | 20.1 | 81.3 | 24.6 | 70.3 | 53.8 | 26.4 | 49.9 | 17.2 | 25.9 | 6.5 | 25.3 | 36.0 | 36.6
AdaptSegNet [58] |  | 86.5 | 36.0 | 79.9 | 23.4 | 23.3 | 23.9 | 35.2 | 14.8 | 83.4 | 33.3 | 75.6 | 58.5 | 27.6 | 73.7 | 32.5 | 35.4 | 3.9 | 30.1 | 28.1 | 42.4
ine AdvEnt [59] | DeepLabv2 | 89.4 | 33.1 | 81.0 | 26.6 | 26.8 | 27.2 | 33.5 | 24.7 | 83.9 | 36.7 | 78.8 | 58.7 | 30.5 | 84.8 | 38.5 | 44.5 | 1.7 | 31.6 | 32.4 | 45.5
ine Source | DeepLabv2 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 29.2
FCAN [60] |  | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 46.6
ine Source | DeepLabv2 | 75.8 | 16.8 | 77.2 | 12.5 | 21.0 | 25.5 | 30.1 | 20.1 | 81.3 | 24.6 | 70.3 | 53.8 | 26.4 | 49.9 | 17.2 | 25.9 | 6.5 | 25.3 | 36.0 | 36.6
DPR [61] |  | 92.3 | 51.9 | 82.1 | 29.2 | 25.1 | 24.5 | 33.8 | 33.0 | 82.4 | 32.8 | 82.2 | 58.6 | 27.2 | 84.3 | 33.4 | 46.3 | 2.2 | 29.5 | 32.3 | 46.5
ine Source | DeepLabv2 | 73.8 | 16.0 | 66.3 | 12.8 | 22.3 | 29.0 | 30.3 | 10.2 | 77.7 | 19.0 | 50.8 | 55.2 | 20.4 | 73.6 | 28.3 | 25.6 | 0.1 | 27.5 | 12.1 | 34.2
PyCDA [62] |  | 90.5 | 36.3 | 84.4 | 32.4 | 28.7 | 34.6 | 36.4 | 31.5 | 86.8 | 37.9 | 78.5 | 62.3 | 21.5 | 85.6 | 27.9 | 34.8 | 18.0 | 22.9 | 49.3 | 47.4
ine ine Source | DeepLabv2 | 71.3 | 19.2 | 69.1 | 18.4 | 10.0 | 35.7 | 27.3 | 6.8 | 79.6 | 24.8 | 72.1 | 57.6 | 19.5 | 55.5 | 15.5 | 15.1 | 11.7 | 21.1 | 12.0 | 33.8
CBST [24] |  | 89.9 | 55.0 | 79.9 | 29.5 | 20.6 | 37.8 | 32.9 | 13.9 | 84.0 | 31.2 | 75.5 | 60.2 | 27.1 | 81.8 | 29.7 | 40.5 | 7.62 | 28.7 | 41.4 | 45.6
CBST+$R_{EBM}$ |  | 91.1 | 53.9 | 80.6 | 31.6 | 21.0 | 40.4 | 35.0 | 19.8 | 86.6 | 35.9 | 76.4 | 63.3 | 31.4 | 83.0 | 22.5 | 38.6 | 24.2 | 32.2 | 39.4 | 47.8
CBST+${\mathcal{L}}_{EBM}$ |  | 92.7 | 56.2 | 79.6 | 27.1 | 21.7 | 39.3 | 35.5 | 24.2 | 85.3 | 38.0 | 77.9 | 63.6 | 29.1 | 84.8 | 25.3 | 39.6 | 17.8 | 34.8 | 43.4 | 48.2
ineine Source | DeepLabv2 | 71.3 | 19.2 | 69.1 | 18.4 | 10.0 | 35.7 | 27.3 | 6.8 | 79.6 | 24.8 | 72.1 | 57.6 | 19.5 | 55.5 | 15.5 | 15.1 | 11.7 | 21.1 | 12.0 | 33.8
CRST [10] |  | 89.0 | 51.2 | 79.4 | 31.7 | 19.1 | 38.5 | 34.1 | 20.4 | 84.7 | 35.4 | 76.8 | 61.3 | 30.2 | 80.7 | 27.4 | 39.4 | 10.2 | 32.2 | 43.3 | 46.6
CRST+$R_{EBM}$ |  | 92.5 | 56.6 | 80.9 | 26.2 | 20.5 | 40.2 | 35.3 | 24.4 | 86.6 | 37.3 | 77.5 | 63.4 | 30.5 | 81.3 | 28.8 | 39.2 | 20.6 | 33.5 | 41.3 | 48.5
CRST+${\mathcal{L}}_{EBM}$ |  | 92.7 | 55.1 | 83.6 | 34.5 | 20.4 | 39.4 | 36.5 | 19.6 | 85.7 | 35.7 | 78.3 | 64.2 | 30.1 | 84.5 | 32.0 | 40.4 | 28.0 | 33.9 | 44.2 | 49.0
ine |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |

Table 3: Experimental results for GTA5 to Cityscapes.

We present the results on VisDA 17 in Table 1 regarding per-class accuracy and
mean accuracy. For each proposed approach, we independently run five times,
and report the average accuracy and the corresponding standard deviation
($\pm$sd).

Among the compared methods, Class-balanced self-training (CBST) and
Conservative regularized self-training (CRST) [10] are the state-of-the-art
self-training UDA methods. Our regularization can be simply added on
CBST/CRST. We denote the CRST with EBM regularization with CRST+$R_{EBM}$.

Contributed by the additional EBMs objective, our proposed CBST+$R_{EBM}$ and
CRST+$R_{EBM}$ can outperform CBST and CRST significantly. More appealingly,
it can outperform the recent UDA methods other than self-training by a large
margin. More appealing, the adversarial training can be simply added on to
further boost the performance of self-training [10]. Our proposed energy-based
constraint is able to adaptively regularize the training w.r.t. the input
$\mathbf{x}$ and the present network parameters, which is more flexible than
the manually pre-defined label smoothing [10].

We can see that CRST+$R_{EBM}$ achieves a better performance than the recent
adversarial training [40], dropout [63] and moment matching methods, which
revokes the potential of self-training in UDA.

To demonstrate the generality of our EBM-based regularizer and loss for
different backbones, we also tested with the more powerful backbones that have
also been applied and shown better results [48, 49]. The EBM objective with
the ResNet152 backbone outperforms the other state-of-the-arts [48, 49]. It
also demonstrates the flexibility of $R_{EBM}$ for different backbones.

Office-31 is another standard UDA dataset containing images belonging to $31$
classes from three different domains, i.e., Amazon (A), Webcam (W), and DSLR
(D), each containing $2,817$, $795$ and $498$ images respectively. We follow
the standard protocol in [55, 49] and evaluate on six transfer tasks
$A\rightarrow W$, $D\rightarrow W$, $W\rightarrow D$, $A\rightarrow D$,
$D\rightarrow A$, and $W\rightarrow A$.

Figure 3: The KL-divergence between the predicted $p_{t}(y)$ and the ground of
truth $p_{t}(y)$ in the testing stage. Left: VisDA17 task with CBST backbone,
right: GTA5 to Cityscapes task with CRST backbone. The small value indicates
the more accurate estimation.

We compare the performances of different methods on Office-31 with the same
basenet ResNet-50 in Table 4. CBST or CRST+$R_{EBM}$ achieves the best
performance that outperforms their baselines CBST and CRST.

Actually, our $R_{EBM}$ can be orthogonal with the recent advanced self-
training UDA methods. CBST [24] is a pioneer of vanilla adversarial UDA, and
its performance on VisDA17 is reported on [10]. We note that the label
smoothing or entropy regularization proposed in CRST [10] or the vanilla self-
training UDA [24] can be simply add-on our $R_{EBM}$ to further improve the
performance. The proposed CRST+ $R_{EBM}$ obtains better or competitive
performances in all of the settings. Especially, it is able to outperform the
sophisticated generative pixel-level domain adaptation method GTA in both
architecture and objectives.

###  4.2 UDA for Semantic Segmentation

ine Method | Base Net | Road | SW | Build | Wall* | Fence* | Pole* | TL | TS | Veg. | Sky | PR | Rider | Car | Bus | Motor | Bike | mIoU | mIoU*
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
ine Source | DRN105 | 14.9 | 11.4 | 58.7 | 1.9 | 0.0 | 24.1 | 1.2 | 6.0 | 68.8 | 76.0 | 54.3 | 7.1 | 34.2 | 15.0 | 0.8 | 0.0 | 23.4 | 26.8
MCD [43] |  | 84.8 | 43.6 | 79.0 | 3.9 | 0.2 | 29.1 | 7.2 | 5.5 | 83.8 | 83.1 | 51.0 | 11.7 | 79.9 | 27.2 | 6.2 | 0.0 | 37.3 | 43.5
ine Source | PSPNet | 56.0 | 24.6 | 76.5 | 5.0 | 0.2 | 19.0 | 5.7 | 7.8 | 77.5 | 78.9 | 44.7 | 7.70 | 35.3 | 7.9 | 1.5 | 24.0 | 29.5 | 34.5
DCAN [64] |  | 82.8 | 36.4 | 75.7 | 5.1 | 0.1 | 25.8 | 8.0 | 18.7 | 74.7 | 76.9 | 51.1 | 15.9 | 77.7 | 24.8 | 4.11 | 37.3 | 38.4 | 44.9
ine Source only | DeepLabv2 | 55.6 | 23.8 | 74.6 | $-$ | $-$ | $-$ | 6.1 | 12.1 | 74.8 | 79.0 | 55.3 | 19.1 | 39.6 | 23.3 | 13.7 | 25.0 | $-$ | 38.6
AdaptSegNet [58] |  | 84.3 | 42.7 | 77.5 | $-$ | $-$ | $-$ | 4.7 | 7.0 | 77.9 | 82.5 | 54.3 | 21.0 | 72.3 | 32.2 | 18.9 | 32.3 | $-$ | 46.7
ine Source only | ResNet-38 | 32.6 | 21.5 | 46.5 | 4.8 | 0.1 | 26.5 | 14.8 | 13.1 | 70.8 | 60.3 | 56.6 | 3.5 | 74.1 | 20.4 | 8.9 | 13.1 | 29.2 | 33.6
CBST [24] |  | 53.6 | 23.7 | 75.0 | 12.5 | 0.3 | 36.4 | 23.5 | 26.3 | 84.8 | 74.7 | 67.2 | 17.5 | 84.5 | 28.4 | 15.2 | 55.8 | 42.5 | 48.4
CBST+$R_{EBM}$ |  | 77.8 | 37.8 | 75.1 | 17.0 | 10.8 | 36.3 | 20.9 | 32.8 | 78.2 | 79.4 | 55.7 | 21.9 | 82.2 | 13.0 | 20.6 | 45.3 | 43.6 | 49.6
CBST+${\mathcal{L}}_{EBM}$ |  | 65.7 | 26.4 | 78.1 | 19.0 | 3.2 | 31.5 | 21.9 | 32.5 | 81.4 | 78.4 | 56.8 | 29.8 | 82.3 | 29.7 | 21.0 | 45.4 | 42.8 | 49.8
ine Source only | DeepLabv2 | 64.3 | 21.3 | 73.1 | 2.4 | 1.1 | 31.4 | 7.0 | 27.7 | 63.1 | 67.6 | 42.2 | 19.9 | 73.1 | 15.3 | 10.5 | 38.9 | 34.9 | 40.3
CBST [24] |  | 68.2 | 32.0 | 76.6 | 15.1 | 2.5 | 29.2 | 20.7 | 33.8 | 73.2 | 78.1 | 60.0 | 25.0 | 83.5 | 15.1 | 20.2 | 40.2 | 42.1 | 48.8
CRST [10] |  | 65.4 | 27.7 | 80.1 | 18.3 | 0.7 | 38.1 | 22.4 | 30.8 | 83.2 | 80.0 | 60.6 | 27.5 | 82.6 | 33.5 | 19.0 | 42.8 | 44.6 | 50.4
CRST+$R_{EBM}$ |  | 67.6 | 24.9 | 78.9 | 18.0 | 9.2 | 36.9 | 26.9 | 35.5 | 82.4 | 79.4 | 59.5 | 29.8 | 85.0 | 29.9 | 22.0 | 47.4 | 45.7 | 51.7
CRST+${\mathcal{L}}_{EBM}$ |  | 64.6 | 30.3 | 78.7 | 16.8 | 5.8 | 39.0 | 25.2 | 31.8 | 85.6 | 80.7 | 64.5 | 29.3 | 83.5 | 35.3 | 20.1 | 44.5 | 45.6 | 51.8
ine |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |

Table 4: Experimental results of SYNTHIA $\rightarrow$ Cityscapes

The typical deep semantic segmentation is essentially making the pixel-wise
classification. We consider the challenging segmentation adaptation settings
in the synthetic-to-real scenario:

1) GTA5 [65] to Cityscapes [3]. There are 19 shared classes between these two
datasets. GTA5 dataset has 24,966 annotated images with the size of
1,052$\times$1,914, which are rendered by the GTA5 game engine.

2) SYNTHIA [66] to Cityscapes [3]. There are 16 shared classes between these
two datasets. We use the SYNTHIA-RAND-CITYSCAPES set, which consists of
$9,400$ labeled images. Following the standard protocols in [56, 58], we use
the full set of SYNTHIA/GTA5, and propose to adapt the segmentation model to
the Cityscapes training set with $2,975$ images. In testing, we evaluate on
the Cityscapes validation set with $500$ images.

To make a fair comparison with the other methods, we use the ResNet101 as the
backbone network as [24, 10]. Noticing that in PSPNET [67], Wide ResNet38 is a
stronger basenet than ResNet101. The basenet is pre-trained in ImageNet and
fine-tuned in source domain by SGD with learning rate $2.5\times 10^{-4}$,
weight decay $5\times 10^{-4}$, momentum $0.9$, batch size $2$, patch size
$512\times 1024$ and data augmentation of multi-scale training ($0.5-1.5$) and
horizontal flipping. All results on this dataset in the main paper are unified
to report the mIoU of the models at the end of the $6^{th}$ epoch.

We compare CBST/CRST+$R_{EBM}$ with the other methods in Table 3. Based on the
previous self-training UDA methods CBST or CRST, our additional EBM objective
outperforms CBST or CRST by about 2% w.r.t. the mean IoUs.

We achieve a new state-of-the-art, even compared with the generative pixel-
level domain adaptation method GTA [60], which is a relatively complex
algorithm in both architecture and objectives. As shown in Fig. 3, $R_{EBM}$
converges stably along the iterations. The performance on SYNTHIA to
Cityscapes is also consistent with the GTA5 to Cityscapes as shown in Table 4.

##  5 Conclusions

In this paper, we introduced the EBM’s objective into the self-training UDA.
Considering the lack of the target label, we resort to the pseudo-label, which
is usually noisy. The EBM optimization objective provides an additional signal
that is independent of the pseudo-labels. This approach can be more promising
in the UDA setting than in the supervised learning setting as it can be added
as either a regularization term or combined with a confidence-aware soft
pseudo-label in an EBM loss. In addition, our solution is orthogonal with the
recent advances in self-training, and can be added in a plug-and-play manner
without large computation and change in network structure. Extensive
experiments on both UDA classification and semantic segmentation demonstrated
the effectiveness and generality of our approach. More advanced EBM training
and self-training methods such as [30] can also be adapted to further improve
the performance, which is subject to our future work.

##  6 Data Availability Statement

This study does not involves new data collection. All of the datasets used in
this study are public available.

## References

## References

  * [1] Z. Wang, B. Du, Y. Guo, Domain adaptation with neural embedding matching, IEEE Transactions on Neural Networks and Learning Systems.
  * [2] S. Li, C. H. Liu, L. Su, B. Xie, Z. Ding, C. P. Chen, D. Wu, Discriminative transfer feature and label consistency for cross-domain image classification, IEEE Transactions on Neural Networks and Learning Systems.
  * [3] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele, The cityscapes dataset for semantic urban scene understanding, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3213–3223.
  * [4] B. Yang, A. J. Ma, P. C. Yuen, Learning domain-shared group-sparse representation for unsupervised domain adaptation, Pattern Recognition 81 (2018) 615–632.
  * [5] B. Yang, P. C. Yuen, Learning adaptive geometry for unsupervised domain adaptation, Pattern Recognition 110 (2021) 107638.
  * [6] X. Liu, X. Liu, B. Hu, W. Ji, F. Xing, J. Lu, J. You, C.-C. J. Kuo, G. E. Fakhri, J. Woo, Subtype-aware unsupervised domain adaptation for medical diagnosis, AAAI.
  * [7] R. Li, W. Cao, Q. Jiao, S. Wu, H.-S. Wong, Simplified unsupervised image translation for semantic segmentation adaptation, Pattern Recognition 105 (2020) 107343.
  * [8] J. Jia, L. Zhai, W. Ren, L. Wang, Y. Ren, L. Zhang, Transferable heterogeneous feature subspace learning for jpeg mismatched steganalysis, Pattern Recognition 100 (2020) 107105.
  * [9] D.-H. Lee, Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks, in: ICML Workshop on Challenges in Representation Learning, 2013.
  * [10] Y. Zou, Z. Yu, X. Liu, B. Kumar, J. Wang, Confidence regularized self-training, ICCV.
  * [11] X. Peng, B. Usman, N. Kaushik, D. Wang, J. Hoffman, K. Saenko, X. Roynard, J.-E. Deschaud, F. Goulette, T. L. Hayes, et al., Visda: A synthetic-to-real benchmark for visual domain adaptation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 2021–2026.
  * [12] H. Bagherinezhad, M. Horton, M. Rastegari, A. Farhadi, Label refinery: Improving imagenet classification through label progression, arXiv preprint arXiv:1805.02641.
  * [13] W. Grathwohl, K. Wang, J. Jacobsen, D. Duvenaud, M. Norouzi, K. Swersky, Your classifier is secretly an energy based model and you should treat it like one, arXiv.
  * [14] X. Liu, B. Hu, X. Liu, J. Lu, J. You, L. Kong, Energy-constrained self-training for unsupervised domain adaptation, 25th International Conference on Pattern Recognition (ICPR 2020).
  * [15] J. Li, B. Zhang, G. Lu, J. You, Y. Xu, F. Wu, D. Zhang, Relaxed asymmetric deep hashing learning: Point-to-angle matching, IEEE Transactions on Neural Networks and Learning Systems.
  * [16] T. Che, X. Liu, S. Li, Y. Ge, R. Zhang, C. Xiong, Y. Bengio, Deep verifier networks: Verification of deep discriminative models with deep generative models, in: ArXiv, 2019.
  * [17] X. Liu, W. Ji, J. You, G. E. Fakhri, J. Woo, Severity-aware semantic segmentation with reinforced wasserstein training, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12566–12575.
  * [18] X. Glorot, A. Bordes, Y. Bengio, Domain adaptation for large-scale sentiment classification: A deep learning approach, in: ICML, 2011, pp. 513–520.
  * [19] M. Long, Y. Cao, J. Wang, M. I. Jordan, Learning transferable features with deep adaptation networks, in: ICML, 2015.
  * [20] P. O. Pinheiro, Unsupervised domain adaptation with similarity learning, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8004–8013.  doi:10.1109/CVPR.2018.00835.
  * [21] E. Tzeng, J. Hoffman, T. Darrell, K. Saenko, Simultaneous deep transfer across domains and tasks, in: 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 4068–4076.  doi:10.1109/ICCV.2015.463.
  * [22] W. M. Kouw, An introduction to domain adaptation and transfer learning, arXiv:1812.11806.
  * [23] G. He, X. Liu, F. Fan, J. You, Classification-aware semi-supervised domain adaptation, in: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020.
  * [24] Y. Zou, Z. Yu, B. V. K. Vijaya Kumar, J. Wang, Unsupervised domain adaptation for semantic segmentation via class-balanced self-training, in: ECCV, 2018.
  * [25] C. Chen, W. Xie, W. Huang, Y. Rong, X. Ding, Y. Huang, T. Xu, J. Huang, Progressive feature alignment for unsupervised domain adaptation, in: CVPR, 2019\.
  * [26] G. Kang, L. Jiang, Y. Yang, A. G. Hauptmann, Contrastive adaptation network for unsupervised domain adaptation, in: CVPR, 2019.
  * [27] X. Gu, J. Sun, Z. Xu, Spherical space domain adaptation with robust pseudo-label loss, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 9101–9110.
  * [28] I. Triguero, S. García, F. Herrera, Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study, Knowledge and Information Systems 42 (2) (2015) 245–284.
  * [29] P. P. Busto, A. Iqbal, J. Gall, Open set domain adaptation for image and action recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence.
  * [30] L. Han, Y. Zou, R. Gao, L. Wang, D. Metaxas, Unsupervised domain adaptation via calibrating uncertainties, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2019, pp. 99–102.
  * [31] Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, F. Huang, A tutorial on energy-based learning, Predicting structured data 1 (0).
  * [32] E. Nijkamp, S.-C. Zhu, Y. N. Wu, On learning non-convergent short-run mcmc toward energy-based model, arXiv.
  * [33] M. Welling, Y. W. Teh, Bayesian learning via stochastic gradient langevin dynamics, in: ICML, 2011.
  * [34] J. Xie, Y. Lu, S.-C. Zhu, Y. Wu, A theory of generative convnet, in: ICML, 2016, pp. 2635–2644.
  * [35] Y. Du, I. Mordatch, Implicit generation and generalization in energy-based models, arXiv.
  * [36] Y. Song, Z. Ou, Learning neural random fields with inclusive auxiliary generators, arXiv.
  * [37] Y. LeCun, F. J. Huang, Loss functions for discriminative training of energy-based models., in: AIStats, Vol. 6, 2005, p. 34.
  * [38] M.-R. Amini, P. Gallinari, Semi-supervised logistic regression, in: ECAI, 2002.
  * [39] G. Celeux, G. Govaert, A classification em algorithm for clustering and two stochastic versions, Computational statistics & Data analysis 14 (3) (1992) 315–332.
  * [40] B. Gholami, P. Sahu, M. Kim, V. Pavlovic, Task-discriminative domain alignment for unsupervised domain adaptation, in: ICCV, 2019.
  * [41] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, V. Lempitsky, Domain-adversarial training of neural networks, The Journal of Machine Learning Research 17 (1) (2016) 2096–2030.
  * [42] Y. Grandvalet, Y. Bengio, Semi-supervised learning by entropy minimization, in: NIPS, 2005.
  * [43] K. Saito, K. Watanabe, Y. Ushiku, T. Harada, Maximum classifier discrepancy for unsupervised domain adaptation.
  * [44] K. Saito, Y. Ushiku, T. Harada, K. Saenko, Adversarial dropout regularization, in: ICLR, 2018.
  * [45] K. You, X. Wang, M. Long, M. Jordan, Towards accurate model selection in deep unsupervised domain adaptation, in: ICML, 2019.
  * [46] D. Hu, J. Liang, Q. Hou, H. Yan, Y. Chen, S. Yan, J. Feng, Panda: Prototypical unsupervised domain adaptation, ECCV.
  * [47] Y. Wu, D. Inkpen, A. El-Roby, Dual mixup regularized learning for adversarial domain adaptation, ECCV.
  * [48] P. O. Pinheiro, Unsupervised domain adaptation with similarity learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 8004–8013.
  * [49] S. Sankaranarayanan, Y. Balaji, C. D. Castillo, R. Chellappa, Generate to adapt: Aligning domains using generative adversarial networks, in: CVPR, 2018\.
  * [50] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [51] M. Long, H. Zhu, J. Wang, M. I. Jordan, Unsupervised domain adaptation with residual transfer networks, in: Advances in Neural Information Processing Systems, 2016, pp. 136–144.
  * [52] E. Tzeng, J. Hoffman, K. Saenko, T. Darrell, Adversarial discriminative domain adaptation, in: CVPR, 2017.
  * [53] M. Long, H. Zhu, J. Wang, M. I. Jordan, Deep transfer learning with joint adaptation networks, in: Proceedings of the 34th International Conference on Machine Learning-Volume 70, JMLR. org, 2017, pp. 2208–2217.
  * [54] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in pytorch.
  * [55] K. Saenko, B. Kulis, M. Fritz, T. Darrell, Adapting visual category models to new domains, in: European conference on computer vision, Springer, 2010, pp. 213–226.
  * [56] J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. A. Efros, T. Darrell, Cycada: Cycle-consistent adversarial domain adaptation, in: ICML, 2018\.
  * [57] Z. Wu, X. Han, Y.-L. Lin, M. Gokhan Uzunbas, T. Goldstein, S. Nam Lim, L. S. Davis, Dcan: Dual channel-wise alignment networks for unsupervised scene adaptation, in: The European Conference on Computer Vision (ECCV), 2018.
  * [58] Y.-H. Tsai, W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang, M. Chandraker, Learning to adapt structured output space for semantic segmentation, in: CVPR, 2018.
  * [59] T.-H. Vu, H. Jain, M. Bucher, M. Cord, P. Pérez, Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation, in: CVPR, 2019.
  * [60] Y. Zhang, Z. Qiu, T. Yao, D. Liu, T. Mei, Fully convolutional adaptation networks for semantic segmentation, in: CVPR, 2018.
  * [61] Y.-H. Tsai, K. Sohn, S. Schulter, M. Chandraker, Domain adaptation for structured output via discriminative representations, ICCV.
  * [62] Q. Lian, F. Lv, L. Duan, B. Gong, Constructing self-motivated pyramid curriculums for cross-domain semantic segmentation: A non-adversarial approach, ICCV.
  * [63] K. Saito, Y. Ushiku, T. Harada, K. Saenko, Adversarial dropout regularization, arXiv preprint arXiv:1711.01575.
  * [64] Z. Wu, X. Han, Y.-L. Lin, M. Gokhan Uzunbas, T. Goldstein, S. Nam Lim, L. S. Davis, Dcan: Dual channel-wise alignment networks for unsupervised scene adaptation, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 518–534.
  * [65] S. R. Richter, V. Vineet, S. Roth, V. Koltun, Playing for data: Ground truth from computer games, in: ECCV, 2016.
  * [66] G. Ros, L. Sellart, J. Materzynska, D. Vazquez, A. M. Lopez, The synthia dataset: A large collection of synthetic images for semantic segmentation of urban scenes, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3234–3243.
  * [67] H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia, Pyramid scene parsing network, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2881–2890.


# A Novel Non-Terrestrial Networks Architecture:
All Optical LEO Constellations with
High-Altitude Ground Stations

Pablo G. Madoery13, Juan A. Fraire23, Jorge M. Finochietto3, Halim
Yanikomeroglu1, Gunes Karabulut Kurt14  1 Non-Terrestrial Networks Lab,
Department of Systems and Computer Engineering, Carleton University, Ottawa,
Canada
2Inria, INSA Lyon, CITI, UR3720, 69621 Villeurbanne, France
3CONICET - Universidad Nacional de Cordoba, Cordoba, Argentina
4Poly-Grames Research Center, Department of Electrical Engineering,
Polytechnique Montréal, Montréal, Canada

###### Abstract

The emergence of low Earth orbit (LEO) satellite mega-constellations is
dynamically transforming the space sector. While free-space optical (FSO)
links efficiently facilitate inter-satellite data forwarding, they suffer from
atmospheric/weather conditions in the space-to-ground link. This study delves
into utilizing high-altitude platform stations (HAPS) as elevated relay
stations strategically positioned above terrestrial ground stations. We
introduce the concept of high-altitude ground stations (HAGS), an innovative
approach to enabling the development of all optical LEO satellite
constellations. The first contribution is an analysis of the HAGS-based
network architecture where the LEO spacecraft only hosts FSO transceivers.
Secondly, we execute an extensive simulation campaign to determine the gain of
HAGS, including a new equivalency model with the traditional ground station
approach. Finally, we examine the research challenges of implementing HAGS-
based, all optical LEO mega-constellations.

###### Index Terms:

High-Altitude Platform Stations, LEO Satellite Networks, LEO Mega-
Constellations, Free-Space Optical Links

##  I Introduction

Integrating high-altitude platform stations (HAPS) ascending beyond the
stratosphere has significantly invigorated the networking field [1]. HAPS has
been notably effective in their versatile applications as cellular base
stations, specifically as international mobile telecommunications base
stations (HIBS) [2]. Building on this concept, this paper ventures into the
space networking and communication domain, proposing using HAPS as elevated
ground stations (GS), or HAGS.

One of HAGS’s most groundbreaking opportunities is enabling all free-space
optical (FSO) LEO constellations. This development encompasses not only inter-
satellite links (ISLs) but also, crucially, ground-satellite (GSL) feeder
links, which have traditionally been hindered by weather-related impairments.
In a HAGS-based setup, satellites can use the same FSO interface for ISL and
HAGS links, eliminating the need for backup or hybrid RF systems. This
capability has two aspects: mitigating weather-related impairments and
extending visibility time due to HAGS’ higher altitude (see Fig. 1).

Figure 1: Opportunities of HAGS: enhanced visibility time and weather
impairment buffer.

HAGS serve a dual function, acting as a relay and a data buffer. HAGS
facilitates rate and weather decoupling, ensuring data delivery even when
satellites are out of reach from the GS. This feature is particularly
advantageous during periods of limited connectivity imposed by orbital
dynamics, allowing for the maximization of LEO-to-HAGS throughput. This aspect
is particularly relevant in Earth observation missions where data can flow in
a store-carry-and-forward fashion. The deployment of HAGS also broadens the
scope of satellite communication systems. By potentially serving as HIBS, HAGS
reduces the barrier to investment and expands operational capabilities.
Moreover, HAGS offers the capability to download and potentially process bulk
data directly on the HAGS. This approach accelerates data availability and
leverages HAGS as mission data centers from which data can be accessed and
analyzed.

This paper contributes significantly to the advancement of HAGS towards all
optical constellations. Firstly, we provide a detailed description of the HAGS
network architecture, laying the foundation for understanding its operational
framework and potential applications. Secondly, we conduct a comprehensive
performance evaluation of HAGS, employing model-based simulations to compare
its efficacy with traditional GS. This comparison quantifies the benefits of
HAGS and enables a GS-to-HAGS equivalency model, offering valuable insights
for decision-makers in the field. Lastly, we identify and discuss the open
challenges within the HAGS network, highlighting areas for future research and
development.

The rest of the paper is organized as follows. Section II offers a detailed
overview of HAPS and FSO satellite communication. We delve into the system
model in Section III. Section IV presents our evaluations and a model-based
equivalency analysis. Section V highlights and discusses the prevailing
challenges in HAGS. Finally, Section VI encapsulates our research’s key
findings and conclusions.

##  II Background

Non-terrestrial networks (NTN) will be a fundamental segment of future 5G/6G
systems [3], which require efficient bidirectional data transport mechanisms
via efficient feeder links to the GS. Likewise, a disruptive shift toward
constellations of small spacecraft with increasing Earth observation data
volume [4] further stresses data rate requirements. Whether for NTN or Earth
observation, RF and FSO feeder links, to and from the GS, are pushed to the
limit.

#### RF and FSO

Although the predominant approach for space-to-ground data transfer has
traditionally centered around RF bands such as Ku and Ka, [5] highlight the
limitations in capacity have largely reached a saturation point. Efforts are
underway to investigate higher bands, such as Q/V/W, which offer broader
spectrum availability, to enhance the capacity of very high throughput
satellite (VHTS) feeder links. However, the propagation impairments
experienced at tropospheric altitudes (ranging from 8 to 15 km) necessitate
implementing spatial diversity techniques across multiple ground gateways. In
this context, FSO links have garnered considerable attention as pivotal for
HAPS [6] and LEO satellites [7]. However, when utilized in space-to-ground
scenarios, FSO encounters significant challenges associated with pointing,
beam wander [5], and weather impairments [8], requiring the employment of
sophisticated acquisition, pointing, and tracking (APT) [9], signal processing
[10] and link diversity techniques [11]. As a result, when considered GSLs,
FSO systems are typically framed in complex hybrid FSO/RF systems where backup
RF links are used during inclement weather conditions [12].

#### HAPS

By utilizing platforms positioned in the stratosphere at approximately 20 km
altitude [13], HAPS can enhance communication coverage over large regions [1].
While HAPS are commonly associated with free-flying configurations,
alternative concepts involving tethered balloons or blimps are also being
contemplated to take advantage of a constant supply of power and data through
the tether. Nevertheless, the presence of the tether imposes altitude
restrictions, typically up to 5 km [14]. HAPS have been widely studied as part
of a space-air-ground integrated network, surveyed in [15]. Implementing
optical terminals in HAPS to link with LEO satellites dates back to 2007 when
authors in [16] introduced the stratospheric optical relay stations. Since
then, several contributions have been made at the system, application, and
link (optical only and hybrid) levels.

#### System and Applications

Multiple works aimed at linking HAPS and satellites. [17] state that the
European Data Relay System (EDRS) is already providing HAPS to satellite data
relay services. On the research front, cooperative HAPS and LEO were assessed
using a time-evolving graph, which is used for resource allocation via mixed
integer linear and non-linear programming in [18]. Authors in [19] also
propose a dynamic resource optimization method for shared spectrum utilization
between LEO and HAPS. Collaborative HAPS-satellite task offloading in networks
with connected HAPS and LEO satellites was studied in [20]. [21] introduced a
collaborative federated learning framework between HAPS and LEO satellites. In
the security domain, authors in [22] proposed optically-linked satellites and
HAPS to increase secrecy in satellite networks.

#### Link Models

Authors in [23] discussed the challenges in APT and the influence of the
atmosphere, background light, and flight qualification requirements in HAPS-
to-LEO FSO links. A two-hop (satellite-to-HAPS-to-ground) assessment was made
in [24]. The focus is on transparent and regenerative all-optical relay
solutions, but no storage or buffering is considered in the HAPS. An
analytical channel model and link design optimization for ground-to-HAPS free-
space optical communication networks were presented in [25]. The point-to-
point optical and radio link budget between HAPS, UAVs, and satellites on
different layers is studied in [26]. The paper in [27] finds that the
exponentiated Weibull (fading best fits different aperture sizes in most
weather conditions.

Figure 2: Downlink data handling and operations in traditional FSO and our
HAGS approach.

#### Hybrid FSO/RF

Connecting HAPS, LEO, and GS with hybrid FSO/RF is discussed in [28]. The
authors later presented an analysis for a dual-hop uplink in [29], confirming
hybrid FSO/RF can improve the performance of HAPS-to-LEO relay links. [30]
considered Hybrid FSO/RF for uplink and downlink with a focus on symbol error
rates. While these papers assumed a gamma–gamma distribution for FSO
communication, a follow-up paper [31] considers the exponential Weibull
fading, which provides a better fit for terminals with larger apertures.

Despite increasing research interest in integrating HAPS and LEO satellites
with FSO links, one critical aspect has been consistently overlooked: the
potential of HAGS to enable all-optical LEO constellations. This oversight is
significant, as it could profoundly impact the sector by allowing LEOs to
utilize optical links for both ISLs and GSLs. The remainder of this paper aims
to address this gap.

##  III System Model

Fig. 2 illustrates the traditional and HAGS-based models that are the focus of
our evaluation in this paper. Each LEO satellite within the constellation is
equipped with an FSO terminal in the depicted models. This terminal enables
the satellite to transmit data directly to a GS or HAGS.

#### Traditional Model

When considering a traditional FSO scheme [32], LEO satellites reach one of
the multiple GS via episodic contacts. If there is line-of-sight between a LEO
and a GS, the LEO can use the contact to download data at the highest possible
data rate (see Fig. 2-a). Suppose optical communication is not possible due to
cloud coverage (see Fig. 2-b). In that case, data will remain in the LEO,
which will take advantage of subsequent contact with another GS.

#### HAGS Model

Each HAGS is positioned 20 km above a corresponding GS, strategically placed
above cloud level (see Fig. 2-c). This positioning assumes that LEO-HAGS
contacts remain consistently available, provided a line-of-sight exists.
However, the connectivity between each HAGS and GS depends on cloud cover. In
scenarios where contact is temporarily unavailable, HAGS can store data in a
buffer (see Fig. 2-d) and forward it to the GS once the link is re-established
(see Fig. 2-e). This store-carry-and-forward capability, a cornerstone of the
delay-tolerant networking (DTN) architecture [33], is effectively utilized in
tandem with advanced routing algorithms like contact graph routing (CGR) [34].
CGR dynamically computes forwarding policies based on a pre-calculated and
distributed contact plan, ensuring efficient data transmission across the
network nodes.

#### Cloud Cover Model

Cloud cover significantly impacts the stability of FSO links, with clouds
acting as a primary obstruction. The unpredictability of cloud cover, its
varying duration, and the likelihood of clear skies can be modeled using an
exponential (Poisson) distribution. It delineates two fundamental outcomes: a
’blocked’ state during cloudy conditions and an ’unblocked’ state under clear
skies. Characterized as a memoryless distribution, this weather model
effectively captures the sporadic nature of cloud cover without any dependency
on past events. The model takes a single known average rate parameter for
cloud cover and can be described by $C(t)=\int^{t}\mu
e^{-\mu\tau}=1-e^{-\mu\tau}$. In this equation, $h(t)=\mu$ represents the
cloud cover rate, and $\tau$ is the time. The cloud cover rate simplifies to
the constant $\mu$ for any time. The mean time to cloud cover (TCC) is
$TCC=1/\mu$. Additionally, we generate the mean time to clear sky (TCS) using
the exponential model. TCC and TCS are the core input parameters for the cloud
cover model. Like a contact plan, a weather plan can be generated using TCC,
TCS, and the weather model. This plan includes all predicted cloud cover
events.

##  IV Evaluation and Analysis

For the comparative analysis of the models outlined in Section III, we
extended DtnSim, a discrete event-driven simulator based on the Omnet++
framework. Originally introduced in [35] and accessible via a public
repository111DtnSim is publicly available at:
https://gitlab.inria.fr/jfraire/dtnsim.git, DtnSim models each node (be it a
LEO, a HAGS, or a GS) as an Omnet module. These modules have sub-modules for
data generation, routing, and forwarding. We implemented the cloud cover
module to study the weather conditions in line with the TCC and TCS parameters
discussed in Section III.

#### Invariants

Utilizing Systems Tool Kit (STK), we simulated a week-long interaction between
the Iridium constellation and the KSAT’s GS network. Additionally, we modeled
interactions between the same LEO satellite and a series of HAGS, each
situated 20 km above its corresponding GS. The satellite must evacuate a
traffic load of 50 files, each 100 GB. All optical links (LEO-GS, LEO-HAGS,
and HAGS-GS) operate at a data rate of 8 Gbps. These links are assumed to be
ideal, with interruptions occurring solely due to cloud cover between the LEO
and GS or between the HAGS and GS.

#### Variables

To facilitate a comprehensive comparison, the traditional architecture varies
the number of GS (1, 2, 5, and 10), while the HAGS-inclusive architecture
adjusts the number of HAGS (1 to 5, each over its respective GS). The TCC is
altered between 0.1 and 40 hours, and the TCS ranges from 5 to 25 hours. Each
configuration undergoes 100 iterations to ensure statistical significance,
with performance metrics averaged and 95% confidence intervals included.

#### Metrics

The evaluation of performance is based on three key metrics: a) Delivery Ratio
(DR), which is determined by the percentage of generated files that
successfully reach their intended destination, b) Delivery Delay (DD),
calculated by subtracting the file’s generation time from its arrival time,
and mean and maximum values of c) Buffer Occupation (BO).

###  IV-A Results Analysis

((a)) Delivery ratio with TCS=5 hours.

((b)) Delivery ratio with TCS=25 hours.

Figure 3: Delivery delay for different TCS and TCC values.

((a)) Delivery delay with TCS=5 hours.

((b)) Delivery delay with TCS=25 hours.

Figure 4: Delivery delay for different TCS and TCC values.

#### Delivery Ratio

Fig. 3 illustrates the DR achieved under varying TCC and TCS values.
Generally, a lower TCC (indicating a higher cloud cover rate) results in fewer
files reaching their destination. Conversely, higher TCC values correlate with
a 100% delivery ratio across all schemes.

As depicted in Fig. 3-a), HAGS schemes maintain a complete DR even at low TCC
values, provided the TCS is fixed at 5 hours. This efficiency is attributed to
the LEO-HAGS link’s immunity to weather disruptions, allowing HAGS to store
files until the cloud-affected HAGS-GS link becomes available. Thus, LEO
satellites can consistently rely on the LEO-HAGS link for transferring data to
the ground, irrespective of temporary adverse conditions on the HAGS-GS link.

In contrast, schemes relying solely on GS lack this benefit. Cloud cover
impeding LEO-GS communication halts data download, forcing the LEO to await
another communication window with a different GS. This limitation is more
pronounced with fewer GS. Remarkably, even with 10 GS, these schemes
underperform compared to HAGS in high cloud cover scenarios (TCC=0.1 hour),
underscoring HAGS’s superior reliability in adverse weather conditions.

Fig. 3-b) demonstrates that while all schemes experience reduced performance
with longer TCS, those relying solely on GS suffer more significantly than
those incorporating HAGS. For instance, in the 1LEO-1GS scheme with a constant
TCC of 5 hours, the delivery ratio dramatically falls from 100% to below 60%
as TCS extends from 5 to 25 hours. In contrast, the 1LEO-1HAGS-1GS scheme
maintains a steady 100% DR across both TCS values, underscoring the gains of
incorporating HAGS into the communication architecture.

#### Delivery Delay

Fig. 4 presents the DD per file across varying TCC and TCS values. For an
accurate comparison, sections of the curves representing DR below 100% (see
Fig. 3) are greyed out. This approach ensures that only schemes delivering an
equal number of files are compared. The figure reveals that a decrease in TCC
generally increases DD for all schemes. Notably, schemes incorporating HAGS
consistently exhibit shorter DD than those using only GS, particularly at
higher cloud rates (lower TCC values).

Examining Fig. 4-a), we observe that at a TCC of 0.5 hours, the performance of
the 2 HAGS scheme closely matches that of the 10 GS scheme. This suggests that
a higher number of GS is necessary to match the efficiency of a smaller number
of HAGS, particularly during periods of high cloud cover. As TCC values
increase, this disparity diminishes. For instance, with a TCC of 2 hours, the
performance of 2 HAGS aligns with that of 5 GS, and at a TCC of 5 hours, a
single HAGS is roughly equivalent to 2 GS in terms of delivery efficiency.

Fig. 4-b), focusing on a TCS of 25 hours, reveals interesting dynamics in DD.
When setting TCC to 5 hours and extending TCS, the performance gap between the
4 HAGS scheme and the 10 GS scheme widens, increasingly favoring the GS-based
approach. This shift occurs because leveraging geographical diversity by
routing traffic through multiple GS becomes more advantageous in scenarios
with less frequent cloud cover and prolonged TCS. This strategy allows quicker
offloading of data compared to waiting for the availability of a HAGS-GS link.
Consequently, the benefits of using HAGS over multiple GS become more
pronounced at lower TCC and TCS values.

#### Buffer Occupation

Measurement of buffer occupancy when using one HAGS reveals that for small
values of TCC, the maximum occupancy can reach up to 60% of the total traffic
generated, while the average occupancy reaches up to 10%. These values
decrease to maximums of 15% and averages of 1% when TCC is greater than 20
hours.

###  IV-B Equivalency Analysis between GS and HAGS

This section analyzes the equivalency between GS and HAGS. We aim to ascertain
the requisite number of GS and HAGS to achieve comparable DD and DR. To
achieve this, we initially identify the intersection points of all curves in
Figs. 4 and 3 across all simulation iterations. This process yields a dataset
comprising nearly 20,000 equivalency points, where configurations of GS and
HAGS demonstrate similar DR and DD. We employ this dataset to train a Support
Vector Machine (SVM) model, utilizing a Radial Basis Function (RBF) kernel.
The RBF kernel is chosen for its ability to measure similarity based on the
spatial proximity of data points.

The generalized equivalency results, as deduced by this model, are depicted in
Fig. 5. The illustration reveals that under most combinations of TCC and TCS,
fewer HAGS are required to match the performance of a larger number of GS. For
instance, it is observed that 2 HAGS can deliver performance on par with 8 GS
for TCS ranging between 5 and 15 hours and a TCC of 0.1. Notably, as weather
conditions ameliorate (characterized by higher TCC and shorter TCS), the
advantage of using HAGS diminishes. Yet, they remain a more efficient
alternative than a larger array of GS, particularly for connecting all optical
LEO constellations.

Figure 5: Equivalency Analysis between GS and HAGS.

##  V Open Challenges

While promising, implementing HAGS poses significant challenges that must be
addressed to fully leverage their potential. Firstly, HAGS necessitates
sophisticated store-carry-and-forward end-to-end mechanisms integral to DTN,
especially when considering ISLs. This includes developing congestion-aware
routing strategies to efficiently direct data via ISL to less occupied buffers
among available HAGS and effective buffer status updating methods.
Additionally, there is a critical need for prioritizing the handling of real-
time telemetry and commands, ensuring their immediate transmission takes
precedence over regular payload or user data. Another significant challenge
lies in the design of HAGS network topology, especially for large-scale mega-
constellations. With more satellites than interfaces on the HAGS, it’s crucial
to devise mechanisms for optimal satellite connections to ensure efficient
data flow. Moreover, the physical layer of HAGS-to-LEO links poses challenges,
such as pointing, acquisition and tracking of a not completely stationary
HAGS. Addressing these challenges is essential for the reliable and effective
operation of HAGS in the dynamic environment of LEO satellite constellations.

##  VI Conclusion

This paper proposed high-altitude ground stations (HAGS) as a transformative
element towards all optical satellite mega-constellations. By enhancing
visibility time and shifting the weather-related bottleneck to a more
manageable, near-Earth platform, HAGS effectively navigates the complexities
of weather uncertainty and transitions to an appealing buffer-based model.
Within this paper, we conducted a comprehensive, simulation-based evaluation
to quantify the performance advantages of HAGS over traditional ground
stations (GS). Our findings demonstrate that HAGS can significantly reduce the
reliance on GS, under certain weather conditions. Furthermore, we established
an equivalency analysis between HAGS and GS, providing insights for strategic
decision-making in satellite network design. By identifying key challenges
within the field of HAGS, this work catalyzes advancements toward the
realization of all optical mega-constellations.

## Acknowledgments

This work has been supported by the National Research Council Canada’s (NRC)
High Throughput Secure Networks program within the Optical Satellite
Communications Consortium Canada (OSC) framework, Mitacs, the EU’s H2020 R&D
program under the Marie Skłodowska-Curie grant agreement No 101008233 (MISSION
project) and the French National Research Agency (ANR) projects
ANR-22-CE25-0014-01.

## References

  * [1] G. Karabulut Kurt, M. G. Khoshkholgh, S. Alfattani, A. Ibrahim, T. S. J. Darwish, M. S. Alam, H. Yanikomeroglu, and A. Yongacoglu, “A vision and framework for the high altitude platform station (HAPS) networks of the future,” _IEEE Communications Surveys & Tutorials_, vol. 23, no. 2, pp. 729–779, 2021.
  * [2] S. Euler, X. Lin, E. Tejedor, and E. Obregon, “High-altitude platform stations as international mobile telecommunications base stations: A primer on HIBS,” _IEEE Vehicular Technology Magazine_ , vol. 17, no. 4, pp. 92–100, 2022.
  * [3] S. Kota and G. Giambene, “6G integrated non-terrestrial networks: Emerging technologies and challenges,” in _IEEE Int. Conference on Communications Workshops (ICC Workshops)_ , 2021.
  * [4] S. Marcuccio, S. Ullo, M. Carminati, and O. Kanoun, “Smaller satellites, larger constellations: Trends and design issues for earth observation systems,” _IEEE Aerospace and Electronic Systems Magazine_ , vol. 34, no. 10, pp. 50–59, 2019.
  * [5] E. Zedini, A. Kammoun, and M.-S. Alouini, “Performance of multibeam very high throughput satellite systems based on FSO feeder links with HPA nonlinearity,” _IEEE Transactions on Wireless Communications_ , vol. 19, no. 9, pp. 5908–5923, 2020.
  * [6] M. Alzenad, M. Z. Shakir, H. Yanikomeroglu, and M.-S. Alouini, “FSO-based vertical backhaul/fronthaul framework for 5G+ wireless networks,” _IEEE Communications Magazine_ , vol. 56, no. 1, pp. 218–224, 2018.
  * [7] M. Toyoshima, “Applicability of space laser communications for low Earth orbit satellite constellations,” in _Optical Fiber Communication Conference (OFC)_ , 2022.
  * [8] J. Liang, A. U. Chaudhry, E. Erdogan, and H. Yanikomeroglu, “Link budget analysis for free-space optical satellite networks,” in _IEEE Int. Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM)_ , 2022\.
  * [9] H. Kaushal and G. Kaddoum, “Optical communication in space: Challenges and mitigation techniques,” _IEEE Communications Surveys & Tutorials_, vol. 19, no. 1, pp. 57–96, 2016.
  * [10] L. Paillier _et al._ , “Space-ground coherent optical links: ground receiver performance with adaptive optics and digital phase-locked loop,” _Journal of Lightwave Technology_ , vol. 38, no. 20, pp. 5716–5727, 2020\.
  * [11] S. Gong _et al._ , “Network availability maximization for free-space optical satellite communications,” _IEEE Wireless Commun. Letters_ , vol. 9, no. 3, pp. 411–415, 2019.
  * [12] H. Kazemi, M. Uysal, and F. Touati, “Outage analysis of hybrid FSO/RF systems based on finite-state Markov chain modeling,” in _Int. Workshop in Optical Wireless Communications (IWOW)_ , 2014.
  * [13] M. S. Alam, G. Karabulut Kurt, H. Yanikomeroglu, P. Zhu, and N. D. Đào, “High altitude platform station based super macro base station constellations,” _IEEE Communications Magazine_ , vol. 59, no. 1, pp. 103–109, 2021.
  * [14] B. E. Y. Belmekki and M.-S. Alouini, “Unleashing the potential of networked tethered flying platforms: Prospects, challenges, and applications,” _IEEE Open Journal of Vehicular Technology_ , vol. 3, pp. 278–320, 2022.
  * [15] J. Liu, Y. Shi, Z. M. Fadlullah, and N. Kato, “Space-air-ground integrated network: A survey,” _IEEE Communications Surveys & Tutorials_, vol. 20, no. 4, pp. 2714–2741, 2018.
  * [16] D. Giggenbach, J. Horwath, and B. Epple, “Optical satellite downlinks to optical ground stations and high-altitude platforms,” in _IST Mobile and Wireless Communications Summit_ , 2007.
  * [17] M. Agnew, L. Renouard, and A. Hegyi, “EDRS-SpaceDataHighway: Near-real-time data relay services for LEO satellites and HAPs,” in _AIAA Int. Communications Satellite System Conference (ICSSC)_ , 2012.
  * [18] Z. Jia, M. Sheng, J. Li, and Z. Han, “Toward data collection and transmission in 6G space–air–ground integrated networks: Cooperative HAP and LEO satellite schemes,” _IEEE Internet of Things Journal_ , vol. 9, no. 13, pp. 10 516–10 528, 2021.
  * [19] Y. Li, N. Deng, and W. Zhou, “A hierarchical approach to resource allocation in extensible multi-layer LEO-MSS,” _IEEE Access_ , vol. 8, pp. 18 522–18 537, 2020.
  * [20] L. Zhang, H. Zhang, C. Guo, H. Xu, L. Song, and Z. Han, “Satellite-aerial integrated computing in disasters: User association and offloading decision,” in _IEEE Int. Conference on Communications (ICC)_ , 2020.
  * [21] M. Elmahallawy and T. Luo, “FedHAP: Fast federated learning for LEO constellations using collaborative HAPs,” in _IEEE Int. Conference on Wireless Communications and Signal Processing (WCSP)_ , 2022.
  * [22] O. B. Yahia, E. Erdogan, and G. Karabulut Kurt, “On the use of haps to increase secrecy performance in satellite networks,” in _IEEE Int. Conference on Communications Workshops (ICC Workshops)_ , 2021.
  * [23] F. Fidler, M. Knapek, J. Horwath, and W. R. Leeb, “Optical communications for high-altitude platforms,” _IEEE Journal of Selected Topics in Quantum Electronics_ , vol. 16, no. 5, pp. 1058–1070, 2010.
  * [24] M. Q. Vu, N. T. Nguyen, H. T. Pham, and N. T. Dang, “Performance enhancement of LEO-to-ground FSO systems using all-optical HAP-based relaying,” _Physical Communication_ , vol. 31, pp. 218–229, 2018.
  * [25] H. Safi, A. Dargahi, J. Cheng, and M. Safari, “Analytical channel model and link design optimization for ground-to-HAP free-space optical communications,” _Journal of Lightwave Technology_ , vol. 38, no. 18, pp. 5036–5047, 2020.
  * [26] Y. Ata and M.-S. Alouini, “HAPS based FSO links performance analysis and improvement with adaptive optics correction,” _IEEE Transactions on Wireless Communications_ , vol. 22, no. 7, pp. 4916–4929, 2023.
  * [27] E. Erdogan, I. Altunbas, G. Karabulut Kurt, M. Bellemare, G. Lamontagne, and H. Yanikomeroglu, “Site diversity in downlink optical satellite networks through ground station selection,” _IEEE Access_ , vol. 9, pp. 31 179–31 190, 2021.
  * [28] R. Swaminathan, S. Sharma, and A. MadhuKumar, “Performance analysis of HAPS-based relaying for hybrid FSO/RF downlink satellite communication,” in _IEEE Vehicular Technology Conference_ , 2020.
  * [29] S. R, S. Sharma, N. Vishwakarma, and A. S. Madhukumar, “HAPS-based relaying for integrated space–air–ground networks with hybrid FSO/RF communication: A performance analysis,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 57, no. 3, pp. 1581–1599, 2021.
  * [30] S. Shah, M. Siddharth, N. Vishwakarma, R. Swaminathan, and A. Madhukumar, “Adaptive-combining-based hybrid FSO/RF satellite communication with and without HAPS,” _IEEE Access_ , vol. 9, pp. 81 492–81 511, 2021.
  * [31] O. B. Yahia, E. Erdogan, G. Karabulut Kurt, I. Altunbas, and H. Yanikomeroglu, “HAPS selection for hybrid RF/FSO satellite networks,” _IEEE Transactions on Aerospace and Electronic Systems_ , vol. 58, no. 4, pp. 2855–2867, 2022.
  * [32] Airbus Defence & Space SAS, “Towards the all optical satellite communications system,” https://nebula.esa.int/sites/default/files/neb_study/1276/C4000117594ExS.pdf, accessed: August 28, 2024.
  * [33] V. Cerf, S. Burleigh, A. Hooke, L. Torgerson, R. Durst, K. Scott, K. Fall, and H. Weiss, “Delay-tolerant networking architecture,” Internet Requests for Comments, RFC 4838, April 2007.
  * [34] J. A. Fraire, O. De Jonckère, and S. C. Burleigh, “Routing in the space internet: A contact graph routing tutorial,” _Journal of Network and Computer Applications_ , vol. 174, p. 102884, 2021.
  * [35] J. A. Fraire, P. G. Madoery, F. Raverta, J. M. Finochietto, and R. Velazco, “DtnSim: Bridging the gap between simulation and implementation of space-terrestrial DTNs,” in _IEEE Int. Conference on Space Mission Challenges for Information Technology (SMC-IT)_ , 2017.