# Leveraging Topology for Domain Adaptive Road Segmentation in Satellite and
Aerial Imagery

Javed Iqbal<EMAIL_ADDRESS>Aliza Masood<EMAIL_ADDRESS>Waqas Sultani<EMAIL_ADDRESS>Mohsen Ali<EMAIL_ADDRESS>Information Technology University, Pakistan

###### Abstract

Getting precise aspects of road through segmentation from remote sensing
imagery is useful for many real-world applications such as autonomous
vehicles, urban development and planning, and achieving sustainable
development goals (SDGs) 111https://sdgs.un.org/goals. Roads are only a small
part of the image, and their appearance, type, width, elevation, directions,
etc. exhibit large variations across geographical areas. Furthermore, due to
differences in urbanization styles, planning, and the natural environments;
regions along the roads vary significantly. Due to these variations among the
train and test domains (domain shift), the road segmentation algorithms fail
to generalize to new geographical locations. Unlike the generic domain
alignment scenarios, road segmentation has no scene structure and generic
domain adaptive segmentation methods are unable to enforce topological
properties like continuity, connectivity, smoothness, etc., thus resulting in
degraded domain alignment. In this work, we propose a topology-aware
unsupervised domain adaptation approach for road segmentation in remote
sensing imagery. During domain adaptation for road segmentation, we predict
road skeleton, an auxiliary task to enforce the topological constraints. To
enforce consistent predictions of road and skeleton, especially in the
unlabeled target domain, the conformity loss is defined across the skeleton
prediction head and the road-segmentation head. Furthermore, for self-
training, we filter out the noisy pseudo-labels by using a connectivity-based
pseudo-labels refinement strategy, on both road and skeleton segmentation
heads, thus avoiding holes and discontinuities. Extensive experiments on the
benchmark datasets show the effectiveness of the proposed approach compared to
existing state-of-the-art methods. Specifically, for SpaceNet to DeepGlobe
adaptation, the proposed approach outperforms the competing methods by a
minimum margin of 6.6%, 6.7%, and 9.8% in IoU, F1-score, and APLS,
respectively.

###### keywords:

Remote sensing, Road segmentation, Domain adaptation, Self-training, Deep
learning, Sustainable cities and communities.

##  1 Introduction

Roads, defined simply as ‘a wide way leading from one place to another’
222https://www.lexico.com/definition/road, play a vital role in limiting or
expanding the opportunities to connect. Building a reliable, up-to-date, and
comprehensive map of roads with attributes like width, number of lanes, type,
etc., is crucial not only for economic analysis, public policy, future
development (building smart cities), and private businesses but to the modern
world of autonomous transportation too. However, segmenting the unlabeled
roads or updating the labels and attributes for the existing ones is labor-
intensive, manually cumbersome, and costly [1], which becomes further crucial
and time-sensitive in the case of natural disasters. Some recent works used
graph-based approaches to extract and map road networks [2, 3]. However, these
graph-based approaches only predict the center lines as edges between two
possible nodes and are not suitable for extracting road characteristics like
road width, number of lanes, type, etc. In this work, we specifically focus on
the segmentation and adaptation of roads in remote-sensing imagery.

Figure 1: The source-trained model fails to generate accurate road
segmentation for target domain images. Predicted roads in the target domain
are broken and disconnected, and as indicated from the probability map, the
probabilities slowly decrease while continuing in the direction of the road is
too small to be labeled as a road. After the adaptation, we see better results
with more spatial connectivity and completeness. Best viewed in color.

Recently, due to the availability of annotated datasets, such as DeepGlobe [4]
and SpaceNet [5], deep learning-based automatic road segmentation from
satellite imagery has shown promising results [6, 2, 3, 7]. However, these
models fail to segment roads accurately in unseen geographical regions [8, 9].
This behavior is attributed to the domain shift between the source (training)
dataset and the unseen target (testing) dataset from another domain. For
satellite imagery, domain shift can occur due to the different image-capturing
modalities, lighting conditions, geographical patterns, and the resolution
difference between the source and target domains.

Compared to generic domain adaptation methods for semantic segmentation [10,
11, 12, 13, 14, 15], the domain adaptation for satellite road segmentation
poses specific challenges. In the case of generic semantic segmentation, the
structure of the scene (roads and objects are mostly in the lower part of the
image and the sky is above) helps in domain adaptation. Whereas,
satellite/aerial imagery does not have such structure or geometry available
[16]. Road’s appearance in satellite imagery is more dependent on the
attributes of the geographical region; buildings on the edge of the road,
natural environment, deserts, etc., and the adaptation performance usually
suffers due to large and unstructured background area. Additionally, based on
the resolution of the satellite/aerial imagery, road width varies across
regions and may only constitute a limited number of pixels making the road
segmentation adaptation problem much more challenging.

To address the challenge of road segmentation domain adaptation, we propose a
topology-based self-supervised adaptation strategy. Depending upon the
construction, geography of the region, and satellite imagery resolution, road
widths vary across the regions (source and target domains in our case) as
illustrated in Figure 1. This further exacerbates the noisy nature of the
pseudo-labels that are generated using the models trained on source data. Fig
1 also shows the deteriorated performance of the source model before
adaptation. We observe that the road skeleton is well defined across domains,
conceptually more domain invariant (i.e., does not depend on road width), and
can help in topology preserving during domain adaptation. Hence, we
incorporate road skeleton (center-line) prediction alongside road segmentation
in a multi-task learning scenario, making the model more generalizable and
using this auxiliary information to help in generating more refined pseudo
labels during self-training.  Note that the road skeleton is not only properly
defined across domains and is conceptually more domain invariant (i.e., does
not depend upon road width), it is helpful in preserving the topological
concepts such as connectivity, continuity, and structure over the skeleton.

To enforce road connectivity, inspired by the classical line detection method
[17], we define a connectivity based pseudo-labels refinement strategy (CBR).
Using connectivity (neighborhood) information helps us improve the quality of
pseudo-labels, which eventually increases the adaptation performance (Figure 6
and Figure 7). To decrease the difference between source and target domain
feature distribution, we perform adversarial learning at the encoder level.
Note that we show that adversarial learning alone cannot align both domain
features due to large data imbalance and is driven by the background (Table
4).

To summarize, this work presents the following contributions. We identify the
major limitation of applying existing domain adaptation methods over the road-
segmentation problem, i.e., these methods do not exploit the topological
properties of the road. To overcome these limitations, we design a multi-task
learning strategy of predicting the skeleton along with the road segmentation
head, resulting in enriched features. Conformity loss between skeleton
prediction and road segmentation is applied as a regularizer to guide the
adaptation over the unlabeled target domain. We exploit the desired properties
of the road such as connectivity and continuity, to clean the pseudo-labels
(connectivity-based refinement of pseudo-labels) resulting in better self-
supervised learning/self-training. State-of-the-art domain adaption
performance is achieved on benchmark road segmentation datasets.

##  2 Related Work

In recent years, the accessibility of satellite imagery has improved multi-
fold, resulting in the collection of large satellite imagery datasets. Deep
learning has been applied to various problems including building/built-up
regions segmentation [18, 16, 19, 20], destruction and change detection [21,
22, 23, 24, 25], houses/structures counting [26, 27, 28, 29, 30], and slums
detection and mapping [31, 32] are explored. Despite the great success of deep
learning-based approaches in remote sensing imagery, very little attention is
devoted to domain adaptive road segmentation. In this section, we briefly
review the road segmentation and adaptation methods related to the proposed
approach.

###  2.1 Road Segmentation

In recent years many deep learning-based models are proposed for semantic
segmentation [33, 34, 35, 36, 37, 38]. Mostly, these approaches are defined
for ground road scenes or other generic scenes. Road segmentation in satellite
and aerial images is a challenging problem due to large variations in
background, shadows of the building, trees, etc. Modifying these generic
segmentation methods, several approaches are designed for road extraction from
aerial and satellite imagery [6, 39]. The authors in [39] $\&$ [40] tried to
learn direction and connectivity alongside road segmentation respectively to
preserve direction and connectivity information. Similarly, authors in [41]
embedded dilated convolutions in [42] to better preserve edges and boundaries
information. Although these approaches work reasonably well, they require
large and precisely annotated road segmentation datasets, which are not
trivial to collect.

Road skeleton segmentation can help in capturing the structure and topology of
the roads which are important characteristics of road networks. The authors in
[43] used skeleton/center-line extraction to improve the broken links in road
segmentation incrementally. Whereas, [44, 45] predict skeleton along road
segmentation. [44] reported only slight improvement in road segmentation when
the skeleton prediction task was added. These methods require labeled data, do
not attempt to perform domain alignment, and do not employ any structural or
topological consistency, hence, producing deteriorated results when exposed to
unseen target domain images. We explicitly enforce structural consistency by
employing conformity loss, and for self-supervised domain adaptation, improve
pseudo-label quality by performing connectivity-based refinement.

###  2.2 Domain Adaptation

Domain adaptation approaches [46, 47, 48, 49, 50, 51, 10, 16, 52] are widely
used to minimize the domain shift between source and target domains.
Adversarial learning is the most common approach to minimize the domain gap
between source and target domain images segmentation either at input space
[53, 54], structured output space [55, 56, 57, 58, 48] or feature space
representations [59, 16]. In recent years, many domain adaptation approaches
based on self-training are proposed with state-of-the-art results for generic
semantic segmentation [60, 61, 10, 51, 52, 62]. The main idea of self-training
based methods is, for each class, to select the most confident pixels as
pseudo-labels and then finetune the source domain trained model using these
pseudo-labels for target images [10, 61]. These approaches exploit contextual
information (such as vehicles will always be on the ground, and the sky will
always be above) and since no such structure exists in satellite images, these
approaches fail to perform well on road segmentation problems. Similarly, the
adversarial learning matches the global image features, or output probability
distributions, which in the case of two classes like road segmentation, is
overwhelmed by background (non-road areas).

Besides great progress in domain adaptation for semantic segmentation, the
road segmentation adaptation is still an open challenge [63, 64, 65, 9, 8].
The authors in [63] presented two-stage adversarial learning, where the first
stage aligns inter-domain global features and the second stage aligns features
of intra-domain hard and easy examples using adversarial learning. Similarly,
authors in [8] tried to match the source and target features at global and
local levels for better adaptation. However, these approaches defined for
domain adaptation of road segmentation still fail to present road-specific
constraints to preserve the characteristics and topology of the roads.
Compared to these existing approaches, we explicitly try to preserve the road
structure and connectivity by employing simultaneous road and skeleton
segmentation and adaptation, structural conformity preservation loss, and
connectivity-based pseudo-labels refinement. These problem-specific components
significantly improve the performance compared to existing state-of-the-art
methods.

##  3 Methodology

In this section, we provide details of our proposed unsupervised domain
adaption (UDA) method for road segmentation. Our approach exploits the
topological property of the road by performing connectivity-based pseudo-label
refinement for improved pseudo-labels selection. During adaptation, the model
is encouraged to learn domain invariant features through the road skeleton
prediction task. Since the target domain is unlabeled, the skeleton prediction
task is trained by applying consistency/conformity loss across the skeleton
prediction and road segmentation head. We use DLinkNet [41], as a base road
segmentation method. Due to dilated convolutions, high receptive field and
skip connections, DLinkNet captures global, multi-scale information and
contextual information, resulting in more accurate segmentation results (Table
1).

Figure 2: Network Architecture: We use a multi-head segmentation network to
predict roads and skeleton segmentation with a shared encoder. Cross entropy
loss for both road and skeleton is back-propagated for source and target
domain images based on ground truth labels and pseudo-labels, respectively. A
structural conformity loss is used to ensure the structural conformity between
road and skeleton predictions using the skeleton pseudo-labels. Moreover, a
discriminator is defined at the encoder to align the features of source and
target domain images. During the evaluation, only the road segmentation head
is used.

###  3.1 Preliminaries

####  3.1.1 Problem Formulation

Let $\mathcal{S}$ be the source domain consisting of labeled satellite image,
$\mathit{I}_{s}$, and the corresponding road segmentation labels,
$\mathit{Y}_{s}$. The objective of unsupervised domain adaptation is to
generalize the model trained on $\mathcal{S}$ to the target domain
$\mathcal{T}$, where satellite images $\mathit{I_{t}}$ are available and
ground-truth labels $\mathit{Y}_{t}$ are not available. Without the loss of
generality, we can assume $\mathit{I}_{s}$,
$\mathit{I_{t}}$$\in\mathbb{R}^{H\times W\times 3}$ and $\mathit{Y}_{s}$,
$\mathit{Y}_{t}$$\in\mathbb{Z}^{H\times W\times C}$, where $H$ and $W$ are
height and width of the image, and $C$ is the total number of classes a pixel
can belong to. In a supervised learning setting for source domain
$\mathcal{S}$, the objective is to learn segmentation model $g\in
G:I\rightarrow Y$, by minimizing the cross-entropy loss function
$\mathcal{L}_{seg}$, defined in Eq. (1).

$\displaystyle\scriptsize\mathcal{L}_{seg}(I_{s},Y_{s})=-\frac{1}{H\times
W}\sum_{h,w}^{H,W}\sum_{c}^{C}Y_{s}^{h,w,c}\log(P_{s}^{h,w,c})$ (1)

let $g(I_{s})=P_{s}\in\mathbb{R}^{H\times W\times C}$ is the segmentation
probability map, where $P_{s}^{h,w,c}$ is probability of pixel $(h,w)$
belonging to class $c$.

Pseudo-Label Selection:  To overcome the non-availability of the $Y_{t}$ in
the target domain, typical self-supervised adaptation methods perform pseudo-
label selection from probable predictions [51, 10, 12, 61]. Let
$P_{t}=g(I_{t})$ be the output probability volume for the target image
$I_{t}$, predicted by the source trained model, then pseudo-labels
$\hat{Y_{t}}$ are computed by setting
$\hat{Y_{t}}^{w,h,c}=\mathbbm{1}[P_{t}^{h,w,c}=\max(P_{t}^{h,w})]$. Since
these predictions might be noisy for $\mathit{I_{t}}$, low-confidence ones are
masked out and not used during the back-propagation.

$\displaystyle\begin{split}&\mathcal{\hat{L}}_{seg}(I_{t},\hat{Y}_{t},m_{t})=\phantom{d}\\\
&-\frac{1}{H\times
W}\sum_{h,w}^{H,W}m^{h,w}_{t}\sum_{c}^{C}\hat{Y}_{t}^{h,w,c}\log(P_{t}^{h,w,c})\end{split}$
(2)

where $m^{h,w}_{t}\in\\{0,1\\}$ depending upon whether this pixel is in the
pseudo-label set or not. Generally, $m^{h,w}_{t}=\mathbbm{1}[P_{t}^{h,w,k}\geq
T]$, where $T$ is user defined threshold and
$k=\operatorname*{arg\,max}_{c}P_{t}^{h,w,c}$. The network is trained over the
$\mathit{Y}_{s}$, and pseudo labels $\hat{Y}_{t}$ (using $m$), which are
updated after each round (set of convergence iterations, more details in Sec.
4.1).

####  3.1.2 Domain adaptation for road segmentation

In the case of road segmentation, Eq. 1 and Eq. 2 are reduced to binary cross
entropy instead of categorical cross-entropy. This results in
$P_{s},P_{t}\in\mathbb{R}^{H\times W\times 1}$; representing the output
probability map of whether a pixel belongs to the road class or not. The
pseudo-labels are defined as
$\hat{Y_{t}}^{h,w}=\mathbbm{1}[P_{t}^{h,w}\geq(1-P_{t}^{h,w})]$. We
backpropagate gradients for those pixels only where we are sure (e.g. have
high predictive probability) that they either belong to the road or they do
not belong road. Therefore, mask $m_{t}$ is computed by checking two
thresholds
$m^{h,w}_{t}=\mathbbm{1}[P_{t}^{h,w}>T_{r}||(1-P_{t}^{h,w})>T_{b}]$, where
$T_{r}\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\ T_{b}$ are
thresholds for the road and background class (not belonging to the road),
respectively.

###  3.2 Multi-task self-supervised domain adaptation

One of the components of the domain shift is the variations in road width
across domains, and how the boundary is delineated due to changes in the
background (buildings on the edge of the road, natural environment, desert,
etc.,). On the other hand, the concept of the center-line or skeleton remains
the same across domains. In addition, the skeleton in general embodies
topological information like continuity and connectivity much better than what
is captured by the road-surface segment. Therefore, in this work, we employ
skeleton prediction as an auxiliary task to improve road segmentation across
domains (Figure 2). The training labels for the skeleton prediction are
obtained by skeletonizing true labels of the source domain $\mathcal{S}$ to
the single pixel width using [66]. The skeleton segmentation head is trained
using Eq. 3,

$\displaystyle\begin{split}\mathcal{L}_{s}^{{\phi}}(I_{s},Y_{s}^{{\phi}})=-\frac{1}{HW}\sum_{h,w}^{H,W}[Y_{s}^{{\phi,(h,w)}}\leavevmode\nobreak\
\log(P_{s}^{{\phi},(h,w)})\\\ +(1-Y_{s}^{{\phi},(h,w)})\leavevmode\nobreak\
\log(1-P_{s}^{{\phi},(h,w)})],\end{split}$ (3)

where $\mathcal{L}_{s}^{{\phi}}(I_{s},Y_{s}^{{\phi}})$ is the skeleton
segmentation loss for source domain image $I_{s}$ with respective labels
$Y_{s}^{{\phi}}\in\mathbb{R}^{H\times W}$ and predicted probability map
$P_{s}^{{\phi}}\in\mathbb{R}^{H\times W}$. For target domain images, similar
to road pseudo-labels, we define skeleton pseudo-labels based on the
probabilities of the skeleton segmentation head and then use Eq. 4.

$\displaystyle\small\begin{split}\mathcal{L}_{t}^{{\phi}}(I_{t},\hat{Y}_{t}^{{\phi}})=-\frac{1}{HW}\sum^{H,W}_{w,h}m^{\phi,(h,w)}_{t}[\hat{Y}_{t}^{{\phi},(h,w)}\leavevmode\nobreak\
\log(P_{t}^{{\phi},(h,w)})\\\
+(1-\hat{Y}_{t}^{{\phi},(h,w)})\leavevmode\nobreak\
\log(1-P_{t}^{{\phi},(h,w)})]\end{split}$ (4)

where
$m^{\phi,(h,w)}_{t}=\mathbbm{1}[P_{t}^{\phi,(h,w)}>T_{r}^{\phi}||(1-P_{t}^{\phi,(h,w)})>T_{b}^{\phi}]$,
and $T_{r}^{\phi}$ and $T_{b}^{\phi}$ are thresholds for the skeleton and
background class, respectively. $m^{\phi,(h,w)}_{t}$ allows the selection of
pseudo-labels, where either the model has predicted the road with high
confidence or the background with high confidence.
$\hat{Y}_{t}^{{\phi}}\in\mathbb{R}^{H\times W}$ and
$P_{t}^{{\phi}}\in\mathbb{R}^{H_{t}\times W_{t}}$ represent the skeleton
pseudo-labels and predicted probability map for input target image $I_{t}$
respectively.

Figure 3: An illustration of the connectivity-based pseudo label refinement
(CBR) approach for both road and skeleton pseudo labels. After applying CBR,
the pseudo-labels are increased while reducing the number of ‘not-selected’
pixels based on improving connectivity and completeness.

###  3.3 Connectivity-Based Pseudo Labels Refinement (CBR)

Effective self-training-based domain adaptation requires a reasonable number
of good pseudo-labels for each class. The confidence-based pseudo-label
assignment and threshold-based masking (Sec. 3.1) results in highly imbalanced
pseudo-labels. In the case of road segmentation, these pseudo-labels might
result in broken segments or holes, thus the concept of continuity might not
be enforced on the target domain as shown in Figure 4 (before CBR).

Note that the pseudo-label assignment and selection are based on the two
thresholds, $T_{r}$ and $T_{b}$, where $T_{r}$ selects the road pixels, and
$T_{b}$ is used for selecting non-road pixels. For ones having predicted
probability in-between $T_{r}$ and $(1-T_{b})$, we set $m^{h,w}_{t}=0$ (since
we are not confident about their label, we call these pixels as ’not-selected’
pixels), and they do not contribute to the loss (Eq. 2) and therefore are not
used to update the weights of the model. Lowering $T_{r}$ will result in the
background being labeled as roads and a high $T_{r}$ will result in disjoint
road pieces. Therefore, we perform connectivity-based pseudo-label refinement,
based on the two thresholds, $T_{h}(=T_{r})$ and $T_{l}(>=1-T_{b})$, to
improve structure and connectivity in the road and skeleton pseudo-labels.

Figure 4: Pseudo-labels before and after connectivity-based refinement (CBR).
The pseudo-labels are more complete and connected after CBR indicating their
quality is improved significantly.

Let $\mathcal{R}=\\{\forall(h,w)|m^{h,w}_{t}=1\leavevmode\nobreak\
\&\leavevmode\nobreak\ P^{h,w}_{t}>T_{h}\\}$ be set of selected pixels
assigned the pseudo-label road. Let
$\mathcal{M}_{t}=\\{\forall(h,w)|m^{h,w}_{t}=0\\}$ be set of all ‘not-
selected’ pixels. Any pixel in $\mathcal{M}_{t}$ that is connected to the road
pseudo-labeled pixels in $\mathcal{R}$ is selected as the road pseudo-label.
For all, $(h,w)\in\\{(P_{t}^{h,w}>T_{l})\leavevmode\nobreak\
\&\leavevmode\nobreak\ (P_{t}^{h,w}<T_{h})\leavevmode\nobreak\
\&\leavevmode\nobreak\ \zeta((h,w),R)=\text{True}\\}$, we set
$m^{h,w}_{t}=\mathbbm{1}$ and $\hat{Y}_{t}^{h,w}=1$. Where $\zeta((h,w),R)$ is
the Boolean function that uses the 8-neighborhood to identify if $(h,w)$ is
connected to any pixel in $\mathcal{R}$.  This process is recursive and
repeated until all pixels in $\mathcal{R}$ are traversed and checked for
connectedness with the existing road labels as well as the newly assigned road
labels.  A similar approach is used for skeleton pseudo-labels selection with
$T_{h}^{\phi}$ and $T_{l}^{\phi}$ as upper and lower thresholds respectively.
This process of pseudo-label selection and connectivity-based refinement is
shown in Figure 3. These connectivity-based refined pseudo-labels are used
during self-supervised domain adaptation.

###  3.4 Structure Conformity Loss

To further guide the adaptation process, we enforce spatial consistency
between thin skeleton prediction and much wider (road) segment predictions by
defining a structural conformity loss (Eq. 5) between the output of two heads.
More specifically, we apply an $L_{2}$-distance loss between road and skeleton
probabilities. The skeleton head tries to predict a single pixel center-line.
During UDA for the source images, we compute the difference between the
prediction of the skeleton and road segmentation, only for the pixels which
are labeled as a skeleton by ground truth. Similarly, in the case of images
from the target domain, we compute the difference between the prediction of
the skeleton and road segmentation, only for the pixels which are pseudo-
labeled as a skeleton  as shown in Figure 2 (red arrow originating from
skeleton pseudo-labels to conformity loss). Combined conformity loss
$\mathcal{L}_{cl}$ is given below.

$\displaystyle\footnotesize\begin{split}\mathcal{L}_{cl}=\sum_{H,W}[Y_{s}^{{\phi}}=1](P_{s}-P_{s}^{{\phi}})^{2}+\sum_{H,W}[\hat{Y}_{t}^{{\phi}}=1](P_{t}-P_{t}^{{\phi}})^{2}.\end{split}$
(5)

Proposed conformity loss enforces that the thinning of the road-surface
results in the same skeleton as the one predicted by the skeleton segmentation
head. An overview of conformity loss for target images is shown in Figure 2.

###  3.5 Adversarial Learning based Feature Alignment

Finally, we employ adversarial learning to align the source and target domain
image features. We define a discriminator network $D$ at the encoder level of
the segmentation network, and try to minimize the gap between source and
target domain features as shown in Figure 2. The discriminator network
($\mathcal{L}_{d}$, Eq. 6) is trained using the cross-entropy loss for source
and target domain images. Similarly, the adversarial loss
($\mathcal{L}_{adv}$, Eq. 7) for the target domain is used to update the
encoder of the segmentation network. Let the features at the output of the
encoder are denoted by $F_{e}\in\mathbb{R}^{h_{f}\times w_{f}\times C_{f}}$,
with $h_{f},w_{f}$ and $C_{f}$ as height, width, and depth of the feature map
respectively, the $\mathcal{L}_{d}$ and $\mathcal{L}_{adv}$ are defined as,

$\displaystyle\begin{split}\mathcal{L}_{d}(F_{e})=-\frac{1}{h_{f}\times
w_{f}}\sum_{h_{f},w_{f}}[y_{d}\leavevmode\nobreak\ \log(D({F_{e}}))\\\
+(1-y_{d})\leavevmode\nobreak\ \log(1-D({F_{e}})))]\end{split}$ (6)
$\displaystyle\mathcal{L}_{adv}(F_{e_{t}})=-\frac{1}{h_{f}\times
w_{f}}\sum_{h_{f},w_{f}}\log(D({F_{e_{t}}}))$ (7)

$F_{e_{t}}$ represents the encoder feature maps of the target images while
$y_{d}$ shows the domain (source, target) label to train the discriminator and
generate an adversarial loss.

Total Loss Function:  During domain adaptation, we simultaneously train the
road and skeleton segmentation head using the labeled source data and the
pseudo-labeled target data. The composite loss function (Eq. 8) is the
summation of the individual losses, Eq. 1, 2, 3, and 4.

$\displaystyle\begin{split}\mathcal{L}_{comp}=\mathcal{L}_{seg}(I_{s},Y_{s})+\mathcal{L}_{s}^{{\phi}}(I_{s},Y_{s}^{{\phi}})\\\
+\hat{\mathcal{L}}_{seg}(I_{t},\hat{Y_{t}})+\hat{\mathcal{L}}_{t}^{{\phi}}(I_{t},\hat{Y}_{t}^{{\phi}})\end{split}$
(8)

Finally, the total loss (Eq. 9) is the summation of composite loss (Eq. 8),
conformity loss (Eq. 5) and adversarial loss (Eq. 7).

$\displaystyle\mathcal{L}_{total}=\mathcal{L}_{comp}+\beta\mathcal{L}_{cl}+\lambda_{adv}\mathcal{L}_{adv}$
(9)

where, hyper-parameters $\beta$, and $\lambda_{adv}$ are chosen empirically
(Sec. 4.1 and 4.4).

Figure 5: Example images showing the visual, infrastructural, and scale
differences in Source and Target domains.

##  4 Experiments

###  4.1 Experimental Setup

Datasets:  For evaluation, we use SpaceNet [5], DeepGlobe [4] and
Massachusetts [67] as benchmark road segmentation datasets. The SpaceNet,
DeepGlobe, and Massachusetts datasets are different with respect to
colorization, geographies, spatial resolution, texture, illuminations, and
road structures. A few sample images from datasets are shown in Fig 5.
SpaceNet dataset consists of annotated satellite imagery for road segmentation
for 4 different cities, i.e., Paris, Las Vegas, Shanghai, and Khartoum. There
are a total of 2780 image tiles with a spatial resolution of 1300$\times$1300
pixels covering 30cm$/$pixel. However, labels are available for a subset of
2548 image tiles. SpaceNet covers around 3000 sq.km of area and around 8000km
annotated roads. DeepGlobe consists of 6226 labeled image tiles of size
1024$\times$1024, at 50 cm$/$pixel. The DeepGlobe dataset is obtained by
capturing satellite imagery over Indonesia, Thailand, and India [4]. For
available labeled images, we have followed the training and validation split
defined by [39]. Massachusetts is an aerial imagery road segmentation dataset
covering 100 cm per-pixel. There are a total of 1171 labeled images of size
1500 $\times$ 1500 with a defined split of 1108 as training, 14 as validation,
and 49 as testing images, respectively.

Model Architecture:  Our proposed method is mainly composed of segmentation
and discriminator networks. The segmentation network (also known as the
generator network) is based on DLinkNet [41] with ResNet-34 [68] as the
backbone feature extractor. We follow the same architecture as the original
DLinkNet [41] for road segmentation to have a fair comparison. A separate
decoder-head is attached to the encoder for the skeleton prediction, as shown
in Figure 2. The discriminator network is inspired by [48] and composed of
four convolution layers each with the filter size 4$\times$4 followed by
LeakyRelu [69] activation except the last layer, where sigmoid is used as the
activation.

Implementation and Training Details:  The proposed domain adaptation approach
is implemented using the PyTorch framework with a single Core-i5 machine with
11GB of GPU memory. From all the images, we randomly crop $1024\times 1024$
image patch for processing, and the batch size is set to two. The initial
learning rates are set to $2e^{-4}$ and $1e^{-4}$ for self-supervised
adaptation and adversarial learning, respectively. The adversarial loss is
scaled with $\lambda_{adv}=0.01$ to reduce the effect of large gradients.
Similarly, the conformity loss is scaled with $\beta=0.1$. The sensitivity
analysis of $\beta$ and $\lambda_{adv}$ are detailed in Sec. 4.4.5. To select
road pseudo-labels and mask $m^{h,w}$, thresholds are set as,
$T_{r}=T_{h}=0.9$, $T_{b}=0.3$ and $T_{l}=1-T_{b}$. Similarly, for skeleton
pseudo-labels and mask $\hat{m}^{h,w}$, the $T_{r}^{\phi}=T_{h}^{\phi}$ and
$T_{b}^{\phi}$ are set to be 0.5 and 0.9 respectively, where
$T_{l}^{\phi}=1-T_{b}^{\phi}$.

The DLinkNet [41] road segmentation network is trained on the source domain.
During adaptation, we follow an iterative process, i.e., generate pseudo-
labels for the whole target dataset (by fixing the segmentation network) and
then train that segmentation model over pseudo-labeled target data alongside
labeled source data. Following the terminology of [70, 51, 71, 72], we define
this iterative process of pseudo-labels generation and model training as
rounds. In the initial round (round-0), the segmentation model (trained on
source data) is further trained for skeleton segmentation alongside road
segmentation using the labeled source data. After round-0, before the start of
the next rounds (round-1 and round-2), pseudo-labels for target images are
generated using the updated model and then used for retraining that model for
two epochs alongside fully labeled source data.

###  4.2 Experimental Results

For all the experiments, SpaceNet is used as the source domain dataset while
DeepGlobe and Massachusetts are used as target domain datasets. Following [16,
63], intersection over union (IoU) & F1-Score are used as evaluation metrics
for road segmentation. To better assess the connectivity and connectivity of
the segmented roads, we follow [2, 3, 73] and also report APLS (Average Path
Length Similarity). For a more fair evaluation, we compare the proposed
approach with generic DA methods ([61, 51, 60, 52, 55, 74] ) as well as road-
specific DA methods including TGN[8], GOAL[8], RoadDA[63] and DOER[75].

Table 1: Comparison between the proposed approach and existing approaches for
SpaceNet to DeepGlobe and SpaceNet to Massachusetts adaptation. Oracle results
(upper-bound) are obtained when the segmentation model is trained over the
target domain images using the ground truth labels [51, 16]. Backbone:
Backbone architecture used in the segmentation network. Adv: Adversarial
adaptation, ST: Self-training based adaptation, Adv+: Adversarial learning
with any other adaptation approach.

|  |  |  SpaceNet $\rightarrow$ DeepGlobe |  SpaceNet $\rightarrow$ Massachusetts
---|---|---|---|---
Method | Backbone | Approach | IoU | F1-Score | APLS | IoU | F1-Score | APLS
Source-only [8] | ResNet-101 | - | 17.7 | 30.1 | 10.3 | - | - | -
RoadDA [63] | Adv | 23.3 | 36.3 | 14.4 | - | - | -
MinEnt [55] | ST | 27.5 | 43.2 | - | - | - | -
MaxSquare [74] | ST | 27.7 | 43.4 | - | - | - | -
TGN [8] | Adv | 27.8 | 43.6 | - | - | - | -
GOAL [8] | Adv+ | 32.1 | 48.6 | - | - | - | -
DOER [75] |  | Adv+ | 32.8 | 49.4 | - | - | - | -
Source-only [33] | ResNet-101 | - | 29.2 | 40.0 | 24.6 | 24.0 | 38.8 | 5.70
CRST-MRKLD [60] | ST | 33.0 | 49.6 | 37.6 | 29.5 | 45.6 | 47.2
Model-Uncertainty[70] | ST | 33.4 | 50.1 | - | 29.8 | 46.0 | -
Source-only [76] | ResNet-38 | - | 37.2 | 54.0 | 29.8 | 26.9 | 42.4 | 12.5
CBST [61] | ST | 39.0 | 56.1 | 39.6 | 28.4 | 44.3 | 49.1
MLSL-SISC [51] | ST | 39.6 | 56.5 | 40.4 | 30.1 | 46.1 | 42.1
Source-only [41] | ResNet-34 | - | 35.2 | 47.4 | 39.2 | 23.0 | 37.4 | 21.0
Ours | ST+Adv | 46.2 | 63.2 | 50.2 | 34.0 | 50.7 | 50.8
Oracle | ResNet-34 | - | 61.2 | 75.0 | 64.9 | 60.0 | 73.5 | 67.9

SpaceNet $\rightarrow$ DeepGlobe: Table 1 presents a comparative analysis of
the proposed road adaptation approach with different source models and
existing state-of-the-art methods. Compared to the Source-only (DLinkNet)
model, the proposed approach improves the IoU, F1-score, and APLS by a margin
of 11%, 15.8%, and 11%, respectively. Compared to TGN[8], GOAL [8], and DOER
[75], recent adversarial learning-based methods for domain adaptation of road
segmentation, the proposed approach outperforms by significant margin of
18.4%, 14.1%, and 13.4% in IoU and 19.6%, 14.6%, and 13.8% in F1-Score,
respectively. Further, compared to another recent approach for road
segmentation adaptation [63], the proposed approach shows an improvement by a
significant margin of 22.9% in IoU, 26.9% in F1-score, and 35.8% in APLS,
respectively.  Similarly, compared to ResNet-38 [76] based self-training
approaches CBST [61] and MLSL-SISC [51], the proposed approach outperforms by
a minimum margin of 6.6%, 6.7%, and 9.8% in IoU, F1-Score, and APLS,
respectively. We also compare our results with DeepLab-v2 [33] based self-
training methods CRST [60] and Model-Uncertainty [70], where the proposed
approach outperforms with a minimum of margins of 12.8% and 13.1% in IoU and
F1-Score, respectively. The considerable improvement in all the evaluation
metrics, APLS, F1-Score, and IoU indicates the improvement of segmented roads
in target images after adaptation. The gain in APLS indicates the segmented
roads after adaptation are more continuous and connected. Compared to
competing methods, the proposed approach results are close to the Oracle
results. The Oracle results, also known as upper-bound, are obtained when the
segmentation model is trained over the target domain images using their
original ground truth labels as defined by [48, 51, 53, 16, 77]. The Oracle
model serves as a benchmark or upper bound for any segmentation model to
compare the performance of the adaptation approach.

|  Target Image |  Ground Truth |  Source-only (DLinkNet [41]) |  CRST [61] |  Ours* (Without CBR) |  Ours
---|---|---|---|---|---|---

Figure 6: SpaceNet $\rightarrow$ DeepGlobe adaptation: Compared to the Source-
only model (DLinkNet[41]) and baseline CRST [60], the proposed approach
produces better segmentation results with comparatively better completeness
and connectivity. “Ours*: Without CBR” and “Ours: With CBR”.

|  Target Image |  Ground Truth |  Source-only (DLinkNet [41]) |  Ours* (Without CBR) |  Ours
---|---|---|---|---|---

Figure 7: From SpaceNet to Massachusetts adaptation. Compared to the source
model (DLinkNet [41]), our approach produces better segmentation results.

SpaceNet $\rightarrow$ Massachusetts: As visible in Figure 5, the
Massachusetts dataset, similar to the SpaceNet dataset, consists of images
from the region where roads, buildings, and other structures appear to be
appropriately planned. The key challenges for adaptation from the SpaceNet to
Massachusetts are the difference in spatial resolution ($100cm^{2}$ compared
to SpaceNet’s $30cm^{2}$ per pixel), visual appearance, and the roads to be
thin when compared to the source domain.

The proposed approach outperforms the competing methods with a minimum margin
of 3.9% in IoU, 4.6% in F1-Score, and 1.6% in APLS (Table 1). More
specifically, we gain 11.0% in IoU, 13.3% in F1-Score, and 29.7% in APLS over
the Source-only model (DLinkNet). Similarly, compared to ResNet-38 [76], CBST
[61] and MLSL-SISC [51], we gain 7.1%, 5.6% and 3.9% in IoU, 8.3%, 6.4% and
4.6% in F1-Score, and 38.3%, 1.7% and 8.7% in APLS, respectively. Compared to
DeepLab-v2 [33], CRTS [60] and Model-Uncertainty [70] by a minimum margin of
4.2%, 4.7% and 3.6% in IoU, F1-Score and APLS, respectively.

###  4.3 Qualitative Results.

Figure 6 & 7 shows the qualitative results. It can be seen that the Source-
only segmentation model (DLinkNet [39]) produces a lot of false negatives
(missing roads or segments) as well as false positives (e.g., predicting
thick/wider roads). This behavior is attributed to the resolution and road-
width differences between the source and target domain images. Specifically,
SpaceNet-3 (source domain dataset), has 0.3m/pixel and wider roads while the
DeepGlobe and Massachusetts (target datasets) have comparatively low
resolution (0.5m/pixel and 1m/pixel, respectively) and thinner roads. The
proposed approach tries to mitigate both these issues by increasing the true
road segmentation and avoiding the non-road areas. Our proposed connectivity-
based pseudo-labels refinement (CBR) further improves the completeness and
connectivity of the road segmentation as shown in Figure 6 & 7, “Ours*:
Without CBR” and “Ours: With CBR”.

###  4.4 Ablation Experiments

To evaluate the effect of different components, we perform domain adaptation
by removing different components (Table 2). Note that for the CL and CBR, the
round-0 is performed where the skeleton head is trained along with fine-tuning
of the segmentation head over the source dataset (Sec. 4.1).

Table 2: Component-wise analysis of our approach, for two sources, one with
skeleton and another without it. SK: Skeleton segmentation, ST: Self-training,
FSA: Feature space adaptation, CL: Conformity loss, CBR: Connectivity-based
pseudo-labels refinement.

| Components | SpaceNet $\rightarrow$ DeepGlobe
---|---|---
Approach | SK | ST | FSA | CL | CBR | IoU | F1-Score | APLS
Source-only | - | - | - | - | - | 35.2 | 47.4 | 35.3
Ours | - | - | ✓ | - | - | 37.8 | 54.9 | -
- | ✓ | - | - | - | 37.4 | 53.4 | 38.4
- | ✓ | ✓ | - | - | 40.3 | 57.4 | 41.8
Source-only | ✓ | - | - | - | - | 36.1 | 53.1 | 39.2
Ours | ✓ | ✓ | - | - | - | 39.7 | 56.8 | 38.1
✓ | - | ✓ | - | - | 39.2 | 56.3 | -
✓ | ✓ | - | ✓ | - | 42.7 | 59.9 | 41.1
✓ | ✓ | ✓ | ✓ | - | 44.2 | 61.3 | 45.4
✓ | ✓ | ✓ | ✓ | ✓ | 46.2 | 63.2 | 50.3

####  4.4.1 Effect of skeleton segmentation and Conformity Loss:

The skeleton (center-line) segmentation helps in preserving the road’s
topology and is a key component of the proposed road-segmentation-adaptation
approach. Due to the multi-task learning, introducing a skeleton segmentation
head improves the accuracy of the road segmentation (source) model from 35.2
IoU to 36.1 IoU, as shown in Table 2 (Source-only Results with and without
skeleton). Similarly, the APLS score is improved from 35.3 to 39.2 indicating
improved connectivity of the segmented roads. Secondly, identifying the
skeleton also allows us to apply the conformity loss between the road and
skeleton heads alongside self-training, resulting in a performance improvement
of 3.0% in IOU, 3.1% in F1-score, and 3.0% in APLS for SpaceNet to DeepGlobe
adaptation. Similarly, we apply CBR over the skeleton head to refine the
skeleton pseudo-labels, making both the self-training and conformity loss more
effective. These additional controls improve our results from 40.3% to 46.2%
in IoU, 57.4% to 63.2% in F1-Score, and from 41.8% to 50.3% in APLS score.
With only a skeleton segmentation head and conformity loss (with self-
training), we see 7.5% points improvement in IoU over the initial source
model.  For details look at Table 2.

####  4.4.2 Connectivity based Pseudo-labels Refinement (CBR)

To understand the effect of CBR, we analyze the quality of pseudo-labels in
terms of IoU, F1-Score, and APLS with ground truth (only used for evaluation
here) with and without CBR (Table 3). We see a considerable improvement in the
quality of pseudo-labels (Figure 4), resulting in a more accurate prediction
of roads in the target domain after adaptation (Table 2). Specifically, the
roads segmented have more connectivity and continuity (Figure 6 & 7), thus
resulting in high APLS.  To further highlight the effect of CBR, we show the
output probability maps and their respective segmentation masks (after
thresholding) without and with CBR in Figure 8. It can be seen that in all the
cases, the probabilities of road prediction improve significantly with the
introduction of CBR. Thus, results in continuous and connected roads.

Table 3: Pseudo-labels quality before and after CBR.

SpaceNet $\rightarrow$ DeepGlobe
---
| Round-1 | Round-2
Process | IoU | F1-Score | APLS | IoU | F1-Score | APLS
Before Refinement | 34.7 | 51.5 | 27.0 | 44.9 | 61.9 | 45.3
After Refinement | 37.0 | 54.0 | 35.7 | 49.3 | 66.0 | 48.2

Figure 8: SpaceNet to DeepGlobe adaptation. The output probabilities and
resulting segmentation are significantly improved after CBR. The roads are
more continuous and connected compared to those without CBR.  Table 4:
Adversarial learning-based adaptation.

SpaceNet $\rightarrow$ DeepGlobe
---
Method | IoU | F1-Score
Source-only (DLinkNet) [41] | 35.2 | 47.4
Output Space Adaptation (OSA) [48] | 37.3 | 54.2
Feature Space Adaptation (FSA) [59] | 37.8 | 54.9

####  4.4.3 Adversarial Learning based Feature Space Adaptation

In general adversarial learning-based domain adaptation is a difficult task
due to high sensitivity and slow convergence. However, many approaches have
been applied to adapt the segmentation models either at structured output
space [55, 56, 57, 58, 48], or latent features space [59, 16]. For road
driving scenes like, GTA [78] and Cityscapes [79] datasets, the output space
adversarial domain is mostly applied because there exists a naturally defined
structure, i.e., the sky always on the top side of the image, roads always on
down rows of the image, etc. However, no such structure is defined in the case
of remote sensing imagery. Secondly, in the case of road segmentation in
satellite imagery, adversarial learning is overwhelmed by large backgrounds
(non-road areas). In this work, we explored both feature space and output
space adversarial learning-based adaptation for satellite imagery.

Table 4 shows that feature space adversarial learning adaptation (FSA)
performs better compared to output space adaptation (OSA), but still, the
improvement due to adversarial learning alone is only comparable to simple
self-training based adaptation. However, when FSA is combined with the self-
training, the road segmentation performance is improved by 2.9% in IoU for
SpaceNet to DeepGlobe adaptation (Table 2 4th row). Hence, performing
adversarial learning along with other components results in better feature
alignment, as visible in the last row of Table 2.

####  4.4.4 Adapting from DeepGlobe to SpaceNet

To further validate the effectiveness of the proposed approach, we performed
experiments in the opposite direction (DeepGlobe (source) to SpaceNet
(target)). Our proposed method improves IoU from 25.6% (before DA) to 29.8%.

####  4.4.5 Sensitivity Analysis of Hyperparameters

In this section, we provide the sensitivity analysis of the hyper-parameters
$\lambda_{adv}$, $\beta$, and pseudo-labels selection thresholds $T_{h}$ and
$T_{l}$, used during our adaptation approach.

Sensitivity of $\beta$ (Conformity loss scaling factor) : We investigate the
effect of conformity loss weight $\beta$ (Eq. 9) on the adaptation performance
of the proposed approach (Table 5). The experiments in Table 5 show that the
conformity loss in general is robust to the scaling factor and improves the
performance even when no scaling factor, i.e., $\beta=1.0$ is used. However,
better performance is reported for $\beta=0.1$.

Table 5: Effect of conformity loss weight $\beta$ on the adaptation process.

SpaceNet $\rightarrow$ DeepGlobe
---
$\beta$ | 0.001 | 0.01 | 0.1 | 1.0
IoU | 43.9 | 45.3 | 46.2 | 44.2
F1-Score | 60.9 | 62.3 | 63.2 | 61.3

Effect of $\lambda_{adv}$ : In Table 6, we show the effect of adversarial loss
scaling factor $\lambda_{adv}$ along with self-supervised domain adaptation.
The adversarial loss helps to improve the adaptation (Table 2), however, is
comparatively more sensitive to loss weight as shown in Table 6. The higher
value of $\lambda_{adv}$ propagates large gradients to the network, while a
very small value of $\lambda_{adv}$ may not help the adaptation process
significantly. A similar observation was reported by [48] as well.

Table 6: Effect of $\lambda_{adv}$, adversarial loss scaling factor in Eq. 9.
The results reported are for adversarial learning combined with self-
supervised learning only.

SpaceNet $\rightarrow$ DeepGlobe
---
$\lambda_{adv}$ | 0.001 | 0.01 | 0.1
IoU | 37.7 | 40.3 | 37.3
F1-Score | 54.1 | 57.4 | 51.3

Effect of $T_{h}$ and $T_{l}$ : We also analyze the effect of upper and lower
thresholds for connectivity-based refinement (CBR) of pseudo-labels. Table 7,
shows the adaptation performance for different combinations of upper and lower
thresholds. It can be seen that reducing the upper threshold to 0.8
deteriorates the overall performance. This is attributed to the false
positives (low precision) selected as pseudo-labels, eventually decreasing the
IoU as well as F1-Score significantly. The lower threshold for CBR is
comparatively less sensitive to overall adaptation performance. However,
reducing $T_{l}$ to 0.5 and less affects the adaptation performance (Table 7).

Table 7: Effect of $T_{h}$ and $T_{l}$ for road pseudo-labels selection.

SpaceNet $\rightarrow$ DeepGlobe
---
$T_{h}$ | $T_{l}$ | IoU | F1-Score
0.9 | 0.7 | 46.2 | 63.2
0.9 | 0.5 | 44.8 | 62.3
0.8 | 0.6 | 43.2 | 60.4

####  4.4.6 Analysis of Backbone Architectures

We have performed additional experiments with our proposed road segmentation
adaptation approach by changing the backbone networks.

Table 8: Comparison of ResNet-34 and large backbone architectures (ResNet-101,
SWIN) used with the DLinkNet segmentation model to evaluate the segmentation
and adaptation performance of the proposed approach. It can be observed that
despite computation limitations, our adaptation strategy improves models’
accuracy considerably over the target domain.

Approach | Backbone | Image Size | IoU | F1-Score | APLS
---|---|---|---|---|---
Source-only | ResNet-34 | 1024 x 1024 | 35.2 | 47.4 | 35.3
Ours | 46.2 | 63.2 | 50.3
Source-only | ResNet-101 | 512 x 512 | 38.0 | 55.1 | 32.6
Ours | 45.94 | 63.0 | 47.9
Source-only | Swin-B | 512 x 512 | 39.0 | 56.1 | 37.7
Ours | 46.7 | 63.7 | 49.2

By using larger backbones, e.g., Swin-B and ResNet-101, the Source Model’s
accuracy improves by 3.8% and 2.8% compared to ResNet-34 based DLinkNet,
respectively. However, these larger backbones bases segmentation models still
suffer from Domain Shift. As indicated in Table 8, our adaptation strategy
improves models’ accuracy considerably over the target domain (7.94% and 7.7%
in IoU and 15.3% and 11.5% in APLS scores, for ResNet-101 and Swin-B based
segmentation models, respectively).

Please note that, due to limited computational resources, we were unable to
train and adapt using the large backbone models (ResNet-101 and Swin-B) with
large image sizes. Therefore we have reduced the size of images to 512 x 512
for ResNet-101 and Swin-B to best utilize our resources, as indicated in Table
8. The results in Table 8 reinforce that regardless of the backbone, road-
segmentation (Source-only) models suffer from the domain shift, and our
adaptation strategy results in performance improvement (ours). We believe that
having large memory GPUs, and increasing the image sizes (as used in the case
of ResNet-34) will further improve the results with ResNet-101 and Swin-B
backbone architectures. Thus, we argue that the proposed approach improves the
performance, is complementary, and can be used with any backbone architecture.

##  5 Conclusions

In this paper, we tackle the challenging problem of overcoming the domain
shift for road segmentation algorithms. The proposed self-supervised domain
adaptation method exploits the topology of the road by having a structural
conformity loss across the skeleton of the road and the road surface itself.
The quality of the pseudo-labels for self-training is improved by introducing
connectivity-based pseudo-labels refinement. To help the adaptation process,
source, and target features are aligned using discriminator-based adversarial
learning. Our thorough experimental results on three different datasets on
multiple evaluation metrics, multiple backbone architectures, detailed
ablation studies, and comparison with several competitive baselines validate
the proposed ideas and framework.

## References

  * [1] B. Herfort, S. Lautenbach, J. P. de Albuquerque, J. Anderson, A. Zipf, The evolution of humanitarian mapping within the openstreetmap community, Scientific reports 11 (1) (2021) 1–15.
  * [2] S. He, F. Bastani, S. Jagwani, M. Alizadeh, H. Balakrishnan, S. Chawla, M. M. Elshrif, S. Madden, M. A. Sadeghi, Sat2graph: road graph extraction through graph-tensor encoding, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIV 16, Springer, 2020, pp. 51–67.
  * [3] Y.-Q. Tan, S.-H. Gao, X.-Y. Li, M.-M. Cheng, B. Ren, Vecroad: Point-based iterative graph exploration for road graphs extraction, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 8910–8918.
  * [4] I. Demir, K. Koperski, D. Lindenbaum, G. Pang, J. Huang, S. Basu, F. Hughes, D. Tuia, R. Raskar, Deepglobe 2018: A challenge to parse the earth through satellite images, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 172–181.
  * [5] A. Van Etten, D. Lindenbaum, T. M. Bacastow, Spacenet: A remote sensing dataset and challenge series, arXiv preprint arXiv:1807.01232 (2018).
  * [6] A. V. Etten, City-scale road extraction from satellite imagery v2: Road speeds and travel times, in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2020, pp. 1786–1795.
  * [7] G. Máttyus, W. Luo, R. Urtasun, Deeproadmapper: Extracting road topology from aerial images, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 3438–3446.
  * [8] X. Lu, Y. Zhong, Z. Zheng, J. Wang, Cross-domain road detection based on global-local adversarial learning framework from very high resolution satellite imagery, ISPRS Journal of Photogrammetry and Remote Sensing 180 (2021) 296–312.
  * [9] S. Wang, X. Mu, H. He, D. Yang, P. Zhao, Feature transfer based adversarial domain adaptation method for cross-domain road extraction, Geocarto International (2020) 1–11.
  * [10] M. N. Subhani, M. Ali, Learning from scale-invariant examples for domain adaptation in semantic segmentation, in: Proceedings of the European Conference on Computer Vision (ECCV), 2020, pp. 1–17.
  * [11] M. Kim, H. Byun, Learning texture invariant representation for domain adaptation of semantic segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12975–12984.
  * [12] F. Pan, I. Shin, F. Rameau, S. Lee, I. S. Kweon, Unsupervised intra-domain adaptation for semantic segmentation through self-supervision, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 3764–3773.
  * [13] B. Zhang, S. Zhao, R. Zhang, Towards adaptive semantic segmentation by progressive feature refinement, in: 2020 IEEE International Conference on Image Processing (ICIP), IEEE, 2020, pp. 2221–2225.
  * [14] Z. Wang, M. Yu, Y. Wei, R. Feris, J. Xiong, W.-m. Hwu, T. S. Huang, H. Shi, Differential treatment for stuff and things: A simple unsupervised domain adaptation method for semantic segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 12635–12644.
  * [15] Y. Zhang, Z. Qiu, T. Yao, D. Liu, T. Mei, Fully convolutional adaptation networks for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6810–6818.
  * [16] J. Iqbal, M. Ali, Weakly-supervised domain adaptation for built-up region segmentation in aerial and satellite imagery, ISPRS Journal of Photogrammetry and Remote Sensing 167 (2020) 263–275.
  * [17] P. Bao, L. Zhang, X. Wu, Canny edge detection enhancement by scale multiplication, IEEE transactions on pattern analysis and machine intelligence 27 (9) (2005) 1485–1490.
  * [18] K. Bittner, S. Cui, P. Reinartz, Building extraction from remote sensing data using fully convolutional networks, International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences-ISPRS Archives 42 (W1) (2017) 481–486.
  * [19] S. Liu, H. Ye, K. Jin, H. Cheng, Ct-unet: Context-transfer-unet for building segmentation in remote sensing images, Neural Processing Letters 53 (2021) 4257–4277.
  * [20] M. Chen, J. Wu, L. Liu, W. Zhao, F. Tian, Q. Shen, B. Zhao, R. Du, Dr-net: An improved network for building extraction from high resolution remote sensing image, Remote Sensing 13 (2) (2021) 294.
  * [21] W. Qiao, L. Shen, J. Wang, X. Yang, Z. Li, A weakly supervised semantic segmentation approach for damaged building extraction from postearthquake high-resolution remote-sensing images, IEEE Geoscience and Remote Sensing Letters 20 (2023) 1–5.
  * [22] M. U. Ali, W. Sultani, M. Ali, Destruction from sky: Weakly supervised approach for destruction detection in satellite imagery, ISPRS Journal of Photogrammetry and Remote Sensing 162 (2020) 115–124.
  * [23] S. Nabiee, M. Harding, J. Hersh, N. Bagherzadeh, Hybrid u-net: Semantic segmentation of high-resolution satellite images to detect war destruction, Machine Learning with Applications 9 (2022) 100381.
  * [24] Z. Lv, H. Huang, L. Gao, J. A. Benediktsson, M. Zhao, C. Shi, Simple multiscale unet for change detection with heterogeneous remote sensing images, IEEE Geoscience and Remote Sensing Letters 19 (2022) 1–5.
  * [25] Z. Lv, F. Wang, G. Cui, J. A. Benediktsson, T. Lei, W. Sun, Spatial–spectral attention network guided with change magnitude image for land cover change detection using remote sensing images, IEEE Transactions on Geoscience and Remote Sensing 60 (2022) 1–12.
  * [26] X. Liu, W. He, H. Zhang, Cross-region plastic greenhouse segmentation and counting using the style transfer and dual-task networks, Computers and Electronics in Agriculture 207 (2023) 107766.
  * [27] A. Shakeel, W. Sultani, M. Ali, Deep built-structure counting in satellite imagery using attention based re-weighting, ISPRS journal of photogrammetry and remote sensing 151 (2019) 313–321.
  * [28] D. Lunga, R. Dhamdhere, S. Walters, L. Bragg, N. Makkar, M. Urban, Learning to count grave sites for cemetery observation models with satellite imagery, IEEE Geoscience and Remote Sensing Letters 19 (2020) 1–5.
  * [29] M. Zakria, H. Rawal, W. Sultani, M. Ali, Cross-region building counting in satellite imagery using counting consistency, arXiv preprint arXiv:2110.13558 (2021).
  * [30] W. Nurkarim, A. W. Wijayanto, Building footprint extraction and counting on very high-resolution satellite imagery using object detection deep learning framework, Earth Science Informatics 16 (1) (2023) 515–532.
  * [31] M. Wurm, T. Stark, X. X. Zhu, M. Weigand, H. Taubenböck, Semantic segmentation of slums in satellite images using transfer learning on fully convolutional neural networks, ISPRS journal of photogrammetry and remote sensing 150 (2019) 59–69.
  * [32] M. F. U. Rehman, I. Aftab, W. Sultani, M. Ali, Mapping temporary slums from satellite imagery using a semi-supervised approach, IEEE Geoscience and Remote Sensing Letters 19 (2022) 1–5.
  * [33] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille, Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs, IEEE transactions on pattern analysis and machine intelligence 40 (4) (2018) 834–848.
  * [34] V. Badrinarayanan, A. Kendall, R. Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE transactions on pattern analysis and machine intelligence 39 (12) (2017) 2481–2495.
  * [35] L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille, Semantic image segmentation with deep convolutional nets and fully connected crfs, in: 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 2015.
  * [36] H. Noh, S. Hong, B. Han, Learning deconvolution network for semantic segmentation, in: Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 1520–1528.
  * [37] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: International Conference on Medical image computing and computer-assisted intervention, Springer, 2015, pp. 234–241.
  * [38] H. Zhao, J. Shi, X. Qi, X. Wang, J. Jia, Pyramid scene parsing network, in: IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 2881–2890.
  * [39] A. Batra, S. Singh, G. Pang, S. Basu, C. Jawahar, M. Paluri, Improved road connectivity by joint learning of orientation and segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 10385–10393.
  * [40] J. Mei, R.-J. Li, W. Gao, M.-M. Cheng, Coanet: Connectivity attention network for road extraction from satellite imagery, IEEE Transactions on Image Processing 30 (2021) 8540–8552.
  * [41] L. Zhou, C. Zhang, M. Wu, D-linknet: Linknet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018, pp. 182–186.
  * [42] A. Chaurasia, E. Culurciello, Linknet: Exploiting encoder representations for efficient semantic segmentation, in: 2017 IEEE Visual Communications and Image Processing (VCIP), IEEE, 2017, pp. 1–4.
  * [43] Y. Wei, K. Zhang, S. Ji, Simultaneous road surface and centerline extraction from large-scale remote sensing images using cnn-based segmentation and tracing, IEEE Transactions on Geoscience and Remote Sensing 58 (12) (2020) 8919–8931.
  * [44] T. Alshaikhli, W. Liu, Y. Maruyama, Simultaneous extraction of road and centerline from aerial images using a deep convolutional neural network, ISPRS International Journal of Geo-Information 10 (3) (2021) 147.
  * [45] G. Cheng, Y. Wang, S. Xu, H. Wang, S. Xiang, C. Pan, Automatic road detection and centerline extraction via cascaded end-to-end convolutional neural network, IEEE Transactions on Geoscience and Remote Sensing 55 (6) (2017) 3322–3337.
  * [46] E. Tzeng, J. Hoffman, K. Saenko, T. Darrell, Adversarial discriminative domain adaptation (2017).  arXiv:1702.05464.
  * [47] G. Mattyus, R. Urtasun, Matching adversarial networks, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 8024–8032.  doi:10.1109/CVPR.2018.00837.
  * [48] Y.-H. Tsai, W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang, M. Chandraker, Learning to adapt structured output space for semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [49] B. Benjdira, Y. Bazi, A. Koubaa, K. Ouni, Unsupervised domain adaptation using generative adversarial networks for semantic segmentation of aerial images, Remote Sensing 11 (11) (2019) 1369.  doi:10.3390/rs11111369.
URL http://dx.doi.org/10.3390/rs11111369

  * [50] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, V. Lempitsky, Domain-adversarial training of neural networks (2016).  arXiv:1505.07818.
  * [51] J. Iqbal, M. Ali, Mlsl: Multi-level self-supervised learning for domain adaptation with spatially independent and semantically consistent labeling, in: The IEEE Winter Conference on Applications of Computer Vision, 2020, pp. 1864–1873.
  * [52] Y. Wang, J. Peng, Z. Zhang, Uncertainty-aware pseudo label refinery for domain adaptive semantic segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 9092–9101.
  * [53] J. Hoffman, E. Tzeng, T. Park, J. Zhu, P. Isola, K. Saenko, A. A. Efros, T. Darrell, Cycada: Cycle-consistent adversarial domain adaptation, in: Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, 2018, pp. 1994–2003.
  * [54] J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440.
  * [55] T.-H. Vu, H. Jain, M. Bucher, M. Cord, P. Pérez, Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 2517–2526.
  * [56] Y. Luo, L. Zheng, T. Guan, J. Yu, Y. Yang, Taking a closer look at domain shift: Category-level adversaries for semantics consistent domain adaptation, in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019\.
  * [57] W.-L. Chang, H.-P. Wang, W.-H. Peng, W.-C. Chiu, All about structure: Adapting structural information across domains for boosting semantic segmentation, in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [58] R. Gong, W. Li, Y. Chen, L. V. Gool, Dlow: Domain flow for adaptation and generalization, in: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [59] Y.-H. Chen, W.-Y. Chen, Y.-T. Chen, B.-C. Tsai, Y.-C. Frank Wang, M. Sun, No more discrimination: Cross city adaptation of road scene segmenters, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1992–2001.
  * [60] Y. Zou, Z. Yu, X. Liu, B. Kumar, J. Wang, Confidence regularized self-training, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 5982–5991.
  * [61] Y. Zou, Z. Yu, B. Vijaya Kumar, J. Wang, Unsupervised domain adaptation for semantic segmentation via class-balanced self-training, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 289–305.
  * [62] J. Iqbal, H. Rawal, R. Hafiz, Y.-T. Chi, M. Ali, Distribution regularized self-supervised learning for domain adaptation of semantic segmentation, Image and Vision Computing 124 (2022) 104504.
  * [63] L. Zhang, M. Lan, J. Zhang, D. Tao, Stagewise unsupervised domain adaptation with adversarial self-training for road segmentation of remote-sensing images, IEEE Transactions on Geoscience and Remote Sensing (2021).
  * [64] P. Shamsolmoali, M. Zareapoor, H. Zhou, R. Wang, J. Yang, Road segmentation for remote sensing images using adversarial spatial pyramid networks, IEEE Transactions on Geoscience and Remote Sensing 59 (6) (2020) 4673–4688.
  * [65] X. Deng, H. L. Yang, N. Makkar, D. Lunga, Large scale unsupervised domain adaptation of segmentation networks with adversarial learning, in: IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Symposium, IEEE, 2019, pp. 4955–4958.
  * [66] T. Zhang, A fast parallel algorithm for thinning digital patterns, Commun. ACM 27 (3) (1997) 337–343.
  * [67] V. Mnih, Machine learning for aerial image labeling, University of Toronto (Canada), 2013.
  * [68] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [69] J. Xu, Z. Li, B. Du, M. Zhang, J. Liu, Reluplex made more practical: Leaky relu, in: 2020 IEEE Symposium on Computers and Communications (ISCC), IEEE, 2020, pp. 1–7.
  * [70] F. Fleuret, et al., Uncertainty reduction for model adaptation in semantic segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 9613–9623.
  * [71] M. A. Munir, M. H. Khan, M. Sarfraz, M. Ali, Ssal: Synergizing between self-training and adversarial learning for domain adaptive object detection, Advances in Neural Information Processing Systems 34 (2021).
  * [72] J. Iqbal, R. Hafiz, M. Ali, Fogadapt: Self-supervised domain adaptation for semantic segmentation of foggy images, Neurocomputing 501 (2022) 844–856.
  * [73] F. Bastani, S. He, S. Abbar, M. Alizadeh, H. Balakrishnan, S. Chawla, S. Madden, D. DeWitt, Roadtracer: Automatic extraction of road networks from aerial images, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 4720–4728.
  * [74] M. Chen, H. Xue, D. Cai, Domain adaptation for semantic segmentation with maximum squares loss, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 2090–2099.
  * [75] X. Lu, Y. Zhong, L. Zhang, Open-source data-driven cross-domain road detection from very high resolution remote sensing imagery, IEEE Transactions on Image Processing 31 (2022) 6847–6862.  doi:10.1109/TIP.2022.3216481.
  * [76] Z. Wu, C. Shen, A. Van Den Hengel, Wider or deeper: Revisiting the resnet model for visual recognition, Pattern Recognition 90 (2019) 119–133.
  * [77] Y.-H. Tsai, W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang, M. Chandraker, Learning to adapt structured output space for semantic segmentation (2020).  arXiv:1802.10349.
  * [78] S. R. Richter, V. Vineet, S. Roth, V. Koltun, Playing for data: Ground truth from computer games, in: B. Leibe, J. Matas, N. Sebe, M. Welling (Eds.), European Conference on Computer Vision (ECCV), Vol. 9906 of LNCS, Springer International Publishing, 2016, pp. 102–118.
  * [79] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele, The cityscapes dataset for semantic urban scene understanding, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3213–3223.