# SoftMatch Distance: A Novel Distance for Weakly-Supervised Trend Change
Detection in Bi-Temporal Images

Yuqun Yang, Xu Tang, Xiangrong Zhang, Jingjing Ma, Licheng Jiao

###### Abstract

General change detection (GCD) and semantic change detection (SCD) are common
methods for identifying changes and distinguishing object categories involved
in those changes, respectively. However, the binary changes provided by GCD is
often not practical enough, while annotating semantic labels for training SCD
models is very expensive. Therefore, there is a novel solution that
intuitively dividing changes into three trends (“appear”, “disappear” and
“transform”) instead of semantic categories, named it trend change detection
(TCD) in this paper. It offers more detailed change information than GCD,
while requiring less manual annotation cost than SCD. However, there are
limited public data sets with specific trend labels to support TCD
application. To address this issue, we propose a softmatch distance which is
used to construct a weakly-supervised TCD branch in a simple GCD model, using
GCD labels instead of TCD label for training. Furthermore, a strategic
approach is presented to successfully explore and extract background
information, which is crucial for the weakly-supervised TCD task. The
experiment results on four public data sets are highly encouraging, which
demonstrates the effectiveness of our proposed model.

##  1 Introduction

As a hot task in computer vision, change detection aims to identify the
foreground (i.e., objects of interest) changes between bi-temporal images
captured at the same locations [23]. It has many real-world applications, such
as land cover monitoring [3, 24], visual surveillance [45], medical diagnosis
[15, 31], autonomous driving [7, 51], etc. However, there are many
difficulties need to be overcome. For example, affected by external conditions
(e.g., illumination, sensor, and shooting angle), the visual appearance of the
same/different foreground objects may be dissimilar/similar in bi-temporal
images. At first, traditional machine learning techniques are popular for
image change detection [37, 35, 13]. However, the low-level visual features
limit their performance. Recently, assisted by the deep learning [27, 44],
change detection technologies are promoted significantly. Many deep-based
methods [29, 16, 38] have been proposed and perform well in their
applications.

Figure 1: Schematic illustration of main challenge. Based on the training of
change label, most existing models conduct the peer prediction and output
simple change map. However, how to realize the upgrade prediction for
performing trend change detection with zero-cost for trend label annotation is
a challenging problem, which also can be viewed as a weakly-supervised manner
that using change labels instead of trend labels for training. We can find
that the change map shows the changed car (marked in yellow). In contrast, the
trend map further provides three trends: “Appear” (blue), “Disappear” (white)
and “Transform” (red), offering a more detailed view of car changes.

Generally, the existing deep-based models can be divided into two groups:
general change detection (GCD) [38, 37, 25] and semantic change detection
(SCD) [36, 50, 48]. GCD models locate changes in foreground objects between
bi-temporal images, while SCD models further recognize the semantic categories
involved in those changes. However, both of them have respective limitations:
1) the outputs of GCD models, which only provide basic information on change
or not, may be insufficient for practical use; and 2) SCD models require
extensive manual labels for various semantics at pixel-level, which is very
expensive. Although some unsupervised and weakly supervised SCD methods [17,
4, 1] have been proposed to alleviate this dilemma, the predicted results are
always not satisfactory due to the absence of semantic labels. As a result,
how to enhance the practicality of change detection and reduce the demands of
manual labels has become an urgent and necessary problem. To address this
issue, some methods [28, 39] attempt to further divide changes into more
intuitive and simple three trends, including “Appear”, “Disappear” and
“Transform” (see Fig. 1) instead of relying on complex semantic categories.
These methods enrich the GCD results without massive semantic labels, and the
obtained results could reflect the variation trend. Thus, we name them trend
change detection (TCD) models in this paper. TCD is a relatively new concept
that offers significant advantages for change detection in real-world
applications, but lacks sufficient datasets with specific trend labels for
support.

In this paper, we summarize a main challenge for this problem as illustrated
in Fig. 1: how to realize the upgrade prediction to perform TCD task with
zero-cost for annotating trend labels. Our solution is to propose a softmatch
distance that is used to construct a weakly-supervised TCD branch in a simple
GCD model. By utilizing only change labels of GCD, softmatch distance can
equip features with the fore-/back-ground directivity (explained in Section
3.2) for predicting trends. As a pluggable module, the softmatch distance has
the potential to enable many existing GCD models for performing the weakly-
supervised TCD task, significantly improving their practicalities.

Three main contributions can be summarized as follows:

  * •

To our knowledge, this is the first successful attempt in weakly-supervised
TCD model, which is upgraded from a GCD model using softmatch distance.
Moreover, we develop a strategic approach to accurately identify the important
background information within bi-temporal images.

  * •

We propose a softmatch distance that not only equips features with the
fore-/back-ground directivity for the weakly-supervised TCD task, but also
outperforms other common distances (i.e., cosine and Euclidean distances) in
GCD task.

  * •

The extensive experiments are conducted on four public data sets to prove the
effectiveness of the proposed method in performing GCD and weakly-supervised
TCD tasks.

##  2 Related Work

###  2.1 General Change Detection

As a fundamental task, GCD plays a vital role in many applications [23], and
it always attracts researchers’ attention. In [33], a kernelized fuzzy modal
variation method is proposed to use background subtraction to map video pixels
into a high-dimensional space, allowing objects of interest to be separated
from the backgrounds for obtaining change information. With the development of
deep learning, many deep-based methods are developed based on the U-shape
encoder-decoder networks [42, 25, 26, 2]. In [19], a hierarchical paired
channel fusion network (HPCFNet) was introduced for street scene change
detection. Incorporating channel fusion and multi-part feature learning
between encoder and decoders improves its detection capacity. Another U-shape
model for street scene change detection, named dynamic receptive temporal
attention network (DRTAM), was proposed [8], which uses the temporal module to
find an optimal dependency scope size for GCD dynamically. Furthermore, Park
et al. [29] developed SimSaC to enhance GCD model robustness in real-world
settings. SimSaC carries out scene flow estimation and change detection
simultaneously, detecting changes with imperfect matches.

Apart from the natural scenario, GCD is also a prevalent and significant task
in the remote sensing (RS) community [22, 9, 21, 12]. In literature [10],
three U-shape models based on the fully convolutional network (FCN) were
developed for RS GCD tasks, which adopt different feature fusion schemes to
capture the multi-scale information hidden in RS images, guaranteeing the
quality of GCD results. Tang et al. [34] presented a GCD network for RS images
using a Siamese FCN and graph convolutional network (GCN) to capture local
information and long-distance pixel relationships, respectively. Then, change
information is determined by a metric-learning algorithm. Considering the high
costs of labeling bi-temporal RS images, a single-temporal supervised learning
method was proposed in [48]. It allows researchers to generate a high-
performance change detection model only using unpaired label images.

Figure 2: Framework of the proposed method. An encoder-decoder is designed to
extract two features groups: $\mathbf{F}_{\mathbf{C}}$ and
$\mathbf{F}_{\mathbf{I}}$, which are applied to conduct the peer/upgrade
prediction for GCD/TCD task with the output of change/trend map, respectively.
In the GCD branch, the softmatch distance is used as distance function to
measure the feature difference of $\mathbf{F}_{\mathbf{C}}$ for generating
change map. In the weakly-supervised TCD branch, the softmatch distance can
improve the fore-/back-ground directivity of $\mathbf{F}_{\mathbf{I}}$ under
the supervision of change label. Meanwhile, the supervision of opposite change
label is applied to explore the background information within bi-temporal
images. Finally, the trend map is generated by the trend prediction module.

###  2.2 Semantic Change Detection

The key to SCD is recognizing the category information of the changed objects.
Semantic segmentation models can intrinsically achieve this by comparing the
semantic labels of pixels.[4]. For example, inspired by human observation
manner, Li et al. [20] proposed a hierarchical model for scene semantic
segmentation that decomposes complex scenes into simpler parts and builds
structured relations between them for obtaining accurate results. Considering
the importance of global semantic relations, a cross-image relational
knowledge distillation strategy was developed in [43], which transfers
structured relations between pixel and region for favorable segmentation
results.

While semantic segmentation can be an emergency plan for SCD, it fails to
consider the temporal relationships between multi-temporal images.
Furthermore, identifying all pixels’ semantic categories for SCD tasks is not
always necessary. This is because the changed parts usually only occupy a
fraction of the total image. To overcome the above limitations, many SCD-
oriented models have been developed based on the general-purpose semantic
segmentation models. Ru et al. [30] presented a CorrFusion module for SCD.
Based on regular convolutional neural network (CNN) and mapping techniques,
CorrFusion can first get the bi-temporal images’ representations and their
instance-level correlation. Then, a cross-temporal fusion approach is used to
explore the temporal relations and generate positive SCD results. A multi-task
learning framework was introduced in [11] to deal with SCD. This framework
consists of two temporal branches and a change branch. By enhancing their
communications, the contributions of triple branches can be maximized. Peri
Akiva et al. [1] proposed a material- and texture-based self-supervision
method and applied it to SCD tasks. The reported results confirm its
usefulness. The prerequisite of the above methods is that the labels are
correct. However, noisy labels are inevitable in practice. To mitigate the
negative influence caused by label noises, a new method was presented in [4].
On the one hand, an iterative learning model is developed to filter the noisy
labels. On the other hand, a guided anisotropic diffusion algorithm is
proposed to ensure the SCD results.

##  3 Methodology

The framework of the proposed method is shown in Fig. 2, which can be divided
into three parts, i.e., feature learning, GCD branch and the weakly-supervised
TCD branch. The feature learning part aims to fuse the valuable information
extracted from the bi-temporal images to the paired common features
$\mathbf{F}_{\mathbf{C}}$ and the paired independent features
$\mathbf{F}_{\mathbf{I}}$. Under the supervision of change label, the GCD/TCD
branch aim to generate change/trend maps using common/independent features
$\mathbf{F}_{\mathbf{C}}$/$\mathbf{F}_{\mathbf{I}}$. Here, the cross-entropy
function is used for all the supervisions.

Figure 3: Diagram of extracting features. It consists of two independent
encoders (marked in green) and decoders (marked in blue) with shared
parameters, and a common decoder (marked in brown).

###  3.1 Feature Extraction

A U-shape encoder-decoder network is constructed to obtain the representative
and discriminative visual features from the bi-temporal images (T1 and T2),
whose structure is exhibited in Fig. 3. Besides two same encoder-decoder nets
(the left and right branches), which focus on capturing the profitable visual
clues from input images, another decoder (the middle branch) is developed to
extra explore temporal relationships between Image T1 and Image T2.

Let us start with the explanation of two encoder-decoder nets. Since they have
the same structure, we explain the left one (in Fig. 3) for clarity. The
encoder contains five $Convs$ layers and four max-pooling layers with the two-
times down-sampling. Note that a $Convs$ layer involves two groups of the
convolution ($3\times 3$), BatchNorm, and ReLU. The decoder has four $Up$
layers and four $Convs$ layers. Here, a $Up$ layer incorporates an up-
sampling, a convolution ($3\times 3$), a BatchNorm, and a ReLU. When Image T1
is input into the encoder, five multi-layer features $\mathbf{e}_{1}^{1}$,
$\mathbf{e}_{2}^{1}$, $\mathbf{e}_{3}^{1}$, $\mathbf{e}_{4}^{1}$, and
$\mathbf{e}_{5}^{1}$ can be produced. Then, $\mathbf{e}_{5}^{1}$ would be fed
into the decoder to generate $\mathbf{d}_{5}^{1}$, $\mathbf{d}_{4}^{1}$,
$\mathbf{d}_{3}^{1}$, and $\mathbf{d}_{2}^{1}$. Here, common skip connections
are added between the encoder and decoder to integrate the various knowledge
within the multi-layer features and reduce information loss. Thus, the
features of the decoder are output by,

$\displaystyle\mathbf{d}_{i}^{1}=\left\\{\begin{matrix}\operatorname{Convs}\left(\operatorname{Up}\left(\mathbf{e}_{i}^{1}\right)\oplus\mathbf{e}_{i-1}^{1}\right),i=5\\\
\operatorname{Convs}\left(\operatorname{Up}\left(\mathbf{d}_{i+1}^{1}\right)\oplus\mathbf{e}_{i-1}^{1}\right),i=4,3,2\\\
\end{matrix}\right.,$ (1)

where $\operatorname{Convs}\left(\cdot\right)$,
$\operatorname{Up}\left(\cdot\right)$, and $\oplus$ indicate the nonlinear
functions of the $Convs$ layer and $Up$ layer, and concatenation,
respectively. Note that the spatial size of $\mathbf{d}_{2}^{1}$ is the same
as that of Image T1. Similarly, after passing through the encoder-decoder net,
Image T2 can be transformed into
$\left\\{\mathbf{e}_{1}^{2},\mathbf{e}_{2}^{2},\mathbf{e}_{3}^{2},\mathbf{e}_{4}^{2},\mathbf{e}_{5}^{2}\right\\}$
and
$\left\\{\mathbf{d}_{5}^{2},\mathbf{d}_{4}^{2},\mathbf{d}_{3}^{2},\mathbf{d}_{2}^{2}\right\\}$,
where the spatial sizes of ${{\bf{d}}_{2}^{2}}$ and Image T2 are the same to
each other. Based on the obtained features, we can get the paired independent
features
${{\mathbf{F}}_{\mathbf{I}}}\text{=}\left\\{\mathbf{F}_{\mathbf{I}}^{1},\mathbf{F}_{\mathbf{I}}^{2}\right\\}$
by,

$\displaystyle{{\mathbf{F}}_{\mathbf{I}}}=\left\\{\operatorname{conv}_{1\times
1}\left(\mathbf{d}_{2}^{1}\right),\operatorname{conv}_{1\times
1}\left(\mathbf{d}_{2}^{2}\right)\right\\},$ (2)

where the output channel of $\operatorname{conv}_{1\times
1}\left(\cdot\right)$ is $c$.

As mentioned above, there is another decoder (see Fig. 3, the middle one) for
modeling the temporal relationship. To this end, the multi-layer features
$\mathbf{e}_{i}^{1}$ and $\mathbf{e}_{i}^{2}$ of Image T1 and Image T2 will be
fused in this decoder. Firstly, $\mathbf{e}_{i}^{1}$ and $\mathbf{e}_{i}^{2}$
are concatenated in channel dimension to generate $\mathbf{e}_{i}^{c}$. Then,
the same fusion scheme with Equation 1 is adopted to fuse $\mathbf{e}_{i}^{c}$
and output $\mathbf{d}_{2}^{c}$. Finally, we can get the paired common
features
${{\mathbf{F}}_{\mathbf{C}}}=\left\\{\mathbf{F}_{\mathbf{C}}^{1},\mathbf{F}_{\mathbf{C}}^{2}\right\\}$
by,

$\displaystyle{{\mathbf{F}}_{\mathbf{C}}}=\left\\{{{\mathop{\rm conv}_{1\times
1}\nolimits}\left({{\bf{d}}_{2}^{1}\oplus{\bf{d}}_{2}^{c}}\right),{\mathop{\rm
conv}_{1\times
1}\nolimits}\left({{\bf{d}}_{2}^{2}\oplus{\bf{d}}_{2}^{c}}\right)}\right\\},$
(3)

where the output channel of $\operatorname{conv}_{1\times
1}\left(\cdot\right)$ also equals $c$. Note that we set $c=3$, and Section 3.3
will explain the reasons.

###  3.2 SoftMatch Distance

Usually, after obtaining the paired bi-temporal features
$\left\\{\mathbf{F}_{\centerdot}^{1},\mathbf{F}_{\centerdot}^{2}\right\\}$, we
can adopt some common distance metrics (e.g., Cosine distance [40]) to measure
their difference for generating change map, which is effective in our GCD
branch. However, due to the specific goals of our weakly-supervised TCD
branch, in addition to the capacity of measuring feature difference, we also
require the distance metric can equip fore-/back-ground directivity to the
involved features based on the supervision of change label. In other words, we
expect that the final
$\left\\{\mathbf{F}_{\centerdot}^{1},\mathbf{F}_{\centerdot}^{2}\right\\}$
(after training) can provide specific information to distinguish different
foregrounds or the foreground and background between bi-temporal images, in
order to predict trend map. In addition, we hope the values of
changed/unchanged pixels in the generated distance maps (DMs) can be pushed
toward 1/0 for aligning with the change label when computing loss.

Figure 4: Diagram of computing softmatch distance between two feature maps.
Here, all of the “$\times$”, “$+$”, and “$-$” operations are carried out at
the pixel-level.

Summing up the above factors that are not achievable with existing common
distances, we develop a simple yet effective softmatch distance. Here,
${{\mathbf{F}}_{\mathbf{I}}}=\left\\{\mathbf{F}_{\mathbf{I}}^{1},\mathbf{F}_{\mathbf{I}}^{2}\right\\}$
is used as an example to illustrate the proposed softmatch distance, while the
computation for
${{\mathbf{F}}_{\mathbf{C}}}=\left\\{\mathbf{F}_{\mathbf{C}}^{1},\mathbf{F}_{\mathbf{C}}^{2}\right\\}$
is the same. To calculate the softmatch distance of feature pixels
$\mathbf{p}=\\{\mathbf{p}^{1},\mathbf{p}^{2}\\},\mathbf{p}^{i}\in\mathbb{R}^{c}$
within ${{\mathbf{F}}_{\mathbf{I}}}$, the normalization of subtracting the
maximum value and the softmax operation are applied first to obtain
$\mathbf{s}=\left\\{{{\mathbf{s}}^{1}},{{\mathbf{s}}^{2}}\right\\},{{\mathbf{s}}^{i}}\in{{\mathbb{R}}^{c}}$.
Then, subtracting $1$ from the inner product of ${{\mathbf{s}}^{1}}$ and
${{\mathbf{s}}^{2}}$ is conducted, i.e.,

$\displaystyle\begin{matrix}sdist\left({{\mathbf{p}}^{1}},{{\mathbf{p}}^{2}}\right)=1-\left\langle\mathbf{s}_{i}^{1},\mathbf{s}_{i}^{2}\right\rangle\\\
=1-\sum\limits_{i=0}^{c}{\left(\frac{\exp\left(p_{i}^{1}\right)}{\sum\limits_{j=1}^{c}{\exp\left(p_{j}^{1}\right)}}\times\frac{\exp\left(p_{i}^{2}\right)}{\sum\limits_{j=1}^{c}{\exp\left(p_{j}^{2}\right)}}\right)}.\\\
\end{matrix}$ (4)

Here, the Proof that the value range of
$sdist\left({{\mathbf{p}}^{1}},{{\mathbf{p}}^{2}}\right)$ is $(0,1)$ is
provided in supplementary materials for clearly measuring feature differences.
When all pixels’ softmatch distances are calculated, the DM of
$\left\\{\mathbf{F}_{\mathbf{I}}^{1},\mathbf{F}_{\mathbf{I}}^{2}\right\\}$ can
be obtained. In the same way, the DM of
$\left\\{\mathbf{F}_{\mathbf{C}}^{1},\mathbf{F}_{\mathbf{C}}^{2}\right\\}$ can
be calculated. The diagram of calculating the softmatch distance between two
features is shown in Fig. 4.

After obtaining DMs, the model parameters can be optimized to perform their
respective tasks by computing the loss between DMs and change labels. Then, in
the GCD branch, it is intuitive that the DM output by the trained model is
discretized to be the predicted change map. Now, let’s delve into how the
fore-/back-ground directivity of
$\mathbf{F}_{\mathbf{I}}=\left\\{\mathbf{F}_{\mathbf{I}}^{1},\mathbf{F}_{\mathbf{I}}^{2}\right\\}$
is obtained in the weakly-supervised TCD branch.

Figure 5: Example of highlighting one channel of $\mathbf{s}^{1}$ and
$\mathbf{s}^{2}$. Here, the values of highlighted/unhighlighted channel (mark
in red/white) are close to 1/0 due to using softmax operation.

For clarity, we will use a paired pixels
$\mathbf{p}=\\{\mathbf{p}^{1},\mathbf{p}^{2}\\}$ within
$\mathbf{F}_{\mathbf{I}}$ as an example to illustrate this point. In fact, the
fore-/back-ground directivity is reflected in that each channel of
$\mathbf{p}^{1}$ and $\mathbf{p}^{2}$ is associated with a particular
foreground or background. In this way, once a pixel channel is highlighted, it
becomes clear which foreground or background it belongs to, enabling the trend
map generation. To do this, we first leverage
$\mathbf{s}=\\{\mathbf{s}^{1},\mathbf{s}^{2}\\}$ output by softmax operation
instead of $\mathbf{p}=\\{\mathbf{p}^{1},\mathbf{p}^{2}\\}$ to ensure two
points: 1) at most one channel can be highlighted; and 2) the
highlighted/unhighlighted values are constrained to be no greater/less than
1/0. Although we have defined two limitations for normalizing the feature
values, it is still unclear how to push the highlighted/unhighlighted values
toward 1/0 for certainly directivity. Fortunately, applying two operations,
inner product and subtraction of $1$, to
$\mathbf{s}=\\{\mathbf{s}^{1},\mathbf{s}^{2}\\}$ can achieve this goal. As
shown in Fig. 5, for the scenarios of changed/unchanged, model has to
highlight different/same channels of $\mathbf{s}^{1}$ and $\mathbf{s}^{2}$ and
push their values toward 1 for generating the result value of 1/0.

After training, an anonymous association plan between feature channels and
foreground/background is created by $\mathbf{s}^{1}$ and $\mathbf{s}^{2}$,
with one channel highlighted to indicate its membership. Therefore, the
fore-/back-ground directivity of $\mathbf{F}_{\mathbf{I}}$ is obtained
successfully during training. However, the current functions are insufficient
to preform TCD task due to a key problem: the anonymous association plan does
not clearly indicate which channel corresponds to the foreground or the
background. Its solution to expose this plan will be provided in Section 3.3.

Finally, let’s look at three possible concerns due to applying similar manner
to conduct softmatch distance in both TCD and GCD branches. Q1) Is the
fore-/back-ground directivity of ${{\mathbf{F}}_{\mathbf{C}}}$ also obtained
in the GCD branch? A1) Yes, but the common information within
$\mathbf{F}_{\mathbf{C}}$ interferes with its ability to distinguish
foreground and background for predicting trend. Nonetheless, it helps
softmatch distance achieve more accurate measurements than common distances,
thereby improving performance of GCD. Q2) Why isn’t the DM of
$\mathbf{F}_{\mathbf{I}}$ in the TCD branch used to generate change maps? A2)
This is unnecessary since the predicted trend map already includes both change
and trend information. Q3) Why not directly use the trend map of TCD branch to
generate the change map, instead of saving the GCD branch? A3) The trend map
can reflect changes, but since its supporting features
$\mathbf{F}_{\mathbf{I}}$ lack common temporal information, the change map
generated by it may be slightly inferior to that generated by
$\mathbf{F}_{\mathbf{C}}$.

###  3.3 Trend Prediction

As mentioned earlier, softmatch distance alone only establish the unclear
anonymous association plan, which is not enough for weakly-supervised TCD.
Therefore, to expose it, we need to further analyze how to definitely assign
$c$ channels of $\mathbf{F}_{\mathbf{I}}$ to foregrounds and the background,
in order to meet the trend prediction requirements.

Again, we define three trends “appear”, “disappear”, and “transform”, i.e.,

  * •

“Appear”: from background to foreground,

  * •

“Disappear”: from foreground to background,

  * •

“Transform”: from foreground A to foreground B.

Where the foreground tags A and B are not specific object categories, but
instead are used to differentiate between two different foregrounds. By
analyzing of above definitions, we can find that identifying background is
paramount in TCD. Thus, only one of $c$ channels of
$\mathbf{F}_{\mathbf{I}}^{1}$ will be selected to learn the background, and we
name it the background channel. The rest $c-1$ channels will be regarded as
the foreground channels, which focus on discovering the objects of interest.
Here, $c-1$ is set to the minimal value $2$ of satisfying “Transformation”,
i.e., $c=3$. The same actions would be carried out to
$\mathbf{F}_{\mathbf{I}}^{2}$. Thus, the three channels of
$\mathbf{F}_{\mathbf{I}}^{1}$ and $\mathbf{F}_{\mathbf{I}}^{2}$ correspond to
foreground A, foreground B, and background. Here, only the background channel
needs to be associated fixedly with one channel, while the foreground channels
can be associated adaptively as they only indicate the difference between
foregrounds. Therefore, fixing the background channel allows us to confirm the
exact channel association plan for performing TCD task.

To fix background channel of $\mathbf{F}_{\mathbf{I}}$, because most
background areas would be unchanged, an opposite change label
(change/unchanged pixels are annotated as 0/1) is constructed to train one
self-determined channel of $\mathbf{F}_{\mathbf{I}}$ using the cross-entropy
loss function. Particularly, a strategic approach for accurately identifying
background needs to be emphasized. The integrated cross-entropy loss will push
the probability value of positive (unchanged) and negative (changed) samples
toward 1 and 0 for representing background and non-background, respectively.
However, this is not appropriate for negative samples, since the background
frequently occurs in the changed parts of “Appear” and “Disappear”. Thus, we
only count the positive samples’ loss for training model. Here, no action is
required for the foreground channels.

Figure 6: Diagram of trend prediction in the inference stage.

As shown in Fig. 6, when $\mathbf{F}_{\mathbf{I}}^{1}$ and
$\mathbf{F}_{\mathbf{I}}^{2}$ are trained, the trend change maps can be
generated by combining the following three cases. “Appear”/“Disappear” can be
represented by the highlighted background/foreground channel of
$\mathbf{F}^{1}_{\mathbf{I}}$ is converted to the highlighted
foreground/background channel of $\mathbf{F}^{2}_{\mathbf{I}}$. “Transform” is
denoted as that the highlighted foreground A/B channel of
$\mathbf{F}^{1}_{\mathbf{I}}$ is converted to the highlighted foreground B/A
channel of $\mathbf{F}^{2}_{\mathbf{I}}$.

(a) Image T1

(b) Image T2

(c) Change label

(d) Trend label

(e) Change map

(f) Trend map

(g) T1 background

(h) T2 background

Figure 7: Six examples of conducting weakly-supervised TCD task, in which the
first three lines and the last three lines are from CDnet-2014 and SVCD,
respectively. (a), (b), (c) and (d) show the inputted image $\textit{T}_{1}$,
image $\textit{T}_{2}$, change and trend labels. The predicted change and
trend maps are exhibited in (e) and (f). As the SVCD dataset is presented for
GCD task, it does not provide trend labels. It is worth noting that the
background channel of $\mathbf{F}^{1}_{\mathbf{I}}$ and
$\mathbf{F}^{2}_{\mathbf{I}}$ are shown in (g) and (h) to visualize extracted
background information (marked in white).

##  4 Experiments

###  4.1 Data Sets and Metrics

To evaluate the model performance, four large-scale public change detection
data sets are employed, including a natural image data set CDnet-2014 [14, 41]
and three remote sensing image data sets SVCD [18],LEVIR-CD [6], and SYSU-CD
[32].

CDnet-2014 consists of 11 scenes, including badWeather, baseline,
cameraJitter, dynamicBackground, intermittentObjectMotion, lowFramerate,
nightVideos, PTZ, shadow, thermal, and turbulence. Each scene consists of 4-6
videos that are converted into image frames for change detection. Some of
these frames are manually annotated with the changed foreground and the
unchanged background. In this paper, four long videos are used to evaluate
trend prediction, including badminton of cameraJitter, blizzard of badWeather,
copyMachine of shadow and office of baseline. For each video, the bi-temporal
images are constructed with 15 annotated image frames at intervals, while
their change and trend labels can be generated according to three trend
definitions in Section 3.3. Finally, this reconstructed data set are randomly
split into training, validation, and testing sets with the ratio of 2:1:1. It
is worth noting that trend labels will not be used during training of the
proposed weakly-supervised TCD model, which are only used to evaluate model
and compare the results visually.

SVCD is obtained from Google Earth, including seven paired real images with
the size of $4725\times 2700$ and four paired synthetic images with the size
of $1900\times 1000$. The synthetic images are generated by manually adding
objects to image. All paired images are cropped into 16000 paired images,
where 10000/3000/3000 paired images are employed as the
training/validation/testing data sets.

LEVIR-CD is composed of 637 paired remote sensing images with the size of
$1024\times 1024$, which focuses on the building changes. In this paper, we
crop one paired images to sixteen sub-images of $256\times 256$. Based on the
division of author to original images, 7120/1024/2048 paired images are
obtained as the training/validation/testing data sets.

SYSU-CD is a remote sensing data set obtained from Hong Kong, which contains
20000 bi-temporal images with the size of $256\times 256$. The
training/validation/testing data sets have 10000/4000/4000 paired images,
respectively.

Evaluate Metrics. In this paper, five metrics are employed to evaluate model
performance, including precision (P), recall (R), F1-score (F), intersection
over union (IoU), and overall accuracy (OA). The above metrics can be defined
as,

P $\displaystyle=\frac{\textit{TP}}{\textit{TP + FP}},\
\textbf{R}=\frac{\textit{TP}}{\textit{TP + FN}},\
\textbf{F}=\frac{2\cdot\textbf{P}\cdot\textbf{R}}{\textbf{P}+\textbf{R}},$ (5)
IoU $\displaystyle=\frac{\textit{TP}}{\textit{TP + FP + FN}},\
\textbf{OA}=\frac{\textit{TP + TN}}{\textit{TP + FP + TN + FN}},$

where TP, FP, TN and FN denote the number of truth positives, false positives,
true negatives and false negatives. To evaluate the trend prediction, the TP,
FP, TN and FN of three change trends would be counted separately.

###  4.2 Implementation Details

Hardware and Training Setup. The proposed method is implemented using Pytorch
on a high-performance computer with two GeForce RTX 3090 GPUs and an Inter
Xenon Silver 4214R processor. The model parameters are initialized randomly,
and Adam optimizer is used to optimize them during training. The batch size
and epoch number are set to 16 and 200. The learning rate is initially set to
$10^{-3}$, and then decreased by a factor of 10 every 60 epochs. The input
size of the image remains unchanged. To simply highlight feature channel, the
temperature of the softmax layer is set to $0.1$. Data Augmentation is applied
to improve the model generalization, including random manners of cropping,
rotation and flipping. All augmentation strategies are performed in the same
way on two paired images.

###  4.3 Result Analyses

Under the supervision of simple change label, the proposed weakly-supervised
model can predict both change and trend maps in the GCD and TCD branches
simultaneously after inputting bi-temporal images. To assess the effectiveness
of model’s two branches, their visual and numerical results will be analyzed
in turn.

For visual results, we select six examples from CDnet-2014 and SVCD, and show
their change and trend prediction maps in Fig. 7. Particularly, the background
channel of $\mathbf{F}^{1}_{\mathbf{I}}$ and $\mathbf{F}^{2}_{\mathbf{I}}$ are
shown to visualize the background information extracted by the proposed
strategic approach. For the GCD task, six predicted change maps are very close
to the corresponding change labels, which demonstrates the effectiveness of
the model’s GCD branch. Moreover, it can be shown that the softmatch distance
successfully measures the difference between $\mathbf{F}_{\mathbf{C}}^{1}$ and
$\mathbf{F}_{\mathbf{C}}^{2}$, indicating a promising capacity for predicting
change map. In terms of the weakly-supervised TCD task, the predicted trend
map of CDnet-2014 are consistent with the trend labels. Additionally, despite
the lack of trend labels for reference, the SVCD results demonstrate a
promising level of accuracy that can be directly observed. Therefore, both the
encouraging performance of the model’s TCD branch and the softmatch distance’s
ability to enhance the fore-/back-ground directivity of features are verified
simultaneously. By observing Fig. 7(g) and 7(h), it is obvious that the
background information of images T1 and T2 is extracted almost perfectly. This
is another crucial factor contributing to the accurate trend maps that are
achieved.

Video | Type | P | R | F | IoU | OA
---|---|---|---|---|---|---
badminton | C | 93.59 | 84.52 | 88.83 | 79.90 | 98.03
A | 81.63 | 73.00 | 77.08 | 62.70 | 98.69
D | 83.60 | 61.35 | 70.77 | 54.76 | 98.29
T | 90.77 | 64.32 | 75.29 | 60.37 | 98.79
blizzard | C | 93.29 | 97.18 | 95.20 | 90.83 | 99.90
A | 96.54 | 92.86 | 94.67 | 89.88 | 99.73
D | 91.51 | 94.87 | 93.16 | 87.19 | 99.64
T | 91.67 | 72.85 | 81.19 | 68.33 | 99.98
copyMachine | C | 92.71 | 94.60 | 93.65 | 88.05 | 98.52
A | 89.96 | 87.88 | 88.91 | 80.03 | 99.16
D | 89.73 | 90.06 | 89.90 | 81.65 | 99.21
T | 84.03 | 88.60 | 86.25 | 75.83 | 98.94
office | C | 95.88 | 97.93 | 96.89 | 93.98 | 99.33
A | 78.97 | 80.73 | 79.84 | 66.44 | 99.50
D | 82.25 | 87.50 | 84.79 | 73.60 | 99.56
T | 96.80 | 94.88 | 95.83 | 91.99 | 99.33
Table 1: Evaluation results of trend prediction on four videos of CDnet-2014
data set. Here, “C”, “A”, “D” and “T” indicate “Change”, “Appear”, “Disappear”
and “Transform”, respectively.

The numerical results listed in Table 1 further illustrate that the trend
prediction of four videos are performed successfully. Specifically, the
comprehensive metrics F and OA values for “blizzard”, “copyMachine” and
“office” are above $80\%$ and $98\%$, respectively. In summary, the proposed
weakly-supervised method, which relies on the simple change labels,
effectively conducts the peer prediction of GCD task and the upgrade
prediction of the weakly-supervised TCD task, thereby proving its
effectiveness.

###  4.4 Comparative Studies

Since we are unable to find existing models for weakly-supervised TCD, the GCD
branch results are adopted to conduct comparative studies. If our proposed
method surpasses the state-of-the-art approaches in GCD task, it will
demonstrate its superiority, particularly the effectiveness of the softmatch
distance. Specifically, thirteen comparative methods are employed, including
FC-EF [10], FC-SC [10], FC-SD [10], CDNet [2], STANet [6], BIT [5], CLNet
[49], DSAMNet [32], ESCNet [47], IFN [46], ISNet [9], SNUNet [12], DARNet
[21]. The corresponding experimental results on three data sets SVCD, SYSU,
and LEVIR-CD are listed in Table 2, 3, and 4, respectively.

According to the results, the proposed method achieves the highest values for
all three comprehensive metrics (F, IoU, and OA) on the three datasets.
Notably, the model shows significant improvement than other methods on the
SVCD data set. Compared to the second-ranking method DARNet, F, IoU and OA are
increased by 1.1%, 2.09% and 0.23%, respectively. In addition, our method
outperforms most others for the P and R metrics, despite their typically
inverse relationship. For SYSU-CD and LEVIR-CD data sets, the encouraging
performance are acquired, which reflects in the balanced P and R and highest
F, IoU, and OA. These experiments provide further evidence of the
effectiveness of the softmatch distance.

###  4.5 Ablation Studies

As a pluggable module, softmatch distance can be easily replaced with other
common distances (Cosine and Euclidean distances) to verify its contribution
to our model. We decided to perform this replacement in the GCD branch for
ablation studies, since the weakly-supervised TCD task must be supported by
using the softmatch distance. Based on this, ablation experiments are
conducted on three data sets SVCD, LEVIR-CD and SYSU-CD, whose results are
listed in Table 5. According to these results, it is easy find that softmatch
distance outperforms Euclidean and cosine distances for all five evaluate
metrics on the SVCD data set. Although P and R do not reach the best scores
simultaneously on both LEVIR-CD and SYSU-CD, the fact that three comprehensive
metrics F, IoU and OA achieve the highest values demonstrates the superior
performance of softmatch distance. Therefore, the contribution of softmatch
distance to our model is verified successfully.

Method | P | R | F | IoU | OA
---|---|---|---|---|---
FC-EF [10] | 89.87 | 66.49 | 76.43 | 61.85 | 94.74
CDNet [2] | 73.40 | 95.31 | 82.93 | 70.84 | 95.37
FC-SC [10] | 89.50 | 73.67 | 80.82 | 67.81 | 95.52
FC-SD [10] | 88.62 | 74.78 | 81.11 | 68.23 | 95.53
STANet [6] | 87.35 | 95.77 | 91.36 | 84.10 | 97.86
BIT [5] | 95.31 | 87.31 | 91.13 | 83.71 | 98.00
CLNet [49] | 93.30 | 89.80 | 91.52 | 84.36 | 98.03
DSAMNet [32] | 90.16 | 98.42 | 94.10 | 88.87 | 98.41
ESCNet [47] | 92.86 | 96.06 | 94.43 | 89.45 | 98.60
IFN [46] | 97.15 | 91.70 | 94.34 | 89.29 | 98.70
ISNet [9] | 95.18 | 94.43 | 94.80 | 90.12 | 98.78
SNUNet [12] | 96.33 | 96.00 | 96.16 | 92.61 | 99.09
DARNet [21] | 97.05 | 96.91 | 96.98 | 94.14 | 99.29
Our | 98.18 | 97.98 | 98.08 | 96.23 | 99.52
Table 2: Comparative experimental results on SVCD data set. Method | P | R | F | IoU | OA
---|---|---|---|---|---
STANet [6] | 70.26 | 86.18 | 77.41 | 63.15 | 88.14
SNUNet [12] | 78.26 | 74.42 | 76.30 | 61.68 | 89.10
CLNet [49] | 79.62 | 74.97 | 77.22 | 62.90 | 89.57
DSAMNet [32] | 77.11 | 79.90 | 78.48 | 64.59 | 89.67
BIT [5] | 79.18 | 77.01 | 78.08 | 64.04 | 89.80
FC-SD [10] | 79.78 | 74.38 | 76.99 | 62.58 | 89.83
FC-EF [10] | 84.77 | 68.63 | 75.85 | 61.09 | 89.84
FC-SC [10] | 83.03 | 71.60 | 76.90 | 62.46 | 89.90
ISNet [9] | 80.27 | 76.41 | 78.29 | 64.44 | 90.01
IFN [46] | 82.39 | 73.57 | 77.73 | 63.57 | 90.06
Our | 83.21 | 74.91 | 78.84 | 65.07 | 90.52
Table 3: Comparative experimental results on SYSU-CD data set. Method | P | R | F | IoU | OA
---|---|---|---|---|---
FC-SC [10] | 90.76 | 58.95 | 71.47 | 55.61 | 97.60
FC-SD [10] | 91.97 | 57.84 | 71.02 | 55.06 | 97.60
IFN [46] | 86.95 | 75.24 | 80.67 | 67.61 | 98.16
DSAMNet [32] | 80.61 | 88.98 | 84.59 | 73.29 | 98.35
CLNet [49] | 90.07 | 85.70 | 87.83 | 78.30 | 98.79
BIT [5] | 89.24 | 89.37 | 89.31 | 80.68 | 98.92
SNUNet [12] | 91.80 | 88.53 | 90.14 | 82.04 | 99.01
ISNet [9] | 92.46 | 88.27 | 90.32 | 82.35 | 99.04
Our | 91.87 | 89.59 | 90.72 | 83.01 | 99.07
Table 4: Comparative experimental results on LEVIR-CD data set. Data set | Distance | P | R | F | IoU | OA
---|---|---|---|---|---|---
SVCD | Euclidean | 97.35 | 97.12 | 97.23 | 94.61 | 99.31
Cosine | 97.44 | 97.43 | 97.43 | 94.99 | 99.36
Softmatch | 98.18 | 97.98 | 98.08 | 96.23 | 99.52
LEVIR-CD | Euclidean | 88.09 | 91.01 | 89.53 | 81.04 | 98.92
Cosine | 89.96 | 90.28 | 90.12 | 82.02 | 98.99
Softmatch | 91.87 | 89.59 | 90.72 | 83.01 | 99.07
SYSU-CD | Euclidean | 83.54 | 73.28 | 78.07 | 64.03 | 90.29
Cosine | 78.51 | 76.44 | 77.46 | 63.21 | 89.51
Softmatch | 83.21 | 74.91 | 78.84 | 65.07 | 90.52
Table 5: Ablation experimental results of three distance metrics on three data
sets

##  5 Conclusion

In this paper, we introduce a novel softmatch distance, which is used to
construct a weakly-supervised TCD branch in a simple GCD model. The modified
model consists of three components: feature extraction, GCD branch and weakly-
supervised TCD branch. Here, softmatch distance is used in the GCD branch to
measure feature difference and generate the change map, while also enhancing
the fore-/back-ground directivity of features in TCD branch to predict the
trend map. Moreover, a strategic approach is developed to explore background
information from bi-temporal images, which is essential for weakly-supervised
TCD task. The promising experimental results demonstrate the effectiveness of
our proposed method.

## References

  * [1] Peri Akiva, Matthew Purri, and Matthew Leotta.  Self-supervised material and texture representation learning for remote sensing tasks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8203–8215, 2022.
  * [2] Pablo F Alcantarilla, Simon Stent, German Ros, Roberto Arroyo, and Riccardo Gherardi.  Street-view change detection with deconvolutional networks.  Autonomous Robots, 42(7):1301–1322, 2018.
  * [3] Adrian Boguszewski, Dominik Batorski, Natalia Ziemba-Jankowska, Tomasz Dziedzic, and Anna Zambrzycka.  Landcover. ai: Dataset for automatic mapping of buildings, woodlands, water and roads from aerial imagery.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1102–1110, 2021.
  * [4] Rodrigo Caye Daudt, Bertrand Le Saux, Alexandre Boulch, and Yann Gousseau.  Guided anisotropic diffusion and iterative learning for weakly supervised change detection.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0, 2019.
  * [5] Hao Chen, Zipeng Qi, and Zhenwei Shi.  Remote sensing image change detection with transformers.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2021\.
  * [6] Hao Chen and Zhenwei Shi.  A spatial-temporal attention-based method and a new dataset for remote sensing image change detection.  Remote Sensing, 12(10):1662, 2020.
  * [7] Long Chen, Shaobo Lin, Xiankai Lu, Dongpu Cao, Hangbin Wu, Chi Guo, Chun Liu, and Fei-Yue Wang.  Deep neural network based vehicle and pedestrian detection for autonomous driving: A survey.  IEEE Transactions on Intelligent Transportation Systems, 22(6):3234–3246, 2021.
  * [8] Shuo Chen, Kailun Yang, and Rainer Stiefelhagen.  Dr-tanet: dynamic receptive temporal attention network for street scene change detection.  In 2021 IEEE Intelligent Vehicles Symposium (IV), pages 502–509. IEEE, 2021.
  * [9] Gong Cheng, Guangxing Wang, and Junwei Han.  Isnet: Towards improving separability for remote sensing image change detection.  IEEE Transactions on Geoscience and Remote Sensing, 2022.
  * [10] Rodrigo Caye Daudt, Bertr Le Saux, and Alexandre Boulch.  Fully convolutional siamese networks for change detection.  In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 4063–4067. IEEE, 2018.
  * [11] Lei Ding, Haitao Guo, Sicong Liu, Lichao Mou, Jing Zhang, and Lorenzo Bruzzone.  Bi-temporal semantic reasoning for the semantic change detection in hr remote sensing images.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–14, 2022\.
  * [12] Sheng Fang, Kaiyu Li, Jinyuan Shao, and Zhe Li.  Snunet-cd: A densely connected siamese network for change detection of vhr images.  IEEE Geoscience and Remote Sensing Letters, 19:1–5, 2021.
  * [13] Wei Feng, Fei-Peng Tian, Qian Zhang, Nan Zhang, Liang Wan, and Jizhou Sun.  Fine-grained change detection of misaligned scenes with varied illuminations.  In Proceedings of the IEEE International Conference on Computer Vision, pages 1260–1268, 2015.
  * [14] Nil Goyette, Pierre-Marc Jodoin, Fatih Porikli, Janusz Konrad, and Prakash Ishwar.  Changedetection. net: A new change detection benchmark dataset.  In 2012 IEEE computer society conference on computer vision and pattern recognition workshops, pages 1–8. IEEE, 2012.
  * [15] Hossam Haick and Ning Tang.  Artificial intelligence in medical sensors for clinical decisions.  ACS nano, 15(3):3557–3567, 2021.
  * [16] Mehrdad Hosseinzadeh and Yang Wang.  Image change captioning by learning from an auxiliary task.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2725–2734, 2021.
  * [17] Anna Khoreva, Rodrigo Benenson, Jan Hosang, Matthias Hein, and Bernt Schiele.  Simple does it: Weakly supervised instance and semantic segmentation.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 876–885, 2017.
  * [18] MA Lebedev, Yu V Vizilter, OV Vygolov, VA Knyaz, and A Yu Rubis.  Change detection in remote sensing images using conditional adversarial networks.  International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 42(2), 2018.
  * [19] Yinjie Lei, Duo Peng, Pingping Zhang, Qiuhong Ke, and Haifeng Li.  Hierarchical paired channel fusion network for street scene change detection.  IEEE Transactions on Image Processing, 30:55–67, 2020.
  * [20] Liulei Li, Tianfei Zhou, Wenguan Wang, Jianwu Li, and Yi Yang.  Deep hierarchical semantic segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1246–1257, 2022.
  * [21] Ziming Li, Chenxi Yan, Ying Sun, and Qinchuan Xin.  A densely attentive refinement network for change detection based on very-high-resolution bitemporal remote sensing images.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–18, 2022\.
  * [22] Zhiyong Lv, Tongfei Liu, Jón Atli Benediktsson, and Nicola Falco.  Land cover change detection techniques: Very-high-resolution optical images: A review.  IEEE Geoscience and Remote Sensing Magazine, 10(1):44–63, 2021\.
  * [23] Murari Mandal and Santosh Kumar Vipparthi.  An empirical review of deep learning frameworks for change detection: model design, experimental frameworks, challenges and research needs.  IEEE Transactions on Intelligent Transportation Systems, 2021.
  * [24] Shawky Mansour, Mohammed Al-Belushi, and Talal Al-Awadhi.  Monitoring land use and land cover changes in the mountainous cities of oman using gis and ca-markov modelling techniques.  Land Use Policy, 91:104414, 2020.
  * [25] Hyeoncheol Noh, Jingi Ju, Minseok Seo, Jongchan Park, and Dong-Geol Choi.  Unsupervised change detection based on image reconstruction loss.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1352–1361, 2022.
  * [26] Islam Osman, Mohamed Abdelpakey, and Mohamed S Shehata.  Transblast: Self-supervised learning using augmented subspace with transformer for background/foreground separation.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 215–224, 2021.
  * [27] Tongyao Pang, Huan Zheng, Yuhui Quan, and Hui Ji.  Recorrupted-to-recorrupted: unsupervised deep learning for image denoising.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2043–2052, 2021.
  * [28] Jin-Man Park, Jae-Hyuk Jang, Sahng-Min Yoo, Sun-Kyung Lee, Ue-Hwan Kim, and Jong-Hwan Kim.  Changesim: towards end-to-end online scene change detection in industrial indoor environments.  In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 8578–8585. IEEE, 2021.
  * [29] Jin-Man Park, Ue-Hwan Kim, Seon-Hoon Lee, and Jong-Hwan Kim.  Dual task learning by leveraging both dense correspondence and mis-correspondence for robust change detection with imperfect matches.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13749–13759, 2022.
  * [30] Lixiang Ru, Bo Du, and Chen Wu.  Multi-temporal scene classification and scene change detection with correlation based fusion.  IEEE Transactions on Image Processing, 30:1382–1394, 2020.
  * [31] Mohammed Yousef Shaheen.  Adoption of machine learning for medical diagnosis.  ScienceOpen Preprints, 2021.
  * [32] Qian Shi, Mengxi Liu, Shengchen Li, Xiaoping Liu, Fei Wang, and Liangpei Zhang.  A deeply supervised attention metric-based network and an open aerial image dataset for remote sensing change detection.  IEEE transactions on geoscience and remote sensing, 60:1–16, 2021\.
  * [33] Badri Narayan Subudhi, Thangaraj Veerakumar, S Esakkirajan, and Ashish Ghosh.  Kernelized fuzzy modal variation for local change detection from video scenes.  IEEE Transactions on Multimedia, 22(4):912–920, 2019.
  * [34] Xu Tang, Huayu Zhang, Lichao Mou, Fang Liu, Xiangrong Zhang, Xiao Xiang Zhu, and Licheng Jiao.  An unsupervised remote sensing change detection method based on multiscale graph convolutional network and metric learning.  IEEE Transactions on Geoscience and Remote Sensing, 60:1–15, 2021\.
  * [35] Nimisha TM, AN Rajagopalan, and Rangarajan Aravind.  Seamless change detection and mosaicing for aerial imagery.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 1–8, 2015.
  * [36] Aysim Toker, Lukas Kondmann, Mark Weber, Marvin Eisenberger, Andrés Camero, Jingliang Hu, Ariadna Pregel Hoderlein, Çağlar Şenaras, Timothy Davis, Daniel Cremers, et al.  Dynamicearthnet: Daily multi-spectral satellite dataset for semantic change segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21158–21167, 2022.
  * [37] Maria Vakalopoulou, Konstantinos Karantzalos, Nikos Komodakis, and Nikos Paragios.  Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data.  In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 61–69, 2015.
  * [38] Sagar Verma, Akash Panigrahi, and Siddharth Gupta.  Qfabric: multi-task change detection dataset.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1052–1061, 2021.
  * [39] Guo-Hua Wang, Bin-Bin Gao, and Chenjie Wang.  How to reduce change detection to semantic segmentation.  Pattern Recognition, page 109384, 2023.
  * [40] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu.  Cosface: Large margin cosine loss for deep face recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5265–5274, 2018.
  * [41] Yi Wang, Pierre-Marc Jodoin, Fatih Porikli, Janusz Konrad, Yannick Benezeth, and Prakash Ishwar.  Cdnet 2014: An expanded change detection benchmark dataset.  In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 387–394, 2014.
  * [42] Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li.  Uformer: A general u-shaped transformer for image restoration.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17683–17693, 2022.
  * [43] Chuanguang Yang, Helong Zhou, Zhulin An, Xue Jiang, Yongjun Xu, and Qian Zhang.  Cross-image relational knowledge distillation for semantic segmentation.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12319–12328, 2022.
  * [44] Yuan-Fu Yang and Min Sun.  Semiconductor defect detection by hybrid classical-quantum deep learning.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2323–2332, 2022.
  * [45] Muhammad Zaigham Zaheer, Arif Mahmood, M Haris Khan, Marcella Astrid, and Seung-Ik Lee.  An anomaly detection system via moving surveillance robots with human collaboration.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2595–2601, 2021.
  * [46] Chenxiao Zhang, Peng Yue, Deodato Tapete, Liangcun Jiang, Boyi Shangguan, Li Huang, and Guangchao Liu.  A deeply supervised image fusion network for change detection in high resolution bi-temporal remote sensing images.  ISPRS Journal of Photogrammetry and Remote Sensing, 166:183–200, 2020.
  * [47] Hongyan Zhang, Manhui Lin, Guangyi Yang, and Liangpei Zhang.  Escnet: An end-to-end superpixel-enhanced change detection network for very-high-resolution remote sensing images.  IEEE Transactions on Neural Networks and Learning Systems, 2021\.
  * [48] Zhuo Zheng, Ailong Ma, Liangpei Zhang, and Yanfei Zhong.  Change is everywhere: Single-temporal supervised object change detection in remote sensing imagery.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15193–15202, 2021.
  * [49] Zhi Zheng, Yi Wan, Yongjun Zhang, Sizhe Xiang, Daifeng Peng, and Bin Zhang.  Clnet: Cross-layer convolutional neural network for change detection in optical remote sensing imagery.  ISPRS Journal of Photogrammetry and Remote Sensing, 175:247–267, 2021.
  * [50] Zhuo Zheng, Yanfei Zhong, Shiqi Tian, Ailong Ma, and Liangpei Zhang.  Changemask: Deep multi-task encoder-transformer-decoder architecture for semantic change detection.  ISPRS Journal of Photogrammetry and Remote Sensing, 183:228–239, 2022.
  * [51] Yunsong Zhou, Yuan He, Hongzi Zhu, Cheng Wang, Hongyang Li, and Qinhong Jiang.  Monocular 3d object detection: An extrinsic parameter free approach.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7556–7566, 2021.