# RPCL: A Framework for Improving Cross-Domain Detection
with Auxiliary Tasks

Kai Li1, Curtis Wigington2, Chris Tensmeyer2, Vlad I. Morariu2,
Handong Zhao2, Varun Manjunatha2, Nikolaos Barmpalios3, Yun Fu1
1Northeastern University, 2Adobe Research, 3Adobe Document Cloud
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Cross-Domain Detection (XDD) aims to train an object detector using labeled
image from a source domain but have good performance in the target domain with
only unlabeled images. Existing approaches achieve this either by aligning the
feature maps or the region proposals from the two domains, or by transferring
the style of source images to that of target image. Contrasted with prior
work, this paper provides a complementary solution to align domains by
learning the same auxiliary tasks in both domains simultaneously. These
auxiliary tasks push image from both domains towards shared spaces, which
bridges the domain gap. Specifically, this paper proposes Rotation Prediction
and Consistency Learning (PRCL), a framework complementing existing XDD
methods for domain alignment by leveraging the two auxiliary tasks. The first
one encourages the model to extract region proposals from foreground regions
by rotating an image and predicting the rotation angle from the extracted
region proposals. The second task encourages the model to be robust to changes
in the image space by optimizing the model to make consistent class
predictions for region proposals regardless of image perturbations.
Experiments show the detection performance can be consistently and
significantly enhanced by applying the two proposed tasks to existing XDD
methods.

##  1 Introduction

Powered by deep learning, the task of recognizing and localizing an object of
interest in a scene, i.e., object detection, has been tremendously advanced in
recent years [15, 16, 40, 32, 37, 38, 39, 18]. While a deep learning based
object detector may have impressive performance on data within the same
distribution as the data the detector was trained on, its performance often
drops significantly when tested on data drawn from a different distribution.
This is the so-called domain shift problem.

Cross-Domain Detection (XDD) addresses the domain shift problem by jointly
training a detector with unlabeled data from the domain of interest (target
domain) and labeled data from an auxiliary domain (source domain) [5]. By
aligning the distributions of the two domains during training, the label
supervision from the source domain becomes more shareable to the target domain
and hence a detector of enhanced generalizability can be obtained.

Various approaches have been proposed to align domain distributions. The first
category of approaches focus on feature alignment where images from both
domains are fed to a detection network and are aligned with feature maps at
different levels or extracted region proposals [57, 43, 5, 19, 31, 20].
Adversarial learning is often used to achieve this where domain classifiers
try to distinguish between domains and the detection model is optimized to
produce features indistinguishable between domains. The second category of
approaches are based on pseudo-labeling where the step of pseudo-label
prediction and the step of model calibration are executed iteratively [25, 23,
24, 41]. A detection model, usually pretrained using the labeled source data,
predicts labels on the target data. Next, the predicted labels of high
confidence are selected to update the model. The third category of approaches
transforms the source images to resemble the target images with generative
models [23, 26]. While similar to the first category in the philosophy of
alignment, these methods operate on image pixels directly instead of the
feature representations.

Figure 1: Illustration demonstrating the use of auxiliary tasks to align
domains. Different shapes represent different classes. (Left): Without our
method, the source and target domains are not well aligned and the decision
boundary suited for the source domain cannot generalize well to the target
domain. (Middle): The proposed auxiliary tasks treat source and target samples
indiscriminately and learn shared representation spaces for all samples.
(Right): With the auxiliary tasks, the source and target domains are now
closely aligned and the decision boundary generalizes well to the target
domain.

We propose techniques that are orthogonal to the three aforementioned
categories and as a result complement various existing algorithms. Our main
idea is to augment existing XDD models with auxiliary learning tasks that are
applied on both domains simultaneously. These auxiliary tasks do not require
annotated labels and thus handle source and target samples indiscriminately.
The learning objectives of these task guide the model’s learning differently
and complement supervised learning, and thus contributing to better alignment
results. Figure 1 illustrate this idea.

Specifically, we propose Rotation Prediction and Consistency Learning (RPCL),
a framework that can incorporate existing XDD algorithms for enhancing
adaptation performance. RPCL includes two auxiliary tasks, namely the Rotation
Prediction task and Consistent Learning task that are applied to images from
both domains simultaneously. For the rotation prediction task, we rotate a
given image by a random angle and then predict the angle based on features of
region proposals extracted from the image. This task encourages the model to
extract region proposals from the foreground regions because background
regions usually lack semantics sufficient to predict the rotation angles. The
consistency learning task first perturbs a given image and then enforces the
consistency of the same set of region proposals for the classification labels
predicted in the original image and the augmented one. This task forces the
model to be robust to changes in the image space and improve its capability of
handling the domain gap.

It is worth noting that the rotation prediction task originates from self-
supervised learning [14] and the consistency learning task from semi-
supervised learning [1, 47, 50]. While we do not propose fundamentally new
auxiliary tasks, we offer insights on drawing connections among seemingly
distinct tasks. We view unsupervised domain adaptation as a special case of
semi-supervised learning, where the unlabeled data are drawn from a different
data distribution due to the domain shift. With this view, we can harvest the
recent progress from semi-supervised learning to address the adaptation
problem. Besides, we cast a unified view towards classification and detection
in the region proposal level; a region proposal, once extracted from a large
scene, can be viewed as a single-object image typically used for
classification. With this framing, techniques applied on images can also be
applied on region proposals, if properly adapted. Based on these insights, we
adapt effective techniques from other tasks, unifying them in a framework that
can be applied to address the target problem and leading to consistently
significant improvement over existing approaches.

##  2 Related Work

###  2.1 Cross-Domain Detection

Previous work in Cross-Domain Detection (XDD) addresses the domain shift
problem by aligning the features or region proposals from the source and
target domains [57, 43, 5, 19]. The alignment is often achieved by adversarial
training where domain classifiers predict the domains of the
pixels/images/proposals, while the detection model aims to deceive the
classifiers. One drawback of these previous methods is that the foreground and
background regions are treated equally, which is undesirable as foreground
regions are more semantically meaningful. Various techniques have been
proposed to emphasize the alignment of foreground regions, including learning
an image classification task to regularize the model to activate regions
containing the main objects [51]; explicitly learning objectness and
centerness for every pixel, and assigning weights accordingly [22]; and
exploiting attention mechanism [3]. Another line of approaches trains the
models iteratively by generating pseudo bounding box labels for target images
and updating the models with the generated pseudo-labels [25, 23, 24, 41].
Different methods vary in how they generate the pseudo-labels or update the
model. Some methods enhance the adaptation performance by improving the input
images. They usually train a style-transfer model (e.g., CycleGAN [56]) using
images from both domains and then apply the model to translate images from the
source domain as the style of the target domain [23, 26]. As the image style
difference narrows, adapting label supervision from the source domain to the
target domain becomes easier. We address the XDD problem in a complementary
way by proposing a generic framework where existing methods can be
incorporated and have performance enhanced by performing two auxiliary tasks
simultaneously in both domains.

Figure 2: Illustration of the proposed framework. Our framework augments a
backbone detector at training time with two additional losses for rotation
prediction ($L_{rp}$) and consistency learning ($L_{cl}$) which improve
performance on the target domain.

###  2.2 Self-Supervised Learning

Self-Supervised Learning (SSL) aims to use the data itself as supervision in a
pretext task where the model can learn to extract informative representations
from unlabeled data. Early efforts focus on designing various pretext tasks
including image colorization [54, 30, 55], image rotation prediction [14],
spatial context prediction [11], solving jigsaw puzzles [35], image inpainting
[36], and contrastive learning [4, 17]. A comparison of some of these
approaches can be found in [27]. It shows that the simple image rotation
prediction task has shown promising results. SSL has also been introduced to
address the domain adaptive classification problem [49, 42, 52] where SSL is
used as an auxiliary task jointly trained along with the main alignment tasks.
We follow this idea but focus on the detection problem instead. Thus, rather
than performing SSL task with entire images, we apply it on region proposals.
To our best knowledge, this is the first use of SSL to address the XDD
problem.

###  2.3 Consistency Learning

Consistency learning regularizes model predictions to be invariant to moderate
changes applied to input examples. It has been a popular technique in recent
semi-supervised learning literature [2, 1, 50, 47]. Different consistency
training methods vary in how data perturbations are generated and how the
consistency loss is composed. Some methods perturb images by compositing
various image transformation techniques, including translation, flipping,
rotation, stretching, shearing, adding noise, etc. [12, 9, 1]. MixUp [53], a
technique that performs linear interpolation between the samples to generate
virtual samples, is used in [2]. Learning based augmentation approaches have
also been proposed, such as AutoAugment [8] and population based augmentation
[21] which employ reinforcement learning to search for the most effective
combinations of transformations. Regarding the consistency loss, early works
use the squared $l_{2}$ loss to minimize the discrepancy of the probabilities
of different version of the same images [44, 29]. The following methods
replace it with the cross-entropy loss [1, 34, 50, 47]. We adopt consistency
learning from semi-supervised learning to address the domain shift problem for
object detection. While existing methods apply the consistency constraint at
the image-level, we enforce it on the region proposals.

##  3 Algorithm

We propose a general framework that can improve different existing cross-
domain detectors by applying our two novel alignment techniques. In this
section, we will first present an overview of the proposed framework and then
introduce the details of the two domain alignment techniques.

###  3.1 Framework Overview

Given a labeled dataset $\mathcal{S}=\\{\mathcal{X}_{s},\mathcal{Y}_{s}\\}$
from the source domain and an unlabeled target dataset
$\mathcal{T}=\\{\mathcal{X}_{t}\\}$, Cross Domain Detection (XDD) learns an
object detector under the following framework:

$L=L_{det}(\mathcal{X}_{s},\mathcal{Y}_{s})+\alpha
L_{uda}(\mathcal{X}_{s},\mathcal{X}_{t}),$ (1)

where $\mathcal{X}_{s}$ and $\mathcal{X}_{t}$ are the images,
$\mathcal{Y}_{s}$ denotes the labels which specify the locations and
categories of the objects, and $\alpha$ is a hyper-parameter. The first term
of Eq. (1) is the standard supervised learning objective for object detection.
It includes the classification objective and bounding box regression objective
using labeled images from the source domain. The second term is the
unsupervised domain alignment objective that aims to align the distributions
of the source and target domains. It is unsupervised in the sense that it
works without the need for ground truth labels. Within this framework, many
effective approaches have been proposed for the unsupervised domain alignment
objective. Some align the features maps of different levels, the extracted
region proposals, or their combination usually by adversarial learning [57,
43, 5, 19]. Other methods use GANs to convert images from one domain to the
other so that source domain labels can be used in the target domain [23, 26].

We address XDD in a way orthogonal to existing approaches. Rather than
proposing another unsupervised domain alignment technique, we investigate how
auxiliary tasks can help address the domain gap. Specifically, we propose a
general framework that can enhance the performance of existing XDD methods by
leveraging two auxiliary tasks. The two tasks are applicable to both the
source and target domains and thus serve to bridge the domain gap. The first
one is the region proposal based image rotation prediction task which rotates
an image and predicts the image rotation angle from the region proposals
extracted from the unrotated image. The second task is the consistency
learning task where the model is trained to make consistent classification
predictions for the same set of region proposals within an image and its
strongly augmented version. Figure 2 illustrates our framework.

Figure 3: Predicting image rotation angle based on region proposals can help
localize foreground regions.

###  3.2 Proposal-Based Rotation Angle Prediction

Training a model to predict the rotation angle of a given image was proposed
in [14] for self-supervised learning. It is based on the intuition that a
model can predict the rotation angle correctly if it has a deep understanding
of the given image, including localization of salient objects, their
orientation, the object type, etc. This inspires us to leverage this task to
address the XDD problem because it does not require manually annotated labels,
which suits the unsupervised domain adaptation setting well, and it helps
localize salient objects and identify the object type, which is exactly the
goal of object detection.

A straightforward way of exploiting this task is to learn the rotation
prediction task jointly with the detection task by rotating the input image
and training the model to predict the rotation angle from the feature
representation of the given image. This is how this task is utilized for the
classification problem [49, 28, 52, 48]. However, this practice is suboptimal
for the detection problem because images used for detection are often much
more complex, containing more salient objects in backgrounds with richer
contexts. It may be too difficult for the model to learn a global
representation for the whole image that encodes the essential information for
all the salient objects.

Our insight is that classification and detection can be unified in the region
proposal level: a region proposal, once extracted from a large scene, can be
viewed as a single-object image typically used for classification. Based on
this insight, we propose to predict the rotation angle from the region
proposals. This practice has two merits. First, it encourages the detection
model to extract region proposals from the foreground since the foreground
contains semantic information that are essential to predict the rotation
angle. As shown in Figure 3, it is easy to tell the rotation angle from the
car region, while hardly possible from the road region. Training the model to
predict the rotation angle correctly encourages it to extract region proposals
from foreground, which benefits for the detection task. Second, this enhances
the feature alignment of foreground regions as the model will activate more on
the foreground regions and thus contribute more when aligning features from
the two domains.

Formally, given a source image $\textbf{s}\in\mathcal{X}_{s}$, we obtain
$\textbf{s}^{r}=Rot(x_{s})$ by rotating s with an random angle $\theta_{s}$
from $[0^{\circ},90^{\circ},180^{\circ},270^{\circ}]$. From $\textbf{s}^{r}$,
we extract a set of region proposals $\mathcal{R}_{s}$ with the same rotation
angle $\theta_{s}$. Similarly, we can get a set of region region proposals
$\mathcal{R}_{t}$ with the same rotation angle $\theta_{t}$ for every target
image $\textbf{t}\in\mathcal{X}_{t}$. We align the domains by applying the
rotation prediction task simultaneously on the two domains. Thus, our learning
objective for this task is as follows:

$\begin{array}[]{cl}L_{rp}(\mathcal{X}_{s},\mathcal{X}_{t})=&\frac{1}{|\mathcal{X}_{s}||\mathcal{R}_{s}|}\sum_{\textbf{s}\sim\mathcal{X}_{s}}\sum_{\textbf{r}_{s}\sim\mathcal{R}_{s}}L(\textbf{r}_{s},\theta_{s})+\\\
&\frac{1}{|\mathcal{X}_{t}||\mathcal{R}_{t}|}\sum_{\textbf{t}\sim\mathcal{X}_{t}}\sum_{\textbf{r}_{t}\sim\mathcal{R}_{t}}L(\textbf{r}_{t},\theta_{t}),\end{array}$
(2)

where $L(\textbf{r}_{s},\theta_{s})$ and $L(\textbf{r}_{t},\theta_{t})$ are
the cross-entropy losses for the source and target proposals, respectively.

###  3.3 Consistency Learning

Consistency learning regularizes model predictions to be invariant to moderate
changes applied to input examples. It has shown impressive performance for
semi-supervised learning [34, 50, 47, 6] recently. Based on the insight that
unsupervised domain adaptation is a special case of semi-supervised learning
where the unlabeled data is drawn from a different data distribution due to
the domain shift, we propose to use consistency learning to address the XDD
problem. Same as the rotation prediction task, we apply consistency learning
on region proposals.

For each source image $\textbf{s}\in\mathcal{X}_{s}$, we apply data
augmentation $\Phi$ and generate

$\hat{\textbf{s}}=\Phi(\mathbf{s}).$ (3)

Following the previous methods [47, 1], we use RandAugment [9] as the data
augmentation $\Phi$, which produces highly perturbed images by uniformly
sampling from the image processing transformations in Python Image Library,
including polarization, solarization, brightness change, color change, etc.
For ease of implementation, we exclude the transformations that change the
positions of pixels (e.g., flipping, rotation, etc.). This ensures s and
$\hat{\textbf{s}}$ have pixel-to-pixel correspondence for every position.
However, our framework could also work with transformations that change the
position of pixels as long as the region proposals in the original image can
be converted to the coordinates of the transformed image.

We extract a set of region proposals $\mathcal{R}_{s}$ from s and map
$\mathcal{R}_{s}$ directly from s to $\hat{\textbf{s}}$, obtaining
$\hat{\mathcal{R}}_{s}$. This ensures that every region proposal
$\mathbf{r}_{s}\sim\mathcal{R}_{s}$ from s can find the corresponding
$\hat{\mathbf{r}}_{s}\sim\hat{\mathcal{R}}_{s}$ from $\hat{\textbf{s}}$ that
localizes the same region in the scene. So, the pair of corresponding region
proposals should be classified consistently by the classification branch of
the detection model.

We enforce this consistency by optimizing the following objective function:

$L^{s}_{cl}=\frac{1}{|\mathcal{R}_{s}|}\underset{{\mathbf{r}_{s}\sim\mathcal{R}_{s},\hat{\mathbf{r}}_{s}\sim\hat{\mathcal{R}}_{s}}}{\sum}\big{[}\mathbbm{1}(\max(\textbf{p}_{s})\geq\sigma)H(\textbf{p}^{\prime}_{s},\hat{\textbf{p}}_{s})\big{]},$
(4)

where $\textbf{p}_{s}$ and $\hat{\textbf{p}}_{s}$ are the classification
probabilities of proposals $\textbf{r}_{s}$ and $\hat{\textbf{r}}_{s}$,
respectively. $\textbf{p}^{\prime}_{s}=\arg\max(\textbf{p}_{s})$ returns a
one-hot vector for the prediction; $H(.,.)$ is the cross-entropy of two
possibility distributions; $\max(\textbf{p}_{s})$ returns the highest
possibility score.

In essence, we enforce consistency of the class predictions for a pair of
corresponding region proposals $(\textbf{r}_{s},\hat{\textbf{r}}_{s})$ by
computing a pseudo label from $\textbf{r}_{s}$ and apply the pseudo label on
$\hat{\textbf{s}}$ by computing the standard cross-entropy loss. To mitigate
the impact of incorrect pseudo labels, only the samples with confident
predictions (the highest probability scores are above a threshold) are used
for loss computation.

Algorithm 1. Proposed RPCL framework
---
Input: Source set $\mathcal{S}=\\{\mathcal{X}_{s},\mathcal{Y}_{s}\\}$ and
target set $\mathcal{T}=\\{\mathcal{X}_{u}\\}$.
Output: Domain adaptive detector.
while not done do
1\. Randomly sample $(\textbf{s},y_{s})\sim\mathcal{S}$ and
$\textbf{t}\sim\mathcal{T}$.
2\. Rotate s and get $(\textbf{s}^{r},\theta_{s})=Rot(\textbf{s})$; rotate t
and get
$(\textbf{t}^{r},\theta_{t})=Rot(\textbf{t})$; augment s and get
$\hat{\textbf{s}}=\Phi(\textbf{s})$;
augment t and get $\hat{\textbf{t}}=\Phi(\textbf{t})$.
3\. Feed-forward
$(\textbf{s},\textbf{s}^{r},\hat{\textbf{s}},\textbf{t},\textbf{t}^{r},\hat{\textbf{t}})$
to the model.
3\. Calculate the detection loss and unsupervised domain
alignment loss in Eq. (1) using $(\textbf{s},y_{s})$ and t.
4\. Calculate the rotation prediction loss in Eq. (2)
using $(\textbf{s},\theta_{s})$ and $(\textbf{t},\theta_{t})$.
5\. Calculate the consistency learning loss in Eq. (5)
using $(\textbf{s},\hat{\textbf{s}})$ and $(\textbf{t},\textbf{t}^{\prime})$.
6\. Back-propagate the loss in Eq. (6).
end while
|  | person | rider | car | truck | bus | train | mbike | bicycle | mAP
---|---|---|---|---|---|---|---|---|---|---
Source only | 17.8 | 23.6 | 27.1 | 11.9 | 23.8 | 9.1 | 14.4 | 22.8 | 18.8
$\textrm{DAF}^{*}$ [5] | 25.0 | 31.0 | 40.5 | 22.1 | 35.3 | 20.2 | 20.0 | 27.1 | 27.6
DAF [5] | 31.5 | 40.9 | 43.9 | 21.4 | 34.2 | 20.2 | 27.8 | 35.4 | 31.9
SWDA [43] | 29.9 | 42.3 | 43.5 | 24.5 | 36.2 | 32.6 | 30.0 | 35.3 | 34.3
SC-DA [57] | 33.5 | 38.0 | 48.5 | 26.5 | 39.0 | 23.3 | 28.0 | 33.6 | 33.8
MAF [19] | 28.2 | 39.5 | 43.9 | 23.8 | 39.9 | 33.3 | 29.2 | 33.9 | 34.0
DAM [26] | 30.8 | 40.5 | 44.3 | 27.2 | 38.4 | 34.5 | 28.4 | 32.2 | 34.6
GA-CA [22] | 41.9 | 38.7 | 56.7 | 22.6 | 41.5 | 26.8 | 24.6 | 35.5 | 36.0
ECR-DAF [51] | 29.7 | 37.3 | 43.6 | 20.8 | 37.3 | 12.8 | 25.7 | 31.7 | 29.9
ECR-SWDA [51] | 32.9 | 43.8 | 49.2 | 27.2 | 45.1 | 36.4 | 30.3 | 34.6 | 37.4
RPCL |  DAF [5] \+ RP | 32.7 | 41.3 | 44.5 | 20.6 | 39.5 | 28.0 | 27.8 | 35.3 | 33.7
DAF [5] \+ CL | 33.8 | 43.0 | 44.7 | 24.3 | 38.3 | 10.9 | 30.5 | 39.4 | 33.1
DAF [5] \+ RP + CL | 34.2 | 47.1 | 49.0 | 25.1 | 37.7 | 13.4 | 33.9 | 38.9 | 34.9
SWDA [43] \+ RP | 39.8 | 37.8 | 48.1 | 32.0 | 32.9 | 41.6 | 31.8 | 25.3 | 36.2
SWDA [43] \+ CL | 41.8 | 34.3 | 47.7 | 30.8 | 33.2 | 43.1 | 34.5 | 28.3 | 36.7
SWDA [43] \+ RP + CL | 47.6 | 35.0 | 49.4 | 33.8 | 33.6 | 44.5 | 31.8 | 28.3 | 38.0
Table 1: Results of adapting Cityscapes to Foggy Cityscapes. “RP” and “CL”
stand for the proposed rotation prediction task and the consistency learning
task, respectively. “$\textrm{DAF}^{*}$” indicates the results reported in the
paper, while “DAF” represents the reimplemented results. “Source only” stands
for training the detection model using source domain data without adaptation.
The best results are in bold.

We apply the same consistency learning task for every target image
$\textbf{t}\in\mathcal{X}_{t}$ as well. So, the learning objective for the
consistency learning task is as follows:

$L_{cl}(\mathcal{X}_{s},\mathcal{X}_{t})=\frac{1}{|\mathcal{X}_{s}|}\sum_{\textbf{s}\sim\mathcal{X}_{s}}L^{s}_{cl}+\frac{1}{|\mathcal{X}_{t}|}\sum_{\textbf{t}\sim\mathcal{X}_{t}}L^{t}_{cl}.$
(5)

Analysis: There are several merits of learning the above consistency learning
task for the XDD problem. First, it introduces a form of consistency
regularization, enforcing the model to be insensitive to the image
perturbations and hence being stronger in detecting objects for unlabeled
target images. Second, we generate pseudo labels for unlabeled target data and
the pseudo labels share the same label space as the labeled source data. This
facilitates label propagation from the labeled source domain to the unlabeled
target domain. Third, we augment images with RandAugment [9], which applies
various image processing transformations. These transformations and their
combinations can model a wide range of factors that cause domain shifts. By
training the detection model to be resistant with these factors, the
generalizability of the model is accordingly enhanced.

###  3.4 Overall Learning Objective

Adding the learning objectives for the two tasks upon Eq. (1), we achieve the
learning objective of our RPCL framework as follows:

$\begin{array}[]{cl}\mathcal{L}&=\mathcal{L}_{det}(\mathcal{X}_{s},\mathcal{Y}_{s})+\alpha\mathcal{L}_{det}(\mathcal{X}_{s},\mathcal{X}_{t})+\\\
&\lambda_{1}\mathcal{L}_{rp}(\mathcal{X}_{s},\mathcal{X}_{t})+\lambda_{2}\mathcal{L}_{cl}(\mathcal{X}_{s},\mathcal{X}_{t})\end{array}$
(6)

where $\lambda_{1}$ and $\lambda_{2}$ are the hyper-parameters.

Algorithm 1 outlines the main steps of the proposed framework.

|  | aero | bike | bird | boat | bot | bus | car | cat | chair | cow | table | dog | horse | mbike | persn | plant | sheep | sofa | train | tv | mAP
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Source only | 35.6 | 52.5 | 24.3 | 23.0 | 20.0 | 43.9 | 32.8 | 10.7 | 30.6 | 11.7 | 13.8 | 6.0 | 36.8 | 45.9 | 48.7 | 41.9 | 16.5 | 7.3 | 22.9 | 32.0 | 27.8
DAF [5] | 26.0 | 58.3 | 24.0 | 23.0 | 28.1 | 44.5 | 29.4 | 10.4 | 32.0 | 39.0 | 17.5 | 15.9 | 31.1 | 58.2 | 49.3 | 44.0 | 19.1 | 19.0 | 30.6 | 43.0 | 32.1
SWDA [43] | 26.2 | 48.5 | 32.6 | 33.7 | 38.5 | 54.3 | 37.1 | 18.6 | 34.8 | 58.3 | 17.0 | 12.5 | 33.8 | 65.5 | 61.6 | 52.0 | 9.3 | 24.9 | 54.1 | 49.1 | 38.1
HTCN [3] | 33.6 | 58.9 | 34.0 | 23.4 | 45.6 | 57.0 | 39.8 | 12.0 | 39.7 | 51.3 | 21.1 | 20.1 | 39.1 | 72.8 | 63.0 | 43.1 | 19.3 | 30.1 | 50.2 | 51.8 | 40.3
DDMRL [26] | 25.8 | 63.2 | 24.5 | 42.4 | 47.9 | 43.1 | 37.5 | 9.1 | 47.0 | 46.7 | 26.8 | 24.9 | 48.1 | 78.7 | 63.0 | 45.0 | 21.3 | 36.1 | 52.3 | 53.4 | 41.8
ATF [20] | 41.9 | 67.0 | 27.4 | 36.4 | 41.0 | 48.5 | 42.0 | 13.1 | 39.2 | 75.1 | 33.4 | 7.9 | 41.2 | 56.2 | 61.4 | 50.6 | 42.0 | 25.0 | 53.1 | 39.1 | 42.1
RPCL | DAF [5] \+ RP | 31.0 | 57.8 | 32.4 | 25.6 | 39.0 | 54.1 | 34.5 | 9.8 | 34.0 | 31.2 | 29.0 | 9.4 | 28.5 | 62.9 | 51.1 | 43.6 | 15.8 | 27.6 | 61.9 | 43.3 | 36.1
DAF [5] \+ CR | 31.4 | 45.3 | 24.3 | 24.2 | 37.8 | 51.1 | 31.1 | 15.1 | 39.1 | 44.8 | 25.3 | 5.0 | 28.7 | 74.2 | 48.1 | 48.4 | 19.9 | 27.5 | 50.3 | 46.1 | 35.9
DAF [5] \+ RP + CL | 39.7 | 67.6 | 28.2 | 33.7 | 36.5 | 44.5 | 41.1 | 11.4 | 47.4 | 37.5 | 16.6 | 8.7 | 27.5 | 84.1 | 52.3 | 48.6 | 17.7 | 28.2 | 47.1 | 50.4 | 38.4
SWDA [43] \+ RP | 29.9 | 57.5 | 37.3 | 26.9 | 47.2 | 51.7 | 40.4 | 18.1 | 43.2 | 52.2 | 14.9 | 24.6 | 39.0 | 82.4 | 68.9 | 46.1 | 24.6 | 35.8 | 52.8 | 47.3 | 42.0
SWDA [43] \+ CL | 39.6 | 61.4 | 27.9 | 26.0 | 38.4 | 62.3 | 34.5 | 15.0 | 42.5 | 27.2 | 20.8 | 15.6 | 30.2 | 79.9 | 56.5 | 47.3 | 10.8 | 30.9 | 59.4 | 44.3 | 38.5
SWDA [43] \+ RP + CL | 46.2 | 62.6 | 36.7 | 27.9 | 47.2 | 53.9 | 40.8 | 17.3 | 42.7 | 55.2 | 18.9 | 20.2 | 37.2 | 87.6 | 70.3 | 45.2 | 29.8 | 35.3 | 52.7 | 46.4 | 43.7

Table 2: Results on adaptation from PASCAL VOC to Clipart.

##  4 Experiments

###  4.1 Datasets

Following the previous methods [43, 5], we conduct experiments on the
following adaptation datasets.

Cityscape to Foggy Cityscape. The Cityscape dataset [7] consists of $3,475$
images captured by a car-mounted camera in a urban scene. Bounding boxes of 8
classes are provided, including bus, bicycle, car, bike, person, rider, train,
and truck. Foggy Cityscape is rendered from Cityscape using the depth
information and a fog mask is applied on every image [45]. This leads to
strict pixel-to-pixel correspondence between every pair of images from the two
datasets. Following the previous method [5, 43], we use $2,975$ images as the
training set and the rest 500 as the validation set for both datasets. For
this adaptation experiment, we use VGG16 [46] as the backbone network and
pretrain it on ImageNet [10].

PASCAL VOC to Clipart. The PASCAL VOC dataset [13] combines the training and
validation sets of both PASCAL VOC 2007 and 2012, which results in $16,551$
training images. The dataset includes ground truth annotations for 20 classes.
The Clipart dataset include $1,000$ comic images from the same 20 classes as
PASCAL VOC. Following [43], all $1,000$ are used for both training and test.
Note that during training, the ground truth annotations are not used. We use
ResNet101 as the backbone and pretrained it on ImageNet.

PASCAL VOC to Watercolor. The source dataset is also PASCAL VOC, but
annotations from only 6 classes are employed for experiments: bike, bird, car,
cat, dog, and person. This is because Watercolor only has annotations for
these 6 classes. Watercolor includes $2,000$ artistic images where $1,000$ are
used for training and the other $1,000$ are used for test [23]. We use
ResNet-101 as the backbone and pretrained it on ImageNet.

###  4.2 Implementation Details

We apply the proposed two tasks as plug-and-play components on the two
different existing XDD models, DAF111We use the PyTorch reimplementation in
https://github.com/tiancity-NJU/da-faster-rcnn-PyTorch, which gets higher mAP
than that reported in the paper for the adaptation from Cityscape to Foggy
Cityscape. We will compare with both the re-implemented one and the reported
one, when available; otherwise, we will only compare with the reimplemented
results. [5] and SWDA [43]. For fair comparison, we do not change any model-
specific hyperparameters, e.g., the learning rate, the optimizer, training
epochs, etc. Please refer to the original papers for the implementation
details. Specific to our RPCL framework, the implementation details are as
follows. When VGG16 is used as the backbone, the rotation prediction branch is
structurally identical to the last three FC layers in the standard VGG16
network, except the output dimension of the last FC layer is 4. When the
backbone is ResNet-101, we use a lighter architecture for the rotation
prediction branch to save GPU memory. The structure is “Conv3 $\rightarrow$
ReLU $\rightarrow$ Conv1 $\rightarrow$ ReLU”. We use mean pooling over the
output feature map to get a vector representation for each proposal, which is
then used for rotation prediction. We set the hyper-parameters
$\lambda_{1}=0.1$ and $\lambda_{2}=0.1$ in the overall loss function Eq. (6)
for all our experiments222The hyper-parameter $\alpha$ is not introduced by
our framework. It varies in different XDD methods. We keep it unchanged when
implementing RPCL.. For the threshold $\sigma$ in Eq. (4), we set it as
$\sigma=0.8$ for all our experiments.

|  | bike | bird | car | cat | dog | person | mAP
---|---|---|---|---|---|---|---|---
Source only | 68.8 | 46.8 | 37.2 | 32.7 | 21.3 | 60.7 | 44.6
DAF [5] | 89.6 | 45.3 | 37.5 | 25.5 | 24.4 | 47.9 | 45.0
WST-BSR [25] | 75.6 | 45.8 | 49.3 | 34.1 | 30.3 | 64.1 | 49.9
MAF [19] | 73.4 | 55.7 | 46.4 | 36.8 | 28.9 | 60.8 | 50.3
SWDA [43] | 82.3 | 55.9 | 46.5 | 32.7 | 35.5 | 66.7 | 53.3
ATF [20] | 78.8 | 59.9 | 47.9 | 41.0 | 34.8 | 66.9 | 54.9
RPCL | DAF [5] \+ RP | 88.7 | 50.5 | 40.9 | 32.3 | 32.9 | 55.1 | 50.1
DAF [5] \+ CL | 76.3 | 46.4 | 57.0 | 37.2 | 26.7 | 62.7 | 51.0
DAF [5] \+ PR + CL | 89.2 | 53.7 | 47.2 | 42.6 | 29.2 | 64.0 | 54.3
SWDA [43] \+ RP | 79.2 | 54.9 | 46.6 | 47.4 | 44.9 | 70.4 | 57.2
SWDA [43] \+ CL | 88.9 | 53.7 | 49.5 | 43.6 | 36.6 | 69.3 | 56.9
SWDA [43] \+ RP + CL | 84.3 | 57.7 | 50.1 | 44.1 | 44.7 | 73.2 | 59.0

Table 3: Results on adpatation from PASCAL VOC to Watercolor.

###  4.3 Experimental Results

We report the results of applying RPCL on DAF [5] and SWDA [43]. For each of
the two methods, we evaluate the performance of adding the proposed Rotation
Prediction (RP) task and Consistency Learning (CL) task, both individually and
combined. By doing so, we can see the impact of each task and how they
complement each other. Apart from the two baseline methods, we also compare
with some very recent XDD algorithms to show how far we have advanced the
baselines towards the state-of-the-art. It is worth noting that RPCL is
orthogonal to existing XDD methods on improving the adaptation performance. We
implement RPCL on top of DAF [5] and SWDA [43] for their popularity. RPCL has
the potential of reaching even better performance if other more recent
algorithms are incorporated into the framework.

Cityscape to Foggy Cityscape. We can see from Table 1 that RPCL significantly
improves the results of the two baseline methods. It raises the mAP of SWDA
from 34.3 to 36.2 with the RP task applied, to 36.7 with the CL task applied,
and further to 38.0 with both tasks jointly applied. Similarly, DAF is
promoted from 31.9 to 33.7, 33.1 and 34.9 with RP, CL, and their combination,
respectively. These results substantiate the effectiveness of RPCL on
enhancing the adaptation performance, as well as the contributing role of each
of the components. With the advancement, the gaps between the baselines to the
state-of-the-art have been significantly narrowed or resolved. RPCL lifts the
performance of SWDA to a level even better than the very recent algorithm ECR-
SWDA [51], which convincingly validates the effectiveness.

| SWDA | SWDA + ImgRot | SWDA + PropRot
---|---|---|---
person | 29.9 | 40.8 | 39.8
rider | 42.3 | 35.3 | 37.8
car | 43.5 | 47.8 | 48.1
truck | 24.5 | 27.9 | 32.0
bus | 36.2 | 32.5 | 32.9
train | 32.6 | 42.7 | 41.6
mbike | 30.0 | 26.6 | 31.8
bicycle | 35.3 | 23.2 | 25.3
mAP | 34.3 | 34.6 | 36.2
Table 4:  Analysis of rotation prediction based on entire images (ImgRot)
versus that based on region proposals (PropRot).

PASCAL VOC to Clipart. Table 2 shows the results where our approach also
improves the mAP scores of SWDA and DAF on the PASCAL VOC to Clipart task.
RPCL enhances the performance of the two baselines with each of its two
components independently and in combination. In particular, RPCL raises SWDA
from 38.1 to 43.7 for the mAP, which is even higher than the state-of-the-art
result. Different from the adaptation from Cityscape to Foggy Cityscape where
the domains are similar, there is severe domain gap between the real dataset
PASCAL VOC and comic dataset Clipart. We can see that RPCL can address the
severe domain gap and significantly enhance the adaptation performance.

Figure 4: Parameter analysis of the proposed RPCL on top of SWDA on the
adaptation from Cityscape and Foggy Cityscape. Figure 5: t-SNE visualization
of the ground truth box features of the car and person categories from
Cityscape and Foggy Cityscape. Shapes “+” and “$\circ$” represent the two
classes. Features from the same domain are drawn with the same color.

PASCAL VOC to Watercolor. We can further see the significant advantage of the
proposed RPCL in Table 3. RPCL raises the mAP of DAF by more than 9 points
from 45.0 to 54.3, and raises the mAP of SWDA by nearly 6 points from 53.3 to
59.0, which is about 4 point improvement over the state-of-the-art
performance. One possible reason for the large improvement is that there are
less classes in this adaptation experiment, and thus the ambiguity between
similar classes (e.g., car and bus) is not very strong. An effective alignment
may lead to significant boost for the performance when the ambiguity is
successfully addressed.

Figure 6: Detection samples. The first and second rows shows the results of
SWDA [43] and RPCL on top of SWDA, respectively.

###  4.4 Further Analysis

Rotation prediction from image vs. from proposals. One of the major
differences of our rotation prediction task from the existing ones is that we
predict the rotation angle based on features of region proposals extracted
from an image, rather than the feature of the entire image. The merit is that
this can encourage the model to extract region proposals from foreground
regions and thus enhance detection performance. To validate this, we implement
the image-based rotation prediction task and train SWDA jointly with this
task. Table 4 shows the comparison on the adaption experiment from Cityscape
to Foggy Cityscape. We can see that the image-based rotation prediction task
(ImgRot) produces only a marginal improvement, which is far lower than our
proposal-based rotation prediction task. This comparison verifies that the
foreground regions are indeed more activated when the model is trained to
extracted region proposals that facilitate to predict the rotation angle.

Parameter analysis. We conducted experiments to evaluate the sensitivity of
the proposed method with respect to the hyper-parameters, i.e., $\lambda_{1}$
and $\lambda_{2}$. When evaluating one parameter, we vary its value and fix
the other parameter unchanged. Figure 4 shows the results of RPCL-SWDA, i.e.,
RPCL on top of SWDA, for the adaptation from Cityscape to Foggy Cityscape. We
can see that RPCL is quite robust with $\lambda_{2}$ \- the performance is
stable when $\lambda_{2}$ varies in a wide range. RPCL is more sensitive to
$\lambda_{1}$ and the performance drops to 0 when $\lambda_{1}$ is greater
tnan 1. This is because the model fails to converge when the rotation
prediction loss is weighted too much.

Feature visualization. To qualitatively evaluate the alignment results, we
plot in Figure 5 the t-SNE [33] visualization of the instance features
obtained by applying RoIAlign on the ground truth instances from Cityscape and
Foggy Cityscape. The features extracted by the RPCL-SWDA model for the car and
person classes from both domains are shown. We can see that the source
features and target features are closely aligned, while features from
different classes are separated.

Detection samples. Figure 6 shows some detection samples from the Clipart
dataset using the RPCL-SWDA [43]. As a comparison, we also show the detection
results of SWDA on the same images. We can see from the figure that RPCL-SWDA
produces fewer false negatives (real objects but not detected) and false
positives (objects detected but not real). This further validates the efficacy
of RPCL on improving the performance of SWDA.

##  5 Conclusions

We introduce in this paper the RPCL framework which can improve the
performance of different existing Cross-Domain Detection (XDD) methods through
the two introduced auxiliary tasks: the rotation prediction task and the
consistency learning task. The rotation prediction task encourages the
detection model to extract region proposals from the foreground. This benefits
both the detection task and domain alignment. The second task encourages the
model to make smooth class predictions for region proposals when the input
image has been applied with various transformations that model domain shifts.
Thus, the learned model should have enhanced generalizability on the target
domain. The experiments show that each of the two tasks contributes to
performance gains for different XDD methods, and the tasks complement each
other, pushing the baseline methods towards new state-of-the-art results.

## References

  * [1] David Berthelot, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel.  Remixmatch: Semi-supervised learning with distribution matching and augmentation anchoring.  In ICLR, 2019.
  * [2] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel.  Mixmatch: A holistic approach to semi-supervised learning.  arXiv preprint arXiv:1905.02249, 2019.
  * [3] Chaoqi Chen, Zebiao Zheng, Xinghao Ding, Yue Huang, and Qi Dou.  Harmonizing transferability and discriminability for adapting object detectors.  In CVPR, 2020.
  * [4] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton.  A simple framework for contrastive learning of visual representations.  In ICML, 2020.
  * [5] Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Domain adaptive faster r-cnn for object detection in the wild.  In CVPR, 2018.
  * [6] Kevin Clark, Minh-Thang Luong, Christopher D Manning, and Quoc V Le.  Semi-supervised sequence modeling with cross-view training.  arXiv preprint arXiv:1809.08370, 2018.
  * [7] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele.  The cityscapes dataset for semantic urban scene understanding.  In CVPR, 2016.
  * [8] Ekin D Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V Le.  Autoaugment: Learning augmentation strategies from data.  In CVPR, 2019.
  * [9] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le.  Randaugment: Practical data augmentation with no separate search.  arXiv preprint arXiv:1909.13719, 2019.
  * [10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In CVPR, 2009.
  * [11] Carl Doersch, Abhinav Gupta, and Alexei A Efros.  Unsupervised visual representation learning by context prediction.  In ICCV, 2015.
  * [12] Alexey Dosovitskiy, Jost Tobias Springenberg, Martin A Riedmiller, and Thomas Brox.  Discriminative unsupervised feature learning with convolutional neural networks.  In NeurIPS, 2014.
  * [13] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman.  The pascal visual object classes (voc) challenge.  International Journal of Computer Vision, 88(2):303–338, 2010.
  * [14] Spyros Gidaris, Praveer Singh, and Nikos Komodakis.  Unsupervised representation learning by predicting image rotations.  In ICLR, 2018.
  * [15] Ross Girshick.  Fast r-cnn.  In ICCV, 2015.
  * [16] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik.  Rich feature hierarchies for accurate object detection and semantic segmentation.  In CVPR, 2014.
  * [17] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In CVPR, 2020.
  * [18] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick.  Mask r-cnn.  In ICCV, 2017.
  * [19] Zhenwei He and Lei Zhang.  Multi-adversarial faster-rcnn for unrestricted object detection.  In ICCV, 2019.
  * [20] Zhenwei He and Lei Zhang.  Domain adaptive object detection via asymmetric tri-way faster-rcnn.  arXiv preprint arXiv:2007.01571, 2020.
  * [21] Daniel Ho, Eric Liang, Xi Chen, Ion Stoica, and Pieter Abbeel.  Population based augmentation: Efficient learning of augmentation policy schedules.  In ICML, 2019.
  * [22] Cheng-Chun Hsu, Yi-Hsuan Tsai, Yen-Yu Lin, and Ming-Hsuan Yang.  Every pixel matters: Center-aware feature alignment for domain adaptive object detector.  In ECCV, 2020.
  * [23] Naoto Inoue, Ryosuke Furuta, Toshihiko Yamasaki, and Kiyoharu Aizawa.  Cross-domain weakly-supervised object detection through progressive domain adaptation.  In CVPR, 2018.
  * [24] Mehran Khodabandeh, Arash Vahdat, Mani Ranjbar, and William G Macready.  A robust learning approach to domain adaptive object detection.  In ICCV, 2019.
  * [25] Seunghyeon Kim, Jaehoon Choi, Taekyung Kim, and Changick Kim.  Self-training and adversarial background regularization for unsupervised domain adaptive one-stage object detection.  In ICCV, 2019.
  * [26] Taekyung Kim, Minki Jeong, Seunghyeon Kim, Seokeon Choi, and Changick Kim.  Diversify and match: A domain adaptive representation learning paradigm for object detection.  In CVPR, 2019.
  * [27] Alexander Kolesnikov, Xiaohua Zhai, and Lucas Beyer.  Revisiting self-supervised visual representation learning.  In CVPR, 2019.
  * [28] D. Zhao etal L. Xiao, J. Xu.  Self-supervised domain adaptation with consistency training.  In ICPR, 2020.
  * [29] Samuli Laine and Timo Aila.  Temporal ensembling for semi-supervised learning.  In ICLR, 2017.
  * [30] Gustav Larsson, Michael Maire, and Gregory Shakhnarovich.  Colorization as a proxy task for visual understanding.  In CVPR, 2017.
  * [31] Kai Li, Curtis Wigington, Chris Tensmeyer, Handong Zhao, Nikolaos Barmpalios, Vlad I Morariu, Varun Manjunatha, Tong Sun, and Yun Fu.  Cross-domain document object detection: Benchmark suite and method.  In CVPR, 2020.
  * [32] Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg.  Ssd: Single shot multibox detector.  In ECCV, 2016.
  * [33] Laurens van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  Journal of Machine Learning Research, 9(Nov):2579–2605, 2008.
  * [34] Takeru Miyato, Shin-ichi Maeda, Masanori Koyama, and Shin Ishii.  Virtual adversarial training: a regularization method for supervised and semi-supervised learning.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(8):1979–1993, 2018.
  * [35] Mehdi Noroozi and Paolo Favaro.  Unsupervised learning of visual representations by solving jigsaw puzzles.  In ECCV, 2016.
  * [36] Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros.  Context encoders: Feature learning by inpainting.  In CVPR, 2016.
  * [37] Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi.  You only look once: Unified, real-time object detection.  In CVPR, 2016.
  * [38] Joseph Redmon and Ali Farhadi.  Yolo9000: better, faster, stronger.  In CVPR, 2017.
  * [39] Joseph Redmon and Ali Farhadi.  Yolov3: An incremental improvement.  arXiv preprint arXiv:1804.02767, 2018.
  * [40] Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun.  Faster r-cnn: Towards real-time object detection with region proposal networks.  In NuerIPS, 2015.
  * [41] Aruni RoyChowdhury, Prithvijit Chakrabarty, Ashish Singh, SouYoung Jin, Huaizu Jiang, Liangliang Cao, and Erik Learned-Miller.  Automatic adaptation of object detectors to new domains using self-training.  In CVPR, 2019.
  * [42] Kuniaki Saito, Donghyun Kim, Stan Sclaroff, and Kate Saenko.  Universal domain adaptation through self supervision.  In NeurIPS, 2020.
  * [43] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada, and Kate Saenko.  Strong-weak distribution alignment for adaptive object detection.  In CVPR, 2019.
  * [44] Mehdi Sajjadi, Mehran Javanmardi, and Tolga Tasdizen.  Regularization with stochastic transformations and perturbations for deep semi-supervised learning.  In NeurIPS, 2016.
  * [45] Christos Sakaridis, Dengxin Dai, and Luc Van Gool.  Semantic foggy scene understanding with synthetic data.  International Journal of Computer Vision, 126(9):973–992, 2018\.
  * [46] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  arXiv preprint arXiv:1409.1556, 2014.
  * [47] Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel.  Fixmatch: Simplifying semi-supervised learning with consistency and confidence.  In NeurIPS, 2020.
  * [48] Jong-Chyi Su, Subhransu Maji, and Bharath Hariharan.  When does self-supervision improve few-shot learning?  In ECCV, 2020.
  * [49] Yu Sun, Eric Tzeng, Trevor Darrell, and Alexei A Efros.  Unsupervised domain adaptation through self-supervision.  arXiv preprint arXiv:1909.11825, 2019.
  * [50] Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, and Quoc V Le.  Unsupervised data augmentation for consistency training.  In NeurIPS, 2020.
  * [51] Chang-Dong Xu, Xing-Ran Zhao, Xin Jin, and Xiu-Shen Wei.  Exploring categorical regularization for domain adaptive object detection.  In CVPR, 2020.
  * [52] Jiaolong Xu, Liang Xiao, and Antonio M López.  Self-supervised domain adaptation for computer vision tasks.  IEEE Access, 7:156694–156706, 2019.
  * [53] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz.  mixup: Beyond empirical risk minimization.  In ICLR, 2018.
  * [54] Richard Zhang, Phillip Isola, and Alexei A Efros.  Colorful image colorization.  In ECCV, 2016.
  * [55] Richard Zhang, Phillip Isola, and Alexei A Efros.  Split-brain autoencoders: Unsupervised learning by cross-channel prediction.  In CVPR, 2017.
  * [56] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros.  Unpaired image-to-image translation using cycle-consistent adversarial networks.  In ICCV, 2017.
  * [57] Xinge Zhu, Jiangmiao Pang, Ceyuan Yang, Jianping Shi, and Dahua Lin.  Adapting object detectors via selective cross-domain alignment.  In CVPR, 2019.