# Real-World Image Super Resolution via Unsupervised Bi-directional Cycle
Domain Transfer Learning based Generative Adversarial Network

Xiang Wang, Yimin Yang, Zhichang Guo, Zhili Zhou, , Yu Liu, , Qixiang Pang,
Shan Du X. Wang is with the Dept. of Computer Science, Lakehead University,
Thunder Bay, ON, Canada, e-mail<EMAIL_ADDRESS>Yang is with the
Dept. of electrical and computer engineering, Western University, London, ON,
Canada, e-mail<EMAIL_ADDRESS>Guo is with the School of Mathematics,
Harbin Institute of Technology, Hei Long Jiang, China, e-mail:
<EMAIL_ADDRESS>Zhou is with the School of Computer and Software, Nanjing
University of Information Science and Technology, Jiangsu, China,
<EMAIL_ADDRESS>Liu is with the School of Microelectronics, Tianjin
University, Tianjin, China, e-mail<EMAIL_ADDRESS>Pang is with the
Okanagan College, Kelowna, BC, Canada, e-mail<EMAIL_ADDRESS>Du is
with the Dept. of Comp. Sci., Math, Physics & Statistics, The University of
British Columbia Okanagan, BC, Canada, e-mail<EMAIL_ADDRESS>
###### Abstract

Deep Convolutional Neural Networks (DCNNs) have exhibited impressive
performance on image super-resolution tasks. However, these deep learning-
based super-resolution methods perform poorly in real-world super-resolution
tasks, where the paired high-resolution and low-resolution images are
unavailable and the low-resolution images are degraded by complicated and
unknown kernels. To break these limitations, we propose the Unsupervised Bi-
directional Cycle Domain Transfer Learning-based Generative Adversarial
Network (UBCDTL-GAN), which consists of an Unsupervised Bi-directional Cycle
Domain Transfer Network (UBCDTN) and the Semantic Encoder guided Super
Resolution Network (SESRN). First, the UBCDTN is able to produce an
approximated real-like LR image through transferring the LR image from an
artificially degraded domain to the real-world LR image domain. Second, the
SESRN has the ability to super-resolve the approximated real-like LR image to
a photo-realistic HR image. Extensive experiments on unpaired real-world image
benchmark datasets demonstrate that the proposed method achieves superior
performance compared to state-of-the-art methods.

###### Index Terms:

Deep convolutional neural networks, Image super-resolution, Unsupervised
manner, Real-world scenes

Figure 1: The overview of the proposed UBCDTL-GAN: In the first stage (left),
the green dot rectangle represents the Unsupervised Bi-directional Cycle
Domain Transfer Network (UBCDTN). The red path indicates the forward cycle
module. Given the input HR image $I^{HR}$, $I^{LR}_{degraded}$ is the
artificially degraded LR image, and $\widehat{I}^{LR}_{real}$ is the real-like
LR image generated by a generator $G_{A}$. $\widehat{I}^{LR}_{recon}$
represents the reconstructed image of $I^{LR}_{degraded}$ produced by another
generator $G_{B}$ from $\widehat{I}^{LR}_{real}$. $\widehat{I}^{LR}_{idt}$ is
produced by $G_{B}$ from the original $I^{LR}_{degraded}$. In addition,
$L^{adv}_{D_{B}}$, $L^{idt}_{degraded}$, $L^{cyc}_{G_{B}}$ and
$L^{percep}_{FE_{A}}$ depicted in red dotted line represent adversarial loss,
identity loss, cycle-consistency loss and cycle-perceptual loss for forward
cycle module. $D_{A}$ is a discriminator and $FE_{A}$ denotes a feature
extractor. Symmetrically, the blue path shows the backward cycle module, where
$I^{LR}_{real}$ is given by a real-world dataset and synthesized LR image
$I^{LR}_{syn}$ is generated by $G_{B}$. Moreover, the $G_{A}$ is able to
translate $I^{LR}_{syn}$ back to reconstructed real-world LR image
$L^{LR}_{recon}$ and generate the identity real-world LR image $I^{LR}_{idt}$.
The blue dotted line represents the adversarial loss $L^{adv}_{D_{A}}$,
identity loss $L^{idt}_{real}$, cycle-consistency loss $L^{cyc}_{G_{A}}$ and
cycle-perceptual loss $L^{percept}_{FE_{B}}$ for backward cycle module
respectively. $D_{B}$ is a discriminator and $FE_{B}$ denotes a feature
extractor. In the second stage (right), the framework of a Semantic Encoder
guided Super Resolution Network (SESRN) is depicted in a yellow dot rectangle,
where it consists of Semantic Encoder $SE$, Generator $G_{SR}$, Joint
Discriminator $D_{SR}$ and Content Extractor $\phi$. There are two paths in
the SESRN, where the red path indicates the process of the real tuple and the
blue path is the process of the fake tuple. $I^{SR}$ is the SR image from
$G_{SR}$. Furthermore, $SE(\cdot)$ denotes the embedded semantics obtained
from $SE$. $D(\cdot)$ represents the output probability of $D_{SR}$.
$\phi(\cdot)$ describes the features learned by $\phi$.

##  I Introduction

Single Image Super-Resolution (SISR) aims to reconstruct a High-Resolution
(HR) image from a single Low-Resolution (LR) image. It has been widely applied
in many computer vision applications, such as surveillance [1], image
enhancement [2] and medical image processing [3]. In the SISR task, the
general degradation formula is expressed as:

$y=(x\bigotimes k)\downarrow s+n$ (1)

where $x$ represents the HR image, $y$ is the degraded LR image, $k$ denotes a
blur kernel, and $\bigotimes$ is the convolution operation performed on $x$
and $k$. $\downarrow s$ denotes a downsampling operation on the image with a
scaling factor $s$, and $n$ is considered as an additive white Gaussian noise.
However, under the real-world scene setting, $k$ is unknown and $n$ should
take into account many possible conditions such as sensor noise, compression
artifacts, and unpredicted noise caused by physical devices. Resulting from
the existence of uncertain kernel $k$ and noise $n$, SISR has become a
particularly ill-posed inverse task since there are infinite HR images that
can be recovered from a given LR image, in which it is required to select the
most plausible solution.

Recently, a great number of deep learning-based SISR models have been
proposed, such as RDN [4], EDSR [5], SRDenseNet [6], and ESRGAN [7]. Despite
the successful progress achieved by the aforementioned methods, there still
exist unnoticed issues. Those models were trained in a supervised manner with
a large number of paired images, synthesized LR images produced by pre-
determined degradation and its HR counterpart, resulting in deteriorative
performance when they are applied to real-world scenarios. The reason is that
paired LR-HR data is unavailable and the degradation of the input LR image is
unknown in the real-world scene. Besides, it is unreasonable to simply apply
the LR images downsampled by the ideally fixed kernel to the real-world SR
problems [8, 9]. There exists a large domain gap between real-world LR images
and artificially synthesized LR images. In addition, the synthesized LR images
may eliminate diverse patterns and complicated characteristics belonging to
real-world LR images such as sensor noise and natural characteristics. Thus,
the existing SR methods normally encounter a serious domain consistency
problem and produce poor performance in practical scenarios [10, 11].

Therefore, it is imperative to explore an effective method that can apply
unpaired images to satisfy the need for real-world SR scenarios. It must be
different from the aforementioned SR methods which do not take into
consideration the domain gap between the LR images generated from a known
degradation (e.g., bicubic downscaling) and the real-world LR images. To
address the above limitations, we propose an Unsupervised Bi-directional Cycle
Domain Transfer Learning-based Generative Adversarial Network (UBCDTL-GAN) for
real-world image super-resolution, which is composed of two networks,
Unsupervised Bi-directional Cycle Domain Transfer Network (UBCDTN) and
Semantic Encoder guided Super Resolution Network (SESRN). To simulate the
real-world data distribution and reduce the domain gap between the generated
LR image domain and the real-world LR image domain, we design UBCDTN to
estimate the inherent degradation kernel from the real-world LR distribution
and translate the artificially degraded LR domain image to the real-world
domain. With the help of the cycle consistency mechanism [12], the proposed
UBCDTN is able to learn bi-directional inverse mapping in an unsupervised
manner, which can ensure the generated real-like LR image preserves desired
characteristics of real-world patterns. Besides, we also enforce auxiliary
constraints on the UBCDTN such as adversarial loss, identity loss, and cycle-
perceptual loss. The designed domain transfer network provides an effective
way to generate real-like LR images, which can construct the paired real-world
LR-HR data for the following SESRN. For the second step, we design the
Semantic Encoder guided Super Resolution Network (SESRN). The goal of the
SESRN is to super-resolve the real-like LR images to the photo-realistic HR
images. It is emphasized that we apply our previously proposed architecture
and optimization strategy of GAFHH-RIDN [13] to the generator and
discriminator of SESEN. We evaluate our method on the NTIRE 2020 Super
Resolution Challenge Track 1 validation dataset. The quantitative and
qualitative comparisons demonstrate the superiority of the proposed UBCDTL-GAN
compared with other state-of-the-art methods. A comparison of visual results
with various latest methods is shown in Fig. 4, and the numeric results can be
seen in Table I. The main contributions of our proposed method can be
summarized as follows:

1) We propose a novel bi-directional cycle domain transfer network, UBCDTN.
According to the domain transfer learning scheme, the designed bi-directional
cycle architecture is able to eliminate the domain gap between the
artificially degraded LR images and real-world LR images in an unsupervised
manner.

2) We further impose the auxiliary constraints on UBCDTN by incorporating
adversarial loss, identity loss, and cycle-perceptual loss, which can
guarantee that the generated real-like LR images contain the same style as
real-world images.

3) We design the SESRN as a deep super-resolution network to generate visually
pleasant SR results under supervised learning settings.

4) Benefiting from the collaborative training strategy, the proposed UBCDTL-
GAN is able to be trained in an end-to-end fashion, which can ease the entire
training procedure and strengthen the robustness of the model.

Figure 2: The proposed SESRN and its components: Semantic Encoder $SE$,
Generator $G_{SR}$, Joint Discriminator $D_{SR}$ and Content Extractor $\phi$.
For $D_{SR}$, ESLDSN represents the Embedded Semantics-Level Discriminative
Sub-Net, ILDSN represents the Image-Level Discriminative Sub-Net, and FCM
denotes the Fully Connected Module. As for the generator $G_{SR}$, there are
three stages: Shallow Feature Module (SFM), Multi-level Dense Block Module
(MDBM), and Upsampling Module (UM). $I^{HR}$ and $\widehat{I}^{LR}_{real}$
denote HR images and LR images respectively. $I^{SR}$ is SR images from
$G_{SR}$. Furthermore, $SE(\cdot)$ denotes the embedded semantics obtained
from the $SE$. $D_{SR}(\cdot)$ represents the output probability of the
$D_{SR}$. $\phi(I^{HR})$ and $\phi(I^{SR})$ describes the features learned by
the content extractor $\phi$.

##  II Related Work

In this section, we first review the paired image super-resolution methods.
Then, we introduce blind and unsupervised learning methods for real-world
scenarios.

###  II-A Paired Image Super Resolution

In recent years, deep learning-based methods have exhibited exceptional
ability to enhance SISR performance. However, most of these methods rely on
supervised settings, where specific pre-defined and paired LR and HR images
are required in the training process. The pioneer method was introduced by
Dong $et\ al.$ [14], namely SRCNN. It is able to recover HR images from
bicubic downsampled LR images in an end-to-end manner. Moreover, the enhanced
deep SR method EDSR [5] was proposed by Lim $et\ al.$, which greatly enlarges
the network to be deeper and wider, resulting in learning richer feature
representations. Later, generative adversarial learning was introduced in
SISR. SRGAN [15] proposed by Ledig $et\ al.$ incorporates adversarial loss and
perceptual loss to update the generator and discriminator respectively, which
can obtain photo-realistic SR images. Furthermore, the ESRGAN [7] introduces
the new generator architecture and adopts enhanced adversarial loss, achieving
promising performance in the visual aspect.

###  II-B Blind Image Super Resolution

The blind SISR is defined as the task that supposes the LR image is produced
by an unknown degradation kernel from its HR version. The Iterative Kernel
Correction (IKC) [16] was proposed by Gu $et\ al$. to estimate blur kernel and
eliminate artifacts caused by kernel discrepancy. Zhang $et\ al$. [17]
proposed IRCNN which includes a set of CNN denoisers to estimate the blur
model. However, the results of IRCNN indicate that it is difficult to estimate
a comprehensive degradation kernel in real-world conditions. The blind SR
methods have the limited capability to approximate unknown degradations. Thus,
there is still room for improvement in dealing with blind image super-
resolution.

###  II-C Unpaired Super Resolution

Recently, many unsupervised methods have been proposed to satisfy real-world
conditions where the paired LR-HR image data is unavailable, and the input
image is corrupted by unknown degradation kernels. By employing the internal
recurrence of information inside an image, ZSSR [18] can use pseudo image
pairs to recover LR images with diverse blur kernels. Inspired by CycleGAN,
Yuan $et\ al$. [10] proposed CinCGAN, which first generates bicubic
downsampled LR images from the input and then super-resolves LR images to HR
images. However, CinCGAN only considers single bicubic degradation, resulting
in poor generalization in complicated real-world SR tasks. By contrast, our
method utilizes UBCDTN to simulate diverse real-world degradation types,
making it perform proficiently in the real world.

##  III Overview

We divide the unsupervised super-resolution problem into two stages, but it is
still in an end-to-end fashion. The proposed method comprises two stages: 1)
Unsupervised image domain translation between real-world LR images and
artificially degraded LR images. 2) Semantic encoder guided generative
adversarial super resolution from the generated real-like LR images to final
HR images in a supervised manner. To obtain the artificially degraded LR image
$I^{LR}_{degraded}$, we perform bicubic downscaling on HR image $I^{HR}$ by
image degradation block.

In the first stage, the proposed UBCDTN generates real-like LR images
belonging to the real-world domain from the artificially degraded LR images.
It aims to transfer the domain of artificially degraded images to the real-
world domain, which can ensure a similar real-world LR pattern in the
generated real-like LR images. After domain translation performed by UBCDTN,
we obtain the real-like LR $\widehat{I}^{LR}_{real}$ treated as the input of
the following SESRN. In the second stage, we utilize SESRN to learn the super-
resolution mapping from the real-like LR space to the HR space. We further
train the SESRN with adversarial loss, content loss, and pixel-wise loss to
generate photo-realistic SR images.

###  III-A Unsupervised Bi-directional Cycle Domain Transfer Network

The designed UBCDTN is able to translate the source domain $I^{LR}_{degraded}$
to the target domain $I^{LR}_{real}$. It provides an effective bi-directional
cycle solution to reduce the domain gap between $I^{LR}_{degraded}$ and
$I^{LR}_{real}$. The forward-cycle module contains $G_{A}$ where it aims to
learn a mapping $I^{LR}_{degraded}\rightarrow\widehat{I}^{LR}_{real}$. The
goal of $G_{B}$ in backward-cycle module is to learn another mapping
$I^{LR}_{real}\rightarrow I^{LR}_{syn}$.

As shown in Fig. 1, the forward-cycle module comprises generator $G_{A}$,
discriminator $D_{B}$ and Feature Extractor $FE_{A}$. The backward-cycle
module consists of $G_{B}$, $D_{A}$, and $FE_{B}$, where these networks have
the same architecture as the forward-cycle module’s corresponding components,
but for different purposes. We utilize U-Net as the basic architecture for
$G_{A}$ and $G_{B}$. $D_{A}$ and $D_{B}$ are standard convolutional neural
networks, where each is composed of nine convolutional layers followed by
BatchNormalization (BN) layers and Leaky ReLU layers. The pre-trained VGG19 is
exploited as feature extractor $FE_{A}$ and $FE_{B}$.

The UBCDTN trains two generators simultaneously, where these two generators
should be translated in bi-direction and inverted to each other. We involve
adversarial learning in both modules, where $G_{A}$ and $G_{B}$ are trained on
discriminator $D_{B}$, $D_{A}$ respectively. However, as mentioned before, the
SISR task is an ill-posed problem in which there exist many possible SR images
reconstructed from one given LR input. Thus, applying adversarial loss alone
to train UBCDTN leads to the problem of model collapse, and it cannot produce
the desired generated images which have the same characteristics and patterns
as target domain images. It is necessary to employ regularization on UBCDTN to
improve the quality of translated images. Thus, we propose a cycle-consistency
constraint to guarantee the domain correlation between
($\widehat{I}^{LR}_{real}$, $I^{LR}_{real}$) and ($I^{LR}_{syn}$,
$I^{LR}_{degraded}$). In addition, in order to avoid color variation, identity
loss is applied to preserve color composition between the input image and
output image [12]. Moreover, we introduce the cycle-perceptual loss as an
additional constraint to preserve the sharper edges and finer structures in
the reconstructed images. The following sections provide more details of the
forward-cycle module and backward-cycle module.

####  III-A1 Forward-cycle Module

The forward-cycle module contains a generator $G_{A}$, discriminator $D_{B}$
and feature extractor $FE_{A}$. The goal of $G_{A}$ is to map the artificially
degraded LR image to the target domain image, i.e.,
$\widehat{I}_{real}^{LR}=G_{A}((I^{LR}_{degraded})_{i})$. The concern of the
unpaired condition can be eliminated by adding additional constraints. We
propose the cycle-consistency to guarantee the relevant content in the
generated $\widehat{I}^{LR}_{real}$ can be preserved, i.e.,
$G_{B}(G_{A}((I^{LR}_{degraded})_{i}))\approx I^{LR}_{degraded}$. In a word,
the generated images of $G_{A}$ and $G_{B}$ should be cycle-consistent with
each other. Thus, the forward cycle consistency is established
$I^{LR}_{degraded}\rightarrow G_{A}(I^{LR}_{degraded})\rightarrow
G_{B}(G_{A}(I^{LR}_{degraded}))\approx I^{LR}_{degraded}$. The forward cycle
consistency loss $L^{cyc}_{G_{B}}$ can be defined as follows:

$\widehat{I}_{recon}^{LR}=G_{B}(G_{A}((I^{LR}_{degraded})_{i}))$ (2)
$\begin{split}L^{cyc}_{G_{B}}=\frac{1}{N}\sum_{i}^{N}||(\widehat{I}^{LR}_{recon})_{i}-(I^{LR}_{degraded})_{i}||_{1}\end{split}$
(3)

where $N$ is the number of training images. As Eqn. 2 shows, with the help of
$G_{B}$, $\widehat{I}^{LR}_{recon}$can be identical to the input
$I^{LR}_{degraded}$. Through the cycle consistency mechanism,
$I^{LR}_{degraded}$ in the source domain can be reconstructed after performing
$G_{A}$ and $G_{B}$ on $I^{LR}_{degraded}$ in turn. In addition, we apply
adversarial losses to $G_{A}$ and $D_{B}$ such that the distribution of
$\widehat{I}^{LR}_{real}$ is indistinguishable from the real distribution
$I^{LR}_{real}$. We apply the Relativistic average GAN (RaGAN) [19] to train
$G_{A}$ and $D_{B}$. $G_{A}$ receives the input $I^{LR}_{degraded}$ and
generates $\widehat{I}^{LR}_{real}$, which can fool $D_{B}$. The discriminator
$D_{B}$ aims to predict the probability that the provided real image
$I^{LR}_{real}$ is more realistic compared to the generated fake image
$\widehat{I}_{real}^{LR}$. The input of $D_{B}$ contains real data
$I^{LR}_{real}$ and fake data $\widehat{I}_{real}^{LR}$ respectively. It can
be summarized as follows:

$D(X_{real})=\sigma(C(I^{LR}_{real})-E_{x_{f}}[C(\widehat{I}_{real}^{LR})])$
(4)
$D(X_{fake})=\sigma(C(\widehat{I}_{real}^{LR})-E_{x_{r}}[C(I^{LR}_{real})])$
(5)

where $\sigma$ denotes sigmoid non-linearity, $C$ denotes the discriminator
without the final sigmoid layer, $E_{x_{f}}$, $E_{x_{r}}$ are mathematical
means of $X_{fake}$ and $X_{real}$ in the training batch respectively, and
$D(X_{real})$, $D({X_{fake}})$ are the predicted probability of
$I^{LR}_{real}$, $\widehat{I}_{real}^{LR}$ to be real by discriminator
$D_{B}$. An adversarial loss for $G_{A}$, $D_{B}$ can be expressed as
$L^{adv}_{G_{A}}$ and $L^{adv}_{D_{B}}$ respectively as follows:

$\begin{split}L^{adv}_{G_{A}}=&-\mathbb{E}_{I^{LR}_{real}}\sim
p_{(I^{LR}_{real})}[log(1-D(X_{real}))]\\\
&-\mathbb{E}_{\widehat{I}_{real}^{LR}}\sim
p_{(\widehat{I}_{real}^{LR})}[log(D(X_{fake}))]\end{split}$ (6)
$\begin{split}L^{adv}_{D_{B}}=&-\mathbb{E}_{I^{LR}_{real}}\sim
p_{(I^{LR}_{real})}[log(D(X_{real}))]\\\
&-\mathbb{E}_{\widehat{I}_{real}^{LR}}\sim
p_{(\widehat{I}_{real}^{LR})}[log(1-D(X_{fake}))]\end{split}$ (7)

where $\mathbb{E}_{I^{LR}_{real}}\sim p_{(I^{LR}_{real})}$ and
$\mathbb{E}_{\widehat{I}_{real}^{LR}}\sim p_{(\widehat{I}_{real}^{LR})}$
indicates the distribution of real image $I^{LR}_{real}$, and fake image
$\widehat{I}^{LR}_{real}$ respectively. We further introduce forward identity
loss $L^{idt}_{degraded}$ to maintain color composition between
$\widehat{I}^{LR}_{idt}$ and $I^{LR}_{degraded}$. The $\widehat{I}^{LR}_{idt}$
can be generated by $G_{B}$, as Eqn. 8 shows. The $L^{idt}_{degraded}$ is
expressed as Eqn. 9:

$\widehat{I}_{idt}^{LR}=G_{B}((I^{LR}_{degraded})_{i})$ (8)
$L^{idt}_{degraded}=\frac{1}{N}\sum_{i}^{N}||(\widehat{I}^{LR}_{idt})_{i}-(I^{LR}_{degraded})_{i}||_{1}$
(9)

Moreover, in order to minimize the perceptual divergence between
$\widehat{I}^{LR}_{recon}$ and $I^{LR}_{degraded}$, we utilize $FE_{A}$ to
extract VGG feature maps for cycle-perceptual loss estimation. It can be
defined as follows:

$\begin{split}L^{percep}_{FE_{A}}=\frac{1}{N}\sum_{i}^{N}||FE_{q,r}((\widehat{I}^{LR}_{recon})_{i})-FE_{q,r}((I^{LR}_{degraded})_{i})||_{2}\end{split}$
(10)

where $FE_{q,r}(\cdot)$ indicates the extracted feature maps of $q$-th
convolution layer (after activation layer) before $r$-th maxpooling layer. The
total objective loss for forward cycle module $L^{Forward}_{total}$ is a
weighted sum of the four loss functions:

$\begin{split}L^{Forward}_{total}=\omega_{1}L^{adv}_{G_{A}}+\omega_{2}L^{cyc}_{G_{B}}+\omega_{3}L^{idt}_{degraded\\_LR}+\omega_{4}L^{percep}_{FE_{A}}\end{split}$
(11)

where the hyper-parameters $\omega_{1}$, $\omega_{2}$, $\omega_{3}$,
$\omega_{4}$ are trade-off factors for each loss. We empirically set
$\omega_{1}$, $\omega_{2}$, $\omega_{3}$, and $\omega_{4}$ to 1, 10, 1 and 1
respectively. It is noticeable that the loss with high weights indicates a
significant proportion of the training process.

####  III-A2 Backward-cycle Module

To transfer the image from the target domain to the source domain, i.e.,
$I^{LR}_{real}\rightarrow I^{LR}_{syn}$, we specifically construct a backward-
cycle module in which the generator $G_{B}$ is able to learn the mapping
$G_{B}(I^{LR}_{real})\approx I^{LR}_{syn}$. The learned mapping is capable of
re-expressing the target domain image $I^{LR}_{real}$ by the source domain
image $I^{LR}_{syn}$ implicitly. The backward-cycle module consists of a
generator $G_{B}$, discriminator $D_{A}$ and feature extractor $FE_{B}$. There
are several constraints required in the backward-cycle module. First, we
symmetrically design the backward cycle consistency constraint, i.e.,
$G_{A}(G_{B}((I^{LR}_{real})_{i}))\approx I^{LR}_{real}$, which further forces
the contents of generated images to be relevant to the input ones. Thus, the
backward cycle is composed of $I^{LR}_{real}\rightarrow
G_{B}(I^{LR}_{real})\rightarrow G_{A}(G_{B}(I^{LR}_{real}))\approx
I^{LR}_{real}$. The backward cycle consistency loss can be defined as follows:

$I_{recon}^{LR}=G_{A}(G_{B}((I^{LR}_{real})_{i}))$ (12)
$L^{cyc}_{G_{A}}=\frac{1}{N}\sum_{i}^{N}||(I^{LR}_{recon})_{i})-(I^{LR}_{real})_{i}||_{1}$
(13)

As Eqn. 12 shows, because of the formulated backward cycle scheme, $G_{B}$ and
$G_{A}$ are capable of inverting the reconstructed images $I^{LR}_{recon}$
back to the $I^{LR}_{real}$. The Eqn. 13 is able to minimize the correlation
discrepancy between $I^{LR}_{recon}$ and $I^{LR}_{real}$ such that the
consistent characteristics can be preserved. Similarly, as the forward cycle
module, we also adopt adversarial learning in the backward cycle module, where
the generator $G_{B}$ and discriminator $D_{A}$ are optimized by RaGAN. The
adversarial loss $L^{adv}_{G_{B}}$ and $L^{adv}_{D_{A}}$ are defined as:

$D(X_{real})=\sigma(C(I^{LR}_{degraded})-E_{x_{f}}[C(I_{syn}^{LR})])$ (14)
$D(X_{fake})=\sigma(C(I_{syn}^{LR})-E_{x_{r}}[C(I^{LR}_{degraded})])$ (15)
$\begin{split}L^{adv}_{G_{B}}=&-\mathbb{E}_{I^{LR}_{degraded}}\sim
p_{(I^{LR}_{degraded})}[log(1-D(X_{real}))]\\\ &-\mathbb{E}_{I_{syn}^{LR}}\sim
p_{(I_{syn}^{LR})}[log(D(X_{fake}))]\end{split}$ (16)
$\begin{split}L^{adv}_{D_{A}}=&-\mathbb{E}_{I^{LR}_{degraded}}\sim
p_{(I^{LR}_{degraded})}[log(D(X_{real}))]\\\ &-\mathbb{E}_{I_{syn}^{LR}}\sim
p_{(I_{syn}^{LR})}[log(1-D(X_{fake}))]\end{split}$ (17)

According to the adversarial loss, $G_{B}$ can generate desired $I^{LR}_{syn}$
under the supervision of $D_{A}$. In order to avoid color variation between
$I^{LR}_{idt}$ and $I^{LR}_{real}$, we further define backward identity loss
$L^{idt}_{real}$. The $I^{LR}_{idt}$ is generated by $G_{A}$, as Eqn. 18 shows
and the $L^{idt}_{real\\_LR}$ is defined as Eqn. 19:

$I_{idt}^{LR}=G_{A}((I^{LR}_{real})_{i})$ (18)
$L^{idt}_{real\\_LR}=\frac{1}{N}\sum_{i}^{N}||(I^{LR}_{idt})_{i}-(I^{LR}_{real})_{i}||_{1}$
(19)

Moreover, the backward cycle-perceptual loss $L^{percep}_{FE_{B}}$ is
calculated to recover visual pleasing details. We utilize $FE_{B}$ to measure
the Euclidean distance between feature maps of $I^{LR}_{recon}$ and
$I^{LR}_{real}$. It can be defined as follows:

$\begin{split}L^{percep}_{FE_{B}}=\frac{1}{N}\sum_{i}^{N}||FE_{q,r}((I^{LR}_{recon})_{i})-FE_{q,r}((I_{real}^{LR})_{i})||_{2}\end{split}$
(20)

In the end, the total optimization loss $L^{Backward}_{total}$ for the
backward cycle module can be formulated as:

$\begin{split}L^{Backward}_{total}=\lambda_{1}L^{adv}_{G_{B}}+\lambda_{2}L^{cyc}_{G_{A}}+\lambda_{3}L^{idt}_{real\\_LR}+\lambda_{4}L^{percep}_{FE_{B}}\end{split}$
(21)

where the hyper-parameters $\lambda_{1}$, $\lambda_{2}$, $\lambda_{3}$,
$\lambda_{4}$ are the set of corresponding weights for $L^{adv}_{G_{B}}$,
$L^{cyc}_{G_{A}}$, $L^{idt}_{real\\_LR}$ and $L^{percep}_{FE_{B}}$. The
$\lambda_{1}$, $\lambda_{2}$, $\lambda_{3}$, and $\lambda_{4}$ are empirically
set to 1, 10, 1 and 1 respectively.

####  III-A3 Total Unsupervised Bi-directional Cycle Domain Transfer Network
Loss

The full optimization objective loss of UBCDTN $L^{UBCDTN}_{total}$, which is
an addition of forward cycle module loss $L^{Forward}_{total}$ and backward
cycle module loss $L^{Backward}_{total}$, can be represented as follows:

$L^{UBCDTN}_{total}=L^{Forward}_{total}+L^{Backward}_{total}$ (22)

Figure 3: Red dotted rectangle: The architecture of the Generator. Blue dotted
rectangle: The architecture of the Joint Discriminator. $F_{SF}$ denotes
shallow features, $F_{MDBM}$ denotes the outputs of MDBM, $F_{GF}$ represents
global features, and $F_{MHF}$ represents multiple hierarchical features. K, n
and s are the kernel size, number of filters, and strides respectively. N is
the number of neurons in the dense layer.

###  III-B Semantic Encoder Guided Super Resolution Network

In this section, we present the proposed SESRN, which aims to generate the
desired $I^{SR}$ from the $\widehat{I}^{LR}_{real}$ produced by UBCDTN. First,
we describe four main components of SESRN: $G_{SR}$, $SE$, $D_{SR}$, $CE$.
Second, we describe the optimization loss function for SESRN. The architecture
of SESRN is illustrated in Fig. 2. In addition, the structure of the joint
discriminator and generator proposed in GAFH-RIDN [13] are shown in Fig. 3.

####  III-B1 Generator

We utilize the Dense Nested Blocks (DNBs) and Residual in Internal Dense
Blocks (RIDBs) that we proposed in GAFH-RIDN [13] as basic units to construct
$G_{SR}$. The goal of $G_{SR}$ is to super-resolve to the SR image $I^{SR}$
from the real-like LR image $\widehat{I}_{real}^{LR}$. The overall super-
resolution process is formulated as:

$I^{SR}=G_{SR}(\widehat{I}_{real}^{LR})$ (23)

As shown at the top of Fig. 3, the generator $G_{SR}$ contains three stages:
Shallow Feature Module (SFM), Multi-level Dense Block Module (MDBM), and
Upsampling Module (UM). Specifically, the MDBM is built up by multiple DNBs
formed by several RIDBs. Each DNB includes 3 RIDBs cascaded by residual
connections and one scaling layer. The designed architecture guarantees the
feature maps of each layer are propagated into all succeeding layers,
promoting an effective way to extract hierarchical features and alleviating
the gradient vanishing problem. It is emphasized that Local Residual Learning
(LRL) is introduced to take effective use of the local residual features
extracted by RIDBs. In addition, in order to help the generator fully take
advantage of hierarchical features, we design the Global Residual Learning
(GRL) to fuse the shallow features $F_{SF}$ and global features $F_{GF}$.
Overall, benefiting from the proposed architecture [13], $G_{SR}$ is capable
of exploiting abundant hierarchical features and super-resolving from the LR
space to the HR space.

####  III-B2 Semantic Encoder

The proposed semantic encoder is supposed to extract embedded semantics (as
shown in Fig. 2), which is used to project visual information (HR, LR) back to
the latent space. The motivation is that the GAN-based SR models [15, 7, 20]
only exploit visual information during the discriminative procedure, ignoring
the semantic information reflected by latent representation. Therefore, the
proposed semantic encoder will complement the missing critical property.
Previous GAN’s work [21, 22] has proved that the semantic representation is
beneficial to the discriminator.

Based on this observation, the proposed semantic encoder is designed to
inversely map the image to the embedded semantics. Significantly, the most
important advantage of the semantic encoder is that it is able to guide the
discriminative process since the embedded semantics obtained from the semantic
encoder can reflect semantic attributes, such as the image features (color,
texture, and shape) and the spatial relationship between various components of
the images. It can be emphasized that the embedded semantics is fed into a
joint discriminator along with HR and LR images, leading to optimizing the
adversarial process. Thanks to this property, the semantic encoder can guide
the discriminative adversarial learning of the joint discriminator, thereby
enhancing its discriminative ability.

In this context, we utilize the pre-trained VGG16 [23] networks as the
semantic encoder. For the purpose of satisfying the different dimensions of HR
and LR images, we adopt two side-by-side semantic encoders, which have the
same structure but different input dimensions, to obtain embedded semantics
from different convolutional layers respectively.

####  III-B3 Joint Discriminator

As shown in Fig. 2, the proposed joint discriminator takes the tuple
incorporating both visual information and embedded semantics as the input,
where Embedded Semantics-Level Discriminative Sub-Net (ESLDSN) aims to
identify the embedded semantics whether it comes from the HR images while
Image-Level Discriminative Sub-Net (ILDSN) distinguishes whether the input
image is from the HR image dataset or the generator. Next, through the
operation of the Fully Connected Module (FCM) on a concatenated vector, the
final probability is predicted.

Thanks to this property, the joint discriminator is capable of learning the
joint probability distribution of image data ($I^{HR},I^{SR}$) and embedded
semantics ($SE(I^{HR}),SE(\widehat{I}^{LR}_{real})$). In order to satisfy this
structure, we design two sets of paths entering into the joint discriminator.
The set of red paths in Fig. 2 represents a real tuple which consists of the
real HR image $I^{HR}$ from the dataset and its embedded semantics
$SE(I^{HR})$. The set of paths in blue indicates the fake tuple constructed by
SR image $I^{SR}$ generated from the generator and
$SE(\widehat{I}^{LR}_{real})$ produced by semantic encoder through real-like
LR image. Therefore, the joint discriminator has the ability to measure the
difference between real tuple $(I^{HR},SE(I^{HR}))$ and the fake tuple
$(I^{SR},SE(\widehat{I}^{LR}_{real}))$.

In addition, we adopt the Relativistic average Least Squares GAN (RaLSGAN)
[19] loss for the joint discriminator by applying the RaD to the least squares
loss function [24]. The real tuple is denoted as
$X_{real}=(I^{HR},SE(I^{HR}))$, and the fake tuple is expressed as
$X_{fake}=(I^{SR},SE(\widehat{I}^{LR}_{real}))$. In the training procedure,
the joint discriminator receives both $X_{real}$ and $X_{fake}$ as the input.
It can be expressed as follows:

$\tilde{C}(X_{real})=(C(X_{real})-E_{x_{f}}[C(X_{fake})])$ (24)
$\tilde{C}(X_{fake})=(C(X_{fake})-E_{x_{r}}[C(X_{real})])$ (25)

where $\tilde{C}(\cdot)$ denotes the probability predicted by the joint
discriminator. Moreover, the least squares loss is applied to evaluate the
distance between HR and SR images. Thus, we define the optimization loss
$L_{D_{SR}}^{RaLS}$ and $L_{G_{SR}}^{RaLS}$ for joint discriminator and
generator respectively:

$\begin{split}L_{D_{SR}}^{RaLS}=&\mathbb{E}_{I^{HR}\sim
p_{(I^{HR})}}[(\tilde{C}(X_{real})-1)^{2}]\\\ &+\mathbb{E}_{I^{SR}\sim
p_{(I^{SR})}}[(\tilde{C}(X_{fake})+1)^{2}]\end{split}$ (26)
$\begin{split}L_{G_{SR}}^{RaLS}=&\mathbb{E}_{I^{SR}\sim
p_{(I^{SR})}}[(\tilde{C}(X_{fake})-1)^{2}]\\\ &+\mathbb{E}_{I^{HR}\sim
p_{(I^{HR})}}[(\tilde{C}(X_{real})+1)^{2}]\end{split}$ (27)

By taking advantage of least squares loss and relativism in RaLS, SESRN is
capable of enhancing model stability and producing visually pleasant SR
images.

####  III-B4 Content Extractor

In SESRN, we further leverage the pre-trained VGG19 network as content
extractor $\phi$ to obtain feature representations, where they are used to
formulate the content loss $L_{content}$. Specifically, we calculate the
$L_{content}$ based on the Euclidean distance between two feature
representations of SR images and HR images. We extract feature representations
from the ‘Conv3_3’ layer in the content extractor, which are the low-level
features before the activation layer. With this loss term, the SESRN is
encouraged to reconstruct finer high-frequency details and improve perceptual
quality.

####  III-B5 Loss Function

We introduce content loss $L_{content}$ to constrain SR images to be faithful
to human visual perception. Besides, we also involve pixel-wise loss
$L_{pixel}$ to optimize our method. Furthermore, adversarial losses
$L_{G_{SR}}^{RaLS}$ and $L_{D_{SR}}^{RaLS}$ are applied to $G_{SR}$ and
$D_{SR}$ respectively, which allows the generator to produce the SR image
consistent with the distribution of the HR image.

Content Loss: $L_{content}$ is able to improve the perceptual similarity
between SR and HR images. It is formulated as:

$L_{content}=||\phi_{i,j}(I^{HR})-\phi_{i,j}(I^{SR})||^{2}_{2}$ (28)

where $\phi_{i,j}(\cdot)$ represents the feature representations obtained from
$j$-th convolution layer before $i$-th maxpooling layer in the fixed content
extractor.

Pixel-wise Loss: The pixel-wise loss is widely applied to enforce the
intensity similarity between the SR and HR images. It is calculated as:

$L_{pixel}=\frac{1}{N}\sum_{i}^{N}||(I^{SR})_{i}-(I^{HR})_{i}||_{2}$ (29)

We use $L_{pixel}$ to minimize the distance between $I^{SR}$ and $I^{HR}$.

Total Loss: Finally, we obtain the total loss function $L_{total}^{SESRN}$ for
SESRN. It is the weighted sum of the three above-discussed losses. The formula
is described as follows:

$L^{SESRN}_{total}=\lambda_{con}L_{content}+\lambda_{adv}L_{G}^{RaLS}+\lambda_{pixel}L_{pixel}$
(30)

where $\lambda_{con}$, $\lambda_{adv}$ and $\lambda_{pixel}$ are empirically
set to 1, $10^{-3}$ and 1 respectively.

###  III-C Full objective loss for UBCDTL-GAN

The full objective loss for the UBCDTL-GAN, which is the combination of
$L^{UBCDTN}_{total}$ and $L^{SESRN}_{total}$, can be defined as:

$L^{UBCDTL-GAN}_{total}=L^{UBCDTN}_{total}+L^{SESRN}_{total}$ (31)

The complete objective loss $L^{UBCDTL-GAN}_{total}$ encourages the proposed
method UBCDTL-GAN can solve the unpaired real-world image super-resolution
problem.

##  IV Experiments

In this section, we first present the datasets and the experimental details.
Then, we compare our method with state-of-the-art SISR methods. The
qualitative comparison can be seen in Fig. 4, and the quantitative comparisons
are provided in Table I. It is emphasized that all the quantitative results
are cited from their official published papers. Moreover, as for qualitative
comparison, we directly download the released code and pre-trained models of
the compared methods. Then, we carefully re-implement referenced methods on
the same validation dataset to obtain effective results.

Figure 4: Qualitative comparison of visual results with state-of-the-art
methods on NTIRE 2020 Real-World Track 1 images. Our method produces photo-
realistic results.

###  IV-A Training Data

At the training stage, we conduct experiments on the DF2K dataset [25, 5],
which is a merge of the DIV2K and Flickr2K datasets, containing a total of
3450 images. Specifically, as for the LR images, we use the real-world LR
images from DIV2K NTIRE 2017 unknown degradation 4× dataset, and Flickr2K LR
images collected from NTIRE 2020 Real-World Track 1 training source dataset,
where all the LR images are corrupted with unknown degradation, and
downsampled 4× by an unpredicted operator to satisfy real-world conditions,
resulting in sensor noise, compression artifacts, etc. Since the goal of our
method is to solve an unsupervised super-resolution problem without LR-HR
paired images, we select the first 1725 images (number: 1-1725) from the DF2K
HR dataset as our HR training dataset, and the LR training dataset is formed
by the other 1725 images (number: 1726-3450) obtained from DF2K real-world LR
dataset. Overall, our method is trained on such an unpaired real-world LR-HR
dataset.

To evaluate the proposed method on real-world data, at the testing stage, we
use the validation dataset from the NTIRE 2020 Real-World SR challenge Track
1. This dataset contains 100 testing LR images (scaling factor: 4×), where all
LR images are processed with unknown degradation operations to simulate the
realistic artifacts and natural characteristics. In order to compare the
qualitative and quantitative results fairly, we use the same validation
dataset for all experiments.

###  IV-B Training Setups

The training procedure is divided into three steps. Instead of randomly
initializing model weights, we pre-train the UBCDTN and SESRN in the first and
second steps, and then we jointly train the whole method in an end-to-end
manner. First, we train the UBCDTN with unpaired artificially degraded images
$I^{LR}_{degraded}$ and real-world images $I^{LR}_{real}$, which aims to
transfer the LR image from the artificially degraded LR domain to the real-
world LR domain. Second, we pre-train the SESRN using the approximated real-
like images $\widehat{I}^{LR}_{real}$ and their HR version $I^{HR}$ to
generate realistic super-resolved images $I^{SR}$. As for pre-trained UBCDTN
and SESRN, we use the same optimization strategy where the Adam optimizer [26]
is applied to train both networks by setting $\beta_{1}=0.9$,
$\beta_{2}=0.999$. The learning rate is initialized as $10^{-4}$ and the
minibatch size is set as 8. We train the UBCDTN and SESRN 50K epochs
separately. After the pre-training process, both UBCDTN and SESRN are able to
acquire exceptional initialization weights, which are beneficial to training
stability and fast training speed. In the third step, we jointly train two
networks together. The proposed UBCDTL-GAN takes an unpaired HR image $I^{HR}$
and $I^{LR}_{real}$ as the input to generate the final result $I^{SR}$. The
UBCDTL-GAN applies the Adam optimizer with an initial learning rate =
$10^{-4}$, $\beta_{1}=0.9$ and $\beta_{2}=0.999$ to the whole training
process. The model is trained for 100K epochs with the minibatch size 8.

TABLE I: Quantitative comparison on NTIRE 2020 Real World Super-Resolution Challenge Track 1 validation dataset of the proposed method against state-of-the-art methods, in terms of average PSNR (dB) and SSIM for upscale factor 4×. The bold results indicate the best performance. Methods |  |  NTIRE_2020_T1
---|---|---
|  |  PSNR |  |  SSIM
Bicubic |  |  23.87 |  |  0.644
Nearest Neighbor |  |  23.39 |  |  0.580
EDSR [5] |  |  25.36 |  |  0.640
ESRGAN [7] |  |  19.04 |  |  0.242
SRGAN [15] |  |  20.78 |  |  0.525
SRFBN [27] |  |  25.37 |  |  0.642
MsDNN [28] |  |  25.08 |  |  0.708
RCAN [29] |  |  25.31 |  |  0.640
USISResNet [30] |  |  21.71 |  |  0.589
CycleGAN [12] |  |  25.01 |  |  0.618
ZSSR [18] |  |  24.87 |  |  0.600
Proposed |  |  26.83 |  |  0.789

###  IV-C Quantitative Comparison

The quantitative results presented in Table I demonstrate that our method has
promising superiority, achieving the highest 26.83dB/0.789 in terms of
PSNR/SSIM values. The results obtained from SRFBN [27] place the second-best
with 25.37dB/0.642. Our method improves the PSNR/SSIM values by 1.46dB/0.147
over their method. EDSR [5] and SRFBN [27] cannot achieve desired numerical
results since they are PSNR-oriented methods and are merely trained on simple
degradation images. Moreover, we found that the performances of ESRGAN [7] and
SRGAN [15] are lower than most of the methods in Table I. Besides, from Fig.
5, the visual results of ESRGAN and SRGAN show over-smoothed textures and
unrealistic artifacts. Interestingly, we found such a phenomenon has also been
presented in [10, 18, 30]. The underlying reason is that these methods only
train on the simple and clean pre-defined LR images, ignoring the difference
of domain distribution between the real-world LR domain and the pre-defined LR
domain. Benefiting from the proposed UBCDTN and SESRN, our method has the
ability to solve the problem of domain distribution shift and boost
quantitative performances when dealing with real-world SR tasks.

###  IV-D Qualitative Comparison

The visual comparisons are provided in Fig. 4. We compare with various SR
methods such as Bicubic, Nearest Neighbor, SRGAN [15], ESRGAN [7], CycleGAN
[12], and ZSSR [18]. For the Bicubic and Nearest Neighbor methods, it is
obvious that the results lack high-frequency contents, producing overly smooth
edges and coarse textures. Regarding the SRGAN, it fails to alleviate the
blurring details on the lines and edges of the SR results. The results of
ESRGAN suffer from apparently broken artifacts and dramatic degradation
problems, which are unfaithful to human perception. As for the unsupervised
method CycleGAN and ZSSR, the SR results are improved, but there are still far
away from the ground truth. Although the SR images of CycleGAN present better
shapes than the previously compared methods, the results still remain
unnatural edges and distortions, leading to poor visual effects. Besides, the
blind method ZSSR was also evaluated, but it fails to reduce visible
corruptions in some degree since there are still over-smoothed textures and
noise-like characteristics existing in the images.

Compared with the aforementioned methods, the SR results of our method
superiorly outperform all other methods, super-resolving visually pleasant SR
images with sharper edges and finer textures. Our SR results are more
realistic than SRGAN and ESRGAN since these two methods are only trained on
simple degradation data (e.g., bicubiced LR images) without introducing any
complicated noise and artifacts from the real-world images while our method
trains on approximated real-like LR images consisting of similar
characteristics as real-world LR images. The unsupervised method CycleGAN is
less effective in super-resolve unclear LR images. Although it involves the
cycle translation model, it lacks a powerful super-resolution network as the
one (SESRN) in our method. Besides, the other unsupervised method ZSSR also
fails to achieve the expected results, since it does not take into account the
domain gap between noise-free LR images and real-world images. In contrast,
benefiting from the domain transfer network (UBCDTN), our method is able to
successfully eliminate the domain gap and produce real-like LR images
comprising real-world patterns. Overall, The SR results verify the powerful
unsupervised learning strategy used in the proposed method for super-resolving
photo-realistic SR images.

TABLE II: The compared variants of the proposed method in the ablation study and the descriptions of the proposed components. The tick indicates that this variant includes this component. Variants | Methods | SESRN | $G_{A}$ | $G_{B}$ | $D_{B}$ | $D_{A}$ | $FE_{B}$ | $FE_{A}$
---|---|---|---|---|---|---|---|---
VariantA | SESRN | $\surd$ |  |  |  |  |  |
VariantB |  SESRN+$G_{A}$+$D_{B}$ | $\surd$ | $\surd$ |  | $\surd$ |  |  |
VariantC |  SESRN+$G_{A}$+$FE_{B}$+$G_{B}$+$FE_{A}$ | $\surd$ | $\surd$ | $\surd$ |  |  | $\surd$ | $\surd$
VariantD |  SESRN+$G_{A}$+$D_{B}$+$G_{B}$+$D_{A}$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ |  |
VariantE |  SESRN+$G_{A}$+$D_{B}$+$G_{B}$+$D_{A}$+$FE_{B}$+$FE_{A}$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$ | $\surd$

##  V Ablation Study

In this section, we conduct the ablation study to further investigate the
components of the proposed method and demonstrate the advantages of UBCDTL-
GAN. The list of compared variants of our method is presented in Table II. We
provide visual SR results of different variants in Fig. 5. The quantitative
comparison of several variants is presented in Table III.

###  V-A Description of Different Variants of the Proposed Method

In ablation studies, we design several variants which consist of different
proposed components. We adopt the SESRN as the baseline variant in the
following experiments and pay more attention to investigating the elements
used in the UBCDTN. To comply with the single variable principle, we gradually
add one of the components to the baseline variant. We describe the details of
designed variants, all of which are specified as follows:

1) VariantA: The VariantA is designed as the baseline variant, which only
contains SESRN. As shown in Table II, VariantA can be considered as removing
all UBCDTN components from the ultimate proposed method. In the following
variants, we successively add each of the components to VariantA.

2) VariantB: In VariantB, we introduce $G_{A}$ and $D_{B}$ while $G_{B}$,
$D_{A}$ and both $FE_{A}$, $FE_{B}$ are removed. Because $G_{A}$ and $D_{B}$
are essential components of the forward cycle module in UBCDTN, VariantB can
be considered as composed of the forward cycle module of UBCDTN and baseline
model, removing the backward cycle module.

3) VariantC: Besides the baseline model, it consists of two generators
$G_{A}$, $G_{B}$ and two feature extractors $FE_{A}$, $FE_{B}$ of UBCDTN,
eliminating discriminators $D_{A}$ and $D_{B}$ involved in UBCDTN.

4) VariantD: It is constructed by the four components $G_{A}$, $G_{B}$,
$D_{B}$ and $D_{A}$ of UBCDTN, while it removes feature extractors $FE_{A}$
and $FE_{B}$ of UBCDTN.

5) VariantE (Proposed): The VariantE represents the ultimate proposed method
which comprises a baseline model and all components of UBCDTN.

Figure 5: Qualitative comparisons of different variants in our ablation study. The visual results on images ”0896”, ”0824” from NTIRE 2020 Track 1 testing dataset with scale factor 4×. The best results are highlighted  TABLE III: Quantitative results of ablation study with different variants on NTIRE 2020 validation T1 dataset, in terms of average PSNR (dB) and SSIM for upscale factor 4×. The bold results indicate the best performance. Variants |  NTIRE_2020_T1
---|---
|  PSNR |  SSIM
VariantA |  25.97 |  0.757
VariantB |  23.47 |  0.698
VariantC |  25.44 |  0.729
VariantD |  25.81 |  0.746
VariantE (Proposed) |  26.83 |  0.789

###  V-B Effect of UBCDTN

This experiment is conducted by VariantA and VariantE. Specifically, because
of removing UBCDTN, VariantA is trained on bicubic downsampled LR images
directly while VariantE takes real-like LR images obtained by UBCDTN as the
training LR inputs. According to the analysis of the performance between
VariantA and VariantE, we can demonstrate advantages originating from UBCDTN.
As shown in Fig. 5, VariantA produces over-smoothed SR images missing high-
frequency details while the SR results of VariantE contain naturally desired
edges and textures. In addition, from Table III, the quantitative results of
VariantA decrease dramatically from 26.83dB/0.789 to 25.97dB/0.757 after
removing UBCDTN. The reason is that VariantA simply trains on bicubic data,
ignoring the domain distribution difference between bicubic data and real-
world data when solving the real-world SR task. By incorporating the UBCDTN in
the variant, there is a noteworthy improvement in terms of both qualitative
and quantitative performance, which is able to verify that UBCDTN plays an
important role in the super-resolution procedure. Thus, we can validate the
effectiveness of the proposed UBCDTN and its necessity.

###  V-C Effect of $G_{B}$ and $D_{A}$

In this experiment, we compare VariantB and VariantE to verify the effect of
$G_{B}$ and $D_{A}$, which is also identical to show the effectiveness of the
backward cycle module. Note that VariantB is equivalent to UBCDTN for domain
transformation without the backward cycle module. In this setting, VariantB
takes the artificially degraded images as the input to $G_{A}$ and generates
real-like LR images through the supervision of $D_{B}$ in the forward cycle
module without the backward cycle module. It can be observed from Table III
that with the absence of the backward cycle module, VariantB performs worse
than VariantE which contains the whole UBCDTN in terms of PSNR/SSIM values,
since there is no restriction to prevent the forward cycle module and backward
cycle module from contradicting each other. A huge enhancement can be observed
after integrating the backward cycle module into VariantE, where it is able to
greatly improve quantitative performance and further produce high quality SR
images with desirable details. This is due to the presence of a backward cycle
module. The variant is capable of utilizing cycle consistency constraint,
which guarantees the correction between two inverse modules in an unsupervised
manner. In a word, by introducing $G_{B}$ and $D_{A}$, the whole bi-
directional cycle consistency learning strategy can be established to produce
real-like LR images which maintain the same characteristics as real-world LR
images. These results validate the significance of involving $G_{B}$ and
$D_{A}$, which also reveals that the backward cycle module can improve
quantitative results and visual quality.

###  V-D Effect of $D_{A}$ and $D_{B}$

In this experiment, we aim to demonstrate the contribution of $D_{A}$ and
$D_{B}$, which can also reflect the importance of an adversarial loss in the
UBCDTN. The VariantC and VariantE are compared to investigate the effect of
$D_{A}$ and $D_{B}$. In this case, VariantC is trained on cycle-consistency
loss, identity loss, and cycle-perceptual loss, while the adversarial loss is
not involved resulting from removing $D_{A}$ and $D_{B}$. As shown in Table
III, it is obvious that performance severely decreases by removing
discriminators from VariantC. By incorporating $D_{A}$ and $D_{B}$ into
VariantE, we observe that VariantE significantly outperforms VariantC by a
large margin of 1.39dB/0.06 in terms of PSNR/SSIM. From Fig. 5, the visual
results of VariantC are degraded, where the SR images contain over-smoothed
textures and unclear artifacts, while VarantE is able to generate visually
realistic SR images with more nature-looking details. The underlying reason
for poor performance is that we cannot employ adversarial loss on VariantC,
which leads to the lack of adequate iterative adversarial training. According
to the quantitative and qualitative comparisons, it can be verified that by
applying adversarial loss performed by $D_{A}$ and $D_{B}$ to the variant, the
SR performance is greatly enhanced, indicating the effectiveness of $D_{A}$,
$D_{B}$ and adversarial loss.

###  V-E Effect of $FE_{A}$ and $FE_{B}$

We compare VariantD and VariantE in this experiment to verify the
effectiveness of $FE_{A}$ and $FE_{B}$, where the advantages of $FE_{A}$ and
$FE_{B}$ are also equal to the benefit of cycle-perceptual loss. By removing
$FE_{A}$ and $FE_{B}$, VariantD no longer employs cycle-perceptual loss on the
training phase to optimize the model. As shown in Table III, VariantE achieves
higher quantitative performance compared to VariantD, increasing PSNR/SSIM
values from 25.81dB/0.746 to 26.83dB/0.789. In addition, Fig. 5 clearly shows
the qualitative comparisons evaluated by the SR results from VariantD and
VariantE with and without cycle-perceptual loss in the training process. It is
apparent that without cycle-perceptual loss, the deteriorated textures are
visible in the SR results of VariantD. In contrast, when integrating $FE_{A}$
and $FE_{B}$ into the variant, the cycle-perceptual loss can be calculated,
which further motivates the variant to produce perceptually natural images
with exceptional details. According to the numerical performance and visual
results, we are able to verify that $FE_{A}$ and $FE_{B}$ have a significant
impact on the super-resolution process, which also identifies that the cycle-
perceptual loss is able to improve the perceptual quality of SR images.

###  V-F Final Effect

The VariantE can be considered as the ultimate proposed method, which includes
all the proposed components. Compared with other variants, the ultimate
proposed method is able to greatly improve quantitative performance and
obviously enhance the quality of visual results. Thus, we can conclude the
effectiveness of the proposed method as well as all the components.

##  VI Conclusion

We proposed an unsupervised super-resolution method UBCDTL-GAN for real-world
scenarios, which does not involve any paired image data and pre-defined
degradation operation. The proposed method comprises two networks, UBCDTN and
SESRN. First, the UBCDTN transfers an artificially degraded image to a real-
like image with real-world artifacts and characteristics. Next, SESRN
reconstructs from the approximate real-like LR image to a visually pleasant
super-resolved image with realistic details and textures. According to the
designed framework and applied optimization constraints, the proposed method
UBCDTL-GAN has the ability to improve real-world super-resolution performance.
The quantitative and qualitative experiments on NTIRE 2020 T1 real-world SR
dataset validate the effectiveness of our method and show superior SR
performances compared to existing state-of-the-art methods.

## References

  * [1] P. Rasti, T. Uiboupin, S. Escalera, and G. Anbarjafari, “Convolutional neural network super resolution for face recognition in surveillance monitoring,” in _International conference on articulated motion and deformable objects_. Springer, 2016, pp. 175–184.
  * [2] D. Capel and A. Zisserman, “Super-resolution enhancement of text image sequences,” in _Proceedings 15th International Conference on Pattern Recognition. ICPR-2000_ , vol. 1. IEEE, 2000, pp. 600–605.
  * [3] W. Shi, J. Caballero, C. Ledig, X. Zhuang, W. Bai, K. Bhatia, A. M. S. M. de Marvao, T. Dawes, D. O’Regan, and D. Rueckert, “Cardiac image super-resolution with global correspondence using multi-atlas patchmatch,” in _International conference on medical image computing and computer-assisted intervention_. Springer, 2013, pp. 9–16.
  * [4] Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, “Residual dense network for image super-resolution,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 2472–2481.
  * [5] B. Lim, S. Son, H. Kim, S. Nah, and K. Mu Lee, “Enhanced deep residual networks for single image super-resolution,” in _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , 2017, pp. 136–144.
  * [6] T. Tong, G. Li, X. Liu, and Q. Gao, “Image super-resolution using dense skip connections,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 4799–4807.
  * [7] X. Wang, K. Yu, S. Wu, J. Gu, Y. Liu, C. Dong, Y. Qiao, and C. Change Loy, “Esrgan: Enhanced super-resolution generative adversarial networks,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 0–0.
  * [8] G. Kim, J. Park, K. Lee, J. Lee, J. Min, B. Lee, D. K. Han, and H. Ko, “Unsupervised real-world super resolution with cycle generative adversarial network and domain discriminator,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2020, pp. 456–457.
  * [9] S. Maeda, “Unpaired image super-resolution using pseudo-supervision,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 291–300.
  * [10] Y. Yuan, S. Liu, J. Zhang, Y. Zhang, C. Dong, and L. Lin, “Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2018, pp. 701–710.
  * [11] T. Zhao, W. Ren, C. Zhang, D. Ren, and Q. Hu, “Unsupervised degradation learning for single image super-resolution,” _arXiv preprint arXiv:1812.04240_ , 2018.
  * [12] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2223–2232.
  * [13] X. Wang, Y. Yang, Q. Pang, Q. Tang, and S. Du, “End-to-end generative adversarial face hallucination through residual in internal dense network,” in _29th European Signal Processing Conference (EUSIPCO)_ , 2021.
  * [14] C. Dong, C. C. Loy, K. He, and X. Tang, “Image super-resolution using deep convolutional networks,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 38, no. 2, pp. 295–307, 2015.
  * [15] C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang _et al._ , “Photo-realistic single image super-resolution using a generative adversarial network,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4681–4690.
  * [16] J. Gu, H. Lu, W. Zuo, and C. Dong, “Blind super-resolution with iterative kernel correction,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 1604–1613.
  * [17] K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep cnn denoiser prior for image restoration,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 3929–3938.
  * [18] A. Shocher, N. Cohen, and M. Irani, ““zero-shot” super-resolution using deep internal learning,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 3118–3126.
  * [19] A. Jolicoeur-Martineau, “The relativistic discriminator: a key element missing from standard gan,” _arXiv preprint arXiv:1807.00734_ , 2018.
  * [20] X. Yu and F. Porikli, “Ultra-resolving face images by discriminative generative networks,” in _European conference on computer vision_. Springer, 2016, pp. 318–333.
  * [21] J. Donahue, P. Krähenbühl, and T. Darrell, “Adversarial feature learning,” _arXiv preprint arXiv:1605.09782_ , 2016.
  * [22] V. Dumoulin, I. Belghazi, B. Poole, O. Mastropietro, A. Lamb, M. Arjovsky, and A. Courville, “Adversarially learned inference,” _arXiv preprint arXiv:1606.00704_ , 2016.
  * [23] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” _arXiv preprint arXiv:1409.1556_ , 2014.
  * [24] X. Mao, Q. Li, H. Xie, R. Y. Lau, Z. Wang, and S. Paul Smolley, “Least squares generative adversarial networks,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 2794–2802.
  * [25] E. Agustsson and R. Timofte, “Ntire 2017 challenge on single image super-resolution: Dataset and study,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2017, pp. 126–135.
  * [26] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” _arXiv preprint arXiv:1412.6980_ , 2014.
  * [27] Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, and W. Wu, “Feedback network for image super-resolution,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 3867–3876.
  * [28] S. Gao and X. Zhuang, “Multi-scale deep neural networks for real image super-resolution,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2019, pp. 0–0.
  * [29] Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu, “Image super-resolution using very deep residual channel attention networks,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 286–301.
  * [30] K. Prajapati, V. Chudasama, H. Patel, K. Upla, R. Ramachandra, K. Raja, and C. Busch, “Unsupervised single image super-resolution network (usisresnet) for real-world data using generative adversarial network,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2020, pp. 464–465.


# NetReAct: Interactive Learning for Network Summarization

Sorour E. Amiri*
<EMAIL_ADDRESS>
Virginia Tech &Bijaya Adhikari$\dagger$
<EMAIL_ADDRESS>
University of Iowa &John Wenskovitch$\ddagger$
<EMAIL_ADDRESS>
PNNL &Alexander Rogriguez$+$
<EMAIL_ADDRESS>
Georgia Tech &Michelle Dowling$\diamond$
<EMAIL_ADDRESS>
GVSU &Chris North*
<EMAIL_ADDRESS>
Virginia Tech &B. Aditya Prakash$+$
<EMAIL_ADDRESS>
Georgia Tech

###### Abstract

Generating useful network summaries is a challenging and important problem
with several applications like sensemaking, visualization, and compression.
However, most of the current work in this space do not take human feedback
into account while generating summaries. Consider an intelligence analysis
scenario, where the analyst is exploring a similarity network between
documents. The analyst can express her agreement/disagreement with the
visualization of the network summary via iterative feedback, e.g. closing or
moving documents (“nodes”) together. How can we use this feedback to improve
the network summary quality? In this paper, we present NetReAct, a novel
interactive network summarization algorithm which supports the visualization
of networks induced by text corpora to perform sensemaking. NetReAct
incorporates human feedback with reinforcement learning to summarize and
visualize document networks. Using scenarios from two datasets, we show how
NetReAct is successful in generating high-quality summaries and visualizations
that reveal hidden patterns better than other non-trivial baselines.

##  1 Introduction

Networks occur in various domains such as social networks, entity networks,
communication networks, population contact networks, and more. A meaningful
summary of these networks can help users with various downstream tasks like
sensemaking, compression, and visualization [1, 2, 10, 11]. However, most
prior work focus on generating summaries without human input [9, 10]. In
contrast, there are several applications, especially exploratory tasks, where
incorporating human feedback in the summarization process is essential for
generating useful summaries. For example, consider intelligence analysis [3],
which often involves making sense of networks of unstructured documents (like
field reports) and extracting hidden information (like a terrorist plot) from
a small subset of documents in the corpus. Users can provide feedback by
interacting directly with the data, providing semantic interactions [5] such
as moving two nodes (documents) closer to each other to express similarity.
This feedback helps the system to determine the relative importance of other
documents with respect to the interests of the user.

Motivated by above, in this paper we tackle the novel problem of learning to
generate interactive network summaries that incorporate user feedback. We
showcase the usefulness of our summaries by focusing on an exploratory
document visualization task. We leverage our summary to generate network
visualizations, with the goal of aiding in investigating document networks and
supporting the human sensemaking process; i.e., help the users “connect the
dots” across different documents and discover the hidden stories behind them.
More specifically, we try to answer: Given a document corpus represented as a
network, can we learn a model that incorporates user feedback alongside the
objectives of the analysis task to generate high quality summary?
Additionally, can we then leverage the summary to generate a meaningful
visualization, and can such a model be re-applied to other document corpora?
Towards solving this problem, we face two major challenges. The first is
simplicity of feedback. Generally, the users are not experts in the
summarization/visualization models and can only provide high-level semantic
interactions. The second challenge is sparsity and inconsistency of human
feedback. Getting human feedback is a slow and expensive process as the user
needs to understand both the data and task in hand. As a result, the feedback
is sparse.

Here the _network_ data structure represents a document corpus. Each node in
the network represents a document, and edge weights indicate the similarity
between the documents based on their word usage. Our goal is to generate a
summary network by grouping similar nodes (i.e., relevant documents together)
and finding hierarchical “super-nodes” (representing collection of documents)
and “super-edges” (representing similarities between groups of documents). We
then visualize this network summary to produce a high-level understanding of
the entire corpus and highlight hidden patterns. We will discuss the concepts
of super-nodes and super-edges later in detail.

Our main idea is to tackle the above challenges via a reinforcement learning
(RL) approach [15] to summarize networks. We believe RL is especially suited
for this problem, as it makes it possible to re-apply the learned model in
similar scenarios to reduce the necessary amount of human feedback to gain
useful information. We design NetReAct, a feedback-based RL algorithm which
integrates user interests with the objective of the summarization. NetReAct
also provides a multi-level understanding of the data by generating summaries
of various sizes, enabling the user to zoom in to each document group to view
these different summaries.

The main contributions of this paper are:

Incorporating human feedback. We introduce NetReAct, a novel approach which
leverages feedback-based reinforcement learning to principally incorporate
human feedback to generate meaningful network summaries.

Meaningful relationships between groups as a summary network. NetReAct not
only groups relevant nodes into super-nodes, but it also defines relationships
between super-nodes. The weight of the edges in the summary network by
NetReAct represent the similarity between groups.

Multi-scale visualization. We leveraged NetReAct to develop a multi-scale,
interactive document network visualization framework. This visualization
groups documents to summarize the network hierarchically, which makes it
possible for the user to get a multilevel understanding of the document
network by looking at the summaries on different levels.

##  2 Proposed Method

In this paper, we focus on document network summarization to support
visualization. Visualizing a document network helps users in their sensemaking
process and understanding the documents, providing a two-dimensional
illustration of the entire network and highlighting hidden patterns. Learning
from the user is essential in designing an intelligent visualization framework
that reflects the user’s interests. Moreover, leveraging user feedback in the
summarization helps to visualize data more effectively and efficiently than
unsupervised approaches. Using supervised approaches are also not realistic in
many real-world applications, particularly when the analyst is not an expert.

We ground our work using a state-of-the-art systems for interactive document
network visualization, StarSPIRE [3]. This system treats the document network
as an attributed network, where each attribute is the frequency of a
particular term in the document. It then builds a multilevel model of user
interests based on the user’s semantic interactions. In the StarSPIRE
framework, a user can interact with the system through semantic interactions
such as minimizing documents, closing documents, annotating a document,
highlight text within a document, and overlapping two documents. From these
interactions, the models infer the importance of each attribute and then
calculate the similarity between each pair of documents (nodes) based on
weighted attribute similarity. However, since the number of unique terms is
very high, the attributes are high-dimensional, and thus generating optimal
weights requires a significant number of interactions.

In NetReAct, we summarize document networks into explicit groups of related
documents, demonstrating the relationship between groups in order to both
generate a high-quality visualization for sensemaking tasks and to detect
underlying stories hidden within the corpus. We design NetReAct to make it
possible to incorporate semantic interaction feedback with network
summarization, using that user guidance to generate a visualization of a
document corpus. A “good” document network summary leads to a high-quality
visualization, which helps a user to identify and read related documents and
make sense of them quickly. More specifically, in a good network summary, each
super-node (i.e., group) contains documents that are most relevant to each
other according to the user’s interest. Further, the structure of the network
summary indicates the relationship between groups, which guides the user on
how to navigate through different groups. Given such a summary, we first can
visualize the summary network. After this, we can expand the super-nodes that
the user is interested in to suggest the most relevant documents. If the user
wants to investigate more documents, we can then expand the closest (most
similar) super-nodes to suggest another group of relevant documents.

###  2.1 User Feedback

We observed several users completing a sensemaking task using the StarSPIRE
framework, and based on their behavior we selected a subset of the supported
semantic interactions for generating user feedback. We divide such
interactions into _positive_ and _negative_ feedback (see Table 1). For
example, positive feedback can indicate the user’s intention to put two
documents close to each other (i.e., group two nodes together), while negative
feedback means they should be far from each other. Overlapping two documents
indicates that the user agrees to display them close to each other. On the
other hand, minimizing a document while reading another one is a sign of the
disagreement with the visualization. This local feedback is then applied to
the entire visualization and corpus. Such feedback is sparse, as the user
cannot evaluate all documents and every aspect of the visualization.

Table 1: NetReAct feedback types and corresponding semantic interactions in StarSPIRE. Feedback Type |  Semantic Interaction
---|---
Negative feedback |  (1) Minimizing document, (2) Closing document
Positive feedback |  (1) Annotation, (2) Highlighting document, (3) Overlapping document

###  2.2 Interactive Summarization Model

In this section, we describe an interactive network summarization framework to
incorporate the user feedback and address its sparsity. Note that our goal is
also to learn the steps to be taken for the summarization process so that the
same approach can be re-applied on other document corpora with similar
characteristics.

Network Construction.  We start by converting the given document corpus into a
network $G(V,E,W)$, where nodes ($V$) represent documents and edges ($E$) and
their weights ($W$) represent document similarity. We define the weight
$w(v_{1},v_{2})$ to be the cosine similarity between the corresponding TF-IDF
vectors of the documents. Note that $G$ is a complete-graph of size $|V|$.

Network Summarization.  Once the network $G$ is constructed, the summarization
process begins. The goal is to generate a smaller network
$G^{s}(V^{s},E^{s},W^{s})$ from the original network $G(V,E,W)$ such that
nodes representing similar/relevant documents in $G$ are grouped into a single
node (a “super-node”) in $G^{s}$. Nodes in $G^{s}$ therefore represent a group
of documents. We call $G^{s}(V^{s},E^{s},W^{s})$ a “summary network,” where
super-nodes ($V^{s}$) are the groups of related documents and super-edges
($E^{s}$), and their weights ($W^{s}$) represent the average similarity
between group of documents represented by the two endpoints. We obtain $G^{s}$
via a series of “assign” operations on $G$. The “assign” operation assigns
nodes to their super-nodes. This operation partitions the original network $G$
and groups each partition to form a super-node in the summary network $G^{s}$.
Next, we must determine how to partition the original network $G$ in a
meaningful manner. In other words, how to decide between two partitions of
$G$, and how to measure the quality of each assigning operation and network
summary?

Reinforcement Learning (RL) is a natural fit to answer the question above, as
we can view our problem as taking actions (assigning nodes to a super-node) to
maximize the reward (the final quality of grouping of documents). The next
step in our summarization process is to formalize the RL framework for our
task.

####  2.2.1 Interactive Reinforcement Learning Formulation

We use Q-learning, as it is known to be more sample efficient than policy
gradient methods [15]. Each RL framework has five main components: states,
actions, transition function, reward, and policy. We further add an additional
feedback component as we design an interactive RL formulation. Brief
descriptions of each follow.

1\. State: The state $s$ is the sequence of actions which assign nodes to
different super-nodes. We use the embedding $s=[l_{1},l_{2},\ldots,l_{n}]$,
$\forall_{i\in\\{1,2,\ldots,n\\}}\quad l_{i}\in\\{1,2,\ldots,|V_{s}|\\}$ in to
represent the states in a vector of n-dimensional space.

2\. Action: An action $a$ at state $s_{t}$ selects a document $i$, assigns it
to a new super-node $v^{s}$, and transfers it to the next state $s_{t+1}$.

3\. Transition: We assume the transition function $\mathcal{T}$ is
deterministic and corresponds to the action of assigning a document to a new
super-node – i.e., $\mathcal{T}(s_{t},a)=s_{t+1}$.

4\. Rewards: We define the reward to be $-1$ for a state $s$, unless it is a
terminal state. A terminal state in our case is a state which satisfies all of
the positive and negative feedback of the user. Intuitively the reward of $-1$
encourages the learner to satisfy the user feedback faster. Formally, we
define our reward function as follows:

$\displaystyle
r(s,a)=\begin{cases}F_{prob}(s_{next})=\sum_{i=1}^{k}\frac{y_{i}^{T}Ay_{i}}{y_{i}^{T}Dy_{i}}&\text{
if }s_{next}\text{ is a terminal state}\\\ -1&\text{otherwise }\end{cases}$
(1)

Here, $A$ is the adjacency matrix of the document graph $G$, $D$ is the
diagonal matrix of node degrees, and $y_{i}$ is an indicator vector for super-
node $v^{s}_{i}\in V^{s}$, i.e., $y_{i}(v)=1$ if a node $v$ belongs to super-
node $v^{s}_{i}$, zero otherwise. In Eq. 1, we compute $F_{prob}(s_{next})$,
which measures the quality of the document groups. By maximizing
$F_{prob}(s_{next})$, we maximize the quality of document groups [16].

5\. Feedback We assume a case that the user is interacting with the system
until she is happy with the visualization, a process of incrementally
formalizing the space [12]. This means that we must learn the model until all
the feedback from the user is satisfied. The feedback is in the form of
positive and negative interactions (See Tab. 1). The user can indicate if she
agrees to group a pair of documents (i.e., positive feedback) or disagrees
with it (i.e., negative feedback). We represent the feedback with two graphs,
which we call feedback graphs. A positive feedback graph $G^{+}$ is created
from the set of positive feedback, i.e., the edges in $G^{+}$ are pair of
related documents that the user indicated. Similarly, the negative feedback
graph $G^{-}$ is created from the set of negative feedback.

To satisfy all the feedback, we must group all positive feedback node pairs in
the same super-nodes and all negative feedback node pairs in different super-
nodes. These constraints can be stated using the positive and negative
feedback graphs $G^{+}$ and $G^{-}$ as follows:

$\displaystyle\sum_{i=1}^{k}y_{i}^{T}A_{G^{+}}y_{i}-\sum_{i=1}^{k}y_{i}^{T}A_{G^{-}}y_{i}=\sum
A_{G^{+}}$ (2)

Here, $y_{i}$ is a super-node, $k$ is the number of super-nodes, and $A_{G}$
is the adjacency matrix of $G$. In the real world, we do not expect the user
to provide all possible feedback, as this would essentially provide the
desired summary without computational assistance. Rather, the provided
feedback are sparse, especially when the task is exploratory in nature. To
handle such problems, we combine the reward in Eq. 1 with feedback (Eq. 2).
Thus, our goal is to achieve a summary that satisfies all feedback and
maximizes the reward.

6\. Policy: The policy function $\pi(a^{*}|s)$ specifies what action to take
at a given state. It is defined as
$\pi(a^{*}|s)=\arg\max\textrm{Q-value}(s,a)$, where $\textrm{Q-value}(s,a)$ is
the Q-value of the state $s$ and action $a$ that estimates the expected
cumulative reward achieved after taking action $a$ at state $s$. Our goal is
to learn an optimal Q-value function resulting in the highest cumulative
reward. We leverage the Q-learning, which iteratively updates
$\textrm{Q-value}(s,a)$ until convergence [13].

####  2.2.2 Q-learning

Our pipeline learns the best super-node for each node in the document graph
$G$ such that its corresponding summary graph $G^{s}$ gives a high-quality
visualization and is generalizable to similar unseen document corpora. We use
Q-learning to learn the pipeline. First, we define how to estimate the Q-value
of a state $s$ and action $a$, $\textrm{Q-value}(s,a)$. We define the Q-value
of a state and action as the expected rewards in the future as follows,

$\displaystyle\textrm{Q-value}(s,a)=\mathbb{E}\left[\sum_{t\geq
0}\gamma^{t}r_{t}|s_{0}=s,a_{0}=a,\pi\right]$ (3)

Our aim is to find the maximum cumulative reward achievable with state $s$ and
action $a$: $\textrm{Q-value}^{*}(s,a)=\max\textrm{Q-value}(s,a)$. We estimate
$\textrm{Q-value}^{*}(s,a)$ iteratively using the Bellman equation:

$\displaystyle\textrm{Q-value}_{i+1}(s,a)=\mathbb{E}\left[r+\gamma\max_{a^{\prime}}\textrm{Q-value}_{i}(s^{\prime},a^{\prime})|s,a\right]$
(4)

We use a Fully Connected (FC) neural network to embed each state $s$ and get a
compact representation of it. We combine the embedding layer with the
$\textrm{Q-value}^{*}(s,a)$ estimator to have an end-to-end framework to
summarize the document network.

In our framework, the input state $s$ is fed into the FC neural network. The
output of this step is a compact representation of the state, which is then
fed into another FC that decides how to update the super-nodes. Alg. 1
presents an overview of our summarization algorithm.

Algorithm 1 Summarization

1:$G$, $G^{+}$, $G^{-}$, $k$

2:Randomly Initialize the deep Q-learning parameters

3:// learning how to summarize

4:for episode=1 to $T$ do

5: Initialize $s_{0}:$ Initialize $\\{y_{1},y_{2},\ldots,y_{k}\\}$

6: while $\textsf{Feedback-value}<k$ do

7: // Take a action

8: $a^{*}=\arg\max\;\textrm{Q-value}(s_{current},a)$

9: $s_{current}=\mathcal{T}(s_{current},a^{*})$

10: // Evaluate

11: Evaluate the corresponding partitioning to $s_{current}$ (Eq. 1)

12: // Optimize (see Section 2.2.2)

13: Update deep Q-learning parameters for better summary

14:Return the trained model and super-nodes $\\{y_{1},y_{2},\ldots,y_{k}\\}$

####  2.2.3 Hierarchical Summaries

Our goal is not only to summarize the network but also to provide a multi-
level understanding of the structure. This is specially useful in large
networks, where it is challenging to meaningfully and efficiently generate the
best summary. Hence, we propose a hierarchical approach, where we intuitively,
zoom out one “level” at a time to generate summaries with different sizes.
Specifically, in each step NetReAct tries to partition the data into two
super-nodes, and then iteratively summarizes each part until reaching a
summary with the desired size.

####  2.2.4 Generate Summaries

After learning the best super-nodes of the network, we _merge_ nodes in the
same super-nodes to generate a corresponding super-node. We also connect each
super-node to others by super-edges. The weight of the super-edge from $y_{1}$
to $y_{2}$ is the average similarity between documents in $y_{1}$ to $y_{2}$.
More formally, we define the merge operation as follows,

###### Definition 1

Merge operation merges nodes $v_{1}$, $v_{2}$, …, $v_{b}$ into a new node $y$,
such that $\forall_{j=1,\ldots,b}\;v_{j}\in V$. We add new edge $(y,i)$ for
all the nodes $i\in\underset{j=1\ldots,b}{\bigcup}NB(v_{j})$ with weight
$\frac{\sum_{j=1}^{b}W(v_{j},i)}{b}$.

We merge nodes in the same super-node using Def. 1 to yield the summary
document network $G^{s}(V^{s},E^{s},W^{s})$, where $|V^{s}|=k$.

###  2.3 Two-step Visualization

Once the summary is generated, our goal is to visualize the document network.
We design a multilevel framework that first leverages the weighted force-
directed layout [4] to visualize the summary graph. This gives us a 2D layout
of the summary network, which we treat as the “backbone” of our visualization
process. Note that each super-node consists of a group of nodes, which induce
sub-graphs in the original network. We separately run the weighted force-
directed layout on each sub-graph induced by the super-nodes. Finally, we
“combine” the layouts within each super-node with the backbone layout of the
entire summary network in a multi-level fashion to visualize with entire
network. Lines 3-7 of Alg. 2 show the pseudocode of this two-step
visualization approach.

Algorithm 2 NetReAct-Viz

1:$G$, $G^{+}$, $G^{-}$, $k$, $current_{k}$

2:$y_{1},\ldots y_{k}=$ Hierarchical-Partitioning$(G,G^{+},G^{-},k,1$)

3:$G^{s}\longleftarrow$ merge nodes in $y_{1},\ldots y_{k}$

4:$\forall_{1\leq i\leq k}loc_{y_{i}}\longleftarrow Layout(G^{s})$

5:for super-node $y_{i}\in\\{y_{1},\ldots,y_{k}\\}$ do

6: $G_{i}\longleftarrow$ Corresponding sub-graph of $y_{i}$

7: $\forall_{v\in y_{i}}loc_{v}\longleftarrow Layout(G_{i})$

8: $\forall_{v\in y_{i}}loc_{v}=\frac{loc_{v}}{k}+loc_{y+1}$

9:Return $\forall_{v\in V}loc_{v}$

##  3 Empirical Studies

We used Python and PyTorch to implement all steps of NetReAct and NetReAct-
Viz, and our code is publicly available for academic and research
purposes111http://bit.ly/2MJzpWy. We explore the effectiveness of NetReAct and
NetReAct-Viz on two document network datasets: Crescent [8] is a document
corpus containing synthetic intelligence reports related to terrorist
activities, and the VAST 2007 Challenge dataset (“Blue Iguanodon”) [7]
contains documents regarding wildlife law enforcement subplots. We compare
performance of NetReAct against several baselines including Spectral [14],
Community-Det [6], CoarseNet [10] and Metric-Learning [17] based approaches.

###  3.1  Quality of Summaries

Here, we demonstrate that NetReAct generates high quality network summaries by
both grouping relevant nodes in the same super-node and satisfying the
constraints posed by users as feedback. In this section, we focus on
quantitative results.

We measure the quality of the summary by quantifying the ease of identifying
relevant documents. To that end, we measure the purity of super-nodes that
contain relevant documents. In other words, we calculate the average
probability of observing a relevant document in a super-node that contains at
least one relevant document. Formally,

$\displaystyle\rho=\frac{1}{|V^{s}_{r}|}\sum_{v^{s}\in
V^{s}_{r}}Pr(doc=relevant|v^{s})$ (5)

where $V^{s}_{r}$ is the set of super-nodes that have at least one relevant
document to the scenario, $v^{s}$ is a super-node in the set $V^{s}_{r}$ and
$Pr(doc=relevant|v^{s})$ is the probability that a document is relevant to the
hidden scenario in $v^{s}$. Intuitively, if the value of $\rho$ is closer to
one, it means the user can easily find relevant documents in a selected super-
node.

We investigate the quality of summary networks generated by NetReAct using
Crescent and three subplots of VAST 2007 datasets with 2, 4, 8, and 16 super-
nodes and calculate their $\rho$ values (Eq. 5). In addition, we compare the
quality of NetReAct with baselines. Figs. 1 and 2 show the quality of summary
networks with various numbers of super-nodes. For each experiment, we randomly
selected positive and negative feedback from the ground-truth (see Sec. 2.1).
More specifically, we randomly choose a few pairs of nodes that are relevant
to the hidden story as positive feedback and similarly pick pairs in which
only one of them is relevant as negative feedback. In all experiments, we
fixed the amount of positive and negative feedback at $10\%$ of all possible
positive feedback and $1\%$ of all possible negative feedback.

(a) Crescent

(b) Chinchilla Bio-terror subplot

(c) Bert subplot

(d) Circus subplot

Figure 1: Ratio of satisfied feedback in (a) Crescent and (b-d) different
subplots of the VAST 2007 dataset. Note, NetReAct satisfies all the user
feedback while other baselines do not.

(a) Crescent

(b) Chinchilla Bio-terror subplot

(c) Bert subplot

(d) Circus subplot

Figure 2: Quality of summaries in (a) Crescent and (b-d) different subplots of
the VAST 2007 dataset. Note, NetReAct generates network summaries with the
highest $\rho$.

Fig. 1 shows the ratio of satisfied feedback. The results indicate that
NetReAct generates the highest-quality summary that matches the interests of
the user, as it satisfies all of the user’s feedback while other baselines can
only satisfy part of the feedback. Fig. 2 shows the quality of summaries
$\rho$. NetReAct generates high-quality summaries (i.e highest $\rho$)
networks for various sizes. This implies that users can easily find the
relevant documents to scenarios while interacting with visualization generated
using NetReAct-Viz. It is interesting to mention that because feedback is
sparse and the TF-IDF vectors are high-dimensional, the Metric-Learning
approach is not able to learn proper weights and does not perform well in some
cases. Similarly, the poor performance of other graph summarization approaches
is explained by the fact that they do not consider user feedback.

Figure 3: The network summary evolves with user feedback. The summary of the
Chinchilla subplot of VAST 2007 dataset. Note the black lines represent
positive feedback and dashed lines represent negative feedback. Also, red
nodes represent relevant documents to the scenario and the gray ones are
irrelevant.

###  3.2 Q2. Effect of Feedback

We investigate how NetReAct evolves the summary of a document network while
the user gives positive and negative feedback. We also objectively measure the
change in quality of super-nodes based on the feedback by tracking the changes
in $\rho$. Similar to Sec. 3.1, feedback is randomly generated from the
ground-truth.

To showcase the quality of NetReAct on the VAST 2007 dataset, for each subplot
we extract a subset of documents relevant to the subplot. Fig. 3 shows the
visualization of the Chinchilla Bio-terror subplot. We depict positive
feedback as solid black lines between documents and negative feedback as
dashed lines. Initially our method can only identify four of the related
documents with the subplot and puts them in a super-node. However, the rest of
the relevant documents are mixed with other irrelevant ones in the largest
super-node of the summary network. Next, the user gives feedback regarding the
similarity of two pairs of documents. NetReAct updates the visualization (Fig.
3(b)). However, this is not enough to improve the quality. When the user adds
the negative feedback, NetReAct can distinguish more relevant documents (Fig.
3(c)). Finally, by giving two more positive feedback interactions, NetReAct
can accurately identify the relevant documents with the subplots and puts them
in a separate super-node (Fig. 3(d)).

##  4 Conclusions and Discussion

In this paper, we explored the problem of learning interactive network
summaries with an application of generating multi-level and generalizable
visualization models for text analysis. We proposed a novel and effective
network summarization algorithm, NetReAct, which leverages a feedback-based
reinforcement learning approach to incorporate human input. We also proposed
NetReAct-Viz as a framework to produce a visualization based on hierarchical
network summaries generated by NetReAct. Our experiments show that NetReAct is
able to summarize and NetReAct-Viz is able visualize a document network
meaningfully to reveal hidden stories in the corpus and connect the dots
between documents.

As NetReAct relies on Q-learning, it can be made faster, which is also a
promising direction for future work. As shown by our experiments, it already
works well on real document networks and solves real tasks in practice. In the
future, we plan to apply this interactive network summarization model to much
larger document datasets and temporal data scenarios. Moreover, the
flexibility we obtain from the reinforcement learning approach makes it
possible to bring learning into summarization and enable better generalization
and personalization. For example, we can build a personalized interactive
summarization model for each user to reflect their interests and quickly
summarize different datasets without requiring user input for each new corpus.

Network summarization can lead to other meaningful visualizations by
incorporating more diverse semantic interactions into the reinforcement
learning approach. For example, we would like to explore how to differentiate
between highlighting, overlapping, and annotating documents in our framework.
Also, leveraging more visual encodings to create a more understandable and
user-friendly summarization is a fruitful direction. We can explore using our
approach for summarizing and visualizing other data types such as social
networks and images as well. Our approach here opens several additional
interesting avenues for future work.

Acknowledgements This paper is based on work partially supported by the NSF
(Expeditions CCF-1918770, CAREER IIS-2028586, RAPID IIS-2027862, Medium
IIS-1955883, NRT DGE-1545362), NEH (HG-229283-15), CDC MInD program, ORNL,
funds/computing resources from Georgia Tech and a Facebook faculty gift.

## References

  * [1] B. Adhikari, Y. Zhang, S. E. Amiri, A. Bharadwaj, and B. A. Prakash.  Propagation-based temporal network summarization.  IEEE Transactions on Knowledge and Data Engineering, 30(4):729–742, 2018.
  * [2] S. E. Amiri, L. Chen, and B. A. Prakash.  Efficiently summarizing attributed diffusion networks.  Data Mining and Knowledge Discovery, 32(5):1251–1274, 2018.
  * [3] L. Bradel, C. North, L. House, and S. Leman.  Multi-model semantic interaction for text analytics.  In 2014 IEEE Conference on Visual Analytics Science and Technology, VAST 2014, Paris, France, October 25-31, 2014, pages 163–172, 2014\.
  * [4] T. Dwyer and Y. Koren.  Dig-cola: directed graph layout through constrained energy minimization.  In INFOVIS. IEEE, 2005.
  * [5] A. Endert, P. Fiaux, and C. North.  Semantic interaction for sensemaking: Inferring analytical reasoning for model steering.  IEEE Trans. Vis. Comput. Graph., 18(12):2879–2888, 2012.
  * [6] M. Girvan and M. E. Newman.  Community structure in social and biological networks.  Proceedings of the national academy of sciences, 99(12):7821–7826, 2002.
  * [7] G. Grinstein, C. Plaisant, S. Laskowski, T. O’Connell, J. Scholtz, and M. Whiting.  Vast 2007 contest-blue iguanodon.  In 2007 IEEE Symposium on Visual Analytics Science and Technology, pages 231–232. IEEE, 2007.
  * [8] F. Hughes and D. Schum.  Discovery-proof-choice, the art and science of the process of intelligence analysis-preparing for the future of intelligence analysis.  Joint Military Intelligence College, Washington, DC., 2003.
  * [9] G. Karypis and V. Kumar.  Multilevel algorithms for multi-constraint graph partitioning.  In Supercomputing, 1998. SC98. IEEE/ACM Conference on, pages 28–28. IEEE, 1998.
  * [10] M. Purohit, B. A. Prakash, C. Kang, Y. Zhang, and V. Subrahmanian.  Fast influence-based coarsening for large networks.  In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1296–1305. ACM, 2014.
  * [11] N. Shah, D. Koutra, T. Zou, B. Gallagher, and C. Faloutsos.  Timecrunch: Interpretable dynamic graph summarization.  In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1055–1064. ACM, 2015.
  * [12] F. M. Shipman and C. C. Marshall.  Formality considered harmful: Experiences, emerging themes, and directions on the use of formal representations in interactive systems.  Computer Supported Cooperative Work (CSCW), 8(4):333–352, 1999\.
  * [13] R. S. Sutton and A. G. Barto.  Introduction to reinforcement learning, volume 135.  MIT press Cambridge, 1998.
  * [14] U. Von Luxburg.  A tutorial on spectral clustering.  Statistics and computing, 17(4):395–416, 2007.
  * [15] C. J. Watkins and P. Dayan.  Q-learning.  Machine learning, 8(3-4):279–292, 1992.
  * [16] J. J. Whang, I. S. Dhillon, and D. F. Gleich.  Non-exhaustive, overlapping _k_ -means.  In Proceedings of the 2015 SIAM International Conference on Data Mining, Vancouver, BC, Canada, April 30 - May 2, 2015, pages 936–944, 2015\.
  * [17] E. P. Xing, M. I. Jordan, S. J. Russell, and A. Y. Ng.  Distance metric learning with application to clustering with side-information.  In Advances in neural information processing systems, pages 521–528, 2003.