11institutetext: National Yang Ming Chiao Tung University, Taiwan
11email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>National Tsing Hua University, Taiwan
22email<EMAIL_ADDRESS><EMAIL_ADDRESS>National Chengchi University, Taiwan
33email<EMAIL_ADDRESS>Qualcomm Technologies, Inc.,
San Diego
44email<EMAIL_ADDRESS>
# Domain-adaptive Video Deblurring via Test-time Blurring

Jin-Ting He ** 1 1 Fu-Jen Tsai ** 2 2 Jia-Hao Wu 11 Yan-Tsung Peng 33 Chung-
Chi Tsai 44 Chia-Wen Lin 22 Yen-Yu Lin 11

###### Abstract

Dynamic scene video deblurring aims to remove undesirable blurry artifacts
captured during the exposure process. Although previous video deblurring
methods have achieved impressive results, they suffer from significant
performance drops due to the domain gap between training and testing videos,
especially for those captured in real-world scenarios. To address this issue,
we propose a domain adaptation scheme based on a blurring model to achieve
test-time fine-tuning for deblurring models in unseen domains. Since blurred
and sharp pairs are unavailable for fine-tuning during inference, our scheme
can generate domain-adaptive training pairs to calibrate a deblurring model
for the target domain. First, a Relative Sharpness Detection Module is
proposed to identify relatively sharp regions from the blurry input images and
regard them as pseudo-sharp images. Next, we utilize a blurring model to
produce blurred images based on the pseudo-sharp images extracted during
testing. To synthesize blurred images in compliance with the target data
distribution, we propose a Domain-adaptive Blur Condition Generation Module to
create domain-specific blur conditions for the blurring model. Finally, the
generated pseudo-sharp and blurred pairs are used to fine-tune a deblurring
model for better performance. Extensive experimental results demonstrate that
our approach can significantly improve state-of-the-art video deblurring
methods, providing performance gains of up to 7.54dB on various real-world
video deblurring datasets. The source code is available at
https://github.com/Jin-Ting-He/DADeblur.

###### Keywords:

Video deblurring Domain adaptation Diffusion model

††footnotetext: * equal contribution

##  1 Introduction

Videos captured in dynamic scenes often appear blurred and have blurry
artifacts due to camera shaking or moving objects captured during the exposure
process. The unwanted blur severely degrades visual qualities and reduces the
accuracy of subsequent computer-vision applications. Dynamic scene video
deblurring aims to restore such blurred videos, considered a highly ill-posed
problem due to the unknown varying blurs.

Video deblurring has made remarkable progress with the development of deep
learning. Numerous methods [36, 22, 64, 63, 60, 56, 50, 3, 16, 17, 34, 48, 25]
have applied Convolutional Neural Networks (CNNs) for video deblurring. Among
these methods, Recurrent Neural Network-based (RNN-based) methods [16, 3, 56,
63] are commonly employed to extract spatio-temporal features. To better
capture motion cues in videos, several methods [36, 22, 48] compute optical
flows to get additional motion information for alignment. In addition,
motivated by the success of Transformer in computer vision [11, 2, 43, 8, 4,
65, 58], Transformer-based methods [26, 25, 28] have been proposed to model
long-range, spatio-temporal dependency. Since these methods have been
dedicated to improving deblurring through architectural designs, their
performance often drops substantially when a domain gap exists between
training and testing videos.

Figure 1: Illustration of the proposed domain adaptation method. It can
generate domain-specific blur conditions for a blurring model to produce
blurred video frames from the chosen pseudo-sharp patch. These blurred frames
can be used to fine-tune video deblurring models and improve their performance
in the target domain.

In real-world cases, cameras may capture blurry videos due to different
settings and scenarios, such as shutter speed, aperture, or light sources,
leading to different blurry patterns with various orientations and magnitudes.
Previous deblurring models often face challenges when dealing with blurry
patterns not seen during training. However, few methods discuss the domain gap
issue in video deblurring. Liu _et al_. [29] attempted to address it via a
blurring network that generates blurred images to enable meta-learning [12]
for adaptation. Even though it could generate blurred data for adaptation, it
did not consider helpful temporal information that exists in consecutive
frames for domain adaptation. That is, continuous motion across consecutive
frames reveals motion blur trajectories during capturing, and the degree of
blurriness implies the blur intensities during exposure. The information
implicitly conveys domain-specific cues regarding blur orientations and
magnitudes in unseen blurry videos. Therefore, motivated by this observation,
we further explore the cues to enhance video deblurring models in unseen
domains.

In this paper, we propose a domain adaptation method for video deblurring,
which relies on a blurring model to generate domain-adaptive training pairs
for adaptation. The generated data can be used to calibrate deblurring models
in unseen domains. The proposed framework is illustrated in Figure 1. Since
blurred and sharp pairs are unavailable for fine-tuning during inference, our
scheme can extract relatively sharp regions from blurry videos using the
proposed Relative Sharpness Detection Module (RSDM). These relatively sharp
regions can be considered pseudo-sharp images. Inspired by the recent
diffusion-based blurring method, ID-Blau [57], which can generate blurred
images based on a sharp image and arbitrarily specified blur conditions, we
adopt ID-Blau as the blurring backbone. Nevertheless, randomly generated blur
conditions are not consistent with blur patterns present in test videos.
Directly applying ID-Blau with such conditions for blurring and fine-tuning a
deblurring model may not help. Considering blurry videos implicitly provide
coherent motion blur cues, we propose a Domain-adaptive Blur Condition
Generation Module (DBCGM) containing a Blur Orientation Estimator and Blur
Magnitude Estimator to create domain-specific blur conditions tailored for
blurring those pseudo-sharp images using ID-Blau. It turns out that the
generated domain-specific blur pairs can be used to fine-tune deblurring
models, thereby achieving domain adaptation during inference. Our
contributions can be summarized as follows:

  * •

We propose a test-time domain adaptation method for video deblurring based on
ID-Blau, which can generate domain-specific blur conditions to achieve test-
time fine-tuning for deblurring models.

  * •

We propose a Relative Sharpness Detection Module to detect relatively sharp
regions as pseudo-sharp images and a Domain-adaptive Blur Condition Generation
Module to generate domain-specific blur conditions for blurring those pseudo-
sharp images.

  * •

Experimental results demonstrate that our method can significantly enhance
state-of-the-art video deblurring models [63, 56, 21, 37] on five real-world
video deblurring datasets, including RealBlur [44], RBVD [3] and three
versions of BSD [63].

##  2 Related Work

####  2.0.1 Video Deblurring

Convolutional neural networks have significantly advanced the video deblurring
task. Compared to image deblurring [32, 51, 59, 13, 49, 38, 39, 19, 53, 54],
video deblurring [36, 22, 64, 63, 60, 56, 50, 3, 16, 17, 34, 48, 25] can
utilize spatial and temporal information from videos to achieve better
performance. Recently, several methods adopted RNN-based models [63, 56, 3,
26, 34, 37, 21] to leverage spatio-temporal information in videos. Zhong  _et
al_. [63] proposed an efficient spatio-temporal RNN architecture to catch
spatially and temporally varying blurs with RNN cells. Wang  _et al_. [56]
utilized a motion magnitude prior as guidance to improve deblurring
performance. Pan  _et al_. [37] developed wavelet-based feature propagation to
transfer features in frequency space recurrently. Li  _et al_. [21] proposed a
grouped spatial-temporal shift operation to aggregate spatio-temporal features
efficiently.

In addition to progressively propagating features through RNNs, several
studies have explored Transformer-based architectures [25, 28, 26] to garner
long-range information for deblurring. Liang  _et al_. [26] proposed a
recurrent video restoration transformer with guided deformable attention. Lin
_et al_. [28] introduced a flow-guided sparse transformer that leverages
optical flows to guide the transformer’s attention mechanism. Although these
architectural designs for RNNs and Transformers were used to improve video
deblurring, they often do not perform up to par when dealing with blurry
videos on unseen domains, especially for videos captured in the real world.

####  2.0.2 Domain Adaptation

Domain adaptation aims to bridge the domain gap between the training set and
testing set, which has been widely discussed in vision tasks, such as object
detection [62, 24, 23] and semantic segmentation [5, 20, 15]. Nevertheless,
little work has been done [6, 33, 29] to address the domain gap issue in
deblurring, especially for video deblurring [29]. Since we solely have blurred
inputs during inference, no ground truth can be used for test-time deblurring
calibration. Some methods apply self-supervised strategies to enable fine-
tuning during testing. Chi  _et al_. [6] utilized a reconstruction branch to
reset its deblurred result to the original blurry input. Nah  _et al_. [33]
employed a fixed reblurring model as a reblurring loss to supervise the
deblurring result. Although these methods could update and improve deblurring
models through self-supervised learning, they ignore domain-specific blur
information in the test domain, which is beneficial to deblurring performance
in unseen domains. To generate blurred images with blur characteristics on
unseen domains, Liu  _et al_. [29] utilized a GAN to supervise a blurring
model to generate blurred images from sharp ones for fine-tuning. In contrast,
our method explores domain-specific blur cues from consecutive testing video
frames for domain adaptation.

####  2.0.3 Diffusion Models

Diffusion models have demonstrated a strong ability to synthesize realistic
images in image generation [14, 47, 10]. Based on the diffusion model, several
studies [35, 46, 42, 55, 31, 45, 30, 27, 61, 57] have utilized it with various
conditions to generate controllable results. Some methods [35, 46] used text
prompts [9, 41, 40] in text-to-image generation. Although text embedding
provides additional information to generate controllable results, it often has
a limitation in spatially guiding the generated images. Recently, several
studies proposed to spatially guide diffusion models with various prompts [55,
31, 45, 30, 27, 61, 57]. Zhang  _et al_. [61] proposed Control-Net to fine-
tune a pre-trained diffusion model with various prompts by zero convolution
layers, making the diffusion model spatially controllable without requiring
high computational costs. Liang  _et al_. [27] proposed a controllable multi-
modal image coloring method based on a content-guided deformable autoencoder.
Wu  _et al_. [57] proposed ID-Blau, a diffusion-based blurring model that can
generate diverse blurred images based on a sharp image and arbitrarily-
specified pixel-wise blur conditions. Motivated by the success of ID-Blau, we
adopt ID-Blau as a blurring model to generate blurred images during testing.
However, simply applying ID-Blau with randomly generated blur conditions would
not improve a deblurring model for blurred images coming from different
domains. Therefore, we aim to leverage the inherent motion blur cues in
testing videos to generate domain-specific blur conditions for adaptation,
thereby significantly boosting deblurring performance on unseen domains.

Figure 2: Pipeline of the proposed domain adaptation scheme. Given a blurred
video in the test domain, we use the Relative Sharpness Detection Module
(RSDM) to extract relatively sharp patches to be pseudo-sharp patches and the
Domain-adaptive Blur Condition Generation Module (DBCGM) to generate domain-
specific blur conditions for blurring the pseudo-sharp patches using ID-Blau
[57]. Finally, the pseudo-sharp and blurred pairs are used to update a
deblurring model for domain adaptation.

##  3 Proposed Method

We proposed a domain adaptation scheme for video deblurring based on ID-Blau
[57], a diffusion-based blurring model that utilizes controllable blur
conditions to generate corresponding blurred images. As shown in Figure 2,
considering blurred and sharp pairs are unavailable during inference, we
generate domain-adaptive training pairs from blurred videos to calibrate a
deblurring model for the target domain. First, we propose a Relative Sharpness
Detection Module (RSDM) to extract relatively sharp patches from blurred
videos. These patches are treated as pseudo-sharp images, which are then
blurred using ID-Blau to create pseudo-training pairs for updating deblurring
models. In ID-Blau, a blur pattern is represented using pixel-wise blur
orientations and magnitudes as a blur condition in a continuous blur condition
field. Simply using ID-Blau with randomly sampled blurred conditions to
produce blurred images may not help a deblurring model in unseen domains.
Therefore, we proposed a Domain-adaptive Blur Condition Generation Module
(DBCGM) to create domain-specific blur conditions for ID-Blau. It allows ID-
Blau to generate blurred images adaptive to the target domain. Finally, the
generated pseudo-sharp and blurred pairs are used to fine-tune a deblurring
model for better performance.

###  3.1 Relative Sharpness Detection Module (RSDM)

The proposed RSDM aims to search for relatively sharp patches in blurred
videos. These patches are regarded as pseudo-sharp patches for domain
adaptation. To achieve this, we propose a Blur Magnitude Estimator (BME) to
predict a blur magnitude map for a blurred image, where the blurriness degree
for each pixel is estimated.

As shown in Figure 3, the BME is a five-stage encoder-decoder network combined
with Multi-Scale Feature Fusion (MSFF) proposed in [7], as

$\tilde{E}_{k}=\begin{cases}MSFF(E_{k-1},E_{k},E_{k+1})&\text{if }k=1,2,3,\\\
MSFF(E_{k},E_{k+1},E_{k+2})&\text{if }k=0,\\\
MSFF(E_{k},E_{k-1},E_{k-2})&\text{if }k=4,\end{cases}$ (1)

where $E_{k}$ is the output of the $k$-th encoder layer and $\tilde{E}_{k}$ is
its fusion result. MSFF includes a $3\times 3$ convolutional layer plus
resizing all the input features to the size of $E_{k}$. After resizing, the
multi-scale features are concatenated, followed by a $1\times 1$ and a
$3\times 3$ convolutional layers.

To optimize the BME, we choose the GoPro [32] dataset, which synthesizes
blurred images by accumulating consecutive sharp frames from a high-speed
camera as

$B=g(\frac{1}{T}\int_{t=1}^{T}H(t)dt)\simeq
g(\frac{1}{N_{s}}\sum_{n=1}^{N_{s}}H\left[n\right]),$ (2)

where $B$, $H$, $T$, $N_{s}$, and $g$ respectively denote the generated
blurred image, the sharp images captured by a high-speed camera, the exposure
time, the number of sampled sharp images, and the camera response function.
Here, the center sharp image $H[\frac{N+1}{2}]$ is chosen to be the ground-
truth sharp image $S$ corresponding to the generated blurred image $B$.

Figure 3: The left figure shows the architecture of the Blur Magnitude
Estimator (BME), which comprises a five-stage encoder-decoder design with
Multi-Scale Feature Fusion (MSFF). The right figure is the architecture of
MSFF. $E_{k}$ denotes the output of the encoder at stage $k$, and
$\tilde{E}_{k}$ denotes its output after MSFF.  Figure 4: Illustration of the
Relative Sharpness Detection Module. We use the Blur Magnitude Estimator to
obtain the blur magnitudes for a blurred image and crop a relatively sharp
patch based on an adaptive sharpness threshold.

Considering continuous motion during the exposure process causes various
degrees of blurriness, we characterize the continuous exposure as motion
trajectories by accumulating optical flows from the sharp image sequence
$\textbf{H}=\\{H\left[1\right],...,H\left[N\right]\\}$ and aggregating them as

$\mathcal{F}=\sum_{n=1}^{N-1}\frac{f(H\left[n\right],H\left[n+1\right])-f(H\left[n\right],H\left[n-1\right])}{2},$
(3)

where $f$ is a pre-trained optical flow network. Here, we adopt the method in
[52] for obtaining the optical flows. The calculated motion trajectory map has
two components: $u$ and $v$, representing the average horizontal and vertical
motion trajectories as $\mathcal{F}=[u;v]\in\mathbb{R}^{H\times W\times 2}$.
Lastly, the blur magnitude map $G\in\mathbb{R}^{H\times W}$ corresponding to
$B$ can be obtained by

$G=\frac{1}{\tau}\sqrt{u^{2}+v^{2}},$ (4)

where $\tau$ serves as a normalization term, set to the maximum value of the
blur magnitudes, and thus $G\in[0,1]$. To this end, a dataset
$\\{B_{k},G_{k}\\}^{K}_{k=1}$ is constructed to train the BME, where $K=2,103$
for the GoPro training set.

Next, we take the trained BME to obtain the blur magnitudes for every frame in
the testing blurred video to sift through relatively sharp patches. As shown
in Figure 4, given a blurred video $V^{(i)}$ in the test set, BME predicts a
blur magnitude map $M^{(i)}_{t}\in\mathbb{R}^{H\times W}$ for each blurred
frame $V^{(i)}_{t}$ as

$M^{(i)}_{t}=BME(V^{(i)}_{t}),$ (5)

where $t$ denotes the frame index. To crop a relatively sharp patch from
$V^{(i)}_{t}$, we use an adaptive sharpness threshold $\eta^{(i)}$ to binarize
$M^{(i)}_{t}$ with magnitude larger than $\eta^{(i)}$ set to $1$ and obtain
the sharp mask. We then crop a sharp patch by the regionprop library [1] based
on the mask to generate a relatively sharp patch $\tilde{S}^{(i)}_{t}$ of size
$256\times 256$. Here, $\eta^{(i)}$ is determined to ensure that the number of
relatively sharp patches extracted from the video $V^{(i)}$ reaches $r\%$ of
the number of total frames in $V^{(i)}$, meaning the top $r\%$ relatively
sharp patches in the video. In our work, $r$ is set to $20$.

Figure 5: Illustration of Domain-adaptive Blur Condition Generation Module.
Given a pseudo-sharp patch and the collocated patches in its neighboring
frames, we use the Blur Orientation Estimator to generate domain-specific blur
orientations and the Blur Magnitude Estimator to generate domain-specific blur
magnitudes. The blur magnitudes estimated from neighboring patches are used to
modulate the pseudo-sharp patch by the Magnitude Adaptation Process. In the
end, the domain-specific blur orientations and magnitudes are used for
blurring.

###  3.2 Domain-adaptive Blur Condition Generation Module (DBCGM)

With the selected pseudo-sharp patches, we utilize a blurring model to
generate their blurred versions for fine-tuning a deblurring model. In this
work, we use ID-Blau [57], a conditional diffusion-based blurring model that
takes a sharp image $S$ and a controllable blur condition map
$C=(x,y,z)\in\mathbb{R}^{H\times W\times 3}$ to synthesize a blurred image $B$
as

$B=\textbf{ID-Blau}(S,C),$ (6)

where $x$, $y$, and $z\in\mathbb{R}^{H\times W}$ denote horizontal and
vertical blur orientations, and blur magnitudes, respectively. Although
randomly generated blur conditions can be used to produce blurred images by
ID-Blau, they do not conform to the test data distribution in the target
domain, which means the generated blurred images do not agree with the blur
patterns that exist in test videos, leading to little performance gain for
deblurring models. Therefore, as shown in Figure 5, we propose a Domain-
adaptive Blur Condition Generation Module (DBCGM) that can generate domain-
specific blur conditions by leveraging motion cues implicitly provided in
blurred videos during inference.

In a blurred video, continuous motion across consecutive frames reveals motion
blur trajectories during capturing, and the degree of blurriness implies the
blur intensities during exposure. Therefore, the proposed DBCGM includes a
Blur Orientation Estimator (BOE) and the previously mentioned BME to generate
domain-specific blur conditions for blurring. First, given a pseudo-sharp
patch $\tilde{S}^{(i)}_{t}$ in the $t$-th frame of the $i$-th video, we take
it and its collocated patches from the two previous and two next frames
$\\{\tilde{S}^{(i)}_{t-2},...,\tilde{S}^{(i)}_{t+2}\\}$ to calculate the
motion trajectory map
${\tilde{F}^{(i)}}_{t}=[\tilde{u};\tilde{v}]\in\mathbb{R}^{H\times W\times 2}$
as

${\mathcal{\tilde{F}}^{(i)}}_{t}={\sum_{n=-2}^{1}f(\tilde{S}^{(i)}_{t+n},\tilde{S}^{(i)}_{t+n+1})},$
(7)

where $f$ denotes the pre-trained optical flow estimator [52]. Next, we obtain
the domain-specific blur orientations
$\tilde{O}^{(i)}_{t}\in\mathbb{R}^{H\times W\times 2}$ tailored for
$\tilde{S}^{(i)}_{t}$ as

$\tilde{O}^{(i)}_{t}=\frac{\mathcal{\tilde{F}}^{(i)}_{t}}{\sqrt{\tilde{u}^{2}+\tilde{v}^{2}}}.$
(8)

To generate domain-adaptive blur magnitudes for $\tilde{S}^{(i)}_{t}$, we
first estimate blur magnitudes of $\tilde{S}^{(i)}_{t}$ by the BME as
${M}^{(i)}_{t}=BME(\tilde{S}^{(i)}_{t})$. Since $\tilde{S}^{(i)}_{t}$ is
relatively sharp, we intend to modulate its blur magnitudes by considering
blur patterns rendered in its neighboring collocated patches
$\\{\tilde{S}^{(i)}_{t-2},\tilde{S}^{(i)}_{t-1},\tilde{S}^{(i)}_{t+1},\tilde{S}^{(i)}_{t+2}\\}$.
To achieve this, we use the average blur magnitudes of neighboring collocated
patches to adjust ${M}^{(i)}_{t}$ by the Magnitude Adaptation Process as

$\tilde{M}^{(i)}_{t}=\text{Norm}(M^{(i)}_{t})\cdot\text{Avg}(M^{(i)}_{t-2},M^{(i)}_{t-1},M^{(i)}_{t+1},M^{(i)}_{t+2}),$
(9)

where $M^{(i)}_{t}$ is normalized in the range of $[0,1]$ and multiplied by
the blur magnitude average to generate domain-specific blur magnitudes
$\tilde{M}^{(i)}_{t}\in\mathbb{R}^{H\times W}$. At last, a domain-specific
blur condition $\tilde{C}^{(i)}_{t}$ that combines blur orientations
$\tilde{O}^{(i)}_{t}\in\mathbb{R}^{H\times W\times 2}$ and magnitudes
$\tilde{M}^{(i)}_{t}\in\mathbb{R}^{H\times W}$ is used to blur
$\tilde{S}^{(i)}_{t}$ by

$\tilde{B}^{(i)}_{t}=\textbf{ID-
Blau}(\tilde{S}^{(i)}_{t},\tilde{C}^{(i)}_{t}),$ (10)

where $\tilde{B}^{(i)}_{t}$ is the generated blurred patch. The pseudo-
training pair $\\{\tilde{B}^{(i)}_{t},\tilde{S}^{(i)}_{t}\\}$ is then utilized
to update a deblurring model for domain adaptation. Ultimately, we collect
these pairs from the total $N$ blurred videos in the target domain to fine-
tune a deblurring model for domain adaptation.

####  3.2.1 Loss Function:

In the proposed scheme, we only need to optimize the BME. We use the L1 loss,

$\mathcal{L}=\mathcal{L}_{\mathrm{1}}(M,G),$ (11)

where $M$ is the predicted blur magnitudes, and $G$ is the calculated blur
magnitudes using Equation 4, used as the ground truth.

††footnotetext: The authors from the universities in Taiwan completed the
experiments on the datasets.

##  4 Experiments

###  4.1 Implementation Details

####  4.1.1 Blurring Model

We choose a diffusion-based blurring ID-Blau [57] as the blurring network to
generate blurred images. Note that we use ID-Blau with its original training
settings. For optimizing the BME, we use the Adam optimizer [18] with the
initial learning rate of $1e^{-3}$, gradually decayed to $1e^{-4}$ by the
cosine annealing strategy. We resize images to $320\times 320$ and adopt
random flipping and rotation for data augmentation with a batch size of $16$
for training the model $50$ epochs. We use the GoPro training set [32], which
contains $22$ training videos with $2,103$ blurred and sharp image pairs for
optimizing ID-Blau and BME.

####  4.1.2 Video Deblurring Models

We adopt four state-of-the-art video deblurring models, including ESTRNN [63],
MMP-RNN [56], DSTNet [37], and Shift-Net [21] to validate the effectiveness of
the proposed domain adaptation method. We regard the GoPro training set as the
source domain and five real-world deblurring datasets as target domains,
including BSD-1ms8ms [63], BSD-2ms16ms [63], BSD-3ms24ms [63], RealBlur [44],
and RBVD [3] test sets. The BSD test set has three subsets: 1ms8ms, 2ms16ms,
and 3ms24ms, indicating two exposure times: the former of which is for sharp
images and the latter of which is for blurred ones. Each subset contains $20$
videos with $3,000$ blurred images. The RealBlur test set consists of $50$
videos with $980$ blurred images, taken in low-light environments. The RBVD
test dataset consists of $7$ videos with $246$ blurred images, encompassing
outdoor scenes, indoor scenes, and high-frequency charts. In the domain
adaptation process, we fine-tune each deblurring model for $10$ epochs on
pseudo-training pairs generated by the proposed RSDM and DBCGM, as we use each
model’s original loss functions for training.

Table 1: Evaluation results on the deblurring datasets, including BSD [63], RealBlur [44], and RVBD [3], where “Baseline” and “+Ours” denote the video deblurring performances w/o or w/ our domain adaptation method. |  | BSD-1ms8ms | BSD-2ms16ms | BSD-3ms24ms | RealBlur | RBVD
---|---|---|---|---|---|---
Model | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM | PSNR | SSIM
ESTRNN | Baseline | 25.57 | 0.747 | 24.64 | 0.726 | 26.01 | 0.748 | 25.87 | 0.773 | 24.47 | 0.725
+Ours | 29.44 | 0.843 | 28.36 | 0.820 | 28.32 | 0.810 | 27.64 | 0.816 | 26.83 | 0.764
MMP-RNN | Baseline | 21.63 | 0.620 | 21.26 | 0.605 | 22.74 | 0.597 | 24.65 | 0.639 | 22.81 | 0.780
+Ours | 29.17 | 0.797 | 26.95 | 0.750 | 26.77 | 0.707 | 27.69 | 0.663 | 25.81 | 0.822
DSTNet | Baseline | 25.42 | 0.821 | 23.50 | 0.760 | 24.68 | 0.788 | 26.57 | 0.750 | 23.15 | 0.768
+Ours | 28.69 | 0.868 | 27.11 | 0.830 | 26.69 | 0.838 | 27.74 | 0.798 | 25.66 | 0.808
Shift-Net | Baseline | 25.00 | 0.837 | 23.75 | 0.807 | 24.98 | 0.819 | 26.01 | 0.797 | 23.98 | 0.870
+Ours | 28.75 | 0.888 | 26.31 | 0.854 | 26.92 | 0.852 | 27.71 | 0.854 | 25.35 | 0.891

###  4.2 Experimental Results

####  4.2.1 Quantitative Analysis

We compare the performance of four video deblurring models, including ESTRNN
[63], MMP-RNN [56], DSTNet [37], and Shift-Net [21], on five real-world
deblurring datasets with (+Ours) or without (Baseline) using the proposed
domain adaptation scheme in Table 1. It shows that our method can consistently
and significantly improve the performance for these video deblurring models by
$4.61$dB, $3.90$dB, $2.57$dB, $1.92$dB, and $2.31$dB in PSNR on average on the
BSD-1ms8ms, BSD-2ms16ms, BSD-3ms24ms, RealBlur and RBVD test sets,
respectively. Particularly, our method can obtain up to $3.87$dB, $7.54$dB,
$3.61$dB, and $3.75$dB performance gains for deblurring models: ESTRNN, MMP-
RNN, DSTNet, and Shift-Net, respectively. These experimental results show that
our method can generate domain-adaptive training pairs to effectively fine-
tune deblurring models originally trained with a synthetic dataset for those
real-world blurred videos.

####  4.2.2 Qualitative Analysis

In Figure 6, we compare deblurred results with or without using our method on
several blurred video frames chosen from the BSD test set. Our domain-adaptive
scheme can significantly improve the visual quality of the deblurred results,
exemplified by the regions of the face, text, and pedestrians. Figure 6 shows
comparisons of the deblurred results with or without using our method on
blurred examples on the RBVD and RealBlur test sets. The visual quality of
deblurred results using our method is significantly better, exemplified by the
regions of the pillar, text, and building. These visualization results show
that our method can truly help boost the deblurring performance compared to
the baseline methods. It also attests to the effectiveness of the proposed
domain adaptation scheme to produce domain-adaptive training pairs for model
fine-tuning in the target domain.

Figure 6: Qualitative results on the BSD datasets, RBVD dataset and RealBlur
dataset with DSTNet(top left), MMP-RNN(top right), Shift-Net(bottom left), and
ESTRNN(bottom right).

###  4.3 Ablation studies

To analyze the impact of the proposed components in our domain adaptation
method, we adopt ESTRNN [63] as the tested deblurring model and analyze its
deblurring performances on the BSD-1ms8ms dataset with various ablative
settings. First, we discuss the effectiveness of the RSDM and DBCGM. Next, we
discuss the designs in RSDM and DBCGM. Lastly, we compare the proposed method
with the existing domain adaptation method [29] for video deblurring.

Table 2: Ablations on the effectiveness of the RSDM and DBCGM.  | Pseudo-Sharp Patches | Blur Condition Generation | PSNR | GAIN
---|---|---|---|---
| Random | RSDM(Ours) | Random | Optical-Flow | DBCGM(Ours) |  |
(a) |  |  |  |  |  | 25.57 | +0.00
(b) | ✓ |  | ✓ |  |  | 23.88 | -1.69
(c) | ✓ |  |  | ✓ |  | 25.51 | -0.06
(d) | ✓ |  |  |  | ✓ | 29.01 | +3.44
(e) |  | ✓ | ✓ |  |  | 24.32 | -1.25
(f) |  | ✓ |  | ✓ |  | 26.19 | +0.62
(g) |  | ✓ |  |  | ✓ | 29.44 | +3.87

####  4.3.1 Effects of RSDM and DBCGM

Table 2 compares the effectiveness of the RSDM and DBCGM to the video
deblurring performance. Table 2(a) shows the deblurred results on BSD-1m8ms
using the GoPro pre-trained model, considered the baseline setting. First, we
analyze the effectiveness of the proposed DBCGM with randomly selected patches
in blurred videos to be pseudo-sharp patches (Table 2(b), (c), and (d)). To
blur pseudo-sharp patches using ID-Blau, we can use randomly sampled blur
conditions (Table 2(b)) or take optical flows as blur conditions (Table 2(c)).
As seen, these two cases do not improve the performance at all. In contrast,
using the proposed DBCGM (Table 2(d)) can largely improve the baseline by
$3.44$dB, which demonstrates that even with random patches cropped from
blurred video frames, the deblurring model can still exploit domain-specific
blur conditions generated by the DBCGM to achieve better performance. Next, we
analyze the effectiveness of the proposed RSDM. In Table 2, we can compare the
settings (b) vs. (e), (c) vs. (f), and (d) vs. (g), which shows that using the
RSDM with different blur condition generation methods can all boost the
deblurring model. Most of all, the combination of the RSDM and DBCGM achieves
the best performance, achieving a $3.87$dB gain compared to the baseline. In
addition, we visualize the blurred results in Figure 7 using different blur
condition generation methods. It shows that the proposed DBCGM can generate
domain-specific blurred images more realistic and consistent with video frames
of a moving scene than optical flows or randomly sampled blur conditions.

Figure 7: Visualization of blurred images obtained using ID-Blau with
different blur condition generation methods: DBCGM, optical flows, and random
sampling.  Figure 8: Example of a blurred image generated from a blurry image
using our method.

####  4.3.2 Effects of the adaptive sharpness threshold for the RSDM

The adaptive sharpness threshold $\eta^{(i)}$ is determined for the video
$V^{(i)}$ in the RSDM to pick the top $r\%$ relatively sharp patches. Choosing
a smaller ratio can result in sharper patches to be pseudo-sharp patches.
However, it may not provide sufficient training data for adaptation. In
contrast, selecting a larger ratio would include more blurred patches as
pseudo-sharp patches, leading to inferior pseudo-training pairs generated,
where those blurred patches are too similar to their blurred versions.
Therefore, we experiment to analyze the effect of using different ratios $r\%$
on the deblurring performance for adaptation. Figure 9(a) shows the deblurring
performance of using different ratios from $10\%$ ($3,000\times 10\%=300$
patches) to $60\%$ ($3,000\times 60\%=1,800$ patches). As seen, setting $r=20$
achieves the best performance. Choosing a larger ratio would include more
inferior training pairs, providing less help for deblurring. In addition, we
conduct another experiment by using patches in different ratio ranges for
blurring, including $0\%-20\%$, $20\%-40\%$, $40\%-60\%$, and $60\%-80\%$. For
example, $20\%-40\%$ means we select the top $40\%$ relatively sharp patches
but exclude the top $20\%$ ones. Based on Figure 9(b), using patches with more
blur to generate pseudo-training pairs is not as beneficial as those with less
blur to obtain quality pairs. That said, compared to the baseline,
intentionally selecting patches with more blur in the proposed domain
adaptation scheme still improves the deblurring performance. Figure 8 shows a
blurred example using a patch with much blur.

(a) Performance of using different ratios for the threshold in RSDM

(b) Performance of using different ratio ranges for picking pseudo-sharp
patches

Figure 9: Illustration of the PSNR results when we set different threshold for
RSDM.

###  4.4 Comparison with an existing domain adaptation method

Considering little work regarding domain adaptation for video deblurring, we
only compare the proposed method with [29], which uses meta-learning to
achieve domain adaptation. Table 3 shows that our method significantly
outperforms [29] on all the benchmark datasets.

Table 3: Comparison between our method and Liu  _et al_. [29]. Model | BSD-1ms8ms | BSD-2ms16ms | BSD-3ms24ms | RealBlur | RBVD
---|---|---|---|---|---
Baseline | 25.57 | 24.64 | 26.01 | 25.87 | 24.47
Liu  _et al_. [29] | 25.58 | 24.53 | 25.15 | 26.12 | 24.83
Ours | 29.44 | 28.36 | 28.32 | 27.64 | 26.83

##  5 Conclusion

We proposed a domain adaptation scheme for video deblurring based on a
conditional diffusion-based blurring model to achieve test-time fine-tuning in
unseen domains. To generate domain-adaptive training pairs for updating
deblurring models during inference, we proposed two modules: the Relative
Sharpness Detection Module (RSDM) and the Domain-adaptive Blur Condition
Generation Module (DBCGM). The RSDM extracts relatively sharp patches from
blurred input frames as pseudo-sharp images. The DBCGM generates domain-
specific blur conditions for blurring the pseudo-sharp images. Lastly, the
generated pseudo-sharp and blurred pairs are used to calibrate a deblurring
model for better performance. Extensive experimental results have demonstrated
that our approach can significantly improve state-of-the-art video deblurring
methods, offering performance gains of up to 7.54dB on various real-world
video deblurring datasets.

## Acknowledgments

This work was supported in part by the National Science and Technology Council
(NSTC) under grants 112-2221-EA49-090-MY3, 111-2628-E-A49-025-MY3,
112-2634-F002-005, 112-2221-E-004-005, 113-2923-E-A49-003-MY2,
113-2221-E-004-001-MY3 and 113-2622-E-004-001 This work was funded in part by
Qualcomm through a Taiwan University Research Collaboration Project.

## References

  * [1] https://scikit-image.org/docs/stable/auto_examples/segmentation/plot_regionprops.html
  * [2] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: ECCV (2020)
  * [3] Chao, Z., Hang, D., Jinshan, P., Boyang, L., Yuhao, H., Lean, F., Fei, W.: Deep recurrent neural network with multi-scale bi-directional propagation for video deblurring. In: AAAI (2022)
  * [4] Chen, C.F., Panda, R., Fan, Q.: Regionvit: Regional-to-local attention for vision transformers. In: ICLR (2022)
  * [5] Chen, L., Wei, Z., Jin, X., Chen, H., Zheng, M., Chen, K., Jin, Y.: Deliberated domain bridging for domain adaptive semantic segmentation. In: NeurIPS (2022)
  * [6] Chi, Z., Wang, Y., Yu, Y., Tang, J.: Test-time fast adaptation for dynamic scene deblurring via meta-auxiliary learning. In: CVPR (2021)
  * [7] Cho, S.J., Ji, S.W., Hong, J.P., Jung, S.W., Ko, S.J.: Rethinking coarse-to-fine approach in single image deblurring. In: ICCV (2021)
  * [8] Chu, X., Tian, Z., Wang, Y., Zhang, B., Ren, H., Wei, X., Xia, H., Shen, C.: Twins: Revisiting the design of spatial attention in vision transformers. In: NeurIPS (2021)
  * [9] Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv (2018)
  * [10] Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. In: NeurIPS (2021)
  * [11] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: ICLR (2021)
  * [12] Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: ICML (2017)
  * [13] Gao, H., Tao, X., Shen, X., Jia, J.: Dynamic scene deblurring with parameter selective sharing and nested skip connections. In: CVPR (2019)
  * [14] Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models. In: NeurIPS (2020)
  * [15] Hoyer, L., Dai, D., Van Gool, L.: HRDA: Context-aware high-resolution domain-adaptive semantic segmentation. In: ECCV (2022)
  * [16] Ji, B., Yao, A.: Multi-scale memory-based video deblurring. In: CVPR (2022)
  * [17] Jiang, B., Xie, Z., Xia, Z., Li, S., Liu, S.: Erdn: Equivalent receptive field deformable network for video deblurring. In: ECCV (2022)
  * [18] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: ICLR (2017)
  * [19] Kupyn, O., Martyniuk, T., Wu, J., Wang, Z.: Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In: ICCV (2019)
  * [20] Lai, X., Tian, Z., Xu, X., Chen, Y., Liu, S., Zhao, H., Wang, L., Jia, J.: Decouplenet: Decoupled network for domain adaptive semantic segmentation. In: ECCV (2022)
  * [21] Li, D., Shi, X., Zhang, Y., Cheung, K.C., See, S., Wang, X., Qin, H., Li, H.: A simple baseline for video restoration with grouped spatial-temporal shift. In: CVPR (2023)
  * [22] Li, D., Xu, C., Zhang, K., Yu, X., Zhong, Y., Ren, W., Suominen, H., Li, H.: Arvo: Learning all-range volumetric correspondence for video deblurring. In: CVPR (2021)
  * [23] Li, W., Liu, X., Yuan, Y.: Sigma: Semantic-complete graph matching for domain adaptive object detection. In: CVPR (2022)
  * [24] Li, Y.J., Dai, X., Ma, C.Y., Liu, Y.C., Chen, K., Wu, B., He, Z., Kitani, K., Vajda, P.: Cross-domain adaptive teacher for object detection. In: CVPR (2022)
  * [25] Liang, J., Cao, J., Fan, Y., Zhang, K., Ranjan, R., Li, Y., Timofte, R., Van Gool, L.: Vrt: A video restoration transformer. In: arXiv (2022)
  * [26] Liang, J., Fan, Y., Xiang, X., Ranjan, R., Ilg, E., Green, S., Cao, J., Zhang, K., Timofte, R., Van Gool, L.: Recurrent video restoration transformer with guided deformable attention. In: NeurIPS (2022)
  * [27] Liang, Z., Li, Z., Zhou, S., Li, C., Loy, C.C.: Control color: Multimodal diffusion-based interactive image colorization. arXiv (2024)
  * [28] Lin, J., Cai, Y., Hu, X., Wang, H., Yan, Y., Zou, X., Ding, H., Zhang, Y., Timofte, R., Van Gool, L.: Flow-guided sparse transformer for video deblurring. In: ICML (2022)
  * [29] Liu, P.S., Tsai, F.J., Peng, Y.T., Tsai, C.C., Lin, C.W., Lin, Y.Y.: Meta transferring for deblurring. In: BMVC (2022)
  * [30] Ma, J., Liang, J., Chen, C., Lu, H.: Subject-diffusion: Open domain personalized text-to-image generation without test-time fine-tuning. arXiv (2023)
  * [31] Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.Y., Ermon, S.: Sdedit: Guided image synthesis and editing with stochastic differential equations. arXiv (2021)
  * [32] Nah, S., Kim, T.H., Lee, K.M.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: CVPR (2017)
  * [33] Nah, S., Son, S., Lee, J., Lee, K.M.: Clean images are hard to reblur: A new clue for deblurring. In: ICLR (2022)
  * [34] Nah, S., Son, S., Lee, K.M.: Recurrent neural networks with intra-frame iterations for video deblurring. In: CVPR (2019)
  * [35] Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., Chen, M.: Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv (2021)
  * [36] Pan, J., Bai, H., Tang, J.: Cascaded deep video deblurring using temporal sharpness prior. In: CVPR (2020)
  * [37] Pan, J., Xu, B., Dong, J., Ge, J., Tang, J.: Deep discriminative spatial and temporal network for efficient video deblurring. In: CVPR (2023)
  * [38] Park, D., Kang, D.U., Kim, J., Chun, S.Y.: Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training. In: ECCV (2020)
  * [39] Purohit, K., Rajagopalan, A.N.: Region-adaptive dense network for efficient motion deblurring. In: AAAI (2020)
  * [40] Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML (2021)
  * [41] Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR (2020)
  * [42] Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., Sutskever, I.: Zero-shot text-to-image generation. In: ICML (2021)
  * [43] Ranftl, R., Bochkovskiy, A., Koltun, V.: Vision transformers for dense prediction. In: ICCV (2021)
  * [44] Rim, J., Lee, H., Won, J., Cho, S.: Real-world blur dataset for learning and benchmarking deblurring algorithms. In: ECCV (2020)
  * [45] Ruiz, N., Li, Y., Jampani, V., Pritch, Y., Rubinstein, M., Aberman, K.: Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In: CVPR (2023)
  * [46] Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E.L., Ghasemipour, K., Gontijo Lopes, R., Karagol Ayan, B., Salimans, T., et al.: Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS (2022)
  * [47] Song, J., Meng, C., Ermon, S.: Denoising diffusion implicit models. In: ICLR (2021)
  * [48] Su, S., Delbracio, M., Wang, J., Sapiro, G., Heidrich, W., Wang, O.: Deep video deblurring for hand-held cameras. In: CVPR (2017)
  * [49] Suin, M., Purohit, K., Rajagopalan, A.N.: Spatially-attentive patch-hierarchical network for adaptive motion deblurring. In: CVPR (2020)
  * [50] Suin, M., Rajagopalan, A.N.: Gated spatio-temporal attention-guided video deblurring. In: CVPR (2021)
  * [51] Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: CVPR (2018)
  * [52] Teed, Z., Deng, J.: Raft: Recurrent all-pairs field transforms for optical flow. In: ECCV (2020)
  * [53] Tsai, F.J., Peng, Y.T., Lin, Y.Y., Tsai, C.C., Lin, C.W.: Stripformer: Strip transformer for fast image deblurring. In: ECCV (2022)
  * [54] Tsai, F.J., Peng, Y.T., Tsai, C.C., Lin, Y.Y., Lin, C.W.: Banet: A blur-aware attention network for dynamic scene deblurring. IEEE TIP (2022)
  * [55] Voynov, A., Aberman, K., Cohen-Or, D.: Sketch-guided text-to-image diffusion models. In: SIGGRAPH (2023)
  * [56] Wang, Y., Lu, Y., Gao, Y., Wang, L., Zhong, Z., Zheng, Y., Yamashita, A.: Efficient video deblurring guided by motion magnitude. In: ECCV (2022)
  * [57] Wu, J.H., Tsai, F.J., Peng, Y.T., Tsai, C.C., Lin, C.W., Lin, Y.Y.: Id-blau: Image deblurring by implicit diffusion-based reblurring augmentation. In: CVPR (2024)
  * [58] Xie, E., Wang, W., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P.: Segformer: Simple and efficient design for semantic segmentation with transformers. In: NeurIPS (2021)
  * [59] Zhang, H., Dai, Y., Li, H., Koniusz, P.: Deep stacked hierarchical multi-patch network for image deblurring. In: CVPR (2019)
  * [60] Zhang, H., Xie, H., Yao, H.: Spatio-temporal deformable attention network for video deblurring. In: ECCV (2022)
  * [61] Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: ICCV (2023)
  * [62] Zhang, Z., Hoai, M.: Object detection with self-supervised scene adaptation. In: CVPR (2023)
  * [63] Zhong, Z., Gao, Y., Zheng, Y., Zheng, B., Sato, I.: Real-world video deblurring: A benchmark dataset and an efficient recurrent neural network. IJCV (2022)
  * [64] Zhou, K., Li, W., Lu, L., Han, X., Lu, J.: Revisiting temporal alignment for video restoration. In: CVPR (2022)
  * [65] Zhu, X., Su, W., Lu, L., Li, B., Wang, X., Dai, J.: Deformable detr: Deformable transformers for end-to-end object detection. In: ICLR (2021)


# Practical PCG Through Large Language Models

αMuhammad U Nasir and βJulian Togelius  αUniversity of the Witwatersrand,
South Africa<EMAIL_ADDRESS>
βNew York University, USA

###### Abstract

Large Language Models (LLMs) have proven to be useful tools in various domains
outside of the field of their inception, which was natural language
processing. In this study, we provide practical directions on how to use LLMs
to generate 2D-game rooms for an under-development game, named Metavoidal. Our
technique can harness the power of GPT-3 by Human-in-the-loop fine-tuning
which allows our method to create $37\%$ Playable-Novel levels from as scarce
data as only 60 hand-designed rooms under a scenario of the non-trivial game,
with respect to (Procedural Content Generation) PCG, that has a good amount of
local and global constraints.

###### Index Terms:

Procedural Content Generation, Large Language Models

††publicationid: pubid: 979-8-3503-2277-4/23/$31.00 ©2023 IEEE

##  I Introduction

There are many ways of generating game levels. While most games featuring
online PCG that are actually shipped rely on domain-specific heuristic
solutions, methods explored by experimenters include evolutionary computation,
constraint satisfaction, grammar expansion, and fractals [1, 2]. Roguelike
games in particular often feature relatively ambitious PCG methods [3]. Over
the last decade, machine learning has turned out to be fruitfully applicable
to essentially everything under the sun. This includes level generation.
Researchers have explored the ways machine learning in general can be applied
to generating levels and other types of game content [4], as well as deep
learning in particular [5]. PCG itself holds utmost importance in many
important research fields, like Open-ended Learning [6] or continual learning
[7].

It’s 2023, and the new new thing that can be applied to everything under the
sun is Large Language Models (LLMs), such as image generation [8] and neural
architecture search [9]. While originally developed for natural language
processing, LLMs have proven effective for anything that can be expressed as
sequences of tokens, including images. The versatility of LLMs go beyond what
would normally be considered text completion, as they are capable of
performing many tasks that would seem to require cognitive efforts from
humans. Could LLMs also be useful for generating game content? Game levels,
like everything else that passes through a computer, are after all just
strings.

Two recent studies examine this. In one of them, GPT-2 and GPT-3 were
finetuned to generate Sokoban levels. The generated levels were good and novel
but, particularly for GPT-2, the dataset requirements were excessive [10].
Another study showed that special-purpose LLM architecture produced good
levels for the classic platformer Super Mario Bros [11].

In this paper we explore the possibility of using LLMs to generate levels for
a game under active development, where only a limited number of levels are
available, forcing us to find a data-efficient method. Furthermore, these
levels are relatively large and have a nontrivial number of constraints. Our
approach is to encode the constraints into the prompt and fine-tune GPT-3. To
efficiently use the limited data available without overfitting we use several
types of data augmentation as well as a form of bootstrapping, where novel
high-quality levels are added back into the dataset.

##  II Metavoidal and Room Generation Setup

Figure 1: Levels of different sizes created by the developers with all assets
on them. Figure 2: A flowchart of how both stages work. The yellow circle
indicates the stage number. Red color indicates automated process and green
indicates human-in-the-loop process.

###  II-A Gameplay

Metavoidal111https://yellowlabgames.itch.io/metavoidal is a roguelite brawler
game, being developed by Yellow Lab Games222https://yellowlab.games/ that
features a metal band trying to hire new drummers. The metal band turns out to
be full of eldritch monsters sacrificing drummers to gain more power. You play
as a drummer in a church where the auditions are happening. You are trying to
escape as you are a bad drummer and hordes of monsters are trying to sacrifice
you. You will have drumsticks as your weapon. You can find power-ups like
music disks to fight enemies. Your goal is to escape the church.

The layout of the game includes 3 levels. Each level with many rooms connected
to hallways. There are some tunnel-like connections. Rooms are essentially the
main areas where assets are discovered to progress in the game. Figure 1 shows
rooms in the game developed by a 2D game artist. There are many tiles to be
considered while generating. There are three types of tiles that make
patterns: wood, marble and moss. There are two types of walls: marble and moss
marble wall. There is one deepwater and one junction point tile. Junction
point tile is used for doors. Both of wall tiles and deepwater tiles are
considered unwalkable tiles.

###  II-B Constraints For Room Generation

The following are the constraints:

  1. 1.

The three pattern tiles that should make the most of the room.

  2. 2.

The unwalkable tiles should be at least two tiles apart if it is within the
path. The path is defined as all walkable tiles that are connected from one
door to another.

  3. 3.

Deepwater tile can be placed in a cluster.

  4. 4.

Wall tiles should be placed as a single row of tiles to look like walls.

  5. 5.

The walkable tiles should be placed such that one of them should be the base
tile and the rest of the two should be supporting pattern tiles.

  6. 6.

The junction tile should be one tile apart for vertical doors and two tiles
apart for horizontal doors, (5) If there is more than one door then at least
two doors should connect to each other.

  7. 7.

The length and the width of the room can vary but it should always be
divisible by two.

##  III Proposed method

The proposed method takes inspiration from Todd et al. [10] and introduces a
training technique such that the method can produce playable-novel levels from
as scarce data as only 60 rooms. This section will introduce all the
methodology in details, starting with the initial dataset, the bootstraping
technique [12], augmentation of our dataset, and training of our final model.

###  III-A Dataset

We received 60 room levels from the developers as our initial dataset. We map
all the rooms to characters as we will give these character-based tiles to our
LLM. This is the method followed by Todd et al.[10] as well. Our LLM is
OpenAI’s GPT-3. GPT-3 requires prompt and completion as one row of the
dataset. As we have many constraints, which is common in any game being
developed, we use controllable prompts rather than simply prompting to create
a level. Our single prompt looks like this:

’”The size of the level is {width x height}, the base tile is ”{base_tile}”,
and the border tile is ”{border_tile}”. There are $2$ pattern tiles,
”{pattern_tiles[0]}” and ”{pattern_tiles[1]}”, ”F” is the water tile, ”J” is
the door tile, and the percentage of pattern tiles is
{percent_pattern_tiles}%.-$>$”’

Where the width and height of the level include the border tile. base_tile is
the tile with the highest number of counts. border_tile is one of the wall
tiles with the most counts on the border. There can be either one or two
pattern_tiles, if there is one pattern_tiles then the statement changes to
”There is 1 pattern tile, ”{pattern_tiles[0]}”. percent_pattern_tiles is
calculated by the percentage of only pattern tiles among all tiles. This lets
LLM knows how many pattern tiles to use. Lastly, $->$ is used as a special
token for GPT-3 to know the prompt has ended. For completions, which are the
labels of the prompts, we selected ”A”, ”B” and ”C” as walkable tiles, ”E” and
”#” as wall tiles, ”F” for water tiles and ”J” for the junction tile. Each row
of tiles ends with `\n` as a newline indicator. After the level ends, we put
”. XUT” as the ending token.

###  III-B Augmenting Dataset

To increase the dataset and get more variation in the dataset, we use the
following techniques, motivated by [10], to augment our data:

  1. 1.

We flip the room horizontally and vertically.

  2. 2.

We rotate the room $90^{\circ}$ and change the door sizes to cater for the
constraint.

  3. 3.

We swap the pattern tiles of the original room levels.

  4. 4.

We repeat $1-2$ for rooms with swapped pattern tiles.

###  III-C Our Method

Our method uses GPT-3 [13] from OpenAI333https://openai.com/ as the LLM to
generate levels. Reference [10] shows that text-davinci-003, a GPT-3’s
variant, has the ability to generate Playable-novel levels from scarce data.
Our method includes two stages of generation. The first stage is the human-in-
the-loop generation where we fine-tune GPT-3 on the data given by developers.
In this stage, we get unplayable rooms that do not follow the constraints. We
observe all generated rooms and try to fix the ones that are fixable. After
fixing the rooms, we add the room to our data if they are novel enough.
Torrado et al. [12] introduced this method and called it bootstrapping, we
will continue with the convention. We repeat bootstrapping till we get enough
data. We commence the second stage by augmenting our data as explained
earlier. Once we have obtained our dataset which is now much larger compared
to the initial dataset, we start to generate levels. This stage does not have
a human-in-the-loop component. We generate 100 rooms in each round, calculate
the metrics, and add the playable-novel rooms back into the dataset. This
could essentially be repeated however many times one desires.

###  III-D Metrics

Our following two metrics are inspired by Todd et al. [10] but adjusted to our
needs:

#### Playable-Novel

has two components, playability and novelty. Playability is measured by the
constraints mentioned in Section II-B. Once all the constraints are passed, we
check the novelty of the level. We check novelty by first checking if at least
the level is novel by a novelty threshold, which is a percentage of the total
number of tiles in the created level. This also includes the border tiles.
When it is novel by at least the novelty threshold, we swap the pattern tiles
and check with the same threshold. The novelty is checked across all the
levels in the dataset that we currently have. If it still holds then we
consider it Playable-Novel level.

#### Accuracy

is a measure of how close the percentage of generated pattern tiles is to the
percentage mentioned in the prompt, and is measured by:

$Accuracy=1-\frac{|Prompt\\_Percent-Generated\\_Percent|}{Prompt\\_Percent}$

Where $Prompt\\_Percent$ is the percentage of pattern tiles, as written in
Section III-A by $percent\\_pattern\\_tiles$, and $Generated\\_Percent$ is the
percentage of the pattern tiles in generated level.

##  IV Experiment Setup and Results

For our first stage of generations, we set the temperature to 0.4. In our
experiments, we observe that GPT-3 can generate random text while generating
at a higher temperature thus we opted for the specific temperature setting.
GPT-3 related settings were set to default. For the novelty score, we set a
novelty threshold of $10\%$ of the total number of tiles in the level. As
mentioned earlier, We received 60 room levels hand created from the Metavoidal
developer team. We generate prompts as discussed earlier. We fine-tune GPT-3
for 5 epochs, generate 100 levels and take the 10 most levels that can be
repaired. We repair them and measure their novelty and playability. If novelty
and playability are passed, we add them to our data and fine-tune our
previously fine-tuned GPT-3 model. We repeat this step till we get 60 more
levels.

With the 120 levels, we move on to the second stage by augmenting our data. We
augment our data in the ways we described earlier and obtain 840 levels. We
further fine-tune our model for 2 epochs. After fine-tuning, we generate 100
levels to get playable-novel levels. We include them in the dataset to fine-
tune our model. We repeat this 5 times to eventually get up to $37\%$
playable-novel levels. Figure 3 illustrated what the output of each round of
level generation looks like.

To show the usefulness of the controllability prompt, we illustrate the
average accuracy over the generated levels in Figure 5. After augmenting the
data we implement the rest of the second stage for 5 seeds to solidify our
experiments. The total cost for all the experiments was $\$300$.

Figure 3: Illustration of how many playable-novel levels are generated per
each round of level generation at stage 2. The shadowed region shows variance
over 5 seeds while the solid line shows the mean. Figure 4: Illustration of
how accuracy improves from stage 1 to stage 2 and further to the last round of
level generation in stage 2. Stage 1 is performed on 1 seed while stage 2 is
performed on 2 seed.

Figure 5: A few generated rooms, randomly placed, to demonstrate constraints
handled by our method.

##  V Conclusion And Future Directions

To conclude, we introduce a method on a new aspiring game, named Metavoidal,
that can have the maximum characteristics and constraints of the content
described in the prompt, while other constraints learnt from scarce data, via
GPT-3. Our method is inspired by the work done by the authors of [10]. We
introduce methods of training that lead to the creation of an increasing
number of playable-novel levels. We show it on a new game, that is currently
under development so that we can introduce the method as a practical tool and
an application that can be used to create content in non-trivial games with
many constraints. Metavoidal also opens up more space in research as it can be
used for PCG, game-playing and game-testing AI research. One of the major
future directions for this method is to be working on 3D under-development
games. A breakthrough for any kind of PCG via LLM would be to have one
generalised model that can generate 2D and 3D levels from a single model.

## References

  * [1] N. Shaker, J. Togelius, and M. J. Nelson, “Procedural content generation in games,” 2016.
  * [2] J. Togelius, G. N. Yannakakis, K. O. Stanley, and C. Browne, “Search-based procedural content generation: A taxonomy and survey,” _IEEE Transactions on Computational Intelligence and AI in Games_ , vol. 3, no. 3, pp. 172–186, 2011.
  * [3] J. Harris, _Exploring roguelike games_. CRC Press, 2020.
  * [4] A. Summerville, S. Snodgrass, M. Guzdial, C. Holmgård, A. K. Hoover, A. Isaksen, A. Nealen, and J. Togelius, “Procedural content generation via machine learning (pcgml),” _IEEE Transactions on Games_ , vol. 10, no. 3, pp. 257–270, 2018.
  * [5] J. Liu, S. Snodgrass, A. Khalifa, S. Risi, G. N. Yannakakis, and J. Togelius, “Deep learning for procedural content generation,” _Neural Computing and Applications_ , vol. 33, no. 1, pp. 19–37, 2021.
  * [6] M. U. Nasir, M. Beukman, S. James, and C. W. Cleghorn, “Augmentative topology agents for open-ended learning,” _arXiv preprint arXiv:2210.11442_ , 2022\.
  * [7] J. Xu and Z. Zhu, “Reinforced continual learning,” _Advances in Neural Information Processing Systems_ , vol. 31, 2018.
  * [8] H. Chang, H. Zhang, J. Barber, A. Maschinot, J. Lezama, L. Jiang, M.-H. Yang, K. Murphy, W. T. Freeman, M. Rubinstein _et al._ , “Muse: Text-to-image generation via masked generative transformers,” _arXiv preprint arXiv:2301.00704_ , 2023.
  * [9] M. U. Nasir, S. Earle, J. Togelius, S. James, and C. Cleghorn, “Llmatic: Neural architecture search via large language models and quality-diversity optimization,” _arXiv preprint arXiv:2306.01102_ , 2023.
  * [10] G. Todd, S. Earle, M. U. Nasir, M. C. Green, and J. Togelius, “Level generation through large language models,” in _Proceedings of the 18th International Conference on the Foundations of Digital Games_ , 2023, pp. 1–8.
  * [11] S. Sudhakaran, M. González-Duque, C. Glanois, M. Freiberger, E. Najarro, and S. Risi, “Mariogpt: Open-ended text2level generation through large language models,” _arXiv preprint arXiv:2302.05981_ , 2023.
  * [12] R. R. Torrado, A. Khalifa, M. C. Green, N. Justesen, S. Risi, and J. Togelius, “Bootstrapping conditional gans for video game level generation,” in _2020 IEEE Conference on Games (CoG)_. IEEE, 2020, pp. 41–48.
  * [13] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell _et al._ , “Language models are few-shot learners,” _Advances in neural information processing systems_ , vol. 33, pp. 1877–1901, 2020.


# Impact of Attention on Adversarial Robustness of Image Classification Models

Prachi Agrawal
Sanjay Kumar Sonbhadra  Dept. of Computer Science and Information Systems
Birla Institute of Technology and Science
Pilani, India
<EMAIL_ADDRESS>Dept. of Information Technology
Indian Institute of Information Technology Allahabad
Prayagraj, India
<EMAIL_ADDRESS>Narinder Singh Punn
Sonali Agarwal  Dept. of Information Technology
Indian Institute of Information Technology Allahabad
Prayagraj, India
<EMAIL_ADDRESS>Dept. of Information Technology
Indian Institute of Information Technology Allahabad
Prayagraj, India
<EMAIL_ADDRESS>
###### Abstract

Adversarial attacks against deep learning models have gained significant
attention and recent works have proposed explanations for the existence of
adversarial examples and techniques to defend the models against these
attacks. Attention in computer vision has been used to incorporate focused
learning of important features and has led to improved accuracy. Recently,
models with attention mechanisms have been proposed to enhance adversarial
robustness. Following this context, this work aims at a general understanding
of the impact of attention on adversarial robustness. This work presents a
comparative study of adversarial robustness of non-attention and attention
based image classification models trained on CIFAR-10, CIFAR-100 and Fashion
MNIST datasets under the popular white box and black box attacks. The
experimental results show that the robustness of attention based models may be
dependent on the datasets used i.e. the number of classes involved in the
classification. In contrast to the datasets with less number of classes,
attention based models are observed to show better robustness towards
classification.

###### Index Terms:

Attention, Classification, Deep learning, Robustness, White and black box
attacks

##  I Introduction

Recently, the vulnerability of deep learning models to adversarial attacks has
been recognized. Adversarial attacks in the context of image classification
involve fooling an image classifier by generating an image with small
perturbation from the original image which results in misclassification of the
image. Previous works in the field have proposed a variety of explanations for
the existence of adversarial examples including high-dimensional geometry of
data manifold [1] and computational limitations of learning algorithms [2].
Ilyas et al. [3] emphasize that adversarial robustness is a human-centric
phenomenon and vulnerability to adversarial examples exists because models
learn to treat non-robust features and robust features with equal importance.

Attention plays a huge role in the human visual system. Humans use a sequence
of partial glimpses and selectively focus on important parts instead of
processing the entire scene at once. Inspired by the attention mechanism in
the human visual system, several methods have been proposed to incorporate
attention in convolution neural network (CNN) based image classification tasks
which have also led to improvement in the accuracy of the classifiers e.g.
residual attention network [4], squeeze and excitation networks [5],
convolutional block attention module [6], etc. [7].

Following this context, the present work focuses on studying the impact of
learning salient features through attention mechanisms on the adversarial
robustness of a model using the image classification problem. The major
contribution of the present research work is as follows:

  * •

Comparative analysis of the robustness of attention based (using mixed
attention) and non-attention based models under the white box and transfer
based black box attacks.

  * •

Exploring the dependence of adversarial robustness of attention and non-
attention based models on datasets used.

  * •

While recent works have proposed attention based techniques to improve
adversarial robustness, our study focuses on understanding how attention
impacts robustness which would give more insights on the existence of
adversarial examples and how defense strategies could be developed.

  * •

To the best of our knowledge, this is also the first attempt to analyze the
adversarial robustness of image classification models with mixed attention.

The rest paper is divided into several sections, where Section 2 describes the
literature survey of the works describing robustness of the classification
models, followed by Section 3 presenting the proposed framework to simulate
white box and black box attacks on classification models with different
datasets. Section 4 presents the the simulation results of the attacks on the
models. Finally, concluding remarks are presented in Section 5.

##  II Related work

In order to defend against adversarial attacks, extensive research has been
conducted to develop defense strategies and build robust models. Some of them
are robust training, input transformation, randomization, model ensemble, and
certified defenses.

In robust training, the classifier is made robust against small noises
internally. Adversarial training [8, 9, 10] is a popular technique based on
robust training which augments training data by adversarial examples. Several
methods based on robust training using losses or regularizations have been
proposed. These include regularization on input gradient [11], perturbation
based regularization [12] and variants on network Lipschitz constant [13].
Another technique is to transform the inputs before feeding them into the
classifier. Total variance minimization [14], autoencoder based denoising
[15], using generative models to project adversarial examples onto the data
distribution [16] are some of the input transformation based strategies.
Adding randomness to the input [17, 18] or the model [19] have also been
proposed to enhance robustness. Another defense strategy is to construct an
ensemble of models and aggregate the output of each model in the ensemble [20,
21]. There are a lot of works based on certified defenses which are guaranteed
to be robust against adversarial perturbations under some attack models [22,
23, 24, 25].

Several recent works have proposed models with attention based mechanisms to
enhance robustness. Zoran et al. [26] use a visual attention component guided
by a recurrent top down approach to improve the adversarial robustness,
whereas Liu et al. [27] propose a nonlinear attention module for feature
prioritization to improve adversarial robustness. In similar work, Zhao et al.
[28] show that self-attention networks may have benefits in terms of
adversarial robustness and later, Vaishnavi et al. [29] use foreground
attention masks i.e separate the background from the foreground and mask the
background before training and classification. Their results show an increase
in adversarial robustness on using foreground masks. Goodman et al. [30]
develop a method called attention and adversarial logit pairing that
encourages both attention map and logit for pairs of clean examples and their
adversarial counterparts to be similar. In this work, we attempt to get a
general understanding of the impact of attention in image classification
models using mixed attention under different kinds of attacks.

##  III Proposed work

Based on the discussion in Section 1, we conduct experiments to understand if
learning of salient features through attention mechanisms translates to
adversarial robustness through better identification of underlying features in
adversarial images. We compare the adversarial robustness of an image
classifier with and without attention under the white box and black box
attacks. Fig 1 shows a schematic representation of the proposed framework in
which the attacks are simulated.

We have considered Resnet-50 [11] for image classification and used
Convolutional Block Attention Module (CBAM) to incorporate attention. CBAM
contains two sequential sub-modules i.e. channel attention module which
feature maps are important and spatial attention module conveys what within
the feature map is important. A comparison of robustness of Resnet-50 and
Resnet-50 with CBAM modules (i.e. CBAM+Resnet-50) is performed under different
attacks.

Figure 1: Simulation framework for the attacks on attention and non-attention
classification models.

###  III-A Attacks

The attacks used in this work can be classified into white box and black box.
In white box attacks, an attacker has complete access to the target model
including the model parameters, architecture, training method and training
data, and exploits this information to craft an attack. In black box attacks,
the attacker has no access to model parameters and uses a different model or a
query based strategy to perform an attack.

####  III-A1 White box attack

Popular white box attacks include fast gradient sign method [8] (FGSM),
projected gradient descent [9] (PGD), deepfool [31], C&W [32]. In this work,
FGSM and PGD have been used to evaluate and compare the performance of
attention and non-attention models under white box attack scenarios.

FGSM is a single step gradient based approach where an adversarial example is
created by maximizing the loss function $\mathcal{L}(x_{adv},y)$ wrt to the
input $x$ and then adding the sign of output gradient to $x$ in order to
produce $x_{adv}$ (adversarial example) and is represented in Eq. 1. In our
analysis, the accuracy of the target models is studied for various values of
epsilon in the range of 0 to 0.3. PGD uses the projected gradient descent to
iteratively create adversarial examples. The perturbed data in step $t+1$ can
be represented in Eq. 2.

$x_{adv}=x+\epsilon*sign(\triangledown_{x}\mathcal{L}(x,y))$ (1)
$x^{t+1}_{adv}=\pi_{x+S}(x^{t}_{adv}+\epsilon+\alpha(sign(\triangledown_{x}\mathcal{L}(x^{t}_{adv},y))$
(2)

Here, $\alpha$ is the step size and $\pi_{x+S}$ denotes projecting
perturbations into set $S$. In all our experiments, we use PGD bounded under
$l_{inf}$ norm with $\epsilon=0.2$ and $\alpha=0.01$. The attack accuracy is
evaluated for different numbers of iterations.

####  III-A2 Black box attack

Transfer based black box attack utilizes cross-model transferability of
adversarial samples, which trains a local substitute model (source model) to
generate adversarial examples and uses these examples to attack the target
model. In this work, we use FGSM and PGD ($\epsilon$=0.2, $\alpha$=0.01) to
generate examples from substitute model and carry out the transfer based black
box attack in the following scenarios:

  * •

Non-attention model as the source and attention based model as target.

  * •

Attention based model as the source and non-attention model as target.

  * •

Comparison of attack against attention based model and non-attention model
with VGG-16 as source.

The first scenario is used to understand if attention module helps in building
defense and thus better robustness towards adversarial examples generated by
non-attention based model. In second scenario, we evaluate if the attack on
non-attention model using images generated by attention based model is better
due to the learning of salient features by attention models. The final
scenario helps in comparison of the robustness of attention and non-attention
model in transfer based attack using a common model as source i.e. VGG16. The
attacks setup of the present research work is presented in Table I for
classification models over multiple datasets.

TABLE I: Summary of attacks used to check the robustness of the classification models. Attack |  Type
---|---
Fast gradient sign method (FGSM) |  White Box (Gradient Based)
Projected gradient descent (PGD) |  White Box (Gradient Based)
Transfer based FGSM & PGD ; Source: Attention based model (CBAM+Resnet-50) , Target: Non-attention model (Resnet-50) |  Black Box (Transfer based)
Transfer based FGSM & PGD, Source: Non-attention model (Resnet-50) , Target: Attention based model (CBAM+Resnet-50) |  Black Box (Transfer based)
Transfer based FGSM & PGD , Source: VGG-16 , Target: Attention model (CBAM+Resnet-50) & non-attention based model (Resnet-50) |  Black Box (Transfer based)

###  III-B Datasets used

Publicly available datasets: CIFAR-10 [33], CIFAR-100 [33] and Fashion MNIST
[34] have been used in our work. CIFAR-10 and CIFAR-100 consist of 60k colour
images of size 32x32 the training set consisting of 50k images and the test
set of 10k images. CIFAR-10 has 10 classes of 6k images each and CIFAR-100 has
100 classes of 600 images each. Fashion MNIST dataset consists of grayscale
images of 10 fashion classes. The size of images is 28x28 and consists of 60k
training images and 10k testing images.

##  IV Results and discussion

###  IV-A White box

Table II shows the performance of the non-attention model and attention based
model trained on CIFAR-10, CIFAR-100 and FashionMNIST datasets under FGSM and
PGD attacks with different values of perturbation and number of iterations
respectively.

TABLE II: Summary of results of white box attacks. Attacks | Dataset
---|---
CIFAR-10 | CIFAR-100 | Fashion MNIST
FGSM | Perturbation |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc)
| 0 |  83.03 |  80.4 |  77.2 |  75.69 |  89.89 |  92.73
| 0.05 |  13.57 |  0.37 |  13.4 |  19.29 |  42.42 |  17.59
| 0.1 |  12.4 |  1.79 |  4.2 |  5.43 |  6.63 |  6.85
| 0.15 |  11.91 |  4.6 |  1.49 |  2.43 |  6.89 |  6.55
| 0.2 |  11.34 |  5.95 |  0.97 |  1.39 |  6.73 |  6.89
| 0.25 |  10.88 |  6.51 |  0.93 |  1.12 |  6.86 |  7.03
| 0.3 |  9.3 |  6.85 |  0.92 |  1.03 |  7.16 |  7.22
PGD | Iterations |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err)
| 5 |  99.98 |  100 |  99.83 |  87.4 |  95.68 |  100
| 10 |  100 |  100 |  100 |  93.41 |  100 |  100
| 20 |  100 |  100 |  100 |  97.16 |  100 |  100

####  IV-A1 CIFAR-10

A comparison of white box FGSM attack for different perturbation values shown
in Fig. 2(a) suggests that CBAM+Resnet-50 is attacked to a much larger extent
as compared to Resnet-50 throughout the range of epsilon values from 0.05 to
0.3, where the accuracy of CBAM+Resnet-50 even drops to $\sim 0$%. The white
box PGD attack in Fig. 2(b) also represents towards better robustness of the
non-attention model (Resnet-50) since it takes a larger number of iterations
as compared to the attention model (CBAM+Resnet-50) for complete attack i.e.
100% error.

Figure 2: a) FGSM attack and b) PGD attack on Resnet-50 and CBAM+Resnet-50 for
CIFAR-10 dataset.

####  IV-A2 CIFAR-100

Unlike the observation for white box attacks on the CIFAR-10 dataset, the
attention based model shows better robustness for the CIFAR-100 dataset. White
box FGSM attack (Fig. 3(a) shows slightly better accuracy for CBAM+Resnet-50
but the PGD attack (Fig. 3(b) convincingly describes that CBAM+Resnet-50 is
much more robust as compared to Resnet-50 since it isn’t attacked completely
even in more than 20 iterations while Resnet-50 is completely attacked in less
than 5 iterations.

Figure 3: a) FGSM attack and b) PGD attack on Resnet-50 and CBAM+Resnet-50 for
CIFAR-100 dataset.

####  IV-A3 Fashion MNIST

Fig. 4(a) shows that Resnet-50 shows better adversarial robustness upto a
perturbation value of 0.1. For epsilon values greater than 0.1, both
CBAM+Resnet-50 and Resnet-50 have approximately equal accuracy under the FGSM
attack. Fig. 4(b) depicts the better robustness of Resnet-50 since it takes a
larger number of iterations as compared to CBAM+Resnet-50 for complete attack.

Figure 4: a) FGSM attack and b) PGD attack on Resnet-50 and CBAM+Resnet-50 for
Fashion MNIST dataset.

###  IV-B Black box

Table III shows the performance of the non-attention model and attention based
model trained on CIFAR-10, CIFAR-100 and FashionMNIST datasets under transfer
based FGSM and PGD attacks under the 3 scenarios of source and target chosen
which has been described in Table 1. The performance of the models have been
observed for different values of perturbation in case of FGSM and different
number of iterations for PGD.

TABLE III: Summary of results of black box attacks. Dataset | Attack |  Source | Target | Source | Target | Source | Target-I | Target-II
---|---|---|---|---|---|---|---|---
CIFAR-10 |  FGSM | Perturbation |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  VGG-16 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc)
| 0 |  83.03 |  74.52 |  80.4 |  74.52 |  80.5 |  74.22 |  72.55
| 0.05 |  13.57 |  37.6 |  0.37 |  42.54 |  5.93 |  25.57 |  28.07
| 0.1 |  12.4 |  19 |  1.79 |  20.91 |  6.64 |  13.82 |  14.33
| 0.15 |  11.91 |  13.11 |  4.6 |  14.12 |  7.84 |  12.66 |  12.43
| 0.2 |  11.34 |  11.65 |  5.95 |  11.83 |  8.66 |  12.89 |  11.2
| 0.25 |  10.88 |  11.49 |  6.51 |  10.88 |  9.13 |  13.15 |  10.98
| 0.3 |  9.3 |  11.58 |  6.85 |  10.47 |  9.47 |  8.84 |  10.69
PGD | Iterations |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  VGG-16 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err)
| 5 |  99.98 |  49.12 |  100 |  45.56 |  100 |  83.65 |  75.76
| 10 |  100 |  64.47 |  100 |  59.03 |  100 |  95.51 |  92.44
| 20 |  100 |  77.3 |  100 |  70.75 |  100 |  98.3 |  97.42
CIFAR-100 |  FGSM | Perturbation |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  VGG-16 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc)
| 0 |  77.2 |  75.69 |  75.69 |  77.2 |  71.46 |  77.2 |  75.69
| 0.05 |  13.4 |  18.3 |  19.29 |  21.3 |  5.37 |  16.23 |  15.74
| 0.1 |  4.2 |  5.63 |  5.43 |  5.16 |  3.33 |  4.78 |  5.16
| 0.15 |  1.49 |  2.21 |  2.43 |  1.58 |  2.21 |  1.72 |  2.18
| 0.2 |  0.97 |  1.37 |  1.39 |  0.95 |  1.53 |  1.01 |  1.42
| 0.25 |  0.93 |  1.16 |  1.12 |  0.99 |  1.11 |  0.88 |  1.1
| 0.3 |  0.92 |  1.05 |  1.03 |  0.88 |  1.02 |  0.88 |  0.95
PGD | Iterations |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  VGG-16 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err)
| 5 |  99.83 |  84.64 |  87.4 |  78.93 |  94.15 |  71.74 |  71.54
| 10 |  100 |  90.21 |  93.41 |  87.11 |  95.24 |  82.82 |  82.18
| 20 |  100 |  93.18 |  97.16 |  92.4 |  95.99 |  89.72 |  88.51
Fashion MNIST |  FGSM | Perturbation |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc) |  VGG-16 (Acc) |  Resnet-50 (Acc) |  CBAM+ Resnet-50 (Acc)
| 0 |  89.89 |  92.73 |  92.73 |  89.89 |  91.93 |  89.89 |  92.73
| 0.05 |  42.42 |  38.34 |  17.59 |  69.29 |  14.69 |  58.83 |  54.67
| 0.1 |  6.63 |  14.93 |  6.85 |  15.38 |  6.04 |  17.54 |  25.03
| 0.15 |  6.89 |  9.95 |  6.55 |  9.67 |  5.5 |  10.33 |  13.85
| 0.2 |  6.73 |  9.15 |  6.89 |  7.99 |  5.64 |  8.26 |  10.67
| 0.25 |  6.86 |  9.05 |  7.03 |  7.06 |  6.27 |  7.76 |  8.93
| 0.3 |  7.16 |  9.02 |  7.22 |  6.69 |  6.95 |  7.27 |  7.98
PGD | Iterations |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err) |  VGG-16 (Err) |  Resnet-50 (Err) |  CBAM+ Resnet-50 (Err)
| 5 |  95.68 |  73.99 |  100 |  35.33 |  90.51 |  37.63 |  60.22
| 10 |  100 |  92.5 |  100 |  64.02 |  90.55 |  57.55 |  70.5
| 20 |  100 |  97.93 |  100 |  92.21 |  90.55 |  71.42 |  79.66
Figure 5: a) Transfer based FGSM, b) Transfer based FGSM with VGG-16 as
source, c) Transfer based PGD and d) Transfer based PGD with VGG-16 as source
for CIFAR-10 dataset.

####  IV-B1 CIFAR10

For the FGSM attack, Resnet-50 showed better transferability leading to a
better attack on CBAM+Resnet-50 as compared to an attack on Resnet-50 by
examples generated by CBAM+Resnet-50 (Fig. 5(a)). Under the FGSM transfer
attack with VGG-16 as the source, the attention and non-attention model both
showed approximately equal accuracy under attack (Fig. 5(b)). Under the
transfer based PGD attack, Resnet-50 was comparatively more robust to attack
against adversarial examples generated by CBAM+Resnet-50 (Fig. 5(c)).
Resnet-50 thus showed better transferability for the attack against attention
based model. However, under transfer based PGD with VGG-16 as the source,
CBAM+Resnet-50 shows better robustness as compared to Resnet-50 (Fig. 5(d)).

Figure 6: a) Transfer based FGSM, b) Transfer based FGSM with VGG-16 as
source, c) Transfer based PGD and d) Transfer based PGD with VGG-16 as source
for CIFAR-100 dataset.

####  IV-B2 CIFAR-100

For the FGSM attack, Resnet-50 showed better transferability leading to a
better attack on CBAM+Resnet-50 as compared to an attack on Resnet-50 by
examples generated by CBAM+Resnet-50 (Fig. 6(a)). Under the FGSM transfer
attack with VGG-16 as the source, the attention and non-attention model both
showed approximately equal accuracy under attack (Fig. 6(b)). Under the
transfer based PGD attack, Resnet-50 was comparatively more robust to attack
against adversarial examples generated by CBAM+Resnet-50 (Fig. 6(c)).
Resnet-50 thus showed better transferability for the attack against attention
based model. However, under transfer based PGD with VGG-16 as the source,
CBAM+Resnet-50 shows better robustness as compared to Resnet-50 (Fig. 6(d)).

####  IV-B3 Fashion MNIST

Resnet-50 is much more robust to transfer based FGSM attack from adversarial
examples generated by CBAM+Resnet-50. Comparison between source and target
accuracy in Fig. 7(a) shows that Resnet-50 has good transferability since the
accuracy of CBAM+Resnet-50 under attack differs from Resnet-50 as a source by
a small margin. The transferability of CBAM+Resnet-50 under FGSM attack is
much poorer which can be observed through the large difference in accuracy
between CBAM+Resmet-50 (source) and Resnet-50 (target) (Fig. 7(b)). Under the
FGSM transfer based attack with VGG-16 as a source, there’s not much
difference in the robustness of the non-attention and attention model. The
attention based model (CBAM+Resnet-50) is slightly more robust for
perturbation values greater than 0.5. The results of the transfer PGD attack
in Fig 17 indicate that Resnet-50 is much more robust under examples generated
by CBAM+Resnet-50 in comparison with the robustness of CBAM+Resnet-50 under
adversarial examples by Resnet-50 (Fig. 7(c)). A similar result is observed
for PGD attack with VGG-16 as source where Resnet-50 shows better robustness
against adversarial examples generated by VGG-16 (Fig. 7(d)).

Figure 7: a) Transfer based FGSM, b) Transfer based FGSM with VGG-16 as
source, c) Transfer based PGD and d) Transfer based PGD with VGG-16 as source
for Fashion MNIST dataset.

##  V Conclusion

While it was expected that attention based models would be more robust under
white box attacks and transfer based black box attack and would also show
better transferability, the results obtained for CIFAR-10 and Fashion MNIST
dataset strongly favour the robustness of the non-attention model i.e.
Resnet-50. However, for the CIFAR-100 dataset, the attention based model i.e.
CBAM+Resnet-50 was comparatively much more robust than the non-attention model
Resnet-50 under white box attacks while the difference in results between the
two models was very low in the transfer based black box attacks. These results
suggest that the impact of attention on the robustness of image classification
models could be dependent on the datasets used i.e. the number of classes in
the datasets. Similarity in results of CIFAR-10 and Fashion MNIST and
deviation of CIFAR-100 from these results could be due to fewer number of
classes i.e. 10 classes in comparison with CIFAR-100 dataset which has 100
classes. The results obtained in this work is a step closer towards a
generalized understanding of the impact of attention on the robustness of
image classification models. Experiments with more datasets with different
number of classes and models with other kinds of attention could be some
directions for future work.

## Acknowledgment

We thank our institute, Indian Institute of Information Technology Allahabad
(IIITA), India and Big Data Analytics (BDA) lab for allocating the centralised
computing facility and other necessary resources to perform this research. We
extend our thanks to our colleagues for their valuable guidance and
suggestions.

## References

  * [1] J. Gilmer, L. Metz, F. Faghri, S. S. Schoenholz, M. Raghu, M. Wattenberg, and I. Goodfellow, “Adversarial spheres,” _arXiv preprint arXiv:1801.02774_ , 2018.
  * [2] S. Bubeck, Y. T. Lee, E. Price, and I. Razenshteyn, “Adversarial examples from computational constraints,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 831–840.
  * [3] A. Ilyas, S. Santurkar, D. Tsipras, L. Engstrom, B. Tran, and A. Madry, “Adversarial examples are not bugs, they are features,” _arXiv preprint arXiv:1905.02175_ , 2019.
  * [4] F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang, “Residual attention network for image classification,” in _Proceedings of the computer vision and pattern recognition_ , 2017, pp. 3156–3164.
  * [5] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _Proceedings of the computer vision and pattern recognition_ , 2018, pp. 7132–7141.
  * [6] S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 3–19.
  * [7] N. S. Punn and S. Agarwal, “Modality specific u-net variants for biomedical image segmentation: A survey,” _arXiv preprint arXiv:2107.04537_ , 2021.
  * [8] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” _arXiv preprint arXiv:1412.6572_ , 2014.
  * [9] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” _arXiv preprint arXiv:1706.06083_ , 2017.
  * [10] F. Tramèr, A. Kurakin, N. Papernot, I. Goodfellow, D. Boneh, and P. McDaniel, “Ensemble adversarial training: Attacks and defenses,” _arXiv preprint arXiv:1705.07204_ , 2017.
  * [11] A. S. Ross and F. Doshi-Velez, “Improving the adversarial robustness and interpretability of deep neural networks by regularizing their input gradients,” in _Thirty-second AAAI conference on artificial intelligence_ , 2018.
  * [12] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [13] M. Cisse, P. Bojanowski, E. Grave, Y. Dauphin, and N. Usunier, “Parseval networks: Improving robustness to adversarial examples,” in _International Conference on Machine Learning_. PMLR, 2017, pp. 854–863.
  * [14] C. Guo, M. Rana, M. Cisse, and L. Van Der Maaten, “Countering adversarial images using input transformations,” _arXiv preprint arXiv:1711.00117_ , 2017\.
  * [15] F. Liao, M. Liang, Y. Dong, T. Pang, X. Hu, and J. Zhu, “Defense against adversarial attacks using high-level representation guided denoiser,” in _Proceedings of the Computer Vision and Pattern Recognition_ , 2018, pp. 1778–1787.
  * [16] P. Samangouei, M. Kabkab, and R. Chellappa, “Defense-gan: Protecting classifiers against adversarial attacks using generative models,” _arXiv preprint arXiv:1805.06605_ , 2018.
  * [17] C. Xie, J. Wang, Z. Zhang, Z. Ren, and A. Yuille, “Mitigating adversarial effects through randomization,” _arXiv preprint arXiv:1711.01991_ , 2017\.
  * [18] T. Pang, K. Xu, and J. Zhu, “Mixup inference: Better exploiting mixup to defend adversarial attacks,” _arXiv preprint arXiv:1909.11515_ , 2019.
  * [19] X. Liu, M. Cheng, H. Zhang, and C.-J. Hsieh, “Towards robust neural networks via random self-ensemble,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 369–385.
  * [20] A. Kurakin, I. Goodfellow, S. Bengio, Y. Dong, F. Liao, M. Liang, T. Pang, J. Zhu, X. Hu, C. Xie _et al._ , “Adversarial attacks and defences competition,” in _The NIPS’17 Competition: Building Intelligent Systems_. Springer, 2018, pp. 195–231.
  * [21] T. Pang, K. Xu, C. Du, N. Chen, and J. Zhu, “Improving adversarial robustness via promoting ensemble diversity,” in _International Conference on Machine Learning_. PMLR, 2019, pp. 4970–4979.
  * [22] A. Raghunathan, J. Steinhardt, and P. Liang, “Certified defenses against adversarial examples,” _arXiv preprint arXiv:1801.09344_ , 2018.
  * [23] A. Sinha, H. Namkoong, R. Volpi, and J. Duchi, “Certifying some distributional robustness with principled adversarial training,” _arXiv preprint arXiv:1710.10571_ , 2017.
  * [24] K. Y. Xiao, V. Tjeng, N. M. Shafiullah, and A. Madry, “Training for faster adversarial robustness verification via inducing relu stability,” _arXiv preprint arXiv:1809.03008_ , 2018.
  * [25] E. Wong, F. R. Schmidt, J. H. Metzen, and J. Z. Kolter, “Scaling provable adversarial defenses,” _arXiv preprint arXiv:1805.12514_ , 2018.
  * [26] D. Zoran, M. Chrzanowski, P.-S. Huang, S. Gowal, A. Mott, and P. Kohli, “Towards robust image classification using sequential attention models,” in _Proceedings of the computer vision and pattern recognition_ , 2020, pp. 9483–9492.
  * [27] C. Liu and J. JaJa, “Feature prioritization and regularization improve standard accuracy and adversarial robustness,” _arXiv preprint arXiv:1810.02424_ , 2018.
  * [28] H. Zhao, J. Jia, and V. Koltun, “Exploring self-attention for image recognition,” in _Proceedings of the Computer Vision and Pattern Recognition_ , 2020, pp. 10 076–10 085.
  * [29] P. Vaishnavi, T. Cong, K. Eykholt, A. Prakash, and A. Rahmati, “Can attention masks improve adversarial robustness?” in _International Workshop on Engineering Dependable and Secure Machine Learning Systems_. Springer, 2020, pp. 14–22.
  * [30] D. Goodman, X. Li, J. Huan, and T. Wei, “Improving adversarial robustness via attention and adversarial logit pairing,” _arXiv preprint arXiv:1908.11435_ , 2019.
  * [31] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: a simple and accurate method to fool deep neural networks,” in _Proceedings of the computer vision and pattern recognition_ , 2016, pp. 2574–2582.
  * [32] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in _2017 ieee symposium on security and privacy (sp)_. IEEE, 2017, pp. 39–57.
  * [33] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” 2009.
  * [34] H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” _arXiv preprint arXiv:1708.07747_ , 2017.