|  Advances in Science, Technology and Engineering Systems Journal
Vol. 5, No. 5, XX-YY (2020)
---|---
www.astesj.com |
ASTES Journal
ISSN: 2415-6698

Advanced Multiple Linear Regression Based Dark Channel Prior Applied on
Dehazing Image and Generating Synthetic Haze

Binghan Li***Binghan Li, Department of Electrical & Computer Engineering,
Texas A&M University, College Station, TX, 77840, USA, Email:
<EMAIL_ADDRESS>,1, Yindong Hua2, Mi Lu3

1Department of Electrical & Computer Engineering, Texas A&M University,
College Station, TX, 77840, USA

2Department of Electrical & Computer Engineering, Stony Brook University,
Stony Brook, NY, 11794, USA

3Department of Electrical & Computer Engineering, Texas A&M University,
College Station, TX, 77840, USA

A R T I C L E I N F O |  |  A B S T R A C T
---|---|---
Article history:
Received: 25 December, 2020
Accepted: 02 March, 2021
Online: 11 March, 2021
Keywords:
Haze Removal
Dark Channel Prior
Multiple Linear Regression
Convolutional Neural Network
Object Detection
Synthetic Haze

|  |  Haze removal is an extremely challenging task, and object detection in the hazy environment has recently gained much attention due to the popularity of autonomous driving and traffic surveillance. In this work, the authors propose a multiple linear regression haze removal model based on a widely adopted dehazing algorithm named Dark Channel Prior. Training this model with a synthetic hazy dataset, the proposed model can reduce the unanticipated deviations generated from the rough estimations of transmission map and atmospheric light in Dark Channel Prior. To increase object detection accuracy in the hazy environment, the authors further present an algorithm to build a synthetic hazy COCO training dataset by generating the artificial haze to the MS COCO training dataset. The experimental results demonstrate that the proposed model obtains higher image quality and shares more similarity with ground truth images than most conventional pixel-based dehazing algorithms and neural network based haze-removal models. The authors also evaluate the mean average precision of Mask R-CNN when training the network with synthetic hazy COCO training dataset and preprocessing test hazy dataset by removing the haze with the proposed dehazing model. It turns out that both approaches can increase the object detection accuracy significantly and outperform most existing object detection models over hazy images.

##  1  Introduction

This paper is an extension of work initially presented in conference name [1].
Computer vision has recently played a major role in broad applications on
urban traffic, such as autonomous and assisted driving, traffic surveillance,
and security maintenance. However, the existence of haze, mist, dust, and
fumes can severely degrade the visibility of images captured outside. Haze
generates reduced contrasts, fainted surfaces, and color distortion to outdoor
scenes, which will inevitably complicate many advanced computer vision tasks,
including object classification and segmentation. Since the depth information
of haze is non-linear and dependent over a global scene, haze removal becomes
a challenging task. Most computer vision algorithms are designed based upon
haze-free input images. They benefit a lot from haze removal, making it a
highly desired task in computational photography and computer vision
applications.

Many algorithms have been proposed to restore clear images from hazy images.
Polarization-based methods [2] [3] [4] analyze the polarization effects of
atmospheric scattering and remove haze through as few as two images with
different degrees of polarization. Depth-based approaches [5] [6] evaluate
depth information upon some assumptions or priors, then estimate transmission
map $t(x)$ and atmospheric light $A$ from it. Some recent CNN-based haze-
removal models [7] [8] [9] are built upon various powerful CNNs to self-learn
transmission map $t(x)$ directly from large-scale image datasets. Among
effective conventional dehazing algorithms, Dark Channel Prior (DCP) [10] is
generally accepted due to its novel prior and outstanding performance. In most
non-sky patches of the haze-free image, at least one color channel contains
dark pixels with extremely low intensity, which is primarily generated by the
air light. However, the estimation on the medium transmission $t(x)$ and
atmospheric light $A$ is not precise, especially when the scene object is
inherently similar to the air light over a large local region and no shadow is
cast on it. And the restored image looks unnaturally dark when there is a sky
region with sunlight.

In this paper, the authors propose a novel Multiple Linear Regression Dark
Channel Prior based model (MLDCP). Trained with the training dataset in
REalistic Single Image DEhazing (RESIDE) [11], the MLDCP model can optimize
the rough estimation of transmission map $t(x)$ and atmospheric light $A$ by
self-learning. The authors show experimentally on RESIDE test dataset that
their model achieves the highest SSIM and PSNR values (two important full-
reference metrics) compared with DCP and some other well-known state-of-the-
art dehazing algorithms and CNN-based architectures. The authors further
evaluate the effect on object detection in the hazy environment when dehazing
the test images by their MLDCP model. Besides, the experimental results
demonstrate that MLDCP not only enhances the performance of object detection
in the hazy environment but also outperforms most dehazing algorithms on this
task with higher detection accuracy.

The authors also present a straightforward and flexible algorithm to generate
synthetic haze to any existing image datasets, inspired by a reversed MLDCP
model. The authors aim to enhance object detection performance in the hazy
environment by utilizing synthetic hazy images as training datasets. In the
experiment, this algorithm is applied to MS COCO training dataset [12] by
adding synthetic haze to the images and build a new Hazy-COCO training
dataset. The authors evaluate the mean average precision (mAP) of Mask R-CNN
[13], a widely adopted object detection and segmentation model, by training
the network with the Hazy-COCO training dataset. The experimental results
indicate that it leads to an impressive improvement when preprocessing
training datasets with the inverse MLDCP algorithm.

##  2 Related Work

###  2.1 Overview of Dehazing Algorithms

####  2.1.1 Background Knowledge

In computer vision, the widely used atmospheric scattering model to describe
the generation of a hazy image is as follows:

$\textbf{{I}}(x)=\textbf{{J}}(x)\textit{t}(x)+\textbf{{A}}(1-\textit{t}(x))$
(1)

where $I(x)$ is the observed intensity (hazy image), $J(x)$ is the scene
radiance (haze-free image), $t(x)$ is the medium transmission map, and $A$ is
the atmospheric light. The first term $J(x)t(x)$ is called attenuation and the
second term $A(1-t(x))$ is called airlight [14].

The medium transmission map $t(x)$ describes the portion of the light that is
not scattered and reaches the camera [10]. When the atmosphere is homogeneous,
the transmission matrix $t(x)$ can be defined as:

$\textit{t}(x)=\textit{e}^{-\beta d(x)}$ (2)

where $\beta$ is the scattering coefficient of the atmosphere, and $d(x)$ is
the scene depth representing the distance between the object and camera.

Most state-of-the-art single image dehazing algorithms exploit the atmospheric
scattering model (1) and estimate the transmission matrix $t(x)$ and the
atmospheric light $A$ in either physically grounded or data-driven ways. Then
the haze-free images $J(x)$ can be recovered by computing the reformulation of
(1):

$\textbf{{J}}(x)=\frac{1}{\textit{t}(x)}\textbf{{I}}(x)-\textbf{{A}}\frac{1}{\textit{t}(x)}+\textbf{{A}}$
(3)

####  2.1.2 Conventional Single Image Dehazing Algorithms

Haze removal is a challenging task due to the non-linear and dependent depth
maps over a global scene in hazy images. Many efforts have been made to tackle
this challenge by exploiting natural images priors and depth statistics. Most
conventional dehazing algorithms focus on predicting two critical parameters,
medium transmission matrix $t(x)$ and global atmospheric light $A$, which are
necessary to recover haze-free images via computing (3). In [14], an automated
method is proposed based on the observation that the contrast of a haze-free
image is higher than that of a hazy image. Furthermore, a Markov Random Fields
(MRFs) framework is implemented to estimate the atmospheric light $A$ by
maximizing the local contrast of a hazy image. The output results are visually
impressive but may not be physically valid. Assuming that the transmission and
surface shading is uncorrelated in local areas, [15] eliminates the scattered
light by locally estimating the optical transmission map of the scene with
constant constraints. Despite its compelling results, it may fail in the cases
with heavy haze and lead to the inaccurate estimation of color and depth maps.

[10] proposed a widely recognized single image dehazing algorithm called Dark
Channel Prior (DCP), which can estimate the transmission map $t(x)$ more
reliably. A regular pattern is found that in most non-sky patches of haze-free
images, at least one color channel (dark channel) has some pixels whose
intensity is very low and even close to zero. Then this pixel-based
observation can be formally described by defining the dark channel $J^{dark}$
as:

$\textbf{{J}}^{dark}(x)=\min_{y\in\Omega(x)}(\min_{c}\textbf{{J}}^{c}(y))\approx
0$ (4)

where $c$ indicates RGB color channels and $y$ refers to the pixel in a local
patch $\Omega(x)$ centered at $x$. Adding minimum operators to both sides of
the transformation of (1):

$\min_{y\in\Omega(x)}(\min_{c}\frac{\textbf{{I}}^{c}(y)}{\textbf{{A}}^{c}})=\widetilde{t}\min_{y\in\Omega(x)}(\min_{c}\frac{\textbf{{J}}^{c}(y)}{\textbf{{A}}^{c}})+1-\widetilde{t}$
(5)

Transmission map $\widetilde{t}$ can be put outside of the minimum operators
based on the fact that $\widetilde{t}$ is a constant in the patch.

Since the dark channel of a haze-free image can be approximately taken as 0,
the multiplicative term in (5) can be eliminated by adding (4). Then
transmission map $\widetilde{t}$ can be predicted by:

$\widetilde{t}=1-\omega\min_{y\in\Omega(x)}(\min_{c}\frac{\textbf{{I}}^{c}(y)}{\textbf{{A}}^{c}})$
(6)

The additional parameter $\omega$ is a constant parameter that optionally
controls the degree of haze removal. Even in a haze-free image, the haze still
exists among distant objects. A small amount of haze will keep the vision
perceptual natural with the sense of depth. The dehazing parameter determines
how much haze will be removed.

In case that the recovered scene radiance $J(x)$ is prone to noise when
transmission map $t(x)$ is extremely low, DCP restricts $t(x)$ by a lower
bound $t_{0}$, which is set to 0.1 in [10]:

$\widetilde{t}=\max(t(x),t_{0})$ (7)

As for the estimation of atmospheric light $A$, [14] defines it as the color
of the most haze-opaque regions, which refers to the brightest pixels in a
hazy image. However, this assumption only applies when there is no sunlight in
local regions. This limitation is optimized in [10] by considering the
sunlight and adopting the dark channel to detect the most haze-opaque. DCP
picks the top 0.1 percent brightest pixels in the dark channel, among which
the pixels with the highest intensity in an input image are selected as the
atmospheric light.

####  2.1.3 Limitations of DCP

DCP has some limitations that it may fail to accurately estimate transmission
map $t(x)$ and atmospheric light $A$, when object surfaces are essentially
analogous to the air light over a local scene without any projected shadows.
Although DCP [10] takes sunlight into consideration, the influence of sunlight
is still tremendous when there is strong sunlight in the sky region. It will
underestimate the transmission map of these objects and overestimate the haze
layer. Thus the brightness of the restored image is darker than the real-world
haze-free image. The authors compare a group of hazy images and recovered
images from DCP, the color distortion in the sky region can be observed
obviously in Fig. 1:

(a) Real-world Hazy image

(b) Dehazed image via DCP

(c) Real-world Hazy image

(d) Dehazed image via DCP

Figure 1: Limitations of DCP

Additionally, the constant parameter $\omega$ in (6) is fixed to 0.95 in [10]
without any changes corresponding to different haze distributions. Excessive
haze removal will create color distortion, and images with insufficient haze
removal will remain blurred. In order to solve this problem, [16] proposes a
dehazing parameter adaptive method based on DCP that estimates dehazing
parameter $\omega$ locally instead of globally. It can automatically adjust
the value of $\omega$ according to the distribution of haze. DCP in [10]
implements soft matting to optimize the estimation of transmission map $t(x)$,
which is further enhanced to be more accurate and efficient by utilizing an
explicit image filter called $guidedfilter$ in [17]. [18] points out that
traditional DCP has not fully exploited its potential and will generate
undesirable artifacts due to inappropriate assumptions or operations. Then it
introduces a novel method that estimates transmission map $t(x)$ by energy
minimization. The energy function combines DCP with piecewise smoothness and
obtains an outstanding performance compared to conventional pixel-based
dehazing algorithms. Several attempts have been made in [19] [20] [21] to
process the color distortion and optimize the restoration in some bright
regions, such as the sky and reflective surfaces.

####  2.1.4 Overview of CNN-based Dehazing Algorithms

In recent years, neural network has made significant progress in numerous
computer vision tasks [22] [23] [24] and natural language processing tasks
[25] [26] [27]. Various Convolutional Neural Networks (CNNs) are designed to
obtain a more accurate estimation of transmission matrix $t(x)$ by self-
learning the mapping between hazy images and corresponding transmission maps,
which outperform most conventional dehazing algorithms. DehazeNet in [8] is an
end-to-end system built upon a deep convolutional neural network whose layers
are specially designed to embody established priors in haze removal.
Furthermore, a novel non-linear activation function is executed to improve the
quality of output recovered images. [9] proposes a multi-scale deep neural
network to dehaze a single image, which consists of a coarse-scale stage that
roughly predicts the transmission matrix $t(x)$ over a global view, and a
fine-scale stage that refines the rough estimation locally. [28] utilize the
DCP energy function as the loss function in a fully-convolutional dilated
residual network. Feeding the network with real-world outdoor images, it
minimizes the loss function completely unsupervised during the training
process. A light-weight CNN model called All-in-One Dehazing Network (AOD-Net)
[7] is designed based on a reformulated atmospheric scattering model. It
generates the recovered images directly and can be widely embedded into other
deep CNN models to enhance the performances of some high-level tasks over hazy
images.

####  2.1.5 Dehazing Benchmark Datasets and Metrics

Traditional haze-removal algorithms used to evaluate and compare dehazing
performances by merely presenting a group of hazy images and dehazed images
restored from various dehazing algorithms. The enhancement of dehazing
performance is expected to be observed from the visual comparison of images.
However, it is not convincing to prove that a new dehazing algorithm
outperforms other algorithms only from human eyes perception. Two widely
adopted image metrics to evaluate and compare single image dehazing algorithms
are PSNR and SSIM [29]. PSNR refers to the peak-signal-to-noise ratio, which
is generally applied to evaluate the image quality. SSIM refers to the
structural similarity index measure, a well-known metric to measure the
similarity between two images. Since it is generally impossible to capture the
same visual scene with and without haze, while all other environmental
conditions stay identical, it is incredibly challenging to measure the SSIM
value between a hazy image and its haze-free ground truth image. Therefore,
recent efforts have been made to create synthetic hazy images from haze-free
images based on the depth information.

[30] builds two sets of images without haze and with synthetic haze from both
real-world camera captured scenes and synthetic scenes to evaluate the
performance of the proposed visibility enhancement algorithm. They utilize the
software to generate 66 synthetic images built upon a physically-based road
environment. By obtaining the depth information, four different fog types are
added to 10 camera images and finally create a dataset with over 400 synthetic
images in total. A fog simulation is proposed in [31] by simulating the
underlying mechanism of hazy image formation (1) and utilizing the standard
optical model for daytime haze. The fog simulation pipeline is leveraged to
add synthetic fog to urban scenes images in Cityscapes dataset [32] and
generate a Foggy Cityscapes dataset. Foggy Cityscapes dataset consists of 550
refined high-quality synthetic foggy images with detailed semantic annotations
and additional 20000 synthetic foggy images without sufficient annotations.

The REalistic Single Image DEhazing (RESIDE) dataset [11] is the first large-
scale dataset for benchmarking single image dehazing algorithms, and it
includes both indoor and outdoor hazy images. RESIDE dataset also contains a
large-scale synthetic training set and two sets designed respectively for
objective and subjective quality evaluations. Moreover, in the supplementary
RESIDE-$\beta$ set, they add annotations and object bounding boxes to an
evaluation set consisting of 4322 real-world hazy images, which can be
utilized to test the performance of object detection in the hazy environment.
[11] also provides a rich variety of criteria beyond PSNR and SSIM to evaluate
the performance of dehazing algorithms, including full-reference metrics, no-
reference metrics, subjective evaluation, and task-driven evaluation. However,
the criteria are only practically applicable to the global performance of
haze-removal. It cannot embody the difference locally between two images,
failing to judge if our MLDCP model outperforms traditional DCP specifically
in the bright region. Therefore, the authors compare dehazing performances
experimentally with both recovered images’ visual quality and two pivotal
metrics PSNR and SSIM.

###  2.2 Overview of Object Detection Models

Object detection is the combination of object classification and localization,
which can both recognize and localize all object instances of specific
categories in an image. Due to its close relationship with image and video
analysis, object detection has been widely applied in various computer vision
tasks, especially in autonomous driving, traffic surveillance, and some other
smart city fields.

Fast R-CNN refers to Fast Region-based Convolutional Network [33]. The network
is fed with an image and a set of object proposals and outputs a convolutional
feature map. A region of interest (RoI) pooling layer is proposed to extract a
fixed-length feature vector and feed it into a sequence of fully connected
layers. The output layers contain a softmax layer that estimates the softmax
probability over $K$ object classes plus a background class, and another layer
with offset values that refine the bounding box positions of an object. Fast
R-CNN overcomes the disadvantages of R-CNN and SPPnet [34] with a higher
detection accuracy as well as faster training and test speed. Later in [35], a
Region Proposal Network (RPN) is added into Fast R-CNN generating a Faster
R-CNN. The RPN aims to simultaneously propose candidate object bounding boxes
and corresponding scores at each position. It then implements RoIPool in Fast
R-CNN to extract features from each candidate box and perform classification
as well as bounding-box regression. Both RPN and Fast R-CNN are trained
independently but share the same convolutional layers. While achieving state-
of-the-art object detection accuracy, the unified network can further increase
the speed significantly.

Mask R-CNN [13] extends the Faster R-CNN by an output branch with a binary
mask for each RoI, in parallel to a branch for bounding-box recognition and
classification. However, the quantization of RoIPool in Faster R-CNN has a
negative effect on predicting binary masks. To remove this harsh quantization,
Mask R-CNN replaces RoIPool with a RoIAligh layer that aligns the extracted
features with the input properly. It can efficiently detect object instances
in an image and simultaneously generate a pixel-accurate segmentation mask for
each instance. Due to its compelling performance and influential architecture,
Mask R-CNN is widely used as a solid baseline to exploit more object detection
tasks.

###  2.3 Domain Adaption Methods

Domain adaptation is a novel strategy that can be utilized to advance object
detection models [36] [37] [38]. And it has been proved effective, especially
in some extreme weather [39] [40], such as hazy, rainy, and snowy. [39] has
proposed a Domain Adaptive Faster R-CNN to enhance the cross-domain robustness
of object detection. Based on H-divergence theory, two domain adaptation
components on image level and instance level are integrated into Faster R-CNN
architecture, aiming to reduce the domain discrepancy at both levels. Training
data with images and full supervision is used as the source domain, and only
unlabeled images in test data are available for the target domain. For both
components, it adapts the classifier trained on a source domain and implements
the adversarial training strategy to learn domain-invariant features. The
method further incorporates a consistency regularization into the Faster R-CNN
model to obtain a domain-invariant region proposal network (RPN).

Inspired by Domain Adaptive Faster R-CNN in [39], [41] adopts a similar idea
and designs a Domain-Adaptive Mask-RCNN (DMask-RCNN). The source domain takes
the clean images in the MS COCO dataset, and the target domain takes
unannotated real-world hazy images in RESIDE dataset [11], and their dehazed
output images by MSCNN [9] respectively. And DMask-RCNN adds a domain-adaptive
branch after the base feature extraction layers in Mask R-CNN architecture,
aiming to mask the generated features to be domain-invariant between the
source domain and target domain. The experimental results in [39] [41]
demonstrate that the domain adaptation method can enhance the performance of
both Faster R-CNN and Mask R-CNN models when tackling the object detection
task in the hazy environment. Moreover, this enhancement can be more effective
when feeding the target domain with images restored by a robust dehazing
algorithm.

###  2.4 Object Detection Datasets

Object recognition is a core task to understand a visual scene in computer
vision, which involves several sub-tasks, including image classification,
object detection, and semantic segmentation. All three tasks have high demands
for image datasets. Object classification requires each image to be labeled
with a sequence of binary numbers, which indicate if object instances exist in
the image or not. Object detection is more challenging, which combines object
classification and localization. It not only identifies which specific class
an object belongs to but also locates it in an image. The object localization
requires collecting bounding boxes that locate one or more objects in an
image, which is a considerable workload in a large-scale image dataset. The
PASCAL VOC challenge [42] is a widely recognized benchmark in visual object
recognition and detection. The VOC2007 dataset contains 20 object categories
spread over 11000 images. The annotation procedure is designed to be
consistent, accurate, and exhaustive, and annotations are made available for
training, validation, and test data. The ImageNet dataset [43] involves
millions of cleanly labeled and full-resolution images in the hierarchical
structure of WordNet [44]. The dataset provides an average of 500-1000 images
to illustrate each synset in a majority of 80000 synsets of WordNet. Many
object recognition algorithms have made a significant breakthrough by using
the training resource of the ImageNet dataset. This benchmark dataset also
plays an essential role in advancing object recognition tasks.

Semantic segmentation requires labeling each pixel in an image to a category,
and it is an extremely time-consuming task to build a large-scale dataset with
detailed semantic scene labeling. The Microsoft COCO dataset [12] consists of
about 3.3 million images, and over 2 million of them are labeled. It collects
91 common object categories, which are fewer than the ImageNet dataset.
However, in contrast to both the VOC2007 dataset and ImageNet dataset, MS COCO
dataset contains significantly more instances per category and considerably
more object instances per image. And every instance of each object category is
fully labeled and segmented by a novel instance-level segmentation mask. MS
COCO dataset has been widely utilized for training some more complex CNN
architectures that aim to make further progress on object recognition as well
as semantic segmentation tasks.

##  3 Proposed Methods

###  3.1 Multiple Linear Regression DCP Model

DCP estimates transmission map $t(x)$ from (6) and selects the pixels in the
dark channel with the highest intensity among the top 0.1 percent brightest
pixels as the atmospheric light $A$. Then hazy images can be recovered from
(3) motivated by the estimations of $t(x)$ and $A$. However, the rough
estimations in DCP introduced in Section 2.1.3 can generate some unpredicted
deviations, which are theoretically impossible to be eliminated during the
estimation process. Now the authors implement a multiple linear regression
model to optimize the haze-removal algorithm in DCP.

Multiple linear regression is a statistical technique using several
explanatory variables to predict the output response variable. As a predictive
analysis, multiple linear regression aims to model the linear relationship
between two or more independent variables and a continuous dependent variable.
The authors still adopt the rough estimations on transmission map $t(x)$ and
atmospheric light $A$ in DCP. Three components
$\frac{\textbf{{I}}(x)}{\textit{t}(x)}$, $\frac{\textit{A}}{\textit{t}(x)}$
and A in (3) can be regarded as explanatory variables, since $\textbf{{I}}(x)$
refers to the pixel of input hazy image while $t(x)$ and $A$ can be estimated
from DCP. The scene radiance $\textbf{{J}}(x)$, which refers to the pixel of
output recovered image, can be regarded as the response variable. The authors
add regression coefficient weights to each explanatory variable and a constant
term (bias) to the atmospheric scattering model. Then (3) can be reformulated
as a multiple linear regression model (8), which describes how the mean
response $\textbf{{J}}(x)$ changes with explanatory variables:

$\textbf{{J}}(x)=\omega_{0}\frac{\textbf{{I}}(x)}{\textit{t}(x)}+\omega_{1}\frac{\textbf{{A}}}{\textit{t}(x)}+\omega_{2}\textbf{{A}}+b$
(8)

As the authors introduced, DCP is already an effective dehazing algorithm even
with the rough estimations on $t(x)$ and $A$. The authors implement a multiple
linear regression model to optimize DCP by learning the relationships between
hazy images with haze-free images, which is extremely challenging to find out
in traditional pixel-based dehazing algorithms. Since both $\textbf{{J}}(x)$
and $A$ are defined and estimated on RGB color channels, the dimensions of
three weights and a bias should be (3,1), intending to refine the parameters
at all three color channels. The Outdoor Training Set (OTS) in RESIDE dataset
[11] provides 8970 outdoor haze-free images, each of which also contains 30
synthetic hazy images with haze intensity ranging from low to high. During the
training process, the network learns the relationships between output
recovered images and their relevant haze-free ground truth images. However, it
is theoretically impossible to obtain the ground truth image of a real-world
hazy image. The most efficient approach is referring to a real-world haze-free
image as ground truth image and its synthetic hazy image as input image
respectively. The various intensity of synthetic haze guarantees that the
MLDCP model can be applied to more than a fixed haze intensity in real-world
scenes.

When MLDCP model is trained on OTS dataset, the authors refer to haze-free
images as target ground truth images J, refer to synthetic hazy images as
input images I, and refer to the recovered images as output images
$\textbf{{J}}_{\omega}$. In order to simplify the formula (8) during the
training process, the authors assign
$\textbf{{x}}_{0}=\frac{\textbf{{I}}(x)}{\textit{t}(x)}$,
$\textbf{{x}}_{1}=\frac{\textit{A}}{\textit{t}(x)}$ and
$\textbf{{x}}_{1}=\textit{A}$. Then (8) can be reformulated as:

$\textbf{{J}}_{\omega}(x)=\omega_{0}\textbf{{x}}_{0}+\omega_{1}\textbf{{x}}_{1}+\omega_{2}\textbf{{x}}_{2}+b$
(9)

The deviations between the output images $\textbf{{J}}_{\omega}$ and the
target ground truth images J are estimated by mean-squared error (MSE):

$\textbf{{MSE}}=(\textbf{{J}}-\textbf{{J}}_{\omega})^{2}$ (10)

MSE measures the average squared difference between an observation’s actual
and predicted values, which is commonly adopted in linear regression models.
The loss function of MLDCP is evaluated based upon MSE:

$\textbf{{L}}(\omega)=\frac{1}{2n}\sum_{i=1}^{n}(\textbf{{J}}^{(i)}-\textbf{{J}}_{\omega}^{(i)})^{2}$
(11)

During the training process, the authors calculate the gradient of loss
function by Stochastic Gradient Descent (SGD), a widespread and effective
technique to address optimization tasks. Then the optimal weights and bias can
be obtained by minimizing MSE through the following steps:

  * •

Implement SGD to compute the derivative of loss function to find the gradient
of the error generated from current weights and bias, and adjusts the weights
by moving in the direction opposite of the gradient to decrease MSE. Then
authors iteratively update three weights via:

$\omega_{k}=\omega_{k}-\alpha\frac{\partial}{\partial\omega}\textbf{{L}}(\omega)$
(12)

where $\alpha$ is the learning rate, and $k\in{(0,1,2)}$.

  * •

Expand the derivative of loss function (11), then (12) can be reformulated as:

$\omega_{k}=\omega_{k}-\alpha\frac{1}{n}\sum_{i=1}^{n}(\textbf{{J}}^{(i)}-\textbf{{J}}_{\omega}^{(i)})x_{k}$
(13)

  * •

Update the bias with the same process repeatedly by training each image in OTS
dataset via:

$b=b-\alpha\frac{1}{n}\sum_{i=1}^{n}(\textbf{{J}}^{(i)}-\textbf{{J}}_{\omega}^{(i)})$
(14)

When the optimal weights and bias are obtained from the training process, they
can be added to the haze-removal model (8) directly, and real-world hazy
images can be dehazed in the same way as that in conventional DCP. The
improved performance is determined by the quality of synthetic hazy images. If
the synthetic haze is more close to the real-world haze, the MLDCP model can
learn more realistic haze information during the training process, and the
dehazing performance can be further enhanced.

###  3.2 Synthetic Haze Generated Model

There have been some large-scale image datasets released for the object
detection task. However, few image datasets are built just for object
detection in extreme weather like hazy and rainy. The Real-world Task-driven
Testing Set (RTTS) in RESIDE [11] contains 4322 real-world hazy images
annotated with both object categories and bounding boxes, which is an
essential contribution taken as a test dataset for object detection task in
the hazy environment. However, even with data augmentation techniques, its
amount of training images is far less efficient in training an object
detection model in comparison with the larger scale of the MS COCO training
dataset [12]. It means a considerable workload to build a large-scale image
dataset with detailed annotations and segmentation instances like MS COCO or
PASCAL VOC. And the rarity of real-world hazy images, especially in urban
cities, makes it more challenging to collect the same amount of high-quality
hazy images as those in the MS COCO training dataset.

The authors propose an algorithm that generates synthetic haze to any existing
large-scale haze-free image datasets without much computation. Inspired by the
multiple linear regression haze-removal model, the authors propose a new
algorithm that implements a similar inverse MLDCP model to add synthetic haze
to an image. If an object detection model is trained on the synthetic hazy
dataset generated by this algorithm, its performance of detecting an object in
hazy weather can be significantly enhanced.

Since DCP is an effective dehazing algorithm, and the multiple linear
regression model can further exploit its potential, the authors suppose that
similar ideas can also be applied to synthesize higher quality hazy images
based on the regular dark channel pattern. Coefficient weights and bias are
added to (1), which is expected to maximally restore the deviations between
haze-free images and their relevant synthetic hazy images. Then the
reformulated atmospheric scattering model is obtained as follows:

$\textbf{{I}}(x)=\beta_{0}\textbf{{J}}(x)\textit{t}(x)+\beta_{1}\textbf{{A}}\textit{t}(x)+\beta_{2}\textbf{{A}}+b$
(15)

The synthetic haze generated model is still trained on OTS in RESIDE dataset
[11]. In contrast to MLDCP model, the authors refer to haze-free images as
input images J and refer to synthetic hazy images in OTS as target images I.
And the output synthetic hazy images are taken as $\textbf{{I}}_{\beta}$. The
cost function is defined based on MSE that evaluates the deviations between
the output images $\textbf{{I}}_{\beta}$ and target images I:

$\textbf{{L}}(\beta)=\frac{1}{2n}\sum_{i=1}^{n}(\textbf{{I}}^{(i)}-\textbf{{I}}_{\beta}^{(i)})^{2}$
(16)

The authors implement SGD during the training process to travel down the slope
of cost function until it reaches the bottom lowest value, and three weights
and a bias are updated iteratively by calculating the derivative of cost
function as follows:

$\beta_{k}=\beta_{k}-\alpha\frac{1}{n}\sum_{i=1}^{n}(\textbf{{I}}^{(i)}-\textbf{{I}}_{\beta}^{(i)})x_{k}$
(17)

$b=b-\alpha\frac{1}{n}\sum_{i=1}^{n}(\textbf{{I}}^{(i)}-\textbf{{I}}_{\beta}^{(i)})$
(18)

where $k\in{(0,1,2)}$ and $x_{k}$ equals to $\textbf{{J}(x)}\textit{t}(x)$,
$\textit{A}\textit{t}(x)$, and A in (15) respectively.

In fact, traditional DCP does not support the inverse functionality of
generating synthetic haze to a haze-free image. Self-learning the haze
information from input synthetic hazy images, the MLDCP model fully utilizes
the superiority of DCP to generate a much higher quality synthetic haze with a
more similar visual and inherent effect to real-world haze. Additionally, the
inverse MLDCP model can add various intensities of haze to an image dataset,
making it possible to train an object detection model on training datasets
with different synthetic haze intensities corresponding to the density of
real-world haze in test images. Since the method focuses on the enhancement by
preprocessing the training dataset, it can be applied simultaneously with any
improvements on the object detection models or preprocessing images in test
datasets with more effective dehazing algorithms.

##  4 Experiment Results and Analysis

###  4.1 MLDCP Dehazing Performance

####  4.1.1 Experiment Setup

In the experiment, both training and test datasets are obtained from RESIDE
dataset [11]. In the training process, 8970 haze-free images and their
corresponding synthetic hazy images in the OTS dataset are utilized for
training the MLDCP model. In the test phase, the Synthesis Object Testing Set
(SOTS) is used to test the dehazing performance. Since MLDCP focuses on
recovering outdoor hazy scenes, the authors do not evaluate its dehazing
performance over the 500 synthetic indoor images in SOTS. The dehazing
performance of MLDCP is compared with conventional DCP on both SSIM and PSNR
metrics over 500 outdoor synthetic hazy images and their haze-free ground
truth images.

####  4.1.2 Dehazing Performance on SOTS Dataset

In Fig. 2, the authors present the comparison of recovered images by
conventional DCP and our MLDCP model. It is obviously observed that the
recovered image by DCP (c) suffers from the color distortion in the sky
region. In contrast, the recovered image by MLDCP (d) performs competently
even in the sky region with strong sunlight. And the non-sky region in (c) is
much darker than that in (d), and it looks unreal. In comparison with the
input synthetic hazy image (a), MLDCP almost removes the haze completely.
Besides, its recovered image (d) looks natural and shares a high similarity
with the ground truth haze-free image (b).

(a) Synthetic hazy image

(b) Haze-free image

(c) Recovered image by DCP

(d) Recovered image by our model

Figure 2: Comparison on synthetic hazy image. (a) is a synthetic hazy image in
SOTS, (b) is the haze-free ground truth image, (c) is recovered by
conventional DCP algorithm, (d) is recovered by our MLDCP model. Table 1:
Average SSIM and PSNR comparison among different dehazing algorithms over 500
outdoor synthetic hazy images in SOTS. 500 outdoor images
---
Dehazing method name | PSNR | SSIM
Improved DCP model | 23.84 | 0.9411
DCP | 18.54 | 0.7100
FVR | 16.61 | 0.7236
CAP | 23.95 | 0.8692
NLD | 19.52 | 0.7328
BCCR | 17.71 | 0.7409
GRM | 20.77 | 0.7617
DehazeNet | 26.84 | 0.8264
MSCNN | 21.73 | 0.8313
AOD-Net | 24.08 | 0.8726

The authors compare the dehazing performance of MLDCP with several
conventional dehazing algorithms as well as some CNN-based haze removal models
over two major image benchmark metrics PSNR and SSIM. PSNR evaluates the
quality of recovered images, while SSIM measures the similarity between
recovered images and haze-free ground truth images. In Table 1, MLDCP model
increases PSNR and SSIM by 5.3 and 0.23 respectively in comparison with the
conventional DCP. Compared with the performances of other widely adopted
dehazing algorithms obtained from [11], the proposed MLDCP model achieves a
reasonably acceptable PSNR value and the highest SSIM value. Considering that
SSIM is essentially more dominant than PSNR when indicating the pixel-wise
effect of haze removal, MLDCP outperforms all other dehazing algorithms in
Table 1.

####  4.1.3 Real-world Hazy Images Dehazing

Figure 3: Visual comparison between real-world hazy images (first column),
recovered images from DCP (second column) and recovered images from MLDCP
(third column).

Even though synthetic benchmark datasets and metrics have achieved great
success in comparing dehazing performances, the visual effect of synthetic
haze is still different from the real-world haze. In Fig. 3, the authors also
present the recovered images by conventional DPC and MLDCP from real-world
nature hazy scenes (first three rows) and hazy scenes in urban cities (last
three rows). It is obviously observed that the MLDCP model performs better
than conventional DCP in both sky and non-sky regions. The output images
restored by MLDCP prevent the color distortion in sky regions, and the
brightness over a global view seems more natural and closer to realistic haze-
free scenes.

###  4.2 Object Detection with Pre-dehazed Test Dataset

####  4.2.1 Experiment Setup

Some object detection tasks on RESIDE RTTS dataset have been tested in [7]
[11] [41]. The Domain-Adaptive Mask R-CNN model proposed in [41] achieves the
highest detection accuracy on RTTS test dataset among several well-known
object detection models including Faster R-CNN [35], Mask R-CNN [13], SSD [45]
and RetinaNet [46]. [7] proposed a concatenation of dehazing algorithm and
object detection modules to detect objects in the hazy environment. [41]
further tried more combinations of effective dehazing algorithms and object
detectors in the cascade and evaluated their performances on the mean average
precision (mAP) values. MLDCP model is utilized to dehaze the RTTS test
dataset, and Mask R-CNN and DMask-RCNN modules are fed with the recovered
images. Then the authors compare the detection accuracy with the results of
dehazing-detection cascades in [41].

In Table 2, the proposed cascade of MLDCP dehazing model and Mask R-CNN or
Mask R-CNN increases the detection accuracy by about 2% or 1.7% respectively,
which is the highest in comparison with other dehazing approaches, including
AOD-Net [7], MSCNN [9] and conventional DCP [10]. And pre-dehazing RTTS
dataset with MSCNN and DCP can both enhance the performance of object
detection in the hazy environment. In [7], AOD-Net outperforms several
dehazing algorithms on PSNR and SSIM values. However, it decreases the mAP
result of DMask R-CNN from 61.72% to 60.47%, which means that a better
dehazing performance on PSNR and SSIM values does not align with a higher
accuracy on object detection in the hazy environment.

Table 2: Object detection accuracy (mAP) comparison among different dehazing-detection cascades Framework | mAP(%)
---|---
Mask R-CNN | 61.01
DMask R-CNN2 | 61.72
MLDCP + Mask R-CNN | 63.06
AOD-Net + DMask R-CNN2 | 60.47
MSCNN + DMask R-CNN2 | 63.36
DCP + DMask R-CNN2 | 62.78
MLDCP + DMask R-CNN2 | 63.42

###  4.3 Object Detection with Hazy-COCO Dataset

####  4.3.1 Experiment Setup

The authors train Mask R-CNN on the new Hazy-COCO training dataset and
evaluate its performance in comparison with the original Mask R-CNN trained on
the MS COCO training dataset. During the training process, the backbone
network is set as resnet101, and the training starts with the pre-trained COCO
weights. The authors also tried various combinations of training stages to
check the effect on its detection performance. It turns out that when training
the head layers by ten epochs with a learning rate of 0.001 and fine-tuned
layers from Resnet stage 4 and up by 60 epochs with a learning rate of 0.0001,
the authors obtained the weights and bias with the highest detection accuracy
on both RTTS and UG2 test datasets.

Figure 4: Comparison on synthetic hazy image. (a) is a haze-free image in MS
COCO, (b) is the synthetic hazy image generated by inverse DCP, (c) is the
synthetic hazy image with 0.1 haze density generated by our algorithm, (d) is
the synthetic hazy image with 0.2 haze density generated by our algorithm.

####  4.3.2 Experiment Results with Hazy-COCO Training Dataset

Table 3: Object detection accuracy (mAP) comparison between MS COCO and Hazy-COCO training datasets Test Dataset | Training Dataset | mAP(%)
---|---|---
RTTS | COCO | 61.01
| Hazy COCO | 66.08
RTTS+MLDCP | COCO | 63.06
| Hazy COCO | 66.15
UG2 | COCO | 32.07
| Hazy COCO | 38.53

In Table.3, the authors evaluate the difference in detection performances when
Mask R-CNN is pre-trained on MS COCO and Hazy-COCO datasets. The mAP results
increase significantly by about 5% and 6% on RTTS and UG2 test datasets
respectively, which presents how effective the Hazy-COCO training dataset is.
Additionally, when training Mask R-CNN on the Hazy-COCO dataset, dehazing the
RTTS test dataset by the MLDCP model can simultaneously increase the mAP
result by 0.07%. However, this increment is much lower than the increment 2%
when we preprocess the RTTS dataset individually. This is caused by the
overlaps of enhancement effects between MLDCP and the advanced inverse DCP
since they share some similarities over the dark channel regular pattern and
multiple linear regression model.

####  4.3.3 Visual Effect of Synthetic Haze

Generally, traditional DCP does not support generating synthetic haze to a
haze-free image by simply implementing the dark channel regular pattern
inversely. This limitation becomes possible after the authors apply the
multiple linear regression techniques to the atmospheric scattering model.

Figure 5: Visual comparison between haze-free outdoor images from MS COCO
dataset (left column) and the synthetic hazy images with 0.2 haze density
generated by our algorithm (right column).

In Fig. 4, the effect of haze generated by the inverse DCP algorithm can be
barely observed in (b), in comparison with the haze-free image (a). Synthetic
hazy images created by our advanced inverse DCP algorithm with haze density
0.1 and 0.2 are presented as (c) and (d) respectively. Both of the two
synthetic hazy images are covered with seemingly realistic haze. And it is
obviously observed that the synthetic haze in (d) is denser than the haze in
(c). The authors also present four more groups of haze-free images from the MS
COCO dataset and their corresponding synthetic hazy images in Fig. 5 to
demonstrate the effectiveness of our algorithm.

##  5 Conclusion

As autonomous driving and traffic surveillance become widespread in smart
cities, object detection in extreme environments like hazy has been paid
special attention to. This paper proposed the approaches that increase the
detection accuracy in the hazy environment from both dehazing test dataset and
preprocessing training dataset.

First, an advanced Multiple Linear Regression Haze-removal Model was proposed,
aiming to overcome the deficiencies of Dark Channel Prior. The authors
implemented Stochastic Gradient Descent to update and find the optimal weights
and bias to refine the rough estimation of two essential parameters
transmission matrix $t(x)$ and atmospheric light $A$. The experimental results
showed that MLDCP not only achieved higher PSNR and SSIM values than other
state-of-the-art dehazing algorithms and CNN-based dehazing models, but also
increased the detection precision by a higher rate when concatenated with
object detection models. It demonstrated that sometimes exploiting practical
conventional dehazing algorithms by machine learning techniques was superior
to building more complicated neural networks.

Second, the authors proposed an inverse DCP algorithm formulated on the
multiple linear regression model that could generate synthetic haze to any
existing image datasets. The authors expected this technique to prevent from
spending excessive time and workload on building a large-scale image dataset
with synthetic or real-world haze, as well as detailed annotations for object
detection and segmentation. Synthetic haze was added to the MS COCO dataset
and trained Mask R-CNN on this Hazy-COCO training set. The experimental
results presented a significant increase in detection accuracy in the hazy
environment. However, this approach could only generate the average synthetic
haze with the same density over all pixels in an image. On the contrary, real-
world haze density normally varies from pixel to pixel. In future work, it is
expected to optimize this limitation and generate more realistic synthetic
haze.

## References

  * [1] B. Li, W. Zhang, M. Lu, “Multiple Linear Regression Haze-removal Model Based on Dark Channel Prior,” in 2018 International Conference on Computational Science and Computational Intelligence (CSCI), 307–312, IEEE, 2018\.
  * [2] Y. Y. Schechner, S. G. Narasimhan, S. K. Nayar, “Instant dehazing of images using polarization,” in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, volume 1, I–I, IEEE, 2001.
  * [3] S. Shwartz, E. Namer, Y. Y. Schechner, “Blind haze separation,” in 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), volume 2, 1984–1991, IEEE, 2006.
  * [4] M. Lu, et al., Arithmetic and logic in computer systems, volume 169, Wiley Online Library, 2004.
  * [5] J. Kopf, B. Neubert, B. Chen, M. Cohen, D. Cohen-Or, O. Deussen, M. Uyttendaele, D. Lischinski, “Deep photo: Model-based photograph enhancement and viewing,” ACM transactions on graphics (TOG), 27(5), 1–10, 2008.
  * [6] S. G. Narasimhan, S. K. Nayar, “Interactive (de) weathering of an image using physical models,” in IEEE Workshop on color and photometric Methods in computer Vision, volume 6, 1, France, 2003.
  * [7] B. Li, X. Peng, Z. Wang, J. Xu, D. Feng, “An all-in-one network for dehazing and beyond,” arXiv preprint arXiv:1707.06543, 2017.
  * [8] B. Cai, X. Xu, K. Jia, C. Qing, D. Tao, “Dehazenet: An end-to-end system for single image haze removal,” IEEE Transactions on Image Processing, 25(11), 5187–5198, 2016.
  * [9] W. Ren, S. Liu, H. Zhang, J. Pan, X. Cao, M.-H. Yang, “Single image dehazing via multi-scale convolutional neural networks,” in European conference on computer vision, 154–169, Springer, 2016.
  * [10] K. He, J. Sun, X. Tang, “Single image haze removal using dark channel prior,” IEEE transactions on pattern analysis and machine intelligence, 33(12), 2341–2353, 2010.
  * [11] B. Li, W. Ren, D. Fu, D. Tao, D. Feng, W. Zeng, Z. Wang, “Benchmarking single-image dehazing and beyond,” IEEE Transactions on Image Processing, 28(1), 492–505, 2018.
  * [12] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision, 740–755, Springer, 2014\.
  * [13] K. He, G. Gkioxari, P. Dollár, R. Girshick, “Mask r-cnn,” in Proceedings of the IEEE international conference on computer vision, 2961–2969, 2017.
  * [14] R. T. Tan, “Visibility in bad weather from a single image,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition, 1–8, IEEE, 2008.
  * [15] R. Fattal, “Single image dehazing,” ACM transactions on graphics (TOG), 27(3), 1–9, 2008.
  * [16] C. Chen, J. Li, S. Deng, F. Li, Q. Ling, “An adaptive image dehazing algorithm based on dark channel prior,” in 2017 29th Chinese Control And Decision Conference (CCDC), 7472–7477, IEEE, 2017.
  * [17] K. He, J. Sun, X. Tang, “Guided image filtering,” IEEE transactions on pattern analysis and machine intelligence, 35(6), 1397–1409, 2012.
  * [18] M. Zhu, B. He, Q. Wu, “Single image dehazing based on dark channel prior and energy minimization,” IEEE Signal Processing Letters, 25(2), 174–178, 2017.
  * [19] J.-H. Kim, W.-D. Jang, J.-Y. Sim, C.-S. Kim, “Optimized contrast enhancement for real-time image and video dehazing,” Journal of Visual Communication and Image Representation, 24(3), 410–425, 2013.
  * [20] Y. Shen, Q. Wang, “Sky region detection in a single image for autonomous ground robot navigation,” International Journal of Advanced Robotic Systems, 10(10), 362, 2013.
  * [21] L. Shi, X. Cui, L. Yang, Z. Gai, S. Chu, J. Shi, “Image Haze Removal Using Dark Channel Prior and Inverse Image,” in MATEC Web of Conferences, volume 75, 03008, EDP Sciences, 2016.
  * [22] Y. Hua, Y. Liu, B. Li, M. Lu, “Dilated Fully Convolutional Neural Network for Depth Estimation from a Single Image,” in 2019 International Conference on Computational Science and Computational Intelligence (CSCI), 612–616, IEEE, 2019.
  * [23] X. Zhang, Y. Wang, H. Wang, Y. Zhang, H. Wu, M. Zhao, “Bend Detection of Bridge Chords in UAV Images via Region-Based Deep Semantic Segmentation Network,” in 2019 IEEE 17th International Conference on Industrial Informatics (INDIN), volume 1, 1459–1463, IEEE, 2019.
  * [24] X. Wei, Y. Zhang, Z. Li, Y. Fu, X. Xue, “DeepSFM: Structure from motion via deep bundle adjustment,” in European conference on computer vision, 230–247, Springer, 2020.
  * [25] Y. Wang, X. Zhang, M. Lu, H. Wang, Y. Choe, “Attention augmentation with multi-residual in bidirectional LSTM,” Neurocomputing, 385, 340–347, 2020.
  * [26] Y. Wang, Z. Zhou, S. Jin, D. Liu, M. Lu, “Comparisons and selections of features and classifiers for short text classification,” in IOP Conference Series: Materials Science and Engineering, volume 261, 012018, IOP Publishing, 2017.
  * [27] Y. Wang, Y. Huang, W. Zheng, Z. Zhou, D. Liu, M. Lu, “Combining convolutional neural network and self-adaptive algorithm to defeat synthetic multi-digit text-based CAPTCHA,” in 2017 IEEE International Conference on Industrial Technology (ICIT), 980–985, IEEE, 2017.
  * [28] A. Golts, D. Freedman, M. Elad, “Unsupervised single image dehazing using dark channel prior loss,” IEEE Transactions on Image Processing, 29, 2692–2701, 2019.
  * [29] A. Hore, D. Ziou, “Image quality metrics: PSNR vs. SSIM,” in 2010 20th international conference on pattern recognition, 2366–2369, IEEE, 2010.
  * [30] J.-P. Tarel, N. Hautiere, L. Caraffa, A. Cord, H. Halmaoui, D. Gruyer, “Vision enhancement in homogeneous and heterogeneous fog,” IEEE Intelligent Transportation Systems Magazine, 4(2), 6–20, 2012.
  * [31] C. Sakaridis, D. Dai, L. Van Gool, “Semantic foggy scene understanding with synthetic data,” International Journal of Computer Vision, 126(9), 973–992, 2018.
  * [32] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 3213–3223, 2016.
  * [33] R. Girshick, “Fast r-cnn,” in Proceedings of the IEEE international conference on computer vision, 1440–1448, 2015.
  * [34] K. He, X. Zhang, S. Ren, J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE transactions on pattern analysis and machine intelligence, 37(9), 1904–1916, 2015.
  * [35] S. Ren, K. He, R. Girshick, J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in Advances in neural information processing systems, 91–99, 2015.
  * [36] V. M. Patel, R. Gopalan, R. Li, R. Chellappa, “Visual domain adaptation: A survey of recent advances,” IEEE signal processing magazine, 32(3), 53–69, 2015.
  * [37] W. Li, L. Duan, D. Xu, I. W. Tsang, “Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation,” IEEE transactions on pattern analysis and machine intelligence, 36(6), 1134–1148, 2013.
  * [38] R. Gopalan, R. Li, R. Chellappa, “Domain adaptation for object recognition: An unsupervised approach,” in 2011 international conference on computer vision, 999–1006, IEEE, 2011.
  * [39] Y. Chen, W. Li, C. Sakaridis, D. Dai, L. Van Gool, “Domain adaptive faster r-cnn for object detection in the wild,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 3339–3348, 2018.
  * [40] Z. Chen, X. Li, H. Zheng, H. Gao, H. Wang, “Domain adaptation and adaptive information fusion for object detection on foggy days,” Sensors, 18(10), 3286, 2018.
  * [41] Y. Liu, G. Zhao, B. Gong, Y. Li, R. Raj, N. Goel, S. Kesav, S. Gottimukkala, Z. Wang, W. Ren, et al., “Improved techniques for learning to dehaze and beyond: A collective study,” arXiv preprint arXiv:1807.00202, 2018.
  * [42] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision, 88(2), 303–338, 2010.
  * [43] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition, 248–255, Ieee, 2009.
  * [44] G. A. Miller, WordNet: An electronic lexical database, MIT press, 1998.
  * [45] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, A. C. Berg, “Ssd: Single shot multibox detector,” in European conference on computer vision, 21–37, Springer, 2016.
  * [46] T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision, 2980–2988, 2017.


# Privacy-preserving and Trusted Threat Intelligence Sharing using Distributed
Ledgers

Hisham Ali1, Pavlos Papadopoulos1, Jawad Ahmad1, Nikolaos Pitropakis1, Zakwan
Jaroucheh1 and William J. Buchanan1
1 Blockpass ID Lab, Edinburgh Napier University, Edinburgh, UK.

###### Abstract

Threat information sharing is considered as one of the proactive defensive
approaches for enhancing the overall security of trusted partners. Trusted
partner organizations can provide access to past and current cybersecurity
threats for reducing the risk of a potential cyberattack—the requirements for
threat information sharing range from simplistic sharing of documents to
threat intelligence sharing. Therefore, the storage and sharing of highly
sensitive threat information raises considerable concerns regarding
constructing a secure, trusted threat information exchange infrastructure.
Establishing a trusted ecosystem for threat sharing will promote the validity,
security, anonymity, scalability, latency efficiency, and traceability of the
stored information that protects it from unauthorized disclosure. This paper
proposes a system that ensures the security principles mentioned above by
utilizing a distributed ledger technology that provides secure decentralized
operations through smart contracts and provides a privacy-preserving ecosystem
for threat information storage and sharing regarding the MITRE ATT&CK
framework.

###### Index Terms:

Threat Information Sharing, MITRE ATT&CK, Distributed Ledger Technology,
InterPlanetary File System, Hyperledger Fabric, Decentralised Identities,
Blockchain, Privacy-Preserving, Cyber Hunting.

##  I Introduction

Organisations can not afford to defend themselves isolated from the threat
landscape due to the emergence of new cyber threats and hence, threat
information sharing is considered an indispensable cybersecurity domain. An
organisation that has faced a specific range of cyberattacks can help other
organizations against adversary attacks of the same type. Threat information
sharing is an essential aspect of the cybersecurity domain regarding the
protection of organizations or individuals against adversary attacks. Threat
information sharing involves the processes of the collection, analysis and
sharing of cyber threat information among multiple organizations [1]. It
ranges from public to private sharing, such as threat intelligence sharing,
regionally or globally. Therefore, finding a secure and trusted way to share
threat information is crucial to ensure the privacy and reliability of the
participant parties. Regardless of the data type, purpose and role, whether
public or private, it should traverse through a secure and trusted
infrastructure, which could be challenging and costly [2]. Thus, providing a
privacy-preserving method to avoid risk is crucial to maintaining
confidentiality, availability, and integrity, such as preventing data loss or
damage, unauthorized disclosure, data unavailability, or unauthorized
alteration.

Traditional systems for information sharing are commonly centralized
networking systems that are controlled by third parties via the internet using
location-based addressing (URLs) to reach resources. As a result, data is
vulnerable to Denial-of-Service (DoS) attacks due to the possibility of a
single point of failure taking place, which is the main drawback of
centralized systems [3]. In contrast, decentralized peer-to-peer networks,
which is our solution’s core concept, represent a practical and efficient
approach for data storage and sharing to combat this issue. Notably, utilizing
the Hyperledger Fabric permissioned blockchain framework fortifies the system
from a single point of failure and DoS attacks due to its decentralized
nature. Additionally, threat information sharing is enabled in a distributed
manner, which is being transmitted securely through a reliable, scalable, and
performance-efficient infrastructure. Hyperledger Fabric’s private and
permissioned nature provides flexible identity systems, smart contracts, and
unique access control policies through private data [4]. It ensures that
private information stored in the ledger cannot be accessed by unauthorized
individuals and only by authorized participants using their authorized
identity certificates [5].

We analyze the overlapping between the threat information’s tactics and
techniques to demonstrate how to use the MITRE Adversarial Tactics,
Techniques, and Common Knowledge (ATT&CK) framework for threat hunting,
modelling the adversaries’ behaviour and incident response using detection
tools and data logs. One can see from Figure 1 depicts the MITRE ATT&CK
framework regarding the threat hunting and information sharing. Additionally,
the data logs are imported to the MITRE ATT&CK navigator to review and compare
the attacker’s behaviors. Furthermore, we present additional data sharing and
threat reporting scenarios to look at scaling into other threat reporting
areas. In this context, we analyze the use case that has been detected in the
healthcare Wicked Panda and Fox Kitten datasets.

Figure 1: MITRE ATT&CK Framework for Threat Hunting and Sharing.

Consequently, in this paper, we propose an integrated distributed
infrastructure that combines various privacy-preserving techniques that
addresses the security and privacy issues with higher efficiency. The
contributions of our work can be summarized as follows:

  * •

We analyse, review and compare a range of healthcare IT infrastructures, and
then we find the overlapping between tactics and techniques using MITRE ATT&CK
navigator.

  * •

We propose a trusted ecosystem for threat sharing based on Hyperledger Fabric,
Decentralised Identities and InterPlanetary File System, a distributed storage
method.

  * •

We empirically evaluate the proposed solution in the scope of collaborative
operation.

This paper is organized as follows: Section 2 presents the literature review
and background knowledge. The proposed methodology for threat sharing is
discussed in Section 3, which includes Data Sources, Data Analysis Framework,
Use Case, the Proposed Method and Collaborative Operations using IPFS,
Decentralised Identities DIDs and Hyperledger Fabric. Section 4 presents a
security evaluation, discusses potential challenges and issues that need to be
carefully considered, and then debates system anonymity consideration,
latency, scalability, and throughput. Finally, Section 5 concludes this work.
Few future directions are also outlined in the conclusion section.

##  II Literature Review and Background Knowledge

This section presents the literature review and background information.
Several approaches, tools and technologies are integrated into the proposed
framework, including MITRE ATT&CK IPFS system and Hyperledger Fabric to create
privacy-preserving method and trusted information sharing.

###  II-A Data Sources and Threat intelligence Sharing

A wide variety of file sharing methods exist, and they differ from one another
in that they allow users to control what, how, and with whom to share. Various
studies have been conducted to investigate these properties [6]. The
requirements for information sharing can range from the simple sharing of
documents to threat intelligence sharing. Threat intelligence is the process
of gathering, processing and disseminating information about threats and
attackers between trusted partners. Threat intelligence sharing aims to
contextualize the information and deliver actionable information that
contributes to the decision-making process.

###  II-B MITRE ATT&CK Framework

The MITRE ATT&CK framework is an open-source that represents a cumulative
knowledge-based on cyber attacker tactics and techniques spotted from previous
experiments of the adversary’s behavior [7]. As an essential knowledge base,
MITRE ATT&CK permits the cyber defense teams to review and contrast attacker
activity and then understand the best options for defense. It describes the
adversary behavior based on actual observations of cybersecurity incidents.
MITRE ATT&CK is used as a navigator to map adversary tactics, techniques, and
procedures (TTPS) to its knowledge base as shown in Figure 2 with their
description presented, as follows:

  * •

Tactics: the goals the attacker is trying to achieve (the column’s title)

  * •

Techniques: the different ways that cyber attackers can achieve the goal of
the tactic (the details under the column)

  * •

Sub-techniques: data that provides a more detailed explanation of the
attacker’s technique

  * •

Procedures: Examples of malware practices used by cyber attackers

Figure 2: Adversary’s Tactics and Techniques on MITRE ATT&CK Framework. [8].

The key notion of using the MITRE ATT&CK framework is to transform the
attacker’s strategies (tactics, techniques) included in the data logs (new
threats information) to IDs, scores and colours, respectively for easier
conceptualizing and understanding. The MITRE ATT&CK navigator is used to
create a new layer of the threats scenario as shown in Figures 5 and 6.
Display and review the existing layer by uploading data logs (JSON) to the
navigator, which will be opened in a layer that shows its threat information
included in the file, as shown in Figure 2. Integrate two layers of different
threats in one layer in order to analyse them and find the overlapping threats
(tactics and techniques) (green) between them (red and yellow), as shown in
Figure 7.

We summarise that the MITRE ATT&CK framework is used to navigate, review and
contrast the strategies of attackers, tactics, and techniques to achieve their
goals, and find suitable mitigation and countermeasures strategies. Concerning
the use case design, MITRE ATT& CK framework can be used to model the
adversary’s behaviour after being detected (data logs source). Then the file
is encrypted and uploaded to the IPFS, being stored there. Thus, we download
its CID (hash key), which will be sent through Hyperledger Fabric for trusted
sharing of threats between the authorized partners. For example, Figure 1
illustrates threat hunting and information sharing using the MITRE ATT&CK
framework.

###  II-C InterPlanetary File System

The InterPlanetary File System (IPFS) is an open-source, peer-to-peer network
that is created as a decentralized system with the remarkable ability to store
big data (scalability) and share files among distributed nodes globally,
without the need of a central administrator [9]. Therefore, it does not have a
single point of failure. In the IPFS, the uploaded file is split into blocks
and then distributed amongst the nodes, as shown in Figure 3 [10]. Each block
has a unique hash value representing the content identifier (CID) used to
reference the file contents that need to be shared. Contrary to this, the
traditional internet uses URL web addresses to share the objects based on the
location approach. Many vital features contributed to emerging IPFS as a
superior technique compared to others. These features result from the
successful thinking of previous peer-to-peer systems, for instance, DHT,
BitTorrent, Git, and SFS. In other words, IPFS is described as integrating
solutions of prominent techniques and protocols in one platform. IPFS contains
various techniques and protocols with different functionality, most prominent
the CID, the DHT, and the Merkle DAG, Libp2p and the BitSwap, which represent
essential pillars that IPFS is sitting on to attain the aim of file sharing
and exchanging between scattered nodes globally. Overall, we conclude that
IPFS combines various eminent technical concepts. Its mechanism involves
several phases in which protocols and techniques work in harmony to share data
between the IPFS network peers efficiently. IPFS achieves data storing,
transferring, requesting, and deleting through key-value data storage
management. On this basis, IPFS provides a high throughput, peers exchange,
and share files in a content-based approach rather than a location-based
approach, based on CID, Merkle DAG, DHT, Libp2p, and the BitSwap protocol that
inspired from the Bit Torrent protocol [11].

Figure 3: Interplanetary file system, store and retrieve file.

###  II-D Hyperledger Fabric

Hyperledger Fabric is a private-permissioned distributed ledger technology
designed to be highly modular and extensible, delivering confidentiality,
privacy and scalability as the de facto enterprise blockchain. Hyperledger
Fabric does not involve any cryptocurrency as opposed to public blockchains;
hence, it has a flexible design that gained a wide reputation in many
different application areas [12, 4]. It is provided as open-source by the
Linux Foundation, and the smart contracts, namely chaincode, written in common
programming languages such as Go, JavaScript, and Java, are being executed in
a distributed operation manner [13]. Hyperledger Fabric allows various
components, such as complex consensus mechanisms and identity services, to be
used as plug-and-play.

A smart contract is a piece of programming code deployed in a blockchain
environment that is being executed automatically if certain conditions are
being met [14]. Before accessing the blockchain ecosystem, all the
participants have to agree on it, and install it. Additionally, smart
contracts involve the security policies and the blockchain rules set by its
administrators [15, 16]. In Hyperledger Fabric, the smart contracts are called
chaincode, and are mutually accepted and installed in all the participating
entities and define their permissions, the actions they can perform, and the
access control policy to specific stored data [13].

Peers is the fundamental entity participating in a Hyperledger Fabric
ecosystem. The Peers can read from and write to the blockchain ledger
according to their authorized identity certificates and defined permissions.
Additionally, the Peers execute their installed chaincode to store their data
to State Databases such as LevelDB and CouchDB that store data in key-value
pairs. State databases are used to track the global state of the blockchain
ledger (the one that all the authorized participants have access to) and the
private data collections, which are stored to private blockchain ledgers
available only to authorized participants with the appropriate permissions. A
group of Peers forms an Organization, and multiple Organizations exist in a
typical Hyperledger Fabric ecosystem. Multiple Organizations (and their Peers)
can access specific blockchain ledgers that are available in their Channel.
Each Channel can have its own privacy rules and permissions; however, the
Organizations can join multiple Channels using the same identity certificates.
Within the Hyperledger Fabric community, it is considered a good practice to
utilize separate Channels if certain participating Organizations do not need
to “share” a blockchain ledger; however, in case that the Organizations may
share some data stored in a blockchain ledger but not others, then Private
Data Collections can be used, allowing access only to specific data fields.
Data can be stored in a blockchain ledger only after Consensus between the
Organizations is being met. This is one of the defined rules during the
creation of the blockchain that specifies the number of Organizations required
to approve a transaction to be considered valid. Hyperledger Fabric allows
complex consensus mechanisms to be utilized, such as the Byzantine Fault-
Tolerance. The Endorsement Policy in Hyperledger Fabric defines which Peers of
an Organization are required to execute the chaincode and approve or reject a
transaction. These transaction proposals are forwarded to the Ordering Service
which checks if the number of Organizations that approve a transaction
corresponds to the defined rules and, if successful, creates a new blockchain
block to be added to the blockchain ledger and broadcasts it to all the
associated Organizations and Peers in order to update their State Databases,
as shown in Figure 4 [12, 4, 13].

In Hyperledger Fabric, the identity and permissions of each participant are
defined through its identity certificates. By default, the Certificate
Authority (CA) creates X.509 digital certificates for all the participants
[13]. These types of credentials follow a similar approach as the Public Key
Infrastructure (PKI), in which the Hyperledger Fabric participants use their
private keys to sign their transactions. In case that an X.509 certificate is
being compromised, the CA can revoke its access to the blockchain ledgers and
issue a new credential for its corresponding legitimate participant. The
Membership Service Provider (MSP) verifies all the identity certificates for
the participants in the ecosystem, as shown in Figure 4. This authentication
and authorization process ensures that the stored data can be accessed only by
legitimate participants with the appropriate permissions and not by
unauthorized third parties. The usage of X.509 certificates can be further
extended in more complex MSP implementations, and other types of digital
identities can be used [5, 17].

Figure 4: Hyperledger Fabric operation phases

###  II-E Decentralised Identities Management (DIDs)

DIDs are a new type of digital identifier tool that provide systems
trustworthy and privacy-preserving. The traditional identities are described
as centralised management, which depends on the third party. Contrary, the
DIDs are a distributed, decentralised self-authenticated system. DIDs are
stored in a decentralised storage system such as IPFS and Distributed ledger
as in the proposed framework. Furthermore, DIDs are managed and controlled by
the organisation owner without third party need.

TABLE I: Comparison of the related literature with our work. Related Work | Distributed Ledger | Accessibility | Verifiabilty | Scalability | GDPR | Content Erasure (RtbF) | Decentralised Identites
---|---|---|---|---|---|---|---
[18] | Ethergram | Yes | Yes | Yes, IPFS | Yes | No | No
[19] | Blockchain Ledger | Yes | Yes | No | Yes | Yes | No
[20] | Public Blockchain | Yes | Yes | Yes, Blockchain | Yes | No | No, Proxy Encryption
[9] | Blockchain | Yes | Yes | Yes, IPFS | Yes | Yes | No
[21] | Hyperledger Fabric | Yes | Yes | Yes, IPFS | Yes | Yes | No
Our Work | Hyperledger Fabric | Yes | Yes | Yes, IPFS | Yes | Yes | Yes

###  II-F Related Works

In the work of [18], the authors proposed the Ethegram framework to solve
challenges of typical social media that derive from the centralization of
systems. In contrast, the authors used the Ethereum blockchain and IPFS
(InterPlanetary File System) peer-to-peer file system to distribute the stored
content and address single point of failure issues. Additionally, the proposed
decentralized system aims to solve censorship issues by using distributed
peers without the need for a trusted intermediary. However, a challenge for
the adoption of this architecture originates from the chosen blockchain since
it involves a cryptocurrency; hence the transactions may be costly.

In the work of [19], the authors introduced the LUCE framework, which is a new
method for data management, taking into account data accountability and their
license terms. This framework utilizes a blockchain ledger to incentivize data
sharing and reuse by facilitating compliance with licensing terms such as
recording the data usage and its purpose. The essential purpose of creating
the LUCE framework is to allow the individual data to be modified and deleted,
complying with regulations such as the GDPR’s rights to access, modification
and deletion. This work is comparable with our proposed infrastructure in
complying with the GDPR and licensing terms and incentivizing data sharing,
but it does not consider the IPFS to distribute the data storage to address
the scalability challenges.

In the work of [20], the authors proposed a novel infrastructure to solve data
sharing issues concerning security, transparency, and traceability. The
proposed solution suggests the use of two blockchains together. One blockchain
is being used for the original data’s storage and the other for the storage of
the transactions. This infrastructure aims to share data securely and provide
features such as linkability and traceability. In addition, this
infrastructure is integrated with another proxy encryption technology to
enhance the privacy and security of the system more. This article proposed
dual-blockchain usage to solve the issues mentioned earlier; however, simply
that is not sufficient to ultimately ensure the privacy of the stored data,
especially since this system was developed with the scope to promote
traceability of the stored data.

In the work of [9], the authors’ combined blockchain technology and IPFS. They
combined blockchain with the IPFS data sharing system to offer a broader data
scale and store personal data off-chain. In other words, the original data is
shared and distributed in the IPFS nodes rather than blockchain to avoid data
censorship and comply with the Right to be Forgotten (RtbF) of the GDPR. The
European Union (GDPR) established the GDPR to enforce personal data protection
within the EU, and all the associated entities in the same context [21]. Their
proposed framework uses an anonymous protocol that aligns with the IPFS’
principle to remove any Personally Identifiable Information (PII) from the
data distributed over the IPFS nodes and ensures that an erasure request
reaches all nodes using the delegated deletion mechanism. Besides that, only
the owner or their delegates can erase the original content. The authors state
that this mechanism for data erasure does not affect the system’s overall
efficiency; hence, it provides robust security with adequate performance for
generating the content-dependent keys.

Table I shows how our work differentiates from the rest of previous work in
the same context. It proposes a combination of a private-permissioned
blockchain framework with IPFS to store threat information and share it
amongst distributed peers. This combination ensures the stored content’s
privacy, whereas it provides high performance in terms of network latency,
transaction throughput, CPU and ram usage. Besides that, we use a
decentralised identities management that ensures distribution identity in a
secure, trusted and available manner.

MITRE ATT&CK navigator is compatible with our proposed infrastructure. In
which threat information consists of the tactics, techniques, and mitigation
in the forms of IDs in the JSON file. It is presented in different colours and
scoring. It will show the threat behaviour nicely, implicitly, simplistic,
clear and comparable. The depictive of the threat scenario could be extracted
in the JSON file format to be shared with the end-user. The MITRE ATT&CK
navigator represents the trusted partners’ user interface, whereby threat
sharing could be reviewed and contrasted to find the mitigation and best
defence points.

The partner will be identified by his node identity to download the sharing
file on the local machine and then easy to upload, review, and contrast the
threat dataset on the MITRE ATT&CK navigator.

##  III Methodology and Technical Architecture

The proposed system consists of Hyperledger Fabric and Interplanetary File
System (IPFS). This Infrastructure depicts the exact threat sharing mechanism
using the MITRE ATT&CK framework. Chaincode in the system are kept distributed
because they are responsible for carrying out different organisations’
policies and rules securely and privately.

###  III-A Data Sources

Data sources represent the cumulative and real-life data associated with the
case study that we need to analyse to find solutions for current and potential
threats. Data sources consist of primary and secondary data. Primary data is
collected from the original sources, reliable and contributes directly to the
critical decision. The secondary data is collected from the literature review
( articles, journals, books, conferences, etc). Furthermore, data obtained
from the expert was the most beneficial regarding the different experiments
and comparisons in practical and theoretical aspects, avoided many
complexities and efforts and saved searching time for valuable data.

Our measurements setup depends on a MacBook Pro with 1TB SSD drive, Memory 16
GB 1600 MHz DDR3, Processor 2.6 GHz Quad-Core Intel Core i7. In addition, we
utilised the activity monitor application to measure the storage usage and
processor utilisation. Our evaluation observed that our approach consumes low
energy and CPU power.

###  III-B Data Analysis Framework

  * •

MITRE ATT&CK navigator

Most threat modelling and attack simulation works are done manually, which can
be time-consuming and prone to errors. MITRE ATT&CK is a globally accessible
knowledge base of adversary tactics and techniques based on real-world
observations[22]. This knowledge base can be used as a foundation for
developing specific threat models and other types of methodologies and tools.
Our focus is on its navigator, the Detect Tactics, Techniques & Combat Threats
(DeTT&CT) Editor.

The data logs (JSON file) will be imported to the MITRE ATT&CK navigator to
analyse and determine suitable defensive coverage. In contrast, we can export
and share files representing the threats group of specific incidents to be
viewed on the navigator back. In other words, the transformation process from
the data logs and navigators is bidirectional. Therefore, this is considered
beneficial concerning the data sharing and displaying process and getting a
trusted and secure way between the scatted partners.

MITRE ATT&CK navigator has other remarkable features that ease the security
analysts, such as colouring comparing, scoring and combining different groups
of the threat actions in a clear view [23].

  * •

DeTT&CT Editor In April 2019, Ruben and Marcus Bakker released the first
version of DeTT&CT. It was created at the Cyber Defence Centre of Rabobank and
built on top of the MITRE ATT&CK [24].

The DeTT&CT Editor has been created to assist blue teams using ATT&CK to score
and compare data log source quality, visibility coverage, detection coverage
and threat actor behaviours. All of which can help, in different ways, to get
more resilient against attacks targeting organisations.

###  III-C Use Case: Threats Information Sharing

The use cases were obtained from the MITRE ATT&CK framework related to the
previous real-world threats experiments of industry and manufacturing fields
subjected to cyber security threats and interested in threat intelligence
sharing. We will describe and analyse some cyberattack incidents identified in
healthcare, such as WICKED PANDA and Fox Kitten.

The WICKED PANDA threat group is identified by ”ID: G0096” and associated
Groups called WICKED PANDA, an espionage group that conducts financially
motivated operations. The group has been observed targeting healthcare,
telecom, technology, and video game industries in 14 countries. We will take
this case study to demonstrate how to use the ATT&CK technique methodology to
determine when and how to develop new techniques for ATT&CK for ICS [25].

Figure 5: WICKED PANDA Threat, Attacker Tactics and Techniques

Fox Kitten threat group targets multiple industries, including oil and gas,
technology, government, defence, healthcare, manufacturing, and engineering.
It is associated with areas and companies such as UNC757, PIONEER KITTEN,
Parisite, the Middle East, North Africa, Europe, Australia, and North America
[26].

Figure 6: Fox Kitten Threat, Attacker Tactics and Techniques Figure 7:
Overlapping between Wicked Panda layer and Fox Kitten layer illustrated by the
green colour

###  III-D The Proposed Method and Collaborative Operations using IPFS and
Hyperledger Fabric

Our proposed ecosystem provides mechanisms to share sensitive data and private
threat information across broad IT and public networks in a secure and trusted
way. We have chosen an eminent use case of real-world observations of the
attacker’s behaviours and systems breach in the healthcare domain. Threats
information obtained in JSON files, viewed and contrasted using MITRE ATT&CK
navigation as shown in Figures 2, 5, 6, and 7.

The function of each operation number and flow of the system is illustrated in
Figure 1, and explained, as follows:

  1. 1.

New threat is stored in a local database system.

  2. 2.

Threat files of WICKED PANDA and Fox Kitten are uploaded to ATT&CK navigator
to analyse and compare the adversary behaviour and find suitable
countermeasures.

  3. 3.

The MSP (DIDs) provides nodes identities and certificates to authenticate the
interacted parties. The sender encrypts and signs the threat file (JSON file).

  4. 4.

Sender uploads the encrypted file to IPFS.

  5. 5.

Sender downloads the hash file (CID) from IPFS.

  6. 6.

Sender submits the transaction (CID) using its own identity. Sender is
authenticated by chaincode, endorsement.

  7. 7.

The trusted partner is authenticated by chaincode and then get the CID using
his private key.

  8. 8.

A trusted partner will use CID to search and get access to the threat file
into IPFS storage.

  9. 9.

A trusted partner will download the threat desired file from IPFS using the
CID.

  10. 10.

The trusted partner retrieves threat file from IPFS, decrypt it and then
display the WICKED PANDA and Fox Kitten threats on the MITRE ATT&CK
navigation.

Figure 8: Threat information flow in proposed infrastructure

As a result, the proposed ecosystem can share and store threat files in a
secure and trusted way. The combination of Hyperledger Fabric and
InterPlanetary File System (IPFS) works together highly efficiently.
Hyperledger Fabric, InterPlanetary File System (IPFS), DIDs and MITRE ATT&CK
framework guarantees privacy-preserving and threat sharing, securely plus
scalability and anonymity.

##  IV Empirical Evaluation

The key objective of our paper is to implement a data-sharing infrastructure
within a permissioned ledger, which creates an ecosystem for the trusted
sharing of threat information. To do so, we consider the system’s security and
anonymity, latency, and scalability.

###  IV-A Security Consideration

From a security point of view, our proposed infrastructure is a decentralised
network that is resistant to denial of service attacks since it does not have
a single administrator service, and its data content is distributed among the
participating nodes. Besides that, Hyperledger Fabric supports transaction
encryption and chaincode that governs the interaction between the trusted
users, ensuring transaction authentication and data confidentiality. Any user
should pass an identity check; otherwise, their transaction will be declined.

Furthermore, we use the MSP entity, which is considered the key element that
offers the system authentication, as a part of the Hyperledger Fabric that
contains certificate authority and Identities management. This entity supplies
all the nodes and channels with unique identities and enables transactions to
be private and secure when traversing the network between the interacting
users. By default, the system is protected against unauthorised users. A
potential vulnerability to our proposed system could be the human factor since
their credentials may be compromised intentionally or negligently.

In light of the above, the certificate authority is the third entity that
supports the interacting parties to trust each other using its signature as
the proof of identity. If any interacting parties are compromised, they can
ask the certificate authority to revoke their transactions. As a result, any
adversary targeting the Hyperledger Fabric by establishing read or write
queries will be exposed since it does not own any system identities. The only
way is to compromise an authorised user’s identity. Another potential threat
is that adversaries may target the blockchain ledgers since they are
disseminated across the participating nodes, and the legitimate users’ access
to them may be blocked.

###  IV-B Anonymity consideration

The Hyperledger Fabric supports transaction anonymity by default since it
includes the MSP entity that is responsible for identity management. On top of
that, this unit is pluggable, meaning that each organization could create its
own MSP.

Data transaction anonymity derives from the transaction data encryption, by
the client’s private key and signed by the trusted third party (identity
management), and the Transport Layer Security cryptographic protocol (TLS)
that is used to encrypt the communication between the interacting
participants. The authorized user will be identified in the chaincode and
should gather enough of the endorsements to be able to send the transaction
for validation, which is governed by the endorsers’ policy without disclosing
or revealing the transaction data.

In the work of [27], the authors propose an Anonymous Endorsement System which
uses a threshold endorsement policy. Several factors contribute to the design
of a new ring signature scheme, called Fabric’ Constant-Sized Linkable Ring
Signature (FCsLRS) with Transaction-Oriented linkability for hiding the
identity of the endorsers. Additionally, this work implemented the signature
scheme in Golang and analyzed its security and performance by varying the RSA
(Rivest-Shamir-Adleman) modulus size. The feasibility of the implementation is
supported by empirical analysis. The signature and tag generation time is
relatively fast. Moreover, it remains constant irrespective of change in
message length or endorsement set size for a given RSA modulus value, assuming
all the endorsers generate their signatures in parallel.

###  IV-C Latency, Scalability, Throughput

Since Hyperledger Fabric is a permissioned-private blockchain framework, the
read and write queries’ latency is comparable to common centralized systems,
such as typical databases (PostgreSQL), instead of public blockchain systems
such as Bitcoin. However, it should be noted that in a higher number of
transaction records, our chosen infrastructure performs more efficiently
contrary to typical local databases, with the transaction times being reduced
[12, 4].

In terms of scalability, since the Hyperledger Fabric is utilizing Docker
containers to act as nodes of the blockchain, it is easily scalable to cloud
infrastructures such as Kubernetes clusters. This adds immense extensibility
to the given use cases by incorporating several Peers and Organizations to
create complex scenarios. Additionally, in our proposed ecosystem, the actual
records can be stored distributed in IPFS; hence, extending the scenario even
more.

The transaction throughput of our system depends on the ordering service and
is capped by the network and ordering nodes capacity size. However, simply by
adding a higher number of ordering nodes, the transaction throughput can be
increased. Examples of adding more nodes include RAFT and KAFKA ordering
clusters [12].

##  V  Conclusion and Future Work

Threat hunting is fundamentally a proactive countermeasure for detecting and
protecting IT systems against malicious behaviour by monitoring current and
historical information related to security breaches. Therefore, it is
necessary to come up with a trusted and secure framework for threat
information sharing. The deployment of trusted computing technologies for
adversary detection and privacy protection can help achieve this goal using
data logs generated by systems, network devices or security applications such
as intrusion detection/prevention systems. Our work proposed a trusted
information sharing solution utilizing both the IPFS system and a permissioned
ledger that ensures the security, privacy, and anonymity of the stored
information that achieves high throughput with immense scalability. The
permisioned distributed ledger technology of our choice is the Hyperledger
Fabric, which meets all the aforementioned expectations and achieves our
work’s aims.

Our proposed infrastructure implements threat information sharing effectively,
using the MITRE ATT&CK Framework, the pluggable certificate authorities, and
the self-executing chaincode when a set of defined conditions is being met,
thus enabling the establishment of trust between the interacting parties and
enhancing the total security of the system. Hence, our ecosystem ensures the
anonymity of the stored information. As future work, we aim to build a
comprehensive proof-of-concept in a cloud infrastructure, utilizing a
Kubernetes cluster to improve the system’s throughput and scalability
considering the expansion of the participating nodes and the transactions’
capacity within the GLASS project, which represents the ongoing European
Union-funded project in this context.

## Acknowledgements

The research leading to these results has been partially funded by the
European Union’s Horizon 2020 research and innovation programme, through
funding of the GLASS project (under grant agreement No 959879).

## References

  * [1] E. Bandara, X. Liang, P. Foytik, and S. Shetty, “Blockchain and self-sovereign identity empowered cyber threat information sharing platform,” in _2021 IEEE International Conference on Smart Computing (SMARTCOMP)_. IEEE, 2021, pp. 258–263.
  * [2] V. Jaiman, L. Pernice, and V. Urovi, “User incentives for blockchain-based data sharing platforms,” _arXiv preprint arXiv:2110.11348_ , 2021.
  * [3] Z. Zheng, S. Xie, H.-N. Dai, W. Chen, X. Chen, J. Weng, and M. Imran, “An overview on smart contracts: Challenges, advances and platforms,” _Future Generation Computer Systems_ , vol. 105, pp. 475–491, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0167739X19316280
  * [4] C. Stamatellis, P. Papadopoulos, N. Pitropakis, S. Katsikas, and W. J. Buchanan, “A privacy-preserving healthcare framework using hyperledger fabric,” _Sensors_ , vol. 20, no. 22, p. 6587, 2020.
  * [5] A. Iftekhar, X. Cui, Q. Tao, and C. Zheng, “Hyperledger fabric access control system for internet of things layer in blockchain-based applications,” _Entropy_ , vol. 23, no. 8, p. 1054, 2021.
  * [6] R. Alsowail and I. Mackie, “Insider threats for file sharing: Characterising file sharing and its protection,” 2014.
  * [7] A. Georgiadou, S. Mouzakitis, and D. Askounis, “Assessing mitre att&ck risk using a cyber-security culture framework,” _Sensors_ , vol. 21, no. 9, p. 3267, 2021.
  * [8] M. Bakker and R. Bouman, “Attack windows,” https://www.mbsecure.nl/blog/2019/5/dettact-mapping-your-blue-team-to-mitre-attack, May 2019.
  * [9] E. Politou, E. Alepis, C. Patsakis, F. Casino, and M. Alazab, “Delegated content erasure in ipfs,” _Future Generation Computer Systems_ , vol. 112, pp. 956–964, 2020.
  * [10] Protocol Labs, “go-ipfs/add.go at master · ipfs/go-ipfs,” 2021. [Online]. Available: https://github.com/ipfs/go-ipfs/blob/master/core/commands/add.go
  * [11] S. Henningsen, M. Florian, S. Rust, and B. Scheuermann, “Mapping the interplanetary filesystem,” in _2020 IFIP Networking Conference (Networking)_. IEEE, 2020, pp. 289–297.
  * [12] P. Papadopoulos, N. Pitropakis, W. J. Buchanan, O. Lo, and S. Katsikas, “Privacy-preserving passive dns,” _Computers_ , vol. 9, no. 3, p. 64, 2020\.
  * [13] E. Androulaki, A. Barger, V. Bortnikov, C. Cachin, K. Christidis, A. De Caro, D. Enyeart, C. Ferris, G. Laventman, Y. Manevich _et al._ , “Hyperledger fabric: a distributed operating system for permissioned blockchains,” in _Proceedings of the thirteenth EuroSys conference_ , 2018, pp. 1–15.
  * [14] P. De Filippi, C. Wray, and G. Sileno, “Smart contracts,” _Internet Policy Review_ , vol. 10, no. 2, 2021.
  * [15] S. Wang, Y. Zhang, and Y. Zhang, “A blockchain-based framework for data sharing with fine-grained access control in decentralized storage systems,” _Ieee Access_ , vol. 6, pp. 38 437–38 450, 2018.
  * [16] G. Wood, “Ethereum: A secure decentralised generalised transaction ledger byzantium version,” _Ethereum Project Yellow Paper_ , 2021.
  * [17] P. Papadopoulos, W. Abramson, A. J. Hall, N. Pitropakis, and W. J. Buchanan, “Privacy and trust redefined in federated machine learning,” _Machine Learning and Knowledge Extraction_ , vol. 3, no. 2, pp. 333–356, 2021.
  * [18] H. Bawane, T. Shinde, A. Kadam, Y. Budukh, and P. Mundhe, “Ethegram-an ethereum and ipfs-based decentralized social network system,” 2020.
  * [19] A. Havelange, M. Dumontier, B. Wouters, J. Linde, D. Townend, A. Riedl, and V. Urovi, “Luce: A blockchain solution for monitoring data license accountability and compliance,” _arXiv preprint arXiv:1908.02287_ , 2019\.
  * [20] Z. Wang, Y. Tian, and J. Zhu, “Data sharing and tracing scheme based on blockchain,” in _2018 8th international conference on logistics, Informatics and Service Sciences (LISS)_. IEEE, 2018, pp. 1–6.
  * [21] C. Grundstrom, K. Väyrynen, N. Iivari, and M. Isomursu, “Making sense of the general data protection regulation—four categories of personal data access challenges,” in _Proceedings of the 52nd Hawaii international conference on system sciences_ , 2019.
  * [22] H. Holm, M. Buschle, R. Lagerström, and M. Ekstedt, “Automatic data collection for enterprise architecture models,” _Software & Systems Modeling_, vol. 13, no. 2, pp. 825–841, 2014.
  * [23] R. trademarks of The MITRE Corporation, “Enterprise matrix,” https://attack.mitre.org/matrices/enterprise/, Oct. 2021.
  * [24] M. Bakker and R. Bouman, “Dett&ct editor,” https://rabobank-cdc.github.io/dettect-editor/#/home, April 2019.
  * [25] K. Kyaw Pyiyt Htet, “Wicked panda,” https://attack.mitre.org/groups/G0096/, Oct. 2021.
  * [26] C. C. Security, “Fox kitten,” https://www.clearskysec.com/fox-kitten/, April 2021.
  * [27] S. Mazumdar and S. Ruj, “Design of anonymous endorsement system in hyperledger fabric,” _IEEE Transactions on Emerging Topics in Computing_ , 2019.