# DSNet: Automatic Dermoscopic Skin Lesion Segmentation

Md. Kamrul Hasan  Computer Vision and Robotics Institute, University of
Girona, Spain 11footnotemark: 1<EMAIL_ADDRESS>Lavsen Dahal
<EMAIL_ADDRESS>Prasad N. Samarakoon<EMAIL_ADDRESS>Université Clermont Auvergne, Clermont-Ferrand, France 22footnotemark: 2
Fakrul Islam Tushar<EMAIL_ADDRESS>Robert Martí
<EMAIL_ADDRESS>
###### Abstract

### Background and Objective

Automatic segmentation of skin lesions is considered a crucial step in
Computer Aided Diagnosis (CAD) for melanoma. Despite its significance, skin
lesion segmentation remains an unsolved challenge due to their variability in
color, texture, and shapes and indistinguishable boundaries.

### Methods

Through this study, we present a new and automatic semantic segmentation
network for robust skin lesion segmentation named Dermoscopic Skin Network
(DSNet). In order to reduce the number of parameters to make the network
lightweight, we used a depth-wise separable convolution in lieu of standard
convolution to project the learnt discriminating features onto the pixel space
at different stages of the encoder. Additionally, we implemented both a U-Net
and a Fully Convolutional Network (FCN8s) to compare against the proposed
DSNet.

### Results

We evaluate our proposed model on two publicly available datasets, namely
ISIC-2017333https://challenge.kitware.com/#challenge/583f126bcad3a51cc66c8d9a
and PH2444https://www.fc.up.pt/addi/ph2%20database.html. The obtained mean
Intersection over Union (mIoU) is $77.5\,\%$ and $87.0\,\%$ respectively for
ISIC-2017 and PH2 datasets which outperformed the ISIC-2017 challenge winner
by $1.0\,\%$ with respect to mIoU. Our proposed network also outperformed
U-Net and FCN8s respectively by $3.6\,\%$ and $6.8\,\%$ with respect to mIoU
on the ISIC-2017 dataset.

### Conclusion

Our network for skin lesion segmentation outperforms the other methods
discussed in the article and is able to provide better segmented masks on two
different test datasets which can lead to better performance in melanoma
detection. Our trained model along with the source code and predicted masks
are made publicly available555https://github.com/kamruleee51/Skin-Lesion-
Segmentation-Using-Proposed-DSNet .

###### keywords:

Skin lesion segmentation , Computer Aided Diagnosis (CAD) , Melanoma detection
, Deep Learning.

††journal: Computers in Biology and Medicine

##  1 Introduction

One in every three cancers is a skin cancer worldwide (Ge et al., 2017). It is
estimated to have 96,480 new cases and 7,230 deaths from melanoma in 2019 in
the United States alone (American Cancer Society, 2019). Melanomas constitute
less than $5\,\%$ of all skin cancers, however, account for around $75\,\%$ of
all skin-cancer-related deaths in the United States alone (Esteva et al.,
2017). Early diagnosis is very important in the case of skin cancer as shown
by the study where the survival rate was as high as $90\,\%$ for melanoma with
early detection (Ge et al., 2017). Multiple non-invasive imaging techniques
such as dermoscopy, photography, Confocal Scanning Laser Microscopy (CSLM),
Optical Coherence Tomography (OCT), ultrasound imaging, Magnetic Resonance
Imaging (MRI), and spectroscopic imaging are currently being used to assist
dermatologists in skin lesion diagnosis (Smith and MacNeil, 2011).
Traditionally, the images output by the above mentioned techniques are
visually inspected by dermatologists to diagnose skin cancer (Haenssle et al.,
2018), which is often considered a complex and tedious task. To assist the
dermatologist and to improve the accuracy, Computer Aided Diagnosis (CAD)
systems have been developed. Nowadays, CAD has become an essential part of the
routine clinical work for abnormality detection in medical images at many
screening sites and hospitals (Doi, 2007). CAD systems generally consist of
multiple components such as raw image acquisition, preprocessing, Region of
Interest (ROI) segmentation, responsible feature extraction, and disease
classification (Jalalian et al., 2017; Fan et al., 2017). Segmentation is a
vital component of skin cancer diagnosis as the features for the
classification are obtained from the ROI of a segmented mask (Korotkov and
Garcia, 2012). However, automatic and robust skin lesion segmentation is a
challenging task due to patient specific properties such as skin color,
texture, size of lesion area, and the presence of numerous artifacts such as
body hair, reflections, air bubbles, rolling lines, shadows, color calibration
charts, non-uniform lighting, non-uniform vignetting, markers, and ink
(Mishraa and Celebi, 2016) as shown in Fig. 1.

Figure 1: A typical pictorial presentation of the skin images in ISIC-2017
test dataset (Codella et al., 2018) with different challenging images for
segmentation. The obtained segmented masks of these challenging images using
our proposed DSNet are provided in A.

In the early days, a mixture of edge-detection, thresholding, active contours
(expectation-maximization, level sets, clustering, adaptive snakes, etc.) or
region based (region growing, iterative stochastic region merging, etc.)
techniques were used for skin lesion segmentation (Celebi et al., 2009;
Oliveira et al., 2016). Celebi et al. (2008) extracted skin lesion borders by
fusing ensembles of several thresholding methods which can usually achieve
accurate and robust results for simple textures and when the contrast between
the lesion (foreground) and background is large. Thresholding based methods
often fail to obtain a precise segmentation due to the presence of different
types of artifacts as mentioned above. A modified region-based active contour
detection method was proposed by Abbas et al. (2011) for multiple lesion
segmentation whereas Zhou et al. (2013) proposed a mean shift based gradient
vector flow algorithm to locate the correct borders. However, when the skin
lesion border is fuzzy, the above mentioned methods provide only a coarse
segmentation. Fan et al. (2017) used saliency based image enhancement followed
by Otsu thresholding. Nevertheless, selecting a proper threshold is often
unattainable due to noise and intensity variability of the skin images. An
automatic segmentation algorithm using color space analysis and clustering-
based histogram thresholding was proposed by Garnavi et al. (2010) to
determine the optimal color channel for segmentation of skin lesions. This
method performed better only when high resolution images were used in addition
to requiring preprocessing steps such as a hair removal algorithm. Melli et
al. (2006) used median cut, k-means, fuzzy-c means, and mean shift for
automatic border extraction of skin lesions. These computer vision algorithms
depend highly on parameter tuning, and finding a robust set of parameters that
leads to accurate segmentation is often difficult. Recently, semantic
segmentation using Convolutional Neural Networks (CNN) has become popular as
they lead to robust solutions for object segmentation by pixel wise
classification (Long et al., 2015; Ronneberger et al., 2015; Ajmal et al.,
2018; Guo et al., 2018). Bi et al. (2017b) presented a multi-stage Fully
Convolutional Network (mFCN) approach for accurate skin lesion segmentation
where they employed a parallel integration method to combine the outputs of
every stage. The first stage consisted of building a probability map using the
original image. Then, both the original image and the previously built
probability map were fed into the remaining (m-1)FCN networks. The results
produced at different stages of the network were complementary to each other
where the early stages were responsible in producing gross skin lesion masks
while the later stages were responsible in generating finer lesion boundaries.
Tang et al. (2018) developed a multi-stage UNet (MS-UNet) with deeply
supervised learning strategy where they injected multiple U-Nets into the
auto-context scheme to segment skin lesions. A full resolution convolutional
network (FrCN) was proposed by Al-masni et al. (2018) where they removed all
the sub-sampling layers of the encoder to get the full resolution of the input
image without losing any spatial information. However, without sub-sampling,
the CNN models often suffer from over-fitting due to redundant features in
addition to having a narrower field of view of the feature maps (Long et al.,
2015). Sarker et al. (2018) introduced SLSDeep, combining skip-connections,
dilated residuals, and pyramid pooling networks. In their model, the encoder
network depends on dilated residual network layers along with pyramid pooling
network with the enhanced ability to learn features from the input dermoscopic
images. The loss function they optimized comprised a Negative Log Likelihood
(NLL) term and an End Point Error (EPE) term. Salimi et al. (2018) proposed
DermoNet having an encoder-decoder paradigm where the encoder consisted of
multiple dense blocks while the decoder was used to recover the full image
resolution. To process high-resolution features from early layers as well as
high-semantic features of deeper layers, they linked the output of each dense
block with its corresponding decoder. Yuan (2017) presented a deep fully
convolutional-deconvolutional neural network where the learning rate was
adjusted based on the first and the second-order moments of the gradient at
each iteration. In their method, the authors included additional channels such
as Hue-Saturation-Value from the HSV color space and the L channel (lightness)
from CIELAB space along with the original RGB channels. Moreover, the authors
used bagging-type ensemble of six networks to obtain the final lesion mask.

In this article, we propose Dermoscopic Skin Network (DSNet), a semantic
segmentation network for automatic and robust dermoscopic skin lesion
segmentation. To eliminate the necessity of learning the redundant features in
the encoder of the proposed network, we use dense blocks and transition blocks
which were inspired by DenseNet (Huang et al., 2017). In order to reduce the
number of parameters that results in a more generic and lightweight network,
depth-wise separable convolution has been used in the decoder similar to the
studies of Chollet (2017) and Kaiser et al. (2017). Skip connections
(Ronneberger et al., 2015) have also been used to learn back the spatial
information lost due to pooling at each stage of the encoder. To highlight the
importance of the overlapping between the true and predicted skin lesion
masks, we propose sum of intersection over union and cross entropy as loss
functions. The novelty of this paper lies in the methodology where the
proposed DSNet has distinct learning capabilities over the existing CNN
networks for the skin lesion segmentation while being very light-weight.
Transfer learning of CNN by leveraging the knowledge from previously trained
model was employed for training the proposed DSNet. Our method was able to
achieve state-of-the-art results on both the IEEE International Symposium on
Biomedical Imaging (ISBI) and PH2 datasets. To the best of our knowledge, the
proposed DSNet is the most lightweight network that produces state-of-the-art
results. We also compare the performance of our DSNet against a number of
well-known deep learning approaches (e.g. FCN, U-Net, etc.) under the same
conditions using the same datasets.

The remaining sections of the article are organized as follows: section 2
describes the architecture of the proposed network and methodologies whereas
section 3 describes the experiments and the results obtained along with an
interpretation of the results. Finally, section 4 concludes the article.

##  2 Methods and Material

###  2.1 Proposed DSNet Architecture

In general, CNN for semantic segmentation consists of two essential
components: the encoder and the decoder (Badrinarayanan et al., 2017). The
encoder is composed of convolution and sub-sampling layers and is responsible
for the automatic feature extraction (Lin et al., 2014). The convolution
layers are used to produce the feature maps whereas the sub-sampling layers
are used to achieve spatial invariance by reducing the resolution of these
generated maps. This reduction in resolution leads to an expansion of the
field of view of the feature map which in turn makes the extraction of more
abstract salient features possible in addition to minimizing the computational
cost (Long et al., 2015). The decoder semantically projects the discriminating
lower resolution features learnt by the encoder onto the pixel space of higher
resolution to obtain a dense pixel wise classification (Garcia-Garcia et al.,
2018). However, the significantly reduced feature maps due to sub-sampling,
often suffer from spatial resolution loss which introduces coarseness, less
edge information, checkerboard artifacts, and over-segmentation in
semantically segmented masks (Long et al., 2015; Ronneberger et al., 2015;
Odena et al., 2016). To overcome these problems, Ronneberger et al. (2015)
introduced skip connections in a U-Net which allowed the decoder to recover
the relevant features learnt at each stage of the encoder that were lost due
to pooling. Similarly, Long et al. (2015) fused features at different
coarseness levels of the encoder in FCN to refine the segmentation. However,
when the deconvolution kernel size is not divisible by the up-scaling factor,
a deconvolution overlap occurs as the number of low resolution features that
contributs to a single high resolution feature is not constant across the high
resolution feature map (Odena et al., 2016). Due to this deconvolution
overlap, checkerboard artifacts may appear in the segmented mask. In that
sense, Al-masni et al. (2018) proposed Full resolution Convolution Network
(FrCN) which does not have any sub-sampling layers in the encoder to preserve
the spatial information of the feature maps for precise segmentation. However,
sub-sampling of feature maps is highly desirable to be employed in CNN due to
the several positive aspects previously mentioned. All the semantic
segmentation networks have similar encoder design but they vary mainly in
their decoder mechanism with respect to how the discriminating features are
projected onto the pixel space.

In the proposed DSNet, the encoder has $121$ layers which mimics the DenseNet
(Huang et al., 2017) architecture to eliminate the necessity of learning the
redundant features. An encoder such as the one proposed in DSNet has the
ability to learn the abstract features of lesions from skin images, reduce the
vanishing-gradient problem occurrences, and strengthen feature propagation.
The design of the DSNet enables each encoding layer to have direct access to
the gradients of the loss functions of all previous encoding layers and also
to the original input image using skip connections as shown in Fig. 2. Each
$n^{th}$ layer of a dense block receives the feature maps of all preceding
layers which can be formulated (Huang et al., 2017) as Eq. 1.

$X_{n}=H_{n}([X_{0},X_{1},X_{2},...,X_{n-1}])$ (1)

where $[X_{0},X_{1},X_{2},...,X_{n-1}]$ are the concatenation of the feature
maps produced in all previous layers and $H_{n}$ is the composite function
(Huang et al., 2017) of the $n^{th}$ layer.

Figure 2: A typical example of a dense block of DenseNet with 4 layers and a
growth rate of 3 where each feature map $H$ concatenates all the feature maps
of its preceding layers of the block for feature reuse throughout the network.
Consequently, this makes the encoder of DSNet more compact (lightweight).

The encoder of the proposed DSNet consisting of feature layers, dense blocks,
and transition blocks. A block representation of the encoder is presented in
Fig. 3. A complete depiction of all layers and the connectivities (among
convolution, batch normalization, activation, deconvolution layers, etc.)
among them are provided on the GitHub link5 as it is impractical to do in the
article due to the space constraints. In a dense block, a dense layer receives
the outputs of all its previous layers (see Fig. 2) where the output depth of
the $L^{th}$ layer $(D_{L})$ is (L-1) $\times$ growth rate $+D_{L-1}$. The
growth rate is the hyperparameter that regulates the amount of information
being added to the layer of the network. A transition layer exists between
every two adjacent dense layers which is responsible for reducing the
computational complexity. In the transition layer the $1\times 1$
convolutional layer (bottleneck layer) ensures that the second convolutional
layer always has a fixed input depth (Huang et al., 2017) and the $2\times 2$
average pooling layer with a stride of $2$ reduces the size of the feature map
by sub-sampling. We have also used skip connections having ladder like
structures (Rasmus et al., 2015) inspired by U-Net to overcome the sub-
sampling limitations and deconvolution overlap. Each pooled layer of the DSNet
is channel-wise concatenated to a deconvoluted feature map having the same
dimensions as shown in Fig. 4 where it acts as a compensatory connection for
the spatial information lost due to sub-sampling.

Figure 3: The encoder of the DSNet generates spatially invariant feature maps.
In a dense block, the feature map resolution remains the same while the number
of filters differ whereas in a transition block, the redundant features are
truncated.The feature maps named as Conv1, Pool1, Conv2-Pool2, and Conv3-Pool3
will be used for skip connection in deocoder of DSNet. (a)  Figure 4: The
decoder of the proposed DSNet to reconstruct a high resolution image of the
predicted lesion mask from a low resolution image.

Moreover, the internal co-variate shift due to the change of parameters during
the training has been addressed in the proposed DSNet by adding batch
normalization (Ioffe and Szegedyet, 2015) in both the encoder and decoder. The
use of depth-wise separable convolution (Chollet, 2017) instead of traditional
standard convolution in the decoder and the reuse of features that leads to a
substantial reduction of the number of parameters in the encoder are the main
factors in making the DSNet lightweight. Depth-wise separable convolution is a
spatial convolution performed independently over each channel of an input
followed by a point-wise convolution, i.e. a $1\times 1$ convolution that
projects each output channel of the depth-wise convolution onto a new channel
space. For any convolution layer, suppose the number of filters, depth and
kernel size are $N_{F}$, $M_{D}$ and $K$ respectively. The the total number of
parameters of that layer is $N_{F}\times M_{D}\times K^{2}$ and
$M_{D}\times(N_{F}+K^{2})$ for standard and depth-wise separable convolution
respectively. Thus, we are able to reduce the number of parameters by a factor
of $(1/N_{F}+1/K^{2})$ per each convolution layer of DSNet.

###  2.2 Datasets and Hardware

The proposed network was trained on the publicly available International Skin
Imaging Collaboration (ISIC-2017) training dataset (Codella et al., 2018) and
was tested on both ISIC-2017 test and PH2 (Mendonça et al., 2013) datasets.
The ISIC-2018 challenge dataset (Codella et al., 2018) was not used as the
organizers haven’t provided the groundtruth data for validation and testing
set. The images in ISIC-2017 dataset are $8$-bit RBG having various
resolutions ranging from $540\times 722$ to $4,499\times 6,748$ pixels whereas
the ones in PH2 dataset are $8$-bit RBG having the same resolution of
$768\times 560$ pixels. The training, validation, and test datasets of ISIC
comprise $2,000$, $150$, and $600$ images respectively. Furthermore, the
percentage of melanoma (mel), seborrheic keratosis (sk), and nevus (nev) are
($18.8\,\%$, $12.7\,\%$, and $68.5\,\%$), ($20.0\,\%$, $28.0\,\%$, and
$52.0\,\%$), and ($19.5\,\%$, $15.0\,\%$, and $65.5\,\%$) in training,
validation, and testing respectively. On the other hand, PH2 dataset has
$20.0\,\%$ and $80.0\,\%$ of mel and nev respectively. A set of images from
the ISIC-2017 test dataset, representing various challenges to overcome is
presented in Fig. 1. The networks were implemented using the python
programming language in keras framework with tensorflow backend and the
experiments were carried out on a desktop running Ubuntu 16.04 LTS operating
system with the following hardware configuration: Intel® CoreTM i7-6850K CPU @
$3.60GHz\times 12$ processor and GeForce GTX 1080/PCle/SSE2 GPU with 8GB GDDR5
memory.

###  2.3 Training and Evaluation Metrics

In the proposed DSNet, the kernels of the encoder were initialized using the
pre-trained weights from ImageNet (Deng et al., 2009) and the kernels of
decoder were initialized using the he normal distribution (He et al., 2015).
It draws samples from a truncated normal distribution centered on $0$ and the
standard deviation is $\sqrt{2/K}$ where $K$ is the number of input units of
the weight tensor. All input images were standardized and rescaled to $[0\,1]$
range before feeding them into the network. As we noticed that most of the
images of the ISIC-2017 dermoscopic dataset have a $3:4$ aspect ratio, all
images were resized to $192\times 256$ pixels. To increase the robustness of
the segmentation, geometric augmentations such as rotation, zooming, shifting,
and flipping were performed. The ISODATA (Iterative Self-Organizing Data
Analysis Technique) (El-Zaart, 2010) method was used to threshold the output
probability map from the $sigmoid$ activator in order to retrieve the skin
lesion ROI. The region having the maximum connected number of pixels was
selected as the skin lesion ROI. The binary or categorical cross-entropy
functions are widely used as loss functions in both classification and
semantic segmentation. However, using binary or categorical cross-entropy as
loss functions may lead to bias effects as the size of a lesions is
drastically smaller than the size of the background. Hence, we minimize the
sum of binary cross-entropy and IoU (Yuan and Lo, 2019) as the loss function
($L_{seg}$) and is expressed in Eq. 2.

$L_{seg}(y,\hat{y})=\frac{1}{N}\sum_{i=1}^{N}[y_{i}\log\hat{y_{i}}+(1-y_{i})\log(1-\hat{y_{i}})]+1-\frac{\sum_{i=1}^{N}y_{i}\times\hat{y_{i}}}{\sum_{i=1}^{N}y_{i}+\sum_{i=1}^{N}\hat{y_{i}}-\sum_{i=1}^{N}y_{i}\times\hat{y_{i}}}$
(2)

where, $y$, $\hat{y}$ and $N$ are the true labels, predicted labels and total
numbers of pixel respectively. The loss function is optimized using $adadelta$
(Zeiler, 2012) with initial learning rate = $1.0$ and decay factor = $0.95$.
For better optimization, the initial learning rate was reduced when the
specified metric had stopped improving. When learning loss stagnated for $8$
epochs, the learning rate was reduced by $40\%$.

The mean Intersection over Union (mIoU), mean Sensitivity (mSn), and mean
Specificity (mSp) have been used for the quantitative analysis of segmented
masks in different experiments. mIoU is used to quantify the percentage
overlap between the true and predicted lesion masks whereas mSn and mSp are
used to quantify the type II error (false negative rate) and type I error
(false positive rate) respectively.

##  3 Results and Discussion

In this section, the qualitative and quantitative segmentation results
obtained by several extensive experiments are reported. First, the proposed
DSNet is compared against two other implemented networks: namely, FCN8s and
U-Net along with a comparison of the loss function ($L_{seg}$) against cross-
entropy and intersection over union (IoU). Then, the proposed DSNet is further
evaluated with respect to different classes (mel, sk, and nev) segmentation
for both ISIC-2017 and PH2 datasets. Finally, the performance of DSNet is
compared against the state-of-the-art networks.

###  3.1 Evaluation of Proposed DSNet and Loss function

The quantitative results of semantic segmentation on ISIC-2017 test dataset
for the three networks (DSNet, FCN8s, and U-Net) employing different loss
functions are shown in Table 1. It is observed that irrespective of the
segmentation network, the specificity of both the cross entropy and IoU loss
functions is higher than the sensitivity which implies that the background
skin is segmented more accurately than the lesions. The true positive rate
(sensitivity) was improved by using the proposed loss function compared to the
IoU loss funtion while the true negative rate (specificity) was improved using
the proposed loss function compared to the cross entropy loss function as
presented in the Table 1. The proposed loss function resulted in a larger
overlap (mIoU) between the true and predicted masks compared to both cross-
entropy and IoU loss functions for all three networks. Furthermore, it is also
observed that the proposed DSNet yields the best performances with respect to
mIoU by improving the U-Net and FCN8s results by $3.6\,\%$ and $6.8\,\%$
respectively when the proposed loss function is employed. The compactness of
DSNet is evidenced by having $3.8$ and $13.8$ times less number of parameters
than U-Net and FCN8s respectively.

Table 1: The quantitative results of semantic segmentation for the three networks (FCN8s, U-Net, and the proposed DSNet) using different loss functions for ISIC-2017 test dataset. Metrics were calculated using the ground truth and the predicted labels obtained using the networks. |  |  | Metric
---|---|---|---
Network | Parameters | loss function | mIoU | mSn | mSp
|  | Cross Entropy | 0.688 | 0.926 | 0.893
|  | IoU Loss | 0.658 | 0.718 | 0.965
FCN8s | 138 M | Proposed Loss | 0.707 | 0.858 | 0.938
|  | Cross Entropy | 0.717 | 0.900 | 0.933
|  | IoU Loss | 0.693 | 0.719 | 0.986
U-Net | 38 M | Proposed Loss | 0.739 | 0.785 | 0.982
|  | Cross Entropy | 0.771 | 0.832 | 0.977
|  | IoU Loss | 0.743 | 0.782 | 0.984
Proposed Network | 10 M | Proposed Loss | 0.775 | 0.875 | 0.955

Fig. 5 presents a sample of the qualitative results obtained by the three
different networks using the proposed loss function. The checkerboard effect
can be observed on the segmented masks of images obtained using FCN8s due to
the non-divisible upscaling factor by the size of the kernels in its decoder
when projecting the lower resolution discriminating features learnt by the
encoder onto the pixel space of higher resolution. It can be also noticed that
the segmented masks using FCN8s have more False Positives (FP) and coarse
boundaries due to deconvolution overlapping in the decoder. As evidenced by
Fig. 5, the masks obtained using the U-Net suffer from under segmentation due
to the lack of learnt salient features and lost spatial resolution in the
encoder. It can also be clearly seen that there are more False Negatives (FN)
with the results obtained using U-Net compared to both FCN8s and DSNet which
is highly undesirable in medical diagnosis because skin lesions would be
predicted as healthy (background) skin by the CAD system. On the other hand,
the proposed network with the proposed loss function demonstrates better
performance in skin lesion segmentation as shown in Fig. 5.

Original | FCN8s | U-Net | Proposed Net
---|---|---|---
|  |  |
|  |  |
|  |  |
|  |  |
Figure 5:  Qualitative segmentation results for ISIC-2017 test dataset using
FCN8s, U-Net, and DSNet network implementations. Green, Red, and Yellow colors
indicate the TP, FN, and FP respectively. The Dice coefficient (top-left) and
IoU (top-right) are provided for quantitative evaluation. More segmented
results are available on GitHub5.

The proposed approach obtains a lower number of false positives and false
negatives compared to the other two implemented networks. Fig. 6 shows
receiver operating characteristic (ROC) curve of the different networks. The
proposed DSNet obtains an area under ROC curve (AUC) of $0.953$ which
indicates that for any given random pixel, the probability of accurate
classification is as high as $95.3\,\%$. Also from Fig. 6 and given a $10\,\%$
false positive rate, the true positive rates of the FCN8s, U-Net, and proposed
DSNet are $86.0\,\%$, $90.0\,\%$, and $92.0\,\%$ respectively. The fact that
the proposed DSNet not only learns highly abstract salient features of skin
lesions using a lower number of parameters (10 M compared to 38 M and 138 M of
FCN8s and U-Net respectively) but also leads to improved semantic segmentation
performance indicate that it can be potentially used as a CAD tool for lesion
segmentation. In some cases as shown in A, DSNet also fails to segment certain
skin lesions. However, in those wrong segmentations, the false positive rate
is generally more than the false negative rate which indicates a better
suitability of DSNet for medical applications.

Figure 6:  Receiver Operating Characteristic (ROC) of the segmented masks
using DSNet, FCN8s, and U-Net. The dashed diagonal line represents the ROC
curve of a random predictor as a baseline.

###  3.2 DSNet Performance on Different Classes

The proposed DSNet was further evaluated on different classes of skin lesion
for both ISIC-2017 test and PH2 datasets. The quantitative and qualitative
results for mel, sk, and nev lesion class segmentation are presented in Table
2 and Fig. 7 respectively.

Table 2: The quantitative results for different skin lesion class segmentation for both ISIC-2017 test and PH2 datasets using the proposed DSNet. Metrics were calculated using the ground truth and the predicted labels obtained using the networks. | ISIC-2017 test dataset | PH2 dataset
---|---|---
Metric | nev | mel | sk | overall | nev | mel | overall
mIoU | 0.808 | 0.730 | 0.684 | 0.775 | 0.891 | 0.835 | 0.870
mSn | 0.907 | 0.836 | 0.832 | 0.875 | 0.945 | 0.929 | 0.929
mSp | 0.956 | 0.939 | 0.953 | 0.955 | 0.976 | 0.849 | 0.969
ISIC-2017 test dataset | PH2 dataset
---|---
Original | Overlaid | Original | Overlaid
|  |  |
|  |  |
|  |  |
|  |  |
Figure 7: Qualitative segmentation results for ISIC-2017 test and PH2 datasets
using the proposed DSNet network. Green, Red, and Yellow colors indicate the
TP, FN, and FP respectively. The Dice co-efficient (top-left) and IoU (top-
right) are provided for quantitative evaluation. More segmented results are
available on GitHub5.

(a)

(b)

Figure 8: Class wise ROC curves for lesion segmentation using the proposed
DSNet. a) ROC curves for mel, sk, and nev classes for ISIC-2017 test dataset
and b) ROC curves for mel and nev classes for PH2 dataset.

Even though the training images were imbalanced for different lesion classes,
the high quality segmentation results obtained for all classes (see Table 2) a
better performance of our proposed network for semantic lesion segmentation.
The qualitative results in Fig. 7 show the accuracy of the lesion segmentation
using DSNet for both ISIC-2017 test and PH2 datasets although PH2 dataset was
neither used for training nor validation of DSNet. The mIoU ($0.87$) on PH2
demonstrates the robustness of the proposed DSNet since the most abstract
features were learnt from the ISIC-2017. From Table 2, it can be also observed
that for both ISIC and PH2 datasets, the nev class is easier to segment than
other two skin lesion classes due to the larger existing percentage of nev
cases in the training dataset (there are $3.7$ and $5.4$ times more nev images
than mel and sk classes respectively). The ROC curves obtained from the
segmented lesion masks using the proposed DSNet for different lesion classes
of both datasets are presented in Fig. 8. In can be observed that for $10\,\%$
false positive rate, the true positive rates of sk, mel, and nev classes
respectively are $82.0\,\%$, $85.0\,\%$, and $95.0\,\%$ for ISIC-2017 test
dataset while the true positive rates of mel and nev classes are $85.0\,\%$
and $99.0\,\%$ respectively for PH2 dataset. These segmentation mask results
on both ISIC and PH2 datasets vouch for the suitability of DSNet for different
skin lesion class segmentation and its success as a CAD tool for melanoma
detection in CAD systems.

###  3.3 Advantages of DSNet over Recent Networks

In this section, the performance of DSNet is compared and contrasted to
several recent networks used for skin lesion segmentation using the same
datasets for training, validation, and testing. The AUC of the segmented masks
for proposed DSNet and FrCN (Al-masni et al., 2018) are provided in Table 3
which were trained and tested on the same datasets. DSNet outperforms FrCN by
substantial margins in all lesion class segmentation for both ISIC-2017 test
and PH2 datasets. Consequently, these results indicate the ability of DSNet to
learn more salient and discriminant features of skin lesions comparative to
FrCN.

Table 3: The AUC of the DSNet and the recent FrCN by Al-masni et al. (2018) for skin lesion class segmentation for ISIC-2017 test and PH2 datasets. Both networks were trained and tested on the same datasets. | ISIC-2017 test dataset | PH2 dataset
---|---|---
Network | mel | sk | nev | overall | mel | nev | overall
FrCN | 0.875 | 0.882 | 0.932 | 0.910 | 0.941 | 0.950 | 0.946
DSNet (Proposed) | 0.928 | 0.917 | 0.970 | 0.953 | 0.955 | 0.996 | 0.987

Skin lesion segmentation performance of the proposed DSNet and a number of
other state-of-the-art networks for ISIC-2017 test and PH2 datasets are
presented in Table 4. DSNet produces the best segmentation for three out of
the six cases while performing similarly with the winning networks on the
other three cases. DSNet produces the best results with respect to the true
positive rate (lesion detected as lesion) beating the state-of-the-art network
(Li and Shen, 2018) by a $2.0\,\%$ margin. With respect to the true negative
rate, DSNet is behind the state-of-the-art network MResNet-Seg (Bi et al.,
2017a) by $3.0\,\%$. However, comparing the true positive rate of DSNet
(0.875) and MresNet-Seg (0.802), we observe that DSNet outperforms the latter
by a $7.3\,\%$ margin providing ample evidence for the higher lesion region
segmentation ability of the former. With respect to mIoU, DSNet produces the
second best results behind SLSDeep (Sarker et al., 2018) by a small margin of
$0.7\,\%$ indicating that the latter network is able to learn the background
more accurately than the former. However, DSNet outperforms SLSDeep with
respect to the true positive rate by a margin of $5.9\,\%$ demonstrating the
higher ability of the former to correctly segment skin lesions. Additionally,
the proposed DSNet is a lighter network compared to SLSDeep as the former has
less number parameters than the latter. Moreover, the proposed network
demonstrates excellent performance on PH2 dataset than the state-of-the-art
networks as presented in Table 4. Since the proposed network also performs
better on PH2 dataset, it can be claimed that DSNet is more generic as it
possesses the capacity to learn more abstract features by efficient reuse of
the features inside the dense blocks in the encoder.

Table 4: Semantic segmentation performance metrics for the proposed DSNet and
other state-of-the-art networks on both ISIC-2017 test and PH2 datasets.

| ISIC-2017 test dataset | PH2 dataset
---|---|---
Network | mIoU | mSn | mSp | mIoU | mSn | mSp
SegNet* (Al-masni et al., 2018) | 0.696 | 0.801 | 0.954 | 0.808 | 0.865 | 0.966
II-FCN(Wen, 2017) | 0.699 | 0.841 | 0.984 | - | - | -
LIN (Li and Shen, 2018) | 0.753 | 0.855 | 0.974 | - | - | -
MResNet-Seg (Bi et al., 2017a) | 0.760 | 0.802 | 0.985 | - | - | -
FrCN (Al-masni et al., 2018) | 0.771 | 0.854 | 0.967 | 0.848 | 0.937 | 0.957
SLSDeep (Sarker et al., 2018) | 0.782 | 0.816 | 0.983 | - | - | -
Proposed DSNet | 0.775 | 0.875 | 0.955 | 0.870 | 0.929 | 0.969

*Originally proposed by Badrinarayanan et al. (2017) and was implemented by Al-masni et al. (2018) for skin lesion segmentation.

Additionally, the average test time for a single dermoscopy image was
$0.595\,sec$ (357 seconds for 600 images) for the proposed DSNet (10M
parameters) compared to the $9.7\,sec$ for FrCN (184M parameters) by Al-masni
et al. (2018). The proposed DSNet appears better suited for eventual CAD
systems due to its extremely fast processing time compared to the current
state-of-the art (roughly 16 times faster than recent FrCN).

The following presents a concise discussion of the possible bottlenecks that
may lead the recent networks shown in Table 4 to perform poorly comparatively
to the proposed DSNet. Pooling layers are required to reduce the in-plane
dimensionality in order to introduce translation invariance, to decrease the
number of subsequent learnable parameters, and to increase the capability of
learning more abstract features (Yamashita et al., 2018). However, FrCN does
not have pooling layers to capture the full resolution feature maps.
Consequently, this may explain the lower AUC values obtained. In SegNet for
skin lesion segmentation implemented by Al-masni et al. (2018), the authors
store the indices at each max-pooling layer in the encoder which are later
used to upsample the corresponding feature map in the decoder in order to
preserve the high-frequency information. Nevertheless, the authors do not take
the neighbouring information into account during upsampling. In II-FCN, Wen
(2017) uses inception blocks which are very deep networks and are prone to
overfitting. LIN (Lesion Indexing Network) (Li and Shen, 2018) is based on
fully convolutional residual networks (FCRN-88) and is the deepest network
among the networks presented in the Table 4. The extensive depth of LIN may
also lead the CNN model to be overfitted due to the limited numbers of images
comparative to its depth. Bi et al. (2017a) uses ResNets architecture in
MResNet-Seg which often has gradient fading problems that lead the CNN model
to be overfitted.

##  4 Conclusion

In this article, a new robust and automatic skin lesion segmentation network
called DSNet has been proposed and implemented. Additionally, a new loss
function comprising a binary cross-entropy term and intersection over union
term has also been proposed. The potential of the proposed network has been
evidenced through multiple extensive experiments. The hurdle of having a
limited number of labeled skin lesion images to train the DSNet was overcome
by transferring the knowledge from another pre-trained model and by augmenting
the images. The proposed new loss function with the addition of the IoU to the
more traditional cross-entropy has proven to be better suited for semantic
segmentation by achieving higher true positive rates in the experiments
carried out. The proposed loss function may play an important role in semantic
segmentation where ROI selection is crucial. The proposed DSNet possesses the
highly desirable trait of having the least number of parameters among all
compared networks, while outperforming the existing baseline segmentation
networks with respect to several metrics. Thus, DSNet is the least memory
intensive among all compared networks. Further tuning of the hyper-parameters
and the introduction of more color space augmentations may yield better
segmentation performances. In the future, the segmented ROIs can be used to
extract features for different lesion type classification (mel, sk, and nev).
The proposed DSNet and the loss function will be applied to other medical
contexts to verify its generality and versatility.

## References

  * Abbas et al. (2011) Abbas, Q., Fondon, I., Rashid, M., 2011.  Unsupervised skin lesions border detection via two-dimensional image analysis.  Computer methods and programs in biomedicine 104, 1–15.
  * Ajmal et al. (2018) Ajmal, H., Rehman, S., Farooq, U., Ain, Q.U., Riaz, F., Hassan, A., 2018\.  Convolutional neural network based image segmentation: a review, in: Pattern Recognition and Tracking XXIX, International Society for Optics and Photonics. p. 106490N.
  * Al-masni et al. (2018) Al-masni, M.A., Al-antari, M.A., Choi, M., Han, S., Kim, T., 2018.  Skin lesion segmentation in dermoscopy images via deep full resolution convolutional networks.  Computer Methods and Programs in Biomedicine 162, 221–231.
  * American Cancer Society (2019) American Cancer Society, 2019.  Key statistics for melanoma skin cancer.  https://www.cancer.org/cancer/melanoma-skin-cancer/about/key-statistics.html.  [Online; accessed 10-May-2019].
  * Badrinarayanan et al. (2017) Badrinarayanan, V., Kendall, A., Cipolla, R., 2017.  SegNet: a deep convolutional encoder-decoder architecture for image segmentation.  IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 2481–2495.
  * Bi et al. (2017a) Bi, L., Kim, J., Ahn, E., Feng, D., 2017a.  Automatic skin lesion analysis using large-scale dermoscopy images and deep residual networks.  arXiv:1703.04197, 2017 .
  * Bi et al. (2017b) Bi, L., Kim, J., Ahn, E., Kumar, A., Fulham, M., Feng, D., 2017b.  Dermoscopic image segmentation via multistage fully convolutional networks.  IEEE Transactions on Biomedical Engineering 64, 2065–2074.
  * Celebi et al. (2008) Celebi, E.M., Kingravi, H.A., Iyatomi, H., Aslandogan, Y.A., Stoecker, W.V., Moss, R.H., Malters, J.M., Grichnik, J.M., Marghoob, A.A., Rabinovitz, H.S., 2008\.  Border detection in dermoscopy images using statistical region merging.  Skin Research and Technology 14, 347–353.
  * Celebi et al. (2009) Celebi, M.E., Iyatomi, H., Schaefer, G., Stoecker, W.V., 2009\.  Lesion border detection in dermoscopy images.  Computerized medical imaging and graphics 33, 148–153.
  * Chollet (2017) Chollet, F., 2017.  Xception: Deep learning with depthwise separable convolutions.  IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 1800–1807.
  * Codella et al. (2018) Codella, N.F., Gutman, D., Celebi, M.E., Helba, B., Marchetti, M.A., Dusza, S.W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., 2018\.  Skin lesion analysis toward melanoma detection: A challenge at the 2017 international symposium on biomedical imaging (isbi), hosted by the international skin imaging collaboration (isic), in: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), IEEE. pp. 168–172.
  * Deng et al. (2009) Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L., 2009.  ImageNet: A large-scale hierarchical image database, IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255.
  * Doi (2007) Doi, K., 2007.  Computer-aided diagnosis in medical imaging: Historical review, current status and future potential.  Comput Med Imaging Graph 31, 198–211.
  * El-Zaart (2010) El-Zaart, A., 2010.  Images thresholding using isodata technique with gamma distribution.  Pattern Recognition and Image Analysis 20, 29–41.
  * Esteva et al. (2017) Esteva, A., Kuprel, B., Novoato, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S., 2017.  Dermatologist-level classification of skin cancer with deep neural networks.  Nature 542, 115–118.
  * Fan et al. (2017) Fan, H., Xie, F., Li, Y., Jiang, Z., Liu, J., 2017.  Automatic segmentation of dermoscopy images using saliency combined with otsu threshold.  Computers in biology and medicine 85, 75–85.
  * Garcia-Garcia et al. (2018) Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., Garcia-Rodriguez, J., 2018.  A survey on deep learning techniques for image and video semantic segmentation.  Applied Soft Computing 70, 41–65.
  * Garnavi et al. (2010) Garnavi, R., Aldeen, M., Celebi, M.E., Bhuiyan, A., Dolianitis, C., Varigos, G., 2010\.  Automatic segmentation of dermoscopy images using histogram thresholding on optimal color channels.  International Journal of Medicine and Medical Sciences 1, 126–134.
  * Ge et al. (2017) Ge, Z., Demyanov, S., Chakravorty, R., Bowling, A., Garnavi, R., 2017.  Skin disease recognition using deep saliency features and multimodal learning of dermoscopy and clinical images, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 250–258.
  * Guo et al. (2018) Guo, Y., Liu, Y., Georgiou, T., Lew, M.S., 2018\.  A review of semantic segmentation using deep neural networks.  International journal of multimedia information retrieval 7, 87–93.
  * Haenssle et al. (2018) Haenssle, H.A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A., Thomas, L., Enk, A., 2018\.  Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists.  Annals of Oncology 29, 1836–1842.
  * He et al. (2015) He, K., Zhang, X., Ren, S., Sun, J., 2015.  Delving deep into rectifiers: Surpassing human-level performance on imagenet classification.  arXiv:1502.01852 .
  * Huang et al. (2017) Huang, G., Liu, Z., v. d. Maaten, L., Weinberger, K.Q., 2017\.  Densely connected convolutional networks, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269.
  * Ioffe and Szegedyet (2015) Ioffe, S., Szegedyet, C., 2015\.  Batch Normalization: accelerating deep network training by reducing internal covariate shift.  arXiv:1502.03167 .
  * Jalalian et al. (2017) Jalalian, A., Mashohor, S., Mahmud, R., Karasfi, B., Saripan, M.I.B., Ramli, A.R.B., 2017\.  Foundation and methodologies in computer-aided diagnosis systems for breast cancer detection.  EXCLI Journal 16, 113–137.
  * Kaiser et al. (2017) Kaiser, L., Gomez, A.N., Shazeer, N., Vaswani, A., Parmar, N., Jones, L., Uszkoreit, J., 2017.  One model to learn them all.  CoRR abs/1706.05137.
  * Korotkov and Garcia (2012) Korotkov, K., Garcia, R., 2012\.  Computerized analysis of pigmented skin lesions: a review.  Artificial intelligence in medicine 56, 69–90.
  * Li and Shen (2018) Li, Y., Shen, L., 2018.  Skin lesion analysis towards melanoma detection using deep learning network.  Sensors 11.
  * Lin et al. (2014) Lin, M., Chen, Q., Yan, S., 2014.  Network in network.  International Conference on Learning Representations (ICLR) .
  * Long et al. (2015) Long, J., Shelhamer, E., Darrell, T., 2015.  Fully convolutional networks for semantic segmentation, in: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440.
  * Melli et al. (2006) Melli, R., Costantino, C., Cucchiara, R., 2006.  Comparison of color clustering algorithms for segmentation of dermatological images, in: Medical Imaging 2006: Image Processing, International Society for Optics and Photonics. p. 61443S.
  * Mendonça et al. (2013) Mendonça, T., Ferreira, P.M., Marques, J.S., Marcal, A.R.S., Rozeira, J., 2013.  Ph 2-a dermoscopic image database for research and benchmarking, in: 2013 35th annual international conference of the IEEE engineering in medicine and biology society (EMBC), IEEE. pp. 5437–5440.
  * Mishraa and Celebi (2016) Mishraa, N.K., Celebi, M.E., 2016\.  An overview of melanoma detection in dermoscopy images using image processing and machine learning.  arXiv:1601.07843 .
  * Odena et al. (2016) Odena, A., Dumoulin, V., Olah, C., 2016.  Deconvolution and checkerboard artifacts, Distill.
  * Oliveira et al. (2016) Oliveira, R.B., Filho, M.E., Z, M., Papa, J.P., Pereira, A.S., Tavares, J.M., 2016\.  Computational methods for the image segmentation of pigmented skin lesions: A review.  Computer Methods and Programs in Biomedicine 131, 127–141.
  * Rasmus et al. (2015) Rasmus, A., Valpola, H., Honkala, M., Berglund, M., Raiko, T., 2015.  Semi-supervised learning with ladder networks.  28th International Conference on Neural Information Processing Systems 2, 3546–3554.
  * Ronneberger et al. (2015) Ronneberger, O., Fischer, P., Brox, T., 2015.  U-net: Convolutional networks for biomedical image segmentation.  International Conference on Medical Image Computing and Computer-Assisted Intervention 9351, 234–241.
  * Salimi et al. (2018) Salimi, S.B., Bozorgtabar, S., Schmid-Saugeon, P., Ekenel, H.K., Thiran, J., 2018.  DermoNet: densely linked convolutional neural network for efficient skin lesion segmentation.  EPFL scientific publications .
  * Sarker et al. (2018) Sarker, M.M.K., Rashwan, H.A., Akram, F., Banu, S.F., Saleh, A., Singh, V.K., Chowdhury, F.H., Abdulwahab, S., Romani, S., Radeva, P., 2018\.  SLSdeep: skin lesion segmentation based on dilated residual and pyramid pooling networks, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer. pp. 21–29.
  * Smith and MacNeil (2011) Smith, L., MacNeil, S., 2011\.  State of the art in non‐invasive imaging of cutaneous melanoma.  Skin Res. Technol. 17, 257–269.
  * Tang et al. (2018) Tang, Y., Yang, F., Yuan, S., Zhan, C.A., 2018.  A multi-stage framework with context information fusion structure for skin lesion segmentation.  CoRR abs/1810.07075.
  * Wen (2017) Wen, H., 2017.  Ii-fcn for skin lesion analysis towards melanoma detection.  arXiv:1702.08699 .
  * Yamashita et al. (2018) Yamashita, R., Nishio, M., Do, R.K.G., Togashi, K., 2018\.  Convolutional neural networks: an overview and application in radiology.  Insights into Imaging 9, 611–629.
  * Yuan (2017) Yuan, Y., 2017.  Automatic skin lesion segmentation with fully convolutional-deconvolutional networks.  arXiv:1703.05165 .
  * Yuan and Lo (2019) Yuan, Y., Lo, Y., 2019.  Improving dermoscopic image segmentation with enhanced convolutional-deconvolutional networks.  IEEE Journal of Biomedical and Health Informatics 23, 519–526.
  * Zeiler (2012) Zeiler, M.D., 2012.  Adadelta: an adaptive learning rate method.  CoRR abs/1212.5701.
  * Zhou et al. (2013) Zhou, H., Li, X., Schaefer, G., Celebi, M.E., Millera, P., 2013.  Mean shift based gradient vector flow for image segmentation.  Computer Vision and Image Understanding 117, 1004–1016.

##  Appendix A Prediction Samples

The predicted segmentation masks (Pred) using the proposed network overlaid on
the ground truth (GT) where TP, FP, and FN are depicted in green, yellow, and
red respectively. The first 6 rows depict the results for the challenging
images presented in Fig. 1 whereas the last 3 rows are the worst predictions
obtained using our network.

Original Image | Pred and GT Overlay | Original Image | Pred and GT Overlay
---|---|---|---
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |
|  |  |