#  An encoder-decoder-based method for COVID-19 lung infection segmentation

Omar Elharrouss<EMAIL_ADDRESS>Noor Almaadeed<EMAIL_ADDRESS>Nandhini Subramanian<EMAIL_ADDRESS>Somaya Al-Maadeed
<EMAIL_ADDRESS>Department of Computer Science and Engineering, Department
of Computer Science and Engineering, Qatar University , Doha, Qatar, Doha,
Qatar

###### Abstract

The novelty of the COVID-19 disease and the speed of spread has created a
colossal chaos, impulse among researchers worldwide to exploit all the
resources and capabilities to understand and analyze characteristics of the
coronavirus in term of the ways it spreads and virus incubation time. For
that, the existing medical features like CT and X-ray images are used. For
example, CT-scan images can be used for the detection of lung infection. But
the challenges of these features such as the quality of the image and
infection characteristics limitate the effectiveness of these features. Using
artificial intelligence (AI) tools and computer vision algorithms, the
accuracy of detection can be more accurate and can help to overcome these
issues. This paper proposes a multi-task deep-learning-based method for lung
infection segmentation using CT-scan images. Our proposed method starts by
segmenting the lung regions that can be infected. Then, segmenting the
infections in these regions. Also, to perform a multi-class segmentation the
proposed model is trained using the two-stream inputs. The multi-task learning
used in this paper allows us to overcome shortage of labeled data. Also, the
multi-input stream allows the model to do the learning on many features that
can improve the results. To evaluate the proposed method, many features have
been used. Also, from the experiments, the proposed method can segment lung
infections with a high degree performance even with shortage of data and
labeled images. In addition, comparing with the state-of-the-art method our
method achieves good performance results.

###### keywords:

Lung infection segmentation , COVID-19 , CT-scan image , Encoder-decoder
network

††journal: Nuclear Physics B

##  1 Introduction

During the first two months, the coronavirus COVID-19 affected thousands of
people around the world with a big number of deaths. Wuhan was the first
epicenter of the coronavirus, then it began to spread to every continent and
most of the countries [1]. The severe acute respiratory syndrome or the
acronym known by SARS-CoV-2 is an infectious disease that appeared in late
2019. SARS-CoV-2 is known also by COVID-19 due to its similarity to solar
corona from the electron microscopic analysis [2]. The novelty of the
Coronavirus and the speed of spread, has created a colossal chaos, impulse
among the researchers worlwide to exploit all resources and capabilities to
understand and analyze characteristics of the coronavirus in term of spread
ways and virus incubation time as well as the exploration of new technics and
imposing some temporal procedures to stop the spread speed. The spreading of
the virus became unstoppable which prompted the governments to mitigate the
impacts of the pandemic using many decisions like stopping the fight and
closing the borders and social distancing. However, these policies were not
efficient in controlling the spread. To contribute to the implementation of
these policies scientists and technology experts attempted to find solutions
to stop the speed of the spread. From the technologies used, we can find
robots which are used to mitigate the contact between the coronavirus patients
and the hospital employees [3]. Also, drones used for monitoring the social
distancing and disinfect the public spaces. For the scientists, especially
health scientist, searching for medicines and cures that can be helpful to
save lives, is the most important mission for them. Whereas the researches in
the other domains like computer sciences involved themselves in discovering
technics that detect infected persons using the existing medical features like
CTscans and X-ray images. Using artificial intelligence (AI) tools and
computer vision algorithms, the accuracy of detection can be more accurate and
with high precision [4, 5, 6, 7, 49, 50]. For that, AI technics can be a good
assistant to mitigate the spread by the early detection of the disease. Also,
it can be another choice besides the laboratory analysis and allows us to make
a large number of tests.

Image processing as well as computer vision combined with AI is a
multidisciplinary domain that can be used in different domains including
medical, astronomy, agriculture [8, 9, 10, 11, 13]. For the medical field,
medical imaging has been used for diagnosing diseases using X-ray and CT
images, also for surgery and therapy[3]. Good progress in this field has been
reached due to the introduction of many technics like machine learning and
deep learning algorithms. This improvement made the computer vision scientists
to contribute to finding solutions for rapid diagnostics, prevention, and
control. For the same purpose, several approaches have been proposed even when
the time is very short.

The detection of infection is the first step for the diagnostic of a disease.
Using CT images, it can be seen that the appearances of the infected regions
are different from the normal regions, so the detection and the extraction of
this region automatically can help the doctor for diagnosis in a short
time[8]. For the same purpose, a deep learning method for segmentation lung
infection for COVID-19 on CT images is proposed. Before starting the learning
process, each image has been composed of Structure and texture components. The
structure represents the homogeneous part of the images whereas the texture
component represented the texture of the image. The structure and texture
features are introduced to the encoder-decoder model first for segmenting the
region of interest, regions that can be affected. Then, segmenting the
specific infected parts on the results of the first segmentation.

The remainder of the paper is organized as follows. The literature overview is
presented in section 2. The proposed method is presented in section 3.
Experiments performed to validate the proposed method are discussed in section
4. The conclusion is provided in section 5.

Table 1: Categories of COVID-19 methods  Task |  Method |  Architecture
---|---|---
Classification |  Ozturk et al. [9] |  DarkNet, CNN
Minaee et al. [10] |  ResNet18, CNN
Apostolopoulos et al. [11] |  MobileNet, CNN
Apostolopoulos et al. [12] |  ResNet18, VGG19, Inception, Xception
Abbas et al. [13] |  ResNet18, CNN
Adhikari et al. [19] |  DenseNet-based CNN
Mobiny et al. [21] |  CNN
Polsinelli et al. [22] |  SqueezeNet-based CNN
Al-karawi et al.[23] |  Machine learning, FFT-Gabor
He et al. [25] |  CNN
Talha et al.[26] |  CNN
Class & segmentation |  Wu et al. [20] |  CNN, encoder-decoder
Amyar et al. [25] |  CNN, encoder-decoder
Segmentation |  Fan et al. [27] |  CNN, partial decoder

##  2 Related Works

Recently, medical imaging has gained attention due to its importance to
diagnose, monitor, and treat several medical problems. Radiography, a medical
imagining technique, uses [14, 15, 16], CT-scan[17, 18], and gamma rays to
create images of the body that requires internal viewing. analyzing images
using different computer vision algorithms provides an alternative for rapid
diagnostics and control of many diseases. For COVID-19, image analysis offers
a good solution for early detection due to the complexity of laboratory
analysis and the importance of early detection that can save lives. For the
same purpose, many approaches have been proposed to COVID-19 detection and
control using X-ray and CT-scans images of the lung. Also, the use of
artificial intelligence makes the precision and the performance of these
results convincing. authors in [19] presented COVID-Net architecture that is
based on DenseNet to diagnose the COVID-19 infections from X-rays and CT-scans
images to decrease the turnaround time of the doctors and check more patients
at that point of time. Using the proposed architecture, the infected regions
are detected and marked.

Working on similar radiographic images, authors in [20] developed a
classification and segmentation system for a real-time diagnostic of COVID-19.
The presented model combines convolutional neural networks and encoder-decoder
networks trained on CT-scans images. The proposed approach succeeds to detect
the infected regions with an accuracy of 92%. In the same context, a learning
architecture named Detail-Oriented Capsule Networks (DECAPS) has been proposed
in[21]. To increases model stability, authors use an Inverted Dynamic Routing
mechanism model implementation. The proposed approach detection the infected
region without segmenting the region edges. The same technique is used in[22]
based on SqueezeNet architecture. The obtained results achieve 89% for
performance accuracy wherein [21] the accuracy reaches 87%. To demonstrate the
efficiency and the automatic testes of COVID-19 infection, the authors in [23]
proposed a machine learning scheme method for CT-scan images analysis for
COVID-19 patients. based on the FFT-Gabor scheme, the proposed method for
predicting the state of the patient in real-time. The performance accuracy
achieves in average 95.37% which represents a convincing rate.

For developing an accurate method for real-time diagnosing of COVID-19 using
CT scan images, deep learning models require a large-scale dataset to train
these models, which might be difficult to obtain at this moment. Hence,
authors in [24], build public CT scans dataset people detected positive for
COVID-19. Also, a deep-learning-based method has been proposed for classifying
the COVID or NON-COVID images. The approach showed an accuracy of 72%, which
means that it is not a very accurate method for COVID-19 testing, . A
multitask deep learning model to identify COVID-19 patient and segmentation of
infected regions from chest CT images has been proposed in [25]. The authors
used an encoder-decoder model for segmentation and perceptron for
classification. The proposed method has the same technic of the method
proposed in [20] but using different architectures. The obtained performance
accuracy of the proposed method reaches 86%. With an accuracy of 89%,
EfficientNet a neural network architecture is proposed for the detection of
COVID-19 patients using CT-scan images [26]. For segmenting the infected
region in a CT-scan image for COVID-19 patient identification, the authors in
[27] proposed a convolutional neural network (CNN) model with a partial
decoder. The proposed model performance achieves an accuracy of 73% for
detecting the infected regions in a CT-scan image. The authors also train
other architectures including Unet [28],Unet++ [29],Attention-Unet [30],Gated-
Unet [31], and Dense-Unet [32] on the same dataset.

All the proposed method attempts to develop a timely and effective method for
testing the coronavirus patients. The proposed methods can be classified into
two general categories: Classification methods [19, 21, 22, 23] and
segmentation [27] of infected region methods. Some of the presented approaches
work on two tasks like [20, 25]. Table 1 presents the proposed method for each
category as well as the architectures used in each one of them.

##  3 Proposed method

In this section the proposed approach for lung infection segmentation is
provided. The method starts by splitting the texture and structure component
of the image, before starting the training of the proposed model for regions
of interest extraction. The extraction of these regions is the operation of
separation or segmentation of the regions that can contain the infection which
are the intern region of the lung. After the extraction of regions of
interest, which is performed using the proposed encoder-decoder network
proposed, the output of the previous operation is used by the segmentation
model which is the sameused for detecting and segmenting the infected part of
the lung. A description is provided for each step including stecture-texture
decomposition, region of interest extraction and lung infection segmentation.

Figure 1: Flowchart of the proposed method.

###  3.1 Structurre and Texture component extraction

Each image can contain information that includes the form of the object (or
the homogenous part) in the image as well as a texture that also contains
information that can be useful in some computer vision tasks. Here, the
texture component is used as input of the proposed encoder-decoder neural
networks. for that, a preprocessing is performed to extract the texture
component. The structure-texture decomposition method proposed in [36] is
adopted using the interval gradient, for adaptive gradient smoothing. Given an
image f, it is a technique that splits the image into S+T (f = S+T) of a
bounded variation component (Structure) and a component that contains the
oscillating part (Texture/Noise) of the image [37, 38]. We applied this
technique on the CT-scan images. Then, we use the texture and structure
components as an input of the proposed encoder-decoder model. The extraction
of the structure component uses gradient rescaling with interval gradients
followed by a color handling operation.

In order to produce a texture-free signal from an input signal I, the
gradients within texture regions should be suppressed. Furthermore, the signal
should be either increasing or decreasing for all local windows $\Omega_{r}$ .
With these objectives, we use the following equation to rescale the gradients
of the input signal with the corresponding interval gradients:

$\begin{array}[]{@{}l}{\left(\nabla^{{}^{\prime}}I\right)}_{p}=\left\\{\begin{array}[]{lc}{\left(\nabla
I\right)}_{p}.w_{p}&if\;sign({\left(\nabla
I\right)}_{p})=sign({\left(\nabla_{\Omega}I\right)}_{p})\\\
0&otherwise\end{array}\right.\end{array}$ (1)

Where$(\nabla^{{}^{\prime}}I)_{p}$ represents the rescaled gradient, and
$w_{p}$ is the rescaling weight:

$\begin{array}[]{@{}l}w_{p}=min\left(1,\frac{\left|{\left(\nabla_{\Omega}I\right)}_{p}\right|+\varepsilon_{s}}{\left|{\left(\nabla
I\right)}_{p}\right|+\varepsilon_{s}}\right)\end{array}$ (2)

Where $\varepsilon_{s}$ is a small constant to prevent numerical instability.
Too small values of $\varepsilon_{s}$ would make the algorithm sensitive to
noise, introducing unwanted artifacts to filtering results. The sensitivity to
noise can be reduced by increasing s but textures may not be completely
filtered if $\varepsilon_{s}$ is too big. We set $\varepsilon_{s}$ = $10^{-4}$
in our implementation.

For filtering color images, we use the gradient sums of color channels in the
gradient rescaling step (Equations (1) and (2)), that is:

$\begin{array}[]{@{}l}w_{p}=min\left(1,\frac{{\displaystyle\sum_{c\in\\{r,g,b\\}}}\left|{\left(\nabla_{\Omega}I^{c}\right)}_{p}\right|+\varepsilon_{s}}{{\displaystyle\sum_{c\in\\{r,g,b\\}}}\left|{\left(\nabla
I^{c}\right)}_{p}\right|+\varepsilon_{s}}\right)\end{array}$ (3)

Figure 3 shows filtering examples to demonstrate the results of structure and
texture extraction extraction using the same parameters
$\varepsilon\in\left[0.01^{2},0.03^{2}\right]$.

Figure 2: Structure and Texture extraction results: Left: original image.
Midle: Structure component. Right: Texture component

###  3.2 Encoder-decoder architecture

Dense pixel-wise classification is a required operation for semantic labeling
of images. To achieve an effective semantic segmentation, many architectures
have been proposed including FCN[39], SegNet [40] architectures. In this
paper, we proposed a model that is based on the SegNet model which is an
encoder-decoder architecture that provides an image output of the same size as
the image input.

The encoder part of SegNet is based on the VGG-16 [41] convolutional layers
that is composed of 5 blocks, where each one contains 2 to 3 convolutional
layers with $3\times 3$ kernels, 1 padding, ReLU, and a batch normalization
(BN)[42]. The convolution block is followed by a max-pooling layer of size
$2\times 2$. At the end of the encoder, each feature map has $H/32,W/32$,
where the original image resolution is $H\times W$.

The decoder part proceeds the operation of upsampling and classification. The
main role of this part is the learning of the method of spatial resolution
restoration by transforming the encoder features maps into the final labels.
The decoder structure is symmetrical with encoder, while the pooling layers in
the encoder part are replaced by unpooling layers in the decoder part. The
role of convolutional blocks after unpooling layer is to densify the sparse
feature maps. This procedure is looped until the feature map reaches the
resolution of the input image.

The final layer in SegNet as well as in most of the proposed architecture in
the same context, SoftMax is used to compute the multinomial loss:

$loss=\frac{1}{N}\sum_{N=1}^{N}\sum_{i=1}^{k}y_{j}^{i}log(\frac{exp(p_{j}^{i})}{\sum_{l=1}^{k}exp(p_{j}^{l})})$
(4)

where N is the number of pixels in the input image, k the number of classes
and, for a specified pixel $i$; $y_{i}$ denote its label and PPP the
prediction vector. This means that we only minimize the average pixel-wise
classification loss without any spatial regularization, as it will be learnt
by the network during training.

In this paper, we followed the same scenarios, but with two streams as input.
The encoder part of the proposed model contains two components. The encoder
for texture component and the encoder for structure component. the encoder
features map is formed by concatenation of these two encoders. In this
architecture, we have 5 encoder blocks. Each block of the encoder is composed
of a $conv+BN+PReLU+pooling$ where, the decoder blocks composed of
$upsampling+conv+BN+PReLU$. The filter size of each convolutional layer in the
encoder part is in the range of 32,64, 128, 256, figure 2 represents the
proposed architecture.

---
Figure 3: Multi-class segemrntation of the lung infected regions  Figure 4:
Region of interest segmentation results. First row : original images. Second
row: groundtruth. third row: segmentation using the propsoed method.

###  3.3 Lung infection segmentation

The current CT-scan dataset of COVID-19 is very limited in terms of the number
of labeled images. Also, the manual segmentation of the infected region of the
lung is very difficult because that needs a domain expert to do it like a
doctor, and also needs time. To solve the current problem of limited data, we
augment this data using rotation and translation of a large number of images
from the labeled data. using the presented encoder-decoder architecture, we
first segment the region of interest or the region that can contain the
infection. The inside part of the lung is segmented before used in the next
stage of lung infection segmentation.

For segmenting the lung infected region using black and white colors, where
the white color represents the infected region, we use the segmented region
which is the output of the first step of segmentation and the original image
as input to the model in the second stage as illustrated in figure 1. The
multi-task segmentation is an effective solution regarding the limitation of
the size of the data discussed above. Also, the segmentation of the infected
region can be used for segmenting these regions using multiclass which
represents the next task in this paper.

###  3.4 Multi-class lung infection segmentation

The segmentation using multiclass or many colors to represent the lung
infection can be more helpful for the diagnostic of COVID-19 and also more
practical. For that, using the previous segmentation of lung infection and the
proposed deep learning model, the multiclass segmentation is performed. Two-
stream inputs of the deep learning model represented by the lung infection
segmentation results and the results of the first segmentation which is the
region of interest. This technique allows learning on the specific region and
can be more accurate according to the data limitation. Figure 3 illustrates
the input and the output of the deep learning model in this stage for
multiclass segmentation.

##  4 Experimental results

In this section, we demonstrate the relevance of our proposed method by
providing the experimental results. the evaluation has been performed on two
segmentation categories including simple segmentation of infected region with
black and white, and multi-class labeling using colors. For the first
category, we compare our results with a set of state-of-the-art methods,
including Unet [28],Unet++ [29],Attention-Unet [30],Gated-Unet [31], Dense-
Unet [32] and Semi-Inf-Net [27]. For multi-class labeling, the obtained
results are compared with four state-of-the-art methods such as Semi-Inf-Net
[27], multi-class U-Net [33], FCN8s [34] and DeepLabV3+ [35]. Also, the
results are visualized by presenting some segmented examples using the
proposed method as well as state-of-the-art methods.

---

---

---

---

---

---

---

---

---

---

---
Dice= 0.849
Sensitivity=0.834
Specitivity=0.987
Precision=0.864
Fmeasure=0.849

---
Dice= 0.870
Sensitivity=0.749
Specitivity=0.990
Precision=0.814
Fmeasure=0.780

---
Dice= 0.850
Sensitivity=0.794
Specitivity=0.996
Precision=0.915
Fmeasure=0.850

---
Dice= 0.765
Sensitivity=0.651
Specitivity=0.998
Precision=0.929
Fmeasure=0.767

---
Dice= 0.676
Sensitivity=0.529
Specitivity=0.997
Precision=0.936
Fmeasure=0.676

Figure 5: infection region segmentation . First row : original images. Second
row: groundtruth. Third row: segmentation using the propsoed method. fourth
row: evaluation results

###  4.1 Datasets

The only segmentation dataset of CT-scan images available for COVID-19 is
111https://medicalsegmentation.com/covid19/. The dataset consists of 100 axial
CT images for 20 COVID-19 patients, collected by the Italian Society of
Medical and Interventional Radiology. The dataset contains CT images labeled.
There are two categories of labels. The first one labels the regions of
interest where the infection can be located. The other one is the specific
infected regions labeled with two colored red and green colors. This part
contains cropped images for training and testing. The training images composed
of 50 images where the infections are labeled with one color (one class) and
multi-class (2 colors). The test folder contains 48 images labeled with the
same labels as the training. in this paper, we train our multitask model on
the two categories of images. We train the first part of our model on the 2000
images for region segmentation. For infection segmentation, we use the labeled
data and we augment the data by rescaling and rotating the images for getting
more data.

Figure 6: Segmentation results using the proposed method. First row: original
images. Second row: binary ground truth. Third row: multi-class ground truth.
Fifth row: Our binary segmentation results. Sixth row: Our multi-class
segmentation results.

###  4.2 Evaluation metrics

To evaluate the segmentation results of the propsoed method, a set of measures
has been exploited including Sorensen-Dice similarity, Sensitivity,
Specitivity, Precision and F-measure used in [45, 46, 47, 48]. For calculation
of these three parameters, the four measures are required, namely true
positive (TP), false positive (FP), true negative (TN), and false negative
(FN). The true positive (TP) represents the number of pixels being correctly
identifed. True negative (TN) describe the number of non-lung infection pixels
being correctly identifed as non-lung infection. False positive (FP) denotes
the number of non-lung infection being wrongly classifed as lung infection,
whereas false negative (FN) means the lung infection pixels being wrongly
classifed as non-lung infecttion.

Sorensen-Dice similarity: Let consider that $A$ is segmented regions that we
need to assess the quality, $B$ is ground truth. The Sorensen-Dice similarity
[46] is computed as follows:

$Dice=\frac{2|A\cap B|}{|A|+|B|}$ (5)

The value of the Sorensen-Dice similarity metric is between 0 and 1. The
higher the Sorensen-Dice value, the better the segmentation result.

Sensitivity: Sensitivity, defned as the ratio of correctly identifed lung
infection to the total number of lung infection pixels, is computed as

$Sensitivity=\frac{TP}{TP+FN}$ (6)

Specifcity: Specifcity, defned as the ratio of correctly detected non-lung
unfection to the total number of non-lung infection pixels, is measured as

$Specifcity=\frac{TN}{TN+FP}$ (7)

Precision: Precision gives the percentage of unnecessary positives through the
compared total number of positive pixels in the detected binary objects mask
[48]

$Precision=\frac{TP}{TP+FP}$ (8)

F-measure: the weighted harmonic mean of precision and sensitivity, computes
the quality of detection.

$F-measure=\frac{2\times Sensitivity\times Precision}{Sensitivity+Precision}$
(9)

Mean Absolute Error (MAE) is also used to evaluate the performance of the
proposed model:

$MAE=\frac{1}{N}\sum_{i=1}^{N}|z_{i}-z_{i}^{gt}|$ (10)

Table 2: Quantitative results of binary infection regions segmentation on COVID-SemiSeg DATASET  Method |  Dice |  Sensitivity |  Specitivity |  Precision |  Fmeasure |  MAE
---|---|---|---|---|---|---
U-Net [28] |  0.439 |  0.534 |  0.858 |  - |  - |  0.186
Attention-UNet [30] |  0.583 |  0.637 |  0.921 |  - |  - |  0.112
Gated-UNet [31] |  0.623 |  0.658 |  0.926 |  - |  - |  0.102
Dense-UNet [32] |  0.515 |  0.594 |  0.840 |  - |  - |  0.184
U-Net++ [29] |  0.422 |  0.379 |  0.976 |  - |  - |  0.120
Semi-Inf-Net [27] |  0.739 |  0.725 |  0.960 |  - |  - |  0.064
Propsed method |  0.786 |  0.711 |  0.993 |  0.856 |  0.784 |  0.076

###  4.3 Discussion

The first step of our proposed model is to segment the regions that can
contain the infection from lung images. This step is sued for segmenting the
lung infection regions in the second part of the proposed method. This step is
an accurate preprocessing operation for segmenting the lung infection region.
Figure 4 illustrates some obtained results for the region of interest
segmentation by presenting the original image with the ground truth as well as
the obtained results. We can observe that the proposed model gives promising
segmentation results. Also, comparing with the ground truth, we obtain a
segmentation without fold segmented regions.

To evaluate the performance of the proposed method for lung infection
segmentation Figure 5 show some examples of the obtained results. From the
visualized results we can observe that the proposed model can detect the
infection effectively with some errors that can be considered negligible.
Also, the segmentation results are close to the ground truth show in the
second line of figure 5. The success of the proposed method to label the
infection is owed to the used architecture that uses two-stream input which
allow robust learning. Also, the use of the results of the region of interest
as input beside the original image gives the model a specification of the
region that can contain the infection. The robustness of the presented
approach is shown also in the presented metrics that demonstrate the
performance evaluation including Dice, Sensitivity, specificity, precision,
and F-measure. For example, for the first CT-scan image, our approach can
segment with a high precision value which achieves 84% for the F-measure
metric which is a convincing value. The presented results can be improved
using a preprocessing on the results images like the morphology operations.

In order to assert the results represented by the binary image in Fig. 5, the
qualitative and quantitative results obtained by each method are represented
in Table 2. From these results, we can see that the proposed method gives high
performance when compared with the state-of-art methods. The effectiveness of
the proposed method comes from the use of multi-task learning and the use of
two-stream input for our model.

The multi-class infection labeling results has also been illustrated in Figure
6. As shown in the figure the proposed method performs an accurate
segmentation of the lung infection using multi-class labeling. The best result
comes from the succession of tasks for performing the multi-class
segmentation. The use of the results of unit-class segmentation with the
original image leads to a precise segmentation of the lung infection. The
evaluation using different metric in Table 3 also demonstrate the advantage of
the proposed method compared with the other existing methods. For example, the
semi-inf-Net method succeed to obtain close results due to the multi-task
learning model. In contrast to the other model that are used as it is like
UNet of FC8s models.

Table 3: Quantitative results of infection regions using multi-class segmentation on COVID-SemiSeg DATASET  Method |  Dice |  Sensitivity |  Specitivity |  Precision |  Fmeasure |  MAE
---|---|---|---|---|---|---
multi-class U-Net [33] |  0.422 |  .0379 |  0.976 |  - |  - |  0.066
DeepLabV3+ [35] |  0.341 |  0.512 |  0.766 |  - |  - |  0.117
FC8s [34] |  0.375 |  0.403 |  0.811 |  - |  - |  0.076
Semi-Inf-Net [27] |  0.541 |  0.564 |  0.967 |  - |  - |  0.057
Propsed method |  0.640 |  0.630 |  0.953 |  0.561 |  0.640 |  0.062

##  5 Conclusion

In this paper, a lung infection segmentation method for COVID-19 has been
proposed. Based on encoder-decoder networks on CT-scan images exploit the
computer vision techniques to identify the lung infected regions for COVID-19
patients. Due to the shortage of data at this moment, multi-task learning has
been performed including the use of two-stream as inputs of the deep learning
model. Also, using computer vision features like structure and texture
component of the images that help for good extraction of the region of
interest that can contain infections has also been utilized. Different
segmentation has been performed including the binary segmentation and multi-
class segmentation of lung infected regions. comparing the proposed approach
with the state-of-the-art method, the experiment shows an accurate
segmentation of the lung infection region in both binary and multiclass
segmentation. The obtained results can be improved by using more data for
training and more labeled data for multi-class segmentation, which represents
our future works.

##  6 Acknowledgements

This publication was made by rEdicting RIsk earLy in COVID-19 project (QUERG-
CENG-2020-1). The statements made herein are solely the responsibility of the
authors.

##  7 Conflict interests

The authors declare that they have no competing interests.

## References

  * [1] NGUYEN, Thanh Thi. Artificial intelligence in the battle against coronavirus (COVID-19): a survey and future research directions. Preprint, DOI, 2020, vol. 10.
  * [2] ULHAQ, Anwaar, KHAN, Asim, GOMES, Douglas, et al. Computer Vision for COVID-19 Control: A Survey. arXiv preprint arXiv:2004.09420, 2020.
  * [3] LATIF, Siddique, USMAN, Muhammad, MANZOOR, Sanaullah, et al. Leveraging Data Science To Combat COVID-19: A Comprehensive Review. 2020.
  * [4] MOUJAHID, Driss, Elharrouss, Omar, et TAIRI, Hamid. Visual object tracking via the local soft cosine similarity. Pattern Recognition Letters, 2018, vol. 110, p. 79-85.
  * [5] O. Elharrouss, N. Almaadeed and S. Al-Maadeed, "LFR face dataset:Left-Front-Right dataset for pose-invariant face recognition in the wild," 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), Doha, Qatar, 2020, pp. 124-130, doi: 10.1109/ICIoT48696.2020.9089530.
  * [6] O. Elharrouss, N. Almaadeed and S. Al-Maadeed, "An image steganography approach based on k-least significant bits (k-LSB)," 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), Doha, Qatar, 2020, pp. 131-135, doi: 10.1109/ICIoT48696.2020.9089566.
  * [7] O. Elharrouss, N. Almaadeed, and S. Al-Maadeed, “Mhad:multi-human action dataset,”Advances in Intelligent Systemsand Computing, vol. 1041, pp. 333–341, 2020.
  * [8] KALKREUTH, Roman et KAUFMANN, Paul. COVID-19: A Survey on Public Medical Imaging Data Resources. arXiv preprint arXiv:2004.04569, 2020.
  * [9] OZTURK, Tulin, TALO, Muhammed, YILDIRIM, Eylul Azra, et al. Automated detection of COVID-19 cases using deep neural networks with X-ray images. Computers in Biology and Medicine, 2020, p. 103792.
  * [10] MINAEE, Shervin, KAFIEH, Rahele, SONKA, Milan, et al. Deep-covid: Predicting covid-19 from chest x-ray images using deep transfer learning. arXiv preprint arXiv:2004.09363, 2020.
  * [11] APOSTOLOPOULOS, Ioannis D., AZNAOURIDIS, Sokratis I., et TZANI, Mpesiana A. Extracting possibly representative COVID-19 Biomarkers from X-Ray images with Deep Learning approach and image data related to Pulmonary Diseases. Journal of Medical and Biological Engineering, 2020, p. 1-8.
  * [12] APOSTOLOPOULOS, Ioannis D. et MPESIANA, Tzani A. Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks. Physical and Engineering Sciences in Medicine, 2020, p. 1.
  * [13] ABBAS, Asmaa, ABDELSAMEA, Mohammed M., et GABER, Mohamed Medhat. Classification of COVID-19 in chest X-ray images using DeTraC deep convolutional neural network. arXiv preprint arXiv:2003.13815, 2020.
  * [14] COHEN, Joseph Paul, MORRISON, Paul, et DAO, Lan. COVID-19 image data collection. arXiv preprint arXiv:2003.11597, 2020.
  * [15] “COVID-19 Patients Lungs X Ray Images 10000,” https://www.kaggle. com/nabeelsajid917/covid-19-x-ray-10000-images, accessed: 2020-04-11.
  * [16] M. E. H. Chowdhury, T. Rahman et al., “Can AI help in screening Viral and COVID-19 pneumonia?” arXiv, 2020.
  * [17] J. Zhao, Y. Zhang, X. He, and P. Xie, “COVID-CT-Dataset: a CT scan dataset about COVID-19,” arXiv, 2020.
  * [18] “COVID-19 CT segmentation dataset,” https://medicalsegmentation. com/covid19/, accessed: 2020-04-11.
  * [19] ADHIKARI, Nimai Chand Das. Infection Severity Detection of CoVID19 from X-Rays and CT Scans Using Artificial Intelligence. International Journal of Computer (IJC), 2020, vol. 38, no 1, p. 73-92.
  * [20] WU, Yu-Huan, GAO, Shang-Hua, MEI, Jie, et al. JCS: An explainable COVID-19 diagnosis system by joint classification and segmentation. arXiv preprint arXiv:2004.07054, 2020.
  * [21] MOBINY, Aryan, CICALESE, Pietro Antonio, ZARE, Samira, et al. Radiologist-Level COVID-19 Detection Using CT Scans with Detail-Oriented Capsule Networks. arXiv preprint arXiv:2004.07407, 2020.
  * [22] POLSINELLI, Matteo, CINQUE, Luigi, et PLACIDI, Giuseppe. A Light CNN for detecting COVID-19 from CT scans of the chest. arXiv preprint arXiv:2004.12837, 2020.
  * [23] AL-KARAWI, Dhurgham, AL-ZAIDI, Shakir, POLUS, Nisreen, et al. Machine Learning Analysis of Chest CT Scan Images as a Complementary Digital Test of Coronavirus (COVID-19) Patients. medRxiv, 2020.
  * [24] HE, Xuehai, YANG, Xingyi, ZHANG, Shanghang, et al. Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans. medRxiv, 2020.
  * [25] AMYAR, Amine, MODZELEWSKI, Romain, et RUAN, Su. Multi-task Deep Learning Based CT Imaging Analysis For COVID-19: Classification and Segmentation. medRxiv, 2020.
  * [26] ANWAR, Talha et ZAKIR, Seemab. Deep learning based diagnosis of COVID-19 using chest CT-scan images. 2020.
  * [27] FAN, Deng-Ping, ZHOU, Tao, JI, Ge-Peng, et al. Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Images. IEEE Transactions on Medical Imaging, 2020.
  * [28] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in MICCAI. Springer, 2015, pp. 234–241.
  * [29] Z. Zhou, M. M. R. Siddiquee, N. Tajbakhsh, and J. Liang, “UNet++: A nested U-Net architecture for medical image segmentation,” IEEE Transactions on Medical Imaging, pp. 3–11, 2019.
  * [30] O. Oktay, J. Schlemper et al., “Attention U-Net: Learning Where to Look for the Pancreas,” in International Conference on Medical Imaging with Deep Learning, 2018.
  * [31] J. Schlemper, O. Oktay, M. Schaap, M. Heinrich, B. Kainz, B. Glocker, and D. Rueckert, “Attention gated networks: Learning to leverage salient regions in medical images,” Medical Image Analysis, vol. 53, pp. 197–207, 2019.
  * [32] X. Li, H. Chen, X. Qi, Q. Dou, C. Fu, and P. Heng, “H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes,” IEEE Transactions on Medical Imaging, vol. 37, no. 12, pp. 2663–2674, 2018.
  * [33] O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in MICCAI. Springer, 2015, pp. 234–241
  * [34] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in CVPR, 2015, pp. 3431–3440.
  * [35] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoderdecoder with atrous separable convolution for semantic image segmentation,” in ECCV, 2018, pp. 801–818.
  * [36] Jan-Mark Geusebroek, R. van den Boomgaard, A.W.M. Smeulders, A. Dev, Color and scale: the spatial structure of color images, in: Proceeding of the 6th European Conference on Computer Vision, vol. 1, 2000, pp. 331–341.
  * [37] O. Elharrouss, D. Moujahid, and H. Tairi. "Motion detection based on the combining of the background subtraction and the structure–texture decomposition". Optik-International Journal for Light and Electron Optics, 2015, vol. 126, no 24, p. 5992-5997.
  * [38] O. Elharrouss, A. Abbad, D. Moujahid, H. Tairi, Moving object detection zone using a block-based background model. IET Comput. Vis.12(1), 86–94 (2017).
  * [39] Long, J., Shelhamer, E., Darrell, T., 2015. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, pp. 3431–3440
  * [40] Badrinarayanan, V., Kendall, A. and Cipolla, R., 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12), pp.2481-2495.
  * [41] SIMONYAN, Karen et ZISSERMAN, Andrew. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  * [42] Ioffe, S., Szegedy, C., 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, Lille, France, pp. 448–456.
  * [43] SHI, Feng, XIA, Liming, SHAN, Fei, et al. Large-scale screening of covid-19 from community acquired pneumonia using infection size-aware classification. arXiv preprint arXiv:2003.09860, 2020.
  * [44] Shan, Fei, Yaozong Gao, Jun Wang, Weiya Shi, Nannan Shi, Miaofei Han, Zhong Xue, and Yuxin Shi. "Lung infection quantification of covid-19 in ct images with deep learning." arXiv preprint arXiv:2003.04655 (2020).
  * [45] Elharrouss, Omar, MOUJAHID, Driss, ELKAH, Samah, et al. Moving object detection using a background modeling based on entropy theory and quad-tree decomposition. Journal of Electronic Imaging, 2016, vol. 25, no 6, p. 061615.
  * [46] Thanh, D. N. H., et al. “BLOOD VESSELS SEGMENTATION METHOD FOR RETINAL FUNDUS IMAGES BASED ON ADAPTIVE PRINCIPAL CURVATURE AND IMAGE DERIVATIVE OPERATORS.” ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XLII-2/W12, Copernicus GmbH, May 2019, pp. 211–18, doi:10.5194/isprs-archives-xlii-2-w12-211-2019.
  * [47] Khan, M.A., Khan, T.M., Soomro, T.A., Mir, N. and Gao, J., 2019. Boosting sensitivity of a retinal vessel segmentation algorithm. Pattern Analysis and Applications, 22(2), pp.583-599.
  * [48] O. Elharrouss, A. Abbad, D. Moujahid, J. Riffi, H. Tairi, A block-based background modelfor moving object detection. ELCVIA: Electronic Letters on Computer Vision and ImageAnalysis.15(3), 0017–31 (2016), DOI: https://doi.org/10.5565/rev/elcvia.855.
  * [49] Elharrouss O, Moujahid D, Elkah S, Tairi H. "Moving object detection using a background modeling based on entropy theory and quad-tree decomposition". J. Electron. Imaging, 2016, vol. 25, no 6 (061615).
  * [50] O. Elharrouss, N. Almaadeed, S. Al-Maadeed, and Y. Akbari, Image inpainting: A review. Neural Processing Letters, 2019, p. 1-22.