# CVR-Net: A deep convolutional neural network for coronavirus recognition
from chest radiography images

Md. Kamrul Hasan11footnotemark: 1 Department of Electrical and Electronic
Engineering (EEE<EMAIL_ADDRESS>Khulna University of Engineering &
Technology (KUET)  Md. Ashraful Alam<EMAIL_ADDRESS>Md. Toufick E
Elahi<EMAIL_ADDRESS>Shidhartho Roy<EMAIL_ADDRESS>Sifat
Redwan Wahid<EMAIL_ADDRESS>
###### Abstract

### Background and Objective

The novel Coronavirus Disease $2019$ (COVID-$19$) is a global pandemic disease
spreading rapidly around the world. A robust and automatic early recognition
of COVID-$19$, via auxiliary Computer-aided Diagnostic (CAD) tools, is
essential for disease cure and control. The Artificial Intelligence (AI)
assisted CAD system can be significant tool for the chest radiography images
such as Computed Tomography (CT) and X-rays.  However, designing such an
automated tool is challenging as a massive number of manually annotated
datasets are not publicly available yet, which is the core requirement of
supervised learning systems.

### Methods

In this article, we propose a robust CNN-based network, called CVR-Net
(Coronavirus Recognition Network), for the automatic recognition of the
coronavirus from CT or X-ray images. The proposed end-to-end CVR-Net is a
multi-scale-multi-encoder ensemble model, where we have aggregated the outputs
from two different encoders and their different scales to obtain the final
prediction probability. We train and test the proposed CVR-Net on three
different datasets, where the images have collected from different open-source
repositories. We compare our proposed CVR-Net with state-of-the-art methods,
which are trained and tested on the same datasets.

### Results

We split three datasets into five different tasks, where each task has a
different number of classes, to evaluate the multi-tasking CVR-Net. Our model
achieves an overall F1-score & accuracy of $0.997$ & $0.998$; $0.963$ &
$0.964$; $0.816$ & $0.820$; $0.961$ & $0.961$; and $0.780$ & $0.780$,
respectively, for task-$1$ to task-$5$.

### Conclusion

As the CVR-Net provides promising results on the small datasets, it can be an
auspicious CAD tool for the diagnosis of coronavirus to assist the clinical
practitioners and radiologists. Our source codes and model are publicly
available (https://github.com/kamruleee51/CVR-Net) for the research community
for further improvements.

###### keywords:

Coronavirus disease , Chest computed tomography and X-ray , Convolutional
neural networks , Ensembling classifier , CAD tools.

††journal: arXivlabel4label4footnotetext: Department of EEE, KUET,
Khulna-9203, Bangladesh.

##  1 Introduction

###  1.1 Problem Presentation and Motivation

A pneumonia of unknown cause detected in Wuhan, China was reported to the
World Health Organization(WHO) office in China on 31 December, 2019 which was
subsequently named severe acute respiratory syndrome coronavirus 2 (SARS-
CoV-2) on 11 February,2020 as the virus causing the disease is genetically
related to the corona virus responsible for the SARS outbreak of 2003. The new
disease was named as COVID-19 by WHO on 11 February 2020 (World Health
Organization, 2020a). As of July $2020$, the outbreak of $2019$ in Wuhan
(China), has extended worldwide (Zhu et al., 2020; Li et al., 2020) with
$12,286,264$ confirmed COVID-$19$ cases including $555,642$ deaths in a short
period of 6 months (11 July 2020) (World Health Organization, 2020b), as
presented in Fig. 1.

(a)  Figure 1: A world heat map of the corona pandemic per capita (COVID, 19)
[Accessed on $14$ July, $2020$].

The clinical attributes, of severe COVID-$19$ epidemic, are bronchopneumonia
that causes cough, fever, dyspnea, and subtle respiratory anxiety ailment
(Wang et al., 2020a; Chen et al., 2020; Li et al., 2020). A person with fever,
cough, and influenza symptoms, is usually screened by the conventional methods
of clinical trials, laboratory testing, and chest radiography to rule out
pneumonia. Reverse Transcription Polymerase Chain Reaction (RT-PCR) is a
commonly employed clinical screening test for COVID-$19$ infection, using
respiratory specimens. Generally, radiologists apply the RT-PCR test as a base
tool for detecting coronavirus, but it is a manual, complicated, tedious, and
time-consuming procedure with a true-positive rate of $63.0\%$ (Wang et al.,
2020b). There is a significant lack of inventory, leading to a delay in
efforts to prevent and cure coronavirus disease (Yang et al., 2020). Many
countries face crises with the incorrect number of positive COVID-$19$ cases
due not only to the lack of test kits but also to the defer in the test
results (A. J. NEWS, 2020b). Such delays may cause affected patients to
interact with and affect healthy people in the process. Furthermore, the RT-
PCR kit is estimated to cost around $120\sim 130$ USD, and requires a
specially designed biosafety laboratory to house the PCR unit, each of which
can cost $15,000\sim 90,000$ USD (A. J. NEWS, 2020a). Nevertheless, using a
costly screening device with delayed test results allows it to spread and
worsen the situation. This scenario is a problem for low-income countries, but
some developed countries also struggle to alleviate that limitation (Wetsman,
2020).

It is observed that most of the COVID-$19$ cases have common characteristics
on radiographic images, such as CT and X-ray, including bilateral, multi-
focal, ground-glass opacities with a peripheral or posterior distribution,
mainly in the lower lobes, early- and late-stage pulmonary consolidation
(Huang et al., 2020; Corman et al., 2020; Xu et al., 2020; Singh et al.,
2020). Those images can be utilized to develop a sensitive CAD tool to detect
COVID-$19$ pneumonia and be considered a screening tool with RT-PRC (Lee et
al., 2020). Application of machine learning methods for automatic diagnosis in
the medical field, via CAD tools, has recently gained popularity by becoming
an adjunct tool for the clinicians (Litjens et al., 2017; Ker et al., 2017;
Shen et al., 2017). The deep CNN-based published methods, for automatic
coronavirus recognition, are briefly presented in the next subsection.

###  1.2 Recent Methods

Ozturk et al. (2020) proposed a model, called DarkCovidNet having $17$
convolutional layers, for binary classification (COVID vs No-Findings) and
multi-class classification (COVID vs No-Findings vs Pneumonia) using chest
X-ray images. The author employed the DarkNet (Redmon and Farhadi, 2017) model
for the You Only Look Once (YOLO) (Redmon and Farhadi, 2017) real-time
coronavirus detection system. A Deep CNN model, called CoroNet, automatically
detect COVID-$19$ infection from chest X-ray images, which was proposed by
Khan et al. (2020). The CoroNet was based on pre-trained Xception architecture
(Chollet, 2017) in ImageNet (Deng et al., 2009), with a dropout layer
(Srivastava et al., 2014) and two fully-connected layers. They trained and
evaluated their model on different tasks like binary and multi-class
recognition. Ghoshal and Tucker (2020) investigated uncertainty of the
coronavirus classification report, using the drop-weights-based Bayesian CNN,
as the availability of uncertainty-aware Deep Learning (DL) can ensure more
extensive adoption of DL in clinical applications. Narin et al. (2020)
implemented three different deep CNN models such as ResNet-$50$ (He et al.,
2016a), InceptionV3 (Szegedy et al., 2017), and Inception-ResNetV$2$ (Szegedy
et al., 2017), where they also used transfer learning for the detection of
coronavirus and pneumonia infected patient. The authors showed that chest
X-ray images and ResNet-$50$ are the best tools for the detection of
COVID-$19$. Hemdan et al. (2020) proposed a DL framework, called COVIDX-Net,
where they experimented on seven different CNN architectures, such as VGG-$19$
(Simonyan and Zisserman, 2014), DenseNet-$121$ (Huang et al., 2017),
ResNetV$2$ (He et al., 2016b), InceptionV$3$, Inception-ResNetV$2$, Xception,
and MobileNetV$2$ (Howard et al., 2017). In the end, their results suggested
that VGG-$19$ and DenseNet-$201$ are better for coronavirus screening systems
with X-ray images. Three comparatively shallow networks, such as
MobileNetV$2$, SqueezeNet (LeCun et al., 2010), and ResNet-$18$, and five deep
networks, such as InceptionV$3$, ResNet-$101$, CheXNet (Rajpurkar et al.,
2018), VGG-$19$, and DenseNet-$201$, were trained and evaluated by Chowdhury
et al. (2020) for coronavirus recognition. Their study experimentally showed
that DenseNet-$201$ outperformed other deep CNN networks, while the authors
trained their model with different image augmentations. Abbas et al. (2020a)
proposed a framework by adopting a deep CNN, called Decompose, Transfer, and
Compose (DeTraC) (Abbas et al., 2020b) for the classification of COVID-$19$
chest X-ray images, where the authors implemented the DeTraC in two phases.
Firstly, they trained, using a gradient descent optimization, the backbone
pre-trained CNN model of DeTraC to extract deep local features from each
image. Secondly, they used the class-composition layer of DeTraC to refine the
final classification of the images. Zhao et al. (2020) developed diagnosis
methods based on multi-task learning and self-supervised learning, where the
authors proposed an open-source COVID dataset of CT images with a binary class
(COVID and NON-COVID). For the classification task, they train DenseNet-$169$
and ResNet-$50$, via a pre-trained model on ImageNet weights, with their newly
proposed dataset. Hall et al. (2020) explored the usefulness of the chest
X-ray images with various deep CNN models for the diagnosis of the COVID-$19$
disease. The authors employed pre-trained ResNet-$50$ and VGG-$16$ (Simonyan
and Zisserman, 2014) on ImageNet. Afshar et al. (2020) proposed a CNN model
named COVID-CAPS, which was based on the Capsule Networks (CapsNets) for
handling the small datasets of coronavirus. CapsNets are alternative models of
CNN, which are capable of capturing spatial information using routing by
agreement, through which capsules try to reach a mutual agreement on the
existence of the objects. Their proposed COVID-CAPS model had $4$
convolutional layers and $3$ capsule layers, where batch normalization (Ioffe
and Szegedy, 2015) followed the former layers. The authors fine-tuned all the
capsule layers, while the conventional layers were frozen with pre-trained
weights of ImageNet. Apostolopoulos and Mpesiana (2020) accomplished
comprehensive experiments on state-of-the-art CNN models applying transfer
learning. In the end, the authors found that VGG-$19$ outperforms other CNNs
for accuracy, while MobileNetv$2$ outperforms VGG-$19$ in terms of
specificity. He et al. (2020a) built a COVID CT dataset, called China
Consortium of Chest CT Image Investigation (CC-CCII), with three classes:
novel coronavirus pneumonia, common pneumonia, and healthy controls. The
authors trained $3$D DenseNet3D-$121$ on their proposed CC-CCII dataset, and
they experimentally validated that $3$D CNNs outperform $2$D CNNs in general.
Singh et al. (2020) implemented a CNN-based model named multi-objective
differential evolution–based CNN for the classification of COVID-$19$. They
fine-tuned the parameters of the CNN model using a multi-objective fitness
function. The differential evolution algorithm was used for the optimization
of the multi-objective fitness function. In differential evolution, the model
was optimized iteratively using mutation, crossover, and selection operation
to determine the best available solution. Apostolopoulos et al. (2020)
extracted massive high-dimensional features, using a pre-trained MobileNetV$2$
architecture, corresponding to six diseases. Finally, they used fully-
connected layers to classify those features for the identification of the
coronavirus. Farooq and Hafeez (2020) employed ResNet-$50$ using transfer
learning with progressively resizing the input images to $128\times 128\times
3$, $224\times 224\times 3$, and $229\times 229\times 3$ pixels, where the
authors also fine-tuned the network at each stage. Ozkaya et al. (2020)
extracted deep features using VGG-$16$, GoogleNet (Szegedy et al., 2015), and
ResNet-$50$ models, which were classified by Support Vector Machine (SVM)
(Furey et al., 2000) with linear kernel function. They also applied the t-test
method to reduce the feature dimension for reducing the overfitting. Rajaraman
et al. (2020) evaluated ImageNet pre-trained CNN models such as VGG-$16$,
VGG-$19$, InceptionV$3$, Xception, Inception-ResNetV$2$, MobileNetV$2$,
DenseNet-$201$, and NasNet-mobile (Pham et al., 2018). Then, they optimized
the hyperparameters of the CNNs using a randomized grid search method
(Bergstra and Bengio, 2012). In the end, the authors proposed an ensemble of
those CNN models for the final coronavirus recognition. Toğaçar et al. (2020)
restructured the data classes using a fuzzy color technique, where they
stacked a structured image with the original images. The authors trained
MobileNetV$2$ and SqueezeNet to extract the deep features, which were then
processed using the social mimic optimization method (Balochian and
Baloochian, 2019). After that, selected features were combined and classified
using the SVM for the recognition of coronavirus.

###  1.3 Our Contribution

The discussions mentioned above, on automated coronavirus recognition systems,
show that the deep CNN approaches are commonly employed methods than the
different systems that rely on handcrafted features. The former approaches
provide auspicious reproducibility of results and amplify the speed of the
diagnosis while being end-to-end methods. While many approaches have already
been developed and implemented for coronavirus recognition, there is still
room for performance improvement for different datasets. However, it is very
impractical to guesstimate the amount of depth of the CNN networks and the
times of subsampling, when utilizing datasets are small in size. In this
article, we propose an end-to-end coronavirus recognition network called CVR-
Net, where we ensemble different scaled feature maps from different encoders
through fully-connected layers. Such an ensembling allows the network to
access different depths and scales of the feature maps of different encoders
for generating the final prediction. In our CVR-Net, the newly added depth and
subsampling can not degrade the final prediction as their previous depths and
subsampling compensate them. To overcome the overfitting and build a generic
CVR-Net, with the limited datasets, we apply geometry-based image
augmentations and transfer learning on ImageNet (Krizhevsky et al., 2012).
Besides, we also rebalance the imbalanced class distribution, as a massive
number of positive coronavirus images are not available yet due to the recent
COVID-$19$ pandemic. We validate our multi-tasking CVR-Net on three different
datasets, of two different modalities, such as CT and X-ray, with a different
number of classes (see in Table 1). We have collected images from different
open-sources, such as Kaggle, GitHub, and grand challenges (see in subsection
2.1). To our best knowledge, the proposed CVR-Net has achieved state-of-the-
art results on three different datasets, having a different number of classes,
while being an end-to-end coronavirus diagnosis system.

The rest of the paper is structured accordingly. We explain the proposed
framework for the recognition of coronavirus and datasets in section 2. The
results and different experiments are reported in section 3. We interpret the
obtained results from the proposed CVR-Net in section 4. Finally, we conclude
this paper in section 5.

##  2 Materials and Methods

This section presents the materials and methods for conducting this research.
Subsection 2.1 briefly describes utilized dataset. The designing of the
proposed network (CVR-Net) is explained in subsection 2.2. Finally, subsection
2.3 describes the training protocol of our network and the evaluation metrics.

###  2.1 Dataset and Hardware

We train and evaluate our multi-tasking CVR-Net on three datasets, CT and
X-ray images, with a different number of classes. We evaluate the proposed
network on three different types of tasks, such as: healthy vs. coronavirus
($2$-class), healthy vs. pneumonia vs. coronavirus ($3$-class), and healthy
vs. bacterial pneumonia vs. viral pneumonia vs. coronavirus ($4$-class). As
COVID-19 is the recent pandemic all over the world, there is still a lack of
suitable annotated public datasets as it requires experts for labeling.
However, we collected the positive COVID-19 X-ray images from an open-source
GitHub222https://github.com/ieee8023/covid-chestxray-dataset repository of
Cohen et al. (2020), the authors compiled the images from various authentic
sources (Radiological Society of North America (RSNA), Radiopaedia, etc). We
collect the Pneumonia (both bacterial & viral) and healthy chest X-ray images
from Kaggle repository Chest X-Ray Images (Pneumonia) (Paul Mooney, 2018).
These two datasets are merged for dataset-$1$, as utilized in CoroNet by Khan
et al. (2020), which has three different tasks, as presented in Table 1. The
utilized dataset-$2$ is the combination of dataset-$1$ and additional images
from another Kaggle repository Pneumonia sample X-Rays (Ahmed Ali, 2020), as
it was utilized by Toğaçar et al. (2020). Finally, the dataset-$3$ is
collected from COVID-$19$ grand challenges (Zhao et al., 2020), which is the
CT images of COVID and healthy patients and collected from Tongji Hospital,
Wuhan, China. The distribution of all the three datasets is presented in Table
1, where we assign three different tasks, with a different number of classes,
for dataset-$1$ and single task for the other two datasets.

Table 1: Utilized data distribution of five different tasks, of three datasets, to validate our proposed CVR-Net, where data was accumulated from different open-sources. Datasets | Task Types | Class categories | No. of Images
---|---|---|---
|  | Normal (NOR) | $5,856$
| Task-1: 2-class | Novel Corona Positive (NCP) | $500$
|  | Normal (NOR) | $1,583$
|  | Common Pneumonia (CPN) | $4,273$
| Task-2: 3-class | Novel Corona Positive (NCP) | $500$
|  | Normal (NOR) | $1,583$
|  | Common Pneumonia Bacterial (CPB) | $2780$
|  | Common Pneumonia Viral (CPV) | $1493$
Dataset-1 | Task-3: 4-class | Novel Corona Positive (NCP) | $500$
|  | Normal (NOR) | $1,648$
|  | Common Pneumonia (CPN) | $4,371$
Dataset-2 | Task-4: 3-class | Novel Corona Positive (NCP) | $500$
|  | Normal (NOR) |  Train/test = $292/105$
Dataset-3 | Task-5: 2-class | Novel Corona Positive (NCP) |  Train/test = $251/98$

Several example of CT and X-ray images for different classes is presented in
Fig. 2.

(a)

(b)

(c)

(d)

(e)

(f)

Figure 2: Samples of chest radiography images from the utilized datasets (a)
Normal (X-ray), (b) Normal (CT), (c) Pneumonia viral, (d) Pneumonia bacterial,
(e) COVID-19 (X-ray), and (f) COVID-19 (CT).

We have applied a $5$-folds cross-validation technique, as presented in Fig.
3, for the first two datasets to select training, validation, and testing
images.

Figure 3: The partitioning of the datasets into five parts for selecting
training, validation, and testing images.

The class-distribution of all the datasets, as shown in Table 1, demonstrates
that the images are imbalanced, which makes the classifier to be biased to the
particular class having more samples. However, we have employed class
rebalancing techniques by penalizing the majority class’s loss to build a
generic classifier even though datasets are imbalanced.

The models were implemented using the Python programming language with
different Python and Keras APIs (Chollet, 2015) and the experiments were
carried out on a machine running Windows-10 operating system with the
following hardware configuration: Intel® CoreTM i$7$-$7700$ HQ CPU @
$3.60\,GHz$ processor with Install memory (RAM): $32.0\,GB$ and GeForce GTX
$1080$ GPU with $8\,GB$ memory.

###  2.2 Proposed CVR-Net Architecture

Efficaciously classifications, of medical images, have an essential role in
aiding clinical cure and control. For instance, an X-ray investigation for
diagnosing pneumonia is the best approach (Organization et al., 2001), but it
needs professional radiologists or experts, which is a rare, costly, and
arduous for some regions. The employment of the conventional machine learning
algorithms, in medical image classification, began long ago (Yadav and Jadhav,
2019), which has several disadvantages, such as the poor performance than the
practical standard; the implementation of them is quite slow; the extraction
and selection of the features are time-consuming and tedious; fluctuate a lot
according to various objects (Kermany et al., 2018). The deep neural networks,
notably the CNNs, are comprehensively applied for image classification or
recognition, which have earned powerful performance since $2012$ (Rawat and
Wang, 2017). Recently, CNN-based medical image classification rivals human
expertise. For instance, CheXNet, a CNN classifier trained on a chest X-rays
dataset with more than $100,000$ frontal-views, achieved better results than
the average performance of four experts. Moreover, Kermany et al. (2018)
proposed a CNN-based classifier, with a transfer learning, to recognize
$108,309$ optical coherence tomography images, where the average error, from
the CNN model, was equivalent the errors from $6$ different human experts.
Currently, the automatic recognition of coronavirus is one of the critical
topics for the researcher, where images are hard to accumulate, as the
collection and annotation of COVID-$19$ data are time-consuming, costly, and
required expert explanations.

However, designing an end-to-end recognition system is a challenging task as
the CNNs may be indirectly limited when employed with highly variable and
distinctive image datasets with limited samples such as COVID datasets.
Moreover, individual CNN architecture may have different capabilities to
characterize or represent the image data, which is often linked to a network’s
depth (Kumar et al., 2016). The number of layers with increasing depth and
amount of subsampling (a downsampling in pooling layers) is also challenging
to guesstimate with the limited datasets. However, in this context, we propose
a CNN-based end-to-end multi-tasking network, where we apply multi-encoder and
multi-scale ensembling, as depicted in Fig. 4.

(a)  Figure 4: The proposed network, called CVR-Net, for the automatic
coronavirus recognition from radiography images, where we ensemble the multi-
encoder and multi-scale of the network, via fully connected blocks, obtain
final recognition probability.

The proposed CVR-Net consists of two encoders, with the same input images,
where each of the encoders has five blocks, namely $E_{1n}$ and $E_{2n}$,
$n=1,2,...,5$, for encoder-$1$ and encoder-$2$, respectively. The encoder-$1$
consists of the residual and convolutional blocks (He et al., 2016a), as
presented in Fig. 5, where the residual connections allow the information to
flow or skip.

(a)  Figure 5: The convolutional (left) and residual (right) blocks (He et
al., 2016a) of the proposed CVR-Net, where the output map is the summation of
the input map and the generated map from the process (convolutions).

The residual connections, also known as skip connections, allow gradients to
flow through a network directly, without passing through non-linear activation
functions and thus avoiding the problem of vanishing gradients in the proposed
CVR-Net (He et al., 2016a). In residual connections, the output of a weight
layer series is added to the original input and then passed through the non-
linear activation function, as shown in Fig. 5. However, in encoder-$1$,
$7\times 7$ input convolution, followed by max-pooling with the stride of $2$,
and pool size of $3\times 3$, is used before identity and convolutional
blocks. By stacking these blocks on top of each other (see Fig. 4), an
encoder-$1$ has formed to get the feature map, where the notation ($n\times$)
under the identity block denotes the number of repetitions ($n$ times). The
different blocks, of encoder-$1$ ($E_{1n}$ and $n=1,2,...,5$), downsample the
input image resolutions in half of the input resolutions, while the resolution
inside the blocks is kept constant. The outputs of those blocks generate the
feature maps with different scales. Within the encoder-$2$, three components,
of information flow blocks, are used, which were initially proposed by Chollet
(2017), such as entry flow, middle flow, and exit flow, as depicted in Fig. 4.
The batch, of input images, firstly passes through the input flow, then the
central flow, eight times ($8\times$) repeated, and finally through the exit
flow. All flows, as in the proposed network (see in Fig. 4), have Depth-wise
Separable Convolution (DwSC) (Chollet, 2017) and residual connections. The
former one has used to create a lightweight network, while the latter has the
advantages discussed earlier. Thus, by utilizing two different types of
encoders, we can learn two types of feature maps from the same input images.
However, the different blocks of encoder-$2$ ($E_{2n}$ and $n=1,2,...,5$),
downsample the input image resolutions in half of the input resolutions, while
the resolution inside the blocks is kept constant. The outputs of those blocks
also generate feature maps with different scales. However, two different $2$D
feature maps of different encoders are concatenated, in channel-wise, to
enhance the depth information of the feature map. We use differently scaled
feature maps to build the proposed CVR-Net, where each feature map is passed
through the Fully Connected Layer (FCL) block. A Global Average Pooling (GAP)
(Lin et al., 2013) layer and four fully connected layers are used in our FCL
block, where the GAP layer performs an extreme dimensionality reduction to
avoid overfitting. An $height\times weight\times depth$ dimensional tensor, in
GAP, is reduced to a $1\times 1\times depth$ vector by transferring
$height\times width$ feature map to a single number, which contributes to the
lightweight design of the proposed CVR-Net. Table 2 presents the
implementational details of the proposed CVR-Net.

Table 2: Details of the proposed CVR-Net have used feature maps, shapes, and the number of parameters, where the input resolution is $M\times N$ pixels. Feature block | Shape of features | Prediction | Parameters
---|---|---|---
$E_{13}$ | $\frac{M}{8}\times\frac{N}{8}\times 512$ | $P_{1}=FCL(E_{13})$ |  | $1,796,867$
---
$E_{14}$ | $\frac{M}{16}\times\frac{N}{16}\times 1024$ | $P_{2}=FCL(E_{14})$ |  | $9,181,827$
---
$[E_{15}+\\!\\!\\!\\!+\,E_{25}]$ | $\frac{M}{32}\times\frac{N}{32}\times 4096$ | $P_{3}=FCL([E_{15}+\\!\\!\\!\\!+\,E_{25}])$ |  | $46,620,971$
---
$E_{23}$ | $\frac{M}{8}\times\frac{N}{8}\times 512$ | $P_{4}=FCL(E_{23})$ |  | $1,371,131$
---
$E_{24}$ | $\frac{M}{16}\times\frac{N}{16}\times 1024$ | $P_{5}=FCL(E_{24})$ |  | $15,954,283$
---
Proposed CVR-Net | $P=Avg(P_{1}\sim P_{5})$ | $48,596,087$

We utilize the feature maps $E_{13}\sim E_{15}$ from encoder-$1$ and
$E_{23}\sim E_{25}$ from encoder-$2$, where we concatenate $E_{15}$ and
$E_{25}$ to increase the depth of the feature information. The final
prediction, in CVR-Net, is the average of different probabilities, such as
$P_{1}$, $P_{2}$, $P_{3}$, $P_{4}$, and $P_{5}$ respectively for $E_{13}$,
$E_{14}$, $[E_{15}+\\!\\!\\!\\!+\,E_{25}]$, $E_{23}$, and $E_{24}$, which was
trained end-to-end fashion. However, designing of such a multi-encoder and
multi-scale network, as CVR-Net, has several benefits, especially for the
small datasets, such as: if one encoder fails to generate responsible
features, another encoder can compensate it and vice-versa; if the feature
quality is reduced in the deeper blocks (lower resolution), the prior blocks
(higher resolution) can also compensate it and vice-versa; if one or more $P$
predicts wrong class, other $P$ can overcome it, as the final result is
average of all $P$’s. Another positive prospect of the CVR-Net is that during
the training, it can be anticipated that if the gradient of one or more
branches vanishes, then other branches can recover it as the final gradient is
the average of all the individual gradient.

###  2.3 Training Protocol and Evaluation

#### Training Protocol

The preprocessing of the input images is the crucial requirement of the deep
CNN models, as images serve as fuel in the learning of those models. However,
as a preprocessing, we apply image augmentations, class rebalancing, and
resizing. The supervised learning systems, in medical imaging domains, suffer
from the limited size of the datasets, which is one reason for the overfitted
model. The data augmentations can partly overcome such overfitting, where
augmentations utilize either data warping or oversampling for augmenting the
training dataset synthetically (Shorten and Khoshgoftaar, 2019). In this
research, we apply different geometric transformations, as an augmentation,
such as rotation, height & width shift, and horizontal & vertical flipping.
Imbalanced class distribution is another common phenomenon in the medical
imaging domain, where the positive class is underrepresented compared to other
classes, as the manual annotation is expensive. However, to alleviate this
problem, we penalize the majority class by weighting the loss function, where
such a weighting pays more attention to samples from the underrepresented
class. We estimate the weights of each class by $W_{j}=N_{j}/N$, where
$W_{j}$, $N$, and $N_{j}$ are the weight for class $j$, the total number of
samples, and the number of samples in class $j$, respectively. As we noticed
that most of the images of all the datasets have a $1:1$ aspect ratio, all
images were resized to $224\times 224$ pixels using nearest-neighbor
interpolation. Additionally, the scarcity of relatively small medical image
datasets has been partially overcome by employing a transfer learning (Shin et
al., 2016; Tajbakhsh et al., 2016) where the previously trained model, so-
called pre-trained model, is used to initialize the kernels rather than random
initialization. We utilize the ImageNet (Krizhevsky et al., 2012) pre-trained
weights to initialize the kernels of the proposed CVR-Net. We employ
categorical cross-entropy as a loss function and accuracy as a metric for
training our CVR-Net for all the datasets. The loss function has been
optimized using the Adam (Kingma and Ba, 2014) optimizer with initial learning
rate ($LR$), exponential decay rates ($\beta{{}_{1}},\,\beta{{}_{2}}$) as
$LR=0.0001$, $\beta{{}_{1}}=0.9$, and $\beta{{}_{2}}=0.999$ respectively
without AMSGrad variant. The initial learning rate is reduced after $12$
epochs by $10.0\,\%$ if validation loss stops improving.

#### Evaluation

We use different metrics, such as recall, precision, F1-score, and accuracy,
to evaluate our multi-tasking CVR-Net for coronavirus recognition, which are
mathematically defined as follows:

$Recall=\frac{TP}{TP+FN}$

$Precision=\frac{TP}{TP+FP}$

$F1-score=\frac{2\times TP}{2\times TP+FN+FP}$

$Accuracy=\frac{TP+TN}{TP+FN+FP+TN}$

where the TP, FN, FP, and TN respectively denote true positive (patient with
coronavirus symptoms recognized as the positive patient), false negative
(patient with coronavirus symptoms recognized as the negative patient), false
positive (patient without coronavirus symptoms recognized as the positive
patient), and true negative (patient without coronavirus symptoms recognized
as the negative patient). The recall quantifies the type-II error (the
patient, with the positive syndromes, inappropriately fails to be nullified),
and precision quantifies the positive predictive values (percentage of truly
positive recognition among all the positive recognition). The F$1$-score
indicates the harmonic mean of recall and precision, which shows the trade-off
between them. Accuracy quantifies the fraction of correct predictions (both
positive and negative).

##  3 Experiments and Results

At the beginning of this section, we present the quantitative results for
coronavirus recognition, applying the proposed CVR-Net with different datasets
having a different number of classes. Finally, in the end, we compare our
multi-tasking results with several recent state-of-the-art results, on the
same datasets, to validate our proposal.

Table 3 presents all the quantitative results, for the coronavirus
recognition, utilizing different datasets and our proposed CVR-Net. The
evaluation metrics, for each fold of each task, with average values are
reported, in Table 3, for evaluating the inter-fold variations.

Table 3: Coronavirus recognition results, applying the proposed CVR-Net, from different comprehensive experiments on the different datasets with different tasks.  |  |  | Metrics
---|---|---|---
Datasets | Task types | Folds | Recall | Precision | F1-score | Accuracy
|  | Fold-1 | $0.995$ | $0.995$ | $0.995$ | $0.998$
|  | Fold-2 | $0.998$ | $0.998$ | $0.998$ | $0.998$
|  | Fold-3 | $0.996$ | $0.996$ | $0.996$ | $0.996$
|  | Fold-4 | $0.998$ | $0.998$ | $0.998$ | $0.998$
|  | Fold-5 | $0.998$ | $0.998$ | $0.998$ | $0.998$
| Task-1: 2-class | Average | $0.997\pm 0.001$ | $0.997\pm 0.001$ | $0.997\pm 0.001$ | $0.998\pm 0.001$
|  | Fold-1 | $0.964$ | $0.964$ | $0.963$ | $0.964$
|  | Fold-2 | $0.966$ | $0.966$ | $0.966$ | $0.966$
|  | Fold-3 | $0.955$ | $0.955$ | $0.955$ | $0.955$
|  | Fold-4 | $0.969$ | $0.968$ | $0.968$ | $0.969$
|  | Fold-5 | $0.965$ | $0.964$ | $0.964$ | $0.965$
| Task-2: 3-class | Average | $0.964\pm 0.005$ | $0.963\pm 0.004$ | $0.963\pm 0.004$ | $0.964\pm 0.005$
|  | Fold-1 | 0.811 | $0.808$ | $0.808$ | $0.811$
|  | Fold-2 | $0.818$ | $0.813$ | $0.813$ | $0.818$
|  | Fold-3 | $0.823$ | $0.819$ | $0.819$ | $0.823$
|  | Fold-4 | $0.817$ | $0.815$ | $0.815$ | $0.817$
|  | Fold-5 | $0.831$ | $0.827$ | $0.826$ | $0.831$
Dataset-1 | Task-3: 4-class | Average | $0.820\pm 0.007$ | $0.816\pm 0.006$ | $0.816\pm 0.006$ | $0.820\pm 0.007$
|  | Fold-1 | $0.972$ | $0.972$ | $0.972$ | $0.972$
|  | Fold-2 | $0.960$ | $0.961$ | $0.960$ | $0.960$
|  | Fold-3 | $0.965$ | $0.965$ | $0.965$ | $0.965$
|  | Fold-4 | $0.952$ | $0.952$ | $0.951$ | $0.952$
|  | Fold-5 | $0.955$ | $0.955$ | $0.955$ | $0.955$
Dataset-3 | Task-4: 3-class | Average | $0.961\pm 0.007$ | $0.961\pm 0.007$ | $0.961\pm 0.007$ | $0.961\pm 0.007$
Dataset-5 | Task-5: 2-class | - | $0.780$ | $0.780$ | $0.780$ | $0.780$

#### Experiment-$1$

We have trained and evaluated the proposed CVR-Net on binary classification
problem (Task-$1$), with $5856$-negative (NOR) and $500$-positive (NCP)
images, applying $5$-fold cross-validations. For Task-$1$, the proposed CVR-
Net achieved an average accuracy of $99.8\,\%$ on dataset-$1$, while the
average recall, precision, and F1-score are $99.7\,\%$, $99.7\,\%$, and
$99.7\,\%$, respectively. The binary results, as presented in Table 3,
demonstrate that $99.7\,\%$ NCP images are correctly recognized as NCP, while
the positive predictive value is also $99.7\,\%$. The inter-fold variations,
for all the metrics, are also as small as $0.1\,\%$, which discloses
commendable robustness of the proposed CVR-Net. The details class-wise
results, from the proposed CVR-Net, are presented in a confusion matrix in
Table 4.

Table 4: Confusion matrix of Task-$1$ ($2$-class) of Dataset-$1$ employing the proposed CVR-Net, where the samples are the summation from each fold. | Actual
---|---
| NOR | NCP
| NOR |  | $5848$
---
$99.86\,\%$
| $7$
---
$1.40\,\%$
Predicted | NCP |  | $8$
---
$0.14\,\%$
| $493$
---
$98.60\,\%$

It exposes that out of $500$-NCP samples, the proposed model successfully can
recognize $493$ samples as NCP, while only $7$ samples are predicted as NOR
(false negative). Table 4 also reveals that the power of a binary test
(probability of rejecting the null hypothesis) is $98.6\,\%$, which is an
excellent outcome, on the utilized dataset, by the proposed CVR-Net.

#### Experiment-$2$

We split the NOR images into NOR and pneumonia (CPN) classes, and then, we
train and evaluate the proposed model on $3$-class problem (Task-2), where we
have $1583$-NOR, $4273$-CPN, and $500$-NCP images, applying $5$-fold cross-
validations. For this task, the obtained accuracy, recall, precision, and
F1-score are $96.4\,\%$, $96.3\,\%$, $96.3\,\%$, and $96.4\,\%$, respectively
(see in Table 3). Those results confess that on an average $96.4\,\%$-samples
are correctly recognized by the proposed CVR-Net with type-II error and
positive predictive value of $3.6\,\%$, and $96.3\,\%$, respectively. It is
also noteworthy, from Table 3, that the inter-fold variation is increasing
with the decreased performances for all the metrics. The details class-wise
results, from the proposed CVR-Net for this task, are presented in a confusion
matrix in Table 5.

Table 5: Confusion matrix of Task-$2$ ($3$-class) of Dataset-$1$ employing the proposed CVR-Net, where the samples are the summation from each fold. |  | Actual
---|---|---
|  | NOR | CPN | NCP
| NOR |  1461 $92.29\,\%$ |  | 80
---
$1.87\,\%$
| 6
---
$1.20\,\%$
| CPN |  | 119
---
$7.52\,\%$
4185 $97.94\,\%$ |  | 15
---
$3.00\,\%$
Predicted | NCP |  | 3
---
$0.19\,\%$
| 8
---
$0.19\,\%$
479 $95.8\,\%$

It shows that the addition of new class (CPN) with earlier two classes (NOR
and NCP), as in Task-$1$, reduces the recognition rate of coronavirus from
$98.6\,\%$ to $95.8\,\%$, where Task-$2$ has $21$-false negatives out of $500$
samples. It is also observable that out of $21$-false negatives, $15$-NCP
samples are recognized as CPN, which affirms that there is a high degree of
similarity between CPN and NCP classes. Moreover, $80$-CPN and $119$-NOR are
recognized as the NOR and CPN, respectively. Which again reveals the inter-
class similarity between CPN and NOR classes.

#### Experiment-$3$

We further break the CPN class into pneumonia bacterial (CPB) and pneumonia
viral (CPV) classes. Thus, we have a total of $4$ classes in Task-$3$, where
it has $1583$-NOR, $2780$-CPB, $1493$-CPV, and $500$-NCP images. We also apply
$5$-fold cross-validation in this experiment. The proposed CVR-Net, for this
experiment (Task-3), produces the recognition results with accuracy, recall,
precision, and F1-score of $82.0\,\%$, $82.0\,\%$, $81.6\,\%$, and $81.6\,\%$,
respectively, with increased standard deviation comparing two previous
experiments (see Table 3). Those results, in this Task-$3$, expose that the
CVR-Net recognizes the classes with higher error rates than two previous
experiments, where it has a false-negative rate and positive predictive value
of $18.0\,\%$ and $81.6\,\%$, respectively. It is also noteworthy that the
positive predictive value and type-II error have been reduced by the margins
of $18.2\,\%$ and $17.8\,\%$, respectively, than binary Task-$1$ (see in Table
3), which indicates that additional $17.8\,\%$-NCP samples are recognized as
other classes (false negative) in Task-$3$. However, a further class-wise
investigation is given in a confusion matrix, as shown in Table 6, which
exhibits that the CVR-Net fails to recognize the coronavirus in $40$ cases.

Table 6: Confusion matrix of Task-$3$ ($4$-class) of Dataset-$1$ employing the proposed CVR-Net, where the samples are the summation from each fold. | Actual
---|---
| NOR | CPB | CPV | NCP
| NOR |  | 1498
---
$94.63\,\%$
| 76
---
$2.73\,\%$
| 74
---
$4.96\,\%$
| 13
---
$2.60\,\%$
| CPB |  | 31
---
$1.96\,\%$
| 2369
---
$85.22\,\%$
| 529
---
$35.43\,\%$
| 14
---
$2.80\,\%$
| CPV |  | 52
---
$3.29\,\%$
| 330
---
$11.87\,\%$
| 885
---
$59.28\,\%$
| 13
---
$2.60\,\%$
Predict | NCP |  | 2
---
$0.12\,\%$
| 5
---
$0.18\,\%$
| 5
---
$0.33\,\%$
| 460
---
$92.00\,\%$

Table 6 shows that the proposed CVR-Net can successfully recognize $460$-NCP
samples as NCP, where it erroneously predict $27$ samples as CPB & CPV and
$13$ samples as NOR. It is also noteworthy that $35.43\,\%$-CPV, and
$11.87\,\%$-CPB are respectively recognized as CPB and CPV.

#### Experiment-$4$ & Experiment-$5$

In these two experiments, to train and evaluate the proposed CVR-Net, we use
dataset-$2$, with $3$-classes (Task-$4$), and dataset-$3$, with $2$-classes
(Task-$5$), as presented in Table 1. Table 3 shows that our model has
accuracy, recall, precision, and F$1$-score of $96.1\,\%$, $96.1\,\%$,
$96.1\,\%$, and $96.1\,\%$, respectively, for dataset-$2$ (Task-$4$), which
are $78.0\,\%$, $78.0\,\%$, $78.0\,\%$, and $78.0\,\%$, respectively, for
dataset-$3$ (Task-$5$). Those results on dataset-$2$ and dataset-$3$
demonstrate that our CVR-Net model recognizes the coronavirus with type-II
errors as $3.9\,\%$, and $22.0\,\%$, respectively. Table 7 and Table 8 show
the confusion matrix from our proposed CVR-Net utilizing the dataset-$2$ and
dataset-$3$, respectively.

Table 7: Confusion matrix of Task-$4$ ($3$-class) of Dataset-$2$ employing the proposed CVR-Net, where the samples are the summation from each fold. |  | Actual
---|---|---
|  | NOR | CPN | NCP
| NOR |  1522 $92.35\,\%$ |  | 88
---
$2.01\,\%$
| 7
---
$1.40\,\%$
| CPN |  | 126
---
$7.65\,\%$
4276 $97.83\,\%$ |  | 26
---
$5.20\,\%$
Predicted | NCP |  | 0
---
$0.00\,\%$
| 7
---
$0.16\,\%$
467 $93.4\,\%$
Table 8: Confusion matrix of Task-$5$ ($2$-class) of Dataset-$3$ employing the proposed CVR-Net, where the samples are the summation from each fold. | Actual
---|---
| NOR | NCP
| NOR |  | $87$
---
$82.86\,\%$
| $26$
---
$26.53\,\%$
Predicted | NCP |  | $18$
---
$17.14\,\%$
| $72$
---
$73.47\,\%$

The matrix, as shown in Table 7, reveals the FN and FP for coronavirus
recognition, where the number of wrongly classified images (type-I or type-II
errors) is $126/1648\,(7.65\,\%)$, $95/4371\,(2.17\,\%)$, and
$33/500\,(6.60\,\%)$ respectively for the NOR, CPN, and NCP. In total, $6263$
images (out of $6519$) are successfully recognized as their respective
classes, especially $467$ images (out of $500$) for coronavirus, which
exhibits the praiseworthy success of the proposed CVR-Net for the correct
recognition of coronavirus. Again, the confusion matrix, as presented in Table
8, shows that $82.86\,\%$-NOR samples are correctly classified as NOR, whereas
$17.14\,\%$-NOR samples are wrongly classified as NCP. On the other hand,
$73.47\,\%$-NCP samples are correctly classified as NCP, whereas
$26.53\,\%$-NCP samples are wrongly classified as NOR. Although the
performance of the CVR-Net on dataset-$3$ is not as high as in the dataset-$1$
and dataset-$2$ (see in Table 3), it is still better as the utilized dataset
is very small in size comparing other two datasets (see in Table 1).

#### Results Comparison

Table 9 represents the performance comparison of the proposed CVR-Net with
other recent state-of-the-art methods with dataset-$1$ (Task-$1$ and Task-$2$)
and dataset-$3$ (Task-$5$). The remaining two other tasks, (Task-$3$ and
Task-$4$) are not reported in Table 9, as state-of-the-art methods were not
trained and tested on these datasets. To enhance the recognition performance,
authors, in several new methods, utilized more the external data to train
their networks, which are not publicly available yet. The improvement of the
recognition network may not be due to the superiority of the network itself,
but the characteristics of the external data, similar to the test datasets.
However, we have reported the results of the methods, which were trained and
tested on the same datasets, for fairness in comparison. The proposed CVR-Net
produces the best recognition results, as presented in Table 9, for seven out
of the nine cases while performing second-best with the winning methods on the
other two cases.

Table 9: The state-of-the-art comparison with proposed CVR-Net, for three different tasks, was trained, validated, and tested on the same dataset. The best, second-best and third-best metrics are denoted by bold font, underline, and double underline. | Task-1 | Task-2 | Task-5
---|---|---|---
Methods | Re | Pr | Ac | Re | Pr | Ac | Re | Pr | Ac
VGG-$19$ (Apostolopoulos and Mpesiana, 2020) | - | - | $0.987$ | - | - | $0.935$ | - | - | -
Xception (Apostolopoulos and Mpesiana, 2020) | - | - | $0.856$ | - | - | $0.928$ | - | - | -
Covid-Net (Wang and Wong, 2020) | - | - | - | $\underline{\underline{0.933}}$ | $\underline{\underline{0.937}}$ | $\underline{\underline{0.933}}$ | - | - | -
ResNet-$50$ (Sethy and Behera, 2020) | $\underline{\underline{0.973}}$ | - | $0.954$ | - | - | - | - | - | -
VGG-$19$ (Hemdan et al., 2020) | $0.900$ | $0.915$ | $0.900$ | - | - | - | - | - |
ResNet-$50$ (Narin et al., 2020) | $0.960$ | 1.0 | $0.980$ | - | - | - | - | - |
InceptionV$3$ (Narin et al., 2020) | $0.940$ | 1.0 | $0.970$ | - | - | - | - | - | -
DarkNet (Ozturk et al., 2020) | $0.951$ | $0.980$ | $\underline{\underline{0.981}}$ | $0.854$ | $0.900$ | $0.870$ | - | - |
CoroNet(Xception) (Khan et al., 2020) | $\underline{0.993}$ | $\underline{\underline{0.983}}$ | $\underline{0.990}$ | 0.969 | $\underline{0.950}$ | $\underline{0.950}$ | - | - | -
VGG-$16$ (He et al., 2020b) | - | - | - | - | - | - | - | - | $\underline{\underline{0.760}}$
ResNet-$18$ (He et al., 2020b) | - | - | - | - | - | - | - | - | $0.740$
EfficientNet-b0 (He et al., 2020b) | - | - | - | - | - | - | - | - | $\underline{0.770}$
CRNet (He et al., 2020b) | - | - | - | - | - | - | - | - | $0.730$
CVR-Net (Proposed, 2020) | 0.997 | $\underline{0.997}$ | 0.998 | $\underline{0.964}$ | 0.963 | 0.964 | 0.780 | 0.780 | 0.780
Re: Recall, Pr: Precision, and Ac: Accuracy.

Firstly, the proposed CVR-Net yields the best results, for Task-$1$,
concerning the accuracy and type-II errors (recall) by beating the second-best
state-of-the-art CoroNet (Khan et al., 2020) with the margins of $0.8\,\%$ and
$0.4\,\%$, respectively. Concerning the positive predictive value, CVR-Net,
for Task-$1$, is behind the state-of-the-art ResNet-$50$ & InceptionV$3$
(Narin et al., 2020) by $0.3\,\%$, but it outperforms the third-best CoroNet
(Khan et al., 2020) by a $1.4\,\%$ margin with respect to the same metric.
Secondly, for Task-$2$, CVR-Net beats the second-best CoroNet (Khan et al.,
2020) by the margins of $1.3\,\%$, and $1.4\,\%$ respectively for precision
and accuracy. Although, for the same task and type-II errors (recall), our
CVR-Net is behind the winner CoroNet (Khan et al., 2020) by $0.5\,\%$, it
outperforms the third-best recall of COVID-Net (Wang and Wong, 2020) by a
margin of $3.1\,\%$. Thirdly, for Task-$5$ and grand challenge dataset, our
proposed CVR-Net outperforms all the methods, such as VGG-$16$ (He et al.,
2020b), ResNet-$18$ (He et al., 2020b), EfficientNet-b0 (He et al., 2020b),
and CRNet (He et al., 2020b), for all the metrics (see in Table 9). The above
discussions show that the performance of the proposed CVR-Net for coronavirus
recognition is praiseworthy for all the utilized datasets.

##  4 Discussion

The COVID-$19$ pandemic has a disastrous effect on the health and well-being
of the global population. Effective and early screening of infected patients
is a critical and crucial step to fight against the COVID-$19$ epidemic, where
the examination and investigation of chest radiography images, via any CAD
tool, is one of the vital screening approaches. Recent studies on COVID-$19$
patients show that there are several coronavirus characteristics in chest
radiography images. Motivated by this and inspired by the research community’s
open-source endeavors, we aimed to design an automated image classifier for
the recognition of the coronavirus utilizing the chest radiography images.

However, to design such a classifier, CNNs are better-choice as they
automatically learn low-, middle-, and high-level features directly from the
input images. Finally, fully-connected neural networks, also known as
multilayer perceptron, classify those features. However, such CNN-based
classifiers’ training is an arduously challenging process, especially when the
training is with a smaller dataset as in the COVID-$19$ datasets. There are
several commonly occurring limitations in current CNN-based classifiers; it is
prone to overfitting, vanishing gradient problem, and amount of the network’s
depth with the following number of times of subsampling. However, in this
article, we proposed an end-to-end network called CVR-Net for automated
coronavirus prediction by considering the limitations mentioned earlier in
network design. In the proposed CVR-Net the aggregation, of the different
encoders and their different scales partially alleviates those limitations, as
if one or more members of the ensembled CVR-Net, fail to predict other can
compensate it. The final cost function can not be zero, as it is the summation
of each cost. Thus, the gradient is always non-zero in our proposed CVR-Net.

The class-wise results, for all the tasks (Task-$1$ to Task-$5$), in Table 4
to Table 8, experimentally show that the metrics for all classes are similar
for each task, although imbalanced data distributions are utilized in our
model. The positive NCP class is highly underrepresented, for all the tasks
(see in Table 1), still it results compatible with other classes. However, the
employment of class rebalancing by penalizing the cost of the overrepresented
class, in this article (see in subsection 2.3), is the crucial reason behind
these balanced performances by the proposed CVR-Net. The results for all the
tasks, as in Table 3, exhibit that the inter-fold variation is very less,
which ensures the better-robustness of the CVR-Net for coronavirus and
pneumonia recognition. The multi-scale-multi-encoder ensemble in CVR-Net and
appliance of the reasonable image augmentations and transferring the weights
from the ImageNet as a preprocessing, during the training of CVR-Net, are the
noteworthy catalyst of obtaining the robust recognition results.

The experimental results on dataset-$1$ from the CVR-Net, in Table 3, also
demonstrated that the recognition results, for Task-$1$ ($2$-class), are
better than the other two tasks, such as Task-$2$: $3$-class and Task-$3$:
$4$-class, with the same number of total training images. Fig. 6 depicts the
impact of adding more classes reduces the performance metrics as such a new
class increase the intra-class similarity.

(a)  Figure 6: Impact of increasing the number of classes with the same number
of total training samples, where bars with blue, green, and black colors
denote the CVR-Net results for $2$-class, $3$-class, and 4-class,
respectively.

Fig. 6 demonstrates that CVR-Net has better recognition results in binary case
(Task-$1$), where it can recognize $493$-NCP samples as NCP with only $7$
samples as false negative (see in Table 4). Such a result is because we keep
all the normal and pneumonia images as NOR class, where the intra-class
similarity, between NOR and NCP classes, in Task-$1$, is very less. Whenever
we break the NOR class into NOR and CPN classes in Task-$2$ ($3$-class), the
recognition results are lower than the former Task-$1$ (see in Table 4, Table
5, and Fig. 6). Remarkably, Table 5 depicts that the false-negative for
coronavirus recognition has increased, where $15$-NCP are predicted as CPN, as
well as many NOR and CPN samples are predicted as CPN and NOR, respectively.
Such false-negative and false-positive results reveal that CPN class has
similarities with both NOR and NCP, which CVR-Net can not recognize due to
fewer samples in the training set. Further breaking of the CPN class into
pneumonia bacterial (CPB) and pneumonia viral (CPV) classes, highly decreases
all the metrics (see in Fig. 6), where, unfortunately, CVR-Net recognize the
coronavirus with $40$ false negative out of $500$ samples. Table 6 shows that
there are many false negative and false positives, which exhibit that with
fewer training samples, the addition of more classes introduces the inter-
class similarity and intra-class diversity. Those discussions reveal that the
coronavirus recognition, from the proposed CVR-Net, is admirable, even with
the less training samples, if we use less number of class as it has a minor
inter-class similarity and intra-class diversity. In the future, the addition
of more distinctive samples in every class, in Task-$2$ and Task-$3$, can lead
CVR-Net to perform markedly even for the increased number of classes, as it
has a better design to avoid the overfitting and vanishing gradient problems.

##  5 Conclusion

The number of people infected by COVID-$19$ is increasing day-by-day, which
can permanently damage the lungs and later provoke death. During this pandemic
emergency, many countries struggle with their shortage of resources for proper
recognition of the coronavirus, where such recognition, with negligible false
negative, is highly essential. This article aimed to design an artificial
system for automated distinguishing people with positive coronavirus. With
this thing in mind, we have proposed and implemented an end-to-end deep
learning-based model, called CVR-Net, to recognize the coronavirus with the
very less false negative from chest radiography images without any
intermediate intervention. The multi-scale-multi-encoder design of the CVR-Net
ensures robustness in recognition, as the final prediction probability is the
aggregation of multiple scales and encoders. In the proposed CVR-Net,
different integral parts of the proposed preprocessing, such as image
augmentations, class rebalancing, and transfer learning, boosted the
performance of coronavirus recognition. The class rebalancing protects the
model from being biased to a particular overrepresent class, as a massive
number of manually annotated positive images are not publicly available yet.
The performance can further be increased by precisely segmenting the lung and
adding more distinctive training samples. We also intend to deploy our trained
CVR-Net to a web application for clinical utilization.

## Acknowledgements

None. No funding to declare.

## Declaration of Competing Interest

The authors have no conflict of interest to disclose.

## References

  * A. J. NEWS (2020a) A. J. NEWS, 2020a.  _Bangladesh scientists create $\$3$ kit. Can it help detect COVID-19? _.  https://bit.ly/aj2020corona [Accessed: 14 July 2020].
  * A. J. NEWS (2020b) A. J. NEWS, 2020b.  _India’s poor testing rate may have masked coronavirus cases._.  https://bit.ly/aj2020covid [Accessed: 11 July 2020].
  * Abbas et al. (2020a) Abbas, A., Abdelsamea, M.M., Gaber, M.M., 2020a.  Classification of covid-19 in chest x-ray images using detrac deep convolutional neural network.  arXiv preprint arXiv:2003.13815 .
  * Abbas et al. (2020b) Abbas, A., Abdelsamea, M.M., Gaber, M.M., 2020b.  Detrac: Transfer learning of class decomposed medical images in convolutional neural networks.  IEEE Access 8, 74901–74913.
  * Afshar et al. (2020) Afshar, P., Heidarian, S., Naderkhani, F., Oikonomou, A., Plataniotis, K.N., Mohammadi, A., 2020\.  Covid-caps: A capsule network-based framework for identification of covid-19 cases from x-ray images.  arXiv preprint arXiv:2004.02696 .
  * Ahmed Ali (2020) Ahmed Ali, 2020.  _Pneumonia sample XRays_.  https://www.kaggle.com/ahmedali2019/pneumonia-sample-xrays [Accessed: 10 July 2020].
  * Apostolopoulos et al. (2020) Apostolopoulos, I.D., Aznaouridis, S.I., Tzani, M.A., 2020.  Extracting possibly representative covid-19 biomarkers from x-ray images with deep learning approach and image data related to pulmonary diseases.  Journal of Medical and Biological Engineering , 1\.
  * Apostolopoulos and Mpesiana (2020) Apostolopoulos, I.D., Mpesiana, T.A., 2020\.  Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks.  Physical and Engineering Sciences in Medicine , 1\.
  * Balochian and Baloochian (2019) Balochian, S., Baloochian, H., 2019\.  Social mimic optimization algorithm and engineering applications.  Expert Systems with Applications 134, 178–191.
  * Bergstra and Bengio (2012) Bergstra, J., Bengio, Y., 2012\.  Random search for hyper-parameter optimization.  The Journal of Machine Learning Research 13, 281–305.
  * Chen et al. (2020) Chen, N., Zhou, M., Dong, X., Qu, J., Gong, F., Han, Y., Qiu, Y., Wang, J., Liu, Y., Wei, Y., et al., 2020\.  Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in wuhan, china: a descriptive study.  The Lancet 395, 507–513.
  * Chollet (2015) Chollet, F., 2015.  Keras.  https://github.com/fchollet/keras.
  * Chollet (2017) Chollet, F., 2017.  Xception: Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1251–1258.
  * Chowdhury et al. (2020) Chowdhury, M.E., Rahman, T., Khandakar, A., Mazhar, R., Kadir, M.A., Mahbub, Z.B., Islam, K.R., Khan, M.S., Iqbal, A., Al-Emadi, N., et al., 2020\.  Can ai help in screening viral and covid-19 pneumonia?  arXiv preprint arXiv:2003.13145 .
  * Cohen et al. (2020) Cohen, J.P., Morrison, P., Dao, L., Roth, K., Duong, T.Q., Ghassemi, M., 2020\.  Covid-19 image data collection: Prospective predictions are the future.  arXiv preprint arXiv:2006.11988 .
  * Corman et al. (2020) Corman, V.M., Landt, O., Kaiser, M., Molenkamp, R., Meijer, A., Chu, D.K., Bleicker, T., Brünink, S., Schneider, J., Schmidt, M.L., et al., 2020\.  Detection of 2019 novel coronavirus (2019-ncov) by real-time rt-pcr.  Eurosurveillance 25, 2000045\.
  * COVID (19) COVID, C., 19.  global cases by the center for systems science and engineering (csse) at johns hopkins university (jhu).  ArcGIS. Johns Hopkins CSSE. Retrieved April 8, 2020.
  * Deng et al. (2009) Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L., 2009.  ImageNet: A large-scale hierarchical image database, IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255.
  * Farooq and Hafeez (2020) Farooq, M., Hafeez, A., 2020\.  Covid-resnet: A deep learning framework for screening of covid19 from radiographs.  arXiv preprint arXiv:2003.14395 .
  * Furey et al. (2000) Furey, T.S., Cristianini, N., Duffy, N., Bednarski, D.W., Schummer, M., Haussler, D., 2000\.  Support vector machine classification and validation of cancer tissue samples using microarray expression data.  Bioinformatics 16, 906–914.
  * Ghoshal and Tucker (2020) Ghoshal, B., Tucker, A., 2020\.  Estimating uncertainty and interpretability in deep learning for coronavirus (covid-19) detection.  arXiv preprint arXiv:2003.10769 .
  * Hall et al. (2020) Hall, L.O., Paul, R., Goldgof, D.B., Goldgof, G.M., 2020\.  Finding covid-19 from chest x-rays using deep learning on a small dataset.  arXiv preprint arXiv:2004.02060 .
  * He et al. (2016a) He, K., Zhang, X., Ren, S., Sun, J., 2016a.  Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778.
  * He et al. (2016b) He, K., Zhang, X., Ren, S., Sun, J., 2016b.  Identity mappings in deep residual networks, in: European conference on computer vision, Springer. pp. 630–645.
  * He et al. (2020a) He, X., Wang, S., Shi, S., Chu, X., Tang, J., Liu, X., Yan, C., Zhang, J., Ding, G., 2020a.  Benchmarking deep learning models and automated model design for covid-19 detection with chest ct scans.  medRxiv .
  * He et al. (2020b) He, X., Yang, X., Zhang, S., Zhao, J., Zhang, Y., Xing, E., Xie, P., 2020b.  Sample-efficient deep learning for covid-19 diagnosis based on ct scans.  medRxiv .
  * Hemdan et al. (2020) Hemdan, E.E.D., Shouman, M.A., Karar, M.E., 2020.  Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images.  arXiv preprint arXiv:2003.11055 .
  * Howard et al. (2017) Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H., 2017\.  MobileNets: Efficient convolutional neural networks for mobile vision applications.  arXiv:1704.04861 .
  * Huang et al. (2020) Huang, C., Wang, Y., Li, X., Ren, L., Zhao, J., Hu, Y., Zhang, L., Fan, G., Xu, J., Gu, X., et al., 2020\.  Clinical features of patients infected with 2019 novel coronavirus in wuhan, china.  The lancet 395, 497–506.
  * Huang et al. (2017) Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q., 2017.  Densely connected convolutional networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708.
  * Ioffe and Szegedy (2015) Ioffe, S., Szegedy, C., 2015\.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  arXiv preprint arXiv:1502.03167 .
  * Ker et al. (2017) Ker, J., Wang, L., Rao, J., Lim, T., 2017.  Deep learning applications in medical image analysis.  Ieee Access 6, 9375–9389.
  * Kermany et al. (2018) Kermany, D.S., Goldbaum, M., Cai, W., Valentim, C.C., Liang, H., Baxter, S.L., McKeown, A., Yang, G., Wu, X., Yan, F., et al., 2018\.  Identifying medical diagnoses and treatable diseases by image-based deep learning.  Cell 172, 1122–1131.
  * Khan et al. (2020) Khan, A.I., Shah, J.L., Bhat, M.M., 2020.  Coronet: A deep neural network for detection and diagnosis of covid-19 from chest x-ray images.  Computer Methods and Programs in Biomedicine , 105581\.
  * Kingma and Ba (2014) Kingma, D.P., Ba, J., 2014.  Adam: A method for stochastic optimization.  arXiv:1412.6980 .
  * Krizhevsky et al. (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E., 2012.  Imagenet classification with deep convolutional neural networks, in: Advances in neural information processing systems, pp. 1097–1105.
  * Kumar et al. (2016) Kumar, A., Kim, J., Lyndon, D., Fulham, M., Feng, D., 2016.  An ensemble of fine-tuned convolutional neural networks for medical image classification.  IEEE journal of biomedical and health informatics 21, 31–40.
  * LeCun et al. (2010) LeCun, Y., Kavukcuoglu, K., Farabet, C., 2010.  Convolutional networks and applications in vision, in: Proceedings of 2010 IEEE international symposium on circuits and systems, IEEE. pp. 253–256.
  * Lee et al. (2020) Lee, E.Y., Ng, M.Y., Khong, P.L., 2020.  Covid-19 pneumonia: what has ct taught us?  The Lancet Infectious Diseases 20, 384–385.
  * Li et al. (2020) Li, Q., Guan, X., Wu, P., Wang, X., Zhou, L., Tong, Y., Ren, R., Leung, K.S., Lau, E.H., Wong, J.Y., et al., 2020.  Early transmission dynamics in wuhan, china, of novel coronavirus–infected pneumonia.  New England Journal of Medicine .
  * Lin et al. (2013) Lin, M., Chen, Q., Yan, S., 2013.  Network in network.  arXiv:1312.4400 .
  * Litjens et al. (2017) Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sánchez, C.I., 2017.  A survey on deep learning in medical image analysis.  Medical image analysis 42, 60–88.
  * Narin et al. (2020) Narin, A., Kaya, C., Pamuk, Z., 2020.  Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks.  arXiv preprint arXiv:2003.10849 .
  * Organization et al. (2001) Organization, W.H., et al., 2001.  Standardization of interpretation of chest radiographs for the diagnosis of pneumonia in children.  Technical Report. World Health Organization.
  * Ozkaya et al. (2020) Ozkaya, U., Ozturk, S., Barstugan, M., 2020.  Coronavirus (covid-19) classification using deep features fusion and ranking technique.  arXiv preprint arXiv:2004.03698 .
  * Ozturk et al. (2020) Ozturk, T., Talo, M., Yildirim, E.A., Baloglu, U.B., Yildirim, O., Acharya, U.R., 2020\.  Automated detection of covid-19 cases using deep neural networks with x-ray images.  Computers in Biology and Medicine , 103792\.
  * Paul Mooney (2018) Paul Mooney, 2018.  _Chest X-Ray Images (Pneumonia)_.  https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia [Accessed: 10 July 2020].
  * Pham et al. (2018) Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J., 2018.  Efficient neural architecture search via parameter sharing.  arXiv preprint arXiv:1802.03268 .
  * Rajaraman et al. (2020) Rajaraman, S., Siegelman, J., Alderson, P.O., Folio, L.S., Folio, L.R., Antani, S.K., 2020\.  Iteratively pruned deep learning ensembles for covid-19 detection in chest x-rays.  arXiv preprint arXiv:2004.08379 .
  * Rajpurkar et al. (2018) Rajpurkar, P., Irvin, J., Ball, R.L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C.P., et al., 2018\.  Deep learning for chest radiograph diagnosis: A retrospective comparison of the chexnext algorithm to practicing radiologists.  PLoS medicine 15, e1002686.
  * Rawat and Wang (2017) Rawat, W., Wang, Z., 2017.  Deep convolutional neural networks for image classification: A comprehensive review.  Neural computation 29, 2352–2449.
  * Redmon and Farhadi (2017) Redmon, J., Farhadi, A., 2017\.  Yolo9000: better, faster, stronger, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7263–7271.
  * Sethy and Behera (2020) Sethy, P.K., Behera, S.K., 2020\.  Detection of coronavirus disease (covid-19) based on deep features.  Preprints 2020030300, 2020\.
  * Shen et al. (2017) Shen, D., Wu, G., Suk, H.I., 2017.  Deep learning in medical image analysis.  Annual review of biomedical engineering 19, 221–248.
  * Shin et al. (2016) Shin, H.C., Roth, H.R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., Summers, R.M., 2016\.  Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning.  IEEE transactions on medical imaging 35, 1285–1298.
  * Shorten and Khoshgoftaar (2019) Shorten, C., Khoshgoftaar, T.M., 2019\.  A survey on image data augmentation for deep learning.  Journal of Big Data 6, 60\.
  * Simonyan and Zisserman (2014) Simonyan, K., Zisserman, A., 2014\.  Very deep convolutional networks for large-scale image recognition.  arXiv:1409.1556 .
  * Singh et al. (2020) Singh, D., Kumar, V., Kaur, M., 2020.  Classification of covid-19 patients from chest ct images using multi-objective differential evolution–based convolutional neural networks.  European Journal of Clinical Microbiology & Infectious Diseases , 1–11.
  * Srivastava et al. (2014) Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., 2014.  Dropout: a simple way to prevent neural networks from overfitting.  The journal of machine learning research 15, 1929–1958.
  * Szegedy et al. (2017) Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A., 2017\.  Inception-v4, inception-resnet and the impact of residual connections on learning, in: Thirty-first AAAI conference on artificial intelligence.
  * Szegedy et al. (2015) Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A., 2015\.  Going deeper with convolutions, in: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9.
  * Tajbakhsh et al. (2016) Tajbakhsh, N., Shin, J.Y., Gurudu, S.R., Hurst, R.T., Kendall, C.B., Gotway, M.B., Liang, J., 2016.  Convolutional neural networks for medical image analysis: Full training or fine tuning?  IEEE transactions on medical imaging 35, 1299–1312.
  * Toğaçar et al. (2020) Toğaçar, M., Ergen, B., Cömert, Z., 2020.  Covid-19 detection using deep learning models to exploit social mimic optimization and structured chest x-ray images using fuzzy color and stacking approaches.  Computers in Biology and Medicine , 103805\.
  * Wang et al. (2020a) Wang, D., Hu, B., Hu, C., Zhu, F., Liu, X., Zhang, J., Wang, B., Xiang, H., Cheng, Z., Xiong, Y., et al., 2020a.  Clinical characteristics of 138 hospitalized patients with 2019 novel coronavirus–infected pneumonia in wuhan, china.  Jama 323, 1061–1069.
  * Wang and Wong (2020) Wang, L., Wong, A., 2020.  Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images.  arXiv preprint arXiv:2003.09871 .
  * Wang et al. (2020b) Wang, W., Xu, Y., Gao, R., Lu, R., Han, K., Wu, G., Tan, W., 2020b.  Detection of sars-cov-2 in different types of clinical specimens.  Jama 323, 1843–1844.
  * Wetsman (2020) Wetsman, N., 2020.  Coronavirus testing shouldn’t be this complicated.  The Verge .
  * World Health Organization (2020a) World Health Organization, 2020a.  _Naming the coronavirus disease (COVID-19)_.  https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/naming-the-coronavirus-disease-(covid-2019)-and-the-virus-that-causes-it [Accessed: 16 July 2020].
  * World Health Organization (2020b) World Health Organization, 2020b.  _WHO Coronavirus Disease (COVID-19) Dashboard_.  https://covid19.who.int/ [Accessed: 11 July 2020].
  * Xu et al. (2020) Xu, X., Jiang, X., Ma, C., Du, P., Li, X., Lv, S., Yu, L., Ni, Q., Chen, Y., Su, J., et al., 2020\.  A deep learning system to screen novel coronavirus disease 2019 pneumonia.  Engineering .
  * Yadav and Jadhav (2019) Yadav, S.S., Jadhav, S.M., 2019\.  Deep convolutional neural network based medical image classification for disease diagnosis.  Journal of Big Data 6, 113\.
  * Yang et al. (2020) Yang, T., Wang, Y.C., Shen, C.F., Cheng, C.M., 2020\.  Point-of-care rna-based diagnostic device for covid-19.
  * Zhao et al. (2020) Zhao, J., Zhang, Y., He, X., Xie, P., 2020.  Covid-ct-dataset: a ct scan dataset about covid-19.  arXiv preprint arXiv:2003.13865 .
  * Zhu et al. (2020) Zhu, N., Zhang, D., Wang, W., Li, X., Yang, B., Song, J., Zhao, X., Huang, B., Shi, W., Lu, R., et al., 2020.  A novel coronavirus from patients with pneumonia in china, 2019.  New England Journal of Medicine .