# ASCENT: Amplifying Power Side-Channel Resilience via Learning & Monte-Carlo
Tree Search

Jitendra Bhandari1*, Animesh Basak Chowdhury1*, Mohammed Nabeel2, Ozgur
Sinanoglu2, Siddharth Garg1, Ramesh Karri1, Johann Knechtel2 1New York
University, USA, 2New York University Abu Dhabi, UAE

(2024)

###### Abstract.

Power side-channel (PSC) analysis is pivotal for securing cryptographic
hardware. Prior art focused on securing gate-level netlists obtained as-is
from chip design automation, neglecting all the complexities and potential
side-effects for security arising from the design automation process. That is,
automation traditionally prioritizes power, performance, and area (PPA),
sidelining security. We propose a “security-first” approach, refining the
logic synthesis stage to enhance the overall resilience of PSC
countermeasures. We introduce ASCENT, a learning-and-search-based framework
that (i) drastically reduces the time for post-design PSC evaluation and (ii)
explores the security-vs-PPA design space. Thus, ASCENT enables an efficient
exploration of a large number of candidate netlists, leading to an improvement
in PSC resilience compared to regular PPA-optimized netlists. ASCENT is up to
120x faster than traditional PSC analysis and yields a 3.11x improvement for
PSC resilience of state-of-the-art PSC countermeasures.

Hardware Security, Power Side-Channel, Logic Synthesis, Design-Space
Exploration, Monte Carlo Tree Search.

††conference: Make sure to enter the correct conference title from your rights
confirmation emai; June 03–05, 2018; Woodstock, NY††isbn:
978-1-4503-XXXX-X/18/06††copyright: acmlicensed††journalyear: 2024††doi:
XXXXXXX.XXXXXXX11footnotetext: J. Bhandari and A. B. Chowdhury contributed
equally to this work.

##  1\. Introduction

Hardware implementations of cryptographic and other sensitive algorithms are
well-known to be vulnerable to side-channel attacks. Kocher et. al (kocher,
24) first demonstrated the power side-channel attack (PSCA) by exploiting
variations in power profiles to extract the secret keys; various advanced PSCA
versions followed throughout the years (survey-PSC, 30). To counter such
attacks, there are many ongoing efforts to enhance the PSCA resilience of
hardware implementations. For example, the state-of-the-art (SOTA)
countermeasures (mask-2005, 29, 25, 21) augment secret data with random noise
to obscure power profiles from secret data. However, all these countermeasures
incur large power, performance, and area (PPA) overheads.

Figure 1. PSCA resilience post-integration of the QuadSeal countermeasure
(7324539, 21) vs. area-delay of various AES netlists.

To tackle PPA overheads, security researchers typically take the outputs of
chip design automation processes—which are optimized for PPA by default—as a
starting point, perform PSCA analysis, and then propose/apply countermeasures
to mitigate PSCAs. However, such an approach can easily overlook circuit
configurations that might be inherently more more effective to support the
resilience of PSCA countermeasures, even if they have some PPA disadvantages.
We demonstrate this in Figure 1, where we show the area-delay plot of various
synthesized netlists versus the final PSCA resilience, for a representative
AES hardware implementation with the QuadSeal countermeasure (7324539, 21)
applied (see also Sec. 2.3 for the latter).

Our findings in Figure 1 clearly demonstrates that optimizing the baseline
netlist for PPA alone does not guarantee the strongest possible defense
against PSCAs in the end. This highlights the need to explore alternative
circuit configurations for inherently better PSCA resilience, leading to our
core research questions:

  1. (1)

What is the right circuit implementation to start with, such that—post-
countermeasure application—the hardware would have the best PSCA resilience?

  2. (2)

How can we guide chip design automation to generate such highly PSCA-resilient
circuits, yet with low PPA overheads?

In this work, we systematically tackle the fact that different logic synthesis
approaches can significantly impact PSCA resilience (for better or worse).
This is because different optimization steps result in different netlists with
varying characteristics for the type, number, driver strengths, etc. of the
standard cells used. Naturally, all these directly impact the power profiles,
thereby impacting the prospects of PSCAs as well. However, the related design-
space exploration is a tedious, parameter-rich challenge. Furthermore, with
commercial synthesis tools, we face a black-box optimization problem, where
the relationship between synthesis choices and PSC resilience is difficult to
model.

Figure 2. High-level view on ASCENT framework for optimized PSC resilience of
countermeasures.

To address these challenges, we propose ASCENT, a framework for amplifying PSC
resilience via learning and Monte-Carlo tree search (MCTS) (Figure 2). Our key
contributions are:

  * •

ASCENT provides a hybrid learning-and-search approach, enabling efficient and
effective exploration of the security-centric design space. ASCENT enables us
to explore 120$\times$ more configurations compared to a naive search.

  * •

ASCENT helps us to achieve up to 3.11$\times$ improvements in PSC resilience
for SOTA countermeasure integration when compared to regular, PPA-optimized
baseline netlists. At the same time, the PPA impact is well controlled and
limited by ASCENT, namely only up to 6.61% more area.

  * •

We open-source ASCENT as a commitment towards reproducible research. All the
algorithms, experiments, and benchmarks are publicly available at
https://github.com/NYU-MLDA/scarl.git.

##  2\. Background and Motivation

###  2.1. Power Side-Channel Attacks (PSCAs)

Cryptographic hardware is vulnerable to PSCAs, which exploit the fluctuations
in a device’s power consumption to reveal sensitive information like keys
(survey-PSC, 30). Thus, these attacks leverage the fundamental connection
between a device’s power consumption and its internal state during operations.
There are different types of PSCAs, including simple power analysis (SPA),
differential power analysis (DPA), and correlation power attack (CPA)
(brier2004CPA, 10).111SPA directly interprets the power profiles during
specific cryptographic operations, aiming to deduce sensitive data. DPA goes
further by comparing power consumption across multiple similar operations with
varying inputs. This technique seeks to isolate variations in power profiles
that directly correlate with the influence of the secret key. CPA employs
statistical tools, most commonly the Pearson correlation coefficient (PCC), to
match hypothesized power consumption patterns against the actual power
measurements. The highest correlation often reveals the correct key. Note that
we utilize CPA in this work; more details are provided further below.

Technology Implications. With continuous advancements for the ever-shrinking
technology nodes, the threat of static power side-channel attacks (S-PSCAs)
has significantly increased over the years (leakage-2010-TCAS, 1, 27, 18).
Unlike dynamic PSCAs, which focus on power fluctuations during active
computations, S-PSCAs exploit the relationship between stored data and static
power consumption (leakage power). More specifically, advanced nodes utilize
standard cells of various types with different power-performance
characteristics, e.g., low-threshold voltage (LVT) and ultra-low threshold
voltage (ULVT) cells are faster than the regular (RVT) cells, thereby helping
to meet faster timing constraints, albeit at the expense of significantly
higher leakage power. LVT and ULVT cells are essential for timing closure,
i.e., the careful final-stage efforts in design automation. In short, these
implications highlight the need for dedicated countermeasures that protect
against S-PSCAs.

Countermeasures. To combat S-PSCAs, security-aware designers can employ
masking (mask-2005, 29, 8), shuffling, and/or balancing (Moos_Moradi_2021, 25,
21) schemes.222Further countermeasures and details are discussed in Section
2.2. Also note that the specific countermeasures employed for this work are
discussed in Section 2.3. In general, these strategies seek to obscure the
power profiles. Despite their demonstrated effectiveness, they all increase
design overheads considerably, necessitating to strive a careful balance
between security and PPA during the design process.

As indicated, here we tackle this challenge through our novel, learning-and-
search based framework for logic synthesis.

Simulation-Based Power Analysis. This is crucial for understanding a design’s
vulnerability to PSCAs before investing into actual tape-outs. Commonly
utilized procedures work along the following lines; we employ such a procedure
as well in this work.

First, through gate-level simulations, a value change dump (VCD) file is
obtained. This captures all the relevant state information of the device under
test. In combination with the post-synthesis netlist and the library files,
this allows for accurate power analysis. Second, power simulation tools
calculate the power consumption of each cell, including static/leakage power
and internal power from input/output switching transitions. Importantly, for
S-PSCA assessments, zero-delay simulations enable precise static power capture
during specific operations (unlike an averaged leakage power analysis provided
by full-timing simulations).

Correlation Power Analysis (CPA). This attack hinges on identifying a
correlation between a device’s power consumption and the intermediate data
processed throughout cryptographic operations. An attacker collects power
consumption data and then hypothesizes on all possible intermediate data
values, which often involve direct correlations to parts or derivatives of the
secret key.

More specifically, power consumption is predicted for each hypothetical
intermediate value, typically using models like Hamming weight or Hamming
distance, which relate binary data representation to power. The core of CPA
involves calculating the PCC between actual power measurements and the
predictions for each hypothesis. The hypothesis yielding the highest
correlation often represents the correct assignment for some part/derivative
of the key, allowing the attacker to reconstruct the full key eventually.

###  2.2. Related Work

S-PSC Attacks and Countermeasures. (giorgetti2007, 18) showed, for the first
time, the potential of S-PSCAs as a severe threat. (amstatic2014, 27) have
conducted one of the first practical experiments for S-PSCAs using FPGAs, with
some follow-up work presented in (7927198, 26). (leakage-2010-TCAS, 1)
highlighted the importance of leakage power and its effect on PSCAs especially
for advanced nodes. (Moos_Moradi_2021, 25) proposed various countermeasures
against S-PSCAs, albeit with considerable PPA overheads.
(bhandari2023lightweight, 7) have shown the impact of various types of
standard cells on S-PSCAs. (bhandari2024lightweight, 8) proposed a lightweight
masking scheme against S-PSCAs. (Karimi_Moos_Moradi_2019, 22) have
demonstrated the important side-effect of aging for S-PSCAs in advanced
technology nodes. (10.1007/978-3-319-57339-7_5, 17) studied multivariate
techniques focused on leakage power consumption to enhance cryptographic
security assessments. (9040870, 6, 5) proposed standard-cell, delay-based
dual-rail pre-charge logic (SC-DDPL) as countermeasure. However, due to its
structural complexity, this countermeasure is incompatible with commercial
design flows.

Design Frameworks for Advancing PSC Resilience. (karna, 31) introduced a
framework that scores and optimizes design parts to minimize PSC
vulnerabilities. This approach is limited by the impractical assumption of
timing slacks being ubiquitously available. It also lacks an actual PSC
evaluation. (rtl_psc, 19) proposed a framework for assessing PSC
vulnerabilities at the register-transfer level (RTL), with the goal to aid
countermeasure implementation. (7364404, 2) studied circuit replication and
SRAM sharing for PSC resilience in FPGAs. This approach notably increases
design costs but significantly improves security while maintaining FPGA
configurability. (10.1145/3488932.3517415, 11) emphasized the importance of
automated modeling for early, system-level detection of potential leaks.
(Tiri2004SecureLS, 32) proposed a methodology for so-called wave dynamic
differential logic (WDDL) on FPGAs.

Summary. Prior art for S-PSC countermeasures is focused on detailed empirical
studies, with only limited considerations for generalized design-time
integration of the countermeasures. At the same time, prior art for frameworks
is limited to D-PSCAs, not S-PSCAs, and was proposed for high-level design
stages and/or for FPGAs. Thus, there is no prior art that proposes a security-
first approach toward the complex, yet critical, challenge of design-space
exploration for S-PSCA countermeasure integration in ASICs. Also recall the
exploratory finding from Section 1. This gap provides the main motivation for
our work.

###  2.3. Representative Countermeasures

As motivated in Sections 2.1 and 2.2, S-PSCAs are becoming ever-more relevant
for advanced technology nodes, and various countermeasures have been proposed.
In our study, we consider the following two representative, SOTA
countermeasures against S-PSCAs.333While these two SOTA countermeasures appear
similar from a high-level view, their implementation details and, thus,
efficiency to hinder S-PSCAs still differ (Moos_Moradi_2021, 25, Table 5).
Importantly, ASCENT is agnostic to the countermeasures a designer likes to
explore and eventually integrate.

Quadruple Algorithmic Symmetrizing (QuadSeal) (7324539, 21). This technique
can protect against both dynamic and static PSCAs. It focuses on achieving a
balance in Hamming weights/distances for the cryptographic operations in
hardware. It operates by quadrupling the unprotected circuit structure and
balancing the arrangement of the so-called substitution boxes (S-Boxes) in
three of those circuit copies in a specific manner. Additionally, it involves
rotating inputs to the resulting balanced structure, to mitigate other real-
world dependencies introduced by, e.g., manufacturing process variations,
timing-path imbalances, aging, etc.

Exhaustive Logic Balancing (ELB) (Moos_Moradi_2021, 25). To reduce the
correlation between input data and the leakage current of standard cells,
selected sensitive cells are duplicated and fed with inverted input data,
which is akin to differential logic. Cell duplication is scaled based on the
number of possible input vectors—a single-input cell is duplicated once, while
two-input cells are quadrupled. This, along with the inverted inputs, ensures
a constantly uniform input distribution across all related cells.

Figure 3. An exemplary PSC-aware logic synthesis framework. Despite the focus
on security, conventional flows utilize PPA-optimized netlists as starting
point to integrate PSC countermeasures right-away on top (red arrows), without
further feedback loops. The latter omission is due to the significant runtime
taken by PSC evaluation (light-red, dotted box). Naturally, this can result in
sub-optimal countermeasure integration. In contrast, our work utilizes such
essential feedback (green arrows) and achieves this by an efficient learning-
and-search approach (detailed in Figure 4).

###  2.4. Logic Synthesis

Logic synthesis transforms a high-level hardware design (e.g., in RTL) into an
optimized and technology-specific gate-level netlist. This process offers
significant flexibility, as any single design can be mapped to many
functionally equivalent but structurally different netlists, all with distinct
PPA characteristics. Toward that end, so-called recipes are devised, which are
sequences of optimization steps. However, the sheer number of possible recipes
and netlists makes this a complex problem, in fact a $\Sigma^{P}_{2}$-complete
problem.

Setting. Commercial tools like Synopsys DC and Cadence Genus are tuned for PPA
optimization and use proprietary optimization algorithms toward that end.
Aside from scripting interfaces tailored for such PPA optimization, these
tools lack direct mechanisms to tune synthesis for other objectives like PSC
resilience.

Thus, research into novel optimization techniques, including our work on
security-first synthesis, often relies on the open-source and customizable
Yosys framework (yosys, 33). In fact, Yosys is the most widely adopted, SOTA
synthesis framework for and by academia.

AIG Representation, Generic Problem Formulation. Within Yosys, ABC (abc, 9) is
used for combinational optimization. First, ABC converts the design into a
homogeneous logic-network implementation called and-inverted graph (AIG).
Next, ABC’s algorithms employ various transformations at the sub-graph-level
(abc, 9). Importantly, users are free to tweak these algorithms in general and
the selection and order of transformations in particular, all to optimize the
AIG circuit representation according to their objectives.

More formally, in line with earlier works (chowdhury2021openabc, 14, 13, 15) a
synthesis recipe $a^{T}$ is a sequence of $T$ transformation steps operating
on an AIG structure to optimize for PPA and/or other metrics (e.g. security
(basak2023almost, 4)), all while preserving the original functionality. We
denote $\mathcal{A}$ of $\mathbf{L}$ unique synthesis transformations,
$\\{a_{0},a_{1},\ldots,a_{L-1}\\}$ ($a_{i}\in\mathcal{A}$), in a synthesis
recipe $a^{T}$. Thus, the number of synthesis recipes of length $T$ is
$\mathbf{L}^{\mathbf{T}}$, including repeatable transformations. This search
space is denoted by $\mathcal{A}^{T}$. The problem of generating a PPA-optimal
synthesis recipe for an AIG is:

(1)
$\displaystyle\operatorname*{arg\,max}_{a^{T}\in\mathcal{A}^{T}}PPA(AIG_{T}),\
\ s.t.\,\,AIG_{t+1}=\eta(AIG_{t},a_{t})\,\forall t\in[0,T-1]$

where $\eta$ is the synthesis function defined as
$\eta:AIG\times\mathcal{A}\longrightarrow AIG$.

###  2.5. Monte-Carlo Tree Search

MCTS is an optimization algorithm best suited for tree-structured search-space
exploration. It has been used in selected prior art for logic synthesis
(yu2020flowtune, 34, 28, 15, 16), albeit only for PPA optimization.

Structure. The MCTS search tree contains a root node representing the initial
state ($S_{0}$). A node is called leaf if there exist an $a_{t}\in\mathcal{A}$
that still remains unexplored and the node is terminal state. Each node
preserves two attributes: (1) node visit count $N(S_{t},a_{t})$ and (2)
cumulative reward $R(S_{t},a_{t})$.444$N(S_{t},a_{t})$ is the number of times
the nodes is visited during exploration. $R(S_{t},a_{t})$ is the total reward
obtained while exploring the sub-tree rooted at that node.

MCTS operates in four stages as follows.

1) Selection: of the “most promising node” in the MCTS tree until a leaf node
is reached for further exploration. The selection is based on the upper
confidence tree (UCT) computation as follows:

(2)
$\pi_{MCTS}(S_{t})=\operatorname*{arg\,max}_{a_{t}\in\mathcal{A}}\left(\underbrace{\frac{R(S_{t},a_{t})}{N(S_{t},a_{t})}}_{\text{Exploitation}}+c\dot{\underbrace{\sqrt{\frac{\log\sum_{a_{t}\in\mathcal{A}}N(S_{t})}{N(S_{t},a_{t})}}}_{\text{Exploration}}}\right)$

That is, UCT computation considers exploitation and exploration.

Exploitation computes the ratio of the reward $R(S_{t},a_{t})$ accumulated
over the sub-tree rooted at that node and the visits count $N(S_{t},a_{t})$.
This average reward is obtained by exploring the sub-tree which, in turn, is
done by picking an action $a_{t}$ from state $S_{t}$.

Exploration prioritizes nodes which have been less explored so far. It loosely
computes the ratio of the parent’s visit count
($\sum_{a_{t}\in\mathcal{A}}N(S_{t},a_{t})$) and the current node’s visit
count ($N(S_{t},a_{t})$). Thus, the score increases if the parent node has
been frequently visited whereas the current node has been less explored.

Starting from state $S_{0}$, $\pi_{MCTS}$ navigates the search space by
selecting the “best” action $a_{t}$ that achieves the maximum score combining
exploitation and exploration terms (Equation 2), effectively performing a
best-first exploration. The selection process continues until a leaf node is
encountered.

2 and 3) Expansion and Rollout: Once a leaf node is selected, an action
$a_{t}$ is chosen (at random) from the set of unexplored actions. A node is
added to the MCTS tree and node attributes are initialized. We call the
trajectory $\tau$ a sequence of nodes visited during MCTS selection and
expansion, which is represented by
$\\{S_{0},(S_{0},a_{0}),(S_{1},a_{1})..,(S_{\tau},a_{\tau})\\}$.

4) Backpropagation: After computing the score for the terminal state $S_{T}$,
the trajectory $\tau$ is backtracked from the leaf node till the root node.
The cumulative reward $R(S_{t},a_{t})$ and node visit count $N(S_{t},a_{t})$
for each node in $\tau$ are updated with $R(S_{T})$ and $1$, respectively.
Next, the process repeats from Stage 1) again.

##  3\. Problem Formulation

We believe that logic synthesis offers ample opportunities to discover
inherently more S-PSCA-resilient netlist structures that can amplify the
resilience of SOTA countermeasure even further. However, while traditional
PPA-focused synthesis does impact S-PSCA outcomes, recall that a dedicated
security-first approach is fundamentally missing (Section 2.2). We understand
and emphasize that realizing such an approach requires enormous efforts in
practice. This is due to two key facts:

  1. (1)

synthesis in general is already complex and its search-space computationally
expansive to explore (see Section 2.4);
and, coming on top,

  2. (2)

actual S-PSC evaluation, which is essential for accurate guidance for a
security-first synthesis method, incurs significant further computation cost
(see further below).

Figure 3 illustrates this problem for an exemplary, PSC-aware synthesis
framework. Next, we formalize this problem in general. Subsequently, we
evidence the practical challenges outlined above in more detail. We also
indicate on the techniques we utilize to address these. Finally, we provide
the specific problem formulation, leading to the proposed ASCENT framework.

Figure 4. The ASCENT framework. ➊ Train a $PT_{score}$ predictor for ultra-
fast PSC evaluation of synthesized netlists. ➋ MCTS-based search-space
exploration to obtain a synthesis recipe that maximizes $PT_{score}$. ➌ use
the obtained recipe for security-first synthesis, apply the countermeasure of
choice on top, and validate $PT_{score}$ of the final netlist.

Formulation. Our goal is to find a netlist that maximizes S-PSCA resilience
after application of SOTA S-PSCA countermeasure. To address these challenges,
we formulate security-first synthesis as an optimization problem, guided by a
Markov decision process (MDP), with distinct states, actions, transitions, and
rewards.

  * •

State $S_{t}$ at step $t$ is the AIG of the design $D$ after applying a
partial synthesis recipe of length $t$. $AIG_{0}$ is the initial AIG extracted
from $D$. The terminal state $AIG_{T}$ is the AIG generated after applying a
synthesis recipe of maximum length $T$.

  * •

Actions $\mathcal{A}$ is the set of $L$ functionality-preserving
transformations $\\{a_{0},a_{1},\ldots,a_{L-1}\\}$ ($a_{i}\in\mathcal{A}$)
provided by a synthesis tool.

  * •

State transition $\eta(S_{t+1}|S_{t},a_{t})$ is the transformation by applying
action $a_{t}$ on state $S_{t}$ resulting in state $S_{t+1}$. Here, the
transition function yields deterministic AIG.

  * •

Reward expresses the S-PSCA resilience as so-called $PT_{score}$, i.e., the
number of power traces required for successful key extraction, of the post-
countermeasure netlist. We consider a delayed reward model and assign zero
reward to every action until we reach terminal state $S_{T}$.

We define the overall problem as $PT_{score}$ maximization:

(3)
$\displaystyle\operatorname*{arg\,max}_{a_{T}\in\mathcal{A}^{T}}PT_{score}(\mathcal{C}(S_{T})),\,\,s.t.\,\,S_{t+1}=\eta(S_{t},a_{t})\,\,\forall
t\in[0,T-1]$

where $\mathcal{C}$ is the countermeasure applied on the synthesized netlist.

Practical Challenges. As indicated, there are critical barriers in terms of
computational complexity associated with this problem. For example, the search
space for synthesis in general is of complexity $L^{T}$, where $L=13$ and
$T=18$,555This exemplary choice of $L=13$ and $T=18$ is in line with the
length of synthesis recipes and unique synthesis transformations available for
Yosys’ compress2rs recipe. is approximately $\sim 10^{19}$.

As our problem has a non-analytical form and no closed-form solution is
available, we must rely on gradient-free optimization methods. This mandates
means for inexpensive reward evaluation. However, our experimentation shows
that running an accurate PSC attack evaluation on post-countermeasure netlists
requires up to 100k test vectors, which takes $\approx 6$ hours of simulation
runtime. Thus, even when evaluating only 100 samples using any gradient-free
optimizer, the process would take $\approx 25$ days.

These important observations raise the following questions toward
computationally-efficient S-PSC-aware logic synthesis:

  * •

Given a runtime budget, how can we quickly, yet accurately, evaluate S-PSC
attacks for some post-countermeasure netlist?

  * •

How can we efficiently explore the search space of synthesis recipes to obtain
S-PSC-resilient netlists?

Outline of Method. Addressing these challenges necessitate an optimization
approach that balances search efficiency with accurate S-PSCA assessment. This
leads us to the design of ASCENT (Section 4), which employs a hybrid learning-
and-search strategy. More specifically, ASCENT utilizes (i) a zero-shot
predictor $\hat{PT}_{score}(\mathcal{C}(S_{T}),\theta)$ to significantly speed
up the PSC evaluation, without loss of accuracy, and (ii) MCTS (Section 2.5)
to explore the large and complex search space for security-first synthesis in
an effective and efficient manner.

Extended Formulation. Assume a predictor
$\hat{PT}_{score}(\mathcal{C}(S_{T}),\theta)$, which predicts the number of
power traces required for PSCAs for a design $\mathbf{D}$ synthesized using a
$T$-length recipe. Then, we seek to solve the following problem:

(4)
$\displaystyle\operatorname*{arg\,max}_{a_{T}\in\mathcal{A}^{T}}\hat{PT}_{score}(\mathcal{C}(S_{T}),\theta),\,\,s.t.\,\,S_{t+1}=\eta(S_{t},a_{t})\,\,\forall
t\in[0,T-1]$

where $\mathbf{\theta}$ represents the predictor’s parameters. In simple
terms, the predictor shall serve as a computationally efficient surrogate for
direct S-PSCA evaluation.

##  4\. ASCENT Framework

We solve the problem formulation in Equation 4 in three steps, as illustrated
in Figure 4. Next, we outline these three steps.

First, we provide an exploratory experiment which clearly demonstrates that
maximizing the S-PSCA resilience of a netlist post-countermeasure integration,
i.e., maximizing $PT_{score}(\mathcal{C}(S_{T}))$, is the same as maximizing
the $PT_{score}(S_{T})$. Therefore, we train a zero-shot $\hat{PT}_{score}$
predictor using $PT_{score}$ values obtained from diverse synthesized
netlists. We show that it suffices to start with some representative data
points to train such zero-shot predictor.

Second, we employ the zero-shot predictor as S-PSCA evaluator for the MCTS-
based exploration of the search space. This predictor within MCTS is essential
to drive the sequential decision-making process. Note that we also improve the
predictor on the fly: we simulate the actual power traces for the top-$k$ and
bottom-$k$ recipes during the MCTS process and accordingly fine-tune the
predictor.

Third, we put all parts together and conduct an end-to-end optimization
process, including a final validation of the PSC resilience by actual PSC
evaluation after the optimization process is done.

###  4.1. Zero-Shot Predictor (➊)

To obtain a zero-shot predictor, we have to collect historical S-PSCA data in
a one-time effort. Key challenges here are (i) high runtime cost of S-PSCA
evaluation even for such one-time effort and (ii) ensuring a representative
dataset of diverse netlists with varying S-PSCA resilience. Next, we discuss
how we tackle these challenges and finally outline the actual training
approach.

####  4.1.1. Pre- vs Post-Countermeasure Evaluation

We observe a monotonic relationship between pre- and post-countermeasure
$PT_{score}$ values. Figure 5 and Figure 6 show $PT_{score}$ values for 50
randomly synthesized netlists pre- and post-countermeasure application for the
two representative countermeasures of choice.666The gaps between 6k and 7k for
the baseline/pre-countermeasure netlists are only due to chance, i.e., none of
the 50 random recipes provide scores in that ranges.

Importantly, this confirms that aiming for pre-countermeasure resilience
suffices for a guided design-space exploration toward best post-countermeasure
resilience. This observation helps to significantly limit the runtime cost for
obtaining training data. In fact, running S-PSCA evaluations on pre-
countermeasure netlists provides a 7.2x speed-up: it takes only 40–50 minutes
as compared to around 6 hours for post-countermasure netlists. This is due to
the lower resilience of the pre-countermeasure netlists; S-PSCAs can find the
correct key with less traces and in shorter time.

Figure 5. $PT_{score}$ on AES before and after applying QuadSeal. Figure 6.
$PT_{score}$ on AES before and after applying ELB.

####  4.1.2. Dataset Diversity

We are inspired by the sampling approaches described in (bai2023towards, 3).
These works inspire us to focus on exploring netlists that result in diverse
scores. Without loss of generality, we utilize simulated annealing (SA) toward
this end. Note that we tune the annealing scheduling for more diversity during
exploration.

####  4.1.3. Training of the Predictor

Our predictor is a regressor model. It uses the pre-countermeasure netlists,
three handcrafted features, and $PT_{score}$ values (obtained by actual S-PSCA
evaluation) as labels. The features are: (1) Overall Diversity: count of the
various cell types found in the synthesized netlists; (2, 3) Specific
Diversity: percentage of area consumed by LVT (2) and HVT (3) cells. We
trained and evaluated various predictor versions, exploring a wide range of
other hand-crafted features (e.g., area, delay, etc.) as well as more
synthesized design and more varied recipes. However, the best performance, as
in the best prediction of post-countermeasure resilience based on pre-
countermeasure resilience, was obtained for the three features above.

###  4.2. Monte-Carlo Tree Search & Fine-Tuning (➋)

Having established a fast and accurate zero-shot predictor, we now have to
integrate it into an search strategy for maximizing PSCA resilience. MCTS,
with its ability to intelligently balance exploration and exploitation, is a
natural fit for this complex task. Its delayed reward model aligns with our
problem.

####  4.2.1. MCTS Implementation

We employ MCTS as outlined in Sections 2.5 and 3. More details are provided
next.

We implement the critical reward component as follows. We assign normalized
$PT_{score}(S_{T})$ values, obtained from the zero-shot predictor
$PT_{score}$, to a terminal state $S_{T}$.

(5) $PT^{norm}_{score}(S_{T})=\begin{dcases}0&\text{if }PT_{score}\leq
PT_{threshold}\\\
\frac{PT_{score}}{PT_{threshold}}&\text{otherwise}\end{dcases}$

In plain words, we compare the predicted scores against a user-defined
threshold ($PT_{threshold}$) and assign $0$ reward if it is less than the
threshold. This reward formulation allows MCTS to skip any unpromising paths
in the search space. The reward for promising paths are normalized so that the
UCT computation maintains the balance for exploitation and exploration.

For the expansion and rollout stages, note the following. A sequence of
actions are taken which in turn synthesize the netlist until the terminal
state is reached. Then, the $\hat{PT}_{score}$ predictor is used to obtain
$PT_{score}$ values of the netlist. A new node is added and assigned the
updated $R(S_{t},a_{t})$ vale.

####  4.2.2. Online Fine-Tuning of Predictor

As indicated, we simulate the actual power traces for the top-$k$ and
bottom-$k$ recipes during the MCTS process and accordingly fine-tune the
predictor $\hat{PT}_{score}$. This is done to ensure the predictor is
continually updated with relevant corner-case data points, all without
hampering the ongoing search-space exploration. For efficiency, we conduct the
S-PSCA evaluation for these netlist in parallel.

####  4.2.3. Justification of MCTS

We note that reinforcement learning (RL)-based methods like (hosny2020drills,
20, 35) are gaining significant attention for advancing logic synthesis.
Important key differences for our work over these works are as follows, making
MCTS uniquely suitable.

In RL-based methods, the asynchronous actor-critic approach encounters
challenges in delayed reward systems. Thus, RL-based methods tuned for PPA
optimization typically assign immediate rewards, like reduced depth of AIGs,
which also correlate well with the final PPA. However, there is no such direct
correlation between AIGs and S-PSCA resilience. Therefore, we cannot utilize
immediate rewards and, thus, chose MCTS. In fact, its notion of back-
propagation (Section 2.5) helps to accurately estimate average rewards even
for intermediate nodes in the search tree.

###  4.3. Integration and End-to-End Validation (➌)

After completion of the MCTS process, we obtain the best synthesis recipe
using a greedy approach following standard procedures (most-rewarding-node
selection and most-visited-node selection). This recipe is expected to
generate the most S-PSCA resilient design post-countermeasure application.
Finally, we synthesize the circuit using this recipe, apply the countermeasure
of choice, and run an actual S-PSCA evaluation to report the final
$PT_{score}$.

##  5\. Experiments

###  5.1. Setup

AES Implementation. We use non-masked Electronic Code Book (ECB) mode AES with
S-Box as a look-up table. ECB mode pads data until the length of the block
which is 128.

We use a commercial 28nm technology library, focusing on the TT corner (25
degrees Celsius, 0.9V), and using different VT cells.

S-PSCA Setup. We implement a C++-based CPA attack following (knechtel22_PSC,
23). It incrementally increases the number of traces through coarse and
thorough sampling for 128 trials, aiming for a 90% success rate with thorough
sampling. In other words, the attack is thoroughly assessed in multiple runs,
not only one-shot trials.

The total number of traces depends on the case study; for baseline AES, up to
10K traces are sufficient, whereas QuadSeal and ELB countermeasures on top
require 50K and 100K traces, respectively.

$PT_{score}$ Predictor. We collected a corpus of 1,000 random, different
synthesis recipes for the baseline AES, with features extracted as in Section
4. This took us $\approx$30 days. We split the dataset into 80% and 20% for
training and testing, respectively.

We used XGBoost (chen2016xgboost, 12), a scalable and distributed, gradient-
boosted decision tree (GBDT) library to implement the predictor model. We
trained our model on an AMD EPYC 7542 server equipped with 128 CPUs and 1TB
RAM, running Red Hat Enterprise Linux Server Release 7.9 (Maipo). The CAD flow
utilized both open-source and commercial tools, including Synopsys VCS
M-2017.03-SP1 for RTL and gate-level simulations, Yosys for logic synthesis,
and Synopsys PrimeTime PX M-2017.06 for power simulations.

ASCENT Framework. We developed ASCENT in-house by implementing MCTS algorithm
and plugged it with $PT_{score}$ . We compute the handcrafted features
generated from the synthesized netlist and pass it to $PT_{score}$ which
provides quick feedback with a latency of 100 ms. We use Yosys (v0.38) to
perform end-to-end synthesis which takes roughly $\sim$2.8 minutes on a single
thread CPU run on our Intel server (Frequency: 2.3GHz, Memory: 256GB).

###  5.2. Results

####  5.2.1. $PT_{score}$ Predictor

Figure 7. Performance of $\hat{PT}_{score}$ predictor on test data.

Figure 7 shows our predictor’s performance on the test datasets, where the
x-axis and y-axis denote “Ground Truth” and “Predicted Scores”, respectively.
The Root Mean Squared Error (RMSE) score for the model on the test datasets is
565.58. However, recall that the predictor will be continuously improved by
fine-tuning to further enhance its performance.

####  5.2.2. ASCENT Framework

Table 1.  Comparison of $PT_{score}$ across recipes. Shown are only recipes with maximal $PT_{score}$.  Synthesis recipes | Baseline AES | AES + QuadSeal | AES + ELB
---|---|---|---
PT | Area | Power | Delay | PT | Area | Power | Delay | PT | Area | Power | Delay
score | ($\mu$m2) | (mW) | (ns) | score | ($\mu$m2) | (mW) | (ns) | score | ($\mu$m2) | (mW) | (ns)
compress2rs | 2800 | 12391 | 0.65 | 0.28 | 12500 | 16430 | 0.83 | 0.31 | 28000 | 24321 | 1.31 | 0.36
resyn2rs | 2900 | 12380 | 0.62 | 0.3 | 13000 | 16712 | 0.86 | 0.33 | 29000 | 24627 | 1.39 | 0.39
SA | 4900 | 12542 | 0.73 | 0.31 | 17500 | 17123 | 0.91 | 0.34 | 42000 | 25242 | 1.48 | 0.41
MCTS | 5300 | 13723 | 0.76 | 0.31 | 18500 | 18204 | 0.93 | 0.35 | 47000 | 26111 | 1.54 | 0.43
ASCENT | 9100 | 13155 | 0.74 | 0.3 | 42500 | 18097 | 0.92 | 0.35 | 87000 | 25928 | 1.53 | 0.44

We started with running the S-PSCA evaluation for the compress2rs recipe,
which we are considering as the baseline. This will be compared against for
all the methods used in this work. The recipe for compress2rs is provided in
(github). We consider three different settings for our experiments, namely
{AES, AES+QuadSeal, AES+ELB}, where AES denotes the baseline without
countermeasures, AES+QuadSeal denotes AES with the countermeasure QuadSeal,
and AES+ELB represents AES with countermeasure ELB incorporated (Section 2.3).

Table 1 details our results and comparison across the various methods studied
in this work. To better understand the results, we have compared the
$PT_{score}$ and overheads with the results from the baseline compress2rs run.
For example for the baseline AES, we would compare the results of each method
with the corresponding values from compress2rs. Here we observed
{1.04$\times$, 1.04$\times$, 1.04$\times$} improvement in the $PT_{score}$
values, along with an additional {-0.09%, 1.7%, 1.26%} area, {-4.6%, 3.61%,
6.11%} power, and {7.14%, 6.45%, 8.33%} delay overhead across those 3
different scenarios, respectively. Thus, with the introduction of the
countermeasures, we can see that the $PT_{score}$ is improved; this is
expected.

We then utilize the same black-box optimizer that was used for our predictor
model, SA, to also explore the security-first search space. Allowing a timeout
of 3 days, there is an improvement across all three scenarios: {1.75$\times$,
1.4$\times$, 1.5$\times$} higher scores, albeit at an overhead of {1.22%,
4.22%, 3.79%} area, {12.31%, 9.64%, 12.98%} power, and {10.71%, 9.68%, 13.89%}
delay, respectively.

We then utilized our ASCENT framework to obtain the synthesis recipes with
maximized $PT_{score}$ values. Here, an increase in resilience of
{1.89$\times$, 1.48$\times$, 1.68$\times$}, along with {10.75%, 10.78%, 7.36%}
area, {16.92%, 12.05%, 17.56%} power, and {10.71%, 12.90%, 19.44%} delay
overheads, respectively, are obtained.

Table 2.  Runtime performance achieved by ASCENT and competitive methods. Iterations denote the number of times the full S-PSCA evaluation can be run. We set a common timeout of 72 hours for fair comparison.  Method | Iterations | Speed-up
---|---|---
SA (AES with countermeasure) | 12 | 1.0$\times$
MCTS (AES with countermeasure) | 12 | 1.0$\times$
SA (baseline AES) | 108 | 9.0$\times$
MCTS (baseline AES) | 107 | 8.3$\times$
ASCENT | 1460 | 121.3$\times$

Finally, we used our full ASCENT framework to more effectively explore the
design space, again with timeout of 3 days. Recall that, thanks to the
predictor model, the time-consuming actual S-PSCA evaluation can be bypassed
in the MCTS exploration phase. We pick the top-3 recipes in terms of max
$PT_{score}$ and run actual S-PSCA evaluation on them, for final validation.
From Table 1, it can be seen that, relative to the baseline, we achieved on
average across these top-3 recipes {3.25$\times$, 3.40$\times$, 3.11$\times$}
higher PSC resilience with an overhead of only {6.16%, 10.15%, 6.61%} area,
{13.85%, 10.84%, 16.79%} power, and {7.14%, 12.90%, 22.22%} delay,
respectively.

Table 2 reports the runtime comparisons of various methods. The first two
cases, { SA (AES with countermeasures), MCTS (AES with countermeasures) }
require running the S-PSCA evaluation end-to-end, with countermeasures in
place. These required to collect 50k and 100k traces for QuadSeal and ELB,
respectively, compared to only 10k traces for the baseline AES. Thus,
5$\times$ and 10$\times$ more time is required, respectively. For the
remaining two cases, namely the baseline AES for SA versus MCTS, we obtained a
speedup of 9.0$\times$ and 8.3$\times$, respectively. In short, ASCENT is able
to explore a much larger design space much more quickly when compared to other
methods; e.g., 120$\times$ faster than the default black-box optimizer SA.

##  6\. Conclusion

In this work, we proposed ASCENT, a novel synthesis framework. We have
successfully enhanced the final resilience of existing power side-channel
countermeasures by carefully guided, yet efficient, exploration of the complex
search space. In fact, ASCENT enables a $3.11\times$ post-countermeasure
improvement when compared to conventional synthesis (tailored for PPA
optimization). For future work, we plan to tailor ASCENT to harden circuits
also against other threats like fault injection.

## References

  * (1) Massimo Alioto  “Leakage Power Analysis Attacks: A Novel Class of Attacks to Nanometer Cryptographic Circuits”  In _IEEE TCAS_ , 2010, pp. 355–367
  * (2) Ziyad Almohaimeed  “Secured-by-design FPGA: look-up tables and switch-boxes”  In _NORCAS_ , 2015, pp. 1–4
  * (3) Yunsheng Bai  “Towards a Comprehensive Benchmark for High-Level Synthesis Targeted to FPGAs”  In _NeurIPS_ 36, 2023, pp. 45288–45299
  * (4) Animesh Basak Chowdhury  “ALMOST: Adversarial Learning to Mitigate Oracle-less ML Attacks via Synthesis Tuning”  In _DAC_ , 2023, pp. 1–6
  * (5) Davide Bellizia  “SC-DDPL as a Countermeasure against Static Power Side-Channel Attacks”  In _Cryptography_ , 2021
  * (6) Davide Bellizia  “SC-DDPL: A Novel Standard-Cell Based Approach for Counteracting Power Analysis Attacks in the Presence of Unbalanced Routing”  In _IEEE TCAS_ , 2020, pp. 2317–2330
  * (7) Jitendra Bhandari  “Beware Your Standard Cells! On Their Role in Static Power Side-Channel Attacks” https://eprint.iacr.org/2023/920, Cryptology ePrint Archive, Paper 2023/920, 2023
  * (8) Jitendra Bhandari  “Lightweight Masking Against Static Power Side-Channel Attacks”  In _arXiv preprint arXiv:2402.03196_ , 2024
  * (9) Robert Brayton  “ABC: An academic industrial-strength verification tool”  In _CAV_ , 2010, pp. 24–40
  * (10) Eric Brier  “Correlation Power Analysis with a Leakage Model”  In _IACR CHES_ , 2004, pp. 16–29
  * (11) Ileana Buhan  “SoK: Design Tools for Side-Channel-Aware Implementations”  In _AsiaCCS_ , 2022, pp. 756–770
  * (12) Tianqi Chen  “Xgboost: A scalable tree boosting system”  In _ACM SIGKDD_ , 2016, pp. 785–794
  * (13) Animesh Basak Chowdhury  “Bulls-Eye: Active few-shot learning guided logic synthesis”  In _IEEE TCAD_ IEEE, 2022
  * (14) Animesh Basak Chowdhury  “OpenABC-D: A Large-Scale Dataset For Machine Learning Guided Integrated Circuit Synthesis”  In _arXiv preprint arXiv:2110.11292_ , 2021
  * (15) Animesh Basak Chowdhury  “Retrieval-Guided Reinforcement Learning for Boolean Circuit Minimization”  In _arXiv preprint arXiv:2401.12205_ , 2024
  * (16) Matthew DeLorenzo  “Make every move count: LLM-based high-quality rtl code generation using MCTS”, 2024
  * (17) Milena Djukanovic  “Multivariate Analysis Exploiting Static Power on Nanoscale CMOS Circuits for Cryptographic Applications”  In _AFRICACRYPT_ , 2017, pp. 79–94
  * (18) Jacopo Giorgetti  “Analysis of Data Dependence of Leakage Current in CMOS Cryptographic Hardware”  In _GLSVLSI_ , 2007, pp. 78–83
  * (19) Miao He  “RTL-PSC: Automated Power Side-Channel Leakage Assessment at Register-Transfer Level”  In _VTS_ , 2019, pp. 1–6
  * (20) Abdelrahman Hosny  “DRiLLS: Deep reinforcement learning for logic synthesis”  In _2020 25th ASP-DAC_ , 2020, pp. 581–586  IEEE
  * (21) D. Jayasinghe  “QuadSeal: Quadruple algorithmic symmetrizing countermeasure against power based side-channel attacks”  In _CASES_ , 2015, pp. 21–30
  * (22) Naghmeh Karimi  “Exploring the Effect of Device Aging on Static Power Analysis Attacks”  In _TCHES_ , 2019, pp. 233–256
  * (23) Johann Knechtel  “Design-time exploration of voltage switching against power analysis attacks in 14 nm FinFET technology”  In _Integration_ , 2022, pp. 27–34
  * (24) Paul C. Kocher  “Differential Power Analysis”  In _CRYPTO_ , 1999, pp. 388–397
  * (25) Thorben Moos  “Countermeasures against Static Power Attacks:– Comparing Exhaustive Logic Balancing and Other Protection Schemes in 28nm CMOS ”  In _IACR CHES_ , 2021, pp. 780–805
  * (26) Thorben Moos  “Static Power Side-Channel Analysis of a Threshold Implementation Prototype Chip”  In _DATE_ , 2017, pp. 1324–1329
  * (27) Amir Moradi  “Side-Channel Leakage through Static Power”  In _CHES_ , 2014, pp. 562–579
  * (28) Zehua Pei  “AlphaSyn: Logic synthesis optimization with efficient monte carlo tree search”  In _ICCAD_ , 2023, pp. 1–9  IEEE
  * (29) Thomas Popp  “Masked Dual-Rail Pre-charge Logic: DPA-Resistance Without Routing Constraints”  In _CHES_ Berlin, Heidelberg: Springer Berlin Heidelberg, 2005, pp. 172–186
  * (30) Mark Randolph  “Power Side-Channel Attack Analysis: A Review of 20 Years of Study for the Layman”  In _Cryptography_ 4.2, 2020  URL: https://www.mdpi.com/2410-387X/4/2/15
  * (31) Patanjali Slpsk  “Karna: A Gate-Sizing based Security Aware EDA Flow for Improved Power Side-Channel Attack Protection”  In _ICCAD_ , 2019, pp. 1–8
  * (32) Kris Tiri  “Secure Logic Synthesis”  In _FPGA_ , 2004
  * (33) Clifford Wolf  “Yosys-a free Verilog synthesis suite”  In _Austrochip_ , 2013, pp. 97
  * (34) Cunxi Yu  “Flowtune: Practical multi-armed bandits in boolean optimization”  In _ICCAD_ , 2020, pp. 1–9
  * (35) Keren Zhu  “Exploring logic optimizations with reinforcement learning and graph convolutional network”  In _2020 ACM/IEEE Workshop on MLCAD_ , 2020, pp. 145–150

github

  *[PSCA]: power side-channel attack
  *[SOTA]: state-of-the-art
  *[PPA]: power, performance, and area
  *[MCTS]: Monte-Carlo tree search
  *[CPA]: correlation power attack
  *[PCC]: Pearson correlation coefficient
  *[AIG]: and-inverted graph
  *[UCT]: upper confidence tree
  *[PSC]: Power side-channel
  *[MDP]: Markov decision process
  *[SA]: simulated annealing


# Classification of COVID-19 in Chest X-ray Images Using Fusion of Deep
Features and LightGBM

Hamid Nasiri  Department of Computer Engineering
Amirkabir University of Technology
Tehran, Iran
<EMAIL_ADDRESS>Ghazal Kheyroddin  Morteza Dorrigiv  Electrical and
Computer Engineering Department
Semnan University
Semnan, Iran
<EMAIL_ADDRESS>Electrical and Computer Engineering Department
Semnan University
Semnan, Iran
<EMAIL_ADDRESS>Mona Esmaeili  Amir Raeisi Nafchi  Electrical and
Computer Engineering Department
University of New Mexico
Albuquerque, USA
<EMAIL_ADDRESS>Electrical and Computer Engineering Department
University of New Mexico
Albuquerque, USA
<EMAIL_ADDRESS>Mohsen Haji Ghorbani  Payman Zarkesh-Ha  Department of
Computer Engineering
Technical and Vocational University (TVU)
Tehran, Iran
<EMAIL_ADDRESS>Electrical and Computer Engineering Department
University of New Mexico
Albuquerque, USA
<EMAIL_ADDRESS>
###### Abstract

The COVID-19 disease was first discovered in Wuhan, China, and spread quickly
worldwide. After the COVID-19 pandemic, many researchers have begun to
identify a way to diagnose the COVID-19 using chest X-ray images. The early
diagnosis of this disease can significantly impact the treatment process. In
this article, we propose a new technique that is faster and more accurate than
the other methods reported in the literature. The proposed method uses a
combination of DenseNet169 and MobileNet Deep Neural Networks to extract the
features of the patient’s X-ray images. Using the univariate feature selection
algorithm, we refined the features for the most important ones. Then we
applied the selected features as input to the LightGBM (Light Gradient
Boosting Machine) algorithm for classification. To assess the effectiveness of
the proposed method, the ChestX-ray8 dataset, which includes 1125 X-ray images
of the patient’s chest, was used. The proposed method achieved 98.54$\%$ and
91.11$\%$ accuracies in the two-class (COVID-19, Healthy) and multi-class
(COVID-19, Healthy, Pneumonia) classification problems, respectively. It is
worth mentioning that we have used Gradient-weighted Class Activation Mapping
(Grad-CAM) for further analysis.

###### Index Terms:

COVID-19, DenseNet169, MobileNet, LightGBM, Univariate Feature Selection,
GradCAM

##  I Introduction

Scientists discovered a family of viruses known as coronaviruses in 1965[1].
These viruses range from the cold virus to the COVID-19 virus. However, seven
different human-transmitted coronaviruses have been discovered. The latest
one, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was found
in Wuhan, China. World Health Organization (WHO) named the disease caused by
SARS-CoV-2 COVID-19. The common symptoms of COVID-19 disease include cough,
shortness of breath, sore throat, fever, and body pain [2].
COVID-19 can cause lung injury and death in severe cases if not treated. By
September 2020, more than 223 million people have contacted the COVID-19, and
over 4.6 million people have died due to COVID-19. The best method of
diagnosing coronavirus is the RT-PCR test. Because taking PCR test takes time,
doctors use the chest computed tomography scan (CT scan) for diagnosis [3].
This study uses the patient’s chest X-ray images and offers a novel method for
early discovery of COVID-19 infection.

This paper suggests MobileNet and DenseNet169 Deep Neural Networks (DNNs) for
feature extraction from the chest X-ray images. After combining the
characteristics extracted from both networks, the most significant features
are selected and given as input to the Light Gradient Boosting Machine
(LightGBM) classification algorithm to classify the images.

##  II Related Works

Ozturk et.al. [4] utilized the DarkNet network in 2020 for their research. The
training model is made up of 17 convolution layers. They obtained an accuracy
of 87.02$\%$ in the three-class classification and 98.8$\%$ in the two-class
classification. Their research was based on a dataset containing 1125 images,
including 500 lung images from healthy people, 500 lung images from people
with Pneumonia, and 125 images from people with COVID-19. Abbas et al. [5]
used the DeTraC deep convolutional neural network architecture to classify the
two classes, with an accuracy of 93.1$\%$. They used a dataset of images from
the chest of 126 people with COVID-19 disease and 80 healthy people to conduct
their research. Minaei and his colleagues[6] have used four models with
convolutional architecture to address the problem of both classes. They
include ResNet18, SqueezeNet, ResNet50 and DenseNet121.

Nasiri and Alavi [7] proposed a novel framework based on deep learning and
analysis of variance (ANOVA) feature selection method. They used XGBoost for
classification. Gunraj and colleagues [8], in their research on convolution
architecture networks, used COVID-Net and reported 93.3$\%$ accuracy. Hemdan
and his colleagues[9] obtained an accuracy of 90$\%$, after examining
different networks using the VGG19 and DenseNet201 networks. Narin[10] and
colleagues selected the ResNet50 neural network by testing five pre-trained
neural networks on three separate datasets, using this neural network to
achieve an average accuracy of 98$\%$ in a two-class problem. Barstugan[11]
extracted the image features from four other datasets and used the support
vector machine (SVM) for classification. Their research was reached 99.68$\%$
accuracy in the problem of two classes. Nasiri and Hassani [12] used the
DenseNet169 deep neural network to extract image features and the XGBoost
algorithm to classify them. They obtained an average accuracy of 98.24$\%$ for
the two classes and 98.70$\%$ for the three classes. In 2022, Hasani and
Nasiri [13] proposed COV-ADSX, an automated COVID-19 detection system, which
used the proposed algorithm in [12]. They used the Django web framework for
the implementation of the COV-ADSX.

Toğaçar[14] used X-ray chest images from three classes (COVID-19, Healthy,
Pneumonia) and trained the dataset using deep learning models (MobileNetV2,
SqueezeNet). Then used SVM for classification. The overall accuracy obtained
was 99.27$\%$. Ucar [15] used Bayes deep SqueezeNet to detect COVID-19 based
on chest X-ray images and showed an accuracy of 76.37$\%$ for the three
classes. Asnaoui[16] used X-ray chest images and demonstrated eight transfer
learning techniques to classify COVID-19 pneumonia using MobileNet-V2 and
Inception-V3 and achieved a 96$\%$ success rate in classification accuracy.
Ezzodding et al. [17] employed DenseNet169 and ANOVA feature selection to
extract and select features from chest X-ray images. They used LightGBM to
classify COVID-19 cases. Thejeshwar[18] used X-ray images of the chest and
suggested a KE Sieve Neural Network architecture. The proposed model achieves
98$\%$ accuracy.

##  III Prerequisites

The proposed method utilizes DenseNet169 and MobileNet deep neural networks.
It also employs LightGBM and the univariate feature selection algorithm. The
rest of this section briefly explains these methods.

###  III-A Deep Neural Networks

The limited amount of data leads to transitional learning to get acceptable
results. This research used pre-trained DenseNet169 and MobileNet models to
extract image features. The DenseNet169 network[19] is a network of large
DenseNet groups already trained in the ImageNet data set. The input of this
network is a single image.
By connecting each current layer to the previous layers of the network, this
network allows the network to be more compact, reducing the number of channels
in the network and computational complexity. The DenseNet169 network is 57 MB
in size with an accuracy of 93.2$\%$ on the ImageNet dataset. Google
researchers implemented MobileNet Deep Neural Network [20] to introduce a
lightweight deep neural network.
The Mobile Net Deep Neural Network contains a new type of convolution called
depth-wise separable convolution. This new convolution includes a deep
convolution filter and a point convolution filter. The deep convolution filter
performs a single convolution on each input channel, and the convolution
filter combines the deep convolution output points linearly with the 1 × 1
convolution. This network has a size of 16 MB and has reached 89.5$\%$
accuracy on the ImageNet dataset.

###  III-B Univariate Feature Selection

In univariate feature selection algorithm, $\chi^{2}$ test is used to find the
relation between the variables and the degree of dependency of each
independent variable. This test aims to establish the relationship between the
two variables. This test distinguishes the degree of independence of the two
variables. This test makes it possible to obtain the most useful features in
the production of classes.

###  III-C LightGBM

The LightGBM algorithm was first developed by part of Microsoft’s R$\&$D
team[21]. Although many researchers used XGBoost in their research [22, 23,
24, 25, 26, 27, 28], LightGBM is a more accurate decision tree algorithm than
the existing tree boosting algorithms, because LightGBM produces more
complicated trees. Although producing complex trees, LightGBM is high-speed
and uses low memory because it utilizes GOSS and EFB algorithms. Existence of
appropriate parameters which allows parameter modification prevents
overfitting of the model[21].

###  III-D Gradient-based One-Side Sampling (GOSS)

In GOSS algorithm the training data are ranked in descending order based on
their absolute gradient values. The data with larger gradients have a more
significant effect on the data amplification, in which high-gradient samples
are sub-sampled by factor a, and low-gradient data samples are sub-sampled by
factor b. Therefore, the subset with larger gradient forms subset A and
$b*(1-a)$ percent of the data is randomly sampled to form subset B.

By having segmented samples, we use (1) to evaluate variance gain of data
point j with respect to point d. In (1), $x_{i}$ is the $i^{th}$ instance of
the training dataset and $g_{i}$ is the negative gradient of the loss function
with respect to the $i^{th}$ data. In (1), $\frac{1-a}{b}$ is used to
normalize the sum of the gradients.

$\bar{V}_{j}(d)=\frac{1}{n}\left(\frac{\left(\sum_{x_{i}\in
A_{i}}g_{i}+\frac{1-a}{b}\sum_{x_{i}\in
B_{i}}g_{i}\right)^{2}}{nl(d)}+\frac{\left(\sum_{x_{i}\in
A_{i}}g_{i}+\frac{1-a}{b}\sum_{x_{i}\in B_{r}}g_{i}\right)^{2}}{nr(d)}\right)$

(1)

where $A_{l}$ is the subset of $A$ whose elements are smaller or equal than
$d$, and where $A_{r}$ is the subset of $A$ whose elements are greater than
$d$. Similarly $B_{l}$ and $B_{r}$ are formed based on set $B$. The definition
of $A_{l},B_{l},A_{r}$ and $B_{r}$ is given by:

$\displaystyle A_{l}=\\{x_{i}\in A:x_{i,j}\leq d\\},A_{r}=\\{x_{i}\in
A:x_{i,j}>d\\}$ (2) $\displaystyle B_{l}=\\{x_{i}\in B:x_{i,j}\leq
d\\},B_{r}=\\{x_{i}\in B:x_{i,j}>d\\}$

###  III-E Exclusive Feature Bundling (EFB)

In deep learning problems, we are confronted with dimensional data; these high
dimensions mean many features that reduce the speed of model training. The EFB
method reduces the number of features by integrating the data without losing
any feature. This process reduces the complexity of the features and speeds up
model training.

##  IV Proposed Method

DNNs are very common in feature extraction from medical images [29], therefore
we used DNNs in this study for feature extraction from chest X-rays.
DenseNet169 and MobileNet are the two DNNs that we used for image feature
extraction in this research. These networks are previously trained by ImageNet
database. Therefore both DenseNet169 and MobileNet can extract features of
each new received image. We combined the extracted features from both DNNs to
increase the classification accuracy. Because the number of features increases
after the combination, the feature selection algorithm is used to select those
features that have the most impact on class differentiation. Finally, the
selected features are given to the LightGBM for classification. Fig. 1
presents the overall framework for the proposed approach.

Figure 1: Framework of the proposed method. Pre-trained DenseNet169 and
MobileNet extract features separately. Then these features are combined
together to increase classification accuracy. Best features are selected to
improve the computational complexity of the algorithm. At the end LightGBM is
fed with selected features and classifies the new chest X-ray images. Figure
2: Sample Images in the Dataset

##  V Experimental Results

To evaluate the proposed method in this paper we used the publicly available
ChestX-ray8 image database created by Wang et al. [30]. ChestX-ray8 contains
total number of 1125 images, of which 500 are labeled Pneumonia, 500 are
labeled healthy, and 125 are labeled COVID-19. The total of 1125 images were
used to study the problem of the three classes (COVID-19, Pneumonia, Healthy),
and 625 images of the dataset were used to study the problem of the two
classes (COVID-19, Healthy).
Our research includes three steps. The first step is to extract the features
by means of pre-trained DNNs; the second step involves combining the features
extracted by DNNs and selecting best extracted features. The third step
consists of classifying the relevant images using the LightGBM classification
algorithm. A sample of images in the ChestX-ray8 database is shown in Fig. 2.

  * •

Step1: In this step, 12 pre-trained neural networks were evaluated, and using
each of them, image features were extracted and trained for each network of
the classifier. Finally, all results have been compared to one another. Due to
the accuracy of the LightGBM classifier, the networks that extracted the best
features were identified. This study chose DenseNet169 and MobileNet pre-
trained neural networks as feature extractors. To extract the features of
these neural networks, first, the size of all images has been modified to 244
$\times$ 244\. Subsequently, the image’s pixel values were normalized and
given as input to the deep neural network. Following the operation, the
DenseNet169 network extracted 1664 features, while the MobileNet network
extracted 1024 features.

  * •

Step2: To improve model accuracy, the characteristics extracted by both
networks are combined. At this point, a feature vector with 2688 elements is
obtained. We know that unnecessary features reduce the efficiency of the
model, furthermore, we need to reduce the number of features and extract the
most significant features to accelerate the training model.
The univariate feature selection method was used to select the feature, which
uses the test $\chi^{2}$ to choose the best K features. This test computes the
degree of independence of each attribute in relation to the classes. Based on
the $\chi^{2}$ test results we chose K to be 2000.

  * •

Step3: The input features of the LightGBM algorithm are indicated and the
classification is carried out. The reason for choosing this algorithm is due
to its high training speed and high accuracy. We selected the LightGBM
parameters by trial and error.

Table I gives the LightGBM classification parameters. The proposed method for
the three-class problem has a 91.11$\%$ accuracy. Furthermore, the average
accuracy of the two-class problem with 5-fold cross-validation is 98.54$\%$.
Tables II and III represent the results for problems with three and two
classes respectively. The results are compared with the results of Ozturk et
al. [4] and Nasiri and Hassani [12]. For a better comparison, in addition to
accuracy, we also reported sensitivity, specificity, precision and F1 score to
achieve a more comprehensive comparison. As shown in Table II, in the problem
of the three classes with respect to the accuracy, the F1 score and the
accuracy of our proposed method gives better results than the other methods.
With respect to specificity, the method presented by Nasiri and Hassani was
the most effective, and our proposed method and the method presented by Nasiri
and Hassani performed achieve the highest sensitivity.

In the case of two classes, based on the results in Table III, the sensitivity
of our proposed method gives the best results and reaches an average of
99.60$\%$, where the methodology presented by Ozturk et al. is the second. In
terms of specificity, the algorithm presented by Nasiri and Hassani
outperformes other methods and averaged 99.78$\%$. Although the results
obtained by the three articles are close to each other in terms of precision,
but the method presented by Nasiri and Hassani has had the best performance.
With respect to the F1 score and accuracy, the best performance is related to
the method proposed in this article. In both of these criteria, the algorithm
presented by Nasiri and Hassani outperformes the method presented by Ozturk et
al. It should be noted that our method, due to the use of LightGBM classifier
and lack of deep neural network training, has less computational complexity
than the method presented by Ozturk et al. Additionally, feature selection and
classification were very quickly compared to the Ozturk et al. method. In
Figures 3 and 4, the confusion matrix for three- and two-class problems is
shown. Table IV compares the results of the proposed method to other studies.

For further assessment of the proposed network, the gradient-based class
activation mapping (Grad-CAM) [31] was employed to depict the decision region
on a heatmap. Fig. 5 illustrates the class activation heatmaps and
superimposed images for three sample images. As can be observed, the proposed
method extracted appropriate features, and the model mainly focused on the
lung area. Radiologists can use superimposed images to examine the chest area
more precisely.

Figure 3: The Confusion Matrix (Three-Class Problem) TABLE I: The LightGBM
Parameter Settings

Parameter | Value
---|---
Learning rate ($\eta$) | 0.24
Number of Trees | 300
Maximum number of leaves | 105
Number of iterations of the algorithm | 250
Maximum tree depth | 7
Minimum number of leaves | 40

TABLE II: The comparison of the proposed method with other methods (Multi-class problem) Method | Performance |
---|---|---
|  Sensitivity |  Specificity |  Precision |  F1-score |  Accuracy
Proposed Method |  95.20 |  97.16 |  95.12 |  95.6 |  91.11
Ozturk et al. |  88.17 |  93.66 |  90.97 |  89.44 |  89.33
Nasiri & Hasani |  95.20 |  100 |  92.50 |  91.20 |  89.70
TABLE III: Comparison of the proposed method with other methods (Two-class problem) Performance | Different-Folds
---|---
|  Methods |  Fold 1 |  Fold 2 |  Fold 3 |  Fold 4 |  Fold 5 |  Average
Sensitivity |  Proposed Method |  100 |  98.97 |  99.06 |  100 |  100 |  99.60
Ozturk et al. |  100 |  96.42 |  90.47 |  93.75 |  93.18 |  95.13
Nasiri & Hasani |  95.20 |  95.40 |  96.70 |  81.40 |  91.40 |  92.08
Specificity |  Proposed Method |  100 |  88.88 |  100 |  86.95 |  92.00 |  93.56
Ozturk et al. |  100 |  96.42 |  90.48 |  93.75 |  93.18 |  95.30
Nasiri & Hasani |  100 |  100 |  100 |  89.90 |  100 |  99.78
Precision |  Proposed Method |  100 |  97.00 |  100 |  97.14 |  98.03 |  98.43
Ozturk et al. |  100 |  94.52 |  97.06 |  99.02 |  99.48 |  98.03
Nasiri & Hasani |  99.50 |  99.50 |  99.40 |  95.30 |  99.02 |  98.54
F1-score |  Proposed Method |  100 |  97.97 |  97.53 |  98.55 |  99.00 |  99.01
Ozturk et al. |  100 |  98.52 |  93.79 |  95.93 |  95.62 |  96.51
Nasiri & Hasani |  98.50 |  98.50 |  98.20 |  92.50 |  97.30 |  97.00
Accuracy |  Proposed Method |  100 |  96.80 |  99.20 |  97.60 |  98.40 |  98.40
Ozturk et al. |  100 |  97.60 |  96.80 |  97.60 |  97.60 |  98.08
Nasiri & Hasani |  99.20 |  99.20 |  99.20 |  95.20 |  98.40 |  98.24

(a) Fold 1

(b) Fold 2

(c) Fold 3

(d) Fold 4

(e) Fold 5

(f) All Data

Figure 4: The Confusion Matrix (Two-Class Problem) TABLE IV: Comparison of the proposed method with other DL-Based methods  Study |  Number of Samples |  Method Used |  Accuracy
---|---|---|---
Apostolopoulos et al.[32] |  1428 |  VGG19 |  93.48
Wang et al.[8] |  13645 |  COVID-Net |  92.40
Hemdan et al.[9] |  50 |  COVIDX-Net |  90.00
Sethy et al.[33] |  50 |  ResNet50+SVM |  95.38
Narin et al.[10] |  100 |  Deep CNN ResNet50 |  98.00
Ozturk et al.[4] |  625 |  DarkCOVIDNet |  98.08
1125 |  DarkCOVIDNet |  89.33
Nasiri & Hasani[12] |  625 |  DenseNet169+XGBoost |  98.23
1125 |  DenseNet169+XGBoost |  89.70
Proposed Method |  625 |  DenseNet169+MobileNet+LightGBM |  98.54
1125 |  DenseNet169+MobileNet+LightGBM |  91.11
Figure 5: The output of GradCAM for three sample images (a) Original image (b)
Class activation heatmap (c) Superimposed image.

##  VI Conclusion

This paper presents a novel deep learning based model for the early diagnosis
of COVID-19 using the patients chest X-rays. Our proposed method uses
DenseNet169 and MobileNet deep neural networks to extract image features, then
selects some of the most essential features using the feature selection
method, and finally classifies the images using the LightGBM algorithm. Our
proposed algorithm achieves 98.54$\%$ accuracy in the classification of two
classes of healthy people and COVID-19 patients, and achieves 91.11$\%$ in the
classification of three classes (Pneumonia, Healthy, COVID-19).

The evaluation results of our proposed method show that this method has a
higher speed and accuracy than previous research in this field due to the use
of the LightGBM algorithm.Our novel classification method can help health care
providers to diagnose COVID-19 disease which is essential for effective
treatment plans. Future work includes verifying the classification results of
our method with medical experts.

##  VII Code Availability

The source code of the proposed method required to reproduce the results is
available at the public Github
repository111https://github.com/gha7all/Covid19-Automated-Detection.

## References

  * [1] E. Mahase, “Covid-19: First coronavirus was described in the bmj in 1965,” 2020\.
  * [2] T. Struyf, J. J. Deeks, J. Dinnes, Y. Takwoingi, C. Davenport, M. M. Leeflang, R. Spijker, L. Hooft, D. Emperador, J. Domen _et al._ , “Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has covid-19,” _Cochrane Database of Systematic Reviews_ , no. 2, 2021.
  * [3] T. Singhal, “A review of coronavirus disease-2019 (covid-19),” _The indian journal of pediatrics_ , vol. 87, no. 4, pp. 281–286, 2020.
  * [4] T. Ozturk, M. Talo, E. A. Yildirim, U. B. Baloglu, O. Yildirim, and U. R. Acharya, “Automated detection of covid-19 cases using deep neural networks with x-ray images,” _Computers in biology and medicine_ , vol. 121, p. 103792, 2020.
  * [5] A. Abbas, M. M. Abdelsamea, and M. M. Gaber, “Classification of covid-19 in chest x-ray images using detrac deep convolutional neural network,” _Applied Intelligence_ , vol. 51, no. 2, pp. 854–864, 2021.
  * [6] S. Minaee, R. Kafieh, M. Sonka, S. Yazdani, and G. J. Soufi, “Deep-covid: Predicting covid-19 from chest x-ray images using deep transfer learning,” _Medical image analysis_ , vol. 65, p. 101794, 2020.
  * [7] H. Nasiri and S. A. Alavi, “A Novel Framework Based on Deep Learning and ANOVA Feature Selection Method for Diagnosis of COVID-19 Cases from Chest X-Ray Images,” _Computational Intelligence and Neuroscience_ , vol. 2022, p. 4694567, 2022.
  * [8] H. Gunraj, L. Wang, and A. Wong, “Covidnet-ct: A tailored deep convolutional neural network design for detection of covid-19 cases from chest ct images,” _Frontiers in medicine_ , p. 1025, 2020.
  * [9] E. E.-D. Hemdan, M. A. Shouman, and M. E. Karar, “Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images,” _arXiv preprint arXiv:2003.11055_ , 2020.
  * [10] A. Narin, C. Kaya, and Z. Pamuk, “Automatic detection of coronavirus disease (covid-19) using x-ray images and deep convolutional neural networks,” _Pattern Analysis and Applications_ , vol. 24, no. 3, pp. 1207–1220, 2021\.
  * [11] M. Barstuğan, U. Özkaya, and Ş. Öztürk, “Coronavirus (covid-19) classification using ct images by machine learning methods,” in _CEUR Workshop Proceedings_ , vol. 2872, 2021.
  * [12] H. Nasiri and S. Hasani, “Automated detection of COVID-19 cases from chest X-ray images using deep neural network and XGBoost,” _Radiography_ , 2022\.
  * [13] S. Hasani and H. Nasiri, “COV-ADSX: An Automated Detection System using X-ray images, deep learning, and XGBoost for COVID-19,” _Software Impacts_ , vol. 11, p. 100210, 2022.
  * [14] M. Toğaçar, B. Ergen, and Z. Cömert, “Covid-19 detection using deep learning models to exploit social mimic optimization and structured chest x-ray images using fuzzy color and stacking approaches,” _Computers in biology and medicine_ , vol. 121, p. 103805, 2020.
  * [15] F. Ucar and D. Korkmaz, “Covidiagnosis-net: Deep bayes-squeezenet based diagnosis of the coronavirus disease 2019 (covid-19) from x-ray images,” _Medical hypotheses_ , vol. 140, p. 109761, 2020.
  * [16] K. El Asnaoui, Y. Chawki, and A. Idri, “Automated methods for detection and classification pneumonia based on x-ray images using deep learning,” in _Artificial intelligence and blockchain for future cybersecurity applications_. Springer, 2021, pp. 257–284.
  * [17] M. Ezzoddin, H. Nasiri, and M. Dorrigiv, “Diagnosis of COVID-19 Cases from Chest X-ray Images Using Deep Neural Network and LightGBM,” in _2022 International Conference on Machine Vision and Image Processing (MVIP)_. IEEE, 2022, pp. 1–7.
  * [18] S. S. Thejeshwar, C. Chokkareddy, and K. Eswaran, “Precise prediction of covid-19 in chest x-ray images using ke sieve algorithm,” _medRxiv_ , 2020\.
  * [19] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 4700–4708.
  * [20] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” _arXiv preprint arXiv:1704.04861_ , 2017.
  * [21] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu, “Lightgbm: A highly efficient gradient boosting decision tree,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [22] Z. Akhavan, M. Esmaeili, D. Sikeridis, and M. Devetsikiotis, “Internet of things-enabled passive contact tracing in smart cities,” _Internet of Things_ , p. 100397, 2021.
  * [23] S. C. Chelgani, H. Nasiri, and M. Alidokht, “Interpretable modeling of metallurgical responses for an industrial coal column flotation circuit by XGBoost and SHAP-A “conscious-lab” development,” _International Journal of Mining Science and Technology_ , vol. 31, no. 6, pp. 1135–1144, 2021\.
  * [24] S. Chehreh Chelgani, H. Nasiri, and A. Tohry, “Modeling of particle sizes for industrial HPGR products by a unique explainable AI tool- A “Conscious Lab” development,” _Advanced Powder Technology_ , vol. 32, no. 11, pp. 4141–4148, 2021.
  * [25] H. Nasiri, A. Homafar, and S. Chehreh Chelgani, “Prediction of uniaxial compressive strength and modulus of elasticity for Travertine samples using an explainable artificial intelligence,” _Results in Geophysical Sciences_ , vol. 8, p. 100034, 2021.
  * [26] A. Bhattacharyya, D. Bhaik, S. Kumar, P. Thakur, R. Sharma, and R. B. Pachori, “A deep learning based approach for automatic detection of covid-19 cases using chest x-ray images,” _Biomedical Signal Processing and Control_ , vol. 71, p. 103182, 2022.
  * [27] R. Kumar, R. Arora, V. Bansal, V. J. Sahayasheela, H. Buckchash, J. Imran, N. Narayanan, G. N. Pandian, and B. Raman, “Classification of covid-19 from chest x-ray images using deep features and correlation coefficient,” _Multimedia Tools and Applications_ , pp. 1–25, 2022.
  * [28] R. Fatahi, H. Nasiri, E. Dadfar, and S. Chehreh Chelgani, “Modeling of energy consumption factors for an industrial cement vertical roller mill by SHAP-XGBoost: a” conscious lab” approach,” _Scientific Reports_ , vol. 12, no. 1, pp. 1–13, 2022.
  * [29] J. Kim, V. D. Calhoun, E. Shim, and J.-H. Lee, “Deep neural network with weight sparsity control and pre-training extracts hierarchical features and enhances classification performance: Evidence from whole-brain resting-state functional connectivity patterns of schizophrenia,” _Neuroimage_ , vol. 124, pp. 127–146, 2016.
  * [30] X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 2097–2106.
  * [31] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra, “Grad-cam: Visual explanations from deep networks via gradient-based localization,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 618–626.
  * [32] I. D. Apostolopoulos and T. A. Mpesiana, “Covid-19: automatic detection from x-ray images utilizing transfer learning with convolutional neural networks,” _Physical and engineering sciences in medicine_ , vol. 43, no. 2, pp. 635–640, 2020.
  * [33] P. K. Sethy, S. K. Behera, P. K. Ratha, and P. Biswas, “Detection of coronavirus disease (covid-19) based on deep features and support vector machine,” _International Journal of Mathematical, Engineering and Management Sciences_ , vol. 5, no. 4, 2020.