#  CloudSense: A Model for Cloud Type Identification using Machine Learning
from Radar data

Mehzooz Nizar Indian Institute of Tropical Meteorology, Ministry of Earth
Sciences, Pune, India Cochin University of Science and Technology, Kochi,
India  Jha K. Ambuj Indian Institute of Tropical Meteorology, Ministry of
Earth Sciences, Pune, India Manmeet Singh Indian Institute of Tropical
Meteorology, Ministry of Earth Sciences, Pune, India Jackson School of
Geosciences, The University of Texas at Austin, Austin, Texas, USA  Vaisakh
S.B Indian Institute of Tropical Meteorology, Ministry of Earth Sciences,
Pune, India G. Pandithurai Indian Institute of Tropical Meteorology, Ministry
of Earth Sciences, Pune, India

ABSTRACT The knowledge of type of precipitating cloud is crucial for radar
based quantitative estimates of precipitation. We propose a novel model called
CloudSense which uses machine learning to accurately identify the type of
precipitating clouds over the complex terrain locations in the Western Ghats
(WGs) of India. CloudSense uses vertical reflectivity profiles collected
during July-August 2018 from an X-band radar to classify clouds into four
categories namely stratiform,mixed stratiform-convective,convective and
shallow clouds. The machine learning(ML) model used in CloudSense was trained
using a dataset balanced by Synthetic Minority Oversampling Technique (SMOTE),
with features selected based on physical characteristics relevant to different
cloud types. Among various ML models evaluated Light Gradient Boosting Machine
(LightGBM) demonstrate superior performance in classifying cloud types with a
BAC of 0.8 and F1-Score of 0.82 . CloudSense generated results are also
compared against conventional radar algorithms and we find that CloudSense
performs better than radar algorithms. For 200 samples tested, the radar
algorithm achieved a BAC of 0.69 and F1-Score of 0.68, whereas CloudSense
achieved a BAC and F1-Score of 0.77. Our results show that ML based approach
can provide more accurate cloud detection and classification which would be
useful to improve precipitation estimates over the complex terrain of the WG.

Keywords: Machine learning, Precipitating clouds, Doppler weather radar,
Western Ghats, LightGBM

##  1 INTRODUCTION

Clouds are an integral component of convection and precipitation and play a
vital role in modulating the global circulation, Earth’s radiative budget and
hydrological cycle. Different cloud types are associated with different
microphysical and radiative properties which influence the vertical
distribution of heating in the atmosphere (Houze 1982; Houze 1997; Schumacher
and Houze 2003). Improved knowledge of cloud types can improve weather and
climate predictions through better representation of clouds in atmospheric
models. From a regional perspective, accurate detection and classification of
precipitating clouds are crucial for accurate quantitative precipitation
estimation (QPE) which is widely used for extreme weather forecasting, climate
studies and hydrological applications (Steiner and Houze, 1997; Gourley and
Vieux 2005; Kühnlein et al.,2014; Thompson et al., 2015; Arulraj and
Barros,2019). QPE algorithms provide rain estimates based on relationships
specific to precipitating cloud type (Rao et al. 2001; Auipong and Trivej
2018) and therefore, any misclassification of precipitating clouds will impact
the accuracy of QPE.

Earlier studies classified precipitating clouds either using space-borne
active and passive sensors (Inoue 1987; Sassen and Wang 2008; Subrahmanyam and
Kumar 2013; So and Shin 2018) or ground-based radars and gauges (Penide et
al., 2013; Loh et al., 2020; Zuo et al.,2022). Satellite-based classification
though has wider spatial coverage but has limitations due to coarser spatial
and temporal sampling. Broadly, precipitating clouds are categorized as
convective and stratiform types (Houze 2014). Convective clouds are
characterised by strong vertical air currents, small areal coverage, and high
rainfall intensities while the converse is associated with stratiform clouds.

Most techniques apply a threshold value to distinguish between different
precipitation types. For example, when precipitation intensity recorded by
rain gauge exceeds a certain threshold, the precipitation cloud is considered
convective but otherwise stratiform (Austin and Houze 1972; Houze 1973; Testud
et al. 2001). Some studies used threshold-based techniques to raindrop spectra
measured by surface disdrometer to classify precipitating systems (Waldvogel
1974; Tokay and Short 1996; Lavanya and Kirankumar 2021). However, these
methods provide point measurements and can easily misinterpret the areal
extent of precipitation in a convective cloud when there is weak precipitation
intensity in its vicinity (Ran et al. 2021). Further, these methods rely only
on rainfall measured near the surface and lack information on the vertical
structure of clouds.

Ground-based weather radars are excellent tools for cloud and precipitation
observations (Battan 1973; Fabry 2015). They provide 3D structures of clouds
and rainfall at finer spatio-temporal scales which can be utilised for more
accurate classification of precipitating systems (Xiao and Liu 2007). Ground-
based radars are also used for calibration and validation of space-based
precipitation products (Zhong et al. 2017; Biswas and Chandrashekhar 2018). A
Doppler radar measures three base data parameters, viz., reflectivity, Doppler
velocity and spectrum width. Using radar reflectivity threshold, Churchill and
Houze (1984) identified the core of the convective precipitation and separated
convective and stratiform regions within cloud clusters. They used a fixed
radius of influence to identify the convective area of the core. The technique
tends to overestimate the extent of the area designated as convective
precipitation. Numerous echoes, despite retaining evident bright band
characteristics, are categorized as convective. Steiner et al. (1995) modified
this method using a variable radius of influence and used criteria of
intensity, peakedness and surrounding area. However, the major limitation of
Steiner’s algorithm was that it does not account for the vertical storm
structure information which could improve the accuracy of precipitation
classification. Later studies modified Steiner’s algorithm using additional
information from the 3-D precipitation field and showed that the accuracy of
the algorithm is improved (DeMott et al. 1995; Biggerstaff and Listemaa 2000).
In contrast to studies which used only reflectivity, Williams et al. (1995)
classified precipitating clouds through the synergy of vertical structures of
reflectivity, Doppler velocity and spectrum width and illustrated the
performance of their algorithm. However, the basic problem with conventional
radar-based algorithms is sensitivity to the adopted threshold of a radar
parameter. If the threshold reflectivity is set to be low, it is likely that
stratiform regions are misclassified as convective. On the other hand, if the
threshold is high, transitioning convective regions or mixed stratiform-
convective areas may be identified as stratiform. Cloud structures are often
complex which makes it difficult to distinguish them into different types. For
example, a convective cloud in transition to stratiform exhibits mixed
characteristics and these algorithms may fail in categorising them. Further,
the threshold value is not always constant and needs to be adjusted and fine-
tuned for different geographical locations, seasons and with changes in
calibration constants of the radar.

In an attempt to solve this long-standing problem, studies have been conducted
in the past that used machine learning (ML) models for classifying
precipitating clouds into convective and stratiform (Anagnostou 2004; Lazri
and Ameur 2018; Wang H et al., 2019; Yang et al., 2019; Wang Y et al., 2021;
Ran et al., 2021; Ghada W et al., 2022). Wang H et al., 2019 developed a deep
learning method of precipitation cloud type identification, based on the data
of a dual-polarization Doppler weather radar. Anagnostou 2004 used a neural
network approach to classify reflectivity measurements from ground-based
volume scanning radar into convective and stratiform precipitation types.
Ghada W et al., 2022, using a total of 20,979 min of rain data measured by a
Micro Rain Radar (MRR) and built seven types of ML models for stratiform and
convective rain type classification. Their results indicate that tree-based
ensembles such as xgboost and random forest, performed better than simpler
models.

Over the Indian region, ML-based classification of precipitating clouds is
either lacking or scarce. In this work, we develop a model using ML named as
“CloudSense” and attempt to classify cloud types from radar reflectivity
profiles in the Western Ghats (WG) of India. This study primarily uses X-band
radar data collected over a high altitude cloud physics laboratory (HACPL) at
Mahabaleshwar (17.92° N, 73.66° E, $\sim$ 1.38 km MSL) in the WG (Fig 1). The
radar site (referred to as MDV) is located at a distance of 26 km from HACPL
and the data collected by the radar during July-August 2018 have been used for
analysis. The WG is one of the heaviest rainfall-receiving regions ($\sim$
6000 mm) during the Indian summer monsoon(ISM) (Nandargi S, Mulye S 2012) and
forms an ideal test bed for cloud and precipitation studies. The topography of
the WG exerts an influence on monsoon rainfall in a manner that amplifies
rainfall on the windward side and inhibits it on the leeward side. Due to
complex topographical features, in-situ measurements of clouds and rainfall
are scarce in this region. The rainfall in the WG is contributed by cloud
systems such as stratiform, mixed stratiform-convective, convective and
shallow convective clouds ( Konwar et al., 2012a; Maheskumar et al., 2014;
Deshpande et al., 2015; Das et al., 2017). Of all clouds, shallow convective
clouds occur most frequently and contribute majorly to the total precipitation
during ISM (Konwar et al., 2014). Earlier, Das et al., 2017 used MRR and
X-band radar to study rain DSD characteristics for different cloud types over
HACPL.

In the present work, we use several ML models to classify cloud types from
vertical profiles of reflectivity (VPRs) and select an optimal ML model to
construct CloudSense. The model-generated results are compared against
conventional radar algorithms and test cases are presented. The paper is
organized as follows. Data and methodology are given in Sect. 2, and Sect. 3
contains results followed by conclusions in Sect. 4.

##  2 DATA AND METHODOLOGY

###  2.1 Data

The Indian Institute of Tropical Meteorology (IITM), Pune, has deployed a
magnetron‐based mobile dual-polarized X-band Doppler radar at Mandhardev
(18.04°N, 73.86°E and $\sim$ 1.3km MSL), a remote location in the WG. The
radar operates at $\sim$ 9.535 GHz with a peak power of about 200 kW. The
technical speciﬁcations of the X-band radar are given in Table 1. The scan
strategy of the radar consists of Range Height Indicator (RHI) and volume scan
modes, repeated every 12 minutes. Each volume scan consists of plan position
indicator (PPI) scans at 18 elevation angles from 0.5° to 90° and takes
approximately 12 minutes. On the other hand, the RHI scan is executed along
239° azimuth which points over the HACPL site (see Fig. 1).

The most important parameter provided by a weather radar is the equivalent
reflectivity factor (Z, henceforth). Z is proportional to the 6th power of
hydrometeor size in the scattering volume (Marshall et al. 1947) . The
relation between Z and raindrop size is given by

$Z=\int_{0}^{\infty}N(D)D^{6}dD$ (1)

where D is the raindrop diameter and N(D) is the precipitation particle size
distribution.

In this work, Z data collected during July-August 2018 has been used. Several
quality checks and corrections have been done for Z before analysis. Mainly
VPR data taken from RHI scans have been used in this study (Fig. 2a). Over
HACPL, zenith pointing radar observations are lacking and therefore, vertical
profiles of Doppler velocity (DV) and spectrum width (W) are not available.
PPI data corresponding to 85º elevation can serve as pseudo zenith pointing
observations and therefore, used here to obtain vertical profiles of DV and W
over MDV(Fig.2b).

###  2.2 Methodology

####  2.2.1 Processing of Z data

Z data collected by the radar is in polar coordinates. Therefore, RHI data is
converted to 2 D Cartesian coordinates with a grid spacing of 1 km in the
horizontal and 100 m in the vertical. The data below 0.5 km are discarded to
avoid noise and clutter near the ground. VPRs with Z$<$5 dBZ at 1.5 km above
the ground are removed to avoid weak and noisy signals and are considered non-
precipitating in nature. Vertical profiles of Z, DV and W are directly taken
from 85º PPI data by averaging the respective profiles. Contoured frequency by
altitude diagrams (CFAD) of radar parameters are also constructed (Yuter and
Houze 1995). The bin size considered for CFADs is 5 dBZ for Z and 1 m/s for DV
and W. The height interval is 0.1 km between 0.5 and 11 km. The number of
occurrences at each level is standardized by the mean and standard deviation
considering all reflectivity bins among all vertical levels.

####  2.2.2 Labelling

The precipitating clouds are classified into four categories: shallow,
convective, stratiform and mixed stratiform-convective (also referred as mixed
clouds) (Williams et al. 1995). Das et al. (2017) have classified clouds over
HACPL into these four categories based on VPR data collected using MRR and the
X-band radar. Here, we have adopted the VPR based definitions of cloud types
described in their study. Note that here the cloud profiles are grouped not
through a conventional approach based on radar algorithms rather they are
manually labelled into these 4 categories after careful visual inspection of
each VPR. Fig 3 shows the logical flow diagram of the classification
algorithm. The first criterion is to check whether hydrometeors exist well
above the melting layer. The climatological melting layer over the study
region has been reported at 4 km (Kalapureddy et al. 2023); if the cloud top
height is below 4 km, the cloud is classified as shallow otherwise the VPR is
scrutinized for a bright band signature. The radar bright band is a prominent
feature of stratiform precipitation and appears as a narrow region of enhanced
Z just below the 0 C level in radar returns (Marshall et al. 1947; Battan
1973). If no bright band is detected, the cloud is placed in the convective
category. Alternatively, if a bright band is present, further examination
focuses on its strength. Mature stratiform clouds exhibit well-defined bright
band features, while mixed clouds display embedded convection along with a
less pronounced bright band (Das et al. 2017). Therefore, VPRs with strong and
prominent bright band signatures are recognized as stratiform whereas those
lacking this feature are labelled as mixed stratiform-convective clouds. These
clouds being manually labelled are considered as true cloud types and the
model results are tested against them.

####  2.2.3  CloudSense

The CloudSense utilizes a ML model for the classification of defined cloud
types except shallow clouds. Shallow clouds are not considered in the ML model
algorithm due to the fact that they are generally confined below the melting
layer (Fig 4d) which makes their identification $>$95 % accurate. Shallow
clouds are obtained by adopting a flexible height threshold of 4 km. For
example, if a cloud crosses the 4 km melting layer but only by a marginal
degree, it will be identified as shallow and therefore, no rigid threshold
criterion is required. For other 3 cloud types, the ML model is trained using
VPR samples and then tested for performance. Fig 4 shows the architecture of
CloudSense.

The number of VPR samples for 3 cloud types is 1492 which includes 808
stratiform, 240 mixed and 444 convective clouds. These samples have been used
for training the ML model. Stratiform clouds account for 54 % of the total
occurrences which is in agreement with the observations of Das et al. 2017.
However, this leads to an imbalanced dataset. To address this, we use a
Synthetic Minority Oversampling Technique (SMOTE) (Chawla et al., 2002, Barua
et al., 2011; Zhu et al., 2017; Elreedy and Atiya 2019) to make the dataset
balanced by increasing the number of samples of convective and mixed clouds
equal to the majority class(see supplementary Fig 1). Fig. 5 shows the CFADs
of Z for all 4 cloud types segregated through manual labelling after
oversampling.

Different cloud types have different physical processes which are linked to
their vertical structures. For training a ML model, it is important to select
features specific to each cloud type which are used as predictors for
modelling. The performance of the ML model depends on these features and
therefore, should be carefully selected. For identifying features, the
following 3 zones are considered for the 3 types of clouds:

a) low-level (1-2 km): This zone is associated with rain drops and shows
higher Z in case of convective clouds due to the presence of bigger rain
drops.

b) mid-level (3-5 km): This zone is associated with melting layer, has mixed
phase hydrometeors and shows a prominent bright band peak for mature
stratiform clouds. Case studies conducted over the WG have found bright band
region situated between $\sim$3-5 km (Jha et al. 2019; Devisetty et al. 2019).

c) high-level (6-7 km): This zone is above the melting layer and mostly
populated with ice phase particles. For mixed clouds, higher turbulence is
observed in this zone (Williams et al. 1995); however, Z does not show any
contrasting features.

A total of 41 predictors are considered from the defined 3 zones.

####  2.2.4 Tuning and Evaluating the model

ML algorithms have several hyperparameters which determine the overall
performance of the algorithm. Different hyperparameter values produce
different model parameter values for a given data set. These values should be
tuned so that the combination of hyper-parameters maximizes the model’s
performance, minimizing a predefined loss function to produce better results
with fewer errors(Probst et al., 2018; Weerts et al., 2020).

Since all four types of clouds will be unevenly distributed in the unseen test
data, we use several evaluation metrics such as balanced accuracy(BAC),
precision, recall and F1-score which are commonly used in classification tasks
( Silva et al., 2020; Nasir et al., 2022). A confusion matrix (Fig 6) of size
n x n associated with a classifier shows the predicted and actual
classification, where n is the number of different classes (Visa et al.,
2011).

##  3 RESULTS AND DISCUSSIONS

###  3.1  Evolution case of a convective system

Figures 7, 8 and 9 depict the temporal evolution of a convective cloud into
stratiform precipitation over MDV. This rainfall event occurred on 26 June
2018 and was associated with a mesoscale convective system (MCS). An MCS
comprises of mainly convective and stratiform regions and has a life cycle
consisting of formation, maturity and dissipation (Houze 2004). A region of
transitional clouds also exist in the MCS which forms when convective clouds
begin to dissipate and merge into the stratiform anvil cloud (Smull and Houze
1987; Williams et al. 1995) . Here, initially, a fully developed convective
cloud is seen (Fig. 7a) which starts to decay and transitions to mixed
stratiform-convective system (Fig. 7b). At this stage, a local maximum in Z
(indicative of bright band) starts to appear between 3 and 4 km height levels
and become more pronounced in Fig. 7c. This suggests the gradual dissipation
of convective features and shift towards stratiform cloud. The bright-band
signature becomes well-marked in Fig. 7d signifying the transition to
stratiform system is complete. Next, the stratiform system matures and
eventually diminishes (Fig. 7e,f).

Corresponding vertical profiles of DV and W are shown in Fig. 8 and Fig. 9
respectively. During the transition phase, a gradient in DV emerges at bright
band levels (3-4 km) (Fig. 8b,c) due to the increase in fall velocities of
rain drops toward the end of melting process (Battan 1973). This gradient
increases and peaks for mature stratiform system (Fig. 8d) when DV rises
sharply from 2 at 3.5 km to 9 m/s at 2.5 km. In later stages, DV decreases
with the dissipation of stratiform cloud (Fig. 8e,f). In case of W, enhanced
values around 2-3 m/s are noted above the bright band during convective and
transition phases indicating higher turbulence (Fig. 9a,b,c). Later, W
decreases at these levels for stratiform system (Fig. 9d, e, f ). Williams et
al. (1995) also noticed enhanced turbulence above the melting level (see their
Fig. 6) in case of mixed clouds and reasoned that large W result from
convective updrafts and downdrafts occurring at these height levels which is
absent in stratiform system. An increase in W is also seen just below the
bright band during the transition and stratiform stages which is associated
with the increasing drop size as hydrometeors transform shape as they fall
through the melting layer (Williams et al 1995).

###  3.2 CloudSense Results

We trained 7 ML models for this multi-classification task using the training
dataset described in Section 2.2.3. Among these, 5 of them were hyper-
parameter tuned models and 2 of them were not-tuned models. The ML algorithms
used are Decision tree, Naïve Bayes, xgboost (Extreme Gradient Boosting),
catboost (Categorical Boosting), KNN (K Nearest Neighbours), Random Forest and
Light GBM (Light Gradient Boosting Machine). The test set consists of 400
clouds (stratiform-140, mixed-77, convective-99, shallow -84) taken over the
HACPL and MDV. The overall results obtained on the test set from the models
are given in Table 2.

It can be seen that hyper-parameter tuned models perform better than not-tuned
models for this classification problem (Table 2). This is in agreement with
the results of Ghada W et al., 2022. LightGBM model gives the best results out
of the 5 tuned models with a BAC of 0.80 and F1-Score of 0.82. Therefore, this
ML model is considered for CloudSense. Table 3 and Table 4 provide the
classification report of CloudSense and the tuned hyper-parameter values for
the LightGBM model.

Shallow clouds, as mentioned earlier, are correctly predicted with highest
percentage and show the highest scores for precision, recall and F1-Score.
Stratiform clouds show an F1-Score of 0.84 followed by convective clouds and
mixed clouds with 0.78 and 0.66 respectively. This shows that the distribution
of stratiform clouds is better understood by the model than convective and
mixed clouds. To get the correctly classified and misclassified number of
clouds by the model we analyse the confusion matrix (Fig 10).

Fig. 10 shows that 123 stratiform clouds (88 %), 45 mixed clouds (58 %), 80
convective clouds (81 %), and 80 shallow clouds (95%) have been accurately
identified and classified by the model. Although the number of clouds
correctly predicted by the model is considerable, there are instances of
misclassifications. Specifically, 10 convective clouds have been classified as
stratiform, and 12 stratiform clouds have been incorrectly classified as
convective clouds. Various factors constrain the performance of the model such
as limited sample size, data imbalance, relevance of features selected for
training the ML model and other external factors. To understand the nature of
misclassifications by the model, we test the model using a RHI scan conducted
by the radar at 3:52 UTC on July 15, 2018 (Fig.11).

Almost 75 % of the precipitating cloud profiles have been correctly identified
and classified. While shallow clouds have been predicted with 100 % accuracy,
there are some instances of misclassification with other cloud types. The
misclassifications primarily involve stratiform and convective clouds being
predicted as mixed stratiform-convective clouds, and vice versa. Notably, no
convective cloud is predicted as stratiform or vice versa. Classifying a cloud
type accurately is an important step in QPE. A stratiform cloud is considered
to produce low intensity rain while a convective cloud is associated with
heavy rain; rainfall from a mixed cloud is moderate in nature. Thus, these
misclassifications directly impact the accuracy of QPE. However, the impact is
relatively less if a stratiform/convective cloud is categorized as mixed and
vice versa. On the other hand, misclassifying a convective cloud as stratiform
can have severe consequences, leading to forecasts with significantly
underestimated rainfall. It may be noted that misclassification of this nature
has not been observed here; however, it cannot be completely ruled out for
other cases. In summary, the model produced cloud types have instances of
misclassifications but with relatively less impact and therefore, the
performance of the model may be considered satisfactory.

###  3.3  CloudSense versus radar algorithm output

In this section, we compare the statistics of CloudSense results with those of
a conventional radar algorithm. Williams et al. (1995) have utilized vertical
profiles of Z, DV and W collected using a vertically pointing 915 MHz wind
profiler radar to classify clouds into the four types.

They used 3 criteria to distinguish the 4 cloud types (see their Fig 4):
melting layer signature, turbulence above the melting level and hydrometeors
above the melting level. A set of thresholds were specified by them for
maximum Doppler velocity gradient (DVG) between the height levels of 3.5 and 5
km and maximum spectral width (MSW) above 7 km.

In this work, we have adapted their methodology and adjusted their algorithm
to account for the differences in the observing instrument and the region of
study. For example, their 915 MHz wind profiler radar is sensitive to both
turbulence in clear air and precipitating hydrometeors while the X-band radar
used here is sensitive to precipitating particles only and provides data at
comparatively higher resolution. Additionally, there are differences in
melting layer altitudes at both sites. A comparison of the CFADs of Z, DV and
W constructed over MDV (Fig. 12) with those of Williams et al. (1995) (their
Fig. 3) highlight these differences. Considering these distinctions, we set
thresholds of DVG between height levels of 2.8-4 km and that of MSW between
4-7 km.

A total of 200 VPRs collected from the MDV site with 50 profiles each for all
four cloud types are taken for testing the algorithm. We tested the
sensitivity of classification results for a set of threshold values for DVG
and MSW (refer Table 1 in supplementary document) and found that a DVG $>$ 2.5
m s-1 km-1 and MSW $>$ 2 m/s yielded the best results. Corresponding to this
threshold, the evaluation metrics are presented in Table 5 and 6 for radar
algorithm and CloudSense model, respectively.

The radar algorithm exhibited a BAC of 0.69 and F1-Score of 0.68, whereas
CloudSense achieved a BAC and F1-Score of 0.77, indicating that the CloudSense
model outperforms the threshold-based radar algorithm. Note that as shallow
clouds are classified using the same criteria for both the model and radar
algorithm, the classification results are similar for both.

##  4 SUMMARY AND CONCLUSIONS

This paper presents an initial attempt to develop an accurate cloud
classification model from radar data with an ML-based approach which is
completely lacking over the Indian region. We proposed a novel model called
CloudSense which uses machine learning (ML) for identifying precipitating
cloud types over the WGs. The clouds are classified into stratiform, mixed
stratiform-convective, convective and shallow. The model used vertical
reflectivity profiles (VPRs) collected from an X-band DWR located at
Mandhardev (18.04°N, 73.86°E and $\sim$ 1.3km MSL) in the WGs. The CloudSense
model classifies shallow clouds using flexible threshold height of the melting
layer whereas other 3 cloud types are classified using ML algorithms. A
typical case showing the evolution of a convective system to stratiform
precipitation was discussed and microphysical properties of the clouds were
studied to select the suitable features to train the ML algorithm. 7 ML
algorithms were trained and tested using the VPRs. Among all ML models, Light
GBM, a tree-based learning algorithm, showed the best test results with a BAC
of 0.8 and F1-Score of 0.82. The performance of CloudSense was compared
against a conventional threshold-based radar algorithm described in by
Williams et al 1995. The radar algorithm achieved a BAC of 0.69 and F1-Score
of 0.68 while the model exhibited a BAC and F1-Score of 0.77 for 200 samples
tested, thus showing that CloudSense can provide more accurate detection and
classification of cloud types.

Future plan is to rigorously train this model using more samples and develop a
more robust model for improved classification of clouds over the WG. Similar
studies can be carried using other radars over other regions to develop
accurate cloud classification models which would be useful to improve the
radar based quantitative estimates of precipitation.

Acknowledgements: The Indian Institute of Tropical Meteorology (IITM), Pune,
is an autonomous organization, fully funded by the Ministry of Earth Sciences
(MoES), Government of India. The authors extend their gratitude to the
Director, IITM for providing unwavering support and encouragement throughout
this research. Author Mehzooz Nizar would love to thank Meenu R. Nair, Adithiy
Raghunathan, his professors at Cochin University of Science and Technology,
the Open Radar Science community and all his workmates at IITM for their
invaluable support and help. The authors gratefully acknowledge the dedicated
efforts of the technical staff at HACPL and the radar site for their diligent
work in observations and maintenance.

Funding: This research did not receive any specific grant from funding
agencies in the public, commercial, or not-for-profit sectors.

REFERENCES

Anagnostou, E.N., 2004. A convective/stratiform precipitation classification
algorithm for volume scanning weather radar observations. Meteorol. Appl. 11,
291–300.

Arulraj, M., Barros, A.P., 2019. Improving quantitative precipitation
estimates in mountainous regions by modelling low-level seeder-feeder
interactions constrained by Global Precipitation Measurement Dual-frequency
Precipitation Radar measurements. Remote Sens. Environ. 231, 111213.

Auipong, N., Trivej, P., 2018. Study of Z-R relationship among different
topographies in Northern Thailand. J. Phys. Conf. Ser. 1144, 012098.

Austin, P.M., Houze, R.A., 1972. Analysis of the Structure of Precipitation
Patterns in New England. J. Appl. Meteorol. 11, 926–935.

Barua, S., Islam, M.M., Murase, K., 2011. A Novel Synthetic Minority
Oversampling Technique for Imbalanced Data Set Learning BT - Neural
Information Processing, in: Lu, B.-L., Zhang, L., Kwok, J. (Eds.), . Springer
Berlin Heidelberg, Berlin, Heidelberg, pp. 735–744.

Battan, L.J., 1973. Radar observation of the atmosphere. L. J. Battan (The
University of Chicago Press) 1973. PP X, 324; 125 figures, 21 tables. £7·15.
Q. J. R. Meteorol. Soc. 99, 793–793.

Biggerstaff, M.I., Listemaa, S.A., 2000. An Improved Scheme for
Convective/Stratiform Echo Classification Using Radar Reflectivity. J. Appl.
Meteorol. 39, 2129–2150.

Biswas, S., Chandrasekar, V., 2018. Cross-Validation of Observations between
the GPM Dual-Frequency Precipitation Radar and Ground Based Dual-Polarization
Radars. Remote Sens. 10, 1773.

Chawla, N. V, Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P., 2002. SMOTE:
Synthetic Minority Over-sampling Technique. J. Artif. Intell. Res. 16,
321–357.

Churchill, D.D., Houze, R.A., 1984. Development and Structure of Winter
Monsoon Cloud Clusters On 10 December 1978. J. Atmos. Sci. 41, 933–960.

Das, S.K., Konwar, M., Chakravarty, K., Deshpande, S.M., 2017. Raindrop size
distribution of different cloud types over the Western Ghats using
simultaneous measurements from Micro-Rain Radar and disdrometer. Atmos. Res.
186, 72–82.

DeMott, C. A., R. Cifelli, and S. A. Rutledge, 1995: An improved method for
partitioning radar data into convective and stratiform components. Preprints,
27th Conf. on Radar Meteorology, Vail, CO, Amer. Meteor. Soc., 233–236.

Deshpande, S.M., Dhangar, N., Das, S.K., Kalapureddy, M.C.R., Chakravarty, K.,
Sonbawne, S., Konwar, M., 2015. Mesoscale kinematics derived from X‐band
Doppler radar observations of convective versus stratiform precipitation and
comparison with GPS radiosonde profiles. J. Geophys. Res. Atmos. 120,
11,511-536,551.

Devisetty, H.K., Jha, A.K., Das, S.K., Deshpande, S.M., Krishna, U.V.M.,
Kalekar, P.M., Pandithurai, G., 2019. A case study on bright band transition
from very light to heavy rain using simultaneous observations of collocated X-
and Ka-band radars. J. Earth Syst. Sci. 128, 136.

Elreedy, D., Atiya, A.F., 2019. A Comprehensive Analysis of Synthetic Minority
Oversampling Technique (SMOTE) for handling class imbalance. Inf. Sci. (Ny).
505, 32–64.

Fabry, F., 2015. Radar Meteorology. Cambridge University Press, Cambridge.

Ghada, W., Casellas, E., Herbinger, J., Garcia-Benadí, A., Bothmann, L.,
Estrella, N., Bech, J., Menzel, A., 2022. Stratiform and Convective Rain
Classification Using Machine Learning Models and Micro Rain Radar. Remote
Sens. 14, 4563.

Gourley, J.J., Vieux, B.E., 2005. A Method for Evaluating the Accuracy of
Quantitative Precipitation Estimates from a Hydrologic Modeling Perspective.
J. Hydrometeorol. 6, 115–133.

Houze, Jr., R.A., 1982. Cloud Clusters and Large-Scale Vertical Motions in the
Tropics. J. Meteorol. Soc. Japan. Ser. II 60, 396–410.

Houze, R.A., 1997. Stratiform Precipitation in Regions of Convection: A
Meteorological Paradox? Bull. Am. Meteorol. Soc. 78, 2179–2196.

Houze, R.A., 1973. A Climatological Study of Vertical Transports by Cumulus-
Scale Convection. J. Atmos. Sci. 30, 1112–1123.

Houze, R.A., 2004. Mesoscale convective systems. Rev. Geophys. 42.

Houze, R.A., 2014. Cloud Dynamics - Second Edition, International Geophysics.

Inoue, T., 1987. A cloud type classification with NOAA 7 split‐window
measurements. J. Geophys. Res. Atmos. 92, 3991–4000.

Jha, A.K., Kalapureddy, M.C.R., Devisetty, H.K., Deshpande, S.M., Pandithurai,
G., 2019. A case study on large-scale dynamical influence on bright band using
cloud radar during the Indian summer monsoon. Meteorol. Atmos. Phys. 131,
505–515.

Kalapureddy, M.C.R., Patra, S., Dhavale, V., Nair, M.R., 2023. CloudSat
inferred contrasting monsoon intra-seasonal variation in the cloud vertical
structure over Indian regions. Clim. Dyn. 61, 1567–1589.

Konwar, M., Das, S.K., Deshpande, S.M., Chakravarty, K., Goswami, B.N., 2014.
Microphysics of clouds and rain over the Western Ghat. J. Geophys. Res. Atmos.
119, 6140–6159.

Konwar, M., Maheskumar, R.S., Kulkarni, J.R., Freud, E., Goswami, B.N.,
Rosenfeld, D., 2012. Aerosol control on depth of warm rain in convective
clouds. J. Geophys. Res. Atmos. 117.

Kühnlein, M., Appelhans, T., Thies, B., Nauss, T., 2014. Improving the
accuracy of rainfall rates from optical satellite sensors with machine
learning — A random forests-based approach applied to MSG SEVIRI. Remote Sens.
Environ. 141, 129–143.

Kumar Das, S., Deshpande, S.M., Shankar Das, S., Konwar, M., Chakravarty, K.,
Kalapureddy, M.C.R., 2015. Temporal and structural evolution of a tropical
monsoon cloud system: A case study using X-band radar observations. J. Atmos.
Solar-Terrestrial Phys. 133, 157–168.

Lavanya, S., Kirankumar, N.V.P., 2021. Classification of tropical coastal
precipitating cloud systems using disdrometer observations over Thumba, India.
Atmos. Res. 253, 105477.

Lazri, M., Ameur, S., 2018. Combination of support vector machine, artificial
neural network and random forest for improving the classification of
convective and stratiform rain using spectral features of SEVIRI data. Atmos.
Res. 203, 118–129.

Loh, J. Le, Lee, D.-I., Kang, M.-Y., You, C.-H., 2020. Classification of
Rainfall Types Using Parsivel Disdrometer and S-Band Polarimetric Radar in
Central Korea. Remote Sens. 12, 642.

Maheskumar, R.S., Narkhedkar, S.G., Morwal, S.B., Padmakumari, B., Kothawale,
D.R., Joshi, R.R., Deshpande, C.G., Bhalwankar, R. V, Kulkarni, J.R., 2014.
Mechanism of high rainfall over the Indian west coast region during the
monsoon season. Clim. Dyn. 43, 1513–1529.

Marshall, J.S., Langille, R.C., Palmer, W.M.K., 1947. MEASUREMENT OF RAINFALL
BY RADAR. J. Meteorol. 4, 186–192.

McKee, J.L., Binns, A.D., 2016. A review of gauge–radar merging methods for
quantitative precipitation estimation in hydrology. Can. Water Resour. J. /
Rev. Can. des ressources hydriques 41, 186–203.

Nandargi, S., Mulye, S.S., 2012. Relationships between Rainy Days, Mean Daily
Intensity, and Seasonal Rainfall over the Koyna Catchment during 1961–2005.
Sci. World J. 2012, 1–10.

Nasir, N., Kansal, A., Alshaltone, O., Barneih, F., Sameer, M., Shanableh, A.,
Al-Shamma’a, A., 2022. Water quality classification using machine learning
algorithms. J. Water Process Eng. 48, 102920.

Özöğür-Akyüz, S., Ünay, D., Smola, A., 2011. Guest editorial: model selection
and optimization in machine learning. Mach. Learn. 85, 1–2.

Penide, G., Protat, A., Kumar, V. V, May, P.T., 2013. Comparison of Two
Convective/Stratiform Precipitation Classification Techniques: Radar
Reflectivity Texture versus Drop Size Distribution–Based Approach. J. Atmos.
Ocean. Technol. 30, 2788–2797.

Probst, P., Boulesteix, A., Bischl, B., 2018. Tunability: Importance of
Hyperparameters of Machine Learning Algorithms. J. Mach. Learn. Res. 20,
53:1-53:32.

Ran, Y., Wang, H., Tian, L., Wu, J., Li, X., 2021. Precipitation cloud
identification based on faster-RCNN for Doppler weather radar. EURASIP J.
Wirel. Commun. Netw. 2021, 19.

Rao, T.N., Rao, D.N., Mohan, K., Raghavan, S., 2001. Classification of
tropical precipitating systems and associated Z ‐ R relationships. J. Geophys.
Res. Atmos. 106, 17699–17711.

Sassen, K., Wang, Z., 2008. Classifying clouds around the globe with the
CloudSat radar: 1‐year of results. Geophys. Res. Lett. 35.

Schumacher, C., Houze, R.A., 2003. The TRMM Precipitation Radar’s View of
Shallow, Isolated Rain. J. Appl. Meteorol. 42, 1519–1524.

Silva, A.A., Tavares, M.W., Carrasquilla, A., Misságia, R., Ceia, M., 2020.
Petrofacies classification using machine learning algorithms. GEOPHYSICS 85,
WA101–WA113.

Smull, B.F., Houze, R.A., 1987. Dual-Doppler Radar Analysis of a Midlatitude
Squall Line with a Trailing Region of Stratiform Rain. J. Atmos. Sci. 44,
2128–2149.

So, D., Shin, D.-B., 2018. Classification of precipitating clouds using
satellite infrared observations and its implications for rainfall estimation.
Q. J. R. Meteorol. Soc. 144, 133–144.

Steiner, M., Houze, R.A., 1997. Sensitivity of the Estimated Monthly
Convective Rain Fraction to the Choice of Z – R Relation. J. Appl. Meteorol.
36, 452–462.

Steiner, M., Houze, R.A., Yuter, S.E., 1995. Climatological Characterization
of Three-Dimensional Storm Structure from Operational Radar and Rain Gauge
Data. J. Appl. Meteorol. 34, 1978–2007.

Subrahmanyam, K. V, Kumar, K.K., 2013. CloudSat observations of cloud-type
distribution over the Indian summer monsoon region. Ann. Geophys. 31,
1155–1162.

Testud, J., Oury, S., Black, R.A., Amayenc, P., Dou, X., 2001. The Concept of
“Normalized” Distribution to Describe Raindrop Spectra: A Tool for Cloud
Physics and Cloud Remote Sensing. J. Appl. Meteorol. 40, 1118–1140.

Thompson, E.J., Rutledge, S.A., Dolan, B., Thurai, M., 2015. Drop Size
Distributions and Radar Observations of Convective and Stratiform Rain over
the Equatorial Indian and West Pacific Oceans. J. Atmos. Sci. 72, 4091–4125.

Tokay, A., Short, D.A., 1996. Evidence from Tropical Raindrop Spectra of the
Origin of Rain from Stratiform versus Convective Clouds. J. Appl. Meteorol.
35, 355–371.

Visa, S., Ramsay, B., Ralescu, A., Knaap, E., 2011. Confusion Matrix-based
Feature Selection., CEUR Workshop Proceedings.

Waldvogel, A., 1974. The N 0 Jump of Raindrop Spectra. J. Atmos. Sci. 31,
1067–1078.

Wang, H., Shao, N., Ran, Y., 2019. Identification of Precipitation-Clouds
Based on the Dual-Polarization Doppler Weather Radar Echoes Using
Deep–Learning Method. IEEE Access 7, 12822–12831.

Wang, Y., Tang, L., Chang, P.-L., Tang, Y.-S., 2021. Separation of convective
and stratiform precipitation using polarimetric radar data with a support
vector machine method. Atmos. Meas. Tech. 14, 185–197.

Weerts, H.J.P., Mueller, A., Vanschoren, J., 2020. Importance of Tuning
Hyperparameters of Machine Learning Algorithms. ArXiv abs/2007.0.

Williams, C.R., Ecklund, W.L., Gage, K.S., 1995. Classification of
Precipitating Clouds in the Tropics Using 915-MHz Wind Profilers. J. Atmos.
Ocean. Technol. 12, 996–1012.

Yang, Z., Liu, P., Yang, Y., 2019. Convective/Stratiform Precipitation
Classification Using Ground-Based Doppler Radar Data Based on the K-Nearest
Neighbor Algorithm. Remote Sens. 11, 2277.

Yan-Jiao, X., Li-Ping, L.I.U., 2007. Identification of Stratiform and
Convective Cloud Using 3D Radar Reflectivity Data. Chinese J. Atmos. Sci. 31,
645–654.

Yuter, S.E., Houze, R.A., 1995. Three-Dimensional Kinematic and Microphysical
Evolution of Florida Cumulonimbus. Part II: Frequency Distributions of
Vertical Velocity, Reflectivity, and Differential Reflectivity. Mon. Weather
Rev. 123, 1941–1963.

Zhong, L., Yang, R., Wen, Y., Chen, L., Gou, Y., Li, R., Zhou, Q., Hong, Y.,
2017. Cross-evaluation of reflectivity from the space-borne precipitation
radar and multi-type ground-based weather radar network in China. Atmos. Res.
196, 200–210.

Zhu, T., Lin, Y., Liu, Y., 2017. Synthetic minority oversampling technique for
multiclass imbalance problems. Pattern Recognit. 72, 327–340.

Zuo, Y., Hu, Z., Yuan, S., Zheng, J., Yin, X., Li, B., 2022. Identification of
Convective and Stratiform Clouds Based on the Improved DBSCAN Clustering
Algorithm. Adv. Atmos. Sci. 39, 2203–2212.

Table 1: Technical specifications of X-band Doppler radar Parameter | Value
---|---
Frequency(GHz) | 9.535
Wavelength(cm) | 3.14
Transmitter | Magnetron
Peak power(kW) | 200
Pulse widths | 0.8-2 µs
Beam width |
Antenna gain(dB) | 44.3
Antenna diameter(m) | 2.4
Cross-pol Isolation(dB) | -30
Minimum detectable signal | -25 dBZ at 20 km
Table 2: Overall results of tuned and not tuned ML models. The bold and underlined portion highlights the best results | Model | BAC | Precision | Recall | F1-Score
---|---|---|---|---|---
Not Tuned Models | Decision tree | 0.67 | 0.68 | 0.68 | 0.68
Naïve Bayes | 0.54 | 0.69 | 0.43 | 0.36
Tuned Models | xgboost | 0.76 | 0.78 | 0.79 | 0.78
catboost | 0.76 | 0.78 | 0.78 | 0.77
KNN | 0.69 | 0.71 | 0.71 | 0.71
Light GBM | 0.80 | 0.82 | 0.82 | 0.82
Random Forest | 0.77 | 0.79 | 0.79 | 0.79
Table 3:  Classification report of CloudSense  | Precision | Recall | F1-Score | Number of samples
---|---|---|---|---
Stratiform | 0.80 | 0.88 | 0.84 | 140
Mixed Stratiform-Convective | 0.75 | 0.58 | 0.66 | 77
Convective | 0.75 | 0.81 | 0.78 | 99
Shallow | 1 | 0.95 | 0.98 | 84
Table 4:  Hyper-parameters and their tuned values used in the LightGBM model. The values are finalised after running 1000 trials and selecting the trial with the highest F1-Score Hyper Parameters | Lower limit | Upper limit | Tuned Values
---|---|---|---
boosting_type | - | - | gbdt
class_weight | 1 | 5 | {0: 5, 1: 5, 2: 4}
objective | - | - | multiclass
metric | - | - | multi_logloss
random_state | - | - | 42
lambda_l1 | 1e-8 | 10.0 | 3.54e-05
lambda_l2 | 1e-8 | 10.0 | 0.0006
feature_fraction | 0.4 | 1 | 0.72
bagging_fraction | 0.4 | 1 | 0.98
bagging_freq | 1 | 7 | 6
min_child_samples | 5 | 100 | 16
Table 5: Classification report of the radar algorithm for 200 cloud samples taken over the MDV site | Precision | Recall | F1-Score | Number of samples
---|---|---|---|---
Stratiform | 0.65 | 0.84 | 0.73 | 50
Mixed Stratiform-Convective | 0.44 | 0.34 | 0.38 | 50
Convective | 0.64 | 0.64 | 0.64 | 50
Shallow | 1 | 0.92 | 0.96 | 50
Table 6: Classification report of CloudSense for 200 cloud samples taken over the MDV site | Precision | Recall | F1-Score | Number of samples
---|---|---|---|---
Stratiform | 0.68 | 0.80 | 0.73 | 50
Mixed Stratiform-Convective | 0.81 | 0.50 | 0.62 | 50
Convective | 0.67 | 0.86 | 0.75 | 50
Shallow | 1 | 0.92 | 0.96 | 50
FIG 1:  Topography map of the Western Ghats. Black dots show the locations of
radar site (MDV) and HACPL. Dotted circle represents the X-band radar’s
maximum range (125 km). RHI scan is taken along the dash-dotted line FIG 2:
a) Typical RHI scan observed on X July, 2018. Vertical black line is drawn at
26 km to show VPRs considered over HACPL. b) Vertical profile of Doppler
velocity from a PPI scan at 85º elevation FIG 3:  Logical flow diagram for
classifying clouds FIG 4:  Architecture of CloudSense FIG 5:  CFADs of Z for
a) Stratiform b) Mixed Stratiform-Convective c) Convective and d)Shallow
clouds FIG 6: A 2X2 Confusion Matrix

The vertical profiles of Z(FIG 7), DV(FIG 8) and W(FIG 9) are analysed every
1̃2 minutes using PPI scan data taken at 85º elevation(pseudo zenith looking).
All times given in UTC

FIG 10:  Confusion Matrix. The main diagonal elements represent the TPs of
each type of cloud. Excluding the TPs, the sum of row elements for each cloud
indicates FNs and the sum of column elements indicates FPs. The sum of every
remaining element indicates TNs for a single type of cloud FIG 11:  a) RHI
scan taken by the radar at 3:52 UTC on July 15 2018 b) Corresponding
Actual/labelled cloud profiles and c) Predicted cloud profiles by the model.
The dotted line at 4 km indicates the melting layer. FIG 12:  CFADs of a) Z b)
DV c) W constructed over the radar site (MDV) site


# Investigating Deep Learning Benchmarks for Electrocardiography Signal
Processing

Hao Wen$1$, Jingsu Kang$2$
$1$LMIB, School of Mathematical Sciences, Beihang University, Beijing, 102206,
China
$2$Tianjin Medical University, Tianjin, 300203, China
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>

###### Abstract

In recent years, deep learning has witnessed its blossom in the field of
Electrocardiography (ECG) processing, outperforming traditional signal
processing methods in various tasks, for example, classification, QRS
detection, wave delineation. Although many neural architectures have been
proposed in the literature, there is a lack of systematic studies and open-
source libraries for ECG deep learning.

In this paper, we propose a deep learning framework, named torch_ecg, which
gathers a large number of neural networks, both from literature and novel, for
various ECG processing tasks. It establishes a convenient and modular way for
automatic building and flexible scaling of the networks, as well as a neat and
uniform way of organizing the preprocessing procedures and augmentation
techniques for preparing the input data for the models. Besides, torch_ecg
provides benchmark studies using the latest databases, illustrating the
principles and pipelines for solving ECG processing tasks and reproducing
results from the literature. torch_ecg offers the ECG research community a
powerful tool meeting the growing demand for the application of deep learning
techniques.

###### Keywords:

ECG processing, deep learning framework, benchmarks

#### Data and Code Availability

This paper involves the the following 6 databases: CinC2020 [1, 2], CinC2021
[3], CPSC2019 [4], CPSC2020 [5], CPSC2021 [6], and LUDB [7, 8], which will be
introduced in more details later in this paper. These databases are hosted
either by PhysioNet [9] or by corresponding challenge organizing committees
and are open, freely accessible to all researchers. PhysioNet [9] is a large
archive hosting a continuously growing number of datasets and software for
physiological signal studies, among which ECG perhaps is the most important.
Despite existing classical databases in PhysioNet, several challenges are held
yearly as special sessions of conferences along with which large novel
databases are released, providing the research community with challenging
frontier problems.

The code is available at https://github.com/DeepPSP/torch_ecg.

##  1 Introduction

Research on computer-aided medical auxiliary diagnosis algorithms and systems
owns a long history, having a large number of practitioners from both academia
and industry. It has abundant research areas and many successful real-world
applications. Driven by the remarkable success of the application of deep
learning in computer vision (CV) [10, 11, 12], research on medical imagery is
the most prominent and attracting [13, 14]. Physiological signal processing, a
traditional medical research field, among which electrocardiogram (ECG) is the
most extensively studied, attracts many researchers of various backgrounds as
well.

Although there has been a steady production of inspiring research work [15,
16] in the field of ECG signal processing, there has always been a lack of an
ECG deep learning research framework or library, whose existence would largely
facilitate the researchers and improve the overall research levels. With the
help of torchvision [17], medical imagery researchers (especially using
ordinary images rather than CT images, MRI images, etc.) can build pipelines
quickly and verify the validity of the models or even do neural network
architecture searching. A similar situation holds for researchers using the
HuggingFace Transformer library [18] from the area of medical text processing.

Based on this motivation, we build a library named torch_ecg, mainly on top of
PyTorch [17], containing readers of popular ECG databases, common
preprocessings and augmentations for ECG, and many neural network
architectures. The databases include the many of the PhysioNet databases [9],
databases from other challenge competitions [19, 4, 5, 6], etc. Preprocessing
like band-pass filtering, detrending, are organized and monitored in a uniform
way; and similarly for the augmentations including Z-score normalization,
mixup [20], etc. torch_ecg also collects neural network models from previous
literature [15, 16, 21, 22, 23], and provides an convenient way to allow the
users to alter the network architecture to search for better ones.

The rest of the paper is organized as follows. In Section 2, preceding work on
ECG deep learning is briefly reviewed. An overview of the torch_ecg library is
presented in Section 3. Afterwards, from Section 4 to Section 6, we dive into
details of torch_ecg from different prospective. We conclude in Section 7 and
propose our future work therein.

##  2 Related Work

To the authors’ knowledge, the earliest effort of computer-aided automated ECG
diagnosis dates back to the 1960s [24]. Since then, this research community
has been growing steadily. The most famous and fundamental software for ECG
processing is the Waveform Database (WFDB) software package hosted at
PhysioNet [25, 9] and other language versions (for example the most widely
used python version [26]) of it. This software package was initially developed
by George Moody and his colleagues at the MIT Laboratory for Computational
Physiology and is still actively updated. WFDB and PhysioNet are epoch-making
products that have greatly promoted the research progress of ECG processing.
Another outstanding library is the BioSPPy toolbox [27] which bundles together
various signal processing methods, not only for ECG. We use these two
libraries for data I/O (Input/Output) and basic signal processing.

Deep learning has long been ordinary among various machine learning methods,
until the last decade when it started occupying an absolute advantage. The
situation is similar for the research field of ECG processing. [15] used a
variant of ResNet [11] for the classification of ambulatory (single-lead)
ECGs, which was claimed to have achieved cardiologist-level. [16] used another
variant of ResNet for classifying the standard 12-lead ECGs, while [22] used
convolutional recurrent neural network (CRNN) architecture with a VGG-like
[28] CNN (convolutional neural network) backbone for similar tasks. Other than
classification, neural network models also dominate other ECG downstream
tasks. For example, [21] used a branched network structure to tackle the
problem of QRS detection, which greatly outperformed existing wavelet-based
algorithms and the classical Pan-Tompkins algorithm [29]. [23] used a U-Net
model for wave delineation on 12-lead ECGs and validated the model using the
Lobachevsky University Electrocardiography Database (LUDB) [7, 8].

Although many powerful deep learning models have been proposed, which greatly
promoted the research progress on the problem of automated ECG diagnosis
systems, there has not yet existed a software package that broadly collects
the state-of-the-art (SOTA) neural network models, just as [30] does for CV
models, for various tasks of ECG processing, as far as the authors know. The
effect of data preprocessing and augmentations on model performance are also
not thoroughly studied, lacking statistics and comparative studies. Moreover,
unlike other research areas where canonical network structures already exist,
for example, convolutions of kernel size 3 for CV models, ECG deep learning
models do not have widely-enough accepted network structures, hence
flexibility for searching for optimized network hyperparameters and
architectures is in desperate need. For the above reasons, torch_ecg mainly
includes

  * •

uniform implementations of data augmenters, which are monitored by a manager
class during the model training process. Similarly for preprocessors.

  * •

flexible and scalable neural network generation from a comprehensive
collection of named neural network architectures from existing literature and
networks newly developed by the authors, for various ECG downstream tasks.

  * •

utilities for building training pipelines, and utilities for logging metrics
for facilitating network architectures and hyperparameters searching.

  * •

benchmark case studies and pretrained models for various tasks using large-
scale databases.

The above will be interpreted in more detail in the following sections.

##  3 System Design

Unlike the situation of commercial ECG production systems where data are
stored in some database like Apache Impala, data of standard ECG datasets are
usually saved in the WFDB file format where signals, annotations and metadata
are stored in separate files. The WFDB package [26] already provides powerful
methods handling a single record in its io submodule. We wrap a standard ECG
dataset using a Python class, where paths, the list of ECG records, overall
statistics, and other status quantities are pre-loaded and maintained. This is
a common practice adopted by frameworks in other research field, for example
by torchtext, torchvision, etc. Methods for manipulating a single record along
with dataset-specific format conversion functions and helper functions, for
example, conversion of annotations of atrial fibrillation into lists of
intervals or masks, are wrapped therein as well. We call such a class a
data_reader, which is task-independent (or model-independent). Combined with
PyTorch’s Dataset and DataLoader, one can quickly set up a pipeline feeding
data for training a neural network. If not used for training neural networks,
a data_reader can also facilitate developing for example an SVM model.

ECG processing usually includes procedures of signal preprocessing, including
resampling, detrending, denoising, normalizing, etc. Each of them is
implemented using a Python class, called a “preprocessor”, sharing a common
calling method and a common abstract base class (ABC). All preprocessors are
arranged and manipulated by a manager. The manager also accepts user-defined
preprocessors inherited from this common abstract base class and can
manipulate them in a uniform way as the built-in preprocessors. A preprocess
manager can be instantiated using an ordered configuration dictionary where
the preprocessors to use and their parameters are specified. It is recommended
to be used inside a PyTorch Dataset. Preprocessors operating on mini-batch
tensors are also implemented but less recommended for the following reasons.
Preprocessors directly deal with raw ECG signals which might be length-
varying. Another reason is that many signal processing algorithms have no
PyTorch (GPU) implementation, which results in frequent conversions between
numpy arrays and PyTorch tensors. More details of the implemented
preprocessors will be given in Section 4.

Training neural networks often involves data augmentation techniques. Data
augmentation plays the role of data extrapolation, hence has the potential to
improve the model performance outside the distribution of the training data.
Common data augmentations include adding Gaussian and/or sinusoidal noise,
random flip, mixup [20], cutmix [31], scaling [22] (named as stretch-or-
compress), etc. Some are inherited from computer vision techniques, while
others are ECG-specific. The augmenters are implemented as child classes of
the PyTorch Module without trainable parameters. The forward functions of them
accept the same set of arguments so that a manager which is also a child class
of the PyTorch Module can manipulate them uniformly. The augmenters are
multiple stochastic, i.e. stochastic in the batch dimension so that the
samples to be augmented within one mini-batch are randomly chosen; and also
stochastic inside each sample, for example, the amplitudes of Gaussian noise
added. In Section 4, the augmenters will be discussed in more details.

A serious problem, compared to computer vision or natural language processing,
for the ECG processing research community is that currently there are still no
commonly accepted neural network structure design principles. For example, for
computer vision neural networks, the convolutions typically have kernel size
3, while ECG neural networks in existing literature usually have larger and
varying convolution kernel sizes. ECG signals have a sampling frequency while
images do not. The kernel sizes should be related to the sampling frequency
(each model should have a fixed sampling frequency input) of the model input.

The most extensively studied architectures are variants of ResNet [15, 16].
They both have kernel size 16111the former paper [15] does not state the
kernel size used explicitly in the paper, but one can find this in the config
file in github.com/awni/ecg/blob/master/examples/irhythm/, however, each using
ECGs of sampling frequencies 200Hz and 400Hz as input respectively. The kernel
sizes of the convolutions in the QRS detection model proposed in [21] have a
decay pattern as the convolutions go deeper in the network. All these indicate
that ECG neural networks still need solid work for architectural optimization.

For this reason, the implementation of the models (ref. Table I) in torch_ecg
are modular, hence extremely flexible to adjust the network architectures for
the users. We extract the common pattern of a model series and make the rest
hyperparameters and components configurable. For example, the vanilla ResNet
[11], and its variants [32, 33, 15, 16, 34] (the convolution part, not
including the final classifying fully connected layer) all share a common
pattern: one stem, plus 4 stages each of which consists of stacked building
blocks of basic type or bottleneck type. DenseNet [35] has the pattern of
stacked dense blocks, within which convolutional layers are densely connected.
Keeping this common pattern, other finer details of the networks, including
extra attention modules, choices of activation, method and positions for down-
sampling in the network, etc. can be altered very easily by just changing one
or more items in the model’s configuration. A bunch of such small modules that
can be integrated into larger networks are implemented in torch_ecg as well,
including squeeze-and-excitation (SE) module [36], global context (GC) module
[37], conditional random field (CRF) module [38], etc. torch_ecg implements
the neural networks in its models module and has a set of built-in
configuration files, a large part of which come from existing literature, in
the model_configs module.

3.5 Network Family | Selected Publications | Highlights
---|---|---
Branched CNN | [21] | CPSC2019 rank 1∗
CinC2021 rank 3†
VGG | [22] | CinC2020 rank 1‡
ResNet | [15, 16] | CPSC2018 rank 1
CinC2020 rank 2-4
CinC2021 rank 1-2
CPSC2021 rank 1
DenseNet | $\backslash$ | CPSC2020 rank 1
Inception | $\backslash$ | CinC2021 rank 4
Xception | $\backslash$ | $\backslash$
MobileNet | $\backslash$ | $\backslash$
U-Net | [23] | CPSC2019 rank 2
CRNN | [22, 21] | CPSC2019 rank 1∗
RR-LSTM | [39, 40] | CPSC2021 rank 5∗∗
3.5 |  |
TABLE I: Neural networks implemented in torch_ecg. For the challenges
CinC2020-2021 [1, 2, 3], CPSC2018-2021 [19, 4, 5, 6], more information can be
found in Section 6. Selected publications are not comprehensive lists, only
including those directly dealing with ECG processing rather than the papers
that initially proposed the network architectures (usually for CV problems).
Details of some networks are not disclosed, for example the rank 1 DenseNet
solution of CPSC2020, which is developed by a company.
† not fully branched
‡ used in conjunction with transformer encoders [41]
∗ Branched CNN is the CNN backbone, CRNN (convolutional recurrent neural
network) is the whole architecture
∗∗ used in conjunction with CRNN

As stated previously, convolutions in neural networks for ECG processing
usually have large kernels, hence the depths of the networks are usually
largely reduced, to prevent them from growing too large in the number of
trainable parameters. To the authors’ experiences, vanilla networks directly
inherited from computer vision, with only 2D operations changed to 1D
operation, perform far worse than their wider and shallower variants. For this
reason, torch_ecg constructs in its model_configs module various wider and
shallower variants of the named neural networks (e.g. TResNet [34], DenseNet
[35], etc.)

To facilitate model training, an abstract base class of BaseTrainer where
common operations for training a neural network model in most cases, including
the setup of the optimizer, controlling of learning rate scheduler, the
workflow of one training epoch, etc., are extracted and implemented.
Statistics are maintained by this BaseTrainer using a logging manager, which
writes to stdout, text files, CSV files as well as tensorboard files. For
training a neural network model for a specific ECG task, one only has to
inherit this class and implement a few functions including the process of one
step (one mini-batch), and the evaluation function for computing and gathering
statistics and metrics. The abstraction of BaseTrainer is moderate, retaining
a certain degree of flexibility for the users.

A set of Outputs classes are implemented to facilitate the maintenance of
outputs of models for ECG downstream tasks (for more details about ECG
downstream tasks, ref. Section 5). Each class has a set of required fields to
meet the demand of the downstream task this class corresponds to. Outputs
classes have additional functions for the computation of a collection of pre-
defined metrics when labels are collected along with model outputs during a
training process. These metrics, implemented also as Python classes, are re-
implementations from existing literature [23, 3, 4] or from
Wikipedia222https://en.wikipedia.org/wiki/Precision_and_recall.

The whole system structure and the workflow of torch_ecg are depicted and
summarized in Figure 1.

Data Reader Preprocessor Manager Augmenter Manager Data FilesRaw
DataUniformized DataTensorsTrainerModelConfig File (Dictionary) Logging
Manager resume from CheckpointTrained ModelRecorded Metrics Figure 1: Main
Modules and Work Flow of torch_ecg.

##  4 Data Preprocessing and Augmentation

As raw ECG signals may contain noise from various sources, especially those
ambulatory ECGs collected from wearable devices, preprocessing sometimes can
be crucial for ECG signal processing. One can not guarantee that the training
data and the data for inference share similar distributions which are
distorted by noises. In extreme cases, even data augmentation can not bridge
this gap. Common preprocessing procedures include

  * •

Resampling to a fixed sampling rate.

  * •

Band-pass filtering, performed using finite impulse response (FIR) filters,
Butterworth filters, etc., which removes noises of frequencies outside the
given pass band.

  * •

Detrending, via median filter, which removes baseline drifts.

  * •

Normalization. There are typically 3 ways to do normalization for ECGs as
follows

$\begin{array}[]{cl}\text{Naïve normalization:}&\frac{\mathbf{x}-m}{s}\\\
\text{Z-score
normalization:}&\left(\frac{\mathbf{x}-\operatorname{mean}(\mathbf{x})}{\operatorname{std}(\mathbf{x})+eps}\right)\cdot
s+m\\\ \text{Min-Max
normalization:}&\frac{\mathbf{x}-\min(\mathbf{x})}{\max(\mathbf{x})-\min(\mathbf{x})+eps}\end{array}$

where $\mathbf{x}$ is an ECG signal, $m,s$ are given values, a small value
$eps$ is added to avoid division by zero error.

The naïve normalization method is a standard transformation for images before
being fed into a computer vision model, because images have a fixed range of
values (0-255 or 0-1), and $m,s$ are usually the mean and standard deviation
of the training data respectively. However, it is usually not the case for
ECGs, hence naïve normalization is not recommended for preprocessing of ECGs.
Z-score normalization is the most commonly used method for ECGs where $m,s$
are typically set $0,1$ respectively. There are researchers [42] adopting min-
max normalization as well.

As is stated in Section 3, preprocessors are implemented as child classes of a
common abstract base class. The hyperparameters, for example, the cutoff
frequencies for the band-pass filter, are accepted as keyword arguments for
the instance initialization function __init__. The calling method
__call__333No type hinting and docstring are used here in the code block for
simplicity. The same applies to the rest code blocks in this paper. Type
hinting is a modern feature of Python that largely improve code readability,
hence is strongly recommended to be used. all accept the signal and sampling
frequency as arguments:

⬇

1def __call__(self, sig, fs):

2 if len(self.preprocessors) == 0:

3 return sig, fs

4 ordering = list(range(len(self.preprocessors)))

5 if self.random:

6 ordering = sample(ordering, len(ordering))

7 for idx in ordering:

8 sig, fs = self.preprocessors[idx](sig, fs)

9 return sig, fs

In this way, all preprocessors are managed by a PreprocManager whose calling
method shares the same set of arguments of the preprocessors.

Data augmentation is an important technique for modern deep learning, and ECG
deep learning is no exception. Data augmentation dose extrapolation on the
training data, broadening and enriching its distribution. Data augmentation
helps alleviate overfitting and over-confidence for neural networks. Most
techniques are initially invented for computer vision tasks and are inherited
for training ECG neural network models. Augmenters are listed as follows

  * •

Addition of Gaussian and/or sinusoidal noise. This technique is sometimes used
by researchers and in challenges (e.g. the authors’ rank 3 solutions to the
problem of supraventricular premature beats (SPB) detection in CPSC2020 [5]),
but its efficacy still needs proving and should not be used when preprocessing
contains band-pass filtering and/or detrending.

  * •

Random Flip, using which the values of the ECGs are multiplied by -1 with a
certain probability. This technique also has to be treated carefully whose
usage might should be restricted to ambulatory ECGs, since for standard-leads
ECGs, altering in relative or absolute values in different leads might
completely change its interpretation, for example, the electrical axis.

  * •

Mixup [20]. Mixup uses convex linear combination of two samples for
extrapolating the data distribution as follows

$\begin{array}[]{l}\mathbf{x}=\alpha\mathbf{x}_{1}+(1-\alpha)\mathbf{x}_{2},\\\
\mathbf{y}=\alpha\mathbf{y}_{1}+(1-\alpha)\mathbf{y}_{2},\end{array}$ (1)

where $(\mathbf{x}_{1},\mathbf{y}_{1})$, $(\mathbf{x}_{2},\mathbf{y}_{2})$ are
data and labels of 2 samples, $(\mathbf{x},\mathbf{y})$ is the generated
sample, $\alpha\in(0,1)$ is randomly generated from some Beta distribution.

  * •

CutMix [31]. CutMix inserts a part of one sample to another for data
distribution extrapolation via

$\begin{array}[]{l}\mathbf{x}=\mathbf{M}\odot\mathbf{x}_{1}+(\mathbf{1}-\mathbf{M})\odot\mathbf{x}_{2},\\\
\mathbf{y}=\alpha\mathbf{y}_{1}+(1-\alpha)\mathbf{y}_{2},\end{array}$ (2)

the annotations are similar to Equation (1), $\mathbf{M}$ is a binary mask, in
which the proportion of ones is equal to $\alpha$, for cutting and joining two
ECGs, $\odot$ is the Hadamard product, i.e. element-wise product. This
technique has proven quite useful in the CPSC2021 rank 1 solution.

  * •

Masking [22]. This technique randomly masks a small proportion of the signal
with zeros, as similar to the Cutout technique [43] in computer vision. This
can also be done randomly at critical points, e.g. randomly masking R peaks
helps reduce the probability of CNN models to misclassify sinus arrhythmia to
atrial fibrillation [40].

  * •

Scaling (stretch-or-compress) [22]. This augmentation method scales the ECG in
the time axis, and has proven its validity in [22].

  * •

Label smoothing [44]. Strictly speaking, label smoothing is not a data
augmentation method, but a technique to prevent the model from overconfidence
thus improving its capability of generalization. This technique generates soft
labels from hard labels in the following way

$\mathbf{y}^{\prime}=(1-\varepsilon)\mathbf{y}+\dfrac{1}{K}\varepsilon\mathbf{1},$
(3)

where $K$ typically equals the number of classes, $\varepsilon$ is a smoothing
factor.

All the augmenters mentioned above have successful applications in training
ECG neural network models. As is mentioned in Section 3, they are all
implemented as child classes of the Pytorch Module, and accept the same set of
parameters for the forward function, hence is manipulated by a
AugmenterManager uniformly as follows:

⬇

1def forward(self, sig, label, *extra_tensors, **kwargs):

2 if len(self.augmenters) == 0:

3 return (sig, label, *extra_tensors)

4 ordering = list(range(len(self.augmenters)))

5 if self.random:

6 ordering = sample(ordering, len(ordering))

7 for idx in ordering:

8 sig, label, *extra_tensors = \

9 self.augmenters[idx](sig, label, *extra_tensors, **kwargs)

10 return (sig, label, *extra_tensors)

##  5 Downstream Tasks

ECG downstream tasks mainly consist of classification and segmentation. Other
research topics such as sequence generation (e.g. for denoising), self-
supervised representation learning are less active, although having no less
importance. Currently, torch_ecg focuses on the former two tasks, i.e.
classification and segmentation.

Classification involves producing a probability vector from an ECG signal,
usually used for detecting arrhythmias. The most commonly used architectures
are CRNN [22] as sketched in Figure 2 and its degeneration without recurrent
part, i.e. CNN [15, 16]. In most cases, an additional recurrent module
slightly improves the model performance at the cost of slowing down the
convergence. The CNN backbone is the core of the CRNN architecture, and is the
most extensively studied in the literature, as can be inferred from Table I.

CNN ResNet, DenseNet, Branched CNN, etc. RNNLSTM, etc., optionalAttnSE, GC,
etc., optionalLinear Figure 2: A typical CRNN architecture. The CNN backbone
is the core of this architecture. The optional RNN module and attention module
provide some effect of regularization. For classification, feature maps
usually go through an additional global pooling layer to cancel the time
dimension.

Finer-grained tasks including wave delineation (i.e. delineating the P, Q, R,
S, T waveforms of the ECG), R peak detection (QRS detection), precise event
localization, etc., could be uniformly abstracted as segmentation tasks. This
type of tasks can be dealt with using U-Nets [23], fully convolutional
networks (FCN), CRNN [21], etc. When using CRNNs, the difference to a
classification model is that the global pooling layer (or layers cancelling
the time dimension) is not needed. Instead, one might need upsampling (done by
interpolation in PyTorch) to recover the length in the time axis for the model
output.

There are also models taking data derived from ECGs as input. For example, in
[40, 39] LSTM models are proposed to predict atrial fibrillation from RR
sequences derived from ECGs. torch_ecg also implements a set of neural
networks of such types, allowing additional structures, for example, attention
modules, CRF modules, etc., which augment the performance of baseline models
[40].

##  6 Benchmark Studies

torch_ecg includes several benchmark studies using databases mentioned in the
Data and Code Availability section, to illustrate the usage of this deep
learning library, clarifying principles for building pipelines for solving ECG
processing tasks, and reproducing results from the literature. The application
of torch_ecg in these benchmark case studies is quite simple. One only needs
to

  * •

choose a model type (e.g. CRNN, U-Net, etc.), build a model from a built-in or
user-defined configuration file (dictionary), and optionally implement an
inference method that includes post-processing of the probability vectors.

  * •

implement a “Dataset” (as a child class of the PyTorch Dataset class) using
corresponding “data_reader” from torch_ecg’s databases module which pre-loads
all training data, or loads training data on the fly during training. The
implementation of the Dataset depends on the input that the model accepts.

  * •

inherit torch_ecg’s abstract base class BaseTrainer to setup a trainer by
implementing a function that controls the workflow of one step and a function
that evaluates the model, and filling in several mandatory properties, for
example the batch dimension of the input, and the monitor for the model
selection, etc. The implementation of the workflow of one step is quite simple
and in most cases can be of the following form

⬇

1def run_one_step(self, *data):

2 signals, labels = data

3 signals = signals.to(self.device)

4 labels = labels.to(self.device)

5 preds = self.model(signals)

6 return preds, labels

The implementation of the evaluation function is slightly more complicated,
directly related to the metrics of the ECG tasks.

###  6.1 PhysioNet/Computing in Cardiology Challenge (CinC) Series

The CinC series started in 2000 whose topic is detecting sleep apnea from the
ECG. In the recent 2 years [1, 2, 3], CinC focuses on the problem of detecting
a wide range of ECG abnormalities from the standard 12-lead ECGs and critical
problems derived from it. torch_ecg creates benchmark studies using CinC2020
and CinC2021 challenge databases. The fundamental model architecture used is
the CRNN family. Examples are provided using various CNN backbones, including
variants from [15, 16] and from [34], as well as from [21], etc. Such models
have proven their effectiveness in the two challenges. The novelty of these
two challenges is their wide range of ECG abnormalities (totally 130+) that
are included in the databases. Another innovation point is the study of data
redundancy of the ECGs, i.e. using a subset of the standard 12 leads for
detecting the ECG abnormalities. torch_ecg develops a mechanism of “lead-wise”
convolutions for this problem.

Typical experiments comparing a “normal” neural network model and its “lead-
wise” (abbreviated as “lw”) variant are illustrated in Figure 3. The
asymmetric loss function [45] is used for model parameter optimization. The
optimizer is the AMSGrad variant of AdamW [46], used in conjunction with the
OneCycleLR [47] learning rate scheduler of maximum learning rate 0.002 and
maximum total epoch number 50. This combination of optimizer and learning rate
scheduler will be frequently used in the rest benchmark case studies.

Figure 3: Statistics of the training processes of two CRNN models on the
CinC2021 database. One uses the CNN backbone proposed in [21], the other uses
its “lead-wise” variant (with “lw” prefix). The computation of the challenge
score can be found in Section 2.2.3 of [2].

###  6.2 China Physiological Signal Challenge (CPSC) Series

The CPSC series started in 2018 with the initial problem of the classification
of 8 types of ECG abnormalities (totally 9, plus the normal sinus rhythm
(NSR)) from the standard 12-lead ECGs, whose database now becomes a part of
the CinC2020/2021 challenge database. Afterwards, the problems of QRS
detection [4], precise localization of premature beats [5] and atrial
fibrillation (AF) events [6] are raised. Such problems have finer granularity,
as opposed to classification problems, hence are more difficult to solve.
torch_ecg provides multiple or hybrid solutions using various models for these
problems. For example, the problem of the detection of AF events raised by
CPSC2021 [6] requires precise (beat-wise) localization of onsets and offsets
of the AF segments in varying-length 2-lead ECGs. It is particularly difficult
for paroxysmal AF ECGs where there are multiple AF segments in one ECG
recording. CRNN models and U-Net models both apply to this problem, and so
does LSTM models using RR intervals derived from the ECGs.

Results of experiments reproducing [21] along with a experiment using U-Net
[23] on the benchmark CPSC2019 is shown in Figure 4. The metric “QRS score” is
defined in Section 3.1 of [4]. Note that its original implementation has bugs
that score a prediction slightly higher than it should be scored. torch_ecg
re-implements this metric function and fixes the bugs in this benchmark study.

Figure 4: Experiments on CPSC2019 using CRNN, pure CNN and U-Net models.
Training setups are similar to the experiments in Figure 3 but with the
maximum total epoch number changed to 150. Early stopping is triggered in each
of the 3 experiments.

###  6.3 Other PhysioNet Datasets

Despite the classical MIT-BIH Arrhythmia Database [48], many new and high-
quality databases are proposed in recent years at PhysioNet. The Lobachevsky
University Electrocardiography Database (LUDB) [8, 7] was released in 2020.
This database consists of standard 12-lead ECGs, with annotations of the P
waves, T waves and the QRS complexes. Along with this database, a U-Net model
[23] was proposed for the task of ECG wave delineation.

torch_ecg re-implements, with slight adjustments, this work as a benchmark
case study. A typical experiment is depicted in Figure 5. Metrics, including
sensitivity, precision, mean error, standard deviation and f1 score which is
the monitor for model selection, are computed following Section 3.2 of [23].
The loss function used is again the focal loss function.

Figure 5: A typical experiment of training a U-Net model for wave delineation
using the LUDB. The f1 score is the macro-averaged of f1 scores of the 3
waveforms, namely the P wave, the T wave and the QRS complex, as defined in
[23]. The focal loss function [49] is used for model parameter optimization. A
burst improvement of the f1 score can be observed near epoch 40.

The full list of currently implemented benchmark studies in torch_ecg is
collected in Table II.

3.5 Database | ECG type | Annotations | Tasks | Models
---|---|---|---|---
2 CinC2020 | standard 12-lead | ECG abnormalities | classification | CRNN
CinC2021 | standard 12-lead | ECG abnormalities | classification | CRNN
CPSC2019 | single-lead | R peaks | segmentation | CRNN, U-Net
CPSC2020 | single-lead | premature beats | segmentation | CRNN
CPSC2021 | 2-lead | R peaks, AF episodes | segmentation | CRNN, U-Net, RR-LSTM
LUDB | standard 12-lead | ECG waveforms | segmentation | U-Net
3.5 |  |  |  |
TABLE II: Summary of benchmark studies included in torch_ecg. Note that beat-
wise classification is treated as segmentation in the table. Most annotations
are given in the WFDB annotation file format, a few are given by MATLAB files.

##  7 Conclusions and Future Work

torch_ecg provides the ECG research community with an open-source deep
learning library, aiming at bridging the gap between its growing demand for
deep learning and the current status that such resources are dispersed. It
offers a convenient and modular way for automatic building and flexible
scaling of the networks and provides a neat and uniform way of organizing the
preprocessing procedures and augmentation techniques for preparing the input
data for the models. It gathers and maintains a comprehensive list of neural
network architectures both from literature and novel. Moreover, it includes
several benchmark case studies as tutorials for using this library, which
serve as guidelines and principles for building pipelines for solving a wide
range of ECG processing tasks. torch_ecg would continue to grow and evolve,
collecting and creating more powerful deep learning techniques and more
effective neural network architectures.

## Institutional Review Board (IRB)

This work does not require IRB approval.

## Acknowledgments

The authors would like to thank professor Deren Han from LMIB, School of
Mathematical Sciences, Beihang University and professor Wenjian Yu from the
Department of Computer Science and Technology, BNRist, Tsinghua University for
generously providing GPU servers to help accomplish this work. This work is
supported by NSFC under grant No. 11625105, 12131004.

## References

  * [1] Matthew A Reyna et al.  “Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020”  In _2020 Computing in Cardiology_ 47, 2020, pp. 1–4  IEEE
  * [2] Erick A Perez Alday et al.  “Classification of 12-lead ECGs: the Physionet/Computing in Cardiology Challenge 2020”  In _Physiological Measurement_ 41.12 IOP Publishing, 2020, pp. 124003
  * [3] Matthew A Reyna et al.  “Will Two Do? Varying Dimensions in Electrocardiography: the PhysioNet/Computing in Cardiology Challenge 2021”  In _Computing in Cardiology_ 48 IEEE, 2021, pp. 1–4
  * [4] Hongxiang Gao et al.  “An Open-Access ECG Database for Algorithm Evaluation of QRS Detection and Heart Rate Estimation”  In _Journal of Medical Imaging and Health Informatics_ 9.9 American Scientific Publishers, 2019, pp. 1853–1858
  * [5] Zhipeng Cai et al.  “An Open-Access Long-Term Wearable ECG Database for Premature Ventricular Contractions and Supraventricular Premature Beat Detection”  In _Journal of Medical Imaging and Health Informatics_ 10.11 American Scientific Publishers, 2020, pp. 2663–2667
  * [6] Xingyao Wang et al.  “Paroxysmal Atrial Fibrillation Events Detection from Dynamic ECG Recordings: The 4th China Physiological Signal Challenge 2021”  PhysioNet, 2021  DOI: 10.13026/KSYA-QW89
  * [7] Alena I Kalyakulina et al.  “Lobachevsky University Electrocardiography Database”  PhysioNet, 2021  DOI: 10.13026/EEGM-H675
  * [8] Alena I Kalyakulina et al.  “LUDB: A New Open-Access Validation Tool for Electrocardiogram Delineation Algorithms”  In _IEEE Access_ 8 IEEE, 2020, pp. 186181–186190
  * [9] Ary L Goldberger et al.  “PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals”  In _Circulation_ 101.23 Am. Heart Assoc., 2000, pp. e215–e220
  * [10] Alex Krizhevsky, Ilya Sutskever and Geoffrey E Hinton  “Imagenet Classification with Deep Convolutional Neural Networks”  In _Advances in Neural Information Processing Systems_ 25 Curran Associates, Inc., 2012, pp. 1097–1105
  * [11] Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun  “Deep Residual Learning for Image Recognition”  In _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ IEEE, 2016, pp. 770–778
  * [12] François Chollet  “Xception: Deep Learning with Depthwise Separable Convolutions”  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 1251–1258
  * [13] Yuan Liu et al.  “A Deep Learning System for Differential Diagnosis of Skin Diseases”  In _Nature Medicine_ 26.6 Nature Publishing Group, 2020, pp. 900–908  DOI: 10.1038/s41591-020-0842-3
  * [14] Tafadzwa L. Chaunzwa et al.  “Deep Learning Classification of Lung Cancer Histology Using CT Images”  In _Scientific Reports_ 11.1 Springer Science and Business Media LLC, 2021
  * [15] Awni Y Hannun et al.  “Cardiologist-level Arrhythmia Detection and Classification in Ambulatory Electrocardiograms using a Deep Neural Network”  In _Nature Medicine_ 25.1 Nature Publishing Group, 2019, pp. 65
  * [16] Antônio H Ribeiro et al.  “Automatic Diagnosis of the 12-lead ECG using a Deep Neural Network”  In _Nature Communications_ 11.1 Nature Publishing Group, 2020, pp. 1–9
  * [17] Adam Paszke et al.  “PyTorch: An Imperative Style, High-Performance Deep Learning Library”  In _Advances in Neural Information Processing Systems 32_ Curran Associates, Inc., 2019, pp. 8024–8035
  * [18] Thomas Wolf et al.  “Transformers: State-of-the-Art Natural Language Processing”  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ Online: Association for Computational Linguistics, 2020, pp. 38–45  URL: https://www.aclweb.org/anthology/2020.emnlp-demos.6
  * [19] Feifei Liu et al.  “An Open Access Database for Evaluating the Algorithms of Electrocardiogram Rhythm and Morphology Abnormality Detection”  In _Journal of Medical Imaging and Health Informatics_ 8.7 American Scientific Publishers, 2018, pp. 1368–1373
  * [20] Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin and David Lopez-Paz  “mixup: Beyond Empirical Risk Minimization”  In _International Conference on Learning Representations_ , 2018  URL: https://openreview.net/forum?id=r1Ddp1-Rb
  * [21] Wenjie Cai and Danqin Hu  “QRS Complex Detection Using Novel Deep Learning Neural Networks”  In _IEEE Access_ 8 IEEE, 2020, pp. 97082–97089
  * [22] Qihang Yao et al.  “Multi-class Arrhythmia Detection from 12-Lead Varied-Length ECG using Attention-based Time-Incremental Convolutional Neural Network”  In _Information Fusion_ 53 Elsevier, 2020, pp. 174–182
  * [23] Viktor Moskalenko, Nikolai Zolotykh and Grigory Osipov  “Deep Learning for ECG Segmentation”  In _International Conference on Neuroinformatics_ , 2019, pp. 246–254  Springer
  * [24] Hubert V Pipberger, Edward D Freis, Leonard Taback and Henry L Mason  “Preparation of Electrocardiographic Data for Analysis by Digital Electronic Computer”  In _Circulation_ 21.3 Am. Heart Assoc., 1960, pp. 413–418
  * [25] George Moody, Tom Pollard and Benjamin Moody  “WFDB Software Package”  PhysioNet, 2021  DOI: 10.13026/ZZPX-H016
  * [26] Chen Xie et al.  “Waveform Database Software Package (WFDB) for Python”  PhysioNet, 2021  DOI: 10.13026/G35G-C061
  * [27] Carlos Carreiras et al.  “BioSPPy: Biosignal Processing in Python” [Online], 2015–  URL: https://github.com/PIA-Group/BioSPPy/
  * [28] Karen Simonyan and Andrew Zisserman  “Very Deep Convolutional Networks for Large-Scale Image Recognition”  In _International Conference on Learning Representations_ , 2015
  * [29] Jiapu Pan and Willis J Tompkins  “A Real-time QRS Detection Algorithm”  In _IEEE Transactions on Biomedical Engineering_ IEEE, 1985, pp. 230–236
  * [30] Ross Wightman  “PyTorch Image Models”  In _GitHub Repository_ GitHub, https://github.com/rwightman/pytorch-image-models, 2019  DOI: 10.5281/zenodo.4414861
  * [31] Sangdoo Yun et al.  “CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features”  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2019, pp. 6023–6032
  * [32] Saining Xie et al.  “Aggregated Residual Transformations for Deep Neural Networks”  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 1492–1500
  * [33] Tong He et al.  “Bag of Tricks for Image Classification with Convolutional Neural Networks”  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2019, pp. 558–567
  * [34] Tal Ridnik et al.  “TResNet: High Performance GPU-Dedicated Architecture”  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2021, pp. 1400–1409
  * [35] Gao Huang, Zhuang Liu, Laurens Van Der Maaten and Kilian Q Weinberger  “Densely Connected Convolutional Networks”  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 4700–4708
  * [36] Jie Hu et al.  “Squeeze-and-Excitation Networks”  In _IEEE Transactions on Pattern Analysis and Machine Intelligence_ 42.8 Institute of Electrical and Electronics Engineers (IEEE), 2020, pp. 2011–2023  DOI: 10.1109/tpami.2019.2913372
  * [37] Yue Cao et al.  “GCNet: Non-Local Networks Meet Squeeze-Excitation Networks and Beyond”  In _2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)_ , 2019, pp. 1971–1980  DOI: 10.1109/ICCVW.2019.00246
  * [38] John D. Lafferty, Andrew McCallum and Fernando C.. Pereira  “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data”  In _Proceedings of the Eighteenth International Conference on Machine Learning_ , ICML ’01  Morgan Kaufmann Publishers Inc., 2001, pp. 282–289
  * [39] Oliver Faust et al.  “Automated Detection of Atrial Fibrillation using Long Short-Term Memory Network with RR Interval Signals”  In _Computers in Biology and Medicine_ 102 Elsevier, 2018, pp. 327–335
  * [40] Hao Wen et al.  “A Scalable Hybrid Model for Atrial Fibrillation Detection” DOI: 10.1142/s0219519421400212  In _Journal of Mechanics in Medicine and Biology_ 21.05 World Scientific Pub. Co. Pte. Lt., 2021, pp. 2140021
  * [41] Ashish Vaswani et al.  “Attention is All You Need”  In _Advances in Neural Information Processing Systems_ , 2017, pp. 5998–6008
  * [42] Xiangyu Zhang et al.  “Over-Fitting Suppression Training Strategies for Deep Learning-Based Atrial Fibrillation Detection”  In _Medical & Biological Engineering & Computing_ 59.1 Springer, 2021, pp. 165–173
  * [43] Terrance DeVries and Graham W Taylor  “Improved Regularization of Convolutional Neural Networks with Cutout”  In _arXiv preprint arXiv:1708.04552_ , 2017
  * [44] Christian Szegedy et al.  “Rethinking the Inception Architecture for Computer Vision”  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 2818–2826
  * [45] Tal Ridnik et al.  “Asymmetric Loss for Multi-Label Classification”  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 82–91
  * [46] Sashank J. Reddi, Satyen Kale and Sanjiv Kumar  “On the Convergence of Adam and Beyond”  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_ OpenReview.net, 2018
  * [47] Leslie N Smith and Nicholay Topin  “Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates”  In _Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications_ 11006, 2019, pp. 1100612  International Society for Optics and Photonics
  * [48] George B Moody and Roger G Mark  “The Impact of the MIT-BIH Arrhythmia Database”  In _IEEE Engineering in Medicine and Biology Magazine_ 20.3 IEEE, 2001, pp. 45–50
  * [49] Tsung-Yi Lin et al.  “Focal Loss for Dense Object Detection”  In _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 2980–2988