# Enhancing ECG Analysis of Implantable Cardiac Monitor Data: An Efficient
Pipeline for Multi-Label Classification

Amnon Bleich, Antje Linnemann, Benjamin Jaidi, Björn H Diem, and Tim OF Conrad

###### Abstract

Implantable Cardiac Monitor (ICM) devices are demonstrating as of today, the
fastest-growing market for implantable cardiac devices. As such, they are
becoming increasingly common in patients for measuring heart electrical
activity. ICMs constantly monitor and record a patient’s heart rhythm and when
triggered - send it to a secure server where health care professionals (denote
HCPs from here on) can review it. These devices employ a relatively simplistic
rule-based algorithm (due to energy consumption constraints) to alert for
abnormal heart rhythms. This algorithm is usually parameterized to an over-
sensitive mode in order to not miss a case (resulting in relatively high
false-positive rate) and this, combined with the device’s nature of constantly
monitoring the heart rhythm and its growing popularity, results in HCPs having
to analyze and diagnose an increasingly growing amount of data. In order to
reduce the load on the latter, automated methods for ECG analysis are nowadays
becoming a great tool to assist HCPs in their analysis. While state-of-the-art
algorithms are data-driven rather than rule-based, training data for ICMs
often consist of specific characteristics which make its analysis unique and
particularly challenging. This study presents the challenges and solutions in
automatically analyzing ICM data and introduces a method for its
classification that outperforms existing methods on such data. As such, it
could be used in numerous ways such as aiding HCPs in the analysis of ECGs
originating from ICMs by e.g. suggesting a rhythm type.

###### Index Terms:

ECG, ICM, Classification, Semi-Supervised-Learning.

##  I Introduction

Automated classification of ECG data has been a field of active research over
the past years, and various methods have been proposed to solve this task (see
e.g. [13, 21]). However, most of these approaches require that not only the
ECG data itself has very good quality, but also that the data labels for each
episode are available and correct. These requirements seem to be fulfilled in
most clinical research setups (with professional ECG devices and trained
staff), but are rarely met in the world of implantable devices, such as with
_Implantable Cardiac Monitors_ (ICMs).

ICMs are an important part of implantable cardiac devices and as such play a
large role in medical diagnostics. They are becoming more and more common as a
diagnostic tool for patients with heart irregularities. These devices are
characterized by a relatively simple classification algorithm that allows
detecting and sending of suspicious ECG episodes to HCPs for further
inspection. Because this algorithm prioritizes sensitivity over specificity
(as it can’t afford to miss heart rhythm irregularities) many of the
detections are false positives, i.e. the recorded episodes are often not a
real indication of a problem. The outcome is an increasing amount of ECG data
that is sent regularly to HCPs for manual diagnoses. As a result, the current
bottleneck of the ICM in treating patients is not in the device itself, but
rather in the scalability in terms of manual analysis done by HCPs. In other
words, due to the rapid growth in the number of sent episodes, it is expected
that in the near future HCPs will not be able to give the proper attention
needed for analyzing it [2]. Therefore, to reduce the load on HCPs that
analyze episodes recorded by ICMs, an automated method to assist HCPs in
annotating and/or suggesting probable arrhythmia in ICM episodes is becoming
essential.

Furthermore, most available methods and existing studies for ECG analysis
focus on the relatively high quality of at least 2-lead ECG signals and
contain high-quality manually assigned labels. In contrast to that, ICMs
produce only 1-(variant) lead ECG data – or more correctly 1 subcutaneous ECG
(sECG) data – with variant morphology due to variable implantation sites and
are of comparably low resolution (128 Hz). In addition, manually labeled
datasets often only provide inaccurate labels. Thus, available analysis
methods for the better quality data (2-or-more leads) are hardly suitable for
the analysis of the lower quality ICM data (see also section VI for a
performance evaluation of these methods on ICM data).

In this paper, we address the above-mentioned problems and present a novel
method that allows de-novo labeling and re-labeling for ECG episodes acquired
from ICM devices. More specifically, the method allows assigning labels to
segments of ECG episodes. Moreover, the method is applicable to low-resolution
data-sets with small sample size, class imbalance, and inaccurate and missing
labels. We compare our method to two other commonly used methods on such (or
similar enough) data and show the superiority of our method (especially on
minority classes). Although the overall performance has room for improvement,
it can nevertheless be seen as a milestone on the way to automatically
classify such data or e.g., serve as an arrhythmia recommender for HCPs.

This paper proposes a novel data analysis pipeline for ICM data. The
highlights of this pipeline are:

  1. 1.

The novel method allows for de-novo labeling and re-labeling of ECG episodes
acquired from Implantable Cardiac Monitors (ICMs). The main goal is to reduce
the load on HCPs by assisting in annotating and suggesting probable arrhythmia
in ICM episodes.

  2. 2.

The method is applicable to low-resolution data-sets with small sample size,
class imbalance, and inaccurate and missing labels. This is a significant
contribution as most available methods and existing studies for ECG analysis
focus on high-quality manually assigned labels for at least 2-lead ECG
signals, which are not suitable for the analysis of low-quality ICM data.

  3. 3.

The results of our experiments suggest that our new approach is superior to
two other commonly used methods on ICM data, especially on minority classes.
Although the overall performance has room for improvement, it can be seen as a
milestone on the way to assisting HCPs as an arrhythmia recommender.

Before we dive into the details of our new method (section IV), we will first
provide background information about ICM devices and the acquired data in the
next section. We will then present our experiments and results (sections V and
VI), including comparison and evaluation with other established methods.
Finally, we discuss and conclude our findings (sections VII and VIII).

##  II Implantable Cardiac Monitors

Implantable Cardiac Monitors (ICMs) continuously monitor the heart rhythm (see
[18] for a detailed overview). The device, also known as implantable loop
recorder (ILR), continuously records a patient’s subcutaneous
electrocardiogram (sECG). Once triggered, it stores the preceding 50 seconds
of the recording up to the triggering event, plus 10 seconds after the
triggering event. (Note that there are cases where the recorded signal is less
than 60 seconds due to compression artifacts). The result is a sECG which is
(up to) 60 seconds long and in which we expect to see the onset and/or offset
of an arrhythmia (i.e. the transition from normal sECG to abnormal one or vice
versa). The stored episodes are sent to a secure remote server once per day.

Possible detection types include: atrial fibrillation (AF), high ventricular
rate (HVR), asystole, bradycardia, and sudden rate drop.

Detections (or triggers) are based on simple rules and thresholds, namely: (i)
the variability of the R-R interval exceeding a certain threshold for a set
amount of time for AF, (ii) a predefined number of beats exceeding a certain
threshold of beats-per-minute for HVR, (iii) mean heart rate below a certain
threshold for a set time for bradycardia, and (iv) pause lasting more than a
set amount of time for asystole.

In summary, data is retrieved daily from the device through a wireless
receiver for long-distance telemetry. The receiver forwards the data to a
unique service center by connecting to the GSM (Global System for Mobile
Communication) network. The Service Center decodes, analyzes and provides data
on a secure website, with a complete overview for the attending hospital
staff. Remote daily transmissions include daily recordings of detected
arrhythmia. Transmitted alerts were reviewed on all working days by a HCP who
is trained in cardiac implantable electronic devices.

###  II-A Challenges with ICM Data

Data acquired from an ICM device has several quite unique characteristics,
compared to data acquired by more commonly used ECG devices, e.g. in a
clinical setting. Some of the key challenges include:

Variant ECG morphologies: Unlike e.g., classical Holter ECG recording device,
the ICM device allows for a variant implant site relative to the heart, which
results in a variant ECG morphology. This variance in ECG morphology
introduces an artifact that makes it difficult for machine learning models to
compensate for.

No “normal” control group: Due to the nature of ICMs, a record is only sent
when an abnormal event occurs. This means that the available data consist of -
almost only - ECG episodes that contain an unusual event - which triggered the
sending of the episode.

Inaccurate labels: The human factor in the manual labeling of thousands of
episodes, each of which requires specialized attention, contributes to a
degree of inaccuracy in the labels assigned to the samples. Furthermore, the
varying definitions of labels among HCPs contribute to this issue. This is
further increased by the fact that many annotation platforms allow the
assignment of global labels only. This means that an annotation doesn’t have a
start and end time, but rather refers to the entire episode. This is
unsuitable for the typical data-set produced by ICMs as these consist in the
majority of cases of 60 seconds long episodes, in which the onset of a
particular arrhythmia is expected - i.e. by definition a single heart rhythm
(viz. label) cannot relate to an entire episode.

Class imbalance: The false detection ratio of the device results in the vast
majority of episodes being Sinus episodes with some artifact that triggered
the storing of the episode.

Small multi-label training-sets: Most supervised learning models of ECG data
require relatively large data-sets of manually annotated ECG episodes.
However, since it involves highly proficient personnel to thoroughly review
every sample in the training data-set (especially in the case of multi-class
labeling), these tend to be relatively small.

Low resolution: ICMs compress data before storing it, resulting in a low
sampling rate (128 Hz compared to a minimum of 300 Hz in comparable
applications).

Noise: Movements of patients might result in noisy amplitude which could
trigger a device recording - this could result in the irregular class
distribution in the data-set, especially regarding noise.

##  III Related work

Various methods have been published in recent years for the automatic
diagnosis of ECG data. In [35], Stracina and co-workers give a good overview
of the developments and possibilities in ECG recordings that have been made
since the first successful recording of the electrical activity of the human
heart in 1887. This review also includes recent developments based on deep
learning. Most of the available analysis methods aim at the classification of
ECG episodes, e.g., to determine whether an ECG signal indicates Atrial
Fibrillation (AF) or any other non-normal heart rhythm.

Most state-of-the-art solutions for ECG classification use either feature-
based models such as bagged/boosted trees [9, 10, 17], SVM, k-nearest
neighbor, etc. [35] or Deep Learning models such as convolutional neural
networks (CNN) [41]. While Deep Learning approaches usually use the raw 1d ECG
signal [19, 5, 29, 30], feature-based ECG arrhythmia classification models
involve the extraction of various features from ECG signals, such as time-
domain and frequency-domain features.

Spectral features are derived from the frequency components of the ECG signal
and are extracted e.g. using Wavelet transform, Wavelet decomposition, and
power spectral density analysis [25].

Time domain features, on the other hand, are based on the P-waves, QRS
complexes, and T-waves, including their regularity, frequency, peaks, and
onsets/offsets [35]. Wesselius and co-workers [38] have evaluated important
time-domain features, such as Poincaré plots or turning point ratios (TPR), as
well as other ECG wave detection algorithms (such as P- and F-waves) and
spectral-domain features, including Wavelet analysis, phase space analysis,
and Lyapunov exponents. These time domain features usually require the
detection of the respective waves, with the majority of algorithms focusing on
the detection of QRS complexes due to their importance in arrhythmia
classification and the fact that they are relatively easy to detect. For
instance, a few of the most important features for ECG signal classification
are based on RR intervals, including the time between two consecutive R peaks,
with R-peaks commonly detected via the detection of QRS complexes [24].
Popular QRS detectors that are used by e.g. [23] are jqrs ([6, 7]) which
consists of a window-based peak energy detector, gqrs [34] which consists of a
QRS matched filter with a custom-built set of heuristics (such as search back)
and wqrs[42] which consists of a low-pass filter, a nonlinearly scaled curve
length transformation, and specifically designed decision rules.

###  III-A PhysioNet/CinC 2017 Challenge

We were also interested in methods that have been shown to be successful in
applied settings, such as challenges or real-world comparisons. For this, we
searched the literature for recent surveys. We used PubMed in January 2023
with the query “single lead ECG machine learning review” OR “single lead ecg
AI review” and restricted the time-frame to 2020 and newer. The results were
manually inspected and led us to a challenge that took place in 2017 and was
related to our approach.

The PhysioNet/Computing in Cardiology 2017 Challenge [12] was an organized
competition by the PhysioNet initiative - a research resource that offers
open-access to databases of physiological signals and time series, and the
Computing in Cardiology (CinC) - an annual conference that focuses on the
application of computer science techniques to problems in cardiovascular
medicine and physiology. In 2017 the two collaborated to conduct the
PhysioNet/CinC 2017 Challenge, which was focused on the detection of atrial
fibrillation (AF) from short ECG recordings. This challenge was intended to
encourage the development of novel algorithms and techniques and attracted
experts from diverse fields, including computer science, cardiology, and
signal processing, thereby facilitating a multidisciplinary approach to the
problem. The results of the competition demonstrated the potential of data-
centered techniques, such as deep learning and ensemble methods, for enhancing
the accuracy of AF detection.

The best-performing approaches include solutions based on deep learning and
decision trees. For example, the winning solution of the challenge by Datta
and co-workers [13] approaches multi-label classification via cascaded binary
Adaboost [17] classifiers along with several pre-processing steps including
noise detection via signal filters, PQRST points detection and 150 features
that are derived from them. Another example is the approach by one of the top-
scoring teams that proposes a deep learning based solution[41], consisting of
a 24-layer CNN and a convolutional recurrent neural network.

Two of the top-scoring algorithms of this challenge, for which source code was
publicly available, are: (1) a feature-based approach that extracts 169
features for each episode and uses an ensemble of bagged trees as well as a
multilayer perceptron for classification and (2) a deep learning-based method
using a 34-layer ResNet. Both approaches have been compared in a paper by
Andreotti and co-workers [4, 20] and are still considered state-of-the-art
approaches for short single-lead ECG data classification. We will use these
two approaches as a baseline to compare to our approach (see section V-A for
more details).

##  IV Method Details

In this section, we provide a detailed description of our pipeline, which we
developed to analyze and classify 60-second episodes of single-lead
subcutaneous ECG (sECG) data. The main steps include segmentation, noise
detection, embedding, and clustering (see Figure 1). The resulting clustering
is then used to construct the final classifier, which allows for the
prediction of labels for new episodes. Each of these pipeline steps will be
discussed in detail in the following subsections.

###  IV-A Input Data

The input to the pipeline is a data set consisting of multiple 60-second sECG
episodes for which at least one label has been assigned by medical experts.
Every episode potentially contains the onset and/or offset of arrhythmia (see
section II) and thus, by definition, more than one rhythm type per episode.
The annotation platform used for our dataset allows assigning labels to
episodes in a manner of selection out of a bank of 38 possible labels.
However, due to the relatively small data-set, in order to reduce the number
of classes and thus increase the number of episodes belonging to each class,
the various labels are grouped into 5 categories: normal, pause, tachycardia,
atrial fibrillation (denote aFib) and noise.

###  IV-B Training

The training pipeline consists of several steps, which are shown in Fig.1. The
following paragraphs explain the building blocks of this pipeline in more
detail.

StartInput: Many 60s ECG episodes Segmentation Denoising (CEEMD) QRS Detection
Transformation (Lorentz Plot) Embedding (2D T-SNE) Clustering (DBSCAN) Output:
ClassifierStop Figure 1: Overview of the method’s main building blocks

#### Segmentation

In order to have homogeneous episodes, i.e., one heart rhythm per episode
(which correlates to one label per episode), we divide the 60-second episodes
into six 10-second, non-overlapping sub-episodes. Each sub-episode is assigned
the label of its parent episode. Note that this label is presumed to be
inaccurate, as it could have been chosen due to a rhythm that appears in
another sub-episode of that episode and the later steps in the pipeline will
correct for this.

#### Noise Detection

(a) Example for the CEEMD method: The first row shows a single sub-episode (10
s) of an sECG episode. The signals in the rows below show the first 8
intrinsic mode functions (IMFs) resulted from running CEEMD on this sub-
episode. The IMFs are different components with decreasing frequencies of the
sub-episode, such that summation of all the IMFs results in the given sECG
sub-episode.

(b) Visualization of the noise detection step, as described in algorithm 1.
The first row shows the ECG data (same as in the left sub-figure). The rows
below (Gn, NZC, rowSum Hn) represent the derived components, as explained in
section IV-B. With Hn being the sum of the first three IMFs (IMFs #1, #2 and
#3 in the left sub-figure), NZC the number of zero-crossings in Hc within a
given window size ($\sim$0.23 in our case) and Gn a ”gate” vector with 1s
where NZC is strictly greater than 1 ($\geq 2$) and 0 otherwise. The component
dependencies are hierarchical (displayed in reverse order): rowSum Hn
computation relies on the IMFs, NZC is dependent on rowSum Hn and Gn is based
on NZC. Finally, if a gate in Gn (consecutive Gn=1 segment) is longer than a
set time (in our case, 0.75 second) The corresponding segment in the sECG is
marked as noise.

Figure 2: Example of our noise detection algorithm. Left: intrinsic mode
functions (IMFs) of an sECG. Right: components derived based on IMFs for
actual noise detection. (For details see description of algorithm 1. Algorithm
1 Our CEEMD-based noise detection algorithm

1:$ceemd=$CEEMD of ecg wave signal with sift num=10, 100 iterations and
Gaussian noise

2:Define:

3:a: $Hn=$Sum of IMF1,IMF2 and IMF3 of $ceemd$

4:b: $threshold$=0.85th quantile of abs(Hn)

5:c: $window\\_size$ = 0.234375 second

6:for $x$ in each $window\\_size$ long, center aligned sliding window on $Hn$
do

7: if $max(abs(x))$ $>$ $threshold$ then

8: $NZC[window\ start]$ = number of zero crossing in $x$

9: else

10: $NZC[window\ start]=0$

11: end if

12:end for

13:Pad NZC with leading 0s to match Hn length

14:for $i$ in $1...length(NZC)$ do

15: if $NZC[i]>1$ then $Gn[i]=1$ else $Gn[i]=0$

16:end for

17:for $gate$ in consecutive 1s series in $Gn$ do

18: if length of gate $>$ 0.75 seconds then

19: mark gate start/end times as start/end of noise segment

20: end if

21:end for

ECG data - and in particular data from ICM devices - typically contains noise,
such as baseline wander, high/low frequency interruptions and the like ([11]).
Although certain technologies in ICM are aimed at reducing this effect ([16]),
filtering out noise is essential and improves downstream processes, such as
detection of the QRS complex. Several approaches have been adjusted from the
general field of signal processing, such as extended Kalman filter, median,
low or high-pass filters or Wavelet transform [32, 14, 28, 3]. In the proposed
method, we suggest using an algorithm based on empirical mode decomposition
(EMD) [8, 33], which has been shown to deal well with high-frequency noise and
baseline wander. The pseudocode in Algorithm 1 describes a step-by-step
description of the CEEMD (Complete Ensemble Empirical Mode Decomposition [36])
- based algorithm used in order to detect noisy ECG signals. It first
decomposes the signal into its IMFs (intrinsic mode function [22]) using CEEMD
- where the first IMFs account for high-frequency signal and the last IMFs
account for low frequency (e.g. baseline wander, see figure 2(a)). It then
sums up the first 3 IMFs (the components accounting for the high-frequency
signal - denoted by Hn) and checks over a sliding window whether Hn in that
window contains a value greater than a certain threshold and if it has more
than a set number of zero-crossings. The result of this is a binary vector
with 1s where both conditions are met and 0s otherwise (denote Gn \- see
figure 2(b)). Finally, if a certain gate (a sequence of 1s in the Gn vector)
is longer than a set threshold, the corresponding part of the original ECG
signal is marked as noise. All the parameters (namely threshold, window size
and gate length) were selected empirically using Jaccard index to quantify the
overlap between the detected noise and the segments annotated as noise in the
test dataset, as the objective function. To conclude, in order to see the
relevance of the noise detection algorithm to our data, we marked all the
noisy segments in it and compared with the assigned labels. Results of this
step can be found in the result section (section VI and in particular Fig. 5).

#### Embedding

To be able to deal with the high-dimensional ECG data and prevent overfitting
(due to the curse-of-dimensionality problem) as much as possible, we perform a
projection of the data (approximately 7600D for an episode or 1300D for a sub-
episode) into a low-dimensional (2D) space. We suggest the following steps for
the dimensionality reduction (see Figure 3). First, we (1) perform QRS complex
detection on the ECG signals, to identify the R-peaks (Figure 3(a)). Based on
the R-peaks, we can (2) transform the ECG signals into a Lorenz plot (Figure
3(b)), which we (3) discretize into a 2D histogram, using a $N\times N$ grid
(Figure 3(c)). Then, we (4) flatten the 2D histogram into a 1D vector with
$N\cdot N$ elements. The last step is (5) to use t-SNE to project the
resulting vectors into the 2D space to get the final embedding. In more
details:

(1) R-peak Detection: The R-peak is a component of the QRS complex which
represents the heart-beats in an ECG signal. There are several algorithms for
QRS detection such as gqrs [34], Pan-Tompkins [27], maxima search [31] and
matched filtering. In our pipeline, we use an aggregation of these four
approaches via a kernel density estimation based voting system.

(2) Lorenz Plot Transformation: In the context of ECG, a _Lorenz plot_ (also
called Poincaré plot) [26], is the plotting of $dRR(i)$ vs. $dRR(i+1)$ for
each R peak $i$ in an ECG signal, with $dRR(i)=RR(i+1)-RR(i)$, where $RR(i)$
is the i’th R-R interval (the time between two consecutive R peaks). This
visualization is often used by HCPs to detect arrhythmia.

(3) Discretization: In order to produce a 2D histogram of the Lorenz plot, the
plot is divided into equal sized 25 bins by overlaying it with a 5-by-5 grid.
The histogram is thus a count of points (viz. dRR(i)/dRR(i+1) intervals) in
each bin. The result is a 5-by-5 matrix that is flattened to form a 25
dimensional vector.

(4) t-SNE Embedding: t-SNE is an algorithm for dimension reduction [37]. It is
primarily meant for visualization (i.e. projection of multi-dimensional data
on a 2D or 3D plane) and since it does not preserve distance between points it
is not suitable for distance-based clustering (such as DBSCAN - the algorithm
used in our work). However, since it maximizes the likelihood that contiguous
data points will not be in the same cluster, it performs better than
alternative embedding methods, such as LDA [40] and PCA [39].

(a)

(b)

(c)

Figure 3: ECG Embedding steps (excluding final t-SNE step). a) raw sECG signal
with R-peaks marked with red dots. b) Lorenz Plot of the sECG signal in sub-
figure 3(a), plotting the relations between consecutive R-R intervals in 3(a)
. c) 2D histogram of the Lorenz Plot in figure 3(b) with 5x5 bins. The colors
(values) of the histograms account for the number of points in the Lorenz Plot
in each bin - brighter colors account for higher values. The latter is
flattened to form a 25 long vector which is given as input to the t-SNE
algorithm to provide a 2D representation of the sECG signal

#### Clustering

There are various clustering approaches described in the literature, with
popular ones including k-means, spectral clustering or DBSCAN [15]. In our
pipeline, we chose the DBSCAN algorithm since it outperformed other approaches
we tried on our datasets. DBSCAN stands for Density-Based Spatial Clustering
of Applications with Noise and as its name suggests it clusters together areas
with high density of points separated by areas with low density while leaving
points in sparse areas as outliers.

In order to select the maximal distance parameter, we created a k-distance
plot with $k=2\times$ number of features, which in the case of 2D t-SNE is
k=4. The reasoning for this is that the distance threshold should be
proportional to the number of dimensions to account for sparsity resulting
from it, with $2\times k$ being a common rule of thumb. Once the minimal point
number is set, the $k$-distance plot is a sorted representation of the average
distance of each point in the data to its k nearest neighbors. The area after
the elbow would therefore represent points in the data that could be
considered outliers in this sense (as finding k nearest neighbors would
require searching increasingly further) so this could mark a reasonable
maximal distance to look for neighbors. The latter resulted in a distance of
eps=1.5. We then explored the neighborhood of that point interchangeably with
the minimal number of points. For a minimal number of points, we simply tried
different values on a scale from a conservative lower to an upper bounds (2 to
30) until we found a visually satisfying results, with the number of outliers
(points not belonging to any cluster) not exceeding $10\%$ of the number of
sub-episodes. The optimal values for our case were 0.75 for maximal distance
and 15 as the minimal number of points.

#### Building the classifier

From the clustering results from the previous paragraph, a classifier that
enables label prediction of previously unseen episodes can be created. In
order to classify a sub-episode that belongs to a certain cluster (denote
cluster c) we compute the p-value of the label proportions for all labels in
that cluster, given the label proportion in the dataset. The p-value is
computed using the cumulative binomial distribution defined as:

$\begin{split}\\\ &\text{for cluster {c}},\forall l\in\text{labels},\\\
&F(k_{cl},n_{c},p_{l}):=Pr(X<k_{cl})=\sum\limits_{i=0}^{k_{cl}-1}\binom{n_{c}}{i}p_{l}^{i}(1-p_{l})^{n-i}\end{split}$
(1)

i.e. the probability for a maximum of $k_{cl}$ successes in $n_{c}$ trials
with probability $p_{l}$ for a success in a single trial. In our case,
$k_{cl}$ denotes the observed number of sub-episodes with label l in cluster
c, $n_{c}$ the size of cluster c and $p_{l}$ the proportion of label l in the
dataset (defined as $p_{l}:=\frac{d_{l}}{d}$ with $d_{l}=\text{label count in
the dataset}$ and $d=\text{dataset size}$ ).

Formula 1 is used to assign p-values of $P(l,c)=Pr(X\geq
k_{cl})=1-Pr(X<k_{cl})$ to each label l in cluster c and the label with the
lowest p-value is assigned to the sub-episode. Or in words - for a given label
and cluster with label count of $k_{cl}$ in that cluster, the p-value computed
is the probability to observe at least $k_{cl}$ sub-episodes of that label in
that cluster given the label’s proportion in the dataset and the cluster size.
The lower the probability, the higher the significance of that cluster being
labeled with that label and therefore the label with the lowest p-value in a
cluster is set to be the label of this cluster and all sub-episodes clustered
to it will be labeled as such.

To use the classifier for predicting the labels of new, unseen episodes, the
given ECG episode is segmented into 10 seconds sub-episodes and each is
projected in the same way as described above. Then, we perform a k-nearest
neighbor search (with $k=1$) on each sub-episode. We assign the cluster of the
nearest neighbor of each sub-episode as the cluster of that sub-episode and
assign the label accordingly.

Note: Several approaches could be taken for the choice of label based on the
p-value, according to the use case. For example, if one wants to ensure high
sensitivity for several classes at the expense of specificity for, say, pause
rhythm, a threshold could be set such that if the p-value for pause is lower
than that threshold, the cluster’s label will be “pause” regardless of other
classes with possibly lower p-values. More on that in section VII

##  V Experimental Setup

In the following sections, we introduce the two algorithms selected as
comparative benchmarks for our novel method, and detail the dataset used for
this study.

###  V-A Baseline Methods

Based on our literature search (Section III-A), we selected two algorithms as
baseline methods that we use to compare our new method to:

(1) a feature-based approach that extracts 169 features for each episode and
uses an ensemble of bagged trees as well as a multilayer perceptron for
classification. (2) A deep learning-based method using a 34-layer ResNet. The
two methods were used as proposed by the authors, with the following
adjustments made to match our study:

  * •

The feature-based approach uses Butterworth band-pass filters that were
adjusted for lower frequency episodes (as the original solution deals with a
frequency 300 Hz and our data has 128 Hz).

  * •

Class imbalance - The original method deals with this problem by adding
episodes from minority classes (namely AF and noise) from additional ECG
datasets. Since we wanted to evaluate the case where the data is imbalanced,
we skipped this step.

  * •

In the deep learning approach, padding is performed for episodes shorter than
the selected size, in our case 60 seconds. Due to the nature of labeling in
our test dataset and the fact that heart rhythms tend to last few seconds
rather than a minute, the test samples must be significantly shorter than 60
seconds (10 seconds in our method) so every sub-episode was padded by 0 values
to complete a 60-second episode.

###  V-B Data

Figure 4: Distribution of labels in the training-set (left) and test-sets
(right). Note the difference in the distributions is due to the way the data
was created, see section V-B for details.

The dataset used for this study was produced by the BIOTRONIK SE & Co KG
BIOMONITOR III and BIOMONITOR IIIm [1] and anonymized for this analysis. The
data produced by the device is characterized by a single-lead, 60 seconds long
ECG episodes with a sample rate of 128 Hz (7680 data points per 60 seconds
episode).

Out of the 60 seconds long stored episodes, 3607 were labeled by up to three
HCPs, each assigning a global label (a label is given to an entire episode
albeit more than one label can be given in case the HCP recognizes more than
one heart rhythm in an episode) and 4710 episodes with similar characteristics
but where the labels were assigned with start and end time (i.e. per segment
as opposed to a global label). In this study, the former was used for learning
(denote training-set) and the latter for evaluation (denote test-set).

It should be noted that although the test-set labels resolve the primary
problem with the training dataset - that a label is assigned to the entire
episode - they might still have the additional inaccuracies, and thus the test
set is still not fully trustworthy as ground truth labels.

The labels of the training- and test-sets are assigned as follows: The
training-set has global labels (i.e. one or multiple labels for the entire
episode without start/end time - see section V-B for details) and therefore
the labels of all sub-episodes are derived from that of the parent episode and
if more than one label is present the episode is duplicated. The test-set has
non-overlapping segmented labels assigned by a different annotation platform
that allows for such annotations (i.e., each label has start and end time -
see section V-B for details) and so a sub-episode is assigned the label that
has a start and end time that overlaps at least half of it (5 or more seconds
in case of 10 seconds segmentation). If there was no such label, the sub-
episode is removed from the dataset. The label distribution of the datasets
can be seen in figure 4. The figure highlights the class imbalance where e.g.
in the training set 2795 episodes represent normal episodes, 546 pause
episodes, 467 tachycardia episodes, 985 aFib episodes, and 932 noise episodes.
(Note that multiple labels per episode are counted once for each label, and
therefore the number of labels sums up to more than the number of episodes).

The reasoning for the selection of test and train sets is that the training-
set presents the difficulties referred to in section II-A and the test-set
contains similar difficulties but due to the different labeling system, the
labels are assumed to be more accurate.

##  VI Results

Performance - Baseline vs. Our method - F1 score (Specificity/Sensitivity)
---
Class |  n |  Baseline A (Feat.-based) |  Baseline B (DL-based) |  Our A (w/o noise det.) |  Our B (w/ noise det.)
Normal |  14492 |  0.71 (0.35/0.81) |  0.47 (0.62/0.40) |  0.65 (0.84/0.54) |  0.62 (0.89/0.48)
Pause |  407 |  0.03 (1.00/0.02) |  0.03 (0.99/0.03) |  0.10 (0.83/0.61) |  0.10 (0.84/0.54)
Tachycardia |  2921 |  0.00 (1.00/0.00) |  0.04 (1.00/0.02) |  0.65 (0.96/0.65) |  0.64 (0.96/0.60)
aFib |  6014 |  0.37 (0.86/0.34) |  0.38 (0.80/0.38) |  0.49 (0.85/0.47) |  0.51 (0.85/0.51)
Noise |  1352 |  0.12 (0.94/0.13) |  0.18 (0.66/0.66) |  0.24 (0.91/0.34) |  0.30 (0.86/0.61)
TABLE I: Results of Baseline vs. our method. ”Baseline A” and ”Baseline B”
correspond respectively to the feature-based (A) and Deep Learning based (B)
methods. ”Our A” and ”Our B” correspond respectively to the method presented
here without and with noise detection pre-step. All methods were trained on
the inaccurately labeled training dataset and tested on our segment-annotated
dataset. The numbers in brackets correspond respectively to specificity and
sensitivity (spec/sens). Numbers in bold mark the best result in each row
(each ground truth arrhythmia type).

The following section presents the results of our experiments and compares the
performance of our proposed method for classifying cardiac rhythms to the
described baseline methods. Our prediction pipeline involves a series of steps
that enable the generation of sECG labels, as described above. The following
sections describe the application of our pipeline to sECG data and the
achieved results, starting with the noise detection step.

###  VI-A Noise Detection: Only Limited Effects

Our proposed pipeline begins with segmentation of the given episodes, followed
by the noise detection step. To understand the influence of this noise
detection step, we performed two experiments: (A) We performed the full
pipeline without the noise detection (and removal) step. (B) We performed the
full pipeline, including the noise detection step. Here, 1444 sub-episodes
(out of 25186) were identified as noise and were removed from further
processing. We then looked at the ground-truth labels of the sub-episodes that
were labeled as “noise” by the noise identification step. As shown in Fig. 5,
the majority of the sub-episodes detected were indeed noisy parts of the ECG
data.

Figure 5: Test-set CEEMD-based noise detected sub-episodes label distribution.
Ground truth label ratio distribution of the 1444 sub-episodes in our test
data, detected by our CEEMD-based noise detection method (values in figure 1
divided by the number of sub-episodes of each label in the entire dataset
(e.g. 0.45 for noise means that 45% of the sub-episodes labeled as noise are
detected by the algorithm - it should be noted that the algorithm only detects
high frequency noise and as such, presumably some of the 55% undetected sub-
episodes labeled as noise are of different kind of noise e.g.
saturation/signal-loss).

Afterwards, we compared the overall performance of pipeline A (“without noise
detection step”) and pipeline B (“with noise detection step”). The comparison
reveals that removing the noisy sub-episodes before the training does have
only a limited effect overall (see Table I for details). The only class with a
significant change in performance is the class “noise” with an increased
F1-score from 0.24 to 0.30 and, more significantly, an increased sensitivity
from 0.34 to 0.61.

###  VI-B Embedding

Figure 6: 2D t-SNE projection of the Lorenz plot histogram embedding of all
ECG sub-episodes (training- and test data). Highlighted in green are sub-
episodes from the training-set having the respective label for each sub-plot
(top-left: Arrest, top-right: Tachycardia, bottom-left: Fibrillation, bottom-
right: Noise). Note that every sub-plot is expected to contain green-
highlighted sub-episodes outside of the high density areas because all sub-
episodes of a sECG episode in the training set inherit the - potentially wrong
- label of its parent episode (see section IV-B for details).

Applied to our 10s segmented combined test- and training dataset, the pipeline
described in section VI resulted in a 2D sub-episode projection that is
illustrated in Figure 6 with the class distribution of the training sub-
episodes highlighted in green. The figure visualizes the effectiveness of the
embedding process, where the different classes tend to concentrate in
different areas of our 2D space.

###  VI-C Clustering

We used the DBSCAN algorithm to cluster the 2D projection of the embedded
points from the previous step. Fig. 7 shows the results of this clustering
step applied to the full dataset. One can see that the clustering algorithm
manages to separate the dataset in an organic manner via areas of lower
density of data points (as opposed to e.g. arbitrary pentagonal clusters, or
many small/few large clusters). In addition, the visualization of the clusters
combined with the class distribution depicted in figure 6 can suggest probable
labels that will be given to certain clusters as part of the labeling
procedure (for clusters with high density of a certain class).

Figure 7: t-SNE base 2D visualization of the DBSCAN clustering of all
training-set and test-set data after embedding of QRS Lorenz plot 5x5 bins
histogram (see Sec. IV-B for more details).

###  VI-D Overall Performance: Better label prediction and lower runtime

The overall performance of our new method compared to the two selected
competitive methods is summarized in table I. The results show that our
method’s prediction outperforms the other two methods in almost all classes,
based on the F1 score, especially in the minority classes. As can be seen in
the table, there is quite a difference in performance between the various
heart rhythm types. The class “normal rhythm” seems to be easiest to classify.
This might be due to the large number of training samples of type “normal” in
the training-set - almost half (49.1%) of all samples have this label. This is
the only class where the baseline method outperforms our method, by a
difference of 0.06 in the F1 score. Moreover, this is arguably the rhythm we
are the least concerned about, and even in this rhythm our method
significantly outperforms the baseline in specificity with 0.84 without noise
(Our A) cleanup and 0.89 with noise cleanup (Our B) compared to 0.35 and 0.62
for the feature-based and deep learning methods respectively. In the case of
Normal heart rhythm, in most use-cases (such as alerting patients of possible
heart issues they might have), specificity is preferred over sensitivity -
i.e. we require to correctly reject Normal labels for non-normal episodes even
at the expense of incorrectly rejecting Normal labels.

Furthermore, it is apparent that our method deals better with class imbalance,
with the performance of minority classes significantly better than the
baseline. In addition, our method has a significantly lower runtime: while our
method needs about 10 minutes on the machine used for this study the baseline
methods need several hours to finish computation on the same machine. Also,
our method has the ability to ’fix’ inaccurate training labels as well as
label shorter segments of the episodes via semi-supervised learning.

##  VII Discussion

The proposed method aims to reduce clinician workload by recommending and
emphasizing certain insights, but not with a focus on making autonomous
decisions. The performance of the method supports this tendency, as while it
outperforms existing methods for the use case for which it is intended, it is
not as accurate as a professional’s manual diagnosis. In this situation, the
use-case can affect model parameters, which could have a significant effect on
the model’s efficacy. According to the authors of this work, one of the
strengths of this method is its adaptability to various scenarios through
small modifications. In order to decrease false-negatives despite an increase
in false-positives (falsely assigning normal episodes with non-normal labels),
one could call the non-normal episode even if it has a higher p-value than the
normal label. Another option is to set distinct thresholds for various labels,
as required by the use case. Such modifications could even be made to a point
where the false-negative rate is close to zero, thereby reducing the number of
episodes sent to clinicians. In such a case, additional research should be
conducted to determine whether its use as a filter on top of the ICM device’s
algorithm reduces the number of false-positives without increasing the number
of false-negatives. As opposed to the Lorenz plot histogram, larger
modifications are possible, such as using a 2D projection of a distinct set of
characteristics. Such modifications, however, necessitate additional research.

Another noteworthy aspect of the method is its potential to be incorporated
into a larger decision support system, which could combine the strengths of
various techniques or algorithms. In such a framework, the proposed method
could serve as a pre-processing phase or supplementary tool to improve the
system’s overall performance. This integration could facilitate stronger
decision-making and reduce the workload of clinicians further.

When implementing the proposed method in a real-world clinical setting, user-
centered design principles should also be considered. Usability,
interpretability, and user acceptability will be vital to the successful
adoption of the method by healthcare professionals. Future research could
investigate methods for enhancing the method’s user interface and providing
meaningful feedback to clinicians, thereby ensuring that the system’s
recommendations are in line with their requirements and expectations.

Finally, the method’s ethical implications must be exhaustively examined. As
the method is intended to reduce clinician burden rather than replace human
decision-making, it is crucial to ensure that its implementation does not
inadvertently absolve healthcare professionals of responsibility or liability.
This may involve developing explicit guidelines for the application of the
method, establishing robust monitoring mechanisms, and promoting transparency
and accountability in the design and implementation of the system.

In conclusion, the proposed method contains great potential for reducing
clinician workload and enhancing the effectiveness of medical diagnosis.
However, additional research is required to investigate its adaptability to
diverse scenarios, optimize its performance for different use cases, and
integrate it into a larger framework for decision support. In order to ensure
its successful adoption in clinical practice, it is also essential to address
the human factors and ethical considerations associated with its
implementation.

##  VIII Conclusion & Outlook

The research presented in this paper demonstrates that automatic
classification of data derived from ICM devices can be applied effectively for
a variety of practical applications, demonstrating superior performance
compared to existing state-of-the-art techniques.

Despite inherent complexities, our study demonstrates that achieving a
satisfactory level of automatic diagnosis with ICM device data is feasible.
This includes the use of ECG clustering as a supplementary tool for medical
professionals managing increasing volumes of ECG episodes and the
implementation of clustering as a preparatory step for segment adaptive
parameter optimization during post-processing. Notably, our proposed method
excels in dealing with underrepresented categories in ICM sECG data sets.

As unlabeled ICM data rapidly becomes one of the most prevalent types of ECG
data, its anticipated volumetric growth emphasizes the importance of precise
labeling. Unfortunately, the lack of specific labels diminishes its practical
utility. This study reveals that contemporary classification algorithms
struggle with such datasets. Nonetheless, the method proposed in this study
provides a promising foundation for annotating such data, which could pave the
way for large ECG episode databases with labels. This could circumvent
existing restrictions and unlock a variety of applications.

The present study pioneers a flexible methodology that enables users to
develop customized extensions and modifications. Several possibilities within
the proposed pipeline have already been investigated, but future research may
consider additional refinements to optimize overall performance based on the
data and devices in use. Limiting label prediction to high-confidence sub-
episodes (as indicated by low p-values) may be an effective strategy for
evaluating automated classification, while leaving more complex sub-episodes
for manual diagnosis. As an alternative to static 10-second sub-episodes,
dynamic episode segmentation may be utilized. This strategy would entail the
identification of specific instances during an episode in which the cardiac
rhythm shifts, enabling the segmentation of episodes based on this criterion.
This would result in sub-episodes with similar cardiac rhythm types. In
addition to the high-frequency noise detector presented in this study, non-
high-frequency noise detection techniques, such as clipping or missing data,
should be considered for comprehensive noise reduction.

In conclusion, this study provides not only a powerful new tool for the
automatic classification of ICM device data, but also a foundation upon which
further advancements can be built.

## Acknowledgment

This work was supported by the German Ministry for Education and Research
(BMBF) within the Berlin Institute for the Foundations of Learning and Data -
BIFOLD (project grants 01IS18025A and 01IS18037I) and the Forschungscampus
MODAL (project grant 3FO18501).

## References

  * [1] _BIOMONITOR III, IIIm_ , https://www.biotronik.com/en-us/products/crm/arrhythmia-monitoring/biomonitor-3, Accessed: 2022-12-08.
  * [2] Muhammad R Afzal, Julie Mease, Tanner Koppert, Toshimasa Okabe, Jaret Tyler, Mahmoud Houmsse, Ralph S Augostini, Raul Weiss, John D Hummel, Steven J Kalbfleisch, et al., _Incidence of false-positive transmissions during remote rhythm monitoring with implantable loop recorders_ , Heart Rhythm 17 (2020), no. 1, 75–80.
  * [3] Mikhled Alfaouri and Khaled Daqrouq, _Ecg signal denoising by wavelet transform thresholding_ , American Journal of applied sciences 5 (2008), no. 3, 276–281.
  * [4] Fernando Andreotti, Oliver Carr, Marco AF Pimentel, Adam Mahdi, and Maarten De Vos, _Comparing feature-based classifiers and convolutional neural networks to detect arrhythmia from short segments of ecg_ , 2017 Computing in Cardiology (CinC), IEEE, 2017, pp. 1–4.
  * [5] Zachi I Attia, Peter A Noseworthy, Francisco Lopez-Jimenez, Samuel J Asirvatham, Abhishek J Deshmukh, Bernard J Gersh, Rickey E Carter, Xiaoxi Yao, Alejandro A Rabinstein, Brad J Erickson, et al., _An artificial intelligence-enabled ecg algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction_ , The Lancet 394 (2019), no. 10201, 861–867.
  * [6] Joachim Behar, Alistair Johnson, Gari D Clifford, and Julien Oster, _A comparison of single channel fetal ecg extraction methods_ , Annals of biomedical engineering 42 (2014), no. 6, 1340–1353.
  * [7] Joachim Behar, Julien Oster, and Gari D Clifford, _Combining and benchmarking methods of foetal ecg extraction without maternal or scalp electrode data_ , Physiological measurement 35 (2014), no. 8, 1569.
  * [8] Manuel Blanco-Velasco, Binwei Weng, and Kenneth E. Barner, _Ecg signal denoising and baseline wander correction based on the empirical mode decomposition_ , Computers in Biology and Medicine 38 (2008), no. 1, 1–13.
  * [9] Leo Breiman, _Random forests_ , Machine learning 45 (2001), no. 1, 5–32.
  * [10] Tianqi Chen and Carlos Guestrin, _Xgboost: A scalable tree boosting system_ , Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, 2016, pp. 785–794.
  * [11] Gari D Clifford, Francisco Azuaje, and Patrick Mcsharry, _Ecg statistics, noise, artifacts, and missing data_ , Advanced methods and tools for ECG data analysis 6 (2006), no. 1, 18.
  * [12] Gari D Clifford, Chengyu Liu, Benjamin Moody, H Lehman Li-wei, Ikaro Silva, Qiao Li, AE Johnson, and Roger G Mark, _Af classification from a short single lead ecg recording: The physionet/computing in cardiology challenge 2017_ , 2017 Computing in Cardiology (CinC), IEEE, 2017, pp. 1–4.
  * [13] Shreyasi Datta, Chetanya Puri, Ayan Mukherjee, Rohan Banerjee, Anirban Dutta Choudhury, Rituraj Singh, Arijit Ukil, Soma Bandyopadhyay, Arpan Pal, and Sundeep Khandelwal, _Identifying normal, af and other abnormal ecg rhythms using a cascaded binary classifier_ , 2017 Computing in cardiology (cinc), IEEE, 2017, pp. 1–4.
  * [14] Philip De Chazal, Maria O’Dwyer, and Richard B Reilly, _Automatic classification of heartbeats using ecg morphology and heartbeat interval features_ , IEEE transactions on biomedical engineering 51 (2004), no. 7, 1196–1206.
  * [15] Martin Ester, Hans-Peter Kriegel, Jörg Sander, Xiaowei Xu, et al., _A density-based algorithm for discovering clusters in large spatial databases with noise._ , kdd, vol. 96, 1996, pp. 226–231.
  * [16] Giovanni B Forleo, Claudia Amellone, Riccardo Sacchi, Leonida Lombardi, Maria Teresa Lucciola, Valentina Scotti, Maurizio Viecca, Marco Schiavone, Daniele Giacopelli, and Massimo Giammaria, _Factors affecting signal quality in implantable cardiac monitors with long sensing vector_ , Journal of Arrhythmia 37 (2021), no. 4, 1061–1068.
  * [17] Yoav Freund, Robert E Schapire, et al., _Experiments with a new boosting algorithm_ , icml, vol. 96, Citeseer, 1996, pp. 148–156.
  * [18] Fabrizio Guarracini, Martina Testolina, Daniele Giacopelli, Marta Martin, Francesco Triglione, Alessio Coser, Silvia Quintarelli, Roberto Bonmassari, and Massimiliano Marini, _Programming optimization in implantable cardiac monitors to reduce false-positive arrhythmia alerts: A call for research_ , Diagnostics 12 (2022), no. 4, 994.
  * [19] Awni Y Hannun, Pranav Rajpurkar, Masoumeh Haghpanahi, Geoffrey H Tison, Codie Bourn, Mintu P Turakhia, and Andrew Y Ng, _Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network_ , Nature medicine 25 (2019), no. 1, 65–69.
  * [20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, _Deep residual learning for image recognition_ , Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [21] Shenda Hong, Meng Wu, Yuxi Zhou, Qingyun Wang, Junyuan Shang, Hongyan Li, and Junqing Xie, _Encase: An ensemble classifier for ecg classification using expert features and deep neural networks_ , 2017 Computing in cardiology (cinc), IEEE, 2017, pp. 1–4.
  * [22] Norden E Huang, Zheng Shen, Steven R Long, Manli C Wu, Hsing H Shih, Quanan Zheng, Nai-Chyuan Yen, Chi Chao Tung, and Henry H Liu, _The empirical mode decomposition and the hilbert spectrum for nonlinear and non-stationary time series analysis_ , Proceedings of the Royal Society of London. Series A: mathematical, physical and engineering sciences 454 (1998), no. 1971, 903–995.
  * [23] Qiao Li, Chengyu Liu, Julien Oster, and Gari D Clifford, _Signal processing and feature selection preprocessing for classification in noisy healthcare data_ , Machine Learning for Healthcare Technologies 2 (2016), no. 33, 2016.
  * [24] V. Mondéjar-Guerra, J. Novo, J. Rouco, M.G. Penedo, and M. Ortega, _Heartbeat classification fusing temporal and morphological information of ecgs via ensemble of classifiers_ , Biomedical Signal Processing and Control 47 (2019), 41–48.
  * [25] Larissa Montenegro, Mariana Abreu, Ana Fred, and Jose M Machado, _Human-assisted vs. deep learning feature extraction: An evaluation of ecg features extraction methods for arrhythmia classification using machine learning_ , Applied Sciences 12 (2022), no. 15, 7404.
  * [26] Laurent Mourot, Malika Bouhaddi, Stéphane Perrey, Sylvie Cappelle, Marie-Thérèse Henriet, Jean-Pierre Wolf, Jean-Denis Rouillon, and Jacques Regnard, _Decrease in heart rate variability with overtraining: assessment by the poincare plot analysis_ , Clinical physiology and functional imaging 24 (2004), no. 1, 10–18.
  * [27] Jiapu Pan and Willis J Tompkins, _A real-time qrs detection algorithm_ , IEEE transactions on biomedical engineering (1985), no. 3, 230–236.
  * [28] Vinícius Queiroz, Eduardo Luz, Gladston Moreira, Álvaro Guarda, and David Menotti, _Automatic cardiac arrhythmia detection and classification using vectorcardiograms and complex networks_ , 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2015, pp. 5203–5206.
  * [29] Pranav Rajpurkar, Awni Y Hannun, Masoumeh Haghpanahi, Codie Bourn, and Andrew Y Ng, _Cardiologist-level arrhythmia detection with convolutional neural networks_ , arXiv preprint arXiv:1707.01836 (2017).
  * [30] Antônio H Ribeiro, Manoel Horta Ribeiro, Gabriela MM Paixão, Derick M Oliveira, Paulo R Gomes, Jéssica A Canazart, Milton PS Ferreira, Carl R Andersson, Peter W Macfarlane, Wagner Meira Jr, et al., _Automatic diagnosis of the 12-lead ecg using a deep neural network_ , Nature communications 11 (2020), no. 1, 1–9.
  * [31] R Sameni, _The open-source electrophysiological toolbox (oset), 2010_ , URL http://www. oset. ir.
  * [32] R. Sameni, M.B. Shamsollahi, C. Jutten, and M. Babaie-Zade, _Filtering noisy ecg signals using the extended kalman filter based on a modified dynamic ecg model_ , Computers in Cardiology, 2005, 2005, pp. 1017–1020.
  * [33] Udit Satija, Barathram Ramkumar, and M Sabarimalai Manikandan, _Automated ecg noise detection and classification system for unsupervised healthcare monitoring_ , IEEE Journal of biomedical and health informatics 22 (2017), no. 3, 722–732.
  * [34] Ikaro Silva and George B Moody, _An open-source toolbox for analysing and processing physionet databases in matlab and octave_ , Journal of open research software 2 (2014), no. 1.
  * [35] Tibor Stracina, Marina Ronzhina, Richard Redina, and Marie Novakova, _Golden standard or obsolete method? review of ecg applications in clinical and experimental context_ , Frontiers in Physiology (2022), 613.
  * [36] María E Torres, Marcelo A Colominas, Gaston Schlotthauer, and Patrick Flandrin, _A complete ensemble empirical mode decomposition with adaptive noise_ , 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, 2011, pp. 4144–4147.
  * [37] Laurens Van der Maaten and Geoffrey Hinton, _Visualizing data using t-sne._ , Journal of machine learning research 9 (2008), no. 11.
  * [38] Fons J Wesselius, Mathijs S van Schie, Natasja MS De Groot, and Richard C Hendriks, _Digital biomarkers and algorithms for detection of atrial fibrillation using surface electrocardiograms: A systematic review_ , Computers in Biology and Medicine 133 (2021), 104404.
  * [39] Svante Wold, Kim Esbensen, and Paul Geladi, _Principal component analysis_ , Chemometrics and intelligent laboratory systems 2 (1987), no. 1-3, 37–52.
  * [40] Hua Yu and Jie Yang, _A direct lda algorithm for high-dimensional data — with application to face recognition_ , Pattern Recognition 34 (2001), no. 10, 2067–2070.
  * [41] Martin Zihlmann, Dmytro Perekrestenko, and Michael Tschannen, _Convolutional recurrent neural networks for electrocardiogram classification_ , 2017 Computing in Cardiology (CinC), IEEE, 2017, pp. 1–4.
  * [42] W Zong, T Heldt, GB Moody, and RG Mark, _An open-source algorithm to detect onset of arterial blood pressure pulses_ , Computers in Cardiology, 2003, IEEE, 2003, pp. 259–262.