# Harnessing FPGA Technology for Enhanced Biomedical Computation
††thanks: *Corresponding author

Nisanur Alici  Biomedical Eng.
Erciyes University
Kayseri, Turkey
<EMAIL_ADDRESS>Kayode Inadagbo  Electrical and Computer Eng.
Prairie View A&M University
Prairie View, TX
<EMAIL_ADDRESS>Murat Isik  Electrical and Computer Eng.
Drexel University
Philadelphia, PA
<EMAIL_ADDRESS>
###### Abstract

This research delves into sophisticated neural network frameworks like
Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long
Short-Term Memory Networks (LSTMs), and Deep Belief Networks (DBNs) for
improved analysis of ECG signals via Field Programmable Gate Arrays (FPGAs).
The MIT-BIH Arrhythmia Database serves as the foundation for training and
evaluating our models, with added Gaussian noise to heighten the algorithms’
resilience. The developed architectures incorporate various layers for
specific processing and categorization functions, employing strategies such as
the EarlyStopping callback and Dropout layer to prevent overfitting.
Additionally, this paper details the creation of a tailored Tensor Compute
Unit (TCU) accelerator for the PYNQ Z1 platform. It provides a thorough
methodology for implementing FPGA-based machine learning, encompassing the
configuration of the Tensil toolchain in Docker, selection of architectures,
PS-PL configuration, and the compilation and deployment of models. By
evaluating performance indicators like latency and throughput, we showcase the
efficacy of FPGAs in advanced biomedical computing. This study ultimately
serves as a comprehensive guide to optimizing neural network operations on
FPGAs across various fields.

###### Index Terms:

FPGA Deployment, ECG Signal Processing, Biomedical Computing, Hardware
Optimization.

##  I Introduction

High-performance computing systems are essential for real-time and accurate
biomedical applications, such as ECG signal analysis. Field-Programmable Gate
Arrays (FPGAs), known for their unique combination of flexibility,
performance, and energy efficiency, are increasingly favored for accelerating
computations across various domains, including biomedical engineering. This
paper focuses on efficiently implementing Convolutional Neural Networks
(CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks
(LSTMs), and Deep Belief Networks (DBNs) on FPGAs for tasks like arrhythmia
detection, heartbeat classification, and risk stratification in
electrocardiogram (ECG) signal analysis.

FPGA-based accelerators offer several advantages in biomedical applications.
They provide high parallelism and are crucial for large-scale data processing
due to their ability to handle multiple tasks simultaneously. Their
customizable architecture enables performance optimization for specific
applications, a feat unachievable with general-purpose processors. Another
significant advantage of FPGAs is their low latency and high bandwidth, making
them ideal for real-time processing and large data transfers. Additionally,
their energy efficiency renders them ideal for power-intensive applications
like image processing, machine learning, and real-time processing [14, 11].

As the demand for biomedical applications grows, the need for high-performance
computing systems capable of processing large data volumes quickly and
accurately becomes increasingly crucial. FPGA-based accelerators, with their
capacity to meet these demands effortlessly, pave the way for significant
advancements in biomedical engineering, especially with the efficient
implementation of CNNs, RNNs, LSTMs, and DBNs for ECG signal analysis.

The rest of the paper is organized as follows: Section II presents the
motivation behind our work and a review of related studies in the field.
Section III introduces open-source machine learning (ML) inference
accelerators, with a focus on the Tensil AI accelerator. In Section IV, we
describe our proposed method for efficient FPGA implementation of CNNs, RNNs,
LSTMs, and DBNs for ECG signal analysis. Section V presents the experimental
results and provides an analysis of the performance and efficiency of our
proposed method. Finally, Section VI concludes the paper, summarizing our
contributions and discussing future directions for research in this area.

##  II Motivation

The relentless pursuit of computational efficiency in high-performance
computing (HPC) has propelled industries toward rapid innovation and enhanced
problem-solving capabilities. Among the myriad technologies that have emerged,
FPGAs stand out as a transformative force in this evolution, especially in
applications that demand real-time processing and analysis. The motivation
behind integrating FPGAs into HPC systems stems from their unique ability to
be reconfigured for specific tasks, offering unparalleled flexibility and
performance benefits compared to traditional CPUs and GPUs [8, 12].

In the biomedical field, the impact of this technology is particularly
palpable. FPGAs can drastically improve the processing of ECG signals, which
are crucial for monitoring and diagnosing cardiac conditions. The parallel
processing capabilities of FPGAs, coupled with their reconfigurable nature,
allow for the design of optimized systems that process ECG signals much more
rapidly than conventional computing systems. This rapid analysis is critical
in medical emergencies, such as in the detection of life-threatening
conditions like arrhythmia or cardiac ischemia [6].

Combining FPGAs with deep learning techniques has advanced ECG signal analysis
substantially. Deep learning algorithms are particularly effective due to
their inherent ability to learn and adapt from data. For example,
Convolutional Neural Networks (CNNs) can automatically learn spatial
hierarchies of features, while Recurrent Neural Networks (RNNs) and Long
Short-Term Memory networks (LSTMs) excel in processing sequential data. FPGAs
facilitate the implementation of these advanced algorithms in a power and
cost-efficient manner [17]. CNNs have proven highly effective in arrhythmia
classification and detection, exploiting their ability to capture spatial
dependencies in ECG data. Studies have demonstrated the successful
implementation of CNNs on FPGAs, achieving high accuracy in real-time patient
monitoring systems. RNNs and LSTMs extend this capability by processing the
temporal characteristics of ECG signals, where the sequence and timing of
heartbeats provide insights into cardiac rhythm and identify abnormalities
[17] [19]. Furthermore, the integration of FPGAs in ECG signal analysis has
facilitated the development of modern implantable cardiac devices such as
pacemakers, where their high-performance and energy-efficient characteristics
are crucial [20].

Beyond ECG signal analysis, FPGAs find applications in various other
biomedical domains. In medical imaging, for example, FPGAs are crucial for
processing large datasets in real-time, as required in MRI, CT scans, and
ultrasound devices [18]. Minimally invasive surgery platforms benefit from
FPGAs’ computational power and real-time processing capabilities, critical for
precision and control in these procedures [24]. Digital image processing
techniques like inverting image operations, brightness control, segmentation,
and contrast stretching in biomedical applications are enhanced by FPGAs for
improved image quality and aid in diagnosis [18]. FPGAs also accelerate real-
time volume rendering for 3D medical images, using techniques such as block-
based ray casting to efficiently process and visualize complex volumetric data
[32].

Apart from the biomedical domain, FPGAs have applications in image and video
processing, where they handle high-resolution data in real-time. FPGA-based
image processing systems have achieved significant throughput, processing up
to 52 frames per second in filtering applications and 20 frames per second in
image segmentation tasks, showcasing FPGAs’ potential in multimedia
applications [1]. In the financial industry, FPGAs are used for complex
mathematical operations in high-frequency trading systems and financial
instrument calculations, such as options and futures, achieving processing
times of less than one microsecond [16, 15]. FPGAs have also emerged as a
vital component in the Internet of Things (IoT) domain, particularly in edge
computing for real-time data analysis and decision-making processes. Their
reconfigurability enables adaptation to the dynamic IoT landscape,
facilitating efficient data traffic management, robust security protocols, and
deployment of machine learning algorithms in IoT gateways [2].

Despite their challenges and limitations, such as limited on-chip memory,
floating-point support, and limited availability of prebuilt IP blocks, the
advantages of FPGAs have made them increasingly popular in high-performance
computing applications. Recent developments in high-level synthesis tools and
the availability of pre-built IP blocks have made FPGAs more accessible for
high-performance computing. As development tools, IP blocks, and FPGA-based
solutions evolve, the adoption of FPGAs in high-performance computing,
particularly in biomedical applications like ECG signal analysis, will
increase [10].

##  III Open-source ML inference accelerators

Machine learning (ML) inference plays a crucial role in numerous high-
performance computing applications. This process involves using models to
analyze input data and generate output results. High-performance computing
systems are instrumental in accelerating ML inference, a task that is often
computationally intensive. ML inference accelerators that are open-source may
be beneficial to high-performance computing applications. A machine learning
inference accelerator is a specialized hardware device designed to efficiently
execute ML inference tasks. Traditionally, ML inference tasks are performed on
general-purpose processors or graphics processing units (GPUs), which are not
specifically optimized for machine learning. A machine learning inference
accelerator streamlines and optimizes the execution of machine learning
inference tasks. A major advantage of open-source ML inference accelerators is
that they are free and can be customized for specific use cases. They offer a
transparent development process that encourages community participation.
Furthermore, open-source accelerators can reduce the cost and time associated
with the development of ML inference accelerators. The Versatile Tensor
Accelerator (VTA), an open-source ML inference accelerator, has recently
garnered significant attention. An optimized hardware accelerator is used for
performing inference tasks using VTA. VTA supports TensorFlow, PyTorch, and
ONNX among other ML frameworks. VTA is compatible with various hardware
platforms, including FPGAs and ASICs [23].

Figure 1: VTA Framework

[23]

The availability of open-source tools and platforms allows developers to
collaborate to develop better ML inference solutions. In addition to VTA [34]
[31], other open-source ML inference accelerators are available, such as
Intel’s OpenVINO and Xilinx’s Deep Learning Processor. These accelerators
provide a variety of options for the development and optimization of machine
learning systems. Open-source ML inference accelerators are flexible and
powerful tools that can be used in high-performance computing applications.
They enable the customized and optimized development of ML inference systems,
reducing development costs and time, fostering collaboration and innovation,
and leading to the wide adoption of ML techniques. Open-source ML inference
accelerators will continue to become more powerful and efficient as the field
of ML grows and evolves. Frameworks such as Nengo and Tensil AI enable the
development of high-performance computing applications utilizing FPGAs. Nengo
software can be used to build large-scale neural models on a variety of
hardware platforms, including FPGAs. Its flexibility and extensibility allow
users to create customized algorithms and models. Nengo is particularly suited
for applications in robotics and cognitive modeling due to its capability to
construct complex models with numerous neurons and synapses. The Tensil AI
hardware accelerator performs machine learning inference tasks. Tensil AI
offers high performance at low power consumption, making it ideal for
applications such as image recognition and natural language processing. Tensil
AI supports a variety of machine learning frameworks, including TensorFlow and
PyTorch, and can easily be integrated with existing hardware architectures.
One major difference between Tensil AI and Nengo is their focus. Tensil AI
aims to accelerate machine learning inference, while Nengo is geared towards
building large-scale neural networks [22] [7] [9] [13].

Nengo can be used for a wider range of tasks compared to the more specific
Tensil AI. The development processes of Nengo and Tensil AI also differ
fundamentally. Nengo is an open-source project actively developed and
maintained by a large developer community, providing a variety of resources
such as documentation, tutorials, and support forums. Tensil AI, on the other
hand, is a commercial product developed and supported by its company, offering
dedicated support and resources, but lacking the extensive community support
available for open-source software. Tensil AI efficiently performs inference
tasks in machine learning. For instance, it enables rapid inferences in self-
driving cars and industrial automation. Nengo, on the other hand, simulates
complex behaviors over long periods using large-scale neural models. Tensil
AI’s potential drawback is its limited flexibility due to the specialized
nature of its hardware accelerator. Users may not be able to customize models
and algorithms as extensively and may have to rely on prebuilt models and
architectures. Both Nengo and Tensil AI are powerful frameworks for developing
high-performance computing applications, with Tensil AI being more suited for
specific tasks like machine learning inference, while Nengo caters to a wider
range of applications. Developers should carefully evaluate the strengths and
weaknesses of each framework before deciding, as the ultimate choice depends
on their specific application needs [27] [26].

##  IV Method

Our framework assesses the performance and power profiles of ML algorithms
across three dimensions: i) applications, ii) algorithms, and iii) hardware
architectures, as depicted in 8. Within the application dimension, we
categorize tasks into computer vision, natural language processing, and time-
series data analysis. These applications are implemented using four distinct
ML algorithms: Long Short-Term Memory (LSTM), Convolutional Neural Networks
(CNN), Recurrent Neural Network (RNN), and Deep Belief Network (DBN), each
tested on different hardware platforms including CPU, GPU, and FPGA. We
classify energy estimation approaches into two types: top-down and bottom-up
techniques. The bottom-up method utilizes a detailed statistical model that
encompasses both workload analysis and measured power consumption, while the
top-down approach estimates power usage based on the observed performance
during algorithm simulations.

Figure 2: Biomedical data simulated with four machine learning algorithms
realized on three types of hardware architectures. Note: Long Short-Term
Memory (LSTM), Convolutional Neural Networks (CNN), Recurrent Neural Network
(RNN), and Deep Belief Network (DBN).

###  IV-A Dataset

The MIT-BIH Arrhythmia Database offers valuable cardiac arrhythmia data
through ECG recordings. Between 1975 and 1979, 48 half-hour excerpts of two-
channel ambulatory ECG recordings were obtained from 47 subjects at Boston’s
Beth Israel Hospital. These recordings were digitized at a rate of 360 samples
per second per channel, with 11-bit resolution over a 10 mV range. A total of
80% of the dataset is devoted to training, while 20% of the dataset is for
validation. The original training data includes approximately 110,000 beats,
each annotated by at least two cardiologists.

In contrast to the MNIST digit dataset, the MIT-BIH Arrhythmia Database does
not inherently contain noisy variations. However, for specific applications,
one can add noise to the ECG recordings. For example, to simulate real-world
noisy conditions in ECG data, random Gaussian noise can be applied to the
dataset elements. Similar to white noise, Gaussian noise follows a Gaussian
distribution. This addition results in impulses with random values.

Using the modified dataset, now containing Gaussian noise, deep-learning
models can be trained for ECG signal analysis, arrhythmia detection, and
algorithm robustness improvement in noisy environments. Our dataset is shown
in Fig. 3.

Figure 3: Project dataset

###  IV-B Training Details

Our approach utilizes a four-layer neural network incorporating LSTM for ECG
signal classification. The LSTM layers understand long-term data dependencies,
while dense layers assist in classification. To prevent overfitting, a dropout
layer randomly disables certain weights, and an EarlyStopping callback halts
training once the validation loss stabilizes. There are 64 and 32 LSTM cells
in the first and second LSTM layers, respectively. The dense layer carries out
classification, producing five outputs corresponding to five classes. During
compilation, a loss function and the ’man’ optimizer are utilized for
training. ’Accuracy’ measures performance, and an EarlyStopping callback
prevents overfitting. Training results are visualized with validation data,
and the model’s efficiency is evaluated using latency and throughput metrics.

The code comprises a neural network featuring several layers, including a
BernoulliRBM for high-level abstractions, LogisticRegression for
classification, and Dropout to prevent overfitting. The Flatten layer
transforms 3D input into 1D, the Reshape layer adjusts dimensions, and the
StandardScaler ensures data consistency. After data checks, the class
imbalance is tackled with SMOTE. Deep Belief Networks (DBNs) are used on
training data. The DBN model is evaluated using different sets. A CNN model
classifies heartbeats in this code. Following data preprocessing and renaming
of columns, the target variable distribution is plotted. Class imbalances are
identified and balanced via SMOTE. The CNN model consists of Conv1D,
MaxPool1D, and Dropout layers. Output is passed to a dense layer after
flattening. This code provides a solid foundation for heartbeat classification
using CNNs, but it can be further enhanced with detailed comments, dataset
information, preprocessing steps, and hyperparameter tuning. The final part
employs an RNN with a Bidirectional LSTM for ECG signal classification. The
MIT-BIH Arrhythmia dataset is processed and split into training and validation
sets. Model performance is assessed using latency, throughput, and other
metrics.

For ECG analysis, the ideal model is one that has low latency and high
throughput. These models can be compared based on their characteristics as
follows:

  * •

LSTM (Long Short-Term Memory): LSTM is a type of recurrent neural network
(RNN) architecture. It has gained significant prominence in the field of deep
learning and artificial intelligence due to its ability to effectively model
and capture long-range dependencies in sequential data. LSTM is a powerful
tool in applications such as natural language processing, speech recognition,
and time series forecasting.

Figure 4: LSTM Architecture

[21]

## LSTM Mathematical Model

LSTMs have three gates:

1\. Forget Gate ($f_{t}$): Determines what information from the previous cell
state should be discarded or retained.

$f_{t}=\sigma(W_{f}\cdot[h_{t-1},x_{t}]+b_{f})$ (1)

Where:

$\displaystyle f_{t}$ is the forget gate activation vector.
$\displaystyle\sigma$ represents the sigmoid activation function.
$\displaystyle W_{f}$ is the weight matrix for the forget gate. $\displaystyle
h_{t-1}$ is the previous cell state. $\displaystyle x_{t}$
$\displaystyle\text{ is the current input at time }t.$ $\displaystyle b_{f}$
is the bias for the forget gate.

2\. Input Gate ($i_{t}$): Determines which values from the input and the
previous cell state should be updated.

$i_{t}=\sigma(W_{i}\cdot[h_{t-1},x_{t}]+b_{i})$ (2)

Where:

$\displaystyle i_{t}$ is the input gate activation vector.
$\displaystyle\sigma$ represents the sigmoid activation function.
$\displaystyle W_{i}$ is the weight matrix for the input gate. $\displaystyle
h_{t-1}$ is the previous cell state. $\displaystyle x_{t}$
$\displaystyle\text{ is the current input at time }t.$ $\displaystyle b_{i}$
is the bias for the input gate.

3\. Cell State Update ($\tilde{C}_{t}$): Combines new information from the
current input and the previous cell state.

$\tilde{C}_{t}=\tanh(W_{C}\cdot[h_{t-1},x_{t}]+b_{C})$ (3)

Where:

$\displaystyle\tilde{C}_{t}$ is the candidate cell state. $\displaystyle\tanh$
represents the hyperbolic tangent activation function. $\displaystyle W_{C}$
is the weight matrix for the cell state update. $\displaystyle h_{t-1}$ is the
previous cell state. $\displaystyle x_{t}$ $\displaystyle\text{ is the current
input at time }t.$ $\displaystyle b_{C}$ is the bias for the cell state
update.

4\. Update and Output Gate ($o_{t}$): Determines the new cell state and the
output at the current time step.

$o_{t}=\sigma(W_{o}\cdot[h_{t-1},x_{t}]+b_{o})$ (4)

$C_{t}=f_{t}\cdot C_{t-1}+i_{t}\cdot\tilde{C}_{t}$ (5)

$h_{t}=o_{t}\cdot\tanh(C_{t})$ (6)

Where:

$\displaystyle o_{t}$ is the output gate activation vector. $\displaystyle
C_{t}$ is the new cell state. $\displaystyle h_{t}$ is the output vector for
the current time step. $\displaystyle C_{t-1}$ is the previous cell state.
$\displaystyle\sigma$ represents the sigmoid activation function.
$\displaystyle\tanh$ represents the hyperbolic tangent activation function.
$\displaystyle W_{o}$ is the weight matrix for the output gate. $\displaystyle
h_{t-1}$ is the previous cell state. $\displaystyle x_{t}$
$\displaystyle\text{ is the current input at time }t.$ $\displaystyle b_{o}$
is the bias for the output gate.

LSTMs are effective for modeling sequences and time-series data, capturing
long-range dependencies and storing information over extended sequences. They
are designed to address the limitations of traditional RNNs, such as the
vanishing and exploding gradient problems. ECG analysis can benefit from LSTMs
when dealing with sequential data that exhibit long-term patterns. However,
due to their complex structure, LSTMs may have higher latency and lower
throughput compared to CNNs.

  * •

CNN (Convolutional Neural Network): CNN is a deep learning model
revolutionizing the field of computer vision. It is specially designed for
processing and analyzing visual data, making it ideal for tasks such as image
recognition, object detection, and video analysis. CNNs consist of multiple
layers, each designed for a specific purpose:

1\. Convolution Layers: These layers learn features automatically from the
input data. They apply filters to detect various patterns, edges, and
textures, enabling CNNs to learn hierarchical representations of visual
features.

2\. Pooling Layers: These layers downsample feature maps from convolutional
layers, reducing spatial dimensions and making the model robust to scale and
position variations.

3\. Fully Connected Layers: These layers make predictions based on learned
features. They classify or regress based on the high-level features extracted
by previous layers.

Figure 5: CNN Architecture

[3]

## CNN Mathematical Model

1\. Convolution Operation (Convolutional Layer):

CNNs apply a set of learnable filters (kernels) to the input data through
convolution operations. These filters convolve across the input to produce
feature maps. The mathematical operation is represented as:

$S(i,j)=(I*K)(i,j)=\sum_{m}\sum_{n}I(i-m,j-n)\cdot K(m,n)$ (7)

Where:

$\displaystyle S(i,j)$ $\displaystyle\text{ represents the value at position
}(i,j)\text{ in the feature map.}$ $\displaystyle I$ denotes the input data.
$\displaystyle K$ is the convolutional kernel (filter). $\displaystyle(i,j)$
are the spatial coordinates of the output feature map. $\displaystyle(m,n)$
are the coordinates within the kernel.

2\. Pooling Layer:

Pooling layers, typically used after convolution, reduce the spatial
dimensions of feature maps. Max pooling, which selects the maximum value from
a local region of the input, is a common operation:

$O(i,j)=\max_{m,n}I(i\cdot s+m,j\cdot s+n)$ (8)

Where:

$\displaystyle O(i,j)$ $\displaystyle\text{ is the value at position
}(i,j)\text{ in the pooled feature map.}$ $\displaystyle I$ refers to the
input feature map. $\displaystyle s$ is the stride, determining the step size
for pooling.

3\. Fully Connected Layer:

CNNs typically include one or more fully connected layers after convolutional
and pooling layers to perform classification or regression. These layers
function like traditional feedforward neural network layers:

$y=\sigma(Wx+b)$ (9)

Where:

$\displaystyle y$ is the layer’s output. $\displaystyle\sigma$ denotes the
activation function (usually ReLU or sigmoid). $\displaystyle W$ is the weight
matrix. $\displaystyle x$ is the input vector. $\displaystyle b$ is the bias
vector.

Due to their ability to capture local patterns in data, CNNs excel in image
and signal processing tasks, offering lower latency and higher throughput than
LSTMs and RNNs for real-time applications.

  * •

RNN (Recurrent Neural Network): RNN, which stands for Recurrent Neural
Network, is a class of neural networks particularly well-suited for tasks that
involve sequential data. Unlike traditional feedforward neural networks, RNNs
have a unique ability to capture patterns and dependencies in data sequences,
making them indispensable in a wide range of applications, including natural
language processing, speech recognition, and time series analysis.

Figure 6: RNN Architecture

[25]

## RNN Mathematical Model

1\. Hidden State Update:

Each time step $t$ in an RNN updates a hidden state vector $h_{t}$, based on
the current input $x_{t}$ and the previous hidden state $h_{t-1}$:

$h_{t}=\sigma(W_{h}\cdot h_{t-1}+U_{x}\cdot x_{t}+b_{h})$ (10)

Where:

$\displaystyle h_{t}$ $\displaystyle\text{ is the hidden state vector at time
}t.$ $\displaystyle\sigma$ $\displaystyle\text{ typically represents
}\tanh\text{ or ReLU activation function.}$ $\displaystyle W_{h}$ is the
weight matrix for the hidden state. $\displaystyle h_{t-1}$ is the previous
time step’s hidden state. $\displaystyle U_{x}$ is the weight matrix for the
input. $\displaystyle x_{t}$ $\displaystyle\text{ is the input vector at time
}t.$ $\displaystyle b_{h}$ is the bias vector for the hidden state.

2\. Output:

The output $y_{t}$ at each time step is often based on the current hidden
state:

$y_{t}=V\cdot h_{t}+b_{y}$ (11)

Where:

$\displaystyle y_{t}$ $\displaystyle\text{ is the output at time }t.$
$\displaystyle V$ is the weight matrix for the output. $\displaystyle h_{t}$
is the current hidden state. $\displaystyle b_{y}$ is the bias vector for the
output.

RNNs are adept at modeling temporal dependencies in sequential data, making
them suitable for tasks like ECG signal analysis. However, they can be
impacted by vanishing and exploding gradient problems, affecting their
performance. RNNs generally have higher latency and throughput than CNNs.

  * •

DBN (Deep Belief Network): Deep Belief Networks (DBNs) are a class of
artificial neural networks that have garnered attention for their remarkable
capabilities in unsupervised learning and feature representation. Developed as
a generative model, DBNs are composed of multiple layers of restricted
Boltzmann machines (RBMs), and they have found applications in diverse domains
such as image recognition, recommendation systems, and dimensionality
reduction.

## DBN Mathematical Model

1\. Restricted Boltzmann Machine (RBM):

The fundamental building block of a DBN is an RBM, consisting of a visible
layer (input data) and a hidden layer. The RBM’s energy is defined as:

$\displaystyle
E(v,h)=-\sum_{i}\sum_{j}w_{ij}v_{i}h_{j}-\sum_{i}a_{i}v_{i}-\sum_{j}b_{j}h_{j}$
(12)

Where:

$\displaystyle E(v,h)$ $\displaystyle\text{ is the energy for visible vector
}v\text{ and hidden vector }h.$ $\displaystyle v_{i},h_{j}$ are the binary
states of visible and hidden units, respectively. $\displaystyle w_{ij}$
$\displaystyle\text{ is the weight between visible unit }v_{i}\text{ and
hidden unit }h_{j}.$ $\displaystyle a_{i},b_{j}$ are biases for visible and
hidden units, respectively.

2\. Joint Probability Distribution:

The joint probability distribution is defined using the energy function:

$P(v,h)=\frac{e^{-E(v,h)}}{Z}$ (13)

Where:

$\displaystyle P(v,h)$ is the joint probability distribution. $\displaystyle
Z$ is the partition function, normalizing the distribution.

3\. Training:

DBN training involves: \- Pretraining: Each RBM is trained layer-wise using
unsupervised learning (e.g., Contrastive Divergence). \- Fine-tuning: The
entire network is fine-tuned using supervised methods like backpropagation.

Figure 7: DBN Architecture

[4]

4\. Stacking RBMs:

Trained RBMs are stacked to form a DBN, capturing increasingly abstract data
features.

DBNs, with multiple layers of RBMs or autoencoders, excel in feature
extraction and classification, but may not be as effective as LSTMs or CNNs
for sequential data like ECG signals. Their performance varies based on
architecture.

Based on the above comparison, CNNs are generally considered the best choice
for ECG analysis when low latency and high throughput are essential, as shown
in Table II. However, the specific choice of model depends on the requirements
of the application and the nature of the ECG data. Experimenting with
different models can be useful to determine which one performs best for a
specific use case.

TABLE I: Resource utilization summary Zynq®-7000 SoC
---
Resource | Utilization | Available | % Utilization
LUT | 17579 | 74000 | 23.75%
FF | 20060 | 106400 | 18.85%
BRAM | 1374 | 3300 | 41.64%
IO | 36 | 150 | 24%
DSP | 85 | 160 | 53.13%
TABLE II: Comparison of LSTM, CNN, RNN, and DBN models based on various parameters Models | Accuracy | Precision | Recall | F1-score | Training time | Model complexity (params) | Throughput [GOP/s] | Latency
---|---|---|---|---|---|---|---|---
LSTM | 81% | 28% | 18% | 16% | 202.43 s | 29,477 | 2439.44 | 37 ms
CNN | 99% | 20% | 4% | 8% | 622.63 s | 3,245,637 | 2039.21 | 14 ms
RNN | 69% | 18% | 20% | 19% | 7.20 s | 276,737 | 378.10 | 43 ms
DBN | 19.43% | 39% | 20% | 66% | 304.18 s | 89,345 | 2237.40 | 30 ms

####  IV-B1 CPU/GPU Implementation

We implemented our model on both CPU and GPU, utilizing Python for execution.
The computational power of NVIDIA’s GeForce RTX 3060 GPU and Intel’s Core i9
12900H CPU was leveraged, each optimized for different tasks, ensuring
efficient execution.

####  IV-B2 Hardware Accelerator MAC Components

The MAC operations for the primary components of the CNN architecture are
quantified as follows:

  1. 1.

Convolutional Layers: These layers involve convolution operations, which are
essentially matrix multiplications between the input data and the
convolutional filters or kernels. Let $F$ be the number of filters, $D$ the
dimension of each filter, and $I$ the size of the input feature map.

$\text{MAC}_{\text{Conv}}=F\times D^{2}\times I^{2}$ (14)

  2. 2.

Pooling Layers: Pooling (e.g., max pooling) reduces the spatial dimensions of
the feature maps. The MAC operations for pooling are typically less
computationally intensive compared to convolutional layers. If $P$ represents
the size of the pooling window:

$\text{MAC}_{\text{Pool}}=I^{2}/P^{2}$ (15)

  3. 3.

Fully Connected Layers: These layers are similar to traditional neural network
layers and involve matrix multiplications between the flattened feature map
and the layer’s weights. Let $C$ be the number of connections in a fully
connected layer.

$\text{MAC}_{\text{FC}}=C$ (16)

The total MAC operations for a CNN architecture are the sum of the MAC
operations from its convolutional, pooling, and fully connected layers:

$\text{MAC}_{\text{Total}}=\text{MAC}_{\text{Conv}}+\text{MAC}_{\text{Pool}}+\text{MAC}_{\text{FC}}$
(17)

####  IV-B3 Throughput Analysis

Throughput is computed using MAC counts and latency:

$\text{Simulation Time}=\frac{\text{Total Inference Time}}{\text{Total number
of inference samples}}$ (18)
$\text{Throughput}=\frac{\\#\text{MACs}}{\text{Latency (Simulation Time)}}$
(19)

###  IV-C FPGA Implementation with Tensil’s Open Source Inference Accelerator

We provide a guide on implementing a ResNet-20 convolution model, trained on
the MIT-BIH Arrhythmia Database, on a PYNQ Z1 FPGA using Tensil’s open-source
inference accelerator (Fig. 8). Resource utilization on a Zynq®-7000 SoC is
detailed in Table I.

Figure 8: Block Diagram of Implementation

This section presents a detailed guide on implementing a ResNet-20 convolution
model, trained not on CIFAR but the MIT-BIH Arrhythmia Database, on a PYNQ Z1
FPGA using Tensil’s open-source inference accelerator, as depicted in Fig. 8.
Table I provided indicates the resource utilization on a Zynq®-7000 SoC when
running the implemented model. It uses 23.75% of available Lookup Tables
(LUT), 18.85% of Flip-Flops (FF), and 41.64% of Block RAM (BRAM). The model
also employs 24% of available Input/Output (IO) resources and 53.13% of
Digital Signal Processors (DSP). This summary highlights the efficient use of
the Zynq®-7000 SoC’s resources by the model, ensuring it operates effectively
within the constraints of the system.

####  IV-C1 Installation and Setup of the Tensil Toolchain

The Tensil toolchain is a suite of tools designed to facilitate FPGA
development, specifically for running machine learning models on FPGA. Docker
is used to host the Tensil toolchain, allowing for simple installation and
setup. For those unfamiliar with Docker, it’s a platform that enables
developers to package and distribute their applications in a manner that is
platform-independent. It’s essential to ensure Docker is installed before
proceeding with the Tensil toolchain setup.

####  IV-C2 Selection of Architecture

The choice of architecture depends on the specific requirements of your
machine learning model and the resources available on your FPGA. It’s crucial
to consider factors such as the complexity of your model, the amount of data
you’ll be processing, and the computational resources of your FPGA.

####  IV-C3 TCU Accelerator Design and Synthesis for PYNQ Z1

In this section, a custom TCU (Tensor Compute Unit) accelerator design is
generated and synthesized for the PYNQ Z1 board using the Tensil toolchain.
The goal here is to create a hardware design that can efficiently execute the
machine learning model on the FPGA.

####  IV-C4 PS-PL Configuration and Smart Interconnect

The next steps involve setting up the configuration between the Processing
System (PS) and Programmable Logic (PL) and establishing a smart interconnect.
These steps ensure efficient data transfer and proper interfacing between the
various components of the design.

####  IV-C5 Compilation of the ML Model

To execute the machine learning model on the FPGA, we compile the model into a
”.tmodel” file. The ”.tmodel” file contains the model’s structure and
parameters in a format that can be executed on the FPGA. Additional files
”.tprog” and ”.tdata” are also generated, providing instructions and data for
the FPGA execution.

####  IV-C6 Execution Using PYNQ

In this stage, all the previously generated and compiled files are deployed
onto the PYNQ environment for execution. A PYNQ environment refers to a
Python-based ecosystem that simplifies the usage and programming of Xilinx
Zynq SoCs. It’s important to set up this environment correctly on the FPGA for
successful execution. An FPGA image is essentially a binary file that contains
the configuration data for the FPGA. This image is crucial for defining the
functionality of the FPGA hardware during runtime.

##  V Results

Figure 9: Our Framework on Different Heterogeneous Devices.

The chart illustrates that CPUs, with their general-purpose design, offer
flexibility but often at the cost of power efficiency. GPUs, with their many-
core architecture, provide higher throughput for parallel computing tasks but
lag in power efficiency compared to FPGAs. FPGAs, with their reconfigurable
nature, achieve superior power efficiency and throughput for specific
applications but require expertise in digital design.

TABLE III: Comparisons with previous implementations.

| [33] | [30] | [28] | [29] | [5] | Our
---|---|---|---|---|---|---
Convolution Type | 1-D | 1-D | 1-D | 2-D | 2-D | 2-D
Platform | FPGA Pynq-Z2 | CPU-i7 | - | iCE40UP5k | GPU RTX 2080 Ti | FPGA Pynq-Z1
No. Input Samples | 512 | 200 | 400 | 10x10 | 64x64 | 187
Activation | - | ReLu | - | bTanH | ReLu | ReLu
Num of MACs | 929,650 | 1,289,312 | 749,620 | 27,153 | 58.1 M | 47,560
Clock | 25 MHz | 3.7 GHz | - | 100 MHz | 1350 MHz | 100 MHz
Accuracy | 98.9 | 99.8 | 98.4 | 96.8 | 99.7 | 99.1
Power | 13.34 $\mu$W | 84 W | 141 mW | 227.3 $\mu$W | 108 W | 1.53 W

##  VI Conclusions

Our research demonstrates the effectiveness of Tensil AI’s open-source
inference accelerator in optimizing neural networks and implementing them on
FPGAs for high-performance computing applications. We achieved significant
results across CPUs, GPUs, and FPGAs, contributing to the growing body of work
on FPGA-based NN inference systems. Our approach, leveraging 2-D convolution
on a PYNQ Z1 FPGA platform with a ReLu activation function, processed 187
input samples with a high accuracy of 99.1% and consumed only 1.53 W of power.
These findings underline the potential of FPGAs in high-throughput, power-
efficient computing applications. We plan to incorporate Dynamic Partial
Reconfiguration (DPR) into our system to improve performance further and
explore Tensil AI’s flexibility in handling diverse machine learning models.

##  VII Future Research Directions

The findings of our current study lay the groundwork for several promising
research trajectories:

###  VII-A Integration of Dynamic Partial Reconfiguration (DPR)

Future research will explore the integration of DPR into FPGA systems. This
advancement aims to enhance the flexibility and performance of FPGAs by
enabling on-the-fly reconfiguration capabilities. This aspect of research will
investigate the potential improvements in computational efficiency and
adaptability to varying workloads.

###  VII-B Optimization of Data Processing Stages

Optimizing the initial and post-data processing stages is another research
avenue. Utilizing tools like Tensil AI, this endeavor will focus on refining
the computational pipeline to enhance efficiency and reduce processing
overheads, thereby streamlining the entire data flow from input to output.

###  VII-C Extending Model Compatibility with Tensil AI

Our research will also extend the scope of Tensil AI’s compatibility with a
wider array of machine learning models. This will involve comprehensive
testing and development to showcase the platform’s versatility across
different machine learning paradigms and its applicability to diverse
computational tasks.

###  VII-D Benchmarking Against Emerging Hardware Technologies

An important future direction is benchmarking FPGA performance against
emerging hardware technologies. This comparative analysis will provide
insights into the relative strengths and limitations of FPGAs in the context
of advancing computational technologies.

###  VII-E Studies on Energy Consumption and Sustainability

In response to the growing emphasis on sustainable technology practices, in-
depth studies focusing on the energy consumption patterns of FPGA
implementations are planned. These studies will seek to develop strategies for
energy-efficient operations and sustainable system designs.

###  VII-F Real-world Deployment and Clinical Trial Collaborations

Collaborating with medical institutions for the real-world deployment and
clinical trials of FPGA-based systems, particularly focusing on real-time ECG
signal analysis, will be a significant step towards practical application.
This collaboration aims to validate the efficacy of our implementations in
clinical settings.

###  VII-G Enhancing the Usability of FPGA Development Toolchain

Recognizing the complexity involved in FPGA development, future work will also
involve efforts to enhance the usability of the FPGA development toolchain.
This initiative aims to make FPGA technology more accessible to a broader
range of practitioners, including those without extensive expertise in
hardware design.

###  VII-H Cross-Domain Application Exploration

Exploring the adaptability of FPGA-based systems to other domains, such as
autonomous systems and IoT devices, represents an exciting frontier. This will
involve adapting the existing framework to meet the specific computational
demands of different applications.

###  VII-I Co-Development Practices for Hardware and Software

Investigating co-development practices that concurrently enhance both the
hardware layout and the software framework is a key research area. This
integrated approach aims to synergize hardware and software development,
maximizing the performance capabilities of FPGAs.

###  VII-J Development of Advanced Compiler Strategies

Lastly, the development of advanced compiler strategies to optimize the
performance of neural networks on FPGAs will be a crucial focus. This will
include exploring innovative forms of parallelism and custom instruction sets
specifically tailored to machine learning tasks, potentially leading to
significant advancements in FPGA-based machine learning implementations.

## References

  * [1] Abdullah Al-Dujaili and Suhaib A. Fahmy  “High Throughput 2D Spatial Image Filters on FPGAs”  In _CoRR_ abs/1710.05154, 2017  arXiv: http://arxiv.org/abs/1710.05154
  * [2] Saman Biookaghazadeh, Ming Zhao and Fengbo Ren  “Are FPGAs Suitable for Edge Computing?”  In _USENIX Workshop on Hot Topics in Edge Computing (HotEdge 18)_ Boston, MA: USENIX Association, 2018  URL: https://www.usenix.org/conference/hotedge18/presentation/biookaghazadeh
  * [3] “Convolution Neural Networks Website” Accessed: 2022-05-08, https://www.baeldung.com/cs/deep-belief-network
  * [4] “Deep Belief Neural Networks Website” Accessed: 2022-11-13, https://www.baeldung.com/cs/deep-belief-network
  * [5] Murside Degirmenci, Mehmet Akif Ozdemir, Elif Izci and Aydin Akan  “Arrhythmic heartbeat classification using 2d convolutional neural networks”  In _Irbm_ 43.5 Elsevier, 2022, pp. 422–433
  * [6] Madhav P. Desai et al.  “A Low-Latency, Low-Power FPGA Implementation of ECG Signal Characterization Using Hermite Polynomials”  In _Electronics_ 10.19, 2021
  * [7] Travis DeWolf, Pawel Jaworski and Chris Eliasmith  “Nengo and low-power AI hardware for robust, embedded neurorobotics”  In _Frontiers in Neurorobotics_ 14 Frontiers Media SA, 2020, pp. 568359
  * [8] Mohamed G. Egila et al.  “FPGA-based electrocardiography (ECG) signal analysis system using least-square linear phase finite impulse response (FIR) filter”  In _Journal of Electrical Systems and Information Technology_ 3.3, 2016, pp. 513–526
  * [9] Jan Gosmann and Chris Eliasmith  “Automatic optimization of the computation graph in the Nengo neural network simulator”  In _Frontiers in Neuroinformatics_ 11 Frontiers Media SA, 2017, pp. 33
  * [10] Sitao Huang et al.  “Accelerating sparse deep neural networks on FPGAs”  In _2019 IEEE High Performance Extreme Computing Conference (HPEC)_ , 2019, pp. 1–7  IEEE
  * [11] Kayode Inadagbo, Baran Arig, Nisanur Alici and Murat Isik  “Exploiting FPGA Capabilities for Accelerated Biomedical Computing”  In _2023 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA)_ , 2023, pp. 48–53  DOI: 10.23919/SPA59660.2023.10274450
  * [12] Murat Isik and Kayode Inadagbo  “Astrocyte-Integrated Dynamic Function Exchange in Spiking Neural Networks”  In _arXiv preprint arXiv:2309.08232_ , 2023
  * [13] Murat Isik, Kayode Inadagbo and Hakan Aktas  “Design optimization for high-performance computing using FPGA”  In _arXiv preprint arXiv:2304.12474_ , 2023
  * [14] Murat Isik, Matthew Oldland and Lifeng Zhou  “An Energy-Efficient Reconfigurable Autoencoder Implementation on FPGA”  In _arXiv preprint arXiv:2301.07050_ , 2023
  * [15] Shigeki Kohda and Kenichi Yoshida  “Characteristics and Forecast of High-frequency Trading”  In _Transactions of the Japanese Society for Artificial Intelligence_ 37.5 The Japanese Society for Artificial Intelligence, 2022, pp. B–M44_1
  * [16] Shigeki Kohda and Kenichi Yoshida  “Characteristics of High-Frequency Trading and Its Forecasts”  In _2021 IEEE 45th Annual Computers, Software, and Applications Conference (COMPSAC)_ , 2021, pp. 1496–1501  IEEE
  * [17] Ashish Kumar, Manjeet Kumar and Rama S. Komaragiri  “FPGA Implementation of Combined ECG Signal Denoising, Peak Detection Technique for Cardiac Pacemaker Systems”  In _High Performance and Power Efficient Electrocardiogram Detectors_ Singapore: Springer Nature Singapore, 2023, pp. 111–129
  * [18] Angelos Kyriakos, Elissaios-Alexios Papatheofanous, Charalampos Bezaitis and Dionysios Reisis  “Resources and Power Efficient FPGA Accelerators for Real-Time Image Classification”  In _Journal of Imaging_ 8.4, 2022  URL: https://www.mdpi.com/2313-433X/8/4/114
  * [19] Wenhan Liu et al.  “A fully-mapped and energy-efficient FPGA accelerator for dual-function AI-based analysis of ECG”  In _Front Physiol_ 14, 2023, pp. 1079503
  * [20] Wenhan Liu et al.  “A fully-mapped and energy-efficient FPGA accelerator for dual-function AI-based analysis of ECG”  In _Front Physiol_ 14, 2023, pp. 1079503
  * [21] “Long Short-Term Memory Website” Accessed: 2022-02-21, https://medium.com/@ottaviocalzone/an-intuitive-explanation-of-lstm-a035eb6ab42c
  * [22] Benjamin Morcos  “Nengofpga: an fpga backend for the nengo neural simulator”, 2019
  * [23] Thierry Moreau et al.  “A hardware–software blueprint for flexible deep learning specialization”  In _IEEE Micro_ 39.5 IEEE, 2019, pp. 8–16
  * [24] Renáta Nagyné Elek and Tamás Haidegger  “Next in Surgical Data Science: Autonomous Non-Technical Skill Assessment in Minimally Invasive Surgery Training”  In _J Clin Med_ 11.24, 2022
  * [25] “Recurrent Neural Networks Website” Accessed: 2017-09-04, https://medium.com/lingvo-masino/introduction-to-recurrent-neural-network-d77a3fe2c56c
  * [26] “Tensil AI Github” Accessed: 2022-12-17, https://github.com/tensil-ai
  * [27] “Tensil AI Website” Accessed: 2022-12-17, https://www.tensil.ai/
  * [28] Ning Wang et al.  “Energy-efficient intelligent ECG monitoring for wearable devices”  In _IEEE transactions on biomedical circuits and systems_ 13.5 IEEE, 2019, pp. 1112–1121
  * [29] David Liang Tai Wong et al.  “An energy efficient ECG ventricular ectopic beat classifier using binarized CNN for edge AI devices”  In _IEEE Transactions on Biomedical Circuits and Systems_ 16.2 IEEE, 2022, pp. 222–232
  * [30] Yufa Xia and Yaoqin Xie  “A novel wearable electrocardiogram classification system using convolutional neural networks and active learning”  In _Ieee Access_ 7 IEEE, 2019, pp. 7989–8001
  * [31] “Xilinx’s DPU Website” Accessed: 2022-12-17, https://docs.xilinx.com/r/en-US/ug1414-vitis-ai/Deep-Learning-Processor-Unit
  * [32] Jianguo Xing, Shufang Liu and Wenmin Zhao  “FPGA-accelerated real-time volume rendering for 3D medical image”  In _2010 3rd International Conference on Biomedical Engineering and Informatics_ 1, 2010, pp. 273–276  DOI: 10.1109/BMEI.2010.5639475
  * [33] Yang Zhao, Zhongxia Shang and Yong Lian  “A 13.34 $\mu$W event-driven patient-specific ANN cardiac arrhythmia classifier for wearable ECG sensors”  In _IEEE transactions on biomedical circuits and systems_ 14.2 IEEE, 2019, pp. 186–197
  * [34] VV Zunin  “Intel OpenVINO Toolkit for Computer Vision: Object Detection and Semantic Segmentation”  In _2021 International Russian Automation Conference (RusAutoCon)_ , 2021, pp. 847–851  IEEE