11institutetext: Intelligent Embedded Systems Lab,
University of Duisburg-Essen, Germany
11email: {chao.qian, christopher.cichiwskyj,
tianheng.ling<EMAIL_ADDRESS>
# Idle is the New Sleep: Configuration-Aware Alternative to Powering Off FPGA-
Based DL Accelerators During Inactivity

Chao Qian  Christopher Cichiwskyj  Tianheng Ling  Gregor Schiele

###### Abstract

In the rapidly evolving Internet of Things (IoT) domain, we concentrate on
enhancing energy efficiency in Deep Learning accelerators on FPGA-based
heterogeneous platforms, aligning with the principles of sustainable
computing. Instead of focusing on the inference phase, we introduce innovative
optimizations to minimize the overhead of the FPGA configuration phase. By
fine-tuning configuration parameters correctly, we achieved a 40.13-fold
reduction in configuration energy. Moreover, augmented with power-saving
methods, our Idle-Waiting strategy outperformed the traditional On-Off
strategy in duty-cycle mode for request periods up to 499.06 ms. Specifically,
at a 40 ms request period within a 4147 J energy budget, this strategy extends
the system lifetime to approximately 12.39$\times$ that of the On-Off
strategy. Empirically validated through hardware measurements and simulations,
these optimizations provide valuable insights and practical methods for
achieving energy-efficient and sustainable deployments in IoT.

###### Keywords:

FPGA Deep Learning Configuration Optimization Sustainable Computing

##  1 Introduction

Embedded Deep Learning (DL) has recently made significant progress in the
Internet of Things (IoT) domain [1]. Nevertheless, IoT devices, limited by the
low-power Microcontroller Units (MCUs), often struggle with performance
constraints [14]. Combining Field-Programmable Gate Arrays (FPGAs) with MCUs
to create a heterogeneous computing platform has proven to be a promising
approach to adapt to these limitations, balancing between computational power
and energy efficiency [8]. However, optimizing task offloading from MCUs to
FPGAs remains crucial to meet the stringent energy budget of IoT devices and
advance sustainable IoT ecosystems.

Previous studies typically assumed that there is continuous data or work
supply for FPGAs [7, 11, 12], justifying their emphasis on energy efficiency
during the inference phase of FPGA-based DL accelerators. However, in common
IoT applications like time series analysis, FPGAs often complete inferences
much faster than sensor data can be gathered, resulting in a data supply gap.

In this context, the FPGA can operate in duty-cycle mode [4], as shown in
Figure 2. This mode involves the MCU gathering sufficient data before
initiating an inference request (indicated by pink arrows in Figure 2) to
offload tasks to the FPGA. We term the interval between these requests as the
request period ($T_{\text{req}}$), which remains constant in our study that
focuses on periodic inference requests. Further, each sequence of operations
performed by the FPGA in response to an inference request is defined as a
workload item (depicted as a gray box in Figure 2), with its processing time
labeled as $T_{\text{latency}}$. In this study, we explore cases where
$T_{\text{latency}}<$$T_{\text{req}}$, allowing the FPGA to be powered off
after finishing a workload item, thereby conserving energy. The duration of
this power-off state is denoted as $T_{\text{off}}$.

Figure 1: Workloads in Duty-Cycle Mode

Figure 2: Energy of a Workload Item

Each workload item for the DL accelerator involves multiple phases:
configuration, data transmission (including loading and offloading), and
inference. Our prior research [5] reveals that the configuration phase
accounts for a substantial 87.15% of the total energy consumption per workload
item, as shown in Figure 2. This underscores the importance of optimizing this
phase to enhance overall energy efficiency. In response, we introduce an
incremental approach to reduce the overhead of FPGA configuration. This
approach has been validated through measurements on an FPGA hardware platform
and is further supported by simulations for scenarios that could not be
achieved by the current design of the hardware platform. The primary
contributions of our research are as follows:

  * •

Utilizing a representative embedded _Spartan-7 XC7S15_ FPGA, we deliver
valuable insights for setting FPGA configuration parameters correctly,
reducing the energy consumption by 40.13-fold per FPGA configuration,
effectively lowering it to a mere 11.85 mJ.

  * •

We introduce the Idle-Waiting strategy as a more efficient alternative for
shorter request periods. In the case of a Long Short-Term Memory (LSTM)
accelerator, we demonstrate that for request periods below 89.21 ms within an
energy budget of 4147 J, this strategy consistently surpasses the traditional
On-Off strategy. At a 40 ms request period, this strategy yields 2.23$\times$
more workload items and a comparable increase in system lifetime.

  * •

By further integrating two power-saving methods, we reduce FPGA idle power by
81.98%. This results in a 5.57-fold increase in the number of executable
workload items, and the estimated system lifetime can be improved to an
average of 47.80 hours within the same energy budget. This optimization also
expands the advantageous request period to 499.06 ms.

The rest of the content is structured as follows: Section 2 details the system
model adopted in our study. Section 3 defines the problem we address. Section
4 describes our proposed solutions. Section 5 discusses the experiments and
results. Section 6 reviews relevant literature. Finally, Section 7 concludes
our research and outlines directions for future work.

##  2 System Model

This section delves into the system model used in our study, laying the
groundwork for understanding the challenges and constraints we address. Figure
3 depicts the architecture of the heterogeneous platform we adopted. This
platform consists of a low-power _RP2040_ MCU for coordination tasks, coupled
with an embedded _Spartan-7 XC7S15_ FPGA. The MCU is usually in sleep mode,
consuming 180 $\mu$A of current. It is woken up by either external hardware
interrupts or through timers to perform periodic tasks. Meanwhile, the FPGA
serves as a hardware accelerator and is activated only when needed, primarily
for processing and accelerating DL tasks.

Figure 3: Architecture for the Hardware designed for DL Accelerator

The communication interface connecting the MCU and the FPGA is a Serial
Peripheral Interface (SPI). The FPGA is connected to flash with a dedicated
SPI interface, supports clock frequencies from 3 to 66 MHz, and can be
programmed to operate in single, dual, or quad buswidths. The FPGA can fetch
bitstreams through this interface, facilitating seamless configuration when
powering up or switching between different accelerators.

The power supply of the system is designed in a dual-mode arrangement,
consisting of both a USB connection and a 320 mAh rechargeable LiPo Battery.
The battery provides an energy capacity of approximately 4147 J, serving as
the system’s designated energy budget, denoted as $E_{\text{Budget}}$. To
better profile the energy consumers in this system, it includes an energy
monitoring subsystem equipped with two PAC1934 sensors, sampling at 1024 times
per second for power rail. Figure 3 also illustrates the seven monitored power
rails that are critical to the system’s operation, with lines and arrows
marking their interconnections.

##  3 Problem Statement

As highlighted in Section 1, with periodic inference requests, the FPGA in our
system can be powered off for a duration of $T_{\text{off}}$ to conserve
energy. However, for SRAM-based FPGAs like ours, powering off results in the
loss of configuration data stored on the chip. This necessitates reconfiguring
the FPGA from external flash for each powering on, adding significant overhead
to every inference request.

Figure 2 demonstrates that enhancements in data transmission and inference
phases have a limited effect on the overall energy consumption per workload
item. Reducing the energy consumption of these phases to zero would only lead
to a 12.85% decrease in the total energy per workload item. In contrast,
eliminating the energy overhead associated with the configuration phase could
potentially enable the execution of up to 6 additional inference requests,
effectively allowing the processing of up to 6$\times$ more workload items
within the same energy budget.

Consequently, our study aims to achieve two primary goals: firstly, to reduce
the energy consumed during the configuration phase in one workload item, and
secondly, to decrease the number of necessary configurations while considering
the request period.

##  4 Proposed Solution

This section details our proposed solutions in three steps. First, we focus on
reducing the energy consumption during the configuration phase of a single
workload item. Next, we extend our approach to optimize average energy
consumption across multiple workload items. Finally, we introduce an
analytical model for estimating the executable workload items of each strategy
under given application requirements.

###  4.1 Reducing Energy for FPGA Configuration Phase

In the first step, we investigated whether it is possible to reduce or
eliminate the energy overhead associated with the FPGA configuration phase by
fine-tuning its parameters. To achieve this, we delved into the detailed
configuration process as outlined in the Xilinx 7-Series FPGA configuration
user guide [2], mainly focusing on the stages where potential energy savings
could be most significant, as shown in Figure 4. Our empirical analysis
highlights that the _Clear Configuration Memory_ and _Load Configuration Data_
stages are the most energy-intensive. In contrast, other stages contribute
minimally to the overall energy consumption due to their short duration.

Figure 4: Breakdown of FPGA Configuration Phase

Our experiments on real hardware show that the _Setup_ stage imposes a
substantial delay of 27 milliseconds for the _Spartan-7 XC7S15_ FPGA after all
power rails are ready. Regrettably, further optimization of this stage proves
infeasible due to its inherent dependence on the FPGA model. Due to these
constraints, our focus shifts to the _Load Configuration Data_ stage. This
stage offers three adjustable parameters: SPI buswidth, SPI clock frequency,
and bitstream compression Option, as detailed in Table 1. Our approach
involves carefully analyzing these parameters to determine the optimal
combination that minimizes the energy cost during this stage. Section 5.2
details the exploration of these parameters, their interplay, and the
resulting impact on the energy efficiency of the FPGA configuration phase.

Table 1: Adjustable Parameters of _Bitstream Loading_ Stage on 7-series FPGAs Parameters | Values
---|---
SPI Buswidth | 1, 2, 4
SPI Clock Frequency | 3, 6, 9, 12, 16, 22, 26, 33, 40, 50, 66
Bitstream Compression Option | False, True

While adjusting these parameters yielded a significant 40.13-fold decrease in
energy consumption, the configuration phase still imposes a notable energy
cost of 11.85 mJ. This observation guides us toward our second solution, which
aims to minimize the number of FPGA configurations within the energy budget.

###  4.2 Minimize Number of Configurations by Idle-Waiting

Taking the _Spartan-7 XC7S15_ FPGA model as an example, energy consumption in
the _Setup_ stage is unavoidable. So even if the energy cost of the _Bitstream
Loading_ stage is optimized to zero, the energy consumption of the
configuration phase can only be reduced from 11.85 mJ to 7 mJ. Thus, it
becomes imperative to adopt higher-level strategies that account for multiple
inference requests to optimize the average energy consumption per inference
request. We focus on two strategies that can impact the overhead imposed by
the configuration phase. The underlying assumption is that the same
accelerator is constantly (re)used for all inference requests. An analysis of
supporting different accelerators is outside the scope of this work.

####  4.2.1 On-Off Strategy

As Section 1 outlines, the FPGA powers off after completing a workload item
and reactivates for incoming inference requests. We define this process for
periodic inference requests as the On-Off strategy. In this strategy, the FPGA
only consumes energy during workload execution, encompassing configuration,
data transmission, and inference phases (as depicted in Figure 5). Notably,
the FPGA does not use energy while powered off, and the transition to off
state happens instantaneously without energy cost. Thus, optimizing the energy
efficiency involves maximizing the off-time ($T_{\text{off}}$) while
maintaining the execution of workload items within the application’s latency
requirements. The strategy proves effective when the execution time
($T_{\text{latency}}$) of a workload item is shorter than the request period
($T_{\text{req}}$), allowing sufficient downtime for energy conservation.

Figure 5: Illustration of On-Off strategy Figure 6: Illustration of Idle-
Waiting Strategy

####  4.2.2 Idle-Waiting Strategy

Extending on the On-Off strategy, we develop an alternative strategy by
integrating an idle-waiting phase to replace the conventional powered-off
period. This modification aims to bypass the energy-intensive configuration
phase. Figure 6 illustrates the core concept of incorporating the idle-waiting
phase amidst inference requests. In this strategy, the FPGA undergoes a one-
time configuration at the outset, referred to as the initial overhead.
Consequently, $T_{\text{latency}}$ excludes the time typically consumed by the
FPGA configuration phase. This strategy is advantageous in scenarios with
frequent inference requests, particularly when the energy overhead of the
idle-waiting phase, determined by the time ($T_{\text{idle}}$) of idle-waiting
phase and the FPGA’s idle power ($P_{\text{idle}}$), is less than the energy
consumed in each configuration phase.

To maximize the effectiveness of the Idle-Waiting strategy, we propose two
methods to reduce the FPGA’s idle power consumption. The first method involves
deactivating non-essential components, such as IOs and clock references, when
the FPGA is idle. The second method is to reduce the voltage of the FPGA
during the idle-waiting phase to a sufficient level to maintain the
configuration but not to enable the operational functions, i.e., data
transmission and execution of the actual inference. By integrating these two
approaches, we anticipate achieving even greater enhancements in reducing
power consumption. However, such hardware optimizations necessitate
specialized components that may not be readily available in the commercial
market. Hence, we employed a simulator to validate and assess the feasibility
of these optimizations. The detailed description of this simulator is
presented in Section 5.1.

###  4.3 Analytical Model

To evaluate energy consumption across different strategies within a designated
energy budget, we develop an analytical model. This model is instrumental in
identifying the maximum number of executable workload items and estimating the
corresponding system lifetime.

For the On-Off strategy, $E_{\text{Sum}}^{\text{OnOff}}(n)$ as outlined in
Equation 1 represents the cumulative energy cost for $n$ workload items. Each
$E_{\text{Item}}^{\text{OnOff}}$ includes the energy consumed during the
configuration, data transmission and inference phases. In the Idle-Waiting
strategy, as detailed in Equation 2, the total energy cost,
$E_{\text{Sum}}^{\text{IdleWait}}(n)$, is comprised of three key components:
1) $E_{\text{Init}}$ represents the one-time initial overhead incurred by the
FPGA at the start of the system. 2)
$\sum_{i=1}^{n}E_{\text{Item}}^{\text{IdleWait}}$ quantifies the energy
required for $n$ workload items, where all configuration-related overheads are
zero. 3) $\sum_{i=1}^{n-1}E_{\text{Idle}}$ accounts for the energy consumed
during idle periods between workload items, where $E_{\text{Idle}}$ is
determined by the idle time ($T_{\text{idle}}$) and the FPGA’s idle power
consumption ($P_{\text{idle}}$).

$E_{\text{Sum}}^{\text{OnOff}}(n)=\sum_{i=1}^{n}E_{\text{Item}}^{\text{OnOff}}$
(1)
$E_{\text{Sum}}^{\text{IdleWait}}(n)=E_{\text{Init}}+\sum_{i=1}^{n}E_{\text{Item}}^{\text{IdleWait}}+\sum_{i=1}^{n-1}E_{\text{Idle}}$
(2)

To ascertain the maximum number ($n_{\text{max}}$) of workload items
executable within the energy budget ($E_{\text{Budget}}$), we set a criterion
ensuring that $E_{\text{Sum}}(n)$ for different strategies optimally aligns
with but does not exceed $E_{\text{Budget}}$, as formulated in Equation 3. The
system lifetime ($T_{\text{lifetime}}$) is then calculated by multiplying the
derived ($n_{\text{max}}$) by the request period ($T_{\text{req}}$), as per
Equation 4.

$n_{\text{max}}=\max\\{n\in\mathbb{N}\mid E_{\text{Sum}}(n)\leq
E_{\text{Budget}}\\}$ (3) $T_{\text{lifetime}}=n_{\text{max}}\times
T_{\text{req}}$ (4)

This analytical model provides a theoretical basis for our research, enabling
a rapid analysis of how various strategies impact energy consumption. We plan
to rigorously validate its effectiveness in upcoming experiments, ensuring its
practical applicability in real-world scenarios.

##  5 Experiments and Results

To validate the effectiveness of our proposed solutions, we conducted three
interconnected experiments. The first experiment focused on reducing energy
consumption during the FPGA configuration phase. The second experiment
assessed the Idle-Waiting strategy’s ability to reduce frequent
configurations. The third experiment explored the power-saving methods in the
idle-waiting phase, further enhancing the strategy’s effectiveness.

###  5.1 Experiments Setup

We utilized the hardware specified in Section 2 for our experiments.
Additionally, to accelerate experimentation and assist in scenarios where
direct hardware testing is impractical, we developed a Python-based simulator,
inspired by [5]. This tool aligns with the analytical model described in
Section 4.3, and outputs the maximum number of executable workload items along
with estimations of the system lifetime.

This simulator enables the specification of overall workload and individual
workload items using YAML files, simplifying the execution of extensive
experiments involving large datasets or complex measurements. A key feature of
this simulator is its ability to incorporate both datasheet specifications and
real hardware measurement, thus enhancing the precision of energy consumption
estimations and offering a more realistic representation of actual scenarios.

The simulator requires two descriptions to operate: 1) the _workload_ and 2)
the _workload item_. The _workload_ description contains the energy budget
$E_{\text{Budget}}$ in joules and the constant request period, as mentioned in
Section 1. The _workload item_ description details each phase’s average power
consumption in milliwatts and duration in milliseconds. With these inputs, the
simulator can effectively model the various strategies discussed in Section
4.2, allowing us to examine them under diverse conditions.

###  5.2 Experiment 1: Optimization on Energy for FPGA Configuration

In the first experiment, we conducted a hardware-based investigation of
various FPGA configuration parameters involving 11 SPI clock frequencies, 3
SPI buswidths, and the bitstream compression option, listed in Table 1. We
aimed to assess their impact on energy consumption during the FPGA
configuration phase. Utilizing an LSTM accelerator with a hidden size of 20,
as detailed in our prior study [13], we generated corresponding bitstreams for
the _Spartan-7 XC7S15_ FPGA. The evaluation metrics include configuration time
(milliseconds), power usage (milliwatts), and energy consumption (millijoules)
during the configuration phase.

Figure 7: Performance Comparison on the _Spartan-7 XC7S15_ FPGA

Due to space constraints, Figure 7 selectively presents our results at SPI
clock frequencies of 3, 33, and 66 MHz. This selection of data points
illustrates the effects across low, medium, and high-frequency settings. The
first column of graphs displays the configuration phase outcomes, while the
second and third columns break down these results into the _Setup_ and
_Bitstream Loading_ stages, respectively. To facilitate a clearer
understanding, the evaluation metrics are displayed in separate rows. The
y-axis of each metric is scaled logarithmically.

Our findings indicate a marked decrease in configuration time with increased
buswidth and clock frequency. Employing bitstream compression further enhanced
efficiency. In the case of Quad SPI at 66 MHz with compression enabled, it
reduced configuration time to 36.15 ms, a 41.4-fold improvement over the least
efficient setting of Single SPI at 3 MHz without compression. This decrease is
mainly attributed to the impact of settings on the _Bitstream Loading_ stage.

The average power consumption during the configuration phase varied,
reflecting an aggregate of the _Setup_ and _Bitstream Loading_ stages. The
_Setup_ stage maintained a consistent power consumption of around 288 mW. In
contrast, the _Bitstream Loading_ stage showed increased power usage,
especially with larger SPI buswidth and higher frequencies. Notably, bitstream
compression led to higher power in this stage, likely due to more switching
activities on the SPI data line.

Using Quad SPI at 66 MHz with compression, the energy consumption during the
configuration phase was 11.85 mJ, compared to 475.56 mJ for Single SPI at 3
MHz without compression, illustrating a significant 40.13-fold reduction in
energy achieved with optimal settings. The trend in energy savings mirrored
the timing pattern, with the _Bitstream Loading_ stage contributing
significantly. Higher SPI frequencies and wider buswidths led to lower energy
costs, attributable to the static power characteristics of Spartan-7 FPGAs.
Accelerating the bitstream loading process shortened the duration of static
power draw, thereby decreasing overall energy consumption.

Similar experiments on the larger _Spartan-7 XC7S25_ FPGA yielded comparable
results. With the optimal settings, the configuration time for the same
accelerator was 38.09 ms, and energy consumption was 13.75 mJ. These results
suggest that the highest clock frequency and widest SPI buswidth optimize
configuration energy, as long as the hardware supports these settings.
However, such settings increase power consumption during the _Bitstream
Loading_ stage, requiring a higher power budget for the hardware.

###  5.3 Experiment 2: Idle-Waiting vs On-Off Strategies

In this experiment, we set out to identify the request period range where the
Idle-Waiting strategy is more efficient than the On-Off strategy.
Additionally, we aimed to validate the effectiveness of our analytical model
using these experimental results. Utilizing the LSTM accelerator described
earlier, we measured timing and power consumption to characterize a workload
item, as listed in Table 2. We applied the optimal settings identified in
Experiment 1 for the configuration phase. Note that the idle power consumption
of 134.3 mW listed in Table 2 is specific to the Idle-Waiting strategy.
Profiling other accelerators is also feasible, simply requiring an adjustment
of the characteristics listed in Table 2 to align with the specific
accelerator being used.

Table 2: Power and Time on Hardware for Simulation

Settings | LSTM Accelerator[13]
---|---
Power (mW) | Time (ms)
Phases | Configuration | 327.9 | 36.145
Data Loading | 138.7 | 0.0100
Inference | 171.4∗ | 0.0281
Data Offloading | 144.1 | 0.0020
Idle-Waiting | 134.3 | varying†

$\dagger$ The idle time varies in response to changes in the request period.
$*$ This power includes the 114 mW for clock reference and flash chip.

Figure 8: Workload Items: Idle-Waiting vs On-Off Strategies

Figure 9: System Lifetime: Idle-Waiting vs On-Off Strategies

Utilizing the simulator, we first estimated the number of executable workload
items within an energy budget of 4147 J for request periods ranging from 10 to
120 ms, in increments of 0.01 ms. This range was chosen to align with our
analytical model’s prediction of an efficiency cross point around 89.21 ms,
enabling a comprehensive performance analysis.

Figure 9 displays the number of executable workload items on a logarithmic
scale. For brevity, we show values at 10 ms intervals. Under the Idle-Waiting
strategy, the number of executable workload items ranges from a minimum of
approximately 257,305 to a maximum of about 3,085,319. Conversely, the On-Off
strategy consistently supports 346,073 executable items. At a 40 ms request
period, the Idle-Waiting strategy yields 2.23$\times$ more workload items.
Note that the On-Off strategy is not represented for request periods below
36.15 ms due to its configuration time cost. When the request period is
shorter than 36.15 ms, the FPGA can not be prepared to process an incoming
workload. The cross point at 89.21 ms shown in Figure 9, as identified by our
analytical model, highlights the effectiveness of the Idle-Waiting strategy
for request periods shorter than this threshold.

Figure 9 provides an estimation of the system lifetime under each strategy.
For the Idle-Waiting strategy, the lifetime averages about 8.58 hours, with a
marginal increase across different request periods. This subtle variation,
while not prominently visible in the figure, aligns with expectations: the
average power consumption in the Idle-Waiting strategy tends to approach idle
power levels, since the other phases consume very little energy for their
duration. The cross point in Figure 9 mirrors the findings in Figure 9. In
contrast, the On-Off strategy exhibits a linear increase in system lifetime as
request periods extend.

To assess the precision of our simulator, we conducted direct hardware
measurements at a 40 ms request period for both strategies, considering the
FPGA and its peripherals’ energy consumption. These measurements exhibited
only minor variations, with a 2.8% difference in executable workload items and
a 2.7% discrepancy in system lifetime, thereby affirming the simulator’s
usability in offering preliminary insights into the accelerator’s behavior.

Our results affirm the potential of the Idle-Waiting strategy for request
periods below 89.21 ms. The main limitation arises from the power consumption
during the idle-waiting phase. We believe that reducing idle power could allow
for the execution of more workload items, thereby expanding the range of
request periods where the Idle-Waiting strategy is effective.

###  5.4 Experiment 3: Optimization on the Idle-Waiting Strategy

In the final experiment, we aimed to improve the Idle-Waiting strategy by
reducing idle power consumption, as proposed in Section 4.2. We evaluated
these enhancements against the initial Idle-Waiting strategy (referred to as
Baseline) from Section 5.3. We utilized the hardware setting described in
Section 2 to assess the feasibility of these methods, and the results are
detailed in Table 3.

Table 3: Idle Power on Hardware for Simulation Metric | Optimization Methods
---|---
Baseline | Method 1 | Method 1+2
Idle Power (mW) | 134.3 | 34.2 | 24.0
Saved Power (%) | - | 74.38 | 81.98

By deactivating the clock reference and FPGA IOs (Method 1), the idle power
can be reduced to 34.2 mW, achieving a 74.38% power saving compared to the
baseline. Further reductions were achieved by lowering the FPGA’s internal and
auxiliary supply voltages (Method 2) from 1.0 V and 1.8 V to 0.75 V and 1.5 V,
respectively. Combining Methods 1 and 2 lowered the idle power to 24 mW,
resulting in an 81.98% reduction compared to the baseline. It is important to
note that our hardware setup includes a flash component with a constant power
consumption of approximately 15.2 mW, which is factored into all power
calculations presented in Table 3. We verified on our hardware that exiting
from these power-saving methods does not affect the FPGA’s configuration,
ensuring it remains retained and functional.

Figure 10: Workload Items: Baseline vs. Optimized Methods Across Request
Periods Figure 11: System Lifetime: Baseline vs. Optimized Methods Across
Request Periods

Due to the lack of dynamic voltage scaling support in our hardware, we relied
on our simulator to estimate potential improvements for test cases that the
hardware could not support. Figure 10 illustrates that Method 1 alone
significantly enhanced the number of executable workload items by 3.92$\times$
relative to the Baseline. Combining both methods yielded even more substantial
gains, increasing the number to 5.57$\times$ the Baseline. Consequently,
Figure 11 shows a proportional increase in system lifetime. Applying method 1
extended the average lifetime to 33.64 hours, a 3.92-fold improvement from
Baseline. Furthermore, the combination of methods 1 and 2 further augmented
the average lifetime to 47.80 hours. These enhancements expanded the
beneficial request period range for the Idle-Waiting strategy from 89.21 ms to
499.06 ms.

The effectiveness of our power-saving methods in increasing executable
workload items is evident, yet further idle power reduction is needed.
Hardware constraints, mainly the flash’s power consumption, limit our method’s
optimal period to 499.06 ms. Addressing this could extend the advantageous
period by up to 5.57$\times$.

##  6 Related Work

Recent research on DL accelerators on FPGAs has made significant progress,
particularly in throughput and energy efficiency [9, 10, 12]. These studies
typically focus on specialized hardware design and optimized inference phase
to boost accelerator performance, primarily during continuous processing
tasks, with the FPGA constantly active. However, these studies frequently
neglect the overhead of FPGA configuration and related phases. For example,
Chen et al. [3] focused on executing a single inference after a power failure,
assuming that the FPGA is pre-configured before the power failure. Thus, they
only need to optimize the performance of the accelerator in the inference
phase. In practical scenarios, DL tasks typically demand a series of
inferences, not just a single inference execution.

Some researchers have started to address the impact of FPGA configuration
overhead on efficiency. Fritzsch et al. [6] proposed a method to compress the
bitstream by 1.05 to 12.2$\times$ to reduce configuration time, yet they did
not explore its energy efficiency implications. Cichiwskyj et al. [5]
introduced the concept of Temporal Accelerators, demonstrating that even with
two reconfigurations, using a smaller FPGA (_Spartan-7 XC7S6_) is more energy-
efficient than a larger one (_Spartan-7 XC7S15_) for a single inference
execution. However, these studies did not integrate the configuration overhead
with request period considerations for energy efficiency.

Our study differs by aiming to enhance the energy efficiency of embedded DL
systems through the lens of periodic workload requests. We adopted a dual-
phase approach: firstly, we optimized configuration parameters to minimize
overhead, and secondly, we employed idle power optimizations to maintain the
FPGA powered on, thus avoiding repeated configurations.

##  7 Conclusion and Future Work

In conclusion, this study significantly reduced the configuration overhead of
FPGA-based DL accelerators in IoT applications. By optimizing the FPGA
configuration phase and introducing an effective Idle-Waiting strategy, we
demonstrated substantial energy savings, thereby increasing the number of
executable workload items. Our Idle-Waiting strategy effectively addresses the
challenges of shorter request periods, a limitation of the traditional On-Off
strategies. Combining the Idle-Waiting strategy with idle power-saving methods
at a 40 ms request period, it achieves 12.39$\times$ more workload items and
system lifetime than the On-Off strategy.

In the future, we plan to extend our power-saving techniques beyond DL use
cases to other periodic processes. Additionally, we aim to investigate methods
for efficiently handling irregularly occurring inference requests, focusing on
optimizing energy efficiency and system performance in more complex scenarios.
These efforts will help validate and refine our approaches in diverse
operational contexts.

Acknowledgements. The authors acknowledge the financial support provided by
the Federal Ministry for Economic Affairs and Climate Action of Germany in the
RIWWER project (01MD22007C).

## References

  * [1] Akkad, G., Mansour, A., Inaty, E.: Embedded deep learning accelerators: A survey on recent advances. IEEE Transactions on Artificial Intelligence (2023)
  * [2] AMD: 7 series fpgas configuration user guide. https://docs.xilinx.com/v/u/en-US/ug470_7Series_Config (2023)
  * [3] Chen, J., Hong, S., He, W., Moon, J., Jun, S.W.: Eciton: Very Low-Power LSTM Neural Network Accelerator for Predictive Maintenance at the Edge. In: 2021 31st International Conference on Field-Programmable Logic and Applications (FPL). pp. 1–8. IEEE (2021)
  * [4] Chéour, R., Khriji, S., El Houssaini, D., Baklouti, M., Abid, M., Kanoun, O.: Recent trends of fpga used for low-power wireless sensor network. IEEE Aerospace and Electronic Systems Magazine 34(10), 28–38 (2019)
  * [5] Cichiwskyj, C., Qian, C., Schiele, G.: Time to learn: Temporal accelerators as an embedded deep neural network platform. In: International Workshop on IoT, Edge, and Mobile for Embedded Machine Learning. pp. 256–267. Springer (2020)
  * [6] Fritzsch, C., Hoffmann, J., Bogdan, M.: Reduction of bitstream size for low-cost ice40 fpgas. In: 2022 32nd International Conference on Field-Programmable Logic and Applications (FPL). pp. 117–122. IEEE (2022)
  * [7] Gan, V.M., Liang, Y., Li, L., Liu, L., Yi, Y.: A cost-efficient digital esn architecture on fpga for ofdm symbol detection. ACM Journal on Emerging Technologies in Computing Systems (JETC) 17(4), 1–15 (2021)
  * [8] Krishnamoorthy, R., Krishnan, K., Chokkalingam, B., Padmanaban, S., Leonowicz, Z., Holm-Nielsen, J.B., Mitolo, M.: Systematic approach for state-of-the-art architectures and system-on-chip selection for heterogeneous iot applications. IEEE Access 9, 25594–25622 (2021)
  * [9] Magyari, A., Chen, Y.: Review of state-of-the-art fpga applications in iot networks. Sensors 22(19), 7496 (2022)
  * [10] Muralidhar, R., Borovica-Gajic, R., Buyya, R.: Energy efficient computing systems: Architectures, abstractions and modeling to techniques and standards. ACM Computing Surveys (CSUR) 54(11s), 1–37 (2022)
  * [11] Olney, B., Mahmud, S., Karam, R.: Efficient nonlinear autoregressive neural network architecture for real-time biomedical applications. In: 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS). pp. 411–414. IEEE (2022)
  * [12] Qian, C., Ling, T., Schiele, G.: Enhancing energy-efficiency by solving the throughput bottleneck of lstm cells for embedded fpgas. In: European Conference on Machine Learning and Knowledge Discovery in Databases. pp. 594–605. Springer (2022)
  * [13] Qian, C., Ling, T., Schiele, G.: Energy efficient lstm accelerators for embedded fpgas through parameterised architecture design. In: International Conference on Architecture of Computing Systems. pp. 3–17. Springer (2023)
  * [14] Situnayake, D., Plunkett, J.: AI at the Edge. ” O’Reilly Media, Inc.” (2023)