# A DSP shared is a DSP earned:
HLS Task-Level Multi-Pumping for
High-Performance Low-Resource Designs

Giovanni Brignone, Mihai T. Lazarescu, Luciano Lavagno  Dipartimento di
Elettronica e Telecomunicazioni
Politecnico di Torino
Turin, Italy
{giovanni.brignone, mihai.lazarescu<EMAIL_ADDRESS>
###### Abstract

HLS enhances digital hardware design productivity through a high abstraction
level. Even if the high-level synthesis (HLS) abstraction prevents fine-
grained manual register-transfer level (RTL) optimizations, it also enables
automatable optimizations that would be unfeasible or hard to automate at RTL.
Specifically, we propose a task-level multi-pumping methodology to reduce
resource utilization, particularly digital signal processors, while preserving
the throughput of HLS kernels modeled as dataflow graphs targeting field-
programmable gate arrays. The methodology exploits the HLS resource sharing to
automatically insert the logic for reusing the same functional unit for
different operations. In addition, it relies on multi-clock DFGs to run the
multi-pumped tasks at higher frequencies. The methodology scales the pipeline
initiation interval (II) and the clock frequency constraints of resource-
intensive tasks by a multi-pumping factor ($M$). The looser II allows sharing
the same resource among $M$ different operations, while the tighter clock
frequency preserves the throughput. We verified that our methodology opens a
new Pareto front in the throughput and resource space by applying it to open-
source HLS designs using state-of-the-art commercial HLS and implementation
tools by Xilinx. The multi-pumped designs require up to 40% fewer DSP
resources at the same throughput as the original designs optimized for
performance (i.e., running at the maximum clock frequency) and achieve up to
50% better throughput using the same DSPs as the original designs optimized
for resources with a single clock.

###### Index Terms:

Dataflow architectures, FPGA, high-level synthesis, multi-pumping, resource
sharing

##  I Introduction

HLS raises the abstraction level of electronic design automation tools to
improve the digital hardware designer’s productivity. The high abstraction
precludes some low-level manual optimizations, making the quality of results
(QoR) of HLS circuits inferior to those manually optimized at the RTL,
especially for the area and maximum clock frequency [1]. On the other hand, we
deem the HLS description introduces new optimization opportunities at a high
level.

We focus on HLS designs modeled as DFGs (e.g., with _dataflow_ in Xilinx
Vivado/Vitis HLS [2], _hierarchy_ in Siemens Catapult HLS [3], or _task
functions_ in Intel HLS compiler [4]). Modeling HLS designs as DFGs proved its
effectiveness both in industrial [5, 6] and academic mmm_hls, rosetta
projects.

DFG DFG is a set of parallel computational tasks (C/C++ functions in HLS)
communicating asynchronously through first-in-first-out (FIFO) queues. HLS
tools typically implement DFGs as single-clock dataflow graphs, where all the
tasks share the same clock signal. Many modern HLS tools do not support multi-
clock designs [2, 4]. Nevertheless, we can generalize SCDFGs to multi-clock
dataflow graphs by assigning each task to a dedicated clock domain. The
generalization enhances the tasks’ flexibility and maximum frequency, limited
only by the critical timing path local to the task rather than the global one.
Clock architectures of modern field-programmable gate array (FPGA) system-on-
chips seamlessly support multiple clocks, and the area overhead for safe clock
domain crossing (CDC) is negligible since the tasks already communicate
through FIFOs, which can be configured with independent read and write clocks
with comparable resource utilization [9].

Multiple clock domains allow optimizations like multi-pumping, which reduces
the area while preserving the throughput by reusing $M$ times a resource,
usually DSP DSP unit in the FPGA context, clocked at a frequency $M$ times
larger than the rest of the system. Designers typically apply the technique at
RTL by manually inserting the custom logic to share the resource and safely
perform CDC.

In this work, we achieve multi-pumping at the task level by tuning only the
high-level parameters of the tasks, in particular the pipeline II, i.e., the
clock cycles between the start of successive pipeline executions, and the
clock constraint at the task granularity, taking advantage of the MCDFG. The
HLS resource sharing algorithm automatically builds the logic for sharing the
resource within a dataflow task. At the same time, the inter-task FIFOs allow
safe CDC. We focused on DSPs, which are critical in compute-intensive kernels
and can run at high frequencies. However, the technique can multi-pump any
shareable resource, including entire sub-functions.

For example, consider a 2D Convolution HLS kernel by [10], implemented as
SCDFG SCDFG, as shown in Fig. 1a,

|  |
---

figures/conv2d_dfg

(a) SCDFG

figures/conv2d_filter_250

(b) SOTA Filter2D

figures/conv2d_filter_500

(c) Double-pumped Filter2D

Figure 1: Task-level multi-pumping saves resources at equal throughput for HLS
of dataflow graphs (DFGs). The Filter2D task from a 2D Convolution kernel [10]
LABEL:sub@subfig:conv2d_srdfg is double-pumped LABEL:sub@subfig:filter2d_500
by doubling its clock frequency and II to save half of the multipliers of the
single-clock solution LABEL:sub@subfig:filter2d_250.

where the rectangular nodes and the arrows represent the tasks and the FIFOs,
respectively. At each iteration, the Filter2D task processes a convolution
window of up to $15\times 15$ elements, which requires computing $225$
multiply and accumulate (MAC) operations bound to DSPs. Thus, II II of
$1\text{\,}\mathrm{cycle}\text{/}$ requires $225$ DSPs. On the other hand,
scaling the II to $2\text{\,}\mathrm{cycles}\text{/}$ implies that a new
pipeline iteration starts every two clock cycles. Therefore, the pipeline has
two cycles to compute the 225 operations. Hence, thanks to resource sharing,
the HLS binding allocates only $\lceil 225/2\rceil=113$ DSPs, each of which
computes two MACs. Assume that we target a throughput of
$250\text{\,}\mathrm{MSa}\text{/}\mathrm{s}$. With the state-of-the-art SCDFG
flow (Fig. 1b), we set the clock frequency of the whole DFG, including the
Filter2D task that allocates 225 DSPs at $250\text{\,}\mathrm{MHz}\text{/}$.
On the other hand, with our multi-pumping approach (Fig. 1c), we optimize the
Filter2D task by scaling its II to $2\text{\,}\mathrm{cycles}\text{/}$, to
save half of the DSPs, and its clock frequency to
$500\text{\,}\mathrm{MHz}\text{/}$, to preserve the throughput.

This paper proposes an area-minimization methodology that preserves the
throughput via task-level multi-pumping for FPGA HLS designs described as
DFGs. Its effectiveness is validated on open-source designs using the workflow
shown in Fig. 2,

[width=]figures/workflow

Figure 2: Given the C/C++ source code of a dataflow graph (DFG) application
and its base clock frequency, the proposed workflow builds the optimized
multi-pumped IP by  (a) analyzing the DFG (_DFG charact._),  (b) optimizing
the multi-pumping factors (_$\mathcal{M}$ opt._), and  (c) synthesizing the
multi-pumped IP (_MCDFG synth._).

which generates an optimized multi-pumped intellectual property (IP) block
from C/C++ source code using state-of-the-art [11] commercial tools [11].

To the best of our knowledge, this is the first work that combines multiple
clock domains with resource sharing in HLS of DFGs for task-level multi-
pumping. The empirical results show that a new Pareto front opens in the
power, performance, and area (PPA) space, with circuits that use up to
$60\text{\,}\mathrm{\char 37\relax}\text{/}$ fewer resources at maximum
throughput or achieve up to $50\text{\,}\mathrm{\char 37\relax}\text{/}$
higher throughput with the same resources.

##  II Related work

Our work is mainly related to QoR improvement of HLS designs by tuning the HLS
directives (i.e., the instructions for the HLS compiler to control hardware
optimizations such as loop pipelining), focusing on multi-clock designs.

Several works [12, 13, 14, 15, 16] optimize for performance the HLS directives
applied to plain software code not intended for HLS via design-space
exploration (DSE). However, the goals of their works differ from ours since we
optimize for resources while preserving the throughput of source code already
optimized for HLS. In addition, our methodology avoids time-consuming DSEs and
analytically computes the multi-pumping factor and, consequently, the
corresponding II and clock frequency constraints. Finally, they all consider
only single clock designs, except for [16] (discussed further in Section
II-A).

HLS design optimizations based on multiple clock domains work at the
_operation level_ , assigning domains at the low-level resource (e.g., adder
or multiplier) granularity, typically during scheduling [17, 18, 19], or at
the _task level_ , assigning domains at function granularity (i.e., MCDFGs)
[20, 16].

###  II-A Operation-level multi-clock in high-level synthesis

[17] use multiple clock domains in HLS to reduce power consumption while
preserving the throughput by halving the operating frequency of two-cycle
operations. We instead focus on area and performance optimizations because
power is only a secondary quality metric for FPGA designs after performance
and area.

[18] and [19] design double-pumped DSP modules and use them in HLS with custom
resource-sharing algorithms. Theoretically, [11] Vitis HLS supports double-
pumped MAC operations through user-callable functions from the `dsp_builtins`
library, but it is undocumented and faulty [21]. Our approach produces similar
results when double-pumping a task. However, our task-level solution does not
require custom modules, changes to the HLS sharing algorithm, or changes to
the source code. In addition, it can select multi-pumping factors greater than
two, resulting in larger resource savings.

###  II-B Task-level multi-clock in high-level synthesis

[20] focus on extending the LegUp HLS tool to support MCDFGs synthesis but
leave the selection of the clock frequencies to a suboptimal, time-consuming
profiling-based approach. Our work focuses instead on a general methodology
for exploiting the multiple clock domains. The workflow we define for building
MCDFGs, based on state-of-the-art Xilinx tools, is just a means to apply our
methodology.

[16] propose DSE DSE methodology for maximizing the throughput under area
constraints for HLS of MCDFG designs. They iteratively push for performance
the HLS loop directives applied to the bottleneck tasks. If a task is still a
bottleneck after maximally pushing the directives (e.g., when the pipeline II
constraint is $1\text{\,}\mathrm{cycle}\text{/}$), they relax the directives
of every task, increase the clock frequency of the bottleneck task, and
restart the procedure. The goal of our work is different since we minimize the
area while preserving the throughput. The optimization approaches differ, too,
since we optimize all the resource-intensive tasks independently of whether
they are bottlenecks, and we never push the II constraints, which the HLS
compiler may fail to meet (e.g., due to data dependencies).

##  III Background

Given the DFG throughput model defined in Section III-A, our multi-pumping
methodology exploits the resource sharing executed by the HLS binding step to
build the sharing logic. The relaxed timing mode for the HLS scheduling step
ensures that the II of the pipelines is independent of the target clock
frequency, as explained in Section III-B.

###  III-A Dataflow graph

DFG DFG $G(V,E)$ is a set of tasks $v\in V$ running in parallel and
communicating asynchronously through FIFO channels $e\in E$.

In HLS, each task is described as a C/C++ function whose core computational
part typically consists of a pipelined loop. Given a task $v_{i}$ clocked at
frequency $f_{i}$ and whose core loop is scheduled with initiation interval
$\mathit{II}_{i}$, an approximation of its throughput is

$\Phi_{i}\coloneqq\frac{f_{i}}{\mathit{II}_{i}}.$ (1)

The maximum external dynamic random access memory (DRAM) bandwidth can also
limit throughput. However, this is out of the scope of our methodology since
it does not change the overall throughput and, consequently, the DRAM
bandwidth requirements.

The overall DFG throughput matches the one of the _bottleneck task_ (i.e., the
task with the lowest throughput)

$\Phi_{G}\coloneqq\min\limits_{v_{i}\in V}\Phi_{i}.$ (2)

According to Eq. 1, the high-level knobs for tuning the throughput of task
$v_{i}$ are its clock frequency ($f_{i}$) and initiation interval
($\mathit{II}_{i}$). All tasks share the same clock in single-clock dataflow
graphs (SCDFGs). Thus, $f_{i}$ is the same for all tasks and is bounded by the
global (i.e., among all tasks) critical path. Therefore, only the
$\mathit{II}_{i}$ can be tuned independently for each task. In MCDFG MCDFG, on
the other hand, the clock frequency can be set individually for each task.
This additional degree of freedom allows for higher flexibility and tasks
frequencies than SCDFG since the clock frequency of a task is limited only by
its local critical path and not the one of the whole DFG.

###  III-B High-level synthesis

Our multi-pumping technique relies on two key concepts of the HLS tools:  (1)
the minimum pipeline II is independent of the clock frequency constraint when
the scheduler works in _relaxed timing_ mode (i.e., the clock frequency is
subordinate to meet the II constraints), and  (2) the level of _resource
sharing_ is directly dependent on the II.  Both are implemented by the HLS
back-end that generates the hardware description. The timing model is used
during _scheduling_ and the resource sharing is done during _binding_ [2].

####  III-B1 Scheduling

Scheduling assigns operations to specific clock cycles; thus, it also
implements loop and function _pipelining_. Designers can constrain the II of
the pipelines, which is lower bound by the resource constraints and the data
dependencies. Consider the data dependence graph (DDG) modeling the data
dependencies in a kernel. Given a cycle $\theta$ in the DDG, we define
$\mathit{delay}_{\theta}$ as the sum of the delays of the operations along
$\theta$ and $\mathit{dist}_{\theta}$ as the total loop-carried dependence
distance along $\theta$. The lower bound of the II is

$\mathit{II}^{\text{min}}\coloneqq\max\limits_{\theta\in\mathrm{DDG}}\left(\frac{\mathit{delay}_{\theta}}{\mathit{dist}_{\theta}}\right).$
(3)

The associated cycle is called _critical_ [22].

For example, consider the following loop to be scheduled:

⬇ for (int i = 0; i < N; i++)  a = a + b;

The read-after-write dependency on a, produced at the $i$-th iteration and
consumed at the $i+1$-th iteration, introduces a cycle $\theta$ in the DDG.
$\mathit{delay}_{\theta}$ is the latency of the adder computing a+b.
$\textit{dist}_{\theta}$ is 1 since a is consumed at the iteration after it is
produced. Therefore, Eq. 3 implies that the minimum II for this loop equals
the latency of the adder.

The clock constraints determine how many operations fit within a clock cycle,
thus affecting the depth of the pipelines. The pipeline depth determines the
latencies of its operations, impacting the critical cycle and, in turn, the II
lower bound. However, the II constraints take precedence over clock
constraints in _relaxed timing_ mode, yielding lower II pipelines in exchange
for potential HLS timing violations. These are usually acceptable at HLS time
since HLS timing estimations may be overly pessimistic [1], and downstream
implementation steps may resolve them.

####  III-B2 Binding

Binding assigns each operation to a compatible functional unit, depending on
resource and performance (e.g., clock frequency, II) constraints.

_Resource sharing_ is a crucial binding optimization that maps operations of
the same type to the same functional unit, scheduled on different clock cycles
or under mutually exclusive conditions (e.g., on different if-then-else
branches). The II constraints directly affect the degree of resource sharing.
In particular, if a pipeline scheduled with II II of
$\text{$\mathit{\acs{II}}_{i}$}\text{\,}\mathrm{cycles}\text{/}$ computes
$N^{\text{\acs{OP}}}_{i}$ operations (OPs) of the same kind at each iteration,
the binding step allocates $N^{\text{\acs{FU}}}_{i}$ functional units (FUs),
with

$N^{\text{\acs{FU}}}_{i}\coloneqq\left\lceil\frac{N^{\text{\acs{OP}}}_{i}}{\mathit{\acs{II}}_{i}}\right\rceil.$
(4)

Note that the operations can be either computations or memory accesses. The
functional units associated with the memory operations are ports proportional
to the partitioning factors (i.e., the number of submemories into which a
memory resource is divided to increase its parallelism). Therefore, larger II
values result in fewer functional units and smaller memory partitioning
factors.

Consider the Filter2D task from the 2D Convolution kernel introduced in
Section I, whose source code is in Fig. 3a.

[width=]figures/conv2d-src

(a)

[scale=.6]figures/conv2d-filter_ii_1

(b)

[scale=.6]figures/conv2d-filter_ii_2

(c)

Figure 3: The pipeline initiation interval (II) directly affects the resource
sharing. For example, in the Filter2D task LABEL:sub@subfig:filter_src, the
pipeline with $\mathit{\acs{II}}=$1\text{\,}\mathrm{cycle}\text{/}$$
LABEL:sub@subfig:filter_ii_1 computes four multiplications per clock cycle in
steady state, while the one with
$\mathit{\acs{II}}=$2\text{\,}\mathrm{cycles}\text{/}$$
LABEL:sub@subfig:filter_ii_2 only two. Thus, the latter datapath allocates
half of the multipliers.

Assuming a filter of size $2\times 2$ (i.e.,
$\text{{FILTER\\_V\\_SIZE}}=\text{{FILTER\\_H\\_SIZE}}=2$), with the schedule
with II II of $1\text{\,}\mathrm{cycle}\text{/}$ (shown in Fig. 3b, where the
nodes represent the operations, and the edges the data dependencies), at the
steady-state, four multiplications are computed in parallel on different data
within the same clock cycle (highlighted by the red rectangle), thus requiring
four DSP-mapped multipliers. With II II of $2\text{\,}\mathrm{cycles}\text{/}$
instead (see Fig. 3c), only two multiplications are computed per clock cycle.
Therefore, the binding step allocates only two multipliers and shares these
among two multiplications.

##  IV Task-level multi-pumping

We multi-pump the resources of task $v_{i}$ by simultaneously scaling by a
multi-pumping factor $M_{i}$ the II and the clock frequency of $v_{i}$.

The underlying principles of our approach are:

  * •

Equation 2 allows tuning each task independently without reducing the overall
DFG throughput, as long as the throughput of the task does not get lower than
the bottleneck task one.

  * •

As discussed in Section III-B2, scaling the II of a pipelined loop by a factor
$M_{i}$ allows reusing the same functional unit for $M_{i}$ operations in
different clock cycles.

  * •

Equation 1 implies that the task throughput is unchanged if we scale by
$M_{i}$ the task clock frequency together with the II.

Assume that $v_{i}$ meets the timing constraints up to $f^{\text{max}}_{i}$
and computes $N^{\text{\acs{OP}}}_{i}$ operations mapped to DSPs. Moreover,
the non-multi-pumped tasks are clocked at $f_{\text{base}}$ (i.e., the clock
constraint given by the designer). The maximum multi-pumping factor for task
$v_{i}$ is

$M^{\text{max}}_{i}\coloneqq\min\left(\left\lfloor\frac{f^{\text{max}}_{i}}{f_{\text{base}}}\right\rfloor,N^{\text{\acs{OP}}}_{i}\right).$
(5)

It is worth noting that our task-level multi-pumping _changes only the HLS
directives while using the HLS tool as a black box and without requiring
manual source code restructuring_. The automation of this step will be the
subject of future work.

##  V Multi-pumping workflow

To validate our task-level multi-pumping, we define a workflow from the C/C++
source code to an optimized MCDFG IP block compatible with [11] tools [11], as
shown in Fig. 2. The main steps of the workflow are  (A) _DFG
characterization_to extract the maximum clock frequency and the number of DSP
operations of each task, needed by the later steps,  (B) _multi-pumping factor
optimization_ to select the multi-pumping factor of each task, and  (C) _MCDFG
synthesis_to generate the multi-pumped IP.

###  V-A Dataflow graph characterization

For each task in the DFG $G(V,E)$, we collect the number of DSP operations
($\mathcal{N}=\\{N^{\text{\acs{OP}}}_{i},\forall v_{i}\in V\\}$) from the
reports of the standard SCDFG HLS. We collect the maximum frequency meeting
the timing constraints ($\mathcal{F}=\\{f^{\text{max}}_{i},\forall v_{i}\in
V\\}$) from the post-implementation reports of the SCDFG. We execute the
implementation with a tight clock constraint (e.g.,
$500\text{\,}\mathrm{MHz}\text{/}$) and at the lowest pipeline II, which is
the worst case for the critical cycle (defined in Section III-B). Indeed, when
multi-pumping increases the II, it relaxes the critical cycle, allowing deeper
pipelines and shorter critical paths, thus higher clock frequencies.

We do not extract $\mathcal{F}$ from the earlier-available HLS clock frequency
estimations since they are unreliable [1]. We run the SCDFG implementation
only once, so the overhead is usually acceptable. However, if a fast flow is
required (e.g., in early design phases), we can run only the logic synthesis
step without placement and routing. The timing estimations at the logic
synthesis step are more accurate than the one of the HLS compiler since they
have access to lower-level information. When the estimated maximum frequency
is less than the actual one, we miss chances of saving resources because of
lower multi-pumping factors, as per Eq. 5. On the contrary, if the frequency
is overestimated, the timing fails during implementation.

###  V-B Multi-pumping factor optimization

We select the multi-pumping factors ($\mathcal{M}=\\{M_{i},\forall v_{i}\in
V\\}$) that minimize the DSP utilization. If $v_{i}$ contains operations
mapped to DSP, we set $M_{i}=M^{\text{max}}_{i}$, as defined by Eq. 5.
Otherwise, we do not apply multi-pumping to $v_{i}$.

###  V-C Multi-clock dataflow graph synthesis

Xilinx Vitis HLS cannot synthesize MCDFGs directly since it supports only one
clock domain per the design. However, the dataflow directive generates several
independent modules, one for each task, and interconnects them in a top-level
module. Thus, we run a _split_ HLS, synthesizing each task separately (i.e.,
setting it as the top module) with its clock constraint.

The Xilinx HLS binding algorithm guarantees optimal resource sharing if guided
by resource constraints only. Therefore, we constrain the number of DSPs
according to Eq. 4. For instance, if we multi-pump with a factor $M_{i}$ a
task $v_{i}$ that originally uses $N^{\text{\acs{DSP}}}_{i}$, we constrain its
DSPs to $\left\lceil N^{\text{\acs{DSP}}}_{i}/M_{i}\right\rceil$.

In principle, we could also scale down the memory partitioning factors by
$M_{i}$ to reduce on-chip memory resource usage, namely block random access
memories and registers. However, we cannot apply this optimization to the test
cases considered in Section VI with Xilinx HLS. Indeed, the tool ignores the
coarser partitioning directives and automatically partitions the memories,
presumably to minimize the pipelines II, regardless of the provided
directives. We plan to revisit the issue as a more recent version of the HLS
tool is available.

Finally, we interconnect the tasks synthesized separately using the Vivado
intellectual property integrator (IPI).

The Xilinx HLS tools use FIFOs as inter-task communication channels when data
are produced and consumed in the same order; otherwise, ping-pong buffers. Our
method could support both, but since the Xilinx IPI flow does not provide a
configurable multi-clock ping-pong buffer, we currently only support FIFO
channels using the [11] FIFO generator [23]. FIFOs are configured with
independent clocks for read and write ports when interconnecting tasks
assigned to different clock domains.

##  VI Evaluation

We verify the applicability and the advantages in the PPA space of our task-
level multi-pumping workflow, described in Section V, by applying it to open-
source HLS designs.

Our experiments target the embedded platform Zynq UltraScale+ FPGA SoC hosted
by the Avnet Ultra96v1 board [24]. We use Vitis HLS 2022.2 [2] and Vivado HLS
2019.2 [25] for the synthesis and Vivado 2022.2 [11] for the implementation.

We collect the resource utilization from the post-implementation reports and
the power estimations from the post-implementation static power analysis. We
verify that the throughput (i.e., the number of output samples produced in the
unit of time) matches the theoretical one by measuring the time for $10\,000$
executions in auto-restart mode [2] (to make the time overhead for control
negligible) of the kernels in hardware, using the PYNQ application programming
interfaces [26].

We apply our flow to some open-source HLS designs, including  (a) the _2D
Convolution_ from the Vitis Tutorials [10] already introduced in Section I,
(b) the _Optical Flow_ from the Rosetta suite [8], and  (c) the _ virtual
molecule screening (VMS)_ [27], a drug discovery accelerator.

For each design, we compare the multi-pumped implementations (_M-Pump_) with
the original ones (_Base_) and with the best SCDFG implementations without
source code changes (_S-Pump_). For the _S-Pump_ implementations, we apply our
flow without the generalization to MCDFG. Thus, if task $v_{i}$ is “single-
pumped” by a factor $S_{i}$, we scale by $S_{i}$ its II, as with our original
workflow, and the clock frequency of the whole kernel. The maximum “single-
pumping” factor for each task is lower than the corresponding maximum multi-
pumping factor (defined by Eq. 5) since it is at most

$S^{\text{max}}_{i}\coloneqq\left\lfloor\dfrac{\min\limits_{\forall v_{i}\in
V}f_{i}^{\text{max}}}{f_{\text{base}}}\right\rfloor.$ (6)

Figure 4

figures/plot-dsp_vs_tput

(a) 2D Convolution

figures/plot-dsp_vs_tput

(b) Optical Flow

figures/plot-dsp_vs_tput

(c) VMS

Figure 4: *DSP allocated for a given throughput. The _M-Pump_ designs are
optimized using the proposed task-level multi-pumping technique. The _M-Pump_
designs are Pareto-optimal compared to the _Base_ designs, whose DSP
utilization is constant since they are optimized by tuning the clock frequency
only, and to the _S-Pump_ designs, which are optimized for area by changing
both the II and the global clock frequency of the tasks. The dashed lines
represent the theoretical throughputs achievable with the allocated DSPs,
which are unreachable in practice due to memory bandwidth limitations. The
dots show the design points implemented in hardware.

shows the tradeoffs between the DSP utilization and the throughput obtained by
varying the base clock frequency within the range allowed by the critical path
of the designs. The dashed lines represent computation throughputs that exceed
the memory throughput. Thus, the effective throughputs are, in practice,
clipped to the maximum non-dashed value, corresponding to the maximum memory
throughput.

TABLE I: *PPA of the benchmarks targeting a Zynq UltraScale+ SoC |  |  |  |  |  | LUT |  |  | Power |
---|---|---|---|---|---|---|---|---|---|---
| Throughput |  |  Base clock |  Pump factors | DSP | Logic | Memory | FF | BRAM | Static | Dynamic |  Clock routing
Design |  ($\mathrm{MSa}\text{/}\mathrm{s}$) | Implem. |  ($\mathrm{MHz}\text{/}$) |  ($\mathrm{1}\text{/}$) |  ($\mathrm{\char 37\relax}\text{/}$) |  ($\mathrm{\char 37\relax}\text{/}$) |  ($\mathrm{\char 37\relax}\text{/}$) |  ($\mathrm{\char 37\relax}\text{/}$) |  ($\mathrm{\char 37\relax}\text{/}$) |  ($\mathrm{W}\text{/}$) |  ($\mathrm{W}\text{/}$) |  ($\mathrm{\char 37\relax}\text{/}$)
|  | Base | 165 | – | 64 | 14 | 12 | $9$ | $4$ | 0.3 | 1.8 | 1.0
| $165$ | S-Pump | 330 | 2 | 33 | 15 | 12 | $18$ | $4$ | 0.3 | 2.3 | 1.0
|  | M-Pump | 165 | 3 | 23 | 14 | 12 | $18$ | $4$ | 0.3 | 2.2 | 2.4
|  | Base | 250 | – | 64 | 13 | 12 | $10$ | $4$ | 0.3 | 2.0 | 1.0
2D Convolution [10] | 250 | M-Pump | 250 | 2 | 33 | 15 | 12 | $19$ | $4$ | 0.3 | 2.8 | 2.4
|  | Base | 150 | – | 55 | 36 | 65 | $23$ | $20$ | 0.3 | 2.5 | 1.0
| $150$ | S-Pump | 300 | 2 | 29 | 37 | 65 | $26$ | $20$ | 0.3 | 3.2 | 1.0
|  | M-Pump | 150 | 2, 3 | 21 | 37 | 64 | $27$ | $20$ | 0.3 | 3.0 | 3.8
|  | Base | 175 | – | 55 | 36 | 65 | $23$ | $20$ | 0.3 | 2.5 | 1.0
Optical Flow [8] | 175 | M-Pump | 175 | 2 | 33 | 38 | 65 | $27$ | $20$ | 0.3 | 2.9 | 2.4
|  | Base | 110 | – | 89 | 32 | 23 | $31$ | $67$ | 0.3 | 2.0 | 1.0
| $28$ | S-Pump | 220 | 2 | 44 | 30 | 23 | $31$ | $67$ | 0.3 | 2.4 | 1.0
VMS [27] |  | M-Pump | 110 | 2, 3 | 42 | 32 | 23 | $37$ | $67$ | 0.3 | 2.9 | 3.8

The number of DSPs used by _Base_ designs is independent of the clock
frequency. The plots of the _Pump_ designs are characterized by a step shape,
whose discontinuities correspond to the IIs changes, which only assume integer
values. The _Pump_ solutions provide different tradeoffs in the throughput
versus DSP space, thanks to the tuning of the pipelines’ II. The additional
degree of freedom of the _M-Pump_ implementations (i.e., the task clock
frequency) makes them always Pareto optimal.

Both _M-Pump_ and _S-Pump_ designs degenerate to _Base_ designs (i.e., all the
pumping factors to one and no resource savings) at the highest throughputs
since they need the lowest IIs to reach the best performance. Note that the
_M-Pump_ designs consistently degenerate to _Base_ at throughputs higher than
_S-Pump_ since the multiple clock domains let the multi-pumped tasks run at
the maximum frequency their local critical path allows. Therefore, the _M-
Pump_ designs achieve up to $52\text{\,}\mathrm{\char 37\relax}\text{/}$
higher throughput than _S-Pump_ with the same DSPs in the 2D Convolution test
case. Moreover, with the Optical Flow benchmark, the _M-Pump_ reaches the
maximum effective throughput using $40\text{\,}\mathrm{\char
37\relax}\text{/}$ fewer DSPs than the _Base_.

Table I reports the post-implementation PPA data for the design points marked
with the dots in Fig. 4. We select those points since their throughputs are
the upper extremes of the last steps of _M-Pump_ and _S-Pump_ within the
memory bound.

Comparing the _M-Pump_ designs with the _Base_ ones, the consistent DSP saving
($54\text{\,}\mathrm{\char 37\relax}\text{/}$ on average) implies power and
flip-flops overheads. The additional power ($24\text{\,}\mathrm{\char
37\relax}\text{/}$ on average) is because the multi-pumped tasks are
characterized by greater switching activity due to higher resource reuse and
clock frequencies. The additional FFs ($33\text{\,}\mathrm{\char
37\relax}\text{/}$ on average) are inserted by the HLS tool in the multi-
pumped tasks to build deeper pipelines and reach higher clock frequencies.

As expected [9], the PPA overhead for CDC in _M-Pump_ is negligible. The
overhead for routing multiple clocks is also marginal, as each additional
clock domain allocates only $1.4\text{\,}\mathrm{\char 37\relax}\text{/}$ of
the available clock routing resources.

In general, the _M-Pump_ solutions Pareto dominate the _S-Pump_ ones. In fact,
at the same throughput, they allocate fewer DSPs, similar look-up tables and
FFs, and consume less power. This is because the _M-Pump_ designs take
advantage of the multiple clock domains to increase the clock frequency of the
multi-pumped tasks only, thus reaching higher multi-pumping factors and
avoiding power and FF overheads in the non-multi-pumped tasks. The VMS test
case is the only exception because only a small fraction of its logic runs at
the base clock frequency, while the rest is double or triple-pumped; thus, the
lower-frequency tasks are not enough to balance the power and FF overhead for
the multi-pumped tasks.

##  VII Conclusion

We propose a task-level multi-pumping technique for saving hardware resources
while maintaining the original throughput for HLS dataflow designs for FPGAs.

Given a state-of-the-art single-clock DFG, our approach first generalizes it
to a multi-clock DFG. Secondly, it tunes the tasks’ high-level parameters
(i.e., clock frequency and pipeline II) to multi-pump their functional units.
The overhead for generalization is negligible, thanks to the DFGs structure,
which consists of independent blocks communicating via FIFOs, allowing for
safe CDC, and modern FPGA clock architectures, which seamlessly handle
multiple clock domains even if current HLS tools do not exploit them.

The experimental results reported in Section VI prove that our method opens a
new Pareto front in the performance versus DSPs space, saving up to
$40\text{\,}\mathrm{\char 37\relax}\text{/}$ of resources at maximum
throughput. Moreover, our method does not require any manual architecture
changes from the designer, since it acts only on the high-level parameters of
the tasks and uses the HLS binding algorithm to automatically generate the
resource sharing logic. Finally, the generalization to multi-clock DFGs simply
requires replacing single-clock with multi-clock FIFOs. Therefore, our
technique is well suited for a fully automated HLS optimization pass, which
will be the subject of future work.

## Acknowledgment

This work was partially supported by the Key Digital Technologies Joint
Undertaking under the REBECCA Project with grant agreement number 101097224,
receiving support from the European Union, Greece, Germany, Netherlands,
Spain, Italy, Sweden, Turkey, Lithuania, and Switzerland.

## References

  * [1] Jason Cong, Jason Lau, Gai Liu, Stephen Neuendorffer, Peichen Pan, Kees Vissers and Zhiru Zhang  “FPGA HLS Today: Successes, Challenges, and Opportunities”  In _ACM Trans. Reconfigurable Technol. Syst._ 15.4, 2022, pp. 1–42  DOI: 10.1145/3530775
  * [2] Xilinx  “Vitis High-Level Synthesis User Guide”, 2022  URL: https://docs.xilinx.com/r/en-US/ug1399-vitis-hls
  * [3] Siemens EDA  “Catapult® Synthesis HLS Bluebook”, 2021  URL: https://resources.sw.siemens.com/en-US/e-book-high-level-synthesis-hls-blue-book
  * [4] Intel  “Intel® High Level Synthesis Compiler Pro Edition Reference Manual”, 2022  URL: https://www.intel.com/content/www/us/en/docs/programmable/683349/22-4/pro-edition-reference-manual.html
  * [5] Xilinx  “Vitis Accelerated Libraries”, 2023  URL: https://docs.xilinx.com/r/en-US/Vitis_Libraries
  * [6] Yaman Umuroglu, Nicholas J. Fraser, Giulio Gambardella, Michaela Blott, Philip Leong, Magnus Jahre and Kees Vissers  “FINN: A Framework for Fast, Scalable Binarized Neural Network Inference”  In _Int. Symp. Field-Prog. Gate Arrays_ , 2017, pp. 65–74  DOI: 10.1145/3020078.3021744
  * [7] Johannes de Fine Licht, Grzegorz Kwasniewski and Torsten Hoefler  “Flexible Communication Avoiding Matrix Multiplication on FPGA with High-Level Synthesis”  In _Int. Symp. Field-Prog. Gate Arrays_ , 2020, pp. 244–254  DOI: 10.1145/3373087.3375296
  * [8] Yuan Zhou, Udit Gupta, Steve Dai, Ritchie Zhao, Nitish Srivastava, Hanchen Jin, Joseph Featherston, Yi-Hsiang Lai, Gai Liu, Gustavo Angarita Velasquez, Wenping Wang and Zhiru Zhang  “Rosetta: A Realistic High-Level Synthesis Benchmark Suite for Software-Programmable FPGAs”  In _Int. Symp. Field-Prog. Gate Arrays_ , 2018, pp. 269–278  DOI: 10.1145/3174243.3174255
  * [9] Xilinx  “Resource Utilization for FIFO Generator v13.2”, 2022  URL: https://www.xilinx.com/cgi-bin/docs/ndoc?t=ip+ru;d=fifo-generator.html
  * [10] Xilinx  “Design and Analysis of Hardware Kernel Module for 2-D Video Convolution Filter”, 2022  URL: https://xilinx.github.io/Vitis-Tutorials/2022-1/build/html/docs/Hardware_Acceleration/Design_Tutorials/01-convolution-tutorial/README.html
  * [11] Xilinx  “Vivado Design Suite User Guide: Release Notes, Installation, and Licensing (UG973)”, 2022  URL: https://docs.xilinx.com/r/en-US/ug973-vivado-release-notes-install-license/Release-Notes
  * [12] Atefeh Sohrabizadeh, Cody Hao Yu, Min Gao and Jason Cong  “AutoDSE: Enabling Software Programmers to Design Efficient FPGA Accelerators”  In _ACM Trans. Des. Autom. Electron. Syst._ 27.4, 2022, pp. 1–27  DOI: 10.1145/3494534
  * [13] Qi Sun, Tinghuan Chen, Siting Liu, Jin Miao, Jianli Chen, Hao Yu and Bei Yu  “Correlated Multi-objective Multi-fidelity Optimization for HLS Directives Design”  In _Design Autom. Test Eur. Conf. Exhib._ , 2021, pp. 46–51  DOI: 10.23919/DATE51398.2021.9474241
  * [14] Xilinx  “Merlin Compiler”, 2022  URL: https://github.com/Xilinx/merlin-compiler
  * [15] Charles Lo and Paul Chow  “Model-based optimization of High Level Synthesis directives”  In _Int. Conf. Field-Prog. Logic Appl._ , 2016, pp. 1–10  DOI: 10.1109/FPL.2016.7577358
  * [16] Tingyuan Liang, Jieru Zhao, Liang Feng, Sharad Sinha and Wei Zhang  “Hi-ClockFlow: Multi-Clock Dataflow Automation and Throughput Optimization in High-Level Synthesis”  In _Int. Conf. Comput.-Aided Des._ , 2019, pp. 1–6  DOI: 10.1109/ICCAD45719.2019.8942136
  * [17] Ghizlane Lhairech-Lebreton, Philippe Coussy and Eric Martin  “Hierarchical and Multiple-Clock Domain High-Level Synthesis for Low-Power Design on FPGA”  In _Int. Conf. Field-Prog. Logic Appl._ , 2010, pp. 464–468  DOI: 10.1109/FPL.2010.94
  * [18] Andrew Canis, Jason H. Anderson and Stephen D. Brown  “Multi-pumping for resource reduction in FPGA high-level synthesis”  In _Design Autom. Test Eur. Conf. Exhib._ , 2013, pp. 194–197  DOI: 10.7873/DATE.2013.053
  * [19] Bajaj Ronak and Suhaib A. Fahmy  “Multipumping Flexible DSP Blocks for Resource Reduction on Xilinx FPGAs”  In _IEEE Trans. Comp.-Aided Des. Integ. Cir. Sys._ 36.9, 2017, pp. 1471–1482  DOI: 10.1109/TCAD.2016.2629421
  * [20] Omar Ragheb and Jason H. Anderson  “High-Level Synthesis of FPGA Circuits with Multiple Clock Domains”  In _Int. Symp. Field-Prog. Custom Comput. Mach._ , 2018, pp. 109–116  DOI: 10.1109/FCCM.2018.00026
  * [21] “double pumping DSP”, 2019  URL: https://support.xilinx.com/s/question/0D52E00006iHilaSAC/double-pumping-dsp
  * [22] Vicki H Allan, Reese B Jones, Randall M Lee and Stephen J Allan  “Software pipelining”  In _ACM Comput. Surv._ 27.3, 1995, pp. 367–432  DOI: 10.1145/212094.212131
  * [23] Xilinx  “FIFO Generator”, 2022  URL: https://www.xilinx.com/products/intellectual-property/fifo_generator.html
  * [24] Avnet  “Ultra96 Hardware User’s Guide”, 2018  URL: https://www.avnet.com/opasdata/d120001/medias/docus/187/Ultra96-HW-User-Guide-rev-1-0-V0_9_preliminary.pdf
  * [25] Xilinx  “Vivado Design Suite User Guide High-Level Synthesis”, 2020  URL: https://docs.xilinx.com/r/en-US/ug1399-vitis-hls
  * [26] Xilinx  “PYNQ: Python productivity for Adaptive Computing platforms”, 2022  URL: https://pynq.readthedocs.io
  * [27] Tom Vander Aa, Tom Haber, Thomas J. Ashby, Roel Wuyts and Wilfried Verachtert  “Virtual Screening on FPGA: Performance and Energy versus Effort”, 2022  arXiv:2210.10386

  *[HLS]: high-level synthesis
  *[RTL]: register-transfer level
  *[DFGs]: dataflow graph
  *[II]: initiation interval
  *[DSP]: digital signal processor
  *[DSPs]: digital signal processor
  *[FPGA]: field-programmable gate array
  *[QoR]: quality of results
  *[DFG]: dataflow graph
  *[FIFO]: first-in-first-out
  *[SCDFGs]: single-clock dataflow graph
  *[CDC]: clock domain crossing
  *[FIFOs]: first-in-first-out
  *[MCDFG]: multi-clock dataflow graph
  *[SCDFG]: single-clock dataflow graph
  *[MAC]: multiply and accumulate
  *[MACs]: multiply and accumulate
  *[IP]: intellectual property
  *[PPA]: power, performance, and area
  *[DSE]: design-space exploration
  *[DSEs]: design-space exploration
  *[MCDFGs]: multi-clock dataflow graph
  *[DRAM]: dynamic random access memory
  *[DDG]: data dependence graph
  *[OPs]: operation
  *[FUs]: functional unit
  *[ HLS]: high-level synthesis
  *[IPI]: intellectual property integrator
  *[SoC]: system-on-chip
  *[VMS]: virtual molecule screening
  *[LUT]: look-up table
  *[FF]: flip-flop
  *[BRAM]: block random access memory
  *[IIs]: initiation interval
  *[FFs]: flip-flop
  *[FPGAs]: field-programmable gate array