# Characterisation of resistive MPGDs with 2D readout

L. Scharenberg,11footnotetext: Corresponding author. F. Brunbauer  H.
Danielson  Z. Fang  K. J. Flöthner  F. Garcia  D. Janssens  M. Lisowska  J.
Liu  Y. Lyu  B. Mehl  H. Muller  R. de Oliveira  E. Oliveri  G. Orlandini  D.
Pfeiffer  O. Pizzirusso  L. Ropelewski  J. Samarati  M. Shao  A. Teixeira  M.
Van Stenis  R. Veenhof  Z. Zhang  and Y. Zhou

###### Abstract

Micro-Pattern Gaseous Detectors (MPGDs) with resistive anode planes provide
intrinsic discharge robustness while maintaining good spatial and time
resolutions. Typically read out with 1D strips or pad structures, here the
characterisation results of resistive anode plane MPGDs with 2D strip readout
are presented. A µRWELL prototype is investigated in view of its use as a
reference tracking detector in a future gaseous beam telescope. A MicroMegas
prototype with a fine-pitch mesh (730 line-pairs-per-inch) is investigated,
both for comparison and to profit from the better field uniformity and thus
the ability to operate the detector more stable at high gains. Furthermore,
the measurements are another application of the RD51 VMM3a/SRS electronics.

##  1 Introduction

In recent years, various experiments and detector R&D lines started the use of
Micro-Pattern Gaseous Detectors (MPGDs) with resistive elements, mainly
because of their discharge robustness [1, 2, 3, 4]. Additionally, parameters
such as the signal induction, i.e. the spread of the charge over a given
number of readout channels, and rate-capability can be tuned to the desired
values of the experiment by adjusting the design of resistive elements. This
is most prominently employed in MPGDs with a single amplification stage, such
as MicroMegas (MM) [5] and µRWELL [6].

In this paper, the results of a characterisation of these two technologies are
shown. The studies are conducted in view of various R&D aspects. At first,
there is the technology of the readout anode, a 2D strip structure underneath
a single layer of resistive material. So far, most resistive MPGDs employ
either 1D strip structures underneath the resistive layer, 1D resistive strips
or resistive pads. The second aspect is specific to the µRWELL detector that
has been investigated. It is a prototype of what is supposed to be used as
reference detectors for particle trajectory reconstruction in a new beam
telescope of the DRD1 collaboration [7, 8] — the preceding RD51 collaboration
[9, 10] provided for its joined test beam campaigns at the H4 beam line of
CERN’s SPS two beam telescopes, one based on MicroMegas and one based on
triple-GEM detectors [11]. While a previous prototype with 2D X-Y strip
readout showed good performance [12], it showed an unequal signal sharing
between the top strip layer and the perpendicular bottom strip layer. For the
new prototype which was investigated for the here presented studies, this
imbalance has been addressed. The third point is specific to the investigated
MicroMegas detector. The particular detector is equipped with a fine mesh in
the amplification stage with 730 line-pairs-per-inch (LPI), corresponding to a
mesh cell pitch of $35\text{\,}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$. The
motivation for investigating a mesh structure like this is a more uniform
electric field and thus the possibility to operate the detector at higher
gains. While the stability measurements are part of a separate study — here
only the results on the spatial resolution, time resolution and charge
behaviour are presented — it can be noted that the MicroMegas detector could
be operated at gains more than twice as high as the µRWELL detector. Another
more general aspect is the application of the new VMM3a/SRS front-end
electronics [13] of the RD51 collaboration to a wider range of detectors.

##  2 Experimental methods

###  2.1 Devices under test

Both detectors (sketched in Fig. 1) have an active area of
$$10$\times$10\text{\,}\mathrm{c}\mathrm{m}^{2}$$ with a 2D X-Y-strip readout
with 256 strips in each direction.

(a) MicroMegas

(b) µRWELL

Figure 1: Sketched cross-section of the MicroMegas (a) and the µRWELL (b)
amplification stages, as used in the here presented measurements.

The strip pitch is $400\text{\,}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$. In
both detectors, the anode is a layer of Diamond-Like Carbon (DLC) with similar
surface resistivities —
$40\text{\,}\mathrm{M}\mathrm{\SIUnitSymbolOhm}\mathrm{/}\mathrm{s}\mathrm{q}$
for the µRWELL and
$37\text{\,}\mathrm{M}\mathrm{\SIUnitSymbolOhm}\mathrm{/}\mathrm{s}\mathrm{q}$
for the MicroMegas. The drift region of each detector had a width of
$3\text{\,}\mathrm{m}\mathrm{m}$. Although both detectors were operated with a
negative high voltage on the cathode, the MicroMegas detector was operated
with a grounded mesh and a positive high voltage on the DLC anode, while the
µRWELL was operated with a negative high voltage at the top layer of the Well
structure and a grounded DLC anode.

When performing gain scans, the drift field was kept constant at around
$2.4\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{c}\mathrm{m}$ for the
µRWELL, which corresponds to $724\text{\,}\mathrm{V}$ difference between drift
cathode and Well, and
$0.9\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{c}\mathrm{m}$ for the
MicroMegas detector, which corresponds to $275\text{\,}\mathrm{V}$ between
cathode and mesh. The amplification voltages were varied from $460$ to
$570\text{\,}\mathrm{V}$ for the µRWELL and $580$ to $700\text{\,}\mathrm{V}$
for the MicroMegas — at higher voltages, the detectors started to show
instabilities in operation due to discharges leading to high voltage trips.
When performing drift scans, the amplification voltage was kept constant at
$680\text{\,}\mathrm{V}$ for the MicroMegas, which corresponds to an effective
gain of around $40\,000$ at
$0.9\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{c}\mathrm{m}$ drift field,
and at $540\text{\,}\mathrm{V}$ for the µRWELL, which corresponds to a gain of
around $15\,000$ at
$2.4\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{c}\mathrm{m}$ drift field.

###  2.2 Beam telescope and readout electronics

Both detectors have been characterised as Devices Under Test (DUTs) with the
RD51 VMM3a/SRS beam telescope [11]. It consists of three COMPASS-like triple-
GEM detectors [14] with an active area of
$$10$\times$10\text{\,}\mathrm{c}\mathrm{m}^{2}$$ almost equally spaced with a
total lever arm of around $1\text{\,}\mathrm{m}$ to provide the position
information. In addition, it contains three scintillators with Photo-
Multiplier Tubes (PMTs) connected to a NIM coincidence unit as a time
reference. All the MPGDs in the telescope have been operated with the same gas
mixture of Ar/CO2 (70/30 %).

For the readout of the detectors, including the output of the NIM coincidence
unit, the ATLAS/BNL VMM3a front-end ASIC [15] in its integration [16] of the
RD51 Scalable Readout System (SRS) [17] has been used. It provides the
acquired charge per channel (10-bit ADC, effectively 7-bit) in a continuous,
multi-channel self-triggered readout mode with around
$1\text{\,}\mathrm{n}\mathrm{s}$ time resolution and a MHz rate-capability.
The adjustable analogue front-end parameters of the VMM3a have been set to
$200\text{\,}\mathrm{n}\mathrm{s}$ peaking time,
$9\text{\,}\mathrm{m}\mathrm{V}\mathrm{/}\mathrm{f}\mathrm{C}$ electronics
gain and around $10\,000$ electrons threshold per channel for both the DUTs
and the reference detectors. The threshold level might seem high, but it
should be considered that the VMM3a is operated in its self-triggered readout,
meaning that each signal above the Threshold Level (THL) will be processed and
become part of the data stream, i.e. a THL that is set too low will results in
a lot of external pick-up noise being part of the raw data. Furthermore, in
terms of the dynamic range of the VMM3a, the THL is still a low value of less
than $2\text{\,}\mathrm{\char 37\relax}$ per channel.

##  3 Charge behaviour

With the previous µRWELL prototype showing an imbalance of the charge
collection between the top and the bottom readout strips [12], the first point
of investigation is the charge sharing and collection behaviour of the
detectors. For this, the charge distribution (Landau distribution) is
generated from all recorded interactions that can be assigned to a
reconstructed particle trajectory from the beam telescope. Then the mean value
of this measured distribution is taken and plotted depending on the effective
gain of the detector (Fig. 2(a)).

(a) Gain dependence

(b) Drift field dependence

Figure 2: Average measured charge, i.e. the calculated mean of the energy loss
distribution after amplification, depending (a) on the effective gain of the
two investigated detectors or (b) on the drift field, in each read out
detector plane.

It can be seen that both strip planes collect almost the same amount of
charge, resulting in a charge-sharing ratio between the top and bottom strips
of each detectors close to one. Another observation is that the mean charge
measured with the Landau distribution increases linearly with the gain for the
µRWELL detector, while for the MicroMegas detector, the increase starts to
flatten out at higher gains. This is due to the ADC saturation of individual
readout channels which at higher gains is more likely to occur.

Another observation is that the total collected charge for the µRWELL is
significantly less than for the MicroMegas detector, despite the same
effective gain. This is due to the signal induction from the anode to the
readout strips. While the effective detector gain was determined from the
current measured on the resistive anode layer, the front-end electronics only
measures the induced current in the capacitively coupled readout strips. This
is important to note, as the gain at which a discharge might occur in the
detector is the one determined via the signal current on the anode plane.

This explanation is strengthened by the behaviour of the cluster size (number
of channels above the THL), with the mean cluster size being plotted against
the effective detector gain (Fig. 3(a)).

(a) Average cluster size

(b) Detector efficiency

Figure 3: In (a), the mean of the cluster size distribution (distribution of
the number of channels above the THL for each recorded interaction) is shown,
depending on the effective gain of the detectors. In (b), the detector
efficiency as defined in Eq. (3.1) is shown, depending on the effective gain.

It can be seen that the cluster size scales accordingly, with the bottom
strips of the µRWELL collecting slightly more charge, also having a slightly
larger cluster size. And an almost doubled cluster size for the MicroMegas
detector at similar gains, leading to almost twice the amount of measured
charge. This behaviour is also reflected in the detector efficiency (Fig.
3(b)), which is defined as

$\displaystyle\epsilon_{\mathrm{det}}=\frac{N_{\mathrm{det}}}{N_{\mathrm{tracks}}}\
.$ (3.1)

Here, $N_{\mathrm{tracks}}$ is the number of particle interactions and
trajectories reconstructed with the reference tracking detectors and
$N_{\mathrm{det}}$ is the number of particle interactions that have been
recorded in the DUT and that can be assigned to an existing trajectory. Due to
the difference in signal induction between the investigated µRWELL and
MicroMegas detectors, reflected by the smaller cluster size and measured
signal amplitude for the µRWELL, higher detector gains are needed to reach
comparable cluster sizes and induced signal charges and thus comparable
efficiencies.

In addition to the dependence on the effective gain of the detector, also the
dependence of the measured charge depending on the electric drift field was
studied, representing the charge collection behaviour by the amplification
stage. The results are shown in Fig. 2(b). Both detectors show the expected
characteristic behaviour, with the charge collection of the µRWELL having a
much broader peak at higher electric fields (e.g. as shown in [6]), while the
MicroMegas shows a more well define peak at lower electric fields (e.g. as
shown in [18]).

##  4 Spatial resolution

###  4.1 Basic results

The spatial resolution of the DUTs is extracted from the width of the residual
distributions that are generated from the difference $\Delta
x=x_{\mathrm{ref}}-x^{\prime}$ between the reference particle position
$x_{\mathrm{ref}}$ that is provided by the reconstructed trajectory from the
reference tracking detectors and the interaction point $x^{\prime}$
reconstructed in the DUTs. To determine the width, two overlapping Gaussian
functions are fitted to the residual distribution, with the mean value being
identical, but a weight factor $w$ to account for the different scales and
different standard deviations $\sigma$ to account for the core and the tails
of the residual distribution. The final width is thus defined as

$\displaystyle\sigma_{\Delta
x}^{2}=w\sigma_{\mathrm{core}}^{2}+(1-w)\sigma_{\mathrm{tail}}^{2}\ .$ (4.1)

The spatial resolution is then obtained by quadratically subtracting the
contribution from the uncertainty on the track reconstruction, as described in
[19, 18, 20].

Using the Centre-Of-Gravity (COG) to determine the position of the particle
interaction within the detector, the gain dependence of the spatial resolution
as shown in Fig. 4(a) is obtained.

(a) Gain dependence

(b) Drift field dependence

Figure 4: Dependence of the spatial resolution of the two DUTs on (a) their
effective detector gain and on (b) the electric drift field, determined for
both readout planes individually.

For both detectors, the spatial resolution improves with increased detector
gain. Taking the cluster size behaviour (Fig. 3(a)) into account, this is
related to more charge information being available and distributed over more
than a single readout strip. This also explains the better spatial resolution
of the MicroMegas compared to the µRWELL, as it has a larger charge collection
and cluster size. A similar dependence can be observed when plotting the drift
behaviour (Fig. 4(b)). The spatial resolution is inversely proportional to the
charge collection depending on the electric drift field (Fig. 2(b)), i.e. the
more charge collected, the better the spatial resolution.

At very high gains (Fig. 4(a)), the spatial resolution starts to decrease for
the MicroMegas detector. This is due to the saturated readout channel that
also caused the flattening of the measured charge-gain-dependence (Fig. 2(a)).
The position reconstruction by COG loses accuracy when the relative amount of
charge is not correctly represented anymore because the corresponding readout
channel is in saturation.

Another observation is that the spatial resolution for the µRWELL is almost
the same for the two readout planes, as it is expected from equal charge
sharing, while for the MicroMegas the behaviour between the top and bottom
strips deviates. This was found to be the result of the so-called ‘readout
modulation’, as illustrated by Fig. 5.

(a) High gain, approximately $40\,000$

(b) Low gain, approximately $8000$

Figure 5: Distribution of the reconstructed interaction points at high
granularity for the MicroMegas detector at high detector gains (a) and low
detector gains (b).

It shows the distribution of the reconstructed interaction points with high
granularity. This makes a peak structure visible which originates from the
readout pattern modulated into the position distribution that is expected to
be uniform due to the detector’s uniform irradiation. Due to this modulation
effect by the discrete readout structure in combination with a threshold-based
zero-suppressed readout electronics, the interaction points are more likely to
be reconstructed to the central strip for odd-strip-count clusters and in-
between the two central strips for even-strip-count clusters. For a more
detailed description of this effect, it is referred to [21], while it should
be noted that the effect has already been observed with Multi-Wire
Proportional Chambers [22]. This behaviour affects the accuracy of the
position determination, with a stronger effect leading to a worse spatial
resolution. This is what can be conducted from Fig. 5, where the modulation
effect is stronger for the top strips than for the bottom strips. It seems to
be an intrinsic behaviour of the detector as it is also observed in laboratory
measurements using an 55Fe source. A possible explanation that was found for
this behaviour is the way the signal induction. With the top strips being much
thinner than the bottom strips, the distribution of the induced signal charge
changes compared to the bottom strips, with more charge being in the cluster’s
central strip for the top strips and less charge being acquired in the outer
strips.

###  4.2 Improving the spatial resolution

While so far, the results have been obtained with COG to calculate the
position, previous studies [21, 11] showed that a different weighting of the
charge in the COG formula

$\displaystyle x^{\prime}=\frac{\sum_{i}Q_{i}^{n}x_{i}}{\sum_{i}Q_{i}^{n}}$
(4.2)

with $i$ the index of the readout strip with a signal above THL and $n$ the
weighting factor reduces the modulation effect and thus improves the spatial
resolution. This is because of low amplitude signals in the tails of the
charge distribution that go above the THL on one side of the charge
distribution but not on the other one. Thus, a certain fraction of charge
information to reconstruct the position is lost and the reconstruction of the
interaction point is forced towards the signal above the THL. To reduce the
weight of the charge distribution’s tails and thus mitigate the bias in the
calculation of the interaction point, $n>1$ can be selected. In previous
studies with triple-GEM detectors [11], $n=2$ was used due to its simplicity
and its proximity to the optimal solution obtained from simulation studies
[23, 24]. Thus, it was investigated, if this $Q^{2}$ weighting can also be
used to improve the spatial resolution of µRWELL and resistive MicroMegas
detectors.

In addition, an orthogonal approach was investigated, making use of a hardware
feature of the VMM3a front-end ASIC, its Neighbouring-Logic (NL). By default,
the neighbouring-logic is turned off, but when enabled, the NL triggers the
acquisition of an induced signal with an amplitude below the THL, if the
neighbouring channel has a signal that surpasses the THL. This allows to
obtain more charge information and thus to improve the reconstructed position.

The results of these two methods, including their combination, are shown in
comparison with the COG reconstruction in Fig. 6.

(a) µRWELL

(b) MicroMegas

Figure 6: Dependence of the spatial resolution on the effective detector gain
for the two different methods to determine the interaction point, with and
without NL enabled. Here only the results of the bottom strips are shown, but
the observed behaviour is the same for the top strops.

It can be seen that the $Q^{2}$ weighting has a large impact and significantly
improves the spatial resolution, in both detector cases. On the other hand,
the effect of the NL is only visible at low gains, i.e. at low efficiencies
and low signal-to-threshold ratios, where the relative amount of collected
charge on the total cluster charge is large, with a large fraction of the
induced signal charge being still below the THL. At higher gains, this effect
gets reduced, which is also reflected in the larger cluster size, i.e. on more
channels a signal above THL was acquired. Thus, the tails of the measured
charge distribution contain less induced signal charge and the probability of
acquiring electronics noise increases. As a result, the spatial resolution
decreases. These two outcomes are exactly in line with what has been observed
in previous studies [11, 20].

##  5 Time resolution

As the last part of the characterisation studies, the time resolution of the
two detectors is investigated. For this, the interaction time of the particle
measured in the DUTs is compared with the reference time measured with the
scintillator/PMT/NIM-coincidence-unit combination, both acquired with
VMM3a/SRS. The reference time is provided as a constant amplitude signal — due
to the NIM output of the coincidence unit — on a single channel of the VMM3a.
The measured interaction time in the DUTs is defined as

$\displaystyle t^{\prime}=\frac{1}{2}(t_{\mathrm{top}}+t_{\mathrm{bottom}})\
.$ (5.1)

The interaction timestamps for each plane, $t_{\mathrm{top}}$ and
$t_{\mathrm{bottom}}$, correspond to the time of the signal within each
cluster with the largest peak amplitude.

With both timestamps found, their difference $\Delta
t=t_{\mathrm{ref}}-t^{\prime}$ is calculated. Fitted to the resulting
distribution is a single Gaussian function, with its width $\sigma_{\Delta
t}$. The final time resolution is then this width, from which the time
resolution contributions of the VMM3a (around $2\text{\,}\mathrm{n}\mathrm{s}$
at the used peaking time of $200\text{\,}\mathrm{n}\mathrm{s}$) and the
scintillator/PMT/NIM-logic (around $1.5\text{\,}\mathrm{n}\mathrm{s}$) are
quadratically subtracted. The obtained time resolutions have been plotted
against the electric drift field (Fig. 7(a)) and the electron drift velocity
(Fig. 7(b)) which was calculated through Magboltz [25].

(a) Field dependence

(b) Velocity dependence

Figure 7: Measured time resolution for the two different detectors, depending
on the electric drift field (a) and the electron drift velocity (b). For
comparison purposes, also the results from measurements with a small-pad
resistive MicroMegas detector from the RHUM project have been added [3].

Added for reference, are data from measurements with a small-pad resistive
MicroMegas detector from the RHUM project [3]. All data points show the same
trend of an improved time resolution with increasing drift velocity. Two
aspects should be noted though in regard to the used gas mixtures. The RHUM
data have been obtained with a mixture of Ar/CO2/iC4H10 (93/5/2 %) for the low
drift velocities and Ar/CF4/iC4H10 (88/10/2 %) for the high drift velocities
($>$6\text{\,}\mathrm{c}\mathrm{m}\mathrm{/}\mathrm{\SIUnitSymbolMicro}\mathrm{s}$$).
In the case of the µRWELL, which has been filled with Ar/CO2 (70/30 %), the
drift velocity does not change significantly at fields above
$2.5\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{c}\mathrm{m}$ — a linear
increase of around
$1\text{\,}\mathrm{c}\mathrm{m}\mathrm{/}\mathrm{\SIUnitSymbolMicro}\mathrm{s}$
over a range of $2.5\text{\,}\mathrm{k}\mathrm{V}$. This explains the observed
saturation behaviour of the time resolution. In the case of the MicroMegas
detector, this saturation behaviour could not be observed, because of the
high-voltage power supply, which did not allow it to go to larger drift
voltages.

In addition to the observation that the time resolution in the three detectors
follows the same trend and that the results are compatible with each other,
two other points are shown by the measurements. Especially highlighted by the
results from the resistive plane MicroMegas detector, it becomes clear that
the working point for the best detector performance in terms of charge
collection and spatial resolution (here at
$0.9\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{c}\mathrm{m}$) is not
necessarily the working point for the best time resolution (here
$>$3\text{\,}\mathrm{k}\mathrm{V}\mathrm{/}\mathrm{c}\mathrm{m}$$). Secondly,
due to the capabilities of VMM3a/SRS, both types of detectors, as well as the
reference timing detectors, could all be read out by the same front-end
electronics, with the corresponding data being contained in a single data
stream. This shows the versatility of the readout system, but also how it
simplifies the data-taking and analysis process as it is not necessary to rely
on additional high-precision timing electronics with an additional data stream
that requires additional effort in the offline data analysis.

##  6 Conclusion and outlook

In this paper, the results from characterising two single-stage resistive
plane MPGDs, a µRWELL detector and a MicroMegas detector, with 2D strip
readout have been presented. Due to the capabilities of the new VMM3a/SRS
readout electronics, it was possible to study simultaneously the charge
behaviour, the spatial resolution and the time resolution of these two
detectors.

One immediate observation was the difference in the amplitude of the induced
signal measured by the front-end electronics between the two detectors,
despite them being operated at the same gain. This shows the importance of a
good signal coupling between the anode and readout structure as otherwise, the
detector efficiency can be still not sufficient although the detectors might
be operated close to the breakdown voltages. Both detectors showed good
performance though.

The µRWELL detector was investigated as a prototype detector for a future beam
telescope of the DRD1 collaboration. With time resolutions of
$10\text{\,}\mathrm{n}\mathrm{s}$, spatial resolutions of better than
$70\text{\,}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$ in Ar/CO2 (70/30 %) and
equal charge sharing between the two readout strip planes, the detector
fulfils the requirements to be used in the new telescope. The MicroMegas
detector was built with a 730 line-pairs-per-inch (LPI) mesh, corresponding to
a mesh cell pitch of $35\text{\,}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$, in
order to increase the operation stability. Although the stability studies will
be presented separately, the detector showed a good performance — e.g. spatial
resolutions of around $50\text{\,}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$ —
while being operated stably at gains of more than $50\,000$. In addition, the
results demonstrated that the optimal working point to achieve the best
spatial resolution and charge collection, is not necessarily the best working
point to achieve the best time resolution.

## Acknowledgments

This work has been supported by the CERN EP R&D Strategic Programme on
Technologies for Future Experiments (https://ep-rnd.web.cern.ch/).

The authors would like to thank Jona Bortfeldt (LMU Munich) for providing the
anamicom software (https://gitlab.physik.uni-
muenchen.de/Jonathan.Bortfeldt/anamicom) to reconstruct the particle
trajectories.

The authors would also like to thank Maria-Teresa Camerlingo (INFN Bari),
Paolo Iengo (INFN Napoli), Mauro Iodice (INFN Roma Tre) and Marco Sessa (INFN
Roma Tor Vergata) for the exchange and discussions on the MicroMegas results
and in particular for providing the RHUM reference data shown in Fig. 7.

## References

  * [1] T. Kawamoto et al. (The ATLAS Collaboration), _New Small Wheel Technical Design Report_ , CERN-LHCC-2013-006, ATLAS-TDR-020 (2013).
URL: https://cds.cern.ch/record/1552862.

  * [2] D. Attié et al., _Characterization of resistive Micromegas detectors for the upgrade of the T2K Near Detector Time Projection Chambers_ , _Nucl. Instrum. Methods Phys. Res. A_ 1025 (2022) 166109.
URL: https://doi.org/10.1016/j.nima.2021.166109.

  * [3] M. Iodice et al., _Towards large size pixelized Micromegas for operation beyond 1 MHz/cm 2_, _J. Instrum._ 18 (2023) C06029.
  * [4] G. Bencivenni et al., _The micro-RWELL detector for the phase-2 upgrade of the LHCb muon system_ , _Nucl. Instrum. Methods Phys. Res. A_ 1049 (2023) 168075.
URL: https://doi.org/10.1016/j.nima.2023.168075

  * [5] Y. Giomataris et al., _MICROMEGAS: a high-granularity position-sensitive gaseous detector for high particle-flux environments_ , _Nucl. Instrum. Methods Phys. Res. A_ 376 (1996) 29–35.
URL: https://doi.org/10.1016/0168-9002(96)00175-1

  * [6] G. Bencivenni et al., _The micro-Resistive WELL detector: a compact spark-protected single amplification-stage MPGD_ , _J. Instrum._ 10 (2015) P02008.
URL: https://doi.org/10.1088/1748-0221/10/02/P02008

  * [7] DRD1 R&D Collaboration, _Development of Gaseous Detectors Technologies_.
URL: https://drd1.web.cern.ch/

  * [8] A. Colaleo et al., _DRD1 Extended R &D Proposal_, CERN-DRDC-2024-003, DRDC-P-DRD1 (2024).
URL: https://cds.cern.ch/record/2885937.

  * [9] RD51 Collaboration, _Development of Micro-Pattern Gas Detectors Technologies_.
URL: https://rd51-public.web.cern.ch/

  * [10] M. Titov, L. Ropelewski, _Micro-Pattern Gaseous Detector Technologies and RD51 Collaboration_ , _Mod. Phys. Lett. A_ 28 (2013) 1340022.
URL: https://doi.org/10.1142/S0217732313400221

  * [11] L. Scharenberg et al., _Performance of the new RD51 VMM3a/SRS beam telescope — studying MPGDs simultaneously in energy, space and time at high rates_ , _J. Instrum._ 18 (2023) C05017.
URL: https://doi.org/10.1088/1748-0221/18/05/C05017

  * [12] Y. Zhou et al., _Fabrication and performance of a µRWELL detector with Diamond-Like Carbon resistive electrode and two-dimensional readout_ , _Nucl. Instrum. Methods Phys. Res. A_ 927 (2019) 31–36.
URL: https://doi.org/10.1016/j.nima.2019.01.036

  * [13] L. Scharenberg et al., _Development of a high-rate scalable readout system for gaseous detectors_ , _J. Instrum._ 17 (2022) C12014.
URL: https://doi.org/10.1088/1748-0221/17/12/C12014

  * [14] C. Altunbas et al., _Construction, test and commissioning of the triple-GEM tracking detector for COMPASS_ , _Nucl. Instrum. Methods Phys. Res. A_ 490 (2002) 177–203.
DOI: https://doi.org/10.1016/S0168-9002(02)00910-5

  * [15] G. de Geronimo et al., _The VMM3a ASIC_ , _IEEE Trans. Nucl. Sci._ 69 (2022) 976.
DOI: https://doi.org/10.1109/TNS.2022.3155818

  * [16] M. Lupberger et al., _Implementation of the VMM ASIC in the Scalable Readout System_ , _Nucl. Instrum. Methods Phys. Res. A_ 903 (2018) 91–98.
DOI: https://doi.org/10.1016/j.nima.2018.06.046

  * [17] S. Martoiu et al., _Development of the scalable readout system for micro-pattern gas detectors and other applications_ , _J. Instrum._ 8 (2013) C03015.
DOI: https://doi.org/10.1088/1748-0221/8/03/C03015

  * [18] J. Bortfeldt, _Development of Floating-Strip Micromegas Detectors_ , PhD Thesis, Ludwig-Maximilians-Universität München (2014).
DOI: https://doi.org/10.5282/edoc.16972

  * [19] S. Horvat, _Study of the Higgs Discovery Potential in the Process $\mathit{pp}\rightarrow\mathit{H}\rightarrow 4\mu$_, PhD Thesis, University of Zagreb (2005).
DOI: https://cds.cern.ch/record/858509

  * [20] L. Scharenberg, _Next-Generation Electronics for the Read-Out of Micro-Pattern Gaseous Detectors_ , PhD Thesis, Rheinische Friedrich-Wilhelms-Universität Bonn (2023).
DOI: https://hdl.handle.net/20.500.11811/10776

  * [21] L. Scharenberg et al., _X-ray imaging with gaseous detectors using the VMM3a and the SRS_ , _Nucl. Instrum. Methods Phys. Res. A_ 1011 (2021) 165576.
DOI: https://doi.org/10.1016/j.nima.2021.165576

  * [22] F. Piuz et al., _Evaluation of systematic errors in the avalanche localization along the wire with cathode strips read-out MWPC_ , _Nucl. Instrum. Methods Phys. Res._ 196 (1982) 451–462.
DOI: https://doi.org/10.1016/0029-554X(82)90113-6

  * [23] H. Pulkkinen, _Basic properties of Gas Electron Multipliers and different methods to determine the cluster characteristics_ , Student Report Tfy-3.5111, Aalto University (2013).
  * [24] D. Janssens, _Resistive Electrodes and Particle Detectors: Modeling and Measurements of Novel Detector Structures_ , PhD Thesis, Vrije Universiteit Brussels (2024).
  * [25] S. F. Biagi, _Magboltz - transport of electrons in gas mixtures_
URL: https://magboltz.web.cern.ch/magboltz/


#  Heterogeneous Dual-Core Overlay Processor for Light-Weight CNNs

Tiandong Zhao, Yunxuan Yu, Kun Wang, Lei He  Electrical and Computer
Engineering Department
University of California, Los Angeles

###### Abstract

Light-weight convolutional neural networks (CNNs) have small complexity and
are good candidates for low-power, high-throughput inference. Such networks
are heterogeneous in terms of computation-to-communication (CTC) ratios and
computation patterns between layers, especially for different layer types.
Yet, existing AI processors either use homogeneous processing elements (PEs),
resulting in low runtime PE efficiency, or run different layers on
heterogeneous PEs in sequential, introducing resource redundancy. This paper
proposes a heterogeneous dual-core architecture (dual-OPU), where one core is
optimized for regular convolution layers and the other for depthwise
convolution layers. PEs are homogeneous with each core. To make full use of
dual-core parallelism, we develop a scheduling algorithm to concurrently
execute layers for different input images on dual-core and balance parallel
workload. Meanwhile, we automatically tune the PE number for a core and tune
the input size for each PE to maximize throughput. Compared with a single-core
processor with the same area for a single network, heterogeneous dual-OPU on
average improves runtime PE efficiency and throughput by 11$\%$ and 31$\%$,
respectively. For a workload of multiple networks, dual-OPU improves average
throughput by 11$\%$ compared with the state-of-the-art processors scaled to
the same area. To the best of our knowledge, it is the first in-depth study on
the heterogeneous dual-core processor for light-weight CNNs.

##  I Introduction

CNNs have achieved extensive success on miscellaneous artificial intelligence
applications such as image classification and object detection. A plethora of
models emerge with different operators and architectures, gradually shifting
attention from accuracy to efficiency in terms of speed and power. Classified
as light-weight CNNs, MobileNets [1][2] adopt depthwise separable convolution
to reduce computation complexity and parameter amount, while other models,
such as SqueezeNet [3], alter model topology to spare computation power.

However, such model-level changes reduce runtime hardware efficiency. To be
more specific, layers in modern light-weight CNNs are heterogeneous concerning
the computation pattern, especially between depthwise convolution layers and
regular convolution layers. Even for the same layer type, CTC (Computation to
Communication) ratio diffs significantly for various layer characteristic
parameters such as input feature map size and kernel size. Consequently,
processors with uniform PEs (Processing Elements) can barely accommodate all
types of layers efficiently, leading to a low runtime PE efficiency. Separate
engines for different layers, however, result in hardware resource redundancy
due to the sequential execution of engines. In other words, the balance
between generality and specificity has not been addressed well. This is
particularly true for workload with multiple CNNs.

There exist two paradigms on how to deal with the heterogeneous CNN workload.
One resorts to a uniform architecture, while the other applies custom
architectures (often called accelerators) to different models. For the first
paradigm, GPGPU focuses on the acceleration of matrix multiplication with
hardware multi-threading on parallel arithmetic cores and performs poorly on
memory-bound depthwise convolution. Similarly, TPU [4] is optimized to perform
fast, bulky matrix multiplication with homogeneous systolic arrays.
Nevertheless, it suffers from poor performance on workload with sparse
computation intensity, such as element-wise algebra. Light-OPU [5] utilizes
uniform overlay architecture on FPGA for light-weight layers. Yet, special-
purpose modules, e.g., line buffer and squeeze-and-excitation block, might be
redundant for the majority of layers, indicating resource inefficiency and
potential room for further speedup. Xilinx DPU [6][7] also includes a non-
negligible additional resource cost for a separate depthwise convolution
engine and channel augmentation module for small output channels.

On the other hand, accelerators like [8] adopt layer-wise architecture
tailored for bottom layers and uniform architecture for top layers based on
the observation that CTC ratios fluctuate significantly between the two groups
but just slightly within top layers. [9][10][11] map one or some specific
layers to heterogeneous convolution accelerators to handle various layer
characteristics. However, light-weight models are missing in these
accelerators, so the performance is unknown when applied to latest light-
weight models that are far more heterogeneous than traditional VGG-like
models. [12] utilizes separate PEs for depthwise and regular convolution
layers to improve individual throughput, which, in contrast, increases the
overall redundancy in hardware resources.

To sum up, existing work with homogeneous PEs suffers from low resource
efficiency on light-weight models, while those with heterogeneous PEs have not
shown support for light-weight models or have exhibited resource redundancy.
To address these problems, we aim to find a balance between generality and
specificity by proposing a dual-core processor (called dual-OPU), where one
core is optimized for regular convolution and the other one for depthwise
convolution with extra resource cost on light-weight hardware modules, e.g.,
line buffer. Still, each core can handle all types of layers but with
different efficiency. To reduce the overall resource redundancy, we run
multiple heterogeneous layers in parallel such that layers that prefer light-
weight modules can be accommodated by a single core. We interleave layers from
different models to increase the chance of parallelizing heterogeneous layers.
Then we tune the PE numbers and PE input sizes of each core to balance the
parallel workload, where the balance is further finetuned in tile granularity
by layer split. As a result, runtime efficiency is maximized. Our
contributions are listed as follows:

  * •

We propose a heterogeneous dual-core architecture with fine-grained PE
configuration space for high run-time PE efficiency.

  * •

We develop a scheduling algorithm to interleave layers from different models
to exploit dual-core parallelism for high throughput.

  * •

Given a set of target CNN models, we can find a PE configuration, with which
high average throughput can be achieved.

##  II Motivation

Light-weight CNN models typically include low-computation-intensity operators,
e.g., depthwise convolution and pointwise convolution, to reduce operation
count and parameter amount. However, this model-level optimization leads to
significant irregularity on the computation complexity between layers, where
memory-intensive light-weight layers are interleaved with compute-intensive
regular convolution layers. Therefore, deployed on processor with uniform PEs,
some layers have low runtime PE efficiency, as defined in Eq.1, where $N_{op}$
is the multiply-and-accumulate (MAC) operation amount performed in the
measurement with frequency $f$. $N_{PE}$ is the number of allocated PEs, and
$\alpha$ is the number of MAC operations performed by each PE per clock cycle.
$T$ is the measured latency in seconds.

$\text{runtime\ PE\ efficiency}=\frac{N_{op}}{\alpha\cdot N_{PE}\cdot T\cdot
f}$ (1)

Runtime PE efficiency measures the ratio between computation time and total
latency of a layer. Memory-intensive layers can barely fully overlap
communication time with computation time, and thus have lower runtime PE
efficiency than compute-intensive layers. Furthermore, the gap between runtime
PE efficiencies of different layers is exaggerated by variation of layer
characteristic parameters such as input/output channels, input feature map
height/width and kernel width/height.

Figure 1: Layer-wise runtime PE efficiency on uniform architecture proposed in
[5] with a single input image

Only when these layer-specific parameters match MAC unit sizes can we make
full use of arithmetic resources.

To show the layer-wise runtime PE efficiency differences, we mimic the
architecture of Light-OPU proposed in [5] to measure the latency of each
layer. Industrial processors like TPU are not good candidates for layer-wise
measurement due to their high runtime variance for an individual layer. As
shown in Fig.1, the average runtime efficiency is 59$\%$, 41$\%$ and 62$\%$
for MobileNet v1, MobileNet v2 and SqueezeNet v1, respectively. We find zigzag
curves for all three models, where high efficiencies are contributed by
regular convolutions and low ones come from depthwise convolutions for
MobileNet v1/v2 and pointwise convolutions for SqueezeNet v1. Both source
layers of low PE efficiency have limited computing parallelism than regular
convolution. Devoid of output channel parallelism, depthwise convolution
layers achieve 42$\%$ and 37$\%$ PE efficiency in MobileNet v1 and v2,
respectively. Pointwise convolution layers with small output channels lead to
41$\%$ PE efficiency in SqueezeNet v1. The significant performance gap between
different layer types calls for heterogeneous PEs, which customize arithmetic
structures for specific layer types to improve runtime PE efficiency.

Even with heterogeneous PEs, we cannot ignore that these light-weight models
are almost purely sequential, which can hardly make full use of the parallel
heterogeneous PEs on hardware. With the emphasis on the throughput of multiple
input images, prior work concurrently run layers for different input images
for better resource efficiency. [7] runs MobileNet v2 in a layer pipeline
schedule so that one pointwise convolution layer and one depthwise convolution
layer for two input images run on different engines in parallel. The fact that
two layers run in parallel, however, still results in low PE runtime
efficiency due to the imbalance of the latency even though its depthwise
engine already utilizes small number of PEs to compensate. Since its PE number
for depthwise engine is not tunable, this imbalance worsens when the CTC
ratios differ more significantly between two types of layers in MobileNet v1.
So we need to optimize PE allocation as well as layer scheduling.

##  III Heterogeneous Dual-core Architecture

###  III-A Proposed Architecture

To overcome the aforementioned problems, i.e., low runtime PE efficiency due
to the heterogeneous workload and imbalanced schedule, we propose a novel
dual-core architecture with fine-grained PE array configuration and an
automatic design flow to find the best configuration for high throughput. We
define a core as computing unit with independent input/output buffers, a PE
array and a post-processing unit. We introduce two types of cores, channel-
parallel core (c-core) and pixel-parallel core (p-core). c-core exploits
input/output channel parallelism for regular convolution, which usually has
large channel numbers. p-core takes advantage of the pixel parallelism in the
kernel window for depthwise convolution, where line buffer is required as
extra hardware support for data fetch due to the reuse of input feature map
pixels from the sliding window. PE configurations of the two cores are
optimized with respect to layer characteristics for high runtime PE
efficiency.

Figure 2: Heterogeneous dual-core architecture. IB, OB and PP are input
buffer, output buffer and post-processing unit.

As shown in Fig.2, our heterogeneous architecture consists of one c-core and
one p-core. Inside a core, PEs are homogenerous, and PE configuration and
buffer sizes are customized. Each core has ping-pong structured buffers for
input feature maps, weights and biases. Once a group of memory blocks for
input feature map, weight and bias are loaded from off-chip memory to input
buffers, the MAC operation pipeline is initiated. Partial sums are stored in
the output buffer for further accumulation. Post-processing operations, such
as pooling and activation, are also included in the computation pipeline and
are initiated once a group of output feature maps is ready.

###  III-B PE Array Configuration

We optimize PE array in terms of ($N_{PE}$, $N_{vector}$), where $N_{PE}$ is
the number of PEs and $N_{vector}$ is the number of multipliers for each PE.
Each PE implements an inner product with $2\times N_{vector}$ inputs including
$N_{vector}$ multiplication products, which are then reduced to one sum with a
balanced adder tree. More adders follow the PE outputs to provide 2 to
$N_{PE}$ accumulated results. The number of accumulated results is dynamically
configured by instructions to obtain high runtime PE efficiency.

Regarding data fetch for c-core, input feature maps are duplicated and
broadcast to PEs to exploit channel parallelism. p-core has an alternative way
of data fetch, by using line buffer. $T_{k_{h}}$ and $T_{k_{w}}$ are tiling
sizes of kernel height and kernel width to compute for one single memory load
from external memory to on-chip buffer. When $T_{k_{h}}$$>$$1$ or
$T_{k_{w}}$$>$$1$, line buffer expands input feature maps by $T_{k_{h}}\times
T_{k_{w}}$ before broadcasting.

Our PE may use multiple DSP macros on FPGA. We decompose each DSP into two
8-bit multipliers to make full use of computation resources. However, the two
decomposed multipliers must share of one input due to hardware constraints.
Two multipliers in c-core share one input feature map pixel with two output
channel weights, while two pixels share one input channel weight in p-core.
Double input feature map buffers are integrated with the p-core PE array. As a
result, two groups of sliding window pixels on the dimension of input feature
map height are computed in parallel to make full use of DSP resources.

While c-core is more computationally powerful for channel parallelism, p-core
is flexible for both channel parallelism and pixel parallelism at the cost of
some computation power and extra hardware components. In addition to
multipliers, p-core requires more resources for auxiliary components, such as
line buffer and extra input feature map buffers, than c-core. Processors using
a single p-core not only leads to low runtime PE efficiency but also results
in inefficient use of auxiliary components. For example, when deploying
regular $3\times 3$ convolution with 64 input channels and 16 output channels
on p-core P(16,64), we prefer not to use line buffer, since PE array
configuration perfectly matches layer channel numbers. Line buffer is not
useful in such case. If line buffer was used, it should generate multiple of
$T_{k_{h}}\times T_{k_{w}}$ pixels as inputs of inner product, which ranges
from 1 to 9 and leaves most of 64 PE inputs idle. Therefore, we need to
carefully allocate resource for p-core and c-core to make full use of all
resources.

###  III-C Design Flow

To determine the resource allocation for c-core and p-core, we propose an
automatic design flow to achieve high throughput of target workload. Our flow
in Fig.3 takes CNN model description and FPGA resource budget as inputs.

Figure 3: Automation Flow

CNN model description is parsed for layer-wise characteristics, such as input
feature map height/width, input/output channels, kernel height/width, as well
as data dependencies between layers. We partition the layers into two groups,
where layers in one group are assigned to c-core and layers in the other one
are assigned to p-core. For each group, we search for the PE configuration
that leads to the highest runtime PE efficiency, under the constraint that the
total allocated resources cannot exceed resource budget. For each
configuration, we decide tiling sizes on all dimensions with the objective to
maximize runtime PE efficiency. Based on the chosen tiling sizes, latency
model and resource model are developed for c-core and p-core to estimate
latency and resource. Moreover, we balance the workload executed in parallel
on c-core and p-core to improve the throughput.

One can see from runtime PE efficiency shown in Fig.1, high efficiency and low
efficiency are interleaved in all three cases. Adjacent layers with large
difference are good candidates for potential balanced parallel execution. To
leverage this, we choose to interleave layers for two input images of the same
CNN model and use topological order. Then PE numbers on c-core and p-core are
tuned based on the interleaved schedule to minimize the latency gap between
parallel workload. We further reduce the latency gap by a heuristic to split
convolution layers along input feature map height dimension for tile
reassignment. Finally, we generate hardware code and instructions based on the
PE allocation and scheduling.

##  IV Modeling

###  IV-A Tile Sizing

We aim to find tiling sizes of each layer for minimal total latency. We use
$(n,v)$ as the abbreviation of ($N_{PE}$, $N_{vector}$) in the following
sections. Given $(n,v)$, we aim to find the buffer sizes and if p-core is
required for each layer so as to estimate the latency and the total resource.
Since tiling sizes compose the tensor size to load and compute at each time
and are correlated with the memory and computation resource, we first
determine the tiling sizes ($T_{c_{i}}$, $T_{c_{o}}$, $T_{k_{h}}$,
$T_{k_{w}}$, $T_{h}$, $T_{w}$) for each layer. Since each PE can implement
inner product with multiple $v$ multiplications, product of $T_{c_{i}}$,
$T_{c_{o}}$, $T_{k_{h}}$ and $T_{k_{w}}$ matches total multiplier count, as
shown in Eq.2, where c-core has $T_{k_{h}}$=1 and $T_{k_{w}}$=1 without line
buffer. The multiple factor of $v$, denoted as $i$, aims to maximize runtime
PE efficiency in Eq.3. In other words, $i$ is the used number of PE among $n$
total PEs. Larger $i$ indicates higher portion of PEs are utilized and thus
higher runtime PE efficiency.

$T_{k_{h}}\cdot T_{k_{w}}\cdot T_{c_{i}}\cdot T_{c_{o}}=n\cdot
v,T_{k_{h}}\cdot T_{k_{w}}\cdot T_{c_{i}}=i\cdot v,i\in\mathbb{N}^{+}$ (2)
$i=\underset{i}{\arg\min}\lceil\frac{C_{o}}{T_{c_{o}}}\rceil\cdot\lceil\frac{C_{i}\cdot
K_{h}\cdot K_{w}}{T_{c_{i}}\cdot T_{k_{h}}\cdot T_{k_{w}}}\rceil$ (3)

We iterate $i$ from $1$ to $\lceil\frac{K_{h}\cdot K_{w}\cdot
T_{c_{i}}}{v}\rceil$. For each $i$, we iterate $T_{k_{h}}$ and $T_{k_{w}}$ to
get $T_{c_{i}}$ = $i\cdot\lceil\frac{v}{T_{k_{h}}\cdot T_{k_{w}}}\rceil$ and
$T_{c_{o}}=\frac{n\cdot v}{T_{k_{h}}\cdot T_{k_{w}}\cdot T_{c_{i}}}$.
($T_{h}$, $T_{w}$) relate to input buffer depth. To simplify, we assume
$T_{h}=T_{w}$, since most convolutions have square input size. Eq.4 decides
($T_{h}$, $T_{w}$) by memory efficiency in terms of buffer depth.

$(T_{h},T_{w})=\text{argmin}\frac{H\cdot
W}{\lceil\frac{H}{T_{h}}\rceil\cdot\lceil\frac{W}{T_{w}}\rceil\cdot T_{h}\cdot
T_{w}}$ (4)

It aims to minimize total input block numbers in size of ($T_{h}\times
T_{w}$). Some available options of tiling sizes might hold the same runtime PE
efficiency, in which case we pick the one with less resource cost using the
resource model to be discussed in Section IV-C.

###  IV-B Latency Modeling

We calculate latency for each layer and then add them up. For each layer,
$T_{load}$ and $T_{compute}$ are latencies of memory load and computation,
respectively. The three terms on the numerator of Eq.5 are the data amount to
load for input feature map in shape of ($H\times W\times C_{i}$), weight in
shape of ($K_{h}\times K_{w}\times C_{i}\times C_{o}$) and bias in shape of
($C_{o}$), which are loaded sequentially through the limited bandwidth of
external DRAM memory. Memory load works in pipeline with $L_{dram}$ as the
last part of latency. $L_{dram}$ is the column address strobe (CAS) latency of
DRAM access, and it stands for the delay between memory read request and the
moment data is available for on-chip buffers. Eq.6 depicts the computational
latency as the product of tile numbers on each dimension. Computation
following by post-processing runs in a deep pipeline. Assuming that the data
bitwidth is $BW_{data}$, whenever
$min(C_{o},\lfloor\frac{BW_{dram}}{BW_{data}}\rfloor)$ number of output
feature map data are ready, they will be passed to post-processing unit and
then stored to DRAM, resulting latency $L_{post}$. Both $L_{dram}$ and
$L_{post}$ are not constant in practice. For estimation, we use average values
based on multiple execution traces on FPGA. We use ceiling operators for the
accuracy of modeling. When the compiler generates and schedules ISA
instructions, it aims to overlap $T_{load}$ and $T_{compute}$ as much as
possible. Therefore, we use the maximum between $T_{load}$ and $T_{compute}$
in Eq.7 to estimate latency of each layer.

$T_{load}=\lceil\frac{H\cdot W\cdot C_{i}+K_{h}\cdot K_{w}\cdot C_{i}\cdot
C_{o}+C_{o}}{BW_{dram}}\rceil+L_{dram}$ (5)
$T_{compute}=\lceil\frac{C_{o}}{T_{c_{o}}}\rceil\cdot\lceil\frac{H}{T_{h}}\rceil\cdot\lceil\frac{W}{T_{w}}\rceil\cdot\lceil\frac{C_{i}}{T_{c_{i}}}\rceil\cdot\lceil\frac{K_{h}}{T_{k_{h}}}\rceil\cdot\lceil\frac{K_{w}}{T_{k_{w}}}\rceil+L_{post}$
(6) $T_{total}=\sum\limits_{l}^{l\in layers}max(T_{compute}^{l},T_{load}^{l})$
(7)

###  IV-C Area Modeling

We will discuss the resource model that varies with PE configuration ($n$,
$v$). Total resource is the sum of the variants from PE and buffers and the
invariant from memory controller, instruction decoder, post-processing unit,
etc. The variant cost on computation and memory resources are discussed as
follows.

##### DSP

We only use DSP to build multipliers for efficiency. $\alpha$ is the MAC
number that one DSP can handle. In our design, each DSP48E1 slice is
decomposed to two 8-bit$\times$8-bit multipliers that can work simultaneously
with one common input. So $\alpha$ is 2. The required number of DSP is
indicated as follows:

$N_{DSP}=\lceil\frac{n}{\alpha}\rceil\cdot v$ (8)

##### Memory Resource

We assume PE array has two types of on-chip input buffers that are built by
BRAM. $B_{fm}$ is for feature maps and $B_{weight}$ is for weights. Bias
amount is usually small so it will be implemented by logic resource. The
buffer depth of $B_{fm}$ is $T_{h}\cdot T_{w}$. Width of $B_{fm}$ and
$B_{weight}$ are $T_{c_{i}}$ and $T_{c_{i}}\cdot T_{c_{o}}$ to match the input
bandwidth of PE array. For p-core, if depthwise convolution is applied,
$B_{fm}$ should have 2 banks as that in the C-core model. The depth of
$B_{weight}$ is
$\lceil\frac{C_{o}}{T_{c_{o}}}\rceil\cdot\lceil\frac{C_{i}}{T_{c_{i}}}\rceil$.
Xilinx FPGAs provide RAMB18K macro for 18kb block RAM. Within total 18kb
memory capacity, RAMB18K has configurable $width$ $\times$ $depth$
combinations, such as 36$\times$512, 18$\times$1k, 9$\times$2k, 4$\times$4k,
2$\times$8k and 1$\times$16k. We count the number of required RAMB18K given
buffer width and depth with priority for width, which means that we tend to
use minimum number of RAMB18K in term of width size.

##### Logic Resource

LUT and FF cost comes from three aspects. (1) the adders following multipliers
in the PE array: For each PE with $v$ inputs, they are accumulated to one with
these adders. Input data width of adder increases with the depth of the adder
tree. (2) the delayers in a PE: Delayers are required when $v$ is not power of
two. Delayers are implemented by simple register insertion. (3) line buffer:
To match memory bandwidth with input buffer, $T_{c_{i}}$ is the number of line
buffer channels required. For each layer with $T_{k_{h}}\times T_{k_{w}}>1$,
the length of line buffer should be $T_{w}\times(T_{k_{h}}-1)+T_{k_{w}}$ such
that pixels in sliding window $T_{k_{h}}\times T_{k_{w}}$ are preloaded before
computation. We select the parameter sizes to accommodate all the layers that
require line buffers. We collect resource costs from different sizes of
adders, delayers and line buffers implemented by Xilinx toolchain and build
model for each component.

##### Validation of Resource Model

| LUT | FF | DSP | BRAM
---|---|---|---|---
[5] | 137816(67.29$\%$) | 251433(57.41$\%$) | 577(68.69$\%$) | 237.5(53.26$\%$)
our resource model | 137149(67.62$\%$) | 234046(61.67$\%$) | 577(68.69$\%$) | 237(53.37$\%$)

TABLE I: Validation of resource modeling on core modules (external-memory-
related modules excluded).

As shown in Table I, [5] is a p-core with input/output buffers as well as
other resource-invariant modules. To show the effectiveness of our resource
model on PEs and buffers, we compare the resource estimation with the real
implementation results. Our model is able to obtain $<$ $3\%$ resource
estimation error.

##  V Optimization

###  V-A Scheduling

Given CNN graph $G(V,E)$ and separated ($n$,$v$) of c-core and p-core, we aim
to find a schedule that maximizes the throughput of heterogeneous dual-OPU. As
shown in Fig.4(a), nodes are layers, and edges indicate the data dependencies
between layers. We first partition the graph to layer groups, and each group
includes one or multiple layers. Groups are assigned to c-core or p-core
following topological order. Since the topology limits the chance for groups
assigned to different cores to be executed in parallel (i.e., c-core-assigned
$g_{3}$ and p-core-assigned $g_{4}$), we interleave layers for two input
images of the same CNN graph such that more groups will be able to run
simultaneously. In Fig.4(b), the second region from top indicates that $g_{2}$
for the first input image and $g_{1}$ for the second image are scheduled on
p-core and c-core in parallel. Due to the fact that layer characteristics and
topology vary with different CNN graphs, two parallel groups can still show
large latency difference even with the best allocation and partitioning
scheme. As a result, one core is idle during the gap, which lowers the
performance. We split some layers to sub-layers along the dimension of input
feature map height to reduce the latency gap. For example, layer 4 in Fig.4(b)
is split to layer $4^{a}$ and $4^{b}$ in Fig.4(c). Layer $4^{a}$ forms $g_{3}$
with layer 3, while layer $4^{b}$ forms $g_{4}$ with layer 5. Consequently,
latency gap between $g_{2}$ and $g_{3}$ is reduced. Although latency gap
between $g_{3}$ and $g_{4}$ increases in Fig.4(c), split is applied as long as
the total throughput is improved. Among multiple partitioning choices with
load balancing strategy, we pick the one with highest throughput estimation as
the final schedule.

Figure 4: Layer scheduling of MobileNet v2 snippet on heterogeneous dual-OPU
(a) MobileNet v2 snippet where layer groups are assigned to c-core and p-core
separately. (b) Execution trace where we interleave layer groups for two input
images. (c) Execution trace where we further split layer 1 and layer 4
according to input feature map height to balance workload between two cores
and improve runtime PE efficiency.

####  V-A1 Allocation-aware Partitioning

We perform layer allocation followed by partitioning to find a scheme suitable
for the scheduling method in Fig.4(b). We adopt three simple allocation
schemes, including greedy allocation, layer-type based allocation and robin-
round allocation. Greedy allocation decides the core assignment based on the
latency estimation of a layer on two cores. Each layer is allocated to the
core with less latency. Regardless of hardware configuration, layer-type based
allocation assigns regular convolution layers to c-core and depthwise
convolution layers to p-core such that we can exploit channel parallelism and
pixel parallelism, respectively. As a work-around in case the two allocation
scheme above cannot work, robin-round allocation assigns layers to two cores
one by one in circular order. We partition the graph to layer groups according
to allocation results. Interleaving layers for two input images, we aim to
find such an allocation that the variance of $g_{i}/g_{i+1}$ for all odd $i$
and $g_{i+1}/g_{i}$ for all even $i$ is minimized. Each group includes one or
more layers instead of single layer for smaller variance. Once we have a small
variance, workload can be balanced well with appropriate PE allocation on
c-core and p-core. The partitioning result is shown in Fig.4(a) as an example,
where layer 3 and layer 4 form $g_{3}$.

####  V-A2 Load Balancing

Partitioning in layer granularity cannot completely eliminate latency gap
between all two parallel layer groups. For example, a regular convolution
layer is scheduled to c-core and run in parallel with a depthwise convolution
layer with same parameter sizes. The former one has far more MAC operations
than the latter, leading to large latency gap when c-core and p-core have same
MAC units. To balance the workload, we propose a heuristic method to split
layers. Since input feature map pixels run in pipeline on either core, we pick
the height dimension to split for simplicity, since it does not complicate the
partial sum accumulation along input channels. As shown in Alg.1, we first
compute latency gap for each parallel group pair and pick the pair with
largest gap. Once layer $l_{split}$ to split is located, we aim to find the
height $h$ to remain in the layer to minimize the average latency of
interleaved two batch runs $T_{b2}$, as defined in Eq.9. $T_{b2}$ is the sum
of the maximal latency between any parallel groups. The input feature map size
is updated from $H\times W$ to $h\times W$. The rest of the layer is
reallocated to the other core. The height $h^{\prime}$ accommodates the new
$T_{k_{h}}$ in the reallocated part. We continue splitting layers until there
is no further improvement of $T_{b2}$.

$T_{b2}=\sum\limits_{i\in[1,N-1]}|T_{g_{i}}-T_{g_{i+1}}|+T_{g_{1}}+T_{g_{N}}$
(9)

Input: Layer groups $\\{g_{i}\\}$

while  _$T_{b2}$ gets improved_ do

compute $|T_{g_{i}}-T_{g_{j}}|$ for each neighboring ($g_{i}$,$g_{j}$)

pick ($g_{p}$,$g_{q}$) with largest latency gap

/* assume $T_{g_{p}}$ $>$ $T_{g_{q}}$ and $g_{p}$ proceeds $g_{q}$ */

find layer to split $l_{split}$ $\leftarrow$ $g_{p}$.back()

find height to remain $h$ $\leftarrow$
$\underset{h\in[1,l_{split}.H-1]}{\arg\min}T_{b2}$

$l_{split}.H\leftarrow h$, $h^{\prime}\leftarrow
l_{split}.H-h+l_{split}.T_{k_{h}}-1$

$g_{q}$.push$\\_$front($l_{split}$.copy($H$=$h^{\prime}$))

update latency of each layer group and $T_{b2}$

end while

Algorithm 1 Load-balance-heuristic Scheduling

###  V-B Co-optimization of PE Allocation and Scheduling

Aforementioned scheduling methods aim to obtain good throughput given
arbitrary PE allocation of c-core and p-core. Clearly, the PE allocation that
better matches the heterogeneity of target workload leads to a higher
throughput. On the other hand, given PE allocation, we need to find a specific
scheduling methods that can make full use the workload-matching PE allocation.
PE allocation and scheduling depend on each other. We will first define the
design space for PE allocation and then discuss how to find the best PE
allocation along with scheduling for target workload.

####  V-B1 Design Space for PE Allocation

Our proposed heterogeneous dual-core design is driven by PE array
configurations of c-core and p-core. We pre-design memory buffers to meet
bandwidth requirements of PE arrays, and define parameter vector
($sch$,$n_{c}$,$v_{c}$,$n_{p}$,$v_{p}$) in Table II as the design space for PE
allocation, where $sch$ specifies the scheduling with respect to layer
allocation according to input model and hardware. ($n_{c}$,$v_{c}$) and
($n_{p}$,$v_{p}$) are PE configuration ($n$,$v$) for c-core and p-core,
respectively. For the constraint from target FPGA device,
($\overline{N}_{DSP}$,$\overline{N}_{BRAM}$,$\overline{N}_{LUT}$,$\overline{N}_{FF}$)
stand for upper-bounds of DSP, BRAM, LUT, FF resources, which define the valid
design space. Another constraint is the bandwidth between core and external
memory, $BW_{dram}$.

Parameters | Constraints
---|---
Scheduling | PE Allocation | ($\overline{N}_{DSP}$,$\overline{N}_{BRAM}$,$\overline{N}_{LUT}$,$\overline{N}_{FF}$)
$sch$ | ($n_{c}$,$v_{c}$,$n_{p}$,$v_{p}$) | $BW_{dram}$

TABLE II: Design space for PE allocation. Tuning knobs are PE array
configuration of c-core/p-core and scheduling.

####  V-B2 Search Algorithm

We use branch-and-bound to find the best PE allocation that minimizes the two-
batch latency $T_{b2}$ within the resource constraints. A naive approach is to
determine if a DSP is included in our design along branches for all DSPs by
enumeration, which, however, leads to huge search space in size of
$2^{\overline{N_{DSP}}}$. This search space is redundant in our dual-core
design, since we only care about the DSP numbers allocated to c-core and
p-core. So, instead, we choose to branch upon the c-core DSP ratio $\theta$,
as defined in Eq.10, where $\alpha$ is the MAC number one DSP Macro can
perform per clock cycle.

$\theta=\frac{n_{c}\cdot v_{c}}{\alpha\cdot\overline{N}_{DSP}}$ (10)

To compute the lower bound given $\theta$, we greedily allocate the remaining
DSPs to p-core until we run out of logic resources or DSP resources. We
estimate the lower bound of $T_{b2}$ by Eq.9, where $T_{compute}$ is estimated
with its lower bound of $T^{lb}_{compute}$ defined in Eq.11.
$N^{core,l}_{DSP}$ is the DSP number of the core allocated for the layer $l$,
which is $\theta\cdot\overline{N_{DSP}}$ for c-core and
$(1-\theta)\cdot\overline{N_{DSP}}$ for p-core.

$T^{lb}_{compute}=\frac{C_{o}\cdot H\cdot W\cdot C_{i}\cdot K_{h}\cdot
K_{w}\cdot 2}{\alpha\cdot N^{core,l}_{DSP}}+L_{post}$ (11)

This is a lower bound of $T_{compute}$ because it does not include the
potential unmatch between layer characteristic parameters and PE array
configuration sizes, which results in higher latency. We try different $sch$
based on the current $\theta$ and choose the lowest $T_{b2}$ as the lower
bound for $\theta$. Then we branch to the two middle points of the unvisited
subsets split by $\theta$. We start with $\theta=0.5$ and search for the
$\theta$ with minimal lower bound of two-batch latency. The early termination
happens whenever we reach the resource limit or we cannot have a better lower
bound.

For the best $\theta$ found in the branch-and-bound global search, we then
locally search for ($n_{c}$,$v_{c}$,$n_{p}$,$v_{p}$) for best throughput. To
reduce the search space, we select limited available options for $v$. Although
our PE array is able to provide runtime configurable $2$ to $n$ outputs, with
fixed $n\times v$, small $v$ leads to huge cost on registers for accessibility
of intermediate results. On the other hand, as the unit input length, large
$v$ can easily result in low runtime PE efficiency. Therefore, we choose
$\\{8,9,10,12,14,15,16,18\\}$ as candidates of $v$. Prime numbers are excluded
since common channel numbers are not multiple of prime numbers. We
exhaustively search all valid pairs of ($n$,$v$) for c-core and p-core based
on the best $\theta$. The ($n$,$v$) and $sch$ corresponding to the best
throughput is our design choice.

##  VI Experiment

###  VI-A Experiment Settings

##### Software

We leverage parser of TVM [13] framework that handles input models from
different CNN developing frameworks (i.e., PyTorch, Tensorflow), and then
transform Relay IR to our customized IR, which is used to generate ISA simar
to that in [14] for dual-OPU.

##### Workload

Our test cases include MobileNet v1, MobileNet v2 and SqueezeNet v1. We denote
SqueezeNet v1 as SqueezeNet for simplicity. These test cases cover typical
light-weight operators such as depthwise convolution in MobileNet v1/v2 and
expand/squeeze layers with small channels in SqueezeNet. We set batch size as
2 for evaluation on throughput and report on average values for each CNN
model.

##### Hardware

We use the notation C($n,v$) for c-core with a PE array with $n$ PEs, which
have $v$ multipliers for each PE. Same notation is applied to p-core, where
PEs are further coupled with line buffers. We run workload on three types of
different designs, including single-core design P(128,9), homogeneous dual-
core design P(64,9)+P(64,9) and heterogeneous design with one c-core and one
p-core C(128,8)+P(64,9) to show the effectiveness of heterogeneous dual-OPU.
For fair comparison, PE configurations in each experiments below have same
equivalent area. To be more specific, three example designs have roughly same
area, so are C(128,8) and P(64,9) in the heterogeneous one. P(64,9) has half
multipliers, buffer depth and line buffer channels of P(128,9). C(128,8) has
the same buffer depth as P(64,9). Without line buffer, C(128,8) saves LUT
resource for more multipliers than P(64,9). Since line buffer only costs LUT
while multipliers primarily cost DSP, it is difficult to compare the total
resource cost. To quantify the resource cost of PE array in c-core and p-core,
we use the equivalent LUT cost as the equivalent area cost. We count the
equivalent LUT cost of each multiplier as the LUT cost to achieve the same
functionality to one decomposed 8-bit multiplier implemented by DSP. p-core PE
array in P(64,9) and c-core PE array C(128,8) have close equivalent LUT cost
as shown in Table III. P(64,9) has 128-channel line buffer to make full use of
double input feature map buffers for extra pixel parallelism on the height
dimension of depthwise convolution. We use single p-core design P(128,9) as
the baseline for comparison.

| Equivalent Area/LUT Cost
---|---
| Line Buffer | Multipliers | Adders | Total
P(64,9) | 39868 | 40896 | 17859 | 98623
C(128,8) | 0 | 72704 | 31749 | 104453

TABLE III: Equivalent LUT cost of PE structures in P(64,9) and C(128,8).
Similar total equivalent cost indicates similar area.

##### Evaluation

We have built a cycle-accurate instruction level latency simulator with
configurable core type (c-core/p-core), PE size ($n$,$v$) and memory
bandwidth. We run the complete compilation flow to generate ISA instructions
for the simulation. For each instruction, we adopts the latency model
discussed in Section IV-B, which takes CAS latency of DRAM access into account
for accuracy. As shown in Table IV, the cycle-accurate instruction-level
simulator shows $<$1$\%$ error on cycle count compared with that on board-
level FPGA implementation. All the results in the experiment section are
measured with cycle-accurate simulation.

| Cycle Count
---|---
| Board-level Performance | Cycle-accurate Simulator
MobileNet v1 | 755857($\pm 0\%$) | 757149(-0.2$\%$)
MobileNet v2 | 637551($\pm 0\%$) | 642940(+0.8$\%$)
SqueezeNet | 447457($\pm 0\%$) | 443129(-0.9$\%$)

TABLE IV: Validation of cycle-accurate instruction-level latency simulator on
P(128,9).

###  VI-B Impact of Scheduling

Table V shows the effectiveness of our scheduling method on different
($N_{PE}$, $N_{vector}$) combinations. C(128,8)+P(64,9) and C(180,8)+P(32,9)
have different $N_{PE}$ ratio between two structures. C(112,9)+P(72,8) further
changes $N_{vector}$. We compare the performance among four scheduling
methods. The first three only apply layer-type based allocation, greedy
allocation and round-robin allocation for layer group partitioning. Then we
measure the average throughput of two interleaved batches. The last one, load-
balance-heuristic scheduling, further balances the parallel workload of layer
groups on two cores based on the three aforementioned schedules. We choose the
best one as our final schedule. Load-balance-heuristic scheduling improves
throughput by 10$\%$ on average from the three basic schemes. MobileNet v1 and
v2 prefer layer-type based schedule as the basic scheme, while SqueezeNet gets
more room for load balancing staring with round-robin scheme.

| PE Array Configuration | Throughput(fps)
---|---|---
| Layer-type | Greedy | Round-robin | Load-balance-heuristic
MobileNet v1 | C(128,8)+P(64,9) | 267.4 | 267.4 | 269.8 | 304.3
C(180,8)+P(32,9) | 318.9 | 259.3 | 266.6 | 320.2
C(112,9)+P(72,8) | 234.7 | 238.5 | 235.0 | 269.9
MobileNet v2 | C(128,8)+P(64,9) | 378.4 | 378.4 | 338.5 | 427.6
C(180,8)+P(32,9) | 392.0 | 304.9 | 214.4 | 384.9
C(112,9)+P(72,8) | 323.7 | 346.6 | 317.0 | 371.1
SqueezeNet | C(128,8),P(64,9) | 413.9 | 413.9 | 391.1 | 529.9
C(180,8)+P(32,9) | 483.9 | 483.9 | 228.4 | 520.4
C(112,9)+P(72,8) | 328.3 | 375.2 | 372.5 | 451.3

TABLE V: Throughput comparison of different scheduling methods on different PE
array configurations. Best throughputs are bolded.

###  VI-C Impact of PE Array Configuration

Within the resource budget, Table VI shows the best PE array configuration for
single workload.

| PE Array Configuration | DSP/PE Eff† | Throughput/fps
---|---|---|---
MobileNet v1 | P(128,9) | 577/59$\%$ | 264.6($\pm 0\%$)
C(128,12)+P(8,16) | 832/$70\%$ | 358.4($+35.4\%$)
MobileNet v2 | P(128,9) | 577/41$\%$ | 313.4($\pm 0\%$)
C(160,8)+P(48,8) | 832/$51\%$ | 438.4($+38.8\%$)
SqueezeNet | P(128,9) | 577/62$\%$ | 446.9($\pm 0\%$)
C(130,8)+P(64,10) | 840/$75\%$ | 534.7($+19.6\%$)

TABLE VI: Throughput comparison between PE array configurations optimized for
single CNN and single-core baseline with same area. PE Eff† is the runtime PE
efficiency.

We use single p-core design P(128,9) as the baseline. Generated configurations
have similar equivalent area in LUT cost on PE structure, including line
buffer, multipliers and adders. The results show that our design flow is able
to generate PE array configuration with 31$\%$ improvement on throughput and
11$\%$ improvement on runtime PE efficiency on average over baseline. On the
generated configurations, we find the best configuration for MobileNet v1
holds the largest $\theta$, indicating highest heterogeneity between parallel
layer groups. With largest workload difference between regular convolution and
light-weight convolution, MobileNet v1 thus needs the largest $\theta$ among
the three workloads to balance the load. The result indicates that more
heterogeneous workload can lead to more improvement by our heterogeneous dual-
OPU.

| PE Array Configuration
---|---
| C(128,12)+P(8,16) | C(160,8)+P(48,8) | C(130,8)+P(64,10) | C(128,10)+P(32,12)
| Optimized for individual CNN | Optimized for average throughput of multiple CNNs
MobileNet v1 |  358.4(+9.9$\%$)† | 249.3(-23.6$\%$) | 314.6(-3.6$\%$) | 326.2($\pm 0\%$)
MobileNet v2 | 329.3(-24.8$\%$) |  438.4(+0.2$\%$)† | 428.1(-2.2$\%$) | 437.8($\pm 0\%$)
SqueezeNet | 527.9(+0.2$\%$) | 436.9(-17.0$\%$) |  534.7(+1.5$\%$)† | 526.6($\pm 0\%$)
Average∗ | 388.5(-6.1$\%$) | 349.6(-15.5$\%$) | 406.2(-1.9$\%$) |  413.9($\pm 0\%$)†

TABLE VII: Throughput comparison of PE array configurations on single-CNN
workload and multiple-CNN workload. Average∗ stands for the average throughput
of multiple-CNN workload. $\dagger$ indicates the configuration found by our
search algorithm. Best throughput for each workload is bolded.

CNN Model | MobileNet v1 | MobileNet v2 | SqueezeNet
---|---|---|---
Design | [15] | [5] | Ours | Xilinx DPU [6] | [5] | Ours | Xilinx DPU [6] | [5] | Ours
Device | Stratix-V | XCK325T | XCK325T | ZCU102 | XCK325T | XCK325T | ZCU102 | XCK325T | XCK325T
PE Precision | Int16 | Int8 | Int8 | Int8 | Int8 | Int8 | Int8 | Int8 | Int8
Allocated DSP | 1278 | 704 | 832 | 2070 | 704 | 832 | 1942 | 704 | 832
Frequency(MHz) | 133 | 200 | 200 | 287 | 200 | 200 | 333 | 200 | 200
Throughput(fps) | 237.1 | 264.6 | 326.2 | 587.2 | 325.7 | 437.8 | 1048 | 420.9 | 526.6
Throughput/DSP∗ | 0.11 | 0.21 | 0.23 | 0.08 | 0.14 | 0.16 | 0.20 | 0.19 | 0.22
(GOPs/DSPslice)

TABLE VIII: Comparison between our work and existing works. Throughput/DSP∗ is
normalized to 8-bit operations.

Targeting on workload consisting of multiple CNN models, our flow is able to
find the PE array configuration with higher average throughput than designs
that are optimized for a single CNN model specifically. We use harmonic mean
of throughput for different models as the average throughput. Our design flow
finds C(128,10)+P(32,12) as the configuration with highest average throughput.
As shown in Table VII, C(128,10)+P(32,12) shows 7.8$\%$ improvement on average
throughput of multiple CNNs with 3.8$\%$ performance loss on individual-best
configurations for single CNN. Results show the effectiveness of our design
space exploration approach on searching for which configuration can lead to
best average throughput.

###  VI-D Comparison with state-of-the-art processors

We compare our work with Xilinx DPUv3 [6] and other processors from industry
and academia, respectively, since they can handle regular convolution and
depthwise convolution. Xilinx DPUv3 is implemented on Xilinx ZCU102 board with
three B4096$\\_$EU cores. In Table VIII, we include extra 48 DSPs per core in
allocated DSP count for DPU on MobileNet v2 due to the depthwise convolution,
besides the basic cost for regular convolution. DPU performance on MobileNet
v1 is not included since it has never been reported by Xilinx. Scaled to same
area, our heterogeneous dual-core processor improves throughput/DSP by up to
85$\%$ and 15$\%$ compared with the latest works from industry (Xilinx DPU)
and academia.

##  VII Conclusions and Discussions

In this paper, we propose a heterogeneous dual-OPU to achieve high throughput
of light-weight CNNs with high runtime PE efficiency. In dual-OPU, one core is
optimized for channel parallelism and regular convolution, and the other core
is optimized for pixel parallelism and depthwise convolution. Moreover, the PE
number of each core and the input size of each PE can be tuned automatically
with our design flow given target CNNs and FPGA device. Meanwhile, we
concurrently run layers for different input images of the same CNN and
schedule with layer split to optimize the overall runtime PE efficiency. The
experiment shows that heterogeneous dual-OPU can improve throughput and
runtime PE efficiency of homogeneous baseline with the same area by 31$\%$ and
11$\%$ for single CNN. For a workload of multiple CNNs, compared with state-
of-the-art processors, our design can improve the throughput by 11$\%$ on
average than the latest works from industry and academia scaled to the same
area.

## References

  * [1] Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  arXiv preprint arXiv:1704.04861, 2017.
  * [2] Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen.  Mobilenetv2: Inverted residuals and linear bottlenecks.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4510–4520, 2018.
  * [3] Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer.  Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size.  arXiv preprint arXiv:1602.07360, 2016.
  * [4] Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al.  In-datacenter performance analysis of a tensor processing unit.  In Proceedings of the 44th Annual International Symposium on Computer Architecture, pages 1–12, 2017.
  * [5] Yunxuan Yu, Tiandong Zhao, Kun Wang, and Lei He.  Light-OPU: An fpga-based overlay processor for lightweight convolutional neural networks.  In The 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pages 122–132, 2020.
  * [6] DPU for Convolutional Neural Network: https://www.xilinx.com/support/documentation/ip_documentation/dpu.  2020\.
  * [7] Di Wu, Yu Zhang, Xijie Jia, Lu Tian, Tianping Li, Lingzhi Sui, Dongliang Xie, and Yi Shan.  A high-performance cnn processor based on fpga for mobilenets.  In 2019 29th International Conference on Field Programmable Logic and Applications (FPL), pages 136–143. IEEE, 2019.
  * [8] Xiaofan Zhang, Hanchen Ye, Junsong Wang, Yonghua Lin, Jinjun Xiong, Wen-mei Hwu, and Deming Chen.  Dnnexplorer: a framework for modeling and exploring a novel paradigm of fpga-based dnn accelerator.  In Proceedings of the 39th International Conference on Computer-Aided Design, pages 1–9, 2020.
  * [9] Xuechao Wei, Yun Liang, Xiuhong Li, Cody Hao Yu, Peng Zhang, and Jason Cong.  Tgpa: tile-grained pipeline architecture for low latency cnn inference.  In Proceedings of the International Conference on Computer-Aided Design, pages 1–8, 2018.
  * [10] Yongming Shen, Michael Ferdman, and Peter Milder.  Maximizing cnn accelerator efficiency through resource partitioning.  In 2017 ACM/IEEE 44th Annual International Symposium on Computer Architecture (ISCA), pages 535–547. IEEE, 2017.
  * [11] Huimin Li, Xitian Fan, Li Jiao, Wei Cao, Xuegong Zhou, and Lingli Wang.  A high performance fpga-based accelerator for large-scale convolutional neural networks.  In 2016 26th International Conference on Field Programmable Logic and Applications (FPL), pages 1–9. IEEE, 2016.
  * [12] Jiang Su, Julian Faraone, Junyi Liu, Yiren Zhao, David B Thomas, Philip HW Leong, and Peter YK Cheung.  Redundancy-reduced mobilenet acceleration on reconfigurable logic for imagenet classification.  In International Symposium on Applied Reconfigurable Computing, pages 16–28. Springer, 2018.
  * [13] Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al.  TVM: An automated end-to-end optimizing compiler for deep learning.  In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 578–594, 2018.
  * [14] Yunxuan Yu, Chen Wu, Tiandong Zhao, Kun Wang, and Lei He.  Opu: An fpga-based overlay processor for convolutional neural networks.  IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 28(1):35–47, 2019.
  * [15] Lin Bai, Yiming Zhao, and Xinming Huang.  A cnn accelerator on fpga using depthwise separable convolution.  IEEE Transactions on Circuits and Systems II: Express Briefs, 65(10):1415–1419, 2018.