# QUANTISENC: A Fully-Configurable Software-Defined Digital Quantized Spiking
Neural Core Architecture

Shadi Matinizadeh, Noah Pacik-Nelson, Ioannis Polykretis, Krupa Tishbi, Suman
Kumar, M. L. Varshika, Arghavan Mohammadhassani, Abhishek Mishra, Nagarajan
Kandasamy, James Shackleford, Eric Gallo, Anup Das S. Matinizadeh, K. Tishbi,
S. Kumar, M. L. Varshika, A. Mohammadhassani, A. Mishra, N. Kandasamy, J.
Shackleford, and A. Das are with the Department of Electrical and Computer
Engineering, Drexel University, Philadelphia, PA, 19104 USA. Correspondance
e-mail<EMAIL_ADDRESS>Pacik-Nelson, I. Polykretis, and E. Gallo are
with Accenture Labs.

# An Open-Source Fully-Configurable Software-Defined
Digital Quantized Spiking Neural Core Architecture

Shadi Matinizadeh, Noah Pacik-Nelson, Ioannis Polykretis, Krupa Tishbi, Suman
Kumar, M. L. Varshika, Arghavan Mohammadhassani, Abhishek Mishra, Nagarajan
Kandasamy, James Shackleford, Eric Gallo, Anup Das S. Matinizadeh, K. Tishbi,
S. Kumar, M. L. Varshika, A. Mohammadhassani, A. Mishra, N. Kandasamy, J.
Shackleford, and A. Das are with the Department of Electrical and Computer
Engineering, Drexel University, Philadelphia, PA, 19104 USA. Correspondance
e-mail<EMAIL_ADDRESS>Pacik-Nelson, I. Polykretis, and E. Gallo are
with Accenture Labs.

# A Fully-Configurable Open-Source Software-Defined Digital Quantized Spiking
Neural Core Architecture

Shadi Matinizadeh, Noah Pacik-Nelson, Ioannis Polykretis, Krupa Tishbi, Suman
Kumar, M. L. Varshika, Arghavan Mohammadhassani, Abhishek Mishra, Nagarajan
Kandasamy, James Shackleford, Eric Gallo, Anup Das S. Matinizadeh, K. Tishbi,
S. Kumar, M. L. Varshika, A. Mohammadhassani, A. Mishra, N. Kandasamy, J.
Shackleford, and A. Das are with the Department of Electrical and Computer
Engineering, Drexel University, Philadelphia, PA, 19104 USA. Correspondance
e-mail<EMAIL_ADDRESS>Pacik-Nelson, I. Polykretis, and E. Gallo are
with Accenture Labs.

###### Abstract

We introduce QUANTISENC, a fully configurable open-source software-defined
digital quantized spiking neural core architecture to advance research in
neuromorphic computing. QUANTISENC is designed hierarchically using a bottom-
up methodology with multiple neurons in each layer and multiple layers in each
core. The number of layers and neurons per layer can be configured via
software in a top-down methodology to generate the hardware for a target
spiking neural network (SNN) model. QUANTISENC uses leaky integrate and fire
neurons (LIF) and current-based excitatory and inhibitory synapses (CUBA). The
nonlinear dynamics of a neuron can be configured at run-time via programming
its internal control registers. Each neuron performs signed fixed-point
arithmetic with user-defined quantization and decimal precision. QUANTISENC
supports all-to-all, one-to-one, and Gaussian connections between layers. Its
hardware-software interface is integrated with a PyTorch-based SNN simulator.
This integration allows to define and train an SNN model in PyTorch and
evaluate the hardware performance (e.g., area, power, latency, and throughput)
through FPGA prototyping and ASIC design. The hardware-software interface also
takes advantage of the layer-based architecture and distributed memory
organization of QUANTISENC to enable pipelining by overlapping computations on
streaming data. Overall, the proposed software-defined hardware design
methodology offers flexibility similar to that of high-level synthesis (HLS),
but provides better hardware performance with zero hardware development
effort. We evaluate QUANTISENC using three spiking datasets and show its
superior performance against state-of-the-art designs.

###### Index Terms:

Neuromorphic Computing, FPGA, PyTorch, Verilog, hardware-software interface,
co-design

##  I Introduction

Neuromorphic computing describes the hardware implementation of the biological
nervous system [1]. Over the years, architects and designers have created such
systems using analog and digital components, offering lower power consumption,
reduced latencies, and many other benefits seen in biological systems.
Although analog designs improve energy efficiency by taking advantage of
electronic and physical laws in implementing neurons and synapses, digital
designs are faster to implement on silicon due to the maturity of their design
flow, while benefiting from technology scaling [2].

Existing digital neuromorphic designs suffer from the following three key
limitations, which we address in this work. First, once implemented on
silicon, existing designs allow only programming of its synaptic weights. They
do not allow configuring the neuron dynamics at run-time. We show that run-
time reconfiguration can be used to explore the trade-off between performance
and power (Section VI-I). Second, existing designs lack a well-defined
hardware-software interface to stream input, visualize output, and load design
configurations and synaptic weights. We show that a hardware-software
interface can also be used to allow pipelining to improve performance (Section
VI-G). Finally, existing neuromorphic designs are not released to the
community.

Figure 1: (a) High-level overview of QUANTISENC’s design blocks. (b)
Computation of spike trains using synaptic weights.

We introduce QUANTISENC (QUANTIzed Spiking-Enabled Neural Core), a fully
configurable layer-based neuromorphic hardware design and its software-defined
hardware design methodology for spiking neural networks (SNNs) [3]. This
design methodology allows researchers and system designers to train an SNN
model in software (e.g., Python) and evaluate its hardware performance, such
as area, power, latency, and throughput through FPGA prototyping and ASIC
design. The following are our key contributions.

  1. 1.

Configurability: QUANTISENC is designed hierarchically using a bottom-up
methodology with neurons, layers, and cores. The number of layers and neurons
per layer can be configured by software in a top-down methodology to generate
hardware for a target SNN model. QUANTISENC supports signed computations of
fixed-point arithmetic with user-defined quantization and decimal precision.
QUANTISENC implements leaky integrate-and-fire neurons (LIF) [4], and current-
based excitatory and inhibitory synapses (CUBA), with all-to-all, one-to-one,
and Gaussian connections between layers. Due to our modular design
methodology, QUANTISENC can be easily extended to support other types of
neurons, e.g., Izhikevich and compartmental [5], and synapse, e.g.,
conductance-based synapse (COBA) [6].

  2. 2.

Hardware-Software Interface: A key novelty of our design methodology is that
it allows the LIF dynamics to be configured from an application software
(e.g., PyTorch code) at runtime by programming the internal control registers
on the hardware through its interface with the system software (e.g., OS).
This provides flexibility to fine-tune hardware performance after the design
has been fabricated on silicon. Our hardware-software interface is also used
to program synaptic weights, drive input, and visualize hardware output.

  3. 3.

Pipelining: QUANTISENC implements a distributed memory organization, where the
synaptic memory of neurons in a layer is instantiated within each layer,
allowing the layers of QUANTISENC to operate independently and parallel to
each other. Although QUANTISENC is a dataflow architecture with layer-by-layer
processing of input data, we leverage the parallelism in its internal layer
design by overlapping the processing of streaming data in a pipelined fashion
using the proposed hardware-software interface. This refinement of dataflow
computations significantly improves throughput.

  4. 4.

Open-Source Release: QUANTISENC is developed over a period of eight years and
is supported by Accenture LLP, three US National Science Foundation research
grants, one US Department of Energy grant, and one European Horizon 2020
research grant. QUANTISENC is developed to foster future research in
neuromorphic computing. To obtain a copy of the design, please email Shadi
Matinizadeh at<EMAIL_ADDRESS>
We evaluate QUANTISENC and our system design methodology using Spiking MNIST
[7], DVS Gesture [8], and Spiking Heidelberg Digit (SHD) [9] datasets. Our
results show that compared to the software implementation, the hardware
implementation of LIF dynamics has a root mean square error (RMSE) of only
0.25 mV using a quantization of 16 bits with a decimal precision of 7 bits.
The RMSE increases to 0.43 mV, when using a quantization of 8 bits with a
decimal precision of 3 bits. For datasets, QUANTISENC significantly
outperforms state-of-the-art designs in terms of resource utilization, power,
latency, and throughput. We show the scalability of QUANTISENC for different
datasets and for the settings of its design parameters.

##  II High-Level Overview of QUANTISENC

Figure 1a shows a high-level overview of the proposed single-core design
QUANTISENC. There are $K$ layers, each instantiating LIF neuron modules. The
number of layers and neurons per layer are design parameters that can be
configured to implement a target SNN model in hardware. LIF parameters are
stored in the control registers of the decoder module. Within each layer, the
connect module consists of connection parameters to specify the organization
of its synaptic weight memory (MEM).

Figure 2: Design of a leaky-integrate-and-fire (LIF) neuron. There are four
main design components – ActGen, VmemDyn, VmemSel, and SpkGen.

At a high level, the synaptic memory of QUANTISENC is distributed among its
layers such that all pre-synaptic weights are stored in their respective
layer. This distributed memory organization allows us to pipeline the design
to improve throughput. Without loss of generality, we illustrate a fully
connected layer in Figure 1b, where a neuron in layer $L$ (post-synaptic
neuron) receives input from all neurons in layer $(L-1)$ (pre-synaptic
neurons). The output spikes of this neuron are routed to all neurons in the
layer $(L+1)$.

At a finer granularity, synaptic memory (MEM) is a $M\times N$ weight matrix.
This weight matrix corresponds to a design configuration of $N$ neurons in the
layer with $M$ pre-synaptic connections per neuron. Spike trains from pre-
synaptic connections are weighed using synaptic weights to generate output
spike trains as illustrated in the figure.

The access granularity of a layer’s synaptic memory is that of a synaptic
weight between a pair of pre-synaptic and post-synaptic neurons. Therefore,
each weight in QUANTISENC can be addressed and programmed individually.

The I/O interface to QUANTISENC is as follows.

  * •

wt_in: This interface is used to program synaptic memory by specifying weight
addresses and data. This interface is clocked using mem_clk.

  * •

cfg_in: This interface is used to configure the neuron parameters and the
neuron dynamics. We implement control registers inside the decoder module to
store these parameters. These registers are clocked using mem_clk.

  * •

spk_in/out: This interface is used for data input and output of QUANTISENC.
This interface is clocked using spk_clk, which is the main design clock. Each
spike is encoded using the address event representation (AER) format, which is
a standard representation technique for spikes in the digital domain.

The I/O interface of QUANTISENC is through the system software, which we
elaborate in Section IV.

##  III Detailed Implementation of QUANTISENC

###  III-A Neuron Implementation

At the core of QUANTISENC are leaky integrate and fire neurons (LIF). Figure 2
shows the design of a digital LIF neuron based on the analog implementation
proposed in [4]. A neuron dynamic is defined by the first-order ordinary
differential equation (ODE) as

$\footnotesize\tau\frac{dU(t)}{dt}=-U(t)+R\cdot I_{in}(t)$ (1)

where $U(t)$ is the membrane potential shown as vmem, $\tau=R\cdot C$ is the
time constant of the neuron defined using membrane resistance $R$ and membrane
capacitance $C$, and $I_{in}(t)$ is the input current to the neuron. Equation
1 can be solved using the forward Euler method as

$\footnotesize U(t+\Delta t)=U(t)+\frac{\Delta t}{\tau}\Big{(}-U(t)+R\cdot
I_{in}(t)\Big{)}$ (2)

Equation 2 can be rewritten using neuron parameters as

$\footnotesize U(t+\Delta t)=U(t)-\texttt{decay\\_rate}\cdot
U(t)+\texttt{growth\\_rate}\cdot I_{in}(t)$ (3)

where

$\footnotesize\texttt{decay\\_rate}=\frac{\Delta t}{\tau}=\frac{\Delta
t}{R\cdot C},\text{ and }$ (4)

$\footnotesize\texttt{growth\\_rate}=R\cdot\frac{\Delta t}{\tau}=\frac{\Delta
t}{C}$ (5)

Decay and growth rates are design parameters that are programmed in the
control registers within the decoder module. The digital implementation of
Equation 3 is shown in Figure 2 inside the bounding box marked as VmemDyn.

The input current $I_{j}(t)$ to the neuron $j$ is the neuron activation (act),
which is calculated as the weighted sum of input spikes from all its pre-
synaptic connections as shown in Equation 6 for current-based synapse (CUBA).

$\footnotesize I_{j}^{j}(t)=\sum_{i}x_{ij}\cdot w_{ij},$ (6)

where $x_{ij}$ is the spike from neuron $i$ to neuron $j$ and $w_{ij}$ is the
synaptic weight between these neurons. The bounding box ActGen in Figure 2
shows the implementation of a neuron activation. The address generator
operates on mem_clk to generate memory addresses for all synaptic weights
$w_{ij}$’s. The act_reg is used to store the weighted sum, which is calculated
as follows. For each pre-synaptic connection, the synaptic weight is added to
the weighted sum if there is an input spike on the connection. In general, it
takes $M$ mem_clk cycles to compute the weighted sum for a neuron with $M$
pre-synaptic connections. This implementation allows synaptic memory to be
implemented using dedicated RAM, such as Block RAM (BRAM) or Ultra RAM (URAM)
in FPGA [10]. QUANTISENC can also be configured to store synaptic weights in
Lookup Tables and Flip-flops. We show the performance trade-off for these
design choices in Sec. VI-G.

Figure 3 shows the impact of the $R$ and $C$ settings on membrane dynamics,
when a neuron is excited with a step input of 40 ms duration. We set a
membrane time constant $\tau=5$ ms and a threshold voltage $V_{th}=10$ mv. We
make the following three key observations.

Figure 3: Impact of $R$ and $C$ settings on a neuron dynamics.

First, with large $R$ and small $C$ ($R=500M\Omega$ and $C=10pF$), the
membrane potential increases significantly due to a large growth_rate
(Equation 5), frequently exceeding the threshold voltage and generating a
large number of spikes. Second, reducing $R$ while increasing $C$ (keeping
$\tau=RC$ constant) reduces the growth_rate, which reduces the number of
spikes. Finally, with a small $R=10M\Omega$ and a large $C=500pF$, the
membrane potential does not cross the threshold voltage. Therefore, no spike
is generated from the neuron. We note that both hardware performance (e.g.,
power consumption) and application performance (e.g., accuracy) are function
of the number of spikes generated from a neuron, which depends on the $R$ and
$C$ settings. QUANTISENC allows exploring these parameters to achieve the
desired performance.

The SpkGen bounding box in Figure 2 generates a spike when the membrane
potential $U(t)$ crosses the threshold voltage $V_{th}$. After firing a spike,
the following two independent processes are triggered in a neuron.

####  III-A1 Reset Mechanism

A membrane potential is reset using one of the following four user-defined
reset mechanisms, i.e.,

$\footnotesize U(t)=\begin{cases}V_{reset}&\textbf{Reset-to-Constant}\\\
0&\textbf{Reset-to-Zero}\\\ U(t)-V_{th}&\textbf{Reset-by-Subtraction}\\\
U(t)-\texttt{decay\\_rate}*U(t)&\textbf{Default}\end{cases}$ (7)

Figure 4 shows the reset dynamics of a neuron for a step input of 40 ms. The
default reset behavior is an exponential decay of the membrane potential.
Without a refractory mechanism (which we describe next), the membrane
potential decays only marginally due to continuous excitation, generating a
total of 37 spikes within the 40 ms time window.

Figure 4: Neuron dynamics for different reset mechanisms.

In the Reset-by-Subtraction mechanism, the threshold voltage is subtracted
from the membrane potential of a neuron after generating a spike. This allows
the membrane potential to decay considerably after firing a spike, resulting
in fewer spikes (= 14) than the default reset mechanism. Finally, in the
Reset-to-Zero mechanism, a neuron membrane potential is reset to zero after
firing a spike, requiring additional time for the membrane potential to cross
the threshold voltage and generate a spike. The total number of spikes
generated using this mechanism is the least among the three reset mechanisms.

####  III-A2 Refractory Mechanism

A neuron membrane potential is held constant for a user-defined time duration
called the refractory period after a neuron fires a spike. The refractory
mechanism prevents the neuron from firing another spike in the refractory
window. The design of the refractory mechanism consists of a counter (RefCnt),
which is activated every time a spike is generated. Once activated, the
counter counts down from the user-defined refractory_period to zero at every
edge of spk_clk. During this period of time, the neuron does not generate
spikes. The refractory mechanism regularizes spike trains generated from a
neuron by setting an upper limit on the spike firing frequency
($f_{\text{Max}}$) as

$\footnotesize f_{\text{Max}}\leq\frac{1}{\texttt{refractory\\_period}}$ (8)

The reset techniques and the refractory mechanism are implemented inside the
VmemSel block in Figure 2.

###  III-B Synapse Implementation

Unlike many previous works in which layers are fully connected to each other,
QUANTISENC supports additional connection modalities, allowing it to implement
many modern SNN models. To formally introduce these connections, we represent
the synaptic weight $w_{ij}$ between neurons $i$ and $j$ as
$w_{ij}=\alpha_{ij}\cdot\beta_{ij}\cdot\omega_{i,j}$, where $\omega_{ij}$ is
the absolute synaptic weight. and $\alpha_{ij}$ and $\beta_{ij}$ are weight
parameters defined as follows.

####  III-B1 Synaptic Connections (Network Topology)

Consider a layer $L$ in the design. The connection parameter
$\alpha_{ij}\in\\{0,1\\}$ for layer $L$ is defined as

$\footnotesize\begin{cases}a)~{}~{}\alpha_{ij}=1~{}\forall
i\in(L-1)~{}\&~{}\forall j\in L&\textbf{: all-to-all}\\\
b)~{}~{}\alpha_{ij}=1~{}\textbf{ if }i=j,\text{ and }0\text{
otherwise}&\textbf{: one-to-one}\\\ c)~{}~{}\alpha_{ij}=1~{}~{}\textbf{ if
}|i-j|\leq 1,\text{ and }0\text{ otherwise}&\textbf{: Gaussian}\\\
\end{cases}$ (9)

Figure 5 shows the three different connection modalities supported in
QUANTISENC. For all-to-all connections (also called full connections), each
neuron in layer $L$ receives input from every neuron in layer $(L-1)$. This is
defined in Equation 9a and is shown in Figure 5a. For one-to-one connections,
a neuron at a specific index in a layer is only connected to the neuron at the
same index in the next layer. This is defined in Equation 9b and is shown in
Figure 5b.

Figure 5: Connection modalities supported in QUANTISENC.

Finally, QUANTISENC supports the implementation of the receptive field using
Gaussian connections, to improve the performance of deep neural networks [11].
Here, each neuron of a layer receives input only from neurons within a
predefined region in its preceding layer. A receptive field is defined in
Equation 9c and is shown in Figure 5c.

These basic connection modalities can be combined and extended to implement
skip connections [12] and other types of connection found in modern SNNs.

####  III-B2 Synaptic Polarity (Excitatory vs. Inhibitory)

The polarity parameter $\beta_{ij}$ is used to implement the
inhibitory/excitatory effect of a synaptic connection on neuron activation.
This enables a neural network to function as a universal approximator [13].
The polarity parameter is defined as:

$\footnotesize\beta_{ij}=\begin{cases}1&\textbf{: excitatory effect}\\\
-1&\textbf{: inhibitory effect}\end{cases}$ (10)

###  III-C Signed Neuronal Computations

QUANTISENC uses fixed-point representations for all internal signals with
configurable quantization and decimal precision. We use the generalized Qn.q
notation to represent a signed binary number with $n$ integer bits and $q$
fraction bits [14], where $n$ and $q$ are the design parameters. QUANTISENC
uses the 2’s compliment format to represent its internal signals, where the
most significant bit (MSB) is used to encode the sign. This signed
representation along with signal computations allows the excitatory and
inhibitory polarities of its synaptic weights.

Figure 6: Fixed-point multiplication of two $(n+q)$-bit numbers.

Fixed-point addition and subtraction of two binary numbers follow the exact
same rules as integer addition and subtraction, respectively. However, fixed-
point multiplication requires additional consideration, which we illustrate in
Figure 6 using two $(n+q)$-bit binary numbers a and b. The result of this
multiplication uses $2n+2q$ bits. The final output is obtained by retaining
$(n+q)$ bits of the result. The discarded most significant bits contribute to
the overflow, whereas those from the least significant bits contribute to the
underflow.

###  III-D Configuration Summary

Table I summarizes the different configurations of QUANTISENC and their
corresponding design enabler.

Configuration | Parameters | Implementation | Enabler
---|---|---|---
static | number of layers ($K$) |  |
| number of neurons per layer ($N$) | HDL | application
| layer-to-layer connections ($\alpha,\beta$) | parameters | software
| precision and quantization ($n,q$) |  |
dynamic | growth rate |  | application
| decay rate | control | software &
| threshold voltage ($V_{th}$) | registers | system
| refractory period |  | software
TABLE I: Different configurations of QUANTISENC.

##  IV Hardware-Software Interface

Figure 7a shows the hardware-software interface of QUANTISENC designed for an
FPGA-based computing system. User code, which forms the application software,
is written using PyTorch-based SNN constructs [15, 16, 17]. With moderate
effort, QUANTISENC can be extended to support other SNN frameworks such as
Brian [18] and CARLsim [19].

In our system architecture, the application software runs on the processing
system (PS) of the FPGA. It interfaces with the hardware implemented on the
programmable logic (PL). For AMD’s Virtex Ultrascale evaluation board, the PS
is composed of MicroBlaze with PetaLinux OS. For AMD’s Zynq MPSoC products,
the MicroBlaze soft core can be replaced with ARM Cortex cores. We designed
open-source Python libraries to generate the configuration parameters of
QUANTISENC in Verilog Hardware Description Language (HDL) and a testbench in
SystemVerilog (sv) language. The testbench is used to verify the design
through (1) behavioral simulation (using the HDL), (2) functional simulation
(using synthesized netlist), and (3) timing simulation (using implemented
netlist) in AMD’s Vivado tool suite. QUANTISENC’s bitstream is implemented on
the PL. The MicroBlaze software stack interfaces with the PL block to program
synaptic weights, drive input, and visualize hardware results using the high-
speed AMBA AXI interconnect. We use timing simulations to extract the toggle
rate of the nets, which is then used to calculate the average dynamic power
consumption. This power estimation framework can be integrated inside a loop
(shown to the left) to do power/energy-aware co-design.

Figure 7: Hardware-software interface of QUANTISENC.

Finally, Figure 7b shows an ASIC design flow using QUANTISENC targeted for
accelerator-based designs. The ASIC flow consists of front-end design steps
that include synthesis and static timing analysis and back-end design steps
that include placement, routing, and layout. After fabrication and testing,
the neuromorphic accelerator is connected to the host CPU inside a system-on-
chip (SoC) using peripheral interconnect express (PCIe). The hardware-software
interface consists of the application software and the system software (e.g.,
Linux OS). A key aspect of our design methodology is that the application
software and the QUANTISENC hardware are the same for both the FPGA and ASIC
flows. Section VI-K provides early results obtained using ASIC synthesis.

A key novelty of QUANTISENC is that synaptic memory is distributed among
hardware layers. This distribution allows the layers to operate in parallel.
Although QUANTISENC performs layer-by-layer processing of spikes for any given
input data stream (inherently a dataflow approach), we exploit pipelined
parallelism in the design by overlapping the processing of streaming data via
the system software [20]. This is in addition to batch-level parallelism that
is supported using QUANTISENC cores in a multi-/many-core setting. 111A batch-
level parallelism is a type of parallel computing environment in a
neuromorphic hardware where multiple batches of input data are processed
concurrently on different processing elements [21].

We illustrate the processing of input data streams in Figure 8. Before
execution begins, the synaptic weights and design parameters are loaded into
the synaptic memory and control registers, respectively. Thereafter, data
streams are scheduled one after another with a delay equal to the processing
time of a layer ($d$) and a waiting time ($s$), as illustrated in the figure.
This waiting time ensures that the membrane potential of a neuron resets to
its resting potential before it begins processing spike trains from the next
input stream. In steady state, the maximum throughput obtained using this
pipelined parallelism is $1/(d+s)$.

Figure 8: Pipelined data processing using the hardware-software interface.

##  V Distinction from State-of-the-Art

Although high-level synthesis (HLS) using commercial tools such as Vitis [22]
and academic platforms such as Bambu [23] can generate HDL starting from a
software specification, there are four key limitations of neuromorphic HLS
that we address using QUANTISENC. First, the HDL generated from HLS is not
parameterizable. Therefore, any change in software description requires
regenerating its HDL using the HLS flow, as illustrated in Figure 9a. Large
run times prohibit the use of HLS flow inside a co-design loop, where several
hundred iterations may be necessary to achieve the required hardware and
software performance for a target SNN model. Second, most HLS tools do not
support neuromorphic computations, and those that do support require a
significant amount of engineering effort at the kernel level for compilation
and mapping, in addition to the effort needed in hardware development to add
new computations or extend existing ones, as shown in our recent work [24].
Third, most HLS tools work with dataflow graphs extracted from machine
learning applications and perform static scheduling on them, exploiting only
batch-level hardware parallelism. There remains a significant opportunity to
improve performance, such as throughput, by taking advantage of other forms of
parallelism that are present in neuromorphic hardware. Finally, the HDL
generated from HLS is not ASIC compatible [25, 26, 27].

Figure 9: (a) A High-Level Synthesis (HLS) flow within a co-design loop. (b)
Our design methodology integrating QUANTISENC within a co-design loop.

Figure 9b shows our design methodology. QUANTISENC can be customized to match
the exact specification of a user code (application software) simply by
changing the design parameters. Our fully automated design flow from model
specification in software to hardware design and verification is the core of
our hardware-software co-design. It does not require optimization at the
kernel level, unlike an HLS. This reduces the time to evaluate each design
refinement, which reduces the overall co-design time.

Table II compares QUANTISENC with recent state-of-the-art designs in terms of
relevant neuromorphic features. A comprehensive treatment of all existing
designs is beyond the scope of this paper. Here, we compare key techniques.

| Year |  Operations |  Neuron Dynamics |  Reset Mechanism |  Refractory Period |  Synapse Model |  Static Configuration |  Dynamic Configuration |  Programming Interface | Parallelism |  Open Source
---|---|---|---|---|---|---|---|---|---|---|---
[28] | 2018 |  Floating Point (32 bits) |  – |  – |  – |  Excitatory |  Model Size |  $\times$ |  Custom C-based | $\times$ |  $\times$
[29] | 2021 |  Floating Point (16 bits) |  Nonlinear |  Reset-to-Constant |  $\surd$ |  Excitatory |  Model Size |  $\times$ |  Bare Metal | $\times$ |  $\times$
[30] | 2021 |  – |  Linear |  Reset-to-Zero |  $\surd$ |  Excitatory |  Model Size, Quantization |  $\times$ |  Custom Python-based | $\times$ |  $\times$
[31] | 2022 |  Fixed Point (16 bits) |  Nonlinear |  Reset-to-Constant |  – |  Excitatory |  Model Size, Quantization |  $\times$ |  Bare Metal | $\times$ |  $\times$
[32] | 2023 |  Fixed Point (8 bits) |  Linear |  Reset-by-Subtraction |  – |  Excitatory |  Model Size, Quantization |  $\times$ |  Bare Metal | $\times$ |  $\times$
QUANTISENC | 2023 |  Fixed Point (Qn.q) |  Nonlinear |  Reset-to-Constant, Reset-to-Zero, Reset-by-Subtraction, Exponential Decay |  – |  Excitatory & Inhibitory |  Model Size, Quantization, Connectivity |  Decay Rate, Growth Rate, Threshold Voltage, Refractory Period |  PyTorch | $\surd$ |  $\surd$
TABLE II: Comparing QUANTISENC against recent state-of-the-art designs in
terms of key neuromorphic features.

##  VI Results

###  VI-A Evaluation Methodology

We evaluate QUANTISENC using Spiking MNIST [7], DVS Gesture [8], and Spiking
Heidelberg Digit (SHD) [9] datasets on AMD’s Virtex Ultrascale (primary),
Virtex 7, and Zynq Ultrascale FPGA development boards. Table III summarizes
these datasets and evaluation boards. For each dataset, we report the number
of classes, training, and test examples (in columns 1-4). For each evaluation
board, we report the technology and resources expressed in terms of Lookup
Tables (LUTs), Flip-Flops (FFs), Block RAMs (BRAMs), and DSP Slices (in
columns 5-7).

Datasets | Classes |  Training Examples |  Test Examples |  Evaluation Board | Technology |  Resources
---|---|---|---|---|---|---
Spiking MNIST [7] | 10 |  60,000 |  100 |  Virtex UltraScale | 16nm FinFET |  LUTs: 537,600 FFs: 1,075,200 BRAMs: 1728 DSPs: 768
DVS Gesture [8] | 11 |  1176 |  288 |  Virtex 7 | 28nm |  LUTs: 303,600 FFs: 607,200 BRAMs: 1030 DSPs: 2800
Spiking Heidelberg Digit (SHD) [9] | 20 |  8156 |  2264 |  Zynq UltraScale | 16nm FinFET |  LUTs: 230,400 FF: 460,800 BRAM: 312 DSP: 1728
TABLE III: Datasets and FPGA boards used to evaluate QUANTISENC.

We use SNNTorch [15] to train SNN models on a Lambda workstation, which has
AMD Threadripper 3960X with 24 cores, 128 MB cache, 128 GB RAM and 2 RTX3090
GPUs. The trained weights are programmed into the QUANTISENC hardware (in its
synaptic memory) to perform classifications.

Figure 10: A classification example with the spiking MNIST dataset using the
handwritten digit 8. QUANTISENC is configured as a $256\times 128\times 10$
architecture.

Figure 10 shows a classification example using the spiking MNIST dataset for
the handwritten digit 8. The configuration of QUANTISENC is set to $(256\times
128\times 10)$ with a quantization of 8 bits and a decimal precision of 3
bits. i.e., using the Q5.3 fixed-point representation.222We scale the original
$28\times 28$ images to $16\times 16$ images to reduce model size. This is the
maximum scaling that can be performed on handwritten images from the dataset
without affecting the classification accuracy.  This baseline configuration of
QUANTISENC gives the best trade-off for the Spiking MNIST dataset in terms of
application performance, e.g., accuracy of digit recognition (96.5%), and
hardware performance, e.g., area, power, latency, and throughput.

In Figure 10, the image is presented to QUANTISENC for a time duration of 150
ms. This is a user-defined parameter and is controlled through the application
software. The figure shows the spikes generated by neurons of the three layers
(input, hidden, and output). We use a spike counter on the output layer to
decode and visualize the classification result of QUANTISENC. This is
illustrated in Figure 11.

Figure 11: Counting output spikes to decode QUANTISENC results.

We observe that output neuron 8 has the highest spike count, so the QUANTISENC
result is correctly interpreted as 8 corresponding to the image. However,
neuron 3 has the second highest number of spikes, followed by neuron 0. This
is because of the structural similarity of the handwritten digit 8 to digits 3
and 0. When analyzing the confusion matrix, we observe that in most cases
where a handwritten digit 8 is predicted incorrectly, it is predicted as a
digit 3 or a digit 0.

###  VI-B LIF Design

Table IV reports the utilization of resources and the dynamic peak power of
our LIF design for five different quantization and precision settings using a
spike frequency of 100 MHz. We make the following three key observations.

|  Quantization & Precision | LUTs | FFs | BRAMs | DSPs |  Dynamic Peak Power (mW)
---|---|---|---|---|---|---
1 |  binary | 14 | 11 | 0 | 0 |  3
2 |  4 bits (Q2.2) | 66 | 19 | 0 | 0 |  4
3 |  8 bits (Q5.3) | 245 | 35 | 0 | 0 |  6
4 |  16 bits (Q9.7) | 242 | 68 | 0 | 2 |  14
5 |  32 bits (Q17.15) | 856 | 132 | 0 | 8 |  27
TABLE IV: Resource utilization of the proposed LIF design for different
settings of quantization and precision.

First, resource utilization (LUTs, FFs, etc.) increases with increasing
quantization. A 32-bit quantized LIF design uses 61x more LUTs and 12x more
FFs compared to a 2-bit quantized design. This increase is due to the
complexity of building an accurate LIF model in hardware. Second, the increase
in dynamic power, on the other hand, is less significant than resource
utilization. This is due to the extensive use of clock gating in the design,
which minimizes dynamic power. A 32-bit quantized LIF design consumes 9x more
dynamic power than a 2-bit quantized design. Third, DSP modules are utilized
in the implementation for a quantization of 16 bits or higher.

###  VI-C Neural Connections

Table V reports the utilization of resources and dynamic power for three
different connections supported in QUANTISENC – one-to-one in row 1, Gaussian
(convolution) in rows 2 & 3, and all-to-all (fully connected) in rows 4, 5, &
6.

|  | Connections | LUTs | FFs | BRAMs | DSPs |  Dynamic Peak Power (mW)
---|---|---|---|---|---|---|---
1 | one-to-one | $1$ | 296 | 56 | 0 | 0 |  12
2 | convolution | $3\times 3$ | 284 | 80 | 0.5 | 0 |  17
3 | $5\times 5$ | 300 | 130 | 0.5 | 0 |  18
4 | fully connected | $128$ | 420 | 443 | 0.5 | 0 |  23
5 | $256$ | 551 | 829 | 0.5 | 0 |  29
6 | $512$ | 822 | 1599 | 0.5 | 0 |  48
TABLE V: Resource utilization and peak dynamic power for different connection modalities in QUANTISENC. | Configuration | Neurons | Synapses |  Quantization & Precision |  LUTs (of 537,600) | $\mathbf{\Delta}$ |  FFs (of 1,075,200) | $\mathbf{\Delta}$ |  BRAMs (of 1728) | $\mathbf{\Delta}$ |  DSPs (of 768) | $\mathbf{\Delta}$ |  Dynamic Power (W) | $\mathbf{\Delta}$
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
1 | $256\times 128\times 10$ | 394 | 34,048 |  Q5.3 |  8.97% | – |  0.98% | – |  3.99% | – |  0% | – |  0.623 | –
2 | $256\times 128\times 10$ | 394 | 34,048 |  Q9.7 |  9.38% | 4.5% |  1.39% | 42.2% |  3.99% | 0% |  35.93% | – |  0.738 | 18.5%
3 | $256\times 256\times 10$ | 522 | 68,096 |  Q5.3 |  17.44% | 1.9$\times$ |  1.85% | 1.9$\times$ |  7.69% | 1.9$\times$ |  0% | – |  1.241 | 2.0$\times$
4 | $256\times 256\times 256\times 10$ | 778 | 133,632 |  Q5.3 |  34.08% | 3.8$\times$ |  3.55% | 3.6$\times$ |  15.10% | 3.8$\times$ |  0% | – |  2.172 | 3.5$\times$
TABLE VI: Resource utilization and dynamic power of QUANTISENC for different
SNN architectures.

We make the following three observations. First, a neuron with a one-to-one
connection to another neuron uses 296 LUTs and 56 FFs for the quantization and
precision of Q5.3. The total dynamic power is 12 mW (after implementation).
The utilization of resources is comparable to that of a single neuron (see row
3 of Table IV) with additional resources to implement synaptic weight between
pre-synaptic and post-synaptic neurons. Second, the Gaussian connection can be
utilized to implement convolution filters. We report the utilization and peak
dynamic power of the $3\times 3$ and $5\times 5$ filters. We observe that
although there are more synaptic weights in a $3\times 3$ convolution filter
than in a one-to-one connection, the former configuration uses fewer LUTs.
This is because it uses BRAMs to implement synaptic weights compared to the
latter configuration, which uses LUTs to implement synaptic weight. Finally,
we report fully connected configurations with 128, 256, and 512 pre-synaptic
connections per neuron. We observe that with $2\times$ more connections (row 5
vs. 4), the number of LUTs and FFs increases by $1.3\times$ and $1.87\times$,
respectively. The peak dynamic power increases by only 6 mW. On the other
hand, with $4\times$ more connections (row 6 vs. 4), the number of LUTs and
FFs increases by $1.95\times$ and $3.6\times$, respectively. The dynamic peak
power increases by $2\times$.

The results presented in Sections VI-B & VI-C clearly show the scalability of
QUANTISENC in terms of its neuron configuration and connection modalities. In
the following, we provide an in-depth architectural analysis of QUANTISENC for
the Spiking MNIST dataset using the baseline configuration of $(256\times
128\times 10)$ with a quantization and precision of Q5.3 (Sections VI-D–VI-I).
We summarize such architectural analysis for the other two datasets in Section
VI-J.

###  VI-D Architectural Analysis for Spiking MNIST Dataset

Row 1 of Table VI reports the utilization and dynamic power consumption of
QUANTISENC for the baseline configuration of $256\times 128\times 10$ and a
quantization of 8 bits with a decimal precision of 3 bits, i.e., using Q5.3
fixed-point representation. The design consists of 394 LIF neurons and 34,048
synapses. It uses BRAMs to implement synaptic memory. The utilization of
resources for the Virtex UltraScale FPGA is as follows: 48,246 (= 8.97%) LUTs,
10,550 (= 0.98%) FFs, and 69 (= 3.99%) BRAMs. No DSP slices are used. Dynamic
power consumption is 623 mW using a spike frequency of 600 KHz. This frequency
gives the maximum performance per watt for the design (see Section VI-G).

Row 2 reports the utilization when the quantization is increased to 16 bits
and the decimal precision to 7 bits, i.e., using Q9.7 fixed-point
representation. Increasing quantization and precision results in an increase
of 4.5% in LUTs and 42.2% in FFs compared to the baseline configuration in row
1. The number of BRAMs remains the same, but the design uses 276 (= 35.93%)
DSPs. Dynamic power increases by 18.5%.

Rows 3 & 4 show the utilization and dynamic power for two additional design
configurations. The configuration in row 3 has 522 neurons (32.5% higher than
baseline) and 68,096 synapses (2$\times$ higher than baseline). For this
configuration, the design requires 1.9$\times$ more LUTs, 1.9$\times$ more
FFs, and 1.9$\times$ more BRAMs than the baseline. The dynamic power is
2$\times$ higher. On the other hand, the configuration in row 4 has 778
neurons (2$\times$ higher than baseline) and 133,632 synapses (4$\times$
higher than baseline). For this configuration, the design requires 3.8$\times$
more LUTs, 3.6$\times$ more FFs, and 3.8$\times$ more BRAMs than the baseline.
The dynamic power is 3.5$\times$ higher.

These results show the scalability of QUANTISENC with the number of neurons
and synapses. One way to utilize this scalability result would be to quickly
evaluate the utilization of FPGA resources for a specific configuration of
QUANTISENC, without having to synthesize the design, which often takes a
considerable amount of time. This is useful when conducting design space
exploration, where multiple iterations may be necessary before making a final
design choice.

###  VI-E Comparison to State-of-the-Art

Table VII compares resource utilization, power, and accuracy for single
neurons and the SNN architecture for the Spiking MNIST dataset against latest
state-of-the-art designs. We make the following four key observations.

First, for a single neuron (columns 2, 3, & 4), the Euler approximation in the
proposed implementation uses more LUTs and FFs than [33] and [34]. This is
because, in the proposed design, a neuron’s refractory period, reset
mechanism, growth rate, decay rate, and threshold voltage can all be
configured at run-time using a combination of the application and system
software. The extra logic to implement such configurability increases the
resource utilization, but at the same time provides flexibility in exploring
the trade-off between accuracy and power consumption even after a design is
implemented on silicon. Neuron implementation in [33] & [34] can only be
configured statically. Once these designs are implemented on silicon, they do
not offer run-time configurability. Second, the power consumption of the
proposed design is lower than [33] due to the extensive use of clock gating.
Specifically, we gate the clock in the design when there is no input spike.

Third, for a complete SNN architecture, the proposed design uses fewer neurons
and synapses to achieve an accuracy comparable to that of the state-of-the-art
[28, 35]. Fourth, the highest accuracy (reported in the literature) is
obtained using [28]. However, this design consumes 3.4 W of dynamic power.
Compared to this design, our design achieves slightly lower accuracy (96.5%
vs. 98.4%) while consuming only 623 mW of dynamic power. Finally, the most
optimized hardware design is proposed in [35]. The resource utilization is
significantly lower because it uses a high-level synthesis approach to
optimize the design, including removing extra design logic that is required to
configure the design at run-time. In addition to the lack of configurability
and ASIC portability, this design consumes 1.03 W of dynamic power and
achieves an accuracy of 93%. Compared to this, the proposed design consumes
less power and achieves higher accuracy.

We conclude that the proposed design achieves accuracy comparable to state-of-
the-art designs while using fewer neurons and synapses. It also consumes less
dynamic power.

| A Single Neuron | SNN Architecture
---|---|---
| Euler [33] | Euler [34] | Ours |  Best Accuracy [28] |  Best Hardware [35] | Ours
Configuration | – | – | – |  784-1024-10 |  784-2048-10 | 256-128-10
Neurons | – | – | – |  1818 |  2932 | 394
Synapses | – | – | – |  813,056 |  1,810,432 | 34,048
LUTs | 95 | 76 | 108 |  78,679 |  16,813 | 40,965
FFs | 85 | 20 | 23 |  16,864 |  7,559 | 7,095
BRAMs | 0 | 0 | 0 |  174 |  129 | 69
Power (W) | 0.25 | NR | 0.05 |  3.4 |  1.03 | 0.623
Accuracy | – | – | – |  98.4% |  93.0% | 96.5%
TABLE VII: comparison to state-of-the-art.

###  VI-F Software vs. Hardware Performance

Figure 12 plots the membrane potential of a neuron in the hidden layer of
QUANTISENC with the baseline configuration of $(256\times 128\times 10)$ for
the handwritten digit 8 image from the spiking MNIST dataset. We evaluate
three quantization and precision settings: Q9.7 (16-bit quantization with
7-bit decimal precision), Q5.3 (8-bit quantization with 3-bit decimal
precision), and Q3.1 (4-bit quantization with 1-bit decimal precision). For
reference, we also include the membrane potential obtained using SNNTorch
(software), which uses a double-precision floating-point representation.

Figure 12: Impact of quantization on neuron membrane potential.

For easier comparison of the results, the subplot inside the plot zooms in on
the time scale between 70 and 80 ms. We observe that the membrane potential
using the Q9.7 fixed-point representation is close to that using the double
precision floating-point representation. The average root mean square error
(RMSE) of all 394 neurons for 100 randomly selected test examples from the
spiking MNIST dataset is 0.25 mV. The average RMSE increases to 0.43 mV using
the Q5.3 representation. Finally, the RMSE using the Q3.1 representation is
2.12 mV.

Table VIII reports the accuracy comparison between SNNTorch (software) and
QUANTISENC (hardware) for the Spiking MNIST dataset. We compare results for
three different quantization and precision settings: Q9.7, Q5.3, and Q3.1. We
make the following three key observations.

Dataset | SNNTorch | Average Test Set Hardware Accuracy
---|---|---
Accuracy |  Q9.7 |  Q5.3 |  Q3.1
Spiking MNIST | 97.8% |  97.1% |  96.5% |  88.3%
TABLE VIII: Accuracy with different quantization and precision settings.

First, the accuracy with a quantization of 16 bits and a decimal precision of
7 bits (Q9.7) is comparable to SNNTorch software. This is because the membrane
potential value obtained in hardware using this setting is close to that in
software (see Figure 12) due to our detailed LIF design (see Figure 2).
Second, the accuracy reduces with a reduction in quantization and decimal
precision. The accuracy using 8-bit and 4-bit quantization is 96.5% and 88.3%,
compared to 97.1% using 16-bit quantization and 97.8% using SNNTorch software.

###  VI-G Design Throughput and Throughput Per Watt

Figure 13 plots the worst setup slack (in ns) of QUANTISENC as we increase the
spike frequency from 100 KHz to 1.2 MHz using the baseline configuration of
$(256\times 128\times 10)$ and Q5.3 quantization and decimal precision. Setup
slack is defined as the difference between the required time and the arrival
time of the data at an endpoint (typically a register). During static timing
analysis (STA), a negative setup slack indicates timing violations. The peak
frequency is one that results in the least positive setup slack. Figure 13
reports the setup slack for three synaptic memory implementations: BRAM,
registers, and distributed LUTs. The subplot reports the dynamic power of
QUANTISENC for these memory settings. We make the following three key
observations.

Figure 13: Maximum frequency and power trade-off.

First, the setup slack is positive for a spike frequency of 100, 200, and 400
KHz for all three memory implementations. As we increase the frequency to 600
KHz, there are multiple timing violations for register-based synaptic memory.
The peak frequency for this implementation is 500 KHz. Second, the setup slack
for distributed LUT-based implementation is 61% lower than for BRAM-based
implementation, which means that the latter supports a higher peak frequency.
Our results show that the peak spike frequencies for these two implementations
are 850 and 925 KHz, respectively. Finally, the distributed LUT implementation
has the least dynamic power for all spike frequencies. It is 23% and 79% lower
than the BRAM and register-based implementations, respectively.

In addition to the peak frequency, we also evaluate QUANTISENC based on its
real-time performance, which is measured as the number of images inferred per
second, and throughput per watt, which is measured as the number of fixed-
point operations performed per watt. These are defined as

$\footnotesize\text{Real-time Performance}=\frac{1}{\text{exposure
time}+N_{\text{reset}}/f}\text{ (see Sec.~{}\ref{sec:software})}$ (11)

where exposure time is the time interval for which each image is exposed to
the SNN model for inference, $N_{\text{reset}}$ is the number of clock cycles
needed to reset the membrane potential, and $f$ is the spike frequency. The
value $N_{\text{reset}}$ depends on the membrane time constant ($\tau$). Our
empirical studies show that $N_{\text{reset}}=4$ clock cycles at $f=1\text{
KHz}$ for $\tau=5$ ms. This results in a real-time performance of 41.67 frames
per second (fps) for an exposure time of 20 ms. This is the performance
obtained by exploiting pipelined parallelism in QUANTISENC. In previous works
such as [30], such parallelism has not been exploited. Therefore, the real-
time performance is $\frac{1}{\text{exposure time}+(K\times L)/f}$, where $K$
is the number of layers and $L$ is the latency (in the number of clock cycles)
of each layer. For the baseline design $(256\times 128\times 10)$ with three
layers, the maximum performance obtained using [30] is 31.25 fps. We conclude
that QUANTISENC improves the real-time performance by 33. 3% by exploiting
pipelined parallelism.

The throughput per watt is computed as the ratio of the total number of fixed-
point operations per second to power consumption. Due to pipeline execution in
QUANTISENC, all synaptic accumulations in all layers occur in parallel, where
each accumulation involves a single fixed-point operation. In addition, all
neuron computations also take place in parallel, working on different inputs.
Therefore, the total number of fixed-point operations in one second is

$\footnotesize\text{Fixed-Point
Ops}=\left(N_{\text{synapse}}+N_{\text{ops}}\times
N_{\text{neurons}}\right)\times f$ (12)

where $f$ is the design frequency and $N_{\text{ops}}$ is the number of fixed-
point operations per neuron.

Figure 14 plots the performance per watt for the three designs evaluated in
Table VI as the spike frequency increases from 100 KHz to 1 MHz. All these
designs use BRAM for synaptic implementation. We make the following
observations.

Figure 14: Performance per watt for different SNN architectures using BRAMs.

First, the performance per watt increases with an increase in frequency until
it reaches the maximum. This is because at lower frequencies the increase in
performance (see Equation 12) exceeds the increase in dynamic power, resulting
in an overall increase in performance per watt for all evaluated designs. With
a further increase in frequency, the dynamic power starts to dominate the
performance-power ratio, resulting in a reduction in performance per watt. The
maximum performance per watt for each design configuration is indicated with a
circle in Figure 14, which occurs at a frequency much lower than the highest
frequency supported by the respective design.

###  VI-H Static Configuration of QUANTISENC

Table IX reports the largest configuration of QUANTISENC that can be
implemented on three evaluated FPGA platforms. We report the configuration for
a wide QUANTISENC design using a single hidden layer and a deep QUANTISENC
design using multiple hidden layers. We also report the power consumption of
these configurations. Our software-defined hardware design methodology allows
us to easily explore different design configurations for a target FPGA
platform.

Platform | Wide Design (Single Hidden Layer) | Deep Design (Multiple Hidden Layers)
---|---|---
Configuration | Power |  Configuration | Power
Virtex UltraScale |  256-1470-10 | 9.557 W |  256-28(64)-10 | 6.371
Virtex 7 |  256-704-10 | 5.818 W |  256-20(64)-10 | 4.833
Zynq UltraScale |  256-640-10 | 3.349 W |  256-12(64)-10 | 1.854
TABLE IX: QUANTISENC configuration on three FPGA platforms.

We observe that the Virtex UltraScale platform has more resources (LUTs, FFs,
etc.) than the other two evaluated platforms (see Table III). Therefore, the
largest configuration supported on this platform is also the highest for both
wide and deep designs. To implement such a large design, the power consumption
is also significantly higher.

###  VI-I Dynamic Configuration of QUANTISENC

Table X reports the average spikes per neuron, dataset accuracy, and dynamic
power for four R & C settings (keeping the time constant $\tau$ = 5 ms), three
reset mechanisms (default  ❶ , reset-by-subtraction  ❷ and reset-to-zero  ❸ ),
and two refractory periods (of 0 and 5 cycles).

| R, C Settings | Reset | Ref.
---|---|---|---
|  R=500M$\Omega$ C=10pF |  R=100M$\Omega$ C=50pF |  R=50M$\Omega$ C=100pF |  R=10M$\Omega$ C=500pF |  ❶ |  ❷ |  ❸ | 0 | 5
Avg. Spikes per Neuron |  26 |  19 |  7 |  0 | 45 | 26 | 22 | 26 | 20
Dataset Accuracy (%) |  96.5 |  94.4 |  67.8 |  – | 92.7 | 96.5 | 96.5 | 96.5 | 95.8
Dynamic Power (mW) |  663 |  541 |  449 |  – | 1,087 | 663 | 625 | 663 | 580
TABLE X: Impact of dynamic settings of QUANTISENC.

We make the following three observations. First, the growth rate of the
membrane potential of a neuron is high for a large R (500 M$\Omega$) and a
small C (10 pF), resulting in an average of 26 spikes per neuron. This is the
same setting that was used in SNNTorch to train the model for the spiking
MNIST dataset. The accuracy is 96.5%. Reducing R to 100 M$\Omega$ and
increasing C to 50 pF results in a reduction in the number of spikes per
neuron. Accuracy is reduced only marginally. However, a reduction in the
number of spikes results in a significant reduction in dynamic power (18.4%).
With a further reduction in R (and increase in C), the number of spikes per
image decreases. Although the dynamic power is reduced by 33%, the accuracy is
much lower. With a small R (10M$\Omega$) and a large C (500 pF), the membrane
potential does not exceed the threshold. As a result, no spikes are generated.

Second, the baseline reset setting is reset-by-subtraction (column 7). Using
the default reset mechanism increases the number of spikes per neuron, which
reduces the accuracy and increases the dynamic power. Reset-to-zero results in
fewer spikes per neuron. The accuracy remains the same as the baseline
configuration. However, the dynamic power is lower.

Finally, the number of spikes per neuron decreases with increasing refractory
period. Using our design methodology, all these parameters can be configured
at run-time to explore the trade-off between performance and power.

###  VI-J Dataset Results

Table XI summarizes the results for the three evaluated datasets. We show a
thorough evaluation for the Spiking MNIST dataset in Sections VI-D–VI-I. These
results are summarized in row 1 of Table XI.

|  | Configuration | Resource Utilization | Accuracy |  Dynamic Peak Power |  Peak Performance per Watt
---|---|---|---|---|---|---
|  |  LUTs |  FFs |  BRAMs |  (W) |  (GOPS/W)
1. |  Spiking MNIST [7] | 256-128-10 |  9% |  1% |  4% | 96.5% |  0.623 |  36.6
2. |  DVS Gesture [8] | 400-300-300-11 |  60% |  15% |  18% | 85.07% |  1.827 |  24.45
3. |  SHD [9] | 700-256-256-20 |  65% |  20% |  24% | 87.8% |  1.629 |  16.09
TABLE XI: Design summary for different datasets.

Row 2 summarizes our design exploration for the DVS Gesture dataset. The
design requires a configuration of $(400\times 300\times 300\times 11)$, which
uses 60% LUTs, 15% FFs, and 18% BRAMs on VirtexUltraScale. The accuracy
obtained is 85.07% compared to an SNNTorch accuracy of 87.1% [36]. The
implemented design has a peak dynamic power of 1.827 W and a performance per
watt of 24.45.

Row 3 summarizes our design exploration for the SHD dataset. The design
requires a configuration of $(700\times 256\times 256\times 20)$, which uses
65% LUTs, 20% FFs, and 24% BRAMs on VirtexUltraScale. The accuracy obtained is
87.8% compared to the SNNTorch accuracy of 90.3% [37]. The implemented design
has a peak dynamic power of 1.629 W and a performance per watt of 16.09.

###  VI-K Early ASIC Synthesis Results

Table XII reports the results of the early ASIC synthesis of the proposed LIF
design using the Synopsys Design Compiler for a spike frequency of 100 MHz.
The proposed design uses 1,574 nets, 944 combinatorial cells, 35 sequential
cells (Flip-Flops), and 309 buffers & inverters, occupying a total area of
2894 $\mu m^{2}$. The design consumes 101.7 $\mu$ W of power, divided into
23.2 $\mu$ W of switching (dynamic) power and 78.5 $\mu$W of leakage power.
Our future work will explore other stages of ASIC design as illustrated in
Figure 7(c).

Technology | Nets |  Comb. Cells |  Seq. Cells | Buf/Inv | Area |  Switching Power |  Leakage Power |  Total Power
---|---|---|---|---|---|---|---|---
32nm | 1574 |  944 |  35 | 309 | 2894 $\mu m^{2}$ |  23.2 $\mu$W |  78.5 $\mu$W |  101.7 $\mu$W
TABLE XII: Early ASIC synthesis results of a Q5.3 LIF neuron.

##  VII Conclusion

We propose QUANTISENC, a fully configurable open source neuromorphic core
architecture designed to advance research in neuromorphic computing.
QUANTISENC distinguishes itself through its hierarchical design methodology,
which allows system designers to configure key parameters such as the number
of layers, neurons per layer, layer-to-layer connectivity, neuron dynamics,
quantization, and precision through the software, both at the design time and
at run-time. The proposed framework allows to define and train SNN models
using PyTorch-based simulators and assess their hardware performance through
FPGA prototyping and ASIC design. QUANTISENC’s performance has been evaluated
using datasets like Spiking MNIST, DVS Gesture, and Spiking Heidelberg Digit
(SHD) on AMD’s Virtex 7, Virtex Ultrascale, and Zynq Ultrascale FPGA
development boards, demonstrating superior resource utilization, power
efficiency, latency, and throughput compared to state-of-the-art designs.

## Acknowledgments

This material is based upon work supported by Accenture LLP, the U.S.
Department of Energy under Award Number DE-SC0022014 and by the National
Science Foundation under Grant Nos. OAC-2209745, CCF-1942697, & CNS-2008167.

## References

  * [1] C. Mead, “Neuromorphic electronic systems,” _Proceedings of the IEEE_ , vol. 78, no. 10, pp. 1629–1636, 1990.
  * [2] A. Joubert, B. Belhadj, O. Temam, and R. Héliot, “Hardware spiking neurons design: Analog or digital?” in _International Joint Conference on Neural Networks (IJCNN)_. IEEE, 2012.
  * [3] W. Maass, “Networks of spiking neurons: The third generation of neural network models,” _Neural Networks_ , vol. 10, no. 9, pp. 1659–1671, 1997.
  * [4] G. Indiveri, “A low-power adaptive integrate-and-fire neuron circuit,” in _International Symposium on Circuits and Systems (ISCAS)_ , 2003.
  * [5] M. Ward and O. Rhodes, “Beyond LIF neurons on neuromorphic hardware,” _Frontiers in Neuroscience_ , vol. 16, p. 881598, 2022.
  * [6] B. Cessac and T. Viéville, “On dynamics of integrate-and-fire neural networks with conductance based synapses,” _Frontiers in Computational Neuroscience_ , vol. 2, p. 228, 2008.
  * [7] M. Fatahi, M. Ahmadi, M. Shahsavari, A. Ahmadi, and P. Devienne, “evt_mnist: A spike based version of traditional mnist,” _arXiv_ , 2016.
  * [8] A. Amir, B. Taba, D. Berg, T. Melano, J. McKinstry, C. Di Nolfo, T. Nayak, A. Andreopoulos, G. Garreau, M. Mendoza _et al._ , “A low power, fully event-based gesture recognition system,” in _Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017.
  * [9] B. Cramer, Y. Stradmann _et al._ , “The Heidelberg spiking data sets for the systematic evaluation of spiking neural networks,” _IEEE Transactions on Neural Networks and Learning Systems (TNNLS)_ , 2020.
  * [10] M. Gungor, K. Huang, S. Ioannidis, and M. Leeser, “Optimizing designs using several types of memories on modern FPGAs,” in _High Performance Extreme Computing (HPEC)_ , 2022.
  * [11] W. Luo, Y. Li, R. Urtasun, and R. Zemel, “Understanding the effective receptive field in deep convolutional neural networks,” _Advances in Neural Information Processing Systems (NeurIPS)_ , 2016.
  * [12] T. Tong, G. Li, X. Liu, and Q. Gao, “Image super-resolution using dense skip connections,” in _Intl. Conf. on Computer Vision (ICCV)_ , 2017.
  * [13] Q. Wang, M. A. Powell, A. Geisa, E. Bridgeford, C. E. Priebe, and J. T. Vogelstein, “Why do networks have inhibitory/negative connections?” in _International Conference on Computer Vision (ICCV)_ , 2023.
  * [14] R. Yates, “Fixed-point arithmetic: An introduction,” _Digital Signal Labs_ , vol. 81, no. 83, p. 198, 2009.
  * [15] J. K. Eshraghian, M. Ward, E. O. Neftci, X. Wang, G. Lenz, G. Dwivedi, M. Bennamoun, D. S. Jeong, and W. D. Lu, “Training spiking neural networks using lessons from deep learning,” _Proc. of the IEEE_ , 2023.
  * [16] W. Fang, Y. Chen, J. Ding, Z. Yu, T. Masquelier, D. Chen, L. Huang, H. Zhou, G. Li, and Y. Tian, “SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligence,” _Science Advances_ , vol. 9, no. 40, p. eadi1480, 2023.
  * [17] P. Spilger, E. Arnold, L. Blessing, C. Mauch, C. Pehle, E. Müller, and J. Schemmel, “hxtorch. snn: Machine-learning-inspired spiking neural network modeling on brainscales-2,” in _Neuro-Inspired Computational Elements Conference_ , 2023.
  * [18] D. F. Goodman and R. Brette, “The brian simulator,” _Frontiers in Neuroscience_ , vol. 3, p. 643, 2009.
  * [19] L. Niedermeier, K. Chen, J. Xing, A. Das, J. Kopsick, E. Scott, N. Sutton, K. Weber, N. Dutt, and J. L. Krichmar, “CARLsim 6: an open source library for large-scale, biologically detailed spiking neural network simulation,” in _International Joint Conference on Neural Networks (IJCNN)_ , 2022.
  * [20] A. Das, “A design flow for scheduling spiking deep convolutional neural networks on heterogeneous neuromorphic system-on-chip,” _ACM Transactions on Embedded Computing Systems (TECS)_ , 2023.
  * [21] ——, “Real-time scheduling of machine learning operations on heterogeneous neuromorphic SoC,” in _International Conference on Formal Methods and Models for System Design (MEMOCODE)_ , 2022.
  * [22] V. Kathail, “Xilinx vitis unified software platform,” in _International Symposium on Field Programmable Gate Arrays (FPGA)_ , 2020.
  * [23] F. Ferrandi, V. G. Castellana, S. Curzel, P. Fezzardi, M. Fiorito, M. Lattuada, M. Minutoli, C. Pilato, and A. Tumeo, “Bambu: An open-source research framework for the high-level synthesis of complex applications,” in _Design Automation Conference (DAC)_ , 2021.
  * [24] S. Curzel, N. B. Agostini, S. Song, I. Dagli, A. Limaye, C. Tan, M. Minutoli, V. G. Castellana _et al._ , “Automated generation of integrated digital and spiking neuromorphic machine learning accelerators,” in _International Conference on Computer-Aided Design (ICCAD)_ , 2021.
  * [25] S. Liu, J. Weng, D. Kupsh, A. Sohrabizadeh, Z. Wang, L. Guo, J. Liu, M. Zhulin, R. Mani, L. Zhang _et al._ , “OverGen: Improving FPGA usability through domain-specific overlay generation,” in _International Symposium on Microarchitecture (MICRO)_ , 2022.
  * [26] J. Cong, J. Lau, G. Liu, S. Neuendorffer, P. Pan, K. Vissers, and Z. Zhang, “FPGA HLS today: Successes, challenges, and opportunities,” _ACM Trans, on Reconfigurable Technology and Systems (TRETS)_ , 2022.
  * [27] H. Ahangari, M. M. Özdal, and Ö. Öztürk, “HLS-based high-throughput and work-efficient synthesizable graph processing template pipeline,” _ACM Trans. on Embedded Computing Systems (TECS)_ , vol. 22, 2023.
  * [28] A. M. Abdelsalam, F. Boulet, G. Demers, J. P. Langlois, and F. Cheriet, “An efficient FPGA-based overlay inference architecture for fully connected DNNs,” in _International Conference on Reconfigurable Computing and FPGAs (ReConFig)_ , 2018.
  * [29] S. Li, Z. Zhang, R. Mao, J. Xiao, L. Chang, and J. Zhou, “A fast and energy-efficient SNN processor with adaptive clock/event-driven computation scheme and online learning,” _IEEE Transactions on Circuits and Systems I: Regular Papers_ , vol. 68, no. 4, pp. 1543–1552, 2021.
  * [30] F. Corradi, G. Adriaans, and S. Stuijk, “Gyro: A digital spiking neural network architecture for multi-sensory data analytics,” in _Drone Systems Engineering and Rapid Simulation and Performance Evaluation: Methods and Tools_ , 2021.
  * [31] A. Carpegna, A. Savino, and S. Di Carlo, “Spiker: An FPGA-optimized hardware accelerator for spiking neural networks,” in _IEEE Computer Society Annual Symposium on VLSI (ISVLSI)_ , 2022.
  * [32] H. Liu, Y. Chen, Z. Zeng, M. Zhang, and H. Qu, “A low power and low latency FPGA-based spiking neural network accelerator,” in _International Joint Conference on Neural Networks (IJCNN)_ , 2023.
  * [33] W. Guo, H. E. Yantır, M. E. Fouda, A. M. Eltawil, and K. N. Salama, “Toward the optimal design and FPGA implementation of spiking neural networks,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 33, no. 8, pp. 3988–4002, 2021.
  * [34] W. Ye, Y. Chen, and Y. Liu, “The implementation and optimization of neuromorphic hardware for supporting spiking neural networks with mlp and cnn topologies,” _IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems_ , vol. 42, no. 2, pp. 448–461, 2022.
  * [35] Z. He, C. Shi, T. Wang, Y. Wang, M. Tian, X. Zhou, P. Li, L. Liu, N. Wu, and G. Luo, “A low-cost FPGA implementation of spiking extreme learning machine with on-chip reward-modulated STDP learning,” _IEEE Transactions on Circuits and Systems II: Express Briefs_ , vol. 69, 2021.
  * [36] W. He, Y. Wu, L. Deng, G. Li, H. Wang, Y. Tian, W. Ding, W. Wang, and Y. Xie, “Comparing SNNs and RNNs on neuromorphic vision datasets: Similarities and differences,” _Neural Networks_ , 2020.
  * [37] I. Hammouamri, I. Khalfaoui-Hassani, and T. Masquelier, “Learning delays in spiking neural networks using dilated convolutions with learnable spacings,” _arXiv preprint arXiv:2306.17670_ , 2023.

|  Shadi Matinizadeh received her master’s degree in electrical engineering
from California State University, Northridge, in 2023. She is currently
working toward the PhD degree under the supervision of Dr.Anup Das from the
Department of Electrical and Computer Engineering, Drexel University,
Philadelphia. Her research interests include neuromorphic computing and
spiking neural networks.
---|---
|  Noah Pacik-Nelson is a Research Scientist at Accenture Labs. He focuses on
neuromorphic computing, IoT devices, and next-gen 3D printing. He received his
BSE in Computer Science from the University of Connecticut, Storrs CT in 2020.
---|---
|  Ioannis Polykretis is a R&D Associate Principal at Accenture Labs in San
Francisco, CA, USA. His research interests include low-power edge processing,
brain-inspired computing, neuromorphic engineering, and robotics. He received
his B.S. and M.S. degrees in Electrical and Computer Engineering from the
National Technical University of Athens in 2016. He received his Ph.D. in
Computer Science from Rutgers University, New Brunswick, NJ, in 2023.
---|---
|  Krupa Tishbi earned two degrees, a B.S. in Electrical Engineering and a
B.S. in Physics, from The College of New Jersey in 2022. Her undergraduate
research involved designing control architectures on an FPAA board, as well as
digital video and image processing with neural networks. She is now working
towards her Ph.D. in Electrical Engineering at Drexel University. Her graduate
research involves the ASIC design of spiking neural networks.
---|---
|  Suman Kumar received his Master’s degree in VLSI Design & Embedded Systems
from Delhi Technological University, India, in 2020. He is currently working
toward the Ph.D. degree under the supervision of Dr. Anup Das from the
Department of Electrical and Computer Engineering, Drexel University,
Philadelphia. His research interests include neuromorphic computing and the
spiking neural network.
---|---
|  M. Lakshmi Varshika earned a B.E. degree from Birla Institute of Technology
and Science, Pilani, India, in 2018, and an MSc from the University of
California, Santa Barbara, in electrical and computer engineering in 2021. She
is currently working towards her PhD degree at Drexel University in
Philadelphia. Her research interests include the development of efficient
hardware architectures for spiking neural network-based neuromorphic
accelerators. Additionally, she is involved in designing neuromorphic circuit
solutions for accelerators. Lakshmi Varshika Mirtinti has received the Best
Research Video Award at DAC in 2021 and 2022, and she is a finalist for the
Best Paper Award at MWSCAS 2023.
---|---
|  Arghavan Mohammadhassani is currently a Ph.D. student at Drexel University.
Her focus of research is developing and optimizing memory/storage systems for
modern heterogeneous multicore systems, which consist of a variety of cores
such as CPUs, GPUs, and Neuromorphic Processing (NP) cores.
---|---
|  Abhishek Kumar Mishra is a Ph.D. candidate at Drexel University. His
research focuses on machine learning, neuromorphic computing, and natural
language processing. He received his MS degree from SUNY Buffalo in 2020.
---|---
|  Nagarajan Kandasamy is a Professor in the Department of Electrical and
Computer Engineering at Drexel University. His current research interests are
in the areas of computer architecture, parallel processing, and embedded and
real-time systems. He received his Ph.D. in Computer Science and Engineering
from the University of Michigan, Ann Arbor. He is a senior member of the IEEE.
---|---
|  James Shackleford (S’03–M’13) received the B.S. degree in electrical
engineering from Drexel University, Philadelphia, PA, USA, in 2006, and the
M.S. degree in electrical engineering and the Ph.D. degree in computer
engineering from Drexel University, in 2009 and 2011, respectively. He was a
Postdoctoral Fellow of the Radiation Oncology Department, Massachusetts
General Hospital, Boston, MA, USA, in 2012 and an Adjunct Assistant Professor
of medicine with the Perelman School of Medicine, University of Pennsylvania,
Philadelphia, PA, USA, from 2017 to 2020. He is currently an Associate
Professor of computer engineering with Drexel University, where he now serves
as the Associate Department Head for Undergraduate Affairs. His research has
been concerned with computer vision algorithm development for medical image
processing with application to image-guided radiotherapy. Dr. Shackleford is a
member of the ACM.
---|---
|  Eric Gallo is a Technical R&D Senior Principal at Accenture Labs. His
current research interests are in areas of low power edge processing,
sustainable electronics and smart materials. He received his PhD in Electrical
and Computer Engineering from Drexel University, Philadelphia PA in 2011.
---|---
|  Anup Das is an Associate Professor at Drexel University and Associate
Department Head for Graduate Studies. He received a Ph.D. in Embedded Systems
from the National University of Singapore in 2014. Following his Ph.D., he was
a postdoctoral fellow at the University of Southampton and a researcher at
IMEC. His research focuses on neuromorphic computing and architectural
exploration. He received the United States National Science Foundation CAREER
Award in 2020 and the Department of Energy CAREER Award in 2021 to investigate
the reliability and security of neuromorphic hardware. He is a senior member
of the IEEE and a member of the ACM.
---|---


# LightSolver - A New Quantum-inspired Solver Cracks the 3-Regular 3-XORSAT
Challenge

Idan Meirzada<EMAIL_ADDRESS>LightSolver LTD., Tel Aviv, Israel  Assaf
Kalinski  LightSolver LTD., Tel Aviv, Israel  Dov Furman  LightSolver LTD.,
Tel Aviv, Israel  Tsafrir Armon  LightSolver LTD., Tel Aviv, Israel  Talya
Vaknin  LightSolver LTD., Tel Aviv, Israel  Harel Primack  LightSolver LTD.,
Tel Aviv, Israel  Chene Tradonsky  LightSolver LTD., Tel Aviv, Israel  Ruti
Ben-Shlomi<EMAIL_ADDRESS>LightSolver LTD., Tel Aviv, Israel

###### Abstract

The increasing complexity of required computational tasks alongside the
inherent limitations in conventional computing calls for disruptive
innovation. LightSolver devised a new quantum-inspired computing paradigm,
which utilizes an all-optical platform for solving hard optimization problems.
In this work, LightSolver introduces its digital simulator and joins the
3-Regular 3-XORSAT (3R3X) challenge, which aims to map the best available
state-of-the-art classical and quantum solvers. So far, the challenge has
resulted in a clear exponential barrier in terms of time-to-solution (TTS),
preventing the inspected platforms from solving problems larger than a few
hundred variables. LightSolver’s simulator is the first to break the
exponential barrier, outperforming both classical and quantum platforms by
several orders-of-magnitude and extending the maximal problem size to more
than 16,000 variables.

## Introduction

Solving optimization problems holds the potential for numerous advancements in
a myriad of applications in both research and industry, from autonomous
transportation to resource management Hoffman and Ralphs (2013). In recent
years, this challenge inspired an increasing number of research groups and
companies to develop dedicated techniques, by designing innovative heuristics
Glover (1990); Finnila  _et al._ (1994); Goto  _et al._ (2019a), utilizing
sophisticated architectures Xia  _et al._ (2002) or realizing novel, cutting-
edge hardware Tsukamoto  _et al._ (2017a); Marandi  _et al._ (2014); McMahon
_et al._ (2016); Traversa  _et al._ (2015a). These solutions offer simplicity
and availability, flexible connectivity or short convergence times. However,
they often depend on cooling to low temperatures, suffer from limited
connectivity or dynamic range, or require vast amounts of energy when applied
to real-life problems.
One year ago, two research groups at the University of Southern California
launched a computing challenge to compare the actual performances of the best
available state-of-the-art solvers Kowalsky  _et al._ (2022). This work
revealed the differences, as well as certain similarities, between their
respective performances. These results were presented, together with other
problems of simpler configurations, in a recent review article Mohseni  _et
al._ (2022).

In this work, we introduce LightSolver, a novel platform that implements a new
concept for solving hard optimization problems, based on a spatially coupled
laser array. We provide a first glance into the performance of this platform
by testing a digital simulation of the optical platform on similar 3R3X
instances, and compare its results to the solving platforms detailed in the
challenge Kowalsky  _et al._ (2022) while also extending the maximal problem
size from 640 variables to 16,384 variables.

In the next sections, we will introduce our optical platform, the 3R3X
problem, and examine the performance of the LightSolver digital simulator on
3R3X instances in the absence and in the presence of noise. We conclude by
discussing these results and their implications for the optical platform.

## LightSolver’s Analog Platform

Similar to many other solving platforms, LightSolver utilizes the flexible
nature of the Ising model to formulate optimization problems into a
Hamiltonian form, consisting of linear and quadratic terms Lucas (2014). This
process allows a single solving method to address a wide variety of problems,
instead of separately tailoring a different process for each problem category.
The Ising Hamiltonian takes the following structure:

$H=\sum_{i=1}^{n}h_{i}s_{i}+\sum_{i,j=1}^{n}J_{ij}s_{i}s_{j}$ (1)

where $s_{i}\in\\{-1,1\\}$, $h_{i}$ defines the external field, $J_{ij}$
defines the interactions and $n$ stands for the number of variables.

In the physical (optical) layer, the spin state is encoded in the lasers’
relative phases, thus promoting a representation of a binary Ising Hamiltonian
via a continuous problem. The spins interact by diffracting light from each
laser to all other lasers in a controllable manner, using a unique optical
coupler composed of programmable diffractive elements and additional optical
components. The coupler controls the interaction between all laser pairs, with
a dynamical range of up to 8 bits. This design allows for a full connectivity
between all lasers, facilitating pairwise all-to-all high-resolution spin
interactions on a desktop-size device, operating at room temperature, while
requiring only a modest amount of energy ($<1kW)$.

Unlike other experimental realizations, LightSolver’s platform implements an
expanded version of the XY model, due to continuity in both phase and
amplitude. This, on the one hand, requires constructing a novel formulation
and algorithm to tackle binary optimization problems, such as Ising. On the
other hand, the continuous nature allows modifying the formulations to
accommodate different requirements according to problem type (e.g. SAT, Max
Cut, Max Clique, etc.), objective attributes (e.g. size, connectivity, dynamic
range, internal structure) or solution preferences (speed, quality,
diversity). The algorithm and formulations remain beyond the scope of this
work.

## Method

The 3R3X problem Barthel  _et al._ (2002); Ricci-Tersenghi (2010) serves as a
test problem for optimization solvers, allowing the generation of a rough
energy topography while knowing in advance the problem’s optimal solution.
These problems consist of $n/2$ exclusive-OR clauses, each of which includes
three binary literals, so that satisfying a clause requires one or three
literals to be True. Each literal appears exactly in three clauses, resulting
in a symmetric connectivity, forming a ’golf-course’ shaped energy landscape.
The $n$-variable Ising Hamiltonians used for this work correspond to $n/2$
3R3X randomly generated problem instances, ranging from $n=32$ to $n=16,384$.
For more details on the nature of the problem, the reader is referred to Ref.
Hen (2019).

Since a variety of state-of-the-art solving platforms and algorithms already
addressed this problem, it is a solid candidate for comparison. Existing
solvers include application-specific integrated circuit (ASIC) Digital
Annealer Unit by Fujitsu (DAU) Tsukamoto  _et al._ (2017b), superconducting
qubits based annealer by D-Wave (Advantage model) Johnson  _et al._ (2010),
tailor-made SAT solver running on GPU (SAT on GPU) Bernaschi  _et al._ (2021),
Virtual MemComputing Machine (MEM) Traversa  _et al._ (2015b), and two
classical algorithms - Simulated Bifurcation Machine by Toshiba (SBM) Goto
_et al._ (2019b) and Parallel Tempering (PT) Swendsen and Wang (1986);
Hukushima and Nemoto (1996), implemented on single GPU and CPU cores,
correspondingly.

To add confidence to the veracity of the results, the performance of the
Lightsolver simulator was tested and analyzed by an external party (namely,
one of the co-authors of Ref. Kowalsky  _et al._ (2022)), asking them to
perform the simulation on our simulator’s alpha version. The following
sections portray the obtained results.

## Results

The LightSolver simulator ran on a single g4dn.4xlarge (64GB RAM) machine
through an Amazon cloud service. To reliably predict the asymptotic behavior
of the algorithm, only one initial state evolved in each run, although the
simulator supports running many initial states in parallel using the GPU. Each
of the 23 problem sizes examined in this work comprises 25 randomly generated
instances. For each instance, the simulation ran until it reached the ground-
state or the maximal number of steps defined (100,000 for noise-free
simulations and 500,000 for noise-injected simulations), and the TTS was
calculated.

The TTS of a single initial state depends on the simulation time ($t_{f}$) and
the probability to reach the ground state ($p_{i}$), and is calculated in the
following way Rønnow  _et al._ (2014):

$TTS=\underset{t_{f}}{min}\biggl{\langle}t_{f}\frac{ln(1-0.99)}{ln[1-p_{i}(t_{f})]}\biggr{\rangle}.$
(2)

###  .1 Noise-free simulations

Figure 1 depicts the LightSolver digital simulator’s performance, and is the
main result of this paper. The figure presents the TTS for the 3R3X problem as
a function of problem size, from $n=32$ to $n=16,384$ variables, on a log-log
scale. The red stars correspond to LightSolver’s simulator, comparing its
performance to other state of the art platforms benchmarked in Kowalsky  _et
al._ (2022), as well as known heuristics, namely Tabu Search and Simulated
Annealing.

Figure 1: Optimal TTS of different solving platforms for 3-XOR-SAT problem as
measured in Kowalsky  _et al._ (2022), including LightSolver simulator (red
stars), as well as tabu search and simulated annealing (dark and light purple
triangles, respectively), for up to $n=16,384$ variables. The LightSolver
simulator displays a power-law scaling of the form $TTS\sim n^{k}$, where the
power $k=2.31\pm 0.03$, is solving a 16,384 variable problem in less than 100
seconds. Tabu search shows similar behavior to the Fujitsu DAU solver, while
simulated annealing performs slightly better than PT.

LightSolver’s simulator demonstrates a power-law scaling of the form $TTS\sim
n^{k}$, where the power $k=2.31\pm 0.03$. For reference, all of the other
solving platforms demonstrated exponential scaling of the form $TTS\sim
10^{\alpha n}$, with the pre-factor $\alpha$ varying from 0.0171 (SAT on GPU)
to 0.08 (D-Wave advantage), resulting in time-to-solution $TTS>10^{4}$ seconds
for a 640 variable problem for the best performing solver. Thus, despite
falling short when facing smaller instances, LightSolver’s simulator proves
superior and exhibits 1-5 orders of magnitude speed-up for every tested
problem larger than $n>64$ variables, while evolving only a single initial
state.

###  .2 Noise-injected simulations

Due to its analog nature, the optical platform may suffer from substantial
noise sources, such as thermal fluctuations, and optical and electrical
instabilities. These would interfere with the ’natural’ evolution of the laser
phases and amplitudes of the coupled lasers, impeding the system’s ability to
effectively solve the problem. Thus, to better predict the optical platform’s
performance, we introduced noise to the simulation in the following way: at
each step (which corresponds to a round trip in the optical platform) the
simulation samples a vector out of a normal distribution around the chosen
noise amplitude, and adds it to the current vector. The noise distorts the
phases and amplitudes of each laser and thus changes the effective
interactions between the Ising spins.

Fig. 2 outlines the LightSolver simulator’s TTS as a function of problem size
in the presence of noise, for noise amplitudes up to 7% of the saturated
amplitude (an intrinsic property in the simulation limiting the lasers’
amplitude according to gain dynamics), for evolutions of up to 500,000 steps.

The simulation is robust in the presence of noise, demonstrating an almost
complete resilience for noise levels of up to 3% for all problem sizes, and
suffering order-of-magnitude increases in TTS only when confronting large
noise levels of above 5.5%. Moreover, the TTS scaling remains constant
regardless of the noise levels inspected here, taking the form of a similar
power law with the power $k=2.3\pm 0.2$. The ’jump’ in TTS between noiseless
evolution and the rest of the curves at the beginning of the figure stems from
noise generation overhead in the simulation code.

Figure 2: TTS as a function of problem size, for noise amplitudes ranging from
0 to 7%. The system seems indifferent to noise levels below 3%, and maintains
similar scaling for all noise amplitudes tested.

## Discussion & Conclusions

The 3R3X problem challenges many state-of-the-art heuristics and physical
Ising solvers. Although Gaussian elimination solves this problem with a cubic
scaling, so far every Ising solver benchmarked against this problem presented
an exponential scaling, with coefficients ranging between $0.0171$ to $0.08$
Kowalsky  _et al._ (2022), resulting in $TTS>10^{4}$ seconds for problems
larger than a few hundred variables.

The LightSolver simulator testing indicates a speed-up of several orders of
magnitude compared to the benchmarked solvers, revealing a power law scaling,
thus allowing solving problems of more than 16,000 spins in less than 100
seconds. Furthermore, we did not exploit all of the parallel capabilities of
the simulator (e.g., running many initial states at the same time) to evaluate
the asymptotic scaling of the TTS as a function of problem size. Relaxing this
restriction would result in better TTS and scaling, especially for evolutions
in the presence of noise.

The increase in computation time originates mostly in the emulation of the
diffractive elements of the physical hardware on the computing platform
(scaling as $n^{2}$). This implies that although the problems increase in
size, their hardness increases very slowly in the eyes of the algorithm.
Additionally, the algorithm demonstrates its ability to reach the global
minima in the presence of noise, for amplitudes up to 7%, while keeping a
similar scaling.

The hardware implementation of the solver consists of a coupled laser array,
with an all-to-all connectivity. Thus, the time required to solve a problem on
the optical platform depends solely on the number of round trips required,
independent of the problem size. The fact that for each noise level the
required number of round trips remains almost constant, suggests that the
optical solver would scale even better than the simulation, resulting in a
speed-up of more than three orders of magnitude compared to the simulation
results (provided that the optical solver includes enough spins to contain the
entire problem).

Lastly, an important aspect to consider when discussing performance of any
solver, concerns the energy consumption required by the proposed solving
platform. The suggested analog hardware, simulated in this work, would perform
the ’heavy’ matrix-vector multiplication using an optical setup, regardless of
problem size or connectivity (up to $\sim$ 1000 variables for the first
generation), while working at room temperature. Thus, the optical platform
will potentially consume a few hundred Watts, comparable to an average
desktop.

## Acknowledgements

We wish to thank Dr. Itay Hen (University of Southern California), for
performing the simulations on our alpha version of the simulator and the in-
depth analysis of the results.

## References

  * Hoffman and Ralphs (2013) K. L. Hoffman and T. K. Ralphs, “Integer and combinatorial optimization,” in  _Encyclopedia of Operations Research and Management Science_, edited by S. I. Gass and M. C. Fu (Springer US, Boston, MA, 2013) pp. 771–783.
  * Glover (1990) F. Glover, Interfaces 20, 74 (1990).
  * Finnila  _et al._ (1994) A. Finnila, M. Gomez, C. Sebenik, C. Stenson, and J. Doll, Chemical Physics Letters 219, 343 (1994).
  * Goto  _et al._ (2019a) H. Goto, K. Tatsumura, and A. R. Dixon, Science Advances 5, eaav2372 (2019a).
  * Xia  _et al._ (2002) Y. Xia, H. Leung, and J. Wang, IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications 49, 447 (2002).
  * Tsukamoto  _et al._ (2017a) S. Tsukamoto, M. Takatsu, S. Matsubara, and H. Tamura, Fujitsu Sci. Tech. J 53, 8 (2017a).
  * Marandi  _et al._ (2014) A. Marandi, Z. Wang, K. Takata, R. Byer, and Y. Yamamoto, Nature Photonics 8, 937 (2014).
  * McMahon  _et al._ (2016) P. L. McMahon, A. Marandi, Y. Haribara, R. Hamerly, C. Langrock, S. Tamate, T. Inagaki, H. Takesue, S. Utsunomiya, K. Aihara, R. L. Byer, M. M. Fejer, H. Mabuchi, and Y. Yamamoto, Science 354, 614 (2016).
  * Traversa  _et al._ (2015a) F. L. Traversa, C. Ramella, F. Bonani, and M. D. Ventra, Science Advances 1, e1500031 (2015a).
  * Kowalsky  _et al._ (2022) M. Kowalsky, T. Albash, I. Hen, and D. A. Lidar, Quantum Science and Technology 7, 025008 (2022).
  * Mohseni  _et al._ (2022) N. Mohseni, P. L. McMahon, and T. Byrnes, Nature Reviews Physics , 1 (2022).
  * Lucas (2014) A. Lucas, Frontiers in Physics 2, 5 (2014).
  * Barthel  _et al._ (2002) W. Barthel, A. K. Hartmann, M. Leone, F. Ricci-Tersenghi, M. Weigt, and R. Zecchina, Phys. Rev. Lett. 88, 188701 (2002).
  * Ricci-Tersenghi (2010) F. Ricci-Tersenghi, Science 330, 1639 (2010).
  * Hen (2019) I. Hen, Phys. Rev. Applied 12, 011003 (2019).
  * Tsukamoto  _et al._ (2017b) S. Tsukamoto, M. Takatsu, S. Matsubara, and H. Tamura, Fujitsu Sci. Tech. J 53, 8 (2017b).
  * Johnson  _et al._ (2010) M. W. Johnson, P. Bunyk, F. Maibaum, E. Tolkacheva, A. J. Berkley, E. M. Chapple, R. Harris, J. Johansson, T. Lanting, I. Perminov, E. Ladizinsky, T. Oh, and G. Rose, Superconductor Science and Technology 23, 065004 (2010).
  * Bernaschi  _et al._ (2021) M. Bernaschi, M. Bisson, M. Fatica, E. Marinari, V. Martin-Mayor, G. Parisi, and F. Ricci-Tersenghi, Europhysics Letters 133, 60005 (2021).
  * Traversa  _et al._ (2015b) F. L. Traversa, C. Ramella, F. Bonani, and M. D. Ventra, Science Advances 1, e1500031 (2015b).
  * Goto  _et al._ (2019b) H. Goto, K. Tatsumura, and A. R. Dixon, Science Advances 5, eaav2372 (2019b).
  * Swendsen and Wang (1986) R. H. Swendsen and J.-S. Wang, Phys. Rev. Lett. 57, 2607 (1986).
  * Hukushima and Nemoto (1996) K. Hukushima and K. Nemoto, Journal of the Physical Society of Japan 65, 1604 (1996).
  * Rønnow  _et al._ (2014) T. F. Rønnow, Z. Wang, J. Job, S. Boixo, S. V. Isakov, D. Wecker, J. M. Martinis, D. A. Lidar, and M. Troyer, Science 345, 420 (2014).