# Scaling Analog Photonic Accelerators for Byte-Size, Integer General Matrix
Multiply (GEMM) Kernels

Oluwaseun Adewunmi Alo  Electrical and Computer Engineering
University of Kentucky, USA
<EMAIL_ADDRESS>Sairam Sri Vatsavai  Electrical and Computer Engineering
University of Kentucky, USA
<EMAIL_ADDRESS>Ishan Thakkar  Electrical and Computer Engineering
University of Kentucky, USA
<EMAIL_ADDRESS>
###### Abstract

Deep Neural Networks (DNNs) predominantly rely on General Matrix Multiply
(GEMM) kernels, which are often accelerated using specialized hardware
architectures. Recently, analog photonic GEMM accelerators have emerged as a
promising alternative, offering vastly superior speed and energy efficiency
compared to traditional electronic accelerators. However, these photonic
cannot support wider than 4-bit integer operands due to their inherent trade-
offs between analog dynamic range and parallelism. This is often inadequate
for DNN training as at least 8-bit wide operands are deemed necessary to
prevent significant accuracy drops. To address these limitations, we introduce
a scalable photonic GEMM accelerator named SPOGA. SPOGA utilizes enhanced
features such as analog summation of homodyne optical signals and in-
transduction positional weighting of operands. By employing an extended
optical-analog dataflow that minimizes overheads associated with bit-sliced
integer arithmetic, SPOGA supports byte-size integer GEMM kernels, achieving
significant improvements in throughput, latency, and energy efficiency.
Specifically, SPOGA demonstrates up to 14.4$\times$, 2$\times$, and
28.5$\times$ improvements in frames-per-second (FPS), FPS/Watt, and
FPS/Watt/mm² respectively, compared to existing state-of-the-art photonic
solutions.

###### Index Terms:

Deep Learning, General Matrix Multiplication, Accelerator, Silicon Photonics,
Bit Slicing

##  I Introduction

Deep Neural Networks (DNNs) can process complex data and perform tasks such as
image recognition, natural language processing, and speech recognition with
remarkable accuracy. For processing DNNs, GEMM functions play a crucial role,
particularly in various linear layers of DNNs such as fully connected,
convolutional, and self/cross attention layers [19].

Some of these DNN layers do not directly employ GEMMs but their comprising
computational kernels are often transformed into GEMM functions for efficient
hardware-based acceleration. For instance, convolution layers are often
converted into input and Toeplitz matrices using Im2Col operations to enable
GEMM functions between these matrices [7]. DNNs also employ non-linear
functions, along with linear functions such as GEMM functions. However, GEMM
functions often comprise about 70% of the execution cycles while training DNNs
[19], making them the key workload in all DNNs.

While GEMM functions continue to remain the favorite abstraction to which the
tensors during the forward and backward passes of DNNs are unrolled, the
performance and energy efficiency demands for processing the GEMM functions of
modern DNNs have grown towards becoming unsustainable over the past decade.
This is due to the number of trainable parameters of DNNs growing from
thousands to more than hundreds of billions, and the speed and energy
consumption of traditional hardware architectures, such as Graphics Processing
Units (GPUs) [19], Tensor Processing Units (TPUs) [25], and Application-
Specific Integrated Circuits (ASICs) [20] based designs, not scaling
sufficiently to keep up. As a result, to meet these demands, hardware
architects dwell in search of ultrafast and extremely energy-efficient
architectures to accelerate GEMM functions.

Fortunately, this search has led to the recent demonstrations of analog
photonic accelerators for GEMM functions [2, 3, 5, 6, 8, 9]. It has been shown
that analog photonic accelerators can achieve two to three orders of magnitude
higher processing speed and energy efficiency for processing GEMM functions
compared to other digital/analog electronic accelerators [1, 2, 3, 4, 6, 9,
22]. These benefits of analog photonic accelerators are attributed to their
massive, multi-dimensional parallelism along with the ultra-low-dissipation,
impedenceless, and high-speed dynamics.

Despite these advantages, however, analog photonic accelerators for GEMM
functions face daunting challenges due to the strong trade-offs among their
achievable multi-dimensional parallelism, operating speeds, and operand bit-
widths. It has been shown that, due to these trade-offs, the achievable
operand bit width for analog multiplications in analog photonic accelerators
for GEMM functions remains constricted to 4-bit with fixed-point precision at
1 gigasamples (GS)/sec speed and 44$\times$44 in-parallel multiplications per
GEMM core [1, 2]. As the operand bit-width increases to fixed-point 8-bit, the
achievable parallelism diminishes to only 1 multiplication per core. This
occurs because a major portion of the already tight optical power budget is
utilized to support the large dynamic range required for 8-bit precision (a
total of 28=256 analog optical power levels are required for 8-bit operands
compared to only 24=16 levels required for 4-bit operands), leaving an
insufficient portion of the optical power budget in support of high optical
parallelism [2].

Although 4-bit fixed-point precision is touted to be sufficient for many DNN
inference applications accelerated on the edge devices, it has been shown that
a minimum of 8-bit fixed-point operands are required for multiplications with
at least 16-bit precision required for intermediate accumulations (before
their rounding to 8-bit precision) during DNN training to keep the drop in the
accuracy tolerable while simultaneously achieving high energy efficiency [26,
27]. Thus, there is a crucial need to scale analog photonic architectures to
accelerate GEMM functions with byte-size integer operands.

To meet this requirement, the analog photonic accelerators in the literature
often utilize bit-sliced operands [3, 9], wherein the INT8 operands are split
into two concatenated INT4 slices. Consequently, the processing of every GEMM
function between matrices comprising INT8 operands is decomposed into four
GEMM functions each between matrices comprising INT4 operands. Each INT4 GEMM
function is implemented on a dedicated analog photonic core to produce an
intermediate output matrix. All four intermediate matrices, thus produced, are
then post-processed to generate the final result of the INT8 GEMM function.
Such processing using INT4 slices of INT8 operands incurs excessive area,
latency, and power overheads for analog-to-digital conversions, memory storage
and access of intermediate matrices, and digital electronic post-processing of
intermediate results. These overheads diminish the inherent throughput and
energy efficiency benefits of employing analog photonic computing.

To overcome these shortcomings, we present a Scalable PhotOnic GEMM
Accelerator (SPOGA) for byte-size integer GEMM kernels in this paper. SPOGA
introduces these two enhanced features in the employed electro-photonic
circuits: (1) analog summation of homodyne optical signals by a time-
integrating optical-to-electrical transduction circuit through incoherent
superposition and charge accumulation, and (2) in-transduction weighting of
accumulated analog results. These features extend the mixed optical/analog
dataflow in SPOGA for the temporary storage and post-processing of
intermediate matrices generated from INT4-sliced GEMM functions. This in turn
eliminates the above-mentioned overheads of processing bit-sliced integer
operands, rendering substantial throughput, latency, and energy-efficiency
benefits to SPOGA for accelerating byte-size integer GEMM functions.

The rest of the paper is organized as follows. Section II discusses related
work and provides background and motivation. Section III provides an overview
of the SPOGA architecture and describes its key enhanced features. The
scalability analysis and the results of benchmark-drivel system-level
evaluations obtained for four modern DNNs are presented in Section IV. Section
V concludes this paper.

##  II Background and Motivation

(a) Illustration of a General Matrix Multiplication (GEMM) operation.

(b) Mapping of GEMM operation from (a) to MAW and AMW types of GEMM core
organisations from prior works.

Figure 1: Illustration of a General Matrix Multiplication (GEMM) operation and
its mapping on photonic GEMM cores.

###  II-A Related Work on Analog Photonic GEMM Accelerators

It has been shown that analog photonic accelerators can achieve two to three
orders of magnitude higher processing speed/throughput and energy efficiency
for processing GEMM functions compared to other digital/analog electronic
accelerators [24]. The higher processing speed and throughput of analog
photonic accelerators are attributed to their massive, multi-dimensional
parallelism [24]. Similarly, their higher energy efficiency is attributed to
their ultra-low-dissipation, impedanceless, and high-speed dynamics. The
analog photonic GEMM accelerators from the literature can be broadly
categorized into two main types: coherent and non-coherent architectures.
Incoherent GEMM accelerators [2, 3, 6, 9] modulate optical signal power to
represent tensor values, whereas coherent GEMM accelerators [5] utilize
optical field amplitude and phase. Our study focuses on incoherent
architectures because of their demonstrated scalability and performance
advantages over their coherent counterparts [5]. Non-coherent architectures
utilize parallel wavelengths and waveguides, with parameters encoded within
optical wavelength signals’ amplitudes using devices like microring resonators
(MRRs).

The MRR-based incoherent photonic accelerators from the literature often
employ multiple GEMM cores that operate concurrently. These GEMM cores are
primarily responsible for executing Vector Dot Product (VDP) operations that
are derived by decomposing the target GEMM functions. Every GEMM core employs
a total of N laser diodes that generate N optical wavelength channels
$\lambda_{1}\ -\lambda_{N}$Ṫhese optical wavelength channels are manipulated
in five different ways by five different blocks of photonic devices and
circuits to implement parallel VDP operations. These blocks include (1) A
splitting block that employs a series of splitters for splitting (copying) N
optical signals in M waveguides to achieve a fan-out degree of M per optical
signal; (2) An aggregation block that employs multiplexers for aggregation
(multiplexing) of N optical signals per waveguide to achieve a fan-in degree
of N per waveguide; (3) A modulation block that employs one or more MRR
modulator (MRM) arrays for modulation of optical signals to imprint input
values onto them; (4) A weighting block that employs multiple parallel MRR
weighting banks for weighting of modulated optical signals to achieve analog
input-weight products; (5) A summation block that comprises a total of M
balance photodetectors (BPDs) each coupled with a time-integrating or trans-
impedance receiver to perform analog summation of optical signals through
incoherent superposition and charge accumulation. The specific order in which
these blocks are arranged distinguishes various prior GEMM core designs. These
designs can be broadly categorized into MAW (Modulation-Aggregation-Weighting)
[8], AMW (Aggregation-Modulation-Weighting) [9], and MWA (Modulation-
Weighting-Aggregation). For each GEMM core, the defined sequence of these
signal manipulations plays a pivotal role in the efficiency and performance of
the core, ultimately shaping the core’s computational capabilities. The reader
is directed to [23] for more details.

While these architectures excel in handling GEMM operations, recent
advancements have also shown all-optical and opto-electronic methods for
implementing non-GEMM operations, such as activation operations, as well [28],
although we only focus on GEMM operations in this work.

###  II-B Mapping of GEMM Functions on Photonic Accelerators

Both electronic and photonic GEMM accelerators support temporal, spatial, and
mixed spatio-temporal mapping of GEMM functions onto GEMM cores. However,
photonic accelerators provide additional advantages in spatial mapping,
offering two distinct approaches: mapping by waveguide and by wavelength. This
increased degree of freedom in spatial mapping provides a notable advantage
over their electronic counterparts. For instance, Fig. 1 illustrates how a
GEMM function is typically mapped onto the MAW and AMW categories of GEMM
cores. Each GEMM core design in Fig. 1(b) is shown to have a total of N
wavelength channels and M waveguides. From Fig. 1(a), the input matrix
$\mathbf{I}$ (of size T$\times$N) is mapped spatially (across N wavelengths)
and temporally (across T time steps), whereas the weight matrix $\mathbf{W}$
(of size N$\times$M) is mapped spatially across N wavelengths and M
waveguides. As a result, during each time step, each of the total M BPDs
produces the sum of a total of N products. Therefore, since there are a total
of M BPDs in each GEMM core design, a total of M sums of products (M dot
products) are produced in parallel in each GEMM core.

###  II-C Bit-Slicing for GEMM Functions with Byte-Size Operands

S. S Vatsavai et al. in [2] inferred that photonic GEMM accelerator cores from
the literature cannot support wider than 4-bit integer operands while
simultaneously supporting expected processing parallelism (i.e., numbers of
parallel wavelength channels and waveguides). This is because these GEMM cores
exhibit a very strong trade-off between the achievable operand bitwidth and
parallelism. To address this shortcoming, the analog photonic accelerators
from prior works often utilize bit-sliced operands, wherein the INT8 operands
are split into two concatenated INT4 slices: one slice comprising the Most
Significant Nibble (MSN) and the other slice comprising the Least Significant
Nibble (LSN). Consequently, the input matrix $\mathbf{I}$ and weight matrix
$\mathbf{W}$ from Fig. 1 are decomposed into two INT4 matrices each,
$\mathbf{I}$MSN/$\mathbf{I}$LSN and $\mathbf{W}$MSN/$\mathbf{W}$LSN,
respectively. As a result, the processing of a GEMM function between the input
and weight matrices comprising INT8 values is decomposed into four GEMM
functions, each function between LSN/MSN matrices comprising INT4 values, as
shown in Fig. 2(a). Each INT4 GEMM function is implemented on a dedicated
analog photonic GEMM core to produce an intermediate output matrix. All four
intermediate matrices, thus produced, are then post-processed using the
Digital Electronic Shifter and Adder (DEAS) block (Fig. 2(a)). This
postprocessing using DEAS is needed to multiply the individual values of each
intermediate result matrix with the weight of the corresponding radix
positions (see 160, 161, and 162 as the radix position weights of the
intermediate result matrices in Fig. 2(a)). This post-processing generates the
final output matrix (Fig. 2(a)).

###  II-D Motivation

In the bit-slicing-based approach, the DEAS-based post-processing of
intermediate result matrices incurs very high overheads. This is because the
intermediate result matrices are generated in the optical analog format by the
analog photonic GEMM cores; therefore, these matrices have to be converted to
the electronic digital format first. This step requires high-speed optical-to-
electrical and analog-to-digital converters (ADCs) because at each BPD one
result needs to be converted to the electrical and then digital format every
time step of 1 nanosecond or sub-nanosecond (corresponding to $\geq$1 GS/sec
speed). Such high-speed optical-to-electrical converters and ADCs consume non-
trivial amounts of power/energy and area. After these digital conversions
using ADCs, the intermediate matrices then need to be stored in digital memory
and accessed from the memory for their subsequent DEAS-based processing. This
consumes additional energy in addition to the extra area and latency overheads
of DEAS units. Overall, these overheads diminish the inherent throughput and
energy efficiency benefits of employing analog photonic computing.

(a) The method used in prior works.

(b) The method used in SPOGA, at matrix-level abstraction. All the ’I’s and
’W’s are matrices.

(c) The method used in SPOGA, at dot product level abstraction. All the ’I’s
and ’W’s are INT4 values.

Figure 2: Different methods of implementing and mapping a GEMM function for
hardware acceleration using bit-sliced integer arithmetic.

##  III SPOGA Architecture: Overview

The main idea of our SPOGA architecture is to extend the optical-analog
dataflow inside a GEMM core to implement the radix-position-based weighting
and addition of the intermediate result matrices opto-electronically directly
in the analog format during the optical-to-electrical transduction, without
requiring to convert the intermediate results matrices in the digital format
nor having to employ sluggish DEAS circuits. SPOGA achieves this by employing
a unique GEMM core architecture that comprises a total of 16 dot product units
(DPU). Each DPU is capable of performing a dot product on large input and
weight vectors comprising INT8 values, with up to 249 INT8 values per vector
(more on this size scalability analysis is discussed in Section IV-A). Each
DPU handles the radix-position-based weighting and addition of the
INT4-operands-based intermediate results in the optical domain. For that, it
assigns a common wavelength channel to all bit-sliced (INT4) multiplication
operations that have the same radix position weight (see Figs. 2(b) and 2(c)).
This arrangement provides several benefits compared to the bit-sliced
processing approach illustrated in Fig. 2(a), as will be discussed in Section
III-B.

###  III-A Structure and Operation of SPOGA DPU

Fig. 3 illustrates the structure of a SPOGA DPU. A SPOGA DPU comprises three
unique types of photonic circuits: (1) multiple Optical Analog Multiplier
Ensembles (OAMEs), (2) radix-position-weight-aware aggregation lanes, and (3)
one Positional Weighting and Accumulation Block (PWAB) for radix-position-
based weighting and accumulation/addition of intermediate results.

####  III-A1 Optical Analog Multiplier Ensemble (OAME)

From Fig. 3(a), each OAME performs a multiplication between two INT8 operands,
i.e., INT8 input I1 and INT8 weight W1, in a bit-sliced manner. Since each
operand is sliced into INT4 MSN and LSN (e.g., I1 is sliced into
I${}_{1}^{MSN}$ and I${}_{1}^{LSN}$, and W1 sliced into W${}_{1}^{MSN}$ and
W${}_{1}^{LSN}$) the OAME performs a total of four INT4 multiplications, i.e.,
(i) I${}_{1}^{MSN}$$\times$W${}_{1}^{MSN}$, (ii)
I${}_{1}^{MSN}$$\times$W${}_{1}^{LSN}$, (iii)
I${}_{1}^{LSN}$$\times$W${}_{1}^{MSN}$, and (iv)
I${}_{1}^{LSN}$$\times$W${}_{1}^{LSN}$. For that, the OAME employs four
parallel optical analog multiplier units (OAMUs) (see Fig. 3(a)). These four
OAMUs are assigned four dedicated wavelength channels ($\lambda_{1}$,
$\lambda_{2}$, $\lambda_{3}$, and $\lambda_{4}$), with one wavelength channel
assigned per OAMU. As a result, the result of the INT4 multiplication
I${}_{1}^{MSN}$$\times$W${}_{1}^{MSN}$ is carried on a dedicated wavelength
$\lambda_{1}$, and so forth for the remaining four INT4 multiplications as
well (see Fig. 3(a)). The four intermediate result values that are produced by
the OAME emerge from the OAME on four different wavelength channels. These
values are unweighted as they emerge, and they need to undergo radix-position-
based weighting and consequent accumulation/addition, to produce the final
output. For that, SPOGA DPU employs aggregation lanes and a PWAB, as discussed
next.

Figure 3: SPOGA Architecture Overview. (a) An Optical Analog Multiplier
Ensemble (OAME), a component that composes a SPOGA GEMM core comprising Dot
Product Units (DPUs) in (c). (b) A Balanced Photo Charge Accumulator (BPCA)
that composes the Positional Weighting and Accumulation Block (PWAB) in (a)
and (c).

####  III-A2 Aggregation Lanes

Each intermediate result emanating on a wavelength from an OAME needs to
undergo a different radix-position-based weighting. For instance, the
weighting associated with the result carried by $\lambda_{1}$ should be 162.
This is because $\lambda_{1}$ corresponds to an OAMU that produces a
multiplication result between two MSNs. Each MSN of an INT8 operand has a
radix-positional weight of 161. As a result, a multiplication of two MSNs
should have a positional weight of 162, making the positional weight of
$\lambda_{1}$ to be 162. Similarly, since there is one MSN multiplied with one
LSN in the OAMUs corresponding to wavelengths $\lambda_{2}$ and $\lambda_{3}$,
the associated weight in this case should be 161 for both wavelengths. Along
the same lines, since there is one LSN multiplied with another LSN in the OAMU
corresponding to wavelengths $\lambda_{4}$, the weight associated with
$\lambda_{4}$ should be 160. All of these positional weights are applied to
the unweighted intermediate results in a PWAB (Fig. 3(a)). Therefore, all the
results-carrying wavelengths emanating from an OAME needs to be sent to the
PWAB. This is achieved using the aggregation lanes (Fig. 3(a)). There are a
total of three sets of aggregation lanes. Each set corresponds to a specific
positional weight. Wavelengths $\lambda_{4}$ and $\lambda_{1}$ are propagated
to the PWAB in the aggregation lane sets corresponding to the positional
weights of 160 and 162 respectively. In contrast, wavelengths $\lambda_{2}$
and $\lambda_{3}$ are aggregated (multiplexed) in the aggregation lane set
corresponding to positional weight 161. Each lane set has one positive (+ve)
lane and one negative (-ve) lane. A wavelength takes either +ve lane or -ve
lane based on the sign of the result values carried onto the wavelength. It is
noteworthy that a SPOGA DPU comprises several (up to 249) OAMEs. The three
aggregation lane sets discussed above are shared among all OAMEs of a DPU
(Fig. 3(c)) so that the unweighted-results-carrying signals of carrier
wavelength $\lambda_{1}$ emanating from all OAMEs are multiplexed into the
lane sets corresponding to positional weight 162, and so forth. These shared
aggregation lane sets take the multiplexed wavelength signals to the PWAB.

####  III-A3 Positional Weighting and Accumulation Block (PWAB)

A PWAB comprises three parallel balance photo-charge accumulators (BPCAs) [1,
22, 23] followed by an analog voltage adder and an ADC. Each BPCA comprises a
BPD connected to a time-integrating receiver with a bank of selectable
accumulation capacitors [2]. The three parallel BPCAs correspond to three
positional weight values 162, 161, and 160. During a time step of interest,
each BPCA acts as an optical-to-electrical transducer and spatial accumulator
to produce an analog output voltage that is proportional to the sum of optical
signals (product values) provided as input during the time step. Thus, it
produces a dot product result in each time step. The BPCA design from our
prior works [1, 2] only sums heterodyne optical signals (signals of different
optical carrier wavelengths). In contrast, in this work, we re-purpose the
BPCA to perform the summation of homodyne optical signals. This is crucial
because each BPCA in Fig. 3 connects with exactly one aggregate lane set that
carries unweighted results from multiple OAMEs on optical signals of the same
carrier wavelength. Hence, summing these homodyne optical signals together in
turn accumulates the unweighted results together into an intermediate partial
result. Thus, the use of three BPCAs produces three positionally unweighted
partial results. In addition, we added another feature to our designed BPCA.
We enabled each BPCA to select one specific accumulation capacitor with the
capacitance of 16-2$\times$C0, 16-1$\times$C0, or C0 to scale its output
analog voltage by the factor of 162, 161, or 160, respectively, for the same
input. This enables each BPCA to multiply its partial result with its
corresponding positional weight (i.e., 162, 161, or 160). This way, the three
BPCAs produce three positionally weighted partial results in the analog
format. These weighted partial results are then summed together using an
analog voltage adder, the result of which is converted into the final digital
result using an ADC (Fig. 3(c)). This final digital result is the dot product
output. Thus, the PWAB block enables the positional weighting and accumulation
of intermediate partial results during the optical-to-electrical traduction of
input optical signals.

###  III-B Benefits of Extending Optical-Analog Dataflow in SPOGA

Due to the unique design of a SPOGA DPU, the generation of one dot product
output in the digital format requires three optical-to-electrical conversions
(of three partial results) and one analog-to-digital conversion. It also does
not require intermediate memory storage or post-processing using DEAS. This
incurs much less overhead than the approach from Fig. 2(a), which requires a
total of four optical-to-electrical conversions, four analog-to-digital
conversions, intermediate memory storage, and post-processing using DEAS. The
notable reduction in the overheads translates to substantially better
processing throughput and energy efficiency for SPOGA, as evident from the
results presented in the next section.

##  IV Evaluation

###  IV-A Scalability Analysis

Our scalability study is based on the methods for modeling photonic GEMM cores
that are presented in prior works [1, 2, 12]. Using the modeling equations and
parameters from [2], we evaluated the maximum number of achievable dot
products per GEMM core (denoted as M) and the maximum allowable vector size
during dot product operations (denoted as N) for SPOGA and other architectures
from prior works, as functions of operating data rate and input optical power.
We considered input analog operand width of 4-bit, i.e., 24 analog levels. The
results of our analysis are summarized in Table I. As evident, SPOGA in
general achieves the highest parallelism, i.e., the largest N$\times$M value.
This is due to the full optical-analog dataflow in SPOGA, which saves optical
power to keep it available in support of higher parallelism.

Figure 4: Schematic of system-level implementation of SPOGA. TABLE I: Results of Scalability Analysis.  | BR/DR
---|---
Architectures | 1GS/s | 5GS/s | 10GS/s
| N | M | N | M | N | M
HOLYLIGHT [3] | 43 | 43 | 21 | 21 | 15 | 15
DEAPCNN [9] | 36 | 36 | 17 | 17 | 12 | 12
MWA (1dBm) | 94 | 16 | 32 | 16 | 5 | 16
MWA (5dBm) | 163 | 16 | 101 | 16 | 74 | 16
MWA (10dBm) | 249 | 16 | 187 | 16 | 160 | 16
TABLE II: Area and Power Overheads of ADC and DACs. Converters | BRs (GS/s) | Area (mm2) | Power (mw)
---|---|---|---
ADC | 1 [13] | 0.002 | 2.55
5 [14] | 0.021 | 11
10 [15] | 0.103 | 29
DAC | 1 [16] | 0.00007 | 0.12
5 [17] | 0.06 | 26
10 [18] | 0.06 | 30

###  IV-B System-Level Simulation Setup

To assess the performance of our SPOGA accelerator, we employed a custom,
transaction-level Python-based simulator. Within this simulator framework, we
modeled SPOGA as depicted in Fig. 4. We used the modeling parameters,
area/energy/latency parameters of hardware components, etc., from our prior
works [1, 2]. The utilized parameter values for the ADCs and DACs are listed
in Table II. Subsequently, we conducted comprehensive simulations for the
inference of four prominent CNN models: MobileNet V2, ShuffleNet V2, ResNet50,
and GoogleNet. Throughout our evaluation, we focused on crucial metrics,
including Frames-Per-Second (FPS), FPS/W (energy efficiency), and FPS/W/mm2
(area efficiency). We conducted comparative assessments against two analog
optical accelerators, namely MAW (HOLYLIGHT) [3] and AMW (DEAPCNN) [9].

###  IV-C Evaluation Results

Fig. 5(a) presents a logarithmic scale comparison of FPS results. Notably,
$SPOGA\\_10$ (SPOGA operating at 10 GS/s) exhibits a remarkable performance
advantage, achieving 14.4$\times$ and 11.1$\times$ higher FPS than
$DEAPCNN\\_10$ and $HOLYLIGHT\\_10$, respectively, as indicated by the
geometric mean (gmean) values. Further, Fig. 5(b) and Fig. 5(c) present the
FPS/W and FPS/W/Area results. Notably, $SPOGA\\_10$ achieves 2$\times$ and
1.3$\times$ better FPS/W than $DEAPCNN\\_10$ and $HOLYLIGHT\\_10$
respectively. In addition, $SPOGA\\_1$ shows 28.5$\times$ better FPS/W/Area
compared to $DEAPCNN\\_1$ and an impressive 22.2$\times$ better FPS/W/Area
compared to $HOLYLIGHT\\_1$. The better results obtained for SPOGA variants
can be attributed to SPOGA’s higher parallelism, extended optical-analog
dataflow, and reduced overheads.

(a) FPS (Log Scale)

(b) FPS/W

(c) FPS/W/Area

Figure 5: Evaluation results for SPOGA versus HOLYLIGHT (MAW) and DEAPCNN
(AMW) accelerators at 5 GS/s and 10 GS/s datarates.

##  V Summary

In this paper, we presented SPOGA, a photonic GEMM accelerator. SPOGA utilizes
homodyne optical signals and in-transduction positional weighting of operands
through an advanced optical-analog dataflow approach. The key advantages of
SPOGA over traditional photonic and electronic GEMM accelerators are its
ability to handle byte-size integer operands and its reduction in the
overheads associated with bit-sliced integer arithmetic. These enable SPOGA to
achieve significantly lower latency and higher throughput and energy
efficiency compared to the state-of-the-art solutions. To validate the
benefits of SPOGA, we evaluated its achievable parallelism, throughput, and
energy efficiency, and compared the obtained results with other photonic GEMM
accelerators. Our analysis shows that SPOGA supports higher parallelism and
demonstrates up to 14.4$\times$ better throughput (FPS), at least 2$\times$
better energy efficiency per watt (FPS/Watt), and up to 28.5$\times$ better
energy efficiency per square millimeter (FPS/Watt/mm²) than prior GEMM
accelerators.

## References

  * [1] S.S. Vatsavai et al., ”SCONNA: A Stochastic Computing Based Optical Accelerator for Ultra-Fast, Energy-Efficient Inference of Integer-Quantized CNNs,” IEEE IPDPS, 2023.
  * [2] S.S. Vatsavai et al., “Photonic reconfigurable accelerators for efficient inference of cnns with mixed-sized tensors,” IEEE TCAD, 2022.
  * [3] Liu et al., “Holylight a nanophotonic accelerator for deep learning in data centers,” IEEE/ACM DATE, 2019.
  * [4] K. Shiflett et al., “Pixel: Photonic neural network accelerator,” IEEE HPCA, 2020.
  * [5] F. Sunny et al., “A survey on silicon photonics for deep learning,” ACM JETC, vol. 17, no.4, Oct. 2021.
  * [6] F. Sunny et al., “Crosslight: A cross-layer optimized silicon photonic neural network accelerator,” in DAC, 2021.
  * [7] Sze et al., “Efficient Processing of Deep Neural Networks: A Tutorial and Survey,” IEEE Proceedings, vol. 105, no. 12 2017.
  * [8] F. Sunny et al., “A survey on silicon photonics for deep learning,” ACM JETC, vol. 17, no.4, Oct. 2021.
  * [9] V. Bangari et al., “Digital electronics and analog photonics for convolutional neural networks (deap-cnns),” JSTQE, 2019.
  * [10] S. V. R. Chittamuru et al., “HYDRA: Heterodyne Crosstalk Mitigation with Double Microring Resonators and Data Encoding for Photonic NoC”, IEEE TVLSI, 26:1, Jan 2018.
  * [11] Q. Cheng et al., “Silicon photonics codesign for deep learning,” Proceedings of the IEEE, 2020.
  * [12] M. A. Al-Qadasi et al., “Scaling up silicon photonic-based accelerators: Challenges and opportunities,” APL Photonics, vol. 7, no. 2, 2022.
  * [13] D.-R. Oh et al., “An 8b 1gs/s 2.55mw sar-flash adc with complementary dynamic amplifiers,” in IVLSIC, 2020.
  * [14] Y.-S. Shu, “A 6b 3gs/s 11mw fully dynamic flash adc in 40nm cmos with reduced number of comparators,” in VLSIC, 2012.
  * [15] M. Guo et al., “A 29mw 5gs/s time-interleaved sar adc achieving 48.5db sndr with fully-digital timing-skew calibration based on digital-mixing,” in VLSIC, 2019.
  * [16] Eslahi, Hossein et al. “Ultra Compact and Linear 4-Bit Digital-to-Analog Converter in 22nm FDSOI Technology.” IEEE ISCAS, 2778–81, 2022.
  * [17] B. Sedighi et al., “8-Bit 5GS/s D/A Converter for Multi-Gigabit Wireless Transceivers,” EMICC, 192–95, 2011.
  * [18] F. N. U. Juanda et al. “A 10-GS/s 4-Bit Single-Core Digital-to-Analog Converter for Cognitive Ultrawidebands.” IEEE TCS II: Express Briefs 64, no. 1, 16–20, 2017.
  * [19] C. Guo et al.,”Accelerating Sparse DNNs Based on Tiled GEMM” in IEEE Transactions on Computers, vol. 73, no. 05, pp. 1275-1289, 2024.
  * [20] E. Qin et al., ”SIGMA: A Sparse and Irregular GEMM Accelerator with Flexible Interconnects for DNN Training,” IEEE HPCA, 2020.
  * [21] X. Xu et al., ”Scaling for edge inference of deep neural networks,” Nature Electronics, 1(4):216–222, Apr. 2018.
  * [22] VSP Karempudi et al. ”A Low-Dissipation and Scalable GEMM Accelerator with Silicon Nitride Photonics,” IEEE ISQED, 2024.
  * [23] S.S Vatsavai et al., ”A Comparative Analysis of Microrings Based Incoherent Photonic GEMM Accelerators,” IEEE ISQED, 2024.
  * [24] P.L. McMahon, ”The physics of optical computing,” Nature Reviews Physics 5, 717 - 734, 2023.
  * [25] N. P. Jouppi et al., ”TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings,” arXiv:2304.01433, 2023.
  * [26] P. Micikevicius et al., ”Mixed precision training,” arXiv preprint arXiv:1710.03740, 2017.
  * [27] M. Wang et al., ”NITI: Training Integer Neural Networks Using Integer-only Arithmetic”, IEEE TPDS, 2022.
  * [28] O. Destras et al., “Survey on Activation Functions for Optical Neural Networks.” ACM Computing Surveys 56, 1 - 30, 2023