# Quantum interference between fundamentally different processes is enabled by
shaped input wavefunctions

Jeremy Lim  Science, Mathematics and Technology, Singapore University of
Technology and Design, 8 Somapah Road, Singapore 487372, Singapore  Suraj
Kumar  School of Electrical and Electronic Engineering, Nanyang Technological
University, 50 Nanyang Avenue, Singapore 639798, Singapore  Yee Sin Ang
Science, Mathematics and Technology, Singapore University of Technology and
Design, 8 Somapah Road, Singapore 487372, Singapore  Lay Kee Ang  Science,
Mathematics and Technology, Singapore University of Technology and Design, 8
Somapah Road, Singapore 487372, Singapore  Liang Jie Wong
<EMAIL_ADDRESS>School of Electrical and Electronic Engineering,
Nanyang Technological University, 50 Nanyang Avenue, Singapore 639798,
Singapore

###### Abstract

We present a general framework for quantum interference (QI) between multiple,
fundamentally different processes. Our framework reveals the importance of
shaped input wavefunctions in enabling QI, and predicts unprecedented
interactions between free electrons, bound electrons, and photons: (i) the
vanishing of the zero-loss peak by destructive QI when a shaped electron
wavepacket couples to light, under conditions where the electron’s zero-loss
peak otherwise dominates; (ii) QI between free electron and atomic (bound
electron) spontaneous emission processes, which can be significant even when
the free electron and atom are far apart, breaking the common notion that
electron and atom must be close by to significantly affect each other’s
processes. Our work shows that emerging quantum waveshaping techniques unlock
the door to greater versatility in light-matter interactions and other quantum
processes in general.

Short title: A general framework for quantum interference

###### pacs:

Valid PACS appear here

##  I Introduction

Interest in controlling quantum processes has led people to seek increasingly
precise ways of manipulating the wavefunctions involved – a process known as
waveshaping. Through waveshaping, many unique wave patterns for particles
including photons and electrons have been realized by introducing a well
defined phase relation between different eigenstates. Shaped free electron
wavepackets Barwick  _et al._ (2009); Park  _et al._ (2010); Kaminer  _et al._
(2015a); Feist  _et al._ (2015); Piazza  _et al._ (2015); Priebe  _et al._
(2017); Vanacore  _et al._ (2018); Liu  _et al._ (2019); Wang  _et al._
(2020); Harvey  _et al._ (2020); Dahan  _et al._ (2020); Liebtrau  _et al._
(2021); García de Abajo and Di Giulio (2021); Vanacore  _et al._ (2020);
Yalunin  _et al._ (2021); Feist  _et al._ (2015); Zhao  _et al._ (2021a);
Morimoto and Baum (2018); Wong  _et al._ (2015); Priebe  _et al._ (2017);
Kozák  _et al._ (2018a, b); Lim  _et al._ (2019); Harris  _et al._ (2015);
Kaminer  _et al._ (2015b); Vanacore  _et al._ (2018); Grillo  _et al._ (2014);
Di Giulio  _et al._ (2019); Vanacore  _et al._ (2019); Kfir (2019); Kfir  _et
al._ (2020), for instance, are useful as probes of light-matter excitations
electron energy-loss spectroscopy (EELS) and its variants Verbeeck and Van
Aert (2004); García de Abajo (2010); Verbeeck  _et al._ (2010); Van Tendeloo
_et al._ (2012); Egoavil  _et al._ (2014); Krehl  _et al._ (2018); Polman  _et
al._ (2019); Liu  _et al._ (2019); Liebtrau  _et al._ (2021); García de Abajo
and Di Giulio (2021); López  _et al._ (2021), cathodoluminescence (CL)
microscopy García de Abajo (2010); Di Giulio  _et al._ (2021); Liebtrau  _et
al._ (2021); García de Abajo and Di Giulio (2021) and photon-induced near-
field electron microscopy (PINEM) Barwick  _et al._ (2009); Park  _et al._
(2010); Feist  _et al._ (2015); Piazza  _et al._ (2015); Priebe  _et al._
(2017); Vanacore  _et al._ (2018); Liu  _et al._ (2019); Wang  _et al._
(2020); Harvey  _et al._ (2020); Dahan  _et al._ (2020); Liebtrau  _et al._
(2021); García de Abajo and Di Giulio (2021), etc.; and also as a means of
tailoring light emission from free electrons Guzzinati  _et al._ (2017);
Roques-Carmes  _et al._ (2018); Karnieli  _et al._ (2021a, b); Gover and Pan
(2018); Pan and Gover (2018, 2019); Gover  _et al._ (2019); Faresab and Yamada
(2015); Talebi (2016); Di Giulio  _et al._ (2021); Kfir  _et al._ (2021); Wong
_et al._ (2021). Likewise, shaped bound electron (i.e., atomic states) form
the basis of many fields including quantum metrology Giovannetti  _et al._
(2011); Glickmanshlomi  _et al._ (2013); Matthews  _et al._ (2016); Sun and
Chen (2018); Sánchez Muñoz  _et al._ (2021); Glickmanshlomi  _et al._ (2013),
quantum information technologies Cirac  _et al._ (1997); Boozer  _et al._
(2007); Kimble (2008); Ritter  _et al._ (2012); Hofmannmichael  _et al._
(2012); Stute  _et al._ (2013); Northup and Blatt (2014); Heshami  _et al._
(2016); Bechler  _et al._ (2018), quantum integrated circuits Carolan  _et
al._ (2015); Kim  _et al._ (2016); Harris  _et al._ (2017); Uppu  _et al._
(2020, 2021), and photon generation and manipulation Kim  _et al._ (2016);
Michler  _et al._ (2000); Faraon  _et al._ (2008); Wu  _et al._ (2009);
Reinhard  _et al._ (2012); Chu  _et al._ (2017); Prasad  _et al._ (2020);
Stiesdal  _et al._ (2021). Many techniques exist to shape photons and quasi-
particles like polaritons Pursley  _et al._ (2018); Morin  _et al._ (2019);
Lib  _et al._ (2020); Uria  _et al._ (2020); Kondakci and Abouraddy (2017);
Bhaduri  _et al._ (2020); Wong  _et al._ (2020); Xu  _et al._ (2014); Machado
_et al._ (2018); Estrecho (2021). Recently, shaped neutron wavefunctions –
especially twisted states – have garnered interest as possible probes for
nuclear structure and interactions Larocque  _et al._ (2018); Afanasev  _et
al._ (2019, 2021), and for neutron interferometry and optics Clark  _et al._
(2015); Nsofini  _et al._ (2016); Pushin  _et al._ (2017); Cappelletti  _et
al._ (2018); Geerits and Sponar (2021).

Here, we show that shaped input wavefunctions are key to enabling quantum
interference (QI) between processes that normally (i.e. for unshaped
wavefunctions) do not affect each other significantly. It should be noted that
specific examples of QI – such as electromagnetically induced transparency
Harris  _et al._ (1990); Boller  _et al._ (1991), QI between nuclear and
electromagnetic processes Schwinger (1948); Pálffy  _et al._ (2007); Afanasev
_et al._ (2019), and QI between photons Craddock  _et al._ (2019) – have been
previously studied. However, a general framework for QI is lacking and the
role of shaped input wavefunctions has not been elucidated. This has prevented
the application of QI to the full range of existing quantum processes. We
therefore present a general framework for QI between arbitrary types and
numbers of quantum systems, where the presence of shaped input wavefunctions
emerges as a fundamental requirement for QI. We also see that QI between more
than two processes is possible, leading to a dominance of QI effects as the
number of systems with shaped input wavefunctions increases.

We use two examples to show that our framework can predict unprecedented
phenomena in light-matter interaction. In the first example, we show that QI
can eliminate the zero-loss peak of the output electron spectrum in free
electron-light interactions at moderate coupling strengths. This is
significant because the zero-loss peak is typically dominant in free electron
gain/loss spectra in PINEM experiments under the conditions we consider. We
also show that QI can dramatically enhance or suppress the other peaks in the
free electron gain/loss spectrum. Our results are achievable with parameters
well within the capabilities of current PINEM setups. In the second example,
we show that QI can occur between free electron and atomic (a.k.a., bound
electron) spontaneous emission (SE) processes. We choose this example for the
following reasons: (i) There has been much excitement and progress in shaping
electron wavepackets both spatially and temporally Barwick  _et al._ (2009);
Park  _et al._ (2010); Kaminer  _et al._ (2015a); Feist  _et al._ (2015);
Piazza  _et al._ (2015); Priebe  _et al._ (2017); Vanacore  _et al._ (2018);
Liu  _et al._ (2019); Wang  _et al._ (2020); Harvey  _et al._ (2020); Dahan
_et al._ (2020); Liebtrau  _et al._ (2021); García de Abajo and Di Giulio
(2021); Vanacore  _et al._ (2020); Yalunin  _et al._ (2021); Feist  _et al._
(2015); Zhao  _et al._ (2021a); Morimoto and Baum (2018); Wong  _et al._
(2015); Priebe  _et al._ (2017); Kozák  _et al._ (2018a, b); Lim  _et al._
(2019); Harris  _et al._ (2015); Kaminer  _et al._ (2015b); Vanacore  _et al._
(2018); Grillo  _et al._ (2014); Di Giulio  _et al._ (2019); Vanacore  _et
al._ (2019); Kfir (2019); Kfir  _et al._ (2020), but the QI this enables with
bound electron processes, such as atomic SE, has never been explored; (ii)
Free electron and bound electron SE processes are not expected to
significantly influence each other unless the free electron and bound electron
are very close by, due to the near-field nature of Coulomb interactions Gover
and Yariv (2020); Ruimy  _et al._ (2021); Zhao  _et al._ (2021b); Zhang  _et
al._ (2021a, b); Rätzel  _et al._ (2021). On the contrary, we find that owing
to QI, shaped free electrons and shaped bound electrons can affect each other
even when both systems are physically far apart. Our results show that maximum
SE enhancement or suppression can be achieved over a wide range of free
electron kinetic energies (e.g., 100 eV to 1 MeV) and emission frequencies
(e.g., optical to terahertz). Our findings fill an important gap in the
understanding of QI. Our work also motivates the development of shaping
techniques for a wider variety of quantum systems to leverage the full
potential of QI for on-demand tailoring of quantum processes in light-matter
interactions and beyond.

##  II Results

General framework for QI Consider a collection of $N$ distinct systems, e.g.,
free electrons, bound electrons, photons, atomic nuclei, neutrons, and any
other fundamental and quasi-particles. We denote the eigenstates of the $j$th
system as $\ket{\alpha_{j}}$ with corresponding eigenvalues $\alpha_{j}$. We
consider an initial state of the form
${\ket{\text{initial}}=\bigotimes_{j=1}^{N}(\sum_{\alpha_{j}}C_{\alpha_{j}}\ket{\alpha_{j}})}$,
where $|C_{\alpha_{j}}|^{2}$ is the probability of finding the $j$th system in
$\ket{\alpha_{j}}$. The probability that $\ket{\text{initial}}$ scatters into
a final state $\ket{\text{final}}=\ket{\beta_{1},...,\beta_{N}}$ after an
arbitrary interaction, described by the scattering operator
$\hat{\mathcal{S}}$, is
$P_{\text{final}}=|\braket{\text{final}}{\hat{\mathcal{S}}}{\text{initial}}|^{2}$.
Expanding $\ket{\text{initial}}$ in full and defining the coherence and
population of the $j$th system as $\rho_{\alpha_{j}\alpha_{j}^{\prime}}\equiv
C_{\alpha_{j}}\bar{C}_{\alpha_{j}^{\prime}}$ (where
$\alpha_{j}\neq\alpha_{j}^{\prime}$) and
$p_{\alpha_{j}}\equiv\rho_{\alpha_{j}\alpha_{j}}$ (overbars denote complex
conjugates) respectively, we express $P_{\text{final}}$ as

$\begin{split}P_{\text{final}}&=\overbrace{\sum_{\alpha_{1},...,\alpha_{N}}p_{\alpha_{1}}...p_{\alpha_{N}}\left|\mathcal{S}^{\text{final}}_{\alpha_{1},...,\alpha_{N}}\right|^{2}}^{\text{without
QI}}+\overbrace{\bigg{[}{N\choose 1}\text{ terms
of}\sum_{\begin{subarray}{c}\text{all except}\\\
\alpha_{j}\end{subarray}}\sum_{\begin{subarray}{c}\alpha_{j}\neq\alpha_{j}^{\prime}\end{subarray}}\rho_{\alpha_{j}\alpha_{j}^{\prime}}\frac{p_{\alpha_{1}}...p_{\alpha_{N}}}{p_{\alpha_{j}}}\mathcal{S}^{\text{final}}_{...,\alpha_{j},...}\bar{\mathcal{S}}^{\text{final}}_{...,\alpha_{j}^{\prime},...}\Bigg{]}}^{\text{1-process
QI}}\\\ &+\underbrace{\bigg{[}{N\choose 2}\text{ terms
of}\sum_{\begin{subarray}{c}\text{all except}\\\
\alpha_{j},\alpha_{k}\end{subarray}}\sum_{\begin{subarray}{c}\alpha_{j}\neq\alpha_{j}^{\prime}\\\
\alpha_{k}\neq\alpha_{k}^{\prime}\end{subarray}}\rho_{\alpha_{j}\alpha_{j}^{\prime}}\rho_{\alpha_{k}\alpha_{k}^{\prime}}\frac{p_{\alpha_{1}}...p_{\alpha_{N}}}{p_{\alpha_{j}}p_{\alpha_{k}}}\mathcal{S}^{\text{final}}_{...,\alpha_{j},\alpha_{k},...}\bar{\mathcal{S}}^{\text{final}}_{...,\alpha_{j}^{\prime},\alpha_{k}^{\prime},...}\Bigg{]}}_{\text{2-process
QI}}\\\ &+\underbrace{\Bigg{[}{N\choose 3}\text{ terms
of}\sum_{\begin{subarray}{c}\text{all except}\\\
\alpha_{j},\alpha_{k},\alpha_{l}\end{subarray}}\sum_{\begin{subarray}{c}\alpha_{j}\neq\alpha_{j}^{\prime}\\\
\alpha_{k}\neq\alpha_{k}^{\prime}\\\
\alpha_{l}\neq\alpha_{l}^{\prime}\end{subarray}}\rho_{\alpha_{j}\alpha_{j}^{\prime}}\rho_{\alpha_{k}\alpha_{k}^{\prime}}\rho_{\alpha_{l}\alpha_{l}^{\prime}}\frac{p_{\alpha_{1}}...p_{\alpha_{N}}}{p_{\alpha_{j}}p_{\alpha_{k}}p_{\alpha_{l}}}\mathcal{S}^{\text{final}}_{...,\alpha_{j},\alpha_{k},\alpha_{l},...}\bar{\mathcal{S}}^{\text{final}}_{...,\alpha_{j}^{\prime},\alpha_{k}^{\prime},\alpha_{l}^{\prime},...}\Bigg{]}}_{\text{3-process
QI}}\\\
&+\quad\dots\quad+\underbrace{\Bigg{[}\sum_{\alpha_{1}\neq\alpha_{1}^{\prime},...,\alpha_{N}\neq\alpha_{N}^{\prime}}\rho_{\alpha_{1}\alpha_{1}^{\prime}}\dots\rho_{\alpha_{N}\alpha_{N}^{\prime}}\mathcal{S}^{\text{final}}_{\alpha_{1},...,\alpha_{N}}\bar{\mathcal{S}}^{\text{final}}_{\alpha_{1}^{\prime},...,\alpha_{N}^{\prime}}\Bigg{]}}_{\text{$N$-process
QI}},\end{split}$ (1)

where
$\mathcal{S}^{\text{final}}_{\alpha_{1},...,\alpha_{N}}\equiv\braket{\text{final}}{\hat{\mathcal{S}}}{\alpha_{1},..,\alpha_{N}}$
is the S-matrix element. The total scattering probability into states that
share the same final values of quantum numbers $\beta_{m},...,\beta_{n}$ is
$P_{\beta_{m},...,\beta_{n}}=\sum_{\text{all
except}\beta_{m},...,\beta_{n}}P_{\text{final}}$. The first term of Eq. (1) is
the total probability in the absence of QI. Note that the absence of QI
corresponds to the scenario where multiple eigenstates exist, but they are
related by random phases, resulting in QI disappearing upon statistical
averaging. The terms in the $R$th square parentheses, where
$R\in\mathbb{Z}^{+}$, contain the ${N\choose R}$ possible QI terms that can
arise between $R$ of the $N$ systems. Crucially, our framework reveals the
importance of shaped input wavefunctions as a prerequisite for (and a means to
tailor) QI: the initial wavefunctions of the systems participating in QI must
be a superposition of eigenstates with well-defined phase relations between
them, such that the coherences of the systems involved in the QI – and hence
the relevant QI terms in Eq. (1) – are non-zero upon statistical averaging.

Figure 1: A general framework for quantum interference (QI). QI is enabled by
shaped input quantum wavefunctions, providing a means to tailor quantum
processes via quantum waveshaping. To illustrate this, we consider the example
of 5 systems (B-F) coupled to system A. (a) When systems B to F have unshaped
(i.e., either single-input state or a superposition of eigenstates with random
phase relations, represented by solid circles) input wavefunctions, QI is
absent and there are only direct coupling pathways between B-F and A. As a
result the total coupling between system A and the other systems is simply the
sum of the individual direct coupling terms considered in isolation. (b) When
some of the systems (B-D here) have shaped input wavefunctions (i.e.,
superposition of eigenstates with fixed phase relations, denoted by
overlapping transluscent circles), QI occurs between the processes associated
with the shaped wavefunctions, resulting in additional coupling pathways.
Notably, QI can involve more than two processes in general, resulting in a
dominance in the number of QI terms as the number of systems with shaped
wavefunctions increases. Note that 1-process QI can occur for instance, when
the scattering events involve classical fields (e.g., self-loop in coupling
between system C and classical field).

Figure 1 illustrates this by considering a system A coupled in a pairwise
manner to systems B to F (note in general that Eq. (1) is not limited to
pairwise coupling). We see that shaping the input wavefunctions of systems B
to D results in 2-process and 3-process QI between the processes associated
with these shaped wavefunctions, providing additional coupling pathways to
couple to A. 1-process QI can arise, for instance, when a system C with its
input wavefunction shaped couples to classical fields (e.g., a light wave).

Figure 2: Quantum interference (QI) in free electron-light interactions at
moderate coupling strengths, resulting in elimination of the zero-loss peak
(ZLP) by destructive QI, and enhancement of satellite peaks by constructive
QI. (a) An incoming shaped free electron (modeled as a quantum electron
wavepacket (QEW)), with input spectrum shown in (b), scatters off a classical
light field with a dimensionless coupling strength $\mathcal{G}=0.7$,
resulting in an output electron spectrum where the ZLP completely vanishes. If
QI contributions are neglected (unfilled bars with blue outlines in (c)),
which occurs, for instance, when there is no fixed phase relation between the
input eigenstates, the ZLP dominates. (d) In contrast, for an unshaped
incoming QEW (input spectrum shown in (e)), the output QEW spectrum in the
presence and absence of QI coincide (as shown in (f)), which implies that QI
contributions vanish for unshaped input QEWs. (g) and (h) compare the output
QEW spectrum in the presence and absence of QI, respectively, as a function of
$\mathcal{G}$. The contribution of QI is already substantial even for weaker
interactions of about $\mathcal{G}\sim 0.1$. As shown in (h), the complete
suppression of gain/loss peaks away from the ZLP can be achieved for coupling
strengths $\mathcal{G}\gtrsim 1$.

Elimination of zero-loss peak by QI in free electron-light interactions When a
shaped incoming free electron, i.e., a quantum electron wavepacket (QEW), is
scattered by a classical light wave at moderate coupling strengths (Figs.
2(a),(b)), the output energy gain/loss spectrum (orange bars in Fig. 2(c))
shows a complete suppression of the zero-loss peak (ZLP) – this is a direct
consequence of QI. If QI is neglected the ZLP remains dominant in the output
spectrum (blue-outlined unfilled bars in Fig. 2(c)). Note that neglecting QI
corresponds to the physical scenario where a random phase relation exists
between the input electron eigenstates, which results in the disappearance of
QI effects upon statistical averaging. In addition to the complete suppression
of the ZLP, QI also enhances the gain/loss peaks away from the ZLP. In
contrast, for an unshaped incoming QEW (Figs. 2(d),(e)), there are no QI
contributions and the ZLP dominates in the output spectrum (Fig. 2(f)). We
model the wavefunction of the incoming QEW with the general form
$\ket{\mathrm{initial}}=\sum_{n}C_{n}\ket{n}$, where
$C_{n}=e^{i\phi_{\mathrm{mod}}}J_{n}(2|\mathcal{G}_{\mathrm{mod}}|)$ is the
initial complex amplitude of the $n$th energy gain/loss peak of the incoming
QEW Feist  _et al._ (2015), $\phi_{\mathrm{mod}}$ is the modulation phase
($\phi_{\mathrm{mod}}=0$), $J_{n}()$ is the Bessel function of the first kind,
and $\mathcal{G}_{\mathrm{mod}}$ is the dimensionless coupling strength of the
shaping stage ($\mathcal{G}_{\mathrm{mod}}=0.5$ in (a)-(c),(g),(h) and
$\mathcal{G}_{\text{mod}}=0$ in (d)-(f)). The exact QEW-light interaction is
described by the S-matrix
$\hat{\mathcal{S}}\equiv\exp(\mathcal{G}^{*}\hat{b}-\mathcal{G}\hat{b}^{\dagger})$
Feist  _et al._ (2015), where $\mathcal{G}$ is the dimensionless coupling
strength between the incoming QEW and the classical light field. Here,
$\hat{b}$ ($\hat{b}^{\dagger}$) decrements (increments) each QEW eigenstate by
a unit photon energy. We obtain the $N$th energy gain/loss peak of the final
output QEW probability spectrum (i.e., the probability that
$\ket{\text{initial}}$ scatters into a final state $\ket{N}$) as

$P_{N}=\underbrace{\sum_{n}|C_{n}|^{2}|\braket{N}{\hat{\mathcal{S}}}{n}|^{2}}_{\text{without
QI}}+\underbrace{\sum_{m\neq
n}C_{m}C_{n}^{*}\braket{N}{\hat{\mathcal{S}}}{m}\braket{N}{\hat{\mathcal{S}}}{n}^{*}}_{\text{1-process
QI}},\quad m,n,N\in\mathbb{Z},$ (2)

which is the sum of the first and second terms in Eq. (1) with only a single
system (the QEW) as input. In Figs. 2(a)-(f), we consider a coupling strength
of $\mathcal{G}=0.7$, which is well within the reach of existing PINEM setups,
where coupling strengths on the order of $\mathcal{G}\sim 100$ has been
demonstrated Dahan  _et al._ (2020). For smaller coupling strengths of
$\mathcal{G}\gtrsim 0.1$, the ZLP appears but remains suppressed by
destructive QI, as we show in Figs. 2(g) and 2(h). Additionally, for
$\mathcal{G}\gtrsim 1$, complete suppression of the gain/loss peaks away from
the ZLP can also occur due to QI. We find that our results still hold for
incoming QEWs of different shapes (Supplemental Material (SM) Section I).

Figure 3: Quantum interference (QI) provides a means to tailor spontaneous
emission from free electrons and bound electrons via quantum waveshaping. (a)
Shaped quantum electron wavepackets (QEWs) and bound electrons (atomic two-
level system) within an electromagnetic (EM) environment separately emit
photons at rates of $\Gamma^{\mathrm{ep}}$ and $\Gamma^{\mathrm{ap}}$
respectively. QI between these spontaneous emission (SE) processes (dotted
line) results in a third emission process at rate $\Gamma^{\mathrm{ap/ep}}$,
which can enhance or suppress the total SE rate as a function of interaction
length $L$ (as shown in (b)) by more than $70\%$. (c) In contrast, for
unshaped incoming QEWs, QI is absent (as shown in (d)) and the total SE rate
is $\Gamma^{\mathrm{ap}}+\Gamma^{\mathrm{ep}}$. QI can be tailored using the
QEW and bound electron shapes, which are determined by bunching factor
$\langle b\rangle=|\langle b\rangle|e^{i\Psi_{b}}$ and coherence
$\rho_{eg}^{a}=|\rho_{eg}^{a}|e^{i\phi_{a}}$ respectively. Defining the figure
of merit
$\gamma=\Gamma^{\mathrm{ap/ep}}/(\Gamma^{\mathrm{ap}}+\Gamma^{\mathrm{ep}})$,
which is a measure of the QI contribution, we see from (e) that larger
$|\gamma_{\mathrm{max}}|=\mathrm{max}(|\gamma|)$ is achieved for larger
$|\langle b\rangle|$, and that QI can be tuned to enhance or suppress the SE
rate by controlling the phase of the bunching factor $\Psi_{b}$. (f) shows the
dependence of the optimal length $L_{\mathrm{opt}}$ – at which $\gamma_{max}$
is achieved – as a function of QEW kinetic energy $E_{\mathrm{K}}$ and bound
electron emission frequency $\omega_{a}$. The polar plots in (g)-(j) show the
value of $|\gamma|$ on the Bloch sphere representing the initial shape of the
bound electron system at various values of $L$, and (h),(i) shows that the
profile of $|\gamma|$ on the initial Bloch sphere can be manipulated by
varying $\Psi_{b}$. Unless otherwise stated, we consider a 30 keV shaped QEW
with $\langle b\rangle=0.99$. Our two-level, bound electron system is a Sn-N
vacancy with $\omega_{a}\approx 3\times 10^{15}$ rad/s and transition dipole
moment of $|\mathbf{d}|=4.33\times 10^{-29}$ Cm aligned parallel to the field.
For (g)-(j), the azimuthal angle $\phi_{a}$ is the phase of the bound electron
coherence and $\theta_{a}$ is related to the excited state population through
$\cos^{2}(\theta_{a}/2)$, and we use a 30 keV shaped QEW with $\langle
b\rangle=0.58$.

QI between free electron and bound electron spontaneous emission processes We
now apply our framework to study QI between free electron and bound electron
SE processes in a cavity (or any electromagnetic environment in general). A
shaped incoming QEW (Fig. 3(a)) can induce a QI SE contribution
$\Gamma^{\mathrm{ap/ep}}$ which enhances or suppresses the sum of the
individual SE processes $\Gamma^{\mathrm{ap}}+\Gamma^{\mathrm{ep}}$ by more
than $70\%$, depending on interaction length $L$ and cavity angular frequency
$\omega_{\text{cav}}$ (Fig. 3(b)). Here $\Gamma^{\mathrm{ap}}$ and
$\Gamma^{\mathrm{ep}}$ denote the bound electron and free electron SE rates
respectively. In contrast, there is no QI for an unshaped (Gaussian) incoming
QEW (Figs. 3(c),(d)). As expected, we find that the SE rates peak sharply at
resonance ($\omega_{\mathrm{cav}}=\omega_{a}$). We consider the resonant case
for the rest of this example. The incoming QEW of velocity
$\mathbf{v}=v_{0}\mathbf{\hat{z}}$ has a central kinetic energy of
$E_{\text{K}}=30$ keV. We treat the bound electron system as a two-level
atomic system by considering a tin-vacancy (SnV) center Trusheim  _et al._
(2020) of emission frequency $\omega_{a}\approx 3\times 10^{15}$ rad/s and
dipole moment $\mathbf{d}=\mathbf{\hat{z}}4.33\times 10^{-29}$ Cm (aligned
parallel to the field).

Importantly, the shapes of the QEW and bound electron can be used to tailor
$\Gamma^{\mathrm{ap/ep}}$. We define the figure of merit
$\gamma=\Gamma^{\mathrm{ap/ep}}/(\Gamma^{\mathrm{ap}}+\Gamma^{\mathrm{ep}})$
as a measure of QI’s relative contribution to the SE rate. Figure 3(e) shows
how the incoming QEW shape – determined by the bunching factor $\langle
b\rangle=|\langle b\rangle|e^{i\Psi_{b}}$ – affects $\gamma_{\mathrm{max}}$,
which is the maximum possible $\gamma$ across all $L$. For the case shown in
Fig. 3(d) (black cross in Fig. 3(e)), we considered $\langle b\rangle=0.99$,
which has recently been shown to be feasible Yalunin  _et al._ (2021). Even
for a more modest bunching factor of $\langle b\rangle\approx 0.58$, which is
attainable using PINEM (see SM Section II and Yalunin  _et al._ (2021)), for
instance, $|\gamma_{\mathrm{max}}|\gtrsim 0.4$ can still be achieved. Varying
the phase $\Psi_{b}$ (e.g., using phase of the modulating field) also allows
us to control the amount by which the overall spontaneous emission rate is
enhanced or suppressed. Figure. 3(f) shows that the optimal interaction length
$L_{\mathrm{opt}}$ needed to achieve $|\gamma_{\mathrm{max}}|\approx 0.707$,
which is on the order of $\sim 10$ nm to $\sim 100$ $\mu$m, falls in the range
of experimentally realizable optical Vahala (2003); Vaidya  _et al._ (2018);
Davis  _et al._ (2019); Muniz  _et al._ (2020) and terahertz Todorov  _et al._
(2009); Derntl  _et al._ (2017); Lu  _et al._ (2018); Liu  _et al._ (2020);
Messelot  _et al._ (2020) waveguide/cavity dimensions. Furthermore, the
required QEW energies, which range from $\sim$100 eV to $\sim 1$ MeV, are
achievable using lab-scale electron sources. This implies that it should be
already feasible to perform experiments to observe QI in SE from
superconducting qubits Kjaergaard  _et al._ (2020) and quantum dots Patel  _et
al._ (2010); Flagg  _et al._ (2010); Loredo  _et al._ (2016); Somaschi  _et
al._ (2016); Ding  _et al._ (2016); Wei  _et al._ (2014); Aharonovich  _et
al._ (2016); Gao  _et al._ (2015); Veldhorst  _et al._ (2015), which radiate
in the terahertz and optical regimes (SM Section III). An approximate
analytical expression for $L_{\mathrm{opt}}$ is presented in SM Section IV.

The QI contribution can also be controlled via the bound electron shape, which
is determined by the coherence $\rho_{eg}^{a}=|\rho_{eg}^{a}|e^{i\phi_{a}}$
between its excited and ground states. We show this in Figs. 3(g)-(j), which
depict $|\gamma|$ as a function of the initial bound electron Bloch sphere at
various values of $L$. Here, $\theta_{a}$ (radial coordinate) and $\phi_{a}$
(angular coordinate) are related to the excited state population
$\rho_{ee}^{a}$ and coherence $\rho^{a}_{eg}$ through
$\rho_{ee}^{a}=\cos^{2}(\theta_{a}/2)$ and
$\rho_{eg}^{a}=(e^{i\phi_{a}}/2)\sin\theta_{a}$ respectively. The bunching
factor phase $\Psi_{b}$ can be used to azimuthally rotate the profile of
$\gamma$ on the bound electron Bloch sphere, as seen in Figs 3(h),(i).

For Fig. 3, we model the QI contribution $\Gamma^{\mathrm{ap/ep}}$ by
considering a QEW of velocity $\mathbf{v}=v_{0}\mathbf{\hat{z}}$, modulated at
frequency $\omega_{\mathrm{mod}}$ and bunching factor $\langle
b\rangle=|\langle b\rangle|e^{i\Psi_{b}}$ passing through an electromagnetic
(EM) environment (e.g., cavity, waveguide) of length $L$ (also the interaction
length) containing a bound electron system of coherence
$\rho_{eg}^{a}=|\rho_{eg}^{a}|e^{i\phi_{a}}$. To first order in perturbation
theory (i.e., weak coupling regime), we find that the single-photon SE rate
arising from the QI between the free electron and bound electron SE processes
is (SM Section V)

$\begin{split}\Gamma^{\mathrm{ap/ep}}=&\frac{\tau}{\hbar}\frac{ev_{0}\omega_{a}|\mathbf{d}|}{\epsilon_{0}V\omega_{\mathrm{cav}}}|\rho^{a}_{eg}||\langle
b\rangle|\cos(\xi)\mathrm{sinc}\bigg{[}\frac{(\omega_{\mathrm{cav}}-\omega_{a})\tau}{2}\bigg{]}\mathrm{sinc}\bigg{[}\frac{(\beta_{0}\omega_{\mathrm{cav}}-\omega_{\mathrm{mod}})\tau}{2}\bigg{]}\mathrm{sinc}\bigg{[}\frac{(\omega_{\mathrm{cav}}-\omega_{\mathrm{mod}})\tau}{2}\bigg{]},\end{split}$
(2)

where the bound electron is located at $\mathbf{r}=(0,0,z_{a})$. The EM
environment supports a single dominant longitundinal field mode of angular
frequency $\omega_{\mathrm{cav}}$ and wavevector
$\mathbf{q}=(0,0,\omega_{\mathrm{cav}}/c)$, where $c$ is the free-space speed
of light. Such a mode is realizable, for instance, using a racetrack waveguide
Kfir (2019). In Eq. (2), $\hbar$ is the reduced Planck constant,
$\epsilon_{0}$ is the free-space permitivitty, $e>0$ is the elementary charge,
$\tau=L/v_{0}$ is the interaction duration, $\mathbf{d}$ is the bound electron
transition dipole moment, $V$ is the mode volume, $m_{e}$ is the electron rest
mass, $\beta_{0}=v_{0}/c$ is the normalized free electron velocity,
$\xi=\phi_{a}-(\omega_{\mathrm{cav}}z_{a}/c)-\pi/2+\Psi_{b}$, and $\phi_{a}$
is the bound electron coherence phase. The total SE rate is simply
$\Gamma^{\mathrm{ap}}+\Gamma^{\mathrm{ep}}+\Gamma^{\mathrm{ap/ep}}$. Unless
otherwise stated, the initial excited and ground state populations are equal,
corresponding to coherence magnitude $|\rho_{eg}^{a}|=1/2$. We set
$\phi_{a}-(\omega_{\mathrm{cav}}z_{a}/c)=\pi/2$, which maximizes the
contribution of $\Gamma^{\mathrm{ap/ep}}$. Note that
$\Gamma^{\mathrm{ap}}+\Gamma^{\mathrm{ep}}$ and $\Gamma^{\mathrm{ap/ep}}$ are
derived from the first and third terms of Eq. (1) respectively when 3 systems
(QEW, bound electron, and photon) are considered and the final state is summed
over all possible output states containing 1 photon. The second term in Eq.
(1) (1-process QI) vanishes. Importantly, we see from Eq. (2) that if either
the free electron state or bound electron state (or both) is unshaped
(corresponding to $\langle b\rangle=0$ and $\rho_{eg}^{a}=0$ respectively),
$\Gamma^{\mathrm{ap/ep}}=0$ and the QI contribution vanishes, which we expect.

In our specific example of QI between free electron SE and bound electron SE,
we find that QI affects the total SE rate substantially even when the free
electron and bound electron systems are physically far apart. This is in sharp
contrast to the Coulomb interaction between the atomic system and the QEW,
which relies on the proximity between the atom and the QEW, and which has been
leveraged in free-electron-bound-electron-resonant interaction (FEBERI) Gover
and Yariv (2020); Ruimy  _et al._ (2021); Zhao  _et al._ (2021b); Zhang  _et
al._ (2021a, b); Rätzel  _et al._ (2021) to encode information on bound
electron coherence and dephasing in electron spectra. In this regard, our work
could provide a complementary route towards free-electron quantum metrology
without the requirement of the bound electron system and QEW being physically
near each other.

##  III Discussion

In essence, the far-reaching implications of our general QI framework are as
follows: (i) Fundamentally distinct quantum processes can be made to affect
each other through QI; (ii) In the presence of multiple shaped wavefunctions,
multiple types of QI can arise, which can lead to the dominance in the number
of QI terms in the overall output rate; (iii) QI can exist not only between
quantum systems, but also between quantum systems and classical fields. These
QI-driven effects are enabled by shaped wavefunctions – a fundamental
requirement revealed by our general framework, which provides the connection
between shaped input wavefunctions and QI, showing that one implies the
another. This motivates the development of innovative shaping techniques for
fundamental particles and other quantum systems. Neutrons, for instance, with
their ability to couple to all four fundamental forces, hypothetical particles
(e.g., dark matter, axions) and interactions (e.g., modified gravity) Sponar
_et al._ (2021), are promising testbeds for the foundations of cosmology and
quantum mechanics. Ongoing efforts to shape the neutron Clark  _et al._
(2015); Cappelletti  _et al._ (2018) imply that it may soon be possible to
observe QI between neutron-driven processes and other types of processes. This
could, for instance, provide additional degrees of freedom through which these
exotic interactions and particles can be probed.

While we have only considered unentangled input quantum states, by using a
more general expression for the input state
$\ket{\text{initial}}=\sum_{\alpha_{1},...,\alpha_{N}}C_{\alpha_{1},...,\alpha_{N}}\ket{\alpha_{1}}\otimes...\otimes\ket{\alpha_{N}}$,
where $C_{\alpha_{1},...,\alpha_{N}}\neq C_{\alpha_{1}}...C_{\alpha_{N}}$ in
general, our framework can be readily extended to study the effects of
entangled input states, opening up a rich field of exploration. Our findings
also suggest exciting prospects for applying QEWs that go beyond controlling
photon emission, such as interference between free electron-photon and free
electron-bound electron interactions for manipulation of free electron
wavepackets and photon statistics, and interference between free electron-
bound electron and bound electron-photon interactions for manipulation of
bound electron population and coherence.

The general framework for QI we present also subsumes phenomena like
electromagnetically induced transparency Harris  _et al._ (1990); Boller  _et
al._ (1991), where destructive QI between transition amplitudes in a three-
level system renders the system transparent in a spectral window, as well as
weakly coupled free electron-photon interactions Pan and Gover (2019); Pan
_et al._ (2019), where QI is analyzed as mixed-order terms arising from the
interference between orders of a perturbative series expansion. Our framework
goes far beyond the prediction of these phenomena, as we show through two
unprecedented physical interactions: the vanishing of zero-loss peak in
electron-light interactions for moderate coupling strengths, and QI between
free electrons and bound electron SE processes. Unlike any other existing
framework, our framework shows that shaped wavefunctions are essential for QI,
and can enable QI between fundamentally different processes.

In summary, we present a general framework that describes QI between arbitrary
types and numbers of quantum systems, revealing that shaped input
wavefunctions are necessary in enabling QI. Intriguingly, we find that QI
between more than two processes is possible, leading to a dominance in the
number of QI terms as the number of shaped input increases. We apply our
general QI framework to two examples and predict unprecedented phenomena in
light-matter interactions. In the first example, we show using experimentally
realistic parameters that QI can eliminate the zero-loss peak of the output
free electron spectrum in free electron-light interactions at moderate
coupling strengths – a significant result since the zero-loss peak is
typically dominant in the output electron spectra of PINEM. We also show that
QI can dramatically enhance or suppress the satellite peaks in the output free
electron spectrum. In the second example, we show that QI can occur between
shaped free electron and bound electron SE processes even when both systems
are physically distant and not able to interact via the Coulomb force. We find
that the total SE rate can be enhanced or suppressed by up to $70\%$ relative
to the sum of isolated free electron and bound electron SE rates as a direct
consequence of QI. Our findings fill an important gap in the understanding of
waveshaping as a prerequisite of QI, and introduce unexplored methods of
tailoring and optimizing quantum interactions. Coupled with growing interest
in shaping a wide range of quantum systems, including free electrons and
neutrons, our work unlocks the possibility of QI for on-demand of tailoring
light-matter interactions and beyond.

###### Acknowledgements.

Acknowledgements We thank I. Kaminer and N. Rivera for insightful discussions.
L.J.W. acknowledges the support of the National Research Foundation (Project
ID NRF2020-NRF-ISF004-3525) and the Nanyang Assistant Professorship Start-up
Grant. J.L. and L.K.A. acknowledge funding from A*STAR AME IRG (Project ID
A2083c0057), MOE PhD Research Scholarship, and USA Office of Naval Research
(Global) grant (Project ID N62909-19-1-2047). Y.S.A. acknowledges funding from
SUTD Startup Research Grant (Project ID SRT3CI21163).

## References

  * Barwick  _et al._ (2009) B. Barwick, D. J. Flannigan, and A. H. Zewail, “Photon-induced near-field electron microscopy,” Nature 462, 902?906 (2009).
  * Park  _et al._ (2010) T. S. Park, M. Lin, and A. H. Zewail, “Photon-induced near-field electron microscopy (pinem): theoretical and experimental,” New J. Phys. 12, 123028 (2010).
  * Kaminer  _et al._ (2015a) I. Kaminer, J. Nemirovsky, M. Rechtsman, R. Bekenstein, and M. Segev, “Self-accelerating dirac particles and prolonging the lifetime of relativistic fermions,” Nature Phys. 11, 261–267 (2015a).
  * Feist  _et al._ (2015) A. Feist, K. Echternkamp, J. Schauss, S. V. Yalunin, S. Schäfer, and C. Ropers, “Quantum coherent optical phase modulation in an ultrafast transmission electron microscope,” Nature 521, 200?203 (2015).
  * Piazza  _et al._ (2015) L. Piazza, T. Lummen, E. Quiñonez, Y. Murooka, B. W. Reed, B. Barwick, and F. Carbone, “Simultaneous observation of the quantization and the interference pattern of a plasmonic near-field,” Nat. Commun. 6, 6407 (2015).
  * Priebe  _et al._ (2017) K.E. Priebe, C. Rathje, S.V. Yalunin, T. Hohage, A. Feist, S. Schäfer, and C. Ropers, “Attosecond electron pulse trains and quantum state reconstruction in ultrafast transmission electron microscopy,” Nat. Photonics 11, 793?797 (2017).
  * Vanacore  _et al._ (2018) G. M. Vanacore, I. Madan, G. Berruto, _et al._ , “Attosecond coherent control of free-electron wave functions using semi-infinite light fields,” Nat. Commun. 9, 2694 (2018).
  * Liu  _et al._ (2019) C. Z. Liu, Y. Wu, Z. Hu, _et al._ , “Continuous wave resonant photon stimulated electron energy-gain and electron energy-loss spectroscopy of individual plasmonic nanoparticles,” ACS Photonics 6, 2499?2508 (2019).
  * Wang  _et al._ (2020) K. P. Wang, R. Dahan, M. Shentcis, Y. Kauffmann, A. Ben Hayun, S. Reinhardt, O. Tsesses, and I. Kaminer, “Coherent interaction between free electrons and a photonic cavity,” Nature 582, 50?54 (2020).
  * Harvey  _et al._ (2020) T. R. Harvey, J.-W. Henke, O. Kfir, H. Lourenço-Martins, A. Feist, F. J. García de Abajo, and C. Ropers, “Probing chirality with inelastic electron-light scattering,” Nano Lett. 20, 4377?4383 (2020).
  * Dahan  _et al._ (2020) R. Dahan, S. Nehemia, M. Shentcis, O. Reinhardt, Y. Adiv, X. Shi, O. Be?er, M. H. Lynch, Y. Kurman, K. Wang, and I. Kaminer, “Resonant phase-matching between a light wave and a free-electron wavefunction,” Nat. Phys. 16, 1123?1131 (2020).
  * Liebtrau  _et al._ (2021) M. Liebtrau, M. Sivis, A. Feist, H. Lourenço-Martins, N. Pazos-Pérez, R. A. Alvarez-Puebla, F. J. García de Abajo, A. Polman, and C. Ropers, “Spontaneous and stimulated electron?photon interactions in nanoscale plasmonic near fields,” Light Sci. Appl. 10, 82 (2021).
  * García de Abajo and Di Giulio (2021) F. J. García de Abajo and V. Di Giulio, “Optical excitations with electron beams: Challenges and opportunities,” ACS Photonics 8, 945–974 (2021).
  * Vanacore  _et al._ (2020) G. M. Vanacore, I. Madan, and F. Carbone, “Spatio-temporal shaping of a free-electron wave function via coherent light?electron interaction,” Riv. del Nuovo Cim. 43, 567?597 (2020).
  * Yalunin  _et al._ (2021) S. V. Yalunin, A. Feist, and C. Ropers, “Tailored high-contrast attosecond electron pulses for coherent excitation and scattering,” Phys. Rev. Research 3, L032036 (2021).
  * Zhao  _et al._ (2021a) Z. Zhao, K. J. Leedle, D. S. Black, O. Solgaard, R. L. Byer, and S. Fan, “Electron pulse compression with optical beat note,” Phys. Rev. Lett. 127, 164802 (2021a).
  * Morimoto and Baum (2018) Y. Morimoto and P. Baum, “Diffraction and microscopy with attosecond electron pulse trains,” Nat. Phys. 14, 252?256 (2018).
  * Wong  _et al._ (2015) L. J. Wong, B. Freelon, T. Rohwer, N Gedik, and S. G. Johnson, “All-optical three-dimensional electron pulse compression,” New J. Phys. 17, 013051 (2015).
  * Kozák  _et al._ (2018a) M. Kozák, T. Eckstein, N. Schönenberger, and P. Hommelhoff, “Inelastic ponderomotive scattering of electrons at a high-intensity optical travelling wave in vacuum,” Nat. Phys. 14, 121?125 (2018a).
  * Kozák  _et al._ (2018b) M. Kozák, N. Schönenberger, and P. Hommelhoff, “Ponderomotive generation and detection of attosecond free-electron pulse trains,” Phys. Rev. Lett. 120, 103203 (2018b).
  * Lim  _et al._ (2019) J. Lim, Y. D. Ching, and L. J. Wong, “Terahertz-optical intensity grating for creating high-charge, attosecond electron bunches,” New J. Phys. 21, 033020 (2019).
  * Harris  _et al._ (2015) J. Harris, V. Grillo, E. Mafakheri, G. C. Gazzadi, S. Frabboni, R. W. Boyd, and E. Karimi, “Structured quantum waves,” Nat. Phys. 11, 629–634 (2015).
  * Kaminer  _et al._ (2015b) I. Kaminer, J. Nemirovsky, M. Rechtsman, R. Bekenstein, and M. Segev, “Self-accelerating dirac particles and prolonging the lifetime of relativistic fermions,” Nat. Phys. 11, 261?267 (2015b).
  * Grillo  _et al._ (2014) V. Grillo, E. Karimi, G. C. Gazzadi, S. Frabboni, M. R. Dennis, and R. W. Boyd, “Generation of nondiffracting electron bessel beams,” Phys. Rev. X 4, 011013 (2014).
  * Di Giulio  _et al._ (2019) V. Di Giulio, M. Kociak, and F. J. Garcia de Abajo, “Probing quantum optical excitations with fast electrons,” Optica 6, 1524 (2019).
  * Vanacore  _et al._ (2019) G. M. Vanacore, G. Berruto, and I. Madan, “Ultrafast generation and control of an electron vortex beam via chiral plasmonic near fields,” Nat. Mater. 18, 573?579 (2019).
  * Kfir (2019) O. Kfir, “Entanglements of electrons and cavity photons in the strong-coupling regime,” Phys. Rev. Lett. 123, 103602 (2019).
  * Kfir  _et al._ (2020) O. Kfir, H. Lourenço-Martins, G. Storeck, M. Sivis, T. R. Harvey, T. J. Kippenberg, A. Feist, and C. Ropers, “Controlling free electrons with optical whispering-gallery modes,” Nature 582, 46?49 (2020).
  * Verbeeck and Van Aert (2004) J. Verbeeck and S. Van Aert, “Model based quantification of eels spectra,” Ultramicroscopy 101, 207–224 (2004).
  * García de Abajo (2010) F. J. García de Abajo, “Optical excitations in electron microscopy,” Rev. Mod. Phys. 82, 209–275 (2010).
  * Verbeeck  _et al._ (2010) J. Verbeeck, H. Tian, and P. Schattschneider, “Production and application of electron vortex beams,” Nature 467, 301–304 (2010).
  * Van Tendeloo  _et al._ (2012) G. Van Tendeloo, S. Bals, S. Van Aert, J. Verbeeck, and D. Van Dyck, “Advanced electron microscopy for advanced materials,” Adv. Mater. 24, 5655–5675 (2012).
  * Egoavil  _et al._ (2014) R. Egoavil, N. Gauquelin, G.T. Martinez, S. Van Aert, G. Van Tendeloo, and J. Verbeeck, “Atomic resolution mapping of phonon excitations in STEM-EELS experiments,” Ultramicroscopy 147, 1–7 (2014).
  * Krehl  _et al._ (2018) J. Krehl, G. Guzzinati, J. Schultz, P. Potapov, D. Pohl, J. Martin, J. Verbeeck, A. Fery, B. Büchner, and A. Lubk, “Spectral field mapping in plasmonic nanostructures with nanometer resolution,” Nat. Commun. 9, 4207 (2018).
  * Polman  _et al._ (2019) A. Polman, M. Kociak, and F. J. García de Abajo, “Electron-beam spectroscopy for nanophotonics,” Nat. Mater. 18, 1158?1171 (2019).
  * López  _et al._ (2021) E. A. López, V. Di Giulio, and F. J. García de Abajo, “Atomic floquet physics revealed by free electrons,” (2021), arXiv:2111.08675 [quant-ph] .
  * Di Giulio  _et al._ (2021) V. Di Giulio, O. Kfir, C. Ropers, and F. J. García de Abajo, “Modulation of cathodoluminescence emission by interference with external light,” ACS Nano 15, 7290–7304 (2021).
  * Guzzinati  _et al._ (2017) G. Guzzinati, A. Béché, H. Lourenço-Martins, J. Martin, M. Kociak, and J. Verbeeck, “Probing the symmetry of the potential of localized surface plasmon resonances with phase-shaped electron beams,” Nat. Commun. 8, 14999 (2017).
  * Roques-Carmes  _et al._ (2018) C. Roques-Carmes, N. Rivera, J. D. Joannopoulos, M. Soljačić, and I. Kaminer, “Nonperturbative quantum electrodynamics in the Cherenkov effect,” Phys. Rev. X 8, 041013 (2018).
  * Karnieli  _et al._ (2021a) A. Karnieli, N. Rivera, A. Arie, and I. Kaminer, “Superradiance and subradiance due to quantum interference of entangled free electrons,” Phys. Rev. Lett. 127, 060403 (2021a).
  * Karnieli  _et al._ (2021b) A. Karnieli, N. Rivera, A. Arie, and I. Kaminer, “The coherence of light is fundamentally tied to the quantum coherence of the emitting particle,” Sci. Adv. 7, eabf8096 (2021b).
  * Gover and Pan (2018) A. Gover and Y. Pan, “Dimension-dependent stimulated radiative interaction of a single electron quantum wavepacket,” Phys. Lett. A 382 (2018).
  * Pan and Gover (2018) Y. Pan and A. Gover, “Spontaneous and stimulated radiative emission of modulated free-electron quantum wavepackets-semiclassical analysis,” J. Phys. Commun. 2, 115026 (2018).
  * Pan and Gover (2019) Y. Pan and A. Gover, “Spontaneous and stimulated emissions of a preformed quantum free-electron wave function,” Phys. Rev. A 99, 052107 (2019).
  * Gover  _et al._ (2019) A. Gover, R. Ianconescu, A. Friedman, C. Emma, N. Sudar, P. Musumeci, and C. Pellegrini, “Superradiant and stimulated-superradiant emission of bunched electron beams,” Rev. Mod. Phys. 91, 035003 (2019).
  * Faresab and Yamada (2015) H. Faresab and M. Yamada, “A quantum mechanical analysis of smith?purcell free-electron lasers,” Nucl. Instrum. Methods. Phys. Res. B . 785, 143–152 (2015).
  * Talebi (2016) N. Talebi, “Schrödinger electrons interacting with optical gratings: quantum mechanical study of the inverse smith?purcell effect,” New J. Phys. 18, 123006 (2016).
  * Kfir  _et al._ (2021) O. Kfir, V. Di Giulio, F. J. García de Abajo, and C. Ropers, “Optical coherence transfer mediated by free electrons,” Sci. Adv. 7, 18 (2021).
  * Wong  _et al._ (2021) L. J. Wong, N. Rivera, C. Murdia, T. Christensen, J. D. Joannopoulos, M. Soljačić, and I. Kaminer, “Control of quantum electrodynamical processes by shaping electron wavepackets,” Nat. Commun. 12, 1700 (2021).
  * Giovannetti  _et al._ (2011) V. Giovannetti, S. Lloyd, and L. Maccone, “Advances in quantum metrology,” Nat. Photonics 5, 222–229 (2011).
  * Glickmanshlomi  _et al._ (2013) Y. Glickmanshlomi, K. Akermanand, and R. Ozeri, “Emergence of a measurement basis in atom-photon scattering,” Science 339, 1187–1191 (2013).
  * Matthews  _et al._ (2016) J. Matthews, XQ. Zhou, H. Cable, P. J. Shadbolt, D. J. Saunders, G. A. Durkin, G. J. Pryde, and J. L. O?Brien, “Towards practical quantum metrology with photon counting,” npj Quantum Inf. 2, 16023 (2016).
  * Sun and Chen (2018) Y. Sun and P.-X. Chen, “Analysis of atom-photon quantum interface with intracavity Rydberg-blocked atomic ensemble via two-photon transition,” Optica 5, 1492–1501 (2018).
  * Sánchez Muñoz  _et al._ (2021) C. Sánchez Muñoz, G. Frascella, and F. Schlawin, “Quantum metrology of two-photon absorption,” Phys. Rev. Research 3, 033250 (2021).
  * Cirac  _et al._ (1997) J. I. Cirac, P. Zoller, H. J. Kimble, and H. Mabuchi, “Quantum state transfer and entanglement distribution among distant nodes in a quantum network,” Phys. Rev. Lett. 78, 3221–3224 (1997).
  * Boozer  _et al._ (2007) A. D. Boozer, A. Boca, R. Miller, T. E. Northup, and H. J. Kimble, “Reversible state transfer between light and a single trapped atom,” Phys. Rev. Lett. 98, 193601 (2007).
  * Kimble (2008) H. Kimble, “The quantum internet,” Nature 453, 1023?1030 (2008).
  * Ritter  _et al._ (2012) S. Ritter, C. Nölleke, C. Hahn, A. Reiserer, A. Neuzner, M. Uphoff, M. Mücke, E. Figueroa, J. Bochmann, and G. Rempe, “An elementary quantum network of single atoms in optical cavities,” Nature 484, 195?200 (2012).
  * Hofmannmichael  _et al._ (2012) J. Hofmannmichael, M. Krug, N. Ortegel, L. Gérard, M. Weber, W. Rosenfeld, and H. Weinfurter, “Heralded entanglement between widely separated atoms,” Science 337, 72–75 (2012).
  * Stute  _et al._ (2013) A. Stute, B. Casabone, B. Brandstätter, K. Friebe, T. E. Northup, and R. Blatt, “Quantum-state transfer from an ion to a photon,” Nat. Photonics 7, 219?222 (2013).
  * Northup and Blatt (2014) T. Northup and R. Blatt, “Quantum information transfer using photons,” Nat. Photonics 8, 356?363 (2014).
  * Heshami  _et al._ (2016) K. Heshami, D. G. England, P. C. Humphreys, P. J. Bustarda, V. M. Acostac, J. Nunnb, and B. J. Sussman, “Quantum memories: emerging applications and recent advances,” J. Mod. Opt. 63, 2005 (2016).
  * Bechler  _et al._ (2018) O. Bechler, A. Borne, S. Rosenblum, _et al._ , “A passive photon-atom qubit swap operation,” Nat. Phys. 14, 996?1000 (2018).
  * Carolan  _et al._ (2015) J. Carolan, C. Harrold, C. Sparro, _et al._ , “Universal linear optics,” Science 349, 711–716 (2015).
  * Kim  _et al._ (2016) J.-H. Kim, C. J. K. Richardson, R. P. Leavitt, and E. Waks, “Two-photon interference from the far-field emission of chip-integrated cavity-coupled emitters,” Nano Lett. 16, 7061?7066 (2016).
  * Harris  _et al._ (2017) N. Harris, G. Steinbrecher, M. Prabhu, _et al._ , “Quantum transport simulations in a programmable nanophotonic processor,” Nat. Photonics 11, 447?452 (2017).
  * Uppu  _et al._ (2020) R. Uppu, F. T. Pedersen, Y. Wang, _et al._ , “Scalable integrated single-photon source,” Sci. Adv. 6 (2020), https://doi.org/10.1126/sciadv.abc8268.
  * Uppu  _et al._ (2021) R. Uppu, L. Midolo, X. Zhou, J. Carolan, and P. Lodahl, “Quantum-dot-based deterministic photon?emitter interfaces for scalable photonic quantum technology,” Nat. Nanotechnol. (2021), https://doi.org/10.1038/s41565-021-00965-6.
  * Michler  _et al._ (2000) P. Michler, A. Kiraz, C. Becher, W. V. Schoenfeld, P. M. Petroff, L. Zhang, E. Hu, and A. Imamoglu, “A quantum dot single-photon turnstile device,” Science 290, 2282?2285 (2000).
  * Faraon  _et al._ (2008) A. Faraon, I. Fushman, D. Englund, N. Stoltz, P. Petroff, and J. Vučković, “Coherent generation of non-classical light on a chip via photon-induced tunnelling and blockade,” Nat. Phys. 4, 859?863 (2008).
  * Wu  _et al._ (2009) J.-H. Wu, M. Artoni, and G. C. La Rocca, “All-optical light confinement in dynamic cavities in cold atoms,” Phys. Rev. Lett. 103, 133601 (2009).
  * Reinhard  _et al._ (2012) A. Reinhard, T. Volz, M. Winger, A. Badolato, K. J. Hennessy, E. L. Hu, and A. Imamoǧlu, “Strongly correlated photons on a chip,” Nat. Photonics 6, 93?96 (2012).
  * Chu  _et al._ (2017) XL. Chu, S. Götzinger, and V. Sandoghdar, “A single molecule as a high-fidelity photon gun for producing intensity-squeezed light,” Nat. Photonics 11, 58?62 (2017).
  * Prasad  _et al._ (2020) A. S. Prasad, J. Hinney, S. Mahmoodian, K. Hammerer, S. Rind, P. Schneeweiss, A. S. Sørensen, J. Volz, and A. Rauschenbeutel, “Correlating photons using the collective nonlinear response of atoms weakly coupled to an optical mode.” Nat. Photonics 14, 719?722 (2020).
  * Stiesdal  _et al._ (2021) N. Stiesdal, H. Busche, K. Kleinbeck, J. Kumlin, M. G. Hansen, H. P. Büchler, and S. Hofferberth, “Controlled multi-photon subtraction with cascaded Rydberg superatoms as single-photon absorbers.” Nat. Commun. 12, 4328 (2021).
  * Pursley  _et al._ (2018) B. C. Pursley, S. G. Carter, M. K. Yakes, A. S. Bracker, and D. Gammon, “Picosecond pulse shaping of single photons using quantum dots.” Nat. Commun. 9, 115 (2018).
  * Morin  _et al._ (2019) O. Morin, M. Körber, S. Langenfeld, and G. Rempe, “Deterministic shaping and reshaping of single-photon temporal wave functions,” Phys. Rev. Lett. 123, 133602 (2019).
  * Lib  _et al._ (2020) O. Lib, G. Hassonand, and Y. Bromberg, “Real-time shaping of entangled photons by classical control and feedback,” Sci. Adv. 6, eabb6298 (2020).
  * Uria  _et al._ (2020) M. Uria, P. Solano, and C. Hermann-Avigliano, “Deterministic generation of large fock states,” Phys. Rev. Lett. 125, 093603 (2020).
  * Kondakci and Abouraddy (2017) H. E. Kondakci and A. F. Abouraddy, “Diffraction-free space?time light sheets,” Nat. Photonics 11, 733–740 (2017).
  * Bhaduri  _et al._ (2020) B. Bhaduri, M. Yessenov, and A. F. Abouraddy, “Anomalous refraction of optical spacetime wave packets,” Nat. Photonics 14, 416–421 (2020).
  * Wong  _et al._ (2020) L. J. Wong, D. N. Christodoulides, and I. Kaminer, “The complex charge paradigm: A new approach for designing electromagnetic wavepackets,” Adv. Sci. 7, 1903377 (2020).
  * Xu  _et al._ (2014) X. G. Xu, B. G. Ghamsari, J.-H. Jiang, L. G. Gregory, O. Andreev, C. Zhi, _et al._ , “One-dimensional surface phonon polaritons in boron nitride nanotubes,” Nat. Commun. 5, 4782 (2014).
  * Machado  _et al._ (2018) F. Machado, N. Rivera, H. Buljan, M. Soljačić, and I. Kaminer, “Shaping polaritons to reshape selection rules,” ACS Photonics 5, 3064?3072 (2018).
  * Estrecho (2021) E. Estrecho, “Laser trapping and manipulation of exciton?polariton quantum fluids,” Nat. Rev. Phys. 3, 536 (2021).
  * Larocque  _et al._ (2018) H. Larocque, I. Kaminer, and V. Grillo, “Twisting neutrons may reveal their internal structure,” Nature Phys. 14, 1–2 (2018).
  * Afanasev  _et al._ (2019) Andrei V. Afanasev, D. V. Karlovets, and V. G. Serbo, “Schwinger scattering of twisted neutrons by nuclei,” Phys. Rev. C 100, 051601 (2019).
  * Afanasev  _et al._ (2021) A. V. Afanasev, D. V. Karlovets, and V. G. Serbo, “Elastic scattering of twisted neutrons by nuclei,” Phys. Rev. C 103, 054612 (2021).
  * Clark  _et al._ (2015) C. Clark, R. Barankov, M. G. Huber, M. Arif, D. G. Cory, and D. A. Pushin, “Controlling neutron orbital angular momentum,” Nature 525, 504–506 (2015).
  * Nsofini  _et al._ (2016) J. Nsofini, D. Sarenac, C. J. Wood, D. G. Cory, M. Arif, C. W. Clark, _et al._ , “Spin-orbit states of neutron wave packets,” Phys. Rev. A 94, 013605 (2016).
  * Pushin  _et al._ (2017) D. A. Pushin, D. Sarenac, D. S. Hussey, H. Miao, M. Arif, D. G. Cory, M. G. Huber, D. L. Jacobson, J. M. LaManna, J. D. Parker, T. Shinohara, W. Ueno, and H. Wen, “Far-field interference of a neutron white beam and the applications to noninvasive phase-contrast imaging,” Phys. Rev. A 95, 043637 (2017).
  * Cappelletti  _et al._ (2018) R. L. Cappelletti, T. Jach, and J. Vinson, “Intrinsic orbital angular momentum states of neutrons,” Phys. Rev. Lett. 120, 090402 (2018).
  * Geerits and Sponar (2021) Niels Geerits and Stephan Sponar, “Twisting neutral particles with electric fields,” Phys. Rev. A 103, 022205 (2021).
  * Harris  _et al._ (1990) S. E. Harris, J. E. Field, and A. Imamoğlu, “Nonlinear optical processes using electromagnetically induced transparency,” Phys. Rev. Lett. 64, 1107–1110 (1990).
  * Boller  _et al._ (1991) K.-J. Boller, A. Imamoğlu, and S. E. Harris, “Observation of electromagnetically induced transparency,” Phys. Rev. Lett. 66, 2593–2596 (1991).
  * Schwinger (1948) J. Schwinger, “On the polarization of fast neutrons,” Phys. Rev. 73, 407–409 (1948).
  * Pálffy  _et al._ (2007) A. Pálffy, Z. Harman, and W. Scheid, “Quantum interference between nuclear excitation by electron capture and radiative recombination,” Phys. Rev. A 75, 012709 (2007).
  * Craddock  _et al._ (2019) A. N. Craddock, J. Hannegan, D. P. Ornelas-Huerta, J. D. Siverns, A. J. Hachtel, E. A. Goldschmidt, _et al._ , “Quantum interference between photons from an atomic ensemble and a remote atomic ion,” Phys. Rev. Lett. 123, 213601 (2019).
  * Gover and Yariv (2020) A. Gover and A. Yariv, “Free-electron-bound-electron resonant interaction,” Phys. Rev. Lett. 124, 064801 (2020).
  * Ruimy  _et al._ (2021) R. Ruimy, A. Gorlach, C. Mechel, N. Rivera, and I. Kaminer, “Toward atomic-resolution quantum measurements with coherently shaped free electrons,” Phys. Rev. Lett. 126, 233403 (2021).
  * Zhao  _et al._ (2021b) Z. Zhao, X.-Q. Sun, and S. Fan, “Quantum entanglement and modulation enhancement of free-electron?bound-electron interaction,” Phys. Rev. Lett. 126, 233402 (2021b).
  * Zhang  _et al._ (2021a) B. Zhang, D. Ran, R. Ianconescu, A. Friedman, J. Scheuer, A. Yariv, and A. Gover, “Quantum wave-particle duality in free-electron-bound-electron interaction,” Phys. Rev. Lett. 126, 244801 (2021a).
  * Zhang  _et al._ (2021b) B. Zhang, D. Ran, R. Ianconescu1, A. Friedman, J. Scheuer, A. Yariv, and A. Gover, “Quantum states interrogation using a pre-shaped free electron wavefunction,” (2021b), arXiv:2111.13130 [quant-ph] .
  * Rätzel  _et al._ (2021) D. Rätzel, D. Hartley, O. Schwartz, and P. Haslinger, “Controlling quantum systems with modulated electron beams,” Phys. Rev. Research 3, 023247 (2021).
  * Trusheim  _et al._ (2020) M. E. Trusheim, B. Pingault, N. H. Wan, _et al._ , “Transform-limited photons from a coherent tin-vacancy spin in diamond,” Phys. Rev. Lett. 124, 023602 (2020).
  * Vahala (2003) K. Vahala, “Optical microcavities,” Nature 424, 839?846 (2003).
  * Vaidya  _et al._ (2018) V. D. Vaidya, Y. Guo, R. M. Kroeze, K. E. Ballantine, A. J. Kollár, J. Keeling, and B. L. Lev, “Tunable-range, photon-mediated atomic interactions in multimode cavity qed,” Phys. Rev. X 8, 011002 (2018).
  * Davis  _et al._ (2019) E. J. Davis, G. Bentsen, L. Homeier, T. Li, and M. H. Schleier-Smith, “Photon-mediated spin-exchange dynamics of spin-1 atoms,” Phys. Rev. Lett. 122, 010405 (2019).
  * Muniz  _et al._ (2020) J. A. Muniz, D. Barberena, R. J. Lewis-Swan, D. J. Young, J. R. K. Cline, A. M. Rey, and J. K. Thompson, “Exploring dynamical phase transitions with cold atoms in an optical cavity,” Nature 580, 602?607 (2020).
  * Todorov  _et al._ (2009) Y. Todorov, A. M. Andrews, I. Sagnes, R. Colombelli, P. Klang, G. Strasser, and C. Sirtori, “Strong light-matter coupling in subwavelength metal-dielectric microcavities at terahertz frequencies,” Phys. Rev. Lett. 102, 186402 (2009).
  * Derntl  _et al._ (2017) C. G. Derntl, D. Bachmann, K. Unterrainer, and J. Darmo, “Disk patch resonators for cavity quantum electrodynamics at the terahertz frequency,” Opt. Express 25, 12311–12324 (2017).
  * Lu  _et al._ (2018) Q. Lu, X. Chen, C.-L. Zou, and S. Xie, “Extreme terahertz electric-field enhancement in high-q photonic crystal slab cavity with nanoholes,” Opt. Express 26, 30851 (2018).
  * Liu  _et al._ (2020) A. Liu, L. Wang, M. Hua, X. Liu, F. Qian, G. Xie, Y. Ning, Y. Shi, X. Wang, and F. Yang, “High-q metamaterials based on cavity mode resonance for thz sensing applications,” AIP Adv. 10, 075014 (2020).
  * Messelot  _et al._ (2020) S. Messelot, C. Symonds, J. Bellessa, J. Tignon, S. Dhillon, J.-B. Brubach, P. Roy, and J. Mangeney, “Tamm cavity in the terahertz spectral range,” ACS Photonics 7, 2906?2914 (2020).
  * Kjaergaard  _et al._ (2020) M. Kjaergaard _et al._ , “Superconducting qubits: Current state of play,” Annu. Rev. Condens. Matter Phys. 11, 369–395 (2020).
  * Patel  _et al._ (2010) R. Patel, A. J. Bennett, I. Farrer, C. A. Nicoll, D. A. Ritchie, and A. J. Shields, “Two-photon interference of the emission from electrically tunable remote quantum dots,” Nat. Photonics 4, 632?635 (2010).
  * Flagg  _et al._ (2010) E. B. Flagg, A. Muller, S. V. Polyakov, A. Ling, A. Migdall, and G. S. Solomon, “Interference of single photons from two separate semiconductor quantum dots,” Phys. Rev. Lett. 104, 137401 (2010).
  * Loredo  _et al._ (2016) J. C. Loredo, N. A. Zakaria, N. Somaschi, _et al._ , “Scalable performance in solid-state single-photon sources,” Optica 3, 433?440 (2016).
  * Somaschi  _et al._ (2016) N. Somaschi, V. Giesz, L. De Santis, _et al._ , “Near-optimal single-photon sources in the solid state,” Nat. Photonics 10, 340?345 (2016).
  * Ding  _et al._ (2016) X. Ding, Y. He, Z.-C. Duan, _et al._ , “On-demand single photons with high extraction efficiency and near-unity indistinguishability from a resonantly driven quantum dot in a micropillar,” Phys. Rev. Lett. 116, 020401 (2016).
  * Wei  _et al._ (2014) Y. J. Wei, Y.-M. He, M.-C. Chen, _et al._ , “Deterministic and robust generation of single photons from a single quantum dot with 99.5% indistinguishability using adiabatic rapid passage,” Nano Lett. 14, 6515?6519 (2014).
  * Aharonovich  _et al._ (2016) I. Aharonovich, D. Englund, and M. Toth, “Solid-state single-photon emitters.” Nat. Photonics 10, 631?641 (2016).
  * Gao  _et al._ (2015) W. Gao, A. Imamoglu, H. Bernien, and R. Hanson, “Coherent manipulation, measurement and entanglement of individual solid-state spins using optical fields,” Nat. Photonics 9, 363–373 (2015).
  * Veldhorst  _et al._ (2015) M. Veldhorst, C. H. Yang, and J. C. C. Hwang, “A two-qubit logic gate in silicon,” Nature 526, 410?414 (2015).
  * Sponar  _et al._ (2021) S. Sponar, R. I. P. Sedmik, M. Pitschmann, H. Abele, and Y. Hasegawa, “Tests of fundamental quantum mechanics and dark interactions with low-energy neutrons,” Nat. Rev. Phys. 3, 309–327 (2021).
  * Pan  _et al._ (2019) Yiming Pan, Bin Zhang, and Avraham Gover, “Anomalous photon-induced near-field electron microscopy,” Phys. Rev. Lett. 122, 183204 (2019).


# EventF2S: Asynchronous and Sparse Spiking AER Framework using Neuromorphic-
Friendly Algorithm

Lakshmi Annamalai, and Chetan Singh Thakur

###### Abstract

Bio-inspired Address Event Representation (AER) sensors have attracted
significant popularity owing to their low power consumption, high sparsity,
and high temporal resolution. Spiking Neural Network (SNN) has become the
inherent choice for AER data processing. However, the integration of the AER-
SNN paradigm has not adequately explored asynchronous processing, neuromorphic
compatibility, and sparse spiking, which are the key requirements of resource-
constrained applications. To address this gap, we introduce a brain-inspired
AER-SNN object recognition solution, which includes a data encoder integrated
with a First-To-Spike recognition network. Being fascinated by the
functionality of neurons in the visual cortex, we designed the solution to be
asynchronous and compatible with neuromorphic hardware. Furthermore, we have
adapted the principle of denoising and First-To-Spike coding to achieve
optimal spike signaling, significantly reducing computation costs.
Experimental evaluation has demonstrated that the proposed method incurs
significantly less computation cost to achieve state-of-the-art competitive
accuracy. Overall, the proposed solution offers an asynchronous and cost-
effective AER recognition system that harnesses the full potential of AER
sensors.

##  I Introduction

Address Event Representation (AER) sensors [5] [6] [7] [16] [17] [18] [19],
which aims to mimic biological eyes, has gained a lot of attention from
different communities owing to its unique features. The inherent working
principle of AER sensors results in the benefit of data sparsity, thus
providing huge energy-saving. In contrast to a conventional camera, each pixel
($u,v$) of an event camera individually monitors the dynamics of the scene and
records in the form of spiking asynchronous events as and when the change
exceeds a pre-defined threshold. Events at pixel $(u,v)$ are generated as
$4$-tuple $(t,u,v,p)$, where $t$ and $p$ represent the events’ time and signed
polarity of occurrence.

The nature of AER data poses severe challenges in processing event data
efficiently [20] [21] [22] [23] [24]. The asynchronous event-driven paradigm
of the AER sensor makes it a natural choice for Spiking Neural Networks (SNN)
[11] [12] [13] [14] [15]. SNN can be broadly classified into two approaches:
Rate-coded SNN [8] [9] [10] and temporal-encoded SNN [29] [30] [31] [32]. The
fundamental problem of rate-coded SNN is that it results in tremendous spike
signaling cost and latency. Unlike the above technique, temporal-encoded SNN
in the AER domain involves hand-crafted spatiotemporal feature extraction
followed by a spiking recognition layer. The feature extraction phase of these
networks makes them challenging to implement in the temporal spiking domain.
Recently a learnable temporal-coded feature extraction has been proposed in
[40]. However, the major limitation of this method comes in the event encoding
process, which incurs a memory cost of $O(N^{2})$, thus making its
implementation non-trivial.

Motivated by the fact that neurons in the visual cortex respond in
asynchronous mode with minimal energy cost, we asked ourselves whether a
similar hypothesis could be achieved in AER recognition systems. Towards this,
we introduce a novel single-spike object recognition system which we refer to
here as EventF2S (Event Camera First-To-Spike Spiking Network).

EventF2S consists of two main components: 1) Sparse Spiking Temporal Encoding
(SS-TE) layer: This layer provides a neuromorphic implementable algorithm to
encode raw events into as few informative early spikes as possible. The SS-TE
encoding scheme delivers a drastically reduced number of spikes to
computationally heavy downstream applications. 2) Spiking Recognition Network:
It consists of a group of differentiable spiking neurons that adapts the
First-To-Spike protocol for inter-neuron communication. The learnable spiking
neural dynamics enable us to backpropagate the error deep down the network and
achieve competitive accuracy with single-spike input encoding. By
incorporating the First-To-Spike communication protocol, we aim to achieve
further reduction in spike signaling.

To summarize, the significant contributions of EventF2S are as follows: i)
Asynchronous processing: EventF2S exhibits asynchronous processing at the
input and output layers. The system processes the events asynchronously as
they arrive at the input layer. At the output layer, the winner neuron fires
asynchronously once it has received sufficient spikes to activate it. Note
that there is no pre-defined accumulation time; ii) Sparse spiking: EventF2S
curtails the number of spikes to one per pixel and one per neuron at the
encoding, feature extraction, and classification layers; iii) Neuromorphic
Friendly: By designing the solution to be end-to-end neuromorphic hardware
implementable, we enable EventF2S to inherit brain’s efficiency and
computational mechanisms.

##  II Related Work

As the proposed framework is a temporal spiking neural network, we have not
ventured into the survey of rate-coded spiking networks. Instead, we provide a
survey specifically in the domain of temporal coding SNN, which falls under
the following two categories: cortical mechanism-based [29] [30] [31] [32]
[35] [36] [37] and statistical learning-based [38] [26] [27].

HMAX [29] is one of the most popular hierarchical models structured as simple
(linear) and complex (non-linear) layers S1, C1, S2, and C2 with SVM as a
classifier. Algorithm of [30] extracted position and size invariant features
for human posture recognition. Zhao. et al. [31] proposed an event-driven
neuron model in the S1 layer with a forgetting mechanism and replacing max-
pooling with MAX operation at the C1 layer. S2 and C2 layers were discarded.
The generated features converted to single-spike encoding have been classified
using a Tempotron-based classification layer. Orchard. et al. [32] came up
with an HMAX-based hierarchical spiking architecture. The winner-Take-All
strategy was implemented in the C1 layer to realize Max operation. However,
the learning layers used statistical methods, thus losing the spiking nature.

Liu. et al. proposed [35] proposed MuST (Multi-Scale spatiotemporal) feature
representation based on Leaky Integrate and Fire (LIF) neurons followed by an
unsupervised recognition approach. MuST features are converted into temporal
coded spikes using a natural logarithm temporal coding scheme. Patterns are
learned from these spatiotemporal spikes in an unsupervised fashion with Spike
Timing Dependent Plasticity (STDP). The architecture of [36] consists of two
modules: Hand-crafted multi-spike encoding followed by tempotron-based SNN
learning. Though this method was fully temporally encoded, multiple spikes
were required to achieve good accuracy as learning is implemented only in the
classification stage.

[37] includes spatiotemporal feature extraction and a novel learning
algorithm. The feature extraction phase is similar to that of [36]. As part of
spike learning, they have introduced a novel learning algorithm known as the
Segmented Probability Maximization (SPA) algorithm, which classifies based on
the firing rate of neurons in the classification layer. Though [37] was able
to achieve better accuracy than [36], the rate coding of the decision layer
discredits its use in the temporal domain.

Recently, [40] ventured into a temporal-encoded spiking object recognition
system, which has three modules: pre-processing of events, temporal coding of
events, and object recognition spiking network. The spike encoding involves
the accumulation of event stream into an image frame, $D(x,y)$, over a period
of time from $t_{0}$ to $t_{k}$, which incurs a memory cost of $O(N^{2})$. As
and when the value at pixel $(x,y)$ of $D(x,y)$ exceeds a pre-defined
threshold, the particular input neuron spikes. This will incur additional
memory costs, thus rendering it not suitable for neuromorphic implementation.

##  III Sparse Temporal-Encoded EventF2S Architecture

The proposed spiking architecture is structured as follows: Single Spike
Temporal Encoding (SS-TE) layer and a learnable First-To-Spike encoded object
recognition spiking network. The SS-TE layer serves as the input layer of the
spiking network. The recognition network has feature extraction convolution
layers followed by a classification layer.

###  III-A Single Spike Temporal Encoding (SS-TE) Layer

SS-TE layer densely covers the pixels of the AER sensor, performing signal-
selective single spike per pixel encoding. The new sensing modality of the AER
sensor has made it more susceptible to noise. This results in an abundant
number of events, potentially overloading the computation of downstream
applications. The noisy events of the AER sensor broadly fall under two
categories: Type I noise, which is uncorrelated with spatiotemporal neighbor
events, and Type II noise, which exhibits self-temporal correlation.

To address this issue, we implement event encoding as a collection of parallel
asynchronous continuous-time LIF (Leaky Integrate-and-Fire) filters whose
spatial sensitiveness is defined to respond best to spatiotemporally
correlated events while penalizing the Type II noise. The degree of
penalization is inversely proportional to the frequency.

####  III-A1 SS-TE Encoding Function

The function of SS-TE encoding can be derived as furnished here. Events are
modeled as a summation of impulse signals in continuous time (Eq. 1),

$E(t)=\sum_{k=1}^{\inf}p_{k}\delta(t-t_{k})\delta_{x_{k}}(x)$ (1)

Where $\delta(t)$ is the Dirac delta function, $p=\pm 1$ is the polarity
depending on the scene brightness change, $x=(u,v)$ is the discrete pixel
address, $t_{k}$ is the timestamp of occurrence of the $k^{th}$ event,
relative to the global clock and $\delta_{x_{k}}(x)$ is the Kronecker delta
function which is $=1$ at $x=x_{k}$.

Considering the events $E_{x_{i}^{-},\vartriangle\tau}(t)$ that occurred on a
local neighborhood
$x_{i}^{-}=\\{(u,v):|u-u_{i}|<\beta,|v-v_{i}|<\beta,x_{i}\nexists{x}$ centered
at $i^{th}$ pixel and activated at $[t_{i},t_{i}-\Delta\tau]$, it can be
represented as a monodimensional signal in continuous time as follows,

$E_{x_{i}^{-},\vartriangle\tau_{i}}(t)=\int_{t_{i}-\Delta\tau}^{t_{i}}\delta(t-\tau)\delta_{x_{i}^{-}}(x)d\tau$
(2)

The events $E_{x_{i},\vartriangle\tau}(t)$ that occured at the $i^{th}$ pixel
in the time duration $[t_{i},t_{i}-\Delta\tau]$ can be expressed as

$E_{x_{i},\vartriangle\tau}(t)=\int_{t_{i}-\Delta\tau}^{t_{i}}\delta(t-\tau)\delta_{x_{i}}(x)d\tau$
(3)

Define the following continuous time single dimensional low pass filter
$h(t)=e^{-\frac{t}{\tau_{c}}}$, with cut-off frequency determined by
$\tau_{c}$. In order to remove the Type I and II noise, we need to maximize
$h(t)\otimes{E_{x_{i}^{-},\vartriangle\tau}}(t)$ and minimize
$h(t)\otimes{E_{x_{i},\vartriangle\tau}}(t)$ respectively. Summarizing, we
need to estimate events such that the following objective function is
maximized,

$h(t)\otimes{E_{x_{i}^{-},\vartriangle\tau}}(t)-h(t)\otimes{E_{x_{i},\vartriangle\tau}}(t)$
(4)

The given objective function can be realized with a set of parallel LIF
filters whose output is modeled as a convolution between asynchronously
sampled continuous time filter $h(t)$ and the events
$\left[E_{x_{i}^{-},\vartriangle\tau}(t),E_{x_{i},\vartriangle\tau}(t)\right]$
weighted with $\left[+1,-1\right]$. The signal spikes are estimated as a
subset of AER spikes emitted by LIF neurons whose corresponding membrane
voltage crosses a threshold. After firing, the particular neuron shunts all
its incoming spikes till a classification decision is reached by the
recognition spiking network. Thus, SS-TE encoding hugely reduces the number of
spikes presented to the recognition spiking network.

###  III-B Learning by First-To-Spike Spiking Network

In the learning stage, a single spike per pixel emitted by the LIF neurons of
the SS-TE layer is transmitted to the convolution-based feature extraction
layers arranged in a feed-forward fashion. The learned multi-channel feature
spikes are fed to the classification layer, which consists of $C$ (number of
classes) neurons. Once a neuron spikes, temporal inhibition is performed,
which will force the active neuron to enter into a dormant stage. As a result,
this promotes sparsity in spike representation.

####  III-B1 Neuron Dynamics

This section describes the neural and synaptic dynamics of the neurons which
form the building block of our spiking recognition network. Inspired by [25],
we define our recognition network with a set of non-leaky IF neurons with
synaptic kernel function $\Theta(t-t_{i})e^{-\frac{(t-t_{i})}{\tau_{m}}}$,
where $\Theta(t-t_{i})=1$ if $t\geq{t_{i}}$ and $0$ otherwise.

As given in [25], the relation between input and output spike time when the
neuron spikes are given as follows,

$e^{t_{out}}=\frac{\sum_{i\in{C}}w_{i}e^{t_{i}}}{\sum_{i\in{C}}w_{i}-1}$ (5)

Where $C$ is the number of causal spikes which causes the neuron to spike and
$w_{i}$ is the weight of the $i^{th}$ spike. The relation between the neuron’s
first spike time and input spike time given by Eq. 5 is differentiable with
respect to the weights, hence making the network trainable. As per Eq. 5,
$\sum_{i}{w_{i}}$ should be greater than one for a neuron to spike. To ensure
a sufficient number of neuron spikes, the following constraint has been
imposed on weights as given in [25].

####  III-B2 Loss Function

The network is trained such that the output neuron belonging to the correct
class fires early. To achieve the same, the following differentiable cross-
entropy loss function is defined, which is minimized using the standard
Gradient Descent technique.

$-\sum_{k=1}^{C}ln\frac{e^{-z_{k}}}{\sum_{i}{e^{-z_{i}}}}$ (6)

Where $z_{k}$ is the output of $k_{th}$ output neuron, the negative of the
output is used in Eq. 6 in order to make the correct class neuron fire
earlier.

##  IV Experiments and Results

In this section, we conduct two types of experiments: 1) To showcase the
capability of the SS-TE encoding layer and 2) To bring out the recognition
capability of the overall EventF2S framework. The neuromorphic datasets
considered are [2], [1], N-MNIST [33] and MNIST-DVS [28].

###  IV-A SS-TE Experiments

This section highlights the benefits reaped by prefixing the recognition
network with the SS-TE encoding layer.

####  IV-A1 Qualitative Analysis

This section provides qualitative evidence to showcase the effectiveness of
the SS-TE encoding layer in extracting sparse and informative spatiotemporal
events from raw events. Fig. 1 shows the 3D plot of the raw input event stream
along with its SS-TE encoded output event stream. The plot demonstrates that
despite the substantial decrease in the number of output events, the pattern
of the digit $5$ is clearly visible in the SS-TE encoded output. The result
highlights the capability of SS-TE to retain relevant information while
discarding less informative events.

Figure 1: Left: Raw event data of digit $5$ drawn from neuromorphic NMNIST
dataset. Right: SS-TE encoding. SS-TE output being visually recognizable
reinstates that SS-TE retains necessary information with just a single spike
per pixel.

SS-TE |  Density [3] |  BAF [4]
---|---|---
TP | FP | TP | FP | TP | FP
0.98 | 0.37 | 0.99 | 0.99 | 0.99 | 0.99
0.95 | 0.32 | 0.89 | 0.85 | 0.99 | 0.99
0.74 | 0.23 | 0.74 | 0.71 | 0.99 | 0.99
0.39 | 0.14 | 0.60 | 0.55 | 0.99 | 0.99
0.25 | 0.09 | 0.53 | 0.43 | 0.99 | 0.99

SS-TE |  Density [3] |  BAF [4]
---|---|---
TP | FP | TP | FP | TP | FP
0.99 | 0.48 | 0.99 | 0.99 | 1.119 | 0.844
0.92 | 0.40 | 0.87 | 0.85 | 1.119 | 0.844
0.69 | 0.28 | 0.74 | 0.71 | 1.119 | 0.844
0.27 | 0.15 | 0.61 | 0.85 | 1.119 | 0.844
0.23 | 0.12 | 0.55 | 0.42 | 1.119 | 0.844

SS-TE |  Density [3] |  BAF [4]
---|---|---
TP | FP | TP | FP | TP | FP
0.99 | 0.54 | 0.99 | 0.99 | 1.119 | 0.844
0.94 | 0.46 | 0.86 | 0.85 | 1.119 | 0.844
0.68 | 0.30 | 0.72 | 0.70 | 1.119 | 0.844
0.27 | 0.15 | 0.60 | 0.54 | 1.119 | 0.844
0.23 | 0.13 | 0.54 | 0.42 | 1.119 | 0.844

TABLE I: Analysis of denoising methods in terms of True Positive (TP) and
False Positive (FP). Left: $0.9$ SNR , Middle: $0.97$ SNR, Right: $0.98$ SNR .
SS-TE encoding passes most of the signals (high TP) while eliminating all
noisy events (low FP)

####  IV-A2 Denoising Experiments

Though the primary focus of our research is the object recognition spiking
task, we include the performance of our SS-TE encoding layer on the denoising
task as a means to validate that the proposed solution retains only the
informative single spike for each pixel. Denoising was evaluated on a publicly
available noise dataset, recorded by a Davis346 camera under bright and dark
conditions [1]. To evaluate the denoising performance, the noise events were
mixed with pure event signals. A conventional camera video from [1] was
converted to pure events using the v2e algorithm [2].

Table. I shows the comparison of the proposed encoding with state-of-the-art
event denoising methods [3] [4] in terms of True Positive (TP) and False
Positive (FP) rates at different SNR. The results demonstrate the
effectiveness of SS-TE in achieving a high True Positive (TP) rate for a low
False Positive (FP) rate across various Signal-to-Noise ratios (SNR). SNR is
adjusted by varying the number of noisy events added.

####  IV-A3 Sparse Temporal Spiking

The spikes generated and transmitted in a spiking network should be as low as
possible to reap the full potential of the AER sensor. The average number of
spikes per pixel ($N_{s}$) generated by the AER sensor, calculated across test
samples, amounts to $49$ and $50$ for NMNIST and MNIST-DVS, respectively. SS-
TE encoding layer reduces the spike count to just one per pixel as against
$49$ and $50$.

###  IV-B Recognition Experiments

For evaluation of our primary object recognition task, the network topology
used is as follows: Two convolution layers of $32$ and $16$ channels, each
with a kernel size of $5\times{5}$ and stride of $2$, followed by a classifier
layer of $10$ neurons. $10$ neurons represent the number of classes. During
the training process, the learning rate was set to $1e^{-2}$ for the initial
$50$ epochs, and later it was reduced to $1e^{-3}$ for the remaining $50$
epochs.

####  IV-B1 comparison in terms of Recognition Accuracy

We have compared our approach with [30], [31], [32], [33] [34] [35] [38] [39].
We have restricted our comparison to non-rate-coded spiking networks as the
rate-coded networks provide good accuracy at par with ANN at the cost of huge
energy consumptions as compared to temporal-coded networks. Results from the
comparison are summarized in Table. II in terms of accuracy. It could be
observed that EventF2S achieves good accuracy at par or above the state-of-
the-art AER recognition networks.

The performance of EventF2S in terms of accuracy is below [40]. While accuracy
is acknowledged as an important metric, the specific goal of the proposed
EventF2S is to provide a neuromorphic-friendly solution. This allows us to
leverage the benefits of neuromorphic hardware, such as asynchronous
processing and low cost. The encoding process mentioned in [40] incurs
additional memory cost, which is a major drawback for hardware implementation.
Additionally, the processing primitives of the encoding layer of [40] do not
align with the device physics of neuromorphic hardware, further limiting the
application of [40]. Thus, EventF2S is a promising asynchronous approach for
low-cost edge applications.

Method | NM | MD | Method | NM | MD
---|---|---|---|---|---
[30] | - | 0.63 | [31] | 0.85 | 0.768
[32] | 0.711 | 0.55 | [33] | 0.83 | -
[34] | 83.44 | - | [35] | 0.89 | 0.79
[36](ss) | 0.79 | - |  [36](ms) | 0.93 | 0.77
[38] | 0.704 | 0.746 | [39] | 0.80 | 0.80 (full)
EventF2S | 0.94 | 0.84 | EventF2S2s | 0.97 | -
TABLE II: Comparison of recognition accuracy with state-of-the-art methods on
NMNIST (NM) and MNIST-DVS (MD) datasets. Accuracy of other methods are as
reported in [35] [38] [37] [36] (MD: 100ms). The proposed method gives better
accuracy than other non-rate-coded AER spiking networks. EventF2S2s refers to
SS-TE emitting maximum two spikes per pixel. Accuracy increases with the
number of spikes, which can be set as per the application requirement.

####  IV-B2 Computation Efficiency

In this section, we perform experiments to analyze the proposed EventtF2S in
terms of computation cost in comparison with [31], [32], [35] [36] and [37].
Initial feature extraction layers contribute to the bulk of the computational
complexity as they operate on larger spatial resolution. Hence, we evaluated
the computation cost as the number of weight multiplications that takes place
in the first layer of the feature extraction stage.

To estimate computation cost, we multiply the number of events input into the
first layer ($N_{e}$) with the number of computations incurred per event
($N_{c}$). Each event was assumed to be surrounded densely by neighboring
events. The values of $N_{e}$ for the raw event and SS-TE encoding are
$(4203,1156)$ and $(27320,16384)$ for NMNIST (NM) and MNIST-DVS (MD),
respectively. $N_{e}$ is calculated as the mean of the number of input events
that enter the feature extractor across the test samples.

The results are provided in Table. III makes it evident that the computation
incurred at the first layer of EventF2S is far lesser as compared to other
methods. State-of-the-art methods perform feature extraction on every input
event, thus increasing the computational complexity. By making the feature
extraction learnable, we were able to reduce the number of computations
drastically with single-spike encoding.

Methods | NM | MD | Methods | NM | MD
---|---|---|---|---|---
[31], [37] | 2.75 | 17.92 | [32] | 2.47 | 16.06
[36] |  $N_{a}\times$2.47 |  $N_{a}\times$16.06 | Ours | 0.46 | 6.55
TABLE III: Computation Complexity $(\times{10^{6}})$ at first layer: $N_{a}$
is the number of events input to the feature extraction layer of [36]. Note
the huge reduction in computation in EventF2S compared to other methods.

##  V Conclusion

In this paper, we proposed an asynchronous, sparse-spiking, neuromorphic
compatible AER sensor object recognition framework known as EventF2S. The
pipeline of EventF2S comprises an event encoding layer followed by an AER
recognition network. The event encoding layer of EventF2S (SS-TE) was
specifically derived to convert raw and noisy AER sensor data into very few
informative early spikes. Experiments conducted demonstrated the denoising
capability of the SS-TE layer and how it contributed to sparse spiking and
resultant reduced computation. Sparse spiking can lead to information loss,
which has been compensated by utilizing learnable feature extraction spiking
neurons. Experiments on neuromorphic datasets revealed the above-mentioned
fact by exhibiting competitive recognition accuracy at par with state-of-the-
art temporal encoded SNNs. Note being neuromorphic hardware friendly, EventF2S
exploits the unique advantages of AER sensors. To summarize, EventF2S provides
an object recognition solution aligned with the benefits of AER sensors, thus
rendering itself more suitable for resource-constrained applications.

## References

  * [1] Guo, Shasha, and Tobi Delbruck. Low cost and latency event camera background activity denoising. IEEE Transactions on Pattern Analysis and Machine Intelligence.
  * [2] Y. Hu, S.-C. Liu, and T. Delbruck, “v2e: From video frames to realistic DVS events,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, 2021
  * [3] Feng Y and et al. Event density based denoising method for dynamic vision sensor. Applied Sciences,
  * [4] Liu H and et al. Design of a spatiotemporal correlation filter for event-based sensor. IEEE International Symposium on Circuits and Systems, pages 722–725, 2015
  * [5] M. Mahowald, “VLSI analogs of neuronal visual processing: A synthesis of form and function,” Ph.D. dissertation, California Institute of Technology, Pasadena, CA, May 1992
  * [6] G. Gallego, T. Delbruck, G. M. Orchard, et al., “Event-based vision: A survey,” IEEE Trans. Pattern Anal. Mach. Intell., pp. 1–1, 2020
  * [7] P. Lichtsteiner, C. Posch, and T. Delbruck, “A 128x128 120dB 15us latency asynchronous temporal contrast vision sensor,” IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, Feb. 2008
  * [8] Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. In- corporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2661–2671, 2021
  * [9] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 1311–1318, 2019
  * [10] Ali Samadzadeh, Fatemeh Sadat Tabatabaei Far, Ali Javadi, Ahmad Nickabadi, and Morteza Haghir Chehreghani. Convolutional spiking neural networks for spatio-temporal feature extraction. arXiv preprint arXiv:2003.12346, 2020
  * [11] Chankyu Lee, Adarsh Kumar Kosta, Alex Zihao Zhu, Kenneth Chaney, Kostas Daniilidis, and Kaushik Roy. Spike-flownet: event-based optical flow estimation with energy-efficient hybrid neural networks. In European Conference on Computer Vision, pp. 366–382. Springer, 2020
  * [12] Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy. Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. arXiv preprint arXiv:2005.01807, 2020
  * [13] Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-learn in networks of spiking neurons. Advances in neural information processing systems, 31, 2018
  * [14] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers in neuroscience, 12:331, 2018.
  * [15] Dongsung Huh and Terrence J Sejnowski. Gradient descent for spiking neural networks. Advances in neural information processing systems, 31, 2018. Systems.
  * [16] P. Lichtsteiner, C. Posch, and T. Delbruck, A 128×128 120 dB 15$\mu$s latency asynchronous temporal contrast vision sensor, IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566–576, 2008
  * [17] C. Posch, D. Matolin, and R. Wohlgenannt, A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS, IEEE J. Solid-State Circuits, vol. 46, no. 1, pp. 259–275, Jan. 2011
  * [18] P. Lichtsteiner and T. Delbruck, “64x64 event-driven logarithmic temporal derivative silicon retina,” in IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors, 2005, pp. 157–160.
  * [19] P. Lichtsteiner, “An AER temporal contrast vision sensor,” Ph. D. Thesis, ETH Zurich, Dept. of Physics (D-PHYS), Zurich, Switzerland, 2006.
  * [20] M. Cook, L. Gugelmann, F. Jug, C. Krautz, and A. Steger, “Interacting maps for fast visual interpretation,” in Int. Joint Conf. Neural Netw. (IJCNN), 2011, pp. 770–776
  * [21] H. Rebecq, T. Horstschafer, G. Gallego, and D. Scaramuzza, EVO: A geometric approach to event-based 6-DOF parallel tracking and mapping in real-time, IEEE Robot. Autom. Lett., vol. 2, no. 2, pp. 593–600, 2017.
  * [22] M. Liu and T. Delbruck, “Adaptive time-slice block-matching optical flow algorithm for dynamic vision sensors,” in British Mach. Vis. Conf. (BMVC), 2018.
  * [23] J. Kogler, C. Sulzbachner, and W. Kubinger, “Bio-inspired stereo vision system with silicon retina imagers,” in Int. Conf. Comput. Vis. Syst. (ICVS), 2009, pp. 174–183.
  * [24] A. I. Maqueda, A. Loquercio, G. Gallego, N. Garcıa, and D. Scaramuzza, “Event-based vision meets deep learning on steering prediction for self-driving cars,” in IEEE Conf. Comput. Vis. Pattern Recog. (CVPR), 2018
  * [25] Hesham Mostafa, Supervised learning based on temporal coding in spiking neural networks, IEEE Transactions on Neural Network and Learning Systems, 2017.
  * [26] Lagorce, Xavier, et al. ”Hots: a hierarchy of event-based time-surfaces for pattern recognition.” IEEE transactions on pattern analysis and machine intelligence 39.7 (2016): 1346-1359.
  * [27] Sironi, Amos, et al. ”HATS: Histograms of averaged time surfaces for robust event-based object classification.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.
  * [28] Serrano-Gotarredona, Teresa, and Bernabé Linares Barranco. ”Poker-DVS and MNIST-DVS. Their history, how they were made, and other details.” Frontiers in neuroscience 9, 481, 2015.
  * [29] T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, and T. Poggio, “Robust object recognition with cortex-like mechanisms,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 29, no. 3, pp. 411–426, 2007
  * [30] S. Chen, P. Akselrod, B. Zhao, J. A. P. Carrasco, B. Linares-Barranco, and E. Culurciello, “Efficient feedforward categorization of objects and human postures with address-event image sensors,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 2, pp. 302–314, 2012.
  * [31] B. Zhao, R. Ding, S. Chen, B. Linares-Barranco, and H. Tang, “Feed-forward categorization on aer motion events using cortex-like features in a spiking neural network,” IEEE transactions on neural networks and learning systems, vol. 26, no. 9, pp. 1963–1978, 2014
  * [32] Garrick Orchard, Cedric Meyer, Ralph Etienne-Cummings, Christoph Posch, Nitish Thakor, Ryad Benosman, ”Hfirst: A temporal approach to object recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015.
  * [33] G. Orchard, A. Jayawant, G. K. Cohen, and N. Thakor, “Converting static image datasets to spiking neuromorphic datasets using saccades,” Frontiers in Neuroscience, vol. 9, p. 437, 2015.
  * [34] Cohen, G. K, Orchard, G, Leng, S. H, Tapson, J, Benosman, R. B and Van Schaik, A. 2016. Skimming digits: neuromorphic classiﬁcation of spike-encoded images. Fron- tiers in neuroscience 10:184.
  * [35] Liu, Q, Pan, G, Ruan, H, Xing, D, Xu, Q, and Tang, H, Unsupervised aer object recognition based on multiscale spatio-temporal features and spiking neurons. arXiv preprint arXiv:1911.08261, 2019
  * [36] R. Xiao, H. Tang, Y. Ma, R. Yan, and G. Orchard, An event-driven categorization model for aer image sensors using multispike encoding and learning, IEEE Transactions on Neural Networks and Learning Systems, 2020
  * [37] Qianhui Liu, Haibo Ruan, Dong Xing, Huajin Tang, Gang Pan, Effective AER Object Classification Using Segmented Probability-Maximization Learning in Spiking Neural Networks, AAAI, 2020
  * [38] X. Peng, B. Zhao, R. Yan, H. Tang, and Z. Yi, “Bag of events: An efficient probability-based feature extraction method for aer image sensors,” IEEE Transactions on Neural Networks and Learning Systems, vol. 28, no. 4, pp. 791–803, 2017
  * [39] Lagorce, X.; Orchard, G.; Galluppi, F.; Shi, B. E.; and Benosman, R. B. 2017. Hots: a hierarchy of event-based time-surfaces for pattern recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 39(7):1346– 1359.
  * [40] Shibo Zhou, Wei Wang, Xiaohua Li and Zhanpeng Jin, ”A spike learning system for event driven object recognition”, 2021