# High efficiency coupling of free electrons to sub-$\lambda^{3}$ modal
volume, high-Q photonic cavities

Malo Bézard  Université Paris-Saclay, CNRS, Laboratoire de Physique des
Solides, 91405, Orsay, France  Imene Si Hadj Mohand  Centre de Nanosciences et
Nanotechnologies, CNRS, Université Paris Saclay, 10 Bd Thomas Gobert, 91120
Palaiseau  Luigi Ruggierio  Université Paris-Saclay, CNRS, Laboratoire de
Physique des Solides, 91405, Orsay, France  Arthur Le Roux  Centre de
Nanosciences et Nanotechnologies, CNRS, Université Paris Saclay, 10 Bd Thomas
Gobert, 91120 Palaiseau  Yves Auad  Université Paris-Saclay, CNRS, Laboratoire
de Physique des Solides, 91405, Orsay, France  Paul Baroux  Centre de
Nanosciences et Nanotechnologies, CNRS, Université Paris Saclay, 10 Bd Thomas
Gobert, 91120 Palaiseau  Luiz H. G. Tizei  Université Paris-Saclay, CNRS,
Laboratoire de Physique des Solides, 91405, Orsay, France  Xavier Checoury
<EMAIL_ADDRESS>Centre de Nanosciences et Nanotechnologies,
CNRS, Université Paris Saclay, 10 Bd Thomas Gobert, 91120 Palaiseau  Mathieu
Kociak<EMAIL_ADDRESS>Université Paris-Saclay,
CNRS, Laboratoire de Physique des Solides, 91405, Orsay, France

###### Abstract

We report on the design, realization and experimental investigation by
spatially resolved monochromated electron energy loss spectroscopy (EELS) of
high quality factor cavities with modal volumes smaller than $\lambda^{3}$,
with $\lambda$ the free-space wavelength of light. The cavities are based on a
slot defect in a 2D photonic crystal slab made up of silicon. They are
optimized for high coupling of electrons accelerated to 100 kV, to quasi-
Transverse Electrical modes polarized along the slot direction. We studied the
cavities in two geometries. The first geometry, for which the cavities have
been designed, corresponds to an electron beam travelling along the slot
direction. The second consists in the electron beam travelling perpendicular
to the slab. In both cases, a large series of modes is identified. The
dielectric slot modes energies are measured to be in the 0.8- 0.85 eV range,
as per design, and surrounded by two bands of dielectric and air modes of the
photonic structure. The dielectric even slot modes, to which the cavity mode
belongs, are highly coupled to the electrons with up to 3.2$\%$ probability of
creating a slot photon per incident electron. Although the experimental
spectral resolution (around 30 meV) alone does not allow to disentangle cavity
photons from other slot photons, the remarkable agreement between the
experiments and finite-difference time-domain (FDTD) simulations allows us to
deduce that amongst the photons created in the slot, around 30$\%$ are stored
in the cavity mode. A systematic study of the energy and coupling strength as
a function of the photonic band gap parameters permits to foresee increase of
coupling strength by fine-tuning phase matching. Our work demonstrates free
electron coupling to high quality factor cavities with low mode density,
sub-$\lambda^{3}$ modal volume, making it an excellent candidate for
applications such as quantum nano-optics with free electrons.

Free electron based spectroscopies, such as electron energy loss spectroscopy
(EELS), photon induced near field electron microscopy (PINEM) or
cathodoluminescence (CL) have been extensively used to investigate optical
properties of nano-objects 1. One of the main drives of early studies was the
impressive spatial resolution of these techniques, with sub-nanometer
resolution in the (scanning) transmission electron microscopes ((S)TEM) or few
nanometer resolution in scanning electron microscopes (SEM). Given the very
high localization of electromagnetic fields in plasmonic nanoparticles, it is
no surprise that the first studies of CL 2, EELS 3, 4, and PINEM 5 focused on
them. With the increase of the spectral resolution in EELS and improvement in
signal-to-noise ratio for CL, excitations with increasingly higher quality
factors (Q) excitations could be studied, should they be surface plasmons 4,
6, surface phonons 7 and hybridized modes 8, or photonic excitations 9, 10,
11.

However, orders of magnitude improvement (from typically $10-100$ to $10^{5}$
or higher) is needed to access the high quality factors (high-Q) cavities
necessary to explore quantum excitations with fast electrons 12, 13. Also, a
high coupling between the electron and the cavity is required 14. Finally, a
low modal volume is extremely pertinent for quantum nano-optical experiments
in general, and well adapted to the possibilities offered by the excellent
spatial resolution of the electrons.

As the linewidths of high-Q cavities (typically in the $\mu$eV range in the
near-infrared) are intrinsically much smaller than the energy resolution of
state-of-the-art EELS systems (at best a few meV 15), other alternatives have
been considered in the past years for measuring their quality-factor. Indeed,
using tunable lasers in conjunction with PINEM, electron energy gain
spectroscopy (EEGS) became recently possible, unravelling the predicted 16
promise of a laser spectral resolution with the spatial resolution of an
electron 17, 18, 19.

At the time being only few types of cavities, including defects in plasmonic
materials 6, 20, spheres subtending whispering-gallery modes 9, 21, 11, 19 or
ring resonators 18 have been investigated with electron beam techniques. These
cavities suffer from either relatively low Q (less than a thousand) and/or
large modal volumes. Among the alternatives, cavities made up of defects in
photonic band gap (PBG) materials are very attractive. Indeed, in the case of
a bidimensional photonic-crystal slab made up of a lattice of cylindrical
holes (hole slab 22) in a high index material, removing one or several holes
creates a highly spatially localized, low modal volume, cavity mode with
energy within the band gap of the photonic crystal. A pioneer work on EELS of
such a $Q\lesssim 2.10^{4}$ PBG-based cavity was reported 23. Unfortunately,
the geometry of the cavity did not allow for either high coupling nor for high
signal to noise ratio. In these cavities, high quality factors can be achieved
by shifting the holes positions close to the created defect. This smooths the
index contrast between the cavity and the rest of the crystal 24. It therefore
prevents the defect mode from scattering too much towards the vertical
direction, avoiding a major source of energy dissipation. This allows to
attain record Q/V values, where V is modal volume, and Q larger than $10^{6}$
24, 25. Efficient coupling of such cavities to electron beams has never been
considered. Therefore, the demonstration of a defect in a PBG design suitable
for free electron coupling is still to be done.

In this letter, we investigate the highly monochromated spatially-resolved
EELS response of cavities based on a slot inside a 2D photonic-crystal hole
slab made up of silicon. The slot is replacing a (missing) row of holes
leading to the formation of new slot modes. Few holes close to the slot are
slightly shifted in order to create a high-Q, sub-$\lambda^{3}$ modal volume
cavity. The design is such that an electron beam passing in the center of the
structure does not experience elastic scattering despite travelling over
microns. The cavity mode is based on a dielectric quasi-TE slot mode, and
designed such that the field is polarized along the beam path and maximum in
the middle of the slot (even mode). Such an engineering aims at efficiently
increasing the coupling of the electron to the field mode. In order to
investigate the structure and identify the different modes observed in the
EELS, we have coupled experimental investigations along two main directions,
parallel and perpendicular to the slot, to finite-difference time-domain
(FDTD) simulations. The dielectric slots modes are experimentally identified.
Their measured field distribution, in both parallel and perpendicular
directions, corresponds to the expected dielectric quasi-TE modes. Their
energy lie between 0.80 eV and 0.85 eV for hole diameters from 180 nm to 280
nm and a 426 nm spacing. It is, as per design, within the band gap of the
photonic-crystal slab. In the parallel direction, a high coupling results in a
photon creation probability of more than 1 per 30 electrons in the dielectric
even slot modes. The other main groups of modes are identified and mapped,
especially those related to the bottom (dielectric) band and upper (air) bands
of the photonic-crystal slab. The energy resolution of the experiment does not
allow to measure the Q factor of the cavity and to disentangle the cavity mode
from the other dielectric slot modes. However, it is sufficient to isolate
dielectric slot modes from the air slot modes as well as from the slab modes.
Simulations permit to estimate the spectral weight of the cavity mode compared
to that of other dielectric slots modes. Comparison of experimental results
and simulations shows that approximately up to 1 photon is created in the
cavity every 100 incoming electrons thanks to our design optimization.
Finally, we investigated the energy and intensity behaviour of the cavity
modes as a function of the hole size as a way to tune the phase matching
condition between the electron and the modes. Therefore, our experiments
supported by simulations show high coupling values, that can be further
enhanced, to high-Q, sub-$\lambda^{3}$ modal volume cavities. This paves the
way to efficient study of quantum optics with free electrons with photonic-
crystal-based cavities.

As introduced, the design of a high-Q, low modal volume cavity optimized for
coupling to free electrons has to rely on a few requirements. First, a long
interaction path for the electrons, in the spirit of dielectric laser
accelerators 26, is needed. This favours 1D cavities designs over 0 D ones
previously studied by EELS 23. 1D defects can be created by removal or
addition of holes series in an air slab photonic-crystal for example. The
electrical field of the cavity mode needs to be aligned with the 1D defect
direction, so that an electron travelling in-plane along the defect can
efficiently couple to it, and its intensity must be located in the center of
the 1D defect. The defect must be transparent to electrons (i. e., a slot 27),
so that the latter do not undergo any elastic or bulk inelastic scattering, in
contrast to an opaque design for the cavity 23. Last but not least, the design
must allow efficient phase matching.

As shown in Figure 1a, we have achieved such a design, consisting in a slot
separating two halves of an hexagonal lattice of air cylinders within a slab
of silicon. Here, the slab is 220 nm thick, the lattice parameter is a = 426
nm and the cylinder radii r = 0.255 a. The slot width is 100 nm. The waveguide
formed by replacing one row of air holes by the slot has been enlarged by
shifting all the holes so that the two halves are separated by $1.2a\sqrt{3}$,
realizing what is called a $W1.2$ waveguide in the photonic crystal
literature. These parameters have been optimized to allow high coupling with
electrons at around 100 kV. We define the $x$, $y$ and $z$ directions to be
along the slot, perpendicular to the slot in the slab plane, and perpendicular
to the slab plane (see Figure 1a and b). Although we have designed the cavity
to be used with electrons travelling along $x$ (parallel direction in Figure
1b), we also studied samples for electron travelling along or close to the $z$
direction (perpendicular direction in Figure 1b) to confirm the consistency
between the design and experimental realization.

To understand the formation of cavity modes and how they couple to free
electrons, we start by considering an hypothetical infinite slot defect in a
hexagonal air slab. The photonic-crystal slab possesses quasi-TE and quasi-TM
modes (”quasi” refers to the fact that although the field is essentially in
the mid-plane of the slab, some $z$ component exists due to the finite
thickness of the slab 22). For simplicity, those modes will be referred to as
TE and TM in the following. The TE dispersion relation is shown on Figure 1c,
and the TM one in the Supplementary Figure S1 (see Methods for simulation
details). The former are field-symmetric around the $z=0$ plane (see Figure 1d
and Supplementary Figure S2), and are polarized so that the $E_{z}$ component
is negligible compared to the in-plane components. The opposite holds for TM
modes. Only the TE polarization band structure possesses a band gap. Finally,
since TM modes are not well localized and are phase matched with 100 kV
electrons only at energies above the TE band gap (see supplementary Figure
S1), they will not be considered further.

The slot defect acts as a waveguide for TE modes, adding several modes within
the band gap as shown in the Figure 1c. For symmetry reasons, the guided TE
modes may be even or odd with respect to the y = 0 plane (see Figure 1d). The
modes with the lowest energies are even and odd TE modes. They lie well within
the band gap (see Figure 1c), and act as dielectric modes 22 with their
$E_{x}$ field intensities peak in-between with the photonic-crystal holes (see
Figure 1d and supplementary Figure S2). They are coined as $D_{e}$ and $D_{o}$
in Figure 1. The modes with higher energies ($A_{o}$ and $A_{e}$, the latter
not shown on Figure 1c) arise already very close to the upper band and are air
bands, i. e. their $E_{x}$ field intensities is shifted by half a period with
respect to the dielectric modes (see Figure 1d and supplementary Figure S2).
As expected, the four TE slots modes exhibit $E_{z}$ values that are typically
one order of magnitude smaller than $E_{x}$, see supplementary Figure S2.

TE modes are easily distinguishable from the photonic-crystal modes, they are
well localized inside the slot where the electron can pass, and they are
essentially polarized along the $x$ direction. Therefore, they are good
candidates for cavity modes. Also, close to the end of the Brillouin zone,
they exhibit large Q factors (Figure 1c). Finally, the $D_{o}$ modes have a
null x component of the electrical field in the center of the slot, while the
$D_{e}$ ones have their $x$ component maximum right in the center of the slot.
The latter are then easier to target with an electron beam, and will therefore
form the basis of the cavity modes.

In a realistic, finite-size design, the TE modes dispersion curve get
discretized. In this case, TE modes form Fabry-Perot (FP) modes of increasing
orders. The FP mode separation scales as $a/L$, where $L$ is the cavity
length.

To increase the quality factor, some of the central holes are shifted by few
nm 28, see Figure 1a. By doing so, a mode of slightly lower energy is forming
the desired cavity mode. In the present study, the effective length of the
cavity is only a few periods in length, resulting in a modal volume smaller
than $0.1\lambda^{3}$ as deduced from simulations, where $\lambda$ is the
wavelength in vacuum.

The length of the cavity can be monitored by changing the number of displaced
holes. In this paper, we have essentially looked at short structures described
in Figure 1a. On both sides of the shifted holes along $x$, a constant number
of 17 holes delineates the whole structure, so that the total length of the
structure is 15.25 $\mu$m. It is noted that we have tried several other
structures with higher number of shifted holes (total length from 22.5 $\mu m$
to 48.5 $\mu m$). However, in these cases, it was practically impossible to
perform hyperspectral imaging on the cavity due to the strong charging of the
dielectric slab induced by the electron beam illumination. The Figure 1b
presents the two main geometries used in this paper, either parallel to the
slot or perpendicular to it. Due to the high thickness of silicon to be
traversed in the perpendicular geometry, not to mention the parallel geometry,
the missing row area was removed allowing a free propagation of the electrons
through a slot. Of course, two different sets of samples, as presented in
Figure 1b have been prepared, because there is no possibility of tilting the
same sample over 90∘. In practice, the samples have been prepared by electron
lithography as described in the Methods. Twin structures with the same design
have shown Q-factors of the order of $2.10^{5}$.

Figure 1:  Design of the EELS-compatible low modal volume and high-Q cavity.
a. The cavity is made up of a slot separating two 2D hole-slabs photonic
crystals, with slight shifts of the holes indicated by coloured arrows. b.
Although designed to be used with electrons travelling parallel to the slot,
we have also prepared samples that can be investigated with electrons
travelling perpendicular to it, as exemplified here with the two red cones
schematizing the propagation of the electron beam in the two geometries. c.
Band diagram of the structure for the TE even modes. The band diagram has been
calculated for $\rho=0.255$. Different modes are enlightened. Bottom of the
air bands and top of the dielectric bands define the band gap and are
indicated by black dashed lines. The electron dispersion relation is presented
as blue dashed lines and calculated for an accelerating voltage of 100 kV. d.
Simulations for the $E_{x}$ electrical field for the modes $D_{e}$ and $A_{o}$
taken in the mid-plane of the slab. e. Simulation for the $E_{x}$ electrical
field of the high-Q cavity mode taken in the mid-plane of the slab.

EELS experiments have been performed on a NION Hermes microscope (CHROMATEM)
at 100 keV, fitted with a Merlin Quantum Detector camera. Data analysis has
been performed using Hyperspy29, see Methods for details. As described in the
Methods, FDTD simulations have been performed for simulating spectra and maps.
Simulated spectra have been convolved by a Gaussian with 40 meV full width at
half maximum to take into account experimental broadening. Contrary to
experiments where the maps are created by integrating the spectral signal over
a given energy window, simulation maps have been drawn for individual modes
only in simulations.

In the following experimental and simulation results, two parameters are of
interest, namely the lattice parameter $a$, and the radius $r$ to lattice
parameter ratio $\rho=r/a$ . The nominal lattice parameter was set to a = 426
nm for both simulations and experiments throughout the paper.

We show in Figure 2 experimental and simulation results for a parallel
geometry sample. The nominal $\rho$ is equal to 0.255. All designs with
different $\rho$ values exhibit essentially the same sequence of modes but
having different energies (see later in the manuscript). A spectrum measured
with the beam centered in the middle of the slot is presented in Figure 2a. A
series of peaks can be directly observed, ranging from $\approx~{}50$ meV to a
few eVs. We are interested in the energy range close to the band gap energy of
the photonic-crystal, corresponding for the sample in Figure 2 to the energy-
resolution-limited sharp peak $\beta$ (0.81 eV) and the broader peaks $\gamma$
(0.95 eV) and $\alpha$ (around 0.7 eV). As shown in Figure 2b, the intensity
of the three peaks are mostly localized in the very center of the structure.
The measured probability of photon creation per incident electron is more than
$2\%$ for the $\beta$ peak, i.e. around 1 creation of excitation every 50
incoming electron. As will be shown later, it can reach up to $3.2\%$ for
other $\rho$ parameters. This relatively strong coupling decreases rapidly as
the beam is moved away from the slot.

The comparison with simulations is excellent in energy position of the
different peaks and global spectral shape as shown in Figure 2a. We also note
that the absolute values given by the simulations, that do not rely on any
fitting parameters, and experimental probabilities are sizable. Differences
between experiments and simulations might rely in slight geometrical
parameters differences, imperfect roundness of the holes, imperfect knowledge
of the silicon index and dispersion, and imperfect energy calibration of the
EELS spectrometer. Simulated spectra before convolution (Supplementary Figure
S3) show that all experimental peaks are a sum of several ones that cannot be
disentangled experimentally in energy position. The simulated $\beta$ peak is
made up of only a restricted number of peaks. Amongst these peaks, a dominant
mode - the cavity mode - can be distinguished because it is several order of
magnitudes more intense and with a quality factor typically two orders of
magnitude larger (see supplementary Figure S3).

Altogether, intensities and quality factors balance in such a way that the
cavity mode represents typically 30 $\%$ of the total spectral weight of the
$\beta$ peak.

The comparison with simulations is also remarkable for the $\gamma$ peak, both
in shape and intensity in the spectrum. However, the absolute value in the
simulated map, is different from the experimental one. Also, the $y$ symmetry
is different between experiments and simulations. In both cases, this is again
because only one specific mode has been simulated out of all those that are
constituting the experimental peak.

At energies higher than around $1~{}eV$, there exists a plethora of modes that
we will not discuss here as they fall above the band gap of Si (not to be
confused with the band gap of the photonic-crystal itself). Typical spectra
and filtered images are shown in the supplementary materials Figure S4 and S5.

Figure 2:  EELS spectral-imaging of a parallel configuration sample a.
Spectrum taken with the beam passing through the center of the slot. The inset
shows a magnified view of the spectrum around the main resonances as well as
the corresponding FDTD simulated spectrum (dotted curve). The position of the
cavity mode, as well as the mode in the $\gamma$ peak used to generate the
maps in c. are indicated with a vertical mark. b. Maps of the $\alpha$,
$\beta$ and $\gamma$ peaks, filtered on the energy windows indicated in the
inset of a. c. Experimental HAADF and FDTD simulations. FDTD simulations have
been performed for individual modes (cav.: cavity mode, and $\gamma$)
indicated in the inset of the a., and therefore are less intense than the
corresponding spectra that include several convoluted modes.

The perpendicular geometry is exemplified in Figure 3 for the same parameters
as the parallel geometry of Figure 2. Here again, several modes are observed
in the same energy ranges (see supplementary Figure S6 for a full spectrum),
but we focus on the energy region between 0.6 and 1 eV. Note that the tail of
the ZLP has been removed by fitting a power-law, see supplementary Figure S7.
In Figure 3a (middle, bottom), two main peaks are seen, around 0.8 eV (peak
$\beta^{\prime}$) and 0.9 eV (peak $\gamma^{\prime}$). The energy position of
these peaks match that of the $\beta$ and $\gamma$ peaks (see top of Figure
3a). The position of the beam is emphasized in the respective insets. From the
energy filtered map in Figure 3b, we can see that their spatial distribution
are aligned (peak $\beta^{\prime}$) and shifted by half a period (peak
$\gamma^{\prime}$) with the hole position. The intensity of the peaks is two
orders of magnitude smaller than in the other geometry. The simulated spectra
and maps in Figure 3 fit very well in energy position and intensity. The
absolute interaction probabilities are also sizable between experiment and
theory in the spectra. However, the experimental and simulation maps show one
to two orders of magnitude difference in intensity. This is because, contrary
to the case of the parallel geometry, where only one mode contributes to
$\sim$ 30 $\%$ of the intensity, here all modes contribute almost equally.
Experimental maps are summed up over all modes while simulations retain only
one. This emphasizes a key feature of our optimization scheme that has been
developed for optimized coupling to the cavity mode in the parallel direction
and not the perpendicular one.

Figure 3:  EELS spectral-imaging of a perpendicular configuration sample a.
Spectra extracted from two representative positions indicated in the inset and
highlighted in b, and compared to the spectrum of Figure 2a at the top
(nominal $a$ and $\rho$ are the same on both figures). The background due to
the tail of the ZLP has been subtracted (see supplementary). The FDTD spectra
are also shown as dashed lines. b. Experimentally filtered maps around peak
$\beta^{\prime}$ and $\gamma^{\prime}$ on the energy windows indicated in a.
Single mode maps corresponding for the cavity mode and and one representative
mode of the $\gamma$ bands. Intensities are substantially lower than the
filtered maps due to a relatively even breakdown of spectral weight of the
modes in the perpendicular direction.

The coupling between the electron and the modes of the structures can be tuned
by phase matching. Usually, one tune the acceleration voltage for a given
sample geometry. Unfortunately, the use of a monochromator does not allow for
sufficient tuning range. An alternative method consists in changing
systematically $\rho$, as it is well-known that the whole band diagram scales
with the it 22.

Therefore, as exemplified on Figure 4 we have performed systematic
measurements on parallel configuration samples as a function of $\rho$.

$\rho$ values were changed from 0.210 to 0.330. Three representative spectra
for increasing hole diameters are displayed in Figure 4a, showing an energy
and intensity increase. These tendencies are synthesized on Figure 4b and c.
In Figure 4b, one can see the energy dependence of the $\beta$ and $\gamma$
peaks as a function of $\rho$ both for experimental and theoretical values.
The slopes are in excellent agreement. There is only a slight and constant
shift in the energy value. Figure 4c presents the intensity dependence of the
modes $\beta$ and $\gamma$. Experiments and simulations results are in good
agreement although simulations are constantly larger by typical factor of 2 to
3. We want here again to emphasize that the simulations have been done without
any fitting parameters. We note that the maximum experimental coupling for the
peak $\beta$ is shown to be 3.2$\%$ per incoming electron.

Figure 4:  Influence of the ratio $\rho$ on modes properties. a. Spectra for
three different holes diameters and same hole separation in the parallel
configuration. Gaussian fitting is also shown, and parameters are indicated in
the legend. b. $\rho$ dependence of the energy of the peak $\beta$ and
$\gamma$ and comparison with FDTD simulations. c. $\rho$ dependence of the
intensity for peak $\beta$ and $\gamma$ and comparison with simulations.

Before trying to identify the different modes, we need to comment on the
effect of monochromation on the experiment. High monochromation leads to a
decrease of the effect of the ZLP tail, making low energy resonances (down to
the far-IR 30) much easier to measure, but we must recognize the finite
resolution of the STEM-EELS system used here, which was from $\approx 20$ to
$\approx 40$ meV at 100 keV. This has to be compared to few physical
quantities at hand here. First, there are few physical quantities that should
be resolved: the bang gap of the photonic-crystal, of the order of 150 meV,
the energy difference between dielectric and air slot modes, of the order of
70 meV, and the bandwidth of the bottom (dieletric) and top (air) photonic-
crystal bands as seen in EELS. Second, there are physical quantities that
cannot be resolved, especially the linewidth of the cavity mode (of the order
of a few $\mu$eV, the energy difference between the air slot modes and the top
band of the photonic-crystal (typically less than 10 meV), and the energy
difference between sub-modes of the slot (see Figure 1).

We are now in the position to detail the link between peaks measured in EELS
and the main modes of the structures.

From their energy, the $\alpha$ feature corresponds to the bottom (dielectric)
band of the photonic-crystal slab. The $\gamma$ mode is made up of a band
encompassing air modes related to the top of the band structure of the PBG and
the air slot modes, which are partly degenerated as seen on Figure 1c. The
$\beta$ peak can be associated to the even dielectric slot modes $D_{e}$.
Beyond the correspondence between experimental and simulated EELS spatial
distribution, they both more generally agree with the $|E_{x}|^{2}$
distribution of $D_{e}$ and confirms our interpretation (see Supplementary
Figure S2) - indeed, EELS is closely related to the electromagnetic density of
state along the electron trajectory 16. Together with the already discussed
argument that the spectral weight of the cavity mode in the simulation which
account for 30% of the total intensity of the $\beta$ peak, this definitely
proves that the $\beta$ peak is dominated by the coupling of the electron beam
to the cavity mode.

If we now switch to the perpendicular case, we note that the $\beta^{\prime}$
energy matches also that of the dielectric slot modes. Both experimental and
simulated maps show maxima aligned to the holes centers, as expected for
dielectric modes. Indeed, EELS roughly resembles the corresponding
$|E_{z}|^{2}$ field distribution (see supplementary Figure S2). $|E_{z}|^{2}$
is mapped in the perpendicular geometry, and $|E_{x}|^{2}$ in the parallel
one. Now, $|E_{z}|^{2}$ and $|E_{x}|^{2}$ are shifted by half a period in the
slab plane, ie, while $|E_{z}|^{2}$ is maximum between two holes as confirmed
by the EELS measurement (see Figure 3), the $|E_{x}|^{2}$ should peak at the
hole positions (see Supplementary Figure S2). Same reasoning applies to
$\gamma^{\prime}$ which corresponds to the air band, possibly mixed with the
air slot modes. This is also confirmed by EELS simulations. Finally, there is
a difference between experimental and simulated maps for $\beta^{\prime}$ and
the cavity mode. The simulation clearly shows an even mode behaviour (as
selected), while the experimental has a mixed character. This is because,
contrary to the parallel case, no special phase matching is expected in that
geometry, therefore all the modes of the $D_{e}$ and $D_{o}$ branches can
contribute to the EELS signal. Beyond the fast periodic modulation of period
$a$, the lack of spectral resolution in EELS does not allow to image the
envelopes of the different modes.

To summarize, the $\alpha$ peak is related to the dielectric band of the
photonic-crystal slab, while $\gamma$ and $\gamma^{\prime}$ correspond to its
air band and/or the air quasi-TE guided mode of the slot. The $\beta$ and
$\beta^{\prime}$ are both made up of dielectric slots modes. However, while
$\beta^{\prime}$ is made up of a mixture of even and odd modes, $\beta$ is
constituted by a finite number of even modes, and dominated by the
contribution of the cavity mode.

Finally, the achieved coupling strength, leading to the creation of few
percents of photons per electrons in a single mode, narrow band cavity, is
relatively high. Indeed, reported experiments in EELS, cathodoluminescence or
PINEM have largely focused the study of plasmons in nanoparticles, for which a
typical value for coupling is an order of magnitude lower. This, despite the
fact that the plasmons have extremely high density of states. In our case,
both the length and access to the phase-matching explains the coupling values.
Much larger coupling values have been reported 31 for the EELS of surface-
plasmon polaritons of surfaces probed in grazing incidence, but again in this
case the density of states is enormous compared to the present case. Our
situation is closer to that of Feist et al. 32, who reported a slitghly larger
coupling efficiency (around 2.5 $\%$) to a single mode of an optical micro-
resonator. The main difference is the modal volume, which can be very roughly
estimated to be $\sim$ 260 $\lambda^{3}$. This means that our design provides
similar coupling but with a modal volume which is typically 3 orders of
magnitude lower, therefore an interesting candidate for quantum optics
applications.

In conclusion, we have developed photonic band gap based cavities compatible
with EELS experiments in a TEM. Spatially resolved EELS experiments combined
to FDTD simulations made it possible to prove that, despite not having enough
spectral resolution to resolve cavity modes, those could be isolated within
the band gap and mapped. We have shown that coupling of slots modes can be as
high as 3.2 $\%$ of photons created per incoming electrons, out of which a
third are directed to the cavity mode with a quality factor Q of the order of
$2.5.10^{5}$. Resolving spectrally the cavity modes could be done in the
future using PINEM or electron energy gain spectroscopy 18, 19. The coupling
between the electron and the cavity is already sufficiently high that we can
hope for use in experiments requiring high-Q cavities 13, 21, 17, 12. It could
be further improved by a better phase-matching - which would require a more
systematic design and production of samples - and workarounds for charging
issues.

## Acknowledgments

This project has been funded in part by the European Union through the Horizon
2020 Research and Innovation Program (grant agreement No. 101017720 (EBEAM)),
the French National Agency for Research under the program of future investment
TEMPOS-CHROMATEM (reference No. ANR-10-EQPX-50), QUENOT (ANR-20-CE30-0033) and
ANR OFELIA (ANR-21-CE24-007). This work was supported by the French RENATECH
network. MK thanks Javier Garcia de Abajo for fruitful discussions on the
topic.

## Methods

### Simulations

Band diagrams and the corresponding mode patterns of the waveguide were
calculated by FDTD with MEEP software 33. A single period of the photonic
crystal is simulated using Bloch boundary condition in the direction of the
waveguide and perfectly matched layers (PML) in the two other directions.

The EELS spectrum $\Gamma(\hbar\omega,y,z)$ for a given position y, z, or
equivalently the probability to create a photon in the cavity per electron and
unit of energy can be calculated by the time-domain Fourier transform of the
field with proper prefactors:

$\Gamma(\omega,y,z)=\frac{ev}{\pi\hbar\omega}\textrm{Re}\int_{-\infty}^{+\infty}\hat{E}_{x}^{\textrm{ind}}(vt,y,z,\omega)e^{i\omega
t}dt$ (1)

with
$\hat{E}_{x}^{\textrm{ind}}(x,y,z,\omega)=\int_{-\infty}^{+\infty}E_{x}^{\textrm{ind}}(x,y,z,t)e^{-i\omega
t}dt$, the Fourier transform of the induced electric field and $v$ the speed
of the electron.

For the simulation of EELS spectra, a digital analog of the continuous
integral in eq. 1 is calculated using an home-made finite-difference in time-
domain (FDTD) software 33, 34. The sample was modelled using the same
parameters as the experimental ones, with the silicon refractive index set to
3.45 and a FDTD resolution of 18 nm. An electron travelling along the slot is
modelled by a single FDTD pixel current source travelling with a velocity
corresponding to an energy of 100 keV as in the EELS experiment. The field is
recorded at every FDTD time step and at every spatial pixel along the whole
path taken by the electron (a straight line). From the resulting field, the
field generated by the crossing of a single electron in an empty simulation
domain is subtracted to remove the transients generated by the entrance and
exit of the charged particle in the simulation domain. Then, at each spatial
pixel, a temporal Fourier transform is calculated using a Padé approximant
technique to increase the accuracy of the digital Fourier transform. Finally a
digital analog of the continuous integral in eq. (1) is calculated.

This method requires one simulation per pixel in the (y,z) plane perpendicular
to the electron trajectory and thus is not well adapted to simulate the
spatial field mapping achieved in EELS experiment. A faster calculation has
been done for a particular mode since its electric field pattern,
$\mathbf{u}_{k}(\mathbf{r},\omega_{k})$, with $\omega_{k}$ the complex
resonant pulsation of the cavity, can be calculated by FDTD quickly. Indeed,
the induced field can be calculated with the Green tensor,
$\overleftrightarrow{\mathbf{G}}(\mathbf{r},\mathbf{r}^{\prime},\omega)$ of
the cavity,

$\mathbf{E}^{\textrm{ind}}(\mathbf{r}(t),\omega)=-i\omega\int_{\mathbb{R}^{3}}\overleftrightarrow{\mathbf{G}}(\mathbf{r},\mathbf{r}^{\prime},\omega)\mu_{0}\mathbf{j}(\mathbf{r}^{\prime}(t),\omega)d^{3}\mathbf{r}^{\prime}$
(2)

with
$\mathbf{j}(\mathbf{r}^{\prime}(t),\omega)=e\frac{\overrightarrow{v}}{|v|}e^{-i\omega
x^{\prime}/v}\delta(y_{0},z_{0})$, the current density generated by the
crossing of a single electron travelling along $x$. Decomposing the Green
tensor on the basis of the cavity modes, we get for an electron travelling in
the slot along the x direction

$\Gamma(\omega,y,z)=\sum_{k}\frac{e^{2}}{2\pi\hbar\omega}\frac{1}{\mathcal{E}_{0}}\textrm{Re}\left(\frac{i\omega}{\omega_{k}^{2}-\omega^{2}}\right)\left|FT[u_{xk}(x^{\prime},y_{0},z_{0},\omega_{k})]\right|_{-\frac{\omega}{v}}^{2}$
(3)

where the $\mathbf{u}_{k}(\mathbf{r},\omega_{k})$ have been normalized so that
$\frac{1}{2}\int_{\mathbb{R}^{3}}\varepsilon_{0}\varepsilon_{r}(\mathbf{r}^{\prime})\bm{u}_{k^{\prime}}^{*T}(\mathbf{r}^{\prime},\omega_{k})\mathbf{u}_{k}(\mathbf{r},\omega_{k})d^{3}\mathbf{r}=\mathcal{E}_{0}\delta_{k^{\prime}k}$
with $\mathcal{E}_{0}=1$ Joule and
$FT[u_{xk}(x^{\prime},y_{0},z_{0},\omega_{k})]_{\frac{\omega}{v}}$ is the
spatial Fourier transform of the mode along the x direction evaluated at a
wave vector $\frac{\omega}{v}$. Similar simulations have been performed for
the perpendicular case, but considering $E_{z}(x,y,vt,\omega$) instead of
$E_{x}(vt,y,z,\omega)$.

Spectra were convoluted by a 40 meV gaussian to account for experimental
spectral broadening.

### Sample fabrication

To produce the samples, a silicon on insulator wafer with a 220-nm top layer
on 2-µm burried oxide was spin-coated with ZEP electronic resist. Photonic
structures were lithographied with a Raith ebpg 5200 system at 100 keV.
Patterns were transfered into the silicon layer by inductively coupled
reactive ion etching (ICP-RIE) using a SF6-C4F8 gas mixture. The resist was
removed with a solvent. The samples are then saw diced to a size of 250-µm
$\times$ 2.8 mm compatible with TEM holders and wet etched by hydrofluoric
acid to remove the buried oxide.

For the perpendicular geometry experiments, a backside lithography aligned
with the photonic crystal and a subsequent etching has been performed before
dicing the sample in order to remove the substrate locally.

### Experiments and data analysis

EELS experiments have been performed on a NION Hermes microscope (CHROMATEM)
at 100 keV, fitted with a Merlin Quantum Detector camera. The spectral
resolution was set to 30-40 meV (as measured on the ZLP) with a dispersion of
6.7 meV/channel. Incident semi-angle was 5 mrd and acceptance semi-angle 30
mrd. Spectral-images with typically $10^{4}$ pixels were recorded with a
typical 30 ms dwell time for the parallel direction and 100 ms for the
perpendicular one. The spectra were normalized by their total integrated
number of counts. For an electron in vacuum, this ensures that the sum of all
events probability is one, as expected as all the electrons are detected on
the spectrometer camera. As this assumption is wrong for electrons travelling
in the bulk, corresponding pixels have been masked in the data presentation.
Then, all spectra were normalized by the dispersion and represented in
percentage. At the end, the spectra amplitude are represented as the ($\%$)
probability to create an excitation in the material per unit of energy. Fitted
or integrated maps are then directly showing this probability at each pixel.
All data were analyzed and processed using Hyperspy 29.

## References

  * Polman et al. 2019 Polman, A.; Kociak, M.; García de Abajo, F. J. Electron-beam spectroscopy for nanophotonics. _Nature Materials_ 2019, _18_ , 1158–1171
  * Yamamoto et al. 2001 Yamamoto, N.; Araya, K.; García de Abajo, F. J. Photon emission from silver particles induced by a high-energy electron beam. _Physical Review B - Condensed Matter and Materials Physics_ 2001, _64_ , 2054191–2054199
  * Nelayah et al. 2007 Nelayah, J.; Kociak, M.; Stéphan, O.; Garcia de Abajo, F. J.; Tence, M.; Henrard, L.; Taverna, D.; Pastoriza-Santos, I.; Liz-Marzan, L. M.; Colliex, C. Mapping surface plasmons on a single metallic nanoparticle. _Nature Physics_ 2007, _3_ , 348–353
  * Rossouw and Botton 2013 Rossouw, D.; Botton, G. A. Plasmonic response of bent silver nanowires for nanophotonic subwavelength waveguiding. _Physical Review Letters_ 2013, _110_ , 1–5
  * Barwick et al. 2009 Barwick, B.; Flannigan, D. J.; Zewail, A. H. Photon-induced near-field electron microscopy. _Nature_ 2009, _462_ , 902–906
  * Saito and Yamamoto 2015 Saito, H.; Yamamoto, N. Control of Light Emission by a Plasmonic Crystal Cavity. _Nano Letters_ 2015, _15_ , 5764–5769
  * Lagos et al. 2022 Lagos, M. J.; Bicket, I. C.; Mousavi M., S. S.; Botton, G. A. Advances in ultrahigh-energy resolution EELS: Phonons, infrared plasmons and strongly coupled modes. _Microscopy_ 2022, _71_ , I174–I199
  * Tizei et al. 2020 Tizei, L. H. G. et al. Tailored Nanoscale Plasmon-Enhanced Vibrational Electron Spectroscopy. _Nano Letters_ 2020, _20_ , 2973–2979
  * Hyun et al. 2008 Hyun, J. K.; Couillard, M.; Rajendran, P.; Liddell, C. M.; Muller, D. A. Measuring far-ultraviolet whispering gallery modes with high energy electrons. _Applied Physics Letters_ 2008, _93_
  * Sapienza et al. 2012 Sapienza, R.; Coenen, T.; Renger, J.; Kuttge, M.; Van Hulst, N. F.; Polman, A. Deep-subwavelength imaging of the modal dispersion of light. _Nature Materials_ 2012, _11_ , 781–787
  * Auad et al. 2022 Auad, Y.; Hamon, C.; Tencé, M.; Lourenço-Martins, H.; Mkhitaryan, V.; Stéphan, O.; García de Abajo, F. J.; Tizei, L. H. G.; Kociak, M. Unveiling the Coupling of Single Metallic Nanoparticles to Whispering-Gallery Microcavities. _Nano Letters_ 2022, _22_ , 319–327
  * Di Giulio et al. 2019 Di Giulio, V.; Kociak, M.; García de Abajo, F. J. Probing quantum optical excitations with fast electrons. _Optica_ 2019, _6_ , 1524
  * Kfir 2019 Kfir, O. Entanglements of Electrons and Cavity Photons in the Strong-Coupling Regime. _Physical Review Letters_ 2019, _123_ , 103602
  * García De Abajo 2013 García De Abajo, F. J. Multiple excitation of confined graphene plasmons by single free electrons. _ACS Nano_ 2013, _7_ , 11409–11419
  * Krivanek et al. 2009 Krivanek, O. L.; Ursin, J. P.; Bacon, N. J.; Corbin, G. J.; Dellby, N.; Hrncirik, P.; Murfitt, M. F.; Own, C. S.; Szilagyi, Z. S. High-energy-resolution monochromator for aberration-corrected scanning transmission electron microscopy/electron energy-loss spectroscopy. _Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences_ 2009, _367_ , 3683–3697
  * Garcia de Abajo and Kociak 2008 Garcia de Abajo, F. J.; Kociak, M. Probing the photonic local density of states with electron energy loss spectroscopy. _Physical Review Letters_ 2008, _100_ , 106804
  * Wang et al. 2020 Wang, K.; Dahan, R.; Shentcis, M.; Kauffmann, Y.; Ben Hayun, A.; Reinhardt, O.; Tsesses, S.; Kaminer, I. Coherent interaction between free electrons and a photonic cavity. _Nature_ 2020, _582_ , 50–54
  * Henke et al. 2021 Henke, J.-W.; Raja, A. S.; Feist, A.; Huang, G.; Arend, G.; Yang, Y.; Kappert, F. J.; Wang, R. N.; Möller, M.; Pan, J.; Liu, J.; Kfir, O.; Ropers, C.; Kippenberg, T. J. Integrated photonics enables continuous-beam electron phase modulation. _Nature_ 2021, _600_ , 653–658
  * Auad et al. 2023 Auad, Y.; Dias, E. J. C.; Tencé, M.; Blazit, J.-d.; Li, X.; Zagonel, L. F.; Stéphan, O.; Tizei, L. H. G.; García de Abajo, F. J.; Kociak, M. $\mu$eV electron spectromicroscopy using free-space light. _Nature Communications_ 2023, _14_ , 4442
  * Saito et al. 2019 Saito, H.; Lourenço-Martins, H.; Bonnet, N.; Li, X.; Lovejoy, T. C.; Dellby, N.; Stéphan, O.; Kociak, M.; Tizei, L. H. G. Emergence of point defect states in a plasmonic crystal. _Physical Review B_ 2019, _100_ , 1–6
  * Kfir et al. 2020 Kfir, O.; Lourenço-Martins, H.; Storeck, G.; Sivis, M.; Harvey, T. R.; Kippenberg, T. J.; Feist, A.; Ropers, C. Controlling free electrons with optical whispering-gallery modes. _Nature_ 2020, _582_ , 46–49
  * Joannopoulos et al. 2008 Joannopoulos, J. D.; Johnson, S. G.; Winn, J. N.; Meade, R. D. _Princeton University Press_ ; Princeton University Press, 2008
  * Le Thomas et al. 2013 Le Thomas, N.; Alexander, D. T.; Cantoni, M.; Sigle, W.; Houdré, R.; Hébert, C. Imaging of high-Q cavity optical modes by electron energy-loss microscopy. _Physical Review B - Condensed Matter and Materials Physics_ 2013, _87_ , 1–7
  * Akahane et al. 2003 Akahane, Y.; Asano, T.; Song, B. S.; Noda, S. High-Q photonic nanocavity in a two-dimensional photonic crystal. _Nature_ 2003, _425_ , 944–947
  * Han et al. 2010 Han, Z.; Checoury, X.; Néel, D.; David, S.; El Kurdi, M.; Boucaud, P. Optimized design for 2 × $10^{6}$ ultra-high Q silicon photonic crystal cavities. _Optics Communications_ 2010, _283_ , 4387–4391
  * England et al. 2014 England, R. J. et al. Dielectric laser accelerators. _Reviews of Modern Physics_ 2014, _86_ , 1337–1389
  * D’Mello et al. 2023 D’Mello, Y.; Dahan, R.; Bernal, S.; Shi, X.; Kaminer, I.; Plant, D. V. Efficient coupling between free electrons and the supermode of a silicon slot waveguide. _Optics Express_ 2023, _31_ , 19443
  * Kuramochi et al. 2006 Kuramochi, E.; Notomi, M.; Mitsugi, S.; Shinya, A.; Tanabe, T.; Watanabe, T. Ultrahigh- Q photonic crystal nanocavities realized by the local width modulation of a line defect. _Applied Physics Letters_ 2006, _88_ , 1–3
  * de la Peña et al. 2022 de la Peña, F. et al. hyperspy/hyperspy: Release v1.7.3. 2022; https://doi.org/10.5281/zenodo.7263263
  * Li et al. 2021 Li, X. Y.; Haberfehlner, G.; Hohenester, U.; Stephan, O.; Kothleitner, G.; Kociak, M. Three-dimensional vectorial imaging of surface phonon polaritons. _SCIENCE_ 2021, _371_ , 1364
  * Adiv et al. 2023 Adiv, Y.; Hu, H.; Tsesses, S.; Dahan, R.; Wang, K.; Kurman, Y.; Gorlach, A.; Chen, H.; Lin, X.; Bartal, G.; Kaminer, I. Observation of 2D Cherenkov Radiation. _Physical Review X_ 2023, _13_ , 11002
  * Feist et al. 2022 Feist, A.; Huang, G.; Arend, G.; Yang, Y.; Henke, J. W.; Raja, A. S.; Kappert, F. J.; Wang, R. N.; Lourenço-Martins, H.; Qiu, Z.; Liu, J.; Kfir, O.; Kippenberg, T. J.; Ropers, C. Cavity-mediated electron-photon pairs. _Science_ 2022, _377_ , 777–780
  * Oskooi et al. 2010 Oskooi, A. F.; Roundy, D.; Ibanescu, M.; Bermel, P.; Joannopoulos, J. D.; Johnson, S. G. Meep: A flexible free-software package for electromagnetic simulations by the FDTD method. _Computer Physics Communications_ 2010, _181_ , 687–702
  * Song et al. 2021 Song, Y.; Monceaux, Y.; Bittner, S.; Chao, K.; Reynoso De La Cruz, H. M.; Lafargue, C.; Decanini, D.; Dietz, B.; Zyss, J.; Grigis, A.; Checoury, X.; Lebental, M. Möbius Strip Microlasers: A Testbed for Non-Euclidean Photonics. _Physical Review Letters_ 2021, _127_ , 1–7


220635.08$+$000323.3 | 0.22 $\pm$ 0.02 | 0.13 $\pm$ 0.0 | 0.12 $\pm$ 0.0 | -0.5 $\pm$ 0.05 | 40.7 | 1.3 $\pm$ 0.0 | 40.3 | 22.1
221142.42$+$002731.2 | -0.56 $\pm$ 0.16 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 39.4 | 25.3
221142.51$+$002728.5 | -0.54 $\pm$ 0.14 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 41.9 | 25.5
222051.55$+$005815.5 | -0.28 $\pm$ 0.0 | 0.75 $\pm$ 0.02 | 0.37 $\pm$ 0.05 |  $\cdots$ $\pm$ $\cdots$ | 40.0 | -0.2 $\pm$ 0.0 | 40.0 | 24.7
222051.70$+$005816.7 |  $\cdots$ $\pm$ $\cdots$ | 1.01 $\pm$ 0.0 | 1.02 $\pm$ 0.0 |  $\cdots$ $\pm$ $\cdots$ | 38.9 | -0.9 $\pm$ 0.0 | 39.7 | 25.1
222907.53$-$000411.1 | 0.43 $\pm$ 0.0 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 40.8 | 25.2
222907.77$-$000410.9 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 36.9 | 24.9
223222.41$+$001226.3 | 0.08 $\pm$ 0.14 | -0.25 $\pm$ 0.04 | -0.36 $\pm$ 0.05 | -1.12 $\pm$ 1.48 | 41.2 | 1.5 $\pm$ 0.0 | 39.9 | 22.9
223222.60$+$001224.7 | -0.21 $\pm$ 0.27 | -0.35 $\pm$ 0.0 | -0.38 $\pm$ 0.0 | -1.31 $\pm$ 3.37 | 41.2 | 1.6 $\pm$ 0.0 | 39.8 | 23.2
223546.28$+$000358.8 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ | 25.0
223546.40$+$000358.7 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ | 24.2
224426.44$+$001051.3 | 0.26 $\pm$ 0.0 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 40.6 | 24.6
224426.61$+$001051.9 | 0.16 $\pm$ 0.11 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 40.8 | 25.0
224532.53$+$005857.9 | 0.66 $\pm$ 0.01 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 41.2 | 24.5
224532.68$+$005855.8 | 0.48 $\pm$ 0.03 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 41.6 | 24.1
225222.52$+$010658.0 | 0.08 $\pm$ 0.05 | 0.54 $\pm$ 0.04 | 0.48 $\pm$ 0.04 | -0.21 $\pm$ 0.24 | 40.1 | 0.1 $\pm$ 0.4 | 40.7 | 22.4
225222.65$+$010700.6 | -0.44 $\pm$ 0.21 | -0.39 $\pm$ 0.0 | -0.55 $\pm$ 0.0 | -1.67 $\pm$ 7.7 | 40.7 | 1.7 $\pm$ 0.0 | 39.5 | 21.9
225817.73$+$003007.7 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 39.2 | 23.4
225817.97$+$003011.5 | -0.15 $\pm$ 0.16 | -0.69 $\pm$ 0.9 | -0.52 $\pm$ 0.62 | -1.45 $\pm$ 2.43 | 40.6 | 1.3 $\pm$ 0.0 | 39.8 | 24.3
230010.18$-$000531.7 | -0.4 $\pm$ 0.0 | 0.38 $\pm$ 0.04 | -0.03 $\pm$ 0.09 |  $\cdots$ $\pm$ $\cdots$ | 40.2 | 0.1 $\pm$ 0.3 | 39.8 | 23.2
230010.24$-$000534.0 | -0.59 $\pm$ 0.37 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 39.9 | 23.0
230342.74$-$011712.7 | -0.37 $\pm$ 0.39 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 40.1 | 24.4
230342.91$-$011711.9 | -0.07 $\pm$ 0.0 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ | 40.5 | 24.6
231953.31$+$003816.7 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ | 25.3
231953.43$+$003813.4 |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ |  $\cdots$ $\pm$ $\cdots$ |  $\cdots$ | 24.6

Note. — Every two rows is a pair, and sources are sorted in ascending R.A. (1)
J2000 coordinate of the optical source; (2)$-$(5) logarithmic emission line
ratios and 1$\sigma$ errors; (6) logarithmic H$\alpha$ luminosity; (7)
H$\alpha$ equivalent width and 1$\sigma$ error; (8) logarithmic [O iii]
luminosity; (9) logarithmic radio power computed from integrated source flux
at 1.4 GHz.

Table 6Derived Optical Properties of Kinematic Pair Sample

Optical Designation | $M_{\star}$ | $\sigma_{\star}$ | $M_{\rm BH}$ | log($\lambda_{\rm Edd}$)
---|---|---|---|---
J2000 | log(M⊙) | (km s-1) | log(M⊙) |
(1) | (2) | (3) | (4) | (5)
005113.93$+$002047.2 | 10.3 | 150.0* | 7.9 | -2.0
005114.11$+$002049.4 | 10.5 | 173.78* | 8.1 | -2.4
013412.78$-$010729.5 | 11.3 | 278.48 | 9.0 | -3.3
013412.84$-$010725.0 | 10.6 | 225.26 | 8.2 | -3.1
014203.08$-$000150.3 | 10.9 |  $\cdots$ | 8.6 | -2.8
014203.24$-$000151.0 | 11.4 | 234.13 | 9.2 | -2.4
014715.86$-$000819.5 | 11.4 |  $\cdots$ | 9.1 | -2.9
014715.94$-$000817.4 | 10.4 | 646.26 | 8.0 | -0.6
014928.38$-$001446.0 | 9.1 |  $\cdots$ | 6.5 | -1.5
014928.41$-$001447.8 | 9.0 |  $\cdots$ | 6.4 | -1.6
015253.79$-$001005.6 | 10.9 |  $\cdots$ | 8.6 | -3.5
015253.92$-$001004.3 | 10.2 |  $\cdots$ | 7.7 | -3.3
020301.40$-$003341.7 | 10.1 |  $\cdots$ | 7.7 | -1.3
020301.43$-$003339.3 | 10.0 |  $\cdots$ | 7.6 | -1.0
021021.00$-$005124.2 | 11.2 |  $\cdots$ | 9.0 | -3.2
021021.13$-$005126.7 | 10.2 |  $\cdots$ | 7.7 | -2.0
220634.97$+$000327.6 | 10.1 | 124.45* | 7.6 | -2.3
220635.08$+$000323.3 | 10.4 | 180.03* | 7.9 | -2.2
221142.42$+$002731.2 | 10.9 |  $\cdots$ | 8.6 | -1.9
221142.51$+$002728.5 | 12.1 | 202.3 | 9.9 | -2.5
222051.55$+$005815.5 | 11.2 | 357.58 | 8.9 | -2.6
222051.70$+$005816.7 | 10.9 | 350.46 | 8.6 | -2.1
222907.53$-$000411.1 | 11.6 | 250.83 | 9.4 | -2.7
222907.77$-$000410.9 | 10.9 |  $\cdots$ | 8.6 | -2.2
223222.41$+$001226.3 | 10.0 | 195.63* | 7.5 | -2.0
223222.60$+$001224.7 | 10.2 | 170.51* | 7.7 | -2.2
223546.28$+$000358.8 | 11.8 | 235.34 | 9.6 |  $\cdots$
223546.40$+$000358.7 | 11.4 | 280.15 | 9.2 |  $\cdots$
224426.44$+$001051.3 | 11.1 |  $\cdots$ | 8.9 | -2.4
224426.61$+$001051.9 | 10.0 |  $\cdots$ | 7.5 | -0.9
224532.53$+$005857.9 | 10.2 | 274.69 | 7.7 | -0.9
224532.68$+$005855.8 | 11.0 |  $\cdots$ | 8.6 | -1.6
225222.52$+$010658.0 | 10.8 | 258.57 | 8.4 | -2.3
225222.65$+$010700.6 | 9.4 | 214.67 | 6.9 | -1.8
225817.73$+$003007.7 | 11.3 |  $\cdots$ | 9.1 | -3.7
225817.97$+$003011.5 | 9.8 | 450.88 | 7.3 | -1.3
230010.18$-$000531.7 | 11.3 | 318.78* | 9.1 | -3.6
230010.24$-$000534.0 | 11.4 | 347.85* | 9.2 | -3.6
230342.74$-$011712.7 | 11.5 | 283.87 | 9.3 | -3.1
230342.91$-$011711.9 | 11.5 | 234.26 | 9.3 | -2.9
231953.31$+$003816.7 | 11.9 |  $\cdots$ | 9.7 |  $\cdots$
231953.43$+$003813.4 | 12.0 |  $\cdots$ | 9.8 |  $\cdots$

Note. — Every two rows is a pair, and sources are sorted in ascending R.A. (1)
J2000 coordinate of the optical source; (2) stellar mass measured from
modeling the optical spectrum with simple stellar populations; (3) intrinsic
stellar velocity dispersion of the stellar populations, where the higher
resolution SDSS spectra are marked with *; (4) black hole mass inferred from
the $M_{\bullet}-M_{\rm bulge}$ relation of Kormendy & Ho (2013), which has an
intrinsic scatter of $\sim$0.29 dex;. (5) Eddington ratio, $\lambda_{\rm
Edd}=(L_{\rm rad}+L_{\rm mech})/L_{\rm Edd}$.

Table 7AGN Classifications of the Kinematic Pair Sample

Optical Designation | BPT1 | BPT2 | BPT3 | Radio | Radio | MIR | Mode
---|---|---|---|---|---|---|---
J2000 | class | class | class | Excess | Loud | color |
(1) | (2) | (3) | (4) | (5) | (6) | (7) | (8)
005113.93$+$002047.2 | Comp | SF | Seyfert | ✓ | ✗ | ✓ | L
005114.11$+$002049.4 | Comp | SF | SF | ✓ | ✗ | ✓ | L
013412.78$-$010729.5 | LINER | LINER |  $\cdots$ | ✓ | ✗ | ✗ | L
013412.84$-$010725.0 | Comp | LINER | LINER | ✓ | ✓ | ✗ | L
014203.08$-$000150.3 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | L
014203.24$-$000151.0 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | L
014715.86$-$000819.5 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | L
014715.94$-$000817.4 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✗ | ✗ | H
014928.38$-$001446.0 | SF | Seyfert | Seyfert | ✓ | ✗ | ✗ | H
014928.41$-$001447.8 | SF | SF | LINER | ✓ | ✗ | ✗ | H
015253.79$-$001005.6 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✓ | ✗ | ✗ | L
015253.92$-$001004.3 | LINER | LINER | LINER | ✓ | ✗ | ✗ | L
020301.40$-$003341.7 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | H
020301.43$-$003339.3 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | H
021021.00$-$005124.2 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✓ | ✓ | ✗ | L
021021.13$-$005126.7 | Comp | LINER | Seyfert | ✓ | ✓ | ✗ | H
220634.97$+$000327.6 | Comp | LINER | LINER | ✓ | ✗ | ✗ | L
220635.08$+$000323.3 | LINER | LINER | LINER | ✓ | ✗ | ✗ | L
221142.42$+$002731.2 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✓ | L
221142.51$+$002728.5 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✓ | H
222051.55$+$005815.5 | LINER | LINER |  $\cdots$ | ✓ | ✓ | ✗ | L
222051.70$+$005816.7 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✓ | ✓ | ✗ | L
222907.53$-$000411.1 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | H
222907.77$-$000410.9 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✗ | ✗ | ✗ | L
223222.41$+$001226.3 | Comp | SF | Seyfert | ✓ | ✗ | ✓ | L
223222.60$+$001224.7 | Comp | SF | SF | ✓ | ✓ | ✓ | L
223546.28$+$000358.8 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✗ | ✗ |  $\cdots$
223546.40$+$000358.7 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✗ | ✗ | ✗ |  $\cdots$
224426.44$+$001051.3 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | L
224426.61$+$001051.9 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | H
224532.53$+$005857.9 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✓ | H
224532.68$+$005855.8 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✗ | ✓ | H
225222.52$+$010658.0 | LINER | LINER | LINER | ✓ | ✗ | ✗ | L
225222.65$+$010700.6 | SF | SF | SF | ✓ | ✗ | ✗ | L
225817.73$+$003007.7 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✗ | ✓ | ✓ | L
225817.97$+$003011.5 | SF | SF | SF | ✓ | ✓ | ✓ | L
230010.18$-$000531.7 | LINER | LINER |  $\cdots$ | ✓ | ✓ | ✗ | L
230010.24$-$000534.0 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✗ | ✗ | L
230342.74$-$011712.7 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | L
230342.91$-$011711.9 |  $\cdots$ |  $\cdots$ |  $\cdots$ | $\heartsuit$ | ✓ | ✗ | L
231953.31$+$003816.7 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✗ | ✗ | ✗ |  $\cdots$
231953.43$+$003813.4 |  $\cdots$ |  $\cdots$ |  $\cdots$ | ✗ | ✗ | ✗ |  $\cdots$

Note. — Every two rows is a pair, and sources are sorted in ascending R.A. (1)
J2000 coordinate of the optical source; (2) $-$ (4) BPT diagnostic
classifications; (5) Radio excess determined via H$\alpha$ SFR ($\heartsuit$
denotes radio excess via Balmer decrement); (6) Radio loudness determined via
$L_{\rm[OIII]}$; (7) mid-IR color AGN status based on criteria of Satyapal et
al. (2017); (8) mode of AGN activity (HERG/LERG) based on criteria of Best &
Heckman (2012).


Further author information: (Send correspondence to Brian McReynolds)
Brian McReynolds: E-mail<EMAIL_ADDRESS>Telephone: +41 77
947 6780

# Re-interpreting the Step-Response Probability Curve to Extract Fundamental
Physical Parameters of Event-Based Vision Sensors

Brian J. McReynolds  Rui Graca  Institute of Neuroinformatics, UZH/ETH Zurich,
Winterthurerstrasse 190, 8057 Zürich, Switzerland  Lucas Kulesza  Air Force
Research Laboratory, Space Vehicles Directorate, Albuquerque, NM, 87117 USA
Peter McMahon-Crabtree  Air Force Research Laboratory, Space Vehicles
Directorate, Albuquerque, NM, 87117 USA

###### Abstract

Biologically inspired event-based vision sensors (EVS) are growing in
popularity due to performance benefits including ultra-low power consumption,
high dynamic range, data sparsity, and fast temporal response. They
efficiently encode dynamic information from a visual scene through pixels that
respond autonomously and asynchronously when the per-pixel illumination level
changes by a user-selectable contrast threshold ratio, $\theta$. Due to their
unique sensing paradigm and complex analog pixel circuitry, characterizing
Event-based Vision Sensor (EVS) is non-trivial. The step-response probability
curve (S-curve) is a key measurement technique that has emerged as the
standard for measuring $\theta$. Though the general concept is
straightforward, obtaining accurate results requires a thorough understanding
of pixel circuitry and non-idealities to correctly obtain and interpret
results. Furthermore, the precise measurement procedure has not been
standardized across the field, and resulting parameter estimates depend
strongly on methodology, measurement conditions, and biasing – which are not
generally discussed. In this work, we detail the method for generating
accurate S-curves by applying an appropriate stimulus and sensor configuration
to decouple 2nd-order effects from the parameter being studied. We use an EVS
pixel simulation to demonstrate how noise and other physical constraints can
lead to error in the measurement, and develop two techniques that are robust
enough to obtain accurate estimates. We then apply best practices derived from
our simulation to generate S-curves for the latest generation Sony IMX636 and
interpret the resulting family of curves to correct the apparent anomalous
result of previous reports suggesting that $\theta$ changes with illumination.
Further, we demonstrate that with correct interpretation, fundamental physical
parameters such as dark current and RMS noise can be accurately inferred from
a collection of S-curves, leading to more accurate parameterization for high-
fidelity EVS simulations.

###### keywords:

Neuromorphic, event-based vision sensors, sensor characterization

##  1 INTRODUCTION

EVS employ a unique sensing paradigm inspired by biological vision. Their
design captures dynamic visual information with high temporal resolution, low
latency, wide dynamic range and limited power consumption, making them ideal
for many diverse applications. These benefits have cemented EVS as an
important tool in computer vision and robotics [1]. In many such application
areas, the goal is simply detecting the spatio-temporal location of an object
or obstacle, and precise knowledge of EVS operating parameters is of lesser
importance. However, interest is growing in applying EVS to scientific
applications where more precise interpretation of EVS output is needed. The
most important EVS performance parameter to characterize is contrast
threshold, $\theta$. Fundamentally, EVS operate on the principle of change
detection–when a pixel senses a change its incident log-illumination level, it
reports either an ON or OFF event, indicating respectively an increase or
decrease in brightness. The change in log-illumination needed to trigger an
event is the threshold. ON and OFF thresholds ($\theta_{\text{on,off}}$) are
independent and able to be adjusted by the user to increase or decrease
sensitivity, depending on sensing requirements. Various measurement techniques
have been proposed to estimate $\theta$, with the most common being the event
probability curve (S-curve) first introduced in [2]. The method consists of
recording pixel response to periodic illumination pulses to a constant
amplitude stimulus, then incrementally increasing the amplitude and
calculating the response probability at each. This results in a curve that
starts at zero and approaches 1 as the pulse amplitude reaches the threshold,
and has emerged as the gold standard for reporting $\theta$ among new EVS
variants [3, 4, 5]. Despite its popularity, the S-curve results in different
estimates of $\theta$ at different brightness levels and bias settings,
leading to uncertainty in the measurement.

In this paper, we revisit the event probability S-curve with a holistic view
of EVS pixel operation and performance to examine the anomalous result that
$\theta$ changes with illumination as reported in previous studies. We first
use an EVS pixel simulation to systematically study how measured S-curves
relate to the pixel’s true threshold(s). Starting with an ideal pixel model,
we incrementally add non-ideal behaviors such as noise, finite bandwidth, and
mismatch to show how measured S-curves are influenced. At each step, we
describe how EVS settings can be adjusted to reduce the impact of non-
idealities, resulting in a prescribed measurement technique that is robust to
noise and mismatch and a new interpretation of the S-curve that allows more
accurate characterization of the contrast threshold. We demonstrate this
method with the latest generation Sony IMX636 EVS. Additionally, we use this
new knowledge to re-interpret measurement results for the Sony IMX636 EVS, and
show that by properly interpreting reported S-curves, fundamental parameters
such as dark current and noise can be inferred to facilitate more accurate
simulations.

##  2 EVS DESCRIPTION

Each EVS pixel operates autonomously and asynchronously, reporting ‘events’
when the log-illumination level changes by a user defined threshold. To
achieve this, each pixel contains a photodiode and analog circuitry to
implement the change detection operation. Fig. 1 shows the standard pixel
schematic introduced in [6] and still used with minor modifications in
commercial EVS designs. The photoreceptor circuit logarithmically compresses
the photocurrent to output $V_{\text{pr}}$. A source-follower buffer decouples
the photoreceptor output from downstream circuit components and low-pass
filters the signal to remove high frequency noise components. The signal is
amplified by the well matched capacitor ratio
$\frac{C_{\text{1}}}{C_{\text{2}}}$, and this signal is compared to the
pixel’s previous memorized brightness level. Independent ON and OFF
comparators trigger if the signal change exceeds $\theta_{\text{on}}$ or
$\theta_{\text{off}}$ respectively, and an ‘event’ is recorded. After each
event, the pixel is held in reset for a finite, adjustable period of time
known as the refractory period. After the refractory period, the pixel
memorizes a new brightness level which is used as a reference for reporting
subsequent events. Each event is read out by peripheral circuitry, and EVS
output consists of a list of events, each containing a pixel address (x,y),
timestamp (t), and a binary polarity output (p) indicating whether the event
is an ON or OFF event.

Figure 1:  Typical EVS pixel circuit schematic and description [7]. The active
logarithmic photoreceptor front-end (A-B) drives a cap-feedback change
amplifier (C) with output $V_{diff}$. When $V_{diff}$ deviates by either an ON
or OFF threshold, comparators (D) report an event, and after a finite
refractory period (E) the change amplifier is reset. F: After each reset, the
pixel again responds to signal changes around a new reference level.

##  3 Step Response Probability Curve

The concept of using a step response probability curve (S-curve) to measure
thresholds is simple. A group of pixels is exposed to a periodic stimulus of
known amplitude, and event probability is calculated as the number of events
recorded divided by the number of stimulus pulses. This procedure is repeated
for several different stimulus amplitudes and event probability is plotted as
a function of the log-contrast of the stimulus,

$\ \text{log
contrast}=log\left(\frac{E_{\text{v,max}}}{E_{\text{v,min}}}\right)\,,$ (1)

where $E_{\text{v,max}}$ and $E_{\text{v,min}}$ are the on-chip illuminance
values of the visual stimulus. An example is shown in Fig. 2. In the ideal
case, this should result in a perfect step function–pulse amplitudes below the
threshold result in zero response probability, and pulse amplitudes above the
threshold result in 100% response probability. Posch and Matolin were the
first to propose this method and fit a Gaussian Cumulative Distribution
Function (CDF) to the resulting curve [2]. Any deviation from an ideal step
response is attributed to noise, and as a result the 50% probability intercept
is selected as the Nominal Contrast Threshold (NCT). This general measurement
procedure and interpretation is broadly accepted throughout the EVS community;
however, McReynolds et al. studied this method using node voltages from a
single EVS test pixel and proposed that the true contrast threshold is near
the 100% response probability intercept [8]. The nature of the stimulus is
another important consideration. [2] describes a more complicated stimulus
consisting of a reset pulse preceeding each test pulse in order to ensure the
pixel reference resets to a desired baseline level before each test pulse, but
specific details of suggested reset pulse amplitude and duration are not
discussed. It is logical that this procedure should produce more
consistent/accurate results, but the amplitude of the reset pulse and duration
of both reset and test pulses must be carefully selected. In our analysis, we
will examine pixel response both with and without a reset pulse.

Figure 2:  Theoretical event probability curves for $\theta$ = 0.3. In the
ideal, noise-free case (black), the pixel response rises from 0 to 1 when the
log contrast (amplitude) of the stimulus equals the contrast threshold. Due to
noise, the response deviates from the ideal case, and more noise (blue curve)
results in a larger deviation from ideal (shallowest slope). NCT is defined as
the 50% event probability intercept, and is broadly accepted as the metric to
characterize EVS contrast threshold, $\theta$. In this paper, we show that
this interpretation does not result in an accurate measurement of $\theta$.

##  4 Experimental Method

To assess the accuracy of the step response probability curve, we devised a
test based on a pixel simulation similar to v2e [9]. The key benefits of using
a simulation are: 1) ground truth is known with 100% accuracy, and 2) multiple
stimulus waveforms and bias parameter combinations can be rapidly tested to
develop best practices and assess robustness of each. Our simulation takes an
input signal, compresses it logarithmically, adds Gaussian white noise, low
pass filters the response, and implements event generation logic by comparing
each time sample to the pixel’s memorized reference level. If the sample
deviates from the reference level by either the ON or OFF threshold, an event
is recorded. We also include a refractory period, which is necessary to
capture realistic pixel behavior. After each event, the simulation skips the
appropriate number of samples corresponding to the length of the refractory
period before resetting its reference. After reset, the event generation logic
recommences. The low-pass filter is implemented by Euler-stepping the
discretized differential equation corresponding to a first-order low-pass
filter,

$\ V_{\text{out}}[n]=V_{\text{out}}[n-1]+\frac{\Delta
t}{\tau}\left(V_{\text{in}}[n]-V_{\text{out}}[n-1]\right)\,.$ (2)

In this case, $\tau=\frac{1}{2\pi f_{\text{3dB}}}$, where $f_{\text{3dB}}$ is
the low-pass corner frequency. For the study, we used a time resolution
($\Delta t$) of 10$\mu$s, which is still many times faster than the highest
low-pass bandwidth tested (2kHz). Appropriate values of noise power were
determined by recording thirty separate 100 second simulations with a DC input
signal. Noise power was then fine-tuned to obtain realistic target background
activity rates, measured as events per pixel per second (typically reported in
Hz). We selected $f_{\text{3dB}}$ values of 2kHz, 200Hz, and 50Hz, within the
range of potential source-follower bandwidth settings, and for each, found
noise power levels resulting in average background activity rates 0.5 Hz and
0.02Hz.

##  5 Results

In this section, we examine the S-curve response to two different stimulus
patterns: 1) a simple periodic square wave with 50% duty cycle, and 2) a Reset
Pulse-Test Pulse (RP-TP) sequence, which is a series of alternating
rectangular reset pulses and test pulses. For the periodic square wave
stimulus, we selected a 5 Hz modulation frequency. We tested various
combinations of reset pulse amplitudes and frequencies, and eventually
selected a 50% contrast reset pulse amplitude, $\approx$ 55% higher than the
contrast threshold in log-signal units, with a 400 ms pulse duration, and 200
ms pulse duration for each test pulse. A sample reset-test pulse train is
shown in Fig. 3. In the case of the reset pulse method, only events that occur
within the temporal window of each test pulse are used to construct the
S-curve. For each S-curve measurement, we apply 30 different contrast
amplitudes ranging from 0.01 to 0.7 (1-70%) linear contrast. In the remainder
of this chapter, we systematically analyze both methods and assess how
accurately they characterize contrast threshold. We start with the ideal case,
then incrementally add in noise, finite bandwidth, and mismatch to demonstrate
how the simulated S-curve measurement evolves. Where appropriate, we
demonstrate biasing techniques that can be used to manage non-ideal behaviors
and restore the integrity of the measurement. Finally, we compare results for
the two stimuli, prescribe a complete methodology for obtaining accurate
threshold measurements, and establish limits on acceptable noise levels over
which accurate results can be attained. For all measurements, we set the ON
and OFF thresholds, $\theta$, at 0.3 unless otherwise noted

Figure 3:  Sample reset pulse - test pulse waveform for testing OFF threshold.
A 400 ms duration, 50% contrast reset pulse precedes each test pulse. OFF
events that occur during each test pulse are summed to determine event
probability at a particular contrast level (test-pulse amplitude).

###  5.1 Ideal Case

As shown in Fig. 4, the noise-free case results in a perfect step response as
predicted. The measurement error depends only on the resolution of the
contrast steps (x-axis), which could be improved by using smaller increments
on the signal amplitude between measurements. Both ON (green) and OFF (red)
curves cross the 50% probability threshold within 1% of the true contrast
threshold and there is no discernible difference between the RP-TP and Square
Wave (SW) methods.

Figure 4:  Noise and mismatch free simulation results for $\theta=0.3$.
’Ideal’ conditions result in perfect step response at $\theta$ as predicted.

###  5.2 Adding Noise

Fig. 5 demonstrates how the S-curve measurement evolves in the presence of
noise. Simulated measurements are shown for 0.5 Hz and 0.02 Hz background
activity rates for each stimulus waveform. Even with very low background
activity rates, NCT does not provide a good estimate of $\theta$, and higher
noise rates shift NCT further to the left. In many applications, high
sensitivity (low $\theta$) is desirable. This highlights a key shortcoming of
the NCT metric, as it results in an overly optimistic estimation of the actual
threshold levels, and can give the false impression of improved contrast
sensitivity in the presence of noise. Our result suggests that selecting the
50% event probability intercept is not a good estimate of $\theta$, but still
has some utility as a practical metric. On the other hand, for an adequately
low noise bandwidth (e.g. 50Hz), we see the curves converge to 100% event
probability near the true value of $\theta$, supporting the interpretation
proposed by McReynolds et al. [8] Noise bandwidth is optimally controlled by
tuning the source-follower bias, although, if accessible in the EVS model
being tested, it can also be adjusted using the photoreceptor bias[10].

To further understand why the Gaussian CDF does not accurately characterize a
single pixel’s event probability curve, we consider a noisy square wave
signal. In general, the frequency of noise components is much faster than the
frequency of the test stimulus, effectively broadening the signal. As a
result, noise can only increase the probability that a signal change smaller
than $\theta$ results in an event, but will never decrease the probability of
an event for a signal change larger than $\theta$. Response curves generated
for RP-TP and SW stimuli have notable differences; however, there is no clear
benefit of one over the other for most bandwidth/background activity
combinations.

Figure 5:  Simulated S-curves with noise for $\theta=0.3$. Even for low
background activity rates, noise shifts the NCT measurement (50 % event
probability intercept) to lower contrast than the true threshold level.
However, if noise bandwidth is limited (e.g. 50 Hz) by adjusting biases
(square datapoints), the 100% event probability intercept is a better
approximation for $\theta$.

###  5.3 Mismatch

In Sec. 5.2 we demonstrated that the 100% intercept of the event probability
curve provides an improved estimate of $\theta$, provided that noise bandwidth
and background activity rates are properly managed. Next, we explore how
measured S-curves are influenced by threshold mismatch. Because ON and OFF
thresholds are determined by independent comparator pathways in the pixel
circuit, they are not guaranteed (or likely) to be equal in a particular
pixel. Due to device mismatch, typical EVS pixels have been shown to exhibit a
1$\sigma$ threshold mismatch of $\approx$ 0.02-0.03 log-contrast. To examine
the influence of mismatch, we simulated the 50Hz bandwidth settings from Sec.
5.2 and applied a mismatch of 0.03. For both ON and OFF threshold measurement
simulations, we examined behavior both when the opposite threshold is higher
and lower than the true threshold of the polarity being measured by 1$\sigma$,
and the results are shown in Fig. 6.

Observing the 100% probability intercept, we see the approximation now has
shifted to the right of $\theta$, leading to another source of uncertainty in
the accuracy of the S-curve measurement. To understand the origin of this
behavior, we examine one period of a simulated waveform entering the
comparator logic stage. Fig. 7 demonstrates how the pixel reset influences
whether or not the next event will occur, highlighting another critical
consideration for collecting accurate S-curve measurements. On the left, the
refractory period is 100 $\mu$s, near the minimum possible setting. To
illustrate, we consider an extremely imbalanced case in which
$\theta_{\text{off}}\approx 0.2$ and $\theta_{\text{off}}\approx 0.3$, and the
signal amplitude is slightly larger than the ON threshold. Because the
refractory period is nearly instantaneous relative to the bandwidth-limited
signal response in the left plot, the new reset level corresponds to the
signal level that caused the OFF event rather than the trough of the waveform.
The new ON threshold is therefore much higher, meaning a larger signal
amplitude is required to consistently generate ON events, leading to the error
we observe in the S-curve measurements of Fig. 6. The plot to the right shows
how increasing the refractory period (e.g. 10 ms) forces the pixel to wait to
reset its reference, resulting in a reference close to the average signal
level at the base of the square wave. Under this condition, the event
probability provides a more accurate measurement of the signal amplitude
relative to the ON threshold.

Fig. 8 demonstrates how increasing the refractory period improves the S-curve
simulation as suggested by analysis of Fig. 7. Again, even with mismatch
applied, the S-curve approaches 100% response probability at the larger of
$\theta_{\text{on}}$/ $\theta_{\text{off}}$. Because mismatch is random,
attempting to bias for balanced ON/OFF thresholds leads to uncertainty in
which threshold is actually measured for a given pixel. However, since
$\theta_{\text{on}}$ and $\theta_{\text{off}}$ are able to be tuned
individually, deliberately setting one lower than the other ensures that the
S-curve’s 100% response will correspond to the less sensitive of the two
contrast thresholds when using the square wave stimulus. The RP-TP method does
not appear to be affected by mismatch as long as the refractory period is
sufficiently long. In Sec. 5.4 we explore the merits of this technique and
assess the robustness of the prescribed bias settings and measurement
procedures.

Figure 6:  Simulation S-curve measurement including mismatch. When threshold
mismatch is applied (i.e. $\theta_{\text{on}}=\theta_{\text{off}}+0.03$) the
simulated S-curve shifts to the right and approaches 100% response probability
at a log contrast level higher than the true threshold.  Figure 7:  One period
of simulated square wave. Pixel reset logic and refractory period explain the
discrepancy between measured and actual thresholds when threshold mismatch is
applied. Increasing the refractory period forces the reset to occur near the
base of the stimulus and improves accuracy of the measurement.  Figure 8:
Simulation of S-curve measurement with 10 ms refractory period for
$\theta=0.3$. Noise power resulting in 0.02Hz background activity and 0.03 log
contrast mismatch were applied. With both noise and mismatch, increasing the
refractory period results in improved accuracy of the S-curve measurement when
using the 100% response to approximate $\theta$.

###  5.4 Robustness of Prescribed Methods

After adding noise, finite response bandwidth, and threshold imbalance, we
have finally developed general biasing techniques to enable accurate S-curve
measurements. Bandwidth must be limited (by source follower or photoreceptor
bias) to limit noise, a long refractory period ensures the pixel reset level
corresponds to the signal baseline prior to each step (pulse), and
deliberately applying imbalanced thresholds (square wave method) defines
whether the measured 100% response probability corresponds to
$\theta_{\text{on}}$ or $\theta_{\text{off}}$. The stimulus light intensity
should be high enough for the photocurrent not to limit the pixel bandwidth,
and also to minimize noise in the pixel pass-band [7]. In EVS with light-
dependent leak rate[11, 7], however, setting the stimulus light intensity too
high results in excessive leakage currents which may significantly affect the
S-curve. Finally, in this section we examine how robust these techniques are
to both elevated noise rates and large threshold imbalances.

Fig. 9 shows that simulated S-curve measurements are relatively robust to
increased background activity rates due to higher noise power; however, the
square wave stimulus is less affected by the higher noise. Similarly, looking
at larger threshold imbalance levels (e.g. the tail ends of the distributions
when $\theta_{\text{on}}$/ $\theta_{\text{off}}$ are deliberately imbalanced)
we observe similar behavior. In both cases, the 100% probability intercept
shifts slightly left for S-curves produced by the RP-TP test sequence.
Although both methods achieve similar results under near ideal testing
conditions (low noise and mismatch), as these non-idealities approach their
upper limits, the square wave method outperforms the more complicated RP-TP
waveform. The RP-TP method could be improved/optimized by selecting a more
appropriate pre-pulse amplitude, duration, and temporal window for observing
the test pulse response. On the other hand, the square wave method produces
robust and accurate measurements without any fine-tuning.

Figure 9:  Simulated S-curve with prescribed biases and elevated background
activity rate of 0.5 Hz.  Figure 10:  Simulated S-curve measurement with
prescribed biases and larger threshold imbalance.

##  6 Camera Measurements

To validate our proposed methodology, we generated S-curve measurements for a
number of on-chip illumination levels for the Sony/Prophesee EVK4/IMX636, and
the result is shown in Fig. 11. A Region of Interest (ROI) of 100$\times$100
pixels limits readout bandwidth, preventing measurement uncertainty due to
readout saturation during each pulse. We used default bias settings other than
refractory period bias which was set to -20. The result was measured for the
EVK4 serial number 00050685 attached to a 35 mm f/4 lens. An OD 3.0 ND filter
was incorporated to facilitate measurement at the four lowest baseline light
levels indicated, mitigating issues with light source flicker and limited
radiometric resolution. To measure each S-curve, a train of 100 square pulses
was generated using programmatic control of our SpectralLED light source
(rise/fall time is about 1 ms). We fine tuned the timing and selected pulse
width of about 20 ms. We used the RP-TP stimulus method, and used a response
window of 40 ms around each test pulse to assess response probability. The
curves shown represent the ON event probability.

Figure 11:  Average measured S-curves from Sony IMX636 EVS using RP-TP
stimulus procedure with prescribed biasing at different baseline illumination
levels for 100$\times$100 pixel region. The 100% response probability
deterministically shifts to higher contrast near the dark current limit,
allowing for an estimation of dark current by Eq. 3.

###  6.1 Inferring Dark Current and Noise

Measuring EVS thresholds with improved fidelity also has implications for
inferring other important performance characteristics necessary to model and
predict sensor performance under extreme conditions such as near the dark
current limit. Pixel analysis does not predict threshold levels to depend on
illumination; however, for illumination levels near the dark current limit, a
larger change in photocurrent will be needed to trigger an event.
Specifically, an ON event occurs when

$\
log\left(\frac{I_{\text{pho2}}+I_{\text{dark}}}{I_{\text{pho1}}+I_{\text{dark}}}\right)>\theta_{on}\,,$
(3)

where the signal change is $I_{\text{pho2}}-I_{\text{pho1}}$ and
$I_{\text{dark}}$ is the dark current. Therefore, with an accurate measurement
of $\theta$, the dark current can be estimated by observing the signal change
required to generate an ON event near the dark current limit and rearranging
Eq. 3 to solve for $I_{\text{dark}}$.

As a result, we propose that a family of S-curves at varied background
illumination levels can be used to infer the sensor’s dark current. Observing
Fig. 11, the $\theta$ estimate moves progressively to larger log contrast
values as the illumination level drops below $\approx$ 300 mlx. At the highest
illumination levels (300 lx), when the dark current is much lower than the
induced photocurrent, we observe the 100% probability intercept at $\approx$
0.35 units log contrast. Observing the noise curve shown in Fig. 12, we see
that noise is elevated at both 3 and 30 lx, which explains why the 100%
probability intercept shifts to the left for those illumination levels.
$I_{\text{pho1}}$, which is the minimum (baseline) photocurrent for the ON
threshold measurement, is calculated from baseline on-chip illuminance
($E_{\text{v1}}$) as

$I_{\text{pho1}}=$1.12\text{\times}{10}^{16}\text{\,}$E_{\text{v}}\eta
qp^{2}\,,$ (4)

where $\eta$ is average spectral quantum efficiency, $p$ is pixel pitch,
$1.12\text{\times}{10}^{16}\text{\,}$ the scalar conversion between lumens and
photon flux, and q the electron charge. We used specs from a similar Combined
Metal-Oxide Semi-conductor (CMOS) image sensor by Sony to estimate quantum
efficiency. $I_{\text{pho2}}$ is a scalar multiple of $I_{\text{pho1}}$
representing the linear change in illumination. For example, observing the 76
mlx curve in Fig. 11, a log contrast of 0.8 is required to achieve $\approx$
100% response probability; therefore, we infer
$\log\left(\frac{I_{\text{pho2}}}{I_{\text{pho1}}}\right)=\log\left(\frac{E_{\text{v2}}}{E_{\text{v1}}}\right)=0.8$,
or rather, $I_{\text{pho2}}=I_{\text{pho1}}e^{0.8}$. Solving Eq. 3 for
$I_{\text{dark}}$ at the five lowest illumination levels (76 - 304 mlx) we
find close agreement across all curves with $I_{\text{dark}}=5.55\pm 0.28$ fA.

Though dark current is not reported for the IMX636, Low Light Cutoff (LLCO) is
defined as the minimum light level required for 50% of pixels to respond to a
linear, 100% contrast step. For an estimated $\theta_{\text{on}}=0.25$ units
log contrast, LLCO is reported as 80 mlx. Given the definition of LLCO, we
make use of Eq. 3 by setting $I_{\text{pho2}}=2I_{\text{pho1}}$ to check our
independent $I_{\text{dark}}$ estimate and find $5.62$ fA, increasing
confidence in the accuracy of our proposed method.

Figure 12:  Sample background activity curve as a function of on-chip
illumination for IMX636 with default bias configuration. Dashed lines are
steady-state background activity rates for ultra-low illumination levels
($<$1\text{\times}{10}^{-5}\text{\,}$$ lx on-chip.

One caveat of this estimation is the possibility that low pixel bandwidth near
the dark current limit attenuates the signal amplitude, leading to an
overestimation of the dark current. For example, if the pixel’s front-end,
low-pass corner frequency is $\approx$2Hz near the dark current limit, the
signal pulse will only reach 40% of its peak value within 40 ms of each test
pulse. The temporal response near the dark current has not been well
characterized for the IMX636, but reported latency at 5 lx (1 ms) suggests a
corner frequency that monotonically depends on illumination could be on the
order of single Hz near the dark current limit. Further tests are required to
characterize and isolate the effects of pixel bandwidth to improve the dark
current estimation.

Another important benefit of correct interpretation and improved methodology
of the S-curve measurement is the ability to devise realistic simulations
including physical noise parameters. Fig. 12 includes background activity
rates as a function of on-chip illuminance. Now that we have confirmed that
the threshold itself does not vary with illumination, we can correctly infer
that the change in background activity is entirely dependent on the noise
profile at a particular illumination level. A brute force approach would be to
simulate background activity at various levels of additive noise power to map
noise power to illumination, but this ignores the key ingredient of noise
bandwidth. A more thorough analysis should also account for the illumination
dependent pixel bandwidth; however, that level of analysis is beyond the scope
of this paper. Regardless, accurate threshold measurements are the crucial
first step for improved modeling of pixel behavior.

##  7 Conclusion

Contrast threshold, $\theta$, is the most important parameter for defining EVS
response, and the pixel response probability curve, or S-curve is an key
method used to estimate $\theta$. In this work, we explored two methods to
obtain S-curves and analyzed the resulting curves. Using a pixel simulation,
we showed that the commonly accepted interpretation of the S-curve does not
actually approximate $\theta$ when subject to real world physical constraints
such as noise and mismatch. Our simulation results validate a prior suggestion
that the true threshold is better approximated by observing where the S-curve
approaches 100% response probability [8]. Additionally, we show that the
approximation is subject to error unless the sensor is carefully biased to
manage noise bandwidth and threshold mismatch. As a result we prescribe a bias
configuration with low-bandwidth, long refractory period, and deliberately
mismatched thresholds where the higher setting corresponds to the threshold
being measured. We also compared methods using a simple square wave stimulus
and a more complicated RP-TP waveform. Our results show that both are viable;
however, the simple square wave demonstrated more robustness to noise and
mismatch. Finally, we demonstrated that accurate measurements along with our
improved interpretation of the S-curve makes it possible to estimate dark
current by observing how the response curves change near the dark current
limit. Our results are important for researchers exploring new applications of
EVS near the edge of their performance limits, and useful for developing high
fidelity EVS simulations for both new and existing commercial sensor models.

###### Acknowledgements.

Rui Graca is funded by the University of Zurich, Swiss National Science
Foundation project SCIDVS (Grant No. 185069).

## References

  * [1] Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A. J., Conradt, J., Daniilidis, K., and Scaramuzza, D., “Event-based vision: A survey,” IEEE Trans. Pattern Anal. Mach. Intell. 44, 154–180 (Jan. 2022).
  * [2] Posch, C. and Matolin, D., “Sensitivity and uniformity of a 0.18µm CMOS temporal contrast pixel array,” in [2011 IEEE International Symposium of Circuits and Systems (ISCAS) ], 1572–1575 (May 2011).
  * [3] Finateu, T., Niwa, A., Matolin, D., Tsuchimoto, K., Mascheroni, A., Reynaud, E., Mostafalu, P., Brady, F., Chotard, L., LeGoff, F., Takahashi, H., Wakabayashi, H., Oike, Y., and Posch, C., “A 1280$\times$720 Back-Illuminated stacked temporal contrast Event-Based vision sensor with 4.86µm pixels, 1.066GEPS readout, programmable Event-Rate controller and compressive Data-Formatting pipeline,” (2020).
  * [4] Niwa, A., Mochizuki, F., Berner, R., Maruyarma, T., Terano, T., Takamiya, K., Kimura, Y., Mizoguchi, K., Miyazaki, T., Kaizu, S., Takahashi, H., Suzuki, A., Brandli, C., Wakabayashi, H., and Oike, Y., “A 2.97$\mu$m-Pitch Event-Based vision sensor with shared pixel Front-End circuitry and Low-Noise intensity readout mode,” in [2023 IEEE International Solid- State Circuits Conference (ISSCC) ], 4–6 (Feb. 2023).
  * [5] Guo, M., Chen, S., Gao, Z., Yang, W., Bartkovjak, P., Qin, Q., Hu, X., Zhou, D., Uchiyama, M., Kudo, Y., Fukuoka, S., Xu, C., Ebihara, H., Wang, A., Jiang, P., Jiang, B., Mu, B., Chen, H., Yang, J., Dai, T. J., and Suess, A., “A 3-Wafer-Stacked hybrid 15MPixel CIS + 1 MPixel EVS with 4.6GEvent/s readout, In-Pixel TDC and On-Chip ISP and ESP function,” in [2023 IEEE International Solid-State Circuits Conference (ISSCC) ], 90–92, IEEE (Feb. 2023).
  * [6] Lichtsteiner, P., Posch, C., and Delbruck, T., “A 128 $\times$ 128 120 db 15 $\mu$s latency asynchronous temporal contrast vision sensor,” IEEE J. Solid-State Circuits 43(2), 566–576 (2008).
  * [7] Graça, R., McReynolds, B., and Delbruck, T., “Shining light on the DVS pixel: A tutorial and discussion about biasing and optimization,” arXiv preprint arXiv:2304.04706 (2023).
  * [8] McReynolds, B., Graca, R., and Delbruck, T., “Experimental methods to predict dynamic vision sensor event camera performance,” Opt. Eng. (2022).
  * [9] Hu, Y., Liu, S.-C., and Delbruck, T., “v2e: From video frames to realistic DVS events,” in [Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ], 1312–1321 (2021).
  * [10] Graca, R., McReynolds, B., and Delbruck, T., “Optimal biasing and physical limits of DVS event noise,” arXiv preprint arXiv:2304.04019 (2023).
  * [11] Nozaki, Y. and Delbruck, T., “Temperature and parasitic photocurrent effects in dynamic vision sensors,” IEEE Trans. Electron Devices 64, 3239–3245 (Aug. 2017).