# Flat-optics generation of broadband photon pairs with tunable polarization
entanglement

Vitaliy Sultanov  Max-Planck Institute for the Science of Light, Staudtstr. 2,
91058 Erlangen, Germany  Friedrich-Alexander Universität Erlangen-Nürnberg,
Staudtstr. 7, 91058 Erlangen, Germany<EMAIL_ADDRESS>Tomás
Santiago-Cruz  Max-Planck Institute for the Science of Light, Staudtstr. 2,
91058 Erlangen, Germany  Friedrich-Alexander Universität Erlangen-Nürnberg,
Staudtstr. 7, 91058 Erlangen, Germany  Maria V. Chekhova  Max-Planck Institute
for the Science of Light, Staudtstr. 2, 91058 Erlangen, Germany  Friedrich-
Alexander Universität Erlangen-Nürnberg, Staudtstr. 7, 91058 Erlangen, Germany

The concept of ‘flat optics’ is quickly conquering different fields of
photonics, but its implementation in quantum optics is still at infancy. In
particular, polarization entanglement, which is central to quantum photonics
due to the simplicity of polarization qubit encoding and control, is so far
not realized on flat platforms. Meanwhile, relaxed phase matching of flat
nonlinear optical sources enables enormous freedom in tailoring their
polarization properties. Here we use this freedom to generate photon pairs
with tunable polarization entanglement via spontaneous parametric down-
conversion (SPDC) in a $400$ nm GaP film. By changing the pump polarization,
we tune the polarization state of photon pairs from maximally entangled to
almost disentangled, which is impossible in a single bulk source of SPDC.
Polarization entanglement, in combination with the broadband frequency
spectrum, results in an ultranarrow (12 fs) Hong-Ou-Mandel effect and promises
extensions to hyperentanglement.

## Introduction

There is a pronounced tendency in photonics towards ‘flat’ optics [1],
involving ultrathin films, 2D materials, and metasurfaces. Linear and
nonlinear ‘flat’ optical elements are not only compact, integrable and
efficient; they are also multifunctional, promising to replace their bulk
counterparts [2]. Quantum optics is also on its way to ‘flat’ platforms [3,
4]. The latter are so far mainly used as hosts for single-photon sources [5]
and linear converters of quantum light, both in space [6, 7, 8] and in
polarization [6, 9]. Meanwhile, ‘flat’ quantum optics was so far unable to
deliver polarization-entangled photons, which are ubiquitous in quantum
technologies - both for quantum communications [10] and optical quantum
computation [11]. Although photon pairs have been generated in ultrathin films
through spontaneous four-wave mixing [12] and spontaneous parametric down-
conversion (SPDC) [13, 14, 15, 16], it is still bulk crystals that provide
polarization entanglement for flat platforms [7, 9]. This shortcoming hinders
the development of ‘flat’ quantum optics.

At the same time, ultrathin platforms provide unprecedented advantages in
engineering photon pairs. Being free from the phase matching constraints, they
can be fabricated of materials with especially large second-order
susceptibility $\hat{\chi}^{(2)}$, to boost the efficiency of pair generation
through SPDC. To produce polarization-entangled photons, now we use another
consequence of relaxed phase matching in ultrathin SPDC sources: they enable
several nonlinear interactions (type-0, type-I, type-II) simultaneously, the
only restriction being the structure of the $\hat{\chi}^{(2)}$ tensor.

Here, we generate photon pairs in an ultrathin layer of gallium phosphide
(GaP) whose $\hat{\chi}^{(2)}$ tensor enables SPDC of different polarization
types. We find that these pairs feature nontrivial polarization properties, in
particular polarization entanglement. Further, we show that by choosing the
polarization of the pump we can easily tune the polarization state of the
photon pairs, from maximally entangled to almost disentangled, and at the same
time maintain the purity of the state. Such performance is impossible with
regular linear polarization elements without introducing polarization-
dependent losses in the system [9]. Moreover, in combination with the nearly
unbounded frequency spectrum of photon pairs emitted from ultrathin materials
[14], polarization entanglement results in a remarkably narrow Hong-Ou-Mandel
dip or peak, depending on the experimental conditions, which we also
demonstrate in experiment. Together with the giant time-frequency [13] and
position-momentum [17] entanglement, the observed polarization entanglement
can be used to create hyperentangled photon pairs on flat platforms.

## Results

### Tunable polarization entanglement

We generate photon pairs via SPDC in a 400 nm-thick film of GaP, pumped by a
continuous-wave laser at $638$ nm (Fig. 1(a)), whose linear polarization can
be rotated by a half-wave plate (HWP). The GaP film is fabricated on the fused
silica substrate with its normal at $15^{\circ}$ to the [100] crystalline
direction, so that the pump field, in the general case, has projections on all
crystallographic axes $x,y,z$. The efficiency of SPDC scales as
$|\hat{\chi}^{(2)}\cdot\vec{e}_{s}\cdot\vec{e}_{i}\cdot\vec{e}_{p}|^{2}$,
where $\vec{e}_{s,i,p}$ are the unit polarization vectors of the signal,
idler, and pump radiation, respectively [18]. Because of the relaxed phase
matching condition, all nonzero elements of the $\hat{\chi}^{(2)}$ tensor of
GaP contribute to SPDC; in particular, there is emission of pairs for any pump
polarization (see the supporting information (SI) for details).

After collecting the SPDC radiation and filtering it from the pump, we send it
to the Hanbury Brown - Twiss setup [19] formed by non-polarizing beam splitter
BS and superconducting nanowire single-photon detectors (SNSPDs) in two output
ports A and B. A time tagger receives photo-detection pulses from both
detectors and builds the histogram of their time differences (Fig. 1(b)),
where the central peak clearly shows the simultaneity of photon arrivals in
both arms. The peak is observed for both vertically (V) and horizontally (H)
polarized pump (red and black points). Events forming this peak, called ‘two-
photon coincidences’, indicate the detections of photon pairs. The background
of the histogram shows accidental two-photon coincidences, and its level is
determined by the product of the two detectors’ count rates. The same level of
the background for the significantly different number of coincidences shows
that accidental coincidences are almost entirely caused by the
photoluminiscence of the GaP film. In all measurements described below, this
background is subtracted.

(a)

(b)

Figure 1: The experiment. (1(a)) A continuous-wave pump is focused by lens L1
into a thin film of GaP, and photon pairs are collected by lens L2 and
filtered from the pump by long-pass filters LP and band-pass filter BP. Non-
polarizing beam splitter BS sends the photons into arms A and B, each
containing a quarter-wave plate (QWP), a half-wave plate (HWP), a polarizer
(P), and a superconducting nanowire single-photon detector (SNSPD). A time
tagger builds a histogram of arrival time differences (1(b)). The histogram of
is plotted in the logarithmic scale for the case of the horizontally (black)
and vertically (red) polarized pump.

We see that, in contrast to SPDC in bulk crystals, photon pairs are emitted
for both V-polarized and H-polarized pump, although at a higher rate in the
latter case (Fig.1(b)). Even more strikingly, the two cases result in
different polarization states of the photon pairs.

A general polarization state of a photon pair in a single frequency and
wavevector mode can be written as a qutrit: a superposition of vertically,
horizontally, and orthogonally polarized photon pairs [20],

$|\Psi\rangle=C_{1}|2\rangle_{H}|0\rangle_{V}+C_{2}|1\rangle_{H}|1\rangle_{V}+C_{3}|0\rangle_{H}|2\rangle_{V},$
(1)

where $|N\rangle_{H,V}$ are the Fock states with $N$ photons polarized
horizontally or vertically, respectively, and $C_{i}$ are the complex
amplitudes satisfying $\sum_{i=1}^{3}|C_{i}|^{2}=1$.

For state (1), the degree of polarization entanglement is given by the
concurrence [21]

$C=\left|2C_{1}C_{3}-C_{2}^{2}\right|,$ (2)

taking the minimal value $C=0$ for a pair of co-polarized photons and the
maximal value $C=1$ for a pair of orthogonally polarized photons. Accordingly,
the Schmidt number $K=2/(2-C^{2})$, another measure of entanglement, takes in
these cases the values $1$ and $2$, respectively [21].

To analyse the polarization state of the pairs generated in our experiment, we
apply two-photon polarization tomography [22], which enables the
reconstruction of state (1) or, for a mixed state, of the corresponding
density matrix $\rho$. The procedure is similar to the Stokes measurements on
single photons [18], but involves coincidence registration. In two-photon
polarization tomography, we independently select different polarization states
in each beam splitter output port and measure the rate of coincidences. The
polarization is selected by a quarter-wave plate (QWP) and a half-wave plate
(HWP), followed by a polarizer (P), see Fig. 1(a). This combination, further
called a polarization analyzer, can filter photons in an arbitrary
polarization state. The combination of two such analyzers in two arms filters
an arbitrary two-photon polarization state. By measuring the rate of
coincidences for nine such combinations, we fully characterize the two-photon
polarization state before the beam splitter (see “Methods”).

We run this procedure for the states produced with H - and V - polarized pump,
using a $50$ nm FWHM band-pass filter centered at 1275 nm to suppress the
photoluminescence. The reconstructed density matrices of photon pairs
generated by the H - and V - polarized pump are shown in Fig. 2(a)-2(b), and
2(c)-2(d), respectively.

|
---|---
|

(a)

(b)

(c)

(d)

(e)

Figure 2: Two-photon state reconstruction and polarization entanglement.
Panels 2(a) (2(c)) and 2(b) (2(d)) show the real and imaginary parts of the
density matrix $\rho$ of the photon pairs generated by the H- (V-) polarized
pump. (2(e)) The number of coincidences versus the linear polarization angle
selected in arm A for the case of V-polarized pump (orthogonally polarized
photons). In arm B, we select the basic linearly polarized states: horizontal
(H), vertical (V), diagonal (D), anti-diagonal (A). The experimental points
are fitted with the theoretical dependence for polarization-entangled photons
(solid lines). The visibility reaches 96%, indicating a high degree of
polarization entanglement.

For the H - polarized pump, the two-photon polarization state is a
superposition of the horizontally and vertically polarized pairs with
different weights: $|C_{1}|^{2}=0.79\pm 0.02$, $|C_{2}|^{2}=0$, and
$|C_{3}|^{2}=0.21\pm 0.02$. Since $|C_{1}|^{2}\gg|C_{3}|^{2}$, two photons
within a pair are almost co-polarized horizontally. In contrast, pairs
generated with the V - polarized pump contain orthogonally (HV) polarized
photons: $|C_{1}|^{2}=0.03\pm 0.01$, $|C_{2}|^{2}=0.97\pm 0.06$, and
$|C_{3}|^{2}=0$. These results demonstrate a vast tunability of the two-photon
polarization state, which is not achievable with a single bulk SPDC source and
requires additional linear elements [23].

Moreover, by changing the polarization of the pump we also tune the degree of
polarization entanglement. In the case of an H - polarized pump, Eq. (2)
yields the concurrence $C=0.4\pm 0.03$ and the Schmidt number $K=1.09\pm
0.01$, i.e., photons within a pair are almost disentangled. For the HV photon
pairs, obtained from a V - polarized pump, the degree of entanglement is
maximal, $C=0.98\pm 0.02$ and $K=1.9\pm 0.1$. Notably, this tunability is
impossible with standard linear polarization elements, which maintain the
degree of entanglement and can modify it only through lossy transformations
[9].

### Bell inequality violation

We further demonstrate the polarization entanglement of photon pairs generated
with the V - polarized pump by measuring the number of coincidences for
different orientations of the analyzers. The analyzer in arm B is fixed at one
of the basic polarization states, horizontal (H), vertical (V), diagonal (D),
or anti-diagonal (A). The analyzer in arm A is then selecting linear
polarization, rotated from $0^{\circ}$ to $180^{\circ}$. The results are shown
in Fig. 2(e). A visibility of 96% witnesses strong polarization entanglement.

By splitting photon pairs generated with the V - polarized pump with the non-
polarizing beam splitter, we create the Bell state $\left|\Psi^{(+)}\right>$
(see SI) and use it to test the Bell inequality in the Clauser-Horne-Shimony-
Holt form [24],

$F\equiv\frac{1}{2}|\langle
ab+a^{\prime}b+ab^{\prime}-a^{\prime}b^{\prime}\rangle|\leq 1.$ (3)

We chose the binary variables $a,\;a^{\prime},\;b,\;b^{\prime}=\pm 1$ for
photons in arms A and B as their Stokes operators $S_{1}^{A}$, $-S_{2}^{A}$,
$\frac{1}{\sqrt{2}}(S_{1}^{B}+S_{2}^{B})$ and
$\frac{1}{\sqrt{2}}(S_{1}^{B}-S_{2}^{B})$, respectively [25]. By measuring
simultaneously the Stokes observables of photons A, B in the two output ports
of the beam splitter and calculating the left-hand side of inequality (3), we
obtain $F=1.36\pm 0.07$, which violates the Bell inequality by five standard
deviations.

Remarkably, generation of polarization-entangled photon pairs with high purity
does not require any additional elements erasing the distinguishability
between pairs of different polarization states, as is the case in bulk phase-
matched sources [26]. For orthogonally polarized photons, we obtain the value
of the purity $Tr(\rho^{2})=1.0\pm 0.1$. The indistinguishability stems from
the isotropic linear optical properties of GaP. A similar feature is expected
in other nonlinear crystals with zinc-blende structures like GaAs or InAs
[27].

---

(a)

(b)

(c)

Figure 3: Broadband polarization entanglement. (3(a)) Calculated spectrum of
the photon pairs, with the width mainly limited by the Fabry-Perot effect in
the film. (3(b)) The delay line for the HOM experiment consists of 4 calcite
plates, 5 mm thick each. The birefringence of calcite introduces a delay
between orthogonally polarized photons, which is varied by tilting the plates
around their optic axes (OA). (3(c)) The HOM dip (blue) and peak (red),
measured (points) and calculated based on the Fourier transform of the
spectrum in panel (3(a)) (dashed lines). Solid lines show the Gaussian fit of
the experimental data. The experimental width of the dip (peak) is $15\pm 2$
fs ($12\pm 2$ fs), which is somewhat broader than that predicted by the
calculated spectrum ($10$ fs).

### Hong-Ou-Mandel effect

As we see in Fig. 2(e), H-V photon pairs generated by the V - polarized pump
result in no coincidences if the polarization analyzers select the D-A
polarization states. This is the polarization version of the Hong-Ou-Mandel
(HOM) effect [28, 29] and another manifestation of polarization entanglement.
Destructive quantum interference leads to the absence of coincidences when the
D-A polarization state is detected. In contrast, when the analyzers are both
oriented diagonally (D-D) or anti-diagonally (A-A), we observe the maximal
rate of coincidences due to the constructive interference. This behavior is
similar to the ‘standard’ version of the HOM effect: two photons arriving from
different input ports of a beam splitter (here, from orthogonal polarization
modes H,V) are both directed into one of the output ports (here, into
polarization mode D or A).

By introducing a time delay between the photons of a pair, one observes a dip
in the case of destructive interference and a peak in the case of constructive
interference. The width of the dip or peak is determined by the spectrum of
the photon pairs [30]. In our case, the spectrum is ultrabroad due to the tiny
thickness of the sample. Figure 3(a) shows the calculated spectrum, with the
etalon effect taken into account [31, 32]. Despite the reduction due to the
etalon effect [14], the spectral width is still as large as $50$ THz, which
should lead to the HOM dip and peak of width about $10$ fs (dashed lines in
Fig. 3(c)).

In the experiment, we introduce a variable time delay between orthogonally
(H-V) polarized photons before the non-polarizing beam splitter by means of
four 5 mm birefringent calcite plates (Fig. 3(b), Methods).

The results of the HOM experiment are shown in Fig. 3(c). The blue (red)
triangles show the number of coincidences versus the delay between the photons
for the D-A (D-D) selected polarization state. The solid curves are
theoretical dependences calculated using the theoretical spectrum in Fig.
3(a). As expected, we see a dip of coincidences at zero delay when the
analyzers select orthogonal polarization states, D-A, and a peak when they
select the same polarization states, D-D. At large time delays, quantum
interference disappears because single-photon wavepackets do not overlap in
time. The widths of the dip and the peak are $15\pm 2$ fs and $12\pm 2$ fs,
respectively, which is somewhat larger than the width predicted from the
spectrum in Fig. 3(a) ($10$ fs). This is because the two-photon spectrum is
additionally narrowed by the spectrally dependent efficiency of the detectors.

## Discussion

We show that the relaxed phase matching for SPDC in an ultrathin nonlinear
film leads to the unprecedented polarization tunability of the produced photon
pairs. Taking advantage of the $\hat{\chi}^{(2)}$ tensor form, by simply
adjusting the pump polarization, we drastically tune the two-photon
polarization state and change it from maximally polarization-entangled to
almost disentangled. This tunability provides enormous freedom in photon pair
polarization engineering, impossible in conventional systems without
introducing losses. With such a system, one can change the degree of
polarization entanglement on demand in a very easy way, which is useful for
various applications of quantum technologies. In particular, two-photon states
with variable degree of entanglement are of interest for quantum key
distribution [33], teleportation [34] and Bell inequality tests [35].

To verify the high degree of polarization entanglement of one of the states,
we experimentally violate the Bell inequality by five standard deviations.
This is the first observation of polarization-entangled photons from ultrathin
nonlinear films. The polarization-entangled state is of high purity and its
preparation does not require linear optical polarization compensators,
unavoidable with bulk SPDC sources.

Polarization entanglement comes in combination with an ultrabroad frequency
spectrum, an extremely high degree of time/frequency entanglement [13] and
ultra-narrow time correlations. Accordingly, we observe very narrow ($12-15$
fs) polarization HOM dip and peak for polarization-entangled photon pairs.
This feature can be used for polarization-sensitive quantum-optical coherence
tomography [36]. We note that the width of the HOM dip and peak can be further
reduced by improving the detection setup.

The combination of polarization with other degrees of freedom enables
achieving hyperentanglement, which allows superdense coding, considerably
increasing the information capacity of two-photon states [37]. So far, SPDC in
ultrathin films resulted in photon pairs with huge entanglement in frequency
[13] and momentum [17]. Now, we complement this set with the polarization
degree of freedom. Further extension can be done by considering entanglement
in orbital-angular momentum (OAM) of photons [38].

Finally, the same flexibility of polarization properties can be used in
metasurface sources of SPDC, where the pair production rate is enhanced due to
geometric resonances [39]. The polarization state of the photon pairs will
then depend not only on the form of the nonlinear tensor, but also on the
resonant field distribution in the meta-atoms.

## Methods

Sample. As a source of photon pairs, we use a $400$ nm film of GaP, which is a
semiconductor with a zinc-blende symmetry. The film is fabricated on a $4$
$\mu$m layer of SiO2, which, in turn, is deposited on a $150$ $\mu$m sapphire
substrate. The fabrication procedure is described in Ref. [40]. The non-zero
$\hat{\chi}^{(2)}$ components of GaP are
$\chi_{xyz}=\chi_{xzy}=\chi_{yzx}=\chi_{yxz}=\chi_{zyx}=\chi_{zxy}\approx 100$
pm/V [27, 40]. To exploit these components, the sample was grown with the
orientation as shown in the inset of Fig. 1(a).

Photon pair generation and detection. The pump (60 mW continuous-wave diode
laser centered at 638 nm) is focused into the sample with lens L1 (Fig. 1(a)),
covering NA=$0.05$. Another lens (L2, NA$=0.16$) collects the generated photon
pairs. We cut off the pump with four long-pass filters (LP) with the cut-on
wavelengths $850$ nm and $1000$ nm. For polarization measurements, we
additionally select frequency-degenerate photon pairs with a band-pass filter
(BP) centered at 1275 nm with 50 nm full width at half maximum. A non-
polarizing beam splitter (BS) splits photon pairs into arms A and B, each of
them containing a quarter-wave plate (QWP), a half-wave plate (HWP), a
polarizer (P), and a superconducting nanowire single-photon detector (SNSPD).

Two-photon polarization reconstruction. To reconstruct the single-mode two-
photon polarization state, we use polarization tomography of single-mode
photon pairs [22]. Since the density matrix of the two-photon polarization
state has a dimension of $n=3$, $n^{2}=9$ measurements (including one extra
measurement for normalization) of the coincidence count rate are required to
fully reconstruct the state and verify its purity. The full protocol of the
measurements for nine combinations of the analyzer settings in arms A,B can be
found in SI. In the more general case of photon pairs generated in two spatial
modes, the dimension of the reduced density matrix is 4, and, as a
consequence, 16 measurements are required for quantum tomography [23].

Time delays in Hong-Ou-Mandel effect. To introduce the time delay between
orthogonally polarized photons, we use four $5$ mm thick calcite crystal
plates (Fig. 3(b)), with the optic axes vertical for two and horizontal for
the other two plates. Tilting the plates around their optic axes changes their
effective thickness. Due to the birefringence of calcite, the optical path for
the horizontally and vertically polarized photons is different, which leads to
a delay between them depending on the tilt angle of a plate. The zero delay is
achieved when two orthogonally oriented crystal plates are tilted by the same
angle. To compensate for the transverse displacement of the beam, we put a
mirrored scheme with another two plates. By rotating the inner or the outer
pair of plates we are able to achieve either negative or positive delay
between two orthogonally polarized photons.

## Data availability

Any data that support the findings of this study are available from the
corresponding author upon reasonable request.

## Aknowledgements

We thank H. Zhang, A. P. Anthur and L. A. Krivitsky for providing the sample
and A. V. Rasputnyi for the help at the initial stage of the experiment. V.S.
and T.S.C. are part of the Max Planck School of Photonics supported by BMBF,
Max Planck Society, and Fraunhofer Society. This work was funded by the
Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID
429529648 – TRR 306 QuCoLiMa (“Quantum Cooperativity of Light and Matter”).

## Author contribution

V.S. built the experimental setup, carried out the experiments, analyzed and
interpreted the experimental data. T.S.C. significantly contributed to the
experiment and the writing of the manuscript. M. V. C. conceived the idea of
the experiment, supervised the project, and significantly revised the
manuscript.

## Competing interest

The authors declare no competing interests.

## References

  * [1] Yu, N., & Capasso, F. Flat optics with designer metasurfaces. Nat. Mater. 13, 139-150 (2014).
  * [2] Chen, W. T., & Capasso, F. Will flat optics appear in everyday life anytime soon? Appl. Phys. Lett. 118, 100503 (2021).
  * [3] Wang, Yu., Jöns, K. D., & Sun, Zh. Integrated photon-pair sources with nonlinear optics. Appl. Phys. Rev. 8, 011314 (2021).
  * [4] Solntsev, A. S., Agarwal, G. S., & Kivshar, Yu. S. Metasurfaces for quantum photonics. Nat. Photon. 15, 327-336 (2021).
  * [5] Toth, M., & Aharonovich, I. Single Photon Sources in Atomically Thin Materials. Annu. Rev. Phys. Chem. 70, 123-142 (2019).
  * [6] Wang, K., et al. Quantum metasurface for multiphoton interference and state reconstruction. Science 361, 1104 (2018).
  * [7] Li, L., et al. Metalens-array–based high-dimensional and multiphoton quantum source. Science 368, 1487 (2020).
  * [8] Li, Q., et al. A non-unitary metasurface enables continuous control of quantum photon–photon interactions from bosonic to fermionic. Nat. Photonics 15, 267–271 (2021).
  * [9] Lung, Sh. et al. Complex-birefringent dielectric metasurfaces for arbitrary polarization-pair transformations. ACS Photonics 7 (11), 3015-3022 (2020).
  * [10] Xu, F., Ma, X., Zhang, Q., Lo, H., & Pan, J. Secure quantum key distribution with realistic devices. Rev. Mod. Phys. 92, 025002-025062 (2020).
  * [11] Zhong, H.-S., et al. Quantum computational advantage using photons. Science 370, 6523 (2020).
  * [12] Lee, K. F., et. al. Photon-Pair Generation with a 100 nm Thick Carbon Nanotube Film. Adv. Mater. 29, 1605978 (2017).
  * [13] Okoth, C., Cavanna, A., Santiago-Cruz, T. & Chekhova, M. V. Microscale Generation of Entangled Photons without Momentum Conservation. Phys. Rev. Lett. 123, 263602-263607 (2019).
  * [14] Santiago-Cruz, T., Sultanov, V., Zhang, H., Krivitsky, L. A., & Chekhova, M. V. Entangled photons from subwavelength nonlinear films. Opt. Lett. 46, 653-656 (2021).
  * [15] Hanh Duong, N. M., Maeder, A., Saerens, G., Kaufmann, F. & Grange, R. Non-phase-matched spontaneous parametric down-conversion from lithium niobate thin films. Conference on Lasers and Electro-Optics Europe & European Quantum Electronics Conference (CLEO/Europe-EQEC) (2021) DOI: 10.1109/CLEO/Europe-EQEC52157.2021.9542676
  * [16] Guo, Q., et al. Ultrathin quantum light source enabled by a nonlinear van der Waals crystal with vanishing interlayer-electronic-coupling. Preprint at https://arxiv.org/abs/2202.03831
  * [17] Okoth, C., et. al. Idealized Einstein-Podolsky-Rosen states from non-phase-matched parametric down-conversion. Phys. Rev. A 101, 011801-011806 (2020).
  * [18] Chekhova, Maria, & Banzer, Peter. Polarization of Light: In Classical, Quantum, and Nonlinear Optics. Berlin, Boston: De Gruyter (2021)
  * [19] Hanbury Brown, R., & Twiss, R. Q. Correlation between Photons in two Coherent Beams of Light. Nature 177, 27–29 (1956).
  * [20] Burlakov, A. V., Chekhova, M. V., Karabutova, O. A., Klyshko, D. N., & Kulik, S. P. Polarization state of a biphoton: Quantum ternary logic. Phys. Rev. A 60, R4209(R) (1999).
  * [21] Fedorov, M. V., & Miklin, N. I. Schmidt modes and entanglement. Contemporary Physics 55, 94-109 (2014).
  * [22] Burlakov, A. V., Krivitskii, L. A., Kulik, S. P., Maslennikov, G. A., & Chekhova, M. V. Measurement of qutrits. Optics and Spectroscopy, 94 (5), 684-690 (2003).
  * [23] White, A. G., James, D. F. V., Eberhard, Ph. H., & Kwiat, P. G. Nonmaximally entangled states: production, characterization, and utilization. Phys. Rev. Lett. 83, 3103 (1999).
  * [24] Clauser, J. F., Horne, M. A., Shimony, A., & Holt, R. A. Proposed experiment to test local hidden-variable theories. Phys. Rev. Lett. 23, 880-884 (1969).
  * [25] Klyshko, D. N. Basic quantum mechanical concepts from the operational viewpoint. Phys. Usp. 41, 885-922 (1998).
  * [26] Kwiat, P. G., et al. New high-intensity source of polarization-entangled photon pairs. Phys. Rev. Lett. 75, 4337 (1995).
  * [27] Boyd, R. W. Nonlinear optics. San Diego, CA: Academic press (2003)
  * [28] Hong, C. K., Ou, Z. Y., & Mandel, L. Measurement of subpicosecond time intervals between two photons by interference. Phys. Rev. Lett. 59, 2044 (1987).
  * [29] Rubin, M. H., Klyshko, D. N., Shih, Y. H., & Sergienko, A. V. Theory of two-photon entanglement in type-II optical parametric down-conversion. Phys. Rev. A 50, 5122 (1994).
  * [30] Burlakov, A. V., Chekhova, M. V., Karabutova, O. A., & Kulik, S. P. Collinear two-photon state with spectral properties of type-I and polarization properties of type-II spontaneous parametric down-conversion: Preparation and testing. Phys. Rev. A 64, 041803(R) (2001).
  * [31] Kitaeva, G.K., & Penin, A.N. Parametric frequency conversion in layered nonlinear media. J. Exp. Theor. Phys. 98, 272–286 (2004).
  * [32] Jeronimo-Moreno, Y., Rodriguez-Benavides, S. & U’Ren, A. B. Theory of cavity-enhanced spontaneous parametric downconversion. Laser Phys. 20, 1221–1233 (2010)
  * [33] Xue, P., Li, C. F., & Guo, G. C. Efficient quantum-key-distribution scheme with nonmaximally entangled states. Phys. Rev. A 64, 032305 (2001).
  * [34] Modławska, J., & Grudka, A. Nonmaximally entangled states can be better for multiple linear optical teleportation. Phys. Rev. Lett. 100, 110503 (2008).
  * [35] Giustina, M., et al. Significant-Loophole-Free Test of Bell’s Theorem with Entangled Photons. Phys. Rev. Lett., 115, 250401 (2015).
  * [36] Booth, M. C., Di Giuseppe, G., Saleh, B. E. A., Sergienko, A. V., & Teich, M. C. Polarization-sensitive quantum-optical coherence tomography. Phys. Rev. A 69, 043815 (2004).
  * [37] Graham, T. M., Bernstein, H. J., Wei, T. C., Junge, M., & Kwiat, P. G. Superdense teleportation using hyperentangled photons. Nature Comm., 6, 7185 (2015).
  * [38] Xu, Ch., et al. Manipulating the orbital-angular-momentum correlation of entangled two-photon states in three-dimensional nonlinear photonic crystals. Phys. Rev. A 104, 063716 (2021).
  * [39] Santiago-Cruz, T., Fedotova, A., et. al. Photon Pairs from Resonant Metasurfaces. Nano Lett. 21, 4423-4429 (2021).
  * [40] Anthur, A. P., et al. Second harmonic generation in gallium phosphide nano-waveguides. Optics Express, 29 (7), 10307-10320 (2021).


# How buildings change the fundamental allometry

Fabiano L. Ribeiro [0000-0002-2719-6061]<EMAIL_ADDRESS>Department of
Physics (DFI), Federal University of Lavras (UFLA), Lavras MG, Brazil  Peiran
Zhang<EMAIL_ADDRESS>School of Systems Science Beijing Jiaotong
University Beijing  Leibniz Institute of Ecological Urban and Regional
Development (IOER), Dresden, Germany  Liang Gao<EMAIL_ADDRESS>School
of Systems Science Beijing Jiaotong University Beijing  Diego Rybski
[0000-0001-6125-7705]<EMAIL_ADDRESS>Leibniz Institute of Ecological Urban
and Regional Development (IOER), Dresden, Germany  Complexity Science Hub
Vienna, Josefstädterstrasse 39, A-1090 Vienna, Austria

###### Abstract

We demonstrate that the original fundamental allometry alone cannot accurately
describe the relationship between urban area and population size. Instead,
building height is a third factor that interplays with area and population. To
illustrate this, we propose a straightforward model based on the idea that
_city area is the result of people’s desire to live close to one another while
also having sufficient living space_. This leads to a more general form of
fundamental allometry (relating area, population, _and_ building height). Our
argument is supported by empirical data from different countries.

## Introduction

_Urban scaling_ refers to an urban indicator that is related to city size in
terms of population, and in many cases, non-linear power-law association is
reported BettencourtLHKW2007 . In a recent review, mathematical models
explaining urban scaling have been presented and compared RibeiroR2023 . A
special form of urban scaling is the relation between city area $A$ and city
population $N$. We call it _fundamental allometry_ because both quantities
represent a measure of size, and relating population and area implicitly
brings density into play, which itself is an important quantity. In the form

$A\sim N^{\beta}\,,$ (1)

most studies report $\beta<1$ BattyF2011 ; BurgerOHWSLBCFHHKEE2022 , implying
higher population density in large cities. This finding is remarkable and even
obtained for ancient urban systems and societies OrtmanCSB2014 ;
HamiltonMWB2007 ; LoboBSO2019 . For a city resembling a hemisphere, assuming
the population is proportional to the volume ($N\sim r^{3}$) and assuming the
area of a circle ($A\sim r^{2}$), one gets $A\sim N^{2/3}$ NordbeckS1971 .
Certainly, cities are not hemispherical, and in particular, the vertical
dimension has a very different character compared to the horizontal ones.
Consequently, more elaborated derivations for the scaling exponent have been
proposed BettencourtLMA2013 ; LoufB2014 ; Deppman2024 . Here, we present a
simple derivation that, as we will see below, leads to the same results as
more complex models. In addition, our results suggest that the average
building height represents a third quantity that interplays with area and
population.

## The model

We begin by considering that the fundamental allometry emerges from an
equilibrium area $A^{*}(N)$; i.e., for a population count $N$, there is an
equilibrium area (indicated by the asterisk) and vice versa. Based on the
following considerations, we argue that this equilibrium results from two
forces.

  1. 1.

Urban dwellers want to be close to others and to facilities, infrastructure,
etc. According to this “ _force_ ”, a given population wants to concentrate on
a small city area, implying high density. In economics, this attraction would
be a “ _centripetal force_ ”. Here, we will refer to this force simply by “
_atractive force_ ”;

  2. 2.

Urban dwellers want to inhabit houses and apartments with large individual
living areas. According to this second “force”, a given population spreads out
in a large city area, implying low density. In economics, this repulsion would
be a “ _centrifugal force_ ”. We will refer to this force simply by “
_repulsive force_ ”.

Analogous to physics, this system of forces can be described by a potential
function $U(A)$, and then the resulting force, representing the intensity with
which the system attempts to restore equilibrium, can be expressed as
$F=-\frac{dU}{dA}$. Using this idea, we can consider that a city will be in
equilibrium when both forces add up to zero, which is the case where the
potential exhibits a minimum, i.e., when $F(A^{*})=-\frac{dU(A)}{dA}=0$
(necessary condition). We must find a mathematical expression for the
potential $U(A)$ to describe the system formally. Involving the two forces,
the potential must be the result of two other potentials, i.e.,
$U(A)=u_{1}(A)+u_{2}(A)$. The first, $u_{1}(A)$, describes the attractive
force and must be an increase function of the area, in the sense that its
minimum will be when the area is small. The second, $u_{2}(A)$, describes the
repulsive force and must be a decreased function of the area because its
minimum will be when the area is large. This idea is illustrated in Fig. 1.

### Attractive force: average distance between the individuals

Accounting for the urge of urban dwellers to be close to one another, we can
use the average distance $\bar{d}$ between locations/individuals in the city,
calculated as

$\bar{d}=\frac{2}{N(N-1)}\sum_{i=1}^{N}\sum_{j\neq i}r_{ij}\,.$ (2)

Here, $r_{ij}$ represents the distance between individuals $i$ and $j$ (more
precisely, the distance between their residences). The respective potential
$u_{1}(A)$ must increase with $\bar{d}$ while its minimum happens as $\bar{d}$
approaches zero, reflecting that dwellers would want to live as close as
possible to each other. It justifies considering $u_{1}\sim\bar{d}$.

In addition, for a fixed population, the average distance increases with the
area. In the recent work RibeiroLBR2024 , it is shown that when a set of
points is distributed in 2D space, forming a macroscopic object with a fractal
dimension $D_{f}$ and area $A$, then the average distance between the points
scales as $\bar{d}\sim A^{1/D_{f}}$ (the solution of the double sum in Eq. 2).
This is precisely the context here, and therefore, it is reasonable to
consider that $u_{1}(A)\sim\bar{d}\sim A^{1/D_{f}}$.

### Repulsive force: individual living area

For the second potential, $u_{2}(A)$, we can define the average floor area per
capita, say $\bar{a}$, which can be calculated by the total floor area of the
city divided by the population. That is, if $\bar{h}$ represents the average
building height, e.g., measured in number of floors, then

$\bar{a}\sim\frac{\bar{h}A}{N}\,.$ (3)

Here, the total floor area $\bar{h}A$ can also be interpreted as the average
building volume. In the particular case where $\bar{h}=1$ floor, the average
floor area per capita is the inverse of the population density (or area per
capita), i.e., $\bar{a}\sim A/N$. The respective potential must decrease with
$\bar{a}$ since its minimization occurs only when $\bar{a}$ approaches
infinity. This indicates that dwellers desire as much individual space as
possible, and then it is reasonable to consider $u_{2}\sim 1/\bar{a}$,
fulfilling the requirements.

Figure 1:  The potential function $U(A)$ \- curve in black - as a combination
of two other potentials by the form $U(A)=u_{1}(A)+u_{2}(A)$, for a fixed
population $N$. The potential $u_{1}(A)$ \- curve in blue - is responsible for
an attractive force (the blue arrows) that represents the desire of the people
to live close to each other – the reason for the cities to exist. The
potential $u_{2}(A)$ \- curve in red - is responsible for a repulsive force
(the red arrows) that represents the desire of the people to live in large
housing/living areas. The minimum of $U(A)$ (black curve) determines the
city’s equilibrium area $A^{*}$. The arrows in black, obtained by the equation
$F=-\frac{dU}{dA}$, represent the intensity and direction (the force) driving
the system back to equilibrium.

## Results

Combining both functions, we get the potential function

$U(A)=A^{1/D_{f}}+\frac{N}{\bar{h}A}\,.$ (4)

This function $U(A)$ depends on the area $A$, and its minimum corresponds to
the equilibrium area $A^{*}$ (see Fig 1), which we obtain from $dU/dA=0$,
yielding

$A^{*}\sim\left(\frac{N}{\bar{h}}\right)^{\frac{D_{f}}{D_{f}+1}}\quad\text{
and }A^{*}\sim N^{\frac{D_{f}}{D_{f}+1}}\text{ for constant }\bar{h}\,.$ (5)

This result allows three observations.

First, since the fractal dimension is positive, the scaling exponent is
smaller than one, $D_{f}/(D_{f}+1)<1$, consistent with the literature. Second,
it is equivalent to the expression by Bettencourt 2013 BettencourtLMA2013  who
– from _different_ arguments111 Bettencourt derives the fundamental allometry
scaling exponent by considering that the probability of encounters between
people decreases with the area. Additionally, he posits that salaries, which
depend on the probability of these encounters, must ensure individuals have
access to the entire city. – derives the _identical_ exponent
$\beta=\frac{D_{f}}{D_{f}+1}$. Formally, Eq. (5) is also compatible with the
result by Louf & Barthelemy LoufB2014  who obtain $\beta=\frac{2\mu}{2\mu+1}$,
where $\mu$ is an exponent reflecting the resilience of the transport network
to congestion RibeiroR2023 .

Third, Eq. (5) indicates that not only the population size is necessary to
infer the area of the city, as the previous models suggest, but also the
building height. In Fig. 2, we examine how the building height affects the
residuals (which are also known as _Adjusted Metropolitan Indicators_
BettencourtLSW2010 ). In this figure, the fundamental allometry is displayed,
where each dot represents one city based on the UCDB City Identification
Standards, and the size of the dots indicates the average building height. One
can see that cities consisting of taller buildings tend to be located below
the regression line, and cities with smaller average building heights are
above – supporting Eq. (5). The analysis presented in this figure shows that
the residuals typically decrease (and become negative) with increasing average
building height.

In this sense, if we redefine the residuals as
$R=\ln{(c(N/\bar{h})^{\beta})}-\ln{(A_{0}N^{\beta})}$ (difference between the
prediction of Eq. (5) and the original fundamental allometry Eq. (1)), where
$c$ and $A_{0}$ are constants, then
$\bar{h}=(A_{0}/c)^{-1/\beta}\text{e}^{-R/\beta}$. Expanding the exponential
function for small $R$ we obtain $R=-a\cdot\bar{h}+b$, where $a$ and $b$ are
constants that depend on $\beta$. That is, a decreasing linear function
between residual and average building height, as in the middle panels of Fig.
2. This result aligns with intuition – smaller city areas require taller
buildings to accommodate the same number of urban dwellers.

A particularly interesting case is the cities of Hong Kong and Sha Tin, both
in China. When considering only the original fundamental allometry, these
cities (UCDB) appear as outliers, positioned significantly below the
fundamental allometry fit curve (see figure 2 \- b and e). This is because
they have large populations within relatively small areas, which contradicts
the simple expectations of the original fundamental allometry. However, their
higher average building heights explain this discrepancy, allowing them to
accommodate more people in smaller areas. This situation aligns with the
results of the approach we are discussing here.

## Final remarks

In conclusion, the vertical growth of the cities must be considered when
relating area and population size. Not taking the building height variability
into account increases the noise of the fundamental allometry (or reduces the
correlations). Thus, the simple fundamental allometry Eq. (1) is just a kind
of first-order approximation. Understanding the complexity of urban scaling,
including the role of building height, as introduced in SchlapferLB2015 ;
MolineroT2021 , represents an ongoing challenge.

It is interesting to observe that if we take above’s argument $A\sim r^{2}$,
$N\sim r^{3}$, and introduce the fractal dimension (instead of $D=2$), i.e.
$A\sim r^{D_{f}}$ and $N\sim r^{D_{f}+1}$, then we get the same
$\beta=\frac{D_{f}}{D_{f}+1}$ only that the vertical dimension is not fractal.
More research is necessary to understand and describe the different
characteristics of vertical and horizontal dimensions in cities. The
equilibrium model we present here is very simple, and one can think of more
complete ones, possibly distinguishing and addressing vertical and horizontal
dimensions.

Acknowledgement: F. L. Ribeiro thanks CNPq (grant numbers 403139/2021-0 and
424686/2021-0) and Fapemig (grant number APQ-00829-21) for financial support.
P.Z. and L.G. are supported by the National Natural Science Foundation of
China (72288101 and 72242102). P.Z. acknowledges support from the program of
China Scholarship Council (No. 202307090047). D. Rybski acknowledges financial
support from German Research Foundation (DFG) for the projects UPon
(#451083179) and Gropius (#511568027).

a) b) c) d) e) f) g)

Figure 2:  Three Figures on top (a,b,c): Urban Area as a Function of
Population Size for Three Continental-Sized Countries: USA, China, and Brazil.
Each dot represents one city (UCDB), with the size of the dots indicating the
city’s average building height. The red line corresponds to the fit of the
original fundamental allometry (i.e., $A\sim N^{\beta}$). Visually, it is
apparent that cities with larger building heights tend to fall below the red
line. To systematically justify this observation, we examine the residuals of
the cities as a function of average building height, as presented in the three
figures in the middle (d,e,f). These residuals, which measure the deviation
from the predicted fundamental allometric relationship, exhibit a negative
slope in relation to average building height. This pattern holds not only for
these three countries but also for the majority of the 32 countries included
in the study, as shown in the histogram below (g). The negative slope of the
residuals indicates that cities with taller average building heights tend to
fall below the line predicted by the original fundamental allometric
relationship. Dataset: _GHSL Global Human Settlement Layer_ ghsl  and _World
Settlement Footprint_ wsf .

## References

  * (1) L. M. A. Bettencourt, J. Lobo, D. Helbing, C. Kühnert, and G. B. West.  Growth, innovation, scaling, and the pace of life in cities.  Proc. Natl. Acad. Sci. U. S. A., 104(17):7301–7306, 2007.
  * (2) F. L. Ribeiro and D. Rybski.  Mathematical models to explain the origin of urban scaling laws.  Phys. Rep., 1012:1–39, 2023.
  * (3) M. Batty and P. Ferguson.  Defining city size.  Env. Plan. B, 38(5):753–756, 2011.
  * (4) J. R. Burger, J. G. Okie, I. A. Hatton, V. P. Weinberger, M. Shrestha, K. J. Liedtke, T. Be, A. R. Cruz, X. Feng, C. Hinojo-Hinojo, A. S. M. G. Kibria, K. C. Ernst, and B. J. Enquist.  Global city densities: Re-examining urban scaling theory.  Front. in Conserv. Sci., 3:879934, 2022.
  * (5) S. G. Ortman, A. H. F. Cabaniss, J. O. Sturm, and L. M. A. Bettencourt.  The pre-history of urban scaling.  PLoS One, 9(2):10.1371/journal.pone.0087902, 2014.
  * (6) M. J. Hamilton, B. T. Milne, R. S. Walker, and J. H. Brown.  Nonlinear scaling of space use in human hunter–gatherers.  Proc. Natl. Acad. Sci. U. S. A., 104(11):4765–4769, 2007.
  * (7) J. Lobo, L. M. A. Bettencourt, M. E. Smith, and S. Ortman.  Settlement scaling theory: Bridging the study of ancient and contemporary urban systems.  Urban Stud., 57(4):731–747, 2019.
  * (8) S. Nordbeck.  Urban allometric growth.  Geogr. Ann. B, 53(1):54–67, 1971.
  * (9) L. M. A. Bettencourt.  The origins of scaling in cities.  Science, 340(6139):1438–1441, 2013.
  * (10) R. Louf and M. Barthelemy.  How congestion shapes cities: from mobility patterns to scaling.  Sci. Rep., 4(1):5561, 2014.
  * (11) A. Deppman, R. L. Fagundes, E. Megías, R. Pasechnik, F. L. Ribeiro, and C. Tsallis.  Dynamics of cities.  in prep., 2024.
  * (12) F. L. Ribeiro, Y. Li, S. Born, and D. Rybski.  Analytical solution for the long- and short-range every-pair-interactions system.  Chaos Soliton. Fract., 183:114771, 2024.
  * (13) L. M. A. Bettencourt, J. Lobo, D. Strumsky, and G. B. West.  Urban scaling and its deviations: Revealing the structure of wealth, innovation and crime across cities.  PLoS One, 5(11):e13541, 2010.
  * (14) M. Schläpfer, J. Lee, and L. M. A. Bettencourt.  Urban skylines: building heights and shapes as measures of city size.  arXiv.org e-Print archive, arXiv:1512.00946 [physics.soc-ph], 2015.
  * (15) C. Molinero and S. Thurner.  How the geometry of cities determines urban scaling laws.  J. R. Soc. Interface, 18(176):20200705, 2021.
  * (16) GHSL Global Human Settlement Layer.  Ghsl global human settlement layer database, 2024.  https://human-settlement.emergency.copernicus.eu/download.php?ds=ucdb.
  * (17) World Settlement Footprint.  World settlement footprint database, 2024.  https://download.geoservice.dlr.de/WSF3D/files/global/.


t72pt

#  BISeizuRe: BERT-Inspired Seizure Data Representation to Improve Epilepsy
Monitoring

Luca Benfenati3, Thorir Mar Ingolfsson1, Andrea Cossettini1,
Daniele Jahier Pagliari3, Alessio Burrello23, Luca Benini12 This project was
supported by the Swiss National Science Foundation (Project PEDESITE) under
grant agreement 193813. 1Integrated Systems Laboratory, ETH Zürich, Zürich,
Switzerland, 2DEI, University of Bologna, Bologna, Italy  3DAUIN, Politecnico
di Torino, Italy

###### Abstract

This study presents a novel approach for EEG-based seizure detection
leveraging a BERT-based model. The model, BENDR, undergoes a two-phase
training process. Initially, it is pre-trained on the extensive Temple
University Hospital EEG Corpus (TUEG), a 1.5 TB dataset comprising over 10,000
subjects, to extract common EEG data patterns. Subsequently, the model is
fine-tuned on the CHB-MIT Scalp EEG Database, consisting of 664 EEG recordings
from 24 pediatric patients, of which 198 contain seizure events. Key
contributions include optimizing fine-tuning on the CHB-MIT dataset, where the
impact of model architecture, pre-processing, and post-processing techniques
are thoroughly examined to enhance sensitivity and reduce false positives per
hour (FP/h). We also explored custom training strategies to ascertain the most
effective setup. The model undergoes a novel second pre-training phase before
subject-specific fine-tuning, enhancing its generalization capabilities. The
optimized model demonstrates substantial performance enhancements, achieving
as low as 0.23 FP/h, 2.5$\times$ lower than the baseline model, with a lower
but still acceptable sensitivity rate, showcasing the effectiveness of
applying a BERT-based approach on EEG-based seizure detection.

Clinical relevance— The model enhances clinical seizure detection, offering
personalized treatments and better generalization to new patients, akin to
successes with transformer-based models, thus significantly improving patient
safety and care.

###### Index Terms:

Healthcare, EEG, Time Series Classification, Deep learning

©2024 IEEE. Personal use of this material is permitted. Permission from IEEE
must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes,
creating new collective works, for resale or redistribution to servers or
lists, or reuse of any copyrighted component of this work in other works.

##  I Introduction & Related Works

Epilepsy is a neurological disorder that affects over 50 million individuals
in the world [1] and is characterized by abnormal electrical activity of the
brain that causes recurrent seizures. While medication management is the
cornerstone of treatment, drug-resistant cases often require more advanced
interventions, including surgery or neurostimulation. In this context,
noninvasive Electroencephalography (EEG) data plays a crucial role in seizure
detection and monitoring to trigger closed-loop actions such as
neurostimulation.

Current methods of classifying raw EEG mostly rely on deep neural networks
(DNNs), which demonstrated higher accuracy than classical machine learning
algorithms [2]. On the other hand, DNNs s often face challenges when
extracting features from relatively long time windows because of the lack of
exploring the global correlation of all the input samples [3, 4]. Recent
approaches [5, 6, 7] showed promising results when working on a reduced number
of EEG channels for the deployment of seizure detection approaches on wearable
low-power devices. However, most of the works mentioned above fail to reach a
False Positive per hour (FP/h) ratio that can enable the clinical
implementation of such methods. Although they can correctly detect almost all
seizure events, they show a FP/h ratio that jeopardizes their real-world
deployment, where wrongly reported seizures will alert the patient and reduce
the detection device reliability. Furthermore, another key open challenge in
all DNN models is the unavailability of very large high-quality labelled
datasets [8], as collecting and labelling seizure data is expensive and
requires complex, human labour-intensive protocols, as well as patient
hospitalization.

Our work investigates whether a model based on the Transformer architecture,
which has recently gained widespread adoption in numerous deep learning
applications, coupled with self-supervision and pre-training on large
unlabeled datasets, can overcome the data labelling bottleneck and improve the
state-of-the-art EEG-based seizure detection for long-term epilepsy
monitoring. Our work focuses on the reduction of the FP/h ratio, even at the
expense of a reduced sensitivity, working towards the objective of clinical
deployment of seizure detection methods: we want a model that predicts a
sufficient number of seizure events, with the least number of false alarms,
thus reducing stress and anxiety of the monitored patients. The state-of-the-
art in this domain is dominated by CNN-based methods [9, 10, 11, 12], with the
best approach achieving $100\%$ sensitivity and $0.58$ FP/h on CHB-MIT dataset
[13]. We take inspiration from a recent study [14] that applied Transformers
and self-supervised sequence learning in Brain–Computer Interface (BCI) with
EEG data. Our contributions are:

  * •

The adaptation for a seizure detection task on CHB-MIT of BERT-inspired Neural
Data Representations (BENDR) [14], an approach inspired by wav2vec 2.0 [15]
and Bidirectional Encoder Representations from Transformers (BERT) [16] which
allows to exploit massive amounts of unlabeled EEG data.

  * •

An extensive task-specific optimization of the base BENDR model, including i)
custom training strategies to improve seizure-detection fine-tuning on CHB-
MIT; ii) the evaluation of the impact of architectural choices; iii) the
application of pre- and post-processing techniques.

  * •

A promising application of a transformer-based approach, pre-trained on a
large amount of unlabelled data, for the seizure detection task.

Thanks to our optimizations, our best model can reduce the FP/h to as low as
0.23, outperforming the best SoA model by 2.5$\times$, with a lower yet
acceptable sensitivity of 72.58% [17].

##  II Materials & Methods

###  II-A Model training and base architectures

####  II-A1 Self-supervised pre-training

The first model training step considered in this work consists in the same
self-supervised pre-training scheme of BENDR, which in turn closely mirrors
wav2vec 2.0 [15]. Namely, the model is pre-trained on a large unlabelled EEG
dataset (TUEG) [18] to learn intrinsic patterns in EEG data. This initial
phase then lays the groundwork for subsequent fine-tuning on our smaller and
task-specific labelled dataset for seizure detection (CHB-MIT). Fig. 1a
illustrates the model architecture used in the self-supervised pre-training
phase. The encoder’s objective is to reconstruct the original sequence
elements. This is achieved thanks to a contrastive loss function, which aims
to align each element of the transformer’s output sequence $c_{i}$ with the
corresponding input $b_{i}$, notwithstanding the masking. In our work, we do
not re-implement this phase, and directly leverage the pre-trained weights of
the original BENDR paper [14].

####  II-A2 Fine-tuning

The masking component is omitted in the fine-tuning phase, and BENDR vectors
are fed directly to the Transformer. A classification block composed of one or
more linear layers with a final Softmax activation is then employed to predict
seizure occurrences based on the first element of the transformer output
sequence. Notably, during pre-training, this first input token was assigned a
special value of $-5$ (selected to be out-of-distribution), and excluded from
the contrastive loss. This step is crucial to ease downstream specialization
to identify seizures accurately. Fig. 1b shows the architecture used in this
phase.

A significant modification of the fine-tuning task with respect to BENDR is
the use of the Sensitivity-Specificity Weighted Cross-Entropy (SSWCE) loss,
introduced by [7].

Figure 1: Model architecture used during (a) self-supervised pretraining (b)
the second supervised pre-training and subject-specific fine-tuning.

Building upon the insights from prior research [19], this study employs a
subject-specific approach for fine-tuning the seizure detection model. Such a
tailored approach is pivotal in capturing the unique EEG patterns and seizure
characteristics inherent to each individual, enhancing sensitivity and
specificity. Precisely, we employ a Leave-One-Out Cross-Validation (LOOCV)
strategy, which involves training the model on all seizure-containing records
of a patient except one. The excluded record is then used as test set, to
assess the model’s efficacy. A further 20% of the training records is
extracted to obtain a validation set, used for early stopping, learning rate
scheduling, etc. The whole procedure is repeated cyclically, considering
different records as test set.

####  II-A3 Second Supervised Pre-training

A novel aspect of this work is the implementation of a second pre-training
phase on CHB-MIT, prior to the subject-specific LOOCV. In this phase, the
model is trained in a supervised way (as described in Sec. II-A2 and with the
architecture of Fig. 1b), to predict seizures on all subjects except the
target one. This step aims to imbue the model with a broader understanding of
EEG patterns associated with seizures across different subjects, before honing
in on the specific characteristics of a single patient. Figure 2 illustrates
the comprehensive training approach employed in the study.

Figure 2: The model training scheme considering the pre-training, the second
pre-training and the subject-specific LOOCV fine-tuning step

###  II-B Task-specific optimizations

In order to maximize seizure detection accuracy, we explore with varying the
model architecture, and apply several pre- and post-processing optimizations.

Keeping the fundamental architecture of Fig. 1, we explored different
classification block architectures, finally converging to a sequence of 4
fully connected layers with decreasing dimensions, progressively reducing the
feature space {(512, 256), (256, 128), (128, 64), (64, 2)} and improving the
discriminative power of the model. We then vary the number of blocks in the
Convolutional stage in {3, 6}, and the number of layers and heads in the
Transformer Encoder in {2, 4, 6, 8, 12} to maximize seizure detection
accuracy. Additionally, we evaluate different weights initialization policies
[20], in order to apply weights pre-trained on TUEG to models of different
sizes (e.g., with a smaller/larger number of layers). Namely, to initialize
the additional layers of a bigger model with respect to the original one, we
tested both the duplication of the pre-trained weights and random
initialization. On the other hand, when testing a smaller model, we initialize
its weights only with the layers shared with the original model.

Based on the hypothesis that early layers may capture the underlying, task-
independent structure of EEG data, we consider freezing the weights of
convolutional layers and fine-tuning only the Transformer encoder. For
completeness, we also test how the model behaves when the Transformer encoder
is frozen as well.

As pre-processing, we consider applying a $5^{th}$-order Butterworth bandpass
filter within a 0.5–50 Hz frequency range to the input, to smooth the signal
and reduce ripple effects, followed by either MinMax or MeanStd normalization.
Moreover, to address the challenge posed by the highly unbalanced nature of
EEG data, we tested two oversampling methods during supervised training
phases, Synthetic Minority Oversampling TEchnique (SMOTE) and Weighted Random
Sampler.

Lastly, we apply post-processing on the model’s output to further reduce FP/h.
Specifically, we go through the predicted labels with a sliding window and
replace the central element with an aggregate prediction over the window. We
consider two aggregation criteria: majority voting and minPooling. In the
latter, we select the smallest predicted value in the window as the output.
Windows of lengths 3, 5, 7 were considered.

###  II-C Training Protocol

We train our models using an Adam optimizer with a learning rate of $1e^{-4}$
and $0.01$ weight decay, reducing the learning rate of factor 0.1 when the
validation loss stops improving for $5$ consecutive epochs. We apply early
stopping on the validation loss with a patience of $15$ epochs. Dropout layers
with 50% probability are added both in the Convolutional stage (between 1D
convolutions and Group Normalization layers) and in the Transformer encoder to
reduce overfitting.

###  II-D Datasets

####  II-D1 Self-supervised Pre-training Data

For the self-supervised pre-training phase, the Temple University Hospital EEG
Corpus (TUEG) [18] was utilized, encompassing 1.5 TB of clinical EEG
recordings. This dataset includes over 10,000 individuals, with diverse
demographics, including 51% females and a wide age range. Pre-existing pre-
training weights were employed as the starting point for model initialization.

####  II-D2 Supervised Fine-tuning Data

The supervised fine-tuning phase utilized the CHB-MIT Scalp EEG Database [13],
comprising EEG recordings from 24 pediatric subjects with intractable seizures
at Boston Children’s Hospital. The dataset, collected using the International
10-20 system, includes 664 EDF files, with 198 containing seizure events. For
this study, only files with seizure occurrences were considered. Additionally,
only 20 out of 23 channels were considered to match the channels used in the
pre-training task on TUEG. All signals were sampled at 256 samples per second
with 16-bit resolution and a window length of 8s was considered, without
overlap.

##  III Experimental Results

All results that are presented were obtained considering the LOOCV setup
described in II-A3, oversampling training data with a Weighted Random sampler.
In terms of metrics, we focus on sensitivity and FP/h: sensitivity represents
the ratio of correctly identified seizure events, i.e., an event for which at
least one of the segments that compose it is assigned a positive label; FP/h
are the number of false alarms in an hour, which is inversely proportional to
the specificity. After an extensive hyperparameters search for the SSWCE loss,
we found $\alpha=0.8$ and $\beta=0.2$ to be the best trade-off to maximize
sensitivity and reduce FP/h.

###  III-A Model Performance

Table I details our seizure detection results on the CHB-MIT dataset. First,
we report the baseline models, i.e., the ones obtained fine-tuning on CHB-MIT
the original BENDR architecture [14] without any of our optimizations. This
serves as a starting reference to demonstrate the improvements achieved by
more refined solutions. Baseline results are reported both with pre-trained
weights from TUEG and without any pre-training. Without pre-training, the
baseline achieves a 50.15% sensitivity and an extremely high FP/h of 132.24.
This is reduced to 10.38 FP/h when using pre-trained weights, demonstrating
the crucial role of self-supervised pre-training for this kind of model. In
subsequent table rows, we report our successive refinements of the model
obtained with optimizations described in the previous section, applied
incrementally to the baseline. Results in bold represent the configurations
that are used after each optimization step as a starting point for the next
one.

First, we explore freezing the weights of the two main parts of the
architecture: as expected, freezing the encoder weights worsens the results,
while freezing the initial convolutional blocks leads to good generalization
capabilities. We impute this to the fact that initial convolutional layers
extract generic enough (i.e., not-task-specific) features from EEG signals
during the pre-training. This model reaches better sensitivity (54.93%) and
FP/h (12.44). On top of this, by applying our second (supervised) pre-training
and filtering pre-processing, we obtain a model with an improved sensitivity
of 69.11% and a reduced FP/h of 6.95. The last steps are the exploration of
the model architecture and the application of pre- and post-processing.
Firstly, MeanStd normalization behaves better than the MinMax one. From what
concerns model architecture, we notice that a more complex classifier, can
learn more discriminative EEG patterns and make more accurate predictions,
reaching 81.87% sensitivity and 2.75 FP/h. Then, we tune the number of
convolutional layers and transformer block, obtaining the best generalization
capabilities with 6 convolutional blocks and 4 attention layers in the
encoder, with 4 heads each. Our best post-processing pipeline applies
minPooling on the output, using a sliding window of length $3$. This leads to
72.58% of seizures detected, with 0.23 FP/h.

TABLE I: Results obtained on optimization dimensions on CHB-MIT | Optimization
---
Dimension
Setup |  | Detected
---
Seizures [%]
FP/h
Baseline | w/o pre-trained weights | 50.15 | 132.24
w pre-trained weights | 43.24 | 10.38
Weight Initialization | freeze conv. layers | 54.93 | 12.44
freeze transf. encoder | 57.96 | 91.56
Pre-training | w second-pretraining | 69.11 | 6.95
Pre-processing | MeanStd + Filtering | 82.44 | 4.01
MinMax + Filtering | 79.12 | 11.32
Model Architecture | new classifier | 81.87 | 2.75
Dimension (conv., transf.) | 6, 12 | 86.55 | 4.59
6, 4 | 83.57 | 2.81
6, 2 | 85.91 | 5.35
3, 4 | 82.01 | 3.88
Post-processing | Majority voting | 78.16 | 0.60
minPooling | 72.58 | 0.23
Majority + minPooling | 73.28 | 0.26

###  III-B Comparison with State-of-the-Art

Table II shows a comparative analysis with the latest state-of-the-art works
that address seizure detection on CHB-MIT considering all patients and using
from 18 up to 23 EEG channels. We distinguish between CNN-based and
Transformer-based approaches. Our approach outperforms all of them in terms of
FP/h, at the cost of lower sensitivity, demonstrating the capability of modern
transformer models to capture complex epileptic patterns in EEG. Our approach
reduces FP/h of the best performing CNN-based model [12] by 2.5$\times$, while
simplifying the heavy pre-processing and feature extraction steps, typical of
CNN-based approaches, at the expense of reduced sensitivity of the model
(27.42% decrease). Our approach significantly improves upon other transformer-
based models, with a 4.7$\times$ lower FP/h with respect to [21]. Noteworthy,
this is crucial for a real-life closed-loop system, given that a high number
of FP/h per hour would cause many warnings in patients, which in turn increase
their stress, as discussed in [22]. Moreover, a sensitivity over $50\%$ has
already been showed to be an acceptable requirement for a seizure detection
method [17].

TABLE II: Comparison with SoA seizure detection on CHB-MIT Type | Work |  | Detected
---
Seizures [%]
FP/h
CNN-based | Qui et al. [23] | 97.1 | 7.51
Wang et al. [9] | 97.52 | 3.42
Zhao et al. [10] | 97.79 | 1.25
Li et al. [11] | 98.47 | 0.63
Sahani et al. [12] | 100 | 0.58
Transformer-based | Yan et al. [24] | 96.01 | 27.14
Hussein et al. [21] | 99.8 | 1.08
Our work | 72.58 | 0.23

##  IV Conclusion

This work proposes a BERT-based approach for seizure detection on the CHB-MIT
dataset, demonstrating the potential of transfer learning from general EEG
data to seizure detection task. We also validate the effectiveness of a
Transformer-based architecture which, once pre-trained on a large amount of
unlabelled data, can partially overcome data labelling bottleneck and improve
the state-of-the-art results. We then extensively explore hyper-parameters and
pre-/post-processing techniques to improve the model performance. The best
model found obtains 0.23 FP/h, detecting 72.58% of seizures. Our future work
will include compressing the model and reducing the EEG channels to explore
the possibility of its deployment on a wearable device, as well as
implementing artefacts detection and removal algorithms [25] to further
improve performances towards the clinical implementation of such methods.

## References

  * [1] World Health Organization, “Epilepsy: a public health imperative,” _who.int/publications/i/item/epilepsy-a-public-health-imperative_ , 2019.
  * [2] A. Shoeibi _et al._ , “Epileptic seizures detection using deep learning techniques: A review,” _International Journal of Environmental Research and Public Health_ , 2021.
  * [3] R. Zanetti _et al._ , “Approximate zero-crossing: A new interpretable, highly discriminative and low-complexity feature for eeg and ieeg seizure detection,” _Journal of Neural Engineering_ , 2022.
  * [4] I. Olokodana _et al._ , “Ezcap: A novel wearable for real-time automated seizure detection from eeg signals,” _IEEE TCE_ , 2021.
  * [5] P. Busia _et al._ , “Reducing false alarms in wearable seizure detection with EEGformer: A compact transformer model for MCUs,” _IEEE TBioCAS_ , 2024.
  * [6] T. M. Ingolfsson _et al._ , “Towards long-term non-invasive monitoring for epilepsy via wearable EEG devices,” in _2021 IEEE BioCAS_ , 2021.
  * [7] ——, “EpiDeNet: An energy-efficient approach to seizure detection for embedded systems,” in _Proc. BioCAS_ , 2023, pp. 1–5.
  * [8] K. Rasheed _et al._ , “A generative model to synthesize eeg data for epileptic seizure prediction,” _IEEE TNSRE_ , 2021.
  * [9] X. Wang _et al._ , “One dimensional convolutional neural networks for seizure onset detection using long-term scalp and intracranial eeg,” _Neurocomputing_ , 2021.
  * [10] S. Zhao _et al._ , “Energy-efficient neural network for epileptic seizure prediction,” _IEEE TBME_ , 2021.
  * [11] C. Li _et al._ , “Seizure onset detection using empirical mode decomposition and common spatial pattern,” _IEEE TNSRE_ , 2021.
  * [12] M. Sahani _et al._ , “Epileptic seizure recognition using reduced deep convolutional stack autoencoder and improved kernel rvfln from eeg signals,” _IEEE TBioCAS_ , 2021.
  * [13] A. Goldberger _et al._ , “Physiobank, physiotoolkit, and physionet : Components of a new research resource for complex physiologic signals,” _Circulation_ , 2000.
  * [14] D. Kostas _et al._ , “Bendr: Using transformers and a contrastive self-supervised learning task to learn from massive amounts of eeg data,” _Frontiers in Human Neuroscience_ , 2021.
  * [15] A. Baevski _et al._ , “wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations,” in _Advances in NIPS_ , 2020.
  * [16] J. Devlin _et al._ , “BERT: Pre-training of deep bidirectional transformers for language understanding,” in _NAACL-HLT_ , 2019.
  * [17] S. Mathieson _et al._ , “Validation of an automated seizure detection algorithm for term neonates,” _Clinical Neurophysiology_ , 2015.
  * [18] A. Harati _et al._ , “The temple university hospital eeg corpus,” in _2013 IEEE GlobalSIP_ , 2013.
  * [19] G. Varone _et al._ , “A comprehensive machine-learning-based software pipeline to classify eeg signals: A case study on pnes vs. control subjects,” _Sensors_ , 2020.
  * [20] M. Narkhede _et al._ , “A review on weight initialization strategies for neural networks,” _Artificial Intelligence Review_ , 2022.
  * [21] R. Hussein _et al._ , “Multi-channel vision transformer for epileptic seizure prediction,” _Biomedicines_ , 2022.
  * [22] E. Bruno _et al._ , “Wearable technology in epilepsy: The views of patients, caregivers, and healthcare professionals,” _E &B_, 2018.
  * [23] S. Qiu _et al._ , “Lightseizurenet: A lightweight deep learning model for real-time epileptic seizure detection,” _IEEE JBHI_ , 2022.
  * [24] J. Yan _et al._ , “Seizure prediction based on transformer using scalp electroencephalogram,” _Applied Sciences_ , 2022.
  * [25] T. M. Ingolfsson _et al._ , “Energy-efficient tree-based eeg artifact detection,” in _2022 IEEE EMBC_ , 2022.