supplementary

# The Portil$\infty$p: a deep learning-based open science tool for closed-loop
brain stimulation

Nicolas Valenchon  Yann Bouteiller  Hugo R. Jourde  Emily B.J. Coffey and
Giovanni Beltrame

###### Abstract

Electroencephalography (EEG) is a method of measuring the brain’s electrical
activity, using non-invasive scalp electrodes. In this article, we propose the
Portiloop, a deep learning-based portable and low-cost device enabling the
neuroscience community to capture EEG, process it in real time, detect
patterns of interest, and respond with precisely-timed stimulation. The core
of the Portiloop is a System on Chip composed of an Analog to Digital
Converter (ADC) and a Field-Programmable Gate Array (FPGA). After being
converted to digital by the ADC, the EEG signal is processed in the FPGA. The
FPGA contains an ad-hoc Artificial Neural Network (ANN) with convolutional and
recurrent units, directly implemented in hardware. The output of the ANN is
then used to trigger the user-defined feedback. We use the Portiloop to
develop a real-time sleep spindle stimulating application, as a case study.
Sleep spindles are a specific type of transient oscillation ($\sim$2.5 s,
12-16 Hz) that are observed in EEG recordings, and are related to memory
consolidation during sleep. We tested the Portiloop’s capacity to detect and
stimulate sleep spindles in real time using an existing database of EEG sleep
recordings. With 71% for both precision and recall as compared with expert
labels, the system is able to stimulate spindles within $\sim$300 ms of their
onset, enabling experimental manipulation of nearly the entire spindle. The
Portiloop can be extended to detect and stimulate other neural events in EEG.
It is fully available to the research community as an open science
project111https://github.com/mistlab/portiloop.

##  I Introduction

Electrical activity within the brain forms the basis of perception, thought
and behaviour. This endogenous electrical activity tends to be oscillatory in
nature, as reciprocal connections within and between brain regions form
functional circuits for processing and communicating information, and it can
be measured on the scalp using electroencephalography (EEG). Correlational
studies have been performed for nearly a century that attempt to link specific
patterns and frequency bands in EEG to cognitive functions or brain states.
These studies are informative and increased our understanding of brain
processes. However, they are unable to establish causal relationships. The
ability to interact with brain oscillations in a precisely-timed fashion to
enhance or inhibit endogenous processes - using sensory [1, 2, 3, 4],
electrical [5] or magnetic [6] stimulation - allows for their functional roles
to be determined [7], and potentially for restoration of processes
deteriorated by aging or pathology [8]. While there is a great deal of
interest in _closed-loop_ stimulation [7, 9], researchers lack flexible,
powerful tools that are easily accessible. Research efforts are also limited
by the portability of systems, and by their complexity and expense. In the
current work, we develop the _Portiloop_ , a complete, portable system for
closed-loop stimulation. We demonstrate the Portiloop on a case study of
scientific relevance: the auditory stimulation of sleep spindles.

Using auditory stimulation, researchers have enhanced _slow oscillations_
(SOs), which are high amplitude waves ($0.5$ \- $1.5$ Hz) known to be involved
in memory consolidation [1]. Ngo et al. showed SOs enhancement caused an
overnight improvement in memory performance, a result that has now been
replicated (see [10, 11, 12] for reviews). Slow oscillations are thought to
work in concert with other faster oscillations, called _sleep spindles_ , to
reactivate recently learned memories and transfer them to long-term memory
[13, 14]. Sleep spindles are transient oscillations observed in both lighter
and deeper non-rapid eye movement (NREM) sleep (i.e., stages 2 and 3). Their
role in memory consolidation is supported by increases in spindle density
following learning (e.g., [15]). Age-related changes in sleep spindles are
correlated with differences in overnight performance gains (e.g., [16, 17];
see [18] for a review of spindle mechanisms and functions).

Unfortunately, current systems that are capable of slow oscillation
stimulation have more difficulty accurately and precisely detecting and
stimulating sleep spindles in real time. Particular challenges of spindle
stimulation are that each oscillatory cycle is only $\sim 60$ ms long and the
entire spindle is between $0.5$ and $2.5$ s, leaving little time for
traditional window-based frequency analysis; there is considerable variability
between the frequency, amplitude, and duration of individuals’ spindles,
particularly in older populations [19, 20]; and even for offline detection of
spindles (which is an easier problem as the entire spindle is available and
can be used in detection), agreement on spindle identification between experts
and algorithms is limited ($\sim 70$%) [21, 22].

If it were possible to influence spindles with sound, as it is to enhance slow
oscillations, researchers could explore their functional role in healthy
adults as well as characterize their involvement in cognitive aging, and even
perhaps restore degraded function. Although auditory stimulation is attractive
due to its non-invasiveness, it is only one of several possibilities: specific
brain areas can be more directly and forcefully stimulated using transcranial
electrical or magnetic stimulation, which could also be triggered by the same
device. Furthermore, by designing the system flexibly such that it can be
extended to detect and stimulate brain oscillations other than spindles, we
can greatly expand its application, for example to theta-band oscillations
that are associated with working memory capacity and task performance [23].

The goal of this work is to design and explore the properties of a deep
learning-based, portable, battery-efficient and low-cost device that will
enable the neuroscience community to collect and process EEG data in real-
time, detect patterns of interest for fundamental research questions, and
respond at low latency with precisely-timed stimulation. We propose a neural
network-based detection algorithm in FPGA and a parallelized design space
exploration algorithm to optimize its parameters. We aim to accelerate
fundamental research on closed-loop stimulation in neuroscience by creating a
highly functional device and offering the code and plans to developers and
scientists in the research community.

##  II Related work

###  II-A Devices for EEG acquisition and stimulation

Various portable devices have been developed to acquire and process EEG
signals. In [24], the authors developed a low-cost device limited to
acquisition. Other portable devices enable closed-loop stimulation [2, 3, 4],
some also based on low-cost hardware [25], but work with simple heuristics and
are generally not sufficiently powerful for deep learning applications.
Closed-loop stimulation has been used in the context of preventing drowsiness
[2], enhancing attention and engagement [4], preventing strokes [3] and
studying memory consolidation [1, 25, 26, 27]. Depending on the application,
real-time constraints can vary from hundreds of ms [28] to a few seconds [4].
The Portiloop is the first system to provide a portable, real-time and deep
learning-based solution for all these applications.

###  II-B Offline sleep spindle detection

Machine learning-based detection algorithms require large sets of accurately
labeled data. The consistent detection and labeling of sleep spindles is a
challenging task, due to variability in their appearance and strength.
Traditionally, spindles are visually identified by experts. One commonly used
dataset [29, 30] is the Montreal Archive of Sleep Studies (MASS) [31], in
which the sleep spindle annotations were provided by two experts. Projects
using MASS for training usually take spindles identified by either expert.
However, the MASS annotations have a low inter-rater agreement, as quantified
by the f1-score222The f1-score, widely used in statistics, is described in
Equation 6. of $0.54$ [22]. The Massive Online Data Annotation (MODA) [22]
project addressed this issue by having 5 experts (on average) annotate
spindles and rate their confidence, in each EEG segment. The experts had an
inter-rater f1-score of $0.72$ with respect to the final MODA labels. This
score is considerably better than the MASS equivalent, and the number of
experts, the scoring and the post-processing steps enable final labels of much
higher precision.

Several offline sleep spindle detectors have been developed and tested on MODA
[21, 32, 33, 34, 35, 36, 37]. However, these are generally heuristics that
compute Fourier transforms or wavelet decomposition on large portions of the
signal, including segments that would be in the future considering an online
application, and thus cannot be implemented for real-time detection.

###  II-C Online sleep spindle detection

Online detectors (i.e., attempting detection during signal acquisition) face
more challenging conditions than offline detectors, due to the unavailability
of “future” data points. For example, the duration of the spindle is not yet
known, and it is one of the identifying criteria commonly used by experts.
Some existing heuristics filter the signal, compute power features and rely on
thresholds to perform detection [25]. Yet, these approaches exhibit poor
f1-scores. Deep learning can also be leveraged to perform online sleep spindle
detection. This is done by first training an artificial neural network offline
through supervised learning to detect sleep spindles, and then feeding the
incoming signal to the trained detector. Several such models have been trained
in previous work [29, 30, 38, 39]. However, these works do not consider
hardware constraints that are central for our purpose: they use large models
that are often unable to run in real time even on high-end GPUs, which makes
them inapplicable on the Portiloop device. Moreover, they are usually trained
and tested on MASS [31] with an ‘OR’ operation performed on its labels, and
therefore limited performance [22]. In this work, we design a Pareto-optimal
neural architecture that performs best on the MODA dataset [22] while
satisfying our hardware and timing constraints. We validate our architecture
against the state-of-the-art SpindleNet [29], initially used with the MASS
dataset. When both architectures are trained and tested on MODA, ours
outperforms the baseline, on top of running in real time with our hardware.

##  III Methods

Figure 1: Main functionalities. Timing is reported from our case study. The
constant delay in stimulation output (4 ms for auditory stimuli) enables
precise experimental control stimulation timing relative to neural oscillatory
phase.

A high-level description of the Portiloop system is provided in Figure 1.
After being retrieved from the subject by electrodes and converted to numeric
format by an Analog to Digital Converter (ADC), the EEG signal is processed
through Finite Impulse Response (FIR) filters and fed to a trained Artificial
Neural Network (ANN). The output of the ANN determines the nature of the
feedback, i.e., whether to stimulate the subject or not. These operations
introduce their own constant delays in the closed-loop system (in Figure 1;
note that the delays are measured on the sleep spindle detector that we derive
in the second part of this article). Such delays are inherent to the order of
the FIR filters, the architecture of the ANN and the nature of the
stimulation. They have to be accounted for when building a Portiloop
application, in order to satisfy the real-time constraints of the task.

###  III-A Real-time constraints

To allow maximum flexibility in when the spindle can be stimulated, it must be
accurately detected as early as possible following spindle onset. We identify
two different sources of delay in the proposed system, _hardware_ and
_software_ delays:

####  III-A1 Hardware delays

By _hardware delays_ we refer to the time it takes to retrieve the signal from
the electrodes, convert it to digital, filter it, process it through the ANN,
and send the resulting feedback stimulation to the subject.

Because we implement the Portiloop in Fast Programmable Gate Array (FPGA)
logic, all these delays are constant. The hardware delays introduced by the
electrodes, ADC and filtering are negligible in the Portiloop applications
(i.e., $\ll 1$ ms). The ANN delay depends on our implementation of the neural
network architecture in FPGA. Since intra-layer operations can be parallelized
in FPGA, it depends primarily on the depth of the ANN, i.e., its number of
layers. However, parallelization in FPGA has a cost in terms of resources,
which are limited in the Portiloop. Thus, in practice, the width of reasonably
large ANNs also has an impact on this delay. Finally, the stimulation delay
depends on the nature of the stimulation. The Portiloop instantly sends a
trigger from the ANN output, but this does not mean that the stimulus reaches
the subject simultaneously, e.g., 4 ms in the case of auditory stimulation.

####  III-A2 Software delays

Figure 2: Software accumulation effect of recurrent units. Dark blue signal:
no phenomenon of interest. Black signal: phenomenon of interest not detected
by the ANN. Green signal; phenomenon of interest correctly detected. Magenta:
output of the ANN. Horizontal grey: detection threshold. The ANN introduces a
software delay: the recurrent units of the ANN act as pseudo-accumulators,
thus the detection happens with a variable delay.

Although the hardware operations performed by FIR filters are near-
instantaneous, this type of filtering introduces a constant software delay in
the filtered signal. This delay is a trade-off related to the order of the FIR
filter. The higher the order of a FIR filter, the more efficient it is at
filtering out undesirable frequencies, but also the longer the software delay
introduced in the signal by the filtering operation. More precisely, the delay
$d$ introduced by an FIR filter of order $o$ on a signal sampled at frequency
$f$ is $d\propto\frac{o}{2f}$. These hardware and software delays sum to a
constant total delay that is the minimum possible response time of the
Portiloop system. Depending on the target timing constraints of the
application, this leaves a certain margin for the actual detection of the
phenomenon of interest in the filtered EEG signal. In practice, this detection
may happen with its own variable software delay. An example of such delay is
illustrated in Figure 2, where an ANN takes a variable amount of time before
correctly detecting a transient pattern in an EEG signal.

###  III-B Architecture of the proposed system

The detailed architecture of the Portiloop device is presented in Figure 3.
The system is implemented on a Pynq Z2 [40] board. This board provides an
integrated FPGA, which we use along the Vivado suite (v2019.1) to convert our
software logic into hardware circuitry operating in real time.

Figure 3: Detailed architecture of the Portiloop device. $(1)$ The user
controls the device from a simple Web User Interface. $(2)$ An EEG front-end
device acquires biosignals from the electrodes. $(3)$ The Microblaze soft
processor centralizes all operations performed within the FPGA. $(4)$ The
Microblaze core sends the digital signal to be filtered through Finite Impulse
Response filters. $(5)$ The Microblaze core sends the digital filtered signal
to an artificial neural network. The neural network returns the likelihood of
the input signal being the desired pattern in order to decide whether a
stimulus should be sent.

####  III-B1 Data acquisition

The analog signal captured by the electrodes is processed by a low-noise ADC
front-end for EEG/ECG developed by Texas Instrument (ADS1299 [41]). This
component can retrieve signal from 8 electrodes simultaneously, plus reference
and bias electrodes. The ADC outputs a digital signal, data point per data
point. Each data point is sent directly to the Pynq Z2 board at up to 16 kHz
through Serial Peripheral Interface (SPI) communication.

####  III-B2 Data preprocessing

##### Microblaze

The Microblaze is a soft processor that is readily implemented by Xilinx on
the FPGA. It acts as a central scheduler for all operations happening within
the FPGA. We use the Microblaze to read data points from the SPI pins in real
time and log them for possible post-analysis. We send either each or a
relevant fraction of these data points to the FIR filters in real time by
Direct Memory Access (DMA).

##### FIR filters

We use FIR filters to preprocess the digital signal by filtering out undesired
frequencies. These filters are readily available as Intellectual Property (IP)
cores provided by Xilinx. We retrieve each unprocessed data point, filter it,
and send the filtered data point back to the Microblaze through DMA. It is
possible to implement as many FIR filters as needed, in the limit of the
available space on the FPGA.

##### Additional preprocessing

We can further preprocess the filtered signal directly within the Microblaze
for simple operations. In particular, we standardize the signal on the fly
though exponential moving average. In other words, we transform the filtered
signal $s(t)$ to $s^{\prime}(t)$ according to:

$s^{\prime}(t)=\frac{s(t)-\hat{\mu}(t)}{\hat{\sigma}(t)}$ (1)

where $\hat{\mu}$ and $\hat{\sigma}$ are estimates of the mean and standard
deviation of the filtered signal, computed as follows:

$\delta(t)=s(t)-\hat{\mu}(t-1)$ (2)
$\hat{\mu}(t)=\hat{\mu}(t-1)+\alpha_{\mu}\delta(t)$ (3)
$\hat{\sigma}^{2}(t)=(1-\alpha_{\sigma})(\hat{\sigma}^{2}(t-1)+\alpha_{\sigma}\delta^{2}(t))$
(4) $\hat{\sigma}(t)=\sqrt{\hat{\sigma}^{2}(t)}$ (5)

$\alpha_{\mu}$ and $\alpha_{\sigma}$ being hyperparameters in $[0,1]$. This
custom real-time standardization makes the signal comparable from one subject
to another, enabling generalizable learning.

####  III-B3 Neural Network

Any real-time decision-making algorithm can be implemented on the Portiloop as
long as it is written in FPGA-ready code. We focus on ANN models. The
Portiloop being a portable, lightweight and battery-efficient system, it does
not include any heavy computing units such as high-end GPUs. Instead, the ANN
is synthesized as hardware circuitry in an FPGA, which provides only a limited
amount of configurable circuitry. Thus, only small ANN models are allowed. We
provide High-Level-Synthesis (HLS) C++ implementation of the following
building blocks, based on [42]:

  * •

1D convolutional layers, used to learn and extract relevant patterns (e.g.,
frequencies) in the signal.

  * •

Layers of Gated Recurrent Units (GRUs), used to learn a memory representation,
so as to keep track of the past. GRUs are usually on par with their famous
Long-Short Term Memory (LSTM) cousin in terms of performance [43, 44], but
they use roughly 30% less hardware resource and thus are more interesting in
the context of FPGAs, where resources are limited.

  * •

Fully connected layers, used to further increase the representational capacity
of the model.

  * •

Rectified Linear Unit (ReLU) non-linearities, which are simple
$\text{max}(x,0)$ operations with minimal footprint.

We train an ANN offline using PyTorch333https://pytorch.org/, and once
trained, we extract its weights from PyTorch and use our C++ library in the
Vivado suite to convert the model (architecture and weights) into the FPGA.
When a data point is fully preprocessed through the steps described in Section
III-B2, the Microblaze sends a buffer of the last preprocessed data points by
DMA to the ANN. In other words, we feed a sliding window of preprocessed
signal as input to the model.

Inference is performed by the neural network in parallel to the acquisition
and preprocessing of the next data point by the Microblaze. Once inference is
done, the result is sent back to the Microblaze through DMA. Depending on this
result, the Microblaze writes a code (e.g., a trigger signal) on the output
pins of the Portiloop device. Any stimulation device can be used with an
appropriate adapter for the trigger signal.

###  III-C Type of ANN

The Portiloop system is designed to process EEG signals in real time. This
type of input involves time-series of data containing oscillatory and
transitory elements. In the realm of deep learning, a natural way of
processing such data is to use either 1D convolutions, recurrent units, or a
combination of both. The type of ANN architecture that we recommend is
inspired by the SpindleNet [29] architecture. In essence, a sliding window
over a few last data points is fed to a Convolutional Neural Network (CNN)
whose purpose is to extract relevant features (e.g., frequencies) in this
signal fragment. Then, these extracted features are fed to a Recurrent Neural
Network (RNN) whose purpose is to keep track of the features extracted in past
forward passes. Note that another family of architectures, called Transformers
[45], is known for exhibiting good results task-wise with this type of data.
However, Transformers are memory-less and not suitable for lightweight real-
time applications, because they need to process the whole signal at each
forward pass. Conversely, RNNs are able to process one single data point at
each forward pass and keep track of the past in memory, which makes them more
relevant in Portiloop applications.

###  III-D Pareto-optimal architecture search

The FPGA used in the Portiloop has a limited amount of available circuitry, so
as to ensure its portability and low price. Thus, typically large ANN
architectures such as SpindleNet [29] are orders of magnitude too large to be
implemented in our device. Fortunately, as we illustrate in Section IV, those
models are often much too large for their actual purpose. When developing a
novel Portiloop application, one needs to devise a neural network that is both
high-performance and lightweight, by selecting the right set of
hyperparameters $H$ in the space of all possible hyperparameter sets
$\mathcal{H}$. Such hyperparameters include the size of the sliding window,
the number of layers in each part of the ANN, the width of each layer, the
time dilation (see Section III-E), the type of optimizer, the hyperparameters
of the optimizer itself, etc. $\mathcal{H}$ can be very large, and finding a
set of hyperparameters that yields a high-performance model within given
hardware constraints is far from trivial. We introduce “Parallel Model-Based
Optimization” (PMBO), a network-based algorithm that automates this process in
a parallel fashion. Released as open-source along with our code, PMBO is
essentially a parallelized and evolved version of “Probabilistic SMBO” [46].
PMBO is a guided-search approach that finds a suitable set of hyperparameters
rapidly. For this matter, it uses one machine (or process) to train a _meta
network_ whose role is to predict non-trivially available costs for any given
set of hyperparameters. Furthermore, PMBO uses any available machines (or
processes) in parallel to train ANNs from sets of hyperparameters selected
based on the cost estimated by the meta model.

####  III-D1 Software and hardware costs

The purpose of PMBO is to find Pareto-optimal sets of hyperparameters that
minimize both a  _software cost_ and a  _hardware cost_. Hyperparameter
selection is a bi-objective problem in our setting:
$\bullet$ On the one hand, we want an ANN that performs well at detecting the
desired patterns. We measure this performance in terms of the f1-score of our
model. The f1-score depends both on the precision (how sure we are that
positive outputs are true positives) and on the recall (how sure we are that
we capture all true positives) of the model. It is defined as:

$\text{f1}=\frac{2*\text{recall}*\text{precision}}{\text{recall}+\text{precision}}$
(6)

where

$\text{precision}=\frac{\text{true positives}}{\text{true
positives}+\text{false positives}}$ (7)

and

$\text{recall}=\frac{\text{true positives}}{\text{true positives}+\text{false
negatives}}$ (8)

The f1-score is in $[0,1]$, with $1$ being a perfect classifier. We cast this
into a minimization problem by defining our _software cost_ as
$L_{s}=1-\text{f1}$.

$\bullet$ On the other hand, we want our ANN to be as lightweight as possible,
so it fits in the FPGA and executes as fast as possible. A precise measurement
of the execution duration and amount of circuitry needed on the board is
difficult. In fact, we do not have access to these results until the model is
actually synthesized on the board, which is a long process. Thus, we use the
number of trainable weights in the ANN as a proxy for these concerns, and call
this number our _hardware cost_ $L_{h}$. This cost is computed directly in
PyTorch. Note that our choice of costs is arbitrary and other custom costs can
be used instead.

####  III-D2 Pareto efficiency

Let us consider a meta dataset $\mathcal{E}$ of previously completed
experiments (i.e., tuples $E=(H,L_{s},L_{h})$ of hyperparameter sets with
their real costs). Let us also consider the following Pareto-domination
relation:

######  Definition 1.

Let $E=(H,L_{s},L_{h})$ and
$E^{\prime}=(H^{\prime},L_{s}^{\prime},L_{h}^{\prime})$ be two experiments in
$\mathcal{E}$. We say that $E$ Pareto-dominates $E^{\prime}$ and we denote
$E<E^{\prime}$ when $L_{s}<L_{s}^{\prime}\wedge L_{h}<L_{h}^{\prime}$. (NB: in
the context of this minimization problem, dominating means having the smallest
costs, hence the notation)

For a given experiment $E$ and meta dataset $\mathcal{E}$, we denote
$D_{\mathcal{E}}(E)$ as the number or experiments in $\mathcal{E}$ that are
Pareto-dominated by $E$, and $d_{\mathcal{E}}(E)$ as the number of experiments
in $\mathcal{E}$ that Pareto-dominate $E$. In other words:

$D_{\mathcal{E}}(E)=|\\{E^{i}\in\mathcal{E},E^{i}>E\\}|$ (9)
$d_{\mathcal{E}}(E)=|\\{E^{i}\in\mathcal{E},E^{i}<E\\}|$ (10)

For a given set of hyperparameters $H\in\mathcal{H}$, we further define an
estimate of the corresponding completed experiment $E$ as
$\hat{E}~{}=~{}(H,\hat{L}_{s},\hat{L}_{h})$, where $\hat{L}_{s}$ and
$\hat{L}_{h}$ are estimates of the real costs, computed by the meta network.
Note that in our setting, $\hat{L}_{h}=L_{h}$ is available and thus only
$\hat{L}_{s}$ is estimated by the meta network. We propose the heuristic
Pareto efficiency $\eta(\hat{E})$:

$\eta(\hat{E})=a_{\mathcal{E}}(\hat{E})+b_{\mathcal{E}}(\hat{E})+s_{\mathcal{E}}(\hat{E})-h_{\mathcal{E}}(\hat{E})$
(11)

where $a_{\mathcal{E}}(\hat{E})$ promotes hyperparameter sets whose predicted
costs are not dominated by many experiments in $\mathcal{E}$:

$a_{\mathcal{E}}(\hat{E})=1-\frac{d_{\mathcal{E}}(\hat{E})}{|\mathcal{E}|}$
(12)

$b_{\mathcal{E}}(\hat{E})$ promotes hyperparameter sets whose predicted costs
dominate many experiments in $\mathcal{E}$:

$b_{\mathcal{E}}(\hat{E})=\frac{D_{\mathcal{E}}(\hat{E})}{|\mathcal{E}|}$ (13)

$s_{\mathcal{E}}(\hat{E})$ promotes hyperparameter sets whose predicted
software costs are better than the best software cost amongst all completed
experiments in $\mathcal{E}$:

$s_{\mathcal{E}}(\hat{E})=\frac{\text{min}(\\{L_{s},(H,L_{s},L_{h})\in\mathcal{E}\\})}{\hat{L}_{s}}$
(14)

and $h_{\mathcal{E}}(\hat{E})$ penalizes hyperparameter sets that have a high
density with respect to experiments present in $\mathcal{E}$ in terms of their
hardware cost. More precisely, we define a range of hardware costs we are
interested in, and we split this range into a number of bins. We then compute
the binned density of experiments in $\mathcal{E}$ over this range, and
multiply this density by the range’s width. The penalty
$h_{\mathcal{E}}(\hat{E})$ is the height of the resulting bin where
$\hat{L}_{h}$ stands. Multiplying the density by the range’s width enforces
$h_{\mathcal{E}}(\hat{E})>1$ in regions of high density and
$h_{\mathcal{E}}(\hat{E})<1$ in regions of low density.

####  III-D3 Parallel Model-Based Optimization

Figure 4: PMBO. The algorithm is based on a single producer and multiple
consumers architecture. The meta learner is in charge of producing relevant
hyperparameter sets in a guided fashion. Then, it sends them to idle workers,
and keeps producing new sets as long as idle workers remain. Each worker that
has received a new set starts training the corresponding ANN. Once this
training ends, the real costs of the hyperparameter set can be computed and
are sent back to the meta learner.

Figure 4 explains the PMBO algorithm. A central meta learner is communicating
with $n$ peripheral workers to find a hyperparameter set
$H^{\ast}\in\mathcal{H}$ that is Pareto-optimal for both the software and
hardware costs (i.e., non-Pareto-dominated by any other set). For this matter,
the algorithm uses a meta dataset $\mathcal{E}$ of tuples
$E^{i}=(H^{i},L_{s}^{i},L_{h}^{i})$ to train a meta network that maps any
hyperparameter set $H\in\mathcal{H}$ to its corresponding (estimated) costs
$\hat{L}_{s}$ and $\hat{L}_{h}$ 444$\hat{L}_{h}$ is an output of the meta
network in the general case. However, with our choice of hardware cost it is
not, since the ground truth $L_{h}$ is available.. Once the meta network is
trained, it is used to guide the sampling process. More exactly, we sample $m$
hyperparameter sets in $\mathcal{H}$ from a multivariate Gaussian distribution
around the hyperparameter set corresponding to the last results received from
the workers by the meta learner. Sets that don’t satisfy user-defined
constraints (e.g., that have already been tested, or, when $L_{h}$ is
available, that fall outside the range we are interested in) are discarded and
resampled. We then select the best set in terms of Pareto efficiency,
estimated thanks to the trained meta network. This selected set is appended to
a bounded buffer, waiting to be consumed by an idle worker. The sampling
process is repeated until the bounded buffer is full. When a worker is idle,
it fetches an ANN architecture and training instructions from the bounded
buffer. The worker yields a measurement of the real hardware and software
costs for the current hyperparameter set, which are sent back to the meta
learner and appended to the meta dataset. The meta learner then uses the
updated meta dataset to train a new meta network, and so on.

###  III-E Virtual ANN parallelization with time dilation

Time dilation [47] is a technique that enables recurrent units such as GRUs to
look further back in time before gradients vanish, at no computational cost.
We propose a version of this technique that allows us to virtually parallelize
a single physical ANN into several decoupled virtual models. Our approach
enables shallow recurrent neural networks to look further back in time by
skipping the redundant information that is inherent to the use of a sliding
window as input, while still acting as fast as possible. Figure 5 shows how
time dilation can be used to look further back in time and avoid redundancy.

Figure 5: Time dilation. In this example, a sliding window of the 4 last
samples (underlined with a dotted curve) is used as input to the model. The
time dilation is the number of samples between two forward passes in the ANN.
When it is small (top), two consecutive windows overlap for the most part (see
e.g., green and red windows). This implies that the recurrent hidden state
which persists from one forward pass to the next (arrows) contains a lot of
redundancy with the next input. When the time dilation is big (bottom), this
issue is corrected, and back-propagation will reach much further back in time
for the same number of forward passes. NB: forward passes happen only at the
end of arrows in this diagram, the ANN is idle during other time-steps.

Although time dilation enables reaching further back in time at no extra
computational cost, this comes with a cost in terms of delays. Since our
technique causes samples 555Here, by _sample_ we refer to all data points
acquired while a forward pass is executed in the ANN. For the sake of
simplicity, our figures assume that there is only one data point per sample.
to be skipped between forward passes in the ANN, a detection delay that can be
as long as the time dilation is introduced. We correct this issue by
implementing a trick that we call _virtual parallelization_. We create a First
In First Out (FIFO) list as big as the time dilation, and fill this list with
independent hidden states. At each time-step, we pop a hidden state from this
list, feed it to the recurrent units of our ANN, perform a forward pass, and
append the resulting hidden state to the list. Doing this without skipping
samples is equivalent to having several decoupled models running in parallel
as illustrated in Figure 6, although one single ANN is physically used.

Figure 6: Virtual parallelization. In this example, the time dilation is 2.
Thus, we keep track of 2 independent hidden states and feed these alternately
to the recurrent units of the ANN. This is equivalent to having two decoupled
models that are used alternately for inference.

This trick allows us to keep acting as fast as possible since it removes the
need for skipping samples, while still reaching far back in time at no extra
computational cost.

###  III-F Case study: sleep spindle auditory stimulation

We now demonstrate the Portiloop implementation of stimulating the brain with
sound during sleep spindles. The long-term goal of this application is to
further clarify the role of sleep spindles in learning and memory, and to
explore therapeutic interventions for memory decline. Stimulating sleep
spindles is more challenging than low frequency neural activity like slow
oscillations, due to their high frequency ($\sim$12 to 16 Hz), rapid evolution
($<$2.5s), and therefore tight timing constraints. We propose the first
portable device able to stimulate sleep spindles in real time with high
detection performance. Because maximum experimental flexibility is attained by
being able to stimulate at anytime during the course of the spindle including
with phase precision, we conduct a thorough time analysis of the proposed
system, with possible trade-offs to maximize specific performance.

####  III-F1 Neural architecture

To the best of our knowledge, the state-of-the-art in previous work regarding
online sleep spindles detection was the SpindleNet [29] model. This
architecture is however much too large to be implemented on the Portiloop
system. Moreover, it is trained and evaluated on the MASS labels, which has
been annotated by experts whose spindle evaluation varies considerably, and it
is closed-source. Nevertheless, we draw inspiration from their work as a
starting point for our ANN architecture design. In particular, we train models
based on the same idea of using CNNs followed by RNNs, and we evaluate the
relevance of the three different inputs used by SpindleNet (namely, the raw
signal, the signal envelope and the power features) in our setting. We use
PMBO along with the MODA dataset to derive a much smaller architecture, and
provide a quantitative comparison with the SpindleNet architecture on MODA.
Since we do not have access to the SpindleNet model, we rebuild the
architecture described in [29] and train it on the MODA dataset with the same
pipeline that we use to train our models.

####  III-F2 Dataset and training

We use the MODA dataset, with ethics approval, for training our ANN, since its
labels are considerably more reliable than e.g., performing an ‘OR’ operation
on the MASS labels [22]. This dataset is divided in two subsets. The first
one, called _phase 1_ , consists of 100 younger subjects, whereas the second
one, _phase 2_ , consists of 80 older subjects. The MODA dataset provides two
types of annotations (labels) on the signal: the first is the mean score given
by the group of experts for each data point; the second is a binary
classification of each data point as a spindle or non-spindle, defined by a
threshold on the aforementioned scores ($0.2$ for phase 1 and $0.35$ for phase
2). Further post-processing steps were applied to obtain these binary labels:
spindles that were too short ($<0.3$ s) and too close ($<0.1$ s) to each other
were merged, then spindles that were too short ($<0.3$ s) or too long ($>2.5$
s) were relabeled as negative. Given this dataset, two types of ANNs are
possible: classifiers and regressors. These two types of ANNs differ only by
the labels and losses used to train them. Classifiers are trained on the
binary labels, by optimizing the binary cross entropy loss. They directly
predict whether the current signal is a spindle or not, according to the very
specific definition given by these binary labels (i.e., taking into account
the thresholds and post-processing applied by MODA). Regressors are trained on
the score labels, by optimizing the mean square error loss. They predict the
score given by the experts (before the aforementioned post-processing steps),
which allows the user to select their own threshold for detection. Note that,
in practice, classifiers also enable the user to select their own threshold,
although in a less interpretable way. We experiment with both types of models.
Finally, note that MODA is a highly unbalanced dataset as only about 5 % of
the signal is labeled as sleep spindles. During the course of this work, we
tried different ways of balancing training for classifiers and regressors.
Interestingly, we found that classifiers highly benefit from oversampling
(i.e., sampling 50 % of spindles and 50 % of non-spindles from the dataset
during training) whereas all the balancing techniques we tried for regression
(including oversampling, Label Distribution Smoothing [48] and a custom
version of the latter) actually hinder the training.

####  III-F3 Signal processing pipeline

Because we use the SpindleNet architecture as a baseline to evaluate our
approach, we need to compute all three inputs used by this model i.e., a clean
signal, an envelope of the signal, and a “power feature ratio” [29]. The
latter compares frequencies between 2 Hz and 8 Hz with frequencies between 9
Hz and 16 Hz from the Fourier transform over the last 500 ms of signal.
Computing this ratio is resource-intensive in the context of the Portiloop
system, plus we found that this did not help our models. Therefore we compute
this ratio offline for the sole purpose of comparison with the SpindleNet
architecture; it is not used in our model. We set the ADC frequency to 500 Hz,
which allows the Microblaze soft processor to log the raw signal at a finer-
grained resolution than what we actually use for detection. The raw signal is
then downsampled to 250 Hz. Figure 7 depicts the FPGA pipeline that computes
cleaned signal and envelope.

Figure 7: Signal processing pipeline extracting relevant inputs for the ANN.
Power features are computed offline only for the SpindleNet architecture and
are not represented in this diagram.

Since FIR filters introduce software delays, we designed both branches of the
pipeline so that they introduce identical software delays to their respective
outputs (i.e., 40 ms at 250 Hz sampling rate), see Figure 7. We used the same
frequency band to prepare our ‘cleaned signal’ in the first step of the
pipeline as is used in standard sleep scoring (i.e., $0.5$ Hz to 30 Hz) [49].

An FIR filter of order 10 works reasonably well to remove frequencies above 30
Hz, but we observed persistent power line noise. To address this issue, we
apply a notch filter whose frequency depends on the geographical area (50 Hz
in Europe, 60 Hz in North America). For removing low frequencies, we rely on
the online standardization process described in Section III-B2. We use a
coefficient $\alpha_{\mu}=0.1$ for our running average, which can be shown to
attenuate frequencies under 4 Hz. We use a smaller coefficient
$\alpha_{\sigma}=0.001$ for our running variance, meaning that our estimate of
the standard deviation takes a larger portion of the signal into account. We
empirically found this choice of $\alpha_{\mu}$ and $\alpha_{\sigma}$ to
reveal EEG features of interest and yield sensible standardization, by visual
inspection. We apply a similar procedure in the second branch to extract the
envelope. First, the signal is filtered with a FIR band-pass between 12 Hz and
16 Hz. Then, it is standardized like in the first branch, except that we use
$\alpha_{\mu}=\alpha_{\sigma}=0.001$. We then square the signal and smooth the
result by computing its moving average according to Equation 3, this time with
$\alpha_{\mu}=0.01$. This method is much simpler and faster than computing
e.g., an Hilbert envelope. We evaluate different types of ANN architectures,
using either both or only one of these preprocessed signals as input.

####  III-F4 Stimulation

The output of the ANN tells whether the model considers the current signal
being a sleep spindle or not. Some further processing is necessary to ensure
that we only send one stimulation per spindle. As seen in Figure 2, the
detection can be noisy around the beginning or the end of a spindle,
especially since we use decoupled virtual parallel networks (see Section
III-E). A stimulation is sent upon initial spindle detection. To avoid
multiple stimulations of the same spindle the subsequent stimulation may only
occur once the current stimulus ended and at least 400 ms following the ending
of the spindle. If a spindle is detected again within this duration the timer
is reset, since we consider it as being part of the previous spindle.

##  IV Experiments

We devise a Pareto-optimal neural architecture using the PMBO algorithm. We
conduct a thorough quantitative and qualitative study of the resulting system,
not only in terms of ethereal detection scores as generally seen in previous
work, but also in terms of real-time stimulation performance. Note that all
our experiments are based on the MODA dataset rather than actual nights spent
wearing the Portiloop device, as we do not have ground truth labels for such
data. Using the final device would require reproducing the experimental
setting of MASS/MODA, which we leave to experts for future work.

###  IV-A Online detection

All results regarding online detection performance are summarized in Table I.
This table shows the f1, precision and recall metrics that statistically
describe how efficient different models are at detecting sleep spindles (on
average over all data points). These metrics are provided separately for phase
1, which groups younger subjects, for phase 2, which groups older subjects,
and for the whole cohort.

Table I: Quantitative results. The nomenclature is “mean (std)” and the
superscripts are for referencing rows in the text.

| (a) Phase 1 (younger) | (b) Phase 2 (older) | (c) Whole Cohort
---|---|---|---
| Recall | Precision | f1 | Recall | Precision | f1 | Recall | Precision | f1
| Experts
IExp1 | 0.76 (0.16) | 0.81 (0.17) | 0.76 (0.1) | 0.66 (0.19) | 0.74 (0.17) | 0.65 (0.12) | 0.72 (0.18) | 0.78 (0.17) | 0.72 (0.12)
| Offline Detection
Ferrarelli[32] | 0.19 | 0.83 | 0.31 | 0.16 | 0.87 | 0.27 | 0.18 | 0.85 | 0.29
Mölle[33] | 0.83 | 0.47 | 0.6 | 0.78 | 0.44 | 0.56 | 0.81 | 0.46 | 0.58
Martin[34] | 0.61 | 0.64 | 0.62 | 0.58 | 0.56 | 0.57 | 0.6 | 0.6 | 0.6
Wamsley[35] | 0.57 | 0.69 | 0.63 | 0.56 | 0.62 | 0.59 | 0.57 | 0.66 | 0.61
Lacourse[21] | 0.75 | 0.73 | 0.74 | 0.7 | 0.69 | 0.7 | 0.73 | 0.71 | 0.72
Ray[36] | 0.73 | 0.47 | 0.57 | 0.75 | 0.32 | 0.45 | 0.74 | 0.4 | 0.51
Parekh[37] | 0.85 | 0.61 | 0.71 | 0.74 | 0.68 | 0.71 | 0.8 | 0.65 | 0.71
| Online Detection
Based on SpindleNet[29]2 | 0.92 (0.04) | 0.24 (0.07) | 0.38 (0.07) | 0.85 (0.06) | 0.19 (0.08) | 0.3 (0.1) | 0.89 (0.05) | 0.22 (0.07) | 0.35 (0.08)
2-input3 | 0.68 (0.04) | 0.6 (0.06) | 0.64 (0.03) | 0.52 (0.09) | 0.58 (0.04) | 0.54 (0.05) | 0.62 (0.06) | 0.6 (0.05) | 0.61 (0.03)
2-input ablation 14 | 0.7 (0.09) | 0.47 (0.08) | 0.55 (0.04) | 0.56 (0.11) | 0.43 (0.09) | 0.47 (0.04) | 0.65 (0.1) | 0.46 (0.08) | 0.52 (0.04)
2-input ablation 25 | 0.72 (0.03) | 0.57 (0.06) | 0.64 (0.03) | 0.57 (0.08) | 0.53 (0.04) | 0.55 (0.04) | 0.67 (0.04) | 0.56 (0.05) | 0.61 (0.03)
1-input6 | 0.7 (0.04) | 0.59 (0.05) | 0.64 (0.03) | 0.54 (0.09) | 0.58 (0.05) | 0.55 (0.05) | 0.64 (0.05) | 0.59 (0.05) | 0.61 (0.03)
1-input ablation td7 | 0.47 (0.1) | 0.6 (0.09) | 0.51 (0.03) | 0.31 (0.12) | 0.59 (0.08) | 0.39 (0.08) | 0.41 (0.1) | 0.6 (0.09) | 0.47 (0.04)
1-input trained on p18 | 0.72 (0.05) | 0.56 (0.05) | 0.63 (0.03) | 0.57 (0.08) | 0.52 (0.07) | 0.54 (0.05) | 0.66 (0.07) | 0.55 (0.05) | 0.6 (0.03)
1-input trained on p29 | 0.75 (0.05) | 0.5 (0.05) | 0.6 (0.02) | 0.62 (0.09) | 0.45 (0.05) | 0.52 (0.03) | 0.7 (0.06) | 0.49 (0.04) | 0.57 (0.02)
1-input regression10 | 0.62 (0.07) | 0.64 (0.06) | 0.63 (0.03) | 0.53 (0.06) | 0.55 (0.08) | 0.53 (0.04) | 0.58 (0.06) | 0.62 (0.06) | 0.6 (0.03)

####  IV-A1 Experts and offline detectors

As previously highlighted, sleep spindle detection is a difficult task and
experts themselves often do not agree when annotating these offline. This
disagreement is quantified by MODA [22] and represented in Table I, row (1)
for reference. The experts annotating the MODA dataset had an average
performance of $0.72$ on the whole cohort in term of the f1-score of their
individual annotations w.r.t. the final labels. They are compared to other
_offline detection_ , i.e., when a virtually infinite computational budget and
the whole signal is available, including future data points, presented under
“offline detection” in Table I (taken from [22]). We instead perform _online
detection_ , which has additional challenges:  (a) computation happens in real
time;  (b) the future signal is not available.

####  IV-A2 ANN training and evaluation

The MODA dataset is relatively small ($\sim$24 h of annotated data) and
heterogeneous. This adds some difficulty for training and properly assessing
the performance of our models, because we choose to use only 10% of subjects
as our validation set (for model selection), and another 10% of subjects as
our test set (for final model evaluation). Since the overall results are
dependent on the assignment of subjects to the three sets, we evaluate our
models through the following procedure:

  * •

we shuffle all subjects 10 times and compute a different
training/validation/test split of the dataset each time;

  * •

for each split, we use the training set to train 3 models, the validation set
being used to estimate their f1-score. We select the best of these 3 models by
its best f1-score on the validation set. We then report the performance of
this model in terms of its f1-score on the test set;

  * •

the above being repeated 10 times, we report the average test f1-score in
Table I, the corresponding standard deviation being indicated in parenthesis.

####  IV-A3 SpindleNet baseline

The SpindleNet [29] architecture is far too large to be implemented on the
Portiloop, and it cannot run in real time unless using a high-end GPU/TPU
Nevertheless, to the best of our knowledge, it is the state-of-the-art among
previous work for the online detection of sleep spindles. Therefore, we use
this architecture as a baseline for evaluating the performance of our own
models, discovered via PMBO. Since SpindleNet is closed-source and trained on
the MASS dataset, we retrain its architecture from scratch with the same
pipeline as used to train our other classifiers. In particular, we balance
training through oversampling (as opposed to the data augmentation technique
used by the authors of the original paper), and more importantly we train and
evaluate SpindleNet on the MODA dataset. The results of this experiment are
presented in Table I, row (2).

####  IV-A4 2-input model

We first derive a lightweight ANN architecture by drawing inspiration from
SpindleNet. More precisely, we use PMBO to find a Pareto-optimal architecture
that uses both the cleaned signal and the envelope as inputs. The resulting
architecture is presented in Appendix G666Appendices are available online at
https://mistlab.ca/papers/Portiloop/. Despite being small compared to the
architectures commonly encountered in the literature, this architecture is
still far too large to be synthesized in a fully-parallel fashion. Instead,
the HLS compiler partially sequences operations in the FPGA. We measure a
total duration of 40 ms for each forward pass in this model on the Portiloop.
The detection performance of this model, reported in Table I, row (3),
outperforms the baseline. This is not particularly surprising, as the baseline
was originally designed using MASS rather than MODA.

####  IV-A5 Envelope ablation

The idea of using the envelope along the raw signal as input to the ANN is
drawn from the baseline. Since the envelope is computed from the raw signal,
it should not contain any additional information that cannot be extracted by
an ANN in theory; we suspect it can be removed. To evaluate the relevance of
this particular input, we perform the following ablation: to keep the same
architecture (and thus the same model capacity), we replace one of the two
inputs by a copy of the other. In Table I, row (4) both inputs are the
envelope, while in row (5) both inputs are the cleaned signal. We find that
the envelope input can be removed: the model in which we replace the envelope
with a copy of the cleaned signal (5) has the same performance as the original
model (3), and performs marginally better on phase 2.

####  IV-A6 Single-input model

Figure 8: Single-input architecture search with PMBO. The hardware cost is the
number of trainable parameters in the neural architecture, and the software
cost is $1-\text{f1-score of the fully-trained model}$. Black: non-Pareto-
optimal models tested by the algorithm. Red dots: Pareto-optimal models found
by the algorithm. Red line: Pareto front.

Since we deemed the use of the envelope input ineffective in the ablation
study, we use PMBO one more time to devise our final Pareto-optimal ANN
architecture, now with only the cleaned signal as input. For this matter, we
run PMBO on 20 Tesla V100 GPU workers over a period of 24h. The detailed
hyperparameters used in this experiment are provided in Appendix C, and the
results are visualized in Figure 8. PMBO brought the architecture search
towards high-performance lightweight models. We select the best such model in
terms of software cost i.e., the model corresponding to the right-hand end of
the Pareto front, since it is rather small with only 25.6k parameters. We
measure the execution time of this architecture to be 20 ms per forward pass
on the Portiloop (vs. 40 ms for the 2-input version).

Figure 9: Final single-input ANN architecture. The dimensions of each layer
are provided in parenthesis using the PyTorch nomenclature.

The selected architecture is described in Figure 9, and its detection
performance is summarized in Table I, row (6). Compared to our 2-input model,
the single-input model exhibits the same performance, with even a marginal
improvement on phase 2, while executing twice as fast. The detailed
hyperparameters of this model are provided in Appendix D.

####  IV-A7 Time-dilation ablation

We illustrate the importance of using virtual parallelization via time-
dilation. This is done by shrinking the time-dilation (set at $168$ ms by
PMBO) to the minimum, i.e., $20$ ms since this is the execution duration of
the ANN per forward pass. This removes the virtual parallelization, since the
same ANN must now be used for each sample. Therefore, each step of back-
propagation reaches 8 times less far back in time during training. The result
of this ablation is presented in Table I, row (7). The highly deteriorated
results illustrate the importance of time-dilation. This hints at the
relevance of looking relatively far back in time to annotate sleep spindles.

####  IV-A8 Training and evaluating on different phases

We now perform an ablation on the MODA dataset itself to highlight a gap
between the data of phase 1 (younger subjects) and the data of phase 2 (older
subjects). Namely, we train the model on subjects drawn only from phase 1 on
the one hand, and on subjects drawn only from phase 2 on the other hand. The
results of these experiments are presented in Table I, rows (8, 9). We observe
that the ANN trained on phase 1 performs almost as well as the ANN trained on
the whole cohort (6) on all subsets, including phase 2, whereas the ANN
trained on phase 2 is noticeably worse on all subsets, even including phase 2.
We hypothesize that this is because phase 2 has fewer sleep spindle examples,
which are furthermore harder to detect. Using phase 2 during training is still
useful in terms of generalization. Indeed, the ANN trained on phase 1 only (8)
has a slightly worse performance when tested on phase 1 than the ANN trained
on the whole cohort (6).

####  IV-A9 Regression

All models presented beforehand are classifiers. We also train a regressor
with the same architecture, as explained in Section III-F2. There is a subtle
difference in what this model measures when compared to our classifiers:
whereas classifiers predict whether the signal is a sleep spindle according to
the full definition given by MODA (including post-processing), the regressor
predicts the mean score given by the experts (excluding post-processing).
Since we are primarily interested in classification in this article, we find
the threshold that maximizes the f1-score on the binary labels. This
experiment is presented in Appendix E. We find that the optimal threshold is
$0.27$ for phase 1, $0.23$ for phase 2 and $0.26$ for the whole cohort. We
then evaluate the regressor with these thresholds on the classification task
and report the results in Table I, row (10). These results are slightly weaker
than those of the classifier (6). We surmise that this effect comes from the
post-processing steps performed by MODA to compute the binary labels. We
choose the 1-input classifier (6) for the remainder of this article.

###  IV-B Real-time stimulation

The performance measured in Section IV-A is not entirely representative of the
performance on the final task. So far, we have only measured the capability of
the model to annotate each data point of the signal individually. Yet, we want
the ability to send one single stimulation per sleep spindle.

####  IV-B1 Qualitative results

Figure 10: Stimulation with a 0.5 threshold. The color code is the same as
Figure 2, with false positives being additionally displayed in red. Note the
first part of the spindle is not detected (false negative), which introduces
an ANN software delay. In addition, a small portion of the signal after the
spindle is still detected as such (false positive), but it has no impact on
our stimulation procedure. Finally, another portion of the signal that was not
annotated as a spindle by MODA is detected as a spindle by our model (false
positive), generating an undesirable stimulus is generated.

Figure 10 shows an example of imperfect sleep spindle detection and
stimulation using our ANN (see Appendix B for more details on failure modes).

Figure 11: Stimulation with a 0.84 threshold.

Although we chose using a classifier as opposed to a regressor, it is still
possible to reduce incorrect stimulation by fine-tuning the detection
threshold ($0.5$ by default). Figure 11 shows an increased threshold reduces
the incorrect stimulation. However, a high threshold worsens the detection
delay since it increases the number of false negatives at spindle onset.

####  IV-B2 Quantitative results

Although the results presented in Section IV-B1 focus on the ANN detection
delay, we must take in consideration the other sources of delay, i.e., the
software delay from FIRs (40 ms), the ANN forward pass duration (20 ms) and
the stimulation hardware delay, to measure our real stimulation performance.
The auditory stimulation delay in the Portiloop is 4 ms, for a total constant
delay of 64 ms.

From now on, we redefine:  (a) True positive: the first stimulus sent within
the duration of a spindle, taking this additional delay into account;  (b)
False positive: any other stimulus;  (c) False negative: any spindle that does
not receive a stimulus within its labeled duration.

Figure 12: Evolution of the actual stimulation performance w.r.t. the chosen
detection threshold.

Figure 12 displays the detection performance (taking all delays into account)
of our final device. We compute the stimulation precision, recall and f1-score
according to the aforementioned definitions of true positives, false positives
and false negatives. This provides a visualization of possible trade-offs in
terms of how many spindles we want to stimulate (recall) versus how sure we
want to be that all stimuli are relevant (precision). In terms of f1-score,
the best such trade-off is attained at a threshold of $0.84$ with our model,
giving a precision and a recall of $0.71$.

Figure 13: Distribution of stimulation delays for a classifier with 0.5 and
0.84 threshold. Delays are negative when spindles are stimulated in advance.

We further study the timing performance of our system. Figure 13 displays the
distribution of stimulation delays, i.e., the distribution of the stimulus
being closest to the beginning of each sleep spindle, all delays being taken
into account. Some of these delays are negative, as spindles are sometimes
stimulated in advance (NB: we count these as false positives, which slightly
harms our reported results). Moreover it shows the effect of increasing the
detection threshold of our model on the stimulation delays. According to
Figure 12, choosing a $0.84$ detection threshold over the $0.5$ default
classification threshold in our ANN yields a better stimulation f1-score and
in particular much more precise stimuli, but this comes at the price of
slightly shifting the stimulation delay distribution to the right, i.e.,
introducing some additional delay to the stimulation, as previously seen in
Figure 11.

Finally, we estimate the Portiloop energy efficiency by running our final
sleep spindle stimulation device continuously, powered by a fully-charged
20000 mAh battery. The battery dies out after 26 hours and 22 minutes,
suggesting that our power consumption is roughly 756 mA.

##  V Discussion and future work

The Portiloop system can be adapted to any application of EEG closed-loop
stimulation. Although classifiers are most relevant in our case study, it
would be straightforward to extend the Portiloop to quantitative tasks using
regressors instead, as seen in Section IV-A9.

As opposed to classical heuristics, our deep learning-based approach does not
require specific knowledge of the phenomenon of interest when defining the
classifier, nor requires a way to extract the relevant information. Instead, a
large dataset of annotated signal suffices to derive a high-performance model
that detects complex patterns such as sleep spindles. Once trained, techniques
such as Integrated Gradients [50] can be used to better understand and fine-
tune the ANN. Moreover, these techniques may help experts by revealing unknown
dependencies in neural activity, as we hint in Appendix A.

We have introduced PMBO to derive Pareto-optimal architectures in a parallel
fashion. Although the algorithm produces high-performance lightweight
architectures, we note that the predictions of the meta-learner are often
near-constant in well-performing areas of the search space, suggesting that
the meta-model could not further predict the software cost. We surmise that
this is due to the large variance in model performance from one training
session to another. This might be alleviated by training the same model
several times, but we forbade this in our implementation to speed up the
search.

Although we compare our architecture to the SpindleNet architecture, we did
not have access to their weights and thus we could not compare their original
model with ours on the MODA dataset. Instead, we retrained their architecture
from scratch on MODA, using our own pipeline. Contrary to Kulkarni et al.
[29], we could remove the envelope and power inputs without harming the
performance of our models. During the course of our work, we have noticed that
our 1-input model was less robust to shrinking the time-dilation than our
2-input model. Since the baseline does not use time-dilation, it cannot reach
far back in time. We believe that this is important for the model to retrieve
the information contained in these additional inputs. Note that it is
necessary not to reduce the capacity of the model (e.g., by cutting a branch
of the ANN instead of duplicating inputs) when performing the ablation study,
which Kulkarni et al. might have overlooked.

While the MODA dataset provides high-quality labels, training on a bigger
dataset of similar quality would likely further improve the performance of our
models. Expanding MODA is a relevant avenue for future work, as is
implementing sim-to-real transfer, because the EEG acquisition and signal may
differ somewhat from the training data. Transfer can be achieved with
techniques such as domain randomization [51]. Alternatively, a dataset can be
collected on the target device and annotated following the same protocol as
MODA.

Long term, we intend to target specific regions of sleep spindles for
stimulation. This harder task will likely involve labeling these regions and
developing more advanced RNNs/Transformers so as to consistently predict sleep
spindles before they even start. Although our model does use information far
back in time to make predictions (see Appendix A), we believe that the main
role currently played by the RNN is to accumulate information regarding
whether the last few windows were spindles or not, rather than actually
predicting the future (see Appendix B). Such models will likely be more
complex and computationally hungry. We will integrate a TPU in the system, as
part of an ad-hoc PCB that will replace the prototyping board currently used,
further miniaturizing the Portiloop. In general, finding an optimal model for
a given Portiloop application involves either retraining our ANN, or re-
executing PMBO with a new architecture. Both activities can be done by
interested practitioners.

##  VI Conclusion

In this article, we introduce the Portiloop, a device that enables the real-
time detection and stimulation of patterns of interest in EEG signal. Our
system is open-source, portable, low-cost, and can be tailored for many such
applications. The Portiloop enables implementing task-specific deep learning-
based detectors directly in efficient hardware circuitry. This is made
possible via an FPGA and a C++ HLS library that we open-source along with the
rest of our code. We propose a pipeline to design neural architectures that
are relevant for processing EEG signal in real time. We further propose PMBO,
an algorithm that automates the process of finding efficient such models,
using one to many parallel workers. We demonstrate our proposed system on the
closed-loop stimulation of sleep spindles, a difficult task of high relevance
for the neuroscience community. Our resulting system is the first portable
device to be able to detect and stimulate sleep spindles in real time with a
f1-score of $0.71$, measured on MODA, a dataset renowned for the reliability
of its labels. In the long range, the Portiloop will help the neuroscience
community non-invasively explore brain functions, such as the role of sleep
spindles in memory consolidation.

## Acknowledgment

We thank Karine Lacourse for expert advice on spindle detection, and the MODA
team for database access. Figures 1 and 3 use icons from https://flaticon.com.

## References

  * [1] H.-V. V. Ngo, T. Martinetz, J. Born, and M. Mölle, “Auditory Closed-Loop Stimulation of the Sleep Slow Oscillation Enhances Memory,” _Neuron_ , vol. 78, no. 3, pp. 545–553, May 2013\.
  * [2] U. Ha and H.-J. Yoo, “A multimodal drowsiness monitoring ear-module system with closed-loop real-time alarm,” in _2016 IEEE Biomedical Circuits and Systems Conference (BioCAS)_ , Oct. 2016, pp. 536–539.
  * [3] A. von Lühmann, J. Addesa, S. Chandra, A. Das, M. Hayashibe, and A. Dutta, “Neural interfacing non-invasive brain stimulation with NIRS-EEG joint imaging for closed-loop control of neuroenergetics in ischemic stroke,” in _2017 8th International IEEE/EMBS Conference on Neural Engineering (NER)_ , May 2017, pp. 349–353.
  * [4] N. Kosmyna and P. Maes, “Attentivu: An EEG-based closed-loop biofeedback system for real-time monitoring and improvement of engagement for personalized learning,” _Sensors_ , vol. 19, no. 23, 2019.
  * [5] G. Zarubin, C. Gundlach, V. Nikulin, and M. Bogdan, “Real-time phase detection for EEG-based tACS closed-loop system,” in _6th International Congress on Neurotechnology, Electronics and Informatics_ , Seville, Spain, 2018, pp. 13–20.
  * [6] S. Shirinpour, I. Alekseichuk, K. Mantell, and A. Opitz, “Experimental evaluation of methods for real-time EEG phase-specific transcranial magnetic stimulation,” _J. Neural Engineering_ , vol. 17, no. 4, 2020.
  * [7] C. Zrenner, P. Belardinelli, F. Müller-Dahlhaus, and U. Ziemann, “Closed-loop neuroscience and non-invasive brain stimulation: a tale of two loops,” _Frontiers in cellular neuroscience_ , vol. 10, p. 92, 2016.
  * [8] A. Vassileva, D. van Blooijs, F. Leijten, and G. Huiskamp, “Neocortical electrical stimulation for epilepsy: Closed-loop versus open-loop,” _Epilepsy research_ , vol. 141, pp. 95–101, 2018.
  * [9] J. Choi, M. Kwon, and S. C. Jun, “A systematic review of closed-loop feedback techniques in sleep studies—related issues and future directions,” _Sensors_ , vol. 20, no. 10, p. 2770, 2020.
  * [10] K. D. Fehér, M. Wunderlin, J. G. Maier, E. Hertenstein, C. Schneider, C. Mikutta, M. A. Züst, S. Klöppel, and C. Nissen, “Shaping the slow waves of sleep: A systematic and integrative review of sleep slow wave modulation in humans using non-invasive brain stimulation,” _Sleep medicine reviews_ , p. 101438, 2021.
  * [11] F. Salfi, A. D’Atri, D. Tempesta, L. De Gennaro, and M. Ferrara, “Boosting slow oscillations during sleep to improve memory function in elderly people: A review of the literature,” _Brain Sciences_ , vol. 10, no. 5, p. 300, 2020\.
  * [12] M. O. Harrington and S. A. Cairney, “Sounding it out: auditory stimulation and overnight memory processing,” _Current Sleep Medicine Reports_ , 2021.
  * [13] T. O. Bergmann and B. P. Staresina, “Neuronal oscillations and reactivation subserving memory consolidation,” in _Cognitive neuroscience of memory consolidation_. Springer, 2017, pp. 185–207.
  * [14] B. Rasch and J. Born, “About sleep’s role in memory,” _Physiological reviews_ , 2013.
  * [15] S. M. Fogel and C. T. Smith, “Learning-dependent changes in sleep spindles and stage 2 sleep,” _Journal of sleep research_ , vol. 15, no. 3, pp. 250–255, 2006.
  * [16] M. Lafortune, J.-F. Gagnon, N. Martin, V. Latreille, J. Dubé, M. Bouchard, C. Bastien, and J. Carrier, “Sleep spindles and rapid eye movement sleep as predictors of next morning cognitive performance in healthy middle-aged and older participants,” _Journal of sleep research_ , vol. 23, no. 2, pp. 159–167, 2014.
  * [17] S. Fogel, C. Vien, A. Karni, H. Benali, J. Carrier, and J. Doyon, “Sleep spindles: a physiological marker of age-related changes in gray matter in brain regions supporting motor skill memory consolidation,” _Neurobiology of aging_ , vol. 49, pp. 154–164, 2017.
  * [18] L. M. J. Fernandez and A. Lüthi, “Sleep Spindles: Mechanisms and Functions,” _Physiological Reviews_ , vol. 100, no. 2, pp. 805–868, 2020\.
  * [19] K. R. Peters, L. B. Ray, S. Fogel, V. Smith, and C. T. Smith, “Age differences in the variability and distribution of sleep spindle and rapid eye movement densities,” _PloS one_ , vol. 9, no. 3, p. e91047, 2014.
  * [20] S. Purcell, D. Manoach, C. Demanuele, B. Cade, S. Mariani, R. Cox, G. Panagiotaropoulou, R. Saxena, J. Pan, J. Smoller _et al._ , “Characterizing sleep spindles in 11,630 individuals from the national sleep research resource,” _Nature communications_ , vol. 8, no. 1, pp. 1–16, 2017\.
  * [21] K. Lacourse, J. Delfrate, J. Beaudry, P. Peppard, and S. C. Warby, “A sleep spindle detection algorithm that emulates human expert spindle scoring,” _J. Neuroscience Methods_ , vol. 316, pp. 3–11, Mar. 2019.
  * [22] K. Lacourse, B. Yetton, S. Mednick, and S. C. Warby, “Massive online data annotation, crowdsourcing to generate high quality sleep spindle annotations from EEG data,” _Sci Data_ , vol. 7, no. 1, p. 190, Jun. 2020.
  * [23] P. Albouy, A. Weiss, S. Baillet, and R. J. Zatorre, “Selective entrainment of theta oscillations in the dorsal stream causally enhances auditory working memory performance,” _Neuron_ , vol. 94, no. 1, pp. 193–206, 2017.
  * [24] C. M. McCrimmon, J. L. Fu, M. Wang, L. S. Lopes, P. T. Wang, A. Karimi-Bidhendi, C. Y. Liu, P. Heydari, Z. Nenadic, and A. H. Do, “Performance Assessment of a Custom, Portable, and Low-Cost Brain-Computer Interface Platform,” _IEEE Transactions on Biomedical Engineering_ , vol. 64, no. 10, pp. 2313–2320, 2017\.
  * [25] S. Zotou, G. K. Kostopoulos, and T. A. Antonakopoulos, “Real-time Spindles Detection for Acoustic Neurofeedback,” in _Brain Function Assessment in Learning_ , ser. Lecture Notes in Computer Science, C. Frasson and G. Kostopoulos, Eds. Cham: Springer International Publishing, 2017, pp. 159–168.
  * [26] H.-V. V. Ngo, M. Seibold, D. C. Boche, M. Mölle, and J. Born, “Insights on auditory closed-loop stimulation targeting sleep spindles in slow oscillation up-states,” _Journal of Neuroscience Methods_ , vol. 316, pp. 117–124, Mar. 2019.
  * [27] L. L. Chen, R. Madhavan, B. I. Rapoport, and W. S. Anderson, “Real-time brain oscillation detection and phase-locked stimulation using autoregressive spectral estimation and time-series forward prediction,” _IEEE Trans. on Biomed. Eng._ , vol. 60, no. 3, pp. 753–762, 2013.
  * [28] R. Xu, N. Jiang, C. Lin, N. Mrachacz-Kersting, K. Dremstrup, and D. Farina, “Enhanced low-latency detection of motor intention from EEG for closed-loop brain-computer interface applications,” _IEEE Transactions on Biomedical Engineering_ , vol. 61, no. 2, pp. 288–296, 2014.
  * [29] P. M. Kulkarni, Z. Xiao, E. J. Robinson, A. S. Jami, J. Zhang, H. Zhou, S. E. Henin, A. A. Liu, R. S. Osorio, J. Wang, and Z. Chen, “A deep learning approach for real-time detection of sleep spindles,” _J. Neural Eng._ , vol. 16, no. 3, p. 036004, Jun. 2019.
  * [30] N. I. Tapia and P. A. Estévez, “RED: Deep Recurrent Neural Networks for Sleep EEG Event Detection,” _arXiv:2005.07795_ , May 2020\.
  * [31] C. O’Reilly, N. Gosselin, J. Carrier, and T. Nielsen, “Montreal Archive of Sleep Studies: An open-access resource for instrument benchmarking and exploratory research,” _Journal of Sleep Research_ , vol. 23, no. 6, pp. 628–635, 2014.
  * [32] F. Ferrarelli, R. Huber, M. J. Peterson, M. Massimini, M. Murphy, B. A. Riedner, A. Watson, P. Bria, and G. Tononi, “Reduced Sleep Spindle Activity in Schizophrenia Patients,” _AJP_ , vol. 164, no. 3, pp. 483–492, Mar. 2007.
  * [33] M. Mölle, L. Marshall, S. Gais, and J. Born, “Grouping of Spindle Activity during Slow Oscillations in Human Non-Rapid Eye Movement Sleep,” _J. Neurosci._ , vol. 22, no. 24, pp. 10 941–10 947, Dec. 2002\.
  * [34] N. Martin, M. Lafortune, J. Godbout, M. Barakat, R. Robillard, G. Poirier, C. Bastien, and J. Carrier, “Topography of age-related changes in sleep spindles,” _Neurobiology of Aging_ , vol. 34, no. 2, pp. 468–476, Feb. 2013.
  * [35] E. J. Wamsley, M. A. Tucker, A. K. Shinn, K. E. Ono, S. K. McKinley, A. V. Ely, D. C. Goff, R. Stickgold, and D. S. Manoach, “Reduced Sleep Spindles and Spindle Coherence in Schizophrenia: Mechanisms of Impaired Memory Consolidation?” _Biological Psychiatry_ , vol. 71, no. 2, pp. 154–161, Jan. 2012.
  * [36] L. Ray, S. Sockeel, M. Soon, A. Bore, A. Myhr, B. Stojanoski, R. Cusack, A. M. Owen, J. Doyon, and S. Fogel, “Expert and crowd-sourced validation of an individualized sleep spindle detection method employing complex demodulation and individualized normalization,” _Front. Hum. Neurosci._ , vol. 9, 2015.
  * [37] A. Parekh, I. W. Selesnick, D. M. Rapoport, and I. Ayappa, “Detection of K-complexes and sleep spindles (DETOKS) using sparse optimization,” _Journal of Neuroscience Methods_ , vol. 251, pp. 37–46, Aug. 2015.
  * [38] N. Yasuhara, T. Natori, M. Hayashi, and N. Aikawa, “A Study on Automatic Detection of Sleep Spindles using a Long Short-Term Memory Network,” in _2019 IEEE 62nd International Midwest Symposium on Circuits and Systems (MWSCAS)_ , Aug. 2019, pp. 45–48.
  * [39] D. Tan, R. Zhao, J. Sun, and W. Qin, “Sleep spindle detection using deep learning: A validation study based on crowdsourcing,” in _37th IEEE Engineering in Medicine and Biology Conference (EMBC)_ , Aug. 2015, pp. 2828–2831.
  * [40] _Pynq Z2 Reference Manual_ , TUL, 2019, rev. 1.1.
  * [41] _ADS1299-x Low-Noise, 4-, 6-, 8-Channel, 24-Bit, Analog-to-Digital Converter for EEG and Biopotential Measurements datasheet_ , Texas Instrument, 2012, rev. C (2017).
  * [42] M. Vohra and S. Fasciani, “PYNQ-Torch: A framework to develop PyTorch accelerators on the PYNQ platform,” in _IEEE International Symposium on Signal Processing and Information Technology (ISSPIT)_ , Dec. 2019, pp. 1–6.
  * [43] J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling,” _arXiv:1412.3555 [cs]_ , Dec. 2014.
  * [44] S. Yang, X. Yu, and Y. Zhou, “LSTM and GRU Neural Network Performance Comparison Study: Taking Yelp Review Dataset as an Example,” in _2020 International Workshop on Electronic Communication and Artificial Intelligence (IWECAI)_ , Jun. 2020, pp. 98–101.
  * [45] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in _Advances in neural information processing systems_ , 2017, pp. 5998–6008.
  * [46] Z. Yin, W. Gross, and B. H. Meyer, “Probabilistic sequential multi-objective optimization of convolutional neural networks,” in _2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)_. IEEE, 2020, pp. 1055–1060.
  * [47] S. Chang, Y. Zhang, W. Han, M. Yu, X. Guo, W. Tan, X. Cui, M. Witbrock, M. Hasegawa-Johnson, and T. S. Huang, “Dilated recurrent neural networks,” _arXiv preprint arXiv:1710.02224_ , 2017.
  * [48] Y. Yang, K. Zha, Y.-C. Chen, H. Wang, and D. Katabi, “Delving into deep imbalanced regression,” _arXiv preprint arXiv:2102.09554_ , 2021.
  * [49] C. Iber, S. Ancoli-Israel, A. L. Chesson, S. F. Quan _et al._ , _The AASM manual for the scoring of sleep and associated events: rules, terminology and technical specifications_. American academy of sleep medicine Westchester, IL, 2007, vol. 1.
  * [50] M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in _International Conference on Machine Learning_. PMLR, 2017, pp. 3319–3328.
  * [51] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” in _International conference on intelligent robots and systems (IROS)_. IEEE, 2017, pp. 23–30.

Supplementary Material
The Portil$\infty$p: a deep learning-based open science tool for closed-loop
brain stimulation

##  Appendix A Model-based explanation of sleep spindles

Figure 14: Integrated gradients (classifier ANN output: $0.136$). The
_integrated gradients_ algorithm enables exploring why the model takes a given
decision (the more a portion of the signal is represented in red, the highest
its influence on the current output of the ANN). Grey windows are past inputs,
whereas the black window is the current input: the past influences the current
output due to the RNN. Here, the model finds that it is looking at the
aftermath of a sleep spindle. With our time dilation and window size, a small
portion of the window overlaps from one sample to the next. We see that this
portion (at the left-hand side of each window) is in fact ignored by the
model. Therefore, it is probably possible to shrink our model even more,
although PMBO did not find this. In the future, this type of visualizations
might also help experts better understand what sleep spindles are by revealing
unknown influences. Figure 15: Integrated gradients (classifier ANN output:
$0.639$). The current window is within an actual sleep spindle. The model
mainly focuses on the spindle itself, but also a few events that happened
further back in time, to make its decision.

##  Appendix B Stimulation visualizations

Figure 16: Different stimulation failure modes (classifier with threshold
0.5). Blue: no sleep spindle and no detection. Black: sleep spindle not
detected. Green: sleep spindle correctly detected. Red: detection where the
signal is not a spindle. Vertical blue: beginning of a spindle. Vertical grey:
stimulation. Magenta: ANN output. Horizontal gey: detection threshold. This
figure illustrates typical ‘failure’ cases of our final sleep spindle
stimulating device. False negative: the first spindle is missed because the
threshold is too big. True positive: the second spindle is correctly
stimulated. False positive: a part of the signal not labeled as a sleep
spindle by MODA is detected as a spindle by the device and stimulated. Almost
true positive: the sleep spindle is stimulated in advance (NB: we count this
case as a false positive). Figure 17: Classifier with threshold 0.84, success
example. Same color code as Figure 16. Increasing the detection threshold from
$0.5$ to $0.84$ removes most false positive stimuli (vertical grey not
following vertical blue). Figure 18: Classifier with threshold 0.84, failure
example. Same color code as Figure 16. Increasing the detection threshold
comes with more false negative stimuli (vertical blue not followed by vertical
grey).

##  Appendix C PMBO hyperparameters

Table II: Hyperparameters used for PMBO hyperparameter | selected value
---|---
range cost hardware | 1000-80000
noise type 1 | 0,25
noise type 2 | 0,1
m | 200
meta network type | MLP
# layers meta network | 3
hidden size meta network | 200
optimizer meta network | SGD
learning rate meta network | 0.05
weight decay meta network | 0.01

Two types of noise are used in our implementation of PMBO to foster
exploration of the hyperparameter space:

  * •

noise type 1: portion of the m sampled models that are sampled randomly in the
whole hyperparameter space, instead of in a Gaussian around the last completed
experiment.

  * •

noise type 2: portion of the time when a model is sampled randomly in the m
models, instead of being selected by its Pareto efficiency.

The ANN used for our meta model is a simple Multi-Layer Perceptron (MLP) of 3
fully connected layers. The hyperparameters we use in PMBO are summarized in
Table II.

##  Appendix D Model hyperparameters

Table III: Hyperparameters used to train the final model Hyperparameter | Selected value | PMBO
---|---|---
Training
optimizer | AdamW |
# epochs max | 150 |
epochs before early stopping | 20 |
early stopping running average factor | 0,1 |
batches per epoch | 1000 |
batch size | 256 | X
dropout on first layer | False |
dropout factor | 0,5 |
adam learning rate | 0,0005 | X
adam weight decay | 0,01 |
balancing mode | oversampling |
type of training | classification |
sequence length | 50 |
Architecture
window size (s) | 0,216 | X
time dilation (s) | 0,168 | X
# CNN layers | 3 | X
# CNN channels | 31 | X
stride convolution | 1 | X
kernel size convolution | 7 | X
dilation convolution | 1 | X
stride max pooling | 1 | X
kernel size max pooling | 1 | X
dilation max pooling | 1 | X
RNN layers | 1 | X
RNN hidden size | 7 | X

The hyperparameters we use in our final model are listed in Table III.
Hyperparameters that were chosen by PMBO are marked with a cross under the
PMBO column.

##  Appendix E Best threshold for classifiers and regressors

Figure 19: F1 score evolution with threshold variation on classification
Figure 20: F1 score evolution with threshold variation on regression

##  Appendix F Detection delay distributions

Figure 21: Stimulation histogram for 1 input network classification with a
0.25 threshold

##  Appendix G 2-input network architecture

Figure 22: 2-input neural network architecture. This architecture consists of
two sequences of CNN and RNN. The first sequence processes the cleaned raw
signal (input 1), whereas the second processes the envelope (input 2). The
latent features from both branches are concatenated and fed to a fully
connected layer to yield the output of the ANN.