11institutetext: Intelligent Science & Technology Academy of CASIC, Beijing
100854, China
11email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
# Real Spike: Learning Real-valued Spikes for Spiking Neural Networks

Yufei Guo Equal contribution. Liwen Zhang⋆ Yuanpei Chen  Xinyi Tong  Xiaode
Liu  YingLei Wang  Xuhui Huang✉ Zhe Ma✉

###### Abstract

Brain-inspired spiking neural networks (SNNs) have recently drawn more and
more attention due to their event-driven and energy-efficient characteristics.
The integration of storage and computation para-digm on neuromorphic hardwares
makes SNNs much different from Deep Neural Networks (DNNs). In this paper, we
argue that SNNs may not benefit from the weight-sharing mechanism, which can
effectively reduce parameters and improve inference efficiency in DNNs, in
some hardwares, and assume that an SNN with unshared convolution kernels could
perform better. Motivated by this assumption, a training-inference decoupling
method for SNNs named as Real Spike is proposed, which not only enjoys both
unshared convolution kernels and binary spikes in inference-time but also
maintains both shared convolution kernels and Real-valued Spikes during
training. This decoupling mechanism of SNN is realized by a re-
parameterization technique. Furthermore, based on the training-inference-
decoupled idea, a series of different forms for implementing Real Spike on
different levels are presented, which also enjoy shared convolutions in the
inference and are friendly to both neuromorphic and non-neuromorphic hardware
platforms. A theoretical proof is given to clarify that the Real Spike-based
SNN network is superior to its vanilla counterpart. Experimental results show
that all different Real Spike versions can consistently improve the SNN
performance. Moreover, the proposed method outperforms the state-of-the-art
models on both non-spiking static and neuromorphic datasets.

##  1 Introduction

Figure 1: The difference of convolution computing process between DNNs and
SNNs. For DNNs, the calculation is conducted in a highly-paralleled way on
conventional hardwares. However, for SNNs, each connection between neurons
will be mapped into a synapse on some neuromorphic hardwares, which cannot
benefit from the advantages of the weight-shared convolution kernel, e.g.,
inference acceleration and parameter reduction.

Spiking Neural Networks (SNNs) have received increasing attention as a novel
brain-inspired computing model that adopts binary spike signals to communicate
between units. Different from the Deep Neural Networks (DNNs), SNNs transmit
information by spike events, and the computation dominated by the addition
operation occurs only when the unit receives spike events. Benefitting from
this characteristic, SNNs can greatly save energy and run efficiently when
implementing on neuromorphic hardwares, e.g., SpiNNaker [17], TrueNorth [1],
Darwin [28], Tianjic [32], and Loihi [5].

The success of DNNs inspires the SNNs in many ways. Nonetheless, the rich
spatio-temporal dynamics, event-driven paradigm, and friendly to neuromorphic
hardwares make SNNs much different from DNNs, and directly applying the
successful experience of DNNs to SNNs may limit the performance of SNNs. As
one of the most widely used techniques in DNNs, the weight-shared convolution
kernel shows great advantages. It can reduce the parameters of the network and
accelerate the inference. However, SNNs show great advantages in the condition
of being implemented on neuromorphic hardwares which is very different from
DNNs being implemented on deep learning hardwares. As shown in Fig. 1, in
DNNs, the calculation is carried out in a highly-paralleled way on deep
learning hardwares, thus sharing convolution kernel can improve the computing
efficiency by reducing the data transferring between separated memory and
processing units. However, for an ideal situation, to take full advantage of
the storage-computation-integrated paradigm of neuromorphic hardwares, each
unit and connection of the SNNs in the inference phase should be mapped into a
neuron and synapse in neuromorphic hardware, respectively. Though these
hardwares could be multiplexed, it also increases the complexity of
deploymention and extra cost of data transfer. As far as we know, at least
Darwin [28], Tianjic [32], and other memristor-enabled neuromorphic computing
systems [41] adopt this one-to-one mapping form at present. Hence, all the
components of an SNN will be deployed as a fixed configuration on these
hardwares, no matter they share the same convolution kernel or not. Unlike the
DNNs, the shared convolution kernels will not bring SNNs the advantages of
parameter reduction and inference acceleration in this situation. Hence we
argue that it would be better to learn unshared convolution kernels for each
output feature map in SNNs.

Unfortunately, whether in theory or technology, it is not feasible to directly
train an unshared convolution kernels-based SNN. First, there is no obvious
proof that learning different convolution kernels directly will surely benefit
the network performance. Second, due to the lack of mature development
platforms for SNNs, many efforts are focusing on training SNNs with DNN-
oriented programming frameworks, which usually do not support the learning of
unshared convolution kernels for each feature map directly. Considering these
limitations, we focus on training SNNs with unshared convolution kernels based
on the modern DNN-oriented frameworks indirectly.

Driven by the above reasons, a training-time and inference-time decoupled SNN
is proposed, where a neuron can emit _real-valued spikes_ during training but
binary spikes during inference, dubbed Real Spike. The training-time real-
valued spikes can be converted to inference-time binary spikes via convolution
kernel re-parameterization and a shared convolution kernel, which can be
derived into multiples then (see details in Sec. 3.3). In this way, an SNN
with different convolution kernels for every output feature map can be
obtained as we expected. Specifically, in the training phase, the SNN will
learn real-valued spikes and a shared convolution kernel for every output
feature map. While in the inference phase, every real-valued spike will be
transformed into a binary spike by folding a part of the value to its
corresponding kernel weight. Due to the diversity of the real-valued spikes,
by absorbing part of the value from each real spike, the original convolution
kernel shared by each output map can be converted into multiple forms. Thus
different convolution kernels for each feature map of SNNs can be obtained
indirectly. It can be guaranteed theoretically that the Real Spike method can
improve the performance due to the richer representation capability of real-
valued spikes than binary spikes (see details in Sec. 3.4). Besides, Real
Spike is well compatible with present DNN-oriented programming frameworks, and
it still retains the advantages of DNN-oriented frameworks in terms of the
convolution kernel sharing mechanism in the training. Furthermore, we extract
the essential idea of training-inference-decoupled and extend Real Spike to a
more generalized form, which is friendly to both neuromorphic and non-
neuromorphic hardwares (see details in Sec. 3.5). The overall workflow of the
proposed method is illustrated in Fig. 2.

Figure 2: The overall workflow of Real Spike. In the training phase, the SNN
learns real-valued spikes and the shared convolution kernel for each output
feature map. In the inference phase, the real spikes can be converted to
binary spikes via convolution kernel re-parameterization. Then an SNN with
different convolution kernels for every feature map is obtained.

Our main contributions are summarized as follows:

  * •

We propose the Real Spike, a simple yet effective method to obtain SNNs with
unshared convolution kernels. The Real Spike-SNN can be trained in DNN-
oriented frameworks directly. It can effectively enhance the information
representation capability of the SNN without introducing training difficulty.

  * •

The convolution kernel re-parameterization is introduced to decouple a
training-time SNN with real-valued spikes and shared convolution kernels, and
an inference-time SNN with binary spikes and unshared convolution kernels.

  * •

We extend the Real Spike to other different granularities (layer-wise and
channel-wise). These extensions can keep shared convolution kernels in the
inference and show advantages independent of specific hardwares.

  * •

The effectiveness of Real Spike is verified on both static and neuromorphic
datasets. Extensive experimental results show that our method performs
remarkably.

##  2 Related Work

This work starts from training more accurate SNNs with unshared convolution
kernels for each feature map. Considering the lack of specialized suitable
platforms that can support the training of deep SNNs, powerful DNN-oriented
programming frameworks are adopted. To this end, we briefly overview recent
works of SNNs in three aspects: (i) learning algorithms of SNNs; (ii) SNN
programming frameworks; (iii) convolutions.

###  2.1 Learning Algorithms of SNNs

The learning algorithms of SNNs can be divided into three categories:
converting ANN to SNN (ANN2SNN) [2, 13, 36, 25], unsupervised learning [6,
14], and supervised learning [16, 29, 26, 37, 12]. ANN2SNN converts a pre-
trained ANN to an SNN by transforming the real-valued output of the activation
function to binary spikes. Due to the success of ANNs, ANN2SNN can generate an
SNN in a short time with competitive performance. However, the converted ANN
inherits the limitation of ignoring rich temporal dynamic behaviors from DNNs,
it cannot handle neuromorphic datasets well. On the other hand, ANN2SNN
usually requires hundreds of timesteps to approach the accuracy of pre-trained
DNNs. Unsupervised learning is considered a more biologically plausible
method. Unfortunately, the lack of sufficient understanding of the biological
mechanism prevents the network from going deep, thus it is usually limited to
small datasets and non-ideal performance. Supervised learning trains SNNs with
error backpropagation. Supervised learning-based methods can not only achieve
high accuracy within very few timesteps but also handle neuromorphic datasets.
Our work falls under this category.

###  2.2 SNN Programming Frameworks

There exist several specialized programming frameworks for SNN modeling.
However, most of them cannot support the direct training of deep SNNs. NEURON
[3], a simulation environment for modeling computational models of neurons and
networks of neurons, mainly focuses on simulating neuron issues with complex
anatomical and physiological characteristics, which is more suitable for
neuroscience research. BRIAN2 [11] and NEST [10], simulators for SNNs, aim at
making the writing of simulation code as quick and easy as possible for the
user, but they are not designed for the supervised learning for deep SNNs.
SpikingJelly [7], an open-source deep learning framework for SNNs based on
PyTorch, provides a solution to establish SNNs by mature DNN-oriented
programming frameworks. However, so far most functions of the SpikingJelly are
still under development. Instead of developing a new framework, we attempt to
investigate an ingenious solution to develop the SNNs that have unshared
convolutional kernels for each output feature map in the DNN-oriented
frameworks, which have demonstrated an easy-to-use interface.

###  2.3 Convolutions

Convolutional Neural Networks have recently harvested a huge success in large-
scale computer vision tasks [22, 20, 38, 39, 15, 35]. Due to the abilities of
input scale adaptation, translation invariance, and parameter reduction, the
stacked convolution layers help train a deep and well-performed neural network
with fewer parameters compared to dense-connected fully connected layers. For
a convolution layer, feature maps of the previous layer are convolved with
some learnable kernels and presented through the activation function to form
the output feature maps as the current layer. Each learnable kernel is
corresponding to an output map. In general, we have that

$\textbf{X}_{i,j}=f(\textbf{I}_{j}*\textbf{K}_{i}),$ (1)

where $\textbf{I}_{j}\in\mathbb{R}^{k\times k\times C}$ denotes $j$-th block
of the input maps with total $C$ channels,
$\textbf{K}_{i}\in\mathbb{R}^{k\times k\times C}$ is the $i$-th convolution
kernel with $i=1,~{}\dots,~{}C$, then $\textbf{X}_{i,j}\in\mathbb{R}$ is the
$j$-th element in $i$-th output feature map. It can be seen that all the
outputs in each channel share a same convolution kernel. However, when
implementing a convolution-based SNN on above mentioned neuromorphic
hardwares, all the kernels will be mapped as synapses no matter they are
shared or not. Hence, keeping shared convolution kernel cannot show the same
advantages for SNNs as DNNs. We argue that learning different convolution
kernels for each output map may improve the performence of the SNN further. In
this case, a convolution layer for SNNs can be written as

$\textbf{X}_{i,j}=f(\textbf{I}_{j}*\textbf{K}_{i,j}),$ (2)

where $\textbf{K}_{i,j}\in\mathbb{R}^{k\times k\times C}$ is the $j$-th
convolution kernel for $i$-th output map. Unfortunately, it is not easy to
directly implement the learning of unshared convolution kernel for SNNs in the
DNN-oriented frameworks. And dealing with this issue is one of the important
works in this paper.

##  3 Materials and Methodology

Aiming at training more accurate SNNs, we adopt the explicitly iterative leaky
Integrate-and-Fire (LIF) model, which can be implemented on mature DNN-
oriented frameworks easily to simulate the fundamental computing unit of SNNs.
Considering that it is not easy to realize unshared convolution kernels for
each output feature map with existing DNN-oriented frameworks, a modified LIF
model that can emit the real-valued spike is proposed first. By using this
modified LIF model, our SNNs will learn real spikes along with the shared
convolution kernel for every output channel as DNNs during training. While for
the inference phase, real spikes will be transformed into binary spikes and
each convolution kernel will be re-parameterized as multiple different
convolution kernels, so the advantages of SNNs can be recovered and unshared
convolution kernel for each output map can be obtained. Then, to make the
proposed design more general, we also propose layer-wise and channel-wise Real
Spike, which can keep shared convolution kernels in both training phase and
inference phase and will introduce no more parameters than its vanilla
counterpart. In this section, we will introduce explicitly iterative LIF
model, Real Spike, re-parameterization, and extensions of Real Spike
successively.

###  3.1 Explicitly Iterative Leaky Integrate-and-Fire Model

The spiking neuron is the fundamental computing unit of SNNs. The LIF neuron
is most commonly used in supervised learning-based methods. Mathematically, a
LIF neuron can be described as

$\tau\frac{\partial u}{\partial t}=-u+I,\quad u<V_{th}$ (3)
$u=u_{rest}\quad\&\quad fire\;a\;spike,\quad u\geq V_{th}$ (4)

where $u$, $u_{rest}$, $\tau$, $I$, and $V_{th}$ represent the membrane
potential, membrane resting potential, membrane time constant that controls
the decaying rate of $u$, pre-synaptic input, and the given firing threshold,
respectively. When $u$ is below $V_{th}$, it acts as a leaky integrator of
$I$. On the contrary, when $u$ exceeds $V_{th}$, it will fire a spike and
propagate the spike to the next layer, then it will be reset to the resting
potential, $u_{rest}$, which is usually set as $0$.

The LIF model has a complex dynamics structure that is incompatible with
nowadays DNN-oriented framework. By discretizing and transforming the LIF
model to an explicitly iterative LIF model, SNNs can be implemented in these
mature frameworks. The hardware friendly iterative LIF model can be described
as

$u(t)=\tau u(t-1)+I(t),\quad u(t)<V_{th}$ (5)
$o(t)=\left\\{\begin{array}[]{lll}1,\ \ if\;u(t)\geq V_{th},\\\ 0,\ \
otherwise.\end{array}\right.$ (6)

where $V_{th}$ is set to 0.5 in this work. Up to now, there is still an
obstacle for training SNNs in a direct way, i.e., Eq. (6) is non-
differentiable. As in other work [33, 40, 4], we appoint a rectangular
function as the particular pseudo derivative of spike firing as follows,

$\frac{do}{du}=\left\\{\begin{array}[]{lll}1,\ \ if\;0\leq u\leq 1,\\\ 0,\ \
otherwise.\end{array}\right.$ (7)

With all these settings, now we can train an SNN on DNN-oriented frameworks.

###  3.2 Real Spike

As aforementioned, driven by the suppressed advantages of the shared
convolution kernel and the expectation of enhancing the capacity of
information representation for SNNs, we turn the problem of learning unshared
convolution kernels to learning real spikes. To be more specific, the output
of our modified LIF model in Eq. (6) is further rewritten as

$\tilde{o}(t)=a\cdotp o(t)$ (8)

where $a$ is a learnable coefficient. With this modification, our LIF model
can emit a _real-valued spike_ , dubbed Real Spike. Then we train SNNs with
this modified LIF model along with the shared convolution kernel for each
output map, which can be easily implemented in DNN-oriented frameworks.
Obviously, unlike the binary spike, the real-valued spike will lose the
advantage of computation efficiency of SNNs, since the corresponding
multiplication cannot be substituted to addition. And another problem is that
the learned convolution kernels for each output map are still shared at this
time. Therefore, to jointly deal with these problems, we propose a training-
inference decoupled framework, which can transform real spikes into binary
spikes and convert the shared convolution kernel as different kernels by using
re-parameterization, which will be introduced in the next subsection.

Figure 3: The diagram of re-parameterization by a simple example.

###  3.3 Re-parameterization

Consider a convolution layer, which takes
$\textbf{F}\in\mathbb{R}^{D_{F}\times D_{F}\times M}$ as input feature maps,
and generates the output feature maps, $\textbf{G}\in\mathbb{R}^{D_{G}\times
D_{G}\times N}$, where $D_{F}$ and $M$ denote the size of the square feature
maps and the number of channels (depths) for the input, respectively; $D_{G}$
and $N$ denote the size of the square feature maps and the number of channels
(depths) for the output, respectively. The convolution layer is actually
parameterized by a group of convolution kernels, which can be denoted as a
tensor, $\textbf{K}\in\mathbb{R}^{D_{K}\times D_{K}\times M\times N}$ with a
spatial dimension of $D_{K}$. Then, each element in G is computed as

$\textbf{G}_{k,l,n}=\sum_{i,j,m}\textbf{K}_{i,j,m,n}\cdotp\textbf{F}_{k+i-1,l+j-1,m}$
(9)

For standard SNNs, the elements of input maps are binary spikes, while in this
work, the SNN is trained with real-valued spikes for the purpose of enhancing
the network representation capacity. In this case, we can further denote the
input feature map, F, according to Eq. (8) as follows

$\textbf{F}=\textbf{a}\odot\textbf{B}$ (10)

where B and a denote a binary tensor and a learnable coefficient tensor,
respectively. With this element-wise multiplication in Eq. (10), we can
extract a part of the value from each element in F, and fold it into the
shared convolution kernel one-by-one according to the corresponding position
during inference. Then the single shared convolution kernel can be turned into
multiples without changing the values of the output maps. Through this
decoupling process, a new SNN that can emit binary spikes and enjoy different
convolution kernels will be obtained. This process can be illustrated from Eq.
(11) to Eq. (13) as follows:

$\textbf{G}_{k,l,n}=\sum_{i,j,m}\textbf{K}_{i,j,m,n}\cdotp(\textbf{a}_{k+i-1,l+j-1,m}\cdotp\textbf{B}_{k+i-1,l+j-1,m})$
(11)
$\textbf{G}_{k,l,n}=\sum_{i,j,m}(\textbf{a}_{k+i-1,l+j-1,m}\cdotp\textbf{K}_{i,j,m,n})\cdotp\textbf{B}_{k+i-1,l+j-1,m}$
(12)
$\textbf{G}_{k,l,n}=\sum_{i,j,m}\tilde{\textbf{K}}_{k,l,i,j,m,n}\cdotp\textbf{B}_{k+i-1,l+j-1,m}$
(13)

where $\tilde{\textbf{K}}$ is the unshared convolution kernel tensor.

The whole process described above is called re-parameterization, which allows
us to convert a real-valued-spike-based SNN into an output-invariant binary-
spike-based SNN with unshared convolution kernels for each output map. That
is, re-parameterization provides a solution to obtain an SNN with unshared
convolution kernels under DNN-oriented frameworks by decoupling the training-
time SNN and inference-time SNN. Figure 3 illustrates the details of re-
parameterization by a simple example.

###  3.4 Analysis and Discussions

In this work, we assume that firing real-valued spikes during training can
help increase the representation capacity of the SNNs. To verify our
assumption, a series of analyses and discussions are conducted by using the
information entropy concept. Given a scalar, vector, matrix, or tensor, X, its
representation capability is denoted as $\mathcal{R}(\textbf{X})$, which can
be measured by the information entropy of X, as follows

$\mathcal{R}(\textbf{X})=\max\mathcal{H}(\textbf{X})=\max(-\sum_{x\in\textbf{X}}p_{\textbf{X}}(x)logp_{\textbf{X}}(x))$
(14)

where $p_{\textbf{X}}(x)$ is the probability of a sample from X. When
$p_{\textbf{X}}(x_{1})=\cdots=p_{\textbf{X}}(x_{n})$,
$\mathcal{H}(\textbf{X})$ reaches its maximum (see Appendix A.1 for detailed
proofs). For a binary spike $o$, it can be expressed with 1 bit, and the
number of samples from $o$ is $2$. While the real-valued spike $\tilde{o}$
needs 32 bits, which consists of $2^{32}$ samples. Hence, $\mathcal{R}(o)=1$
and $\mathcal{R}$ $(\tilde{o})=32$ according to Eq. (14). Obviously, the
representation capability of real spikes far exceeds that of binary spikes.
This indicates that real spikes will enhance the information expressiveness of
SNNs, which accordingly benefit the performance improvement. To further show
the difference between real spikes and binary spikes intuitively, the
visualizations of some channels expressed by real spikes and binary spikes are
given respectively in the appendix.

Another intuitive conjecture to explain why the SNN with real-valued spikes
performs better than its counterpart with binary spikes is that, for the
former one, the information loss can be restrained to some extent by changing
the fixed spike value to an appropriate value with a scalable coefficient,
$a$; while for the later one, the firing function would inevitably induce the
quantization error.

Figure 4: The difference of adjacent layers in SNNs implemented on
neuromorphic hardwares before re-parameterization and after re-
parameterization. The convolution kernel re-parameterization will only change
the values of the connections between neurons without introducing any
additional computation and storage resource since the entire architecture
topology of an SNN must be completely mapped into the hardwares.

It can be concluded from the above analysis that, learning real-valued spikes
instead of the binary spikes in the training phase, enables us to train a more
accurate SNN. And by performing re-parameterization in the inference phase,
real spikes can be converted to binary spikes and unshared convolution kernels
can be obtained. In another word, learning Real Spike is actually used to
generate a better information encoder, while the re-parameterization will
transfer the rich information encoding capacity from Real Spike into
information decoding. Moreover, the transformed model after re-
parameterization can also retain the advantage coming from the binary spike
information processing paradigm in standard SNNs. As shown in Fig. 4, when
deploying an SNN into some neuromorphic hardwares, all the units and their
connections of the network will be mapped as neurons and synaptic connections
one by one. Hence, the SNN with multiple different convolution kernels will
not introduce any computation and storage resource.

###  3.5 Extensions of Real Spike

The key observation made in the Real Spike is that re-scaling the binary spike
of the LIF neuron with a real-valued coefficient, $a$, can increase the
representation capability and accurancy of SNNs. As shown in Eq. (10), the
default Real Spike is performed in the element-wise way. In a similar fashion,
we argue that introducing scaling coefficent by layer-wise or channel-wise
manners will also retain the benefit to some extent. Then, we further propose
to re-formulate Eq. (8) for one layer as follows

$\tilde{\textbf{o}}(t)=\textbf{a}\cdotp\textbf{o}(t)$ (15)

With this new formulation we can explore various ways of introducing a during
training. Specifically, we propose to introduce a for each layer in the
following 3 ways:

Layer-wise:

$\textbf{a}\in\mathbb{R}^{1\times 1\times 1}$ (16)

Channel-wise:

$\textbf{a}\in\mathbb{R}^{C\times 1\times 1}$ (17)

Element-wise:

$\textbf{a}\in\mathbb{R}^{C\times H\times H}$ (18)

where $C$ is the number of channels and $H$ is the spatial dimension of a
square feature map. Obviously, for layer-wise and channel-wise Real Spike, re-
parameterization will only re-scale the shared convolution kernels without
transferring them to different ones. In these two forms, our Real Spike act in
the same way as conventional SNNs on any hardwares. That is to say, starting
from the motivation of how to take full advantage of the integration of memory
and computation, we propose element-wise Real Spike. Then we extract the
essential idea of training-inference-decoupled and extend Real Spike to a more
generalized form, which is friendly to both neuromorphic and non-neuromorphic
hardware platforms.

##  4 Experiment

The performance of the Real Spike-based SNNs were evaluated on several
traditional static datasets (CIFAR-10 [19], CIFAR-100 [19], ImageNet [20]) and
one neuromorphic dataset (CIFAR10-DVS [24]). And multiple widely-used spiking
archetectures including ResNet20 [33, 36], VGG16 [33], ResNet18 [8], and
ResNet34 [8] were used to verify the effectiveness of our Real Spike. Detailed
introduction of the datasets and experimental settings are provided in the
appendix. Extensive ablation studies were conducted first to compare the SNNs
with real-valued spikes and their binary spike counterpart. Then, we
comprehensively compared our SNNs with the existing state-of-the-art SNN
methods.

###  4.1 Ablation Study for Real Spike

Table 1: Ablation study for Real Spike.

Dataset |  Architecture |  Timestep |  Accuracy
---|---|---|---
CIFAR10 |  ResNet20 w/ BS |  2 |  88.91%
4 |  91.73%
6 |  92.98%
ResNet20 w/ RS |  2 |  90.47%
4 |  92.53%
6 |  93.44%
CIFAR100 |  ResNet20 w/ BS |  2 |  62.59%
4 |  63.27%
6 |  67.12%
ResNet20 w/ RS |  2 |  63.40%
4 |  64.87%
6 |  68.60%

RS represents real spikes and BS represents binary spikes.

The ablation study of Real Spike was conducted on CIFAR-10/100 datasets based
on ResNet20. The SNNs with real-valued spikes and binary spikes were trained
with the same configuration, with timestep varying from 2 to 6. Results in
Tab. 1 show that the test accuracy of the SNNs with real-valued spikes is
always higher than that with binary spikes. This is due to the richer
representation capacity from Real Spike, which benefits the performance
improvement of SNNs. Figure 5 illustrates the test accuracy curves of ResNet20
with real-valued spikes and its counterpart with binary spikes on CIFAR-10/100
during training. It can be seen that the SNNs with real-valued spikes
obviously perform better on the accuracy and convergence speed.

Figure 5:  The accuracy curves of ResNet20 with real-valued spikes and binary
spikes with a timestep = 4 on CIFAR-10(left) and CIFAR-100(right). The real-
valued spikes-based SNNs obviously enjoy higher accuracy and easier
convergence.

###  4.2 Ablation Study for Extensions of Real Spike

The performance of SNNs with different extensions of Real Spike was evaluated
on CIFAR-10(100). Results in Tab. 2 show that element-wise Real Spike always
outperforms the other versions. But all the Real Spike-based SNNs conformably
outperform the vanilla (standard) SNN. Another observation is that the
accuracy difference between layer-wise Real Spike and the binary spike is
greater than that between layer-wise Real Spike and element-wise Real Spike.
This shows that our method touches on the essence of improving the SNN
accuracy and is very effective.

Table 2: Performance comparison for different Real Spike versions. Dataset |  Architecture |  version |  Accuracy
---|---|---|---
CIFAR10 |  ResNet20 |  Vanilla |  91.73%
Layer-wise |  92.12%
Channel-wise |  92.25%
Element-wise |  92.53%
CIFAR100 |  ResNet20 |  Vanilla |  63.27%
Layer-wise |  64.28%
Channel-wise |  64.71%
Element-wise |  64.87%

###  4.3 Comparison with the State-of-the-art

Table 3: Performance comparison with SOTA methods. Dataset | Method | Type | Architecture | Timestep | Accuracy
---|---|---|---|---|---
CIFAR-10 |  SpikeNorm [36] | ANN2SNN | VGG16 | 2500 | 91.55%
Hybrid-Train [34] | Hybrid | VGG16 | 200 | 92.02%
SBBP [23] | SNN training | ResNet11 | 100 | 90.95%
STBP [40] | SNN training | CIFARNet | 12 | 90.53%
TSSL-BP [42] | SNN training | CIFARNet | 5 | 91.41%
PLIF [9] | SNN training | PLIFNet | 8 | 93.50%
Diet-SNN [33] | SNN training | VGG16 | 5 | 92.70%
10 | 93.44%
ResNet20 | 5 | 91.78%
10 | 92.54%
STBP-tdBN [43] | SNN training | ResNet19 | 2 | 92.34%
4 | 92.92%
6 | 93.16%
Real Spike | SNN training | ResNet19 | 2 | 94.01%$\pm 0.10$
4 | 95.60%$\pm 0.08$
6 | 95.71%$\pm 0.07$
ResNet20 | 5 | 93.01%$\pm 0.07$
10 | 93.65%$\pm 0.05$
VGG16 | 5 | 92.90%$\pm 0.09$
10 | 93.58%$\pm 0.06$
CIFAR-100 |  BinarySNN [27] | ANN2SNN | VGG15 | 62 | 63.20%
Hybrid-Train [34] | Hybrid | VGG11 | 125 | 67.90%
T2FSNN [31] | ANN2SNN | VGG16 | 680 | 68.80%
Burst-coding [30] | ANN2SNN | VGG16 | 3100 | 68.77%
Phase-coding [18] | ANN2SNN | VGG16 | 8950 | 68.60%
Diet-SNN [33] | SNN training | ResNet20 | 5 | 64.07%
VGG16 | 5 | 69.67%
Real Spike | SNN training | ResNet20 | 5 | 66.60%$\pm 0.11$
VGG16 | 5 | 70.62%$\pm 0.08$
VGG16 | 10 | 71.17%$\pm 0.07$
ImageNet |  Hybrid-Train [34] | Hybrid | ResNet34 | 250 | 61.48%
SpikeNorm [36] | ANN2SNN | ResNet34 | 2500 | 69.96%
STBP-tdBN [43] | SNN training | ResNet34 | 6 | 63.72%
SEW ResNet [8] | SNN training | ResNet18 | 4 | 63.18%
ResNet34 | 4 | 67.04%
Real Spike | SNN training | ResNet18 | 4 | 63.68%$\pm 0.08$
ResNet34 | 4 | 67.69%$\pm 0.07$
CIFAR10-DVS |  Rollout [21] | Streaming | DenseNet | 10 | 66.80%
STBP-tdBN [43] | SNN training | ResNet19 | 10 | 67.80%
Real Spike | SNN training | ResNet19 | 10 | 72.85%$\pm 0.12$
ResNet20 | 10 | 78.00%$\pm 0.10$

In this section, we compared the Real Spike-based SNNs with other state-of-
the-art SNNs on several static datasets and a neuromorphic dataset. For each
run, we report the mean accuracy as well as the standard deviation with 3
trials. Experimental results are shown in Tab. 3.

CIFAR-10(100). On CIFAR-10, our SNNs achieve higher accuracy than the other
state-of-the-art methods, and the best result of 95.71% top-1 accuracy is
achieved by ResNet19 with 6 timesteps. And even trained with much fewer
timesteps, i.e., 2, our ResNet19 still outperforms the STBP-tdBN under 6
timesteps with 0.85% higher accuracy. This comparison shows that our method
also enjoys the advantage of latency reduction for SNNs. On CIFAR-100, Real
Spike also performs better and achieves a 2.53% increment on ResNet20 and a
0.95% increment on VGG16.

ImageNet. The ResNet18 and ResNet34 were selected as the backbones.
Considering that the image size of the samples is much larger, the channel-
wise Real Spike was used. For a fair comparison, we made our architectures
consistent with SEW-ResNets, which are not typical SNNs, where the IF model
and modified residual structure are adopted. Results show that, even with 120
fewer epochs of training (200 for ours, 320 for SEW-ResNets), the channel-wise
Real Spike-based SNNs can still outperform SEW-ResNets. In particular, our
method achieves a 0.5% increment on ResNet18 and a 0.65% increment on
ResNet34. Moreover, Real Spike-based ResNet34 with 4 timesteps still
outperforms STBP-tdBN-based RersNet34 with 6 timesteps by 3.97% higher
accuracy.

CIFAR10-DVS. To further verify the generalization of the Real Spike, we also
conducted experiments on the neuromorphic dataset CIFAR10-DVS. Using ResNet20
as the backbone, Real Spike achieves the best performance with 78.00% accuracy
in 10 timesteps. For ResNet19, Real Spike obtains 5.05% improvement compared
with STBP-tdBN. It’s worth noting that, as a more complex model, ResNet19
performs worse than ResNet20. This is because that neuromorphic datasets
usually suffer much more noise than static ones, thus more complex models are
easier to overfit on these noisier datasets.

##  5 Conclusions

In this work, we focused on the difference between SNNs and DNNs and
speculated that the unshared convolution kernel-based SNNs would enjoy more
advantages than those with shared convolution kernels. Motivated by this idea,
we proposed Real Spike, which aims at enhancing the representation capacity
for an SNN by learning real-valued spikes during training and transferring the
rich representation capacity into inference-time SNN by re-parameterizing the
shared convolution kernel to different ones. Furthermore, a series of Real
Spikes in different granularities were explored, which are enjoy shared
convolution kernels in both training and inference phases and friendly to both
neuromorphic and non-neuromorphic hardware platforms. Proof of why Real Spike
has a better performance than vanilla SNNs was provided. Extensive experiments
verified that our proposed method consistently achieves better performance
than the existed state-of-the-art SNNs.

## References

  * [1] Akopyan, F., Sawada, J., Cassidy, A., Alvarez-Icaza, R., Arthur, J., Merolla, P., Imam, N., Nakamura, Y., Datta, P., Nam, G.J., et al.: Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip. IEEE transactions on computer-aided design of integrated circuits and systems 34(10), 1537–1557 (2015)
  * [2] Cao, Y., Chen, Y., Khosla, D.: Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision 113(1), 54–66 (2015)
  * [3] Carnevale, N.T., Hines, M.L.: The NEURON book. Cambridge University Press (2006)
  * [4] Cheng, X., Hao, Y., Xu, J., Xu, B.: Lisnn: Improving spiking neural networks with lateral interactions for robust object recognition. In: IJCAI. pp. 1519–1525 (2020)
  * [5] Davies, M., Srinivasa, N., Lin, T.H., Chinya, G., Cao, Y., Choday, S.H., Dimou, G., Joshi, P., Imam, N., Jain, S., et al.: Loihi: A neuromorphic manycore processor with on-chip learning. Ieee Micro 38(1), 82–99 (2018)
  * [6] Diehl, P.U., Cook, M.: Unsupervised learning of digit recognition using spike-timing-dependent plasticity. Frontiers in computational neuroscience 9, 99 (2015)
  * [7] Fang, W., Chen, Y., Ding, J., Chen, D., Yu, Z., Zhou, H., Tian, Y., other contributors: Spikingjelly. https://github.com/fangwei123456/spikingjelly (2020)
  * [8] Fang, W., Yu, Z., Chen, Y., Huang, T., Masquelier, T., Tian, Y.: Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems 34, 21056–21069 (2021)
  * [9] Fang, W., Yu, Z., Chen, Y., Masquelier, T., Huang, T., Tian, Y.: Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 2661–2671 (2021)
  * [10] Gewaltig, M.O., Diesmann, M.: Nest (neural simulation tool). Scholarpedia 2(4), 1430 (2007)
  * [11] Goodman, D.F., Brette, R.: The brian simulator. Frontiers in neuroscience p. 26 (2009)
  * [12] Guo, Y., Tong, X., Chen, Y., Zhang, L., Liu, X., Ma, Z., Huang, X.: Recdis-snn: Rectifying membrane potential distribution for directly training spiking neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 326–335 (June 2022)
  * [13] Han, B., Roy, K.: Deep spiking neural network: Energy efficiency through time based coding. In: European Conference on Computer Vision. pp. 388–404. Springer (2020)
  * [14] Hao, Y., Huang, X., Dong, M., Xu, B.: A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule. Neural Networks 121, 387–395 (2020)
  * [15] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [16] Huh, D., Sejnowski, T.J.: Gradient descent for spiking neural networks. Advances in neural information processing systems 31 (2018)
  * [17] Khan, M.M., Lester, D.R., Plana, L.A., Rast, A., Jin, X., Painkras, E., Furber, S.B.: Spinnaker: mapping neural networks onto a massively-parallel chip multiprocessor. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence). pp. 2849–2856. Ieee (2008)
  * [18] Kim, J., Kim, H., Huh, S., Lee, J., Choi, K.: Deep neural networks with weighted spikes. Neurocomputing 311, 373–386 (2018)
  * [19] Krizhevsky, A., Nair, V., Hinton, G.: Cifar-10 (canadian institute for advanced research). URL http://www. cs. toronto. edu/kriz/cifar. html 5(4), 1 (2010)
  * [20] Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012)
  * [21] Kugele, A., Pfeil, T., Pfeiffer, M., Chicca, E.: Efficient processing of spatio-temporal data streams with spiking neural networks. Frontiers in Neuroscience 14, 439 (2020)
  * [22] LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)
  * [23] Lee, C., Sarwar, S.S., Panda, P., Srinivasan, G., Roy, K.: Enabling spike-based backpropagation for training deep neural network architectures. Frontiers in neuroscience p. 119 (2020)
  * [24] Li, H., Liu, H., Ji, X., Li, G., Shi, L.: Cifar10-dvs: an event-stream dataset for object classification. Frontiers in neuroscience 11, 309 (2017)
  * [25] Li, Y., Deng, S., Dong, X., Gong, R., Gu, S.: A free lunch from ann: Towards efficient, accurate spiking neural networks calibration. In: International Conference on Machine Learning. pp. 6316–6325. PMLR (2021)
  * [26] Li, Y., Guo, Y., Zhang, S., Deng, S., Hai, Y., Gu, S.: Differentiable spike: Rethinking gradient-descent for training spiking neural networks. Advances in Neural Information Processing Systems 34, 23426–23439 (2021)
  * [27] Lu, S., Sengupta, A.: Exploring the connection between binary and spiking neural networks. Frontiers in Neuroscience 14, 535 (2020)
  * [28] Ma, D., Shen, J., Gu, Z., Zhang, M., Zhu, X., Xu, X., Xu, Q., Shen, Y., Pan, G.: Darwin: A neuromorphic hardware co-processor based on spiking neural networks. Journal of Systems Architecture 77, 43–51 (2017)
  * [29] Neftci, E.O., Mostafa, H., Zenke, F.: Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks. IEEE Signal Processing Magazine 36(6), 51–63 (2019)
  * [30] Park, S., Kim, S., Choe, H., Yoon, S.: Fast and efficient information transmission with burst spikes in deep spiking neural networks. In: 2019 56th ACM/IEEE Design Automation Conference (DAC). pp. 1–6. IEEE (2019)
  * [31] Park, S., Kim, S., Na, B., Yoon, S.: T2fsnn: Deep spiking neural networks with time-to-first-spike coding. In: 2020 57th ACM/IEEE Design Automation Conference (DAC). pp. 1–6. IEEE (2020)
  * [32] Pei, J., Deng, L., Song, S., Zhao, M., Zhang, Y., Wu, S., Wang, G., Zou, Z., Wu, Z., He, W., et al.: Towards artificial general intelligence with hybrid tianjic chip architecture. Nature 572(7767), 106–111 (2019)
  * [33] Rathi, N., Roy, K.: Diet-snn: Direct input encoding with leakage and threshold optimization in deep spiking neural networks. arXiv preprint arXiv:2008.03658 (2020)
  * [34] Rathi, N., Srinivasan, G., Panda, P., Roy, K.: Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation. arXiv preprint arXiv:2005.01807 (2020)
  * [35] Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems 28 (2015)
  * [36] Sengupta, A., Ye, Y., Wang, R., Liu, C., Roy, K.: Going deeper in spiking neural networks: Vgg and residual architectures. Frontiers in neuroscience 13, 95 (2019)
  * [37] Shrestha, S.B., Orchard, G.: SLAYER: Spike layer error reassignment in time. In: Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., Garnett, R. (eds.) Advances in Neural Information Processing Systems 31. pp. 1419–1428. Curran Associates, Inc. (2018), http://papers.nips.cc/paper/7415-slayer-spike-layer-error-reassignment-in-time.pdf
  * [38] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  * [39] Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1–9 (2015)
  * [40] Wu, Y., Deng, L., Li, G., Zhu, J., Xie, Y., Shi, L.: Direct training for spiking neural networks: Faster, larger, better. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp. 1311–1318 (2019)
  * [41] Yao, P., Wu, H., Gao, B., Tang, J., Zhang, Q., Zhang, W., Yang, J.J., Qian, H.: Fully hardware-implemented memristor convolutional neural network. Nature 577(7792), 641–646 (2020)
  * [42] Zhang, W., Li, P.: Temporal spike sequence learning via backpropagation for deep spiking neural networks. Advances in Neural Information Processing Systems 33, 12022–12033 (2020)
  * [43] Zheng, H., Wu, Y., Deng, L., Hu, Y., Li, G.: Going deeper with directly-trained larger spiking neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 11062–11070 (2021)


††E-mail<EMAIL_ADDRESS>
# FRBSTATS: A web-based platform for visualization of
fast radio burst properties

Apostolos Spanakis-Misirlis1,2
1 Department of Informatics, University of Piraeus, Greece
2 Institute of Astrophysics, FORTH, Dept. of Physics, University of Crete,
Voutes, University Campus, GR-71003, Heraklion, Greece

###### Abstract

The study of fast radio bursts (FRBs) is of great importance, and is a topic
that has been extensively researched, particularly in recent years. While the
extreme nature of FRBs can serve as a tool for researchers to probe the
intergalactic medium and study exotic aspects of the Universe, the rapid
discovery of FRBs has recently deemed the track of new events challenging.
FRBSTATS provides a user-friendly web interface to an open-access catalogue of
FRBs published up to date, along with a statistical overview of the observed
events. The platform supports the retrieval of fundamental FRB data either
directly through the FRBSTATS API, or in the form of a CSV/JSON-parsed
database, while enabling the plotting of parameters and their distributions,
for a variety of visualizations. These features allow researchers to conduct
population studies and comparisons with astrophysical models, describing the
origin and emission mechanism behind these sources. So far, the redshift
estimates of 806 bursts have been computed and derived, providing the first
public database that includes redshift entries for nearly all observed FRBs.
Lastly, the platform provides a visualization tool that illustrates
associations between primary bursts and repeaters, complementing basic
repeater information provided by the Transient Name Server. In this work, we
present the structure of the platform, the established version control system,
as well as the strategy for maintaining such an open database up to date.
Additionally, we introduce a novel, computationally-efficient, clustering-
based approach that enables unsupervised classification of hundreds of bursts
into repeaters and non-repeaters, resulting in the discovery of three new FRB
repeaters.

##  1 Introduction

Fast radio bursts (FRBs) are bright, short-duration pulses of unknown origin,
observed at radio wavelengths. Since the first detection by Lorimer et al.
(2007), hundreds of such events have been observed, with varying emission
characteristics (see Fig. 1). Despite the 15-year period of FRB observations
however, the emission mechanism behind these sources remains unknown (Petroff
et al., 2022). Although the exact origin of FRBs is highly debated, nearly all
FRBs detected so far are extragalactic, with varying spectral, polarization,
and time-domain emission properties. With the exception of the Galactic
magnetar SGR 1935+2154 (CHIME/FRB Collaboration et al., 2020), the dispersion
measure of FRBs hint that their origin is extragalactic. It is worth noting,
that some FRBs exhibit a repeating behaviour (Spitler et al., 2016), which has
ruled out, altered, and even introduced a variety of new theoretical models
describing the radiation process (Platts et al., 2019). An unusual subclass of
repeating FRBs appears to show a periodic activity (Chime/Frb Collaboration et
al., 2020; Rajwade et al., 2020), which may further narrow down the list of
candidate models.

Because of their enigmatic nature and the rapid rate of discoveries, gaining a
better and more complete understanding of these astrophysical objects is an
essential prerequisite to uncovering a significant piece of the puzzle behind
the mysterious presence of this class of sources. Due to the unpredictability
of FRBs however, the consistent monitoring of individual sources is a highly
time-consuming and challenging task. While each observation comes with its own
derived properties, carrying out literature reviews for each individual burst
becomes unpractical, especially as the number of detected bursts increases.
Without the most significant properties of each detected burst indexed in an
accessible database, population studies and extensive analysis of FRBs (taking
every observed event into consideration to build statistical models) become
laborious.

Figure 1: Total number of fast radio bursts observed as a function of their
detection time. The notable increase of observed bursts in July 2019 came with
the commissioning of the Canadian Hydrogen Intensity Mapping Experiment Fast
Radio Burst (CHIME/FRB) Project. FRB 20180725A marks the very first detection
by CHIME, representing the first ever FRB observed below 700 MHz.

In this work, we present a platform that hosts an accurate and complete
catalogue of FRBs, with the primary goal of enabling researchers to easily
derive fundamental properties of bursts, with minimal effort. In Sect. 2, we
begin by describing the high-level architecture of the system, as well as the
strategy for maintaining and keeping the catalogue up to date, as autonomously
as possible. Accessible via a user-friendly web interface, FRBSTATS includes
an open-access catalogue of all FRBs published to date (Sect. 3), along with a
statistical overview of their properties. Furthermore, the platform hosts a
page that illustrates the association between primary bursts and repeaters
(Subsect. 3.1), complementing basic repeater information provided by the
Transient Name Server111https://www.wis-tns.org (Yaron et al., 2020). In
addition to the ability of plotting of parameter distributions for a variety
of visualizations (Subsect. 3.2), the platform supports the retrieval of FRB
data, directly through the FRBSTATS API (Subsect. 3.3). We finally conclude by
summarizing our work and discussing future enhancements for the platform
(Sect. 4).

##  2 Platform architecture

In order to ensure all data and code supporting the platform is findable,
accessible, interoperable, and reusable, the source of the
website222https://www.herta-experiment.org/frbstats is hosted on a Git
repository.333https://github.com/HeRTA/FRBSTATS This enables and encourages
researchers to openly contribute with code, data, and ideas for improvement.
Additionally, the project utilizes continuous integration and continuous
deployment (CI/CD) pipelines, which address the problem of delayed development
and updates to the platform. Fig. 2 describes the high-level architecture of
the FRBSTATS platform. Upon the submission of modifications to catalogue.csv,
the database is automatically updated, and the properties of each burst are
distributed to the corresponding subprocesses for further management.

Based on a non-relational document model,
MongoDB444https://github.com/mongodb/mongo offers a suitable NoSQL database,
which serves as the catalogue infrastructure for the platform. By
automatically parsing the CSV catalogue as a JSON-formatted file, the
catalogue page permits quick search queries, based on either the designation
of the FRB of interest, or its parameters.

Figure 2: System architecture of the FRBSTATS platform, indicating the flow of
data from one subsystem to another. The parser.py process refers to a Python
script that is part of the CI/CD pipeline. Its role is to convert the input
CSV catalogue to a JSON-formatted file for improved accessibility by the
MongoDB database infrastructure.

##  3 Catalogue

The catalogue consists of the observed properties for each FRB: namely, the
topocentric UTC and Modified Julian Date (MJD) of the observed event, the
corresponding telescope that detected the burst, and its sky location in
equatorial (right ascension $\alpha$ and declination $\delta$), as well as
Galactic coordinates (longitude $l$ and latitude $b$). Additionally, the
database includes the frequency of each burst, its dispersion measure,
redshift (computed with the help of the fruitbat555The package takes the
coordinates and dispersion measure of the FRB source, and based on the YMW16
free-electron distribution model (Yao et al., 2017), the tool then computes an
estimate for the redshift based on the derived excess dispersion measure.
package; Batten, 2019), peak flux density, burst width and fluence, as well as
the signal-to-noise ratio of the detection. Provided with a reference for each
set of parameters, building statistical models and population studies becomes
straightforward.

Event data are obtained from the Transient Name Server (TNS), FRBCAT (Petroff
et al., 2016), the First CHIME/FRB Fast Radio Burst
Catalog,666https://www.chime-frb.ca/catalog The Astronomer’s
Telegram777https://www.astronomerstelegram.org (Rutledge, 1998), and the
corresponding publications cited on the FRBSTATS catalogue. For some bursts,
parameters obtained and merged from other catalogues have been reviewed and
corrected based on the publications associated with each burst.

###  3.1 Repeaters

As mentioned in Sect. 1, a fraction of FRB sources appear to exhibit a
repeating activity, in other words, a single FRB source is capable of
producing more than one bursts, potentially through a similar emission
mechanism. This clue becomes especially significant if the observed properties
appear identical to the previously-emitted bursts.

The discovery of the first repeater (FRB 20121102) by Spitler et al. (2016)
demonstrated that such a class of FRBs could not originate from cataclysmic
events involving the collapse of supramassive neutron stars (Falcke &
Rezzolla, 2014) or neutron star mergers (Hansen & Lyutikov, 2001). Although it
is not yet clear whether a single emission mechanism is responsible for all
FRBs (repeaters and non-repeaters), the repository of FRB
theories888https://frbtheorycat.org (Platts et al., 2019) consists of studies
that rule out even more theories for the progenitor of FRB repeaters, such as
pulsar-black hole mergers (Bhattacharyya, 2017), binary white dwarf mergers
(Kashiyama et al., 2013), the collapse of strange-star crusts (Zhang et al.,
2018), neutron star-white dwarf collisions (Liu, 2018), among other theories.

It thus becomes apparent that the study of FRB repeaters is a valuable method
for investigating the potential emission mechanisms responsible for each
burst. However, the manual classification of FRBs into repeaters and non-
repeaters (one-offs) can be a tedious task. FRBSTATS makes this process
easier, by automatically identifying repeating bursts and distinguishing them
from one-off events. This enables researchers to easily associate the observed
properties of each burst, without having to investigate the parameters of
hundreds of FRBs.

####  3.1.1 Repeater classification using DBSCAN

Considering the vast number of FRBs detected to date, in order to characterise
bursts into repeaters and one-offs, an automated classification method is
required. We describe a novel method for automatically classifying repeaters
from non-repeaters using density-based spatial clustering of applications with
noise (DBSCAN). Originally proposed by Ester et al. (1996), this data
clustering algorithm is ideal for certain applications that require the
grouping of a set of neighbouring points.

With the objective of rapidly classifying nearly a thousand bursts into
repeaters and non-repeaters, we find that this approach can be applied to FRBs
using merely three parameters: the candidate’s Galactic coordinates ($l$,
$b$), as well as its dispersion measure (DM).

The DM describes the integrated free-electron column density between the
observer and the FRB of interest,
and based on the model of Macquart et al. (2020), the observed DM can be split
into four contributing factors,

$\mathrm{DM_{\mathrm{FRB}}}=\mathrm{DM}_{\mathrm{ISM}}+\mathrm{DM}_{\mathrm{halo}}+\mathrm{DM}_{\mathrm{cosmic}}+\mathrm{DM}_{\mathrm{host}},$
(1)

where the contribution of the Galactic interstellar medium (ISM) and halo, the
extragalactic distribution of ionized gas and the host galaxy of the FRB
source are all taken into account.

Despite the uncertainty of $\mathrm{DM_{\mathrm{host}}}$ described by James et
al. (2022), Yang & Zhang (2017) show that variations in the dispersion measure
of repeating FRBs are negligible. The stability of this parameter makes the DM
a great alternative to using source distance as a clustering parameter, which,
due to the limitation of free-electron distribution models being confined to
Galactic sources (Cordes, 2004; Schnitzeler, 2012), this property remains
unknown.

Assuming the three parameters ($l,b,\mathrm{DM}$) of repeaters used for
clustering possess negligible variance over time/remain constant, we can
employ a custom metric to reliably evaluate the distance between two given
points (FRBs), using the three-dimensional Euclidean distance that separates
them.

Let $x_{1},y_{1},z_{1}$ and $x_{2},y_{2},z_{2}$ be the Cartesian coordinates
of $\mathrm{FRB}_{1}$ and $\mathrm{FRB}_{2}$, respectively. The position of
each burst expressed in terms of $l,b,$ and $\mathrm{DM}$ is then given by

$\displaystyle x_{i}$ $\displaystyle=\mathrm{DM}_{i}\cos{b_{i}}\sin{l_{i}}$
(2) $\displaystyle y_{i}$
$\displaystyle=\mathrm{DM}_{i}\cos{b_{i}}\cos{l_{i}}$ (3) $\displaystyle
z_{i}$ $\displaystyle=\mathrm{DM}_{i}\cos{b_{i}},$ (4)

where $i\in\\{1,2\\}$ refers to the FRB of interest. We can then compute the
Euclidean distance $d$ between $\mathrm{FRB}_{1}$ and $\mathrm{FRB}_{2}$ using

$\displaystyle
d\big{(}\mathrm{FRB}_{1},\mathrm{FRB}_{2}\big{)}=\sqrt{\sum_{j=1}^{n}{\big{(}\mathrm{FRB}_{1}-\mathrm{FRB}_{2}\big{)}}^{2}}\
$ (5) $\displaystyle\stackrel{{\scriptstyle
n=3}}{{=\joinrel=\joinrel=}}\sqrt{\big{(}x_{1}-x_{2}\big{)}^{2}+\big{(}y_{1}-y_{2}\big{)}^{2}+\big{(}z_{1}-z_{2}\big{)}^{2}}.$
(6)

Assuming the threshold for the number of neighbours, minPts, is equal to $2$
(definition of FRB repeater; i.e., present at least 1 detection following the
primary burst), we find that, by using Equation (5) as a metric for the
distance function, accurate results are obtained when the radius (arbitrary
distance measure; Schubert et al., 2017) is $\epsilon\approx 15$. Given
$\epsilon$ is the most important parameter for evaluating the association
between two bursts using the DBSCAN algorithm, the radius value was optimized
(fine-tuned) by manually inspecting the reasonableness of the output repeater
clusters.

Figure 3: Output scatter plots resulting from the DBSCAN algorithm for
$\epsilon=15$. The one-off events are counted as noisy samples/stray data
points (left), while the detected 25 clusters are grouped and represented with
a corresponding colour associated with an individual repeater (right). While
the distance metric for estimating whether a burst is part of a cluster is
given by Equation (5), the Galactic coordinates and the dispersion measure of
the bursts are instead displayed for clarity.

Expressed in terms of the miss rate or false negative rate (FNR), $\epsilon$
values significantly less than $15$ provide too little clustering sensitivity,
yielding numerous false negatives:

$\mathrm{FNR}={\frac{\mathrm{FN}}{\mathrm{FN}+\mathrm{TP}}}=1-\mathrm{TPR},$
(7)

where $\mathrm{FN}$ and $\mathrm{TP}$ are the false negatives and true
positives, respectively, and $\mathrm{TPR}$ is the true positive rate. To the
contrary, $\epsilon$ values far greater than $15$ provide excessive
contamination to the resulting clusters, yielding a high false positive rate
(FPR):

$\mathrm{FPR}={\frac{\mathrm{FP}}{\mathrm{FP}+\mathrm{TN}}}=1-\mathrm{TNR},$
(8)

where $\mathrm{FP}$ and $\mathrm{TN}$ are the false positives and true
negatives, respectively, and $\mathrm{TNR}$ is the true negative rate.

By using the cluster.DBSCAN class999https://scikit-
learn.org/stable/modules/generated/sklearn.cluster.DBSCAN provided by the
scikit-learn library (Pedregosa et al., 2011), we identify 25 repeating FRB
sources. Only 1 repeater listed on the CHIME/FRB repeaters
page101010https://www.chime-frb.ca/repeaters (Amiri et al., 2021) was not
identified by the clustering algorithm (namely, FRB 20171019A; Kumar et al.,
2019), but that was due to inadequate reporting (i.e., no sub-burst data has
been published to an online catalogue). An additional repeater (FRB
190907.J08+46; Fonseca et al., 2020) was detected under a different
designation (FRB 20190907A), with 3 identified sub-bursts (FRB 20190915A, FRB
20190925A and FRB 20200307A).

Table 1: Fundamental FRB properties available for direct plotting through the
platform, along with a brief description for each parameter.

Parameter | Description
---|---
Source Position
RA | Right ascension ($\alpha$)
Dec. | Declination ($\delta$)
Gal. Long. | Galactic longitude ($l$)
Gal. Lat. | Galactic latitude ($b$)
Burst Properties
Frequency | Reported burst frequency (MHz)
DM | Reported dispersion measure $\big{(}\mathrm{pc\ cm}^{-3}\big{)}$
Flux Density | Peak flux density (Jy)
Burst Width | Derived burst width (ms)a
Fluence | Measured burst fluence (Jy ms)

$\ \ {}^{a}$ Commonly reported as the fitted full width at half maximum (FWHM)
of the burst.

Figure 4: Flux–width distribution of all reported FRBs, along with their
dispersion measure. Bursts whose flux or width have not been measured and
published have been excluded from the chart. The burst with the greatest flux
density corresponds to FRB 20210415A, yielding a burst fluence of
${334\mathrm{\ Jy\ ms}\pm 10}$ per cent. This particular burst was observed
with the Stockert 25-m Radio Telescope in L band (Herrmann, 2021).

The clustering algorithm identified three additional candidate repeater
sources: FRB 20190107B with one additional sub-burst association (FRB
20190308B), FRB 20181225B (linked to FRBs 20190320E and 20190609B), and FRB
20181119C (linked to FRB 20190131E). Upon visual inspection, we find that
these candidate parent/daughter associations have similar DMs, with variations
($\sim$8, 2 and 1 per cent, respectively) consistent with what has previously
been observed for FRB 121102 (Spitler et al., 2016; Law et al., 2017). Two of
the parent/daughter events also appear to have consistent coordinates,
considering the CHIME beam size (CHIME/FRB Collaboration et al., 2018). In
regard to FRB 20181225B, the maximum angular separation between two bursts is
3.5∘, which could be the result of a

sidelobe detection bearing CHIME’s beam pattern (Newburgh et al., 2014; Berger
et al., 2016) in mind. We conclude that based on the available information,
these sources (highlighted in red on the FRBSTATS platform) could be true
repeaters that require further investigation.

Given the efficiency of the algorithm, repeater clustering of 811 bursts is
carried out in less than 20 seconds by the FRBSTATS server (1 Intel vCPU; 1 GB
of memory). This includes the total data read time of the catalogue, the
clustering itself, as well as the plotting of the results. The output of the
clustering algorithm for 798 events is provided in Fig. 3.

####  3.1.2 Repeater tree visualization

Once repeater classification is complete, the data is stored as a JSON file,
which is finally picked up by the frontend and displayed to the user in the
form of a repeater tree for convenient visualization. Upon clicking on an FRB
repeater source, its time series plot is displayed, along with the
corresponding telescope and signal-to-noise ratio of each detected sub-burst.

###  3.2 Plots

Since the analysis of burst properties is a crucial step required to derive
parameter distributions of FRBs and conduct population studies, the platform
also offers an easy-to-use plotting tool for quick analysis. Similar to the
Australia Telescope National Facility (ATNF) Pulsar Catalogue (Manchester et
al., 2005), FRBSTATS enables direct parameter visualizations, without the need
to download the catalogue and manually run scripts locally, in an offline
manner. The FRB parameters available for plotting are listed in Table 1.

An example plot obtained using FRBSTATS is shown in Fig. 4, highlighting the
distribution of the flux, width, and dispersion measure of FRBs. These plots
are maintained up to date automatically, and are updated as soon as a new
database entry/modification is submitted to the system. This ensures that
newly detected events are displayed, and simple comparisons with the remaining
FRB population can be made fairly easily.

###  3.3 API

An application programming interface (API) enables a convenient integration
layer between a server and its clients. In the case of this particular
platform, the FRBSTATS API provides researchers and developers access to FRB
parameters in a lightweight, rapid, and easy manner, supporting numerous
automation applications. All entries present in the database can be obtained
with the help of the API, whose functionality is documented on the web
platform, along with some examples. The client may submit a GET request to the
provided endpoint URL, and query an FRB of interest. If the FRB is found in
the database, the JSON-encoded API response will consist of the parameters of
the queried FRB, as well as the reference linked to the corresponding
catalogue/publication. Thus, each parameter value can be reliably traced back
to its original publication source for further verification. In the case of
queries that are not associated with any burst, the API returns an empty array
(i.e., []).

It is worth noting that the PHP-based utility automatically neglects the ‘FRB’
prefix, underscores and spaces, in order to improve the parsing of the query
and handle different designations (corresponding to the same event) in a
compatible fashion. Finally, unlike the catalogue-querying system in place,
the server ensures that the search queries submitted to the API are directed
to solely match the name of the FRB of interest, and does not return
irrelevant bursts that could happen to match the values of certain parameters.

##  4 Conclusions

We have built and presented a versatile web-platform consisting of an accurate
open-access catalogue of fast radio bursts, a parameter visualization utility,
as well as an application programming interface available for querying burst
properties. Furthermore, we expanded on the method of automatically
classifying FRBs into repeaters and non-repeaters using the DBSCAN algorithm
with a custom distance metric. We hope that the platform provides researchers
with access to an up-to-date database of FRBs to work with and conduct better
population studies, with as little maintenance work as possible.

In future work, we plan on further expanding on the catalogue by providing
additional information regarding each burst, such as the Galactic,
intergalactic, and local contributions to the DM, as well as polarization
properties. Additionally, we hope to make dynamic spectra of each individual
burst available and accessible, directly through the online catalogue.

## Acknowledgements

I wish to acknowledge the help provided by Dr. John Antoniadis for the
valuable feedback collected and provided to improve the utility of the
platform.

## References

  * Amiri et al. (2021) Amiri M., et al., 2021, ApJS, 257, 59
  * Batten (2019) Batten A., 2019, The Journal of Open Source Software, 4, 1399
  * Berger et al. (2016) Berger P., et al., 2016, in Hall H. J., Gilmozzi R., Marshall H. K., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 9906, Ground-based and Airborne Telescopes VI. p. 99060D (arXiv:1607.01473), doi:10.1117/12.2233782
  * Bhattacharyya (2017) Bhattacharyya S., 2017, arXiv e-prints, p. arXiv:1711.09083
  * CHIME/FRB Collaboration et al. (2018) CHIME/FRB Collaboration et al., 2018, ApJ, 863, 48
  * CHIME/FRB Collaboration et al. (2020) CHIME/FRB Collaboration et al., 2020, Nature, 587, 54
  * Chime/Frb Collaboration et al. (2020) Chime/Frb Collaboration et al., 2020, Nature, 582, 351
  * Cordes (2004) Cordes J. M., 2004, in Clemens D., Shah R., Brainerd T., eds, Astronomical Society of the Pacific Conference Series Vol. 317, Milky Way Surveys: The Structure and Evolution of our Galaxy. p. 211
  * Ester et al. (1996) Ester M., Kriegel H.-P., Sander J., Xu X., 1996, in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining. KDD’96. AAAI Press, p. 226–231
  * Falcke & Rezzolla (2014) Falcke H., Rezzolla L., 2014, A&A, 562, A137
  * Fonseca et al. (2020) Fonseca E., et al., 2020, ApJ, 891, L6
  * Hansen & Lyutikov (2001) Hansen B. M. S., Lyutikov M., 2001, MNRAS, 322, 695
  * Herrmann (2021) Herrmann W., 2021, The Astronomer’s Telegram, 14556, 1
  * James et al. (2022) James C. W., Prochaska J. X., Macquart J. P., North-Hickey F. O., Bannister K. W., Dunning A., 2022, MNRAS, 509, 4775
  * Kashiyama et al. (2013) Kashiyama K., Ioka K., Mészáros P., 2013, ApJ, 776, L39
  * Kumar et al. (2019) Kumar P., et al., 2019, ApJ, 887, L30
  * Law et al. (2017) Law C. J., et al., 2017, ApJ, 850, 76
  * Liu (2018) Liu X., 2018, Ap&SS, 363, 242
  * Lorimer et al. (2007) Lorimer D. R., Bailes M., McLaughlin M. A., Narkevic D. J., Crawford F., 2007, Science, 318, 777
  * Macquart et al. (2020) Macquart J. P., et al., 2020, Nature, 581, 391
  * Manchester et al. (2005) Manchester R. N., Hobbs G. B., Teoh A., Hobbs M., 2005, AJ, 129, 1993
  * Newburgh et al. (2014) Newburgh L. B., et al., 2014, in Stepp L. M., Gilmozzi R., Hall H. J., eds, Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series Vol. 9145, Ground-based and Airborne Telescopes V. p. 91454V (arXiv:1406.2267), doi:10.1117/12.2056962
  * Pedregosa et al. (2011) Pedregosa F., et al., 2011, Journal of Machine Learning Research, 12, 2825
  * Petroff et al. (2016) Petroff E., et al., 2016, PASA, 33, e045
  * Petroff et al. (2022) Petroff E., Hessels J. W. T., Lorimer D. R., 2022, A&A Rev., 30, 2
  * Platts et al. (2019) Platts E., Weltman A., Walters A., Tendulkar S. P., Gordin J. E. B., Kandhai S., 2019, Phys. Rep., 821, 1
  * Rajwade et al. (2020) Rajwade K. M., et al., 2020, MNRAS, 495, 3551
  * Rutledge (1998) Rutledge R. E., 1998, PASP, 110, 754
  * Schnitzeler (2012) Schnitzeler D. H. F. M., 2012, MNRAS, 427, 664
  * Schubert et al. (2017) Schubert E., Sander J., Ester M., Kriegel H. P., Xu X., 2017, ACM Trans. Database Syst., 42
  * Spitler et al. (2016) Spitler L. G., et al., 2016, Nature, 531, 202
  * Yang & Zhang (2017) Yang Y.-P., Zhang B., 2017, ApJ, 847, 22
  * Yao et al. (2017) Yao J. M., Manchester R. N., Wang N., 2017, ApJ, 835, 29
  * Yaron et al. (2020) Yaron O., Ofek E., Gal-Yam A., Sass A., 2020, Transient Name Server AstroNote, 70, 1
  * Zhang et al. (2018) Zhang Y., Geng J.-J., Huang Y.-F., 2018, ApJ, 858, 88