# Microcanonical Truncations of Observables in Quantum Chaotic Systems

Fernando Iniguez<EMAIL_ADDRESS>Mark Srednicki<EMAIL_ADDRESS>Department of Physics, University of California, Santa Barbara, CA 93106

###### Abstract

We consider the properties of an observable (such as a single spin component
that squares to the identity) when expressed as a matrix in the basis of
energy eigenstates, and then truncated to a microcanonical slice of energies
of varying width. For a quantum chaotic system, we model the unitary or
orthogonal matrix that relates the spin basis to the energy basis as a random
matrix selected from the appropriate Haar measure. We find that the spectrum
of eigenvalues is given by a centered Jacobi distribution that approaches the
Wigner semicircle of a random hermitian matrix for small slices. For slices
that contain more than half the states, there is a set of eigenvalues of
exactly $\pm 1$. The transition to this qualitatively different behavior at
half size is similar to that seen in other quantities such as entanglement
entropy. Our results serve as a benchmark model for numerical calculations in
realistic physical systems.

##  I Introduction

The eigenstate thermalization hypothesis (ETH) [1, 2, 3, 4, 5, 6, 7] is now
widely accepted as a microscopic mechanism that is able to explain how an
isolated quantum many-body system can come to thermal equilibrium when
starting from an initial pure state that appears to be far from equilibrium.
ETH is expected to hold for a “chaotic” quantum system that is sufficiently
far (in a parameter space of possible hamiltonians) from any point of
integrability and which also does not exhibit many-body localization due to
strong disorder. ETH then takes the form of an ansatz for the matrix elements
(in the energy-eigenstate basis) of each observable $A$ that would be measured
in order to determine whether or not the system is in thermal equilibrium.
This ansatz is

$A_{ij}=\mathcal{A}(E)\delta_{ij}+e^{-S(E)/2}f(E,\omega)R_{ij},$ (1)

where $E=(E_{i}+E_{j})/2$ is the average energy of the two eigenstates,
$\omega=E_{i}-E_{j}$ is their energy difference, $S(E)$ is the thermodynamic
entropy (logarithm of the density of states) at energy $E$, $\mathcal{A}(E)$
and $f(E,\omega)$ are smooth, real functions of their arguments, with
$f(E,\omega)=f(E,-\omega)$, and $R_{ij}$ varies erratically, with overall zero
mean and unit variance in local ranges of $E$ and $\omega$.

A question of interest is whether more can be said about the statistical
properties of the $R_{ij}$’s. An argument based on the central limit theorem
would indicate that they can be treated as independent gaussian random
variables, and numerical investigations in specific systems have generally
been consistent with this. However, as has been pointed out before [8, 9],
this gaussianity cannot be an exact property, as it would yield various
unphysical predictions, including an expression of any $n$-point time
correlation function of $A$ in terms of the 2-point function. Furthermore, the
operator $A$ has a spectrum of eigenvalues, and this spectrum must somehow be
encoded in the energy-basis matrix elements $A_{ij}$. Because of this, as
noted in [8], it is more useful to think of the unitary matrix $U_{ai}$ that
transforms basis states in which $A$ is diagonal to the energy-eigenstate
basis as a statistically random matrix.

In [9] (see also [10]), the observable $A$ was taken to be a component of a
single spin in a lattice spin system, and the eigenvalues of $A_{ij}$ computed
when $i$ and $j$ were restricted to particular ranges of energies. If this
submatrix had the statistical properties of a gaussian random matrix, then a
Wigner semi-circular distribution of eigenvalues would be expected. This was
found for small energy ranges, but significant deviations appeared at larger
ranges.

Our goal here is to provide a theoretical benchmark for these calculations,
computing the expected eigenvalue spectrum for a single-spin-component
operator $A$ (which obeys $A^{2}=I$) when its matrix $A_{ij}$ in the energy-
eigenstate basis is truncated, with the energies $E_{i}$ and $E_{j}$ each in
the same finite range. We refer to this as a microcanonical slicing. We
specialize to the case where the ${\cal A}(E)$ function in Eq. (1) is zero;
this is equivalent to

$\operatorname{Tr}e^{-\beta H}\\!A=0$ (2)

for all inverse temperatures $\beta$. This corresponds to a system in which
the hamiltonian $H$ is invariant under $A\to-A$. We then treat the
diagonalizing matrix $U$ as either a unitary or orthogonal Haar-random matrix.
(The result is the same in both cases.) This is the strongest possible
assumption of random-matrix behavior; for an actual physical system, we expect
correlations that result in $U$ having an approximately banded structure. We
hope that our benchmark results can be used in the future to help elucidate
this structure in different physical systems of interest.

##  II Setup

We consider an operator $A$ that obeys $A^{2}=I$ and $\operatorname{Tr}A=0$.
We take the dimension of the full Hilbert space to be $2D$; for a system of
$N$ two-component spins, we would have $2D=2^{N}$. The eigenvalues of $A$ are
hence $\pm 1$, with $D$ eigenvalues of each sign.

We then write

$A=U^{\dagger}\tilde{A}U,$ (3)

where $\tilde{A}$ is a $2D\times 2D$ diagonal matrix,

$\tilde{A}=\begin{pmatrix}+I&0\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr
0&-I\end{pmatrix},$ (4)

and $U$ is a unitary (or orthogonal) matrix that transforms from the
computational basis (in which a component of each spin is diagonal) to the
energy basis (in which the hamiltonian is diagonal).

We are now interested in a microcanonical slicing of $A$, defined as

$A_{K}=\mathit{\Pi}_{K}A\mathit{\Pi}_{K}$ (5)

where $\mathit{\Pi}_{K}$ is a projection operator onto an energy window that
spans $K$ energy eigenstates.

We first specialize to the case $K\leq D$. We treat $U$ as either a unitary or
orthogonal matrix that is selected at random from the corresponding Haar
measure, and consider the expected distribution $\rho(\lambda)$ of the
eigenvalues $\lambda$ of $A_{K}$ in the limit of large $D$.

This problem has been solved in a different context [11], and the result (for
either unitary or orthogonal $U$) is a special case of the centered Jacobi
distribution,

$\rho(\lambda)=\frac{\sqrt{4\alpha(1-\alpha)-\lambda^{2}}}{2\pi\alpha(1-\lambda^{2})},$
(6)

where we have defined

$\alpha=\frac{K}{2D},$ (7)

and where $\rho(\lambda)$ vanishes for values of $\lambda$ for which the
argument of the square-root is negative. We have normalized $\rho(\lambda)$ to
integrate to one.

For a thin microcanonical slice, $K\ll D$ and hence $\alpha\ll 1$, the maximum
value of $\lambda^{2}$ is $4\alpha\ll 1$, and then we can replace
$1-\lambda^{2}$ in the denominator of Eq. (6) with $1$. We then have

$\rho_{\alpha\ll
1}(\lambda)\simeq\frac{1}{2\pi\alpha}\sqrt{4\alpha-\lambda^{2}}.$ (8)

We can compare this result with the Wigner semicircle for an $K\times K$
hermitian random matrix $H$ with matrix elements $H_{ij}$ drawn from a
gaussian distribution with the expected value of $|H_{ij}|^{2}$, $i\neq j$,
given by $v^{2}$; this is [12]

$\rho_{\mathrm{W}}(\lambda)=\frac{1}{2\pi Kv^{2}}\sqrt{4Kv^{2}-\lambda^{2}}.$
(9)

Our $2D\times 2D$ matrix $A$ obeys $A^{2}=I$, and hence

$\sum_{j=1}^{2D}|A_{ij}|^{2}=1.$ (10)

This implies that the expected value of each $|A_{ij}|^{2}$ is $1/2D$. If we
set $v^{2}=1/2D$ in Eq. (9), we find that Eq. (8) matches it. This result
agrees with the expectation from ETH that, for small energy differences, the
energy-basis matrix elements of a local observable should have the statistics
of independent gaussian random variables.

For larger values of $\alpha$, Eq. (6) begins to differ from the Wigner
semicircle. The curvature at the origin $\rho^{\prime\prime}(0)$ is one
diagnostic; it is negative for
$\alpha<\alpha_{\mathrm{c}}=(2-\sqrt{2})/4=0.146$ but turns positive for
$\alpha>\alpha_{\mathrm{c}}$. At $\alpha=1/2$, $K=D$, we find an arcsine
distribution with integrable singularities at $\lambda=\pm 1$,

$\rho_{\alpha=1/2}(\lambda)=\frac{1}{\pi\sqrt{1-\lambda^{2}}}.$ (11)

In Fig. 1(a,b,c), we show the eigenvalue distribution for a matrix with
$2D=10{,}000$ and with $U$ a particular orthogonal matrix selected at random
from the Haar measure, and with $K/2D=\alpha=1/8,1/4,1/2$, along with the
predicted distribution of Eq. (6). We find very good agreement.

Next we consider the case $K\geq D$. This can be related to the case $K\leq D$
by considering the complementary microcanonical projection operator,

$\mathit{\Pi}_{2D-K}=I-\mathit{\Pi}_{K}.$ (12)

We show in the Appendix that for $K>D$ the eigenvalues of $A_{K}$ are minus
those of $A_{2D-K}$, plus $K-D$ extra pairs of eigenvalues of exactly $\pm 1$.
This is true for any specific individual $U$. Hence, after averaging $U$ over
a Haar measure, the result will be a continuous spectrum given by Eq. (6)
(though with the normalizing factor of $\alpha$ in the denominator replaced by
$1-\alpha$), plus a discrete spectrum of $K-D$ eigenvalues $+1$ and $K-D$
eigenvalues $-1$.

In Fig. 1(d,e), we show the eigenvalue distribution for $K/2D=\alpha=3/4,7/8$.
We find, as predicted, a continuous distribution that matches that of the
matrix with $K/2D=1-\alpha$, with all remaining eigenvalues exactly equal to
$\pm 1$.

These results for $K>D$ are contrary to our initial expectations. We expected
to find the Wigner semicircle for $K\ll D$, and for this to gradually morph
into a set of only $\pm 1$ at $K=2D$. Our expectations are met by the results
for $K\leq D$, but the sudden appearance of some exact $\pm 1$ eigenvalues for
every $K>D$, along with an additional continuous distribution that mirrors the
distribution for $K<D$ and eventually becomes a Wigner semicircle for small
$2D-K$, came as a surprise to us. We will discuss this further in our
conclusions.

##  III Conclusions

Motivated by the investigations of [9], we have considered the properties of a
single spin-component operator (with eigenvalues that are an equal number of
plus ones and minus ones) in a many-body quantum-chaotic system. We are
interested in the statistical properties of the matrix elements of such an
operator in the energy-eigenstate basis. We model this by treating the unitary
(or orthogonal) matrix $U$ that relates the spin-eigenstate basis to the
energy-eigenstate basis as a random matrix selected from the Haar measure. We
then consider microcanonical truncations of this matrix in the energy basis,
and study their eigenvalues. For truncations to a much smaller matrix, we find
the distribution agrees with the Wigner semicircle expected for a hermitian
random matrix. For larger truncations, we find that the truncated matrix
begins to “remember” that the eigenvalues of the full matrix are $\pm 1$.

Once the truncation is to a matrix larger than half the size of the original,
we find that there are now a set of eigenvalues of exactly $\pm 1$, along with
a continuous distribution that matches that of the complementary truncation.
This result was counter to what we initially expected, and shows a kind of
phase transition at half system size. This is reminiscent of a similar
transition in the behavior of entanglement and Rényi entropies [13], which
also exhibit sudden changes of behavior at half system size. Similar
transitions have also been discussed in [14] for operators that include more
than half the degrees of freedom of the system.

Our results are based on the most chaotic possible behavior of a physical
system, in which the energy eigenstates are completely random superpositions
of basis states, without any additional structure. Though this is unlikely to
be true for any realistic physical system, our results serve as a useful
benchmark of comparison for numerical calculations in these systems.

###### Acknowledgements.

The work of F.I. was supported by an NSF Graduate Research Fellowship under
Grant No. 2139319 and funds from the University of California. M.S. thanks
Lauri Foini, Jorge Kurchan, and Silvia Pappalardi for helpful discussions.

|
---|---
(a) | (b)
|
(c) | (d)
(e)
Figure 1: Histograms of eigenvalues for different microcanonical truncation
sizes of an originally $10000\times 10000$ matrix ($2D=10000$). The blue curve
represents the analytical prediction given by the Jacobi distribution of Eq.
(6). Figs. (a)–(e) show results for $K/2D=0.125$, 0.25, 0.5, 0.75, 0.875.
Figs. (d) and (e) are truncated in height and do not show the full count of
$\pm 1$ eigenvalues.

##  Appendix A Truncations Greater than Half

We write $A=U^{\dagger}\tilde{A}U$ in block-diagonal form,

$A=\begin{pmatrix}A_{K}&B\\\ \vskip 3.0pt plus 1.0pt minus 1.0pt\cr
B^{\dagger}&A_{2D-K}\end{pmatrix}.$ (13)

We take $K\leq 2D$. Since the eigenvalues of $A$ are $\pm 1$, we have

$A^{2}=I.$ (14)

This yields the three equations

$\displaystyle A_{K}^{2}+BB^{\dagger}$ $\displaystyle=I,$ (15) $\displaystyle
A_{K}B+BA_{2D-K}$ $\displaystyle=0,$ (16) $\displaystyle
B^{\dagger}\\!B+A_{2D-K}^{2}$ $\displaystyle=I.$ (17)

We can make independent unitary transformations on the upper $K\times K$ block
and on the lower $(2D{-}K)\times(2D{-}K)$ block that render $A_{K}$ and
$A_{2D-K}$ diagonal. We then write

$(A_{K})_{ij}=\lambda_{i}\delta_{ij},\quad(A_{2D-K})_{i^{\prime}j^{\prime}}=\kappa_{i^{\prime}}\delta_{i^{\prime}j^{\prime}},$
(18)

where $i,j=1,\ldots,K$ and $i^{\prime},j^{\prime}=K{+}1,\ldots,2D{-}K$. Taking
the $ij^{\prime}$ matrix element of Eq. (16), we get

$(\lambda_{i}+\kappa_{j^{\prime}})B_{ij^{\prime}}=0.$ (19)

This shows that a nonvanishing matrix element of $B$ is possible if and only
if there is an eigenvalue $\kappa_{j^{\prime}}$ of $A_{2D-K}$ that is the
negative of an eigenvalue $\lambda_{i}$ of $A_{K}$. For $K<D$, there are more
eigenvalues of $A_{2D-K}$ than there are of $A_{K}$, and Eq. (17) then implies
that these extra eigenvalues must be $\pm 1$. Since $\operatorname{Tr}A=0$,
these extra eigenvalues must come in $\pm 1$ pairs.

We conclude that, for $K\leq D$, the $2D{-}K$ eigenvalues of $A_{2D-K}$
consist of $K$ eigenvalues that are equal in magnitude and opposite in sign to
the $K$ eigenvalues of $A_{K}$, $D{-}K$ eigenvalues $+1$, and $D{-}K$
eigenvalues $-1$.

## References

  * [1] J. M. Deutsch.  Quantum statistical mechanics in a closed system.  Phys. Rev. A, 43:2046–2049, Feb 1991.
  * [2] Mark Srednicki.  Chaos and quantum thermalization.  Phys. Rev. E, 50:888–901, Aug 1994.
  * [3] Mark Srednicki.  Thermal fluctuations in quantized chaotic systems.  J. Phys. A: Math. Gen., 29(4):L75–L79, feb 1996.
  * [4] Mark Srednicki.  The approach to thermal equilibrium in quantized chaotic systems.  J. Phys. A: Math. Gen., 32(7):1163, 1999.
  * [5] Marcos Rigol, Vanja Dunjko, and Maxim Olshanii.  Thermalization and its mechanism for generic isolated quantum systems.  Nature, 452:854, Apr 2008.
  * [6] Luca D’Alessio, Yariv Kafri, Anatoli Polkovnikov, and Marcos Rigol.  From quantum chaos and eigenstate thermalization to statistical mechanics and thermodynamics.  Adv. Phys., 65(3):239–362, 2016.
  * [7] Joshua M Deutsch.  Eigenstate thermalization hypothesis.  Rep. Prog. Phys., 81(8):082001, jul 2018.
  * [8] Laura Foini and Jorge Kurchan.  Eigenstate thermalization hypothesis and out of time order correlators.  Phys. Rev. E, 99(4):042139, 2019.
  * [9] Jonas Richter, Anatoly Dymarsky, Robin Steinigeweg, and Jochen Gemmer.  Eigenstate thermalization hypothesis beyond standard indicators: Emergence of random-matrix behavior at small frequencies.  Phys. Rev. E, 102:042127, Oct 2020.
  * [10] Silvia Pappalardi, Laura Foini, and Jorge Kurchan.  Microcanonical windows on quantum operators, 2023.
  * [11] Benoît Collins.  Product of random projections, jacobi ensembles and universality problems arising from free probability.  Probability Theory and Related Fields, 133(3):315–344, 2005.
  * [12] Madan Lal Mehta.  Random Matrices.  Elsevier, Amsterdam, 3rd edition, 2004.
  * [13] Tsung Cheng Lu and Tarun Grover.  Renyi entropy of chaotic eigenstates.  Physical Review E, 99(3):032111, 2019.
  * [14] James R. Garrison and Tarun Grover.  Does a single eigenstate encode the full hamiltonian?  Phys. Rev. X, 8:021026, Apr 2018.


# Evaluation of Semantic Search and its Role in Retrieved-Augmented-Generation
(RAG) for Arabic Language

Ali Mahboub, Muhy Eddin Za’ter, Bashar Alfrou, Yazan Estaitia, Adnan Jaljuli,
Asma Hakouz
Maqsam
Amman, Jordan
{ali.mahbob, muhyeddin, bashar.alrfooh, yazan.estaitia, adnan.jaljuli,
<EMAIL_ADDRESS>

###### Abstract

The latest advancements in machine learning and deep learning have brought
forth the concept of semantic similarity, which has proven immensely
beneficial in multiple applications and has largely replaced keyword search.
However, evaluating semantic similarity and conducting searches for a specific
query across various documents continue to be a complicated task. This
complexity is due to the multifaceted nature of the task, the lack of standard
benchmarks, whereas these challenges are further amplified for Arabic
language. This paper endeavors to establish a straightforward yet potent
benchmark for semantic search in Arabic. Moreover, to precisely evaluate the
effectiveness of these metrics and the dataset, we conduct our assessment of
semantic search within the framework of retrieval augmented generation (RAG).

_K_ eywords Semantic Search $\cdot$ Retrieved Augmented Generation (RAG)
$\cdot$ Arabic Natural Language Processing

##  1 Introduction

The abundance of information has driven the development of semantic search
technologies that surpass traditional keyword-based search engines by
understanding the context and intent of user queries through natural language
processing (NLP) and machine learning [1]. Unlike conventional search methods
that focus on matching keywords, semantic search interprets the meaning and
relationships between words, aiming to mimic human understanding. This
advancement enhances user experience across various applications, including
web search engines, knowledge discovery, and personalized content
recommendation systems, and most recently Retriever-Augmented Generation (RAG)
[2].

RAG represents an innovative approach at the crossroads of information
retrieval and natural language generation, leveraging the strengths of both
fields to refine Artificial Intelligence (AI) based systems ability to
comprehend and generate human-like text [3, 4]. By combining a sophisticated
retrieval mechanism with a powerful generation model, RAG systems can produce
detailed, contextually relevant responses that significantly improve
standalone language models limitations in terms of precision and human-like
generation. The integration of semantic search into RAG systems is crucial,
particularly for processing complex queries or those requiring deep contextual
understanding, making it a cornerstone for enhancing retrieval accuracy and
the quality of generated content.

Similar to the majority of research endeavors and NLP tasks, the Arabic
language semantic search and RAG lags behind other languages due to the
challenges posed by the Arabic language, including its complex morphology the
diversity of its dialects and the shortage of datasets [5, 6]. The above-
mentioned challenges underscore the need for NLP techniques and studies
tailored to Arabic in the context of semantic search and RAG systems. This
paper aims to evaluate semantic searchś effectiveness in processing Arabic
alongside its impact on the performance of RAG systems specifically designed
for Arabic Question Answering use-case. By evaluating the effect of different
text encoders on the performance of RAG systems, the study seeks to provide
insights into optimizing NLP applications for Arabic-speaking users and
advancing the development of linguistically inclusive AI systems.

The rest of the paper is presented as follows; section 2 presents a simple
overview of the previous literature, followed by section 3 which describes the
methodology and experiment designed to evaluate the semantic search in context
of RAG, while section 4 presents the results and the discussion finally
followed by the conclusion.

##  2 Literature review

The evolution of semantic search can be traced back to before the widespread
adoption of machine learning, initially relying on keyword-based methods and
statistical techniques like latent semantic indexing (LSI) in the late 1990s
[7]. These early methods aimed to understand document similarity beyond exact
keyword matches, setting the stage for more sophisticated approaches.

In the 2000s, machine learning transformed semantic search with algorithms
like support vector machines (SVMs)[8], moving beyond keyword matching to a
deeper understanding of queries through the use of more advanced and
sophisticated textual features. However, these techniques had their
limitations regarding the contextual meaning it is capable of grasping,
therefore more advanced technique that are capable of leveraging the
increasing influx of data were required.

Therefore, deep learning was introduced to semantic search which marked a
significant milestone, with techniques like Word2Vec [9] and GloVe [10]
enhancing the understanding of word relationships through unsupervised
learning of huge corpus. This era also saw the development of advanced neural
networks based architecture, such as attention mechanism [11] and transformers
(e.g., BERT, GPT)[12], revolutionizing the field by enabling a deeper grasp of
query intent and contextual relevance, which played a pivotal role the
advancements of deep learning.

Parallel to the advancements of text encoders using deep learning, the use of
approximate nearest neighbors (ANN) techniques became crucial for semantic
search [13]. ANN algorithms, emerging in the early 2010s, facilitated
efficient similarity search in high-dimensional spaces, essential for managing
the vast data processed by deep learning models, which allowed users to
replace slow unscalable techniques such as cosine similarity. The integration
of ANN with state-of-the-art language models has continuously improved of
semantic search, enhancing its scalability and efficiency [14].

Significant progress has been made in Arabic semantic search thanks to deep
learning technologies, especially with the creation of Arabic-focused encoders
employing various frameworks like Word2Vec and Transformer-based models [15,
16, 17]. Additionally, incorporating Arabic into universal encoders has also
proved beneficial, utilizing patterns from other languages to improve
understanding. This study highlights the effectiveness of using advanced deep
learning methods to better grasp the semantic nuances of Arabic inquiries,
leading to high precision in recognizing comparable questions in a customer
support setting.

Conversely, RAG—Retrieved Augmented Generation—remains an emerging field
within the burgeoning domain of generative artificial intelligence. The
particular challenges associated with Arabic, such as its complex morphology
and the relative paucity of resources, have hindered the attention and
research it has received. Arabic RAG has not yet emerged as a focal point of
scholarly inquiry to the degree that perhaps it warrants [18, 19].

This study endeavors to assess the current landscape of semantic search
capabilities within the Arabic language, addressing the notable lack of
benchmarks and baseline data. Additionally, we seek to explore the impact of
semantic search on the efficacy of retrieved augmented generation, positing
that enhanced search mechanisms could significantly bolster the generation
process.

##  3 Evaluation Methodology

In this section, we present the procedures employed for the evaluation of
various semantic search modules. Our methodological framework is structured
around three core components: dataset generation, evaluation metrics, and the
configuration of semantic search modules.

###  3.1 Dataset Generation

The structure of the dataset required for the effective evaluation of semantic
search ranking is pivotal. The dataset needs to encompass:

  * •

A collection of documents, which, in our study, consist of Arabic summaries of
customer support calls for real-world companies.

  * •

A set of user search queries, where each query is associated with all or a
subset of the documents.

  * •

A relevance score or label for each (query, document) pair, indicating the
documentś relevance to the query. This can be a binary label
(relevant/irrelevant) or a score value with higher scores indicating higher
relevance.

To circumvent the resource-intensive process of data collection and labeling,
we leverage the capabilities of Large Language Models (LLMs), notably GPT-4,
to generate search queries that:

  * •

Mimic realistic searches as might be conducted by customer support agents.

  * •

Assign each query to a set of five summaries.

  * •

Assign a relevance label/score to each (query, document) pair, with the
scoring system of (irrelevant, somewhat relevant, very relevant) designated as
(0, 1, 2) respectively.

A prompt was devised to meet these requirements and to generate an Arabic
search query for every set of five summaries, ensuring that at least one
summary is highly relevant.

The evaluation dataset comprises 2030 customer support call summaries and 406
search queries. A manual inspection of 10% of the dataset (205 summaries with
41 queries) revealed only 2 misclassifications between very relevant and
irrelevant summaries, suggesting the dataset is in a highly accurate state.
Further validation on another set of randomized samples confirmed the datasetś
robustness, qualifying it for semantic search evaluation.

###  3.2 Evaluation Metrics

Assessing the proficiency and effectiveness of various semantic search
methodologies necessitates the use of specific evaluation metrics. Our
attention centers on the pivotal metrics: Normalized Discounted Cumulative
Gain (nDCG), Mean Reciprocal Rank (MRR), and Mean Average Precision (mAP).
Each metric is integral to an in-depth evaluation of the ability of semantic
search methods to accurately retrieve and rank documents. The formulas for
calculating these metrics are delineated below:

####  3.2.1 Normalized Discounted Cumulative Gain (nDCG)

This metric quantifies the ranking efficacy across search outputs, considering
the varying degrees of relevance each document holds. It takes into account
the weighted relevance of documents, giving precedence to those at the top of
the list. A superior nDCG value signifies that documents of relevance are
appropriately prioritized in the rankings, thus underscoring the significance
of their order in the search results. The nDCG is calculated using the
following equation:

$nDCG@k=\frac{DCG@k}{IDCG@k}$ (1)

where $DCG@k$ (Discounted Cumulative Gain at rank k) is defined as:

$DCG@k=\sum_{i=1}^{k}\frac{2^{rel_{i}}-1}{\log_{2}(i+1)}$ (2)

and $IDCG@k$ is the ideal DCG at k, representing the optimal ranking which
maps to the maximum possible DCG up to position k, ensuring a fair comparison
by normalizing the score. $rel_{i}$ represents the graded relevance of the
result at position $i$.

####  3.2.2 Mean Reciprocal Rank (MRR)

MRR centers on the ranking of the first highly relevant document for a given
search query, offering insights into the speed at which the ranking system is
able to locate and display the most pertinent information. The calculation is
as follows:

$MRR=\frac{1}{|Q|}\sum_{i=1}^{|Q|}\frac{1}{rank_{i}}$ (3)

Here, $|Q|$ is the total number of queries, and $rank_{i}$ is the rank
position of the first very relevant document for the $i^{th}$ query.

####  3.2.3 Mean Average Precision (mAP)

mAP serves as a comprehensive indicator of precision for all pertinent
documents associated with each query, with an average taken across all
queries. Higher mAP scores signify the systemś enhanced consistency in
identifying and retrieving relevant documents across the entire ranking.

The equation for the Average Precision (AP) for a single query is:

$AP=\frac{\sum_{k=1}^{n}(P(k)\times rel_{k}}{\text{\\# relevant\\_documents}}$
(4)

where: \- $n$ is the number of retrieved documents. \- $P(k)$ is the precision
at cutoff $k$ in the list of retrieved documents. \- $rel_{k}$ is the
relevancy score of the document at rank $k$ whether it is very relevant,
somewhat relevant or irrelevant.

To find the Mean Average Precision (MAP) we average the AP scores across all
queries:

$MAP=\frac{\sum_{q=1}^{Q}AP_{q}}{Q}$ (5)

where: \- $AP_{q}$ is the Average Precision for the $q^{th}$ query. \- $Q$ is
the total number of queries.

Employing these metrics allows for the evaluation of different methods’
ranking effectiveness, taking into account essential aspects of accuracy and
efficiency in document ranking that reflect their real-world use in semantic
search scenarios.

###  3.3 Semantic Search Approach

As discussed in the Literature Review section, our evaluation framework is
based on the idea of using encoders to convert documents and queries into
embedding vectors that capture their content. We then determine the cosine
similarity between the embedding vector of a query and those of the documents,
ordering the documents based on these similarity scores.

####  3.3.1 Assessment of Encoders

The success of semantic search ranking relies significantly on the caliber of
encoders used; higher-quality encoders produce more detailed embedding
vectors, which in turn enable more accurate evaluations of similarity between
search queries and documents. In our study, we chose encoders that show the
best performance for Arabic by comparing their results against those obtained
from random document rankings for each query (calculating average evaluation
scores from 30 random rankings samples) and against the worst document
rankings for each query (where documents are sorted by diminishing relevance
to set a benchmark for the lowest achievable scores).

Evaluated encoders:

  * •

Encoder #1: Paraphrase Mulitlingual MiniLM111https://huggingface.co/sentence-
transformers/paraphrase-multilingual-MiniLM-L12-v2:

This is a multi-lingual embedding model that was taught on 50+ languages
covering Arabic and outputs a 384 dimensional embedding vector of the given
sentence. It is mainly implemented for clustering and semantic search.

  * •

Encoder #2: Cmlm Multilingual 222https://huggingface.co/sentence-
transformers/use-cmlm-multilingual:

It is a universal sentence encoder, that is designed for mapping 109 languages
to a common vector space. Leveraging LaBSE as its base model with embedding
vector dimension of 768. Trained for multiple downstream tasks.

  * •

Encoder #3: Paraphrase Mulitlingual Mpnet 333https://huggingface.co/sentence-
transformers/paraphrase-multilingual-mpnet-base-v2:

This embedding model maps sentences & paragraphs to a 768 dimensional dense
vector space, and it works on 50+ languages including Arabic. Trained for
tasks like clustering and semantic similarity.

  * •

Encoder #4: Multilingual Distil Bert 444https://huggingface.co/sentence-
transformers/distiluse-base-multilingual-cased-v1:

It is an embedding model that operates by mapping sentences to a dense vector
space with a dimensionality of 512. It is effective across 15 languages,
including Arabic. Can be used for tasks like clustering or semantic search.

  * •

Encoder #5: Xlm Roberta 555https://huggingface.co/symanto/sn-xlm-roberta-base-
snli-mnli-anli-xnli:

An embedding model that converts texts into a dense vector with 768
dimensions, it works well in Arabic and 12 other languages, it was trained on
SNLI, MNLI, ANLI and XNLI corpora.

###  3.4 RAG Evaluation Setup

Retrieval-Augmented Generation for Arabic semantic search leverages both
retrieval of relevant documents and generation of text to provide answers that
are semantically aligned with the userś query. The complete RAG pipeline is
shown in figure 1.

Figure 1: Retrieved-Augmented-Generation with Semantic Search Pipeline

####  3.4.1 Dataset Creation

In constructing a comprehensive dataset for Retrieval Augmented Generation
(RAG), we curated an extensive collection of Frequently Asked Questions
(FAQs)from four different domains. This compilation contains a total of 816
distinct questions accompanied by their verifiable answers. To build a test
set and ensure its robustness, we employed the advanced capabilities of GPT-4,
generating three nuanced variations for each original question. This approach
yielded a synthesized dataset, optimally prepared for the subsequent testing
of the Retrieval-Augmented Generation (RAG) framework.

####  3.4.2 RAG Pipeline Implementation

The RAG framework was precisely designed to undertake a multi-step process,
aiming to test the semantic search capabilities within the Arabic language
domain. This process is outlined as follows:

  1. 1.

Semantic Encoding: Each generated query undergoes a semantic encoding process
alongside all ground truth questions of the same domain. This step utilizes a
semantic search encoder to identify and retrieve the three closest
semantically relative to the input query.

  2. 2.

Knowledge-Based Answer Generation: The answers corresponding to the
semantically aligned questions, along with the generated query, are presented
to a Large Language Model (LLM). For this purpose, GPT-3.5-turbo was selected
to generate responses based on the knowledge extracted from the answers of the
identified questions. This phase emphasizes the modelś ability to synthesize
and repurpose existing knowledge to address previously unseen queries.

  3. 3.

Assessment Phase: To fact checking of the generated responses, a subsequent
LLM, specifically GPT-4-turbo, evaluates each response. This step involves
comparing the generated answer, the original query, and the ground truth
answer, thereby assessing whether the generated response properly addresses
the query with the same information as the ground truth answer.

  4. 4.

Accuracy Calculation: The final stage in the RAG evaluation framework is the
quantification of accuracy. This metric is derived by calculating the
proportion of queries for which the responses generated by the LLM were
correct, based on the criteria established in the assessment phase.

This systematic evaluation presents potential applicability of semantic search
methods and comparing their performance to the Arabic RAG pipeline.

##  4 Results

This section presents the results for the stand-alone evaluation of semantic
search, followed by assessment of the RAG with the different encoders.

###  4.1 Semantic Search Evaluation results

Table 1: Semantic Search Evaluation Results Model | NDCG@3 | MRR@3 | mAP @3 | Emb. Size
---|---|---|---|---
Encoder #1 | 0.853 | 0.888 | 0.863 | 384
Encoder #2 | 0.789 | 0.798 | 0.793 | 768
Encoder #3 | 0.879 | 0.911 | 0.888 | 768
Encoder #4 | 0.868 | 0.89 | 0.876 | 512
Encoder #5 | 0.837 | 0.848 | 0.854 | 768
Random Ranking | 0.669 | 0.623 | 0.703 | —
Worst Ranking | 0.32 | 0.138 | 0.401 | —

From results represented in Table 1, it is shown that Encoder #3 (paraphrase-
multilingual-mpnet-base-v2) is performing best for Arabic semantic search,
however it has the largest embedding size which allow it to carry more
semantic information, yet require more computational & memory costs.

###  4.2 Correlation between Semantic search accuracy and RAG

Table 2: RAG Using GPT-3.5 Evaluation Encoder | Top 3 Accuracy | Top 1 Accuracy
---|---|---
Encoder #1 | 59.31% | 61.15%
Encoder #2 | 62.01% | 63.23%
Encoder #3 | 63.11% | 63.84%
Encoder #4 | 62.5% | 63.24%
Encoder #5 | 57.84% | N/A
Random Ranking | 6.62% | N/A

In Table 2, some findings were observed. Initially, a minor decline in
accuracy from top-1 to top-3 was detected, attributed to GPT-3.5 merging of
information from the three outcomes of semantic search and failing to identify
the correct answer for a subset of questions.

Furthermore, while Encoder #1 demonstrated commendable performance in the
Semantic Search Evaluation, its effectiveness diminished in the context of
Retrieval Augmented Generation (RAG) evaluation. Contrarily, Encoder #2
exhibited an inverse pattern of performance. This discrepancy can be
attributed to the nature of the semantic search evaluation, which closely
resembles an Asymmetric Semantic Search scenario, where embeddings of
extensive text summaries and brief queries, averaging four words, are
compared. On the other hand, the RAG scenario aligns more with a Symmetric
Semantic Search, where the queries and the reference questions are nearly
equal in length, thereby examining distinct facets and constraints of the
semantic search component. The size of the embedding vector also plays a
crucial role, as larger vectors can encapsulate more information, potentially
enhancing overall performance, particularly for Arabic. This language poses
greater challenges in language modeling compared to English.

##  5 Conclusion

The analysis presented above clearly demonstrates the viability and
significance of incorporating semantic search into the Retrieval Augmented
Generation (RAG) system. This integration has notably enhanced the quality and
precision of generated content. Furthermore, employing semantic search for the
retrieval of documents relevant to a query offers several advantages, such as
the use of shorter prompts in terms of tokens. This not only contributes to
more precise outcomes but also results in cost-effective and quicker
inference. However, further investigations are still required to conclude that
superior encoders lead to superior RAG results.

## References

  * [1] Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck.  Learning deep structured semantic models for web search using clickthrough data.  In Proceedings of the 22nd ACM international conference on Information & Knowledge Management, pages 2333–2338, 2013.
  * [2] Chetana Gavankar, Taniya Bhosale, Dhanashree Gunda, Anindita Chavan, and Shafa Hassan.  A comparative study of semantic search systems.  In 2020 International Conference on Computer Communication and Informatics (ICCCI), pages 1–7. IEEE, 2020.
  * [3] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang.  Retrieval-augmented generation for large language models: A survey.  arXiv preprint arXiv:2312.10997, 2023.
  * [4] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  Advances in Neural Information Processing Systems, 33:9459–9474, 2020.
  * [5] Khaled Shaalan, Sanjeera Siddiqui, Manar Alkhatib, and Azza Abdel Monem.  Challenges in arabic natural language processing.  In Computational linguistics, speech and image processing for arabic language, pages 59–83. World Scientific, 2019.
  * [6] Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel.  Arabic natural language processing: An overview.  Journal of King Saud University-Computer and Information Sciences, 33(5):497–507, 2021.
  * [7] Thomas Hofmann.  Probabilistic latent semantic indexing.  In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pages 50–57, 1999.
  * [8] Chung-Hong Lee and Hsin-Chang Yang.  A classifier-based text mining approach for evaluating semantic relatedness using support vector machines.  In International Conference on Information Technology: Coding and Computing (ITCC’05)-Volume II, volume 1, pages 128–133. IEEE, 2005.
  * [9] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean.  Efficient estimation of word representations in vector space.  arXiv preprint arXiv:1301.3781, 2013.
  * [10] Jeffrey Pennington, Richard Socher, and Christopher D Manning.  Glove: Global vectors for word representation.  In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014.
  * [11] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [12] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [13] Wen Li, Ying Zhang, Yifang Sun, Wei Wang, Mingjie Li, Wenjie Zhang, and Xuemin Lin.  Approximate nearest neighbor search on high dimensional data—experiments, analyses, and improvement.  IEEE Transactions on Knowledge and Data Engineering, 32(8):1475–1488, 2019.
  * [14] Matthijs Douze, Alexandr Guzhva, Chengqi Deng, Jeff Johnson, Gergely Szilvasy, Pierre-Emmanuel Mazaré, Maria Lomeli, Lucas Hosseini, and Hervé Jégou.  The faiss library.  arXiv preprint arXiv:2401.08281, 2024.
  * [15] Mahmoud Al-Ayyoub, Aya Nuseir, Kholoud Alsmearat, Yaser Jararweh, and Brij Gupta.  Deep learning for arabic nlp: A survey.  Journal of computational science, 26:522–531, 2018.
  * [16] Ons Meddeb, Mohsen Maraoui, and Mounir Zrigui.  Deep learning based semantic approach for arabic textual documents recommendation.  In 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA), pages 1–6. IEEE, 2021.
  * [17] Aya M Al-Zoghby and Khaled Shaalan.  Semantic search for arabic.  In The Twenty-Eighth International Flairs Conference, 2015.
  * [18] Hazem Abdelazim, Mohamed Tharwat, and Ammar Mohamed.  Semantic embeddings for arabic retrieval augmented generation (arag).
  * [19] Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham.  In-context retrieval-augmented language models.  Transactions of the Association for Computational Linguistics, 11:1316–1331, 2023.


# RMP-Loss: Regularizing Membrane Potential Distribution for Spiking Neural
Networks

Yufei Guo, Xiaode Liu111, Yuanpei Chen, Liwen Zhang, Weihang Peng, Yuhan
Zhang,
Xuhui Huang, Zhe Ma
Intelligent Science & Technology Academy of CASIC, China
Scientific Research Laboratory of Aerospace Intelligent Systems and
Technology, China
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Equal
contribution.Corresponding author.

###### Abstract

Spiking Neural Networks (SNNs) as one of the biology-inspired models have
received much attention recently. It can significantly reduce energy
consumption since they quantize the real-valued membrane potentials to 0/1
spikes to transmit information thus the multiplications of activations and
weights can be replaced by additions when implemented on hardware. However,
this quantization mechanism will inevitably introduce quantization error, thus
causing catastrophic information loss. To address the quantization error
problem, we propose a regularizing membrane potential loss (RMP-Loss) to
adjust the distribution which is directly related to quantization error to a
range close to the spikes. Our method is extremely simple to implement and
straightforward to train an SNN. Furthermore, it is shown to consistently
outperform previous state-of-the-art methods over different network
architectures and datasets.

##  1 Introduction

Recently, many efforts have been done to make deep neural networks (DNNs)
lightweight, so that they can be deployed in devices where energy consumption
is limited. To this end, several approaches have been proposed, including
network pruning [65], network quantization [18, 40, 15], knowledge
transfer/distillation [49], neural architecture search [69, 42], and spiking
neural networks (SNNs) [23, 19, 22, 17, 56, 39, 47, 59, 58, 67, 53, 60, 61].
The SNN provides a special way to reduce energy consumption following the
working mechanism of the brain neuron. Its neurons accumulate spikes from
previous neurons and present spikes to posterior neurons when the membrane
potential exceeds the firing threshold. This information transmission paradigm
will convert the computationally expensive multiplication to computationally
convenient additions thus making SNNs energy-efficient when implemented on
hardware. Specialized neuromorphic hardware based on an event-driven
processing paradigm is currently under various stages of development, e.g.,
SpiNNaker [31], TrueNorth [1], Tianjic [48], and Loihi [9], where SNNs can be
efficiently implemented further. Due to the advantage of computational
efficiency and rapid development of neuromorphic hardware, the SNN has gained
more and more attention.

Figure 1: The overall workflow of the proposed method. We embed a membrane
potential regularization loss in the task loss to redistribute the membrane
potential in the training phase to reduce the quantization error.

Although SNNs have been widely studied, their performance is still not
comparable with that of DNNs. This performance gap is largely related to the
quantization of the real-valued membrane potential to 0/1 spikes for the
firing of the SNN in implementation [21]. The excessive information loss
induced by the firing activity forcing all information only to two values will
cause accuracy to decrease. Although information loss is important for
computer vision tasks and the quantization will cause information loss too,
the essential role of the activation function is to introduce non-linearity
for neural networks [46]. Therefore, how to effectively reduce the information
loss of membrane potential quantization is of high research importance.
However, as far as we know, few studies have focused on directly solving this
problem. The quantization error problem also exists in these methods that
convert the ANN model to an SNN model [11, 4, 39]. However, these methods
solve the problem by changing the activation function in ANNs or increasing
timesteps in SNNs, which don’t work for SNN training. InfLoR-SNN [21] adds a
membrane potential redistribution function in the spiking neuron to reduce the
quantization error via redistributing the membrane potential. However, it will
decrease the biological plausibility of the SNN and increase the inference
burden. This paper focuses on reducing the quantization error in SNN training
directly and aims to introduce no burden for the SNN.

Quantization error is smaller when the membrane potential is close to the
spiking threshold or reset values [21]. Hence, to mitigate the information
loss, we suggest redistributing the membrane potential to where the membrane
potential is closer to the 0/1 spike. Then an additional loss term aims at
regularizing membrane potential is presented, called RMP-Loss, which can
encourage the membrane potentials to gather around binary spike values during
the training phase. The workflow of our method is shown in Fig. 1. Our main
contributions can be concluded as follows:

  * •

To our best knowledge, there have been few works noticing the quantization
error in direct training of SNNs. To mitigate the quantization error, we
present the RMP-Loss, which is of benefit to training an SNN model that enjoys
a narrow gap between the membrane potential and its corresponding 0/1 spike.
Furthermore, we also provide theoretical proof to clarify why the RMP-Loss can
prevent information loss.

  * •

Some existing methods can address information loss too. While achieving
comparable performance, more parameters or computation burdens are also
introduced in the inference phase. Different from those methods, the RMP-Loss
can handle information loss directly without introducing any additional
parameters or inference burden.

  * •

Extensive experiments on both static and dynamic datasets show that our method
performs better than many state-of-the-art SNN models.

##  2 Related Work

There are two main problems to train an SNN model [22]. The first is that the
firing activity is non-differentiable thus the SNN model cannot be trained
with these back-propagation methods directly. One way to mitigate the
optimization difficulty is through surrogate functions. This kind of method
replaces the non-differentiable firing activity with a differentiable
surrogate function to calculate the gradient in the back-propagation [37, 55,
45, 54]. In [2], the derivative of a truncated quadratic function was used to
approximate the derivative of the firing activity function. In [63] and [7],
the derivatives of a sigmoid [63] and a rectangular function were respectively
adopted to construct the surrogate gradient. Furthermore, a dynamic
evolutionary surrogate gradient that could maintain accurate gradients during
the training was proposed in [41, 20, 5, 6]. Another way to solve the non-
differentiable problem is converting an ANN model to an SNN model, known as
ANN-SNN conversion methods [27, 26, 39, 32, 3, 29, 28]. The ANN-SNN method
trains a high-accuracy ANN with ReLU activation first and then transforms the
network parameters into an SNN under the supposition that the activation of
the ANN can be approximated by the average firing rates of an SNN. However,
conversion methods will introduce conversion errors inevitably. Many efforts
were made to tackle this problem, such as long inference time [14], threshold
rescaling [52], soft reset [27], threshold shift [39], and the quantization
clip-floor-shift activation function [4]. This paper adopts the surrogate
gradient method.

The second problem is information loss. Though transmitting information with
binary spikes is more efficient than that with real values, quantizing these
real-valued membrane potentials to 0/1 spikes will cause information loss.
There have been some works handling this problem indirectly. Such as learning
the appropriate membrane leak-and-firing threshold [50, 68]. These are of
benefit to finding a preferable quantization choice. Similarly, in [17, 62],
some learnable neuron parameters are incorporated into the SNN to optimize the
quantization choice. In [24], three regularization losses are introduced to
penalize three undesired shifts of membrane potential distribution to make
easy optimization and convergence for the SNN. The regularization is also
beneficial to reduce quantization error. However, all these methods do not
handle the information loss problem directly and most of them require more
parameters. InfLoR-SNN [21] suggests adding a membrane potential
redistribution function in the spiking neuron to redistribute the membrane
potential closer to spike values, which can be seen as a direct method to
reduce the quantization error. However, adding another function in spiking
neurons will decrease the biological plausibility of the SNN and increase the
inference burden. In this work, we focus on handling the quantization error
problem directly without introducing extra parameters or computation burden in
the inference phase.

##  3 Preliminary

In this section, a brief review of the primary computing element of SNNs is
first provided in Sec. 3.1. Then we introduce the proposed surrogate gradient
that handles the non-differentiability challenge in Sec. 3.2. Finally, in Sec.
3.3, we describe the threshold-dependent batch normalization technique used in
this work.

###  3.1 Spiking Neuron Model

Spiking neuron. The primary computing element, i.e., the neuron of an SNN is
much different from that of an ANN. The neuron of an ANN plays the role of
nonlinear transformation and can output real-valued values. Meanwhile, the
neuron in an SNN accumulates the information from previous neurons into its
membrane potential and presents the spike to the following neurons when its
membrane potential exceeds the firing threshold. The SNN is more efficient
with this special information transmission paradigm while suffering the
information loss problem. A unified form of this spiking neuron model can be
formulated as follows,

$u^{(t),{\rm pre}}=\tau u^{(t-1)}+x^{(t)},$ (1)
$o^{(t)}=\left\\{\begin{array}[]{lll}1,\ \ {\rm if}\;u^{(t),{\rm pre}}\geq
V_{\rm th}\\\ 0,\ \ {\rm otherwise}\end{array},\right.$ (2)
$u^{(t)}=u^{(t),{\rm pre}}(1-o^{(t)}).$ (3)

Where $u^{(t),{\rm pre}}$ and $u^{(t)}$ are pre-membrane potential and
membrane potential, respectively, $x^{(t)}$ is input information, $o^{(t)}$ is
output spike at the timestep $t$. The range of constant leaky factor $\tau$ is
$(0,1)$, and we set $\tau$ as $0.25$. $V_{\rm th}$ is the firing threshold and
is set to $0.5$ here.

Output layer neuron. In the general classification task, the output of the
last layer will be presented to the Softmax function first and then find the
winner class. We make the neuron model in the output layer only accumulate the
incoming inputs without any leakage as final output like doing in recent
practices[50, 66], described by

$u^{(t)}=u^{(t-1)}+x^{(t)}.$ (4)

Then the cross-entropy loss is calculated according to the final membrane
potential at the last timesteps, $u^{(T)}$.

###  3.2 Surrogate Gradients of SNNs

As shown in Eq. 2. the firing activity of SNNs can be seen as a step function.
Its derivative is $0$ everywhere except at $V_{\rm th}$. This non-
differentiability problem of firing activity will cause gradient vanishing or
explosion and make the back-propagation unsuitable for training SNNs,
directly. As mentioned before, many prior works adopted the surrogate function
to replace the firing function to obtain a suitable gradient [7, 66, 56]. In
InfLoR-SNN [21], an extra membrane potential redistribution function is added
before the firing function. It argues that this redistribution function will
reduce the information loss in forward propagation. However, investigating it
from a backward propagation perspective and considering the redistribution
function being next to the firing function, it can be seen as a more suitable
surrogate gradient if we gather the STE (used in [21]) gradients of the firing
function and redistribution function together. In this sense, the suitable
surrogate gradient can also be seen as a route to reduce quantization error.
Therefore, in this paper, we adopt the gathered redistribution function in
InfLoR-SNN as our surrogate function, which indeed performs better than other
methods [7, 66, 56] in our experiments, denoted as

$\varphi(u)=\left\\{\begin{array}[]{ll}0,&u\textless 0,\\\ \frac{1}{2{\rm
tanh}(3/2)}{\rm tanh}(3(u-1/2))+1/2,&0\leq u\leq 1,\\\ 1,&u\textgreater
1.\end{array}\right.$ (5)

###  3.3 Threshold-dependent Batch Normalization

Normalization techniques can effectively reduce the training time and
alleviate the gradient vanishing or explosion problem for training DNNs. A
most widely used technique called batch normalization (BN) [30] uses the
distribution of the summed input to a neuron over a mini-batch of training
cases to compute a mean and variance which are then used to normalize the
summed input to that neuron on each training case. This is significantly
effective for convolutional neural networks (CNNs). However, directly applying
BN to SNNs will ignore the temporal characteristic and can not achieve the
desired effect. To this end, a more suitable normalization technique for SNNs,
named threshold-dependent BN (tdBN) was further proposed in [66]. It
normalizes feature inputs on both temporal and spatial dimensions. Besides, as
its name implies, tdBN makes normalized variance dependent on the firing
threshold, i.e., the pre-activations are normalized to $N(0,(\alpha V_{\rm
th})^{2})$ instead of $N(0,1)$. this can balance pre-synaptic input and
threshold to maintain a reasonable firing rate. In this paper, we also adopt
the tdBN, as follows,

$\bar{\textbf{X}}=\alpha V_{\rm
th}\frac{\textbf{X}-\mathrm{mean}(\mathbf{X})}{\sqrt{\mathrm{mean}((\mathbf{X}-\mathrm{mean}(\mathbf{X}))^{2})+\epsilon}},$
(6) $\tilde{\textbf{X}}=\lambda\bar{\textbf{X}}+\beta.$ (7)

where $\textbf{X}\in\mathbb{R}^{T\times B\times C\times H\times W}$ ($T$:
timesteps; $B$: batch size; $C$: channel; $(H,W)$: spatial domain) is a 5D
tensor and $\alpha$ is a hyper-parameter set as 1.

##  4 RMP-Loss

As aforementioned, we suggest reducing the quantization error to avoid
information loss in supervised training-based SNNs. In this section, we first
try to provide a metric to measure the quantization error in SNNs. Then based
on the metric, we introduce our method, a loss term for regularizing membrane
potential, i.e., RMP-Loss, which can push the membrane potentials close to
spike values to reduce the quantization error. Finally, theoretical proof to
clarify why the RMP-Loss can prevent information loss is given.

###  4.1 Quantization Error Estimation

Estimating the quantization error precisely is important for designing a
method to reduce the prediction error. Hence, before giving the full details
of the RMP-Loss, here we try to formulate the quantization error first. As
shown in Eq. 2, the output of a neuron is dependent on the magnitude of the
membrane potential. When the membrane potential exceeds the firing threshold,
the output is $1$, otherwise, is $0$; that is, these full-precision values
will be mapped to only two spike values. The difference between output, $o$,
and membrane potential, $u$, can be measured as

$\mathrm{Dis}(o,u)=\left\\{\begin{array}[]{lll}1-u,\ \ {\rm if}\;u\geq V_{\rm
th}\\\ 0-u,\ \ {\rm otherwise}\end{array}.\right.$ (8)

Obviously, the closer a membrane potential is to its corresponding spike, the
smaller the quantization error. The quantization error depends on the margin
between the membrane potential and its corresponding spike. Then we define the
quantization error as

$q(u)=|\mathrm{Dis}(o,u)|^{p},$ (9)

where $p>0$, which is usually set to $1$ and/or $2$ corresponding to L1-norm
and/or L2-norm. Unless otherwise specified, we set it as 2 in the paper.

###  4.2 Regularizing Membrane Potential Distribution

To mitigate the information loss problem in SNNs, the quantization error
should be reduced to some extent. To this end, we propose RMP-Loss to carve
the distribution to a range close to the spikes based on $q(u)$, as

$\mathcal{L}_{\rm RMP}=\frac{1}{TLBCWH}\sum_{t,l,b,c,w,h}{q(u)}.$ (10)

where $T$, $L$, $B$, $C$, $W$, and $H$ denote the number of timesteps, the
number of layers, batch size, channel length, width, and height of the
membrane potential map, respectively. Finally, taking classification loss into
consideration, the total loss can be written as

$\mathcal{L}_{\rm CE-RMP}=\mathcal{L}_{\rm CE}+\lambda(n)\mathcal{L}_{\rm
RMP},$ (11)

where $\mathcal{L}_{CE}$ is the cross-entropy loss, and $\lambda(n)$ is a
balanced coefficient, which changes dynamically with the training epoch.
Obviously, $\mathcal{L}_{RMP}$ does not take into account the classification
problem at hand and will add a new constraint in the network optimization. To
make the network parameter update have enough freedom at the beginning, which
is important to train a well-performance network, and still focus on the
classification task at the end. We adopt a strategy of increasing first and
decreasing later to adjust the $\lambda(n)$ as follows,

$\lambda(n)=\left\\{\begin{array}[]{lll}2k\frac{n}{N},\ \ {\rm if}\;n\leq
N/2\\\ 2k(1-\frac{n}{N}),\ \ {\rm otherwise}\end{array},\right.$ (12)

where $n$ is the $n$-th epoch and $k$ is a coefficient, which controls the
amplitude of regularization. Unless otherwise specified, we set it as 0.1 in
the paper.

Algorithm 1 Training an SNN with RMP-Loss.

Input: An SNN to be trained; Training dataset, total number of training epochs
$N$, total number of training iterations per epoch $I$.
Output: The trained SNN.

1: for All $n=1,2,\dots,N-th$ epoch do

2: for All $i=1,2,\dots,I-th$ iteration do

3: Feed-Forward:

4: Calculate the SNN output, $\mathbf{O}^{i}$, membrane potential,
$\mathbf{U}$, and class label, $\mathbf{Y}^{i}$;

5: Compute classification loss $\mathcal{L}_{\rm CE}={{\mathcal{L}}_{\rm
CE}}(\mathbf{O},\mathbf{Y}^{i})$;

6: Compute RMP-Loss ${\mathcal{L}}_{\rm RMP}={\mathcal{L}}_{\rm
RMP}(\mathbf{U})$;

7: Compute the total loss $\mathcal{L}_{\rm CE-RMP}=\mathcal{L}_{\rm
CE}+\lambda(n)\mathcal{L}_{\rm RMP}$. Backward-Propagation:

8: Calculate the gradients $\Delta\mathbf{W}=\frac{\partial{\mathcal{L}_{\rm
CE-RMP}}}{\partial\mathbf{W}}=\sum_{t}{\frac{\partial({\mathcal{L}_{\rm
CE}+\lambda(n)\mathcal{L}_{\rm
RMP}})}{\partial\mathbf{y}^{t}}\frac{\partial\mathbf{y}^{t}}{\partial\mathbf{{U}}}\frac{\partial\mathbf{U}}{\partial\mathbf{W}}}$,
where $\mathbf{y}^{t}$ is feature maps at $t$-th timesteps and
$\frac{\partial\mathbf{y}^{t}}{\partial\mathbf{{U}}}$ is the gradient of
firing function, which can be calculated as
$\frac{\partial\varphi(\mathbf{U})}{\partial\mathbf{{U}}}$. Parameters Update:

9: Update all parameters with the learning rate $\eta$.

10: end for

11: end for

12: return the trained SNN.

The final resulting RMP-Loss training method is defined in Algo. 1.

###  4.3  Analysis and Discussion

In this work, we assume that RMP-Loss can help reduce the quantization loss of
the SNNs. To verify our assumption, a theoretical analysis is conducted by
using the information entropy concept. To mitigate the information loss, the
output spike tensor $\mathbf{O}$ should reflect the information of the
membrane potential tensor $\mathbf{U}$ as much as possible. Since KL-
divergence is a metric to measure the difference between two random variable
distributions. We use KL-divergence to compute the difference between
$\mathbf{O}$ and $\mathbf{U}$, which refers to the difficulty of
reconstructing $\mathbf{U}$ from $\mathbf{O}$, i.e., information loss degree.
Then we provide the theoretical analysis below.

Let $x^{u}\sim\mathcal{U}$ where $x^{u}$ are samples $\in\mathbf{U}$ and
$\mathcal{U}$ represents the distribution of $x^{u}$. Similarily, we have
$x^{o}\sim\mathcal{O}$ where $x^{o}\in\mathbf{O}$. We use $P_{U}(x)$ and
$P_{O}(x)$ to denote the probability density function for $\mathcal{U}$ and
$\mathcal{O}$, respectively. Then the information loss for the spike firing
process can be described as

$\mathcal{L}_{KL}(\mathcal{U}||\mathcal{O})=\int_{-\infty}^{\infty}P_{U}(x^{u}){\rm
log}\frac{P_{U}(x^{u})}{P_{O}(x^{o})}dx.$ (13)

Since output spike $x^{o}$ is discrete, we can update it as

$\mathcal{L}_{KL}(\mathcal{U}||\mathcal{O})=\int_{o-\epsilon}^{o+\epsilon}P_{U}(x^{u})\log\frac{P_{U}(x^{u})}{P_{O}(x^{o})}dx,$
(14)

where $\epsilon$ is a small constant, $o$ is the spike value, $P_{O}(x^{o})$
can be seen as a very large constant in this situation, and
$\int_{o-\epsilon}^{o+\epsilon}P_{O}(x^{o})dx=1$. With RMP-Loss, it will
become

$\mathcal{L}_{KL}(\hat{\mathcal{U}}||\mathcal{O})=\int_{o-\epsilon}^{o+\epsilon}P_{\hat{U}}(x^{\hat{u}})\log\frac{P_{\hat{U}}(x^{\hat{u}})}{P_{O}(x^{o})}dx,$
(15)

where $x^{\hat{u}}\in\hat{\mathcal{U}}$ and $\hat{\mathcal{U}}$ is the new
distribution of $x^{\hat{u}}$ adjusted by RMP-Loss. Here, we have the
following propositions.

Proposition 1  $\frac{d\mathcal{L}_{KL}}{dP_{U}(x^{u})}<0$, i.e.,
$\mathcal{L}_{KL}$ $\downarrow$ as $P_{U}(x^{u})$ $\uparrow$.

proof: When a membrane potential $u$ is below the firing threshold, it will be
pushed to 0 by the RMP-Loss, considering its effect, and otherwise, 1. Hence
the firing rate of the neuron can be seen as the same, whether using the RMP-
Loss or not, i.e., $P_{O}(x^{o})$ keep the same for $\hat{\mathbf{U}}$ and
$\mathbf{U}$.

$\displaystyle\frac{d\mathcal{L}_{KL}}{dP_{U}(x^{u})}$
$\displaystyle=\frac{d\int_{o-\epsilon}^{o+\epsilon}P_{U}(x^{u})\log\frac{P_{U}(x^{u})}{P_{O}(x^{o})}dx}{dP_{U}(x^{u})}$
(16)
$\displaystyle=\int_{o-\epsilon}^{o+\epsilon}(\log\frac{P_{U}(x^{u})}{P_{O}(x^{o})}+\frac{1}{\rm{ln2}})dx.$
(17)

Since $P_{O}(x^{o})$ is much larger than $P_{U}(x^{u})$, we can get
$\int_{o-\epsilon}^{o+\epsilon}(\log\frac{P_{U}(x^{u})}{P_{O}(x^{o})}+\frac{1}{\rm{ln2}})dx<0$,
then $\frac{d\mathcal{L}_{KL}}{dP_{U}(x^{u})}<0$. $\blacksquare$

Proposition 2  $P_{\hat{U}}(x^{\hat{u}})>P_{U}(x^{u})|x=o$.

proof: We assume that there are $n^{u}_{\epsilon}$ samples in the interval
$(o-\epsilon,o+\epsilon)$ for $\mathcal{U}$, where $\epsilon$ is a small
constant. Then we can get
$P_{U}(x^{u})\approx\frac{n^{u}_{\epsilon}}{2\epsilon}$. And we can simply see
the RMP-Loss as a function to push the vanilla $\mathbf{U}$ to
$\hat{\mathbf{U}}$, which is closer to $\mathbf{O}$, based on its effect.
Therefore, we can assume that these samples will be gathered to a new interval
$(o-{\epsilon}_{l},o+{\epsilon}_{r})$, where
${\epsilon}_{l},{\epsilon}_{r}<{\epsilon}$. Then we can get
$P_{\hat{U}(x^{\hat{u}})}\approx\frac{n^{u}_{\epsilon}}{{\epsilon}_{l}+{\epsilon}_{r}}$.
Thereby we can have $P_{\hat{U}(x^{\hat{u}})}>P_{U}(x^{u})|_{x=o}$.
$\blacksquare$

Along with Proposition 1 and Proposition 2, we can conclude that
$\mathcal{L}_{KL}(\hat{U}||O)<\mathcal{L}_{KL}({U}||O)$, i.e., our method with
RMP-Loss enjoys lesser information loss.

##  5 Experiment

In this section, we first conducted extensive ablation studies to compare the
SNNs with RMP-Loss and their vanilla counterparts to verify the effectiveness
of the method. Next, we fully compared our method with other state-of-the-art
(SoTA) methods. Finally, some further experimental visualizations are provided
to understand the RMP-Loss.

Table 1: Ablation Study for RMP-Loss Dataset | Architecture | Type | Timestep | Accuracy | Quantization error
---|---|---|---|---|---
CIFAR-10 | ResNet20 | Without RMP-Loss | 4 | 91.26% | 0.186
With RMP-Loss | 4 | 91.89% | 0.121
ResNet19 | Without RMP-Loss | 4 | 95.04% | 0.128
With RMP-Loss | 4 | 95.51% | 0.104
VGG16 | Without RMP-Loss | 4 | 92.80% | 0.174
With RMP-Loss | 4 | 93.33% | 0.135

###  5.1 Datasets and Settings

Datasets. We conducted experiments on four benchmark datasets: CIFAR-10 (100)
[33], CIFAR10-DVS [38], and ImageNet (ILSVRC12) [10]. The CIFAR-10 (100)
dataset includes 60,000 images in 10 (100) classes with $32\times 32$ pixels.
The numbers of training images and test images are 50,000 and 10,000. The
CIFAR10-DVS dataset is the neuromorphic version of the CIFAR-10 dataset. It is
composed of 10,000 images in 10 classes. We split the dataset into 9000
training images and 1000 test images similar to [56]. ImageNet dataset
consists of 1,250,000 training images and 50,000 test images.

Preprocessing. We applied data normalization on all static datasets to make
input images have $0$ mean and $1$ variance. Besides, we conducted random
horizontal flipping and cropping on these datasets to avoid overfitting. For
CIFAR, the AutoAugment [8] and Cutout [13] were also used for data
augmentation as doing in [24, 20]. For the neuromorphic dataset, we resized
the training image frames to $48\times 48$ as in [66] and adopted random
horizontal flip and random roll within $5$ pixels for augmentation. And the
test images were merely resized to $48\times 48$ without any additional
processing.

Training setup. For all the datasets, the firing threshold $V_{\rm th}$ was
set as $0.5$. For static image datasets, the images were encoded to binary
spike using the first layer of the SNN, as in recent works [50, 17, 16]. This
is similar to rate coding. For the neuromorphic image dataset, we used the
$0/1$ spike format directly. The neuron models in the output layer accumulated
the incoming inputs without generating any spike as the output like in [50].
For CIFAR-10 (100) and CIFAR10-DVS datasets, we used the SGD optimizer with
the momentum of $0.9$ and learning rate of $0.01$ with cosine decayed [43] to
$0$ as in [21]. All models were trained within $400$ epochs with the same
batch size of $128$. For the ImageNet dataset, we adopted the SGD optimizer
with a momentum of $0.9$ and a learning rate of $0.1$ with cosine decayed to
$0$. All models are trained within $320$ epochs.

###  5.2 Ablation Study for RMP-Loss

We conducted a set of ablation experiments on CIFAR-10 using ResNet20,
ResNet19, and VGG16 as backbones. The results are shown in Tab. 1. It can be
seen that with the RMP-Loss, these SNNs can achieve higher accuracy than their
vanilla counterparts. We also show the membrane potential distribution of the
first layer of the second block in the ResNet20 with and without RMP-Loss on
the test set of CIFAR-10 in Fig. 2. It can be seen that the models trained
with RMP-Loss can shrink the membrane potential distribution range which
enjoys less quantization error.

Figure 2: The effect of RMP-Loss. The overall original membrane potential distribution (left) and the redistributed membrane potential distribution by RMP-Loss (right) of the first layer of the second block in ResNet20 on CIFAR-10 test sets. Table 2: Comparison with SoTA methods on CIFAR-10/100. Dataset | Method | Type | Architecture | Timestep | Accuracy
---|---|---|---|---|---
CIFAR-10 | SpikeNorm [52] | ANN2SNN | VGG16 | 2500 | 91.55%
Hybrid-Train [51] | Hybrid training | VGG16 | 200 | 92.02%
Spike-Thrift [35] | Hybrid training | VGG16 | 100 | 91.29%
Spike-basedBP [36] | SNN training | ResNet11 | 100 | 90.95%
STBP [56] | SNN training | CIFARNet | 12 | 90.53%
TSSL-BP [64] | SNN training | CIFARNet | 5 | 91.41%
PLIF [17] | SNN training | PLIFNet | 8 | 93.50%
DSR [44] | SNN training | ResNet18 | 20 | 95.40%
Joint A-SNN [23] | SNN training | ResNet18 | 4 | 95.45%
Diet-SNN [50] | SNN training | VGG16 | 5 | 92.70%
10 | 93.44%
ResNet20 | 5 | 91.78%
10 | 92.54%
RecDis-SNN [24] | SNN training | ResNet19 | 2 | 93.64%
4 | 95.53%
6 | 95.55%
STBP-tdBN [66] | SNN training | ResNet19 | 2 | 92.34%
4 | 92.92%
6 | 93.16%
TET [12] | SNN training | ResNet19 | 2 | 94.16%
4 | 94.44%
6 | 94.50%
InfLoR-SNN [21] | SNN training | ResNet19 | 2 | 94.44%
4 | 96.27%
6 | 96.49%
RMP-Loss | SNN training | ResNet19 | 2 | 95.31%$\pm 0.07$
4 | 95.51%$\pm 0.08$
6 | 96.10%$\pm 0.08$
| ResNet20 | 4 | 91.89%$\pm 0.05$
| 6 | 92.55%$\pm 0.06$
| VGG16 | 4 | 93.33%$\pm 0.07$
| 10 | 94.39%$\pm 0.08$
CIFAR-100 | DSR [44] | SNN training | ResNet18 | 20 | 78.50%
InfLoR-SNN [21] | SNN training | VGG16 | 5 | 71.56%
IM-Loss [20] | SNN training | VGG16 | 5 | 70.18%
Diet-SNN [50] | SNN training | ResNet20 | 5 | 64.07%
VGG16 | 5 | 69.67%
Real Spike [25] | SNN training | ResNet20 | 5 | 66.60%
VGG16 | 5 | 70.62%
TET [12] | SNN training | ResNet19 | 2 | 72.87%
4 | 74.47%
6 | 74.72%
RMP-Loss | SNN training | ResNet20 | 4 | 66.65%$\pm 0.10$
VGG16 | 4 | 72.55%$\pm 0.08$
10 | 73.30%$\pm 0.11$
ResNet19 | 2 | 74.66%$\pm 0.12$
4 | 78.28%$\pm 0.10$
6 | 78.98%$\pm 0.08$

###  5.3 Comparison with State-of-the-art Methods

We evaluated the proposed method with the accuracy performance on various
widely used static and neuromorphic datasets using spiking ResNet20 [50, 52],
VGG16 [50], ResNet18 [16], ResNet19 [66], and ResNet34 [16]. The results with
the mean accuracy and standard deviation of 3-trials are listed in Tab. 2.

CIFAR-10. For CIFAR-10, we tested three different network architectures. The
performance of our method is shown in Tab. 2. It can be seen that our method
achieves the best results in most of these cases. In special, using ResNet19
[66], the RMP-Loss achieves 96.10% averaged top-1 accuracy, which improves
1.60% absolute accuracy compared with the existing state-of-the-art TET [12].
Though InfLoR-SNN [21] is slightly better than our method for 4 and 6
timesteps, it will decrease the biological plausibility of the SNN and
increase the inference burden since a complex transform function is added in
its spiking neuron. On ResNet20, our method can achieve higher accuracy with
only 6 timesteps, while Diet-SNN [50] with 10 timesteps. On VGG16, the RMP-
Loss also shows obvious advantages.

CIFAR-100. For CIFAR-100, we also experimented with these three different
network structures. For all these configurations, our method achieves
consistent and significant accuracy over prior work. ResNet20-based and
VGG16-based RMP-Loss achieve 66.65% and 72.55% top-1 accuracy with only 4
timesteps, which outperform their Diet-SNN counterparts with 2.58% and 2.88%
higher accuracy and Real spike counterparts with 0.05% and 1.93% respectively
but with fewer timesteps. Noteworthy, our method significantly surpasses TET
with 4.26% higher accuracy on ResNet19, which is not easy to achieve in the
SNN field. Overall, our results on CIFAR-100 show that, when applied to a more
complex dataset, the RMP-Loss achieves an even more favorable performance
compared to competing methods.

Table 3: Comparison with SoTA methods on CIFAR10-DVS. Dataset | Method | Type | Architecture | Timestep | Accuracy
---|---|---|---|---|---
CIFAR10-DVS | Rollout [34] | Rollout | DenseNet | 10 | 66.80%
STBP-tdBN [66] | SNN training | ResNet19 | 10 | 67.80%
LIAF [57] | Conv3D | LIAF-Net | 10 | 71.70%
LIAF [57] | LIAF | LIAF-Net | 10 | 70.40%
RecDis-SNN [24] | SNN training | ResNet19 | 10 | 72.42%
InfLoR-SNN [21] | SNN training | ResNet19 | 10 | 75.50%
ResNet20 | 10 | 75.10%
RMP-Loss | SNN training | ResNet19 | 10 | 76.20%$\pm 0.20$
ResNet20 | 10 | 75.60%$\pm 0.30$
Table 4: Comparison with SoTA methods on ImageNet. Method | Type | Architecture | Timestep | Spike form | Accuracy
---|---|---|---|---|---
Hybrid-Train [51] | Hybrid training | ResNet34 | 250 | Binary | 61.48%
SpikeNorm [52] | ANN2SNN | ResNet34 | 2500 | Binary | 69.96%
STBP-tdBN [66] | SNN training | ResNet34 | 6 | Binary | 63.72%
TET [12] | SNN training | ResNet34 | 6 | Binary | 64.79%
SEW ResNet [16] | SNN training | ResNet18 | 4 | Integer | 63.18%
ResNet34 | 4 | Integer | 67.04%
Spiking ResNet [16] | SNN training | ResNet18 | 4 | Binary | 62.32%
ResNet34 | 4 | Binary | 61.86%
RMP-Loss | SNN training | ResNet18 | 4 | Binary | 63.03%$\pm 0.07$
ResNet34 | 4 | Binary | 65.17%$\pm 0.07$

CIFAR10-DVS. We also verified our method on the popular neuromorphic dataset,
CIFAR10-DVS. It can be seen that RMP-Loss also shows amazing performance. RMP-
Loss outperforms STBP-tdBN by 12.39%, RecDis-SNN by 3.78%, and InfLoR-SNN by
0.70% respectively with 76.20% top-1 accuracy in 10 timesteps using ResNet19
as the backbone. With ResNet20 as the backbone, RMP-Loss can also achieve
well-performed results.

ImageNet. For ImageNet, we conducted experiments with ResNet18 and ResNet34 as
backbones. Our results are presented in Tab. 4. In these normal spiking
structures, our method achieves the highest accuracy among these SoTA prior
works. However, our method is slightly worse than SEW ResNet [16]. This is
because that SEW ResNet uses an atypical architecture, which abandons binary
spike forms but will output arbitrary integers to transmit information, thus
enjoying higher accuracy but the efficiency from multiplication free of SNNs
will be lost. Hence, it would be limited in some applications while our method
not.

###  5.4 Visualization

Figure 3: The original membrane potential distribution (left) and the
redistributed membrane potential distribution by RMP-Loss (right) of the first
layer of the second block in ResNet20/19 on CIFAR-10 test sets.

Furthermore, we also provide some experimental visualizations to show the
regularizing effects of RMP-Loss. Fig. 3 shows the membrane potential
distribution of the first layer of the second block in the ResNet20/19 with
and without RMP-Loss on the test set of CIFAR-10. It can be seen that the SNN
models trained with RMP-Loss can shrink the membrane potential distribution
range which enjoys less quantization error. On the other hand, comparing the
membrane potential distribution difference between ResNet20 and ResNet19, it
can be found that the membrane potential distribution of ResNet19 is thinner
than that of ResNet20 too. Considering that ResNet19 can achieve higher
accuracy than ResNet20, it also shows that reducing quantization error is
effective to improve the accuracy of SNN models and our route to improve the
SNN accuracy by reducing the quantization error is reasonable.

##  6 Conclusion

This paper aims at addressing the information loss problem caused by the $0/1$
spike quantization of SNNs. We introduce RMP-Loss to adjust the membrane
potential to reduce the quantization error. Different from other methods that
reduce the quantization error indirectly or will induce more parameters, RMP-
Loss focuses on handling this problem directly and will introduce no extra
parameters in the inference phase. We show that our method outperforms SoTA
methods on both static and neuromorphic datasets.

## Acknowledgment

This work is supported by grants from the National Natural Science Foundation
of China under contracts No.12202412 and No.12202413.

## References

  * [1] F. Akopyan, J. Sawada, A. Cassidy, R. Alvarez-Icaza, and Dharmendra S. Modha.  Truenorth: Design and tool flow of a 65 mw 1 million neuron programmable neurosynaptic chip.  IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 34(10):1537–1557, 2015.
  * [2] S. M. Bohte.  Error-backpropagation in networks of fractionally predictive spiking neurons.  Springer Berlin Heidelberg, 2011.
  * [3] Tong Bu, Jianhao Ding, Zhaofei Yu, and Tiejun Huang.  Optimized potential initialization for low-latency spiking neural networks.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11–20, 2022.
  * [4] Tong Bu, Wei Fang, Jianhao Ding, PENGLIN DAI, Zhaofei Yu, and Tiejun Huang.  Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks.  In International Conference on Learning Representations, 2022.
  * [5] Kaiwei Che, Luziwei Leng, Kaixuan Zhang, Jianguo Zhang, Qinghu Meng, Jie Cheng, Qinghai Guo, and Jianxing Liao.  Differentiable hierarchical and surrogate gradient search for spiking neural networks.  In Advances in Neural Information Processing Systems.
  * [6] Yi Chen, Silin Zhang, Shiyu Ren, and et al.  Gradual surrogate gradient learning in deep spiking neural networks.  In ICASSP, pages 8927–8931, 2022.
  * [7] X. Cheng, Y. Hao, J. Xu, and B. Xu.  Lisnn: Improving spiking neural networks with lateral interactions for robust object recognition.  In Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence IJCAI-PRICAI-20, 2020.
  * [8] Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le.  Autoaugment: Learning augmentation policies from data, 2019.
  * [9] M. Davies, N. Srinivasa, T. H. Lin, G. Chinya, P. Joshi, A. Lines, A. Wild, and H. Wang.  Loihi: A neuromorphic manycore processor with on-chip learning.  IEEE Micro, pages 82–99, 2018.
  * [10] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li.  Imagenet: a large-scale hierarchical image database.  pages 248–255, 06 2009.
  * [11] S. Deng and S. Gu.  Optimal conversion of conventional artificial neural networks to spiking neural networks.  2021\.
  * [12] Shikuang Deng, Yuhang Li, Shanghang Zhang, and Shi Gu.  Temporal efficient training of spiking neural network via gradient re-weighting.  In International Conference on Learning Representations, 2022.
  * [13] Terrance DeVries and Graham W. Taylor.  Improved regularization of convolutional neural networks with cutout, 2017\.
  * [14] P. U. Diehl, D. Neil, J. Binas, M. Cook, and S. C. Liu.  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing.  In International Joint Conference on Neural Networks, 2015.
  * [15] Jason K Eshraghian, Corey Lammie, Mostafa Rahimi Azghadi, and Wei D Lu.  Navigating local minima in quantized spiking neural networks.  In 2022 IEEE 4th International Conference on Artificial Intelligence Circuits and Systems (AICAS), pages 352–355. IEEE, 2022.
  * [16] Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, and Yonghong Tian.  Deep residual learning in spiking neural networks.  2021\.
  * [17] W. Fang, Z. Yu, Y. Chen, T. Masquelier, and Y. Tian.  Incorporating learnable membrane time constant to enhance learning of spiking neural networks.  2020\.
  * [18] Ruihao Gong, Xianglong Liu, Shenghu Jiang, Tianxiang Li, Peng Hu, Jiazhen Lin, Fengwei Yu, and Junjie Yan.  Differentiable soft quantization: Bridging full-precision and low-bit neural networks.  In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4852–4861, 2019.
  * [19] Yufei Guo and Yuanpei Chen.  Neuroclip: Neuromorphic data understanding by clip and snn.  arXiv preprint arXiv:2306.12073, 2023.
  * [20] Yufei Guo, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Yinglei Wang, Xuhui Huang, and Zhe Ma.  Im-loss: Information maximization loss for spiking neural networks.  In Advances in Neural Information Processing Systems.
  * [21] Yufei Guo, Yuanpei Chen, Liwen Zhang, YingLei Wang, Xiaode Liu, Xinyi Tong, Yuanyuan Ou, Xuhui Huang, and Zhe Ma.  Reducing information loss for spiking neural networks.  In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XI, pages 36–52. Springer, 2022.
  * [22] Yufei Guo, Xuhui Huang, and Zhe Ma.  Direct learning-based deep spiking neural networks: a review.  Frontiers in Neuroscience, 17:1209795, 2023.
  * [23] Yufei Guo, Weihang Peng, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Xuhui Huang, and Zhe Ma.  Joint a-snn: Joint training of artificial and spiking neural networks via self-distillation and weight factorization.  Pattern Recognition, page 109639, 2023.
  * [24] Yufei Guo, Xinyi Tong, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Zhe Ma, and Xuhui Huang.  Recdis-snn: Rectifying membrane potential distribution for directly training spiking neural networks.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 326–335, June 2022.
  * [25] Yufei Guo, Liwen Zhang, Yuanpei Chen, Xinyi Tong, Xiaode Liu, YingLei Wang, Xuhui Huang, and Zhe Ma.  Real spike: Learning real-valued spikes for spiking neural networks.  In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XII, pages 52–68. Springer, 2022.
  * [26] B. Han and K. Roy.  Deep Spiking Neural Network: Energy Efficiency Through Time Based Coding.  Computer Vision – ECCV 2020, 2020.
  * [27] B. Han, G. Srinivasan, and K Roy.  Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network.  IEEE, 2020.
  * [28] Zecheng Hao, Tong Bu, Jianhao Ding, Tiejun Huang, and Zhaofei Yu.  Reducing ann-snn conversion error through residual membrane potential, 2023.
  * [29] Zecheng Hao, Jianhao Ding, Tong Bu, Tiejun Huang, and Zhaofei Yu.  Bridging the gap between anns and snns by calibrating offset spikes, 2023\.
  * [30] Sergey Ioffe and Christian Szegedy.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift.  arXiv e-prints, page arXiv:1502.03167, Feb. 2015.
  * [31] M. M. Khan, D. R. Lester, L. A. Plana, A. D. Rast, and S. B. Furber.  Spinnaker: Mapping neural networks onto a massively-parallel chip multiprocessor.  In IEEE International Joint Conference on Neural Networks, 2008\.
  * [32] S. Kim, S. Park, B. Na, and S. Yoon.  Spiking-yolo: Spiking neural network for energy-efficient object detection.  2019\.
  * [33] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton.  Cifar-10 (canadian institute for advanced research).
  * [34] A. Kugele, T. Pfeil, M. Pfeiffer, and E. Chicca.  Efficient processing of spatio-temporal data streams with spiking neural networks.  Frontiers in Neuroscience, 14:439, 2020.
  * [35] Souvik Kundu, Gourav Datta, Massoud Pedram, and Peter A. Beerel.  Spike-thrift: Towards energy-efficient deep spiking neural networks by limiting spiking activity via attention-guided compression.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 3953–3962, January 2021.
  * [36] C. Lee, S. S. Sarwar, Priyadarshini Panda, Gopalakrishnan Srinivasan, and K. Roy.  Enabling spike-based backpropagation for training deep neural network architectures.  2019\.
  * [37] J. H. Lee, T. Delbruck, and M. Pfeiffer.  Training deep spiking neural networks using backpropagation.  Frontiers in Neuroscience, 10, 2016.
  * [38] H. Li, H. Liu, X. Ji, G. Li, and L. Shi.  Cifar10-dvs: An event-stream dataset for object classification.  Frontiers in Neuroscience, 11, 2017.
  * [39] Yuhang Li, Shikuang Deng, Xin Dong, Ruihao Gong, and Shi Gu.  A free lunch from ann: Towards efficient, accurate spiking neural networks calibration.  arXiv preprint arXiv:2106.06984, 2021.
  * [40] Yuhang Li, Ruihao Gong, Xu Tan, Yang Yang, Peng Hu, Qi Zhang, Fengwei Yu, Wei Wang, and Shi Gu.  Brecq: Pushing the limit of post-training quantization by block reconstruction.  arXiv preprint arXiv:2102.05426, 2021.
  * [41] Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu.  Differentiable spike: Rethinking gradient-descent for training spiking neural networks.  Advances in Neural Information Processing Systems, 34, 2021.
  * [42] Hanxiao Liu, Karen Simonyan, and Yiming Yang.  Darts: Differentiable architecture search.  arXiv preprint arXiv:1806.09055, 2018.
  * [43] I. Loshchilov and F. Hutter.  Sgdr: Stochastic gradient descent with warm restarts.  2016\.
  * [44] Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo.  Training high-performance low-latency spiking neural networks by differentiation on spike representation, 2023.
  * [45] E. Neftci, C. Augustine, S. Paul, and G. Detorakis.  Event-driven random backpropagation: Enabling neuromorphic deep learning machines.  In 2017 IEEE International Symposium on Circuits and Systems (ISCAS), 2017.
  * [46] C. Nwankpa, W. Ijomah, A. Gachagan, and S. Marshall.  Activation functions: Comparison of trends in practice and research for deep learning.  2018\.
  * [47] Alexander Ororbia.  Spiking neural predictive coding for continual learning from data streams.  arXiv preprint arXiv:1908.08655, 2019.
  * [48] J. Pei, L. Deng, S. Song, M. Zhao, and L. Shi.  Towards artificial general intelligence with hybrid tianjic chip architecture.  Nature, 572(7767):106, 2019.
  * [49] A. Polino, R. Pascanu, and D. Alistarh.  Model compression via distillation and quantization.  2018\.
  * [50] N. Rathi and K. Roy.  Diet-snn: Direct input encoding with leakage and threshold optimization in deep spiking neural networks.  2020\.
  * [51] N. Rathi, G. Srinivasan, P. Panda, and K. Roy.  Enabling deep spiking neural networks with hybrid conversion and spike timing dependent backpropagation.  2020\.
  * [52] A. Sengupta, Y. Ye, R. Wang, C. Liu, and K. Roy.  Going deeper in spiking neural networks: Vgg and residual architectures.  Frontiers in Neuroscience, 13, 2019.
  * [53] Jiangrong Shen, Qi Xu, Jian K Liu, Yueming Wang, Gang Pan, and Huajin Tang.  Esl-snns: An evolutionary structure learning strategy for spiking neural networks.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 86–93, 2023.
  * [54] Sumit Bam Shrestha and Garrick Orchard.  Slayer: Spike layer error reassignment in time.  2018\.
  * [55] Y. Wu, L. Deng, G. Li, J. Zhu, and L. Shi.  Spatio-temporal backpropagation for training high-performance spiking neural networks.  2017\.
  * [56] Y Wu, L. Deng, G. Li, J. Zhu, and L. Shi.  Direct training for spiking neural networks: Faster, larger, better.  2018\.
  * [57] Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang.  LIAF-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing.  IEEE Transactions on Neural Networks and Learning Systems, pages 1–14, 2021.
  * [58] Qi Xu, Yaxin Li, Xuanye Fang, Jiangrong Shen, Jian K Liu, Huajin Tang, and Gang Pan.  Biologically inspired structure learning with reverse knowledge distillation for spiking neural networks.  arXiv preprint arXiv:2304.09500, 2023.
  * [59] Qi Xu, Yaxin Li, Jiangrong Shen, Jian K Liu, Huajin Tang, and Gang Pan.  Constructing deep spiking neural networks from artificial neural networks with knowledge distillation.  arXiv preprint arXiv:2304.05627, 2023.
  * [60] Qi Xu, Yaxin Li, Jiangrong Shen, Pingping Zhang, Jian K Liu, Huajin Tang, and Gang Pan.  Hierarchical spiking-based model for efficient image classification with enhanced feature extraction and encoding.  IEEE Transactions on Neural Networks and Learning Systems, 2022\.
  * [61] Qi Xu, Jiangrong Shen, Xuming Ran, Huajin Tang, Gang Pan, and Jian K Liu.  Robust transcoding sensory information with neural spikes.  IEEE Transactions on Neural Networks and Learning Systems, 33(5):1935–1946, 2021.
  * [62] B. Yin, F. Corradi, and SM Bohté.  Effective and efficient computation with multiple-timescale spiking recurrent neural networks.  2020\.
  * [63] F. Zenke and S. Ganguli.  Superspike: Supervised learning in multi-layer spiking neural networks.  Neural Computation, 30(6):1514–1541, 2017.
  * [64] W. Zhang and P. Li.  Temporal spike sequence learning via backpropagation for deep spiking neural networks.  2020\.
  * [65] X. Zhang, Y. He, and S. Jian.  Channel pruning for accelerating very deep neural networks.  In IEEE International Conference on Computer Vision, 2017.
  * [66] H. Zheng, Y. Wu, L. Deng, Y. Hu, and G. Li.  Going deeper with directly-trained larger spiking neural networks.  2020\.
  * [67] Zhaokun Zhou, Yuesheng Zhu, Chao He, Yaowei Wang, Shuicheng YAN, Yonghong Tian, and Li Yuan.  Spikformer: When spiking neural network meets transformer.  In The Eleventh International Conference on Learning Representations, 2023.
  * [68] R. Zimmer, T. Pellegrini, S. F. Singh, and T. Masquelier.  Technical report: supervised training of convolutional spiking neural networks with pytorch.  2019\.
  * [69] Barret Zoph and Quoc V. Le.  Neural architecture search with reinforcement learning, 2016.