# No circular birefringence in a chiral medium: an analysis of single-mode
refraction

Zhi-Juan Hu<EMAIL_ADDRESS>Department of Physics, Shanghai Normal
University, 100 Guilin Road, 200233 Shanghai, China  Chun-Fang Li
<EMAIL_ADDRESS>Department of Physics, Shanghai University, 99 Shangda Road,
200444 Shanghai, China

###### Abstract

It is generally believed that the right-handed circularly polarized (RCP) and
left-handed circularly polarized (LCP) waves in an isotropic chiral medium
propagate at different velocities, known as circular birefringence. Here we
show that this is not the case. After obtaining the refraction and reflection
coefficients of any elliptically polarized wave at the surface of a chiral
medium, we derive the conditions for single-mode refraction. By means of the
process of single-mode refraction, we demonstrate that both the refracted RCP
and the refracted LCP waves at normal incidence can be expressed as a coherent
superposition of a pair of orthogonal linearly polarized waves that are
rotated simultaneously. As a result, they must propagate at the same velocity
as the linearly polarized waves. A physical interpretation is also given in
detail. In particular, we show that the state of polarization of any
elliptically polarized wave in a chiral medium is rotated with propagation.
Such a rotation amounts to the rotation of polarization bases without
involving the change of the Jones vector. The rotation of the RCP and LCP
waves, as special cases of elliptically polarized waves, results in two
opposite phases as if they propagated at different phase velocities with their
polarization states transmitted unchanged.

##  I Introduction

A chiral medium is an optically active material [1]. It has the ability to
rotate the plane of polarization of a linearly polarized light wave. Such an
ability is associated with the absence of mirror symmetry of its molecular or
crystalline configuration [2]. A helically-coiled optical fiber also behaves
as a chiral medium [3, 4, 5, 6]. Chiral objects appear in either right- or
left-handed form. They provide a coupling of the magnetic field to the
electric polarizability and of the electric field to the magnetic
polarizability, which is described macroscopically by the constitutive
relations. A well-accepted set of constitutive relations for an isotropic and
transparent chiral medium can be written as follows [7, 8, 9],

$\displaystyle\mathbf{D}$
$\displaystyle=\varepsilon\mathbf{E}-g\partial{\mathbf{H}}/\partial{t},$ (1a)
$\displaystyle\mathbf{B}$
$\displaystyle=\mu\mathbf{H}+g\partial{\mathbf{E}}/\partial{t},$ (1b)

where $\mathbf{E}$, $\mathbf{H}$, $\mathbf{D}$, and $\mathbf{B}$ are, as
usual, the vectors of electric field, magnetic field, electric displacement,
and magnetic induction, respectively, $\varepsilon$ is the permittivity, $\mu$
is the permeability, and the pseudo-scalar constant $g$ is the gyrotropic
parameter. As early as in 1825, Fresnel [1] proposed a simple phenomenological
description of optical activity. Since a linearly polarized wave can be
expressed as a coherent superposition of right-handed circularly polarized
(RCP) and left-handed circularly polarized (LCP) waves, he suggested that
these two circularly polarized waves propagate at different velocities. A
chiral medium shows allogyric birefringence [10] or circular birefringence;
that is to say, it possesses two indices of refraction, one for RCP wave and
the other for LCP wave. Nowadays, circular birefringence has seemed to be
widely accepted as true [11, 9, 8, 12, 7, 13, 14]. In particular, the
difference between these two indices of refraction is even defined [2, 15, 16,
17, 18, 19, 20] plainly as the circular birefringence. We will show here that
this is not the case.

It is indeed true, as demonstrated recently by Ghosh and Fischer [16], that
circular double refraction [21] occurs at the planar surface of a chiral
liquid though it is isotropic. The problem, however, is that at oblique
incidence, the two refracted circularly polarized modes do not propagate in
the same direction. They cannot combine to yield a linearly polarized wave no
matter what the incidence polarization is. In order for the refracted wave in
a chiral medium to be linearly polarized, one needs to make use of normal
incidence [8, 12, 16]. The incident wave in this case should also be linearly
polarized. Furthermore, single-mode refraction is permissible at the surface
of a chiral medium. That is to say, one of the refracted circularly polarized
modes vanishes under suitable conditions. As a matter of fact, Jaggard and Sun
[22] predicted that at any incidence angle there exist two states of incidence
polarization for single-mode refraction, one for each handedness of circular
polarizations. By this it is meant that given the incidence angle, changing
only the incidence polarization from one state to another will convert the
refracted wave from RCP to LCP or the other way around. So if the two
circularly polarized waves in a chiral medium propagate at different
velocities, one will be able to alter the velocity of the refracted wave by
changing only the incident wave from one particular polarization state to
another. This is rather puzzling.

According to Ref. [22], the conditions for single-mode refraction require the
incident wave at normal incidence to be circularly polarized. It is known that
an incident circularly polarized wave in the achiral region can be represented
as a coherent superposition of two orthogonal linearly polarized waves. It is
also known that an incident linearly polarized wave normal to the surface of a
chiral medium gives rise to a refracted linearly polarized wave. So at normal
incidence, either of the refracted circularly polarized waves under the
conditions for single-mode refraction can be represented as a coherent
superposition of two orthogonal linearly polarized waves. This means that the
velocities of the two circularly polarized waves in a chiral medium are the
same as that of their linearly polarized components and thus cannot be
different. That is to say, there is not circular birefringence at all in a
chiral medium. The key point here is that the polarization states of
circularly polarized waves in a chiral medium are rotated in the same way as
those of linearly polarized or any other elliptically polarized waves. After a
same propagation distance, the rotation of the polarization states of the RCP
and LCP waves yields two opposite phases as if they propagated at different
phase velocities with their polarization states transmitted unchanged. The
purpose of the present article is to report these new findings. The contents
are arranged as follows.

In Section II, we derive the refraction and reflection coefficients of any
elliptically polarized wave at the surface of an isotropic chiral medium. To
do this, we expand the polarization vector of the incident wave in terms of
circular-polarization bases. The combination of the expansion coefficients
plays the role of specifying the incidence polarization. As expected, the
refraction and reflection coefficients depend not only on the incidence angle
but also on the incidence polarization. Conditions for single-mode refraction
are deduced in Section III from the refraction coefficients. Conditions under
which the reflected wave is circularly polarized are also given. The main
result of this paper is put forward in Section IV. It is demonstrated that
whether the RCP or the LCP wave in a chiral medium can be expressed as a
coherent superposition of two orthogonal linearly polarized waves. They should
propagate at the velocity at which the linearly polarized waves propagate. In
other words, there cannot be circular birefringence in a chiral medium. A
physical interpretation is further given in Section V. Firstly, the common
phase velocity of the linearly polarized and the circularly polarized waves is
discussed. Secondly, it is shown that the state of polarization of any
elliptically polarized wave in a chiral medium is rotated. This is expressed
by the rotatory polarization vector. Such a rotation amounts to the rotation
of polarization bases. The result of the rotation of the RCP and LCP waves, as
special cases of elliptically polarized waves, appears mathematically as two
opposite phases as if they propagated at different phase velocities with their
polarization states transmitted unchanged. It is these two opposite phases
that account for the circular double refraction at the surface of a chiral
medium. Section VI summarizes the paper with remarks.

##  II Refraction and reflection coefficients of any elliptically polarized
waves

Jaggard and Sun [22] once gave the conditions for single-mode refraction at
the interface between two different chiral media where the incident wave is a
coherent superposition of RCP and LCP components that propagate in different
directions. For our purpose, we will re-derive these conditions at the
interface between the air of permittivity $\varepsilon_{0}$ and permeability
$\mu_{0}$ and an isotropic chiral medium such as the chiral liquid considered
in Ref. [16]. A well-known fact is that single-mode refraction is permissible
at the surface of a dielectric medium, which is achiral. The refracted
characteristic wave in that case is linearly polarized. In particular, the
incident and refracted waves under the conditions for single-mode refraction
have the same polarization. They are either $s$-polarized or $p$-polarized.
Probably influenced by that fact, some authors [16] thought that at the
surface of a chiral medium, the incident and refracted waves under the
conditions for single-mode refraction also have the same polarization. They
are either RCP or LCP. However this is incorrect. To avoid such a
misunderstanding, it is better to have the refraction and reflection
coefficients of any elliptically polarized wave.

###  II.1 Expressions for incident and reflected waves

Figure 1:  Illustration for the circular double refraction at the surface of
an isotropic chiral medium.

As is shown in Fig. 1, a plane wave is incident from the air upon the surface
of the chiral medium at the incidence angle $\theta_{i}$. Considering that the
refracted characteristic modes are circularly polarized, we expand the
incident wave in terms of RCP and LCP components rather than in terms of $s$\-
and $p$-polarized components as is usually done in the literature [8, 7, 9,
11]. Letting $\bar{x}$, $\bar{y}$, and $\bar{z}$ be the unit vectors along the
corresponding Cartesian axes, the electric field of the incident wave can be
expressed as follows,

$\mathbf{E}_{i}(\mathbf{x})=\bigg{[}\frac{c_{+}}{\sqrt{2}}(\mathbf{s}_{i}+i\mathbf{p}_{i})+\frac{c_{-}}{\sqrt{2}}(\mathbf{s}_{i}-i\mathbf{p}_{i})\bigg{]}E_{i}\exp(i\mathbf{k}_{i}\cdot\mathbf{x}),$
(2)

where the time-dependence $\exp(-i\omega t)$ is assumed,
$\mathbf{k}_{i}=k_{0}(\bar{y}\sin\theta_{i}+\bar{z}\cos\theta_{i})$ is the
wavevector, $k_{0}=(\varepsilon_{0}\mu_{0})^{1/2}\omega$,
$\mathbf{s}_{i}=\bar{x}$,
$\mathbf{p}_{i}=\frac{\mathbf{k}_{i}}{k_{0}}\times\mathbf{s}_{i}=\bar{y}\cos\theta_{i}-\bar{z}\sin\theta_{i}$,
the expansion coefficients $c_{+}$ and $c_{-}$ satisfy

$|c_{+}|^{2}+|c_{-}|^{2}=1,$ (3)

and $E_{i}$ is the amplitude. The unit vectors
$\frac{1}{\sqrt{2}}(\mathbf{s}_{i}+i\mathbf{p}_{i})$ and
$\frac{1}{\sqrt{2}}(\mathbf{s}_{i}-i\mathbf{p}_{i})$ in expression (2) are the
circular-polarization bases. They stand for the RCP and LCP states,
respectively. So the combination of $c_{+}$ and $c_{-}$ plays the role of
specifying the incidence polarization, giving the polarization ellipticity
$\sigma=|c_{+}|^{2}-|c_{-}|^{2}$. When $c_{-}=0$, it is RCP. When $c_{+}=0$,
it is LCP. According to Maxwell’s equation
$\nabla\times\mathbf{E}=-\partial\mathbf{B}/\partial t$ and the constitutive
relation $\mathbf{B}=\mu_{0}\mathbf{H}$, the magnetic field of the incident
wave takes the form

$\mathbf{H}_{i}(\mathbf{x})=\frac{1}{i\eta_{0}}\bigg{[}\frac{c_{+}}{\sqrt{2}}(\mathbf{s}_{i}+i\mathbf{p}_{i})-\frac{c_{-}}{\sqrt{2}}(\mathbf{s}_{i}-i\mathbf{p}_{i})\bigg{]}E_{i}\exp(i\mathbf{k}_{i}\cdot\mathbf{x}),$

where $\eta_{0}=\sqrt{\mu_{0}/\varepsilon_{0}}$. Similarly, letting be
$\theta_{r}$ the reflection angle, the electric field of the reflected wave
can be expressed as

$\mathbf{E}_{r}(\mathbf{x})=\bigg{[}\frac{E_{+,r}}{\sqrt{2}}(\mathbf{s}_{r}+i\mathbf{p}_{r})+\frac{E_{-,r}}{\sqrt{2}}(\mathbf{s}_{r}-i\mathbf{p}_{r})\bigg{]}\exp(i\mathbf{k}_{r}\cdot\mathbf{x}),$
(4)

where $\mathbf{k}_{r}=k_{0}(\bar{y}\sin\theta_{r}-\bar{z}\cos\theta_{r})$,
$\mathbf{s}_{r}=\bar{x}$,
$\mathbf{p}_{r}=\frac{\mathbf{k}_{r}}{k_{0}}\times\mathbf{s}_{r}=-\bar{y}\cos\theta_{r}-\bar{z}\sin\theta_{r}$,
and $E_{+,r}$ and $E_{-,r}$ are the amplitudes of the RCP and LCP components,
respectively. Here the unit vectors
$\frac{1}{\sqrt{2}}(\mathbf{s}_{r}+i\mathbf{p}_{r})$ and
$\frac{1}{\sqrt{2}}(\mathbf{s}_{r}-i\mathbf{p}_{r})$ stand for the RCP and LCP
states in the reflected wave, respectively. Accordingly, the magnetic field of
the reflected wave assumes

$\mathbf{H}_{r}(\mathbf{x})=\frac{1}{i\eta_{0}}\bigg{[}\frac{E_{+,r}}{\sqrt{2}}(\mathbf{s}_{r}+i\mathbf{p}_{r})-\frac{E_{-,r}}{\sqrt{2}}(\mathbf{s}_{r}-i\mathbf{p}_{r})\bigg{]}\exp(i\mathbf{k}_{r}\cdot\mathbf{x}).$

It is pointed out that expression (4) for the reflected wave is not parallel
to expression (2) for the incident wave. This is because the incident wave is
supposed to be known so that both the amplitude $E_{i}$ and the expansion
coefficients $c_{+}$ and $c_{-}$ are given; whereas the reflected wave needs
to be determined by use of the boundary conditions. We will see below that
what we obtain directly from the boundary conditions are $E_{+,r}$ and
$E_{-,r}$. If defined as $\frac{E_{+,r}}{E_{i}}$ and $\frac{E_{-,r}}{E_{i}}$,
the reflection coefficients will depend on the incidence polarization
specified by $c_{+}$ and $c_{-}$.

###  II.2 Expressions for refracted wave in the chiral medium

As was experimentally observed [16], the refracted wave consists of RCP and
LCP components that propagate in different directions. According to
constitutive relations (1), the “wave numbers” of the RCP and LCP components
are given by [9, 7]

$\begin{split}k_{+,t}&=k-\tau,\\\ k_{-,t}&=k+\tau,\end{split}$ (5)

respectively, where $k=(\varepsilon\mu)^{1/2}\omega$ and $\tau=-g\omega^{2}$.
Letting be $\theta_{+,t}$ the refraction angle of the RCP component, its
electric field can be expressed as

$\mathbf{E}_{+,t}(\mathbf{x})=\frac{E_{+,t}}{\sqrt{2}}(\mathbf{s}_{+,t}+i\mathbf{p}_{+,t})\exp(i\mathbf{k}_{+,t}\cdot\mathbf{x}),$

where $E_{+,t}$ is the amplitude,
$\mathbf{k}_{+,t}=k_{+,t}(\bar{y}\sin\theta_{+,t}+\bar{z}\cos\theta_{+,t})$ is
the wavevector, $\mathbf{s}_{+,t}=\bar{x}$, and
$\mathbf{p}_{+,t}=\frac{\mathbf{k}_{+,t}}{k_{+,t}}\times\mathbf{s}_{+,t}=\bar{y}\cos\theta_{+,t}-\bar{z}\sin\theta_{+,t}$.
The magnetic field of the RCP component is found, upon making use of Maxwell’s
equation $\nabla\times\mathbf{E}=-\partial\mathbf{B}/\partial t$ and
constitutive relation (1b), to be
$\mathbf{H}_{+,t}(\mathbf{x})=-\frac{ik}{\mu\omega}\mathbf{E}_{+,t}(\mathbf{x})$
and is given by

$\mathbf{H}_{+,t}(\mathbf{x})=\frac{E_{+,t}}{i\sqrt{2}\eta}(\mathbf{s}_{+,t}+i\mathbf{p}_{+,t})\exp(i\mathbf{k}_{+,t}\cdot\mathbf{x}),$

where $\eta=\sqrt{\mu/\varepsilon}$. Similarly, letting be $\theta_{-,t}$ the
refraction angle of the LCP component, its electric field can be expressed as

$\mathbf{E}_{-,t}(\mathbf{x})=\frac{E_{-,t}}{\sqrt{2}}(\mathbf{s}_{-,t}-i\mathbf{p}_{-,t})\exp(i\mathbf{k}_{-,t}\cdot\mathbf{x}),$

where $E_{-,t}$ is the amplitude,
$\mathbf{k}_{-,t}=k_{-,t}(\bar{y}\sin\theta_{-,t}+\bar{z}\cos\theta_{-,t})$,
$\mathbf{s}_{-,t}=\bar{x}$, and
$\mathbf{p}_{-,t}=\frac{\mathbf{k}_{-,t}}{k_{-,t}}\times\mathbf{s}_{-,t}=\bar{y}\cos\theta_{-,t}-\bar{z}\sin\theta_{-,t}$.
Correspondingly, the magnetic field of the LCP component is found to be
$\mathbf{H}_{-,t}(\mathbf{x})=\frac{ik}{\mu\omega}\mathbf{E}_{-,t}(\mathbf{x})$
and is given by

$\mathbf{H}_{-,t}(\mathbf{x})=\frac{iE_{-,t}}{\sqrt{2}\eta}(\mathbf{s}_{-,t}-i\mathbf{p}_{-,t})\exp(i\mathbf{k}_{-,t}\cdot\mathbf{x}).$

Consequently, the total electric field of the refracted wave,
$\mathbf{E}_{t}=\mathbf{E}_{+,t}+\mathbf{E}_{-,t}$, reads

$\mathbf{E}_{t}=\frac{E_{+,t}}{\sqrt{2}}(\mathbf{s}_{+,t}+i\mathbf{p}_{+,t})\exp(i\mathbf{k}_{+,t}\cdot\mathbf{x})+\frac{E_{-,t}}{\sqrt{2}}(\mathbf{s}_{-,t}-i\mathbf{p}_{-,t})\exp(i\mathbf{k}_{-,t}\cdot\mathbf{x}),$
(6)

and the total magnetic field,
$\mathbf{H}_{t}=\mathbf{H}_{+,t}+\mathbf{H}_{-,t}$, reads

$\mathbf{H}_{t}=\frac{1}{i\eta}\bigg{[}\frac{E_{+,t}}{\sqrt{2}}(\mathbf{s}_{+,t}+i\mathbf{p}_{+,t})\exp(i\mathbf{k}_{+,t}\cdot\mathbf{x})-\frac{E_{-,t}}{\sqrt{2}}(\mathbf{s}_{-,t}-i\mathbf{p}_{-,t})\exp(i\mathbf{k}_{-,t}\cdot\mathbf{x})\bigg{]}.$

###  II.3 Refraction and reflection coefficients

Having written out the electric and magnetic fields of the incident,
reflected, and refracted waves, we are in a position to make use of the
boundary conditions to determine the reflection angle $\theta_{r}$ as well as
the refraction angles $\theta_{+,t}$ and $\theta_{-,t}$ and to find the
reflection and refraction coefficients. The phase-matching condition at the
interface between the air and the chiral medium leads to Snell’s law

$\sin\theta_{r}=\sin\theta_{i}$ (7)

for the reflection angle $\theta_{r}$ as well as Snell’s laws

$k_{+,t}\sin\theta_{+,t}=k_{-,t}\sin\theta_{-,t}=k_{0}\sin\theta_{i}$ (8)

for the refraction angles $\theta_{+,t}$ and $\theta_{-,t}$. After these
equations are taken into account, the continuity of the $x$-component of the
electric field at the interface yields

$(c_{+}+c_{-})E_{i}+E_{+,r}+E_{-,r}=E_{+,t}+E_{-,t}$ (9)

and the continuity of the $y$-component of the electric field at the interface
yields

$(c_{+}-c_{-})E_{i}-E_{+,r}+E_{-,r}=\frac{\cos\theta_{+,t}}{\cos\theta_{i}}E_{+,t}-\frac{\cos\theta_{-,t}}{\cos\theta_{i}}E_{-,t}.$
(10)

Similarly, after Eqs. (7) and (8) are taken into account, the continuity of
the $x$-component of the magnetic field at the interface means

$\frac{1}{\eta_{0}}[(c_{+}-c_{-})E_{i}+E_{+,r}-E_{-,r}]=\frac{1}{\eta}(E_{+,t}-E_{-,t})$
(11)

and the continuity of the $y$-component of the magnetic field at the interface
means

$\frac{1}{\eta_{0}}[(c_{+}+c_{-})E_{i}-E_{+,r}-E_{-,r}]=\frac{1}{\eta}\Big{(}\frac{\cos\theta_{+,t}}{\cos\theta_{i}}E_{+,t}+\frac{\cos\theta_{-,t}}{\cos\theta_{i}}E_{-,t}\Big{)}.$
(12)

From Eqs. (9)-(12) one readily finds the refraction coefficients for the two
characteristic modes, which are defined as $\frac{E_{+,t}}{E_{i}}\equiv T_{+}$
and $\frac{E_{-,t}}{E_{i}}\equiv T_{-}$ and are given by

$\displaystyle T_{+}$
$\displaystyle=2\cos\theta_{i}\frac{(n+1)(\cos\theta_{-,t}+\cos\theta_{i})c_{+}-(n-1)(\cos\theta_{-,t}-\cos\theta_{i})c_{-}}{2n(\cos^{2}\theta_{i}+\cos\theta_{+,t}\cos\theta_{-,t})+(n^{2}+1)(\cos\theta_{+,t}+\cos\theta_{-,t})\cos\theta_{i}},$
(13a) $\displaystyle T_{-}$
$\displaystyle=2\cos\theta_{i}\frac{(n+1)(\cos\theta_{+,t}+\cos\theta_{i})c_{-}-(n-1)(\cos\theta_{+,t}-\cos\theta_{i})c_{+}}{2n(\cos^{2}\theta_{i}+\cos\theta_{+,t}\cos\theta_{-,t})+(n^{2}+1)(\cos\theta_{+,t}+\cos\theta_{-,t})\cos\theta_{i}},$
(13b)

where $n=\eta_{0}/\eta$. So defined refraction coefficients depend not only on
the incidence angle $\theta_{i}$ but also on the incidence polarization
specified by $c_{+}$ and $c_{-}$. It is worth noting that such a definition
gets rid of the so-called cross-polarized refraction coefficients [22]. From
Eqs. (9)-(12) one can also find the reflection coefficients for the two
circularly polarized components, which are defined as
$\frac{E_{+,r}}{E_{i}}\equiv R_{+}$ and $\frac{E_{-,r}}{E_{i}}\equiv R_{-}$
and are given by

$\displaystyle R_{+}$
$\displaystyle=\frac{2n(\cos\theta_{i}+\cos\theta_{-,t})(\cos\theta_{i}-\cos\theta_{+,t})c_{+}-(n^{2}-1)\cos\theta_{i}(\cos\theta_{+,t}+\cos\theta_{-,t})c_{-}}{2n(\cos^{2}\theta_{i}+\cos\theta_{+,t}\cos\theta_{-,t})+(n^{2}+1)(\cos\theta_{+,t}+\cos\theta_{-,t})\cos\theta_{i}},$
(14a) $\displaystyle R_{-}$
$\displaystyle=\frac{2n(\cos\theta_{i}+\cos\theta_{+,t})(\cos\theta_{i}-\cos\theta_{-,t})c_{-}-(n^{2}-1)\cos\theta_{i}(\cos\theta_{+,t}+\cos\theta_{-,t})c_{+}}{2n(\cos^{2}\theta_{i}+\cos\theta_{+,t}\cos\theta_{-,t})+(n^{2}+1)(\cos\theta_{+,t}+\cos\theta_{-,t})\cos\theta_{i}}.$
(14b)

Akin to the refraction coefficients, so defined reflection coefficients depend
not only on the incidence angle but also on the incidence polarization.

##  III Conditions for single-mode refraction

In general, double refraction occurs at the surface of the chiral medium. The
refracted wave consists of RCP and LCP components that propagate in different
directions. But when the incidence angle and the state of incidence
polarization satisfy certain conditions, only one mode, either RCP or LCP,
exists in the refracted wave. In fact, expression (13b) tells that only the
RCP mode will be refracted if

$\frac{c_{-}}{c_{+}}=\frac{n-1}{n+1}\frac{\cos\theta_{+,t}-\cos\theta_{i}}{\cos\theta_{+,t}+\cos\theta_{i}}.$
(15)

On the other hand, expression (13a) tells that only the LCP mode will be
refracted if

$\frac{c_{+}}{c_{-}}=\frac{n-1}{n+1}\frac{\cos\theta_{-,t}-\cos\theta_{i}}{\cos\theta_{-,t}+\cos\theta_{i}}.$
(16)

Clearly, at any incidence angle there exist two states of incidence
polarization for single-mode refraction, one for each handedness of circular
polarizations. Since usually $n\neq 1$, condition (15) does not mean $c_{-}=0$
and condition (16) does not mean $c_{+}=0$ unless the incidence angle is equal
to zero. This shows that at oblique incidence the incidence polarization for
single-mode refraction is not circular. Frankly speaking, one has $n>1$ for
conventional chiral media. In this case, condition (15) means
$|c_{-}|<|c_{+}|$, the incident wave being right-handed elliptically
polarized. In contrast, condition (16) means $|c_{+}|<|c_{-}|$, the incident
wave being left-handed elliptically polarized.

Of course, the reflected wave in the air is in general elliptically polarized.
But Eqs. (14) show that there exist conditions under which the reflected wave
is circularly polarized. Eq. (14b) means that the reflected wave will be RCP
if

$\frac{c_{+}}{c_{-}}=\frac{2n}{n^{2}-1}\frac{(\cos\theta_{i}+\cos\theta_{+,t})(\cos\theta_{i}-\cos\theta_{-,t})}{(\cos\theta_{+,t}+\cos\theta_{-,t})\cos\theta_{i}}.$

Similarly, Eq. (14a) means that the reflected wave will be LCP if

$\frac{c_{-}}{c_{+}}=\frac{2n}{n^{2}-1}\frac{(\cos\theta_{i}-\cos\theta_{+,t})(\cos\theta_{i}+\cos\theta_{-,t})}{(\cos\theta_{+,t}+\cos\theta_{-,t})\cos\theta_{i}}.$

##  IV There is not circular birefringence

According to conditions (15) and (16) for single-mode refraction, at any
incidence angle there exist two states of incidence polarization for the
refracted wave to be circularly polarized. Importantly, the incidence
polarization at normal incidence is required to be circular, too. This
provides a possibility for us to logically analyze the propagation velocities
of the RCP and LCP waves in the chiral medium.

Since the incident circularly polarized wave can be expressed as a coherent
superposition of two orthogonal linearly polarized waves, the refracted
circularly polarized wave could also be expressed as a coherent superposition
of two orthogonal linearly polarized waves that are rotated in the same way.
To show this, let us first find out such two refracted linearly polarized
waves. When $\theta_{i}=0$, expression (2) for the electric field of the
incident wave reads

$\mathbf{E}_{i}(\mathbf{x})=\frac{1}{\sqrt{2}}[\bar{x}(c_{+}+c_{-})+i\bar{y}(c_{+}-c_{-})]E_{i}\exp(ik_{0}z).$
(17)

Since the angles of refraction at normal incidence are equal to zero,
$\theta_{+,t}=\theta_{-,t}=0$, the refraction coefficients (13) become

$\displaystyle T_{+}$ $\displaystyle=$ $\displaystyle Tc_{+},$ $\displaystyle
T_{-}$ $\displaystyle=$ $\displaystyle Tc_{-},$

where $T=\frac{2}{n+1}$. As a result, expression (6) for the electric field of
the refracted wave assumes

$\mathbf{E}_{t}(\mathbf{x})=\frac{1}{\sqrt{2}}[(\bar{x}\cos\tau
z+\bar{y}\sin\tau z)(c_{+}+c_{-})+i(\bar{y}\cos\tau z-\bar{x}\sin\tau
z)(c_{+}-c_{-})]TE_{i}\exp(ikz),$ (18)

by virtue of Eqs. (5). A comparison with expression (17) shows that for this
wave to be linearly polarized, the incident wave also needs to be linearly
polarized. In particular, when $c_{+}=c_{-}=\frac{\sqrt{2}}{2}$, expression
(17) reduces to

$\mathbf{E}_{1,i}=\bar{x}E_{i}\exp(ik_{0}z);$ (19)

and expression (18) reduces to

$\mathbf{E}_{1,t}=(\bar{x}\cos\tau z+\bar{y}\sin\tau z)TE_{i}\exp(ikz).$ (20)

In addition, when $c_{+}=-c_{-}=-i\frac{\sqrt{2}}{2}$, expression (17) reduces
to

$\mathbf{E}_{2,i}=\bar{y}E_{i}\exp(ik_{0}z);$ (21)

and expression (18) reduces to

$\mathbf{E}_{2,t}=(\bar{y}\cos\tau z-\bar{x}\sin\tau z)TE_{i}\exp(ikz).$ (22)

It is not difficult to see that the refracted waves (20) and (22), which are
rotated with propagation in the same way, are orthogonal to each other at the
same propagation distance $z$. This is understandable, because their
corresponding incident waves (19) and (21) are orthogonal to each other.

To see further how these two linearly polarized waves can be utilized to
express a circularly polarized wave in the chiral medium, we resort to the
familiar fact to expand them in terms of the RCP and LCP waves. When $c_{+}=1$
and $c_{-}=0$, the refracted wave (18) is RCP,

$\mathbf{E}_{+,t}=\frac{1}{\sqrt{2}}(\bar{x}+i\bar{y})TE_{i}\exp(ik_{+,t}z).$
(23)

And when $c_{+}=0$ and $c_{-}=1$, the refracted wave (18) is LCP,

$\mathbf{E}_{-,t}=\frac{1}{\sqrt{2}}(\bar{x}-i\bar{y})TE_{i}\exp(ik_{-,t}z).$
(24)

As expected [1], the linearly polarized wave $\mathbf{E}_{1,t}$ can be
expressed as a coherent superposition of the circularly polarized waves
$\mathbf{E}_{+,t}$ and $\mathbf{E}_{-,t}$ of the form,

$\mathbf{E}_{1,t}=\frac{1}{\sqrt{2}}(\mathbf{E}_{+,t}+\mathbf{E}_{-,t}).$ (25)

Similarly, the linearly polarized wave $\mathbf{E}_{2,t}$ can also be
expressed as a coherent superposition of $\mathbf{E}_{+,t}$ and
$\mathbf{E}_{-,t}$,

$\mathbf{E}_{2,t}=-\frac{i}{\sqrt{2}}(\mathbf{E}_{+,t}-\mathbf{E}_{-,t}).$
(26)

From Eqs. (25) and (26) it follows that the RCP wave $\mathbf{E}_{+,t}$ can be
expressed as a coherent superposition of the linearly polarized waves
$\mathbf{E}_{1,t}$ and $\mathbf{E}_{2,t}$ as follows,

$\mathbf{E}_{+,t}=\frac{1}{\sqrt{2}}(\mathbf{E}_{1,t}+i\mathbf{E}_{2,t}).$
(27)

In addition, the LCP wave $\mathbf{E}_{-,t}$ can also be expressed as a
coherent superposition of the same two linearly polarized waves of the form,

$\mathbf{E}_{-,t}=\frac{1}{\sqrt{2}}(\mathbf{E}_{1,t}-i\mathbf{E}_{2,t}).$
(28)

In a word, whether the RCP wave $\mathbf{E}_{+,t}$ or the LCP wave
$\mathbf{E}_{-,t}$ can be expressed as a coherent superposition of the
orthogonal linearly polarized waves $\mathbf{E}_{1,t}$ and $\mathbf{E}_{2,t}$.

We are now ready to give our main result that the two refracted characteristic
modes in the chiral medium do not propagate at different velocities. Rotated
in the same way, the linearly polarized waves $\mathbf{E}_{1,t}$ and
$\mathbf{E}_{2,t}$ in the chiral medium must propagate at the same velocity if
they have definite propagation velocities. As a result of Eqs. (27) and (28),
the two circularly polarized waves $\mathbf{E}_{+,t}$ and $\mathbf{E}_{-,t}$
should propagate at the velocity at which the linearly polarized waves
propagate. That is to say, they should propagate at the same velocity. No
circular birefringence exists in the chiral medium. What should be pointed out
is that even though expression (23) correctly describes the RCP wave in the
chiral medium, it does not mean that the RCP wave is a coherent superposition
of the following linearly polarized waves,

$\begin{split}\mathbf{E}^{+}_{x,t}&=\bar{x}TE_{i}\exp(ik_{+,t}z),\\\
\mathbf{E}^{+}_{y,t}&=\bar{y}TE_{i}\exp(ik_{+,t}z).\end{split}$ (29)

This is because there does not exist in the chiral medium such a linearly
polarized wave the polarization plane of which does not change. Similarly,
expression (24) does not mean that the LCP wave is a coherent superposition of
the following linearly polarized waves,

$\begin{split}\mathbf{E}^{-}_{x,t}&=\bar{x}TE_{i}\exp(ik_{-,t}z),\\\
\mathbf{E}^{-}_{y,t}&=\bar{y}TE_{i}\exp(ik_{-,t}z).\end{split}$ (30)

It is also noted that if the RCP and LCP waves $\mathbf{E}_{+,t}$ and
$\mathbf{E}_{-,t}$ were considered to propagate at different velocities, one
could not deduce from Eqs. (25) and (26) that the two linearly polarized waves
$\mathbf{E}_{1,t}$ and $\mathbf{E}_{2,t}$ have the same propagation velocity.
A further interpretation of this result is given below.

##  V Further discussions

Let us first look at the propagation velocity of circularly polarized waves in
the chiral medium. To this end, we rewrite linearly polarized waves (20) and
(22) explicitly as

$\displaystyle\mathbf{E}_{1,t}$
$\displaystyle=\bar{x}^{\prime}(z)TE_{i}\exp(ikz),$ (31a)
$\displaystyle\mathbf{E}_{2,t}$
$\displaystyle=\bar{y}^{\prime}(z)TE_{i}\exp(ikz),$ (31b)

where

$\begin{split}\bar{x}^{\prime}(z)&=\exp[-i(\mathbf{\Sigma}\cdot\bar{z})\tau
z]\bar{x},\\\ \bar{y}^{\prime}(z)&=\exp[-i(\mathbf{\Sigma}\cdot\bar{z})\tau
z]\bar{y},\end{split}$ (32)

are the polarization vectors depending on the propagation distance $z$ and
$(\Sigma_{k})_{ij}=-i\epsilon_{ijk}$ with $\epsilon_{ijk}$ the Levi-Civitá
pseudotensor. The rotation of the polarization planes of linearly polarized
waves (31) is properly represented by expressions (32), the rotatory
polarization vectors. That is to say, the rotation operator
$\exp[-i(\mathbf{\Sigma}\cdot\bar{z})\tau z]$ in (32) properly conveys the
role of the gyrotropic parameter $g$ in constitutive relations (1). This
indicates that only the common phase factor $\exp(ikz)$ in (31) is the
propagation factor. It determines the phase velocity to be $\frac{\omega}{k}$.
Substituting Eqs. (31) into Eqs. (27) and (28), we have

$\displaystyle\mathbf{E}_{+,t}$ $\displaystyle=\mathbf{r}(z)TE_{i}\exp(ikz)$
(33a) $\displaystyle\mathbf{E}_{-,t}$
$\displaystyle=\mathbf{l}(z)TE_{i}\exp(ikz),$ (33b)

where

$\begin{split}\mathbf{r}(z)&=\frac{1}{\sqrt{2}}\exp[-i(\mathbf{\Sigma}\cdot\bar{z})\tau
z](\bar{x}+i\bar{y}),\\\
\mathbf{l}(z)&=\frac{1}{\sqrt{2}}\exp[-i(\mathbf{\Sigma}\cdot\bar{z})\tau
z](\bar{x}-i\bar{y}).\end{split}$ (34)

A comparison of Eqs. (34) with (32) shows that the polarization vectors of
circularly polarized waves (33) are rotated in the same way as those of
linearly polarized waves (31). This means that the same as the states of
polarization of linearly polarized waves are rotated, the states of
polarization of circularly polarized waves are also rotated. Such a rotation
is also represented by their rotatory polarization vectors. That is, the
factors $\mathbf{r}(z)$ and $\mathbf{l}(z)$ express the rotation of the
polarization states of RCP and LCP waves, respectively. Only the common phase
factor $\exp(ikz)$ in (33) is the propagation factor. It determines the phase
velocity to be $\frac{\omega}{k}$, the same as the phase velocity of linearly
polarized waves (31). The quantity $k=(\varepsilon\mu)^{1/2}\omega$ for the
chiral medium is not “merely a shorthand notation” [23] at all. It is the wave
number.

Secondly, since $\bar{x}+i\bar{y}$ and $\bar{x}-i\bar{y}$ are eigen vectors of
operator $\mathbf{\Sigma}\cdot\bar{z}$ with eigen values $+1$ and $-1$,
respectively, expressions (34) can be rewritten as

$\begin{split}\mathbf{r}(z)&=\frac{1}{\sqrt{2}}(\bar{x}+i\bar{y})\exp(-i\tau
z),\\\ \mathbf{l}(z)&=\frac{1}{\sqrt{2}}(\bar{x}-i\bar{y})\exp(i\tau
z),\end{split}$ (35)

meaning that the rotation of the polarization states of RCP and LCP waves in
the chiral medium gives rise to phases of equal magnitude but opposite sign.
In view of this, the circular double refraction indicated by the difference
between $k_{+,t}$ and $k_{-,t}$ in (5) just reflects these two opposite
phases. It is unreasonable to interpret the RCP and LCP waves, expressed
respectively by (23) and (24), as propagating at different phase velocities.

Then how do we understand the rotation of circularly polarized waves in the
chiral medium? As discussed above, the linearly polarized waves in (31) are
orthogonal to each other at the same propagation distance. They can serve as
base modes to expand the refracted wave (18) in the following way,

$\mathbf{E}_{t}=\alpha_{1}\mathbf{E}_{1,t}+\alpha_{2}\mathbf{E}_{2,t}=\mathbf{a}_{t}(z)TE_{i}\exp(ikz),$
(36)

where

$\mathbf{a}_{t}(z)=\alpha_{1}\bar{x}^{\prime}(z)+\alpha_{2}\bar{y}^{\prime}(z)$
(37)

is the polarization vector, and $\alpha_{1}=\frac{1}{\sqrt{2}}(c_{+}+c_{-})$
and $\alpha_{2}=\frac{i}{\sqrt{2}}(c_{+}-c_{-})$ obey
$|\alpha_{1}|^{2}+|\alpha_{2}|^{2}=1$ by virtue of Eq. (3). As a matter of
fact, it is the unit vectors $\bar{x}^{\prime}$ and $\bar{y}^{\prime}$ that
are orthogonal to each other at the same propagation distance as can be seen
from Eqs. (32). They serve as polarization bases to expand
$\mathbf{a}_{t}(z)$. The expansion coefficients $\alpha_{1}$ and $\alpha_{2}$
therefore form the well-known Jones vector
$\alpha=\bigg{(}\begin{array}[]{c}\alpha_{1}\\\
\alpha_{2}\end{array}\bigg{)}$. It is important to note that the polarization
bases mean any two orthogonal polarization vectors that are physically
permissible. Any linear combination of them such as (37) describes a possible
state of polarization. With this in mind, the constant vectors $\bar{x}$ and
$\bar{y}$ in expressions (29) or (30), though they are orthogonal to each
other, cannot be regarded as polarization bases for the plane wave in the
chiral medium. Because the polarization bases $\bar{x}^{\prime}$ and
$\bar{y}^{\prime}$ are rotated with propagation in the same way, the
polarization state of refracted wave (36) is also rotated with propagation no
matter what the Jones vector is. When
$\alpha=\frac{1}{\sqrt{2}}\bigg{(}\begin{array}[]{c}1\\\ i\end{array}\bigg{)}$
or $\alpha=\frac{1}{\sqrt{2}}\bigg{(}\begin{array}[]{c}1\\\
-i\end{array}\bigg{)}$, Eq. (36) goes back to circularly polarized wave (33a)
or (33b). That is, the circularly polarized waves in (33) are just special
cases of elliptically polarized wave (36). It is incorrect to say [10] that
circularly polarized waves are transmitted unchanged in the chiral medium. The
optical activity of helically-coiled optical fibers [3, 4, 5, 6] can be
discussed in a similar manner with the result that the rotation of RCP and LCP
waves yields opposite phases [24, 25, 26].

By the way, the two circularly polarized waves in (33) are also orthogonal to
each other. They can serve as base modes to expand the refracted wave (18) in
the form,

$\mathbf{E}_{t}=c_{+}\mathbf{E}_{+,t}+c_{-}\mathbf{E}_{-,t}=\mathbf{a}_{t}(z)TE_{i}\exp(ikz),$
(38)

where

$\mathbf{a}_{t}(z)=c_{+}\mathbf{r}(z)+c_{-}\mathbf{l}(z).$ (39)

Here the unit vectors $\mathbf{r}$ and $\mathbf{l}$, which are given in (34),
also serve as polarization bases to expand the polarization vector
$\mathbf{a}_{t}(z)$ because they are orthogonal to each other at the same
propagation distance. When $c_{+}=c_{-}=\frac{\sqrt{2}}{2}$, Eq. (38) goes
back to linearly polarized wave (31a). Whereas when
$c_{+}=-c_{-}=-i\frac{\sqrt{2}}{2}$, it goes back to linearly polarized wave
(31b). It is thus clear that the rotation of the polarization plane of
linearly polarized wave (31a) or (31b), when considered as a coherent
superposition of RCP and LCP waves, comes from the rotation of the
polarization states of their circularly polarized components.

##  VI Conclusions and Remarks

In conclusion, we showed through a detailed logical analysis that there is not
circular birefringence in an isotropic chiral medium. We proved that both the
RCP and the LCP waves propagate at the same phase velocity as the linearly
polarized wave. In addition, we found that any elliptically polarized wave in
a particular chiral medium is rotated in the same way. This is described by
its rotatory polarization vector and is indicated by expression (37) or,
equivalently, by expression (39). The circularly polarized waves are so
special that the result of the rotation of their polarization states appears
as opposite phases as if, as expressed by Eqs. (23) and (24), they propagated
at different phase velocities with their polarization states transmitted
unchanged.

What is remarkable is that the rotation of the polarization state of the plane
wave (36) in the chiral medium comes only from the rotatory polarization bases
(32). It does not involve the change of the Jones vector. This is in
consistency with the observation [2, 20, 18, 6] that the optical rotation in
the chiral medium shows up as the rotation of the axes of polarization
ellipse. It is hence improper to describe the optical activity as the
transformation of the Jones vector [27]. The optical activity in the chiral
medium implies a new mechanism to change the state of polarization of light.
It amounts to the change of the polarization bases with the Jones vector
remaining fixed. The physics that underlies such a mechanism needs further
discussions, which are beyond the scope of the present paper and will be
presented elsewhere.

## Acknowledgments

This work was supported in part by National Natural Science Foundation of
China under Grant No. 11974251.

## References

  * [1] E. Hecht, Optics, 4th ed. (Addison-Wesley, San Francisco, 2001).
  * [2] L. D. Barron, Molecular Light Scattering and Optical Activity, 2nd ed. (Cambridge University Press, Cambridge, 2004).
  * [3] A. Papp and H. Harms, Polarization optics of liquid-core optical fibers, Appl. Opt. 16, 1315–1319 (1977).
  * [4] R. Ulrich and A. Simon, Polarization optics of twisted single-mode fibers, Appl. Opt. 18, 2241–2251 (1979).
  * [5] J. N. Ross, The rotation of the polarization in low birefringence monomode optical fibres due to geometric effects, Opt Quant Electron 16, 455–461 (1984).
  * [6] Y. Chen and P. St. J. Russell, Frenet-Serret analysis of helical Bloch modes in N-fold rotationally symmetric rings of coupled spiraling optical waveguides, J. Opt. Soc. Am. B 38, 1173–1183 (2021).
  * [7] E. Georgieva, Reflection and refraction at the surface of an isotropic chiral medium: eigenvalue-eigenvector solution using a $4\times 4$ matrix method, J. Opt. Soc. Am. A 12, 2203–2211 (1995).
  * [8] S. Bassiri, C. H. Papas, and N. Engheta, Electromagnetic wave propagation through a dielectric-chiral interface and through a chiral slab, J. Opt. Soc. Am. A 5, 1450–1459 (1988).
  * [9] M. P. Silverman, Reflection and refraction at the surface of a chiral medium: comparison of gyrotropic constitutive relations invariant or noninvariant under a duality transformation, J. Opt. Soc. Am. A 3, 830–837 (1986).
  * [10] R. W. Ditchburn, Light (Blackie & Son, London, 1958).
  * [11] A. Lakhtakia, V. V. Varadan, and V. K. Varadan, A parametric study of microwave reflection characteristics of a planar achiral-chiral interface, IEEE Trans. Electromagn. Compat. 28, 90–95 (1986).
  * [12] H. Cory and I. Rosenhouse, Electromagnetic wave reflection and transmission at a chiral-dielectric interface, J. Mod. Opt. 38, 1229–1241 (1991).
  * [13] V. K. Valev, J. J. Baumberg, C. Sibilia, and T. Verbiest, Chirality and chiroptical effects in plasmonic nanostructures: fundamentals, recent progress, and outlook, Adv. Mater. 25, 2517–2534 (2013).
  * [14] C. N. Alexeyev, E. V. Barshak, B. P. Lapin, and M. A. Yavorsky, Reciprocal optical activity in multihelicoidal optical fibers, Phys. Rev. A 98, 023824 (2018).
  * [15] W. Kaminsky, Experimental and phenomenological aspects of circular birefringence and related properties in transparent crystals, Rep. Prog. Phys. 63, 1575–1640 (2000).
  * [16] A. Ghosh and P. Fischer, Chiral molecules split light: reflection and refraction in a chiral liquid, Phys. Rev. Lett. 97, 173002 (2006).
  * [17] M. Pfeifer and P. Fischer, Weak value amplified optical activity measurements, Opt. Express 19, 16508–16517 (2011).
  * [18] X. M. Xi, T. Weiss, G. K. L. Wong, F. Biancalana, S. M. Barnett, M. J. Padgett, and P. St. J. Russell, Optical activity in twisted solid-core photonic crystal fibers, Phys. Rev. Lett. 110, 143903 (2013).
  * [19] C. N. Alexeyev, B. P. Lapin, G. Milione, and M. A. Yavorsky, Optical activity in multihelicoidal optical fibers, Phys. Rev. A 92, 033809 (2015).
  * [20] C. N. Alexeyev, E. V. Barshak, B. P. Lapin, D. V. Vikulin, and M. A. Yavorsky, Amplification of optical activity in a fiber loop resonator, Appl. Opt. 59, 10160–10167 (2020).
  * [21] Here circular double refraction refers to the phenomenon that the refracted wave consists of RCP and LCP components propagating in different directions. It does not mean circular birefringence, which refers to different velocities or different indices of refraction for RCP and LCP components.
  * [22] D. L. Jaggard and X. Sun, Theory of chiral multilayers, J. Opt. Soc. Am. A 9, 804–813 (1992).
  * [23] A. Lakhtakia, V. V. Varadan, and V. K. Varadan, Reflection of plane waves at planar achiral-chiral interfaces: independence of the reflected polarization state from the incident polarization state, J. Opt. Soc. Am. A 7, 1654–1656 (1990).
  * [24] R. Y. Chiao and Y.-S. Wu, Manifestations of Berry’s topological phase for the photon, Phys. Rev. Lett. 57, 933–936 (1986).
  * [25] A. Tomita and R. Y. Chiao, Observation of Berry’s topological phase by use of an optical fiber, Phys. Rev. Lett. 57, 937–940 (1986).
  * [26] M. V. Berry, Interpreting the anholonomy of coiled light, Nature 326, 277–278 (1987).
  * [27] J. N. Damask, Polarization optics in telecommunications (Springer Science+Business Media, New York, 2005).


# Optimizing the energy consumption of spiking neural networks for
neuromorphic applications

Martino Sorbaro,1,2 Qian Liu,1 Massimo Bortone,1 and Sadique Sheik1,
1aiCTX AG, Zürich, Switzerland,
2 Institute of Neuroinformatics, Univ. of Zürich and ETH Zürich, Switzerland
<EMAIL_ADDRESS>
###### Abstract

In the last few years, spiking neural networks have been demonstrated to
perform on par with regular convolutional neural networks. Several works have
proposed methods to convert a pre-trained CNN to a Spiking CNN without a
significant sacrifice of performance. We demonstrate first that quantization-
aware training of CNNs leads to better accuracy in SNNs. One of the benefits
of converting CNNs to spiking CNNs is to leverage the sparse computation of
SNNs and consequently perform equivalent computation at a lower energy
consumption. Here we propose an optimization strategy to train efficient
spiking networks with lower energy consumption, while maintaining similar
accuracy levels. We demonstrate results on the MNIST-DVS and CIFAR-10
datasets.

Keywords neuromorphic computing, neural networks, spiking networks, loss
function, synaptic operations, energy consumption, convolutional networks,
CIFAR10, MNIST-DVS

##  1 Introduction

Since the early 2010s, computer vision has been dominated by the introduction
of convolutional neural networks (CNNs), which have yielded unprecedented
success in previously challenging tasks such as image recognition, image
segmentation or object detection, among others. Considering the theory of
neural networks was mostly developed decades earlier, one of the main driving
factors behind this evolution was the widespread availability of high-
performance computing devices and general purpose Graphic Processing Units
(GPU). In parallel with the increase in computational requirements (Strubell
et al., 2019), the last decades have seen a considerable development of
portable, miniaturized, battery-powered devices, which pose constraints on the
maximum power consumption.

Attempts at reducing the power consumption of traditional deep learning models
have been made. Typically, these involve optimizing the network architecture,
in order to find more compact networks (with fewer layers, or fewer neurons
per layer) that perform equally well as larger networks. One approach is
energy-aware pruning, where connections are removed according to a criterion
based on energy consumption, and accuracy is restored by fine-tuning of the
remaining weights (Yang et al., 2017; Molchanov et al., 2016). Other work
looks for more efficient network structures through a full-fledged
architecture search (Cai et al., 2018). The latter work was one of the winners
of the Google “Visual Wake Words Challenge” at CVPR 2019, which sought models
with memory usage under 250 kB, model size under 250 kB and per-inference
multiply-add count (MAC) under 60 millions.

Using spiking neural networks (SNNs) on neuromorphic hardware is an entirely
different, and much more radical, approach to the energy consumption problem.
In SNNs, like in biological neural networks, neurons communicate with each
other through isolated, discrete electrical signals (spikes), as opposed to
continuous signals, and work in continuous instead of discrete time.
Neuromorphic hardware (Indiveri et al., 2011; Furber, 2016; Thakur et al.,
2018; Esser et al., 2016) is specifically designed to run such networks with
very low power overhead, with electronic circuits that faithfully reproduce
the dynamics of the model in real time, rather than simulating it on
traditional von Neumann computers. Some of these architectures (including
Intel’s Loihi, IBM’s TrueNorth, and aiCTX’s DynapCNN) support convolution
operations, which are necessary for modern computer vision techniques, by an
appropriate weight sharing mechanism.

The challenge of using SNNs for machine learning tasks, however, is in their
training. Mimicking the learning process used in the brain’s spiking networks
is not yet feasible, because neither the learning rules, nor the precise
fitness functions being optimized are sufficiently well understood, although
this is currently a very active area of research (Marblestone et al., 2016;
Richards et al., 2019). Supervised learning routines for spiking networks have
been developed (Bohte et al., 2002; Mostafa, 2017; Nicola and Clopath, 2017;
Neftci et al., 2019; Shrestha and Orchard, 2018), but are slow and challenging
to use. For applications which have little or no dependence on temporal
aspects, it is more efficient to train an analog network (i.e., a traditional,
non-spiking one) with the same structure, and transfer the learned parameters
onto the SNN, which can then operate through rate coding. In particular, the
conversion of pre-trained CNNs to SNNs has been shown to be a scalable and
reliable process, without much loss in performance (Diehl et al., 2015;
Rueckauer et al., 2017; Sengupta et al., 2019). But this approach is still
challenging, because the naive use of analog CNN weights does not take into
account the specific characteristics and requirements of SNNs. In particular,
SNNs are more sensitive than analog networks to the magnitude of the input.
Naive weight transfer can, therefore, lead to a silent SNN, or, conversely, to
one with unnecessarily high firing rates, which have a high energy cost.

Here, we propose a hybrid training strategy which maintains the efficiency of
training analog CNNs, while accounting for the fact that the network is being
trained for eventual use in SNNs. Furthermore, we include the energy cost of
the network’s computations directly in the loss function during training, in
order to minimize it automatically and dynamically. We demonstrate that
networks trained with this strategy perform better per Joule of energy
utilized. While we demonstrate the benefit of optimizing based on energy
consumption, we believe this strategy is extendable to any approach that uses
back-propagation to train the network, be it through a spiking network or a
non-spiking network.

In the following sections, we will detail the training techniques we devised
and applied for these purposes. We will test our networks on two standard
problems. The first is the MNIST-DVS dataset of Dynamic Vision Sensor
recordings. DVSs are event-based sensors, and, as such, the analysis of their
recordings is an ideal application of spike-based neural networks. The second
is the standard CIFAR-10 object recognition benchmark, which provides a
reasonable comparison on computation cost to non-spiking networks. For each of
these tasks, we will demonstrate the energy-accuracy trade-off of the networks
trained with our methods. We show that significant amounts of energy can be
saved with a small loss in performance, and conclude that ours is a viable
strategy for training neuromorphic systems with a limited power budget.

##  2 Materials and Methods

In most state-of-the art neuromorphic architectures with time multiplexed
units like Merolla et al. (2014); Davies et al. (2018); Furber et al. (2014),
the various states need to be fetched from memory and rewritten. Such
operations happen every time a neuron receives a synaptic event. Whenever one
of these operations is performed, the neuromorphic hardware consumes a certain
amount of energy. For instance in Indiveri and Sandamirskaya (2019) the
authors show that this energy consumption is usually of the order of
$10^{-11}$ J. While there are several other processes that consume power on a
neuromorphic device, the bulk of the active power on these devices is used by
the synaptic operations. Reducing their number is therefore the most natural
way to keep energy usage low.

In this paper we explore strategies to lower synaptic operations and evaluate
their effect on the network’s computational performance. We suggest to train,
or fine-tune, networks with an additional loss term which explicitly enforces
lower activations in the trained network — and consequently lower firing rates
of the corresponding spiking network. This is analogous to the $L_{1}$ term
used by Esser et al. (2016) and Neil et al. (2016), but applied on synaptic
operations directly rather than firing rates, and set up so that a target
SynOp count value can be set. Additionally, we introduce quantization of the
activations on each layer, which mimics the discretization effect of spiking
networks, so that the network activity remains at reasonable levels even when
the regularization term is strong. The following sections illustrate the
technical details and introduce the datasets and networks we use for
evaluation.

###  2.1 Training strategies

####  2.1.1 Parameter scaling

By scaling the weights, biases and/or thresholds of neurons in different
layers, we can influence the number of spikes generated in each layer, thereby
allowing us to tune the synaptic activity of the model. This is easy to do,
even with pre-trained weights. For a scale-invariant network, such as any
network whose only non-linearities are ReLUs, this method attains perfect
results, because a linear rescaling of the weights causes a linear rescaling
of the output, which gives identical results for classification tasks where we
select the class that receives the highest activation.

We use this method as a baseline comparison for our results. We chose to
rescale the weights of the first convolutional layer of our network by a
variable factor $\rho$:

$w_{0}^{\prime}=\rho w_{0},$

which is equivalent to a rescaling of the input signal by the same factor.
Note that an increase/decrease in the first layer’s output firing rate causes
a correspondent increase/decrease in the activation of all the subsequent
layers, and thereby reduces the global energy consumption of the whole
network.

For baseline comparisons, we also apply the “robust” weight scaling suggested
by Rueckauer et al. (2017). This consists of a per-layer scaling of weights,
in such a way that the maximum level of activation is constant along the
network. For robustness, the 99th percentile of activations is taken as a
measure of output magnitude in each layer, estimated from forward passes over
25600 samples of the training set. In this way, the activity of the network is
balanced over its layers, in the sense that no layer unnecessarily amplifies
or reduces the activity level compared to its input.

The scale-invariance property of ReLU functions does not hold for the
corresponding spiking network, and small activation values could cause
discretization errors, or even yield a completely silent spiking network from
a perfectly functional analog network.

####  2.1.2 Synaptic Operation optimization

We measure the activity of the network, for each layer group, in
correspondence with the ReLU operations, which effectively correspond to the
spikes from an equivalent SNN (Supplementary Figure 2). We denote the activity
of neuron $i$ in layer $\ell$ as $a_{i}^{\ell}$. We define the _fan-out_ of
each group of layers, $f_{\text{out}}^{\ell}$, as the number of units of layer
$\ell+1$ that receive the signal emitted by a single neuron in layer $\ell$.
This measure is essential in estimating the number of synaptic operations
(SynOps) $s^{\ell}$ elicited by each layer:

$s^{\ell}=f_{\text{out}}^{\ell}\sum_{i}a_{i}^{\ell}$ (1)

We directly add this number to the loss we want to minimize, optionally
specifying a target value $S_{0}$ for the desired number of SynOps:

$\mathcal{L}=\mathcal{C}(a^{\text{output}},t)+\alpha\left({S_{0}-\sum_{\ell}s^{\ell}}\right)^{2}$
(2)

where $\mathcal{C}$ is the cross-entropy loss, $t$ is the target label, and
$\alpha$ is a constant. We will refer to this additional term as _SynOp loss_.
In this work, we will always choose $\alpha=1/S_{0}^{2}$, in order to keep the
SynOp loss term normalized independently of $S_{0}$. Although setting
$S_{0}=0$ and tweaking the value of $\alpha$ instead is also a valid choice,
we found it easier to set a direct target for the power budget, which leads to
more predictable results.

Additionally, we performed some experiments where an $L_{1}$ penalty on
activations was used, without fanout-based weighting. Against our
expectations, we did not find a significant difference in power consumption
between the models trained with or without per-layer weighting (Supplementary
Figure 1). However, we use the fanout-based penalty throughout this paper,
since this addresses the power consumption more directly, and we cannot rule
out that this difference may be more significant in larger networks.

####  2.1.3 Quantization-aware training and surrogate gradient

Optimizing for energy consumption with the SynOp loss mentioned above has
unintended consequences. During training, the optimizer tries to achieve
smaller activations, but cannot account for the fact that, when the
activations are too small, discretization errors become more prominent.
Throughout this paper, by discretization error we mean the discrepancy that
occurs when a real number needs to be represented in a discrete way — namely,
the value of each neuron’s activation, which is continuous, needs to be
translated in a finite number of spikes, leading to inevitable approximations.
To solve this issue, we introduce a form of quantization during training. The
quantization of activations mimics, in the context of an analog network, a
form of discretization analogous to what happens in a spiking network.
Therefore, the network can be already aware of the discretization error at
training time, and automatically adjust its parameters in order to properly
account for it. To this end, we turn all ReLU activation functions into
“quantized” (i.e. step-wise) ReLUs, which additionally truncate the inputs to
integers, as follows:

$\operatorname{QReLU}(x)=\begin{cases}0&x\leq 0\\\ \lfloor
x\rfloor&x>0\end{cases}$ (3)

where $\lfloor\cdot\rfloor$ indicates the floor operation. This choice
introduces a further problem: this function is discontinuous, and its
derivative is uniformly zero wherever it is defined. To avoid the zeroing of
gradients during the backward pass, we use a _surrogate gradient_ method
(Neftci et al., 2019), whereby the gradient of QReLU is approximated with the
gradient of a normal ReLU during the backward pass:

$\nabla_{x}\operatorname{QReLU}(x)\approx\begin{cases}0&x\leq 0\\\
1&x>0\end{cases}$ (4)

This is not the only way to approximate the gradient of a step-wise function
in a meaningful way, and closer approximations are certainly possible;
however, we found that this linear approximation works sufficiently well for
our purposes.

In this work, we apply QReLUs in combination with the SynOp loss term
illustrated in the previous section, but quantization on activations could be
independently used for a more accurate training of spiking networks. We note
that quantization-aware training in different forms has been used before (Guo,
2018; Hubara et al., 2017), but its typical purpose is to sharply decrease the
memory consumption of CNNs, by storing both activations and weights as lower-
precision numbers (e.g. as int8 instead of the typical float32). PyTorch
recently started providing support utilities for this
purpose.111https://pytorch.org/docs/stable/quantization.html

###  2.2 Spiking network simulations with Sinabs

After training, we tested our trained weights on spiking network simulations.
Unlike tests done on analog networks, these are time-dependent simulations,
which fully account for the time dynamics of the input spike trains, and
closely mimic the behaviour of a neuromorphic hardware implementation, like
DynapCNN (Liu et al., 2019). Our simulations are written using the Sinabs
Python library222https://aictx.gitlab.io/sinabs/, which uses non-leaky
integrate-and-fire neurons with a linear response function. The sub-threshold
neuron dynamics of the non-leaky integrate and fire neurons are described as
follows:

$\displaystyle\dot{v}$ $\displaystyle=$ $\displaystyle
R\cdot\big{(}I_{syn}(t)+I_{bias}\big{)}$ (5) $\displaystyle I_{syn}(t)$
$\displaystyle=$ $\displaystyle WS(t)$ (6)

where $v$ is the membrane potential of the neuron, $R$ is a constant,
$I_{syn}$ is the synaptic input current, $I_{bias}$ is a constant input
current term, $W$ is the synaptic weight matrix and $S(t)$ is a vector of
input spike trains. For the results presented in this paper, we assume $R=1$
without any loss of generality. Upon reaching a spiking threshold $v_{th}$ the
neuron’s membrane potential is reduced by a value $v_{th}$ (not reset to
zero).

As a result of the above, between times $t$ and $t+\delta t$, for a total
input current $I(t)=I_{syn}(t)+I_{bias}$, the neurons generate a number of
spikes $n(t,t+\delta t)$ given by the following equation:

$n(t,t+\delta t)=\left\lfloor\frac{R\cdot I(t)\cdot\delta
t}{v_{th}}\right\rfloor.$ (7)

In order to simulate the equivalent SNN model on Sinabs, the CNN’s pre-trained
weights are directly transferred to the equivalent SNN.

###  2.3 Digit recognition on DVS recordings

Figure 1: Illustration of the MNIST-DVS dataset, as used in this work, and of
the network model we used for the task. A: a single accumulated frame of 3000
spikes, as used for training. B: the corresponding 3000 spikes, in location
and time. C: example single-millisecond frames, as sequentially shown to the
Sinabs spiking network during the tests presented in figure 2. D: The
convolutional network model we used for this task. All convolutional layers
and the linear layer are followed by ReLUs. Dropout is used before the linear
layer at training time.

####  2.3.1 Task and Dataset

As a benchmark to assess the performance of the above training methods, we
used an image recognition task on real data recorded by a Dynamic Vision
Sensor (DVS). Given a spike train generated by the DVS, our spiking networks
identify the class to which the object belongs—corresponding to the fastest-
firing neuron in the output layer. For this task, we used the MNIST-DVS
dataset at scale 16 (Serrano-Gotarredona and Linares-Barranco, 2015; Liu et
al., 2016), a collection of DVS recordings where digits from the classic MNIST
dataset (LeCun et al., 1998) are shown to the DVS camera as they move on a
screen.

During the training phase, we presented the (analog) neural network with
images formed of accumulated DVS events, i.e. DVS spike trains divided into
chunks and collapsed along the time dimension. The value of each pixel (0-255
in the image encoding we chose) was determined simply by the number of events
on that pixel. The DVS recordings were split into frames not based on time
length, but according to event count: the accumulation of each frame was
stopped when the total number of events per frame reached a value of 3000
(figure 1A). This ensured all frames had comparable pixel values without the
need for normalization, and all contained similar amounts of information
regardless of the type of activity presented. The information regarding event
polarity was discarded, resulting in a 1-channel input frame (analogous to
gray-scale image).

During testing on the spiking network simulation, the corresponding spike
trains were presented to the network with 1 ms time resolution. This value was
chosen to enable reasonable simulation times but could be lowered if needed.
(Figure 1C), to simulate the real-time event transmission between the DVS and
a neuromorphic chip. The network state was reset between the presentation of a
data chunk and the next. The polarity of events was ignored. Of the original
10000 recordings (1000 per digit from zero to nine), we set $20\%$ aside as
test set.

####  2.3.2 Network architecture

In order to solve the task mentioned above, we used a simple convolutional
neural network, with three 2D convolutional layers (3x3 filters), each
followed by an average pooling layer (2x2 filters) and a rectified linear
unit. The choice of average pooling is due to the difficulties of implementing
max pooling in spiking networks Rueckauer et al. (2017). The last layer is a
linear (fully connected) layer, which outputs the class predictions (Figure
1D). We used a cross-entropy loss function to evaluate the model predictions
and optimized the network weights using the Adam optimizer (Kingma and Ba,
2014) with a learning rate of $10^{-3}$. Bias parameters were deactivated
everywhere in the network. A 50% dropout was used just before the fully
connected layer at training time. The network was implemented using PyTorch
(Paszke et al., 2017).

The whole procedure can be summarized as follows:

  1. 1.

The dataset is prepared by dividing the original DVS recordings in sections of
3000 spikes each, ignoring event polarity. From these the following is saved:

    1. (a)

The spike train itself, used for testing

    2. (b)

An image, corresponding to the time-collapsed spike train, with pixel values
equal to the number of spikes at that location, used for training

  2. 2.

A neural network is trained, applying quantization in correspondence with
every ReLU. The loss used for training is binary cross-entropy with the
addition of the synoploss term (eq. 2).

  3. 3.

The trained weights are transferred to a spiking network simulation,
implemented in Sinabs. The network dynamics is simulated with 1 ms time
resolution. The network prediction is defined as the neuron that spikes the
most over the 3000-spike input. Synaptic operations are counted as the sum of
spikes emitted by each layer, weighted on the fan-out of that layer.

For reproducibility, the python code implementing these methods is available
at gitlab.com/aiCTX/synoploss.

###  2.4 Object recognition on CIFAR-10

####  2.4.1 Task and Dataset

In order to validate the approach on a dataset with higher complexity than
MNIST, we also benchmarked our work on CIFAR-10 (Krizhevsky et al., 2009), a
visual object classification task. The input images were augmented with random
crop and horizontal flip, and then normalized to $[-1,1]$. A 20% dropout rate
was applied to the input layer to further augment the input data.

For the experimental results on this dataset, we directly injected the image
pixel analog value to the first layer of convolutions as input current in each
simulation time step for $N_{dt}$ time steps. The magnitude of the current was
scaled down by the same value $N_{dt}$, in order to have an accumulated
current, over the whole simulation, equal to the analog input value. The
Sinabs simulations were run for $N_{dt}=10$ time steps, obtaining SynOps and
accuracy values. The network state was reset between the presentation of an
image and the next.

####  2.4.2 Network architecture and training procedure

In order to solve the task mentioned above, we used an All-ConvNet
(Springenberg et al., 2014), a 9-layer convolutional network, without bias
terms, which has 1.9M parameters in total. The ReLU layers in the model,
including the last output layer, were replaced with QReLUs. All the
convolutional layers in this network are followed by a dropout layer with a
rate of 10%, which not only prevents over-fitting, but also compensates the
SNN’s discrete representations of analog values. As illustrated in
Springenberg et al. (2014), training lasts 350 epochs, and the learning rate
is initialized at $2.5\times 10^{-4}$ and scaled down by a factor of 10 at
epochs [200, 250, 300]. We use the Adam optimizer with weight decay of
$10^{-3}$. Note that the model was trained without ReLU on the last output
layer, since it is harder to train the classification layer when the outputs
are only positive, while the classification accuracy was tested with ReLU on
the output layer, in order to have an equivalent network to the spiking model.

The entire experiment is as follows:

  1. 1.

Train an ANN network, anet, get its MAC and test the accuracy with original
CIFAR10 dataset.

  2. 2.

Scale up the weights of the first layer of anet by $\rho$, and transfer the
weights to the SNN equivalence, snet.

    1. (a)

test the accuracy and SynOps of snet with $N_{dt}=10$, the input current is
1/10 of a pixel value.

    2. (b)

increase $\rho$, and repeat 2(a) till the accuracy reached about the ANN
accuracy.

  3. 3.

Select a $\rho$ from Step 2, where the snet have SynOps $>$ MAC, and start
quantization-aware training with the SynOp loss.

    1. (a)

set the target SynOp to half of the current SynOps and train.

    2. (b)

test the accuracy and get the SynOps, then repeat 3(a) until the accuracy is
too low to be meaningful, and thus a full accuracy/SynOps curve is obtained.

####  2.4.3 SynOps Optimization

Before training the network with QReLU activations, the network was first
trained with ReLU to get an initial set of parameters. The network with QReLU
was then initialized with the scaled parameters (scaling up by $\rho$ on the
first layer). The scaling factor $\rho$ was chosen to initialize the network
in a state where enough information is propagated through layers so that the
network performs reasonably well. Consequently, the weights of the last
weighted layer were scaled by $1/\rho$, in order to adapt the classification
loss back to its original range.

During testing, we measured the ANN and SNN performance in terms of their
accuracy and SynOps, and found a mismatch of SynOps between training and
testing. There are two main reasons: 1) The output of a dropout layer (with a
dropout rate $p$) is always scaled down by $1-p$ to compensate the dropped out
activations, however the mismatch could be large after a sequence of dropout
layers. 2) Due to discrete spike events operated in the network where the
order (not only the count) of the spikes matters, the mismatch occurs between
the spike count-based analog activation and the actual spiking ones.

To compensate for this mismatch, for all the trained models we tested the
performance with both $1.5\times$ and $2\times$ scaled-up first layer weights.
Lastly, we optimized the QReLU-based model with the objective of minimizing
the classification error given a target SynOps. We trained 30 models with
lower and lower target SynOps, and each model was initialized with the trained
weights of the previous one.

##  3 Results

###  3.1 The SynOp loss term leads to a reduction in network activity on DVS
data

Figure 2: Results on the MNIST-DVS dataset. Left: SynOps-accuracy curves
computed on a spiking network simulation with Sinabs. Each point represents a
different model, trained for a different value of $S_{0}$ or rescaled by a
different value of $\rho$. The red star represents the original model,
standard CNN weights transferred to the SNN without changes. The solid lines
are smoothed versions of the curves described by the data points, provided as
a guide to the eye. Center: a zoomed-in version of the left panel, showing
direct comparison between layer-wise gain scaling (Rueckauer et al., 2017) and
our method. Right: SynOps per layer, compared between the baseline model and a
selected model trained with the SynOp loss and quantization. This is the model
indicated by a black circle in the first panel. Note that the input SynOps
depend only on the input data, and cannot be changed by training.

In figure 2, we show the results of four methods to reduce the activity of the
network, in a way that yields energy savings. First, as a baseline, we trained
a traditional CNN using a cross-entropy loss function, and rescaled down the
weights of its first layer. This is equivalent to rescaling the input values,
and has the effect of proportionally reducing the activity in all subsequent
layers of the network. The “baseline” model in figure 2 is the same network,
with no input rescaling: weights are transferred from the CNN to the
corresponding layers of the SNN without any changes or special considerations.
Thresholds are set to 1 on all layers. Second, following Rueckauer et al.
(2017), we rescaled the weights of each layer in such a way that the maximum
activation in each layer stays constant (see Methods). The input weights are
again rescaled as stated above. Third, we introduced an additional term in the
loss function, the SynOp loss, which directly pushes the estimated number of
SynOps to a given value. We trained CNN models, each with a different target
number of synaptic operations, independently of each other. Furthermore,
excessive reduction of the SynOps leads to the silencing of certain neurons,
and other discretization errors, causing an immediate drop in accuracy. To
account for this we jointly use the SynOp loss term and quantization-aware
training.

We tested our training methods on a real-world use case of spiking neural
networks. Dynamic Vision Sensors (DVS) are used in neuromorphic engineering as
very-low-power sources of visual information, and are a natural data source
for SNNs simulated on neuromorphic hardware. We transferred the weights
learned with the methods described above onto a spiking network simulation,
and used it to identify the digits presented to the DVS in the MNIST-DVS
dataset.

Our results show that adding a requirement on the number of synaptic
operations to the loss yields better results in terms of accuracy compared to
rescaling input weights and layer-wise activation gains (Figure 2, orange).
Using the SynOp loss together with quantization during training outperforms
the simpler methods, allowing for further reduction of the SynOps value with
smaller losses in accuracy (Figure 2, blue).

Among the models trained in this way, we selected one with a good balance
between energy consumption and accuracy, and used it for a direct comparison
with the baseline (that is, weights from an ANN without quantization and no
additional loss terms). The second and third panels of figure 2 graphically
show the large decrease in the number of synaptic operations required by each
layer of our model, and the very small reduction in performance. This
particular model brings accuracy down from 96.3% to 95.0%, while reducing the
number of synaptic operations from 3.86M to 0.63M, an 84% reduction of the
SynOp-related energy consumption.

###  3.2 The SynOp loss leads to a lower operations count compared to ANNs on
CIFAR10

Figure 3: Accuracy vs. SynOps curves on the non-spiking CIFAR-10 task. ‘ANN’
results (red crosses and purple dots) are SynOp count estimations based on the
quantized activations of an analogue network. ‘SNN’ results (blue crosses and
green dots) are SynOp values obtained by Sinabs simulation. The performance of
models fine-tuned with SynOp loss and quantization (dots) shows a clear
advantage over weight rescaling (crosses). Right panel: a zoomed-in plot of
the dashed region. A high-light blue cross represents a good performance of
the arbitrary weight scaling. The good trade-off points, trained with SynOp
loss and quantization, listed in table 1 are marked with green ‘+’. The
results from other work are marked with orange triangles. The original ANN
model, for which the MAC is plotted instead of the SynOps, is marked with a
red star.

SNNs are a natural way of working with DVS events, having advantages over ANNs
in event-driven processing. However, it is also interesting to highlight the
benefits of using SNNs over ANNs in conventional non-spiking computer vision
tasks, e.g. CIFAR-10, where SNNs can still offer advantages in power
consumption. As stated in Section 2.4 we have trained the network with two
approaches: 1) conventional ANN training plus weight scaling as the baseline;
2) further training with QReLU and SynOp Loss for performance optimization.

####  3.2.1 Weight Scaling

We first trained the analog All-ConvNet on CIFAR-10, attaining an accuracy of
91.37% and a MAC of 306M (red star in Figure 3). Then, we transferred the
trained weights directly on the equivalent SNN and scaled the weights of its
first layer to manipulate the overall activity level. This is shown by the
blue crosses in Figure 3: as the SynOp count grows, so does the accuracy.
However, the SynOps are around 10 times to the MAC of ANN when the accuracy
reaches an acceptable rate of 90.7%. To improve on this result, we fine-tuned
this training by adding quantization and the SynOp loss.

A faster way to measure the same quantities is by testing the analog model,
with ReLU layers all replaced with QReLU, and count the activation levels
instead of the Sinabs spike counts. Estimations based on this quantized
activation layers are shown as red crosses in Figure 3. The performance on
accuracy and SynOps of the analog network and its spiking equivalent are well
aligned, showing that quantized activations are a good proxy for the firing
rates of the simulated SNN, at least in this regime.

####  3.2.2 SynOp loss optimization

We further fine-tuned one of the weight-scaled models obtained above, with the
addition of quantization-aware training and the SynOp loss. Figure 3 also
shows the classification accuracy and SynOps for both quantized-analog and
spiking models (blue and green dots respectively) trained with this method.

Multiple SNN test trials achieve better accuracy than the original ANN model
(red star, 91.37%), thanks to the further training with QReLU. As the SynOp
goes down, the accuracy stays above the original ANN model until 91.43% when
SynOps are at 277M (see one of the green ‘+’ in Figure 3). Note that, the SNN
has outperformed ANN both on accuracy and operations count, where the number
of MAC in the original ANN is 306M. As another good example of accuracy-SynOp
trade-off (90.37% at 127M), our model could perform reasonably well, above
90%, by reducing 58% (Syn-MAC ratio is 0.42) of computing operations from the
original ANN. Therefore, running the SNN model on neuromorphic hardware will
benefit on energy efficiency not only from the lower computation cost of
SynOps but also from the significant reduction on operation counts.
Additionally, the plot shows how this method outperforms weight scaling in
terms of operation counts by roughly a factor of 10 for all accuracy values.

As far as we know, our converted SNN model from the AllConvNet reached the
state-of-the-art accuracy at 91.75% among SNN models (see detailed comparison
in Table 1 and Figure 3). In addition, our model is the smallest, at 1.9M
parameters, while the BinaryConnect model (Rueckauer et al., 2017) is 7 times
larger in size and WeightNorm, consisting of a VGG-16 (Sengupta et al., 2019),
is 8-fold in size. Although achieving the best accuracy requires a SynOp of
2,179M, this can easily be reduced by 27% by giving up 0.02% in accuracy, see
the two green ‘+’ on the top-right of Figure 3. Comparing to the result from
Sengupta et al. (2019) (orange triangle on the right of Figure 3), our model
achieves 91.47% in accuracy at 368M SynOps, thus only loses 0.08% in accuracy
but saves 41% of SynOps and energy. Thanks to the optimization of the SynOp
loss, the number of SynOps is continuously pushed down while keeping an
appropriate accuracy, e.g. 85.71% at a SynOp of 64M. This result not only
outperforms most of the early attempts of SNN models for the CIFAR-10 task
(Panda and Roy, 2016; Cao et al., 2015; Hunsberger and Eliasmith, 2015), but
also brings down the SynOps to only 1/5 of the MAC and saves 86% energy
compared to Rueckauer et al. (2017).

In a brief summary, 1) the energy-aware training strategy pushes down the
SynOps 10 times compared to its weight scaling baseline; 2) the QReLU-trained
SNN achieves the state-of-the-art accuracy in CIFAR-10 task; and 3) the trade-
off performances between accuracy and energy show a significant save in
computation cost/energy comparing to existing SNN models and the equivalent
non-spiking CNN.

SNN Models | Net Architecture | Best Accuracy | Accuracy-SynOps trade-off
---|---|---|---
| N. par. | MAC | Acc. | SynOps |  | Syn-MAC
---
ratio
Acc. | SynOps |  | Syn-MAC
---
ratio
BinaryConnect | 14M | 616M | 90.85 | N/A | N/A | 84.87 | 460M | 0.75
WeightNorm | 15M | 313M | 91.55 | 618M | 1.98 | 91.55 | 618M | 1.98
Ours | 1.9M | 306M |  | 91.75
---
91.73
| 2179M
---
1593M
| 7.12
---
5.21
| 91.47
---
91.43
90.37
85.71
| 368M
---
277M
127M
64M
| 1.20
---
0.91
0.42
0.21
Table 1: Comparison with best SNN models on CIFAR-10. BinaryConnect: 8-layer
ConvNet from Rueckauer et al. (2017); WeightNorm: VGG-16 model from Sengupta
et al. (2019). The most efficient model and the best performance are
highlighted as bold fonts. Regarding to the comparisons on accuracy-SynOps
trade-off, blue colored result in our models refers to the performance close
to that of Rueckauer et al. (2017); while green shows the result approximates
to Sengupta et al. (2019). And the numbers in bold only highlight the winner
in these two comparisons.

####  3.2.3 SynOp vs. Accuracy for shorter inference times

Unlike the DVS data, which has its own time dynamics, there are no
restrictions on how static images should be presented to the network in time.
Therefore we measure total spike count instead of firing rates, thus
calculating the total energy cost per image, independently of time. For
example, Figure 3 shows how SynOp loss optimization pushes the SynOp to a
value lower than the MAC during training. This is one of the approaches in
which SNNs outperform ANNs in the accuracy-operations trade-off; while the
other benefit SNNs naturally bring is the temporal encoding and computation.
SNNs continuously output a prediction from the moment when the input currents
are injected. This prediction becomes more accurate with more time. For the
experiments presented in the previous sections, we only measure the
classification accuracy when the input is completely forwarded into the
network, $N_{dt}=10$: the input currents are chosen so that the total input
accumulated over $N_{dt}=10$ time steps is equivalent to that of the analog
network during training. In Figure 4, we measure how SNN models perform in the
course of the entire process, $N_{dt}=1,2,3,...,10$. The figure shows how
classification accuracy increases when more simulation time-steps are allowed,
and therefore more accumulated current is injected to the network. Each gray
curve represents a single trained model, and the SynOp and accuracy are tested
with increasing $N_{dt}$. The colored dots mark the accuracy-SynOps pair at
$N_{dt}=4,6,8,10$ over all trained networks. The same-colored dots approach to
the expected SNN result (green dots at $N_{dt}=10$) as $N_{dt}$ increases.

On the other hand, understanding the relationship between inference time and
accuracy is very relevant when dealing with DVS data. In general, a global
reduction of spike rates in a rate-based network causes a corresponding
increase in the latency, since more time is needed to accumulate enough spikes
for a reliable prediction. We compared one of our networks with an equivalent
model prepared through the ‘robust’ layer-wise normalization technique from
Rueckauer et al. (2017), with a few different values of input weight scale.
Figure 5 shows that the dependency of accuracy on the inference time follows a
similar trajectory for all these models. We conclude that the increase in
latency does not depend on the specific method used for optimization, and our
network’s latency is similar to that of other models with similar accuracy,
despite the much lower power consumption (shown in previous sections).

Figure 4: Total activity and accuracy on the CIFAR-10 benchmark for increasing
inference times (current injected for longer times, thus leading to more
SynOps). Gray lines correspond to individual SNN models, tested at
$N_{dt}=1,...,10$. To facilitate same $N_{dt}$ comparison, coloured dots were
added for all models at $N_{dt}=4,6,8,10$. Figure 5: Limited-time inference on
MNIST-DVS. Here, the accuracy of the networks is measured at a limited input
length of 10, 20, 30, 40, 50 ms. The accuracy of the network trained with our
method (again, we chose the one indicated by the black circle in figure 2)
behaves, in relation to observation time, similarly to that of other networks,
but with lower power consumption. The models and color choices are the same as
in figure 2. The different purple lines correspond to different rescaling of
the input layer weights, chosen so to have a comparable accuracy with the
other curves.

####  3.2.4 Effects on weight statistics

Common regularization techniques in ordinary neural networks often involve the
inclusion of an $L_{1}$ or $L_{2}$ cost on the network weights. In rough,
intuitive terms, $L_{1}$ regularization has a sparsifying effect, pushing
smaller connections towards zero; $L_{2}$ regularization generally keeps the
weights from growing to excessively large values. Conversely, the effect on
weights of penalizing synaptic operations or reducing the network’s activity,
as we do with the SynOp loss term, is not immediately clear. We investigate
whether imposing low synaptic operations count has a sparsifying effect on the
weight structure. To this end, we examine how many synaptic connections in our
models are null connections, which we define as weights $w$ such that
$|w|<10^{-9}$ (this threshold can be changed by several orders of magnitude
without impacting the conclusions). We performed this analysis on the networks
trained on the CIFAR-10 dataset, as explained in the previous sections. These
networks are much wider and deeper than the ones used for the MNIST-DVS task,
and therefore can better show weight sparseness effects. Figure 6 (left) shows
how the fraction of null weights changes with the SynOp count (and thus, of
the regularization strength), and compares it with the model’s accuracy. When
the number of synaptic operations is forced to be extremely low, the fraction
of null weights reaches values above 90%. A large increase in null
connections, however, is already noticeable for models above 80% accuracy,
showing that the SynOp loss term does have a sparsifying effect, and that this
is desirable. For the sake of completeness, in figure 6 (right), we also show
a depiction of the distribution of weights as a function of the number of
synaptic operations.

Setting synaptic weights to zero is effectively equivalent to pruning certain
connections between a layer and the next. Other than $L_{1}$ regularization of
the weights, more sophisticated pruning-and-retraining algorithms have been
studied in the machine learning literature (LeCun et al., 1990; Hassibi and
Stork, 1993). However, advanced pruning methods (such as those based on Fisher
information) are usually coupled with partial retraining of the network, and
are therefore more alike a form of architectural search (Crowley et al.,
2018). Due to the retraining of the remaining weights, these forms of pruning
are not guaranteed to reduce the activity levels if not coupled to other forms
of regularization.

Figure 6: Effect of the SynOp loss on the network’s weights. Left: the
fraction of near-zero weights ($|w|<10^{-9}$) greatly increases in models
where a stricter reduction of SynOp counts were imposed. The test-set accuracy
values for each model are also shown for comparison. Right: the distribution
of weights as a function of the SynOp count. Shaded areas indicate, from
lighter to darker, the following inter-quantile ranges: 10%–90%; 20%–80%;
30%–70%; 40%–60%. The solid line is the median weight. The models used for
this test are the same as those shown in figure 3, trained with quantization
on static CIFAR images.

##  4 Discussion and Conclusion

We used two techniques which significantly improve the energy requirements of
machine learning models that run on neuromorphic hardware, while maintaining
similar performances.

The first improvement consisted in optimising the energy expenditure by
directly adding it to the loss function during training. This method
encourages smaller activations in all neurons, which is not in itself an issue
in analog models, but can lead to discretization errors, due to the lower
firing rates, once the weights are transferred to a spiking network. To solve
this problem, we introduced the second improvement; quantization-aware
training, whereby the network activity is quantized at each layer, i.e. only
integer activations are allowed. Discretizing the network’s activity would
normally reduce all gradients to zero: this can be solved by substituting the
true gradient with a surrogate.

Applying these two methods together, we achieved an up to ten-fold drop in the
number of synaptic operations and the consequent energy consumption in the
DVS-MNIST task, with only a minor (1-2%) loss in performance, when comparing
to simply transferring the weights from a trained CNN to a spiking network. To
demonstrate the scalability of this approach, we also show that, as the
network grows bigger to solve a much more complex task of CIFAR-10 image
classification, the SynOps are reduced to 42% of the MAC, while losing 1% of
accuracy (90.37% at 127M). The accuracy-energy trade-off can be flexibly tuned
at training time. We also showed the consequences of using this method on the
distribution of network weights and the network’s accuracy as a function of
time.

While training based on static frames is not the optimal approach to leverage
all the benefits of spike-based computation, it enables fast training with the
use of state-of-the-art deep learning tools. In addition, the hybrid strategy
to train SNNs based on a target power metric is unique to SNNs. Conversely,
optimizing the energy requirement of an ANN/CNN requires modification of the
network architecture itself, which can require large amounts of computational
resources (Cai et al., 2018). In this work, we demonstrated that we can train
an SNN to a target energy level without a need to alter the network
hyperparameters. A potential drawback of this approach of (re)training the
model as opposed to simply transferring the weights of a pre-trained model is
brought to light when attempting to convert very deep networks trained over
large datasets such as IMAGENET. Pre-trained deep CNNs trained over large
datasets are readily available on the web and can be used to quickly
instantiate a spiking CNN. The task becomes much more cumbersome to optimize
for power utilization using the method described in this paper, ie. one has to
retrain the network over the relevant dataset for optimal performance.
However, our method can also be effectively used to fine-tune a pre-trained
network, removing the need for training from scratch (Supplementary Figure 1).
Furthermore, no large event-based datasets of the magnitude of IMAGENET exist
currently, and perhaps when such datasets are generated, the corresponding
models optimized for spiking CNNs will also be developed and made readily
available.

The quantization and SynOp-based optimization used in this paper can
potentially be applied, beyond the method illustrated here, in more general
contexts such as algorithms based on back-propagation through time to reduce
power usage. Such a reduction in power usage can make a large difference when
the model is ran on a mobile, battery-powered, neuromorphic device, with
potential for a significant impact in the industrial applications.

## Conflict of Interest Statement

All authors were employed by aiCTX AG during the course of the work published
in this article.

## Author Contributions

SS designed research; QL and SS contributed to the methods; MS, QL, and MB
contributed code and performed experiments; all authors wrote the paper.

## Funding

This work is supported in part by H2020 ECSEL grant TEMPO (826655). The funder
was not involved in the study design, collection, analysis, interpretation of
data, the writing of this article or the decision to submit it for
publication.

## Acknowledgments

The authors would like to thank Mr. Felix Bauer, Mr. Ole Richter, Dr. Dylan
Muir and Dr. Ning Qiao for their support and feedback on this work.

## Data and Code Availability

The third-party datasets used in this study are available from their
respective authors, cited in the main text. The Python/PyTorch code used for
training and analysis is publicly available at gitlab.com/aiCTX/synoploss.
Reuse and feedback are encouraged, within the terms of the license provided.

## References

  * Bohte et al. (2002) Bohte, S. M., Kok, J. N., and La Poutre, H. (2002).  Error-backpropagation in temporally encoded networks of spiking neurons.  _Neurocomputing_ 48, 17–37
  * Cai et al. (2018) Cai, H., Zhu, L., and Han, S. (2018).  Proxylessnas: Direct neural architecture search on target task and hardware.  _arXiv preprint arXiv:1812.00332_
  * Cao et al. (2015) Cao, Y., Chen, Y., and Khosla, D. (2015).  Spiking deep convolutional neural networks for energy-efficient object recognition.  _International Journal of Computer Vision_ 113, 54–66
  * Crowley et al. (2018) Crowley, E. J., Turner, J., Storkey, A., and O’Boyle, M. (2018).  Pruning neural networks: is it time to nip it in the bud?
  * Davies et al. (2018) Davies, M., Srinivasa, N., Lin, T.-H., Chinya, G., Cao, Y., Choday, S. H., et al. (2018).  Loihi: A neuromorphic manycore processor with on-chip learning.  _IEEE Micro_ 38, 82–99
  * Diehl et al. (2015) Diehl, P. U., Neil, D., Binas, J., Cook, M., Liu, S.-C., and Pfeiffer, M. (2015).  Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing.  In _2015 International Joint Conference on Neural Networks (IJCNN)_ (IEEE), 1–8
  * Esser et al. (2016) Esser, S. K., Merolla, P. A., Arthur, J. V., Cassidy, A. S., Appuswamy, R., Andreopoulos, A., et al. (2016).  Convolutional networks for fast energy-efficient neuromorphic computing.  _Proc. Nat. Acad. Sci. USA_ 113, 11441–11446
  * Furber (2016) Furber, S. (2016).  Large-scale neuromorphic computing systems.  _Journal of neural engineering_ 13, 051001
  * Furber et al. (2014) Furber, S. B., Galluppi, F., Temple, S., and Plana, L. A. (2014).  The spinnaker project.  _Proceedings of the IEEE_ 102, 652–665
  * Guo (2018) Guo, Y. (2018).  A survey on methods and theories of quantized neural networks.  _arXiv preprint arXiv:1808.04752_
  * Hassibi and Stork (1993) Hassibi, B. and Stork, D. G. (1993).  Second order derivatives for network pruning: Optimal brain surgeon.  In _Advances in neural information processing systems_. 164–171
  * Hubara et al. (2017) Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., and Bengio, Y. (2017).  Quantized neural networks: Training neural networks with low precision weights and activations.  _The Journal of Machine Learning Research_ 18, 6869–6898
  * Hunsberger and Eliasmith (2015) Hunsberger, E. and Eliasmith, C. (2015).  Spiking deep networks with LIF neurons.  _arXiv preprint arXiv:1510.08829_
  * Indiveri et al. (2011) Indiveri, G., Linares-Barranco, B., Hamilton, T. J., Van Schaik, A., Etienne-Cummings, R., Delbruck, T., et al. (2011).  Neuromorphic silicon neuron circuits.  _Frontiers in neuroscience_ 5, 73
  * Indiveri and Sandamirskaya (2019) Indiveri, G. and Sandamirskaya, Y. (2019).  The importance of space and time for signal processing in neuromorphic agents: The challenge of developing low-power, autonomous agents that interact with the environment.  _IEEE Signal Processing Magazine_ 36, 16–28
  * Kingma and Ba (2014) Kingma, D. P. and Ba, J. (2014).  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_
  * Krizhevsky et al. (2009) Krizhevsky, A., Hinton, G., et al. (2009).  _Learning multiple layers of features from tiny images_.  Tech. rep., Citeseer
  * LeCun et al. (1998) LeCun, Y., Cortes, C., and Burges, C. J. (1998).  The mnist database of handwritten digits, 1998.  _URL http://yann.lecun.com/exdb/mnist_ 10, 34
  * LeCun et al. (1990) LeCun, Y., Denker, J. S., and Solla, S. A. (1990).  Optimal brain damage.  In _Advances in neural information processing systems_. 598–605
  * Liu et al. (2016) Liu, Q., Pineda-García, G., Stromatias, E., Serrano-Gotarredona, T., and Furber, S. B. (2016).  Benchmarking spike-based visual recognition: a dataset and evaluation.  _Frontiers in neuroscience_ 10, 496
  * Liu et al. (2019) Liu, Q., Richter, O., Nielsen, C., Sheik, S., Indiveri, G., and Qiao, N. (2019).  Live demonstration: Face recognition on an ultra-low power event-driven convolutional neural network asic.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_. 0–0
  * Marblestone et al. (2016) Marblestone, A. H., Wayne, G., and Kording, K. P. (2016).  Toward an integration of deep learning and neuroscience.  _Frontiers in computational neuroscience_ 10, 94
  * Merolla et al. (2014) Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., et al. (2014).  A million spiking-neuron integrated circuit with a scalable communication network and interface.  _Science_ 345, 668–673.  doi:10.1126/science.1254642
  * Molchanov et al. (2016) Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. (2016).  Pruning convolutional neural networks for resource efficient inference.  _arXiv preprint arXiv:1611.06440_
  * Mostafa (2017) Mostafa, H. (2017).  Supervised learning based on temporal coding in spiking neural networks.  _IEEE transactions on neural networks and learning systems_ 29, 3227–3235
  * Neftci et al. (2019) Neftci, E. O., Mostafa, H., and Zenke, F. (2019).  Surrogate gradient learning in spiking neural networks.  _arXiv preprint arXiv:1901.09948_
  * Neil et al. (2016) Neil, D., Pfeiffer, M., and Liu, S.-C. (2016).  Learning to be efficient: Algorithms for training low-latency, low-compute deep spiking neural networks.  In _ACM Symposium on Applied Computing_ (Proceedings of the 31st Annual ACM Symposium on Applied Computing)
  * Nicola and Clopath (2017) Nicola, W. and Clopath, C. (2017).  Supervised learning in spiking neural networks with force training.  _Nature communications_ 8, 2208
  * Panda and Roy (2016) Panda, P. and Roy, K. (2016).  Unsupervised regenerative learning of hierarchical features in spiking deep networks for object recognition.  In _2016 International Joint Conference on Neural Networks (IJCNN)_ (IEEE), 299–306
  * Paszke et al. (2017) Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., et al. (2017).  Automatic differentiation in PyTorch.  In _NIPS Autodiff Workshop_
  * Richards et al. (2019) Richards, B. A., Lillicrap, T. P., Beaudoin, P., Bengio, Y., Bogacz, R., Christensen, A., et al. (2019).  A deep learning framework for neuroscience.  _Nature neuroscience_ 22, 1761–1770
  * Rueckauer et al. (2017) Rueckauer, B., Lungu, I.-A., Hu, Y., Pfeiffer, M., and Liu, S.-C. (2017).  Conversion of continuous-valued deep networks to efficient event-driven networks for image classification.  _Frontiers in neuroscience_ 11, 682
  * Sengupta et al. (2019) Sengupta, A., Ye, Y., Wang, R., Liu, C., and Roy, K. (2019).  Going deeper in spiking neural networks: VGG and residual architectures.  _Frontiers in neuroscience_ 13
  * Serrano-Gotarredona and Linares-Barranco (2015) Serrano-Gotarredona, T. and Linares-Barranco, B. (2015).  Poker-dvs and mnist-dvs. their history, how they were made, and other details.  _Frontiers in neuroscience_ 9, 481
  * Shrestha and Orchard (2018) Shrestha, S. B. and Orchard, G. (2018).  Slayer: Spike layer error reassignment in time.  In _Advances in Neural Information Processing Systems_. 1412–1421
  * Springenberg et al. (2014) Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (2014).  Striving for simplicity: The all convolutional net.  _arXiv preprint arXiv:1412.6806_
  * Strubell et al. (2019) Strubell, E., Ganesh, A., and McCallum, A. (2019).  Energy and policy considerations for deep learning in nlp.  _arXiv preprint arXiv:1906.02243_
  * Thakur et al. (2018) Thakur, C. S. T., Molin, J., Cauwenberghs, G., Indiveri, G., Kumar, K., Qiao, N., et al. (2018).  Large-scale neuromorphic spiking array processors: A quest to mimic the brain.  _Frontiers in neuroscience_ 12, 891
  * Yang et al. (2017) Yang, T.-J., Chen, Y.-H., and Sze, V. (2017).  Designing energy-efficient convolutional neural networks using energy-aware pruning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_. 5687–5695