appendAppendices References

# Increasing Liquid State Machine Performance with Edge-of-Chaos Dynamics
Organized by Astrocyte-modulated Plasticity

Vladimir A. Ivanov
Department of Computer Science
Rutgers University
Piscataway, NJ
<EMAIL_ADDRESS>
Konstantinos P. Michmizos
Department of Computer Science
Rutgers University
Piscataway, NJ
<EMAIL_ADDRESS>

###### Abstract

The liquid state machine (LSM) combines low training complexity and biological
plausibility, which has made it an attractive machine learning framework for
edge and neuromorphic computing paradigms. Originally proposed as a model of
brain computation, the LSM tunes its internal weights without backpropagation
of gradients, which results in lower performance compared to multi-layer
neural networks. Recent findings in neuroscience suggest that astrocytes, a
long-neglected non-neuronal brain cell, modulate synaptic plasticity and brain
dynamics, tuning brain networks to the vicinity of the computationally optimal
critical phase transition between order and chaos. Inspired by this disruptive
understanding of how brain networks self-tune, we propose the neuron-astrocyte
liquid state machine (NALSM)111Code and data available at
https://github.com/combra-lab/NALSM that addresses under-performance through
self-organized near-critical dynamics. Similar to its biological counterpart,
the astrocyte model integrates neuronal activity and provides global feedback
to spike-timing-dependent plasticity (STDP), which self-organizes NALSM
dynamics around a critical branching factor that is associated with the edge-
of-chaos. We demonstrate that NALSM achieves state-of-the-art accuracy versus
comparable LSM methods, without the need for data-specific hand-tuning. With a
top accuracy of $97.61\%$ on MNIST, $97.51\%$ on N-MNIST, and $85.84\%$ on
Fashion-MNIST, NALSM achieved comparable performance to current fully-
connected multi-layer spiking neural networks trained via backpropagation. Our
findings suggest that the further development of brain-inspired machine
learning methods has the potential to reach the performance of deep learning,
with the added benefits of supporting robust and energy-efficient neuromorphic
computing on the edge.

##  1 Introduction

With the recent rise of neuromorphic [1, 2, 3, 4] and edge computing [5, 6],
the liquid state machine (LSM) learning framework [7] has become an attractive
alternative [8, 9, 10, 11] to deep neural networks owing to its compatibility
with energy-efficient neuromorphic hardware [12, 13, 14] and inherently low
training complexity. Originally proposed as a biologically plausible model of
learning, LSMs avoid training via backpropagation by using a sparse,
recurrent, spiking neural network (liquid) with fixed synaptic connection
weights to project inputs into a high dimensional space from which a single
neural layer can learn the correct outputs. Yet, these advantages over deep
networks come at the expense of 1) sub-par accuracy and 2) extensive data-
specific hand-tuning of liquid weights. Interestingly, these two limitations
have been targeted by several studies that tackle one [15, 16] or the other
[17, 18], but not both. This has limited the widespread use of LSMs in real-
world applications [8]. In that sense, there is an unmet need for a unified,
brain-inspired approach that is directly applicable to the emerging
neuromorphic and edge computing technologies, facilitating them to go
mainstream.

As a general heuristic, LSM accuracy is maximized when LSM dynamics are
positioned at the edge-of-chaos [19, 20, 21] and specifically in the vicinity
of a critical phase transition [22, 23, 24, 25] that separates: 1) the sub-
critical phase, where network activity decays, and 2) the super-critical
(chaotic) phase, where network activity gets exponentially amplified.
Strikingly, brain networks have also been found to operate near a critical
phase transition [26, 27, 28] that is modeled as a branching process [25, 26].
Current LSM tuning methods organize network dynamics at the critical branching
factor by adding forward and backward communication channels on top of the
liquid [15, 16]. This, however, results in significant increases in training
complexity and violates the LSM’s brain-inspired self-organization principles.
For example, these methods lack local plasticity rules that are widely
observed in the brain and considered a key component for both biological [29]
and neuromorphic learning [3, 2, 4]. A particular local learning rule, spike-
timing-dependent plasticity (STDP), is known to improve LSM accuracy [17, 18].
Yet, current methods of incorporating STDP into LSMs further exacerbate the
limitations of data-specific hand-tuning as they require additional mechanisms
to compensate for the STDP-imposed saturation of synaptic weights [17, 30, 31,
32, 33]. This signifies the scarcity of LSM tuning methods that are both
computationally efficient and data-independent.

A long-neglected non-neuronal cell in the brain, astrocytes, is now known to
play key roles in modulating brain networks [34, 35, 36, 37, 38, 39], from
modifying synaptic plasticity [40, 41, 42] to facilitating switching between
cognitive states [43, 44, 45, 46] that have been linked to a narrow spectrum
of dynamics around the critical phase transition [47, 48, 49, 50, 51]. The
mechanisms that astrocytes use to modulate neurons include the integration of
the activity of thousands of synapses into a slow intracellular continuous
signal that feeds back to neurons by affecting their synaptic plasticity [52,
53, 54, 55, 42]. The unique spatio-temporal attributes [56, 57] identified in
astrocytes align well with the brain’s remarkable ability to self-organize its
massive and highly recursive networks near criticality. That is why astrocytes
present a fascinating possibility of forming a unified feedback modulation
mechanism required to improve baseline LSM accuracy while eliminating data-
specific hand-tuning.

Here, we propose the neuron-astrocyte liquid state machine (NALSM), where a
biologically inspired astrocyte model organized liquid dynamics near a
critical phase transition, by modulating STDP. We show that NALSM combined the
computational benefits of both STDP and critical branching dynamics by
demonstrating its accuracy advantage compared to other LSM methods on two
datasets: 1) MNIST [58], and 2) N-MNIST [59]. We demonstrate that, similar to
its biological counterpart that handles new and unstructured information with
robustness and versatility, NALSM maintains the state-of-the-art LSM
performance without re-tuning training parameters for each tested dataset. We
also show that a NALSM with a large enough liquid can attain comparable
accuracy to fully-connected multi-layer spiking neural networks trained via
backpropagation on 1) MNIST [58], 2) N-MNIST [59], as well as 3) Fashion-MNIST
[60]. Our results suggest that the under-performance and high training
difficulty of current neuromorphic methods can be addressed by harvesting
neuroscience knowledge and further translating biological principles to
computational mechanisms.

##  2 Methods

###  2.1 The neuron-astrocyte liquid state machine

To construct the NALSM, we started with a baseline LSM model consisting of 2
layers: 1) a spiking liquid, and 2) a linear output layer. Next, we added STDP
to the LSM liquid, forming the LSM+STDP model. We developed a biologically
faithful leaky-integrate-and-modulate (LIM) astrocyte model, which we embedded
in the LSM+STDP liquid, to form the NALSM. The process is formalized below.

##### LSM Model

We implemented the baseline LSM as a 3-dimensional neural network (liquid)
consisting of $1,000$ neurons surrounded by 1-dimensional layers of input and
output neurons. Number of input neurons was $784$ and $2,312$ for MNIST and
N-MNIST, respectively (See Appendix A.1). We used the leaky-integrate-and-fire
(LIF) model [3] for input and liquid neurons, modeled as:

$\frac{dv_{i}}{dt}=-\frac{1}{\tau_{v}}v_{i}(t)+u_{i}(t)-\theta_{i}\sigma_{i}(t)\\\
$ (1) $u_{i}(t)=\sum_{j\neq i}w_{ij}(\alpha_{u}*\sigma_{j})(t)+b_{i}\\\ $ (2)

where $v_{i}$ is the membrane potential and $u_{i}$ is the synaptic response
current of neuron $i$, $\theta_{i}$ is the membrane potential threshold,
$\sigma_{i}(t)=\sum_{k}\delta(t-t_{i}^{k})$ is the spike train of neuron $i$
with $t_{i}^{k}$ being the time of the $k$-th spike, $w_{ij}$ is the weight
connecting neuron $j$ to $i$, $b_{i}$ is the bias of neuron $i$, and
$\alpha_{u}(t)=\tau_{u}^{-1}\exp(-t/\tau_{u})H(t)$ is the synaptic filter with
H(t) being the unit step function (See Appendix A.2). All LIF neurons had a
$2$ $ms$ absolute refractory period. Liquid neurons were excitatory and
inhibitory with $80\%/20\%$ ratio. Input neurons did not have an
excitatory/inhibitory distinction and had random excitatory and inhibitory
connections to liquid neurons. From here on, we will refer to connections
between input neurons and liquid neurons as IL connections, inter-liquid
connections as LL, and liquid to output connections as LO. In line with [7],
we created LL connections using probabilities based on Euclidean distance,
$D(i,j)$, between any two neurons $i$, $j$:

$P(i,j)=C\cdot exp\left(-\left(\frac{D(i,j)}{\lambda}\right)^{2}\right)\\\ $
(3)

with closer neurons having higher connection probability. Parameters $C$ and
$\lambda$ set the amplitude and horizontal shift, respectively, of the
probability distribution (See Appendix A.3). Density of IL connections was
15%. The output layer was a dense layer consisting of $10$ linear neurons.

##### LSM+STDP Model

We added unsupervised, local learning to the LSM model by letting STDP change
each LL and IL connection [61], modeled as:

$\frac{dw}{dt}=A_{+}T_{pre}\sum_{o}\delta(t-t^{o}_{post})-A_{-}T_{post}\sum_{i}\delta(t-t^{i}_{pre})\\\
$ (4)

where $A_{+}=A_{-}=0.15$ are the potentiation/depression learning rates and
$T_{pre}$/$T_{post}$ are the pre/post-synaptic trace variables, modeled as,

$\tau_{+}^{*}\frac{dT_{pre}}{dt}=-T_{pre}+a_{+}\sum_{i}\delta(t-t_{pre}^{i})\\\
$ (5)
$\tau_{-}^{*}\frac{dT_{post}}{dt}=-T_{post}+a_{-}\sum_{o}\delta(t-t_{post}^{o})\\\
$ (6)

where $a_{+}=a_{-}=0.1$ are the discrete contributions of each spike to the
trace variable, $\tau_{+}^{*}=\tau_{-}^{*}=10$ $ms$ are the decay time
constants, $t_{pre}^{i}$ and $t_{post}^{o}$ are the times of the pre-synaptic
and post-synaptic spikes, respectively. We constrained connection weights to:
1) IL: $[-3,3]$ , 2) excitatory LL: $[0,3]$, and 3) inhibitory LL: $[-3,0]$.
We used the same STDP parameters for all models and experiments.

##### LIM Astrocyte Model

We developed the astrocyte model as a leaky integrator with a continuous
output value $A_{-}^{astro}$, expressed as:

$\tau_{asto}\frac{dA_{-}^{astro}}{dt}=-A_{-}^{astro}+w_{astro}\sum_{i\in
N_{liq}}\delta(t-t_{i})-w_{astro}\sum_{j\in
N_{inp}}\delta(t-t_{j})+b_{astro}\\\ $ (7)

where $A_{-}^{astro}$ directly mapped to $A_{-}$ in equation (4),
$b_{astro}=A_{+}$ adjusted the astrocyte output to the fixed STDP potentiation
learning rate, $N_{liq}$ and $N_{inp}$ are the sets of liquid and input
neurons, respectively, and $w_{astro}$ set astrocyte responsiveness to network
activity (See Appendix A.4). Ignoring the decay and bias terms, the astrocyte
model computed the difference in the number of spikes produced by liquid
neurons and input neurons. Functionally, this is equivalent to computing the
ratio of spikes emitted by the liquid over the input neurons:

$BF_{proxy}(t)=\frac{\sum_{i\in N_{liq}}\delta(t-t_{i})}{\sum_{j\in
N_{inp}}\delta(t-t_{j})}\\\ $ (8)

Specifically, when the liquid produced more spikes than the input neurons,
their difference was positive which translated to $BF_{proxy}>1.0$, and vice
versa. This approach to measure liquid dynamics acted as a network level
approximation of the branching factor, $\sigma_{BF}$, which is normally
evaluated for each neuron (See 2.3.1). We empirically confirmed that
$BF_{proxy}=1.0$ aligned with the critical branching factor, $\sigma_{BF}=1.0$
(Fig. 1 B). Hence, as dynamics became progressively super-critical
($\sigma_{BF}>1.0$), $BF_{proxy}$ became greater than $1$, which caused the
LIM astrocyte to increase STDP depression learning rate above the fixed STDP
potentiation learning rate ($A_{-}^{astro}\rightarrow A_{-}>A_{+}$). This
caused STDP to decrease the average weight of LL and IL connections, which
decreased number of spikes produced by the liquid and made dynamics less
super-critical (Fig. 1 B). The reverse occurred as dynamics became
progressively sub-critical. As a result of astrocyte modulation, liquid
dynamics oscillated between sub-critical and super-critical until eventual
stabilization near the critical branching factor (See Appendix A.4).

##### NALSM Model

We completed NALSM by adding the LIM astrocyte to the LSM+STDP model’s liquid
(Fig. 1 A). As described above, the LIM astrocyte integrated activity from
input and liquid neurons, and continuously controlled the STDP depression
learning rate.

Figure 1: NALSM architecture and astrocyte modulation of liquid dynamics. ( A
) The neuron-astrocyte liquid was modeled as a 3-dimensional network of
excitatory and inhibitory spiking neurons connected with sparse, recurrent
connections with spike-timing-dependent plasticity (STDP). Input neurons
projected excitatory and inhibitory connections to the liquid. Receiving each
liquid neuron’s spike count per input sample, a dense linear output layer was
trained via gradient descent to classify inputs. ( B ) To organize liquid
dynamics at the critical branching factor, an astrocyte integrated input and
liquid neuron activity and, in turn, set the global STDP depression learning
rate. Data points are binned averages over ‘BF approximation’ metric. Error
bars are standard deviation. See Appendix A.10 for polynomial fits.

###  2.2 Training

Model training was done in $3$ steps: 1) initialization of IL and LL liquid
connections, 2) passing all data through the liquid resulting in liquid neuron
spike counts, and 3) training the output layer on the spike counts. The steps
are further detailed below.

####  2.2.1 Liquid initialization

##### LSM

We initialized all IL and LL connections with a single weight value,
maintaining originally defined connection signs [62]. Weights were constant
during spike count collection.

##### LSM+STDP

We initialized IL and LL connections with STDP by consecutively presenting all
the training images to the liquid. Starting with initially maximal
connections, $3(-3)$ for excitatory(inhibitory) connections, we let STDP
continuously adjusted weights while presenting the liquid with a randomly
ordered series of MNIST training image snapshots, each lasting $20$ $ms$. For
N-MNIST, we randomly sampled each $20$ $ms$ snapshot from the $0-250$ $ms$
range, as a way to account for the variability in the temporal dimension (See
2.3). In each case, we used a total of $50,000$ snapshots, each corresponding
to a unique training image. We used STDP only for weight initialization.
Initialized weights were fixed during spike count collection.

##### NALSM

We used the LSM+STDP weight initialization process with the exception of added
STDP modulation by the LIM astrocyte. We used this set of initialized weights
as the starting point for each sample in the spike counting phase, during
which astrocyte-modulated STDP continued to adjust synaptic weights to
compensate for slight deviations in dynamics caused by each input sample’s
different level of activity. For each sample, parameters $A_{+}$ from (4) and
$b_{astro}$ from (7) were both initialized to $0.15$ and decayed at a rate of
$0.99$ for the duration of sample input.

####  2.2.2 Output layer training

We assembled spike counts by presenting each sample image to the liquid for
$250$ $ms$ and counting the number of spikes emitted by each liquid neuron for
the full duration of input. We used Adam optimizer to batch train the output
layer on spike count vectors by minimizing the cross entropy loss with
L2-regularization,

$\mathcal{L}(y_{i},\hat{y})=-\frac{1}{m}\sum^{m}_{i=1}y_{i}log(\hat{y}_{i})+(1-y_{i})log(1-\hat{y}_{i})+\frac{\lambda_{reg}}{2m}||W_{out}||^{2}_{F}$
(9)

where $m=250$ is the batch size, $W_{out}$ is the output layer weight matrix,
$\lambda_{reg}=5\times 10^{-10}$ is the regularization hyperparameter, $y_{i}$
and $\hat{y}_{i}$ are the normalized vectors denoting the predicted label and
the target label, respectively. Prior to training, we initialized output layer
weights/biases to $0.0$, and the learning rate to $0.1$. We trained the output
layer until validation accuracy peaked (up to a maximum of $5,000$ epochs), at
which point we evaluated model test accuracy.

###  2.3 Experiments

We performed all LSM comparison experiments on MNIST and N-MNIST datasets (See
A.1). Using 10 randomly generated networks for each dataset, we trained 1) the
baseline LSM model, 2) the LSM+STDP model, 3) the NALSM model (See 2.1), and
4) the LSM+AP-STDP model, a method for incorporating STDP in the LSM liquid
[17] (See Appendix A.5). First, we evaluated LSM model accuracy with respect
to liquid weight, which ranged in $0.4-1.2$ for MNIST and $0.8-1.35$ for
N-MNIST. We used a random seed for each training session. Next, we evaluated
corresponding network dynamics of each network/weight combination by measuring
the liquid’s branching factor on 20 randomly sampled inputs (See 2.3.1). To
have comparable results for each network, we trained the remaining models
using the same seed that resulted in peak LSM accuracy. For NALSM, we used the
same initialization and parameters for all networks and datasets. For LSM+AP-
STDP, we hand-tuned STDP control parameters for each network and dataset
combination to maximize validation accuracy (See Appendix A.5). Additionally,
we trained NALSM on Fashion-MNIST dataset (See A.1) using the same 10 randomly
generated networks that we had used for MNIST.

##### Sparse neuron-astrocyte connectivity

We tested NALSM’s accuracy as a function of neuron-astrocyte connection
density on $3$ best performing networks (per dataset). Keeping the proportion
of neurons sampled by the astrocyte the same for both input neurons and liquid
neurons, we trained NALSM with $10\%$, $20\%$, $40\%$, $60\%$, and $80\%$
neuron-astrocyte density over $3$ seeds for each of $3$ networks. Regardless
of connection sparseness, all IL/LL connections were modulated by
$A_{-}^{astro}$ (See 2.1)

##### NALSM with larger liquid sizes

We tested NALSM performance for larger liquids. For each size, we trained $3$
randomly generated networks, each on a random seed. All parameters and
initialization were same as for $1,000$ neuron liquid. For maximum accuracy,
we trained NALSM with an $8,000$ neuron liquid. For each dataset, we used $5$
randomly generated networks trained on a random seed. Parameter
$w_{astro}=0.0075$ for all datasets. All other parameters and initialization
were as before.

####  2.3.1 Branching factor of liquid

To evaluate a liquid’s dynamics, we used the network branching factor,
$\sigma_{BF}$, which quantifies network information flow amplification/decay.
Liquid dynamics are sub-critical, near-critical and super-critical when
$\sigma_{BF}<1.0$, $\sigma_{BF}\approx 1.0$, and $\sigma_{BF}>1.0$. We
calculated $\sigma_{BF}$ as done in [30], with offset $\phi=0$ and time window
$\Delta=4$ $ms$ as per [26].

####  2.3.2 Kernel quality of liquid

We evaluated liquid 1) linear separation, and 2) generalization capability
using methods from [63]. For the MNIST and N-MNIST test sets, we computed the
rank of matrix $M$ assembled from $k$ randomly selected spike count vectors,
resulting in shape $N_{liq}\times k$. We repeated this on $1,000$ shuffles of
spike vectors. For linear separation, we used spike counts from model testing
phase. For generalization capability, we added noise to input data and
evaluated new spike count vectors (See 2.2.2). For MNIST, we added
$\mathcal{N}(0,125)$ noise to each pixel value. For N-MNIST, we time-shifted
each event by $\mathcal{N}(0,10)$. Taken together across all models and both
datasets, we rescaled ranks of each measure to $0-1$ range and subtracted
measure $1$ from measure $2$ as in [63]. Due to negative differences, we again
rescaled all differences to $0-1$.

Figure 2: LSM accuracy depended on liquid weight and dynamics. ( A ) LSM
accuracy shown as a function of its liquid weight, averaged over a set of $10$
randomly generated networks for MNIST and N-MNIST datasets. ( B ) LSM accuracy
shown with respect to liquid dynamics set by liquid synaptic weight. For each
weight, liquid dynamics were measured and averaged over all $10$ networks.
Similarly, accuracy and resulting liquid dynamics are shown for each model: 1)
NALSM, 2) LSM+AP-STDP, and 3) LSM+STDP. ( C ) Liquid dynamics shown with
respect to liquid weight, averaged over all $10$ networks. Error bars are
standard deviation. See Appendix A.10 for polynomial fits.

##  3 Results

###  3.1 Baseline LSM performance

We established a benchmark accuracy for the baseline LSM on MNIST and N-MNIST
datasets (See 2.3). We acquired our baseline by averaging over 10 randomly
generated liquids with $1,000$ neurons (See 2.1). The LSM achieved a top
accuracy of $95.44\%$ ($95.30\pm 0.11\%$) on MNIST, and $95.35\%$ ($95.02\pm
0.15\%$) on N-MNIST (See 2.2). For MNIST, this was comparable to the
previously reported state-of-the-art LSM accuracy [64], using the same sized
liquid. Further, LSM accuracy was very sensitive to the liquid’s weight (Fig.
2 A).

###  3.2 LSM performance peaked at the critical branching factor

The peak LSM accuracy on each dataset corresponded to a different liquid
synaptic weight. Specifically, there were cases where a liquid with weights
tuned for maximum accuracy on MNIST, would catastrophically fail on N-MNIST
(Fig. 2 A). Also, LSM accuracy on MNIST plateaued for a wider range of weights
than on N-MNIST, which can be attributed to N-MNIST’s greater difficulty
caused by its variability over the temporal dimension (See Appendix A.1).
Taken together, this indicated that LSM training requires extensive hand-
tuning of weights for each specific dataset.

Since critical dynamics are well known to result in near-maximum LSM
performance [65, 19, 20, 21], dataset-specific hand-tuning can be
significantly reduced by replacing accuracy with liquid dynamics as the target
output of weight tuning. Indeed, LSM accuracy was near-maximum for both
datasets, when the liquid’s branching factor was in $1.0-1.2$ range, or
slightly super-critical (Fig. 2 B) (See 2.3.1). This agrees with studies
showing that information transfer in finite sized systems peaks at slightly
super-critical dynamics [66]. Although each dataset still had different weight
ranges corresponding to the critical branching factor, the relationship
between liquid dynamics and weight was positive for both datasets (Fig. 2 C).
Known to generalize beyond specific datasets [20], this relationship suggested
that near-critical dynamics can be organized using STDP, by providing it
directional feedback from current liquid dynamics.

###  3.3 Astrocyte-modulated plasticity organized liquid dynamics near
criticality

The LIM astrocyte model stabilized liquid dynamics near the critical branching
factor as we presented a continuous stream of samples to the neuron-astrocyte
liquid (Fig. 2 B). The NALSM’s slightly super-critical stabilization suggested
that liquid dynamics were at the edge-of-chaos. While chaotic activity is
known to correspond to super-critical branching dynamics in some cases [25],
such correspondence is not guaranteed. Hence, we examined additional network
properties that are necessary and indicative of chaotic activity (See Appendix
A.6). Specifically, the astrocyte-modulated liquid had coexistence of small
and large synaptic weights (Fig. S2), as well as a balance of excitation and
inhibition, both of which are necessary for the existence of chaotic network
activity [67, 68]. Further supporting a chaotic activity, the neuron-astrocyte
liquid spike activity appeared irregular (Fig. S3). We also performed
autocorrelation analysis on liquid neuron spike trains, which further
suggested the existence of chaotic activity with a correspondence between
edge-of-chaos dynamics and critical branching dynamics (Fig. S4) [69, 70] (See
Appendix A.6). Given that liquid dynamics were directly approximated by the
LIM astrocyte, which directly controlled the STDP depression learning rate
(See 2.1), NALSM required no dataset-specific hand-tuning. As a result, we
used the same weight initialization and parameters to benchmark NALSM (See
2.3).

Figure 3: Comparison of model accuracy and liquid computational capacity. ( A
) Accuracy performance of the proposed NALSM model was compared, on MNIST and
N-MNIST, against $3$ related models: 1) the baseline LSM, 2) LSM with
activity-based STDP (LSM+AP-STDP), and 3) LSM with unregulated STDP
(LSM+STDP). For each dataset and model, accuracy was evaluated using $10$
randomly generated networks, each of which was trained on a random seed. This
set of $10$ seeds was used for all models. ( B ) Computational capacity of
each model was measured using a kernel quality metric that encompassed the
linear separation and generalization capability of the liquid (See 2.3.2).
Error bars are standard deviation.  Figure 4: NALSM maintains accuracy
advantage with sparse neuron-astrocyte connectivity. For MNIST and N-MNIST,
NALSM accuracy was evaluated with respect to neuron-astrocyte connection
density. For each density, NALSM performance was compared to LSM and LSM+AP-
STDP average accuracy. NALSM data points are average values over $9$
experiments ($3$ networks $\times$ $3$ seeds). Error bars and shaded areas are
standard deviation. See Appendix A.10 for polynomial fits.

###  3.4 Benchmarking NALSM performance on MNIST and N-MNIST

On both datasets, NALSM achieved superior performance to comparable LSM models
of the same size. Using $1,000$ liquid neurons, NALSM achieved a top accuracy
of $96.15\%$ ($95.96\pm 0.13\%$) on MNIST and $96.13\%$ ($95.90\pm 0.16\%$) on
N-MNIST; outperforming LSM model’s top accuracy by $0.71\%$ on MNIST and
$0.78\%$ on N-MNIST (Fig. 3 A). We also compared NALSM to a state-of-the-art
LSM STDP method, AP-STDP (See 2.3). The LSM+AP-STDP model required more
extensive dataset-specific hand-tuning than the baseline LSM due to its
additional STDP control parameters (See Appendix A.5). Resulting in top
accuracy of $95.62\%$ ($95.49\pm 0.09\%$) and $95.43\%$ ($95.23\pm 0.16\%$),
the LSM+AP-STDP model was superseded by NALSM by $0.53\%$ and $0.70\%$ on
MNIST and N-MNIST, respectively. As a control measure, we also trained a LSM
with unregulated STDP (See 2.1). The LSM+STDP model significantly under-
performed compared to all other models achieving a top accuracy of $90.52\%$
($89.47\pm 0.45\%$) on MNIST and $87.71\%$ ($86.80\pm 0.51\%$) on N-MNIST. We
attributed this under-performance to the LSM+STDP liquid’s excessive super-
critical dynamics (Fig. 2 B) which are well known to decrease liquid
computational capacity [65, 66].

The NALSM had the most robust accuracy performance across the two datasets out
of all the compared LSM models. With no dataset-specific tuning, NALSM’s
average accuracy on N-MNIST was lower than the accuracy on MNIST by only
$-0.05\%$. This was $5-50$ times less than for the LSM+AP-STDP ($-0.26\%$),
LSM ($-0.29\%$), and LSM+STDP ($-2.66\%$) models.

We attributed the NALSM’s performance advantage to the improved computational
properties of its liquid. For both tested datasets, the NALSM achieved
slightly super-critical branching dynamics where baseline LSM performance
peaked right before it started to decline with increasing super-critical
dynamics (Fig. 2 B). This suggested that NALSM’s performance advantage,
compared to a LSM with similar dynamics, was due to the addition of astrocyte-
modulated STDP (See 2.1, 2.2.1). While LSM+AP-STDP and LSM+STDP models also
had STDP, their lower performance can be explained by their excessively sub-
critical and super-critical dynamics, respectively (Fig. 2 B). We further
confirmed that NALSM’s increased performance resulted from the improved
computational properties of its liquid by measuring each model’s liquid kernel
quality. This encompassed both the linear separation and generalization
capability of the liquid (See 2.3.2). Higher model accuracy corresponded to
higher kernel quality for all $4$ models (Fig. 3 B). This is a further
indication that near-critical dynamics and astrocyte-modulated STDP
contributed to the NALSM’s performance increase.

###  3.5 NALSM maintained performance with sparse neuron-astrocyte
connectivity

The NALSM maintained its accuracy advantage even with neuron-astrocyte
connection densities as low as $10\%$ (Fig. 4). In the brain, astrocytes
contact only approximately $65\%$ of all synapses in their surroundings [71].
We tested NALSM performance as a function of neuron-astrocyte connection
density (See 2.3). The NALSM mean accuracy decreased marginally with
increasingly sparse connectivity, while variability in performance was minimal
across densities. At 10% connectivity, average NALSM accuracy decreased by
$0.36\%$ for MNIST and $0.14\%$ for N-MNIST compared to $100\%$ connection
density. In both cases, average NALSM accuracy was still above LSM and
LSM+APSTDP average accuracy.

###  3.6 Larger liquids increased NALSM accuracy

The NALSM accuracy improved with increased liquid size, saturating at
approximately $8,000$ neurons (See Appendix A.7). NALSM$8000$ achieved a top
accuracy of $97.61\%$ ($97.49\pm 0.11\%$) on MNIST, $97.51\%$ ($97.42\pm
0.07\%$) on N-MNIST, and $85.84\%$ ($85.61\pm 0.18\%$) on Fashion-MNIST.
Compared to previously reported benchmarks on MNIST and Fashion-MNIST, the
NALSM$8000$ outperformed all brain-inspired learning methods that do not use
backpropagation of gradients or its approximation through feedback alignment
[72, 73], with the exception of [74] for MNIST. While [74] demonstrated that a
fully-connected 2-layer spiking network can achieve high accuracy through a
combination of biologically-plausible plasticity rules, it is not clear how
such an approach would scale to more layers without some form of
backpropagation. Conversely, multi-layered LSMs have been shown to work
without backpropagation [8, 10]. Further, NALSM$8000$ used approximately $1/3$
($\approx 1,199,407\pm 453$) of number of trainable(plastic) connections as in
[74]. Compared to top accuracies reported for fully-connected multi-layered
spiking neural networks trained with backpropagation, the NALSM$8000$ achieved
comparable performance on all datasets; outperforming multiple reported
results on MNIST and N-MNIST (Table 1) (See Appendix A.8).

Table 1: Comparison to brain-inspired and fully-connected multi-layer spiking neural networks. Model | Layers | Learning Method | Accuracy
---|---|---|---
Dataset: MNIST
Unsupervised-SNN [75] | 2 | STDP | $95\%$
Multi-liquid LSM [64] | 2 | GD on last layer | $95.5\%$
NALSM1000 | 2 | astro-STDP, GD on last layer | 96.15%
LIF-BA [73] | 3 | Broadcast feedback alignment | $97.09\%$
Temporal SNN [76] | 2 | Temporal backpropagation | $97.2\%$
STiDi-BP [77] | 2 | Backpropagation | $97.4\%$
NALSM8000 | 2 | astro-STDP, GD on last layer | 97.61%
SN [78] | 3 | Backpropagation | $97.93\%$
GLSNN [72] | 4 | Global feedback alignment, STDP | $98.62\%$
Balance-SNN [74] | 2 | Equi-prop, STDP, STP | $98.64\%$
BPSNN [79] | 3 | Backpropagation | $98.88\%$
STBP [80] | 2 | Spatial and temporal backpropagation | $98.89\%$
Dataset: N-MNIST
DECOLLE [81] | 3 | Backpropagation | $96\%$
NALSM1000 | 2 | astro-STDP, GD on last layer | 96.13%
AER-SNN [82] | 2 | Backpropagation | $96.3\%$
NALSM8000 | 2 | astro-STDP, GD on last layer | 97.51%
BPSNN [79] | 3 | Backpropagation | $98.74\%$
STBP [80] | 2 | Spatial and temporal backpropagation | $98.78\%$
SLAYER [83] | 3 | Backpropagation | $98.89\%$
Dataset: Fashion-MNIST
VPSNN [84] | 2 | Equi-prop, STDP | $82.69\%$
NALSM1000 | 2 | astro-STDP, GD on last layer | 83.54%
Unsupervised-SNN [85] | 2 | STDP | $85.31\%$
NALSM8000 | 2 | astro-STDP, GD on last layer | 85.84%
BS4NN [86] | 2 | Temporal backpropagation | $87.3\%$
GLSNN [72] | 4 | Global feedback alignment, STDP | $89.05\%$
*GD: gradient descent |  |  |

##  4 Discussion and Broader Impact

Ironically, LSMs are one of the most brain-like and at the same time one of
the most difficult to train learning models. Here, we proposed an astrocyte
model that merged critical branching dynamics and STDP into a single liquid,
thereby simultaneously improving LSM performance and decreasing data-specific
tuning. We showed that the synergy of STDP and near-critical branching
dynamics improved the computational capacity of the liquid, which translated
to better than state-of-the-art LSM accuracy on MNIST and N-MNIST, and do so
with minimal added computational cost (See Appendix A.9). Our results indicate
that, given a large enough liquid, NALSM performance compares to current
fully-connected multi-layer spiking neural networks trained via
backpropagation.

The reported narrowing of the performance gap between brain-inspired LSM and
deep networks suggests that studying the interaction among the brain’s
computational principles can help our learning models to reach human-like
performance. Indeed, our results demonstrate that the synergy of brain-
inspired astrocyte-modulated STDP and near-critical dynamics resulted in the
superior performance of NALSM compared to 1) a LSM with critical dynamics but
without STDP, and 2) a LSM with STDP, but without critical dynamics. Aligning
with other studies showing that liquid topology impacts LSM accuracy [87], we
also showed that a brain-inspired, sparse, 3D-distance-based network
architecture can improve the computational capacity of a single liquid.
Specifically, our baseline 3D LSM achieved comparable accuracy to the multi-
liquid LSM [64], which improved performance of a single dimensionless liquid
by partitioning it into multiple liquids. While we demonstrated NALSM
performance using only the 3D-distance-based network architecture, our
proposed astrocyte modulation method does not depend on network topology and,
therefore, is applicable to other types of topology. In fact, our approach is
also extendable to multi-liquid architectures and other local plasticity rules
that follow STDP’s separation of potentiation and depression components.

The astrocyte-modulated LSM learning framework is also compatible with the
emerging neuromorphic hardware. This is because the gradient descent that we
used for training the linear output can be replaced by a single-layer spike
based learning rule [88, 89, 90]. This makes NALSM compatible with
neuromorphic hardware, exploiting in full its advantages [91]. For example,
our method can leverage even further the energy efficiency of neuromorphic
chips, by virtue of its low spiking rates. In line with biological ranges
[92], NALSM had spiking rates that ranged from $12$ $Hz$ to $37$ $Hz$,
depending on the input sample. These rates can be reduced further, by
modifying input encoding, since liquid spiking rates are directly adjusted by
the astrocyte based on input spiking rates (Fig. 1).

Here, we demonstrated a possible connection between the near-critical
branching dynamics of the NALSM liquid and the edge-of-chaos transition (See
Appendix A.6). The critical branching transition has been extensively used to
model critical dynamics in brain networks [27, 26]. Focusing on the
computational benefits of criticality, machine learning has mostly examined
network dynamics at the edge-of-chaos transition. Although the presence of one
transition does not guarantee the existence of the other, both transitions are
well connected to the same result, an improved computational performance [25].
Indeed, the computational performance of systems poised at a critical phase
transition has been widely studied both experimentally [22] and theoretically
[23], and are well-connected to both edge-of-chaos [19, 20] and critical
branching transitions [25, 15]. Networks operating at near-criticality are
believed to have simultaneous access to the computational properties (learning
and memory) of both phases, which results in 1) maximizing their information
processing capacity [22], 2) optimizing their dynamical range [93, 24], and 3)
expanding their number of metastable states [25]. Hence, it is not surprising
that the NALSM’s astrocyte imposed near-critical branching dynamics resulted
in improved accuracy and generalization capabilities as observed in LSMs with
edge-of-chaos dynamics [19, 20], while adding the benefit of a neuromorphic
compatibility and self-organized criticality.

Our work shows how insights from modern cellular neuroscience can synergize
with neuromorphic computing, and lead to novel intelligent systems, spurring
the dialogue between artificial intelligence and brain sciences. Indeed, given
that the known neuronal mechanisms are too slow and uncoordinated in the brain
to modulate STDP [31, 94, 32], it is an open question how neurons modulate
synaptic plasticity. Our demonstration that the distinct temporal and spatial
mechanisms of astrocytes may modulate STDP and subsequently regulate network
dynamics, questions the neuron as the only processing unit in the brain [95,
96, 97]. In that sense, it helps in dismantling the 100-year old dogma that
“brain = neurons”, and tackle the absence of astrocytes in both prevailing
computational hypotheses on how the brain learns and efforts to translate such
knowledge to effective models of intelligence.

By showing how astrocyte-modulated STDP can maximize computational performance
near criticality, we aimed to broaden the applicability of the LSM to complex
spatio-temporal problems that require integration of data over multiple
sources and time-scales, thereby, making LSMs suitable for real-life
applications of edge computing. Our so far results suggest that this is a
direction worth pursuing.

## Acknowledgements

This work is supported by the National Center for Medical Rehabilitation
Research (NIH/NICHD) $K12HD093427$ Grant and by the Rutgers Office of Research
and Innovation. Any findings, conclusions, and opinions expressed in this
material are those of the authors and do not necessarily reflect the views of
the NIH or Rutgers University.

## References

  * Kendall and Kumar [2020] Jack D. Kendall and Suhas Kumar.  The building blocks of a brain-inspired computer.  _Applied Physics Reviews_ , 7(1):011305, 2020\.  doi: 10.1063/1.5129306.
  * Tang et al. [2020] Guangzhi Tang, Neelesh Kumar, and Konstantinos P. Michmizos.  Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware.  In _2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 6090–6097, 2020.  doi: 10.1109/IROS45743.2020.9340948.
  * Davies et al. [2018] Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, Yuyun Liao, Chit-Kwan Lin, Andrew Lines, Ruokun Liu, Deepak Mathaikutty, Steven McCoy, Arnab Paul, Jonathan Tse, Guruguhanathan Venkataramanan, Yi-Hsin Weng, Andreas Wild, Yoonseok Yang, and Hong Wang.  Loihi: A neuromorphic manycore processor with on-chip learning.  _IEEE Micro_ , 38(1):82–99, 2018.  doi: 10.1109/MM.2018.112130359.
  * Tang et al. [2019a] Guangzhi Tang, Arpit Shah, and Konstantinos P. Michmizos.  Spiking neural network on neuromorphic hardware for energy-efficient unidimensional slam.  In _2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_ , pages 4176–4181, 2019a.  doi: 10.1109/IROS40897.2019.8967864.
  * Cao et al. [2020] Keyan Cao, Yefan Liu, Gongjie Meng, and Qimeng Sun.  An overview on edge computing research.  _IEEE Access_ , 8:85714–85728, 2020.  doi: 10.1109/ACCESS.2020.2991734.
  * Chen and Ran [2019] Jiasi Chen and Xukan Ran.  Deep learning with edge computing: A review.  _Proceedings of the IEEE_ , 107(8):1655–1674, 2019.  doi: 10.1109/JPROC.2019.2921977.
  * Maass et al. [2002] Wolfgang Maass, Thomas Natschläger, and Henry Markram.  Real-time computing without stable states: A new framework for neural computation based on perturbations.  _Neural Computation_ , 14(11):2531–2560, 2002\.  doi: 10.1162/089976602760407955.
  * Soures Nicholas [2019] Kudithipudi Dhireesha Soures Nicholas.  Deep liquid state machines with neural plasticity for video activity recognition.  _Frontiers in Neuroscience_ , 13:686, 2019.  doi: 10.3389/fnins.2019.00686.
  * Ponghiran et al. [2019] Wachirawit Ponghiran, Gopalakrishnan Srinivasan, and Kaushik Roy.  Reinforcement learning with low-complexity liquid state machines.  _Frontiers in Neuroscience_ , 13:883, 2019.  ISSN 1662-453X.  doi: 10.3389/fnins.2019.00883.
  * Wang and Li [2016] Qian Wang and Peng Li.  D-lsm: Deep liquid state machine with unsupervised recurrent reservoir tuning.  In _2016 23rd International Conference on Pattern Recognition (ICPR)_ , pages 2652–2657, 2016.  doi: 10.1109/ICPR.2016.7900035.
  * Liu et al. [2020a] Shiya Liu, Lingjia Liu, and Yang Yi.  Quantized reservoir computing on edge devices for communication applications.  In _2020 IEEE/ACM Symposium on Edge Computing (SEC)_ , pages 445–449, 2020a.  doi: 10.1109/SEC50012.2020.00068.
  * Li et al. [2020] Shiming Li, Lei Wang, Shiying Wang, and Weixia Xu.  Liquid state machine applications mapping for noc-based neuromorphic platforms.  In Dezun Dong, Xiaoli Gong, Cunlu Li, Dongsheng Li, and Junjie Wu, editors, _Advanced Computer Architecture_ , pages 277–289, Singapore, 2020\. Springer Singapore.  ISBN 978-981-15-8135-9.
  * Ku et al. [2018] Bon Woong Ku, Yu Liu, Yingyezhe Jin, Sandeep Samal, Peng Li, and Sung Kyu Lim.  Design and architectural co-optimization of monolithic 3d liquid state machine-based neuromorphic processor.  In _2018 55th ACM/ESDA/IEEE Design Automation Conference (DAC)_ , pages 1–6, 2018.  doi: 10.1109/DAC.2018.8465837.
  * Rosselló et al. [2016] Josep L. Rosselló, Miquel L. Alomar, Antoni Morro, Antoni Oliver, and Vincent Canals.  High-density liquid-state machine circuitry for time-series forecasting.  _International Journal of Neural Systems_ , 26(05):1550036, 2016.  doi: 10.1142/S0129065715500367.  PMID: 26906454.
  * Balafrej and Rouat [2020] Ismael Balafrej and Jean Rouat.  P-critical: A reservoir autoregulation plasticity rule for neuromorphic hardware, 2020.
  * Brodeur and Rouat [2012] Simon Brodeur and Jean Rouat.  Regulation toward self-organized criticality in a recurrent spiking neural reservoir.  In Alessandro E. P. Villa, Włodzisław Duch, Péter Érdi, Francesco Masulli, and Günther Palm, editors, _Artificial Neural Networks and Machine Learning – ICANN 2012_ , pages 547–554, Berlin, Heidelberg, 2012. Springer Berlin Heidelberg.
  * Jin and Li [2016] Yingyezhe Jin and Peng Li.  Ap-stdp: A novel self-organizing mechanism for efficient reservoir computing.  In _2016 International Joint Conference on Neural Networks (IJCNN)_ , pages 1158–1165, 2016.  doi: 10.1109/IJCNN.2016.7727328.
  * Norton and Ventura [2006] D. Norton and D. Ventura.  Preparing more effective liquid state machines using hebbian learning.  In _The 2006 IEEE International Joint Conference on Neural Network Proceedings_ , pages 4243–4248, 2006.  doi: 10.1109/IJCNN.2006.246996.
  * Legenstein and Maass [2007] Robert Legenstein and Wolfgang Maass.  Edge of chaos and prediction of computational performance for neural circuit models.  _Neural Networks_ , 20(3):323 – 334, 2007.  ISSN 0893-6080.  doi: 10.1016/j.neunet.2007.04.017.  Echo State Networks and Liquid State Machines.
  * Bertschinger and Natschläger [2004] Nils Bertschinger and Thomas Natschläger.  Real-time computation at the edge of chaos in recurrent neural networks.  _Neural Computation_ , 16(7):1413–1436, 2004\.  doi: 10.1162/089976604323057443.
  * Langton [1990] Chris G. Langton.  Computation at the edge of chaos: phase transitions and emergent computation.  _Physica D: Nonlinear Phenomena_ , 42(1):12 – 37, 1990.  ISSN 0167-2789.  doi: 10.1016/0167-2789(90)90064-V.
  * Shew et al. [2011] Woodrow L. Shew, Hongdian Yang, Shan Yu, Rajarshi Roy, and Dietmar Plenz.  Information capacity and transmission are maximized in balanced cortical networks with neuronal avalanches.  _Journal of Neuroscience_ , 31(1):55–63, 2011\.  ISSN 0270-6474.  doi: 10.1523/JNEUROSCI.4637-10.2011.
  * de Arcangelis and Herrmann [2010] Lucilla de Arcangelis and Hans J. Herrmann.  Learning as a phenomenon occurring in a critical state.  _Proceedings of the National Academy of Sciences_ , 107(9):3977–3981, 2010.  ISSN 0027-8424.  doi: 10.1073/pnas.0912289107.
  * Kinouchi and Copelli [2006] Osame Kinouchi and Mauro Copelli.  Optimal dynamical range of excitable networks at criticality.  _Nature Physics_ , 2(5):348–351, 2006.  doi: 10.1038/nphys289.
  * Haldeman and Beggs [2005] Clayton Haldeman and John M. Beggs.  Critical branching captures activity in living neural networks and maximizes the number of metastable states.  _Phys. Rev. Lett._ , 94:058101, Feb 2005.  doi: 10.1103/PhysRevLett.94.058101.
  * Beggs and Plenz [2003] John M. Beggs and Dietmar Plenz.  Neuronal avalanches in neocortical circuits.  _Journal of Neuroscience_ , 23(35):11167–11177, 2003.  ISSN 0270-6474.  doi: 10.1523/JNEUROSCI.23-35-11167.2003.
  * Shriki et al. [2013] Oren Shriki, Jeff Alstott, Frederick Carver, Tom Holroyd, Richard N.A. Henson, Marie L. Smith, Richard Coppola, Edward Bullmore, and Dietmar Plenz.  Neuronal avalanches in the resting meg of the human brain.  _Journal of Neuroscience_ , 33(16):7079–7090, 2013.  ISSN 0270-6474.  doi: 10.1523/JNEUROSCI.4286-12.2013.
  * Chialvo [2010] Dante R. Chialvo.  Emergent complex neural dynamics.  _Nature Physics_ , 6:744–750, 2010.  ISSN 1745-2481.  doi: 10.1038/nphys1803.
  * Feldman [2009] Daniel E. Feldman.  Synaptic mechanisms for plasticity in neocortex.  _Annual Review of Neuroscience_ , 32(1):33–55, 2009.  doi: 10.1146/annurev.neuro.051508.135516.  PMID: 19400721.
  * Stepp et al. [2015] Nigel Stepp, Dietmar Plenz, and Narayan Srinivasa.  Synaptic plasticity enables adaptive self-tuning critical networks.  _PLOS Computational Biology_ , 11(1):1–28, 01 2015.  doi: 10.1371/journal.pcbi.1004043.
  * Zenke and Gerstner [2017] Friedemann Zenke and Wulfram Gerstner.  Hebbian plasticity requires compensatory processes on multiple timescales.  _Philosophical Transactions of the Royal Society B: Biological Sciences_ , 372(1715):20160259, 2017.  doi: 10.1098/rstb.2016.0259.
  * Watt and Desai [2010] Alanna Watt and Niraj Desai.  Homeostatic plasticity and stdp: keeping a neuron’s cool in a fluctuating world.  _Frontiers in Synaptic Neuroscience_ , 2:5, 2010.  ISSN 1663-3563.  doi: 10.3389/fnsyn.2010.00005.
  * Abbott and Nelson [2000] L. F. Abbott and Sacha B. Nelson.  Synaptic plasticity: taming the beast.  _Nature Neuroscience_ , 3(11):1178–1183, 2000\.  ISSN 1546-1726.  doi: 10.1038/81453.
  * Perea and Araque [2010] Gertrudis Perea and Alfonso Araque.  Glia modulates synaptic transmission.  _Brain Research Reviews_ , 63(1):93–102, 2010\.  ISSN 0165-0173.
  * Han et al. [2013] Xiaoning Han, Michael Chen, Fushun Wang, Martha Windrem, Su Wang, Steven Shanz, Qiwu Xu, Nancy Ann Oberheim, Lane Bekar, and Sarah Betstadt.  Forebrain engraftment by human glial progenitor cells enhances synaptic plasticity and learning in adult mice.  _Cell Stem Cell_ , 12(3):342–353, 2013.  ISSN 1934-5909.
  * Tewari and Parpura [2013] Shivendra Tewari and Vladimir Parpura.  A possible role of astrocytes in contextual memory retrieval: An analysis obtained using a quantitative framework.  _Frontiers in Computational Neuroscience_ , 7:145, 2013\.  ISSN 1662-5188.  doi: 10.3389/fncom.2013.00145.
  * Navarrete and Araque [2010] Marta Navarrete and Alfonso Araque.  Endocannabinoids potentiate synaptic transmission through stimulation of astrocytes.  _Neuron_ , 68(1):113–126, 2010.  ISSN 0896-6273.
  * Adamsky et al. [2018] Adar Adamsky, Adi Kol, Tirzah Kreisel, Adi Doron, Nofar Ozeri-Engelhard, Talia Melcer, Ron Refaeli, Henrike Horn, Limor Regev, and Maya Groysman.  Astrocytic activation generates de novo neuronal potentiation and memory enhancement.  _Cell_ , 174(1):59–71. e14, 2018.  ISSN 0092-8674.
  * Petrelli et al. [2020] Francesco Petrelli, Glenn Dallérac, Luca Pucci, Corrado Calì, Tamara Zehnder, Sébastien Sultan, Salvatore Lecca, Andrea Chicca, Andrei Ivanov, Cédric S. Asensio, Vidar Gundersen, Nicolas Toni, Graham William Knott, Fulvio Magara, Jürg Gertsch, Frank Kirchhoff, Nicole Déglon, Bruno Giros, Robert H. Edwards, Jean-Pierre Mothet, and Paola Bezzi.  Dysfunction of homeostatic control of dopamine by astrocytes in the developing prefrontal cortex leads to cognitive impairments.  _Molecular Psychiatry_ , 25(4):732–749, 2020\.  doi: 10.1038/s41380-018-0226-y.
  * Manninen et al. [2020] Tiina Manninen, Ausra Saudargiene, and Marja-Leena Linne.  Astrocyte-mediated spike-timing-dependent long-term depression modulates synaptic properties in the developing cortex.  _PLOS Computational Biology_ , 16(11):1–29, 11 2020.  doi: 10.1371/journal.pcbi.1008360.
  * Sibille et al. [2014] Jérémie Sibille, Ulrike Pannasch, and Nathalie Rouach.  Astroglial potassium clearance contributes to short-term plasticity of synaptically evoked currents at the tripartite synapse.  _The Journal of Physiology_ , 592(1):87–102, 2014\.  doi: 10.1113/jphysiol.2013.261735.
  * Min and Nevian [2012] Rogier Min and Thomas Nevian.  Astrocyte signaling controls spike timing–dependent depression at neocortical synapses.  _Nature Neuroscience_ , 15(5):746–753, 2012.  doi: 10.1038/nn.3075.
  * Thrane et al. [2012] Alexander Stanley Thrane, Vinita Rangroo Thrane, Douglas Zeppenfeld, Nanhong Lou, Qiwu Xu, Erlend Arnulf Nagelhus, and Maiken Nedergaard.  General anesthesia selectively disrupts astrocyte calcium signaling in the awake mouse cortex.  _Proceedings of the National Academy of Sciences_ , 109(46):18974–18979, 2012.  ISSN 0027-8424.  doi: 10.1073/pnas.1209448109.
  * Foley et al. [2017] Jeannine Foley, Tamara Blutstein, SoYoung Lee, Christophe Erneux, Michael M. Halassa, and Philip Haydon.  Astrocytic ip3/ca2+ signaling modulates theta rhythm and rem sleep.  _Frontiers in Neural Circuits_ , 11:3, 2017.  ISSN 1662-5110.  doi: 10.3389/fncir.2017.00003.
  * Bojarskaite et al. [2020] Laura Bojarskaite, Daniel M. Bjørnstad, Klas H. Pettersen, Céline Cunen, Gudmund Horn Hermansen, Knut Sindre Åbjørsbråten, Rolf Sprengel, Koen Vervaeke, Wannan Tang, Rune Enger, and Erlend A. Nagelhus.  Astrocytic ca2+ signaling is reduced during sleep and is involved in the regulation of slow wave sleep.  _Nature Communications_ , 11:3240, 2020.  ISSN 2041-1723.  doi: 10.1038/s41467-020-17062-2.
  * Ingiosi et al. [2020] Ashley M. Ingiosi, Christopher R. Hayworth, Daniel O. Harvey, Kristan G. Singletary, Michael J. Rempe, Jonathan P. Wisor, and Marcos G. Frank.  A role for astroglial calcium in mammalian sleep and sleep regulation.  _Current Biology_ , 30:4373–4383.e7, 2020.  doi: 10.1016/j.cub.2020.08.052.
  * Tagliazucchi et al. [2016] Enzo Tagliazucchi, Dante R. Chialvo, Michael Siniatchkin, Enrico Amico, Jean-Francois Brichant, Vincent Bonhomme, Quentin Noirhomme, Helmut Laufs, and Steven Laureys.  Large-scale signatures of unconsciousness are consistent with a departure from critical dynamics.  _Journal of The Royal Society Interface_ , 13(114):20151027, 2016.  doi: 10.1098/rsif.2015.1027.
  * Bellay et al. [2015] Timothy Bellay, Andreas Klaus, Saurav Seshadri, and Dietmar Plenz.  Irregular spiking of pyramidal neurons organizes as scale-invariant neuronal avalanches in the awake state.  _eLife_ , 4:e07224, jul 2015.  ISSN 2050-084X.  doi: 10.7554/eLife.07224.
  * Hahn et al. [2017] Gerald Hahn, Adrian Ponce-Alvarez, Cyril Monier, Giacomo Benvenuti, Arvind Kumar, Frédéric Chavane, Gustavo Deco, and Yves Frégnac.  Spontaneous cortical activity is transiently poised close to criticality.  _PLOS Computational Biology_ , 13(5):1–29, 05 2017.  doi: 10.1371/journal.pcbi.1005543.
  * Priesemann et al. [2013] Viola Priesemann, Mario Valderrama, Michael Wibral, and Michel Le Van Quyen.  Neuronal avalanches differ from wakefulness to deep sleep – evidence from intracranial depth recordings in humans.  _PLOS Computational Biology_ , 9(3):1–14, 03 2013\.  doi: 10.1371/journal.pcbi.1002985.
  * Fagerholm et al. [2015] Erik D. Fagerholm, Romy Lorenz, Gregory Scott, Martin Dinov, Peter J. Hellyer, Nazanin Mirzaei, Clare Leeson, David W. Carmichael, David J. Sharp, Woodrow L. Shew, and Robert Leech.  Cascades and cognitive state: focused attention incurs subcritical dynamics.  _Journal of Neuroscience_ , 35(11):4626–4634, 2015.  ISSN 0270-6474.  doi: 10.1523/JNEUROSCI.3694-14.2015.
  * Parpura and Haydon [2000] Vladimir Parpura and Philip G. Haydon.  Physiological astrocytic calcium levels stimulate glutamate release to modulate adjacent neurons.  _Proceedings of the National Academy of Sciences_ , 97(15):8629–8634, 2000.  ISSN 0027-8424.  doi: 10.1073/pnas.97.15.8629.
  * Christian and Huguenard [2013] Catherine A. Christian and John R. Huguenard.  Astrocytes potentiate gabaergic transmission in the thalamic reticular nucleus via endozepine signaling.  _Proceedings of the National Academy of Sciences_ , 110(50):20278–20283, 2013.  doi: 10.1073/pnas.1318031110.
  * Mederos et al. [2018] Sara Mederos, Candela González-Arias, and Gertrudis Perea.  Astrocyte–neuron networks: a multilane highway of signaling for homeostatic brain function.  _Frontiers in Synaptic Neuroscience_ , 10(45), 2018.  ISSN 1663-3563.  doi: 10.3389/fnsyn.2018.00045.
  * Shigetomi et al. [2016] Eiji Shigetomi, Sandip Patel, and Baljit S. Khakh.  Probing the complexities of astrocyte calcium signaling.  _Trends in Cell Biology_ , 26(4):300–312, 2016\.  ISSN 0962-8924.
  * Perea et al. [2014] Gertrudis Perea, Mriganka Sur, and Alfonso Araque.  Neuron-glia networks: integral gear of brain function.  _Frontiers in Cellular Neuroscience_ , 8(378), 2014.  ISSN 1662-5102.  doi: 10.3389/fncel.2014.00378.
  * Araque et al. [2014] Alfonso Araque, Giorgio Carmignoto, Philip G Haydon, Stéphane HR Oliet, Richard Robitaille, and Andrea Volterra.  Gliotransmitters travel in time and space.  _Neuron_ , 81(4):728–739, 2014.  ISSN 0896-6273.
  * Lecun et al. [1998] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, 1998.  doi: 10.1109/5.726791.
  * Orchard et al. [2015] Garrick Orchard, Ajinkya Jayawant, Gregory K. Cohen, and Nitish Thakor.  Converting static image datasets to spiking neuromorphic datasets using saccades.  _Frontiers in Neuroscience_ , 9:437, 2015.  ISSN 1662-453X.  doi: 10.3389/fnins.2015.00437.
  * Xiao et al. [2017] Han Xiao, Kashif Rasul, and Roland Vollgraf.  Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017.
  * Morrison et al. [2008] Abigail Morrison, Markus Diesmann, and Wulfram Gerstner.  Phenomenological models of synaptic plasticity based on spike timing.  _Biological Cybernetics_ , 98(6):459–478, 2008\.  ISSN 1432-0770.  doi: 10.1007/s00422-008-0233-1.
  * Zhang and Li [2019] Wenrui Zhang and Peng Li.  Information-theoretic intrinsic plasticity for online unsupervised learning in spiking neural networks.  _Frontiers in Neuroscience_ , 13:31, 2019.  ISSN 1662-453X.  doi: 10.3389/fnins.2019.00031.
  * Maass et al. [2004] Wolfgang Maass, Robert Legenstein, and Nils Bertschinger.  Methods for estimating the computational power and generalization capability of neural microcircuits.  In _Proceedings of the 17th International Conference on Neural Information Processing Systems_ , NIPS’04, page 865–872, Cambridge, MA, USA, 2004\. MIT Press.
  * Wijesinghe et al. [2019] Parami Wijesinghe, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy.  Analysis of liquid ensembles for enhancing the performance and accuracy of liquid state machines.  _Frontiers in Neuroscience_ , 13:504, 2019.  ISSN 1662-453X.  doi: 10.3389/fnins.2019.00504.
  * Boedecker et al. [2012] Joschka Boedecker, Oliver Obst, Joseph T Lizier, N Michael Mayer, and Minoru Asada.  Information processing in echo state networks at the edge of chaos.  _Theory Biosci._ , 131(3):205–13, 2012.  doi: 10.1007/s12064-011-0146-8.
  * Ribeiro et al. [2008] Andre S. Ribeiro, Stuart A. Kauffman, Jason Lloyd-Price, Björn Samuelsson, and Joshua E. S. Socolar.  Mutual information in random boolean models of regulatory networks.  _Phys. Rev. E_ , 77:011901, Jan 2008.  doi: 10.1103/PhysRevE.77.011901.
  * Kuśmierz et al. [2020] Łukasz Kuśmierz, Shun Ogawa, and Taro Toyoizumi.  Edge of chaos and avalanches in neural networks with heavy-tailed synaptic weight distribution.  _Phys. Rev. Lett._ , 125:028101, Jul 2020.  doi: 10.1103/PhysRevLett.125.028101.
  * van Vreeswijk and Sompolinsky [1996] C. van Vreeswijk and H. Sompolinsky.  Chaos in neuronal networks with balanced excitatory and inhibitory activity.  _Science_ , 274(5293):1724–1726, 1996.  doi: 10.1126/science.274.5293.1724.
  * Ostojic [2014] Srdjan Ostojic.  Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons.  _Nature Neuroscience_ , 17:594–600, 2014.  ISSN 1546-1726.  doi: 10.1038/nn.3658.
  * Rajan et al. [2010] Kanaka Rajan, L. F. Abbott, and Haim Sompolinsky.  Stimulus-dependent suppression of chaos in recurrent neural networks.  _Phys. Rev. E_ , 82:011903, Jul 2010.  doi: 10.1103/PhysRevE.82.011903.
  * Zhou et al. [2019] Bin Zhou, Yun-Xia Zuo, and Ruo-Tian Jiang.  Astrocyte morphology: diversity, plasticity, and role in neurological diseases.  _CNS Neuroscience & Therapeutics_, 25(6):665–673, 2019.  doi: 10.1111/cns.13123.
  * Zhao et al. [2020] Dongcheng Zhao, Yi Zeng, Tielin Zhang, Mengting Shi, and Fei fei Zhao.  Glsnn: A multi-layer spiking neural network based on global feedback alignment and local stdp plasticity.  _Frontiers in Computational Neuroscience_ , 14, 2020.
  * Samadi et al. [2017] Arash Samadi, T. Lillicrap, and D. Tweed.  Deep learning with dynamic spiking neurons and fixed feedback weights.  _Neural Computation_ , 29:578–602, 2017.
  * Zhang et al. [2018a] Tielin Zhang, Yi Zeng, Dongcheng Zhao, and Bo Xu.  Brain-inspired balanced tuning for spiking neural networks.  In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18_ , pages 1653–1659. International Joint Conferences on Artificial Intelligence Organization, 7 2018a.  doi: 10.24963/ijcai.2018/229.
  * Diehl and Cook [2015] Peter Diehl and Matthew Cook.  Unsupervised learning of digit recognition using spike-timing-dependent plasticity.  _Frontiers in Computational Neuroscience_ , 9:99, 2015.  ISSN 1662-5188.  doi: 10.3389/fncom.2015.00099.
  * Mostafa [2018] Hesham Mostafa.  Supervised learning based on temporal coding in spiking neural networks.  _IEEE Transactions on Neural Networks and Learning Systems_ , 29:3227–3235, 2018.
  * Mirsadeghi et al. [2021] Maryam Mirsadeghi, Majid Shalchian, Saeed Reza Kheradpisheh, and Timothée Masquelier.  Stidi-bp: Spike time displacement based error backpropagation in multilayer spiking neural networks.  _Neurocomputing_ , 427:131–140, 2021.  ISSN 0925-2312.  doi: https://doi.org/10.1016/j.neucom.2020.11.052.
  * O’Connor and Welling [2016] Peter O’Connor and M. Welling.  Deep spiking networks.  _ArXiv_ , abs/1602.08323, 2016.
  * Lee et al. [2016] Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer.  Training deep spiking neural networks using backpropagation.  _Frontiers in Neuroscience_ , 10:508, 2016.  ISSN 1662-453X.  doi: 10.3389/fnins.2016.00508.
  * Wu et al. [2018] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi.  Spatio-temporal backpropagation for training high-performance spiking neural networks.  _Frontiers in Neuroscience_ , 12:331, 2018.  ISSN 1662-453X.  doi: 10.3389/fnins.2018.00331.
  * Kaiser et al. [2020] Jacques Kaiser, Hesham Mostafa, and Emre Neftci.  Synaptic plasticity dynamics for deep continuous local learning (decolle).  _Frontiers in Neuroscience_ , 14:424, 2020.  ISSN 1662-453X.  doi: 10.3389/fnins.2020.00424.
  * Liu et al. [2020b] Qianhui Liu, Haibo Ruan, Dong Xing, H. Tang, and Gang Pan.  Effective aer object classification using segmented probability-maximization learning in spiking neural networks.  In _AAAI_ , 2020b.
  * Shrestha and Orchard [2018] S. Shrestha and G. Orchard.  Slayer: Spike layer error reassignment in time.  In _NeurIPS_ , 2018.
  * Zhang et al. [2018b] Tielin Zhang, Yi Zeng, Dongcheng Zhao, and Mengting Shi.  A plasticity-centric approach to train the non-differential spiking neural networks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , 2018b.
  * Hao et al. [2020] Yunzhe Hao, Xuhui Huang, Meng Dong, and Bo Xu.  A biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule.  _Neural Networks_ , 121:387–395, 2020.  ISSN 0893-6080.  doi: https://doi.org/10.1016/j.neunet.2019.09.007.
  * Kheradpisheh et al. [2020] Saeed Reza Kheradpisheh, Maryam Mirsadeghi, and Timothée Masquelier.  BS4NN: binarized spiking neural networks with temporal coding and learning.  _CoRR_ , abs/2007.04039, 2020.
  * Ju et al. [2013] Han Ju, Jian-Xin Xu, Edmund Chong, and Antonius M.J. VanDongen.  Effects of synaptic connectivity on liquid state machine performance.  _Neural Networks_ , 38:39–51, 2013.  ISSN 0893-6080.  doi: https://doi.org/10.1016/j.neunet.2012.11.003.
  * Ponulak and Kasiński [2010] Filip Ponulak and Andrzej Kasiński.  Supervised learning in spiking neural networks with resume: Sequence learning, classification, and spike shifting.  _Neural Comput._ , 22(2):467–510, February 2010\.  ISSN 0899-7667.  doi: 10.1162/neco.2009.11-08-901.
  * Urbanczik and Senn [2009] Robert Urbanczik and Walter Senn.  A Gradient Learning Rule for the Tempotron.  _Neural Computation_ , 21(2):340–352, 02 2009\.  ISSN 0899-7667.  doi: 10.1162/neco.2008.09-07-605.
  * Gütig and Sompolinsky [2006] Robert Gütig and Haim Sompolinsky.  The tempotron: a neuron that learns spike timing–based decisions.  _Nature Neuroscience_ , 9(3):420–428, 2006.  doi: 10.1038/nn1643.
  * Tang et al. [2019b] Guangzhi Tang, Ioannis E. Polykretis, Vladimir A. Ivanov, Arpit Shah, and Konstantinos P. Michmizos.  Introducing astrocytes on a neuromorphic processor: Synchronization, local plasticity and edge of chaos.  _ACM Proceedings of 2019 Neuroinspired Computing Elements (NICE 2019)_ , 1(1):1–10, 2019b.  doi: arXiv:1907.01620.
  * Roxin et al. [2011] Alex Roxin, Nicolas Brunel, David Hansel, Gianluigi Mongillo, and Carl van Vreeswijk.  On the distribution of firing rates in networks of cortical neurons.  _Journal of Neuroscience_ , 31(45):16217–16226, 2011.  ISSN 0270-6474.  doi: 10.1523/JNEUROSCI.1677-11.2011.
  * Shew et al. [2009] Woodrow L. Shew, Hongdian Yang, Thomas Petermann, Rajarshi Roy, and Dietmar Plenz.  Neuronal avalanches imply maximum dynamic range in cortical networks at criticality.  _Journal of Neuroscience_ , 29(49):15595–15600, 2009.  ISSN 0270-6474.  doi: 10.1523/JNEUROSCI.3864-09.2009.
  * Zenke et al. [2013] Friedemann Zenke, Guillaume Hennequin, and Wulfram Gerstner.  Synaptic plasticity in neural networks needs homeostasis with a fast rate detector.  _PLOS Computational Biology_ , 9(11):1–14, 11 2013.  doi: 10.1371/journal.pcbi.1003330.
  * Halassa and Haydon [2010] Michael M. Halassa and Philip G. Haydon.  Integrated brain circuits: astrocytic networks modulate neuronal activity and behavior.  _Annual Review of Physiology_ , 72(1):335–355, 2010.  doi: 10.1146/annurev-physiol-021909-135843.
  * Benarroch [2005] Eduardo E. Benarroch.  Neuron-astrocyte interactions: partnership for normal function and disease in the central nervous system.  _Mayo Clinic Proceedings_ , 80(10):1326–1338, 2005.  ISSN 0025-6196.
  * Theodosis et al. [2008] Dionysia T. Theodosis, Dominique A. Poulain, and Stéphane H. R. Oliet.  Activity-dependent structural and functional plasticity of astrocyte-neuron interactions.  _Physiological Reviews_ , 88(3):983–1008, 2008\.  doi: 10.1152/physrev.00036.2007.

##  Appendix A Appendix

###  A.1 Datasets

We tested models on MNIST mnist_bib_0, its temporal, event-driven version,
N-MNIST nmnist_bib_0, and Fashion-MNIST fmnist_bib_0. We modified the original
$60,000$/$10,000$ train/test split to $50,000$/$10,000$/$10,000$
train/validate/test split, by partitioning away the last $10,000$ training
samples to the validation set. We normalized and rescaled each $28\times 28$
MNIST and Fashion-MNIST image to $0-1$ range, which we Poisson encoded into
the spiking activity of input neurons. For N-MNIST, we treated all discrete
events the same way and transformed each image into $300\times 68\times 34$
matrix, with the first dimension being temporal. Using first $250$ timesteps,
we converted each event at each timestep into a spike in the corresponding
input neuron.

###  A.2 Neuron Parameters

The LIF neuron parameters we used in all networks are shown in Table S1.

Table S1: LIF neuron parameters Parameter name | Description | Value
---|---|---
$\theta$ | membrane potential threshold | $20.0$
$\tau_{v}$ | membrane potential time constant | $64.0$
$\tau_{u}$ | synaptic conductance time constant | $1.0$
$b$ | membrane potential bias | $0.0$

###  A.3 Liquid Connectivity Parameters

The parameters we used in the distance based connection probability function,
(3), depended on the connection type. Connection types were determined by the
pre- and post-synaptic neurons, which resulted in 4 types of connections:

  1. 1.

$EE$: excitatory to excitatory

  2. 2.

$EI$: excitatory to inhibitory

  3. 3.

$II$: inhibitory to inhibitory

  4. 4.

$IE$: inhibitory to excitatory

For each connection type $[EE,EI,II,IE]$, parameter $C$ values were
$[0.2,0.1,0.3,0.05]$. For all connection types, $\lambda=3.0$.

###  A.4 Neuron-astrocyte connection weight

The weight of neuron-astrocyte connections, $w_{astro}$ in (7), impacted both
liquid dynamics and NALSM accuracy. Controlling the responsiveness of the LIM
astrocyte to liquid neuron activity, larger $w_{astro}$ resulted in lower
branching factor, and vice versa. For both datasets, accuracy peaked in the
vicinity of $w_{astro}=0.01$ with slightly super-critical branching factor of
$\approx 1.3$ for MNIST and $\approx 1.2$ for N-MNIST (Fig. S1).

Figure S1: Neuron-astrocyte connection weight impacts liquid dynamics and
NALSM accuracy. ( Top ) NALSM accuracy shown with respect to neuron-astrocyte
connection weight for MNIST and N-MNIST. ( Bottom ) NALSM liquid dynamics
shown as a function of neuron-astrocyte connection weight for MNIST and
N-MNIST. Data points are averaged over 10 random networks. Error bars are
standard deviation.

###  A.5 LSM+AP-STDP model

We implemented AP-STDP from apstdp_bib_0 on top of LSM+STDP model by making
STDP weight changes conditionally dependent on neuronal activity.
Specifically, we implemented rule (4) from apstdp_bib_0, with $p=1.0$. The
spiking rate of each neuron $i$, $C_{i}$, was approximated using (3) from
apstdp_bib_0, with $\tau_{C}=1000$ $ms$. Parameters $C_{\theta}$ and $\Delta
C$ set the neuronal activity range in which STDP changes were enforced. We
hand-tuned parameters $C_{\theta}$ and $\Delta C$ for each specific network
and dataset to maximize the validation accuracy of LSM+AP-STDP model. We used
the same initialization process as was used for LSM+STDP model, with two
exceptions 1) weights were set to $1.0$ prior to initialization, and 2) STDP
synaptic weight changes were conditioned on neuron activity ranges using
$C_{\theta}$ and $\Delta C$. As with LSM+STDP model, the liquid’s weights were
fixed during spike generation phase.

###  A.6 Evidence for edge-of-chaos dynamics in NALSM

Here, we provide evidence suggesting that NALSM’s slightly super-critical
branching dynamics (Fig. 2) corresponded to the edge-of-chaos. First, NALSM
exhibited coexistence of small and large synaptic weights, which is necessary
for chaotic activity in spiking networks kus_2020_0. NALSM had concentrations
of near-maximum excitatory weights and near-zero weights, with weights also
covering the full range in between these extremes. Inhibitory weights exhibit
the same kind of bimodal distribution (Fig. S2).

Second, NALSM exhibited excitation/inhibition (E/I) balance, which is thought
to be necessary for existence of deterministic chaos
Vreeswijk_1996_0,Krauss_2019_0. We used three different methods to evaluate
E/I balance. First, we confirmed synaptic weight E/I balance, $W_{E/I}$, in
initialized NALSM liquids, which was found to align with edge-of-chaos
dynamics in Krauss_2019_0 and was evaluated as:

$W_{E/I}=\frac{n_{w>0}-n_{w<0}}{n_{w\neq 0}}\\\ $ (10)

where $n_{w>0}$, $n_{w<0}$, and $n_{w\neq 0}$ are total number of IL and LL
synaptic weights that are positive, negative, and non-zero, respectively.
Indicative of E/I balance, we obtained $W_{E/I}=-0.0029\pm 0.018$ averaged
over all NALSM initializations on both MNIST and N-MNIST ($W_{E/I}$ ranges
from $-1$ to $1$, with $0$ representing perfect E/I balance). Second, we
measured the difference in spiking rates between liquid excitatory and liquid
inhibitory neuron populations by evaluating:

$f_{E/I}=\frac{|\hat{f}_{e}-\hat{f}_{i}|}{\hat{f}_{l}}\\\ $ (11)

where $\hat{f}_{e}$, $\hat{f}_{i}$, and $\hat{f}_{l}$ are the average spiking
rate of excitatory liquid neurons, inhibitory liquid neurons, and all liquid
neurons, respectively. Averaged over all NALSM network initializations and
both MNIST and N-MNIST datasets, we obtained $f_{E/I}=0.074\pm 0.083$, which
was indicative of E/I balance since $f_{E/I}$ ranges from $0$ to $1$, with $0$
representing perfect E/I balance. Finally, we measured the net current
received by each neuron at each timestep from all active input and liquid
neurons. Averaged over $100$ different MNIST input samples, net current
received by each neuron was $-0.99\pm 4.91$. The near $0$ average net current
combined with its large standard deviation suggests that excitatory and
inhibitory inputs were balanced and that neurons were primarily driven by
network fluctuations. This is believed to give rise to the irregular activity
observed in the brain Ostojic_2014_0 and has been associated with
deterministic chaos Vreeswijk_1996_0 (shown in Fig. 2 A in Vreeswijk_1996_0).

Indeed, NALSM also exhibited spiking activity that was irregular across the
network and across time (Fig. S3). Evidence for chaotic activity was further
confirmed by autocorrelation analysis performed on neuronal spike trains
generated during generation of spike counts for output layer training (See
2.2.2). Specifically, spike autocorrelation function,
$A_{spikes}(\tau_{auto})$, was computed as:

$A_{spikes}(\tau_{auto})=\frac{1}{NT}\sum_{i=1}^{N}\sum_{t=1}^{T}\sigma_{i}(t+\tau_{auto})\sigma_{i}(t)$
(12)

where $N=1000$ liquid neurons, $T=125$ $ms$ is the duration of neuronal spike
trains, $\sigma_{i}$ is the spike train of neuron $i$. As the branching factor
became increasingly greater than $1.0$, the decay of liquid neuron spike
autocorrelation functions became broader and increased in magnitude (Fig. S4).
Alternatively, when the branching factor became progressively less than $1.0$,
decay of liquid neuron spike autocorrelation functions was narrower and
magnitudes were marginally greater than that of input neuron spike
autocorrelation functions. As expected, spike train autocorrelation functions
of input neurons remained flat showing no decay with respect to lag time. This
suggested that the transition from a sub-critical to a super-critical
branching factor possibly corresponded to a transition to chaos in NALSM’s
spiking rate dynamics Ostojic_2014_0,Rajan_2010_0.

Figure S2: Initialized NALSM synaptic weights. Distribution of IL and LL
synaptic weights after NALSM liquid initialization. Figure S3: NALSM network
spike activity. For each input sample class, a raster plot shows spike
activity of input (black), liquid inhibitory (green), and liquid excitatory
(blue) neurons for a $100$ $ms$ duration. Figure S4: Spike autocorrelation
versus branching factor dynamics. Spike autocorrelation as a function of lag
time for sub-critical (left), near-critical (middle), and super-critical
(right) branching factor dynamics. Spike autocorrelation was computed using
equation (12) on input (gray) and liquid (blue) neuron spike trains. Data
points are averaged over 100 MNIST input samples. Error bars are standard
deviation.

###  A.7 NALSM performance with respect to liquid size

NALSM performance increased with the number of neurons in the liquid,
saturating at approximately $8,000$ neurons (Fig. S5)

Figure S5: NALSM accuracy increases with liquid size. NALSM accuracy shown
with respect to number of neurons in the liquid. Data points are averaged over
5 random networks. Error bars are standard deviation.

###  A.8 Number of plastic parameters in NALSM

For NALSM, we counted all IL, LL, and LO connections as either plastic with
STDP or trainable with gradient descent. For NALSM$8000$, the number of LO
connections was constant at $80,000$. The number of IL, LL connections varied
based on the randomly generated liquid. The average number of total
plastic/trainable connections for NALSM$8000$ trained on MNIST was
$1,199,406.70\pm 453.47$ with a maximum(minimum) of $1,200,105$($1,198,916$).
For N-MNIST, the average was $3,033,045.40\pm 268.70$ with a maximum(minimum)
of $3,033,446$($3,032,737$). The significant difference in the number of
plastic connections used for MNIST and N-MNIST training was due to the
$\approx 3$ times larger input layer needed for N-MNIST.

###  A.9 Added computational cost of the LIM astrocyte model

Our proposed method adds a negligible computational cost to the LSM.
Specifically, we used a single astrocyte unit with the same functional form as
the LIF neuron, making it $0.01\%$ of all the LIF neurons used in NALSM8000
(we used a total of $8,784$ input and liquid neurons for MNIST). In terms of
connections, we used $8,784$ neuron-astrocyte connections, which was $0.78\%$
of the number of neuron-neuron connections (we used $1,119,407$ input-liquid
and liquid-liquid connections). Further, we showed in Fig. 4 that even with
$90\%$ of neuron-astrocyte connections removed, NALSM still maintains a
performance advantage versus LSM+AP-STDP and LSM models; in which case only
$878$ neuron-astrocyte connections are used or $0.078\%$ of neuron-neuron
connections. Finally, fixed neuron-astrocyte connections are computationally
less expensive than the plastic neuron-neuron connections, since the ms-
precision STDP mechanism (Eqs. 4, 5, 6) adds extra computations on top of each
neuronal connection that does not exist in the neuron-astrocyte connections.

###  A.10 Curve Fitting

We fit $2$nd and $3$rd degree polynomial functions. All polynomial fit
parameters and residual sum values are shown in Table S2.

Table S2: Polynomial Curve Fitting Parameters Figure/Plot | Degree | Coefficients | Residuals Sum
---|---|---|---
Fig. 1 B | 3 | $(0.1143,-0.7563,1.7492,-0.0658)$ | $0.00143$
Fig. 2 C MNIST | 2 | $(-0.9151,2.7587,-0.6077)$ | $0.00229$
Fig. 2 C N-MNIST | 2 | $(0.3000,0.6104,0.0752)$ | $0.00050$
Fig. 4 MNIST | 3 | $(0.0187,-0.0344,0.0209,0.9561)$ | $0.000000422$
Fig. 4 N-MNIST | 3 | $(0.0044,-0.0084,0.0060,0.9585)$ | $0.000000477$

###  A.11 Hardware

We used Tesla K$80$ GPU to train all models.

unsrtnat bib_append