# Noise-Induced Phase Separation and Local Entropy Production Rate in Scalar
Field Theories Driven by Persistent Noise

Matteo Paoluzzi<EMAIL_ADDRESS>Istituto per le Applicazioni del
Calcolo del Consiglio Nazionale delle Ricerche, Via Pietro Castellino 111
80131 Napoli (NA)  Departament de Física de la Matèria Condensada, Universitat
de Barcelona, C. Martí Franquès 1, 08028 Barcelona, Spain.  Demian Levis
Departament de Física de la Matèria Condensada, Universitat de Barcelona, C.
Martí Franquès 1, 08028 Barcelona, Spain.  UBICS University of Barcelona
Institute of Complex Systems, Martí i Franquès 1, E08028 Barcelona, Spain.
Andrea Crisanti  Dipartimento di Fisica, Sapienza Università di Roma Piazzale
A. Moro 2, I-00185 Rome, Italy.  Ignacio Pagonabarraga  Departament de Física
de la Matèria Condensada, Universitat de Barcelona, C. Martí Franquès 1, 08028
Barcelona, Spain.  UBICS University of Barcelona Institute of Complex Systems,
Martí i Franquès 1, E08028 Barcelona, Spain.

###### Abstract

Within the Landau-Ginzburg picture of phase transitions, scalar field theories
can undergo phase separation because of a spontaneous symmetry-breaking
mechanism that makes homogeneous field configurations unstable. This picture
works in thermodynamics but also in the dynamics of phase separation. Here we
show that scalar non-equilibrium field theories undergo phase separation just
because of non-equilibrium fluctuations driven by a persistent noise. The
mechanism is similar to what happens in Motility-Induced Phase Separation
where persistent motion introduces an effective attractive force. In this
work, we observe that Noise-Induced Phase Separation occurs in a region of the
phase diagram where disordered field configurations would otherwise be stable
at equilibrium. Moreover, looking at the local entropy production rate, we
find that the breaking of time-reversal symmetry is concentrated on the
boundary between the two phases.

††preprint: APS/123-QED

## Introduction.

Dynamical field theories provide a powerful framework for investigating
collective properties in a variety of systems, ranging from critical phenomena
and non-equilibrium phase transitions [1, 2], to growing of interfaces [3].
Such continuum descriptions are also suitable for modeling different materials
[4], cell colonies [5], single-cell motion [6] or phase transitions in cell
aggregates [7, 8, 9, 10].

Focusing our attention on systems described by a scalar order parameter, as in
the case of gas-liquid phase transition, the so-called Model A and Model B (in
the Halperin-Hohenberg notation [1]) can be extended to capture the large-
scale phenomenology of active systems, e.g., collections of self-propelled
agents [11], once opportune non-equilibrium terms have been taken into account
[12, 13, 14, 15]. Moreover, the statistical properties of the noise, which
represents the effect of the fast degree of freedom on the slow ones, might be
another natural source of non-equilibriumness in field theories [16, 17, 18].
In particular, there are no reasons a priori to consider a noise that is
delta-correlated, while a natural choice might be to rather consider a noise
whose two-point correlation function decays exponentially fast on a non-
vanishing time scale $\tau$, much smaller than the experimental time scale
[19, 20].

Models with exponentially correlated noise have been largely employed during
the last years in active matter [21, 22, 23, 24]. They are quite effective for
capturing the properties of non-thermal fluctuations typical of active baths
[25, 26]. Moreover, the persistence time $\tau$ allows to interpolate between
a genuine equilibrium system at $\tau=0$ up to a far from equilibrium system
as $\tau$ increases [24, 27, 28, 29]. Despite the efforts devoted to study
stochastic dynamics driven by correlated noise [30, 31, 32], the impact of
such non-equilibrium dynamics on field theory has been only partially explored
[16, 33, 20, 19].

Figure 1: Noise-Induced Phase Separation (NIPS). (a)-(h) Stationary
configurations of Model A ((a)-(d)) and Model B ((e)-(h)) in a region of the
phase diagram where the equilibrium dynamics leads to a disordered phase
($a(\tau)-a(0)>0$). Upon increasing $\tau$ (Model A $\tau=0.02,0.2,2,10$,
Model B $\tau=0.05,1,10,40$, from left to right), Model A orders and Model B
develops phase separation. The lattice size is $L=100$ ($D=1$). (i) Phase
diagram for Model A and B. The green region indicates where the probability
distribution function $\mathcal{P}[\varphi]$ is double-peaked. The dashed
magenta line is the one parameter fit to Eq. (6). (j) Rescaled Binder
parameter for $\tau=1$ with $\nu=1$ and different system sizes from $L=40$ to
$L=80$ (see legend). (k) Scaling of the susceptibility for $\tau=1$ is
consistent with $\eta=1/4$.

In this work, we perform a systematic study of scalar field theories in $2d$,
driven out-of-equilibrium by an exponentially correlated noise. We consider
the case of both, non-conserved (Model A) and conserved (Model B) scalar order
parameter dynamics. We document that the non-equilibrium noise is the driver
of phase separation in a region of the phase diagram where the corresponding
equilibrium system does not display any ordered phase. This Noise-Induced
Phase Separation (NIPS) reminds us of Motility-Induced Phase Separation
(MIPS), where self-propulsion destabilizes the homogenous states, stable at
equilibrium. From the computation of the entropy production rate, we observe
that time-reversal symmetry breaking is concentrated around the boundary
between the two coexisting phases.

## Model.

We consider the relaxation dynamics of a scalar field
$\varphi\equiv\varphi(\bm{x},t)$ in the presence of a persistent noise
$\psi\equiv\psi(\bm{x},t)$. $\varphi$ represents the slow variable we are
interested in, as in the case of density fluctuations in the gas-liquid phase
transition. The dynamics of $\varphi$ results from the competition between a
deterministic force $F$ and a fluctuating one $\psi$. The latter represents
the effect of the fast degrees of freedom on $\varphi$. Instead of considering
the stochastic force as a zero mean and delta correlated noise, we assume
$\psi$ to be an annealed variable characterized by a time scale $\tau$ which
is another control parameter of the model (in the limit $\tau\to 0$, we
recover a delta-correlated noise). The dynamics reads

$\displaystyle\dot{\varphi}=F[\varphi]+\psi$ (1)

with the stochastic field $\psi$ such as $\langle\psi\rangle=0$ and

$\displaystyle\langle\psi(\bm{x},t)\psi(\bm{y},s)\rangle=2DL(\bm{x}-\bm{y},t-s)\;,$
(2)

where $D$ sets the strength of the noise and the operator $L$ keeps into
account both, a suitable differential operator for describing conserved or
non-conserved dynamics, and the time-correlation function of the noise. The
deterministic force $F$ combines conservative terms, i.e. reversible, and non-
conservative ones, i.e., non-integrable terms as in the case of the
$\lambda|\nabla\varphi|^{2}$ term in KPZ. In the following, we restrict our
study to the case where $F[\varphi]$ can be written as the functional
derivative of the standard Landau-Ginzburg energy functional

$\displaystyle H_{LG}[\varphi]=\int
d\bm{x}\,\left[\frac{\mu}{2}(\nabla\varphi)^{2}+\frac{a}{2}\varphi^{2}+\frac{u}{4}\varphi^{4}\right]\;,$
(3)

so that non-equilibrium is caused solely by the presence of the time-
correlated noise $\eta$. The parameter $a$ sets the distance from the
equilibrium critical point $a\\!=\\!0$. In finite dimensions, this critical
value will be renormalized to lower values because fluctuations below the
upper critical dimensions are not negligible (in numerical simulations, by
discretization effects too).

Figure 2:  Non-equilibrium dynamics enhances phase separation. Phase diagrams
of Model B for different values of $\tau$ (see legend). The green area
indicates the phase separation region (corresponding to $\tau\\!=\\!40$) that
increases monotonically for increasing values of $\tau$.

In Model A, the order parameter is not conserved and the dynamics is specified
by the following choices

$\displaystyle L(\bm{x},t)$
$\displaystyle=\delta(\bm{x})K(t)\,,\;\;K(t)\equiv\frac{e^{-|t|/\tau}}{\tau}\,,\;\;F[\varphi]=-\frac{\delta
H_{LG}}{\delta\varphi}\;.$

The dynamics in Model B can be written as a continuity equation in terms of
the current $\bm{J}$ so that

$\displaystyle\dot{\varphi}=-\bm{\nabla}\cdot\bm{J}\,,\;\;\bm{J}\equiv-\bm{\nabla}\frac{\delta
H_{LG}}{\delta\varphi}+\bm{\theta}$ (4)

where the statistical properties of each component of the vectorial noise
$\bm{\theta}$ satisfies Eq. (2). The dynamics can be made in the form of Eq.
(1) upon defining $F[\varphi]$ and $L$ as follows [2]

$\displaystyle F[\varphi]\equiv\nabla^{2}\frac{\delta
H_{LG}}{\delta\varphi}\,,\;\;L(\bm{x},t)\equiv-\nabla^{2}\delta(\bm{x})K(t)\;.$

## Phase diagram & NIPS.

We start our study by considering the mean-field (MF) picture of the model,
i.e., neglecting any spatial dependency of $\varphi$ within an effective
equilibrium approach. Through the Markovian approximation named as Unified
Colored Noise (UCN) approximation [34], it has been shown that the correlated
noise shifts the critical point of the Landau model at higher temperatures,
i.e, the critical value $a_{c}(\tau)$ is in general $a_{c}(\tau)>a$ [33].
Although UCN dynamics does not reproduce the real trajectories [35], it
provides useful insight into the stationary properties of the system,
especially when the potential generating deterministic forces in the system
are characterized by positive curvature, as in the case of a $\varphi^{4}$
theory approaching the critical point from above. However, in order to
overcome any pathology that might arise, we employ a small-$\tau$
approximation to UCN and we work within that limit.

In the small-$\tau$ regime, we can write
$H_{eff}\\!\simeq\\!H_{LG}+\frac{\tau}{2}(H_{LG}^{\prime})^{2}-\tau
H^{\prime\prime}_{LG}$, where the prime indicates the derivative with respect
to $\varphi$. A phase transition to $\varphi\\!\neq\\!0$ takes place if the
configuration $\varphi\\!=\\!0$ is not stable anymore. We can check the
stability of $\varphi\\!=\\!0$ looking at the second derivative of $H_{eff}$,
given by
$H^{\prime\prime}_{eff}\\!=\\!H_{LG}^{\prime}+\tau\left[(H_{LG}^{\prime\prime})^{2}+H_{LG}^{\prime}H_{LG}^{\prime\prime\prime}\right]-\tau
H_{LG}^{\prime\prime\prime\prime}$. Since
$H^{\prime\prime}_{eff}[0]\\!=\\!-3u\tau$, negative for any $\tau\\!>\\!0$,
the system undergoes a phase transition to $\varphi\\!\neq\\!0$. Since the
parameter driving the phase transition is the non-equilibrium noise, we refer
to this mechanism as NIPS.

The stationary homogeneous configurations of $\varphi$ are thus regulated by
[36]

$\displaystyle H_{eff}$
$\displaystyle=\frac{\tilde{a}}{2}\varphi^{2}+\frac{\tilde{u}}{4}\varphi^{4}+O(\varphi^{6})$
(5) $\displaystyle\tilde{a}$ $\displaystyle\equiv a(1+\tau a)-3u\tau$
$\displaystyle\tilde{u}$ $\displaystyle\equiv u(1+4\tau)\;.$

We obtain that the combination of non-equilibrium noise, parametrized by
$\tau$, and non-linear interactions, whose intensity is tuned by $u$,
renormalizes the coupling constants $a$ and $u$ in an effective way, i.e.,
$\tilde{u}\\!\geq\\!u$ so that the non-linear interaction becomes more
important, and $\tilde{a}=\tilde{a}(\tau u)$, so that it can change its sign.
Without non-linear interactions ($u\\!=\\!0$), the location of the transition
remains untouched at $a\\!=\\!0$. For $u\\!>\\!0$, the shift of the critical
point is given by $\tilde{a}\\!=\\!0\\!\equiv\\!a_{c}(\tau)$, that is

$\displaystyle
a_{c}(\tau)=\frac{1}{2\tau}\left(\sqrt{1+12u\tau^{2}}-1\right)\;.$ (6)

Moving out from the MF regime (and out from the effective equilibrium), it is
not clear how finite dimension fluctuations and non-equilibrium dynamics
change the MF picture in $2d$.

We now move to $2d$ using numerical simulations. As in MF, we observe that the
correlated noise drives a phase transition. This is qualitatively shown in
Fig. (1) where we report representative configurations of Model A ((a)–(d))
and B ((e)–(h)) in a region of the phase diagram where the corresponding
equilibrium model is in a disordered homogeneous state. As one can see, for
increasing values of $\tau$, disordered configurations become unstable so that
phase transition takes place (in the case of Model B, the initial
configuration is a drop $\varphi\\!=\\!1$ of radius $R\\!=\\!20$ immersed into
a background configuration $\varphi\\!=\\!-1$, Model A starts from a random
configuration).

To make quantitative progress, we compute the phase diagram of the model in
both cases, Model A and Model B. The result is shown in Fig. (1i). We observe
that, by tuning $\tau$ the system undergoes a non-equilibrium phase transition
in a wide region of the phase diagram where the corresponding equilibrium
system is homogeneous. We compare the numerical results with the theoretical
prediction (6) using $u$ as a fitting parameter. As one can see, the theory
reproduces remarkably well the order-disorder transition in a wide range of
$\tau$. This is counterintuitive since eq. (6) has been obtained within the
small-$\tau$ limit. We can rationalize this by noticing that the order
parameter $\varphi$ in the Landau theory of continuous transition is supposed
to be small at the transition so that the correction to the mass $\tau
H^{\prime\prime}_{LG}\simeq\tau u\varphi^{2}$ around the critical point is
small in a wide range of $\tau$.

In Model A, we now compute the Binder parameter
$U_{4}\\!\equiv\\!1-\langle\varphi^{4}\rangle/3\langle\varphi^{2}\rangle^{2}$
for system sizes $L\\!=\\!40,50,60,70,80$ and $\tau\\!=\\!1$. As shown in Fig.
(1j), we observe a good scaling collapse within the Ising universality class,
i.e., $\nu\\!=\\!1$. Measuring the exponent $\eta$ through the scaling of the
peak of the susceptibility $\chi$, we observe again a value consistent with
Ising universality class (Fig. (1k)).

Figure 3:  Entropy production rate and phase separation. Total entropy
production rate $\mathcal{S}$ as a function of the distance from the
transition point, i.e., $a_{c}(\tau)-a$, in the case of Model B
($\tau\\!=\\!1$). $\mathcal{S}$ shows a crossover at the transition point.
Insets: Representative stationary configuration of the conserved field
$\varphi$ and the corresponding local entropy production rate $\sigma$.

Next, we measure how non-equilibrium fluctuations modify the phase separation
region in Model B. We thus performed numerical simulations for
$\tau=0.1,1,10,40$ and several values of the initial density
$\varphi_{a}\equiv\int d\bm{x}\,\varphi(\bm{x},0)$. Fig. (2) reports the
phase-separation region as $\tau$ increases. We observe that not only $\tau$
is the trigger for the phase separation, but it also quantitatively impacts
the size of the phase-separated region making it larger and larger for
increasing values of $\tau$.

## Time-Reversal Symmetry Breaking (TRSB).

As NIPS is driven by the non-equilibrium nature of the noise, a natural
question is whether the noise is just a trigger for an equilibrium-like
transition or other hints of non-equilibriumness are still present. To answer
this question we look at the total entropy production rate $\mathcal{S}$,
which is the natural observable to quantify the degree of TRSB in the system.
$\mathcal{S}$ is defined as the long-time behavior of Kullback-Leibler
divergence [37]

$\displaystyle\mathcal{S}\equiv\lim_{T\to\infty}\frac{1}{T}\left\langle\log\frac{P[\varphi]}{P_{R}[\varphi]}\right\rangle\;.$
(7)

with $P[\varphi]$ indicating the probability of the path $\varphi(\bm{x},t)$
with $t\in[t_{0},T]$ and $P_{R}[\varphi]$ the probability of the time-reversed
path $\varphi_{R}$ obtained through the transformation $t\to T-t$. In the case
of a field theory, $\mathcal{S}$ can be written in terms of a spatial resolved
entropy production rate $\sigma(\bm{x})$ so that $\mathcal{S}\\!=\\!\int
d\bm{x}\,\sigma(\bm{x})$ where $\sigma(\bm{x})$ is a model-dependent composite
operator of the field $\varphi$ [13, 20, 38]. In the case of Model B, the
computation of $\mathcal{S}$ brings to [36]

$\displaystyle\sigma(\bm{x})=\frac{\tau^{2}}{2D}\left\langle\dot{\varphi}^{3}\frac{\delta^{3}H_{LG}}{\delta\varphi^{3}}\right\rangle$
(8)

where the angular brackets indicate the averaging over time performed on a
stationary configuration.

We employ (8) for computing $\mathcal{S}$ in our simulations. Fig. (3a)
reports $\mathcal{S}$ normalized to its maximum value in the case of
$\tau\\!=\\!1$. As an initial condition, we consider a drop of radius
$R\\!=\\!20$ (here $L\\!=\\!60$). We observe that $\mathcal{S}$ increases as
the system approaches the transition. Similar to recent studies on the
behavior of $\mathcal{S}$ across non-equilibrium phase transitions [39, 40],
$\mathcal{S}$ undergoes a crossover, i.e., it follows different scaling
regimes before and after the transition. Because $\mathcal{S}$ is non-zero,
despite the phase separation being equilibrium-like, it is not just triggered
by non-equilibrium fluctuations but the persistent noise continuously
maintains the two phases separated, breaking continuously the time-reversal
symmetry.

A natural question is whether TRSB is accompanied by some kind of non-
equilibrium pattern formation. To answer this question, we look at the heat
map produced in space by $\sigma(\bm{x})$. In Fig. (3) we display the field
configurations and the corresponding $\sigma(\bm{x})$ (see the insets). In the
symmetric phase, $\varphi$ is disordered in space and the same happens for
$\sigma$. Once the system phase separates, the corresponding map of $\sigma$
develops a pattern at the boundary between the two phases, indicating that
most TRSB is concentrated in that region. This is precisely the kind of
pattern predicted in the picture of Active Model B [13] and tested against
experiments and simulations of Active Systems in a model-independent fashion
[41]. We can rationalize that from (8) noticing that terms proportional to
$\varphi^{m}\nabla^{2}\varphi$, with $m>1$, that give a contribution to
$\sigma$ on the boundary between the two phases, arise once we replace
$\dot{\varphi}$ by eq. (1), so that
$\sigma\sim\langle(\mu\nabla^{2}\varphi-a\varphi+u\varphi^{2}+\psi)^{3}\varphi\rangle$.
These terms are irrelevant in the Renormalization Group sense [20], however,
similarly to the case of Active Model B, they contribute to a space-dependent
TRSB. In the case of Model A, the order parameter is not conserved and thus
interfaces are suppressed in favor of homogeneous phases that spontaneously
break the $\varphi\\!\to\\!-\varphi$ symmetry. In this situation, $\sigma$
might develop anomalous fluctuations around the upper critical dimensions
$d_{c}\\!=\\!4$ [20, 14], which is far away from our numerical study.

## Discussion.

We have shown that scalar field theories in MF and $2d$ undergo a phase
separation driven by non-equilibrium noise and controlled by its persistence
time $\tau$. The phase separation takes place in a region of the phase diagram
where the corresponding equilibrium model is homogeneous. We showed that the
emerging phenomenology can be rationalized within a simple MF theory within
the small-$\tau$ limit. The theory highlights how the combination of non-
linearities (parametrized by $u$) and non-equilibrium noise (parametrized by
$\tau$) can trigger the transition. We computed numerically the critical
exponents of the theory in the case of Model A. The exponents are compatible
with the Ising universality class, i.e., $\nu=1$ and $\eta\\!=\\!1/4$, (we
stress that devitations from Ising exponents require a separate work). The
phenomenology is similar to that of MIPS in active particle models where
homogeneous density profiles become unstable because of the active motion that
generates feedback between local density and local velocity [42].

We have quantified the non-equilibirum character of NIPS by computing the
entropy production rate $\mathcal{S}$, which quantifies the degree of TRSB in
the system. The computation shows that the non-equilibrium nature of the phase
separation is due to the combination of non-linear interactions and non-
equilibrium dynamics: both ingredients are fundamental and complementary. In
other words, even though the critical point of the phase separation belongs to
the Ising universality class, the low-temperature phase-separated phase is
maintained alive because of the continuous energy injection on the mesoscopic
scale performed by the noise, rather than by the standard spontaneous
symmetry-breaking mechanism of the mesoscopic equilibrium description. This
difference is signaled by a non-vanishing entropy production rate for any
arbitrary small value of $\tau$.

Finally, we observed that most of the TRSB is concentrated at the boundaries
between the two coexisting phases. It is worth noting that the picture we
obtain is quite similar to that of Active Model B where the entropy production
rate is non-zero at the interface because of the presence of local and non-
integrable terms. On the other hand, in our model, TRSB at the interface is
due solely to the presence of non-equilibrium fluctuations that couple with
each other because of non-linear interactions.

We highlight that an exponentially correlated noise naturally emerges in the
continuum description. This is because the stochastic forces in the dynamics
of the slow degrees of freedom represent the effect of time scales smaller
than the mesoscopic time scale. Because of the central limit theorem, the
Gaussian assumption for the noise is reasonable but, on the other hand, the
instantaneous decorrelation of fast fluctuations is a suitable simplification
of the problem [2]. In Active Matter, one usually considers perturbations due
to delta-correlated noise on a set of continuum equations [43], e.g., the
density field $\rho$, such as
$\partial_{t}\rho\\!=\\!-\nabla(J-D(\rho)\nabla\rho)$ with a current $J$
decaying on a finite time-scale, i.e., $\partial_{t}J\\!=\\!-D_{r}J+\dots$.
The ground assumption for performing the linear stability analysis is to
consider $J$ as a fast variable so that $\dot{J}\\!=\\!0$ [11, 44]. Once we
remove this simplification $\nabla J$ acts as an Ornstein-Uhlenbeck field on
$\rho$: this does not change the linear stability analysis. However, our
results suggest that, once we include non-linear interactions, the Ornstein-
Uhlenbeck field destabilizes homogenous profiles even in the small-$\tau$
regime.

## Acknowledgments.

D.L. acknowledges DURSI for financial support under project 2021SGR00673. I.P.
acknowledges MICINN, DURSI, and SNSF for financial support under Projects No.
PGC2018-098373-B-I00, No. 2017SGR-884, and No. 200021-175719, respectively.

## References

  * Hohenberg and Halperin [1977] P. C. Hohenberg and B. I. Halperin, _Reviews of Modern Physics_ , 1977, 49, 435.
  * Täuber [2014] U. C. Täuber, _Critical dynamics: a field theory approach to equilibrium and non-equilibrium scaling behavior_ , Cambridge University Press, 2014.
  * Kardar _et al._ [1986] M. Kardar, G. Parisi and Y.-C. Zhang, _Phys. Rev. Lett._ , 1986, 56, 889–892.
  * Kendon _et al._ [2001] V. M. Kendon, M. E. Cates, I. Pagonabarraga, J.-C. Desplat and P. Bladon, _Journal of Fluid Mechanics_ , 2001, 440, 147–203.
  * Cates _et al._ [2010] M. E. Cates, D. Marenduzzo, I. Pagonabarraga and J. Tailleur, _Proceedings of the National Academy of Sciences_ , 2010, 107, 11715–11720.
  * Tjhung _et al._ [2012] E. Tjhung, D. Marenduzzo and M. E. Cates, _Proceedings of the National Academy of Sciences_ , 2012, 109, 12381–12386.
  * Ziebert _et al._ [2012] F. Ziebert, S. Swaminathan and I. S. Aranson, _Journal of The Royal Society Interface_ , 2012, 9, 1084–1092.
  * Loewe _et al._ [2020] B. Loewe, M. Chiang, D. Marenduzzo and M. C. Marchetti, _Phys. Rev. Lett._ , 2020, 125, 038003.
  * Wenzel and Voigt [2021] D. Wenzel and A. Voigt, _Phys. Rev. E_ , 2021, 104, 054410.
  * Mueller _et al._ [2019] R. Mueller, J. M. Yeomans and A. Doostmohammadi, _Phys. Rev. Lett._ , 2019, 122, 048004.
  * Marchetti _et al._ [2013] M. C. Marchetti, J. F. Joanny, S. Ramaswamy, T. B. Liverpool, J. Prost, M. Rao and R. A. Simha, _Rev. Mod. Phys._ , 2013, 85, 1143–1189.
  * Wittkowski _et al._ [2014] R. Wittkowski, A. Tiribocchi, J. Stenhammar, R. J. Allen, D. Marenduzzo and M. E. Cates, _Nature communications_ , 2014, 5, 4351.
  * Nardini _et al._ [2017] C. Nardini, É. Fodor, E. Tjhung, F. Van Wijland, J. Tailleur and M. E. Cates, _Physical Review X_ , 2017, 7, 021007.
  * Caballero and Cates [2020] F. Caballero and M. E. Cates, _Phys. Rev. Lett._ , 2020, 124, 240604\.
  * Cates [2012] M. E. Cates, _Reports on Progress in Physics_ , 2012, 75, 042601\.
  * Sancho _et al._ [1998] J. Sancho, J. Garcia-Ojalvo and H. Guo, _Physica D: Nonlinear Phenomena_ , 1998, 113, 331–337.
  * García-Ojalvo _et al._ [1992] J. García-Ojalvo, J. M. Sancho and L. Ramirez-Piscina, _Physical Review A_ , 1992, 46, 4670.
  * García-Ojalvo and Sancho [1994] J. García-Ojalvo and J. M. Sancho, _Phys. Rev. E_ , 1994, 49, 2769–2778.
  * Maggi _et al._ [2022] C. Maggi, N. Gnan, M. Paoluzzi, E. Zaccarelli and A. Crisanti, _Communications Physics_ , 2022, 5, 1–10.
  * Paoluzzi [2022] M. Paoluzzi, _Phys. Rev. E_ , 2022, 105, 044139.
  * Szamel [2014] G. Szamel, _Physical Review E_ , 2014, 90, 012111.
  * Maggi _et al._ [2015] C. Maggi, U. M. B. Marconi, N. Gnan and R. Di Leonardo, _Scientific reports_ , 2015, 5, 1–7.
  * Farage _et al._ [2015] T. F. F. Farage, P. Krinninger and J. M. Brader, _Phys. Rev. E_ , 2015, 91, 042310.
  * Fodor _et al._ [2016] E. Fodor, C. Nardini, M. E. Cates, J. Tailleur, P. Visco and F. van Wijland, _Phys. Rev. Lett._ , 2016, 117, 038103.
  * Maggi _et al._ [2014] C. Maggi, M. Paoluzzi, N. Pellicciotta, A. Lepore, L. Angelani and R. Di Leonardo, _Phys. Rev. Lett._ , 2014, 113, 238303.
  * Maggi _et al._ [2017] C. Maggi, M. Paoluzzi, L. Angelani and R. Di Leonardo, _Scientific reports_ , 2017, 7, 17588.
  * Caprini _et al._ [2019] L. Caprini, U. M. B. Marconi, A. Puglisi and A. Vulpiani, _Journal of Statistical Mechanics: Theory and Experiment_ , 2019, 2019, 053203\.
  * Dal Cengio _et al._ [2021] S. Dal Cengio, D. Levis and I. Pagonabarraga, _Journal of Statistical Mechanics: Theory and Experiment_ , 2021, 2021, 043201.
  * O’Byrne _et al._ [2022] J. O’Byrne, Y. Kafri, J. Tailleur and F. van Wijland, _Nature Reviews Physics_ , 2022, 4, 167–183.
  * Jung and Hänggi [1987] P. Jung and P. Hänggi, _Physical review A_ , 1987, 35, 4464\.
  * Fox [1986] R. F. Fox, _Physical Review A_ , 1986, 33, 467.
  * McKane _et al._ [1990] A. J. McKane, H. C. Luckock and A. J. Bray, _Phys. Rev. A_ , 1990, 41, 644–656.
  * Paoluzzi _et al._ [2016] M. Paoluzzi, C. Maggi, U. Marini Bettolo Marconi and N. Gnan, _Phys. Rev. E_ , 2016, 94, 052602.
  * Hänggi and Jung [1995] P. Hänggi and P. Jung, _Advances in chemical physics_ , 1995, 89, 239–326.
  * Crisanti and Paoluzzi [2023] A. Crisanti and M. Paoluzzi, _Phys. Rev. E_ , 2023, 107, 034110\.
  * [36] _See Supplemental Material at ……, which contains additional details on the model, additional results complementing those shown in the main text, and includes Refs. [….]._
  * Lebowitz and Spohn [1999] J. L. Lebowitz and H. Spohn, _Journal of Statistical Physics_ , 1999, 95, 333–365.
  * Pruessner and Garcia-Millan [2022] G. Pruessner and R. Garcia-Millan, _arXiv preprint arXiv:2211.11906_ , 2022\.
  * Alston _et al._ [2023] H. Alston, L. Cocconi and T. Bertrand, _arXiv preprint arXiv:2304.08661_ , 2023\.
  * Suchanek _et al._ [2023] T. Suchanek, K. Kroy and S. A. Loos, _arXiv preprint arXiv:2303.16701_ , 2023\.
  * Ro _et al._ [2022] S. Ro, B. Guo, A. Shih, T. V. Phan, R. H. Austin, D. Levine, P. M. Chaikin and S. Martiniani, _Phys. Rev. Lett._ , 2022, 129, 220601.
  * Tailleur and Cates [2008] J. Tailleur and M. E. Cates, _Phys. Rev. Lett._ , 2008, 100, 218103\.
  * Marconi _et al._ [2021] U. M. B. Marconi, L. Caprini and A. Puglisi, _New Journal of Physics_ , 2021, 23, 103024.
  * Fily and Marchetti [2012] Y. Fily and M. C. Marchetti, _Phys. Rev. Lett._ , 2012, 108, 235702\.


# FPGA Implementation of An Event-driven Saliency-based Selective Attention
Model ††thanks: This work was supported by the European Research Council (ERC)
under the European Union’s Horizon 2020 Research and Innovation Program Grant
Agreement No. 724295 (NeuroAgents).

Hajar Asgari  Institute of Neuroinformatics
University of Zurich and ETH Zurich
Zurich, Switzerland
<EMAIL_ADDRESS>Nicoletta Risi  Bio-Inspired Circuits and Systems Lab
University of Groningen
Groningen, Netherlands
<EMAIL_ADDRESS>Giacomo Indiveri  Institute of Neuroinformatics
University of Zurich and ETH Zurich
Zurich, Switzerland
<EMAIL_ADDRESS>
###### Abstract

Artificial vision systems of autonomous agents face very difficult challenges,
as their vision sensors are required to transmit vast amounts of information
to the processing stages, and to process it in real time. One first approach
to reduce the data transmission is to use event-based vision sensors, whose
pixels produce events only when there are changes in the input. However, even
for event-based vision, transmission and processing of visual data can be
quite onerous. Currently these challenges are solved by using high-speed
communication links and powerful machine vision processing hardware. But if
resources are limited, instead of processing all the sensory information in
parallel, an effective strategy is to divide the visual field into several
small sub-regions, choose the region of highest saliency, process it and shift
serially the focus of attention to regions of decreasing saliency. This
strategy, commonly used also by the visual system of many animals, is
typically referred to as “selective attention”. Here we present a digital
architecture implementing a saliency-based selective visual attention model
for processing asynchronous event-based sensory information received from a
Dynamic Vision Sensor (DVS). For ease of prototyping, we use a standard
digital design flow and map the architecture on a Field Programmable Gate
Array (FPGA). We describe the architecture block diagram highlighting the
efficient use of the available hardware resources demonstrated through
experimental results exploiting a hardware setup where the FPGA interfaced
with the DVS camera.

###### Index Terms:

Neuromorphic engineering, selective visual attention, event-driven, foveal
vision, FPGA.

##  I Introduction

Event-based vision sensors are becoming increasingly popular for a wide range
of vision processing applications that require low latency and low power
consumption [1]. As these constraints are the same ones faced by biological
vision systems, it can be useful to study the computational strategies used by
animal vision systems to optimize the performance of artificial event-based
ones. In particular, a prominent strategy that biology uses to carry out
efficient vision processing with limited resources is to use _selective
attention_ [2, 3, 4]. Instead of simultaneously processing all the information
that can be provided by the input sensors, this strategy serially selects
_salient_ sub-regions of the input space shifting from sub-region to sub-
region, and processing them sequentially. These sub-regions are selected by
choosing the region of the visual space that has the highest value in the
corresponding “saliency map”. In Itti’s bottom-up selective attention model
[4], saliency maps are computed by combining multiple visual features, such as
color contrast, texture, motion, etc. [4].

As one of the most relevant features in these saliency maps is given by motion
cues and dynamic changes in the visual scene (e.g., produced by stationary but
flashing LEDs), here we focus on the activity of the DVS sensor [5], which
naturally correlates with these salient features [6]. The use of this sensor
reduces the amount of data transferred to the processing areas and,
consequently, the amount of power required to process it. Based on this, we
present an event-based visual processing system that uses a biologically
plausible computational model of a saliency-based form of bottom-up attention,
to select and sequentially process sub-regions of the visual field. While
early attempts to implement selective attention models in neuromorphic
hardware have focused on mixed-signal analog/digital implementations [7, 8, 9,
10, 11], few have been done using a standard digital design flow [12, 13].
Here we implement a saliency-based event-driven attention mechanism at pixel
level which depend on the exponential decay of the pixel states. We use
digital FPGA devices for fast prototyping, while still allowing real-time
processing of events streamed from the DVS. The active-vision system that we
present carries out computation in parallel using an event-based asynchronous
communication infrastructure that employs the Address-Event Representation
(AER) [14, 15, 16].

In Section II we present the details of the selective attention model. Section
III describes the proposed event-based attention mechanism. In Section IV we
introduce the hardware architectures, and in Section V we presents the
simulation and experimental results of the proposed system. Finally, Section
VI concludes the paper.

##  II Background

Selective attention is a powerful computational strategy used by biological
systems to optimize the use of the limited computing resources and minimize
latency and reaction times. It acts by selecting the most salient location of
the visual scene, ignoring distractors and irrelevant information, and
allocating the system’s computational resources for fast and accurate
processing of the information in that region. Subsequently, the system chooses
other regions of the visual scene with decreasing saliency, processing them in
a sequential manner. The choice of the regions to process can be influenced by
“bottom-up” stimuli (e.g., such as high contrast or moving objects) or “top-
down” preferences (such regions of the scene that ignore the sky, or that
include faces). Several computational models of selective attention under both
top-down and bottom-up influences have been proposed [17, 18, 19, 20]. The
first explicit, biologically plausible computational model of bottom-up
saliency-based selective visual attention was proposed by Koch and Ullman in
1985 [3].

Inspired by biological vision systems, the first processing stage in any
bottom-up attention model is computing different features such as intensity,
contrast, color opponency, orientation, direction, or motion [4]. The
different feature maps contribute with different weights to create a unique
saliency map. The strength of the feature contributions can be influenced by
top-down modulation or training [4]. A Winner-Take-All (WTA) network selects
the most salient location in the saliency map and focuses the system’s
attention to it. As the region is being attended, an inhibition of return
mechanism is used to suppress it, so that less salient regions can win the WTA
competition. Depending on the parameters governing its dynamics, this
mechanism can allow the system to switch only between the two most salient
regions, the top three, and so on.

To test selective attention in real-time visual processing tasks, several
event-driven saliency-based models have been proposed [21, 20], as well as
their hardware implementations, using neuromorphic circuits [22, 23, 11], and
using FPGAs [12, 13]. However, almost all the FPGA implementations made use of
frame-based cameras [13], and very few (only one to our knowledge) interfaced
to silicon retina devices [12]. In this paper we proposed a digital event-
driven saliency-based attention model implemented on FPGA for prototyping to
process event-streams of the DVS camera.

##  III Event-based Saliency Attention Model With Top-down Modulation

Here we present our FPGA-compatible event-based selective attention
architecture. As bottom-up selective attention models are especially appealing
for hardware implementations, due to their modular and easily expandable
nature [10], we focus the bulk of our work on these sub-sets of models. First,
we describe how the model processes visual inputs from event-cameras and
exploits the event-based nature of these cameras to compute saliency from
high-contrast changing visual stimuli; then we extend the bottom-up saliency
model to incorporate a top-down modulation mechanism, which allows steering
the focus of attention (FOA) according to the target task.

###  III-A Event-based Bottom-up saliency

As opposed to the original computational model [17], the system we designed
uses only one feature channel that encodes the changes present in the visual
scene, sensed by the DVS. Event-camera pixels, such as those present in the
DVS, operate independently and respond only to local changes in the light
intensity, which makes them inherently act as local edge detectors of moving
objects. In this respect, saliency is defined as the region of the visual
scene that produces the highest number of events, which in turn corresponds to
the area that has the highest number of high contrast moving objects.

Let the FOA be defined as a window of size $[m_{x},m_{y}]$ centered around the
most salient pixel. The event $e(x,y,t,p)$ represents the brightness change
detected by the pixel with coordinates (x,y) at time t and with polarity p
(with p=1 for contrast increases, on ON-event, and p=-1, or OFF-event, for
contrast decreases). Then, the most salient pixel at time t can be defined as
follows:

$P^{*}(t)=\max_{P}s_{P}(t)$

with $s_{P}$ being the pixel state at time t. See IV-C for a detailed
description of the algorithm, which updates the pixels state in an event-based
manner.

###  III-B Top-down Biasing

After receiving a pixel event and extracting information about the pixel’s
location, we can have a top-down bias ahead of the bottom-up processing. The
top-down biasing can be executed in two methods: top-down gating and top-down
modulation. In the “top-down gating” approach, all the areas out of a
predefined region of interest are cut out. In the “top-down modulation”
approach, all the arriving pixels are processed but the changing gain of the
state for each pixel is modulated according to the predefined conditions for
the top-down biasing.

##  IV Hardware Design and Implementation

Fig. 1 shows a schematic of the proposed architecture, which consists of seven
blocks. The Handshake Receiver (HSR) handles the input events. HSR implements
the standard 4-phase handshake AER protocol between the sender and the
receiver block. The output events are monitored through a USB connection but
can also be sent to the other event-based devices using a Handshake Sender
(faded HSS block in Fig. 1). To prevent meta-stable states at the interface
between different clock domains, a double flip-flop synchronizer (FMS) is
applied before the input handshake. The core blocks of the architecture are
described in more detail in the following sections.

Figure 1: High-level block diagram of the network architecture. FMS: Flip-flop
Meta-stability Synchronizer. HSR: Handshake Receiver. DPE (Exp): Data
Processing Element in Exploring Mode. DPE (Fov): Data Processing Element in
Fovea Mode. TDG: Top-Down Gating. TDM: Top-Down Modulation. SAL: Saliency
block. HSS (faded block): Handshake Sender.

###  IV-A Data Processing Element(DPE)

The event-camera included in our architecture is the DAVIS240C which employs a
serial data format, i.e. x and y coordinates of the events are sent one after
the other [5, 24]. Thus, the Data Processing Element block (or DPE) takes care
of generating the 2D event coordinate by merging x and y input words and
forwards the resulting pixel_ID to the receiver block. Two instances of this
block are used, which differ in the spatial resolution of the output events.

  * •

DPE Exploring, or DPE (Exp), which operates on the full input resolution,
extracts the pixel_ID from the input events and forwards it to the Saliency
block (SAL); SAL defines the most salient pixel according to the input events
and a history of the previous events.

  * •

DPE Fovea, or DPE (Fov), which receives the coordinate of the most salient
pixel by the SAL block and extracts only the events inside the FOA.

###  IV-B Top-Down Biasing Blocks

After exploring the input events and extracting the pixel_ID, the top-down
mechanism works on top of the bottom-up saliency. In the proposed system, the
top-down biasing mechanism is implemented using two different methods. Both
are explained in more detail in III-B.

####  IV-B1 Top-Down Gating (TDG)

This block receives the extracted pixel_ID from DPE (Exp) block. When this
block is enabled, via the input parameter Enable_TDG, input events cross the
TDG only if they meet the input top-down biasing conditions (TDB_parameters).

####  IV-B2 Top-Down Modulation (TDM)

TDM also receives the pixel_IDs from the DPE (Exp). This block handles all the
input events with a modulated state-changing gain. The amplitude of the
modulation depends on the top-down biasing conditions. The highest modulation
happens when the input events meet the TDM requirements.

###  IV-C Saliency block (SAL)

Figure 2: High-level block diagram of the saliency block (SAL). This block
receives the input pixel_ID and determines the most salient pixel.

Fig. 2 illustrates the core blocks of the Saliency Block (or SAL)
architecture. SAL extracts the location of the most salient pixel. In the SAL
architecture two separate Dual Port BRAM are used: RAM_FR stores the pixel
state $s_{P}(t)$ as 21-bits registers (with fixed-point representation: s=1
bit for the sign, i=12 integer bits, and f=8 fractional bits); RAM_TIME stores
timestamp of the last event generated at each pixel location.

Let $P^{*}$ be the current most salient pixel location with state $s^{*}$,
which is selected based on all the previous input events. At each incoming
event from pixel $P$ at time point $t$, the SAL block performs the following
steps:

  1. 1.

Read the last pixel event time stamp $t_{old}$ from the corresponding entry of
the RAM_TIME.

  2. 2.

Read the latest pixel state $s_{P(t_{old})}$ at the last pixel event timestamp
$t_{old}$ from the corresponding entry of the RAM_FR.

  3. 3.

Compute the updated pixel state $s_{P}(t)$ as follows:

$s_{P}(t)=1+s_{P(t_{old})}\cdot\exp\left[(t_{old}-t)/\tau\right]$ (1)

with $\tau$ stored as input parameter. This operation is handled by Exp_Func
block (Fig. 2), where the exponential decay term is implemented using the
piece-wise linear approximation, to reduce the hardware resource utilization
on the FPGA.

  4. 4.

Store the new pixel state and timestamp in the corresponding BRAM entries.

  5. 5.

Update the state of the current most salient pixel by repeating steps 1, 2,
and 3 for $P^{*}$.

  6. 6.

Compare the new pixel state to the state of the current most salient pixel
$P^{*}$. If $s_{P}(t)>s*$:

     * •

Update the most salient pixel location, which is forwarded to the DPE(Fov)
block.

     * •

Excite the state of the updated most salient pixel as the center of FOV or
$P^{*}$:

$s[x,y]=s[x,y]+S_{plus},\quad\forall(x,y)\in FOA$ (2)

     * •

Inhibit the state of all pixels in the previous FOA:

$s[x,y]=s[x,y]-S_{minus},\quad\forall(x,y)\in FOA$ (3)

with $S_{plus}$ and $S_{minus}$ stored as input parameters for the highest
excitation and inhibition values, respectively.

By penalizing the pixels in the previously selected salient region, the last
step adapts the Inhibition of Return (IOR) mechanism proposed in [4] to our
event-based implementation.

##  V Results and Discussion

The proposed system implements a digital electronic event-driven foveal
vision. All parts of the design are described using the standard top-bottom
digital design flow.
As a proof of concept we implemented our electronic Foveal vision for
processing event-camera inputs on a Kintex-7 XC7KT160T FPGA, which is hosted
on the Opal Kelly XEM7360 board. Based on the results from Table I, the entire
proposed network uses only less than 1 percent of the available Slice FFs and
LUTRAMs and DSPs, 2.31 percent of available Slice LUTs and 38 percent of
available BRAMs. Furthermore, since the whole pipeline operates directly on
the hardware, the system’s latency is in the order of a few microseconds.

TABLE I: Resource utilization of the proposed digital electronic Foveal vision on Kintex-7(XC7KT160T). | Proposed | Available | Utilization
---|---|---|---
Slice FFs | 1646 | 202800 | $0.81\%$
Slice LUTs | 3240 | 102400 | 2.31$\%$
LUTRAMs | 33 | 35000 | 0.09 $\%$
BRAMs | 123.5 | 325 | 38 $\%$
DSPs | 2 | 600 | 0.33 $\%$
Device | Kintex.7 |  |

To validate the architecture in more real-world scenarios, the design was
tested with samples from the DVS128 Gesture Dataset [25], which contains DVS
recordings from different hand gestures and illumination conditions. Fig. 3
shows the trajectory of the most salient locations of a sample from the left
hand clockwise gesture. Fig. 3-(Left) shows the time surface plot of the input
events collected within a time window (t=[0;43.1] ms). The darker events are
the most recent ones. The pixels shown in red are the most salient pixels
selected across time. Fig. 3-(Right) shows the output of the pipeline. Events
are confined within a shifting $16\times 16$ window, which corresponds to the
most salient region at a given time point.

Figure 3: Time surface plots of the input (Left) and output (Right) of the
system.

To examine the behavior of the proposed architecture in terms of the ability
to attend to different salient locations, a controlled data stream was used to
assess the system behavior. The data consists of 3 active pixels with
different firing rates (depicted in gray scale levels in Fig. 4). The
architecture attends first to the pixel with the highest firing rate (dark
gray point). Then it progressively shifts to the other active locations with
decreasing firing rate (from gray to light gray regions). Thus, this result
highlights how the proposed system can effectively shift the focus of
attention towards different salient locations according to the local
instantaneous firing rate.

Figure 4: Examining the forgetting ability of the proposed system by applying
a controlling input data stream. Darker colors show higher firing rates.

Fig. 5 illustrates the results with a top-down biasing condition. Exploring
the events of half of the screen based on the application demands is an
example of top-down biasing conditions. In this experiment, we have used input
recordings from the DAVIS240. Unlike in the previous scenarios, here the
pipeline is only attending to the most salient locations on the upper-half
(Fig. 5a) and left-half (Fig. 5b) parts of the scene. The fovea window stays
at one location for a limited time window (indicated on top of each plot),
until the next most salient location is selected.

Figure 5: Bottom-up attention mechanism with two different top-down biasing
parameters. The region of interest for finding the most salient location is:
(a) upper-half. (b) left-half.

##  VI Conclusion

We presented an implementation of a digital bio-inspired event-driven foveal
vision on an FPGA. The system implements a simplified model of the bottom-up
stimulus-driven saliency-based selective-attention mechanism. The proposed
network has been synthesized and as a proof of concept implemented on a Xilinx
Kintext-7 FPGA device in a setup containing a DAVIS240 camera connected to a
dedicated FPGA. We demonstrated how the system can sequentially select the
spatial locations of the most salient inputs present in the camera’s field of
view with experimental results. The system receives input signals in the form
of address events, selects the spatial locations of the most salient inputs
present in the camera’s field of view, and only transmits events from the
selected sub-region. This reduces the amount of data that needs to be
processed, e.g., to extract features or detect specific targets, and enables
the construction of a complex saliency-based vision processing system. In
particular, it enables further neuromorphic event-based processing chips to
process high-resolution images by using their limited resources (e.g., cores
of 256 neurons [26, 27]) sequentially, as it happens in biology.

## References

  * [1] G. Gallego, T. Delbrück, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis, et al., “Event-based vision: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 154–180, 2020.
  * [2] G. Rizzolatti, Mechanisms of Selective Attention in Mammals, pp. 261–297.  Boston, MA: Springer US, 1983.
  * [3] C. Koch and S. Ullman, “Shifts in selective visual-attention – towards the underlying neural circuitry,” Human Neurobiology, vol. 4, no. 4, pp. 219–227, 1985.
  * [4] L. Itti and C. Koch, “Computational modeling of visual attention,” Nature Reviews Neuroscience, vol. 2, no. 3, pp. 194–203, 2001.
  * [5] “Product of inilabs: Dynamic vision sensor.” http://www.inilabs.com/products/dynamic-vision-sensors, 2009.
  * [6] J. Thompson, “Attention, biological motion, and action recognition,” Neuroimage, vol. 59, no. 1, pp. 4–13, 2012.
  * [7] T. Horiuchi, T. Morris, C. Koch, and S. DeWeerth, “Analog VLSI circuits for attention-based, visual tracking,” in Advances in Neural Information Processing Systems (NIPS) (M. Mozer, M. Jordan, and T. Petsche, eds.), vol. 9, pp. 706–712, MIT Press, 1997.
  * [8] T. Morris, T. Horiuchi, and S. DeWeerth, “Object-based selection within an analog VLSI visual attention system,” IEEE Transactions on Circuits and Systems II, vol. 45, no. 12, pp. 1564–1572, 1998.
  * [9] G. Indiveri, “Modeling selective attention using a neuromorphic analog VLSI device,” Neural Computation, vol. 12, no. 12, pp. 2857–2880, 2000.
  * [10] G. Indiveri, “A neuromorphic VLSI device for implementing 2-D selective attention systems,” IEEE Transactions on Neural Networks, vol. 12, pp. 1455–1463, Nov. 2001.
  * [11] C. Bartolozzi and G. Indiveri, “Selective attention in multi-chip address-event systems,” Sensors, vol. 9, no. 7, pp. 5076–5098, 2009.
  * [12] B. Kim, H. Okuno, T. Yagi, and M. Lee, “Implementation of visual attention system using artificial retina chip and bottom-up saliency map model,” in Neural Information Processing (B.-L. Lu, L. Zhang, and J. Kwok, eds.), (Berlin, Heidelberg), pp. 416–423, Springer Berlin Heidelberg, 2011.
  * [13] J. L. Molin, C. S. Thakur, E. Niebur, and R. Etienne-Cummings, “A neuromorphic proto-object based dynamic visual saliency model with a hybrid fpga implementation,” IEEE Transactions on Biomedical Circuits and Systems, vol. 15, no. 3, pp. 580–594, 2021.
  * [14] J. Lazzaro, J. Wawrzynek, M. Mahowald, M. Sivilotti, and D. Gillespie, “Silicon auditory processors as computer peripherals,” IEEE Transactions on Neural Networks, vol. 4, pp. 523–528, 1993.
  * [15] K. Boahen, “Multiple pathways: Retinomorphic chips that see quadruple images,” in Proceedings of the Seventh International Conference on Microelectronics for Neural, Fuzzy and Bio-inspired Systems; Microneuro’99, (Los Alamitos, CA), pp. 12–20, IEEE Computer Society, Apr. 1999.
  * [16] S. Deiss, R. Douglas, and A. Whatley, “A pulse-coded Communications infrastructure for neuromorphic systems,” in Pulsed Neural Networks (W. Maass and C. Bishop, eds.), ch. 6, pp. 157–78, MIT Press, 1998.
  * [17] L. Itti, E. Niebur, and C. Koch, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254–1259, 1998.
  * [18] S. Park, S. Ban, S. J., L. Minho, O. Kaynak, E. Alpaydin, E. Oja, and X. Lei, “Implementation of visual attention system using bottom-up saliency map model,” in Lecture notes in computer science, Artificial neural networks and neural information processing - ICANN / ICONIP (B. Springer, ed.), vol. 2714, pp. 678–685, June 2003.
  * [19] A. Borji and L. Itti, “State-of-the-art in visual attention modeling,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 185–207, 2013.
  * [20] T. Uejima, E. Niebur, and R. Etienne-Cummings, “Proto-object based saliency model with texture detection channel,” Frontiers in Computational Neuroscience, vol. 14, 2020.
  * [21] H. Akolkar, D. R. Valeiras, R. Benosman, and C. Bartolozzi, “Visual-auditory saliency detection using event-driven visual sensors,” in International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), pp. 1–6, 2015.
  * [22] C. S. Thakur, J. L. Molin, T. Xiong, J. Zhang, E. Niebur, and R. Etienne-Cummings, “Neuromorphic visual saliency implementation using stochastic computation,” in International Symposium on Circuits and Systems, (ISCAS), pp. 1–4, IEEE, May 2017.
  * [23] F. Galluppi, K. Brohan, S. Davidson, T. Serrano-Gotarredona, J. Carrasco, B. Linares-Barranco, and S. Furber, “A real-time, event-driven neuromorphic system for goal-directed attentional selection,” in Neural Information Processing, pp. 226–233, Springer, 2012.
  * [24] C. Brandli, R. Berner, M. Yang, S.-C. Liu, and T. Delbruck, “A 240$\times$180 130 dB 3 $\mu$s latency global shutter spatiotemporal vision sensor,” IEEE Journal of Solid-State Circuits, vol. 49, no. 10, pp. 2333–2341, 2014.
  * [25] “Dvs128 gesture dataset.” IBM Website.
  * [26] P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura, B. Brezzo, I. Vo, S. K. Esser, R. Appuswamy, B. Taba, A. Amir, M. D. Flickner, W. P. Risk, R. Manohar, and D. S. Modha, “A million spiking-neuron integrated circuit with a scalable communication network and interface,” Science, vol. 345, pp. 668–673, Aug. 2014.
  * [27] S. Moradi, N. Qiao, F. Stefanini, and G. Indiveri, “A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAPs),” Biomedical Circuits and Systems, IEEE Transactions on, vol. 12, pp. 106–122, Feb. 2018.

  *[DVS]: Dynamic Vision Sensor
  *[FPGA]: Field Programmable Gate Array
  *[AER]: Address-Event Representation
  *[WTA]: Winner-Take-All
  *[FPGAs]: Field Programmable Gate Array