# Influence of Furnace Baking on Q-E Behavior of Superconducting Accelerating
Cavities

H. Ito<EMAIL_ADDRESS>High Energy Accelerator Research Organization (KEK),
305-0801 Tsukuba, Ibaraki, Japan  H. Araki  High Energy Accelerator Research
Organization (KEK), 305-0801 Tsukuba, Ibaraki, Japan  K. Takahashi  The
Graduate University for Advanced Studies, SOKENDAI, 305-0801 Tsukuba, Ibaraki,
Japan  K. Umemori  High Energy Accelerator Research Organization (KEK),
305-0801 Tsukuba, Ibaraki, Japan  The Graduate University for Advanced
Studies, SOKENDAI, 305-0801 Tsukuba, Ibaraki, Japan

###### Abstract

The performance of superconducting radio-frequency (SRF) cavities depends on
the niobium surface condition. Recently, various heat-treatment methods have
been investigated to achieve unprecedented high quality factor (Q) and high
accelerating field (E). We report the influence of a new baking process called
furnace baking on the Q-E behavior of 1.3 GHz SRF cavities. Furnace baking is
performed as the final step of the cavity surface treatment; the cavities are
heated in a vacuum furnace for 3 h, followed by high-pressure rinsing and
radio-frequency measurement. This method is simpler and potentially more
reliable than previously reported heat-treatment methods, and it is therefore,
easier to apply to the SRF cavities. We find that the quality factor is
increased after furnace baking at temperatures ranging from 300∘C to 400∘C,
while strong decreasing the quality factor at high accelerating field is
observed after furnace baking at temperatures ranging from 600∘C to 800∘C. We
find significant differences in the surface resistance for various processing
temperatures.

††preprint: AIP/123-QED

A superconducting radio-frequency (SRF) cavity is a key component of particle
accelerators used to generate charged particle beams. An SRF cavity exhibits a
lower energy dissipation and a lower surface resistance ($R_{\rm s}$) under a
radio frequency (RF) field, compared to a normal-conducting accelerating
cavity, which enables continuous-wave operation at a high accelerating field
($E_{\rm acc}$). Owing to decades of research focused on the improvement of
SRF cavities Padamsee (2017); Gurevich (2017), various surface treatment
techniques have been established; thus SRF cavities with superior performance
in terms of the quality factor ($Q_{0}$) and $E_{\rm acc}$ have been developed
Geng  _et al._ (2007); Watanabe  _et al._ (2013); Kubo  _et al._ (2014).

In recent years, further surface treatment techniques, such as nitrogen doping
Grassellino  _et al._ (2013); Dhakal (2020), nitrogen infusion Grassellino
_et al._ (2017); Dhakal (2020), and two-step baking Grassellino  _et al._
(2018), have been investigated to increase the $Q_{0}$ and $E_{\rm acc}$. The
nitrogen-doped cavities have an extremely high $Q_{0}$ and show increasing the
$Q_{0}$ as a function of the $E_{\rm acc}$ which is referred to as the anti-Q
slope. However, the maximum $E_{\rm acc}$ obtained with nitrogen doping is
lower than that of the conventional surface-treated cavity. Moreover, the
nitrogen-doped cavities are highly sensitive to trapped magnetic flux compared
with standard treated cavities Martinello  _et al._ (2016). The nitrogen
doping process has been applied in the Linac Coherent Light Source (LCLS$\rm-
II$) cavity fabrication process Bishop  _et al._ (2015); Gonnella  _et al._
(2018) because it is highly reproducible and has resulted in high $Q_{0}$ and
anti-Q slope in several studies Dhakal (2020). In the nitrogen infusion
technique, the Q-E behavior does not change significantly, and both $Q_{0}$
and $E_{\rm acc}$ are improved compared with those values obtained using the
standard surface treatment methods Dhakal  _et al._ (2018); Umemori (2020);
Wenskat  _et al._ (2020); however, the reproducibility has been limited to a
few laboratories. The two-step baking process developed at Fermi National
Accelerator Laboratory (FNAL) can produce cavities with a maximum $E_{\rm
acc}$ of approximately 50 MV/m Grassellino  _et al._ (2018); Bafia  _et al._
(2019), and other laboratories are currently verifying the effectiveness of
two-step baking.

In the typical surface treatment planned for the International Linear Collider
(ILC), the following procedure is implemented: after fabricating the SRF
cavity, a 100 $\mu$m layer of the cavity inner surface is removed by bulk
electropolishing; this results in the elimination of the surface layer damaged
during cavity fabrication. After electropolishing, the surface is thoroughly
rinsed with ultrapure water, and ultrasonic cleaning is performed by filling
ultrapure water with a surfactant, followed by high-pressure ultrapure water
rinsing (HPR). Then, annealing is performed in a vacuum furnace at
approximately 800∘C to desorb the hydrogen that was absorbed on the niobium
surface during electropolishing. Subsequently, light electropolishing is used
to remove a layer of approximately 20 $\mu$m of the cavity inner surface to
eliminate dirt from the inner surface, followed by sufficient water rinsing,
ultrasonic cleaning, HPR, and assembly in a cleanroom. Next, as the final step
in the surface treatment process, the cavity is vacuumed and heat-treated at
120∘C for 48 h.

In this study, a new heat-treatment method, which is simpler and more reliable
than the surface treatment method described above, is investigated from the
viewpoint of oxygen diffusion of a niobium oxide layer, and the effects on the
properties of $Q_{0}$, the Bardeen-Cooper-Schrieffer (BCS) resistance ($R_{\rm
BCS}$), and the residual resistance ($R_{\rm res}$) for each $E_{\rm acc}$ are
studied. In the 1980s and 1990s, SRF cavities that were heat-treated in vacuum
in the range of 250 to 300∘C and 1100 to 1400∘C were investigated to
understand the effect of an oxide layer on the SRF cavity performance Palmer
(1987); Palmer  _et al._ (1990); Palmer and Tigner (1985). It was revealed
that the heat-treatment at 250∘C dissolves the oxide layer and decreases
$R_{\rm BCS}$. Therefore, it is expected that the heat treatment in this study
can be performed in the same temperature range to create a cavity with high
$Q_{0}$. We used several 1.3 GHz TESLA and STF (TESLA-like) single-cell
cavities that had undergone various surface treatments. As a first step, a 10
or 20 $\mu$m layer of the cavity inner surface was removed by the light
electropolishing to reset the surface conditions in the cavity, followed by
HPR to eliminate any remaining impurities on the surface. Subsequently, the
cavities were placed in a large vacuum furnace. The inner diameter of the
furnace chamber is $\phi$ 950 mm, and the length is 2080 mm Umemori  _et al._
(2018). This vacuum furnace can be depressurized to $1\times 10^{-6}$ Pa at
room temperature using a cryopump. Due to the insufficient cooling capacity of
the cryopump, heat treatment at 800∘C increases the temperature of the
cryopump. In such a case, we switch from the cryopump to a turbo molecular
pump for achieving the desired level of vacuum Umemori  _et al._ (2018). The
Quadrupole Mass Spectrometer (Q-mass) was equipped with a vacuum furnace to
monitor the partial pressure of each element during heat treatment. The
cavities were baked in a temperature range of 200 to 800∘C for 3 h in this
vacuum furnace. This baking process is referred to as “furnace baking”, which
is different from the “medium-temperature bake” (mid-T bake) that is performed
at FNAL and Institute of High Energy Physics (IHEP) Posen  _et al._ (2020);
Zhou  _et al._ (2020). The mid-T bake process requires special heat treatment
equipment to perform the RF measurement without exposing the inner surface of
the cavity to the air after heat treatment at 250 to 400∘C, whereas the
furnace baking process is a simple method that can be performed with existing
cavity treatment systems because the heat treatment is performed in a vacuum
furnace. In the first step, the baking time was fixed at 3 h to investigate
the optimum temperature and achieve high $Q_{0}$. To perform the furnace
baking, the cavity temperature was ramped up from room temperature at a ramp
rate of 200∘C/h to a target temperature of 200 to 800∘C. The vacuum was
1-2$\times 10^{-6}$ Pa at room temperature, and it was maintained at the order
of $10^{-4}$ Pa even during baking. After the furnace baking process and
cooling down to below 50∘C, the furnace was purged with $\rm N_{2}$ gas, and
the cavity was packed and placed on the HPR stand for final rinsing before
assembly. A new niobium oxide layer grows on the inner surface during this
step because of exposure to air and water; however, this is not expected to
affect the oxygen diffusion region formed by furnace baking. After assembly,
no further baking was performed, and the RF measurements were conducted.

The cavity was then cooled down by depressurizing liquid helium to 1.5 K,
which is the lowest temperature that can be achieved in the cryostat available
at High Energy Accelerator Research Organization (KEK). Then, the Q-E curve at
each temperature was obtained by calculating the input, reflected, and
transmitted RF powers for 0.1 K temperature increments. Finally, the RF
measurement was performed up to the quench field at 2 K. To minimize the
magnetic flux trapping during the cooling process, the magnetic field around
the cavity was reduced to less than $\sim$1 mG using the magnetic shield and
solenoid coil, and a heater placed at the top beam tube was used to provide a
temperature gradient in the cavity for flux expulsion Huang, Kubo, and Geng
(2016); Posen  _et al._ (2016); Dhakal, Ciovati, and Gurevich (2020). Figure 1
shows the Q-E curve for the cavity that was furnace-baked at 350∘C. The cavity
was quenched once during the measurement at 1.5 K, which caused it to trap the
magnetic flux and subsequently decrease the $Q_{0}$. However, higher $Q_{0}$
and anti-Q slope were still clearly observed in the 2 K measurement results
compared to the standard treated cavity (see Fig. 3).

Figure 1:  Q-E curve at each temperature for the cavity that was furnace-baked
at 350∘C. Colored circles show Q-E curves at each temperature, and purple
squares show radiation levels at 2 K measurement.

$R_{\rm s}$ at each temperature and $E_{\rm acc}$ is calculated using $R_{\rm
s}=G/Q_{0}$, where G is the geometric factor that is independent of material
properties Padamsee, Knobloch, and Hays (1998). $R_{\rm s}$ can be expressed
as the sum of $R_{\rm BCS}$, which decreases exponentially with temperature,
and $R_{\rm res}$, which is a weak temperature-dependent or temperature-
independent term that cannot be accounted for in $R_{\rm BCS}$. $R_{\rm s}$ is
decomposed into $R_{\rm BCS}$ and $R_{\rm res}$ at each $E_{\rm acc}$ using
the data set at the same $E_{\rm acc}$ from the Q-E curve. The decomposition
is performed using the following fitting equation:

$R_{\rm s}(T)=\frac{A\omega^{2}}{T}e^{-(\Delta/kT)}+R_{\rm res},$ (1)

where A is a fitting constant that depends on superconducting properties, T is
the temperature, $k$ is the Boltzmann constant, 2$\Delta$ is the energy gap of
the superconductor, which is treated as a fitting parameter, and $\omega$ is
the frequency of the cavity. The first term in this equation corresponds to
$R_{\rm BCS}$. The colored curves in the upper figure of Fig. 2 show fitting
parameters for $R_{\rm s}(T)$ at each $E_{\rm acc}$. The red closed circles in
the lower figure of Fig. 2 illustrate the behavior of $R_{\rm res}$ for
$E_{\rm acc}$. The blue closed circles depict the behavior of $R_{\rm BCS}$,
which decreases sharply as $E_{\rm acc}$ increases in the case of 350∘C
furnace baking. This behavior is considerably different from the behavior of
standard treated cavities (120∘C and 48 h baked cavities), and it is similar
to the behavior observed in a nitrogen-doped cavity Grassellino  _et al._
(2013). Red open circles depict the estimated behavior of $R_{\rm res}$ before
the flux trapping; $R_{\rm res}$ is smaller than that of the standard treated
cavities for 350∘C furnace baking.

Figure 2: Temperature dependence of $R_{\rm s}$ for each $E_{\rm acc}$ (upper
figure) and behavior of $R_{\rm BCS}$ and $R_{\rm res}$ for $E_{\rm acc}$
(lower figure). $R_{\rm res}$ before the trap was obtained by subtracting
$R_{\rm BCS}$ obtained after the trap from the $R_{\rm s}$ at 1.5 K before the
trap.

Figure 3 shows a comparison of the Q-E curves measured at 2 K for cavities
that were furnace-baked at various temperatures (200 to 800∘C) and a standard
treated cavity (120∘C and 48 h baking under vacuum directly followed by RF
measurement, no exposure to air or water). The Q-E behavior of the 200∘C and 3
h furnace-baked cavity (purple points) is similar to that of the standard
treated cavity (black points). This indicates that the conventional
performance can be achieved by replacing the 120∘C and 48 h baking with 200∘C
and 3 h furnace baking, which may be significantly effective for mass
production of cavities. Four furnace-baked cavities, baked at 300 to 400∘C,
have high $Q_{0}$ and anti-Q slope, but a low $E_{\rm acc}$ compared with the
standard treated cavity. This behavior is typically associated with nitrogen-
doped cavities Grassellino  _et al._ (2013). In particular, 300∘C furnace
baking produces an extremely high Q cavity, with a $Q_{0}$ of over 5$\times
10^{10}$ at 16 MV/m. Furthermore, 300∘C furnace baking has the same effect for
two different cavities, indicating good reproducibility. These results are in
good agreement with those obtained for single-cell cavities that are furnace-
baked in the temperature range of 250 to 400∘C at IHEP He  _et al._ (2020).
The high-temperature furnace-baked cavities baked at 600∘C and 800∘C did not
reach high $Q_{0}$, and the Q values were comparable to those of the standard
treated cavity. A phenomenon called high field Q slope (HFQS), in which the
$Q_{0}$ decreases significantly at high $E_{\rm acc}$ safa (2001), was
observed in these cavities. This HFQS is considered to be related to the
diffusion of oxygen and hydrogen on the inner surface of the cavity Benvenuti,
Calatroni, and Ruzinov (2001); Ciovati (2006); Ciovati  _et al._ (2010);
Romanenko  _et al._ (2013); Checchin and Grassellino (2020), and the effect of
suppressing the HFQS diminished due to oxygen diffusion at high temperatures.
These results indicate that diverse Q-E behaviors were obtained when the
furnace baking temperature was changed from 200 to 800∘C. In particular,
furnace baking at 300 to 400∘C resulted in high $Q_{0}$ and anti-Q slope. The
low $E_{\rm acc}$ is similar to that of the nitrogen-doped cavity and may be
related to the low superheating field at the dirty limitKubo (2020a).
Theoretical considerations suggest that the $Q_{0}$ varies depending on the
cavity surface conditionGurevich and Kubo (2017); Kubo and Gurevich (2019);
Kubo (2020b).

Figure 3:  Comparison of Q-E behavior measured at 2 K for cavities that were
furnace-baked at various temperatures (200 to 800∘C) and a standard treated
cavity (120∘C and 48 h baking). Colored closed points show the Q-E curve for
each furnace-baked cavity, and the colored opened points show the radiation
levels corresponding to each colored closed point.

The upper panel of Fig. 4 shows the relationship between $R_{\rm BCS}$ at 2 K
and $E_{\rm acc}$ for each furnace-baked and standard treated cavity. $R_{\rm
BCS}$ behavior is classified into three types: one that increases with
increasing $E_{\rm acc}$, one that decreases with increasing $E_{\rm acc}$,
and one that does not increase as much as the first type. The 200∘C furnace-
baked cavity and the standard treated cavity correspond to the first type
mentioned above, and the slope of $R_{\rm BCS}$ is steep compared with those
obtained for other cavities. The 300 to 400∘C furnace-baked cavities
correspond to the second type. In these cavities, $R_{\rm BCS}$ decreases as
$E_{\rm acc}$ increases, which is the origin of the anti-Q slope. This
behavior is the most pronounced in 300∘C furnace-baked cavities. The 600∘C and
800∘C furnace-baked cavities correspond to the third type. These cavities
already have a high $R_{\rm BCS}$ at low $E_{\rm acc}$; however, the slope is
less steep compared with the first type. From these results, it was found that
$R_{\rm BCS}$ behavior varies significantly with differences in the baking
temperature, resulting in the variation of Q-E behavior. Further, it was
suggested that there is an inflection point between 200∘C and 300∘C, where the
behavior of $R_{\rm BCS}$ changes significantly, and that a similar inflection
point exists in the region between 400∘C and 600∘C. The lower panel of Fig. 4
shows the relationship between $R_{\rm res}$ and $E_{\rm acc}$ for each
furnace-baked cavity and standard treated cavity. It was found that $R_{\rm
res}$ is lower for all the furnace-baked cavities compared with that of the
standard treated cavity, and $R_{\rm res}$ behavior changes with differences
in baking temperature but not as drastically as $R_{\rm BCS}$ behavior.
Notably, the 600∘C furnace-baked cavity has an extremely low $R_{\rm res}$ of
0.2 $\rm n\Omega$, which corresponds to a $Q_{0}$ of over $1\times 10^{12}$.
Because $R_{\rm res}$ dominates $R_{\rm s}$ at temperatures of approximately 1
K, this 600∘C furnace baking has the potential to be a useful processing
method for superconducting devices, such as those used at cryogenic
temperatures in the mK region rather than the SRF accelerator application
operating at 2 K.

Figure 4: $R_{\rm BCS}$ behavior at 2 K for $E_{\rm acc}$ for each furnace-
baked cavity and standard treated cavity (upper panel). Relationship between
$R_{\rm res}$ and $E_{\rm acc}$ for each furnace-baked cavity and standard
treated cavity (lower panel).

The sensitivity of the mid-T (300 to 400∘C) furnace-baked cavity was estimated
by measuring the Q-E curve after cooling slowly in a 20 mG field. (Slow
cooling allowed the magnetic field of 95% to be trapped in the cavity.) The
sensitivity $S$ describes the amount of increase in $R_{\rm s}$ per unit of
trapped field $B_{\rm trap}$ and can be expressed as

$S=\frac{\Delta R_{\rm s}}{B_{\rm trap}}.$ (2)

Figure 5 shows the measurement results of the sensitivity of the mid-T
furnace-baked cavity and the comparison to a standard treated cavity. The
mid-T furnace-baked cavities have a high sensitivity compared with the
standard treated cavity and resemble the nitrogen-doped cavity Martinello  _et
al._ (2016). The sensitivity of the 300∘C furnace-baked cavity is higher than
that of the nitrogen-doped cavity. When such a high-sensitivity cavity is
installed as an accelerator component, it is necessary to consider the effect
of the magnetic field trapping due to the ambient magnetic field more severely
than in the standard treated cavity because it is difficult to realize the
ideal magnetic field shielding in real accelerator components as in the RF
measurement of this study.

Figure 5:  Sensitivity of mid-T furnace-baked cavities and standard treated
cavity.

In this study, a new heat-treatment process called furnace baking has been
investigated at several baking temperatures ranging from 200 to 800∘C and a
baking time of 3 h. The behavior of Q-E is found to be sensitive to the baking
temperature. Furthermore, the quench field changes with the baking
temperature. The mid-T (300 to 400∘C) furnace baking produces a cavity with
high $Q_{0}$ and anti-Q slope. In particular, the 300∘C furnace-baked cavity
has an extremely high $Q_{0}$ of over 5$\times 10^{10}$ at 16 MV/m and 2 K.
The quench field is 20 to 25 MV/m for the mid-T furnace-baked cavities, which
is lower than the quench field of the standard treated cavity. Although the
achievable $E_{\rm acc}$ is low, its high $Q_{0}$ is very impressive, and
combined with the simplicity of the furnace baking procedure, it is clear that
the mid-T furnace baking can be successfully adapted to various SRF
applications in the future. $R_{\rm res}$ is lower for all the furnace-baked
cavities compared with that of the standard treated cavity. For the 600∘C
furnace-baked cavity, an extremely low $R_{\rm res}$ of 0.2 $\rm n\Omega$ is
obtained. The sensitivity of the mid-T furnace-baked cavity is higher than
that of the standard treated cavity and resembles that of the nitrogen-doped
cavity. The 300∘C furnace-baked cavity, which has the highest $Q_{0}$, has a
higher sensitivity than that of the nitrogen-doped cavity. Further studies
will be undertaken by focusing on the surface analysis based on the sample
study to reveal the relationship between these behaviors and the cavity
surface condition. Further, process optimization will be performed by changing
the baking time.

This work was supported by JSPS Grant-in-Aid for Scientific Research(B) No.
19H04402.

## References

  * Padamsee (2017) H. Padamsee, Supercond. Sci. Technol. 30, 053003 (2017).
  * Gurevich (2017) A. Gurevich, Supercond. Sci. Technol. 30, 034004 (2017).
  * Geng  _et al._ (2007) R. L. Geng, G. V. Eremeev, H. Padamsee, and V. D. Shemelin, in  _Proc. PAC2007_ (2007) pp. 2337–2339.
  * Watanabe  _et al._ (2013) K. Watanabe, S. Noguchi, E. Kako, K. Umemori, and T. Shishido, Nuclear Inst. and Methods in Physics Research, A 714, 67 – 82 (2013).
  * Kubo  _et al._ (2014) T. Kubo, Y. Ajima, H. Inoue, K. Umemori, Y. Watanabe, and M. Yamanaka, in  _Proc. IPAC’14_ (2014) pp. 2519–2521.
  * Grassellino  _et al._ (2013) A. Grassellino, A. Romanenko, D. Sergatskov, O. Melnychuk, Y. Trenikhina, A. Crawford, A. Rowe, M. Wong, T. Khabiboulline, and F. Barkov, Supercond. Sci. Technol. 26, 102001 (2013).
  * Dhakal (2020) P. Dhakal, Physics Open 5, 100034 (2020).
  * Grassellino  _et al._ (2017) A. Grassellino, A. Romanenko, Y. Trenikhina, M. Checchin, M. Martinello, O. S. Melnychuk, S. Chandrasekaran, D. A. Sergatskov, S. Posen, A. C. Crawford, S. Aderhold, and D. Bice, Supercond. Sci. Technol. 30, 094004 (2017).
  * Grassellino  _et al._ (2018) A. Grassellino, A. Romanenko, D. Bice, O. Melnychuk, A. C. Crawford, S. Chandrasekaran, Z. Sung, D. A. Sergatskov, M. Checchin, and S. Posen, arXiv e-prints , arXiv:1806.09824 (2018).
  * Martinello  _et al._ (2016) M. Martinello, A. Grassellino, M. Checchin, A. Romanenko, O. Melnychuk, D. A. Sergatskov, S. Posen, and J. F. Zasadzinski, Applied Physics Letters 109, 062601 (2016).
  * Bishop  _et al._ (2015) P. Bishop, M. Checchin, H. Conklin, A. Crawford, E. Daly, G. Davis, M. Drury, R. Eichhorn, J. Fischer, F. Furuta, G. M. Ge, D. Gonnella, A. Grassellino, C. Grimm, T. Gruber, D. Hall, A. Hocker, G. Hoffstaetter, J. Kaufman, G. Kulina, M. Liepe, J. Maniscalco, M. Martinello, O. Melnychuk, T. O’Connel, J. Ozelis, A. D. Palczewski, P. Quigley, C. Reece, A. Romanenko, M. Ross, A. Rowe, D. Sabol, J. Sears, D. A. Sergatskov, W. Soyars, R. Stanek, V. Veshcherevich, R. Wang, and G. Wu, in  _Proc. SRF’15_ (2015) pp. 159–163.
  * Gonnella  _et al._ (2018) D. Gonnella, S. Aderhold, A. Burrill, E. Daly, K. Davis, A. Grassellino, C. Grimm, T. Khabiboulline, F. Marhauser, O. Melnychuk, A. Palczewski, S. Posen, M. Ross, D. Sergatskov, A. Sukhanov, Y. Trenikhina, and K. Wilson, Nuclear Inst. and Methods in Physics Research, A 883, 143 – 150 (2018).
  * Dhakal  _et al._ (2018) P. Dhakal, S. Chetri, S. Balachandran, P. J. Lee, and G. Ciovati, Phys. Rev. Accel. Beams 21, 032001 (2018).
  * Umemori (2020) K. Umemori, “New results of kek infusion and mid-t bake,” in  _TTC 2020_ (2020).
  * Wenskat  _et al._ (2020) M. Wenskat, C. Bate, A. D. Pandey, A. Jeromin, T. F. Keller, J. Knobloch, J. Köszegi, F. Kramer, O. Kugeler, S. Kulkarni, D. Reschke, J. Schaffran, G. D. L. Semione, S. Sievers, L. Steder, A. Stierle, and N. Walker, Supercond. Sci. Technol. 33, 115017 (2020).
  * Bafia  _et al._ (2019) D. Bafia, A. Grassellino, O. Melnychuk, A. Romanenko, Z.-H. Sung, and J. Zasadzinski, in  _Proc. SRF’19_ , 19 (2019) pp. 586–591.
  * Palmer (1987) F. Palmer, IEEE Transactions on Magnetics 23, 1617–1619 (1987).
  * Palmer  _et al._ (1990) F. Palmer, R. Kirby, F. King, and E. L. Garwin, Nuclear Inst. and Methods in Physics Research, A 297, 321 – 328 (1990).
  * Palmer and Tigner (1985) F. Palmer and M. Tigner, IEEE Transactions on Magnetics 21, 1011–1013 (1985).
  * Umemori  _et al._ (2018) K. Umemori, M. Egi, E. Kako, T. Konomi, S. Michizono, and H. Sakai, in  _Proc. 29th International Linear Accelerator Conference_ (2018).
  * Posen  _et al._ (2020) S. Posen, A. Romanenko, A. Grassellino, O. Melnychuk, and D. Sergatskov, Phys. Rev. Applied 13, 014024 (2020).
  * Zhou  _et al._ (2020) Q. Zhou, F.-S. He, W. Pan, P. Sha, Z. Mi, and B. Liu, Radiation Detection Technology and Methods (2020).
  * Huang, Kubo, and Geng (2016) S. Huang, T. Kubo, and R. L. Geng, Phys. Rev. Accel. Beams 19, 082001 (2016).
  * Posen  _et al._ (2016) S. Posen, M. Checchin, A. C. Crawford, A. Grassellino, M. Martinello, O. S. Melnychuk, A. Romanenko, D. A. Sergatskov, and Y. Trenikhina, Journal of Applied Physics 119, 213903 (2016).
  * Dhakal, Ciovati, and Gurevich (2020) P. Dhakal, G. Ciovati, and A. Gurevich, Phys. Rev. Accel. Beams 23, 023102 (2020).
  * Padamsee, Knobloch, and Hays (1998) H. Padamsee, J. Knobloch, and T. Hays,  _RF Superconductivity for Accelerators, Wiley-VCH_ (1998).
  * He  _et al._ (2020) F. He, W. Pan, P. Sha, J. Zhai, Z. Mi, X. Dai, S. Jin, Z. Zhang, C. Dong, B. Liu, H. Zhao, R. Ge, J. Zhao, Z. Mu, L. Du, L. Sun, L. Zhang, C. Yang, and X. Zheng, arXiv e-prints , arXiv:2012.04817 (2020).
  * safa (2001) H. safa, in  _Proc. SRF’01_ (2001).
  * Benvenuti, Calatroni, and Ruzinov (2001) C. Benvenuti, S. Calatroni, and V. Ruzinov, in  _Proc. SRF’01_ (2001).
  * Ciovati (2006) G. Ciovati, Applied Physics Letters 89, 022507 (2006).
  * Ciovati  _et al._ (2010) G. Ciovati, G. Myneni, F. Stevie, P. Maheshwari, and D. Griffis, Phys. Rev. ST Accel. Beams 13, 022002 (2010).
  * Romanenko  _et al._ (2013) A. Romanenko, F. Barkov, L. D. Cooley, and A. Grassellino, Supercond. Sci. Technol. 26, 035003 (2013).
  * Checchin and Grassellino (2020) M. Checchin and A. Grassellino, Applied Physics Letters 117, 032601 (2020).
  * Kubo (2020a) T. Kubo, Phys. Rev. Research 2, 033203 (2020a).
  * Gurevich and Kubo (2017) A. Gurevich and T. Kubo, Phys. Rev. B 96, 184515 (2017).
  * Kubo and Gurevich (2019) T. Kubo and A. Gurevich, Phys. Rev. B 100, 064522 (2019).
  * Kubo (2020b) T. Kubo, Phys. Rev. Research 2, 013302 (2020b).


# EchoVPR: Echo State Networks for
Visual Place Recognition

Anıl Özdemir1,†, Mark Scerri1,†, Andrew B. Barron2, Andrew Philippides3,
Michael Mangan1,∗, Eleni Vasilaki1,4,∗, and Luca Manneschi1,∗ Manuscript
received: September 9, 2021; Revised December 3, 2021; Accepted January 20,
2022.This paper was recommended for publication by Editor Eric Marchand upon
evaluation of the Associate Editor and Reviewers’ comments.This work was
supported by Engineering and Physical Sciences Research Council under Grant
EP/P006094/1, EP/S030964/1, EP/S009647/1 and EP/V006339/1, and by Templeton
World Charity Foundation Grant № 0539.1Anıl Özdemir, Mark Scerri, Michael
Mangan, Eleni Vasilaki, and Luca Manneschi are with Department of Computer
Science, The University of Sheffield, UK<EMAIL_ADDRESS>B.
Barron is with Department of Biological Sciences, Macquarie University,
Australia3Andrew Philippides is with School of Engineering and Informatics,
University of Sussex, UK4Eleni Vasilaki is Institute for Neuroinformatics,
University of Zurich and ETH Zurich, Switzerland†These authors have equally
contributed to the work.∗Joint senior authors.

###### Abstract

Recognising previously visited locations is an important, but unsolved, task
in autonomous navigation. Current visual place recognition (VPR) benchmarks
typically challenge models to recover the position of a query image (or
images) from sequential datasets that include both spatial and temporal
components. Recently, Echo State Network (ESN) varieties have proven
particularly powerful at solving machine learning tasks that require spatio-
temporal modelling. These networks are simple, yet powerful neural
architectures that—exhibiting memory over multiple time-scales and non-linear
high-dimensional representations—can discover temporal relations in the data
while still maintaining linearity in the learning time. In this paper, we
present a series of ESNs and analyse their applicability to the VPR problem.
We report that the addition of ESNs to pre-processed convolutional neural
networks led to a dramatic boost in performance in comparison to non-recurrent
networks in five out of six standard benchmarks (GardensPoint, SPEDTest,
ESSEX3IN1, Oxford RobotCar, and Nordland), demonstrating that ESNs are able to
capture the temporal structure inherent in VPR problems. Moreover, we show
that models that include ESNs can outperform class-leading VPR models which
also exploit the sequential dynamics of the data. Finally, our results
demonstrate that ESNs improve generalisation abilities, robustness, and
accuracy further supporting their suitability to VPR applications.

###### Index Terms:

Vision-Based Navigation; Deep Learning for Visual Perception; Visual Learning

©2022 IEEE. Personal use of this material is permitted. Permission from IEEE
must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes,
creating new collective works, for resale or redistribution to servers or
lists, or reuse of any copyrighted component of this work in other works.
DOI:10.1109/LRA.2022.3150505.

##  I Introduction

Visual Place Recognition (VPR) challenges algorithms to recognise previously
visited places despite changes in appearance caused by illuminance, viewpoint,
and weather conditions (for reviews see [1, 2, 3]). Unlike in many machine
learning tasks, typical VPR benchmarks challenge models to learn image
locations following a single route traversal, which are then compared with
data during another route traversal. Thus, there are very few examples to
learn from (typically only the images within a few metres of the correct
location) making the task particularly challenging.

One approach is to recognise places based on matching single views using image
processing methods to remove the variance between datasets. For instance,
models have been developed that use different image descriptors to obtain
meaningful image representations that are robust to visual change (e.g.
DenseVLAD [4], NetVLAD [5], AMOSNet [6], SuperGlue [7], DELG [8], Patch-
NetVLAD [9], and HEAPUtil [10]). While matching single images is successful in
many scenarios, it can suffer from the effects of aliasing, individual image
corruption, or sampling mismatches between datasets (e.g. it is challenging to
ensure that images sampled along the same route precisely overlap).

Milford and Wyeth [11] were the first to demonstrate that such issues could be
overcome by matching sequences of images using a global search to overcome
individual image mismatches. This has led to a family of models that improve
VPR performance by exploiting the temporal relationships inherent in images
sampled along routes [11, 12, 13, 14, 15, 16, 17]. While achieving state-of-
the-art performance on challenging real-world datasets (e.g. Oxford RobotCar
[18], Extended CMU Seasons [19], Pittsburgh [20], Tokyo24/7 [4], Nordland
[21], Mapillary Street Level Sequences [22]), such models often include an
explicit encoding of non-visual information to limit the image search space,
require a cache of input images for global searches, and can be
computationally expensive in both learning and deployment phases. Neither of
these features are desirable for autonomous robots that may have limited
computational resources and external sensing capabilities.

Echo State Networks (ESN) [23] are a class of computationally efficient
recurrent neural networks, ideally suited to addressing VPR problems without
the need for additional support cues or input data caching (see Fig. 1). ESNs
are a subset of reservoir computing models in which the reservoir neurons
possess fixed, random and recurrent interconnections that sustain recent
memories, i.e. echoes [24], with the practical benefit that only the output
layer weights require training. ESNs thus act as a temporal kernel [25] over a
variety of time-scales, creating a form of working memory dispensing of the
need for input caching. ESNs have excelled when applied to problems that
involve sequential data including dynamical system predictions [26, 27],
robotic motion and navigation tasks [28, 29, 30]. Hence, they appear well-
suited to the VPR problem but this hypothesis is yet to be tested. Recently
ESNs have been augmented by novel techniques e.g., [31, 32]. Among these, in
the present work we focus on SPARCE, which stands for Sparse Reservoir
Computing.

Figure 1:  An illustration of EchoVPR framework. Echo State Networks (ESN)
incorporate temporality while still maintaining real-time prediction
capability, which is a key feature for a robotic system in real-world
applications. Given an input image at a time (from snowy Nordland [21] in this
example), an image descriptor (class-leading NetVLAD [5] via a hidden layer,
see Section II-D) provides a meaningful representation to the ESN to update
the fixed reservoir.

The contributions of this work are:

  * •

a novel approach that combines state-of-the-art image descriptors augmented
with ESNs to exploit temporal dynamics of image sequences;

  * •

validation of the proposed models in four evaluation datasets that present
different input variances (e.g. indoor/outdoor, day/night, and viewpoint);

  * •

benchmarking against contemporary single- and sequential-view based models in
challenging, long-range VPR datasets sampled from moving vehicles across
illumination and seasonal changes;

  * •

assessment of the models’ capacity to recognise routes from a random start-
point (akin to kidnapped robot problem).

##  II Methods

###  II-A Problem Formulation

VPR algorithms are provided with a sequence of places (in the form of images)
sampled along a route. They are then asked to correctly match (within an
acceptable threshold) the places by the image key-frames along the same route
at a different time. The input data is composed of videos where the network
has to correctly infer the location, i.e. the image key-frame that is
processed at the considered time. In all the tasks there are at least two
sequences of images, one used as a training set (i.e. reference) and the other
used as a test set (i.e. query), acquired by visiting the same locations and
following the same path twice. Even though there is a one-to-one mapping
between training and test samples, the latter is acquired by visiting the
locations at different times, leading to differences in visual appearances,
such as seasonal or illuminance as well as viewpoint changes. Perfect matching
is not always possible, hence, there can be a tolerance term that allows close
matches to be accepted. A match is considered successful, if
$||\text{reference}-\text{query}||\leq\text{tolerance}$.

In our specific implementation, we consider supervised learning with the ESNs
as a predictor, hence, forming a classification problem. The number of read-
out nodes is equal to the number of places, and therefore, specific to the
given dataset. The read-out nodes (the final and only learnable layer) output
a probability distribution, $\mathcal{P}_{\text{query}}$, for each given query
image. The prediction (i.e. key-frame of the query) is the number of the read-
out node, i.e. $\arg\max\mathcal{P}_{\text{query}}$.

###  II-B Standard Echo State Networks

An ESN is a reservoir of recurrently connected nodes, whose temporal dynamics
$\mathbf{x}(t)$ evolves following [23]:

$\displaystyle\mathbf{x}(t+\delta t)$
$\displaystyle=(1-\alpha)\mathbf{x}(t)+\alpha f\left(\mathbf{h}(t)\right),$
(1) $\displaystyle\mathbf{h}(t)$ $\displaystyle=\gamma\mathbf{W}_{\rm
in}\mathbf{s}(t)+\rho\mathbf{W}\mathbf{x}(t),$ (2)

where $\alpha$ is the leakage term and defines the rate of integration of
information, $f$ is a non-linear activation function (usually
$\operatorname{tanh}$), $\mathbf{s}(t)$ is the input signal, $\mathbf{W}_{\rm
in}$ is the input connectivity matrix, which is commonly drawn from a random
Gaussian distribution, and $\gamma$ is a multiplicative factor of the external
signal. The recurrent connectivity $\mathbf{W}$ is a sparse, random and fixed
matrix whose eigenvalues are constrained inside the unit circle of the
imaginary plane, with a hyper-parameter $\rho$ (usually in the range of
$[0,1]$) set to further control the spectral radius. Learning occurs via
minimisation of a cost function. Only the read-out weights $\mathbf{W}_{\rm
out}$ that connect the reservoir neurons $\mathbf{x}$ to the output change.
Optimisation of $\mathbf{W}_{\rm out}$ can be accomplished through different
techniques such as ridge regression or iterative gradient descent methods
[33]. In our work, we use exclusively gradient decent methods.

###  II-C Sparse Reservoir Computing

The definition of sparse representations through the SPARCE model [31] can
enhance the capacity of the reservoir to learn associations by introducing
specialised neurons through the definition of learnable thresholds.
Considering the representation $\mathbf{x}$ from which the read-out is
defined, as in Eq. (1), SPARCE consists of the following normalisation
operation:

$\displaystyle x^{\prime}_{i}$
$\displaystyle=\operatorname{sign}\big{(}x_{i}\big{)}\operatorname{ReLU}\Big{(}|x_{i}|-\theta_{i}\Big{)}$
(3) $\displaystyle\theta_{i}$
$\displaystyle=P_{n}\big{(}|\mathbf{x}_{i}|\big{)}+\bar{\theta}_{i}$ (4)

where $i$ is the $i$-th dimension, $\operatorname{sign}$ is the sign function
and $\operatorname{ReLU}$ is the rectified linear unit. Of course, the new
read-out is defined from $x^{\prime}_{i}$, that is after the transformation
given in Eq. (3) and (4), which leaves the dynamics of the system unaltered
and can be easily applied to any reservoir representation. The threshold
$\theta_{i}$ is composed of two factors:
$P_{n}\big{(}|\mathbf{x}_{i}|\big{)}$, i.e. the $n$-th percentile of
$\mathbf{x}_{i}$, which stands for the distribution of activities of dimension
$i$ after the presentation of a number of samples with sufficient statistics,
and a learnable part $\bar{\theta}_{i}$, which is adapted through gradient
descent and is initialised to arbitrarily small values at the beginning of
training. The percentile $n$ can be considered as an additional interpretable
hyper-parameter that controls the sparsity level of the network at the start
of the training phase111For different methodologies to estimate the percentile
operation, see [31].

###  II-D Image Pre-processing

Convolutional neural networks (CNN) are the best performing architectures for
processing images and discovering high-level features from visual data.
However, they are static and lack temporal dynamics. Thus, after a pre-
processing module composed of NetVLAD [5], a pre-trained CNN, we adopted a
system composed by ESNs. Considering that the reservoir computing paradigm is
more effective when the reservoir expands the dimensionality of its
corresponding input, we first decreased the dimensionality of NetVLAD output
(original dimension is $4096$) by training a feedforward network composed of
one hidden layer (with $500$ nodes) on the considered classification task. The
hidden layer representation is then considered as the input to the reservoir
computing system. With NV we denote the full model consisting of NetVLAD, a
hidden layer and an output layer. When NV is combined with another system,
e.g., NV-ESN, it means that we have used the trained hidden layer as an input
representation to the ESN.

###  II-E Proposed Architecture

The reservoir is then trained to distinguish the different locations, which
are processed successively in the natural order of acquisition by the overall
architecture. The two reservoir computing models we study are summarised
below222The supplementary material and source-code for the ESN implementations
can be found in https://anilozdemir.github.io/EchoVPR/:

  * •

Echo State Network (ESN), where learning happens on the output weights only.
The critical hyper-parameters of the system for the cases studied are
$\alpha,\gamma,\eta$ (leakage term, input factor, learning rate). The overall
architecture that exploits this reservoir will be named NV-ESN in the rest of
the work.

  * •

Echo State Network with SPARCE (SPARCE-ESN), where thresholds are applied to
the reservoir following Eq. (3) and learning occurs on $\bar{\theta}$ and
$\mathbf{W}_{\rm out}$. The hyper-parameters are the same as the standard ESN
with the addition of the starting percentile $P_{n}$ of Eq. (4). For this
case, the overall architecture will be named NV-SPARCE-ESN.

###  II-F Error Functions

Learning of $\mathbf{W}_{\rm out}$ and $\bar{\theta}$ is accomplished through
mini-batches and by minimisation of softmax cross-entropy loss via gradient
descent:

$E=\sum_{j}^{\rm N_{batch}}\sum_{i}{y^{\rm
target}_{ij}}\operatorname{log}\Bigg{(}\dfrac{\exp\big{(}y_{ij}\big{)}}{\sum_{i}\exp(y_{ij})}\Bigg{)},$
(5)

where $\rm N_{batch}$ is the minibatch size, $y$ the output of the neural
network, $y^{\rm target}$ the target output, and the indices $i$ and $j$
correspond to the sample number and to the output node considered. The models
are trained for up to $60$ epochs, i.e. each training image is seen $60$
times.

Figure 2: ESN performance compared with single view matching algoirthms in
evaluation datasets. The utilisation of reservoir computing models permits the
temporal dynamics of the problem to be captured and improves the performance
of CNNs. NV-ESN and NV-SPARCE-ESN are shown in blue-green colours, while the
performance of static neural networks is reported in red-yellow colours. The
performance of AMOSNet, DenseVLAD and NetVLAD were taken from [34], where
image matching was achieved by computing distances among the representation.
NetVLAD-C and NV correspond to models in which a simple read-out or a hidden
layer with a read-out were trained from the representation of the
convolutional network respectively. This was achieved through the minimisation
of Eq. (5) on the specific task considered, similar to the approach used for
ESNs. The bar plots for our method shows average performance over $20$ trials.

For larger datasets, we used the sigmoid cross-entropy loss as the error
function, which led to better performance:

$\small E=\sum_{j}^{\rm
N_{batch}}\sum_{i}y^{target}_{ij}\operatorname{log}{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\big{[}\sigma(y_{ij})\big{]}+(1-y^{target}_{ij})\operatorname{log}\big{[}1-\sigma(y_{ij})\big{]}},$
(6)

where $\sigma$ is the sigmoid function and the remaining terms correspond to
the ones of Eq. (5). A description of the datasets used is provided in Section
III-A.

##  III Experiments

###  III-A Datasets And Metrics

To allow benchmarking of different models across datasets, the VPR community
is converging upon a standardised methodology [34, 35, 36]. We therefore
evaluate model performance using the recommended metrics: prediction accuracy,
recall@n and precision-recall area-under-curve (AUC).

Initial analysis was performed on four datasets that challenge models in a
variety of conditions:

  * •

GardensPoint [37] consists of $200$ indoor, outdoor and natural environments
with both viewpoint and conditional changes throughout the dataset. A
tolerance of $2$ is accepted.

  * •

ESSEX3IN1 [38] consists of $210$ images taken at the university campus and
surroundings, focusing on perceptual aliasing and confusing places. There is
no tolerance for this dataset.

  * •

SPEDTest [39] consists of $607$ low-quality but high-depth images collected
from CCTV cameras around the World; it includes environmental changes
including variations in weather, seasonal, and illumination conditions. There
is no tolerance for this dataset.

  * •

Corridor [12] consists of 111 images of an indoor environment - three
traverses of the same corridor.

Models were then evaluated in two larger datasets captured from moving
vehicles that are more representative of the data that would be presented to
an autonomous robot:

  * •

Nordland [9] comprises of images taken at train traversals in four different
seasons in Norway; the viewpoint angle is fixed although there is a high
weather, seasonal and illumination variability. Each traversal consists of
$27592$ images. A tolerance of $10$ is acceptable—the same as the models [16,
9, 17] we compare against (see Section IV-E for more details).

  * •

Oxford RobotCar [40] comprises of images taken from a car travelling a fixed
10 km route around Oxford, UK at different times of the day across seasons.
Each traversal consists of roughly $30000$ images.

To allow direct comparison of ESNs with different published results, we also
consider a subset of the Nordland dataset composed of $1000$ images as in
[16], and then the full dataset as in [9]. For the Oxford RobotCar dataset
[40], we used 4599 and 4550 images captured during the day
(2015-03-17-11-08-44) and at night (2014-12-16-18-44-24) sampled at $2$m apart
for training and testing respectively, with a tolerance of $20$m, in a similar
manner to [17].

###  III-B Benchmarking Process

We compared NV-ESN and NV-SPARCE-ESN (see Section II-E) on the smaller
datasets (GardensPoint, SPEDTest, ESSEX3IN1) to the following models, taken
from the literature: AMOSNet [6], DenseVLAD [4] and NetVLAD [5]. We also used
our own NetVLAD variants, NetVLAD-C, which consists of NetVLAD with an output
layer trained as a classifier, and NV (see Section II-D).

For the Nordland dataset, we compare our methods with NetVLAD [5], SuperGlue
[7], DELG global [8], DELG local [8] and Patch-NetVLAD [9]. The latter uses
NetVLAD in combination with local features (Patch). We were able to combine
our ESNs variants with local features similar to Patch-NetVLAD. In particular,
after classification of the top $100$ locations performed by NV-ESN or NV-
SPARCE-ESN, we used such local features to select the final classified
location. For simplicity, we named the resulting models PatchL-ESN and PatchL-
SPARCE-ESN for the case with or without SPARCE respectively, where L stands
for “Light”. The “Light” model refers to the variant Single-Spatial-Patch-
NetVLAD as defined in [9] which employs a simple spatial verification method
applied on a single patch of size $5$ and $512$ PCA dimensions. In contrast
the results that are reported from Patch-NetVLAD [9] utilises the variant
Multi-RANSAC-Patch-NetVLAD which employs a more complex spatial scoring method
applied on a fusion of multiple patches of sizes ($2$, $5$ and $8$) and $4096$
PCA dimensions. This selection process among the top $100$ images is analogous
to the one in the original paper [9].

Again on Nordland, we compared NV-ESN to sequential models. More specifically,
we considered three NetVLAD variants (NetVLAD+Smoothing, NetVLAD+Delta and
Netvlad+SeqMatch), SeqNet($S_{5}$) and HVPR($S_{5}$ to $S_{1}$). For the
description of all these models see [17]. SeqNet, in particular, is a recently
formulated sequential model where authors used the images of the Nordland
dataset captured during summer and winter as training set, and then tested the
model on previously unseen locations during Spring and Fall. In [17], the
generalisation on unseen locations is possible because of the computation of
distances among images and of the minimisation of a triplet loss function.
Considering that we are training a classifier, where each image corresponds to
a different output node similarly to [16], it is difficult to make a direct
comparison. Instead, we formulated a methodology inspired by the one adopted
for SeqNet [17]. We selected single images or pairs of consecutive images
randomly from the summer and winter datasets and excluded them from the
training. We used these images for validation and images captured at the same
locations, but during Spring and Fall, for testing. In this way, our models
are tested on images for which training was absent.

For the reduced version of the Nordland dataset, we compared our methods NV-
ESN and NV-SPARCE-ESN to FlyNet+CANN and FlyNet+RNN. In that case the
comparison was straightforward as all networks are classifiers presuming each
place is a class.

We also compared NV-ESN, PatchL-ESN, NV-SPARCE-ESN and PatchL-SPARCE-ESN on
the Oxford RobotCar dataset with NetVLAD and Patch-NetVLAD, which were not
available from the literature. For these two models, we produced results using
the source-code provided by the authors.

###  III-C Hyper-parameter Tuning On ESNs

The lack of a validation set for the considered tasks makes hyper-parameter
selection challenging. This difficulty is increased by the small number of
samples in the training set (i.e. one sample per place) and by the major
statistical differences between the training and test data. In particular, the
seasonal differences in the acquisition of reference and query data lead to
the possible presence or absence of snow and shifts in colour intensity. In
our preliminary experiments, different hyper-parameters often reached perfect
accuracy (i.e. $100\%$) on the training set and had reduced performance on the
test set. For this reason, a validation test was required.

We believe that there is a lack of clarity in previous research works
regarding the definition of a methodology to overcome the problem of hyper-
parameter selection. We therefore tuned the hyper-parameters of the reservoir
by using a small percentage (i.e. $10\%$) of samples of the test set as
validation. In other words, while the read-out was always optimised from
training samples, hyper-parameters were optimised through grid search over the
performance achieved on $10\%$ of the test data. We did not estimate
performance on validation data when computing scores on the test dataset.
Being aware of the limitations of this methodology, we show how it is possible
to use the test set of one task as validation for another task with little
performance loss, demonstrating that the model performs well if the hyper-
parameters were selected to be robust to non-excessive statistical changes
(see Section IV-B).

##  IV Results

###  IV-A Assessing ESNs On Visual Place Recognition

The performance of the NV-ESN and NV-SPARCE-ESN models were first evaluated in
four datasets (GardensPoint, SPEDTest, ESSEX3IN1 and Corridor). Fig. 2 shows
that both ESN variants outperform single-view matching models (including
NetVLAD with read-out and hidden layers) in all four conditions (with the
exception of AMOSNet in Corridor). The NV-ESN achieves mean accuracy scores of
$0.75$, $0.99$, $1.0$ and 0.88 and mean AUC scores of $0.9$, $1.0$, $1.0$ and
$0.93$. The addition of the NV-SPARCE layer provides additional improvement
with accuracy scores of $0.77$, $0.99$, $1.0$, and $0.91$ and mean AUC scores
of $0.93$, $1.0$, $1.0$ and $0.93$. The ESNs used $N=1000$ reservoir neurons.

###  IV-B Robustness Of Hyper-parameters

We also analysed the sensitivity of the ESNs with respect to hyper-parameter
selection. Fig. 3 shows accuracy scores for hyper-parameters tuned by training
the models on GardensPoint and maintaining them when training in SPEDTest,
ESSEX3IN1 and Corridor. The reason we chose the hyper-parameters for
GardensPoint is that generalisation is more likely to occur when the baseline
task is more complex than the new tasks to which it is applied. Indeed, richer
and more difficult datasets can lead neural networks to discover high-level
features that are transferable to simpler datasets, while the contrary is
difficult. Fig. 3 demonstrates how, even with sub-optimal hyper-parameters,
the introduction of ESNs leads to higher performance in comparison to single-
view matching models: NetVLAD-C and NV. Moreover, the performance remains
above $86\%$ for both accuracy and AUC compared to the virtually perfect
scores achieved when hyper-parameters were tuned using the same dataset (see
Fig 2).

Figure 3:  Robustness of hyper-parameters. Performance is maintained despite
using the hyper-parameters optimised for a different dataset (GardensPoint).
The two variants of ESN are well above the accuracy achieved by NetVLAD-C and
NV (horizontal lines). The box plots represent the distribution of $20$
trials.  TABLE I:  ESNs outperform state-of-the-art single view matching
models in the challenging Nordland and Oxford RobotCar benchmarks. The larger
datasets pose a greater challenge for algorithms with an order of magnitude
more images and extreme variance between training and test sets (seasonal and
day/night). Best results are found for PatchL-ESN in Nordland, and PatchL-
SPARCE-ESN in Oxford RobotCar.

| Nordland | Oxford RobotCar
---|---|---
| Summer vs Winter | Day vs Night
Method | R@1 | R@5 | R@10 | R@1 | R@5 | R@10
NetVLAD [5] | $13.0$ | $20.6$ | $25.0$ | $31.2$ | $48.6$ | $58.3$
SuperGlue [7] | $29.1$ | $33.4$ | $35.0$ | $-$ | $-$ | $-$
DELG global [8] | $23.4$ | $35.4$ | $41.7$ | $-$ | $-$ | $-$
DELG local [8] | $60.1$ | $63.5$ | $64.6$ | $-$ | $-$ | $-$
Patch-NetVLAD [9] | $44.9$ | $50.2$ | $52.2$ | $80.7$ | $84.8$ | $86.4$
NV-ESN | $47.5$ | $61.0$ | $66.4$ | $44.0$ | $58.1$ | $65.5$
NV-SPARCE-ESN | $46.7$ | $60.3$ | $65.9$ | $80.4$ | $87.9$ | $91.5$
PatchL-ESN | $\mathbf{66.4}$ | $\mathbf{76.0}$ | $\mathbf{78.7}$ | $75.9$ | $85.2$ | $87.7$
PatchL-SPARCE-ESN | $66.3$ | $75.7$ | $78.6$ | $\mathbf{88.8}$ | $\mathbf{97.0}$ | $\mathbf{98.2}$

###  IV-C Performance On Larger Datasets

To assess the scalability of the ESNs, and to compare their performance to
state-of-the-art models, performance was analysed in the full Nordland and
Oxford RobotCar datasets as in [17]. The methodology adopted is the same as in
Section IV, but the number of reservoir nodes in the ESN is increased to
$8000$ and $6000$ for the Nordland and the Oxford RobotCar datasets,
respectively. Results presented in Table I, show that best performance was
achieved for PatchL-ESN in the Nordland dataset, when compared with state-of-
the-art single view matching models.  We would like to note that, in our
models we have used exclusively the “light” version of Patch-NetVLAD for
computational efficiency, while the performance reported in Table I of the
original Patch-NetVLAD exploited its most complex version. Considering the
results for Nordland (summer vs winter) and Oxford RobotCar (day vs night), it
is evident that the temporal features captured by the ESNs cause it to reach
competitive performance. NV-ESN reports recall@n that are higher than the
original Patch-NetVLAD for the Nordland dataset, while NV-SPARCE-ESN is
surprisingly successful on the Oxford RobotCar dataset, with performance that
is inferior only to its more complex variant PatchL-SPARCE-ESN and comparable
to Patch-NetVLAD. The utilisation of the local features from [9] further
improve these results, and the model PatchL+SPARCE-ESN is the best performing
network overall. Indeed, PatchL+SPARCE-ESN reports the highest score for
Oxford RobotCar and it is only slightly worse than PatchL-ESN for Nordland. It
is likely that these results might improve by using the complete variant of
Patch-NetVLAD. Here, we chose to keep the model as simple as possible, given
that it still outperforms the other techniques. Thus, ESNs constitute an
addition to other neural architectures that can lead to considerable
performance improvements and to state-of-the-art results.

###  IV-D Robustness With Respect To Start-Point

We investigate if the proposed models satisfy two conditions that are
necessary for a potential implementation on a robot. First, the model should
maintain high performance if we start from different locations of the input
sequence. Second, the amount of temporal information that needs to be
processed before the system can reach satisfactory results should be low. Fig.
4 shows the recall@n of an ESN as the number of images being processed
increases from the starting location. The recall@n are averaged across
different possible initial locations and computed on the test set of the
Nordland dataset. The average performance increases quickly and steadily,
showing that the model needs to process in the order of $50$ images before the
trends converge. The fact that the performance of Fig. 4 is obtained by
averaging from various starting points further demonstrates the robustness of
the model with respect to variation of the initial location.

Figure 4:  Robustness with respect to start-point. Performance of NV-ESN as
the image number from a starting location increases. The trends are averaged
across $500$ different starting locations of the Winter Nordland datasets.

###  IV-E Comparing ESN With Sequential VPR Models

In this section, we benchmark the performance of ESNs against sequence
matching VPR models.  The majority of sequential models for VPR benchmarks
adopt different learning and testing methodologies, and it is consequently
difficult to make direct comparisons to previously published results. However,
it is possible to directly compare our networks with with two models recently
reported to achieve great performance [16] in a subset of the Nordland dataset
[21] composed by 1000 images. Both previous models use a bio-inspired
feedforward neural network (FlyNet) to encode visual information and either a
recurrent neural network (RNN) or a continuous attractor neural network (CANN)
to introduce temporality. Fig. 5 shows accuracy scores of $0.72$ and $0.92$
for the standard NV-ESN and NV-SPARCE-ESN respectively. For the AUC test, NV-
ESN achieves scores of $0.95$, with NV-SPARCE-ESN improving results to $0.98$.
This compares favourably to both static view matching models (e.g. NetVLAD+HL)
which score $0.24$, and sequential models which score $0.21$ (FlyNet+RNN) and
$0.91$ (FlyNet+CANN). While it is not possible to directly and fairly compare
with models such as SeqNet [17] that use distances between images, we have
devised a methodology that allows us to provide scores of our methods in
classifying previously unseen data.

The performance obtained for the larger version of the Nordland dataset are
shown in Table II (summer-winter vs spring-fall). The two results reported for
each recall@n correspond to the case where single, or consecutive image pairs,
were randomly selected for testing respectively (see Section III-B for more
details). In the latter case, because the connections to adjacent pairs of
output neurons are not trained, the performance degrades. However, even in a
learning and testing paradigm not completely suited to our model, the proposed
networks are able to generalise. This additional comparison between ESNs and
previously published results (Table II) confirms that the former can achieve
high performance in relation to sequential techniques. Moreover, the memory of
ESNs is solely reliant on the internal dynamics, while the vast majority of
other sequential models rely on a direct comparison among sequences, which
need to be stored in memory.

Figure 5:  Direct comparison to sequential classification models in the
Nordland dataset. The NV-ESN model and in particular NV-SPARCE-ESN, outperform
FlyNet+RNN and FlyNet+CANN, taken from [16] and NV when compared with reported
AUC scores. The horizontal lines report the performance of FlyNet and NV
models. The box plots represent the distribution of $20$ trials. TABLE II:
Indicative comparison of ESNs to published results with sequential models in
the Nordland dataset. We note that the NV-ESN cannot be evaluated in the same
way as the other models so a direct comparison is not appropriate.

| Nordland
---|---
| Summer-Winter vs Spring-Fall
Method | R@1 | R@5 | R@20
NetVLAD+Smoothing [17] | $44.0$ | $59.0$ | $72.0$
NetVLAD+Delta [17] | $56.0$ | $70.0$ | $80.0$
NetVLAD+SeqMatch [17] | $61.0$ | $71.0$ | $78.0$
SeqNet ($S_{5}$) [17] | $79.0$ | $90.0$ | $94.0$
HVPR ($S_{5}$ to $S_{1}$) [17] | $79.0$ | $89.0$ | $94.0$
NV-ESN | $82.7/77.6$ | $91.2/85.9$ | $95.1/90.9$

##  V Conclusions

Here we have demonstrated the viability of ESNs as a solution to the VPR
problem. Both ESN variants achieve comparable or better performance than
single-view matching models in four evaluation datasets. On larger datasets
such as full Nordland and Oxford RobotCar, the two variants, combined with
local features, were the best models. With respect to sequential models, where
a fair comparison was permitted, two ESN variants achieved performance
above/equal to the class-leading results on the reduced Nordland dataset.
While performance is comparable, we note that FlyNet [16] has many fewer
parameters. However, the ESN does not require images to be cached during
multiple comparisons or any external cues. A comparison with sequential models
is, in general, difficult due to their use of distances that allow unseen
images to be compared. In this sense, these sequential models have an
advantage at the cost that caching images is required in order to recognise a
new place.

The addition of SPARCE to the standard ESN improved performance considerably
in almost all cases, showing how the introduction of sparse representations
can efficiently help the classification process. An exception was the full
Nordland dataset, where we did not find any advantage by adding this
technique, and the reasons are worthy of further investigation.

When it comes to ESN advances, an intriguing future course of action is to
take inspiration from invertebrate mini-brains. An interesting point being
that they possess analogous structural motifs of both deep and shallow ESNs. A
simple example is the insect mushroom body. This is considered the cognitive
centre of the insect brain [41] and is necessary for learning relationships
sequences and patterns in honey bees [41, 42]. Structurally the mushroom body
is a three-layer network with a compact input layer, an expanded middle layer
of inter-neurons called Kenyon cells (KC), and a small layer of output neurons
[43]. The connections between the KC and output neurons are plastic and
modified by learning [44], and there are chemical and electrical synapses
between the KC [45]. These features are analogous to the recurrent connections
in the reservoir layer of an ESN, and it has been hypothesised [31] that these
recurrent connections in the KC layer could contribute to the reverberant
activity of the mushroom body that supports forms of memory [46]. Further, the
technique SPARCE is inspired by the low activity exhibited by the KC. Given
the similar structures of ESN and mushroom body, insights gained from
neurobiology could help shape the future ESN investigations and in turn,
analysis of the optimal structure for VPR could shed light on the function of
different brain areas. For instance, in [32], hierarchical structures with
fast and slow information processing times are investigated, reminiscent of
neural processing observed in insect brains. See [47] (supplementary
materials) for a preliminary study of such networks.

ESN efficiently exploit information in sequences but in practice it is also
desirable that places are recognised from a single input image allowing
robotics to truly solve the kidnapped robot problem. However, in the cases
where such methods fail, traversing portions of a familiar path can help to
disambiguate input. In this respect, our models require “viewing” of
approximately $50$ images (for the Nordland set) in order to start correctly
identifying their location.

ESNs provide a means to exploit such temporal dynamics using only visual data
but more powerful variants require tuning of a large number of parameters
which may not be possible when only a small amount of training examples are
provided. Other methods [11, 16] have focused on low-parameter models but
often require additional cues such as velocity to focus the image search.
Ensemble methods [48, 49] that combine these features are emerging that may
provide the best of both worlds.

## References

  * [1] S. Lowry, N. Sünderhauf, P. Newman, J. J. Leonard, D. Cox, P. Corke, and M. J. Milford, “Visual place recognition: A survey,” _IEEE Transactions on Robotics_ , vol. 32, no. 1, pp. 1–19, 2015.
  * [2] C. Masone and B. Caputo, “A survey on deep visual place recognition,” _IEEE Access_ , vol. 9, pp. 19 516–19 547, 2021.
  * [3] X. Zhang, L. Wang, and Y. Su, “Visual place recognition: A survey from deep learning perspective,” _Pattern Recognition_ , vol. 113, p. 107760, 2021\.
  * [4] A. Torii, R. Arandjelovic, J. Sivic, M. Okutomi, and T. Pajdla, “24/7 place recognition by view synthesis,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2015, pp. 1808–1817.
  * [5] R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic, “NetVLAD: CNN architecture for weakly supervised place recognition,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2016, pp. 5297–5307.
  * [6] Z. Chen, A. Jacobson, N. Sünderhauf, B. Upcroft, L. Liu, C. Shen, I. Reid, and M. Milford, “Deep learning features at scale for visual place recognition,” in _IEEE International Conference on Robotics and Automation_ , 2017, pp. 3223–3230.
  * [7] P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 4938–4947.
  * [8] B. Cao, A. Araujo, and J. Sim, “Unifying deep local and global features for image search,” in _European Conference on Computer Vision_. Springer, 2020, pp. 726–743.
  * [9] S. Hausler, S. Garg, M. Xu, M. Milford, and T. Fischer, “Patch-netvlad: Multi-scale fusion of locally-global descriptors for place recognition,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 14 141–14 152.
  * [10] N. V. Keetha, M. Milford, and S. Garg, “A hierarchical dual model of environment-and place-specific utility for visual place recognition,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 4, pp. 6969–6976, 2021\.
  * [11] M. J. Milford and G. F. Wyeth, “SeqSLAM: Visual route-based navigation for sunny summer days and stormy winter nights,” in _IEEE International Conference on Robotics and Automation_ , 2012, pp. 1643–1649.
  * [12] M. J. Milford, “Vision-based place recognition: how low can you go?” _The International Journal of Robotics Research_ , vol. 32, no. 7, pp. 766–789, 2013.
  * [13] P. Hansen and B. Browning, “Visual place recognition using HMM sequence matching,” in _2014 IEEE/RSJ International Conference on Intelligent Robots and Systems_ , 2014, pp. 4549–4555.
  * [14] E. Kagioulis, A. Philippides, P. Graham, J. C. Knight, and T. Nowotny, “Insect inspired view based navigation exploiting temporal information,” in _Conference on Biomimetic and Biohybrid Systems_. Springer, 2020, pp. 204–216.
  * [15] L. Zhu, M. Mangan, and B. Webb, “Spatio-temporal memory for navigation in a mushroom body model,” in _Conference on Biomimetic and Biohybrid Systems_. Springer, 2020, pp. 415–426.
  * [16] M. Chancán, L. Hernandez-Nunez, A. Narendra, A. B. Barron, and M. Milford, “A hybrid compact neural architecture for visual place recognition,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 993–1000, 2020\.
  * [17] S. Garg and M. Milford, “SeqNet: Learning descriptors for sequence-based hierarchical place recognition,” _IEEE Robotics and Automation Letters_ , vol. 6, no. 3, pp. 4305–4312, 2021.
  * [18] W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The Oxford Robotcar dataset,” _The International Journal of Robotics Research_ , vol. 36, no. 1, pp. 3–15, 2017.
  * [19] T. Sattler, W. Maddern, C. Toft, A. Torii, L. Hammarstrand, E. Stenborg, D. Safari, M. Okutomi, M. Pollefeys, J. Sivic, _et al._ , “Benchmarking 6DOF outdoor visual localization in changing conditions,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 8601–8610.
  * [20] A. Torii, J. Sivic, T. Pajdla, and M. Okutomi, “Visual place recognition with repetitive structures,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2013, pp. 883–890.
  * [21] N. Sünderhauf, P. Neubert, and P. Protzel, “Are we there yet? Challenging SeqSLAM on a 3000 km journey across all four seasons,” in _Proc. of Workshop on Long-Term Autonomy, IEEE International Conference on Robotics and Automation_. Citeseer, 2013, p. 2013.
  * [22] F. Warburg, S. Hauberg, M. Lopez-Antequera, P. Gargallo, Y. Kuang, and J. Civera, “Mapillary street-level sequences: A dataset for lifelong place recognition,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 2626–2635.
  * [23] H. Jaeger, M. Lukoševičius, D. Popovici, and U. Siewert, “Optimization and applications of Echo State Networks with leaky-integrator neurons,” _Neural Networks_ , vol. 20, no. 3, pp. 335–352, 2007.
  * [24] H. Jaeger, “The “echo state” approach to analysing and training recurrent neural networks-with an erratum note,” _Bonn, Germany: German National Research Center for Information Technology GMD Technical Report_ , vol. 148, no. 34, p. 13, 2001.
  * [25] M. Hermans and B. Schrauwen, “Recurrent kernel machines: Computing with infinite echo state networks,” _Neural Computation_ , vol. 24, no. 1, pp. 104–133, 2012.
  * [26] D. Li, M. Han, and J. Wang, “Chaotic time series prediction based on a novel robust Echo State Network,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 23, no. 5, pp. 787–799, 2012.
  * [27] A. Deihimi and H. Showkati, “Application of Echo State Networks in short-term electric load forecasting,” _Energy_ , vol. 39, no. 1, pp. 327–340, 2012.
  * [28] P. G. Plöger, A. Arghir, T. Günther, and R. Hosseiny, “Echo State Networks for mobile robot modeling and control,” in _Robot Soccer World Cup_. Springer, 2003, pp. 157–168.
  * [29] K. Ishu, T. van Der Zant, V. Becanovic, and P. Ploger, “Identification of motion with Echo State Network,” in _MTS/IEEE Techno-Ocean_ , vol. 3, 2004, pp. 1205–1210.
  * [30] C. Hartland and N. Bredeche, “Using Echo State Networks for robot navigation behavior acquisition,” in _IEEE International Conference on Robotics and Biomimetics_ , 2007, pp. 201–206.
  * [31] L. Manneschi, A. C. Lin, and E. Vasilaki, “SpaRCe: Improved Learning of Reservoir Computing Systems through Sparse Representations,” _IEEE Transactions on Neural Networks and Learning Systems_ , 2021.
  * [32] L. Manneschi, M. O. Ellis, G. Gigante, A. C. Lin, P. Del Giudice, and E. Vasilaki, “Exploiting Multiple Timescales In Hierarchical Echo State Networks,” _Frontiers in Applied Mathematics and Statistics_ , 2021.
  * [33] M. Lukoševičius, “A practical guide to applying echo state networks,” in _Neural networks: Tricks of the trade_. Springer, 2012, pp. 659–686.
  * [34] M. Zaffar, S. Garg, M. Milford, J. Kooij, D. Flynn, K. McDonald-Maier, and S. Ehsan, “VPR-Bench: Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change,” _International Journal of Computer Vision_ , pp. 1–39, 2021.
  * [35] S. Garg, T. Fischer, and M. Milford, “Where is your place, visual place recognition?” _arXiv preprint arXiv:2103.06443_ , 2021.
  * [36] C. Toft, W. Maddern, A. Torii, L. Hammarstrand, E. Stenborg, D. Safari, M. Okutomi, M. Pollefeys, J. Sivic, T. Pajdla, _et al._ , “Long-term visual localization revisited,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2020.
  * [37] A. Glover, “Day and night, left and right,” 2014. [Online]. Available: https://doi.org/10.5281/zenodo.4590133
  * [38] M. Zaffar, S. Ehsan, M. Milford, and K. D. McDonald-Maier, “Memorable maps: A framework for re-defining places in visual place recognition,” _IEEE Transactions on Intelligent Transportation Systems_ , 2020.
  * [39] Z. Chen, L. Liu, I. Sa, Z. Ge, and M. Chli, “Learning context flexible attention model for long-term visual place recognition,” _IEEE Robotics and Automation Letters_ , vol. 3, no. 4, pp. 4015–4022, 2018.
  * [40] W. Maddern, G. Pascoe, C. Linegar, and P. Newman, “1 year, 1000 km: The oxford robotcar dataset,” _The International Journal of Robotics Research_ , vol. 36, no. 1, pp. 3–15, 2017.
  * [41] R. Menzel and M. Giurfa, “Cognitive architecture of a mini-brain: the honeybee,” _Trends in Cognitive Sciences_ , vol. 5, no. 2, pp. 62–71, 2001\.
  * [42] A. J. Cope, E. Vasilaki, D. Minors, C. Sabo, J. A. Marshall, and A. B. Barron, “Abstract concept learning in a simple neural network inspired by the insect brain,” _PLoS Computational Biology_ , vol. 14, no. 9, p. e1006435, 2018\.
  * [43] S. E. Fahrbach, “Structure of the mushroom bodies of the insect brain,” _Annual Review of Entomology_ , vol. 51, pp. 209–232, 2006.
  * [44] B. Gerber, H. Tanimoto, and M. Heisenberg, “An engram found? evaluating the evidence from fruit flies,” _Current Opinion in Neurobiology_ , vol. 14, no. 6, pp. 737–744, 2004.
  * [45] Z. Zheng, J. S. Lauritzen, E. Perlman, C. G. Robinson, M. Nichols, D. Milkie, O. Torrens, J. Price, C. B. Fisher, N. Sharifi, _et al._ , “A complete electron microscopy volume of the brain of adult drosophila melanogaster,” _Cell_ , vol. 174, no. 3, pp. 730–743, 2018.
  * [46] P. Cognigni, J. Felsenberg, and S. Waddell, “Do the right thing: neural network mechanisms of memory formation, expression and update in drosophila,” _Current Opinion in Neurobiology_ , vol. 49, pp. 51–58, 2018\.
  * [47] A. Ozdemir, M. Scerri, A. B. Barron, A. Philippides, M. Mangan, E. Vasilaki, and L. Manneschi, “EchoVPR: Echo State Networks for Visual Place Recognition,” _arXiv preprint arXiv:2110.05572_ , 2021.
  * [48] S. Hausler, A. Jacobson, and M. Milford, “Multi-process fusion: Visual place recognition using multiple image processing methods,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 2, pp. 1924–1931, 2019.
  * [49] T. Fischer and M. Milford, “Event-based visual place recognition with ensembles of temporal windows,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 4, pp. 6924–6931, 2020.

See pages - of supp.pdf