# System Size and Flavour Dependence of Chemical Freeze-out Temperatures in
ALICE pp, pPb and PbPb Collisions at LHC Energies

Fernando Antonio Flor<EMAIL_ADDRESS>Gabrielle Olinger  René Bellwied
Department of Physics, University of Houston, Houston, Texas 77204, USA

(August 27, 2024)

###### Abstract

We present the system size and flavour dependence of the chemical freeze-out
temperature ($T_{\mathrm{ch}}$) at vanishing baryo-chemical potential
calculated via thermal fits to experimental yields for several centrality
classes in pp, pPb and PbPb collisions measured by ALICE. Using the Thermal-
FIST Hadron Resonance Gas model package, we compare the quality of fits across
various treatments of strangeness conservation under different freeze-out
conditions as a function of the charged particle multiplicity density
($\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$). Additionally, we examine
how the anti-hadron to pion yield ratios of light and strange baryons, as well
as the $\phi$ meson, evolve within a flavour-dependent model. Through a unique
two-temperature chemical-freezeout approach, we show that flavour dependence
of $T_{\mathrm{ch}}$ leads to a natural explanation of strangeness enhancement
from small to large systems at LHC energies without requiring any canonical
suppression for small $\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$.

###### keywords:

Strangeness Enhancement , Sequential Flavour Freeze-out , Statistical
Hadronization , Hadron Resonance Gas

##  1 Introduction

Hadronization and chemical freeze-out have been suggested to coincide at the
phase boundary in the Quantum Chromodynamics (QCD) phase diagram based on
results from Statistical Hadronization Models (SHMs) using particle yields
measured at the Large Hadron Collider (LHC) and the Relativistic Heavy Ion
Collider (RHIC), when compared to pseudo-critical temperature calculations
from temperature dependent continuum extrapolations of the chiral
susceptibilities on the lattice [1, 2, 3, 4, 5]. A point of interest emerges
concerning whether the phase transition from quark to hadron degrees of
freedom occurs at the same temperature for all particle species and/or quark
flavours.
Final state particle yields have been successfully reproduced by SHMs to nine
orders of magnitude over a wide energy range in high energy collisions of
heavy ions [6, 7]. SHMs typically use experimental hadron yields from central
events (0 - 10%) in relativistic heavy ion collisions as an anchor for
determining common freeze-out parameters in the QCD phase diagram – namely,
the chemical freeze-out temperature ($T_{\mathrm{ch}}$) and the baryo-chemical
potential ($\mu_{\mathrm{B}}$) – within a Grand Canonical Ensemble (GCE),
where baryon number, electric charge and strangeness are conserved on average.
The GCE treatment has been shown to inadequately reproduce experimental
results in a centrality dependent manner across pp, pPb and PbPb collision
systems measured by the ALICE Collaboration – particularly where
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}\leq 20$ [8]. The
aforementioned is often attributed to the presence of canonical suppression in
the smaller systems and may be partially remedied by employing a Strangeness
Canonical Ensemble (SCE), in which strangeness is explicitly conserved within
some correlation volume $V_{\mathrm{C}}$ using a single value of
$T_{\mathrm{ch}}$ for all particle species (1CFO). The question arises as to
whether additional canonical strangeness suppression in small systems provides
an adequate description of the overall production of final state
(multi)strange hadrons as measured at LHC energies. The presence of such a
suppression, commonly represented by a strangeness saturation parameter
($\gamma_{\mathrm{{S}}}$) within the fireball volume of hadronic collisions,
is known to decrease with increasing collision energy [9, 10].

Thus, canonical strangeness suppression scaling as an inverse function of
collision energy can be equated with an increasing value of
$\gamma_{\mathrm{{S}}}$. At LHC energies, $\gamma_{\mathrm{{S}}}$ is expected
to be asymptotic to unity [8, 11], i.e. full saturation of strangeness is
achieved. The dominating presence of strangeness enhancement at ALICE, even in
small systems, is evident from an energy dependent comparison of final state
anti-hadron to $\pi^{+}$ yields measured by ALICE and STAR [12], where
production of strange baryons is fully saturated for collision energy values
above $\sqrt{s_{\rm{NN}}}=62.4$ GeV.

Nevertheless, in order to fully describe the experimental data, further
considerations are made regarding the interplay between $V_{\mathrm{C}}$, the
fireball volume ($V$), the experimental rapidity window ($\Delta$y) and
$\gamma_{\mathrm{{S}}}$ [13, 14]. This letter aims at providing a novel
description of strangeness enhancement across all three collision systems
measured by ALICE, assuming full saturation of strangeness is inherently
present at LHC energies. Our description of final state hadron yields within
the SHM framework relies not on the canonical suppression paradigm, but rather
on flavour-dependent freeze-out temperatures and volumes across increasing
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ values.

##  2 Sequential Strangeness Freeze-out

Flavour-dependent freeze-out temperatures in the crossover region of the QCD
phase diagram have been predicted by continuum extrapolated susceptibilities
of single flavour quantum numbers on the lattice [15, 16]. Flavour specific
susceptibility ratios ${\chi_{4}}/{\chi_{2}}$, suggested as an observable for
directly determining freeze-out temperatures [17], show a deviation of the
lattice and Hadron Resonance Gas (HRG) model calculations at the peaks of the
lattice data, which occur at flavour-dependent temperatures differing by
$15-20$ MeV from light to strange quarks [16]. Similar temperature differences
between the light and strange mesons have also been shown in net-particle
fluctuation measurements by the STAR Collaboration [18, 19, 20, 21]. An HRG-
based study was also performed in a similar analysis on both off-diagonal and
diagonal second order correlators of conserved charges [22].

In a previous letter, at vanishing $\mu_{\rm{B}}$, we calculated a light
flavour freeze-out temperature $T_{\mathrm{L}}=150.2\pm 2.6$ MeV and a strange
flavour freeze-out temperature $T_{\mathrm{S}}=165.1\pm 2.7$ MeV, by employing
the GCE approach to heavy ion collisions within the framework of the Thermal
FIST (‘The FIST’) HRG model package and varying the particle species included
in each thermal fit [23].

In this letter, we extend this flavour-dependent two chemical freeze-out
(2CFO) temperature approach to pp, pPb and PbPb collision systems measured by
ALICE as a function of increasing
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$, rendering a natural
explanation of strangeness enhancement from small to large systems at
vanishing $\mu_{\rm{B}}$ without further modifying $\Delta$y, $V_{\mathrm{C}}$
nor $\gamma_{\mathrm{S}}$.

##  3 Model and Data Preparation

All calculations shown here were performed using the open source ‘The FIST’
thermal model package [24]. In this iteration, we modeled an ideal non-
interacting gas of hadrons and resonances within both, the GCE and the SCE
scenarios, for the sake of comparison. The analysis was two-fold: to gauge the
sensitivity of the chemical freeze-out temperature at vanishing $\mu_{\rm{B}}$
relative to the ensemble of choice, and to employ a 2CFO treatment onto the
reigning ensemble.

As the HRG input list, we used the PDG2016+ hadronic spectrum [25]. The
PDG2016+ hadronic spectrum has been shown to be an optimized compromise
between too few and too many resonant states when compared to lattice QCD
predictions [25]; it includes a total of 738 states (i.e. *, **, *** and ****
states from the 2016 Particle Data Group Data Book [26]).

Yield data for $\pi^{\text{+}}$, $\pi^{\text{-}}$, $K^{\text{+}}$,
$K^{\text{-}}$, $p$, $\bar{p}$, $\Lambda$, $\bar{\Lambda}$, $\Xi^{-}$,
$\bar{\Xi}^{+}$, $\Omega^{-}$, $\bar{\Omega}^{+}$, $K_{\rm{S}}^{0},$ and
$\phi$ for ALICE pp collisions at $\sqrt{s}=7.00$ TeV [27], pPb collisions at
$\sqrt{s_{\rm{NN}}}=5.02$ TeV [28, 29, 30] and PbPb collisions at
$\sqrt{s_{\rm{NN}}}=2.76$ TeV [31, 32, 33, 34] across all available
multiplicity classes were included in our analysis.

Table 1: Available centrality bins and corresponding values of charged
particle multiplicity density for ALICE pp collisions at $\sqrt{s}=7.00$ TeV,
pPb collisions at $\sqrt{s_{\rm{NN}}}=5.02$ TeV and PbPb collisions at
$\sqrt{s_{\rm{NN}}}=2.76$ TeV. For the pp sample, the multiplicity classes are
labelled in accordance to their generalized definitions in [27]. pp at 7.00
TeV
---
Multiplicity Class | Centrality Bin | $\langle dN_{\mathrm{ch}}/d\eta\rangle$
I-II | 0-5$\%$ | 17.47$\pm$ 0.524
III-VI | 5-28$\%$ | 10.383$\pm$0.313
VII-VIII | 28-48$\%$ | 6.057$\pm$0.19
IX-X | 48-100$\%$ | 2.886$\pm$0.135
pPb at 5.02 TeV
| 0-5$\%$ | 45$\pm$1
| 5-10$\%$ | 36.2$\pm$0.8
| 10-20$\%$ | 30.5$\pm$0.7
| 20-40$\%$ | 23.2$\pm$0.5
| 40-60$\%$ | 16.1$\pm$0.4
| 60-80$\%$ | 9.8$\pm$0.2
| 80-100$\%$ | 4.3$\pm$0.1
PbPb at 2.76 TeV
| 0-10$\%$ | 1447.5$\pm$54.5
| 10-20$\%$ | 966$\pm$37
| 20-40$\%$ | 537.5$\pm$19
| 40-60$\%$ | 205$\pm$7.5
| 60-80$\%$ | 55.5$\pm$3

All throughout, we follow a shorthand notation when naming our fits to
(anti)particle species (e.g. $\Omega$ refers to both $\Omega^{-}$ and
$\bar{\Omega}^{+}$, etc.), unless explicitly noted otherwise.

In the GCE configuration, the thermal fits were performed with $T_{\rm{ch}}$
(MeV) and V (fm3) as free parameters, keeping $\mu_{\rm{B}}=0$ – as is the
case at LHC energies – and setting $\gamma_{\mathrm{S}}$ and
$\gamma_{\mathrm{q}}$ to unity in order to ensure a full saturation of
strangeness and electric charge. Systematic HRG-based studies on the
determination of the latter three parameters at top LHC energies can be found
in Refs. [35, 36]. We focused on varying the particle species included in the
fit in order to gauge the sensitivity of $T_{\rm{ch}}$ to each fit. The
particle species included in our flavour-dependent temperature fits were $\pi
Kp$, $\pi Kp\Lambda\Xi\Omega K^{0}_{\rm{S}}\phi$ and $K\Lambda\Xi\Omega
K^{0}_{\rm{S}}\phi$, hereinafter referred to as light, all and strange,
respectively. Since the kaon yields have been shown to be insensitive to the
freeze-out temperature [37], we included them in light fit for the sake of
having sufficient degrees of freedom. We compared the extracted $T_{\rm{ch}}$
values, for the light, strange and all fits, and their corresponding
$\chi^{2}$/dof measures as a function of their
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ values across all three
collision systems.

In the SCE configuration, our thermal fits were instead performed with
$T_{\rm{ch}}$ (MeV) and V (fm3) as free parameters, keeping $\mu_{\rm{B}}=0$,
$V_{\mathrm{C}}=V$ and setting $\gamma_{\mathrm{S}}$ and $\gamma_{\mathrm{q}}$
to unity. Setting $V_{\mathrm{C}}=V$ was done in order to guarantee a local
conservation of strangeness within the calculated fireball volume per unit
rapidity.

The initial SCE analysis followed the same procedure than the early GCE trial,
however, the following procedures were not performed for the GCE approach due
to the deterioration of the quality of its fits across all three collision
systems. A similar approach was presented in [38, 39], with a notable
difference in the manner in which the pion yields were calculated in the
thermal model. In our case, the pion, and all other, yields were based on
experimental values measured over a single unit of rapidity around mid-
rapidity.

For our SCE analysis, the $\pi Kp\Lambda\Xi\Omega K_{\rm{S}}^{0}\phi$ yields
were calculated by fixing the temperature to the flavour specific freeze-out
temperatures at $\mu_{\rm{B}}=0$ from our original study [23], such that
$T_{\mathrm{L}}=150$ MeV for $\pi p$ and $T_{\mathrm{S}}=165$ MeV for
$K\Lambda\Xi\Omega K_{\rm{S}}^{0}\phi$. We calculated the anti-hadron to
$\pi^{+}$ ratio as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ and compared our results
with the experimental data. The choice of the anti-hadron to $\pi^{+}$ ratio
was made, as in [12], in order to use only particles produced during the
evolution of the fireball. This will allow us to facilitate future centrality
dependent comparisons to lower collision energy measurements, where
$\mu_{\mathrm{B}}\neq 0$. In order to explicitly show the temperature
dependence of the fireball volumes, we calculated the volumes across all three
systems as a function of $\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$
with temperatures fixed to the aforementioned flavour specific temperatures,
as well as $T=158$ MeV for a non-flavour-dependent temperature.

##  4 Results and Discussion

Figure 1: Top Panel: Flavour-dependent ‘The FIST’ GCE Fits to Yields using the
PDG2016+ hadronic spectrum for ALICE pp collisions at $\sqrt{s}=7.00$ TeV, pPb
collisions at $\sqrt{s_{\rm{NN}}}=5.02$ TeV and PbPb collisions at
$\sqrt{s_{\rm{NN}}}=2.76$ TeV, respectively, shown as as closed circles, open
diamonds and closed squares as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$. For all fits,
$\mu_{\mathrm{B}}=0$ and $\gamma_{\mathrm{S}}=1$, while $T_{\mathrm{ch}}$, and
$V$ were used as free parameters. Red, green, and blue points represent the
light, all, and strange fits, respectively. Bottom Panel: Flavour-dependent
‘The FIST’ SCE Fits to Yields using the PDG2016+ hadronic spectrum for ALICE
pp collisions at $\sqrt{s}=7.00$ TeV, pPb collisions at
$\sqrt{s_{\rm{NN}}}=5.02$ TeV and PbPb collisions at $\sqrt{s_{\rm{NN}}}=2.76$
TeV as a function of $\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$. For
all SCE fits, $\mu_{\mathrm{B}}=0$, $\gamma_{\mathrm{S}}=1$ and
$V_{\mathrm{C}}=V$, while $T_{\mathrm{ch}}$, and $V$ were used as free
parameters. The bottom panel follows the same labeling convention used in the
top panel. Figure 2: Top Panel: ‘The FIST’ $\chi^{2}$/dof values of the GCE
(cyan) and SCE (magenta) fits used in the all fits of the top and bottom
panels of Figure fig. 1, respectively, using the PDG2016+ hadronic spectrum
for ALICE pp collisions at $\sqrt{s}=7.00$ TeV, pPb collisions at
$\sqrt{s_{\rm{NN}}}=5.02$ TeV and PbPb collisions at $\sqrt{s_{\rm{NN}}}=2.76$
TeV as a function of $\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$.
Bottom Panel: ‘The FIST’ $\Delta$($\chi^{2}$/dof) values for the SCE fits used
in the light (orange) and strange (purple) fits of the bottom panel of Figure
fig. 1 via the PDG2016+ hadronic spectrum for ALICE pp collisions at
$\sqrt{s}=7.00$ TeV, pPb collisions at $\sqrt{s_{\rm{NN}}}=5.02$ TeV and PbPb
collisions at $\sqrt{s_{\rm{NN}}}=2.76$ TeV as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$. For both the top and
bottom panels, the corresponding $\chi^{2}$/dof and $\Delta$($\chi^{2}$/dof)
values for pp, pPb and PbPb collision systems are respectively shown as as
closed circles, open diamonds and closed squares.

We extracted freeze-out temperatures $T_{\mathrm{ch}}$ via ‘The FIST’ for the
light, all, and strange particle thermal fits across increasing
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ values for pp, pPb, and
PbPb collision systems at ALICE in the GCE and SCE configurations using
experimental particle yields.

The top panel of Figure fig. 1 shows the extracted freeze-out temperatures
$T_{\mathrm{ch}}$ as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ at $|\eta|<0.5$ for the
three different fits within the GCE treatment. The pp, pPb, PbPb points are
shown as closed circles, open diamonds and closed squares, respectively. The
red, green, and blue points pertain to the extracted $T_{\mathrm{ch}}$ values
for the light, all, and strange fits, respectively. The coloured bands were
determined from the propagation of the error of the fits onto the freeze-out
parameters. The bottom panel of Figure fig. 1 shows the extracted freeze-out
temperatures $T_{\mathrm{ch}}$ as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ at $|\eta|<0.5$ for the
three different fits within the SCE treatment, following the same labeling
convention as in the top panel of the figure. Qualitatively contrasting both
top and bottom panels, we observe a deterioration of the flavour-dependent
2CFO approach occurring in the pp and pPb systems when using the GCE scenario,
starting at values of $\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}<50$.
This is not the case in the SCE treatment, where a flavour-dependent
temperature separation is consistently present as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ across all three systems.
It also is worth noting that in the grand canonical limit, i.e. for values of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}>50$, both the GCE and SCE
treatments render almost identical $T_{\mathrm{ch}}$ values. Nevertheless, in
order to qualitatively differentiate between the two treatments, we performed
an in-depth comparison of the fit quality of the aforementioned
$T_{\mathrm{ch}}$ values as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$, which is separately shown
in Figure fig. 2.

The top panel of Figure fig. 2 depicts the corresponding $\chi^{2}$/dof values
for the GCE and SCE all fits of the top and bottom panels of Figure fig. 1,
shown in cyan and magenta, respectively. The pp, pPb, PbPb points are shown as
closed circles, open diamonds and closed squares. In the large system (PbPb),
we observe a consistent quality of fit for all multiplicity classes in the GCE
and SCE scenarios, with almost identical values for both approaches. However,
in the GCE case, we observe a deteriorating quality of fits in the small
systems (pp and pPb), with $\chi^{2}/\rm{dof}>5$ for nearly all fits to pp
data. This renders an inconsistent description of temperatures across
multiplicity within the GCE treatment of smaller systems. Instead, using the
SCE approach, we observe a consistent quality of fit, specifically,
$\chi^{2}/$dof$\;\lesssim 5$, across all small and large systems with
corresponding values $\big{\langle}dN_{\rm{ch}}/d\eta\big{\rangle}\geq 10$.

The bottom panel of Figure fig. 2 shows the $\Delta$($\chi^{2}$/dof) values
corresponding to the SCE fits performed in the light and strange fits from the
bottom panel of Figure fig. 1 – shown in orange and purple, respectively. As
in the top panel, the pp, pPb, PbPb points are respectively shown as closed
circles, open diamonds and closed squares. The $\Delta$($\chi^{2}$/dof) values
for the orange points in the bottom panel of Figure fig. 2 were calculated as
the difference between the corresponding $\chi^{2}$/dof values of the SCE all
(green) fits and the SCE light (red) fits from the bottom panel in Figure fig.
1. Similarly, the $\Delta$($\chi^{2}$/dof) values for the purple points in the
bottom panel of Figure fig. 2 were calculated as the difference between the
corresponding $\chi^{2}$/dof values of the SCE all (green) fits and the SCE
strange (blue) fits from the bottom panel in Figure fig. 1. This procedure was
done in order to explicitly discriminate between notable differences in the
quality of the fits when employing the 2CFO approach within the SCE, where any
improvements of the fit quality would render $\Delta$($\chi^{2}$/dof) values
approximately equal to the magenta $\chi^{2}$/dof values shown in the top
panel. Conversely, any $\Delta$($\chi^{2}$/dof) values below zero reflect a
worsened fit quality. We note that for all three systems, across all values of
$\big{\langle}dN_{\rm{ch}}/d\eta\big{\rangle}$, the calculated
$\Delta$($\chi^{2}$/dof) values for both the light (orange) and strange
(purple) points generally follow the same trend as the all (cyan) points in
the top panel. Without loss of generality, this suggests an overall
improvement of the fits when employing the flavour-dependent 2CFO approach in
the SCE treatment. The presence of negative $\Delta$($\chi^{2}$/dof) values
can only be seen for a few of the light fits and can be attributed to a
decreasing number of degrees of freedom, which causes an overall increase in
the calculated $\chi^{2}$/dof. Moreover, the final $\chi^{2}$/dof values for
the flavour specific fits can be obtained by subtracting the values in the
lower panel from the magenta points in the upper panel of Figure fig. 2. One
can thus observe a consistent quality of fit – specifically,
$\chi^{2}/$dof$\;\lesssim 5$ – for each of the three types of fits across the
small and large systems with corresponding values of
$\big{\langle}dN_{\rm{ch}}/d\eta\big{\rangle}\geq 10$. We note that the
quality of fit is consistent for both the light and strange fits to a lower
corresponding value of $\big{\langle}dN_{\rm{ch}}/d\eta\big{\rangle}\geq 5$.
Generally, the $\chi^{2}/$dof values for the light and strange fits are lower
than those for the all fit.

Figure 3: 2CFO SCE ‘The FIST’ Thermal Model anti-hadron to $\pi^{+}$ yield
ratio calculations via the PDG2016+ hadronic spectrum for ALICE pp collisions
at $\sqrt{s}=7.00$ TeV, pPb collisions at $\sqrt{s_{\rm{NN}}}=5.02$ TeV, and
PbPb collisions at $\sqrt{s_{\rm{NN}}}=2.76$ TeV as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$. From top to bottom:
$\bar{p}$/$\pi^{+}$, $\bar{\Lambda}$/$\pi^{+}$, $\bar{\Xi}$/$\pi^{+}$, and
$\bar{\Omega}$/$\pi^{+}$, respectively. ALICE experimental points for pp, pPb
and both PbPb collision systems are respectively shown as grey closed circles,
black open diamonds and black closed squares. Purple open crosses depict to
our calculated SCE 2CFO anti-hadron to $\pi^{+}$ ratios at vanishing baryo-
chemical potential in the SCE framework based on the flavour specific
temperatures extracted in [23]. For all calculations, $\mu_{\mathrm{B}}=0$,
$\gamma_{\mathrm{S}}=1$ and $V_{\mathrm{C}}=V$. Figure 4: ‘The FIST’ SCE
Thermal Model temperature dependent volume per unit rapidity calculations via
the PDG2016+ hadronic spectrum for ALICE pp collisions at $\sqrt{s}=7.00$ TeV,
pPb collisions at $\sqrt{s_{\rm{NN}}}=5.02$ TeV, and PbPb collisions at
$\sqrt{s_{\rm{NN}}}=2.76$ TeV as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ based on the flavour
specific temperatures extracted in [23]. From top to bottom: Volume
determinations at 150 MeV, 158 MeV and 165 MeV. Our calculated volumes per
unit rapidity of pp, pPb and both PbPb collision systems are respectively
shown as grey orange circles, orange open diamonds and orange closed squares.
For all calculations, $\mu_{\mathrm{B}}=0$, $\gamma_{\mathrm{S}}=1$ and
$V_{\mathrm{C}}=V$.

Figure fig. 3 shows the anti-hadron to $\pi^{+}$ ratio as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$, starting from the top
panel down to the bottom: $\bar{p}$/$\pi^{+}$, $\bar{\Lambda}$/$\pi^{+}$,
$\bar{\Xi}$/$\pi^{+}$ and $\bar{\Omega}$/$\pi^{+}$, respectively. The
experimental points for ALICE pp, pPb and both PbPb collision systems are
shown as grey closed circles, black open diamonds and black closed squares,
respectively. The purple open crosses indicate our calculated 2CFO anti-hadron
to $\pi^{+}$ ratios within the SCE framework in ‘The FIST’ using the flavour
specific temperatures $T_{\mathrm{L}}=150$ MeV for $\pi^{+}$ and $\bar{p}$,
and $T_{\mathrm{S}}=165$ MeV for $\bar{\Lambda}\bar{\Xi}$ and $\bar{\Omega}$.
The corresponding model calculations of the $\phi$ to $\pi^{+}$ ratio are
shown separately in Figure fig. 5, keeping the same purple cross colour
convention, for the traditional treatment of net-strangeness of the $\phi$
meson of $S=0$. For this study we chose the chemical temperatures extracted in
our previous letter [23] for central (0 - 10%) PbPb collisions at
$\mu_{\mathrm{B}}$ = 0 across all centralities and system sizes. We also
tested that using instead the temperatures shown in the bottom panel of Figure
fig. 1 will not make a difference in final state particle yield ratios since a
simultaneous change in volume from the values shown in Figure fig. 4 will
compensate any temperature differences.

Our results show an excellent agreement with the experimental yield ratios
measured by ALICE across all three systems and are consistent with those shown
by [38, 39]. Early calculations [40] requiring an additional canonical
suppression factor, i.e. $\gamma_{S}<1$, can be excluded based on our studies
and Ref. [39]. At these high collision energies, strangeness seems saturated
even in the smallest systems. The main differences to Ref. [39] emerge from
the fact that our results do not require different rapidity windows for pions
and strange particles and no additional normalization factor was used to
reproduce the experimental results. In our analysis, the accurate
representation of final state particle yields relies solely on the use of
flavour-dependent temperatures and fireball volumes – the relevant volumes as
a function of $\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ as determined
by the model are shown in Figure fig. 4.

Lastly, we varied the treatment of net-strangeness of the $\phi$ meson in our
model between $S=0$, $S=1$ and $S=2$ to gauge the sensitivity of the
calculated yields and to elucidate the question of strangeness enhancement in
the case of the vector meson. Figure fig. 5 shows the $\phi$ to $\pi^{+}$
ratio as a function of $\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$,
following the same labeling convention for experimental points and 2CFO
thermal model calculations as Figure fig. 3, using an ad hoc variation of the
total strangeness (S) value within the model for the $\phi$ meson. To
reiterate, the $\pi^{+}$ and $\phi$ values were calculated at
$T_{\mathrm{L}}=150$ MeV and $T_{\mathrm{S}}=165$ MeV, respectively. For the
comparison to the data, we arbitrarily fixed the total strangeness of the
$\phi$ meson to $S=0$ (purple open crosses), $S=1$ (cyan open crosses) and
$S=2$ (magenta open crosses), respectively.

Figure 5: 2CFO SCE ‘The FIST’ Thermal Model $\phi$ to $\pi^{+}$ yield ratio
calculations via the PDG2016+ hadronic spectrum for ALICE pp collisions at
$\sqrt{s}=7.00$ TeV, pPb collisions at $\sqrt{s_{\rm{NN}}}=5.02$ TeV, and PbPb
collisions at $\sqrt{s_{\rm{NN}}}=2.76$ TeV as a function of $\langle
dN_{\mathrm{ch}}/d\eta\rangle$. ALICE experimental points for pp, pPb, PbPb
collisions are respectively shown in grey as closed circles, open diamonds,
and closed squares. Purple, cyan and magenta open crosses depict our
calculated SCE 2CFO $\phi$ to $\pi^{+}$ ratios in the SCE framework based on
the flavour specific freeze-out temperatures at vanishing baryo-chemical
potential shown in [23] fixing the total strangeness values for the $\phi$
meson to $S=0$, $S=1$ and $S=2$, respectively. For all calculations,
$\mu_{\mathrm{B}}=0$, $\gamma_{\mathrm{S}}=1$ and $V_{\mathrm{C}}=V$.

In the case of $S=0$, the calculated yield ratios are well described by the
2CFO approach only in the large systems, for $\langle
dN_{\mathrm{ch}}/d\eta\rangle$ > 50\. Our model vastly overestimates the value
of the ratio for both the pp and pPb systems. In the case $S=1$, we see an
improvement particularly in the smaller systems, with our calculated values
almost falling within the errors of the experimental data for all multiplicity
bins. Lastly, for the case of $S=2$, the model underestimates the experimental
yield ratios for values of $\langle dN_{\mathrm{ch}}/d\eta\rangle$ < 10, but
does otherwise quite well. Our results suggest that for $\phi$ production in
small systems the $\phi$ should not be considered a $S=0$ particle, because
simple flavor conservation and recombination arguments require more than a
single string to fragment to form an $s\bar{s}$ state. Therefore, $\phi$
yields can be more accurately calculated, within the HRG framework, by
assuming a strangeness content between 1 and 2.

##  5 Conclusion

We presented determinations of freeze-out temperatures $T_{\mathrm{ch}}$ for
the light, full, and strange particle thermal fits across increasing
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ values for pp, pPb, and
PbPb collision systems at ALICE in the GCE and SCE configurations from
experimental particle yields via ‘The FIST’ using the PDG2016+ hadronic
spectrum. Moreover, we also showed thermal model anti-hadron to $\pi^{+}$
yield ratio calculations in these same collision systems as a function
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$, with particular attention
given to the treatment of the total strangeness content of the $\phi$ meson.
In the scope of the Strangeness Canonical Ensemble within the framework of the
Thermal FIST HRG model package, we show an excellent description of
experimental yield ratios across all three systems, measured by the ALICE
Collaboration as a function of
$\big{\langle}dN_{\mathrm{ch}}/d\eta\big{\rangle}$ at LHC energies, when
employing flavour-dependent chemical freeze-out temperatures under the
assumption of fully saturated strangeness. In conclusion, the flavour-
dependent (Tch) separation established in heavy ion collisions, seems to
prevail also at low $\langle dN_{\mathrm{ch}}/d\eta\rangle$ values
corresponding to the pp and pPb systems. This sustained separation may also be
seen as an indication of QGP formation in small systems.

##  6 Acknowledgments

The authors acknowledge fruitful discussions with Volodymyr Vovchenko, Claudia
Ratti, Paolo Parotto, Jamie Stafford, Livio Bianchi, Boris Hippolyte, Horst
Sebastian Scheid and Dhevan Gangadharan. This work was supported by the DOE
grant DEFG02-07ER4152. F.A.F. acknowledges ongoing support from the Franco-
American Fulbright Commission.

## References

  * Stachel  _et al._ [2014] J. Stachel, A. Andronic, P. Braun-Munzinger, and K. Redlich, Journal of Physics: Conference Series 509, 012019 (2014).
  * Bazavov  _et al._ [2014] A. Bazavov _et al._ (HotQCD Collaboration), Phys. Rev. D 90, 094503 (2014).
  * Borsányi  _et al._ [2014] S. Borsányi, Z. Fodor, C. Hoelbling, S. D. Katz, S. Krieg, and K. K. Szabó, Physics Letters B 730, 99 (2014).
  * Borsányi  _et al._ [2010] S. Borsányi, Z. Fodor, C. Hoelbling, S. D. Katz, S. Krieg, C. Ratti, and K. K. Szabó, Journal of High Energy Physics 2010, 73 (2010).
  * Bellwied  _et al._ [2015] R. Bellwied, S. Borsányi, Z. Fodor, J. Günther, S. Katz, C. Ratti, and K. Szabó, Physics Letters B 751, 559 (2015).
  * Cleymans and Redlich [1998] J. Cleymans and K. Redlich, Phys. Rev. Lett. 81, 5284 (1998).
  * Andronic  _et al._ [2018] A. Andronic, P. Braun-Munzinger, K. Redlich, and J. Stachel, Nature 561, 321 (2018).
  * Sharma  _et al._ [2019] N. Sharma, J. Cleymans, B. Hippolyte, and M. Paradza, Phys. Rev. C 99, 044914 (2019).
  * Tounsi and Redlich [2001] A. Tounsi and K. Redlich, (2001), arXiv:hep-ph/0111159 .
  * Tounsi  _et al._ [2003] A. Tounsi, A. Mischke, and K. Redlich, Nuclear Physics A 715, 565c (2003), quark Matter 2002.
  * Sahoo and Biswas [2021] R. Sahoo and D. Biswas, Adv. High Energy Phys. 2021, 6611394 (2021).
  * Bellwied [2018] R. Bellwied, EPJ Web Conf. 171, 02006 (2018).
  * Vovchenko  _et al._ [2019] V. Vovchenko, B. Dönigus, and H. Stoecker, Phys. Rev. C 100, 054906 (2019).
  * Adam  _et al._ [2016a] J. Adam _et al._ , Physics Letters B 758, 389 (2016a).
  * Ratti  _et al._ [2012] C. Ratti, R. Bellwied, M. Cristoforetti, and M. Barbaro, Phys. Rev. D 85, 014004 (2012).
  * Bellwied  _et al._ [2013] R. Bellwied, S. Borsanyi, Z. Fodor, S. D. Katz, and C. Ratti, Phys. Rev. Lett. 111, 202302 (2013).
  * Karsch [2012] F. Karsch, Central European Journal of Physics 10, 1234 (2012).
  * Adamczyk  _et al._ [2014] L. Adamczyk _et al._ (STAR Collaboration), Phys. Rev. Lett. 112, 032302 (2014).
  * Bellwied  _et al._ [2019] R. Bellwied, J. Noronha-Hostler, P. Parotto, I. Portillo Vazquez, C. Ratti, and J. M. Stafford, Phys. Rev. C 99, 034912 (2019).
  * Bluhm and Nahrgang [2019] M. Bluhm and M. Nahrgang, Eur. Phys. J. C. 79, 155 (2019).
  * Adam  _et al._ [2020] J. Adam _et al._ (STAR), Phys. Rev. C 102, 024903 (2020), arXiv:2001.06419 [nucl-ex] .
  * Bellwied  _et al._ [2020] R. Bellwied, S. Borsányi, Z. Fodor, J. N. Guenther, J. Noronha-Hostler, P. Parotto, A. Pásztor, C. Ratti, and J. M. Stafford, Phys. Rev. D 101, 034506 (2020).
  * Flor  _et al._ [2021] F. A. Flor, G. Olinger, and R. Bellwied, Physics Letters B 814, 136098 (2021).
  * Vovchenko and Stoecker [2019] V. Vovchenko and H. Stoecker, Computer Physics Communications 244, 295 (2019).
  * Alba  _et al._ [2017] P. Alba, R. Bellwied, S. Borsányi, Z. Fodor, J. Günther, S. D. Katz, V. Mantovani Sarti, J. Noronha-Hostler, P. Parotto, A. Pasztor, I. P. Vazquez, and C. Ratti, Phys. Rev. D 96, 034517 (2017).
  * Patrignani  _et al._ [2016] C. Patrignani _et al._ (Particle Data Group), Chin. Phys. C40, 100001 (2016).
  * Adam  _et al._ [2017] J. Adam _et al._ (ALICE Collaboration), Nature Phys 13, 535 (2017).
  * Abelev  _et al._ [2014a] B. Abelev _et al._ , Phys. Lett. B 728, 25 (2014a).
  * Adam  _et al._ [2016b] J. Adam _et al._ , Eur. Phys. J 76, 245 (2016b).
  * Adam  _et al._ [2016c] J. Adam _et al._ , Physics Letters B 758, 389 (2016c).
  * Abelev  _et al._ [2013a] B. Abelev _et al._ (ALICE Collaboration), Phys. Rev. C 88, 044910 (2013a).
  * Abelev  _et al._ [2013b] B. Abelev _et al._ (ALICE Collaboration), Phys. Rev. Lett. 111, 222301 (2013b).
  * Abelev  _et al._ [2015] B. Abelev _et al._ (ALICE Collaboration), Phys. Rev. C 91, 024609 (2015).
  * Abelev  _et al._ [2014b] B. Abelev _et al._ (ALICE Collaboration), Physics Letters B 728, 216 (2014b).
  * Bhattacharyya  _et al._ [2019] S. Bhattacharyya, D. Biswas, S. K. Ghosh, R. Ray, and P. Singha, Phys. Rev. D 100, 054037 (2019).
  * Bhattacharyya  _et al._ [2020] S. Bhattacharyya, D. Biswas, S. K. Ghosh, R. Ray, and P. Singha, Phys. Rev. D 101, 054002 (2020).
  * Magestro [2002] D. Magestro, Journal of Physics G: Nuclear and Particle Physics 28, 1745 (2002).
  * Acharya  _et al._ [2019] S. Acharya _et al._ (ALICE Collaboration), Phys. Rev. C 99, 024906 (2019).
  * Vislavicius and Kalweit [2019] V. Vislavicius and A. Kalweit, (2019), arXiv:1610.03001 [nucl-ex] .
  * Chatterjee  _et al._ [2017] S. Chatterjee, A. K. Dash, and B. Mohanty, J. Phys. G 44, 105106 (2017).


# Spotlight Attention: Robust Object-Centric Learning With a Spatial Locality
Prior

Ayush Chakravarthy
Mila, Université de Montréal
<EMAIL_ADDRESS>
Trang Nguyen
Mila, Université de Montréal
Anirudh Goyal
Google DeepMind
Yoshua Bengio
Mila, Université de Montréal
Michael C. Mozer
Google Research, Brain Team

###### Abstract

The aim of object-centric vision is to construct an explicit representation of
the objects in a scene. This representation is obtained via a set of
interchangeable modules called _slots_ or _object files_ that compete for
local patches of an image. The competition has a weak inductive bias to
preserve spatial continuity; consequently, one slot may claim patches
scattered diffusely throughout the image. In contrast, the inductive bias of
human vision is strong, to the degree that attention has classically been
described with a spotlight metaphor. We incorporate a spatial-locality prior
into state-of-the-art object-centric vision models and obtain significant
improvements in segmenting objects in both synthetic and real-world datasets.
Similar to human visual attention, the combination of image content and
spatial constraints yield robust unsupervised object-centric learning,
including less sensitivity to model hyperparameters.

##  1 Introduction

Learning about objects and their interactions is a cornerstone of human
cognition (Spelke and Kinzler, 2007). Understanding the nature of objects and
their properties is necessary to achieve symbol-like mental representations
(Whitehead, 1928) and systematicity of reasoning (Fodor and Pylyshyn, 1988).
Although language has a natural tokenization that supports systematicity
(Chakravarthy et al., 2022), progress in visual reasoning in Artificial
Intelligence hinges on tokenizing visual input. Visual tokenization
corresponds to the problem of object-centric representation learning—learning
to partition images into a set of discrete slots in an unsupervised manner.
The desiderata for these slots are that they induce a bijection with the
objects in an image and are interchangeable.

With no or limited supervision, object-centric representation learning is very
difficult (Greff et al., 2020) and requires appropriate forms of inductive
bias (Schölkopf et al., 2021; Ke et al., 2021; Goyal and Bengio, 2022). The
biases explored for building models with object-centric representation have
been based on instance based segmentation (Greff et al., 2017, 2019a;
Locatello et al., 2020), sequential object extraction (Gregor et al., 2015;
Burgess et al., 2019; Engelcke et al., 2021; Goyal et al., 2021a), invariance
(Crawford and Pineau, 2019; Lin et al., 2020; Jiang et al., 2020; Biza et al.,
2023), type-token distinction (Bao et al., 2022, 2023; Goyal et al., 2020) and
the sparsity of interactions among different slots (Goyal et al., 2021b, c).

However, none these methods leverage a bias that is considered fundamental to
visual attention in the psychological literature: the preference for spatial
continuity of an attended region. Traditionally, attention was considered to
be a _spotlight_ on a region of interest in the image (Posner, 1980). The
spotlight metaphor was extended to be a zoom lens (LaBerge, 1983), allowing
for large or small regions, but nonetheless the regions needed to be convex.
The metaphor was further extended to allow the selected region to blanket
around a shape (Mozer, 1988). In the psychological literature, even the notion
of _object-based attention_ (Duncan, 1984) is spatial in nature (Vecera and
Farah, 1994), although it allows noncontiguous features of an object to be
selected together e.g., if two ends of an occluded object are visible (Zemel
et al., 2002).

Although current techniques in object-centric representation learning make
available a positional encoding to guide the mapping between image patches and
slots, the slot extraction process itself has no explicit pressure to choose
patches in a spatially contiguous manner. Consequently, one slot may claim
patches scattered diffusely throughout the image. In this paper, we introduce
an inductive bias in the form of a _spatial locality prior_ (_SLP_) that
encourages slots to select spatially contiguous patches in the input image.

Summary. We present an algorithm that can be incorporated into models for
object-centric representation learning that biases slot assignments based on
spatial locality. We show consistent improvements in the quality of object
representations for three object-centric architectures, eight distinct data
sets, both synthetic and natural, and multiple different performance measures
that have been used in the literature. We show that the SLP also makes the
baseline models more robust to hyperparameter selection and supports robust
out-of-distribution generalization.

##  2 Background

Unsupervised Object-Centric Learning. Our work falls into the line of research
in machine vision known as unsupervised object-centric representation learning
(Eslami et al., 2016; Greff et al., 2017, 2019a; Burgess et al., 2019; Goyal
et al., 2020; Lin et al., 2020; Locatello et al., 2020; Engelcke et al., 2021;
Singh et al., 2022a; Jia et al., 2023). Broadly, this work has the objective
of mapping image elements to a low-dimensional set of objects, or _slots_ ,
where the goal is for grouped image elements to be semantically similar. As
‘semantic similarity’ is an ambiguous objective, several additional sources of
information have been explored including video sequences (Jiang et al., 2020;
Weis et al., 2021; Singh et al., 2022b; Traub et al., 2023) and optical flow
(Kipf et al., 2022; Elsayed et al., 2022; Bao et al., 2022, 2023). However, as
pointed out by Yang and Yang (2022), such methods suffer from problems in
scaling to large real-world datasets. Toward alleviating this problem, recent
work has moved from CNNs to Transformer-based models which have greater
expressivity (Singh et al., 2022a; Seitzer et al., 2023). Nonetheless, the
basic mechanism of slot assignment in Slot Attention (Locatello et al., 2020)
and its variants (Chang et al., 2022; Jia et al., 2023) has proven a critical
building block to unsupervised object-centric learning.

Spatially-biased OCL. The key novelty of our work is a spatial-proximity-based
mechanism that modulates the slot-pixel assignment. One might argue that
because current methods for object centric representation learning includes
explicit positional encodings in its input (Locatello et al., 2020; Goyal et
al., 2020, 2021b), the existing methods have sufficient information to
discover the principle of spatial coherence. However, the results we present
clearly indicate otherwise, or at least that spatial constraints have yet to
be fully exploited. Other work has incorporated spatial information into
models, such as using shape priors for weak supervision (Elich et al., 2020),
spatial position as a means to bootstrap learning (Kim et al., 2023), and
incorporating spatial symmetries (Biza et al., 2023). Furthermore, in the
context of video, SaVi (Kipf et al., 2022) and SaVi++ (Elsayed et al., 2022)
use ground-truth spatial information such as center of mass and bounding boxes
extracted from the first frame of the video. However, our method is the first
to use spatial constraints to steer the slot-pixel assignment with no
supervision.

##  3 Method

###  3.1 Augmenting Key-Query Match with Spatial Bias

The input image is first pre-processed by an _image encoder_. This processing
preserves the image topography, yielding an embedding at each image patch $p$,
which corresponds to an $(x,y)$ position in a coarse grid over the image. (The
embedding is intended to encode high-level visual features, and thus the grid
of patches is coarser than the input dimensions in pixels.) The embedding at
each patch $p$ is mapped to a key, $\bm{\kappa}_{p}$, which is matched to a
query from each slot $k$, denoted $\bm{q}_{k}$. The evidence supporting a key-
query match is

$\gamma_{kp}=\frac{\bm{q}_{k}^{\mathrm{T}}\bm{\kappa}_{p}}{\sqrt{d}},$

where $d$ is the dimensionality of the query and key vectors.

In various different methods of object centric representation learning (Goyal
et al., 2020; Locatello et al., 2020; Goyal et al., 2021b), slots compete for
each grid position via a softmax renormalization of the match scores
$\gamma_{kp}$. Here, we introduce a _Spatial Locality Prior_ to modulate the
competition among slots. This prior takes the form of an additive term,
$\alpha_{kp}$, in the softmax:

$s_{kp}=\mathrm{softmax}_{k}\big{(}\gamma_{kp}+\alpha_{kp}\big{)},$

where $s_{kp}$ is the affinity between position $p$ and slot $k$, and
$\mathrm{softmax}_{k}$ denotes the $k$’th element of the softmax vector. We
use $\bm{s}_{k}\equiv\\{s_{k}\\}$ to denote the distribution of activation (or
attention) over positions for a given slot $k$. The $\bm{\alpha}$ matrix is
used to bias this distribution by spatial locality.

###  3.2 Encouraging Spatial Locality

Conditioned on an input, the $\bm{\alpha}$ matrix is determined by a
constraint satisfaction process (CSP) that encourages a roughly spotlight-like
distribution of activation over positions in $\bm{s}_{k}$ for each slot $k$.
Additionally, the CSP discourages overlap in the spotlights of any pair of
slots. The CSP converges on $\bm{\alpha}$ by iterative gradient-descent steps
in a loss that penalizes spotlights that are non-compact and overlapping.

The spotlight associated with each slot $k$ is characterized by its center,
$\bm{m}_{k}$, and isotropic spread, $v_{k}$, defined to be the central
tendency and variance of $\bm{s}_{k}$:

$\bm{m}_{k}=\frac{\sum_{p}s_{kp}\bm{p}}{\sum_{p}s_{kp}}\text{~{}~{}~{}~{}and~{}~{}~{}~{}}v_{k}=\frac{\sum_{p}s_{kp}|\bm{p}-\bm{m}_{k}|^{2}}{\sum_{p}s_{kp}}$
(1)

Note that while $\sum_{k}s_{kp}=1$ due to the softmax, the sum over positions
is not normalized.

The CSP’s loss consists of two terms. First, a penalty is imposed to the
degree that each pair of slots fail to have spatially distinct attentional
profiles, as characterized by a distance measure summed over slot pairs:

$\mathcal{L}_{\mathrm{distinct}}=\sum_{k,k^{\prime}>k}\exp\left(-\frac{|\bm{m}_{k}-\bm{m}_{k^{\prime}}|^{2}}{v_{k}+v_{k^{\prime}}}\right)$
(2)

This loss is designed to push apart the slot means (the numerator term)
relative to the intra-slot variance (the denominator term). Second, to prevent
degenerate solutions in which attention collapses to a point, we impose a
penalty on the Froebenius norm of $\bm{\alpha}$:

$\mathcal{L}_{\mathrm{norm}}=\sum_{k,p}\alpha_{kp}^{2}$ (3)

The overall loss
$\mathcal{L}=\mathcal{L}_{\mathrm{distinct}}+\lambda\mathcal{L}_{\mathrm{norm}}$
is minimized with respect to $\bm{\alpha}$ from an initial state
$\bm{\alpha}^{0}$, which we discuss next.

Algorithm 1 Spatial Locality Prior. The algorithm takes image embedding
features $Z\in\mathbb{R}^{N\times C}$; the number of slots, $K$; the number of
slot-update iterations, $T_{\textit{slot}}$; the number of spatial-update
iterations, $T_{\textit{spat}}$; and the projection dimensionality, $d$. The
learned model parameters are: the learned projections $q,k,v$ each with
dimensionality $d$, the alpha initialization
$\bm{\alpha}^{0}\in\mathbb{R}^{K\times N}$; the GRU and the MLP layers; and a
Gaussian mean and variance $\mu,\sigma\in\mathbb{R}^{d}$.

$S\sim\mathcal{N}(\mu,\sigma)$

$Z=\textrm{{LayerNorm}}(Z)$

$\bm{\alpha}=\bm{\alpha}^{0}\ /\ ||\bm{\alpha}^{0}||_{2}$

for $i=1\dots T_{\textit{slot}}$ do

$S=\textrm{{LayerNorm}}(S)$

$L=\frac{1}{\sqrt{d}}q(S)\cdot k(Z)^{\textsc{T}}$

for $j=1\dots T_{\textit{spat}}$ do

if $j=T_{\textit{spat}}-1$ then

$\bm{\alpha}=\textrm{{StopGradient}}(\bm{\alpha})+\bm{\alpha}^{0}-\textrm{{StopGradient}}(\bm{\alpha}^{0})$

end if

$A=\textrm{{Softmax}}(L+\bm{\alpha},\textrm{axis}=\textrm{"slots"})$

$m,v=\textrm{{ComputeDistribution}}(Z,A)$ $\triangleright$ Compute Equation 1

$l=\textrm{{GetLoss}}(\bm{\alpha},m,v)$ $\triangleright$ Compute Equations 2
and 3

$\bm{\alpha}=\bm{\alpha}-\bm{\alpha}_{\textrm{lr}}\cdot\frac{\partial
l}{\partial\bm{\alpha}}$

end for

$A=\textrm{{Softmax}}(L+\bm{\alpha},\textrm{axis}=\textrm{"slots"})$

$A=A\ /\ \textrm{{sum}}(A,\textrm{axis}=\textrm{"embeddings"})$

$U=A\cdot v(Z)$

for $n=1\dots N$ in parallel do

$S_{n}=\textrm{{GRU}}(S_{n},U_{n})$

$S_{n}\mathrel{+}=\textrm{{MLP}}(\textrm{{LayerNorm}}(S_{n}))$

end for

end for

return $S$

###  3.3 Learning Initial State Through Bilevel Optimization

To break symmetry, it is vital to learn an initial state $\bm{\alpha}^{0}$
which partitions the image by dispersing initial slot means across the image.
We take inspiration from Jia et al. (2023) and perform meta-learning to
determine $\bm{\alpha}^{0}$. On each training trial, after $j$ steps of the
CSP optimization, we obtain an approximately optimal attentional bias, let’s
call it $\bm{\alpha}^{*}$. We detach $\bm{\alpha}^{*}$ and optimize for
$\bm{\alpha}^{0}$ on the last CSP step. We use the straight-through estimator
(Bengio et al., 2013; van den Oord et al., 2017) to additionally propagate
gradients into $\bm{\alpha}^{0}$. Through this design, we are able to learn
generalized dataset-wide statistics about
$\operatorname{\mathbb{E}}[\bm{m}_{k}]$ and $\operatorname{\mathbb{E}}[v_{k}]$
for each slot $k$. Note that this procedure breaks slot symmetry because
$\bm{\alpha}^{0}$ assigns slots to default regions of the image.

##  4 Experiments

In this section, we evaluate the benefit of incorporating the spatial-locality
prior into Slot Attention, as well as into two recent object-centric methods
that build upon Slot Attention to achieve state-of-the-art performance, BO-QSA
(Jia et al., 2023) and DINOSAUR (Seitzer et al., 2023). We find that for all
three models, across eight diverse datasets, the spatial-locality prior boosts
performance. We refer to the base models (Slot Attention, BO-QSA, and
DINOSAUR) augmented with the spatial locality prior (hereafter, _SLP_) by
adding the modifier ‘+SLP’ to the name. All comparisons we report use a given
base model with and without SLP for highly controlled experimentation.

Here we include details about our datasets, architectural decisions, and
evaluation methods; but hyperparameters and other simulation details are
presented in the Supplementary Materials. All experiments were run on a single
50GB Quatro RTX 8000 GPU.

Table 1: Foreground ARI (%) Segmentation Accuracy
(mean ± 1 SEM across 3 replications of each simulation)

(a) Results for Vanilla Slot Attention augmented with SLP

Method | ObjectsRoom | MultidSprites | ShapeStacks | ClevrTex
---|---|---|---|---
MONet (Burgess et al., 2019) | 0.54  ± 0.05 | 0.89  ± 0.05 | 0.70  ± 0.11 | 0.19  ± 0.05
GENESIS-V2 (Engelcke et al., 2021) | 0.86  ± 0.05 | 0.52  ± 0.15 | 0.81  ± 0.05 | 0.31  ± 0.20
Slot Attention (Locatello et al., 2020) | 0.86  ± 0.14 | 0.91  ± 0.10 | 0.80  ± 0.08 | 0.62  ± 0.08
Slot Attention + SLP | 0.87  ± 0.05 | 0.94  ± 0.05 | 0.83  ± 0.05 | 0.71  ± 0.05

(b) Varying hyperparameters (number of slot-update iterations and
spatial-update iterations) for ClevrTex

Method | 3 slot iterations | 5 slot iterations | 10 slot iterations
---|---|---|---
Slot Attention (Locatello et al., 2020) | 0.45  ± 0.23 | 0.35  ± 0.20 | 0.34  ± 0.16
Slot Attention + SLP (1 spatial iteration) | 0.54  ± 0.05 | 0.61  ± 0.05 | 0.60  ± 0.14
Slot Attention + SLP (5 spatial iterations) | 0.64  ± 0.10 | 0.60  ± 0.08 | 0.63  ± 0.11
Slot Attention + SLP (10 spatial iterations) | 0.65  ± 0.08 | 0.63  ± 0.08 | 0.62  ± 0.05

(a) Clevr6 examples: Visualizations for Image Reconstructions, Slot
Extractions, and Alphas.

(b) Visualization of meta-learned $\bm{\alpha}^{0}$ on Clevr6

Figure 1: Results from Slot Attention + SLP

###  4.1 Object Discovery in Synthetic Images

For synthetic datasets, we focus on the task of Object Discovery (Burgess et
al., 2019), which is to produce a set of masks that cover each of the objects
that appear in the image. We first isolate the effect of SLP through
experimenting with vanilla Slot Attention (Locatello et al., 2020) on CLEVR6
(Johnson et al., 2016), ObjectsRoom (Kabra et al., 2019), MultidSprites
(Burgess et al., 2019), ShapeStacks (Groth et al., 2018), and ClevrTex
(Karazija et al., 2021). To show that SLP works for other variants of Slot
Attention, we examine BO-QSA (Jia et al., 2023) on ShapeStacks, ObjectsRoom,
and ClevrTex datasets. We primarily focus on Foreground-ARI (FG-ARI) (Hubert
and Arabie, 1985) as our dependent measure of performance. We then show
results with DINOSAUR (Seitzer et al., 2023) on the MoVi-C and MoVi-E datasets
(Greff et al., 2022), where we evaluate using FG-ARI and mean-best-overlap
(mBO) (Pont-Tuset et al., 2015).

####  4.1.1 Vanilla Slot Attention

Table 2: FG-ARI (%) Accuracy
(mean ± 1 SEM across 3 runs)
Varying number of slots for ClevrTex Model | 7 Slots | 11 Slots
---|---|---
Slot Attention | 0.45  ± 0.23 | 0.62  ± 0.08
Slot Attention + SLP | 0.65  ± 0.08 | 0.71  ± 0.05

Methodology. In order to isolate the effect of SLP, we setup Slot Attention as
described in Locatello et al. (2020), with a Mixture Decoder from Watters et
al. (2019) and a 4-layer CNN encoder, on the object-discovery task. SLP is
integrated into Slot Attention as described in Algorithm 1. Slots are
initialized with a learned Gaussian mean and variance.

Results. Table 1(a) presents results for object discovery on synthetic
datasets. Slot Attention + SLP yields improvements on MultidSprites,
ShapeStacks, and, most notably, on ClevrTex. We obtain an almost 10%
improvement on ClevrTex; given challenging views with diverse textures and
complicated lighting effects, Slot Attention itself is not able to effectively
segment the objects. SLP does not completely solve the task, but it is a
significant step forward. Further, in Table 2, we show that SLP allows an
underparametrized Slot Attention with 7 slots to match the 11 slot baseline.
Additionally, the improvement on the 7 slot experiment is robust as a two-
tailed t-test yielded $t(3)=3.61,p=.02$.

Another limitation of vanilla Slot Attention is its fragility as the number of
slot-update iterations across training and evaluation phase. In Table 1(b), we
show that SLP effectively solves this problem. In this Table, we manipulate
both the number of slot-update iterations ($T_{\textit{slot}}$) and the number
of spatial-update iterations ($T_{\textit{spat}}$)—the columns and rows of the
table, respectively. As the number of slot-update iterations increases,
performance of Slot Attention drops but Slot Attention + SLP is not
systematically affected. The Table also indicates that a single SLP
iteration—minimal additional computation—provides a sufficient bias to improve
FG-ARI.

The first and second columns of Figure 1(a) show three sample Clevr6 images
and their reconstruction, respectively. To the right of each image pair is the
$\bm{\alpha}_{k}$ spatial-attention biases for each of the $k\in\\{1...7\\}$
slots (the dark images), and the the post-competition masked slot
representations for each slot (color images). Note that the spatial-attention
biases do not reflect the final selection, but rather the fact that SLP is
encouraging slots to claim patches near already claimed patches. For some
objects, especially in cases where the object seems to be distant, the spatial
bias seems important, but not for all objects in these simple images.

Figure 1(b) visualizes the meta-learned initial spatial bias distribution,
$\bm{\alpha}^{0}$, for the 7 slots. The initialization essentially carves up
the image, ignoring regions of the image that never contain objects. Note that
these initial biases do not determine the final slot assignments, as one can
see by inspection of the spatial distribution of objects claimed by a given
slot in Figure 1(a).

Figure 2 presents ClevrTex examples comparing Slot Attention and Slot
Attention (SA) + SLP via image reconstructions and slot extractions. SA +
SLP’s image reconstruction is far more accurate as compared to that of Slot
Attention, with the clearest difference being the quality of the background
scene and the overall image sharpness. This can be explained by the
observation that empty slots of SA + SLP do a much better job of representing
the background pattern. We also note that the slot decompositions are
effectively localized with SA + SLP, even though the precise boundaries of the
object are not sharp.

Figure 2: ClevrTex examples: Visualizing Image Reconstructions and Slot
Extractions for Slot Attention (SA) and Slot Attention + SLP (SA + SLP)

####  4.1.2 BO-QSA

Methodology. Here, the setup is identical to that in Section 4.1.1, the only
difference being that the slots are set to learnable queries which are
themselves learned through gradient updates via the straight-through estimator
(Bengio et al., 2013).

Results. Table 3 presents FG-ARI scores for BO-QSA. Consistent with our
previous results (4.1.1), we observe the largest improvements on ClevrTex,
while also observing statistically reliable improvements on ShapeStacks and
ObjectsRoom.

Table 3: Foreground ARI (%) Segmentation Accuracy
(mean ± 1 SEM across 3 replications of each simulation)
BO-QSA (Jia et al., 2023) augmented with SLP Method | ShapeStacks | ObjectsRoom | ClevrTex
---|---|---|---
MONet (Burgess et al., 2019) | 0.70  ± 0.11 | 0.54  ± 0.05 | 0.19  ± 0.05
GENESIS-V2 (Engelcke et al., 2021) | 0.81  ± 0.05 | 0.86  ± 0.05 | 0.31  ± 0.20
SLATE (Singh et al., 2022a) | 0.65  ± 0.10 | 0.57  ± 0.10 | 0.73  ± 0.05
I-SA (Chang et al., 2022) | 0.90  ± 0.08 | 0.85  ± 0.05 | 0.78  ± 0.10
BO-QSA (Jia et al., 2023) | 0.93  ± 0.05 | 0.87  ± 0.05 | 0.80  ± 0.08
BO-QSA + SLP | 0.95  ± 0.08 | 0.93  ± 0.05 | 0.87  ± 0.05

####  4.1.3 DINOSAUR

Table 4: FG-ARI and mBO measures of object discovery
on MoVi-C and MoVi-E with DINOSAUR (Seitzer et al., 2023) (mean ± 1 SEM across
3 runs)
Method | MoVi-C | MoVi-E
---|---|---
| FG-ARI | mBO | FG-ARI | mBO
DINOSAUR (ViT-B/8) | 68.9  ± 0.3 | 38.0  ± 0.2 | 65.1  ± 0.6 | 33.5  ± 0.1
DINOSAUR + SLP | 72.8  ± 0.6 | 41.5  ± 0.3 | 70.4  ± 0.4 | 35.9  ± 0.2

Methodology. We closely follow the setup of DINOSAUR (Seitzer et al., 2023),
using a frozen pre-trained ViT encoder (Kolesnikov et al., 2021), Slot
Attention with $11$ slots for MoVi-C and $24$ slots for MoVi-E, a Transformer
Decoder similar to SLATE (Singh et al., 2022a) and feature reconstruction loss
as the primary learning signal instead of image reconstruction loss.

Results. Table 4 compares measures of object discovery for MoVi-C and MoVi-E
datasets on DINOSAUR and our augmented variant with SLP. On both data sets and
two performance measures—FG-ARI and mBO—DINOSAUR + SLP reliably outperforms
DINOSAUR.

Table 5: Real-World Image Experiments

(a) Unsupervised foreground extraction performance on BO-QSA (Jia et al.,
2023) with and without SLP. Results from earlier baseline models are based on
the _best_ replication, whereas the our results are _mean_ ± 1 SEM performance
across three replications per simulation.

Method | CUB | Stanford Dogs | Stanford Cars
---|---|---|---
| IoU | Dice | IoU | Dice | IoU | Dice
ReDO (Chen et al., 2019) | 0.46 | 0.60 | 0.55 | 0.70 | 0.52 | 0.68
IODINE (Greff et al., 2019b) | 0.30 | 0.44 | 0.54 | 0.67 | 0.51 | 0.67
OneGAN (Benny and Wolf, 2019) | 0.55 | 0.69 | 0.71 | 0.81 | 0.71 | 0.82
SLATE (Singh et al., 2022a) | 0.36 | 0.51 | 0.62 | 0.76 | 0.75 | 0.85
I-SA (Chang et al., 2022) | 0.63 | 0.72 | 0.80 | 0.89 | 0.85 | 0.92
BO-QSA (Jia et al., 2023) | 0.61  ± 0.12 | 0.74  ± 0.12 | 0.78  ± 0.12 | 0.68  ± 0.04 | 0.76  ± 0.05 | 0.86  ± 0.05
BO-QSA + SLP | 0.68  ± 0.02 | 0.80  ± 0.02 | 0.78  ± 0.12 | 0.87  ± 0.12 | 0.82  ± 0.08 | 0.91  ± 0.05

(b) Unsupervised multi-object segmentation using BO-QSA with and without SLP,
following Yang and Yang (2022) (mean ± 1 SEM across 3 replications of each
simulation)

Method | COCO | ScanNet
---|---|---
| AP@05 | Precision | Recall | PQ | AP@05 | Precision | Recall | PQ
BO-QSA | 0.093  ± 0.152 | 0.169  ± 0.194 | 0.208  ± 0.210 | 0.114  ± 0.159 | 0.234  ± 0.057 | 0.339  ± 0.054 | 0.387  ± 0.060 | 0.234  ± 0.051
BO-QSA + SLP | 0.126  ± 0.115 | 0.206  ± 0.129 | 0.251  ± 0.142 | 0.139  ± 0.100 | 0.261  ± 0.060 | 0.349  ± 0.048 | 0.399  ± 0.051 | 0.242  ± 0.040

Figure 3: Slot assignments for BO-QSA + SLP on COCO and ScanNet.

###  4.2 Object Discovery in Real-World Images

For real-world data, following Jia et al. (2023) we use two tasks to evaluate
BO-QSA + SLP: unsupervised foreground extraction and unsupervised multi-object
segmentation. For unsupervised foreground extraction, we experiment on the CUB
(Wah et al., 2011), Stanford Dogs (Khosla et al., 2012), and Stanford Cars
(Krause et al., 2013) datasets and evaluate using Intersection-over-Union
(IoU) and Dice evaluation metrics. For unsupervised multi-object segmentation,
we experiment on COCO (Lin et al., 2014) and ScanNet (Dai et al., 2017)
datasets and evaluate using the metrics followed in Yang and Yang (2022)

Methodology. As our base model, we use BO-QSA (Jia et al., 2023) with the
SLATE encoder-decoder setup (Singh et al., 2022a), which consists of a
$4$-layer CNN encoder and a Transformer decoder in a dVAE setup. The slots are
initialized as learned embeddings and the initializations are optimized
directly as in Jia et al. (2023). During evaluation, we select predicted
foreground as the one with the maximum intersection between slot’s mask
prediction and ground-truth foreground mask.

Results. Table 5(a) presents results for unsupervised foreground extraction
and Table 5(b) presents results for unsupervised multi-object segmentation. On
all data sets and on both tasks, SLP consistently improves the performance of
BO-QSA, the most significant improvement being on the Dice metric for the
Stanford Dogs dataset. It appears that I-SA (Chang et al., 2022) marginally
outperforms BO-QSA + SLP on Stanford Dogs and Stanford Cars. _However_ , the
baselines reported in Table 5(a)—including I-SA results—are the best
performing replication of multiple runs, whereas we report mean performance
across three replications for BO-QSA and BO-QSA + SLP. Figure 3 presents
visualizations of slot assignments for several COCO images (first and second
rows) and ScanNet images (third and fourth rows).

Table 6: FG-ARI (%) and Mean-Squared Error (MSE)
BO-QSA Jia et al. (2023) with and without SLP
(mean ± 1 SEM across 3 experiment trials) Method | ClevrTex-OOD
---|---
| FG-ARI | MSE
MONet (Burgess et al., 2019) | 0.37  ± 0.05 | 409  ± 0.5
GENESIS-V2 (Engelcke et al., 2021) | 0.29  ± 0.19 | 539  ± 7.0
Slot-Attention (Locatello et al., 2020) | 0.58  ± 0.05 | 487  ± 2.3
I-SA (Chang et al., 2022) | 0.83  ± 0.05 | 241  ± 1.1
BO-QSA (Jia et al., 2023) | 0.86  ± 0.05 | 265  ± 2.9
BO-QSA + SLP | 0.88  ± 0.05 | 243  ± 1.6

###  4.3 Out-of-Distribution Generalization

One potential weakness of our model is the explicit learning of
$\bm{\alpha}^{0}$ based on training-set-wide statistics of the likely spatial
locations of objects. This initialization may result in poor OOD
generalization. However, we observe that this issue does not adversely affect
our model. As suggested by Figure 1(b), learning $\bm{\alpha}^{0}$ does help
break symmetry: the learned $\bm{\alpha}$ disperses the slot means across the
scene while maintaining a high slot variance. As a result, the attention
distribution forces the slots to model different objects in the scene.
However, since the learned initialization is normalized before further
optimization, the effects of an excessively large $\bm{\alpha}^{0}$
initialization should be effectively nullified.

As evidence in support of our conjecture, we tested Slot Attention with SLP
and initialized slots as learned embeddings as in Section 4.1.2. We evaluate
object discovery on the ClevrTex-OOD split variants and present results in
Table 6. Empirically, as we see improvements both in FG-ARI and in MSE
Reconstruction loss, we are able to prove that learning dataset wide
statistics merely has the effect of learning an $\bm{\alpha}^{0}$
initialization that supports symmetry breaking between slots.

##  5 Discussion

In this paper, we proposed a spatial-locality prior that is consistent with
both human visual attention and statistics of objects in natural images.
Incorporating this prior into unsupervised object-centric models biases slot
decompositions of the visual scene when direct evidence for objects and their
boundaries is weak. The result is improved models that are more robust to
hyperparameter selection and that yield better object segmentations. We show
consistent improvements with three object-centric architectures (Slot
Attention, BO-QSA, and DINOSAUR), eight distinct data sets, and various
performance measures that have been used in the literature, including FG-ARI,
mBO, IoU, and Dice. In all cases, models incorporting SLP advance state-of-
the-art performance.

Limitations and Future Work. A key limitation of the proposed method—as well
as of any slot-based object-centric model, is the hard requirement to specify
the maximum number of slots that the model can represent. Another limitation
of SLP in particular is the increased computational complexity due to the bi-
level optimization algorithm. Fortunately, we have found that a single spatial
iteration, which is relatively efficient, yields significant benefits. In
future we hope to extend the proposed method from static images to video
streams. SLP should be even more fruitful for video streams where the spatial
modulations, $\bm{\alpha}$, inferred for one frame should be a suitable
initialization point for the next frame. We also hope to extend the method to
include the depth dimension of spatial attention, allowing SLP to operate in
depth and to thereby predict occlusions.

##  6 Acknowledgement

This research was enabled in part by compute resources provided by Mila
(mila.quebec). We would like to thank Vedant Shah and Aniket Didolkar for
reviewing early versions of the manuscript. We would also like to thank Mihir
Prabhudesai and Katerina Fragkiadaki for useful discussions.

## References

  * Bao et al. [2022] Zhipeng Bao, Pavel Tokmakov, Allan Jabri, Yu-Xiong Wang, Adrien Gaidon, and Martial Hebert.  Discorying object that can move.  In _CVPR_ , 2022.
  * Bao et al. [2023] Zhipeng Bao, Pavel Tokmakov, Yu-Xiong Wang, Adrien Gaidon, and Martial Hebert.  Object discovery from motion-guided tokens.  In _CVPR_ , 2023.
  * Bengio et al. [2013] Yoshua Bengio, Nicholas Léonard, and Aaron C. Courville.  Estimating or propagating gradients through stochastic neurons for conditional computation.  _ArXiv_ , abs/1308.3432, 2013.
  * Benny and Wolf [2019] Yaniv Benny and Lior Wolf.  Onegan: Simultaneous unsupervised learning of conditional image generation, foreground segmentation, and fine-grained clustering.  _ArXiv_ , abs/1912.13471, 2019.
  * Biza et al. [2023] Ondrej Biza, Sjoerd van Steenkiste, Mehdi S. M. Sajjadi, Gamaleldin F. Elsayed, Aravindh Mahendran, and Thomas Kipf.  Invariant slot attention: Object discovery with slot-centric reference frames.  _ArXiv_ , abs/2302.04973, 2023.
  * Burgess et al. [2019] Christopher P. Burgess, Loic Matthey, Nicholas Watters, Rishabh Kabra, Irina Higgins, Matt Botvinick, and Alexander Lerchner.  Monet: Unsupervised scene decomposition and representation, 2019.
  * Chakravarthy et al. [2022] Ayush K Chakravarthy, Jacob Labe Russin, and Randall O’Reilly.  Systematicity emerges in transformers when abstract grammatical roles guide attention.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop_ , pages 1–8, Hybrid: Seattle, Washington + Online, July 2022. Association for Computational Linguistics.  doi: 10.18653/v1/2022.naacl-srw.1.  URL https://aclanthology.org/2022.naacl-srw.1.
  * Chang et al. [2022] Michael Chang, Thomas L. Griffiths, and Sergey Levine.  Object representations as fixed points: Training iterative refinement algorithms with implicit differentiation.  _ArXiv_ , abs/2207.00787, 2022.
  * Chen et al. [2019] Mickaël Chen, Thierry Artières, and Ludovic Denoyer.  Unsupervised object segmentation by redrawing.  In _Neural Information Processing Systems_ , 2019.
  * Crawford and Pineau [2019] Eric Crawford and Joelle Pineau.  Spatially invariant unsupervised object detection with convolutional neural networks.  In _Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence_ , AAAI’19/IAAI’19/EAAI’19. AAAI Press, 2019\.  ISBN 978-1-57735-809-1.  doi: 10.1609/aaai.v33i01.33013412.  URL https://doi.org/10.1609/aaai.v33i01.33013412.
  * Dai et al. [2017] Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner.  Scannet: Richly-annotated 3d reconstructions of indoor scenes.  In _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2432–2443, 2017.  doi: 10.1109/CVPR.2017.261.
  * Dittadi et al. [2022] Andrea Dittadi, Samuele S Papa, Michele De Vita, Bernhard Schölkopf, Ole Winther, and Francesco Locatello.  Generalization and robustness implications in object-centric learning.  In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, _Proceedings of the 39th International Conference on Machine Learning_ , volume 162 of _Proceedings of Machine Learning Research_ , pages 5221–5285. PMLR, 17–23 Jul 2022.  URL https://proceedings.mlr.press/v162/dittadi22a.html.
  * Duncan [1984] J. Duncan.  Selective attention and the organization of visual information.  _Journal of Experimental Psychology: General_ , 113:501–517, 1984.
  * Elich et al. [2020] Cathrin Elich, Martin R. Oswald, Marc Pollefeys, and Joerg Stueckler.  Weakly supervised learning of multi-object 3d scene decompositions using deep shape priors.  _Comput. Vis. Image Underst._ , 220:103440, 2020.
  * Elsayed et al. [2022] Gamaleldin F. Elsayed, Aravindh Mahendran, Sjoerd van Steenkiste, Klaus Greff, Michael C. Mozer, and Thomas Kipf.  SAVi++: Towards end-to-end object-centric learning from real-world videos.  In _Advances in Neural Information Processing Systems_ , 2022.
  * Engelcke et al. [2021] Martin Engelcke, Oiwi Parker Jones, and Ingmar Posner.  Genesis-v2: Inferring unordered object representations without iterative refinement.  In _Neural Information Processing Systems_ , 2021.
  * Eslami et al. [2016] S. M. Ali Eslami, Nicolas Heess, Theophane Weber, Yuval Tassa, David Szepesvari, Koray Kavukcuoglu, and Geoffrey E. Hinton.  Attend, infer, repeat: Fast scene understanding with generative models.  In _Proceedings of the 30th International Conference on Neural Information Processing Systems_ , NIPS’16, page 3233–3241, Red Hook, NY, USA, 2016. Curran Associates Inc.  ISBN 9781510838819.
  * Fodor and Pylyshyn [1988] Jerry A. Fodor and Zenon W. Pylyshyn.  Connectionism and cognitive architecture: A critical analysis.  _Cognition_ , 28(1-2):3–71, March 1988.  ISSN 00100277.  doi: 10.1016/0010-0277(88)90031-5.
  * Goyal et al. [2021a] A. Goyal, A. Lamb, J. Hoffmann, S. Sodhani, S. Levine, Y. Bengio, and B. Schölkopf.  Recurrent independent mechanisms.  In _9th International Conference on Learning Representations (ICLR)_ , May 2021a.  URL https://openreview.net/pdf?id=mLcmdlEUxy-.
  * Goyal and Bengio [2022] Anirudh Goyal and Yoshua Bengio.  Inductive biases for deep learning of higher-level cognition.  _Proceedings of the Royal Society A_ , 478(2266):20210068, 2022.
  * Goyal et al. [2020] Anirudh Goyal, Alex Lamb, Phanideep Gampa, Philippe Beaudoin, Sergey Levine, Charles Blundell, Yoshua Bengio, and Michael Mozer.  Object files and schemata: Factorizing declarative and procedural knowledge in dynamical systems, 2020.
  * Goyal et al. [2021b] Anirudh Goyal, Aniket Didolkar, Nan Rosemary Ke, Charles Blundell, Philippe Beaudoin, Nicolas Heess, Michael C Mozer, and Yoshua Bengio.  Neural production systems.  _Advances in Neural Information Processing Systems_ , 34:25673–25687, 2021b.
  * Goyal et al. [2021c] Anirudh Goyal, Aniket Didolkar, Alex Lamb, Kartikeya Badola, Nan Rosemary Ke, Nasim Rahaman, Jonathan Binas, Charles Blundell, Michael Mozer, and Yoshua Bengio.  Coordination among neural modules through a shared global workspace.  _arXiv preprint arXiv:2103.01197_ , 2021c.
  * Greff et al. [2017] Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber.  Neural expectation maximization.  In _Proceedings of the 31st International Conference on Neural Information Processing Systems_ , NIPS’17, page 6694–6704, Red Hook, NY, USA, 2017. Curran Associates Inc.  ISBN 9781510860964.
  * Greff et al. [2019a] Klaus Greff, Raphael Lopez Kaufman, Rishabh Kabra, Nicholas Watters, Christopher P. Burgess, Daniel Zoran, Loïc Matthey, Matthew M. Botvinick, and Alexander Lerchner.  Multi-object representation learning with iterative variational inference.  In _International Conference on Machine Learning_ , 2019a.
  * Greff et al. [2019b] Klaus Greff, Raphaël Lopez Kaufman, Rishabh Kabra, Nick Watters, Chris Burgess, Daniel Zoran, Loic Matthey, Matthew M. Botvinick, and Alexander Lerchner.  Multi-object representation learning with iterative variational inference.  In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, _Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 2424–2433. PMLR, 2019b.  URL http://proceedings.mlr.press/v97/greff19a.html.
  * Greff et al. [2020] Klaus Greff, Sjoerd van Steenkiste, and Jürgen Schmidhuber.  On the binding problem in artificial neural networks.  _ArXiv_ , abs/2012.05208, 2020.
  * Greff et al. [2022] Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J. Fleet, Dan Gnanapragasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh-Ti (Derek) Liu, Henning Meyer, Yishu Miao, Derek Nowrouzezahrai, Cengiz Oztireli, Etienne Pot, Noha Radwan, Daniel Rebain, Sara Sabour, Mehdi S. M. Sajjadi, Matan Sela, Vincent Sitzmann, Austin Stone, Deqing Sun, Suhani Vora, Ziyu Wang, Tianhao Wu, Kwang Moo Yi, Fangcheng Zhong, and Andrea Tagliasacchi.  Kubric: A scalable dataset generator.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 3749–3761, June 2022.
  * Gregor et al. [2015] Karol Gregor, Ivo Danihelka, Alex Graves, Danilo Rezende, and Daan Wierstra.  Draw: A recurrent neural network for image generation.  In Francis Bach and David Blei, editors, _Proceedings of the 32nd International Conference on Machine Learning_ , volume 37 of _Proceedings of Machine Learning Research_ , pages 1462–1471, Lille, France, 07–09 Jul 2015. PMLR.  URL https://proceedings.mlr.press/v37/gregor15.html.
  * Groth et al. [2018] Oliver Groth, Fabian B. Fuchs, Ingmar Posner, and Andrea Vedaldi.  Shapestacks: Learning vision-based physical intuition for generalised object stacking.  In Vittorio Ferrari, Martial Hebert, Cristian Sminchisescu, and Yair Weiss, editors, _Computer Vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part I_ , volume 11205 of _Lecture Notes in Computer Science_ , pages 724–739. Springer, 2018.  doi: 10.1007/978-3-030-01246-5\\_43.  URL https://doi.org/10.1007/978-3-030-01246-5_43.
  * Hubert and Arabie [1985] Lawrence J. Hubert and Phipps Arabie.  Comparing partitions.  _Journal of Classification_ , 2:193–218, 1985.
  * Jia et al. [2023] Baoxiong Jia, Yu Liu, and Siyuan Huang.  Improving object-centric learning with query optimization.  _International Conference on Learning Representations (ICLR)_ , 2023\.
  * Jiang et al. [2020] Jindong Jiang, Sepehr Janghorbani, Gerard de Melo, and Sungjin Ahn.  Scalor: Generative world models with scalable object representations.  In _Proceedings of ICLR 2020_. OpenReview.net, 2020.  URL https://openreview.net/pdf?id=SJxrKgStDH.
  * Johnson et al. [2016] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross B. Girshick.  Clevr: A diagnostic dataset for compositional language and elementary visual reasoning.  _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 1988–1997, 2016.
  * Kabra et al. [2019] Rishabh Kabra, Chris Burgess, Loic Matthey, Raphael Lopez Kaufman, Klaus Greff, Malcolm Reynolds, and Alexander Lerchner.  Multi-object datasets.  https://github.com/deepmind/multi-object-datasets/, 2019.
  * Karazija et al. [2021] Laurynas Karazija, Iro Laina, and C. Rupprecht.  Clevrtex: A texture-rich benchmark for unsupervised multi-object segmentation.  _ArXiv_ , abs/2111.10265, 2021.
  * Ke et al. [2021] Nan Rosemary Ke, Aniket Didolkar, Sarthak Mittal, Anirudh Goyal, Guillaume Lajoie, Stefan Bauer, Danilo Rezende, Yoshua Bengio, Michael Mozer, and Christopher Pal.  Systematic evaluation of causal discovery in visual model based reinforcement learning.  _arXiv preprint arXiv:2107.00848_ , 2021.
  * Khosla et al. [2012] Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei.  Novel dataset for fine-grained image categorization : Stanford dogs.  2012\.
  * Kim et al. [2023] Jinwoo Kim, Janghyuk Choi, Ho-Jin Choi, and Seonhoon Kim.  Shepherding slots to objects: Towards stable and robust object-centric learning.  _ArXiv_ , abs/2303.17842, 2023.
  * Kipf et al. [2022] Thomas Kipf, Gamaleldin F. Elsayed, Aravindh Mahendran, Austin Stone, Sara Sabour, Georg Heigold, Rico Jonschkowski, Alexey Dosovitskiy, and Klaus Greff.  Conditional Object-Centric Learning from Video.  In _International Conference on Learning Representations (ICLR)_ , 2022.
  * Kolesnikov et al. [2021] Alexander Kolesnikov, Alexey Dosovitskiy, Dirk Weissenborn, Georg Heigold, Jakob Uszkoreit, Lucas Beyer, Matthias Minderer, Mostafa Dehghani, Neil Houlsby, Sylvain Gelly, Thomas Unterthiner, and Xiaohua Zhai.  An image is worth 16x16 words: Transformers for image recognition at scale.  2021\.
  * Krause et al. [2013] Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei.  3d object representations for fine-grained categorization.  _2013 IEEE International Conference on Computer Vision Workshops_ , pages 554–561, 2013.
  * LaBerge [1983] D. LaBerge.  Spatial extent of attention to letters and words.  _Journal of Experimental Psychology: Human Perception and Performance_ , 9:371–379, 1983.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll’a r, and C. Lawrence Zitnick.  Microsoft COCO: common objects in context.  _CoRR_ , abs/1405.0312, 2014.  URL http://arxiv.org/abs/1405.0312.
  * Lin et al. [2020] Zhixuan Lin, Yi-Fu Wu, Skand Vishwanath Peri, Weihao Sun, Gautam Singh, Fei Deng, Jindong Jiang, and Sungjin Ahn.  Space: Unsupervised object-oriented scene representation via spatial attention and decomposition.  _ArXiv_ , abs/2001.02407, 2020.
  * Locatello et al. [2020] Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf.  Object-centric learning with slot attention, 2020.
  * Mozer [1988] M. C. Mozer.  A connectionist model of selective attention in visual perception.  In V. L. Patel and G. J. Groen, editors, _Proceedings of the Tenth Annual Conference of the Cognitive Science Society_ , pages 195–201, Hillsdale, NJ, 1988. Earlbaum Associates.
  * Pont-Tuset et al. [2015] Jordi Pont-Tuset, Pablo Arbeláez, Jonathan T. Barron, Ferran Marqués, and Jitendra Malik.  Multiscale combinatorial grouping for image segmentation and object proposal generation.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 39:128–140, 2015.
  * Posner [1980] M. I. Posner.  Orienting of attention.  _Quarterly Journal of Experimental Psychology_ , 32:3–25, 1980.
  * Schölkopf et al. [2021] B. Schölkopf, F. Locatello, S. Bauer, N. R. Ke, N. Kalchbrenner, A. Goyal, and Y. Bengio.  Toward causal representation learning.  _Proceedings of the IEEE_ , 109(5):612–634, 2021\.  doi: 10.1109/JPROC.2021.3058954.  URL https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9363924.
  * Seitzer et al. [2023] Maximilian Seitzer, Max Horn, Andrii Zadaianchuk, Dominik Zietlow, Tianjun Xiao, Carl-Johann Simon-Gabriel, Tong He, Zheng Zhang, Bernhard Schölkopf, Thomas Brox, and Francesco Locatello.  Bridging the gap to real-world object-centric learning.  In _ICLR 2023_ , 2023.  URL https://www.amazon.science/publications/bridging-the-gap-to-real-world-object-centric-learning.
  * Singh et al. [2022a] Gautam Singh, Fei Deng, and Sungjin Ahn.  Illiterate dall-e learns to compose.  In _International Conference on Learning Representations_ , 2022a.  URL https://openreview.net/forum?id=h0OYV0We3oh.
  * Singh et al. [2022b] Gautam Singh, Yi-Fu Wu, and Sungjin Ahn.  Simple unsupervised object-centric learning for complex and naturalistic videos.  In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, _Advances in Neural Information Processing Systems_ , 2022b.  URL https://openreview.net/forum?id=eYfIM88MTUE.
  * Spelke and Kinzler [2007] Elizabeth S. Spelke and Katherine D. Kinzler.  Core knowledge.  _Developmental Science_ , 10(1):89–96, 2007.  doi: https://doi.org/10.1111/j.1467-7687.2007.00569.x.  URL https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-7687.2007.00569.x.
  * Traub et al. [2023] Manuel Traub, Sebastian Otte, Tobias Menge, Matthias Karlbauer, Jannik Thümmel, and Martin V. Butz.  Learning what and where: Disentangling location and identity tracking without supervision, 2023.
  * van den Oord et al. [2017] Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu.  Neural discrete representation learning.  In _NIPS_ , 2017.
  * Vecera and Farah [1994] S. P. Vecera and M. J. Farah.  Does visual attention select objects or locations?  _Journal of Experimental Psychology: General_ , 123:146–160, 1994.
  * Wah et al. [2011] Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge J. Belongie.  The caltech-ucsd birds-200-2011 dataset.  2011\.
  * Watters et al. [2019] Nicholas Watters, Loïc Matthey, Christopher P. Burgess, and Alexander Lerchner.  Spatial broadcast decoder: A simple architecture for learning disentangled representations in vaes.  _ArXiv_ , abs/1901.07017, 2019.
  * Weis et al. [2021] Marissa A. Weis, Kashyap Chitta, Yash Sharma, Wieland Brendel, Matthias Bethge, Andreas Geiger, and Alexander S. Ecker.  Benchmarking unsupervised object representations for video sequences.  _J. Mach. Learn. Res._ , 22(1), jan 2021.  ISSN 1532-4435.
  * Whitehead [1928] Alfred North Whitehead.  Symbolism: Its meaning and effect.  _Humana Mente_ , 3(12):527–530, 1928.
  * Yang and Yang [2022] Yafei Yang and Bo Yang.  Promising or elusive? unsupervised object segmentation from real-world single images.  _NeurIPS_ , 2022.
  * Zemel et al. [2002] R. S. Zemel, M. Behrmann, M. C. Mozer, and D. Bavelier.  Experience-dependent perceptual grouping and object-based attention.  _Journal of Experimental Psychology: Human Perception and Performance_ , 28:202–217, 2002.

##  7 SLP Configuration

Table 7: Spatial Locality Prior Configuration Sweep $\bm{\alpha}_{\textrm{lr}}$ | [1.0, 0.5, 0.1]
---|---
$\lambda_{norm}$ | 0.1
spatial iterations | [1, 5, 10, 20, 25]

In order to ensure robust experimentation, we adhere to the default
hyperparameters111For details on the default hyperparameters, please refer to
the original papers. used in the previously reported results. We achieve this
by augmenting various open-sourced and official implementations of each model,
which serve as our baselines. The only exception we made was in determining
the batch size, which was influenced by the hardware available to us.

Since each dataset and model exhibit different learning dynamics and scene
complexities, we tune the hyperparameters of the SLP according to the sweep
configuration outlined in Table 7. Through experimentation, we have found that
setting $\bm{\alpha}_{\textrm{lr}}$ to $1.0$ consistently yields the best
results. Consequently, we exclude the inferior results obtained with other
values of $\bm{\alpha}_{\textrm{lr}}$ from our analysis.

Generally, we observe that a larger number of spatial iterations is necessary
to learn the appropriate spatial bias for scenes derived from more complex
real-world datasets like COCO and ScanNet. Conversely, smaller values of
spatial iterations tend to suffice for simpler and synthetic datasets such as
ClevrTex. For example, as shown in Table 1(b), setting $T_{spat}=1$ brings
about a significant performance improvement for 7-slot Slot Attention on
ClevrTex.

Furthermore, we have discovered that annealing $\bm{\alpha}_{\textrm{lr}}$ by
a factor of $(T_{spat}-j)\ /\ T_{spat}$ enhances optimization dynamics. Our
hypothesis is that initializing $\bm{\alpha}_{\textrm{lr}}$ at $1.0$ and
subsequently annealing it results in a few aggressive steps towards the
optimum, followed by several fine-tuning steps.

###  7.1 Vanilla Slot Attention Experiments

For the experiments conducted in Section 4.1.1, we utilized the open-source
implementation provided by Dittadi et al.
[2022]222https://github.com/addtt/object-centric-library. In Table 8, an
extension of Table 1(b), we include higher values of spatial iterations.
However, we observe minimal gains in performance when increasing the spatial
iterations from 5 to 15. This leads us to hypothesize that the design of
annealing $\bm{\alpha}_{\textrm{lr}}$ starting from $1.0$ encourages rapid
convergence to a near-optimal $\bm{\alpha}$, and further increasing the number
of spatial iterations does not yield significant performance improvements.
Importantly, we did not observe any degradation in performance for any number
of spatial iterations, indicating that SLP does not introduce additional
gradient instability as observed when increasing the number of slot iterations
in Slot Attention and its variants.

Table 8: Foreground ARI (%) Segmentation Accuracy
(mean ± 1 SEM across 3 replications of each simulation)
Varying spatial iterations for 7-slot Slot Attention on ClevrTex
Slot Attention [Locatello et al., 2020] | 0.47  ± 0.22
---|---
Slot Attention + SLP (1 spatial iteration) | 0.54  ± 0.05
Slot Attention + SLP (5 spatial iterations) | 0.64  ± 0.10
Slot Attention + SLP (10 spatial iterations) | 0.65  ± 0.08
Slot Attention + SLP (15 spatial iterations) | 0.66  ± 0.05

###  7.2 BO-QSA experiments

For the experiments conducted in Sections 4.1.2 and 4.2, we built on the open-
source implementation provided by Jia et al.
[2023]333https://github.com/YuLiu-LY/BO-QSA. In Tables 9 and 10, we extend
Tables 3 and 6 respectively to include higher values of spatial iterations.
Through these extensions, we were able to validate our hypothesis regarding
the lack of improvement beyond a certain number of spatial iterations.

Specifically, in Table 3, we observe that increasing the number of spatial
iterations to 5 does not result in a significant performance improvement on
ClevrTex. However, as demonstrated in Table 8, setting SLP to 5 spatial
iterations leads to a significant enhancement in performance. This suggests
that the hyperparameter for spatial iterations is sensitive not only to
dataset complexity but also to the learning dynamics of the model being used.

Table 9: Foreground ARI (%) Segmentation Accuracy
(mean ± 1 SEM across 3 replications of each simulation)
BO-QSA [Jia et al., 2023] augmented with SLP
Varying spatial iterations
Method | ShapeStacks | ObjectsRoom | ClevrTex
---|---|---|---
BO-QSA [Jia et al., 2023] | 0.93  ± 0.05 | 0.87  ± 0.05 | 0.80  ± 0.08
BO-QSA + SLP (5 spatial iterations) | 0.93  ± 0.10 | 0.92  ± 0.08 | 0.82  ± 0.05
BO-QSA + SLP (10 spatial iterations) | 0.95  ± 0.08 | 0.93  ± 0.05 | 0.87  ± 0.05
Table 10: FG-ARI (%) and Mean-Squared Error (MSE)
BO-QSA Jia et al. [2023] with and without SLP
(mean ± 1 SEM across 3 experiment trials)
Varying spatial iterations
Method | ClevrTex-OOD
---|---
| FG-ARI | MSE
BO-QSA [Jia et al., 2023] | 0.86  ± 0.05 | 265  ± 2.9
BO-QSA + SLP (10 spatial iterations) | 0.88  ± 0.05 | 243  ± 1.6
BO-QSA + SLP (20 spatial iterations) | 0.89  ± 0.05 | 239  ± 1.4
BO-QSA + SLP (25 spatial iterations) | 0.89  ± 0.05 | 242  ± 0.8

###  7.3 DINOSAUR experiments

For the experiments conducted in Section 4.1.3, we built upon the open-source
implementation provided by Seitzer et al. [2023]444https://github.com/amazon-
science/object-centric-learning-framework. In these experiments, we were able
to run higher values of spatial iterations, as demonstrated in Table 11. This
was possible because DINOSAUR utilizes a frozen pre-trained ViT-B/8 encoder,
resulting in a lower number of effective model parameters. Interestingly, we
observed significant improvements for higher values of spatial iterations,
which can be attributed to the fact that the DINOSAUR model does not apply any
positional encoding to the encoded image representation. These results provide
further evidence that SLP offers a stronger spatial bias compared to the
positional encoding layer. They also suggest that existing methods which use
positional encoding layers, need a stronger spatial bias which SLP provides.

Table 11: FG-ARI and mBO measures of object discovery
on MoVi-C and MoVi-E with DINOSAUR [Seitzer et al., 2023] (mean ± 1 SEM across
3 runs).
Method | MoVi-C | MoVi-E
---|---|---
| FG-ARI | mBO | FG-ARI | mBO
DINOSAUR (ViT-B/8) | 68.9  ± 0.3 | 38.0  ± 0.2 | 65.1  ± 0.6 | 33.5  ± 0.1
DINOSAUR + SLP (10 spatial iterations) | 70.1  ± 0.4 | 38.5  ± 0.2 | 66.3  ± 0.6 | 34.5  ± 0.3
DINOSAUR + SLP (20 spatial iterations) | 72.8  ± 0.6 | 41.5  ± 0.3 | 70.4  ± 0.4 | 35.9  ± 0.2
DINOSAUR + SLP (50 spatial iterations) | 74.6 | 42.6 | 73.1 | 36.7