††thanks: current address: Vienna Center for Quantum Science and Technology,
Technische Universität Wien, Atominstitut, Vienna, Austria

# Observation of two non-thermal fixed points for the same microscopic
symmetry

Stefan Lannig<EMAIL_ADDRESS>Maximilian Prüfer  Yannick Deller
Ido Siovitz  Jan Dreher  Thomas Gasenzer  Helmut Strobel  Markus K. Oberthaler
Kirchhoff-Institut für Physik, Universität Heidelberg, Im Neuenheimer Feld
227, 69120 Heidelberg, Germany

###### Abstract

Close to equilibrium, the underlying symmetries of a system determine its
possible universal behavior. Far from equilibrium, however, different
universal phenomena associated with the existence of multiple non-thermal
fixed points can be realized for given microscopic symmetries. Here, we study
this phenomenon using a quasi-one-dimensional spinor Bose-Einstein condensate.
We prepare two different initial conditions and observe two distinct universal
scaling dynamics with different exponents. Measurements of the complex-valued
order parameter with spatial resolution allow us to characterize the phase-
amplitude excitations for the two scenarios. Our study provides new insights
into the phenomenon of universal dynamics far from equilibrium and opens a
path towards mapping out the associated basins of non-thermal fixed points.

Universality is a powerful concept for characterizing systems by means of
effective models based on features that are independent of microscopic
details. This concept led to the identification of universality classes for
systems at and near equilibrium [1, 2]. For example, close to a phase
transition, they are characterized by a few universal exponents, which
describe the relevant properties as functions of the distance to the critical
point [3].

In such close-to-equilibrium scenarios, universal exponents are typically
associated with the symmetry properties of the underlying Hamiltonian in the
respective phase; generically, that means that for a fixed Hamiltonian, there
exists a unique set of universal exponents and scaling functions. The
situation changes for systems far from equilibrium. Here, possible excitations
are richer and not strictly tied to the symmetry properties associated with
the ground state of the Hamiltonian. Thus, universal phenomena far from
equilibrium are expected to have more defining elements than the underlying
microscopic Hamiltonian.

Motivated by renormalization-group ideas, the concept of non-thermal fixed
points (NTFPs) has been developed as a means of characterizing universal types
of dynamics far from equilibrium [4, 5]. NTFPs imply universal dynamics, that
is, time evolution which can be described by rescaling in time and space
according to a set of universal exponents [6, 7]. Quantum simulators utilizing
ultracold atoms have proven to be a versatile platform for studying non-
equilibrium behavior of quantum many-body systems [8, 9, 10]. During recent
years, studies of universal phenomena far from equilibrium have intensified,
both in theory [4, 5, 11, 12, 13, 14, 6, 15, 7, 16, 17, 18, 19, 20, 21, 22,
23, 24, 25, 26, 27, 28, 29, 30] and experiment [31, 32, 33, 34, 35, 36, 37,
38]. Universal scaling associated with NTFPs was observed experimentally in
different systems [31, 32, 33, 35]; recently, distinct universal phenomena,
depending on the Hamiltonian symmetries, have been reported [37].

Far from equilibrium, there is numerical evidence that the universal behavior
can exhibit a dependence of the exponents on the initial condition [18], see
also [33] for related experiments. Intuitively, this is due to the possible
existence of, e.g., different configurations of linear or nonlinear, such as
topological, excitations dominating the dynamics. Nevertheless, to be
physically relevant, a range of initial conditions must lead to the same
universal dynamics (see Fig. 1). Thus, it is sensible to define, for every
NTFP, an associated basin encompassing initial states, from which the system
can evolve into the vicinity of that fixed point [13].

Figure 1:  Schematic representation of the equilibration of a many-body system
starting from different initial conditions. During this evolution a transient
period of slow self-similar behavior in the vicinity of a NTFP may occur. The
associated basins are defined by the set of initial conditions which evolve
towards the same NTFP. This is the simplest scenario, more complicated ones
are possible, such as the existence of associated basins that lead to a
subsequent time evolution to two or more NTFPs.

In this work, we experimentally confirm the existence of two distinct NTFPs in
a system with the same microscopic Hamiltonian. We achieve this by generating
initial conditions (ICs) belonging to two different associated basins. Both
ICs are obtained by quenching to the same parameter regime. Prior to the
quench, one is prepared in the ground state of another phase, while for the
other one we additionally generate localized excitations, employing local spin
control. For the two scaling evolutions we find distinct sets of universal
exponents and the corresponding universal scaling functions.

Figure 2:  Time evolution from two different initial conditions (ICs) after
quenching the control parameter $q$. (a) Quasi-one-dimensional BEC in a box
potential (at the start of the evolution at $t=0$) formed by an elongated
dipole trap with repulsive walls. (b) The relevant energy difference $q$
between the $m_{\text{F}}=0$ and $m_{\text{F}}=\pm 1$ levels is controlled
with off-resonant microwave dressing. This allows tuning spin-changing
collisions into resonance, which redistribute population between the hyperfine
levels. (c) Absorption pictures of the atom clouds after a short Stern-Gerlach
pulse for the polar (left) and rotation ICs (right). For the rotation IC, six
local spin rotations transfer atoms from $m_{\text{F}}=0$ to $m_{\text{F}}=\pm
1$. (d) Time evolution of $F_{x}$-$F_{y}$-histograms for the polar (left) and
rotation ICs (right). The black lines correspond to the spatial profiles of a
representative single realization. For the polar IC the system relaxes to an
approx. constant transverse spin length with phase excitations along the ring.
In contrast, combined spin-length and phase fluctuations persist for the
rotation IC.

In our experiments, we employ a Bose-Einstein condensate of 87Rb in the $F=1$
hyperfine manifold. The atom cloud is contained in a quasi one-dimensional box
consisting of a red-detuned dipole trap with blue-detuned end caps. The latter
confine the atoms within the central part of the longitudinal harmonic
potential created by the red-detuned beam along the $x$-direction (the
resulting density is shown in Fig. 2(a)). After preparing the different
initial conditions, we study their respective dynamics after a quench. For
brevity, we denote the ICs without and with additional excitations as polar IC
and rotation IC, respectively. Absorption pictures of these ICs are shown in
Fig. 2(c).

The polar IC is set by a gas with $\sim 1.6\cdot 10^{5}$ atoms in the polar
state, all of them occupying the $m_{\text{F}}=0$ magnetic sub-state (cf. Fig.
2(c)). For the rotation IC, we prepare the polar state with a lower atom
number of $\sim 4\cdot 10^{4}$ and additionally generate excitations via 6
equally spaced local spin rotations. Each rotation angle is chosen greater
than $\pi/2$, which, in the polar phase, leads to the generation of vector
solitons [39]. The lower atom number is chosen to ensure a sufficiently long
lifetime of excitations.

Following the preparation of each IC, the parameter $q$, denoting the energy
offset between $m_{\text{F}}=0$ and $m_{\text{F}}=\pm 1$ (see Fig. 2(b) for
the internal level structure and [40] for the microscopic Hamiltonian), is
quenched to a value within the easy-plane phase [41, 42], by instantaneously
applying off-resonant microwave dressing [43]. The quench tunes spin-changing
collisions into resonance, which redistribute atoms between the magnetic
levels for both ICs (cf. Fig. 2(b)) and lead to a build-up of transverse spin
$F_{\perp}=F_{x}+\mathrm{i}F_{y}$, with only small excitations arising in
$F_{z}$ (see Fig. S1 in [40]). To access the order-parameter field
$F_{\perp}$, we simultaneously measure orthogonal spin projections transverse
to the magnetic offset field with spatial resolution along the $x$-axis [44].

Figure 3:  Scaling evolution of the transverse-spin structure factor $f$ for
the polar (a-c) and rotation ICs (d-f). Unscaled (a,d) and scaled (b,e)
$F_{\perp}$ power spectra as a function of $k=2\pi/\lambda$ with $\lambda$ the
wavelength and corresponding residuals (insets) with respect to the scaling
function $f_{\text{s}}(k)$ at reference time
$t_{\text{r}}=$42\text{\,}\mathrm{s}$$ are displayed over the time range which
shows self-similar evolution. The scaling exponents are extracted via a
$\chi^{2}$ minimization with respect to the function $f_{\text{s}}$, scaled
according to Eq. (1) (see [40] for details). For this analysis, only the
points in the gray shaded areas are used, where the spectra have the same
shape. All error bars indicate 1 s.d. of the mean; where no error bars are
visible they are smaller than the plot marker. This leads to the likelihood
function $L\propto\exp(-\chi^{2}/2)$ (normalized to 1 at the maximum) of the
residuals $\chi^{2}$ shown in the colored contour plot in (c,f). The black and
gray ellipses indicate the $2\sigma$ and $5\sigma$ statistical error ranges of
the extracted exponents. The optimal scaling exponents given in the text are
extracted at the position of the minimal residuals (marked by dashed lines and
dots).

We observe that, after around $17\text{\,}\mathrm{s}$, the system reaches a
regime where long-wavelength correlations only change slowly in time (cf. Fig.
S2 in [40]), however, excitations in the transverse spin plane differ
drastically for the distinct ICs. For the polar IC the ensuing dynamics after
the quench causes the transverse spin $F_{\perp}$ to settle to the bottom of
the corresponding Mexican-hat mean-field potential in the transverse-spin
observables, with Goldstone excitations along the bottom in angular direction
(see the left col. of Fig. 2(d)). Contrary to this, when starting from the
rotation IC, large fluctuations filling the transverse plane build up and
persist during the subsequent evolution (right col. of Fig. 2(d)). We estimate
that, for both ICs, the energy input by the quench leaves the system well
below the critical temperature for which the easy-plane phase vanishes [45,
40]. Thus, we conclude that the expected thermal state for these energies has
the same properties as a thermal state in the easy-plane phase [46]; that is,
one would expect finding a ring structure in the $F_{x}$-$F_{y}$-histograms.

For both initial conditions we infer spatial dynamics which shows scaling in
space and time with distinct parameters. To investigate this phenomenon, we
evaluate the time-evolving structure factor
$f(k,t)=\langle|\text{dFT}[F_{\perp}(x,t)](k)|^{2}\rangle$ from the
transverse-spin momentum spectra, cf. Fig. 3(a,d). Here, $\text{dFT}[\cdot]$
denotes the discrete Fourier transform and $\langle\cdots\rangle$ the ensemble
average. We analyze the evolution of $f(k,t)$ with respect to spatio-temporal
scaling of the form [7, 22]

$f(k,t)=\left({t}/{t_{\text{r}}}\right)^{\alpha}f_{\text{s}}\left(\left[t/t_{\text{r}}\right]^{\beta}k\right)\,,$
(1)

where $\alpha$ and $\beta$ denote the amplitude and momentum scaling
exponents, $f_{\text{s}}$ represents a time-independent scaling function and
$t_{\text{r}}$ is a reference time within the scaling regime.

After an initial period of $\sim$17\text{\,}\mathrm{s}$$, the spectra $f$ have
approached the form of the scaling function $f_{\text{s}}$. Thereafter, we
observe, independently for both ICs, a self-similar shift towards lower
momenta that is in accordance with Eq. (1) (see Figs. 3(a,d)). Figs. 3(b,e)
show the respective rescaled spectra. These coincide well with scaling
functions $f_{\text{s}}(k)\propto 1/[1+(k/k_{\text{s}})^{\zeta}]$ (solid gray
lines in Figs. 3(b,e)), with inverse-length scale
$k_{\text{s}}=2\pi/\lambda_{\text{s}}$ (for
$t_{\text{r}}=$42\text{\,}\mathrm{s}$$ we find $\lambda_{\text{s}}=$(147\pm
13_{\,\text{stat}})\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ for the polar IC
and $\lambda_{\text{s}}=$(33.5\pm
1.3_{\,\text{stat}})\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ for the rotation
IC). This matches the previous observations in a harmonic trap [31]. In the
re-scaling analysis for the polar IC we find the values

$\displaystyle\alpha$ $\displaystyle=$0.64\pm 0.09_{\,\text{stat}}\pm
0.50_{\,\text{sys}}$\,,$ $\displaystyle\beta$ $\displaystyle=$0.58\pm
0.04_{\,\text{stat}}\pm 0.26_{\,\text{sys}}$\,,$ $\displaystyle\zeta$
$\displaystyle=$2.51\pm 0.06_{\,\text{stat}}$\,,$

while the rotation IC gives rise to scaling with

$\displaystyle\alpha$ $\displaystyle=$0.24\pm 0.04_{\,\text{stat}}\pm
0.03_{\,\text{sys}}$\,,$ $\displaystyle\beta$ $\displaystyle=$0.28\pm
0.04_{\,\text{stat}}\pm 0.05_{\,\text{sys}}$\,,$ $\displaystyle\zeta$
$\displaystyle=$2.87\pm 0.18_{\,\text{stat}}$\,.$

Here, we have chosen $t_{\text{r}}=$42\text{\,}\mathrm{s}$$ for the extraction
of the scaling functions. The statistical error is extracted from the root-
mean-square width of the marginal likelihood distributions shown in Fig.
3(c,f) and the systematic errors are estimated from the variability of the
exponents when varying the momentum cutoff (see [40] for details). The large
systematic error for the exponent $\alpha$ in the polar IC is a result of the
finite system size, which obstructs the observation of a clear infrared
plateau. Nevertheless, under the assumption of the transport of conserved
$F_{\perp}$ excitations [21, 47], implying $\alpha=\beta$, we obtain
$\beta=$0.54\pm 0.02_{\,\text{stat}}\pm 0.05_{\,\text{sys}}$$ for the polar IC
and $\beta=$0.28\pm 0.03_{\,\text{stat}}\pm 0.04_{\,\text{sys}}$$ for the
rotation IC.

For the extraction of the scaling exponents and function parameters we apply a
multi-step $\chi^{2}$ optimization procedure, described in more detail in
[40]. First, approximate scaling exponents are extracted by minimizing the
squared deviations between measured spectra with respect to the scaling
hypothesis (1). Then, the parameters of the scaling function are obtained from
a fit to all rescaled data. Finally, these parameters enter into a functional
scaling model, which is used to define the squared deviations
$\chi^{2}=\sum_{k,t}[f_{k,t}-f(k,t)]^{2}/\sigma_{k,t}^{2}$ of the unscaled
measured values $f_{k,t}$ from the scaling prediction $f(k,t)$, relative to
the measurement uncertainties $\sigma_{k,t}$. The resulting likelihood
$L\propto\exp[-\chi^{2}(\alpha,\beta)/2]$ is shown in Figs. 3(c,f), which is
used to obtain the scaling exponents with statistical errors as stated above.
We select the momentum scaling regime (gray shaded area in Fig. 3) from the
lowest $k>0$ up to $k_{\text{max}}$, where the shape of the measured spectra
starts to differ for different times. The insets in Fig. 3 depict the
residuals of the spectra, obtained by dividing the measured data by the
scaling function.

To gain more insight into the difference between the two scaling scenarios we
evaluate the space-resolved profiles of the transverse spin
$F_{\perp}(x)=|F_{\perp}(x)|e^{\mathrm{i}\varphi_{\text{L}}(x)}$ of single
realizations in Fig. 4(a). The solid lines show the spin length
$|F_{\perp}(x)|$ and the dashed ones the Larmor phase $\varphi_{\text{L}}(x)$
at $t=$29\text{\,}\mathrm{s}$$. For the polar IC, mostly phase excitations
with roughly constant $|F_{\perp}|$ are present in the system while, for the
rotation IC, strong localized phase-amplitude defects are present (marked by
vertical blue lines).

Figure 4:  Spatial structure of the spin excitations in the system. (a)
Transverse-spin length $\lvert F_{\perp}\rvert$ (upper plot: solid black line)
and Larmor phase $\varphi_{\text{L}}$ (upper plot: dashed red line) extracted
from the same single realization at $t=$42\text{\,}\mathrm{s}$$ shown as black
line in Fig. 2. The lower plot shows the argument
$S(x)=\Delta|F_{\perp}(x)|\cdot\left|\nabla\varphi_{\text{L}}(x)\right|$ of
the correlator given in Eq. (2) for the single realizations. All positions
with large correlations between spin length reduction and phase gradient with
$S(x)>0.25$ (horizontal blue line) are marked with vertical blue lines. (b)
Evolution of correlator amplitude $\mathscr{C}(\Delta x=0,t)$ and spatial
profiles (insets) of the full cross correlator $\mathscr{C}$. This shows the
larger abundance and persistence of excitations for the rotation IC as
compared to the polar IC, which shows a decay.

To characterize these phase-amplitude excitations, we evaluate the spatial
cross-correlation function

$\mathscr{C}(\Delta
x,t)=\sum_{x}\left\langle\Delta\left|F_{\perp}(x,t)\right|\cdot\left|\nabla\varphi_{\text{L}}(x+\Delta
x,t)\right|\right\rangle$ (2)

between spin-length variations and gradients of the Larmor phase.
Specifically,
$\Delta|F_{\perp}|(x,t)=\langle{|F_{\perp}|}\rangle_{x}(t)-|F_{\perp}|(x,t)$
describes the local deviation of the spin length $\left|F_{\perp}\right|$ from
the mean value $\langle{|F_{\perp}|}\rangle_{x}$ taken over all positions and
realizations. The time evolution of the local correlator amplitude
$\mathscr{C}(\Delta x=0,t)$ in Fig. 4(b) shows a clear distinction between the
decay of excitations for the polar IC as compared to the rotation IC: The
spatial correlator profiles show a distinct peak at $\Delta x=0$ in accordance
with the structures identified in Fig. 4(a). While, for the polar IC, the
local amplitude $\mathscr{C}(\Delta x=0,t)$ decays over time, it remains
approximately unchanged for the rotation IC. We find the conservation of the
correlator amplitude to be enhanced for lower total atomic densities, while
larger densities lead to a decay.

Numerical simulations for the rotation IC demonstrate that the phase kinks are
rather stable, in line with the conservation of $\mathscr{C}(\Delta x=0,t)$,
and propagate through the system while interacting with the strongly
fluctuating spin [40]. The corresponding scaling exponents agree with the
experimental results. This is in contrast to simulations for the polar IC,
where a similarly slow scaling with exponents $\alpha=0.27\pm 0.06$,
$\beta=0.25\pm 0.04$ has been observed numerically [48, 49].

We report the measurement of two distinct scaling evolutions characterized by
different exponents and scaling functions in a regime with the same
microscopic Hamiltonian symmetry of a spinor gas in the easy-plane phase. Our
findings show the existence of distinct associated basins of initial
conditions, that is, the sets of states which evolve towards their respective
NTFP. Thus, for a classification of universal phenomena far from equilibrium,
not only the scaling properties in the vicinity of NTFPs are important but
also the structure of their associated basins. This hints towards the
necessity to include the initial condition for identifying universality
classes of far-from-equilibrium dynamics; this is still an unsettled question.

###### Acknowledgements.

The authors thank S. Erne for useful comments on error estimation and K.
Boguslavski, P. Heinen, P.G. Kevrekidis, A.N. Mikheev, and C.M. Schmied for
discussions and collaboration on related topics. They acknowledge support by
the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) through
SFB 1225 ISOQUANT – 273811115, and GA677/10-1, as well as under Germany’s
Excellence Strategy – EXC-2181/1 – 390900948 (the Heidelberg STRUCTURES
Excellence Cluster), and by the state of Baden-Württemberg through bwHPC and
DFG through grants INST 35/1134-1 FUGG, INST 35/1503-1 FUGG, INST 35/1597-1
FUGG, and 40/575-1 FUGG.

## References

  * Hohenberg and Halperin [1977] P. C. Hohenberg and B. I. Halperin, Rev. Mod. Phys. 49, 435 (1977).
  * Bray [1994] A. Bray, Adv. Phys. 43, 357 (1994).
  * Zinn-Justin [2004] J. Zinn-Justin,  _Quantum Field Theory and Critical Phenomena_ (Clarendon Press, Oxford, UK, 2004).
  * Berges  _et al._ [2008] J. Berges, A. Rothkopf, and J. Schmidt, Phys. Rev. Lett. 101, 041603 (2008).
  * Nowak  _et al._ [2011] B. Nowak, D. Sexty, and T. Gasenzer, Phys. Rev. B 84, 020506(R) (2011).
  * Berges  _et al._ [2015] J. Berges, K. Boguslavski, S. Schlichting, and R. Venugopalan, Phys. Rev. Lett. 114, 061601 (2015).
  * Piñeiro Orioli  _et al._ [2015] A. Piñeiro Orioli, K. Boguslavski, and J. Berges, Phys. Rev. D 92, 025041 (2015).
  * Polkovnikov  _et al._ [2011] A. Polkovnikov, K. Sengupta, A. Silva, and M. Vengalattore, Rev. Mod. Phys. 83, 863 (2011).
  * Langen  _et al._ [2016] T. Langen, T. Gasenzer, and J. Schmiedmayer, J. Stat. Mech. Theor. Exp. 2016, 064009 (2016).
  * Ueda [2020] M. Ueda, Nat. Rev. Phys. 2, 669 (2020).
  * Schole  _et al._ [2012] J. Schole, B. Nowak, and T. Gasenzer, Phys. Rev. A 86, 013624 (2012).
  * Nowak  _et al._ [2014] B. Nowak, J. Schole, and T. Gasenzer, New J. Phys. 16, 093052 (2014).
  * Berges  _et al._ [2014a] J. Berges, K. Boguslavski, S. Schlichting, and R. Venugopalan, JHEP 2014 (5), 54.
  * Hofmann  _et al._ [2014] J. Hofmann, S. S. Natu, and S. Das Sarma, Phys. Rev. Lett. 113, 095702 (2014).
  * Maraga  _et al._ [2015] A. Maraga, A. Chiocchetta, A. Mitra, and A. Gambassi, Phys. Rev. E 92, 042151 (2015).
  * Williamson and Blakie [2016] L. A. Williamson and P. B. Blakie, Phys. Rev. Lett. 116, 025301 (2016).
  * Bourges and Blakie [2017] A. Bourges and P. B. Blakie, Phys. Rev. A 95, 023616 (2017).
  * Karl and Gasenzer [2017] M. Karl and T. Gasenzer, New J. Phys. 19, 093014 (2017).
  * Fujimoto  _et al._ [2018] K. Fujimoto, R. Hamazaki, and M. Ueda, Phys. Rev. Lett. 120, 073002 (2018).
  * Walz  _et al._ [2018] R. Walz, K. Boguslavski, and J. Berges, Phys. Rev. D 97, 116011 (2018).
  * Mikheev  _et al._ [2019] A. N. Mikheev, C.-M. Schmied, and T. Gasenzer, Phys. Rev. A 99, 063622 (2019).
  * Schmied  _et al._ [2019a] C.-M. Schmied, A. N. Mikheev, and T. Gasenzer, Int. J. Mod. Phys. A 34, 1941006 (2019a).
  * Fujimoto  _et al._ [2019] K. Fujimoto, R. Hamazaki, and M. Ueda, Phys. Rev. Lett. 122, 173001 (2019).
  * Williamson and Blakie [2019] L. A. Williamson and P. B. Blakie, SciPost Physics 7, 029 (2019).
  * Schmied  _et al._ [2019b] C. M. Schmied, T. Gasenzer, and P. B. Blakie, Phys. Rev. A 100, 033603 (2019b).
  * Gao  _et al._ [2020] C. Gao, M. Sun, P. Zhang, and H. Zhai, Phys. Rev. Lett. 124, 040403 (2020).
  * Wheeler  _et al._ [2021] M. T. Wheeler, H. Salman, and M. O. Borgh, EPL 135, 30004 (2021).
  * Gresista  _et al._ [2022] L. Gresista, T. V. Zache, and J. Berges, Phys. Rev. A 105, 013320 (2022).
  * Rodriguez-Nieva  _et al._ [2022] J. F. Rodriguez-Nieva, A. Piñeiro Orioli, and J. Marino, PNAS 119, e2122599119 (2022).
  * Preis  _et al._ [2023] T. Preis, M. P. Heller, and J. Berges, Phys. Rev. Lett. 130, 031602 (2023).
  * Prüfer  _et al._ [2018] M. Prüfer, P. Kunkel, H. Strobel, S. Lannig, D. Linnemann, C.-M. Schmied, J. Berges, T. Gasenzer, and M. K. Oberthaler, Nature 563, 217 (2018).
  * Erne  _et al._ [2018] S. Erne, R. Bücker, T. Gasenzer, J. Berges, and J. Schmiedmayer, Nature 563, 225 (2018).
  * Johnstone  _et al._ [2019] S. P. Johnstone, A. J. Groszek, P. T. Starkey, C. J. Billington, T. P. Simula, and K. Helmerson, Science 364, 1267 (2019).
  * Navon  _et al._ [2019] N. Navon, C. Eigen, J. Zhang, R. Lopes, A. L. Gaunt, K. Fujimoto, M. Tsubota, R. P. Smith, and Z. Hadzibabic, Science 366, 382 (2019).
  * Glidden  _et al._ [2021] J. A. P. Glidden, C. Eigen, L. H. Dogra, T. A. Hilker, R. P. Smith, and Z. Hadzibabic, Nat. Phys. 17, 457 (2021).
  * García-Orozco  _et al._ [2022] A. D. García-Orozco, L. Madeira, M. A. Moreno-Armijos, A. R. Fritsch, P. E. S. Tavares, P. C. M. Castilho, A. Cidrim, G. Roati, and V. S. Bagnato, Phys. Rev. A 106, 023314 (2022).
  * Huh  _et al._ [2023] S. Huh, K. Mukherjee, K. Kwon, J. Seo, S. I. Mistakidis, H. R. Sadeghpour, and J. Choi, (2023), arXiv:2303.05230 .
  * Martirosyan  _et al._ [2023] G. Martirosyan, C. J. Ho, J. Etrych, Y. Zhang, A. Cao, Z. Hadzibabic, and C. Eigen, (2023), arXiv:2304.06697 .
  * Lannig  _et al._ [2020] S. Lannig, C.-M. Schmied, M. Prüfer, P. Kunkel, R. Strohmaier, H. Strobel, T. Gasenzer, P. G. Kevrekidis, and M. K. Oberthaler, Phys. Rev. Lett. 125, 170401 (2020).
  * [40] See Supplemental Material for details on the experimental system, the microscopic model, the extraction of the scaling parameters and data from numerical simulations. Additionally, spectra for all times and the full $k$-ranges are provided.
  * Kawaguchi and Ueda [2012] Y. Kawaguchi and M. Ueda, Phys. Rep. 520, 253 (2012).
  * Stamper-Kurn and Ueda [2013] D. M. Stamper-Kurn and M. Ueda, Rev. Mod. Phys. 85, 1191 (2013).
  * Gerbier  _et al._ [2006] F. Gerbier, A. Widera, S. Fölling, O. Mandel, and I. Bloch, Phys. Rev. A 73, 041602(R) (2006).
  * Kunkel  _et al._ [2019] P. Kunkel, M. Prüfer, S. Lannig, R. Rosa-Medina, A. Bonnin, M. Gärttner, H. Strobel, and M. K. Oberthaler, Phys. Rev. Lett. 123, 063603 (2019).
  * Phuc  _et al._ [2011] N. T. Phuc, Y. Kawaguchi, and M. Ueda, Phys. Rev. A 84, 043645 (2011).
  * Prüfer  _et al._ [2022] M. Prüfer, D. Spitz, S. Lannig, H. Strobel, J. Berges, and M. K. Oberthaler, Nat. Phys. 18, 1459 (2022).
  * Prüfer  _et al._ [2020] M. Prüfer, T. V. Zache, P. Kunkel, S. Lannig, A. Bonnin, H. Strobel, J. Berges, and M. K. Oberthaler, Nat. Phys. 16, 1012 (2020).
  * Schmied  _et al._ [2019c] C.-M. Schmied, M. Prüfer, M. K. Oberthaler, and T. Gasenzer, Phys. Rev. A 99, 033611 (2019c).
  * Siovitz  _et al._ [2023] I. Siovitz, S. Lannig, Y. Deller, H. Strobel, M. K. Oberthaler, and T. Gasenzer, (2023), arXiv:2304.09293 .
  * Kunkel  _et al._ [2018] P. Kunkel, M. Prüfer, H. Strobel, D. Linnemann, A. Frölian, T. Gasenzer, M. Gärttner, and M. K. Oberthaler, Science 360, 413 (2018).
  * Dalfovo  _et al._ [1999] F. Dalfovo, S. Giorgini, L. P. Pitaevskii, and S. Stringari, Rev. Mod. Phys. 71, 463 (1999).
  * Berges  _et al._ [2014b] J. Berges, K. Boguslavski, S. Schlichting, and R. Venugopalan, Phys. Rev. D 89, 114007 (2014b).

##  Appendix A Supplemental material

###  A.1 Experimental System

Trapping Potential
The dipole trap in the experiment is formed by a focused
$1030\text{\,}\mathrm{n}\mathrm{m}$ laser beam, leading to an approximately
harmonic confinement with trap frequencies of
$(\omega_{\parallel},\omega_{\perp})=2\pi\times(1.6,160)\,$\mathrm{H}\mathrm{z}$$,
in conjunction with two blue-detuned beams at
$760\text{\,}\mathrm{n}\mathrm{m}$. The latter implement walls separated by
$\sim$130\text{\,}\mathrm{\SIUnitSymbolMicro m}$$, which lead to an
approximately homogeneous system with a rapid decrease of the total density
within $\sim$10\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ at the edges.

Microscopic Hamiltonian
The Hamiltonian of the system is described by [41, 42]

$\hat{H}=\hat{H}_{0}+\int
dV\biggl{[}\,:\frac{c_{0}}{2}\hat{n}^{2}+\frac{c_{1}}{2}\left(\hat{F}_{x}^{2}+\hat{F}_{y}^{2}+\hat{F}_{z}^{2}\right):+p\left(\hat{n}_{+1}-\hat{n}_{-1}\right)+q\left(\hat{n}_{+1}+\hat{n}_{-1}\right)\biggr{]}\,,$
(S1)

where $\hat{H}_{0}$ describes the kinetic energy and trapping potential and
$\hat{n}=\sum_{j}\hat{n}_{j}$,
$\hat{n}_{j}=\hat{\psi}_{j}^{\dagger}\hat{\psi}_{j}$, with
$\hat{\psi}_{j}^{(\dagger)}$ being the bosonic field annihilation (creation)
operator in the magnetic substate $j\in\\{0,\pm 1\\}$,
$F_{x}=[\hat{\psi}_{0}^{\dagger}(\hat{\psi}_{+1}+\hat{\psi}_{-1})+\text{h.c.}]/\sqrt{2}$,
$\hat{F}_{y}=[i\hat{\psi}_{0}^{\dagger}(\hat{\psi}_{+1}-\hat{\psi}_{-1})+\text{h.c.}]/\sqrt{2}$,
$\hat{F}_{z}=\hat{n}_{+1}-\hat{n}_{-1}$ describe the spin field operators,
$:\,:$ denotes normal ordering, and $p$ and $q$ label the linear and second-
order Zeeman shifts, respectively. For 87Rb in the $F=1$ hyperfine manifold
the spin interactions are ferromagnetic ($c_{1}<0$), such that spin-changing
collisions (SCCs) of the form
$\hat{\psi}_{+1}^{\dagger}\hat{\psi}_{-1}^{\dagger}\hat{\psi}_{0}\hat{\psi}_{0}+\text{h.c.}$,
contained in the terms $\hat{F}_{x}^{2}+\hat{F}_{y}^{2}$ of the Hamiltonian,
lead to the build-up of the transverse spin, as shown in the left column of
Fig. 2(d).

Magnetic Offset Field and Control of $\bm{q}$
During the experiment the atom cloud is subject to a homogeneous magnetic
offset field of $B=$0.894\text{\,}\mathrm{G}$$, leading to a linear and
quadratic Zeeman splitting of
$p=h\times$626\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$$ and
$q_{\text{B}}=h\times$58\text{\,}\mathrm{H}\mathrm{z}$$, respectively. The
spin dynamics is controlled by varying the energy detuning
$q=q_{\text{B}}+q_{\text{MW}}$ by applying off-resonant microwave dressing
[43] at a Rabi frequency $\Omega_{\text{MW}}\approx
2\pi\times$7.2\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$$ and blue-detuned by
$\delta_{\text{MW}}=2\pi\times$246.256\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$$
(polar IC) or
$\delta_{\text{MW}}=2\pi\times$248.645\text{\,}\mathrm{k}\mathrm{H}\mathrm{z}$$
(rotation IC) to the
$(F=1,m_{\text{F}}=0)\longleftrightarrow(F=2,m_{\text{F}}=0)$ transition. This
corresponds to a difference in $q_{\text{MW}}$ of approx.
$h\times$0.5\text{\,}\mathrm{H}\mathrm{z}$$. The quench of $q$ at $t=0$ is
implemented by instantaneously switching on the microwave power. The detuning
$\delta_{\text{MW}}$ for the polar IC is chosen such that the transverse spin
length $|F_{\perp}|$ after $t=$30\text{\,}\mathrm{s}$$ becomes maximal at a
value of 0.73. Also the $q$ value used for the rotation IC is close to the
maximum of $|F_{\perp}|$ in this measurement for the corresponding density
without local rotations. For both initial conditions we confirm to stay in the
easy-plane phase throughout the dynamical evolution by observing only small
excitations in $F_{z}$ (see Fig. S1).

Magnetic field gradients are precisely compensated by tuning $q$ to the left
half of the SCC resonance, where $F_{z}$ excitations are expected. Here,
magnetic field gradients along the cloud lead to a build-up of spatial
imbalances in the averaged $m_{\text{F}}=\pm 1$ density profiles at large $t$.
The magnetic field gradient is compensated by ensuring that the averaged
$m_{\text{F}}=\pm 1$ density profiles are homogeneous at
$t=$30\text{\,}\mathrm{s}$$ after the quench from the polar IC.

Figure S1:  Histograms of the $F_{x}$-$F_{y}$ and $F_{x}$-$F_{z}$ cross
sections for a simultaneous measurement of all three spin projections during
the time evolution. Throughout the scaling regime
($t=$1742\text{\,}\mathrm{s}$$) the $F_{z}$ fluctuations stay small for both
ICs.

Atom Loss
Over the duration of the experiment the atom number decreases and between
$t=$1042\text{\,}\mathrm{s}$$ the atom number is described well by an
exponential decrease with $1/e$ lifetime of $\tau=$37\text{\,}\mathrm{s}$$
(polar IC) and $\tau=$52\text{\,}\mathrm{s}$$ (rotation IC). At the start of
the scaling regime ($t=$17\text{\,}\mathrm{s}$$) the one-dimensional densities
along axial direction are approx.
$700\text{\,}\mathrm{a}\mathrm{t}\mathrm{o}\mathrm{m}\mathrm{s}\mathrm{/}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$
(polar IC) and
$200\text{\,}\mathrm{a}\mathrm{t}\mathrm{o}\mathrm{m}\mathrm{s}\mathrm{/}\mathrm{\SIUnitSymbolMicro}\mathrm{m}$
(rotation IC). After the initial build-up of transverse spin up to
$t=$17\text{\,}\mathrm{s}$$ the average spin length
$\langle|F_{\perp}|\rangle_{x}$ stays approximately constant for both ICs.

Local Spin Rotations
For the rotation IC 6 local spin rotations are successively applied along the
cloud within a total duration of $<$1\text{\,}\mathrm{m}\mathrm{s}$$, which is
much faster than the spin dynamics (characteristic spin time scale
$\gtrsim$600\text{\,}\mathrm{m}\mathrm{s}$$). The central rotation angle of
each spin rotation is tuned to be larger than $\pi/2$, such that phase jumps
are generated in the $m_{\text{F}}=0$ component, as described in [39]. Due to
the additional spin dynamics induced by the quench of $q$ the excitations
discussed in [39] are presumably not stable in this system. All rotations are
performed phase-coherently around the same rotation axis.

Transverse Spin Readout
To extract the transverse spin field $F_{\perp}=F_{x}+iF_{y}$ with spatial
resolution a simultaneous readout of both transverse projections is applied in
conjunction with Stern-Gerlach (SG) absorption imaging as detailed in [47].
Additionally, at half the time between the two global $\pi/2$ spin rotations,
used to map the transverse observables to the projection direction of the SG
gradient, a spin-echo $\pi$-pulse is inserted to mitigate effects of magnetic
offset field drifts to the readout. In the analysis of the pictures three
adjacent pixels are binned together to a size of
$1.2\text{\,}\mathrm{\SIUnitSymbolMicro m}$, which also corresponds to the
resolution of our imaging setup [50]. In order to exclude edge effects from
the varying total density only the central
$\sim$105\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ of the atom cloud are
evaluated.

Estimation of the Quench Energy
A numerical comparison of the energies between the ICs shows that the local
spin rotations of the rotation IC induce an initial energy offset of approx.
$k_{\text{B}}\times$210\text{\,}\mathrm{p}\mathrm{K}$$, where $k_{\text{B}}$
denotes Boltzmann’s constant. The additional energy input from the quench
across the polar to easy-plane phase transition is on the order of
$n|c_{1}|\sim k_{\text{B}}\times$120\text{\,}\mathrm{p}\mathrm{K}$$ (polar IC)
and $n|c_{1}|\sim k_{\text{B}}\times$30\text{\,}\mathrm{p}\mathrm{K}$$
(rotation IC) for the initial densities. We estimate the experimental spin-
interaction energy $n|c_{1}|$ by observing the density-dependence of the spin-
changing-collision instability. Zero-temperature Bogoliubov instabilities
occur below $q_{\text{c}}=2n|c_{1}|$ for a quench of the polar state. From the
measured atom numbers we estimate a lower limit of the critical temperature
$T_{\text{c}}$ of Bose-Einstein condensation [51] and obtain
$T_{\text{c}}\approx$60\text{\,}\mathrm{n}\mathrm{K}$$ for the atom number of
the polar IC and $T_{\text{c}}\approx$40\text{\,}\mathrm{n}\mathrm{K}$$ for
the rotation IC at $t=$42\text{\,}\mathrm{s}$$. Previously, we have shown that
the system can thermalize to a temperature of
$\sim$3\text{\,}\mathrm{n}\mathrm{K}$$ [46]. Therefore, the total temperatures
in both systems remain well below the estimated critical temperature of $\sim
T_{\text{c}}/4\gg n|c_{1}|$ for the polar to easy-plane phase transition at
$q\sim n|c_{1}|$ [45]. Hence, the equilibrium state, which is expected to be
reached, is, for both ICs, in the easy-plane phase, where
$\langle|F_{\perp}|\rangle>0$.

###  A.2 Extraction of Scaling Exponents

As outlined in the main text the scaling exponents are extracted in two
stages. The first model-agnostic approach does not make assumptions about the
scaling function but enables a first extraction of the scaling exponents.
These values are then used to obtain a re-scaled data set which allows the
determination of the scaling function with high fidelity. Based on this a
model of the spectra is constructed and it facilitates a standard $\chi^{2}$
analysis based on the scaling hypothesis together with the scaling function.

Scaling $\bm{k}$-Range
Before determining the scaling exponents it is imperative to select the
momentum range over which to evaluate re-scaling of the structure factor. This
selection process is based on the expected physical process of the transport
of excitations to lower $k$ and transport of energy to larger $k$ occurring
during the period of self-similar dynamics in the spin-1 system [31]. Here,
two self-similar regimes have been identified [48, 35], where exponents are
positive for the infra-red (IR) fixed point dynamics at low $k$ and negative
for the ultra-violet fixed point at large $k$. Therefore, the $k$-region
evaluated for each self-similar process should extend no more than to half the
distance between the IR and UV regions marked by the crossing points of the
spectra measured at different times to avoid systematic distortion of the
extracted scaling exponents. In order not to affect the gradient of the re-
scaled spectra at large k by this, we limit the evaluation ranges in $k$-space
(gray shaded regions in Figs. 3(a,d)) to the values

$\displaystyle 1/\lambda$
$\displaystyle=$0.0090.038\text{\,}{\mathrm{\SIUnitSymbolMicro
m}}^{-1}$\;\;\text{(polar IC)},$ $\displaystyle 1/\lambda$
$\displaystyle=$0.0090.047\text{\,}{\mathrm{\SIUnitSymbolMicro
m}}^{-1}$\;\;\text{(rotation IC)}.$

In this publication we are only evaluating the IR fixed points experimentally.
See Fig. S2 for the full $k$-range of the recorded spectra.

Figure S2:  Full structure factors for the polar IC (a) and rotation IC (b).
The data points to the left of the gray shaded region indicate the amplitude
of the $k=0$ values.

Error Estimation of the Scaling Exponents
For the extracted values of the scaling exponents provided in the main text
and in Tabs. SI–SIII both statistical and systematic errors are specified.
After calculating the likelihood functions shown in Figs. 3(c,f) or S3 the
statistical errors are obtained from Gaussian fits to the corresponding
marginal likelihood functions. Similarly, the gray ellipses in Figs. 3(c,f)
and S3 are obtained as $n\times\sigma$ intervals extracted from Gaussian fits
to the marginal likelihood distributions along rotated axes in
$(\alpha,\beta)$-space. The systematic error specifies the change of the
exponent values when changing the upper limit of the scaling $k$-range
specified above to the neighbouring values.

####  A.2.1 Model-Agnostic Re-Scaling Analysis

The approach outlined here resembles a modified version of the method applied
in [32]. For optimizing the scaling exponents without assumptions about the
scaling function we use the data measured at an arbitrarily selected reference
time $t_{\text{r}}$ as ”model” for the remaining measured data. Applying Eq.
(1) to the data taken at $t_{\text{r}}$ (where $f_{\text{s}}$ indicates this
measured data) this data is re-scaled to the data at $t\neq t_{\text{r}}$.
This process requires interpolating the reference data at $t_{\text{r}}$ to
the re-scaled $k$ values at $(t/t_{\text{r}})^{\beta}k$. For this task we
perform a piece-wise power law interpolation (i.e. a linear interpolation in a
bi-logarithmic plot). Based on this approach we define a $\chi^{2}$ distance
measure between the measured data based on the scaling hypothesis (similar to
[52, 32]) based on the standard errors $\sigma(t,k)$ of the measured mean
values of the structure factors $f(t,k)$:

$\chi^{2}_{\text{nofct}}(\alpha,\beta)=\frac{1}{N_{\text{ref}}}\sum_{t_{\text{r}}}\sum_{t,k}\frac{\left\\{(t/t_{\text{r}})^{-\alpha}f[t,(t/t_{\text{r}})^{\beta}k]-f(t_{\text{r}},k)\right\\}^{2}}{\left\\{(t/t_{\text{r}})^{-\alpha}\sigma[t,(t/t_{\text{r}})^{\beta}k]\right\\}^{2}+\sigma(t_{\text{r}},k)^{2}}\,.$
(S2)

Note that experimental data is described by the ”function” notation, e.g.
$f(t,k)$ for improved readability (main text: $f_{t,k}$). To remove a possible
bias from the particular choice of $t_{\text{r}}$ we average the $\chi^{2}$
measure over all $N_{\text{ref}}=6$ reference times (first sum in Eq. (S2) in
the scaling regime (times $t$ shown in Fig. 3(a,b,d,e)). Fig. S3 shows the
corresponding normalized likelihood function

$L(\alpha,\beta)\propto\exp\left[-\chi^{2}(\alpha,\beta)/2\right]$ (S3)

with $\chi^{2}=\chi^{2}_{\text{nofct}}$.

Figure S3:  Likelihood functions (based on Eqs. (S2) and (S3) for the model-
agnostic optimization of the scaling exponents for the polar IC (a) and the
rotation IC (b). The black (for polar IC) and gray (for rotation IC) ellipses
indicate the $2\sigma$ and $5\sigma$ statistical error ranges.

Its minimization over $\alpha$ and $\beta$ yields the optimal exponents given
in Tab. SI. The table also includes the average number $d$ of measured data
points for all $t\neq t_{\text{r}}$ which still lie in the scaling $k$-range
after re-scaling and interpolation.

Initial Condition | $\bm{\alpha}$ | $\bm{\beta}$ |  minimal $\bm{\chi^{2}_{\text{nofct}}}$ |  # Points $\bm{\tilde{d}}$ |  $\bm{\chi_{\text{red}}^{2}=\chi_{\text{nofct}}^{2}/\nu}$
---|---|---|---|---|---
Polar IC |  $0.65\pm 0.30_{\,\text{stat}}\pm 0.48_{\,\text{sys}}$ |  $0.58\pm 0.13_{\,\text{stat}}\pm 0.24_{\,\text{sys}}$ |  $25.4$ | 135 |  $1.24$
Rotation IC | $0.22\pm 0.08_{\,\text{stat}}\pm 0.02_{\,\text{sys}}$ | $0.25\pm 0.05_{\,\text{stat}}\pm 0.02_{\,\text{sys}}$ | $25.3$ | 165 | $0.99$
Table SI: Results of the model-agnostic likelihood maximization for the data
given in Fig. S3. Here, the number of points $\tilde{d}$ includes data across
all $N_{\text{ref}}$ reference times, such that $d=\tilde{d}/N_{\text{ref}}$.
By accounting for the two scaling exponents, the number of degrees of freedom
is estimated to be $\nu=d-2$.

####  A.2.2 Re-Scaling Analysis with Scaling Function

From the re-scaled data set, obtained with the exponents from Tab. SI, we
extract the parameters of the scaling function $f_{\text{s}}$, as provided in
the main text, from a fit in the scaling $k$-range, weighted with the scaled
and interpolated errors
$1/[(t/t_{\text{r}})^{-\alpha}\sigma(t,(t/t_{\text{r}})^{\beta}k)]^{2}$.

Initial Condition | $\bm{\alpha}$ | $\bm{\beta}$ |  minimal $\bm{\chi^{2}}$ |  # Points $\bm{d}$ |  $\bm{\chi_{\text{red}}^{2}=\chi^{2}/\nu}$
---|---|---|---|---|---
Polar IC |  $0.64\pm 0.09_{\,\text{stat}}\pm 0.50_{\,\text{sys}}$ |  $0.58\pm 0.04_{\,\text{stat}}\pm 0.26_{\,\text{sys}}$ |  $29.5$ | 24 | $1.55$
Rotation IC | $0.24\pm 0.04_{\,\text{stat}}\pm 0.03_{\,\text{sys}}$ | $0.28\pm 0.04_{\,\text{stat}}\pm 0.05_{\,\text{sys}}$ | $42.9$ | 30 | $1.72$
Table SII: Results of the likelihood maximization based on the scaling
function and hypothesis as model for the data given in Fig. 3. The $p$-values
are estimated assuming a $\chi^{2}$ distribution of $\nu=d-5$ degrees of
freedom and $\gamma$ specifies the angle of the semi-major axis of the
likelihood distribution in Fig. 3(c,f) with respect to the horizontal axis.

Using this scaling function and the scaling hypothesis as model to define
$\chi^{2}$ and applying a maximum likelihood optimization as described in the
main text we obtain the values provided in Tab. SII. Because no interpolation
is involved in the calculation of $\chi^{2}$ in this case the total number $d$
of measured data points across all times in the scaling regime stays constant.
As the number of degrees of freedom we set $\nu=d-5$ to account for the two
re-scaling exponents and amplitude, scale $\lambda_{\text{s}}$ and power law
exponent $\zeta$ of the scaling function as model parameters.

####  A.2.3 Re-Scaling Analysis under the Constraint $\alpha=\beta$

The finite size of atom cloud limits the infra-red resolution of the structure
factor in the experiment, which induces a large uncertainty in primarily the
exponent $\alpha$. From earlier investigations of the polar IC one expects a
transport of conserved $F_{\perp}$ excitations near the NTFP. This implies a
conservation of the integral of the structure factor, which fixes
$\alpha=\beta$ in a one-dimensional system. Therefore, we are also performing
the re-scaling analysis described above under this constraint. Instead of
evaluating the two-dimensional distributions in the $\alpha$-$\beta$-plane
(cf. S3), here only the intersection of the likelihood $L$ along the diagonal
is evaluated. The results are given in Tab. SIII.

Initial Condition + Evaluation |  Exponent $\bm{\alpha=\beta}$ |  minimal $\bm{\chi^{2}}$ |  # Points |  $\bm{\chi_{\text{red}}^{2}=\chi^{2}/\nu}$
---|---|---|---|---
Polar IC (model-agnostic) |  $0.53\pm 0.04_{\,\text{stat}}\pm 0.04_{\,\text{sys}}$ |  $\chi^{2}_{\text{nofct}}=$25.4$$ | $\tilde{d}=135$ | $1.24$
Polar IC | $0.54\pm 0.02_{\,\text{stat}}\pm 0.05_{\,\text{sys}}$ | $\chi^{2}=$29.0$$ | $d=24$ | $1.53$
Rotation IC (model-agnostic) | $0.24\pm 0.08_{\,\text{stat}}\pm 0.01_{\,\text{sys}}$ | $\chi^{2}_{\text{nofct}}=$24.6$$ | $\tilde{d}=165$ | $0.96$
Rotation IC | $0.28\pm 0.03_{\,\text{stat}}\pm 0.04_{\,\text{sys}}$ | $\chi^{2}=$44.1$$ | $d=30$ | $1.76$
Table SIII: Results of the likelihood maximization under the constraint
$\alpha=\beta$. Refer to Tabs. SI and SII for the definitions of the
quantities given here. Due to the constraint on the exponents, the number of
degrees of freedom is $\nu=d-4$ for the evaluation based on the scaling
function. The fit of the scaling function under the constraint $\alpha=\beta$
results in the same parameters as given in the main text.

###  A.3 Numerical Simulations

We compare the experimental data to Truncated-Wigner (TW) simulations of a
homogeneous one-dimensional spinor gas, simulated by solving the spin-1 Gross-
Pitaevskii equations subject to periodic boundary conditions, similar to the
simulations performed in [49, 48]. Local spin rotations with an angle of
$2\pi/3$ are imprinted onto the initial condensate to roughly result in the
same density of excitations as in the experiment with respect to the spin-
healing length $\xi_{\text{s}}=\hbar/\sqrt{2mn|c_{1}|}$, where $\hbar$ denotes
the reduced Planck constant, $m$ the atomic mass, $n$ the density and $c_{1}$
the spin-interaction constant. To initiate dynamics the second-order shift is
quenched to $q=0.9n|c_{1}|$. The dynamical evolution is then characterized in
terms of the spin-changing-collision time scale $t_{\text{s}}=h/(n|c_{1}|)$.
We note that the simulations are performed with a much larger atom number
$\sim 3\cdot 10^{6}$ compared to the experiment, resulting in a physical
density of $n=$13\,600\text{\,}{\mathrm{\SIUnitSymbolMicro m}}^{-1}$$.

The time evolution of the spin field $F_{\perp}$, displayed in Fig. S4, shows
long-lived reductions in length associated with a jump of the Larmor phase
$\varphi_{\text{L}}$.

Figure S4:  Numerical simulation of the time evolution after a quench for the
rotation IC in the observables (a) $n_{0}/n$, (b) $|F_{\perp}|$, (c) the phase
$\varphi_{0}$ of the $m_{\text{F}}=0$ field and (d) $\varphi_{\text{L}}$. The
data shows a classically propagated single realization of the full TW
simulation. Around $x\approx$312\text{\,}\xi_{\text{s}}$$ long-lived density
depletion in the $m_{\text{F}}=0$ component is observed to propagate through
the condensate, accompanied by a phase jump of $\approx\pi$. This excitation
is also seen in the transverse spin degree of freedom.

Upon evaluation of the transverse-spin phase-amplitude excitations, the
numerical simulations show qualitative agreement with the experimental results
provided in Fig. 4. The corresponding numerical data is shown in Fig. S5,
where the data for the polar IC is the same as presented in [49].

Figure S5:  Comparison of experimental data with the numerically extracted
structure of spin excitations from TW simulations analogous to Fig. 4. (a) An
excerpt of a single-time snapshot of the TW run shown in Fig. S4 at time
$t=18t_{\mathrm{s}}$ and $x\in[300,320]\xi_{\mathrm{s}}$. The lower panel
shows the correlator density as defined in Fig. 4. Events which are larger
than a standard deviation of the data are marked by blue lines. (b) Evaluation
of the cross-correlator amplitudes $\mathscr{C}(\Delta x=0,t)$ on a double
logarithmic scale for the numerical simulation for polar and rotation ICs. The
spatial profiles shown in the insets display the same behavior as the
experimental data. (c) For comparison, the experimentally extracted correlator
amplitudes $\mathscr{C}(\Delta x=0,t)$ of Fig. 4(b) are shown double-
logarithmically, along with the profiles $\mathscr{C}(\Delta x,t)$. Error bars
indicate 1 s.d. of the mean.

A numerical analysis of the transverse-spin power spectra results in scaling
exponents $\alpha=0.25\pm 0.04$ and $\beta=0.26\pm 0.04$ (see Fig. S6), hence
agreeing with experimental results within the error bounds. By imprinting
local spin rotations, overoccupied modes in the transverse spin are enhanced
by Bogoliubov instabilities and redistribute to form a time-independent
scaling function $f_{\mathrm{s}}$ with a large plateau. After around
$30t_{\mathrm{s}}$, the self-similar scaling regime is reached.

Figure S6:  Self-similar scaling of the structure factor
$f(k,t)=\langle|\mathrm{dFT}\left[|F_{\perp}(x,t)|\right](k)|^{2}\rangle$ for
the numerical simulations of the rotation IC. (a) Initial condition and short-
time dynamics after imprinting an ensemble of 80 rotations onto a condensate
of length $L=$220\text{\,}\mathrm{\SIUnitSymbolMicro m}$$ and quenching the
quadratic Zeeman shift to $q=0.9n|c_{1}|$. The build-up of structure through
instabilities and redistribution of momentum leads to the formation of the
universal scaling function. (b) Unscaled spectra in the self-similar scaling
regime. We extract the power law $\zeta$ via a least-squares fit of the
function $f_{\mathrm{s}}$ and find $\zeta=2.002\pm 0.005$. (c) Inverse
residuals $1/\Sigma^{2}$ of the rescaling procedure computed according to the
prescription given in [7]. (d) Rescaled spectra of the rotation IC shown in
panel (b). The numerical analysis finds scaling exponents $\alpha=0.25\pm
0.04$ and $\beta=0.26\pm 0.04$. The inset shows the residuals after the
rescaling.


# Analysis of Deep Image Quality Models

Pablo Hernández-Cámara
Image Processing Lab
Universitat de València, Spain
<EMAIL_ADDRESS>Jorge Vila-Tomas
<EMAIL_ADDRESS>Valero Laparra
<EMAIL_ADDRESS>Jesús Malo
<EMAIL_ADDRESS>
###### Abstract

Subjective image quality measures based on deep neural networks are very
related to models of visual neuroscience. This connection benefits engineering
but, more interestingly, the freedom to optimize deep networks in different
ways, make them an excellent tool to explore the principles behind visual
perception (both human and artificial). Recently, a myriad of networks have
been successfully optimized for many interesting visual tasks. Although these
nets were not specifically designed to predict image quality nor other
psychophysics, they have shown surprising human-like behavior. The reasons for
this remain unclear.

In this work, we perform a thorough analysis of the perceptual properties of
pre-trained nets (in particular their ability to predict image quality) by
isolating different factors: the goal (the function), the data (the learning
environment), the architecture, and the readout: selected layer(s), fine-
tuning of channel relevance, and use of statistical descriptors as opposed to
plain readout of responses.

Several conclusions can be drawn. All the studied models correlate better with
human opinion than SSIM (a _de-facto_ standard). More importantly, some of the
nets are in pair of the state-of-the-art with no extra refinement nor
perceptual information. Nets trained for supervised tasks such as
classification correlate substantially better with humans than LPIPS (a recent
net specifically tuned for image quality). Interestingly self-supervised tasks
such as jigsaw also perform better than LPIPS. Simpler architectures are
better than very deep nets. In simpler nets, correlation with humans increases
with depth as if deeper layers were closer to human judgement. This is not
true in very deep nets. Consistently with reports on illusions and contrast
sensitivity, small changes in the image environment does not make a big
difference in performance. Finally, the explored statistical descriptors and
concatenations had no major impact.

##  1 Introduction

Humans have evolved to adapt our visual system to the statistics of nature.
Neural networks learn the statistics of the data they have been trained on
[9]. However, how human perception and machine learning models trained with
natural images are related is an open issue. There is a long tradition on
relating human visual system with machine learning models and there are many
options to relate them. One option is trying to reproduce fundamental
signatures of the visual system that have been directly measured, such us
contrast sensitivity functions (CSF) [15, 1]. Other option that we explore
here is the image quality assessment (IQA) problem, which tries to predict
image distances in a similar way to what a human would do.

Architecture | Goal | Training data | Read-out | Fine tuning
---|---|---|---|---
AlexNet | Supervised | ImageNet-1K | Euclidean, no concat. | No
VGG-16
ResNet-50
DenseNet-121
EfficientNet-B0
AlexNet | Supervised | ImageNet-1K | Euclidean, no concat. | No
Self-Supervised RotNet
Self-Supervised Jigsaw
Self-Supervised Colorization
Self-Supervised DeepCluster
AlexNet | Supervised | ImageNet-1K | Euclidean, no concat. | No
Places-365
Cifar-10
AlexNet | Supervised | ImageNet-1K | Euclidean, no concat. | No
Euclidean, concat.
Means, no concat.
Means, concat.
Means-sigmas, no concat.
Means-sigmas, concat.
Gram, no concat.
Gram, concat.
AlexNet | Supervised | ImageNet-1K | Euclidean | No
TID-2008
train-KADID-10K
Table 1: Resume of the factors analyzed. Each column show the different model
factors that we have varied and each row shows the different options explored.

A classical tradition to predict human perception is to use biologically
plausible models to define the quality metrics [8]. However, this is not
mandatory and it is possible to approach the problem from other perspectives.
For many years, statistical based quality methods have been widely used and
awarded [24]. In the last decade, features extracted from deep learning models
have been used to calculate the distances without taking care of the
biological plausibility of the architecture (LPIPS [26] and DISTS [5]). These
models use convolutional feature extractors (pre-trained to perform
classification) as a backbone to define image quality metrics in their inner
domain and perform a fine-tune with few perceptual data to maximize the
correlation with human perception. Results from these models correlate
surprisingly well with human perception and they have become state-of-the-art
in image quality assessment. However, the reason why these models designed for
other tasks correlate so well with human perception remains unclear.

In recent years, some studies tried to analyze different aspects of this
relation between machine learning models and human perception [9, 15, 1, 14].
They found, not only that low level tasks have better properties than high
level tasks for reproduce human-CSF, but also that models with higher accuracy
on ImageNet classification perform worse when correlating with human
perception. However, there are still many open issues to clarify these
behaviors.

In this work, we do an analysis of the perceptual properties of already
trained models when facing image quality problems. We analyze how different
factors such as architecture, objective, data, and ways of computing the
distance (output deepness, output concatenation, channel relevance fine-
tuning, and using statistics) affect the correlation between human perception
and model distances. Table 1 shows a summary of the different tested factors.
To the best of our knowledge, this is the first work that analyzes how well
the features extracted (layer by layer) correlate with human perception.

##  2 Related work

Since its appearance in 2004, SSIM became the standard model for predicting
image distances as close to humans as possible [24]. It is based on
calculating statistical descriptors of image patches to compare them an obtain
a distance measurement. However, SSIM recently was surpassed by models based
on pre-trained artificial neural networks. For example, LPIPS [26]
concatenates features of different VGG layers. Once an image and its distorted
version have passed through the network and the outputs have been read, each
feature is weighted to maximize the correlation with human perception. Even
fewer years ago, in 2020, DISTS [5] unified both SSIM and LPIPS. It uses
different layers of VGG to extract features from the images, and computes
statistical descriptors as in SSIM to compare them in order to get a distance
measurement between an image and its distorted version.

These IQA models based on neural networks correlate surprisingly well with
human perception, although the reason is not known. In the last years, some
works have tried to explore and understand this relation between deep learning
models and human perception. On the one hand, some studies studied fundamental
signatures of human visual system to check if deep learning models also have
the same signatures. They show that human-CSFs appear more in low/middle-level
tasks than in high-level objectives [1, 15]. They also show that depending on
the training objective, CSFs appear more in some layers than in others. This
shows that, as expected, neither all training targets nor all layers of deep
learning models have the same properties, and that some of them are more
closely related to human perception than others.

On the other hand, other works explore the relation between deep learning
models classification accuracy and human perception. They show that there is
an inverse-V relation between ImageNet accuracy and correlation with human
perception in IQA problems [14], where models with high and very low
accuracies have lower correlation with humans.

##  3 Method

Here we describe the databases and models we used in our experiments and the
different ways we employed to calculate distances between images in the inner
representation of a certain model.

###  3.1 Databases and models

We restrict ourselves to already trained models by third-party people in order
to avoid dependence on architecture design or training procedures. Following
this, we used different pre-trained models, trained on ImageNet [4],
Places-365 [28] and Cifar-10 [12] databases, in supervised or self-supervised
ways. More particularly, for the supervised models, we used AlexNet [13],
VGG-16 [21], DenseNet-121 [10], ResNet-50 [7] and EfficientNet-B0 [22]. We
downloaded AlexNet and EfficientNet-B0 from TorchVision [17] and VGG-16,
DenseNet-121 and ResNet-50 from Keras [3]. We downloaded all of them with
ImageNet pre-trained weights. We also downloaded the supervised AlexNet model
pre-trained in Places-365 from MIT CSAIL Computer Vision [27] and pre-trained
in Cifar-10 from [20]. For the self-supervised models, we select one
architecture (AlexNet) and use different self-supervised training goals with
ImageNet images. We tested RotNet [11], Jigsaw [18], Colorization [25] and
DeepCluster [2] tasks. We downloaded all the self-supervised models from
Facebook research VISSL [6]. Table 2 shows a resume of the supervised and
self-supervised models trained with ImageNet-1K data and their ImageNet-1k Top
1 accuracy.

Architecture | Training process | ImageNet Top 1
---|---|---
AlexNet | Supervised | 56.5%
VGG-16 | Supervised | 71.3%
DenseNet-121 | Supervised | 75.0%
ResNet-50 | Supervised | 74.9%
EfficientNet-B0 | Supervised | 77.7%
AlexNet | Self RotNet | 39.5%
AlexNet | Self Jigsaw | 34.8%
AlexNet | Self Colorization | 30.4%
AlexNet | Self DeepCluster | 37.9%
Table 2: Summary of the tested models, their training process and their
ImageNet Top 1 accuracy.

To test the models described previously we used two image quality databases:
TID-2013 [19] and KADID-10K [16]. Both consists on pairs of images and
distorted versions of the same image with a mean opinion score (MOS) for each
image - distorted image pair, which represents the distance between them as
estimated by humans. More particularly, we used the whole TID-2013 (3000 image
pairs) and $30\%$ of KADID-10K, which we call val-KADID-10K (3038 image
pairs), for testing the different models.

We also tested weighting the features extracted by the models (what we call
fine-tuning), similarly to LPIPS. We used TID-2008 (1700 image pairs) and the
remaining $70\%$ of KADID-10K (train-KADID-10K) (7087 image pairs) to weight
each output feature in order to maximize the correlation with the MOS in these
training databases.

###  3.2 Distance measurement and correlation

There are several ways to measure the distance between two images. For,
example, let $x_{0}\in\mathbb{R}^{(H,W,C)}$ and $y_{0}\in\mathbb{R}^{(H,W,C)}$
denote an image and its distorted version, and
$\hat{x}^{l}\in\mathbb{R}^{(H^{l},W^{l},C^{l})}$ and
$\hat{y}^{l}\in\mathbb{R}^{(H^{l},W^{l},C^{l})}$ their feature maps at the
$l^{th}$ layer of a network. Then, their euclidean distance is just:

$d^{l}(x_{0},y_{0})=\sqrt{\sum_{H^{l},W^{l},C^{l}}(\hat{x}^{l}-\hat{y}^{l})^{2}}$
(1)

However, it is also possible to statistically summarize the layer output
before computing the distance. Here, we used three different ways of summarize
the layer output. First, we can use the mean of each feature:

$d^{l}_{\mu}(x_{0},y_{0})=\sqrt{\sum_{C^{l}}(\hat{\mu}^{l}_{x}-\hat{\mu}^{l}_{y})^{2}}$
(2)

where $\hat{\mu}^{l}_{x}\in\mathbb{R}^{C^{l}}$ and
$\hat{\mu}^{l}_{y}\in\mathbb{R}^{C^{l}}$ are the spatial average of the
outputs of the $l^{th}$ layer for the image $x_{0}$ and its distorted version
$y_{0}$. In this case, we compute the spatial averages to calculate the
distance with them. Second, we can use not only the spatial averages but also
the spatial standard deviations. In this case, we concatenate the spatial
averages and standard deviation before computing the distance:

$d^{l}_{\mu,\sigma}(x_{0},y_{0})=\sqrt{\sum_{C^{l}}(concat(\hat{\mu}^{l}_{x},\hat{\sigma}^{l}_{x})-concat(\hat{\mu}^{l}_{y},\hat{\sigma}^{l}_{y}))^{2}}$
(3)

where $\hat{\mu}^{l}_{x}\in\mathbb{R}^{C^{l}}$,
$\hat{\mu}^{l}_{y}\in\mathbb{R}^{C^{l}}$ and
$\hat{\sigma}^{l}_{x}\in\mathbb{R}^{C^{l}}$,
$\hat{\sigma}^{l}_{y}\in\mathbb{R}^{C^{l}}$ are the spatial average and
standard deviation of the outputs of the $l^{th}$ layer for the image $x_{0}$
and its distorted version $y_{0}$ respectively. Finally, we can summarize the
outputs through their Gram matrix:

$d^{l}_{G}(x_{0},y_{0})=\sqrt{\sum_{C^{l}}(\hat{G}^{l}_{x}-\hat{G}^{l}_{y})^{2}}$
(4)

where $\hat{G}^{l}_{x}\in\mathbb{R}^{(C^{l},C^{l})}$ and
$\hat{G}^{l}_{y}\in\mathbb{R}^{(C^{l},C^{l})}$ are the Gram matrices of the
outputs at the $l^{th}$ layer for the image $x_{0}$ and its distorted version
$y_{0}$.

Inspired by some of the SOTA image quality models [26], one can also
concatenate the outputs of different layers in order to introduce more
information to calculate the distance. Not only that, it is also possible to
weight (fine-tune) the output features of a model (concatenating or not) so
that the correlation with a specific database is maximized.

Summing up, to obtain the results we pass each image - distorted image pair
through the different models and record the outputs at different layers. Then,
we calculate the distance between them at the different layers using one of
the distance definitions from above ($d^{l}$-$d^{l}_{\mu}$-$d^{l}_{G}$;
concatenating or not the outputs of different layers; weighting or not the
outputs). Finally, we calculate the Spearman correlation between the distance
of the model at different layers with the experimental MOS.

##  4 Experiments and results

###  4.1 Architecture

In our first experiment, we tested how different architectures (AlexNet,
VGG-16, ResNet-50, DenseNet-121 and EfficientNet-B0) correlate with human
perception. Here we fixed the goal of the models (ImageNet supervised
classification task), the training data (ImageNet-1K), the way we perform the
read-out to calculate the distance (euclidean: $d^{l}(x_{0},y_{0})$) and we
did not weight the output features, so we just calculate the euclidean
distance at the inner domain of each layer.

Figure 1 shows how different architectures correlate with human perception
(MOS) at different depths (different layers) for TID-2013 and val-KADID-10K.

Figure 1: TID-2013 (left) and val-KADID-10K (right) Spearman correlation at
different model depths (different layers) for different model architectures.
Note that each model has different number of layer so that to plot all
together, the x-axis represents the percentage of the network. Colors
represent the different AlexNet blocks, which achieve the higher correlation
at the second half of the network. Some published IQA models results are shown
in black (solid and dashed lines).

There are several results to notice from this figure. First, all the models
perform better than classical statistical image quality models (SSIM [24]:
black solid line). Also, simpler models (AlexNet and VGG-16) perform better
than modern image quality algorithms based on neural networks (DISTS [5],
LPIPS [26]: black dashed lines). Only some specific biologically inspired
image quality algorithms [8] perform better than the majority of the models in
TID-2013. Second, simpler models achieve higher correlations. Specifically,
AlexNet and VGG-16, which do NOT have skip connections, get more correlation
with human perception than modern models with higher ImageNet accuracies. The
fact that skip connections hurt the models is in agreement with the results of
style transfer, where it was found that using ResNet with skip connections
performs worse than networks without skip connections such as VGG [23]. Third,
in the simplest models (AlexNet and VGG-16), there is a relationship between
the depth of the layer used to measure image distances and the correlation
with perception, with higher correlations obtained in deeper layers. However,
both models show a decrease in correlation for the last two layers. More
complicated models (ResNet-50, DenseNet and EfficientNet) have much more
complex correlation diagrams with depth, not showing a clear relation between
deepness and correlation.

This suggests that image quality algorithms that used deep learning models
should use classical networks without skip connections even if they do not
achieve high accuracy in ImageNet classification.

###  4.2 Goal function

In the last years, some studies showed that human-CSF emerge more in low level
tasks models such as denoising autoencoders [15, 1]. However, what about self-
supervised tasks? Here we select the network architecture that obtained the
higher correlation in the supervised scenario, AlexNet, and check if some
self-supervised goals achieve higher correlation with human perception. To do
this, we fixed the architecture (AlexNet), the training data (ImageNet-1K),
the way we perform the read-out to calculate the distance (euclidean,
$d^{l}(x_{0},y_{0})$) and we did not weight the features, so that we just
calculate the euclidean distance in the inner domain of each layer for AlexNet
trained for different goals.

Figure 2 shows how AlexNet trained with different objectives correlates with
human perception (MOS) at different layers for TID-2013 and val-KADID-10K.

Figure 2: TID-2013 (left) and val-KADID-10K (right) Spearman correlation at
different model depths (different layers) for different goals. Colors
represent the different AlexNet blocks. Some IQA models results are shown in
black solid and dashed lines.

While all training objectives have good perceptual properties, there are some
that have better properties than others. In fact, all the models achieve
higher correlation than SSIM. We obtain that the supervised model and RotNet
model obtain the higher correlations. However, it is important to highlight
that the RotNet model was pre-trained on the supervised ImageNet task. Between
the other self-supervised tasks, Jigsaw obtains correlations at the level of
the supervised model only in its first layers but, as depth increases, the
correlation goes down. Colorization goal shows a small linear increase in
correlation with layer depth but it always has lower correlation than the
supervised one. Finally, DeepCluster model remains always almost at the same
correlation level than in the RGB (input) domain, which is just the
correlation with the RMSE.

This suggests that image quality algorithms based on deep learning models
should not use models trained for self-supervised tasks, but rather they
should use supervised or low-level tasks.

###  4.3 Training data and learning environment

We check how human perception correlates with the distances from networks
trained with different data. We fixed the architecture that obtained a better
correlation (AlexNet) and the training objective, supervised. We also keep
fixed the way we perform the read-out to calculate the distance (euclidean,
$d^{l}(x_{0},y_{0})$) and we did not weight the features. The only difference
between them is data used to train the models, and we analyze how the
correlation depends on training on ImageNet-1k, Places-365 and Cifar-10.
Figure 3 shows how different different training data correlate with human
perception (MOS) at different layers for TID-2013 and val-KADID-10K.

Figure 3: TID-2013 (left) and val-KADID-10K (right) Spearman correlation at
different model depths (different layers) for different training data. Colors
represent the different AlexNet blocks. Some IQA model results are shown in
black solid and dashed lines.

There are not big differences between the data used in the training process.
However, the best result is obtained with ImageNet-1K, which are around 1
million of natural images. Places-365 (ten million place locations images) and
Cifar-10 (50K small natural images) achieve less correlation. It implies that
not using natural images or using smaller/not enough images doesn’t hurt the
model too much.

It suggests that image quality algorithms using deep learning models should
use as natural and big images as possible, while the data quantity isn’t as
important.

###  4.4 Readout strategies and statistical descriptors

Here we tested how the way we perform the read-out affects the correlation. To
test it, we use the supervised AlexNet model trained in a supervised way with
ImageNet-1K and we calculate the correlation with human perception using the
different ways to calculate the distances in the model inner domain. We tested
not only the different distance definitions (summarizing the layer outputs
with statistics or not) but also we check what happens when we concatenate the
outputs of the three max pooling layers. Figure 4 shows how different distance
measurements correlate with human perception (MOS) at different layers for
TID-2013 and val-KADID-10K.

Figure 4: TID-2013 (left) and val-KADID-10K (right) Spearman correlation at
different model depths (different layers) for different options to calculate
the distance: just the euclidean distance with the full output, using
statistical descriptors and/or concatenating the outputs of different layers.
Colors represent the different AlexNet blocks. Some IQA model results are
shown in black solid and dashed lines.

First, figure 4 shows that the best correlation is obtained when the whole
output is used to calculate the distances, without the use of any statistical
descriptor. When statistical descriptors are used, using the spatial Gram
Matrix performs worst. Using the spatial mean together with the spatial
standard deviation lead to higher correlation than using only the spatial
means, because it implies using more information to calculate the distances.
With regard to concatenating different layer outputs (such as in LPIPS [26]
and DISTS [5]), it does not have a big effect.

This suggests that image quality algorithms that use deep learning models
should utilize the full output of the layers without using any statistical
descriptor. Also, using a concatenation of the outputs of different layers
seems to have no benefits while increasing the computational complexity.

###  4.5 Fine-tuning strategies.

Finally, we perform an analysis of what happen when we weight each feature to
maximize the correlation in some database. In this scenario we take the model
that gives the higher correlation (AlexNet) trained in a supervised way on
ImageNet-1K. We used only the euclidean read-out without any statistical
descriptor measurement with and without concatenating the layer outputs.
Figure 5 shows how different ways of weight features correlate with human
perception. More specifically, we fine-tune the outputs with TID-2008 and
train-KADID-10K and we tested at different layers with TID-2013 and val-
KADID-10K.

Figure 5: TID-2013 (left) and val-KADID-10K (right) Spearman correlation at
different model depths (different layers) for different fine-tune, i.e.
different feature weights. Colors represent the different AlexNet blocks. Some
IQA model results are shown in black solid and dashed lines.

The first conclusion is that fine-tuning with TID-2008 when evaluating in
TID-2013 results in an increase in the correlation, which makes sense because
the images are similar. It also occurs when fine-tuning with train-KADID-10K
and evaluating with val-KADID-10K. More interesting results happened when we
performed cross fine-tune: fine-tuning with train-KADID-10K gives no
substantially changes when evaluating with TID-2013. However, fine-tuning with
TID-2008 gives much worse results when evaluating in val-KADID-10K.

###  4.6 Relation between ImageNet-1K classification accuracy and perceptual
correlation

Finally, here we tried to compare the ImageNet-1K classification correlation
with the maximum correlation obtained both in TID-2013 and val-KADID-10K.
Figure 6 shows this relation between the supervised and self-supervised models
ImageNet-1K classification accuracy and the Spearman correlation.

Figure 6: TID-2013 (left) and val-KADID-10K (right) maximum Spearman
correlation relation with ImageNet-1K classification accuracy for the
different models.

It is interesting to note that there is an inverse relation, for the
supervised models, between classification accuracy and the maximum correlation
they obtained. It implies that the simpler the models, the better correlation.
However, for the self-supervised models, there is a direct relation between
classification accuracy and correlation.

##  5 Conclusion

In this work, we explore the perceptual properties of deep learning models. We
restrict ourselves to already trained models by third-party people in order to
avoid dependence on architecture design or training procedures. Following this
idea, we analyze the perceptual properties using two classical image quality
databases already accepted by the community [19] and [16].

Multiple factors are analyzed and conclusions can be raised for all of them:

  * •

Different models: All models are better than SSIM on prediction of human
correlation. Simpler models have better perceptual behavior than complex
models and are better than usual image quality metrics.

  * •

Training objectives: While all training objectives have good perceptual
properties, there are some that have better properties than others. In
particular, the best results are obtained by the model trained for
classification.

  * •

Amount of training data: Differences in the amount of data used for training
didn’t have a big effect on perceptual behavior, but training the models with
over a million big natural images seems to be beneficial.

  * •

Measure concatenating outputs or using statistical descriptor: Concatenating
outputs of different layers (as in LPIPS [26] and DISTS [5]) or just taking
the output of one layer does not have a big effect on the correlation. The use
of statistical descriptors as the channel mean, channel mean and standard
deviation or the Gram Matrix has a bad effect in the correlations.

  * •

Fine-tuning the measures: Fine-tuning the channel relevance for a particular
image quality database seems to have a negative effect when measuring
correlation in a different database. A relevant output is that even without
fine-tuning, the correlation of already trained models surpasses most image
quality metrics. In fact, they surpass LPIPS [26] and DISTS [5].

  * •

The best behavior is obtained by the fifth layer of the original AlexNet
model: trained for classification, without using fine-tuning on perceptual
data, without concatenating outputs, and without using statistics.

## References

  * [1] Arash Akbarinia, Yaniv Morgenstern, and Karl R. Gegenfurtner.  Contrast sensitivity function in deep networks.  bioRxiv, 2023.
  * [2] Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze.  Deep clustering for unsupervised learning of visual features.  In European Conference on Computer Vision, pages 132–149, 2018\.
  * [3] Francois Chollet et al.  Keras.  https://github.com/fchollet/keras, 2015.
  * [4] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009.
  * [5] Keyan Ding, Kede Ma, Shiqi Wang, and Eero P Simoncelli.  Image quality assessment: Unifying structure and texture similarity.  IEEE transactions on pattern analysis and machine intelligence, 44(5):2567–2581, 2020.
  * [6] Priya Goyal, Quentin Duval, Jeremy Reizenstein, Matthew Leavitt, Min Xu, Benjamin Lefaudeux, Mannat Singh, Vinicius Reis, Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Ishan Misra.  Vissl.  https://github.com/facebookresearch/vissl, 2021.
  * [7] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of Computer Vision and Pattern Recognition, pages 770–778, 2016.
  * [8] Alexander Hepburn, Valero Laparra, Jesús Malo, Ryan McConville, and Raul Santos-Rodriguez.  Perceptnet: A human visual system inspired neural network for estimating perceptual distance.  In 2020 IEEE International Conference on Image Processing (ICIP), pages 121–125, 2020.
  * [9] Alexander Hepburn, Valero Laparra, Raúl Santos-Rodríguez, Johannes Ballé, and Jesus Malo.  On the relation between statistical learning and perceptual distances.  In International Conference on Learning Representations, 2022.
  * [10] Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In Proceedings of the Computer Vision and Pattern Recognition, pages 4700–4708, 2017.
  * [11] Nikos Komodakis and Spyros Gidaris.  Unsupervised representation learning by predicting image rotations.  In International Conference on Learning Representations, 2018.
  * [12] Alex Krizhevsky, Geoffrey Hinton, et al.  Learning multiple layers of features from tiny images.  2009\.
  * [13] Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton.  Imagenet classification with deep convolutional neural networks.  In Proceedings of the 25th Neural Information Processing Systems, page 1097–1105, 2012.
  * [14] Manoj Kumar, Neil Houlsby, Nal Kalchbrenner, and Ekin Dogus Cubuk.  Do better imagenet classifiers assess perceptual similarity better?  Transactions of Machine Learning Research, 2022.
  * [15] Qiang Li, Alex Gomez-Villa, Marcelo Bertalmío, and Jesús Malo.  Contrast sensitivity functions in autoencoders.  Journal of Vision, 22(6):8–8, 2022.
  * [16] Hanhe Lin, Vlad Hosu, and Dietmar Saupe.  Kadid-10k: A large-scale artificially distorted iqa database.  In 2019 Tenth International Conference on Quality of Multimedia Experience (QoMEX), pages 1–3. IEEE, 2019.
  * [17] TorchVision maintainers and contributors.  Torchvision: Pytorch’s computer vision library.  https://github.com/pytorch/vision, 2016.
  * [18] Mehdi Noroozi and Paolo Favaro.  Unsupervised learning of visual representations by solving jigsaw puzzles.  In European Conference on Computer Vision, pages 69–84, 2016.
  * [19] Nikolay Ponomarenko, Lina Jin, Oleg Ieremeiev, Vladimir Lukin, Karen Egiazarian, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, and C.-C. Jay Kuo.  Image database tid2013: Peculiarities, results and perspectives.  Signal Processing: Image Communication, 30:57–77, 2015.
  * [20] Sebastian Raschka.  Deep learning models.  https://github.com/rasbt/deeplearning-models, 2023.
  * [21] Karen Simonyan and Andrew Zisserman.  Very deep convolutional networks for large-scale image recognition.  In Proceedings of the 3rd International Conference on Learning Representations, pages 1–14, 2015.
  * [22] Mingxing Tan and Quoc Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  In International Conference on Machine Learning, pages 6105–6114, 2019.
  * [23] Pei Wang, Yijun Li, and Nuno Vasconcelos.  Rethinking and improving the robustness of image style transfer.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 124–133, 2021.
  * [24] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli.  Image quality assessment: from error visibility to structural similarity.  IEEE Transactions on Image Processing, 13(4):600–612, 2004.
  * [25] Richard Zhang, Phillip Isola, and Alexei A Efros.  Colorful image colorization.  In European conference on computer vision, pages 649–666, 2016\.
  * [26] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018.
  * [27] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba.  Places: A 10 million image database for scene recognition.  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017\.
  * [28] Bolei Zhou, Agata Lapedriza, Jianxiong Xiao, Antonio Torralba, and Aude Oliva.  Learning deep features for scene recognition using places database.  In Advances in Neural Information Processing Systems, volume 27. Curran Associates, Inc., 2014.


11institutetext: Lomonosov Moscow State University, Moscow, Russia 11email:
<EMAIL_ADDRESS>
# Deep Two-Stage High-Resolution Image Inpainting

Andrey Moskalenko  Mikhail Erofeev  Dmitriy Vatolin

###### Abstract

In recent years, the field of image inpainting has developed rapidly, learning
based approaches show impressive results in the task of filling missing parts
in an image. But most deep methods are strongly tied to the resolution of the
images on which they were trained. A slight resolution increase leads to
serious artifacts and unsatisfactory filling quality. These methods are
therefore unsuitable for interactive image processing. In this article, we
propose a method that solves the problem of inpainting arbitrary-size images.
We also describe a way to better restore texture fragments in the filled area.
For this, we propose to use information from neighboring pixels by shifting
the original image in four directions. Moreover, this approach can work with
existing inpainting models, making them almost resolution independent without
the need for retraining. We also created a GIMP plugin that implements our
technique. The plugin, code, and model weights are available at
https://github.com/a-mos/High˙Resolution˙Image˙Inpainting.

###### Keywords:

Image inpainting Image restoration High-resolution Deep learning CNN

##  1 Introduction

Image inpainting is the process of realistically filling unknown or damaged
regions of an image. An inpainting algorithm receives as input a corrupted
image and a mask; its output is a restored image.

${512\times 512}$ |  |  |  |  |
---|---|---|---|---|---
${1024\times 1024}$ |  |  |  |  |
${512\times 512}$ |  |  |  |  |
${1024\times 1024}$ |  |  |  |  |
|  DFNet |  DeepFillv2 |  HiFill |  ProFill |  Ours
Figure 1: Methods outputs at different resolutions

In recent years, the progress of neural networks has led to the development of
deep inpainting methods. Neural-network methods are strongly tied to the
resolution at which they are trained, owing to the lack of receptive field.
Most models have an input size less than or equal to 512 pixels. As a result,
they are unable to handle images of arbitrary shape — for instance, those in
interactive image-processing tools. When the resolution increases, serious
artifacts appear in the models. Fig. 1 shows several examples.

In this article, we describe a method that can restore images regardless of
resolution. It uses the coarse-to-fine approach, restoring the image structure
at low resolution and the texture at high resolution. Also, to improve texture
filling, we propose using shifts of the original image that fill the hole and
that artificially expand the receptive field by the shift amount. Our approach
theoretically works for any inpainting method without retraining.

##  2 Related Work

The solution to image inpainting problem can take the classical approach of
choosing the most suitable patch [1, 2, 3] from the image and sequentially
filling the hole. Such methods are good at filling the texture component but
poor at filling the structural component. In recent years, the development of
neural networks has led to the creation of various deep inpainting methods [4,
5, 6]. Such algorithms avoid using external memory and operate only on the
basis of knowledge gained during training. They are better than classical
algorithms at restoring an image’s structural features.

Adding to the difficulty of training neural-network methods is that the
solution to the inpainting problem is nonunique. Thus, formulating the most
suitable loss function for training is difficult. Deep-neural-network features
are the best way to evaluate hole-filling quality [7].

Also, the appearance of generative adversarial networks (GANs) [8] formed the
basis for creating generative image-filling methods [9, 4, 5, 10, 11, 12],
which use adversarial loss as one component of their loss functions.

###  2.1 Gated Convolutions and Contextual Attention Module

In [10], researchers proposed replacing some of the neural network’s classical
convolutions with gated convolutions, an extension of partial convolutions
[4]. They also introduced a contextual-attention module, which is a neural-
network analog of patch-based algorithms for filling image areas. Their method
employs a GAN. Instead of a high-computational-cost contextual-attention
module, we propose common shifts as a means of filling texture.

###  2.2 Deep Fusion Network

The authors of [6] created a fusion block, which allows the network to, at its
output, alpha blend each pixel in accordance with the predicted alpha map.
They implemented the U-Net [13] architecture in a non-generative-adversarial
manner, using the high-level features of VGG16 [14] as loss functions. Our
network implements a pretrained DFNet in the first stage, yielding the
structural component in low resolution. We avoided a fusion block in our
refinement network, since it changes even the unmasked area.

###  2.3 High-Resolution Inpainting

In [11], researchers proposed modified gated convolutions that decrease the
computational complexity, reducing the number of weights. They also suggested
splitting the image at high and low frequencies by subtracting the blurry
version from the image and using the modified contextual-attention module for
aggregation. They trained their refinement network on the small but full
images. The goal of our refinement network, on the other hand, is only to
restore texture, so we train it on small patches of the original image; when
testing, we use the entire image.

##  3 Proposed Approach

###  3.1 Data Preparation

For training, we selected all images from DIV2K [15] and some from the
Internet. From each image, we cut out three random largest squares. In total,
the training sample contained 7,218 images, with an additional 1,650 for
validation. We applied to each image a random irregular mask from [4]. Our
testing used natural images with a square mask at the center.

Figure 2: Whole pipeline illustration

Figure 3: Refinement network architecture

###  3.2 Whole Pipeline

####  3.2.1 Stage one

The first stage of our algorithm restores the image structure at a low
resolution. We initially downscale the image and mask to ${512\times 512}$,
then apply the pretrained DFNet [6] to get the coarse result in low
resolution. Next we perform an upscale and replace the known region with the
corresponding region from the high-resolution image.

Finally we generate shifts of the original image in four directions: left,
right, down, up. For our experiments, the shifts were 20% of the image size.
Note that during this process, we also recount the masks and mark the pixels
in the open areas as invalid. Thus, the first stage yields a 20-channel image:
five RGB images (main plus four shifts) and five masks.

####  3.2.2 Stage two

The second stage restores the texture. To prevent the network from being
attached to the structure and to increase training efficiency, we cut out
random ${512\times 512}$ patches from the input tensor of depth 20, with the
condition that the masked area (from the main mask) is at least 10% but not
more than 90% of the patch area. Note that during testing, we skip the patch
extraction and simply transmit images in full resolution. The refinement
network’s output is a fine filled result. Fig. 3 shows the pipeline.

###  3.3 Network Architecture

The refinement-network architecture implements the U-Net [13] approach. (An
illustration appears in Fig. 3). Note that after each layer — except for the
last one — we used Batch Normalization [16]. The figure shows encoder filter
sizes; all decoder filters are ${3\times 3}$.

###  3.4 Loss Function

We trained our network in a non-generative-adversarial manner. Following [6]
as a loss function, we used a linear combination:

$\mathcal{L}=0.1\cdot\mathcal{L}_{\\_}{tv}+6.0\cdot\mathcal{L}_{\\_}{1}+0.1\cdot\mathcal{L}_{\\_}{p}+240.0\cdot\mathcal{L}_{\\_}{s}$
(1)

Where for reference image $\mathbf{I}$ and predicted $\hat{\mathbf{I}}$:

${L}_{\\_}{tv}$ — total variation distance in the masked area

${L}_{\\_}{1}$ — distance which is calculated as

$\mathcal{L}_{\\_}{1}=\frac{1}{CHW}\left\|\mathbf{I}-\hat{\mathbf{I}}\right\|_{\\_}{1}$
(2)

Where $C,W,H$ are the number of channels, width, and height respectively.

${L}_{\\_}{p},{L}_{\\_}{s}$ — Perceptual and Style Losses [17]:

$\mathcal{L}_{\\_}{p}=\sum_{\\_}{j\in
J}\left\|\psi_{\\_}{j}\left(\mathbf{I}\right)-\psi_{\\_}{j}\left(\hat{\mathbf{I}}\right)\right\|_{\\_}{1}$
(3) $\mathcal{L}_{\\_}{s}=\sum_{\\_}{j\in
J}\left\|G_{\\_}{j}\left(\mathbf{I}\right)-G_{\\_}{j}\left(\hat{\mathbf{I}}\right)\right\|_{\\_}{1}$
(4)

Where $J$ is set of indices in VGG16, $\psi_{\\_}{j}$ is $j-th$ feature layer.
And $G_{\\_}{j}$ is Gram matrix of $j-th$ feature layer.

##  4 Experiments

Our network training used the Adam [18] optimizer with default settings. It
took two days on two Nvidia Tesla P100 GPUs with a batch size of 18 images.
Note that we trained only the network from the second stage. For optimization,
we calculated the first stage’s output separately.

|  |  |  |  |
---|---|---|---|---|---
|  |  |  |  |
|  |  |  |  |
|  |  |  |  |
DFNet |  DeepFillv2 |  HiFill |  Photoshop |  ProFill |  Ours
Figure 4: Methods outputs with ${1024\times 1024}$ images

###  4.1 Comparisons

Fig. 1, Fig. 4, and Fig. 7 show examples of our work. Although the method
functions almost identically at any resolution, we limited ourselves to
${1024\times 1024}$. More pictures and resolutions appear in the repository.

$0$$1$$2$$3$$4$$5$$6$$7$DFNetDeepFill v2HiFillOursGround-
truth$1.1$$0.39$$2.98$$3.19$$6.34$ Figure 5: Subjective comparison scores
(resolution 2048x2048) $0$$1$$2$$3$$4$$5$$6$$7$DFNetDeepFill
v2ProFillHiFillOursGround-truth$2.24$$0.44$$1.61$$2.2$$3.21$$6.4$ Figure 6:
Subjective comparison scores (resolution 1024x1024)

####  4.1.1 Subjective evaluation

To conduct a subjective evaluation, we selected a set of 34 natural images
mostly from [7] with a full resolution of ${2048\times 2048}$. Participants
were shown two images and asked to choose the one with the highest visual
quality. We also added two validation questions to exclude unscrupulous
viewers. Due to the implementation limitations of the [12] we made two
separate comparisons at ${2048\times 2048}$ and ${1024\times 1024}$
resolutions. In first comparison 150 people participated, yielding 3,750 valid
votes. In second comparison 248 people participated, yielding 6200 valid
votes. Then paired estimates were converted into numerical ones using [19]
model. The results appear in Fig. 5, Fig. 6.

####  4.1.2 Objective evaluation

Due to the complexity of subjective evaluation, we only conducted objective
comparisons for other resolutions, although this may not be confirmed by
observers ratings. For the objective comparison, we also added output images
from Adobe Photoshop 2020, a commercial package that implements the classical
inpainting approach. Our quality metrics were the mean L1 distance, SSIM [20],
and PSNR. To reduce resolution, we used Nearest-Neighbor downsampling. Table 1
shows the results for this objective comparison.

Table 1: Objective comparison with different resolutions Methods Metrics | L1 | PSNR | SSIM |  | L1 | PSNR | SSIM
---|---|---|---|---|---|---|---
DeepFill v2 | 4.981 | 22.389 | 0.938 |  | 5.397 | 21.956 | 0.944
DFNet | 3.592 | 24.910 | 0.946 |  | 4.132 | 23.836 | 0.946
HiFill | 4.422 | 23.667 | 0.935 |  | 4.373 | 23.738 | 0.943
Photoshop 2020 | 4.115 | 23.789 | 0.944 |  | 13.320 | 23.756 | 0.949
ProFill | 3.906 | 24.677 | 0.950 |  | — | — | —
Ours | 3.524 | 25.175 | 0.944 |  | 3.474 | 25.311 | 0.950
Resolution | 1024x1024 |  | 1536x1536
Methods Metrics | L1 | PSNR | SSIM |  | L1 | PSNR | SSIM
DeepFill v2 | 5.546 | 21.834 | 0.946 | 5.669 | 21.697 | 0.948
DFNet | 4.345 | 23.424 | 0.948 | 4.633 | 22.915 | 0.950
HiFill | 4.403 | 23.696 | 0.944 | 4.391 | 23.710 | 0.947
Photoshop 2020 | 4.047 | 23.863 | 0.952 | 4.153 | 23.659 | 0.952
Ours | 3.447 | 25.374 | 0.952 | 3.434 | 25.412 | 0.954
Resolution | 1792x1792 | 2048x2048

###  4.2 GIMP Plugin

Inspired by [21], we embedded our method in the GNU Image Manipulation Program
(GIMP). The final plugin, which implements our method, appears in the
repository.

##  5 Conclusion

This work proposes an inpainting technique that handles images of different
sizes. We conducted both objective and subjective comparisons with existing
learning-based models and a popular commercial package, showing that our
method produces a more satisfactory result. Also, our approach can
theoretically apply to any inpainting model, making it resolution independent.
In the future, we would like to train a new model using the proposed method,
but in an end-to-end manner with dynamic shift size.

####  5.0.1 Acknowledgments

This work was partially supported by Russian Foundation for Basic Research
under Grant 19-01-00785 a. Model training for this work employed the IBM Polus
computing cluster of the Faculty of Computational Mathematics and Cybernetics
at Moscow State University.

|  |  |
---|---|---|---
|  |  |
Ours |  GT |  Ours |  GT
Figure 7: Failures of our model

## References

  * [1] Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment-based image completion. ACM Transactions on Graphics 22 (08 2003)
  * [2] Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing 13(9), 1200–1212 (2004)
  * [3] Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.: Patchmatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28 (08 2009)
  * [4] Liu, G., Reda, F.A., Shih, K.J., Wang, T.C., Tao, A., Catanzaro, B.: Image inpainting for irregular holes using partial convolutions. In: The European Conference on Computer Vision (ECCV) (2018)
  * [5] Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.S.: Generative image inpainting with contextual attention. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (Jun 2018)
  * [6] Hong, X., Xiong, P., Ji, R., Fan, H.: Deep fusion network for image completion. In: Proceedings of the 27th ACM International Conference on Multimedia. pp. 2033–2042. MM ’19, ACM, New York, NY, USA (2019)
  * [7] Molodetskikh, I., Erofeev, M., Vatolin, D.: Perceptually motivated method for image inpainting comparison (2019)
  * [8] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. ArXiv (06 2014)
  * [9] Iizuka, S., Simo-Serra, E., Ishikawa, H.: Globally and locally consistent image completion. ACM Transactions on Graphics 36, 1–14 (07 2017)
  * [10] Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., Huang, T.: Free-form image inpainting with gated convolution. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (Oct 2019)
  * [11] Yi, Z., Tang, Q., Azizi, S., Jang, D., Xu, Z.: Contextual residual aggregation for ultra high-resolution image inpainting (2020)
  * [12] Zeng, Y., Lin, Z., Yang, J., Zhang, J., Shechtman, E., Lu, H.: High-resolution image inpainting with iterative confidence feedback and guided upsampling. In: European Conference on Computer Vision. pp. 1–17. Springer (2020)
  * [13] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015 p. 234–241 (2015)
  * [14] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv 1409.1556 (09 2014)
  * [15] Timofte, R., Gu, S., Wu, J., Van Gool, L., Zhang, L., Yang, M.H., Haris, M., et al.: Ntire 2018 challenge on single image super-resolution: Methods and results. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (June 2018)
  * [16] Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015)
  * [17] Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. Lecture Notes in Computer Science p. 694–711 (2016)
  * [18] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization (2014)
  * [19] Bradley, R.A., Terry, M.E.: Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika 39(3/4), 324–345 (1952)
  * [20] Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E.: Image quality assessment: From error visibility to structural similarity. Image Processing, IEEE Transactions on 13, 600 – 612 (05 2004)
  * [21] Soman, K.: Gimp-ml: Python plugins for using computer vision models in gimp. arXiv preprint arXiv:2004.13060 (2020)