#  Model-independent test of T violation in neutrino oscillations

Thomas Schwetz  Institut für Astroteilchenphysik, Karlsruher Institut für
Technologie (KIT), 76131 Karlsruhe, Germany  Alejandro Segarra  Institut für
Theoretische Teilchenphysik, Karlsruher Institut für Technologie (KIT), 76131
Karlsruhe, Germany

(September 3, 2024)

###### Abstract

We propose a method to establish time reversal symmetry violation at future
neutrino oscillation experiments in a largely model-independent way. We
introduce a general parametrization of flavour transition probabilities which
holds under weak assumptions and covers a large class of new-physics
scenarios. This can be used to search for the presence of T-odd components in
the transition probabilities by comparing data at different baselines but at
the same neutrino energies. We show that this test can be performed already
with experiments at three different baselines and might be feasible with
experiments under preparation/consideration.

Introduction. The violation of time reversal (T) and charge-parity (CP)
symmetries are central topics in particle physics. CP violation (CPV) is one
of the necessary conditions to generate a matter-antimatter asymmetry in the
early Universe [1], and under the well founded assumption of CPT conservation,
CPV is equivalent to T violation (TV). A particularly active field is the
search for CPV in neutrino oscillations [2, 3, 4]. Unfortunately, the
experimental signature is rather indirect, and it is not possible to construct
model-independent CP-asymmetric observables in neutrino oscillation
experiments. This is related to the fundamental obstacle that experiments and
detectors are made out of matter (and not antimatter). Moreover, the passage
of the neutrino beam through Earth matter introduces environmental CPV due to
matter effects [5].

The standard approach to this problem is to perform a model-dependent fit to
data. This involves the assumptions that neutrino production, detection and
propagation is fully understood in terms of Standard Model (SM) interactions,
that neutrino mixing is unitary, and only the three SM neutrino flavours
exist. In this case oscillation physics can be parametrized in terms of a
unitary $3\times 3$ lepton-mixing matrix [6, 7] and two neutrino mass-squared
differences. CPV is then described by a complex phase $\delta$ in the mixing
matrix [2, 8] which can be fitted against data. “Observation of CPV” is
considered equivalent to establishing that $\delta$ is different from 0 and
$\pi$ at a certain confidence level. Within this restricted framework, current
data start to provide first indications of preferred regions for the parameter
$\delta$ [9, 10, 11, 12, 13].

Large activity is devoted to study the impact of non-standard scenarios on the
search for CPV in neutrino oscillations. Examples are non-unitary mixing [14,
15], non-standard neutrino interactions [16, 17, 18], or the presence of
sterile neutrinos [19, 20, 21]. In such new-physics scenarios, additional
complex phases appear, which can act as new sources for CP and T violation.
Typically one adopts a specific parameterization of new-physics and again
performs a parametric fit in the extended model. Our aim in this letter is to
go a step beyond such approaches and develop a largely model-independent test,
covering a wide class of non-standard scenarios. Our approach is based on
fundamental principles about TV noted in the seminal paper by N. Cabibbo [2].

Model-independent description of flavour evolution. Here we specify our
approach to describe the neutrino survival and transition probabilities
$P_{\alpha\beta}$, with $\alpha,\beta=e,\mu,\tau$. $P_{\alpha\beta}$ is the
probability for a neutrino $\nu_{\alpha}$ produced at the neutrino source to
arrive as $\nu_{\beta}$ at the detector. We adopt the following assumptions:

$(i)$ Propagation of the three SM neutrino states is described by a hermitian
Hamiltonian $H(E,x)$, which depends on neutrino energy $E$ and in general on
the matter density at the position $x$ along the neutrino path.

$(ii)$ We assume that for the experiments of interest, medium effects can be
described to sufficient accuracy by a constant matter density which is
approximately the same for all considered experiments. This is a good
approximation for experiments with baselines less than several 1000 km [22,
23]. In an accompanying paper [PRD] we show that for the baselines relevant
for our test, the effect of non-constant density is negligible.

Assumption $(ii)$ implies that the matter effect does not introduce
environmental TV by itself [24, 25] [PRD]. Therefore, any observation of TV
can be related to fundamental TV of the theory. (General discussions about TV
in neutrino oscillations can be found e.g. in [2, 26, 27, 28, 29, 30, 31, 32,
33, 34].) Furthermore, the Hamiltonian becomes position independent and we can
diagonalize it as $H(E)=W\lambda W^{\dagger}$, with $W$ being a unitary matrix
and $\lambda=(\lambda_{i})$ is a diagonal matrix of the real eigenvalues
$\lambda_{1},\lambda_{2},\lambda_{3}$ of $H$. Both $W$ and $\lambda$ depend on
the neutrino energy. Note that we allow for arbitrary non-standard matter
effects. In general, $W$ and $\lambda$ will be different for neutrinos and
antineutrinos.

$(iii)$ We allow for arbitrary (non-unitary) mixing of the energy eigenstates
$\nu_{i}$ with the flavour states $\nu_{\alpha}$ relevant for detection and
production,

$|\nu_{\alpha}\rangle=\sum_{i=1}^{3}N_{\alpha
i}^{\text{prod,det}}|\nu_{i}\rangle\,.$ (1)

We make no specific assumption on the complex coefficients $N_{\alpha i}$. In
particular, we do not relate them to the unitary matrix $W$, we allow them to
be arbitrary (sufficiently smooth) functions of energy, and they can be
different for neutrino production and detection. But we do assume that they
are the same for different experiments (at the same energy).

Below we focus on the experimentally relevant $\nu_{\mu}\to\nu_{\mu}$
disappearance and $\nu_{\mu}\to\nu_{e}$ appearance channels. Under the
assumptions $(i),(ii),(iii)$ the corresponding probabilities are obtained as
[PRD]

$\displaystyle P_{\mu\alpha}=$
$\displaystyle\left|\sum_{i=1}^{3}c^{\alpha}_{i}e^{-i\lambda_{i}L}\right|^{2}$
(2) $\displaystyle=$
$\displaystyle\sum_{i}|c^{\alpha}_{i}|^{2}+2\sum_{j<i}\real(c_{i}^{\alpha}c_{j}^{\alpha*})\cos(\omega_{ij}L)$
$\displaystyle-2\sum_{j<i}\imaginary(c_{i}^{\alpha}c_{j}^{\alpha*})\sin(\omega_{ij}L)\,,$
(3)

with $c_{i}^{\alpha}\equiv(N_{\alpha i}^{\rm det})^{*}N^{\rm prod}_{\mu i}$
and $\omega_{ij}\equiv\lambda_{j}-\lambda_{i}$. Similar expressions are
obtained in the context of non-unitary mixing, e.g., [35, 36, 37]. As usual we
have traded the time dependence in the evolution equation with the space
coordinate, leading to the appearance of the baseline $L$ in the above
probabilities. Therefore, T reversal is formally equivalent to $L\to-L$ [2,
25]. The first line of Eq. (3) is invariant under T, whereas the second line
is T-odd. Fundamental TV can be established by proving the presence of the
$L$-odd term in the probability.

For practical reasons we will introduce one more assumption. We list it here
for completeness and provide further discussion and motivation below:

$(iv)$ We impose that the oscillation frequencies $\omega_{ij}$ deviate only
weakly from the ones corresponding to the standard three-flavour oscillation
case.

Note that our assumptions $(i),(ii),(iii)$ are rather general and cover a
large class of new physics scenarios, including non-standard interactions in
production, detection, and propagation [38], generic non-unitarity [35, 36,
37], or the presence of sterile neutrinos, as long as their associated
oscillation frequencies are large compared to $\omega_{ij}$ [39].

The TV test. The strategy we propose to probe the $L$-odd terms is to measure
the oscillation probability as a function of $L$ at a fixed energy and check
whether $L$-even terms are enough to describe the data or if TV is required.
Under these conditions, the effective frequencies and mixings in the
Hamiltonian are the same, and so the data at different baselines (but at the
same energy) can be consistently combined. Notice that antineutrino data
cannot be analyzed together with neutrino data, as their effective frequencies
and mixings are in general different from the neutrino’s; nevertheless,
separate tests could be made for neutrino and antineutrino data.

In the absence of TV, all $c^{\alpha}_{i}$ are real and the data points could
be described by the $L$-even part of the oscillation probability. We define
($c^{\alpha}_{i}$ real)

$P_{\mu\alpha}^{\mathrm{even}}(L,E;\theta)=\sum_{i}(c_{i}^{\alpha})^{2}+2\sum_{j<i}c_{i}^{\alpha}c_{j}^{\alpha}\cos(\omega_{ij}L)\,.$
(4)

For the two relevant channels, these probabilities depend on 8 parameters,
which we collectively denote by $\theta$: 6 real coefficients
$c_{i}^{\mu},c_{i}^{e}$ $(i=1,2,3)$ and two independent $\omega_{ij}$, e.g.,
$\omega_{21}$ and $\omega_{31}$. We assume now that the probabilities
$P_{\mu\mu}$ and $P_{\mu e}$ are measured at a fixed energy at several
baselines $L_{b}$. We denote the corresponding measured values by
$p_{b}^{\mathrm{dis}}$ and $p_{b}^{\mathrm{app}}$ with the uncertainties
$\sigma_{b}^{\mathrm{dis}}$ and $\sigma_{b}^{\mathrm{app}}$, respectively.
Below we are going to assume that $p_{b}^{\mathrm{dis}}$ and
$p_{b}^{\mathrm{app}}$ correspond to the values predicted by standard three-
flavour neutrino (3$\nu$) oscillations in matter.

We now ask the question if we can exclude the hypothesis of T conservation
parametrised by Eq. (4), if the data correspond to $3\nu$ oscillations _with_
TV, i.e., for a CP phase $\delta$ different from 0 or $\pi$. To this aim we
construct the $\chi^{2}$ function

$\displaystyle\chi^{2}_{\mathrm{even}}(E;\theta)=$
$\displaystyle\sum_{b=1}^{N_{L}}\left[\frac{P^{\mathrm{even}}_{\mu\mu}(L_{b},E;\theta)-p_{b}^{\mathrm{dis}}}{\sigma_{b}^{\mathrm{dis}}}\right]^{2}$
$\displaystyle+\sum_{b=1}^{N_{L}}\left[\frac{P^{\mathrm{even}}_{\mu
e}(L_{b},E;\theta)-p_{b}^{\mathrm{app}}}{\sigma_{b}^{\mathrm{app}}}\right]^{2}.$
(5)

The best-fit T-conserving model is obtained by considering
$\chi^{2}_{\text{min}}(E)=\text{min}_{\theta}\left[\chi^{2}_{\mathrm{even}}(E;\theta)\right]$.
We will take the value of $\chi^{2}_{\text{min}}(E)$ as a rough indication of
how strongly T conservation can be excluded by data, and leave a more detailed
statistical analysis for future work. Considering that each baseline provides
2 data points (appearance and disappearance) and that the T-even model has 8
parameters, it is clear that we need more than 4 experiments at different
baselines. Let us note, however, that our parameterization includes so-called
zero-distance effects, due to the non-unitary mixing in Eq. (1). Therefore,
the near-detector(s) of long-baseline experiments provide already two data
points at $L\approx 0$ and effectively only more than 3 experiments are
needed.

This requirement can even be further relaxed if we impose one additional
assumption, which can be motivated by the fact that we have overwhelming
evidence that the standard three-flavour scenario is approximately correct and
any new physics effect can only be sub-leading. Therefore, we introduce
assumption $(iv)$ mentioned above: we assume that the oscillation frequency
$\omega_{21}$ deviates only weakly from the one corresponding to the standard
$3\nu$ case. Technically we impose this requirement by calculating the
effective mass-squared difference in matter, $\Delta\tilde{m}^{2}_{21}(E)$,
assuming the standard matter effect and add the following prior to Eq. (5):

$\chi^{2}_{\mathrm{even}}(E;\theta)\mapsto\chi^{2}_{\mathrm{even}}(E;\theta)+\left[\frac{\Delta\tilde{m}^{2}_{21}(E)-2E\omega_{21}}{\sigma_{21}}\right]^{2}\,.$
(6)

The frequencies $\omega_{ij}$ are determined by the effective evolution
Hamiltonian, assumption $(i)$, and are independent of the mixing in Eq. (1). A
general parameterization of the Hamiltonian is provided by the non-standard
neutrino interaction scenario, see e.g., Ref. [40]. Using the results of Ref.
[40] we estimate the possible deviation to
$\sigma_{21}=0.1\Delta\tilde{m}^{2}_{21}$, see the appendix for more details.
It turns out that the other independent frequency, $\omega_{31}$, is
effectively constrained by the long-baseline data used in our fit, and
therefore it is not necessary to impose an analogous prior for it. The prior
in Eq. (6) acts as an additional data point for each energy bin (note that
also the prior is energy dependent). Therefore, under this additional
assumption, we come to the remarkable result that our model-independent test
can be performed already with 3 experiments at different baselines plus near
detectors.

The crucial requirement, however, is sufficient overlap in neutrino energy. If
experiments have overlapping energy ranges, we can combine information from
different energies. However, to be completely model-independent, the
minimization has to be done individually for each energy, since we do not want
to make any assumptions about the energy dependence of the unknown new
physics. This is an important difference to usual model-dependent analyses.

Realistic baselines and energies. Let us now consider planned long-baseline
accelerator experiments in order to see if such a test realistically can be
carried out in the future. We consider the following experiments: the DUNE
project in USA ($L=1300$ km) [41, 42], T2HK in Japan ($L=295$ km) [43], with
the option of a second detector in Korea, T2HKK ($L=1100$ km, $1.5^{\circ}$
off axis) [44], and a long-baseline experiment at the European Spalation
Source in Sweden, ESS$\nu$SB ($L=540$ km) [45, 46].

Figure 1:  Estimated number of appearance signal events at future accelerator
experiments, assuming normal mass ordering and true $\delta=90^{\circ}$. Data
from Refs. [47, 48] (DUNE), [43] (T2HK), [44] (T2HKK), and [46] (ESS$\nu$SB).

Expected event numbers are obtained from Design Reports or detailed studies of
the physics potential and are shown for the appearance channel in the case of
$3\nu$ oscillations and $\delta=90^{\circ}$ in Fig. 1. In practice, we will
see that only the two energy bins between 0.7 and 0.9 GeV provide relevant
sensitivity, as data points with sufficient statistics are needed at 1st and
2nd oscillation maxima. We note that the energy spectrum from the NO$\nu$A
experiment [10] has no overlap with the T2K beam and therefore it cannot be
used for this analysis. We use the information from Fig. 1 (and the
corresponding data for the disappearance channel) to estimate the statistical
uncertainties in Eq. (5) as $\sigma_{br}/P^{\rm
even}(L_{b},E_{r})=\sqrt{S_{br}+B_{br}}/S_{br}$ at baseline $b$ and energy bin
$r$. We take the background events $B_{br}$ directly from the experimental
studies and estimate the number of signal events from the $N_{br}$ in the
Figure assuming $S_{br}=N_{br}\times P^{\rm
even}(L_{b},E_{r};\theta)/P^{3\nu}(L_{b},E_{r})$. For the near detector data
points, we assume the standard $P_{\alpha\beta}(L\to 0)=\delta_{\alpha\beta}$
with $\sigma=0.01$.

Figure 2:  Data points for the disappearance (top) and appearance (bottom)
channels at the baselines of DUNE, T2HK, T2HKK, ESS$\nu$SB and a near detector
location for $E=0.75$ GeV. Data points are generated for standard three-
flavour oscillations in matter with normal mass ordering and
$\delta=90^{\circ}$, and the corresponding oscillation probability is shown as
black-dashed. Error bars show $1\sigma$ statistical errors. The solid curves
show the best-fit model-independent $L$-even probabilities using all baselines
(4L, blue), DUNE + T2HK + T2HKK (3L (HKK), red), or DUNE + T2HK + ESS$\nu$SB
(3L (ESS), green). Left (right) panels are without (with) the smearing due to
a $10\%$ energy resolution.

In Fig. 2 we show the data points for the appearance and disappearance
probabilities as a function of the baseline for the 0.7–0.8 GeV energy bin. We
can see that the disappearance data points essentially fix the oscillation
frequency, whereas the appearance data are crucial for the TV test. The “true”
oscillation probability assumed to generate the data points correspond to
standard $3\nu$ oscillations with maximal TV ($\delta=90^{\circ}$) and normal
mass ordering. We find that no satisfactory $L$-even fit is possible for the
4L and 3L (HKK) combinations at this energy. The essential information is
obtained from the relative heights of the first and second appearance
oscillation peaks, see the appendix for further discussion. Note that
disappearance probabilities can reach values larger than one in our fit, since
we do not impose unitarity in our effective parameterization of the T-even
transitions.

In order to connect our test with experiments, one should take into account
the fact that finite energy resolution effectively changes the $L$-dependence
in their measurements, which will in turn affect the sensitivity of the TV
test. We assume a given energy resolution $\Delta E$ around the central bin
energy $E_{0}$, and smear the transition probability by convoluting it with a
Gaussian with mean $E_{0}$ and width $\Delta E$. To illustrate the effect we
assume here $\Delta E=0.1E_{0}$. In order to perform the convolution one must
assume a certain energy dependence of the transition probability. Our
assumption is that the energy dependence of the amplitudes $c_{i}^{\alpha}$ is
slow enough, such that it can be neglected within an interval of few $\Delta
E$. The only significant energy dependence would thus be in the oscillation
phases $\omega_{ij}$. According to assumption $(iv)$ introduced above, we
assume that $\omega_{31}\propto 1/E$, as in the standard $3\nu$ oscillation
case. We have checked that our results are independent of energy smearing of
$\omega_{21}$ terms. The impact of the finite energy resolution is illustrated
in the right panels of Figure 2.

Table 1:  Fit to data with the $\Delta m^{2}_{21}$ prior $\sigma_{21}=0.1$ in Eq. (6) assuming normal mass ordering and a true $\delta=90^{\circ}$. Units of $E$ are GeV. Columns correspond to different combinations of DUNE, T2HK, T2HKK, ESS$\nu$SB. The values outside (inside) the brackets show the $\min(\chi^{2})$ without (with) smearing the data with a $10\%$ energy resolution.  $E$ | w/o HKK | w/o DUNE | w/o ESS | all
---|---|---|---|---
0.65 | 0.07 [0.03] | 0.76 [0.65] | 0.04 [0.21] | 0.79 [0.67]
0.75 | 0.04 [0.04] | 6.95 [4.78] | 7.92 [4.82] | 8.60 [4.86]
0.85 | 0.54 [0.53] | 0.76 [2.18] | 2.75 [2.96] | 3.15 [3.06]
0.95 | - | - | 0.42 [0.98] | -
Tot. | 0.65 [0.60] | 8.46 [7.60] | 11.13 [8.97] | 12.54 [8.59]

Our results for maximal TV are summarized in Tab. 1, which shows the
$\chi^{2}_{\rm min}$ values for the various energy bins for different
experiment combinations, with and without including the energy smearing. We
observe that 0.75 GeV is the most relevant energy bin, whereas the one at 0.85
GeV still provides some sensitivity. The strong impact of the energy
resolution is apparent. We also find that the detector in Korea is essential,
whereas both DUNE and ESS provide little sensitivity but at least one of them
is needed to fix the $\omega_{ij}$ from disappearance data.

Figure 3: $\chi^{2}_{\rm min}$ summed for the energy bins around 0.75 and 0.85
GeV, with perfect (solid) or $10\%$ (dashed) energy resolution. We show the
fit to all 4 experimental baselines (4L), DUNE + T2HK + T2HKK (3L (HKK)), and
DUNE + T2HK + ESS$\nu$SB (3L (ESS)), as well as the effect in 4L of DUNE
having twice as many events (2xDUNE). Neutrino data is assumed, with normal
(inverted) mass ordering for the left (right) panel.

In Fig. 3 we show the summed $\chi^{2}_{\rm min}$ contributions from the 0.75
and 0.85 GeV bins as a function of the value of the $3\nu$ CP phase $\delta$
assumed to calculate the “data” to which the T-even model is fitted. In
addition to the features mentioned above, we see from Fig. 3 that the test is
sensitive only to $\delta\simeq 90^{\circ}$, whereas no sensitivity appears
around $270^{\circ}$. This behaviour stems from the enhancement of the second
oscillation maximum in the latter case (contrary to its suppression around
$90^{\circ}$): only when the second oscillation maximum is smaller than the
first one does the $P_{\mu e}^{\mathrm{even}}(L)$ fail to fit the data. Bins
with $E>1$ GeV are not useful in the test because of the absence of
measurements at both maxima. See the appendix for further discussion. For
illustration purpose we show in Fig. 3 the effect doubling the event numbers
in DUNE. This shows that there is significant potential to increase the
sensitivity of the test by suitable optimizations. The increased sensitivity
emerges from the 0.85 GeV bin, since at this energy the DUNE baseline is close
to the 2nd oscillation maximum.

The results for inverted mass ordering (IO) are qualitatively similar to the
one from normal ordering (for IO we show only the relevant range of $\delta$
in Fig. 3). Further details on IO are given in the appendix. If antineutrino
data are assumed (instead of neutrino data) the result is roughly obtained for
$\delta\to 2\pi-\delta$ in Fig. 3, with highest sensitivity around
$\delta\simeq 270^{\circ}$. This is to be expected, since antineutrino
oscillation probabilities are obtained from the neutrino ones by replacing
$\delta\to-\delta$ (in addition to the sign-flip of the matter potential).

Summary. We propose a largely model-independent test to search for T violation
in neutrino oscillations by comparing transition probabilities at the same
energy and different baselines. The test can be done under rather general
assumptions covering a wide range of new physics scenarios. Within some modest
assumptions, the test can be performed already with experiments at three
different baselines plus near detectors. The crucial requirements are
sufficient event numbers in the neutrino energy overlap region between the
experiments and good neutrino energy reconstruction [48, 49]. Our estimates
show that with the planned long-baseline experiments DUNE, T2HK, and T2HKK,
this test can be potentially carried out. In order to cover all T-violating
values of $\delta$, data for neutrinos and antineutrinos are necessary. We
stress that a detector at the Tokai-Korea baseline is required in addition to
DUNE and T2HK. Some optimization studies, especially in the low-energy region
of the DUNE and high-energy region of the T2HKK beams, may be required. The
results presented here warrant more detailed sensitivity studies based on
realistic experiment simulations and statistical analyses, which we leave for
future work.

###### Acknowledgements.

This project has received support from the European Union’s Horizon 2020
research and innovation programme under the Marie Sklodowska-Curie grant
agreement No 860881-HIDDeN, and from the Alexander von Humboldt Foundation.

## References

  * Sakharov [1967] A. D. Sakharov, Violation of CP Invariance, C Asymmetry, and Baryon Asymmetry of the Universe, Pisma Zh. Eksp. Teor. Fiz. 5, 32 (1967).
  * Cabibbo [1978] N. Cabibbo, Time Reversal Violation in Neutrino Oscillation, Phys. Lett. 72B, 333 (1978).
  * Bilenky  _et al._ [1980] S. M. Bilenky, J. Hosek, and S. T. Petcov, On Oscillations of Neutrinos with Dirac and Majorana Masses, Phys. Lett. 94B, 495 (1980).
  * Barger  _et al._ [1980] V. D. Barger, K. Whisnant, and R. J. N. Phillips, CP Violation in Three Neutrino Oscillations, Phys. Rev. Lett. 45, 2084 (1980).
  * Wolfenstein [1978] L. Wolfenstein, Neutrino Oscillations in Matter, Phys. Rev. D17, 2369 (1978).
  * Pontecorvo [1957] B. Pontecorvo, Mesonium and anti-mesonium, Sov. Phys. JETP 6, 429 (1957), [Zh. Eksp. Teor. Fiz. 33 (1957) 549].
  * Maki  _et al._ [1962] Z. Maki, M. Nakagawa, and S. Sakata, Remarks on the Unified Model of Elementary Particles, Prog. Theor. Phys. 28, 870 (1962).
  * Kobayashi and Maskawa [1973] M. Kobayashi and T. Maskawa, CP Violation in the Renormalizable Theory of Weak Interaction, Prog. Theor. Phys. 49, 652 (1973).
  * Abe  _et al._ [2020] K. Abe _et al._ (T2K), Constraint on the Matter–Antimatter Symmetry-Violating Phase in Neutrino Oscillations, Nature 580, 339 (2020), [Erratum: Nature 583,E 16 (2020)], arXiv:1910.03887 [hep-ex] .
  * Acero  _et al._ [2019] M. Acero _et al._ (NOvA), First Measurement of Neutrino Oscillation Parameters Using Neutrinos and Antineutrinos by Nova, Phys. Rev. Lett. 123, 151803 (2019), arXiv:1906.04907 [hep-ex] .
  * Esteban  _et al._ [2020] I. Esteban, M. C. Gonzalez-Garcia, M. Maltoni, T. Schwetz, and A. Zhou, The fate of hints: updated global analysis of three-flavor neutrino oscillations, JHEP 09, 178, arXiv:2007.14792 [hep-ph] .
  * de Salas  _et al._ [2020] P. de Salas, D. Forero, S. Gariazzo, P. Martínez-Miravé, O. Mena, C. Ternes, M. Tórtola, and J. Valle, 2020 Global Reassessment of the Neutrino Oscillation Picture, (2020), arXiv:2006.11237 [hep-ph] .
  * Capozzi  _et al._ [2020] F. Capozzi, E. Di Valentino, E. Lisi, A. Marrone, A. Melchiorri, and A. Palazzo, Addendum To: Global Constraints on Absolute Neutrino Masses and Their Ordering 10.1103/PhysRevD.101.116013 (2020), [Addendum: Phys.Rev.D 101, 116013 (2020)], arXiv:2003.08511 [hep-ph] .
  * Fernandez-Martinez  _et al._ [2007] E. Fernandez-Martinez, M. B. Gavela, J. Lopez-Pavon, and O. Yasuda, CP-violation from non-unitary leptonic mixing, Phys. Lett. B 649, 427 (2007), arXiv:hep-ph/0703098 .
  * Escrihuela  _et al._ [2017] F. J. Escrihuela, D. V. Forero, O. G. Miranda, M. Tórtola, and J. W. F. Valle, Probing CP violation with non-unitary mixing in long-baseline neutrino oscillation experiments: DUNE as a case study, New J. Phys. 19, 093005 (2017), arXiv:1612.07377 [hep-ph] .
  * Ge and Smirnov [2016] S.-F. Ge and A. Y. Smirnov, Non-standard interactions and the CP phase measurements in neutrino oscillations at low energies, JHEP 10, 138, arXiv:1607.08513 [hep-ph] .
  * de Gouvêa and Kelly [2016] A. de Gouvêa and K. J. Kelly, Non-standard Neutrino Interactions at DUNE, Nucl. Phys. B 908, 318 (2016), arXiv:1511.05562 [hep-ph] .
  * Denton  _et al._ [2021] P. B. Denton, J. Gehrlein, and R. Pestes, $CP$ -Violating Neutrino Nonstandard Interactions in Long-Baseline-Accelerator Data, Phys. Rev. Lett. 126, 051801 (2021), arXiv:2008.01110 [hep-ph] .
  * Gandhi  _et al._ [2015] R. Gandhi, B. Kayser, M. Masud, and S. Prakash, The impact of sterile neutrinos on CP measurements at long baselines, JHEP 11, 039, arXiv:1508.06275 [hep-ph] .
  * Palazzo [2016] A. Palazzo, 3-flavor and 4-flavor implications of the latest T2K and NO$\nu$A electron (anti-)neutrino appearance results, Phys. Lett. B 757, 142 (2016), arXiv:1509.03148 [hep-ph] .
  * Berryman  _et al._ [2015] J. M. Berryman, A. de Gouvêa, K. J. Kelly, and A. Kobach, Sterile neutrino at the Deep Underground Neutrino Experiment, Phys. Rev. D 92, 073012 (2015), arXiv:1507.03986 [hep-ph] .
  * Miura  _et al._ [2001] T. Miura, T. Shindou, E. Takasugi, and M. Yoshimura, The Matter Fluctuation Effect to T Violation at a Neutrino Factory, Phys. Rev. D 64, 073017 (2001), arXiv:hep-ph/0106086 .
  * Yokomakura  _et al._ [2002] H. Yokomakura, K. Kimura, and A. Takamura, Overall Feature of CP Dependence for Neutrino Oscillation Probability in Arbitrary Matter Profile, Phys. Lett. B 544, 286 (2002), arXiv:hep-ph/0207174 .
  * Krastev and Petcov [1988] P. I. Krastev and S. T. Petcov, Resonance Amplification and t Violation Effects in Three Neutrino Oscillations in the Earth, Phys. Lett. B 205, 84 (1988).
  * Akhmedov  _et al._ [2001] E. K. Akhmedov, P. Huber, M. Lindner, and T. Ohlsson, T Violation in Neutrino Oscillations in Matter, Nucl. Phys. B 608, 394 (2001), arXiv:hep-ph/0105029 .
  * Kuo and Pantaleone [1987] T.-K. Kuo and J. T. Pantaleone, $T$ Nonconservation in Three Neutrino Oscillations, Phys. Lett. B 198, 406 (1987).
  * Toshev [1989] S. Toshev, Maximal $T$ Violation in Matter, Phys. Lett. B 226, 335 (1989).
  * Arafune and Sato [1997] J. Arafune and J. Sato, CP and T violation test in neutrino oscillation, Phys. Rev. D 55, 1653 (1997), arXiv:hep-ph/9607437 .
  * Parke and Weiler [2001] S. J. Parke and T. J. Weiler, Optimizing T violating effects for neutrino oscillations in matter, Phys. Lett. B 501, 106 (2001), arXiv:hep-ph/0011247 .
  * Xing [2013] Z.-z. Xing, Leptonic Commutators and Clean T Violation in Neutrino Oscillations, Phys. Rev. D 88, 017301 (2013), arXiv:1304.7606 [hep-ph] .
  * Petcov and Zhou [2018] S. T. Petcov and Y.-L. Zhou, On Neutrino Mixing in Matter and CP and T Violation Effects in Neutrino Oscillations, Phys. Lett. B 785, 95 (2018), arXiv:1806.09112 [hep-ph] .
  * Bernabéu and Segarra [2018a] J. Bernabéu and A. Segarra, Disentangling genuine from matter-induced CP violation in neutrino oscillations, Phys. Rev. Lett. 121, 211802 (2018a), arXiv:1806.07694 [hep-ph] .
  * Bernabéu and Segarra [2018b] J. Bernabéu and A. Segarra, Signatures of the genuine and matter-induced components of the CP violation asymmetry in neutrino oscillations, JHEP 11, 063, arXiv:1807.11879 [hep-ph] .
  * Bernabéu and Segarra [2019] J. Bernabéu and A. Segarra, Do T asymmetries for neutrino oscillations in uniform matter have a CP-even component?, JHEP 03, 103, arXiv:1901.02761 [hep-ph] .
  * Antusch  _et al._ [2006] S. Antusch, C. Biggio, E. Fernandez-Martínez, M. B. Gavela, and J. Lopez-Pavon, Unitarity of the Leptonic Mixing Matrix, JHEP 10, 084, arXiv:hep-ph/0607020 .
  * Escrihuela  _et al._ [2015] F. J. Escrihuela, D. V. Forero, O. G. Miranda, M. Tortola, and J. W. F. Valle, On the description of nonunitary neutrino mixing, Phys. Rev. D 92, 053009 (2015), [Erratum: Phys.Rev.D 93, 119905 (2016)], arXiv:1503.08879 [hep-ph] .
  * Fong  _et al._ [2019] C. S. Fong, H. Minakata, and H. Nunokawa, Non-Unitary Evolution of Neutrinos in Matter and the Leptonic Unitarity Test, JHEP 02, 015, arXiv:1712.02798 [hep-ph] .
  * Falkowski  _et al._ [2020] A. Falkowski, M. González-Alonso, and Z. Tabrizi, Consistent QFT description of non-standard neutrino interactions, JHEP 11, 048, arXiv:1910.02971 [hep-ph] .
  * Blennow  _et al._ [2017] M. Blennow, P. Coloma, E. Fernandez-Martínez, J. Hernandez-Garcia, and J. Lopez-Pavon, Non-Unitarity, Sterile Neutrinos, and Non-Standard Neutrino Interactions, JHEP 04, 153, arXiv:1609.08637 [hep-ph] .
  * Esteban  _et al._ [2018] I. Esteban, M. C. Gonzalez-Garcia, M. Maltoni, I. Martinez-Soler, and J. Salvado, Updated constraints on non-standard interactions from global analysis of oscillation data, JHEP 08, 180, [Addendum: JHEP 12, 152 (2020)], arXiv:1805.04530 [hep-ph] .
  * Abi  _et al._ [2020a] B. Abi _et al._ (DUNE), Deep Underground Neutrino Experiment (Dune), Far Detector Technical Design Report, Volume I Introduction to Dune, JINST 15 (08), T08008, arXiv:2002.02967 [physics.ins-det] .
  * Abi  _et al._ [2020b] B. Abi _et al._ (DUNE), Deep Underground Neutrino Experiment (Dune), Far Detector Technical Design Report, Volume Ii: Dune Physics, (2020b), arXiv:2002.03005 [hep-ex] .
  * Abe  _et al._ [2018a] K. Abe _et al._ (Hyper-Kamiokande), Hyper-Kamiokande Design Report (2018a), arXiv:1805.04163 [physics.ins-det] .
  * Abe  _et al._ [2018b] K. Abe _et al._ (Hyper-Kamiokande), Physics potentials with the second Hyper-Kamiokande detector in Korea, PTEP 2018, 063C01 (2018b), arXiv:1611.06118 [hep-ex] .
  * Baussan  _et al._ [2014] E. Baussan _et al._ (ESSnuSB), A Very Intense Neutrino Super Beam Experiment for Leptonic CP Violation Discovery Based on the European Spallation Source Linac, Nucl. Phys. B 885, 127 (2014), arXiv:1309.7022 [hep-ex] .
  * Blennow  _et al._ [2020] M. Blennow, E. Fernandez-Martinez, T. Ota, and S. Rosauro-Alcaraz, Physics potential of the ESS$\nu$SB, Eur. Phys. J. C 80, 190 (2020), arXiv:1912.04309 [hep-ph] .
  * Acciarri  _et al._ [2015] R. Acciarri _et al._ (DUNE), Long-Baseline Neutrino Facility (LBNF) and Deep Underground Neutrino Experiment (DUNE): Conceptual Design Report, Volume 2: The Physics Program for DUNE at LBNF (2015), arXiv:1512.06148 [physics.ins-det] .
  * De Romeri  _et al._ [2016] V. De Romeri, E. Fernandez-Martinez, and M. Sorel, Neutrino oscillations at DUNE with improved energy reconstruction, JHEP 09, 030, arXiv:1607.00293 [hep-ph] .
  * Chatterjee  _et al._ [2021] S. S. Chatterjee, P. S. B. Dev, and P. A. N. Machado, Impact of Improved Energy Resolution on DUNE sensitivity to Neutrino Non-Standard Interactions, (2021), arXiv:2106.04597 [hep-ph] .

## Supplemenatry Material

###  .1 Estimation of the $\Delta m_{21}^{2}$ prior

If the neutrino data come from a theory with T violation, the $L$-even fit
converges to oscillation frequencies different from the real ones trying to
compensate the wrong functional form. These different oscillation frequencies
thus show either the presence of T violation or an underlying BSM mechanism
that shifts the effective mass-squared differences $\Delta\tilde{m}_{ij}^{2}$
from their (energy-dependent) standard values in matter.

We characterize the possible size of this second effect using the constraints
on non-standard neutrino interactions from current global oscillation data
[36]. The NSI effects can be parametrized in the matter part of the neutrino
Hamiltonian as

$H_{\mathrm{mat}}=\sqrt{2}G_{F}N_{e}(x)\matrixquantity[1+\mathcal{E}_{ee}(x)&\mathcal{E}_{e\mu}(x)&\mathcal{E}_{e\tau}(x)\\\
\mathcal{E}_{\mu e}(x)&\mathcal{E}_{\mu\mu}(x)&\mathcal{E}_{\mu\tau}(x)\\\
\mathcal{E}_{\tau
e}(x)&\mathcal{E}_{\tau\mu}(x)&\mathcal{E}_{\tau\tau}(x)]\,,$ (7)

where
$\mathcal{E}_{\alpha\beta}(x)=\epsilon_{\alpha\beta}^{e}+\epsilon_{\alpha\beta}^{p}+Y_{n}(x)\epsilon_{\alpha\beta}^{n}$,
$Y_{n}$ is the neutron-to-electron number ratio, and
$\epsilon_{\alpha\beta}^{q}$ are the non-standard couplings to electron,
protons, neutrons for $q=e,p,n$, respectively.

Considering a typical value $\epsilon_{\alpha\beta}^{q}\lesssim 0.1$ from the
global fit results in Ref. [36], we generate random values for the non-
standard couplings and diagonalize the Hamiltonian to study their effect on
the neutrino mass squared differences. We find that
$\Delta\tilde{m}_{21}^{2}(E)$ is shifted around its standard value in matter
with $\sigma=0.09$. Therefore, we conservatively choose
$\sigma_{21}(E)=10\%\Delta\tilde{m}_{21}^{2}(E)$ to describe the allowed
deviations of the smallest oscillation frequency from the SM value via the
prior term Eq. (6).

In the case of the large oscillation frequency, we find that such effect is
negligible: the T-even disappearance data always fix its best-fit point to a
value within the range allowed by NSIs.

Let us emphasize that this approach remains largely model independent. We
consider the NSI as an effective parametrization to study how much the
eigenvalues of the Hamiltonian are allowed to deviate from their standard
values. This argument does not depend on the specific new physics scenario and
is independent of the mixing coefficients in Eq. (1).

###  .2 Discussion of particular behaviours of the $L$-even fit

The consistency of the test requires that $\chi^{2}_{\rm min}=0$ in the
absence of T violation, and we checked that we indeed recover this result
whenever data are generated for $3\nu$ oscillations with $\sin\delta=0$.
However, depending on the specific values of the data points to be fitted, it
may happen that one obtains a small $\chi^{2}_{\rm min}$ even in the presence
of T violation. In this Section, we present some cases that illustrate the
typical behaviour of the $L$-even fit.

Figure 4:  Data points for the disappearance (top) and appearance (bottom)
channels at the baselines of DUNE, T2HK, T2HKK, ESS$\nu$SB and a near detector
location for neutrino energies $E=0.65$ GeV (left), 0.75 GeV (middle), and
0.85 GeV (right). Data points are generated for standard three-flavour
oscillations in matter with normal mass ordering and $\delta=90^{\circ}$, and
the corresponding oscillation probability is shown as black-dashed. Error bars
show $1\sigma$ statistical errors. The solid curves show the best-fit model-
independent $L$-even probabilities using all baselines (4L, blue), DUNE + T2HK
+ T2HKK (3L (HKK), red), or DUNE + T2HK + ESS$\nu$SB (3L (ESS), green). We
assume perfect energy resolution.  Figure 5:  Same as Fig. 4 for
$\delta=270^{\circ}$.

We show in Figure 4 the data and best-fit curves for $\delta=90^{\circ}$ at
the energies $E=0.65,0.75,0.85$ GeV. For the clarity of the following
discussion, we show the results without the smearing effect due to a finite
energy resolution; the qualitative features apply also to the case including
energy smearing. The middle panels of Figure 4 are identical to the left
panels of Figure 2. In general, the disappearance data fix the largest
oscillation frequency, whereas appearance data provide the sensitivity to TV.
The two higher-energy bins are the most sensitive ones, where the fit is
unable to describe the data from the T-violating oscillation probability. On
the other hand, we see that for $E=0.65$ GeV (left panels) the $L$-even model
can provide a reasonable fit to the 5 appearance data points, even if the
fitted function looks nothing like the true oscillation probability. It
remains an interesting question, whether such a drastic deviation from the
standard behaviour could be tested model-independently by other observations,
for instance atmospheric neutrinos. Notice also the relatively large zero-
distance effect for the fit with all baselines, which show the importance of
the near detector measurement in constraining non-standard
$P_{\alpha\beta}(L\to 0)\neq\delta_{\alpha\beta}$. Increasing the error bar of
this data point by one order of magnitude to $0.1$ decreases $\chi^{2}_{\rm
min}(0.75\leavevmode\nobreak\ {\rm GeV})$ from 7.40 to 5.05.

The analogue plots for $\delta=270^{\circ}$ are shown in Fig. 5, where one
sees that the shape of the oscillation probability is much more oscillatory-
like than for $\delta=90^{\circ}$ and the $L$-even fit can quite accurately
reproduce the true oscillation probability. This feature explains the lack of
sensitivity of the TV test in the region around the maximally T-violating
value $\sin\delta=-1$. From this Figure it becomes also clear that this
behaviour does not depend on the specific baselines used here; even many more
relatively accurate data points at different baselines would not be sensitive
to this case.

To understand why this happens, one should look into the functional form
$f(L)$ of the fitting function and the true probability. We provide here a
qualitative discussion. For simplicity we set $\theta_{23}=\pi/4$ and consider
oscillations in vacuum; the arguments remain valid also when the matter effect
is included. The $3\nu$ appearance probability is then given by

$\displaystyle P_{3\nu}$ $\displaystyle\approx
2s_{13}^{2}\sin^{2}\Delta+2s_{13}\tilde{\alpha}\Delta\sin\Delta\cos(\Delta+\delta)+\frac{1}{2}\tilde{\alpha}^{2}\Delta^{2}\,,$
(8)

where $s_{13}=\sin\theta_{13}\approx 0.15$, $\Delta=\Delta m^{2}_{31}L/4E$,
$\tilde{\alpha}=\sin 2\theta_{12}\Delta m^{2}_{21}/\Delta m^{2}_{31}$ with
$|\tilde{\alpha}|\approx 0.028$. The sign of $\Delta m^{2}_{31}$ determines
the mass ordering. Eq. (8) holds for neutrinos; for antineutrinos the sign in
front of $\delta$ should be reversed. Evaluating this expression for
$\delta=\pm 90^{\circ}$ we obtain

$P_{3\nu}(\delta=\pm 90^{\circ})\approx
2s_{13}\sin^{2}\Delta[s_{13}\mp(\pm)\tilde{\alpha}\Delta]+\frac{1}{2}\tilde{\alpha}^{2}\Delta^{2}$
(9)

for neutrinos (antineutrinos).

In performing the TV test, we are attempting to fit the probability (9) with
an $L$-even function. According to our requirement ($iv$) it will have a
similar $\Delta$-dependence as Eq. (8) with $\delta=0$ or $\pi$:

$P_{\mathrm{even}}(\Delta^{\prime})\approx
a\sin^{2}\Delta^{\prime}+b\Delta^{\prime}\sin
2\Delta^{\prime}+c{\Delta^{\prime}}^{2}\,,$ (10)

where $a,b,c$ are three fit amplitudes with $a,c\geq 0$. Disappearance data
require $|\Delta^{\prime}|\approx|\Delta|$. The expression in Eq. (10) is
equivalent to our $L$-even fit assuming that zero-distance effects are small,
as it happens with the data we used in the analysis.

Let us focus now on the behaviour at the first and second oscillation maxima,
i.e., $|\Delta|=\pi/2,3\pi/2$. We can neglect the term proportional to $b$,
and Eq. (10) implies

$\displaystyle P_{\rm even}(3\pi/2)\geq P_{\rm even}(\pi/2)\,.$ (11)

Comparing this with Eq. (9), it becomes immediately clear that it is not
possible to fit the T-violating probabilities when the second oscillation
maximum is suppressed with respect to the first one, i.e., for
$\delta=+(-)90^{\circ}$ for neutrinos (antineutrinos). For the opposite case,
i.e., $\delta=-(+)90^{\circ}$ for neutrinos (antineutrinos), the second
oscillation maximum is enhanced and consistent with the condition (11).
Indeed, one can show that in this case, choosing three reference values for
$\Delta$ close to (but not exactly at) the first and second oscillation
maximum and the first oscillation minimum, all three coefficients, $a,b,c$,
can be fitted.

Hence, this simple considerations confirm the behaviour visible in figs. 4 and
5 (which are based on full oscillation probabilities including matter
effects): the suppression (enhancement) of the second oscillation maximum due
to the sign of the term proportional to $\sin\delta$ is responsible for the
(lack of) sensitivity around $\delta=90^{\circ}\,(270^{\circ})$ for neutrinos,
and the opposite for antineutrinos. Higher energy bins have no sensitivity to
TV due to the fact that there is no longer any data point at the second
oscillation maximum.

Figure 6:  Same as Fig. 4 for inverted hierarchy.  Figure 7:  Same as Fig. 6 for $\delta=270^{\circ}$.  Table 2:  Same as Tab. I for inverted hierarchy.  $E$ | w/o HKK | w/o DUNE | w/o ESS | all
---|---|---|---|---
0.65 | 0.01 [0.06] | 0.09 [0.70] | 0.00 [0.68] | 0.11 [0.70]
0.75 | 0.00 [0.05] | 2.46 [4.43] | 5.21 [4.36] | 6.01 [4.49]
0.85 | 0.71 [0.59] | 1.32 [1.38] | 4.87 [2.35] | 4.92 [2.40]
0.95 | - | - | 0.71 [0.62] | -
Tot. | 0.72 [0.71] | 3.87 [6.50] | 10.78 [7.99] | 11.04 [7.59]

Furthermore, it becomes also clear why this behaviour is independent of the
mass ordering. This follows from the observation that the sign of the term
$\tilde{\alpha}\Delta$ in Eq. (9) is independent of the mass ordering. We give
$\chi^{2}_{\mathrm{min}}$ results for inverted ordering in Tab. 2, and figs. 6
and 7 show some of the corresponding best-fit curves, which are qualitatively
similar to their normal-hierarchy counterparts. Matter effects induce
deviations from the above analytical arguments; however, at the energies under
consideration, the matter potential is small enough that these deviations are
subleading.


# How Real is Real: Evaluating the Robustness of Real-World Super Resolution

Athiya Deviyani  Efe Sinan Hoplamaz  Alan Savio Paul

###### Abstract

Image super-resolution (SR) is a field in computer vision that focuses on
reconstructing high-resolution images from the respective low-resolution
image. However, super-resolution is a well-known ill-posed problem as most
methods rely on the downsampling method performed on the high-resolution image
to form the low-resolution image to be known. Unfortunately, this is not
something that is available in real-life super-resolution applications such as
increasing the quality of a photo taken on a mobile phone. In this paper we
will evaluate multiple state-of-the-art super-resolution methods and gauge
their performance when presented with various types of real-life images and
discuss the benefits and drawbacks of each method. We also introduce a novel
dataset, WideRealSR, containing real images from a wide variety of sources.
Finally, through careful experimentation and evaluation, we will present a
potential solution to alleviate the generalization problem which is imminent
in most state-of-the-art super-resolution models.

Machine Learning, ICML

##  1 Introduction

Super-resolution is a process of reconstructing high-resolution images from
their low-resolution counterparts. However, it is well-known that super-
resolution is an ill-posed problem as a large amount of current state-of-the-
art methods are trained only to reconstruct images artificially downsampled by
a simple and uniform degradation (Dong et al., 2014; Tai et al., 2017; Tong et
al., 2017). One of the most commonly used downsampling methods is bicubic
downsampling. While these state-of-the-art methods produce impressive results
on bicubically downsampled images (Wang et al., 2018; Dai et al., 2019), they
perform poorly on low-resolution images downsampled by an unknown, realistic
image degradation method (Cai et al., 2019a; Lugmayr et al., 2019a). Such
realistic and natural degradations are what we are often presented with
through cameras, smartphones, TV, etc.

One of the reasons why natural image super-resolution is a big challenge is
that there is a limited number of natural low-resolution and corresponding
high-resolution image pairs. Therefore, methods such as bicubic downsampling
are employed to artificially generate LR-HR pairs (Wang et al., 2020). Other
work has also involved using synthetically generating real low-resolution to
high-resolution pairs through unsupervised learning or blind kernel estimation
(Lugmayr et al., 2019b; Zhou & Susstrunk, 2019). Others have shifted their
focus away from generating LR-HR pairs to simulating more complex image
degradation models (Lugmayr et al., 2019c). These methods have proven to
improve current super resolution models that were trained on only bicubically
downsampled images to generalise better to images found in ‘the wild’.
Supervised methods for “in-the-wild” real images have seen recent significant
advancements compared to unsupervised methods, which are more practical in the
real-world setting as it is difficult to obtain real LR-HR image pairs. We aim
to perform detailed analysis of existing state-of-the-art methods to develop a
better understanding of their strengths and weaknesses.

It is crucial that the trained super-resolution methods are able to generalise
to real-world images, which are naturally occurring low-resolution images that
have unknown and complicated downsampling kernels and noises. Examples of
their applications include facial recognition in video surveillance, remote
sensing, and healthcare applications such as detecting anomalies in medical
images. We hope that, with this paper, we will be able to contribute to the
development of SR models that are robust to the varied adverse effects (such
as noises) of different equipment in these applications.

In this paper we will compare and contrast the performance of several state-
of-the-art methods on real-world low-resolution images, and explore methods
that could improve their generalization performance. There are multiple
metrics which indicate how well the network reconstructs the LR image to its
HR version. However, calculating distortion metrics such as Peak Signal-to-
Noise Ratio (PSNR) and Structural Similarity Index (SSIM) (Wang et al., 2004)
is not possible in some cases if we are using real-world LR images for testing
because no ground-truth is available. These measures are objective and do not
take any subjective evaluation of human perception. We will thus use other
perceptual-based metrics for qualitative evaluation on our dataset which
consists of real-world images from a variety of sources.

Additionally, we note that there is a lack of datasets that contain a wide
variety of naturally downsampled, real images from different types of sources.
Having a wide variety of sources will help obtain more accurate measurements
of generalisability of real-world super-resolution models. Popular existing
datasets such as DPED and RealSR do not achieve this. DPED contains images
taken from 3 smartphone cameras, which together might have different noises
compared to TV streams, CCTV footage and satellite images. RealSR contains
images taken from the same camera and only helps evaluate the performance of
models on the particular downsampling method of such cameras. In this paper,
we aim to introduce a carefully curated dataset, WideRealSR, that can be used
as a test set for thoroughly evaluating real-world performance.

In short, the goal of this paper can be summed as the following:

  1. 1.

Evaluate the generalisability of various supervised and unsupervised super-
resolution models,

  2. 2.

Identify the reasons why the models perform poorly or well, and

  3. 3.

Investigate a method to devise a practical solution to potentially alleviate
the generalization problem.

##  2 Dataset and task

As mentioned in the previous section, most of the existing datasets used in
the real-world super-resolution field lack a diverse range of sensor noises.
The DPED dataset (Ignatov et al., 2017a), for example, only contains images
from 3 different smartphone cameras - iPhone 3GS, BlackBerry Passport, and
Sony Xperia Z - as the Low-Resolution images. The respective High-Resolution
counterparts are captured using a Canon 70D DSLR. This dataset provided
support for the authors to present an end-to-end deep learning approach that
bridges the gap between ordinary photos into higher-quality DSLR-like images.
The authors have proposed to learn the translation function using a residual
convolutional neural network that improves both color rendition and image
sharpness (Ignatov et al., 2017b). Since its release, the DPED dataset has
been the base dataset for numerous super-resolution model proposals, including
the award-winning RealSR model.

A popular alternative to the DPED dataset is DIV2K (Agustsson & Timofte,
2017), containing 800 high-resolution images and their corresponding low-
resolution images that are obtained artificially through a variety of
downsampling methods. Recent work has emerged (Cai et al., 2019b) in the
super-resolution field which aim to identify the best image downgrading
methods that best generalise to images in the real world, i.e. images with
unknown sensor noises. The DIV2K dataset is commonly used in this particular
field of super-resolution.

To evaluate the performance of existing super-resolution models, it is not
sufficient to sample from a limited number of sensor noises and downsampling
techniques. This is because, as mentioned previously, images in the ‘real-
world’ inherently come with arbitrary kernels and noises. Thus, following this
limitation, we decided to meticulously curate a dataset to be used for
evaluating several super-resolution models. We have obtained 1-3 images for 35
different sensor noises, from sources such as (but not limited to) Google
Maps, satellites, drones, microscopes, smartphones (iPhone, BlackBerry,
Samsung Galaxy), WhatsApp, Facebook, tablets (iPad, Samsung Galaxy Tab), BBC
broadcasts. To maintain the ‘real-world’ factor, we decided to do minimal
preprocessing and only cropping the image when necessary. We call this dataset
WideRealSR. We have displayed sample images from our dataset collected from 10
different sources in Figure 1.

Figure 1: Sample images from the WideRealSR dataset

The main task in this paper is to perform real-world super-resolution using
state-of-the-art models on images obtained with a diversity of sensor noises.
From this, we will be able to identify which models generalise best, and thus
performs real-world super resolution to a satisfactory level. Due to the lack
of ‘ground-truth’ high-resolution images, we will not be able to use
quantitative metrics such as PSNR to evaluate the performance of the models.
Instead, we decided to perform an extensive user study to obtain a human
rating on the super-resolved images, which was also done for (Rad et al.,
2021). The details of the survey will be discussed in the proceeding sections.

##  3 Methodology

###  3.1 Real-World Super Resolution Methods

RealSR

The authors of the RealSR paper (Cai et al., 2019a) propose a novel
degradation framework based on kernel estimation and noise injection. The
method they propose is mainly divided into two stages; the first stage is to
estimate the degradation from real data and generate realistic LR images, the
second is to train the SR model based on the constructed data. They take two
sets of images as input: a real-world image set $X$, and an HR image set $Y$.
They estimate the kernel (using KernelGAN (Bell-Kligler et al., 2019)) and
noise from the $X$ set, which is then applied for downsampling the HR set to
acquire an LR image set.

They then train a modified version of ESRGAN (super-resolution network,
described below) with a patch discriminator, which is capable of producing HR
results with better perceptual qualities. The patch discriminator helps avoid
the addition of artifacts. Compared with previously existing methods, the
RealSR model produces much fewer noise and artifacts, implying that the noise
estimated by noise injection is closer to the real noise.

Their experiments on real-world images from the DPED dataset show that their
model outperforms state-of-the-art methods such as EDSR (Lim et al., 2017),
ESRGAN (Wang et al., 2018), ZSSR (Shocher et al., 2018), and K-ZSSR, resulting
in lower noise and better visual quality. They also won the NTIRE 2020
challenge by a significant margin when scored using human perception and the
Perceptual Index (PI). This is why we are keen to investigate RealSR’s
performance on our WideRealSR dataset.

USRNet

The authors of the USRNet model proposed a deep unfolding super-resolution
network. Inspired by the flexibility of model-based methods and advantages of
learning-based methods, this model aims to get the ‘best of both worlds’. The
main novelty aspect of the model is that it is the first paper that has
successfully attempted to handle the degradation model via a single model. The
core idea of this model is the deep unfolding image restoration method: it
unfolds the MAP (maximum a posteriori) inference through the semi-quadratic
splitting algorithm and a mixed number of iterations, consisting of
alternately solving data subproblems and obtaining the prior subproblems.
These two subproblems can be solved by the neural module to obtain a trainable
end-to-end iterative network.

The USRNet is obtained through unsupervised learning. During training, the
model takes in a high-resolution image and obtains the low-resolution image
through a specially designed degradation model, which takes into account three
key user-defined inputs: scale factor, the blur kernel, and the noise level.
After a series of thorough experimentation, the authors were able to show that
the USRNet prevailed over its competitors (such as RCAN (Zhang et al., 2018),
ZSSR (Shocher et al., 2018)). Thus, they concluded that USRNet has a high
potential when used in a real-world setting.

While the experimental results look promising, it is interesting to evaluate
the network’s potential on our WideRealSR dataset containing images with an
inherently unknown downsampling method, as the model relied heavily on
computationally generated low-resolution images and their high-resolution
counterpart during training.

DPSR

The Deep Plug-and-Play Super-Resolution model (Zhang et al., 2019), also known
as DPSR. The novelty of this model lies in the image restoration method: it
treats each restored model as a flexible plug-and-play (hence, the name)
toolbox. The main objective of the paper is to build a new degradation model
that will allow existing blind deblurring methods to estimate the blur kernel
from an out-of-distribution low-resolution image. This is done in hopes to
improve the robustness of existing super-resolution models in real-world
applications.

Similar to USRNet (Zhang et al., 2020), the authors of DPSR introduce an
energy function to optimize the new degradation model. The plug-and-play
models are obtained through a semi-quadratic splitting technique. From this,
they were able to plug any super-resolver prior rather than the denoiser prior
as a modular part. The model was then evaluated through comparison between
artificially generated images and real images.

Through a wide range of experiments, the authors have successfully shown that
their model allowed them to insert the super-parser prior in the plug-and-play
framework in the place of the denoising prior. This signifies that they are
able to use the advantages of the existing neural-network-based single-image
super-resolution moles to pre-design and train the super decomposer. The
experiments also show that the proposed degradation model can potentially
solve the super-resolution problem of arbitrary fuzzy kernels and can be used
in a real-world setting to produce satisfactory results. We were curious as to
what extent the new degradation model affects the generalisability of the
model, thus we decided to include this model in our experiments.

###  3.2 Non-Real-World Super Resolution Methods

ESRGAN

This super-resolution model (Wang et al., 2018) is an enhanced version of a
previous implementation - SRGAN (Ledig et al., 2017). The general architecture
design of the network is retained but few new concepts are added and changed
which ultimately lead to increase in efficiency of the network. To do this,
the authors studied three key components of SRGAN: network architecture,
adversarial loss, and perceptual loss, and improved each of them to derive an
Enhanced SRGAN (ESRGAN).

The network structure of the generator is improved by introducing the
Residual-in-Residual Dense Block which replaces the original basic blocks in
SRGAN and removing all Batch Normalization layers, which contributed to the
reduction of computation complexity and increase in performance. The authors
have managed to improve the adversarial loss by employing a relativistic
discriminator that probabilistically judges ‘whether one image is more
realistic than the other’ instead of ‘whether one image is real or fake’.
Finally, the authors have introduced perceptual loss to optimize super-
resolution models in feature space instead of pixel space, which leads to
brightness consistency and texture recovery.

The final model that was proposed in the paper was trained as an unsupervised
learning task. It creates the LR-HR pairs by bicubic downsampling the training
HR images (obtained from DIV2K). It is trained to generate high-resolution
images from low-resolution test images with an up-scaling factor of 4. The
model was then evaluated with the aforementioned losses. Although the ESRGAN
is not intended for real-world applications, it is a common backbone for
models like RealSR, and models that aim to solve the real-world super
resolution problems such as face images from surveillance cameras (Aakerberg
et al., 2021).

EnhanceNet

A common problem in objective metric-based Single Image Super-Resolution CNN
models is the generation of over-smoothed images, regardless of high PSNR
values. EnhanceNet (Sajjadi et al., 2017) aims to address this problem by
focusing primarily on generating realistic textures along with higher
perceptual quality rather than just optimizing PSNR values.

The EnhanceNet is built in the likes of a generic CNN architecture. The feed-
forward fully convolutional network contains 10 residual blocks to help faster
model convergence. Upsampling is done through nearest neighbor upsampling
followed by a convolution layer instead of the convolution transpose layers.
To ensure training stability and avoid any unwanted colour shifts, a bicubic
interpolation of the low-resolution input image is added to the reconstructed
output. The authors evaluated the performance of the model when implemented
with a variety of combinations of loss functions, including the pixel-wise
MSE-loss in the image domain, the perceptual loss in the feature space, the
texture matching loss, and the adversarial loss.

The authors have presented through thorough experimentation that EnhanceNet
optimized with a combination of perceptual loss, adversarial loss and texture
loss generates an image that is very close to the original high-resolution
image. Unlike the models presented previously, EnhanceNet aims to solve only a
specific problem that is common in real-world image super-resolution model
design attempts. Investigating the performance of this model on our WideRealSR
dataset will help us identify whether or not alleviating the over-smoothing
problem is an important factor to generate more realistic high-resolution
images from low-resolution input.

###  3.3 Evaluation method

As mentioned briefly in the previous section, we evaluate the model using its
pretrained weights on our dataset, WideRealSR, which contains images with a
diverse range of sensor noises. We have kept the image preprocessing to a
minimum as we would like to keep the images as close to actual real-world
images as possible.

However, due to the various sensor noises present and the unavailability of
the complementary high-resolution images in the WideRealSR dataset, we were
not able to quantitatively evaluate the performance of the models. Existing
quantitative super-resolution metrics such as PSNR and SSIM calculate the
distortion between a model-generated high-resolution image to its respective
gold-standard counterpart. In addition, recent studies have proven that the
PSNR and SIM metrics do not always correlate well with super-resolution
performance, signifying a discrepancy between the objective evaluation and
subjective human perception (Ozcinar & Rana, 2021).

As a workaround, we turned to (Rad et al., 2021) paper and designed a similar
qualitative evaluation pipeline which relies on the human perception ability.
We performed a user study in the form of a survey where a respondent would be
presented with a series of super-resolved images from the WideRealSR. These
images are generated from the five super-resolution models discussed
previously. Out of the five unlabeled images, the respondent is required to
identify one image that they think has the highest resolution while still
maintaining a realistic ‘look’, i.e. the images do not contain oversaturated
colors, unrealistic contrast and brightness, etc. From this user study, we
were able to identify which image super-resolution models are most resistant
to unknown kernels, and thus proving its robustness for real-world
applications.

##  4 Experiments

The experiments in this section were run on an NVIDIA Geforce GTX 1060 GPU in
a virtual environment under Scientific Linux version 7.8. We have utilised the
official PyTorch implementations of the models discussed in the previous
section.

In this section, we will perform experiments to prove or disprove the
‘robustness’ claims made by the proponents of the state-of-the-art super-
resolution models, thus evaluating the feasibility of implementing them in a
real-world setting.

The first experiment we conducted consisted of several straightforward sub-
experiments described as the following:

Input: WideRealSR Real-world LR Images $lr$, set of models $m$

for each $m_{i}$ in $m$ do

Load pre-trained weights $w_{i}$ of $m_{i}$, where $w$ is obtained with the
optimal model hyperparameters

for each $lr_{j}$ in $lr$ do

Generate high-resolution image $hr_{j}$ from $lr_{j}$ using $m_{i}$

Output $hr_{j}$

end for

end for

Algorithm 1 Experiments using WideRealSR

After performing the experiment above, we were able to visually observe the
performance of each model on out-of-distribution images (images with arbitrary
kernels) by physically comparing the generated high-resolution images.

We picked one image from each domain to avoid a burdensome evaluation process.
Then, we collated their corresponding generated high-resolution images from
the various models to be used in our user study, to be presented to a pool of
30 respondents. We have shown a sample of the images below:

Table 1 shows the number of times where a user identified an image generated
by a particular model as the ‘best’ super-resolved image.

Model | Counts (per domain)
---|---
Smartphone | Google StreetView | Other
RealSR | 21 | 7 | 537
USRNet | 5 | 9 | 120
ESRGAN | 0 | 3 | 12
EnhanceNet | 0 | 1 | 6
DPSR | 4 | 10 | 315
Table 1: User Study Results: Number of votes obtained by each model Figure 2:
Comparison of the performances of RealSR, EnhanceNet, ESRGAN, DPSR, and USRNet
on a KITTI driving image from our WideRealSR dataset. Figure 3: Comparison of
the performances of RealSR, EnhanceNet, ESRGAN, DPSR, and USRNet on a iPhone11
fish image from our WideRealSR dataset.

From the detailed user study, we note the inconsistency in the notably award-
winning high-performing RealSR model: it performs significantly well on images
taken on many smartphones, however does significantly poorly on images
captured from other sources such as Google StreetView and KITTI driving
dataset. Figure 3 shows one instance where RealSR outperforms other models
when evaluated on the downsampling kernel it was trained on, while Figure 2
illustrates RealSR adding artifacts and ruining the image when evaluated on an
out-of-distribution downsampling kernel. It is important to note that the
RealSR model was trained on an iPhone 3GS, a BlackBerry Passport, and a Sony
Xperia Z, nevertheless it performs well on sample images captured through
other smartphones such as Samsung S3 mini, OnePlus 6, and Microsoft Lumia.
From the results of this evaluation, we were able to formulate a hypothesis
concerning the correlation between the input data sensor noise diversity to
model performance: Incorporating images from a broad range of sensor noises
into the training phase will improve the generalisability of the model.

However, simply generating LR-HR pairs using all sensor noises for building
the training dataset would not work because the resulting model would be
“confused” and super-resolve the image poorly. This is why the training
dataset must be LR-HR pairs containing a single downsampling kernel. However,
our goal is to make the model capable of super-resolving images with any
unknown downsampling kernel. Thus, in order to do this, we propose a novel
approach to training the RealSR.

In our new approach, we first group together kernels that are similar to each
other using k-means clustering after finding the optimal number of clusters.
Each cluster group will represent one type of kernel. For each cluster, we use
the cluster centre as the downsampling kernel to create a training set (LR-HR
pair) and train the RealSR model. Thus we obtain 1 model per cluster.

For every test image, we estimate its kernel using KernelGAN (Bell-Kligler et
al., 2019) and find which cluster it belongs to using distance from the
cluster centres. We use its cluster’s RealSR model to super-resolve the LR
real-world test image. Due to limited time budget, we first (1) verify that
training the model on a domain apart from DPED’s sensor domain will lead to
performance improvements on that domain, when compared to the performance of
the model trained on the DPED domain. Then, (2) we apply K-Means clustering on
the kernel groups to understand the similarities between kernels and prepare
to train multiple models, which will be the last step in building a model that
is more generalizable than RealSR.

To verify hypothesis (1), we first pick a poor super-resolved output image
generated by the pre-trained DPED model, to compare it against our new model’s
performance.

To train the new model from scratch, we estimate the downsampling kernel in
Google StreetView images and create a train and validation dataset containing
LR-HR pairs by downsampling HR images (from DIV2K (Agustsson & Timofte, 2017))
using the estimated kernel. We then train the RealSR (ESRGAN-based) model
using this dataset. Details of this model are mentioned in the RealSR section
above. The comparison of the performance of this model versus the pretrained
models on the selected image is illustrated in Figure 4.

Figure 4: Comparison of the performances of pre-trained RealSR, EnhanceNet,
ESRGAN, DPSR, USRNet and RealSR trained on Google Street View images on a
Google Street View image from our WideRealSR dataset.

The newly trained model performs significantly better than the model trained
only on smartphone images. Particularly, we note that the new model has more
realistic textures and does not include undesirable artifacts.

(2) With the basic hypothesis proved, we further explore the clusters of
kernels that we would train models on. We flatten the kernel matrices to get
(1, 1089)-shaped vectors, which we use as inputs to SkLearn’s k-means
clustering algorithm with 7 clusters and illustrate the clusters in Figure 5
with further details in Table 2. For visualizing the clusters, we used PCA to
convert the 1089 dimensional feature space to a 2 dimensional vector.

Thus, we successfully verified that we can make the RealSR model more
generalizable by training on a dataset that has a downsampling kernel similar
to the test input. We also proposed using the K-Means clustering technique to
scale this improvement to arbitrary test input kernels. We leave the
verification of this final proposal as potential future work.

Cluster |  Source/Equipment |  Name/Names
---|---|---
0 |  Mobile Phone |  iPhone 6, iPhone 11, LG, Huawei, Samsung S3 Mini
Satellite camera |  Satellite
Camera |  Kodak
Microscope |  Malaria
Car camera |  LYFT-overhead
Tablet camera |  Galaxy Tab, iPad2
Facebook compression |  ipad2_fb
Whatsapp compression |  samsungs3mini_whatsapp
Drone camera |  Drone
1 |  Camera |  Nikon, Kodak, Canon, Kinect
Laptop webcam |  Lenovo_webcam
Google Maps |  Google Street View
Webcam |  webcam
2 |  TV Broadcast |  BBC Pose
Whatsapp compression |  iPad 2_wa
Mobile Phone |  iPhone3, iPhone 5c
3 |  Mobile Phone |  Blackberry, Microsoft Lumia
4 |  CCTV |  indoor_cam
5 |  Facebook compression |  Samsungs3mini_facebook
6 |  FLIR Camera |  KITTI
Table 2: Cluster labels and noise sources of kernels illustrated in Figure 5
Figure 5: Visualisation of dimension-reduced kernel clusters

##  5 Related work

There has not been an extensive amount of research done on comparing the
overall generalisability of available state-of-the-art super-resolution
models. Most of the existing work that evaluates the performance of these
models are very domain specific, such as in biomedical optics for retinal
imagery (Thapa et al., 2014). Similar papers would also investigate the
corresponding hardware to take into consideration during the evaluation
process.

On the contrary, there have been plenty of attempts to create datasets to
train and evaluate super-resolution models, as we did with WideRealSR, except
none of them contain a wide range of downsampling methods. A rather popular
dataset is the NTIRE dataset (Nah et al., 2019). The authors introduced a
novel large dataset for video deblurring, video super-resolution and
evaluation of state-of-the-art models. Originating from the 2019 NTIRE
challenges, the video deblurring and video super-resolution challenges are the
first challenges of its kind receiving numerous participants and proposed
solutions. Another attempt of collecting datasets for conducting image super-
resolution experiments is the Wieldfield2SIM (Zhou et al., 2020) dataset.
Unlike NTIRE, this dataset is more domain-specific, as it consists of 100K+
real fluorescence microscopy images, containing noisy LR images with different
noise levels, noise-free LR images with corresponding high-quality HR images.
This dataset was used to benchmark the combinations of 6 denoising methods and
6 super-resolution methods.

Most of the methods proposed to solve the generalization models commonly
involve improving and tweaking existing models. This is done by selectively
choosing different aspects of the super-resolution method to amend, such as
the downsampling method, image reconstruction, and more. Some of these
improvements were implemented by the models that we have discussed in the
preceding sections in this report.

##  6 Conclusions

In this paper, we found that the models are not as generalizable as they
claim, by testing them on a wide variety of sensor noises in the form of a
dataset that we propose, called WideRealSR. We recognise the reason for the
generalization problem: inability to deal with random unknown noise because of
the lack of diversity in sensor noises available for training.

We then found a simple and potentially highly effective solution to increase
the generalisability of the RealSR model. While we propose a method to improve
an existing model, we find it important to point out a crucial flaw in this
high-performing model: the objective loss function. We discuss that human
evaluation is a better metric, currently, to evaluate the model performance.
However, these models have been trained on flawed objective loss functions
such as PSNR and SSIM. We encourage researchers to focus on developing a
reliable metric that highly correlates with perceptual quality of the HR
image.

We would like to encourage researchers to completely verify our final proposal
that could potentially outperform state-of-the-art real-world super-resolution
methods.

## References

  * Aakerberg et al. (2021) Aakerberg, A., Nasrollahi, K., and Moeslund, T. B.  Real-world super-resolution of face-images from surveillance cameras.  _arXiv preprint arXiv:2102.03113_ , 2021.
  * Agustsson & Timofte (2017) Agustsson, E. and Timofte, R.  Ntire 2017 challenge on single image super-resolution: Dataset and study.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , pp. 126–135, 2017.
  * Bell-Kligler et al. (2019) Bell-Kligler, S., Shocher, A., and Irani, M.  Blind super-resolution kernel estimation using an internal-gan.  _arXiv preprint arXiv:1909.06581_ , 2019.
  * Cai et al. (2019a) Cai, J., Zeng, H., Yong, H., Cao, Z., and Zhang, L.  Toward real-world single image super-resolution: A new benchmark and a new model.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 3086–3095, 2019a.
  * Cai et al. (2019b) Cai, J., Zeng, H., Yong, H., Cao, Z., and Zhang, L.  Toward real-world single image super-resolution: A new benchmark and a new model.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 3086–3095, 2019b.
  * Dai et al. (2019) Dai, T., Cai, J., Zhang, Y., Xia, S.-T., and Zhang, L.  Second-order attention network for single image super-resolution.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 11065–11074, 2019.
  * Dong et al. (2014) Dong, C., Loy, C. C., He, K., and Tang, X.  Learning a deep convolutional network for image super-resolution.  In _European conference on computer vision_ , pp. 184–199. Springer, 2014.
  * Ignatov et al. (2017a) Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., and Van Gool, L.  Dslr-quality photos on mobile devices with deep convolutional networks.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 3277–3285, 2017a.
  * Ignatov et al. (2017b) Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K., and Van Gool, L.  Dslr-quality photos on mobile devices with deep convolutional networks.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 3277–3285, 2017b.
  * Ledig et al. (2017) Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al.  Photo-realistic single image super-resolution using a generative adversarial network.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pp. 4681–4690, 2017.
  * Lim et al. (2017) Lim, B., Son, S., Kim, H., Nah, S., and Mu Lee, K.  Enhanced deep residual networks for single image super-resolution.  In _Proceedings of the IEEE conference on computer vision and pattern recognition workshops_ , pp. 136–144, 2017.
  * Lugmayr et al. (2019a) Lugmayr, A., Danelljan, M., and Timofte, R.  Unsupervised learning for real-world super-resolution.  In _2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)_ , pp. 3408–3416. IEEE, 2019a.
  * Lugmayr et al. (2019b) Lugmayr, A., Danelljan, M., and Timofte, R.  Unsupervised learning for real-world super-resolution.  In _2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)_ , pp. 3408–3416. IEEE, 2019b.
  * Lugmayr et al. (2019c) Lugmayr, A., Danelljan, M., Timofte, R., Fritsche, M., Gu, S., Purohit, K., Kandula, P., Suin, M., Rajagoapalan, A., Joon, N. H., et al.  Aim 2019 challenge on real-world image super-resolution: Methods and results.  In _2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)_ , pp. 3575–3583. IEEE, 2019c.
  * Nah et al. (2019) Nah, S., Baik, S., Hong, S., Moon, G., Son, S., Timofte, R., and Mu Lee, K.  Ntire 2019 challenge on video deblurring and super-resolution: Dataset and study.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , pp. 0–0, 2019.
  * Ozcinar & Rana (2021) Ozcinar, C. and Rana, A.  Quality assessment of super-resolved omnidirectional image quality using tangential views.  _arXiv preprint arXiv:2101.10396_ , 2021.
  * Rad et al. (2021) Rad, M. S., Yu, T., Musat, C., Ekenel, H. K., Bozorgtabar, B., and Thiran, J.-P.  Benefiting from bicubically down-sampled images for learning real-world image super-resolution.  In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , pp. 1590–1599, 2021.
  * Sajjadi et al. (2017) Sajjadi, M. S., Scholkopf, B., and Hirsch, M.  Enhancenet: Single image super-resolution through automated texture synthesis.  In _Proceedings of the IEEE International Conference on Computer Vision_ , pp. 4491–4500, 2017.
  * Shocher et al. (2018) Shocher, A., Cohen, N., and Irani, M.  “zero-shot” super-resolution using deep internal learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 3118–3126, 2018.
  * Tai et al. (2017) Tai, Y., Yang, J., Liu, X., and Xu, C.  Memnet: A persistent memory network for image restoration.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 4539–4547, 2017.
  * Thapa et al. (2014) Thapa, D., Raahemifar, K., Bobier, W. R., and Lakshminarayanan, V.  Comparison of super-resolution algorithms applied to retinal images.  _Journal of biomedical optics_ , 19(5):056002, 2014.
  * Tong et al. (2017) Tong, T., Li, G., Liu, X., and Gao, Q.  Image super-resolution using dense skip connections.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 4799–4807, 2017.
  * Wang et al. (2018) Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., and Change Loy, C.  Esrgan: Enhanced super-resolution generative adversarial networks.  In _Proceedings of the European Conference on Computer Vision (ECCV) Workshops_ , pp. 0–0, 2018.
  * Wang et al. (2004) Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P.  Image quality assessment: from error visibility to structural similarity.  _IEEE transactions on image processing_ , 13(4):600–612, 2004.
  * Wang et al. (2020) Wang, Z., Chen, J., and Hoi, S. C.  Deep learning for image super-resolution: A survey.  _IEEE transactions on pattern analysis and machine intelligence_ , 2020.
  * Zhang et al. (2019) Zhang, K., Zuo, W., and Zhang, L.  Deep plug-and-play super-resolution for arbitrary blur kernels.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 1671–1681, 2019.
  * Zhang et al. (2020) Zhang, K., Gool, L. V., and Timofte, R.  Deep unfolding network for image super-resolution.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 3217–3226, 2020.
  * Zhang et al. (2018) Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., and Fu, Y.  Image super-resolution using very deep residual channel attention networks.  In _Proceedings of the European conference on computer vision (ECCV)_ , pp. 286–301, 2018.
  * Zhou & Susstrunk (2019) Zhou, R. and Susstrunk, S.  Kernel modeling super-resolution on real low-resolution images.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 2433–2443, 2019.
  * Zhou et al. (2020) Zhou, R., Helou, M. E., Sage, D., Laroche, T., Seitz, A., and Süsstrunk, S.  W2s: a joint denoising and super-resolution dataset.  _arXiv preprint arXiv:2003.05961_ , 2020.