# Quantum entanglement between neutrino eigenstates in the presence of the
subsequent phase shift of the neutrino oscillations

Hoda Abdolalizadeh<EMAIL_ADDRESS>Theoretical and Computational
Physics Research Laboratory, Istanbul University, 34134 İstanbul, Türkiye
Institute of Graduate Studies in Science, İstanbul University, 34134,
İstanbul, Türkiye  Ekrem Aydiner<EMAIL_ADDRESS>
https://orcid.org/0000-0002-0385-9916 Theoretical and Computational Physics
Research Laboratory, Istanbul University, 34134 İstanbul, Türkiye  Department
of Physics, Faculty of Science, İstanbul University, 34134 İstanbul, Türkiye

###### Abstract

In this Letter, using von Neumann entropy we examine the entanglement entropy
for the neutrino oscillations in the presence of the subsequent phase shift.
We numerically show that the entanglement entropy for the subsequent periods
of the two-flavor neutrino oscillations increases asymmetrically with time
depending on the space-time deformation. We also explored the obtained results
for the three-flavor neutrino oscillations to show that this result is also
valid for the three-flavor neutrino oscillations. These results, obtained for
the first time in this Letter, are quite different from the computing for the
standard neutrino oscillation theory. We concluded that these interesting
results play an important role in the cosmology.

††preprint: APS/123-QED

##  I Introduction

Recently, quantum entanglement and the phase shift in neutrino oscillations
have attracted much attention in neutrino physics. In this Letter, we will
discuss two issues together. Before delving into the subject, we would like to
briefly present the work done in both fields.

Entanglement entropy as Measuring the degree of entanglement is a fundamental
concept in quantum mechanics. In the case of neutrino particles, the
entanglement entropy indicates the degree of entanglement between the
eigenstates of the neutrino mass. Considering the importance of various
aspects of entanglement entropy in neutrino oscillations, many theoretical
studies have addressed this issue. It can be seen in the literature that
Blasano et al have done a series of studies on quantum entanglement in
neutrino oscillations. For instance, Blasone et al quantified in detail the
amount and the distribution of entanglement in the physically relevant cases
of flavor mixing in quark and neutrino systems [1]. In 2009, they showed that
mode entanglement can be expressed in terms of flavor transition probabilities
[2]. In 2010, they studied single-particle entanglement arising in two-flavor
neutrino mixing, first in the context of quantum mechanics, and then in the
case of quantum field theory [3]. In 2013, they described neutrino
oscillations in terms of (dynamical) entanglement of neutrino flavor modes
[4]. In 2014, they extended their studies to the framework of quantum field
theory and used concurrence as a suitable measure of entanglement [5], in
another work, they analyzed the entanglement in the states of flavor neutrinos
and antineutrinos within the framework of quantum information theory and,
revealed its correlation with experimentally measurable quantities such as the
variances of lepton numbers and charges [6]. In 2015, they employed two
measures: the concurrence and the logarithmic negativity, to quantify the
entanglement, and they analyzed the behavior of multipartite entanglement
within the (three-qubit) Hilbert space of flavor neutrino eigenstates [7].
Similarly, Kayser et al examined the role of entanglement in theoretical
models of neutrino oscillation, and they found that a theoretical approach
considering the entanglement between neutrinos and their interaction partners
still produces the standard result for the neutrino oscillation wavelength
[8]. Martin et al [9] investigated measurements of quantum information,
including entanglement entropy and purity in quantum neutrino oscillations at
macroscopic scales in extreme astrophysical environments, such as the early
universe, core-collapse supernovae, and merging neutron stars. Siwach et al
examined entanglement in neutrino systems by quantifying entropies and
polarization vector components in the context of three-flavor oscillations
[10]. Amol et al presented a significant connection between the entanglement
entropy of individual neutrinos and the occurrence of spectral splits in their
energy spectra due to collective neutrino oscillations [11]. Mallick et al
explored the entanglement entropy between spins and position space during
neutrino propagation [12]. Cervia et al measured the entanglement entropy and
the Bloch vector of the reduced density matrix, which are used to assess the
interactions between constituent neutrinos in the many-body system [13].

On the other hand, investigating the phase shift of neutrino oscillations,
such as the entanglement entropy, is a fascinating topic proposed by Ahluwalia
and Burgard [14, 15]. They theoretically showed that the effect of gravity
appears in the form of phase shift in neutrino oscillations. After these
seminal works, this concept has received attention in various studies within
the literature. For instance, Grossman et al argued that the gravitational
oscillation phase shift in the neutrino oscillation might have a significant
effect on supernova explosion [16]. Capozziello et al discussed the
gravitational phase shift of neutrino oscillation in the framework of
$f(R)$-gravity [17]. Additionally, Capozziello and Lambiase analyzed the
flavor oscillations of neutrinos in the framework of Brans-Dicke theory of
gravity [18]. Ren and Zhang presented general equations for oscillation phases
under the influence of gravitational rotation and electric charge in Kerr-
Newman spacetime [19]. Koranga showed that there would be a phase shift in
neutrino oscillation due to quantum gravity effects at the Planck scale [20].
In a similar study, Swami computed the phase shift in neutrino oscillations
due to the rotation of gravitational sources for both non-lensed and lensed
neutrinos Ref. [21]. More recently, Aydiner theoretically proposed that phase
shift between eigenstates may gradually increase in the neutrino oscillation
[22]. This means that phase shift in Aydiner model is not constant, unlike,
the phase shift increases each subsequent oscillation period depending on the
deformation of the space-time.

In this Letter, we will investigate the entanglement entropy between neutrino
eigenstates in the presence of the subsequent phase shift based on Ref.[22].
We will show that quantum entanglement between eigenstates also increases
depending on the deformation of the space-time.

This work is organized as follows: In Section II, we first analyze the
neutrino flavor oscillation model and then delve into the concept of
entanglement entropy in neutrino oscillations. Subsequently, we calculate the
entropy of neutrino oscillations between two flavors using Von Neumann’s
method. Section III, we present the impact of varying degrees of space-time
deformation on the entanglement entropy and phase shift of neutrino
oscillations. Finally, in the last section, a discussion and conclusion are
given.

##  II Neutrino Oscillation and Quantum Entanglement

###  II.1 Neutrino Oscillation

Neutrino oscillation is the probability of transition from a flavor to the
other one during the neutrino propagation. The discrepancy between mass and
flavor eigenstates and also the non-zero and non-degenerate neutrino mass give
rise to the neutrino oscillation. Neutrino oscillation can be characterized by
three mixing angles and a phase. The flavor eigenstates of the neutrino
denoted by $|\nu_{\alpha}\rangle$ can be represented as linear superposition
of the mass eigenstates $|\nu_{j}\rangle$

$|\nu_{\alpha}\rangle=\sum_{j}U^{*}_{\alpha
j}|\nu_{j}\rangle,\quad\alpha=e,\mu,\tau,\quad j=1,2,3$ (1)

where $U$ is a unitary and non-diagonal mixing matrix that specifies the
composition of each neutrino flavor state. It can be given in the general form

$U=\begin{pmatrix}c_{12}c_{13}&s_{12}c_{13}&s_{13}e^{-i\delta_{CP}}\\\
-s_{12}c_{23}-c_{12}s_{23}s_{13}e^{i\delta{CP}}&c_{12}c_{23}-s_{12}s_{23}s_{13}e^{i\delta{CP}}&s_{23}c_{13}\\\
s_{12}s_{23}-c_{12}c_{23}s_{13}e^{i\delta{CP}}&-c_{12}s_{23}-s_{12}c_{23}s_{13}e^{i\delta{CP}}&c_{23}c_{13}\\\
\end{pmatrix}$ (2)

where $c_{ij}=\cos\theta_{ij}$, $s_{ij}=\sin\theta_{ij}$ and $\delta$ is the
Dirac CP violating phase. This matrix is called the
Pontecorvo–Maki–Nakagawa–Sakata (PMNS) matrix [23, 24]. The PMNS matrix
represents the transformation between flavor and mass eigenstates, providing
insights into how the probabilities of different flavor outcomes are entangled
with the underlying mass states. For three flavors Eq.(1) can be written as

$\begin{pmatrix}\nu_{e}\\\ \nu_{\mu}\\\ \nu_{\tau}\\\
\end{pmatrix}=\begin{pmatrix}U_{e1}&U_{e2}&U_{e3}\\\ U_{\mu 1}&U_{\mu
2}&U_{\mu 3}\\\ U_{\tau 1}&U_{\tau 2}&U_{\tau 3}\\\
\end{pmatrix}\begin{pmatrix}\nu_{1}\\\ \nu_{2}\\\ \nu_{3}\\\ \end{pmatrix}$
(3)

However, Eq.(3) for two flavors can be given in the simple form:

$\begin{pmatrix}\nu_{e}\\\ \nu_{\mu}\\\
\end{pmatrix}=\begin{pmatrix}\cos\theta&\sin\theta\\\
-\sin\theta&\cos\theta\end{pmatrix}\begin{pmatrix}\nu_{1}\\\ \nu_{2}\\\
\end{pmatrix}$ (4)

where $\nu_{e}$ and $\nu_{\mu}$ can be given in the eigenstate forms

${|\nu_{e}(t=0)\rangle}=\cos{\theta}{|\nu_{1}\rangle}+\sin{\theta}{|\nu_{2}\rangle},\qquad{|\nu_{\mu}(t=0)\rangle}=-\sin{\theta}{|\nu_{1}\rangle}+\cos{\theta}{|\nu_{2}\rangle}.$
(5)

These equations describe the initial states of electron neutrinos and muon
neutrinos as linear combinations of the flavor eigenstates $|\nu_{1}\rangle$
and $|\nu_{2}\rangle$. The weak eigenstates are rotated by an angle $\theta$
with respect to the mass eigenstates ${|\nu_{1}\rangle}$ and
${|\nu_{2}\rangle}$ to allow mixing between $\nu_{\mu}$ and $\nu_{e}$. The
time-evolved state of flavor eigenstate $\nu_{\alpha}$, with $\alpha=e$, $\mu$
is a superposition of the mass eigenstates $\nu_{k}$, with $k=1$ and 2,

${|{\nu}_{\alpha}(x,t)\rangle}=\sum_{k}U_{\alpha
k}\exp[-i\phi_{k}]{|{\nu}_{k}\rangle}$ (6)

where the phase factor is

$\phi_{k}=E_{k}t-\vec{P}_{k}.\vec{x}$ (7)

where $E_{k}$ and $\vec{P}$ are the energy and momentum of the mass
eigenstates ${|{\nu}_{k}\rangle}$. The oscillation probability that the
neutrino produced as ${|\nu_{e}\rangle}$ is detected as ${|\nu_{\mu}\rangle}$
is

$P_{(}{\nu_{e}}\rightarrow{\nu_{\mu}})={\mid|\nu_{e}\rangle\langle\nu_{\mu}(x,t)|\mid}^{2}=\sin^{2}{2\theta}\sin^{2}{\frac{\phi_{21}}{2}}$
(8)

If the neutrino produced at a space-time point $A(t_{A},\vec{x}_{A})$ and
detected at $B(t_{B},\vec{x}_{B})$, the expression for the phase [25]:

$\phi_{21}=(\frac{m_{2}^{2}}{2E_{2}}-\frac{m_{1}^{2}}{2E_{1}})\mid\vec{x}_{B}-\vec{x}_{A}\mid$
(9)

$\mid\vec{x}_{B}-\vec{x}_{A}\mid=L$, L is the length of the source from the
detector also known as the baseline; Usually this distance is much larger than
the sizes of the production and detections regions [26].

$P_{(}{\nu_{e}}\rightarrow{\nu_{\mu}})=\sin^{2}{2\theta}\sin^{2}{\frac{\Delta
m^{2}L}{4E_{0}}}$ (10)

The probability of a neutrino oscillating from one flavor to another is given
by a sinusoidal function, and it depends on parameters like the baseline
distance traveled by the neutrinos, the energy of the neutrinos, and the mass-
squared differences between the mass eigenstates. It should be noted that the
functional dependence on $L/E$ is called a spectral dependence [27, 28, 29].

However, Aydiner proposed that the Dirac equation could be written in
fractional form in the case of the neutrinos being coupled to deformed space-
time [22]:

$iD_{{t}}^{\eta}\Psi_{j}(t,x)=-\left[|P|+\frac{(m_{j})^{2}c^{3}}{2|P|}\right]\frac{\delta.P}{2|P|}\Psi_{j}(t,x)$
(11)

where $D_{{t}}^{\eta}$ denotes the Caputo fractional derivative operator,
$\eta$ denotes coupling strength or deformation of the space-time, which is
given by $0<\eta<1$ and $\Psi(t,x)$ is the wave function of the neutrino. This
equation leads to a fractional transition probability between quantum states
$|\nu_{\alpha}\rangle\rightarrow\ |\nu_{\beta}\rangle$. This probability is
given by

$P_{|\nu_{\alpha}\rangle\rightarrow\ |\nu_{\beta}\rangle}=\sum_{j,k}U_{\alpha
j}U_{\beta j^{*}}U_{\alpha k^{*}}U_{\beta
k}\exp\left[-i\left(\frac{(m_{j}^{2}-m_{k}^{2})c^{3}}{2h|P|}\right)^{\eta}t^{\eta}\right]$
(12)

which indicates anomalous cyclic behavior in the neutrino oscillation [22].
This model provides that the phase shift will not be constant as in previous
models, but the phase shift increases with each subsequent oscillation.
Additionally, it should be noted that in the Pontecorvo’s oscillation scheme,
oscillation between mass and flavour eigenstates of the neutrino is the
periodic. This Markovian behavior of the oscillation can be represented by
Eq.(10). However, Aydiner [22] suggested that the gravitational waves, the
mass distribution in the universe or decoherence and deformation of the
Hilbert space at the quantum level can cause the space-time deformations at
both the local and cosmological scales. In this case, if the neutrino mass or
flavor eigenvector is coupled to deformed space-time, time-dependent equation
of the neutrino oscillation can be given in fractional form as in Eq.(11).

###  II.2 Quantum Entanglement Between Neutrino Eigen-States

It is known that von Neumann entropy serves as a fundamental tool for
quantifying entanglement between subsystems in bipartite quantum systems. When
dealing with mixed states, where subsystems are described by density matrices,
Von Neumann entropy, denoted as $S(\rho)$, offers a precise measure of
entanglement by characterizing the degree of correlation between the
subsystems. The density matrix is a mathematical representation that describes
the quantum state of a system, taking into account the probabilities of
different possible states, and the reduced density matrix is obtained by
tracing out (or integrating over) the degrees of freedom of one of the
subsystems from the full density matrix of the entire system [30]. The Von
Neumann entropy is defined as follows:

$S(\rho)=-\text{Tr}(\rho\ln\rho)$ (13)

where $\rho$ is the density matrix of neutrino flavor states (density matrices
are normalized so that $\text{Tr}\rho=1$):

$\rho={|{\nu}_{\alpha}(t)\rangle}{\langle{\nu}_{\alpha}(t)|}$ (14)

The density matrix is a mathematical object that contains all of the
information about the state of the system. We limit ourselves to two flavors
$\nu_{\alpha}$ and $\nu_{\beta}$, we can take them as the upper and lower
components in the neutrino flavor iso-spin SU(2) algebra [31], So by assuming
the neutrino occupation number associated with a given flavor (mode) as a
reference quantum number, one can establish the following correspondences with
two-qubit states:

${|{\nu}_{\alpha}\rangle}\equiv|{1\,\,0}\rangle=\begin{pmatrix}0\\\ 1\\\ 0\\\
0\end{pmatrix},\quad{|{\nu}_{\beta}\rangle}\equiv|{0\,\,1}\rangle=\begin{pmatrix}0\\\
0\\\ 1\\\ 0\end{pmatrix}$ (15)

States $|0\rangle$ and $|1\rangle$ correspond, respectively, to the absence
and the presence of a neutrino in mode $\alpha$ or $\beta$. Entanglement is
thus established among flavor modes, in a single-particle setting. The free
evolution of the neutrino state ${|{\nu}_{\alpha}\rangle}$ can be written in
the form:

${|{\nu}_{\alpha}(t)\rangle}=\sum_{j}U_{\alpha
j}\exp^{\frac{-iE_{j}t}{h}}(\sum_{\alpha}U^{-1}_{\alpha
j}{|{\nu}_{\alpha}\rangle})$ (16)

explicit expression of the Eq.(16) is

${|{\nu}_{\alpha}(t)\rangle}=U_{\alpha\alpha}(t){|{\nu}_{\alpha}\rangle}+U_{\alpha\beta}(t){|{\nu}_{\beta}\rangle}$
(17)

where $U_{\alpha\alpha}(t)$ and $U_{\alpha\beta}(t)$ are given by

$U_{\alpha\alpha}(t)=\cos^{2}\theta\exp^{\frac{-iE_{j}t}{h}}+\sin^{2}{\theta}\exp^{\frac{-iE_{k}t}{h}},\qquad
U_{\alpha\beta}(t)=\cos\theta\sin\theta[\exp^{\frac{-iE_{k}t}{h}}-\exp^{\frac{-iE_{j}t}{h}}]$
(18)

As a result, Eq.(17) can be rewritten as

${|{\nu}_{\alpha}(t)\rangle}=U_{\alpha\alpha}(t){|{\nu}_{\alpha}\rangle}+U_{\alpha\beta}(t){|{\nu}_{\beta}\rangle}=\begin{pmatrix}0\\\
U_{\alpha\alpha}(t)\\\ U_{\alpha\beta}(t)\\\ 0\end{pmatrix}$ (19)

Hence, by inserting Eq.(19) into Eq.(14), density matrix in Eq.(14) can be
obtained as

$\rho=\begin{pmatrix}0&0&0&0\\\
0&|U_{\alpha\alpha}|^{2}&U_{\alpha\alpha}U^{*}_{\alpha\beta}&0\\\
0&U^{*}_{\alpha\alpha}U_{\alpha\beta}&|U_{\alpha\beta}|^{2}&0\\\
0&0&0&0\end{pmatrix}$ (20)

where elements of the density matrix are given by

${\mid
U_{\alpha\alpha}\mid}^{2}=\cos^{4}{\theta}+\sin^{4}{\theta}+2\cos^{2}{\theta}\cos{\frac{\Delta
m^{2}t}{2E}}$ (21)
$U_{\alpha\alpha}U_{\alpha\beta}^{*}=\sin{\theta}\cos{\theta}\left[\cos^{2}{\theta}\,e^{i\frac{\Delta
m^{2}t}{2E}}-\sin^{2}{\theta}\,e^{-i\frac{\Delta
m^{2}t}{2E}}+\sin^{2}{\theta}-\cos^{2}{\theta}\right]$ (22)
$U_{\alpha\alpha}^{*}U_{\alpha\beta}=\sin{\theta}\cos{\theta}\left[\cos^{2}{\theta}\,e^{-i\frac{\Delta
m^{2}t}{2E}}-\sin^{2}{\theta}\,e^{i\frac{\Delta
m^{2}t}{2E}}+\sin^{2}{\theta}-\cos^{2}{\theta}\right]$ (23) ${\mid
U_{\alpha\beta}\mid}^{2}=\sin^{2}{2\theta}\sin\frac{\Delta m^{2}t}{2E}$ (24)

On the other hand, reduced density matrices of matrix $\rho$ are given by

$\rho_{\alpha}=\text{Tr}_{\beta}(\rho)={\mid
U_{\alpha\alpha}\mid}^{2}{|{1}\rangle}{\langle{1}|}+{\mid
U_{\alpha\beta}\mid}^{2}{|{0}\rangle}{\langle{0}|}$ (25)

$\rho_{\beta}=\text{Tr}_{\alpha}(\rho)={\mid
U_{\alpha\alpha}\mid}^{2}{|{0}\rangle}{\langle{0}|}+{\mid
U_{\alpha\beta}\mid}^{2}{|{1}\rangle}{\langle{1}|}.$ (26)

The linear entropy associated with the reduced state after tracing over one
mode (flavor) can be computed straightforwardly (the entropies of the muon
neutrino state and the electron neutrino state yield similar results [3])

$\displaystyle S(\rho_{\alpha})$ $\displaystyle=$ $\displaystyle
S(\rho_{\beta})$ $\displaystyle=$
$\displaystyle-\left(\cos^{4}\theta+\sin^{4}\theta+2\cos^{2}\theta\sin^{2}\theta\cos\frac{\Delta
m^{2}t}{2E}\right)$ $\displaystyle\times$
$\displaystyle\log\left(\cos^{4}\theta+\sin^{4}\theta+2\cos^{2}\theta\sin^{2}\theta\cos\frac{\Delta
m^{2}t}{2E}\right)$ $\displaystyle-$
$\displaystyle\left(\sin^{2}2\theta\sin\frac{\Delta
m^{2}t}{2E}\right)\log\left(\sin^{2}2\theta\sin\frac{\Delta
m^{2}t}{2E}\right)$

In other words, entanglement entropy is equal to:

$S(\rho_{\alpha})=S(\rho_{\beta})=-(P_{sur})\log(P_{sur})-(P_{osc})\log(P_{osc})$
(28)

where $P_{sur}$ refers to the possibility in Eq.(21) that a neutrino remains
in its initial flavor state without undergoing a flavor change, and $P_{osc}$
signifies the probability in Eq.(24) that a neutrino transitions from one
flavor state to another during its propagation through space. This
relationship highlights how the entanglement entropy, as quantified by the Von
Neumann entropy, is connected to the probabilities that describe the behavior
of neutrinos in terms of flavor survival and oscillation [2].

##  III Numerical Results

In the previous Section, we discussed entanglement entropy between neutrino
eigenstates using Neumann entropy. In this section, we explore the
entanglement entropy to the phase shift of the neutrino oscillation based on
Eq.(12).

First, we consider $\nu_{e}\rightarrow\nu_{e}$ survival probability,
$\nu_{e}\rightarrow\nu_{\mu}$ transition probability and entanglement entropy
for the Eq.(10). For numerical analyses we set $\theta_{12}=33$, $\Delta
m^{2}_{12}=7.37\times 10^{-5}$ eV 2 and $c=3\times 10^{8}$ Km/s. The above
equation expresses the fact that flavor neutrino states at any time can be
regarded as entangled super-positions of the mass qubits $|{\nu_{i}}\rangle$,
where the entanglement is a function of the mixing angle only. The
entanglement entropy of the neutrino for two flavors; $\alpha=e$ and
$\beta=\mu$ is plotted in Fig.1.

Figure 1: The blue curve denotes the survival probability
$P(\nu_{e}\rightarrow\nu_{e})$, the red curve corresponds to the transition
probability from electron neutrino to a muon neutrino
$P(\nu_{e}\rightarrow\nu_{\mu})$ and the black curve corresponds to the
entropy

In this figure, we show the behavior of $S(e,\mu)$ as a function of the
scaled, dimensionless time $t$. The plots have a clear physical
interpretation, at time $t=0$, the two flavors are not mixed, the entanglement
is zero, and the global state of the system is factorized. For $t\geq 0$,
flavors start to oscillate, and the entanglement is maximal at the largest
mixing, $P(\nu_{e}\rightarrow\nu_{e})=P(\nu_{e}\rightarrow\nu_{\mu})=0.5$, and
minimum at $t=\pi$. Accordingly, entropy is maximum at points where the
probability of survival of an electron neutrino and the probability of
oscillating a muon neutrino are equal. High values of entropy suggest greater
mixing between the flavor states, indicating a more entangled quantum system.
In the area between these points, that is, in the range where the probability
of oscillation is greater than the probability of survival, entropy decreases
again. At points where the probability of survival is maximum, entropy takes
its minimum values.

Now, we numerically analyzed entanglement entropy by using Eq.(12) to see
entropy change in the presence of anomalous behavior in the neutrino
oscillations. By using Von Neumann entanglement entropy formalism, introduced
above, we computed entanglement for various $\eta$ values. The obtained
results are shown in Figs.2, 3 and 4. Firstly, we plotted the entanglement
entropy results for the transition between two flavors from $\nu_{e}$ to
$\nu_{\mu}$ for various $\eta$ in Figs.2 and 3. Additionally, we also
presented entanglement entropy results for the transition between three
flavors in Fig.4.

Figure 2: In (a), the entropy of neutrino oscillation $S(e,\mu)$ Minkowski
space-time for $\eta=1$ indicates the ordinary Minkowski space-time. The
entropy in the deformed Minkowski space-time is given in (b–d) sub-figures.
The deformation parameters of the Minkowski space-time are $\eta=0.9$ in (b)
$\eta=0.8$ in (c) and $\eta=0.7$ in (d)

In Fig.2(a) entanglement entropy is given for $\eta=1$. This figure is the
same as the entropy given in Fig.1. As seen from Fig.2(a) the distribution of
the entanglement is quite periodic and has a symmetric shape. However, for
$\eta<1$ values, it can be seen that the symmetry of the entropy distribution
is disrupted and the periodicity disappears. This situation becomes more
evident for small values of $\eta$. For instance, for $\eta=1$ in (a) while
the period of oscillation is equal to $2\pi$, for $\eta=0.9$ in (b) the period
of oscillation shifts to $2.5\pi$, for $\eta=0.8$ in (c) period shifts to
$3.25\pi$, and finally, for $\eta=0.7$ in (d) the period shifts to $4.5\pi$.
These results indicate that the entanglement entropy of each subsequent
oscillation of the neutrino increases with time in the deformed space-time
depending on the deformation parameter $\eta$. Additionally, to show phase
shift due to $\eta$ we plotted four entanglement figures as superimposed in
Fig.3. It was observed that the phase shift between eigenstates is not
constant and gradually increases with each subsequent oscillation period. This
figure may be helpful for the reader to compare the changes of entropy and
phase shift, more easily.

Figure 3: Comparison of the entropy of neutrino oscillation $S(e,\mu)$ for
various fractional parameters

Furthermore, we plotted entanglement entropy for three-flavor neutrino
oscillations for various $\eta$ parameters as seen in Fig.4. Here we computed
entanglement entropy using three flavor transition probability. As can be
seen, the results obtained for the three flavors are compatible with the
results obtained for the two flavors.

Figure 4: Comparison of the entropy of neutrino oscillation for three flavors

##  IV Conclusion

In this Letter, we discussed entanglement entropy between eigenstates of the
neutrino which moves in the deformed space-time. Before presenting the
numerical results, we introduced the theoretical framework of the neutrino
oscillation and the entanglement computing method for the neutrino
oscillations in Section II. Afterward, we briefly summarized phase shift
studies that may appear in the neutrino oscillation and fractional neutrino
oscillation probability proposed by Aydiner in Ref. [22].

We numerically obtained entanglement entropy using the fractional probability
function of the neutrino oscillation in Eq.(12) for the various values of the
deformation parameter $\eta$. As seen from Fig.2-4, we show that entanglement
entropy also shifts in the case of the presence of the phase shift in the
neutrino oscillation for two and three flavor cases unlike the conventional
approach given in Fig.1. Numerical results indicate that when the space-time
deformation increases, the entanglement entropy also increases and
asymmetrically shifts in time like fractional transition probability in
Eq.(12).

The increment of the area for the subsequent oscillation in the entanglement
entropy may denote the increment of the information entropy. This is the
result of the phase shift in the oscillation probabilities between neutrino
eigenstates. It may be early at this stage to interpret the physical
counterpart of the increase in entropy with each subsequent oscillation.
However, we can give a possible interpretation. Here, entanglement entropy in
Eq.(13) can be considered as information entropy. As known from information
theory, information entropy is equal to the thermodynamic entropy, So, the
information may lead to thermodynamic works. If such a relationship is
established between information and thermodynamics, a relationship can be
established between the increment of the entanglement entropy and cosmology.

Finally, we have noticed that entanglement is compatible with neutrino flavor
transition probabilities, as evidenced by prior studies [1, 2, 3, 4, 5, 6, 7].
Similarly, in our study, we have also shown that entanglement entropy in
Eq.(28) is compatible with flavor transition probability in Eq.(12).

## References

  * Blasone  _et al._ [2008] M. Blasone, F. Dell’Anno, S. D. Siena, and F. Illuminati, Multipartite entangled states in particle mixing, Phys. Rev. D 77, 096002 (2008).
  * Blasone  _et al._ [2009] M. Blasone, F. Dell’Anno, S. D. Siena, and F. Illuminati, Entanglement in neutrino oscillations, EPL 85, 50002 (2009).
  * Blasone  _et al._ [2010] M. Blasone, F. Dell’Anno, S. DeSiena, and F. Illuminati, On entanglement in neutrino mixing and oscillations, J. Phys.: Conf. Ser. 237, 012007 (2010).
  * Blasone  _et al._ [2013] M. Blasone, F. Dell’Anno, S. D. Siena, and F. Illuminati, Neutrino flavor entanglement, Nucl. Phys. B Proc. Suppl. 237-238, 320 (2013).
  * Blasone  _et al._ [2014a] M. Blasone, F. Dell’Anno, S. D. Siena, and F. Illuminati, Entanglement in a qft model of neutrino oscillations, Adv. High Energy Phys. 2014, 6 (2014a).
  * Blasone  _et al._ [2014b] M. Blasone, F. Dell’Anno, S. De Siena, and F. Illuminati, A field-theoretical approach to entanglement in neutrino mixing and oscillations, EPL 106, 30002 (2014b).
  * Blasone  _et al._ [2015] M. Blasone, F. Dell’Anno, S. D. Siena, and F. Illuminati, Flavor entanglement in neutrino oscillations in the wave packet description, EPL 112, 20007 (2015).
  * Kayser  _et al._ [2010] B. Kayser, J. Kopp, R. G. H. Robertson, and P. Vogel, Theory of neutrino oscillations with entanglement, Phys. Rev. D 82, 093003 (2010).
  * Martin  _et al._ [2022] J. D. Martin, A. Roggero, H. Duan, J. Carlson, and V. Cirigliano, Classical and quantum evolution in a simple coherent neutrino problem, Phys. Rev. D 105, 083020 (2022).
  * Siwach  _et al._ [2023] P. Siwach, A. Suliga, and A. Balantekin, Entanglement in three-flavor collective neutrino oscillations, Phys. Rev. D 107, 023019 (2023).
  * Amol V  _et al._ [2021] P. Amol V, C. Michael J, and A. Balantekin, Spectral splits and entanglement entropy in collective neutrino oscillations, Phys. Rev. D 104, 123035 (2021).
  * Mallick  _et al._ [2017] A. Mallick, S. Mandal, and C. Chandrashekar, Neutrino oscillations in discrete-time quantum walk framework, Eur. Phys. J. C 77, 85 (2017).
  * Cervia  _et al._ [2019] M. J. Cervia, A. V. Patwardhan, A. Balantekin, S. Coppersmith, and C. W. Johnson, Entanglement and collective flavor oscillations in a dense neutrino gas, Phys. Rev. D 100, 083001 (2019).
  * Ahluwalia and Burgard [1996] D. V. Ahluwalia and C. Burgard, Gravitationally induced neutrino-oscillation phases., Gen. Relativ. Gravit. 28, 1161–1170 (1996).
  * Ahluwalia and Burgard [1998] D. V. Ahluwalia and C. Burgard, Interplay of gravitation and linear superposition of different mass eigenstates, Phys. Rev. D 57, 4724 (1998).
  * Grossman and Lipkin [1997] Y. Grossman and H. J. Lipkin, Flavor oscillations from a spatially localized source: A simple general treatment, Phys. Rev. D 55, 2760 (1997).
  * Capozziello  _et al._ [2010] S. Capozziello, M. De Laurentis, and D. Verieri, Neutrino oscillation phase dynamically induced by $f(r)$ gravity, Mod. Phys. Lett. B 25, 1163–1168 (2010).
  * Capozziello and Lambiase [1999] S. Capozziello and G. Lambiase, Neutrino oscillation in brans-dicke theory of gravity, Modern Physics Letters A 14, 2193–2200 (1999).
  * Ren and Zhang [2010] J. Ren and C.-M. Zhang, Neutrino oscillations in the kerr–newman spacetime, Class. Quantum Grav. 27, 065011 (2010).
  * Koranga [2012] B. S. Koranga, Neutrino oscillation phase shift from quantum gravity, Int J Theor Phys 51, 3688–3693 (2012).
  * Swami [2022] H. Swami, Neutrino flavor oscillations in a rotating spacetime, Eur. Phys. J. C 82, 974 (2022).
  * Aydiner [2023] E. Aydiner, Anomalous cyclic in the neutrino oscillations, Sci. Rep. 13, 12651 (2023).
  * Gribov and Pontecorvo [1969] G. Gribov and B. Pontecorvo, Neutrino astronomy and lepton charge, Phys. Lett. B 28, 493 (1969).
  * Maki  _et al._ [1962] Z. Maki, M. Nakagawa, and S. Sakata, Remarks on the unified model of elementary particles, Prog. Theor. Phys. 28, 870 (1962).
  * Follin  _et al._ [2015] B. Follin, L. Knox, M. Millea, and Z. Pan, First detection of the acoustic oscillation phase shift expected from the cosmic neutrino background, Phys. Rev. Lett. 115, 091301 (2015).
  * Akhmedov and Smirnov [2011] E. Akhmedov and A. Smirnov, Neutrino oscillations: Entanglement, energy-momentum conservation and qft, Found Phys. 41, 1279–1306 (2011).
  * Upadhyay and Batra [2013] A. Upadhyay and M. Batra, Phenomenology of neutrino mixing in vacuum and matter, Int. Sch. Res. Notices 2013, 206516 (2013).
  * Kajita [2010] T. Kajita, Atmospheric neutrinos and discovery of neutrino oscillations, Proc Jpn Acad Ser B Phys Biol Sci. 84, 303–321 (2010).
  * Kimura and Takamura [2021] K. Kimura and A. Takamura, Exact oscillation probabilities of neutrinos in three generations derived from relativistic equation, arXiv: High Energy Physics , 24 (2021).
  * Amico  _et al._ [2008] L. Amico, R. Fazio, A. Osterloh, and V. Vedral, Entanglement in many-body systems, Rev. Mod. Phys. 80, 517 (2008).
  * Balantekin [2022] A. Balantekin, Quantum entanglement and neutrino many-body systems, J. Phys.: Conf. Ser 2191, 012004 (2022).


# Optimal Linear Subspace Search: Learning to Construct Fast and High-Quality
Schedulers for Diffusion Models

Zhongjie Duan  East China Normal UniversityShanghaiChina
<EMAIL_ADDRESS>,  Chengyu Wang  Alibaba GroupHangzhouChina
<EMAIL_ADDRESS>,  Cen Chen  East China Normal
UniversityShanghaiChina<EMAIL_ADDRESS>,  Jun Huang  Alibaba
GroupHangzhouChina<EMAIL_ADDRESS>and  Weining Qian  East China
Normal UniversityShanghaiChina<EMAIL_ADDRESS>
(2018)

###### Abstract.

In recent years, diffusion models have become the most popular and powerful
methods in the field of image synthesis, even rivaling human artists in
artistic creativity. However, the key issue currently limiting the application
of diffusion models is its extremely slow generation process. Although several
methods were proposed to speed up the generation process, there still exists a
trade-off between efficiency and quality. In this paper, we first provide a
detailed theoretical and empirical analysis of the generation process of the
diffusion models based on schedulers. We transform the designing problem of
schedulers into the determination of several parameters, and further transform
the accelerated generation process into an expansion process of the linear
subspace. Based on these analyses, we consequently propose a novel method
called Optimal Linear Subspace Search (OLSS), which accelerates the generation
process by searching for the optimal approximation process of the complete
generation process in the linear subspaces spanned by latent variables. OLSS
is able to generate high-quality images with a very small number of steps. To
demonstrate the effectiveness of our method, we conduct extensive comparative
experiments on open-source diffusion models. Experimental results show that
with a given number of steps, OLSS can significantly improve the quality of
generated images. Using an NVIDIA A100 GPU, we make it possible to generate a
high-quality image by Stable Diffusion within only one second without other
optimization techniques.

diffusion, computational efficiency, path optimization

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Applied computing Media arts

##  1\. Introduction

In recent years, diffusion models (Ho et al., 2020; Nichol and Dhariwal, 2021;
Song and Ermon, 2019) have become the most popular framework and have achieved
impressive success in image synthesis (Feng et al., 2022; Ramesh et al., 2022;
Saharia et al., 2022a). Unlike generative adversarial networks (GANs)
(Goodfellow et al., 2020), diffusion models generate high-quality images
without relying on adversarial training processes, and do not require careful
hyperparameter tuning. In addition to image synthesis, diffusion models have
also been applied to image super-resolution (Saharia et al., 2022b), music
synthesis (Liu et al., 2021) and video synthesis (Ho et al., 2022). The
research community and industry have witnessed the impressive effectiveness of
diffusion models in generative tasks.

The major weakness of diffusion models, however, is the extremely slow
sampling procedure, which limits their practicability (Yang et al., 2022). The
diffusion model is a family of iterative generative models and typically
requires hundreds to thousands of steps to generate contents. The idea of
diffusion models is inspired by the diffusion process in physics. In image
synthesis, various levels of Gaussian noise are incrementally added to images
(or latent variables corresponding to images) in a stepwise manner, while the
model is trained to denoise the samples and reconstruct the original images.
The generation process is the reverse process of diffusion process and both
processes include the same time discretization steps to ensure the
consistency. The number of steps in the training stage required is usually
very large to improve the image quality, making the generation process
extremely slow.

To tackle the efficiency problem of diffusion models, existing studies
proposed several types of methods. For instance, Salimans et al. (Salimans and
Ho, 2022) proposed a distillation method that can reduce the number of
sampling steps to half and can be applied repeatable to a model. By maximizing
sample quality scores, DDSS (Watson et al., 2022) attempted to optimize fast
samplers. However, these speedup methods require additional training, which
makes it difficult to deploy them with limited computing resources. Another
family of studies focused on designing a scheduler to control the generation
process, including DDIM (Song et al., 2020), PNDM (Liu et al., 2022), DEIS
(Zhang and Chen, 2022), etc. The schedulers can reduce the number of steps
without training but may introduce new noise when we use a small number of
steps, thus resulting in a trade-off between the computing resource
requirement and performance.

In order to speed up the generation process, we focus on devising an adaptive
scheduler that requires only a little time for learning. In this paper, we
first theoretically analyze the generation process of diffusion models. With
the given steps, most of the existing schedulers (Song et al., 2020; Karras et
al., 2022; Liu et al., 2022) generate the next sample in the linear subspace
spanned by the previous samples and model outputs, thus the generation process
is essentially the expansion process of linear subspaces. We also analyze the
correlation of intermediate variables when generating images, and we observe
that the model outputs contain redundant duplicate information at each step.
Therefore, the number of steps must be reduced to prevent redundant
calculation. In light of these analyses, we replace the coefficients in the
iterative formula with trainable parameters to control the expansion of the
linear subspaces, and then use simple least square methods (Björck, 1990; Kerr
et al., 2009) to solve these parameters. Leveraging a path optimization
algorithm, we further improve the performance by tuning the sampling steps.
Path optimization and parameter solving can be performed within only a few
minutes, thus it is easy to train and deploy. Our proposed scheduler, named
Optimal Linear Subspace Search (OLSS), is designed to be lightweight and
adaptive. We apply OLSS to several popular open-source diffusion models
(Rombach et al., 2022) and compare the performance of OLSS with the state-of-
the-art schedulers. Experimental results prove that the approximate generation
process built by OLSS is an accurate approximation of the complete generation
process. The source code of our proposed method will be released on
GitHub111https://github.com/alibaba/EasyNLP/tree/master/diffusion. The main
contribution of this paper includes:

  * •

We theoretically and empirically analyze the generation process of diffusion
models and model it as an expansion process of linear subspaces. The analysis
provides valuable insights for researchers to design new diffusion schedulers.

  * •

Based on our analysis, we propose a novel scheduler, OLSS, which is capable of
finding the optimal path to approximate the complete generation process and
generating high-quality images with very few steps.

  * •

Using popular diffusion models, we benchmark OLSS against existing schedulers.
The experimental results demonstrate that OLSS achieves the highest image
quality with the same number of steps.

##  2\. Related Work

###  2.1. Image Synthesis

Image synthesis has been widely investigated. In early years, GANs (Reed et
al., 2016; Huang et al., 2017; Odena et al., 2017) are the most popular
methods. By adversarial training a generator and a discriminator, we can
obtain a network that can directly generate images. However, it is difficult
to stabilize the training process (Salimans et al., 2016). Diffusion models
overcome this issue by modeling the synthesis task as a Markovian diffusion
process, including Denoising Diffusion Probabilistic Models (Sohl-Dickstein et
al., 2015), Score-Based Generative Models (Song and Ermon, 2019), etc.
Leveraging large-scale text-image datasets (Gu et al., 2022; Schuhmann et al.,
2022), diffusion models with billions of parameters have achieved impressive
success in text-to-image synthesis. Given text prompts, the diffusion models
can generate semantically meaningful images with text-guided mechanisms (Ho
and Salimans, 2022). Diffusion models are proved to outperform GANs in image
quality (Dhariwal and Nichol, 2021), and are even competitive with human
artists (Feng et al., 2022; Ramesh et al., 2022; Saharia et al., 2022a).
However, the slow sampling procedure becomes a critical issue.

###  2.2. Efficient Sampling for Diffusion Models

To speed up the generation process of diffusion models, existing studies focus
on reducing the number of inference steps. Specifically, some schedulers are
proposed for controlling this denoising process. DDIM (Denoising Diffusion
Implicit Models) (Song et al., 2020) is a straightforward scheduler that
converts the stochastic process to a deterministic process and skips some
steps. Some numerical ODE algorithms (Butcher, 2000; Karras et al., 2022) are
also introduced to improve efficiency. Liu et al. (Liu et al., 2022) pointed
out that numerical ODE methods may introduce additional noise and are
therefore less efficient than DDIM with only a small number of steps. To
overcome this pitfall, they modified the iterative formula and improved their
effectiveness. DEIS (Diffusion Exponential Integrator Sampler) (Zhang and
Chen, 2022), another study based on ODE, leverages an exponential integrator
and a semilinear structure, stabilizing the approximated generation process.
DPM-Solver (Lu et al., 2022a) made further refinements by calculating a part
of the solution analytically. Recently, an enhanced version (Lu et al., 2022b)
of DPM-Solver adopted thresholding methods and achieved state-of-the-art
performance.

##  3\. Methodology

This section begins with a brief theoretical review of diffusion models and a
summary of the central issues in scheduler design. Next, we empirically
analyze the generation process of general diffusion models. Finally, we
propose our scheduler based on these existing studies.

###  3.1. Review of Diffusion Models

Different from GAN-based generative models, diffusion-based models require
multi-step inference. The iterative generation process significantly increases
the computation time. In the training stage, the number of steps may be very
large. For example, the number of steps in Stable Diffusion (Rombach et al.,
2022) while training is $1000$.

In the complete generation process, starting from random Gaussian noise
$\boldsymbol{x}_{T}$, we need to calculate
$\boldsymbol{x}_{T-1},\dots,\boldsymbol{x}_{0}$ step by step, where $T$ is the
total number of steps. At each step $t$, the diffusion model
$\epsilon_{\theta}$ takes $\boldsymbol{x}_{t}$ as input and output
$\boldsymbol{e}_{t}=\epsilon_{\theta}(\boldsymbol{x}_{t},t)$. Thus, we obtain
$\boldsymbol{x}_{t-1}$ via:

(1)
$\begin{split}\boldsymbol{x}_{t-1}=\sqrt{\alpha_{t-1}}\underbrace{\left(\frac{\boldsymbol{x}_{t}-\sqrt{1-\alpha_{t}}\boldsymbol{e}_{t}}{\sqrt{\alpha_{t}}}\right)}_{\text{predicted
}\boldsymbol{x}_{0}}+\underbrace{\sqrt{1-\alpha_{t-1}-\sigma_{t}^{2}}\boldsymbol{e}_{t}}_{\text{direction
pointing to
}\boldsymbol{x}_{t}}+\underbrace{\sigma_{t}\boldsymbol{\epsilon}_{t}}_{\text{random
noise}},\end{split}$

where $\alpha_{t},\sigma_{t}$ are hyper-parameters used for training. Note
that $\sigma_{t}\boldsymbol{\epsilon}_{t}$ is the additional random noise to
increase the diversity of generated results. In DDIM (Song et al., 2020),
$\sigma_{t}$ is set to $0$, making this process deterministic given
$\boldsymbol{x}_{T}$.

To reduce the steps, in most existing schedulers, a few steps
$t(1),\dots,t(n)$ are selected as a sub-sequence of $\\{T,T-1,\dots,0\\}$, and
the scheduler only calls the model to calculate $\boldsymbol{e}_{t(i)}$ at
these $n$ steps. For example, DDIM directly transfers Formula (1) to an
$n$-step generation process:

(2)
$\begin{split}\boldsymbol{x}_{t(i+1)}=\sqrt{\alpha_{t(i+1)}}\left(\frac{\boldsymbol{x}_{t(i)}-\sqrt{1-\alpha_{t(i)}}\boldsymbol{e}_{t(i)}}{\sqrt{\alpha_{t(i)}}}\right)+\sqrt{1-\alpha_{t(i+1)}}\boldsymbol{e}_{t(i)}.\end{split}$

The final latent variable $\boldsymbol{x}_{0}$ obtained by DDIM is an
approximate result of that in the complete generation process. Another study
(Karras et al., 2022) focuses on modeling the generation process as an
ordinary differential equation (ODE) (Butcher, 2000). Consequently, forward
Euler, a numerical ODE algorithm, can be employed to calculate the numerical
solution of $\boldsymbol{x}_{0}$:

(3)
$\boldsymbol{x}_{t(i+1)}=\boldsymbol{x}_{t(i)}+\Big{(}t(i+1)-t(i)\Big{)}\frac{\mathrm{d}\boldsymbol{x}_{t(i)}}{\mathrm{d}t(i)}.$

PNDM (Liu et al., 2022) is another ODE-based scheduler. It leverages Linear
Multi-Step Method and constructs a pseudo-numerical method. We simplify the
iterative formula as:

(4)
$\boldsymbol{x}_{t(i+1)}=\frac{\sqrt{\alpha_{t(i+1)}}}{\sqrt{\alpha_{t(i)}}}\boldsymbol{x}_{t(i)}-\frac{1}{\sqrt{\alpha_{t(i)}}}\alpha_{t(i)}^{\prime}\boldsymbol{e}_{t(i)}^{\prime},$

where

(5)
$\boldsymbol{e}_{t(i)}^{\prime}=\frac{1}{24}(55\boldsymbol{e}_{t(i)}-59\boldsymbol{e}_{t(i-1)}+37\boldsymbol{e}_{t(i-2)}-9\boldsymbol{e}_{t(i-3)}),$
(6)
$\alpha_{t(i)}^{\prime}=\frac{\alpha_{t(i+1)}-\alpha_{t(i)}}{\sqrt{(1-\alpha_{t(i+1)})\alpha_{t(i)}}+\sqrt{(1-\alpha_{t(i)})\alpha_{t(i+1)}}}.$

Figure 1. The heat map of the correlation coefficients of latent variables,
which records the whole generation process.

In DDIM (2) and forward Euler (3), $\boldsymbol{x}_{t(i+1)}$ is a linear
combination of $\\{\boldsymbol{x}_{t(i)},\boldsymbol{e}_{t(i)}\\}$. In PNDM
(4), $\boldsymbol{x}_{t(i+1)}$ is a linear combination of
$\\{\boldsymbol{x}_{t(i)},\boldsymbol{e}_{t(i)},\boldsymbol{e}_{t(i-1)},\boldsymbol{e}_{t(i-2)},\boldsymbol{e}_{t(i-3)}\\}$.
Generally, all these schedulers satisfy

(7)
$\boldsymbol{x}_{t(i+1)}\in\text{span}\\{\boldsymbol{x}_{t(i)},\boldsymbol{e}_{t(1)},\boldsymbol{\dots},\boldsymbol{e}_{t(i)}\\}.$

Recursively, we can easily prove that

(8)
$\boldsymbol{x}_{t(i+1)}\in\text{span}\\{\boldsymbol{x}_{t(1)},\boldsymbol{e}_{t(1)},\boldsymbol{\dots},\boldsymbol{e}_{t(i)}\\}.$

Therefore, the generation process is the expansion process of a linear
subspace. This linear subspace is spanned by the initial Gaussian noise and
the previous model outputs. At each step, we obtain the model output and add
it to the vector set. The core issue of designing a scheduler is to determine
the coefficients in the iterative formula. The number of non-zero coefficients
does not exceed $\frac{1}{2}n^{2}+\frac{3}{2}n$.

###  3.2. Empirical Analysis of Generation Process

To empirically analyze what happens in the whole generation process, we use
Stable Diffusion to generate several images and store the latent variables,
including
$\boldsymbol{x}_{T},\dots,\boldsymbol{x}_{0},\boldsymbol{e}_{T},\dots,\boldsymbol{e}_{1}$.
As shown in Figure 1, we plot a heat map showing the correlation coefficients
between these variables. We have the following findings:

  1. (1)

$\boldsymbol{x}_{t}$ is similar to that in neighbouring steps and differs from
that in non-neighbouring steps. As the denoising process proceeds,
$\boldsymbol{x}_{t}$ is updated continuously.

  2. (2)

The correlation between $\boldsymbol{x}_{t}$ and the predicted noise
$\boldsymbol{e}_{t}$ is strong in the beginning and becomes weak in the end.
The reason is that $\boldsymbol{x}_{t}$ consists of little noise in the last
few steps.

  3. (3)

The correlation between $\boldsymbol{e}_{T},\dots,\boldsymbol{e}_{1}$ is
significantly strong, indicating that the outputs of the model contain
redundant duplicate information.

Figure 2. Three methods to generate $\boldsymbol{x}_{t(i+1)}$ starting from
$\boldsymbol{x}_{t(i)}$, where the method (c) is the one that we adopt.

###  3.3. Constructing a Faster Scheduler

On the basis of existing schedulers and our analysis, we propose OLSS, a new
diffusion scheduler. In this method, we first run the complete generation
process to catch the intermediate variables and then construct an approximate
process using these variables, instead of leveraging mathematical theories to
design an iterative formula.

####  3.3.1. Skip Estimation

Assume that we have selected $n$ steps $\\{t(1),\dots,t(n)\\}$, which is a
sub-sequence of $\\{T,T-1,\dots,0\\}$ with $t(1)=T$. Calling the diffusion
model for inference is only allowed at these $n$ steps. At the $i$-th step, we
have obtained intermediate variables
$\boldsymbol{x}_{t(1)},\dots,\boldsymbol{x}_{t(i)}$ and
$\boldsymbol{e}_{t(1)},\dots,\boldsymbol{e}_{t(i)}$. In Figure 2, we present
three methods to generate $\boldsymbol{x}_{t(i+1)}$. The complete generation
process is presented in Figure 2 (a). It needs to call the model for
$t(i+1)-t(i)-1$ times. To reduce the time consumption, we can naturally come
up with a naive method. As we mentioned in Section 3.2, the correlation
between $\boldsymbol{e}_{T},\dots,\boldsymbol{e}_{1}$ is significantly strong,
thus we can estimate the model’s output $\boldsymbol{e}_{t(i)-1}$ using a
simple linear model trained with intermediate variables. Formally, let

(9)
$\hat{\boldsymbol{e}}_{t(i)-1}=\mathop{\arg\min}_{\boldsymbol{e}\in\mathcal{E}}||\boldsymbol{e}-\boldsymbol{e}_{t(i)-1}||_{2}^{2},$

where the feasible region is

(10)
$\mathcal{E}=\text{span}\\{\boldsymbol{x}_{t(1)},\boldsymbol{e}_{t(1)},\dots,\boldsymbol{e}_{t(i)}\\}.$

In other words, we use the orthogonal projection of $\boldsymbol{e}_{t(i)-1}$
in $\mathcal{E}$ as the estimation of $\boldsymbol{e}_{t(i)-1}$. Similarly, we
can obtain all estimated intermediate variables in the missing steps before
$t(i+1)$ and finally calculate $\hat{\boldsymbol{x}}_{t(i+1)}$. The flow
diagram of this naive method is shown in Figure 2 (b). According to Equations
(8, 9, 10), it is obvious to see the estimated $\boldsymbol{x}_{t(i+1)}$ still
satisfies Equation (8). However, note that the estimation may have non-
negligible errors and the errors are accumulated into subsequent steps. To
address this issue, we design a simplified end-to-end method directly
estimating $\boldsymbol{x}_{t(i+1)}$. The simplified method in Figure 2 (c)
only contains one linear model, containing $i+1$ coefficients
$w_{i,0},w_{i,1},\dots,w_{i,i}$. The estimated $\boldsymbol{x}_{t(i+1)}$ is
formulated as:

(11)
$\hat{\boldsymbol{x}}_{t(i+1)}=w_{i,0}\boldsymbol{x}_{t(1)}+\sum_{j=1}^{i}w_{i,j}\boldsymbol{e}_{t(j)}.$

Leveraging least square methods (Björck, 1990), we can easily minimize the
mean square error
$||\hat{\boldsymbol{x}}_{t(i+1)}-\boldsymbol{x}_{t(i+1)}||_{2}^{2}$. Note that
we use only $\boldsymbol{x}_{t(1)}$ (i.e., $\boldsymbol{x}_{T}$) instead of
$\\{\boldsymbol{x}_{t(1)},\dots,\boldsymbol{x}_{t(i)}\\}$ in Equation (11),
because
$\\{\boldsymbol{x}_{T},\boldsymbol{e}_{t(1)},\dots,\boldsymbol{e}_{t(i)}\\}$
is a linearly independent set and
$\\{\boldsymbol{x}_{t(2)},\dots,\boldsymbol{x}_{t(i)}\\}$ can be linearly
represented by these vectors. Additionally, the linear independence makes it
easier to solve the least squares problem using QR-decomposition algorithms
(Kerr et al., 2009), which is faster and more numerically stable than directly
computing the pseudo-inverse matrix (Regalia, 1993).

Algorithm 1 Path optimization with error upper bound $D$

1: Input: Error upper bound $D$

2: $t(1)\leftarrow T$

3: for $i=1,2,\dots,n$ do

4: Finding next step $t(i+1)$ using binary search w.r.t. $D$

5: if $t(i+1)$ does not exist then

6: return false

7: end if

8: end for

9: if $t(n+1)>0$ then

10: return false

11: else

12: return $\\{t(1),t(2),\dots,t(n)\\}$

13: end if

####  3.3.2. Path Optimization

Another problem to be addressed is how to select the $n$ steps
$\\{t(1),\dots,t(n)\\}$. In Section 3.2, we observe that the correlation
between $\boldsymbol{x}_{t}$ and $\boldsymbol{e}_{t}$ is different at each
step, indicating that the difficulty level of generating low-error
$\boldsymbol{x}_{t}$ is also varied. In most existing schedulers, these steps
are selected in $\\{T,T-1,\dots,0\\}$ with equal intervals. For example, in
PNDM, Equation (5) comes from the Linear Multi-Step Method, which enforces
that the steps must be uniformly selected. However, there is no restriction on
the selection of steps in our method. We can search for the optimal path to
generate high-quality images.

For convenience, we use $d(t(1),\dots,t(i+1))$ to denote the distance from
$\boldsymbol{x}_{t(i+1)}$ to its orthogonal projection in
$\text{span}\\{\boldsymbol{x}_{t(1)},\boldsymbol{e}_{t(1)},\dots,\boldsymbol{e}_{t(i)}\\}$
(i.e., the error of $\hat{\boldsymbol{x}}_{t(i+1)}$). We add an additional
step $t(n+1)=0$. Setting an error upper bound $D$, we hope the error of every
step does not exceed $D$. On the one hand, we need to reduce the errors to
generate high-quality images; we also have to ensure that the path ends with
$0$. Thus, we model the step-searching problem as an optimization problem:

(12) $\min t(i+1)\ \ \ \text{s.t.}\begin{cases}0\leq t(i+1)\leq t(i),\\\
d(t(1),\dots,t(i+1))\leq D.\end{cases}$

According to the conclusions in Section 3.2, the more steps we skip, the
larger errors we have. Therefore, we can use a simple binary search algorithm
to solve the optimization problem (12). Based on this step-searching
algorithm, we subsequently design a path-searching algorithm. The pseudo-code
of finding the path with error limitation is presented in Algorithm 1. Such a
path always exists if $D$ is sufficiently large, thus we can use another
binary search algorithm to compute the minimal error upper bound when a path
exists. Finally, we obtain the optimal path for constructing the generation
process.

###  3.4. Efficiency Analysis

We analyze the time complexity of constructing an OLSS scheduler. The most
time-consuming component is solving the least square problem. We need to solve
the least square problem $\mathcal{O}(n)$ times if the path is fixed, and
$\mathcal{O}(n\log\frac{1}{\epsilon}\log T)$ times to perform path
optimization, where $\epsilon$ is the absolute error of optimal $D$.
Empirically, when we use OLSS to improve the efficiency of Stable Diffusion,
usually a few minutes of computation on CPU is sufficient to compute the
optimal path and solve all the parameters $\\{w_{i,j}\\}$, after all the
required intermediate variables are collected from the complete generation
process.

##  4\. Experiments

To demonstrate the effectiveness of OLSS, we conduct comparative experiments.
We further investigate the factors that affect the quality of images generated
by OLSS.

###  4.1. Comparison of Diffusion Schedulers

We compare OLSS with $7$ baseline schedulers, including its variant OLSS-P and
$6$ existing schedulers.

  * •

OLSS-P: A variant of our proposed OLSS that selects the sampling steps
uniformly instead of using the path optimization algorithm.

  * •

DDIM (Song et al., 2020): A straightforward method that directly skips some
steps in the generation process. It has been widely used in many diffusion
models.

  * •

Euler (Karras et al., 2022): A general algorithm of calculating ODEs’
numerical solution. We follow the implementation of
k-diffusion222https://github.com/crowsonkb/k-diffusion and use the ancestral
sampling version.

  * •

PNDM (Liu et al., 2022): An pseudo numerical method that improves the
performance of Linear Multi-Step method.

  * •

DEIS (Zhang and Chen, 2022): A fast high-order solver designed for diffusion
models. It consists of an exponential integrator and a semi-linear structure
to reduce the error.

  * •

DPM-Solver (Lu et al., 2022a): A high-order method that analytically computes
the linear part of the latent variables.

  * •

DPM-Solver++ (Lu et al., 2022b): An enhanced scheduler based on DPM-Solver. It
solves ODE with a data prediction model and uses thresholding methods. DPM-
Solver++ is the most recent state-of-the-art scheduler.

####  4.1.1. Experimental Settings

Table 1. The FID $\downarrow$ scores between diffusion schedulers with different steps. The best results are in bold, and the second best are underscored. ${\dagger}$: the generation result of 1000-step DDIM is the most high-quality irrespective of the computational efficiency. Model | Steps / Scheduler | 100 Steps | 1000 Steps†
---|---|---|---
DDIM | Euler | PNDM | DEIS |  | DPM
---
-Solver
| DPM
---
-Solver++
DDIM
Stable Diffusion ($512\times 512$) | 5 Steps | DDIM | 82.62 | 78.14 | 84.40 | 84.91 | 85.00 | 85.24 | 84.55
Euler | 132.19 | 111.18 | 134.62 | 135.06 | 135.28 | 135.72 | 134.28
PNDM | 75.53 | 89.53 | 74.29 | 75.50 | 75.31 | 75.13 | 75.41
DEIS | 56.31 | 55.89 | 58.44 | 58.03 | 58.21 | 58.54 | 57.68
DPM-Solver | 55.39 | 55.94 | 57.49 | 57.09 | 57.27 | 57.59 | 56.76
DPM-Solver++ | 54.78 | 56.11 | 56.82 | 56.41 | 56.59 | 56.91 | 56.08
OLSS-P | 48.67 | 61.27 | 48.62 | 48.80 | 48.77 | 48.72 | 48.94
OLSS | 48.34 | 61.11 | 48.40 | 48.61 | 48.60 | 48.57 | 48.79
10 Steps | DDIM | 52.52 | 59.87 | 53.71 | 54.37 | 54.46 | 54.66 | 54.13
Euler | 79.63 | 60.04 | 81.97 | 81.70 | 81.91 | 82.33 | 81.32
PNDM | 55.77 | 72.31 | 54.84 | 56.01 | 55.88 | 55.77 | 56.03
DEIS | 39.11 | 55.07 | 41.75 | 40.96 | 41.13 | 41.40 | 40.97
DPM-Solver | 38.93 | 55.33 | 41.53 | 40.75 | 40.92 | 41.17 | 40.73
DPM-Solver++ | 38.42 | 55.72 | 41.06 | 40.17 | 40.33 | 40.59 | 40.02
OLSS-P | 37.33 | 59.28 | 38.64 | 38.36 | 38.36 | 38.38 | 38.60
OLSS | 36.46 | 58.09 | 37.68 | 37.66 | 37.66 | 37.67 | 37.79
Stable Diffusion 2 ($768\times 768$) | 5 Steps | DDIM | 93.50 | 80.83 | 95.12 | 98.10 | 98.16 | 98.38 | 97.62
Euler | 170.79 | 119.12 | 173.09 | 175.24 | 175.28 | 175.51 | 174.43
PNDM | 105.08 | 99.88 | 105.82 | 108.61 | 108.54 | 108.67 | 108.28
DEIS | 55.88 | 58.62 | 57.30 | 59.04 | 59.20 | 59.50 | 58.65
DPM-Solver | 54.90 | 58.77 | 56.31 | 58.02 | 58.18 | 58.48 | 57.63
DPM-Solver++ | 54.25 | 59.13 | 55.67 | 57.30 | 57.45 | 57.75 | 56.91
OLSS-P | 50.03 | 65.20 | 51.38 | 51.95 | 51.87 | 51.83 | 51.95
OLSS | 49.18 | 64.89 | 50.64 | 51.17 | 51.11 | 51.07 | 51.11
10 Steps | DDIM | 55.32 | 62.05 | 56.01 | 58.74 | 58.83 | 59.03 | 58.47
Euler | 96.86 | 61.75 | 98.57 | 100.88 | 100.99 | 101.26 | 100.15
PNDM | 56.44 | 64.58 | 56.94 | 59.52 | 59.62 | 59.81 | 59.28
DEIS | 39.83 | 60.85 | 42.11 | 42.59 | 42.74 | 43.02 | 42.33
DPM-Solver | 39.70 | 61.22 | 41.93 | 42.41 | 42.56 | 42.81 | 42.17
DPM-Solver++ | 39.40 | 61.81 | 41.62 | 41.89 | 42.04 | 42.30 | 41.66
OLSS-P | 38.39 | 64.21 | 40.46 | 40.70 | 40.70 | 40.74 | 40.63
OLSS | 38.10 | 63.81 | 40.15 | 40.46 | 40.48 | 40.54 | 40.45
Table 2. The FID $\downarrow$ scores of diffusion schedulers with different steps on two real-world datasets. The underlying diffusion model is fine-tuned for 5,000 steps. Dataset | Scheduler | Steps
---|---|---
5 Steps | 10 Steps
CelebA-HQ | DDIM | 80.55 | 31.72
Euler | 83.80 | 35.85
PNDM | 57.63 | 25.33
DEIS | 24.12 | 11.68
DPM-Solver | 23.03 | 11.68
DPM-Solver++ | 21.82 | 11.44
OLSS-P | 14.24 | 11.40
OLSS | 11.65 | 11.37
LSUN-Church | DDIM | 109.57 | 38.55
Euler | 216.07 | 84.21
PNDM | 29.55 | 14.58
DEIS | 55.27 | 13.67
DPM-Solver | 51.66 | 12.99
DPM-Solver++ | 48.93 | 11.77
OLSS-P | 19.46 | 11.10
OLSS | 15.18 | 10.21

The comparative experiments consist of two parts. The first part is to
benchmark the speed-up effect of these schedulers on open-domain image
synthesis and analyze the relationship between different schedulers. We
compare the schedulers on two popular large-scale diffusion models in research
community, including Stable Diffusion333https://huggingface.co/CompVis/stable-
diffusion-v1-4 and Stable Diffusion
2444https://huggingface.co/stabilityai/stable-diffusion-2-1. The architecture
of both models consists of a CLIP-based text encoder (Radford et al., 2021), a
U-Net (Ronneberger et al., 2015), and an autoencoder (Kingma and Welling,
2013), where the U-Net is trained to capture the pattern of noise. The
implementation of baseline schedulers is mainly based on Diffusers (von Platen
et al., 2022). We randomly sample $1000$ prompts in LAION-Aesthetics
V2555https://laion.ai/blog/laion-aesthetics as the conditional information
input to models. The guidance scale is set to $7.0$. The second part is to
further investigate the effect of these schedulers on close-domain image
synthesis. We fine-tune Stable Diffusion on CelebA-HQ (Karras et al., 2017)
($256\times 256$) and LSUN-Church (Yu et al., 2015) ($256\times 256$) for 5000
steps respectively. CelebA-HQ is a high-quality version of CelebA, which is a
human face dataset. LSUN-Church is a part of LSUN and includes photos of
churches. The training and generating process is performed without textual
conditional information. We generate images using the fine-tuned model and
compare them with real-world images in each dataset. In both two parts of the
experiments, in order to avoid the influence of random noise on the
experimental results, we use the same random seed and the same pre-generated
$\boldsymbol{x}_{T}$ for every scheduler. For DPM-Solver and DPM-Solver++, we
use their multi-step version to bring out their best generative effects. For
OLSS, we run the complete generation process to generate $32$ images, and then
let our algorithm construct the approximate process. Furthermore, We
investigate the improvement made by the path optimization algorithm. The
variant of OLSS that uniformly selects steps (same as DDIM) is denoted as
OLSS-P.

####  4.1.2. Evaluation Metrics

We use FID (Frechet Inception Distance) (Heusel et al., 2017), i.e., the
Frechet Distance of features extracted by Inception V3 (Szegedy et al., 2016),
to measure the similarity between two sets of images. A smaller FID indicates
that the distributions of the two sets of images are more similar. In the
first part, For each scheduler, we run the generation program with 5 steps and
10 steps respectively. Considering that the generation process with fewer
steps is an approximate process of the complete generation process, we compare
each one with the complete generation process (i.e., DDIM, 1000 steps).
Additionally, we also compare each one with the $100$-step schedulers to
investigate the consistency. In the second part, we compute the FID scores
between 10,000 generated images and real-world images. The experimental
results of the two parts are shown in Table 1 and Table 2.

####  4.1.3. Experimental Results

In Table 1, we can clearly see that OLSS reaches the best performance with the
same steps. The FID between images generated by OLSS and those by 1000-step
DDIM is lower than any other schedulers, and the gap is significant when we
use only 5 steps. Considering the consistency of different schedulers, we
observe that most schedulers generate similar images with the same settings
except Euler. The FID of Euler is even larger than that of DDIM. This is
because Euler’s method computes the numerical solution of $\boldsymbol{x}_{0}$
iteratively along a straight line (Liu et al., 2022), making the solution far
from the origin $\boldsymbol{x}_{0}$. PNDM, another ODE-based scheduler,
overcomes this pitfall by adjusting the linear transfer part in the iterative
formula. Comprehensively, DPM-Solver++ performs the best among all baseline
methods but still cannot outperform our method. Comparing OLSS and OLSS-P, we
find that OLSS performs better than OLSS-P. It indicates that the path
optimization algorithm further improves the performance. We also conduct a
_paired t-test_ on FID scores of OLSS and the best baseline scheduler. The
$p$-value is $0.0047$, indicating that the _t-test_ rejects the null
hypothesis at the significance level $<0.5\%$. This shows OLSS is the best
approximation of the complete generation process.

In Table 2, the FID scores of OLSS are also the lowest. Even without the path
optimization algorithm, OLSS-P can still outperform other schedulers. The gap
between OLSS and other schedulers is more significant than that in Table 1,
indicating that OLSS is more effective in generating images with a similar
style. The main reason is that the generation process of images in a close
domain follows a domain-specific pattern, and the learnable parameters in OLSS
make it more suitable for the generation task. Additionally, the FID of PNDM
is lower than DDIM, which is different from the first part of the experiments.
We suspect that PNDM constructs a new generation pattern to generate realistic
images rather than constructing an approximate process of the original
process.

###  4.2. Efficiency Study

Figure 3. The FID scores (between images generated by the approximate generation process and the complete generation process) and inference time of OLSS. The inference time is the averaged result after repeating the generation process 1000 times. Figure 4. The generation path $\\{\boldsymbol{x}_{t(0)},\boldsymbol{x}_{t(1)},\dots,\boldsymbol{x}_{t(n)}\\}$ of three schedulers. We embed the latent variables to 2D using PCA to see the generation process intuitively. |  |  |
---|---|---|---
DDIM, 5 steps | DPM-Solver++, 5 steps | OLSS, 5 steps | DDIM, 1000 steps
(a) Prompt: “Fantasy magic castle on floating island. A very beautiful art
painting.”
|  |  |
DDIM, 5 steps | DPM-Solver++, 5 steps | OLSS, 5 steps | DDIM, 1000 steps
(b) Prompt: “The leader of the wolves bared its ferocious fangs. High-
resolution digital painting.”
Figure 5. Some examples generated by Stable Diffusion 2 with different
schedulers and steps. For more examples, please refer to our supplementary
materials.

When a scheduler is applied to a diffusion model, there is a trade-off between
efficiency and quality. Fewer steps can reduce the inference time but usually
results in lower image quality. With the same settings as above, we apply OLSS
to Stable Diffusion and calculate the FID scores with varying numbers of
steps. We run the program on an NVIDIA A100 GPU and record the time of
generating an image. The results are plotted in Figure 3. As the number of
steps increases, the FID score decreases rapidly at the beginning and
gradually converges. The inference time increases almost linearly with the
number of steps. Setting the number of steps to $5$, OLSS is able to generate
an image within only one second, while still achieving satisfactory quality.

###  4.3. Visualization

To intuitively see how diffusion models generate images with different
schedulers, we select $32$ examples randomly generated by Stable Diffusion in
the above experiments. We catch the generation path
$\\{\boldsymbol{x}_{t(0)},\boldsymbol{x}_{t(1)},\dots,\boldsymbol{x}_{t(n)}\\}$
and then embed these latent variables into a 2D plane using Principal
Component Analysis (PCA). The embedded generation paths of three schedulers
are shown in Figure 4. Starting from the same Gaussian noise
$\boldsymbol{x}_{t(0)}$, these generation processes finally reach different
$\boldsymbol{x}_{t(n)}$. In the complete generation process,
$\boldsymbol{x}_{i}$ is updated gradually along a curve. The three 10-step
schedulers construct approximate generation process. We can see that the
errors in the beginning steps are accumulated in the subsequent steps, thus
the errors at the final steps become larger than those at the beginning. The
generation path of OLSS is the closest one to the complete generation process,
and the generation path of DPM-Solver++ is the second closest.

###  4.4. Case Study

In Figure 5, we present examples of images generated using Stable Diffusion 2
with DDIM, DPM-Solver++, and OLSS. Benefiting from the excellent generation
ability of Stable Diffusion 2, we can generate exquisite artistic images. With
only 5 steps, DDIM may make destructive changes to images. For instance, the
castle in the first example looks smoggy. DPM-Solver++ tends to sharpen the
entity at the expense of fuzzing details up. Only OLSS can clearly preserve
the texture in the generated images (see the fur of the wolf in the second
example). Despite being generated with significantly fewer steps than
1000-step DDIM, the images generated by OLSS still look satisfactory. Hence,
OLSS greatly improves the quality of generated images with very few steps.

##  5\. Conclusion and Future Work

In this paper, we investigate the schedulers in diffusion models.
Specifically, we propose a new scheduler (OLSS) that is able to generate high-
quality images within a very small number of steps. OLSS is a simple yet
effective method that utilizes linear models to determine the coefficients in
the iterative formula, instead of using mathematical theories. Leveraging the
path optimization algorithm, OLSS can construct a faster process as an
approximation of the complete generation process. The experimental results
demonstrate that the quality of images generated by OLSS is higher than the
existing schedulers with the same steps. In future work, we will continue
investigating the generation process and explore improving the generative
quality based on the modification in the latent space.

## References

  * (1)
  * Björck (1990) Åke Björck. 1990\.  Least squares methods.  _Handbook of numerical analysis_ 1 (1990), 465–652.
  * Butcher (2000) John C Butcher. 2000\.  Numerical methods for ordinary differential equations in the 20th century.  _J. Comput. Appl. Math._ 125, 1-2 (2000), 1–29.
  * Dhariwal and Nichol (2021) Prafulla Dhariwal and Alexander Nichol. 2021.  Diffusion models beat gans on image synthesis.  _Advances in Neural Information Processing Systems_ 34 (2021), 8780–8794.
  * Feng et al. (2022) Zhida Feng, Zhenyu Zhang, Xintong Yu, Yewei Fang, Lanxin Li, Xuyi Chen, Yuxiang Lu, Jiaxiang Liu, Weichong Yin, Shikun Feng, et al. 2022\.  ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts.  _arXiv preprint arXiv:2210.15257_ (2022).
  * Goodfellow et al. (2020) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020\.  Generative adversarial networks.  _Commun. ACM_ 63, 11 (2020), 139–144.
  * Gu et al. (2022) Jiaxi Gu, Xiaojun Meng, Guansong Lu, Lu Hou, Minzhe Niu, Hang Xu, Xiaodan Liang, Wei Zhang, Xin Jiang, and Chunjing Xu. 2022\.  Wukong: 100 million large-scale chinese cross-modal pre-training dataset and a foundation framework.  _arXiv preprint arXiv:2202.06767_ (2022).
  * Heusel et al. (2017) Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. 2017\.  Gans trained by a two time-scale update rule converge to a local nash equilibrium.  _Advances in neural information processing systems_ 30 (2017).
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising diffusion probabilistic models.  _Advances in Neural Information Processing Systems_ 33 (2020), 6840–6851.
  * Ho and Salimans (2022) Jonathan Ho and Tim Salimans. 2022.  Classifier-free diffusion guidance.  _arXiv preprint arXiv:2207.12598_ (2022).
  * Ho et al. (2022) Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. 2022.  Video diffusion models.  _arXiv preprint arXiv:2204.03458_ (2022).
  * Huang et al. (2017) Xun Huang, Yixuan Li, Omid Poursaeed, John Hopcroft, and Serge Belongie. 2017.  Stacked generative adversarial networks. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 5077–5086.
  * Karras et al. (2017) Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017\.  Progressive growing of gans for improved quality, stability, and variation.  _arXiv preprint arXiv:1710.10196_ (2017).
  * Karras et al. (2022) Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. 2022\.  Elucidating the design space of diffusion-based generative models.  _arXiv preprint arXiv:2206.00364_ (2022).
  * Kerr et al. (2009) Andrew Kerr, Dan Campbell, and Mark Richards. 2009\.  QR decomposition on GPUs. In _Proceedings of 2nd Workshop on General Purpose Processing on Graphics Processing Units_. 71–78.
  * Kingma and Welling (2013) Diederik P Kingma and Max Welling. 2013.  Auto-encoding variational bayes.  _arXiv preprint arXiv:1312.6114_ (2013).
  * Liu et al. (2021) Jinglin Liu, Chengxi Li, Yi Ren, Feiyang Chen, Peng Liu, and Zhou Zhao. 2021\.  Diffsinger: Singing voice synthesis via shallow diffusion mechanism.  _arXiv preprint arXiv:2105.02446_ 2 (2021).
  * Liu et al. (2022) Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao. 2022\.  Pseudo numerical methods for diffusion models on manifolds.  _arXiv preprint arXiv:2202.09778_ (2022).
  * Lu et al. (2022a) Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. 2022a.  Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps.  _arXiv preprint arXiv:2206.00927_ (2022).
  * Lu et al. (2022b) Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. 2022b.  Dpm-solver++: Fast solver for guided sampling of diffusion probabilistic models.  _arXiv preprint arXiv:2211.01095_ (2022).
  * Nichol and Dhariwal (2021) Alexander Quinn Nichol and Prafulla Dhariwal. 2021.  Improved denoising diffusion probabilistic models. In _International Conference on Machine Learning_. PMLR, 8162–8171.
  * Odena et al. (2017) Augustus Odena, Christopher Olah, and Jonathon Shlens. 2017\.  Conditional image synthesis with auxiliary classifier gans. In _International conference on machine learning_. PMLR, 2642–2651.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021\.  Learning transferable visual models from natural language supervision. In _International conference on machine learning_. PMLR, 8748–8763.
  * Ramesh et al. (2022) Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022.  Hierarchical text-conditional image generation with clip latents.  _arXiv preprint arXiv:2204.06125_ (2022).
  * Reed et al. (2016) Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. 2016\.  Generative adversarial text to image synthesis. In _International conference on machine learning_. PMLR, 1060–1069.
  * Regalia (1993) Phillip A Regalia. 1993\.  Numerical stability properties of a QR-based fast least squares algorithm.  _IEEE Transactions on Signal Processing_ 41, 6 (1993), 2096–2109.
  * Rombach et al. (2022) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022\.  High-resolution image synthesis with latent diffusion models. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 10684–10695.
  * Ronneberger et al. (2015) Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015\.  U-net: Convolutional networks for biomedical image segmentation. In _Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18_. Springer, 234–241.
  * Saharia et al. (2022a) Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. 2022a.  Photorealistic text-to-image diffusion models with deep language understanding.  _arXiv preprint arXiv:2205.11487_ (2022).
  * Saharia et al. (2022b) Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. 2022b.  Image super-resolution via iterative refinement.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ (2022).
  * Salimans et al. (2016) Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016\.  Improved techniques for training gans.  _Advances in neural information processing systems_ 29 (2016).
  * Salimans and Ho (2022) Tim Salimans and Jonathan Ho. 2022.  Progressive distillation for fast sampling of diffusion models.  _arXiv preprint arXiv:2202.00512_ (2022).
  * Schuhmann et al. (2022) Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022\.  Laion-5b: An open large-scale dataset for training next generation image-text models.  _arXiv preprint arXiv:2210.08402_ (2022).
  * Sohl-Dickstein et al. (2015) Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015.  Deep unsupervised learning using nonequilibrium thermodynamics. In _International Conference on Machine Learning_. PMLR, 2256–2265.
  * Song et al. (2020) Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020\.  Denoising diffusion implicit models.  _arXiv preprint arXiv:2010.02502_ (2020).
  * Song and Ermon (2019) Yang Song and Stefano Ermon. 2019.  Generative modeling by estimating gradients of the data distribution.  _Advances in neural information processing systems_ 32 (2019).
  * Szegedy et al. (2016) Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016.  Rethinking the inception architecture for computer vision. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2818–2826.
  * von Platen et al. (2022) Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, and Thomas Wolf. 2022\.  Diffusers: State-of-the-art diffusion models.  https://github.com/huggingface/diffusers.
  * Watson et al. (2022) Daniel Watson, William Chan, Jonathan Ho, and Mohammad Norouzi. 2022.  Learning fast samplers for diffusion models by differentiating through sample quality. In _International Conference on Learning Representations_.
  * Yang et al. (2022) Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. 2022\.  Diffusion models: A comprehensive survey of methods and applications.  _arXiv preprint arXiv:2209.00796_ (2022).
  * Yu et al. (2015) Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. 2015\.  Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.  _arXiv preprint arXiv:1506.03365_ (2015).
  * Zhang and Chen (2022) Qinsheng Zhang and Yongxin Chen. 2022.  Fast sampling of diffusion models with exponential integrator.  _arXiv preprint arXiv:2204.13902_ (2022).