# Semi-classical understanding of flux quantization in superconductors

Kolahal Bhattacharya  St. Xavier’s College (Autonomous), Kolkata-700016
<EMAIL_ADDRESS>
###### Abstract

Like electric charge, magnetic flux is also quantised. Theoretically, one can
show that the flux quantum must be $h/e$, as observed in the quantum Hall
effect. However, in the superconducting systems, the flux quantum is
experimentally observed as $h/2e$. There is no fundamental explanation for the
empirical result. In this article, we argue that this phenomenon is
fundamentally linked to the nonlocality problem of the Aharonov-Bohm effect
and present a new semi-classical explanation for the magnetic flux quantum in
superconductivity. This work will also show why the flux quantum should be
$h/e$ in the case of the quantum Hall effect.

  *

##  1 Introduction

In the first half of the last century, it was experimentally discovered that
the magnetic field is expelled from a superconductor during its transition to
the superconducting state when it is cooled below the critical temperature.
This expulsion pushes off a nearby magnet. This phenomenon is known as the
Meissner effect [1]. Later, it was also discovered that the magnetic flux, is
quantized within the superconductors in the units of $\frac{h}{2e}$ [2] where
$h$ is Planck’s constant and $e$ is electronic charge. This was shocking since
in general the magnetic flux is quantized in the units of $\frac{h}{e}$ (see:
Chapter 21 of [3]). The extra factor of $\frac{1}{2}$ was a hint that somehow
electrons form a pair in superconductors. This was quite surprising, due to
the Coulomb repulsion between the electrons in the pair. At this point, the
BCS theory provided a microscopic model of the pairing of electrons, taking
into account their phonon-mediated interaction with the superconducting
lattice. Several other unconventional superconductors, those for which BCS
theory is not applicable, were invented by the end of the last century.
Irrespective of whether or not a superconductor is conventional, it exhibits
the Meissner effect [4]. For unconventional superconductors which are strongly
correlated electron systems, presently there is no universally accepted
mechanism of electron pairing.

Apart from the introductory chapters in classic popular texts [3], formal
treatment of the topic can be found in the standard books [5, 6, 7]. These
texts, as well as review articles [8, 9] introduce the concept of Cooper pairs
of electrons in superconductors. Such pairing can be achieved through the
phenomenological Landau-Ginzburg model [10], or the microscopic BCS theory
[11, 12], and can lead to the flux quantum of $\frac{h}{2e}$. However, one
must acknowledge that these models accept the pairing up of electrons in
superconductors as an experimental fact. They do not explain $why$ the flux
would necessarily be quantized in the units of $\frac{h}{2e}$ (and not in the
units of $\frac{h}{e}$, say). Not only that, to the best of the knowledge of
the author, there is no clear answer to the following question: what makes the
magnetic flux equal to $\frac{h}{e}$ in some cases (e.g. quantum Hall effect),
and $\frac{h}{2e}$ in the superconductors.

Four decades ago, Crawford [13] gave an “elementary” derivation of the
magnetic flux quantum. But close inspection of his work shows that it is a
mere justification of the quantization of magnetic flux, in the units of
$\frac{h}{q}$ using the Meissner effect (where $q$ is the electric charge).
Unless one already knows about the electron pairs, his reasoning does not
explain why the magnetic flux must be quantized in the units of $\frac{h}{2e}$
in superconductors. Thus, on the fundamental ground, the empirical fact of
electron pairing in superconductors remains an enigma even today. In this
paper, we address this question and argue that the Meissner effect is
intimately connected to the flux quantization in superconductors.

The phenomenon of flux quantization is related to the magnetic Aharonov-Bohm
effect, which is an example of the nonlocal phenomena in quantum physics. A
recent development [14] has presented a local explanation of the phenomenon in
terms of a semi-classical theory of static curl-free fields, somewhat
different from the standard QED. In this theory, one talks about the
wavefunctions of the fields, different from the quantum states of the
electromagnetic four potential $A^{\mu}$. Apart from resolving the nonlocality
problem of the Aharonov-Bohm effect, this paper also sheds light on
quantization of magnetic flux. An interesting prediction of this theory is
that wavefunctions of a conservative static field may exist in a region of
space, even if the corresponding classical field vanishes. It is similar to
vacuum fluctuations in QED, but perhaps they are not identical. In the regions
devoid of classical fields in the Aharonov-Bohm effect, the wavefunctions of
the field operate on the electrons as unitary phases.

In this paper, we show that the semi-classical theories of both the
magnetostatic field $\vec{\mathcal{H}}$ and the vector potential
$\vec{\mathcal{A}}$ are relevant in superconductors. It is possible to express
wavefunctions of these fields which operate on the electrons. From this
standpoint, we will try to throw light on flux quantization.

We shall start our discussion by summarising the basic principles of the semi-
classical model of the curl-free vector fields in section 2. Relevant examples
are vector potential $\vec{\mathcal{A}}$ in a region devoid of the magnetic
field, or magnetic field $\vec{\mathcal{H}}$ in a region devoid of free source
current density $\vec{\mathcal{J}}_{free}$, for which
$\nabla\times\vec{\mathcal{H}}={\bf 0}$). When an electron, represented by
$\psi_{e}$, is subjected to vector potential $\vec{\mathcal{A}}$ in the
absence of a magnetic field, the wavefunction $\psi_{A}$ of
$\vec{\mathcal{A}}$ field operates on $\psi_{e}$. We shall show in section 3
that this electron satisfies the standard Schr$\ddot{\rm o}$dinger equation of
a charge under the influence of a vector potential. In the following section
4, we will obtain the basis wavefunctions of the magnetic field
$\vec{\mathcal{H}}$ when $\mathcal{H}\neq 0$. Then, in section 5, we shall
first express the basis wavefunction in terms of the vector potential, and
then argue why the electrons must form a pair inside superconductors leading
to a flux quantum of $\frac{h}{2e}$.

##  2 Summary of semi-classical theory of magnetic fields

Under specific conditions, the vector potential $\vec{\mathcal{A}}$, or the
magnetic field vector $\vec{\mathcal{H}}$ become curl-free. One can express
such a curl-free vector field $\vec{\mathcal{F}}$ as a gradient of a scalar
potential $\vec{\mathcal{F}}=-\nabla{U}$. Such a vector field satisfies a
variational principle: $\delta\int\mathcal{F}ds=0$, similar to the wave vector
$\vec{k}$ in optics (see section 53 of [15]). It is understood that the
integral is evaluated along a curve that is always superimposed with the
direction of $\vec{\mathcal{F}}$. The similarity of the principle with
Fermat’s principle of optics suggests the possibility of Hamiltonian
formulation of the curl-free fields (similar to the Hamiltonian formulation of
geometrical optics [16]) and subsequent quantization. This treatment shows
that the momenta conjugate to the coordinates $x,y$ and $z$ are
$\mathcal{F}_{x}$, $\mathcal{F}_{y}$ and $\mathcal{F}_{z}$ respectively. Taken
together, $(x,y,z,\mathcal{F}_{x},\mathcal{F}_{y},\mathcal{F}_{z})$ constitute
a phase space, every point of which represents a field point characterised by
Hamiltonian $H=0$.

Taking this further ahead, one can make a transition to the quantum domain, by
requiring that the components of $\vec{\mathcal{F}}$ are operators. These
operate on the wave functions $\psi_{F}$ of the field through the eigenvalue
equation:

$\hat{\mathcal{F}}\psi_{F}=-i\bar{\zeta}\nabla\psi_{F}=\vec{\mathcal{F}}\psi_{F}$
(1)

-where $\bar{\zeta}=\frac{\zeta}{2\pi}$ is a scale factor which plays the role similar to that of the Planck’s constant. This factor has the dimension of $U=-\int\vec{\mathcal{F}}\cdot d\vec{s}$. The semi-classical aspects of $\vec{\mathcal{F}}$ is manifested, if $\zeta\centernot\rightarrow{0}$, with respect to $U$ (scalar potential), of the problem. In the source-free region, $\psi_{F}$ satisfies:

$\bar{\zeta}^{2}\nabla^{2}\psi_{F}+\mathcal{F}^{2}\psi_{F}=0$ (2)

The scalar wave equation (Eq.(2)) is the differential equation of the wave
function of the field $\vec{\mathcal{F}}$. The continuum structure of points
in the classical phase space does not hold any longer, since
$\bar{\zeta}\centernot\rightarrow 0$ in the quantum domain. Thus, the
configuration of a system can be best determined up to the minimum unit
$\bar{\zeta}$ that denotes the area of an elementary cell in the phase space.

A non-normalisable basis solution of Eq.(2) is given by $\psi_{F}={\bf
e}^{i\frac{U}{\bar{\zeta}}}$. This plane wave form of solution is valid even
if $\mathcal{F}=0$ and is useful to explain the nonlocality problem in the
Aharonov-Bohm effect as noted in [14]. However, in a region where
$\mathcal{F}\neq 0$, a normalisable solution $\Psi_{F}$ can be constructed by
superposing several such plane waves with appropriate coefficients
$u(\mathcal{F})$: $\Psi_{F}=\int u(\vec{\mathcal{F}}){\bf
e}^{i\frac{U}{\bar{\zeta}}}d\vec{\mathcal{F}}$.

If in a region the magnetic field is zero, then
$\nabla\times\vec{\mathcal{A}}={\bf 0}$. So, a similar treatment would result
in the wavefunction $\psi_{A}$ of the vector potential which satisfies:

$\bar{\eta}^{2}\nabla^{2}\psi_{A}+\vec{\mathcal{A}}^{2}\psi_{A}=0$ (3)

under the gauge condition $\nabla\cdot\vec{\mathcal{A}}=0$. It has been shown
[14] that the wavefunction $\psi_{A}$ has the form $\psi_{A}={\bf
e}^{-ie\frac{\int{\mathcal{A}}d{s}}{\hbar}}$. A general normalisable solution
of Eq.(3) is constructed as: $\Psi_{A}=\int a(\vec{\mathcal{A}}){\bf
e}^{-i\frac{e\int\vec{\mathcal{A}}\cdot
d\vec{s}}{\hbar}}d{\vec{\mathcal{A}}}$.

##  3 Transformation of electron state in zero magnetic field

An electron wave packet $\psi_{e}$, under $\vec{\mathcal{A}}$ is transformed
as:

$\psi_{e}\rightarrow\psi^{\prime}_{e}=\Psi_{A}\psi_{e}=\left[\int
a(\vec{\mathcal{A}}){\bf e}^{-i\frac{e\int\vec{\mathcal{A}}\cdot
d{\vec{s}}}{\hbar}}d{\vec{\mathcal{A}}}\right]\psi_{e}$ (4)

Time variation of the transformed state is given by:

$\displaystyle i\hbar\frac{\partial\psi_{e}^{\prime}}{\partial
t}=i\hbar\left[\int a(\vec{\mathcal{A}}){\bf
e}^{-i\frac{e\int\vec{\mathcal{A}}\cdot
d{\vec{s}}}{\hbar}}\frac{\partial}{\partial
t}\left(-i\frac{e\int{\vec{\mathcal{A}}}\cdot
d{\vec{s}}}{\hbar}\right)d{\vec{\mathcal{A}}}\right]\psi_{e}+i\hbar\Psi_{A}\frac{\partial\psi_{e}}{\partial
t}=\Psi_{A}\left(i\hbar\frac{\partial\psi_{e}}{\partial t}\right)$ (5)

Here we can drop the first term, taking into account the fact that the partial
temporal derivative of $\int\vec{\mathcal{A}}\cdot d\vec{s}$ is zero, as far
as magnetostatics is concerned. On the other hand, by expressing the
normalized electron wavefunction as $\psi_{e}=\int c(\vec{k}){\bf
e}^{i({\vec{k}}\cdot{\vec{r}}-\omega t)}d{\vec{k}}$, the kinetic term of the
transformed state is given by:

$\displaystyle-\frac{\hbar^{2}}{2m}\nabla^{2}\psi_{e}^{\prime}$
$\displaystyle=-\frac{\hbar^{2}}{2m}\left(\nabla^{2}\Psi_{A}\psi_{e}+2\nabla\psi_{A}\cdot\nabla\psi_{e}\right)+\Psi_{A}\left(-\frac{\hbar^{2}}{2m}\nabla^{2}\psi_{e}\right)$
$\displaystyle\implies\Psi_{A}\left(-\frac{\hbar^{2}}{2m}\nabla^{2}\psi_{e}\right)$
$\displaystyle=-\frac{\hbar^{2}}{2m}\nabla^{2}\psi_{e}^{\prime}+\frac{\hbar^{2}}{2m}\left(-\frac{e^{2}\vec{\mathcal{A}}^{2}}{\hbar^{2}}\Psi_{A}\right)\psi_{e}+\frac{\hbar^{2}}{2m}2(-ie)\frac{\vec{\mathcal{A}}\Psi_{A}}{\hbar}\cdot(i\vec{k})\psi_{e}$
$\displaystyle=\frac{1}{2m}\left[-\hbar^{2}\nabla^{2}\psi_{e}^{\prime}-e^{2}\vec{\mathcal{A}}^{2}\psi_{e}^{\prime}+2e\hbar(\vec{\mathcal{A}}\cdot\vec{k})\psi_{e}^{\prime}\right]$
(6)

Let us note that:

$\displaystyle-i\hbar\nabla(\Psi_{A}\psi_{e})$
$\displaystyle=-i\hbar\nabla\Psi_{A}\psi_{e}-i\hbar\Psi_{A}\nabla\psi_{e}$
$\displaystyle=-i\hbar\frac{-ie\vec{\mathcal{A}}}{\hbar}(\Psi_{A}\psi_{e})-i\hbar(i\vec{k})(\Psi_{A}\psi_{e})$
$\displaystyle\implies 2e\vec{\mathcal{A}}\cdot\hat{\vec{p}}\psi_{e}^{\prime}$
$\displaystyle=-2e^{2}\vec{\mathcal{A}}^{2}\psi_{e}^{\prime}+2e\hbar\vec{\mathcal{A}}\cdot\vec{k}\psi_{e}^{\prime}$
(7)

From Eq.(3) and Eq.(3), we find that:

$\displaystyle\Psi_{A}\left(-\frac{\hbar^{2}}{2m}\nabla^{2}\psi_{e}\right)$
$\displaystyle=\frac{1}{2m}\left[-\hbar^{2}\nabla^{2}\psi_{e}^{\prime}+e^{2}\vec{\mathcal{A}}^{2}\psi_{e}^{\prime}+2e\vec{\mathcal{A}}\cdot\hat{\vec{p}}\psi_{e}^{\prime}\right]$
$\displaystyle=\frac{1}{2m}\left[-\hbar^{2}\nabla^{2}\psi_{e}^{\prime}-ie\hbar\cancelto{0}{(\nabla\cdot\vec{\mathcal{A}})}\psi_{e}^{\prime}-2i\hbar
e\vec{\mathcal{A}}\nabla\psi_{e}^{\prime}+e^{2}\vec{\mathcal{A}}^{2}\psi_{e}^{\prime}\right]$
$\displaystyle=\frac{1}{2m}\left[(\hat{\vec{p}}+e\vec{\mathcal{A}})^{2}\right]\psi_{e}^{\prime}$
(8)

Hence, from Eq.(5) and Eq.(3), we find that the transformed state follows the
following equation:

$i\hbar\frac{\partial\psi_{e}^{\prime}}{\partial
t}=\frac{1}{2m}(\hat{\vec{p}}+e\vec{\mathcal{A}})^{2}\psi_{e}^{\prime}$ (9)

This exercise shows that the semi-classical theory of magnetostatic fields is
consistent with standard formalism.

##  4 wavefunctions of non-zero magnetic field

If in a region, the magnetic field is non-zero, but the source current
$\vec{\mathcal{J}}_{free}$ is zero, we have
$\nabla\times\vec{\mathcal{H}}={\bf 0}$. So, one has $\delta\int\mathcal{H}\
ds=0$ when the integral is evaluated along a curve superimposed with the local
direction of the magnetic field. Hence, one can develop the semi-classical
description of the magnetic field. The wavefunctions $\psi_{M}$ of the
magnetic field is seen to satisfy an equation similar to Eq.(1):

$\bar{\kappa}^{2}\nabla^{2}\psi_{M}+\vec{\mathcal{H}}^{2}\psi_{M}=0$ (10)

Clearly, the wavefunction $\psi_{M}$ has the form $\psi_{M}={\bf
e}^{i\frac{\int\vec{\mathcal{H}}\cdot d\vec{s}}{\bar{\kappa}}}$. This can be
expressed in an alternative form which is more useful for our future
discussion.

The linear momentum of a particle of mass $m$ and charge $q$ moving at a
constant speed $v$ in a region of non-zero magnetic field is:
$\vec{p}=m\vec{v}+q\vec{\mathcal{A}}$. This implies:

$\displaystyle\nabla\times\vec{p}$
$\displaystyle=m\nabla\times\vec{v}+q\nabla\times\vec{\mathcal{A}}$
$\displaystyle{\bf 0}$ $\displaystyle=2m\vec{\omega}+q\vec{\mathcal{B}}$
$\displaystyle\implies\vec{\mathcal{H}}$
$\displaystyle=-\frac{2m}{\mu_{0}q}\vec{\omega}$ (11)

where $\mu_{0}$ denotes the permeability of the free space and $\vec{\omega}$
is the angular frequency. Using this expression, we shall evaluate the
integral $\int\mathcal{H}ds$ along a curve $\mathcal{C}$ aligned with the
local direction of $\vec{\mathcal{H}}$, say the $z$ direction. We assume that
the motion of the charge produces a constant magnetic dipole moment:
$\vec{\mu}$. If the velocity $\vec{v}$ of the particle locally makes an angle
$\theta$ with the $z$ direction, then $ds=v\cos\theta\ dt$. So, the integral
can be expressed as:

$\displaystyle\int\mathcal{H}ds$
$\displaystyle=-\int\frac{2m}{\mu_{0}q}\omega\ ds$
$\displaystyle=-\int\frac{||\vec{\mathcal{L}}||}{\mu_{0}||\vec{\mu}||}\frac{d\phi}{dt}v\cos\theta\
dt$ (12)
$\displaystyle=-\frac{v\cos\theta}{\mu_{0}||\vec{\mu}||}\int{||\vec{\mathcal{L}}||\
d\phi}$ (13)

-where in Eq.(12), we have used the well-known relation between angular momentum vector $\vec{\mathcal{L}}$ and the magnetic dipole moment of the test current distribution $\vec{\mu}$, i.e. $\vec{\mathcal{L}}=\frac{2m}{q}\vec{\mu}$. Now, the minimum value of $\bar{\kappa}\equiv\int\mathcal{H}ds$ occurs, if we set the minimum allowed value of the action $\int||\vec{\mathcal{L}}||\ d\phi$ to be $\hbar$ and $\cos\theta=1$. Then, from Eq.(13), we see that $\bar{\kappa}=-\frac{v\hbar}{\mu_{0}||\vec{\mu}||}$. Therefore, the wavefunction $\psi_{M}$ can be expressed as:

$\displaystyle\psi_{M}=Exp\left[-i\frac{\mu_{0}||\vec{\mu}||\int\mathcal{H}\
v\cos\theta\
dt}{v\hbar}\right]=Exp\left[-i\frac{\int\vec{\mu}\cdot\vec{\mathcal{B}}dt}{\hbar}\right]$
(14)

One can easily identify this as the unitary time evolution operator of a
magnetic dipole moment inside a magnetic field.

##  5 Flux quantization in superconductors

Now, let us note what happens in case of the electron pairs in the
superconductors. To understand this aspect, we consider a chunk of
superconducting material to which an external uniform magnetic field is
applied, say in the $z$ direction. Now, the temperature is gradually reduced
below the critical temperature such that the magnetic field is expelled from
the chunk and we have a superconductor. The electrons in this chunk are in the
field of $\vec{\mathcal{A}}$. Since $\nabla\times\vec{\mathcal{A}}=0$, the
electrons are subjected to $\psi_{A}=\int a(\mathcal{A}){\bf
e}^{-i\frac{e\int{\mathcal{A}}d{s}}{\hbar}}d{\mathcal{A}}$. On the other hand,
$\vec{\mathcal{H}}={\bf 0}$ in this chunk due to the Meissner effect. Though
the electrons in this bulk superconducting region are not subjected to
macroscopic magnetic field, these could experience the semi-classical
wavefunction $\psi_{M}$ of the same [14]. The expression of the basis wave
function of $\psi_{M}$ was obtained in the previous section.

If we assume that the dipoles, created due to the motion of the electrons, are
aligned with the direction of the magnetic field ($z$ direction), then
$\vec{\mu}\cdot\vec{\mathcal{B}}=\mu\mathcal{B}$. Noting that
$\vec{\mu}=\frac{1}{2}\int_{\tau}{\bf r}\times\vec{\mathcal{J}}d\tau$,
$\vec{\mathcal{J}}=\rho\vec{v}$ and the charge by $e=\rho d\tau$, we see that
$\vec{\mu}=\mu\hat{z}=\frac{e}{2}{\bf
r}\times\vec{v}=\frac{1}{2}e(xv_{y}-yv_{x})\hat{z}$. Hence,
$\vec{\mu}\cdot\vec{\mathcal{B}}=\mu\mathcal{B}=\frac{1}{2}e(xv_{y}-yv_{x})B$.
Now, for uniform magnetic field
$\vec{\mathcal{A}}=\frac{1}{2}\vec{\mathcal{B}}\times{\bf r}$. If we remove
the vector notation, $\mathcal{A}=\frac{1}{2}\mathcal{B}r$, then $\psi_{M}$
can be written as:

$\displaystyle\psi_{M}$
$\displaystyle=Exp\left[-i\frac{\int\frac{e}{2}(xv_{y}-yv_{x})\frac{2\mathcal{A}}{r}dt}{\hbar}\right]$
$\displaystyle=Exp\left[-i\frac{\int e\mathcal{A}\left(\frac{x\cdot dy-y\cdot
dx}{\sqrt{x^{2}+y^{2}}}\right)}{\hbar}\right]$
$\displaystyle=Exp\left[-i\frac{\int e\mathcal{A}rd\phi}{\hbar}\right]$
$\displaystyle=Exp\left[-i\frac{\int e\mathcal{A}ds}{\hbar}\right]$ (15)

Hence, the effective wavefunction operating on the electrons in
superconductors in the Meissner state is given by
$\Psi_{AM}=\psi_{A}\psi_{M}=\int a(\mathcal{A}){\bf
e}^{-2i\frac{e\int{\mathcal{A}}d{{s}}}{\hbar}}d{\mathcal{A}}$. Now, we impose
the condition that the vector potential remains unchanged when we add a
constant $\vec{\mathcal{A}_{0}}$ to it: i.e.
$\vec{\mathcal{A}}\rightarrow{\vec{\mathcal{A}}+\vec{\mathcal{A}_{0}}}$. Then
we have:

$\displaystyle a(\vec{\mathcal{A}}+\vec{\mathcal{A}_{0}})$
$\displaystyle=a(\vec{\mathcal{A}})$ (16) $\displaystyle{\bf
e}^{-i\frac{2e\int(\vec{\mathcal{A}}+\vec{\mathcal{A}_{0}})\cdot
d{\vec{s}}}{\hbar}}$ $\displaystyle={\bf
e}^{-i\frac{2e\int\vec{\mathcal{A}}\cdot d{\vec{s}}}{\hbar}}$ (17)

In fact, for the system we considered, the vector potential can be shown to be
$\propto\frac{1}{r}$ in plane polar coordinates. Hence, the transformation by
a constant vector potential $\vec{\mathcal{A}}_{0}$ implies a transformation
within the same circle (of same radius). Thus, Eq.(16) means that for any such
transformation
$\vec{\mathcal{A}}\rightarrow{\vec{\mathcal{A}}+\vec{\mathcal{A}_{0}}}$, the
coefficient $a(\vec{\mathcal{A}})$ remains the same. This reasoning is
applicable also in the case of the magnetic Aharonov-Bohm effect, where two
$\psi_{A}$-s from disjoint paths of equal radius interfere [14].

On the other hand, Eq.(17) leads to a flux quantization condition, as shown
below. Collecting the argument of the cosine part of both sides of the
Eq.(17), we find that:

$\displaystyle\left(\frac{2e\int(\vec{\mathcal{A}}+\vec{\mathcal{A}_{0}})\cdot
d{\vec{s}}}{\hbar}\right)$
$\displaystyle=\left(\frac{2e\int\vec{\mathcal{A}}\cdot
d{\vec{s}}}{\hbar}\right)+2k\pi\ ............\rm{where\ k\in\mathbb{N}}$
$\displaystyle\implies 2e\int\vec{\mathcal{A}_{0}}\cdot d\vec{s}$
$\displaystyle=k\ 2\pi\frac{h}{2\pi}=k\ h$
$\displaystyle\implies\int\vec{\mathcal{A}_{0}}\cdot d\vec{s}$
$\displaystyle=k\frac{h}{2e}$ (18)

The above exercise demonstrates that in superconductors under the Meissner
state, the magnetic flux is quantized in the integral multiples of
$\frac{h}{2e}$. This in turn, implies that charge must be quantized in the
units of $2e$. It can be achieved through the model proposed in the BCS
theory.

##  6 Discussions

The quantized nature of the magnetic flux in the units of $\frac{h}{e}$ has
been pointed out by many authors in past [3, 13]. However, the connection
between the usual flux quantum and the flux quantum in superconductivity
($\frac{h}{2e}$) was not very clear on a fundamental ground. The present
article uses a simple semi-classical approach to bridge this gap. From this
discussion, we find that the result will continue to hold irrespective of
type-I, or type-II superconductors, as long as it is in a Meissner state.
Elsewhere, it has been argued that the BCS theory cannot explain the Meissner
effect [17, 18]. In this context, we comment that if one accepts the flux
quantum and possible mechanisms behind it (e.g. the BCS theory), then the
present work clearly shows why the magnetic field must be zero inside
superconductors.

This work also shows why the observed magnetic flux quantum should be
$\frac{h}{e}$ (and not $\frac{h}{2e}$) in say, the integer quantum Hall
effect. In this case, the magnetic field is not zero. So, one can have a semi-
classical wavefunction $\psi_{M}$ (leading to one of the factors), but not
$\psi_{A}$ due to a non-zero magnetic field.

##  7 References

## References

  * [1] Walther Meissner and Robert Ochsenfeld.  A new effect when the superconductivity occurs.  natural sciences, 21(44):787–788, 1933.
  * [2] Bascom S Deaver Jr and William M Fairbank.  Experimental evidence for quantized flux in superconducting cylinders.  Physical Review Letters, 7(2):43, 1961.
  * [3] Richard P Feynman, Robert B Leighton, and Matthew Sands.  The Feynman lectures on physics: volume III: quantum mechanics.  1965\.
  * [4] JE Hirsch.  The origin of the meissner effect in new and old superconductors.  Physica Scripta, 85(3):035704, 2012.
  * [5] AC Rose Innes and EH Rhoderick.  Introduction to superconductivity.  Pergamon Press, 1980.
  * [6] Vladimir Z Kresin and Stuart A Wolf.  Fundamentals of superconductivity.  Springer Science & Business Media, 2013.
  * [7] Michael Tinkham.  Introduction to superconductivity.  Courier Corporation, 2004.
  * [8] RG Sharma.  A review of theories of superconductivity.  Superconductivity, pages 109–133, 2015.
  * [9] Ibrahim Olanrewaju Alade, Md Safiqur Rahaman, and Talal F Qahtan.  A comprehensive review of superconductivity research productivity.  Journal of Superconductivity and Novel Magnetism, pages 1–17, 2022\.
  * [10] Vitaly L Ginzburg and Lev D Landau.  On the theory of superconductivity.  In On Superconductivity and Superfluidity, pages 113–137. Springer, 2009.
  * [11] Leon N Cooper.  Bound electron pairs in a degenerate fermi gas.  Physical Review, 104(4):1189, 1956.
  * [12] John Bardeen, Leon N Cooper, and John Robert Schrieffer.  Theory of superconductivity.  Physical review, 108(5):1175, 1957.
  * [13] Frank S Crawford.  Elementary derivation of the magnetic flux quantum.  American Journal of Physics, 50(6):514–516, 1982.
  * [14] Kolahal Bhattacharya.  Demystifying the nonlocality problem in aharonov-bohm effect.  Physica Scripta, 2021.
  * [15] Lev Davidovich Landau.  The classical theory of fields, volume 2.  Elsevier, 2013.
  * [16] Vasudevan Lakshminarayanan, Ajoy Ghatak, and K Thyagarajan.  Lagrangian optics.  Springer Science & Business Media, 2002.
  * [17] JE Hirsch.  Bcs theory of superconductivity: it is time to question its validity.  Physica Scripta, 80(3):035702, 2009.
  * [18] JE Hirsch.  Explanation of the meissner effect and prediction of a spin meissner effect in low and high tc superconductors.  Physica C: Superconductivity and its applications, 470:S955–S956, 2010.


Recent advancements in deep generative models present new opportunities for music production but also pose challenges, such as high computational demands and limited audio quality. Moreover, current systems frequently rely solely on text input and typically focus on producing complete musical pieces, which is incompatible with existing workflows in music production. To address these issues, we introduce "Diff-A-Riff," a Latent Diffusion Model designed to generate high-quality instrumental accompaniments adaptable to any musical context. This model offers control through either audio references, text prompts, or both, and produces 48kHz pseudo-stereo audio while significantly reducing inference time and memory usage. We demonstrate the model's capabilities through objective metrics and subjective listening tests, with extensive examples available on the accompanying website.[<sonycslparis.github.io/diffariff-companion/>]

§ INTRODUCTION

Deep generative modeling has recently made significant strides, greatly expanding the toolbox for synthesizing visual and auditory art [1, 2, 3, 4, 5, 6] and signaling a new era of enhanced creative expression.
These technologies promise more intuitive, high-level control over digital creations, yet their deployment in music production comes with inherent challenges.
Generative music systems frequently rely solely on text inputs for control and typically focus on generating complete musical pieces rather than individual sounds or instruments. This approach can limit their integration into existing musical workflows and may compromise the artist's control over the final product.
Furthermore, the computational demands of these advanced models often necessitate access to specialized hardware or online services. Additionally, they often fail to meet professional audio standards, such as true stereo output at 48 kHz.

In this paper, we introduce "Diff-A-Riff," a novel Latent Diffusion Model designed for generating single-instrument accompaniments. A distinct feature of our approach is the ability to condition on musical audio contexts. This specific form of control crucially allows the music to dynamically adapt to the artist's style, enabling a more personalized creation process. Additionally, the model supports conditioning using joint text-and-audio embeddings from CLAP [7], which can be derived from either textual descriptions or audio references, providing versatile input options for directing the music generation.

At the core of "Diff-A-Riff" are two pivotal technological elements. First, the efficiency of a Consistency Autoencoder with a high compression rate enhances the system's performance in terms of inference time and memory usage. Second, the model employs the expressiveness of Elucidated Diffusion Models (EDMs), known for their robust handling of complex data distributions and improved efficiency in model parameterization and inference [8].

We validate "Diff-A-Riff" through comprehensive evaluations, assessing its performance in ablation studies using objective metrics, and we compare it to other models and estimate context adherence using subjective listening tests. The results, detailed in Sections <ref> and <ref>, demonstrate that our model not only achieves state-of-the-art audio quality (statistically not distinguishable from real audio) but also effectively adapts to various conditional settings confirming its potential for practical applications in music production.

The paper is organized as follows: after a review of related work in Section <ref> and background in Section <ref>, we describe our methodology in Section <ref>. We then present our results in Section <ref> and conclude with a discussion and potential future research directions in Section <ref>.

§ RELATED WORK

End-to-end models. The landscape of generative models for music has undergone transformative advancements in recent years. End-to-end Autoregressive Models (AMs) have traditionally been at the forefront of sound fidelity, diversity, and long-term coherence [9, 10]. Nonetheless, their high computational demands render AMs unsuitable for music production settings (i.e., sample rate $\geq$ 44.1 kHz, stereo). In contrast, Generative Adversarial Networks [11]

and Variational Autoencoders [12]

exhibit exceptional generation speed at high sampling rates [13, 14, 15], positioning them as valuable assets for commercial music production technologies [13, 16]. However, these strategies typically require simple datasets with reduced diversity [13, 14], and often restrict generation to fixed lengths [15, 16]. Recently, diffusion models showed a balanced equilibrium between generation quality, diversity, and efficiency [17, 18, 19, 20]. Nevertheless, these rely on an iterative denoising process that, while faster than AMs, still demands long and heavy computations.

Latent models. To address the challenges inherent in end-to-end modeling, generative models have recently pivoted towards operating on compressed representation spaces learned via autoencoders [3, 4, 5, 6, 21]. By doing so, generative systems can allocate representational capacity separately for learning immediate auditory characteristics of sound and longer-term music structure. Additionally, they facilitate the interpretation and integration of multi-modal control data, such as text [3, 5, 6], audio [22, 23], or melody [4]. Within this evolved framework, AMs leverage discrete representation spaces crafted through vector-quantized variational autoencoders

[24, 25], resulting in faster models with better long-term structure [3, 4].

Recent developments have equipped AMs with parallel decoding using masked token modeling techniques [26, 22, 27], enabling sample rates as high as 44.1 kHz with acceptable inference speed.

Latent Diffusion Models (LDMs) also operate on compressed representation spaces, which are typically continuous [5, 6, 28]. This evolution has catalyzed the development of various LDMs capable of generating high-resolution musical audio with long-term structure [6, 28, 29, 5]. Notably, some works can generate audio at sampling rates as high as 48 kHz [6] and stereo [28, 29]. Other works like Stable Audio [5] improve inference efficiency, enabling the generation of 44.1 kHz sampling rate and stereo audio at an unprecedented speed.[95 seconds of audio in 8 seconds on an A100 GPU]

Following this spirit, our system leverages a pre-trained Consistency Autoencoder[paper under review and will be added upon acceptance.] which enables Diff-A-Riff to function within a highly compressed representation space, allowing faster generation than previous systems. Further, our LDM employs the framework of Elucidated Diffusion Models (EDMs) [8, 19], a departure from the Denoising Diffusion Implicit Models (DDIMs) [30] used in previous approaches [6, 5, 28].

Control mechanisms. As evidenced by the state-of-the-art, text prompts currently serve as the most common interface for users to guide audio generative models [4, 3, 5, 6]. To facilitate finer control, Jukedrummer [31] and Music ControlNet [32]
utilize time-varying controls such as rhythmic and dynamic envelopes and melodic lines. Another method for influencing audio output involves conditioning on audio signals, a technique primarily used in style transfer and accompaniment tasks. In style transfer, the objective is to emulate specific aspects of the source audio, e.g., melody [4], timbre [23]. For accompaniment, the focus is on generating musical content that complements or enhances the conditioning audio [33, 23, 34, 22, 35].
Inspired by these control mechanisms, our system introduces conditioning on audio and textual features derived from CLAP [7] alongside audio signals that serve as music context, widening the scope of generative capabilities, e.g., accompaniment generation, text-driven generation, and style transfer.

§ BACKGROUND

In this section, we provide a brief overview of Consistency Models and Denoising Diffusion Models. For an in-depth explanation, we encourage the reader to review the corresponding references.

Consistency Models (CMs) [36, 37] are a novel class of generative models that can produce high-quality samples in a single forward pass without adversarial training. CMs learn a mapping between noisy and clean data samples via a probability flow Ordinary Differential Equation (ODE) [30]. Given a noise level $t$, the consistency function $f$ transforms a noisy sample $x_t \sim p_t(x)$ to a clean sample $x \sim  p_{data}(x)$ by mapping $f(x_t, t) \mapsto x$. This consistency function is approximated by a neural network $f_\theta(x_t, t)$ with parameters $\theta$. It must satisfy the boundary condition $f_\theta(x, t_{\text{min}}) = x$ and is trained by minimizing the discrepancy between its output and a teacher CM at adjacent noise levels $t_i$ and $t_{i+1}$.

Denoising Diffusion Models (DDMs) [38] are generative models originally inspired by the concept of thermodynamic diffusion [39]. DDMs first add noise to data in a forward diffusion process and then use a neural network to reverse this process by removing the noise iteratively. The forward diffusion process is detailed by a Stochastic Differential Equation (SDE), introducing noise to the original data $x_0$ over $T$ steps, resulting in a noisy version $x_T$. This process is defined by $dx_t = f(x_t, t)dt + g(t)dB_t$. Here, $dB_t$ is the increment of a Wiener process (the random noise), $f(x_t, t)$ is the drift term, $g(t)$ is the diffusion term, and $t$ represents the diffusion time step. The reverse process aims to reconstruct the original data from its noisy version by removing the noise. This is achieved by modeling the score of the data distribution, i.e., the gradient of the log probability density function of the noisy data with respect to the data itself, $\nabla_x \log p(x|t)$. The reverse process is described by another SDE, which guides the denoising $dx_t = [f(x_t, t) - g(t)^2 \nabla_x \log p(x_t|t)]dt + g(t)dB_t$, where a neural network $g_{\theta}$, with parameters $\theta$, is trained to estimate this score function, i.e., $g_{\theta}(x_t, t) \approx \nabla_{x} \log p(x_t|t)$. During inference, by performing this process iteratively, we can progressively transform pure noise inputs into data points following the training data distribution.

§ METHODOLOGY

§.§ Dataset

We train our model on a proprietary dataset comprising 12,000 multi-track recordings of diverse music genres (e.g., pop/rock, R&B, rap, country). Each multi-track has various instrument tracks, including bass, guitars, pianos, vocals, and more. We resample each track to 48 kHz, convert it to mono and segment it into overlapping windows of approximately 10 seconds with a 3-second hop size.

In training, we randomly select a target accompaniment track (excluding vocals) and construct the music context by mixing a random subset of the remaining tracks for each segment. We apply this data segmentation and sampling strategy offline to obtain 1M training pairs of audio segments. Following the same methodology, we derive a validation set from 1,200 multi-tracks.

§.§ Diff-A-Riff

§.§.§ Consistency Autoencoder

In this work, we employ a consistency model-based Autoencoder (CAE). We use it pre-trained and freeze its parameters to train a generative model on its latent embeddings (see Fig. <ref>). The  CAE encodes audio samples into a continuous representation space with a $64\times$ compression ratio.

It operates on complex Short-Time Fourier Transform (STFT) spectrograms, with real and imaginary components as separate channels. The architecture uses convolutional residual blocks interleaved with down/up-sampling layers. The CAE Encoder produces 64-dimensional encodings in the range $(-1, 1)$ with a sample rate of 12 Hz for 48 kHz input audio.

The model has $\sim58$ million parameters and is trained following the consistency training framework [37]. For a detailed description of the architecture and training procedure, we refer the reader to the original reference.[paper under review and will be added upon acceptance.]

§.§.§ Latent Diffusion Model

In this work, we train a Latent Diffusion Model (LDM) on the latent space learned by the CAE. The proposed LDM follows the framework of Elucidated Diffusion Models (EDMs) [8], a departure from DDIMs [30] for improved model parametrization and inference.
The architecture follows DDPM++ [40], an upgraded version of the originally proposed Diffusion Probabilistic Model [38]. We only adapt the network's input dimensionality to that of the CAE's latent space (64 channels, see Sec. <ref>). Also, we increase the dimensionality of the conditional embedding input with that of CLAP [7] (i.e., 512 dimensions).

Our UNet is composed of four down/up-sampling blocks with convolutional layers and skip connections, both for the encoder and the decoder (see Fig. <ref> Right). Self-attention is employed in the penultimate resolution layer. We use 512 base channels and double their number at each resolution block. Additionally, the model relies on two dense layers to project the concatenation of CLAP embeddings and the sinusoidal denoising step embeddings into a joint representation. The resulting embedding is used in all down/up-sampling blocks to condition the denoising process as illustrated in Fig <ref>.

§.§.§ Training

Fig. <ref> provides a high-level overview of Diff-A-Riff's setup. Given a pair of input context and target accompaniment audio segments, the model is trained to reconstruct the accompaniment given the context and a CLAP embedding derived from a randomly selected sub-segment of the target itself. This prevents the model from relying on CLAP for temporal alignment. In order to use Classifier-Free Guidance (see Sec. <ref>) and allow the model to optionally operate unconditionally, we drop the audio context and clap embeddings both with a 50% probability. We train Diff-A-Riff over 1M iterations using a batch size of 256 (2 weeks on a single RTX 3090 GPU). We use AdamW [41] as the optimizer and a base learning rate of $10^{-4}$. We use a learning rate schedule with an initial warm-up phase and a reduce-in-plateau process that decreases the learning rate to a minimum value of $10^{-6}$. We keep an Exponential Moving Average (EMA) on the weights with a momentum of $0.9999$. The resulting model has  500M parameters (including the CAE and not CLAP) and occupies 3 GB of memory.

§.§ Evaluation

Objective comparison of Diff-A-Riff with existing state-of-the-art models [5, 6, 4] is challenging as these are generally trained to perform a substantially different task (generation of fully mixed music with no accompaniment conditioning). Even though StemGen [22] and SingSong [34] are trained to generate accompaniments, their implementation and pretrained weights are not publicly available.

Therefore, we focus on subjective listening tests to compare with available music generation models. Additionally, we perform objective evaluations to analyze different inference parameters (e.g., number of diffusion steps, conditioning information; see Section <ref>) to understand which configurations of our proposed model perform the best on our set of metrics. We then apply the gained insights in order to generate the samples that are proposed in the user studies. In the following sections, we describe how we generate the samples used for evaluation, the objective metrics, the listening test methodology, and the baselines we compare against.

§.§.§ Inference Configurations

In this section, we describe the inference configurations that we use for the objective and subjective evaluations.

Conditioning Signals: Different conditioning signals are evaluated. $\textit{CLAP}_A$ refers to the audio-derived CLAP embeddings, which are obtained by using CLAP to encode real audio from a track of the evaluation set.
We can also condition the model on text-derived CLAP embeddings, despite them never being fed during training to the model, since CLAP offers a joint embedding space for both modalities. Because our dataset does not contain audio/text pairs, we create $\textit{CLAP}_T$ embeddings by asking ChatGPT to write text descriptions of single-stem tracks. Finally, Context refers to the conditioning signal obtained by solely encoding the music context into the CAE Encoder.

Classifier-Free Guidance (CFG) [42] allows to improve generation quality by increasing the influence of conditioning signals in the sampling process.

Given the guidance strength CFG, we implement guidance as $x_{t-1} = f_{\theta}(x_t) + \text{CFG}\cdot(f_{\theta}(x_t, c) - f_{\theta}(x_t))$. At inference time, we can use different guidance strengths for Context and CLAP embeddings, denoted as $\text{CFG}_\textit{Context}$ and $\text{CFG}_\textit{CLAP}$, respectively.

Number of Diffusion Steps: At inference time, the number of denoising steps $T$ allows to trade between audio quality and generation speed.

Pseudo-Stereo Generation: We generate pseudo-stereo audio by denoising until a given diffusion time step, and then by independently concluding a stochastic denoising process twice, one for each audio channel. We define the stereo width as the proportion of denoising steps used for stereo generation over the total number of steps. In the user study, we set this parameter to $0.4$.

§.§.§ Objective Metrics

We evaluate Diff-A-Riff through objective metrics to assess various aspects of the generated audio. These include the standard Squared Maximum Mean Discrepancy (MMD2) [43] and Fréchet Audio Distance (FAD) [44] for audio quality as well as Density and Coverage [45] for evaluating fidelity and diversity. To study the system's responsiveness to text prompts, we employ the Clap Score (CS) [46], which calculates the cosine similarity between text and audio embeddings. In order to evaluate the alignment of the generated accompaniment with the context, we employ the Audio Prompt Adherence (APA),[paper under review and will be added upon acceptance.] a newly proposed metric based on FAD tailored to evaluate accompaniment systems. APA is calculated as

\begin{equation*}
\text{APA} = 0.5 + \frac{\text{FAD}_{YX'} - \text{FAD}_{YX}}{2 \cdot \text{FAD}_{XX'}},
\end{equation*}

where $X$ is the background set containing matching context and accompaniment pairs; $X'$ is a set with misaligned pairs; $Y$ is the candidate pairs. All metrics are calculated by averaging five batches of 500 candidate samples. We use CLAP [7] as the embedding space for metrics that compare distributions (like MMD2 and FAD) using a reference set of 5,000 real audio examples.

§.§.§ Listening Tests

Subjective Audio Quality (SAQ): We perform a Mean Opinion Score (MOS) test to assess audio quality. Participants were presented with 5-second audio segments from real data as well as generations from the baselines and the proposed system. Their task is to rate the audio quality of these segments on a 5-level Likert scale ranging from poor (1) to excellent quality (5). For all items (real data, Diff-A-Riff, and baselines), we compare both complete music pieces as well as solo instruments.

We generate solo instruments with Diff-A-Riff by conditioning the model on text or audio-derived CLAP embeddings ($\textit{CLAP}_{A\text{ or }T}$) and without an input context (for a fair comparison with the baselines, which do not rely on contextual audio inputs).
Despite the model not being trained for this task, we can also generate complete music pieces using CLAP and Context embeddings, following an iterative approach: First, we create sets of $\textit{CLAP}_{A\text{ or }T}$ embeddings as described in <ref>. Then, from an initially empty context, we generate new tracks from those CLAP embeddings, iteratively summing the resulting generation into the input context from which we derive the next Context embeddings.

We compare Diff-A-Riff against three state-of-the-art text-to-music baselines: AudioLDM2 [6], MusicGen [4], and Stable Audio [5].[We use the open-source AudioLDM2 model ‘AudioLDM2-48kHz’ operating at 48 kHz. For MusicGen, we use the open-source model ‘MusicGen-large’ operating at 32 kHz, and for Stable Audio, we use their public API.] For each baseline, we generate 20 5-second excerpts of complete music and solo instruments using text prompts generated by ChatGPT.

Subjective Audio Prompt Adherence (SAPA): We also conduct a subjective assessment of audio-prompt adherence. Participants are provided with a reference 10-second music segment and are asked to rate the compatibility of five distinct accompaniments on a scale ranging from 0 (indicating no adherence) to 100 (indicating perfect adherence), according to harmonic, rhythmic, and overall music style compatibility.

The reference segments are derived from music pieces within the evaluation set by summing all tracks in a multitrack, excluding one track, which is reserved to serve as the original accompaniment. Each accompaniment is presented mixed with the reference segment, with slight panning applied to the right to aid in distinguishing between them.
The five accompaniments include the original accompaniment, a randomly selected one from the evaluation set, and three generated by our model under different conditional setups: ($\textit{CLAP}_{A}$ + Context), ($\textit{CLAP}_{T}$ + Context) and (Context only).
To remove a potential bias toward better-quality audio, the original and random segments are encoded and decoded through the CAE.

For both studies, all audio segments are normalized to a loudness of -20 dB LUFS and not cherry-picked. Sample questions are available on the accompanying website.

§ RESULTS & DISCUSSION

§.§ Objective Evaluation

Fig. <ref> shows the MMD2 score of our model as a function of the number of denoising steps, with each line corresponding to a different conditional setting (see Sec. <ref>), all without classifier-free guidance (CFG).
Note that MMD2 compares the distributions of embeddings of generated and real audio in the CLAP latent space. This means it indicates not only audio quality but also how well the distributions of generated instrument types and timbres match the test data distribution. For this reason, when using audio CLAP conditioning ($\textit{CLAP}_{A}$), the results are considerably better, as we force the timbre distribution to be equal to the test data distribution (by using embeddings of that distribution as conditioning).
However, the improvement of the results when increasing the number of denoising steps can be considered independent of the timbre distribution. As a result, we find that $T=30$ steps provide good enough quality.

Based on the results described above we perform a grid search over diffusion steps and classifier-free guidance strength resulting in $T=30$ steps and  $\text{CFG}_\textit{Context} = \text{CFG}_\textit{CLAP} = 1.25$, what we denote as $\textit{Diff-A-Riff}_{T=30}^{\text{CFG}=1.25}$ in Tab. <ref>.
In addition, we compare to a specific configuration that achieves real-time performance[95 seconds of audio in 89 seconds on an NVIDIA RTX3090] on a CPU with acceptable quality using $T=10$ steps and $\text{CFG}=1$, denoted as $\textit{Diff-A-Riff}_{T=10}^{\text{CFG}=1}$.

For reference, we also include metrics computed on real data and a lower bound, calculated from white noise in the case of quality metrics or random real pairs for input adherence metrics (APA and CS).
The overall trend suggests that dense conditioning information helps the model in all metrics: audio quality, coverage and density, as well as APA.
Also, the results could suggest a slight dependency of the model on $\textit{CLAP}_{A}$ embeddings, with metrics close to real data, even in the absence of music context. Only density exhibits a minor drop without context conditioning, suggesting that generating from a silent context leads to samples that sometimes fall in low-density regions of the CLAP space.

The impact of Classifier-free Guidance (CFG) on MMD2 can be estimated by comparing Fig. <ref>, that displays results without CFG, with Tab. <ref>, where CFG was used. In Fig. <ref>, the metrics for both ($\textit{CLAP}_{A} + \textit{Context}$) and ($\textit{CLAP}_{A}$ only) converge towards an MMD2 of $0.5$, while the corresponding values in Tab. <ref> show a reduction to about half this figure. For ($\textit{CLAP}_{T} + \textit{Context}$) and ($\textit{CLAP}_{T}$ only), the MMD2 drops from approximately 5 to about 4 with CFG, and for Context only, it decreases from around 6 to approximately 5.

Finally, we calculate the Clap Score (CS) for text-conditioned generation ($\textit{CLAP}_{T}$ + $\textit{Context}$, $\textit{CLAP}_{T}$ only). We compare $\textit{Diff-A-Riff}_{T=30}^{\text{CFG}=1.25}$ against random pairs of text and real audio (CS=-0.07), suggesting that the model is only somewhat responsive to text prompts (CS=0.25).

§.§ Subjective Evaluation

Table <ref> presents the outcomes of the Subjective Audio Quality (SAQ) test based on $74$ users who each rated $32$ audio segments, resulting in $2368$ ratings. In this test, we compare results against leading baselines (see Sec. <ref>) and real audio data. Our analysis includes a comparison between these benchmarks and the audio generated by Diff-A-Riff in both mono (ch=1) and pseudo-stereo (ch=2) formats (see Sec. <ref>). Results show that the pseudo-stereo samples generated by Diff-A-Riff received ratings that are statistically indifferent from real audio ratings (p-value=0.79), indicating that participants found the audio quality of the generations indistinguishable from real data.

This outcome is particularly remarkable given that Diff-A-Riff was not explicitly trained on complete musical pieces nor stereo music generation, but is still competitive to other models. Further, it highlights the influence of stereo imaging on the perceived audio quality.

Tab. <ref> shows the results for the Subjective Audio Prompt Adherence (SAPA) listening test based on 35 users, each rating 25 accompaniments. The results include ratings scored by real accompaniments

, random accompaniments

, and the various conditional settings described in Sec. <ref>.
Following the trend of previous results, the default setting ($\textit{CLAP}_{A} + \textit{Context}$) scores the closest to real accompaniments,

suggesting that the model can effectively adapt to the context under this setting. When conditioned on

(Context only)

, Diff-A-Riff is rated worse but still significantly better than random accompaniments. Further, for ($\textit{CLAP}_{T} + \textit{Context}$), the accompaniments are rated the lowest. A reason could be that the overall quality is worse because CLAP embeddings of text prompts have not been shown during training. Another problem could be that the randomly chosen text prompt is incompatible with the provided music context (e.g., "A drum machine with electronic textures" with an acoustic blues context), which reduces perceived adherence due to conflicting styles.

Overall, SAPA results are interesting given that objective validation for APA (see Tab. <ref>) suggested rather pessimistic results for ($\textit{CLAP}_{T} + \textit{Context}$)  and (Context only). This could potentially be attributed to APA's sensitivity to audio quality and timbre differences between the reference and candidate set.

§.§ Control Mechanisms

In the accompanying website, we show examples of Diff-A-Riff generations for different inference settings (see Sec. <ref>).

We also showcase other controls that naturally emerge from the denoising process, such as in/out-painting or the generation of variations and loops, as well as controls derived from the manipulation of CLAP embeddings, e.g., text-audio Interpolations.

§ CONCLUSION

This work introduced Diff-A-Riff, a Latent Diffusion Model capable of generating instrumental accompaniments adapted to a user-provided musical audio context. It can be controlled based on style audio references, text prompts, or both. We also proposed a simple method for producing pseudo-stereo audio.

By exploiting the efficiency of a Consistency Autoencoder, Diff-A-Riff is capable of generating 48 kHz sample rate pseudo-stereo audio with unprecedented speed and quality. Through extensive objective and subjective evaluation, we showed that our model achieves state-of-the-art audio quality, can meaningfully adapt to various conditional settings, and generates content that adheres to pre-existing musical audio contexts. We believe that this work represents a significant step towards enhancing accessibility and audio quality in AI-assisted music production tools while prioritizing artist-centric interactions, enriching the landscape of human-machine music co-creation.

§ ETHICS STATEMENT

Sony Computer Science Laboratories is committed to exploring the positive applications of AI in music creation. We collaborate with artists to develop innovative technologies that enhance creativity. We uphold strong ethical standards and actively engage with the music community and industry to align our practices with societal values. Our team is mindful of the extensive work that songwriters and recording artists dedicate to their craft. Our technology must respect, protect, and honour this commitment.

Diff-A-Riff supports and enhances human creativity and emphasises the artist's agency by providing various controls for generating and manipulating musical material. By generating a stem at a time, the artist remains responsible for the entire musical arrangement.

Diff-A-Riff has been trained on a dataset that was legally acquired for internal research and development; therefore, neither the data nor the model can be made publicly available. We are doing our best to ensure full legal compliance and address all ethical concerns.

For media, institutional, or industrial inquiries, please contact us via the email address provided.

[1]
U. Singer, A. Polyak, T. Hayes, X. Yin, J. An, S. Zhang, Q. Hu, H. Yang, O. Ashual, O. Gafni, D. Parikh, S. Gupta, and Y. Taigman, “Make-A-Video: Text-to-video generation without text-video data,” in Proc. of the 11th International Conference on Learning Representations, ICLR, 2023.

[2]
J. Betker, G. Goh, L. Jing, TimBrooks, J. Wang, L. Li, LongOuyang, JuntangZhuang, JoyceLee, YufeiGuo, WesamManassra, PrafullaDhariwal, CaseyChu, YunxinJiao, and A. Ramesh, “Improving image generation with better captions,” in CoRR, 2023.

[3]
A. Agostinelli, T. I. Denk, Z. Borsos, J. H. Engel, M. Verzetti, A. Caillon, Q. Huang, A. Jansen, A. Roberts, M. Tagliasacchi, M. Sharifi, N. Zeghidour, and C. H. Frank, “MusicLM: Generating Music From Text,” in CoRR, 2023.

[4]
J. Copet, F. Kreuk, I. Gat, T. Remez, D. Kant, G. Synnaeve, Y. Adi, and A. Défossez, “Simple and controllable music generation,” in Advances in Neural Information Processing Systems 36 NeurIPS, 2023.

[5]
Z. Evans, C. Carr, J. Taylor, S. H. Hawley, and J. Pons, “Fast timing-conditioned latent audio diffusion,” in CoRR, 2024.

[6]
H. Liu, Q. Tian, Y. Yuan, X. Liu, X. Mei, Q. Kong, Y. Wang, W. Wang, Y. Wang, and M. D. Plumbley, “AudioLDM 2: Learning holistic audio generation with self-supervised pretraining,” in arXiv, 2023.

[7]
B. Elizalde, S. Deshmukh, M. A. Ismail, and H. Wang, “CLAP learning audio concepts from natural language supervision,” in Proc. of the IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP, 2023.

[8]
T. Karras, M. Aittala, T. Aila, and S. Laine, “Elucidating the design space of diffusion-based generative models,” in Proc. of the 36th Conference on Neural Information Processing Systems (NeurIPS), 2022.

[9]
A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. W. Senior, and K. Kavukcuoglu, “WaveNet: A generative model for raw audio,” in Proc. of the 9th ISCA Speech Synthesis Workshop, 2016.

[10]
S. Mehri, K. Kumar, I. Gulrajani, R. Kumar, S. Jain, J. Sotelo, A. C. Courville, and Y. Bengio, “SampleRNN: An unconditional end-to-end neural audio generation model,” in Proc. of 5th International Conference on Learning Representations, ICLR, 2017.

[11]
I. J. Goodfellow, J. Pouget-Abadie et al., “Generative adversarial nets,” in Advances in Neural Information Processing Systems 27, Dec. 2014, pp. 2672–2680.

[12]
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in 2nd International Conference on Learning Representations (ICLR), Apr. 2014.

[13]
A. Caillon and P. Esling, “RAVE: A variational autoencoder for fast and high-quality neural audio synthesis,” in CoRR, 2021.

[14]
M. Pasini and J. Schlüter, “Musika! Fast infinite waveform music generation,” in Proc. of the 23rd International Society for Music Information Retrieval Conference, ISMIR, 2022.

[15]
J. Nistal, S. Lattner, and G. Richard, “DrumGAN: Synthesis of drum sounds with timbral feature conditioning,” in Proc. of the 21st International Society for Music Information Retrieval Conference, ISMIR, 2020.

[16]
J. Nistal, C. Aouameur, I. Velarde, and S. Lattner, “DrumGAN VST: A plugin for drum sound analysis/synthesis with autoencoding generative adversarial networks,” in Proc. of International Conference on Machine Learning ICML, Workshop on Machine Learning for Audio Synthesis, MLAS, 2022.

[17]
S. Rouard and G. Hadjeres, “CRASH: raw audio score-based generative modeling for controllable high-resolution drum sound synthesis,” in Proc. of the 22nd International Society for Music Information Retrieval Conference, ISMIR, 2021.

[18]
Q. Huang, D. S. Park, T. Wang, T. I. Denk, A. Ly, N. Chen, Z. Zhang, Z. Zhang, J. Yu, C. H. Frank, J. H. Engel, Q. V. Le, W. Chan, and W. Han, “Noise2Music: Text-conditioned music generation with diffusion models,” in CoRR, 2023.

[19]
G. Zhu, Y. Wen, M. Carbonneau, and Z. Duan, “EDMSound: Spectrogram based diffusion models for efficient and high quality audio synthesis,” in CoRR, 2023.

[20]
F. Schneider, “ArchiSound: Audio generation with diffusion,” in CoRR, 2023.

[21]
P. Dhariwal, H. Jun, C. Payne, J. W. Kim, A. Radford, and I. Sutskever, “Jukebox: A generative model for music,” in CoRR, 2020.

[22]
J. D. Parker, J. Spijkervet, K. Kosta, F. Yesiler, B. Kuznetsov, J. Wang, M. Avent, J. Chen, and D. Le, “StemGen: A music generation model that listens,” in CoRR, 2023.

[23]
M. Pasini, M. Grachten, and S. Lattner, “Bass accompaniment generation via latent diffusion,” in IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2024, Seoul, South Korea, April 14-19, 2024.1em plus 0.5em minus 0.4emIEEE, 2024.

[24]
N. Zeghidour, A. Luebs, A. Omran, J. Skoglund, and M. Tagliasacchi, “SoundStream: An end-to-end neural audio codec,” in IEEE ACM Trans. Audio Speech Lang. Process., 2022.

[25]
A. Défossez, J. Copet, G. Synnaeve, and Y. Adi, “High fidelity neural audio compression,” in CoRR, 2022.

[26]
H. F. García, P. Seetharaman, R. Kumar, and B. Pardo, “VampNet: Music generation via masked acoustic token modeling,” in Proc. of the 24th International Society for Music Information Retrieval Conference, ISMIR, 2023.

[27]
A. Ziv, I. Gat, G. L. Lan, T. Remez, F. Kreuk, A. Défossez, J. Copet, G. Synnaeve, and Y. Adi, “Masked audio generation using a single non-autoregressive transformer,” in CoRR, 2024.

[28]
F. Schneider, O. Kamal, Z. Jin, and B. Schölkopf, “Moûsai: Text-to-music generation with long-context latent diffusion,” in CoRR, 2023.

[29]
P. Li, B. Chen, Y. Yao, Y. Wang, A. Wang, and A. Wang, “JEN-1: text-guided universal music generation with omnidirectional,” in CoRR, 2023.

[30]
J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in Proc. of the 9th International Conference on Learning Representations, ICLR, 2021.

[31]
Y. Wu, C. Chiu, and Y. Yang, “Jukedrummer: Conditional beat-aware audio-domain drum accompaniment generation via transformer VQ-VAE,” in Proc. of the 3rd International Society for Music Information Retrieval Conference, ISMIR, 2022.

[32]
S.-L. Wu, C. Donahue, S. Watanabe, and N. J. Bryan, “Music controlnet: Multiple time-varying controls for music generation,” in CoRR, 2023.

[33]
M. Grachten, S. Lattner, and E. Deruty, “BassNet: A variational gated autoencoder for conditional generation of bass guitar tracks with learned interactive control,” in Applied Sciences, 2020.

[34]
C. Donahue, A. Caillon, A. Roberts, E. Manilow, P. Esling, A. Agostinelli, M. Verzetti, I. Simon, O. Pietquin, N. Zeghidour, and J. H. Engel, “Singsong: Generating musical accompaniments from singing,” in CoRR, 2023.

[35]
S. Lattner and M. Grachten, “High-level control of drum track generation using learned patterns of rhythmic interaction,” in 2019 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, WASPAA 2019, New Paltz, NY, USA, October 20-23, 2019.1em plus 0.5em minus 0.4emIEEE, 2019, pp. 35–39. [Online]. Available: <https://doi.org/10.1109/WASPAA.2019.8937261>

[36]
Y. Song, P. Dhariwal, M. Chen, and I. Sutskever, “Consistency models,” in International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, ser. Proceedings of Machine Learning Research, A. Krause, E. Brunskill, K. Cho, B. Engelhardt, S. Sabato, and J. Scarlett, Eds., vol. 202.1em plus 0.5em minus 0.4emPMLR, 2023, pp. 32 211–32 252. [Online]. Available: <https://proceedings.mlr.press/v202/song23a.html>

[37]
Y. Song and P. Dhariwal, “Improved techniques for training consistency models,” arXiv preprint arXiv:2310.14189, 2023.

[38]
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, NeurIPS, 2020.

[39]
J. Sohl-Dickstein, E. A. Weiss et al., “Deep unsupervised learning using nonequilibrium thermodynamics,” in Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, ser. JMLR Workshop and Conference Proceedings, vol. 37.1em plus 0.5em minus 0.4emJMLR.org, 2015, pp. 2256–2265.

[40]
T. Karras, M. Aittala, T. Aila, and S. Laine, “Score-based generative modeling through stochastic differential equations,” in Proc. of the International Conference on Learning Representations (ICLR), 2021.

[41]
I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in 7th International Conference on Learning Representations, ICLR, 2019.

[42]
J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv preprint arXiv:2207.12598, 2022.

[43]
M. Binkowski, D. J. Sutherland, M. Arbel, and A. Gretton, “Demystifying MMD gans,” in Proc. of the 6th International Conference on Learning Representations, ICLR, 2018.

[44]
K. Kilgour, M. Zuluaga, D. Roblek, and M. Sharifi, “Fréchet audio distance: A reference-free metric for evaluating music enhancement algorithms,” in Proc. of the 20th Conference of the International Speech Communication Association (InterSpeech), 2019.

[45]
M. F. Naeem, S. J. Oh, Y. Uh, Y. Choi, and J. Yoo, “Reliable fidelity and diversity metrics for generative models,” in Proc. of the 37th International Conference on Machine Learning, ICML, 2020.

[46]
R. Huang, J. Huang, D. Yang, Y. Ren, L. Liu, M. Li, Z. Ye, J. Liu, X. Yin, and Z. Zhao, “Make-an-audio: Text-to-audio generation with prompt-enhanced diffusion models,” in Proc. of the International Conference on Machine Learning, ICML, 2023.