Current address:]Argonne National Laboratory, Argonne, Illinois 60439 Current
address:]Thomas Jefferson National Accelerator Facility, Newport News,
Virginia 23606

The CLAS Collaboration

# First measurement of hard exclusive $\pi^{-}\Delta^{++}$ electroproduction
beam-spin asymmetries off the proton

S. Diehl  II Physikalisches Institut der Universitaet Giessen, 35392 Giessen,
Germany  University of Connecticut, Storrs, Connecticut 06269  N. Trotta
University of Connecticut, Storrs, Connecticut 06269  K. Joo  University of
Connecticut, Storrs, Connecticut 06269  P. Achenbach  Thomas Jefferson
National Accelerator Facility, Newport News, Virginia 23606  Z. Akbar
University of Virginia, Charlottesville, Virginia 22901  Florida State
University, Tallahassee, Florida 32306  W.R. Armstrong  Argonne National
Laboratory, Argonne, Illinois 60439  H. Atac  Temple University, Philadelphia,
PA 19122  H. Avakian  Thomas Jefferson National Accelerator Facility, Newport
News, Virginia 23606  L. Baashen  Florida International University, Miami,
Florida 33199  N.A. Baltzell  Thomas Jefferson National Accelerator Facility,
Newport News, Virginia 23606  L. Barion  INFN, Sezione di Ferrara, 44100
Ferrara, Italy  M. Bashkanov  University of York, York YO10 5DD, United
Kingdom  M. Battaglieri  INFN, Sezione di Genova, 16146 Genova, Italy  I.
Bedlinskiy  National Research Centre Kurchatov Institute - ITEP, Moscow,
117259, Russia  F. Benmokhtar  Duquesne University, 600 Forbes Avenue,
Pittsburgh, PA 15282  A. Bianconi  Universit‘a degli Studi di Brescia, 25123
Brescia, Italy  INFN, Sezione di Pavia, 27100 Pavia, Italy  A.S. Biselli
Fairfield University, Fairfield CT 06824  F. Bossù  IRFU, CEA, Université
Paris-Saclay, F-91191 Gif-sur-Yvette, France  K.-T. Brinkmann  II
Physikalisches Institut der Universitaet Giessen, 35392 Giessen, Germany  W.J.
Briscoe  The George Washington University, Washington, DC 20052  D. Bulumulla
Old Dominion University, Norfolk, Virginia 23529  V. Burkert  Thomas Jefferson
National Accelerator Facility, Newport News, Virginia 23606  R. Capobianco
University of Connecticut, Storrs, Connecticut 06269  D.S. Carman  Thomas
Jefferson National Accelerator Facility, Newport News, Virginia 23606  J.C.
Carvajal  Florida International University, Miami, Florida 33199  A. Celentano
INFN, Sezione di Genova, 16146 Genova, Italy  G. Charles  Universit’e Paris-
Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay, France  Old Dominion University,
Norfolk, Virginia 23529  P. Chatagnon  Thomas Jefferson National Accelerator
Facility, Newport News, Virginia 23606  Universit’e Paris-Saclay, CNRS/IN2P3,
IJCLab, 91405 Orsay, France  V. Chesnokov  Skobeltsyn Institute of Nuclear
Physics, Lomonosov Moscow State University, 119234 Moscow, Russia  G. Ciullo
INFN, Sezione di Ferrara, 44100 Ferrara, Italy  Universita’ di Ferrara , 44121
Ferrara, Italy  P.L. Cole  Lamar University, 4400 MLK Blvd, PO Box 10046,
Beaumont, Texas 77710  M. Contalbrigo  INFN, Sezione di Ferrara, 44100
Ferrara, Italy  G. Costantini  Universit‘a degli Studi di Brescia, 25123
Brescia, Italy  INFN, Sezione di Pavia, 27100 Pavia, Italy  V. Crede  Florida
State University, Tallahassee, Florida 32306  A. D’Angelo  INFN, Sezione di
Roma Tor Vergata, 00133 Rome, Italy  Universita’ di Roma Tor Vergata, 00133
Rome Italy  N. Dashyan  Yerevan Physics Institute, 375036 Yerevan, Armenia  R.
De Vita  INFN, Sezione di Genova, 16146 Genova, Italy  A. Deur  Thomas
Jefferson National Accelerator Facility, Newport News, Virginia 23606  C.
Djalali  Ohio University, Athens, Ohio 45701  University of South Carolina,
Columbia, South Carolina 29208  R. Dupre  Universit’e Paris-Saclay,
CNRS/IN2P3, IJCLab, 91405 Orsay, France  M. Ehrhart  [  Universit’e Paris-
Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay, France  A. El Alaoui  Universidad
Técnica Federico Santa María, Casilla 110-V Valparaíso, Chile  L. El Fassi
Mississippi State University, Mississippi State, MS 39762-5167  L. Elouadrhiri
Thomas Jefferson National Accelerator Facility, Newport News, Virginia 23606
S. Fegan  University of York, York YO10 5DD, United Kingdom  A. Filippi  INFN,
Sezione di Torino, 10125 Torino, Italy  G. Gavalian  Thomas Jefferson National
Accelerator Facility, Newport News, Virginia 23606  D.I. Glazier  University
of Glasgow, Glasgow G12 8QQ, United Kingdom  A.A. Golubenko  Skobeltsyn
Institute of Nuclear Physics, Lomonosov Moscow State University, 119234
Moscow, Russia  G. Gosta  Universit‘a degli Studi di Brescia, 25123 Brescia,
Italy  INFN, Sezione di Pavia, 27100 Pavia, Italy  R.W. Gothe  University of
South Carolina, Columbia, South Carolina 29208  Y. Gotra  Thomas Jefferson
National Accelerator Facility, Newport News, Virginia 23606  K. Griffioen
College of William and Mary, Williamsburg, Virginia 23187-8795  K. Hafidi
Argonne National Laboratory, Argonne, Illinois 60439  H. Hakobyan  Universidad
Técnica Federico Santa María, Casilla 110-V Valparaíso, Chile  M. Hattawy  Old
Dominion University, Norfolk, Virginia 23529  Argonne National Laboratory,
Argonne, Illinois 60439  T.B. Hayward  University of Connecticut, Storrs,
Connecticut 06269  D. Heddle  Christopher Newport University, Newport News,
Virginia 23606  Thomas Jefferson National Accelerator Facility, Newport News,
Virginia 23606  A. Hobart  Universit’e Paris-Saclay, CNRS/IN2P3, IJCLab, 91405
Orsay, France  M. Holtrop  University of New Hampshire, Durham, New Hampshire
03824-3568  I. Illari  The George Washington University, Washington, DC 20052
D.G. Ireland  University of Glasgow, Glasgow G12 8QQ, United Kingdom  E.L.
Isupov  Skobeltsyn Institute of Nuclear Physics, Lomonosov Moscow State
University, 119234 Moscow, Russia  H.S. Jo  Kyungpook National University,
Daegu 41566, Republic of Korea  R. Johnston  Massachusetts Institute of
Technology, Cambridge, Massachusetts 02139-4307  D. Keller  University of
Virginia, Charlottesville, Virginia 22901  M. Khachatryan  Old Dominion
University, Norfolk, Virginia 23529  A. Khanal  Florida International
University, Miami, Florida 33199  A. Kim  University of Connecticut, Storrs,
Connecticut 06269  W. Kim  Kyungpook National University, Daegu 41566,
Republic of Korea  V. Klimenko  University of Connecticut, Storrs, Connecticut
06269  A. Kripko  II Physikalisches Institut der Universitaet Giessen, 35392
Giessen, Germany  V. Kubarovsky  Thomas Jefferson National Accelerator
Facility, Newport News, Virginia 23606  S.E. Kuhn  Old Dominion University,
Norfolk, Virginia 23529  V. Lagerquist  Old Dominion University, Norfolk,
Virginia 23529  L. Lanza  INFN, Sezione di Roma Tor Vergata, 00133 Rome, Italy
Universita’ di Roma Tor Vergata, 00133 Rome Italy  M. Leali  Universit‘a degli
Studi di Brescia, 25123 Brescia, Italy  INFN, Sezione di Pavia, 27100 Pavia,
Italy  S. Lee  Argonne National Laboratory, Argonne, Illinois 60439  P. Lenisa
INFN, Sezione di Ferrara, 44100 Ferrara, Italy  Universita’ di Ferrara , 44121
Ferrara, Italy  X. Li  Massachusetts Institute of Technology, Cambridge,
Massachusetts 02139-4307  I .J .D. MacGregor  University of Glasgow, Glasgow
G12 8QQ, United Kingdom  D. Marchand  Universit’e Paris-Saclay, CNRS/IN2P3,
IJCLab, 91405 Orsay, France  V. Mascagna  Universit‘a degli Studi di Brescia,
25123 Brescia, Italy  Università degli Studi dell’Insubria, 22100 Como, Italy
INFN, Sezione di Pavia, 27100 Pavia, Italy  G. Matousek  Duke University,
Durham, North Carolina 27708-0305  B. McKinnon  University of Glasgow, Glasgow
G12 8QQ, United Kingdom  C. McLauchlin  University of South Carolina,
Columbia, South Carolina 29208  Z.E. Meziani  Argonne National Laboratory,
Argonne, Illinois 60439  Temple University, Philadelphia, PA 19122  S.
Migliorati  Universit‘a degli Studi di Brescia, 25123 Brescia, Italy  INFN,
Sezione di Pavia, 27100 Pavia, Italy  R.G. Milner  Massachusetts Institute of
Technology, Cambridge, Massachusetts 02139-4307  T. Mineeva  Universidad
Técnica Federico Santa María, Casilla 110-V Valparaíso, Chile  M. Mirazita
INFN, Laboratori Nazionali di Frascati, 00044 Frascati, Italy  V. Mokeev
Thomas Jefferson National Accelerator Facility, Newport News, Virginia 23606
P. Moran  Massachusetts Institute of Technology, Cambridge, Massachusetts
02139-4307  C. Munoz Camacho  Universit’e Paris-Saclay, CNRS/IN2P3, IJCLab,
91405 Orsay, France  P. Naidoo  University of Glasgow, Glasgow G12 8QQ, United
Kingdom  K. Neupane  University of South Carolina, Columbia, South Carolina
29208  S. Niccolai  Universit’e Paris-Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay,
France  G. Niculescu  James Madison University, Harrisonburg, Virginia 22807
M. Osipenko  INFN, Sezione di Genova, 16146 Genova, Italy  P. Pandey  Old
Dominion University, Norfolk, Virginia 23529  M. Paolone  New Mexico State
University, PO Box 30001, Las Cruces, NM 88003, USA  Temple University,
Philadelphia, PA 19122  L.L. Pappalardo  INFN, Sezione di Ferrara, 44100
Ferrara, Italy  Universita’ di Ferrara , 44121 Ferrara, Italy  R. Paremuzyan
Thomas Jefferson National Accelerator Facility, Newport News, Virginia 23606
University of New Hampshire, Durham, New Hampshire 03824-3568  S.J. Paul
University of California Riverside, 900 University Avenue, Riverside, CA
92521, USA  W. Phelps  Christopher Newport University, Newport News, Virginia
23606  The George Washington University, Washington, DC 20052  N. Pilleux
Universit’e Paris-Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay, France  M. Pokhrel
Old Dominion University, Norfolk, Virginia 23529  J. Poudel  [  Old Dominion
University, Norfolk, Virginia 23529  J.W. Price  California State University,
Dominguez Hills, Carson, CA 90747  Y. Prok  Old Dominion University, Norfolk,
Virginia 23529  A. Radic  Universidad Técnica Federico Santa María, Casilla
110-V Valparaíso, Chile  B.A. Raue  Florida International University, Miami,
Florida 33199  T. Reed  Florida International University, Miami, Florida 33199
J. Richards  University of Connecticut, Storrs, Connecticut 06269  M. Ripani
INFN, Sezione di Genova, 16146 Genova, Italy  J. Ritman  GSI Helmholtzzentrum
fur Schwerionenforschung GmbH, D-64291 Darmstadt, Germany  Institute fur
Kernphysik (Juelich), Juelich, Germany  P. Rossi  Thomas Jefferson National
Accelerator Facility, Newport News, Virginia 23606  INFN, Laboratori Nazionali
di Frascati, 00044 Frascati, Italy  F. Sabatié  IRFU, CEA, Université Paris-
Saclay, F-91191 Gif-sur-Yvette, France  C. Salgado  Norfolk State University,
Norfolk, Virginia 23504  S. Schadmand  GSI Helmholtzzentrum fur
Schwerionenforschung GmbH, D-64291 Darmstadt, Germany  A. Schmidt  The George
Washington University, Washington, DC 20052  Massachusetts Institute of
Technology, Cambridge, Massachusetts 02139-4307  Y.G. Sharabian  Thomas
Jefferson National Accelerator Facility, Newport News, Virginia 23606  U.
Shrestha  University of Connecticut, Storrs, Connecticut 06269  Ohio
University, Athens, Ohio 45701  D. Sokhan  IRFU, CEA, Université Paris-Saclay,
F-91191 Gif-sur-Yvette, France  University of Glasgow, Glasgow G12 8QQ, United
Kingdom  N. Sparveris  Temple University, Philadelphia, PA 19122  M. Spreafico
INFN, Sezione di Genova, 16146 Genova, Italy  S. Stepanyan  Thomas Jefferson
National Accelerator Facility, Newport News, Virginia 23606  I. Strakovsky
The George Washington University, Washington, DC 20052  S. Strauch  University
of South Carolina, Columbia, South Carolina 29208  M. Turisini  INFN,
Laboratori Nazionali di Frascati, 00044 Frascati, Italy  R. Tyson  University
of Glasgow, Glasgow G12 8QQ, United Kingdom  M. Ungaro  Thomas Jefferson
National Accelerator Facility, Newport News, Virginia 23606  S. Vallarino
INFN, Sezione di Ferrara, 44100 Ferrara, Italy  L. Venturelli  Universit‘a
degli Studi di Brescia, 25123 Brescia, Italy  INFN, Sezione di Pavia, 27100
Pavia, Italy  H. Voskanyan  Yerevan Physics Institute, 375036 Yerevan, Armenia
E. Voutier  Universit’e Paris-Saclay, CNRS/IN2P3, IJCLab, 91405 Orsay, France
D.P. Watts  University of York, York YO10 5DD, United Kingdom  X. Wei  Thomas
Jefferson National Accelerator Facility, Newport News, Virginia 23606  R.
Williams  University of York, York YO10 5DD, United Kingdom  R. Wishart
University of Glasgow, Glasgow G12 8QQ, United Kingdom  M.H. Wood  Canisius
College, Buffalo, NY  M. Yurov  Mississippi State University, Mississippi
State, MS 39762-5167  N. Zachariou  University of York, York YO10 5DD, United
Kingdom  Z.W. Zhao  Duke University, Durham, North Carolina 27708-0305  Old
Dominion University, Norfolk, Virginia 23529  M. Zurek  Argonne National
Laboratory, Argonne, Illinois 60439

###### Abstract

The polarized cross section ratio $\sigma_{LT^{\prime}}/\sigma_{0}$ from hard
exclusive $\pi^{-}\Delta^{++}$ electroproduction off an unpolarized hydrogen
target has been extracted based on beam-spin asymmetry measurements using a
10.2 GeV / 10.6 GeV incident electron beam and the CLAS12 spectrometer at
Jefferson Lab. The study, which provides the first observation of this channel
in the deep-inelastic regime, focuses on very forward-pion kinematics in the
valence regime, and photon virtualities ranging from 1.5 GeV2 up to 7 GeV2.
The reaction provides a novel access to the $d$-quark content of the nucleon
and to $p\rightarrow\Delta^{++}$ transition generalized parton distributions.
A comparison to existing results for hard exclusive $\pi^{+}n$ and $\pi^{0}p$
electroproduction is provided, which shows a clear impact of the excitation
mechanism, encoded in transition generalized parton distributions, on the
asymmetry.

###### pacs:

13.60.Le, 14.20.Dh, 14.40.Be, 24.85.+p

Hard exclusive meson electroproduction provides a powerful tool to study the
structure of the nucleon and the underlying reaction dynamics as the process
amplitude depends on Generalized Parton Distributions (GPDs) [1, 2, 3]. GPDs
enable us to access the three-dimensional (3D) structure of the nucleon by
correlating the transverse position and the longitudinal momentum of the
quarks and gluons inside the nucleon. For longitudinally polarized virtual
photons, the factorization of the process amplitude into a perturbatively
calculable hard-scattering part and two soft parts has been proven at large
photon virtuality $Q^{2}$, large invariant energy $W$ and fixed Bjorken-x
($x_{B}$) [4, 2]. The contribution of transversely polarized virtual photons
for which factorization is not explicitly proven, is typically treated as a
higher twist-effect in current phenomenological models [5]. While the GPD
framework is already well established for the study of the three-dimensional
structure of the ground state nucleon, theoretical attempts have been made to
extend this framework to excited nucleon states [6, 7]. For this purpose, a
new set of GPDs, the so-called transition GPDs, have been introduced.

For the special case of the $N\to\Delta$ transition, there are in total 16
transition GPDs [7]. The first eight are helicity-non-flip (twist-2)
transition GPDs (compared to four chiral even GPDs for the ground state
nucleon), of which four are unpolarized and the other four polarized. The
unpolarized twist-2 transition GPDs $G_{1}$ \- $G_{4}$ can be related to the
Jones-Scadron electromagnetic form factors for the $N\to\Delta$ transition [8,
9], while the polarized transition GPDs $\widetilde{G}_{1}$ \-
$\widetilde{G}_{4}$ are related to the Adler form factors [9, 10, 11]. Similar
to ordinary deeply virtual meson production, where the description of the
twist-3 sector requires the introduction of four additional transversity GPDs,
the description of hard exclusive $N\to N^{*}$ pion production ($eN\to
e^{\prime}N^{*}\pi$) requires the introduction of eight additional helicity
flip / transversity transition GPDs ($G_{T_{1}}$ \- $G_{T_{8}}$), which
describe the impact of the transversely polarized virtual photons on the
twist-3 amplitudes [7]. Hard exclusive electroproduction of
$\pi^{-}\Delta^{++}$ has been theoretically described based on transition GPDs
in Ref. [7]. It has been shown that in total 12 of the 16 transition GPDs
contribute to the exclusive electroproduction of $\pi^{-}\Delta^{++}$. So far,
these transition GPDs are only poorly known based on symmetry relations in the
large $N_{C}$ limit [13, 14, 15, 12, 7] and no experimental data exists that
would allow access to these GPDs.

The measurement of hard exclusive
($\gamma^{*}p\to\pi^{-}\Delta^{++}\to\pi^{-}[p\pi^{+}]$) electroproduction
beam-spin asymmetries in this work is expected to represent a first observable
sensitive to $N\to\Delta$ transversity transition GPDs and $N\to\Delta$
transition GPDs in general. In analogy to ordinary GPDs [16], it is expected
that the production of charged pions is especially sensitive to the tensor
charge of the $\Delta$ resonance, which is so far completely unexplored. As
shown in Fig. 1, the soft parts of the convolution can be described with
transition GPDs and a meson distribution amplitude (DA).

Figure 1: Hard exclusive $\pi^{-}\Delta^{++}$ electroproduction off the proton
in very forward kinematics ($-t/Q^{2}\ll 1$) with the virtuality $Q^{2}$ and
the four-momentum transfer $t$ to the $\Delta^{++}$.

It is assumed for this process that the QCD factorization theorem is valid
within the Bjorken limit: $-t/Q^{2}\ll 1$ and $x_{B}$ fixed, with the
additional condition $Q^{2}>>m^{2}_{\Delta}$ as discussed in Ref. [12] for the
$p\to\Delta$ deeply virtual Compton scattering (DVCS) process. However, a
direct proof of the factorization for the investigated channel is not
available yet.

Previous attempts to extract observables sensitive to $N\to N^{*}$ transition
GPDs based on $p\to N^{*}$ DVCS ($ep\to e^{\prime}N^{*}\gamma$), but also
based on $p\to N^{*}$ $\pi^{0}$ and $\pi^{+}$ production [17, 18], suffered
from low statistics and a clean separation between the produced $\Delta$
events and the overlapping events from other nucleon resonances, as well as
from non-resonant background. In contrast to these, the $p\to N^{*}$ $\pi^{-}$
production studied in this work with high statistics, focuses on the
$\Delta^{++}$ resonance with an isospin $I_{z}=+3/2$, which is only fulfilled
by $\Delta$ resonances. Therefore, a large gap exists between the
$\Delta(1232)$ and the higher-mass $\Delta$ resonances, starting at masses of
1.6 GeV and showing a strongly suppressed branching ratio compared to the
$\Delta(1232)$, which allows a relatively clean extraction of the dominant
$\Delta(1232)$ resonance and a clear identification and subtraction of the
non-resonant background. Previous studies of this channel [19, 17] were
strongly limited by low statistics and therefore constrained to the low
$Q^{2}$ regime.

In exclusive electroproduction experiments, GPDs are typically accessed
through differential cross sections and beam and target polarization
asymmetries [20, 21, 22]. The focus of this work is on the extraction of the
structure function ratio $\sigma_{LT^{\prime}}/\sigma_{0}$ from beam-spin
asymmetry (BSA) measurements. In the one-photon exchange approximation the BSA
is defined as [20, 21]:

$\displaystyle
BSA=\frac{\sqrt{2\epsilon(1-\epsilon)}\frac{\sigma_{LT^{\prime}}}{\sigma_{0}}\sin\phi}{1+\sqrt{2\epsilon(1+\epsilon)}\frac{\sigma_{LT}}{\sigma_{0}}\cos\phi+\epsilon\frac{\sigma_{TT}}{\sigma_{0}}\cos
2\phi},$ (1)

where the structure functions $\sigma_{L}$ and $\sigma_{T}$ that contribute to
$\sigma_{0}=\sigma_{T}+\epsilon\sigma_{L}$, correspond to coupling to
longitudinal and transverse virtual photons, and $\epsilon$ describes the flux
ratio of longitudinally and transversely polarized virtual photons.
$\sigma_{LT}$, $\sigma_{TT}$ and the polarized structure function
$\sigma_{LT^{\prime}}$ describe the interference between their amplitudes.
$\phi$ is the azimuthal angle between the electron scattering and the hadronic
reaction plane.

For the present study, hard exclusive $\pi^{-}\Delta^{++}$ electroproduction
was measured at Jefferson Laboratory (JLab) with CLAS12 (CEBAF Large
Acceptance Spectrometer for experiments at 12 GeV) [23]. The incident
longitudinally polarized electron beam had energies of 10.2 GeV and 10.6 GeV,
impinging on an unpolarized liquid-hydrogen target. The CLAS12 forward
detector consists of six identical sectors within a toroidal magnetic field.
The momentum and the charge of the particles were determined by 3 regions of
drift chambers from the curvature of the particle trajectories in the magnetic
field. The electron identification was based on a lead-scintillator
electromagnetic sampling calorimeter in combination with a Čherenkov counter.
Pions and protons were identified by time-of-flight measurements.

For the selection of deeply inelastic scattered electrons, cuts on $Q^{2}>$
1.5 GeV2, the energy fraction of the beam carried by the virtual photon $y<$
0.75 and the invariant mass of the hadronic final state $W>$ 2 GeV, were
applied. To select the exclusive $e^{\prime}\pi^{-}\Delta^{++}$ final state,
events with exactly one electron, one $\pi^{-}$ and one proton were detected,
and the missing $\pi^{+}$ was selected via a cut on the $\pi^{+}$ peak in the
$e^{\prime}p\pi^{-}X$ missing mass spectrum and assigned to the missing
4-vector. In addition, only forward kinematics were selected by a cut on
$-t<1.5$ GeV2. The selected events are displayed in the mass distribution
shown in Fig. 2.

Figure 2: Distribution of the $p\pi^{+}$ versus $\pi^{+}\pi^{-}$ invariant
mass for exclusive events. The red lines indicate the applied cuts on
$M_{\pi^{+}\pi^{-}}>1.1$ GeV and $M_{p\pi^{+}}<1.3$ GeV to select the events
of interest.

It can be clearly seen that besides $\pi^{-}\Delta^{++}$ production, dominant
contributions from $p\rho^{0}$ and other non-resonant background sources are
also present. The dominant background from exclusive $\rho$ production was
reduced by a cut on the invariant two pion mass $M_{\pi^{+}\pi^{-}}>1.1$ GeV
to a level of less than 0.8%. In addition, a cut on the $p\pi^{+}$ invariant
mass $M_{p\pi^{+}}<1.3$ GeV was applied for the final analysis to select the
$\pi^{-}\Delta^{++}$ events and to reduce the non-resonant background that
dominates below the tail of the resonance mass at larger $p\pi^{+}$ invariant
masses.

The remaining non-resonant background was studied by comparing the data to two
Monte Carlo (MC) samples. The first sample was based on a full semi-inclusive
deep-inelastic scattering (SIDIS) generator [24], which contains all non-
resonant background channels but not the exclusive $\pi^{-}\Delta^{++}$
production in forward kinematics. It can therefore be used to reproduce the
background shape. As a second sample, an exclusive $\pi^{-}\Delta^{++}$
generator with literature values [25] for the mass and full width at half
maximum of the $\Delta^{++}$ was used to reproduce the signal events. Both
samples were processed through the full Geant4-based [26, 27] simulation and
reconstruction chain. Good agreement for all underlying variables was
observed. Figure 3 (upper row) shows the $\Delta^{++}$ peak in the $p\pi^{+}$
invariant mass of the experimental data (without a cut on this mass) in
comparison to the non-resonant background obtained with the SIDIS MC for
selected bins of $-t$ in the forward region, integrated over $Q^{2}$ and
$x_{B}$. Figure 3 (lower row) shows the $\Delta^{++}$ peak in the same bins
after the subtraction of the background in comparison to the result from the
exclusive MC. Both MC samples were scaled to match the data.

Figure 3: Upper row: $\Delta^{++}$ peak in the $p\pi^{+}$ invariant mass of
the experimental data (blue) in comparison to the non-resonant background
obtained with the SIDIS MC (black) for selected bins of $-t$ in the forward
region ($Q^{2}$ = 2.48 GeV2, $x_{B}$ = 0.27). Lower row: $\Delta^{++}$ peak in
the same bins after the subtraction of the background (blue) in comparison to
the result from the exclusive MC (green) and a fit (red, see text).

It can be observed that the non-resonant background is small close to
$t_{min}$ but increases to $\approx$ 40% for the largest $-t$ bins considered,
making a background subtraction necessary. The signal-to-background ratios
were directly determined from the SIDIS MC in comparison to the experimental
data.

Figure 4 shows the $Q^{2}$ versus $x_{B}$ distribution of the exclusive
events, together with the applied binning scheme.

Figure 4: Distribution of $Q^{2}$ versus $x_{B}$. The bin boundaries are shown
as red lines (bin 1: $x_{B}<$ 0.23; bin 2: $x_{B}>$ 0.23, $Q^{2}<$ 2.6 GeV2;
bin 3: $x_{B}>$ 0.23, $Q^{2}>$ 2.6 GeV2).

For each of the three $Q^{2}$-$x_{B}$ bins, up to seven bins in $-t$ and 9
bins in $\phi$ were defined to extract the BSA. The BSA was determined
experimentally from the number of counts with positive and negative helicity
($N^{\pm}_{i}$), in a specific bin $i$ as:

$\displaystyle
BSA_{i}=\frac{1}{P_{e}}\frac{N^{+}_{i}-N^{-}_{i}}{N^{+}_{i}+N^{-}_{i}},$ (2)

where $P_{e}$ = 86.6% $\pm$ 2.7% is the average magnitude of the beam
polarization, which was measured with a Møller polarimeter upstream of CLAS12
[23].

The raw asymmetry was extracted from the defined signal region
($M_{p\pi^{+}}<1.3$ GeV) and the background asymmetry was determined from the
sideband in the region 1.45 GeV $<$ $M_{p\pi^{+}}$ $<$ 1.65 GeV. It was
checked that the non-resonant background from the neutral resonances
($N^{*}\to p\pi^{-}$) homogeneously spreads over the sideband and signal
region in the $p\pi^{+}$ invariant mass. The background asymmetry, which was
found to be between 0.0 and -0.04, was determined for each $Q^{2}$, $x_{B}$
and $-t$ bin and subtracted from the signal asymmetry in each bin.

As a second completely independent method, a bin-by-bin background subtraction
was performed based on a fit of the complete distribution (signal +
background) with a so-called “Sill” function, which is a Breit-Wigner
distribution including threshold effects [28] plus a fifth-order polynomial
background in each $Q^{2}$, $x_{B}$, -$t$ and $\phi$ bin and for each helicity
state. After the combined fit, the signal and background contributions were
separated and the asymmetry was calculated based on the pure signal events. It
was found that both methods provided consistent results for the signal
asymmetry within the statistical uncertainty.

To extract the structure function ratio $\sigma_{LT^{\prime}}/\sigma_{0}$, the
BSA was plotted as a function of the azimuthal angle $\phi$ (see Fig. 5). A
fit of the data with a $\sin\phi$ function was then applied.

Figure 5: BSA as a function of $\phi$ for representative $-t$ bins ($Q^{2}$ =
2.48 GeV2, $x_{B}$ = 0.27). The red line shows the $\sin\phi$ fit.

It can be seen that a precise measurement of the $\phi$ dependence, which can
be well described by a $\sin\phi$ shape, is possible.

The main source of systematic uncertainty is given by the background
subtraction. It was determined by varying the signal-to-background ratio and
the background asymmetry within the estimated uncertainty ranges and was found
to be on the order of 1.5 - 12.5% (depending on the $-t$ bin). Also the
difference between the sideband-based background subtraction and the bin-by-
bin-based background subtraction was found to be within these limits. Also the
impact of the denominator terms in Eq. (1) on
$\sigma_{LT^{\prime}}/\sigma_{0}$ was evaluated and found to be on the order
of 2.8%, which was treated as part of the systematic uncertainty. The
systematic effect due to the uncertainty of the beam polarization (3.1%) was
determined based on the uncertainty of the measurement with the Møller
polarimeter. To estimate the impact of acceptance and bin-migration effects, a
Geant4-based MC simulation [27] was performed. The impact of these effects was
evaluated by comparing the injected and reconstructed asymmetries and was
found to be on the order of 2.9%. Also acceptance effects from the decay
products of the $\Delta^{++}$ were evaluated and found to be of the same
order. Systematic uncertainties due to radiative effects have been studied
based on Ref. [29], and were found to be on the order of 3.0%. Several
additional sources of systematic uncertainty, including particle
identification and the effect of fiducial volume definitions, were found to be
small ($<$2.0%). The total systematic uncertainty in each bin was defined as
the square-root of the quadratic sum of the uncertainties from all sources. On
average it was found to be on the order of 7.1 - 14.3% (depending on the $-t$
bin), which is smaller than the statistical uncertainty in most kinematic
bins.

Figure 6 shows the final results for $\sigma_{LT^{\prime}}/\sigma_{0}$ in the
region of small $-t$, where a description based on transition GPDs is expected
to be applicable, and compares them to measurements from the hard exclusive
$\pi^{+}n$ and $\pi^{0}p$ electroproduction from Refs. [32, 33], which can be
described with ground state GPDs. The structure function ratio
$\sigma_{LT^{\prime}}/\sigma_{0}$ for $\pi^{-}\Delta^{++}$ is clearly negative
in all kinematic bins and shows a shape that can be explained by the
contributing structure functions. The integrated cross section
$\sigma_{0}=\sigma_{T}+\epsilon\sigma_{L}$, which provides the denominator of
the ratio, is typically forward peaked due to the pion pole term contribution,
while $\sigma_{LT^{\prime}}$ which is the numerator of the measured ratio, is
constrained to be zero at $t=t_{min}$ due to angular momentum conservation.
This behavior can be observed for $\pi^{-}$ as well as for $\pi^{+}$, while
$\pi^{0}$ shows a more constant behavior with a continuous decrease for
increasing $-t$ over all kinematic bins due to the missing pion pole term.

Figure 6: $\sigma_{LT^{\prime}}/\sigma_{0}$ and its statistical uncertainty
for $\pi^{-}\Delta^{++}$ (black, this work) as a function of
$-t^{\prime}=(|t|-|t_{min}|)$ in the forward kinematic regime and its
systematic uncertainty (gray band). The sub-figures correspond to the results
for the different $Q^{2}$ and $x_{B}$ bins defined in Fig. 4. The mean
kinematics are shown on top of each sub-figure. The corresponding result
tables can be found in the supplemental material [30] and can be downloaded
from Ref. [31]. For comparison, the results from the hard exclusive $\pi^{+}n$
(red, Ref. [32]) and $\pi^{0}p$ (blue, Ref. [33]) electroproduction with
similar kinematics are shown.

As an interesting feature, the magnitude of the structure function ratio for
$\pi^{-}$ production is approximately two times larger than for $\pi^{+}$
production and has an opposite sign. The opposite sign can be directly
explained by the quark polarization. For $ep\to e^{\prime}n\pi^{+}$, the
polarized $\gamma^{*}$ removes a longitudinally polarized $u$-quark from the
proton ($uud$). This $u$ quark then combined with a $\bar{d}$ from a
$d\bar{d}$ vacuum pair with the $d$-quark returning to create the final state
neutron ($udd$). In contrast to this, for $ep\to
e^{\prime}\Delta^{++}\pi^{-}$, the polarized $\gamma^{*}$ kicks out a
longitudinally polarized $d$-quark from the proton, which combines with a
$\bar{u}$ from a $u\bar{u}$ vacuum pair with the $u$-quark returning to
produce the final state $\Delta^{++}$ ($uuu$). For the $\pi^{0}$ a clear
assignment to one quark type cannot be made due to the mixed content of its
wave function. It is known that within the valence region, the polarization of
$d$-quarks, $\Delta d$, in the proton is negative, while the polarization of
$u$-quarks, $\Delta u$, is positive and that the polarization of $\bar{u}$\-
and $\bar{d}$-quarks in the proton is small [34]. If we now look into the
$\pi^{+}$ ($\left|u\bar{d}\right\rangle$) and $\pi^{0}$
($1/\sqrt{2}\left[\left|u\bar{u}\right\rangle-\left|d\bar{d}\right\rangle\right]$)
BSAs, both are positive. The $\pi^{+}$ production is clearly dominated by
$u$-quarks, while for $\pi^{0}$ the negative sign in front of the $d\bar{d}$
part of the wave function turns the polarization contribution from the
$d$-quark around and causes a sizable positive asymmetry. This asymmetry is in
some regions similar to $\pi^{+}$ even though there is no amplification by the
pion pole. On the other hand, $\pi^{-}$ ($\left|d\bar{u}\right\rangle$) BSAs
are negative, since they are dominated by $d$-quarks. The reaction $ep\to
e^{\prime}\Delta^{++}\pi^{-}$ can therefore provide access to the polarized
$d$-quark content within the proton, which is otherwise hard to probe. Since
the described effects are coming from polarized quarks, even bigger effects
with asymmetries on the order of 40% [7] are expected for double spin
asymmetries with a longitudinally polarized beam and a longitudinally
polarized target.

The absolute magnitude of the $u$\- and $d$-quark polarization is similar in
the proton [34]. Therefore, a similar absolute magnitude of the BSA would be
expected for $\pi^{+}$ and $\pi^{-}$ based on these simple considerations. To
correctly model the magnitude of the asymmetry, the excitation process from
the ground state proton to the $\Delta^{++}$ resonance has to be considered
through transition GPDs. According to this formalism, the measured cross
section ratio $\sigma_{LT^{\prime}}/\sigma_{0}$ is expected to be a twist-3
quantity. The $Q^{2}$ dependence of the presented results is consistent with
this assumption, since $\sigma_{LT^{\prime}}/\sigma_{0}$ in bin 3 (high
$Q^{2}$, high $x_{B}$) shows a decrease of the magnitude compared to
$\sigma_{LT^{\prime}}/\sigma_{0}$ in bin 2 (low $Q^{2}$, high $x_{B}$) as
expected based on the $1/Q$ suppression for twist-3 observables. In Ref. [7]
the first transition GPD-based predictions for the unpolarized partial cross
sections of the $ep\to e^{\prime}\Delta^{++}\pi^{-}$ process have been made.
In the large $N_{C}$ limit it is expected that the process is dominated by the
transversity transition GPDs $G^{3}_{T_{5}}$ and $G^{3}_{T_{7}}$ (the
superscript indicates the twist-3 nature), which can be related to the ground
state transversity GPD $H_{T}$ [7]. Therefore, it can be assumed that the
polarized structure function $\sigma_{LT^{\prime}}$, which is given by
products of convolutions of transversity and helicity non-flip transition GPDs
with sub-process amplitudes, shows the following relation to the two dominant
transversity transition GPDs:

$\sigma_{LT^{\prime}}\sim\sqrt{-t^{\prime}}~{}~{}\Im\left[G^{3}_{T_{5}}\cdot
A+c~{}G^{3}_{T_{7}}\cdot A^{\prime}\right],$ (3)

with an unknown kinematic factor $c$ and helicity amplitudes for
longitudinally polarized virtual photons $A$ and $A^{\prime}$, which are
determined by the helicity non-flip transition GPDs $\widetilde{G}_{3}$ and
$\widetilde{G}_{4}$ within the large $N_{C}$ limit.

Due to the large uncertainties of the so far nearly unconstrained transition
GPDs, the existing predictions on unpolarized cross sections have large
uncertainties [7]. For asymmetries, these uncertainties are expected to be
even larger due to the dependence on the imaginary part of helicity amplitude
products and the related, so far poorly known, relative phases of the helicity
amplitudes, making reliable predictions at the present stage impossible [7].
So far, only transition GPD-based predictions for the BSA of the $p\to\Delta$
DVCS process, based on the twist-2 transition GPDs, exist [12]. For this
process, the BSA of the $\Delta$ production is excepted to be approximately
20-40% larger than for the regular DVCS process of the ground state proton
[12]. This difference can be directly related to the increase of the magnitude
of the underlying twist-2 transition GPDs. Assuming that the twist-3
transition GPDs are affected from the inelasticity in a similar way, this
would lead to an increase by a factor 1.2-1.4. Considering the additional
amplification of $\sigma_{LT^{\prime}}/\sigma_{0}$ by the pion pole term for
charged pions, the expected theoretical prediction based on transition GPDs
would be roughly consistent with the experimentally observed factor of two in
the magnitude of $\sigma_{LT^{\prime}}/\sigma_{0}$ for $\pi^{-}$ in comparison
to $\pi^{+}$. However, more theoretical investigations and especially more
experimental constraints are necessary to obtain a reliable parameterization
of the transition GPDs and a reliable description of the hard exclusive
$N^{*}\pi$ production process.

In summary, we have performed a first multidimensional measurement of the
structure function ratio $\sigma_{LT^{\prime}}/\sigma_{0}$ for $\vec{e}p\to
e^{\prime}\pi^{-}\Delta^{++}$ at large photon virtualities above the resonance
region. The results have been discussed in the context of quark polarizations
and in relation to $p\to\Delta$ transition GPDs. The measurement can give us a
direct access to the $d$-quark content of the nucleon and can be seen as a
first measured observable sensitive to $p\to\Delta$ transition GPDs. The
observed results in comparison to the $\pi^{+}n$ and $\pi^{0}p$ final state,
agree well with the expectations for the effects of the inelasticity
introduced to the GPDs for the $p\to\Delta$ transition. The measurements
presented in this work have initiated first theoretical investigations of the
hard exclusive $\pi^{-}\Delta^{++}$ production based on transition GPDs [7].
This opens the path to the investigation of the 3D structure of resonances
from future measurements of the $N\to N^{*}$ DVCS process, as well as other
$N\to N^{*}$ deeply virtual meson production (DVMP) channels at JLab and at
the future EIC with an extension to the strangeness sector.

We acknowledge the outstanding efforts of the staff of the Accelerator and the
Physics Divisions at Jefferson Lab in making this experiment possible. We owe
much gratitude to P. Kroll for many fruitful discussions concerning the
interpretation of our results. This work was supported in part by the U.S.
Department of Energy, the National Science Foundation (NSF), the Italian
Istituto Nazionale di Fisica Nucleare (INFN), the French Centre National de la
Recherche Scientifique (CNRS), the French Commissariat pour l′Energie
Atomique, the UK Science and Technology Facilities Council, the National
Research Foundation (NRF) of Korea, the Helmholtz-Forschungsakademie Hessen
für FAIR (HFHF), the Deutsche Forschungsgemeinschaft (DFG) and the Chilean
Agency of Research and Development (ANID). The Southeastern Universities
Research Association (SURA) operates the Thomas Jefferson National Accelerator
Facility for the U.S. Department of Energy under Contract No. DE-
AC05-06OR23177.

## References

  * [1] A. V. Radyushkin, Phys. Rev. D 56, 5524 (1997).
  * [2] J. C. Collins, L. Frankfurt and M. Strikman, Phys. Rev. D 56, 2982 (1997).
  * [3] S. J. Brodsky, L. Frankfurt, J. F. Gunion, A. H. Mueller, M. Strikman, Phys. Rev. D 50, 3134 (1994).
  * [4] A. V. Radyushkin, Phys. Lett. B 385, 333 (1996).
  * [5] S. V. Goloskokov and P. Kroll, Eur. Phys. J. A 47, 112 (2011).
  * [6] A. V. Belitsky, A. V. Radyushkin, Phys. Rept. 418, 1 (2005).
  * [7] P. Kroll, K. Passek-Kumericki, Phys. Rev. D 107, 054009 (2023).
  * [8] H. F. Jones, M. D. Scadron, Ann. Phys. 81, 1 (1973).
  * [9] K. Goeke et al., Prog. Part. Nucl. Phys. 47, 401 (2001).
  * [10] S. Adler, Ann. of Phys. 50, 189 (1968).
  * [11] S. Adler, Phys. Rev. D 12, 2644 (1975).
  * [12] P. A. M. Guichon, L. Moss$\acute{e}$, M. Vanderhaeghen, Phys. Rev. D 68, 034018 (2003).
  * [13] L. L. Frankfurt, P. V. Pobylitsa, M. V. Polyakov and M. Strikman, Phys. Rev. D 60, 014010 (1999).
  * [14] L. L. Frankfurt, M. V. Polyakov, M. Strikman, and M. Vanderhaeghen, Phys. Rev. Lett. 84, 2589 (2000).
  * [15] K. Goeke, M. V. Polyakov, and M. Vanderhaeghen, Prog. Part. Nucl. Phys. 47, 401 (2001).
  * [16] G. R. Goldstein, J. O. G. Hernandez and S. Liuti, Phys. Rev. D 91, 114013 (2015).
  * [17] C. T. Day, D. J. Harding, J. S. Klinger et al., Phys. Rev. D 23, 576 (1981).
  * [18] S. Diehl, Proceedings of the DIS2022 conference (2022). https://doi.org/10.5281/zenodo.7233861
  * [19] K. Wacker, G. Drews, J. Knobloch et al. Nucl. Phys. B 144, 269 (1978).
  * [20] D. Drechsel and L. Tiator, J. Phys. G 18, 449 (1992).
  * [21] T. Arens, O. Nachtmann, M. Diehl and P. V. Landshoff, Z. Phys. C 74, 651 (1997).
  * [22] M. Diehl and S. Sapeta, Eur. Phys. J. C 41, 515 (2005).
  * [23] V. D. Burkert et al. (CLAS Collaboration), Nucl. Instr. and Meth. A 959, 163419 (2020).
  * [24] L. Mankiewicz, A. Schafer, and M. Veltri, Comput. Phys. Commun. 71, 305 (1992).
  * [25] R.L. Workman et al. (Particle Data Group), Prog. Theor. Exp. Phys. 2022, 083C01 (2022).
  * [26] S. Agostinelli et al. (GEANT4 Collaboration), Nucl. Instrum. Meth. A 506, 250 (2003).
  * [27] M. Ungaro et al., Nucl. Instrum. Meth. A 959, 163422 (2020).
  * [28] F. Giacosa, A. Okopinska, V. Shastry, The Europ. Phys. Journ. A 57, 336 (2021).
  * [29] A. Afanasev, I. Akushevich, V. Burkert, K. Joo, Phys. Rev. D 66, 074004 (2002).
  * [30] See supplemental material for the result tables (link).
  * [31] CLAS physics database, https://clas.sinp.msu.ru/cgi-bin/jlab/db.cgi.
  * [32] S. Diehl, A. Kim, K. Joo et al. (CLAS Collaboration), Phys. Lett. B 839, 137761 (2023).
  * [33] A. Kim, S. Diehl, K. Joo et al. (CLAS Collaboration), to be submitted to Phys. Lett. B (2023).
  * [34] C. Cocuzza, W. Melnitchouk, A. Metz, N. Sato, Phys. Rev. D 106, L031502 (2022).


# Solving Audio Inverse Problems with a Diffusion Model

###### Abstract

This paper presents CQT-Diff, a data-driven generative audio model that can,
once trained, be used for solving various different audio inverse problems in
a problem-agnostic setting. CQT-Diff is a neural diffusion model with an
architecture that is carefully constructed to exploit pitch-equivariant
symmetries in music. This is achieved by preconditioning the model with an
invertible Constant-Q Transform (CQT), whose logarithmically-spaced frequency
axis represents pitch equivariance as translation equivariance. The proposed
method is evaluated with objective and subjective metrics in three different
and varied tasks: audio bandwidth extension, inpainting, and declipping. The
results show that CQT-Diff outperforms the compared baselines and ablations in
audio bandwidth extension and, without retraining, delivers competitive
performance against modern baselines in audio inpainting and declipping. This
work represents the first diffusion-based general framework for solving
inverse problems in audio processing.

Index Terms— Audio systems, deep learning, inverse problems, signal
restoration.

##  1 Introduction

Audio restoration tasks, such as bandwidth extension, inpainting, and
declipping, are inverse problems with the aim to restore the signal from
observations that have suffered a known type of degradation. These problems
are ill-posed in the sense that several different restorations may be equally
plausible. Algorithms are typically constructed for a particular type of
degradation using domain knowledge, such as signal sparsity [1, 2], or the
low-rankness of the power spectrum [3]. Data-driven generative models have
also been recently proposed [4, 5]. A common shortcoming is that an algorithm
engineered for one type of degradation (e.g., declipping) is typically not
useful for others (e.g., inpainting).

In this work, we explore whether a single pre-trained generative audio model
could be useful for many different restoration tasks _without knowledge of
which types of degradations it will be applied to at inference time_. We base
our work on diffusion models [6, 7], a family of generative models that have
shown outstanding performance in different modalities, such as image [8],
video [9], speech [10, 11, 12, 13, 14], and audio generation [15, 16, 17].
Particularly relevant to us is that unconditional (purely generative)
diffusion models have been shown to be useful also in conditional restoration
settings where parts of the result are pre-determined, particularly in the
image domain [18, 19, 20, 21, 22]. Crucially, as these approaches do not
require paired data for training and the degradation model is only used during
inference, the models adapt to a new problem without retraining. To the best
of our knowledge, analogous methods have not been employed in audio audio
restoration. This paper addresses this research gap.

Training an unconditional audio diffusion model is the first step towards this
objective. We argue that this is overly challenging unless suitable inductive
biases are built in to the model. While unconditional audio generation has
been demonstrated using a raw audio waveform representation, the experiments
are limited to small speech datasets [10, 11], or non-tonal audio [15, 17],
with performance expected to decrease for more challenging datasets. Building
diffusion models using mel-spectrograms or magnitude spectrograms [16] leads
to a more expressive generative performance. However, the lack of
invertibility of these transforms blocks the required conditioning for solving
inverse problems. We design our diffusion model in the time domain, allowing
for maximum flexibility, and use an invertible time-frequency transform as a
preprocessing step.

Choosing the right transform plays a critical role in the performance of the
model. In diffusion-based speech enhancement, Richter et al. [13] used the
Short-Time Fourier Transform (STFT) and applied a neural network based on 2-D
convolutions adapted from image generation [7]. While this approach succeeded
for speech, we argue that, contrary to image processing, it is non-ideal for
signals with strong harmonics (music), as translation equivariance of
harmonics does not hold true in linear frequency. The Constant-Q Transform
(CQT), in which the frequency axis is logarithmically spaced, is a possible
solution to this problem. Since pitch-shifting corresponds to translation in
the CQT spectrogram, applying the convolutional operator is now appropriate.
The CQT is widely used in Music Information Retrieval [23], but, despite some
exceptions [24], is not common in audio generation tasks.

We first show that CQT preconditioning yields a high-quality diffusion model
for audio. We present a versatile framework called CQT-Diff for solving audio
restoration problems, from which we evaluate three: bandwidth extension, audio
inpainting, and declipping. Pitted against specialized algorithms, our model
yields state-of-the art or strongly competitive performance in listening
tests.

##  2 CQT-Diff framework

###  2.1 Score-based Generative Modeling

Consider a diffusion process where data $\bm{x}_{0}\sim p_{\text{data}}$ is
progressively diffused into Gaussian noise
$\bm{x}_{t_{\text{max}}}\sim\mathcal{N}(\mathbf{0},\sigma_{\text{max}}^{2}\mathbf{I})$
over time111The “diffusion time” $t$ must not be confused with the “audio
time”.  $t$ by applying a perturbation kernel
$p_{t}(\bm{x}_{t})=\mathcal{N}(\bm{x}_{t};\bm{x}_{0},\sigma_{t}^{2}\mathbf{I})$.
Diffusion models, also referred to as score-based generative models, generate
data samples by reversing the aforementioned diffusion process. Specifically,
diffusion models estimate the gradient of the log probability density with
respect to data $\nabla_{\bm{x}_{t}}\log p_{t}(\bm{x}_{t})$, properly known as
the score function.

In this paper, we follow the parameterization from [25]. The reverse diffusion
process can be defined by the following Stochastic Differential Equation
(SDE):

$\text{d}\bm{x}=-\sigma_{t}(\beta(t)\sigma_{t}+\dot{\sigma}_{t})\;\nabla_{\bm{x}}\log
p_{t}(\bm{x}_{t})\text{d}t+\sqrt{2\beta(t)}\sigma_{t}\;d\text{w},$ (1)

where d$t$ is an infinitesimal negative timestep, $d\text{w}$ is the standard
Wiener process when time flows backwards, $\dot{\sigma}_{t}$ is the time
derivative of the noise variance schedule, and $\beta(t)$ is, intuitively, a
function that defines the amount of stochasticity added into the process. Note
that stochasticity is, in fact, not needed and (1) could be defined as an
ordinary differential equation [7, 25]. However, stochasticity has a role in
correcting approximation errors during sampling [25] and is empirically found
to improve the generation quality in our model.

The score is intractable but can be approximated with a deep neural network
$s_{\theta}(\bm{x}_{t},\sigma_{t})\simeq\nabla_{\bm{x}_{t}}\log
p_{t}(\bm{x}_{t};\sigma_{t})$. The parameterized score can be written as
$s_{\theta}(\bm{x}_{t},\sigma_{t})=(D_{\theta}(\bm{x}_{t},\sigma_{t})-\bm{x}_{t})/\sigma_{t}^{2},$
where $D_{\theta}(\bm{x}_{t},\sigma_{t})$ is a denoiser to be trained with the
L2 objective:

$\mathbb{E}_{\bm{x}\sim
p_{\text{data}},\bm{\epsilon}\sim\mathcal{N}(\mathbf{0},\mathbf{I})}\left[\lambda(\sigma_{t})\lVert
D_{\theta}(\bm{x}+\sigma_{t}\bm{\epsilon},\sigma_{t})-\bm{x}\rVert_{2}^{2}\right],$
(2)

where $\lambda(\sigma_{t})$ is a weighting function. Once the model has been
trained, one can generate new data instances by substituting the estimated
score $s_{\theta}(\bm{x}_{t},\sigma_{t})$ in (1), and solving the SDE with a
numerical solver of choice.

###  2.2 Diffusion Models for Inverse Problems

Consider the goal of retrieving an unknown audio signal $\bm{x}$ from a set of
measurements or observations $\bm{y}=\mathcal{A}(\bm{x})+\bm{\epsilon}$, where
$\mathcal{A}(\cdot)$ is a known and differentiable degradation function, and
$\bm{\epsilon}\sim\mathcal{N}(0,\sigma_{y}^{2}\bm{I})$ is measurement noise.
The objective would be to sample from the posterior distribution given the
observations $p(\bm{x}|\bm{y})$. In the context of a diffusion model, this
would require substituting the score in (1) for the conditional score
$\nabla_{\bm{x}_{t}}\log p_{t}(\bm{x}_{t}|\bm{y})$.

One solution is to train a score model $s_{\theta}(\bm{x}_{t},\bm{y},t)$ that
depends explicitly on the observations [14, 12, 8]. However, these approaches
lack versatility as they require training with paired data and cannot
effectively adapt to unseen degradations. Instead, one can make use that,
applying Bayes, the posterior factorizes as $p(\bm{x}|\bm{y})\propto
p(\bm{x})p(\bm{y|x})$, which leads to

$\nabla_{\bm{x}_{t}}\log p_{t}(\bm{x}_{t}|\bm{y})=\nabla_{\bm{x}}\log
p_{t}(\bm{x}_{t})+\nabla_{\bm{x}_{t}}\log p_{t}(\bm{y}|\bm{x}_{t}).$ (3)

Then, an unconditional diffusion model can be used as a generative prior,
while the conditioning is applied by some other means. In some cases, the
likelihood can be indirectly applied by ensuring data consistency during
sampling, replacing parts of the internal predictions
$\hat{\bm{x}}_{0}=D_{\theta}(\bm{x}_{t},\sigma_{t})$ with the measurements
$\bm{y}$ during every discretized step [7, 19, 18, 20]. We refer to this
strategy as the data consistency (DC) method. Note that this method can only
be used in cases where an explicit replacement is available. Degradations such
as distortions or nonlinear-phase filtering cannot be treated by replacing
parts of the signal in the time domain.

Recent works [21, 22, 9] demonstrated that the aforementioned data consistency
trick produces suboptimal results, so they proposed an alternative strategy
defining the second term of (3) as

$\nabla_{\bm{x}_{t}}\log
p_{t}(\bm{y}|\bm{x}_{t})\simeq-\xi(t)\;\nabla_{\bm{x}_{t}}\lVert\bm{y}-\mathcal{A}(D_{\theta}(\bm{x}_{t},\sigma_{t}))\rVert^{2}.$
(4)

This solution derives from approximating the likelihood as a Gaussian
distribution of the form
$p_{t}(\bm{y}|\bm{x}_{t})\sim\mathcal{N}(\mathcal{A}(D_{\theta}(\bm{x}_{t},\sigma_{t})),\sigma_{y}^{2}\mathbf{I})$.
Following [9], we refer to this method as reconstruction guidance (RG). The
gradient operator $\nabla_{\bm{x}_{t}}$ requires backpropagating through the
degradation function $\mathcal{A}(\cdot)$, and through the neural network of
the denoiser $D_{\theta}(\bm{x}_{t},\sigma_{t})$. The scaling function
$\xi(t)$ defines the amount of guidance that is applied during the reverse
diffusion process. We empirically found that a robust parameterization of the
scaling function is $\xi(t)=\xi^{\prime}\sqrt{N}/(t\lVert G\rVert^{2})$, where
$G=\nabla_{\bm{x}_{t}}\lVert
y-\mathcal{A}(D_{\theta}(\bm{x}_{t},\sigma_{t}))\rVert^{2}$ are the guidance
gradients and $N$ is the audio segment size in samples. Then, choosing
$\xi^{\prime}=0$ would lead to an unconditional sampler, but selecting too
large a value for $\xi^{\prime}$ would often result with degenerated
solutions. Note that, if there is no noise in the observations, using
reconstruction guidance does not deprive us from applying data consistency
steps. In some cases, combining both strategies can help stabilize the
sampling.

###  2.3 Data Representation and Preprocessing

To minimize error propagation during sampling, we apply a similar
preconditioning as Karras et al. [25]. The denoiser is defined as

$D_{\theta}(\bm{x}_{t},\sigma_{t})=c_{\text{skip}}(\sigma_{t})\bm{x}_{t}+c_{\text{out}}(\sigma_{t})F_{\theta}(c_{\text{in}}(\sigma_{t})\bm{x}_{t},\tfrac{1}{4}\ln(\sigma_{t})),$
(5)

where $c_{\text{skip}}(\sigma_{t})$, $c_{\text{out}}(\sigma_{t})$, and
$c_{\text{in}}(\sigma_{t})$ are scaling parameters optimized in such a way
that the input and the output of $F_{\theta}$ have always close-to-unit
variance222These parameters are defined in Table 1 in [25] and depend on the
standard deviation of the dataset, which needs to be estimated beforehand..
Following the same approach, the loss weighting in (2) is defined as
$\lambda(\sigma_{t})=1/c_{\text{out}}(\sigma_{t})^{2}$.

Throughout this paper, $\bm{x}$ and $\bm{y}$ refer to signals in the raw
waveform domain. However, in our early exploration, we found that using the
time-domain waveform representation resulted in unsatisfactory results,
especially for music bandwidth extension, which requires preserving the
harmonic structure. Instead, the data is transformed using an invertible CQT
(ICQT), which is applied as a data preprocessing step inside $F_{\theta}$. As
stated in Sec. 1, the motivation for using the CQT is to explore the partial
pitch-equivariance symmetry that audio data exhibits in the log-spaced
frequency range. Thus, $F_{\theta}$ is composed as
$F_{\theta}=\text{ICQT}\circ F_{\theta}^{\prime}\circ\text{CQT}$, where
$F_{\theta}^{\prime}$ refers to the raw neural network layers operating in the
transform domain. Using this approach, the gradient is backpropagated through
the forward CQT and its inverse, or ICQT, and, thus, the chosen CQT
implementation must be both invertible and differentiable.

We use the CQT implementation based on non-stationary Gabor frames [26], which
guarantees perfect reconstruction and is differentiable, since it is a linear
transform implemented by applying FFT-processing. If set up for minimum
redundancy, this transform yields a non-uniform sampling grid where the time
resolution increases with frequency. Alternatively, we use a rasterized CQT
representation where the hop sizes for all frequencies are set to the minimum,
hence keeping the time resolution constant. This solution allows CQT
processing as a 2D matrix at the cost of adding redundancy to the data. In our
experiments, we use a frequency resolution of 64 bins per octave, and a span
of seven octaves.

Fig. 1: Diagram of the neural network architecture adapted for the CQT domain.
The design follows a U-Net structure, where only the time resolution is
modified. Each intermediate RBlock contains a stack of dilated convolutions in
the frequency axis.

###  2.4 Architecture Design

Next, we summarize how CQT-Diff fits audio data in the CQT-domain using the
neural network architecture shown in Fig. 1. The architecture is a U-Net where
only the time resolution is downscaled after each subsequent residual block
(RBlock), while the frequency resolution is maintained. The non-local
dependencies in the frequency domain are tackled with stacks of exponentially-
increasing dilated convolutions without bias, accompanied with GELU
nonlinearities and residual connections. Inspired by [7], we use anti-aliased
downsampling operators and a progressive growing structure for the encoder and
decoder sides [27]. The diffusion step is shown to the network as
$\tfrac{1}{4}\ln(\sigma_{t})$, and is encoded using Random Fourier Features
(RFF) [28] and a multi-layer perceptron. The embeddings are applied as a
conditioner in every RBlock using FiLM modulations [29]. We concatenate RFF-
based absolute frequency positional embeddings to the CQT-transformed input,
meant to propagate absolute frequency information in the early layers of the
network.

##  3 Experiments and results

This section summarizes our experiments on three different audio inverse
problems: bandwidth extension, inpainting, and declipping. Audio examples are
included in the companion
webpage333research.spa.aalto.fi/publications/papers/icassp23-cqt-diff, and
reproducible code for all the experiments can be found in the official
repository444github.com/eloimoliner/CQTdiff, to which we refer for further
implementation details.

We train and test our model on the MAESTRO dataset [30], which contains
classical solo piano recordings. For ease of use, all tracks are resampled to
22.05 kHz. For sampling, we use the second-order stochastic sampler from [25]
(Algorithm 2), with the stochasticity parameter $S_{\text{churn}}=5$. Also
following [25], we assign the noise variance as $\sigma_{t}=t$, with $t$
distributed according to the schedule

$t_{i<T}=\left(\sigma_{\text{max}}^{\;\;\;\frac{1}{\rho}}+\tfrac{i}{T-1}\left(\sigma_{\text{min}}^{\;\;\;\frac{1}{\rho}}-\sigma_{\text{max}}^{\;\;\;\frac{1}{\rho}}\right)\right)^{\rho}.$
(6)

In these experiments , we use $\sigma_{\text{min}}=10^{-4}$,
$\sigma_{\text{max}}=1$, and $\rho=13$. We found this schedule to be
empirically effective for sampling with a reduced number of discretization
steps. We use $T=35$ discretization steps for the bandwidth extension and
inpainting experiments, while, for declipping, we raised the number of steps
to $T=140$. Regarding the inference speed, with $T=35$ steps, CQT-Diff
requires 17 s for processing a 2.97-s segment in a GPU. The inference time
augments up to 84 s when reconstruction guidance is used, as this feature
requires gradient computation.

The neural network $F_{\theta}^{\prime}$ was trained using the Adam optimizer
with a learning rate of $2\cdot 10^{-4}$. During training, the diffusion time
$t$ is randomly sampled following (6), but the parameters are set as
$\sigma_{\text{min}}=10^{-6}$, $\sigma_{\text{max}}=10$, and $\rho=10$. We set
a segment size of 2.97 s ($2^{16}$ samples) and a batch size of 4, but, as the
architecture is fully convolutional, the segment size can be freely modified
during inference. The model was trained for 320k iterations, which took about
three days on a single NVIDIA A100 GPU.

###  3.1 Bandwidth Extension

The first set of experiments concerns audio bandwidth extension, where the
degradation function is a lowpass filter
$\bm{y}=\mathcal{A}(\bm{x})=\text{LPF}(\bm{x})$. In contrast to previous works
[4, 14, 31], our method does not suffer from filter generalization as a
consequence of its fully generative nature, and, as we demonstrate in the
companion webpage, can be applied after many different types of lowpass
filters, even when the observations contain Gaussian noise. Nevertheless, we
restrict the evaluation to FIR filters designed with a Kaiser window, with
order 500 and two cutoff frequencies ($f_{\text{c}}=$ 1 and 3 kHz). In this
setting, it is possible to apply data consistency steps, where the observed
low frequencies are replaced with the denoised estimate following
$\bar{\bm{x}}_{0}=\bm{y}+\hat{\bm{x}}_{0}-\text{LPF}(\hat{\bm{x}}_{0})$, in a
way similar to [18].

We conduct an objective evaluation using 53 complete recordings (about 6 h of
music) from the MAESTRO test set, which were not used during training. As
objective metrics, we report in Table 1 the averaged Log-Spectral-Distance
(LSD), Frechet Audio Distance (FAD), and the F1 score (onset) of a
transformer-based piano transcription model [32] (F1)555The MAESTRO dataset
includes the ground-truth MIDI transcriptions.. For this task, we experiment
with applying our method (CQT-Diff) in two conditioning settings: using only
data consistency (DC) and combining it with reconstruction guidance (DC+RG),
with the scaling parameter $\xi^{\prime}=0.1$. We include two conditions where
the CQT-based neural architecture is replaced with one using the STFT (STFT-
Diff) and raw audio (SaShiMi [11]). We also compare with BEHM-GAN [4], a
recent method for music bandwidth extension which we retrained using the same
dataset. The BEHM-GAN was trained using lowpass filters with randomized cutoff
frequencies at around 3 kHz, so it could only be used as the baseline in the
$f_{\text{c}}=$ 3 kHz setting. Table 1 also reports the number of trainable
parameters (#prms) as an indicator of the model capacity.

We conducted a subjective listening test following the MUSHRA recommendation.
In the test, 12 volunteers were asked to rate the different methods in terms
of perceptual audio quality in comparison with a reference. The test was split
into two parts, each evaluating the bandwidth extension from the cutoff
frequencies of, respectively, 1 kHz and 3 kHz. While the same low anchor was
used for both parts (the lowpass-filtered sample at $f_{\text{c}}$ = 1 kHz),
the compared conditions differed. In the first part, we included the two
variants of our method and the STFT-Diff and SaShiMi-Diff conditions. In the
second part, the lowpass-filtered sample at $f_{\text{c}}$ = 3 kHz was
included as an extra condition, as well as the BEHM-GAN baseline, but the
STFT-Diff and SaShiMi-Diff conditions were excluded for length reasons. We
included four pages containing different audio examples for each part of the
test. The results of the subjective experiments are presented in Fig. 2.

After analyzing the evaluation results, we reached the following conclusions.
(i) The CQT data representation is advantageous against the STFT or raw audio
(SaShiMi), as its MUSHRA and F1 scores are significantly better. (ii) The use
of reconstruction guidance is not critical for this task, as the (DC) and
(DC+RG) conditions obtained comparable results. The reason may be that the
architecture is explicitly designed to capture long-range data dependencies in
the frequency range, thanks to the dilated convolutions. (iii) In the
$f_{\text{c}}=3$ kHz setting, the CQT-Diff (DC) condition obtained
significantly better subjective scores than the baseline BEHM-GAN (p-value
1.4$\cdot 10^{-5}$ in a paired t-test). (iv) As far as we are aware, this is
the first work succeeding at audio bandwidth extension with a cutoff frequency
as low as 1 kHz.

Table 1: Objective metrics for bandwidth extension. The best result in each
column is highlighted.

|  $f_{\text{c}}$ = 1 kHz |  $f_{\text{c}}$ = 3 kHz |
---|---|---|---
Method |  LSD $\downarrow$ |  FAD $\downarrow$ |  F1 $\uparrow$ |  LSD $\downarrow$ |  FAD $\downarrow$ |  F1 $\uparrow$ | #prms
LPF | 1.29 | 12.47 | 0.67 | 0.81 | 3.50 | 0.73 | -
BEHM-GAN | - | - | - | 0.68 | 0.37 | 0.93 | 82M
STFT-Diff | 0.95 | 0.48 | 0.68 | 0.71 | 0.21 | 0.93 | 65M
SaShiMi-Diff | 0.97 | 4.06 | 0.63 | 0.70 | 1.37 | 0.93 | 19M
CQT-Diff (DC) | 0.86 | 0.51 | 0.79 | 0.66 | 0.22 | 0.94 | 15M
CQT-Diff (DC+RG) | 0.86 | 0.50 | 0.80 | 0.65 | 0.17 | 0.94 | 15M

Fig. 2: Results of the bandwidth extension listening test.

###  3.2 Long Audio Inpainting

We apply our framework for the task of long audio inpainting, where the
degradation function is a linear binary mask
$\mathcal{A}(\bm{x})=(1-\mathbbm{1}_{[t_{\text{start}},t_{\text{end}}]})\bm{x}$,
which deletes a long signal segment between $t_{\text{start}}$ and
$t_{\text{end}}$. For this task, the conditioning can be applied by using the
data consistency steps as
$\bar{\bm{x}}_{0}=\bm{y}+\mathbbm{1}_{[t_{\text{start}},t_{\text{end}}]}\hat{\bm{x}}_{0}$,
and/or using reconstruction guidance, directly applying (4).

We have evaluated the success of our method in long audio inpainting by means
of a subjective experiment, which consisted of a multi-stimulus listening
test. In the test, an 11-s excerpt of piano music with a long gap, i.e.,
silence, in the middle was presented to the participants as the reference. The
participants were asked to rate the rest of the different conditions in terms
of plausibility of the reconstruction. The test included four audio examples
with three different gap lengths, forming a total of 12 pages of comparisons.
We evaluate CQT-Diff in the same conditioning settings as in Sec. 3.1, relying
solely on the data consistency (DC) and combining it with reconstruction
guidance (DC+RG), using the scaling parameter $\xi^{\prime}=0.35$. We compare
our method against two recent baselines: GACELA [5] and Catch-A-Waveform [33]
(CAW).

Fig. 3: Results of the audio inpainting listening test.

The main conclusions from the results of the subjective experiment reported in
Fig. 3 are the following: (i) In contrast with the outcomes from Sec. 3.1,
reconstruction guidance is critical for inpainting, given the significant
difference in the results. Our explanation is that the neural architecture
$F_{\theta}$ has not been explicitly designed to account for long-term time
dependencies and, thus, the guidance term is necessary to ensure coherence.
(ii) Although the confidence intervals overlap, the MUSHRA scores of CQT-Diff
are marginally higher than GACELA. We conclude that the proposed method shows
at least comparable performance against the baseline while being more
flexible.

###  3.3 Audio Declipping

Audio declipping refers to canceling the distortion in an audio signal that
has been hard-clipped. We treat declipping as a nonlinear inverse problem,
where the degradation function is a saturating nonlinearity
$\mathcal{A}(\bm{x})=(|\bm{x}+c|-|\bm{x}-c|)/2$, where $c$ is the clipping
threshold. In this case, we apply the conditioning on the clipped observations
based on, uniquely, the reconstruction guidance strategy.

We conducted an objective evaluation where CQT-Diff was compared against two
popular audio declipping methods: Analysis SPArse DEclipper (A-SPADE) [1] and
Social Sparsity with Persistent Empirical Wiener (SS-PEW) [2]. We report three
objective metrics: FAD [34], PEAQ, and PEMO-Q. The test set consists of 149
segments from different piano recordings of the MAESTRO test set, each of 2.97
s. Table 2 shows the evaluation results on two Signal-to-Distortion Ratio
(SDR) settings: 1 dB and 10 dB.

Table 2: Objective metrics on audio declipping tests.

| SDR = 1 dB | SDR = 10 dB
---|---|---
|  FAD $\downarrow$ |  PEAQ $\downarrow$ |  PEMO-Q $\downarrow$ |  FAD $\downarrow$ |  PEAQ $\downarrow$ |  PEMO-Q $\downarrow$
Clipped | 23.09 | -3.88 | -3.82 | 7.85 | -3.87 | -3.78
CQT-Diff (RG) | 1.84 | -3.53 | -3.36 | 0.68 | -2.99 | -2.56
A-SPADE | 8.62 | -3.87 | -3.58 | 0.59 | -2.97 | -2.60
SS-PEW | 9.69 | -3.78 | -3.62 | 0.46 | -2.05 | -0.93

Thanks to the generative priors, CQT-Diff has an advantage over the other
approaches in the most severe circumstances: when SDR = 1 dB, it achieved
significantly higher scores than the compared baselines in the three reported
metrics in Table 2. However, in moderate cases (SDR = 10 dB), the performance
of CQT-Diff is upper-bounded by the still imperfect generation quality of the
diffusion model. This explains why CQT-Diff is evidently outperformed by SS-
PEW when SDR = 10 dB, although having comparable performance to A-SPADE.

##  4 Conclusions

We proposed CQT-Diff, a diffusion-based generative model which can be used as
a flexible framework for audio restoration. In objective and subjective
experiments, CQT-Diff excels at the task of audio bandwidth extension and
shows competitive performance to the compared baselines in long audio
inpainting and declipping, without the need of retraining for each task in
particular. Apart from the three evaluated tasks, we emphasize that the
proposed model could be flexibly applied for solving a much wider range of
linear or nonlinear ill-posed problems, as long as the degradation is known.
Future work should target adapting the model to be trained in a large-scale
data regime, which would allow for more practical applications. This work
shows the potential of diffusion models for audio restoration, and how they
can benefit from using invertible time-frequency representations.

##  5 Acknowledgments

We thank Tero Karras, Miika Aittala, Samuli Laine and Timo Aila for the
helpful discussions and for proposing the idea of using a Constant-Q
Transform. We acknowledge the computational resources provided by the Aalto
Science-IT project.

## References

  * [1] S. Kitić, N. Bertin, and R. Gribonval,  “Sparsity and cosparsity for audio declipping: A flexible non-convex approach,”  in Proc. Int. Conf. Latent Variable Analysis and Signal Separation, Aug. 2015, pp. 243–250.
  * [2] K. Siedenburg, M. Kowalski, and M. Dörfler,  “Audio declipping with social sparsity,”  in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), May 2014, pp. 1577–1581.
  * [3] Ç. Bilen, A. Ozerov, and P. Pérez,  “Solving time-domain audio inverse problems using nonnegative tensor factorization,”  IEEE Trans. Signal Process., vol. 66, no. 21, pp. 5604–5617, Sep. 2018.
  * [4] E. Moliner and V. Välimäki,  “BEHM-GAN: Bandwidth extension of historical music using generative adversarial networks,”  IEEE/ACM Trans. Audio Speech Lang. Process. (Early Access), Jul. 2022.
  * [5] A. Marafioti, P. Majdak, N. Holighaus, and N. Perraudin,  “GACELA: A generative adversarial context encoder for long audio inpainting of music,”  IEEE J. Selected Topics Signal Process., vol. 15, no. 1, pp. 120–131, Nov. 2020.
  * [6] J. Ho, A. Jain, and P. Abbeel,  “Denoising diffusion probabilistic models,”  in Proc. NeurIPS, Dec. 2020, pp. 6840–6851.
  * [7] Y. Song, J. Sohl-Dickstein, D. P Kingma, et al.,  “Score-based generative modeling through stochastic differential equations,”  in Proc. ICLR, May 2021.
  * [8] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer,  “High-resolution image synthesis with latent diffusion models,”  in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Jun. 2022, pp. 10684–10695.
  * [9] J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet,  “Video diffusion models,”  in Proc. NeurIPS, Dec. 2022.
  * [10] Z. Kong, W. Ping, J. Huang, K. Zhao, and B. Catanzaro,  “Diffwave: A versatile diffusion model for audio synthesis,”  in Proc. ICLR, May 2021.
  * [11] K. Goel, A. Gu, C. Donahue, and C. Re,  “It’s raw! Audio generation with state-space models,”  in Proc. 39th Int. Conf. Machine Learning, Mar 2022, pp. 7616–7633.
  * [12] J. Serrà, S. Pascual, J. Pons, R. O. Araz, and D. Scaini,  “Universal speech enhancement with score-based diffusion,”  arXiv preprint arXiv:2206.03065, 2022.
  * [13] J. Richter, S. Welker, J. Lemercier, et al.,  “Speech enhancement and dereverberation with diffusion-based generative models,”  arXiv preprint arXiv:2208.05830, 2022.
  * [14] S. Han and J. Lee,  “NU-Wave 2: A general neural audio upsampling model for various sampling rates,”  in Proc. Interspeech, Sep. 2022.
  * [15] S. Rouard and G. Hadjeres,  “Crash: Raw audio score-based generative modeling for controllable high-resolution drum sound synthesis,”  in Proc. ISMIR, Nov. 2021.
  * [16] C. Hawthorne, I. Simon, A. Roberts, et al.,  “Multi-instrument music synthesis with spectrogram diffusion,”  arXiv preprint arXiv:2206.05408, 2022.
  * [17] E. Moliner and V. Välimäki,  “Realistic gramophone noise synthesis using a diffusion model,”  in Proc. Int. Conf. Digital Audio Effects (DAFX), Vienna, Austria, Sep. 2022.
  * [18] J. Choi, S. Kim, Y. Jeong, et al.,  “ILVR: Conditioning method for denoising diffusion probabilistic models,”  in Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV), Oct. 2021, pp. 14347–14356.
  * [19] B. Kawar, M. Elad, S. Ermon, and J. Song,  “Denoising diffusion restoration models,”  in Proc. ICLR Workshop on Deep Generative Models for Highly Structured Data, May 2022.
  * [20] A. Lugmayr, M. Danelljan, A. Romero, et al.,  “Repaint: Inpainting using denoising diffusion probabilistic models,”  in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Jun. 2022, pp. 11461–11471.
  * [21] H. Chung, B. Sim, D. Ryu, and J. C. Ye,  “Improving diffusion models for inverse problems using manifold constraints,”  arXiv preprint arXiv:2206.00941, 2022.
  * [22] H. Chung, J. Kim, M. T. Mccann, et al.,  “Diffusion posterior sampling for general noisy inverse problems,”  arXiv preprint arXiv:2209.14687, 2022.
  * [23] M. Won, S. Chun, O. Nieto, and X. Serra,  “Data-driven harmonic filters for audio representation learning,”  in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), May 2020, pp. 536–540.
  * [24] S. Huang, Q. Li, C. Anil, et al.,  “Timbretron: A wavenet(cycleGAN(CQT(audio))) pipeline for musical timbre transfer,”  in Proc. ICLR, May 2019.
  * [25] T. Karras, M. Aittala, T. Aila, and S. Laine,  “Elucidating the design space of diffusion-based generative models,”  in Proc. NeurIPS, Dec. 2022.
  * [26] G. A. Velasco, N. Holighaus, M. Dörfler, and T. Grill,  “Constructing an invertible constant-Q transform with non-stationary Gabor frames,”  in Proc. Int. Conf. Digital Audio Effects (DAFX), Paris, France, 2011.
  * [27] T. Karras, S. Laine, M. Aittala, et al.,  “Analyzing and improving the image quality of StyleGAN,”  in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition, Jun. 2020, pp. 8110–8119.
  * [28] M. Tancik, P. Srinivasan, B. Mildenhall, et al.,  “Fourier features let networks learn high frequency functions in low dimensional domains,”  in Proc. NeurIPS, Dec. 2020, vol. 33, pp. 7537–7547.
  * [29] E. Perez, F. Strub, H. De Vries, V. Dumoulin, and A. Courville,  “Film: Visual reasoning with a general conditioning layer,”  in Proc. AAAI Conf. Artificial Intelligence, Feb. 2018, vol. 32.
  * [30] C. Hawthorne, A. Stasyuk, A. Roberts, et al.,  “Enabling factorized piano music modeling and generation with the MAESTRO dataset,”  in Proc. ICLR, May 2019.
  * [31] S. Sulun and M. E. P. Davies,  “On filter generalization for music bandwidth extension using deep neural networks,”  IEEE J. Sel. Topics Signal Process., vol. 15, no. 1, pp. 132–142, Nov. 2020.
  * [32] C. Hawthorne, I. Simon, R. Swavely, et al.,  “Sequence-to-sequence piano transcription with transformers,”  in Proc. ISMIR, Dec. 2021.
  * [33] G. Greshler, T. Shaham, and T. Michaeli,  “Catch-a-waveform: Learning to generate audio from a single short example,”  in Proc. NeurIPS, 2021, vol. 34, pp. 20916–20928.
  * [34] K. Kilgour, M. Zuluaga, D. Roblek, and M. Sharifi,  “Fréchet audio distance: A reference-free metric for evaluating music enhancement algorithms,”  in Proc. Interspeech, Aug. 2019, pp. 2350–2354.