Computed Tomography (CT) is a prominent example of Imaging Inverse Problem
(IIP), highlighting the unrivalled performances of data-driven methods in
degraded measurements setups like sparse X-ray projections. Although a
significant proportion of deep learning approaches benefit from large
supervised datasets to directly map experimental measurements to medical
scans , they cannot generalize to unknown acquisition setups. In contrast,
fully unsupervised techniques, most notably using score-based generative
models, have recently demonstrated similar or better performances compared
to supervised approaches to solve IIPs while being flexible at test time
regarding the imaging setup. However, their use cases are limited by two
factors: (a) they need considerable amounts of training data to have good
generalization properties and (b) they require a backward operator, like
Filtered-Back-Projection in the case of CT, to condition the learned prior
distribution of medical scans to experimental measurements. To overcome
these issues, we propose an unsupervised conditional approach to the
Generative Latent Optimization framework (cGLO), in which the parameters of
a decoder network are initialized on an unsupervised dataset. The decoder is
then used for reconstruction purposes, by performing Generative Latent
Optimization with a loss function directly comparing simulated measurements
from proposed reconstructions to experimental measurements. The resulting
approach, tested on sparse-view CT using multiple training dataset sizes,
demonstrates better reconstruction quality compared to state-of-the-art
score-based strategies in most data regimes and shows an increasing
performance advantage for smaller training datasets and reduced projection
angles. Furthermore, cGLO does not require any backward operator and could
expand use cases even to non-linear IIPs.

§ INTRODUCTION

Imaging Inverse Problems (IIPs) are a large and swiftly growing area of
research. IIPs cover a large variety of applications, ranging from generic
image processing topics such as denoising, super-resolution and
deconvolution to more specific domains usually involving the
characterization of a system of interest from indirect observations. In this
latter case, the problem of reconstructing scans from medical imaging
devices such as Computed Tomography (CT), Magnetic Resonance Imaging (MRI)
and Positron Emission Tomography (PET) offers major challenges yet to be
solved. Among those, CT reconstruction is a prime example of medical IIPs.
The search of reduced ionizing radiations for patients has led to a
sparsification of X-ray projection angles and/or a lowering of X-ray
intensities resulting in noisier projection images. Using such degraded
measurement setups, reconstructions obtained by conventional numerical
methods, e.g. Filtered Back-Projection (FBP)
<cit.>, are severely worsened.

Recent progresses have been made using deep learning approaches for CT
reconstruction. However, a great part of these novel data-driven methods
employ a supervised training setup [2, 3, 4, 5, 6, 7].

In supervised training pairs, ground truths consist of 3D volumetric scans
reconstructed from conventional techniques such as the well-known FBP
algorithm, using high-dose and densely sampled X-ray measurements. To each
ground truth is associated a degraded set of measurements, obtained from
sparsification and/or dose reduction from the original X-ray projections.
Building such supervised datasets thus implies that a fixed physical model
has been employed for all training pairs, which must be identical to the one
used for inference. In other words, supervised reconstruction strategies
need a new training whenever the acquisition process changes. This can be
problematic when the viewing angles, X-ray spectrum or beam geometry vary.
To circumvent this drawback, several unsupervised strategies have recently
been introduced.

Current approaches dealing with ill-posed IIPs in an unsupervised way are
mostly based on the use of generative models. As such, the infamous
Generative Adversarial Networks (GANs) [8] are
largely employed for unsupervised reconstruction tasks [9].
As a serious contender to GANs, Score-based Generative Models (SGMs)
[10, 11] have gained a lot of
attention in the past few years, as recent improvements lead to the
generation of high-quality image samples without requiring a complex
adversarial optimization. The basic principle of SGMs is to add noise of
increasing intensity to data during the training process and learn to
reverse this process in order to make a generative model from a sequence of
trained denoisers. Using this general strategy, two types of approaches can
be distinguished: Denoising Diffusion Probabilistic Models (DDPM)
[11, 12, 13] and
Score Matching methods [10, 14, 15]. In particular, Score Matching has been used to
perform sparse CT reconstruction [16]. This method
estimates the gradient of the prior log-likelihood, also called the score
function, for each level of noise. In the continuous approach
[15], the discrete sequence of noise levels is
replaced by a continuum of distributions progressively diffusing data points
into random noise following a Stochastic Differential Equation (SDE)
describing a Markovian diffusion process. By estimating the score function
with a time-dependent neural network, the reverse-time SDE is approximated.
Using standard iterative sampling techniques then leads to generative

Both GANs and SGMs are very efficient to learn unconditional prior
distributions on slices of 3D reconstructed volumes. However, difficulties
arise when a conditional sampling is needed to reconstruct slices from
experimental X-ray projections. Various strategies have been proposed to
deal with this conditional sampling issue.

GAN inversion approaches [9] start from a trained
and fixed decoder, and aim at estimating the latent code corresponding to a
given observation. The inversion process can be learning-based
[17, 18], e.g. learning an
encoder to inverse the decoder, optimization-based where the optimization
problem is solved by finding the latent code minimizing the given objective
function [19, 20], or both
[21, 22, 23]. Other
approaches using a parameterization of the latent space have been proposed,
where the decoder and the encoder are jointly learned using an adversarial
process [24, 25, 26, 27, 28, 29, 30]. Similar to GANs, conditional inference
using SGMs is also quite unnatural. It is generally performed using a biased
sampling of the posterior distribution to generate approximate samples from
a stochastic process conditioned to experimental measurements. For instance,
in the context of sparse CT reconstruction, conditioning a SGM to X-ray
projections, as in [16], involves computing the FBP on
experimental measurements filled with simulated measurements from the
candidate reconstruction. To the best of our knowledge, this conditioned
version of a SGM (cSGM) currently achieves state-of-the-art for unsupervised
sparse CT reconstruction. However, cSGM is restricted to IIPs for which both
a forward and an approximate pseudo-inverse operators are available. While
such an inverse operator exists for single-material CT using for instance
the FBP algorithm, this approach cannot be extended to other IIPs where
pseudo-inverse operators are not defined, e.g. multi-material CT.

In addition to the conditional sampling issue, unsupervised generative
models require large amounts of training data, which can be difficult to
collect in the context of medical imaging. Furthermore, while these
approaches have good generalization given infinite training data, in
practice it is not clear how much is necessary.

On the other side of the data-consumption spectrum, Deep Image Prior (DIP)
[31] has demonstrated that the structure of a deep
convolutional network inherently captures a significant amount of low-level
image statistics without any learning. In other words, it induces an
implicit structural bias, i.e. a prior. DIP uses a randomly initialized
U-Net like architecture [32], and a fixed input
noise. The network weights can then be optimized to solve any ill-posed IIP
with known forward model, such as inpainting or super-resolution. It has
been shown that this IIP reparameterization, when optimized with gradient
descent, hierarchically reconstructs from low to high frequencies, i.e.    converging faster towards "natural images" than noise
[33, 34]. The method has
recently been adapted in [35] to solve low-dose and
sparse CT reconstruction. While this strategy showed good results on use
cases with missing or small datasets, it imposes a quite rough prior which
cannot compete with the reconstruction performances of its data-driven
counterparts in degraded measurement setups [16].

Considering, on one side, data-hungry generative models offering strong
priors and limited to specific IIPs, and data-free methods providing weak
priors on the other side, the question of a possible trade-off between these
two extremes naturally arises.

As a way to alleviate this dilemma, this paper proposes a conditional
version of the approach described as Generative Latent Optimization (GLO)
[36]. The core idea of the original GLO
technique is to use a decoder network and a set of learnable unit
noise vectors, i.e. latent codes, where each code is associated to a single
individual of the unsupervised training dataset. During training, the
gradient is back-propagated both on the weights of the decoder and on the
latent codes. In this GLO framework, the manifold defined by the latent
codes is not parameterized. A direct consequence is that it is not trivial
to sample from it. Decent generated samples can be produced when evaluating
the decoder on linear combinations of latent codes or when sampling from a
single full-covariance Gaussian fitted to the latent codes distribution.
While these generations do not suffer from mode collapse, they are
outperformed by GANs except when trained on small datasets. The vast
majority of articles building on GLO are focused on improving its generative
characteristics [37, 38, 39].

This paper differs from these recent works and from standard GAN inversion
techniques which focus on parameterizing the manifold entailed by the latent
codes. Similar to DIP, our conditioned version of GLO (cGLO) exploits the
inherent structural prior induced by the convolutional network of the
decoder. However, contrary to DIP, cGLO can benefit from an unsupervised
training dataset of any size to initialize its decoder weights, resulting in
greatly improved reconstruction results. The higher the quantity of training
data for initialization is used, the stronger the induced prior is.

Moreover, cGLO provides what we call self-regularization, by reconstructing
several slices at once, i.e. regularizing with the measurements themselves.
Finally, cGLO does not need a backward operator like the FBP algorithm at
any point to perform a CT reconstruction. The resulting approach is
unsupervised and very flexible, depending on the type and quantity of
available data.

As illustrated on the special case of sparse CT reconstruction, cGLO can be
used as a plug-and-play reconstruction method. This article demonstrates
that cGLO is a parsimonious reconstruction method that outperforms current
state-of-the-art unsupervised IIPs solvers in most data regimes.

§ METHODS

§.§ Computed Tomography (CT)

Parallel beam geometry profile acquisition for one viewing angle

Given a narrow beam of X-ray photons of energy $E \in [0, \varepsilon]$,
with a normalized dose spectrum $S(E)$, tracing through a material $m$
with homogeneous density distribution $\rho_m$, the beam intensity
decreases in accord with the Beer-Lambert law:

\begin{equation}
\label{spectrum_beam_attenuation}
\frac{I}{I_0} = \int_0^\varepsilon S(E) \exp \Biggl[
- \biggl(\frac{\mu_m}{\rho_m}\biggr)_E \, \rho_m L
\Biggr]dE
\end{equation}

where $I_0$ and $I$ are respectively the input and observed intensities,
$L$ is the beam path length and $\mu_m$ the linear attenuation
coefficient of material $m$. The ratio
$\Bigl(\frac{\mu_m}{\rho_m}\Bigr)_E$ corresponds to the mass attenuation
of material $m$. Depending on the energy level $E$, this macroscopic
attenuation is obtained by adding up the effect of several absorption
and scattering mechanisms, e.g. Rayleigh and Compton scattering,
photoelectric absorption or pair production.

In medical tomodensitometry, the quantity of interest is the
heterogeneous density distribution reflecting the patient internals.
However, further assumptions are made to recover this distribution from
experimental measurements: bones and other tissues are reduced to one
equivalent material with water mass attenuation and the X-ray beam is
considered monoenergetic.

Now, by considering an horizontal plane section through the patient
body, i.e. a slice $x$, and by moving the source and detector as
indicated in figure <ref>, it is possible to obtain a profile
$y$, on the detector axis $p$, for the viewing angle $\phi$:

\begin{equation}
\label{viewing_angle_profile}
y(p, \phi) = -\log\biggl(\frac{I}{I_0}\biggr)
= \int_L x\, ds
\end{equation}

For continuous $p$ and $\phi$ the profiles $y$ may be identified with
the two-dimensional Radon transform <cit.>, also
called the forward operator $\mathcal{T}$, of the slice $x$, such that
$y = \mathcal{T}(x)$. However, in practical use cases, projection
profiles are acquired for a set of incremental values $\Phi = \phi_1,
..., \phi_N$, which constitutes a sampling of the Radon transform.
Moreover, shot noise following a Poisson process $\tau$ usually appears
in experimental profiles for lower doses. Resulting experimental
measurements can then be described by:

\begin{equation}
\label{experimental_profiles}
y_\Phi = \mathcal{T}_\Phi(x) + \tau
\end{equation}

Given a number of viewing angles close to the Nyquist sampling criterion
<cit.>, and sufficiently low noise,
the inverse of the sampled Radon transform $\mathcal{T}_\Phi^\dagger$
can be computed with standard methods, e.g. FBP, to retrieve the slice
density map without producing artefacts. By stacking several slices, the
two-dimensional information may be converted to three-dimensional

In the case of insufficient viewing angles, e.g. sparse CT, or in
presence of too much noise, e.g. low-dose CT, the inverse problem
becomes ill-posed. It is classically solved using iterative optimization
algorithm [41] using a Maximum A Posteriori
(MAP) formulation of the problem:

\begin{equation}
\label{maximum_a_posteriori}
x^* = \argmin_x \lVert \mathcal{T}_\Phi(x)
- y_\Phi \rVert^2_2 + \mathcal{R}(x)
\end{equation}

where $\mathcal{R}$ is a real valued function used to regularize the
optimization process by injecting prior information on the desired
output $x^*$. Most common operators encountered in the literature to
model this information are $\lVert x \rVert_1$, $\lVert x \rVert_2$ and
Total Variation (TV), $\mathrm{TV}(x) = \lVert \nabla x \rVert_1$.

§.§ Conditioning a Score-Based Generative Model (cSGM)

The method developed by Song et al. [15] is
generative. With a dataset of i.i.d. samples $\mathbf{x} \in X$ from an
unknown distribution $p(\mathbf{x})$, it aims at generating new data
samples. To this end, it firstly defines a continuous diffusion process,
that progressively perturbs samples from $p(\mathbf{x})$ into samples
from a tractable prior distribution. It is formulated as a linear SDE as

\begin{equation}
\label{sde}
\mathrm{d}\mathbf{x}_t = f(t) \mathbf{x}_t \mathrm{d}t
+ g(t) \mathrm{d}\mathbf{w}_t
\end{equation}

where $t\in[0,1]$, $\mathrm{d}t$ is an infinitesimal time step, $f$ and
$g$ are real valued functions, $\mathbf{w}_t$ is a standard Wiener
process, and $\mathbf{x}_t$ represents the samples through the
perturbation process. The marginal probability distribution of
$\mathbf{x}_t$, $p_t(\mathbf{x}_t)$ can be further derived from Eq.        (<ref>), as well as the transition distribution from $\mathbf{x}_0$
to $\mathbf{x}_t$, $p_{0t}(\mathbf{x}_t | \mathbf{x}_0)$. $f$ and $g$
are carefully designed so that, for any $p_0(\mathbf{x}) \equiv
p(\mathbf{x})$, Eq. (<ref>) ensures convergence at $t=1$ towards a
distribution $p_1(\mathbf{x})$ close to a Gaussian noise.

As demonstrated by Anderson [42], the
reverse of the diffusion process, in Eq. (<ref>), is also a
diffusion process which can be written as the following reverse-time

\begin{equation}
\label{reverse_sde}
\mathrm{d}\mathbf{x}_t = [
f(t) \mathbf{x}_t
+ g(t)^2 \nabla_{\mathbf{x}_t}\mathrm{log}p_t(\mathbf{x}_t)
] \mathrm{d}t + g(t) \mathrm{d}\mathbf{\overline{w}}_t
\end{equation}

where $\mathbf{\overline{w}}_t$ is a standard Wiener process in the
reverse-time direction, and $\mathrm{d}t$ indicates an infinitesimal
negative time step. Solving Eq. (<ref>) from $t=0$ to $t=1$
corresponds to a continuous denoising, yielding data samples
$\mathbf{x}_0 \sim p_0(\mathbf{x}) \equiv p(\mathbf{x})$ from noise
samples [15]. However, it requires the score
function $\nabla_{\mathbf{x}_t} \mathrm{log}p_t(\mathbf{x}_t)$ of
$p_t(\mathbf{x}_t)$.  Using a time-dependent neural network
$s_\theta(\mathbf{x},t)$ called the score model, it is learned on $X$
with denoising score matching. As described in
[43] and [15], the
score model is trained by approximating the known score functions of the
transition distributions from $\mathbf{x}_0$ to $\mathbf{x}_t$,
$p_{0t}(\mathbf{x}_t | \mathbf{x}_0)$. Optimized parameters $\theta^*$
ensure that $s_{\theta^*}(\mathbf{x},t) \approx \nabla_{\mathbf{x}_t}
\mathrm{log}p_t(\mathbf{x}_t)$ according to denoising score matching
theory. The score model can then be plugged into Eq.        (<ref>), yielding:

\begin{equation}
\label{score_reverse_sde}
\mathrm{d}\mathbf{x}_t = [
f(t) \mathbf{x}_t + g(t)^2 s_{\theta^*}(\mathbf{x},t)
] \mathrm{d}t + g(t) \mathrm{d}\mathbf{\overline{w}}_t
\end{equation}

from here, one can sample from $p_0(\mathbf{x})$ using a
sequence of time steps ${0=t_0<t_1<...<t_N=1}$ with standard iterative
sampling techniques:

\begin{equation}
\label{unconditioned_iteration}
\mathbf{x}_{t_{i-1}} = \boldsymbol{h}(
\mathbf{x}_{t_i},
\mathbf{z}_i, s_{\theta^*}(\mathbf{x}_{t_i},t_i)
\end{equation}

where $\mathbf{z}_i \sim \mathcal{N}(\mathbf{0}, \boldsymbol{I})$ and
$\boldsymbol{h}$ denotes the iterative function related to the chosen
sampling algorithm, such as annealed Langevin dynamics
[10] or Predictor-Corrector samplers

Although it is natural to unconditionally sample from Eq.        (<ref>), conditioning the sampling process to
measurements, i.e. being able to sample from the posterior distribution
$p(\mathbf{x}|\mathbf{y})$, is not trivial. Building on the
unconditional case, Song et al. [16] introduce
perturbed measurements $\mathbf{y}_t$ given the experimental observation
$\mathbf{y}$. They design an iterative sampling algorithm that promotes,
at each time-step $t_i$, consistency of conditioned samples
$\mathbf{x}'_{t_i}$ simultaneously with the perturbed measurements
$\mathbf{y}_{t_i}$ and the unconditioned samples
$\mathbf{x}_{t_i}$ by solving a proximal optimization step:

\begin{equation}
\label{proximal_step}
\mathbf{x}'_{t_i} = \argmin_{w}\lbrace
(1-\lambda) \lVert w - \mathbf{x}_{t_i} \rVert^2_{\mathcal{T}_{\phi_e}}
+ \lambda \min_{u} \lVert w - u \rVert^2_{\mathcal{T}_{\phi_e}}
\rbrace \quad s.t. \quad \mathcal{T}_{\phi_e}(u) = \mathbf{y}_{t_i}
\end{equation}

where $\mathcal{T}_{\phi_e}$ is the Radon transform sampled on $\phi_e$,
the set of experimental viewing angles, and $\lVert.\rVert^2_\mathcal{T}
= \lVert \mathcal{T(.)} \rVert^2_2$. The hyper-parameter $\lambda \in
[0, 1]$ balances consistency regarding experimental measurements
($\lambda \rightarrow 1$) and unconditioned samples ($\lambda
\rightarrow 0$).

Song et al. [16] demonstrated that Eq.        (<ref>) has a tractable closed-formed solution, such that
in practice, conditioned samples are computed through:

\begin{equation}
\label{conditioned_sampling}
\mathbf{x}'_{t_i} = \mathcal{T}_\Phi^\dagger[
P_{\phi_e} (
\lambda \mathbf{y}_{t_i}
+ (1 - \lambda) \mathcal{T}_{\phi_e}(\mathbf{x}_{t_i})
+ P_{\phi_s} \circ \mathcal{T}_{\phi_s}(\mathbf{x}_{t_i})
\end{equation}

where the set $\phi_e$ of experimental viewing angles is completed with
simulated viewing angles $\phi_s$ to constitute an appropriate sampling,
$\Phi = \phi_e \cup \phi_s$, above the Nyquist criterion, so that the
inverse Radon transform $\mathcal{T}_\Phi^\dagger$ is well-defined. $P$
denotes a profile padding operator ensuring dimensionality consistency.

Given conditioned samples $\mathbf{x}'_{t_i}$, at time-step
$t_i$, the same iterative sampling strategy as Eq.        (<ref>) is applied to draw
unconditioned samples $\mathbf{x}_{t_i - 1}$:

\begin{equation}
\label{conditioned_iteration}
\mathbf{x}_{t_{i-1}} = \boldsymbol{h}(
\mathbf{x}'_{t_i},
\mathbf{z}_i, s_{\theta^*}(\mathbf{x}_{t_i},t_i)
\end{equation}

where $s_{\theta^*}(\mathbf{x},t)$ is the score model trained on $X$ in
an unsupervised manner, i.e. without assuming any measurement process.
The reconstruction is ultimately computed by iterating sequentially on
steps described in Eq. (<ref>) and Eq.        (<ref>).

§.§ Deep Image Prior (DIP)

Ulyanov et al. [31] introduced DIP to solve classic
IIPs such as denoising and super-resolution. The core idea of DIP is to
regularize IIPs by taking advantage of the structural bias of a U-Net
[32] $f_\theta$ parameterized with a set of
weights $\theta$. It operates through two mechanisms: reparameterization
and early stopping ($ES$). For instance, in the case of denoising, the
optimization problem is reparametrized as:

\begin{equation}
\label{unconditioned_dip}
x^* = \lbrace
f_{\theta^*}(z) \bigm|
\theta^* = \argmin_{\theta} \lVert
f_{\theta}(z) - (x_0 + \eta)
\rVert^2_2
\rbrace
\end{equation}

where $x_0$ is the initial image perturbed with unknown white noise
$\eta$ and $z$ is a fixed white noise with the same dimensions as $x_0$.
Experiments in [31] showed that given sufficient
capacity and time/iterations, via gradient descent, the randomly
initialized and over-parameterized U-Net can fit the output signal
almost perfectly, including the noise $\eta$. However, Ulyanov et al.        [31] demonstrated that the weights descent sequence
$\theta_1, ..., \theta_N$, with $f_{\theta_N} \approx x_0+ \eta$,
contains an early stopping point $\theta_{ES}$, such that
$f_{\theta_{ES}} \approx x_0$. This phenomenon has been proven to be a
consequence of the structure of the generative network, more
specifically the convolutional and upsampling layers
[33, 34]. They induce a
spectral bias in which the decoder learns to construct the image from
low to high frequencies, meaning that with an appropriate choice of
$ES$, one can prevent the decoder from fitting the high frequency

This methodology has recently been tailored to reconstruct 3D CT volumes
by Baguer et al. [35], using an adaptation of
Eq. (<ref>) further regularized with TV:

\begin{equation}
\label{conditioned_dip}
x^* = \lbrace
f_{\theta^*}(z) \bigm|
\theta^* = \argmin_{\theta} \lVert
\mathcal{T}_{\phi_e} \circ f_{\theta}(z) - y_{\phi_e}
\rVert^2_2 + \alpha \mathrm{TV} \circ f_{\theta}(z)
\rbrace
\end{equation}

where $\mathcal{T}_{\phi_e}$ is the Radon transform sampled on
experimental viewing angles $\phi_e$, $y_{\phi_e}$ are the experimental
measurements and $\alpha$ is an hyper-parameter balancing the
regularization. In this formulation, the issue of finding an appropriate
early stopping for denoising is replaced with the necessity to find an
optimal value for $\alpha$, depending on the ill-posedness of the IIP at

Further experiments in [35] showed improvements
by initializing the decoder weights using the result $x_s$ of a
supervised method, e.g. Learned Primal-Dual reconstruction

\begin{equation}
\label{IDIP}
\theta_0 = \argmin_{\theta} \lVert f_{\theta}(z) - x_s \rVert^2_2
\end{equation}

However, replacing random weights of the decoder with such an
initialization restricts the applicability of the approach to the
supervised use case, i.e. fixed acquisition parameters.

§.§ Generative Latent Optimization (GLO)

Attempting to generate competitive results when compared to samples from
GANs without using an adversarial training protocol, Bojanowski et al.        [36] proposed a novel approach consisting
in mapping one freely learned unit latent vector $z_i$ to each
image $x_i$ of a given dataset $X = \lbrace x_1, ..., x_N \rbrace$,
through a decoder network $f_\theta$ with parameters $\theta$, e.g. a
DCGAN [44]. With gradient descent, it comes
to jointly learn decoder network parameters and latent codes $Z =
\lbrace z_1, ..., z_N \rbrace$, both randomly initialized, with the
objective function:

\begin{equation}
\label{glo_objective}
Z^*, \theta^* = \argmin_{Z, \theta}
\frac{1}{N} \sum_{i=1}^N \lVert f_\theta(z_i) - x_i \rVert^2_2
\quad s.t. \quad \lVert z_i \rVert_2 = 1
\end{equation}

It is mentioned in [36] that, as is
conventionally done for training GANs, vectors sampled from an
n-dimensional normal distribution are close to the surface of an
n-sphere with radius $\sqrt{n}$. Following this idea, during the
optimization process, the latent vectors are constrained to the unit
n-sphere by projection after each update.

Experiments in [36] and
[37], have shown that the GLO approach does not
suffer from mode collapse and significantly outperform GANs when trained
on small datasets. However, like Variational Auto-Encoders (VAE)
[45], the quality of generated samples
deteriorates when the data variability overcomes the network capacity
[46]. We also observe this behaviour in our own

§.§ Conditioning GLO (cGLO)

The method proposed to solve IIPs in this article builds on the
framework described by GLO [36]. Attempting
to invert GLO, i.e. sampling or exploring the manifold entailed by
latent vectors learned through Eq. (<ref>), is not
straightforward [38, 39]. The latent vectors are not uniformly distributed
on the surface of the n-sphere. Furthermore, the decoder generation
quality rapidly worsens when evaluated on vectors outside geodesic
interpolation lines between pairs of latent vectors. In other words, the
manifold on which the trained decoder can produce plausible images can
be described as a complete graph, heterogeneously distributed on the
surface of the unit n-sphere.

Instead of inverting GLO, our method cGLO uses a reparameterization of
the IIP, like DIP [31], to benefit from the
structural bias of the convolutional decoder. However, the objective
function described in Eq. (<ref>), is cast into the
framework of GLO [36], detailed in Eq.        (<ref>) such that the reconstruction is composed of two

Firstly, given availability of previous full-dose CT scan
reconstructions, i.e. a set of slices $X = \lbrace x_1, ..., x_N
\rbrace$, prior knowledge may be enforced by learning an appropriate
initial set of parameters $\theta_0$, for the decoder $f_\theta$, in an
unsupervised manner following Eq. (<ref>) and Eq.        (<ref>):

\begin{equation}
\label{cglo_init}
\theta_0 = \argmin_{Z, \theta}
\frac{1}{N} \sum_{i=1}^N \lVert f_\theta(z_i) - x_i \rVert^2_2
\quad s.t. \quad \lVert z_i \rVert_2 = 1
\end{equation}

Secondly, at examination time, the $K$ slices $\Bar{X} = \lbrace
\Bar{x}_1, ..., \Bar{x}_K \rbrace$ corresponding to experimental
measurements are reconstructed together such that the parameters
$\theta$,  and the latent unit vectors $\Bar{Z} = \lbrace \Bar{z}_1, \,
..., \Bar{z}_K \rbrace$ are jointly learned through:

\begin{equation}
\label{cglo_objective}
\Bar{Z}^*, \theta^* = \argmin_{\Bar{Z}, \theta}
\frac{1}{K} \; \sum_{i=1}^K \lVert
\mathcal{T}_{\phi_e} \circ f_\theta(\Bar{z}_i) - y_{i, \: \phi_e}
\rVert^2_1
\quad s.t. \quad \lVert \Bar{z}_i \rVert_2 = 1
\end{equation}

\begin{equation}
\label{cglo_reconstruction}
\Bar{X}^* = f_{\theta^*}(\Bar{Z}^*)
\end{equation}

where $\mathcal{T}_{\phi_e}$ is the Radon transform sampled on
experimental viewing angles $\phi_e$ and $y_{i, \: \phi_e}$ are the
profiles obtained from slice $\Bar{x}_i$. Because of the shared decoder
between the entire set of slice reconstructions $\Bar{X}^*$, and with
sufficient slices to reconstruct, this strategy avoids the overfitting
issue of DIP. It is expected for the set of parameters $\theta_0$ to be
closer to $\theta^*$ than random weights, thus easing the optimization
of Eq. (<ref>), especially when $\phi_e$ is very sparse.
The latent vectors from Eq. (<ref>) are not reused so that
the set $\Bar{Z}$ of Eq. (<ref>) is always randomly

Consequently, cGLO is an unsupervised reconstruction methodology that is
very flexible as it can be adapted to various practical use cases
regardless of the quantity of data at hand.

§ EXPERIMENTS

§.§ Datasets

Experiments are conducted on two commonly used datasets which are
publicly available on The Cancer Imaging Archive (TCIA) platform
[47]. The Lung Image Database Consortium (LIDC)
[48] image collection was collected using lung
cancer screenings from 1018 patients. It consists of thoracic CT scans
completed with radiologists annotations for nodule segmentation. The
Low-dose CT image and projection dataset (LDCT)
[49] is comprised of 299 CT scans of patients
heads, chests and abdomens and their respective (full) clinical doses.
In addition, the LDCT dataset also includes simulated reduced doses, by
Poisson noise insertion, and the location and diagnosis for positive
findings. In the presented experiments, only CT scan slices from both
datasets are used. Depending on the optimization process, slices are
either directly compared using a pixel-wise metric, or indirectly via
simulated measurements obtained from the application of the same forward
operator, i.e. a full dose parallel-beam geometry with viewing angles
equally distributed across 180 degrees. Similarly to Song et al.        [16], our approach treats every slice independently.
Therefore, though 3D scan geometric parameters such as pixel spacing and
slice thickness may vary accros patients, slices are not resampled, such
that 3D volumes can have different number of slices and spatial scales.
The native resolution of slices is 512x512 pixels for both datasets, but
the LIDC slices are downsized to a resolution of 320x320 pixels. For
experiments to be representative of various realistic situations with
sparse or abundant data, training sub-datasets consisting of portions of
the LIDC and LDCT collections are prepared.  Details of such datasets
are given in table <ref>.  The LIDC and the LDCT test sets
consist of 5 scans respectively totalling 791 and 1089 slices. Unless
stated otherwise, Peak Signal to Noise Ratio (PSNR) and Structural
SIMilarity (SSIM) metrics are estimated on the total test set where each
3D volume is reconstructed independently, meaning that 5 independent
reconstructions are computed.

Portion     # of scans     # of slices

50%     500     104 953

35%     350     65 033

20%     200     35 697

10%     100     17 898

2%     20     2 906

LIDC sub-datasets

Portion     # of scans     # of slices

50%     147     24 464

35%     103     16 924

20%     59     10 463

10%     29     4 989

2%     6     930

LDCT sub-datasets

Sub-datasets used for training with their corresponding number
of 3D scans and slices. 3D scans are always fully included.

§.§ Implementation Details

In the presented experiments, the geometry of the decoder's latent space
is set to the unit n-sphere, similarly to state-of-the art
representation learning approaches [36].
The latent space dimension is set to 320 for experiments on the LIDC
dataset and 512 for the LDCT dataset. Being a reference in the
literature of generative models, the Deep Convolutional Generative
Adversarial Network (DCGAN) [44]
architecture is used for the decoder. As illustrated in figure
<ref>, the latent code firstly goes through a linear block
and is mapped to a feature map of shape
$2\,\mathrm{x}\,2\,\mathrm{x}\,\mathbf{C}$, with $\mathbf{C}$ the
desired number of channels for the first convolutional layer. From layer
to layer the input is upscaled by a factor 2 (last layer upscale factor
may vary to fit the desired output image resolution) and the number of
channels is cut by half. Hence, the image resolution, the latent
dimension and $\mathbf{C}$ completely define the structure of the
decoder. The number of input channels $\mathbf{C}$ is set to 8192 for
both experiments on the LIDC and the LDCT sub-datasets.

Generator DCGAN-like architecture, latent dimension (512) and final
resolution (512x512) correspond to the LDCT sub-datasets experiments.

Optimizations are conducted with the Adam algorithm
[50], with parameters kept at default values. The
learning rates for the latent codes and the decoder weights are both set
to $10^{-3}$ while training on CT reconstruction, i.e. the first step
described in Eq. (<ref>), and respectively set to $10^{-2}$
and $10^{-4}$ when reconstructing from experimental measurements, i.e.        the second step as shown in Eq. (<ref>). Although
learning rates are kept constant, batch sizes are increased along
specified schedules. It leads to faster training and is statistically
equivalent to decreasing learning rates [51]. Finally,
the optimization described in Eq. (<ref>) is further
stabilized by artificially augmenting the number of experimental
measurements, with a linear interpolation along the vertical axis, i.e.        the axis orthogonal to the slices plane. It simulates a larger set of
experimental measurements, which includes true experimental
measurements. Reconstructions are conducted using the entire augmented
set, then reconstructions corresponding to interpolated measurements are
discarded. Practically, measurements are augmented by a factor 8 for
reconstructions conducted on both the LIDC and the LDCT test sets.

Regarding cSGM, hyper-parameters, model architecture and other
implementation details cSGM are set to what is provided by Song et al.        [16] for sparse-view CT experiments.

PSNR (left side) and SSIM (right side) median values curves
corresponding to reconstructions of slices from the LIDC dataset
(upper row) and the LDCT dataset (lower row) test sets given 9, 23,
50 and 100 experimental viewing angles. Each curve is associated
with one of the training sub-datasets presented in table

§.§ Results

In this section, our method cGLO is compared to cSGM
[16], which currently achieves state-of-the-art
performance for unsupervised sparse CT. The two approaches are compared
on quantitative pixel-wise (PSNR) and structural (SSIM) metrics which
are given in figure <ref> and detailed in Table
<ref> for reconstructions computed from 100
experimental viewing angles. Qualitative results consists of examples of
reconstructions from cGLO and cSGM trained on the smallest sub-dataset
consisting of a 2% portion of the LIDC and the LDCT datasets. These
reconstructions from the LIDC and the LDCT test sets are respectively
presented in figure <ref> and figure <ref>.
Additional results and examples, for experimental setups corresponding
to different combinations of training dataset sizes and number of
viewing angles are provided in appendix A and B.

§.§.§ Quantitative performance

As shown in figure <ref>, cGLO outperforms cSGM in most
data and viewing angles regimes. cSGM achieves better median PSNR
values only in scenarios involving more than 100 experimental
viewing angles, when the portion of the employed sub-datasets is
superior to 20%. The details of the performance of both models in
this context are presented in table <ref>. In
every other experimental setups, our method cGLO outperforms cSGM
regarding median PSNR values. Furthermore, cGLO achieves higher
median SSIM metrics values in all experimental setups, even in
abundant data regimes with large training sub-datasets and many
experimental viewing angles. Also, median PSNR and especially median
SSIM curves in figure <ref> indicate that the performance
gap, at the advantage of cGLO over cSGM, increases as the set of
experimental viewing angles gets sparsier and/or the training
dataset gets smaller. In other words, our method is more
parsimonious and more robust for solving increasingly sparse CT when
compared to cSGM.

2c|LIDC 320x320
2cLDCT 512x512
PSNR $\uparrow$     SSIM $\uparrow$
PSNR $\uparrow$     SSIM $\uparrow$

cSGM     2%      39.07 $\pm 0.89$     0.946 $\pm 0.015$
34.51 $\pm 0.53$     0.906 $\pm 0.012$

cGLO     2%      41.48 $\pm 1.69$     0.987 $\pm 0.006$
38.97 $\pm 1.12$     0.976 $\pm 0.007$

cSGM     10%     41.82 $\pm 1.47$     0.965 $\pm 0.015$
39.00 $\pm 1.08$     0.962 $\pm 0.008$

cGLO     10%     41.88 $\pm 1.64$     0.987 $\pm 0.006$
40.38 $\pm 1.45$     0.981 $\pm 0.006$

cSGM     20%     42.06 $\pm 1.61$     0.967 $\pm 0.014$
39.99 $\pm 1.18$     0.969 $\pm 0.008$

cGLO     20%     41.48 $\pm 1.55$     0.986 $\pm 0.006$
40.26 $\pm 1.61$     0.980 $\pm 0.007$

cSGM     35%     42.07 $\pm 1.67$     0.967 $\pm 0.013$
40.27 $\pm 1.18$     0.970 $\pm 0.007$

cGLO     35%     41.12 $\pm 1.44$     0.985 $\pm 0.006$
40.14 $\pm 1.47$     0.980 $\pm 0.007$

cSGM     50%     42.24 $\pm 1.64$     0.968 $\pm 0.013$
39.81 $\pm 1.08$     0.968 $\pm 0.008$

cGLO     50%     40.72 $\pm 1.39$     0.984 $\pm 0.006$
39.55 $\pm 1.15$     0.977 $\pm 0.006$

PSNR and SSIM median $\pm$ half Inter Quartile Range (IQR)
values for reconstructions of slices from the LIDC and the LDCT
test datasets given 100 experimental viewing angles.

§.§.§ Effect of training data

As illustrated in figure <ref>, contrary to cSGM, our
model performance plateaus as the number of experimental viewing
angles increases and even slightly worsens, regarding median PSNR
values, as the training datasets becomes larger. Given a fixed
representation capacity for our decoder architecture, i.e. a fixed
number of input channels $\mathbf{C}$, the model becomes
under-parameterized after a given training dataset size is reached.
Since the model parameters and the latent codes are jointly
optimized in Eq. (<ref>) with respect to a
data-likelihood objective function, the model error distribution is
widely spread over the entire training dataset
[46]. Consequently, when the data variability
overcomes the model capacity, each additional training example
further deteriorates the average prediction quality. Ideally, the
model capacity should be tuned to best fit each experimental setup.
The experiments realized in this paper however demonstrate that even
with a fixed decoder architecture, cGLO produces superior
reconstructions for a wide range of experimental setups.

Examples of reconstructions given 9, 23, 50 and 100 experimental
viewing angles, obtained with cGLO (upper row) and cSGM
(lower row). Both methods are trained on the sub-dataset consisting
of a 2% portion of the LIDC dataset.

Examples of reconstructions given 9, 23, 50 and 100 experimental
viewing angles, obtained with cGLO (upper row) and cSGM
(lower row). Both methods are trained on the sub-dataset consisting
of a 2% portion of the LDCT dataset.

§.§.§ Reconstruction quality

Figure <ref> and figure <ref> compare
reconstructions computed with cGLO and cSGM respectively from the
LIDC and the LDCT test datasets. Reconstruction quality clearly
degrades for both models as the number of experimental viewing
angles diminishes.  The type of degradation, however, differs
between cGLO and cSGM.  While cGLO reconstructions lack sharp
details and get noisier for very sparse CT, cSGM tends to alter the
structural integrity of the slices. In addition, cSGM reproduces as
a texture on top of its reconstructions the high frequency artefacts
resulting from the FBP operator, when those exists in the training
dataset, e.g. in the LDCT dataset. This effect is especially
noticeable in reconstructions shown in figure <ref>
which may be zoomed in for more details. Our model does not show
this behaviour in any experiment. These two observations corroborate
and explain the larger gap in performance between cGLO and cSGM when
comparing median SSIM curves rather than median PSNR curves in
figure <ref>.

The fact that cSGM and cGLO yield reconstruction results at
decreasing viewing angles can be explained by the way each model
employs experimental data. cSGM uses a proximal optimization step,
described in Eq. (<ref>) to mix experimental data
with unconditioned samples. The cost funtion of Eq.            (<ref>), expresses a balance between data likelihood
and prior, which only enforces a loose constraint on the conformity
of the proposed reconstruction to experimental data. When the number
of viewing angles diminishes, this is responsible for the structural
deformations appearing in figure <ref> and figure

On the contrary, the cost function of cGLO, described in equation
(<ref>), enforces a strict constraint on experimental
data. With decreasing viewing angles, the energy landscape of this
cost function becomes less smooth, leading to the presence of local
minima and harder convergence of the optimization process. As seen
in figure <ref>, this leads to degraded
reconstructions with more noise and less details at 9 viewing angles
for example. However, contrary to cSGM, cGLO does not tend to create
hallucinated structures arising from a too strong prior.

§ CONCLUSION

In this work, a novel unsupervised method to solve IIPs named cGLO was
presented, built upon the general framework of GLO
[36]. Contrary to supervised strategies, our
method does not require a fixed experimental setup. cGLO was tested on
sparse-view CT which is a widely studied reconstruction problem arising in
medical imaging and compared to the current state-of-the-art unsupervised
reconstruction approach cSGM [16].  Experiments
conducted in this paper cover a wide range of realistic setups with varying
amount of available training data and experimental viewing angles.
Quantitative results demonstrate that cGLO is a parsimonious and robust
reconstruction method, as the performance advantage over cSGM increases for
smaller training datasets and sparsier experimental viewing angles.
Moreover, reconstruction examples illustrate that cGLO also exhibits
potentially attractive properties like a propensity to preserve the
structural integrity of reconstructions, even for very ill-posed IIPs,
thanks to its straightforward conditioning to experimental measurements.

While the performance of cGLO have been tested on a tomographic
reconstruction problem, the framework developed in ths article can be
readily applied to other ill-posed IIPs, such as MRI reconstruction from a
sparsely sampled k-space, as done with cSGM. Furthermore, since the cGLO
approach only requires the knowledge of the forward operator, it could in
principle be extended to solve non-linear IIPs. One of the main interests of
cGLO is that it does not require any backward operator, such as FBP for
tomographic reconstruction. It is thus a method of choice for IIPs where
such operators do not exist, such as multi-material CT reconstruction, used
for example in security screening. Future work will explore the application
of cGLO to such IIPs. Since cGLO efficiently learns correlations between the
output channels of its decoder network, further developments will also focus
on muti-task applications, such as jointly learning reconstruction and

§ APPENDIX A: QUANTITATIVE RESULTS TABLES

2c|LIDC 320x320
2cLDCT 512x512
PSNR $\uparrow$     SSIM $\uparrow$
PSNR $\uparrow$     SSIM $\uparrow$

cSGM     2%      28.47 $\pm 1.04$     0.784 $\pm 0.027$
24.46 $\pm 1.10$     0.664 $\pm 0.059$

cGLO     2%      31.60 $\pm 1.30$     0.923 $\pm 0.016$
30.53 $\pm 0.98$     0.903 $\pm 0.014$

cSGM     10%     29.25 $\pm 1.00$     0.800 $\pm 0.020$
26.91 $\pm 1.08$     0.744 $\pm 0.041$

cGLO     10%     35.31 $\pm 0.94$     0.957 $\pm 0.008$
31.35 $\pm 0.99$     0.917 $\pm 0.017$

cSGM     20%     29.54 $\pm 0.88$     0.808 $\pm 0.022$
27.60 $\pm 1.15$     0.765 $\pm 0.036$

cGLO     20%     35.82 $\pm 1.50$     0.961 $\pm 0.009$
31.11 $\pm 1.03$     0.915 $\pm 0.015$

cSGM     35%     30.28 $\pm 1.13$     0.821 $\pm 0.031$
27.59 $\pm 1.31$     0.759 $\pm 0.038$

cGLO     35%     35.11 $\pm 0.97$     0.958 $\pm 0.006$
31.19 $\pm 1.08$     0.913 $\pm 0.016$

cSGM     50%     30.19 $\pm 1.16$     0.819 $\pm 0.033$
27.46 $\pm 1.05$     0.757 $\pm 0.035$

cGLO     50%     34.64 $\pm 1.41$     0.954 $\pm 0.009$
31.70 $\pm 1.18$     0.925 $\pm 0.014$

PSNR and SSIM median $\pm$ half Inter Quartile Range (IQR) values
for reconstructions of slices from the LIDC and the LDCT test datasets given
9 experimental viewing angles.

2c|LIDC 320x320
2cLDCT 512x512
PSNR $\uparrow$     SSIM $\uparrow$
PSNR $\uparrow$     SSIM $\uparrow$

cSGM     2%      33.89 $\pm 1.01$     0.879 $\pm 0.019$
29.09 $\pm 0.61$     0.793 $\pm 0.018$

cGLO     2%      37.60 $\pm 1.67$     0.972 $\pm 0.009$
35.68 $\pm 1.03$     0.955 $\pm 0.010$

cSGM     10%     36.31 $\pm 0.79$     0.912 $\pm 0.015$
33.35 $\pm 1.02$     0.887 $\pm 0.017$

cGLO     10%     39.82 $\pm 1.26$     0.981 $\pm 0.006$
37.34 $\pm 1.40$     0.967 $\pm 0.010$

cSGM     20%     36.87 $\pm 0.82$     0.917 $\pm 0.015$
34.62 $\pm 0.96$     0.904 $\pm 0.015$

cGLO     20%     39.90 $\pm 1.26$     0.982 $\pm 0.005$
37.03 $\pm 1.33$     0.965 $\pm 0.010$

cSGM     35%     36.62 $\pm 1.61$     0.916 $\pm 0.024$
34.90 $\pm 1.08$     0.907 $\pm 0.015$

cGLO     35%     39.85 $\pm 1.19$     0.982 $\pm 0.005$
36.61 $\pm 1.45$     0.962 $\pm 0.010$

cSGM     50%     36.97 $\pm 1.33$     0.920 $\pm 0.023$
34.35 $\pm 0.95$     0.899 $\pm 0.015$

cGLO     50%     39.52 $\pm 1.17$     0.980 $\pm 0.005$
37.18 $\pm 1.18$     0.966 $\pm 0.008$

PSNR and SSIM median $\pm$ half Inter Quartile Range (IQR) values
for reconstructions of slices from the LIDC and the LDCT test datasets given
23 experimental viewing angles.

2c|LIDC 320x320
2cLDCT 512x512
PSNR $\uparrow$     SSIM $\uparrow$
PSNR $\uparrow$     SSIM $\uparrow$

cSGM     2%      36.90 $\pm 0.83$     0.922 $\pm 0.016$
32.46 $\pm 0.56$     0.867 $\pm 0.015$

cGLO     2%      40.50 $\pm 1.56$     0.983 $\pm 0.007$
38.24 $\pm 1.05$     0.972 $\pm 0.007$

cSGM     10%     40.03 $\pm 1.05$     0.950 $\pm 0.016$
36.78 $\pm 0.96$     0.937 $\pm 0.013$

cGLO     10%     41.45 $\pm 1.53$     0.985 $\pm 0.006$
39.73 $\pm 1.31$     0.979 $\pm 0.007$

cSGM     20%     40.41 $\pm 1.19$     0.953 $\pm 0.016$
37.96 $\pm 1.03$     0.949 $\pm 0.011$

cGLO     20%     41.22 $\pm 1.47$     0.985 $\pm 0.006$
39.53 $\pm 1.38$     0.978 $\pm 0.008$

cSGM     35%     40.04 $\pm 1.45$     0.950 $\pm 0.019$
38.37 $\pm 1.05$     0.953 $\pm 0.010$

cGLO     35%     40.89 $\pm 1.38$     0.984 $\pm 0.006$
39.48 $\pm 1.43$     0.978 $\pm 0.008$

cSGM     50%     40.29 $\pm 1.50$     0.955 $\pm 0.020$
37.71 $\pm 1.03$     0.946 $\pm 0.011$

cGLO     50%     40.53 $\pm 1.35$     0.984 $\pm 0.006$
39.08 $\pm 1.12$     0.975 $\pm 0.006$

PSNR and SSIM median $\pm$ half Inter Quartile Range (IQR) values
for reconstructions of slices from the LIDC and the LDCT test datasets given
50 experimental viewing angles.

§ APPENDIX B: RECONSTRUCTION EXAMPLES

Examples of reconstructions of slices from the LIDC test set given : 9,
23, 50 and 100 experimental viewing angles. They are obtained with cGLO
trained on the following portions of the LIDC dataset : 2%, 10%, 20%,
35% and 50%.

Examples of reconstructions of slices from the LIDC test set given : 9,
23, 50 and 100 experimental viewing angles. They are obtained with cSGM
trained on the following portions of the LIDC dataset : 2%, 10%, 20%,
35% and 50%.

Examples of reconstructions of slices from the LDCT test set given : 9,
23, 50 and 100 experimental viewing angles. They are obtained with cGLO
trained on the following portions of the LDCT dataset : 2%, 10%, 20%,
35% and 50%.

Examples of reconstructions of slices from the LDCT test set given : 9,
23, 50 and 100 experimental viewing angles. They are obtained with cSGM
trained on the following portions of the LDCT dataset : 2%, 10%, 20%,
35% and 50%.

[1]
Stanley R. Deans.
The Radon transform and some of its applications.
Wiley, 1983.

[2]
Jonas Adler and Ozan Öktem.
Deep Bayesian Inversion.
arXiv:1811.05910 [stat.ML].

[3]
Jonas Adler and Ozan Öktem.
Learned Primal-Dual Reconstruction.
IEEE Transactions on Medical Imaging, 37(6):1322–1332,
doi: 10.1109/TMI.2018.2799231.

[4]
Hoyeon Lee, Jongha Lee, Hyeongseok Kim, Byungchul Cho, and Seungryong Cho.
Deep-Neural-Network-Based Sinogram Synthesis for
Sparse-View CT Image Reconstruction.
IEEE Transactions on Radiation and Plasma Medical
Sciences, 3(2):109–119, 2019.
doi: 10.1109/TRPMS.2018.2867611.

[5]
Bo Zhu, Jeremiah Z. Liu, Stephen F. Cauley, Bruce R. Rosen, and Matthew S.
Image reconstruction by domain-transform manifold learning.
Nature, 555(7697):487–492, 2018.
doi: 10.1038/nature25988.

[6]
Francesco Tonolini, Jack Radford, Alex Turpin, Daniele Faccio, and Roderick
Variational inference for computational imaging inverse problems.
Journal of Machine Learning Research, 21(1):7285–7330,
arXiv:1904.06264 [cs.LG].

[7]
Jonas Adler and Ozan Öktem.
Solving ill-posed inverse problems using iterative deep neural
Inverse Problems, 33(12):124007, 2017.
doi: 10.1088/1361-6420/aa9581.

[8]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley,
Sherjil Ozair, Aaron Courville, and Yoshua Bengio.
Generative adversarial networks.
In Advances in Neural Information Processing Systems,
arXiv:1406.2661 [stat.ML].

[9]
Weihao Xia, Yulun Zhang, Yujiu Yang, Jing-Hao Xue, Bolei Zhou, and Ming-Hsuan
GAN Inversion: A Survey.
IEEE Transactions on Pattern Analysis and Machine
Intelligence, 45(3):3121–3138, 2023.
doi: 10.1109/TPAMI.2022.3181070.

[10]
Yang Song and Stefano Ermon.
Generative Modeling by Estimating Gradients of the Data
In Advances in Neural Information Processing Systems,
arXiv:1907.05600 [cs.LG].

[11]
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising Diffusion Probabilistic Models.
In Advances in Neural Information Processing Systems,
arXiv:2006.11239 [cs.LG].

[12]
Alexander Quinn Nichol and Prafulla Dhariwal.
Improved Denoising Diffusion Probabilistic Models.
In International Conference on Machine Learning, 2021.
arXiv:2102.09672 [cs.LG].

[13]
Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and
Luc Van Gool.
RePaint: Inpainting Using Denoising Diffusion
Probabilistic Models.
In Computer Vision and Pattern Recognition, 2022.
doi: 10.1109/CVPR52688.2022.01117.

[14]
Yang Song and Stefano Ermon.
Improved Techniques for Training Score-Based Generative
In Advances in Neural Information Processing Systems,
arXiv:2006.09011 [cs.LG].

[15]
Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano
Ermon, and Ben Poole.
Score-Based Generative Modeling through Stochastic
Differential Equations.
In International Conference on Learning
Representations, 2021.
arXiv:2011.13456 [cs.LG].

[16]
Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon.
Solving Inverse Problems in Medical Imaging with
Score-Based Generative Models.
In International Conference on Learning
Representations, 2022.
arXiv:2111.08005 [eess.IV].

[17]
Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav
Shapiro, and Daniel Cohen-Or.
Encoding in Style: A StyleGAN Encoder for Image-to-Image
In Computer Vision and Pattern Recognition, 2021.
doi: 10.1109/CVPR46437.2021.00232.

[18]
Guim Perarnau, Joost van de Weijer, Bogdan Raducanu, and Jose M. Àlvarez.
Invertible Conditional GANs for image editing.
arXiv:1611.06355 [cs.CV].

[19]
Rameen Abdal, Yipeng Qin, and Peter Wonka.
Image2StyleGAN: How to Embed Images Into the StyleGAN
Latent Space?
In International Conference on Computer Vision, 2019.
arXiv:1904.03189 [cs.CV].

[20]
Minyoung Huh, Richard Zhang, Jun-Yan Zhu, Sylvain Paris, and Aaron Hertzmann.
Transforming and Projecting Images into Class-Conditional
Generative Networks.
In European Conference on Computer Vision, 2020.
doi: 10.1007/978-3-030-58536-5_2.

[21]
Jun-Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A. Efros.
Generative Visual Manipulation on the Natural Image
In European Conference on Computer Vision, 2016.
doi: 10.1007/978-3-319-46454-1_36.

[22]
Jiapeng Zhu, Yujun Shen, Deli Zhao, and Bolei Zhou.
In-Domain GAN Inversion for Real Image Editing.
In European Conference on Computer Vision, 2020.
doi: 10.1007/978-3-030-58520-4_35.

[23]
David Bau, Jun-Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei
Zhou, and Antonio Torralba.
Seeing What a GAN Cannot Generate.
In International Conference on Computer Vision, 2019.
arXiv:1910.11626 [cs.CV].

[24]
Jeff Donahue, Philipp Krähenbühl, and Trevor Darrell.
Adversarial Feature Learning.
In International Conference on Learning
Representations, 2017.
arXiv:1605.09782 [cs.LG].

[25]
Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb,
Martin Arjovsky, and Aaron Courville.
Adversarially Learned Inference.
In International Conference on Learning
Representations, 2017.
arXiv:1606.00704 [stat.ML].

[26]
Jiapeng Zhu, Deli Zhao, Bo Zhang, and Bolei Zhou.
Disentangled Inference for GANs With Latently Invertible
International Journal of Computer Vision,
130(5):1259–1276, 2022.
doi: 10.1007/s11263-022-01598-5.

[27]
Giannis Daras, Joseph Dean, Ajil Jalal, and Alexandros G. Dimakis.
Intermediate Layer Optimization for Inverse Problems using
Deep Generative Models.
arXiv:2102.07364 [cs.LG].

[28]
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky.
It Takes (Only) Two: Adversarial Generator-Encoder
In AAAI Conference on Artificial Intelligence, 2018.
doi: 10.1609/aaai.v32i1.11449.

[29]
Varun A. Kelkar and Mark Anastasio.
Prior Image-Constrained Reconstruction using Style-Based
Generative Models.
In International Conference on Machine Learning, 2021.
arXiv:2102.12525 [eess.IV].

[30]
Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin.
PULSE: Self-Supervised Photo Upsampling via Latent
Space Exploration of Generative Models.
In Computer Vision and Pattern Recognition, 2020.
doi: 10.1109/CVPR42600.2020.00251.

[31]
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky.
Deep Image Prior.
In Computer Vision and Pattern Recognition, 2018.
arXiv:1711.10925 [cs.CV].

[32]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox.
U-Net: Convolutional Networks for Biomedical Image
In Medical Image Computing and Computer-Assisted
Intervention, 2015.
doi: 10.1007/978-3-319-24574-4_28.

[33]
Reinhard Heckel and Mahdi Soltanolkotabi.
Denoising and Regularization via Exploiting the Structural
Bias of Convolutional Generators.
arXiv:1910.14634 [cs.LG].

[34]
Prithvijit Chakrabarty and Subhransu Maji.
The Spectral Bias of the Deep Image Prior.
arXiv:1912.08905 [cs.CV].

[35]
Daniel Otero Baguer, Johannes Leuschner, and Maximilian Schmidt.
Computed tomography reconstruction using deep image prior and learned
reconstruction methods.
Inverse Problems, 36(9):094004, 2020.
doi: 10.1088/1361-6420/aba415.

[36]
Piotr Bojanowski, Armand Joulin, David Lopez-Paz, and Arthur Szlam.
Optimizing the Latent Space of Generative Networks.
In International Conference on Machine Learning, 2018.
arXiv:1707.05776 [stat.ML].

[37]
Idan Azuri and Daphna Weinshall.
Generative Latent Implicit Conditional Optimization when
Learning from Small Sample.
In International Conference on Pattern Recognition,
doi: 10.1109/ICPR48806.2021.9413259.

[38]
Yedid Hoshen, Ke Li, and Jitendra Malik.
Non-Adversarial Image Synthesis With Generative Latent
Nearest Neighbors.
In Computer Vision and Pattern Recognition, 2019.
doi: 10.1109/CVPR.2019.00596.

[39]
Bi Xiao and Xiaokang Yang.
Optimizing Generative Adversarial Networks in Latent Space.
In International Conference on Intelligent Computing,
Automation and Systems, 2019.
doi: 10.1109/ICICAS48597.2019.00020.

[40]
Frank Natterer and Frank Wübbeling.
Mathematical Methods in Image Reconstruction.
Society for Industrial and Applied Mathematics, 2001.
doi: 10.1137/1.9780898718324.

[41]
Amir Beck and Marc Teboulle.
A Fast Iterative Shrinkage-Thresholding Algorithm for
Linear Inverse Problems.
SIAM Journal on Imaging Sciences, 2(1):183–202, 2009.
doi: 10.1137/080716542.

[42]
Brian D. O. Anderson.
Reverse-time diffusion equation models.
Stochastic Processes and their Applications,
12(3):313–326, 1982.
doi: 10.1016/0304-4149(82)90051-5.

[43]
Pascal Vincent.
A Connection Between Score Matching and Denoising
Neural Computation, 23(7):1661–1674, 2011.
doi: 10.1162/NECO_a_00142.

[44]
Alec Radford, Luke Metz, and Soumith Chintala.
Unsupervised Representation Learning with Deep Convolutional
Generative Adversarial Networks.
In International Conference on Learning
Representations, 2016.
arXiv:1511.06434 [cs.LG].

[45]
Diederik P. Kingma and Max Welling.
Auto-Encoding Variational Bayes.
In International Conference on Learning
Representations, 2014.
arXiv:1312.6114 [stat.ML].

[46]
Lucas Theis, Aäron van den Oord, and Matthias Bethge.
A note on the evaluation of generative models.
In International Conference on Learning
Representations, 2016.
arXiv:1511.01844 [stat.ML].

[47]
Kenneth Clark, Bruce Vendt, Kirk Smith, John Freymann, Justin Kirby, Paul
Koppel, Stephen Moore, Stanley Phillips, David Maffitt, Michael Pringle,
Lawrence Tarbox, and Fred Prior.
The Cancer Imaging Archive (TCIA): Maintaining and
Operating a Public Information Repository.
Journal of Digital Imaging, 26(6):1045–1057, 2013.
doi: 10.1007/s10278-013-9622-7.

[48]
Samuel G. Armato III.
The Lung Image Database Consortium (LIDC) and Image
Database Resource Initiative (IDRI): A Completed Reference
Database of Lung Nodules on CT Scans.
Medical Physics, 38(2):915–931, 2011.
doi: 10.1118/1.3528204.

[49]
Taylor R. Moen, Baiyu Chen, David R. Holmes III, Xinhui Duan, Zhicong Yu,
Lifeng Yu, Shuai Leng, Joel G. Fletcher, and Cynthia H. McCollough.
Low-dose CT image and projection dataset.
Medical Physics, 48(2):902–911, 2021.
doi: 10.1002/mp.14594.

[50]
Diederik P. Kingma and Jimmy Ba.
Adam: A Method for Stochastic Optimization.
In International Conference on Learning
Representations, 2015.
arXiv:1412.6980 [cs.LG].

[51]
Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, and Quoc V. Le.
Don't Decay the Learning Rate, Increase the Batch Size.
In International Conference on Learning
Representations, 2018.
arXiv:1711.00489 [cs.LG].