# Unifying GANs and Score-Based Diffusion as Generative Particle Models

Jean-Yves Franceschi
Criteo AI Lab, Paris, France
<EMAIL_ADDRESS>&Mike Gartrell
Criteo AI Lab, Paris, France
<EMAIL_ADDRESS>&Ludovic Dos Santos
Criteo AI Lab, Paris, France
<EMAIL_ADDRESS>&Thibaut Issenhuth11footnotemark: 1
Criteo AI Lab, Paris, France
LIGM, Ecole des Ponts, Univ Gustave Eiffel,
CNRS, Marne-la-Vallée, France
<EMAIL_ADDRESS>&Emmanuel de Bézenac11footnotemark: 1
Seminar for Applied Mathematics,
D-MATH, ETH Zürich, Rämistrasse 101,
Zürich-8092, Switzerland
<EMAIL_ADDRESS>&Mickaël Chen11footnotemark: 1
Valeo.ai, Paris, France
<EMAIL_ADDRESS>&Alain Rakotomamonjy11footnotemark: 1
Criteo AI Lab, Paris, France
<EMAIL_ADDRESS>Authors listed in a randomly chosen order.

###### Abstract

Particle-based deep generative models, such as gradient flows and score-based
diffusion models, have recently gained traction thanks to their striking
performance. Their principle of displacing particle distributions by
differential equations is conventionally seen as opposed to the previously
widespread generative adversarial networks (GANs), which involve training a
pushforward generator network. In this paper, we challenge this interpretation
and propose a novel framework that unifies particle and adversarial generative
models by framing generator training as a generalization of particle models.
This suggests that a generator is an optional addition to any such generative
model. Consequently, integrating a generator into a score-based diffusion
model and training a GAN without a generator naturally emerge from our
framework. We empirically test the viability of these original models as
proofs of concepts of potential applications of our framework.

##  1 Introduction

Score-based diffusion models (Song et al., 2021) have recently received
significant attention within the machine learning community, due to their
striking performance on generative tasks (Rombach et al., 2022; Ho et al.,
2022). Similarly to gradient flows, these models involve systems of particles,
where the displacement of the particle distribution is described by a
differential equation parameterized by a gradient vector field. Such particle-
based deep generative models are typically seen as opposed to generative
adversarial networks (GANs, Goodfellow et al., 2014), as the latter involves
adversarial training of a generator network (Dhariwal & Nichol, 2021; Song,
2021; Xiao et al., 2022).

In this paper, we challenge the conventional view that particle and
adversarial generative models are opposed to each other. We make the following
contributions.

A unified framework. We present a novel framework that unifies both classes of
models, showing that they are based on similar particle evolution equations.
Particle models follow a gradient vector field during inference; in a similar
fashion, the generator’s outputs can be seen as following the same gradient
field during training, up to a specific smoothing due to the generator.
Building upon prior scattered literature, we propose a new framework that
encompasses a variety of methods, which are listed in Table 1 together with
their respective flow type.

Decoupling generators and flows. By uncovering the role of the generator as a
smoothing operator, we suggest that the existence of a generator and the flow
that particles follow can be decoupled. We deduce that it is possible to train
a generator with score-based gradients which replace adversarial training
(which we call a Score GAN); and that a GAN can be trained without a
generator, using only the discriminator to synthesize samples (which we call a
Discriminator Flow); cf. Table 1. We introduce these new models as proofs of
concept, which we empirically assess to support the validity of our framework
and illustrate the new perspectives it opens up in this active field of
research.

Throughout the paper, we call out some specifics of the contributions of our
framework as Definitions and Findings (italicized in the main text).

Outline. We begin with a discussion of particle models without generators in
Section 2, covering Wasserstein gradient flows and score-based diffusion
models. In Section 3, we discuss how generator training can be framed as a
particle model with a generator, including GANs and Stein gradient flows.
Section 4 then highlights how our framework allows us to decouple the
generator and flow components of particle models, leading to the
aforementioned new models, Score GANs and Discriminator Flows. Finally, we
discuss the implications of our findings and conclude the paper in Section 5.

Notations. We consider the evolution of generated distributions $\rho_{t}$
over $\mathbb{R}^{D}$ w.r.t. time $t\in\mathbb{R}_{+}$, with $t$ being
respectively inference time for particle models without generators or training
time for generator training. This evolution until some finite or infinite end
time $T\in\mathbb{R}_{+}\cup\left\\{+\infty\right\\}$ then yields a final
generated distribution $p_{\mathrm{g}}=\rho_{T}$, which ideally approaches the
data distribution $p_{\rm{data}}$.

Table 1:  Taxonomy of particle models, including our proposed hybrid models: Score GANs and Discriminator Flows.  Model | Generator | Flow type ${\nabla}h_{\rho_{t}}$
---|---|---
Wasserstein gradient flows | ✗ | Wasserstein gradient $-{\nabla}_{W}{\mathcal{F}}\left(\rho_{t}\right)$
Stein gradient flows | ✓
Score-based diffusion models | ✗ | $\alpha_{t}{\nabla}\log\left[p_{\rm{data}}\star k_{\mathrm{RBF}}^{\sigma\left(t\right)}\right]-\beta_{t}{\nabla}\log\rho_{t}$
Score GANs | ✓
Discriminator Flows | ✗ |  $-{\nabla}\left(c\circ f_{\rho_{t}}\right)$ where $f_{\rho_{t}}$ is a discriminator between $\rho_{t}$ and $p_{\rm{data}}$
GANs | ✓

##  2 Particle Models without Generators: Non-interacting Particles

In this section, we formally introduce the notion of generative particle
models that do not use a generator, and present two standard instances:
Wasserstein gradient flows and score-based diffusion models. They involve the
manipulation of particles $x_{t}\sim\rho_{t}$ following a differential
equation parameterized by some vector field. We characterize these models by
noticing that their particles actually optimize, independently from each
other, a loss that depends on the current particle distribution.

######  Definition 1 (Particle Models, PMs).

PMs model particles $x_{t}\sim\rho_{t}$ starting from a prior $\rho_{0}=\pi$:

$\displaystyle x_{0}\sim\pi=\rho_{0},$
$\displaystyle\operatorname{d\\!}x_{t}={\nabla}h_{\rho_{t}}\left(x_{t}\right)\operatorname{d\\!}t,$
(1)

where $h_{\rho_{t}}\colon\mathbb{R}^{D}\to\mathbb{R}$ is a functional that
depends on the current particle distribution $\rho_{t}$. Time $t$ corresponds
to generation / inference time from $\rho_{0}$ to the final distribution
$p_{\mathrm{g}}=\rho_{T}$.

######  Finding 1.

In PMs, the evolution of Equation 1 makes each generated particle $x_{t}$
individually follow a gradient ascent path on the objective
$h_{\rho_{t}}\left(x_{t}\right)$.

In prior work, the prior $\pi$ is chosen to be easy to sample from, such as a
Gaussian. $h_{\rho_{t}}$ is usually defined in a theoretical manner so that
the flow of Equation 1 conveys good convergence properties for $\rho_{t}$
towards $p_{\rm{data}}$ when $t\to T$. Because analytically computing
$h_{\rho_{t}}$ is often intractable, it is empirically estimated and replaced
by a neural network in practice.

###  2.1 Wasserstein Gradient Flows

A Wasserstein gradient flow is an absolute continuous curve of probability
distributions in a Wasserstein metric space ${\mathcal{P}}_{2}$ over
$\mathbb{R}^{D}$ that satisfies a continuity equation (Santambrogio, 2017),
and equivalently an evolution of particles $x_{t}\sim\rho_{t}$ under mild
hypotheses (Jordan et al., 1998):

$\displaystyle\partial_{t}\rho_{t}-{\nabla}\cdot\left(\rho_{t}{\nabla}_{W}{\mathcal{F}}\left(\rho_{t}\right)\right)=0,$
$\displaystyle\operatorname{d\\!}x_{t}=-{\nabla}_{W}{\mathcal{F}}\left(\rho_{t}\right)\left(x_{t}\right)\operatorname{d\\!}t,$
(2)

where $\mathcal{F}\colon{\mathcal{P}}_{2}\to\mathbb{R}$ is the functional to
minimize in the Wasserstein space. This definition involves the Wasserstein
gradient of the functional ${\mathcal{F}}\left(\rho_{t}\right)$, which for
some functionals can be obtained in closed form by computing the first
variation of the functional ${\mathcal{F}}$ (Santambrogio, 2017, Section 4.3):

${\nabla}_{W}{\mathcal{F}}\left(\rho_{t}\right)={\nabla}\mathinner{\dfrac{\partial{}{\mathcal{F}}\left(\rho_{t}\right)}{\partial{\rho_{t}}}}\colon\mathbb{R}^{D}\to\mathbb{R}^{D}.$
(3)

######  Finding 2.

Wasserstein gradient flows are PMs with
$h_{\rho_{t}}=-\frac{\partial{\mathcal{F}}\left(\rho_{t}\right)}{\partial\rho_{t}}+\mathrm{cst}$
and $T=+\infty$.

Table 2:  Gradient flows for standard objectives ${\mathcal{F}}$.  Objective ${\mathcal{F}}\left(\rho\right)$ | $h_{\rho}$
---|---
Forward KL | $\textstyle\mathbb{E}_{\rho}\log\nicefrac{{\rho}}{{p_{\rm{data}}}}$ | $\textstyle-\log\nicefrac{{\rho}}{{p_{\rm{data}}}}$
$f$-divergence | $\textstyle\mathbb{E}_{p_{\rm{data}}}f\left(\nicefrac{{\rho}}{{p_{\rm{data}}}}\right)$ | $\textstyle-f^{\prime}\left(\nicefrac{{\rho}}{{p_{\rm{data}}}}\right)$
Squared MMD w.r.t. kernel $k$ | $\textstyle\textstyle{\begin{subarray}{c}\displaystyle\mathbb{E}\\\ x,x^{\prime}\sim\rho\\\ y,y^{\prime}\sim p_{\rm{data}}\end{subarray}}\left[\begin{subarray}{c}k\left(x,x^{\prime}\right)\\\ +k\left(y,y^{\prime}\right)\\\ -2k\left(x,y\right)\end{subarray}\right]$ | $\textstyle\begin{subarray}{c}\mathbb{E}_{y\sim p_{\rm{data}}}\left[k\left(y,\cdot\right)\right]\\\ -\mathbb{E}_{x\sim\rho}\left[k\left(x,\cdot\right)\right]\end{subarray}$
Entropy | $\textstyle\mathbb{E}_{\rho}\log\rho$ | $\textstyle-\log\rho$

The partial differential equation governing the particle evolution, as well as
its convergence properties toward $p_{\rm{data}}$, strongly depends on the
functional ${\mathcal{F}}$. We detail the standard examples of the forward
Kullback-Leibler (KL) divergence (Kullback & Leibler, 1951), $f$-divergences
(Rényi, 1961), the squared Maximum Mean Discrepancy (MMD, Gretton et al.,
2012; Arbel et al., 2019), and entropy regularization in Table 2. They can be
additively combined for a variety of objectives ${\mathcal{F}}$. More examples
exist in the literature (Liutkus et al., 2019; Mroueh et al., 2019; Glaser et
al., 2021).

Several methods have been explored in the literature to solve Equation 2 in
practice, either using input-convex neural networks (Mokrov et al., 2021;
Alvarez-Melis et al., 2022) to discretize the continuous flow, or
parameterizing ${\nabla}h_{\rho}$ by a neural network (Gao et al., 2019; Fan
et al., 2022; Heng et al., 2023). In all cases, these methods fit with the
class of PMs as framed in Definition 1.

###  2.2 Score-Based Diffusion Models

Early score-based models (Noise Conditional Score Networks [NCSN], Song &
Ermon, 2019) rely on Langevin dynamics as described in the following
stochastic differential equation, converging towards $p_{\rm{data}}$ when
$t\to\infty$:

$\operatorname{d\\!}x_{t}={\nabla}\log
p_{\rm{data}}\left(x_{t}\right)\operatorname{d\\!}t+\sqrt{2}\operatorname{d\\!}W_{t}.$
(4)

Several methods that use neural networks to estimate the score function of the
data distribution ${\nabla}\log p_{\rm{data}}$ (Hyvärinen, 2005), coupled with
the use of Langevin dynamics, can work in practice even for high-dimensional
distributions. Nonetheless, because of ill-definition and estimation issues
for the score for discrete data on manifolds, a Gaussian perturbation of the
data distribution is introduced to stabilize the dynamics. Thus,
$p_{\rm{data}}$ in Equation 4 is replaced by the distribution
$p_{\rm{data}}^{\sigma}$ of $x+\sigma\varepsilon$, where $x\sim p_{\rm{data}}$
and $\varepsilon\sim{\mathcal{N}}\left(0,I_{D}\right)$. Denoting $\star$ as
the convolution of a probability distribution $p$ by a kernel
$k\colon\mathbb{R}^{D}\times\mathbb{R}^{D}\to\mathbb{R}$, we notice that
$p_{\rm{data}}^{\sigma}$ is actually the convolution of $p_{\rm{data}}$ by a
Gaussian kernel:

$\displaystyle p_{\rm{data}}^{\sigma}=p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma},$ $\displaystyle p\star
k\triangleq\int_{x}k\left(x,\cdot\right)\operatorname{d\\!}p\left(x\right),$
$\displaystyle
k_{\mathrm{RBF}}^{\sigma}\left(x,y\right)\triangleq\frac{1}{\sigma\sqrt{2\pi}}\mathrm{e}^{-\frac{\left\|x-y\right\|^{2}_{2}}{2\sigma^{2}}}.$
(5)

NCSN then follows this equation, estimating the score with denoising score
matching (Vincent, 2011) and repeating the process for a decreasing sequence
of $\sigma$s:

$\operatorname{d\\!}x_{t}={\nabla}\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}\right]\left(x_{t}\right)\operatorname{d\\!}t+\sqrt{2}\operatorname{d\\!}W_{t}.$
(6)

Building on Song et al. (2021), newer score models (Karras et al., 2022) make
$\sigma$ a continuous function of time $\sigma(t)$, decreasing towards $0$ in
finite time to improve convergence. Many of these approaches share the
following generation equation, corresponding to the reverse of a noising
process for $p_{\rm{data}}$ (Elucidating the Design Space of Diffusion-Based
Generative Models [EDM], Karras et al., 2022):

$\operatorname{d\\!}x_{t}=2\sigma^{\prime}\left(t\right)\sigma\left(t\right){\nabla}\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma\left(t\right)}\right]\left(x_{t}\right)\operatorname{d\\!}t+\sqrt{2\sigma^{\prime}\left(t\right)\sigma\left(t\right)}\operatorname{d\\!}W_{t},$
(7)

where $\sigma^{\prime}$ is the derivative of $\sigma(t)$. These approaches
admit an equivalent deterministic flow yielding the same probability path
$\rho_{t}$:

$\operatorname{d\\!}x_{t}=\sigma^{\prime}\left(t\right)\sigma\left(t\right){\nabla}\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma\left(t\right)}\right]\left(x_{t}\right)\operatorname{d\\!}t.$
(8)

However, we notice that this only holds under the implicit assumption that
Equation 7 perfectly reverses the initial noising process, i.e.,
$\rho_{t}=p_{\rm{data}}\star k_{\mathrm{RBF}}^{\sigma\left(t\right)}$. This
assumption can be broken in practice when the score is not well estimated or
for coarse time discretization. In the general case for both NCSN and EDM, by
going through the Fokker-Planck equation (Jordan et al., 1998), which allows
us to substitute the stochastic component $\operatorname{d\\!}W_{t}$ by the
negative of the score of the generated distribution, we obtain the following
equivalent exact probability flows:

$\displaystyle\mathinner{\dfrac{\operatorname{d\\!}{}x_{t}}{\operatorname{d\\!}{t}}}={\nabla}\log\left[\frac{p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}}{\rho_{t}}\right]\left(x_{t}\right),$
$\displaystyle\mathinner{\dfrac{\operatorname{d\\!}{}x_{t}}{\operatorname{d\\!}{t}}}=\sigma^{\prime}\left(t\right)\sigma\left(t\right){\nabla}\log\left[\frac{1}{\rho_{t}}\left(p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma\left(t\right)}\right)^{2}\right]\left(x_{t}\right).$
(9)

######  Finding 3.

Score-based diffusion models are PMs:

  * •

NCSN (Song & Ermon, 2019) with $h_{\rho_{t}}=\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}\right]-\log\rho_{t}$ and $T=+\infty$;

  * •

EDM (Karras et al., 2022) with
$h_{\rho_{t}}=\sigma^{\prime}\left(t\right)\sigma\left(t\right)\left(2\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma\left(t\right)}\right]-\log\rho_{t}\right)$ and
$T<+\infty$.

NCSN actually implements through Langevin dynamics a forward KL gradient flow,
which is common knowledge in the related literature (Jordan et al., 1998; Yi
et al., 2023).

##  3 Particle Models with Generators: Training of Interacting Particles

In the previous section we presented a framework for particle models (PMs)
that, in the absence of a generator, individually manipulate particles in the
data space, and optimize a distribution-dependent objective $h_{\rho_{t}}$ via
a differential equation. In this section we frame generator training as a
generalization of PMs involving direct interaction between particles. We show,
supported by prior literature, that our framework applies to the case of GANs
and Stein gradient flows.

###  3.1 Generator Training as a Modified Particle Model

We begin with the training of a neural generator
$g_{\theta}\colon\mathbb{R}^{d}\to\mathbb{R}^{D}$ parameterized by $\theta$.
Associated with a prior distribution on its latent space $p_{z}$, $g_{\theta}$
produces a generated distribution $p_{\theta}$ as the pushforward of $p_{z}$
through $g_{\theta}$: $p_{\theta}=g_{\theta}\sharp p_{z}$, by which we seek to
imitate $p_{\rm{data}}$. Unlike PMs which progressively construct synthesized
samples directly in the data space $\mathbb{R}^{D}$, generators enable models
like GANs to generate samples starting from a different latent space. When
$d<D$, this latent space allows the resulting distribution to be naturally
embedded into a lower-dimensional manifold, thereby integrating the manifold
hypothesis (Bengio et al., 2013). Parameters $\theta$ evolve during training,
making the generated distribution move accordingly: $\rho_{t}=p_{\theta_{t}}$.

We characterized in Section 2, Finding 1, PMs as models that make free
generated particles optimize an objective
$h_{\rho}\colon\mathbb{R}^{D}\to\mathbb{R}$ that convey desirable convergence
properties. We leverage this observation to show that generator training can
be framed as a PM as well. We see that generators involve generated particles
$x_{t}\sim\rho_{t}$ as generator outputs $x_{t}=g_{\theta_{t}}\left(z\right)$,
with $z\sim p_{z}$, which move during training. We proceed by making the
generator optimize the same objective $h_{\rho_{t}}$ as in PMs, that is, the
generator parameters are trained to minimize at each optimization step (where
$\eta$ is the learning rate):

${\mathcal{L}}_{\mathrm{gen}}\left(\theta\right)=-\eta\mathbb{E}_{z\sim
p_{z}}\left[h_{\rho_{t}}\left(g_{\theta}\left(z\right)\right)\right].$ (10)

In the optimization of Equation 10, we intentionally ignore the dependency of
$\rho$ on $\theta$, i.e. in practice
$\rho=\mathrm{StopGradient}\left(g_{\theta}\sharp p_{z}\right)$. This allows
us to mimic PMs where generated particles $x_{t}$s optimize the objective
$h_{\rho_{t}}$, without taking into account that $\rho_{t}$ is actually a
mixture of all the $x_{t}$s.

By optimizing $\theta_{t}$ via gradient descent for the loss of Equation 10,
idealized in the continuous training time setting, we obtain using the chain
rule:

$\mathinner{\dfrac{\operatorname{d\\!}{}\theta_{t}}{\operatorname{d\\!}{t}}}={\nabla}_{\theta_{t}}\mathbb{E}_{z\sim
p_{z}}\left[h_{\rho_{t}}\left(g_{\theta_{t}}\left(z\right)\right)\right]=\mathbb{E}_{z\sim
p_{z}}\left[{\nabla}_{\theta_{t}}g_{\theta_{t}}\left(z\right){\nabla}h_{\rho_{t}}\left(g_{\theta_{t}}\left(z\right)\right)\right].$
(11)

As a consequence, using the chain rule again, each generated particle
$x_{t}=g_{\theta_{t}}\left(z\right)\sim\rho_{t}$ evolves as:

$\mathinner{\dfrac{\operatorname{d\\!}{}g_{\theta_{t}}\left(z\right)}{\operatorname{d\\!}{t}}}={\nabla}_{\theta_{t}}g_{\theta_{t}}\left(z\right)^{\top}\mathinner{\dfrac{\operatorname{d\\!}{}\theta_{t}}{\operatorname{d\\!}{t}}}=\mathbb{E}_{z^{\prime}\sim
p_{z}}\left[k_{g_{\theta_{t}}}\left(z,z^{\prime}\right){\nabla}h_{\rho_{t}}\left(g_{\theta_{t}}\left(z^{\prime}\right)\right)\right],$
(12)

where $k_{g_{\theta}}\colon
z,z^{\prime}\mapsto{\nabla}_{\theta_{t}}g_{\theta_{t}}\left(z\right)^{\top}{\nabla}_{\theta_{t}}g_{\theta_{t}}\left(z^{\prime}\right)$
is the matrix Neural Tangent Kernel (NTK, Jacot et al., 2018) of the
generator. Equation 12 describes the dynamics of the generated particles as a
modified version of Equation 1 for PMs. We formalize this as follows.

######  Definition 2 (Interacting Particle Models, Int-PMs).

Int-PMs model particles resulting from the pushforward $g_{\theta_{t}}\sharp
p_{z}$ of a generator $g_{\theta_{t}}$ applied to a prior $p_{z}$, with the
following training dynamics:

$\displaystyle\operatorname{d\\!}g_{\theta_{t}}\left(z\right)=\left[{\mathcal{A}}_{\theta_{t}}\left(z\right)\right]\left({\nabla}h_{\rho_{t}}\right)\operatorname{d\\!}t,$
(13)

where $h_{\rho_{t}}\colon\mathbb{R}^{D}\to\mathbb{R}$ is a functional that
depends on the current distribution $\rho_{t}$, time $t$ is training time, and
${\mathcal{A}}_{\theta_{t}}\left(z\right)$ is a linear operator operating on
vector fields (Sriperumbudur et al., 2010), defined as:

$\displaystyle\left[{\mathcal{A}}_{\theta_{t}}\left(z\right)\right]\left(V\right)\triangleq\mathbb{E}_{z\sim
p_{z}}\left[k_{g_{\theta_{t}}}\left(z,z^{\prime}\right)V\left(g_{\theta_{t}}\left(z\right)\right)\right],$
$\displaystyle
k_{g_{\theta_{t}}}\left(z,z^{\prime}\right)\triangleq{\nabla}_{\theta_{t}}g_{\theta_{t}}\left(z^{\prime}\right)^{\top}{\nabla}_{\theta_{t}}g_{\theta_{t}}\left(z\right).$
(14)

Similarly to PMs, the vector field ${\nabla}h_{\rho_{t}}$ in Int-PMs indicates
which direction each generated particle will follow to get closer to the data
distribution. However, ${\mathcal{A}}_{\theta_{t}}$ smooths this gradient
field using the generator’s NTK, and generated particles thus interact with
each other. Indeed, moving one particle makes its neighbors move because of
their parameterization by the generator. Notably, Int-PMs generalize PMs: in
the degenerate case where $k\left(z,z^{\prime}\right)=\delta_{z-z^{\prime}}$,
with $\delta$ the Dirac delta function centered on $0$, i.e., when particles
can move freely with a sufficiently powerful generator, the effect of
parameterization disappears with ${\mathcal{A}}_{\theta_{t}}=\mathrm{id}$, and
therefore Equation 13 reduces to Equation 1.

######  Finding 4.

Int-PMs generalize PMs. Each Int-PM is therefore defined by two components:
the loss function $h_{\rho_{t}}$ and the choice of generator architecture
$g_{\theta}$.

We will show in the remainder of this section that Int-PMs encompass both GANs
and Stein gradient flows, borrowing Franceschi et al. (2022)’s results which
the previous reasoning generalizes.

###  3.2 GANs as Interacting Particle Models

In GANs, each generator $g_{\theta}$ is accompanied by a discriminator
$f_{\rho}$ that depends on the generated distribution. $f_{\rho}$ is optimized
as a neural network via gradient ascent (GA) to maximize an objective of the
following form:

$f_{\rho}=\mathrm{GA}_{f}\left\\{{\mathcal{L}}_{\mathrm{d}}\left(f;\rho,p_{\rm{data}}\right)\triangleq\mathbb{E}_{\rho}\left[a\circ
f\right]-\mathbb{E}_{p_{\rm{data}}}\left[b\circ
f\right]+{\mathcal{R}}\left(f;\rho,p_{\rm{data}}\right)\right\\},$ (15)

for some functions $a,b\colon\mathbb{R}\to\mathbb{R}$ (e.g. for the WGAN of
Arjovsky et al. (2017), $a=b=\mathrm{id}$) and regularization ${\mathcal{R}}$
(e.g., the gradient penalty of Gulrajani et al. (2017)). In this work, we
remain oblivious to how the discriminator is trained in practice as a single
network alongside the generator. Nonetheless, we note that this GA is usually
stopped early and not run until convergence, because the discriminator is
trained only for a few steps between generator updates.

This discriminator is then used to train the generator, as usually framed in a
min-max optimization setting. However, several works (Metz et al., 2017;
Franceschi et al., 2022; Yi et al., 2023) showed that generator optimization
deviates from min-max optimization, because alternating updates between the
generator and the discriminator make the generator minimize a loss function of
the form (Franceschi et al., 2022):

${\mathcal{L}}_{\mathrm{GAN}}\left(g_{\theta}\right)=\eta\mathbb{E}_{z\sim
p_{z}}\left[\left(c\circ
f_{\rho}\right)\left(g_{\theta}\left(z\right)\right)\right],$ (16)

for some $c\colon\mathbb{R}\to\mathbb{R}$ (e.g., for WGAN, $c=\mathrm{id}$).
Using Definition 2, we deduce the following observation.

######  Finding 5.

GANs are Int-PMs with $h_{\rho}=-c\circ f_{\rho}$, where $f_{\rho}$ is the
current discriminator.

Under some assumptions on the outcome of discriminator training in Equation
15, the resulting ${\nabla}h_{\rho}$ for GANs has been proven to implement a
Wasserstein gradient $-{\nabla}_{W}{\mathcal{F}}\left(\rho\right)$. Two
notable examples are (see also Section 2.1): $f$-divergence GANs (Nowozin et
al., 2016), which are linked to the forward KL divergence gradient flow (Yi et
al., 2023) and therefore diffusion models; and Integral Probability Metrics
GANs, which are linked to the squared MMD gradient flow w.r.t. the NTK of the
discriminator (Franceschi et al., 2022). However, these links have been made
under strong simplifying assumptions, and the GAN formulation as an Int-PM in
this paper is far more general.

######  Finding 6.

GANs as Int-PMs generalize gradient flows and diffusion models.

###  3.3 Stein Gradient Flows as Int-PMs

Int-PMs as framed in Definition 2 are similar to Stein gradient flows (Liu &
Wang, 2016; Liu, 2017; Duncan et al., 2023). The latter are a generalization
of Wasserstein gradient flows in another geometry shaped by a kernel
$k\colon\mathbb{R}^{D}\times\mathbb{R}^{D}\to\mathbb{R}^{D}\times\mathbb{R}^{D}$
defined in the data space:

$\operatorname{d\\!}x_{t}=-\mathbb{E}_{x^{\prime}_{t}\sim\rho_{t}}\left[k\left(x_{t},x^{\prime}_{t}\right){\nabla}_{W}{\mathcal{F}}\left(\rho_{t}\right)\left(x_{t}\right)\right]\operatorname{d\\!}t.$
(17)

Prior works showed strong links between such flows and GAN optimization, which
help us to see that GANs are Int-PMs. In the following, we will see that Stein
Gradient flows serve as an example of a functional generator-based counterpart
of a generator-less PM.

We begin by generalizing the reasonings of Chu et al. (2020), Durr et al.
(2022) and Franceschi et al. (2022), which initially applied only to the case
of GANs. We assume that for Int-PMs in Equations 13 and 14, the generator’s
NTK is constant throughout training, i.e. $k_{g_{\theta_{t}}}=k_{g}$, like for
many networks with infinite width (Jacot et al., 2018; Liu et al., 2020).
Then, when ${\nabla}h_{\rho}=-{\nabla}_{W}{\mathcal{F}}\left(\rho_{t}\right)$,
e.g., for gradient flows as in Finding 2 or for GANs following Finding 5, we
obtain an equation similar to Equation 17:

$\mathinner{\dfrac{\operatorname{d\\!}{}g_{\theta_{t}}\left(z\right)}{\operatorname{d\\!}{t}}}=-\mathbb{E}_{z^{\prime}\sim
p_{z}}\left[k_{g}\left(z,z^{\prime}\right){\nabla}_{W}{\mathcal{F}}\left(\rho_{t}\right)\left(g_{\theta_{t}}\left(z^{\prime}\right)\right)\right].$
(18)

This is a special case of Equation 17 with an invertible generator (Chu et
al., 2020). However, in the general case, the kernel
$k_{g}\colon\mathbb{R}^{d}\times\mathbb{R}^{d}\to\mathbb{R}^{D}\times\mathbb{R}^{D}$
acts on the latent space, making Equation 18 define a generalized latent-
driven Stein flow that was uncovered by Franceschi et al. (2022).

######  Finding 7.

Stein gradient flows are Int-PMs with an invertible generator in the NTK
regime and
$h_{\rho_{t}}=-\frac{\partial{\mathcal{F}}\left(\rho_{t}\right)}{\partial\rho_{t}}+\mathrm{cst}$.
They can be generalized to non-invertible generators with Equation 18.

##  4 Decoupling the Generator and the Flow in Particle Models

We saw in the last section that Int-PMs, such as GANs, generalize PMs, such as
diffusion and Wasserstein gradient flows, by applying their generative
equations to generator training. This suggests that many generative models can
be defined by their PM flow, which allows an optional generator to be trained.
As we saw in Section 3.3, this is the case for gradient flows, which can
either define a PM or an Int-PM with the same $h$ as a Wasserstein gradient.
Consequently, we formulate the following claim that directly results from our
framework.

######  Claim 1.

A generator can be trained using the gradient flow of a score-based diffusion
model instead of adversarial training, and it is possible to remove the
generator in a GAN by synthesizing samples with the discriminator only.

We confirm this hypothesis in this section by introducing corresponding new
hybrid models, which we call respectively Score GANs and Discriminator Flows
(see Table 1), and by empirically demonstrating their viability. Note that we
introduce these models as proofs of concepts of the applications of our
framework. Since they challenge many assumptions and standard practices of
generative modeling, they do not benefit from the same wealth of accumulated
knowledge that classic models have access to, and thus are harder to tune than
standard models.

Table 3:  Test FID of studied models.  Dataset | PMs (no generator) | Int-PMs (generator)
---|---|---
EDM | Discr. Flow | GAN | Score GAN
MNIST | $3$ | $4$ | $3$ | $15$
CelebA | $10$ | $41$ | $19$ | $35$

##### Experimental setting.

We conduct experiments on the unconditional generation task for two standard
datasets composed of images: MNIST (LeCun et al., 1998) and $64\times 64$
CelebA (Liu et al., 2015). We consider two reference baselines, EDM (the
score-based diffusion model of Karras et al. (2022)) and GANs, and use the
Fréchet Inception Distance (FID, Heusel et al., 2017) to test generative
performance in Table 3. Training details are given in Appendix C; our source
code is open-sourced at https://github.com/White-Link/gpm. We refer to
Appendix B and the code for more experimental results and samples for each
baseline.

Figure 1:  From left to right, generation process on MNIST of EDM and
Discriminator Flow for every $8$ evaluations of ${\nabla}h_{\rho_{t}}$. The
last row shows the first $7$ steps of Discriminator Flow.  Figure 2:
Uncurated samples of studied models on CelebA and MNIST.

###  4.1 Training Generators with Score-Based Diffusion: Score GANs

Input: Noise distribution $p_{\sigma}$, number of intermediate score training
steps $K$, learning rates $\lambda,\eta\in\mathbb{R}_{+}$, previous generator
$g_{\theta}\colon\mathbb{R}^{D}\to\mathbb{R}$, previous $\rho$ score model
$s^{\rho}_{\phi}\colon\mathbb{R}^{D}\times\mathbb{R}\to\mathbb{R}^{D}$,
pretrained $p_{\rm{data}}$ score model
$s^{p_{\rm{data}}}_{\psi}\colon\mathbb{R}^{D}\times\mathbb{R}\to\mathbb{R}^{D}$.

Output: Updated generator $g_{\theta}$ and $\rho$ score model
$s^{\rho}_{\phi}$.

1 for  _$k=1$ to $K$_ do // Updates of $s^{\rho}$ by denoising score matching

2 $x\sim p_{\rm{data}}$, $z\sim p_{z}$, $x=g_{\theta}\left(z\right)$,
$\sigma\sim p_{\sigma}$,
$x^{\sigma}\sim{\mathcal{N}}\left(x,\sigma^{2}I_{D}\right)$;

3
$\phi\leftarrow\phi-\lambda{\nabla}_{\phi}\left\|s^{\rho}_{\phi}\left(x^{\sigma},\sigma\right)+\frac{x^{\sigma}-x}{\sigma^{2}}\right\|_{2}^{2}$

// Score matching with generator, Equations 11 and 19

4 $x\sim p_{\rm{data}}$, $z\sim p_{z}$, $x=g_{\theta}\left(z\right)$,
$\sigma\sim p_{\sigma}$, $\varepsilon\sim{\mathcal{N}}\left(0,I_{D}\right)$;

$\theta\leftarrow\theta+\eta\cdot{\nabla}_{\theta}g_{\theta}\left(z\right)^{\top}\left(s_{\psi}^{p}{}_{\rm{data}}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)-s_{\phi}^{\rho}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)\right)$

Algorithm 1 Training iteration of Score GANs; all operations can be performed
in parallel for batching. See Appendix A for details on the practical
implementation of lines $3$ and $5$.

We propose training a generator with the score-based diffusion flow of NCSN,
Equation 9, left. This involves applying Equation 13 with
$h_{\rho_{t}}=\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}\right]-\log\rho_{t}$. To do so, we directly use the
generator weight update formula of Equation 11, as it avoids the problem of
estimating $h_{\rho_{t}}$ and only requires its gradient
${\nabla}h_{\rho_{t}}={\nabla}\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}\right]-{\nabla}\log\rho_{t}$. Composed of the scores
of, respectively, the noised data distribution and the generated distribution,
${\nabla}h_{\rho_{t}}$ can be efficiently estimated via score matching
techniques.

In practice, we use a score network $s_{\psi}^{p}{}_{\rm{data}}$ pretrained
with the latest denoising score matching techniques (Karras et al., 2022) to
estimate the static term ${\nabla}\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}\right]$. Moreover, as ${\nabla}\log\rho_{t}$ is
dynamic and needs to be continuously estimated, we leverage GAN discriminator
practices and train a network $s_{\phi}^{\rho}$ by alternating with generator
updates to estimate this score.

However, our proposed solution remains impractical for two reasons. First,
since the dynamics would match $p_{\rm{data}}\star k_{\mathrm{RBF}}^{\sigma}$
and $\rho_{t}$, we would need to schedule $\sigma$s during training, similar
to what Song & Ermon (2019) do during inference. Second, while
${\nabla}\log\rho_{t}$ can be estimated using sliced score matching (Song et
al., 2020), this approach is less performant than denoising score matching and
leads to estimation issues when $\rho_{t}$ lies on a manifold (Song & Ermon,
2019), as in our case with a pushforward generator. Both of these problems can
be solved by instead matching $p_{\rm{data}}\star k_{\mathrm{RBF}}^{\sigma}$
and $\rho_{t}\star k_{\mathrm{RBF}}^{\sigma}$ for a range of $\sigma$s, using
Equations 12 and 13:

$h_{\rho_{t}}=\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}\right]-\log\left[\rho_{t}\star
k_{\mathrm{RBF}}^{\sigma}\right].$ (19)

This approach allows us to leverage denoising score matching to train
$s_{\phi}^{\rho}$ and to avoid scheduling $\sigma$s by instead sampling them
during training and noising the generated distribution with the chosen noise
levels. Overall, we obtain the algorithm for Score GANs described in Algorithm
1.

######  Finding 8.

Score GANs are Int-PMs with $h_{\rho}=\mathbb{E}_{\sigma\sim
p_{\sigma}}\left[\log\left[\frac{p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}}{\rho_{t}\star
k_{\mathrm{RBF}}^{\sigma}}\right]\right]$ and $T=+\infty$.

###  4.2 Removing the Generator from GANs: Discriminator Flows

Input: Initial distribution $\pi=\rho_{0}$, gradient strength
$\eta\in\mathbb{R}_{+}$, learning rate $\lambda\in\mathbb{R}_{+}$, previous
discriminator $f_{\phi}\colon\mathbb{R}^{D}\times\mathbb{R}\to\mathbb{R}$.

Output: Updated discriminator $f_{\phi}$.

$x\sim p_{\rm{data}},x_{0}\sim\pi,t\sim{\mathcal{U}}\left(0,1\right)$;

// Initialization, random sampling time

$x_{t}\leftarrow x_{0}-\eta\int_{0}^{t}{\nabla}_{x_{s}}\left[\left(c\circ
f_{\phi}\right)\left(x_{s},s\right)\right]\operatorname{d\\!}s$;

// Partial generation, Equation 20

$\phi\leftarrow\phi+\lambda{\nabla}_{\phi}\left\\{{\mathcal{L}}_{\mathrm{d}}\left(f_{\phi}\left(\cdot,t\right);\delta_{x_{t}},\delta_{x}\right)\right\\}$;

// Train $f_{\phi}$ at time $t$, cf. Equation 15

Algorithm 2 Training iteration of Discr. Flows. Cf. batching and
discretization in Appendix A.

Based on Findings 4 and 5, we see that removing the generator from GANs to
make them PMs, as in Definition 1, simply requires that we define
$h_{\rho_{t}}=-c\circ f_{\rho_{t}}$, where $f_{\rho_{t}}$ is the discriminator
between $\rho_{t}$ and $p_{\rm{data}}$ at sampling time $t$ in Equation 1:

$\operatorname{d\\!}x_{t}=-{\nabla}\left(c\circ
f_{\rho_{t}}\right)\left(x_{t}\right)\operatorname{d\\!}t,$ (20)

with each particle following the pointwise GAN generator loss of Equation 16.
This makes Equation 20 the equivalent of GAN training, but as a PM without
parameterization by a generator.

This approach requires training a neural discriminator
$f_{\phi}\colon\mathbb{R}^{D}\times\mathbb{R}\to\mathbb{R}$, that takes as
input both a sample $x_{t}\sim\rho_{t}$ and its corresponding time
$t\in\mathbb{R}$. Each $x_{t}\sim\rho_{t}$ must be computed both in training
and inference from $x_{0}\sim\rho_{0}=\pi$ using Equation 20. For practical
convenience, we restrict, without loss of generality, $t\in\left[0,1\right]$,
i.e. $T=1$. This results in the training procedure of Algorithm 2 for
Discriminator Flows.

######  Finding 9.

Discriminator Flows are PMs with $h_{\rho}=-c\circ f_{\rho_{t}}$ and
$T=+\infty$.

Unlike diffusion models, for which the score can be freely estimated at each
$t$ because $\rho_{t}$ is assumed to equal $p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma\left(t\right)}$, the generation process of
Discriminator Flows is not known in advance. In fact, we must simulate the
entire process during training, making each training iteration slower.
However, our approach has the advantage of generalizing the diffusion process
and allowing the discriminator to learn another path from the initial
distribution $\rho_{0}=\pi$ to $p_{\rm{data}}$. We believe that, when properly
tuned, Discriminator Flows may benefit from faster sampling times than
diffusion models because they can learn shorter paths towards $p_{\rm{data}}$.

###  4.3 Experimental results

We see uncurated samples from our proposed Score GAN and Discriminator Flow
models in Figure 2, as well as the baseline EDM and GAN models. In
LABEL:tab:fid we report the FID scores for all models. We observe that both of
our introduced models produce reasonable results, which experimentally
confirms our initial claim. Nonetheless, these models exhibit worse (higher)
FID scores than our baselines, although Discriminator Flows provide good
performance on the MNIST dataset. We attribute these performance results to
the fact that our proposed models are novel and do not benefit from the
accumulated knowledge regarding training best practices that the standard
models possess.

Interestingly, the proposed models exhibit typical properties of generator-
free and generator-based models. Since Score GANs use a generator, as in GANs,
only one function evaluation is required to draw a sample, making it orders of
magnitude faster to sample from than EDM; like GANs, Score GANs may produce
mode collapse, as shown in Appendix B. Of course, Discriminator Flows are
slower at both training and inference time than such generator-based models,
but this comes with the additional flexibility of operating directly in the
data space (Voleti et al., 2022; Couairon et al., 2023). However, as shown in
Figure 1 and in the numerical results in Appendix B, Discriminator Flows have
the expected advantage of converging faster to the target distribution as
compared to the state-of-the-art diffusion model EDM, and thus have higher
efficiency during inference.

###  4.4 Relationship with Prior Work

Score GANs. Conceptually, Score GANs implement for each $\sigma$ a forward KL
gradient flow, similarly to Yi et al. (2023) who theoretically proved that
some GAN models approximate it without noising, under optimality assumptions
on the discriminator. However, Score GANs differ from traditional GANs as they
involve no discriminator but rather split the flow ${\nabla}h_{\rho_{t}}$ into
two parts: one that can be estimated before generator training as it depends
only on the data distribution, and one that must be continuously estimated
during training as it depends on the generated distribution. While
$h_{\rho_{t}}$ could be estimated by adding noise to the inputs of a
discriminator (Wang et al., 2022), Score GANs instead only need to estimate
the score of the generated distribution, which is no longer adversarial.

Discriminator Flows. As a general concept, Discriminator Flows provide an
encompassing framework that helps to reveal the connections of various
approaches to GAN training.

Recently, Heng et al. (2023) introduced deep generative Wasserstein gradient
flows (DGGF), a method that relies on $f$-divergence gradient flows
approximated by estimating the ratio $\nicefrac{{\rho_{t}}}{{p_{\rm{data}}}}$
with a neural network. Examined under our framework, DGGF’s training
objectives correspond to that of a discriminator in $f$-divergence GANs,
allowing us to frame DGGF as a special case of Discriminator Flows.
Nonetheless, we stress that Discriminator Flows have a larger scope than DGGF
since we can handle all types of GAN objectives; all our experiments were
performed with WGAN objectives. Moreover, unlike DGGF, which in practice
removes the time dependency in their estimation without a theoretical
justification, our method does handle time as input to the discriminator.

Discriminator Flows also relate to, and generalize, methods that finetune GAN
outputs with gradient flows (Tanaka, 2019; Che et al., 2020; Ansari et al.,
2021). The latter use discriminator gradients to approximate such flows,
making them naturally expressible as Discriminator Flows in our framework. In
practice, they only apply their sampling procedures in the latent space of the
generator, as applying them in pixel space leads to artifacts. We resolve this
issue with a principled training procedure for the discriminator conditioned
on sampling time.

Finally, we provide intuition on the sampling efficiency of Discriminator
Flows observed in Figure 1. We observe that diffusion models smooth the data
distribution with a Gaussian kernel by Equation 5, while discriminators were
shown to smooth the data distribution with their NTK (Franceschi et al.,
2022). This observation brings diffusion models and GANs closer while
explaining the sampling efficiency of Discriminator Flows thanks to the
properties of NTKs for generative modeling. Indeed, Franceschi et al. (2022)
presented empirical evidence that using NTKs of standard discriminators as
kernels in MMD gradient flows (which some GAN models implement) instead of
Gaussian kernels accelerates convergence towards $p_{\rm{data}}$ of Equation 1
by several orders of magnitude.

##  5 Conclusion

In this paper we have unified score-based diffusion models, GANs, and gradient
flows under a single framework based on particle models which can be
complemented with a generator. Since this framework unifies models that have
been customarily opposed in the literature, this work paves the way for new
perspectives in generative modeling. As an example of potential applications,
we have shown that our framework naturally leads to two novel generative
models: a generator that follows score-based gradients, and a generator-free
GAN that uses a discriminator-guided generation process.

Of course, generator-less and generator-based models each retain their unique
attributes. On the one hand, generator training provides a simple and
efficient sampling procedure and endows the generative model with a low-
dimensional structured latent space, at the cost of potential instability and
mode collapse. On the other hand, generator-less models, despite their slow
sampling, may be easier to train since the generator component has been
removed from their flow and are more flexible as they rely on a continuous-
time process directly defined in the data space. We believe that our
framework, by unveiling the close relationship between these models, can help
them to improve upon one another, or can even help create other new hybrid
models.

Beyond potential applications, our study could be enhanced and expanded in
many ways for future work. On the theoretical side, we would like to tackle
the challenging task (Hsieh et al., 2021) of taking into account the fact that
the discriminator in GANs is actually continuously trained with the generator.
On the practical side, while we have proposed new models that function
reasonably well, we would be interested in refining them further for state-of-
the-art generative performance. Furthermore, Score GANs could serve as a
distillation method for score-based diffusion models (Salimans & Ho, 2022),
while Discriminator Flows could outperform diffusion models for generation
efficiency.

## Acknowledgements

We would like to thank Lorenzo Croissant and Ugo Tanielian for helpful
discussions and comments on the paper, as well as Edouard Delasalles for
inspiring the architecture of our code.

This work was granted access to the HPC / AI resources of IDRIS under the
allocation 2022-AD011013503R1 made by GENCI (Grand Equipement National de
Calcul Intensif). Emmanuel de Bézenac is financially supported by the ETH
Foundations of Data Science.

## References

  * Alvarez-Melis et al. (2022) Alvarez-Melis, D., Schiff, Y., and Mroueh, Y.  Optimizing functionals on the space of probabilities with input convex neural networks.  _Transactions on Machine Learning Research_ , 2022.
  * Ansari et al. (2021) Ansari, A. F., Ang, M. L., and Soh, H.  Refining deep generative models via discriminator gradient flow.  In _International Conference on Learning Representations_ , 2021.
  * Arbel et al. (2019) Arbel, M., Korba, A., Salim, A., and Gretton, A.  Maximum mean discrepancy gradient flow.  In Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché Buc, F., Fox, E., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 32, pp. 6484–6494. Curran Associates, Inc., 2019\.
  * Arjovsky et al. (2017) Arjovsky, M., Chintala, S., and Bottou, L.  Wasserstein generative adversarial networks.  In Precup, D. and Teh, Y. W. (eds.), _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , pp. 214–223. PMLR, August 2017.
  * Bengio et al. (2013) Bengio, Y., Courville, A., and Vincent, P.  Representation learning: A review and new perspectives.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 35(8):1798–1828, August 2013.
  * Brock et al. (2019) Brock, A., Donahue, J., and Simonyan, K.  Large scale GAN training for high fidelity natural image synthesis.  In _International Conference on Learning Representations_ , 2019.
  * Che et al. (2020) Che, T., Zhang, R., Sohl-Dickstein, J., Larochelle, H., Paull, L., Cao, Y., and Bengio, Y.  Your GAN is secretly an energy-based model and you should use discriminator driven latent sampling.  In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.-F., and Lin, H.-T. (eds.), _Advances in Neural Information Processing Systems_ , volume 33, pp. 12275–12287. Curran Associates, Inc., 2020.
  * Chu et al. (2020) Chu, C., Minami, K., and Fukumizu, K.  The equivalence between Stein variational gradient descent and black-box variational inference.  _arXiv preprint arXiv:2004.01822_ , 2020.
  * Couairon et al. (2023) Couairon, G., Verbeek, J., Schwenk, H., and Cord, M.  DiffEdit: Diffusion-based semantic image editing with mask guidance.  In _International Conference on Learning Representations_ , 2023.
  * Dhariwal & Nichol (2021) Dhariwal, P. and Nichol, A.  Diffusion models beat GANs on image synthesis.  In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), _Advances in Neural Information Processing Systems_ , volume 34, pp. 8780–8794. Curran Associates, Inc., 2021.
  * Duncan et al. (2023) Duncan, A., Nüsken, N., and Szpruch, L.  On the geometry of stein variational gradient descent.  _Journal of Machine Learning Research_ , 24(56):1–39, 2023.
  * Durr et al. (2022) Durr, S., Mroueh, Y., Tu, Y., and Wang, S.  Effective dynamics of generative adversarial networks.  _arXiv preprint arXiv:2212.04580_ , 2022.
  * Fallis (2021) Fallis, D.  The epistemic threat of deepfakes.  _Philosophy & Technology_, 34:623––643, 2021.
  * Fan et al. (2022) Fan, J., Zhang, Q., Taghvaei, A., and Chen, Y.  Variational Wasserstein gradient flow.  In Chaudhuri, K., Jegelka, S., Song, L., Szepesvári, C., Niu, G., and Sabato, S. (eds.), _Proceedings of the 39th International Conference on Machine Learning_ , volume 162 of _Proceedings of Machine Learning Research_ , pp. 6185–6215. PMLR, July 2022.
  * Franceschi et al. (2022) Franceschi, J.-Y., De Bézenac, E., Ayed, I., Chen, M., Lamprier, S., and Gallinari, P.  A neural tangent kernel perspective of GANs.  In Chaudhuri, K., Jegelka, S., Song, L., Szepesvári, C., Niu, G., and Sabato, S. (eds.), _Proceedings of the 39th International Conference on Machine Learning_ , volume 162 of _Proceedings of Machine Learning Research_ , pp. 6660–6704. PMLR, July 2022.
  * Gao et al. (2019) Gao, Y., Jiao, Y., Wang, Y., Wang, Y., Yang, C., and Zhang, S.  Deep generative learning via variational gradient flow.  In Chaudhuri, K. and Salakhutdinov, R. (eds.), _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pp. 2093–2101. PMLR, June 2019\.
  * Glaser et al. (2021) Glaser, P., Arbel, M., and Gretton, A.  KALE flow: A relaxed KL gradient flow for probabilities with disjoint support.  In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), _Advances in Neural Information Processing Systems_ , volume 34, pp. 8018–8031. Curran Associates, Inc., 2021.
  * Goodfellow (2016) Goodfellow, I.  NIPS 2016 tutorial: Generative adversarial networks.  _arXiv preprint arXiv:1701.00160_ , 2016.
  * Goodfellow et al. (2014) Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y.  Generative adversarial nets.  In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N. D., and Weinberger, K. Q. (eds.), _Advances in Neural Information Processing Systems_ , volume 27, pp. 2672–2680. Curran Associates, Inc., 2014.
  * Gretton et al. (2012) Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola, A.  A kernel two-sample test.  _Journal of Machine Learning Research_ , 13(25):723–773, 2012.
  * Gulrajani et al. (2017) Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., and Courville, A.  Improved training of Wasserstein GANs.  In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 30, pp. 5769–5779. Curran Associates, Inc., 2017.
  * Hendrycks & Gimpel (2016) Hendrycks, D. and Gimpel, K.  Gaussian error linear units (GeLUs).  _arXiv preprint arXiv:1606.08415_ , 2016.
  * Heng et al. (2023) Heng, A., Ansari, A. F., and Soh, H.  Deep generative Wasserstein gradient flows, 2023.  URL https://openreview.net/forum?id=zjSeBTEdXp1.
  * Heusel et al. (2017) Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S.  GANs trained by a two time-scale update rule converge to a local Nash equilibrium.  In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 30, pp. 6629––6640. Curran Associates, Inc., 2017.
  * Ho et al. (2022) Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., and Salimans, T.  Imagen Video: High definition video generation with diffusion models.  _arXiv preprint arXiv:2210.02303_ , 2022.
  * Hsieh et al. (2021) Hsieh, Y.-P., Mertikopoulos, P., and Cevher, V.  The limits of min-max optimization algorithms: Convergence to spurious non-critical sets.  In Meila, M. and Zhang, T. (eds.), _Proceedings of the 38th International Conference on Machine Learning_ , volume 139 of _Proceedings of Machine Learning Research_ , pp. 4337–4348. PMLR, July 2021\.
  * Hyvärinen (2005) Hyvärinen, A.  Estimation of non-normalized statistical models by score matching.  _Journal of Machine Learning Research_ , 6(24):695–709, 2005.
  * Ioffe & Szegedy (2015) Ioffe, S. and Szegedy, C.  Batch normalization: Accelerating deep network training by reducing internal covariate shift.  In Bach, F. and Blei, D. (eds.), _Proceedings of the 32nd International Conference on Machine Learning_ , volume 37 of _Proceedings of Machine Learning Research_ , pp. 448–456, Lille, France, July 2015. PMLR.
  * Jacot et al. (2018) Jacot, A., Gabriel, F., and Hongler, C.  Neural tangent kernel: Convergence and generalization in neural networks.  In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 31, pp. 8580–8589. Curran Associates, Inc., 2018.
  * Jordan et al. (1998) Jordan, R., Kinderlehrer, D., and Otto, F.  The variational formulation of the Fokker-Planck equation.  _SIAM Journal on Mathematical Analysis_ , 29(1):1–17, 1998.
  * Kang et al. (2022) Kang, M., Shin, J., and Park, J.  StudioGAN: A taxonomy and benchmark of GANs for image synthesis.  _arXiv preprint arXiv:2206.09479_ , 2022.
  * Karras et al. (2022) Karras, T., Aittala, M., Aila, T., and Laine, S.  Elucidating the design space of diffusion-based generative models.  In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), _Advances in Neural Information Processing Systems_ , volume 35, pp. 26565–26577. Curran Associates, Inc., 2022.
  * Kingma & Ba (2015) Kingma, D. P. and Ba, J.  Adam: A method for stochastic optimization.  In _International Conference on Learning Representations_ , 2015.
  * Kloeden & Platen (1992) Kloeden, P. E. and Platen, E.  _Introduction to Stochastic Time Discrete Approximation_ , pp. 305–337.  Applications of Mathematics. Springer Berlin Heidelberg, Berlin - Heidelberg, Germany, 1992.
  * Kullback & Leibler (1951) Kullback, S. and Leibler, R. A.  On information and sufficiency.  _The Annals of Mathematical Statistics_ , 22(1):79–86, 1951.
  * LeCun et al. (1998) LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P.  Gradient-based learning applied to document recognition.  _Proceedings of the IEEE_ , 86(11):2278–2324, November 1998.
  * Li et al. (2022) Li, X., Thickstun, J., Gulrajani, I., Liang, P., and Hashimoto, T. B.  Diffusion-LM improves controllable text generation.  In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), _Advances in Neural Information Processing Systems_ , volume 35, pp. 4328–4343. Curran Associates, Inc., 2022.
  * Lim & Ye (2017) Lim, J. H. and Ye, J. C.  Geometric GAN.  _arXiv preprint arXiv:1705.02894_ , 2017.
  * Liu et al. (2021) Liu, B., Zhu, Y., Song, K., and Elgammal, A.  Towards faster and stabilized GAN training for high-fidelity few-shot image synthesis.  In _International Conference on Learning Representations_ , 2021.
  * Liu et al. (2020) Liu, C., Zhu, L., and Belkin, M.  On the linearity of large non-linear models: when and why the tangent kernel is constant.  In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.-F., and Lin, H.-T. (eds.), _Advances in Neural Information Processing Systems_ , volume 33, pp. 15954–15964. Curran Associates, Inc., 2020.
  * Liu et al. (2018) Liu, L., Lu, Y., Yang, M., Qu, Q., Zhu, J., and Li, H.  Generative adversarial network for abstractive text summarization.  In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence_ , AAAI’18, pp. 8109–8110. AAAI Press, 2018.
  * Liu (2017) Liu, Q.  Stein variational gradient descent as gradient flow.  In Guyon, I., von Luxburg, U., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S. V. N., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 30, pp. 3118––3126. Curran Associates, Inc., 2017.
  * Liu & Wang (2016) Liu, Q. and Wang, D.  Stein variational gradient descent: A general purpose Bayesian inference algorithm.  In Lee, D. D., Sugiyama, M., von Luxburg, U., Guyon, I., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 29, pp. 2378–2386. Curran Associates, Inc., 2016.
  * Liu et al. (2015) Liu, Z., Luo, P., Wang, X., and Tang, X.  Deep learning face attributes in the wild.  In _IEEE International Conference on Computer Vision (ICCV)_ , pp. 3730–3738, December 2015.
  * Liutkus et al. (2019) Liutkus, A., Simsekli, U., Majewski, S., Durmus, A., and Stöter, F.-R.  Sliced-Wasserstein flows: Nonparametric generative modeling via optimal transport and diffusions.  In Chaudhuri, K. and Salakhutdinov, R. (eds.), _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pp. 4104–4113. PMLR, June 2019\.
  * Lucy & Bamman (2021) Lucy, L. and Bamman, D.  Gender and representation bias in GPT-3 generated stories.  In _Proceedings of the Third Workshop on Narrative Understanding_ , pp. 48–55, Virtual, June 2021. Association for Computational Linguistics.
  * Metz et al. (2017) Metz, L., Poole, B., Pfau, D., and Sohl-Dickstein, J.  Unrolled generative adversarial networks.  In _International Conference on Learning Representations_ , 2017.
  * Miyato et al. (2018) Miyato, T., Kataoka, T., Koyama, M., and Yoshida, Y.  Spectral normalization for generative adversarial networks.  In _International Conference on Learning Representations_ , 2018.
  * Mokrov et al. (2021) Mokrov, P., Korotin, A., Li, L., Genevay, A., Solomon, J., and Burnaev, E.  Large-scale Wasserstein gradient flows.  In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), _Advances in Neural Information Processing Systems_ , volume 34, pp. 15243–15256. Curran Associates, Inc., 2021.
  * Mroueh et al. (2019) Mroueh, Y., Sercu, T., and Raj, A.  Sobolev descent.  In Chaudhuri, K. and Sugiyama, M. (eds.), _Proceedings of the Twenty-Second International Conference on Artificial Intelligence and Statistics_ , volume 89 of _Proceedings of Machine Learning Research_ , pp. 2976–2985. PMLR, April 2019.
  * Müller (1997) Müller, A.  Integral probability metrics and their generating classes of functions.  _Advances in Applied Probability_ , 29(2):429–443, 1997.
  * Nichol et al. (2022) Nichol, A. Q., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M.  GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models.  In Chaudhuri, K., Jegelka, S., Song, L., Szepesvári, C., Niu, G., and Sabato, S. (eds.), _Proceedings of the 39th International Conference on Machine Learning_ , volume 162 of _Proceedings of Machine Learning Research_ , pp. 16784–16804. PMLR, July 2022.
  * Nowozin et al. (2016) Nowozin, S., Cseke, B., and Tomioka, R.  $f$-GAN: Training generative neural samplers using variational divergence minimization.  In Lee, D. D., Sugiyama, M., von Luxburg, U., Guyon, I., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 29, pp. 271––279. Curran Associates, Inc., 2016.
  * Paszke et al. (2019) Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Köpf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., and Chintala, S.  PyTorch: An imperative style, high-performance deep learning library.  In Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché Buc, F., Fox, E., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 32, pp. 8026–8037. Curran Associates, Inc., 2019\.
  * Radford et al. (2016) Radford, A., Metz, L., and Chintala, S.  Unsupervised representation learning with deep convolutional generative adversarial networks.  In _International Conference on Learning Representations_ , 2016.
  * Rombach et al. (2022) Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B.  High-resolution image synthesis with latent diffusion models.  In _IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 10684–10695, June 2022.
  * Ronneberger et al. (2015) Ronneberger, O., Fischer, P., and Brox, T.  U-Net: Convolutional networks for biomedical image segmentation.  In Navab, N., Hornegger, J., Wells, W. M., and Frangi, A. F. (eds.), _Medical Image Computing and Computer-Assisted Intervention — MICCAI 2015_ , pp. 234–241, Cham, Switzerland, 2015. Springer International Publishing.
  * Rényi (1961) Rényi, A.  On measures of entropy and information.  In Neyman, J. (ed.), _Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability_ , volume 1, pp. 547–561. University of California Press, 1961.
  * Salimans & Ho (2022) Salimans, T. and Ho, J.  Progressive distillation for fast sampling of diffusion models.  In _International Conference on Learning Representations_ , 2022.
  * Santambrogio (2017) Santambrogio, F.  $\\{$Euclidean, metric, and Wasserstein$\\}$ gradient flows: an overview.  _Bulletin of Mathematical Sciences_ , 7:87–154, 2017.
  * Seymour & Tully (2018) Seymour, J. and Tully, P.  Generative models for spear phishing posts on social media.  _arXiv preprint arXiv:1802.05196_ , 2018.
  * Skafte Detlefsen et al. (2022) Skafte Detlefsen, N., Borovec, J., Schock, J., Harsh, A., Koker, T., Di Liello, L., Stancl, D., Quan, C., Grechkin, M., and Falcon, W.  TorchMetrics - measuring reproducibility in PyTorch, February 2022\.  URL https://github.com/Lightning-AI/torchmetrics.
  * Song (2021) Song, Y.  Generative modeling by estimating gradients of the data distribution.  https://yang-song.net/blog/2021/score/, 2021.  Accessed: 2023-05-17.
  * Song & Ermon (2019) Song, Y. and Ermon, S.  Generative modeling by estimating gradients of the data distribution.  In Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché Buc, F., Fox, E., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 32, pp. 11918–11930. Curran Associates, Inc., 2019\.
  * Song et al. (2020) Song, Y., Garg, S., Shi, J., and Ermon, S.  Sliced score matching: A scalable approach to density and score estimation.  In Adams, R. P. and Gogate, V. (eds.), _Proceedings of The 35th Uncertainty in Artificial Intelligence Conference_ , volume 115 of _Proceedings of Machine Learning Research_ , pp. 574–584. PMLR, July 2020\.
  * Song et al. (2021) Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B.  Score-based generative modeling through stochastic differential equations.  In _International Conference on Learning Representations_ , 2021.
  * Sriperumbudur et al. (2010) Sriperumbudur, B. K., Gretton, A., Fukumizu, K., Schölkopf, B., and Lanckriet, G. R. G.  Hilbert space embeddings and metrics on probability measures.  _Journal of Machine Learning Research_ , 11(50):1517–1561, 2010.
  * Tanaka (2019) Tanaka, A.  Discriminator optimal transport.  In Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché Buc, F., Fox, E., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems_ , volume 32, pp. 6816––6826. Curran Associates, Inc., 2019\.
  * Tanielian et al. (2020) Tanielian, U., Issenhuth, T., Dohmatob, E., and Mary, J.  Learning disconnected manifolds: a no GAN’s land.  In Daumé, III, H. and Singh, A. (eds.), _Proceedings of the 37th International Conference on Machine Learning_ , volume 119 of _Proceedings of Machine Learning Research_ , pp. 9418–9427. PMLR, July 2020\.
  * Vincent (2011) Vincent, P.  A connection between score matching and denoising autoencoders.  _Neural Computation_ , 23(7):1661––1674, July 2011.
  * Voleti et al. (2022) Voleti, V., Jolicoeur-Martineau, A., and Pal, C.  MCVD - masked conditional video diffusion for prediction, generation, and interpolation.  In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A. (eds.), _Advances in Neural Information Processing Systems_ , volume 35, pp. 23371–23385. Curran Associates, Inc., 2022.
  * Wang et al. (2021) Wang, D., Li, C., Wen, S., Nepal, S., and Xiang, Y.  Man-in-the-middle attacks against machine learning classifiers via malicious generative models.  _IEEE Transactions on Dependable and Secure Computing_ , 18(5):1941–0018, September 2021.
  * Wang et al. (2022) Wang, Z., Zheng, H., He, P., Chen, W., and Zhou, M.  Diffusion-GAN: Training GANs with diffusion.  _arXiv preprint arXiv:2206.02262_ , 2022.
  * Xiao et al. (2022) Xiao, Z., Kreis, K., and Vahdat, A.  Tackling the generative learning trilemma with denoising diffusion GANs.  In _International Conference on Learning Representations_ , 2022.
  * Yi et al. (2023) Yi, M., Zhu, Z., and Liu, S.  MonoFlow: Rethinking divergence GANs via the perspective of differential equations.  _arXiv preprint arXiv:2302.01075_ , 2023.
  * Zhang et al. (2019) Zhang, H., Goodfellow, I., Metaxas, D., and Odena, A.  Self-attention generative adversarial networks.  In Chaudhuri, K. and Salakhutdinov, R. (eds.), _Proceedings of the 36th International Conference on Machine Learning_ , volume 97 of _Proceedings of Machine Learning Research_ , pp. 7354–7363. PMLR, June 2019\.

##  Appendix A Algorithmic Details

We detail in this section some aspects of the Score GAN and Discriminator Flow
algorithms that were described at a high level in Section 4.

###  A.1 Score GANs

Algorithm 1 involves two major steps: line 3 performs denoising score matching
for the generated distribution, and line 5 updates the generator parameters
using the resulting gradient flow in Equation 19. We describe some
implementation tricks for these steps in the following subsections.

####  A.1.1 Denoising Score Matching

Denoising score matching was described in line 3 of Algorithm 1, as originally
used by Song & Ermon (2019). Following best practices later introduced by
Karras et al. (2022), instead of directly training a network $s_{\phi}^{\rho}$
to estimate the score, we instantiate and train a denoising network
$d_{\phi}^{\rho}$ using the following update:

$\phi\leftarrow\phi-\lambda{\nabla}_{\phi}\left\|d_{\phi}^{\rho}\left(x^{\sigma},\sigma\right)-x\right\|_{2}^{2},$
(21)

where $d_{\phi}^{\rho}$ is implemented as a U-Net (Ronneberger et al., 2015),
with additional input-output skip connections for better preconditioning
(Karras et al., 2022). We then use $d_{\phi}^{\rho}$ to compute the estimated
score:

$s_{\phi}^{\rho}\left(x,\sigma\right)=\frac{d_{\phi}^{\rho}\left(x,\sigma\right)-x}{\sigma^{2}}.$
(22)

We use the same tricks to pretrain the score model of the data distribution,
$s_{\psi}^{p_{\rm{data}}}$.

####  A.1.2 Generator Training

Line 5 of Algorithm 1 indicates how to update generator parameters, following
Equation 11. In order to facilitate its implementation in deep learning
frameworks, we instead use the equivalent weight update:

$\theta\leftarrow\theta+\eta{\nabla}_{\theta}\left[g_{\theta}\left(z\right)^{\top}\mathrm{StopGradient}\left(s_{\psi}^{p}{}_{\rm{data}}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)-s_{\phi}^{\rho}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)\right)\right],$
(23)

that is, we optimize the loss
$-g_{\theta}\left(z\right)^{\top}\mathrm{StopGradient}\left(s_{\psi}^{p}{}_{\rm{data}}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)-s_{\phi}^{\rho}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)\right)$.
We also adapt the weight update as follows. While keeping the same $h_{\rho}$
as in Equation 19, we multiply it in the weight update by $\sigma$ (which
amounts to changing the noise level sampling distribution $p_{\sigma}$
accordingly). When implemented with a denoiser network as described in Section
A.1.1, the score estimation $s_{\phi}^{\rho}\left(x,\sigma\right)$ can be
numerically unstable for small $\sigma$s when using Equation 22. This explains
why Karras et al. (2022) consistently multiply scores by $\sigma$, which is a
choice that we follow for Score GANs. Furthermore, we can generalize Score
GANs to other functionals $h_{\rho}$ encompassing the EDM formulation of
Equation 9, right:

$h_{\rho_{t}}=\mu_{p_{\rm{data}}}\log\left[p_{\rm{data}}\star
k_{\mathrm{RBF}}^{\sigma}\right]-\mu_{\rho}\log\left[\rho_{t}\star
k_{\mathrm{RBF}}^{\sigma}\right],$ (24)

where $\mu_{p_{\rm{data}}}$ and $\mu_{\rho}$ are constant hyperparameters
(e.g., for EDM, $\mu_{p_{\rm{data}}}=2$ and $\mu_{\rho}=1$).

Overall, in practice we implement the generator weight update corresponding to
line 5 of Algorithm 1 as:

$\theta\leftarrow\theta+\eta{\nabla}_{\theta}\left[\sigma\cdot
g_{\theta}\left(z\right)^{\top}\mathrm{StopGradient}\left(\mu_{p_{\rm{data}}}s_{\psi}^{p}{}_{\rm{data}}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)-\mu_{\rho}s_{\phi}^{\rho}\left(g_{\theta}\left(z\right)+\sigma\varepsilon,\sigma\right)\right)\right].$
(25)

###  A.2 Discriminator Flows

Input: Batch size $B\in{\mathbb{N}}^{\ast}$, number of steps
$N\in{\mathbb{N}}^{\ast}$, initial distribution $\pi=\rho_{0}$, gradient
strength $\eta\in\mathbb{R}_{+}$, previous discriminator
$f_{\phi}\colon\mathbb{R}^{D}\times\mathbb{R}\to\mathbb{R}$.

Output: Updated discriminator $f_{\phi}$.

1 for  _$b=1$ to $B$_ do // In parallel

$x^{b}\sim p_{\rm{data}},x_{0}^{b}\sim\pi$;

// Initialization

// Partial generation

2 for  _$i=0$ to $N-1$_ do // Solve Equation 20

3 $x_{\nicefrac{{i+1}}{{N}}}^{b}\leftarrow
x_{\nicefrac{{i}}{{N}}}^{b}-\frac{\eta}{N}{\nabla}_{x_{\nicefrac{{i}}{{N}}}^{b}}\left[\left(c\circ
f_{\phi}\right)\left(x_{\nicefrac{{i}}{{N}}}^{b},\frac{i}{N}\right)\right]$;

4

$i_{b}\sim{\mathcal{U}}\left(\left\llbracket 0,N-1\right\rrbracket\right)$;

// Select random step

5

// Train discriminator $f_{\phi}$ at the chosen random steps, cf. Equation 15

$\phi\leftarrow\mathrm{GA}_{\phi}\left\\{{\mathcal{L}}_{\mathrm{d}}\left(f_{\phi}\left(\cdot,\frac{i_{b}}{N}\right);{\mathcal{U}}\left(\left\\{x_{\nicefrac{{i_{b}}}{{N}}}^{b}\right\\}_{b\in\left\llbracket
1,B\right\rrbracket}\right),{\mathcal{U}}\left(\left\\{x^{b}\right\\}_{b\in\left\llbracket
1,B\right\rrbracket}\right)\right)\right\\}$

Algorithm 3 Training iteration for Discriminator Flow (detailed).

In Algorithm 3 we provide a detailed implementation of the high-level
Algorithm 2 for Discriminator Flows, including how batching and differential
equation discretization are handled. For the latter, we use the Euler method
with a uniform temporal grid, with $N$ steps, in $\left[0,1\right]$; while
higher-order solvers and more optimal temporal grid choice could have been
used, as in EDM (Karras et al., 2022), we avoid using any refinement of time
discretization for Discriminator Flows for the sake of simplicity. For batch-
parallel execution on GPUs, we solve the differential equation of Equation 20
over $\left[0,1\right]$ for all batch samples, and then select a random time
for each sample to compute the discriminator loss and update its parameters.

##  Appendix B Further Experiments

In this section we present additional experiments that will help the reader
develop better intuition on the behavior of both Score GANs and Discriminator
Flows in practice.

###  B.1 Additional Samples

Figure 3:  Uncurated samples from studied models trained on CelebA and MNIST.

Figure 3 shows additional samples for all four tested models. Furthermore, for
better visualization purposes, our public repository https://github.com/White-
Link/gpm includes animated images illustrating the generation process for EDM
and Discriminator Flows, as illustrated in Figure 1.

###  B.2 Mode Collapse on Score GANs

Figure 4:  Uncurated samples of a Score GAN variant which shows strong mode
collapse on MNIST. Note that the Score GAN parameters were intentionally
chosen to obtain this behavior.

A widely known issue for GANs, identified soon after their introduction, is
known as mode collapse (Goodfellow, 2016), which occurs when the generator
only covers a fraction of the generated distribution. Interestingly, we
observe the same phenomenon in Score GANs. We illustrate this with an extreme
example in Figure 4, where we intentionally change parameters in our original
model in order to induce the mode collapse issue. We induce mode collapse by
choosing $\mu_{p_{\rm{data}}}=2$ and $\mu_{\rho}=1$ in Equation 25 (i.e., EDM
parameters instead of NCSN parameters in the original model).

Since mode collapse is absent from the generator-less particle models that we
tested (the score models on which Score GANs are based, and also Discriminator
Flows), this observation suggests that mode collapse is primarily caused by
the generator. This is in line with previous theoretical and empirical
findings identifying the generator as the cause of mode collapse (Tanielian et
al., 2020; Durr et al., 2022).

###  B.3 Time Efficiency of Discriminator Flows

(a) Changing the differential equation time discretization of Equation 20 for
Discriminator Flow.

(b) Stopping the generative process of Equation 20 at an earlier time
$T^{\prime}$ than in training ($0<T^{\prime}<T=1$).

Figure 5:  FID performance versus NFE (neural of function evaluations, i.e.,
the number of times ${\nabla}h_{\rho}$ is queried to produce a single image)
for EDM and Discriminator Flow on MNIST. We test different NFE modulation
methods for the Discriminator Flow, in order to evaluate the time efficiency
of each method.

Qualitative experimental results in Section 4 suggested that Discriminator
Flows, by learning a path towards the data distribution with a discriminator,
might be more efficient at inference time than diffusion models for which the
path towards the data distribution is determined by the diffusion equation. We
support this observation by quantitative results in this subsection.

Since Discriminator Flows are based on a discretized differential equation, we
can measure the time efficiency of this model by decreasing the number of
neural function evaluations (NFEs) in two ways: by using a larger time
discretization of Equation 20 (see Figure 5(a)), or by an early stop of the
generative process at time $T^{\prime}<T$ (see Figure 5(b)). We compare time
efficiency of Discriminator Flows against two inference methods for EDM based
on a (first-order) Euler solver (Kloeden & Platen, 1992), and (second-order)
Heun / EDM sampler (Karras et al., 2022). In both cases, when comparing the
two first-order methods, we observe that Discriminator Flows consistently
outperform the Euler-based EDM model.

In Figure 5(a) we evaluate the robustness of Discriminator Flows w.r.t. time
discretization on MNIST. We trained the evaluated model with $128$ generative
steps and $\eta=1$. By increasing the time discretization, in addition to
decreasing the number of NFEs, the discriminator is queried at times not
observed during the training phase, thus measuring the performance of the
discriminator w.r.t. time interpolation. We observe that, even if the
discretized Discriminator Flow is based on a first-order solver of a
differential equation, its behavior is comparable to the (second-order) Heun /
EDM sampler version. Furthermore, compared with the (first-order) Euler solver
based EDM, the Discriminator Flow model shows a performance improvement of an
order of magnitude for low NFEs. For example, with only $13$ NFEs, both
Discriminator Flow and the second-order version of EDM achieve an FID of
approximately $13$, while the first-order version of EDM achieves a FID of
$230$.

In Figure 5(b), we report the performance of Discriminator Flows w.r.t. the
generative process time when stopping the generation early. We trained the
evaluated model with $64$ generative steps and $\eta=2$ , i.e., with half as
many generative steps as the model evaluated in Figure 5(a) but with a doubled
velocity, c.f. Algorithm 3. Again, the Discriminator Flow model, while being
first-order based, suffers a lower drop in performance than the first-order
EMD, and a drop comparable to the second order EDM. Furthermore, we observe
that the first $10$ generative steps of the Discriminator Flows model brings
better FID values than both first- and second- order EDMs.

Overall, the first-order Discriminator Flow performance behavior w.r.t. NFE
is, in both settings, closer to the second-order EDM than the first-order EDM,
which supports our initial claim. We stress that this is achieved while
Discriminator Flows discretize Equation 20 on a regular temporal grid, while
EDM discretize time for both first- and second-order methods using a custom
temporal grid that improves discretization performance.

###  B.4 Experiments on Gaussians

As a toy example, we train all considered models (Discriminator Flows, GANs,
EDM, and Score GANs) on synthetic samples generated from a two-dimensional
mixture of Gaussian distributions. We provide a visualization of the particle
evolution w.r.t. training and inference time in our repository
https://github.com/White-Link/gpm, as well as in Figures 7, 8, 6, 9 and 10.
See the figure captions for more information.

We again observe that for a limited number of NFEs, Discriminator Flows remain
closer to the target distribution than EDM.

##  Appendix C Experimental Details

We provide all details in this section that are necessary to reproduce our
experiments. Our Python source code (tested on version 3.10.4), based on
PyTorch (Paszke et al., 2019) (tested on version 1.13.1), is open-sourced at
https://github.com/White-Link/gpm.

###  C.1 Datasets and Evaluation Metric

##### MNIST.

MNIST is a standard dataset introduced in LeCun et al. (1998), with no clear
license to the best of our knowledge, composed of monochrome images of hand-
written digits. Each MNIST image is single-channel, of size $28\times 28$. We
preprocess MNIST images by extending them to $32\times 32$ frames (padding
each image with black pixels), in order to better fit as inputs and outputs of
standard convolutional networks. We linearly scale pixels values so that they
lie in $\left[-1,1\right]$. MNIST is comprised of a training and testing
dataset, but no validation set; we create one for each model training by
randomly selecting $10\%$ of the training images.

##### CelebA.

CelebA (Liu et al., 2015) is a dataset composed of celebrity pictures. Its
license permits use for non-commercial research purposes. Each CelebA image
has three color channels, and is of size $178\times 218$. We preprocess these
images by center-cropping each to a square image and resizing to $64\times 64$
with a Lanczos filter. We linearly scale pixels values so that they lie in
$\left[-1,1\right]$.

##### Gaussians.

Our Gaussian dataset is composed of a mixture of $5$ two-dimensional Gaussian
distributions with standard deviation $\frac{1}{2}$, with means evenly spaced
over a circle of radius $5$. Training, validation, and testing datasets all
consist of i.i.d. samples from this mixture.

##### FID.

Throughout the paper we use the Fréchet Inception Distance (FID, Heusel et
al., 2017) to measure the generative performance of the models we consider. We
use in our code the PyTorch implementation of TorchMetrics (Skafte Detlefsen
et al., 2022).

###  C.2 Hyperparameters

We summarize the model hyperparameters used during training in Tables 4, 5, 6
and 7. See our code for more information. We further discuss some aspects of
our implementation choices in the remainder of this subsection.

##### Networks.

Beyond multi-layer perceptrons (MLP) and EDM’s U-Nets, we use three kinds of
model architectures: DCGAN (Radford et al., 2016), a ResNet (Kang et al.,
2022) based on SAGAN with self-attention (Zhang et al., 2019), and FastGAN
(Liu et al., 2021). We adapt these models in the following ways:

  * •

For MNIST images, we changed the first (respectively, last) convolution of the
DCGAN discriminator (respectively, generator) to adapt it for $32\times 32$
inputs (respectively, outputs).

  * •

We adapted FastGAN to operate on images of size $32\times 32$ and $64\times
64$ by removing layers with higher resolutions.

  * •

We added a bias to the first convolutional layers of discriminators which did
not have one.

  * •

We enhanced these models so that they can accept a vectorial embedding of time
$t$ for Discriminator flows by modulating most of their convolution outputs
(before the activation) channel-wise with an affine transformation. The affine
transformation parameters are the outputs of a MLP of depth $2$, with SiLU
activations (Hendrycks & Gimpel, 2016) and a hidden width that is twice as
large as the input time embeddings.

##### Final generator activation.

Since image pixel values lie in $\left[-1,1\right]$, we add a final hyperbolic
tangent activation to the output of the generators operating on image data.

##### Score GAN noise distribution $p_{\sigma}$.

Since during training Score GANs mimic the inference procedure of a diffusion
model, we choose to sample $\sigma$ during Score GAN training time following
the schedule chosen by EDM at inference time (Karras et al., 2022). In
practice, we choose a minimal value $\sigma_{\mathrm{min}}$, a maximal value
$\sigma_{\mathrm{max}}$, and an interpolation parameter $\rho$. To sample from
$p_{\sigma}$, we first sample an interpolation value
$\alpha\sim{\mathcal{U}}\left(\left[0,1\right]\right)$, and then compute
$\sigma$ as:

$\sigma=\left(\sigma_{\mathrm{max}}^{\frac{1}{\rho}}+\alpha\left(\sigma_{\mathrm{min}}^{\frac{1}{\rho}}-\sigma_{\mathrm{max}}^{\frac{1}{\rho}}\right)\right)^{\rho}.$
(26)

Note that here $\rho$ denotes a scalar hyperparameter following the EDM
notation, and not the generated distribution as in the main paper.

##### Usual generative modeling tricks.

For simplicity in our proof-of-concept experiments, we avoided using standard
performance improvement tricks such as exponential moving average and
truncation tricks (Brock et al., 2019), EDM sampling tricks (Karras et al.,
2022), or spectral normalization (Miyato et al., 2018).

###  C.3 Compute

For all experiments we use one or two Nvidia V100 GPUs with CUDA 11.8. When
using two V100 Nvidia GPUs, the training time of both Discriminator Flow and
Score GAN models is at most one day for the largest dataset (CelebA).

##  Appendix D Broader Impacts

Our work aims to better understand recent generative modeling methods. As
such, from a practical point of view, our work shares much of the impact of
other work in this domain. While the models we propose do not yet have the
quality required for broad application, generative models have a wide range of
potential positive and negative impacts. Some of the positive impacts include
enabling faster and more accurate natural language processing (Li et al.,
2022), improving automated tasks like summarization (Liu et al., 2018), and
automating certain aspects of content creation (Nichol et al., 2022). However,
generative models are also susceptible to producing undesirable output, such
as unethical text, adversarial attacks (Wang et al., 2021), and malicious
manipulation of data. These models also have the potential to exacerbate
issues such as bias and discrimination in AI systems (Lucy & Bamman, 2021),
enabling the creation of more effective fake text and videos, and expanding
the scope and complexity of cyberattacks that can be launched by malicious
actors (Seymour & Tully, 2018). For a thorough discussion on the potential
dangers of deepfakes, see Fallis (2021).

Discussions and debates around the responsible use of generative models and
the potential broader impacts of deploying them more widely are still ongoing.
We hope that our principled framework aimed at improving our understanding of
generative models will contribute to these discussions and to a better control
of such models.

Table 4:  Chosen hyperparameters for Discriminator Flows for each dataset. GP stands for Gradient Penalty (Gulrajani et al., 2017). IPM stands for Integral Probability Metric (Müller, 1997). BN stands for Batch Normalization (Ioffe & Szegedy, 2015).  Hyperparameters | Gaussians | MNIST | CelebA
---|---|---|---
GAN loss | Non-saturating vanilla | IPM | IPM
$a$ (Equation 15) | $\log\left(1-\mathrm{sigmoid}\right)$ | $\mathrm{id}$ | $\mathrm{id}$
$b$ (Equation 15) | $-\log\mathrm{sigmoid}$ | $\mathrm{id}$ | $\mathrm{id}$
$c$ (Equation 15) | $-\log\mathrm{sigmoid}$ | $\mathrm{id}$ | $\mathrm{id}$
${\mathcal{R}}$ (Equation 15) | $0$ | GP | GP
GP strength | $0$ | $0.04$ | $0.05$
GP center | — | $0$ | $0$
$\pi$ (Equation 1) | ${\mathcal{N}}\left(0,I_{D}\right)$
$\eta$ (Algorithm 3) | $20$ | $\left[2,1\right]$ | $2$
$N$ (Algorithm 3) | $56$ | $\left[64,128\right]$ | $25$
Network architecture | MLP | DCGAN | ResNet
Width | $512$ | $64$ | $64$
Activation | Leaky ReLU, negative slope of $-0.2$
Depth | $4$ | — | —
BN | No
Initialization | Normal | Orthogonal | Normal
Initialization gain | $0.02$ | $1.41$ | $0.02$
Time embedding type | Fourier
Frequency scale | $16$
Time embedding size | $128$
Batch size | $128$ | $128$ | $64$
Number of optimization steps | $4000$ | $200\,000$ | $200\,000$
Optimizer | Adam (Kingma & Ba, 2015)
Learning rate | $0.0002$
$\left(\beta_{1},\beta_{2}\right)$ | $\left(0.5,0.999\right)$
Model selection | — | Best validation FID
Validation frequency | — | $1000$
Number of validation samples | — | $6000$ | $1000$
Table 5:  Chosen hyperparameters for EDM for each dataset. Cf. Karras et al. (2022) and our code for more details.  Hyperparameters | Gaussians | MNIST | CelebA
---|---|---|---
$\sigma_{\mathrm{min}}$ | $0.002$
$\sigma_{\mathrm{max}}$ | $40$
$\sigma_{\mathrm{data}}$ | $0.5$
$\rho$ | $7$
Equation & Solver | Heun solver on the deterministic ODE of Equation 8
Number of solver steps | $7$ | $32$ | $25$
Network architecture | MLP | EDM | EDM
Width | $512$ | $16$ | $128$
Number of residual blocks | — | $1$ | $2$
Dropout | — | $0.13$ | $0.1$
Depth | $4$ | — | —
Activation | Leaky ReLU, slope of $-0.2$ | — | —
Initialization |  Uniform Kaiming for convolutions, Unit weight and zero bias for group normalization layers, otherwise PyTorch default
Time embedding type | Fourier | Positional | Positional
Frequency scale | $16$ | — | —
Time embedding size | $128$ | $256$ | $256$
Batch size | $128$ | $128$ | $64$
Number of optimization steps | $10\,000$ | $500\,000$ | $100\,000$
Optimizer | Adam (Kingma & Ba, 2015)
Learning rate | $0.0002$
$\left(\beta_{1},\beta_{2}\right)$ | $\left(0.9,0.999\right)$
Model selection | —
Table 6:  Chosen hyperparameters for GANs for each dataset. Hinge refers to Hinge GANs (Lim & Ye, 2017).  Hyperparameters | Gaussians | MNIST | CelebA
---|---|---|---
GAN loss | Non-saturating vanilla | Hinge
Number of discriminator steps per generator update | $1$
Latent space size | $128$
$p_{z}$ (Definition 2) | ${\mathcal{N}}\left(0,I_{d}\right)$
Discriminator architecture | MLP | DCGAN | FastGAN
Width | $512$ | $64$ | $32$
Activation | Leaky ReLU, negative slope of $-0.2$
Depth | $4$ | — | —
BN | No | Yes | Yes
Generator architecture | MLP | DCGAN | FastGAN
Width | $512$ | $64$ | $32$
Activation |  Leaky ReLU, slope of $-0.2$ | ReLU |  Leaky ReLU, slope of $-0.2$
Depth | $4$ | — | —
BN | No | Yes | Yes
Initialization | Normal
Initialization gain | $0.02$
Batch size | $128$ | $128$ | $64$
Number of optimization steps | $1900$ | $10\,000$ | $100\,000$
Optimizers | Adam (Kingma & Ba, 2015)
Learning rate | $0.0002$
$\left(\beta_{1},\beta_{2}\right)$ | $\left(0.5,0.999\right)$
Model selection | — | Best validation FID
Validation frequency | — | $1000$
Number of validation samples | — | $6000$ | $1280$
Table 7:  Chosen hyperparameters for Score GANs for each dataset.  Hyperparameters | Gaussians | MNIST | CelebA
---|---|---|---
$\sigma_{\mathrm{min}}$ (Section C.2) | $0.1$ | $0.32$ | $0.32$
$\sigma_{\mathrm{max}}$ (Section C.2) | $10$ | $40$ | $40$
$\rho$ (Section C.2) | $3$
$K$ (Algorithm 1) | $10$ | $1$ | $4$
Latent space size | $128$
$p_{z}$ (Definition 2) | ${\mathcal{N}}\left(0,I_{d}\right)$
Data score $s_{\psi}^{p}{}_{\rm{data}}$ | EDM from Table 5
Gen. score $s_{\phi}^{\rho}$ architecture | MLP | EDM | EDM
$\sigma_{\mathrm{data}}$ | $0.5$
Width | $512$ | $64$ | $128$
Number of residual blocks | — | $2$ | $2$
Dropout | — | $0.13$ | $0$
Depth | $4$ | — | —
Activation |  Leaky ReLU, slope of $-0.2$ | — | —
Initialization | EDM from Table 5
Generator architecture | MLP | DCGAN | FastGAN
Width | $512$ | $64$ | $32$
Activation | ReLU
Depth | $4$ | — | —
BN | No | Yes | Yes
Initialization | PyTorch default | Normal | Orthogonal
Initialization gain | — | $0.02$ | $1.41$
Batch size | $128$ | $256$ | $32$
Number of generator optimization steps | $3150$ | $100\,000$ | $150\,000$
Optimizers | Adam (Kingma & Ba, 2015)
Score learning rate | $0.0002$ | $0.001$ | $0.0004$
Generator learning rate | $0.0002$
$\left(\beta_{1},\beta_{2}\right)$ | $\left(0.9,0.999\right)$
Model selection | — | Best validation FID | —
Validation frequency | — | $2500$ | —
Number of validation samples | — | $6000$ | —

(a) Initialization (sampling from $\rho_{0}=\pi$). (b) $t=\frac{1}{7}T$. (c)
$t=\frac{2}{7}T$. (d) $t=\frac{3}{7}T$. (e) $t=\frac{4}{7}T$. (f)
$t=\frac{5}{7}T$. (g) $t=\frac{6}{7}T$. (h) $t=T$.

Figure 6:  Sampling steps for EDM on a Gaussian mixture; $128$ samples are
shown for both the generated (⚫) and the data (◆) distributions. The second-
order solver was used with $13$ NFEs. Arrows show the gradients
${\nabla}h_{\rho_{t}}$ associated with each generated sample, corresponding to
the direction provided by the score function, cf. Equation 8.

(a) Initialization (sampling from $\rho_{0}=\pi$). (b) $t=\frac{1}{7}T$. (c)
$t=\frac{2}{7}T$. (d) [$t=\frac{3}{7}T$. (e) $t=\frac{4}{7}T$. (f)
$t=\frac{5}{7}T$. (g) $t=\frac{6}{7}T$. (h) $t=T$.

Figure 7:  Sampling steps for Discriminator Flows on a Gaussian mixture; cf.
Figure 6. Our chosen discretization yields $14$ NFEs. The colored background
represents the pointwise generator loss function $-h_{\rho_{t}}=-c\circ
f_{\rho_{t}}$ (darker is lower, renormalized for every snapshot), from which
the particle gradients, shown in the figures, are derived.

(a) Initialization ($\rho_{0}=g_{\theta_{0}}\sharp p_{z}$). (b)
$t=\frac{1}{7}T$. (c) $t=\frac{2}{7}T$. (d) $t=\frac{3}{7}T$. (e)
$t=\frac{4}{7}T$. (f) $t=\frac{5}{7}T$. (g) $t=\frac{6}{7}T$. (h) $t=T$.

Figure 8:  Training snapshots of a GAN on a Gaussian mixture; cf. Figure 6.
Here time $t$ represents training time, and $T$ is the end of training. The
colored backgrounds represent the pointwise generator loss function
$-h_{\rho_{t}}=-c\circ f_{\rho_{t}}$ (darker is lower, renormalized for every
snapshot), from which the particle gradients, shown in the figures, are
derived and then fed to the generator following Equations 11 and 13.

(a) Initialization ($\rho_{0}=g_{\theta_{0}}\sharp p_{z}$). (b)
$t=\frac{1}{7}T$. (c) $t=\frac{2}{7}T$. (d) $t=\frac{3}{7}T$. (e)
$t=\frac{4}{7}T$. (f) $t=\frac{5}{7}T$. (g) $t=\frac{6}{7}T$. (h) $t=T$.

Figure 9:  Training snapshots of a Score GAN on a Gaussian mixture; cf. Figure
6. Here time $t$ represents training time, and $T$ is the end of training.

(a) Initialization ($\rho_{0}=g_{\theta_{0}}\sharp p_{z}$). (b)
$t=\frac{1}{7}T$. (c) $t=\frac{2}{7}T$. (d) $t=\frac{3}{7}T$. (e)
$t=\frac{4}{7}T$. (f) $t=\frac{5}{7}T$. (g) $t=\frac{6}{7}T$. (h) $t=T$.

Figure 10:  Training snapshots of a Score GAN on a Gaussian mixture identical
to Figure 9, but with generated samples perturbed by Gaussian noise of
standard deviation $\sigma=0.2$. Arrows show the gradients
${\nabla}h_{\rho_{t}}$ received by the generated particles at this noise
level, corresponding to Equation 19, split into the data score (in blue) and
minus the score of the generated distribution (in yellow). They are then fed
to the generator following Equations 11 and 13.