#  Evaluating Generative Adversarial Networks on Explicitly Parameterized
Distributions

Shayne O’Brien, Matt Groh, Abhimanyu Dubey
{shayneob, groh<EMAIL_ADDRESS>
Massachusetts Institute of Technology

###### Abstract

> The true distribution parameterizations of commonly used image datasets are
> inaccessible. Rather than designing metrics for feature spaces with unknown
> characteristics, we propose to measure GAN performance by evaluating on
> explicitly parameterized, synthetic data distributions. As a case study, we
> examine the performance of 16 GAN variants on six multivariate distributions
> of varying dimensionalities and training set sizes. In this learning
> environment, we observe that: GANs exhibit similar performance trends across
> dimensionalities; learning depends on the underlying distribution and its
> complexity; the number of training samples can have a large impact on
> performance; evaluation and relative comparisons are metric-dependent;
> diverse sets of hyperparameters can produce a “best” result; and some GANs
> are more robust to hyperparameter changes than others. These observations
> both corroborate findings of previous GAN evaluation studies and make novel
> contributions regarding the relationship between size, complexity, and GAN
> performance.

##  1 Introduction

Generative adversarial network (GAN) optimization stability and convergence
properties remain poorly understood despite the introduction of hundreds of
GAN variants since their conception [8, 11]. While GAN learning and
performance behavior has been studied [10, 19, 23], most existing work
examining this relationship focuses on image datasets for which the underlying
distribution parameterization is inaccessible [1, 2, 4, 18, 25]. This is
problematic since claims of behavior that are made by modeling an unknown
target distribution require a strong assumption for generalizability.

The goal of generative modeling is to approximate a distribution $p_{d}$ by
learning a parameterized distribution $p_{g}$, where both $p_{d}$ and $p_{g}$
are defined over samples. If we do not have full access to $p_{d}$,
generalizability requires us to assume that the modeled dataset is a
reasonable proxy for the family of distributions from which it was sampled.
Without this assumption that is often only implicitly made, using images to
understand GAN behavior limits conclusions to the data context being modeled.

We seek to address a gap in the literature by investigating GAN variant
performance on datasets for which we have full access to the distribution
parameterization. This allows us to study empirical performance on data where
we can make claims of model behavior that generalize to the full distribution,
as opposed to on image datasets for which this is not necessarily true. To
this end, we examine the performance of 16 GAN variants on six explicitly
parameterized multivariate distributions of four different dimensionalities
and three different training set sizes.

Across 20 grid search trials, we observe that: (1) GANs exhibit similar
performance trends across dimensionalities, (2) learning depends on the
underlying distribution and its complexity, (3) the number of training samples
can have a large impact on performance, (4) evaluation and relative
comparisons are metric-dependent, (5) diverse sets of hyperparameters can
produce a “best” result, and (6) some GANs are more robust to hyperparameter
changes than others. These findings corroborate those of previous GAN
evaluation studies as well as contribute novel insights regarding the
relationship between size, complexity, and GAN performance.111All code is
publicly available at https://github.com/shayneobrien/explicit-gan-eval.

##  2 Related Work

One notable work in this area by Lucic et al. [19] compares seven GAN variants
in terms of modeling ability and optimization stability. The authors find that
as computational budget increases, all tested models reach similar Frechét
Inception Distance on the MNIST, Fashion-MNIST, CIFAR10, and CelebA datasets;
and F1, precision, and recall on a synthetic dataset of convex polygons. They
also discuss the difficulties of comparing GANs due to multiple valid ways to
analyze performance.

Santurkar et al. [24] measure Inception Score and classification accuracy and
report that the five GAN variants they train do not succeed at capturing
distributional properties of the training set on the CelebA and LSUN datasets.
The authors observe that the GAN distributions exhibit significantly less
diversity at test time compared to the evaluation dataset, suggesting $p_{g}$
is far from $p_{d}$.

In another study, Im et al. [13] evaluate GAN variant performance based on the
original GAN criterion, least squares, maximum mean discrepancy, and improved
Wasserstein distance. They show that for the three GAN variants they consider,
test-time metrics do not favor networks that use the same training-time
criterion on the MNIST, CIFAR10, LSUN, and Fashion-MNIST image datasets. The
authors also examine performance as a function of sample size and show that
some GANs exhibit faster performance increases than others as the number of
training samples increases.

Lastly, Borji [4] provide a thorough discussion of the strengths and
weaknesses of 26 quantitative and qualitative measures used for evaluating
GANs trained on image datasets. They conclude that there is no single, best
GAN evaluation measure. The authors suggest benchmarking models under
identical architectures and computational budgets, and using more than a
single metric to make comparisons.

##  3 Experimental Setup

In GANs, we define a prior probability distribution on input noise variables
$p_{z}(\mathbf{z})$ and represent a mapping to the target data space
$p_{d}(\mathbf{x})$ as $G(\mathbf{z},\theta_{G})$, where $G$ is a fully
differentiable neural network called the generator and $\theta_{G}$ are its
parameters. We train $G$ by simultaneously learning a fully differentiable
network $D$, called the discriminator or critic and defined by
$D(\mathbf{x},\theta_{D})$, that helps $G$ during training. Whereas $G$ is
trained to mimic $p_{d}$, the learning objective, output, and precise task of
$D$ vary depending upon the GAN variant.

### Models

As a case study, we examine the same seven GAN variants evaluated by Lucic et
al. [19] and nine additional GAN variants that have been popularly discussed
since their study was published. The primary difference between considered
variants is whether the discriminator output can be interpreted as a
probability (MMGAN, NSGAN [8], RaGAN [14], DRAGAN [16], FisherGAN [21],
InfoGAN [5], ForwGAN, RevGAN, HellingerGAN, PearsonGAN, JSGAN [22]) or is
unbounded (WGAN [1], WGANGP [9], LSGAN [20], BEGAN [3]). We summarize these
models in Table 1.

GAN Variant Loss Functions |
---|---
$\mathcal{L}^{\textnormal{MMGAN}}$ = $\mathbb{E}$[$\log(D(\mathbf{x}))]+\mathbb{E}$[$\log(1-D(G(\mathbf{z})))]$ |  $\mathcal{L}^{\textnormal{RaGAN}}$ = $\mathbb{E}$[$\log(D(\mathbf{x})-D(G(\mathbf{z})))]+\mathbb{E}[\log(1-(D(G(\mathbf{z}))-D(\mathbf{x})))]$
$\mathcal{L}^{\textnormal{NSGAN}}$ = $\mathbb{E}$[$\log(D(\mathbf{x}))]-\mathbb{E}$[$\log(D(G(\mathbf{z})))]$ |  $\mathcal{L}^{\textnormal{LSGAN}}$ = $-\mathbb{E}[(D(\mathbf{x})-1)^{2}]+\mathbb{E}[D(G(\mathbf{z}))^{2}]$
$\mathcal{L}^{\textnormal{WGAN}}$ = $-\mathbb{E}$[$D(\mathbf{x})]+\mathbb{E}$[$D(G(\mathbf{z}))]$ |  $\mathcal{L}^{\textnormal{BEGAN}}$ = $\mathbb{E}[\|\mathbf{x}-D_{\textnormal{AE}}(\mathbf{x})\|_{1}]-k_{t}\mathbb{E}[\|G(\mathbf{z})-D_{\textnormal{AE}}(G(\mathbf{z}))\|_{1}]$
$\mathcal{L}^{\textnormal{WGANGP}}$ = $\mathcal{L}^{\textnormal{WGAN}}+\lambda\mathbb{E}$ [($\|\nabla_{\mathbf{z}}D(G(\mathbf{z}))\|_{2}-1)^{2}]$ |  $\mathcal{L}^{\textnormal{DRAGAN}}$ = $\mathcal{L}^{\textnormal{MMGAN}}+\lambda\mathbb{E}[(\|\nabla_{\mathbf{x}}D(\mathbf{x}+\delta))\|_{2}-1)^{2}]]$
$\mathcal{L}^{\textnormal{FisherGAN}}$ = $\mathcal{L}^{\textnormal{WGAN}}+\lambda(1-\hat{\Omega}(D,G))-\frac{\rho}{2}(\hat{\Omega}(D,G)-1)$ |  $\mathcal{L}^{\textnormal{InfoGAN}}$ = $\mathcal{L}^{\textnormal{MMGAN}}$ \- $\lambda(\mathbb{E}[\log(Q(\mathbf{c}^{\prime}|\mathbf{x}))])$
$\mathcal{L}^{\textnormal{PearsonGAN}}$ = $\mathbb{E}[D(\mathbf{x})]+\mathbb{E}[\frac{1}{4}D(G(\mathbf{z}))^{2}+D(G(\mathbf{z}))]$ |  $\mathcal{L}^{\textnormal{TVGAN}}$ = $-\frac{1}{2}\mathbb{E}[\tanh(D(\mathbf{x}))]+\frac{1}{2}\mathbb{E}[\tanh(D(G(\mathbf{z})))]$
$\mathcal{L}^{\textnormal{ForwGAN}}$ = $\mathbb{E}[D(\mathbf{x})]+\mathbb{E}[\textnormal{exp}(D(G(\mathbf{z})))-1]$ |  $\mathcal{L}^{\textnormal{RevGAN}}$ = $\mathbb{E}[-\textnormal{exp}(D(\mathbf{x}))]+\mathbb{E}[-1-(D(G(\mathbf{z})))]$
$\mathcal{L}^{\textnormal{HellingerGAN}}$ = $\mathbb{E}[1-\textnormal{exp}(-D(\mathbf{x}))]+\mathbb{E}[\frac{1-\textnormal{exp}(D(G(\mathbf{z})))}{\textnormal{exp}(D(G(\mathbf{z})))}]$ |  $\mathcal{L}^{\textnormal{JSGAN}}$ = $\mathbb{E}[2-(1+\textnormal{exp}(-D(\mathbf{x})))]-\mathbb{E}[2-\textnormal{exp}(D(G(\mathbf{z})))]$

Table 1:  The loss function for each GAN variant with slight abuse of
parameterization notation on the expectations, $G$, and $D$. Note that $G$ is
parameterized by $\theta_{G}$, $D$ is parameterized by $\theta_{D}$, $x\sim
p_{d}$, $z\sim p_{g}$, $\delta\sim\mathcal{N}(0,cI)$,
$\hat{\Omega}(D,G)=\frac{1}{2}\mathbb{E}[D(\mathbf{x})]^{2}-\frac{1}{2}\mathbb{E}[D(G(\mathbf{z}))]^{2}$,
$D_{AE}$ indicates that $D$ is an autoencoder,
$\mathbf{c}=[\mathbf{c_{1}},\mathbf{c_{2}}]$ are structured latent variables
where $\mathbf{c}^{\prime}$ is sampled from the approximated distribution
$p_{\mathbf{c}}(\mathbf{c}|\mathbf{x})$, $\nabla_{(\mathbf{\cdot})}$ is the
gradient of the loss with respect to $(\mathbf{\cdot})$, and $k_{t}$,
$\lambda$, and $\rho$ are introduced hyperparameters.

In our implementations, both $D$ and $G$ consist of two feedforward network
layers each; the full architecture has four layers total. We apply a ReLU
activation function to the output of each layer and sample the noise prior
$\mathbf{z}$ from $\mathcal{N}(0,\frac{h}{4}I)$, where $h$ is the hidden
dimension size. All models have the same number of trainable parameters except
InfoGAN and BEGAN due to their use of latent variables as inputs to $D$ and
formation of $D$ as an autoencoder, respectively. Chen et al. [5] argue that
this difference is negligible for InfoGAN and we do not observe that it gives
BEGAN any tangible advantage over other models. Trainable parameter counts can
be found in Table 8.

### Data

We train each of these variants by randomly sampling 1,000, 10,000, and
100,000 data points from the following six explicitly parameterized
multivariate distributions: Gaussian with mean $\mathbf{\mu}$ and symmetric,
full rank covariance $\mathbf{\Sigma}$ both from $[0,1]$; exponential with
inverse mean shape $\mathbf{\lambda}$ from $[0,1]$; beta with shape parameters
$a$ and $\beta$ both from $[0,1]$; gamma with shape $k$ from $[0,10]$ and
scale $\theta$ from $[0,2]$; Gumbel with location $\mu$ and scale $\beta$ both
from $[0,1]$; and Laplace with location $\mu$ and scale $\beta$ both from
$[0,1]$. For each of these distributions and numbers of samples, we generate
datasets of 16, 32, 64, and 128 dimensions. We note that by the Universal
Approximation Theorem, our proposed network architecture should be able to
model each of these distributions without exception.

### Hyperparameters

For all models and data distributions, we conduct 20 grid search trials with
random network initializations for learning rates
$\gamma\in[2\text{e}^{-1},2\text{e}^{-2},2\text{e}^{-3}]$, hidden dimension
sizes $h\in[32,64,128,256,512]$, and batch size $b=1024$.222We also ran full
experiments for $b\in[128,256,512]$, but limit our analyses to $b$ = 1024 as
results across different batch sizes are not comparable due to greater noise
in the data generation process at lower values of $b$. For models with
introduced hyperparameters, we use those given in the original the paper. We
use the Adam optimizer with default settings [15] and train for 25 epochs.

### Measuring Divergence

We evaluate the difference between $p_{d}$ and $p_{g}$ using Kullback-Leibler
divergence (KL), Jensen-Shannon divergence (JS), and Wasserstein Distance
(WD). KL and JS focus on the alignment of the modes of the distributions and
WD emphasizes how much $p_{g}$ must be modified to reach $p_{d}$. Whereas JS
and WD are symmetric, KL is not. For any of these measures, a value of 0 can
be interpreted as indicating the two distributions being compared are
identical [6, 17, 26]. We report results as the divergence between a generated
batch and a test batch of size $b=1024$ at the end of every epoch.

### Estimating $p_{g}$

Although we have access to the true data distribution $p_{d}$, we must
estimate the probability distribution of $p_{g}$. Since the data
dimensionality is low, we construct a dimension-wise histogram for each data
point.333Kernel density estimation was found to give similar outputs while
being more computationally expensive. In doing so, we assume that each
dimension is independent from the others. This assumption is valid in the case
of all experiments involving non-Gaussian data, which follows from the
multivariate model being a product of the marginal distributions. To select
the optimal bin width $B_{w}$ in the histogram, we follow the Freedman-
Diaconis rule: $B_{w}=\frac{2\cdot IQR({\bf\tilde{x}})}{\sqrt[3]{M}}$, where
$IQR$ is the inter-quartile range of the $M$ samples
${\bf\tilde{x}}=\\{x_{1},...,x_{M}\\}$ from the distribution being
approximated. This initialization minimizes the difference between the areas
under the empirical and theoretical probability distributions [7].

##  4 Results

In our analyses, we take the same approach as Lucic et al. [19] and Im et al.
[13]: we let the “best” hyperparameter setting be the one that achieved the
lowest minimum performance on average across all trials for each distribution,
metric, and number of training samples, respectively. We include results,
visualizations, and evidence to support all conclusions in the appendices. For
the best hyperparameter settings in our learning environment, we find that:

  1. 1.

GANs exhibit similar learning trends across dimensionalities: For many of the
models, performance under the best hyperparameter setting consistently follows
a trend across dimensionalities for all three tested metrics. At the same
time, performance generally worsens with increased dimensionality. See Figures
1, 2, 3.

  2. 2.

Learning depends on the underlying distribution and its complexity: Models
which do well on some distributions perform poorly on the same distribution
with higher dimensionality, or on other distributions of the same
dimensionality. It is not immediately apparent that these differences are due
to model design. Lucic et al. [19] make a similar finding in the case of image
datasets with varying complexities. See Figures 1, 2, 3.

  3. 3.

Number of training samples can have a large impact on performance: Some GAN
variants are able to achieve the same performance learning from 1,000 samples
as 10,000 or 100,000 samples, while others show large performance jumps with
increased amounts of data. At the same time, almost all GAN variants begin to
worsen in performance within five epochs for 1,000 training samples. The
number of training samples seems to be critical to some models’ performances,
which was also noted by Im et al. [13]. See Figures 4, 5, and 6.

  4. 4.

Evaluation and comparison are metric-dependent: Relative ranking of GAN
variants according to performance varies depending on the evaluation metric
used to rank them. No single GAN performed best across all metrics for any
dataset or dimensionality. We concur with previous studies that GANs generally
perform the same, although there are variants that perform worse than others
on some distributions [4, 12, 13, 19, 24]. We warn against ranking models as
relative differences can be marginal. See Tables 2, 3, 4, and 5.

  5. 5.

Diverse sets of hyperparameters can produce a “best” result: Many, diverse
hyperparameter settings yielded superior performances to the best average
minimum performance, but these models did not achieve those minima with tight
confidence bounds. Furthermore, we see that even on the best performing
hyperparameter settings, our tested models preferred widely different hidden
dimensionalities and learning rates; some variants with less parameters
outperformed others that had more. We agree with previous work that it is
important to present results that are able to be consistently reproduced [4,
13, 19, 18, 25]. See Table 6.

  6. 6.

Some GANs are more robust to hyperparameter changes than others: With respect
to the distribution, dimensionality, and training set size being approximated,
some models yielded average minimum performances for more hyperparameters than
others that fell within the confidence interval of the best average minimum
performance under consideration. This is an indication that some GANs can
perform well under a greater range of hyperparameter settings than others. See
Table 7.

##  5 Future Work

In future work, we plan to analyze cases where GAN variants underperform
relative to others and relate the characteristics of the distribution being
modeled to the assumptions made in designing the variant, e.g. by empirically
considering whether a normally distributed prior hurts performance on non-
normal distributions. We would also like to use longer training times and more
complex models to evaluate additional synthetic datasets such as multivariate
mixture models, colored circles, and autoencoded image datasets.

## Acknowledgements

We would like to thank Deb Roy, Iyad Rahwan, Manolis Zampetakis, and the Media
Lab Consortium for their support of this research. Aleksander Mądry and Costis
Daskalakis provided us with inspiration to pursue this project. Shayne O’Brien
is partially funded through a National Science Foundation Graduate Research
Fellowship.

## References

  * Arjovsky et al. [2017] Martín Arjovsky, Soumith Chintala, and Léon Bottou.  Wasserstein generative adversarial networks.  In _International Conference on Machine Learning_ , pages 214–223, 2017.
  * Arora et al. [2017] Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang.  Generalization and equilibrium in generative adversarial nets (GANs).  In _Proceedings of the 34th International Conference on Machine Learning_ , pages 224–232, 2017.
  * Bethelot et al. [2017] David Bethelot, Thomas Schumm, and Luke Metz.  BEGAN: Boundary equilibrium generative adversarial networks.  _arXiv preprint arXiv:1703.10717_ , 2017.
  * Borji [2018] Ali Borji.  Pros and cons of GAN evaluation measures.  _CoRR_ , abs/1802.03446, 2018.  URL http://arxiv.org/abs/1802.03446.
  * Chen et al. [2016] Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel.  InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets.  In _Advances in Neural Information Processing Systems_ , pages 2172–2180, 2016.
  * Endres and Schindelin [2003] Dominik Endres and Johannes Schindelin.  A new metric for probability distributions.  In _IEEE Transactions on Information Theory_ , pages 1858–1860, 2003\.
  * Freedman and Diaconis [1981] David Freedman and Persi Diaconis.  On the histogram as a density estimator: L2 theory.  _Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete_ , pages 453–476, 1981.
  * Goodfellow et al. [2014] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial nets.  In _Advances in neural information processing systems_ , pages 2672–2680, 2014.
  * Gulrajani et al. [2017] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville.  Improved training of wasserstein GANs.  In _Advances in Neural Information Processing Systems_ , pages 5767–5777. 2017.
  * Heusel et al. [2017] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Günter Klambauer, and Sepp Hochreiter.  GANs trained by a two time-scale update rule converge to a Nash equilibrium.  _arXiv preprint arXiv:1706.08500_ , 2017.
  * Hindupur [2017] Avinash Hindupur.  The GAN zoo.  _GitHub Repository_ , 2017.  URL https://github.com/hindupuravinash/the-gan-zoo.
  * Huang et al. [2018] Gao Huang, Yang Yuan, Qiantong Xu, Chuan Guo, Yu Sun, Felix Wu, and Kilian Weinberger.  An empirical study on evaluation metrics of generative adversarial networks.  2018\.  URL https://openreview.net/forum?id=Sy1f0e-R-.
  * Im et al. [2018] Daniel Jiwoong Im, Alllan He Ma, Graham W. Taylor, and Kristin Branson.  Quantitatively evaluating GANs with divergences proposed for training.  In _International Conference on Learning Representations_ , 2018.  URL https://openreview.net/forum?id=SJQHjzZ0-.
  * Jolicoeur-Martineau [2018] Alexia Jolicoeur-Martineau.  The relativistic discriminator: A key element missing from standard GAN.  _arXiv preprint arXiv:1807.00734_ , 2018.
  * Kingma and Ba [2015] Diederik Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _International Conference on Learning Representations_ , 2015.
  * Kodali et al. [2017] Naveen Kodali, Jacob Abernethy, James Hays, and Kira Zsolt.  On convergence and stability of GANs.  _arXiv preprint arXiv:1705.07215_ , 2017.
  * Kullback and Leibler [1951] Soloman Kullback and Richard Leibler.  On information and sufficiency.  pages 79–86, 1951.
  * Lee et al. [2017] Holden Lee, Rong Ge, Tengyu Ma, Andrej Risteski, and Sanjeev Arora.  On the ability of neural nets to express distributions.  In _Journal of Machine Learning Research_ , pages 1–26, 2017.
  * Lucic et al. [2017] Mario Lucic, Karol Kurach, Marcin Michalski, Sylvain Gelly, and Olivier Bousquet.  Are GANs created equal? A large-scale study.  _arXiv preprint arXiv:1711.10337_ , 2017.
  * Mao et al. [2017] Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley.  Least squares generative adversarial networks.  In _2017 IEEE International Conference on Computer Vision (ICCV)_ , pages 2813–2821, 2017.
  * Mroueh and Sercu [2017] Youssef Mroueh and Tom Sercu.  Fisher GAN.  In _Advances in Neural Information Processing Systems_ , pages 3310–3320, 2017.
  * Nowozin et al. [2016] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka.  f-GAN: Training generative neural samplers using variational divergence minimization.  In _Advances in Neural Information Processing Systems_ , pages 271–279. 2016.
  * Salimans et al. [2016] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen.  Improved techniques for training GANs.  In _Advances in Neural Information Processing Systems_ , pages 2234–2242, 2016.
  * Santurkar et al. [2017] Shibani Santurkar, Ludwig Schmidt, and Aleksander Madry.  A classification-based perspective on GAN distributions.  _arXiv preprint arXiv: 1711.00970_ , 2017.
  * Theis et al. [2015] Lucas Theis, Aäron van den Oord, and Matthias Bethge.  A note on the evaluation of generative models.  _arXiv preprint arXiv:1511.01844_ , 2015.
  * Villani [2008] Cédric Villani.  _Optimal Transport: Old and New_ , volume 338.  Springer Science & Business Media, 2008.

##  Appendix A GANs exhibit similar learning trends across dimensionalities,
and learning depends on the underlying distribution and its complexity

Figure 1: Performance of GAN variants for their best hyperparameter settings,
respectively, trained on 100,000 samples across normal, beta, Gumbel, Laplace,
exponential, and gamma multivariate distributions of dimensionalities $N=16$
(rows 1 and 2), $N=32$ (rows 3 and 4), $N=64$ (rows 5 and 6), and $N=128$
(rows 7 and 8). Plots display metric performance as a function of epoch.
Shaded areas represent the region of the 95% confidence interval of the
respective model computed over 20 trials. “Expected” indicates the empirical
average divergence of a generated batch of size $b=1024$ for the given
distribution across 20 trials where samples are generated independently, i.e.
one at a time. Best viewed in color. Figure 2: Performance of GAN variants for
their best hyperparameter settings, respectively, trained on 10,000 samples
across normal, beta, Gumbel, Laplace, exponential, and gamma multivariate
distributions of dimensionalities $N=16$ (rows 1 and 2), $N=32$ (rows 3 and
4), $N=64$ (rows 5 and 6), and $N=128$ (rows 7 and 8). Plots display metric
performance as a function of epoch. Shaded areas represent the region of the
95% confidence interval of the respective model computed over 20 trials.
“Expected” indicates the empirical average divergence of a generated batch of
size $b=1024$ for the given distribution across 20 trials where samples are
generated independently, i.e. one at a time. Best viewed in color. Figure 3:
Performance of GAN variants for their best hyperparameter settings,
respectively, trained on 1,000 samples across normal, beta, Gumbel, Laplace,
exponential, and gamma multivariate distributions of dimensionalities $N=16$
(rows 1 and 2), $N=32$ (rows 3 and 4), $N=64$ (rows 5 and 6), and $N=128$
(rows 7 and 8). Plots display metric performance as a function of epoch.
Shaded areas represent the region of the 95% confidence interval of the
respective model computed over 20 trials. “Expected” indicates the empirical
average divergence of a generated batch of size $b=1024$ for the given
distribution across 20 trials where samples are generated independently, i.e.
one at a time. Best viewed in color.

##  Appendix B Number of training samples can have a large impact on
performance

Figure 4: Confidence intervals of the Jensen-Shannon divergence performances
for the best performing hyperparameter over 20 trials as a function of sample
size for dimensionalities $N=16$ (rows 1 and 2), $N=32$ (rows 3 and 4), $N=64$
(rows 5 and 6), and $N=128$ (rows 7 and 8). “Expected” indicates the empirical
average divergence of a generated batch of size $b=1024$ for the given
distribution would be across 20 trials where samples are generated
independently, i.e. one at a time. Best viewed in color. Figure 5: Confidence
intervals of the Kullback-Leibler divergence performances for the best
performing hyperparameter over 20 trials as a function of sample size for
dimensionalities $N=16$ (rows 1 and 2), $N=32$ (rows 3 and 4), $N=64$ (rows 5
and 6), and $N=128$ (rows 7 and 8). “Expected” indicates the empirical average
divergence of a generated batch of size $b=1024$ for the given distribution
would be across 20 trials where samples are generated independently, i.e. one
at a time. Best viewed in color. Figure 6: Confidence intervals of the
Wasserstein Distance performances for the best performing hyperparameter over
20 trials as a function of sample size for dimensionalities $N=16$ (rows 1 and
2), $N=32$ (rows 3 and 4), $N=64$ (rows 5 and 6), and $N=128$ (rows 7 and 8).
“Expected” indicates the empirical average divergence of a generated batch of
size $b=1024$ for the given distribution would be across 20 trials where
samples are generated independently, i.e. one at a time. Best viewed in color.

##  Appendix C Evaluation and comparisons are metric-dependent

Model | Normal | Beta | Gumbel | Laplace | Exponential | Gamma
---|---|---|---|---|---|---
WGAN | [7, 7, 13] | [0, 0, 5] | [0, 0, 1] | [5, 7, 13] | [0, 0, 3] | [0, 0, 2]
| [8, 9, 10] | [0, 1, 4] | [9, 5, 7] | [9, 9, 9] | [1, 3, 4] | [0, 0, 1]
| [7, 2, 11] | [0, 0, 4] | [1, 2, 9] | [8, 8, 11] | [3, 6, 4] | [5, 0, 2]
WGANGP | [8, 10, 14] | [13, 2, 0] | [13, 8, 4] | [9, 9, 14] | [13, 1, 0] | [9, 1, 0]
| [9, 10, 11] | [11, 3, 1] | [13, 12, 9] | [12, 11, 10] | [12, 4, 1] | [9, 4, 0]
| [9, 9, 0] | [2, 1, 1] | [8, 11, 3] | [10, 10, 2] | [9, 4, 3] | [10, 3, 0]
NSGAN | [4, 6, 6] | [6, 3, 4] | [2, 4, 6] | [2, 8, 5] | [5, 4, 6] | [6, 2, 3]
| [5, 1, 5] | [7, 2, 3] | [1, 4, 4] | [5, 4, 5] | [2, 0, 5] | [7, 8, 5]
| [4, 7, 10] | [10, 4, 6] | [5, 6, 1] | [4, 9, 3] | [4, 5, 10] | [6, 4, 4]
MMGAN | [14, 14, 11] | [7, 5, 6] | [12, 11, 7] | [14, 13, 11] | [3, 5, 4] | [13, 5, 4]
| [14, 13, 12] | [10, 6, 5] | [12, 9, 10] | [13, 13, 13] | [14, 11, 6] | [14, 9, 2]
| [14, 13, 12] | [14, 5, 5] | [14, 4, 2] | [14, 11, 5] | [14, 13, 12] | [13, 12, 1]
RAGAN | [10, 8, 7] | [11, 13, 14] | [8, 9, 13] | [10, 10, 9] | [6, 13, 14] | [10, 9, 14]
| [6, 7, 7] | [12, 13, 14] | [7, 8, 8] | [6, 8, 7] | [11, 14, 14] | [10, 6, 11]
| [8, 10, 13] | [8, 15, 13] | [4, 12, 10] | [7, 4, 13] | [2, 8, 13] | [0, 6, 12]
LSGAN | [9, 12, 8] | [9, 4, 2] | [9, 10, 8] | [6, 12, 8] | [2, 2, 1] | [14, 7, 1]
| [10, 12, 9] | [9, 4, 0] | [11, 11, 11] | [8, 12, 11] | [13, 10, 3] | [12, 11, 3]
| [11, 8, 1] | [11, 3, 3] | [11, 8, 0] | [11, 14, 0] | [11, 12, 7] | [12, 13, 5]
DRAGAN | [13, 13, 10] | [14, 14, 1] | [14, 15, 11] | [13, 15, 10] | [14, 14, 2] | [12, 14, 11]
| [13, 14, 13] | [14, 14, 10] | [14, 15, 15] | [14, 15, 15] | [10, 13, 10] | [11, 14, 14]
| [13, 14, 7] | [5, 7, 0] | [13, 14, 4] | [13, 13, 10] | [13, 14, 0] | [14, 14, 6]
BEGAN | [15, 15, 15] | [15, 15, 15] | [15, 14, 15] | [15, 14, 15] | [15, 15, 15] | [15, 15, 15]
| [15, 15, 15] | [15, 15, 15] | [15, 14, 14] | [15, 14, 12] | [15, 15, 15] | [15, 15, 15]
| [15, 15, 15] | [15, 14, 14] | [15, 15, 15] | [15, 15, 15] | [15, 15, 15] | [15, 15, 15]
InfoGAN | [11, 9, 9] | [1, 1, 3] | [10, 12, 14] | [11, 6, 6] | [1, 3, 5] | [4, 3, 5]
| [11, 8, 8] | [1, 0, 2] | [10, 10, 13] | [10, 5, 8] | [6, 1, 0] | [8, 1, 7]
| [10, 11, 4] | [1, 2, 2] | [9, 7, 7] | [9, 5, 8] | [7, 7, 8] | [11, 10, 3]
FisherGAN | [5, 2, 5] | [4, 8, 8] | [5, 5, 5] | [3, 4, 4] | [10, 8, 11] | [5, 10, 7]
| [2, 4, 4] | [5, 10, 7] | [2, 6, 5] | [2, 6, 1] | [4, 2, 8] | [3, 5, 10]
| [5, 5, 9] | [7, 12, 7] | [0, 0, 11] | [6, 7, 6] | [8, 9, 11] | [8, 5, 9]
ForwGAN | [0, 3, 0] | [5, 9, 10] | [3, 1, 0] | [4, 3, 2] | [12, 6, 7] | [1, 6, 8]
| [0, 0, 6] | [3, 8, 11] | [4, 7, 1] | [3, 1, 3] | [8, 9, 9] | [4, 7, 8]
| [0, 1, 3] | [3, 8, 10] | [7, 10, 8] | [2, 2, 1] | [10, 3, 5] | [7, 8, 8]
RevGAN | [2, 4, 2] | [2, 10, 9] | [4, 6, 2] | [1, 0, 1] | [4, 9, 12] | [2, 4, 12]
| [4, 3, 1] | [4, 7, 6] | [5, 3, 2] | [1, 2, 2] | [7, 5, 13] | [2, 2, 13]
| [2, 0, 8] | [6, 9, 11] | [2, 9, 5] | [0, 1, 4] | [5, 1, 1] | [2, 1, 13]
JSGAN | [12, 11, 12] | [12, 11, 11] | [11, 13, 12] | [12, 11, 12] | [11, 10, 8] | [11, 13, 13]
| [12, 11, 14] | [13, 12, 13] | [8, 13, 12] | [11, 10, 14] | [9, 12, 11] | [13, 12, 12]
| [12, 12, 14] | [13, 10, 15] | [12, 13, 13] | [12, 12, 14] | [12, 11, 14] | [9, 11, 14]
TVGAN | [3, 0, 3] | [8, 6, 12] | [6, 2, 9] | [7, 2, 3] | [7, 7, 9] | [7, 11, 6]
| [3, 5, 3] | [6, 9, 9] | [6, 2, 6] | [7, 7, 4] | [3, 7, 7] | [6, 10, 4]
| [3, 4, 5] | [9, 6, 8] | [10, 5, 14] | [5, 0, 9] | [0, 0, 6] | [1, 2, 11]
HellingerGAN | [1, 1, 1] | [3, 7, 7] | [1, 3, 3] | [0, 1, 0] | [8, 12, 13] | [3, 8, 9]
| [1, 2, 0] | [2, 5, 8] | [0, 1, 0] | [0, 0, 0] | [5, 8, 12] | [1, 3, 9]
| [1, 3, 2] | [4, 11, 12] | [3, 3, 6] | [3, 3, 7] | [6, 2, 2] | [4, 7, 7]
PearsonGAN | [6, 5, 4] | [10, 12, 13] | [7, 7, 10] | [8, 5, 7] | [9, 11, 10] | [8, 12, 10]
| [7, 6, 2] | [8, 11, 12] | [3, 0, 3] | [4, 3, 6] | [0, 6, 2] | [5, 13, 6]
| [6, 6, 6] | [12, 13, 9] | [6, 1, 12] | [1, 6, 12] | [1, 10, 9] | [3, 9, 10]
Table 2: Performance-based relative ranking of GAN variants for 1,000 samples
(first entry of each row), 10,000 samples (second entry), and 100,000 samples
(third entry) with respect to Jensen-Shannon divergence (first row), Kullback-
Leibler divergence (second row), and Wasserstein distance (third row) for the
best hyperparameter settings for dimensionality $N=128$. Note that differences
between performances are sometimes marginal. Dimensionalities
$N\in\left\\{16,32,64\right\\}$ yielded similar ranking results to these with
slight variations.

Model | Normal | Beta | Gumbel | Laplace | Exponential | Gamma
---|---|---|---|---|---|---
WGAN | 0.128 $\pm$ 0.019 | 0.099 $\pm$ 0.010 | 0.062 $\pm$ 0.007 | 0.201 $\pm$ 0.019 | 0.156 $\pm$ 0.010 | 0.078 $\pm$ 0.011
| 0.160 $\pm$ 0.029 | 0.109 $\pm$ 0.011 | 0.066 $\pm$ 0.009 | 0.271 $\pm$ 0.027 | 0.128 $\pm$ 0.017 | 0.059 $\pm$ 0.009
| 0.299 $\pm$ 0.037 | 0.114 $\pm$ 0.018 | 0.064 $\pm$ 0.014 | 0.349 $\pm$ 0.050 | 0.108 $\pm$ 0.013 | 0.062 $\pm$ 0.013
WGANGP | 0.129 $\pm$ 0.026 | 0.190 $\pm$ 0.020 | 0.174 $\pm$ 0.023 | 0.259 $\pm$ 0.042 | 0.248 $\pm$ 0.019 | 0.143 $\pm$ 0.032
| 0.213 $\pm$ 0.048 | 0.121 $\pm$ 0.011 | 0.092 $\pm$ 0.013 | 0.287 $\pm$ 0.033 | 0.122 $\pm$ 0.014 | 0.068 $\pm$ 0.012
| 0.291 $\pm$ 0.022 | 0.066 $\pm$ 0.016 | 0.073 $\pm$ 0.011 | 0.341 $\pm$ 0.012 | 0.083 $\pm$ 0.010 | 0.051 $\pm$ 0.011
NSGAN | 0.106 $\pm$ 0.021 | 0.161 $\pm$ 0.019 | 0.079 $\pm$ 0.009 | 0.197 $\pm$ 0.022 | 0.213 $\pm$ 0.013 | 0.125 $\pm$ 0.025
| 0.120 $\pm$ 0.018 | 0.120 $\pm$ 0.013 | 0.085 $\pm$ 0.012 | 0.277 $\pm$ 0.028 | 0.146 $\pm$ 0.015 | 0.089 $\pm$ 0.020
| 0.126 $\pm$ 0.025 | 0.105 $\pm$ 0.013 | 0.080 $\pm$ 0.013 | 0.242 $\pm$ 0.021 | 0.110 $\pm$ 0.014 | 0.060 $\pm$ 0.016
MMGAN | 0.422 $\pm$ 0.040 | 0.168 $\pm$ 0.014 | 0.148 $\pm$ 0.026 | 0.444 $\pm$ 0.033 | 0.202 $\pm$ 0.013 | 0.311 $\pm$ 0.037
| 0.401 $\pm$ 0.025 | 0.142 $\pm$ 0.032 | 0.100 $\pm$ 0.009 | 0.440 $\pm$ 0.036 | 0.142 $\pm$ 0.018 | 0.098 $\pm$ 0.023
| 0.246 $\pm$ 0.033 | 0.110 $\pm$ 0.027 | 0.081 $\pm$ 0.009 | 0.334 $\pm$ 0.042 | 0.107 $\pm$ 0.015 | 0.060 $\pm$ 0.009
RaGAN | 0.160 $\pm$ 0.029 | 0.180 $\pm$ 0.019 | 0.100 $\pm$ 0.013 | 0.269 $\pm$ 0.027 | 0.216 $\pm$ 0.023 | 0.146 $\pm$ 0.035
| 0.169 $\pm$ 0.027 | 0.184 $\pm$ 0.016 | 0.099 $\pm$ 0.021 | 0.284 $\pm$ 0.023 | 0.220 $\pm$ 0.015 | 0.128 $\pm$ 0.028
| 0.156 $\pm$ 0.029 | 0.199 $\pm$ 0.016 | 0.104 $\pm$ 0.014 | 0.293 $\pm$ 0.029 | 0.224 $\pm$ 0.019 | 0.092 $\pm$ 0.020
LSGAN | 0.142 $\pm$ 0.035 | 0.170 $\pm$ 0.025 | 0.115 $\pm$ 0.015 | 0.204 $\pm$ 0.017 | 0.198 $\pm$ 0.019 | 0.340 $\pm$ 0.052
| 0.306 $\pm$ 0.021 | 0.127 $\pm$ 0.014 | 0.094 $\pm$ 0.019 | 0.333 $\pm$ 0.022 | 0.128 $\pm$ 0.020 | 0.102 $\pm$ 0.027
| 0.206 $\pm$ 0.016 | 0.080 $\pm$ 0.026 | 0.088 $\pm$ 0.010 | 0.288 $\pm$ 0.019 | 0.090 $\pm$ 0.012 | 0.056 $\pm$ 0.014
DRAGAN | 0.354 $\pm$ 0.041 | 0.212 $\pm$ 0.025 | 0.272 $\pm$ 0.031 | 0.392 $\pm$ 0.032 | 0.260 $\pm$ 0.021 | 0.301 $\pm$ 0.045
| 0.385 $\pm$ 0.064 | 0.198 $\pm$ 0.026 | 0.248 $\pm$ 0.051 | 0.457 $\pm$ 0.037 | 0.259 $\pm$ 0.054 | 0.216 $\pm$ 0.035
| 0.239 $\pm$ 0.017 | 0.069 $\pm$ 0.020 | 0.098 $\pm$ 0.010 | 0.297 $\pm$ 0.021 | 0.096 $\pm$ 0.020 | 0.087 $\pm$ 0.016
BEGAN | 0.549 $\pm$ 0.050 | 0.231 $\pm$ 0.018 | 0.329 $\pm$ 0.069 | 0.507 $\pm$ 0.051 | 0.301 $\pm$ 0.026 | 0.446 $\pm$ 0.040
| 0.457 $\pm$ 0.056 | 0.226 $\pm$ 0.016 | 0.179 $\pm$ 0.057 | 0.445 $\pm$ 0.050 | 0.340 $\pm$ 0.027 | 0.287 $\pm$ 0.061
| 0.348 $\pm$ 0.055 | 0.223 $\pm$ 0.017 | 0.120 $\pm$ 0.025 | 0.381 $\pm$ 0.041 | 0.307 $\pm$ 0.025 | 0.218 $\pm$ 0.052
InfoGAN | 0.190 $\pm$ 0.023 | 0.122 $\pm$ 0.019 | 0.113 $\pm$ 0.012 | 0.261 $\pm$ 0.024 | 0.151 $\pm$ 0.011 | 0.119 $\pm$ 0.019
| 0.204 $\pm$ 0.028 | 0.105 $\pm$ 0.011 | 0.104 $\pm$ 0.007 | 0.261 $\pm$ 0.026 | 0.134 $\pm$ 0.014 | 0.089 $\pm$ 0.019
| 0.202 $\pm$ 0.032 | 0.085 $\pm$ 0.015 | 0.104 $\pm$ 0.005 | 0.243 $\pm$ 0.022 | 0.102 $\pm$ 0.010 | 0.066 $\pm$ 0.018
FisherGAN | 0.112 $\pm$ 0.030 | 0.156 $\pm$ 0.013 | 0.083 $\pm$ 0.013 | 0.197 $\pm$ 0.029 | 0.223 $\pm$ 0.020 | 0.117 $\pm$ 0.023
| 0.109 $\pm$ 0.027 | 0.160 $\pm$ 0.011 | 0.081 $\pm$ 0.010 | 0.221 $\pm$ 0.026 | 0.193 $\pm$ 0.013 | 0.124 $\pm$ 0.024
| 0.110 $\pm$ 0.022 | 0.176 $\pm$ 0.021 | 0.078 $\pm$ 0.012 | 0.222 $\pm$ 0.020 | 0.184 $\pm$ 0.009 | 0.078 $\pm$ 0.015
ForwGAN | 0.083 $\pm$ 0.016 | 0.159 $\pm$ 0.017 | 0.073 $\pm$ 0.008 | 0.196 $\pm$ 0.021 | 0.233 $\pm$ 0.020 | 0.107 $\pm$ 0.019
| 0.106 $\pm$ 0.021 | 0.179 $\pm$ 0.020 | 0.075 $\pm$ 0.013 | 0.210 $\pm$ 0.021 | 0.187 $\pm$ 0.011 | 0.098 $\pm$ 0.022
| 0.096 $\pm$ 0.026 | 0.181 $\pm$ 0.030 | 0.055 $\pm$ 0.011 | 0.205 $\pm$ 0.021 | 0.168 $\pm$ 0.014 | 0.075 $\pm$ 0.014
RevGAN | 0.097 $\pm$ 0.027 | 0.147 $\pm$ 0.022 | 0.074 $\pm$ 0.010 | 0.172 $\pm$ 0.014 | 0.206 $\pm$ 0.022 | 0.104 $\pm$ 0.021
| 0.103 $\pm$ 0.025 | 0.173 $\pm$ 0.021 | 0.080 $\pm$ 0.009 | 0.188 $\pm$ 0.018 | 0.197 $\pm$ 0.013 | 0.087 $\pm$ 0.019
| 0.101 $\pm$ 0.015 | 0.174 $\pm$ 0.016 | 0.062 $\pm$ 0.011 | 0.187 $\pm$ 0.019 | 0.185 $\pm$ 0.015 | 0.085 $\pm$ 0.019
JSGAN | 0.245 $\pm$ 0.034 | 0.183 $\pm$ 0.016 | 0.116 $\pm$ 0.020 | 0.352 $\pm$ 0.036 | 0.229 $\pm$ 0.020 | 0.147 $\pm$ 0.026
| 0.243 $\pm$ 0.035 | 0.175 $\pm$ 0.013 | 0.124 $\pm$ 0.019 | 0.324 $\pm$ 0.025 | 0.195 $\pm$ 0.046 | 0.150 $\pm$ 0.034
| 0.257 $\pm$ 0.038 | 0.187 $\pm$ 0.012 | 0.097 $\pm$ 0.018 | 0.339 $\pm$ 0.055 | 0.167 $\pm$ 0.026 | 0.086 $\pm$ 0.016
TVGAN | 0.095 $\pm$ 0.022 | 0.165 $\pm$ 0.018 | 0.081 $\pm$ 0.014 | 0.204 $\pm$ 0.024 | 0.215 $\pm$ 0.022 | 0.126 $\pm$ 0.026
| 0.098 $\pm$ 0.016 | 0.157 $\pm$ 0.012 | 0.079 $\pm$ 0.009 | 0.205 $\pm$ 0.025 | 0.188 $\pm$ 0.013 | 0.123 $\pm$ 0.024
| 0.101 $\pm$ 0.016 | 0.183 $\pm$ 0.016 | 0.085 $\pm$ 0.013 | 0.204 $\pm$ 0.035 | 0.176 $\pm$ 0.012 | 0.062 $\pm$ 0.011
HellingerGAN | 0.088 $\pm$ 0.021 | 0.148 $\pm$ 0.014 | 0.067 $\pm$ 0.011 | 0.167 $\pm$ 0.012 | 0.215 $\pm$ 0.024 | 0.102 $\pm$ 0.022
| 0.097 $\pm$ 0.018 | 0.152 $\pm$ 0.013 | 0.072 $\pm$ 0.015 | 0.196 $\pm$ 0.014 | 0.210 $\pm$ 0.012 | 0.104 $\pm$ 0.018
| 0.099 $\pm$ 0.022 | 0.159 $\pm$ 0.018 | 0.062 $\pm$ 0.016 | 0.179 $\pm$ 0.021 | 0.212 $\pm$ 0.022 | 0.076 $\pm$ 0.014
PearsonGAN | 0.112 $\pm$ 0.023 | 0.172 $\pm$ 0.021 | 0.087 $\pm$ 0.013 | 0.238 $\pm$ 0.024 | 0.217 $\pm$ 0.024 | 0.124 $\pm$ 0.023
| 0.118 $\pm$ 0.021 | 0.177 $\pm$ 0.014 | 0.088 $\pm$ 0.008 | 0.256 $\pm$ 0.027 | 0.203 $\pm$ 0.018 | 0.121 $\pm$ 0.026
| 0.100 $\pm$ 0.013 | 0.180 $\pm$ 0.020 | 0.084 $\pm$ 0.009 | 0.250 $\pm$ 0.030 | 0.175 $\pm$ 0.016 | 0.074 $\pm$ 0.015

Table 3: Confidence intervals of the best average minimum Jensen-Shannon
divergence across 20 trials for 1,000 samples (first row), 10,000 samples
(second row), and 100,000 samples (third row) of the best hyperparameter
settings for dimensionality $N=128$.

Model | Normal | Beta | Gumbel | Laplace | Exponential | Gamma
---|---|---|---|---|---|---
WGAN | 0.832 $\pm$ 0.295 | 0.445 $\pm$ 0.043 | 0.251 $\pm$ 0.037 | 1.174 $\pm$ 0.249 | 0.609 $\pm$ 0.061 | 0.312 $\pm$ 0.038
| 0.985 $\pm$ 0.188 | 0.521 $\pm$ 0.082 | 0.232 $\pm$ 0.025 | 1.359 $\pm$ 0.221 | 0.639 $\pm$ 0.052 | 0.229 $\pm$ 0.037
| 1.185 $\pm$ 0.194 | 0.566 $\pm$ 0.083 | 0.216 $\pm$ 0.025 | 1.430 $\pm$ 0.285 | 0.552 $\pm$ 0.040 | 0.244 $\pm$ 0.040
WGANGP | 0.989 $\pm$ 0.280 | 0.985 $\pm$ 0.119 | 1.139 $\pm$ 0.303 | 1.916 $\pm$ 0.351 | 0.955 $\pm$ 0.122 | 0.904 $\pm$ 0.144
| 1.520 $\pm$ 0.430 | 0.610 $\pm$ 0.091 | 0.440 $\pm$ 0.085 | 1.943 $\pm$ 0.412 | 0.661 $\pm$ 0.142 | 0.393 $\pm$ 0.177
| 1.380 $\pm$ 0.134 | 0.401 $\pm$ 0.131 | 0.318 $\pm$ 0.041 | 1.533 $\pm$ 0.184 | 0.463 $\pm$ 0.042 | 0.216 $\pm$ 0.042
NSGAN | 0.544 $\pm$ 0.238 | 0.814 $\pm$ 0.256 | 0.177 $\pm$ 0.010 | 0.891 $\pm$ 0.202 | 0.687 $\pm$ 0.206 | 0.603 $\pm$ 0.299
| 0.381 $\pm$ 0.181 | 0.583 $\pm$ 0.080 | 0.221 $\pm$ 0.075 | 0.874 $\pm$ 0.256 | 0.546 $\pm$ 0.170 | 0.534 $\pm$ 0.204
| 0.522 $\pm$ 0.179 | 0.473 $\pm$ 0.064 | 0.165 $\pm$ 0.011 | 0.788 $\pm$ 0.249 | 0.556 $\pm$ 0.078 | 0.302 $\pm$ 0.069
MMGAN | 2.458 $\pm$ 0.184 | 0.932 $\pm$ 0.114 | 0.864 $\pm$ 0.228 | 2.796 $\pm$ 0.373 | 1.028 $\pm$ 0.115 | 1.108 $\pm$ 0.158
| 1.946 $\pm$ 0.219 | 0.697 $\pm$ 0.094 | 0.374 $\pm$ 0.041 | 2.290 $\pm$ 0.347 | 0.999 $\pm$ 0.146 | 0.584 $\pm$ 0.185
| 1.449 $\pm$ 0.327 | 0.572 $\pm$ 0.127 | 0.310 $\pm$ 0.076 | 2.013 $\pm$ 0.155 | 0.598 $\pm$ 0.086 | 0.254 $\pm$ 0.038
RaGAN | 0.730 $\pm$ 0.299 | 0.992 $\pm$ 0.126 | 0.211 $\pm$ 0.077 | 0.900 $\pm$ 0.187 | 0.940 $\pm$ 0.101 | 0.935 $\pm$ 0.368
| 0.795 $\pm$ 0.342 | 1.067 $\pm$ 0.129 | 0.262 $\pm$ 0.186 | 1.167 $\pm$ 0.323 | 1.034 $\pm$ 0.126 | 0.451 $\pm$ 0.160
| 0.741 $\pm$ 0.242 | 1.183 $\pm$ 0.337 | 0.221 $\pm$ 0.080 | 1.040 $\pm$ 0.243 | 1.052 $\pm$ 0.126 | 0.357 $\pm$ 0.085
LSGAN | 1.012 $\pm$ 0.369 | 0.901 $\pm$ 0.155 | 0.616 $\pm$ 0.157 | 1.006 $\pm$ 0.196 | 0.950 $\pm$ 0.062 | 1.035 $\pm$ 0.118
| 1.608 $\pm$ 0.316 | 0.633 $\pm$ 0.103 | 0.390 $\pm$ 0.112 | 2.004 $\pm$ 0.324 | 0.925 $\pm$ 0.221 | 0.675 $\pm$ 0.257
| 1.073 $\pm$ 0.102 | 0.365 $\pm$ 0.128 | 0.316 $\pm$ 0.038 | 1.728 $\pm$ 0.221 | 0.495 $\pm$ 0.070 | 0.274 $\pm$ 0.075
DRAGAN | 2.326 $\pm$ 0.230 | 1.075 $\pm$ 0.153 | 1.281 $\pm$ 0.120 | 3.223 $\pm$ 0.270 | 0.930 $\pm$ 0.089 | 0.993 $\pm$ 0.094
| 2.444 $\pm$ 0.213 | 1.367 $\pm$ 0.187 | 1.314 $\pm$ 0.162 | 3.348 $\pm$ 0.301 | 1.008 $\pm$ 0.144 | 1.001 $\pm$ 0.092
| 1.608 $\pm$ 0.226 | 0.683 $\pm$ 0.192 | 0.594 $\pm$ 0.139 | 2.119 $\pm$ 0.376 | 0.732 $\pm$ 0.286 | 0.879 $\pm$ 0.326
BEGAN | 2.774 $\pm$ 0.268 | 1.360 $\pm$ 0.118 | 1.445 $\pm$ 0.170 | 3.721 $\pm$ 0.347 | 1.060 $\pm$ 0.129 | 1.144 $\pm$ 0.131
| 3.004 $\pm$ 0.339 | 1.394 $\pm$ 0.147 | 1.008 $\pm$ 0.452 | 2.389 $\pm$ 0.421 | 1.269 $\pm$ 0.149 | 1.391 $\pm$ 0.217
| 2.099 $\pm$ 0.368 | 1.446 $\pm$ 0.134 | 0.589 $\pm$ 0.183 | 1.834 $\pm$ 0.421 | 1.645 $\pm$ 0.257 | 1.209 $\pm$ 0.316
InfoGAN | 1.021 $\pm$ 0.187 | 0.510 $\pm$ 0.103 | 0.561 $\pm$ 0.121 | 1.464 $\pm$ 0.220 | 0.826 $\pm$ 0.118 | 0.664 $\pm$ 0.208
| 0.882 $\pm$ 0.158 | 0.505 $\pm$ 0.115 | 0.371 $\pm$ 0.024 | 0.906 $\pm$ 0.110 | 0.593 $\pm$ 0.081 | 0.336 $\pm$ 0.051
| 0.965 $\pm$ 0.197 | 0.428 $\pm$ 0.109 | 0.370 $\pm$ 0.022 | 1.213 $\pm$ 0.076 | 0.456 $\pm$ 0.051 | 0.321 $\pm$ 0.044
FisherGAN | 0.391 $\pm$ 0.173 | 0.709 $\pm$ 0.060 | 0.171 $\pm$ 0.015 | 0.779 $\pm$ 0.240 | 0.803 $\pm$ 0.290 | 0.381 $\pm$ 0.149
| 0.609 $\pm$ 0.181 | 0.818 $\pm$ 0.061 | 0.235 $\pm$ 0.081 | 0.906 $\pm$ 0.182 | 0.621 $\pm$ 0.202 | 0.394 $\pm$ 0.180
| 0.483 $\pm$ 0.172 | 0.646 $\pm$ 0.166 | 0.176 $\pm$ 0.013 | 0.637 $\pm$ 0.129 | 0.716 $\pm$ 0.224 | 0.346 $\pm$ 0.100
ForwGAN | 0.321 $\pm$ 0.137 | 0.680 $\pm$ 0.202 | 0.190 $\pm$ 0.009 | 0.771 $\pm$ 0.219 | 0.909 $\pm$ 0.218 | 0.397 $\pm$ 0.134
| 0.377 $\pm$ 0.138 | 0.808 $\pm$ 0.160 | 0.232 $\pm$ 0.151 | 0.653 $\pm$ 0.150 | 0.873 $\pm$ 0.210 | 0.466 $\pm$ 0.280
| 0.541 $\pm$ 0.310 | 0.766 $\pm$ 0.219 | 0.159 $\pm$ 0.014 | 0.735 $\pm$ 0.265 | 0.712 $\pm$ 0.150 | 0.334 $\pm$ 0.087
RevGAN | 0.518 $\pm$ 0.174 | 0.690 $\pm$ 0.133 | 0.191 $\pm$ 0.074 | 0.698 $\pm$ 0.195 | 0.820 $\pm$ 0.188 | 0.352 $\pm$ 0.132
| 0.519 $\pm$ 0.211 | 0.745 $\pm$ 0.197 | 0.214 $\pm$ 0.019 | 0.741 $\pm$ 0.208 | 0.779 $\pm$ 0.222 | 0.369 $\pm$ 0.151
| 0.370 $\pm$ 0.170 | 0.630 $\pm$ 0.175 | 0.150 $\pm$ 0.013 | 0.691 $\pm$ 0.135 | 0.863 $\pm$ 0.210 | 0.400 $\pm$ 0.137
JSGAN | 1.919 $\pm$ 0.639 | 1.014 $\pm$ 0.115 | 0.219 $\pm$ 0.071 | 1.705 $\pm$ 0.445 | 0.905 $\pm$ 0.087 | 1.050 $\pm$ 0.137
| 1.560 $\pm$ 0.439 | 0.904 $\pm$ 0.175 | 0.521 $\pm$ 0.217 | 1.855 $\pm$ 0.491 | 0.998 $\pm$ 0.130 | 0.687 $\pm$ 0.136
| 1.879 $\pm$ 0.384 | 1.034 $\pm$ 0.226 | 0.346 $\pm$ 0.130 | 2.077 $\pm$ 0.561 | 0.858 $\pm$ 0.201 | 0.359 $\pm$ 0.083
TVGAN | 0.486 $\pm$ 0.188 | 0.762 $\pm$ 0.244 | 0.201 $\pm$ 0.075 | 0.911 $\pm$ 0.273 | 0.694 $\pm$ 0.242 | 0.590 $\pm$ 0.237
| 0.602 $\pm$ 0.187 | 0.800 $\pm$ 0.248 | 0.197 $\pm$ 0.079 | 0.962 $\pm$ 0.240 | 0.798 $\pm$ 0.214 | 0.581 $\pm$ 0.220
| 0.460 $\pm$ 0.189 | 0.670 $\pm$ 0.181 | 0.180 $\pm$ 0.015 | 0.752 $\pm$ 0.169 | 0.594 $\pm$ 0.200 | 0.273 $\pm$ 0.058
HellingerGAN | 0.377 $\pm$ 0.208 | 0.548 $\pm$ 0.112 | 0.155 $\pm$ 0.012 | 0.645 $\pm$ 0.195 | 0.801 $\pm$ 0.170 | 0.337 $\pm$ 0.103
| 0.409 $\pm$ 0.190 | 0.678 $\pm$ 0.195 | 0.187 $\pm$ 0.026 | 0.628 $\pm$ 0.148 | 0.804 $\pm$ 0.238 | 0.379 $\pm$ 0.111
| 0.352 $\pm$ 0.100 | 0.659 $\pm$ 0.200 | 0.149 $\pm$ 0.010 | 0.586 $\pm$ 0.191 | 0.855 $\pm$ 0.152 | 0.337 $\pm$ 0.076
PearsonGAN | 0.764 $\pm$ 0.242 | 0.869 $\pm$ 0.216 | 0.177 $\pm$ 0.015 | 0.874 $\pm$ 0.286 | 0.578 $\pm$ 0.147 | 0.409 $\pm$ 0.202
| 0.600 $\pm$ 0.229 | 0.815 $\pm$ 0.202 | 0.175 $\pm$ 0.025 | 0.862 $\pm$ 0.415 | 0.786 $\pm$ 0.214 | 0.744 $\pm$ 0.169
| 0.391 $\pm$ 0.143 | 0.867 $\pm$ 0.262 | 0.154 $\pm$ 0.017 | 0.820 $\pm$ 0.246 | 0.481 $\pm$ 0.097 | 0.318 $\pm$ 0.099

Table 4: Confidence intervals of the best average minimum Kullback-Leibler
divergence across 20 trials for 1,000 samples (first row), 10,000 samples
(second row), and 100,000 samples (third row) of the best hyperparameter
settings for dimensionality $N=128$.

Model | Normal | Beta | Gumbel | Laplace | Exponential | Gamma
---|---|---|---|---|---|---
WGAN | 2.376 $\pm$ 0.428 | 6.301 $\pm$ 0.543 | 5.910 $\pm$ 1.368 | 7.698 $\pm$ 2.711 | 9.615 $\pm$ 2.024 | 3.235 $\pm$ 1.053
| 2.655 $\pm$ 0.950 | 6.777 $\pm$ 2.090 | 8.019 $\pm$ 1.944 | 8.577 $\pm$ 3.843 | 11.899 $\pm$ 5.086 | 2.995 $\pm$ 1.149
| 3.042 $\pm$ 0.449 | 5.927 $\pm$ 0.366 | 7.037 $\pm$ 2.640 | 6.988 $\pm$ 1.975 | 8.405 $\pm$ 1.402 | 1.881 $\pm$ 0.543
WGANGP | 3.191 $\pm$ 0.400 | 8.336 $\pm$ 0.654 | 7.611 $\pm$ 2.508 | 8.480 $\pm$ 3.452 | 11.127 $\pm$ 2.078 | 4.279 $\pm$ 1.734
| 4.195 $\pm$ 1.431 | 7.808 $\pm$ 2.472 | 9.871 $\pm$ 4.512 | 9.882 $\pm$ 5.517 | 11.367 $\pm$ 6.599 | 3.672 $\pm$ 1.154
| 2.043 $\pm$ 0.147 | 5.038 $\pm$ 0.290 | 4.768 $\pm$ 1.626 | 4.941 $\pm$ 1.163 | 8.142 $\pm$ 3.307 | 1.348 $\pm$ 0.396
NSGAN | 1.749 $\pm$ 0.413 | 9.099 $\pm$ 0.734 | 7.445 $\pm$ 2.463 | 6.187 $\pm$ 1.414 | 10.076 $\pm$ 3.777 | 3.401 $\pm$ 1.031
| 3.358 $\pm$ 1.237 | 8.643 $\pm$ 2.142 | 8.464 $\pm$ 2.808 | 8.950 $\pm$ 3.988 | 11.447 $\pm$ 4.605 | 3.737 $\pm$ 1.477
| 3.029 $\pm$ 0.229 | 7.469 $\pm$ 0.535 | 4.117 $\pm$ 1.023 | 5.327 $\pm$ 1.703 | 10.218 $\pm$ 3.821 | 2.223 $\pm$ 0.816
MMGAN | 10.201 $\pm$ 1.561 | 10.464 $\pm$ 0.755 | 11.390 $\pm$ 2.192 | 13.033 $\pm$ 1.864 | 17.059 $\pm$ 3.348 | 10.312 $\pm$ 1.613
| 5.495 $\pm$ 1.363 | 10.551 $\pm$ 3.462 | 8.330 $\pm$ 3.214 | 10.662 $\pm$ 5.150 | 18.233 $\pm$ 6.909 | 5.926 $\pm$ 5.555
| 3.110 $\pm$ 0.452 | 6.617 $\pm$ 0.661 | 4.452 $\pm$ 1.029 | 5.534 $\pm$ 1.823 | 10.873 $\pm$ 2.759 | 1.843 $\pm$ 0.506
RaGAN | 2.740 $\pm$ 0.566 | 8.983 $\pm$ 0.788 | 7.380 $\pm$ 2.084 | 7.509 $\pm$ 2.185 | 8.976 $\pm$ 2.146 | 2.791 $\pm$ 1.016
| 4.519 $\pm$ 1.452 | 13.032 $\pm$ 4.097 | 10.823 $\pm$ 4.832 | 8.052 $\pm$ 2.342 | 12.526 $\pm$ 3.182 | 4.056 $\pm$ 1.408
| 3.537 $\pm$ 0.381 | 10.291 $\pm$ 0.733 | 7.178 $\pm$ 1.257 | 7.852 $\pm$ 2.715 | 10.936 $\pm$ 2.011 | 3.654 $\pm$ 1.024
LSGAN | 4.280 $\pm$ 0.518 | 9.232 $\pm$ 0.563 | 10.503 $\pm$ 3.403 | 8.753 $\pm$ 2.167 | 11.754 $\pm$ 3.206 | 9.023 $\pm$ 0.929
| 4.096 $\pm$ 1.636 | 8.427 $\pm$ 2.414 | 8.978 $\pm$ 4.606 | 13.346 $\pm$ 4.851 | 15.695 $\pm$ 5.691 | 7.326 $\pm$ 5.143
| 2.286 $\pm$ 0.190 | 5.827 $\pm$ 0.778 | 3.460 $\pm$ 0.814 | 4.262 $\pm$ 0.798 | 9.539 $\pm$ 3.441 | 2.345 $\pm$ 0.962
DRAGAN | 7.051 $\pm$ 0.851 | 8.892 $\pm$ 0.572 | 11.208 $\pm$ 1.052 | 11.004 $\pm$ 1.971 | 15.461 $\pm$ 2.439 | 10.695 $\pm$ 2.428
| 9.638 $\pm$ 2.113 | 11.497 $\pm$ 4.364 | 17.183 $\pm$ 6.580 | 12.998 $\pm$ 3.925 | 19.594 $\pm$ 9.288 | 9.541 $\pm$ 5.206
| 2.704 $\pm$ 0.338 | 4.614 $\pm$ 0.641 | 5.762 $\pm$ 2.537 | 6.923 $\pm$ 1.181 | 7.696 $\pm$ 2.245 | 2.713 $\pm$ 0.689
BEGAN | 14.073 $\pm$ 0.759 | 11.215 $\pm$ 0.477 | 20.592 $\pm$ 2.197 | 18.819 $\pm$ 2.514 | 21.311 $\pm$ 4.300 | 16.569 $\pm$ 0.759
| 19.480 $\pm$ 5.667 | 12.881 $\pm$ 3.939 | 23.892 $\pm$ 7.131 | 22.189 $\pm$ 6.844 | 24.741 $\pm$ 7.144 | 14.040 $\pm$ 9.001
| 13.694 $\pm$ 1.408 | 10.468 $\pm$ 0.663 | 16.832 $\pm$ 3.186 | 20.066 $\pm$ 1.192 | 23.807 $\pm$ 3.008 | 9.741 $\pm$ 1.698
InfoGAN | 4.184 $\pm$ 0.693 | 7.700 $\pm$ 0.388 | 7.702 $\pm$ 2.137 | 8.213 $\pm$ 2.313 | 10.682 $\pm$ 1.933 | 5.653 $\pm$ 1.414
| 4.707 $\pm$ 3.152 | 8.039 $\pm$ 3.353 | 8.623 $\pm$ 3.280 | 8.085 $\pm$ 3.983 | 12.106 $\pm$ 8.355 | 5.034 $\pm$ 3.126
| 2.535 $\pm$ 0.277 | 5.789 $\pm$ 0.597 | 6.710 $\pm$ 2.561 | 6.351 $\pm$ 2.116 | 9.892 $\pm$ 3.821 | 2.207 $\pm$ 0.638
FisherGAN | 1.823 $\pm$ 0.550 | 8.931 $\pm$ 0.997 | 5.493 $\pm$ 1.880 | 6.836 $\pm$ 1.186 | 10.790 $\pm$ 3.162 | 3.579 $\pm$ 2.350
| 3.105 $\pm$ 1.345 | 11.792 $\pm$ 3.672 | 7.582 $\pm$ 2.777 | 8.348 $\pm$ 4.066 | 12.528 $\pm$ 3.745 | 3.895 $\pm$ 1.670
| 2.768 $\pm$ 0.433 | 9.146 $\pm$ 0.449 | 7.666 $\pm$ 1.595 | 5.885 $\pm$ 0.885 | 10.648 $\pm$ 2.411 | 3.388 $\pm$ 0.929
ForwGAN | 1.450 $\pm$ 0.426 | 8.835 $\pm$ 0.506 | 7.515 $\pm$ 2.029 | 5.828 $\pm$ 1.839 | 11.519 $\pm$ 3.067 | 3.431 $\pm$ 0.767
| 2.634 $\pm$ 0.958 | 11.460 $\pm$ 3.812 | 9.553 $\pm$ 4.423 | 7.038 $\pm$ 2.696 | 11.161 $\pm$ 2.903 | 4.347 $\pm$ 1.576
| 2.380 $\pm$ 0.413 | 9.478 $\pm$ 0.281 | 6.854 $\pm$ 2.344 | 4.799 $\pm$ 0.825 | 8.786 $\pm$ 1.829 | 3.220 $\pm$ 0.977
RevGAN | 1.537 $\pm$ 0.456 | 8.915 $\pm$ 0.627 | 6.060 $\pm$ 1.804 | 4.752 $\pm$ 0.978 | 10.211 $\pm$ 3.843 | 2.889 $\pm$ 1.359
| 2.358 $\pm$ 0.593 | 11.499 $\pm$ 3.694 | 9.208 $\pm$ 3.809 | 6.812 $\pm$ 2.425 | 10.693 $\pm$ 3.902 | 3.420 $\pm$ 1.207
| 2.714 $\pm$ 0.336 | 9.645 $\pm$ 0.424 | 5.959 $\pm$ 1.767 | 5.355 $\pm$ 1.093 | 7.898 $\pm$ 1.725 | 3.665 $\pm$ 1.149
JSGAN | 5.042 $\pm$ 0.709 | 10.094 $\pm$ 0.601 | 10.782 $\pm$ 2.684 | 9.379 $\pm$ 2.631 | 12.464 $\pm$ 1.795 | 4.090 $\pm$ 1.270
| 5.276 $\pm$ 1.471 | 11.595 $\pm$ 2.963 | 12.447 $\pm$ 5.515 | 10.820 $\pm$ 2.957 | 14.465 $\pm$ 5.005 | 5.057 $\pm$ 2.490
| 4.272 $\pm$ 0.908 | 10.478 $\pm$ 1.770 | 8.104 $\pm$ 1.919 | 9.201 $\pm$ 1.930 | 12.593 $\pm$ 2.252 | 3.932 $\pm$ 0.745
TVGAN | 1.572 $\pm$ 0.425 | 9.026 $\pm$ 0.685 | 7.761 $\pm$ 1.566 | 6.326 $\pm$ 0.902 | 8.706 $\pm$ 1.955 | 2.868 $\pm$ 1.068
| 3.024 $\pm$ 1.161 | 11.396 $\pm$ 3.091 | 8.336 $\pm$ 2.476 | 6.074 $\pm$ 2.376 | 10.191 $\pm$ 3.094 | 3.592 $\pm$ 1.347
| 2.543 $\pm$ 0.421 | 9.273 $\pm$ 0.436 | 8.183 $\pm$ 2.106 | 6.407 $\pm$ 2.109 | 8.852 $\pm$ 2.168 | 3.529 $\pm$ 0.848
HellingerGAN | 1.524 $\pm$ 0.341 | 8.882 $\pm$ 0.431 | 6.097 $\pm$ 1.572 | 5.846 $\pm$ 1.577 | 10.578 $\pm$ 2.485 | 3.140 $\pm$ 1.179
| 2.694 $\pm$ 0.781 | 11.558 $\pm$ 3.325 | 8.278 $\pm$ 1.857 | 7.158 $\pm$ 2.250 | 10.792 $\pm$ 4.651 | 4.148 $\pm$ 1.731
| 2.376 $\pm$ 0.346 | 9.982 $\pm$ 0.520 | 6.650 $\pm$ 1.888 | 6.172 $\pm$ 2.072 | 7.931 $\pm$ 1.591 | 2.925 $\pm$ 1.307
PearsonGAN | 1.946 $\pm$ 0.439 | 9.315 $\pm$ 0.996 | 7.482 $\pm$ 3.103 | 5.619 $\pm$ 1.029 | 8.804 $\pm$ 1.810 | 2.961 $\pm$ 0.885
| 3.243 $\pm$ 1.122 | 11.982 $\pm$ 4.928 | 7.740 $\pm$ 2.779 | 8.301 $\pm$ 3.403 | 13.876 $\pm$ 4.201 | 4.569 $\pm$ 1.747
| 2.639 $\pm$ 0.246 | 9.390 $\pm$ 0.469 | 7.866 $\pm$ 1.670 | 7.304 $\pm$ 1.037 | 10.162 $\pm$ 2.995 | 3.465 $\pm$ 0.858

Table 5: Confidence intervals of the best average minimum Wasserstein distance
across 20 trials for 1,000 samples (first row), 10,000 samples (second row),
and 100,000 samples (third row) of the best hyperparameter settings for
dimensionality $N=128$.

##  Appendix D Diverse sets of hyperparameters can produce a best result

Model | Normal | Beta | Gumbel | Laplace | Exponential | Gamma
---|---|---|---|---|---|---
WGAN | [8, 3, 13] | [5, 3, 3] | [6, 6, 7] | [6, 5, 7] | [8, 11, 9] | [11, 8, 7]
| [14, 12, 10] | [5, 3, 4] | [5, 10, 5] | [14, 9, 9] | [10, 9, 10] | [11, 9, 7]
| [8, 11, 9] | [7, 6, 7] | [13, 11, 14] | [8, 10, 12] | [13, 13, 13] | [8, 10, 12]
WGANGP | [4, 8, 9] | [2, 3, 5] | [8, 7, 3] | [4, 5, 6] | [3, 6, 5] | [9, 4, 6]
| [5, 7, 7] | [3, 3, 3] | [6, 5, 2] | [4, 5, 4] | [8, 8, 10] | [11, 5, 6]
| [8, 6, 7] | [8, 8, 8] | [13, 13, 13] | [13, 12, 14] | [15, 12, 14] | [11, 10, 14]
NSGAN | [6, 5, 5] | [7, 4, 4] | [5, 7, 10] | [7, 4, 5] | [9, 4, 9] | [11, 8, 8]
| [9, 7, 8] | [9, 4, 4] | [5, 8, 7] | [8, 5, 6] | [9, 10, 12] | [11, 8, 12]
| [5, 6, 7] | [3, 5, 6] | [11, 10, 12] | [8, 11, 14] | [11, 9, 14] | [10, 9, 12]
MMGAN | [9, 8, 6] | [5, 4, 5] | [5, 4, 6] | [12, 11, 5] | [4, 5, 8] | [9, 7, 6]
| [8, 6, 4] | [3, 4, 5] | [6, 4, 5] | [7, 4, 2] | [5, 7, 10] | [5, 3, 7]
| [15, 8, 8] | [7, 3, 9] | [14, 6, 13] | [14, 9, 15] | [17, 11, 12] | [14, 15, 11]
RaGAN | [11, 8, 7] | [4, 6, 9] | [8, 10, 9] | [8, 9, 7] | [8, 5, 5] | [13, 12, 13]
| [11, 12, 10] | [6, 9, 10] | [9, 9, 13] | [13, 13, 11] | [8, 4, 4] | [12, 9, 11]
| [9, 5, 10] | [5, 8, 12] | [9, 14, 12] | [12, 10, 13] | [12, 12, 10] | [14, 12, 13]
LSGAN | [4, 9, 9] | [5, 5, 7] | [3, 5, 5] | [2, 6, 8] | [6, 5, 5] | [3, 7, 7]
| [5, 4, 3] | [5, 7, 6] | [3, 4, 5] | [2, 7, 4] | [8, 7, 7] | [4, 6, 8]
| [4, 6, 7] | [4, 8, 8] | [8, 10, 12] | [6, 11, 11] | [9, 12, 11] | [9, 7, 12]
DRAGAN | [12, 11, 7] | [9, 4, 5] | [9, 9, 6] | [9, 11, 9] | [9, 8, 4] | [14, 13, 7]
| [8, 6, 9] | [6, 4, 9] | [5, 4, 6] | [11, 6, 8] | [7, 4, 11] | [10, 7, 3]
| [13, 13, 5] | [6, 12, 5] | [10, 10, 10] | [12, 12, 9] | [12, 9, 9] | [11, 13, 7]
BEGAN | [11, 4, 5] | [8, 5, 5] | [7, 4, 3] | [6, 3, 3] | [6, 3, 6] | [4, 5, 3]
| [7, 3, 5] | [6, 6, 4] | [5, 5, 3] | [6, 4, 2] | [4, 3, 5] | [4, 2, 4]
| [7, 9, 4] | [12, 8, 9] | [17, 15, 4] | [13, 9, 3] | [15, 13, 15] | [9, 9, 3]
InfoGAN | [10, 10, 5] | [3, 4, 5] | [9, 5, 9] | [8, 10, 7] | [8, 4, 2] | [9, 4, 7]
| [7, 5, 3] | [4, 4, 5] | [8, 3, 8] | [5, 6, 5] | [9, 3, 4] | [12, 4, 9]
| [7, 7, 9] | [7, 10, 3] | [12, 11, 11] | [13, 13, 10] | [13, 14, 12] | [12, 4, 12]
FisherGAN | [6, 7, 5] | [9, 8, 7] | [6, 8, 9] | [8, 9, 7] | [10, 9, 13] | [10, 11, 9]
| [9, 12, 7] | [9, 10, 9] | [5, 8, 6] | [9, 9, 7] | [10, 7, 8] | [10, 10, 12]
| [7, 6, 10] | [4, 6, 6] | [12, 10, 13] | [10, 11, 10] | [11, 13, 15] | [11, 10, 9]
ForwGAN | [5, 6, 6] | [7, 9, 11] | [6, 6, 6] | [7, 6, 7] | [4, 10, 12] | [8, 7, 10]
| [8, 8, 8] | [8, 6, 7] | [3, 6, 4] | [9, 7, 7] | [11, 12, 9] | [11, 8, 10]
| [5, 6, 6] | [6, 8, 6] | [12, 14, 13] | [10, 11, 8] | [11, 11, 13] | [9, 11, 10]
RevGAN | [5, 8, 9] | [5, 10, 7] | [7, 7, 8] | [5, 5, 5] | [6, 11, 9] | [9, 8, 8]
| [8, 9, 9] | [9, 5, 7] | [4, 5, 3] | [7, 8, 7] | [11, 8, 10] | [9, 13, 12]
| [7, 7, 8] | [7, 8, 6] | [11, 11, 13] | [10, 11, 10] | [12, 10, 14] | [11, 12, 9]
JSGAN | [9, 11, 11] | [6, 6, 7] | [10, 11, 11] | [9, 7, 7] | [7, 5, 7] | [11, 6, 12]
| [6, 10, 8] | [5, 5, 5] | [9, 9, 9] | [9, 9, 8] | [6, 4, 10] | [12, 10, 13]
| [13, 12, 10] | [4, 9, 11] | [12, 10, 12] | [12, 14, 16] | [12, 11, 11] | [12, 10, 13]
TVGAN | [7, 6, 7] | [7, 7, 9] | [7, 8, 9] | [9, 8, 8] | [10, 11, 11] | [10, 11, 9]
| [9, 11, 12] | [10, 10, 9] | [9, 6, 5] | [9, 6, 9] | [7, 9, 10] | [11, 12, 11]
| [7, 7, 7] | [5, 5, 7] | [11, 10, 15] | [9, 9, 9] | [12, 12, 10] | [11, 8, 11]
HellingerGAN | [7, 5, 5] | [7, 8, 8] | [6, 6, 4] | [4, 4, 5] | [7, 8, 6] | [11, 9, 11]
| [9, 6, 9] | [6, 6, 4] | [5, 4, 3] | [7, 5, 6] | [9, 6, 6] | [11, 8, 8]
| [7, 6, 8] | [5, 5, 9] | [11, 10, 13] | [10, 11, 9] | [12, 14, 14] | [10, 11, 9]
PearsonGAN | [9, 9, 8] | [6, 8, 8] | [6, 7, 6] | [6, 8, 7] | [8, 6, 10] | [11, 9, 11]
| [13, 9, 10] | [8, 9, 8] | [8, 8, 10] | [11, 13, 11] | [10, 7, 10] | [14, 10, 11]
| [7, 6, 9] | [6, 11, 6] | [11, 9, 13] | [10, 11, 12] | [9, 12, 16] | [11, 11, 12]
Table 6: Number of unique hyperparameter settings that yielded a best result
out of 20 trials for 1,000 samples (first entry of each row), 10,000 samples
(second entry), and 100,000 samples (third entry) with respect to Kullback-
Leibler divergence (first row), Jensen-Shannon divergence (second row), and
Wasserstein distance (third row) for dimensionality $N=128$. The maximum value
for any given entry would be 20 provided that a diferent hyperparameter output
the minimum performance for every trial conducted, and the minimum value would
be 1 if only one hyperparameter setting always outperformed the rest.
Dimensionalities $N\in\left\\{16,32,64\right\\}$ yielded similar results.

##  Appendix E Some GANs are more robust to hyperparameter changes than others

Model | Jensen-Shannon | Kullback-Leibler | Wasserstein Distance | Total (% of max)
---|---|---|---|---
WGAN | 167 | 215 | 346 | 728 (1.40%)
WGANGP | 115 | 146 | 492 | 753 (1.45%)
NSGAN | 43 | 64 | 221 | 328 (0.63%)
MMGAN | 34 | 127 | 416 | 577 (1.11%)
RaGAN | 80 | 58 | 69 | 207 (0.40%)
LSGAN | 73 | 120 | 386 | 579 (1.12%)
DRAGAN | 126 | 138 | 380 | 644 (1.24%)
BEGAN | 54 | 137 | 352 | 543 (1.05%)
InfoGAN | 127 | 179 | 366 | 672 (1.30%)
FisherGAN | 42 | 38 | 159 | 239 (0.46%)
ForwGAN | 45 | 51 | 166 | 262 (0.51%)
RevGAN | 43 | 44 | 137 | 224 (0.43%))
JSGAN | 72 | 31 | 179 | 282 (0.54%)
TVGAN | 38 | 47 | 126 | 211 (0.41%)
HellingerGAN | 41 | 57 | 176 | 274 (0.53%)
PearsonGAN | 32 | 42 | 165 | 239 (0.46%)
Table 7: The number of hyperparameter settings that yielded a minimum average
performance that fell within the confidence interval of the best average
performance across all 16 models, three evaluation metrics, six distributions,
15 hyperparameter settings, four dimensionalities, and three sample sizes. The
last column indicates the sum of the values to its left followed by this
number divided by 51,840 (the maximum possible number in this table) to its
right, rounded up to the nearest hundredth.

##  Appendix F Model specifications

| $d=16$ | $d=32$ | $d=64$ | $d=128$
---|---|---|---|---
$h=32$ | [1,393 - 1,821 - 1,888] | [2,433 - 2,861 - 3,456] | [4,513 - 4,941 - 6,592] | [8,673 - 9,101 - 12,864]
$h=64$ | [3,281 - 4,125 - 4,256] | [5,345 - 6,189 - 7,360] | [9,473 - 10,317 -13,568] | [17,729 - 18,573 - 25,984]
$h=128$ | [8,593 - 10,269 - 10,528] | [12,705 - 14,381 - 16,704] | [20,929 - 22,605 - 29,056] | [37,377 - 39,053 - 53,760]
$h=256$ | [25,361 - 28,701 - 29,216] | [33,569 - 36,909 - 41,536] | [49,985 - 53,325 - 66,176] | [82,817 - 86,157 - 115,456]
$h=512$ | [83,473 - 90,141 - 91,168] | [99,873 - 106,541 - 115,776] | [132,673 - 139,341 - 164,992] | [198,273 - 204,941 - 263,424]

Table 8: Number of model parameters for data dimensionality $d$ and hidden
dimension size $h$. The first entries are for all models except InfoGAN and
BEGAN, the second entries are for InfoGAN, and the third are for BEGAN. All
architectures consisted of four feedforward neural network layers in total:
two for $G$ and two for $D$. Since InfoGAN used latent variables as inputs to
$D$ and BEGAN’s $D$ was an autoencoder, they necessitated slightly more
parameters. We did not observe these differences to give neither InfoGAN nor
BEGAN any significant advantage over other models.


# Neck Rupture and Scission Neutrons in Nuclear Fission

Ibrahim Abdurrahman  Theoretical Division, Los Alamos National Laboratory, Los
Alamos, NM 87545, USA  Matthew Kafker  Department of Physics, University of
Washington, Seattle, WA 98195–1560, USA  Aurel Bulgac  Department of Physics,
University of Washington, Seattle, WA 98195–1560, USA  Ionel Stetcu
Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545,
USA

###### Abstract

Just before a nucleus fissions a neck is formed between the emerging fission
fragments. It is widely accepted that this neck undergoes a rather violent
rupture, despite no direct experimental evidence, and only a few contentious
theoretical treatments of this fission stage were ever performed in the more
than eight decades since nuclear fission was experimentally observed by Hahn
and Strassmann and described by Meitner and Frisch in 1939. In the same year,
Bohr and Wheeler conjectured that the fission of the nuclear liquid drop would
likely be accompanied by the rapid formation of tiny droplets, later
identified with either scission neutrons or other ternary fission fragments, a
process which has not yet been discussed in a fully quantum many-body
framework. The main difficulty in addressing both of these stages of nuclear
fission is both are highly non-equilibrium processes. Here we will present the
first fully microscopic characterization of the scission mechanism, along with
the spectrum and the spatial distribution of scission neutrons, and some upper
limit estimates for the emission of charged particles.

††preprint: NT@UW-23-08,LA-UR-23-25884

Nuclear fission was experimentally discovered by Hahn and Strassmann (1939) in
1939. Later in 1939, it was named and its main mechanism was explained by
Meitner and Frisch (1939). It is a quantum many-body process of extreme
complexity, with various parts of the process occurring at vastly different
timescales. The total time it takes from the moment a neutron initiates the
formation of a compound nucleus until all final fission products have attained
their equilibrium state after $\beta$-decay, can be as long as the life-time
of the Universe ${\cal O}(10^{17})$ sec., and spans almost 40 orders of
magnitude relative to the time it takes a nucleon to cross a nucleus ${\cal
O}(10^{-22})$ sec.

The compound system, formed by a low-energy neutron Hahn and Strassmann (1939)
interacting with a target nucleus, evolves through many distinct stages. The
first stage is a relatively slow quasi-equilibrium evolution, that lasts until
the compound system Bohr (1936) reaches the outer saddle-point at $\approx
10^{-14}$ sec. Gönnenwein (2014). During this stage, the nucleus, with an
initial prolate intrinsic shape and axial symmetry, evolves into a nucleus
with triaxial shape, and eventually into a reflection asymmetric and axially
symmetric elongated shape near the outer fission barrier Ryssens  _et al._
(2015), these being clear examples of quantum phase transitions. The second
stage is a highly non-equilibrium evolution from saddle-to-scission Bulgac
_et al._ (2016, 2019, 2020), when the primordial fission fragments (FFs)
properties are defined within a duration of $\approx 5\times 10^{-21}$ sec.
Gönnenwein (2014). Even though this second stage is much faster than the first
stage, it corresponds to rather slow dynamics, relative to the third stage
(scission). In this stage, the compound nucleus undergoes a relatively rapid
“divorce” into two FFs, lasting $\approx 10^{-22}$ sec. This stage is also
known in literature as the neck rupture and is another example of a quantum
phase transition. This is followed by a fourth stage in which the FFs are
Coulomb accelerated during an interval of time of ${\cal O}(10^{-18})$ sec.,
when the FFs achieve an equilibration of their shapes. In spite of the fact
that the initial compound nucleus is a relatively cold system with a very
small spin, the primordial FFs are very hot and have relatively large spins as
well Bulgac  _et al._ (2019, 2021). These highly excited FFs emit prompt
neutrons for an interval of time up to about ${\cal O}(10^{-14})$ sec.,
followed by the emission of the majority of prompt $\gamma$-rays for an
interval of time until about ${\cal O}(10^{-3})$ sec., which is further
followed by much slower $\beta$-decays, virtually to times of the order of the
life-time of the Universe. Other processes are also possible, such as delayed
neutron or $\gamma$ emission after $\beta$-decay, or fission after
$\beta$-decay.

With the exception of the saddle-to-scission configuration and the neck
rupture, all the other stages of fission are relatively slow quasi-equilibrium
processes. The fission dynamics after the compound nucleus reaches to the
outer saddle point historically has typically been described in terms of the
potential energy surface of the nucleus, determined by its shape Gönnenwein
(2014); Ring and Schuck (2004); Krappe and Pomorski (2012); Schunck and
Robledo (2016), not compatible with either experiment or recent microscopic
studies Bulgac  _et al._ (2016, 2019); M. Bender et al. (2020). Approximately
at the top of the outer saddle, the nucleus starts forming a barely seen
“wrinkle”, where eventually the neck between the two FFs is formed. This
“wrinkle” tends to appear when the fissioning nucleus is at a young age, and,
as our results below will show, their position hardly changes in time, at
least until the FFs fully separate. This is similar to the formation of “skin-
wrinkles” in humans. At the top of the outer saddle the nucleus starts a
relatively slow dissipative evolution towards scission Bulgac  _et al._
(2016). In this period, the fissioning nucleus becomes more elongated and the
neck becomes more and more pronounced. The nuclear fluid behaves as nuclear
molasses, with a very small collective velocity Bulgac  _et al._ (2016, 2019,
2020), while at the same time the intrinsic temperature of the system
gradually increases. The bond between the two partners slowly weakens until
the neck, which was still keeping them together, reaches a critical small
diameter of approximately 3 fm and ruptures, exactly where the initial
“wrinkle” formed much earlier at the top of the outer saddle. This dramatic
“divorce” of the two emerging FFs is a rather short-time event. For Brosa  _et
al._ (1990) scission was the defining stage of fission, where the total
kinetic energy (TKE) of the FFs is defined along with the average FF
properties. The Brosa model assumes that the nucleus is a very viscous fluid,
with a long neck that ruptures at a random position, and is widely invoked
today in many models, even though it has no microscopic foundation and its
claimed grounding in experimental data do not necessarily support a unique
interpretation. Additionally, the Brosa random neck rupture model contradicts
the theoretical assumptions of other popular approaches, such as the scission-
point model of Wilkins  _et al._ (1976), where the FF formation is based on
statistical equilibrium Lemaître  _et al._ (2015, 2021), and the recently
updated Brownian motion model Albertsson  _et al._ (2020). The drama of
scission is followed by unavoidable debris characteristic of such dramatic
separations, the scission neutrons, envisioned as early as 1939 by Bohr and
Wheeler (1939). Potentially other heavier fragments, usually termed as ternary
fission products Vandenbosch and Huizenga (1973); Rose and Jones (1984);
Wagemans (1991), are created as well. We relegate a brief review of the
history of scission neutrons as an online supplementary material sup (2023).

Figure 1: The left/right columns show a time series of the neutron/proton
number densities in log scale for a typical fission trajectory. The bar
relates the color to the decimal logarithm of the number density.

Figure 2:  In the upper panel we display the integrated nucleon density along
the fission axis $n_{z,\tau}(t)=\int dxdy\,n_{\tau}(x,y,z,t)$ at several
times; before scission at -258.53 fm/c (black lines), when the neck is barely
formed; after scission at 129.27 fm/c (red lines); and after the FFs separated
respectively at 517.06 fm/c (blue lines). The position of the initial
“wrinkle”, where the neck ruptures and the FFs eventually separate, is
determined well before the neck is formed. Neutrons/protons are represented
via solid/dashed lines respectively. The simulation is started when the
compound nucleus is near the top of the outer barrier. In the bottom panel,
the nucleon number density integrated over the cross section of the neck as a
function of time. A fit around the scission time, shows that integrated over
the cross section of the neck neutron number density decay exponentially,
$n_{\mathrm{neck},\tau}(t)=\int\\!\\!dxdy\,n_{\tau}(x,y,0,t)\sim\exp(-t/\tau)$,
with $\tau\approx 35.0\pm 2.2$ fm/c for neutrons and $15.3\pm 0.3$ fm/c for
protons.  Figure 3: The solid/dashed regions show the number of scission
neutrons/protons respectively; $\tau=n,p$. The shaded regions represent the
standard deviation from considering various trajectories. $N_{p,sci}$ was
enhanced by a factor of 10.  Figure 4: The inset shows the total kinetic
energy of the scission neutrons vs time. The shaded regions represent the
standard deviation from considering various trajectories. The main panel shows
the scission neutron kinetic energy distribution with error bars corresponding
to different trajectories. The distributions are normalized to the total
number of scission neutrons i.e.
$\sum\mathrm{P}(\langle\epsilon_{n}\rangle)\times\frac{\Delta
E}{\mathrm{MeV}}=N_{\mathrm{sci}}$, with $\Delta E=2$ MeV.

As recognized by Meitner and Frisch (1939) nuclear fission must proceed from
an initially relatively compact shape with positive Gaussian curvatures
everywhere, to an intermediate shape, when the neck is formed, with a negative
Gaussian curvature, before the systems breaks up into two separate fission
fragments, both with positive Gaussian curvatures everywhere. (For any 3D
smooth surface, the Gaussian curvature is defined as the product of the
inverse principal curvature radii.) The nuclear shape changes from positive
Gaussian curvature and axially symmetric, to axially asymmetric, back to
axially symmetric Ryssens  _et al._ (2015), to negative Gaussian curvature and
axially symmetric shape, and eventually to shapes with positive overall
Gaussian curvatures, which are all examples of quantum phase transitions. This
has been demonstrated in a real-time description of nuclear fission dynamics,
starting at the top of the outer barrier Bulgac  _et al._ (2016).

In these simulations, we started by placing the initial compound nucleus near
the top of the outer barrier in a very large simulation volume, in order to
allow to the emitted nucleons enough time to decouple from the FFs after the
neck rupture. We have performed a range of simulations for 235U(nth,f),
239Pu(nth,f), and 252Cf(sf), using the nuclear energy density functional
(NEDF) SeaLL1 Bulgac  _et al._ (2018a) in simulation volumes $48^{2}\times
120$ and $48^{2}\times 100$ fm3, with a lattice constant of 1 fm, for further
technical details see Ref. Jin  _et al._ (2020). The SeaLL1 NEDF is defined by
only 8 basic nuclear parameters, each related to specific nuclear properties
known for decades, and contains the smallest number of phenomenological
parameters of any NEDF to date Bulgac  _et al._ (2018a, 2023). We started the
simulations at various points near the outer fission barrier, by constraining
the system deformations $Q_{20},Q_{30}$, see Refs. Bulgac  _et al._ (2016,
2019, 2020), where one can find more details about how the final results vary
with the choice of initial conditions. Our simulation volume of $48^{2}\times
120$ fm3 required the use of the entire supercomputer Summit, and we still
could not follow the emission of nucleons for a long time, since the emitted
nucleons are reflected back at the boundary relatively rapidly, see the lowest
row of Fig. 1. In the transversal direction the reflection from the boundaries
occurs earlier than along the fission axis, and that has effected some of the
properties of the nucleons emitted perpendicular to the fission axis, however
the effect on the total number of nucleons emitted is minor. Additionally, the
number of scission nucleons is not large enough to noticeably alter the FF
properties.

From here, we will concentrate on the dynamics of the neck formation and
rupture, followed by the emission of nucleons, all treated within the time-
dependent density functional extended to superfluid fermionic systems Bulgac
(2019). The integrated neck density, shown in Fig. 2, is defined as

$\displaystyle
n_{\mathrm{neck},\tau}(t)=\int\\!\\!dxdy\,n_{\tau}(x,y,z_{\mathrm{neck}},t),\quad\tau=n,\,p,$
(1)

separately for neutrons and protons, where the $z_{neck}$ is the position
along the fission axis $Oz$ where the neck has the smallest radius. The neck
decays relatively slowly at scission, until its diameter reaches about 3 fm,
after-which it undergoes a very rapid decay. Different curves illustrated in
the lower panel correspond to trajectories started at various initial
conditions for the deformations $Q_{20},Q_{30}$ close to the outer fission
barrier. The time to reach scission can vary significantly, depending on the
initial values of the deformations $Q_{20},Q_{30}$ and on the NEDF used,
typically ranging from 1,000 to 3,000 fm/c.

These microscopic results illustrate several points, which were unknown until
now, due to the absence of any detailed quantum many-body simulation of
fission dynamics. First, the “wrinkle” in the nuclear density, where the neck
is eventually formed and where the nucleus eventually scissions, is determined
a long time before the nucleus reaches scission. Within the TDDFT framework,
it is not random, and instead is determined by the quasiparticle energy
spectrum and/or the shell structure around the outer fission barrier, where
the trajectory was initiated. At the time when the neck reaches a critical
diameter of $\approx 3$ fm, the nuclear surface tension and the shape of the
compound around the neck region, can no longer counteract the strong Coulomb
repulsion between the preformed FFs causing the system to snap. One should
keep in mind that as intrinsic temperature of the compound nucleus increases
the surface tension decreases. The geometry of the nuclear shape changes
dramatically at this stage, from exhibiting a neck region where the Gaussian
curvature is negative, to two separated FFs with surfaces characterized by
predominantly positive Gaussian curvatures.

Second, the proton neck completes its rupture earlier than the neutron neck
does, see lower panel in Fig. 2, resulting in the neck being mostly sustained
by the neutrons just before the full rupture. In this time interval, the
number of neutrons per unit area at the neck varies by an order of magnitude.
The protons in the emerging FFs separate about 50-100 fm/c before the neutrons
neck ruptures. Additionally, the integrated neutron and proton densities at
$z_{\mathrm{neck}}$ asymptotically reach almost equilibrium values, after the
neck ruptures.

Third, the rupture is unarguably the fastest stage of the fission dynamics,
starting from the capture the incident neutron and formation of the compound
nucleus, until all fission products have emitted. The decay times are 15 fm/c
and 35 fm/c for proton and neutron necks respectively, which are significantly
faster processes than the time it takes the fastest nucleon to communicate any
information or facilitate any kind of equilibrium between the two preformed
FFs, which is at minimum $\sim 160$ fm/c.

Fourth, the neck decay dynamics displays a clear universality irrespective of
the initial conditions, as shown in the lower panel of Fig. 2. The time to
full FF separation, the FF masses and charges, FF excitation energies, and the
TKE all vary significantly Bulgac  _et al._ (2019, 2020), however, the neck
dynamics is essentially unchanged for both proton and neutron components.

Last, the scission mechanism emerging from a fully microscopic treatment of
the fission dynamics is totally at odds with previous models, including the
Brosa random rupture model and the scission-point models. TDDFT extended to
superfluid systems is the only theoretical framework so far in literature in
which the scission is treated without any unchecked assumptions or fitting
parameters, fitted to achieve agreement with data.

Consider the case of a long neck described in the Brosa model, whose existence
was argued for in order to describe both TKE and FF yields distributions. If a
long neck was to randomly rupture somewhere in the middle, the surface of the
emerging FFs would initially have at least one rather long “nose,” where the
Gaussian curvature is negative. These noses would eventually have to be
swallowed into the FFs, leading to a particle current lasting a noticeable
time interval. This involves a larger number of particles, thus more scission
nucleons, longer equilibration times, and would also cause the FFs
deformations to relax at a later time. Before particle equilibration can be
reached however, the presence of a long remnant neck in the FFs and its re-
absorption would likely lead to a significant number of nucleons being emitted
in the direction of motion of each fragment, an aspect which will discuss in
more detail below.

In reality, the neck rupture is a very fast “healing” process of the nuclear
surface in the neck region. Unlike a gas in a punctured balloon, which would
rapidly escape the enclosure, due to the presence of the nuclear “skin” and
strong surface tension, the nucleus behaves as a fluid. The surface tension
quickly “heals” the “wound,” however a small fraction of matter manages to
escape like a gas, with no droplet formation, see Fig. 1. The potential
condensation of this emitted gas cannot be described within the present
framework. In Fig. 1 we show several representative frames for neutrons and
protons of the neck formation and emission of nucleons. The most remarkable
features of this process are the following.

As visible in Fig. 2 the proton neck completes its rupture before the neutron
neck, however in two stages. Immediately at the scission, which in Fig. 1 is
identified with the rupture of the proton neck, a number of nucleons are
emitted in the plane perpendicular to the fission axis (see second time
frame). After a sufficient time for nucleons to propagate from the neck to the
nose of each FF, scission nucleons also appear propagating in front of each
FF. In 1984, it was suggested by Mädler (1985), that the reabsorption of the
neck stumps by the FFs, being a relatively rapid process, could act as a
“catapult”, which is more appropriately described as a slingshot, and “push”
nucleons out of the front of the FFs. It is also important to note, the three
neutron clouds, two in front of the FFs and one transverse ring perpendicular
to the fission axis, appears across all considered trajectories and nuclei
hence far.

Remembering that the TKE is roughly 171-186 MeV, at the end of the full
Coulomb acceleration the light and heavy FFs will have an average kinetic
energy per nucleon of about 1 MeV and 0.5 MeV respectively, which is
significantly lower than the average kinetic energy of the scission neutrons,
see Fig. 4, given by 3.51 $\pm$ 0.25 MeV, 3.42 $\pm$ 0.27 MeV, and 2.67 $\pm$
0.24 MeV for 236U, 240Pu, and 252Cf respectively. As a result, the FFs will
never have a chance to catch up with them. Additionally, the scission neutrons
are essentially free, since their total interaction energy, estimated via
Bulgac  _et al._ (2018b),

$E_{n}^{\mathrm{int}}=\int
dV[a_{n}n_{n}^{5/3}+b_{n}n_{n}^{2}+c_{n}n_{n}^{7/3}]\ll E_{n}^{\mathrm{kin}},$
(2)

comprises less than 1 percent of their kinetic energy.

The total number of emitted neutrons is about $0.30\pm 0.05$, $0.26\pm 0.05$,
and $0.55\pm 0.02$ per fission event for 236U, 240Pu, and 252Cf respectively,
which is a considerable portion (roughly 15 %) of the total emitted prompt
neutrons, see Refs. Vandenbosch and Huizenga (1973); Wagemans (1991); Bulgac
_et al._ (2019, 2020); Stetcu  _et al._ (2021). At the same time the number of
emitted protons is about two orders of magnitude lower, see Fig. 3. The
nucleons are emitted in roughly equal numbers both transverse to the scission
axis and in front of the FFs.

Here, we presented the first fully microscopic study of the neck in fission
dynamics and the emission of scission nucleons, performed without any
uncontrollable approximations and unchecked assumptions. We have clarified
several aspects of, what is arguably, the most non-equilibrium and fastest
stage of nuclear fission dynamics. Within TDDFT, the neck rupture is not a
random process, as previously argued in various phenomenological models.
Additionally, it appears that the neck rupture has universal dynamics,
irrespective of nucleus considered or the initial conditions, beyond the top
of the outer fission barrier. This universality carries over to the emission
of scission neutrons, whose signal always appears as three distinct clouds,
one transverse to the fission axis and two in front of each FF, in almost
equal proportions. The aspects of the neck dynamics discussed above, can serve
as a firm theoretical input for any semi-phenomenological approach to study FF
properties Becker  _et al._ (2013); Vogt  _et al._ (2009); Litaize  _et al._
(2012). Additionally, we have established that an average of about 15 % of the
total neutrons are emitted at scission per fission event. The idea of scission
neutrons, proposed by Bohr and Wheeler (1939), is almost as old as nuclear
fission itself. The existence of scission neutrons has been debated over the
years Wilson (1947); Debenedetti  _et al._ (1948); Stavinsky (1959); Fuller
(1962); Bowman  _et al._ (1962); Kapoor  _et al._ (1963); Skarsvåg and
Bergheim (1963); Gavron and Fraenkel (1974); Boneh and Fraenkel (1974);
Pringle and Brooks (1975); Franklyn  _et al._ (1978); Franklyn (1986); Budtz-
Jørgensen and Knitter (1988); Milek  _et al._ (1988); Brosa and Knitter
(1992); Samant  _et al._ (1995); J. K. Hwang et al. (1999); Carjan  _et al._
(2007); Rizea  _et al._ (2008); Carjan and Rizea (2010, 2015); Guseva  _et
al._ (2018); Vorobyev  _et al._ (2018); Carjan and Rizea (2019), see also
Historical Note in Ref.Wagemans (1991), and their experimental confirmation is
still an open question. This small fraction of scission neutrons carry
noticeable kinetic energy, and what is more surprising is that their energy
spectrum ranges up to roughly 18 MeV. Along with scission neutrons, a very
small fraction of protons are emitted as well, and their fraction suggest a
reasonable theoretical upper bound for the emission of $\alpha$-particles and
other charged nuclei.

Acknowledgements

We thank Kyle Godbey and Guillaume Scamps for many useful discussions. The
work of I.A. and I.S. was supported by the U.S. Department of Energy through
the Los Alamos National Laboratory. The Los Alamos National Laboratory is
operated by Triad National Security, LLC, for the National Nuclear Security
Administration of the U.S. Department of Energy Contract No.
89233218CNA000001. I.A. and I.S. gratefully acknowledges partial support and
computational resources provided by the Advanced Simulation and Computing
(ASC) Program. The work of M.K. and A.B. was supported by the US DOE, Office
of Science, Grant No. DE-FG02-97ER41014 and also partially by NNSA cooperative
Agreement DE-NA0003841, and is greatly appreciated. This research used
resources of the Oak Ridge Leadership Computing Facility, which is a U.S. DOE
Office of Science User Facility supported under Contract No. DE-
AC05-00OR22725.

## References

  * Hahn and Strassmann (1939) O. Hahn and F. Strassmann, “Über den Nachweis und das Verhalten der bei der Bestrahlung des Urans mittels Neutronen entstehenden Erdalkalimetalle,” Naturwissenschaften 27, 11 (1939).
  * Meitner and Frisch (1939) L. Meitner, L. and O. R. Frisch, “Disintegration of Uranium by Neutrons: a New Type of Nuclear Reaction,” Nature 143, 239 (1939).
  * Bohr (1936) N. Bohr, “Neutron Capture and Nuclear Constitution,” Nature 137, 344 and 351 (1936).
  * Gönnenwein (2014) F. Gönnenwein, “Neutron and gamma emission in fission,” LANL Fiesta 2014 Lectures (2014).
  * Ryssens  _et al._ (2015) W. Ryssens, P.-H. Heenen, and M. Bender, “Numerical accuracy of mean-field calculations in coordinate space,” Phys. Rev. C 92, 064318 (2015).
  * Bulgac  _et al._ (2016) A. Bulgac, P. Magierski, K. J. Roche, and I. Stetcu, “Induced Fission of ${}^{240}\mathrm{Pu}$ within a Real-Time Microscopic Framework,” Phys. Rev. Lett. 116, 122504 (2016).
  * Bulgac  _et al._ (2019) A. Bulgac, S. Jin, K. J. Roche, N. Schunck, and I. Stetcu, “Fission dynamics of ${}^{240}\mathrm{Pu}$ from saddle to scission and beyond,” Phys. Rev. C 100, 034615 (2019).
  * Bulgac  _et al._ (2020) A. Bulgac, S. Jin, and I. Stetcu, “Nuclear Fission Dynamics: Past, Present, Needs, and Future,” Frontiers in Physics 8, 63 (2020).
  * Bulgac  _et al._ (2021) A. Bulgac, I. Abdurrahman, S. Jin, K. Godbey, N. Schunck, and I. Stetcu, “Fission fragment intrinsic spins and their correlations,” Phys. Rev. Lett. 126, 142502 (2021).
  * Ring and Schuck (2004) P. Ring and P. Schuck,  _The Nuclear Many-Body Problem_ , 1st ed. (Springer-Verlag, Berlin Heidelberg New York, 2004).
  * Krappe and Pomorski (2012) J. K. Krappe and K. Pomorski,  _Theory of Nuclear Fission_ (Springer Heidelberg, 2012).
  * Schunck and Robledo (2016) N. Schunck and L. M. Robledo, “Microscopic theory of nuclear fission: a review,” Rep. Prog. Phys. 79, 116301 (2016).
  * M. Bender et al. (2020) M. Bender et al., “Future of nuclear fission theory,” Journal of Physics G: Nuclear and Particle Physics 47, 113002 (2020).
  * Brosa  _et al._ (1990) U. Brosa, S. Grossmann, and A. Müller, “Nuclear scission,” Physics Reports 197, 167 (1990).
  * Wilkins  _et al._ (1976) B. D. Wilkins, E. P. Steinberg, and R. R. Chasman, “Scission-point model of nuclear fission based on deformed-shell effects,” Phys. Rev. C 14, 1832 (1976).
  * Lemaître  _et al._ (2015) J.-F. Lemaître, S. Panebianco, J.-L. Sida, S. Hilaire, and S. Heinrich, “New statistical scission-point model to predict fission fragment observables,” Phys. Rev. C 92, 034617 (2015).
  * Lemaître  _et al._ (2021) J.-F. Lemaître, S. Goriely, A. Bauswein, and H.-T. Janka, “Fission fragment distributions and their impact on the $r$-process nucleosynthesis in neutron star mergers,” Phys. Rev. C 103, 025806 (2021).
  * Albertsson  _et al._ (2020) M. Albertsson, B.G. Carlsson, T. Døssing, P. Möller, J. Randrup, and S. Åberg, “Excitation energy partition in fission,” Phys. Lett. B 803, 135276 (2020).
  * Bohr and Wheeler (1939) N. Bohr and J. A. Wheeler, “The Mechanism of Nuclear Fission,” Phys. Rev. 56, 426–450 (1939).
  * Vandenbosch and Huizenga (1973) R. Vandenbosch and J. R. Huizenga, “Nuclear Fission,” Academic Press, New York (1973).
  * Rose and Jones (1984) H. J. Rose and G. A. Jones, “A new kind of radioactivity,” Nature 307, 245 (1984).
  * Wagemans (1991) C. Wagemans, ed.,  _The Nuclear Fission Process_ (CRC Press, Boca Raton, 1991).
  * sup (2023) “See Supplemental Material for a historical overview.” (2023).
  * Bulgac  _et al._ (2018a) A. Bulgac, M. M. Forbes, S. Jin, R. N. Perez, and N. Schunck, “Minimal nuclear energy density functional,” Phys. Rev. C 97, 044313 (2018a).
  * Jin  _et al._ (2020) S. Jin, K. J. Roche, I. Stetcu, I.Abdurrahman, and A. Bulgac, “The LISE package: solvers for static and time-dependent superfluid local density approximation equations in three dimentions,” Comp. Phys. Comm. 269, 108130 (2020).
  * Bulgac  _et al._ (2023) A. Bulgac, M. Kafker, and I. Abdurrahman, “Measures of complexity and entanglement in many-fermion systems,” Phys. Rev. C 107, 044318 (2023).
  * Bulgac (2019) A. Bulgac, “Time-Dependent Density Functional Theory for Fermionic Superfluids: from Cold Atomic gases, to Nuclei and Neutron Star Crust,” Physica Status Solidi B 256, 1800592 (2019).
  * Mädler (1985) P. Mädler, “Catapult mechanism for fast particle emission in fission and heavy ion reactions,” Z. Physik A 321, 343 (1985).
  * Bulgac  _et al._ (2018b) A. Bulgac, M. M. Forbes, S. Jin, R. Navarro Perez, and N. Schunck, “Minimal nuclear energy density functional,” Phys. Rev. C 97, 044313 (2018b).
  * Stetcu  _et al._ (2021) I. Stetcu, A. E. Lovell, P. Talou, T. Kawano, S. Marin, S. A. Pozzi, and A. Bulgac, “Angular Momentum Removal by Neutron and $\gamma$-Ray Emissions during Fission Fragment Decays,” Phys. Rev. Lett. 127, 222502 (2021).
  * Becker  _et al._ (2013) B. Becker, P. Talou, T. Kawano, Y. Danon, and I. Stetcu, “Monte Carlo Hauser-Feshbach predictions of prompt fission $\gamma$ rays: Application to ${n}_{\mathrm{th}}+{}^{235}$U, ${n}_{\mathrm{th}}+{}^{239}$Pu, and 252Cf (sf),” Phys. Rev. C 87, 014617 (2013).
  * Vogt  _et al._ (2009) R. Vogt, J. Randrup, J. Pruet, and W. Younes, “Event-by-event study of prompt neutrons from ${}^{239}\mathrm{Pu}(n,f)$,” Phys. Rev. C 80, 044611 (2009).
  * Litaize  _et al._ (2012) O. Litaize, O. Serot, D. Regnier, S. Theveny, and S. Onde, “New Features of the FIFRELIN Code for the Investigation of Fission Fragments Characteristics,” Physics Procedia 31, 51 (2012).
  * Wilson (1947) R. R. Wilson, “Directional Properties of Fission Neutrons,” Phys. Rev. 72, 189 (1947).
  * Debenedetti  _et al._ (1948) S. Debenedetti, J. E. Francis, W. M. Preston, and T. W. Bonner, “Angular Dependence of Coincidences between Fission Neutrons,” Phys. Rev. 74, 1645 (1948).
  * Stavinsky (1959) V. S. Stavinsky, “On the emission mechanism of prompt fission neutrons,” JETP (Soviet Physics) 9, 437 (1959).
  * Fuller (1962) R. W. Fuller, “Dependence of neutron production in fission on rate of change of nuclear potential,” Phys. Rev. 126, 684 (1962).
  * Bowman  _et al._ (1962) H. R. Bowman, S. G. Thompson, J. C. D. Milton, and W. J. Swiatecki, “Velocity and angular distributions of prompt neutrons from spontaneous fission of Cf 252,” Phys. Rev. 126, 2120 (1962).
  * Kapoor  _et al._ (1963) S. S. Kapoor, R. Ramanna, and P. N. Rama Rao, “Emission of prompt neutrons in the thermal neutron fission of u 235,” Phys. Rev. 131, 283 (1963).
  * Skarsvåg and Bergheim (1963) K. Skarsvåg and K. Bergheim, “Energy and angular distributions of prompt neutrons from slow neutron fission of U235,” Nuclear Physics 45, 72 (1963).
  * Gavron and Fraenkel (1974) A. Gavron and Z. Fraenkel, “Neutron correlations in spontaneous fission of ${}^{252}\mathrm{Cf}$,” Phys. Rev. C 9, 632–645 (1974).
  * Boneh and Fraenkel (1974) Y. Boneh and Z. Fraenkel, “Dynamic single-particle effects in fission,” Phys. Rev. C 10, 893 (1974).
  * Pringle and Brooks (1975) J. S. Pringle and F. D. Brooks, “Angular Correlation of Neutrons from Spontaneous Fission of Cf 252,” Phys. Rev. Lett. 35, 1563 (1975).
  * Franklyn  _et al._ (1978) C. B. Franklyn, C. Hofmeyer, and D. W. Mingay, “Angular correlation of neutrons from thermal-neutron fission of 235U,” Phys. Lett. B 78, 564 (1978).
  * Franklyn (1986) C. B. Franklyn, “Neutron-fragment angular correlations in 235U (nth, f),” Radiation Effects 92, 323 (1986).
  * Budtz-Jørgensen and Knitter (1988) C. Budtz-Jørgensen and H.-H. Knitter, “Simultaneous investigation of fission fragments and neutrons in 252Cf (SF),” Nuclear Physics A 490, 307 (1988).
  * Milek  _et al._ (1988) B. Milek, R. Reif, and J. Revai, “Model for particle emission from a fissioning system,” Phys. Rev. C 37, 1077 (1988).
  * Brosa and Knitter (1992) U. Brosa and H.-H. Knitter, “The scission neutron spectrum of 252CF (SF),” Z. Physik A 343, 39 (1992).
  * Samant  _et al._ (1995) M. S. Samant, R. P. Anand, R. K. Choudhury, S. S. Kapoor, and D. M. Nadkarni, “Prescission neutron emission in ${}^{235}\mathrm{U}$(${\mathit{n}}_{\mathrm{th}}$,f) through fragment-neutron angular correlation studies,” Phys. Rev. C 51, 3127 (1995).
  * J. K. Hwang et al. (1999) J. K. Hwang et al., “Search for scission neutrons in the spontaneous fission of ${}^{252}\mathrm{Cf}$,” Phys. Rev. C 60, 044616 (1999).
  * Carjan  _et al._ (2007) N. Carjan, P. Talou, and O. Serot, “Emission of scission neutrons in the sudden approximation,” Nuclear Physics A 792, 102 (2007).
  * Rizea  _et al._ (2008) M. Rizea, V. Ledoux, M. V. Daele, G. V. Berghe, and C. Nicolae, “Finite difference approach for the two-dimensional schrödinger equation with application to scission-neutron emission,” Comp. Phys. Comm. 179, 466 (2008).
  * Carjan and Rizea (2010) N. Carjan and M. Rizea, “Scission neutrons and other scission properties as function of mass asymmetry in U 235 (nth, f),” Phys. Rev. C. 82, 014617 (2010).
  * Carjan and Rizea (2015) N. Carjan and M. Rizea, “Similarities between calculated scission-neutron properties and experimental data on prompt fission neutrons,” Phys. Lett. B. 747, 178 (2015).
  * Guseva  _et al._ (2018) I. S. Guseva, A. M. Gagarski, V. E. Sokolov, G. A. Petrov, A. S. Vorobyev, G. V. Val’sky, and T. A. Zavarukhina, “Detailed investigations of neutron–neutron angular correlations in slow-neutron-induced fission of 233 U, 235 U, and 239 Pu,” Phys. Atom. Nuclei 81, 447 (2018).
  * Vorobyev  _et al._ (2018) A. S. Vorobyev, O. A. Shcherbakov, A. M. Gagarski, G. A. Petrov, G. V. Val’ski, and T. E. Kuz’mina, “Estimation of the Yield of “Scission” Neutrons from Thermal Neutron-Induced Fission of 239 Pu,” J. Exp. Theor. Phys. 127, 659 (2018).
  * Carjan and Rizea (2019) N. Carjan and M. Rizea, “Structures in the energy distribution of the scission neutrons: Finite neutron-number effect,” Phys. Rev. C. 99, 034613 (2019).