# Large Scale Qualitative Evaluation of Generative Image Model Outputs

Yannick Assogba, Adam Pearce and Madison Elliott Yannick Assogba, Adam Pearce
and Madison Elliot are with Google Research.
E-mail<EMAIL_ADDRESS>
###### Abstract

Evaluating generative image models remains a difficult problem. This is due to
the high dimensionality of the outputs, the challenging task of representing
but not replicating training data, and the lack of metrics that fully
correspond to human perception and capture all the properties we want these
models to exhibit. Therefore, qualitative evaluation of model outputs is an
important part of model development and research publication practice.
Quantitative evaluation is currently under-served by existing tools, which do
not easily facilitate structured exploration of a large number of examples
across the latent space of the model. To address this issue, we present Ravel,
a visual analytics system that enables qualitative evaluation of model outputs
on the order of hundreds of thousands of images. Ravel allows users to
discover phenomena such as mode collapse, and find areas of training data that
the model has failed to capture. It allows users to evaluate both quality and
diversity of generated images in comparison to real images or to the output of
another model that serves as a baseline. Our paper describes three case
studies demonstrating the key insights made possible with Ravel, supported by
a domain expert user study.

###### Index Terms:

Information visualization, Picture/Image Generation, Machine learning

##  1 Introduction

Generative image models are a class of neural network based models that aim to
produce novel, high-quality and diverse images that faithfully model a target
image distribution. A variety of architectures and training methods have been
designed to learn such models, such as Generative Adversarial Networks (GANs)
[1], Variational Auto-Encoders (VAEs) [2], Flow Based Models [3] and Diffusion
Models [4].

Evaluating these models remains difficult [5]. The high dimensionality of the
output architectures used make likelihood estimates of model outputs
difficult, and in some cases intractable. It has also been demonstrated that
measures like average log likelihood do not always correlate with human
perceptual judgments of sample quality [6]. Additionally, while we want models
to capture the target distribution well, we do not want them to produce images
that are actually in the training set (an issue commonly referred to as
memorization).

A number of metrics have emerged in the literature around generative image
models [7] [8], with Fréchet Inception Distance (FID) [9] being the most
popular. However, issues have been identified with FID, leading to the
development of more granular metrics such as precision and recall [10] [11].

Single-number metrics such as FID, while necessary for forward progress in the
field, do not capture the full range of qualities desired of these models.
Because of this, human visual inspection often plays a critical role in the
evaluation and dissemination of advances in generative image modeling. However
with existing evaluation tools, practitioners can typically only look at a
small fraction of the output space of these models, on the order of 10s to
100s of images (e.g., [12], [13], [14], [15], [16]).

Our interviews with domain experts confirm that human evaluation is a critical
part of practitioner workflows. Some experts rely on human evaluation with
crowd-sourced evaluators, they however recognize that these are often
expensive or time consuming and are thus left to the final stages of
evaluation if done at all, leaving them to primarily rely on small scale
qualitative evaluation during the model development process.

At the same time, experts in the field are concerned about cherry picking of
results for publication but typically have no means to expansively explore
model outputs in the rare occasions that these are published alongside
academic manuscripts.

To address these needs, we built a system called Ravel, which enables users to
perform visual inspection of model outputs on scales up to three orders of
magnitude greater that typical user workflows. We demonstrate usage of this
system on datasets varying from 50k - 120k images. These dataset sizes are
comparable to those used in standard quantitative evaluation of generative
image models.

Our primary contributions include:

  * •

A visual analytics system that supports multiple evaluation tasks (e.g.
evaluating quality & diversity, discovering mode collapse or gaps in model
output) for generative image models and is agnostic to model architecture and
internals.

  * •

Interactive exploration of large generative image model datasets, facilitated
by clustering and the use of fine grained visualization of cluster metrics to
guide qualitative evaluation.

  * •

A user interface that uses visual comparison driven by semantically meaningful
embedding spaces to support reasoning about differences between image
distributions and generate hypotheses about model behaviour.

##  2 Background

###  2.1 Generative Image Models

The capabilities of generative image models have greatly increased over the
last several years. Since the original GAN paper [1] that broke the dam on
modelling of faces, we now have systems like BigGAN [12], StyleGAN [13], GLOW
[14], VQ-VAE [15], CDM [16] and many others that present a wide variety of
model architecture and training algorithms and are capable of producing very
realistic images in a wide variety of domains.

###  2.2 Quantitative Metrics for Evaluating Generative Image Models

In this section, we outline the most commonly cited metrics in the research
literature:

  * •

Fréchet Inception Distance [9]: Uses a pre-trained InceptionV3 classifier [17]
to generate embeddings for both real and generated images, then uses a
statistical measure to compare the distribution of embeddings from the two
sources. FID is the most popular metric in the literature. It requires a large
number of samples to produce an accurate estimate (generally at least 50k
generated images), and cannot detect memorization of the training set. Karras
et al [18] point out that the texture bias in ImageNet based CNNs like
InceptionV3 [19] imply that metrics derived from them will not capture all
aspects of image quality.

  * •

Inception Score [20]: Uses a pre-trained inception classifier to measure, a)
how well each generated image matches a single ImageNet class, and b) if the
full the set of generated images has uniform coverage over all the ImageNet
classes. Similar to FID, Inception Score requires a fairly large number of
images and cannot be used to detect memorization. It also cannot measure
intra-class diversity or detect mode collapse. See Barrat and Sharma [21] for
a detailed discussion of issues with this metric. Both Inception Score and FID
are scalar scores designed to capture both image quality and diversity, and
thus cannot reveal if the model is trading off one of these properties for the
other to achieve a better score.

  * •

Precision and Recall Metrics: To disentangle the measurement of image quality
and diversity, Sajjadi et al [10] and Kynkäänniemi et al [11] propose
precision and recall metrics to measure each independently. Broadly speaking,
precision corresponds to the sample quality, whereas recall corresponds to the
coverage of the sample distribution with respect to the target distribution -
i.e. diversity.

These metrics are generally computed over the entire dataset, and thus have
low granularity. Even when they indicate that a model is better or worse, they
don’t specify where in the distribution of generated images improvements or
regressions lie. Ravel increases the granularity of these metrics, providing a
way to find specific clusters of images that score poorly on some metric

###  2.3 Qualitative Evaluation / Visualization of Generative Image Model
Outputs

Due to the limitations of quantitative metrics discussed above, researchers
also rely on visual inspection of model outputs to evaluate model performance.
Visual inspection is typically performed during training, to monitor that the
process has not immediately failed, as well as after training, to evaluate
overall quality of the model. We validate and elaborate on this workflow and
strategy with domain experts in Section 6.1. Although models often output 100s
of thousands of images, our user study found that researchers are only able to
inspect a small portion of images (in the 100s) during their analyses.

Visually impressive samples are paramount for successfully publishing model
advancements in scientific venues. Relevant papers in the field of generative
image models typically include 10s-100s of images [12] [13] [18] [22]. This
represents a very limited sample of the variety of images these models are
typically trained to generate. Our user study also found that there is an
assumption that authors ”cherry-pick” the best images to include in their
publications. Relatively few authors have published large datasets of un-
cherry-picked output images alongside their publications.

There are currently no purpose-built interfaces that make these convenient to
browse or examine. For example, [13] [18] each publish 100k output images to a
publicly accessible Google drive. These images are organized into 1000 sub-
folders to make the interface more usable given the large number of files. To
view this output, users must either click through the folders individually, or
bring their own interface to browse the images.

##  3 Related Work

Borji [7] [8] catalogues many of the metrics for automatic evaluation of
generative models. Ravel utilizes the precision and recall metrics from [10]
and [11], but does not propose any new metrics. We thus situate this work in
primarily relation to work on qualitative evaluation and interfaces to explore
generative model output.

Crowd-worker Evaluation

Denton et al. [23] use a small volunteer sample of human annotators to
estimate quality by asking whether they can distinguish real from generated
images. Zhou et al. [24] refine and scale this technique, asking crowd-sourced
workers from Amazon’s Mechanical Turk to make psychophysical judgments about
real vs. generated images. While these methods are good at scaling up
evaluation to larger dataset sizes, they are more time-consuming and expensive
than manual inspection by researchers. Thus they are typically reserved for
later stages of the evaluation pipeline. They also tend to focus on measures
that can be evaluated on individual images (e.g. image quality), rather that
corpus level properties (such as image diversity).

By contrast, Ravel is designed to support researcher evaluation earlier in the
development pipeline before the use of external raters, and allows researchers
to evaluate both quality and diversity in the same interface. It fits in
between initial monitoring of training dynamics to ensure that the model is
converging and larger scale human rater evaluation typically performed closer
to model release.

Explaining Model Internals

Bau et al. [25] explore finding interpretable units (neurons) within GANs and
visualizing the causal effects of ablating these neurons. Their method depends
on having access to model internals and having a pre-trained object
segmentation network to find objects within the scene to establish the casual
relationship between neuron activation and network output. [26] Bau et al. use
a pre-trained segmentation network to compare the distribution of objects
found in generated images with those found in a set of real images. This
provides a measure of diversity of the models outputs with respect to the
objects that can be segmented by the pre-trained network. The authors also
propose a method (Layer Inversion), to train networks that compute approximate
inversions of real images into the latent space of the model, to see what the
network generates instead of the missing objects.

While these approaches are critical to better understanding of the internal
mechanisms that drive model behavior, they are typically specific to a
particular model architecture. Ravel treats models as black boxes and is thus
agnostic to model architecture. Ravel does use pre-trained networks to compute
vector representations of images, but is less sensitive to the final task the
pre-trained network is trained to perform. For example one could use the
embeddings from the model under examination or any model that has learned
semantically useful features such as InceptionV3.

Online Exploration of Model Outputs

White [27] explores a variety of ways to sample images from latent space that
enable repeatable visual comparisons between models. They introduce a number
of visualizations designed to examine how models perform with respect to
specific input images that are used to test model behaviour. In a follow-up
work White & Loh [28] introduce a novel visual interface based on a
spreadsheet metaphor that allows users to use geometric operations in the
latent space to interactively query these models and thus explore their
output.

While online methods enable users to explore specific hypotheses, they
generally suffer from supporting relatively small exploration spaces due to
the slow generation of images. Ravel focuses on offline analysis of generated
images, which allows examining much larger datasets and can thus complement
online methods as means of generating hypotheses for further, more targeted
investigation.

Embedding Based Visualizations

A number of works have visualized embedding spaces of large, high-dimensional
datasets [29] [30] [31] [32].

Liu et al. [33] present a visual analysis system which uses the latent space
learned by the encoder of a VAE to explore variation in an existing image
dataset. This task is conceptually similar to what we support in Ravel,
however we do not attempt to learn a new latent space as we want to focus on
the generator output and its latent space (rather than fixed input data).

Ravel builds on these earlier works visualizing embedding spaces, and
incorporates clustering to make navigating these spaces more tractable. We
focus on dataset comparison as a way to ground exploration of the output space
and cater to the needs of the image generation use case.

Xiang et al. [34] present a visual analytics system for correcting labelling
errors in large datasets. They visualize t-SNE projections of image embeddings
color coded by label to highlight data points that are mislabelled. In
contrast, Ravel is designed to support unconditional generation scenarios,
where the generated (and ground truth images) have no labels. Thus, rather
than relying on labels for grounding, Ravel enables comparative evaluation of
datasets to allow grounding evaluation of generated images in the distribution
of real images.

##  4 Approach & System Design

Figure 1: The Ravel interface primarily consists of: A) Dataset & view
options. B) Summary charts & linked cluster plots. C) Side by side image grids
for visual comparison of clusters. This view shows a cluster comparing real
images on the left to generated images on the right.

Ravel focuses on enabling visual inspection of model outputs in comparison
with some baseline set of images, typically real ’ground truth’ images. From
our interviews (section 6.1), we know that practitioners perform visual
analysis of image samples coming out of models, but typically on small numbers
of images. Our aim is to support and scale up those workflows to allow more
systematic visual inspection from 10s or 100s of images to 10,000s or 100,000s
of images. We leverage four key concepts to facilitate this comparison across
a large set of images:

  1. 1.

Image Embeddings: Neural image embeddings provide a semantically rich vector
space for images that allow computing similarity scores between pairs of
images [35]. They have become the standard in evaluation pipelines for
generative image models (section 4.1.2) and we want to leverage the
familiarity researchers have with embedding based metrics in Ravel. The
ability to compute semantic similarity between images allows us to group
similar generated images together and allow comparison of those images to
similar ground truth images. By default we compute the standard InceptionV3
embeddings used in metrics like FID, but can also add embeddings from other
models including pre-trained models or from the model under examination if
those are available.

  2. 2.

Clustering Images: In order to enable visual exploration of hundreds of
thousands of images we need to group them into a smaller number of meaningful
groups. We use unsupervised clustering of images to reduce the number of top-
level items the user has to consider from e.g. 100k images to 1000, 500 or
even 250 clusters. We use k-means clustering and provide more details about
this in the pipeline details section below

  3. 3.

Cluster Metrics: Once we have reduced the number of top-level elements to a
manageable number we wish to provide hints to the user as to which clusters
might be most interesting to explore first, as well as scaffold a repeatable
workflow that can guide exploration. We compute metrics over each cluster that
enable sorting clusters into predictable order. Many of the metrics we compute
are designed to surface differences between the generated data and the
baseline data in a cluster, with others show general properties of the cluster
itself. We provide more details about the metrics we compute in the pipeline
details section below. By sorting clusters by these metrics we allow discovery
of outlier clusters as well as analyzing properties of different parts of the
generative image space (e.g. seeing what kinds of output images have low
precision)

  4. 4.

Interactive Visualization: Finally we provide responsive interactive
visualization of images, clusters and associated cluster metrics. Because we
compute all the data offline, the user interface is quite responsive and
enable fast and free exploration of a large number of data-points.

###  4.1 Pipeline Details

####  4.1.1 Clustering

Ravel clusters the image embeddings using the implementation of k-means
clustering from scikit-learn. k-means was an attractive clustering algorithm
to start with because it is a well known algorithm that has a single easily
understood hyper-parameter, namely the number of clusters. This allows the
user to directly specify values for this hyper-parameter that they believe
make sense for their dataset. In addition to this, compared to other
algorithms with similar hyper parameter options such as spectral clustering,
k-means scales well with the number of examples and number of clusters
selected [36]. k-means also tends to produce fairly evenly sized clusters,
which is helpful for displaying their contents in a uniform way.

All clustering methods suffer from issues of imperfect cluster assignment,
particularly at the boundaries of clusters. However, Ravel mitigates this
issue somewhat by visualizing clusters according to their positions in latent
space. This allows users to quickly discover and examine clusters that are
near to each other and ascertain whether they are truly a single semantic
cluster. For more details see the dimensionality reduction section (4.1.3)
below.

####  4.1.2 Cluster Metrics

We compute a number of metrics for each cluster. Having multiple metrics
provides different lenses through which to consider the clusters. For example
one might be interested in where recall or precision are low, or alternatively
where clusters are tightly packed or more spread out.

Clusters may have images from either the generated data or the baseline/ground
truth, in the UI we refer to these data sources as splits and typically
display the baseline data on the left, though the user can change this
interactively. Many metrics are geared toward exposing differences between the
two splits that are currently being explored.

  * •

Percent of Split 2: The percentage of images in the cluster that come from the
split displayed on the right (usually the generated data).

  * •

Recall: Recall is a metric that describes what fraction of samples in the
ground truth split have support in the alternate split. We compute recall as
defined in [11] but aggregate the per-image values for each cluster. When
comparing generated images to real ones a high recall implies the model is
producing images as diverse as the real data.

  * •

Precision: Precision is an metric that describes what fraction of samples in
the generated data have support in the ground truth. As above we compute
precision as defined in [11] but aggregate the per-image values for each
cluster. When comparing generated images to real ones a high precision implies
better image quality/realism.

  * •

Distance between split centroids: Measures the distance between centroids of
the samples in a cluster that belong to each split. Larger values suggest that
the data from the two splits are more visually different while smaller values
suggest that the left and right split are more visually similar.

  * •

Median distance to centroid: Median distance of samples in the cluster to the
centroid of that cluster. Smaller values here imply more compact clusters with
more similar samples, higher values suggest the cluster has a greater variety
of images.

####  4.1.3 Dimensionality Reduction

In addition to visualizing clusters by the metrics described above, we also
visualize the positions of cluster centroids in the embedding space itself.
Since these are high dimensional embeddings spaces we need to use
dimensionality reduction in order to visualize them in 2D. We use the UMAP
algorithm [37] to do this projection. Other options for dimensionality
reduction include PCA [38] or t-SNE [39]. Compared to t-SNE, UMAP has been
reported to preserve more of the global structure of the original space and is
significantly more computationally efficient both as the number of dimensions
increases and as the size of the dataset grows [37] [40]. The InceptionV3
embedding we use is 2048 dimensional embedding, making a computationally
efficient method particularly attractive. While not as directly interpretable
as linear methods like PCA, UMAP is better able to capture some of the complex
non-linear relationships between images encoded in the embedding space.

###  4.2 User Interface

The user interface is a browser-based application. We describe the design of
the user interface and further discuss the utility of these affordances in the
case study section.

Figure 1 shows the Ravel interface, divided into 3 main sections:

####  4.2.1 (A): Dataset and View Controls

The user can select which embedding to use, the number of clusters and which
split to show on the left or right.

####  4.2.2 (B): Charts

There are three main kinds of data display in the charts section of the UI.

Summary Statistics

First is a static table showing summary statistics and metrics for the dataset
as a whole. These include things like the number of samples in the whole
dataset, the number of samples in each split as well as dataset level metrics
such as Fréchet distance111When using the Inceptionv3 embedding this is
Fréchet Inception Distance (FID), recall and precision [11].

Cluster Metric Plots

Below the summary charts are a series of beeswarm plots for the per-cluster
metrics that were computed (see Cluster Metrics section above for details).
Each beeswarm shows each cluster as a dot and plots the distribution of
clusters over that metric. These charts are interactive, individual dots can
be selected to show the images from that cluster in the sample viewer.

Figure 2: Beeswarm plot showing distribution of cluster precision scores. Each
dot is a cluster which the currently selected dot shown in orange. A
description of the metric can be accessed by clicking on the ? icon.

A color encoding can also be applied by clicking on the rainbow icon toggle
next to that plot. This color encoding is applied across all charts allowing
comparison of one metric to another (Figure 3).

Figure 3: Recall and Precision beeswarm plots colored by precision score. High
recall, low precision clusters can be identified using the position encoding
in the recall plot and the color encoding applied from the precision plot. To
save space in the display we do not display a legend, as the relationship
between high-low values and hue is directly visible in the chart whose rainbow
icon was toggled and exact values are of less importance than relative
comparisons

UMAP Plots

Below the beeswarm plots are two plots showing 2D UMAP projections of: a) all
the clusters and b) images from the currently selected cluster (See Figure 4).
The Cluster UMAP view in particular allows browsing clusters by visual
similarity rather than by the metric scores. Previews of individual images are
shown on mouseover of points on the Samples UMAP plot

Figure 4: Top: UMAP projection of cluster centroids in embedding space.
Bottom: UMAP projection of currently selected cluster, real samples are shown
in blue and generated samples are show in red

Highlight Classes

In cases where the dataset does contain class labels, Ravel will display a
search interface that allows users to search for and select any number of
matching classes in the dataset.

If one or more classes is selected Ravel will dim clusters that do not contain
any images from those classes in the cluster plots (5).

Figure 5:  Clusters containing at least one image with the selected classes
are highlighted in all cluster plots.

All the cluster plot interactions are linked, enabling users to see where a
cluster falls in multiple metrics or in embedding space.

Any chart can be collapsed by by clicking on its title, this allows making
more room for the charts the user finds most useful for their analysis.

####  4.2.3 Sample Viewer

To the right of the charts is the sample viewer. This consists of a pair of
scrollable views displaying image thumbnails from the selected cluster divided
by split. If there are classes/labels associated with the images they are
displayed below the thumbnail and images from the same class are grouped
together, otherwise they are displayed in the order they were processed by the
pipeline (effectively random). Users can click on images to get a larger view
of the image and see any additional metadata.

Image thumbnails are displayed at a maximum size of 150x150px and a minimum
size of 100x100px depending on how large the browser window is. On a 3840 x
2160 resolution display222This resolution is the default 4K resolution which
is widely available. It is also close to the native resolution of a 2021 14
inch MacBook Pro (3024 x 1964) one can typically see 72 images per split for a
total of 144 images in a single screenful. However one clear limitation is
that if there are more images than this in a cluster a user cannot see them
all at once and does have to scroll through to get a better understanding of
the cluster. One way to mitigate this is to use choose a higher number of
clusters to view. Which results in smaller, more granular clusters, at the
cost of having more to explore.

##  5 Case Studies

In this section, we illustrate how the design features of Ravel support user
exploration in a series of case studies. These case studies highlight
discoveries the authors made when using the tool. Our 3 use cases are: 1)
Unconditional image generation in a single domain (faces), 2) Class
conditioned generation on ImageNet data and 3) Analyzing downstream use of a
generative model for another task - in this case, super-resolution.

While Ravel supports the use of multiple embeddings for clustering and
exploration, in this paper we focus primarily on the same Inceptionv3
embeddings used in FID score and other metrics, as they have become a de-facto
standard in the generative model space and are publicly available.

####  5.0.1 Unconditional Image Generation

Unconditional generation refers to situations where a trained model is used to
generate images with no ’conditioning’ other than an input vector (often
referred to as a ’noise’ or ’Z’ vector) drawn from a random distribution. In
this case study, we generate 60k images from an implementation of the
StyleGAN2 architecture 333We want to make clear that we are using an
independently trained StyleGAN2 and not the StyleGAN2 weights released by the
original authors [18] trained on the FFHQ dataset. The images are generated
using Z vectors drawn from a random distribution with a mean of zero and a
standard deviation of one. We load those images, along with 67,542 images from
the training set for a total of 127,542 images.

_Are there images that this model generates that are not part of the input
distribution?_

Our analyst opens the Ravel interface in her browser and sets the number of
clusters to 250, with images from the training data on the left and generated
images on the right. Knowing that low precision generally implies less
realistic images, she begins by exploring clusters on the low end of the
precision chart and notes that the clusters with the 10 lowest values each
contain generated images with visually obvious defects. Seeing these images
contrasted with ground truth data gives her a sense of what the model is
struggling to capture in each category. For example she notes that the model
has a difficult time modeling complex backgrounds, portraits where there are
more than one face, or portraits with occluding objects like hands or
microphones. Another problem that stands out to her is the difficulty the
model has generating faces with face-paint as in Figure 6. Seeing these
corrupted images juxtaposed with real images from the training data allows her
to hypothesize why the model struggles with this, in particularly she observes
that these types of images are relatively rare in the training data.

In examining 10 clusters, she has quickly scanned through approximately 4234
generated images and 2718 real images.

Figure 6: The StyleGAN2-based model struggles to model images of faces with
facepaint and other colorful accessories occluding the face, it instead
produces these artefacts.

####  5.0.2 Class Conditioned Image Generation

Class conditioned generation refers to models that have been trained to
generate output images for a fixed set of distinct classes. Here, we look at
directly comparing the output of two models trained on the same data and task:
namely, generating images from 1000 classes of the ImageNet dataset [41] at a
resolution of 128x128px. We use model implementations from Lucic et al’s ”Are
GANs Created Equal? A Large-Scale Study” [42].

We set up a Ravel instance with 50k images generated by BigGAN 128 and BigGAN-
deep 128 [12], and 50k images from the validation set for ImageNet444Data
retrived from https://www.tensorflow.org/datasets/ catalog/imagenet2012. Input
vectors for each model are drawn from a random normal distribution, as
described in [42], and no truncation is applied to the input vectors.

Using these models demonstrates a workflow where researchers are trying to
determine how one variant of a model, or an improved model architecture,
behaves differently from some baseline model. This comparison is a common
workflow in machine learning, typically achieved by reporting performance on
key metrics such as FID. We demonstrate how Ravel can complement traditional
metrics to find specific differences in model behaviour.

Figure 7: Mode collapse of the ”space bar” and ”typewriter keyboard” classes
in the BigGAN-deep model. All 100 images in this cluster look identical as
both classes have collapsed onto the same output. Figure 8: Output of the
”space bar” and ”typewriter keyboard” classes in BigGAN show much more
variety. This model does not exhibit mode collapse for these classes

_How does BigGAN compare to BigGAN-deep in terms of diversity of output_

Our analyst opens Ravel and sets the number of clusters to 500, with the left
split showing output from BigGAN, and the right split showing output from
BigGAN-deep.

It immediately jumps out to him that there are a number of clusters that have
scores of 0 in the recall chart. He clicks on one of them and discovers it
only contains images from BigGAN-deep. All 100 images from this model are from
two classes, appear virtually identical, and are of low visual quality (see
Figure 7). This is a classic case of mode collapse [43]; the model was unable
to learn the true distribution for these classes and has ’collapsed’ all
output for these classes to a single image. In this case, both classes have
been collapsed into the same output image. The analyst clicks on other
clusters with zero recall and finds similar mode collapse in BigGAN-deep for
the following classes: ”space bar”, ”typewriter keyboard”, ”pickelhaube”,
”rapeseed”, ”cardoon”, ”chambered nautilus, pearly nautilus, nautilus”, and
”odometer”.

Using the ”highlight classes” feature of Ravel 5, our analyst is able to find
all clusters that contain images from these classes, and confirms that the
BigGAN model does not show mode collapse for these classes (Figure 8).

What about classes where both models do okay at generation (i.e. neither model
exhibits a pathological failure like mode collapse)?

Our analyst now chooses to increase the granularity of clusters by setting the
number of clusters to 1000. He turns on the color by option of the precision
chart, and then looks at the recall chart to find clusters with high recall
but relatively low precision (though not as low as in the previous section).
Looking at higher recall clusters allows finding ones that have at least some
overlap in their distribution, while keeping an eye on precision suggests
differences in quality. These appear as light yellow dots on the right side of
the recall chart. He randomly selects one, and discovers a cluster of sea
urchins. Both generators produce high quality outputs, however visual
inspection shows that BigGAN-deep produces more diverse output (Figure 9).

Figure 9: A cluster of sea urchin samples from two generators, both produce
high quality images, but the generator on the right produces more diverse
output. The images on the right show a greater variety of colors, textures and
poses.

A benefit of clustering by visual similarity, rather than grouping by label,
is making it easier to discover overlap or leakage of visual semantics between
classes. Using the same method as above, our analyst selects another cluster
that consists of relatively realistic images of dishrags (Figure 10). He then
highlights all clusters containing dishrags. He can now individually examine
all these clusters. His findings, shown in (Figure 11 and Figure 12), suggests
that other classes, namely handkerchief’ and ’doormat’, are ’leaking’ into how
each generator models dishrags.

Figure 10: A cluster of ’dishrag’ samples from BigGAN on the left and BigGAN-
deep on the right, both produce somewhat realistic dishrags, but the generator
on the right produces images with more diverse textures and colors. Figure 11:
BigGAN cluster with a few dishrags but many colorful handkerchiefs that have
similar patterns to those seen in the main dishrag cluster. This suggests some
leakage in visual representation between the two concepts. Figure 12: BigGAN-
deep cluster with a few dishrags but many doormats, this suggests a hypothesis
for why many of the BIgGAN-deep dishrags have a very rough texture.

####  5.0.3 Downstream Applications of Generative Models: Super-Resolution

In our final case study we consider the PULSE (Photo Upsampling via Latent
Space Exploration) algorithm [22], a super-resolution algorithm that searches
the manifold of a generative image model (in this case StyleGAN2) to create
plausible upsampled images corresponding to low resolution input images. The
post-publication release of this model received sharp critique, as users
quickly discovered weaknesses in the models ability to upsample images of
people with non-caucasian features [44]. We use this example to illustrate how
broader exploration of output manifolds could help researchers and
practitioners who are using these kinds models in downstream applications to
understand their weaknesses and mitigate risks before release.

In this scenario, we use the original CelebA-HQ dataset from Karras, et al.
[13]. This dataset consists 30,000 facial portraits of celebrities at
1024x1024px. We take CelebA-HQ images at 32x32px and upsample them back to
1020x1024px using the PULSE algorithm555This scale factor (24x) is within the
range of scale factors (8x-64x) the original authors used in evaluation. We
invoke PULSE with the default settings provided by the authors in their model
release 666https://github.com/adamian98/pulse, however we increased the number
of steps we run the algorithm from 100 to 200 steps to increase the chances of
PULSE producing a result for a given input.777We did this because PULSE will
not always produce a result for an input image, and we found increasing the
number of steps allowed substantially more input images to produce some
result.

We then compare the original 1024x1024px images and the PULSE up-sampled ones
using Ravel. In total, we have 30000 original CelebA-HQ images and 22092
images output from PULSE. 7908 images failed to produce any output after 200
steps of the PULSE algorithm.

_What kinds of images does PULSE struggle to produce any output for?_

Our analyst sets the number of clusters to 250 and the original CelabA-HQ
images on the left split, and immediately notices a group of clusters on the
lowest end of the precision chart. She notices that many clusters also have
low recall and are mostly composed of images only from the ground truth (see
Figure 13). She clicks on some low-recall/low-precision clusters and observes
a few categories where the algorithm struggles to produce any output. These
include: people wearing hats or other headgear, images where the person has a
a microphone or hand in front of them, faces with a lot of facepaint or
makeup, and images where people are wearing sunglasses. Each of these clusters
has between 60-120 images and a few samples are shown in Figure 14.

Figure 13: Cluster metric charts colored by cluster precision. On the left of
the precision chart is a group of clusters with very low precision that are
mostly composed of images only from the ground truth data, many of these also
have low recall, suggesting the algorithm struggles to produce any output for
them. Figure 14: Each row contains samples from a different cluster with that
only has images from the ground truth data and none from the algorithm output.
These are examples of images the algorithm struggles to upscale.

_What kinds of images does PULSE struggle to produce high quality output for?_

She continues to explore low precision clusters by scanning left to right
along the chart. One of the lowest precision clusters contains ground truth
images of people wearing spectacles (and a few wearing sunglasses), however
from the algorithm output she sees a number of low quality (i.e. unrealistic)
images that are likely up-scaled from ones where the subject is wearing
sunglasses (Figure 15). She refines her hypothesis about what the algorithm
does to portraits with sunglasses, determining that, ”if a person is wearing
sunglasses, the algorithm often fails to produce any image, and when it does,
it produces unrealistic output”.

Figure 15: Samples from a cluster of upscaled images of people wearing
sunglasses.

_What kinds of images does PULSE do well at upsampling?_

The precision chart can also be used to look at what images PULSE does well
at. As our analyst browses clusters on the right side of the distribution, she
observes that a majority of high quality outputs seem to be of lighter skin
tone faces that look relatively young or middle aged.

Her manual inspection reveals some of the model’s failure modes and strengths
along demographic categories even though she does not have access to
quantitative measures of performance across sensitive features such as skin
tone or age.

##  6 Domain Expert User Study

We evaluated Ravel in a two-stage study, where the first stage identified
domain experts’ model evaluation goals, workflows and existing tools, and the
second stage investigated the usability and utility of the Ravel UI.
Participants were recruited from a convenience sample of full time employees
as a large technology company, and there was a diversity in gender identity
($n_{female}$ = 1, $n_{Male}$ = 5), product area (5 different teams), and
office location ($n_{Israel}$ = 2, $n_{UnitedStates}$ = 4). Stage one (n = 4)
included four expert research scientists and software engineers who currently
work on training and evaluating generative models, and stage two (n = 5)
included three of the users from stage one plus two additional users who have
experience working with generative models. Both evaluation stages used remote
moderated Google Meet video calls to speak with users and allow them to share
their screen to show how they interacted with Ravel.

###  6.1 Stage One: The Current State of Evaluation

Stage one aimed to understand the current state of evaluation for generative
image model output. Semi-structured interviews were used, with questions about
participants’ familiarity and day to day work with generative models, goals
and motivation for model evaluation, as well as current workflows and
practices. All four users currently work on generative models, and were
familiar with StyleGAN and its variants, BigGAN and its variants, as well as
other models trained on ImageNet and FFHQ. There was diversity in
architectures they work with and the tasks they apply generative models to.
Here, we report the most common practices among users.

####  6.1.1 Evaluation Goals & Workflows

All four participants reported that publication of model
performance/improvement was a primary goal for their work. All four
participants also expressed that evaluating output image quality was critical
to understanding model performance.

All four participants reported a mix of quantitative and qualitative
evaluation methods. In general, their workflows could be characterized by a
common pattern of: model training accompanied by limited visual inspection for
sanity-checking $\rightarrow$ examination of metrics $\rightarrow$ continued
training $\rightarrow$ reexamination of metrics until a predetermined
threshold is reached $\rightarrow$ qualitative visual inspection of image
output.

Two participants indicated that they believed the current “best practice” for
determining image quality was human qualitative evaluation, often from
crowdsourced studies. Importantly, all four participants also emphasized the
ubiquitous reliance upon and limitations of FID scores in model evaluation.
While one user suggested that “FID scores are much better than inception
scores and other metrics”, they also conceded that “there is not a gold
standard for evaluation of generated images”. All users indicated that they
were dissatisfied with FID score as a primary evaluation metric, and that they
had all experienced and read cases where they felt that FID score did not
align with human inference, a sentiment supported in Zhou, et al. [24].

####  6.1.2 Evaluation Tools

All four users mentioned using TensorBoard [45] or
Colab888https://colab.research.google.com/ (a computational notebook
envrionment similar to Jupyter Notebook [46]) to examine model output. For
Colab, the main strength reported was flexibility. The flexibility of Colab
enables bespoke analyses that we elaborate on in section 6.2.5. Its main
limitations were difficulties reusing code between projects and sharing
results, especially with non-technical collaborators or stakeholders not
directly involved with model training. For TensorBoard, the main strengths
reported were ease of use during training to quickly visualize metrics and see
sample output at different stages of training. The main limitations mentioned
about TensorBoard were its latency when loading many images, and lack of
customization compared to an open ended tool like Colab.

####  6.1.3 Qualitative Visual Evaluation Tasks

Determining image quality was reported as the most important task. Determining
the diversity of images was also important to all four users, and one user
mentioned that looking for the occurrence of mode collapse was an explicit
part of their evaluation workflow. All users indicated that it was important
to do more granular and bespoke image generation, including looking at samples
within certain classes or samples close to each other in embedding space. One
user discussed examining different levels of truncation for latent vectors to
make decisions about both realism and aesthetics in the output. Users reported
viewing a small number of images. One user reported looking at approximately
64 images, and never more than 100 images, per evaluation step of the model
training pipeline. Another user explained that when they work with a team on
an evaluation, they typically generate and inspect about 50 images. However,
when working alone, this user said that they would inspect at least 200
images, noting that it was difficult to get a group consensus on more than 50
images at a time.

We determined four categories of critical evaluation tasks from this part of
the study: image diversity, image quality, mode collapse, and a “catch-all”
category of additional explorations of classes and samples.

###  6.2 Stage Two: Observing how Ravel is Used

The objective of this stage was to determine how the Ravel UI can be used for
the critical evaluation tasks identified in stage one. A brief slide deck and
video explaining the UI components was sent to users upon confirmation of
their participation, which they were asked to review before the study session.
The study sessions themselves lasted for one hour and consisted of task based
user exploration of the tool and semi-structured interview questions.

####  6.2.1 Task one: Diversity

The first task we asked users to attempt was to decide whether BigGAN or
BigGAN-deep was performing better in terms of the diversity of sea urchin
images, in a setup mirroring the Ravel instance described in Section 8. This
instance showed 128x128 pixel images from both models, with 50,000 images per
model and 50 images per class from each model. The UI resolution was set to
show the same number of images for each user: 5 images per row in each split.

Users were presented with an instance of Ravel with the ”sea urchin” class
selected in the Highlighted Classes menu, showing results from BigGAN and
BigGAN-deep in the left split and right split, respectively. On initial load
the instance displayed the cluster containing most of sea urchin images
selected: 39 (out of 50) urchins from BigGAN and 45 (out of 50) urchins from
BigGAN-deep.

All users started by attending to images in each split. No users examined
metrics or asked about metrics as a first step. This emphasizes the importance
of Ravel intuitively depicting some of the most important information for
assessing image diversity: a salient grid of sample images within a cluster of
interest. Three users immediately noted the number of samples in each split,
paying close attention to which model had a greater number of samples in the
cluster of interest. The most common flow involved inspecting individual
images and counting or “eyeballing” the number of images in each model with
unique background colors, sea urchin poses, and sea urchin colors (all users
mentioned these three features). Three users scanned the metrics charts and
clicked on the other highlighted clusters with sea urchins. All five users
made the determination that BigGAN-deep was performing better in terms of the
diversity of sea urchin images, with one user explicitly stating that this
confirmed their prior expectations. This demonstrates consistency in how users
complete this critical task with Ravel, and shows the potential for convergent
decision making and operationalization of workflows in qualitative visual
evaluation. It is also notable that Ravel helped confirm one user’s prior
expectations about BigGAN-deep’s diversity performance, although that could
also have been a potential biasing factor in their evaluation process.

####  6.2.2 Task two: Quality

For the second task, users were asked to decide whether BigGAN or BigGAN-deep
was performing better in terms of the quality of golden retriever images.
Users were presented with the same Ravel instance as task one, but this time
the ”golden retriever” class was selected in the Highlighted Classes menu and
a cluster showing most of the golden retriever images was selected: 46 (out of
50) in the BigGAN split and 46 (out of 50) in the BigGAN-deep split.

All users immediately noted many artifacts in output images from both models.
All users clicked on individual images and narrated particular issues, such as
the shape of dogs’ noses, the number of legs, and the accuracy of the form and
pose of the dogs in each sample. The most common workflow was to count or
“eyeball” the number of artifacts in each split to make an initial
determination, but evaluation workflows were notably more diverse between
users for the rest of the task. Five users reported that this task was more
difficult than the diversity task while one user reported that it was easier.
Some users indicated that FID and Inception score for each model would be an
important part of making this determination in their typical workflow. Four
out of the five users made the determination that BigGAN-deep also performed
better in terms of the quality of golden retriever images, and one user did
not make an explicit determination for this task. Once again, Ravel supported
consistency in the primary image quality evaluation strategy across all users,
but individual exploration varied after this initial decision. Some users
validated their visual judgments by looking at recall and precision charts,
but others simply explored the charts without forming additional opinions
about the task.

####  6.2.3 Task three: Mode Collapse

Following the Quality task, users were asked if they were familiar with the
term mode collapse and if it was something they used to evaluate generative
model output. Four out of five users were familiar with mode collapse and
reported it as a useful discovery in the model evaluation process, while one
user was unfamiliar with the term. Users who were familiar with mode collapse
were then asked to use Ravel to determine whether it had occurred for either
model within any class.

Because this task was not constrained to a single class, it was more open-
ended, and thus more challenging for users to make a determination about.
Workflows varied widely between users, as they were given no guidance about
how to accomplish the task or make a determination. Several users mentioned
that they expected BigGAN to exhibit more mode collapse after observing that
BigGAN-deep had better diversity in the first task. Four users who found
concrete instances of mode collapse (e.g. Fig. 6) did so by selecting clusters
with the smallest values in the Recall chart. Four out of five users viewed
the cluster samples displayed in figure 7 at some point during their
exploration, but one user did not identify it as mode collapse, and one user
did not select it at all. Nevertheless, this was a promising observation, and
demonstrates further consistency of Ravel’s usability and specific utility of
the recall visualization. There was majority agreement that the recall chart
can be used to identify mode collapse, and it was revealed to occur more often
in BigGAN-deep.

####  6.2.4 Task four: Additional exploration of classes, samples, and
features

For their final task, users were asked to view an instance of Ravel showing
generated images from a StyleGAN2-based model in the right split, and images
from its ground-truth training data, FFHQ, in the left split. Users were told
they could freely explore the interface, either repeating the quality and
diversity assessments or trying a new task that they might be interested in.

This task was fully unguided, but most users started by assessing image
quality for the StyleGAN2-based model. It was common for users to select
clusters at the low and high ends of the Precision distribution. One user
observed that clicking on a cluster with high precision revealed “typical FFHQ
images…very good images without occlusion, faces looking at the camera,
showing people with straight hair, and the model output is very similar to
these images.”. Four out of five users discovered and explicitly verbalized
that StyleGAN2-based model struggled to generate images of people with
facepaint. They did this by clicking on clusters with low precision and
noticing artifacts on many of the generated images. All four of these users
expressed surprise upon making this new discovery about the model. This
demonstrates that even without a specific prompt, many users will make the
same kinds of discoveries and follow the same types of evaluation processes
with Ravel. One of our participants was on the team that had originally
trained the StyleGAN2-based model that we used and discovered that the model
was unable to generate faces of people wearing a particular style of fuzzy
winter hat that was fairly common in the ground truth data. He remarked that
he wasn’t aware of that inability and was pleasantly surprised to be able to
discover it.

The other common tasks were “semantic explorations” of the clusters and the
embedding space. Four out of five users examined images in the FFHQ split to
make decisions about whether the clusters were semantically meaningful, and to
explore the diversity of FFHQ. Two users pondered whether or not the embedding
space was doing a good job of capturing what they, as humans, would group
together. These users investigated the proximity of samples in the Samples
UMAP chart and determined that the ImageNet feature space is not optimal for
clustering faces, since they could not find consistent visual similarity
between nearby samples in some of the clusters.

####  6.2.5 Discussion of Expert Feedback

Users were overall impressed with the tool. All five users thought that Ravel
would fit into their current evaluation process, and that it was especially
useful for researchers who publish generative models. Here, we summarize
additional feedback from the reflection portion of stage two.

Two users reported that their exploration made them doubt whether the
Inceptionv3 features were good for evaluating face portrait generation. In
reflection, one of these users stated that Ravel could be used to learn about
the feature space itself, which could be broadly useful in generative image
model evaluation.

Upon discovering mode collapse in BigGAN-deep, one user stated that Ravel
would be useful for probing state-of-the-art models to learn which classes
they can’t generate images for, which could point to systematic failure modes
for researchers to focus on improving.

One user noted that Ravel could help their team reach conclusions about model
performance more quickly than using Colab, especially for understanding the
diversity of images. They explained that using Colab required developing a
bespoke visualization tool and manual calculations of FID in each cluster,
whereas the same type of useful information was readily available in the Ravel
UI. Two users emphasized that Ravel could help operationalize how researchers
evaluate quality and diversity, one of which explained that Ravel could be “a
forcing function for having a standard UI/pipeline for results”, arguing that
this would make it “easier to share results with someone on another team, or
someone non-technical…even with my manager who doesn’t have time to run my
code”. This confirms for us that there is a place for bespoke purpose built
interfaces like Ravel, that while less flexible than Colab, are more purpose
built for common evaluation tasks that researchers perform.

Overall, the expert feedback from stage two was positive and enthusiastic,
with several users expressing excitement about continued exploration in Ravel
and incorporating it into their own workflows.

##  7 Limitations and Future Work

Our qualitative user study was performed with a small, domain expert sample
from a single company, and therefore may not be an externally valid
representation of all researcher experiences with generative image models. The
study sessions were also limited to one hour per user, with each evaluation
task time-boxed to 15 minutes or less. Richer and more diverse interactions
and discoveries could be possible with a longer duration of tool use.

Two users wanted to see images at a higher resolution, and two other users
wanted to see the original resolution of the images when examining them for
artifacts; this is not an inherent issue with Ravel but is an important design
affordance for the future. Two users wanted to be able to mark images in each
split once they had viewed them, and enabling this would support the
’counting’ based workflows we saw participants use to complete the tasks.

When using the class conditional model, users initially reported that they
would prefer to see all images from a given class in the same view (i.e.
clustering by class label) to make a judgement about that class. However on
further exploration they noted it was useful to see ’outlier’ images for a
given label in context with the other images they are most similar to. This
suggests that both workflows are important to support for class conditioned
models and should be supported by tools like Ravel.

The authors also note a number of limitations of the system we observed while
watching users use the tool:

User’s cannot always interpret the ’meaning’ of clusters (i.e., construct a
rationale for why a set of images are clustered together). This is a general
issue with unsupervised methods like clustering, but we found that users would
often try to attach some semantically meaningful description to each cluster
to ground their comparison.

Exploring ways to ’describe’ clusters or summarize differences between
clusters could be important future work to aid user comprehension. We also
think exploring other clustering methods, in particular hierarchical methods,
could be a particularly attractive means to produce clusters at different
levels of granularity to help build understanding of groups within the
dataset.

In describing the Sample viewer (section 4.2.3) we noted that one limitation
is that not all of the images are visible in one screen if there are many
images present in the cluster. While one mitigation is to decrease the size of
clusters by increasing the number of clusters, we believe that future work
could provide better ways to get the visual gestalt of the entire cluster in
one view. Possibly adapting techniques such as those described in Activation
Atlas [47], creating stacks of very similar images within a cluster, or other
ways of sub-sampling or sorting to ensure that we are displaying the maximum
variety about a cluster in a single screen.

One user task that Ravel does not directly support is detecting memorization.
One user commented that adding a real time nearest-neighbor search to the
interface would likely make it useful for this task.

##  8 Conclusion

We presented Ravel, a visual analysis tool that enables researchers to perform
large scale qualitative evaluation of generative model outputs. Our primary
contributions included:

  * •

A visual analytics system that supports multiple evaluation tasks (e.g.
evaluating quality & diversity, discovering mode collapse or gaps in model
output) for generative image models and is agnostic to model architecture and
internals.

  * •

Interactive exploration of large generative image model datasets, facilitated
by clustering and the use of fine grained visualization of cluster metrics to
guide qualitative evaluation.

  * •

A user interface that uses visual comparison driven by semantically meaningful
embedding spaces to support reasoning about differences between image
distributions and generate hypotheses about model behaviour.

The expert users in our study were able to generate consistent insights about
model behaviour including identifying areas of the true data distribution the
model was not capturing, such as face paint or certain kinds of headgear in
the StyleGAN2-based model or mode collapse in BigGAN-deep. This kind of
insight is an example of one that is not possible to get from just looking at
quantitative metrics like FID.

Our study participants confirmed our hypotheses that single number metrics are
not fully sufficient measures of model performance. In addition to exploring
metrics at greater granularity, Ravel allows users to explore metrics at finer
granularity, revealing areas of model output where metrics like recall or
precision do not capture problems in the generated images. Our users
hypothesized that the underlying InceptionV3 embedding, used both in our tool
and in the primarily metrics in the field, may not attend to certain kinds
visual artefacts that are easily visible to humans. We believe that future
work in this direction could enable better understanding of limits of the
embedding spaces themselves and how they affect both metrics and the workflows
that use them.

## Acknowledgments

The authors wish to thank Mario Lucic, Marvin Ritter, Ben Poole, Han Zhang,
Chitwan Saharia, James Wexler and Lucas Dixon for their help and feedback on
this work. We also thank our study participants for their feedback.

## References

  * [1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” in _Advances in Neural Information Processing Systems_ , vol. 27, 2014\. [Online]. Available: https://proceedings.neurips.cc/paper/2014/hash/5ca3e9b122f61f8f06494c97b1afccf3-Abstract.html
  * [2] D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” Dec. 2013. [Online]. Available: https://arxiv.org/abs/1312.6114v10
  * [3] L. Dinh, D. Krueger, and Y. Bengio, “NICE: Non-linear Independent Components Estimation,” _arXiv:1410.8516 [cs]_ , Apr. 2015, arXiv: 1410.8516. [Online]. Available: http://arxiv.org/abs/1410.8516
  * [4] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep Unsupervised Learning using Nonequilibrium Thermodynamics,” in _Proceedings of the 32nd International Conference on Machine Learning_ , Jun. 2015, pp. 2256–2265, iSSN: 1938-7228. [Online]. Available: https://proceedings.mlr.press/v37/sohl-dickstein15.html
  * [5] A. Odena, “Open Questions about Generative Adversarial Networks,” _Distill_ , vol. 4, no. 4, p. e18, Apr. 2019. [Online]. Available: https://distill.pub/2019/gan-open-problems
  * [6] L. Theis, A. v. d. Oord, and M. Bethge, “A note on the evaluation of generative models,” _arXiv:1511.01844 [cs, stat]_ , Apr. 2016, arXiv: 1511.01844. [Online]. Available: http://arxiv.org/abs/1511.01844
  * [7] A. Borji, “Pros and Cons of GAN Evaluation Measures,” _arXiv:1802.03446 [cs]_ , Oct. 2018.
  * [8] ——, “Pros and Cons of GAN Evaluation Measures: New Developments,” _arXiv:2103.09396 [cs]_ , Mar. 2021.
  * [9] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,” in _Advances in Neural Information Processing Systems_ , vol. 30, 2017. [Online]. Available: https://proceedings.neurips.cc/paper/2017/hash/8a1d694707eb0fefe65871369074926d-Abstract.html
  * [10] M. S. M. Sajjadi, O. Bachem, M. Lucic, O. Bousquet, and S. Gelly, “Assessing Generative Models via Precision and Recall,” in _Advances in Neural Information Processing Systems_ , vol. 31, 2018. [Online]. Available: https://proceedings.neurips.cc/paper/2018/hash/f7696a9b362ac5a51c3dc8f098b73923-Abstract.html
  * [11] T. Kynkäänniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila, “Improved Precision and Recall Metric for Assessing Generative Models,” _arXiv:1904.06991 [cs, stat]_ , Oct. 2019, arXiv: 1904.06991. [Online]. Available: http://arxiv.org/abs/1904.06991
  * [12] A. Brock, J. Donahue, and K. Simonyan, “Large Scale GAN Training for High Fidelity Natural Image Synthesis,” _arXiv:1809.11096 [cs, stat]_ , Feb. 2019, arXiv: 1809.11096. [Online]. Available: http://arxiv.org/abs/1809.11096
  * [13] T. Karras, S. Laine, and T. Aila, “A Style-Based Generator Architecture for Generative Adversarial Networks,” _arXiv:1812.04948 [cs, stat]_ , Mar. 2019, arXiv: 1812.04948. [Online]. Available: http://arxiv.org/abs/1812.04948
  * [14] D. P. Kingma and P. Dhariwal, “Glow: Generative Flow with Invertible 1x1 Convolutions,” _arXiv:1807.03039 [cs, stat]_ , Jul. 2018, arXiv: 1807.03039. [Online]. Available: http://arxiv.org/abs/1807.03039
  * [15] A. v. d. Oord, O. Vinyals, and K. Kavukcuoglu, “Neural Discrete Representation Learning,” _arXiv:1711.00937 [cs]_ , May 2018, arXiv: 1711.00937. [Online]. Available: http://arxiv.org/abs/1711.00937
  * [16] J. Ho, C. Saharia, W. Chan, D. J. Fleet, M. Norouzi, and T. Salimans, “Cascaded Diffusion Models for High Fidelity Image Generation,” _arXiv:2106.15282 [cs]_ , Dec. 2021, arXiv: 2106.15282. [Online]. Available: http://arxiv.org/abs/2106.15282
  * [17] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the Inception Architecture for Computer Vision,” _arXiv:1512.00567 [cs]_ , Dec. 2015.
  * [18] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen, and T. Aila, “Analyzing and improving the image quality of stylegan,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 8110–8119.
  * [19] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness,” _arXiv:1811.12231 [cs, q-bio, stat]_ , Jan. 2019.
  * [20] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X. Chen, and X. Chen, “Improved Techniques for Training GANs,” in _Advances in Neural Information Processing Systems_ , vol. 29, 2016. [Online]. Available: https://proceedings.neurips.cc/paper/2016/hash/8a3363abe792db2d8761d6403605aeb7-Abstract.html
  * [21] S. Barratt and R. Sharma, “A Note on the Inception Score,” _arXiv:1801.01973 [cs, stat]_ , Jun. 2018, arXiv: 1801.01973. [Online]. Available: http://arxiv.org/abs/1801.01973
  * [22] S. Menon, A. Damian, S. Hu, N. Ravi, and C. Rudin, “PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models,” in _2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , Seattle, WA, USA, Jun. 2020, pp. 2434–2442.
  * [23] E. Denton, S. Chintala, A. Szlam, and R. Fergus, “Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networks,” _arXiv:1506.05751 [cs]_ , Jun. 2015.
  * [24] S. Zhou, M. Gordon, R. Krishna, A. Narcomey, L. F. Fei-Fei, and M. Bernstein, “HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models,” p. 13.
  * [25] D. Bau, J.-Y. Zhu, H. Strobelt, B. Zhou, J. B. Tenenbaum, W. T. Freeman, and A. Torralba, “GAN Dissection: Visualizing and Understanding Generative Adversarial Networks,” _arXiv:1811.10597 [cs]_ , Dec. 2018.
  * [26] D. Bau, J.-Y. Zhu, J. Wulff, W. Peebles, H. Strobelt, B. Zhou, and A. Torralba, “Seeing What a GAN Cannot Generate,” _arXiv:1910.11626 [cs, eess]_ , Oct. 2019.
  * [27] T. White, “Sampling Generative Networks,” _arXiv:1609.04468 [cs, stat]_ , Dec. 2016.
  * [28] T. White and B. Loh, “SpaceSheet: Navigating Conceptual Space with a Spreadsheet Interface,” p. 8.
  * [29] D. Smilkov, N. Thorat, C. Nicholson, E. Reif, F. B. Viégas, and M. Wattenberg, “Embedding Projector: Interactive Visualization and Interpretation of Embeddings,” _arXiv:1611.05469 [cs, stat]_ , Nov. 2016.
  * [30] K. Ho, “Font Map,” https://experiments.withgoogle.com/font-map.
  * [31] C. Diagne and S. Doury, “Curator Table,” https://experiments.withgoogle.com/curator-table.
  * [32] K. McDonald, M. Tan, and Y. Mann, “Bird Sounds,” https://experiments.withgoogle.com/bird-sounds.
  * [33] Y. Liu, E. Jun, Q. Li, and J. Heer, “Latent Space Cartography: Visual Analysis of Vector Space Embeddings,” _Computer Graphics Forum_ , vol. 38, no. 3, pp. 67–78, Jun. 2019.
  * [34] S. Xiang, X. Ye, J. Xia, J. Wu, Y. Chen, and S. Liu, “Interactive Correction of Mislabeled Training Data,” in _2019 IEEE Conference on Visual Analytics Science and Technology (VAST)_ , Oct. 2019, pp. 57–68.
  * [35] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The Unreasonable Effectiveness of Deep Features as a Perceptual Metric,” 2018, pp. 586–595. [Online]. Available: https://openaccess.thecvf.com/content_cvpr_2018/html/Zhang_The_Unreasonable_Effectiveness_CVPR_2018_paper.html
  * [36] “Scikit-learn Overview of Clustering.” [Online]. Available: https://scikit-learn.org/stable/modules/clustering.html#overview-of-clustering-methods
  * [37] L. McInnes, J. Healy, N. Saul, and L. Grossberger, “Umap: Uniform manifold approximation and projection,” _The Journal of Open Source Software_ , vol. 3, no. 29, p. 861, 2018.
  * [38] K. P. F.R.S., “Liii. on lines and planes of closest fit to systems of points in space,” _The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science_ , vol. 2, no. 11, pp. 559–572, 1901.
  * [39] L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” _Journal of machine learning research_ , vol. 9, no. 11, 2008.
  * [40] A. Coenen and A. Pearce, “Understanding UMAP,” 2019. [Online]. Available: https://pair-code.github.io/understanding-umap/
  * [41] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” _International Journal of Computer Vision (IJCV)_ , vol. 115, no. 3, pp. 211–252, 2015.
  * [42] M. Lucic, K. Kurach, M. Michalski, S. Gelly, and O. Bousquet, “Are GANs Created Equal? A Large-Scale Study,” in _Advances in Neural Information Processing Systems_ , vol. 31, 2018. [Online]. Available: https://proceedings.neurips.cc/paper/2018/hash/e46de7e1bcaaced9a54f1e9d0d2f800d-Abstract.html
  * [43] I. Goodfellow, “NIPS 2016 Tutorial: Generative Adversarial Networks,” _arXiv:1701.00160 [cs]_ , Apr. 2017.
  * [44] “Lessons from the PULSE Model and Discussion,” https://thegradient.pub/pulse-lessons/, Jun. 2020.
  * [45] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mane, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viegas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems,” _arXiv:1603.04467 [cs]_ , Mar. 2016.
  * [46] T. Kluyver, B. Ragan-Kelley, F. Pérez, B. Granger, M. Bussonnier, J. Frederic, K. Kelley, J. Hamrick, J. Grout, S. Corlay, P. Ivanov, D. Avila, S. Abdalla, and C. Willing, “Jupyter notebooks – a publishing format for reproducible computational workflows,” in _Positioning and Power in Academic Publishing: Players, Agents and Agendas_ , F. Loizides and B. Schmidt, Eds. IOS Press, 2016, pp. 87 – 90.
  * [47] S. Carter, Z. Armstrong, L. Schubert, I. Johnson, and C. Olah, “Activation Atlas,” _Distill_ , vol. 4, no. 3, p. e15, Mar. 2019. [Online]. Available: https://distill.pub/2019/activation-atlas