tcb@breakable

# Considering Visualization Example Galleries

Junran Yang  University of Washington
Seattle, USA
<EMAIL_ADDRESS>Andrew M. McNutt  University of Washington
Seattle, USA
<EMAIL_ADDRESS>Leilani Battle  University of Washington
Seattle, USA
<EMAIL_ADDRESS>
###### Abstract

Example galleries are often used to teach, document, and advertise visually-
focused domain-specific languages and libraries, such as those producing
visualizations, diagrams, or webpages. Despite their ubiquity, there is no
consensus on the role of “example galleries”, let alone what the best
practices might be for their creation or curation. To understand gallery
meaning and usage, we interviewed the creators (N=11) and users (N=9) of
prominent visualization-adjacent tools. From these interviews we synthesized
strategies and challenges for gallery curation and management (e.g. weighing
the costs/benefits of adding new examples and trade-offs in richness vs ease
of use), highlighted the differences between planned and actual gallery usage
(e.g. opportunistic reuse vs search-engine optimization), and reflected on
parts of the gallery design space not explored (e.g. highlighting the
potential of tool assistance). We found that galleries are multi-faceted
structures whose form and content are motivated to accommodate different
usages—ranging from marketing material to test suite to extended
documentation. This work offers a foundation for future support tools by
characterizing gallery design and management, as well as by highlighting
challenges and opportunities in the space (such as how more diverse galleries
make reuse tasks simpler, but complicate upkeep).

###### Index Terms:

Example gallery, qualitative research, interview study, DSL documentation

##  I Introduction

Examples are a powerful medium of knowledge sharing in both design and coding.
Designers and artists look into examples as sources of inspiration [2, 30, 63]
and as tools to communicate concrete feedback [27]. In software engineering,
code examples provide resources to understand how things work [41], learn
usage patterns [56, 4], and as a means of researching opportunistic reuse [7]
(such as how developers will copy and tweak examples from StackOverflow).
Leveraging the power of examples, _galleries_ are a common entry point into
many domain-specific languages and libraries—we use the latter of these terms
throughout for simplicity. Example galleries are especially common for
libraries which include a visual component, such as those that produce
visualizations (such as Vega [52]), diagrams (like Penrose [66]), art (à la
Processing [49]), and even webpages (e.g. Idyll [8]). Compared to individual
examples, galleries illustrate the gamut of capabilities through curated
collections of code with accompanying visual outputs.

Galleries come in many forms and shapes. Some consist of every conceivable
usage, as with Plot.ly. Others are tightly managed sets of minimal examples
that are meant to advertise the tool, as with Deck.gl. In some contexts [2],
galleries act as tutorials or starter code. In others, galleries are
intertwined with the library infrastructure (e.g. as test suite). These
discordant approaches make it hard to identify what makes an example gallery
effective. Questions like _how many examples should be included?_ and _which
examples are necessary?_ remain unanswered. The interaction between “official”
channels and “folk” documentation (e.g. StackOverflow) are unclear. Without
clear guidelines creators are left to copy the works of other gallery
creators, who may have different motivations and resources than they do.

We explore these questions via an interview study. This consisted of semi-
structured interviews with (N=11) authors of galleries as well as (N=9)
_users_ of a subset of those example galleries. We identify three central
reasons why galleries are formed: to advertise the library, to accelerate
library usage via reuse, and to support gallery sustainability by acting as a
test suite. We then analyze a series of common cross-cutting issues that
appear that are affected by these various curatorial stances including the
contents of the gallery, how the gallery is maintained over time, and role of
education and communication in gallery architectural and organization. Across
these analyses we observe strategies and challenges for gallery construction
and management (such as the burdens of maintaining the examples during library
development), highlight the differences between planned and actual gallery
usage (such as between search engine optimization and opportunistic reuse
[7]), and highlight opportunities in the space of example galleries (such as
providing integrated generative assistance alongside gallery search). Broadly,
the effectiveness of a gallery seems to depend on the goals of both the
creators (_do they integrate examples into the test suite or just the
documentation?_) and the user community around it (_how do they tend to
use/contribute examples for this library?_).

We identify practices that make it easier to form and maintain galleries in
the future (summarized in Fig. 4). For instance, diversifying the types of
tasks covered in a gallery can make it easier for users to find specific tasks
that are relevant to them and provide a more useful test cases for creators.
Yet, smaller galleries are easier to maintain and put less burden on the
developers. Like most socio-technical artifacts, there are few unambiguous
guidelines; for a system with fixed resources, compromises will need to be
made to support conflicting usage patterns (e.g. maintainers vs users). Our
work in this paper is to make the space of trade-offs clearer, so that design
decisions might be made more clearly.

##  II Background and Related Work

Here we set definitions and review prior work in this space.

Documentation Formats. Galleries are but one part of a constellation of
different types and forms of documentation written by software creators and
consumed by end users [15]. These include various common forms which describe
here to provide language for comparing and contrasting with galleries.
Diataxis [48] taxonomizes documentation types based on task and form. We draw
on this theory in organizing our analysis of galleries as documentation form.
We use examples from Vega as exemplars of these forms.

_Tutorials and usage documentation_ demonstrate step-by-step processes someone
can follow to achieve goals specified by the software authors such as
installing the DSL or recreating a recommended example. They are a common
entry point to documentation, designed for beginners to learn by doing. Code
examples in tutorials focus on capturing the process. They are organized into
digestible chunks and often display the intermediate results of the segmented
code [18]. Textual information and visual output are used to help elicit the
context of changes. For instance, Vega’s “Let’s Make A Bar Chart Tutorial”
[60] is an example of this usage. While tutorials highlight step-wise changes
and reference code examples focus on specific usage and assumed basic
understanding of the library, gallery examples are self-contained so that
users can copy and paste the code to successfully execute it. Some of gallery
examples focus on a single learning objective, while others (_i.e_. showcases)
include more narrativized context.

_References_ describe the function of features exposed by the library such as
functions, methods, attributes and parameters. This most basic form of
documentation characterizes the specific way in which the user can interact
with the library aspect by aspect. Libraries like Sphinx [13] automatically
generate documentation from code comments. Vega’s “Documentation” [57] section
demonstrates this pattern. In contrast to galleries, which feature standalone
examples, references are short and focused on an aspect of a particular
feature.

_Explanation and discussion_ elucidate a topic in a blogpost style. Unlike a
tutorial, the goal is not to accomplish a specific task but to broaden the
understanding of domain knowledge from a high level or a different perspective
[48]. A library architecture overview or a background section would fall into
this category, such as in “How Vega Works” [58]. It describes the way in which
the internals of the library operates. While informative, these resources do
not visually demonstrate functionality the way that galleries do, nor do they
offer the re-usable cookbook-like guide [48] to accomplish tasks.

Documentation Studies. Code examples and documentations are essential for
learning programming and accomplishing programming tasks. Nasehi et al. [41]
find that well-received answers on StackOverflow often incorporate more code
examples with explanation. Links to existing resources that provide reliable
background knowledge are frequently shared, indicating that the general
audience have difficulties in finding useful information. Nam et al. [38]
analyze documentation page-view logs from popular cloud-based services,
finding a correlation between visit patterns and user characteristics. Other
work investigates the challenges using code examples for different user
groups, including lack of API knowledge [56]; identifying relevant parts,
integrating and adapting existing code [24, 62]. While code examples are a key
part of galleries, we investigate the interaction with example collections as
a whole.

Studies have explored how documentation might be enhanced by supporting
sustainable maintenance [22], annotation [21], automated creation [17, 29,
10], enhancements to the development environment [47, 35], among others. For
instance, ActiveDocumentation [36] compares documented design rules against
users’ codebase to provide feedback. Glassman et al. [16] develop a
visualization for summarizing code collections in a synthetic code skeleton,
which can help users explore how others have used an unfamiliar API. Jernigan
et al. [26] provided debugging hints to end-users who are not interested in
programming. Our work explore effective galleries as the premise towards
dynamic enhancement. Related to some of the tasks galleries attempt to serve
are code search tools (e.g. Blueprint [6] or Calcite [37]). Whereas search
supports identification of relevant examples, galleries support browsing what
is possible within a given context.

Several works studied the design of gallery-like entities. Bako et al. [2]
explore how visualization designers search and use examples. They find people
select examples by examining the trade-off between the effectiveness and
aesthetics, and reuse them by modifying and merging the visual elements (à la
opportunistic programming [7]). Kruchten et al. [28] introduce a specialized
form of gallery for comparing features among visualization libraries. We
consider the more common case of single-tool multipurpose galleries.

##  III Methodology

We next describe the methods used in our studies.

Gallery Survey. To situate our interview study with users and creators of
example galleries, we develop a baseline answer to the question _what is a
gallery?_ by surveying 14 visualization-adjacent notations (Fig. 3). We
conveniently sampled via searching for terms like “visualization” and
“gallery” on web search engines. We selected galleries using a human-usable
computer language (per Fowler [14]) that consist of an official collection of
visual examples rendered from self-contained code. This excludes SQL-like
query languages or dataframe-like APIs for data processing and data
transformation. Similarly, this filters out natural language tools (e.g. NL4DV
[39]) as well as partial-specification DSLs that operate over other DSLs (e.g.
Draco [65]). We excluded benchmark or dataset repositories (e.g. vizNet [23])
which are curated to train or test systems rather than be used by humans. Our
definition of galleries used to identify elements in our survey is
provisional: since there was no prior formal definition, we describe the
general characteristics of instances regarded as “galleries” by their creators
and users. Similarly, this survey is not exhaustive, but merely develops a
baseline for gallery form from which to measure interviewee responses.

Interview Studies. We conducted a semi-structured interview study with (N=11)
gallery creators and (N=9) with users of those galleries. Interviews with
creators aimed to answer why and how do authors create example galleries? Our
interviews with users focused on corroborating our findings from creators.

Gallery creators, denoted C${}_{\scriptstyle\textit{example}}$ and “quoted”,
were recruited via email. They were identified from our gallery survey and
snowball sampling. We interviewed two sets of overlapping authors:
C${}_{\scriptstyle\textit{Jeff}}$ and C${}_{\scriptstyle\textit{Dominik}}$
both worked on Vega and Vega-Lite, while C${}_{\scriptstyle\textit{Xiaoji}}$
and C${}_{\scriptstyle\textit{Ib}}$ both worked on Deck.GL. All but one
participant identified as male. Most creators had at $\geq 5$ years experience
designing DSLs or APIs and had designed at $\geq 2$ galleries. Users (referred
to as U${}_{\scriptstyle\textit{1}}$ and quoted “like this”) were recruited
via social media. We sought users with experience using Altair, Vega-Lite, and
Plot.ly. We limited our focus to these tools to narrow our scope and because
their conceptual overlap provides points of direct comparison: Altair is a
python wrapper for Vega-Lite, while Plot.ly is strongly influenced by Vega-
Lite. We used a screening survey to vary our sample across background and
notational experience. (8/9) have $\geq$2 years experience, (7/9) hold degrees
in computing, (4/9) identified as female. We piloted the study with two
creators and three users. It was marked exempt by the University of Washington
IRB.

We conducted all interviews over Zoom. Consent was elicited via email prior to
the interview. Creators received a $50 gift card in compensation. Users
received a $15 gift card. Our interview scripts can be found in the appendix.
Interviews lasted $\sim$60 minutes with creators and $\sim$30 minutes with
users. Demographics were solicited via post-interview survey. Interviews were
transcribed by the first author using Otter.ai and then open-coded to identify
the initial themes. The research team iteratively refined these concepts.
Themes were finalized via an affinity diagramming [32] session.

Positionality. Several participants have worked with the research team, see
appendix for an enumeration of these connections. In an evaluative study, this
would pose ethical quandaries about participants’ ability to give unbiased
answers. We seek to understand and synthesize these expert’s experiences.
While we try to circumvent these issues via iterative theming and reflection,
our relationships with participants will inevitably bias our results. Of
particular note is our connection with Heer (as colleague, advisor, and
collaborator) and his collaborators—for instance he has worked with
C${}_{\scriptstyle\textit{Dominik}}$ (on Vega),
C${}_{\scriptstyle\textit{Fil}}$ (on D3), and
C${}_{\scriptstyle\textit{Matt}}$ (on Idyll). This centers our results on the
approach to tool-making expressed through his work and the (large) surrounding
community he has fostered.

Name |  Main notation
---|---
Jeff Heer |  Vega [52] _grammar of interactive visualizations_
Matt Conlen |  Idyll [8] _markup language for online interactive articles_
Nicolas Kruchten |  Plot.ly [45] _commercial open-source graphing library_
Carlos Scheidegger |  Quarto [46] _open-source scientific and technical publishing system_
Xiaoji Chen |  deck.gl [42] _WebGL-powered framework for large-scale data visualization_
Ib Green
Josh Sunshine |  Penrose [66] _platform for creating diagrams illustrating concepts_
Dominik Moritz |  Vega-Lite [51] _high-level grammar of interactive graphics_
Philippe
Rivière (Fil) |  D3 [5] _low-level JavaScript library for interactive data visualizations_
Will
Crichton |  Nota [11] _a language for writing modern documents_
Peter Vidos |  ipyvizzu [61] _animated charting library_
Figure 1: We interviewed 11 gallery creators or maintainers. We include their
de-anonymized names (with consent) to credit them for their work and insights.

##  IV What are galleries?

As with many things in engineering, there is no one definition of a gallery.
The type of galleries employed seems to be informed by library nature and
construct. We identified three forms of example galleries: showcase (analogous
to a carefully curated museum gallery, in which each piece is a demonstration
of a realistic use case), reference (analogous to code examples in API
references that are minimal and focused), and index (analogous to a phone book
or table of contents). Each library contains at least one main gallery that
can typically be accessed from the navigation bar. For example, (a) and (c) of
Fig. 2 show small slices of Vega and Plot.ly’s main galleries. More complex
and customized examples appear either as a “showcase” or “miscellaneous”
category in the main gallery or as an independent gallery focused on that
topic (such as ipyvizzu), depending on the number of examples. Some libraries
incorporate a smaller reference gallery that resembles the references type of
documentation: it details the lowest level breakdown of library usage.
However, unlike the API references where the textual information appears first
followed by an optional code example, the reference galleries are dominated by
leading visual examples followed by explanations. To wit, Idyll’s [8] main
gallery (Fig. 2 (e)) accommodates interactive articles, whereas its “built-in
components” section (Fig. 2 (f)) exhaustively lists the components orthogonal
to each other in terms of functionality.

Similarly, creators did not express a cohesive opinion on what constitutes a
gallery. Some seemed to regard them as the interface to a database of self-
contained examples (Matplotlib), whereas others saw them as more bespoke
curated collections (deck.gl). This heterogeneity can lead to strong opinions,
for instance C${}_{\scriptstyle\textit{Nicolas}}$ argued that Plot.ly’s
“Examples” webpage (Fig. 2 (c)) is not a gallery, noting that the thumbnails
are not selected from their examples, but are rather visually appealing images
representing the themes of visualization collections. In NotaScope [28],
C${}_{\scriptstyle\textit{Nicolas}}$ and co-authors define galleries as a way
to evaluate the expressive or generative power of a notation. While it
conforms to one of the goals for galleries in our findings, we extend the
definition and regard Plot.ly’s “Examples” webpage as an index gallery. From
these views we synthesize the following definition:
A gallery is a (potentially) organized _collection_ of examples, which consist
of code, visual output, and metadata, and an _interface_ through which to use
those examples.

Figure 2: Vega, Plot.ly and Idyll Galleries. Gallery organization and content
depend on the goals and scope of the project they describe. Larger galleries
are often categorized by features or task. To wit, Vega groups the examples as
bar chart, line charts and so on, whereas Plot.ly groups by data domain like
finance or statistics. In contrast Idyll’s gallery is bifurcated into large
high-level article examples and small low-level component examples.

##  V Gallery Motivations

Here we identify motivations guiding gallery design such as using the gallery
for marketing, as objects for quick reuse, and as a component of the system
infrastructure.

Gallery as Marketing or Advertising. An example gallery is often the face of
the library, acting as an ambassador for the library as a whole. Rich
evocative galleries can serve as marketing materials by demonstrating the
design space of possibilities for the library, possibly attracting new users
or helping prospective users differentiate from similar offerings. While the
design space is interpolated from the collection components, effectively
communicating it depends on the interface component.
C${}_{\scriptstyle\textit{Matt}}$ observed that “people come to the site, and
the first thing  $[isthat]$ they’ll scroll through the examples to see ‘if I
were to try to make something like this? Is this up to my quality standards?”’
Galleries designed with motivation emphasize demonstrating expressiveness and
unique facilities of the language—a signal of development status and evidence
of community—as well as sketching what is possible with a notation—so as to
elicit adoption or usage.

This interest in marketing affects the selection of gallery elements.
C${}_{\scriptstyle\textit{Jeff}}$ and C${}_{\scriptstyle\textit{Fil}}$ noted
that some highly customized examples are aimed and are not expected to be
frequently reused. For instance, a community contributed example shows that it
is possible to construct a working Pac Man game in Vega, which can serve to
highlight the power of the library and promote interest in it. Such examples
are added to show the upper bound of what it can achieve as a testament of
power. Similarly, C${}_{\scriptstyle\textit{Matt}}$,
C${}_{\scriptstyle\textit{Josh}}$, and C${}_{\scriptstyle\textit{Xiaoji}}$
noted that they consider aesthetics to be an important factor. To this end
C${}_{\scriptstyle\textit{Josh}}$ noted “we have other examples that are
available in the source code repository in GitHub. But they’re not as visual
appealing or widely useful. So the gallery is intended to be a subset of those
that are visually appealing and also representative of the kind of things that
we expect our users to want to do”. Besides the official gallery, to help
answer users’ questions C${}_{\scriptstyle\textit{Fil}}$ also maintains a
personal collection of Observable notebooks, and creates one-time examples
that are archived online afterward. These suggest a multi-tier example
database with a front-facing curated gallery and a repository for additional
examples.

Commercialized libraries such as Plot.ly put in enormous effort to compete for
market share, and often leverage their gallery not only for marketing its
features, but also as a means of Search Engine Optimization (SEO). This is
done with the explicit goal to be at the top of search results for keywords
like (e.g. “Python scatter plot”). C${}_{\scriptstyle\textit{Nicolas}}$
explained that the strategy involved ensuring that “every feature gets an
example…It’s not something that’s meant to be browsable as a unit; it’s meant
to be linked directly to any sort of scroll up and down and optimize for using
CMD+F within a page or Google or our search feature”. For example,
U${}_{\scriptstyle\textit{2}}$ started exploring Plot.ly after finding a
StackOverflow answer implementing the design they need (e.g. Kernel Density
plot with zooming).

An important usage in this advertising situation is supporting comparisons
with other prominent tools—potentially helping users answer the question _why
should I use this tool instead of another?_ For instance,
C${}_{\scriptstyle\textit{Jeff}}$ observed that “galleries from D3 will
typically involve lots of visualizations that you can’t make with ggplot2, or
Observable Plot or with Vega-Lite, because they target different levels of
expressiveness. And so as you start to do some of that sort of comparative
research across tools, you will also get a sense”. Multiple galleries together
help users select a suitable library by comparison [55, 28]. Users recalled
needing to use a tool’s gallery to “figure out what it can and cannot do”
(U${}_{\scriptstyle\textit{2}}$) and “learn the visualization vocabulary”
(U${}_{\scriptstyle\textit{1,3,6}}$) when they started to learn the libraries.
User participants suggested that they tended to select tools based on the
interactions the library supports (U${}_{\scriptstyle\textit{1,2,4,6,7}}$),
ease-of-use (U${}_{\scriptstyle\textit{1,5-7}}$), and its popularity
(U${}_{\scriptstyle\textit{2-7}}$). Gallery’s can signal these traits, such as
by showing GitHub stars, impressive quick starts, or just by showing the gamut
of possible interactions.

Users consider adopting a library based on certain qualities that facilitate
their specific goals. For instance, in comparing Plot.ly, Altair and Vega-
Lite, our user participants expressed a variety of usage. These include
analyzing datasets (U${}_{\scriptstyle\textit{1,2,4}}$), presenting findings
(U${}_{\scriptstyle\textit{1,2}}$), academic publishing
(U${}_{\scriptstyle\textit{1,3,5,6}}$), testing research prototypes
(U${}_{\scriptstyle\textit{1,3}}$), and teaching data science classes
(U${}_{\scriptstyle\textit{1,7}}$). This is then sometimes reflected in
gallery organization, for instance matplotlib explicitly includes sections
focused on various domain tasks relevant to those roles. Some differentiating
features can be simple. For instance, U${}_{\scriptstyle\textit{3}}$
integrated Vega-Lite to generate SVGs used in their research prototype. While
other tools support that output format, Vega-Lite did so more explicitly.

Gallery as Reusable Template Corpus. Another important motivation for gallery
construction is to help users overcome the blank page problem, such as by
providing inspiration and facilitating reuse. Gallery collection determines
the existence of reusable contents, whereas its interface affects the effort
to locate them. Galleries developed with this motivation seek to provide ideas
for common tasks (e.g. making quotidian chart types like bar charts) as well
as design alternatives by displaying the possibilities.
C${}_{\scriptstyle\textit{Jeff}}$ observed that a gallery can “give them an
idea for visualizations they might not have considered otherwise. And so it’s
part of showing what’s possible, and the way it might inspire someone.”

Some creators believe that examples are designed to be repurposed, functioning
almost like a slow auto-complete. C${}_{\scriptstyle\textit{Dominik}}$ added
that this use of examples is similar to templates [3, 34], noting that
“Galleries are just fairly lightweight way to get lots and lots of different
programs that show different aspects and different patterns. And hopefully
provide good starting points for people to copy and modify. So they don’t have
to start from scratch”. This follows a pattern known as opportunistic reuse
[7], in which users copy and paste code, and then modify it to meet their
needs. Users expressed exactly this behavior, explicitly noting that they seek
examples for reuse when they have a target chart type in mind. For instance,
U${}_{\scriptstyle\textit{5}}$ browses the gallery examples looking for
additional accessories when they have a base design.
U${}_{\scriptstyle\textit{3}}$ is familiar with the gallery and has a mental
map of each chart’s location, so they can easily jump to the example needed
and use it as starter code. U${}_{\scriptstyle\textit{4}}$ prefers to view the
examples as an aid to iterate over the design they brainstormed: “so it’s more
of a learning process…there were visualizations that you did not imagine, but
now that you know them makes more sense than the things before. ” When using
tools like D3 or ggplot2, they often search for more elaborate designs from
sources like “Information is Beautiful” [33] (U${}_{\scriptstyle\textit{1}}$)
and search engines (U${}_{\scriptstyle\textit{5}}$).

Galleries do not serve every (re)use case—with folk documentation sometimes
being more accessible. For instance, when using D3,
U${}_{\scriptstyle\textit{5}}$ prefers searching online to the gallery because
searching is likely to direct them to Stack Overflow where a lot of questions
are directly answered. Similarly, U${}_{\scriptstyle\textit{1,2,6,7}}$ added
that they make common charts types (e.g. line charts) from scratch because it
only requires a few lines of code from memory. They refer to the gallery when
in need of examples that are “not the simplest, like anything stacked”
(U${}_{\scriptstyle\textit{6}}$) and when “it’s just easier to start from
something exists than from scratch” (U${}_{\scriptstyle\textit{3}}$).
Similarly, not every gallery is built to service this style of usage.
C${}_{\scriptstyle\textit{Nicolas}}$ was skeptical about the benefits of
opportunistic reuse in reality for smaller galleries (e.g. Seaborn): “the
likelihood that one of these things happens to look like what you’re trying to
make is pretty low. I mean, the design space for charts is huge, right? I just
think that the browsing approach to like, providing this information to users
doesn’t scale.” Opportunistic reuse is less likely to be applied to showcases
and highly-customized examples since they also require the users to interpret
the code, decompose and extract relevant pieces.

Gallery as Test Infrastructure. Finally, some creators were motivated by using
gallery examples as part of their test infrastructure, namely as unit tests.
Unit testing is an established practice motivated by developers’ conviction
and management requirements to ensure software quality [12]. However,
programmers often look for usage examples from unit tests, when there are no
better-documented options, even though it can be time-consuming [40]. In
contrast, galleries contain self-exploratory usage examples mainly documented
for the users, but they benefit the creators as end-to-end test cases.
Compared to low-level unit tests, they give an opportunity to check how well
things work when fused together (C${}_{\scriptstyle\textit{Jeff}}$). They help
developers quickly validate the ideas, serve as a debugging tool for local
development via visual inspection (C${}_{\scriptstyle\textit{Xiaoji}}$), and
aid in the discovery of performance issues (C${}_{\scriptstyle\textit{Matt,
Ib}}$). While the testing is mainly carried out through the collection, the
interface provides a means of inspection. Similarly, while this approach does
not directly benefit the end users, it can affect the perception of the
library, such as via coverage badges and commit activity (as signal for
stability).

However, using galleries for testing requires setting up testing
infrastructure in addition to the general gallery maintenance. This can be
beneficial: Penrose guarantees every example in the repository keeps running
using continuous integration (C${}_{\scriptstyle\textit{Josh}}$). The
granularity of these tests can vary. For instance Vega-Lite and Penrose
programmatically check the rendered SVGs for each gallery example against
snapshots to assure consistency across code changes, while Quarto merely
checks that the documentation site successfully loads and produces documents
using browser automation. To refine the testing pipeline,
C${}_{\scriptstyle\textit{Carlos}}$ planned on inspecting the example contents
“to not only generate output, but also assert things about the output.”
However, the utility of automation has limits, with manual inspections still
being unavoidable in some cases. For instance, the effect of code changes can
be subtle and not visually noticeable (e.g. slight changes from Penrose’s
layout optimization algorithm), or they can be significant and still correct
by definition. As a result, they require human intervention as part of the
testing process. C${}_{\scriptstyle\textit{Jeff}}$ mentioned checking if all
the charts looked correct before a big release. Narrowing the gap of
evaluation between programs and their intended interfaces/outputs seems to be
a crucial but also incredibly challenging goal for the creators to achieve,
and the gap seems to widen as libraries become more expressive.

Figure 3:  Example counts in galleries we explored to situate our analysis.

##  VI Cross-cutting Themes

We now explore several cross-cutting themes from across the gallery and their
relationship with different motivations.

###  VI-A Education and Communication

Galleries are often the main point of contact between users and creators,
meaning they are laden with communicative tasks, such as showing how to use a
library. C${}_{\scriptstyle\textit{Nicolas}}$ emphasized that “if it’s not in
the docs, you may as well not have built it. No one is ever going to use it if
it’s not in the docs.” Here we discuss issues related to communication of
functionality.

Are galleries a teaching tool? Some creators saw galleries as a teaching aid
that helps engage and educate people in exploring the library.
C${}_{\scriptstyle\textit{Jeff}}$ noted that while they lack the detail of a
tutorial, galleries allow users to see how things are constructed and how
goals are achieved with the library. In contrast,
C${}_{\scriptstyle\textit{Josh}}$ expressed a marketing-centric view, arguing
that the gallery is not the best place to teach a library. He argued that “the
main purpose of the galleries is an advertisement, not an explanation or
something that people are intended to reuse.”
C${}_{\scriptstyle\textit{Matt}}$ echoed this perspective, noting that while
people learn from examples, but “if you want to teach something, you want a
simple example that highlights the thing you’re trying to teach, which are not
usually good fits for the gallery because of the visual complexity and their
applicability to a lot of people”.

The user participants all started their journey with the charting libraries by
playing with the gallery examples. As beginners, they were not “trying to
become an expert, but just wanted to use it” (U${}_{\scriptstyle\textit{5}}$).
They browse the overview page and select basic examples to replicate. Then
they slightly tweak the examples by changing the parameters, and that was when
they refer to the references for detail. In addition,
U${}_{\scriptstyle\textit{3}}$ highlighted that he learned about visualization
research and vocabulary by studying Vega-Lite’s gallery. We infer that the
effectiveness of gallery as a learning tool depends on gallery structure,
library construct, and the nature of the domain. Galleries of charting
libraries are broadly used as learning tools because they are often
categorized based on well-studied taxonomies, and provide basic and
straightforward examples. In contrast, Idyll and Penrose have no such
categorization to lean on.

Examples are sometimes used as a means to educate about a particular topic.
C${}_{\scriptstyle\textit{Jeff}}$ sought to have Vega gallery examples to
convey useful data facts in addition to features of the system. Correll [9],
referring to example data more generally, calls for selecting datasets that
are of social import or impact as means for advocating for positive change. We
suggest that this usage of examples is under-explored and may benefit library
marketing (via novel examples) as well as social good.

Crichton [10] (_i.e_. C${}_{\scriptstyle\textit{Will}}$) argued that
documentation generation can be seen as a form of information visualization.
We connect this observation with Lee-Robbins’ [31] characterization of
visualization as being a means of fulfilling learning objectives. We argue
that galleries are then, in essence, a visualization of what is possible in a
gallery in a means that is meant to support learning of those features—whether
or not that learning is intentional. Although the creators and users agree on
the tutorials being a more conventional teaching material for the beginners,
gallery examples provide shortcuts for more focused further learning. We
suggest that developing designs to help users bridge from gallery use to
tutorial usage is an important documentation challenge, and may be an area
where new tools may be helpful.

Community Examples. Eliciting external examples is a commonly used means to
expand galleries. For instance, after making an initial collection of showcase
examples for Idyll, C${}_{\scriptstyle\textit{Matt}}$ gathered more instances
in the form of student projects from a course he taught at University of
Washington. C${}_{\scriptstyle\textit{Jeff}}$,
C${}_{\scriptstyle\textit{Dominik}}$, C${}_{\scriptstyle\textit{Carlos}}$, and
C${}_{\scriptstyle\textit{Josh}}$ echoed the value of selecting existing
external examples as gallery entries. To this end, some libraries (e.g. p5.js
[44]) specifically facilitate new users’ ability to contribute examples (and
other forms of documentation), and try to make the process of adding to that
documentation a welcoming and easy process.

In some cases external community resources form to support folk curation of
these examples, such as on StackOverflow or in a dedicated website like
bl.ocks.org. The latter of these is a (defunct) means for the d3 user
community to collect and browse examples. Making a formal connection with this
type of folk resource is a potential area of tool support.
C${}_{\scriptstyle\textit{Dominik}}$ envisioned a two-tier system where a
maintainer can “check a box” for a community example to promote it into the
official gallery. Forked versions could be tracked to show variations—perhaps
inspiring a remix culture à la OpenProcessing [54].

Community examples are not a silver bullet. Despite the example creation labor
being outsourced, the curation and management overhead for quality control
still apply to monitoring user-contributed examples. Similarly,
C${}_{\scriptstyle\textit{Jeff}}$ noted that they can muddy the utility of a
tailored collection of examples. We note that this tends to cause these
examples to be more like showcase examples (which is how Quarto uses them),
which can limit their re-usability. Imposing strong restrictions may support
higher quality examples, but can also severely reduce the number of community
submissions to the gallery.

Communication & Expectations. Barriers exist both in delivering information
from the creators via documentation and receiving questions and feedback from
the users. While a number of channels are available (ranging from the
galleries themselves, to secondary channels like GitHub or Discord), designing
an appropriate mixture of communication forms is dependent on the community
around those tools.

The inherent knowledge asymmetry between users and creators makes it difficult
to form and select examples. For instance, C${}_{\scriptstyle\textit{Carlos}}$
highlighted that Quarto has a large and diverse set of users with varied
technical background, noting “our examples use pandas because we expect it to
be a familiar tool for Python users, but it’s a very real problem that we have
to guess what’s the knowledge base, and have different ways to explain things
in code”. User participants struggled with integrating multiple tools
(U${}_{\scriptstyle\textit{6}}$) and spent considerable time adapting data to
match visualization specifications (U${}_{\scriptstyle\textit{1-7}}$).

Some creators interact with users on social media platforms to answer
questions, collect feedback, and observe how users review the tool. Most users
would not interact with the creators directly, however, they typically find
answers to their questions through StackOverflow or GitHub issues
(U${}_{\scriptstyle\textit{1,2,4,6}}$), and reviews containing research done
by others (U${}_{\scriptstyle\textit{2}}$). When users express confusion about
gallery examples in submitted GitHub issues,
C${}_{\scriptstyle\textit{Xiaoji}}$ adds more explanation to the Deck.gl
examples. In these interactions users contribute feedback that can help shape
the tool collaboratively in manners that increase its usability (supporting
both marketing and reuse goals). They can also act as a slow form of testing
by communicating the pain points they encounter with the tool. To this end,
one surprising channel of communication from user to creators is through user-
contributed examples. For instance C${}_{\scriptstyle\textit{Carlos}}$ uses
the community contributed Quarto showcase as a way to monitor how people use
the tool: “a number of the things that people have built are not things we
would ever have considered. There’s some very advanced things that people do
with styles that are really nice”. Correctly positioning the role of
contributed examples can then serve a dual purpose of providing documentation
and soliciting feedback.

Communicating correct usage patterns is essential, both to ensure that users
understand what the notation can do (as in the comparative research use case
of marketing), as well as show how to do those things. Ineffective
articulation of these patterns may lead to user confusion. For example,
mismatches in example-suggested programming patterns and the user’s target
implementation can hamper code reuse, as observed by
U${}_{\scriptstyle\textit{1}}$: “sometimes they wrap the visual functions in a
class, and it’s hard to extend to some edge cases.”
C${}_{\scriptstyle\textit{Ib}}$ observed different strength and hurdles for
community in different groups for developers familiar with functional
paradigm, which Deck.GL uses, or imperative programmers. It is intended to be
used by web developers who are familiar with React and its functional
paradigm, and can be confusing to other groups. In contrast, React users can
have their own problem: “it’s very hard to combine React with an imperative
API, which Deck.GL $[oftenneedsand]$ works extremely well with”
(C${}_{\scriptstyle\textit{Ib}}$). The usability of code examples and
documentation is affected by how the notation is designed, just as other parts
of technical systems [25]. Creators make assumptions about usage patterns and
work backwards to inform their design decisions. While these decisions might
benefit the _assumed_ user group, they could make it less flexible for less
expected or expert users.

Similarly, concepts from within the domain are difficult to capture
unambiguously. For instance, in selecting appropriate names for examples,
C${}_{\scriptstyle\textit{Nicolas}}$ observed that “It wasn’t super clear,
what would you call things. Like do you call things facet and sort of
trellises or small multiples or panel charts or lattice? I don’t know. So we
tended to kind of go with everything, like, make sure we can stick all the
keywords in the page.” Communicating usage patterns is extremely difficult for
libraries supporting interactions and animations because they add extra
dimensions to the design space, which also requires more domain knowledge. For
instance, U${}_{\scriptstyle\textit{4}}$ complained about the difficulty of
identifying valid combinations of static design and animation operations.
Users care about the end-results rather than the steps that lead to them,
however, the space of possible usage patterns can be large, which can make
searching for specific solutions challenging. _Instead of merely enumerating
the options (via the gallery), generative recommenders might offer result
tuned to user questions._

###  VI-B Maintenance

Gallery creation and maintenance is a huge burden. Here we review several
challenges latent to this process.

Resource constraints. Growing a gallery can be extremely demanding without
immediately demonstrable benefits. Moreover, it is unclear how to most
effectively distribute limited developer time. Should galleries be weighted
above over tutorials? Is new documentation or a new feature more important?

Weighing changes to the maintenance workload against the library goals is one
means to determine resource allocation. For example, Plot.ly explicitly
focuses on Search Engine Optimization (SEO), likely in part because they have
such strong marketing centered goals, and so the costs of maintenance are
offset by the potential value of increased SEO. In contrast,
C${}_{\scriptstyle\textit{Ib}}$ believed in spending more time building great
features to attract users rather than add more examples and improve the
documentation. Libraries like Seaborn are led by a few part-time developers
with “probably too little time to consider the best things to do with
galleries” (C${}_{\scriptstyle\textit{Nicolas}}$). This contrast can be seen
in Fig. 3: Plot.ly has nearly 13$\times$ more examples than Seaborn. Is this a
better allocation of resources? It depends on the intent of the library.

Creating a gallery is usually easier at the beginning as presenting
straightforward use cases that justify the need for new features in a library.
Although, after developing new features, C${}_{\scriptstyle\textit{Ib}}$
described how they implement the corresponding examples, noting that they
“have to do all the extra work of actually create a beautiful example or
finding a dataset that’s public with good license. So sometimes it’s just like
an extra tax on the work to do it”. Further, as a gallery grows, it requires a
series of careful considerations to select and organize the examples.
Additional interface design (such as search bars, categories, or other
affordances) for the gallery becomes essential for efficient example
discovery, but it can introduce more cognitive overhead.
C${}_{\scriptstyle\textit{Josh}}$ explained that “if you have a category or
directory or a search, then you have to maintain that. It’s another feature to
deal with. And it adds complexity to the user experience a little bit, because
it’s another thing they have to go through in order to get started. So there’s
no reason to do it till you have to”. This may suggest that tools to support
the automated construction and organization of galleries would be useful, as
it would relieve some of this burden. However, the prominence of auto-doc
tools like Sphinx [13] would suggest that this problem is not as simple as
building a new documentation system.

Maintaining and Coordinating Artifacts. There are multiple artifacts around
galleries that require coordination. These include the gallery examples,
software features, testing suite, and the software ecosystem more broadly.
Actions on any of them would likely cause consequences on another component.
For example, C${}_{\scriptstyle\textit{Dominik}}$ described that when a new
example is added to Vega-Lite, it is required to be included in the Vega
Editor (_i.e_. online playground). Further, if there are changes to an
example’s data, the vega-datasets repository needs to be updated, which in
turn involves ensuring that other examples do not break. Notebooks such as
Observable provide a web-based platform for hosting customized examples
written in all sorts of libraries a user can import.
C${}_{\scriptstyle\textit{Fil}}$ mentioned how using Observable reduced the
workload for managing the gallery interfaces of D3 and Observable Plot since
the examples are just links to notebooks. However, making a system seamless
for someone inevitably makes it frictional for someone else [19]. For
instance, moving documentation into notebooks (or Discords in some cases) can
make that documentation less discoverable (e.g. for search engines) and
potentially more intimidating to some users.

All creators expressed that it is challenging to maintain galleries when the
core software changes. C${}_{\scriptstyle\textit{Matt}}$ explained that
“maintaining documentation for an open source project is a real drag for the
most part”, going on to note that “every time you do that [fixing bugs,
updating APIs, etc.] you have to go and update the documentation too. So it’s
like this added layer of work, and it becomes hard to keep things in sync.”
Manually inspecting each example is a common task before releasing.
C${}_{\scriptstyle\textit{Jeff}}$ mentioned going through examples manually as
a sanity check before big releases because not all examples have a one-to-one
mapping to testing and validation. C${}_{\scriptstyle\textit{Josh}}$ reflected
on the experience of having to re-implement every example after updating
Penrose’s features, noting that even if everything works well, “[adding more
examples] adds time to the testing [because] we run a test suite every time we
commit something to the repository.” U${}_{\scriptstyle\textit{5}}$ disliked
how software version updates can affect the community examples, potentially
breaking examples and confusing readers (particularly of unofficial
documentation, like on StackOverflow). C${}_{\scriptstyle\textit{Nicolas}}$
stated that software with a larger and more stable audience (such as the
marketing-centric Plot.ly) enforces a stronger commitment for backward
compatibility and instead “treats everything as the documentation problem,
because [the library] is pretty hard to evolve”. Newer libraries are likely to
go through major changes, requiring additional labor to fix the examples. In
contrast, mature libraries tend to keep everything as is, which ensures
stability but can impede new features.

As a library phases into the maintenance period from active development, the
frequency of adding new examples also slows down. The creators occasionally
get inspirations and implement new use cases. Yet more often, enthusiastic
users contributed a burst of examples in short period of time as the user base
grow (C${}_{\scriptstyle\textit{Xiaoji}}$). Another way to create new examples
is demand-driven in that multiple users asking about a similar question
indicates the need for a new example C${}_{\scriptstyle\textit{Nicolas, Jeff,
Fil}}$. Library creators and maintainers answer questions by showing examples.
And some of the examples become part of the gallery, while others are never
revisited by the authors (C${}_{\scriptstyle\textit{Fil}}$). Maintenance is
typically invisible labor [1] that rarely garners the attention that new
feature work does, but may offer a marketing-style signal that the library is
actively maintained.

###  VI-C Gallery Contents

Finally, galleries are agglomerative objects. They consist of many constituent
objects, the organization and selection of which is guided by balancing
different views and constraints.

What’s an example? The main contents in an example are the caption, the
visual, the explanation, and the code. The galleries all contain these
elements but with different levels of detail. For example, a detailed chart
example can incorporate the background of the data source and an explanation
of why the visual encodings were selected to be more educational.

Creators often took inspiration from other galleries in the selection and
design of their examples. This expresses a marketing-tact: it allows them to
show that they have the parity of what others can do and more. For instance,
being competitive for market share, Plot.ly reproduces examples from ggplot2
and Matplotlib, “literally saying ‘do you like this Matplotlib plot and here’s
exactly how you make it”’ (C${}_{\scriptstyle\textit{Nicolas}}$). This is akin
to how a new library is also a reaction to the flaws (_i.e_. negative space)
of its alternatives, which C${}_{\scriptstyle\textit{Ib}}$ highlighted as
being especially important. C${}_{\scriptstyle\textit{Dominik}}$ noted that
these considerations started from the beginning of Vega-Lite: “then we have
some examples of showcasing different things people often want to do. And
that’s how we actually designed language: we made examples and invert the
language  $[fromtheexamples]$ so that it works with the examples that we
designed”.

Creators become more selective as more examples are added
(C${}_{\scriptstyle\textit{Matt}}$), and they ensure that the difference of a
new example from existing ones has a clear target in functionality or learning
objectives by carefully designed ordering (C${}_{\scriptstyle\textit{Jeff}}$).
In general, creators avoid repetitive examples that overwhelm users by the
quantity. C${}_{\scriptstyle\textit{Jeff}}$,
C${}_{\scriptstyle\textit{Dominik}}$, and C${}_{\scriptstyle\textit{Josh}}$
noted that try to keep the examples intriguing while filtering out niche
examples that distract users with irrelevant details. For example,
C${}_{\scriptstyle\textit{Dominik}}$ noted that new Vega-Lite examples reuse
datasets from previous examples when possible to share data domain knowledge
across examples. C${}_{\scriptstyle\textit{Ib}}$ observed a related tension
between performance and understandability, noting that: “to make sure they
load fast we apply some compression on those datasets, and the example code is
no longer easy to understand by the end user.” Selecting examples requires
balancing their need to interesting, understandable, re-usable, and to not
unduly burden the infrastructure.

Gallery Organization. The order in which gallery examples are presented and
the means by which they are organized strongly affects the readability. Most
galleries navigate this challenge by sorting and clustering by complexity or
similarity.

The galleries tend to start with simpler examples and progress to more
specialized or complex cases (e.g., layers and interactions in Vega-Lite),
both throughout the entire gallery and within each category. D3, as a special
case, arranges the more complicated categories such as animation and
interaction at the beginning and the basic charts at the end.
C${}_{\scriptstyle\textit{Fil}}$ explained that they intentionally arrange the
gallery in this way: “it serves the users better to use Observable Plot (a
high-level charting library by the D3 creators) because they will have all the
defaults that help them create basic charts.” Conceptually and visually
similar examples are placed closer together to avoid losing track or
confusion.

Several other strategies were prominent as well. For instance,
C${}_{\scriptstyle\textit{Carlos}}$ explained that they would shape Quarto’s
components around how they present the usage in the documentation. If a
feature does not fit well with the narratives they will redesign it to match
the concepts behind. In contrast, Plot.ly’s emphasis on marketing affects how
their gallery is organized: an extensive set of design variants with
descriptive headings are stocked in the same webpage, so that the webpage can
be better indexed by search engines (_i.e_. for SEO).

Users have a finely tuned process for sorting through examples and deciding
relevant instances. They look at different cues in the environment, such as
the example titles and the code structures. Even when they look at unfamiliar
galleries, they are capable of interpolating from their experience with other
libraries or programming in general. Vega-Lite and Altair users
(U${}_{\scriptstyle\textit{3-6}}$) browse the galleries when searching for
relevant examples, while two of the Plot.ly users
(U${}_{\scriptstyle\textit{2,7}}$) start from Google searches as expected by
the gallery designers. U${}_{\scriptstyle\textit{3}}$ found the categorization
useful for quickly locating the charts, compared to
U${}_{\scriptstyle\textit{6}}$ who is not as conscious but became aware of the
organization: “I have seen that page a fair amount of time by now. So I know
how it’s organized. I know that the slightly crazier things and the maps are
at the bottom. ” U${}_{\scriptstyle\textit{1}}$ spoke about how they wish
there were an additional overview of all variant examples with a “more
scientific” designed arrangement—highlighting the potential value of design
interventions for helping users understand gallery contents.

Gallery Peripherals. Finally, augmenting this gallery organization and content
are a family of other tools, that provide context, support inter-connections
with the rest of the documentation, and bridge gaps left by the default
presentation.

Besides common elements like text and image, a few galleries employ cross-
links to references, tutorials and other relevant examples, and tags of
keywords. A playground or online editor is a common documentation component
for external DSL-style libraries [14] (e.g. Vega/Vega-Lite, Penrose, Idyll,
Nota, etc.). These tools allow users to play with the examples and make
modifications in-situ. For instance, the Vega and Vega-Lite editor [59]
includes specialized affordances for those libraries, such as debugging [20]
and performance [64] tools. Similarly, it supports link sharing so that
specifications, examples, and demos can be distributed. This design supports
both conceptual reuse—as it make it easy to make adjustments to existing
examples—as well as marketing—as it can demonstrate the facilities of the
library more clearly than static images of gifs. Secondary tools that are
woven into the documentation can help support its usage and is an opportunity
for tool research.

Galleries are one of the many tools utilized in user’s workflows. For
instance, U${}_{\scriptstyle\textit{6}}$ processes data with Pandas, generates
the base chart in Altair, and uses Adobe Illustrator to modify the aesthetics
and add annotations. U${}_{\scriptstyle\textit{3}}$ also switches between
Vega-Lite and other tools so that he repurposes Vega-lite generated SVGs to
the data storytelling usage scenario by adding annotations and animation. Most
galleries only provide succinct examples which required less reading and code
decomposition, leaving a gap for integrating the library with other
environments. Many users employed AI-assistance as a component of their daily
workflows. For instance, U${}_{\scriptstyle\textit{1}}$,
U${}_{\scriptstyle\textit{2}}$,U${}_{\scriptstyle\textit{4}}$, and
U${}_{\scriptstyle\textit{5}}$ mentioned using ChatGPT, Phind, and Copilot
either in combination with the galleries or as the replacement for specific
tasks. U${}_{\scriptstyle\textit{4}}$ explained that they first inspect the
code recommended by Copilot based on previous code (e.g. data manipulation
code) or comments. U${}_{\scriptstyle\textit{2}}$ turns to Phind as a
replacement for Google search, while U${}_{\scriptstyle\textit{1}}$ and
U${}_{\scriptstyle\textit{5}}$ use ChatGPT for implementation and debugging
tasks. Framing the Google search queries gets “progressively and exponentially
more difficult” when they are in later stages inof a project to “get a bit
more creative to format the search” (U${}_{\scriptstyle\textit{2}}$). With
LLMs, U${}_{\scriptstyle\textit{2}}$ can “explain more about what  $[theyare]$
trying to do”, even when they could not figure out an appropriate query “as a
non-native speaker  $[inEnglish]$ ”. Users encountered well known bugs with
these tools. U${}_{\scriptstyle\textit{5}}$ complained about the model mixing
up syntax versions or hallucinating invalid code, especially for newer or less
popular libraries, forcing them to “spend more time on fixing the errors and
then realize that the code doesn’t work at all.” For the same reason,
U${}_{\scriptstyle\textit{1}}$ choose Matplotlib as the target library when
using ChatGPT.

Clearly, using AIs to support documentation is becoming increasingly common.
To wit, many documentation sites now including chatbots as a sibling feature
to traditional documentation. However, we note that this can in turn burden
maintainers with disentangling that confusion. For instance,
C${}_{\scriptstyle\textit{Fil}}$ observed that “people try GPT and then they
come to us with code that doesn’t work. But they don’t understand what it’s
trying to do. And we have no idea how they come to this code and what they are
trying to achieve”. Moreover, the long term benefit of these generative
structures rather than browsable structure is unclear, although we suggest in
the future they might be hybridized in useful ways.

Marketing |  Reuse |  Testing |  Takeaway
---|---|---|---
X | X |  |  Galleries can bear learning objectives, but not all are conceived as the main tool for teaching; setting expectations is critical
| X | X |  Community-contributed examples can capture new use cases, but they need to be maintained
X | X | X |  There is no silver bullet for user-creator communication, any seamless solution will inevitably be friction-ful for someone
X | X | X |  Effective allocation of developer time depends on project goals: sometimes improving SEO can be more important than new features
|  | X |  More examples can capture more use cases, but can add to upkeep
X | X |  |  Different priorities yield different shapes and sizes of examples, impeding some expected uses
X | X |  |  Users often have specific expectations about how to navigate the gallery, which must be considered to ensure legibility
| X | X |  Generative AI can help fill documentation gaps, but can burden communication channels
Figure 4: Takeaways by theme: communication, maintenance, and content.

##  VII Discussion

In this work we analyzed the user and creator perspectives on visualization
galleries. We find that galleries serve as marketing materials, reusable
templates, and testing infrastructure. Navigating these different usages forms
inherently leads to trade-offs and compromises. For instance, the
organization, structure, and size are shaped by the goals of the creators and
the community that surrounding the library. We summarize these and other
findings in Fig. 4.

Whose gallery is it anyway? Creator and user needs from galleries evolve over
time, but in different and sometimes conflicting ways. For the creators,
galleries start as materialization of design space and means to test
implementations. Marketing and communication needs subsequently arise to
attract and keep users, especially when the software transitions from active
feature development into maintenance phases. In contrast, galleries give users
a first impression and serve as the entry point to learning the tool. As they
get more advanced, galleries become cheat sheets for quick inspiration and
reuse. Expert users revisit them less frequently because they know the
examples by memory or need customized cases from other sources. Creators
anticipate diminished utility for each new example added to larger collections
as they foresee heavier burden for maintenance, the need to redesign, and
users’ cognitive load. Unaware of this backstage work, users desire ever
larger galleries, being overly optimistic of effectively locating examples in
need. Reconciling the difference in needs and resources does not have a clear
solution, but seems most effectively aided by iterative communication between
both groups. Beyond this, scaling searching and browsing of examples keeps
being an essential interface design challenge.

Evaluating Galleries. Many interviewees stressed the difficulty in assessing
gallery quality, especially when isolated from the rest of the documentation.
For instance, C${}_{\scriptstyle\textit{Josh}}$ observed that “It’s hard to
say how much the current instance of the gallery makes a difference versus
just a small number of examples that are highlighting things. It’s very hard
to tease out all the different parts of the system.”
C${}_{\scriptstyle\textit{Nicolas}}$ and C${}_{\scriptstyle\textit{Fil}}$
noted that they keep track of statistics such as webpage traffic in order to
analyze what examples are well-received. Creators often monitor services like
StackOverflow (C${}_{\scriptstyle\textit{Nicolas}}$,
C${}_{\scriptstyle\textit{Carlos}}$, and C${}_{\scriptstyle\textit{Fil}}$),
Discord (C${}_{\scriptstyle\textit{Matt}}$, C${}_{\scriptstyle\textit{Josh}}$,
C${}_{\scriptstyle\textit{Dominik}}$, and C${}_{\scriptstyle\textit{Fil}}$),
and GitHub (all). Creators empirically construct and evaluate the galleries
without standard validation mechanisms, which may limit a gallery’s potential.
Finding means and metrics to automatically evaluate galleries is mostly
unexplored. Kruchten et al. [28] explore metrics for a particular form of
gallery, but their connection to utility is unknown. We highlight this as a
useful area for research. For example, measuring the cognitive distances
between examples might help with better gallery organization and navigation.
Similarly, developing a readability score (à la TuTVis [50]) for galleries
might highlight places for improvement. A related coverage metric might
analyze not just the fraction of the library’s API represented, but how much
of the essential ideas for its operation are covered. Realization of such
metric might guide both better practice by creators but also automated gallery
organization and management.

Docs as database. As we observed galleries can be a variety of different
things. One persistent view that we observed across creators, users, and
galleries themselves is that they are sometimes viewed as databases. For
instance, prior work explored gallery-like exploration of visualization
collections drawn from databases [43, 53, 63]. Extending this metaphor is
enticing: it suggests that we can conceptualize ensembles of examples in
different collections—as in showcases, indexes, and so on—as views over an
underlying data structure. Tool support under this metaphor might empower
galleries across our themes without negatively affecting the others. This is a
potentially useful venue for AI-assistance. For instance, an example might be
defined not by its code, but instead by its task, allowing a generative system
to only materialize at the last minute, allowing it to be more contextual and
specific to a user’s interests. However, this may clash with the curation of
bespoke experiences that are sometimes intended for galleries—a collection of
representative examples of sustainable size. Future research could explore
malleable galleries that unify these perspectives by ranging between select
all-style queries to more bespoke personalized presentations that adapt to
user needs and experience, and, ideally, strike a balance between facilitating
maintenance and learning.

Limitations. Like any work, ours has its share of limitations. We interviewed
a relatively limited set of creators and users. A survey would have had
further reach, but we instead elected for a smaller number of higher quality
responses. Such a survey is useful future work. Our user interviews are
influenced by our selection process and may not capture the usage patterns of
those who do not identify as users of the selected tools. Our interviews were
centered on high level practices, which may miss low-level usage details.
Future work might follow Nam et al.’s [38] logging study to understand how
galleries are used in day-to-day practice rather than in recollection. The
libraries studied in this work have a strong influence from Heer (who is also
a collaborator of ours) and some of our findings may be bound to the specific
(although prominent) perspectives espoused in the communities surrounding his
work. Galleries outside of visualization may operate differently (e.g. non-
visual libraries like for data processing), and so investigating them is
important future work.

## Acknowledgments

We thank our participants for sharing their time and insights with us. We also
thank our reviewers for their thoughtful commentary. This work was supported
by the NSF (award # 2141506) and the Moore Foundation.

## References

  * [1] D. Akbaba, D. Lange, M. Correll, A. Lex, and M. Meyer.  Troubling collaboration: Matters of care for visualization design study.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 1–15, 2023. doi: 10 . 1145/3544548 . 3581168
  * [2] H. K. Bako, X. Liu, L. Battle, and Z. Liu.  Understanding How Designers Find and Use Data Visualization Examples.  Transactions on Visualization and Computer Graphics, pp. 1–11, 2022. doi: 10 . 1109/TVCG . 2022 . 3209490
  * [3] H. K. Bako, A. Varma, A. Faboro, M. Haider, F. Nerrise, B. Kenah, J. P. Dickerson, and L. Battle.  User-driven support for visualization prototyping in d3.  In International Conference on Intelligent User Interfaces, p. 958–972. Association for Computing Machinery, New York, NY, USA, 2023. doi: 10 . 1145/3581641 . 3584041
  * [4] L. Battle, D. Feng, and K. Webber.  Exploring d3 implementation challenges on stack overflow.  In IEEE Visualization Conference (Short Papers), pp. 1–5, 2022. doi: 10 . 1109/VIS54862 . 2022 . 00009
  * [5] M. Bostock, V. Ogievetsky, and J. Heer.  D3 data-driven documents.  Transactions on Visualization and Computer Graphics, 17(12):2301–2309, 2011. doi: 10 . 1109/TVCG . 2011 . 185
  * [6] J. Brandt, M. Dontcheva, M. Weskamp, and S. R. Klemmer.  Example-centric programming: integrating web search into the development environment.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 513–522, 2010. doi: 10 . 1145/1753326 . 1753402
  * [7] J. Brandt, P. J. Guo, J. Lewenstein, M. Dontcheva, and S. R. Klemmer.  Two studies of opportunistic programming: interleaving web foraging, learning, and writing code.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 1589–1598, 2009. doi: 10 . 1145/1518701 . 1518944
  * [8] M. Conlen and J. Heer.  Idyll: A markup language for authoring and publishing interactive articles on the web.  In ACM Symposium on User Interface Software and Technology, pp. 977–989, 2018. doi: 10 . 1145/3242587 . 3242600
  * [9] M. Correll.  Ethical dimensions of visualization research.  In SIGCHI conference on human factors in computing systems, pp. 1–13, 2019. doi: 10 . 1145/3290605 . 3300418
  * [10] W. Crichton.  Documentation generation as information visualization.  PLATEAU Annual Workshop on the Intersection of HCI and PL, 2020. doi: abs/2011 . 05600
  * [11] W. Crichton.  A new medium for communicating research on programming languages.  In Workshop on Human Aspects of Types and Reasoning Assistants, 2021.
  * [12] E. Daka and G. Fraser.  A survey on unit testing practices and problems.  In International Symposium on Software Reliability Engineering, pp. 201–211, 2014. doi: 10 . 1109/ISSRE . 2014 . 11
  * [13] T. S. developers.  Sphinx.  https://www.sphinx-doc.org/, 2007.
  * [14] M. Fowler.  Domain-specific languages.  Pearson Education, 2010.
  * [15] R. S. Geiger, N. Varoquaux, C. Mazel-Cabasse, and C. Holdgraf.  The Types, Roles, and Practices of Documentation in Data Analytics Open Source Software Libraries: A Collaborative Ethnography of Documentation Work.  Computer Supported Cooperative Work, 27(3-6):767–802, 2018. doi: 10 . 1007/s10606-018-9333-1
  * [16] E. L. Glassman, T. Zhang, B. Hartmann, and M. Kim.  Visualizing API Usage Examples at Scale.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 1–12. ACM, 2018. doi: 10 . 1145/3173574 . 3174154
  * [17] A. Head, C. Appachu, M. A. Hearst, and B. Hartmann.  Tutorons: Generating context-relevant, on-demand explanations and demonstrations of online code.  In Symposium on Visual Languages and Human-Centric Computing, pp. 3–12. IEEE, 2015. doi: 10 . 1109/VLHCC . 2015 . 7356972
  * [18] A. Head, J. Jiang, J. Smith, M. A. Hearst, and B. Hartmann.  Composing Flexibly-Organized Step-by-Step Tutorials from Linked Source Code, Snippets, and Outputs.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 1–12. ACM, 2020. doi: 10 . 1145/3313831 . 3376798
  * [19] N. Hengesbach.  Undoing seamlessness: Exploring seams for critical visualization.  In CHI Conference on Human Factors in Computing Systems Extended Abstracts, pp. 1–7, 2022. doi: 10 . 1145/3491101 . 3519703
  * [20] J. Hoffswell, A. Satyanarayan, and J. Heer.  Visual debugging techniques for reactive data visualization.  In Computer Graphics Forum, vol. 35, pp. 271–280. Wiley, 2016. doi: 10 . 1111/cgf . 12903
  * [21] A. Horvath, M. X. Liu, R. Hendriksen, C. Shannon, E. Paterson, K. Jawad, A. Macvean, and B. A. Myers.  Understanding how programmers can use annotations on documentation.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 1–16, 2022. doi: 10 . 1145/3491102 . 3502095
  * [22] A. Horvath, A. Macvean, and B. A. Myers.  Support for long-form documentation authoring and maintenance.  In Symposium on Visual Languages and Human-Centric Computing, pp. 109–114. IEEE, 2023. doi: 10 . 1109/VL-HCC57772 . 2023 . 00020
  * [23] K. Hu, S. Gaikwad, M. Hulsebos, M. A. Bakker, E. Zgraggen, C. Hidalgo, T. Kraska, G. Li, A. Satyanarayan, and Ç. Demiralp.  Viznet: Towards a large-scale visualization learning and benchmarking repository.  In SIGCHI conference on human factors in computing systems, pp. 1–12, 2019. doi: 10 . 1145/3290605 . 3300892
  * [24] M. Ichinco and C. Kelleher.  Exploring novice programmer example use.  In Symposium on Visual Languages and Human-Centric Computing, pp. 63–71. IEEE, 2015. doi: 10 . 1109/VLHCC . 2015 . 7357199
  * [25] J. Jakubovic, J. Edwards, and T. Petricek.  Technical Dimensions of Programming Systems.  Art, Science, and Engineering of Programming, 7(3), 2023. doi: 10 . 22152/programming-journal . org/2023/7/13
  * [26] W. Jernigan, A. Horvath, M. Lee, M. Burnett, T. Cuilty, S. Kuttal, A. Peters, I. Kwan, F. Bahmani, and A. Ko.  A principled evaluation for a principled idea garden.  In Symposium on Visual Languages and Human-Centric Computing, pp. 235–243. IEEE, 2015. doi: 10 . 1109/VLHCC . 2015 . 7357222
  * [27] H. B. Kang, G. Amoako, N. Sengupta, and S. P. Dow.  Paragon: An Online Gallery for Enhancing Design Feedback with Visual Examples.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 1–13. ACM, 2018. doi: 10 . 1145/3173574 . 3174180
  * [28] N. Kruchten, A. M. McNutt, and M. J. McGuffin.  Metrics-Based Evaluation and Comparison of Visualization Notations.  IEEE Transactions on Visualization and Computer Graphics, 2023. doi: 10 . 1109/TVCG . 2023 . 3326907
  * [29] G. Le Moulec, A. Blouin, V. Gouranton, and B. Arnaldi.  Automatic production of end user documentation for DSLs.  Computer Languages, Systems & Structures, 54:337–357, 2018. doi: 10 . 1016/j . cl . 2018 . 07 . 006
  * [30] B. Lee, S. Srivastava, R. Kumar, R. Brafman, and S. R. Klemmer.  Designing with interactive example galleries.  In SIGCHI Conference on Human Factors in Computing Systems, pp. 2257–2266. ACM, 2010. doi: 10 . 1145/1753326 . 1753667
  * [31] E. Lee-Robbins and E. Adar.  Affective learning objectives for communicative visualizations.  IEEE Transactions on Visualization and Computer Graphics, 29(1):1–11, 2022. doi: 10 . 1109/TVCG . 2022 . 3209500
  * [32] A. Lucero.  Using affinity diagrams to evaluate interactive prototypes.  In Human-Computer Interaction - INTERACT, vol. 9297 of Lecture Notes in Computer Science, pp. 231–248. Springer, 2015. doi: 10 . 1007/978-3-319-22668-2_19
  * [33] D. McCandless.  Information is beautiful.  Accessed March 25, 2024.
  * [34] A. M. McNutt and R. Chugh.  Integrated visualization editing via parameterized declarative templates.  In SIGCHI Conference on Human Factors in Computing Systems. ACM, NYC, 2021. doi: 10 . 1145/3411764 . 3445356
  * [35] A. M. McNutt, A. Outkine, and R. Chugh.  A Study of Editor Features in a Creative Coding Classroom.  In SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, NYC, 2023. doi: 10 . 1145/3544548 . 3580683
  * [36] S. Mehrpour, T. D. LaToza, and R. K. Kindi.  Active Documentation: Helping Developers Follow Design Decisions.  In Symposium on Visual Languages and Human-Centric Computing, pp. 87–96. IEEE, 2019. doi: 10 . 1109/VLHCC . 2019 . 8818816
  * [37] M. Mooty, A. Faulring, J. Stylos, and B. A. Myers.  Calcite: Completing code completion for constructors using crowds.  In 2010 IEEE Symposium on Visual Languages and Human-Centric Computing, pp. 15–22. IEEE, 2010. doi: 10 . 1109/VLHCC . 2010 . 12
  * [38] D. Nam, A. Macvean, B. Myers, and B. Vasilescu.  Understanding documentation use through log analysis an exploratory case study of four cloud services.  SIGCHI Conference on Human Factors in Computing Systems, 2024.  To Appear. doi: 10 . 48550/arXiv . 2310 . 10817
  * [39] A. Narechania, A. Srinivasan, and J. Stasko.  Nl4dv: A toolkit for generating analytic specifications for data visualization from natural language queries.  Transactions on Visualization and Computer Graphics, 27(2):369–379, 2020. doi: 10 . 1109/TVCG . 2020 . 3030378
  * [40] S. M. Nasehi and F. Maurer.  Unit tests as api usage examples.  In 2013 IEEE International Conference on Software Maintenance, pp. 1–10. IEEE Computer Society, Los Alamitos, CA, USA, sep 2010. doi: 10 . 1109/ICSM . 2010 . 5609553
  * [41] S. M. Nasehi, J. Sillito, F. Maurer, and C. Burns.  What makes a good code example?: A study of programming Q&A in StackOverflow.  In International Conference on Software Maintenance, pp. 25–34. IEEE, 2012. doi: 10 . 1109/ICSM . 2012 . 6405249
  * [42] OpenJS Foundation.  deck.gl.  https://deck.gl/.  Accessed March 25, 2024.
  * [43] M. Oppermann and T. Munzner.  VizSnippets: Compressing Visualization Bundles Into Representative Previews for Browsing Visualization Collections.  Transactions on Visualization and Computer Graphics, 28(1):747–757, 2022. doi: 10 . 1109/TVCG . 2021 . 3114841
  * [44] p5.js.  examples.  https://p5js.org/examples/.  Accessed March 25, 2024.
  * [45] Plotly.  Low-code python data app.  https://plotly.com/.  Accessed March 25, 2024.
  * [46] Posit.  Quarto.  https://quarto.org/.  Accessed March 25, 2024.
  * [47] H. Potter, A. Madadi, R. Just, and C. Omar.  Contextualized programming language documentation.  In ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, pp. 1–15, 2022. doi: 10 . 1145/3563835 . 3567654
  * [48] D. Procida.  Diátaxis: A systematic approach to technical documentation authoring.  https://diataxis.fr/#.  Accessed March 29, 2024.
  * [49] C. Reas and B. Fry.  Processing: a programming handbook for visual designers and artists, vol. 6812.  Mit Press, 2007. doi: oclc/73993935
  * [50] S. A. Sabab, A. Khan, P. K. Chilana, J. McGrenere, and A. Bunt.  An automated approach to assessing an application tutorial’s difficulty.  In 2020 IEEE Symposium on Visual Languages and Human-Centric Computing, pp. 1–10. IEEE, 2020. doi: 10 . 1109/VL/HCC50065 . 2020 . 9127271
  * [51] A. Satyanarayan, D. Moritz, K. Wongsuphasawat, and J. Heer.  Vega-lite: A grammar of interactive graphics.  Transactions on Visualization and Computer Graphics, 23(1):341–350, 2016. doi: 10 . 1109/TVCG . 2016 . 2599030
  * [52] A. Satyanarayan, R. Russell, J. Hoffswell, and J. Heer.  Reactive vega: A streaming dataflow architecture for declarative interactive visualization.  Transactions on Visualization and Computer Graphics, 22(1):659–668, 2015. doi: 10 . 1109/TVCG . 2015 . 2467091
  * [53] V. Setlur, A. Kanyuka, and A. Srinivasan.  Olio: A Semantic Search Interface for Data Repositories.  In ACM Symposium on User Interface Software and Technology, pp. 1–16. ACM, 2023. doi: 10 . 1145/3586183 . 3606806
  * [54] B. Subbaraman, S. Shim, and N. Peek.  Forking a sketch: How the openprocessing community uses remixing to collect, annotate, tune, and extend creative code.  In ACM Designing Interactive Systems Conference, pp. 326–342, 2023. doi: 10 . 1145/3563657 . 3595969
  * [55] M. H. Tanzil, G. Uddin, and A. Barcomb.  ” how do people decide?”: A model for software library selection.  In IEEE/ACM International Conference on Cooperative and Human Aspects of Software Engineering, 2024. doi: 10 . 48550/arXiv . 2403 . 16245
  * [56] K. Thayer, S. E. Chasins, and A. J. Ko.  A Theory of Robust API Knowledge.  ACM Transactions on Computing Education, 21(1):1–32, 2021. doi: 10 . 1145/3444945
  * [57] Vega.  Documentation.  https://vega.github.io/vega/docs.  Accessed March 29, 2024.
  * [58] Vega.  Documentation.  https://observablehq.com/@vega/how-vega-works.  Accessed March 29, 2024.
  * [59] Vega.  Editor.  https://vega.github.io/editor/.  Accessed March 29, 2024.
  * [60] Vega.  Let’s make a bar chart tutorial.  https://vega.github.io/vega/tutorials/bar-chart/.  Accessed March 29, 2024.
  * [61] Vizzu Inc.  ipyvizzu.  https://ipyvizzu.vizzuhq.com/latest/.  Accessed March 25, 2024.
  * [62] W. Wang, A. Kwatra, J. Skripchuk, N. Gomes, A. Milliken, C. Martens, T. Barnes, and T. Price.  Novices’ Learning Barriers When Using Code Examples in Open-Ended Programming.  ACM Conference on Innovation and Technology in Computer Science Education, pp. 394–400, 2021.  ACM Conference on Innovation and Technology in Computer Science Education ISBN: 9781450382144 Place: Virtual Event Germany Publisher: ACM. doi: 10 . 1145/3430665 . 3456370
  * [63] X. T. Xu, R. Xiong, B. Wang, D. Min, and S. P. Dow.  IdeateRelate: An Examples Gallery That Helps Creators Explore Ideas in Relation to Their Own.  Proceedings of the ACM on Human-Computer Interaction, 5:1–18, 2021. doi: 10 . 1145/3479496
  * [64] J. Yang, A. Bäuerle, D. Moritz, and Ç. Demiralp.  Vegaprof: Profiling vega visualizations.  In ACM Symposium on User Interface Software and Technology, pp. 1–11, 2023. doi: 10 . 1145/3586183 . 3606790
  * [65] J. Yang, P. F. Gyarmati, Z. Zeng, and D. Moritz.  Draco 2: An extensible platform to model visualization design.  In IEEE Visualization and Visual Analytics, pp. 166–170, 2023. doi: 10 . 1109/VIS54172 . 2023 . 00042
  * [66] K. Ye, W. Ni, M. Krieger, D. Ma’ayan, J. Wise, J. Aldrich, J. Sunshine, and K. Crane.  Penrose: from mathematical notation to beautiful diagrams.  ACM Transactions on Graphics, 39(4):144–1, 2020. doi: 10 . 1145/3386569 . 3392375

##  Appendix A Interview Guides

In this appendix we reproduced the interview guides used in our semi-
structured interviews. Follow up questions are collapsed into single bullet
items to simplify the presentation.

###  A-A Creator Backgrounds

Here we review in greater detail the relationships that the authors of this
work have with our participants as a means to better concretize our
positionality. The three creators, Jeffrey Heer, Dominik Moritz, and Matt
Conlen are, or have been, affiliated with the same research lab (UW IDL) as
all three of the authors of this work. Carlos Scheidegger, Jeffrey Heer, and
Dominik Moritz are or have been current collaborators of Leilani’s. Andrew co-
authored a paper with Nicolas Kruchten, worked with Xiaoji Chen and Ib Green,
and was advised by Jeffrey Heer. Junran has co-authored papers with Dominik
Moritz and Jeffrey Heer. This work focuses soliciting the experiences of a
small subject population who might potentially share common preferences and
habits for gallery designing from previous collaboration experience; future
research engaging with tool builders more exclusively is likely to discover
additional insights in gallery creation. This work also focused on users of
those galleries, but there was no overlap between the research team and that
group.

###  A-B Gallery Creator Interview Guide

Study introduction. Hi, _name_! Thank you for participating in today’s
interview and volunteering your time to help us understand example gallery
design.

_Interviewers introduce themselves (name, position)._

To quickly re-summarize our work, we are investigating why and how DSL
developers create example galleries as a part of their documentation. In this
interview, you will share the design decisions and the process for creating
the galleries. Feel free to share your screen at any time to answer our
questions.

If you don’t have any questions and you consent to participating I will begin
to record this meeting, is that okay with you? If yes, start the recording.

Opening questions - background and introduction.

  * •

I’m familiar with [the DSL], but can you tell me about how you view it and
your role in developing it? What makes it different from existing tools

  * •

Who are the target groups and how do you expect them to start using it?

  * •

Like domain expert or practiced programmer, one-time user or frequent user,
for job or for fun? Have you noticed any unexpected user groups?

  * •

In your library and documentation, what things are considered as galleries?
What is the purpose of each of them? Like advertisement, learning resources,
direct reuse, test, standard; for users or for your own

Documentation and gallery.

  * •

Besides galleries, what resources do you include? For example, tutorial, blog
post, detailed API docs, and code examples What’s the most important format of
resources? How are these different pieces connected? How should a user work
with these resources to use your library efficiently?

  * •

How is a gallery different from all of the other documentation formats? Can
you give some examples for how they complement the others?

  * •

Are there tools you use to create and manage the documentation? Could you
describe your experience using these tools? Which part of the documentation do
you use them for? Are you satisfied with these tools and what features do you
think would be great to have in an ideal tool?

Gallery design and maintenance.

  * •

If you were going to show only a few examples (from your DSL gallery) to
someone, which example would it be? Which are the top or bottom well-designed
(pick one)? What about them do you like or hate? For first-time users, which
would you suggest? How do you think they would benefit? Would you suggest any
examples to programming novices but domain users? How do you think they would
benefit from them?

  * •

Tell us about the process of designing a new example? Do you make them
manually or repurpose existing ones? Are there galleries that inspired you and
can you explain the connection with your gallery examples?

  * •

From where did you learn to do gallery design? Are there papers, blog posts or
books you used? Galleries other than the previous ones we talked about? Do you
find them useful? Do you disagree with any of the information? Examples? What
would you suggest to someone with less experience?

  * •

What is the process of creating the gallery? Can you describe the timeline?
When do you start to work on documentation and galleries? When do you consider
the gallery mostly done?

  * •

What are some key decisions you have to make for design and modification? What
do you need to help make these decisions?

  * •

How do you figure out how many examples to include? What’s the largest gallery
you’ve created? And why that many? Would you include as many examples as
possible? Are there any organizational challenges?

  * •

If you were to write down your own “lessons learned” or “rules of thumb” or
“best practices” about gallery design, what would it include? E.g. Maximize
the coverage of features; From simple to complex examples; categorize. How did
you learn these?

  * •

Is there any connection between the gallery and your test suite?

  * •

How do you maintain the examples? When do you add, update or delete examples?
How do you decide whether or not to include a certain example? Can you give
some examples? When do you need to sort or categorize the examples? How do you
order the categories? Do you put the same examples under multiple categories?
If not, how do you choose which category to put when there are multiple
options?

  * •

Are there additional features you would provide to support searching and
browsing the examples? E.g. meaningful names, keywords, keyword search,
categories, links to documentations, link to related examples

  * •

given infinite resources (time/money) what would you want your gallery to look
like?

  * •

What do you care about when you evaluate the gallery? Like usability,
learnability, quality. Any metrics? Does this differ based on the library
itself?

  * •

Were there any challenges you had creating or maintaining the galleries?

  * •

Any hidden assumptions, usage patterns that are difficult to perceive by the
gallery reader, compared to the visual things? How do you incorporate them in
the docs

User feedbacks.

  * •

Do your users have a communication channel back to you or the other
developers? for instance, GitHub issue, a survey you all did, etc. What do
they request about docs and galleries? Has this been helpful? Do you modify
anything based on their feedback? Examples? Do you think they understand why
you do things in the way you do? Have you seen any complaints about the
gallery examples? Have you seen examples being misused?

  * •

Are there any other kinds of data that you collect from users? Like user
surveys?

  * •

In general, how much do you think the gallery affects the adoption of your
library?

You can answer these question either now or in the exit survey:

  * •

Any other thoughts on galleries that we haven’t covered?

  * •

Would you recommend anyone as our interviewee?

Thank you very much! These have been very useful!

###  A-C Gallery User Interview Guide

Intro as we talked about during the meeting “Today we’re interested in talking
about the galleries for vl/altair/plotly. Which of these are you most familiar
with and how long have you been using it

The goals of the interviews are to understand how people use example galleries
(to learn or to accomplish their tasks efficiently), and to understand
opportunities for better galleries and tools to help people learn and create
visualizations.

  * •

What made you start to use [DSL]? For what tasks? Why did you choose it over
other alternatives? What are your strategies to start with a new DSL? Do you
read their documentation, search for videos, or blog posts? Which part of the
documentation do you start with? How do you navigate through the
documentation? Which part of the documentation do you find most useful as a
beginner, and as you get experienced? Which parts confuse you or make you
overwhelmed?

  * •

When do you find yourself looking into the gallery examples? What information
are you looking for? How to do a certain thing, inspirations, possibilities or
general language construct? How often? In which way do you find them useful?

  * •

What types of tasks do you do with a gallery? Which do you find more
difficult: figuring out what the design should look like or implementing it
with the DSL? Which step is the documentation or gallery more helpful with?

  * •

Can you describe the most recent or most common process for you to create a
visualization using the [DSL]? Do you start from scratch, template, examples
similar to your goal? Do you integrate different examples? How did it make you
more fluent with the library? What’s your leaning and usage trajectory? How
long does it take? What are some decisions you made? How did you decide?

  * •

Which code examples are more useful to you, the ones from tutorials and
library references, or the gallery examples? Or what are the different
situations when you search for these code examples?

  * •

What strategies do you use to find relevant examples? (For which tasks do you
use each of these strategies?) Do you search for external resources? Do you
browse the gallery examples? Do you search with keywords on search engines or
inside the doc site? How useful are the keywords? How did you decide what is
helpful?

  * •

How do you use these examples after you find them? Do you try to understand
every detail before making edits? Do you look into tutorials and references
specific to the example? Do you look for help from the library maintainers,
the communities and Stack Overflow? Do you find them helpful?

  * •

What aspects are the most important when you look at an example? Environment:
Being able to make changes and see the result right away (online editor)
Explanations and code comments Simple and clear, easy to read

  * •

What aspects are the most important when you look at the gallery? Categorized,
Sorted, aesthetically intriguing, Clear descriptions, Visuals that are easy to
perceive, Number of available examples

  * •

What are the challenges for you to find relevant examples in a gallery?

  * •

What are the challenges for you to understand or repurpose relevant examples
in a gallery?

  * •

Are you satisfied with the gallery of [DSL]? What’s hard or annoying about the
galleries or creating visualizations in general? In an ideal world, what
features or tools would you want to help you write code with [DSL]

  * •

Do you share examples you made with other people? For example, send pr to
contribute or share blocks style posts with colleagues etc.

  * •

The ability to leverage carefully selected examples or the ability to
recognize a good example from a collection?

###  A-D Post Interview Survey for the Creators

Study introduction. Thank you for sharing your knowledge and experience with
us in the interview session. We are still learning how DSL/library developers
design and manage example galleries. We would love for you to fill out this
5-10 minute survey. This survey contains 6 close-ended questions and 2 open-
ended questions.

  * •

What is your occupation?

  * •

Which gender identity do you most identify with?

  * •

How long have you been designing DSLs/API and libraries? Give your best
estimate.

  * •

How many DSLs/API and libraries have you designed? Give your best estimate.

  * •

How many galleries have you created for the DSLs/API and libraries you
designed? Give your best estimate.

  * •

Are we allowed to refer to you by name or not?

  * •

Do you have additional feedback for the interview?

  * •

Is there anyone you know would be a good fit for our interviewee?