# Report of the 1st Workshop on Generative AI and Law

A. Feder Cooper Equal contribution. Correspondence<EMAIL_ADDRESS>We
thank Matthew Jagielski, Andreas Terzis, and Jonathan Zittrain for feedback on
this report. We thank Google, Microsoft, Schmidt Futures, OpenAI, Anthropic,
Cornell Law School, and ML Collective for their generous sponsorship. GenLaw
Organizers  Cornell University  Google Research  Katherine Lee††footnotemark:
GenLaw Organizers  Cornell University  Google DeepMind  James
Grimmelmann††footnotemark:  GenLaw Organizers  Cornell Tech  Cornell Law
School  Daphne Ippolito††footnotemark:  Google DeepMind  Carnegie Mellon
University
Christopher Callison-Burch  University of Pennsylvania  Christopher A.
Choquette-Choo  Google DeepMind  Niloofar Mireshghallah  GenLaw Organizers
University of Washington  Miles Brundage  OpenAI  David Mimno  GenLaw
Organizers  Cornell University  Madiha Zahrah Choksi  GenLaw Organizers
Cornell University  Cornell Tech  Jack M. Balkin  Yale Law School  Nicholas
Carlini  Google DeepMind  Christopher De Sa  Cornell University  Jonathan
Frankle  DataBricks (MosaicML)  Deep Ganguli  GenLaw Organizers  Anthropic
Bryant Gipson  Google  Andres Guadamuz  University of Sussex  Swee Leng Harris
Luminate Group  Abigail Z. Jacobs  University of Michigan  Elizabeth Joh  U.C.
Davis, School of Law  Gautam Kamath  University of Waterloo  Mark Lemley
Stanford Law School  Cass Matthews  Microsoft  Christine McLeavey  OpenAI
Corynne McSherry  Electronic Frontier Foundation  Milad Nasr  Google Research
Paul Ohm  Georgetown Law School  Adam Roberts  Google DeepMind  Tom Rubin
OpenAI  Pamela Samuelson  U.C. Berkeley, School of Law  Ludwig Schubert
GenLaw Organizers  Kristen Vaccaro  U.C. San Diego  Luis Villa  Tidelift
Felix Wu  Cardozo Law  Elana Zeide  Nebraska College of Law

###### Abstract

This report presents the takeaways of the inaugural Workshop on Generative AI
and Law (GenLaw), held in July 2023. A cross-disciplinary group of
practitioners and scholars from computer science and law convened to discuss
the technical, doctrinal, and policy challenges presented by law for
Generative AI, and by Generative AI for law, with an emphasis on U.S. law in
particular. We begin the report with a high-level statement about why
Generative AI is both immensely significant and immensely challenging for law.
To meet these challenges, we conclude that there is an essential need for 1) a
shared knowledge base that provides a common conceptual language for experts
across disciplines; 2) clarification of the distinctive technical capabilities
of generative-AI systems, as compared and contrasted to other computer and AI
systems; 3) a logical taxonomy of the legal issues these systems raise; and,
4) a concrete research agenda to promote collaboration and knowledge-sharing
on emerging issues at the intersection of Generative AI and law. In this
report, we synthesize the key takeaways from the GenLaw workshop that begin to
address these needs. All of the listed authors contributed to the workshop
upon which this report is based, but they and their organizations do not
necessarily endorse all of the specific claims in this report.

This work is licensed under CC BY 4.0. To view a copy of this license, visit
http://creativecommons.org/licenses/by/4.0/.

## 1 Introduction

The inaugural Generative AI and Law (GenLaw) workshop took place on July 29th
and 30th, in Honolulu, Hawai’i, where it was co-located with the 40th
International Conference on Machine Learning (ICML). The workshop was
organized in response to the intense interest in (and scrutiny of) recent
public advancements in generative-AI technology. The primary goal was to bring
together experts in machine learning (ML) and law to discuss the legal
challenges that generative-AI technology raises. To promote concrete and
focused discussions, we chose to make intellectual property (IP) and privacy
the principal legal topics of the first workshop. Other significant topics
discussed included free speech, products liability, and transparency. For this
first workshop, most discussion was limited to considerations of U.S. law.

The workshop was convened over two days. The first day (July 29th) was a
public session held as part of ICML, consisting of keynote lectures, panel
discussions, lightning talks, and a poster session, all dealing with research
issues at the intersection of Generative AI and law. The second day (July
30th), was held off-site, at which approximately forty participants conducted
a series of roundtable discussions to dig deeper into significant issues
identified on the first day.

This report reflects the takeaways from the roundtable discussions. They are
organized into five broad headings, reflecting the participants’ consensus
about the most urgently needed contributions to the research area of
Generative AI and law:

  1. 1.

A high-level statement about why Generative AI is both immensely significant
and immensely challenging for law (Section 2 The Impact of Generative AI on
Law);

  2. 2.

The beginnings of a shared knowledge base that provides a common conceptual
language for experts across disciplines (Section 3 Developing a Shared
Knowledge Base);

  3. 3.

Clarification of the unique capabilities and issues of generative-AI systems,
setting them in relation to the broader landscape of artificial-intelligence
and machine-learning technologies (Section 4 Pinpointing Unique Aspects of
Generative AI);

  4. 4.

An initial taxonomy of the legal issues at play (Section 5 A Preliminary
Taxonomy of Legal Issues); and,

  5. 5.

A concrete research agenda to promote collaboration and progress on emerging
issues at the intersection of Generative AI and law (Section 6 Toward a Long-
Term Research Agenda).

To best serve these ends, this report does not delve into the technical
details of specific generative-AI systems, the legal details of complaints and
lawsuits involving those systems, or policy proposals for regulators. Our
intended audience is scholars and practitioners who are already interested in
engaging with issues at the intersection of Generative AI and law, for
example, ML researchers who have familiarity with some of the ongoing lawsuits
regarding Generative AI, and lawyers who have familiarity with terms like
“large language model.” We focus our attention on synthesizing reflections
from the workshop to highlight key issues that need to be addressed for
successful research progress in this emerging and fundamentally
interdisciplinary area.

## 2 The Impact of Generative AI on Law

Generative AI is “generative” because it generates text, images, audio, or
other types of output. But it is also “generative” in the sense of Jonathan
Zittrain’s theory of generative technologies: it has the “capacity to produce
unanticipated change through unfiltered contributions from broad and varied
audiences” [116]. As a result, generative-AI systems will be both immensely
societally significant — too significant for governments to ignore or to delay
dealing with — and present an immensely broad range of legal issues. To see
why, it is useful to consider Zittrain [116]’s five dimensions of
generativity:

Leverage

A technology provides leverage when it makes difficult tasks easier.
Generative AI is widely recognized for its use in creativity; programming;
retrieving and synthesizing complex bodies of knowledge; and automating
repetitive tasks.

Adaptability

A technology is adaptable when it can be applied to a wide range of uses.
Generative AI is celebrated for its adaptability. It has been applied to
programming, painting, language translation, drug discovery, fiction,
educational testing, graphic design, and much more.

Ease of mastery

A technology is easy to master when users without specialized training can
readily adopt and adapt it. While some generative-AI methodologies, such as
model pre-training, still require technical skills, the ability to use chat-
style, interactive, natural-language prompting to control generative-AI
systems greatly reduces the difficulty of adoption. Users without programming
or ML backgrounds have been able to use Generative AI for numerous tasks.

Accessibility

A technology is accessible when there are few barriers to its use. Cost is the
most obvious barrier, but other barriers can include regulation, secrecy, and
linguistic limits. The creation of cutting-edge generative-AI models from
scratch requires enormous inputs of data, compute, and human expertise —
currently limiting model creation to a handful of institutions — but services
allowing inference with these models are widely available to the public. These
services are inexpensive for users making small numbers of queries, and they
tend to operate close to real-time, making generative outputs (at least
appear) low-cost to produce, in terms of time.

Transferability

A technology is transferable when changes in it can easily be conveyed to
others. Once pre-trained or fine-tuned, generative-AI models can be easily
shared, prompts and prompting techniques are trivially easy to describe, and
systems built around generative-AI models can be made broadly available at
increasingly low effort and cost.

In short, Generative AI hits the generativity jackpot. It provides enormous
leverage across a wide range of tasks, is readily built on by a huge range of
users, and facilitates rapid iterative improvement as those users share their
innovations with each other.

Zittrain [116]’s two examples of supremely generative technologies from 2008
are computers and the Internet. Generative AI seems likely to be a third. No
other technology of the last two decades even comes close. Regulators and
legal scholars should expect that Generative AI will raise legal and policy
challenges that are comparable in scope, scale, and complexity to those raised
by computers and the Internet.

The Internet-law analogy also provides guidance on how technologists and
lawyers can approach this shared challenge. They must have a common vocabulary
so that their contributions are mutually intelligible (Section 3 Developing a
Shared Knowledge Base). Lawyers must have a sufficient foundation of technical
understanding to be able to apply their expertise in law accurately (Section 4
Pinpointing Unique Aspects of Generative AI). Technologists, for their part,
must have a sufficient foundation of legal knowledge to identify legally
significant technical interventions (Section 5 A Preliminary Taxonomy of Legal
Issues). And both groups need a common research agenda to collaborate and
iterate rapidly on effective projects that advance a shared understanding how
Generative AI and the legal system interact (Section 6 Toward a Long-Term
Research Agenda). The aim of this report is to lay down a starting framework
for these tasks.

## 3 Developing a Shared Knowledge Base

It became apparent over the course of the GenLaw roundtable discussions that
some commonly used terms have different meanings in machine learning and in
law. Sometimes, both groups have been working to develop deep understandings
of important but hard-to-capture concepts. The term privacy is a prominent
such example. Technologists’ formal definitions (such as differential privacy)
do not always encompass the wide range of interests protected by privacy law;
similarly, it can be hard to put the holistic definitions used by legal
scholars into computationally tractable forms that can be deployed in actual
systems.111The U.S. Census has sparked debate over its use of differential
privacy: a technique that provides strong theoretical guarantees of privacy
preservation. Critics question whether or not the definition of privacy
reflected in differential privacy accords with the census’s broader goals of
privacy preservation. Differential privacy is also sometimes used in
Generative AI, though it is context-dependent whether its definition of
“privacy” is meaningful for generative-AI applications [13]. These
communication barriers are real, but, as was clear during GenLaw, the two
communities have generally understood that they mean something different by
“privacy” and have read each others’ work with a working understanding of
these differences in mind.

We also observed, however, that there are also various types of
misunderstandings in terminology across disciplines. At GenLaw, some terms
were used in different ways because members of one community did not even
realize a loosely-defined term in their community was a term of art in the
other, or because the meaning they assumed a term had was subtly different
from how it was actually used in writing. These conflicting definitions and
translation gaps hampered our ability to collaborate on assessing emerging
issues. For example, technologists use the term pre-training to refer to an
early, general-purpose phase of the model training process, but legal scholars
assumed that the term referred to a data preparation stage prior to and
independent of training. Similarly, many technologists were not aware of the
importance of harms as a specific and consequential concept in law, rather
than a general, non-specific notion of unfavorable outcomes. We found our way
to common understandings only over the course of our conversations, and often
only after many false starts.

Thus, first and foremost, we need to have a shared understanding of baseline
concepts in both Generative AI and law. Even when it is not possible to pin
down terms with complete precision, it is important to have clarity about
which terms are ambiguous or overloaded. We believe that there three
significant ways that computer scientists and legal scholars can contribute to
creating this shared understanding:

  1. 1.

They can build glossaries of definitions of important terms in machine
learning and law, which can serve both as textbooks and as references (Section
3.1 Identifying and Defining Terms). Throughout this piece, glossary terms are
identified in green and serve as links to the corresponding glossary entry in
the Appendix.

  2. 2.

They can develop well-crafted metaphors to clarify complex concepts across
disciplinary boundaries. Even imperfect metaphors are useful, as they can
serve to highlight where concepts in Generative AI deviate from intuitions
that draw on more traditional examples (Section 3.2 Crafting Useful
Metaphors).

  3. 3.

They can keep current with the state of the art, and help others to do so.
This does not just mean the fundamentals of machine learning and Generative AI
(although these are certainly important). It also means being alert to the
plethora of ways that generative-AI systems are be deployed in practice, and
the commonalities and differences between these systems (Section 3.3
Understanding Evolving Business Models).

### 3.1 Identifying and Defining Terms

The GenLaw organizers and participants have collaborated to create an initial
glossary of important terms at the intersection of law and Generative AI. (It
follows as Appendix A Glossary.) The glossary has two primary goals.

First, it identifies terms of art with technical or multiple meanings. Both
law and machine learning commonly give specific, technical definitions to
words that also have general, colloquial meanings, such as “attention” or
“harm.”222See the entry for attention in the glossary for the machine-learning
definition. Sometimes, the redefinition runs the other way, when a technical
term has taken on a broader meaning in society at large. To technologists, an
algorithm is simply a precise rule for carrying out a procedure, but the term
has come to be popularly associated with specific technologies for ranking
social media posts based on expected interest. The words “goal” and
“objective” are sometimes used interchangeably in English, but objective is a
term of art in machine learning, describing a mathematical function used
during training. On the other hand, “goal” does not have an agree-upon
technical definition. In machine learning, it is typically used colloquially
to describe our overarching desire for model behaviors, which cannot be
written directly in math.

Second, the glossary provides succinct definitions of the most critical
concepts for a non-expert in either law or ML (or both). These definitions are
not intended to cover the full complexity of a concept or term from the expert
perspective. For example, one could write volumes on privacy — and many have,
arguably for thousands of years.333This is why we do not attempt a definition
of privacy in Appendix A Glossary. Instead, our purpose here is simply to show
technologists that there is more to privacy than removing personally
identifiable information (PII).

The glossary is offered as a starting point, not a finish line. The field is
in flux; its terminology will evolve as new technologies and controversies
emerge. We will host and update this glossary on the GenLaw website.444See
https://genlaw.github.io/glossary.html. We hope that these definitions will
serve as a baseline for more effective communication across disciplines about
emerging issues.

### 3.2 Crafting Useful Metaphors

Well-chosen metaphors can provide a useful mental model for thinking through
complex concepts. Metaphors are also widely used in both machine learning and
law. For machine-learning practitioners, these metaphors can also be sources
of inspiration; the idea of an “artificial neural network” was inspired by the
biology of neurons in the human brain [9]. Analogy and metaphor are central to
legal rhetoric [98]; they provide a rational framework for thinking through
the relevant similarities and differences between cases. Metaphors, however,
can also simplify and distort; nevertheless, understanding the ways that a
metaphor fails to correctly describe a concept can still be instructive in
helping to clarify one’s thinking.

At the GenLaw workshop, we discussed instructive metaphors for Generative AI
extensively. We give two examples from this discussion here (anthropomorphism
and memorization) and describe additional ones in Appendix B Metaphors.

Metaphorical anthropomorphism. Metaphorical anthropomorphism is the
personification of a non-human entity; it applies metaphors that compare the
entity’s traits to human characteristics, emotions, and behaviors. Machine-
learning practitioners commonly use terms that anthropomorphize machine-
learning models, for example, saying models “learn,” “respond,” “memorize,” or
“hallucinate.” Such metaphors can lead people to conclude that a machine-
learning system is completing such actions using the same mechanisms and
thought processes that a human would. However, while practitioners may
sometimes be inspired in their designs by biological phenomena (e.g., neural
networks contain “neurons” that “fire” analogously to those in the human
brain), they by and large do not mean that machine-learning models “learn” or
“memorize” in exactly the same way that humans complete these actions.
Instead, these should be considered terms of art — perhaps inspired by human
actions, but grounded in technical definitions that bear little resemblance to
human mechanisms.

Some within the GenLaw community have advocated for using different terms to
describe these processes — ones that do not elicit such strong comparisons to
human behavior [29, and citations therein]. However, until we have better
terms, understanding when a term is indeed a term of art, and the ways that it
is inspired by (but not equivalent to) colloquial understandings, will remain
a critical part of any interdisciplinary endeavour.

Memorization. The terms memorization and regurgitation are very common in the
machine-learning literature. Roughly speaking, memorization and regurgitation
can be treated interchangeably. They both signify when a machine-learning
model encodes details from its training data, such it is capable of generating
outputs that closely resemble its training data.555Some differentiate
“memorization” and “regurgitation,” with regurgitation referring to a model’s
ability to output its training data (via generation) and memorization
referring to a model containing a perfect copy of its training data
(regardless of whether or not it is regurgitated). In practice, the two terms
are commonly used interchangeably.  The machine-learning experts who coined
these terms created precise definitions that can be translated into quantified
metrics; these definitions refer to specific ways to measure the amount of
memorization (as it is technically definition) present in a model or its
outputs [18, 53, 2, 59].

Unfortunately, the connection to the colloquial meanings of these words can
cause confusion. Some outside of the machine-learning community misinterpet
“Generative AI memorization” to include functionality that goes beyond what
machine-learning practitioners are actually measuring with their precise
definitions. One such example is discussions around text-to-image generative
models “memorizing” an artist’s style [20]. While measuring stylistic
similarity is an active area of machine-learning research [19], it is _not_
equivalent to memorization in the technical sense of the word.

Other misunderstandings can arise due to the fact that “memorization” and
“regurgitation” are imperfect analogies for the underlying processes that
machine-learning scientists are measuring. People deliberately memorize; for
example, an actor will actively commit a script to memory. In contrast, models
do not deliberately memorize; the training examples that end up memorized by a
model were not treated any differently during training than the ones that were
not memorized.666Models are trained using an objective function that rewards
the model for producing data that looks similar to the training data during
the training phase. However, the goal of model training is not to reproduce
the training data, so other techniques, such as regularization or alignment
are used to help a model generalize. See Section 3.1 Identifying and Defining
Terms for a discussion on “goal” versus objective in machine learning.
Importantly, humans distinguish between memorizing (which has intentionality)
and “remembering” (when a detail is recalled without the intent to memorize
it). Generative-AI models have no such distinction. For humans, it is how we
personally feel about a thought, action, or vocalization that leads us to call
it “memorized.” But for models, which lack intent or feeling, memorization is
merely a property assigned to their outputs and weights through technical
definitions.

### 3.3 Understanding Evolving Business Models

Developing glossaries and metaphors can help with staying current about
generative-AI technology, but they do not necessarily capture all of the
different ways that generative-AI functionality can be put to use in practice.
To understand real-world uses, it is also important to have a working
understanding of different generative-AI production processes.

There are a variety of evolving business models that have yet to solidify into
definitive patterns. In this uncertain environment, myths and folk wisdom can
proliferate. Real, current information about business models can be very
useful for understanding who is involved in the production, maintenance, and
use of different parts of generative-AI systems. Business models prove useful
for appreciating the quantity and diversity of actors (not just technologies)
that enable generative-AI functionality.

We highlight four patterns from discussion at GenLaw:

  1. 1.

Business-to-consumer (B2C) hosted services: Several companies have released
direct-to-consumer applications and application programming interfaces (APIs)
for producing generations. For example, on the large end of the spectrum,
there are chatbots powered by main players in language modeling (e.g.,
OpenAI’s ChatGPT, Anthropic’s Claude, Google’s Bard, etc.). There are also
smaller companies that have released similar tools (e.g., Midjourney’s [71] or
Ideogram’s [51] text-to-image generation applications). These companies
provide a mix of entry points to their systems and models, including user
interfaces and APIs, often offered via subscription-based services. Typically,
the systems and models developed by these companies are hosted in proprietary
services; users can access these services to produce generations but, with
some notable exceptions (e.g., fine-tuning APIs), cannot directly alter or
interact with the models embedded within them.

  2. 2.

Business-to-business (B2B) integration with hosted services: Other business
models allow for businesses to integrate generative-AI functionality into
their products either via direct partnership/integration or through the use of
APIs. For example, ChatGPT functionality is integrated into Microsoft Bing
search (via a close partnership between Microsoft and OpenAI). Poe is
developed based on a partnership between Anthropic and Quora [3].777We cannot
tell from press releases whether Poe also uses the API or if their partnership
is of a different nature. Often, we will not be able to tell the nature of the
business relationship (unless disclosed publicly) between corporate partners.
In other cases, companies develop generative-AI products by becoming corporate
customers of generative-AI-company APIs, as opposed to bespoke business
partners. These types of business relationships can either lead to developing
new features for existing products, or new products altogether.

  3. 3.

Products derived from open models and datasets: The business models discussed
above depend on proprietary systems, models, and/or datasets. Other options
include (or directly rely on) open-source software, models, and datasets,
which can be downloaded and put to use (e.g., training new models or fine-
tuning existing model checkpoints). Some companies operate distinctly (or
partially) with open-source product offerings, such as some versions of Stable
Diffusion [86] offered by Stability AI [102]. Others operate in a mixed
fashion, for example, individuals can openly download Meta’s Llama-model
family (the different models’ weights), but the details of the training data
are closed [105].

  4. 4.

Companies that operate at specific points in the generative-AI supply chain:
Any link (or subset of links) in the supply chain [62] could potentially
become a site of specific business engagement with generative-AI technology.
For those interested in issues at the intersection of Generative AI and law,
it will be important not only to be familiar with the roles and scope of
companies in these areas, but also how they interact and inter-operate. We
provide three examples below of emerging sites of business engagement.

     * •

_Datasets_ : There may be companies that engage only with the dataset
collection and curation aspects of Generative AI (similar to how data brokers
function in other industries). Scale AI [92] is one such company that works on
data example annotation for generative-AI training datasets.

     * •

_Training diagnostics_ : There are some companies that handle aspects of data
analysis and diagnostics for generative-AI-model training dynamics, like
Weights & Biases [110].

     * •

_Training and deployment_ : While it is generally tremendously costly to train
and deploy large generative-AI models, advancements in open-source technology
and at smaller companies (in both software and hardware) have helped make
training custom models more efficient and affordable. There are now several
companies that develop solutions for bespoke model training and serving, such
as MosaicML (acquired by DataBricks) [72] and Together AI [104].

We defer additional discussion of open\- vs. closed-source software to
Appendix A Glossary. The important point that we want to highlight here is
that there are many ways that generative-AI models may be integrated into
software systems, and that there are many different types of business models
associated with the training and use of these models. This landscape is likely
to continue to evolve as new business players enter the field.

## 4 Pinpointing Unique Aspects of Generative AI

During the roundtable discussions, the legal scholars and practitioners had a
recurring question for the machine-learning experts in the room: What’s so
special about Generative AI? Clearly, the outputs created by Generative AI
today are better than anything we have seen before, but what is the “magic”
that makes this the case? It became clear that answering this question, even
just in broad strokes, could be useful for providing more precise analysis of
the legal issues at play (Section 5 A Preliminary Taxonomy of Legal Issues).
In this section, we summarize three aspects of Generative AI for which it can
be productive to consider recent developments in AI as meaningfully novel or
different in comparison to past technology. These include (1) the transition
from training models to perform narrowly-defined tasks to training them for
open-ended ones (Section 4.1 The Transition to Very Flexible Generative
Models), (2) the role of the modern training data (Section 4.2 Developments in
the Training Pipeline: Pre-Training and Fine-Tuning) and generative-AI
pipelines (Section 4.3 Generative-AI Systems and the Supply Chain) and (3) how
the scaling up of pre-existing techniques has enabled the quality and variety
we see in generations today (Section 4.4 The Massive Scale of Generative-AI
Models)

### 4.1 The Transition to Very Flexible Generative Models

In the past, machine-learning models tended to be trained to perform narrowly
defined discriminative tasks, such as labeling an image according to the class
of object it depicts [33, 32] or classifying the sentiment of a sentence as
positive or negative [57]. Modern generative-AI models change this paradigm in
two ways.888For a longer summary of this transition from task-specific,
discriminative models to generative models in Generative AI, see Parts I.A and
I.B of Lee et al. [62].

First, there has been a shift from discriminative models, which have simple
outputs like a class label (e.g., dog or cat for an image classifier), to
generative models, which output complex content, such as entire images or
paragraphs of text (e.g., given the input of cat, outputting a novel image of
a cat, sampling from the near-infinite space of reasonable cat images it could
create) [62, Part I.A].999As we discuss at the top of Section 5 A Preliminary
Taxonomy of Legal Issues, today’s generative-AI models are used to solve both
discriminative tasks (e.g., sentiment classification) and generative tasks
that result in expressive content (e.g., producing paragraphs of text).

Second, there has been a shift toward using single, general-purpose models to
solve many different tasks, rather than employing a model customized to each
task we would like to perform. Even a few years ago, it was common to take a
base model and fine-tune it once per each task domain. This would, for
example, result in one model that specializes in sentiment classification,
another which specializes in automatic summarization, another in part-of-
speech tagging, and so on. Many state-of-the-art systems today handle a wide
variety of tasks using a single model.101010It is rumored that the models
underlying ChatGPT are actually an _ensemble_ of on the order of $10$ expert
models, in which different types of requests get routed to specific experts.
Nevertheless, if true, these experts are still more flexible than task-
specific models from the past.

These models are able to do all sorts of things [39]. In Section 2 The Impact
of Generative AI on Law, we discuss how Generative AI is a generative
technology, in the sense of Jonathan Zittrain’s theory of generativity [116].
The scaling up of Generative AI (Section 4.4 The Massive Scale of Generative-
AI Models) has facilitated generativity across a wide range of applications
and modalities, not just in the text-to-text and text-to-image applications
that are most commonly reported on in the news. As a non-exhaustive list, this
scaling has enabled huge breakthroughs in image captioning [68], music
generation [1], speech generation [60] and transcription [83], tools for
lowering the barrier to learning to program [113], and research questions in
the physical sciences (including on protein folding, drug design, and
materials science) [31].

### 4.2 Developments in the Training Pipeline: Pre-Training and Fine-Tuning

Machine-learning models are trained on a training dataset of examples of the
task that the model is supposed to be able to accomplish. The nature of these
training datasets has changed drastically over the years, leading to the
capabilities seen in generative-AI systems today [63]. In particular, we have
seen a shift toward multi-stage training pipelines, in which models are first
trained on large (but possibly lower-quality) datasets to create a base model
and then progressively trained on smaller, more-curated datasets that better
align with the model creators’ goals.

Typically, a base model is constructed by training on an enormous, often web-
scraped dataset, which instills a “base” of knowledge about the world within
the model. This step is called pre-training because it is the training that
occurs before the final training of the model. As described by Callison-Burch
[15] in his testimony to the U.S. House of Representatives Judiciary
Committee, during pre-training, models learn underlying patterns from their
input data. When pre-training on large-scale data that has a wide variety of
information content, base models capture abundant, “general-knowledge”
information. For example, large language models (LLMs) learn syntax and
semantics, facts (and fictions) about the world, and opinions, which can be
used to produce summaries and perform limited reasoning tasks; image-
generation models learn to produce different shapes and objects, which can be
composed together in coherent scenes.111111Though, notably, not a collage! See
Appendix B Metaphors for a discussion of why the metaphor (Section 3.2
Crafting Useful Metaphors) of a collage for generative-AI outputs can be
misleading. Pre-training gives these models the unprecedented flexibility to
generate all sorts of outputs through synthesizing information in the input
training data.

This flexibility allows base models to be re-used in a variety of ways. For
example, to adapt it to more specific tasks and domains, one can further train
(i.e., fine-tune) the base model on domain-specific data (e.g., legal texts
and case documents) to specialize the model’s behavior (e.g., performing
better at legal document summarization). Alternatively, one might fine-tune
the base model to understand a dialog-like format (as ChatGPT has done [74]).
Pre-training is very expensive, which means it only happens once (or a small
handful of times),121212It can cost millions of dollars to train a large
language model [8, 5, 62]. but subsequent fine-tuning tends to be much faster
(due to the smaller size of the datasets involved), so it can more tractably
occur many times.

Choosing what is pre-training and what is fine-tuning. Despite our discussion
above of what is unique about pre-training and fine-tuning, it is worth
emphasizing that this division is not well-defined. It is predominantly an
artifact of choices made regarding training, rather than an essential aspect
of the training process. Both pre-training and fine-tuning are just training
(though perhaps configured differently). The reasons we differentiate between
these two stages have to do with how large-scale model training is done in
practice; the distinction is only meaningful because researchers frequently
choose to divide stages of training along these lines (in turn, ascribing
meaning to this division). For example, one actor in the supply chain may
release a pre-trained model, a different actor may fine-tune that model and
release it as well, and a third actor may fine-tune the already fine-tuned
model [62].131313Is the third model fine-tuned from a pre-trained model, or a
fine-tuned model? This is all just semantics. Additionally, researchers frame
concrete research questions specifically for pre-training or fine-tuning [69,
e.g.].

Note. During the GenLaw roundtable discussions, it became apparent that legal
experts shared some misconceptions about the roles of pre-training and fine-
tuning, and that it is therefore important for machine-learning researchers to
emphasize the influence of pre-training on generative-AI model capabilities.
Additionally, we re-emphasize our note from the top of Section 3 Developing a
Shared Knowledge Base that pre-training is _training_ , and not a data
preparation stage.

### 4.3 Generative-AI Systems and the Supply Chain

There are numerous decisions and intervention points throughout the system,
which extend to elements beyond choices in pre-training and fine-tuning. Since
many actors can be involved in the generative-AI supply chain, and decisions
made in one part of the supply chain can impact other parts of the supply
chain, it can be useful to identify each intervention and decision point and
think about them in concert. We defer to Lee et al. [62, Part I.C] for
detailed discussion of the supply chain and the numerous stages, actors, and
design choices that it involves.

These choices affect the quality of the model, both in terms of its
characteristics/capabilities and the model’s consequent effectiveness. For
example, consider the intervention point at which the training data is chosen.
Creating a training dataset requires answering questions like: (1) which data
examples should be included in the training dataset; (2) where will the data
be stored (e.g., on whose servers); (3) for how long will the data be
retained; (4) where will the resulting trained model be deployed,
etc?141414For more on choices in training data, see Chapter 1: The Devil is in
the Training data from Lee et al. [63] Choices made about where the model will
be deployed can affect what training data can be used. A model training on
private user data has a very different privacy-risk profile if such a model
were never to leave the user’s personal device, compared to if it is be shared
across many users’ devices.

Not all design choices are about models and how they are trained. Models are
embedded within overarching systems, which consist of many component pieces
that both individually and together reflect the outcomes of relevant
sociotechnical design decisions [27, 26, 78, 14]. There are numerous other
intervention points throughout the supply chain, which involve systems-level
choices [62]. Such intervention points include prompt input filters,
generation output filters, rate limiting (e.g., how many prompts a user can
supply in a given time window to a system), access controls, terms of use,
use-case policies for APIs,151515Google, Anthropic, OpenAI, and Cohere all
have such policies. user interface (UI) and experience (UX) design (e.g., to
guard against over-reliance on generative-AI systems), and so on [78, 44, 14].
Each of these involve design decisions that can have their own legal
implications.

### 4.4 The Massive Scale of Generative-AI Models

Ultimately, the capacity to facilitate “magical,” flexible, open-ended
functionality with Generative AI (Section 4.1 The Transition to Very Flexible
Generative Models) comes from the massive scale at which generative-AI models
are trained [95]. State-of-the-art models today are an order of magnitude
larger and trained on significantly more data than the biggest models from
five years ago.

Techniques for scaling up models have demonstrated a uniquely important role
in unlocking generative-AI capabilities. This includes research into more
efficient neural architectures and better machine-learning systems for
handling model training and inference at scale [84]. For example, many experts
have studied methods for collecting and curating massive, web-scraped datasets
[63, e.g.], as well as the “emergent behaviors” of models trained at such such
large scales [109, e.g.].

In spite of these changes, it is worth noting that many techniques used in
Generative AI today are not new. Language models, for example, have existed
since at least the 1980s [87]. The difference is that, in recent years, we
have figured out how to scale these techniques tremendously (e.g., modern
language models use context windows of thousands of input tokens, compared to
the 5 to 10 input tokens used by language models in the early 2000s). We defer
to machine-learning experts to provide more specific details on the
methodologies and outcomes of scaling.161616Part II.B of Lee et al. [62] has
useful discussion and citations on this topic.

Finally, one of the implications of scale is that machine-learning
practitioners are training fewer state-of-the art models today than were being
trained in the past. When models were small relative to available computing
resources, it was common to re-train a machine-learning system several times,
changing hyperparameters or other configuration details to find the best-
quality model. Today’s model scale means the cost of training just one state-
of-the-art model can be hundreds of thousands or even millions of dollars [8,
5, 62], This further incentivizes the push toward general-purpose models
described in Section 4.1 The Transition to Very Flexible Generative Models.

## 5 A Preliminary Taxonomy of Legal Issues

One significant outcome of the GenLaw discussions was progress toward a
taxonomy of the legal issues that Generative AI raises. We say “progress
toward” because the initial analysis presented here is very much an interim
contribution as part of an ongoing project. The GenLaw workshop was explicitly
scoped to privacy and intellectual property (IP) issues, so this analysis
should be considered non-exhaustive, and the omission of other topics is not a
judgment that they are unimportant. Further, we note that not all
capabilities, consequences, risks, and harms of Generative AI are _legal_ in
nature, so this taxonomy is not a complete guide to generative-AI policy.
Other reports have made significant attempts to catalog such concerns [37,
e.g.]. We instead focus on highlighting the ways in which specifically legal
issues may arise.

We begin with an important high-level point: Generative AI inherits
essentially _all_ of the issues of AI/ML technology more generally. This is so
because Generative AI can be used to perform a large and increasing number of
tasks for which these other types of ML systems have been used. For example,
instead of using a purpose-built sentiment-analysis model, one might simply
prompt an LLM with labeled examples of text and ask it to classify text of
interest; one could use a trained LLM to answer questions with “yes” or “no”
answers (i.e., to perform classification). The resulting classifications may
or may not be as reliable as ones from a purpose-built model, but insofar as
one is using a machine-learning model in both cases, any legal issues raised
by the purpose-built model are also present with the LLM.

Further, any crime or tort that involves communication could potentially be
conducted using a generative-AI system. One could use an LLM to write the text
used for fraud, blackmail, defamation, or spam, or use an image-generation
system to produce deepfakes, obscene content, or false advertisements. Almost
any speech-related legal issue is likely to arise in some fashion in
connection with Generative AI.

With these broader observations in mind, in the remainder of this section we
discuss four legal areas that will need to deal with Generative AI: intention
in torts and criminal law (Section 5.1 Intent), privacy (Section 5.2 Privacy),
misinformation and disinformation (Section 5.3 Misinformation and
Disinformation), and intellectual property (Section 5.4 Intellectual
Property).

### 5.1 Intent

Numerous aspects of law turn on an actor’s intention. For example, in criminal
law, the defendant’s “criminal intent” (_mens rea_), not just the act and its
resulting harms, is often an element of a crime [52]. Intent is not a
universal requirement. Some crimes and torts are “strict liability” (e.g., a
manufacturer is liable for physical harm caused by a defective product
regardless of whether they intended that harm, which they almost always did
not). But where intent is required, a defendant’s lack of wrongful intent
means they cannot be convicted or held liable. For example, fraud is an
intentional tort and crime. A defendant who speaks falsely but honestly to the
best of their knowledge does not commit fraud.

Generative AI will force us to rethink the role of intent in the law. In
contrast to most prior types of ML systems,171717Highly autonomous vehicles
are another such example, but currently remain in relatively limited use [25].
Generative AI can cause harms that are similar to those brought about by human
actors but _without_ human intention. For example, an LLM might emit false and
derogatory claims about a third party – claims that would constitute
defamation if they had been made by a human [107].

There is unlikely to be a simple across-the-board answer as to how the
“intent” of a generative-AI system should be measured, in part because the
legal system uses intention in so many ways and so many places. Consider an
example from a GenLaw discussion. One participant noted that it may be useful
to move to a _respondeat superior_ model — a legal doctrine (often used in
tort law) that ascribes the legal responsibility of an employee to their
employer (if the tort or other wrongful conduct was conducted within the scope
of employment). For this kind of liability model, one could treat the
generative-AI system as the “employee,” and then ascribe responsibility for
harm to the “employer” – i.e., the user. Such an approach appears to sidestep
the need to deal with intent; _respondeat superior_ is strict liability as to
the employer. However, another participant noted that in the usual application
of _respondeat superior_ , there is still an embedded notion of intention.
That is because the employee’s intentions are still relevant in determining
whether a tort has been committed at all; only if there has can liability then
also be placed on the employer. This is not to say that _respondeat superior_
has no role to play, only that it does not avoid difficult questions of
intent.

Another line of discussion at GenLaw concerned whether these difficulties
might lead to a greater focus on the human recipients of generative-AI
outputs. Some authors, for example, have argued that the rise of AI systems
creates a world of “intentionless free speech” in which communications should
be assessed purely based on their utility to the listener [23]. Such a
framework helps establish clearly a First Amendment basis for a right _to use_
Generative AI. But it also raises difficult questions about how to protect
users _from_ Generative AI in cases of false or harmful outputs. These issues
will cut across many legal areas.

### 5.2 Privacy

As discussed above (Section 3 Developing a Shared Knowledge Base), “privacy”
is a notoriously difficult term to define. Different disciplines rely on
different definitions that simplify the concept in different ways, which can
make it very difficult to communicate about privacy across fields. In
particular, computer science and law are known to operate using very different
notions of privacy.181818Even subfields vary greatly. Cryptographers use
different conceptions of “privacy” than ML researchers; decisional privacy in
constitutional law is very different than data privacy in technology law.

For example, in many subfields of computer science, it is common to employ
definitions of privacy based on mathematical formalisms that are
computationally tractable (often differential privacy). In contrast, privacy
in the law is often defined contextually, based on social norms and reasonable
expectations. It is typically necessary to first identify which norms are at
play in a given context, after which it is then possible to determine if those
norms have been violated (and what to do about it). Such definitions of
privacy are fundamentally nuanced; they resist quantification. The tensions
between legal and computer-science approaches to privacy are a source of
communication challenges. At GenLaw, one of the legal experts provided a
useful intuition for this tension: Computer scientists often want to be able
to quantify policy, including policy for handling privacy concerns; in the
law, the mere desire to quantify complex concepts like privacy can itself be
the source of significant problems.

Despite these difficulties, it is still important to be able to reason about
privacy (and when it is violated) in both computing and law. This is not a new
problem: it has been a source of significant practical and research challenges
for essentially as long as computers have been in use. As long as personally
identifiable information (PII) like addresses and phone numbers has been
stored on computers, there have been risks that such information could be seen
or leveraged by others who otherwise would not have had access. Since the
introduction of machine-learning methods to software systems, it has become
possible to predict user behavior and personal preferences (sometimes with
high fidelity); in turn, having access to such arguably private information
has opened up the possibility to develop software that relies on this
information to guide or manipulate user behavior. We understand how difficult
these privacy challenges are only because of decades of research in law and
computer science. Legal scholars have articulated the real-world harms that
people can suffer from through misuse of “private” information; computer
scientists have demonstrated real-world attack vectors through which “private”
information can be leaked.

Generative AI is poised to make these privacy challenges even harder. As noted
above (Section 4.4 The Massive Scale of Generative-AI Models), in contrast to
prior machine-learning models, generative-AI models are typically trained on
large-scale web-scraped datasets. These datasets can contain all sorts of
private information (e.g., PII) [13], which in turn can be memorized and then
leaked in generations [17, 99]. A traditional search engine only locates
individual data points, but a generative-AI model could link together
information in novel ways that reveal sensitive information about individuals.
Adversarially designed prompts can extract other sensitive information, such
as internal instructions used within chatbots [34].191919This is potentially
also a trade-secret issue (Section 5.4 Intellectual Property), depending on
the nature of the information leaked.

### 5.3 Misinformation and Disinformation

Generative AI can be used to produce plausible-seeming but false content at
scale. As such, it may be a significant source of misinformation, and amplify
the speech of actors engaged in disinformation campaigns.202020By
misinformation, we mean material that is false or misleading, regardless of
the intent behind it. In contrast, disinformation consists of deliberately
false or misleading material, often with the purpose of manipulating human
behavior. These capabilities will present issues in any area of law that
prohibits false speech — from lies about people to lies about products to lies
about elections. They will also challenge the assumptions of areas of law that
tolerate false speech out of the belief that such speech will be comparatively
rare and easy to counter. Taken together, these include a wide variety of
legal topics, including defamation, national security, impersonation, bad (or,
in regulated contexts, illegal) advice, over-reliance [14], amplification,
spear-phishing, spam, elections, consumer-protection law (e.g., addiction,
deception, false advertising, products liability), deepfakes, and much else.

From a misinformation perspective, generative-AI models are very sensitive to
their training data, which may itself include misinformation or
disinformation. For example, in August of this year, it was discovered that a
book about mushroom foraging, which was produced with the assistance of an
LLM, contained misinformation about which mushrooms are poisonous (likely due
to inaccurate information learned from the data on which the LLM was trained)
[22]. Similarly, it appears that LLMs are subject to _sycophancy_ , where a
model answers subjective questions in a way that flatters their user’s stated
beliefs, and _sandbagging_ , where models are more likely to endorse common
misconceptions when their user appears to be less educated” [12, 80].

From a disinformation perspective, models can be used to deliberately generate
persuasive but false content scraped from the Internet. But they can also be
deliberately manipulated through adversarially selected fine-tuning data or
through alignment. These processes could be used to skew models to
deliberately produce misleading content.

One point raised by legal scholars at GenLaw is that generated disinformation
about individuals (e.g., deepfakes) will potentially contribute to new types
of defamation-related harms. Other experts, particularly those with a privacy
background in computing, questioned whether such harms could also be
classified as intimate privacy violations [21]. In many respects, the harms
caused by sufficiently convincing forgeries are very similar to those caused
by truthful revelations [115]. Indeed, this is something that Generative AI
seems well-positioned to enable: large-scale, inexpensive production of
believable deepfakes that use a person’s likeness, and depict fake intimate
acts or convey fake intimate information [70, 88].

In response, lawyers with expertise in defamation said they believed that this
would likely not constitute a cognizable _privacy_ harm under the law,
although it would still be actionable as defamation or false light. In turn,
this response raised questions about whether Generative AI could create new
types of harms that blur current conceptions of disinformation and privacy
harms.

### 5.4 Intellectual Property

Naturally, given the recent spate of lawsuits about copyright and Generative
AI (as well as the stated thematic focus for the first GenLaw Workshop), IP
was a frequent topic for emerging legal issues. It has also been one of the
first generative-AI subjects explored in detail by scholars, [62, 89, 91, 15,
108, e.g.], and we leave discussion of the doctrinal details to their work.
Instead, we focus here on a few high-level observations about current and
impending IP issues — many of which also have applications beyond IP.

  * •

_Volition_ : Human volition plays an important and subtle role in defining IP
infringement. For example, copyright infringement normally requires that a
human _intentionally made_ a copy of a protected work, but not that the human
was consciously aware that they were infringing. Generative-AI systems may
occasionally produce outputs that look like duplicates of the training data.
Some participants at GenLaw were concerned that it may be easy to deflect the
role of human-made design choices (Section 4.2 Developments in the Training
Pipeline: Pre-Training and Fine-Tuning) by making such choices seem “internal”
to the system (when, in fact, such choices are typically not foregone
conclusions or strict technical requirements [29]). Since “purely internal”
copies tend to be fair use, such deflection could serve as a copyright
liability shield. Legal experts will need to contend with this possibility in
their analysis of generative-AI systems.212121For a more general treatment of
“scapegoating the system,” see Cooper et al. [29]. Its opposite, in which a
human is held wholly responsible for a harm caused by a technical system, is
the “moral crumple zone,” described in Elish [36]. Additionally, since
generative-AI systems typically take in input from human users, it is also
possible for a user to intentionally cause a model to output potentially
infringing content.

  * •

_Market externalities_ : There are many concerns that Generative AI will lead
to mass labor displacement, significant market changes, and the concentration
of market power. These issues extend beyond IP, but necessarily invoke related
questions of ownership. These are matters for labor law, international trade
law, and other areas of law. But they are also IP issues, because doctrines
such as fair use invite courts to consider such societal effects in weighing
the propriety of particular copying.

  * •

_Trade secrecy_ : Fine-tuning on proprietary data is poised to become a
potentially useful pattern in the adoption of generative-AI technology.
However, existing generative-AI models are known to memorize their training
data [17, 18]. In turn, this raises the possibility that an adversarial user
could extract proprietary information in training data, thereby presenting
issues related to trade secrecy [34].

  * •

_Scraping_ : Similarly, the legality of scraping training data is
intextricable from the IP treatment of Generative AI. Generative-AI companies
both rely on scraped data as an input and take measures (both technical and
legal) to prevent outputs from their systems from being used as inputs to
other systems without permission.

  * •

_Authorship_ : As alluded to above, IP law may need to reconsider authorship
eligibility in light of Generative AI. Computer authorship is not a new topic
of analysis in the law [46, 90, e.g.], but Generative AI is likely to present
new variations on old themes. For example, purely computer-generated works are
not currently covered by copyright. However, some argue that this situation
not sustainable [64, e.g.]. Where AI-generated works have significant value,
there will be strong economic pressures on courts to gives users copyright in
those works.

  * •

_Patent_ : Given that generative-AI modeling techniques also have applications
in the physical sciences (e.g., in drug design, see Section 4 Pinpointing
Unique Aspects of Generative AI), it seems likely that there will be
implications for patent law. For example, U.S. patent law requires a human
inventor as a condition of patent eligibility. Just as copyright’s human
authorship requirement has been challenged (but so far upheld [103, e.g.]),
similar challenges may arise with respect to patents.

  * •

_Idea-expression dichotomy_ : Generative AI seems to further blur the already
often-murky line between idea and expression in copyright law. For example,
one could attempt to analogize the prompt to an idea and the associated
generation to its expression, but this presents several problems. For one
thing, there seems to be a bit of an inversion from the typical pattern: it
suggests that the AI, rather than the human, is responsible for the creative
expression (which is not currently protectable by copyright law). For another,
there may be sufficient creativity for copyrightability of the prompt itself,
even if it is ultimately (by the prior analogy) responsible for the idea in
the resulting generation. Lastly, there is a tenable argument that the human
prompter and generative-AI system are acting in concert to produce the
resulting generation [62], and that the way that an idea is expressed in a
prompt makes it intextricably indivisible from the resulting expressive
generation. In short, as others have noted [67], Generative AI seems to turn
the idea-expression dichotomy “upside down.”

## 6 Toward a Long-Term Research Agenda

Participants in the GenLaw workshop and roundtable identified several
important and promising future research directions. Notably, these topics have
several elements in common. First, each showcases how technical design choices
play a crucial role in legal research questions. Many of the architectures and
applications of generative-AI systems are genuinely novel, compared to
previous technologies that the legal system has had to contend with.
Understanding the legal issues that they raise will require close engagement
with the technical details.

Second, just as design choices can inspire questions for the legal scholars,
it is also important to consider how legal scholarship can influence the
choices that generative-AI researchers make when designing systems (Sections
4.2 Developments in the Training Pipeline: Pre-Training and Fine-Tuning, 4.3
Generative-AI Systems and the Supply Chain). Understanding not just the
current legal framework, but also how that framework may evolve, provides
important guidance for system designers about which technical changes are and
are not legally significant. In addition, a clear sense of the legal
possibility space can help direct generative-AI research toward novel designs,
algorithms, attacks, and characterizations that have beneficial
characteristics.

The list that follows is just a sample of emerging research areas at the
intersection of Generative AI and law. It gives a flavor of how these two
disciplines can concretely inform each other. We believe it is the starting
point of a rich, long-term research agenda with the potential to influence and
inform public policy, education, and industrial best practices.

### 6.1 Centralization and Decentralization

One crucial question about the future of Generative AI concerns the relative
degree of centralization versus decentralization. Consider, as an example, the
controversies over the use of closed-licensed data (within web-scraped
datasets) as training data for generative-AI models (e.g., LAION datasets [4,
94], The Pile [40], Books3 [58], etc.), especially if training involves
removing copyright management information. While such datasets are often
released with open licenses (e.g., the LAION organization has released their
datasets under the MIT license, which allows for use and copying), this does
not guarantee that the associated and constituent data examples in those
datasets can be licensed for use in this way [62]. Many examples within
datasets have closed licenses.222222As Lee et al. [62] notes, this is
particularly complex for datasets used to train multimodal models, like text-
to-image models; the examples to train text-to-image models are image-caption
_pairs_ , where for each pair the image and the text caption could be subject
to their own copyrights (and even hypothetically could be subject to a
copyright as a compilation). Legal scholars have made arguments that run the
gamut of possible fair-use outcomes for the use of these datasets in
Generative AI [91, 67, 62, 96, 89, 47, e.g.]. Nevertheless, it remains to be
seen whether courts will rule that the use of such datasets constitutes fair
use.

In the interim, an alternative path is to invest in producing open,
permissively licensed datasets that avoid the alleged legal issues of using
web-scraped data. This means not only releasing datasets with such licenses,
but ensuring that the underlying data examples in the dataset have clear
provenance [63] and are openly licensed. This is a rich problem domain. It
involves significant technical innovation, both in techniques for collecting
such datasets at scale while respecting licensing conditions and also in
training models that make best use of the limited materials available in them.
(Current attempts to train models on such openly licensed datasets have
yielded mixed results in terms of generation quality [45].) It also requires
substantial legal innovation, including the development of appropriate
licenses that function as intended across jurisdictions, and organizational
innovation in creating authorities to steward such datasets.

These same issues and tensions recur at every stage in the development of
generative-AI systems. This is partly a technical question; current methods
require centralized pre-training at scale based on datasets typically gathered
from highly decentralized creators. Whether either or both of these
constraints will change in the future is an important and open question.
Improvements in training algorithms may reduce the investment required to pre-
train a powerful base model, opening it up to greater decentralization. At the
same time, improvements in synthetic data may enable well-resourced actors to
generate their own training data, partially centralizing the data-collection
step.

Centralization versus decentralization is also partly a business question
(Section 3.3 Understanding Evolving Business Models). There is currently
substantial investment both in large centralized companies that are developing
large base models, and in a large ecosystem of smaller entities developing
fine-tuned models or smaller special-purpose models. The relationships among,
and relative balance between, these different entities is likely to evolve
rapidly in the coming years.

And, most significantly, centralization versus decentralization is a
fundamentally legal question. As noted above, licensing law may inform who can
use a dataset. Competition and antitrust law are likely to play a major role
going forward. Every important potential bottleneck in Generative AI – from
copyright ownership to datasets to compute to models and beyond – will be the
focus of close scrutiny. These novel markets will require technical, economic,
and legal analysis to determine the most appropriate competition policy. In
addition to antitrust enforcement, possible policies include government
subsidies, open-access requirements, “public option” generative-AI
infrastructure, export restrictions, and structural separation. These
questions cannot be discussed intelligently without contributions from both
technical and legal scholars.

### 6.2 Rules, Standards, Reasonableness, and Best Practices

Since the technological capabilities of today’s generative-AI systems are so
new, it is unclear what duties the creators and users of these systems should
be. This overarching problem is not unprecedented for either law or computing.
In some cases, these duties take bright-line rule-like forms; HIPAA strictly
regulates which kinds of data are treated as personally identifying and
subject to stringent security standards. In other cases, these are more
flexible standards that require greater exercises of discretion. In some
cases, the legal system defaults to a general standard of reasonableness: did
a person behave reasonably when developing or using a system? And sometimes,
even when there is no law on point, practitioners have developed best
practices that they follow to do their jobs effectively. We anticipate that
the legal system will need to articulate these duties for generative-AI
creators and users, and to determine which modalities of rules and standards
to employ.

These expectations have always been technology-specific and necessarily change
over time as technology evolves. For example, under the Uniform Trade Secrets
Act (UTSA) information must be “the subject of efforts that are reasonable
under the circumstances to maintain its secrecy.” The threshold for what
efforts are considered “reasonable” has changed over time in response to
developments in information security. Similarly, in cybersecurity, the FTC
monitors the state of the art. As state-of-the-art practices improve, the FTC
has been willing to argue that companies engage in unfair and deceptive trade
practices by failing to implement widely used cost-effective measures.
Further, the definition of reasonableness is contextual; what is considered
reasonable for a large company (e.g., in terms of system development
practices) is typically different than what is considered reasonable for
smaller actors.

In short, legal scholars urgently need to study – and technical scholars
urgently need to explain – which generative-AI safety and security measures
are recognized as efficient and effective. Nor will a one-time exchange
suffice. The legal system must be attuned to the dynamism of generative-AI
development. What is currently an effective countermeasure against extracting
memorized examples from models may fail completely in the face of a newly
developed techniques. But conversely, new techniques of training and alignment
may be developed that are so clearly effective that it is appropriate to
expect future generative-AI creators to employ them. Indeed, the legal system
must be attuned to this dynamism itself, to the fact that our current
understanding of the frontier between the possible and the impossible in
Generative AI is provisional and constantly being refined. There is work here
for many researchers from both communities.

Once technology begins to stabilize, it becomes easier to define concrete
standards (e.g., safety standards). Accordingly, by definition, compliance
with such standards is sufficient for meeting the bar of reasonableness. Until
there is some stability, when harms occur, there will necessarily be some
flexibility; there will be some deference to system builder’s self-assessments
of whether their design choices reflected reasonable best efforts to construct
safe systems. In turn, today’s best efforts will guide future standards-
setting and determination of best practices.

Both the legal and machine-learning research communities should face this
reality head-on; they should take hold of the opportunity to actively engage
in research and public policy regarding today’s generative-AI systems, such
that they can help shape the development of future standards. This work will
require understanding the complexity and particulars of different generative-
AI technologies; effective standards will differ by model modality and other
system capabilities (e.g., generative-AI systems that interact with APIs to
bring in additional content, such as plugins [76]).

To meet this challenge, one clear need is useful metrics to effectively
evaluate the behaviors of generative-AI systems. As we discuss below (Section
6.4 Evaluation Metrics), effective ways to evaluate generative-AI systems
currently remain elusive. System capabilities and harms are not readily
quantifiable; designing useful metrics will be an important, related area of
research for Generative AI and law.

### 6.3 Notice and Takedown $\neq$ Machine Unlearning

Notice and takedown is well-known in both software and legal communities
because of search engines and Section 512 of the US Copyright Act. Enabling
notice-and-takedown functionality has a variety of sociotechnical challenges,
which are notably even more complicated for Generative AI.

For generative-AI models, there is no straightforward analogue for
simply232323Notice and take down can be technically challenging (but
nevertheless feasible) for large-scale software systems that involve
distributed databases that work in concert. removing a piece of data from a
database (as might be the case for removing a file from a video-hosting
platform). Once a model has been trained, the impact of each data example in
the training data is dispersed throughout the model and cannot be easily
traced. In order to remove an example from a trained model, one must either
track down all the places where the example has an impact and identify a way
to negate its influence, or re-train the entire model. This is challenging
because “impact” is not well defined, and neither is
“removal.”242424Alternatively, we must first define what it means to “take
down” a training example from a generative-AI model, which itself is a ill-
defined problem.

There are entire subfields of machine learning devoted to problems like these.
For example, the subfield of “machine unlearning” [11, 16] attempts to define
the desired goals for removing an example and to design algorithms that
satisfy these goals.252525This subfield is also heavily motivated by the The
Right to be Forgotten clause in GDPR. Another line of work attempts to
quantify data-example attribution and influence; it seeks to define
“attribution” and then attribute generations from a model to specific data
examples in the training data. Both machine unlearning and attribution are
very young fields, and their strategies are (for the most part) not yet
computationally feasible to implement in practice for deployed generative-AI
systems. Machine unlearning and attribution are of significant interest for ML
researchers and practitioners. There has been intense (and growing) investment
in this area. How these fields will develop remains to be seen.

### 6.4 Evaluation Metrics

Evaluation is far from a new topic in machine learning (or computing more
generally). Nevertheless, there is a clear need for useful metric definitions
for Generative AI. We discuss some issues of interest below.

Defining metrics for evaluation. It is well-known that the force of legal
rules depends on how they are implemented and interpreted. Many decisions are
made on a case-by-case basis, taking into account specific facts and context.
In contrast to this approach, machine-learning practitioners evaluate systems
at scale. It is common practice to define metrics that can be applied directly
to every situation (or at least a large majority of them). These metrics
necessarily use a pre-specified sets of features that may leave out
considerations that may be important to forming a decision that appropriately
accounts for broader context.

This is hardly a new observation; it has had significant influence machine-
learning subfields, such as algorithmic fairness. More generally, the
challenges of operationalizing or concretizing societal concepts into math has
been discussed at length in prior works [54, 38, 24, 27, e.g.], and developing
reasonable definitions for legal concepts is an active and evolving area of
research [28, 30, 93, e.g.].

Nevertheless, it is worth emphasizing that these observations hold true for
Generative AI.262626There are also specific complexities for Generative AI
that have not been so readily apparent in prior work in machine learning
(e.g., in other areas of machine learning, there are accepted (though
imperfect) notions of “ground truth” labels, which are absent in Generative
AI). For example, as we discussed in Section 3.2 Crafting Useful Metaphors,
researchers create different, precise, definitions of memorization for
different purposes. The definition of memorization for an image-generation
model will differ greatly from a code-generation model or a text-generation
model. Similarly, since “removing” the impact of a training example from a
trained model is an ill-defined problem, researchers may develop different
metrics for quantifying whether or not training data points are successfully
removed.

Evaluation is dynamic. Many metrics are defined in terms of technical
capabilities. For example, the evaluation of the amount of memorized training
data in a model depends on the ability to extract and discover the memorized
training data [18, 53]. As the techniques for data extraction improve, the
evaluation of the model will change. Additionally, the way models are used
alters the way that they should be evaluated. For example, a machine-
unlearning method may be applied to a model to remove the effect of a specific
individual’s data. However, that model may later be fine-tuned on additional
data that is very similar to the removed individual’s data. This may cause the
individual’s data to effectively “resurface.”272727This is presently
speculation, though not all-together baseless. Recent research shows that the
effects of alignment methods may be negated through the course of fine-tuning
[82]. The way that the supply chain is constructed for a particular
generative-AI model may alter the way that the systems (in which that model is
embedded) can and should be evaluated. For example, the analysis may change
depending on whether a model is aligned or not.282828Further, alignment is not
binary. There are different possible degrees of alignment. In turn, as another
example, it is possible that some actors may not have the relevant information
to perform necessary evaluations; some actors may not even know if a
particular model is aligned or not.

##  7 Conclusion and the Future of GenLaw

In this report, we discussed the main topics broached at the first GenLaw
workshop: the importance of developing a shared knowledge base for improved
communication (Section 3 Developing a Shared Knowledge Base), the unique
aspects of Generative AI that present novel challenges and opportunities
(Section 4 Pinpointing Unique Aspects of Generative AI), a taxonomy of
emerging legal issues (Section 5 A Preliminary Taxonomy of Legal Issues), and
associated open research questions at the intersection of Generative AI and
law (Section 6 Toward a Long-Term Research Agenda).

As is clear from the diversity of issues discussed within these topics, it is
difficult to pithily sum up the main takeaways of GenLaw. Nevertheless, we
will attempt to do so, and will sketch out our hopes for the future of GenLaw
as an organization.

  1. 1.

_Expanding beyond copyright concerns_ : Perhaps the fairest overarching
assessment from this report is that GenLaw’s participants believe that
copyright concerns just scratch the surface of potential issues. Put
differently, a common belief was that the legal questions currently under
consideration in U.S. courts only touch on a small area of the potential legal
issues that Generative AI will raise. In part, this is because the underlying
technology is continuing the evolve and be adopted at such a rapid pace. As a
result, the research agenda that we suggest here (Section 6 Toward a Long-Term
Research Agenda) will necessarily evolve over time.

  2. 2.

_Shaking off disciplinary boundaries_ : There remain major open questions
about how best to evaluate the behavior of generative-AI systems. Answering
these questions necessarily will involve machine-learning-technical knowledge,
but they will also involve much more. This report illustrates just how central
legal considerations are to effective evaluation. But we do not intend to
suggest that accounting for these considerations will on its own be
sufficient. As we continue to understand how Generative AI will transform our
interactions, expectations, economy, education, etc., we will need to continue
to shake off disciplinary boundaries in order to design useful and
comprehensive evaluation methodologies.

  3. 3.

_Evolving resources and engagement_ : Given the generative and evolving nature
of generative-AI systems and products, GenLaw’s work to help educate and
facilitate engagement between technologists, legal experts, policymakers, and
the general public will necessarily require ongoing effort. The resources that
we develop (such as those in this report) will need to be frequently updated
to keep pace with technological changes.

In response to these takeaways, we are growing GenLaw into a
nonprofit292929GenLaw is in the process of obtaining 501(c)(3) status. home
for research, education, and interdisciplinary discussion. Thus far, we have
written pieces that make complex and specialized knowledge about law and
Generative AI accessible to both a general audience [63] and to subject-matter
experts [62]. We have worked to provide additional resources, such as
recordings of our events [42, 55], collections of external resources [43], and
the initial glossary on the GenLaw website. For our first in-person workshop,
we engaged with participants that have various expertise in Generative AI,
law, policy, and other computer-science disciplines across 25 different
institutions. We are excited to continue engaging with experts across
industry, academia, and government. While our first event and materials have
had a U.S.-based orientation, we are actively focusing on expanding our
engagement globally. We will be maintaining the GenLaw
website303030https://genlaw.github.io with the most up-to-date information
about future events and resources.

## References

  * Agostinelli et al. [2023] Andrea Agostinelli, Timo I. Denk, Zalán Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, Matt Sharifi, Neil Zeghidour, and Christian Frank.  Musiclm: Generating music from text, 2023.
  * Anil et al. [2023] Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al.  Palm 2 technical report.  _arXiv preprint arXiv:2305.10403_ , 2023.
  * Anthropic [2023] Anthropic.  Introducing Claude, 2023.  URL https://www.anthropic.com/index/introducing-claude.
  * Beaumont [2022] Romain Beaumont.  LAION-5B: A New Era of Large-Scale Multi-Modal Datasets.  _LAION Blog_ , March 2022.  URL https://laion.ai/blog/laion-5b/.
  * Bekman [2022] Stas Bekman.  The Technology Behind BLOOM Training.  _HuggingFace_ , July 2022.  URL https://huggingface.co/blog/bloom-megatron-deepspeed.
  * Ben Zimmer [2023] Ben Zimmer.  ’Hallucination’: When Chatbots (and People) See What Isn’t There, 2023.  URL https://www.wsj.com/articles/hallucination-when-chatbots-and-people-see-what-isnt-there-91c6c88b?st=wns4rqlp2dl1ly5&reflink=desktopwebshare_permalink.
  * Bender et al. [2021] Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell.  On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’21, page 610–623, New York, NY, USA, 2021. Association for Computing Machinery.  ISBN 9781450383097.  doi: 10.1145/3442188.3445922.  URL https://doi.org/10.1145/3442188.3445922.
  * BigScience [2022] BigScience.  BigScience Large Open-science Open-access Multilingual Language Model.  _HuggingFace_ , July 2022.  URL https://huggingface.co/bigscience/bloom.
  * Boers et al. [1993] Egbert JW Boers, Herman Kuiper, Bart LM Happel, and Ida G Sprinkhuizen-Kuyper.  Biological metaphors in designing modular artificial neural networks.  In _ICANN’93: Proceedings of the International Conference on Artificial Neural Networks Amsterdam, The Netherlands 13–16 September 1993 3_ , pages 780–780. Springer, 1993.
  * Bommasani et al. [2021] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al.  On the opportunities and risks of foundation models.  _arXiv preprint arXiv:2108.07258_ , 2021.  URL https://fsi.stanford.edu/publication/opportunities-and-risks-foundation-models.
  * Bourtoule et al. [2021] Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot.  Machine unlearning.  In _2021 IEEE Symposium on Security and Privacy (SP)_ , pages 141–159. IEEE, 2021.
  * Bowman [2023] Samuel R Bowman.  Eight things to know about large language models.  _arXiv preprint arXiv:2304.00612_ , 2023.
  * Brown et al. [2022] Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr.  What Does It Mean for a Language Model to Preserve Privacy?  In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’22, page 2280–2292, New York, NY, USA, 2022. Association for Computing Machinery.  ISBN 9781450393522.  doi: 10.1145/3531146.3534642.  URL https://doi.org/10.1145/3531146.3534642.
  * Brundage et al. [2022] Miles Brundage, Katie Mayer, Tyna Eloundou, Sandhini Agarwal, Steven Adler, Gretchen Krueger, Jan Leike, and Pamela Mishkin.  Lessons learned on language model safety and misuse, 2022.  URL https://openai.com/research/language-model-safety-and-misuse.
  * Callison-Burch [2023] Chris Callison-Burch.  Understanding Generative Artificial Intelligence and Its Relationship to Copyright.  Testimony before The U.S. House of Representatives Judiciary Committee, Subcommittee on Courts, Intellectual Property, and the Internet, May 2023.  Hearing on Artificial Intelligence and Intellectual Property: Part I – Interoperability of AI and Copyright Law.
  * Cao and Yang [2015] Yinzhi Cao and Junfeng Yang.  Towards making systems forget with machine unlearning.  In _2015 IEEE symposium on security and privacy_ , pages 463–480. IEEE, 2015.
  * Carlini et al. [2023a] Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace.  Extracting Training Data from Diffusion Models, 2023a.
  * Carlini et al. [2023b] Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang.  Quantifying Memorization Across Neural Language Models.  In _International Conference on Learning Representations_ , 2023b.
  * Casper et al. [2023] Stephen Casper, Zifan Guo, Shreya Mogulothu, Zachary Marinov, Chinmay Deshpande, Rui-Jie Yew, Zheng Dai, and Dylan Hadfield-Menell.  Measuring the Success of Diffusion Models at Imitating Human Artists, 2023.
  * Chayka [2023] Kyle Chayka.  Is A.I. Art Stealing from Artists?  _The New Yorker_ , February 2023.  URL https://www.newyorker.com/culture/infinite-scroll/is-ai-art-stealing-from-artists.
  * Citron and Solove [2022] Danielle Keats Citron and Daniel J. Solove.  Privacy Harms.  _Boston University Law Review_ , 102, 2022.
  * Cole [2023] Samantha Cole.  ‘Life or Death:’ AI-Generated Mushroom Foraging Books Are All Over Amazon.  _404 Media_ , August 2023.  URL https://www.404media.co/ai-generated-mushroom-foraging-books-amazon/.
  * Collins and Skover [2018] Ronald KL Collins and David M Skover.  _Robotica: speech rights and artificial intelligence_.  Cambridge University Press, 2018.
  * Cooper and Abrams [2021] A. Feder Cooper and Ellen Abrams.  Emergent Unfairness in Algorithmic Fairness-Accuracy Trade-Off Research.  In _Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society_ , page 46–54, New York, NY, USA, 2021. Association for Computing Machinery.  ISBN 9781450384735.
  * Cooper and Levy [2022] A. Feder Cooper and Karen Levy.  Fast or Accurate? Governing Conflicting Goals in Highly Autonomous Vehicles.  _Colorado Technology Law Journal_ , 20, 2022.
  * Cooper and Vidan [2022] A. Feder Cooper and Gili Vidan.  Making the Unaccountable Internet: The Changing Meaning of Accounting in the Early ARPANET.  In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’22, page 726–742, New York, NY, USA, 2022. Association for Computing Machinery.  ISBN 9781450393522.  doi: 10.1145/3531146.3533137.
  * Cooper et al. [2021] A. Feder Cooper, Karen Levy, and Christopher De Sa.  Accuracy-Efficiency Trade-Offs and Accountability in Distributed ML Systems.  In _Equity and Access in Algorithms, Mechanisms, and Optimization_ , New York, NY, USA, 2021. Association for Computing Machinery.  ISBN 9781450385534.  URL https://doi.org/10.1145/3465416.3483289.
  * Cooper et al. [2022a] A. Feder Cooper, Jonathan Frankle, and Christopher De Sa.  Non-Determinism and the Lawlessness of Machine Learning Code.  In _Proceedings of the 2022 Symposium on Computer Science and Law_ , CSLAW ’22, page 1–8, New York, NY, USA, 2022a. Association for Computing Machinery.  ISBN 9781450392341.  doi: 10.1145/3511265.3550446.
  * Cooper et al. [2022b] A. Feder Cooper, Emanuel Moss, Benjamin Laufer, and Helen Nissenbaum.  Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning.  In _2022 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’22, pages 864–876, New York, NY, USA, 2022b. Association for Computing Machinery.  ISBN 9781450393522.  doi: 10.1145/3531146.3533150.
  * Cooper et al. [2023] A. Feder Cooper, Katherine Lee, Solon Barocas, Christopher De Sa, Siddhartha Sen, and Baobao Zhang.  Is My Prediction Arbitrary? Measuring Self-Consistency in Fair Classification.  _arXiv preprint arXiv:2301.11562_ , 2023.
  * Corso et al. [2023] Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola.  DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docki.  In _International Conference on Learning Representations_ , 2023.
  * Deng et al. [2009] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. Ieee, 2009.
  * Deng [2012] Li Deng.  The mnist database of handwritten digit images for machine learning research.  _IEEE Signal Processing Magazine_ , 29(6):141–142, 2012.
  * Edwards [2023] Benj Edwards.  I-powered Bing Chat spills its secrets via prompt injection attack [Updated].  _Ars Technica_ , 2023.  URL https://arstechnica.com/information-technology/2023/02/ai-powered-bing-chat-spills-its-secrets-via-prompt-injection-attack/.
  * EleutherAI [2023] EleutherAI.  the_pile_deduplicated, 2023.  URL https://huggingface.co/datasets/EleutherAI/the_pile_deduplicated.
  * Elish [2019] Madeleine Clare Elish.  Moral crumple zones: Cautionary tales in human-robot interaction (pre-print).  _Engaging Science, Technology, and Society (pre-print)_ , 2019.
  * Fergusson et al. [2023] Grant Fergusson, Caitriona Fitzgerald, Chris Frascella, Megan Iorio, Tom McBrien, Calli Schroeder, Ben Winters, and Enid Zhou.  Generating Harms: Generative AI’s Impact & Paths Forward.  Technical report, Electronic Privacy Information Center, 2023.
  * Friedman and Nissenbaum [1996] Batya Friedman and Helen Nissenbaum.  Bias in Computer Systems.  _ACM Trans. Inf. Syst._ , 14(3):330–347, July 1996.  ISSN 1046-8188.
  * Ganguli et al. [2022] Deep Ganguli, Danny Hernandez, Liane Lovitt, et al.  Predictability and Surprise in Large Generative Models.  In _Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’22, page 1747–1764, New York, NY, USA, 2022. Association for Computing Machinery.  ISBN 9781450393522.  doi: 10.1145/3531146.3533229.  URL https://doi.org/10.1145/3531146.3533229.
  * Gao et al. [2020] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy.  The Pile: An 800GB Dataset of Diverse Text for Language Modeling, 2020.
  * GDPR.eu [2023] GDPR.eu.  General Data Protection Regulation (GDPR), 2023.  URL https://gdpr.eu/tag/gdpr/.
  * GenLaw [2023a] GenLaw.  Generative AI + Law (GenLaw) ’23 – 29 July 2023 [livestream], 2023a.  URL https://www.youtube.com/watch?v=5j4U2UzJWfI.
  * GenLaw [2023b] GenLaw.  GenLaw [Resources], 2023b.  URL https://genlaw.github.io/resources.html.
  * GitHub [2023] GitHub.  About GitHub Copilot for Individuals, GitHub, 2023.  URL https://docs.github.com/en/copilot/overview-of-github-copilot/about-github-copilot-for-individuals.
  * Gokaslan et al. [2023] Aaron Gokaslan, A. Feder Cooper, Jasmine Collins, Landan Seguin, Austin Jacobson, Mihir Patel, Jonathan Frankle, Cory Stephenson, and Volodymyr Kuleshov.  CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images.  _arXiv preprint arXiv:2310.16825_ , 2023.
  * Grimmelmann [2016] James Grimmelmann.  There’s No Such Thing as a Computer-Authored Work – And It’s a Good Thing, Too.  _Columbia Journal of Law and the Arts_ , 39:403, 2016.
  * Henderson et al. [2023] Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A. Lemley, and Percy Liang.  Foundation Models and Fair Use, 2023.
  * Ho et al. [2020] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising Diffusion Probabilistic Models.  In _Advances in Neural Information Processing Systems_ , volume 33, 2020.
  * Hoffmann et al. [2022] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre.  Training Compute-Optimal Large Language Models, 2022.
  * Huckabee v. Meta Platforms, Inc., Bloomberg L.P., Bloomberg Finance, L.P., Microsoft Corporation, and The EleutherAI Institute [2023] Huckabee v. Meta Platforms, Inc., Bloomberg L.P., Bloomberg Finance, L.P., Microsoft Corporation, and The EleutherAI Institute, 2023.  No. 1:23-cv-09152 (S.D. NY Oct. 17, 2023).
  * Ideogram.AI [2023] Ideogram.AI, 2023.  URL https://ideogram.ai/.
  * Institute [2023] Legal Information Institute.  Mens rea, 2023.  URL https://www.law.cornell.edu/wex/mens_rea.
  * Ippolito et al. [2023] Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, and Nicholas Carlini.  Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy, 2023.
  * Jacobs and Wallach [2021] Abigail Z. Jacobs and Hanna Wallach.  Measurement and Fairness.  In _Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency_ , FAccT ’21, page 375–385, New York, NY, USA, 2021. Association for Computing Machinery.  ISBN 9781450383097.  doi: 10.1145/3442188.3445901.
  * James Grimmelmann [2023] James Grimmelmann.  GenLaw [liveblog], 2023.  URL https://3d.laboratorium.net/2023-07-29-genlaw.
  * Katz et al. [2023] Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo.  Gpt-4 passes the bar exam.  _Social Science Research Network 4389233_ , 2023.
  * Kiritchenko et al. [2014] Svetlana Kiritchenko, Xiaodan Zhu, and Saif M Mohammad.  Sentiment analysis of short informal texts.  _Journal of Artificial Intelligence Research_ , 50:723–762, 2014.
  * Knibbs [2023] Kate Knibbs.  The Battle Over Books3 Could Change AI Forever.  _Wired_ , September 2023.  URL https://www.wired.com/story/battle-over-books3/.
  * Kudugunta et al. [2023] Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Christopher A Choquette-Choo, Katherine Lee, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, et al.  Madlad-400: A multilingual and document-level large audited dataset.  _arXiv preprint arXiv:2309.04662_ , 2023.
  * Le et al. [2023] Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu.  Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale, 2023.
  * Lee et al. [2022] Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini.  Deduplicating Training Data Makes Language Models Better.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics_ , volume 1, pages 8424–8445, 2022.
  * Lee et al. [2023a] Katherine Lee, A. Feder Cooper, and James Grimmelmann.  Talkin’ ’Bout AI Generation: Copyright and the Generative-AI Supply Chain.  _arXiv preprint arXiv:2309.08133_ , 2023a.
  * Lee et al. [2023b] Katherine Lee, A. Feder Cooper, James Grimmelmann, and Daphne Ippolito.  AI and Law: The Next Generation, 2023b.  URL http://dx.doi.org/10.2139/ssrn.4580739.
  * Lee [2023] Timothy B. Lee.  Opinion: The Copyright Office is making a mistake on AI-generated art.  _Ars Technica_ , 2023.  URL https://arstechnica.com/tech-policy/2023/09/opinion-dont-exclude-ai-generated-art-from-copyright/.
  * Legal Information Institute (2023a) [Cornell] Legal Information Institute (Cornell).  17 U.S. Code § 102 - Subject matter of copyright: In general, 2023a.  URL https://www.law.cornell.edu/uscode/text/17/102.
  * Legal Information Institute (2023b) [Cornell] Legal Information Institute (Cornell).  17 U.S. Code § 107 - Limitations on exclusive rights: Fair use, 2023b.  URL https://www.law.cornell.edu/uscode/text/17/107.
  * Lemley [2023] Mark A. Lemley.  How Generative AI Turns Copyright Law on its Head, 2023.  URL https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4517702.
  * Li et al. [2023] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi.  BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models, 2023.
  * Longpre et al. [2023] Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, and Daphne Ippolito.  A Pretrainer’s Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity, 2023.
  * Maiberg [2023] Emanuel Maiberg.  Inside the AI Porn Marketplace Where Everything and Everyone Is for Sale.  _404 Media_ , August 2023.  URL https://rb.gy/yylwk.
  * Midjourney [2023] Midjourney, 2023.  URL https://www.midjourney.com/.
  * MosaicML [2023] MosaicML, 2023.  URL https://www.mosaicml.com/.
  * Murphy [2022] Kevin P. Murphy.  _Probabilistic Machine Learning: An introduction_.  The MIT Press, 2022.
  * OpenAI [2022] OpenAI.  ChatGPT: Optimizing Language Models for Dialogue, 2022.  URL https://web.archive.org/web/20221130180912/https://openai.com/blog/chatgpt/.
  * OpenAI [2022] OpenAI.  Aligning language models to follow instructions, 2022.  URL https://openai.com/research/instruction-following.
  * OpenAI [2023a] OpenAI.  ChatGPT plugins, March 2023a.  URL https://openai.com/blog/chatgpt-plugins.
  * OpenAI [2023b] OpenAI.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ , 2023b.
  * OpenAI [2023c] OpenAI.  GPT-4 System Card.  Technical report, March 2023c.  URL https://cdn.openai.com/papers/gpt-4-system-card.pdf.
  * Patrick von Platen [2020] Patrick von Platen.  How to generate text: using different decoding methods for language generation with Transformers, 2020.  URL https://huggingface.co/blog/how-to-generate.
  * Perez et al. [2022] Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al.  Discovering language model behaviors with model-written evaluations.  _arXiv preprint arXiv:2212.09251_ , 2022.
  * Piantadosi and Hill [2022] Steven T Piantadosi and Felix Hill.  Meaning without reference in large language models.  _arXiv preprint arXiv:2208.02957_ , 2022.
  * Qi et al. [2023] Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson.  Fine-tuning aligned language models compromises safety, even when users do not intend to!, 2023.
  * Radford et al. [2022] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever.  Robust Speech Recognition via Large-Scale Weak Supervision, 2022.  URL https://cdn.openai.com/papers/whisper.pdf.
  * Ratner et al. [2019] Alexander Ratner, Dan Alistarh, Gustavo Alonso, David G. Andersen, et al.  MLSys: The New Frontier of Machine Learning Systems, 2019.
  * Riedl [2023] Mark Riedl.  A Very Gentle Introduction to Large Language Models without the Hype, April 2023.  URL https://rb.gy/tkfw5.
  * Rombach et al. [2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-Resolution Image Synthesis with Latent Diffusion Models.  In _2022 IEEE Conference on Computer Vision and Pattern Recognition_ , 2022.
  * Rosenfeld [2000] Ronald Rosenfeld.  Two decades of statistical language modeling: Where do we go from here?  _Proceedings of the IEEE_ , 88(8):1270–1278, 2000.
  * Rubin [2023] April Rubin.  Teens exploited by fake nudes illustrate threat of unregulated AI.  _Axios_ , November 2023.  URL https://www.axios.com/2023/11/03/ai-deepfake-nude-images-new-jersey-high-school.
  * Sag [2023] Matthew Sag.  Copyright Safety for Generative AI.  _Houston Law Review_ , 2023.  Forthcoming.
  * Samuelson [1985] Pamela Samuelson.  Allocating Ownership Rights in Computer-Generated Works.  _University of Pittsburgh Law Review_ , 47:1185, 1985.
  * Samuelson [2023] Pamela Samuelson.  Generative AI meets copyright.  _Science_ , 381(6654):158–161, 2023.  doi: 10.1126/science.adi0656.  URL https://www.science.org/doi/abs/10.1126/science.adi0656.
  * Scale AI [2023] Scale AI, September 2023.  URL https://scale.com/.
  * Scheffler et al. [2022] Sarah Scheffler, Eran Tromer, and Mayank Varia.  Formalizing Human Ingenuity: A Quantitative Framework for Copyright Law’s Substantial Similarity.  In _Proceedings of the Symposium on Computer Science and Law_ , pages 37–49, 2022.
  * Schuhmann et al. [2022] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev.  LAION-5B: An open large-scale dataset for training next generation image-text models.  In _Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2022.
  * Smith et al. [2023] Samuel L. Smith, Andrew Brock, Leonard Berrada, and Soham De.  ConvNets Match Vision Transformers at Scale, 2023.
  * Sobel [2021] Benjamin L.W. Sobel.  A Taxonomy of Training Data: Disentangling the Mismatched Rights, Remedies, and Rationales for Restricting Machine Learning.  In Jyh-An Lee and Kung-Chung Liu, editors, _Artificial Intelligence and Intellectual Property_. 2021.
  * Sohl-Dickstein et al. [2015] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathany, and Surya Ganguli.  Deep Unsupervised Learning using Nonequilibrium Thermodynamics.  In _Proceedings of the 32nd International Conference on Machine Learning_ , 2015.
  * Solove [2001] Daniel Solove.  Privacy and Power: Computer Databases and Metaphors for Information Privacy.  _Stanford Law Review_ , 53(6), 2001.
  * Somepalli et al. [2022] Gowthami Somepalli, Vasu Singla, Micah Goldblum, Jonas Geiping, and Tom Goldstein.  Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models, 2022.
  * Song and Ermon [2019] Yang Song and Stefano Ermon.  Generative Modeling by Estimating Gradients of the Data Distribution.  In _Advances in Neural Information Processing Systems_ , volume 32, 2019.
  * Stability AI [2022] Stability AI.  Stable Diffusion Public Release, August 2022.  URL https://stability.ai/blog/stable-diffusion-public-release.
  * Stability AI [2023] Stability AI, 2023.  URL https://stability.ai/.
  * Thaler v. Perlmutter [2023] Thaler v. Perlmutter, 2023.  No. 22-1564 (D.D.C. August 18, 2023).
  * Together AI [2023] Together AI, 2023.  URL https://together.ai/.
  * Touvron et al. [2023] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, et al.  Llama 2: Open Foundation and Fine-Tuned Chat Models, 2023.
  * Vaswani et al. [2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkorei, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and llia Polosukhin.  Attention Is All You Need.  In _Advances Neural Information Processing Systems_ , volume 30, 2017.
  * Volokh [2023] Eugene Volokh.  Large Libel Models? Liability for AI Output.  _Journal of Free Speech Law_ , 3:489–558, 2023.  URL https://www.journaloffreespeechlaw.org/volokh4.pdf.
  * Vyas et al. [2023] Nikhil Vyas, Sham Kakade, and Boaz Barak.  On Provable Copyright Protection for Generative Models, 2023.
  * Wei et al. [2022] Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus.  Emergent Abilities of Large Language Models, 2022.
  * Weights & Biases [2023] Weights & Biases, 2023.  URL https://wandb.ai/site.
  * Wikipedia [Accessed October 2023a] Wikipedia.  Differential privacy, Accessed October 2023a.  URL https://en.wikipedia.org/wiki/Differential_privacy.
  * Wikipedia [Accessed October 2023b] Wikipedia.  Generalization error, Accessed October 2023b.  URL https://en.wikipedia.org/wiki/Generalization_error.
  * Yilmaz and Karaoglan Yilmaz [2023] Ramazan Yilmaz and Fatma Gizem Karaoglan Yilmaz.  The effect of generative artificial intelligence (AI)-based tool use on students’ computational thinking skills, programming self-efficacy and motivation.  _Computers and Education: Artificial Intelligence_ , 4:100147, 2023.  ISSN 2666-920X.  doi: https://doi.org/10.1016/j.caeai.2023.100147.
  * Yiu et al. [2023] Eunice Yiu, Eliza Kosoy, and Alison Gopnik.  Imitation versus innovation: What children can do that large language and language-and-vision models cannot (yet)?  _arXiv preprint arXiv:2305.07666_ , 2023.
  * Zipursky and Goldberg [2023] Benjamin C Zipursky and John CP Goldberg.  A tort for the digital age: False light invasion of privacy reconsidered.  2023\.
  * Zittrain [2008] Jonathan Zittrain.  _The Future of the Internet–And How to Stop It_.  Yale University Press, USA, 2008.  ISBN 0300124872.

## A Glossary

We roughly divide our glossary into two sections: terms from machine learning
and Generative AI (Appendix A.1 \- A.2), and those from the law (Appendix A.3
\- A.4). We further subdivide these sections, and then alphabetize terms
within them.

[sections] paragraphpsections [sections]l3

###  A.1 Concepts in Machine Learning and Generative AI

  1. Algorithm An algorithm is a formal, step-by-step specification of a process. Machine learning uses algorithms for training models and for applying models (a process called inference). In training, the algorithm takes a model architecture, training data, hyperparameters, and a random seed (to enable random choices during statistical computations) to produce trained model parameters.

In public discourse around social media, the term algorithm is often used to
refer to methods for optimizing the probability that a user will engage with a
post; however, it is important to note that algorithms describe many processes
including, for example, the process of sorting social media posts by date.

  2. Alignment Alignment refers to the process of taking a pre-trained model and further tuning it so that its outputs are aligned with a policy set forth by the model developer. Alignment can also refer to the state of being aligned — some academic papers might compare and contrast between an aligned and an unaligned model. The goals included in an alignment policy (sometimes called a constitution) vary from developer to developer, but common ones include:

     * •

Following the intent of user-provided instructions;

     * •

Abiding by human values (e.g., not emitting swear-words);

     * •

Being polite, factual, or helpful;

     * •

Avoiding generating copyrighted text.

While the goals of alignment are vast and pluralistic, current techniques for
achieving better alignment are broadly applicable across goals. These
techniques include reinforcement learning with human feedback and full model
fine-tuning. Specifying the desired properties of alignment often requires a
special dataset. These datasets may include user-provided feedback, supplied
through the user interface in a generative-AI product.

  3. Application Programming Interface (API) Companies choose between releasing generative-AI model functionality in a variety of ways: They can release the model directly by open-sourcing it; they can embed the model in a software system, which they release as a product; or, they can make the model (or the system its embedded in) available via an Application Programming Interface (API). When a model is open-source, anyone can take the checkpoint and load it onto a personal computer in order to use it for generation (by embedding the checkpoint in a program). In contrast, when a company only makes their generative-AI model or system available via an API, that means that users access it in code. The user writes a query in a format specified by the company, then sends the query to the company’s server. The company then runs the model or system on their own computers, and provides a response to the user with the generated content. API access usually requires accepting the company’s Terms of Service, and companies may add extra layers of security on top of the model (such as rejecting queries identified as being in violation of the ToS).

  4. Architecture The design of a model is called its architecture. For a neural network, architectural decisions include the format of inputs that can be accepted (e.g., images with a certain number of pixels), the number of layers, how many parameters per layer, how the parameters in each layer are connected to each other, and how we represent the intermediate state of the model as each layer transforms input to output. The most common architecture for language tasks is called a Transformer [106], for which there are many variations.

Many contemporary models appear in model families that have similar
architectures but different sizes, often differentiated by the total number of
parameters in the model. For example, Meta originally released four sizes of
the LLaMA family that had almost the exact same architectures, differing only
in the number of layers and size of the intermediate vector representations.
More layers and wider internal representations can improve the capability of a
model, but can also increase the amount of time it takes to train the model or
to do inference.

  5. Attention An attention mechanism is a sub-component of a transformer architecture. This mechanism allows a neural network to selectively focus on (i.e., attend to) specific tokens in the input sequence by assigning different attention weights to each token. Attention also refers to the influence an input token has on the token that is generated by the model.

  6. Base Model A base model, which is sometimes called a “foundation” or “pre-trained model,” is a neural network that has been pre-trained on a large, general-purpose dataset (e.g., a web crawl). Base models can be thought of as the first step of training and a good building block for other models. Thus, base models are not typically exposed to Generative AI users; instead they are adapted to be more usable either through alignment or fine-tuning or performing in-context learning for specific tasks. For example, OpenAI trained a base model called GPT-3 then adapted it to follow natural-language instructions from users to create a subsequent model called InstructGPT [75].

See also: pre-training and fine-tuning.

  7. Checkpoint While a model is being trained, all of its parameters are stored in the computer’s memory, which gets reset if the program terminates or the computer turns off. To keep a model around long-term, it is written to long-term memory, i.e., a hard drive in a file called a checkpoint. Often, during training, checkpoints are written to disk every several thousand steps of training. The minimum bar for a model to be considered open-source is if there has been a public release of one of its checkpoints, as well as the code needed to load the checkpoint back into memory.

  8. Context Window Also called prompt length. Generative AI prompts typically have a fixed context window. This is the maximum accepted input length for the model and arises because models are trained with data examples that are no longer than this maximum context window. Inputs longer than this maximum context window may result in generations with performance degradations.

  9. Data Curation and Pre-Processing Data curation is the process of creating and curating a dataset for training a model. In the past, when datasets were smaller, they could be manually curated, with human annotators assessing the quality of each example. Today, the final datasets used to train generative machine-learning models are typically automatically curated. For example, data examples identified as “toxic” or “low quality” may be removed. This filtering is done using an algorithm. These algorithms may include heuristic rules (e.g., labeling examples containing forbidden words as toxic) or may use a different machine-learning model trained to classify training examples as low-quality or toxic. Another common curation step is to deduplicate the dataset by identifying examples that are very similar to each other and removing all but one copy [61].

Data pre-processing involves transforming each example in the dataset to a
format that is useful for the task we want our machine-learning model to be
able to do. For example, a web page downloaded from a web crawl might contain
HTML markup and site navigation headers. We may want to remove these
components and keep only the content text.

Data curation and pre-processing happens directly to the data and are
independent of any model. Different models can be trained on the same dataset.
For more information about dataset curation and pre-processing, see Lee et al.
[63, Chapter 1]

  10. Datasets Datasets are collections of data examples. Datasets are used to train machine-learning models. This means that the parameters of a machine-learning model depend on the dataset. Different machine-learning models may require different types of data and thus different datasets. The choice of dataset depends on the task we want the machine-learning model to be able to accomplish. For example, to train a model that can generate images from natural-language descriptions, we would need a dataset consisting of aligned image-text pairs.

Datasets are often copied and reused for multiple projects because (a) they
are expensive and time-consuming to create, and (b) reuse makes it easier to
compare new models and algorithms to existing methods (a process called
benchmarking). Datasets are usually divided into training and testing
portions. The testing portion is not used during training, and is instead used
to measure how well the resulting model generalizes to new data (how well the
model performs on examples that look similar to the training data but were not
actually used for training).

Datasets can be either created directly from a raw source, such as Wikipedia
or a web crawl, or they can be created by assembling together pre-existing
datasets. Here are some popular “source” datasets used to train generative
machine learning models:

     * •

Wikipedia

     * •

Project Gutenberg

     * •

Common Crawl

     * •

C4

     * •

ImageNet

     * •

LAION-400M

     * •

ROOTS

Here are some popular datasets that were created by collecting together
several pre-existing datasets:

     * •

The Pile (The Pile was formerly listed online, but was removed following the
Huckabee v. Meta Platforms, Inc., Bloomberg L.P., Bloomberg Finance, L.P.,
Microsoft Corporation, and The EleutherAI Institute class action complaint
[50]. Notably, a de-duplicated version of the dataset is still available on
HuggingFace via EleutherAI, as of the writing of this report [35].)

     * •

Dolma

     * •

RedPajama

Collection-based datasets tend to have a separate license for each constituent
dataset rather than a single overarching license.

  11. Decoding A decoding algorithm is used by a language model to generate the next word given the previous words in a prompt. There are many different types of decoding algorithms including: greedy algorithm, beam-search, and top-k [79]. For language models, the term decoding may be used interchangeably with inference, generation, and sampling.

  12. Diffusion-Based Modeling Diffusion-based modeling is an algorithmic process for model training. Diffusion is _not_ itself a model architecture, but describes a process for training a model architecture (typically, an underlying neural network) [97, 86, 48, 100]. Diffusion-based models are commonly used for image generation, such as in Stable Diffusion text-to-image models [101]. Diffusion probabilistic model, diffusion model, and latent diffusion model have become terms that refer to types of models (typically, neural networks) that are trained using diffusion processes.

  13. Embedding (Also called vector representation.) Embeddings are numerical representations of data. There are different types of embeddings, such as word embeddings, sentence embeddings, image embeddings, etc. [73, p. 26, pp. 703-10]. Embeddings created with machine-learning models seek to model statistical relationships between the data seen during training.

“Common embedding strategies capture semantic similarity, where vectors with
similar numerical representations (as measured by a chosen distance metric)
reflect words with similar meanings.

Needless to say, such quantified data are not identical to the entities they
reflect, _however_ , they can capture certain useful information about said
entities” Quoted from Lee et al. [62, p. 7]:

  14. Examples An example is a self-contained piece of data. Examples are assembled into datasets. Depending on the dataset, an example can be an image, a piece of text (such as content of a web page), a sound snippet, a video, or some combination of these. Often times examples are labeled — they have an input component that can be passed into a machine-learning model, and they have a target output, which is what we would like the model to predict when it sees the input. This format is usually referred to as an “input-output” or “input-target” pair. For example, an input-target example pair for training an image-classification model would consist of an image as the input, and a label (e.g. whether this animal is a dog or a cat) as the target.

  15. Generalization Generalization in machine learning refers to a model’s ability to perform well on unseen data, i.e., data it was not exposed to during training. Generalization error is usually measured evaluating the model on training data and comparing it with the evaluation of the model on test data [112]. (See datasets for more on training and test data.)

  16. Generation Generative models produce complex, human-interpretable outputs, such as full sentences or natural-looking images, called generations. Generation also refers to the process of applying the generative-AI model to an input and generating an output. The input to a generative model is often called a prompt. More traditional machine-learning models are limited to ranges of numeric outputs (regression) or discrete output labels like “cat” and “dog” (classification). More commonly the word inference is used to describe applying a traditional machine-learning model to inputs.

Generative models can output many different generations for the same prompt,
which all may be valid to a user. For example, there may be many different
kinds of cats that would all look great wearing a hat. This makes evaluating
the performance of generative models challenging (i.e., how can we tell which
is the “best” such cat?). Recent developments in Generative AI have
significantly increased generation quality. However, even high-quality
generations can reach an uncanny valley with subtly wrong details (e.g.,
incorrect number of fingers on a typical human hand).

See also: inference.

  17. Fine-Tuning See description alongside pre-training.

  18. Foundation Model Foundation model is a term coined by researchers at Stanford University [10] to refer to neural networks that are trained on very large general-purpose datasets, after which they can be adapted to many different applications. Another common word, used interchangeably with foundation model, is base model.

  19. Hallucination There are two definitions of the word hallucination [6]. First, hallucination can refer to generation that does not accord with our understanding of reality, e.g., the generation for the text prompt “bibliographies” consisting of research papers that do not exist. These hallucinations may occur because generative models do not have explicit representations of facts or knowledge. Second, generations that have nothing to do with the input may also be termed a hallucination, e.g., an image generated of a cat wearing a hat in response to the prompt “Science fiction from the 80s.”

  20. Hyperparameter Hyperparameters are settings for the model or training process that are manually specified prior to training. Unlike parameters, they are not typically learned (often, they are hand-selected). Examples of hyperparameters for a model include properties of the architecture, such as input sequence length, the number of model parameters per layer, and number of layers. Examples of hyperparameters that determine the behavior of the training algorithm include the learning rate, which controls how much we update model parameters after each input/output training example (i.e., the magnitude of the update). The process of picking hyperparameters is typically called hyperparameter optimization, which is its own field of research.

  21. In-Context Learning (Zero-Shot / Few-Shot) A base model can be used directly without creating new checkpoints through fine-tuning. In-context learning is a method of adapting the model to a specific application by providing additional context to the model through the prompt. This can be used instead of the more computationally expensive fine-tuning process, though it may not be as effective. In-context learning involves creating an input or prompt to a model in a way that constrains a desired output. Typically the input includes either instructions (zero-shot: a natural language description of the output) or a small number of examples of input-output pairs (few-shot). “Shot” refers to the number of examples provided.

  22. Inference More traditional machine-learning methods (like regression and classification) often use the word inference instead of generation to refer to the process of applying a trained model to input data. However, both inference and generation are used to describe the generation process for a generative-AI model.

See also: generation.

  23. Language Model A language model (LM) is a type of model that takes a sequence of text as input and returns a prediction for what the next word in the text sequence should be. This prediction is usually in the form of a probability distribution. For example, when passed the input “It is raining,” the language model might output that the probability of “outside” is 70%, and the probability of “inside” is 5%. Language models used to be entirely statistical; the probability of a “outside” coming next in the phrase would be computed by counting the number of times “outside” occurred after the sequence “It is raining” in their training dataset. Modern language models are implemented using neural networks, which have the key advantage that they can base their output probabilities on complex relationships between sequences in the training data, rather than just counting how often each possible sequence occurs. As an illustrative example, a neural network may be able to use information from a phrase like “It’s a monsoon outside” occurring in the training data to increase the probability of the word “outside.”

A language model can be used for generation by employing an algorithm that
selects a word to generate given the probabilities output by the model for
some prompt sequence. After each word is selected, the algorithm appends that
word to the previous prompt to create a new prompt, which is then used to pick
the next word, and so on. Such an algorithm is referred to as a decoding
algorithm.

Language-model generation can be used to implement many tasks, including
autocomplete (given the start of a sentence, how should the sentence be
completed?) and translation (translate a sentence from English to Chinese).
The probabilities output by the language model can also be used directly for
tasks. For example, for a sentiment classification task, we might ask the
language model whether “but” or “because” is a more probable next word given
the prompt “the food was absolutely delicious.”

For more information, see Riedl [85].

  24. Large Language Model (LLM) The term large language model (LLM) has become popular as a way to distinguish older language models from more modern ones, which use the transformer architecture with many parameters and are trained on web-scale datasets. Older models used different model architectures and fewer parameters, and were often trained on smaller, more narrowly scoped datasets. Consensus for what constitutes “large” has shifted over time as previous generations of large language models are replaced with models with even more parameters that are trained for even more steps.

  25. Loss Neural networks (and other models) take an input and predict an output. The distance (measured by some specified function) between this output and the output we expect the model to predict (i.e., the target) is called the loss. Neural networks are trained by an algorithm that repeatedly passes examples into the the network, measures the loss compared to the expected output, and then updates the network’s parameters so as to reduce the size of the loss on these examples. The goal of this training process is to minimizing the loss over all the exampling in the training dataset.

Some research areas (for example, reinforcement learning) refer to “maximizing
a reward” rather than “minimizing a loss.” These concepts are largely
interchangeable; a loss can be turned into a reward by adding a negative sign
to it.

See also: objective.

  26. Memorization Memorization generally refers to being able to deduce or produce a model’s given training example.

There are further delineations in the literature about different types of
memorization. A training example may be memorized by a model if information
about that training example can be discovered inside the model through any
means. A training example is said to be extracted from a model if that model
can be prompted to generate an output that looks exactly or almost exactly the
same as the training example. A training example may be regurgitated by the
model if the generation looks very similar or almost exactly the same as the
training example (with or without the user’s intention to extract that
training example from the model).

To tease these words apart: a training example is memorized by a model and can
be regurgitated in the generation process regardless of whether the intent is
to extract the example or not.

The word memorization itself may be used to refer to other concepts that we
may colloquially understand as “memorization.” For example, facts and style
(artists style) may also be memorized, regurgitated, and extracted. However,
this use should not be confused with technical words (e.g., extraction) with
precise definitions that correspond to metrics.

  27. Model Models are at the core of contemporary machine learning. A model is a mathematical tool that takes an input and produces an output. A simple example might be a model that tells you whether a temperature is above or below average for a specific geographic location. In this case the input is a number (temperature) and the output is binary (above/below).

There could be many versions of this model depending on geographic location.
The behavior of the model is defined by an internal parameter (or, weight). In
our temperature example, the model has one parameter, the average temperature
for the location. The process for setting the value of the parameter for a
specific version of the model is called training. In this case we might train
a model for New York City by gathering historical temperature data and
calculating the average. The process of gathering historical data is called
data collection. The process of training — in this case, calculating the
average — is an algorithm.

A saved copy of a model’s trained parameters is called a checkpoint. We might