# Text Data Augmentation in Low-Resource Settings
via Fine-Tuning of Large Language Models

Jean Kaddour
University College London
&Qi Liu
University of Hong Kong

###### Abstract

The in-context learning ability of large language models (LLMs) enables them
to generalize to novel downstream tasks with relatively few labeled examples.
However, they require enormous computational resources to be deployed.
Alternatively, smaller models can solve specific tasks if fine-tuned with
enough labeled examples. These examples, however, are expensive to obtain. In
pursuit of the best of both worlds, we study the annotation and generation of
fine-tuning training data via fine-tuned teacher LLMs to improve the
downstream performance of much smaller models. In four text classification and
two text generation tasks, we find that both data generation and annotation
dramatically improve the respective downstream model’s performance,
occasionally necessitating only a minor fraction of the original training
dataset.

##  1 Introduction

Large language models (LLMs) have demonstrated _in-context learning_ (ICL)
capabilities in various natural language processing tasks, which allow us to
perform an unseen downstream task by prompting the model with a collection of
input-target pairs and a single unlabeled example [8]. Crucially, ICL requires
relatively few labeled examples but large model sizes [35]. However, deploying
LLMs in real-world systems is challenging due to their computational costs and
inference latencies [20].

An alternative paradigm that enables good results with much smaller models is
to specialize a pre-trained model for a single task through gradient-based
_supervised fine-tuning_ (SFT) [10, 11]. The drawback of this approach is that
it relies on labeled examples, which require human annotators and, therefore,
is expensive and time-consuming. Especially in low-resource settings with only
a handful of examples, SFT can be challenging [37].

In this work, we attempt to yield the best of both worlds by fine-tuning
smaller models with training data generated by an LLM. Following recent work
on training data generation [30, 26, 20], we show that by (i) _annotating_
unlabeled examples or (ii) _generating_ entirely new ones, we can effectively
transfer knowledge from the LLM (_teacher_) to the specialized model
(_student_), which can be several magnitudes of orders smaller, akin to
knowledge distillation [17] but only via the exchange of data.

To our knowledge, our work is the first to show that _d ata augmentation via
fine-tuning the teacher LLM_ (DAFTT), even on only extremely limited data,
improves the synthetic data quality as measured by the downstream model
generalization performance. For example, fine-tuning a 20B LLM on as few as
125 examples (5% of the RTE dataset [32]) increases the augmented-data-fine-
tuned downstream model’s performance by multiple percentage points.

We empirically verify our approach on four text classification and two natural
language generation tasks, finding that both (i) and (ii) consistently improve
the student model’s downstream performance. We provide ablation studies on
varying amounts of synthetic data, comparisons with GPT3.5 [8] as a teacher
model, and evaluate the teacher LLMs directly on the downstream tasks.

Figure 1: Data Augmentation via Fine-Tuning the Teacher LLM (DAFTT): _Step 1:_
We fine-tune (FT) an LLM (GPT-NeoX-20B [6]). _Step 2:_ We either annotate
unlabeled instances or generate entirely new ones. _Step 3:_ We train a small
downstream model on the augmented training dataset.

##  2 Method

###  2.1 Problem Definition

Consider a task-specific and labeled dataset
${\mathcal{D}}^{{\mathcal{T}}}:=\\{({\bm{x}}_{i},{\bm{y}}_{i})\\}_{i=1}^{N}$
of text input-output pairs for some task ${\mathcal{T}}$, a teacher LLM
paratemerized by $\bm{\theta^{\text{T}}}$ and lightweight student model
$\bm{\theta^{\text{S}}}$, with
$|\bm{\theta^{\text{S}}}|\ll|\bm{\theta^{\text{T}}}|$. We aim to yield
$\bm{\theta^{\text{S}}}$ that achieves similar performance to
$\bm{\theta^{\text{T}}}$ in solving ${\mathcal{T}}$. The premise is that
$\bm{\theta^{\text{T}}}$ performs much better than $\bm{\theta^{\text{S}}}$
but is more expensive to deploy. Our strategy is to generate synthetic data
$\widetilde{{\mathcal{D}}}^{{\mathcal{T}}}$ using $\bm{\theta^{\text{T}}}$,
and then fine-tune $\bm{\theta^{\text{S}}}$ on
${\mathcal{D}}^{{\mathcal{T}}}\cup\widetilde{{\mathcal{D}}}^{{\mathcal{T}}}$.

We further distinguish the cases of (i) _annotating_ an unlabeled set of
instances $\\{{\bm{x}}_{i}\\}_{i=1}^{M}$ such that
$\widetilde{{\mathcal{D}}}^{{\mathcal{T}}}:=\\{({\bm{x}}_{i},\hat{\bm{y}}_{i})\\}_{i=1}^{M}$,
or (ii) _generating_ entire input-output pairs
$\widetilde{{\mathcal{D}}}^{{\mathcal{T}}}:=\\{(\hat{\bm{x}}_{i},\hat{\bm{y}}_{i})\\}_{i=1}^{M}$,
where $\hat{\bm{x}}_{i},\hat{\bm{y}}_{i}$ refer to synthetic input and output
sequences, respectively. (i) is relevant in scenarios where an abundance of
unlabeled text is available (e.g., scraped from the internet), but obtaining
annotations is expensive. In turn, (ii) is suitable even in situations without
unlabeled instances.

###  2.2 Teacher model

The teacher LLM is the 20B GPT-NeoX [6], which, at the time of our
experimentation phase, was considered state-of-the-art 111In the meantime,
many better-performing models have been released [20, 1], which would likely
further improve the effectiveness of our approach.. We adopt the text-to-text
framework [27], where the LLM receives a text query as input and produces text
output. For instance, when generating dialog responses, the query corresponds
to incomplete dialog utterances, and the model generates the response. As for
classification tasks, the query corresponds to the textual input, and the
model generates the class label.

Further, we fine-tune GPT-NeoX for multi-task instruction-following on several
task mixtures, using the Super-NaturalInstructions data set [34]. Each prompt
consists of a task description prefix and a small subset of training input-
output pairs, separated by [INPUT] and [OUTPUT] tags. Further, depending on
whether we annotate unlabeled text or generate entirely new examples, the
prompt ends with either [OUTPUT] or [INPUT], respectively.

We fine-tune all layers and use Adam for up to $5$ epochs with a cosine
schedule and a learning rate of $1e-5$. Our code relies upon the GPT-Neox
library [2], which utilizes DeepSpeed [28] to facilitate distributed training.
We use NVIDIA A100 GPUs for all experiments as provided by an internal compute
cluster.

An important inference hyper-parameter is the generation temperature, which
controls the LLM output randomness. We find that a low temperature of $0.1$
works best for annotations, while a high temperature of $0.8$ works best for
generations (due to higher diversity in the samples).

###  2.3 Student models

For text classification and generation tasks, we use RoBERTa-Large [25], and
BART-Large [22] models, respectively. We fine-tune their publicly available
checkpoints. We fine-tune up to $320$ epochs for TC tasks using Adam, a batch
size of $50$, and a learning rate of $1e-5$. We fine-tune with Adam for $5$
epochs for TG tasks, using a linear learning schedule, batch size $32$, and
learning rate $5e-5$. We report the results using the model with the best
validation loss.

##  3 Experiments

###  3.1 Datasets

Dataset | # Examples
---|---
Train | Dev | Test
SLURP [4] | 11514 | 2033 | 2974
RTE [32] | 2500 | 277 | 300
BoolQ [9] | 9427 | 3270 | 3245
MultiRC [21] | 5100 | 953 | 1800
PubMedQA [19] | 212300 | - | 1000
SGD [29] | 164982 | 10000 | 10000
WebNLG [14] | 35426 | 1667 | 1779

Table 1: Dataset Statistics.

We list the dataset sizes in Table 1. One can apply the fractions in later
tables to this table to calculate the number of fine-tuning samples for the
teacher LLM.

#### Text classification

We report classification accuracies across four classification tasks. _SLURP_
(Spoken Language Understanding Resource Package) [4] is a multi-domain dataset
for end-to-end spoken language understanding, consisting of single-turn user
interactions with a home assistant. _BoolQ_ [9] contains naturally occurring
yes/no questions about a provided paragraph. _RTE_ (Recognizing Textual
Entailment) [31] is a dataset for recognizing whether, given two texts, the
meaning of one is entailed (can be inferred) from the other. _MultiRC_ (Multi-
Sentence Reading Comprehension) [21] is a multi-sentence question-answering
dataset, where each input consists of a short paragraph and question, which
requires combining information from multiple sentences.

#### Natural language generation

We report Rouge-{1,2,L} scores [24] on both development and test sets across
two tasks. _Schema Guided Dialog_ (SGD) [29] includes task-oriented
conversations between a human and a virtual assistant. Here, we follow
Gehrmann et al. [15] and construct the task of generating an utterance given
an unfinished dialog as input. _WebNLG_ is a data-to-text [14] task, which
aims at generating a text that verbalizes the input triples grammatically
correct.

###  3.2 Main results

Dataset | Type | Data amount in % | Dev Acc | Test Acc
---|---|---|---|---
Original | DAFTT
SLURP [4] |  | 1% | 0 % | 42.25 | 43.95
$X,Y$ | 26 % | 54.57 | 54.25
$Y|X$ | 78 % | 76.14 | 76.09
| 5% | 0 % | 73.49 | 71.59
$X,Y$ | 43 % | 77.39 | 76.89
$Y|X$ | 78 % | 85.00 | 83.96
- | 10% | 0% | 80.12 | 80.04
$X,Y$ | 43 % | 82.59 | 81.91
$Y|X$ | 43 % | 86.13 | 86.48
- | 100 % | 0 % | 88.64 | 87.70
BoolQ [9] |  | 1% | 0 % | 62.84 | N/A
$X,Y$ | 31 % | 68.96
$Y|X$ | 44 % | 79.72
| 5% | 0 % | 62.97
$X,Y$ | 31 % | 66.02
$Y|X$ | 44 % | 80.09
- | 10% | 0 % | 68.29
$X,Y$ | 31 % | 77.22
$Y|X$ | 44 % | 81.93
- | 100 % | 0 % | 85.05
RTE [31] |  | 5% | 0 % | 60.65 | N/A
$X,Y$ | 80 % | 66.79
$Y|X$ | 80 % | 83.20
| 10% | 0 % | 65.43
$X,Y$ | 80 % | 69.68
$Y|X$ | 80 % | 83.75
- | 20% | 0 % | 74.73
$X,Y$ | 80 % | 76.84
$Y|X$ | 80 % | 85.20
- | 100 % | 0 % | 86.60
MultiRC [21] |  | 1% | 0 % | 57.50 | 8.18
$X,Y$ | 40 % | 63.40 | 15.11
$Y|X$ | 254 % | 71.46 | 24.24
| 5% | 0 % | 67.70 | 18.05
$X,Y$ | 40 % | 72.85 | 21.86
$Y|X$ | 254 % | 71.51 | 32.63
- | 10% | 0 % | 70.63 | 22.35
$X,Y$ | 40 % | 73.88 | 24.97
$Y|X$ | 254 % | 76.90 | 34.94
- | 100% | 0 % | 82.12 | 48.16

Figure 2: Text Classification.

Dataset | Type | Data amount in % | Validation scores | Test scores
---|---|---|---|---
Original | DAFTT | R-1 | R-2 | R-L | R-1 | R-2 | R-L
SGD [29] | - | 1% | 0 % | 21.78 | 9.29 | 20.09 | 21.91 | 9.40 | 20.06
$X,Y$ | 10% | 43.72 | 25.09 | 39.84 | 40.82 | 22.36 | 37.00
$Y|X$ | 47.44 | 28.19 | 43.47 | 43.37 | 24.49 | 39.58
- | 5% | 0 % | 33.17 | 16.86 | 30.32 | 30.92 | 14.86 | 28.09
$X,Y$ | 10% | 45.27 | 26.39 | 41.48 | 41.71 | 23.25 | 37.98
$Y|X$ | 48.99 | 29.87 | 45.02 | 44.60 | 25.76 | 40.66
- | 10% | 0 % | 35.48 | 18.48 | 32.47 | 33.40 | 16.40 | 30.37
$X,Y$ | 10% | 48.49 | 29.59 | 44.62 | 43.89 | 25.29 | 40.16
$Y|X$ | 49.31 | 30.15 | 45.32 | 44.89 | 25.94 | 40.98
- | 100% | 0 % | 57.62 | 39.64 | 53.63 | 50.23 | 32.03 | 46.30
WebNLG [14] | - | 1% | 0 % | 53.94 | 31.76 | 42.58 | 50.21 | 28.44 | 39.72
$X,Y$ | 10% | 72.99 | 47.16 | 56.06 | 69.63 | 42.99 | 52.98
$Y|X$ | 75.69 | 51.18 | 59.60 | 70.21 | 44.34 | 54.51
- | 5% | 0 % | 56.58 | 35.74 | 46.46 | 53.17 | 32.57 | 44.65
$X,Y$ | 10% | 76.57 | 51.23 | 60.12 | 71.63 | 44.38 | 54.37
$Y|X$ | 78.02 | 53.60 | 61.46 | 72.11 | 45.50 | 55.29
- | 10% | 0 % | 59.02 | 38.21 | 48.37 | 54.07 | 32.47 | 44.06
$X,Y$ | 10% | 77.72 | 53.56 | 61.60 | 72.02 | 45.20 | 54.71
$Y|X$ | 78.38 | 54.56 | 62.72 | 72.73 | 46.02 | 55.96
- | 100% | 0 % | 80.65 | 58.50 | 65.73 | 73.27 | 46.90 | 55.76

Figure 3: Natural Language Generation.

Figure 4: Comparison of Performances with and without DAFTT. We find that
DAFTT is most effective in settings with the smallest amount of initial data
despite the teacher LLM being fine-tuned on that small amount of data. For
reference, we list the performance of the student model when fine-tuned on
100% training data.

Figure 4 shows the main results. We calculate the data amounts relatively
w.r.t. the original training dataset size. For example, an entry with 1%
original and 10% synthetic data means we construct the augmented training
dataset by concatenating 1% of data randomly sampled from the original
training dataset and 10% synthetic data.

We observe that both data annotations and generations improve performance
across all scores. Interestingly, the smaller the initial training dataset,
the larger the gains. This is interesting because it implies that we do not
need many samples for the teacher’s fine-tuning phase to succeed.

###  3.3 Varying amount of synthetic data

To understand the relationship between the amount of synthetic data added and
its downstream performance improvements, we vary the amount of synthetic data
for the NLG tasks.

Dataset | Type | Data amount | Rouge-L
---|---|---|---
Original | DAFTT | Dev | Test
SGD | - | 1% | 0% | 20.09 | 20.06
$Y|X$ | 1% | 10% | 43.47 | 39.58
20% | 45.22 | 40.89
30% | 46.02 | 41.29
$X,Y$ | 1% | 10% | 39.84 | 37.00
20% | 40.92 | 37.62
30% | 42.27 | 39.30
- | 100% | 0% | 53.63 | 46.30
WebNLG | - | 1% | 0% | 42.58 | 39.72
$Y|X$ | 1% | 1% | 57.59 | 52.92
2% | 58.94 | 54.50
3% | 59.44 | 54.44
4% | 59.79 | 54.42
5% | 59.56 | 54.44
$X,Y$ | 1% | 1% | 57.09 | 52.81
2% | 57.00 | 52.29
3% | 57.08 | 52.64
4% | 56.92 | 52.72
5% | 56.92 | 52.76
- | 100% | 0% | 65.73 | 55.76

Figure 5: Different amounts of DAFTT data added to the student model training
set. While both the SGD and WebNLG tasks benefit from the synthetic data, we
observe diminishing returns.

Dataset | Type | Data amount | Rouge-L
---|---|---|---
Original | DAFTT | Dev | Test
SGD | - | 1% | 0% | 20.09 | 20.06
$Y|X$ | 1% | 10% | 43.47 | 39.58
$X,Y$ | 1% | 10% | 39.84 | 37.00
$Y|X$; $X,Y$ | 1% | 5% each | 42.46 | 39.18
WebNLG | - | 1% | 0% | 42.58 | 39.72
$Y|X$ | 1% | 10% | 59.60 | 54.51
$X,Y$ | 1% | 1% | 56.06 | 52.98
$Y|X$; $X,Y$ | 1% | 5% each | 59.41 | 54.17

Figure 6: Combining Annotation ($Y|X$) and Generation ($X,Y$) equally performs
almost as well as annotating the same amount, while the latter assumes access
to unlabeled instances, which can be difficult in practice.

Figure 6 shows the results of different augmentation amounts with 1% original
training data. For SGD, we observe that up until 30%, the downstream model
performance increases. However, for WebNLG, we observe diminishing returns and
even slightly worse performance as the amount of synthetic data increases.

###  3.4 Evaluating the teacher LLMs directly

We report the performances of the teacher models, ie., without generating
additional data points and fine-tuning a student model, in Figure 8.
Interestingly, the fine-tuned NeoX 20B model outperforms the fine-tuned
davinci-002 model with 175B parameters (fine-tuned with default hyper-
parameters as provided by OpenAI’s API).

###  3.5 Comparing Teacher LLMs NeoX-20B and GPT-3.5 175B

We investigate how effective the DAFTT framework is when combined with other
teacher models. We compare NeoX-20B with OpenAI’s GPT-3.5 175B [8] (called
davinci-002 in their API) in Figure 7. The fine-tuned NeoX model consistently
produces better training data as measured in the student LM’s downstream
performance on the tasks’ test sets. This is an interesting result, given the
difference in model size.

Model | Dev | Test
---|---|---
Rouge 1 | Rouge 2 | Rouge L | Rouge 1 | Rouge 2 | Rouge L
davinci-002 | 35.47 | 17.99 | 32.35 | 34.62 | 17.19 | 31.46
NeoX | 47.44 | 28.19 | 43.47 | 43.37 | 24.49 | 39.59
Figure 7: Comparing the Teacher LLMs: We report the student LM’s downstream performance after being fine-tuned on 11% of the original training dataset size, where 1% is the original data (also used to fine-tune the teacher LLMs) and 10% teacher-annotated examples. The task is SGD. Model | Dev | Test
---|---|---
Rouge 1 | Rouge 2 | Rouge L | Rouge 1 | Rouge 2 | Rouge L
davinci-002 | 30.71 | 14.38 | 28.06 | 30.54 | 14.14 | 27.70
NeoX | 38.38 | 20.84 | 35.44 | 37.78 | 20.13 | 34.58
Figure 8: Evaluating the Teacher LLMs on SGD after being fine-tuned with 1%
training data.

###  3.6 Combining annotation and generation

In Section 3.2, we observe that annotations of existing, unlabeled data yield
more significant gains than generating entire input-output pairs. In many
real-world settings, however, access to unlabeled data may still be more
costly than generating data points entirely de novo. Hence, we want to
investigate whether adding generated data points to annotated ones in an equal
amount can yield performances closer to a larger set of annotated ones while
potentially being much cheaper in practice.

In the setting of 1% original training data, we confirm this hypothesis in the
affirmative, as shown in Figure 6. When mixing both data sources with 5% each,
we yield almost the same performance as if we had annotated 10% of the whole
data set.

###  3.7 Qualitative Analysis of Generated Text

In Appendix A, we showcase both annotated instances as well as entirely
generated one. By manually inspecting and verifying them, we find that all
generated examples are coherent and syntactically correct. However, not all
instances are factually truthful, a phenomenon often referred to as
“hallucination”. A post-hoc factuality filtering process of the generated
examples could be an exciting avenue for future work [23].

##  4 Related Work

There is extensive literature on data augmentation for text data leveraging
large teacher models [5, 20]. However, most of these works do not fine-tune
the teacher LLM on limited data to improve the quality of the generated data.
Efrat & Levy [13] examine a model’s ability to follow natural language
instructions, including annotating unlabeled dataset examples. Schick &
Schütze [30] study synthetic data generation on semantic textual similarity
datasets without task-specific training examples. Similarly, other work has
focused on information retrieval [7], code generation [3], and reasoning tasks
[18].

Yoo et al. [36] propose to transfer knowledge from LLMs to student models by
generating synthetic examples and knowledge distillation using soft labels.
Wang et al. [33] explore GPT-3 as a low-cost data labeler to train other
models. For NLG and NLU tasks, they find that it costs 50% to 96% less to use
GPT-3 generated labels than human annotations. Similarly, Ding et al. [12]
evaluates GPT-3’s effectiveness as a data annotator on classification and
named entity recognition tasks.

Gunasekar et al. [16], Li et al. [23] follow a similar approach to ours, where
they augment pre-training data with synthetically generated documents,
including textbooks and exercises to train a comparatively small model, which
outperforms much bigger ones on coding tasks. In contrast, we consider the
fine-tuning phase and consider classification and natural language generation
tasks.

In spirit, our approach is similar to _knowledge distillation_ (KD) [17],
where one uses the teacher model’s output logits as targets when training the
student model. However, state-of-the-art LLMs are often served via cloud-based
commercial APIs, exposing only a truncated output distribution (e.g., top-5
tokens). In contrast, our synthetic data generation approach does not require
the teacher’s output distribution.

##  5 Conclusion, Limitations and Future Work

In this work, we have elucidated that fine-tuning teacher LLMs to both
annotate unlabeled instances and generate new data points can effectively
improve a downstream model’s performance. Our empirical investigations spanned
six tasks, four in classification and two pertaining to natural language
generation. A potential constraint of our approach is that fine-tuning a large
model necessitates significant resources. In future work, we aim to delve
deeper into quantifying the extend of fine-tuning requisite to steer the
teacher model towards producing high-quality synthetic data.

## References

  * [1] Open LLM Leaderboard - a Hugging Face Space by HuggingFaceH4.  URL https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard.
  * Andonian et al. [2021] Andonian, A., Biderman, S., Black, S., Gali, P., Gao, L., Hallahan, E., Levy-Kramer, J., Leahy, C., Nestler, L., Parker, K., Pieler, M., Purohit, S., Songz, T., Phil, W., and Weinbach, S.  GPT-NeoX: Large Scale Autoregressive Language Modeling in PyTorch, August 2021.  URL https://www.github.com/eleutherai/gpt-neox.
  * Azerbayev et al. [2022] Azerbayev, Z., Ni, A., Schoelkopf, H., and Radev, D.  Explicit knowledge transfer for weakly-supervised code generation, 2022.  URL https://arxiv.org/abs/2211.16740.
  * Bastianelli et al. [2020] Bastianelli, E., Vanzo, A., Swietojanski, P., and Rieser, V.  Slurp: A spoken language understanding resource package, 2020.  URL https://arxiv.org/abs/2011.13205.
  * Bayer et al. [2022] Bayer, M., Kaufhold, M.-A., and Reuter, C.  A survey on data augmentation for text classification.  _ACM Computing Surveys_ , 55(7):1–39, 2022.
  * Black et al. [2022] Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al.  Gpt-neox-20b: An open-source autoregressive language model.  _arXiv preprint arXiv:2204.06745_ , 2022.
  * Bonifacio et al. [2022] Bonifacio, L., Abonizio, H., Fadaee, M., and Nogueira, R.  Inpars: Data augmentation for information retrieval using large language models, 2022.  URL https://arxiv.org/abs/2202.05144.
  * Brown et al. [2020] Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Clark et al. [2019] Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K.  Boolq: Exploring the surprising difficulty of natural yes/no questions.  _arXiv preprint arXiv:1905.10044_ , 2019.
  * Dai & Le [2015] Dai, A. M. and Le, Q. V.  Semi-supervised sequence learning, 2015.  URL https://arxiv.org/abs/1511.01432.
  * Devlin et al. [2018] Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K.  Bert: Pre-training of deep bidirectional transformers for language understanding, 2018.  URL https://arxiv.org/abs/1810.04805.
  * Ding et al. [2022] Ding, B., Qin, C., Liu, L., Bing, L., Joty, S., and Li, B.  Is gpt-3 a good data annotator?, 2022.  URL https://arxiv.org/abs/2212.10450.
  * Efrat & Levy [2020] Efrat, A. and Levy, O.  The turking test: Can language models understand instructions?  _arXiv preprint arXiv:2010.11982_ , 2020.
  * Gardent et al. [2017] Gardent, C., Shimorina, A., Narayan, S., and Perez-Beltrachini, L.  Creating training corpora for nlg micro-planning.  In _55th annual meeting of the Association for Computational Linguistics (ACL)_ , 2017.
  * Gehrmann et al. [2021] Gehrmann, S., Adewumi, T., Aggarwal, K., Ammanamanchi, P. S., Anuoluwapo, A., Bosselut, A., Chandu, K. R., Clinciu, M., Das, D., Dhole, K. D., et al.  The gem benchmark: Natural language generation, its evaluation and metrics.  _arXiv preprint arXiv:2102.01672_ , 2021.
  * Gunasekar et al. [2023] Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Del Giorno, A., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G., Saarikivi, O., et al.  Textbooks are all you need.  _arXiv preprint arXiv:2306.11644_ , 2023.
  * Hinton et al. [2015] Hinton, G., Vinyals, O., Dean, J., et al.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ , 2(7), 2015.
  * Ho et al. [2022] Ho, N., Schmid, L., and Yun, S.-Y.  Large language models are reasoning teachers.  _arXiv preprint arXiv:2212.10071_ , 2022.
  * Jin et al. [2019] Jin, Q., Dhingra, B., Liu, Z., Cohen, W., and Lu, X.  PubMedQA: A dataset for biomedical research question answering.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pp. 2567–2577, Hong Kong, China, November 2019. Association for Computational Linguistics.  doi: 10.18653/v1/D19-1259.  URL https://aclanthology.org/D19-1259.
  * Kaddour et al. [2023] Kaddour, J., Harris, J., Mozes, M., Bradley, H., Raileanu, R., and McHardy, R.  Challenges and Applications of Large Language Models, July 2023.  URL http://arxiv.org/abs/2307.10169.  arXiv:2307.10169 [cs].
  * Khashabi et al. [2018] Khashabi, D., Chaturvedi, S., Roth, M., Upadhyay, S., and Roth, D.  Looking beyond the surface: A challenge set for reading comprehension over multiple sentences.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pp. 252–262, 2018.
  * Lewis et al. [2019] Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L.  Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  _arXiv preprint arXiv:1910.13461_ , 2019.
  * Li et al. [2023] Li, Y., Bubeck, S., Eldan, R., Del Giorno, A., Gunasekar, S., and Lee, Y. T.  Textbooks are all you need ii: phi-1.5 technical report.  _arXiv preprint arXiv:2309.05463_ , 2023.
  * Lin [2004] Lin, C.-Y.  Rouge: A package for automatic evaluation of summaries.  In _Text summarization branches out_ , pp. 74–81, 2004.
  * Liu et al. [2019] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ , 2019.
  * Meng et al. [2022] Meng, Y., Huang, J., Zhang, Y., and Han, J.  Generating training data with language models: Towards zero-shot language understanding.  In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), _Advances in Neural Information Processing Systems_ , 2022.  URL https://openreview.net/forum?id=4G1Sfp_1sz7.
  * Raffel et al. [2020] Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P. J., et al.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _J. Mach. Learn. Res._ , 21(140):1–67, 2020.
  * Rasley et al. [2020] Rasley, J., Rajbhandari, S., Ruwase, O., and He, Y.  Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters.  In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_, pp. 3505–3506, 2020.
  * Rastogi et al. [2019] Rastogi, A., Zang, X., Sunkara, S., Gupta, R., and Khaitan, P.  Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset, 2019.  URL https://arxiv.org/abs/1909.05855.
  * Schick & Schütze [2021] Schick, T. and Schütze, H.  Generating datasets with pretrained language models, 2021.  URL https://arxiv.org/abs/2104.07540.
  * Wang et al. [2018] Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pp. 353–355, Brussels, Belgium, November 2018. Association for Computational Linguistics.  doi: 10.18653/v1/W18-5446.  URL https://aclanthology.org/W18-5446.
  * Wang et al. [2019] Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S.  Superglue: A stickier benchmark for general-purpose language understanding systems.  _Advances in neural information processing systems_ , 32, 2019.
  * Wang et al. [2021] Wang, S., Liu, Y., Xu, Y., Zhu, C., and Zeng, M.  Want to reduce labeling cost? gpt-3 can help, 2021.  URL https://arxiv.org/abs/2108.13487.
  * Wang et al. [2022] Wang, Y., Mishra, S., Alipoormolabashi, P., Kordi, Y., Mirzaei, A., Arunkumar, A., Ashok, A., Dhanasekaran, A. S., Naik, A., Stap, D., Pathak, E., Karamanolakis, G., Lai, H. G., Purohit, I., Mondal, I., Anderson, J., Kuznia, K., Doshi, K., Patel, M., Pal, K. K., Moradshahi, M., Parmar, M., Purohit, M., Varshney, N., Kaza, P. R., Verma, P., Puri, R. S., Karia, R., Sampat, S. K., Doshi, S., Mishra, S., Reddy, S., Patro, S., Dixit, T., Shen, X., Baral, C., Choi, Y., Smith, N. A., Hajishirzi, H., and Khashabi, D.  Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks, 2022.
  * Wei et al. [2022] Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al.  Emergent abilities of large language models.  _arXiv preprint arXiv:2206.07682_ , 2022.
  * Yoo et al. [2021] Yoo, K. M., Park, D., Kang, J., Lee, S.-W., and Park, W.  Gpt3mix: Leveraging large-scale language models for text augmentation.  _arXiv preprint arXiv:2104.08826_ , 2021.
  * Zhang et al. [2021] Zhang, T., Wu, F., Katiyar, A., Weinberger, K. Q., and Artzi, Y.  Revisiting Few-sample BERT Fine-tuning, March 2021.  URL http://arxiv.org/abs/2006.05987.  arXiv:2006.05987 [cs].

##  Appendix A Exemplary Generations

Examples of annotations for WebNLG [14]. Ground-truth references are shown in
the {braces}.
---
Input: [CONTEXT] WrittenWork [DATA] A_Glastonbury_Romance | mediaType | Hardcover A_Glastonbury_Romance | numberOfPages | 1174 | isbnNumber | 0-7156-3648-0
Output: A Glastonbury Romance, with 1144 pages, is available in print and is
ISBN number 0-7156-3648.
{A Glastonbury Romance has the ISBN number 0-7156-3648-0, can be found in
hardcover and has 1174 pages.}
Input: [CONTEXT] MeanOfTransportation [DATA] A-Rosa_Luna | builder | Germany MTU_Friedrichshafen | city | Friedrichshafen A-Rosa_Luna | powerType | MTU_Friedrichshafen
Output: A-Rosa Luna was designed by MTU Friedrichshafen and is powered by
them. The city of Friedrichshafen is where the builder of the A-Rosa Luna is
located.
{The A-Rosa Luna is powered by a MTU Friedrichshafen engine in the city of
Friedrichshafen, Germany.}
Input: [CONTEXT] Astronaut [DATA] Alan_Bean | nationality | United_States Alan_Bean | occupation | Test_pilot Alan_Bean | almaMater | UT Austin, B.S. 1955 Alan_Bean | mission | Apollo_12 Alan_Bean | birthPlace | Wheeler,_Texas
Output: Alan Bean was a national of the United States, who was a student at
the University of Texas at Austin (B.S. 1955). He was an American test pilot
and was a member of the Apollo 12 crew. Mr. Bean was born in Wheeler, Texas.
{Alan Bean is a United States citizen born in Wheeler Texas. He served as a
test pilot and was a crew member of the Apollo 12 mission.}

Examples of generations for WebNLG [14].
---
Input: [CONTEXT] Astronaut [DATA] Elliot_See | almaMater | University_of_Texas_at_Austin University_of_Texas_at_Austin | affiliation | University_of_Texas_System Elliot_See | occupation | Test_pilot Elliot_See | nationality | United_States Elliot_See | birthPlace | Dallas Elliot_See | deathPlace | St._Louis Elliot_See | status | Deceased
Output: Elliot See was a US national, who was born in Dallas. He was a student
at the University of Texas at Austin (affiliated to the university of Texas
system). He was a test pilot who died in St. Louis.
Input: [CONTEXT] Airport [DATA] Al_Asad_Airbase | operatingOrganisation | United_States_Air_Force United_States_Air_Force | transportAircraft | Lockheed_C-130_Hercules
Output: Al Asad Airbase is operated by the United States Air Force which use
the Lockheed C-130 Hercules as a transport aircraft.
Input: [CONTEXT] Food [DATA] Ajoblanco | ingredient | Almond Spain | leader | Felipe_VI_of_Spain Ajoblanco | region | Andalusia Andalusia | leader | Susana_D0̆0edaz Ajoblanco | country | Spain
Output: Ajoblanco originates from Spain where Felipe VI is leader. Ajoblanco
is found in Andalusia where Susana Diaz is leader. One of the ingredients of
ajoblanco is almonds.

Examples of annotations for SGD [29]. Ground-truth references are shown in the
{braces}.
---
Input: [CONTEXT] User: I want to make the most of the free time I have coming
up by finding something fun to do. Let’s say I have no idea what to do but my
trusty friend, that’s you, will help me. Am I right?
Output: I’ll be happy to help. What type of events do you have in mind?
{Sure, I’ll be happy to help you with that. Let’s start with this: tell me
what type of activity and the area you’re interested in.}
Input: [CONTEXT] User: Can you find me oneway flights?
System: Where will you be flying from?
User: I am flying out of LA.
System: When do you want to leave?
User: I need to leave on the 10th of this month.
System: Where will you be flying too.
User: I am flying to Paris, France with 4 people. I prefer Delta Airlines.
Output: Please confirm the following: You will need 1 Economy ticket on Delta
Airlines. You will be flying from Los Angeles to Paris on March 10th. You will
be traveling with 4 people.
{There is a direct Delta Airlines flight that leaves at 8:25 am and has 0
stops for $255.}
Input: [CONTEXT] User: I’d like to find a rental car to pick up in New York.
System: When will you need it?
User: I would like to pick it up at 12 in the afternoon.
System: What date would you like to return the car?
User: I’d like to return it on the 9th of March.
System: What date would you like to pick it up?
User: I want to pick it up Friday next week.
System: I found a standard size Honda Accord for you that’s available at Grand
Central Station on March 8th, does that work?
User: Is there anything else available? I would like a compact size and will
actually need it until March 10th.
System: Sure, I found a compact size Chevrolet Bolt that is available at Port
Authority Station on March 8th, does that work for you?
User: What’s the cost for the car?
System: The total cost is $56.
User: Is there anything available in Paris, France?
Output: There are 10 cars available. How about a compact size Chevrolet Cruze?
It’s available to pick up from Charles de Gaulle International Airport on
March 8th.
{Yes, there is a compact size Honda Fit available at the Charles de Gaulle
International Airport on March 8th.}

Examples of generations for SGD [29].
---
Input: [CONTEXT] User: I want to eat at a place in San Francisco.
System: What type of food? You can say Indian, Italian, Vegetarian, or
something else.
User: Small plates would be perfect.
System: Aatxe has small plates and is in San Francisco.
User: What’s the address and is their live music?
System: Their address is 2174 Market Street and they don’t have live music.
User: What else is available? Check Redwood City.
System: La Viga Seafood & Cocina Mexicana is in Redwood City.
User: Yes, that will work.
System: Do you want to book a table?
User: Not now. Can you find a salon in Redwood City?
System: August Barbershop is in Redwood City.
User: Yes, I’ve heard of them.
System: Shall I make an appointment.
User: Please do, book it for half past 2 in the afternoon.
System: What date do you want it for?
User: The 4th of March and book it for two pm.
System: Please confirm the following: Booking an appointment with August
Barbershop for March 4th at 2 pm.
User: No, do it for the 1st of March.
System: Please confirm the following: Booking an appointment with August
Barbershop for March 1st.
User: Yes, that’s correct.
System: Your appointment went through successfully!
User: Could you try booking a table at that La Viga place?
System: What time is it for?
User: It’s for half past 6 in the evening.
System: Please confirm the following: Booking a table for 2 at La Viga Seafood
& Cocina Mexicana in Redwood City on March 1st at 6:30 pm.
User: Yes, that’s correct.
System: Your appointment went through successfully!
User: Could you try booking a table for 4 on the 7th of this month?
System: Please confirm the following: Booking a table for 4 at La Viga Seafood
& Cocina Mexicana in Redwood City on March 7th.
User: Yes, that’s correct.
System: Your appointment went through successfully!
User: Thanks a lot.
Output: Do you need anything else?
Input: [CONTEXT] User: I’m looking for some events.
System: What city should I look in?
User: I want to see something in New York City.
System: What kind of events are you looking for?
User: I want to see Carmina Burana.
System: I found 2 occurrences of the event Carmina Burana. The first is at the
Sony Hall on March 5th.
User: What is the address of the venue?
System: The venue is located at 235 West 46th Street.
User: What is the venue’s address?
System: The address is 235 West 46th Street.
User: That sounds great.
System: Do you want to purchase tickets?
User: No, I don’t want to buy tickets right now.
System: Is there anything else I can help you with?
User: I want to buy bus tickets to get there.
System: How many tickets do you need? What time do you want to depart?
User: I need Economy tickets for 4 people. I want to leave at one pm.
System: Where will you be departing from?
User: I’m leaving from Washington.
System: Okay, I have 4 Economy bus tickets leaving from Washington and going
to New York on March 5th at 1 pm. Is that correct?
User: Yes, that sounds good. What station does the bus depart from and what
station does it arrive at?
Output: I’ve purchased the tickets. The bus leaves from Union Station and
arrives at Port Authority Station.
Input: [CONTEXT] I’m looking for a rental car.
System: What day and time do you want to pick it up? How long will you be
needing it?
User: I need it at three pm on the 8th of March, until the 11th of March.
System: Where do you want to pick it up?
User: I want to get it in Anaheim, CA.
System: I have 5 cars you might like. How about a Standard Chevrolet Cruze?
It’s available at John Wayne Airport on March 8th.
User: What else do you have?
System: I have a Standard Honda Accord at John Wayne Airport on March 8th.
User: What else do you have?
System: I have a Standard Toyota Camry at Anaheim Intermodal Center on March
8th.
User: That one sounds good.
System: Do you want to reserve it?
User: Yes, please reserve it.
System: Please confirm the following: You are renting a Standard car from
Anaheim Intermodal Center on March 8th at 3 pm. You’ll return it on March
11th.
User: Actually, I want to keep it until March 12th.
System: Please confirm the following details: You are keeping the car until
March 12th.
User: That’s correct. How much does it cost?
System: Your car has been reserved. The total cost is $170.
User: I want to find a movie near there.
System: Do you want to look for movies showing in Anaheim?
User: Yes, that’s what I want.
System: How about some movies by Cameron Crowe? There’s also dogman, which is
a drama.
User: Is there anything else? I want something more family-friendly.
System: There’s also Family, which is a comedy.
User: Family sounds good.
System: Do you want to buy tickets?
User: No, not right now.
Output: Is there anything else you need?