250

# Agent Instructs Large Language Models to be General Zero-Shot Reasoners

Nicholas Crispino1, Kyle Montgomery1, Fankun Zeng1, Dawn Song2, Chenguang
Wang1∗
1Washington University in St. Louis, 2UC Berkeley
{ncrispino, kylemontgomery, fankun<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

We introduce a method to improve the zero-shot reasoning abilities of large
language models on general language understanding tasks. Specifically, we
build an autonomous agent to instruct the reasoning process of large language
models. We show this approach further unleashes the zero-shot reasoning
abilities of large language models to more tasks. We study the performance of
our method on a wide set of datasets spanning generation, classification, and
reasoning. We show that our method generalizes to most tasks and obtains
state-of-the-art zero-shot performance on 20 of the 29 datasets that we
evaluate. For instance, our method boosts the performance of state-of-the-art
large language models by a large margin, including Vicuna-13b (13.3%),
Llama-2-70b-chat (23.2%), and GPT-3.5 Turbo (17.0%). Compared to zero-shot
chain of thought, our improvement in reasoning is striking, with an average
increase of 10.5%. With our method, Llama-2-70b-chat outperforms zero-shot
GPT-3.5 Turbo by 10.2%.

${}^{*}$${}^{*}$footnotetext: Corresponding
author.$\color[rgb]{1,1,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,1,1}\pgfsys@color@gray@stroke{1}\pgfsys@color@gray@fill{1}^{*}$$\color[rgb]{1,1,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,1,1}\pgfsys@color@gray@stroke{1}\pgfsys@color@gray@fill{1}^{*}$footnotetext:
The code is available at https://github.com/wang-research-lab/agentinstruct.

##  1 Introduction

Figure 1: Summary of our approach and results. Top: Our zero-shot
AgentInstruct generalizes the zero-shot reasoning abilities of large language
models to a wide set of language understanding tasks including generation,
classification, and reasoning. Our agent produces task-specific instructions
to instruct the reasoning process of large language models to solve a task.
Both the agent instructions and task-specific reasoning process are
highlighted. Bottom: Performance of zero-shot AgentInstruct compared with
standard zero-shot and zero-shot chain of thought (CoT). Zero-shot
AgentInstruct improves the performance of three large language models
substantially on the 29 datasets we evaluate.

Large language models (LLMs) (Brown et al., 2020; Wang & Komatsuzaki, 2021;
Zhang et al., 2022a; Smith et al., 2022; Chowdhery et al., 2022; Hoffmann et
al., 2022; Scao et al., 2023; OpenAI, 2023; Anil et al., 2023; Touvron et al.,
2023a; b; Penedo et al., 2023) have significantly advanced the state-of-the-
art on a wide range of language understanding tasks. These models have led to
widespread deployment and adoption in applications (Araci, 2019; Huang et al.,
2020; Bolton et al., 2022; Wu et al., 2023; Driess et al., 2023; Huang et al.,
2023). In particular, the emerging capabilities of LLMs such as complex
reasoning (Wei et al., 2022b; Wang et al., 2023c; Kıcıman et al., 2023) have
made them the subject of research in recent years. Among these, zero-shot
reasoning (Kojima et al., 2022; Wan et al., 2023) has drawn substantial public
interest and achieved promising results in specific task domains. However, the
reasoning ability of LLMs on general tasks remains unclear.

In this paper, we improve the zero-shot reasoning abilities of LLMs on general
language understanding tasks. To solve a task, we build an agent to instruct
the reasoning process of LLMs for the task (Figure 1). More specifically, our
autonomous agent generates task-specific instructions to better align the
chain of thought reasoning process of LLMs with each task. We refer to this
approach as zero-shot agent instructed reasoning (AgentInstruct). The basic
idea of our approach is motivated by two lines of work. First, language agents
(Yao et al., 2023b; Shinn et al., 2023; Park et al., 2023; Wang et al., 2023a;
Xi et al., 2023) have been developed to automatically complete a task. Instead
of completing the task, our agent produces instructions on how to complete the
task. We enable this by adapting an existing agent to access a wide range of
task-relevant knowledge on the web, given basic task information such as the
name of the dataset and several input examples. As a result, the agent
synthesizes high-quality step-by-step instructions for tasks verified by the
web resources. We follow the recent design of agents where an LLM is used to
produce the plans for this process. Second, zero-shot chain of thought (CoT)
reasoning of LLMs has obtained promising results on tasks such as arithmetic
reasoning (Kojima et al., 2022; Wang et al., 2023b). Standard in-context zero-
shot learning prompts an LLM to directly output predictions without task
examples. In contrast, CoT decomposes a task into intermediate steps and
solving each will lead to the final output. We further align the CoT reasoning
steps with a particular task by prompting with the task-specific agent
instructions. The design of zero-shot AgentInstruct is important: We
generalize the zero-shot reasoning abilities of LLMs to more tasks with the
combination of task-specific instructions from a language agent and task-
specific reasoning of LLMs.

We empirically evaluate the zero-shot reasoning abilities of LLMs on a wide
set of language understanding tasks across 29 datasets (including 53 subsets),
spanning generation, classification, and reasoning. Zero-shot AgentInstruct
obtains state-of-the-art performance on 20 datasets. We conduct our evaluation
on three state-of-the-art LLMs, namely, Vicuna (Chiang et al., 2023),
Llama-2-chat (Touvron et al., 2023b), and GPT-3.5 Turbo (OpenAI, 2022). We
show that zero-shot AgentInstruct boosts the performance of these models by
17.8% on average. When compared to zero-shot CoT, the overall performance
improvement is significant (6.5%), and in particular, the improvement in
reasoning is substantial with an average increase of 10.5%, leading to the
best performance on 10 out of 12 reasoning tasks. Notably, Llama-2-70b-chat
with zero-shot AgentInstruct outperforms standard zero-shot GPT-3.5 Turbo by
an average of 10.2%. We hope the results help foster future research on
further unlocking the zero-shot reasoning abilities of large foundation models
and exploring the broader usage of agents.

##  2 Approach

We present zero-shot AgentInstruct in this section. To solve a task, zero-shot
AgentInstruct employs an agent to instruct an LLM to reason toward the final
prediction. Intuitively, humans often rely on specific instructions to more
effectively guide their thought process as they work towards a solution to a
problem. For instance, to understand the sentiment in the movie reviews,
instructions such as “1\. Understand the Dataset: … Movie Reviews dataset … 2.
Analyze the Passage: Pay attention to … the tone of the review …” help humans
decompose the problem into task-specific reasoning steps and solve each to
deliver the final answer (Figure 1). Zero-shot AgentInstruct follows this
intuition.

Figure 2: An example of our agent producing instructions for a classification
dataset IMDB. The task-specific instructions are highlighted. Agent
instructions for all datasets are in Appendix D.

##### Agent Instructions

Instead of handwriting task-specific instructions, we build an agent to
automate the process. The intuition is that an agent is able to synthesize
high-quality instructions with access to a wide range of existing task
knowledge on the web. We design our agent based on ReAct (Yao et al., 2023b).
This is motivated by the recent developments of language agents for task
solving. Our agent highlights two features (Figure 2): (i) Instruction
generation. Our agent follows ReAct which uses an LLM to propose a series of
thoughts. The agent then receives observations and takes actions following the
thoughts. Different from ReAct which aims to directly solve the task, our
agent outputs step-by-step instructions on how to solve the task. We use GPT-4
(OpenAI, 2023) as the default agent. Once the action is finish, the
corresponding output is our task-specific instructions. (ii) Action space. We
design a question answering API with two types of actions to support the
instruction generation: (a) ask_about_dataset[string], which returns the top
relevant web pages containing the information about the dataset from the API.
(b) finish[instructions], which finishes the instruction generation with the
task-specific instructions. As shown in Figure 2, to produce the instructions,
our agent takes basic dataset information such as the name of the dataset
(e.g., IMDB) and a few input-only examples as its input. Using the task
knowledge from the web, our agent forms observations (e.g., “Observation 1: …
labeled as either positive or negative …”) and thoughts (e.g., “Thought 2: …
creating instructions …”) which trigger the agent to perform actions, such as
the finish action to output the task-specific instructions.

##### Chain of Thought Reasoning

Chain of thought (CoT) (Wei et al., 2022b; Kojima et al., 2022) prompts LLMs
to break down the task into intermediate reasoning steps that lead to the
final answer. Unlike zero-shot CoT which uses a fixed prompt “Let’s think step
by step”, we prepend our task-specific agent instructions to the input to
prompt the LLMs to optimize their reasoning processes for the task. LLMs will
then follow our task-specific instructions to decompose the task into a chain
of more specific intermediate steps to solve the task. As shown in Figure 1,
the agent instructions “… Pay attention to … explicit or implicit expressions
of sentiment towards the movie …” are the key to producing the critical
reasoning path “… the movie is worth a view only for the performances of the
three actors …”, which leads to the correct prediction where standard zero-
shot and zero-shot CoT fail. We follow zero-shot CoT, which consists of a
reasoning extraction prompt to produce the intermediate reasoning steps, and
an answer extraction prompt to collect the answers. For simplicity of
implementation, we replace zero-shot CoT’s fixed prompt with zero-shot
AgentInstruct’s task-specific instructions in the reasoning extraction prompt.

Zero-shot AgentInstruct enjoys several unique properties: (i) Zero-shot
AgentInstruct is a new way to improve the zero-shot reasoning of LLMs. Zero-
shot AgentInstruct decouples the language agent and reasoning process of LLMs,
which helps zero-shot AgentInstruct generalize to more tasks. Additionally,
the agent instructions provide more task-specific controls to the reasoning
paths of LLMs, which benefits human alignment and improves the safety of LLMs.
(ii) Our agent instructions are customized for different tasks and verified by
existing task knowledge. For each task, different LLMs use the same set of
instructions, and we find the instructions transfer well among these models.
This is important in practice as the agent LLMs are often more powerful and
costly than the reasoning LLMs, so our approach is a cost-effective
alternative to using agents directly. (iii) By providing task-specific
instructions, chain of thought reasoning abilities are further generalized to
more tasks beyond reasoning tasks. We show general language understanding
tasks such as generation and classification also benefit from chain of thought
reasoning. To better embrace the emerging reasoning capabilities of LLMs, our
task-specific instructions align the reasoning process with a particular task
better than general or fixed instructions.

##  3 Experiment

We show that zero-shot AgentInstruct successfully improves the zero-shot
reasoning abilities of LLMs, namely, Vicuna (Chiang et al., 2023),
Llama-2-chat (Touvron et al., 2023b), and GPT-3.5 Turbo (OpenAI, 2022), by a
large margin on average. We evaluate zero-shot AgentInstruct on an exhaustive
selection of 29 benchmarking datasets containing 53 subsets. As shown in
Figure 4, each dataset is either a generation or classification task and a
portion of the datasets in each category are also reasoning tasks. The
datasets consist of all HELM core scenarios from Liang et al. (2023), as well
as the reasoning datasets from Kojima et al. (2022). The details of the
experimental setup, including datasets and models, are described in Appendix
A.

###  3.1 Main Results

Results are shown in Figure 5. We compare zero-shot AgentInstruct to standard
zero-shot and zero-shot CoT. We focus our analysis on three models:
Vicuna-13b, Llama-2-70b-chat, and GPT-3.5 Turbo.

Figure 3: Datasets for generation (blue), classification (green), and
reasoning (orange). Reasoning contains generation and classification tasks.

((a)) All datasets.

((b)) Generation datasets.

((c)) Classification datasets.

((d)) Reasoning datasets.

Figure 4: Winning rate (%) between zero-shot, zero-shot CoT, and zero-shot
AgentInstruct based on the average results over three models.

We first compare zero-shot AgentInstruct to standard zero-shot prompting
(Figure 1). The zero-shot prompt design follows Liang et al. (2023). On each
model, zero-shot AgentInstruct wins on the majority of datasets, with no less
than a 13.0% increase on average. Average performance versus the zero-shot
setting is best on Llama-2-70b-chat with a 23.2% improvement. Figure 5(a) and
Figure 5(b) show the results for generation and classification tasks
respectively. On average, with zero-shot AgentInstruct, the three models beat
the zero-shot setup by 23.1% for generation and 13.5% for classification. We
hypothesize that generation datasets generally require more specific
instructions than classification datasets, as the model does not know the best
format of the generation output unless it has sufficient task information.
This shows that our agent is able to instruct the reasoning process to improve
the final outputs for different tasks, and zero-shot AgentInstruct is able to
generalize LLMs’ reasoning abilities across tasks. With zero-shot
AgentInstruct, we also observe a large margin of improvement for zero-shot
performance across different models. Significantly, Llama-2-70b-chat beats the
performance of zero-shot GPT-3.5 Turbo by 10.2% on average across all
datasets. This indicates our agent instructions are the key to improving the
reasoning performance of LLMs.

The most immediate comparison to zero-shot AgentInstruct is zero-shot CoT, as
zero-shot AgentInstruct uses task-specific instructions instead of a fixed
manual instruction. On average, across all three models, zero-shot
AgentInstruct beats zero-shot CoT by 6.5%, with the largest growth being
Vicuna-13b at 9.5%. On both generation and classification datasets, across
three models, zero-shot AgentInstruct wins by 5.0% and 7.8% on each category
respectively. This suggests that zero-shot AgentInstruct is able to generalize
the zero-shot reasoning abilities of LLMs to both generation and
classification tasks, and optimize the performance of specific tasks.

In particular, we look into the performance of reasoning tasks (Figure 5(c)).
Of our three models, the average difference between zero-shot AgentInstruct
and the zero-shot setting on reasoning tasks is 31.3%, whereas the difference
between zero-shot AgentInstruct and zero-shot CoT is 10.5%. This shows that
our task-specific instructions are more helpful for LLMs to break down tasks
into more specific intermediate reasoning steps compared to the task-agnostic
instructions in zero-shot CoT, which leads to improved final predictions.

Overall, zero-shot AgentInstruct wins on 9 of the 13 generation datasets, 11
of the 16 classification datasets, and 10 of the 12 reasoning datasets (Figure
4). See Appendix B for additional results and analysis, including results on
individual datasets and subsets.

((a)) Generation datasets.

((b)) Classification datasets.

((c)) Reasoning datasets.

Figure 5: Results on Vicuna-13b, Llama-2-70b-chat, and GPT-3.5 Turbo across
tasks. Top: generation. Middle: classification. Bottom: reasoning.

###  3.2 Ablation Study

| AddSub | IMDB | NarrativeQA
---|---|---|---
Zero-Shot AgentInstruct | 79.5 | 94.0 | 65.0
w/o Agent Instructions | 73.2 | 89.0 | 62.3
w/o Input Examples | 72.4 | 88.0 | 60.1
w/o Labels | 74.9 | 93.8 | 63.9
w/o GPT-4 | 75.2 | 92.6 | 63.5

Table 1: Ablation over different facets of zero-shot AgentInstruct with
Llama-2-70b-chat.

Figure 6: Comparison on GPT-4 using zero-shot, zero-shot CoT, ReAct, and zero-
shot AgentInstruct on AddSub.

We examine how different components of zero-shot AgentInstruct impact its
zero-shot reasoning performance. Results are shown in Table 1 on AddSub
(reasoning), IMDB (classification), and NarrativeQA (generation). We use
Llama-2-70b-chat for reasoning. The four settings examine the importance of
agent instructions in zero-shot AgentInstruct. Descriptions of each setting
are as follows: (i) w/o Agent Instructions: We compare the zero-shot
AgentInstruct methodology to zero-shot CoT. (ii) w/o Input Examples: We remove
the input-only examples from the input to the agent. (iii) w/o Labels: We
remove the description of the labels from the input to the agent. (iv) w/o
GPT-4: We use GPT-3.5 Turbo, instead of GPT-4, as the agent to generate
instructions. The results suggest that all components of zero-shot
AgentInstruct are effective in providing high-quality instructions and
eliciting high-quality reasoning steps before making predictions.

Next, we focus our analysis on GPT-4. We test the following methods: zero-
shot, zero-shot CoT, ReAct (Yao et al., 2023b), and zero-shot AgentInstruct.
Figure 6 shows the performance of each method on AddSub. Zero-shot
AgentInstruct outperforms zero-shot GPT-4 by 8.6% and ties the performance of
zero-shot CoT GPT-4 for approximately one-tenth of the cost. Zero-shot
AgentInstruct is using GPT-3.5 Turbo for the CoT reasoning. This indicates
that both task-specific instructions and CoT reasoning help improve the zero-
shot performance of LLMs. Though ReAct narrowly outperforms zero-shot
AgentInstruct, it costs nearly 100 times more. This result implies that
decoupling the agent instruction generation and reasoning steps further
unleashes the zero-shot reasoning abilities of LLMs, and that zero-shot
AgentInstruct is a cost-effective alternative to using agents directly,
resulting in only a minimal performance loss.

###  3.3 Truncated Context Length

In Figure 8, we test how zero-shot AgentInstruct performs on various context
lengths. We artificially reduce the context length of Llama-2-70b-chat from
4,000 (maximum context length) to 2,048, 1,024, 768, and 640. The results
suggest that the performance of zero-shot AgentInstruct is worse for models
with smaller context lengths. While the impact is minimal on AddSub and IMDB,
performance on NarrativeQA steeply declines below a context length of 2,048,
which is due to truncation. This is because the instances of NarrativeQA are
much longer, with an average length of 855 tokens compared to the instances of
AddSub and IMDB, with average lengths of 330 tokens and 48 tokens
respectively.

Figure 7: Truncating context lengths on Llama-2-70b-chat with zero-shot
AgentInstruct on AddSub, IMDB, and NarrativeQA.

Figure 8: Model scaling results of zero-shot, zero-shot CoT, and zero-shot
AgentInstruct with Llama-2-chat on all datasets.

###  3.4 Model Scaling

As it is often the case that larger models substantially improve the reasoning
capabilities of LLMs (Wei et al., 2022b), we test zero-shot AgentInstruct’s
performance on models of various sizes. Specifically, we test on three
Llama-2-chat models with 7 billion, 13 billion, and 70 billion parameters.
Figure 8 shows the average score across all 29 datasets for zero-shot, zero-
shot CoT, and zero-shot AgentInstruct. These results confirm that the average
performance of all three methods increases with model size. Each time model
size increases, zero-shot CoT and zero-shot AgentInstruct show consistent
gains of around 6%, while zero-shot has smaller gains near 2%. This is because
reasoning steps are best produced with more powerful models. In fact, at just
13b parameters, Llama-2-13b-chat with zero-shot AgentInstruct surpasses the
performance of zero-shot GPT-3.5 Turbo by over 2%. Additionally, zero-shot
AgentInstruct’s superiority over zero-shot and zero-shot CoT appears
independent of model size.

CoT Reasoning |  Prompt |  Quasi-Exact Match (%)
---|---|---
Reasoning extraction |  Follow the instructions to generate an explanation that reasons towards the correct answer to the task above. End the explanation with the correct answer.\n\nExplanation: |  79.5
|  Let’s think step-by-step according to the instructions. End with the correct answer. Explanation: |  77.5
|  Let’s think step-by-step according to the instructions. First, |  75.2
|  Use the instructions to guide you towards your answer.\nExplanation: |  79.0
|  Explanation: |  78.5
Answer extraction |  Therefore, the answer to the task is below. Give the answer in the shortest form possible that will still be correct. \nAnswer: |  79.5
|  Therefore, the answer to the task is below.\nAnswer: |  79.7
|  Therefore, the answer is |  79.2
|  Answer: |  77.7

Table 2: Prompt sensitivity analysis of chain of thought reasoning of zero-
shot AgentInstruct with Llama-2-70b-chat on AddSub. The default prompts are
highlighted. Higher scores are better.

###  3.5 Manual Prompt Sensitivity

Zero-shot AgentInstruct has two manual prompts in the CoT reasoning step: (1)
the reasoning extraction prompt, which asks for intermediate reasoning steps,
and (2) the answer extraction prompt, which collects the final answer. To test
the sensitivity of each prompt, we vary a single prompt while keeping the
default zero-shot AgentInstruct prompt for the other. Results are shown in
Table 2 based on Llama-2-70b-chat on AddSub. Overall, zero-shot
AgentInstruct’s performance does not appear particularly sensitive to changes
in the manual prompts, suggesting that the methodology behind zero-shot
AgentInstruct is robust. Additional prompt sensitivity experiments are
conducted in Appendix C.2.

###  3.6 Error Analysis

| Error Type | Percentage
---|---|---
Reasoning | Incorrect reasoning | 32.0
| Not factual | 12.0
Answer | Ambiguity | 22.7
| Invalid label | 14.7
| Short answer | 10.6
| Incorrect format | 8.0

Table 3: Error analysis for zero-shot AgentInstruct Llama-2-70b-chat on
AddSub, IMDB, and NewsQA.

To investigate the reasons for errors made by zero-shot AgentInstruct, we
select 25 samples from AddSub, IMDB, and NewsQA respectively where zero-shot
AgentInstruct results in incorrect predictions on Llama-2-70b-chat. We define
incorrect predictions as those with a quasi-exact match or F1 score less than
1. Table 3 shows our error analysis. The most common error across datasets is
incorrect reasoning, i.e., not correctly reasoning through the problem when
applying the accurate agent instructions. For example, on AddSub, zero-shot
AgentInstruct chooses the wrong operation due to a misleading verb. On IMDB,
zero-shot AgentInstruct misreads the sentiment due to emphasizing words
describing the movie, not the review. See Figure 10 for an example of
incorrect reasoning on IMDB. The answer ambiguity is another main source of
errors. For example, in a review where the reviewer clearly enjoyed the movie
even though the reviewer acknowledged it was a typical bad movie, our
prediction is “Positive” while the ground truth is “Negative” on IMDB. For
many errors, either the instructions are taken too literally or partially
ignored. As larger models become better at reasoning, such errors should be
minimized. More thorough error analysis and full examples for each error
category are in Appendix C.1.

    Passage: As an avid Disney fan, I was not totally impressed by this movie, certainly not motivated to catch it in the theaters.
    I am, however, so glad that I caught it on DVD and watched the special features.
    You MUST check out the ‘‘Moose commentary’’
    The enjoyment I got from this commentary completely made up for the tepid reaction I had to the film itself.

    CoT Reasoning:
    Based on the language used in the passage, it is clear that the reviewer has a positive sentiment towards the movie. The reviewer uses positive adjectives such as ‘‘enjoyment’’ to describe their experience with the movie...

    Answer: Positive ✗

Figure 9: An incorrect reasoning example for Llama-2-70b-chat with zero-shot
AgentInstruct on IMDB for error analysis. Here, the model mistook the love of
the commentary as the love of the movie (highlighted).

    Passage: This is a good movie. Terrance Stamp is great, the music is sweet, Carol White is very believeable. The only thing that marred this was the shakey acting of Carol’s first husband, but if you can get past that, you’re OK. Donovan provides some of the most languid, mellow, bittersweet lyrics from the 60s.

    CoT Reasoning:
    The reviewer mentions that the movie is ‘‘good’’ and that it has a ‘‘sweet’’ soundtrack. They also mention that the acting by Carol White is ‘‘believeable’’. However, they also mention that the acting by Carol’s first husband is ‘‘shakey.’’

    Answer: Positive ✓

Figure 10: Case study example for Llama-2-70b-chat with zero-shot
AgentInstruct on IMDB. Here, the answer is correct and the task-specific
reasoning is helpful for finding the answer (highlighted).

###  3.7 Case Study

Next, we analyze the quality of the CoT reasoning steps when predictions are
correct. On three datasets (AddSub, IMDB, and NewsQA), we randomly select 25
examples from each dataset with a perfect quasi-exact match or F1 score. We
find that the reasoning capabilities are further enhanced by the combination
of effective agent instructions and the task-specific reasoning process of
LLMs. An example is in Figure 10.

###  3.8 Comparison to Related Methods

##### Few-Shot

We compare Llama-2-70b-chat zero-shot AgentInstruct results with few-shot
results on AddSub, IMDB, and NarrativeQA in Figure 12. Surprisingly, zero-shot
AgentInstruct reaches competitiveness with few-shot prompting. Zero-shot
AgentInstruct, without any few-shot examples, outperforms few-shot performance
on AddSub and NarrativeQA, by over 4.3% and 23.7% respectively, and loses by
0.7% on IMDB. Ideally, all the information encoded in the few-shot examples
can be presented in a clear manner within the agent instructions to better
utilize the reasoning capabilities of models. As shown, zero-shot
AgentInstruct has the potential to reach or even beat few-shot performance.

Figure 11: Comparison between zero-shot AgentInstruct and few-shot on
Llama-2-70b-chat on AddSub, IMDB, and NarrativeQA.

Figure 12: Comparison between zero-shot AgentInstruct and zero-shot self-
consistency on Llama-2-70b-chat.

##### Self-Consistency

Finally, we compare zero-shot AgentInstruct results with self-consistency
(Wang et al., 2023c) results of Llama-2-70b-chat on AddSub, IMDB, and
NarrativeQA in Figure 12. We adapt self-consistency to the zero-shot setting
as follows: We sample three responses using a temperature of 0.7, top-$k$
sampling with $k=40$, and a randomly generated seed for each request. After
cleaning the output, we use a majority vote to determine the consensus answer,
choosing at random to break ties. On AddSub, IMDB, and NarrativeQA, zero-shot
AgentInstruct outperforms self-consistency by 5.8%, 7.5%, and 1.9%
respectively. Besides, zero-shot AgentInstruct is more computationally
efficient than self-consistency as there is no need for sampling the reasoning
paths multiple times.

##  4 Related Work

Large language models, such as GPT-4 (OpenAI, 2023), GPT-3 (Brown et al.,
2020), PaLM (Chowdhery et al., 2022), PaLM-2 (Anil et al., 2023), BLOOM (Scao
et al., 2023), OPT (Zhang et al., 2022a), LLaMA (Touvron et al., 2023a),
Llama-2 (Touvron et al., 2023b), and many others (Radford et al., 2019; Wang &
Komatsuzaki, 2021; Black et al., 2021; Smith et al., 2022; Hoffmann et al.,
2022; Penedo et al., 2023) have shown remarkable performance on natural
language processing (NLP) tasks. Following the pretraining phase, additional
finetuning enables models (e.g., FLAN (Wei et al., 2022a), FLAN-T5 (Chung et
al., 2022), InstructGPT (Ouyang et al., 2022)) to better align with human
instructions to complete tasks. Moreover, instruction-tuning has enabled LLMs
(e.g., GPT-3.5 Turbo (OpenAI, 2022), Llama-2-chat (Touvron et al., 2023b),
Self-Instruct (Wang et al., 2023d), Alpaca (Taori et al., 2023), Vicuna
(Chiang et al., 2023), Koala (Geng et al., 2023)) to better engage with users
through single-turn or multi-turn dialogue. Zero-shot AgentInstruct builds on
these instruction-following language models, enabling better zero-shot
reasoning abilities through the use of agent instructions.

Language agents (Yao et al., 2023b; Shinn et al., 2023; Xu et al., 2023; Park
et al., 2023; Zhou et al., 2023b; Andreas, 2022; Wang et al., 2023a; Xi et
al., 2023; Sumers et al., 2023; Chan et al., 2023) have recently emerged due
to the task planning capabilities of LLMs. Given a task that is often
demonstrated in natural language, these agents aim to complete the task
directly. Unlike existing agents, the goal of our agent is to generate task-
specific instructions on how to complete the given task, decoupling the agent
planning and reasoning steps. Besides reaching competitive performance with
using agents directly, our design turns out to be more cost-effective.

Finetuning is an effective method for generating higher-quality responses from
LLMs on downstream tasks (Liu et al., 2019; Howard & Ruder, 2018). As model
scale increases, finetuning becomes less practical, which lightweight tuning
methods (such as prefix tuning (Li & Liang, 2021), prompt learning (Lester et
al., 2021; Liu et al., 2023), LoRA (Hu et al., 2022)) have tried to solve.
Even with such methods, in-context prompting techniques have gained attention
as an alternative. Few-shot learning, which involves providing a few examples
demonstrating the task before prompting the models during inference, is often
effective on a range of tasks (Brown et al., 2020; Dong et al., 2023). Chain
of thought (CoT) prompting (Wei et al., 2022b) involves generating a series of
intermediate reasoning steps, which can dramatically increase the performance
of LLMs on complex reasoning tasks. While this reasoning behavior is
traditionally learned from few-shot demonstrations, Kojima et al. (2022)
extends CoT prompting to the zero-shot setting. More recently, new approaches
such as self-consistency (Wang et al., 2023c), plan-and-solve prompting (Wang
et al., 2023b), tree of thought (Yao et al., 2023a), and graph of thought
(Besta et al., 2023) have further improved the reasoning quality. On the other
hand, zero-shot AgentInstruct focuses on the zero-shot setup. It generalizes
the reasoning abilities of LLMs to more tasks by utilizing task-specific
instructions generated from our agent to better align the reasoning process
with a particular task.

NLP benchmarking datasets provide a standardized interface to evaluate LLMs on
specific downstream tasks. Common benchmarks (e.g., HELM (Liang et al., 2023),
MMLU (Hendrycks et al., 2021), and many others (Wang et al., 2019a; b;
Rajpurkar et al., 2016; Srivastava et al., 2023; Zhong et al., 2023; Suzgun et
al., 2023; Chen et al., 2021)) have become part of the standard evaluation of
LLMs. We benchmark our method on 29 datasets, including the core scenarios
from HELM (Liang et al., 2023) and the reasoning datasets from Kojima et al.
(2022). We also include results from these benchmarks as task-specific model
results for comparison purposes. Besides reasoning tasks, our method
generalizes to general language understanding benchmark tasks including
generation and classification. For reasoning tasks, our method outperforms
existing zero-shot approaches (Brown et al., 2020; Kojima et al., 2022) by a
large margin.

##  5 Conclusion

Our work proposes a new way of improving the zero-shot reasoning abilities of
large language models on general language understanding tasks. We build an
agent to instruct the reasoning process of LLMs. Our agent automatically
generates task-specific instructions for a wide set of tasks. The instructions
are used to guide LLMs to reason better across these tasks to make high-
quality predictions. Our method is zero-shot so no input-output examples are
required to solve the task. Our results confirm the overall efficacy of our
approach, leading to substantial improvements across various NLP tasks
spanning generation, classification, and reasoning. Average score enhancements
of 13.3%, 23.2%, and 17.0% are achieved over the standard zero-shot setting
across 29 datasets for Vicuna-13b, Llama-2-70b-chat, and GPT-3.5 Turbo
respectively. Our method wins on 20 of the 29 datasets used for evaluation. We
believe zero-shot AgentInstruct’s style of human-understandable reasoning,
along with its utilization of an autonomous agent with access to a wide range
of dataset knowledge, can replace more traditional styles of zero or few-shot
prompting as models become equipped with stronger reasoning capabilities.

## Acknowledgements

We thank Ce Zhang and Together AI for giving us access to some computational
resources.

## References

  * Alex et al. (2021) Neel Alex, Eli Lifland, Lewis Tunstall, Abhishek Thakur, Pegah Maham, C. Riedel, Emmie Hine, Carolyn Ashurst, Paul Sedille, Alexis Carlier, Michael Noetel, and Andreas Stuhlmüller.  RAFT: A Real-World Few-Shot Text Classification Benchmark.  In _NeurIPS_ , 2021.
  * Andreas (2022) Jacob Andreas.  Language Models as Agent Models.  In _EMNLP_ , 2022.
  * Anil et al. (2023) Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, and Yonghui Wu.  PaLM 2 Technical Report.  _arXiv_ , 2023.
  * Araci (2019) Dogu Araci.  FinBERT: Financial Sentiment Analysis with Pre-trained Language Models.  _arXiv_ , 2019.
  * Bajaj et al. (2016) Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang.  MS MARCO: A Human Generated MAchine Reading COmprehension Dataset.  In _NeurIPS_ , 2016.
  * Besta et al. (2023) Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Michal Podstawski, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler.  Graph of Thoughts: Solving Elaborate Problems with Large Language Models.  _arXiv_ , 2023.
  * Black et al. (2021) Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman.  GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow.  _GitHub_ , 2021.
  * Bolton et al. (2022) Elliot Bolton, David Hall, Michihiro Yasunaga, Tony Lee, Chris Manning, and Percy Liang.  BioMedLM.  _CRFM_ , 2022.
  * Borkan et al. (2019) Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman.  Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification.  In _WWW_ , 2019.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.  Language Models are Few-Shot Learners.  In _NeurIPS_ , 2020.
  * Chan et al. (2023) Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu.  ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate.  _arXiv_ , 2023.
  * Chase (2022) Harrison Chase.  LangChain.  _GitHub_ , 2022.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba.  Evaluating Large Language Models Trained on Code.  _arXiv_ , 2021.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing.  Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* ChatGPT Quality.  _LMSYS Org_ , 2023.
  * Choi et al. (2018) Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer.  QuAC: Question Answering in Context.  In _EMNLP_ , 2018.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel.  PaLM: Scaling Language Modeling with Pathways.  _arXiv_ , 2022.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei.  Scaling Instruction-Finetuned Language Models.  _arXiv_ , 2022.
  * Clark et al. (2019) Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova.  BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions.  In _NAACL-HLT_ , 2019.
  * Cobbe et al. (2021) Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman.  Training Verifiers to Solve Math Word Problems.  _arXiv_ , 2021.
  * Ding et al. (2021) SiYu Ding, Junyuan Shang, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang.  ERNIE-Doc: A Retrospective Long-Document Modeling Transformer.  In _ACL_ , 2021.
  * Dong et al. (2023) Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui.  A Survey on In-context Learning.  _arXiv_ , 2023.
  * Driess et al. (2023) Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence.  PaLM-E: An Embodied Multimodal Language Model.  In _ICML_ , 2023.
  * Geng et al. (2023) Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song.  Koala: A Dialogue Model for Academic Research.  _BAIR_ , 2023.
  * Geva et al. (2021) Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant.  Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies.  _TACL_ , 2021.
  * Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.  Measuring Massive Multitask Language Understanding.  In _ICLR_ , 2021.
  * Hoffmann et al. (2022) Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre.  Training Compute-Optimal Large Language Models.  _arXiv_ , 2022.
  * Howard & Ruder (2018) Jeremy Howard and Sebastian Ruder.  Universal Language Model Fine-tuning for Text Classification.  In _ACL_ , 2018.
  * Hu et al. (2022) Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  LoRA: Low-Rank Adaptation of Large Language Models.  In _ICLR_ , 2022.
  * Huang et al. (2019) Hsin-Yuan Huang, Eunsol Choi, and Wen-tau Yih.  FlowQA: Grasping Flow in History for Conversational Machine Comprehension.  In _ICLR_ , 2019.
  * Huang et al. (2022) Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han.  Large Language Models Can Self-Improve.  _arXiv_ , 2022.
  * Huang et al. (2020) Kexin Huang, Jaan Altosaar, and Rajesh Ranganath.  ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission.  _arXiv_ , 2020.
  * Huang et al. (2023) Siyuan Huang, Zhengkai Jiang, Hao Dong, Yu Qiao, Peng Gao, and Hongsheng Li.  Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model.  _arXiv_ , 2023.
  * Järvelin & Kekäläinen (2002) Kalervo Järvelin and Jaana Kekäläinen.  Cumulated Gain-Based Evaluation of IR Techniques.  _ACM TOIS_ , 2002.
  * Joshi et al. (2020) Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy.  SpanBERT: Improving Pre-training by Representing and Predicting Spans.  _TACL_ , 2020.
  * Kočiský et al. (2018) Tomáš Kočiský, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette.  The NarrativeQA Reading Comprehension Challenge.  _TACL_ , 2018.
  * Koh et al. (2021) Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton Earnshaw, Imran S. Haque, Sara M. Beery, Jure Leskovec, Anshul Kundaje, Emma Pierson, Sergey Levine, Chelsea Finn, and Percy Liang.  WILDS: A Benchmark of in-the-Wild Distribution Shifts.  In _ICML_ , 2021.
  * Kojima et al. (2022) Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa.  Large Language Models are Zero-Shot Reasoners.  In _NeurIPS_ , 2022.
  * Koncel-Kedziorski et al. (2015) Rik Koncel-Kedziorski, Hannaneh Hajishirzi, Ashish Sabharwal, Oren Etzioni, and Siena Dumas Ang.  Parsing Algebraic Word Problems into Equations.  _TACL_ , 2015.
  * Koncel-Kedziorski et al. (2016) Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi.  MAWPS: A Math Word Problem Repository.  In _HLT-NAACL_ , 2016.
  * Kwiatkowski et al. (2019) Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov.  Natural Questions: A Benchmark for Question Answering Research.  _TACL_ , 2019.
  * Kıcıman et al. (2023) Emre Kıcıman, Robert Ness, Amit Sharma, and Chenhao Tan.  Causal Reasoning and Large Language Models: Opening a New Frontier for Causality.  _arXiv_ , 2023.
  * Lester et al. (2021) Brian Lester, Rami Al-Rfou, and Noah Constant.  The Power of Scale for Parameter-Efficient Prompt Tuning.  In _EMNLP_ , 2021.
  * Li & Liang (2021) Xiang Lisa Li and Percy Liang.  Prefix-Tuning: Optimizing Continuous Prompts for Generation.  In _ACL_ , 2021.
  * Liang et al. (2023) Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda.  Holistic Evaluation of Language Models.  In _TMLR_ , 2023.
  * Lin (2004) Chin-Yew Lin.  ROUGE: A Package for Automatic Evaluation of Summaries.  In _ACL_ , 2004.
  * Lin et al. (2022) Stephanie Lin, Jacob Hilton, and Owain Evans.  TruthfulQA: Measuring How Models Mimic Human Falsehoods.  In _ACL_ , 2022.
  * Ling et al. (2017) Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom.  Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems.  In _ACL_ , 2017.
  * Liu et al. (2023) Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig.  Pre-Train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing.  _ACM Comput. Surv._ , 2023.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  RoBERTa: A Robustly Optimized BERT Pretraining Approach.  _arXiv_ , 2019.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts.  Learning Word Vectors for Sentiment Analysis.  In _ACL_ , 2011.
  * MacAvaney et al. (2021) Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, and Nazli Goharian.  Simplified Data Wrangling with ir_datasets.  In _SIGIR_ , 2021.
  * Mihaylov et al. (2018) Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal.  Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering.  In _EMNLP_ , 2018.
  * Narayan et al. (2018) Shashi Narayan, Shay B. Cohen, and Mirella Lapata.  Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization.  In _EMNLP_ , 2018.
  * Nogueira et al. (2020) Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin.  Document Ranking with a Pretrained Sequence-to-Sequence Model.  In _EMNLP_ , 2020.
  * OpenAI (2022) OpenAI.  Introducing ChatGPT.  _OpenAI_ , 2022.
  * OpenAI (2023) OpenAI.  GPT-4 Technical Report.  _arXiv_ , 2023.
  * Ouyang et al. (2022) Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe.  Training language models to follow instructions with human feedback.  In _NeurIPS_ , 2022.
  * Park et al. (2023) Joon Sung Park, Joseph C. O’Brien, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein.  Generative Agents: Interactive Simulacra of Human Behavior.  In _UIST_ , 2023.
  * Patel et al. (2021) Arkil Patel, Satwik Bhattamishra, and Navin Goyal.  Are NLP Models really able to Solve Simple Math Word Problems?  In _NAACL-HLT_ , 2021.
  * Penedo et al. (2023) Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay.  The RefinedWeb dataset for Falcon LLM: outperforming curated corpora with web data, and web data only.  _arXiv_ , 2023.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever.  Language Models are Unsupervised Multitask Learners.  _OpenAI_ , 2019.
  * Rajpurkar et al. (2016) Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang.  SQuAD: 100,000+ Questions for Machine Comprehension of Text.  In _EMNLP_ , 2016.
  * Roy & Roth (2015) Subhro Roy and Dan Roth.  Solving General Arithmetic Word Problems.  In _EMNLP_ , 2015.
  * Scao et al. (2023) Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, Jonathan Tow, Alexander M. Rush, Stella Biderman, Albert Webson, Pawan Sasanka Ammanamanchi, Thomas Wang, Benoît Sagot, Niklas Muennighoff, Albert Villanova del Moral, Olatunji Ruwase, Rachel Bawden, Stas Bekman, Angelina McMillan-Major, Iz Beltagy, Huu Nguyen, Lucile Saulnier, Samson Tan, Pedro Ortiz Suarez, Victor Sanh, Hugo Laurençon, Yacine Jernite, Julien Launay, Margaret Mitchell, Colin Raffel, Aaron Gokaslan, Adi Simhi, Aitor Soroa, Alham Fikri Aji, Amit Alfassy, Anna Rogers, Ariel Kreisberg Nitzav, Canwen Xu, Chenghao Mou, Chris Emezue, Christopher Klamm, Colin Leong, Daniel van Strien, David Ifeoluwa Adelani, Dragomir Radev, Eduardo González Ponferrada, Efrat Levkovizh, Ethan Kim, Eyal Bar Natan, Francesco De Toni, Gérard Dupont, Germán Kruszewski, Giada Pistilli, Hady Elsahar, Hamza Benyamina, Hieu Tran, Ian Yu, Idris Abdulmumin, Isaac Johnson, Itziar Gonzalez-Dios, Javier de la Rosa, Jenny Chim, Jesse Dodge, Jian Zhu, Jonathan Chang, Jörg Frohberg, Joseph Tobing, Joydeep Bhattacharjee, Khalid Almubarak, Kimbo Chen, Kyle Lo, Leandro Von Werra, Leon Weber, Long Phan, Loubna Ben allal, Ludovic Tanguy, Manan Dey, Manuel Romero Muñoz, Maraim Masoud, María Grandury, Mario Šaško, Max Huang, Maximin Coavoux, Mayank Singh, Mike Tian-Jian Jiang, Minh Chien Vu, Mohammad A. Jauhar, Mustafa Ghaleb, Nishant Subramani, Nora Kassner, Nurulaqilla Khamis, Olivier Nguyen, Omar Espejel, Ona de Gibert, Paulo Villegas, Peter Henderson, Pierre Colombo, Priscilla Amuok, Quentin Lhoest, Rheza Harliman, Rishi Bommasani, Roberto Luis López, Rui Ribeiro, Salomey Osei, Sampo Pyysalo, Sebastian Nagel, Shamik Bose, Shamsuddeen Hassan Muhammad, Shanya Sharma, Shayne Longpre, Somaieh Nikpoor, Stanislav Silberberg, Suhas Pai, Sydney Zink, Tiago Timponi Torrent, Timo Schick, Tristan Thrush, Valentin Danchev, Vassilina Nikoulina, Veronika Laippala, Violette Lepercq, Vrinda Prabhu, Zaid Alyafeai, Zeerak Talat, Arun Raja, Benjamin Heinzerling, Chenglei Si, Davut Emre Taşar, Elizabeth Salesky, Sabrina J. Mielke, Wilson Y. Lee, Abheesht Sharma, Andrea Santilli, Antoine Chaffin, Arnaud Stiegler, Debajyoti Datta, Eliza Szczechla, Gunjan Chhablani, Han Wang, Harshit Pandey, Hendrik Strobelt, Jason Alan Fries, Jos Rozen, Leo Gao, Lintang Sutawika, M Saiful Bari, Maged S. Al-shaibani, Matteo Manica, Nihal Nayak, Ryan Teehan, Samuel Albanie, Sheng Shen, Srulik Ben-David, Stephen H. Bach, Taewoon Kim, Tali Bers, Thibault Fevry, Trishala Neeraj, Urmish Thakker, Vikas Raunak, Xiangru Tang, Zheng-Xin Yong, Zhiqing Sun, Shaked Brody, Yallow Uri, Hadar Tojarieh, Adam Roberts, Hyung Won Chung, Jaesung Tae, Jason Phang, Ofir Press, Conglong Li, Deepak Narayanan, Hatim Bourfoune, Jared Casper, Jeff Rasley, Max Ryabinin, Mayank Mishra, Minjia Zhang, Mohammad Shoeybi, Myriam Peyrounette, Nicolas Patry, Nouamane Tazi, Omar Sanseviero, Patrick von Platen, Pierre Cornette, Pierre François Lavallée, Rémi Lacroix, Samyam Rajbhandari, Sanchit Gandhi, Shaden Smith, Stéphane Requena, Suraj Patil, Tim Dettmers, Ahmed Baruwa, Amanpreet Singh, Anastasia Cheveleva, Anne-Laure Ligozat, Arjun Subramonian, Aurélie Névéol, Charles Lovering, Dan Garrette, Deepak Tunuguntla, Ehud Reiter, Ekaterina Taktasheva, Ekaterina Voloshina, Eli Bogdanov, Genta Indra Winata, Hailey Schoelkopf, Jan-Christoph Kalo, Jekaterina Novikova, Jessica Zosa Forde, Jordan Clive, Jungo Kasai, Ken Kawamura, Liam Hazan, Marine Carpuat, Miruna Clinciu, Najoung Kim, Newton Cheng, Oleg Serikov, Omer Antverg, Oskar van der Wal, Rui Zhang, Ruochen Zhang, Sebastian Gehrmann, Shachar Mirkin, Shani Pais, Tatiana Shavrina, Thomas Scialom, Tian Yun, Tomasz Limisiewicz, Verena Rieser, Vitaly Protasov, Vladislav Mikhailov, Yada Pruksachatkun, Yonatan Belinkov, Zachary Bamberger, Zdeněk Kasner, Alice Rueda, Amanda Pestana, Amir Feizpour, Ammar Khan, Amy Faranak, Ana Santos, Anthony Hevia, Antigona Unldreaj, Arash Aghagol, Arezoo Abdollahi, Aycha Tammour, Azadeh HajiHosseini, Bahareh Behroozi, Benjamin Ajibade, Bharat Saxena, Carlos Muñoz Ferrandis, Daniel McDuff, Danish Contractor, David Lansky, Davis David, Douwe Kiela, Duong A. Nguyen, Edward Tan, Emi Baylor, Ezinwanne Ozoani, Fatima Mirza, Frankline Ononiwu, Habib Rezanejad, Hessie Jones, Indrani Bhattacharya, Irene Solaiman, Irina Sedenko, Isar Nejadgholi, Jesse Passmore, Josh Seltzer, Julio Bonis Sanz, Livia Dutra, Mairon Samagaio, Maraim Elbadri, Margot Mieskes, Marissa Gerchick, Martha Akinlolu, Michael McKenna, Mike Qiu, Muhammed Ghauri, Mykola Burynok, Nafis Abrar, Nazneen Rajani, Nour Elkott, Nour Fahmy, Olanrewaju Samuel, Ran An, Rasmus Kromann, Ryan Hao, Samira Alizadeh, Sarmad Shubber, Silas Wang, Sourav Roy, Sylvain Viguier, Thanh Le, Tobi Oyebade, Trieu Le, Yoyo Yang, Zach Nguyen, Abhinav Ramesh Kashyap, Alfredo Palasciano, Alison Callahan, Anima Shukla, Antonio Miranda-Escalada, Ayush Singh, Benjamin Beilharz, Bo Wang, Caio Brito, Chenxi Zhou, Chirag Jain, Chuxin Xu, Clémentine Fourrier, Daniel León Periñán, Daniel Molano, Dian Yu, Enrique Manjavacas, Fabio Barth, Florian Fuhrimann, Gabriel Altay, Giyaseddin Bayrak, Gully Burns, Helena U. Vrabec, Imane Bello, Ishani Dash, Jihyun Kang, John Giorgi, Jonas Golde, Jose David Posada, Karthik Rangasai Sivaraman, Lokesh Bulchandani, Lu Liu, Luisa Shinzato, Madeleine Hahn de Bykhovetz, Maiko Takeuchi, Marc Pàmies, Maria A Castillo, Marianna Nezhurina, Mario Sänger, Matthias Samwald, Michael Cullan, Michael Weinberg, Michiel De Wolf, Mina Mihaljcic, Minna Liu, Moritz Freidank, Myungsun Kang, Natasha Seelam, Nathan Dahlberg, Nicholas Michio Broad, Nikolaus Muellner, Pascale Fung, Patrick Haller, Ramya Chandrasekhar, Renata Eisenberg, Robert Martin, Rodrigo Canalli, Rosaline Su, Ruisi Su, Samuel Cahyawijaya, Samuele Garda, Shlok S Deshmukh, Shubhanshu Mishra, Sid Kiblawi, Simon Ott, Sinee Sang-aroonsiri, Srishti Kumar, Stefan Schweter, Sushil Bharati, Tanmay Laud, Théo Gigant, Tomoya Kainuma, Wojciech Kusa, Yanis Labrak, Yash Shailesh Bajaj, Yash Venkatraman, Yifan Xu, Yingxin Xu, Yu Xu, Zhe Tan, Zhongli Xie, Zifan Ye, Mathilde Bras, Younes Belkada, and Thomas Wolf.  BLOOM: A 176B-Parameter Open-Access Multilingual Language Model.  _arXiv_ , 2023.
  * See et al. (2017) Abigail See, Peter J. Liu, and Christopher D. Manning.  Get To The Point: Summarization with Pointer-Generator Networks.  In _ACL_ , 2017.
  * ShareGPT (2023) ShareGPT.  ShareGPT: Share your wildest ChatGPT conversations with one click.  _ShareGPT_ , 2023.
  * Shinn et al. (2023) Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao.  Reflexion: Language Agents with Verbal Reinforcement Learning.  _arXiv_ , 2023.
  * Smith et al. (2022) Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zhang, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro.  Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model.  _arXiv_ , 2022.
  * Srivastava et al. (2023) Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakaş, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartłomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, César Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodola, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocoń, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Hernandez-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras-Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Şenel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramírez Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael Ivanitskiy, Michael Starritt, Michael Strube, Michał Swędrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha S. Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Miłkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima, Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, and Ziyi Wu.  Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models.  _TMLR_ , 2023.
  * Sumers et al. (2023) Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths.  Cognitive Architectures for Language Agents.  _arXiv_ , 2023.
  * Suzgun et al. (2023) Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei.  Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them.  In _ACL_ , 2023.
  * Talmor et al. (2019) Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant.  CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge.  In _NAACL-HLT_ , 2019.
  * Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin , Percy Liang, and Tatsunori B. Hashimoto.  Alpaca: A Strong, Replicable Instruction-Following Model.  _arXiv_ , 2023.
  * Touvron et al. (2023a) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.  LLaMA: Open and Efficient Foundation Language Models.  _arXiv_ , 2023a.
  * Touvron et al. (2023b) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom.  Llama 2: Open Foundation and Fine-Tuned Chat Models.  _arXiv_ , 2023b.
  * Trischler et al. (2017) Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman.  NewsQA: A Machine Comprehension Dataset.  In _ACL_ , 2017.
  * Wan et al. (2023) Xingchen Wan, Ruoxi Sun, Hanjun Dai, Sercan O. Arik, and Tomas Pfister.  Better Zero-Shot Reasoning with Self-Adaptive Prompting.  In _ACL_ , 2023.
  * Wang et al. (2019a) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.  SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems.  In _NeurIPS_ , 2019a.
  * Wang et al. (2019b) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman.  GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding.  In _ICLR_ , 2019b.
  * Wang & Komatsuzaki (2021) Ben Wang and Aran Komatsuzaki.  GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model.  _GitHub_ , 2021.
  * Wang et al. (2023a) Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen.  A Survey on Large Language Model based Autonomous Agents.  _arXiv_ , 2023a.
  * Wang et al. (2023b) Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim.  Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models.  In _ACL_ , 2023b.
  * Wang et al. (2023c) Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou.  Self-Consistency Improves Chain of Thought Reasoning in Language Models.  In _ICLR_ , 2023c.
  * Wang et al. (2023d) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi.  Self-Instruct: Aligning Language Models with Self-Generated Instructions.  In _ACL_ , 2023d.
  * Wei et al. (2022a) Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le.  Finetuned Language Models are Zero-Shot Learners.  In _ICLR_ , 2022a.
  * Wei et al. (2022b) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou.  Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.  In _NeurIPS_ , 2022b.
  * Wu et al. (2023) Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann.  BloombergGPT: A Large Language Model for Finance.  _arXiv_ , 2023.
  * Xi et al. (2023) Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhang, Wenjuan Qin, Yongyan Zheng, Xipeng Qiu, Xuanjing Huang, and Tao Gui.  The Rise and Potential of Large Language Model Based Agents: A Survey.  _arXiv_ , 2023.
  * Xu et al. (2023) Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao.  ExpertPrompting: Instructing Large Language Models to be Distinguished Experts.  _arXiv_ , 2023.
  * Xu et al. (2022) Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, and Xuedong Huang.  Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention.  In _IJCAI_ , 2022.
  * Yao et al. (2023a) Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan.  Tree of Thoughts: Deliberate Problem Solving with Large Language Models.  _arXiv_ , 2023a.
  * Yao et al. (2023b) Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao.  ReAct: Synergizing Reasoning and Acting in Language Models.  In _ICLR_ , 2023b.
  * Zellers et al. (2019) Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi.  HellaSwag: Can a Machine Really Finish Your Sentence?  In _ACL_ , 2019.
  * Zhang et al. (2020) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu.  PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization.  In _ICML_ , 2020.
  * Zhang et al. (2022a) Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer.  OPT: Open Pre-trained Transformer Language Models.  _arXiv_ , 2022a.
  * Zhang et al. (2022b) Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola.  Automatic Chain of Thought Prompting in Large Language Models.  In _ICLR_ , 2022b.
  * Zhao et al. (2023) Xu Zhao, Yuxi Xie, Kenji Kawaguchi, Junxian He, and Qizhe Xie.  Automatic Model Selection with Large Language Models for Reasoning.  _arXiv_ , 2023.
  * Zheng et al. (2023) Chuanyang Zheng, Zhengying Liu, Enze Xie, Zhenguo Li, and Yu Li.  Progressive-Hint Prompting Improves Reasoning in Large Language Models.  _arXiv_ , 2023.
  * Zhong et al. (2023) Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan.  AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models.  _arXiv_ , 2023.
  * Zhou et al. (2023a) Aojun Zhou, Ke Wang, Zimu Lu, Weikang Shi, Sichun Luo, Zipeng Qin, Shaoqing Lu, Anya Jia, Linqi Song, Mingjie Zhan, and Hongsheng Li.  Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verification.  _arXiv_ , 2023a.
  * Zhou et al. (2023b) Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan.  Agents: An Open-source Framework for Autonomous Language Agents.  _arXiv_ , 2023b.
  * Zoph et al. (2022) Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus.  ST-MoE: Designing Stable and Transferable Sparse Expert Models.  _arXiv_ , 2022.

##  Appendix A Experimental Setup

###  A.1 Datasets

We evaluate zero-shot AgentInstruct on a wide selection of datasets consisting
of all HELM core scenarios (Liang et al., 2023) and all the reasoning datasets
used to benchmark zero-shot CoT (Kojima et al., 2022). The datasets are listed
in Table 4, along with their categorizations. See Figure 13 for general
details and A.4 for the collection and processing of each dataset in more
detail.

###  A.2 Models

We focus on the following large language models:

  * •

Vicuna-13b (Chiang et al., 2023): A finetuned version of LLaMA-13b (Touvron et
al., 2023a) which was trained on an additional 70 thousand user-shared
conversations from ShareGPT (ShareGPT, 2023). Version 1.1 weight deltas were
used.

  * •

Llama-2-chat (Touvron et al., 2023b): A family of chat models based on
Llama-2. Llama-2-chat was further trained using supervised finetuning, reward
modeling, and reinforcement learning from human feedback, and comes in three
sizes: 7b, 13b, and 70b. We focus on the 70b model, but share results from the
7b and 13b models in Figure 8 and Appendix B.

  * •

GPT-3.5 Turbo (OpenAI, 2022): A finetuned version of GPT-3.5 which was further
trained using reinforcement learning from human feedback. Specifically, the
0301 version was used.

Dataset | Task Category | Metric | Number of Instances
---|---|---|---
AddSub (Koncel-Kedziorski et al., 2016) | Generation, Reasoning | QEM | 395
AQuA (Ling et al., 2017) | Classification, Reasoning | EM | 254
BoolQ (Clark et al., 2019) | Classification | QPEM | 1,000
CivilComments (Koh et al., 2021; Borkan et al., 2019) | Classification | QPEM | 6,688
All | Classification | QPEM | 765
Black | Classification | QPEM | 735
Christian | Classification | QPEM | 733
Female | Classification | QPEM | 730
LGBTQ | Classification | QPEM | 735
Male | Classification | QPEM | 748
Muslim | Classification | QPEM | 760
Other Religions | Classification | QPEM | 767
White | Classification | QPEM | 715
CNN/Daily Mail (See et al., 2017) | Generation | ROUGE-2 | 466
Coin Flip (Wei et al., 2022b) | Classification, Reasoning | QEM | 500
CommonsenseQA (Talmor et al., 2019) | Classification, Reasoning | EM | 1,000
Date Understanding (Srivastava et al., 2023) | Classification, Reasoning | QEM | 369
GSM8K (Cobbe et al., 2021) | Generation, Reasoning | QEM | 1,000
HellaSwag (Zellers et al., 2019) | Classification | EM | 1,000
IMDB (Maas et al., 2011) | Classification | QEM | 1,000
Last Letter Concatenation (Wei et al., 2022b) | Generation, Reasoning | QEM | 500
MMLU (Hendrycks et al., 2021) | Classification | EM | 514
Abstract Algebra | Classification | EM | 100
College Chemistry | Classification | EM | 100
Computer Security | Classification | EM | 100
Econometrics | Classification | EM | 114
US Foreign Policy | Classification | EM | 100
MS MARCO (Regular) (Bajaj et al., 2016) | Classification | RR@10 | 1,000
MS MARCO (TREC) (Bajaj et al., 2016) | Classification | NDCG@10 | 43
MultiArith (Roy & Roth, 2015) | Generation, Reasoning | QEM | 600
NarrativeQA (Kočiský et al., 2018) | Generation | F1 | 355
NaturalQuestions (closed-book) (Kwiatkowski et al., 2019) | Generation | F1 | 1,000
NaturalQuestions (open-book) | Generation | F1 | 1,000
NewsQA (Trischler et al., 2017) | Generation | F1 | 1,000
OpenBookQA (Mihaylov et al., 2018) | Classification | EM | 500
QuAC (Choi et al., 2018) | Generation | F1 | 1,000
RAFT (Alex et al., 2021) | Classification | QEM | 440
ADE Corpus v2 | Classification | QEM | 40
Banking 77 | Classification | QEM | 40
Neurips Impact Statement | Classification | QEM | 40
One Stop English | Classification | QEM | 40
Overruling | Classification | QEM | 40
Semiconductor Org Types | Classification | QEM | 40
Systematic Review Inclusion | Classification | QEM | 40
Tai Saftey Research | Classification | QEM | 40
Terms of Service | Classification | QEM | 40
Tweet Eval Hate | Classification | QEM | 40
Twitter Complaints | Classification | QEM | 40
Shuffled Objects (Srivastava et al., 2023) | Classification, Reasoning | QEM | 2,750
Five Objects | Classification, Reasoning | QEM | 1,000
Seven Objects | Classification, Reasoning | QEM | 1,000
Three Objects | Classification, Reasoning | QEM | 750
SingleEq (Koncel-Kedziorski et al., 2016) | Generation, Reasoning | QEM | 508
StrategyQA (Geva et al., 2021; Srivastava et al., 2023) | Classification, Reasoning | QEM | 458
SVAMP (Patel et al., 2021) | Generation, Reasoning | QEM | 1,000
TruthfulQA (Lin et al., 2022) | Classification | EM | 654
XSUM (Narayan et al., 2018) | Generation | ROUGE-2 | 518

Table 4: Overview of each dataset used.

###  A.3 Implementation Details

####  A.3.1 Zero-Shot AgentInstruct Details

##### Agent Instructions

When generating the instructions using zero-shot AgentInstruct, we query our
question answering (QA) API built based on the Bing Web Search API to retrieve
five relevant websites, each of which is added to a vector database. Our QA
API is a retrieval question answering system built from that database, which
is used as the only tool in our main agent. We implement our agent based on
LangChain’s ReAct implementation (Chase, 2022; Yao et al., 2023b). To initiate
instruction generation, we build the agent input based on basic dataset
information including the name of the dataset, task information (e.g.,
generation or classification), and a few input examples. We use GPT-4 (OpenAI,
2023) inside our agent. We use the default temperature of 0.3 and the snapshot
gpt-4-0613 when generating instructions. Given the temperature, if rerun it is
not guaranteed to generate the same outputs each time, nor is there any
guarantee that the same steps of our pipeline are reached. In the case that
instructions were not returned, we reran the pipeline until there were valid
instructions. We also reran the pipeline on a select number of datasets
(NaturalQA (openbook), GSM8K) where the name of the dataset was unhelpful. In
this case, we use an alias for the name of the dataset.

##### Chain of Thought Reasoning

The reasoning extraction prompt of the chain of thought reasoning pipeline is
in Table 13. The answer extraction prompts for generation, classification
(excluding multi-choice), and classification (multi-choice) datasets are in
Tables 14, 15, and 16 respectively. The prompts for reasoning tasks follow the
corresponding prompts of either generation or classification based on their
general categorization. We take the output of the answer extraction prompt as
the answer, applying parsing as dictated by the dataset. Note that for
generation, we encouraged shorter answers in the answer extraction prompt due
to the nature of the datasets we used.

    You will be provided instructions for completing a task followed by a task to complete.

    Instructions:

    {Agent Instructions}

    {Test Instance}

    Follow the instructions to generate an explanation that reasons towards the correct answer to the task above. End the explanation with the correct answer.

    Explanation:

Figure 13: Reasoning extraction prompt for the CoT reasoning of zero-shot
AgentInstruct.

    {Reasoning Extraction Prompt}{Reasoning Extraction Prompt Output}

    Therefore, the answer to the task is below. Give the answer in the shortest form possible that will still be correct.

Figure 14: Answer extraction prompt for the CoT reasoning of zero-shot
AgentInstruct on a generation dataset. Generation datasets that also belong to
the reasoning category use the same prompt.

    {Reasoning Extraction Prompt}{Reasoning Extraction Prompt Output}

    Therefore, the correct label among {labels} (just the label) to the original task is below.

Figure 15: Answer extraction prompt for the CoT reasoning of zero-shot
AgentInstruct on a classification (excluding multi-choice) dataset.
Classification (excluding multi-choice) datasets that also belong to the
reasoning category use the same prompt.

    {Reasoning Extraction Prompt}{Reasoning Extraction Prompt Output}

    Therefore, the correct multiple choice label (just the letter) to the task is below.

Figure 16: Answer extraction prompt for the CoT reasoning of zero-shot
AgentInstruct on a classification (multi-choice) dataset. Classification
(multi-choice) datasets that also belong to the reasoning category use the
same prompt.

Inference requests on Vicuna and Llama-2-chat were submitted from HELM to a
TorchServe API running on a local cluster containing 2 nodes, each with 8x
NVIDIA RTX A6000 GPUs. As a general rule, models were loaded at the highest
precision while still allowing each GPU to contain an entire copy of the full
model. Llama-2-7b-chat, Llama-2-13b-chat, and Vicuna-13b were run in FP16,
while Llama-2-70b-chat was quantized to NF4. NaturalQuestions (open-book),
NewsQA, and QuAC runs with Llama-2-70b-chat required 2 GPUs per worker in
order to handle longer sequence lengths. Inference requests on GPT-3.5 Turbo
were submitted to OpenAI’s API.

For Llama-2-chat runs, we prepend the necessary special tokens, including an
empty system message. Also, we include a special designation [\INST] during
the reasoning extraction prompt to signify the transition from User to
Assistant roles. Initial experimentation with formatting revealed that
Llama-2-chat was very sensitive to this designation, and would often fail to
generate thoughtful reasoning steps if omitted during runs with zero-shot
AgentInstruct or standard zero-shot CoT. We made no such User/Assistant
distinction before the answer extraction prompts. For Vicuna and GPT-3.5 Turbo
runs, we made no User/Assistant distinctions either, effectively treating all
model input as part of the User role. Generally, we use each dataset’s default
generation parameters outlined in HELM. All inference was done using a
temperature of 0.0, except on datasets that involved summarization, where a
temperature of 0.3 was used. For the reasoning extraction prompt, we request a
maximum of 512 new tokens, and for the answer extraction prompt, the number of
tokens requested is specific to each dataset.

####  A.3.2 Evaluation Details

Each dataset uses one of the following accuracy metrics:

  * •

Exact Match (EM): A binary indicator that awards correctness if and only if
the output exactly matches one or more of the correct references. We primarily
use this metric for classification, in particular for multi-choice
classification tasks.

  * •

Quasi-Exact Match (QEM): A more lenient version of Exact Match which awards
correctness if the output text matches the correct reference after both have
been normalized (i.e., after removing extra white space, articles,
punctuation, and capitalization) (Choi et al., 2018). This metric is primarily
used for classification tasks, as well as some generation tasks.

  * •

Quasi-Prefix Exact Match (QPEM): A more lenient version of Quasi-Exact Match
that awards correctness if the normalized output begins with the normalized
reference. Hence, “Yes, Elmendorf” would be marked correct if the reference
was “Yes”. The metric is useful on several classification tasks where the
model has a tendency to generate additional tokens beyond that of the correct
answer.

  * •

F1: Instead of checking for an exact match, this metric will award partial
correctness for any string overlap (Rajpurkar et al., 2016). The metric is
used for a number of question answering tasks with open-ended generation.

  * •

ROUGE-2: This metric scores correctness based on bigram overlap (Lin, 2004).
We use ROUGE-2 as the accuracy measure for generation tasks involving
summarization.

  * •

RR@10: This metric is used for MS MARCO (Regular) and it uses the reciprocal
rank of the first relevant document.

  * •

NDCG@10: This metric is used for MS MARCO (TREC) to assess the normalized
discounted cumulative gain using the set of graded rankings (Järvelin &
Kekäläinen, 2002).

Prior to evaluation, we often perform some post-processing to clean the
generated text. On mathematical generation datasets, we parse out the first
number in the output and use that for our final answer. Similarly, on
classification (multi-choice) datasets, we grab the first letter in the
output. Appendix A.4 gives a compact description of each dataset, as well as
the accuracy metric used, and any initial or post-processing done. These
choices followed HELM or other relevant papers.

In general, we report evaluation results using the default accuracy metric
outlined in HELM. For datasets with various subsets, we report the macro-
average accuracy score across all subsets. Prompts, parsing techniques, and
instructions were tested using 50 samples on most datasets to ensure the
explanations and parsing were valid. Note that we are including these 50
samples in the full (up to) 1,000 sample evaluation, so there is a possibility
of bias, especially regarding some of the smaller datasets. Note that all
experiments done in this paper are the result of a single run on our pipeline.
We only generate instructions once and run inference once as well due to the
cost of running our models on all datasets with a high amount of samples.

###  A.4 Additional Information About Each Dataset

####  A.4.1 AddSub

##### Description

AddSub (Koncel-Kedziorski et al., 2016) is a generation dataset containing
addition and subtraction math word problems. We also classify AddSub as a
reasoning dataset because of its inclusion in Kojima et al. (2022). The
dataset contains 395 test instances.

##### Implementation Details

AddSub is a generation dataset. Specifically, we prompt for an answer that is
at most 10 tokens. During few-shot runs, we use the 5 in-context learning
examples extracted from the math word problem instances presented in the chain
of thought paper (Wei et al., 2022b). Responses undergo a post-processing step
where the first number in the output is extracted for evaluation. The
evaluation metric used is quasi-exact match.

##### Examples

An example is shown in Figure 18.

##### Comparison Methods

The few-shot results on the GPT-4 model achieved the state-of-the-art (Zhao et
al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.2 AQuA

##### Description

AQuA (Ling et al., 2017) is a classification (multi-choice) dataset containing
various math word problems. Because it is in Kojima et al. (2022), we consider
it a reasoning dataset. The dataset contains 254 test instances.

##### Implementation Details

We prompt for an answer that is at most 5 tokens. Responses undergo a post-
processing step where the first letter in the output is extracted for
evaluation. The evaluation metric used is exact match.

##### Examples

An example is shown in Figure 19.

##### Comparison Methods

The few-shot results on the GPT-4 model achieved the state-of-the-art (Zheng
et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.3 BoolQ

##### Description

BoolQ (Clark et al., 2019) is a classification benchmark where yes/no
questions are asked about a given passage. We sample 1,000 test instances
following HELM.

##### Implementation Details

BoolQ is a core scenario in HELM (Liang et al., 2023), and we use the
implementation parameters outlined in HELM. Specifically, we prompt for an
answer that is at most 5 tokens. Quasi-prefix exact match is used for
evaluation.

##### Examples

An example is shown in Figure 20.

##### Comparison Methods

The result of finetuned ST-MoE-32B achieved the state-of-the-art (Zoph et al.,
2022).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.4 CivilComments

##### Description

Civil Comments (Koh et al., 2021; Borkan et al., 2019) is a classification
dataset to identify toxicity in online comments. The dataset is split by
demographic (All, Black, Christian, Female, LGBTQ, Male, Muslim, Other
religions, White). We sample 1,000 test instances from each subset following
HELM.

##### Implementation Details

CivilComments is a core scenario in HELM, and we use the implementation
parameters outlined in HELM. We prompt for an answer that is at most 5 tokens.
Following HELM, we report the macro-average accuracy of each subset below with
respect to the toxic/non-toxic classes, as well as the average accuracy across
all subsets. We found it necessary to map yes/no outputs to true/false during
zero-shot runs. Quasi-prefix exact match is used for evaluation.

##### Examples

Examples are shown in Figures 21, 22, 23, 24, 25, 26, 27, 28, 29.

##### Comparison Methods

The few-shot results on the GPT-3.5 Turbo model achieved the state-of-the-art
(Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.5 CNN/Daily Mail

##### Description

CNN/Daily Mail (See et al., 2017) is a generation dataset where the goal is to
summarize a selection of news articles. The dataset contains 466 test
instances.

##### Implementation Details

CNN/Daily Mail is a core scenario in HELM. We generate a summary of no more
than 128 tokens and use a temperature of 0.3. The ROUGE-2 metric is used for
evaluation.

##### Examples

An example is shown in Figure 30.

##### Comparison Methods

The result of the PEGASUSLARGE (HugeNews) model achieved the state-of-the-art
(Zhang et al., 2020).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.6 Coin Flip

##### Description

Coin Flip (Wei et al., 2022b) is a classification task where the objective is
to determine the state of a coin after a number of successive flips. We
consider Coin Flip a reasoning task because it is included in Kojima et al.
(2022). The dataset contains 500 test instances.

##### Implementation Details

We recreated the Coin Flip dataset following the details in Kojima et al.
(2022). We generate a response of no more than 4 tokens and use quasi-exact
match for evaluation.

##### Examples

An example is shown in Figure 31.

##### Comparison Methods

The result of the GPT-3.5 (text-davinci-002) model achieved the state-of-the-
art (Zhang et al., 2022b).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.7 CommonsenseQA

##### Description

CommonsenseQA (Talmor et al., 2019) is a classification (multi-choice) dataset
focusing on common sense. Because it is in Kojima et al. (2022), we consider
it a reasoning dataset. We sample 1,000 instances from the test set mirroring
the existing implementation in HELM.

##### Implementation Details

Though CommonsenseQA was already implemented into HELM, we added a separate
implementation to better align with Kojima et al. (2022). We use the joint
multiple-choice adapter in HELM rather than the separate multiple-choice
adapter. We generate a response of no more than 5 tokens, and grab the first
letter from the output and then use exact match for evaluation.

##### Examples

An example is shown in Figure 32.

##### Comparison Methods

The result of the DeBERTaV3-large model achieved the state-of-the-art (Xu et
al., 2022).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.8 Date Understanding

##### Description

Date Understanding (Srivastava et al., 2023; Suzgun et al., 2023) is a
classification (multi-choice) dataset from the BIG-bench benchmark (Srivastava
et al., 2023), which aims to assess pretrained language models’ capacity to
comprehend dates through inquiries about specific days’ dates. The dataset
contains 369 test instances.

##### Implementation Details

We use the Date Understanding instances from Suzgun et al. (2023). We generate
up to 60 tokens, though in practice, the generation ends much quicker when it
hits a stop token: either “\n” or “)”. As with other classification multi-
choice, we extract the first letter from the response. Quasi-exact match is
used as the evaluation metric.

##### Examples

An example is shown in Figure 33.

##### Comparison Methods

The few-shot CoT results on the PaLM 2 model achieved the state-of-the-art
(Anil et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.9 GSM8K

##### Description

GSM8K (Cobbe et al., 2021) is a generation dataset focused on mathematical
reasoning. Because it is in Kojima et al. (2022), we consider it a reasoning
dataset. We sample 1,000 test instances following HELM.

##### Implementation Details

GSM8K is included in HELM as a target scenario. We leverage this
implementation, which generates up to 400 new tokens. Responses undergo a
post-processing step where the first number in the output is extracted for
evaluation. The evaluation metric used is quasi-exact match.

##### Examples

An example is shown in Figure 34.

##### Comparison Methods

The result of the GPT-4 code interpreter achieved the state-of-the-art (Zhou
et al., 2023a).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.10 HellaSwag

##### Description

HellaSwag (Zellers et al., 2019) is a classification (multi-choice) dataset
comprised of 70,000 multiple-choice questions about grounded situations,
originating from ActivityNet and wikiHow domains, with human-verified
adversarial incorrect answers and one correct answer aimed at studying
grounded commonsense inference. We sample 1,000 test instances following HELM.

##### Implementation Details

We use the implementation of HellaSwag in HELM, but use the joint adapter to
mimic a standard multiple-choice question, rather than the separate adapter
which scores each reference separately using the sentence probability
normalized by the sentence length. We generate only a single token. During
zero-shot and zero-shot CoT runs, we include the following instructions from
HELM: “The following are multiple choice questions (with answers) about common
sense.” The exact match metric is used for evaluation. Like with other
classification (multi-choice) datasets, we parse out the first letter from the
response for evaluation.

##### Examples

An example is shown in Figure 35.

##### Comparison Methods

The few-shot results on the GPT-4 model achieved the state-of-the-art (OpenAI,
2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.11 IMDB

##### Description

IMDB (Maas et al., 2011) is a classification dataset comprising 50,000 movie
reviews, with 25,000 for training and 25,000 for testing, focused on binary
sentiment classification. We sample 1,000 test instances following HELM.

##### Implementation Details

IMDB is a core scenario in HELM, and we follow their implementation. We
generate up to 5 new tokens. During few-shot runs, we sampled 5 in-context
examples from the IMDB training set. Quasi-exact match is used for evaluation.

##### Examples

An example is shown in Figure 36.

##### Comparison Methods

The result of the ERNIE-DOC-Large model achieved the state-of-the-art (Ding et
al., 2021).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.12 Last Letter Concatenation

##### Description

Last Letter Concatenation (Wei et al., 2022b) is a generation task where the
objective is to concatenate the last letters of a series of two words. Because
it is in Kojima et al. (2022), we consider it a reasoning dataset. We generate
full names by randomly concatenating names from the top one-thousand first and
last names from name census data (https://namecensus.com/), as referenced in
Wei et al. (2022b). The dataset contains 500 test instances.

##### Implementation Details

We recreated the Last Letter Concatenation dataset following the details in
Kojima et al. (2022); Wei et al. (2022b). Unlike Kojima et al. (2022) which
does concatenation on 4 words, we only do so on 2. We generate a response of
no more than 4 tokens. We noticed that quasi-exact match unfairly penalized
zero-shot CoT runs, so for all runs we extract the answer directly from the
generated text, grabbing only the first 2 letters after stripping away white
space and punctuation.

##### Examples

An example is shown in Figure 37.

##### Comparison Methods

The few-shot CoT results on the PaLM-540B model achieved the state-of-the-art
(Wei et al., 2022b).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.13 MMLU

##### Description

MMLU (Hendrycks et al., 2021) is a classification (multi-choice) benchmark
comprised of 57 diverse subjects, ranging from elementary to professional
levels, encompassing STEM, humanities, social sciences, and specialized
fields. Following HELM, we focus on 5 subjects (Abstract Algebra, College
Chemistry, Computer Security, Econometrics, and US Foreign Policy) totaling
514 test instances.

##### Implementation Details

MMLU is a core HELM scenario, and we use HELM’s implementation of the dataset.
We prompt for a single token. HELM’s implementation includes the following
instructions, where “{subject}” is replaced with the subject of the dataset:
“The following are multiple choice questions (with answers) about {subject}.”
We prepend these instructions to instances during zero-shot and zero-shot CoT
runs. We parse out the first letter from the response, and the exact match
metric is used for evaluation.

##### Examples

Examples are shown in Figures 38, 39, 40, 41, 42.

##### Comparison Methods

The few-shot results on the Palmyra X (43B) model achieved the state-of-the-
art (Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.14 MS MARCO (Regular)

##### Description

MS MARCO (Bajaj et al., 2016; Nogueira et al., 2020; MacAvaney et al., 2021)
is a classification dataset containing real Bing search queries and nearly 9
million web passages, annotated for relevance. It consists of two tracks: the
Regular track uses binary RR@k metric, while the TREC track employs graded
annotations with NDCG as the main metric, featuring more extensively annotated
queries and passages with relevance grades. We sample 1,000 test instances for
the Regular track following HELM.

##### Implementation Details

We use the default HELM implementation for MS MARCO (Regular) and evaluate
using RR@10.

##### Examples

An example is shown in Figures 43.

##### Comparison Methods

The few-shot results on the Cohere Command beta (52.4B) model achieved the
state-of-the-art (Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.15 MS MARCO (TREC)

##### Description

This is the TREC track of MS MARCO (Bajaj et al., 2016; Nogueira et al., 2020;
MacAvaney et al., 2021) as introduced in Appendix A.4.14. The TREC track
contains 43 instances.

##### Implementation Details

We use the default HELM implementation for MS MARCO (TREC) and evaluate using
NDCG@10.

##### Examples

An example is shown in Figure 44.

##### Comparison Methods

The few-shot results on the Cohere Command beta (52.4B) model achieved the
state-of-the-art (Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.16 MultiArith

##### Description

MultiArith (Roy & Roth, 2015) is a generation dataset containing a
comprehensive collection of arithmetic word problems, encompassing various
mathematical operations and ranging in complexity. Because it is in Kojima et
al. (2022) as a reasoning task, we consider it a reasoning dataset. The
dataset contains 600 test instances.

##### Implementation Details

We prompt for up to 10 new tokens. As with other math generation datasets, we
extract the first number from the generated text and use that as the final
answer. Quasi-exact match is used for evaluation.

##### Examples

An example is shown in Figure 45.

##### Comparison Methods

The result of the self-consistency strategy on the GPT-3 (code-davinci-002)
model achieved the state-of-the-art (Wang et al., 2023c).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.17 NarrativeQA

##### Description

NarrativeQA (Kočiský et al., 2018) is a generation dataset focused on reading
comprehension where the model answers a question about a passage. The dataset
is comprised of 355 test instances.

##### Implementation Details

We use the implementation of NarrativeQA in HELM. We generate up to 100 new
tokens, and use F1 score for evaluation. For few-shot runs, we sampled 5 in-
context examples from the training set.

##### Examples

An example is shown in Figure 46.

##### Comparison Methods

The few-shot results on the Llama 2 (70B) model achieved the state-of-the-art
(Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.18 NaturalQuestions (closed-book)

##### Description

NaturalQuestions (Kwiatkowski et al., 2019) is a generation dataset comprised
of anonymized real search queries posed to Google. We focus on two subsets:
closed-book where the model is not given any external context, and open-book
long answer, where a passage is given prior to the question. We sample 1,000
test instances following HELM.

##### Implementation Details

We use the implementation of the Natural Questions dataset within HELM. Up to
300 new tokens are generated, and we use F1 score as the evaluation metric.

##### Examples

An example is shown in Figure 47.

##### Comparison Methods

The few-shot results on the Llama 2 (70B) model achieved the state-of-the-art
on the NaturalQuestions (closed-book) dataset (Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.19 NaturalQuestions (open-book)

##### Description

This is the open-book subset of NaturalQuestions (Kwiatkowski et al., 2019) as
introduced in Appendix A.4.18.

##### Implementation Details

We use the same implementation as introduced in Appendix A.4.18.

##### Examples

An example is shown in Figure 48.

##### Comparison Methods

The few-shot results on the text-davinci-003 model achieved the state-of-the-
art on the NaturalQuestions (open-book) dataset (Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.20 NewsQA

##### Description

NewsQA (Trischler et al., 2017) is a generation dataset that contains human-
generated question-answer pairs sourced from more than 10,000 CNN news
articles. We sample 1,000 test instances following HELM.

##### Implementation Details

We followed the instructions in HELM to collect and process the NewsQA
dataset. We generate up to 50 tokens. F1 score is used as the evaluation
metric.

##### Examples

Because of restrictive licensing, we cannot publish an example.

##### Comparison Methods

The result of the SpanBERT model achieved the state-of-the-art (Joshi et al.,
2020).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.21 OpenBookQA

##### Description

OpenBookQA (Mihaylov et al., 2018) is a classification (multi-choice) dataset
focused on commonsense knowledge utilization. It contains 500 test instances.

##### Implementation Details

We follow the implementation of OpenbookQA in HELM, but use the joint adapter
to mimic a standard multiple-choice question, rather than the separate adapter
which scores each reference separately using the sentence probability
normalized by the sentence length. We generate only a single token. During
zero-shot and zero-shot CoT runs, we include the following instructions from
HELM: “The following are multiple choice questions (with answers) about common
sense.” The first letter from the response is parsed out and the exact match
metric is used for evaluation.

##### Examples

An example is shown in Figure 49.

##### Comparison Methods

The result of the fine-tuning method with the self-consistency prompting
method on the PaLM-540B model achieved the state-of-the-art (Huang et al.,
2022).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.22 QuAC

##### Description

QuAC (Choi et al., 2018) is a generation dataset that facilitates information-
seeking dialogue modeling by presenting interactive conversations between a
student asking open-ended questions and a teacher providing short text
excerpts from a hidden Wikipedia passage. There are 1,000 test instances
following HELM.

##### Implementation Details

We use the default QuAC implementation in HELM. Responses are limited to 100
tokens, and F1 is used to measure accuracy.

##### Examples

An example is shown in Figure 50.

##### Comparison Methods

The result of the FLOWQA model achieved the state-of-the-art (Huang et al.,
2019).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.23 RAFT

##### Description

RAFT (Alex et al., 2021) is a classification benchmark designed to assess
language models’ performance across diverse domains. Specifically, we focus on
11 subsets, containing 40 test instances each.

##### Implementation Details

We use the default implementation of RAFT in HELM. Responses are limited to 30
tokens. For zero-shot and zero-shot CoT runs, we prepend subset-specific task
instructions provided in HELM. We report the average score across the 11
subsets using quasi-exact match as the evaluation metric.

##### Examples

Examples are shown in Figures 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61.

##### Comparison Methods

The few-shot results on the GPT-3.5 Turbo model achieved the state-of-the-art
(Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.24 Shuffled Objects

##### Description

Shuffled Objects (Srivastava et al., 2023; Suzgun et al., 2023) is a
classification (multi-choice) dataset of the BIG-bench benchmark. In this
dataset, objects initially owned by different people are swapped through
pairwise trades, and the model’s goal is to accurately determine which person
ends up with a particular object. There are 3 subtasks (three objects, five
objects, and seven objects) in this dataset. The five and seven objects
subtasks each contain 1000 instances, and the three object subtask contains
750 test instances.

##### Implementation Details

We use the Shuffled Objects instances from Suzgun et al. (2023). We generate
up to 60 tokens, though in practice, the generation ends much quicker when it
hits a stop token: either "\n" or ")". Unlike Kojima et al. (2022), we report
results on the three, five, and seven object cases, instead of only the three
object case. As with other classification (multi-choice) datasets, we parse
out the first letter in the response. Quasi-exact match is used as the
evaluation metric.

##### Examples

Examples are shown in Figures 63, 64, 65.

##### Comparison Methods

The few-shot CoT results on the PaLM 2 model achieved the state-of-the-art
(Anil et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.25 SingleEq

##### Description

SingleEq (Koncel-Kedziorski et al., 2016) is a generation dataset containing
math word problems requiring the use of single mathematical equations. Because
the dataset is included in Kojima et al. (2022), we consider it a reasoning
task in addition to generation. The dataset contains 508 test instances.

##### Implementation Details

We followed the instructions in Kojima et al. (2022) to collect the SingleEQ
dataset. We limit generated responses to 10 tokens. Like other math generation
datasets, we parse out the first number from the response, and then use quasi-
exact match to measure accuracy.

##### Examples

An example is shown in Figure 62.

##### Comparison Methods

The few-shot results on the GPT-4 model achieved the state-of-the-art (Zhao et
al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.26 StrategyQA

##### Description

StrategyQA (Geva et al., 2021) is a yes/no classification dataset, though we
follow the BIG-bench implementation which treats instances as multiple-choice
questions with two options (A. Yes, B. No). Because it is in Kojima et al.
(2022), we consider it a reasoning dataset. The dataset contains 458 test
instances.

##### Implementation Details

We use the StragetyQA implementation through BIG-bench. As a result, our
implementation varies from that in Kojima et al. (2022) in that the instances
are formulated as a classification (multi-choice) task, rather than an open-
ended classification task. We prompt for no more than 64 new tokens. We
extract the first letter from the response, and use exact match to measure
accuracy.

##### Examples

An example is shown in Figure 66.

##### Comparison Methods

The few-shot CoT results with self-consistency on the PaLM 2 model achieved
the state-of-the-art (Anil et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.27 SVAMP

##### Description

SVAMP (Patel et al., 2021) is a generation dataset focused on mathematical
reasoning containing 1,000 test instances. Because it is in Kojima et al.
(2022), we also consider it a reasoning dataset.

##### Implementation Details

We prompt for at most 10 tokens. Like other math generation datasets, we parse
the first number out of the response, and use quasi-exact match for
evaluation.

##### Examples

An example is shown in Figure 67.

##### Comparison Methods

The few-shot results on the GPT-4 model achieved the state-of-the-art (Zhao et
al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.28 TruthfulQA

##### Description

TruthfulQA (Lin et al., 2022) is a classification (multi-choice) task designed
to test LLMs’ propensity to dispense harmful information. The dataset contains
654 test instances.

##### Implementation Details

We use the implementation of TruthfulQA in HELM, which prompts for only a
single token. We extract the first letter and use exact match for evaluation.

##### Examples

An example is shown in Figure 68.

##### Comparison Methods

The few-shot results on the Palmyra X (43B) model achieved the state-of-the-
art (Liang et al., 2023).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

####  A.4.29 XSUM

##### Description

XSUM (Narayan et al., 2018) is a generation dataset focused on summarization.
The dataset contains 518 test instances.

##### Implementation Details

XSUM is a core scenario in HELM. We generate a summary of no more than 128
tokens and use a temperature of 0.3. The ROUGE-2 metric is used for
evaluation.

##### Examples

An example is shown in Figure 69.

##### Comparison Methods

The result of the PEGASUSLARGE (HugeNews) model achieved the state-of-the-art
(Zhang et al., 2020).

##### Results

The results are shown in Table LABEL:table:results-numerical-table.

##  Appendix B Additional Results

Table LABEL:table:results-numerical-table presents the results of zero-shot,
zero-shot CoT, and zero-shot AgentInstruct on Vicuna-13b, Llama-2-70b-chat,
and GPT-3.5 Turbo, as well as the smaller 7b and 13b sizes of Llama-2-chat,
across 29 datasets and 53 subsets. Also, scores of task-specific models are
included for comparison. If there are multiple subsets, the score of task-
specific models is only given for all subsets on average, not for individual
subsets. Figure 17 plots the results for each dataset on Vicuna-13b,
Llama-2-70b-chat, and GPT-3.5 Turbo, and Table B gives the average for each of
the five models on each task category. Similar to our three main models,
Llama-2-7b-chat and Llama-2-13b-chat with zero-shot AgentInstruct outperform
both zero-shot and zero-shot CoT, on average, in each category of tasks.

Table 5: Results of zero-shot, zero-shot CoT, and zero-shot AgentInstruct on Vicuna-13b, Llama-2-7b-chat, Llama-2-13b-chat, Llama-2-70b-chat, and GPT-3.5 Turbo across 29 datasets and 53 subsets. Dataset | Subset | Task-Specific Model | Model | Results
---|---|---|---|---
Zero-Shot(%) | Zero-Shot CoT(%) | Zero-Shot AgentInstruct(%)
AddSub (Koncel-Kedziorski et al., 2016) | - |  95.7 (Zhao et al., 2023) | Vicuna-13b | 10.9 | 29.1 | 60.8
Llama-2-7b-chat | 50.4 | 56.7 | 63.8
Llama-2-13b-chat | 52.7 | 52.4 | 65.1
Llama-2-70b-chat | 36.2 | 73.2 | 79.5
GPT-3.5 Turbo | 78.7 | 79.0 | 88.1
AQuA (Ling et al., 2017) | - |  79.9 (Zheng et al., 2023) | Vicuna-13b | 25.6 | 16.5 | 24.0
Llama-2-7b-chat | 20.9 | 1.2 | 22.0
Llama-2-13b-chat | 18.9 | 24.0 | 25.2
Llama-2-70b-chat | 19.7 | 35.8 | 37.8
GPT-3.5 Turbo | 16.5 | 60.2 | 57.9
BoolQ (Clark et al., 2019) | - |  92.4 (Zoph et al., 2022) | Vicuna-13b | 77.2 | 71.0 | 74.2
Llama-2-7b-chat | 77.5 | 76.4 | 72.8
Llama-2-13b-chat | 79.9 | 76.1 | 76.1
Llama-2-70b-chat | 78.5 | 83.2 | 84.6
GPT-3.5 Turbo | 74.0 | 82.1 | 83.6
| Average across all 9 subsets |  68.4 (Liang et al., 2023) | Vicuna-13b | 56.1 | 51.2 | 60.6
| Llama-2-7b-chat | 57.7 | 54.3 | 58.1
| Llama-2-13b-chat | 66.1 | 52.0 | 64.7
| Llama-2-70b-chat | 66.9 | 57.4 | 65.5
| GPT-3.5 Turbo | 45.0 | 64.4 | 67.5
| All | - | Vicuna-13b | 60.3 | 51.6 | 63.6
| Llama-2-7b-chat | 65.3 | 58.8 | 59.6
| Llama-2-13b-chat | 63.9 | 59.1 | 59.0
| Llama-2-70b-chat | 64.8 | 61.4 | 70.2
| GPT-3.5 Turbo | 52.1 | 67.7 | 68.8
| Black | - | Vicuna-13b | 55.0 | 51.4 | 61.1
| Llama-2-7b-chat | 54.7 | 51.1 | 64.3
| Llama-2-13b-chat | 71.1 | 52.7 | 68.1
| Llama-2-70b-chat | 71.9 | 58.6 | 68.2
| GPT-3.5 Turbo | 44.5 | 66.7 | 63.0
| Christian | - | Vicuna-13b | 59.5 | 54.4 | 61.4
| Llama-2-7b-chat | 62.0 | 58.7 | 66.6
| Llama-2-13b-chat | 66.4 | 48.4 | 62.8
| Llama-2-70b-chat | 65.1 | 56.2 | 72.7
| GPT-3.5 Turbo | 53.2 | 67.5 | 74.7
CivilComments (Koh et al., 2021; Borkan et al., 2019) | Female | - | Vicuna-13b | 55.1 | 48.3 | 65.1
Llama-2-7b-chat | 56.3 | 54.6 | 57.1
Llama-2-13b-chat | 68.6 | 52.6 | 71.0
Llama-2-70b-chat | 72.2 | 55.8 | 55.1
GPT-3.5 Turbo | 47.1 | 67.0 | 71.2
LGBTQ | - | Vicuna-13b | 55.1 | 49.0 | 57.3
Llama-2-7b-chat | 53.8 | 56.6 | 56.1
Llama-2-13b-chat | 63.4 | 49.5 | 64.0
Llama-2-70b-chat | 68.9 | 56.7 | 64.7
GPT-3.5 Turbo | 41.6 | 62.3 | 65.6
| Male | - | Vicuna-13b | 57.5 | 51.0 | 54.4
| Llama-2-7b-chat | 59.4 | 54.9 | 55.6
| Llama-2-13b-chat | 67.1 | 55.7 | 60.8
| Llama-2-70b-chat | 66.8 | 56.3 | 66.7
| GPT-3.5 Turbo | 49.2 | 66.0 | 67.7
| Muslim | - | Vicuna-13b | 52.6 | 51.8 | 60.0
| Llama-2-7b-chat | 55.3 | 52.7 | 53.2
| Llama-2-13b-chat | 63.5 | 52.3 | 67.0
| Llama-2-70b-chat | 64.3 | 55.7 | 65.4
| GPT-3.5 Turbo | 40.4 | 61.0 | 68.2
| Other Religions | - | Vicuna-13b | 57.9 | 52.1 | 63.6
| Llama-2-7b-chat | 60.6 | 51.7 | 58.9
| Llama-2-13b-chat | 69.0 | 46.5 | 68.1
| Llama-2-70b-chat | 67.3 | 56.3 | 67.0
| GPT-3.5 Turbo | 41.2 | 60.9 | 68.4
| White | - | Vicuna-13b | 51.7 | 51.8 | 58.6
| Llama-2-7b-chat | 52.0 | 49.9 | 51.0
| Llama-2-13b-chat | 61.8 | 51.6 | 61.7
| Llama-2-70b-chat | 60.9 | 59.4 | 59.2
| GPT-3.5 Turbo | 36.1 | 60.3 | 60.2
CNN/Daily Mail (See et al., 2017) | - |  21.5 (Zhang et al., 2020) | Vicuna-13b | 15.0 | 12.3 | 12.6
Llama-2-7b-chat | 14.1 | 13.8 | 13.4
Llama-2-13b-chat | 14.2 | 12.5 | 13.0
Llama-2-70b-chat | 14.4 | 13.4 | 13.4
GPT-3.5 Turbo | 17.4 | 15.8 | 15.0
Coin Flip (Wei et al., 2022b) | - |  99.9 (Zhang et al., 2022b) | Vicuna-13b | 49.0 | 27.0 | 52.4
Llama-2-7b-chat | 0.0 | 15.8 | 62.2
Llama-2-13b-chat | 0.0 | 18.8 | 65.0
Llama-2-70b-chat | 0.0 | 5.6 | 99.4
GPT-3.5 Turbo | 32.6 | 61.8 | 95.2
CommonsenseQA (Talmor et al., 2019) | - |  91.2 (Xu et al., 2022) | Vicuna-13b | 60.4 | 48.0 | 61.8
Llama-2-7b-chat | 60.5 | 16.4 | 54.3
Llama-2-13b-chat | 63.1 | 60.0 | 62.6
Llama-2-70b-chat | 71.6 | 62.4 | 72.8
GPT-3.5 Turbo | 77.2 | 72.1 | 74.1
Date Understanding (Srivastava et al., 2023) | - |  91.2 (Anil et al., 2023) | Vicuna-13b | 63.2 | 50.8 | 60.8
Llama-2-7b-chat | 38.8 | 46.4 | 46.4
Llama-2-13b-chat | 45.2 | 57.2 | 56.4
Llama-2-70b-chat | 56.8 | 68.4 | 70.8
GPT-3.5 Turbo | 35.6 | 70.8 | 71.6
GSM8K (Cobbe et al., 2021) | - |  97.0 (Zhou et al., 2023a) | Vicuna-13b | 2.3 | 14.7 | 19.3
Llama-2-7b-chat | 3.1 | 30.0 | 21.9
Llama-2-13b-chat | 5.2 | 32.6 | 31.6
Llama-2-70b-chat | 2.2 | 54.6 | 51.6
GPT-3.5 Turbo | 8.4 | 77.8 | 73.4
HellaSwag (Zellers et al., 2019) | - |  95.3 (OpenAI, 2023) | Vicuna-13b | 45.5 | 39.9 | 49.9
Llama-2-7b-chat | 41.5 | 42.8 | 33.5
Llama-2-13b-chat | 59.7 | 43.0 | 46.8
Llama-2-70b-chat | 65.0 | 61.9 | 60.6
GPT-3.5 Turbo | 75.8 | 57.1 | 73.5
IMDB (Maas et al., 2011) | - |  97.1 (Ding et al., 2021) | Vicuna-13b | 5.5 | 69.5 | 94.2
Llama-2-7b-chat | 29.9 | 77.7 | 90.1
Llama-2-13b-chat | 41.2 | 78.4 | 90.6
Llama-2-70b-chat | 43.5 | 89.0 | 94.0
GPT-3.5 Turbo | 84.2 | 88.9 | 93.9
Last Letter Concatenation (Wei et al., 2022b) | - |  99.4 (Wei et al., 2022b) | Vicuna-13b | 2.4 | 3.0 | 18.8
Llama-2-7b-chat | 0.2 | 3.2 | 25.4
Llama-2-13b-chat | 1.8 | 3.4 | 31.8
Llama-2-70b-chat | 3.0 | 3.4 | 35.6
GPT-3.5 Turbo | 4.8 | 3.4 | 99.8
| Average across all 5 subsets |  60.9 (Liang et al., 2023) | Vicuna-13b | 44.1 | 44.2 | 44.5
| Llama-2-7b-chat | 41.5 | 37.8 | 35.1
| Llama-2-13b-chat | 46.8 | 45.8 | 46.6
| Llama-2-70b-chat | 53.3 | 52.2 | 50.3
| GPT-3.5 Turbo | 58.6 | 62.3 | 58.9
| Abstract Algebra | - | Vicuna-13b | 31.0 | 30.0 | 29.0
| Llama-2-7b-chat | 25.0 | 24.0 | 18.0
| Llama-2-13b-chat | 30.0 | 30.0 | 33.0
| Llama-2-70b-chat | 37.0 | 28.0 | 35.0
| GPT-3.5 Turbo | 38.0 | 50.0 | 36.0
MMLU (Hendrycks et al., 2021) | College Chemistry | - | Vicuna-13b | 37.0 | 37.0 | 29.0
Llama-2-7b-chat | 26.0 | 30.0 | 26.0
Llama-2-13b-chat | 31.0 | 35.0 | 29.0
Llama-2-70b-chat | 39.0 | 41.0 | 32.0
GPT-3.5 Turbo | 43.0 | 50.0 | 55.0
Computer Security | - | Vicuna-13b | 60.0 | 60.0 | 63.0
Llama-2-7b-chat | 57.0 | 56.0 | 48.0
Llama-2-13b-chat | 66.0 | 64.0 | 68.0
Llama-2-70b-chat | 64.0 | 73.0 | 72.0
GPT-3.5 Turbo | 75.0 | 80.0 | 78.0
| Econometrics | - | Vicuna-13b | 23.7 | 29.8 | 31.6
| Llama-2-7b-chat | 30.7 | 22.8 | 25.4
| Llama-2-13b-chat | 27.2 | 28.1 | 27.2
| Llama-2-70b-chat | 39.5 | 43.0 | 40.4
| GPT-3.5 Turbo | 53.0 | 44.7 | 44.7
| US Foreign Policy | - | Vicuna-13b | 69.0 | 64.0 | 70.0
| Llama-2-7b-chat | 69.0 | 56.0 | 58.0
| Llama-2-13b-chat | 80.0 | 72.0 | 76.0
| Llama-2-70b-chat | 87.0 | 76.0 | 72.0
| GPT-3.5 Turbo | 84.0 | 87.0 | 81.0
MS MARCO (Bajaj et al., 2016) | Regular |  47.2 (Liang et al., 2023) | Vicuna-13b | 11.7 | 14.0 | 19.0
Llama-2-7b-chat | 17.5 | 10.6 | 19.2
Llama-2-13b-chat | 7.7 | 12.4 | 18.8
Llama-2-70b-chat | 15.6 | 13.2 | 20.4
GPT-3.5 Turbo | 26.0 | 20.5 | 30.0
TREC |  76.2 (Liang et al., 2023) | Vicuna-13b | 27.9 | 31.0 | 34.6
Llama-2-7b-chat | 32.8 | 26.3 | 33.3
Llama-2-13b-chat | 20.9 | 31.1 | 38.9
Llama-2-70b-chat | 37.8 | 28.7 | 33.5
GPT-3.5 Turbo | 47.7 | 38.1 | 54.1
MultiArith (Roy & Roth, 2015) | - |  100.0 (Wang et al., 2023c) | Vicuna-13b | 3.8 | 33.7 | 56.3
Llama-2-7b-chat | 4.8 | 60.5 | 65.2
Llama-2-13b-chat | 7.3 | 73.0 | 76.2
Llama-2-70b-chat | 5.8 | 88.3 | 88.2
GPT-3.5 Turbo | 50.3 | 96.7 | 96.5
NarrativeQA (Kočiský et al., 2018) | - |  77.0 (Liang et al., 2023) | Vicuna-13b | 40.3 | 45.9 | 41.4
Llama-2-7b-chat | 38.8 | 55.6 | 61.1
Llama-2-13b-chat | 50.0 | 61.4 | 64.1
Llama-2-70b-chat | 45.9 | 62.3 | 65.0
GPT-3.5 Turbo | 49.1 | 67.7 | 65.2
NaturalQuestions (Kwiatkowski et al., 2019) | NaturalQuestions (closed-book) |  45.8 (Liang et al., 2023) | Vicuna-13b | 14.6 | 18.3 | 27.5
Llama-2-7b-chat | 15.3 | 21.2 | 27.3
Llama-2-13b-chat | 18.7 | 26.7 | 33.2
Llama-2-70b-chat | 17.0 | 33.1 | 35.0
GPT-3.5 Turbo | 23.5 | 40.6 | 39.7
NaturalQuestions (open-book) |  77.0 (Liang et al., 2023) | Vicuna-13b | 39.3 | 47.2 | 55.5
Llama-2-7b-chat | 43.9 | 56.3 | 59.4
Llama-2-13b-chat | 43.3 | 61.7 | 64.1
Llama-2-70b-chat | 38.4 | 57.7 | 65.9
GPT-3.5 Turbo | 41.5 | 63.9 | 42.7
NewsQA (Trischler et al., 2017) | - |  73.6 (Joshi et al., 2020) | Vicuna-13b | 37.2 | 27.2 | 33.7
Llama-2-7b-chat | 44.0 | 40.7 | 49.0
Llama-2-13b-chat | 40.9 | 45.5 | 51.5
Llama-2-70b-chat | 39.5 | 47.1 | 50.7
GPT-3.5 Turbo | 30.9 | 52.0 | 50.2
OpenBookQA (Mihaylov et al., 2018) | - |  94.4 (Huang et al., 2022) | Vicuna-13b | 61.6 | 54.8 | 57.6
Llama-2-7b-chat | 56.8 | 57.0 | 56.2
Llama-2-13b-chat | 61.6 | 63.8 | 62.0
Llama-2-70b-chat | 73.8 | 75.0 | 70.4
GPT-3.5 Turbo | 81.6 | 78.6 | 79.2
QuAC (Choi et al., 2018) | - |  64.1 (Huang et al., 2019) | Vicuna-13b | 37.5 | 17.9 | 16.8
Llama-2-7b-chat | 40.8 | 18.0 | 13.4
Llama-2-13b-chat | 42.5 | 19.3 | 14.0
Llama-2-70b-chat | 46.8 | 15.6 | 16.6
GPT-3.5 Turbo | 38.6 | 33.1 | 34.7
| Average across all 11 subsets |  76.8 (Liang et al., 2023) | Vicuna-13b | 19.1 | 54.5 | 51.4
| Llama-2-7b-chat | 21.8 | 39.1 | 47.3
| Llama-2-13b-chat | 17.3 | 43.0 | 56.6
| Llama-2-70b-chat | 6.8 | 19.8 | 57.0
| GPT-3.5 Turbo | 68.2 | 78.0 | 68.9
| ADE Corpus v2 | - | Vicuna-13b | 5.0 | 77.5 | 75.0
| Llama-2-7b-chat | 0.0 | 60.0 | 45.0
| Llama-2-13b-chat | 62.5 | 75.0 | 57.5
| Llama-2-70b-chat | 0.0 | 30.0 | 37.5
| GPT-3.5 Turbo | 82.5 | 82.5 | 80.0
| Banking 77 | - | Vicuna-13b | 15.0 | 12.5 | 12.5
| Llama-2-7b-chat | 20.0 | 15.0 | 2.0
| Llama-2-13b-chat | 2.5 | 10.0 | 2.5
| Llama-2-70b-chat | 7.5 | 17.5 | 12.5
| GPT-3.5 Turbo | 45.0 | 65.0 | 45.0
| Neurips Impact Statement | - | Vicuna-13b | 0.0 | 75.0 | 70.0
| Llama-2-7b-chat | 7.5 | 7.5 | 45.0
| Llama-2-13b-chat | 0.0 | 47.5 | 72.5
| Llama-2-70b-chat | 0.0 | 0.0 | 90.0
| GPT-3.5 Turbo | 20.0 | 87.5 | 90.0
| One Stop English | - | Vicuna-13b | 17.5 | 25.0 | 22.5
| Llama-2-7b-chat | 27.5 | 32.5 | 25.0
| Llama-2-13b-chat | 17.5 | 27.5 | 37.5
| Llama-2-70b-chat | 0.0 | 32.5 | 37.5
| GPT-3.5 Turbo | 35.0 | 35.0 | 25.0
RAFT (Alex et al., 2021) | Overruling | - | Vicuna-13b | 20.0 | 75.0 | 87.5
Llama-2-7b-chat | 2.5 | 55.0 | 77.5
Llama-2-13b-chat | 0.0 | 37.5 | 80.0
Llama-2-70b-chat | 0.0 | 0.0 | 85.0
GPT-3.5 Turbo | 95.0 | 97.5 | 95.0
Semiconductor Org Types | - | Vicuna-13b | 30.0 | 25.0 | 40.0
Llama-2-7b-chat | 42.5 | 57.5 | 67.5
Llama-2-13b-chat | 0.0 | 52.5 | 90.0
Llama-2-70b-chat | 32.5 | 30.0 | 92.5
GPT-3.5 Turbo | 92.5 | 95.0 | 95.0
| Systematic Review Inclusion | - | Vicuna-13b | 22.5 | 42.5 | 2.5
| Llama-2-7b-chat | 65.0 | 40.0 | 15.0
| Llama-2-13b-chat | 25.0 | 57.5 | 12.5
| Llama-2-70b-chat | 15.0 | 2.5 | 7.5
| GPT-3.5 Turbo | 85.0 | 95.0 | 12.5
| Tai Safety Research | - | Vicuna-13b | 55.0 | 57.5 | 55.0
| Llama-2-7b-chat | 57.5 | 60.0 | 45.0
| Llama-2-13b-chat | 77.5 | 57.5 | 57.5
| Llama-2-70b-chat | 20.0 | 80.0 | 60.0
| GPT-3.5 Turbo | 77.5 | 72.5 | 65.0
| Terms of Service | - | Vicuna-13b | 45.0 | 72.5 | 47.5
| Llama-2-7b-chat | 17.5 | 40.0 | 55.0
| Llama-2-13b-chat | 0.0 | 20.0 | 50.0
| Llama-2-70b-chat | 0.0 | 15.0 | 35.0
| GPT-3.5 Turbo | 85.0 | 90.0 | 85.0
| Tweet Eval Hate | - | Vicuna-13b | 0.0 | 70.0 | 70.0
| Llama-2-7b-chat | 0.0 | 32.5 | 67.5
| Llama-2-13b-chat | 0.0 | 40.0 | 80.0
| Llama-2-70b-chat | 0.0 | 0.0 | 85.0
| GPT-3.5 Turbo | 47.5 | 50.0 | 80.0
| Twitter Complaints | - | Vicuna-13b | 0.0 | 67.5 | 82.5
| Llama-2-7b-chat | 0.0 | 30.0 | 57.5
| Llama-2-13b-chat | 5.0 | 47.5 | 82.5
| Llama-2-70b-chat | 0.0 | 10.0 | 85.0
| GPT-3.5 Turbo | 85.0 | 87.5 | 85.0
| Average across all 3 subsets |  79.3 (Anil et al., 2023) | Vicuna-13b | 20.0 | 20.1 | 18.0
| Llama-2-7b-chat | 13.5 | 21.5 | 23.5
| Llama-2-13b-chat | 7.6 | 22.7 | 23.9
| Llama-2-70b-chat | 8.0 | 38.7 | 28.9
| GPT-3.5 Turbo | 25.1 | 39.1 | 44.3
Shuffled Objects (Srivastava et al., 2023) | Five Objects | - | Vicuna-13b | 13.6 | 18.0 | 16.8
Llama-2-7b-chat | 17.6 | 17.2 | 22.0
Llama-2-13b-chat | 4.0 | 18.8 | 21.6
Llama-2-70b-chat | 6.4 | 40.0 | 28.0
GPT-3.5 Turbo | 18.8 | 35.6 | 34.8
Seven Objects | - | Vicuna-13b | 15.2 | 15.6 | 10.8
Llama-2-7b-chat | 10.8 | 14.4 | 12.8
Llama-2-13b-chat | 6.8 | 16.8 | 16.8
Llama-2-70b-chat | 2.8 | 24.4 | 17.2
GPT-3.5 Turbo | 16.4 | 20.8 | 41.2
| Three Objects | - | Vicuna-13b | 31.2 | 26.8 | 26.4
| Llama-2-7b-chat | 12.0 | 32.8 | 35.6
| Llama-2-13b-chat | 12.0 | 32.4 | 33.2
| Llama-2-70b-chat | 14.8 | 51.6 | 41.6
| GPT-3.5 Turbo | 40.0 | 60.8 | 56.8
SingleEq (Koncel-Kedziorski et al., 2016) | - |  98.8 (Zhao et al., 2023) | Vicuna-13b | 11.0 | 29.1 | 63.4
Llama-2-7b-chat | 44.3 | 66.1 | 63.6
Llama-2-13b-chat | 46.9 | 51.0 | 65.0
Llama-2-70b-chat | 31.9 | 78.1 | 78.5
GPT-3.5 Turbo | 77.4 | 90.4 | 90.6
StrategyQA (Geva et al., 2021) | - |  90.4 (Anil et al., 2023) | Vicuna-13b | 57.0 | 49.1 | 55.0
Llama-2-7b-chat | 53.5 | 55.2 | 52.4
Llama-2-13b-chat | 56.3 | 63.1 | 61.1
Llama-2-70b-chat | 60.9 | 68.6 | 67.2
GPT-3.5 Turbo | 59.2 | 67.7 | 69.0