# Rationale-Augmented Ensembles in Language Models

Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Denny Zhou
Google Research, Brain Team
<EMAIL_ADDRESS>

###### Abstract

Recent research has shown that rationales, or step-by-step chains of thought,
can be used to improve performance in multi-step reasoning tasks. We
reconsider rationale-augmented prompting for few-shot in-context learning,
where (input $\rightarrow$ output) prompts are expanded to (input, rationale
$\rightarrow$ output) prompts. For rationale-augmented prompting we
demonstrate how existing approaches, which rely on manual prompt engineering,
are subject to sub-optimal rationales that may harm performance. To mitigate
this brittleness, we propose a unified framework of rationale-augmented
ensembles, where we identify rationale sampling in the output space as the key
component to robustly improve performance. This framework is general and can
easily be extended to common natural language processing tasks, even those
that do not traditionally leverage intermediate steps, such as question
answering, word sense disambiguation, and sentiment analysis. We demonstrate
that rationale-augmented ensembles achieve more accurate and interpretable
results than existing prompting approaches—including standard prompting
without rationales and rationale-based chain-of-thought prompting—while
simultaneously improving interpretability of model predictions through the
associated rationales.

##  1 Introduction

Recent progress on improving few-shot in-context learning in pretrained large
language models has been achieved by expanding prompt exemplars with
rationales, delivering successes in a variety of natural language reasoning
tasks (Wei et al., 2022; Chowdhery et al., 2022; Lampinen et al., 2022;
Creswell et al., 2022; Kojima et al., 2022; Jung et al., 2022). These
prompting-based approaches typically adopt manually-written rationales and
therefore rely on the quality of prompt engineering, which usually does not
ensure optimal rationales are provided for a given task. Previous work has
also shown that “rationales” can be useful for supervised learning in natural
language tasks when added in the training data (Zaidan et al., 2007; Ling et
al., 2017; Cobbe et al., 2021; Zelikman et al., 2022), but it remains unclear
whether such rationales can be reliably useful in few-shot in-context learning
(Ye & Durrett, 2022).

In this paper, we investigate the role of rationales in few-shot in-context
learning by conducting a systematic study over a wide range of NLP tasks. In
particular, we seek to answer the following questions: (1) Why do rationales
sometimes hurt task performance in few-shot learning? and (2) How can one
reliably leverage rationales in few-shot learning for general natural language
tasks?

Below we show that, when shifting from the simpler paradigm of (input
$\rightarrow$ output) prompts to expanded (input, rationale $\rightarrow$
output) prompts, there is indeed a large variance in final task performance
for few-shot in-context learning. We identify the primary source of
sensitivity as the sub-optimality of the rationales used for prompting. To
overcome such sub-optimality, we develop a unified framework of rationale-
augmented ensembles, where the idea is to aggregate over multiple rationales
generated from the language model to reduce the brittleness of the results.
Ensemble aggregation can be achieved in a few different ways depending on how
randomness over the rationales is introduced in the input or the output space,
including (1) self-consistency, where existing work (Wang et al., 2022) has
shown that task performance can be improved by sampling multiple language
model outputs for ensembling, (2) prompt-order ensembling, where previous work
(Lu et al., 2021; Zhao et al., 2021) has shown that task performance is
sensitive to the order of the exemplars in the prompts, and (3) input-
rationale ensembling, where human-written rationales can be replaced by model-
generated rationales, leveraging the ability of language models to generate
high-quality explanations (Wiegreffe et al., 2022). Figure 1 provides an
overview of rationale-augmented ensembling approaches.

Figure 1: An overview of different ways of composing rationale-augmented
ensembles, depending on how the randomness of rationales is introduced. Here
$q$, $a$, $r$ correspond to question, answer, and rationale, respectively.
Rationales are human-written unless specified as model-generated.

A key finding of this study is that rationale sampling in the output space is
a central aspect of rationale-augmented ensembles contributing to their
success. That is, regardless of how the input or the prompt vary, task
performance is best improved when sufficient diversity is introduced by
sampling rationales from the language model’s decoder. We also find that
rationale-augmented ensembles reliably outperform existing rationale-based
few-shot and zero-shot prompting methods (Wei et al., 2022; Kojima et al.,
2022) across a variety of natural language processing tasks. Moreover, in
cases where human-written rationales hurt task performance due to the sub-
optimality of the rationales, rationale-augmented ensembling is able to fill
the gap and reliably outperform standard few-shot prompting (Brown et al.,
2020) on most tasks.

Perhaps surprisingly, we also find that the proposed framework can be used to
improve few-shot learning in common natural language processing tasks, even
including tasks where explicit intermediate steps might not be necessary, such
as question answering (BoolQ; Clark et al., 2019), word sense disambiguation
(WiC; Pilehvar & Camacho-Collados, 2019), sentiment analysis (SST-2; Socher et
al., 2013), and paraphrase identification (QQP; Iyer et al., 2017). We
conjecture that, in principle, any natural language processing task can be
usefully augmented with “rationales” that represent the thought processes
needed to achieve accurate and interpretable results in few-shot in-context
learning.

Existing work on interpretability usually focuses on improving the explanation
of model predictions via supervised learning, which requires large amounts of
human labeled explanations to be collected (Zaidan et al., 2007; Camburu et
al., 2018; Rajani et al., 2019; Narang et al., 2020), while remaining agnostic
to improving final task performance. In contrast, we show that the framework
proposed in this paper can leverage very few human-written rationales (as
$K$-shot exemplars where $K$ is usually very small, e.g., 3 to 6) and still
generate ensembles that can improve task performance significantly. The
proposed framework does not require additional fine-tuning (Thoppilan et al.,
2022; Zelikman et al., 2022), verifiers (Cobbe et al., 2021), calibrators (Ye
& Durrett, 2022), or any use of an auxiliary dataset (Zelikman et al., 2022;
Li et al., 2022), making it applicable to any off-the-shelf large language
model. As a general approach to obtaining more accurate and more interpretable
natural language understanding, rationale-augmented ensembles also provide
more accurate assessments of the performance gains contributed by rationales
in few-shot in-context learning.

##  2 Rationale-Augmented Ensembles in Language Models

We investigate the role of rationales in few-shot in-context learning, first
interrogating the sensitivity of final performance to rationale quality, then
developing a unified perspective on rationale-augmented ensembles that seek to
reduce sensitivity and improve final performance.

###  2.1 Optimality of the rationales in few-shot learning.

Given that rationale-augmented prompting has been shown to exhibit variable
performance (Wei et al., 2022; Ye & Durrett, 2022), we first investigate the
sensitivity of task performance to rationale quality across a range of natural
language tasks, including e-SNLI (Camburu et al., 2018), BoolQ (Clark et al.,
2019), WiC (Pilehvar & Camacho-Collados, 2019), and SST-2 (Socher et al.,
2013), finding that human-generated rationales can indeed be sub-optimal.

For each task, we choose $K$ (4 to 6) exemplars from the training set,
manually produce a set of rationales for each exemplar, then use these as
seeds to generate additional rationales from the language model: we leave one
question from the exemplars out, and use the rest of the exemplars with human-
written rationales as prompts, then we can sample from the language model’s
decoder to obtain a large number of generated rationales for this
question.111Specifically, we sample 1,024 rationales for each exemplar and
only keep those where the final answer is consistent with the ground truth
answer. Each new prompt is then composed as follows: for each of the $K$
exemplars, we replace its human-written rationale with a random sample from
the generated rationales, while keeping the rationales of the other $K-1$
exemplars fixed. We repeat this for every exemplar and report the final task
performance using the new prompts in Figure 2 (denoted as sampled-r-$k$ if the
$k$-th rationale is replaced).

Figure 2: The performance varies depending on which rationales are used in the
prompts for few-shot in-context learning. The exemplars in the prompts are
exactly the same, only the rationales differ. The performance is evaluated
with accuracy on the greedy decoded output using PaLM-540B (Chowdhery et al.,
2022).

First, one can observe that compared with standard few-shot prompting (“no-
rationale”), the addition of human-written rationales does not always yield
better performances. Moreover, the performance induced by sampled-rationales
exhibits substantial variance, implying that the quality of the rationales in
the prompts has a significant effect on final performance. Often the sampled
rationales exhibit better performance than the human-written ones, indicating
that manually provided rationales can be far from “optimal” in terms of task
performance. Table 2.1 shows examples of human-written rationales and two
model-generated rationales for the same question, demonstrating that the model
is able to generate diverse but reasonable rationales using its pre-trained
knowledge.

o 1 X[l] [e-SNLI] Premise: “Children smiling and waving at camera." Based on
this premise, can we conclude the hypothesis “The kids are frowning." is true?
---
Human-written rationale (crowd-sourced): One cannot be smiling and frowning at
the same time.
Model-generated rationale: Smiling and waving is not frowning. $\mid\mid$
’Smiling’ and ’frowning’ are opposites.
[BoolQ] can you use oyster card at epsom station?
Human-written rationale: Epsom railway station serves the town of Epsom in
Surrey and is not in the London Oyster card zone.
Model-generated rationale: Epsom Station is not currently a part of the Oyster
Card Network. $\mid\mid$ ’Oyster card’ is a pre-pay card for public transport
in London. Epsom Station is in Surrey.
[WiC] Approach a task. To approach the city. Q: Is the word “approach" used in
the same way in the two sentences above?
Human-written rationale: The first “approach" means “deal with", the second
“approach" means “come near".
Model-generated rationale: 1) To tackle or deal with a task. 2) To come near.
$\mid\mid$ ’To approach the city’ means “to get close to the city". ’Approach
a task’ means “to deal with a task".
[QQP] Is the following question “What can one do after MBBS?" the same as
“What do i do after my MBBS?"
Human-written rationale: Both are asking what can a person do after MBBS.
Model-generated rationale: ‘One’ means the same as ‘I’. $\mid\mid$ ’What can
one do’ and ’what do I do’ mean the same thing in the context of the question.
[SST-2] What is the sentiment of the following sentence? “contains no wit ,
only labored gags"
Human-written rationale: “contains no wit" is clearly a negative sentiment.
Model-generated rationale: “no wit" and “labored gags" both mean negative
sentiments. $\mid\mid$ ’no wit’ indicates something negative and “only labored
gags" is also negative.

Table 1: Examples of human-written rationale and model-generated rationales
from PaLM-540B (Chowdhery et al., 2022), representing multiple ways to reason
about the same question. These examples show that many common NLP tasks can be
augmented with rationales, even when the task itself does not require explicit
intermediate steps.

#### Simply including a rationale does not always improve task performance.

From Figure 2 one can also see that, due to the sub-optimality of the
rationales used, task performance can degrade when rationales are added to
prompts in few-shot in-context learning. For example, on e-SNLI, WiC and
SST-2, the performance achieved by written-rationales is significantly worse
than standard few-shot prompting without rationales, consistent with the
findings in (Ye & Durrett, 2022).

###  2.2 Rationale-augmented ensembles

Given that determining “optimal” rationales for few-shot in-context learning
is difficult,222A line of existing work uses a train/validation set to
determine the optimal prompts (either discrete or continuous), e.g., Lester et
al. (2021); Gao et al. (2021). Such a setting is closer to fine-tuning rather
than few-shot learning, due to the use of an additional dataset for
performance validation. it is natural to consider the use of rationale-
augmented ensembles that can automatically aggregate across diverse rationales
to overcome the brittleness of performance to sub-optimal human-written
rationales.

Rationale-augmented ensembles | Input/Prompt | Output
---|---|---
Self-consistency (Wang et al., 2022) | fixed | sampled
Prompt-order ensemble (Lu et al., 2021; Zhao et al., 2021) | shuffled | greedy/sampled
Input-rationale ensemble, adapted from (Wiegreffe et al., 2022) | sampled | greedy/sampled
Table 2: Methods for generating rationale-augmented ensembles in language
models.

We define a rationale-augmented ensemble as introducing an additional latent
variable (the “rationales”) that can be sampled and ultimately marginalized
out (see Figure 1 for examples). Depending on the stage where the sampling
occurs, the approaches to rationale ensembling can be categorized as follows
(summarized in Table 2):

  * •

Self-consistency (Wang et al., 2022), where the input/prompt is fixed, and
multiple rationales are sampled from the language model’s decoder.

  * •

Prompt-order ensemble: Given that task performance has been observed to be
sensitive to prompt ordering (Lu et al., 2021; Zhao et al., 2021), the order
of exemplars in prompts can be permuted to elicit multiple rationales in the
decoder.

  * •

Input-rationale ensemble: Leveraging the ability of large language models to
generate high-quality explanations (Wiegreffe et al., 2022), model-generated
rationales can replace human-written rationales in the input prompts (e.g.,
via the process described in Section 2.1), which can then be used to elicit
multiple rationales in the decoder.

For each of these ensembling approaches, the model couples the generation of
rationales and answers before taking a majority vote (more precisely, a
plurality vote) to produce the final ensemble answer. For both prompt-order
ensembling and input-rationale ensembling, since the randomness is introduced
in the input space, one can either decode an output greedily with a rationale,
or sample an output with a rationale in the output space for each new prompt.
Interestingly, below we find that rationale sampling in the output space is
the most important component in the overall rationale-augmented ensemble
framework. In particular, regardless of how the input/prompt varies, sampling
in the output space is the key to achieving better task performance across a
variety of natural language processing tasks. With this key component, we find
that rationale-ensembling can significantly improve results over both standard
prompting (Brown et al., 2020) and rationale-based prompting (Wei et al.,
2022; Kojima et al., 2022) on common NLP tasks; the framework also provides
rationales at no additional cost that can be used to better interpret model
predictions.

##  3 Experiments

We conducted a series of experiments to compare the performance of rationale-
augmented ensembles against existing approaches, across a variety of natural
language processing tasks. Overall, the results demonstrate that rationale-
augmented ensembles can robustly improve task performance across alternative
language models and model scales.

###  3.1 Experiment setup

#### Tasks and datasets.

We considered a set of natural language tasks from GLUE (Wang et al., 2018),
SuperGLUE (Wang et al., 2019), and other natural language processing
benchmarks. These tasks can be categorized as follows:333We use the test split
for all tasks if the test split is available and has labels for evaluation,
otherwise we use the dev split. Specifically, test split: ANLI, e-SNLI,
OpenBookQA, ARC; dev/validation split: MNLI, RTE, BoolQ, Hotpot-QA, WiC,
SST-2, QQP. In addition, some of the datasets are too large to run large
language models on, so we used the first 1,000 data points for HotpotQA,
e-SNLI, MNLI, and QQP for evaluation.

  * •

Question Answering: For question answering, we include BoolQ (Clark et al.,
2019), HotpotQA (Yang et al., 2018), and OpenBookQA (Mihaylov et al., 2018).

  * •

Natural Language Inference: For these tasks, we include ANLI (Nie et al.,
2020) with the three subsets (R1, R2, R3), e-SNLI (Camburu et al., 2018), MNLI
(matched/mis-matched) (Williams et al., 2018), and RTE (Dagan et al., 2005;
Bar-Haim et al., 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009).

  * •

Word Sense Disambiguation: Here we use Word-in-Context (WiC; Pilehvar &
Camacho-Collados, 2019).

  * •

Sentiment Analysis: we use the Stanford Sentiment Treebank v2 (SST-2; Socher
et al., 2013).

  * •

Paraphrase Identification: Here we use Quora Question Pairs (QQP; Iyer et al.,
2017).

  * •

Reasoning. For reasoning tasks, we consider the AI2 Reasoning Challenge (ARC)
(Clark et al., 2018) for open-domain question answering with commonsense
reasoning, as well as the grade-school math problems (GSM8K; Cobbe et al.,
2021) for arithmetic reasoning.

#### Language models and prompts.

To investigate whether rationale-augmented ensembles can robustly improve
performance across language models, we evaluated the framework with two dense
left-to-right, decoder-only transformer language models with varying scale:
(1) PaLM-540B, a language model with 540-billion parameters (Chowdhery et al.,
2022) and (2) the public GPT-3 model with 175-billion parameters (Brown et
al., 2020; Ouyang et al., 2022).

All experiments are conducted in the few-shot setting except the zero-shot CoT
baseline (Kojima et al., 2022), without any fine-tuning. For each task, we
randomly choose $K$ examples from the training set as $K$-shot prompts, while
maintaining a balanced label distribution and manually providing a set of
rationales as the initial prompts; see Appendix A.1 for the full set of
initial prompts and rationales used in each experiment. We use the exact same
exemplars in the few-shot prompts for all baselines and rationale-augmented
ensembles. For standard few-shot prompting we omit the rationales.

#### Parameter settings.

Across all tasks, each rationale-augmented ensemble is generated by ensembling
$m=40$ outputs from the language model. For sampling in the language model, we
use temperature sampling (Ackley et al., 1985; Ficler & Goldberg, 2017) with
temperature $T=0.7$. The maximum number of decoded steps is set to $128$ in
every case, except for GSM8K where we use $256$ to accommodate longer
rationales needed to express extended reasoning chains.

###  3.2 Results

The results for the PaLM-540B model are shown in Table 3, Table 4 and Table 6,
and give a comparison to two baseline approaches: (1) standard few-shot
prompting without rationales (Brown et al., 2020), and (2) rationale-based
prompting, including few-shot chain-of-thought (CoT) prompting (Wei et al.,
2022), and zero-shot CoT (Kojima et al., 2022) where the model is prompted
with “Let’s think step by step” to generate initial rationales then prompted
with “Therefore, the answer is” to obtain the final answer.444We have found
the zero-shot CoT approach yields slightly less controlled responses compared
to few-shot based approaches, i.e., the model is less likely to generate a
desired fixed answer like “yes/no”, “(a)-(e)” even when we add guided prompts
like “The answer (yes or no) is”, “among options (a) through (e)”.

For each of the rationale-augmented ensembles, we specify the inputs as
“fixed", “shuffled" (for prompt-order ensemble), or “sampled" (for input-
rationale ensemble); and the outputs as “greedy" or “sampled" depending on
whether we decode the outputs greedily or sample the outputs from the language
model’s decoder. Based on the results shown in the tables, a few key
observations follow:

  * •

For each rationale-augmented ensemble strategy, the “output-sampled” version
yields better final performance than the “output-greedy” version for almost
every task. This remains true regardless of whether randomness is introduced
in the input space (i.e., whether the exemplars are shuffled in a prompt-order
ensemble, or whether rationales in the exemplars are sampled in an input-
rationale ensemble). Although self-consistency has an “output-sampled” only
version, given that the input/prompt is fixed, it also achieves comparable
performance to the “output-sampled” versions of the other ensembling
approaches. These findings indicate that rationale sampling in the output
space is the critical component for improving task performance, more so than
the specific ensembling method used.

  * •

The “output-sampled” version of each rationale-ensembling method almost always
improves performance over standard prompting (Brown et al., 2020) without
rationales, as well as rationale-based few-shot and zero-shot prompting (Wei
et al., 2022; Kojima et al., 2022). There are a few exceptions, including
MNLI-m/mm, SST-2, and QQP, from GLUE (Wang et al., 2018), where standard-
prompting still exhibits the best performance. We conjecture that the
questions and answers in these tasks already appear frequently in the pre-
training corpus, which allows simple memorization to perform well, whereas
forcing the model to additionally provide rationales slightly degrades
performance.

  * •

Simply adding rationales as in (Wei et al., 2022; Kojima et al., 2022) can
sometimes degrade task performance compared to standard prompting (also
observed in (Ye & Durrett, 2022)), but rationale-augmented ensembling reliably
boosts performance beyond both rationale-based and standard prompting in most
tasks. This finding suggests that rationale-augmented ensembles provide a
reliable approach to improving the final task performance of rationale-based
few-shot in-context learning. Interpretability of model predictions is also
enhanced by the presence of generated rationales in the model outputs.

| Input | Output | ANLI R1 / R2 / R3 | e-SNLI | RTE | MNLI-m/mm
---|---|---|---|---|---|---
Zero-shot CoT (Kojima et al., 2022) | fixed | greedy | 49.7 / 45.1 / 44.8 | 70.4 | 72.2 | 60.0 / 62.2
Standard-prompting (no-rationale) | fixed | greedy | 69.1 / 55.8 / 55.8 | 85.8 | 84.8 |  82.7 / 81.5
CoT-prompting (Wei et al., 2022) | fixed | greedy | 68.8 / 58.9 / 60.6 | 81.0 | 79.1 | 72.0 / 74.0
Prompt-order ensemble | shuffled | greedy | 72.0 / 60.7 / 61.3 | 84.2 | 78.0 | 74.5 / 75.7
shuffled | sampled |  78.7 / 64.9 / 66.0 | 89.0 | 84.8 | 80.3 / 81.2
Input-rationale ensemble | sampled | greedy | 70.1 / 60.1 / 61.1 | 87.1 | 79.1 | 73.4 / 75.9
sampled | sampled |  78.3 / 64.5 / 64.3 | 88.8 | 85.2 | 78.8 / 81.0
Self-consistency (Wang et al., 2022) | fixed | sampled |  78.5 / 64.5 / 63.4 | 88.4 | 86.3 | 79.5 / 80.5
Table 3: Performance comparison over natural language inference tasks, on
PaLM-540B.

We explain these experiments in more detail. Table 3 shows the results
obtained across a range of natural language inference tasks. One can see that
the three rationale-augmented ensembling strategies (“output-sampled”) all
achieve significantly higher accuracy than chain-of-thought prompting with
human-written rationales (Wei et al., 2022). On e-SNLI, RTE, and MNLI, the
chain-of-thought approach produces worse performance than standard prompting,
while rationale-augmented ensembling is able to boost the performance
significantly, outperforming chain-of-thought prompting in every case, and
outperforming standard prompting in all cases except MNLI.

| Input | Output |  BoolQ (q only) |  BoolQ (w/ passage) |  HotpotQA (q only, EM/F1) |  OpenBookQA (q only)
---|---|---|---|---|---|---
Zero-shot CoT (Kojima et al., 2022) | fixed | greedy | 55.4 | 71.7 | 17.1 / 23.0 | 67.6
Standard-prompting (no-rationale) | fixed | greedy | 71.3 | 89.7 | 27.1 / 36.8 | 84.4
CoT-prompting (Wei et al., 2022) | fixed | greedy | 74.2 | 85.4 | 28.9 / 39.8 | 86.4
Prompt-order ensemble | shuffled | greedy | 73.3 | 87.4 | 30.3 / 41.3 | 87.6
shuffled | sampled | 78.0 | 91.0 |  34.7 / 45.4 | 91.0
Input-rationale ensemble | sampled | greedy | 75.0 | 86.5 | 30.9 / 41.7 | 87.4
sampled | sampled | 78.6 | 90.9 | 32.4 / 43.7 | 90.0
Self-consistency (Wang et al., 2022) | fixed | sampled | 78.4 | 90.6 | 33.8 / 44.6 | 90.0
Table 4: Performance comparison over question answering tasks on PaLM-540B.
For BoolQ we evaluated both closed-book setting (question only, passage not
provided) and the setting where passage is provided. For HotpotQA we evaluated
the closed-book setting (question only).

Similarly, Table 4 shows the results obtained in four question answering
tasks. For BoolQ, we conducted an evaluation in both the closed-book setting
(the model is given a question only, without providing a relevant passage), as
well as the setting where both the question and a relevant passage are
provided. In both settings, rationale-augmented ensembling (“output-sampled”)
outperforms standard and chain-of-thought prompting. For HotpotQA, we
conducted the evaluation in the closed-book setting, where rationale-augmented
ensembling achieves a higher EM and F1 score compared to both baselines. An
important question for open-domain question answering in the closed-book
setting is how to utilize a language model to retrieve relevant knowledge and
compose the final answer (Roberts et al., 2020). Our results show that
rationale-augmented ensembling is an effective technique for encouraging a
language model to retrieve multiple relevant facts for a given question. To
illustrate this phenomenon, Table 5 demonstrates some of the retrieved facts
used to yield final answers.

Task / Question |  Golden passage |  Retrieved facts and composed rationales from the language model (closed-book setting, passage not provided)
---|---|---
[BoolQ] can u drive in canada with us license (answer: yes) |  Persons driving into Canada must have their vehicle’s registration document and proof of insurance. |  Yes, you can drive in Canada with a U.S. licence. You need to have a signed letter from your car rental company. |  49 U.S. states and 6 Canadian provinces have a reciprocal agreement for the recognition of driver licensing. As long as you have a valid U.S. driver’s license, you may drive in Canada. The answer is yes.
[HotpotQA] What government position was held by the woman who portrayed Corliss Archer in the film Kiss and Tell? (answer: Chief of Protocol) |  Kiss and Tell is … starring then 17-year-old Shirley Temple as Corliss Archer. $\mid$ Shirley Temple Black… was named United States ambassador to Ghana and to Czechoslovakia and also served as Chief of Protocol of the United States. |  Shirley Temple Black played Corliss Archer in Kiss and Tell. Black was the United States Ambassador to Ghana and Czechoslovakia. The answer is Ambassador. |  Corliss Archer was a fictional character. Actress Shirley Temple portrayed Corliss Archer in the film Kiss and Tell. In 1967, Shirley Temple became the first female Chief of Protocol in the United States. The answer is Chief of Protocol.
Table 5: Examples of how the language model retrieves multiple relevant facts and composes rationales for open-domain question-answering in the closed-book setting. | Input | Output | WiC | SST-2 | QQP | ARC-e | ARC-c | GSM8K
---|---|---|---|---|---|---|---|---
Zero-shot CoT (Kojima et al., 2022) | fixed | greedy | 54.1 | 76.8 | 55.8 | 87.0 | 79.6 | 43.0
Standard-prompting (no-rationale) | fixed | greedy | 67.6 | 94.6 | 84.1 | 95.9 | 87.1 | 17.9
CoT-prompting (Wei et al., 2022) | fixed | greedy | 65.2 | 87.8 | 75.6 | 95.3 | 85.2 | 56.5
Prompt-order ensemble | shuffled | greedy | 62.1 | 88.1 | 76.6 | 94.5 | 85.6 | 59.6
shuffled | sampled | 62.5 | 91.2 | 80.9 | 96.4 | 88.5 | 75.4
Input-rationale ensemble | sampled | greedy |  66.5 / 72.1 | 92.3 | 76.6 | 95.5 | 86.6 | 58.9
sampled | sampled |  65.2 / 70.8 | 93.1 | 81.2 | 96.7 | 88.6 | 73.8
Self-consistency (Wang et al., 2022) | fixed | sampled | 66.9 | 91.1 | 78.9 | 96.4 | 88.7 | 74.4
Table 6: Performance comparison over other common NLP tasks, on PaLM-540B.

Finally, Table 6 provides results for other common natural language processing
tasks. Interestingly, for tasks that do not require explicit intermediate
steps, such as SST-2 and QQP, adding manual rationales to prompts can degrade
performance significantly. Yet, in these cases, rationale-augmented ensembles
(“output-sampled”) are able to significantly close the gap. For WiC, ARC-
easy/challenge and GSM8K, rationale-augmented ensembling outperforms both
standard and chain-of-thought prompting by a large margin. Here, for WiC, we
evaluated an alternative variant of the input-rationale ensemble: instead of
replacing one rationale in each prompt, we replace every original rationales
by a generated one in each prompt. This variant generally yields similar or
slightly worse performance compared to replacing one rationale at a time, but
on the WiC task we observed a performance improvement (70.8% versus 65.2% when
only one rationale is replaced), which indicates that this task might require
greater rationale diversity to support strong task performance.

###  3.3 Results on GPT-3

To control for the effects of the language model and aid reproducibility, we
repeat the above studies with the publicly available GPT-3 model (Brown et
al., 2020; Ouyang et al., 2022). Once again, we find similar outcomes where
rationale-augmented ensembling robustly improves performance across natural
language tasks. Here we use the code-davinci-002 engine (Chen et al., 2021),
which has been observed to yield slightly better performance than text-
davinci-002. The results of this study are given in Table 7, showing that
rationale-augmented ensembles with GPT-3 obtain similar improvements to those
obtained with PaLM-540B above. Once again, human-written rationales in few-
shot learning can sometimes degrade performance compared to standard prompting
(e.g., on RTE, OpenBookQA, WiC, ARC-challenge), while rationale-augmented
ensembling with sampling in the output space (“output-sampled”) reliably
improves performance over both baselines. Similarly, for WiC, introducing
greater diversity in sampled rationales improves performance (67.6%) compared
to sampling a single rationale for each prompt (57.4%). These results
reinforce the finding that the improvements are robust to the specific
language model, provided it is of sufficient size/quality.

| Input | Output | RTE | BoolQ | OpenBookQA | WiC | ARC-c
---|---|---|---|---|---|---|---
Standard-prompting (no-rationale) | fixed | greedy | 85.2 | 69.9 | 81.4 | 65.5 | 85.9
CoT-prompting (Wei et al., 2022) | fixed | greedy | 84.1 | 73.5 | 80.4 | 55.5 | 83.6
Prompt-order ensemble | shuffled | greedy | 83.0 | 74.2 | 83.4 | 56.4 | 84.0
shuffled | sampled | 88.8 | 78.5 | 87.8 | 56.7 | 88.2
Input-rationale ensemble | sampled | greedy | 85.2 | 75.0 | 85.4 |  57.1 / 68.0 | 84.7
sampled | sampled | 87.4 | 78.4 | 87.0 |  57.4 / 67.6 | 87.6
Self-consistency (Wang et al., 2022) | fixed | sampled | 85.6 | 78.2 | 88.4 | 55.6 | 87.5
Table 7: Performance comparison on GPT-3 (code-davinci-002 engine).

###  3.4 Additional Studies

#### Effect of $K$ in $K$-shot in-context learning.

In Table 8, we provide an ablation study that examines the effect of choosing
different $K$ in $K$-shot in-context learning. While increasing the number of
exemplars $K$ generally improves performance, rationale-augmented ensembling
robustly improves performance over standard and chain-of-thought prompting for
all values of $K$.

| Input | Output | 3-shot | 6-shot/T-1 | 9-shot | T-2 | T-3
---|---|---|---|---|---|---|---
Standard-prompting (no-rationale) | fixed | greedy | 67.9 | 69.1 | 69.3 | 66.1 | 66.4
CoT-prompting (Wei et al., 2022) | fixed | greedy | 71.6 | 68.8 | 72.2 | 67.9 | 68.3
Prompt-order ensemble | shuffled | sampled | 76.0 | 78.7 | 80.1 | 78.4 | 75.6
Input-rationale ensemble | sampled | sampled | 76.1 | 78.3 | 78.4 | 77.8 | 76.0
Self-consistency (Wang et al., 2022) | fixed | sampled | 77.9 | 78.5 | 78.7 | 76.6 | 76.9
Table 8: Performance comparison on ANLI-R1 using PaLM-540B, with (1) varying
$K$ (3, 6, 9) in $K$-shot learning; and (2) using different
templates/verbalizers (T-1, T-2, T-3), fixing $K=6$.

#### Effect of templates and verbalizers.

We also investigate whether rationale-augmented ensembling is robust to
different templates or verbalizers, since previous work has shown that
templates or verbalizers can have a significant effect on final performance
(Bach et al., 2022). Here we choose three alternative templates from
PromptSource555https://github.com/bigscience-workshop/promptsource for the NLI
task, as follows:

  * •

Template-1: Premise:\\\{premise}"\\\Based on this premise, can we conclude the
hypothesis "{hypothesis}" … is true?\\\options

  * •

Template-2: "{premise}"\\\Does it follow that "{hypothesis}"?\\\options

  * •

Template-3: Suppose "premise"\\\Can we infer that "hypothesis"?\\\options

The results in Table 8 reveal that, although different templates can induce
variable performance, rationale-augmented ensembling outperforms standard and
chain-of-thought prompting under all three templates.

#### Effect of using existing explanations vs newly-written ones in the
prompts.

To control for the bias of manually written rationales, we also investigate
performance on the e-SNLI dataset using crowd-sourced rationales (Camburu et
al., 2018). As shown in Table 3, the improvement of rationale-augmented
ensemble appears to be stable regardless of whether the rationales are crowd-
sourced or author-supplied.

Note that in this paper, we focus on the role of “rationales”, and conduct the
studies in a manner that fixes other factors that might affect task
performance. Due to the large performance variance across alternative set-ups,
it is clear that a rigorous evaluation of few-shot in-context learning
requires the specification of all these factors, including (1) the exact
prompts used, including the specific exemplars, templates/verbalizers,
instructions, or rationales/explanations used; and (2) the exact prompt order
and the number of exemplars $K$ used.

##  4 Related work

#### Rationalization and interpretability in NLP.

One relevant line of work tries to improve rationalization and
interpretability in natural language processing models, for example, by
extracting rationales using task-specific approaches (Xu et al., 2021; Asai et
al., 2020; Chen et al., 2019). In the supervised learning setting, one
typically fine-tunes a model using human-annotated rationales as training data
(Zaidan et al., 2007; Ling et al., 2017; Narang et al., 2020; Cobbe et al.,
2021). Zelikman et al. (2022) propose to use prompting to augment a training
dataset with rationales, then fine-tune a language model using this dataset to
further improve reasoning ability. Li et al. (2022) propose to sample
“diverse” prompts from the training set augmented by rationales, plus an
additional voting verifier to improve model performance on reasoning tasks.
However, the use of an additional training set is closer to the fine-tuning
setting rather than the few-shot setting. Compared to these approaches,
rationale-augmented ensembles focus more on the few-shot setting, where there
is no additional training or fine-tuning, hence no human annotation nor
training/development datasets are required.

Recent work has also considered prompting language models with human-written
rationales to further improve performance, such as (Wei et al., 2022; Kojima
et al., 2022; Wang et al., 2022; Jung et al., 2022). Lampinen et al. (2022)
show that hand-tuned explanations can improve task performance substantially.
By contrast, rationale-augmented ensembling requires no hand-tuning on
rationales. Instead, we leverage the language model to automatically sample
rationales to overcome the sub-optimality of manually provided rationales.

#### Prompt optimization and ensembles in language models.

Previous work has shown that the prompt order (Lu et al., 2021), how each task
is verbalized (Bach et al., 2022), and the distribution of labels in the
prompts (Zhao et al., 2021) can all affect final task performance. In this
paper, we find that, when shifting from the paradigm of (input $\rightarrow$
output) pairs to (input, rationale $\rightarrow$ output) pairs, there is also
a large variance in the final task performance when the rationales used in the
prompts differ. Recent work has also proposed ways to further improve a
model’s reasoning ability under specific constraints. For example, when the
final label is binary, Jung et al. (2022) induce a tree of explanations, then
use an SAT solver and an NLI verifier to infer the satisfiability of each
explanation. For commonsense reasoning tasks, Liu et al. (2022a) generate
relevant knowledge as additional inputs to the model, to improve the
performance. Another line of work proposes to better retrieve prompts closer
to the target question to further improve task performance (Liu et al., 2022b;
Rubin et al., 2021).

#### Learn to execute programs with intermediate computations.

Although much of the work on rationales has come from the natural language
processing literature, there has been growing interest in similar mechanisms
in the area of program synthesis. Nye et al. (2021) use pretrained language
models to execute a program by predicting the intermediate states of a program
behaviour line-by-line. This work shows that eliciting step-by-step reasoning
described by a formal language can dramatically improve the execution
prediction accuracy. Other recent work (Pi et al., 2022) pre-trains language
models as program executors and shows that this can improve reasoning task
performance.

##  5 Conclusion

In this paper, we have presented a unified framework for rationale-augmented
ensembles, and found that rationale sampling in the output space is a key
component for achieving improved performance in natural language processing
tasks. By sampling diverse rationales and ensembling the results, we have
shown that rational-ensembling methods in the proposed framework can reliably
outperform standard prompting and rationale-based few-shot prompting, across a
wide range of natural language tasks and alternative language models. Overall,
rationale-augmented ensembling appears to be a reliable way to shift from the
paradigm of (input $\rightarrow$ output) pairs to (input, rationale
$\rightarrow$ output) pairs to achieve more accurate and interpretable natural
language processing.

Although the proposed framework mitigates sensitivity to human-written
rationales, some human-written seed rationales are still required, which could
still bias generation of output rationales. We have observed that patterns
expressed in the written rationales can affect a model’s generated rationales.
For example, if all seed rationales are written in a similar style, like “The
first…the second…”, subsequently generated rationales will tend to follow the
same pattern. Therefore, some diversity in seed rationales still appears to be
important for inducing sufficient diversity in generated rationales.

Overall, through this study, we hope to motivate more research on
understanding how language models respond differently to variations in few-
shot exemplars, which can lead to the development of more robust and
autonomous approaches for generating effective prompts for a given target
task.

## References

  * Ackley et al. (1985) Ackley, D. H., Hinton, G. E., and Sejnowski, T. J.  A learning algorithm for boltzmann machines.  _Cognitive Science_ , 9(1):147–169, 1985.  ISSN 0364-0213.  URL https://www.sciencedirect.com/science/article/pii/S0364021385800124.
  * Asai et al. (2020) Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., and Xiong, C.  Learning to retrieve reasoning paths over wikipedia graph for question answering.  In _International Conference on Learning Representations_ , 2020.  URL https://openreview.net/forum?id=SJgVHkrYDH.
  * Bach et al. (2022) Bach, S. H., Sanh, V., Yong, Z.-X., Webson, A., Raffel, C., Nayak, N. V., Sharma, A., Kim, T., Bari, M. S., Fevry, T., Alyafeai, Z., Dey, M., Santilli, A., Sun, Z., Ben-David, S., Xu, C., Chhablani, G., Wang, H., Fries, J. A., Al-shaibani, M. S., Sharma, S., Thakker, U., Almubarak, K., Tang, X., Tang, X., Jiang, M. T.-J., and Rush, A. M.  Promptsource: An integrated development environment and repository for natural language prompts, 2022.
  * Bar-Haim et al. (2006) Bar-Haim, R., Dagan, I., Dolan, B., Ferro, L., Giampiccolo, D., Magnini, B., and Szpektor, I.  The second pascal recognising textual entailment challenge.  In _Proceedings of the second PASCAL challenges workshop on recognising textual entailment_ , 2006.
  * Bentivogli et al. (2009) Bentivogli, L., Clark, P., Dagan, I., and Giampiccolo, D.  The fifth pascal recognizing textual entailment challenge.  In _TAC_ , 2009.
  * Brown et al. (2020) Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , 2020.  URL https://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.
  * Camburu et al. (2018) Camburu, O.-M., Rocktäschel, T., Lukasiewicz, T., and Blunsom, P.  e-snli: Natural language inference with natural language explanations.  In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), _Advances in Neural Information Processing Systems 31_ , pp. 9539–9549. Curran Associates, Inc., 2018.  URL http://papers.nips.cc/paper/8163-e-snli-natural-language-inference-with-natural-language-explanations.pdf.
  * Chen et al. (2019) Chen, J., Lin, S., and Durrett, G.  Multi-hop question answering via reasoning chains.  _CoRR_ , abs/1910.02610, 2019.  URL http://arxiv.org/abs/1910.02610.
  * Chen et al. (2021) Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al.  Evaluating large language models trained on code.  _arXiv preprint arXiv:2107.03374_ , 2021.
  * Chowdhery et al. (2022) Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., Schuh, P., Shi, K., Tsvyashchenko, S., Maynez, J., Rao, A., Barnes, P., Tay, Y., Shazeer, N., Prabhakaran, V., Reif, E., Du, N., Hutchinson, B., Pope, R., Bradbury, J., Austin, J., Isard, M., Gur-Ari, G., Yin, P., Duke, T., Levskaya, A., Ghemawat, S., Dev, S., Michalewski, H., Garcia, X., Misra, V., Robinson, K., Fedus, L., Zhou, D., Ippolito, D., Luan, D., Lim, H., Zoph, B., Spiridonov, A., Sepassi, R., Dohan, D., Agrawal, S., Omernick, M., Dai, A. M., Pillai, T. S., Pellat, M., Lewkowycz, A., Moreira, E., Child, R., Polozov, O., Lee, K., Zhou, Z., Wang, X., Saeta, B., Diaz, M., Firat, O., Catasta, M., Wei, J., Meier-Hellstern, K., Eck, D., Dean, J., Petrov, S., and Fiedel, N.  Palm: Scaling language modeling with pathways, 2022.  URL https://arxiv.org/abs/2204.02311.
  * Clark et al. (2019) Clark, C., Lee, K., Chang, M.-W., Kwiatkowski, T., Collins, M., and Toutanova, K.  Boolq: Exploring the surprising difficulty of natural yes/no questions.  In _NAACL_ , 2019.
  * Clark et al. (2018) Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O.  Think you have solved question answering? try arc, the ai2 reasoning challenge.  _ArXiv_ , abs/1803.05457, 2018.
  * Cobbe et al. (2021) Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J.  Training verifiers to solve math word problems, 2021.
  * Creswell et al. (2022) Creswell, A., Shanahan, M., and Higgins, I.  Selection-inference: Exploiting large language models for interpretable logical reasoning, 2022.  URL https://arxiv.org/abs/2205.09712.
  * Dagan et al. (2005) Dagan, I., Glickman, O., and Magnini, B.  The pascal recognising textual entailment challenge.  In _Machine Learning Challenges Workshop_ , pp. 177–190. Springer, 2005.
  * Ficler & Goldberg (2017) Ficler, J. and Goldberg, Y.  Controlling linguistic style aspects in neural language generation.  In _Proceedings of the Workshop on Stylistic Variation_ , pp. 94–104, Copenhagen, Denmark, September 2017. Association for Computational Linguistics.  doi: 10.18653/v1/W17-4912.  URL https://aclanthology.org/W17-4912.
  * Gao et al. (2021) Gao, T., Fisch, A., and Chen, D.  Making pre-trained language models better few-shot learners.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pp. 3816–3830, Online, August 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.acl-long.295.  URL https://aclanthology.org/2021.acl-long.295.
  * Giampiccolo et al. (2007) Giampiccolo, D., Magnini, B., Dagan, I., and Dolan, B.  The third pascal recognizing textual entailment challenge.  In _Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing_ , pp. 1–9. Association for Computational Linguistics, 2007\.
  * Iyer et al. (2017) Iyer, S., Dandekar, N., and Csernai, K.  First quora dataset release: Question pairs, 2017.  URL https://data.quora.com/First-Quora-Dataset-Release-Question-Pairs.
  * Jung et al. (2022) Jung, J., Qin, L., Welleck, S., Brahman, F., Bhagavatula, C., Bras, R. L., and Choi, Y.  Maieutic prompting: Logically consistent reasoning with recursive explanations, 2022.  URL https://arxiv.org/abs/2205.11822.
  * Kojima et al. (2022) Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y.  Large language models are zero-shot reasoners, 2022.  URL https://arxiv.org/abs/2205.11916.
  * Lampinen et al. (2022) Lampinen, A. K., Dasgupta, I., Chan, S. C. Y., Matthewson, K., Tessler, M. H., Creswell, A., McClelland, J. L., Wang, J. X., and Hill, F.  Can language models learn from explanations in context?, 2022.  URL https://arxiv.org/abs/2204.02329.
  * Lester et al. (2021) Lester, B., Al-Rfou, R., and Constant, N.  The power of scale for parameter-efficient prompt tuning.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pp. 3045–3059, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.  doi: 10.18653/v1/2021.emnlp-main.243.  URL https://aclanthology.org/2021.emnlp-main.243.
  * Li et al. (2022) Li, Y., Lin, Z., Zhang, S., Fu, Q., Chen, B., Lou, J.-G., and Chen, W.  On the advance of making language models better reasoners, 2022.  URL https://arxiv.org/abs/2206.02336.
  * Ling et al. (2017) Ling, W., Yogatama, D., Dyer, C., and Blunsom, P.  Program induction by rationale generation: Learning to solve and explain algebraic word problems.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , 2017.  doi: 10.18653/v1/P17-1015.  URL https://aclanthology.org/P17-1015.
  * Liu et al. (2022a) Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Le Bras, R., Choi, Y., and Hajishirzi, H.  Generated knowledge prompting for commonsense reasoning.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pp. 3154–3169, Dublin, Ireland, May 2022a. Association for Computational Linguistics.  doi: 10.18653/v1/2022.acl-long.225.  URL https://aclanthology.org/2022.acl-long.225.
  * Liu et al. (2022b) Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., and Chen, W.  What makes good in-context examples for GPT-3?  In _Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures_ , pp. 100–114, Dublin, Ireland and Online, May 2022b. Association for Computational Linguistics.  doi: 10.18653/v1/2022.deelio-1.10.  URL https://aclanthology.org/2022.deelio-1.10.
  * Lu et al. (2021) Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P.  Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity.  _ArXiv_ , abs/2104.08786, 2021.
  * Mihaylov et al. (2018) Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A.  Can a suit of armor conduct electricity? a new dataset for open book question answering.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 2381–2391, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.  doi: 10.18653/v1/D18-1260.  URL https://www.aclweb.org/anthology/D18-1260.
  * Narang et al. (2020) Narang, S., Raffel, C., Lee, K., Roberts, A., Fiedel, N., and Malkan, K.  Wt5?! training text-to-text models to explain their predictions, 2020\.  URL https://arxiv.org/abs/2004.14546.
  * Nie et al. (2020) Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., and Kiela, D.  Adversarial NLI: A new benchmark for natural language understanding.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, 2020\.
  * Nye et al. (2021) Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., Sutton, C., and Odena, A.  Show your work: Scratchpads for intermediate computation with language models, 2021.  URL https://arxiv.org/abs/2112.00114.
  * Ouyang et al. (2022) Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R.  Training language models to follow instructions with human feedback, 2022\.  URL https://arxiv.org/abs/2203.02155.
  * Pi et al. (2022) Pi, X., Liu, Q., Chen, B., Ziyadi, M., Lin, Z., Gao, Y., Fu, Q., Lou, J.-G., and Chen, W.  Reasoning like program executors, 2022.
  * Pilehvar & Camacho-Collados (2019) Pilehvar, M. T. and Camacho-Collados, J.  WiC: the word-in-context dataset for evaluating context-sensitive meaning representations.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pp. 1267–1273, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.  doi: 10.18653/v1/N19-1128.  URL https://aclanthology.org/N19-1128.
  * Rajani et al. (2019) Rajani, N. F., McCann, B., Xiong, C., and Socher, R.  Explain yourself! Leveraging language models for commonsense reasoning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , 2019.  doi: 10.18653/v1/P19-1487.  URL https://aclanthology.org/P19-1487.
  * Roberts et al. (2020) Roberts, A., Raffel, C., and Shazeer, N.  How much knowledge can you pack into the parameters of a language model?  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pp. 5418–5426, Online, November 2020\. Association for Computational Linguistics.  doi: 10.18653/v1/2020.emnlp-main.437.  URL https://www.aclweb.org/anthology/2020.emnlp-main.437.
  * Rubin et al. (2021) Rubin, O., Herzig, J., and Berant, J.  Learning to retrieve prompts for in-context learning, 2021.  URL https://arxiv.org/abs/2112.08633.
  * Socher et al. (2013) Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pp. 1631–1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics.  URL https://aclanthology.org/D13-1170.
  * Thoppilan et al. (2022) Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al.  Lamda: Language models for dialog applications.  _arXiv preprint arXiv:2201.08239_ , 2022.  URL https://arxiv.org/abs/2201.08239.
  * Wang et al. (2018) Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S.  GLUE: A multi-task benchmark and analysis platform for natural language understanding.  In _Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP_ , pp. 353–355, Brussels, Belgium, November 2018. Association for Computational Linguistics.  doi: 10.18653/v1/W18-5446.  URL https://aclanthology.org/W18-5446.
  * Wang et al. (2019) Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. R.  _SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems_.  Curran Associates Inc., Red Hook, NY, USA, 2019.
  * Wang et al. (2022) Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D.  Self-consistency improves chain of thought reasoning in language models, 2022.  URL https://arxiv.org/abs/2203.11171.
  * Wei et al. (2022) Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D.  Chain of thought prompting elicits reasoning in large language models.  _arXiv preprint arXiv:2201.11903_ , 2022.  URL https://arxiv.org/pdf/2201.11903.
  * Wiegreffe et al. (2022) Wiegreffe, S., Hessel, J., Swayamdipta, S., Riedl, M., and Choi, Y.  Reframing human-ai collaboration for generating free-text explanations.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , Online and Seattle, USA, July 2022. Association for Computational Linguistics.  URL https://arxiv.org/abs/2112.08674/.
  * Williams et al. (2018) Williams, A., Nangia, N., and Bowman, S.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pp. 1112–1122. Association for Computational Linguistics, 2018.  URL http://aclweb.org/anthology/N18-1101.
  * Xu et al. (2021) Xu, W., Deng, Y., Zhang, H., Cai, D., and Lam, W.  Exploiting reasoning chains for multi-hop science question answering.  In _Findings of the Association for Computational Linguistics: EMNLP 2021_ , pp. 1143–1156, Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics.  URL https://aclanthology.org/2021.findings-emnlp.99.
  * Yang et al. (2018) Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., and Manning, C. D.  HotpotQA: A dataset for diverse, explainable multi-hop question answering.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pp. 2369–2380, Brussels, Belgium, October-November 2018. Association for Computational Linguistics.  doi: 10.18653/v1/D18-1259.  URL https://aclanthology.org/D18-1259.
  * Ye & Durrett (2022) Ye, X. and Durrett, G.  The unreliability of explanations in few-shot in-context learning, 2022\.
  * Zaidan et al. (2007) Zaidan, O., Eisner, J., and Piatko, C.  Using “annotator rationales” to improve machine learning for text categorization.  _NAACL_ , 2007.  URL https://aclanthology.org/N07-1033.
  * Zelikman et al. (2022) Zelikman, E., Wu, Y., Mu, J., and Goodman, N. D.  Star: Bootstrapping reasoning with reasoning, 2022.  URL https://arxiv.org/abs/2203.14465.
  * Zhao et al. (2021) Zhao, Z., Wallace, E., Feng, S., Klein, D., and Singh, S.  Calibrate before use: Improving few-shot performance of language models.  In Meila, M. and Zhang, T. (eds.), _Proceedings of the 38th International Conference on Machine Learning_ , volume 139 of _Proceedings of Machine Learning Research_. PMLR, 2021.  URL https://proceedings.mlr.press/v139/zhao21c.html.

##  Appendix A Appendix

###  A.1 Full Sets of Prompts

Table 9:  Few-shot exemplars for RTE.  Premise:
---
"No Weapons of Mass Destruction Found in Iraq Yet."
Based on this premise, can we conclude the hypothesis "Weapons of Mass
Destruction Found in Iraq." is true?
A: "No Weapons of Mass Destruction Found" contradicts "Weapons of Mass
Destruction Found". The answer is no.
Premise:
"A place of sorrow, after Pope John Paul II died, became a place of
celebration, as Roman Catholic faithful gathered in downtown Chicago to mark
the installation of new Pope Benedict XVI."
Based on this premise, can we conclude the hypothesis "Pope Benedict XVI is
the new leader of the Roman Catholic Church." is true?’
A: "installation of new Pope Benedict XVI." means "Pope Benedict XVI is the
new leader". The answer is yes.
Premise:
"A man is due in court later charged with the murder 26 years ago of a
teenager whose case was the first to be featured on BBC One’s Crimewatch.
Colette Aram, 16, was walking to her boyfriend’s house in Keyworth,
Nottinghamshire, on 30 October 1983 when she disappeared. Her body was later
found in a field close to her home. Paul Stewart Hutchinson, 50, has been
charged with murder and is due before Nottingham magistrates later."
Based on this premise, can we conclude the hypothesis "Paul Stewart Hutchinson
is accused of having stabbed a girl." is true?
A: The premise does not say Paul Stewart Hutchinson "stabbed" this girl. The
answer is no.
Premise:
"Herceptin was already approved to treat the sickest breast cancer patients,
and the company said, Monday, it will discuss with federal regulators the
possibility of prescribing the drug for more breast cancer patients."
Based on this premise, can we conclude the hypothesis "Herceptin can be used
to treat breast cancer." is true?
A: "Herceptin was approved to treat breast cancer" implies that "Herceptin can
be used to treat breast cancer". The answer is yes.
Table 10:  Few-shot exemplars for ARC easy/challenge.  Q: George wants to warm
his hands quickly by rubbing them. Which skin surface will produce the most
heat? (a) dry palms. (b) wet palms. (c) palms covered with oil. (d) palms
covered with lotion.
---
A: Dry surfaces will more likely cause more friction via rubbing than other
smoother surfaces, hence dry palms will produce the most heat. The answer is
(a).
Q: Which factor will most likely cause a person to develop a fever? (a) a leg
muscle relaxing after exercise. (b) a bacterial population in the bloodstream.
(c) several viral particles on the skin. (d) carbohydrates being digested in
the stomach.
A: Option (b), bacterial population is the most likely cause for a person
developing fever. The answer is (b).
Q: Which change in the state of water particles causes the particles to become
arranged in a fixed position? (a) boiling. (b) melting. (c) freezing. (d)
evaporating.
A: When water is freezed, the particles are arranged in a fixed position; the
particles are still moving for all other options. The answer is (c).
Q: When a switch is used in an electrical circuit, the switch can (a) cause
the charge to build. (b) increase and decrease the voltage. (c) cause the
current to change direction. (d) stop and start the flow of current.
A: The function of a switch is to start and stop the flow of a current. The
answer is (d).
Table 11:  Few-shot exemplars for NLI tasks, including ANLI and MNLI.
Premise:
---
"Conceptually cream skimming has two basic dimensions - product and
geography."
Based on this premise, can we conclude the hypothesis "Product and geography
are what make cream skimming work." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: Based on "cream skimming has two basic dimensions" we can’t infer that
these two dimensions are what make cream skimming work. The answer is it is
not possible to tell.
Premise:
"One of our member will carry out your instructions minutely."
Based on this premise, can we conclude the hypothesis "A member of my team
will execute your orders with immense precision." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: "one of" means the same as "a member of", "carry out" means the same as
"execute", and "minutely" means the same as "immense precision". The answer is
yes.
Premise:
"Fun for adults and children."
Based on this premise, can we conclude the hypothesis "Fun for only children."
is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: "adults and children" contradicts "only children". The answer is no.
Premise:
"He turned and smiled at Vrenna."
Based on this premise, can we conclude the hypothesis "He smiled at Vrenna who
was walking slowly behind him with her mother." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: the premise does not say anything about "Vrenna was walking". The answer is
it is not possible to tell.
Premise:
"well you see that on television also"
Based on this premise, can we conclude the hypothesis "You can see that on
television, as well." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: "also" and "as well" mean the same thing. The answer is yes.
Premise:
"Vrenna and I both fought him and he nearly took us."
Based on this premise, can we conclude the hypothesis "Neither Vrenna nor
myself have ever fought him." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: "Vrenna and I both" contradicts "neither Vrenna nor myself". The answer is
no.
Table 12:  Few-shot exemplars for e-SNLI (the rationales are crowd-sourced
annotations from the original dataset).  Premise:
---
"A person on a horse jumps over a broken down airplane."
Based on this premise, can we conclude the hypothesis "A person is training
his horse for a competition." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: The person is not necessarily training his horse. The answer is it is not
possible to tell.
Premise:
"A person on a horse jumps over a broken down airplane."
Based on this premise, can we conclude the hypothesis "A person is at a diner,
ordering an omelette." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: One jumping horse cannot be in a diner ordering food. The answer is no.
Premise:
"A person on a horse jumps over a broken down airplane."
Based on this premise, can we conclude the hypothesis "A person is outdoors,
on a horse." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: A broken down airplane is outdoors. The answer is yes.
Premise:
"Children smiling and waving at camera."
Based on this premise, can we conclude the hypothesis "They are smiling at
their parents." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: Just because they are smiling and waving at a camera does not imply their
parents or anyone is anyone behind it. The answer is it is not possible to
tell.
Premise:
"Children smiling and waving at camera."
Based on this premise, can we conclude the hypothesis "The kids are frowning."
is true? OPTIONS:
\- yes
\- no
\- it is not possible to tell
A: One cannot be smiling and frowning at the same time. The answer is no.
Premise:
"Children smiling and waving at camera."
Based on this premise, can we conclude the hypothesis "There are children
present." is true?
OPTIONS:
\- yes
\- no
\- it is not possible to tell
A:The children must be present to see them smiling and waving. The answer is
yes.
Table 13:  Few-shot exemplars for BoolQ (closed-book setting).  Q: does system
of a down have 2 singers?
---
A: System of a Down currently consists of Serj Tankian, Daron Malakian, Shavo
Odadjian and John Dolmayan. Serj and Daron do vocals, so the band does have
two singers. The answer is yes.
Q: do iran and afghanistan speak the same language?
A: Iran and Afghanistan both speak the Indo-European language Persian. The
answer is yes.
Q: is a cello and a bass the same thing?
A: The cello is played sitting down with the instrument between the knees,
whereas the double bass is played standing or sitting on a stool. The answer
is no.
Q: can you use oyster card at epsom station?
A: Epsom railway station serves the town of Epsom in Surrey and is not in the
London Oyster card zone. The answer is no.
Table 14:  Few-shot exemplars for BoolQ (with passage provided).  Q: System of
a Down, sometimes shortened to System and abbreviated as SOAD, is an Armenian-
American heavy metal band from Glendale, California, formed in 1994. The band
currently consists of Serj Tankian (lead vocals, keyboards), Daron Malakian
(vocals, guitar), Shavo Odadjian (bass, backing vocals) and John Dolmayan
(drums).
---
Based on the above text, does system of a down have 2 singers?
A: System of a Down currently consists of Serj Tankian, Daron Malakian, Shavo
Odadjian and John Dolmayan. Serj and Daron do vocals, so the band does have
two singers. The answer is yes.
Q: Persian, also known by its endonym Farsi, is one of the Western Iranian
languages within the Indo-Iranian branch of the Indo-European language family.
It is primarily spoken in Iran, Afghanistan, and Tajikistan, and some other
regions which historically were Persianate societies and considered part of
Greater Iran.
Based on the above text, do iran and afghanistan speak the same language?’
A: Iran and Afghanistan both speak the Indo-European language Persian. The
answer is yes.
Q: Both the violin and viola are played under the jaw. The viola, being the
larger of the two instruments, has a playing range that reaches a perfect
fifth below the violin’s. The cello is played sitting down with the instrument
between the knees, and its playing range reaches an octave below the viola’s.
The double bass is played standing or sitting on a stool, with a range that
typically reaches a minor sixth, an octave or a ninth below the cello’s.
Based on the above text, is a cello and a bass the same thing?
A: The cello is played sitting down with the instrument between the knees,
whereas the double bass is played standing or sitting on a stool. The answer
is no.
Q: Epsom railway station serves the town of Epsom in Surrey. It is located off
Waterloo Road and is less than two minutes’ walk from the High Street. It is
not in the London Oyster card zone unlike Epsom Downs or Tattenham Corner
stations. The station building was replaced in 2012/2013 with a new building
with apartments above the station.
Based on the above text, can you use oyster card at epsom station?
A: Epsom railway station serves the town of Epsom in Surrey and is not in the
London Oyster card zone. The answer is no.
Table 15:  Few-shot exemplars for HotpotQA (closed-book setting).  Q: Which
magazine was started first Arthur’s Magazine or First for Women?
---
A: Arthur’s Magazine started in 1844. First for Women started in 1989. So
Arthur’s Magazine was started first. The answer is Arthur’s Magazine.
Q: The Oberoi family is part of a hotel company that has a head office in what
city?
A: The Oberoi family is part of the hotel company called The Oberoi Group. The
Oberoi Group has its head office in Delhi. The answer is Delhi.
Q: What nationality was James Henry Miller’s wife?
A: James Henry Miller’s wife is June Miller. June Miller is an American. The
answer is American.
Q: The Dutch-Belgian television series that "House of Anubis" was based on
first aired in what year?
A: "House of Anubis" is based on the Dutch–Belgian television series Het Huis
Anubis. Het Huis Anubis is first aired in September 2006. The answer is 2006.
Table 16:  Few-shot exemplars for OpenBookQA.  Q: Poison causes harm to which
of the following? (a) a Tree (b) a robot (c) a house (d) a car
---
A: Poison will harm living things, only a tree is a living thing. The answer
is (a).
Q: As you look deeper into a Marbel you can see (a) the future (b) minut
defects (c) colors (d) the other side
A: Marbel is not transparent, so you can not see the other side. Marbel does
not necessarily have multiple colors. You will see minut defects. The answer
is (b).
Q: When food is reduced in the stomach (a) the mind needs time to digest (b)
take a second to digest what I said (c) nutrients are being deconstructed (d)
reader’s digest is a body of works
A: The food is being deconstructed in the stomach during digestion. The answer
is (c).
Q: The sun is responsible for (a) puppies learning new tricks (b) children
growing up and getting old (c) flowers wilting in a vase (d) plants sprouting,
blooming and wilting
A: The sun can affect the growing of living things, like plants. The answer is
(d).
Table 17:  Few-shot exemplars for WiC.  Do you want to come over to my place
later?
---
A political system with no place for the less prominent groups.
Q: Is the word "place" used in the same way in the two sentences above?
A: The first "place" means "home", the second "place" means "room". The answer
is no.
Approach a task.
To approach the city.
Q: Is the word "approach" used in the same way in the two sentences above?
A: The first "approach" means "deal with", the second "approach" means "come
near". The answer is no.
The general ordered the colonel to hold his position at all costs.
Hold the taxi.
Q: Is the word "hold" used in the same way in the two sentences above?
A: Both "hold" mean "keep" or "detain". The answer is yes.
We like to summer in the Mediterranean.
We summered in Kashmir.
Q: Is the word "summer" used in the same way in the two sentences above?
A: Both "summer" mean "spend the summer". The answer is yes.
Table 18:  Few-shot exemplars for SST-2.  What is the sentiment of the
following sentence?
---
"that loves its characters and communicates something rather beautiful about
human nature"
A: "loves its characters" indicates positive sentiment. The answer is
positive.
What is the sentiment of the following sentence?
"hide new secretions from the parental units"
A: If people are hiding something, it means the sentiment is on the negative
side. The answer is negative.
What is the sentiment of the following sentence?
"the greatest musicians"
A: By saying someone being the "greatest", it means positive sentiment. The
answer is positive.
What is the sentiment of the following sentence?
"contains no wit , only labored gags"
A: "contains no wit" is clearly a negative sentiment. The answer is negative.
What is the sentiment of the following sentence?
"demonstrates that the director of such hollywood blockbusters as patriot
games can still turn out a small , personal film with an emotional wallop ."
A: "can still turn out a small , personal film with an emotional wallop ."
indicates sentiment on the positive side. The answer is positive.
What is the sentiment of the following sentence?
"that ’s far too tragic to merit such superficial treatment"
A: "far too tragic" and "to merit such superficial treatment" both mean
negative sentiments. The answer is negative.
Table 19:  Few-shot exemplars for QQP.  Is the following question "What causes
stool color to change to yellow?" the same as "What can cause stool to come
out as little balls?"
---
A: "change to yellow" and "come out as little balls" mean different things.
The answer is no.
Is the following question "What can one do after MBBS?" the same as "What do i
do after my MBBS?"
A: Both are asking what can a person do after MBBS. The answer is yes.
Is the following question "How is air traffic controlled?" the same as "How do
you become an air traffic controller?"
A: "How is air traffic controlled" means differently as "how to become a
controller". The answer is no.
Is the following question "How do I control my horny emotions?" the same as
"How do you control your horniness?"
A: "horny emotions" means the same as "horniness". The answer is yes.