#  FollowIR: Evaluating and Teaching Information Retrieval Models to Follow
Instructions

Orion Weller${}^{\hskip 0.70004pt\hskip
0.70004pt{\color[rgb]{0,0,1}\boldsymbol{\iota}}}$ Benjamin Chang${}^{\hskip
0.70004pt\color[rgb]{0,0,1}\boldsymbol{\iota}}$ Sean MacAvaney${}^{\hskip
0.70004pt\color[rgb]{0,0,1}\boldsymbol{\lambda}}$ Kyle Lo${}^{\hskip
0.70004pt\color[rgb]{0,0,1}\boldsymbol{\alpha}}$
Arman Cohan${}^{\hskip 0.70004pt\color[rgb]{0,0,1}\boldsymbol{\gamma\hskip
0.70004pt\alpha}}$ Benjamin Van Durme${}^{\hskip
0.70004pt\color[rgb]{0,0,1}\boldsymbol{\iota}}$ Dawn Lawrie${}^{\hskip
0.70004pt\color[rgb]{0,0,1}\boldsymbol{\iota}}$ Luca Soldaini${}^{\hskip
0.70004pt\color[rgb]{0,0,1}\boldsymbol{\alpha}}$
${}^{\color[rgb]{0,0,1}\iota\hskip 0.70004pt}$Johns Hopkins University
${}^{\color[rgb]{0,0,1}\alpha\hskip 0.70004pt}$Allen Institute for AI
${}^{\color[rgb]{0,0,1}\lambda\hskip 0.70004pt}$University of Glasgow
${}^{\color[rgb]{0,0,1}\gamma\hskip 0.70004pt}$Yale University
<EMAIL_ADDRESS>
###### Abstract

Modern Large Language Models (LLMs) are capable of following long and complex
instructions that enable a diverse amount of user tasks. However, despite
Information Retrieval (IR) models using LLMs as the backbone of their
architectures, nearly all of them still only take queries as input, with no
instructions. For the handful of recent models that do take instructions, it’s
unclear how they use them. We introduce our dataset FollowIR, which contains a
rigorous instruction evaluation benchmark as well as a training set for
helping IR models learn to better follow real-world instructions. FollowIR
builds off the long history of the TREC conferences: as TREC provides human
annotators with instructions (also known as narratives) to determine document
relevance, so should IR models be able to understand and decide relevance
based on these detailed instructions. Our evaluation benchmark starts with
three deeply judged TREC collections and alters the annotator instructions,
re-annotating relevant documents. Through this process, we can measure how
well IR models follow instructions, through a new pairwise evaluation
framework. Our results indicate that existing retrieval models fail to
correctly use instructions, using them for basic keywords and struggling to
understand long-form information. However, we show that it is possible for IR
models to learn to follow complex instructions: our new FollowIR-7B model has
significant improvements (over 13%) after fine-tuning on our training set.

##  1 Introduction

Large language models (LLMs) have shown impressive results in understanding
long complex instructions and have enabled a broad array of tasks (Brown et
al., 2020; Achiam et al., 2023). The Information Retrieval (IR) community has
adopted the use of LLMs as the main backbone of their neural retrieval
architectures and has seen an explosion of progress in semantic search
(Karpukhin et al., 2020; Khattab & Zaharia, 2020; Reimers & Gurevych, 2019).
However, these existing systems rely on under-specified keywords and short
sentences, despite the growing capabilities of LLMs. And even though the
retrieval community is increasingly adopting larger IR models for their
backbones (such as LLaMA 7B in Pradeep et al. (2023); Ma et al. (2023); Weller
et al. (2023)), they are only used to estimate document relevance without any
instructions. Moving past this ad-hoc search to search with instructions would
enable experts to narrow in on complex information needs using only flexible
natural language, adapting to new tasks on the fly. For example, imagine a
researcher seeking to identify papers that must contain numerous qualities to
be relevant (from a given venue, using a particular method, etc.) while also
making sure to avoid conditions that would make the document not-relevant (on
the wrong domain, from the wrong time period, etc.).

Recent work has started to move towards search with instructions, but is still
understudied, with only a handful of papers on the topic (Muennighoff et al.,
2024; Su et al., 2022; Asai et al., 2022). Although these works have shown
great initiative in adapting retrieval models to take instructions, these
instructions are typically short (less than 10 words), repetitive (only one
instruction per dataset e.g. Su et al. (2022); Asai et al. (2022); Wang et al.
(2023); Xiao et al. (2023)), and lack evaluation datasets that explicitly
measure instruction following – instead using standard retrieval benchmarks.

To address this we introduce FollowIR, which contains a benchmark that
explicitly measures the instruction following ability of retrieval models,
while also providing training data that includes diverse and realistic
instructions. Our key intuition is that many ad-hoc retrieval collections
developed as part of TREC (Text REtrieval Conference) conferences provide
instructions to human annotators who judge document relevance for a given
query, as TREC recognizes that information needs often cannot be specified in
only a simple query. These instructions are thorough and complex, including
minute details about what makes a document relevant vs not-relevant. Thus if
humans can use these TREC instructions to annotate document relevance, so
should instruction-following retrieval models (example query and instruction
pairs are shown in Figure 1 and Table 1).

We use three deeply-judged111i.e. that a large number of documents have been
judged relevant or non-relevant, see Section 2.1 for more. TREC collections as
the basis of our evaluation set: TREC Robust 2004 (Voorhees, 2005), TREC
Common Core 2017 (Allan et al., 2017), and TREC News 2021 (Soboroff et al.,
2020). These collections have been thoroughly annotated in order to evaluate
recall in retrieval, frequently with hundreds (or even thousands) of documents
judged as relevant or not-relevant. We take the instructions given to human
annotators (called narratives) and alter them slightly, re-annotating the
relevant documents. We then have paired instructions, which can be used to
test how models respond to changing instructions – seeing if they change the
set of relevant documents to correspond to the relevance definition in the new
altered instructions.

Feature |  Description |  Example | %
---|---|---|---
N/A: Query-only |  Used for keyword and semantic similarity matching |  What impact has the Chunnel had on the British economy and/or the life style of the British? | –
Specifics |  Detailed specifics on what qualifies as relevant, potentially including or or and conditions |  Documents discussing the following issues are relevant: - projected and actual impact on the life styles of the British - Long term changes to economic policy and relations - major changes to other transportation systems linked with the Continent … | 90.4
Irrelevant Info |  Not keyword relevant, but helpful for background |  … The Channel Tunnel, sometimes referred to informally as the Chunnel, is a 50.46-kilometre undersea railway tunnel, opened in 1994 … | 34.6
Negation |  Specifics on what type of documents are not relevant and should be excluded |  … Documents discussing the following issues are not relevant: - expense and construction schedule - routine marketing ploys by other channel crossers (i.e., schedule changes, price drops, etc.) … | 52.9
Table 1: How standard retrieval queries differ from instructions (or
narratives): instructions contain more specific details about what is
relevant, include less directly-relevant background information, and often
have directives about what documents are not relevant, using negation. %
indicates the percent of the time these features are present in the original
TREC instructions used in FollowIR.

As there are no existing methods to compare pairwise queries in IR, we develop
a new evaluation framework to do so, measuring rank-wise score changes (which
we call $p$-MRR) of documents given a pair of different instructions with the
same query. Results on FollowIR indicate that current models generally fail to
follow instructions in retrieval unless they are 3B+ parameters or have not
been trained for retrieval. Our analysis shows that these failures are due to
two phenomena: the length of the instruction (as models are not used to long
instructions) as well as using the instruction for keyword search rather than
as an instruction that defines relevance.

To further progress in building retrieval models that can understand
instructions, we build a training set of real-world human-used instructions
and fine-tune a model on them (FollowIR-7B). Our results show marked
improvement on FollowIR for both standard IR metrics and for $p$-MRR,
indicating a starting point for future progress on instruction following.

Thus, we provide the following contributions: (1) a benchmark for evaluating
instruction following in retrieval (FollowIR) consisting of human annotations
on top of three already highly-judged corpora, (2) analysis of why current
models fail to understand instructions, and (3) training data for teaching
retrieval models to follow instructions along with a new open-sourced IR
model, FollowIR-7B, that can handle long instructions in IR.222Links to the
code, data, and models are available at https://github.com/orionw/FollowIR

Figure 1: A visual depiction of the pairwise evaluation framework: models are
evaluated on both the original query and instruction, as well as the altered
instruction. If the model correctly understands the instructions, it will
change which documents are relevant w.r.t. the alteration (right). Note that
the real-world instructions (left) given to TREC annotators includes fine-
grained details about what relevance is, as well as instructions containing
negation (in bold).

##  2 Related Work

###  2.1 TREC Conferences

The United States National Institute of Science and Technology (NIST) created
the TREC organization in 1993. Each year TREC sponsors many tracks, or shared
tasks, on a given dataset. These tracks range from a variety of topics:
anywhere from standard ad-hoc retrieval on news (Soboroff et al., 2018;
Soboroff, 2021) to more complex domains such as legal retrieval (Baron et al.,
2006; Oard et al., 2008) or retrieval-augmented generation/report-generation
(Lawrie et al., 2024).

As part of this process, NIST sponsors annotations for these collections.
Typically, this is done by pooling a set of results (runs) from a variety of
retrieval models and then annotating them in rank order until funding runs
out. To help facilitate annotation, track organizers provide a narrative (or
instruction) for each query that will be given to the annotators – however, IR
models are only ever given the query. As evaluating total recall would require
annotating every document in the collection for every query (which is not
feasible for collections with millions of documents), recall error is tested
using post-hoc sampling and annotation. Although not every query and document
pair can be evaluated, recall for queries is very high. We build off the
rigorous evaluation done at TREC by using several of their collections to
build FollowIR.

###  2.2 Instructions for LLMs

Instruction-following LLMs have been popularized by models such as GPT-3
(Brown et al., 2020) and have become a large area of interest for the natural
language processing community (Touvron et al., 2023a; Jiang et al., 2023;
Groeneveld et al., 2024; Black et al., 2022). There has been much work in
evaluating if they can generalize to new instructions (Weller et al., 2020;
Wang et al., 2022c; Ouyang et al., 2022), if we can train them to follow
instructions without human-annotated data (Wang et al., 2022b; Qin et al.,
2023), and applying them to various domains (Zhao et al., 2021; Singhal et
al., 2023; Shaghaghian et al., 2020). As the IR community uses LLMs in their
models, we seek to broaden the scope of IR to include instructions, aligning
it with the broader NLP community.

###  2.3 Instructions for Retrieval

Using instructions in retrieval is a nascent area of exploration. Su et al.
(2022) and Asai et al. (2022) were two of the earliest works that trained a
retrieval model to use instructions along with the query. However, these
instructions are typically very short, such as “Retrieve a Wikipedia paragraph
that answers this question." Recent work incorporates instructions in smaller
models (Xiao et al., 2023; Chen et al., 2023, 2024) as well as others which
use Mistral (Jiang et al., 2023) as the backbone of a larger retrieval model
that can use instructions: GritLM (Muennighoff et al., 2024) trains Mistral to
do both generation and embedding, while Wang et al. (2023) uses Mistral for
embeddings only.

Despite this flurry of activity, these efforts do not have an explicit
instruction-related retrieval benchmark to evaluate on. Instead, they evaluate
on standard retrieval benchmark suites such as MTEB (Muennighoff et al., 2022)
and BEIR (Thakur et al., 2021) which do not contain instructions. Thus, these
newer instruction-IR models hand-write a few instructions, where typically
each instruction is applied to an entire dataset, irrespective of the query.
This makes these instructions generic: focused only on the task format, format
of the “document" (paragraph, sentence, etc.), and the broad domain. Note that
because of this, no current instructions contain any extra background
information or negation (Weller et al., 2024) which are commonly found in
real-world instructions (Table 1).

In work concurrent to ours, Oh et al. (2024) also propose a dataset to
evaluate instructions in retrieval models. Their dataset uses the MSMarco
collection, and differs in several crucial aspects: it only has one relevant
document per query (e.g. sparsely judged), is GPT-4 generated and validated,
focuses on the background of the user (“I am a school teaching looking for
…"), and evaluates using the lowest score over $N$ instructions for the same
query (measuring robustness). In contrast, we use highly-judged corpora to
ensure we can measure recall, use real-world human generated instructions,
have human-validated relevance judgements, propose a new paired evaluation
protocol, and provide a training dataset and model for teaching instruction-
following.

Dataset |  # $Q$ | $|Q|$ | $|I|$ | Rel. D/Q |  # $Q$ | $|I|$ | Rel. D/Q
---|---|---|---|---|---|---|---
TREC News ’21 (Soboroff et al., 2020) | 50 | 15.3 | 40.1 | 50.1 | 32 | 46.9 | 19.2
TREC Core ’17 (Allan et al., 2017) | 50 | 16.6 | 44.0 | 180.0 | 20 | 53.5 | 32.7
TREC Robust ’04 (Voorhees, 2005) | 249 | 11.9 | 68.2 | 69.9 | 52 | 75.2 | 19.8
Table 2: FollowIR evaluation set statistics before (left) and after (right)
annotation. We use a subset of the queries in three popular TREC tracks for
variety in queries and documents. $|Q|$ is the word length of the queries and
$|I|$ is the word length of the instructions. Rel. D/Q indicates the number of
relevant annotated documents in the collection, excluding irrelevant
annotations. As designed, there are less relevantly-judged documents in the
FollowIR portion (as the annotations change the relevance of documents on
purpose for evaluation).

##  3 Building FollowIR

We build FollowIR by starting with three existing TREC collections: TREC News
2021 that uses the Washington Post v4 corpus (Soboroff et al., 2020), TREC
Robust 2004 using the Disks 4 and 5 collections of various news articles
(Voorhees, 2005), and TREC Common Core 2017 using the New York Times Annotated
corpus (Allan et al., 2017). Each of these was human annotated to include
hundreds of annotations per query (see Table 2), with around 50-180 relevant
documents per query on average (and many more not-relevant annotations).

Each of these TREC tracks includes instructions for the human annotators that
we use as the instructions. Although using these alone can provide some
indication of how well models can follow instructions, it doesn’t explicitly
test their instruction following ability. To more carefully isolate this
ability in our benchmark, we test whether models can respond to small changes
in the instruction.

To accomplish this, we ask two expert human annotators to modify the TREC
instructions. However, doing this in a naive way would require re-annotating
all the document judgements, a non-trivial task requiring immense annotation
efforts.333NIST spends roughly 1-2 million per year:
https://trec.nist.gov/pubs/2010.economic.impact.pdf We crucially ask the
annotators to make the instructions more specific by including additional
aspects to the relevance definition or to include relevance instructions with
negation. This decreases the set of documents to re-annotate to only those
documents deemed relevant by the original TREC annotators (as it is then a
subset of the original relevant documents by definition). This makes the
annotation tractable, with only dozens or hundreds of documents to re-annotate
per query instead of needed to re-annotate the whole collection (e.g.
thousands).

We annotate a subset of the original TREC queries due to cost and overlap: we
sample 50 queries from TREC Robust 2004 that do not overlap with TREC Common
Core (as Common Core used 50 queries from Robust04 on a new collection), and
30 queries from TREC News 2021. Table 2 shows dataset statistics of judged
documents and the final benchmark size. Annotators were asked to change the
instructions so that the number of relevant documents was cut in half, thus
including a sizeable number of changed relevance judgements. We note that
although the number of queries seems small by NLP standards, 30-50 queries is
both effective (Webber et al., 2008) and standard in the IR community due to
the expense of careful annotation over many documents per query.

Due to differences in retriever quality, if we evaluate by searching over the
full collection, each model will retrieve a different number of relevant
documents. However, because we evaluate instruction following based on
changing the document relevance, models that do poorly in the initial
retrieval will have fewer documents which change relevance in the instruction-
following evaluation. To rectify this, we instead turn to a reranking task
where we include all relevant documents, and use a pool of five models444We
use BM25, BGE-base, E5-base-v2, TART-Contriever, and INSTRUCTOR-XL to select
the top non-relevant documents. To be able to freely distribute the data due
to fair-use laws, we chunk the documents into 400-word passages with 200-word
overlap and select the highest scoring passages using MaxP (Dai & Callan,
2019). This enables us to distribute our data, which we do by extending the
MTEB evaluation framework (Muennighoff et al., 2022).

###  3.1 Evaluation Metrics for FollowIR

Our benchmark provides two ways of measuring instruction following: (1)
standard retrieval metrics when using the instructions with the queries and
(2) pairwise evaluation of instruction following. (1) We use typical IR
evaluation metrics but use the instruction along with the query: these metrics
are mean average precision (MAP) for Core17/Robust04 and normalized discounted
cumulative gain at 5 (nDCG@5) for News21. For (2) we use our novel pairwise
evaluation metric that measures the delta in scores when following the
modified instructions instead of the original.555Note that we do not show
standard retrieval results on the modified instruction’s relevant document
set, as standard retrieval scores cannot be directly compared across different
query relevance annotations (qrels).

Our new pairwise evaluation metric, $p$-MRR, measures rank-wise changes
between queries. In developing this metric we had the following desiderata: it
should compare the results of the original instruction to those of the new
instruction, it should have a standardized range from worst possible change in
instruction-following score (i.e. -1) to best possible instruction-following
score (i.e. 1) with an option for no change when using different instructions
(i.e. 0), and finally should take into account the document rank so that
changes from rank 1 to rank 2 are more prominent than changes from rank 99 to
100. Given the above qualifications, we use the following equation applied to
each relevant document per query (where MRR is mean reciprocal rank, $R_{og}$
is the rank of the document when using the original instruction and $R_{new}$
is the new rank):

$\text{$p$-MRR}=\begin{cases}\frac{MRR_{og}}{MRR_{new}}-1&\text{if
$R_{og}>R_{new}$}\\\\[10.0pt]
1-\frac{MRR_{new}}{MRR_{og}}&\text{otherwise}\end{cases}$ (1)

For the final score, we average first within a given query and then over all
queries in the corpora – i.e. macro-averaging across queries, to handle the
different number of relevant documents per query.

##  4 Evaluating Instruction Following

In this section we describe the models we evaluate, results of those models on
FollowIR, and ablations performed to better understand the behavior of current
models.

###  4.1 Evaluation Settings

We evaluate a wide variety of IR models (trained with and without
instructions), including neural models ranging from 100 million to 7 billion
parameters. We evaluate on the original TREC instructions in the FollowIR
benchmark and then on the new instructions, showing both standard IR metrics
and the new pairwise metric $p$-MRR. We group models into four categories:

#### No Instructions in Training

These retrieval models did not see instructions in training and typically
aren’t given them: this includes Contriever (Izacard et al., 2021), E5 (Wang
et al., 2022a), MonoBERT Nogueira et al. (2019), MonoT5 (Nogueira et al.,
2020), and BM25 (Robertson et al., 1995).

#### Instructions in IR Training

Most retrieval models using instructions received roughly one instruction per
retrieval dataset, which generally defined the domain (e.g. “Financial"),
document size (sentence, passage, etc.), and task format. This includes
INSTRUCTOR models (Su et al., 2022), the bi-encoder TART model trained from
Contriever (Asai et al., 2022), the reranker TART trained from FLAN-T5 (Chung
et al., 2022), E5 Mistral-Instruct (Wang et al., 2023), and GritLM
(Muennighoff et al., 2024). We also include BGE models (Xiao et al., 2023) in
this category, although they are trained with only one instruction total for
each broad task (retrieval, clustering, etc.).

#### API Models

We use two of the best performing API embedding models: Cohere’s v3 English
and OpenAI’s Text-Embedding-v3-Large. It is unknown what these model’s
training procedures were – including if they were trained on instructions or
not – thus we place them in a distinct category.

#### Instruction-Tuned LLMs

We also evaluate several instruction-tuned LLMs to be used as rerankers,
including FLAN-T5 (Chung et al., 2022), Llama v2 (Touvron et al., 2023b), and
Mistral-Instruct-v0.2 (Jiang et al., 2023). We evaluate these models in the
same fashion as MonoT5 rerankers, comparing the true and false tokens. Note
that these models were not trained on any retrieval-specific data.

|  | Robust04 | News21 | Core17 | Average
---|---|---|---|---|---
Model | mAP |  $p$-MRR | nDCG |  $p$-MRR | mAP |  $p$-MRR | Score |  $p$-MRR
No-Instruction IR | BM25 | 12.2 |  -2.8 | 21.3 |  +2.5 | 8.1 |  -0.7 | 13.9 |  -0.3
E5-base-v2 | 14.5 |  -6.8 | 21.6 |  -4.1 | 14.0 |  -2.9 | 16.7 |  -4.6
E5-large-v2 | 18.1 |  -4.1 | 24.9 |  -2.2 | 17.0 |  +0.1 | 20.0 |  -2.1
Contriever | 20.3 |  -6.1 | 24.0 |  -1.8 | 15.3 |  -2.5 | 19.9 |  -3.5
MonoBERT | 21.5 |  -9.7 | 26.3 |  -4.4 | 18.4 |  -1.3 | 22.1 |  -5.1
MonoT5-base | 16.3 |  -5.8 | 11.9 |  -1.2 | 12.2 |  -3.5 | 13.5 |  -3.5
MonoT5-3B | 27.8 |  +5.6 | 18.6 |  +7.5 | 18.1 |  +1.7 | 21.5 |  +4.9
Instruction-IR | BGE-base | 17.5 |  -6.4 | 23.8 |  -0.2 | 14.6 |  -2.7 | 18.6 |  -3.1
BGE-large | 18.1 |  -7.8 | 26.4 |  +0.1 | 15.0 |  +0.1 | 19.8 |  -2.5
TART-Contriever | 14.1 |  -7.8 | 21.9 | +0.0 | 12.4 |  -1.3 | 16.1 |  -3.0
INSTRUCTOR-base | 14.4 |  -5.6 | 16.3 |  -2.5 | 14.7 |  -2.2 | 15.1 |  -3.4
INSTRUCTOR-xl | 15.5 |  -2.1 | 14.6 |  -4.3 | 14.4 |  -0.6 | 14.8 |  -2.3
TART-FLAN-T5-xl | 25.2 |  -0.8 | 20.3 |  -1.1 | 17.0 |  +2.8 | 20.8 |  +0.3
GritLM-7B | 29.0 |  -1.4 | 25.2 |  +2.1 | 20.8 |  +2.6 | 25.0 |  +1.1
APIs | Cohere v3 English | 22.9 |  -3.3 | 23.6 |  -3.1 | 20.6 |  +2.7 | 22.4 |  -1.2
OpenAI v3 Large | 27.9 |  -5.7 | 30.0 |  -3.3 | 21.4 |  -0.2 | 26.4 |  -3.1
Instruct LMs | FLAN-T5-base | 6.8 |  +5.0 | 2.2 |  +1.1 | 6.5 |  -3.2 | 5.2 |  +1.0
FLAN-T5-large | 15.1 |  +4.0 | 8.5 |  +7.7 | 11.5 |  +1.2 | 11.7 |  +4.3
Llama-2-7B-chat | 6.9 |  +1.6 | 13.3 |  +2.1 | 5.4 |  +3.6 | 8.5 |  +2.4
Mistral-7B-instruct | 24.1 |  +12.2 | 22.9 |  +10.5 | 19.6 |  +13.4 | 22.2 |  +12.0
FollowIR-7B | 25.9 |  +13.6 | 25.7 |  +10.8 | 20.0 |  +16.3 | 23.9 |  +13.6

Table 3: Evaluating instruction-following on FollowIR. $p$-MRR is a new
pairwise evaluation metric measuring instruction following when instructions
change, ranging from -100 to 100 (higher is better). We see that the only
models that show any success at following instructions are large models (3B+
parameters) or instruction-tuned LLMs that haven’t been trained on retrieval
tasks.

###  4.2 Results

Table 3 shows the main results on FollowIR, with the standard IR score shown
(either MAP or nDCG@5) as well as the pairwise evaluation metric, $p$-MRR.

#### No-Instruction IR Models

We see that the no-instruction models range in standard IR metrics (in terms
of nDCG@5 and MAP) but generally have negative scores for $p$-MRR (up to
-9.7). The only non-instruction model to score positively on average is
MonoT5-3B.

#### Instruction IR Models

We again see that these models have generally negative scores, with the
exception being TART-FLAN-T5-xl (with scores averaging roughly zero) and
GritLM which has slightly positive scores for two of the three datasets.

Figure 2: Score difference when changing from no instructions to using
instructions formatted as keywords, short text, or the full text. We see that
models that can correctly use instructions see gains with the additional
information while most other models see decreasing performance as instruction
length increases. This indicates that most models struggle with instruction
length, using these instructions for keyword search rather than as
instructions that define document relevance.

#### API Models

We see that the API models perform strongly in terms of standard IR metrics,
with OpenAI’s model performing the highest overall. However, both OpenAI and
Cohere’s models perform poorly at instruction-following with negative scores
(-3.1 and -2.8 on average, respectively).

#### Instruct-Tuned LLMs

In contrast to the previous results, all instruction-tuned LLMs show positive
results for instruction following, although they have the widest range of
performance using standard IR metrics (ranging from very poor scores to some
of the higher scores). We see that the best performing model in this category
is FollowIR–7B, which we describe in more detail in Section 5.

#### Overall

We see that the only models that show positive results at following
instructions are either larger IR models (3B+ parameters) or those that have
been explicitly trained to follow instructions (FLAN-T5) without retrieval-
specific training data. This aligns with work in the natural language
processing community which has shown that the instruction-following ability
improves with scale (Brown et al., 2020) and instruct-tuning (Longpre et al.,
2023).

###  4.3 Analysis

Why do so many models fail to correctly follow instructions when they do well
on typical IR metrics such as nDCG and MAP? We answer this question by
ablating several components that may impact results: (1) whether IR models are
not used to instructions that cannot be used for keyword search and (2)
whether they are unused to the length of the longer instruction (as
instruction-IR models have been trained on much shorter instructions).

We compare the original query-only result to those where we additionally give
the model either the full instruction, a shorter instruction, or keywords from
the instruction. We gather these short instructions and keywords by prompting
GPT-4-Turbo-1106 to generate them from the original full instruction (for TREC
data) or the original short instruction used by the model (for BEIR data). For
the full prompt text, please see Appendix E.

We show results for these ablations in Table 2, where positive scores indicate
that adding information improves the model while negative scores indicate a
drop in performance. We see a consistent trend where models that did poorly on
longer instructions perform better on keywords and shorter instructions than
with the full instruction. However, models that are able to follow
instructions see better results with the additional information, on average.

These results show that models are (1) using the instruction text as keywords
and (2) are unused to longer (with less relevant) text that is in the
instructions.

We also confirm that these results hold on datasets outside of TREC
collections and show results on three BEIR datasets: SciFact, NFCorpus, and
FiQA. We show in Table 4 the original score (using the short instructions from
their papers) and the change in score when using just keywords from the
instruction (again extracted from GPT-4). We show results only for models
which performed poorly for instruction-following. We see that the scores for
keywords vs the short instruction are generally similar, with most models
seeing a change of around $\pm$ 1 point, except for the strongest instruction-
following model, E5-Mistral, seeing a larger drop on some datasets.

#### Overall

We find overall (on both TREC and BEIR datasets) that models use instructions
for keyword matching and are unused to longer instructions that may contain
slightly less relevant words.

|  | SciFact | NFCorpus | FiQA
---|---|---|---|---
Model | OG |  $\Delta$ w/Key. | OG |  $\Delta$ w/Key. | OG |  $\Delta$ w/Key.
No-Instruction | BM25 | 67.9 | -1.7 | 32.2 | -5.1 | 23.6 | -1.6
E5-base-v2 | 71.9 | -2.7 | 35.4 | -2.5 | 39.9 | -0.4
Contriever | 64.9 | +0.4 | 31.7 | +0.0 | 24.5 | -3.2
MonoT5-base | 73.1 | -0.6 | 35.6 | -0.9 | 41.2 | -0.3
Uses Instruction | TART-Contriever | 67.6 | -0.3 | 33.4 | -5.3 | 31.8 | -0.4
INSTRUCTOR-base | 57.8 | +1.0 | 31.6 | -0.4 | 39.2 | -0.1
BGE-base | 73.2 | -0.5 | 35.5 | +0.0 | 40.8 | -2.3
TART-FLAN-xl | 74.2 | +1.6 | 33.9 | +0.4 | 39.6 | -0.3
INSTRUCTOR-xl | 62.4 | +0.2 | 36.0 | -0.6 | 46.9 | +0.8
E5-Mistral | 77.1 | -5.1 | 38.8 | +0.3 | 56.7 | -6.5
Table 4: Ablation on BEIR benchmarks for models that do poorly with longer
instructions, comparing their original short instructions vs domain keywords
extracted from those instructions (see Appendix D for a list). If models had
learned to use the instructions correctly we would see a divergence between
the behavior of instruct and non-instruct models, however, for both we see
that using keywords instead of the instruction results in comparable
performance ($\pm$ one point).

##  5 Teaching Instruction Following

Is it possible to improve model performance in following instructions? We seek
to show that fine-tuning on a training set of longer instructions can provide
a method for doing so.

We start by gathering a training set to teach models. We collect all TREC
narratives (i.e. instructions) from tasks not in FollowIR, consisting of 1836
pairs of queries and narratives. However, we note that this does not provide
any positive or negative documents for fine-tuning.

In order to obtain documents for training, we prompt GPT-3.5-Turbo-1106 to
generate relevant and not-relevant documents, generating roughly two relevant
and non-relevant instances per query. However, these synthetic documents are
noisy and contains errors w.r.t. the labels – to remedy this, we perform a
round of filtering and use the best performing open-source model from Table 3
(Mistral-7B-Instruct-v0.2) to score each of the generated documents according
to the instruction. We then filter the documents according to whether Mistral
correctly predicts the generated label, and finally balance the relevant and
non-relevant samples, choosing only one relevant and non-relevant document per
query. Our total is $\sim$1800 training instances on $\sim$1200 unique
query/instructions pairs.

We then train our instruction-following model, FollowIR-7B, by fine-tuning
Mistral-7B-Instruct-v0.2 on our data using the Llama-Factory framework
(Hiyouga, 2023) with LoRA (Hu et al., 2021). Full training hyperparameter
details are found in Appendix A.

When we evaluate this model on FollowIR (Table 3), we find that the scores
consistently improve. Compared to the original Mistral-7B-Instruct-v0.2, our
model improves on both standard IR metrics (+7.6% relative improvement) and on
instruction following (+13.3% relative improvement). Thus, we can see that it
is possible to train IR models to be better instruction followers.

##  6 Conclusion

Despite the use of LLMs as the backbone of neural retrieval models, most
existing IR models do not take instructions that define document relevance.
Further, there is no existing resource that measures how well retrieval models
can follow instructions. We build a new benchmark that explicitly measures the
instruction following ability of retrieval models and find that nearly all
retrieval models do not follow instructions, with the exception of very large
models (3B+ parameters) or instruction-tuned LLMs that typically are not used
for retrieval. However, we show that it is possible to improve their
instruction following ability, and build and release a training corpus for
teaching retrieval models to follow instructions. Our new model, FollowIR-7B,
shows improvement on both standard retrieval metrics as well as in instruction
following. We hope that these resources will allow the community to develop
more capable instruction-following retrieval models that can quickly adapt to
a relevance definition given flexible natural language text.

##  7 Limitations

#### Reranking vs Full Retrieval

As evaluating instruction following requires evaluating the documents which
changed relevance, we cannot use the full collection for retrieval (as each
retriever finds different relevant documents by design). Further, due to
licensing restrictions, we cannot distribute the full corpora from the TREC
tracks – thus we distribute passages due to fair use laws. However, we show
full corpus retrieval results for a subset of models in Appendix C and note
similar trends.

#### Possible Errors

Our work is built on the TREC document collections and judgements, as well as
new annotation efforts. We do not check for potential errors in the TREC
annotations, and our newly gathering annotations may have small errors.
Despite these caveats, we see that our dataset still provides a useful
evaluation setup for measuring instruction following.

## Acknowledgments

Orion Weller is supported by a National Science Foundation Graduate Research
Fellowship Program grant. We also thank Marc Marone for their feedback and
comments that helped improve this work.

## References

  * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ , 2023.
  * Allan et al. (2017) James Allan, Donna Harman, Evangelos Kanoulas, Dan Li, Christophe Van Gysel, and Ellen M Voorhees.  Trec 2017 common core track overview.  In _TREC_ , 2017.
  * Asai et al. (2022) Akari Asai, Timo Schick, Patrick Lewis, Xilun Chen, Gautier Izacard, Sebastian Riedel, Hannaneh Hajishirzi, and Wen-tau Yih.  Task-aware retrieval with instructions.  _arXiv preprint arXiv:2211.09260_ , 2022.
  * Baron et al. (2006) Jason R Baron, David D Lewis, and Douglas W Oard.  Trec 2006 legal track overview.  In _TREC_. Citeseer, 2006.
  * Black et al. (2022) Sid Black, Stella Biderman, Eric Hallahan, Quentin G. Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Martin Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Benqi Wang, and Samuel Weinbach.  Gpt-neox-20b: An open-source autoregressive language model.  _ArXiv_ , abs/2204.06745, 2022.  URL https://api.semanticscholar.org/CorpusID:248177957.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Chen et al. (2024) Haonan Chen, Zhicheng Dou, Kelong Mao, Jiongnan Liu, and Ziliang Zhao.  Generalizing conversational dense retrieval via llm-cognition data augmentation.  _arXiv preprint arXiv:2402.07092_ , 2024.
  * Chen et al. (2023) Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu.  Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2023.
  * Chung et al. (2022) Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al.  Scaling instruction-finetuned language models.  _arXiv preprint arXiv:2210.11416_ , 2022.
  * Dai & Callan (2019) Zhuyun Dai and Jamie Callan.  Deeper text understanding for ir with contextual neural language modeling.  In _Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval_ , pp. 985–988, 2019.
  * Groeneveld et al. (2024) Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, A. Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Daniel Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, and Hanna Hajishirzi.  Olmo: Accelerating the science of language models.  2024\.  URL https://api.semanticscholar.org/CorpusID:267365485.
  * Hiyouga (2023) Hiyouga.  Llama factory.  https://github.com/hiyouga/LLaMA-Factory, 2023.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_ , 2021.
  * Izacard et al. (2021) Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave.  Unsupervised dense information retrieval with contrastive learning.  _arXiv preprint arXiv:2112.09118_ , 2021.
  * Jiang et al. (2023) Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L’elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed.  Mistral 7b.  _ArXiv_ , abs/2310.06825, 2023.  URL https://api.semanticscholar.org/CorpusID:263830494.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih.  Dense passage retrieval for open-domain question answering.  _arXiv preprint arXiv:2004.04906_ , 2020.
  * Khattab & Zaharia (2020) Omar Khattab and Matei Zaharia.  Colbert: Efficient and effective passage search via contextualized late interaction over bert.  In _Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval_ , pp. 39–48, 2020.
  * Lawrie et al. (2024) Dawn Lawrie, Sean MacAvaney, James Mayfield, Paul McNamee, Douglas W. Oard, Luca Soldaini, and Eugene Yang.  Overview of the TREC 2023 NeuCLIR track.  2024\.
  * Longpre et al. (2023) Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al.  The flan collection: Designing data and methods for effective instruction tuning.  In _International Conference on Machine Learning_ , pp. 22631–22648. PMLR, 2023.
  * Ma et al. (2023) Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin.  Fine-tuning llama for multi-stage text retrieval.  _arXiv preprint arXiv:2310.08319_ , 2023.
  * Muennighoff et al. (2022) Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers.  Mteb: Massive text embedding benchmark.  _arXiv preprint arXiv:2210.07316_ , 2022.
  * Muennighoff et al. (2024) Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela.  Generative representational instruction tuning.  _arXiv preprint arXiv:2402.09906_ , 2024.
  * Nogueira et al. (2019) Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin.  Multi-stage document ranking with bert.  _arXiv preprint arXiv:1910.14424_ , 2019.
  * Nogueira et al. (2020) Rodrigo Nogueira, Zhiying Jiang, and Jimmy Lin.  Document ranking with a pretrained sequence-to-sequence model.  _arXiv preprint arXiv:2003.06713_ , 2020.
  * Oard et al. (2008) Douglas W Oard, Björn Hedin, Stephen Tomlinson, and Jason R Baron.  Overview of the trec 2008 legal track.  In _TREC_ , pp. 500–277, 2008.
  * Oh et al. (2024) Hanseok Oh, Hyunji Lee, Seonghyeon Ye, Haebin Shin, Hansol Jang, Changwook Jun, and Minjoon Seo.  Instructir: A benchmark for instruction following of information retrieval models.  _arXiv preprint arXiv:2402.14334_ , 2024.
  * Ouyang et al. (2022) Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.  Training language models to follow instructions with human feedback.  _Advances in Neural Information Processing Systems_ , 35:27730–27744, 2022.
  * Pradeep et al. (2023) Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin.  Rankzephyr: Effective and robust zero-shot listwise reranking is a breeze!  _arXiv preprint arXiv:2312.02724_ , 2023.
  * Qin et al. (2023) Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al.  Toolllm: Facilitating large language models to master 16000+ real-world apis.  _arXiv preprint arXiv:2307.16789_ , 2023.
  * Reimers & Gurevych (2019) Nils Reimers and Iryna Gurevych.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084_ , 2019.
  * Robertson et al. (1995) Stephen E Robertson, Steve Walker, Susan Jones, Micheline M Hancock-Beaulieu, Mike Gatford, et al.  Okapi at trec-3.  _Nist Special Publication Sp_ , 109:109, 1995.
  * Shaghaghian et al. (2020) Shohreh Shaghaghian, Luna Yue Feng, Borna Jafarpour, and Nicolai Pogrebnyakov.  Customizing contextualized language models for legal document reviews.  In _2020 IEEE International Conference on Big Data (Big Data)_ , pp. 2139–2148. IEEE, 2020.
  * Singhal et al. (2023) Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al.  Large language models encode clinical knowledge.  _Nature_ , 620(7972):172–180, 2023.
  * Soboroff (2021) Ian Soboroff.  Overview of trec 2021.  In _30th Text REtrieval Conference. Gaithersburg, Maryland_ , 2021.
  * Soboroff et al. (2018) Ian Soboroff, Shudong Huang, and Donna Harman.  Trec 2018 news track overview.  In _TREC_ , volume 409, pp. 410, 2018.
  * Soboroff et al. (2020) Ian Soboroff, Shudong Huang, and Donna Harman.  Trec 2020 news track overview.  In _TREC_ , 2020.
  * Su et al. (2022) Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu.  One embedder, any task: Instruction-finetuned text embeddings.  2022\.  URL https://arxiv.org/abs/2212.09741.
  * Thakur et al. (2021) Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych.  Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models.  _arXiv preprint arXiv:2104.08663_ , 2021.
  * Touvron et al. (2023a) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023a.
  * Touvron et al. (2023b) Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom.  Llama 2: Open foundation and fine-tuned chat models.  _ArXiv_ , abs/2307.09288, 2023b.  URL https://api.semanticscholar.org/CorpusID:259950998.
  * Voorhees (2005) Ellen M Voorhees.  The trec robust retrieval track.  In _ACM SIGIR Forum_ , volume 39, pp. 11–20. ACM New York, NY, USA, 2005.
  * Wang et al. (2022a) Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei.  Text embeddings by weakly-supervised contrastive pre-training.  _arXiv preprint arXiv:2212.03533_ , 2022a.
  * Wang et al. (2023) Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei.  Improving text embeddings with large language models.  _arXiv preprint arXiv:2401.00368_ , 2023.
  * Wang et al. (2022b) Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi.  Self-instruct: Aligning language model with self generated instructions.  _arXiv preprint arXiv:2212.10560_ , 2022b.
  * Wang et al. (2022c) Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al.  Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks.  _arXiv preprint arXiv:2204.07705_ , 2022c.
  * Webber et al. (2008) William Webber, Alistair Moffat, and Justin Zobel.  Statistical power in retrieval experimentation.  In _Proceedings of the 17th ACM conference on Information and knowledge management_ , pp. 571–580, 2008.
  * Weller et al. (2020) Orion Weller, Nicholas Lourie, Matt Gardner, and Matthew E Peters.  Learning from task descriptions.  _arXiv preprint arXiv:2011.08115_ , 2020.
  * Weller et al. (2023) Orion Weller, Kyle Lo, David Wadden, Dawn Lawrie, Benjamin Van Durme, Arman Cohan, and Luca Soldaini.  When do generative query and document expansions fail? a comprehensive study across methods, retrievers, and datasets.  _arXiv preprint arXiv:2309.08541_ , 2023.
  * Weller et al. (2024) Orion Weller, Dawn J Lawrie, and Benjamin Van Durme.  Nevir: Negation in neural information retrieval.  _Conference of the European Chapter of the Association for Computational Linguistics_ , 2024.  URL https://api.semanticscholar.org/CorpusID:258676146.
  * Xiao et al. (2023) Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff.  C-pack: Packaged resources to advance general chinese embedding, 2023.
  * Zhao et al. (2021) Ming Zhao, Peter Anderson, Vihan Jain, Su Wang, Alexander Ku, Jason Baldridge, and Eugene Ie.  On the evaluation of vision-and-language navigation instructions.  _arXiv preprint arXiv:2101.10504_ , 2021.

##  Appendix A Hyperparameters for Fine-Tuning Mistral

We fine-tune Mistral-7B-Instruct-v0.2 using the Mistral template for 8 epochs.
We use LoRA and tune the “q_proj,v_proj,o_proj,k_proj" parameters, using a
rank of 8 and an alpha of 16. We use bfloat16, a batch size of 32, and a
learning rate of 3e-5.

##  Appendix B Hyperparameters for Inference

We use default parameters for inference, taken from the original code of the
authors of the papers we use (from their MTEB evaluations).

##  Appendix C Full Retrieval Results

In Table 5 we show results for models searching on the full collections of the
TREC tasks included in FollowIR. Note that because each model retrieves
different relevant documents, the instruction-following evaluation has a
different set of instances that each model is evaluated on (as it can only be
evaluated on documents it retrieved that then become not-relevant).

|  | Robust04 | News21 | Core17
---|---|---|---|---
|  | (mAP) | (nDCG@5) | (mAP)
Model | OG | $\Delta$ | OG | $\Delta$ | OG | $\Delta$
No
Instruct | BM25 | 21.4 | -1.2 | 30.1 | +5.3 | 16.8 | -0.2
E5-base-v2 | 22.7 | -7.0 | 33.6 | +1.8 | 19.7 | -3.0
Contriever | 19.2 | -7.7 | 22.5 | +9.0 | 22.6 | -7.6
Uses
Instruct | TART-Contriever | 25.5 | -10.1 | 40.0 | -5.0 | 22.6 | -7.6
BGE-base | 23.6 | -3.1 | 36.5 | -7.8 | 23.0 | -2.1
INSTRUCTOR-base | 22.5 | -2.2 | 33.3 | -2.8 | 20.0 | -0.2
INSTRUCTOR-XL | 30.4 | -3.1 | 38.1 | -0.1 | 29.9 | -2.8
Table 5: FollowIR scores on the full retrieval collection (thus rerankers are
not included). As the base score is different, there are different numbers of
relevant documents they are being evaluated on for $p$-MRR. Thus, we only
report the original (no-instruction) score and the delta when using the TREC
instructions. We note that it shows similar results to the main text –
retrieval models are not effectively using instructions and see performance
degradations with longer text.

##  Appendix D Keywords used for BEIR experiments

GPT-4-Turbo-1106 extracted the following keywords (Table 6) from the
instructions these models used, which generated the results in Table 4.

Model | Dataset | Keywords
---|---|---
BM25/Contriever/E5/MonoT5 | FiQA | Finance web
BM25/Contriever/E5/MonoT5 | SciFact | science paper verify
BM25/Contriever/E5/MonoT5 | NFCorpus | medicine relevant
TART-dual | FiQA | financial web
TART-dual | SciFact | scientific paper verify
TART-dual | NFCorpus | scientific paper paragraph
INSTRUCTOR-base | FiQA | financial supporting:
INSTRUCTOR-base | SciFact | scientific supporting passage:
INSTRUCTOR-base | NFCorpus | medicine relevant
BGE-base | FiQA | relevant passages:
BGE-base | SciFact | relevant passages:
BGE-base | NFCorpus | relevant passages:
INSTRUCTOR-xl | FiQA | finance supporting:
INSTRUCTOR-xl | SciFact | scientific supporting passages:
INSTRUCTOR-xl | NFCorpus | nutrition facts public medical:
E5-Mistral | FiQA | financial replies
E5-Mistral | SciFact | scientific
E5-Mistral | NFCorpus | retrieve relevant
TART-T5-FLAN-xl | FiQA | financial web
TART-T5-FLAN-xl | SciFact | scientific paper verify
TART-T5-FLAN-xl | NFCorpus | Scientific paper paragraph
Table 6: Keywords used for the BEIR keyword analysis. Note that non-
instruction models received the keywords used in INSTRUCTOR-base and TART-dual
(as shown in the table).

##  Appendix E Prompts Used

We use these prompts for generating the short instructions, the keywords, and
the synthetic documents. The examples used in the prompt for the “Full
Instructions to Short Instructions" prompt were partially created by the
authors, as only the short instructions were provided by TART/INSTRUCTOR.

Full Instructions to Short Instructions I have instructions that are specific
to a question, but I need your help abstracting them to a general task format
that I can give to someone else. I need you to turn them into an abstract
command that just describe the general abstract task instead (e.g., where the
data is from, what the type of document looks like). It is crucial that you
read and follow these instructions, you will get a large bonus if you are
successful ($200).
The abstract command should only mention the **task format**. Do **not** refer
to any entities or specific text in the original instruction. Your response
should be around 10 words. The command should be as if you were speaking to
another human.
## Examples
### Example 1:
Original Instruction: A relevant document would provide information about the
whole blood-base perfusate and whether or not it provides superior
preservation of myocardial function during ex vivo heart perfusion. This may
include research experiments, commentary, or survey/review papers. Information
about whole blood-base perfusate alone is not relevant, unless it also
mentions it’s effect on myocardial function during ex vivo heart perfusion.
Abstract Command: Help me to find a highly related PubMed paper to answer this
question.
### Example 2:
Original Instruction: A relevant document will contain information that about
the right of way in international waters that can be used to determine who
should have the right of way in a given situation. For example, it should
contain instances about who is at fault in an accident, if it depends on the
size of the boat, or details about how this differs according to nationality.
Especially relevant are documents describing who is at fault in a crash
situation.
Abstract Command: Retrieve a Wikipedia paragraph that answers this question.
### Example 3:
Original Instruction: A relevant instance will be a question that is
semantically equivalent to the query given. For example, it may contain
different lexical words or be a paraphrase of the other, but the underlying
meaning will be the same. If the instance is not semantically the same as the
query, it is irrelevant.
Abstract Command: Check if a Quora question is duplicated with this question.
### Example 4:
Original Instruction: A relevant document would include details about the
timing of medicare and what age patients can start using it for healthcare. It
may include information about laws, insurance, or other details that describe
the age the medicare begins. Less relevant are documents talking about
potential laws or facts about medicare that do not answer the question of what
age medicare begins. Just the mention of an age and when to start medicare
would be relevant.
Abstract Command: I want to know the answer to the question. Can you find good
evidence on the web?
Now you can easily see that the abstract command is vague and describes only a
short command about how to get the information you need. Follow this exactly –
do not reference specifics (like in the above, "international waters" and
"medicare" are not included in the abstract command). You should instead keep
the abstract command vague and well, abstract about the task only. Use the
word "question".
## Your turn
Original Instruction: FILL_TEXT_HERE
Abstract Command (remember to use the word "question"):

Short Instructions to Keywords I have instructions that are specific to a
style of retrieval, but I want you to instead just focus on the relevant
keywords that are in these instructions. Your job is to return a list of these
keywords that are relevant in the query. There are probably one or two
relevant keywords to extract only. ## Examples
### Example 1:
Instruction: Help me to find a highly related PubMed paper to answer this
question.
Keywords: ["PubMed"]
### Example 2:
Instruction: I want to find an answer for this Trivia question. Can you find
some paragraphs that provide evidence from Wikipedia?
Keywords: ["Trivia", "Wikipedia"]
### Example 3:
Instruction: Check if a Quora question is duplicated with this question.
Keywords: ["Quora", "duplicated"]
### Example 4:
Instruction: I want to find a related question asked in StackExchange. Can you
find one for me?
Keywords: ["related", "StackExchange"]
## Your turn
Instruction: FILL_TEXT_HERE
Keywords (either one or two keywords, that are not "documents", "questions",
"answer", or "articles"):

Synthetic Document Creation I need you to annotate some data for my business
and it is super important that you follow instructions precisely or you will
be fired. Given a Google search query and instructions regarding what makes a
document relevant, I need you to write two documents: one that would be
relevant and one that would not. Search: TITLE_HERE Instructions:
NARRATIVE_HERE I need some different options to choose from, so give me three
**different** options for both a relevant document and an irrelevant document.
They should be **long** paragraph-sized documents ($\sim$300 words each), one
on each line. If there is no negation in the instructions, your irrelevant
document should be slightly off topic:


# RAGAS: Automated Evaluation of Retrieval Augmented Generation

Shahul Es†, Jithin James†, Luis Espinosa-Anke∗♢, Steven Schockaert∗
†Exploding Gradients
∗CardiffNLP, Cardiff University, United Kingdom
♢AMPLYFI, United Kingdom
shahules786<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

We introduce RAGAs (Retrieval Augmented Generation Assessment), a framework
for reference-free evaluation of Retrieval Augmented Generation (RAG)
pipelines. RAG systems are composed of a retrieval and an LLM based generation
module, and provide LLMs with knowledge from a reference textual database,
which enables them to act as a natural language layer between a user and
textual databases, reducing the risk of hallucinations. Evaluating RAG
architectures is, however, challenging because there are several dimensions to
consider: the ability of the retrieval system to identify relevant and focused
context passages, the ability of the LLM to exploit such passages in a
faithful way, or the quality of the generation itself. With RAGAs, we put
forward a suite of metrics which can be used to evaluate these different
dimensions without having to rely on ground truth human annotations. We posit
that such a framework can crucially contribute to faster evaluation cycles of
RAG architectures, which is especially important given the fast adoption of
LLMs.

##  1 Introduction

Language Models (LMs) capture a vast amount of knowledge about the world,
which allows them to answer questions without accessing any external sources.
This idea of LMs as repositories of knowledge emerged shortly after the
introduction of BERT Devlin et al. (2019) and became more firmly established
with the introduction of ever larger LMs Roberts et al. (2020). While the most
recent Large Language Models (LLMs) capture enough knowledge to rival human
performance across a wide variety of question answering benchmarks Bubeck et
al. (2023), the idea of using LLMs as knowledge bases still has two
fundamental limitations. First, LLMs are not able to answer questions about
events that have happened after they were trained. Second, even the largest
models struggle to memorise knowledge that is only rarely mentioned in the
training corpus Kandpal et al. (2022); Mallen et al. (2023). The standard
solution to these issues is to rely on _Retrieval Augmented Generation (RAG)_
Lee et al. (2019); Lewis et al. (2020); Guu et al. (2020). Answering a
question then essentially involves retrieving relevant passages from a corpus
and feeding these passages, along with the original question, to the LM. While
initial approaches relied on specialised LMs for retrieval-augmented language
modelling Khandelwal et al. (2020); Borgeaud et al. (2022), recent work has
suggested that simply adding retrieved documents to the input of a standard LM
can also work well Khattab et al. (2022); Ram et al. (2023); Shi et al.
(2023), thus making it possible to use retrieval-augmented strategies in
combination with LLMs that are only available through APIs.

While the usefulness of retrieval-augmented strategies is clear, their
implementation requires a significant amount of tuning, as the overall
performance will be affected by the retrieval model, the considered corpus,
the LM, or the prompt formulation, among others. Automated evaluation of
retrieval-augmented systems is thus paramount. In practice, RAG systems are
often evaluated in terms of the language modelling task itself, i.e. by
measuring perplexity on some reference corpus. However, such evaluations are
not always predictive of downstream performance Wang et al. (2023c). Moreover,
this evaluation strategy relies on the LM probabilities, which are not
accessible for some closed models (e.g. ChatGPT and GPT-4). Question answering
is another common evaluation task, but usually only datasets with short
extractive answers are considered, which may not be representative of how the
system will be used.

To address these issues, in this paper we present RAGAs111RAGAs is available
at https://github.com/explodinggradients/ragas., a framework for the automated
assessment of retrieval augmented generation systems. We focus on settings
where reference answers may not be available, and where we want to estimate
different proxies for correctness, in addition to the usefulness of the
retrieved passages. The RAGAs framework provides an integration with both
llama-index and Langchain, the most widely used frameworks for building RAG
solutions, thus enabling developers to easily integrate RAGAs into their
standard workflow.

##  2 Related Work

### Estimating faithfulness using LLMs

The problem of detecting hallucinations in LLM generated responses has been
extensively studied Ji et al. (2023). Several authors have suggested the idea
of predicting factuality using a few-shot prompting strategy Zhang et al.
(2023). Recent analyses, however, suggest that existing models struggle with
detecting hallucination when using standard prompting strategies Li et al.
(2023); Azaria and Mitchell (2023). Other approaches rely on linking the
generated responses to facts from an external knowledge base Min et al.
(2023), but this is not always possible.

Yet another strategy is to inspect the probabilities assigned to individual
tokens, where we would expect the model to be less confident in hallucinated
answers than in factual ones. For instance, BARTScore Yuan et al. (2021)
estimates factuality by looking at the conditional probability of the
generated text given the input. Kadavath et al. (2022) use a variation of this
idea. Starting from the observation that LLMs provide well-calibrated
probabilities when answering multiple-choice questions, they essentially
convert the problem of validating model generated answers into a multiple-
choice question which asks whether the answer is true or false. Rather than
looking at the output probabilities, Azaria and Mitchell (2023) propose to
train a supervised classifier on the weights from one of the hidden layers of
the LLM, to predict whether a given statement is true or not. While the
approach performs well, the need to access the hidden states of the model
makes it unsuitable for systems that access LLMs through an API.

For models that do not provide access to token probabilities, such as ChatGPT
and GPT-4, different methods are needed. SelfCheckGPT Manakul et al. (2023)
addresses this problem by instead sampling multiple answers. Their core idea
is that factual answers are more stable: when an answer is factual, we can
expect that different samples will tend to be semantically similar, whereas
this is less likely to be the case for hallucinated answers.

### Automated evaluation of text generation systems

LLMs have also been leveraged to automatically evaluate other aspects of
generated text fragments, beyond factuality. For instance, GPTScore Fu et al.
(2023) uses a prompt that specifies the considered aspect (e.g. fluency) and
then scores passages based on the average probability of the generated tokens,
according to a given autoregressive LM. This idea of using prompts was
previously also considered by Yuan et al. (2021), although they used a smaller
fine-tuned LM (i.e. BART) and did not observe a clear benefit from using
prompts. Another approach directly asks ChatGPT to evaluate a particular
aspect of the given answer by providing a score between 0 and 100, or by
providing a rating on a 5-star scale Wang et al. (2023a). Remarkably, strong
results can be obtained in this way, although it comes with the limitation of
being sensitive to the design of the prompt. Rather than scoring individual
answers, some authors have also focused on using an LLM to select the best
answer among a number of candidates Wang et al. (2023b), typically to compare
the performance of different LLMs. However, care is needed with this approach,
as the order in which the answers is presented can influence the result Wang
et al. (2023b).

In terms of how ground truth answers or, more generally, generations, have
been typically used in the literature, most approaches have relied on the
availability of one or more reference answers. For instance, BERTScore Zhang
et al. (2020) and MoverScore Zhao et al. (2019) use contextualised embeddings,
produced by a pre-trained BERT model, to compare the similarity between the
generated answer and the reference answers. BARTScore Yuan et al. (2021)
similarly uses reference answers to compute aspects such as precision
(estimated as the probability of generating the generated answer given the
reference) and recall (estimated as the probability of generating the
reference given the generated answer).

##  3 Evaluation Strategies

We consider a standard RAG setting, where given a question $q$, the system
first retrieves some context $c(q)$ and then uses the retrieved context to
generate an answer $a_{s}(q)$. When building a RAG system, we usually do not
have access to human-annotated datasets or reference answers. We therefore
focus on metrics that are fully self-contained and reference-free. We focus in
particular three quality aspects, which we argue are of central importance.
First, Faithfulness refers to the idea that the answer should be grounded in
the given context. This is important to avoid hallucinations, and to ensure
that the retrieved context can act as a justification for the generated
answer. Indeed, RAG systems are often used in applications where the factual
consistency of the generated text w.r.t. the grounded sources is highly
important, e.g. in domains such as law, where information is constantly
evolving. Second, Answer Relevance refers to the idea that the generated
answer should address the actual question that was provided. Finally, Context
Relevance refers to the idea that the retrieved context should be focused,
containing as little irrelevant information as possible. This is important
given the cost associated with feeding long context passages to LLMs.
Moreover, when context passages are too long, LLMs are often less effective in
exploiting that context, especially for information that is provided in the
middle of the context passage Liu et al. (2023).

We now explain how these three quality aspects can be measured in a fully
automated way, by prompting an LLM. In our implementation and experiments, all
prompts are evaluated using the gpt-3.5-turbo-16k model, which is available
through the OpenAI API222https://platform.openai.com.

### Faithfulness

We say that the answer $a_{s}(q)$ is faithful to the context $c(q)$ if the
claims that are made in the answer can be inferred from the context. To
estimate faithfulness, we first use an LLM to extract a set of statements,
$S(a_{s}(q))$. The aim of this step is to decompose longer sentences into
shorter and more focused assertions. We use the following prompt for this
step333To help clarify the task, we include a demonstration as part of the
prompt. This demonstration is not explicitly shown in the listing of the
prompts throughout this paper.:

> Given a question and answer, create one or more statements from each
> sentence in the given answer.
> question: [question]
> answer: [answer]

where [question] and [answer] refer to the given question and answer. For each
statement $s_{i}$ in $S$, the LLM determines if $s_{i}$ can be inferred from
$c(q)$ using a verification function $v(s_{i},c(q))$. This verification step
is carried out using the following prompt:

> Consider the given context and following statements, then determine whether
> they are supported by the information present in the context. Provide a
> brief explanation for each statement before arriving at the verdict
> (Yes/No). Provide a final verdict for each statement in order at the end in
> the given format. Do not deviate from the specified format.
> statement: [statement 1]
> …
> statement: [statement $n$]

The final faithfulness score, $F$, is then computed as $F=\frac{|V|}{|S|}$,
where $|V|$ is the number of statements that were supported according to the
LLM and $|S|$ is the total number of statements.

### Answer relevance

We say that the answer $a_{s}(q)$ is relevant if it directly addresses the
question in an appropriate way. In particular, our assessment of answer
relevance does not take into account factuality, but penalises cases where the
answer is incomplete or where it contains redundant information. To estimate
answer relevance, for the given answer $a_{s}(q)$, we prompt the LLM to
generate $n$ potential questions $q_{i}$ based on $a_{s}(q)$, as follows:

> Generate a question for the given answer.
> answer: [answer]

We then obtain embeddings for all questions using the text-embedding-ada-002
model, available from the OpenAI API. For each $q_{i}$, we calculate the
similarity $\text{sim}(q,q_{i})$ with the original question $q$, as the cosine
between the corresponding embeddings. The answer relevance score, AR, for
question $q$ is then computed as:

$\text{AR}=\frac{1}{n}\sum_{i=1}^{n}\text{sim}(q,q_{i})$ (1)

This metric evaluates how closely the generated answer aligns with the initial
question or instruction.

### Context relevance

The context $c(q)$ is considered relevant to the extent that it exclusively
contains information that is needed to answer the question. In particular,
this metric aims to penalise the inclusion of redundant information. To
estimate context relevance, given a question $q$ and its context $c(q)$, the
LLM extracts a subset of sentences, $S_{ext}$, from $c(q)$ that are crucial to
answer $q$, using the following prompt:

> Please extract relevant sentences from the provided context that can
> potentially help answer the following question. If no relevant sentences are
> found, or if you believe the question cannot be answered from the given
> context, return the phrase "Insufficient Information". While extracting
> candidate sentences you’re not allowed to make any changes to sentences from
> given context.

The context relevance score is then computed as:

$\text{CR}=\frac{\text{number of extracted sentences}}{\text{total number of
sentences in }c(q)}$ (2)

##  4 The WikiEval Dataset

To evaluate the proposed framework, we ideally need examples of question-
context-answer triples which are annotated with human judgments. We can then
verify to what extent our metrics agree with human assessments of
faithfulness, answer relevance and context relevance. Since we are not aware
of any publicly available datasets that could be used for this purpose, we
created a new dataset, which we refer to as
WikiEval444https://huggingface.co/datasets/explodinggradients/WikiEval. To
construct the dataset, we first selected 50 Wikipedia pages covering events
that have happened since the start of 2022555That is, beyond the reported
training cutoff of the model we used in our experiments.. In selecting these
pages, we prioritised those with recent edits. For each of the 50 pages, we
then asked ChatGPT to suggest a question that can be answered based on the
introductory section of the page, using the following prompt:

> Your task is to formulate a question from given context satisfying the rules
> given below:
> 1\. The question should be fully answered from the given context.
> 2\. The question should be framed from a part that contains non-trivial
> information.
> 3\. The answer should not contain any links.
> 4\. The question should be of moderate difficulty.
> 5\. The question must be reasonable and must be understood and responded to
> by humans.
> 6\. Do not use phrases that ’provided context’, etc in the question
> context:

We also used ChatGPT to answer the generated question, when given the
corresponding introductory section as context, using the following prompt:

> Answer the question using the information from the given context.
> question: [question]
> context: [context]

All questions were annotated along the three considered quality dimensions by
two annotators. Both annotators were fluent in English and were given clear
instructions about the meaning of the three considered quality dimensions. For
faithfulness and context relevance, the two annotators agreed in around 95% of
cases. For answer relevance, they agreed in around 90% of the cases.
Disagreements were resolved after a discussion between the annotators.

### Faithfulness

To obtain human judgements about faithfulness, we first used ChatGPT to answer
the question without access to any additional context. We then asked the
annotators to judge which of the two answers was the most faithful (i.e. the
standard one or the one generated without context), given the question and
corresponding Wikipedia page.

### Answer relevance

We first used ChatGPT to obtain candidate answers with lower answer relevance,
using the following prompt:

> Answer the given question in an incomplete manner.
> question: [question]

We then asked human annotators to compare this answer, and indicate which of
the two answers had the highest answer relevance.

### Context relevance

To measure this aspect, we first added additional sentences to the context by
scraping back-links to the corresponding Wikipedia page. In this way, we were
able to add information to the context that was related but less relevant for
answering the question. For the few pages without any back-links, we instead
used ChatGPT to complete the given context.

| Faith. | Ans. Rel. | Cont. Rel.
---|---|---|---
RAGAs | 0.95 | 0.78 | 0.70
GPT Score | 0.72 | 0.52 | 0.63
GPT Ranking | 0.54 | 0.40 | 0.52
Table 1: Agreement with human annotators in pairwise comparisons of
faithfulness, answer relevance and context relevance, using the WikEval
dataset (accuracy).

##  5 Experiments

Table 1 analyses the agreement between the metrics proposed in Section 3 and
the human assessments from the proposed WikiEval dataset. Each WikiEval
instance requires the model to compare two answers or two context fragments.
We count how often the answer/context preferred by the model (i.e. with
highest estimated faithfulness, answer relevance, or context relevance)
coincides with the answer/context preferred by the human annotators. We report
the results in terms of accuracy (i.e. the fraction of instances on which the
model agrees with the annotators).

To put the results in context, we compare our proposed metrics (shown as RAGAs
in Table 1) with two baseline methods. For the first method, shown as _GPT
Score_ , we ask ChatGPT to assign a score between 0 and 10 for the three
quality dimensions. To this end, we use a prompt that describes the meaning of
the quality metric and then asks to score the given answer/context in line
with that definition. For instance, for evaluating faithfulness, we used the
following prompt:

> Faithfulness measures the information consistency of the answer against the
> given context. Any claims that are made in the answer that cannot be deduced
> from context should be penalized.
> Given an answer and context, assign a score for faithfulness in the range
> 0-10.
> context: [context]
> answer: [answer]

Ties, where the same score is assigned by the LLM to both answer candidates,
were broken randomly. The second baseline, shown as _GPT Ranking_ , instead
asks ChatGPT to select the preferred answer/context. In this case, the prompt
again includes a definition of the considered quality metric. For instance,
for evaluating answer relevance, we used the following prompt:

> Answer Relevancy measures the degree to which a response directly addresses
> and is appropriate for a given question. It penalizes the present of
> redundant information or incomplete answers given a question. Given an
> question and answer, rank each answer based on Answer Relevancy.
> question: [question]
> answer 1: [answer 1]
> answer 2: [answer 2]

The results in Table 1 show that our proposed metrics are much closer aligned
with the human judgements than the predictions from the two baselines. For
faithfulness, the RAGAs prediction are in general highly accurate. For answer
relevance, the agreement is lower, but this is largely due to the fact that
the differences between the two candidate answers are often very subtle. We
found context relevance to be the hardest quality dimension to evaluate. In
particular, we observed that ChatGPT often struggles with the task of
selecting the sentences from the context that are crucial, especially for
longer contexts.

##  6 Conclusions

We have highlighted the need for automated reference-free evaluation of RAG
systems. In particular, we have argued the need for an evaluation framework
that can assess faithfulness (i.e. is the answer grounded in the retrieved
context), answer relevance (i.e. does the answer address the question) and
context relevance (i.e. is the retrieved context sufficiently focused). To
support the development of such a framework, we have introduced WikiEval, a
dataset which human judgements of these three different aspects. Finally, we
have also described RAGAs, our implementation of the three considered quality
aspects. This framework is easy to use and can provide deverlopers of RAG
systems with valuable insights, even in the absence of any ground truth. Our
evaluation on WikiEval has shown that the predictions from RAGAs are closely
aligned with human predictions, especially for faithfulness and answer
relevance.

## References

  * Azaria and Mitchell (2023) Amos Azaria and Tom M. Mitchell. 2023.  The internal state of an LLM knows when its lying.  _CoRR_ , abs/2304.13734.
  * Borgeaud et al. (2022) Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack W. Rae, Erich Elsen, and Laurent Sifre. 2022.  Improving language models by retrieving from trillions of tokens.  In _International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA_ , volume 162 of _Proceedings of Machine Learning Research_ , pages 2206–2240. PMLR.
  * Bubeck et al. (2023) Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023.  Sparks of artificial general intelligence: Early experiments with gpt-4.  _arXiv preprint arXiv:2303.12712_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Fu et al. (2023) Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, and Pengfei Liu. 2023.  Gptscore: Evaluate as you desire.  _CoRR_ , abs/2302.04166.
  * Guu et al. (2020) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020.  Retrieval augmented language model pre-training.  In _International conference on machine learning_ , pages 3929–3938. PMLR.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12):1–38.
  * Kadavath et al. (2022) Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. 2022.  Language models (mostly) know what they know.  _CoRR_ , abs/2207.05221.
  * Kandpal et al. (2022) Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2022.  Large language models struggle to learn long-tail knowledge.  _CoRR_ , abs/2211.08411.
  * Khandelwal et al. (2020) Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020.  Generalization through memorization: Nearest neighbor language models.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Khattab et al. (2022) Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. 2022.  Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive NLP.  _CoRR_ , abs/2212.14024.
  * Lee et al. (2019) Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019.  Latent retrieval for weakly supervised open domain question answering.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6086–6096.
  * Lewis et al. (2020) Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Li et al. (2023) Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023.  Halueval: A large-scale hallucination evaluation benchmark for large language models.  _CoRR_ , abs/2305.11747.
  * Liu et al. (2023) Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023.  Lost in the middle: How language models use long contexts.
  * Mallen et al. (2023) Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023.  When not to trust language models: Investigating effectiveness of parametric and non-parametric memories.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 9802–9822, Toronto, Canada. Association for Computational Linguistics.
  * Manakul et al. (2023) Potsawee Manakul, Adian Liusie, and Mark J. F. Gales. 2023.  Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models.  _CoRR_ , abs/2303.08896.
  * Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023.  Factscore: Fine-grained atomic evaluation of factual precision in long form text generation.  _CoRR_ , abs/2305.14251.
  * Ram et al. (2023) Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023.  In-context retrieval-augmented language models.  _CoRR_ , abs/2302.00083.
  * Roberts et al. (2020) Adam Roberts, Colin Raffel, and Noam Shazeer. 2020.  How much knowledge can you pack into the parameters of a language model? In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 5418–5426, Online. Association for Computational Linguistics.
  * Shi et al. (2023) Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023.  REPLUG: retrieval-augmented black-box language models.  _CoRR_ , abs/2301.12652.
  * Wang et al. (2023a) Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023a.  Is chatgpt a good NLG evaluator? A preliminary study.  _CoRR_ , abs/2303.04048.
  * Wang et al. (2023b) Peiyi Wang, Lei Li, Liang Chen, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023b.  Large language models are not fair evaluators.  _CoRR_ , abs/2305.17926.
  * Wang et al. (2023c) Shufan Wang, Yixiao Song, Andrew Drozdov, Aparna Garimella, Varun Manjunatha, and Mohit Iyyer. 2023c.  KNN-LM does not improve open-ended text generation.  _CoRR_ , abs/2305.14625.
  * Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021.  Bartscore: Evaluating generated text as text generation.  In _Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual_ , pages 27263–27277.
  * Zhang et al. (2023) Tianhua Zhang, Hongyin Luo, Yung-Sung Chuang, Wei Fang, Luc Gaitskell, Thomas Hartvigsen, Xixin Wu, Danny Fox, Helen Meng, and James R. Glass. 2023.  Interpretable unified language checking.  _CoRR_ , abs/2304.03728.
  * Zhang et al. (2020) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020.  Bertscore: Evaluating text generation with BERT.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Zhao et al. (2019) Wei Zhao, Maxime Peyrard, Fei Liu, Yang Gao, Christian M. Meyer, and Steffen Eger. 2019.  MoverScore: Text generation evaluating with contextualized embeddings and earth mover distance.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 563–578, Hong Kong, China. Association for Computational Linguistics.

##  Appendix A Examples from WikiEval

Tables 2, 3 and 4 show examples from the WikiEval dataset, focusing in
particular on answers with high and low faithfulness (Table 2), high and low
answer relevance (Table 3), and high and low context relevance (Table 4).

Question |  Context |  Answer
---|---|---
Who directed the film Oppenheimer and who stars as J. Robert Oppenheimer in the film? |  Oppenheimer is a 2023 biographical thriller film written and directed by Christopher Nolan. Based on the 2005 biography American Prometheus by Kai Bird and Martin J. Sherwin, the film chronicles the life of J. Robert Oppenheimer, a theoretical physicist who was pivotal in developing the first nuclear weapons as part of the Manhattan Project, and thereby ushering in the Atomic Age. Cillian Murphy stars as Oppenheimer, with Emily Blunt as Oppenheimer’s wife Katherine "Kitty" Oppenheimer. |  High Faithfulness: Christopher Nolan directed the film Oppenheimer. Cillian Murphy stars as J. Robert Oppenheimer in the film. Low Faithfulness: James Cameron directed the film Oppenheimer. Tom Cruise stars as J. Robert Oppenheimer in the film.
Table 2: Example from WikiEval, showing answers with high and low faithfulness.  Question |  Answer
---|---
When is the scheduled launch date and time for the PSLV-C56 mission, and where will it be launched from? |  High answer relevance: The PSLV-C56 mission is scheduled to be launched on Sunday, 30 July 2023 at 06:30 IST / 01:00 UTC. It will be launched from the Satish Dhawan Space Centre, Sriharikota, Andhra Pradesh, India. Low answer relevance: The scheduled launch date and time for the PSLV-C56 mission have not been provided.The PSLV-C56 mission is an important space mission for India. It aims to launch a satellite into orbit to study weather patterns.
Table 3: Example from WikiEval, showing answers with high and low answer relevance.  Question |  Context
---|---
When was the Chimnabai Clock Tower completed, and who was it named after? |  High context relevance: The Chimnabai Clock Tower, also known as the Raopura Tower, is a clock tower situated in the Raopura area of Vadodara, Gujarat, India. It was completed in 1896 and named in memory of Chimnabai I (1864–1885), a queen and the first wife of Sayajirao Gaekwad III of Baroda State. Low context relevance: The Chimnabai Clock Tower, also known as the Raopura Tower, is a clock tower situated in the Raopura area of Vadodara, Gujarat, India. It was completed in 1896 and named in memory of Chimnabai I (1864–1885), a queen and the first wife of Sayajirao Gaekwad III of Baroda State. It was built in Indo-Saracenic architecture style. History. Chimnabai Clock Tower was built in 1896. The tower was named after Chimnabai I (1864–1885), a queen and the first wife of Sayajirao Gaekwad III of Baroda State. It was inaugurated by Mir Kamaluddin Hussainkhan, the last Nawab of Baroda. During the rule of Gaekwad, it was a stoppage for horse drawn trams. The clock tower was erected at the cost of 25,000 (equivalent to 9.2 million or USD 120,000 in 2023).
Table 4: Example from WikiEval, showing answers with high and low context
relevance.