# Efficient LLM Comparative Assessment: a Product of Experts Framework for
Pairwise Comparisons

Adian Liusie
ALTA Institute, Engineering Department
University of Cambridge
<EMAIL_ADDRESS>
&Vatsal Raina
ALTA Institute, Engineering Department
University of Cambridge
<EMAIL_ADDRESS>
Yassir Fathullah
ALTA Institute, Engineering Department
University of Cambridge
<EMAIL_ADDRESS>
&Mark J. F. Gales
ALTA Institute, Engineering Department
University of Cambridge
<EMAIL_ADDRESS>

###### Abstract

LLM-as-a-judge approaches are a practical and effective way of assessing a
range of text tasks, aligning with human judgements especially when applied in
a comparative assessment fashion. However, when using pairwise comparisons to
rank a set of candidates the computational costs scale quadratically with the
number of candidates, which can have practical limitations. This paper
introduces a Product of Expert (PoE) framework for efficient LLM Comparative
Assessment. Here individual comparisons are considered experts that provide
information on a pair’s score difference. The PoE framework combines the
information from these experts to yield an expression that can be maximized
with respect to the underlying set of candidates, and is highly flexible where
any form of expert can be assumed. When Gaussian experts are used one can
derive simple closed-form solutions for the optimal candidate ranking, as well
as expressions for selecting which comparisons should be made to maximize the
probability of this ranking. Our approach enables efficient comparative
assessment, where by using only a small subset of the possible comparisons,
one can generate score predictions that correlate as well to human judgements
as the predictions when all comparisons are used. We evaluate the approach on
multiple NLG tasks and demonstrate that our framework can yield considerable
computational savings when performing pairwise comparative assessment. When
$N$ is large, with as few as 2% of comparisons the PoE solution can achieve
similar performance to when all comparisons are used.

##  1 Introduction

The advent of instruction-following [50, 42] Large Language Models (LLMs) [6,
46] has enabled systems to exhibit impressive zero-shot capabilities on a
range of Natural Language Processing (NLP) tasks. One such practical
application is in Natural Language Generation (NLG) evaluation [17], where
LLMs can be prompted to assess the quality of texts for particular attributes
[49, 29, 59]. A popular and effective methodology is LLM comparative
assessment, where pairwise comparisons are used to determine which of two
texts is better [59, 45, 32]. Although using pairwise comparisons has shown to
better align with human preferences [31] than LLM scoring approaches [49, 29],
the set of all comparisons scales quadratically with the number of input
texts, which can be impractical for many real-world use cases. Therefore, one
may instead consider methods that only use a subset of comparisons to
determine the scores/ranks, which with the benefits of computational
efficiency, should not incur any significant performance degradation.

Due to its applicability to sports, gaming, search and many other domains, the
task of going from a partial set of comparisons between $N$ candidates to a
final ranking/scoring has been well-studied and extensively explored [14, 13,
34, 8]. However, in the majority of set-ups, the comparative decisions are
binary (win/loss, although occasionally also win/loss/tie). LLMs, however, not
only provide the outcome of the comparison but also additional information,
such as the associated probability that A is better than B. Despite this
available information, current LLM comparative works often leverage naive
metrics such as win-ratio [45, 32] and average probability [44, 39], with
little theoretical justification and analysis on how to maximally extract the
information from the comparisons.

This paper introduces a theoretical framework for viewing comparative
assessment that enables practical scoring in cases when the full set of
comparisons is not used. We conceptualize the process as a Product of Experts
(PoE) [21, 51], where each comparative decision is assumed to provide
information on the quality difference between the two competing texts. By
considering two forms of experts, namely 1) the Gaussian distribution with
linear assumptions and 2) an extension of the Bradley-Terry model that deals
with soft probabilities, we demonstrate that the PoE framework for comparative
assessment can achieve efficient and effective NLG assessment. With the
Gaussian expert, the framework conveniently yields a closed-form solution for
the scores, where using the full set of comparisons conveniently yields
standard metrics. We demonstrate that our Product of Expert framework leads to
considerable performance boosts across models, datasets, and assessment
attributes and, when using a fraction of the possible comparisons, can achieve
performance with minimal performance degradation.

This paper makes several contributions. 1) We introduce a theoretical
framework for probabilistic pairwise comparative assessment, the PoE
perspective. To the best of our knowledge, we are the first to extend existing
approaches that only deal with discrete outcomes to those where confidence is
also available (such as with LLMs). 2) We provide theoretical analysis of the
PoE framework, exploitig the closed-form solutions of Gaussian experts for
comparative assessment, as well as approximations for selecting the most
informative comparisons. 3) We demonstrate practically that the PoE solution
yields significant computational savings and empirically show that convergence
is reached much faster than when using other baseline approaches.

##  2 Background and Related Work

Traditional/Tailored NLG Evaluation: Initially, the outputs of NLG systems
were evaluated by comparing them to ground-truth human-annotated references
using N-gram overlap metrics such as BLEU [43], ROUGE [27] and METEOR [2], or
similarity-based metrics such as BERTScore [58]. To conduct more fine-grained
NLG evaluation, later studies developed bespoke evaluators for particular task
dimensions such as consistency in summarization [48, 35, 25] or coherence in
dialogue generation [16, 54]. Further extensions considered unified
evaluators, which evaluate the response quality on multiple different
independent attributes [37, 55, 60], often through multiple independent
systems. A drawback with these traditional NLG evaluation approaches is that
they are typically bespoke towards particular tasks and attributes and
therefore not easily extendible to new domains.

LLM-Based NLG Evaluation: Given the current impressive instruction-following
[42, 11] and emergent capabilities of LLMs such as GPT-4 [1] and open-sourced
variants [11, 46], recent works have studied methods of leveraging these LLMs
for general zero-shot NLG evaluation. Methods include GPTScore [18], which
computes the LLM likelihood of generating the response, and LLM-as-a-judge
approaches [59], which either prompt models to provide independent scores on
the quality of responses [49, 24, 29] or use pairwise comparisons to determine
which of two responses are better [45, 32]. These methods have shown to not
only work effectively for a range of tasks and datasets, but also yield
performance that is competitive, if not better, than traditional bespoke
solutions [32].

LLM Comparative Assessment: Various recent papers have used pairwise LLM
comparative assessment for ranking texts: [32] demonstrated that for moderate-
sized LLMs, LLM comparative assessment outperforms LLM scoring and various
bespoke baselines. They compute the win-ratio using all $N(N\\!-\\!1)$
comparisons and further consider using a subset of comparisons, albeit with
significant performance degradation. Further, [45] leveraged pairwise
comparisons for retrieving relevant sources for a query with methods using all
comparisons, as well as sorting-based algorithms (though assuming
transitivity). [44] applied comparative assessment for dialogue evaluation,
computing the average probability over a randomly sampled set of comparisons
as the score quality. They also adapted the model through supervised training
on labelled pairwise preferences. [30] demonstrated limitations of calibration
in aligning LLM scoring approaches with human judgements, though considered
pairwise comparisons as a promising solution. They introduced PAirwise-
preference Search (PAIRS), which is a variant of the merge sort algorithm,
although using LLM comparisons.

Comparisons to Scores: Although LLMs have only recently been used as pairwise
evaluators, the problem of ranking a set of candidates from a set of pairwise
comparisons arises in many different contexts, such as sports and games [3,
12], information retrieval [7, 28] and social studies [36, 33]. Arguably, the
most widely used parametric model is the Bradley-Terry model [5], which uses
the outcomes of events and defines the win probabilities in terms of the
difference between the latent scores of the compared items. The latent scores
are deduced by maximizing the likelihood of the observed pairwise comparison
data, with various works discussing algorithms that converge to the solution
[14, 13, 8]. Additionally, [10] investigated predicting the full ranking under
the Bradley-Terry-Luce model [34], while TrueSkill [20, 38], extended the
Bradley-Terry model to incorporate uncertainties in player skills (in a sports
context) under a Bayesian framework.

Product of Experts: Products of Experts (PoE) [21, 51] combines a number of
individual models (the experts) by taking their product and normalizing the
result. They are typically used to model complex high-dimensional
distributions, done by breaking them down as a product of simpler
distributions that each model a different subset of dimensions [56, 53, 47,
26, 22]. In [56], several different probabilistic acoustic models are
multiplied with each other and trained jointly under a PoE framework to
generate a stronger overall system. Meanwhile, the works of [53, 47, 26, 22]
approximate a high-dimensional joint distribution into a product of univariate
or lower-dimensional ‘experts’. Furthermore, while any choice can be made for
the individual expert, the works of [52, 19] show that using the Gaussian
expert leads to closed-form expressions for the joint distribution,
simplifying the modeling process.

##  3 A Product of Experts Perspective of Comparative Assessment

###  3.1 Task Definition

Let $x_{1:N}\in\mathcal{X}$ be a set of $N$ candidate texts and
$s_{1:N}\in\mathbb{R}$ the scores of the texts for a particular assessed
attribute. Given a set of $K$ pairwise comparisons, $\mathcal{C}_{1:K}$, the
objective is to determine a predicted set of scores, $\hat{s}_{1:N}$, that are
close to the true scores, $s^{*}_{1:N}$.

###  3.2 The Bradley–Terry Model

For traditional comparative assessment set-ups, outcomes are usually discrete
and either binary (win/loss) or ternary (win/draw/loss). One standard approach
of going from a set of discrete comparisons $\mathcal{C}_{1:K}$ to predicted
scores $\hat{s}_{1:N}$ is the Bradley–Terry model [5, 57]. Assuming each
comparison $C_{k}$ is of the form $(i,j,y_{ij})$, where $y_{ij}\in\\{0,1\\}$
represents the binary outcome of whether $x_{i}$ is better than $x_{j}$, one
can adopt a probabilistic binomial model where the probability of victory
depends solely on the difference of scores,
$P(y_{ij}|s_{i}\\!-\\!s_{j})=\sigma(s_{i}\\!-\\!s_{j})$. The most popular form
is the sigmoid function, $\sigma(x)=1/(1+e^{-x})$. The Bradley-Terry model
treats the scores as parameters of the model, and aims to maximize the
likelihood of the observations,

$\hat{s}_{1:N}=\operatorname*{arg\,max}_{s_{1:N}}{\tt
P}(\mathcal{C}_{1:K}|s_{1:N})=\operatorname*{arg\,max}_{s_{1:N}}\\!\\!\prod_{i,j\in\mathcal{C}_{1:K}}\\!\\!\\!\sigma(s_{i}-s_{j})^{y_{ij}}(1-\sigma(s_{i}-s_{j}))^{1-y_{ij}}$
(1)

Although no closed-form solution exists, Zermello’s algorithm [57] can be used
to iterate the solution until convergence is reached. Furthermore, while
Zermello’s algorithm is known to be slow to converge [15, 23], later
improvements have demonstrated faster convergence rates [41].

###  3.3 A Product of Expert Perspective

A Product of Experts (PoE) [21, 51] combines the information gained from many
individual models (the experts) by taking their product and normalizing the
result. In this work, we explore reformulating the scores as a PoE. One may
consider the information gained from each comparison as independent experts,
which enables the probability of a set of scores to be written in the form of
a PoE:

${\tt
p}(s_{1:N}|\mathcal{C}_{1:K})=\frac{1}{Z}\prod_{i,j\in\mathcal{C}_{1:K}}{\tt
p}(s_{i}\\!-\\!s_{j}|C_{k})$ (2)

Each expert therefore models the distribution of the score difference given
the observations and provides flexibility in the choice of the experts. For
LLM comparative assessment, as opposed to traditional binary comparative
decisions, one has access to richer information including the associated
probability of the decision. Each comparison can therefore be extended to be
of the form $(i,j,p_{ij})$, where $p_{ij}={\tt P}_{\tt
lm}(y_{i}>y_{j}|x_{i},x_{j})$ is the associated LLM probability of the
comparative decision. Each expert can be conditioned on the observed LLM
probability such that ${\tt p}(s_{i}\\!-\\!s_{j}|C_{k})={\tt
p}(s_{i}\\!-\\!s_{j}|p_{ij})$. Inspired by the form of the Bradley-Terry
Model, we propose to extend Equation 1 to a soft form and consider this as the
conditional distribution of the scores,

$\displaystyle{\tt
p}(s_{i}-s_{j}|p_{ij})=\frac{1}{Z_{ij}}\sigma(s_{i}-s_{j})^{p_{ij}}(1-\sigma(s_{i}-s_{j}))^{1-p_{ij}}$
(3)

Defined within the range $0<p_{ij}<1$, where
$Z_{ij}=\pi\\!/\\!\sin(p_{ij}\pi)$ is a normalization constant to ensure a
valid probability density function. However, the form is not restricted to
sigmoid-based modeling. One can alternatively select any other family of
probability distributions and run gradient-based optimization approaches to
find the maximum-likelihood solution. One option for the probability densities
of the experts is the Gaussian distribution, which has favorable properties
such as a closed-form solution.

###  3.4 Properties of Gaussian Experts

Having Gaussian experts yields convenient properties in the PoE framework,
such as a closed-form expression for the solution [56]. If the underlying
distribution is assumed to be Gaussian with the mean $f_{\mu}(p_{ij})$ and
variance $f_{\sigma}(p_{ij})$ only dependent on the probability such that
${\tt
p}(s_{i}\\!-\\!s_{j}|p_{ij})=\mathcal{N}{\big{(}}s_{i}\\!-\\!s_{j};f_{\mu}(p_{ij}),f_{\sigma}(p_{ij}){\big{)}}$,
then by representing the scores in vector form, ${\bf s}\\!=\\![s_{1:N}]$, one
can express the distribution as,

${\tt p}({\bf W}{\bf s}|\mathcal{C}_{1:K})=\mathcal{N}\left({\bf W}{\bf
s};\bm{\mu},\text{diag}(\bm{\sigma^{2}})\right)$ (4)

Where ${\bf W}\\!\in\\!R^{K\times N}$ is a matrix representing the set of
comparisons, such that for the $k^{\text{th}}$ comparison between $i$ and $j$
${\bf W}_{ki}\\!=\\!1$, ${\bf W}_{kj}\\!=\\!-1$, and ${\bf
W}_{km}\\!=\\!0\;\;\forall m\neq i,j$, ${\bf s}$ is the N-dimensional column
vector of $s_{1:N}$, $\bm{\mu}\\!\in\\!R^{K}$ is a vector of the means and
$\bm{\sigma^{2}}\\!\in\\!R^{K}$ equivalently represents the variances,

$\bm{\mu}=[f_{\mu}(p_{ij}^{(1)}),f_{\mu}(p_{ij}^{(2)}),...f_{\mu}(p_{ij}^{(K)})]^{\scriptstyle\sf
T}\qquad{\bm{\sigma}^{2}}=[f_{\sigma}(p_{ij}^{(1)}),f_{\sigma}(p_{ij}^{(2)}),...f_{\sigma}(p_{ij}^{(K)})]^{\scriptstyle\sf
T}$ (5)

The matrix $\bf{W}$ is not full rank since any shift of the scores $\bm{s}$
will yield an equivalent output. To address this, an additional expert on the
first element can be added such that,

${\tt p}(s_{1}|\mathcal{C}_{0})=\mathcal{N}(0,\sigma_{0}^{2})$ (6)

Prepending an extra row to all of ${\bf W}$, $\bm{\mu}$ and $\bm{\sigma^{2}}$,
yielding ${\bf\tilde{W}}$, $\bm{\tilde{\mu}}$ and $\bm{\tilde{\sigma}^{2}}$
respectively (as shown in Appendix A.1).The distribution takes a similar form,
${\tt p}({\bf\tilde{W}}{\bf
s}|\mathcal{C}_{1:K})=\mathcal{N}({\bf\tilde{W}}{\bf
s};\bm{\tilde{\mu}},\text{diag}(\bm{\tilde{\sigma}^{2}}))$, which can be
rearranged (shown in Appendix A.4) to provide the probability for a given set
of scores,

${\tt p}({s}_{1:N}|C_{1:K})=\mathcal{N}\left({\bf
s};({\bf\tilde{W}}^{\scriptstyle\sf
T}{\bf\tilde{\Sigma}}^{-1}{\bf\tilde{W}})^{-1}{\bf\tilde{W}}{\bf\tilde{\Sigma}}^{-1}\bm{\tilde{\mu}}\;,\;({\bf\tilde{W}}^{\scriptstyle\sf
T}{\bf\tilde{\Sigma}}^{-1}{\bf\tilde{W}})^{-1}\right)$ (7)

where ${\bf\tilde{\Sigma}}=\text{diag}(\bm{\tilde{\sigma}^{2}})$, yielding the
expression of the maximum probability solution, $\hat{s}_{1:N}$,

${\bf\hat{s}}=\operatorname*{arg\,max}_{{s}_{1:N}}{\tt
p}({s}_{1:N}|\mathcal{C}_{1:K})=({\bf\tilde{W}}^{\scriptstyle\sf
T}{\bf\tilde{\Sigma}}^{-1}{\bf\tilde{W}})^{-1}{\bf\tilde{W}}{\bf\tilde{\Sigma}}^{-1}\bm{\tilde{\mu}}$
(8)

However, a drawback with the Gaussian Expert is that knowing
$\bm{\tilde{\mu}}$ and $\bm{\tilde{\sigma}^{2}}$ requires knowledge of both
$f_{\mu}(p)$ and $f_{\sigma}(p)$ for all $0\\!<p\\!<1$. This is not available
without a large quantity of human-annotated NLG assessment data for the task
and model at hand, making the approach impractical for zero-shot applications.

###  3.5 Further Gaussian Assumptions

To enable a practical solution applicable in zero-shot settings, one can make
two assumptions on the Gaussian experts, assumption 1) that the variance is
constant regardless of the predicted probability $f_{\sigma}(p)=\sigma^{2}$,
and assumption 2) that the mean scales linearly with the probability
$f_{\mu}(p)=\alpha\\!\cdot\\!(p-\beta)$. This simplifies the solution,

${\bf\hat{s}}=\alpha\cdot({\bf\tilde{W}}^{{\scriptstyle\sf
T}}{\bf\tilde{W}})^{-1}{\bf\tilde{W}}\bm{\tilde{\mu}}$ (9)

where $\bm{\tilde{\mu}}^{\scriptstyle\sf
T}=[0,p_{ij}^{(1)}\\!\\!-\\!\beta,...,p_{ij}^{(K)}\\!\\!-\\!\beta]$. Note that
one can set $\beta=0.5$, since when inputting texts of equal quality into an
unbiased system, one would expect an average output probability of 0.5.
Further, the value of $\alpha$ is insignificant as it only determines the
relative spacing and only influences the subjective scale used to score the
texts.

###  3.6 Optimal Set of Comparisons

The previous theory detailed how to determine the predicted scores
$\hat{s}_{1:N}$ given a set of observed comparisons $\mathcal{C}_{1:K}$. As an
extension, one may also consider how one can select the optimal set of
comparisons to gain as much information from a fixed number of comparisons.
Under the Gaussian model, the probability of the most likely set of scores is
given as,

${\tt p}({\hat{s}}_{1:N}|\mathcal{C}_{1:K})=\frac{\sqrt{{\tt
det}({\bf\tilde{W}}^{{\scriptstyle\sf
T}}{\bf\tilde{W}})}}{(2\pi\sigma^{2})^{N/2}}$ (10)

Therefore for a fixed number of comparisons $K$, one may aim to find the
comparison matrix ${\bf\tilde{W}}^{*}$ that minimizes the uncertainty,

${\bf\tilde{W}}^{*}=\operatorname*{arg\,max}_{{\bf\tilde{W}}}{\tt
p}({\hat{s}}_{1:N}|\mathcal{C}_{1:K})\equiv\operatorname*{arg\,max}_{{\bf\tilde{W}}}{\tt
det}({\bf\tilde{W}}^{{\scriptstyle\sf T}}{\bf\tilde{W}})$ (11)

This can be approximated through an iterative greedy search. Assume that
${\bf\tilde{W}}^{(k)*}$ is the optimal comparison matrix using $k$
comparisons. Considering an additional comparison $(i,j)$ is equivalent to
adding an extra row ${\bf r}\in R^{N}$ where ${\bf r}_{i}\\!=\\!1$, ${\bf
r}_{j}\\!=\\!-1$ and ${\bf r}_{l}\\!=\\!0\;\;\forall l\neq i,j$. By noting
that,

${\tt det}\left([{\bf\tilde{W}};{\bf r}]^{\scriptstyle\sf
T}[{\bf\tilde{W}};{\bf r}]\right)={\tt det}({\bf\tilde{W}}^{{\scriptstyle\sf
T}}{\bf\tilde{W}}+{\bf r}^{\scriptstyle\sf T}{\bf r})={\tt
det}({\bf\tilde{W}}^{{\scriptstyle\sf T}}{\bf\tilde{W}})(1+{\bf
r}^{\scriptstyle\sf T}({\bf\tilde{W}}^{{\scriptstyle\sf
T}}{\bf\tilde{W}})^{-1}{\bf r})$ (12)

Given ${\bf\tilde{W}}^{(k)*}$ and ${\bf
A}^{(k)*}\\!=\\!({\bf\tilde{W}}^{(k)*^{\scriptstyle\sf
T}}{\bf\tilde{W}}^{(k)*})^{-1}$, the next optimal comparison
$(\hat{i},\hat{j})$ can be calculated,

$\hat{i},\hat{j}=\operatorname*{arg\,max}_{i,j}{\bf A}^{(k)*}_{ii}+{\bf
A}^{(k)*}_{jj}-2\cdot{\bf A}^{(k)*}_{ij}$ (13)

Note the inverse matrix ${\bf A}^{(k+1)*}$ can be updated from ${\bf
A}^{(k)*}$ in $O(N)$ as shown in Appendix A.5.

##  4 Experimental Setup

###  4.1 Datasets

To determine whether the product of expert approach works for real-world
tasks, we consider a range of NLG evaluation datasets which have available
ground-truth scores. For summary evaluation we use SummEval [17], a dataset of
100 articles each with 16 machine-generated summaries evaluated on coherency
(COH), consistency (CON), fluency (FLU), and relevancy (REL). For dialogue
response generation we use TopicalChat [37], a dataset of 60 dialogue contexts
with six responses per context assessed on coherency (COH), continuity (CNT),
engagingness (ENG), and naturalness (NAT). We further consider question
difficulty ranking and use CMCQRD [40], a dataset with 658 multiple choice
reading comprehension questions, each annotated with perceived question
difficulty. For CMCQRD, we compare question difficulty across all 658
questions.

###  4.2 Methodology

Base Large Language Models Two different model families are used as judge LLMs
for Summeval and TopicalChat: FlanT5 (3B, 11B) [11] and Llama2-chat (7B, 13B)
[46]. For CMCQRD, ChatGPT is used as the base judge-LLM. Although ChatGPT is
accessible only by API access, it returns the logits for the top 20 tokens and
therefore can be used to get implicit confidence in its decisions.

LLM Pairwise Probability Calculation To get the model probability associated
with a pairwise comparison, we follow [32] and take the ratio of the label
word LLM probability, e.g. ${\tt P(A)}/({\tt P(A)}+{\tt P(B)})$. Since LLMs
have inconsistencies and are prone to exhibiting positional bias [59, 9, 31],
for each comparison, we consider both orderings and then take the average
probability. This means that each comparison takes two LLM calls to ensure
that $p_{ij}\\!=\\!1\\!-\\!p_{ji}$, eliminating positional bias.

Comparison Selection When considering comparative assessment with a subset of
comparisons, the default setting takes a randomly drawn set of comparisons
where each comparison is equally likely to be drawn. Let $K$ denote the total
number of comparisons made for a particular set of candidates. For a set of
inputs $x_{1:N}$ and randomly select $K$ unique pairs $(x_{i},x_{j})$ to be
judged by the LLM, ensuring that each text $x_{i}$ is involved in at least one
comparison. Experiments begin with $K\\!=\\!N$ comparisons and is incremented
to the full set of comparisons, $K\\!=\\!N\\!\cdot\\!(N\\!-\\!1)/2$.

Scoring Methods Several different methods of mapping a set of pairwise
comparisons to scores are considered in this paper, which can be categorized
into binary decision-based or probability-based. For the methods where only
binary outcomes of the comparisons are used, our first baseline is the win-
ratio which calculates the number of comparisons won as the quality score, as
used in [45, 32]. The second baseline is the Bradley-Terry model, BT, [5],
where the solution is found by Zermelo [57] with a convergence threshold of
$e^{-4}$. Since any candidate that wins/loses all games will have an infinite
score, a prior of $1/(N\\!-\\!1)$ wins is added to each selected comparison.
For the methods that leverage the LLM probabilities, the baseline is the
average probability avg-prob of a text in all its comparisons, as used in [44,
39]. To better leverage the probabilistic information, our paper proposes to
decompose the probability into a product of experts. We propose two variants;
1) PoE-BT which uses a variant of the Bradley-Terry model extended to soft
probabilities (Equation 3), optimized using gradient descent and 2) PoE-g
which uses the Gaussian expert (Equation 70) with the linear mean and constant
variance assumptions described in Section 3. Lastly, the final method is PoE-
g-hard, which applies the POE-gaussian framework, however, using hard binary
decisions and not the soft probabilities.

Evaluation For SummEval and TopicalChat, the summary-level Spearman score is
used as the assessment metric. For each context, we do pairwise comparisons
using the LLM on the full set of $N(N\\!-\\!1)/2$ comparisons. We then
simulate using a subset of comparisons by randomly selecting $K$ of these
outcomes. This process is repeated 100 times for a particular number of total
comparisons, $K$, and we calculate both the mean and standard deviation of
performance over the entire dataset. Equivalent results for Pearson are
provided in Appendix C.

##  5 Results

###  5.1 Data Analysis

(a) FlanT5-3B, SummEval COH

(b) Llama2-13B, TopicalChat CNT

(c) ChatGPT, CMCQRD DIFF

Figure 1: Joint distribution of the LLM probabilities and true scores.

In the POE framework, each expert models the distribution ${\tt
p}(s_{i}\\!-\\!s_{j}|p_{ij})$. To determine a suitable form of the expert, and
whether the Gaussian and/or the extended Bradley-Terry experts are sensible
assumptions, Figure 1 displays the joint bivariate distribution between the
true score difference $s_{i}\\!-\\!s_{j}$ and the observed probability
$p_{ij}$. For a particular LLM, all comparisons over all the contexts of the
dataset are assessed. The frequency count of the LLM probability and true
score difference (calculated using the gold-standard annotator labels) is then
plotted. The plots illustrate a clear correlation between the probabilities
and score difference, implying that considerable scoring information can be
gained from leveraging probabilities and decisions. However, the mapping is
not deterministic, and there is considerable noise present. Empirically, The
distributions appear to be well approximated by Gaussian distributions,
implying that the conditional distributions will also be well-modeled by
Gaussian distributions (with further plots shown in the Appendix).

(a) FlanT5-3B, SummEval COH

(b) Llama2-13B, TopicalChat CNT

(c) ChatGPT, CMCQRD DIFF

Figure 2: Expected score difference and variance given the LLM probability.

We further analyze the relationship between the LLM probability $p$ and the
expected score difference,
$\delta(p)=E_{p_{ij}}[s_{i}\\!-\\!s_{j}\;|\;|p_{ij}\\!-\\!p|\\!<\\!\epsilon]$.
Figure 2 demonstrates that 1) the probability is quite linearly correlated
with the expected score difference; and 2) the variance across all score
distributions given the probability is quite constant. Therefore the Gaussian
assumptions discussed in Section 3.5 appear to be reasonable. Note that
TopicalChat is a smaller dataset (with 1800 total comparisons) and hence has
more observed noise.

###  5.2 Efficient Comparative Assessment

system | $K$ | win-ratio | BT | PoE-g-hard | avg-prob | PoE-BT | PoE-g
---|---|---|---|---|---|---|---
Flant5-3B | 24 |  34.1$\pm$0.8 |  36.6$\pm$0.6 |  34.9$\pm$0.7 |  38.4$\pm$0.6 |  42.7$\pm$0.4 |  42.4$\pm$0.4
120 |  43.6$\pm$0.0 |  43.6$\pm$0.0 |  43.4$\pm$0.0 |  44.3$\pm$0.0 |  44.2$\pm$0.0 |  44.3$\pm$0.0
Flant5-11B | 24 |  31.2$\pm$0.8 |  33.4$\pm$0.7 |  32.0$\pm$0.7 |  34.7$\pm$0.7 |  38.4$\pm$0.4 |  38.4$\pm$0.4
120 |  40.0$\pm$0.0 |  40.0$\pm$0.0 |  39.7$\pm$0.0 |  40.5$\pm$0.0 |  40.5$\pm$0.0 |  40.5$\pm$0.0
Llama2-7B | 24 |  21.6$\pm$0.8 |  23.4$\pm$0.7 |  22.5$\pm$0.7 |  24.0$\pm$0.7 |  26.8$\pm$0.5 |  26.6$\pm$0.5
120 |  27.8$\pm$0.0 |  27.9$\pm$0.0 |  27.6$\pm$0.0 |  28.4$\pm$0.0 |  28.4$\pm$0.0 |  28.4$\pm$0.0
Llama2-13B | 24 |  30.8$\pm$0.7 |  33.1$\pm$0.7 |  31.6$\pm$0.7 |  33.7$\pm$0.6 |  37.7$\pm$0.4 |  37.3$\pm$0.4
120 |  39.3$\pm$0.0 |  39.3$\pm$0.0 |  39.2$\pm$0.0 |  39.3$\pm$0.0 |  39.3$\pm$0.0 |  39.3$\pm$0.0
Table 1: Spearman correlations for SummEval, averaged over all attributes
(COH, CON, FLU, REL). $K$ is the number of balanced comparisons made, where
$K\\!=\\!120$ is the full set of comparisons.

In this Section, we investigate whether using our Product of Experts framework
enables the effective use of the soft-LLM probability information for better
performance in efficient settings when using fewer LLM calls. Table 1 presents
the Spearman rank correlation coefficient averaged over all contexts of
SummEval and scoring attributes, when considering various base LLMs. The
following observations can be made:

Average probability is superior to win-ratio in efficient settings When
considering the full set of comparisons ($K=120$) the performance of average
probability is observed to only be marginally better than using win-ratio
(within 1 SCC). However, when using 20% of the comparisons ($K=24$) the
average probability yields significant gains of 3-4 SCC. This highlights that
especially in scenarios when only a subset of comparisons are being made,
leveraging the soft probabilistic information can be very useful to minimize
the performance degradation.

The PoE solution yields larger gains in efficient settings Even when
considering the hard-decisions, for $K\\!=\\!24$ both the Bradley-Terry model
(BT) and the PoE Gaussian with hard decisions (PoE-g-hard) have mild
performance gains over the win-ratio. However, the real benefits come when
considering the PoE perspective with the soft-probabilities, with both POE-BT
and PoE-g significantly outperforming the average probability. With these
methods, when using only 20% of the comparisons, one can achieve performance
very close to when using the full comparison set (within 1-2 SCC) which would
otherwise have had large degradations when using the naive win-ratio (up to 10
SCC). The findings are general and hold across the different SummEval
attributes and models (equivalent tables for TopicalChat are provided in
Appendix C.2 where similar trends are seen).

(a) FlanT5-3B, SummEval COH

(b) Llama2-13B, TopicalChat CNT

(c) ChatGPT, CMCQRD DIFF

Figure 3: Efficiency curves when sweeping $K$, the number of comparisons per
context, with 95% confidence intervals over 100 samples per $K$.

Both Gaussian PoE and BT PoE result in similar performing solutions Note that
when using full-comparisons, the Gaussian PoE solution can be shown to be
equivalent to the average win-ratio (with the proof in Appendix A.3) however
the optimized BT PoE approach will lead to a different solution. Nonetheless,
the performance for both PoE-BT and PoE-g are very comparable for most
models/datasets, in both the hard and soft set-ups. The Gaussian solution has
the benefit of having a convenient closed form solution, while the PoE-BT has
to be solved with gradient based optimization (with Zermelo no longer
applicable). PoE-BT does achieve marginally better in several set-ups,
although within the standard deviation ranges.

Convergence rates The results in Table 1 showed performance for the arbitrary
chosen operating point of $K\\!=\\!24$. Figure 3 shows the performance for a
few selected models and attributes while sweeping $K$ from $K\\!=\\!N$ to the
full set of comparisons, $K\\!=\\!N(N\\!-\\!1)/2$. The curves show that the
performance improves smoothly with increasing number of comparisons, with the
convergence rates considerably better with the PoE methods. Further plots for
other models/tasks are provided in Appendix C.3.

Performance gains are most significant for large N CMCQRD is a dataset with
658 sample texts, which therefore requires over 200,000 comparisons when
considering all of them. Figure 5(f), which uses a log x-axis, shows that PoE-
gaussian reaches the solution with as few as 2000 comparisons, significantly
faster than the other baseline methods which require orders of magnitude more
comparisons.

###  5.3 Comparison Selection

system | method | win-ratio | avg-prob | PoE-BT | PoE-g
---|---|---|---|---|---
Flant5-3B | random |  34.1$\pm$0.8 |  38.4$\pm$0.6 |  42.7$\pm$0.4 |  42.4$\pm$0.4
optimal |  36.0$\pm$0.6 |  39.3$\pm$0.6 |  43.2$\pm$0.4 |  42.9$\pm$0.3
Flant5-11B | random |  31.2$\pm$0.8 |  34.7$\pm$0.7 |  38.4$\pm$0.4 |  38.4$\pm$0.4
optimal |  33.1$\pm$0.6 |  35.7$\pm$0.7 |  39.2$\pm$0.4 |  39.0$\pm$0.4
Llama2-7B | random |  21.6$\pm$0.8 |  24.0$\pm$0.7 |  26.8$\pm$0.5 |  26.6$\pm$0.5
optimal |  23.0$\pm$0.8 |  24.5$\pm$0.6 |  27.3$\pm$0.4 |  27.2$\pm$0.4
Llama2-13B | random |  30.8$\pm$0.7 |  33.7$\pm$0.6 |  37.7$\pm$0.4 |  37.3$\pm$0.4
optimal |  32.4$\pm$0.7 |  34.6$\pm$0.6 |  38.2$\pm$0.3 |  38.0$\pm$0.4

Table 2: SummEval Spearman correlations when using the greedy optimal set of
comparisons, for $K=24$.

Figure 4: FlanT5-3B, SummEval COH

The previous Section considered the set-up where the comparisons were randomly
selected from all possible comparisons. An alternative would be to select a
set of comparisons that maximizes the information gained from a fixed number
of comparisons. Section 3.6 discusses how under the Gaussian-POE, this can be
achieved with a practical greedy approximation. Table 2 shows that at the
operating point of $K\\!=\\!24$, one can gain further performance boosts by
selecting the comparisons, with the average performance of the probabilistic
PoE approaches consistently increasing by $0.5-0.7$ for all approaches, at no
extra LLM cost. Further, although the theory was derived using the Gaussian
assumptions, the performance boosts are observed across all the different
methods, and are more significant when considering the win-ratio. Lastly,
Figure 4 shows that the performance gains are significant when few comparisons
are made, but as the number of comparison grows, the performance difference
between random and optimal selection is negligible.

##  6 Conclusions

Comparative assessment using LLMs has been demonstrated to be effective for
text assessment. This paper investigates framing the scoring process within a
Product of Experts framework, where the comparison information (including
model confidence) can be easily combined to determine a set of scores that
effectively capture text quality. This enables comparative assessment to not
suffer from the $O(N^{2})$ complexity, as now only a subset of the possible
comparisons can be used to predict scores that are similarly as accurate as
those from the full set of comparisons. Further, using Gaussian experts yields
a closed-form solution and provides a basis for deriving a greedy-optimal set
of comparisons. The paper demonstrated the effectiveness of the approach on
multiple different standard NLG evaluation datasets, such as SummEval and
TopicalChat, as well as for question difficulty ranking with $N\\!>\\!500$,
which led to substantial computational savings against standard methods.

## Acknowledgements

The research in this paper was supported by Cambridge University Press &
Assessment (CUP&A), a department of The Chancellor, Masters, and Scholars of
the University of Cambridge. This work was also supported by the Gates
Cambridge Trust (grant OPP1144 from the Bill & Melinda Gates Foundation) and
EPSRC (The Engineering and Physical Sciences Research Council) Doctoral
Training Partnership (DTP) PhD studentship.

## References

  * [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman and Shyamal Anadkat  “Gpt-4 technical report”  In _arXiv preprint arXiv:2303.08774_ , 2023
  * [2] Satanjeev Banerjee and Alon Lavie  “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments”  In _Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization_ , 2005, pp. 65–72
  * [3] David Beaudoin and Tim Swartz  “A computationally intensive ranking system for paired comparison data”  In _Operations Research Perspectives_ 5 Elsevier, 2018, pp. 105–112
  * [4] Stephen P Boyd and Lieven Vandenberghe  “Convex optimization”  Cambridge university press, 2004
  * [5] Ralph Allan Bradley and Milton E Terry  “Rank analysis of incomplete block designs: I. The method of paired comparisons”  In _Biometrika_ 39.3/4 JSTOR, 1952, pp. 324–345
  * [6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry and Amanda Askell  “Language models are few-shot learners”  In _Advances in neural information processing systems_ 33, 2020, pp. 1877–1901
  * [7] Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai and Hang Li  “Learning to rank: from pairwise approach to listwise approach”  In _Proceedings of the 24th international conference on Machine learning_ , 2007, pp. 129–136
  * [8] Manuela Cattelan  “Models for paired comparison data: A review with emphasis on dependent data”, 2012
  * [9] Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang and Benyou Wang  “Humans or LLMs as the Judge? A Study on Judgement Biases”, 2024  arXiv:2402.10669 [cs.CL]
  * [10] Pinhan Chen, Chao Gao and Anderson Y Zhang  “Optimal full ranking from pairwise comparisons”  In _The Annals of Statistics_ 50.3 Institute of Mathematical Statistics, 2022, pp. 1775–1805
  * [11] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani and Siddhartha Brahma  “Scaling instruction-finetuned language models”  In _arXiv preprint arXiv:2210.11416_ , 2022
  * [12] László Csató  “Ranking by pairwise comparisons for Swiss-system tournaments”  In _Central European Journal of Operations Research_ 21 Springer, 2013, pp. 783–803
  * [13] Herbert Aron David  “The method of paired comparisons”  London, 1963
  * [14] Roger R Davidson and Peter H Farquhar  “A bibliography on the method of paired comparisons”  In _Biometrics_ JSTOR, 1976, pp. 241–252
  * [15] Otto Dykstra  “A Note on the Rank Analysis of Incomplete Block Designs–Applications Beyond the Scope of Existing Tables”  In _Biometrics_ 12.3 JSTOR, 1956, pp. 301–306
  * [16] Nouha Dziri, Ehsan Kamalloo, Kory Mathewson and Osmar R Zaiane  “Evaluating Coherence in Dialogue Systems using Entailment”  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , 2019, pp. 3806–3812
  * [17] Alexander R Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher and Dragomir Radev  “Summeval: Re-evaluating summarization evaluation”  In _Transactions of the Association for Computational Linguistics_ 9 MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info …, 2021, pp. 391–409
  * [18] Jinlan Fu, See-Kiong Ng, Zhengbao Jiang and Pengfei Liu  “Gptscore: Evaluate as you desire”  In _arXiv preprint arXiv:2302.04166_ , 2023
  * [19] M.J.F. Gales and S.S. Airey  “Product of Gaussians for speech recognition”  In _Computer Speech & Language_ 20.1, 2006, pp. 22–40  DOI: https://doi.org/10.1016/j.csl.2004.12.002
  * [20] Ralf Herbrich, Tom Minka and Thore Graepel  “TrueSkill™: a Bayesian skill rating system”  In _Advances in neural information processing systems_ 19, 2006
  * [21] Geoffrey E. Hinton  “Products of experts”  In _Artificial Neural Networks, 1999. ICANN 99. Ninth International Conference on (Conf. Publ. No. 470)_ 1, 1999, pp. 1–6  IET
  * [22] Xun Huang, Arun Mallya, Ting-Chun Wang and Ming-Yu Liu  “Multimodal Conditional Image Synthesis with Product-of-Experts GANs”  In _Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XVI_ <conf-loc content-type="InPerson">Tel Aviv, Israel</conf-loc>: Springer-Verlag, 2022, pp. 91–109  DOI: 10.1007/978-3-031-19787-1_6
  * [23] David R Hunter  “MM algorithms for generalized Bradley-Terry models”  In _The annals of statistics_ 32.1 Institute of Mathematical Statistics, 2004, pp. 384–406
  * [24] Tom Kocmi and Christian Federmann  “Large language models are state-of-the-art evaluators of translation quality”  In _arXiv preprint arXiv:2302.14520_ , 2023
  * [25] Wojciech Kryściński, Bryan McCann, Caiming Xiong and Richard Socher  “Evaluating the Factual Consistency of Abstractive Text Summarization”  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , 2020, pp. 9332–9346
  * [26] Svetlana Kutuzova, Oswin Krause, Douglas McCloskey, Mads Nielsen and Christian Igel  “Multimodal variational autoencoders for semi-supervised learning: In defense of product-of-experts”  In _arXiv preprint arXiv:2101.07240_ , 2021
  * [27] Chin-Yew Lin  “Rouge: A package for automatic evaluation of summaries”  In _Text summarization branches out_ , 2004, pp. 74–81
  * [28] Tie-Yan Liu  “Learning to rank for information retrieval”  In _Foundations and Trends® in Information Retrieval_ 3.3 Now Publishers, Inc., 2009, pp. 225–331
  * [29] Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu and Chenguang Zhu  “G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment”  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ Singapore: Association for Computational Linguistics, 2023, pp. 2511–2522  DOI: 10.18653/v1/2023.emnlp-main.153
  * [30] Yinhong Liu, Han Zhou, Zhijiang Guo, Ehsan Shareghi, Ivan Vulić, Anna Korhonen and Nigel Collier  “Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators”, 2024  arXiv:2403.16950 [cs.CL]
  * [31] Adian Liusie, Yassir Fathullah and Mark JF Gales  “Teacher-Student Training for Debiasing: General Permutation Debiasing for Large Language Models”  In _arXiv preprint arXiv:2403.13590_ , 2024
  * [32] Adian Liusie, Potsawee Manakul and Mark Gales  “LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models”  In _Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)_ St. Julian’s, Malta: Association for Computational Linguistics, 2024, pp. 139–151  URL: https://aclanthology.org/2024.eacl-long.8
  * [33] Jordan J Louviere, David A Hensher and Joffre D Swait  “Stated choice methods: analysis and applications”  Cambridge university press, 2000
  * [34] R Duncan Luce  “Individual choice behavior: A theoretical analysis”  Courier Corporation, 2005
  * [35] Potsawee Manakul, Adian Liusie and Mark JF Gales  “Mqag: Multiple-choice question answering and generation for assessing information consistency in summarization”  In _arXiv preprint arXiv:2301.12307_ , 2023
  * [36] Charles F Manski  “The structure of random utility models”  In _Theory and decision_ 8.3 Kluwer Academic Publishers, 1977, pp. 229
  * [37] Shikib Mehri and Maxine Eskenazi  “USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generation”  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 2020, pp. 681–707
  * [38] Tom Minka, Ryan Cleven and Yordan Zaykov  “Trueskill 2: An improved bayesian skill rating system”  In _Technical Report_ , 2018
  * [39] Piotr Molenda, Adian Liusie and Mark J.. Gales  “WaterJudge: Quality-Detection Trade-off when Watermarking Large Language Models”, 2024  arXiv:2403.19548 [cs.CL]
  * [40] Andrew Mullooly, Øistein Andersen, Luca Benedetto, Paula Buttery, Andrew Caines, Mark JF Gales, Yasin Karatay, Kate Knill, Adian Liusie and Vatsal Raina  “The Cambridge Multiple-Choice Questions Reading Dataset”  Cambridge University PressAssessment, 2023
  * [41] MEJ Newman  “Efficient computation of rankings from pairwise comparisons”  In _Journal of Machine Learning Research_ 24.238, 2023, pp. 1–25
  * [42] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama and Alex Ray  “Training language models to follow instructions with human feedback”  In _Advances in Neural Information Processing Systems_ 35, 2022, pp. 27730–27744
  * [43] Kishore Papineni, Salim Roukos, Todd Ward and Wei-Jing Zhu  “Bleu: a method for automatic evaluation of machine translation”  In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_ , 2002, pp. 311–318
  * [44] ChaeHun Park, Minseok Choi, Dohyun Lee and Jaegul Choo  “PairEval: Open-domain Dialogue Evaluation with Pairwise Comparison”  In _arXiv preprint arXiv:2404.01015_ , 2024
  * [45] Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler and Xuanhui Wang  “Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting”  In _arXiv preprint arXiv:2306.17563_ , 2023
  * [46] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava and Shruti Bhosale  “Llama 2: Open foundation and fine-tuned chat models”  In _arXiv preprint arXiv:2307.09288_ , 2023
  * [47] Ramakrishna Vedantam, Ian Fischer, Jonathan Huang and Kevin Murphy  “Generative Models of Visually Grounded Imagination”  In _International Conference on Learning Representations_ , 2018  URL: https://openreview.net/forum?id=HkCsm6lRb
  * [48] Alex Wang, Kyunghyun Cho and Mike Lewis  “Asking and Answering Questions to Evaluate the Factual Consistency of Summaries”  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , 2020, pp. 5008–5020
  * [49] Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu and Jie Zhou  “Is chatgpt a good nlg evaluator? a preliminary study”  In _arXiv preprint arXiv:2303.04048_ , 2023
  * [50] Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai and Quoc V Le  “Finetuned Language Models are Zero-Shot Learners”  In _International Conference on Learning Representations_ , 2021
  * [51] M. Welling  “Product of experts” revision #137078  In _Scholarpedia_ 2.10, 2007, pp. 3879  DOI: 10.4249/scholarpedia.3879
  * [52] Christopher Williams, Felix Agakov and Stephen Felderhof  “Products of Gaussians”  In _Advances in Neural Information Processing Systems_ 14 MIT Press, 2001  URL: https://proceedings.neurips.cc/paper_files/paper/2001/file/8232e119d8f59aa83050a741631803a6-Paper.pdf
  * [53] Mike Wu and Noah Goodman  “Multimodal Generative Models for Scalable Weakly-Supervised Learning”  In _Advances in Neural Information Processing Systems_ 31 Curran Associates, Inc., 2018  URL: https://proceedings.neurips.cc/paper_files/paper/2018/file/1102a326d5f7c9e04fc3c89d0ede88c9-Paper.pdf
  * [54] Zheng Ye, Liucun Lu, Lishan Huang, Liang Lin and Xiaodan Liang  “Towards Quantifiable Dialogue Coherence Evaluation”  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , 2021, pp. 2718–2729
  * [55] Weizhe Yuan, Graham Neubig and Pengfei Liu  “Bartscore: Evaluating generated text as text generation”  In _Advances in Neural Information Processing Systems_ 34, 2021, pp. 27263–27277
  * [56] Heiga Zen, Mark JF Gales, Yoshihiko Nankaku and Keiichi Tokuda  “Product of experts for statistical parametric speech synthesis”  In _IEEE Transactions on Audio, Speech, and Language Processing_ 20.3 IEEE, 2011, pp. 794–805
  * [57] Ernst Zermelo  “Die berechnung der turnier-ergebnisse als ein maximumproblem der wahrscheinlichkeitsrechnung”  In _Mathematische Zeitschrift_ 29.1 Springer, 1929, pp. 436–460
  * [58] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger and Yoav Artzi  “Bertscore: Evaluating text generation with bert”  In _arXiv preprint arXiv:1904.09675_ , 2019
  * [59] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li and Eric Xing  “Judging LLM-as-a-judge with MT-Bench and Chatbot Arena”  In _arXiv preprint arXiv:2306.05685_ , 2023
  * [60] Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji and Jiawei Han  “Towards a Unified Multi-Dimensional Evaluator for Text Generation”  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ Abu Dhabi, United Arab Emirates: Association for Computational Linguistics, 2022, pp. 2023–2038  DOI: 10.18653/v1/2022.emnlp-main.131

A PoE Perspective of LLM Comparative Assessment: Efficient Scoring from
Pairwise Comparisons
(Supplementary Material)

Adian Liusie, Vatsal Raina, Yassir Fathullah, Mark J. F. Gales
Engineering Department, University of Cambridge
{al826, vr311, yf286<EMAIL_ADDRESS>
##  Appendix A Additional Product of Expert Theory

###  A.1 Structure of $\bf{\tilde{W}}$ Matrix

The paper discussed the comparison matrix
${\bf\tilde{W}}\\!\in\\!R^{K\\!+\\!1\times N}$, where each row represents the
particular comparison being considered. It was discussed how for the
$k^{\text{th}}$ comparison between $i$ and $j$, ${\bf W}_{ki}\\!=\\!1$, ${\bf
W}_{kj}\\!=\\!-1$, and ${\bf W}_{km}\\!=\\!0\;\;\forall m\neq i,j$. Further,
an extra row was prepended to ${\bf W}$ adding constraints on the first score,
forming ${\bf\tilde{W}}$ and ensuring the corresponding matrix is not
defective. To illustrate the structure of ${\bf\tilde{W}}$, consider the case
where one has 4 elements $x_{1:4}$ and all possible comparisons are
considered,

$\displaystyle{\bf\tilde{W}}=\left[\begin{array}[]{c c c c}1&0&0&0\\\
1&-1&0&0\\\ 1&0&-1&0\\\ §1&0&0&-1\\\ 0&1&-1&0\\\ 0&1&0&-1\\\ 0&0&1&-1\\\
\end{array}\right]$ (21)

###  A.2 Structure of ${\bf W^{\scriptstyle\sf T}W}$ Matrix

In the Gaussian-Products of Experts, the variance was shown to be directly
related to the matrix ${\bf\tilde{W}}^{\scriptstyle\sf T}{\bf\tilde{W}}$. For
the full comparison case previously considered, this would yield a matrix of
the form,

$\displaystyle{\tilde{\bf W}}^{\scriptstyle\sf T}{\tilde{\bf
W}}=\left[\begin{array}[]{c c c c}4&-1&-1&-1\\\ -1&3&-1&-1\\\ -1&-1&3&-1\\\
-1&-1&-1&3\\\ \end{array}\right]$ (26)

Let ${\bf\tilde{A}}={\bf\tilde{W}}^{\scriptstyle\sf T}{\bf\tilde{W}}$. For any
set of selected comparisons,
${\bf\tilde{A}}_{ij}={\bf\tilde{w}}_{i}\cdot{\bf\tilde{w}}_{j}$. Therefore by
taking into account the structure of $\tilde{W}$, it’s easily shown that the
diagonal elements represent the number of comparisons the element has been
involved in, while the off-diagonal elements are -1 if the comparison is made,

${\bf\tilde{A}}_{kk}=\sum_{i}1(x_{k}\in\mathcal{C}_{i})\qquad\qquad{\bf\tilde{A}}_{ij}=\begin{cases}-1&\text{if
}(x_{i},x_{j})\in\mathcal{C}_{K},\\\ \;0&\text{otherwise}.\end{cases}$ (27)

This means that for the full comparison matrix, irrespective of $N$, the
matrix ${\tilde{\bf W}}^{\scriptstyle\sf T}{\tilde{\bf W}}$ will have the
form,

$\displaystyle{\tilde{\bf W}}^{\scriptstyle\sf T}{\tilde{\bf
W}}=\left[\begin{array}[]{c c c c c}N&-1&-1&\ldots&-1\\\
-1&N\\!-\\!1&-1&\ldots&-1\\\ -1&-1&N\\!-\\!1&\ldots&-1\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\ -1&-1&-1&\ldots&N\\!-\\!1\\\
\end{array}\right]$ (33)

###  A.3 Equivalence of Gaussian PoE Solution with Average Probability

Given the structure of ${\tilde{\bf W}}^{\scriptstyle\sf T}{\tilde{\bf W}}$,
when considering the full-comparison set-up, the inverse is given by,

$\displaystyle\left({\tilde{\bf W}}^{\scriptstyle\sf T}{\tilde{\bf
W}}\right)^{-1}$ $\displaystyle=$ $\displaystyle\left[\begin{array}[]{c c c c
c}1&1&1&\ldots&1\\\ 1&1+\frac{2}{N}&1+\frac{1}{N}&\ldots&1+\frac{1}{N}\\\
1&1+\frac{1}{N}&1+\frac{2}{N}&\ldots&1+\frac{1}{N}\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\
1&1+\frac{1}{N}&1+\frac{1}{N}&\ldots&1+\frac{2}{N}\\\ \end{array}\right]$ (39)
$\displaystyle=$ $\displaystyle\frac{N+1}{N}\left[\begin{array}[]{c c c c
c}1&1&1&\ldots&1\\\ 1&1&1&\ldots&1\\\ 1&1&1&\ldots&1\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\ 1&1&1&\ldots&1\\\
\end{array}\right]+\frac{1}{2N}\left[\begin{array}[]{c c c c
c}-1&-1&-1&\ldots&-1\\\ -1&1&0&\ldots&0\\\ -1&0&1&\ldots&0\\\
\vdots&\vdots&\vdots&\ddots&\vdots\\\ -1&0&0&\ldots&1\\\ \end{array}\right]$
(50)

For the Gaussian PoE with linear mean and constant Gaussian assumptions, the
solution was shown to be of form
${\bf\hat{s}}=\alpha\cdot({\bf\tilde{W}}^{{\scriptstyle\sf
T}}{\bf\tilde{W}})^{-1}{\bf\tilde{W}}\bm{\tilde{\mu}}$. By noting that
$\bm{\tilde{\mu}}$ represents the LLM probabilities for each comparative
decision, we observe that ${\bf\tilde{W}}\bm{\tilde{\mu}}$ simply represents
the sum of probabilities for all comparisons that each element has been a part
of. Therefore, Equation 50 shows that the solution will be a constant shift of
the average probability for any particular sample.

###  A.4 Form of the PoE Score Distribution

To determine ${\tt p}({\bf s}|\mathcal{C}_{1:K})$ given ${\tt P}({\bf
Ws}|\mathcal{C}_{1:K})=\mathcal{N}\left({\bf
Ws};\bm{\tilde{\mu}},{\bf\tilde{\Sigma}}\right)$, one can expand the
expression and isolate all terms that have an ${\bf s}$, yielding,

$\displaystyle{\tt p}({\bf Ws}|\mathcal{C}_{1:K})$
$\displaystyle=\mathcal{N}\left({\bf
Ws};\bm{\tilde{\mu}},{\bf\tilde{\Sigma}}\right)$ (51)
$\displaystyle\propto\exp\left(-\frac{1}{2}\;({\bf
Ws}-{\bm{\tilde{\mu}}})^{\scriptstyle\sf T}{\bf\tilde{\Sigma}}^{-1}({\bf
Ws}-{\bm{\tilde{\mu}}})\right)$ (52)
$\displaystyle\propto\exp\left(-\frac{1}{2}\left({\bf s}^{\scriptstyle\sf
T}{\bf W}^{\scriptstyle\sf T}{\bf\tilde{\Sigma}}^{-1}{\bf Ws}+2{\bf
s}^{\scriptstyle\sf T}{\bf\tilde{\Sigma}}^{-1}{\bf
W}\tilde{\bm{\mu}}\right)\right)$ (53)

As the distribution over scores will be Gaussian, ${\tt p}({\bf
s}|C_{1:K})\sim\mathcal{N}({\bf s};{\bm{\mu}^{*}},{\bf\Sigma^{*}})$, one can
equate coefficients to derive the form used in the paper,

${\tt p}({\bf s}|C_{1:K})=\mathcal{N}\left({\bf
s};({\bf\tilde{W}}^{\scriptstyle\sf
T}{\bf\tilde{\Sigma}}^{-1}{\bf\tilde{W}})^{-1}{\bf\tilde{W}}{\bf\tilde{\Sigma}}^{-1}\bm{\tilde{\mu}}\;,\;({\bf\tilde{W}}^{\scriptstyle\sf
T}{\bf\tilde{\Sigma}}^{-1}{\bf\tilde{W}})^{-1}\right)$ (54)

###  A.5 Efficient GreedyOptimal Selection

It was shown that given ${\bf\tilde{W}}^{(k)*}$ and ${\bf
A}^{(k)*}\\!=\\!({\bf\tilde{W}}^{(k)*^{\scriptstyle\sf
T}}{\bf\tilde{W}}^{(k)*})^{-1}$, the next optimal comparison
$(\hat{i},\hat{j})$ can be calculated as,

$\hat{i},\hat{j}=\operatorname*{arg\,max}_{i,j}{\bf A}^{(k)*}_{ii}+{\bf
A}^{(k)*}_{jj}-2\cdot{\bf A}^{(k)*}_{ij}$ (55)

Updating ${\bf\tilde{W}}^{(k)*}$ is trivial, since considering an additional
comparison $(i,j)$ is equivalent to adding an extra row ${\bf r}\in R^{N}$ to
${\bf\tilde{W}}^{(k)*}$, where ${\bf r}_{i}\\!=\\!1$, ${\bf r}_{j}\\!=\\!-1$
and ${\bf r}_{l}\\!=\\!0\;\;\forall l\neq i,j$. Therefore

${\bf\tilde{W}}^{(k+1)*}=[{\bf\tilde{W}}^{(k)*};{\bf r}]$ (56)

However one can also efficiently update the inverse using the Sherman-Morrison
inversion lemma,

$\displaystyle{\bf A}^{(k+1)*}$
$\displaystyle=\left([{\bf\tilde{W}}^{(k)*};{\bf r}]^{\scriptstyle\sf
T}[{\bf\tilde{W}}^{(k)*};{\bf r}]\right)^{-1}$ (57)
$\displaystyle=\left({\bf\tilde{W}}^{(k)*^{\scriptstyle\sf
T}}{\bf\tilde{W}}^{(k)*}+{\bf r}^{\scriptstyle\sf T}{\bf r}\right)^{-1}$ (58)
$\displaystyle={\bf A}^{(k)*}+\frac{{\bf A}^{(k)*}{\bf r}{\bf
r}^{\scriptstyle\sf T}{\bf A}^{(k)*}}{1+{\bf r}^{\scriptstyle\sf T}{\bf
A}^{(k)*}{\bf r}}$ (59)

Note that to initialize ${\bf\tilde{W}}$, the simplest option would be to use
$N-1$ comparisons and follow a stripped diagonal matrix, e.g.

$\displaystyle{\bf\tilde{W}}=\left[\begin{array}[]{c c c c}1&0&0&0\\\
1&-1&0&0\\\ 0&1&-1&0\\\ 0&0&1&-1\\\ \end{array}\right]$ (64)

###  A.6 Gaussian Assumptions under a Biased System

In Section 3.5, it was discussed how one could set the mean of the Gaussian
Product-of-Expert model to $f_{\mu}(p)=\alpha\\!\cdot\\!(p-\beta)$, where
$\beta=0.50$ for an unbiased comparative assessment system. However, in a
single comparative assessment call where we task the underlying LLM with
predicting which candidate is better, the predicted probabilities suffer from
positional bias. To accommodate this, one can either permute the options and
call the LLM again to eliminate positional bias, or change the modelling
structure. The latter can be done by shifting the bias term $\beta$ according
to:

$\displaystyle\beta=\frac{1}{K}\sum_{i,j\in C_{1:K}}p_{ij}$ (65)

Using this $\beta$ will allow the Gaussian expert to accommodate for the bias
that exists in the outputs of the underlying LLM, without resorting to calling
the LLM for both permutations.

##  Appendix B Product of Laplacians

The Product of Experts framework is a simple approach to modelling a high-
dimensional distribution by breaking it down into simpler and smaller models,
the so-called experts. As observed in Section 3, choosing a Gaussian expert is
highly convenient due to the closed-form solution one can obtain. This is
particularly desirable for cases with large numbers of candidates and
comparisons.

Another valid choice is using the Laplace distribution as an expert:

$\displaystyle{\tt
p}(s_{i}\\!-\\!s_{j}|p_{ij})=\mathcal{L}{\big{(}}s_{i}\\!-\\!s_{j};f_{\mu}(p_{ij}),f_{\sigma}(p_{ij}){\big{)}}=\frac{1}{2f_{\sigma}(p_{ij})}\exp\left(-\frac{\lvert
s_{i}\\!-\\!s_{j}-f_{\mu}(p_{ij})\rvert}{f_{\sigma}(p_{ij})}\right)$ (66)

The longer tails of the Laplacian could help it be more robust to outlier or
very noisy comparisons that could affect the predicted scores $\hat{s}$.
However unlike the Gaussian, the optimal solution for the Product of
Laplacians does not have a closed form.

###  B.1 Product of Laplacians

The aim is to find the solution to the following problem:

$\displaystyle\hat{s}_{1:N}$
$\displaystyle=\operatorname*{arg\,max}_{s_{1:N}}\hskip 2.84526pt{\tt
p}(s_{1:N}|\mathcal{C}_{1:K})$ (67)
$\displaystyle=\operatorname*{arg\,max}_{s_{1:N}}\prod_{i,j\in\mathcal{C}_{1:K}}{\tt
p}(s_{i}\\!-\\!s_{j}|p_{ij})$ (68)
$\displaystyle=\operatorname*{arg\,max}_{s_{1:N}}\prod_{i,j\in\mathcal{C}_{1:K}}\frac{1}{2f_{\sigma}(p_{ij})}\exp\left(-\frac{\lvert
s_{i}\\!-\\!s_{j}-f_{\mu}(p_{ij})\rvert}{f_{\sigma}(p_{ij})}\right)$ (69)
$\displaystyle=\operatorname*{arg\,min}_{s_{1:N}}\sum_{i,j\in\mathcal{C}_{1:K}}\frac{\lvert
s_{i}\\!-\\!s_{j}-f_{\mu}(p_{ij})\rvert}{f_{\sigma}(p_{ij})}$ (70)

While Equation 70 is a non-linear optimization problem, we can introduce a new
set of parameters $\eta\geq 0$ to recast this as a Linear Program [4]. Let:

$\displaystyle\frac{\lvert
s_{i}\\!-\\!s_{j}-f_{\mu}(p_{ij})\rvert}{f_{\sigma}(p_{ij})}\leq\eta_{ij}$
(71)

Then, the optimization can be recast as minimising the sum of $\eta_{ij}$:

$\displaystyle
s_{1:N}^{*},\\{\eta_{ij}^{*}\\}_{i,j\in\mathcal{C}_{1:K}}=\operatorname*{arg\,min}_{s_{1:N},\\{\eta_{ij}\\}}$
$\displaystyle\sum_{i,j}\eta_{ij}$ (72) subject to
$\displaystyle\frac{s_{i}\\!-\\!s_{j}-f_{\mu}(p_{ij})}{f_{\sigma}(p_{ij})}\leq\eta_{ij}$
(73)
$\displaystyle\frac{s_{i}\\!-\\!s_{j}-f_{\mu}(p_{ij})}{f_{\sigma}(p_{ij})}\geq-\eta_{ij}$
(74)

This follows the form of a Linear Program, and can be solved efficiently using
a wide range of approaches applicable to this class of convex optimization
problems. Instead of minimizing a non-linear problem with $N$ variables, we
have extended it to $N+K$ unknowns in a linear format.

##  Appendix C Extra Results

###  C.1 Summeval Pearson Performance Tables

The main paper illustrated the context-level Spearman correlations for
SummEval. For certain applications, one may not only care about the rank
ordering of the points, but also the relative spacing between them as this
provides information of the predicted quality difference between any two
texts. Table 3 therefore presents the Pearson correlations for SummEval, where
very trends to the Spearman table are observed.

system | $R$ | win-ratio | BT | PoE-g-hard | avg-prob | PoE-BT | PoE-g
---|---|---|---|---|---|---|---
Flant5-3B | 24 |  34.3$\pm$0.8 |  37.2$\pm$0.7 |  35.0$\pm$0.7 |  42.3$\pm$0.5 |  48.3$\pm$0.3 |  47.1$\pm$0.3
120 |  44.1$\pm$0.0 |  45.0$\pm$0.0 |  44.1$\pm$0.0 |  49.4$\pm$0.0 |  50.0$\pm$0.0 |  49.4$\pm$0.0
Flant5-11B | 24 |  31.7$\pm$0.7 |  34.2$\pm$0.7 |  32.3$\pm$0.7 |  37.3$\pm$0.6 |  41.7$\pm$0.4 |  41.4$\pm$0.5
120 |  40.8$\pm$0.0 |  41.4$\pm$0.0 |  40.8$\pm$0.0 |  43.7$\pm$0.0 |  44.0$\pm$0.0 |  43.7$\pm$0.0
Llama2-7B | 24 |  21.7$\pm$0.7 |  23.5$\pm$0.6 |  22.3$\pm$0.7 |  24.3$\pm$0.6 |  26.9$\pm$0.4 |  26.8$\pm$0.4
120 |  27.8$\pm$0.0 |  27.8$\pm$0.0 |  27.8$\pm$0.0 |  28.4$\pm$0.0 |  28.4$\pm$0.0 |  28.4$\pm$0.0
Llama2-13B | 24 |  31.3$\pm$0.7 |  33.8$\pm$0.6 |  32.0$\pm$0.7 |  36.0$\pm$0.5 |  40.6$\pm$0.4 |  39.9$\pm$0.4
120 |  39.8$\pm$0.0 |  40.4$\pm$0.0 |  39.9$\pm$0.0 |  42.1$\pm$0.0 |  42.5$\pm$0.0 |  42.1$\pm$0.0
Table 3: Pearson correlations for SummEval, averaged over all attributes (COH,
CON, FLU, REL). $K$ is the number of balanced comparisons made, where
$K\\!=\\!120$ is the full set of comparisons.

###  C.2 TopicalChat Performance Tables

Table 4 and 5 demonstrate performance for comparative assessment when applied
to dialogue evaluation. The PoE approaches continue to provide considerable
performance improvements at the operating point $K\\!=\\!9$, albeit since $N$
is no longer that large ($N\\!=\\!6$), the full set of comparisons is no
longer as infeasible to compute, and savings are less significant.

system | $R$ | win-ratio | BT | PoE-g-hard | avg-prob | PoE-BT | POE-g
---|---|---|---|---|---|---|---
FlanT5-3B | 9 |  41.3$\pm$1.3 |  41.8$\pm$1.2 |  41.6$\pm$1.3 |  43.4$\pm$1.2 |  45.5$\pm$0.9 |  45.2$\pm$0.8
15 |  45.3$\pm$0.0 |  44.8$\pm$0.0 |  45.3$\pm$0.0 |  44.7$\pm$0.0 |  44.7$\pm$0.0 |  44.7$\pm$0.0
FlanT5-11B | 9 |  51.2$\pm$1.2 |  52.4$\pm$1.1 |  51.9$\pm$1.1 |  53.8$\pm$1.1 |  56.1$\pm$0.8 |  56.1$\pm$0.8
15 |  57.0$\pm$0.0 |  56.6$\pm$0.0 |  56.0$\pm$0.0 |  58.1$\pm$0.0 |  58.1$\pm$0.0 |  58.0$\pm$0.0
Llama-7B | 9 |  28.4$\pm$1.2 |  28.9$\pm$1.0 |  28.7$\pm$1.1 |  27.7$\pm$1.4 |  29.8$\pm$1.0 |  29.5$\pm$1.0
15 |  31.5$\pm$0.0 |  31.6$\pm$0.0 |  31.6$\pm$0.0 |  31.5$\pm$0.0 |  31.5$\pm$0.0 |  31.5$\pm$0.0
Llama-13B | 9 |  37.4$\pm$1.1 |  38.1$\pm$1.1 |  37.9$\pm$1.0 |  38.4$\pm$1.2 |  40.6$\pm$0.7 |  40.5$\pm$0.9
15 |  41.6$\pm$0.0 |  41.7$\pm$0.0 |  41.8$\pm$0.0 |  41.6$\pm$0.0 |  41.6$\pm$0.0 |  41.6$\pm$0.0
Table 4: Spearman correlations for TopicalChat, averaged over all attributes (COH, CNT, ENG, NAT). $K$ is the number of balanced comparisons made, where $K\\!=\\!15$ is the full set of comparisons. system | $R$ | win-ratio | BT | PoE-g-hard | avg-prob | PoE-BT | PoE-g
---|---|---|---|---|---|---|---
FlanT5-3B | 9 |  42.1$\pm$1.2 |  43.1$\pm$1.1 |  42.8$\pm$1.1 |  45.7$\pm$1.0 |  48.1$\pm$0.6 |  47.9$\pm$0.7
15 |  46.5$\pm$0.0 |  46.5$\pm$0.0 |  46.5$\pm$0.0 |  48.7$\pm$0.0 |  48.6$\pm$0.0 |  48.7$\pm$0.0
FlanT5-11B | 9 |  51.5$\pm$1.2 |  53.3$\pm$1.0 |  52.9$\pm$1.0 |  56.3$\pm$0.9 |  58.0$\pm$0.7 |  58.3$\pm$0.6
15 |  57.5$\pm$0.0 |  57.4$\pm$0.0 |  57.4$\pm$0.0 |  59.8$\pm$0.0 |  59.7$\pm$0.0 |  59.8$\pm$0.0
Llama2-7B | 9 |  28.5$\pm$1.1 |  29.4$\pm$0.8 |  29.1$\pm$1.0 |  29.1$\pm$1.1 |  29.4$\pm$0.8 |  30.2$\pm$0.7
15 |  31.6$\pm$0.0 |  31.6$\pm$0.0 |  31.6$\pm$0.0 |  31.5$\pm$0.0 |  30.7$\pm$0.0 |  31.5$\pm$0.0
Llama2-13B | 9 |  37.5$\pm$1.1 |  38.7$\pm$1.0 |  38.4$\pm$1.0 |  40.2$\pm$1.0 |  41.9$\pm$0.5 |  41.8$\pm$0.6
15 |  41.4$\pm$0.0 |  41.5$\pm$0.0 |  41.4$\pm$0.0 |  42.5$\pm$0.0 |  42.6$\pm$0.0 |  42.5$\pm$0.0
Table 5: Pearson correlations for TopicalChat.

###  C.3 Extra Efficiency Plots from other Models and Datasets

(a) FlanT5-3B, SummEval CON

(b) FlanT5-11B, SummEval COH

(c) SummEval, REL, llama2-7B

(d) FlanT5-3B, TopicalChat COH

(e) FlanT5-11B, TopicalChat ENG

(f) Llama2-7B, TopicalChat NAT

Figure 5: Efficiency Curves when sweeping $R$, the number of comparisons per
context, with 95% confidence intervals over 100 samples per $R$.