# On Reality and the Limits of Language Data:
Aligning LLMs with Human Norms

Nigel H. Collier1 Fangyu Liu1 Ehsan Shareghi2
1University of Cambridge, UK 2Monash University, Australia
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Recent advancements in Large Language Models (LLMs) harness linguistic
associations in vast natural language data for practical applications.
However, their ability to understand the physical world using only language
data remains a question. After reviewing existing protocols, we explore this
question using a novel and tightly controlled reasoning test (ART) and compare
human norms against versions of GPT-3. Our findings highlight the categories
of common-sense relations models that could learn directly from data and areas
of weakness. GPT-3 offers evidence for verbal reasoning on a par with human
subjects for several relations including Synonymy, Antonymy, and Default
inheritance, Without reinforcement learning from human judgements, it appears
GPT-3 performs at the lower end of the reference interval for Has-part and
Contained-in. Weaknesses were observed also in affordance characteristics
through Necessary-quality, Order-of-size and Order-of-intensity. Combining
LLMs with symbolic world grounding is a promising direction to address
associative learning.

Keywords: large language models, common sense, human norms

## Introduction

We consider the relationship between the real world and the meaning
representations that are derived from large-scale language data using large
language models (LLMs) that have been trained using self-supervision, i.e.
learning to predict a part of the input from the other parts.

A widely discussed problem in the Philosophy of Language concerns the power of
language to represent reality - the world of ‘sticks and stones; cats and
trees’ (?, ?), of ‘molecules, galaxies, and babies’ (?, ?) that we know about
from advances in scientific discovery. This question finds voice in recent
works that try to close the gap between LLMs and normative human performance
on language tasks (?, ?, ?).

Do the limitations of LLMs arise from data or fundamental properties inherent
to their structure? A variety of evaluation metrics (?, ?) across multiple
tasks and models have been pursued to gain insight into this question (?, ?,
?). Whilst evaluations have spurred remarkable achievements in many tasks, we
argue that they obscure a fundamental question about the limitation of LLMs to
represent reality. We look at these models’ capacity to tell true from false
propositions about _naïve physics_ , a subset of what is called common sense
reasoning (?, ?, ?, ?), that focuses on the intuitive truth that human
cognition endorses about the physical world (?, ?).

Whilst common sense reasoning has long been considered a part of the natural
language understanding evaluations, e.g. the Winograd Schema Challenge (WSC)
(?, ?), the exact kinds of knowledge that model builders should target is
rarely made explicit.

The characteristic of reasoning with _expressed_ facts along with _unstated_
facts (enthymemes) from a store of common sense knowledge appears fundamental
to human understanding (?, ?). Likewise, LLMs will require access to
enthymematic mechanisms if they are to make predictions about and correctly
explain the long tail of _unstated_ facts not explicitly mentioned in training
data. We triangulate between three synergistic fields: philosophical ontology,
cognitive psychology and NLP. A philosophical approach to objective reality is
taken to motivate some of the necessary goals of common sense reasoning. We
then present a psychologically motivated test, based on analogical reasoning
to interpret model capabilities. The analogy test focuses on understanding
word meanings when presented as ordered pairs, i.e. of the form a is to b as c
is to d. Analogical Reasoning Tests (ARTs) such as the Miller Analogies Test
(?, ?) are widely trusted as indicators of human academic achievement and have
long been viewed in AI as crucial instruments to elucidate the role of
knowledge in problem solving (?, ?, ?, ?).

We present a new bipartite ART test, ART A&B,111Available to download in
BigBench json format from https://sites.google.com/site/nhcollier/projects/art
consisting both of analogy questions in Section A and a structured set of
semantic relation questions in Section B (covering 16 semantic relations) that
underlie the analogy test. The test is intended for use in zero-shot settings
to focus on true generalization from foundation knowledge. Additionally, we
use the test responses of 61 participants222Native English speaker residents
of the UK. to construct two normative profiles: the first against human expert
judgements and the second against a common sense judgement where at least 80%
of participants agreed. We report the results of five GPT-3 language models
and ChatGPT against the two profiles. To rule out results based on associative
behaviour we further examined the results of nonsymmetric semantic relation
items where the largest models (text-Davinci-002, code-Davinci-002 and
ChatGPT) fell within human norms. We observe that the three smallest GPT-3
models fall outside both expert and common sense norms. After ruling out
associative behaviour, the text-Davinci-002 falls within norms for half the
relations but at the lower end or outside human norms for relations that
require understanding of mereotopological and affordance relations. ChatGPT
which is augmented with data selection and reinforcement learning, performs
far better at understanding object qualities but still falls at the lower end
of human performance for several mereotopological relations.

We hope that this test will help to clarify several misconceptions about
common sense reasoning in current state-of-the-art LLMs, draw attention to
algorithmic deficiencies, and facilitate coherent strategies for achievement
outcomes.

## Related work

Here we briefly review the case for why naïve physics is a core part of common
sense, why corpora might be deficient in allowing us to reconstruct common
sense, and provide a survey of important common sense benchmarks.

#### What is common sense?

Despite its apparently obvious nature, common sense remains an uneasy
obscurity in the NLP literature which often describes it via its absence, e.g.
as the output of a system that elicits assertions within a large corpus (?, ?)
or as a database (?, ?). Arguably, a critical examination of what is common
sense knowledge? and its distinction from other types of knowledge feels long
overdue if we are to construct productive research questions, e.g. how stable
is such knowledge? how malleable are LLMs for representing such knowledge? A
satisfying discussion is beyond the scope of this work but for now we
highlight Brachman and Levesque’s (?, ?) working definition: Common sense is
the ability to make effective use of ordinary, everyday, experiential
knowledge in achieving ordinary, practical goals. The definition, with
parallels to Gibson’s world of affordances (?, ?), is ambitious and draws
attention to the need for models to act holistically upon common sense
knowledge within everyday tasks. We favour this as an end goal but within this
complex endeavour (?, ?) we are concerned to systematically investigate the
essential core of non-holistic common sense knowledge, i.e. commonly held
knowledge about physical reality which is capable of scientific treatment.
Recent commentaries on human intelligence support this view that an intuitive
understanding of the physical world is core in human infants and animals but
currently lacking in artificial intelligence (?, ?, ?).

In ontological terms what might obvious physical facts be? It is a difficult
question but in the tradition of philosophical realism, Smith (?, ?, ?) has
argued for a stable and culturally invariant notion of common sense. Following
this, the common sense we seek to capture is a subset of objective reality -
independent of an individual’s cognition - to which ordinary (non-expert)
cognition can relate via contact with the physical environment we perceive.

Are corpora Gestalt sources of common sense knowledge? A premise in NLP is
that LLMs might be able to recover implicit physical common sense knowledge
(?, ?). However, when thinking about facts reported in large-scale corpus
data, as Russell noted (?, ?), what might be commented on is not the whole
sense at one time but rather the part that individuals single out for
attention.

When we come to analyse factual descriptions arising from perceptual evidence
within a population it seems natural that we should expect variation due to
individual’s thinking disposition. A question therefore arises: when the
pieces of perceptual evidence are combined from a large group of individuals
we find on the Web, does a combination of data, the algorithm and the physical
computer (?, ?) allow an LLM to represent the sum of common sense reality as
precisely as humans perceive it from the environment? Such an important
question is almost never explicitly posed in probes of natural language
generation (NLG) which often lack sufficiency for semantic proposition content
(?, ?).

Common Sense Benchmarks. A substantial body of literature exists that observes
how models make predictions on common sense benchmarks. For a recent and
comprehensive survey we refer readers to Storks et al. (?, ?). One of the most
influential evaluations is the Winograd Schema Challenge (WSC) (?, ?), a
discriminative alternative to the generative probe set out in Turing’s 1950’s
Imitation Game (?, ?). Each question in the test has an ambiguous pronoun and
a binary choice of antecedents, for example, The trophy doesn’t fit in the
brown suitcase because it’s too big. Options: trophy or suitcase. The example
is designed so that humans easily resolve the ambiguity based on their
understanding that part of the physical and social reality of suitcase is its
use as a container.

The WSC has inspired much work, however it has been criticised on a variety of
grounds (?, ?) such as the high cognitive load required to craft the examples,
the limited range of test sentences, the sensitivity of some examples to
selectional restrictions, the presence of switchable referents, and the
possibility to employ simple associative tricks (?, ?) based on a high mutual
information between a trigger verb and an antecedent. Despite recent
applications of adversarial filtration to create Winogrande debiased (WG) a
large-scale version of the WSC that is immune to associative solutions (?, ?),
the limitations of WSC call into question the external validity of the test
and whether system performance on the WSC is indicative of advances in common
sense reasoning (?, ?). In response to these limitations, alternative
benchmarks have emerged which vary greatly in style and complexity such as
common sense question answering with OpenBook QA (?, ?, OBQA), CommonsenseQA
(?, ?, CSQA), Cosmos QA (?, ?, CQA), and Commonsense QA 2.0 (?, ?, CSQA2),
everyday situation understanding (?, ?, SWAG), and likely causes of sentence
endings with the Choice of Plausible Alternatives (?, ?, COPA). Closest to our
work is PROST (?, ?), focusing on physical common sense which uses an
alternative task construction to the one we present here: a small closed world
of objects serves as the source for the generation of multiple choice
questions using a set of fixed templates. Ten relations explore model
understanding of object attributes and affordances. As reported in Table 1
(upper part), whilst previous benchmarks have provided insights into common
sense reasoning by LLMs, these insights are often limited to the data set due
to their holistic nature and low level of ontological control.

Table 1: Attributes of modern common sense reasoning benchmarks. $N$ denotes
the relation did not appear to be explicitly controlled and vice versa.
1WordNet is included for reference; 2Social common sense corresponds to
linguistic/interpersonal intelligence (?, ?) and Physical common sense
corresponds to logical-mathematical/naturalistic intelligence; 3Expert
selection of objects and question patterns followed by automated generation;
${}^{4}12.2k$ Obtained after de-biasing; 5Hardening the data through bias
reduction; 6Hardening the data through distractor answers; 7Hardening the data
through gamification; 8Hardening of the data set is done indirectly via
restriction to zero-shot setting to avoid biasing training to testing data.

|  WSC |  WG |  CSQA |  CSQA2 |  COPA |  WNet1 |  PROST |  $\text{ART}_{\text{A\&B}}$
---|---|---|---|---|---|---|---|---
Physical2 |  Y |  Y |  Y |  Y |  Y |  Y |  Y |  Y
Social2 |  Y |  Y |  Y |  N |  Y |  Y |  N |  N
Source |  Expert |  Crowd |  Crowd |  Crowd |  Expert |  Expert |  Expert3 |  Expert
Size |  273 |  44k4 |  9.5k |  14.3k |  1K |  207k |  18.7k |  368
Hardening |  N |  Y5 |  Y6 |  Y7 |  N |  N |  Y8 |  Y8
Explicit relation controls
Synonym |  N |  N |  N |  N |  N |  Y |  N |  Y
Necessary quality |  N |  N |  N |  Y |  N |  N |  N |  Y
Associated quality |  N |  N |  N |  Y |  N |  N |  N |  Y
Has part |  N |  N |  Y |  Y |  N |  Y |  N |  Y
Order of intensity |  N |  N |  N |  N |  N |  N |  Y |  Y
Order of size |  N |  N |  N |  Y |  N |  N |  Y |  Y
Cause and effect |  N |  N |  Y |  Y |  Y |  N |  N |  Y
Capable of |  N |  N |  Y |  Y |  N |  N |  Y |  Y
Default inheritance |  N |  N |  N |  N |  N |  N |  N |  Y
Precondition |  N |  N |  Y |  Y |  N |  N |  N |  Y
Boundary (bona fide) |  N |  N |  N |  N |  N |  N |  N |  Y
Antonym |  N |  N |  Y |  N |  N |  Y |  N |  Y
Contained in |  N |  N |  N |  N |  N |  N |  N |  Y
Has member |  N |  N |  Y |  N |  N |  Y |  N |  Y
Troponym |  N |  N |  N |  N |  N |  Y |  N |  Y
Boundary (fiat) |  N |  N |  N |  N |  N |  N |  N |  Y
Example prompts for each relation
Synonym: | Do congeal and thicken have a similar meaning?
Necessary quality: | Must a rope be bendy?
Associated quality: | Can a mug be mined?
Has part: | Is a core a part of an apple?
Order of intensity: | Is coal shinier than aluminium foil?
Order of size: | Is a country longer than a town?
Cause and effect: | Will a snow flake cause a flood?
Capable of: | Can a helmet be driven?
Default inheritance: | Did Napoleon have a mouth?
Precondition: | Is a sunrise possible only after seeing a cloud?
Boundary (bona fide): | Is a trouser waist belt joined to a leg opening?
Antonym: | Does pursue have an opposite meaning to retreat?
Contained in: | Can a credit card be contained in a wool fibre?
Has member: | Is haiku a type of art?
Troponym: | Is crushing a type of breaking?
Boundary (fiat): | Is Iran joined to Pakistan?
|  |  |  |  |  |  |  |

## Analogical Reasoning Tests A & B

Drawing on objective reality we craft a high-quality physical common sense
benchmark that is balanced and stratified for its relation types with
verifiable common knowledge.

#### Two Tasks.

Due to its importance in everyday reasoning we expect analogies to be one of
the central mechanism for applying core physical common sense to new
situations (?, ?, ?). Two experimental tasks were employed to establish norms:
inductive reasoning with uncued cross-domain analogies (Section A: 48 unique
questions), and analogy relations (Section B: 320 unique questions) that
underlie the analogies. The questions were designed using expert
introspection, collectively agreed after independent answering by the co-
authors, and Google checking. Both ART A and B had an equal balance of the 16
semantic relations (described in Table 1, lower part) that we discuss below.
Overall, approximately half the questions in each analogy and each analogy
relation were judged by the paper authors to be Yes (True) and half were
judged to be No (False).333ART A: 25 Yes, 23 No with 8 joint resolutions; ART
B: 154 Yes, 166 No with 52 joint resolutions. A simple and consistent syntax
was chosen for all items within a semantic relation.

Consider this example of an analogy question: Do congeal and thicken have a
similar relationship to whisper and murmur? The expected answer would be Yes
(True). Analogical reasoning (?, ?, ?) requires participants to grasp that the
source entities (e.g. ‘congeal’ and ‘thicken’) are related by a specific
relationship (e.g. synonymy) based on their shared attributes. Participants
then need to map this source analog to a target domain. In the case of ‘False’
questions, the second pair of words would be related by a different semantic
relation from within the set of 16 types. The following is an example of an
analogy relation question: Is coal shinier than aluminium foil? The expected
answer would be No (False). In terms of intensity, ‘coal’ is less shiny than
‘aluminium foil’. For analogy relation questions, questions judged True and
False by ourselves were considered to represent expert truth and falsehood
about objective reality, respectively.

#### Semantic Relations.

We now turn to consider an inventory of physical common sense relationships
that we expect to be present within the core of common sense reasoning.

The strength and stability of norms within this inventory will be tested by
our human subject study. As reported in Table 1 (lower part), the semantic
relations are all general purpose, can be applied across domains, and various
subsets have been widely used in prior classification schemes (?, ?, ?). We
leave to future work, the intra-rater stability of the scores over time, as
well as a test over a wider demographic. Each of the 16 relations is measured
by 20 questions which require a Yes/No answer. The choice of questions is
based on an intuition that each of the semantic relations contributes to an
understanding of the physical world. Each question was broadly intended to
surprise readers so that answering would engage both knowledge and processes
beyond association and rote learning. For example, comparing two objects for
size at vastly different scales or in highly unlikely domains, or considering
whether possible attributes such as colour were necessary attributes of an
object. Questions were designed also to contain misleading cues, for example
asking about whether sibling concepts are part of each other. In this respect
the questions try to create a mildly hostile environment although not one that
is actively informed by adversarial testing on any model. Note that analogies
were not restricted by domain, e.g. Do jeans and cotton have a similar
relationship to tree and leaf?

#### Web Checking.

We eliminated candidates with exact or trivial variations of the question and
answer on the Web, arriving at the final set of 48 ART A and 320 ART B
questions.

#### Equivalence and Gold Standard.

To obtain an objective benchmark, a panel of three experts (co-authors)
independently annotated the tests with a Fleiss’s $\kappa$ of 0.75 (8 item
disagreements giving moderate-to-substantial agreement with 95% CI) on Section
A and 0.77 (52 item disagreements giving substantial agreement with 95% CI) on
Section B. Sources of difference were resolved by discussion to produce a gold
standard by making minor modifications to the test question or by agreeing a
single interpretation. All disagreements could be resolved. Disagreements were
due to word sense understanding, e.g. impure as mixed with foreign matter vs.
morally corrupt, semantic errors in the question, e.g. a comparison to deck
vs. keel where spatial orientation was important, and the exact degree of
probability implied by modal verbs such as will X cause Y, ranging from
tentative to required.

## Methods

Participants. 63 participants were recruited from Prolific Academic with at
least a master degree who were native English speakers resident in the
UK.444Ethical approval was granted by University of Cambridge’s Modern and
Medieval Languages and Linguistics Ethics committee on condition of informed
consent, anonymisation, and secured storage. Two participants were excluded
due to non-completion. Hence 61 participants, aged 22 to 68 (mean = 34.4, SD =
10.4) where 13 identified their gender as male and 48 as female, were our
subject group. The participants were given all questions to complete in one
hour and ten minutes, with a recommendation to take a rest every 40 to 50
questions. Subjects were free to go over the guide time up to a limit of three
hours. Questions were presented in random order. For each test we provided
four complete and three warm up examples. In practice all subjects exceeded
70% without being asked to re-do the test. The participants were asked to not
use other sources such as the Web.

LLMs. We used five variants of GPT-3 with different parameter sizes, Ada
(350M), Babbage (1.3B), Curie (6.7B), text-Davinci-002 (175B), code-
Davinci-002 (175B) and ChatGPT (175B). For all GPT-3 models, to ensure all
models’ performance reflects their parameter sizes, and not their ability to
follow the instructions, we used the conditional probability of “Yes” and “No”
tokens given a prompt to select model’s response. The temperature parameter
for models was set to 0 in all experiments, switching off stochastic behavior.
The prompts are either Direct questions, e.g. _Does a missing wheel stop a car
from running?_ or Categorical questions including the semantic relations, e.g.
_In terms of order of size, is a country longer than a town?_ For the code-
Davinci-002 and ChatGPT models, we also experimented with Chain-of-Thoughts
(?, ?, ?) prompting (denoted as CoT). Default prompting style is asking Direct
questions unless stated otherwise.

## Results

### Human Variation on the ART A&B555We expect human variation due to
performance errors (?, ?) or algorithmic differences (?, ?).

Ceiling/Floor Effect. We adopted the common threshold to test for ceiling and
floor effects. These were considered present if 15% of the human subjects
achieved the worst (chance level) or the best score on ART A or B as a whole.
A secondary objective was to test for ceiling/floor effects on any subset of
ART B. No ceiling or floor effect was observed for either of the tests as a
whole. However a floor effect was observed for Has part and a ceiling effect
was observed for Order of intensity, Order of size and Cause and effect.

Internal Consistency. Since our test items are dichotomous with varying levels
of difficulty, reliability was measured using the Kuder-Richardson Formula 20
(KR20) on the collectively agreed gold answers. The value for ART A for all
questions was 0.72 (moderate/high), and for ART B was 0.87 (high) indicating
the adequacy of the tests.

Establishing Levels of Agreement. Table 2 shows human subject performance
against the gold standard. The 61 participants scored a mean of 36.3 correct
answers (76%) on ART A and 260.5 (81%) on ART B. We further examined our data
and noted cases where a large majority ($\geq 80\%$) disagreed with the gold
standard (2 on ART A and 7 on ART B).

Relationship between Part A and B. We used the Jarque-Bera non-normality test
taking account of both skewness and kurtosis to determine with $p<0.05$ that
human subject scores on both ART A&B were not normally distributed. On
inspection both had heavy left tails (Skewness: ART A -0.89, ART B -1.32) and
ART B was leptokurtik (Kurtosis: ART A 0.25, ART B 3.60). We therefore
employed Spearman rank correlation coefficient and found moderate evidence
($\rho=0.46$) for dependence between the rankings in performance.

Table 2: Human performance on the ART A&B. AG@N% is the percentage of
questions on which a $N\%$ of participants agreed. Maximum agreement is 48 for
the part A, 320 for part B, and 20 for each semantic relation. 1 shows the
range of correct answers that are possible for individual participants where
the gold standard is the expert committee; 2 shows the range of correct
answers for individual participants where the gold standard is the expert
committee; 3,4 Show the mean and standard deviation over all participants; 5
C/F@15 reports the ceiling or floor effect, which is reported as No if less
than 15% of test takers (9 out of 61) obtained the maximum or minimum possible
scores, otherwise Yes; 6 A composite score across all the semantic relations.

|  Missing |  Potential |  Observed |  M3 |  SD4 |  C/F5 |  AG |  AG
---|---|---|---|---|---|---|---|---
Measure |  % |  range1 |  range2 |  |  |  @15% |  @80% |  @90%
Analogy Test |  0% |  0–48 |  22–43 |  36.3 |  4.6 |  N/N |  26 |  14
Analogy Relation Test6 |  0% |  0–320 |  199–285 |  260.5 |  15.5 |  N/N |  228 |  183
1.Synonym |  0% |  0–20 |  11–20 |  16.4 |  2.0 |  N/N |  14 |  11
2.Necessary quality |  0% |  0–20 |  12–19 |  15.7 |  1.7 |  N/N |  12 |  10
3.Associated quality |  0% |  0–20 |  10–18 |  14.8 |  1.6 |  N/N |  15 |  10
4.Has part |  0% |  0–20 |  10–19 |  15.5 |  1.6 |  Y/N |  15 |  12
5.Order of intensity |  0% |  0–20 |  12–20 |  18.7 |  1.5 |  N/Y |  19 |  16
6.Order of size |  0% |  0–20 |  14–20 |  18.8 |  1.4 |  N/Y |  19 |  18
7.Cause and effect |  0% |  0–20 |  13–20 |  18.0 |  1.6 |  N/Y |  16 |  13
8.Capable of |  0% |  0–20 |  12–20 |  16.2 |  1.8 |  N/N |  12 |  12
9.Default inheritance |  0% |  0–20 |  12–19 |  16.7 |  1.5 |  N/N |  16 |  14
10.Precondition |  0% |  0–20 |  11–20 |  17.0 |  1.7 |  N/N |  15 |  11
11.Boundary (bona fide) |  0% |  0–20 |  9–19 |  14.6 |  2.1 |  N/N |  13 |  6
12.Antonym |  0% |  0–20 |  9–18 |  15.2 |  2.1 |  N/N |  12 |  6
13.Contained in |  0% |  0–20 |  14–20 |  17.6 |  1.4 |  N/N |  15 |  14
14.Has member |  0% |  0–20 |  10–19 |  15.2 |  2.0 |  N/N |  12 |  11
15.Troponym |  0% |  0–20 |  14–20 |  17.4 |  1.4 |  N/N |  16 |  14
16.Boundary (fiat) |  0% |  0–20 |  7–17 |  13.0 |  2.1 |  N/N |  7 |  5

### Identifying a Common Sense Subset of ART A&B

We cannot simply assume that different people perceive all the relationships
between concepts in our test set in the same way. We therefore try to
establish the common sense norms of naïve physics by establishing the set of
questions on which a majority of participants agreed. In the common sense
subset, the gold answers are the majority answers chosen by the human
subjects. Table 2 shows the number of questions on which participants agreed,
both as a composite and grouped by individual semantic relations.
Agreement@80%, the value we choose for our common sense subset, shows the
number of questions on which 80% or more of participants agreed. After
filtering for 80% agreement threshold, 26 out of 48 ART A questions (54%)
remained and 228 out of 320 ART B questions (71%) remained for evaluating
LLMs.

### LLMs vs. the Full Set of ART A&B

Having established an approximation of a human norm we compare LLMs to human
performance. Since LLMs do not exhibit random performance errors due to the
environment, we consider that any difference between the LLM and the human
subject reference interval exhibits a potential divergence in rationality.
Table 3 highlights both alignment and divergence on naïve physical common
sense understanding between LLMs and human subjects on ART A&B.

LLMs on ART A. Babbage, code-Davinci-002 and ChatGPT had composite scores that
were within the lower part of the reference range, whilst Ada, Curie and text-
Davinci-002 fell outside the reference range. Performing chain-of-thought
(CoT) prompting with categories (Categorical) increased performance, bringing
small improvements to code-Davinci-002 and ChatGPT.Despite the difference in
question formulation, these findings are consistent with what was observed in
the zero-shot SAT Analogy experiments of (?, ?).

LLMs on ART B. Ada, Babbage, Currie and text-Davinci-002 composite scores for
ART B fell under the reference intervals. For text-Davinci-002 the composite
score is approaching human levels but this result obscures an uneven
understanding across categories. For text-Davinci-002, Order of intensity,
Order of size, Precondition, Necessary quality, and Fiat boundary show marked
divergence to the reference intervals. This outcome supports the view of
Aroca-Ouellette et al. (?, ?) on GPT-1 and -2 that such models are challenged
on object attributes (e.g. height) and affordances (e.g. breakability). Moving
to ChatGPT we observed that performance has improved markedly for Necessary
quality, Associated quality, Order of intensity and Precondition. We speculate
that this is a result of the combination of careful selection of training data
(e.g. instructional texts) as well as reinforcement learning. A group of
ChatGPT tests showed quantitative performance at the upper end of the
reference interval on Necessary quality, Associated quality, Precondition,
Troponym and somewhat surprisingly Cause and effect. This result provides a
challenge to the view that only rationalistic (i.e. logic based) language
models can be successful in answering common sense questions correctly. Yet we
should be cautious in accepting this result at its face value. As we discuss
later, models draw on a wide range of associations that sometimes allows them
to display seemingly correct reasoning. Such effects however are hard to
isolate.

Table 3: LLM performance on the full set and common sense subet of ART A&B
where green shows performance within the reference range and pink outside the
reference range. 1 Shows the potential range of correct answers that are
possible for individual participants where the gold standard is the expert
committee; 2 Shows the reference interval calculated as the range within which
95% of the human subject reference population values fall. + Later we run
another test on the sensitivity of the results on the order of arguments in
nonsymmetric relations by reversing the order of arguments, and observe the
inability of the model to correctly answer _Cause and Effect_ and _Troponym_
questions.

| Full ART A&B | Common Sense Subset of ART A&B
---|---|---
| Range | GPT-3 | code-davinci+CoT | ChatGPT | Range | GPT-3 | code-davinci+CoT | ChatGPT
Measure |  Potential1 |  Reference2 |  Ada |  Babbage |  Currie |  text-davinci |  Direct |  Categorical |  Direct |  Categorical |  Potential1 | Reference2 |  Ada |  Babbage |  Currie |  text-davinci |  Direct |  Categorical |  Direct |  Categorical
ART Section A |  0–48 |  28–43 |  22 |  32 |  24 |  27 |  28 |  32 |  30 |  32 |  0–26 | 18–25 |  9 |  21 |  19 |  15 |  17 |  16 |  18 |  19
ART Section B |  0–320 |  231–285 |  168 |  169 |  177 |  221 |  238 |  241 |  250 |  255 |  0–228 | 192–220 |  148 |  135 |  137 |  180 |  191 |  191 |  203 |  202
Synonym |  0–20 |  13–20 |  11 |  10 |  12 |  17 |  16 |  17 |  17 |  17 |  0–14 | 12–14 |  10 |  10 |  8 |  12 |  14 |  14 |  13 |  13
Necessary quality |  0–20 |  13–18 |  12 |  13 |  13 |  11 |  13 |  14 |  18 |  19 |  0–12 | 11–12 |  9 |  7 |  8 |  9 |  10 |  10 |  11 |  11
Associated quality |  0–20 |  12–17 |  8 |  7 |  8 |  15 |  13 |  12 |  14 |  16 |  0–15 | 10–14 |  10 |  6 |  4 |  14 |  11 |  10 |  13 |  13
Has part |  0–20 |  13–18 |  11 |  11 |  11 |  13 |  15 |  12 |  14 |  14 |  0–15 | 12–15 |  11 |  12 |  12 |  13 |  13 |  12 |  13 |  13
Order of intensity |  0–20 |  16–20 |  10 |  12 |  10 |  12 |  16 |  15 |  19 |  17 |  0–19 | 16–19 |  10 |  11 |  9 |  11 |  15 |  14 |  18 |  16
Order of size |  0–20 |  17–20 |  10 |  9 |  10 |  11 |  13 |  14 |  13 |  16 |  0–19 | 16–19 |  10 |  9 |  10 |  11 |  13 |  14 |  13 |  16
Cause and effect |  0–20 |  15–20 |  9 |  12 |  10 |  $18^{+}$ |  $18^{+}$ |  $18^{+}$ |  $19^{+}$ |  $18^{+}$ |  0–16 | 14–16 |  8 |  10 |  10 |  15 |  15 |  15 |  16 |  15
Capable of |  0–20 |  13–19 |  12 |  11 |  9 |  17 |  14 |  13 |  15 |  15 |  0–12 | 11–12 |  10 |  3 |  4 |  11 |  11 |  11 |  11 |  11
Default inheritance |  0–20 |  14–19 |  10 |  11 |  13 |  17 |  16 |  17 |  18 |  17 |  0–16 | 13–16 |  10 |  11 |  13 |  16 |  14 |  15 |  15 |  14
Precondition |  0–20 |  14–20 |  11 |  8 |  11 |  10 |  18 |  18 |  16 |  19 |  0–15 | 13–15 |  11 |  8 |  11 |  10 |  14 |  13 |  13 |  14
Boundary (bona fide) |  0–20 |  11–17 |  11 |  14 |  12 |  12 |  11 |  16 |  14 |  16 |  0–13 | 9–12 |  8 |  8 |  8 |  10 |  9 |  11 |  10 |  10
Antonym |  0–20 |  12–18 |  10 |  5 |  9 |  14 |  14 |  14 |  16 |  16 |  0–12 | 8–12 |  7 |  3 |  5 |  9 |  8 |  9 |  10 |  11
Contained in |  0–20 |  15–20 |  10 |  14 |  13 |  13 |  14 |  13 |  15 |  14 |  0–15 | 13–15 |  9 |  12 |  11 |  10 |  11 |  10 |  13 |  11
Has member |  0–20 |  12–19 |  11 |  11 |  14 |  16 |  17 |  17 |  15 |  14 |  0–12 | 10–12 |  10 |  11 |  10 |  12 |  11 |  11 |  11 |  11
Troponym |  0–20 |  15–20 |  11 |  10 |  13 |  $16^{+}$ |  $17^{+}$ |  $18^{+}$ |  $17^{+}$ |  $18^{+}$ |  0–16 | 14–16 |  11 |  10 |  12 |  13 |  16 |  16 |  16 |  16
Boundary (fiat) |  0–20 |  10–17 |  11 |  11 |  9 |  9 |  13 |  13 |  10 |  9 |  0–7 | 5–6 |  4 |  4 |  2 |  4 |  5 |  5 |  6 |  6

### LLMs vs. the Common Sense Subset of ART A&B

The Table 3 (right block) shows the LLM performance on the common sense subset
of ART A&B. Reference ranges have unsurprisingly narrowed compared to the full
tests, and unfortunately for two relations (Antonym and Fiat Boundary) the
lower reference bound falls very close to the random baseline. As shown by the
upper value of the potential range, several relations appear to have markedly
strong consensus at $\geq 80\%$ agreement among human subjects including Order
of intensity, Order of size, Cause and effect, Precondition, Contained in and
Troponym. On the other hand Capable of, Bona fide boundary and Antonym had
fewer consensus items at 80% and Fiat boundary had the fewest consensus items
indicating how disordered the judgements were in this category. The common
sense subset favoured ChatGPT over other models showing that a combination of
reinforcement learning and data selection led to performance on a par with
humans in many categories. The subset revealed churn on a category by category
basis. Across all semantic relations Curie dropped out of three reference
ranges and gained two, Babbage dropped out of three and gained two, Ada
dropped two and gained two. Percentage improvements on ART B reveal modest
gains for all models on the common sense subset (Ada +12%, Babbage +6%, Curie
+5%, text-Davinci-002 +10%, code-Davinci-002 +9%, ChatGPT +11%).

## Discussion

Our investigation has drawn attention to several algorithmic deficiencies in
the LLMs to reason about objects in the physical world. We highlight a few
that warrant more investigation:

Mereo-Topology. Humans understand objects at various granular levels. Although
we cannot be entirely sure of ChatGPT’s algorithm, ART seems to demonstrate
that GPT-3 using text alone without reinforcement learning from human
judgements performs at the lower end or outside the reference interval for Has
part and Contained in. We tend to discount the result for Boundary (fiat) due
to the low level of agreement among human subjects. Interestingly, GPT-3
performance on demarcating boundaries appears less variant to model size,
indicating that a combination of more data with ever larger numbers of model
parameters is not a viable pathway to improvements.

Affordance Characteristics.

Object properties seem to be understood by humans in terms of their affordance
characteristics, or in other words the opportunities and limitations that they
offer to humans (?, ?, ?). ART has shown up weaknesses in GPT-3 models to
understand affordance characteristics through Necessary quality, Order of size
and Order of intensity. Severe performance limitations on Order of size appear
consistent with previous reports (?, ?), and supports the view that there is
an algorithmic weakness in the ability of language-only distributed semantic
representations to capture affordance characteristics (?, ?, ?). Our finding
is particularly salient in the light of strong human performance and low human
variance for Order of size and Order of intensity. ChatGPT’s progress on
Necessary and Associated quality shows how reinforcement learning has been
able to skillfully leverage human judgements to compensate for weaknesses in
the data.

In other areas of object understanding GPT-3 appears to offer evidence for
verbal reasoning on a par with our human subjects. For example in Synonym,
Antonym, Default inheritance, and intriguingly, Cause and effect. We take this
finding not as a conclusion but as the next starting point for further
investigation.

Stability on Nonsymmetric Semantic Relations. In order to test whether the
responses arise from associative fortuity (?, ?) in the data we tested Davinci
and ChatGPT sensitivity to argument reversal for nonsymmetric relations (e.g.
Original: Does a missing wheel stop a car from running; Modified: Does a car
stop a missing wheel from running?) and seeing if this changed Davinci’s
response. Results of this on 9 semantic relations with nonsymmetric items
(i.e., _Cause and effect_ , _Order of intensity_ , _Order of size_ , _Has
part_ , _Default inheritance_ , _Precondition_ , _Contained in_ , _Has member_
, _Troponym_) indicated that both Davinci and ChatGPT responses are stable
under 7 relations but exhibit potential signs of associative behavior for
_Cause and effect_ and _Troponym_ with GPT-3 and ChatGPT failing (e.g.,
replying yes for both of the examples provided above) on 70% and 50% of the
cases, respectively. Combining LLMs with symbolic (?, ?) or physical world (?,
?) grounding are promising directions to address these side effects of
associative learning.

## Conclusion

This work builds on foundations that are already firmly established in
different fields: the importance of lexical semantic relationships for
language processing (?, ?), the importance of conceptual modeling based on
basic ontological principles (?, ?), and prior evidence that establishes the
characteristics and norms of human reasoning (?, ?).

We have questioned therefore the strong empirical assumption of much prior
work in NLP about the ability of LLMs to represent physical common sense using
language data alone (?, ?) and identified some promising areas for algorithmic
investigation and theory formation. Going forward we hope that the discussion
in this paper will be a useful adjunct for intrinsic evaluation of natural
language generation models and further to LLMs that aim to combine machine
learning from text with nature-inspired cognitive capabilities.

## References

  * Ahn et al.Ahn et al. Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., … others  (2022).  Do as i can, not as i say: Grounding language in robotic affordances.  _arXiv preprint arXiv:2204.01691_.
  * Anderson, Fernando, Johnson, GouldAnderson et al. Anderson, P., Fernando, B., Johnson, M., Gould, S.  (2016).  Spice: Semantic propositional image caption evaluation.  In _European conference on computer vision_ (pp. 382–398).
  * Angeli ManningAngeli Manning Angeli, G., Manning, C. D.  (2014).  Naturalli: Natural logic inference for common sense reasoning.  In _Proceedings of the 2014 conference on empirical methods in natural language processing (emnlp)_ (pp. 534–545).
  * Aroca-Ouellette, Paik, Roncone, KannAroca-Ouellette et al. Aroca-Ouellette, S., Paik, C., Roncone, A., Kann, K.  (2021).  Prost: Physical reasoning about objects through space and time.  In _Findings of the association for computational linguistics: Acl-ijcnlp 2021_ (pp. 4597–4608).
  * Bejar, Chaffin, EmbretsonBejar et al. Bejar, I. I., Chaffin, R., Embretson, S.  (2012).  _Cognitive and psychometric analysis of analogical problem solving_.  Springer Science & Business Media.
  * Brachman LevesqueBrachman Levesque Brachman, R. J., Levesque, H. J.  (2021).  Toward a new science of common sense.  _arXiv preprint arXiv:2112.12754_.
  * Brown et al.Brown et al. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., … others  (2020).  Language models are few-shot learners.  _arXiv preprint arXiv:2005.14165_.
  * CherniakCherniak Cherniak, C.  (1990).  _Minimal rationality_.  MIT Press.
  * Chomsky et al.Chomsky et al. Chomsky, N., et al. (2006).  _Language and mind_.  Cambridge University Press.
  * Crosby et al.Crosby et al. Crosby, M., Beyret, B., Shanahan, M., Hernández-Orallo, J., Cheke, L., Halina, M.  (2020).  The animal-AI testbed and competition.  In _Neurips 2019 competition and demonstration track_ (pp. 164–176).
  * Da KasaiDa Kasai Da, J., Kasai, J.  (2019).  Cracking the contextual commonsense code: Understanding commonsense reasoning aptitude of deep contextual representations.  In _Proceedings of the first workshop on commonsense inference in natural language processing_ (pp. 1–12).
  * Davis MarcusDavis Marcus Davis, E., Marcus, G.  (2015).  Commonsense reasoning and commonsense knowledge in artificial intelligence.  _Communications of the ACM_ , _58_(9), 92–103.
  * Devitt SterelnyDevitt Sterelny Devitt, M., Sterelny, K.  (1999).  _Language and reality: An introduction to the philosophy of language_.  MIT Press.
  * Forbes, Holtzman, ChoiForbes et al. Forbes, M., Holtzman, A., Choi, Y.  (2019).  Do neural language representations learn physical commonsense?  _arXiv preprint arXiv:1908.02899_.
  * Gatt KrahmerGatt Krahmer Gatt, A., Krahmer, E.  (2018).  Survey of the state of the art in natural language generation: Core tasks, applications and evaluation.  _Journal of Artificial Intelligence Research_ , _61_ , 65–170.
  * Gentner, Holyoak, KokinovGentner et al. Gentner, D., Holyoak, K. J., Kokinov, B. N.  (2001).  _The analogical mind: Perspectives from cognitive science_.  MIT press.
  * GibsonGibson Gibson, J. J.  (1979).  _The ecological approach to visual perception_.  Houghton, Mifflin and Company.
  * Glenberg RobertsonGlenberg Robertson Glenberg, A. M., Robertson, D. A.  (2000).  Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning.  _Journal of memory and language_ , _43_ , 379–401.
  * HallHall Hall, R. P.  (1989).  Computational approaches to analogical reasoning: A comparative analysis.  _Artificial intelligence_ , _39_(1), 39–120.
  * Hassabis, Kumaran, Summerfield, BotvinickHassabis et al. Hassabis, D., Kumaran, D., Summerfield, C., Botvinick, M.  (2017).  Neuroscience-inspired artificial intelligence.  _Neuron_ , _95_(2), 245–258.
  * Hofstadter SanderHofstadter Sander Hofstadter, D. R., Sander, E.  (2013).  _Surfaces and essences: Analogy as the fuel and fire of thinking_.  Basic books: New York.
  * House KeeleyHouse Keeley House, J. D., Keeley, E. J.  (1996).  Differential prediction of adult student performance from miller analogies test scores.  _The Journal of genetic psychology_ , _157_(4), 501–503.
  * Huang, Le Bras, Bhagavatula, ChoiHuang et al. Huang, L., Le Bras, R., Bhagavatula, C., Choi, Y.  (2019).  Cosmos QA: Machine reading comprehension with contextual commonsense reasoning.  In _Proceedings of the 2019 conference on empirical methods in natural language processing (emnlp)_ (pp. 2391–2401).
  * HumphreyHumphrey Humphrey, N.  (2007).  The society of selves.  _Philosophical transactions of the royal society b: Biological sciences_ , _362_(1480), 745–754.
  * Jones et al.Jones et al. Jones, C. R., Chang, T. A., Coulson, S., Michaelov, J. A., Trott, S., Bergen, B.  (2022).  Distrubutional semantics still can’t account for affordances.  In _Proceedings of the annual meeting of the cognitive science society_ (Vol. 44).
  * Kaplan et al.Kaplan et al. Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., … Amodei, D.  (2020).  Scaling laws for neural language models.  _arXiv preprint arXiv:2001.08361_.
  * Kocijan, Cretu, Camburu, Yordanov, LukasiewiczKocijan et al. Kocijan, V., Cretu, A.-M., Camburu, O.-M., Yordanov, Y., Lukasiewicz, T.  (2019).  A surprisingly robust trick for the winograd schema challenge.  In _Proceedings of the 57th annual meeting of the association for computational linguistics_ (pp. 4837–4842).
  * Kocijan, Lukasiewicz, Davis, Marcus, MorgensternKocijan et al. Kocijan, V., Lukasiewicz, T., Davis, E., Marcus, G., Morgenstern, L.  (2020).  A review of winograd schema challenge datasets and approaches.  _arXiv preprint arXiv:2004.13831_.
  * Kojima, Gu, Reid, Matsuo, IwasawaKojima et al. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., Iwasawa, Y.  (2022).  Large language models are zero-shot reasoners.  _arXiv preprint arXiv:2205.11916_.
  * Lake, Ullman, Tenenbaum, GershmanLake et al. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., Gershman, S. J.  (2017).  Building machines that learn and think like people.  _Behavioral and brain sciences_ , _40_.
  * Lenat, Prakash, ShepherdLenat et al. Lenat, D., Prakash, M., Shepherd, M.  (1985).  Cyc: Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks.  _AI Magazine_ , _6_(4).
  * Levesque, Davis, MorgensternLevesque et al. Levesque, H., Davis, E., Morgenstern, L.  (2012).  The Winograd schema challenge.  In _Thirteenth international conference on the principles of knowledge representation and reasoning._
  * Lin, Hilton, EvansLin et al. Lin, S., Hilton, J., Evans, O.  (2021).  Truthfulqa: Measuring how models mimic human falsehoods.  _arXiv preprint arXiv:2109.07958_.
  * Mihaylov, Clark, Khot, SabharwalMihaylov et al. Mihaylov, T., Clark, P., Khot, T., Sabharwal, A.  (2018).  Can a suit of armor conduct electricity? a new dataset for open book question answering.  In _Proceedings of the 2018 conference on empirical methods in natural language processing (emnlp)._
  * MillerMiller Miller, G. A.  (1995).  Wordnet: a lexical database for english.  _Communications of the ACM_ , _38_(11), 39–41.
  * Milton SmithMilton Smith Milton, S. K., Smith, B.  (2004).  Top-level ontology: The problem with naturalism.  In _Formal ontology in information systems_ (pp. 85–94).
  * MinskyMinsky Minsky, M.  (1975).  A framework for representing knowledge.  _The Psychology of Computer Vision_.
  * Nye, Tessler, Tenenbaum, LakeNye et al. Nye, M., Tessler, M., Tenenbaum, J., Lake, B. M.  (2021).  Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning.  _Advances in Neural Information Processing Systems_ , _34_ , 25192–25204.
  * PearlPearl Pearl, J.  (2009).  _Causality_.  Cambridge university press.
  * Rae et al.Rae et al. Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., … others  (2021).  Scaling language models: Methods, analysis & insights from training gopher.  _arXiv preprint arXiv:2112.11446_.
  * Roemmele, Bejan, GordonRoemmele et al. Roemmele, M., Bejan, C. A., Gordon, A. S.  (2011).  Choice of plausible alternatives: An evaluation of commonsense causal reasoning.  In _Aaai spring symposium: logical formalizations of commonsense reasoning_ (pp. 90–95).
  * Rogers, Kovaleva, RumshiskyRogers et al. Rogers, A., Kovaleva, O., Rumshisky, A.  (2020).  A primer in bertology: What we know about how bert works.  _Transactions of the Association for Computational Linguistics_ , _8_ , 842–866.
  * RussellRussell Russell, B.  (1914).  _The relation of sense-data to physics_.  Nicola Zanichelli.
  * Sakaguchi, Le Bras, Bhagavatula, ChoiSakaguchi et al. Sakaguchi, K., Le Bras, R., Bhagavatula, C., Choi, Y.  (2020).  Winogrande: An adversarial winograd schema challenge at scale.  In _Proceedings of the aaai conference on artificial intelligence_ (pp. 8732–8740).
  * SchubertSchubert Schubert, L.  (2002).  Can we derive general world knowledge from texts?  In _Proceedings of the second international conference on human language technology research_ (pp. 94–97).
  * SearleSearle Searle, J. R.  (2010).  The basic reality and the human reality.  In D. Franken, A. Karakus, J. G. Michel (Eds.), _John r. searle: Thinking about the real world_ (pp. 19–44).  De Gruyter.
  * Singer, Halldorson, Lear, AndrusiakSinger et al. Singer, M., Halldorson, M., Lear, J. C., Andrusiak, P.  (1992).  Validation of causal bridging inferences in discourse understanding.  _Journal of Memory and Language_ , _31_(4), 507–524.
  * SmithSmith Smith, B.  (1995).  Formal ontology, common sense and cognitive science.  _International journal of human-computer studies_ , _43_(5-6), 641–667.
  * SmithSmith Smith, B.  (2012).  Ontology.  In _The furniture of the world: Essays in ontology and metaphysics_ (pp. 47–68).  Brill.
  * Smith CasatiSmith Casati Smith, B., Casati, R.  (1994).  Naive physics.  _Philosophical psychology_ , _7_(2), 227–247.
  * Smith MarkSmith Mark Smith, B., Mark, D. M.  (2003).  Do mountains exist? towards an ontology of landforms.  _Environment and Planning B: Planning and Design_ , _30_(3), 411–427.
  * SpradleySpradley Spradley, J. P.  (2016).  _Participant observation_.  Waveland Press.
  * Srivastava et al.Srivastava et al. Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., … others  (2022).  Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.  _arXiv preprint arXiv:2206.04615_.
  * Stanovich WestStanovich West Stanovich, K. E., West, R. F.  (1998).  Individual differences in rational thought.  _Journal of experimental psychology: general_ , _127_(2), 161.
  * SternbergSternberg Sternberg, R. J.  (1977).  Component processes in analogical reasoning.  _Psychological review_ , _84_(4), 353.
  * Storks, Gao, ChaiStorks et al. Storks, S., Gao, Q., Chai, J. Y.  (2019).  Recent advances in natural language inference: A survey of benchmarks, resources, and approaches.  _arXiv preprint arXiv:1904.01172_.
  * Talmor, Herzig, Lourie, BerantTalmor et al. Talmor, A., Herzig, J., Lourie, N., Berant, J.  (2019).  Commonsenseqa: A question answering challenge targeting commonsense knowledge.  In _Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: Human language technologies_ (pp. 4149–4158).
  * Talmor et al.Talmor et al. Talmor, A., Yoran, O., Bras, R. L., Bhagavatula, C., Goldberg, Y., Choi, Y., Berant, J.  (2022).  CommonsenseQA 2.0: Exposing the limits of ai through gamification.  _arXiv preprint arXiv:2201.05320_.
  * Tenney, Das, PavlickTenney et al. Tenney, I., Das, D., Pavlick, E.  (2019).  Bert rediscovers the classical nlp pipeline.  In _Proceedings of the 57th annual meeting of the association for computational linguistics_ (pp. 4593–4601).
  * Trichelair, Emami, Trischler, Suleman, CheungTrichelair et al. Trichelair, P., Emami, A., Trischler, A., Suleman, K., Cheung, J. C. K.  (2019).  How reasonable are common-sense reasoning tasks: A case-study on the winograd schema challenge and swag.  In _Proceedings of the 2019 conference on empirical methods in natural language processing (emnlp)_ (pp. 3382–3387).
  * TuringTuring Turing, A. M.  (2009).  Computing machinery and intelligence.  In _Parsing the turing test_ (pp. 23–65).  Springer.
  * Wang et al.Wang et al. Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., … Bowman, S.  (2019).  Superglue: A stickier benchmark for general-purpose language understanding systems.  _Advances in neural information processing systems_ , _32_.
  * Wei et al.Wei et al. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., Zhou, D.  (2022).  Chain of thought prompting elicits reasoning in large language models.  _arXiv preprint arXiv:2201.11903_.
  * Zellers, Bisk, Schwartz, ChoiZellers et al. Zellers, R., Bisk, Y., Schwartz, R., Choi, Y.  (2018).  Swag: A large-scale adversarial dataset for grounded commonsense inference.  _arXiv preprint arXiv:1808.05326_.