# Strong hallucinations from negation and how to fix them

Nicholas Asher
IRIT, CNRS / Toulouse, France
Affiliation / Address line 3
asher@irit
&Swarnadeep Bhar
Université Paul Sabatier / Toulouse, France
<EMAIL_ADDRESS>

###### Abstract

Despite great performance on many tasks, language models (LMs) still struggle
with reasoning, sometimes providing responses that cannot possibly be true
because they stem from logical incoherence. We call such responses strong
hallucinations and prove that they follow from an LM’s computation of its
internal representations for logical operators and outputs from those
representations. Focusing on negation, we provide a novel solution in which
negation is treated not as another element of a latent representation, but as
an operation over an LM’s latent representations that constrains how they may
evolve. We show that our approach improves model performance in cloze
prompting and natural language inference tasks with negation without requiring
training on sparse negative data.

Strong hallucinations from negation and how to fix them

Nicholas Asher IRIT, CNRS / Toulouse, France Affiliation / Address line 3
asher@irit Swarnadeep Bhar Université Paul Sabatier / Toulouse, France
<EMAIL_ADDRESS>

##  1 Introduction

In this paper we examine a reasoning problem for language models that stems
from their representation of the meaning of logical terms. We call this
problem strong hallucination, because it involves faulty representations that
lead LMs into logical incoherence, and hence into error, regardless of the
facts. Concentrating on negation, we extend prior empirical work Kassner and
Schütze (2020); Hosseini et al. (2021); Jang et al. (2022b) to prove that how
an LM computes meaning representations for negation, and how that affects
outputs, leads to logical incoherence.

To eliminate strong hallucinations from negation, we alter both the role of
negation in computing an LM’s output and the nature of that output itself. Our
idea is simple but radical: negation’s latent representation is not treated as
just another token contributing to a latent representation via linear
algebraic operations. Instead, the meaning of negation triggers an algorithm
that uses latent representations of positive content to provide consistent and
appropriate continuations.

Many tasks involve negation in their conception as well, which requires
linking negation in the data to the logic of the task. While hallucinations
are often observed for text generation, strong hallucinations are difficult to
evaluate in an open textual setting. We illustrate our approach with
algorithms for negation in three easily evaluable tasks: yes|no question
answering, masked knowledge retrieval (MKR) and natural language inference
(NLI) using both encoder LMs and LLMs. Our algorithms modify the output
distribution of the original model and continuations of the context. They also
change the meaning of negation from a component of a string with a particular
distribution into a function over sets of strings and distributions, which
avoids the problems we describe for standard treatments of negation in LMs. We
calculate an appropriate continuation of a given context recursively, starting
with positive content and then applying the negation algorithm when needed.

Our hybrid approach yields good performance on MKR and NLI; we increase
accuracy by 13% on one NLI dataset and by 91% on another, by having the LM
infer or train only on positive data, which avoids the problem of finding
sparse negative data. We offer two modified, linguistically validated versions
of Dagan et al. (2005); Hossain et al. (2020)’s datasets with negation and
2306 annotated inferences. Code for our experiments and data are in
https://anonymous.4open.science/r/Negation-6320

Section 2 provides relevant background on hallucinations, distributions and
linguistic meaning. Section 3 proves strong hallucinations follow from basic
assumptions about an LM’s treatment of negation. Section 4 details our
positive proposal, and Section 5 provides experiments with our approach on
Q&A, MKR and NLI. Section 6 discusses related work. We then conclude.

##  2 Background

Strong Hallucinations. A hallucinating agent $A$ sincerely affirms something
that is false and for which $A$ has no reasonable grounds. Strong
hallucinations are hallucinations in which $A$ affirms claims that are
inherently false, false regardless of how the facts of the world are. These
affirmations are logically false; no possible evidence can support them. These
point to inherent problems with an LM’s latent representation of linguistic
meaning, rather than factual problems with sources of information.

As an LM’s affirmations depend on a probability distribution, we link strong
hallucinations to probability distributions. Throughout, $\hat{f}$ represents
an LM. Let $\mu_{\hat{f}}(.|E)$ be the conditional probability distribution of
LM $\hat{f}$ over a set of truth evaluable strings $X$, given available
evidence $E$. $E$ could be a context given to $\hat{f}$, an external
information source, evidence from its pretraining, or grounds it provides when
asked about its predictions. Let $P$ be the objective or actual distribution
over $X$ at the actual world; $P$ is the true distribution over events or
propositions described by strings in $X$. $P_{w}$ is an objective distribution
for a logically possible way $w$ the world could be.

######  Definition 1.

$\hat{f}$ strongly hallucinates that $p$, if $\mu_{\hat{f}}(p|E)\neq
P_{w}(p|E)$ for any possible objective distribution $P_{w}$ and state
$w$.111In the same spirit, we can define standard hallucinations: $\hat{f}$
hallucinates that $p$, if $\mu_{\hat{f}}(p|E)$ is significantly different from
$P(p|E)$ and $\mu_{\hat{f}}(E)$ is significantly different from $P(E)$.

LMs. We assume LMs trained on transformer architectures Vaswani et al. (2017)
over very large corpora using masked or causal language modeling. In decoder
models, the decoder is trained to predict the next token $y_{i}$ given the
context matrix $c_{i}$ and all the previously predicted words
{$y_{1},···,y_{i-1}$} using a conditional distribution
$\mu(y_{i}/y_{1},...,y_{i-1},c_{i})$. In encoder models, $c_{i}$ is typically
the input to an MLP layer using these representations to make predictions, in
our case, of labels or short strings.

$c_{i}$ is the LM’s internal representation at $i$ of all the tokens in its
context assigned to each token $j$ in the context: it is a concatenation
$y_{j}$ of multiple hidden states $z^{h}_{j}$, one for each attention head
such that:

$z^{h}_{j}=\Sigma^{K}_{k=1}\alpha^{h}_{j,k}W^{h}_{V}x_{k}$ (1)

$\alpha_{j,k}^{h}$ are the attention weights providing the import of token
(representation) $x_{k}$ to $x_{j}$ and $W^{h}_{V}$ is the value weight matrix
of head $h$. Note that all tokens play a similar though perhaps task dependent
role given Equation 1.222As distributional semantics dictates Baroni and
Zamparelli (2010). These representations together contain a large amount of
linguistic information Devlin et al. (2019); Liu et al. (2019); Tenney et al.
(2019); Talmor et al. (2020); Zhu and Rudzicz (2020)

This representation enables LMs to learn a function $\sigma:(C\times
X)\rightarrow[0,1]$ that assigns probabilities for string $x\in X\subset
V^{m}$ given context $C\subset V^{n}$ of strings of length $n$ with vocabulary
$V$. For encoder and decoder models and tasks we consider, $m$ = 1 typically.
So we will abstract away from architecture and learning details for our models
(the tasks we examine all fit in this category); they learn a similar function
from contexts to tokens of different kinds. To assign a distribution to novel
strings and strings of different length, generative LMs use conditionalization
as in Equation 2. For string $s=(w_{1},w_{2},...,w_{m+1})$ and context $C$, we
have:

$\mu(s|C)=\mu(w_{1}|C)\times\mu(w_{2}\ |C,w_{1})\times$ (2)

$...\times\mu(w_{m+1})|C,w_{1},...,w_{m})$

Generative LMs but also encoders for certain tasks use this distribution over
strings to select continuations or affirmations given a context $C$. Various
methods ${\cal M}$ (greedy or beam search, or sampling) over the distribution
$\mu(.|C)$ have been proposed to find the maximally plausible continuation
$\phi\in X^{n}$ of $C$ .

######  Definition 2.

$\hat{f}$ affirms $\phi$ given $C$, if $\phi={\cal M}\\{x\in X:\mu(x|C)\\}$

Probability and linguistic meaning According to epistemologists, an agent’s
ideal probability distribution $\kappa$ conditioned on science’s best
confirmed laws $T$ and history $H$ should equal the objective distribution $P$
conditional on $T$ Lewis (1981); Hall (1994). An agent with $\kappa$ conforms
to the full truth about a non deterministic world given best scientific theory
$T$ and as such will avoid all hallucinations in the sense of Definition 1.

There is an important but subtle difference between a LM’s distribution over
strings and an ideal distribution $\kappa$; $\kappa$ is a distribution over
propositions, which, in linguistics, are sets of circumstances of evaluation
or set of worlds $\|\phi\|$ in which a sentence $\phi$ is true Dowty et al.
(1981). The linguistic meanings of linguistic expressions that define logical
operators and quantifiers impose a particular algebraic structure on the set
of propositions; for the propositional connectives $\neg$ (not), $\vee$ (or),
$\wedge$ (and) it is a Boolean algebra $\mathbb{P}$.

Linguists construct modal models of linguistic meaning Blackburn et al. (2001)
that respect $\mathbb{P}$. Modal models consist of a set of points of
evaluation and an interpretation function assigning appropriate meanings to
tokens. Following Leblanc (1979), any probability function $P$ over sentences
of a language that respects $\mathbb{P}$ makes every sentence expressing a
logical truth have probability 1, every inconsistent string have probability 0
and if $\phi$ is a logical consequence of $\psi$ then $P(\phi|\psi)=1$.
Conversely, any model defines such a probability function (for formal details
See Appendix A.1). In addition, every ideal distribution defines a modal
model, and every modal model defines an ideal distribution. (See Appendix A.1)

##  3 LMs, negation and strong hallucinations

Section 2 showed a tight connection between ideal distributions, meanings and
probability axioms. This connection does not hold for an LM’s objective
function.

######  Proposition 1.

Every LM $\hat{f}$ whose out probability distribution $\mu_{\hat{f}}$ is a
probability distribution over $V^{n}$, individual strings of length $n$ and
whose outputs are governed by Definition 2 must strongly hallucinate if
either: (i) $\sum_{x\in V^{n}\setminus\\{\phi,\neg\phi\\}}\mu_{\hat{f}}(x)>0$;
(ii) $\mu_{\hat{f}}$ assigns values to strings in $V^{n}$ that are logical
truths or deductively valid reasoning steps.

Proof: (i). Given the assumptions in (i),
$\mu_{\hat{f}}(\phi)+\mu_{\hat{f}}(\neg\phi)<1$. But for every objective
distribution $P$, $\mu_{\hat{f}}(\phi)+\mu_{\hat{f}}(\neg\phi)=1$. So
$\mu_{\hat{f}}$ diverges from any possible objective distribution. (ii). Any
objective distribution must assign probability 1 to every logically true
sentence $\ell$ of $V^{*}$. fFor each positive clause $Q$ there is such an
$\ell$ of the form Not ($Q$ and Not $Q$). As a distribution over individual
strings, $\mu$ cannot assign 1 to all such $\ell$. Moreover, suppose given
context the model offers a chain of deductive reasoning,
$\phi.\ell.\ell^{\prime}$. But then by the axioms of probability and Equation
2, an objective distribution gives $P(\ell|\phi).P(\phi)=P(\phi)$. So
$P(\ell|\phi)=1$. If $\mu(\ell|\phi)=1$, however, $\mu$ assigns no other
continuation of $\phi$ non $0$ probability. In particular
$\mu(\ell^{\prime}.\ell|\phi)=0$. Thus, $\mu$ diverges from any possible
objective distribution. In either case, by Definition 2, the LM strongly
hallucinates. $\Box$

Our LM is forcibly subject to strong hallucinations under minimal assumptions.
Appendix A.2 shows why LMs cannot respect logical equivalence.

Others have already given empirical evidence that language models have trouble
with negation (e.g., Kassner and Schütze, 2020; Hosseini et al., 2021; Jang et
al., 2022b). But we are the first, we believe, to point out that the
distribution governing an LM’s output is incapable of representing negation
adequately and thus must lead to strong hallucinations. We also note that
Proposition 1 part (ii) applies to logical operators in general.

Betting behavior links pathological probability distributions to irrational
behavior, Ramsey (1931); De Finetti (1937), and our results predict that LMs
should place pathological bets called Dutch Books (Appendix A.4). But most NLP
practitioners won’t care that their LM bets badly. Proponents of LMs should
nevertheless worry. All LMs that generate responses from a distribution over
individual strings derived from masked language or causal language modeling
meet the conditions in Proposition 1. If they produce strings representing
reasoning, their distribution over the steps in the reasoning will diverge
from anY objective distribution and lead to an almost sure expectation of
logical error. (See Appendix A.3).

Our results show we can’t use an LM’s distribution over strings to lead to
assertions reflecting a non hallucinatory understanding of negation or other
logical operators. While LMs may have some grasp of negation in simple
contexts Gubelmann and Handschuh (2022), this does not fix an LM’s underlying,
objective function determining its output, which is the source of the problem
with negation. We turn to fixing that now.

##  4 Fixing an LM’s treatment of negation

Expressions like negation structure meaning recursively by performing a
distinctive operation over the content of tokens in its scope. We use this
idea to define how to treat negation in LMs. We use negation to transform
continuations that are determined by the latent representations of tokens
within negation’s scope into continuations reflecting the effects of negation.
Using the mechanisms of continuation semantics Asher et al. (2017, 2023), the
meaning of negation becomes a function from one set of continuations to
another, making negation operate over propositions, as it should, or in our
framework sets of strings.

To develop our idea about continuations, we will make use of the analysis of
logical operators in dynamic semantics (DS)Kamp (1981); Kamp and Reyle (1993);
van Eijck and Kamp (1997). (DS) understands logical operators as transitions
between contexts, we will make use of their formalism, which is natural since,
as Li et al. (2021) have noted, there is a similarity between DS and the
latent representations of LMs.

Central to DS’s analysis is the notion of an embedding Chang and Keisler
(1973) and a structure $(U,P)$, with $U$ a set of tokens representing objects,
and $P$ a set of properties ascribed to objects in $U$. An embedding of a
structure $(U,P)$ of ${\cal L}$ defines satisfaction or truth relative to a
structure, $(U^{\prime},P^{\prime})$ from another information source ${\cal
V}$.333Our formulation here echoes maps used in multimodal approaches Li et
al. (2019); Lu et al. (2019); Driess et al. (2023); Devillers et al. (2023);
VanRullen and Kanai (2021).

######  Definition 3.

$A$ has an embedding $\mathfrak{b}:U_{A}\rightarrow U_{B}$ in $B$ (written
$A\leq_{\mathfrak{b}}B)$ iff for each property ascription $\phi\in P_{A}$ and
for all $x_{1},...x_{n}\in U_{A}$ if $\phi$ holds of or is satisfied by
$x_{1},...x_{n}$ in $A$ then $\phi$ is satisfied by
$\mathfrak{b}(x_{1}),...\mathfrak{b}(x_{n})$ in $B$.

Kamp and Reyle (1993) uses embeddings to define the meaning of negation.
Negation structurally takes scope over a representation $(U,P)$, and converts
it into a property of tokens in a larger structure; embeddings and their
extensions interpret this structural feature. We write
$(U,P)\preceq(U^{\prime},P^{\prime})$ if $U\subseteq U^{\prime}$ and
$P\subseteq P^{\prime}$. If $f$ embeds $(U,P)$, we say $g$ extends $f$ to an
embedding of $(U^{\prime},P^{\prime})$, if:
$(U,P)\preceq(U^{\prime},P^{\prime})$, $g\supseteq f$ and $Dom(g)=U^{\prime}$
and $g$ embeds $(U^{\prime},P^{\prime})$.

Let
$\overrightarrow{\phi}=(U_{\overrightarrow{\phi}},P_{\overrightarrow{\phi}})$
represent $\phi$ for an LM.

######  Definition 4.

$\overrightarrow{\neg\phi}$ is satisfied relative to some
$\mathfrak{v}\in\mathfrak{V}$ and map $\mathfrak{b}:$ iff there is no
extension $\mathfrak{c}$ of $\mathfrak{b}$ over $U_{\overrightarrow{\phi}}$
such that $\overrightarrow{\phi}\leq_{\mathfrak{c}}\mathfrak{v}$.

###  4.1 Negation as a constraint on continuations

Pure LMs can’t appeal to an external source to characterize negation’s effect
on their internal representations. Our key and novel idea uses continuations
and their semantics Asher et al. (2017, 2023). Negation tells us how the
content in their scope will affect continuations $(V,Q)$ of some context
$(U,P)$, with $(U,P)\preceq(V,Q)$. For example, if an object (or token
representation) $o\in U$ has the property $\neg A$ in $P$, no consistent or
coherent continuation of the list of properties for $o$ in $Q$ should contain
$A$.

Now for an LM, a continuation $y$ with structure
$A_{2}=(U_{A_{2}},P_{A_{2}})$, of a context $A_{1}=(U_{A_{1}},P_{A_{1}})$, has
a certain probability $\alpha$. So we say that $A_{2}$ extends $A_{1}$ iff
$U_{A_{1}}\subseteq U_{A_{2}}$ and $P_{A_{1}}\subseteq P_{A_{2}}$ with
probability $\alpha$—$A_{1}\leq_{\mathfrak{c},\alpha}A_{2}$.

Negation imposes three constraints on coherent or admissable continuations on
a distribution $\mu$.

######  Definition 5.

Let $A_{1}\leq_{\mathfrak{c},\gamma}A_{2}$. $A_{2}$ is an admissible
continuation of $A_{1}$ relative to $\mathfrak{c}$ only if:
(i) If $\\{\neg A_{k}\\}\cup P_{A_{1}}=P_{A_{2}}$ with
$\mu(A_{k}|A_{1})=\alpha$ then $\gamma=1-\alpha$; and $\mu(\phi|\neg
A_{k})=\frac{\mu(\phi)(1-\mu(A_{k}|\phi))}{1-\mu(A_{k})}$;444For details see
Appendix A.4. (ii) if $\gamma=1$, then for any context $C$,
$\mu(A_{1}|C)\leq\mu(A_{2}|C)$; (iii) for any $\mathfrak{c}$, if $A_{1}\preceq
A_{2}$, $A_{2}\leq_{\mathfrak{c},1}A_{2}+A_{1}$.

By treating negation as a function over output distributions, we have changed
the LM meaning for negation. Our view defines negation by the set of
admissible continuations it provides for different input contexts. Definition
5 applies recursively to any formula of the form $\neg(\phi\wedge\neg\phi)$ to
assign it probability 1, regardless of $\phi$. Now distributions behave
properly with respect to negation and positive content. Our interpretation of
the output distribution must also change to become
$\mu^{\prime}:2^{V^{n}}\rightarrow[0,1]$, a distribution over sets of strings,
bringing us closer to ideal distributions.

##  5 Experiments with our approach on Q&A, cloze prompts, NLI and negation

To illustrate our approach from Section 4, we look at three applications with
straightforward evaluations: simple Q&A, masked knowledge retrieval (MKR) and
natural language inference (NLI), each with a different output and input. We
use both encoder LMs and autoregressive LLMs. In each of these tasks $T$, the
task structure interacts with Definition 5 to determine how the probabilities
of future continuations can evolve. We leverage an LM’s training over
plentiful positive information $C$ for $T$ with a set of possible outputs
$\o\in{\cal O}$ and use the distribution $\mu(o|C)$ together with our
Definition 5 to compute $\mu(o^{\prime}|C^{\prime})$ $o^{\prime}\in{\cal
O}^{\prime}$ with negative information $C^{\prime}$ and ${\cal O}^{\prime}$.
Or tasks require generating only strings of one element, but the method we
have just described generalizes to the generation of strings of any length.
The problem lies in the complex evaluation of such strings Li et al. (2023),
which is beyond this paper’s scope.

#### Q&A tasks.

We first looked at the BERT large and basic encoder models BERT on a simple
question answering task. We built a simple, synthetic dataset SYN with Y|N
questions and positive and negative contexts by exploiting schemas like the
following: there was a <col> car (positive context); there was no <col> car
(negative context). “<col>” is a placeholder for a color term. Given such
schemas, we asked, Was there a <col> car? (Appendix B has more schemas).

Given an input <cls> QUESTION <sep> CONTEXT <sep> and a BERT model, we plotted
cosine similarity (cossim) histograms of CLS representations from a positive
context and from its corresponding negative context, before and after fine
tuning using the training regime of Chaturvedi et al. (2024) on the Coqa
dataset Reddy et al. (2019). We used BERT, as its pretraining gives a
meaningful CLS representation Devlin et al. (2019). The cossim values on
pretrained BERT ranged between .986 and 1, showing that pretraining changes
BERT’s representations negligibly in the presence of negation.

After Q&A fine tuning, we got a much lower cossim for the CLS representations
positive and negative contexts, indicating that both BERT models (especially
BERT-large with values between 0,34 and 0,38) learned to differentiate
positive and negative content in CLS tokens. (Histogram plots are in Appendix
C.)

We then tested fine-tuned BERT and Roberta LMs on Q&A with SYN. While the
large models did well, Roberta-basic answered "no" to most questions in the
positive context. But their problems were not with negation. Looking into the
model, our task as a binary classification forced the model to consider only
two mutually exclusive and exhaustive possibilities or continuations, $P$ and
$\neg P$. Our task also involves minimal reasoning; so Proposition 1’s
assumptions are not met in this restricted setting. The model learned what
logic and the probability axioms dictate.

We also used our method $\lambda$ on the Q&A task with SYN. We fine-tuned BERT
and RoBERTa models only on positive contexts in SYN. We accessed the logits
which after softmax gives us a probability distribution over the Y|N classes.
Using the probabilities only over the positive contexts $(Y|?Q,C)$, applying
Definition 5 defines probabilities or outputs for the negative contexts
$(N|?Q,C)$, $(Y|?Q,\neg C)$ and $(N|?Q,\neg C)$. This operation replicated the
perfect results of BERT-large and yielded the best results for the small
models, given their problems on positive contexts.

Model | APSyn | ANSyn | APSyn$\lambda$ | ANSyn$\lambda$
---|---|---|---|---
RoBERTa-L | 100 | 100 | 100 | 100
BERT-L | 100 | 100 | 100 | 100
RoBERTa-B | 53 | 100 | 53 | 53
BERT-B | 57 | 100 | 57 | 57
Table 1: Accuracy on Y|N question task with positive (AP) and negative (AN)
contexts on SYN with simple Fine tuning and our method $\lambda$.

#### Masked Knowledge Retrieval.

In a second experiment, we explored a cloze prompt sentence completion task,
Masked Knowledge Retrieval (MKR), in which the model must compute a sentence
completion and lexical item for mask in context– where the mask is either in a
positive $C$ or negative context $\neg C$. Regardless of the facts, a
prediction for $\neg C(\mathit{<mask>})$ should not match the prediction given
$C(\mathit{<mask>})$; answers preferred in $C$ should be dispreferred in $\neg
C$

We used Kassner and Schütze (2020)’s (KS) and Jang et al. (2022a)’s (JS)
“negated” versions of the LAMA dataset Petroni et al. (2019). We converted the
negative contexts for the MKR task provided in KS and JS into a positive
context $C$, which we need for our approach. KS provided 51 prompt examples,
while JS yielded 2926 examples.

Pretrained RoBERTa-large and BERT-base had significant numbers of examples
with an exact match (EM) for both positive and negative contexts in KS and so
showed an inconsistent treatment of negation. For instance, given positive and
negative contexts, A teacher is most likely teaching at a [mask] and A teacher
is not most likely teaching at a [mask], it returned the completions, A
teacher is most likely teaching at a [school] and A teacher is not most likely
teaching at a [school] (See Appendix E for more examples).

With cloze prompts, an LM originally outputs only the token with the highest
score. But an important feature of negation is that it presupposes a set of
relevant alternatives Rooth (1992). The response The capital of France is not
Marseille to the question What is the capital of France? conveys the
information that while the capital of France is not Marseille it is some other
city or place where people live and work; thus, The capital of France is not
an eggplant while not false is not really a relevant response to the question.

To capture the idea of relevant alternatives of negation we modified the LM’s
output to get its top 5 candidate completions in its distribution. We then
applied Definition 5 by computing the probabilities $p_{i}$ over the top 5
completions $\sigma_{i}$ $1\leq i\leq 5$ of the positive prompt
$C(\mathit{<mask>})$; for the negative contexts $\neg C(\mathit{<mask>})$, we
assigned $1-p_{i}$ to $\sigma_{i}$, reversing the LM’s ranking of positive
completions.

Our method produced $0$ EM on both KS and JS, and all completions were
meaningful on KS and only relatively few were ungrammatical on JS. Table 2
gives the performance of BERT and RoBERTA models on this task out of the box
and with our method.

Model | Dataset | Pretrained |  $\lambda$(EM/NM)
---|---|---|---
RoBERTa-L | KS | 32/51 | 0 / 0
JS | 1038/2926 | 0/6
BERT-B | KS | 26/51 | 0/ 0
JS | 936/2926 | 0/162
Table 2: MKR Accuracy for pretrained only Roberta-large and BERT-base and with
our treatment of negation $\lambda$. Under OB, we have #EM / # examples. For
$\lambda$ we have #EM/ # non meaningful completions (NM).

#### NLI.

In a third, more complex application, we looked at natural language inference
(NLI) with SNLI Bowman et al. (2015) and RTE Dagan et al. (2005) datasets
modified for negation by Hossain et al. (2020). Each contains pairs of
sentences, a context $C$ and a hypothesis $h$ and are labeled either with
entailment ($E$) or non-entailment $\neg E$ (RTE) or entailment ($E$),
contradiction ($Cn$) or neutral ($N$) (SNLI). Hossain et al. (2020) then
negated manually either $C$ or $h$ or both in portions of RTE and SNLI.
Hossain’s datasets, $\neg$RTE and $\neg$SNLI, contain no examples of positive
cases $(C,h)$.

To apply our method of training only on positive information, we reconstructed
212 positive examples (41% of $\neg$RTE) and completed 117 examples of Hossain
et al. (2020)’s $\neg$SNLI with four inferential patterns: $(C,h)$, $(C,\neg
h)$, $(\neg C,h)$ and $(\neg C,\neg h)$. In addition, in $\neg$SNLI, negations
of $(C,h)$ pairs were often wrongly labeled because of non full scope
negation. For instance, there were examples where the original $h$ was of the
form a man is smoking and a context $C$ such that $(C,h)$ was correctly
labeled as $E$, but where with the negative hypothesis $\neg h$ was a man was
not smoking $C,\neg h$ received $Cn$, when $C$ and $\neg h$ were plainly
consistent and $C,\neg h$ should have been labeled $N$. We rendered $h$ and
$\neg h$ true contradictories in many cases, using a definite description the
NP in $h$ to refer to a NP from $C$ or replacing a NP in $h$ with no NP (for
examples see Appendix F).

In addition our data often featured less than full sentential scopes for
negation on $C$, as well as presuppositional elements (definite noun phrases,
adverbial clauses), which scope out of negation. To deal with this, we divided
$C$ into two components $P$ and $C^{\prime}$, with negation having scope only
over $C^{\prime}$. We then added annotated inferential patterns for both
material under the scope of negation and material outside of it. For instance,
if we had the context a man with blue shoes was not sleeping and a hypothesis
$h$, we annotated the inference to h from a man with blue shoes $(P,h)$, and
from $h$ to a man was not sleeping $(h,C\textquoteright)$. Our datasets
containing 2306 annotated inferences are in supplementary material.

NLI asks us to learn relations between two sentences, whose definitions
involve negation. This imposes a logical structure on the labels given their
intuitive meaning; e.g., semantic entailment (E) between $C$ and $h$ means
that there are no situations where $C$ and $\neg h$ hold. From this definition
of entailment and Definition 5 it follows that $\mu(E|C,\neg h)=1-\mu(E|C,h)$.
And so if the model predicts $(C,h)$:$E$, it should predict $(C,\neg h)$:
$\neg E$. To infer labels for $(\neg C,\neg h)$ from positive data, if
$(h,C)$:$\neg E/E$, then $(\neg C,\neg h)$: $\neg E/E$. Similarly for $(\neg
C,h)$ contexts: if $\neg C,\neg h$:$E$, then $(\neg C,h)$:$\neg E$. This
illustrates the interactions between negation in the data and in the task
definition. We note that our rules are provably correct but incomplete; using
only valid rules, we cannot infer $(C,\neg h)$:$E$ from just positive data in
the 2 label $E$ and $\neg E$ task. So our method perforce missed those cases.

To account for less than wide scope negations, we developed an algorithm
inspired by Karttunen and Peters (1979) that determines values for all
configurations based on entailment predictions for $(C,h)$, $(P,h)$ and
$(h,C^{\prime})$. In Algorithms 1 and 2 are the scoped algorithm for $(\neg
C,h)$ and $(\neg C,\neg h)$ in a 2 label NLI problem and scoped $C$; as we did
not scope $h$, only those cases are relevant.

Algorithm 1 Algorithm for $(\neg C,h)$

1:procedure Algorithm$(\neg C,H)$

2: $(C,h):\leftarrow\text{defined in paper}$

3: $(P,h):\leftarrow\text{P defined in paper}$

4: if $(h,C):\textit{$\neg E$}$ then

5: if $(C,h):\textit{E}$ and $(P,h):\textit{E}$ then

6: $(\neg C,h):\textit{E}$

7: else

8: $(\neg C,h):\textit{$\neg E$}$

9: if $(h,C):\textit{E}$ then $(\neg C,h)$: $\neg E$

Algorithm 2 Algorithm for $(\neg C,\neg h)$

1:procedure Algorithm$(\neg C,\neg H)$

2: $(C,h):\leftarrow\text{C,h defined in paper}$

3: $(P,h):\leftarrow\text{P defined in paper}$

4: $(h,C^{\prime}):\leftarrow\text{C' defined in paper}$

5: if $(h,C):\textit{E}$ then

6: if $(P,h):\textit{E}$ then

7: $(\neg C,h):\textit{E}$

8: else

9: $(\neg C,h):\textit{$\neg E$}$

10: if $(h,C):\textit{$\neg E$}$ then

11: if $(h,C^{\prime}):\textit{E}$ then

12: $(\neg C,h):\textit{E}$

The three labels $E$, $Cn$ and $N$ in our$\neg$SNLI also make reference to
logical relations involving negation, relations which $\lambda$ encodes in its
algorithm. For instance, exploiting these relations for $C,\neg h$ contexts
gives: $(C,h)$:$E/Cn/N$ iff $(C,\neg h)$:$Cn/E/N$. Rules for scoped negation
in the 3 label NLI task are in appendix G. For this task, our rules for
negation capture the cases we missed in RTE.

On our $\neg$RTE, the scoped algorithm $\lambda$ yielded a theoretical
maximum, overall accuracy of 94% on all $C$,$h$ configurations, while a basic
algorithm assuming wide scope negation over $C$, $\lambda$ basic, was 11% less
accurate, showing the importance of scoping Kletz et al. (2023). We scored
only (89%) was on $C,\neg h$, because we missed the $(C,\neg h)$:$E$ cases. On
our $\neg$SNLI, $\lambda$ yielded a 96% accuracy overall. For $\lambda$’s
calculations we used gold $(C,h)$, $(P,h)$ and $(h,C^{\prime})$ labels on both
datasets. Table 3 contains scores for the different configurations of $C$, $h$
and negation.

We then prompted a Llama2 7B LLM on the two NLI tasks to compute labels for
positive $(C,h)$, $(P,h)$ and $(h,C^{\prime})$ data to be used with $\lambda$.
Long prompts (in supplementary material) in the chain of thought style did
best for $(C,h)$ Wei et al. (2023); shorter prompts worked better for $(P,h)$
and $(h,C^{\prime})$. We experimented with different formats; the model did
best with a method that involved answering Y|N questions.

On our $\neg$RTE, Llama got 73% correct for $(C,h)$, 76% correct on $(P,h)$
and 83% on $(h,C)$ labels. These predictions combined with our algorithm
(L$\lambda$) achieved an average accuracy of 80% on the various $C$ $h$
configurations with negation; details for $C,\neg h$, $\neg C,h$ and $\neg
C,\neg h$ are in Table 3. With the same complex prompts on $\neg$RTE, Llama
alone had an average accuracy of 71% , and it predicted correctly on 76% of
$(C,\neg h)$ cases, 63% for $(\neg C,h)$ and 70% for $(\neg C,\neg h)$. Still
Llama prompted only on positive data with $\lambda$ significantly outperformed
Llama by itself on this task.

Though it did relatively well on the binary labeling task in RTE, Llama had
more trouble on our $\neg$SNLI, with the three label classification. On
positive data only, Llama predicted 78% correct on $(C,h)$, 69% on $(P,h)$ and
70% on $(h,C^{\prime})$. Combined with $\lambda$, this gave an overall
accuracy of 72%. Llama alone was able to achieve a decent accuracy on $(C,\neg
h)$ (accuracy 71%); but floundered on the $(\neg C,h)$, and $(\neg C,\neg h)$
cases. Llama with $\lambda$ far surpassed what Llama could do on its own. See
Table 3.

Data | Env. | $C,\neg h$ | $\neg C,h$ | $\neg C,\neg h$ | Full
---|---|---|---|---|---
$\neg$RTE |  $\lambda$ basic | .89 | .76 | .91 | .85
$\lambda$ | .89 | .96 | .98 | .94
| L $\lambda(.73)$ | .89 | .76 | .80 | .8
| L (.73) | .76 | .63 | .70 | .71
$\neg$SNLI | $\lambda$ | .96 | .97 | .95 | .96
L $\lambda$ (.78) | .77 | .67 | .67 | .72
L (.78) | .71 | .12 | .11 | .43
Table 3: Accuracy on NLI tasks for $\neg$RTE and $\neg$SNLI datasets. $\lambda
basic$ accuracies for basic algorithm assuming sentence wide scope. $\lambda$:
accuracies for the full algorithm with scoping on the ($\neg$)C,($\neg$)h
configurations, given gold labeled $(C,h)$, $(h,C^{\prime})$, $P,h)$. L: Llama
predictions with best prompts. L$\lambda$: predictions using $\lambda$ given
Llama predictions for $(C,h)$, $(h,C^{\prime})$, $P,h)$.

Our NLI algorithm applies to other tasks involving inference. Consider a Q&A
task where a model is queried with a yes/no question $?\phi$ about information
in a text. A general algorithm for using our method would run as follows.
First we determine whether there are structural negations in the text and
their scope. We then separate out the text as we did for NLI tasks into a part
outside the scope of negation $P$ and a part within $C^{\prime}$. We then
answer yes to $?\phi$ if the model with our method predicts $(T,\phi):E$, no
if $(T,\phi):Cn$ and unknown if $(T,\phi):N$. Our method generalizes to the
semantics of other questions Groenendijk and Stokhof (1984).

##  6 Related work

Ji et al. (2023) defines content as hallucinatory if it is nonsensical or
unfaithful to a provided source. Definition 1 defines strong hallucinations
and shows they involve necessarily unfaithful content. Researchers have
proposed various causes for hallucinations— noisy information sources
Filippova (2020), faulty decoding strategies or imperfect representation
learning Parikh et al. (2020), or knowledge in parameters from pretraining
interfering with information extracted from an explicitly given source Longpre
et al. (2021). As far as we know, we are the first to define and to analyze
strong hallucinations and to formally derive strong hallucinations from an
LM’s representation of negation’s meaning.

Various proposals to avoid hallucinations have also surfaced–exploiting
diverse information sources Nakano et al. (2021); Asher and Hunter (2022),
idealized or specialized training Merrill et al. (2022); Gubelmann and
Handschuh (2022), scaling up to larger LLMs. Scaling has drawn criticims
Filippova (2020); Huang et al. (2021); Li et al. (2023) Detecting
hallucinations and evaluating fixes for them is also challenging Goyal and
Durrett (2020); Sellam et al. (2020); Li et al. (2023). Our method for
correcting strong hallucinations from negation is formally correct; it’s not a
form of training but acctually shifts LM output from its last layer. We have
evaluated our method on tasks where there are clear evaluation metrics, and we
have extended and refined two data sets on NLI from Hossain et al. (2020) to
this end.

Prior research Kassner and Schütze (2020); Hossain et al. (2020); Hosseini et
al. (2021); Jang et al. (2022a) has analyzed the effects of negation MKR and
NLI on LMs for which they reported mediocre results. Proposition 1 provides a
formal foundation for these observations, and our tests on pretrained BERT
models confirms that they doesn’t adequately represent negation. Gubelmann and
Handschuh (2022) adopted a pragmatic approach that improved LM performance on
MKR; our novel, semantic approach gives completely logically coherent MKR
performance. Finally, Truong et al. (2023) investigated GPT style LLMs and
found NLMs unable to reason with negation. Our prompting of Llama2 7B LLM gave
variable to bad results on our NLI datasets. Our method for fixing strong
hallucinations from negation is novel, and it substantially improved Llama’s
NLI performance. The embeddings we use to define our method refine and extend
lexical matching Dhingra et al. (2019) and relation triple extraction
techniques Cao et al. (2018); Huang et al. (2020). They can serve to check
factual hallucinations.

Jang et al. (2022b) improve LM performance on MKR tasks with negation by
training the model on an intermediate task, in which they use a dictionary
corpus to give the model as input a word and a sentence describing it
(dictionary meaning). The model then predicts whether the word is described by
the sentence. They use an encoder with such training on MKR tasks. We believe
their approach profits from the binary classifier task to help understand
negation, as our formal results and Q&A experiments indicate.

##  7 Conclusions

We have investigated strong hallucinations in LMs, originating from faulty
meaning representations, in particular faulty representations of negation. We
proved that hallucinations must result from an LM’s distributions defined over
strings of a finite length. To remedy this we proposed a new treatment of
negation on which negation introduces an operation over latent
representations, imposing constraints on how those representations evolve in
coherent continuations. These constraints trigger algorithms for tasks that
have to integrate the logical structure with the semantics of negation.

We illustrated our approach on Q&A, sentence completion and NLI tasks, and
showed that it can increase LM performance. Notably, our approach only
requires LM training on positive data, which is an advantage, since negative
data is sparse in real corpora. We’ve also supplied two datasets for NLI
inferencing with LLMs that improve on the datasets in Hossain et al. (2020)
and that were the basis of our NLI inferencing experiments.

In the future, we will test our approach with longer string generation. We
will need new, annotated data sets Li et al. (2023), as well as a more
developed algorithm integrating other logical operators to improve and
evaluate multiple, natural language, reasoning steps.

##  8 Limitations

While we have provided empirical support for our view on strong hallucinations
and negation using both LMs like BERT and RoBERTa as well as LLMs like
Llama2-7b, we haven’t linked negation problems directly to full text
generation and long strings. We have concentrated on problems where we are
adding a single token to a string rather than a longer string. Our problems
have simple and precise evaluation metrics, unlike those for text generation.
As we have concentrated on strong hallucinations, the application where strong
hallucinations would be most evident is in generation of reasoning chains. We
don’t see how to do this, however, without having integrated more deductive
inference. In addition, this would take a large scale annotation and data set
creation effort to the best of our knowledge.

$\lambda$ only deals with one logical operator. It doesn’t get perfect scores
on our NLI data because it doesn’t capture the complex interactions between
other logical operators and quantifiers and negation. Our approach is general
enough to incorporate all logical operators that formal, continuation
semantics currently can handle, but working out the details and coding this to
scale remains a complex theoretical and empirical endeavor. Our link from LM
implicit representations to formal semantics is a first step; a full analysis
will require us to investigate empirically and theoretically various issues
essential to inference: the scope of logical operators, the predicate argument
structure of clauses and constraints from other logical operators, which is
problematic for LMs Chaturvedi et al. (2024).

The scope of operators is particularly important as can be seen from our Table
3. We did not train our models to do scoping, in future work we will do this
following Kletz et al. (2023). For our proposal to fully work on NLI data, we
need reliable semantic parsers to get the scopes of negation. This in itself
remains a difficult problem, though preliminary experiments show LLama
relatively capable in providing logical forms of short and relatively simple
context inputs.

Another issue is the weakness of LLMs for instance in dealing with just
positive data in complex tasks like NLI. We need to fine tune a Llama model
rather than simply prompting it, as prompting is not a very robust but rather
unstable process on these LLMs we have found.

To do fine tuning, we will probably also need more and better data. Our
examination of the data on NLI shows that we need a much more comprehensive
and varied dataset. We will continue to enlarge the revised datasets we have
provided here in supplementary material.

## References

  * Asher et al. (2023) Nicholas Asher, Swarnadeep Bhar, Akshay Chaturvedi, Julie Hunter, and Soumya Paul. 2023.  Limits for learning with large language models.  arXiv preprint arXiv:2306.12213.
  * Asher and Hunter (2022) Nicholas Asher and Julie Hunter. 2022.  When learning becomes impossible.  In _2022 ACM Conference on Fairness, Accountability, and Transparency_ , pages 107–116.
  * Asher et al. (2017) Nicholas Asher, Soumya Paul, and Antoine Venant. 2017.  Message exchange games in strategic conversations.  _Journal of Philosophical Logic_ , 46.4:355–404.
  * Baroni and Zamparelli (2010) Marco Baroni and Roberto Zamparelli. 2010.  Nouns are vectors, adjectives are matrices: Representing adjective-noun constructions in semantic space.  In _Proceedings of the 2010 conference on empirical methods in natural language processing_ , pages 1183–1193.
  * Blackburn et al. (2001) Patrick Blackburn, Maarten De Rijke, and Yde Venema. 2001.  _Modal logic, Cambridge Tracts in Theoretical Computer Science No.53_.  Cambridge University Press.
  * Bowman et al. (2015) Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015\.  A large annotated corpus for learning natural language inference.  _arXiv preprint arXiv:1508.05326_.
  * Cao et al. (2018) Ziqiang Cao, Furu Wei, Wenjie Li, and Sujian Li. 2018.  Faithful to the original: Fact aware neural abstractive summarization.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 32.
  * Chang and Keisler (1973) Chen Chung Chang and H Jerome Keisler. 1973.  _Model theory_.  North Holland, Elsevier.
  * Chaturvedi et al. (2024) Akshay Chaturvedi, Swarnadeep Bhar, Soumadeep Saha, Utpal Garain, and Nicholas Asher. 2024.  Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answering.  _Computational Linguistics_ , pages 1–37.
  * Dagan et al. (2005) Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005.  The pascal recognising textual entailment challenge.  In _Machine learning challenges workshop_ , pages 177–190. Springer.
  * De Finetti (1937) Bruno De Finetti. 1937.  La prévision : ses lois logiques, ses sources subjectives.  _Annales de l’institut Henri Poincaré_ , 7:1–68.
  * Devillers et al. (2023) Benjamin Devillers, Léopold Maytié, and Rufin VanRullen. 2023.  Semi-supervised multimodal representation learning through a global workspace.  _arXiv preprint arXiv:2306.15711_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  Bert: Pre-training of deep bidirectional transformers for language understanding.
  * Dhingra et al. (2019) Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming-Wei Chang, Dipanjan Das, and William W Cohen. 2019.  Handling divergent reference texts when evaluating table-to-text generation.  _arXiv preprint arXiv:1906.01081_.
  * Dowty et al. (1981) David R Dowty, Robert Wall, and Stanley Peters. 1981.  _Introduction to Montague semantics_.  Dordrecht.  Synthese Library vol. 11.
  * Driess et al. (2023) Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. 2023\.  Palm-e: An embodied multimodal language model.  _arXiv preprint arXiv:2303.03378_.
  * Filippova (2020) Katja Filippova. 2020.  Controlled hallucinations: Learning to generate faithfully from noisy data.  _arXiv preprint arXiv:2010.05873_.
  * Gaifman (1964) Haim Gaifman. 1964.  Concerning measures on boolean algebras.  _Pacific Journal of Mathematics_ , (14):61–73.
  * Goyal and Durrett (2020) Tanya Goyal and Greg Durrett. 2020.  Evaluating factuality in generation with dependency-level entailment.  _arXiv preprint arXiv:2010.05478_.
  * Groenendijk and Stokhof (1984) Jeroen Groenendijk and Martin Stokhof. 1984.  _Studies on the Semantics of Questions and the Pragmatics of Answers_.  Ph.D. thesis, Centrale Interfaculteit, Amsterdam.
  * Gubelmann and Handschuh (2022) Reto Gubelmann and Siegfried Handschuh. 2022.  Context matters: A pragmatic study of plms’ negation understanding.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 4602–4621.
  * Hall (1994) Ned Hall. 1994.  Correcting the guide to objective chance.  _Mind_ , 103(412):505–517.
  * Hossain et al. (2020) Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, and Eduardo Blanco. 2020.  An analysis of natural language inference benchmarks through the lens of negation.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9106–9118.
  * Hosseini et al. (2021) Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, and Aaron Courville. 2021.  Understanding by understanding not: Modeling negation in language models.  In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1301–1312.
  * Huang et al. (2020) Luyang Huang, Lingfei Wu, and Lu Wang. 2020.  Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward.  _arXiv preprint arXiv:2005.01159_.
  * Huang et al. (2021) Yichong Huang, Xiachong Feng, Xiaocheng Feng, and Bing Qin. 2021.  The factual inconsistency problem in abstractive text summarization: A survey.  _arXiv preprint arXiv:2104.14839_.
  * Jang et al. (2022a) Mj Jang, Frank Mtumbuka, and Thomas Lukasiewicz. 2022a.  Beyond distributional hypothesis: Let language models learn meaning-text correspondence.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 2030–2042.
  * Jang et al. (2022b) Myeongjun Jang, Deuk Sin Kwon, and Thomas Lukasiewicz. 2022b.  Becel: Benchmark for consistency evaluation of language models.  In _Proceedings of the 29th International Conference on Computational Linguistics_ , pages 3680–3696.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12):1–38.
  * Kamp and Reyle (1993) H. Kamp and U. Reyle. 1993.  _From Discourse to Logic: Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory_.  Kluwer Academic Publishers.
  * Kamp (1981) Hans Kamp. 1981.  A theory of truth and semantic representation.  In J. Groenendijk, T. Janssen, and M. Stokhof, editors, _Formal Methods in the Study of Language_ , pages 277–322. Mathematisch Centrum, Amsterdam.
  * Karttunen and Peters (1979) Lauri Karttunen and Stanley Peters. 1979.  Conventional lmplicature.  In _Presupposition_ , pages 1–56. Brill.
  * Kassner and Schütze (2020) Nora Kassner and Hinrich Schütze. 2020.  Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7811–7818.
  * Kletz et al. (2023) David Kletz, Marie Candito, and Pascal Amsili. 2023.  Probing structural constraints of negation in pretrained language models.  In _Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa)_ , pages 541–554.
  * Leblanc (1979) Hugues Leblanc. 1979.  Probabilistic semantics for first-order logic.  _Zeitschrift für mathematische Logik und Grundlagen der Mathematik_ , 25(32):497–509.  Wiley Online Library.
  * Lewis (1981) David Lewis. 1981.  A subjectivist’s guide to objective chance.  In _IFS: Conditionals, Belief, Decision, Chance and Time_ , pages 267–297. Springer.
  * Li et al. (2021) Belinda Z Li, Maxwell Nye, and Jacob Andreas. 2021.  Implicit representations of meaning in neural language models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 1813–1827.
  * Li et al. (2023) Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023.  HaluEval: A large-scale hallucination evaluation benchmark for large language models.  In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing_ , pages 6449–6464, Singapore. Association for Computational Linguistics.
  * Li et al. (2019) Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019\.  Visualbert: A simple and performant baseline for vision and language.  _arXiv preprint arXiv:1908.03557_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Longpre et al. (2021) Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021.  Entity-based knowledge conflicts in question answering.  _arXiv preprint arXiv:2109.05052_.
  * Lu et al. (2019) Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.  _Advances in neural information processing systems_ , 32.
  * Merrill et al. (2022) William Merrill, Alex Warstadt, and Tal Linzen. 2022.  Entailment semantics can be extracted from an ideal language model.  _arXiv preprint arXiv:2209.12407_.
  * Nakano et al. (2021) Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021.  Webgpt: Browser-assisted question-answering with human feedback.  _arXiv preprint arXiv:2112.09332_.
  * Parikh et al. (2020) Ankur P Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020.  Totto: A controlled table-to-text generation dataset.  _arXiv preprint arXiv:2004.14373_.
  * Petroni et al. (2019) F. Petroni, T. Rocktäschel, A. H. Miller, P. Lewis, A. Bakhtin, Y. Wu, and S. Riedel. 2019.  Language models as knowledge bases?  In _In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019_.
  * Ramsey (1931) Frank Plumpton Ramsey. 1931.  _The foundations of mathematics and other logical essays_.  K. Paul, Trench, Trubner & Company, Limited.
  * Reddy et al. (2019) Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019.  CoQA: A Conversational Question Answering Challenge.  _Transactions of the Association for Computational Linguistics_ , 7:249–266.
  * Rooth (1992) M. Rooth. 1992.  A theory of focus interpretation.  _Natural Language Semantics_ , 1(1):75–116.
  * Sellam et al. (2020) Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020.  Bleurt: Learning robust metrics for text generation.  _arXiv preprint arXiv:2004.04696_.
  * Talmor et al. (2020) Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2020.  olmpics-on what language model pre-training captures.  _Transactions of the Association for Computational Linguistics_ , 8:743–758.
  * Tarski (1944) Alfred Tarski. 1944.  The semantic conception of truth: and the foundations of semantics.  _Philosophy and phenomenological research_ , 4(3):341–376.
  * Tenney et al. (2019) Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R Bowman, Dipanjan Das, et al. 2019.  What do you learn from context? probing for sentence structure in contextualized word representations.  In _International Conference on Learning Representations_.
  * Truong et al. (2023) Thinh Hung Truong, Timothy Baldwin, Karin Verspoor, and Trevor Cohn. 2023.  Language models are not naysayers: An analysis of language models on negation benchmarks.  _arXiv preprint arXiv:2306.08189_.
  * van Eijck and Kamp (1997) J. van Eijck and H. Kamp. 1997.  Representing discourse in context.  In Johan van Benthem and Alice ter Meulen, editors, _Handbook of Logic and Linguistics_ , pages 179–237. Elsevier.
  * VanRullen and Kanai (2021) Rufin VanRullen and Ryota Kanai. 2021.  Deep learning and the global workspace theory.  _Trends in Neurosciences_ , 44(9):692–704.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ , 30.
  * Wei et al. (2023) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023.  Chain-of-thought prompting elicits reasoning in large language models.
  * Zhu and Rudzicz (2020) Zining Zhu and Frank Rudzicz. 2020.  An information theoretic view on selecting linguistic probes.  _arXiv preprint arXiv:2009.07364_.

## Appendix

A.1 Language, probabilities and propositions 3 and 4

Let $\mathfrak{A}$ be a modal model Blackburn et al. (2001) with a set of
possible worlds or points of evaluation $W_{\mathfrak{A}}$. For $w\in
W_{\mathfrak{A}}$, we write $\mathfrak{A},w\models\phi$ for the fact that
$\phi$ is true in $\mathfrak{A}$ at $w$, and $\|\phi\|_{\mathfrak{A}}=\\{w\in
W_{\mathfrak{A}}:\mathfrak{A},w\models\phi\\}$.

######  Definition 6.

Let $\phi,\psi\in Sent({\cal L})$. $\phi$ is a semantic or logical consequence
of $\psi$ ($\psi\models\phi$) if for all models $\mathfrak{A}$ and worlds
$w\in W_{\mathfrak{A}}$ such that $\mathfrak{A},w\models\psi$, then
$\mathfrak{A},w\models\phi$ Chang and Keisler (1973).

######  Proposition 2.

Let $P$ be a probability distribution that respects the $\sigma$ algebra
$\mathbb{P}$. Then $P$ determines a probability distribution $\pi$ over
$Sent({\cal L})$ such that the following axioms A hold:
$\pi(\neg\phi)=1-\pi(\phi)$; If $\models\phi$, then $\pi(\phi)=1$; if
$\phi\models\psi$, then $\pi(\phi)\leq\pi(\psi)$; if
$\models\neg(\phi\wedge\psi)$, then $\pi(\phi\vee\psi)=\pi(\phi)+\pi(\psi)$;
$\|\phi\wedge\psi\|=\|\phi\|\cap\|\psi\|$, and $\pi(\forall
x\phi)=lim_{n\rightarrow\infty}\bigwedge_{a_{1},...a_{n}\in C_{\cal
L}}\pi(\phi(\frac{x}{a}))$.

A straightforward consequence Proposition 2 is that $\pi$ must assign the same
probability to sentences $\phi,\psi$ if $\models\phi\leftrightarrow\psi$ (are
logically equivalent).

Finally, we can relate probability distributions $\pi$ to models of linguistic
meaning. Let ${\cal L}$ be a first order language.

######  Proposition 3.

(i) Every probability function $\pi:Sent({\cal L})\rightarrow[0,1]$ that
respects $\mathbb{P}$ defines a modal model $\mathfrak{A}$ of ${\cal L}$ with
worlds $W$ where: if $\phi$ is a logical truth, $\|\phi\|_{\mathfrak{A}}=W$;
if $\phi$ is inconsistent, then $\|\phi\|_{\mathfrak{A}}=\varnothing$; If
$\models\phi\leftrightarrow\psi$, then
$\|\phi\|_{\mathfrak{A}}=\|\psi\|_{\mathfrak{A}}$ and all identities of
$\mathbb{P}$ hold in ${\cal A}$; finally if $\phi\models\psi$,
$\pi(\psi|\phi)=1$. (ii) Every model of ${\cal L}$ defines a probability
function $\pi:Sent({\cal L})\rightarrow[0,1]$ verifying the axioms $A$ of
Proposition 2, and (iii) if $\pi:Sent({\cal L})\rightarrow[0,1]$ does not
verify the axioms $A$, it does not define a model of ${\cal L}$, or preserve
semantic consequence.

Proof of Proposition 3. To prove (i), we consider a set of worlds $W$ where
$\pi$ provides a uniform distribution over $w\in W$. If $\pi(\phi)=\alpha$, we
build a model $\mathfrak{A}$ such that where $\|\phi\|=\\{w\in
W:\mathfrak{A},w\models\phi\\}$, $\sum_{w\in\|\phi\|}\pi(w)=\alpha$. Now
suppose $\phi$ is a logical truth. Then $\pi(\phi)=1$. So
$\sum_{w\in\|\phi\|}\pi(w)=1$. Given that $\pi$ is a probability measure over
$W$, $\|\phi\|=W$. Similarly if $\phi$ is inconsistent, then $\pi(\phi)=0$.
And so $\|\phi\|=\varnothing$. Given the axioms in A, if
$\models\phi\rightarrow\psi$, then $\pi(\phi)\leq\pi(\psi)$; we then set
$\|\phi\|\subseteq\|\psi\|$ in $\mathfrak{A}$. So if
$\models\phi\leftrightarrow\psi$, $\|\phi\|=\|\psi\|$. This means that all the
identities of $\mathbb{P}$ and all logical equivalences hold in
$\mathfrak{A}$. Finally, if $\phi\models\psi$, $\|\phi\wedge\psi\|=\|\psi\|$
given our definition of $\mathfrak{A}$. Then
$\pi(\phi|\psi)=\frac{\pi(||\phi||\
\cap||\psi||)}{\pi(||\psi||)}=\frac{\pi(||\psi||)}{\pi(||\psi||)}=1$.
To prove (ii), assume a modal model $\mathfrak{A}$ and define a set of
measures $\Pi$ on $W_{\mathfrak{A}}$ such that $\pi\in\Pi$,
$\pi(W_{\mathfrak{A}})=1$; $\pi(\varnothing)=0$; and for $p,q\subseteq
W_{\mathfrak{A}}$, if $p\subset q$ then $\pi(p)\leq\pi(q)$; and if $p\cup
q=0$, then $\pi(p\cup q)=\pi(p)\cup\pi(q)$. Such measures exist Gaifman (1964)
and in virtue of Proposition LABEL:boolean obey axioms $A$.
To prove (iii), if $\pi$ does not obey axioms $A$, then one of the clauses of
satisfaction for L formulas Tarski (1944) will fail; since a model of ${\cal
L}$ must by definition satisfy those clauses, $\pi$ cannnot define a model.
$\Box$

######  Proposition 4.

Every ideal distribution defines a modal model of ${\cal L}$.

Proof of Proposition 4.
By hypothesis the objective probability distribution $P$ respects the
structure of $\mathbb{P}$ and thus $P$ obeys the axioms in $A$. By Proposition
3 $P$ defines a model ${\mathfrak{A}}_{P}$ of ${\cal L}$. By definition of
$\kappa$, $\kappa(\phi|T)=P(\phi)$, so $\kappa(.|T)$ defines a model
${\mathfrak{A}}_{\kappa}$ of ${\cal L}$ that is elementarily equivalent to
${\mathfrak{A}}_{P}$ Chang and Keisler (1973). But this means that $\kappa$
must respect the structure of $\mathbb{P}$ and has the requisite properties
via Proposition 3 to define a model

A.2 Dutch books

Let’s look at betting behavior and degrees of belief. For instance, if I
believe $p$ to degree $.5$, if I am rational I will bet that $p$ is true only
given even odds; but if my degree of belief is $.6$, then I will take the bet
given 4/6 odds or greater (I will receive 4 euros on winning and will pay out
6 euros if I lose). Linking probability to betting behavior leads to the idea
of a Dutch book. A Dutch book is a set of odds and bets, established by a
bookmaker, that ensures that the bookmaker will profit at the expense of the
gambler no matter what the facts are. It’s a way of showing the behavioral
effects of strong hallucinations.

We predict that LMs should be subject to Dutch Books. Given how $\hat{f}$
calculates probabilities for strings, $1-\mu(\phi)$ is the probability of the
set $Sent({\cal L})\setminus\phi$, the set of all sentences other than $\phi$.
If $\mu(Sent({\cal L})\setminus\\{\phi,\neg\phi\\})>0$, then
$1-\mu(\phi)>\mu(\neg\phi)$, and so $\mu(\neg\phi)+\mu(\phi)<1$ and
$\mu(\phi\vee\neg\phi)<1$. But now a bookie can offer $\hat{f}$ bets that on
the set of possibilities $D=\\{\phi,\neg\phi\\}$. Given $\hat{f}$’s
probabilities, $\hat{f}$ should bet that none of $D$’s possibilities holds
only if the bookie gives $\hat{f}$ odds reflecting
$1-\mu(\neg\phi)+\mu(\phi)$. Since this is positive, $\hat{f}$ will take a bet
the bookie can’t lose and so $\hat{f}$ will necessarily lose money.

Discussion of the import of a Dutch book argument: $\hat{f}$ can have a
distribution where it is not induced to affirm both $\phi$ and $\neg\phi$ but
still be subject to a Dutch book argument. Consider a language $L$ with just
four sentences or strings $L=\\{p,\neg p,q,p\vee\neg p\\}$, and let
$\mu_{\hat{f}}(p)=\mu_{\hat{f}}(q)=\frac{1}{4}$ with $\mu_{\hat{f}}(\neg
p)=0$. This will not induce $\hat{f}$ by our definitions to affirm an outright
contradiction. The incoherence is more subtle. What is the probability that
$\hat{f}$ could assign to $p\vee\neg p$ in this case? Since $\sum_{\phi\in
L}\mu(\phi)=1$, $\mu(p\vee\neg p)=.5$ as a maximum. So the bookie can offer
$\hat{f}$ an astronomical reward to bet against $p\vee\neg p$, and $\hat{f}$
should accept the bet if it’s maximizing its expected gain. Inevitably,
$\hat{f}$ will then lose money.

We prompted ChatGPT to bet on semantically equivalent, short sentences (for an
interaction see Appendix G). ChatGPT failed to recognize semantic equivalences
and so was subject to Dutch books.

We offer another reason why LLM distributions lead to strong hallucinations
besides those given in Proposition 1.

######  Proposition 1.

i) If $\mu:Sent({\cal L})\rightarrow[0,1]$ and $\hat{f}$’s betting behavior
reflects $\mu_{\hat{f}}$ and computes distributions according to equation 2,
$\hat{f}$ will strongly hallucinate concerning some logically equivalent
propositions.

Suppose two logically equivalent propositions, $\chi,\psi$ with $\psi$ longer
than $\chi$, but such that $\mu(\chi)=\mu(\psi)$. Suppose that $\chi$ is a
subformula or substring of $\psi$. So this means that $\psi=\zeta.\chi$ and
$\mu(\psi)=\mu(\zeta.\chi)$. But then by conditionalization equation 2,
$\mu(\zeta|\chi).\mu(\chi)=\mu(\chi)$. But then $\mu(\zeta|\chi)=1$. So no
other continuation of $\chi$ has non $0$ probability. Take an extension (via
conjunction) $\zeta^{\prime}.\zeta.\chi$ that’s also logically equivalent to
$\chi$. But $\mu(\zeta^{\prime}.\zeta|\chi)=0$, given that all the probability
mass in the distribution is on $\mu(\zeta|\chi)$. In either case $\hat{f}$
will be open to a Dutch book argument. As a result, $\mu$ diverges strongly
from any objective distribution $P$. By Definition 2, $\hat{f}$ strongly
hallucinates. $\Box$

A.3 Proposition 5 and proof

######  Proposition 5.

Any LM $\hat{f}$ whose output distribution is defined over individual strings
will strongly hallucinate when it outputs a reasoning sequence.

A reasoning task results in a string $\sigma$ consisting of a sequence
$\sigma_{1}.\sigma_{2}.\cdots$ of logically valid steps starting from a
premise $\sigma_{1}$ and ending up with a conclusion $\sigma_{n}$. We suppose
that the coherent continuations $\sigma_{i+1}$ in a reasoning task
$\sigma_{1},...,\sigma_{i}$ are only a small fraction $\frac{1}{m}$ of all
possible continuations $\chi$ of the sequence. We now suppose that $\hat{f}$
has learned a transition matrix for $i$ reasoning steps in the sequence
$\vec{\sigma}$ such that
$\mu(\chi|\vec{\sigma_{i}})<\mu(\sigma_{i+1}|\vec{\sigma_{i}})$ for $\chi$ but
$\Sigma_{j}\mu(\chi_{j}|\vec{\sigma_{i}})>\mu(\chi|\vec{\sigma_{i}})$/ Given
Proposition 3, given $\hat{f}$, $\mu_{\hat{f}}$ should be such that
$\mu_{\hat{f}}(\sigma_{2}|\sigma_{1})=1$, but in addition
$\mu_{\hat{f}}(\sigma_{2}.\sigma_{3}|\sigma_{1})=1$. But that is impossible
since $\mu_{\hat{f}}$ is a distribution over individual strings. So
$mu_{\hat{f}}(\sigma_{2}.\sigma_{3}|\sigma_{1})<\mu(\sigma_{2}|\sigma_{1})<1.$
That is, the transition matrix governing the reasoning chain outputs lower and
lower probabilities. So as the reasoning chain gets longer, the probability of
a valid continuation, the next step $\sigma_{i+1}$ in the reasoning process
monotonically decreases and the probability that an LM introduces an arbtrary,
invalid step in the reasoning chain increases monotonically. Eventually the
expectation is almost sure that $\mu$ will assign some other logically
erroneous continuation of the sequence a higher probability. $\Box$

A.4 Explanation of constraint in admissable continuations

$P(a|b^{c})=\frac{P(a\cap b^{c})}{P(b^{c})}$ (3)

where $b^{c}$ is the complement of $b$.

$(a\cap b^{c})\cup(a\cap b)=a$ (4)

Since the 2 sets on the left hand side of (4) are disjoint, $P(a)=P(a\cap
b^{c})+P(a\cap b)$, so $P(a\cap b^{c})=P(a)-P(a\cap b)$. Putting that together
with what we had above:

$\frac{P(a\cap b^{c})}{P(b^{c})}=\frac{P(a)-P(a\cap b)}{1-P(b)}$ (5)

$P(a|b^{c})=\frac{P(a)-P(b|a).P(a)}{1-P(b)}$ (6)
$P(a|b^{c})=\frac{P(a)(1-P(b|a))}{1-P(b)}$ (7)

B: Schemas Used to Construct Our synthetic dataset SYN
We design two datasets, one containing positive and another containing the
negative samples.
The positive schemas are as follows :-

  1. 1.

CONTEXT: There was a <col> car. QUESTION: Was there a <col> car ?

  2. 2.

CONTEXT: John played with a <col> ball.
QUESTION: Did john play with a <col> ball ?

  3. 3.

CONTEXT: The man was wearing a <col> shirt.
QUESTION: Did the man wear a <col> shirt ?

  4. 4.

CONTEXT: The house had a <col> window.
QUESTION: Did the house have a <col> window ?

  5. 5.

CONTEXT: A <col> glass was on the table.
QUESTION: Was there a <col> glass on the table?

The negative schemas are given by :-

  1. 1.

CONTEXT: There was no <col> car. QUESTION: Was there a <col> car ?

  2. 2.

CONTEXT: John played with no <col> ball.
QUESTION: Did john play with a <col> ball ?

  3. 3.

CONTEXT: The man was wearing no <col> shirt.
QUESTION: Did the man wear a <col> shirt ?

  4. 4.

CONTEXT: The house had no <col> window.
QUESTION: Did the house have a <col> window ?

  5. 5.

CONTEXT: No <col> glass was on the table.
QUESTION: Was there a <col> glass on the table?

We generate different data points by choosing <col> from a list of colors
(red,blue,green,yellow,black,white)

C.Histograms Showing that Specialised training on QA teaches the models
something about negation:

Figure 1: Histogram plot of cossim values for positive and negative contexts
with BERT-large (above) BERT-base (below) before (red) and after fine-tuning
(blue).

The values in red, the cossim values of CLS tokens without fine-tuning are on
the higher end of the historgram, indicating a high cosssim between positive
and negative samples.

D. Model behavior on SYN and SIM datasets.

The smaller LM models, BERT and RoBERTa basic, were worse on the positive
contexts than the negative ones, tending to answer “no” more often than not.
They also sometimes got confused by non-typical color choices (given the
context, John played with a black ball, the base models sometimes answered no
to did John play with a black ball?. We separated out a subset SIM of SYN,
where we filled the schemas in with typical colors and other properties of the
objects in question. The smaller models far fewer difficulties with the (Sim)
examples; this suggests that their mediocre performance on SYNis not about
negation but rather about its pretraining inducing it to reject non standard
properties, like non standard colors for bats and balls and the like.

E.Some Examples from Cloze Prompts Showing Wrong Predictions from LMs without
our negation constraint and the change due to our negation constraint.
We show the positive and negative strings predicted by out of the box,
bert-base

  1. 1.

A teacher is most likely teaching at a [school].
A teacher is not most likely teaching at a [school].

  2. 2.

The Teatr Wielki is a [museum]
The Teatr Wielki is not a [museum]

  3. 3.

The LDS Church focuses on [individual] mentorship.
The LDS Church does not focus on [individual] mentorship.

  4. 4.

Warsaw is the most diverse [city] in Poland.
Warsaw is not the most diverse [city] in Poland.

  5. 5.

The 1893 World’s Columbian Exposition was held in [chicago].
The 1893 World’s Columbian Exposition was not held in [chicago].

With our approach, we look at the top 5 probabilities which are predicted for
the positive string and reverse them (1-P), this gives us the following [MASK]
completions,

  1. 1.

A teacher is not most likely teaching at a [hospital].

  2. 2.

The Teatr Wielki is not a [club]

  3. 3.

The LDS Church does not focus on [community] mentorship.

  4. 4.

Warsaw is not the most diverse [suburb] in Poland.

  5. 5.

The 1893 World’s Columbian Exposition was not held in [philadelphia].

roberta-large

  1. 1.

Quran is a <religious> text.
Quran is not a <religious> text.

  2. 2.

Isaac’s chains made out of <wood>.
Isaac’s chains made out of <wood>.

  3. 3.

The sporting capital of Australia is <Melbourne>.
The sporting capital of Australia is not <Melbourne>.

  4. 4.

Warsaw is the most diverse [city] in Poland.
Warsaw is not the most diverse [city] in Poland.

  5. 5.

The 1893 World’s Columbian Exposition was held in [chicago].
The 1893 World’s Columbian Exposition was not held in [chicago].

Our approach on the other hand gives:

  1. 1.

Quran is not a <biblical> text.

  2. 2.

Isaac’s chains made out of <glass>.

  3. 3.

The sporting capital of Australia is not <Brisbane>.

  4. 4.

Warsaw is not the most diverse [place] in Poland.

  5. 5.

The 1893 World’s Columbian Exposition was not held in [philadelphia].

We assume that the model can generate correct completions for the positive
sentences,which when reversed might give us a likely completion for the
negative string.

F. SNLI examples and corrections

Here is a typical example from SNLI:
(C) "Tattooed young woman chains her bicycle to a signpost while juggling a
guitar on her back",
($\neg h$) "A woman with tattoos does not chain her bike to a post",

which is (incorrectly) labelled with contradiction, when the judgment should
be neutral due to the presence of the two indefinites. To get make $\neg h$ a
contradictory of $C$, we had two strategies:

"Tattooed young woman chains her bicycle to a signpost while juggling a guitar
on her back",
"No woman with tattoos chains her bike to a post",

and "Tattooed young woman chains her bicycle to a signpost while juggling a
guitar on her back",
"The woman with tattoos did not chain her bike to a post",

Both of these devices restore the desired contradictory status and contrast
nicely with the original entailment pair from SNLI:

"Tattooed young woman chains her bicycle to a signpost while juggling a guitar
on her back",
"A woman with tattoos chains her bike to a post",

G: Scoped algorithms for NLI We first give the full unscoped rules for NLI on
the two label problem in $\neg$RTE.

For $(C\neg h)$.
(i) If $(C,h)$:$E$, then $(C,\neg h)$: $\neg E$.
(ii) If $(C,h)$:$\neg E$ and if $(h,C)$:$\neg E$, then $(C,\neg h)$: $\neg E$.

For $(\neg C,\neg h)$.
If $(h,C)$:$\neg E/E$, then $(\neg C,\neg h)$: $\neg E/E$.

For $(\neg C,h)$.
(i) if $\neg C,\neg h$:$E$, then $(\neg C,h)$:$\neg E$.
(ii) If $[(C,h)$:$\neg E$ and $(h,C)$:$E$, then $(C,\neg h)$:$\neg E$.
(iii) If $\\{[(C,h)$:$\neg E$ and $(h,C)$:$E]$ or $[(C,h)$:$E$ and
$(h,C)$:$\neg E]\\}$, then $(\neg C,h)$:$\neg E$. Else, $(\neg C,(\neg)h)$:
$\neg E$ .

Here again are the full scoped rules for the two label ($E,\neg E)$ problem.

Algorithm 3 RTE Algorithm for $(\neg C,h)$

1:procedure Algorithm$(\neg C,H)$

2: $(C,h):\leftarrow\text{defined in paper}$

3: $(P,h):\leftarrow\text{P defined in paper}$

4: if $(h,C):\textit{$\neg E$}$ then

5: if $(C,h):\textit{E}$ and $(P,h):\textit{E}$ then

6: $(\neg C,h):\textit{E}$

7: else

8: $(\neg C,h):\textit{$\neg E$}$

9: if $(h,C):\textit{E}$ then $(\neg C,h)$: $\neg E$

Algorithm 4 RTE Algorithm for $(\neg C,\neg h)$

1:procedure Algorithm$(\neg C,\neg H)$

2: $(C,h):\leftarrow\text{C,h defined in paper}$

3: $(P,h):\leftarrow\text{P defined in paper}$

4: $(h,C^{\prime}):\leftarrow\text{C' defined in paper}$

5: if $(h,C):\textit{E}$ then

6: if $(P,h):\textit{E}$ then

7: $(\neg C,h):\textit{E}$

8: else

9: $(\neg C,h):\textit{$\neg E$}$

10: if $(h,C):\textit{$\neg E$}$ then

11: if $(h,C^{\prime}):\textit{E}$ then

12: $(\neg C,h):\textit{E}$

Here are the fully scoped rules for the three label NLI problem and the $(\neg
C,h)$ and $(\neg C,\neg h)$ cases. Let $\neg C=(P,\neg[C^{\prime}])$.

Algorithm 5 SNLI Algorithm for $(\neg C,h)$

1:procedure Algorithm$(\neg C,H)$

2: $(C,h):\leftarrow\text{defined earlier}$

3: $(P,h):\leftarrow\text{defined earlier}$

4: $(h,C^{\prime}):\leftarrow\text{defined earlier}$

5: if $(C,h):\textit{E}$ then

6: if $(P,h):\textit{E}$ then

7: $(\neg C,h):\textit{E}$

8: if $(P,h):\textit{N}$ and $(h,C^{\prime}):\textit{E}$ then

9: $(\neg C,h):\textit{$C_{n}$}$

10: else

11: if $(h,C^{\prime}):\textit{N}$ then

12: $(\neg C,h):\textit{N}$

13: if $(C,h):\textit{$C_{n}$}$ then

14: if $(P,h):\textit{$C_{n}$}$ then

15: $(\neg C,h):\textit{$C_{n}$}$

16: if $(P,h):\textit{N}$ and $(h,C^{\prime}):\textit{E}$ then

17: $(\neg C,h):\textit{$C_{n}$}$

18: else

19: if $(h,C^{\prime}):\textit{$C_{n}$}$ then

20: $(\neg C,h):\textit{E}$

21: if $(h,C^{\prime}):\textit{N}$ then

22: $(\neg C,h):\textit{N}$

23: if $(C,h):\textit{N}$ and $(h,C):\textit{E}$ then

24: $(\neg C,h):\textit{$C_{n}$}$

25: else

26: $(\neg C,h):\textit{N}$

These complex rules for $\neg C,h$ enable a simple rule for $(\neg C,\neg h)$:

Algorithm 6 SNLI Algorithm for $(\neg C,\neg h)$

1:procedure Algorithm$(\neg C,\neg h)$

2: $(\neg C,h):\leftarrow\text{calculated earlier}$

3: if $(\neg C,h):\textit{$C_{n}$}$ then

4: $(\neg C,\neg h):\textit{$E$}$

5: if $(\neg C,h):\textit{$E$}$ then

6: $(\neg C,\neg h):\textit{$C_{n}$}$

7: if $(\neg C,h):\textit{$N$}$ then

8: $(\neg C,\neg h):\textit{$N$}$

Each one of these rules is provably valid. For instance, If $(h,C):negE$, we
can conclude $\neg C,\neg h:\neg E$. Suppose h,C not E that means either
$h\cap C=0$ or they overlap but $h\not\subseteq C$ in terms of events or
worlds. In the first case where they are disjoint, the complement of C
contains h and so neg C, neg h : neg E. In the second case where they overlap
but we don’t have h completely included in C then the complement of C contains
some h and some not h cases. So again $\neg C,\neg h$: $\neg E$ in this case.

H: Screenshots of interacting with ChatGPT on betting behavior

Figure 2: Betting Prompts With ChatGPT