# Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical
Reasoning Capabilities of Language Models

Man Luo 1,211footnotemark: 1 Shrinidhi Kumbhar111footnotemark: 1 Ming shen1
Mihir Parmar1
Neeraj Varshney1 Pratyay Banerjee3 Somak Aditya4 Chitta Baral1
1 Arizona State University, 2 Mayo Clinic, 3 Amazon Alexa AI, 4 IIT KGP
{mluo26<EMAIL_ADDRESS>
###### Abstract

Logical reasoning is fundamental for humans yet presents a substantial
challenge in the domain of Artificial Intelligence. Initially, researchers
used Knowledge Representation and Reasoning (KR) systems that did not scale
and required non-trivial manual effort. Recently, the emergence of large
language models (LLMs) has demonstrated the ability to overcome various
limitations of formal Knowledge Representation (KR) systems. Consequently,
there’s a growing interest in using LLMs for logical reasoning via natural
language.

This work strives to understand the proficiency of LLMs in logical reasoning
by offering a brief review of the latest progress in this area; with a focus
on the logical reasoning datasets, tasks, and the methods adopted to utilize
LLMs for reasoning. To offer a thorough analysis, we’ve compiled a benchmark
titled LogiGLUE. This includes 24 varied datasets encompassing deductive,
abductive, and inductive reasoning. We have standardized these datasets into
Seq2Seq tasks to facilitate straightforward training and evaluation for future
research. Utilizing LogiGLUE as a foundation, we have trained an instruction
fine-tuned language model, resulting in LogiT5. We study single-task training,
multi-task training, and a "chain-of-thought" knowledge distillation fine-
tuning technique to assess the model’s performance across the different
logical reasoning categories. By this comprehensive process, we aim to shed
light on the capabilities and potential pathways for enhancing logical
reasoning proficiency in LLMs, paving the way for more advanced and nuanced
developments in this critical field.

Towards LogiGLUE: A Brief Survey and A Benchmark for Analyzing Logical
Reasoning Capabilities of Language Models

Man Luo 1,211footnotemark: 1 Shrinidhi Kumbhar111footnotemark: 1 Ming shen1
Mihir Parmar1 Neeraj Varshney1 Pratyay Banerjee3 Somak Aditya4 Chitta Baral1 1
Arizona State University, 2 Mayo Clinic, 3 Amazon Alexa AI, 4 IIT KGP {mluo26,
<EMAIL_ADDRESS>

00footnotetext: *Equal contribution

##  1 Introduction

Figure 1: Logical Reasoning Survey: Datasets and Language Model Application.

With logical reasoning, humans can explain an answer to a question via step-
wise deduction, make robust planning and decision, or even reason about the
workings in an unseen universe. As an example of logical reasoning, physicist
Stephen Hawking derived the area theorem in 1971, which indicates the boundary
beyond which nothing can ever escape (black hole). If Hawking’s area theorem
holds, then using logical reasoning, we can conclude that the horizon area of
the new black hole should not be smaller than the total horizon area of its
parent black holes. This theorem was later confirmed by real observation by
MIT researchers in 2015, fifty years later after the theorem had been
derived111https://news.mit.edu/2021/hawkings-black-hole-theorem-confirm-0701.

In the field of Artificial Intelligence (AI), there has been significant
attention directed towards the aspiration to develop machines equipped with
logical reasoning capabilities (McCarthy, 1989; Colmerauer and Roussel, 1996).
Early approaches in logical reasoning were primarily dedicated to the design
of formal logic languages to encapsulate rules and knowledge, along with the
development of automated theorem provers (Lifschitz, 2019). This paradigm,
however, necessitated a deep understanding of the syntax and semantics of the
formal logic for manual rule formulation – making knowledge representation and
knowledge acquisition hard and expert-driven endeavor. Due to these
challenges, contemporary research has progressively turned towards addressing
logical reasoning tasks (Clark et al., 2020; Tian et al., 2021a; Han et al.,
2022) by employing transformer-based Vaswani et al. (2017) pre-trained
language models Devlin et al. (2019a); Brown et al. (2020).

The Language models (LMs) that are pretrained using objectives such as the
mask language modeling Devlin et al. (2019a) and next word prediction Brown et
al. (2020) enables them to acquire adequate syntax and semantics of language,
alongside commonsense knowledge. These language models can excel in numerous
natural language understanding tasks, owing to the unsupervised pretraining on
a vast array of unstructured text data. However, it is unclear if the current
pretraining objectives are sufficient enough for the models to infer logical
reasoning because this involves understanding structure; coupled with
inductive, deductive, and abductive reasoning skills. This question has drawn
intense attention and inspired different research directions to examine if LMs
can learn logical reasoning ability Wu et al. (2023); Lanham et al. (2023);
Clark et al. (2020); Joshi et al. (2020). For instance, Clark et al. (2020)
shows that pre-trained language models can serve as a “soft-reasoner" based on
their near-perfect performance on synthetic datasets. Creswell et al. (2022)
showed that large LMs are few-shot logical reasoners. On the other hand, Liu
et al. (2020); Joshi et al. (2020); Han et al. (2022) shows that logical
reasoning remains challenging for language models. Furthermore, Wu et al.
(2023); Lanham et al. (2023) showed that LLMs maybe retrieving or reciting
previously seen facts and steps, instead of actually reasoning. Liu et al.
(2023b) shows that while ChatGPT and GPT-4 generally perform well on some
benchmarks, their performance noticeably diminishes when faced with new or
out-of-distribution datasets.

To better understand the progress of logical reasoning ability in the current
language model era, we first provide a concise survey of its role within
current language models. Based on the insights gathered through the survey, we
assembled a logical reasoning benchmark termed as LogiGLUE. Subsequently, we
trained a model on this benchmark by utilizing diverse training strategies;
our contributions are summarized below.

##### Concise Survey.

We provide a brief survey of the recent development of logical reasoning using
natural language (see Figure 1). First we discuss three types of logical
reasoning. Then we focus on the relevant benchmarks and the methodologies for
applying LMs to logical reasoning tasks.

##### LogiGLUE.

One result of this survey is a benchmark for logical reasoning (LogiGLUE),
with the aim to facilitate a consistent progress of logical reasoning in NLP.
The importance for the LogiGLUE benchmark arises from several critical
considerations. First, it encompasses diverse logical reasoning tasks and
generalization evaluation, ensuring a comprehensive assessment of how a model
performs across varied logical paradigms. Second, the unique format of each
dataset within LogiGLUE simplifies both training and evaluation processes,
facilitating swift integration into research workflows. Lastly, researchers
can easily compare with established baselines, and the LogiGLUE offers the
flexibility to seamlessly integrate new datasets in the future, ensuring its
lasting relevance in logical reasoning evaluation.

##### LogiT5.

Drawing inspiration from recent successes in multi-task learning and
instruction-fine-tuned models, we trained seq2seq models, specifically Flan-T5
Chowdhery et al. (2022), using multi-task learning on LogiGLUE’s in-domain
data. The resulting model, named LogiT5, demonstrated effective generalization
on out-of-domain data.

##  2 A Concise Survey of Logical Reasoning in NLP: Types of Reasoning,
Datasets and Language Models Approach

The advent of large language models has been transformative for the AI
community; prompting many to speculate that we are on the cusp of achieving
general artificial intelligence (GAI). Yet, as astounding as their
capabilities are, these models grapple with numerous challenges, particularly
with logical reasoning Han et al. (2022); Valmeekam et al. (2023, 2022); Guan
et al. (2023). Recognizing the significance of this, our survey aims to
provide a timely comprehensive overview of advancements in logical reasoning
within the context of language models, elucidating their performance,
limitations, and the obstacles that remain, which casts a vision for future
research directions. While other surveys have touched upon the broader theme
of logical reasoning using natural language Helwe et al. (2022); Yu et al.
(2023); Yang et al. (2023), our survey has led us to propose a comprehensive
bnechmark collection, and include a systematic review of techniques to adopt
LLMs for logical reasoning tasks. More importantly, we categorize different
ways of using LMs on logical reasoning tasks, highlighting the intricacies and
challenges faced by models for such tasks.

Figure 2: Examples (top) of three types of logical reasoning and explanations
(bottom) correlating each example with its respective reasoning type.

###  2.1 Three Types of Logical Reasoning

##### Deductive Reasoning.

In this predominant form of reasoning, we start with a set of premises which
can be facts or rules, and derive a specific conclusion based on the available
premises with a valid logical derivation path. In short, deductive reasoning
derives specific conclusion(s) from generic observation(s) Byrne et al.
(2019). There are two characteristics related to a deductive reasoning system,
validity and soundness. A conclusion is valid if and only if it is fully
supported by the premises irrespective of the factuality of the premises. A
conclusion is sound if and only if it is valid and the premises are true. For
example in Figure 2, the conclusion is valid but it is not sound because it is
not true that “All kids love animals.” Most of the synthetic deductive
reasoning datasets such as RuleTaker Clark et al. (2020) has valid
conclusions, but may not be sound as the rules in the premises are often
synthetically generated and may be untrue in the real world. Datasets such as
PrOntoQA Saparov and He (2023a) offer a broader view, by sourcing the premise
rules from a true, a false and a fictional ontology.

##### Inductive Reasoning.

For inductive reasoning, one starts with a set of observations, and derives a
general conclusion that is merely true, but not certain Heit (2007); Sauce and
Matzel (2017). In contrast to deductive reasoning, inductive reasoning is a
bottom-up reasoning process which starts from specific observations and
derives a generic conclusion. Many Knowledge graph completion task requires
inductive reasoning such as WN18RR222Here, we exclude this task since we are
more interested in natural language input. In this paper, we do not discuss
about knowledge graph completion tasks since most of them are not in natural
language forms.. To apply inductive reasoning, one usually relies on a large
number of observations (both positive and negative in support or against an
induced rule). Since large language models are pretrained on large amount of
free-text, it learns several generic patterns or conclusions, therefore
reasoning inductively (even if the rules may not be represented symbolically
or a human-readable fashion) Han et al. (2023). In general, commonsense
reasoning tasks in NLP require both inductive and deductive reasoning.

##### Abductive Reasoning.

Abductive reasoning typically begins with an incomplete set of observations
and proceeds to derive most likely explanations for the observations to be
true Paul (1993); Hobbs et al. (1993). Similar to inductive reasoning, this
also involves uncertainty, as there can be different explanations. Compared to
inductive reasoning, the deductive reasoning is a process from known facts or
rules to derive a new conclusion, while abductive reasoning is from an
observation to “guess” what can be the reason to cause the observation. It is
used more often in our daily decision-making, such as medical diagnoses based
on a set of incomplete symptoms.

In previous paragraphs, we mentioned how both inductive and abductive
reasoning inherently encompass uncertainty. In fact, deductive reasoning can
also operate within the realm of uncertainty De Raedt et al. (2007);
Richardson and Domingos (2006); Lee and Wang (2016); Bach et al. (2017); Lee
and Luo (2018). Such reasoning paradigm uses “soft rules” to indicate the
likelihood of a rule being true rather than its absolute truth. Consequently,
conclusions derived may carry probabilistic true/false values. Reasoning under
uncertainty is particularly useful because the real world is inherently
unpredictable and full of unknown variables. While many datasets operate under
the assumption that rules are unequivocally true, Rulebert Saeed et al. (2021)
deviates by attributing weight values to each rule.

###  2.2 Logical Reasoning Tasks and Datasets

We discuss the task and datasets in terms of format of the tasks and how they
are created.

####  2.2.1 Four Types of Tasks

##### Multiple Choice Question Answering (MCQA).

In the MCQA task, the given inputs are a paragraph which forms a context, a
question, and a list of answer candidates (typically four choices). The goal
is to predict which candidate is (most likely) correct. All datasets are pure-
text Yu et al. (2019); Liu et al. (2020)333ReClor and LogiQA sources the
datasets from real examination questions that may involve images or charts.
But they remove such questions and retain only those which are self-contained
and answerable from the provided text..

##### Free Form Question Answering.

Unlike MCQA, where a set of answer choices are given, freeform QA only has a
context and a question, and the answer to the question can be any format,
including but not limited to a single word, a list of words, and a number
Weston et al. (2015b); Banerjee et al. (2020).

##### Fact Checking.

In fact verification, given a context and a fact, the goal is to classify the
binary truth value of the fact according to the given information Clark et al.
(2020); Saeed et al. (2021); He et al. (2021).

##### Natural language inference (NLI)

NLI is the task of detecting inferential relationships between a premise and a
hypothesis. For most NLI datasets, there are three relationships, entailment
(the hypothesis follows or can be inferred by the premise), neutral (the truth
of hypothesis is undetermined by the premise), and contradiction (the
hypothesis contradicts the premise or some facts in the premise) Tian et al.
(2021a).

####  2.2.2 Dataset Creation Techniques

##### Human Annotation.

Crowdsourcing is one of the major approaches to create datasets, such as for
NLI tasks. The advantages of this methodology include a richer linguistic
grammar and potentially increased task complexity. However, it comes with
drawbacks. In addition to being a cost-intensive process, crowdsourced
datasets tend to harbor biases (as highlighted in numerous previous studies Yu
et al. (2019)). These biases can be leveraged by neural models to artificially
inflate accuracy scores. Furthermore, assembling a dataset for logical
reasoning tasks demands a level of expertise that poses a significant
challenge.

##### Extraction from Academic Challenge.

It is hard for crowdsourcing workers to produce questions requiring
complicated logical reasoning since such reasoning tasks require extensive
training and practice. Fortunately, questions in some standardized tests are
aligned with the goal of logical reasoning and can be utilized to create such
datasets after some preprocessing Yu et al. (2019); Liu et al. (2020).
However, the domains of these examinations are limited and the dataset size is
small.

##### Synthetic Generation.

Synthetic generation is more efficient to create large data than manually
created ones Luo et al. (2022b). There are two ways, simulation based Weston
et al. (2015b) and rule-based Clark et al. (2020); Saeed et al. (2021);
Banerjee et al. (2020). In rule based methods, logic programs (either written
by humans or mined from knowledge graphs) are generated, and then implications
are drawn by automatic theorem prover. Last, the rules and facts in the logic
programs are converted into English form using natural language patterns.
Synthetic generation has issues that the rules or facts do not have real-world
meaning and the language could be simple.

###  2.3 Language Models for Logical Reasoning over Natural Language

Language models (LMs) have been actively studied these days for logical
reasoning tasks. Dasgupta et al. (2022) demonstrates that large language
mdoels (LLMs) show human-level abstract reasoning skill. Creswell et al.
(2022) proposes a selection-inference pipeline that given a context and
question, the model can firstly select which facts or rules given in the
context are important to answer the question and decompose the question into
step by step reasoning. Wei et al. (2022) demonstrates that language models
have the capacity to engage in chain-of-thoughts (CoT) reasoning. This
approach facilitates a step-by-step reasoning process that enhances the
performance of the model in downstream tasks such as mathematical reasoning.
In following section, we summarize the five prevalent trends in utilizing
language models for logical reasoning over language.

####  2.3.1 Supervised Finetuning

Fine-tuning a language model on the downstream tasks has been a standard way
to teach a model to perform a task. Such a paradigm has also been the
prevalent method for logical reasoning tasks Clark et al. (2020); Liu et al.
(2021); Tian et al. (2021b); Saeed et al. (2021); Han et al. (2022); Chan et
al. (2023). In general, such a method is usually applied to a moderate size of
the language model such as BERT Devlin et al. (2019b), GPT2 Radford et al. ,
RoBERTa Liu et al. (2019), and XLNet Yang et al. (2019). It has been shown
that transformer based models perform better than the other types of neural
models such as LSTM Yu et al. (2019); Liu et al. (2020), probably because such
pretrained models have a certain degree of commonsense and logical reasoning
Huang et al. (2019). This has been further proven in Clark et al. (2020). They
show that when every word in the passage is replaced by a random word
resulting in no grammaticality, the performance of a transformer-based model
dramatically decreases. In addition, the larger model performs better than
smaller ones, indicating that the deeper a model is, the more complicated
reasoning it can execute He et al. (2021). While the IID performance of a
fine-tuned model can be nearly perfect, such model has poor generalization.
For example, model can not generalize from lower depth to higher depth
reasoning Clark et al. (2020), from low level language diversity to high level
diversity Richardson and Sabharwal (2021); Tafjord et al. (2021), from one
domain to another domain Banerjee et al. (2020). Such observations indicate
that models might just learn the inductive pattern in the training data rather
than the underline logical reasoning skill Zhang et al. (2022).

####  2.3.2 Logical Reasoning Pretraining

The next word prediction or mask language modeling pretraining tasks allow the
language models to learn the language syntax and semantic as well as the world
knowledge, however, it does not guarantee a model to learn logical operations.
Thus, researchers have been exploring logical-oriented pretraining tasks to
teach a model of logical reasoning from large free data. APOLLO Sanyal et al.
(2022)improves the logical reasoning of a model by two pre-training tasks. The
first pretraining task is selective mask language modeling (MLM). Unlike the
naive MLM which randomly masks the words, s-MLM selects and masks the logical
words (defined by Spacy POS tags). The second pretraining task is entailment
classification which aims to classify if there is an entailment relationship
within a masked sentence or not. MERIt Fangkai Jiao (2022) proposes a meta-
path-guided pretraining task to teach a model to learn logical reasoning by
self-supervised learning. They construct the training data by converting any
document into a graph with entities as the node and the relation between the
entities as edges. Then, given a pair of entities, the positive candidates are
the sentences that connect this pair of entities, and the negative candidates
are obtained by data augmentation. Such training data allows the model trained
by contrastive learning manner to identify the positive sentence from the
negative sentences. MERIt+ Jiao et al. (2023) combines MERIt with the
autoregression training objective: rather than using contrastive learning,
MERIt+ optimizes the probability of positive candidate sentences.

####  2.3.3 Proof Generation

Proof generation is found to be harder than answer generation Saha et al.
(2020); Tafjord et al. (2021). However, models developed for a proof
generation task have better performance on out-of-domain datasets or unseen
depth reasoning (e.g., train on lower depth and test on higher depth). Kaiyu
Yang and Chen (2022) introduce NLProofS, a novel method for generating step-
by-step logically valid and relevant proofs given a set of supporting facts
and hypothesis. In their proposed method, they employ a prover which generates
candidate proofs step-by-step, a verifier to measure the validity of generated
proof steps to avoid the prover from hallucinating proof steps, and an
algorithm for retrieving the entire proof with highest validity score by
aggregating proof step scores. ProofWriter Tafjord et al. (2021) proposed two
ways to generate proofs based on T5 models. The first one is to predict the
sequence of proof in one output; the second one is to iteratively generate a
proof and specifically, predict one intermediate conclusion and combine it
with the given facts and rules as a new input to predict the following
conclusion, and repeat this process until no new conclusion is predicted.

####  2.3.4 CoT Knowledge Distillation

The previous approach relies on the proof annotations in the datasets,
however, in many cases, the dataset does not come with the proof. It is shown
that large language model (LLM) can generate step-by-step reasoning (similar
as the proof) Saparov and He (2023a); Liu et al. (2023c). Namgyu Ho (2022)
propose Fine-tune-CoT (i.e. chain-of-thought Wei et al. (2022)) approach which
involves three key steps. In the first step, a large teacher model is prompted
to address intricate queries, generating multi-step reasoning explanations.
These explanations are then filtered based on the accuracy of the final
prediction. In the second step, a reasoning sample is constructed,
incorporating the question, rationale, and answer, thereby forming a
comprehensive prompt and multi-step solution. This collection of carefully
curated reasoning samples is leveraged to fine-tune a compact student model,
imbuing it with the ability to engage in reasoning tasks. Nonetheless, LLMs
encounter difficulties in planning proofs, occasionally making wrong
selections when presented with multiple valid choices. This challenge leads to
proofs that are not fully developed and consequently produces inaccurate
responses.

####  2.3.5 Neural Symbolic

Recent advancements in pre-trained language models have demonstrated
impressive reasoning abilities using explanations or "chain-of-thought" for
in-context learning. Conversely, reasoning tasks are considered more
straightforward for symbolic programming. A promising way is to use LLM to
translate a natural language input into a symbolic program which can be
consumed by a symbolic solver. Such a paradigm has been shown to effectively
avoid unfaithfulness of LLM Pan et al. (2023). Hanlin Zhang1 (2022) employs
LLMS as Logic Programmers (LMLP) to learn logic rules and examples and reason
over knowledge bases (KBs) using Prolog’s backward chaining algorithm. They
show that LMLP outperforms CoT in deductive reasoning settings, achieving over
25% higher accuracy on length generalization benchmarks, even with fewer
parameters. Pan et al. (2023) propose Logic-LM to handle deductive reasoning,
first-order logic, and constraint programming tasks. They leverage GPT-3 and
in-context learning (providing a few examples) to translate a natural language
input to a formal language formulation that can be executed by symbolic
engines. They also show that the error messages of the symbolic engines can
refine the output of an LLM. Such a paradigm has been investigated for
addressing other challenges, wherein LLMs act as planners, and external tools
are utilized to execute the plan Lu et al. (2023); Sumers et al. (2023);
Paranjape et al. (2023); Guan et al. (2023); Schick et al. (2023).

###  2.4 Survey Summary

The survey delineates how current datasets address three types of logical
reasoning distributed across four task formats. Additionally, the curation
process of a dataset can influence its inherent difficulty level. We’ve also
identified five approaches for utilizing LLMs in addressing these reasoning
tasks. This structured insight serves as a foundation for future research,
offering a roadmap to optimize model performance and curation methodologies.
In the following section, we will present a logical reasoning benchmark,
positioned alongside established benchmarks like SuperGlue Wang et al. (2019),
BigBench Srivastava et al. (2023), and Unicorn Lourie et al. (2021), all aimed
at exhaustively gauging system capabilities.

##  3 LogiGLUE: General Logical Reasoning Benchmark

Dataset | Train size | Dev size | Test size | Synthetic | Task Type | Reasoning Type
---|---|---|---|---|---|---
In-domain datasets |  |  |  |
$\alpha$ARCT 2019 | 2420 | 632 | 888 | ✗ | MCQA | Abductive
$\alpha$NLI 2019 | 169,654 | - | 1532 | ✗ | NLI | Abductive
CLUTTR-Robust 2019 | 10,100 | - | 144 | ✓ | FF | Inductive
AbductionRule-Animal 2019 | 23,100 | 3,300 | 6,600 | ✓ | FF | Abductive
ANLI 2020 | 162,865 | 3,200 | 3,200 | ✗ | NLI | Deductive
LogiQA 2021 | 7,376 | 651 | 651 | ✗ | MCQA | Mixed
LogicNLI 2021b | 16,000 | 2,000 | 2000 | ✓ | NLI | Deductive
ProofWriter 2021 | 69,814 | 10,158 | 20,058 | ✓ | FV | Deductive
Rulebert-Union 2021 | 56,000 | 4,666 | 9,334 | ✓ | FV | Deductive
FOLIO 2022 | 1004 | 204 | 227 | ✗ | FV | Deductive
Out-of-domain datasets |  |  |  |
bAbi 2015a | - | - | 5000 | ✓ | FF | Inductive
bAbi 2015a | - | - | 5000 | ✓ | FF | Deductive
CLUTTR-Systematic 2019 | - | - | 10100 | ✓ | FF | Inductive
AbductionRule-person 2019 | - | - | 4,864 | ✓ | FF | Abductive
ReClor 2020 | - | - | 500 | ✗ | MCQA | Mixed
Bird-Electricity 2021 | - | - | 5270 | ✗ | FV | Deductive
NatlLang 2021 | - | - | 8,008 | ✗ | FV | Deductive
Winologic 2021 | - | - | 562 | ✗ | FV | Deductive
WaNLI 2022 | - | - | 5000 | ✓ | NLI | Deductive
Rulebert-Union 2021 | - | - | 5000 | ✓ | FV | Deductive
BigBench 2022 | - | - | 1300 | ✗ | FF | Deductive
BigBench 2022 | - | - | 32 | ✗ | FF | Inductive
LogiQA 2.0 2023a | - | - | 3238 | ✗ | NLI | Deductive
PrOntoQA 2023b | - | - | 200 | ✗ | MCQA | Deductive

Table 1: Statistics of In-domain (IID) and out-of-domain (OOD) datasets of
LogiGLUE benchmark.

As mentioned in the introduction, the reasoning ability of language models as
assessed by various studies seems to differ. One plausible explanation for
this variance is the inconsistency in the benchmarks used or differences in
task formats, leading to performance disparities. To rectify this, our goal is
to offer a standardized testbed. It becomes imperative to meticulously
formulate our selection criteria to create a testbed that evaluates a system’s
logical reasoning capabilities. Guiding our dataset choice are two primary
principles outlined in §3.1. These endeavors have led to the formation of a
diverse and comprehensive logical reasoning benchmark, which we’ve named
LogiGLUE (§3.2).

###  3.1 Principle of Collecting LogiGLUE

Numerous logical reasoning datasets have been accessible since 2015, such as
bAbi Weston et al. (2015b). Our selection process for including a dataset in
LogiGLUE is primarily driven by principles of diversity and generalization
Gokhale et al. (2022).

##### Diversity.

There are two aspects to Diversity. First aspect concerns the types of
reasoning in the dataset. We ensure that our coverage encompasses three main
reasoning types, which collectively represent the full spectrum of logical
reasoning. These three categories have been previously discussed. The second
aspects concerns the level of difficulty, with datasets ranging from easy to
hard. Our experimental results indicate a varied model performance across
different datasets – excelling in some, delivering average results on others,
and struggling significantly on a few. We discovered a strong correlation
between the complexity of a dataset and the methodology employed in its
creation. Datasets built using simple templates and basic rule structures tend
to be easier. In contrast, those with more sophisticated rules and uncertain
elements are relatively more challenging. However, the most difficult datasets
are those meticulously crafted by human hands.

##### Generalization.

We also consider the axis of generalization, which aims to quantify (or
assess) whether a model trained on the logical reasoning tasks can genuinely
acquire reasoning skills. Previous studies have found that the superior
performance of a fine-tuned language model primarily stems from learning the
patterns exhibited within the dataset, which unfortunately often leads to poor
generalization to other datasets. Consequently, the model’s performance tends
to be overestimated due to the identical and independently distributed (IID)
nature of the testing data. To counteract this, LogiGLUE includes an out-of-
domain testing set that also encompasses the three types of reasoning. The
out-of-domain testing set is readily adaptable to incorporate future logical
reasoning tasks.

##### Excluded Datasets.

Lexicographically speaking, reasoning is defined as “the process of forming
conclusions, judgments, or inferences from facts or
premises."444https://www.dictionary.com/browse/reasoning Reasoning is usually
associated with an entailment relation, where given a premises, the truth
value of a hypothesis depends on if the latter is entailed by the premise or
not. There are many datasets that require reasoning which we decided to
exclude from the scope of this work. This includes some well-known NLI
datasets, such as, SNLI Bowman et al. (2015) and MultiNLI Williams et al.
(2018). These datasets use many linguistic forms, unstated background
knowledge, and sometimes unsupported inference steps Clark et al. (2020). We
also exclude datasets where reasoning with external domain knowledge is
required since for such tasks, retrieving the external knowledge is essential
and it is hard to diagnose whether the noisy retrieved knowledge affects
systems or systems lack of logical reasoning capacity. This includes QuAIL
Rogers et al. (2020), WSC Levesque et al. (2012), QuaRTz Tafjord et al.
(2019), ROPES Lin et al. (2019). Commonsense reasoning datasets are not
covered in this survey either since they focus on solving a task using
commonsense knowledge Sap et al. (2020) and thus it is more important to
acquire the commonsense knowledge rather than to do logical reasoning. Other
datasets that we exclude are ones that require logical reasoning but are not
presented in the natural language form such as logical entailment Evans et al.
(2018), NeuroSAT Selsam et al. (2019), and LTL Hahn et al. (2020).

###  3.2 Statistic of LogiGLUE

A suite of natural language logical reasoning benchmarks with 10 in-domain and
12 out-domain datasets that cover different types of logical reasoning. In
addition, LogiGLUE includes three task formats, multiple choice question
answer (MCQA), natural language inference (NLI), and fact verification (FV)/
fact checking (FC). Table 1 shows the statistics.

Unique Format. There are many existing practicing for standardizing different
datasets into a consistent format Mishra et al. (2022); Lourie et al. (2021),
such as transforming all tasks to question answering McCann et al. (2018) or
NLI Poliak et al. (2018) styles. Through our model analysis, it’s evident that
certain models can only manage specific reasoning tasks. For instance,
classification models are commonly used for NLI and MCQA tasks, where the
number of classification heads matches the number of choices (like 3 for NLI
and 4 for MCQA). Yet, these models struggle when confronted with free-form
question answering, thus limiting their versatility. Hence, to develop a model
adept at logical reasoning regardless of task structure, we convert them into
a singular format. An added advantage of this standardized format is that it
ensures consistency in the input, ruling out performance disparities arising
from different inputs. Every dataset is then adapted to this specific format.
In the case of MCQA/FV/NLI tasks, each instance encompasses a context, a
question, and potential answer options. Conversely, FF tasks don’t present any
answer choices. The correct answer is the expected output for every instance.
For FV tasks, we use the statement as the question with true/false as
potential answers. In NLI tasks, the options include natural, contradictory,
and entailment.

##  4 Experiments and Results

We selected Flan-T5-large Chowdhery et al. (2022) as our base model for
training due to two pivotal reasons. Firstly, Flan-T5 stands as an
instruction-fine-tuned iteration of T5, exhibiting enhanced performance when
compared to its peers. Secondly, Flan-T5’s manageable size renders it to be
trainable on a machine that is conducive to an academic setting. In the
following, we present results that encompass both quantitative and qualitative
aspects.

Dataset | Single-Task (Flan-T5) | Multi-Task (LogiT5) | Single Task (LogiT5)
---|---|---|---
$\alpha$ARCT 2019 | 72.31 | 77.22 | 76.74
$\alpha$NLI 2019 | 78.26 | 76.37 | 78.46
CLUTTR-Robust 2019 | 97.22 | 96.53 | 97.22
AbductionRule-Animal 2019 | 100 | 100 | 100
ANLI 2020 | 61.16 | 59.53 | 60.53
LogiQA 2021 | 37.94 | 38.56 | 39.94
LogicNLI 2021b | 82.60 | 88.40 | 88.65
ProofWriter 2021 | 99.42 | 98.85 | 99.55
Rulebert-Union 2021 | 99.69 | 99.36 | 99.70
FOLIO 2022 | 66.66 | 74.02 | 72.06
Average | 79.52 | 80.88 | 81.28

Table 2: Three training strategies for models and the performance on In-domain
Dataset.

###  4.1 In-Domain Performance

####  4.1.1 Single Task Fine-tuning

We fine-tune the Flan-T5-large on each individual dataset and the result is
presented in Table 2. We draw some interesting observations. It is apparent
that the model exhibits superior performance when operating on synthetic data
compared to hand-crafted alternatives. In support of this, the datasets that
garnered the top 5 performances are predominantly synthetic. This trend holds
even when considering the ANLI dataset, which, despite having a more
substantial training set than its synthetic counterparts, yielded inferior
results. Moreover, we ventured to explore if the model displayed a
predilection for one form of reasoning over another. Preliminary insights
suggest a potential preference towards abductive reasoning in comparison to
deductive reasoning, as evidenced in the disparity in performance between the
$\alpha$NLI and ANLI datasets – both of which are similar in terms of training
size and are hand-crafted. This, however, is a mild observation and warrants
further exploration to derive a conclusive statement. For instance, our
statistical analysis revealed that the average context length for ANLI is 105,
whereas for $\alpha$NLI, it is 66, potentially leading to varying degrees of
difficulty.

####  4.1.2 Multitask Fine-tuning

We fine-tune the Flan-T5 on all in-domain datasets utilizing a weighted
sampling technique to accommodate for the unbalanced size of the training
datasets. We find that this sampling is better than random sampling and the
comparison is given in the Appendix. We termed this model as LogiT5.

One benefit of multi-task training compared to the single task training is
that the low resources data can benefit from other tasks Parmar et al. (2022);
Luo et al. (2022a). From Table 2, it is apparent that the multi-task training
model holds a significant advantage when dealing with tasks with small
training set. It showcases higher proficiency compared to its single-task
counterpart, notably performing better by 5% and 8% on the $\alpha$ARCT and
FOLIO tasks, respectively. These datasets, characterized by their smaller
training size (limited to 1/2 K training samples), benefited notably from the
multi-task training approach. Contrastingly, the tasks with large trainig set
did not reap any benefits from multi-task training, such as $\alpha$NLI and
ANLI datasets. A potential explanation for this could be the substantial
training set already facilitates optimum learning for the model, rendering the
multi-task training approach redundant. This observation underlines a critical
limitation in leveraging multi-task training when the individual training
datasets are already sufficiently large.

####  4.1.3 Fine-tuned LogiT5 on Single Dataset

Here, we further fine-tune LogiT5 on each dataset. However, upon analyzing the
performance displayed in Table 2, we did not observe any notable advantages
from this additional fine-tuning even though small margin gains are achieved.
This suggests that LogiT5 has likely already learned the majority of knowledge
from these tasks.

Dataset | Flan-T5 | LogiT5 | LLAMA | LLAMA-CoT
---|---|---|---|---
bAbi(induction) 2015a | 59.44 | 13.12 | 33.46 | 27.06
bAbi(deduction) 2015a | 7.36 | 16.74 | 58.76 | 67.36
CLUTTR-Systematic 2019 | 15.43 | 100 | 38.82 | 40.31
AbductionRule-person 2019 | 00.00 | 95.97 | 31.31 | 43.19
ReClor 2020 | 39.80 | 46.80 | 39.80 | 39.00
Bird-Electricity 2021 | 41.29 | 65.14 | 50.00 | 45.73
NatlLang 2021 | 56.16 | 70.24 | 49.52 | 50.43
Winologic 2021 | 68.86 | 62.10 | 48.93 | 48.39
WaNLI 2022 | 50.96 | 62.28 | 32.80 | 15.54
Rulebert-Union 2021 | 27.29 | 62.92 | 87.40 | 61.50
BigBench(logical-deduction) 2022 | 49.85 | 38.00 | 22.38 | 23.23
BigBench(logical args) 2022 | 59.38 | 40.62 | 40.62 | 25.00
LogiQA 2.0 2023a | 52.66 | 55.00 | 50.74 | 50.46
PrOntoQA 2023b | 6.50 | 56.50 | 70.00 | 29.50
Average | 38.63 | 54.50 | 43.63 | 37.78

Table 3: Performance on out-domain Datasets.

###  4.2 Out-of-Domain Generation

When we study the out-of-domain generalization, we compare three models,
Flan-T5, LogiT5, and LLama-2 (7B) Touvron et al. (2023). In addition, for
Llama-2, we also study the chain-of-thought prompting Wei et al. (2022). Here,
we evaluate the model’s zero-shot capabilities rather than its few-shot in-
context learning performanceLuo et al. (2023). Investigating the latter will
be reserved for future research. More specifically, we add a prompt "let’s
think step by step" after the question. However, by our results, we do not see
the advantage of CoT prompting, probably because the model already generates
the reasoning even without such a prompt. Evaluating the LLama-2 answer poses
a challenge since the output is usually a free form and not use the exact
answer option. On the other hand Flan-T5 generate answer in a more structure
way that is easier for evaluation, probably because Flan-T5 is already trained
on instruction fine-tuned data which are already in a structured templates.
The preliminary results of LLama-2 were poor. Upon manually reviewing the
predictions, we observed that LLama-2 occasionally produces synonyms of the
ground truth. To address this, we employed ConceptNET Speer et al. (2017) to
identify synonyms and verify if the prediction aligns with any of them, a
strategy akin to the one explored in Luo et al. (2021). Furthermore, on the
babi dataset, we have seen that sometimes the llama model ignores the input
text and generates answer based on its pre-trained knowledge. This is similar
to the findings revealed in Varshney et al. (2023).

###  4.3 CoT Distillation

Flan-T5 | CoT, 3K | CoT, 6K | CoT, 15K
---|---|---|---
0.37 | 0.38 | 0.37 | 0.41

Table 4: Performance of Flan-T5 trained with CoT fine-tuning on the LogiQA
dataset. First column is without using CoT and trained on the given training
set of LogiQA.

As shown by previous work Namgyu Ho (2022), distill the chain-of-thoughts from
a large model to a small student model can boost the performance of the
student model. We apply such a CoT finetuning strategy and conduct experiments
on LogiQA, identified as the most challenging task, by distilling the CoT from
LLama-7B to Flan-T5. Initially, we generated a single answer for each
question, retaining only the samples where the predicted answer was correct,
resulting in approximately 3K valid samples. Alternatively, we created 10
answers for each question and preserved the samples with at least one correct
predicted answer, which generated a unique set of 6K questions. It is worth
noting that some questions offered multiple correct reasoning paths. In such
cases, we either opted for a singular path or utilized all available paths,
the latter approach amassing a total of 15K training samples. With the CoT
fine-tuning, we observe that the fine-tuning takes longer time and a larger
learning rate in the beginning is helpful. Thus, instead of using 1e-4 as the
learning rate, we use 3e-4. we train the model with 40 epochs. We do see that
the model performance increase when the number of epoch increase.

Following this, we trained the Flan-T5 model utilizing datasets consisting of
3K, 6K, and 15K samples, derived from the generated CoT,], with the results
delineated in Table 4. Our findings indicate that the training with 3K and 6K
samples did not enhance the CoT’s fine-tuning efficacy. However, an increased
dataset size of 15K samples facilitated a 4% improvement in performance,
suggesting that CoT distillation becomes more beneficial with a larger volume
of data.

##  5 Conclusion

In this study, we concentrate our efforts on a crucial area of research:
logical reasoning over natural language. Initially, we offer a survey to
provide a thorough comprehension of this domain, emphasizing the role of large
language models in addressing this demanding task. Following this, we assemble
a benchmark for logical reasoning named LogiGLUE, set to be publicly available
to aid forthcoming research. Finally, we refine a language model utilizing
LogiGLUE, demonstrating encouraging results across both in-domain and out-of-
domain datasets.

## References

  * Bach et al. (2017) Stephen H. Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. 2017.  Hinge-loss markov random fields and probabilistic soft logic.  _J. Mach. Learn. Res._ , 18(1):3846–3912.
  * Banerjee et al. (2020) Pratyay Banerjee, Chitta Baral, Man Luo, Arindam Mitra, Kuntal Pal, Tran C Son, and Neeraj Varshney. 2020.  Can transformers reason about effects of actions?  _arXiv:2012.09938_.
  * Bhagavatula et al. (2019) Chandra Bhagavatula, Ronan Le Bras, Chaitanya Malaviya, Keisuke Sakaguchi, Ari Holtzman, Hannah Rashkin, Doug Downey, Wen-tau Yih, and Yejin Choi. 2019.  Abductive commonsense reasoning.  In _International Conference on Learning Representations_.
  * Bowman et al. (2015) Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Byrne et al. (2019) Ruth MJ Byrne, Jonathan St BT Evans, and Stephen E Newstead. 2019.  _Human reasoning: The psychology of deduction_.  Psychology Press.
  * Chan et al. (2023) Chunkit Chan, Xin Liu, Tsz Ho Chan, Jiayang Cheng, Yangqiu Song, Ginny Wong, and Simon See. 2023.  Self-consistent narrative prompts on abductive natural language inference.  _arXiv preprint arXiv:2309.08303_.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022.  Palm: Scaling language modeling with pathways.  _arXiv preprint arXiv:2204.02311_.
  * Clark et al. (2020) Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020.  Transformers as soft reasoners over language.  In _IJCAI_.
  * Colmerauer and Roussel (1996) Alain Colmerauer and Philippe Roussel. 1996.  The birth of prolog.  In _History of programming languages—II_.
  * Creswell et al. (2022) Antonia Creswell, Murray Shanahan, and Irina Higgins. 2022.  Selection-inference: Exploiting large language models for interpretable logical reasoning.  In _The Eleventh International Conference on Learning Representations_.
  * Dasgupta et al. (2022) Ishita Dasgupta, Andrew K Lampinen, Stephanie CY Chan, Antonia Creswell, Dharshan Kumaran, James L McClelland, and Felix Hill. 2022.  Language models show human-like content effects on reasoning.  _arXiv preprint arXiv:2207.07051_.
  * De Raedt et al. (2007) Luc De Raedt, Angelika Kimmig, and Hannu Toivonen. 2007.  Problog: A probabilistic prolog and its application in link discovery.  In _Proceedings of the 20th International Joint Conference on Artifical Intelligence_ , IJCAI’07, page 2468–2473, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
  * Devlin et al. (2019a) J. Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019a.  Bert: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL-HLT_.
  * Devlin et al. (2019b) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019b.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _NAACL_ , Minneapolis, Minnesota. ACL.
  * Evans et al. (2018) Richard Evans, David Saxton, David Amos, and et al. 2018.  Can neural networks understand logical entailment?  In _ICLR_.
  * Fangkai Jiao (2022) Xuemeng Song Liqiang Nie Fangkai Jiao, Yangyang Guo. 2022.  Merit: Meta-path guided contrastive learning for logical reasoning.  _arXiv preprint arXiv:2203.00357_.
  * Gokhale et al. (2022) Tejas Gokhale, Swaroop Mishra, Man Luo, Bhavdeep Sachdeva, and Chitta Baral. 2022\.  Generalized but not robust? comparing the effects of data modification methods on out-of-domain generalization and adversarial robustness.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pages 2705–2718.
  * Guan et al. (2023) Lin Guan, Karthik Valmeekam, Sarath Sreedharan, and Subbarao Kambhampati. 2023.  Leveraging pre-trained large language models to construct and utilize world models for model-based task planning.  _arXiv preprint arXiv:2305.14909_.
  * Hahn et al. (2020) Christopher Hahn, Frederik Schmitt, Jens U Kreber, Markus Norman Rabe, and Bernd Finkbeiner. 2020.  Teaching temporal logics to neural networks.  In _ICLR_.
  * Han et al. (2022) Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Luke Benson, Lucy Sun, Ekaterina Zubova, Yujie Qiao, Matthew Burtell, et al. 2022.  Folio: Natural language reasoning with first-order logic.  _arXiv preprint arXiv:2209.00840_.
  * Han et al. (2023) Simon Jerome Han, Keith J Ransom, Andrew Perfors, and Charles Kemp. 2023.  Inductive reasoning in humans and large language models.  _Cognitive Systems Research_ , page 101155.
  * Hanlin Zhang1 (2022) Li Erran Li3 Eric Xing Hanlin Zhang1, Yi-Fan Zhang2. 2022.  The impact of symbolic representations on in-context learning for few-shot reasoning.  _arXiv preprint arXiv:2212.08686_.
  * He et al. (2021) Weinan He, Canming Huang, Yongmei Liu, and Xiaodan Zhu. 2021.  WinoLogic: A zero-shot logic-based diagnostic dataset for Winograd Schema Challenge.  In _EMNLP_.
  * Heit (2007) Evan Heit. 2007.  _What Is Induction and Why Study It?_, page 1–24. Cambridge University Press.
  * Helwe et al. (2022) Chadi Helwe, Chloé Clavel, and Fabian Suchanek. 2022.  Logitorch: A pytorch-based library for logical reasoning on natural language.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_.
  * Hobbs et al. (1993) Jerry R Hobbs, Mark E Stickel, Douglas E Appelt, and Paul Martin. 1993.  Interpretation as abduction.  _Artificial intelligence_ , 63(1-2):69–142.
  * Huang et al. (2019) Lifu Huang, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019.  Cosmos QA: Machine reading comprehension with contextual commonsense reasoning.  In _EMNLP-IJCNLP_.
  * Ignatiev et al. (2019) Alexey Ignatiev, Nina Narodytska, and Joao Marques-Silva. 2019.  Abduction-based explanations for machine learning models.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, pages 1511–1519.
  * Jiao et al. (2023) Fangkai Jiao, Zhiyang Teng, Shafiq Joty, Bosheng Ding, Aixin Sun, Zhengyuan Liu, and Nancy F Chen. 2023.  Logicllm: Exploring self-supervised logic-enhanced training for large language models.  _arXiv preprint arXiv:2305.13718_.
  * Joshi et al. (2020) Pratik Joshi, Somak Aditya, Aalok Sathe, and Monojit Choudhury. 2020.  TaxiNLI: Taking a ride up the NLU hill.  In _CoNLL_.
  * Kaiyu Yang and Chen (2022) Jia Deng Kaiyu Yang and Danqi Chen. 2022.  Generating natural language proofs with verifier-guided search.  _arXiv preprint arXiv:2205.12443_.
  * Lanham et al. (2023) Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, Kamile Lukosiute, Karina Nguyen, Newton Cheng, Nicholas Joseph, Nicholas Schiefer, Oliver Rausch, Robin Larson, Sam McCandlish, Sandipan Kundu, Saurav Kadavath, Shannon Yang, Thomas Henighan, Timothy Maxwell, Timothy Telleen-Lawton, Tristan Hume, Zac Hatfield-Dodds, Jared Kaplan, Jan Brauner, Samuel R. Bowman, and Ethan Perez. 2023.  Measuring faithfulness in chain-of-thought reasoning.  _CoRR_ , abs/2307.13702.
  * Lee and Luo (2018) Joohyung Lee and Man Luo. 2018.  Strong equivalence for lpmln programs.  In _35th International Conference on Logic Programming (ICLP 2019)_.
  * Lee and Wang (2016) Joohyung Lee and Yi Wang. 2016.  Weighted rules under the stable model semantics.  In _KRR_.
  * Levesque et al. (2012) Hector Levesque, Ernest Davis, and Leora Morgenstern. 2012.  The winograd schema challenge.  In _KRR_.
  * Lifschitz (2019) Vladimir Lifschitz. 2019.  _Answer set programming_.  Springer Berlin.
  * Lin et al. (2019) Kevin Lin, Oyvind Tafjord, Peter Clark, and Matt Gardner. 2019.  Reasoning over paragraph effects in situations.  In _MRQA_.
  * Liu et al. (2022) Alisa Liu, Swabha Swayamdipta, Noah A Smith, and Yejin Choi. 2022.  Wanli: Worker and ai collaboration for natural language inference dataset creation.  _arXiv preprint arXiv:2201.05955_.
  * Liu et al. (2023a) Hanmeng Liu, Jian Liu, Leyang Cui, Zhiyang Teng, Nan Duan, Ming Zhou, and Yue Zhang. 2023a.  Logiqa 2.0—an improved dataset for logical reasoning in natural language understanding.  _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , 31:2947–2962.
  * Liu et al. (2023b) Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. 2023b.  Evaluating the logical reasoning ability of chatgpt and gpt-4.  _arXiv preprint arXiv:2304.03439_.
  * Liu et al. (2023c) Hanmeng Liu, Zhiyang Teng, Leyang Cui, Chaoli Zhang, Qiji Zhou, and Yue Zhang. 2023c.  Logicot: Logical chain-of-thought instruction-tuning data collection with gpt-4.  _arXiv preprint arXiv:2305.12147_.
  * Liu et al. (2020) Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2020\.  Logiqa: A challenge dataset for machine reading comprehension with logical reasoning.  In _IJCAI_.
  * Liu et al. (2021) Jian Liu, Leyang Cui, Hanmeng Liu, Dandan Huang, Yile Wang, and Yue Zhang. 2021\.  Logiqa: a challenge dataset for machine reading comprehension with logical reasoning.  In _Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence_ , pages 3622–3628.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, and et al. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv:1907.11692_.
  * Lourie et al. (2021) Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021.  Unicorn on rainbow: A universal commonsense reasoning model on a new multitask benchmark.  _arXiv preprint arXiv:2103.13009_.
  * Lu et al. (2023) Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. 2023.  Chameleon: Plug-and-play compositional reasoning with large language models.  _arXiv preprint arXiv:2304.09842_.
  * Luo et al. (2022a) Man Luo, Kazuma Hashimoto, Semih Yavuz, Zhiwei Liu, Chitta Baral, and Yingbo Zhou. 2022a.  Choose your qa model wisely: A systematic study of generative and extractive readers for question answering.  _Spa-NLP 2022_ , page 7.
  * Luo et al. (2021) Man Luo, Shailaja Keyur Sampat, Riley Tallman, Yankai Zeng, Manuha Vancha, Akarshan Sajja, and Chitta Baral. 2021.  ‘just because you are right, doesn’t mean i am wrong’: Overcoming a bottleneck in development and evaluation of open-ended vqa tasks.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2766–2771.
  * Luo et al. (2022b) Man Luo, Sharad Saxena, Swaroop Mishra, Mihir Parmar, and Chitta Baral. 2022b.  Biotabqa: Instruction learning for biomedical table question answering.  In _CEUR Workshop Proceedings_ , volume 3180, pages 291–304. CEUR-WS.
  * Luo et al. (2023) Man Luo, Xin Xu, Zhuyun Dai, Panupong Pasupat, Mehran Kazemi, Chitta Baral, Vaiva Imbrasaite, and Vincent Y Zhao. 2023.  Dr. icl: Demonstration-retrieved in-context learning.  _arXiv preprint arXiv:2305.14128_.
  * McCann et al. (2018) Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018.  The natural language decathlon: Multitask learning as question answering.  _arXiv preprint arXiv:1806.08730_.
  * McCarthy (1989) John McCarthy. 1989.  Artificial intelligence, logic and formalizing common sense.  In _Philosophical logic and artificial intelligence_. Springer.
  * Mishra et al. (2022) Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022.  Cross-task generalization via natural language crowdsourcing instructions.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 3470–3487.
  * Namgyu Ho (2022) Se-Young Yun Namgyu Ho, Laura Schmid. 2022.  Large language models are reasoning teachers.  _arXiv preprint arXiv:2212.10071_.
  * Nie et al. (2020) Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020.  Adversarial nli: A new benchmark for natural language understanding.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4885–4901.
  * Niven and Kao (2019) Timothy Niven and Hung-Yu Kao. 2019.  Probing neural network comprehension of natural language arguments.  In _ACL_.
  * Pan et al. (2023) Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023.  Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning.  _arXiv preprint arXiv:2305.12295_.
  * Paranjape et al. (2023) Bhargavi Paranjape, Scott Lundberg, Sameer Singh, Hannaneh Hajishirzi, Luke Zettlemoyer, and Marco Tulio Ribeiro. 2023.  Art: Automatic multi-step reasoning and tool-use for large language models.  _arXiv preprint arXiv:2303.09014_.
  * Parmar et al. (2022) Mihir Parmar, Swaroop Mishra, Mirali Purohit, Man Luo, Murad Mohammad, and Chitta Baral. 2022.  In-boxbart: Get instructions into biomedical multi-task learning.  In _Findings of the Association for Computational Linguistics: NAACL 2022_ , pages 112–128.
  * Paul (1993) Gabriele Paul. 1993.  Approaches to abductive reasoning: an overview.  _Artificial intelligence review_ , 7(2):109–152.
  * Poliak et al. (2018) Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, and Benjamin Van Durme. 2018.  Collecting diverse natural language inference problems for sentence representation evaluation.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 67–81, Brussels, Belgium. Association for Computational Linguistics.
  * (63) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.  Language models are unsupervised multitask learners.
  * Richardson and Sabharwal (2021) Kyle Richardson and Ashish Sabharwal. 2021.  Pushing the limits of rule reasoning in transformers through natural language satisfiability.  _arXiv:2112.09054_.
  * Richardson and Domingos (2006) Matthew Richardson and Pedro Domingos. 2006.  Markov logic networks.  _Mach. Learn._ , 62(1–2):107–136.
  * Rogers et al. (2020) Anna Rogers, Olga Kovaleva, Matthew Downey, and Anna Rumshisky. 2020.  Getting closer to ai complete question answering: A set of prerequisite real tasks.  In _AAAI_ , volume 34.
  * Saeed et al. (2021) Mohammed Saeed, Naser Ahmadi, Preslav Nakov, and Paolo Papotti. 2021.  RuleBERT: Teaching soft rules to pre-trained lms.  In _EMNLP_.
  * Saha et al. (2020) Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava, and Mohit Bansal. 2020.  Prover: Proof generation for interpretable reasoning over rules.  In _EMNLP_.
  * Sanyal et al. (2022) Soumya Sanyal, Yichong Xu, Shuohang Wang, Ziyi Yang, Reid Pryzant, Wenhao Yu, Chenguang Zhu, and Xiang Ren. 2022.  Apollo: A simple approach for adaptive pretraining of language models for logical reasoning.  _arXiv preprint arXiv:2212.09282_.
  * Sap et al. (2020) Maarten Sap, Vered Shwartz, Antoine Bosselut, Yejin Choi, and Dan Roth. 2020.  Introductory tutorial: Commonsense reasoning for natural language processing.  _ACL 2020_ , page 27.
  * Saparov and He (2023a) Abulhair Saparov and He He. 2023a.  Language models are greedy reasoners: A systematic formal analysis of chain-of-thought.  In _The Eleventh International Conference on Learning Representations_.
  * Saparov and He (2023b) Abulhair Saparov and He He. 2023b.  Language models are greedy reasoners: A systematic formal analysis of chain-of-thought.  In _The Eleventh International Conference on Learning Representations_.
  * Sauce and Matzel (2017) Bruno Sauce and Louis D Matzel. 2017.  Inductive reasoning.  _Encyclopedia of animal cognition and behavior_ , 6:1–8.
  * Schick et al. (2023) Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023.  Toolformer: Language models can teach themselves to use tools.  _arXiv preprint arXiv:2302.04761_.
  * Selsam et al. (2019) Daniel Selsam, Matthew Lamm, Benedikt Bünz, Percy Liang, Leonardo de Moura, and David L Dill. 2019.  Learning a sat solver from single-bit supervision.  In _ICLR_.
  * Sinha et al. (2019) Koustuv Sinha, Shagun Sodhani, Jin Dong, and et al. 2019.  Clutrr: A diagnostic benchmark for inductive reasoning from text.  In _EMNLP_.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 31.
  * Srivastava et al. (2022) Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022.  Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.  _arXiv preprint arXiv:2206.04615_.
  * Srivastava et al. (2023) Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2023.  Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.  _Transactions on Machine Learning Research_.
  * Sumers et al. (2023) Theodore Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L Griffiths. 2023.  Cognitive architectures for language agents.  _arXiv preprint arXiv:2309.02427_.
  * Tafjord et al. (2021) Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. 2021.  ProofWriter: Generating implications, proofs, and abductive statements over natural language.  In _Findings-ACL-IJCNLP_.
  * Tafjord et al. (2019) Oyvind Tafjord, Matt Gardner, Kevin Lin, and Peter Clark. 2019.  Quartz: An open-domain dataset of qualitative relation questions.  _arXiv:1909.03553_.
  * Tian et al. (2021a) Jidong Tian, Yitian Li, Wenqing Chen, Liqiang Xiao, Hao He, and Yaohui Jin. 2021a.  Diagnosing the first-order logical reasoning ability through LogicNLI.  In _EMNLP_.
  * Tian et al. (2021b) Jidong Tian, Yitian Li, Wenqing Chen, Liqiang Xiao, Hao He, and Yaohui Jin. 2021b.  Diagnosing the first-order logical reasoning ability through logicnli.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 3738–3747.
  * Touvron et al. (2023) Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023.  Llama 2: Open foundation and fine-tuned chat models.  _arXiv preprint arXiv:2307.09288_.
  * Valmeekam et al. (2022) Karthik Valmeekam, Alberto Olmo, Sarath Sreedharan, and Subbarao Kambhampati. 2022\.  Large language models still can’t plan (a benchmark for llms on planning and reasoning about change).  In _NeurIPS 2022 Foundation Models for Decision Making Workshop_.
  * Valmeekam et al. (2023) Karthik Valmeekam, Sarath Sreedharan, Matthew Marquez, Alberto Olmo, and Subbarao Kambhampati. 2023.  On the planning abilities of large language models (a critical investigation with a proposed benchmark).  _arXiv preprint arXiv:2302.06706_.
  * Varshney et al. (2023) Neeraj Varshney, Mihir Parmar, Nisarg Patel, Divij Handa, Sayantan Sarkar, Man Luo, and Chitta Baral. 2023.  Can nlp models correctly reason over contexts that break the common assumptions?  _arXiv preprint arXiv:2305.12096_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, and et al. 2017.  Attention is all you need.  In _NeurIPS_ , volume 30. Curran Associates, Inc.
  * Wang et al. (2019) Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019.  Superglue: A stickier benchmark for general-purpose language understanding systems.  _Advances in neural information processing systems_ , 32.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in Neural Information Processing Systems_ , 35:24824–24837.
  * Weston et al. (2015a) Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart Van Merriënboer, Armand Joulin, and Tomas Mikolov. 2015a.  Towards ai-complete question answering: A set of prerequisite toy tasks.  _arXiv preprint arXiv:1502.05698_.
  * Weston et al. (2015b) Jason Weston, Antoine Bordes, and et al. 2015b.  Towards ai-complete question answering: A set of prerequisite toy tasks.  _arXiv:1502.05698_.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122. Association for Computational Linguistics.
  * Wu et al. (2023) Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. 2023.  Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks.  _CoRR_ , abs/2307.02477.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, and et al. Carbonell. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  In _NeurIPS_ , volume 32. Curran Associates, Inc.
  * Yang et al. (2023) Zonglin Yang, Xinya Du, Rui Mao, Jinjie Ni, and Erik Cambria. 2023.  Logical reasoning over natural language as knowledge representation: A survey.  _arXiv preprint arXiv:2303.12023_.
  * Yu et al. (2023) Fei Yu, Hongbo Zhang, and Benyou Wang. 2023.  Nature language reasoning, a survey.  _arXiv preprint arXiv:2303.14725_.
  * Yu et al. (2019) Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. 2019.  Reclor: A reading comprehension dataset requiring logical reasoning.  In _ICLR_.
  * Yu et al. (2020) Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. 2020.  Reclor: A reading comprehension dataset requiring logical reasoning.  In _International Conference on Learning Representations_.
  * Zhang et al. (2022) Honghua Zhang, Liunian Harold Li, Tao Meng, Kai-Wei Chang, and Guy Van den Broeck. 2022.  On the paradox of learning to reason from data.  _arXiv preprint arXiv:2205.11502_.