# Think about it! Improving defeasible reasoning by first modeling the
question scenario

Aman Madaan, Niket Tandon†, Dheeraj Rajagopal, Peter Clark†,
Yiming Yang, Eduard Hovy
Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA,
USA
† Allen Institute for Artificial Intelligence, Seattle, WA, USA
<EMAIL_ADDRESS>
{nikett<EMAIL_ADDRESS>

###### Abstract

Defeasible reasoning is the mode of reasoning where conclusions can be
overturned by taking into account new evidence. Existing cognitive science
literature on defeasible reasoning suggests that a person forms a mental model
of the problem scenario before answering questions. Our research goal asks
whether neural models can similarly benefit from envisioning the question
scenario before answering a defeasible query. Our approach is, given a
question, to have a model first create a graph of relevant influences, and
then leverage that graph as an additional input when answering the question.
Our system, curious, achieves a new state-of-the-art on three different
defeasible reasoning datasets. This result is significant as it illustrates
that performance can be improved by guiding a system to “think about” a
question and explicitly model the scenario, rather than answering
reflexively.111Code and data located at https://github.com/madaan/thinkaboutit

##  1 Introduction

Defeasible inference is a mode of reasoning where additional information can
modify conclusions Koons (2017). Here we consider the specific formulation and
challenge in Rudinger et al. (2020): Given that some premise $\mathbf{P}$
plausibly implies a hypothesis $\mathbf{H}$, does new information that the
situation is $\mathbf{S}$ weaken or strengthen the conclusion $\mathbf{H}$?
For example, consider the premise “The drinking glass fell” with a possible
implication “The glass broke”. New information that “The glass fell on a
pillow” here weakens the implication.

Figure 1: curious improves defeasible reasoning by modeling the question
scenario with an inference graph $G$ adapted from cognitive science
literature. The graph is encoded judiciously using our graph encoder $h(.)$,
resulting in end task performance improvement.

We borrow ideas from the cognitive science literature that supports defeasible
reasoning for humans with an _inference graph_ (Pollock, 2009, 1987).
Inference graphs formulation in Madaan et al. (2021), which we use in this
paper, draws connections between the $\mathbf{P}$, $\mathbf{H}$, and
$\mathbf{S}$ through mediating events. This can be seen as a _mental model_ of
the question scenario before answering the question Johnson-Laird (1983). This
paper asks the natural question: can modeling the question scenario with
inference graphs help machines in defeasible reasoning?

Our approach is as follows. First, given a question, generate an inference
graph describing important influences between question elements. Then, use
that graph as an additional input when answering the defeasible reasoning
query. Our proposed system, curious, comprises a graph generation module and a
graph encoding module to use the generated graph for the query (Figure 2).

To generate inference graphs, we build upon past work that uses a sequence to
sequence approach Madaan et al. (2021). However, our analysis revealed that
the graphs can often be erroneous, and curious also includes an error
correction module to generate higher quality inference graphs. This was
important because we found that better graphs are more helpful in the
downstream QA task.

Figure 2: An overview of curious

The generated inference graph is then used for the QA task on three existing
defeasible inference datasets from diverse domains, viz., $\delta$-snli
(natural language inference) Bowman et al. (2015), $\delta$-social (reasoning
about social norms) Forbes et al. (2020), and $\delta$-atomic (commonsense
reasoning) Sap et al. (2019). We show that the way the graph is encoded for
input is important. If we simply augment the question with the generated
graphs, there are some gains on all datasets. However, the accuracy improves
substantially across all datasets with a more judicious encoding of the graph-
augmented question that accounts for interactions between the graph nodes. To
achieve this, we use the mixture of experts approach Jacobs et al. (1991) to
include a mixture of experts layers during encoding, enabling the ability to
attend to specific nodes while capturing their interactions selectively.

In summary, our contribution is in drawing on the idea of an inference graph
from cognitive science to show benefits in a defeasible inference QA task.
Using an error correction module in the graph generation process, and a
judicious encoding of the graph augmented question, curious achieves a new
state-of-the-art over three defeasible datasets. This result is significant
also because our work illustrates that guiding a system to “think about" a
question before answering can improve performance.

Figure 3: An overview of our method to perform graph-augmented defeasible
reasoning using a hierarchical mixture of experts. First, moe-v selectively
pools the node representations to generate a representation
$\mathbf{h}_{\mathbf{G}}$ of the inference graph. Then, moe-gx pools the query
representation $\mathbf{h}_{\mathbf{x}}$ and the graph representation
generated by moe-v to pass to the upstream classifier.

##  2 Task

We use the defeasible inference task and datasets defined in Rudinger et al.
(2020), namely given an input $\mathbf{x}$ =
($\mathbf{P}$,$\mathbf{H}$,$\mathbf{S}$), predict the output
$\mathbf{y}\in\\{strengthens,weakens\\}$, where $\mathbf{P}$, $\mathbf{H}$,
and $\mathbf{S}$ are sentences describing a premise, hypothesis, and scenario
respectively, and $y$ denotes whether $S$ strengthens/weakens the plausible
conclusion that $\mathbf{H}$ follows from $\mathbf{P}$, as described in
Section 1.

##  3 Approach

Inspired by past results Madaan et al. (2021) that humans found inference
graphs useful for defeasible inference, we investigate whether neural models
can benefit from envisioning the question scenario using an inference graph
before answering a defeasible inference query.

##### Inference graphs

As inference graphs are central to our work, we give a brief description of
their structure next. Inference graphs were introduced in philosophy by
Pollock (2009) to aid defeasible reasoning for humans, and in NLP by Tandon et
al. (2019) for a counterfactual reasoning task. We interpret the inference
graphs as having four kinds of nodes Pollock (2009); Madaan et al. (2021):

  * i.

Contextualizers (C-, C+): these nodes set the context around a situation and
connect to the $\mathbf{P}$.

  * ii.

Situations (S, S-): these nodes are new situations that emerge which might
overturn an inference.

  * iii.

Hypothesis (H-, H+): Hypothesis nodes describe the outcome/conclusion of the
situation.

  * iv.

Mediators (M-, M+): Mediators are nodes that help bridge the knowledge gap
between a situation and a hypothesis node by explaining their connection
explicitly. These node can either act as a _weakener_ or _strengthener_.

Each node in an influence graph is labeled with an event (a sentence or a
phrase). The signs - and + capture the nature of the influence event node.
Concrete examples are present in Figures 1, 4, and in Appendix §D.

###  3.1 Overview of curious

Our system, curious, comprises three components, (i) a graph generator
$\textsc{gen}_{\text{init}}$, (ii) a graph corrector
$\textsc{gen}_{\text{corr}}$, (iii) a graph encoder (Figure 1).
$\textsc{gen}_{\text{init}}$ generates an inference graph from the input
$\mathbf{x}$. We borrow the sequence to sequence approach of
$\textsc{gen}_{\text{init}}$ from Madaan et al. (2021) without any
architectural changes. However, we found that the resulting graphs can often
be erroneous (which hurts task performance), so curious includes an error
correction module $\textsc{gen}_{\text{corr}}$ to generate higher quality
inference graphs that are then judiciously encoded using the graph encoder.
This encoded representation is then passed through a classifier to generate an
end task label. The overall architecture is shown in Figure 2.

###  3.2 Graph generator

Figure 4: The graphs generated by $\textsc{gen}_{\text{init}}$.The input graph
has repetitions for nodes $\\{C{-},C{+}\\}$ and $\\{S,S{-}\\}$. The corrected
graph generated by $\textsc{gen}_{\text{corr}}$ replaces the repetitions with
meaningful labels.

As the initial graph generator, we use the method described in Madaan et al.
(2021) ($\textsc{gen}_{\text{init}}$) to generate inference graphs for
defeasible reasoning.222We use their publicly available code and data Their
approach involves first training a graph-generation module, and then
performing zero-shot inference on a defeasible query to obtain an inference
graph. They obtain training data for the graph-generation module from wiqa
dataset Tandon et al. (2019). wiqa is a dataset of 2107
$(\mathbf{T}_{i},\mathbf{G}_{i})$ tuples, where $\mathbf{T}_{i}$ is the
passage text that describes a process (e.g., waves hitting a beach), and
$\mathbf{G}_{i}$ is the corresponding influence graph. The graph generator
$\textsc{gen}_{\text{init}}$ is trained as a seq2seq model, by setting
$\texttt{input}=\text{[Premise] }\mathbf{T}_{i}\mid\text{[Situation]
}\mathbf{S}_{i}\mid\text{[Hypothesis] }\mathbf{H}_{i}$, and
$\texttt{output}=\mathbf{G}_{i}$. Note that $\mathbf{S}_{i}$ and
$\mathbf{H}_{i}$ are nodes in the influence graph $\mathbf{G}_{i}$, allowing
grounded generation. [Premise], [Situation], [Hypothesis] are special tokens
used to demarcate the input.

###  3.3 Graph corrector

We found that 70% of the randomly sampled 100 graphs produced by
$\textsc{gen}_{\text{init}}$ (undesirably) had repeated nodes (an example of
repeated nodes is in Figure 4). Repeated nodes introduce noise because they
violate the semantic structure of a graph, e.g., in Figure 4, nodes C+ and C-
are repeated, although they are expected to have opposite semantics. Higher
graph quality yields better end task performance when using inference graphs
(as we will show in §4.3.1)

To repair such problems, we train a graph corrector,
$\textsc{gen}_{\text{corr}}$, that takes as input $\mathbf{G}^{\prime}$, and
as output it gives a graph $\mathbf{G}^{*}$, with repetitions fixed. To train
the model, we require ($\mathbf{G}^{\prime}$, $\mathbf{G}^{*}$) examples,
which we generate using a data augmentation technique described in the
Appendix §A. Because the nodes in the graph are from an open vocabulary, we
then train a T5 sequence-to-sequence model Raffel et al. (2020) with input =
$\mathbf{G}^{\prime}$ and output = $\mathbf{G}^{*}$. In summary, given a
defeasible query $\mathbf{PHS}$, we generate a potentially incorrect initial
graph $\mathbf{G}^{\prime}$ using $\textsc{gen}_{\text{init}}$. We then feed
$\mathbf{G}^{\prime}$ to $\textsc{gen}_{\text{corr}}$ to obtain an improved
graph $\mathbf{G}$.

###  3.4 Graph Encoder

For each defeasible query ($\mathbf{P}$, $\mathbf{H}$, $\mathbf{S}$), we add
the inference graph $\mathbf{G}$ from curious (the corrected graph from §3.3),
to provide additional context for the query, as we now describe.

We concatenate the components ($\mathbf{P}$, $\mathbf{H}$, $\mathbf{S}$) of
the defeasible query into a single sequence of tokens
$\mathbf{x}=(\mathbf{P}\|\mathbf{H}\|\mathbf{S})$, where $\|$ denotes
concatenation. Thus, each sample of our graph-augmented binary-classification
task takes the form $((\mathbf{x},\mathbf{G}),\mathbf{y})$, where
$\mathbf{y}\in\\{\text{{strengthener}, {weakener}}\\}$. Following Madaan et
al. (2021), we do not use edge labels and treat all the graphs as undirected
graphs.

##### Overview:

We first use a language model $\mathcal{L}$ to obtain a dense representation
$\mathbf{h}_{\mathbf{x}}$ for the defeasible query $\mathbf{x}$, and a dense
representation $\mathbf{h}_{\mathbf{v}}$ for each node
$\mathbf{v}\in\mathbf{G}$. The node representations $\mathbf{h}_{\mathbf{v}}$
are then pooled using a hierarchical mixture of experts (MoE) to obtain a
graph representation $\mathbf{h}_{\mathbf{G}}$. The query representation
$\mathbf{h}_{\mathbf{x}}$ and the graph representation
$\mathbf{h}_{\mathbf{G}}$ are combined to solve the defeasible task. We now
provide details on obtaining $\mathbf{h}_{\mathbf{x}}$,
$\mathbf{h}_{\mathbf{v}}$, $\mathbf{h}_{\mathbf{G}}$.

####  3.4.1 Encoding the query and nodes

Let $\mathcal{L}$ be a pre-trained language model (in our case RoBERTa Liu et
al. (2019)). We use
$\mathbf{h}_{\mathbf{S}}=\mathcal{L}(\mathbf{S})\in\mathbb{R}^{d}$ to denote
the dense representation of sequence of tokens $\mathbf{S}$ returned by the
language model $\mathcal{L}$. Specifically, we use the pooled representation
of the beginning-of-sequence token <s> as the sequence representation.

We encode the defeasible query $\mathbf{x}$ and the nodes of the graph using
$\mathcal{L}$. Query representation is computed as
$\mathbf{h}_{\mathbf{x}}=\mathcal{L}(\mathbf{x})$, and we similarly obtain a
matrix of node representations $\mathbf{h}_{\mathbf{V}}$ by encoding each node
$\mathbf{v}$ in $\mathbf{G}$ with $\mathcal{L}$ as follows:

$\displaystyle\mathbf{h}_{\mathbf{V}}=[\mathbf{h}_{v_{1}};\mathbf{h}_{v_{2}};\ldots;\mathbf{h}_{|\mathbf{V}|}]$
(1)

where $\mathbf{h}_{v_{i}}\in\mathbb{R}^{d}$ refers to the dense representation
for the $i^{th}$ node of $\mathbf{G}$ derived from $\mathcal{L}$ (i.e.,
$\mathbf{h}_{v_{i}}=\mathcal{L}(v_{i})$), and
$\mathbf{h}_{\mathbf{V}}\in\mathbb{R}^{|\mathbf{V}|\times d}$ to refer to the
matrix of node representations.

####  3.4.2 Graph representations using MoE

Recently, mixture-of-experts Jacobs et al. (1991); Shazeer et al. (2017);
Fedus et al. (2021) has emerged as a promising method of combining multiple
feature types. Mixture-of-experts (MoE) is especially useful when the input
consists of multiple facets, where each facet has a specific semantic meaning.
Previously, Gu et al. (2018); Chen et al. (2019) have used the ability of MoE
to pool disparate features on low-resource and cross-lingual language tasks.
Since each node in the inference graphs used by us plays a specific role in
defeasible reasoning (contextualizer, situation node, or mediator), we take
inspiration from these works to design a hierarchical MoE model Jordan and Xu
(1995) to pool node representations $\mathbf{h}_{\mathbf{V}}$ into a graph
representation $\mathbf{h}_{\mathbf{G}}$.

An MoE consists of $n$ expert networks
$\mathbf{E_{1}},\mathbf{E_{2}},\ldots,\mathbf{E_{n}}$ and a gating network
$\mathbf{M}$. Given an input $\mathbf{x}\in\mathbb{R}^{d}$, each expert
network $\mathbf{E_{i}}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{k}$ learns a
transform over the input. The gating network
$\mathbf{M}:\mathbb{R}^{d}\rightarrow\Delta^{d}$ gives the weights
$\mathbf{p}=\\{p_{1},p_{2},\ldots,p_{n}\\}$ to combine the expert outputs for
input $\mathbf{x}$. Finally, the output $\mathbf{y}$ is returned as a convex
combination of the expert outputs:

$\displaystyle\mathbf{p}$ $\displaystyle=\mathbf{M}(\mathbf{x})$
$\displaystyle\mathbf{y}$ $\displaystyle=\sum_{i=1}^{n}p_{i}\mathbf{E_{i}(x)}$
(2)

The output $\mathbf{y}$ can either be the logits for an end task Shazeer et
al. (2017); Fedus et al. (2021) or pooled features that are passed to a
downstream learner Chen et al. (2019); Gu et al. (2018). The gating network
$\mathbf{M}$ and the expert networks
$\mathbf{E_{1}},\mathbf{E_{2}},\ldots,\mathbf{E_{n}}$ are trained end-to-end.
During learning, the gradients to $\mathbf{M}$ train it to generate a
distribution over the experts that favors the best expert for a given input.
Appendix §B presents a further discussion on our MoE formulation and an
analysis of the gradients.

##### Hierarchical MoE for defeasible reasoning

Different parts of the inference graphs might help answer a query to a
different degree. Further, for certain queries, graphs might not be helpful
(and could even be distracting), and the model could rely primarily on the
input query alone. This motivates a two-level architecture that can: (i)
select a subset of the nodes in the graph and ii) selectively reason across
the query and the graph to varying degrees.

Given these requirements, a hierarchical MoE Jordan and Jacobs (1994) model
presents itself as a natural choice to model this task. The first MoE (moe-v)
creates a graph representation by taking a convex combination of the node
representations. The second MoE (moe-gx) then takes a convex-combination of
the graph representation returned by moe-v and query representation and passes
it to an MLP for the downstream task.

  * $\bullet$

moe-v consists of five node-experts and gating network to selectively pool
node representations $\mathbf{h}_{\mathbf{v}}$ to graph representation
$\mathbf{h}_{\mathbf{G}}$:

$\displaystyle\mathbf{p}$ $\displaystyle=\mathbf{M}(\mathbf{h}_{\mathbf{V}})$
$\displaystyle\mathbf{h}_{\mathbf{G}}$
$\displaystyle=\sum_{v\in\mathbf{V}}p_{v}E_{v}(v)$ (3)

  * $\bullet$

moe-gx contains two experts (graph expert $E_{\mathbf{G}}$ and question expert
$E_{\mathbf{Q}}$) and a gating network to combine the graph representation
$\mathbf{h}_{\mathbf{G}}$ returned by moe-gx and the query representation
$\mathbf{h}_{\mathbf{x}}$:

$\displaystyle\mathbf{p}$
$\displaystyle=\mathbf{M}([\mathbf{h}_{\mathbf{G}};\mathbf{h}_{\mathbf{Q}}])$
$\displaystyle\mathbf{h}_{\mathbf{y}}$
$\displaystyle=E_{\mathbf{G}}(\mathbf{h}_{\mathbf{G}})+E_{\mathbf{Q}}(\mathbf{h}_{\mathbf{Q}})$
(4)

$\mathbf{h}_{\mathbf{y}}$ is then passed to a 1-layer MLP to perform
classification. The gates and the experts in our MoE model are single-layer
MLPs, with equal input and output dimensions for the experts.

##  4 Experiments

In this section, we empirically investigate if curious can improve defeasible
inference by first modeling the question scenario using inference graphs. We
also study the reasons for any improvements.

###  4.1 Experimental setup

Dataset | Split | # Samples | Total
---|---|---|---
$\delta$-atomic | train | 35,001 | 42,977
test | 4137
dev | 3839
$\delta$-social | train | 88,675 | 92,295
test | 1836
dev | 1784
$\delta$-snli | train | 77,015 | 95,795
test | 9438
dev | 9342
Table 1: Number of samples in each dataset by split.

##### Datasets

Our end task performance is measured on the three existing datasets for
defeasible inference provided by Rudinger et al. (2020):333
github.com/rudinger/defeasible-nli $\delta$-atomic, $\delta$-snli,
$\delta$-social (Table 1). These datasets exhibit substantial diversity
because of their domains: $\delta$-snli (natural language inference),
$\delta$-social (reasoning about social norms), and $\delta$-atomic
(commonsense reasoning). Thus, it would require a general model to perform
well across these diverse datasets.

##### Baselines and setup

The previous state-of-the-art (SOTA) is the RoBERTa Liu et al. (2019) model
presented in Rudinger et al. (2020), and we report the published numbers for
this baseline. For an additional baseline, we directly use the initial
inference graph $\mathbf{G}^{\prime}$ generated by
$\textsc{gen}_{\text{init}}$, and provide it to the model simply as a string
(i.e., sequence of tokens; a simple, often-used approach). This baseline is
called e2e-STR. We use the same hyperparameters as Rudinger et al. (2020), and
add a detailed description of the hyperparameters in Appendix §C. For all the
qa experiments, we report the accuracy on the test set using the checkpoint
with the highest accuracy on the development set. We use the McNemar’s test
McNemar (1947); Dror et al. (2018) and use $p<0.05$ as a threshold for
statistical significance. All the p-values are reported in Appendix §G.

###  4.2 Results

Table 2 compares QA accuracy on these datasets without and with modeling the
question scenario. The results suggest that we get consistent gains across all
datasets, with $\delta$-snli gaining about 4 points. curious achieves a new
state-of-the-art across three datasets, as well as now producing
justifications for its answers with inference graphs.

|  $\delta$-atomic |  $\delta$-snli |  $\delta$-social
---|---|---|---
Prev-SOTA |  78.3 |  81.6 |  86.2
e2e-STR |  78.8 |  82.2 |  86.7
curious |  80.2* |  85.6* |  88.6*
Table 2: curious is better across all the datasets. This demonstrates that
understanding the question scenario through generating an inference graph
helps. * indicates statistical significance.

###  4.3 Understanding curious gains

In this section, we study the contribution of the main components of the
curious pipeline.

####  4.3.1 Impact of graph corrector

We ablate the graph corrector module $\textsc{gen}_{\text{corr}}$ in curious
by directly supplying the output from $\textsc{gen}_{\text{init}}$ to the
graph encoder. Table 3 shows that this ablation consistently hurts across all
the datasets. $\textsc{gen}_{\text{corr}}$ provides 2 points gain across
datasets. This indicates that better graphs lead to better task performance,
assuming that $\textsc{gen}_{\text{corr}}$ actually reduces the noise. Next,
we investigate if $\textsc{gen}_{\text{corr}}$ can produce more informative
graphs.

|  $\delta$-atomic |  $\delta$-snli |  $\delta$-social
---|---|---|---
$\mathbf{G}^{\prime}$ | 78.5 | 83.8 | 88.2
$G$ |  80.2* |  85.6* | 88.6
Table 3: Performance w.r.t. the graph used. $\mathbf{G}^{\prime}$ is the
initial graph from $\textsc{gen}_{\text{init}}$, $G$ is the corrected graph
from $\textsc{gen}_{\text{corr}}$. Better graphs lead to better task
performance. * indicates statistical significance.

##### Do graphs corrected by $\textsc{gen}_{\text{corr}}$ show fewer
repetitions?

We evaluate the repetitions in the graphs produced by
$\textsc{gen}_{\text{init}}$ and $\textsc{gen}_{\text{corr}}$ using two
metrics: (i) repetitions per graph: average number of repeated nodes in a
graph. (ii) % with repetitions: % of graphs with at least one repeated node.

| Repetitions | $\textsc{gen}_{\text{init}}$ | $\textsc{gen}_{\text{corr}}$
---|---|---|---
$\delta$-atomic | per graph | 2.05 | 1.26
| % graphs | 72 | 48
$\delta$-snli | per graph | 2.09 | 1.18
| % graphs | 73 | 46
$\delta$-social | per graph | 2.2 | 1.32
| % graphs | 75 | 49
overall | per graph |  |  $\Delta$ -40%
| % graphs |  |  $\Delta$ -25.7%
Table 4: $\textsc{gen}_{\text{corr}}$ reduces the inconsistencies in graphs.
The number of repetitions per graph and percentage of graphs with some
repetition, both improve.

Table 4 shows $\textsc{gen}_{\text{corr}}$ does reduce repetitions by
approximately 40% (2.11 to 1.25) per graph across all datasets, and also
reduces the fraction of graphs with at least one repetition by 25.7% across.

####  4.3.2 Impact of graph encoder

We experiment with two alternative approaches to graph encoding to compare our
MoE approach by using the graphs generated by $\textsc{gen}_{\text{corr}}$:

1\. Graph convolutional networks: We follow the approach of Lv et al. (2020)
who use gcn Kipf and Welling (2017) to learn rich node representations from
graphs. Broadly, node representations are initialized by $\mathcal{L}$ and
then refined using a gcn. Finally, multi-headed attention Vaswani et al.
(2017) between question representation $\mathbf{h}_{\mathbf{x}}$ and the node
representations is used to yield $\mathbf{h}_{\mathbf{G}}$. We add a detailed
description of this method in Appendix §H.

2\. String based representation: Another popular approach Sakaguchi et al.
(2021) is to concatenate the string representation of the nodes, and then
using $\mathcal{L}$ to obtain the graph representation
$\mathbf{h}_{\mathbf{G}}$ $=\mathcal{L}(v_{1}\|v_{2}\|..)$ where $\|$ denotes
string concatenation.

Table 5 shows that MoE graph encoder improves end task performance
significantly compared to the baseline.444Appendix §E provides an analysis on
time and memory requirements. In the following analysis, we study the reasons
for these gains in-depth.

We hypothesize that gcn is less resistant to noise than MoE in our setting,
thus causing a lower performance. The graphs augmented with each query are not
human-curated and are instead generated by a language model in a zero-shot
inference setting. Thus, the gcn style message passing might amplify the noise
in graph representations. On the other hand, moe-v first selects the most
useful nodes to answer the query to form the graph representation
$\mathbf{h}_{\mathbf{G}}$. Further, moe-gx can also decide to completely
discard the graph representations, as it does in many cases where the true
answer for the defeasible query is weakens.

To further establish the possibility of message passing hampering the
downstream task performance, we experiment with a gcn-MoE hybrid, wherein we
first refine the node representations using a 2-layer gcn as used by Lv et al.
(2020), and then pool the node representations using an MoE. We found the
results to be about the same as ones we obtained with gcn (3rd-row Table 5),
indicating that bad node representations are indeed the root cause for the bad
performance of gcn. This is also supported by Shi et al. (2019) who found that
noise propagation directly deteriorates network embedding and gcn is sensitive
to noise.

Interestingly, graphs help the end-task even when encoded using a relatively
simple str based encoding scheme, further establishing their utility.

|  $\delta$-atomic |  $\delta$-snli |  $\delta$-social
---|---|---|---
str | 79.5 | 83.1 | 87.2
gcn | 78.9 | 82.4 | 88.1
gcn \+ MoE | 78.7 | 84.3 | 87.8
MoE | 80.2 | 85.6 | 88.6
Table 5: Contribution of MoE-based graph encoding compared with alternative
graph encoding methods. The gains of MoE over gcn are statistically
significant for all the datasets, and the gains over str are significant for
$\delta$-snli and $\delta$-social.

####  4.3.3 Detailed MoE analysis

We now analyze the two MoEs used in curious: (i) the MoE over the nodes
(moe-v), and (ii) the MoE over $\mathbf{G}$ and input $x$ (moe-gx).

Figure 5: moe-gx gate values for the classes strengthens and weakens, averaged
over the three datasets.

#####  moe-gx performs better for $y=\text{strengthens:}$

Figure 5 shows that the graph makes a stronger contribution than the input,
when the label is strengthens. In instances where the label is weakens, the
gate of moe-gx gives a higher weight to the question. This trend was present
across all the datasets. We conjecture that this happens because language
models are tuned to generate events that happen rather than events that do
not. In the case of a weakener, the nodes must be of the type event1 leads to
less of event2, whereas language models are naturally trained for event1 leads
to event2. Understanding this in-depth requires further investigation in the
future.

#####  moe-v relies more on specific nodes:

We study the distribution over the types of nodes and their contribution to
moe-v. Recall from Figure 3 that C- and C+ nodes are contextualizers that
provide more background context to the question, and S- node is typically an
inverse situation (i.e., inverse $\mathbf{S}$), while M- and M+ are the
mediator nodes leading to the hypothesis. Figure 6 shows that the situation
node S- was the most important, followed by the contextualizer and the
mediator. Notably, our analysis shows that mediators are less important for
machines than they were for humans in the experiments conducted by Madaan et
al. (2021). This is probably because humans and machines use different pieces
of information. As our error analysis shows in §5, the mediators can be
redundant given the query $\mathbf{x}$. Humans might have used the redundancy
to reinforce their beliefs, whereas machines leverage the unique signals
present in S- and the contextualizers.

Figure 6: moe-v gate values for the three datasets.

#####  moe-v, moe-gx have a peaky distribution:

A peaky distribution over the gate values implies that the network is
judiciously selecting the right expert for a given input. We compute the
average entropy of moe-v and moe-gx and found the entropy values to be 0.52
(max 1.61) for moe-v, and 0.08 (max 0.69) for moe-gx. The distribution of the
gate values of moe-v is relatively flat, indicating that specialization of the
node experts might have some room for improvement (additional discussion in
Appendix §B). Analogous to scene graphs-based explanations in visual QA Ghosh
et al. (2019), peaky distributions over nodes can be considered as an
explanation through supporting nodes.

#####  moe-v learns the node semantics:

The network learned the semantic grouping of the nodes (contextualizers,
situation, mediators), which became evident when plotting the correlation
between the gate weights. As Figure 7 shows, there is a strong negative
correlation between the situation nodes and the context nodes, indicating that
only one of them is activated at a time.

Figure 7: Correlation between probability assigned to each semantic type of
the node by moe-v

##  5 Error analysis

|  |
---|---|---
| now fail | now succ
prev. fail |  $\delta$-atomic 615 $\delta$-snli 197 $\delta$-social 772 |  $\delta$-atomic 294 $\delta$-snli 124 $\delta$-social 398
prev. succ |  $\delta$-atomic 207 $\delta$-snli 68 $\delta$-social 302 |  $\delta$-atomic 3022 $\delta$-snli 1448 $\delta$-social 7967
Table 6: Confusion matrix: can curious fix previously failing or successful
examples?

Table 6 shows that curious is able to correct several previously wrong
examples. When curious corrected previously failing cases, the moe-v relied
more on mediators, as the average mediator probabilities go up from 0.09 to
0.13 averaged over the datasets. curious still fails, and more concerning are
the cases when previously successful examples now fail. To study this, we
annotate 50 random dev samples over the three datasets (26/24 examples for
weakens/strengthens label). For each sample, a human-annotated if the graph
had errors. We observe the following error categories:555Concrete examples in
Appendix §F

  * $\bullet$

All nodes off-topic (4%): The graph nodes were not on topic. This (rarely)
happens when curious cannot distinguish the sense of a word in the input
question. For instance, $\mathbf{S}$ = there is a water fountain in the center
– curious generated based on an incorrect word sense of natural water spring.

  * $\bullet$

Repeated nodes (20%): These may be exact or near-exact matches. Node pairs
with similar effects tend to be repeated in some samples. E.g., the S- node is
often repeated with contextualizer C- perhaps because these nodes indirectly
affect graph nodes in a similar way.

  * $\bullet$

Mediators are uninformative (34%): The mediating nodes are not correct or
informative. One source of these errors is when the $\mathbf{H}$ and
$\mathbf{S}$ are nearly connected by a single hop, e.g., $\mathbf{H}$ =
personX pees, and $\mathbf{S}$ = personX drank a lot of water previously.

  * $\bullet$

Good graphs are ineffective (42%): These graphs contained the information
required to answer the question, but the gating MOE mostly ignored this graph.
This could be attributed in part to the observation in the histogram in Figure
5, that samples with weakens label disproportionately ignore the graph.

In accordance with the findings of Rudinger et al. (2020), the maximum
percentage of errors was in $\delta$-atomic, in part due to low question
quality.

##  6 Explainability

In this section, we analyze the explainability of curious model. Jacovi and
Goldberg (2020) note that an explanation should aim towards two complementary
goals: i) Plausibility: provide an interpretation of system outputs that is
convincing for humans, and ii) Faithfulness: capture the actual reasoning
process of a model. We discuss how our approach takes a step towards
addressing these goals.

##### Plausibility

In a prior work, Madaan et al. (2021) show that human annotators selectively
picked and chose parts of the graph that explained a model decision and
enabled them in improving on the task of defeasible reasoning. We show in
§4.3.3, the MoE gate values gives insights into the part of the graph
(contextualizer, mediator, situation node) that the model leveraged to answer
a query. Our model thus produces a reasoning chain that is similar to the
explanation that humans understand, providing a step towards building
inherently plausible models, while also achieving better performance.

##### Measuring faithfulness w.r.t. graphs

Since faithfulness is a widely debated term, we restrict its definition to
measure faithfulness w.r.t to the reasoning graph. This can be measured by the
correlation between the model performance and graph correctness. A high
correlation implies that the model uses both the graph and query to generate
an answer and thus is faithful to the stated reasoning mechanism (i.e., graphs
used to answer a question). Our analysis reveals this to be the case: in cases
where the model answers incorrectly, 42% of the graphs were entirely correct
(§5). In contrast, when the model answers correctly, 82% of the graphs are
correct. In summary, we hope that curious serves as a step towards building
reasoning models that are both plausible and faithful.

##  7 Related work

##### Mental Models

Cognitive science has long promoted mental models - coherent, constructed
representations of the world - as central to understanding, communication, and
problem-solving Johnson-Laird (1983); Gentner and Stevens (1983); Hilton
(1996). Our work draws on these ideas, using inference graphs to represent the
machine’s “mental model” of the problem at hand. Building the inference graph
can be viewed as first asking clarification questions about the context before
answering. This is similar to self-talk Shwartz et al. (2020) but directed
towards eliciting chains of influence.

##### Injecting Commonsense Knowledge

Many prior systems use commonsense knowledge to aid question-answering, e.g.,
using sentences retrieved from a corpus Yang et al. (2019); Guu et al. (2020),
or with knowledge generated from a separate source Shwartz et al. (2020);
Bosselut et al. (2019); and injected either as extra sentences fed directly to
the model Clark et al. (2020), via the loss function Tandon et al. (2018), or
via attention Ma et al. (2019). Unlike prior work, we use conditional language
generation techniques to create graphs that are relevant to answering a
question.

##### Encoding Graph Representations

Several existing methods use graphs as an additional input for commonsense
reasoning Sun et al. (2018); Lin et al. (2019); Lv et al. (2020); Feng et al.
(2020); Bosselut et al. (2021); Ma et al. (2021); Kapanipathi et al. (2020).
These methods first retrieve a graph relevant to a question using information
retrieval techniques and then encode the graph using graph representation
techniques like gcn Kipf and Welling (2017) and graph attention Velickovic et
al. (2018). Different from these works, we use a graph generated from the
query for answering the commonsense question. The graphs consumed by these
works contain entities grounded in knowledge graphs like ConceptNet Speer et
al. (2017), whereas we perform reasoning over event inference graphs where
each node describes an event. Our best model uses a mixture-of-experts (MoE)
Jacobs et al. (1991) model to pool multi-faceted input. Prior work has shown
the effectiveness of using MoE for graph classification Zhou and Luo (2019);
Hu et al. (2021), cross-lingual language learning Chen et al. (2019); Gu et
al. (2018), and model ensemble learning Fedus et al. (2021); Shazeer et al.
(2017). To the best of our knowledge, we are the first to use MoE for learning
and pooling graph representations for qa task.

##  8 Summary and Conclusion

Cognitive science suggests that people form “mental models” of a situation to
answer questions about it. Drawing on those ideas, we have presented a simple
instantiation in which the situational model is an inference graph. Different
from gcn-based models popular in graph learning, we use mixture-of-experts to
pool graph representations. Our experiments show that MoE-based pooling can be
a strong (both in terms of performance and explainability) alternative to gcn
for graph-based learning for reasoning tasks. Our method establishes a new
state-of-the-art on three defeasible reasoning datasets. Overall, our method
shows that performance can be improved by guiding a system to “think about” a
question and explicitly model the scenario, rather than answering reflexively.

## Acknowledgments

We thank the anonymous reviewers for their feedback. Special thanks to
reviewer 2 for their insightful comments on our MoE formulation. This material
is partly based on research sponsored in part by the Air Force Research
Laboratory under agreement number FA8750-19-2-0200. The U.S. Government is
authorized to reproduce and distribute reprints for Governmental purposes
notwithstanding any copyright notation thereon. The views and conclusions
contained herein are those of the authors and should not be interpreted as
necessarily representing the official policies or endorsements, either
expressed or implied, of the Air Force Research Laboratory or the U.S.
Government.

## References

  * Bosselut et al. (2021) Antoine Bosselut, Ronan Le Bras, and Yejin Choi. 2021.  Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering.  In _Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI)_.
  * Bosselut et al. (2019) Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. 2019.  COMET: Commonsense transformers for automatic knowledge graph construction.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4762–4779, Florence, Italy. Association for Computational Linguistics.
  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015\.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Chen et al. (2019) Xilun Chen, Ahmed Hassan Awadallah, Hany Hassan, Wei Wang, and Claire Cardie. 2019\.  Multi-source cross-lingual model transfer: Learning what to share.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 3098–3112, Florence, Italy. Association for Computational Linguistics.
  * Clark et al. (2020) P. Clark, Oren Etzioni, Daniel Khashabi, Tushar Khot, B. D. Mishra, Kyle Richardson, Ashish Sabharwal, Carissa Schoenick, Oyvind Tafjord, Niket Tandon, Sumithra Bhakthavatsalam, Dirk Groeneveld, Michal Guerquin, and Michael Schmitz. 2020.  From ’f’ to ’a’ on the n.y. regents science exams: An overview of the aristo project.  _AI Mag._ , 41:39–53.
  * Dror et al. (2018) Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. 2018.  The hitchhiker’s guide to testing statistical significance in natural language processing.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1383–1392, Melbourne, Australia. Association for Computational Linguistics.
  * Falcon (2019) et al. Falcon, WA. 2019.  Pytorch lightning.  _GitHub. Note: https://github.com/PyTorchLightning/pytorch-lightning_ , 3.
  * Fedus et al. (2021) William Fedus, Barret Zoph, and Noam Shazeer. 2021.  Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.  _arXiv preprint arXiv:2101.03961_.
  * Feng et al. (2020) Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, Peifeng Wang, Jun Yan, and Xiang Ren. 2020.  Scalable multi-hop relational reasoning for knowledge-aware question answering.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1295–1309, Online. Association for Computational Linguistics.
  * Forbes et al. (2020) Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. 2020\.  Social chemistry 101: Learning to reason about social and moral norms.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 653–670, Online. Association for Computational Linguistics.
  * Gentner and Stevens (1983) Dedre Gentner and Albert L. Stevens. 1983.  _Mental Models_.  Lawrence Erlbaum Associates.
  * Ghosh et al. (2019) S. Ghosh, Giedrius Burachas, Arijit Ray, and Avi Ziskind. 2019.  Generating natural language explanations for visual question answering using scene graphs and visual attention.  _ArXiv_ , abs/1902.05715.
  * Gu et al. (2018) Jiatao Gu, Hany Hassan, Jacob Devlin, and Victor O.K. Li. 2018.  Universal neural machine translation for extremely low resource languages.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 344–354, New Orleans, Louisiana. Association for Computational Linguistics.
  * Guu et al. (2020) Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020.  Retrieval augmented language model pre-training.  In _International Conference on Machine Learning_ , pages 3929–3938. PMLR.
  * Hilton (1996) D. Hilton. 1996.  Mental models and causal explanation: Judgements of probable cause and explanatory relevance.  _Thinking & Reasoning_, 2:273–308.
  * Hu et al. (2021) Fenyu Hu, Liping Wang, Shu Wu, Liang Wang, and Tieniu Tan. 2021.  Graph classification by mixture of diverse experts.  _arXiv preprint arXiv:2103.15622_.
  * Jacobs et al. (1991) Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991\.  Adaptive mixtures of local experts.  _Neural computation_ , 3(1):79–87.
  * Jacovi and Goldberg (2020) Alon Jacovi and Yoav Goldberg. 2020.  Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4198–4205, Online. Association for Computational Linguistics.
  * Johnson-Laird (1983) P. Johnson-Laird. 1983.  _Mental Models : Towards a Cognitive Science of Language_.  Harvard University Press.
  * Jordan and Jacobs (1994) Michael I Jordan and Robert A Jacobs. 1994.  Hierarchical mixtures of experts and the em algorithm.  _Neural computation_ , 6(2):181–214.
  * Jordan and Xu (1995) Michael I Jordan and Lei Xu. 1995.  Convergence results for the em approach to mixtures of experts architectures.  _Neural networks_ , 8(9):1409–1431.
  * Kapanipathi et al. (2020) Pavan Kapanipathi, Veronika Thost, Siva Sankalp Patel, Spencer Whitehead, Ibrahim Abdelaziz, Avinash Balakrishnan, Maria Chang, Kshitij P. Fadnis, R. Chulaka Gunasekara, Bassem Makni, Nicholas Mattei, Kartik Talamadupula, and Achille Fokoue. 2020.  Infusing knowledge into the textual entailment task using graph convolutional networks.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 8074–8081. AAAI Press.
  * Kipf and Welling (2017) Thomas N. Kipf and Max Welling. 2017.  Semi-supervised classification with graph convolutional networks.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Koons (2017) Robert Koons. 2017.  Defeasible Reasoning.  In Edward N. Zalta, editor, _The Stanford Encyclopedia of Philosophy_ , Winter 2017 edition. Metaphysics Research Lab, Stanford University.
  * Lin et al. (2019) Bill Yuchen Lin, Xinyue Chen, Jamin Chen, and Xiang Ren. 2019.  KagNet: Knowledge-aware graph networks for commonsense reasoning.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2829–2839, Hong Kong, China. Association for Computational Linguistics.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Lv et al. (2020) Shangwen Lv, Daya Guo, Jingjing Xu, Duyu Tang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, and Songlin Hu. 2020.  Graph-based reasoning over heterogeneous external knowledge for commonsense question answering.  In _The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020_ , pages 8449–8456. AAAI Press.
  * Ma et al. (2019) Kaixin Ma, Jonathan Francis, Quanyang Lu, Eric Nyberg, and Alessandro Oltramari. 2019.  Towards generalizable neuro-symbolic systems for commonsense question answering.  In _Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing_ , pages 22–32, Hong Kong, China. Association for Computational Linguistics.
  * Ma et al. (2021) Kaixin Ma, Filip Ilievski, Jonathan Francis, Yonatan Bisk, Eric Nyberg, and Alessandro Oltramari. 2021.  Knowledge-driven data construction for zero-shot evaluation in commonsense question answering.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 13507–13515.
  * Madaan et al. (2021) Aman Madaan, Dheeraj Rajagopal, Niket Tandon, Yiming Yang, and Eduard Hovy. 2021\.  Could you give me a hint ? generating inference graphs for defeasible reasoning.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 5138–5147, Online. Association for Computational Linguistics.
  * McNemar (1947) Quinn McNemar. 1947.  Note on the sampling error of the difference between correlated proportions or percentages.  _Psychometrika_ , 12(2):153–157.
  * Paszke et al. (2017) Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017.  Automatic differentiation in pytorch.  _NIPS 2017 Workshop Autodiff Submission_.
  * Pollock (1987) J. Pollock. 1987.  Defeasible reasoning.  _Cogn. Sci._ , 11:481–518.
  * Pollock (2009) J. Pollock. 2009.  A recursive semantics for defeasible reasoning.  In _Argumentation in Artificial Intelligence_.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _Journal of Machine Learning Research_ , 21:1–67.
  * Rudinger et al. (2020) Rachel Rudinger, Vered Shwartz, Jena D. Hwang, Chandra Bhagavatula, Maxwell Forbes, Ronan Le Bras, Noah A. Smith, and Yejin Choi. 2020.  Thinking like a skeptic: Defeasible inference in natural language.  In _Findings of the Association for Computational Linguistics: EMNLP 2020_ , pages 4661–4675, Online. Association for Computational Linguistics.
  * Sakaguchi et al. (2021) Keisuke Sakaguchi, Chandra Bhagavatula, Ronan Le Bras, Niket Tandon, Peter Clark, and Yejin Choi. 2021.  proscript: Partially ordered scripts generation via pre-trained language models.  _arxiv_.
  * Sap et al. (2019) Maarten Sap, Ronan Le Bras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. 2019.  ATOMIC: an atlas of machine commonsense for if-then reasoning.  In _The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, The Thirty-First Innovative Applications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, Honolulu, Hawaii, USA, January 27 - February 1, 2019_ , pages 3027–3035. AAAI Press.
  * Shazeer et al. (2017) Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017.  Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.  In _5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings_. OpenReview.net.
  * Shi et al. (2019) M. Shi, Yufei Tang, Xingquan Zhu, and J. Liu. 2019.  Feature-attention graph convolutional networks for noise resilient learning.  _ArXiv_ , abs/1912.11755.
  * Shwartz et al. (2020) Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020\.  Unsupervised commonsense question answering with self-talk.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 4615–4629, Online. Association for Computational Linguistics.
  * Speer et al. (2017) Robyn Speer, Joshua Chin, and Catherine Havasi. 2017.  Conceptnet 5.5: An open multilingual graph of general knowledge.  In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA_ , pages 4444–4451. AAAI Press.
  * Sun et al. (2018) Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhutdinov, and William Cohen. 2018.  Open domain question answering using early fusion of knowledge bases and text.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 4231–4242, Brussels, Belgium. Association for Computational Linguistics.
  * Tandon et al. (2018) Niket Tandon, Bhavana Dalvi, Joel Grus, Wen-tau Yih, Antoine Bosselut, and Peter Clark. 2018.  Reasoning about actions and state changes by injecting commonsense knowledge.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 57–66, Brussels, Belgium. Association for Computational Linguistics.
  * Tandon et al. (2019) Niket Tandon, Bhavana Dalvi, Keisuke Sakaguchi, Peter Clark, and Antoine Bosselut. 2019.  WIQA: A dataset for “what if…” reasoning over procedural text.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6076–6085, Hong Kong, China. Association for Computational Linguistics.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5998–6008.
  * Velickovic et al. (2018) Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018.  Graph attention networks.  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net.
  * Wolf et al. (2019) Thomas Wolf, L Debut, V Sanh, J Chaumond, C Delangue, A Moi, P Cistac, T Rault, R Louf, M Funtowicz, et al. 2019.  Huggingface’s transformers: State-of-the-art natural language processing.  _ArXiv, abs/1910.03771_.
  * Yang et al. (2019) Wei Yang, Yuqing Xie, Aileen Lin, Xingyu Li, Luchen Tan, Kun Xiong, Ming Li, and Jimmy Lin. 2019.  End-to-end open-domain question answering with BERTserini.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations)_ , pages 72–77, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Yang and Liu (1999) Yiming Yang and Xin Liu. 1999.  A re-examination of text categorization methods.  In _Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval_ , pages 42–49.
  * Zhou and Luo (2019) Xuanyu Zhou and Yuanhang Luo. 2019.  Explore mixture of experts in graph neural networks.  _Stanford CS224W_.

##  Appendix A Training graph corrector

As mentioned in Section §3.2, the graph generator $\textsc{gen}_{\text{init}}$
is trained as a seq2seq model from wiqa with $\texttt{input}=\text{[Premise]
}\mathbf{T}_{i}\mid\text{[Situation] }\mathbf{S}_{i}\mid\text{[Hypothesis]
}\mathbf{H}_{i}$, and $\texttt{output}=\mathbf{G}_{i}$. Graphs in wiqa
additionally capture the influence that the situation has on the hypothesis.
Denoting this influence label by $y_{i}$ can be either helps or hurts

From our experiments, we observe that appending $y_{i}$ to the training data
(from $\texttt{input}=\text{[Premise] }\mathbf{T}_{i}\mid\text{[Situation]
}\mathbf{S}_{i}\mid\text{[Hypothesis] }\mathbf{H}_{i}$ to
$\texttt{input}=\text{[Premise] }\mathbf{T}_{i}\mid\text{[Situation]
}\mathbf{S}_{i}\mid\text{[Hypothesis] }\mathbf{H}_{i}\mid y_{i}$) reduces
repetitions by 13%.

We refer to this data generator as $\textsc{gen}_{\text{init}}^{*}$, and the
graphs produced by it as $\mathbf{G}^{*}$. However, we do not have access to
$y$ during test time, and thus $\textsc{gen}_{\text{init}}^{*}$ cannot be used
directly to produce $\mathbf{G}^{*}$ for defeasible queries. We circumvent
this by using $\textsc{gen}_{\text{init}}^{*}$ to train a graph-to-graph
generation model, that takes as input $\mathbf{G}^{\prime}$ and generates
$\mathbf{G}^{*}$ as output ($\mathbf{G}^{\prime}\rightarrow\mathbf{G}^{*}$).
We call this system $\textsc{gen}_{\text{corr}}$. We give an overview of the
process in Figure 8. In Figure 9, we give examples of an intial graph produced
by $\textsc{gen}_{\text{init}}$, the corresponding graph produced by
$\textsc{gen}_{\text{init}}^{*}$, and the graph produced by
$\textsc{gen}_{\text{corr}}$.

Figure 8: Training data generation to train $\textsc{gen}_{\text{corr}}$.

Figure 9: The graphs generated by $\textsc{gen}_{\text{init}}$ (left),
$\textsc{gen}_{\text{init}}^{*}$ (middle), and $\textsc{gen}_{\text{corr}}$
(right).The input graph has repetitions for nodes $\\{C{-},S{-}\\}$,
$\\{C{+},H{+}\\}$, and $\\{M{-},M{+}\\}$. The corrected graph replaces the
repetitions with meaningful labels.

##  Appendix B MoE gradient analysis

Figure 10: MoE gradient analysis setup: we consider a simple setting where the
weighted output of the experts (using the expert weights $p$) is directly fed
to a softmax and is used for generating class probabilities $\hat{y}$.

We restate Equation 2 for quick reference:

$\displaystyle\mathbf{p}$ $\displaystyle=\mathbf{M}(\mathbf{x})$
$\displaystyle\mathbf{o}$ $\displaystyle=\sum_{i=1}^{n}p_{i}\mathbf{E_{i}(x)}$

where we have changed the notation slightly to use $\mathbf{o}$ as the MoE
output instead of $\mathbf{y}$. We also refer to $\mathbf{E_{i}}(x)$ as
$\mathbf{E_{i}}$. Further, $o_{j}=\sum_{i=1}^{n}p_{i}E_{ij}$. We present the
analysis for a generic multi-class classification setting with $k$ classes,
with training done using a cross-entropy loss $\mathcal{L}$ (Figure 10) Let
$\hat{y}_{c}$ be the normalized probability of the correct class $c$
calculated using softmax:

$\displaystyle\hat{y}_{c}$
$\displaystyle=\frac{\exp(o_{c})}{\sum_{j=1}^{k}\exp(o_{j})}$
$\displaystyle=\frac{\exp(\sum_{i=1}^{n}p_{i}E_{ic})}{\sum_{j=1}^{k}\exp(\sum_{i=1}^{n}p_{i}E_{ij})}$

Let $\mathcal{L}$ be the cross-entropy loss:

$\displaystyle\mathcal{L}$
$\displaystyle=-\log\hat{y}_{c}=-o_{c}+\log{\sum_{j=1}^{k}\exp(o_{j})}$
$\displaystyle=-\sum_{i=1}^{n}p_{i}E_{ic}+\log{\sum_{j=1}^{k}\exp(\sum_{i=1}^{n}p_{i}E_{ij})}$

##### Evaluating $\frac{\partial\mathcal{L}}{\partial p_{m}}$

The derivatives w.r.t. the $m^{th}$ expert gate probability $p_{m}$ is given
by:

$\displaystyle\frac{\partial\mathcal{L}}{\partial p_{m}}$
$\displaystyle=-E_{mc}+\frac{\sum_{j=1}^{k}E_{mj}\exp(\sum_{i=1}^{n}p_{i}E_{ij})}{\sum_{j=1}^{k}\exp(\sum_{i=1}^{n}p_{i}E_{ij})}$
$\displaystyle=-E_{mc}+\sum_{j=1}^{k}\hat{y}_{j}E_{mj}$
$\displaystyle=-E_{mc}(1-\hat{y}_{c})+\sum_{j=1,j\not=c}^{k}\hat{y}_{j}E_{mj}$
(5)

##### Evaluating $\frac{\partial\mathcal{L}}{\partial E_{mc}}$

the derivatives w.r.t. the logits $E_{mc}$ (logit for the correct class by
$m^{th}$ expert) is given by:

$\displaystyle\frac{\partial\mathcal{L}}{\partial E_{mc}}$
$\displaystyle=-p_{m}+\frac{\exp(o_{c})p_{m}}{\sum_{j=1}^{k}\exp o_{j}}$
$\displaystyle=-p_{m}(1-\hat{y}_{c})$ (6)

Equations 5 and 6 have natural interpretations: the gradient on both the
mixture probability $p_{m}$ and the logits $E_{mc}$ will be 0 (note that for
Equation 5, $\mathbf{y}^{c}=1\implies\mathbf{y}^{j}=0$ for $j\not=c$) when the
network makes perfect predictions ($\hat{y}_{c}=1$). As noted by Jacobs et al.
(1991) (Section 1), this might cause the network to specialize slower, as the
gradient will be small for experts that are helping in making the correct
prediction. They suggest a different loss function that promotes faster
specialization by redefining the error function in terms of a mixture
distribution, with the mixture weights supplied by the $p_{i}$ terms.
Analyzing the effect of loss function for applications where the MoE is used
to pool representations remains an interesting future work.

##  Appendix C Hyperparameters

##### Training details

All of our experiments were done on a single Nvidia GeForceRTX 2080 Ti. We
base our implementation on PyTorch Paszke et al. (2017) and also use PyTorch
Lightning Falcon (2019) and Huggingface Wolf et al. (2019). The gates and the
experts in our MoE model were a single layer MLP. For the experts, we set the
input size set to be the same as output size. Table 7 shows the parameters
shared by all the methods, and 8 shows the hyperparameters applicable to gcn
encoder.

Hyperparameter | Value
---|---
Pre-trained model | RoBERTa-base
Learning rate | 2e-5
Gradient accumulation batches | 2
Num epochs | 30
Optimizer | AdamW
Dropout | 0.1
Learning rate scheduling | linear
Warmup | 3 epochs
Batch size | 16
Weight decay | 0.01
Gradient clipping | 1.0
Table 7: General hyperparameters used by all the models. Hyperparameter | Value
---|---
# Layers | 2
Layer dropout | 0.1
Number of attention heads | 1
Attention dimension | 256
Table 8: Hyperparameters specific to gcn.

##  Appendix D Schema of an influence graph

Figure 11 shows the skeleton of an influence graph.

Figure 11: Schema of an inference graph.

##  Appendix E Runtime Analysis

Finally, we discuss the cost-performance tradeoffs for various encoding
mechanisms (Table 9). As Table 9 shows, both gcn and MoE take about 7% more
number of parameters than the str encoding scheme and have about 2x the
runtime. Further, as we use one expert per node, the number of parameters
scales linearly with the number of nodes. While this is not prohibitive in our
setting (each graph has a small number of nodes), our analysis shows that the
behavior of the nodes that have similar semantics is correlated, indicating
that the experts for those nodes can share parameters. Alternatively, MoE with
more than two layers Jordan and Xu (1995) can also help in scaling the number
of parameters only logarithmically with the number of nodes.

Method | str | gcn | MoE
---|---|---|---
#Params | 124M | 131M | 133M
Runtime | 0.17 | 0.47 | 0.40
Table 9: Number of parameters in the different encoding methods. Runtime
reports the number of seconds to process one training example.

##  Appendix F Error Analysis Examples

We show three examples with different types of errors. These examples are
taken from Dev set, and these are for the cases where curious introduced a
wrong answer, while baseline answered this correctly without the graph.

  * $\bullet$

Figure 12 shows a failure case when a good graph is unused. Example from
$\delta$-atomic dev set.

  * $\bullet$

Figure 13 shows a failure case when an off topic graph is produced due to
confusion in the sense of water fountain. Example from $\delta$-snli dev set.

  * $\bullet$

Figure 14 shows a failure case when the mediator is wrong. Example from
$\delta$-social dev set.

Figure 12: Example of a failure case: A good graph is unused. Example from
$\delta$-atomic dev set. Figure 13: Example of a failure case: The generated
graph is off topic (wrong sense of water fountain is used). Example from
$\delta$-snli dev set. Figure 14: Example of a failure case: The mediator
nodes (second last level in the graph) are unhelpful. Example from
$\delta$-social dev set.

##  Appendix G Significance Tests

We perform two statistical tests for verifying our results: i) The micro-sign
test (s-test) Yang and Liu (1999), and ii) McNermar’s test Dror et al. (2018).

Dataset | s-test | McNemar’s test
---|---|---
$\delta$-atomic | 5.07e-05 | 1.1e-04
$\delta$-snli | 2.65e-05 | 6.5e-05
$\delta$-social | 1.4e-04 | 3.2e-04
Table 10: p-values for the three datasets and two different statistical tests while comparing the results with and without graphs (Table 2). As the p-values show, the results in Table 2 are highly significant Dataset | s-test | McNemar’s test
---|---|---
$\delta$-atomic | 0.001 | 0.003676
$\delta$-snli | 0.01 | 0.026556
$\delta$-social | 0.06 | 0.146536
Table 11: p-values for the three datasets and two different statistical tests while comparing the results with noisy vs. cleaned graphs (Table 3). |  $\delta$-atomic |  $\delta$-snli |  $\delta$-social
---|---|---|---
str | 0.13 | 1.8e-06 | 8.7e-06
gcn | 0.006 | 1.31e-05 | 0.03
Table 12: p-values for the s-test for Table 5. |  $\delta$-atomic |  $\delta$-snli |  $\delta$-social
---|---|---|---
str | 0.28 | 4e-06 | 2e-05
gcn | 0.015127 | 3.2e-05 | 0.06
Table 13: p-values for the McNemar’s for Table 5.

##  Appendix H Description of gcn encoder

We now describe our adaptation of the method by Lv et al. (2020) to pool
$\mathbf{h}_{\mathbf{V}}$ into $\mathbf{h}_{\mathbf{G}}$ using gcn. Figure 15
captures the overall design.

##### Refining node representations

The representation for each node $v\in\mathbf{V}$ is first initialized using:

$\mathbf{h}_{\mathbf{v}}^{0}=\mathbf{W}^{0}\mathbf{h}_{\mathbf{v}}$

Where $\mathbf{h}_{\mathbf{v}}\in\mathbb{R}^{d}$ is the node representation
returned by the $\mathcal{L}$, and $\mathbf{W}^{0}\in\mathbb{R}^{d\times k}$.
This initial representation is then refined by running L-layers of a GCN Kipf
and Welling (2017), where each layer $l+1$ is updated by using representations
from the $l^{th}$ layer as follows:

$\displaystyle\mathbf{h}_{v}^{(l+1)}$
$\displaystyle=\sigma\left(\frac{1}{|\mathbf{A}(v)|}{\sum_{w\in\mathbf{A}(v)}\mathbf{W}^{l}\mathbf{h}_{w}^{l}}+\mathbf{W}^{l}\mathbf{h}_{v}^{l}\right)$
$\displaystyle\mathbf{H}^{L}$
$\displaystyle=[\mathbf{h}_{0}^{L};\mathbf{h}_{1}^{L};\ldots;\mathbf{h}_{|\mathbf{V}|-1}^{L}]$
(7)

Where $\sigma$ is a non-linear activation function,
$\mathbf{W}^{l}\in\mathbb{R}^{k\times k}$ is the GCN weight matrix for the
$l^{th}$ layer, $\mathbf{A}(v)$ is the list of neighbors of a vertex $v$, and
$\mathbf{H}^{L}\in\mathbb{R}^{|\mathbf{V}|\times k}$ is a matrix of the
$L^{th}$ layer representations the $|\mathbf{V}|$ nodes such that
$\mathbf{H}^{L}_{i}=\mathbf{h}_{i}^{L}$.

##### Learning graph representation

We use multi-headed attention Vaswani et al. (2017) to combine the query
representation $\mathbf{h}_{\mathbf{Q}}$ and the nodes representations
$\mathbf{H}^{L}$ to learn a graph representation $\mathbf{h}_{\mathbf{G}}$.
The multiheaded attention operation is defined as follows:

$\displaystyle\mathbf{a}_{i}$
$\displaystyle=\text{softmax}\left(\frac{(\mathbf{W^{q}_{i}}\mathbf{h}_{\mathbf{Q}})(\mathbf{W^{k}_{i}}\mathbf{H}^{L})^{T}}{\sqrt{d}}\right)$
$\displaystyle\text{head}_{i}$
$\displaystyle=\mathbf{a}_{i}(\mathbf{W^{v}_{i}}\mathbf{H}^{L})$
$\displaystyle\mathbf{h}_{\mathbf{G}}$
$\displaystyle=Concat(head_{1},\ldots,head_{h})\mathbf{W}^{O}$
$\displaystyle=\text{MultiHead}(\mathbf{h}_{Q},\mathbf{H}^{L})$ (9)

Where $h$ is the number of attention heads,
$\mathbf{W}^{q}_{i},\mathbf{W}^{k}_{i},\mathbf{W}^{v}_{i}\in\mathbb{R}^{k\times
d}$ and $\mathbf{W}^{O}\in\mathbb{R}^{hd\times d}$.

Finally, the graph representation generated by the the MultiHead attention
$\mathbf{h}_{\mathbf{G}}\in\mathbb{R}^{n}$ is concatenated with with the
question representation $\mathbf{h}_{\mathbf{Q}}$ to get the prediction:

$\hat{y}=\text{softmax}([\mathbf{h}_{\mathbf{G}},\mathbf{h}_{\mathbf{Q}}]\mathbf{W}_{out})$

where $\mathbf{W}_{out}\in\mathbb{R}^{d\times 2}$ is a single linear layer
MLP.

##  Appendix I All results

Our experiments span two types of graphs ($\mathbf{G}^{\prime}$, $G$), three
datasets ($\delta$-snli, $\delta$-social, $\delta$-atomic), and three graph
encoding schemes (str, gcn, MoE). Table 14 above shows the results on all 18
combinations of $\\{\text{graph
types}\\}\times\\{\text{datasets}\\}\times\\{\text{graph encoding schemes}\\}$

Dataset |  Encoder |  Graph Type |  Accuracy
---|---|---|---
$\delta$-atomic |  |  n/a |
str |  $\mathbf{G}^{\prime}$ |  78.78
str |  $G$ |  79.48
gcn |  $\mathbf{G}^{\prime}$ |  78.25
gcn |  $G$ |  78.85
MoE |  $\mathbf{G}^{\prime}$ |  78.83
MoE |  $G$ |  80.15
$\delta$-snli |  |  n/a |
str |  $\mathbf{G}^{\prime}$ |  82.16
str |  $G$ |  83.11
gcn |  $\mathbf{G}^{\prime}$ |  82.63
gcn |  $G$ |  83.09
MoE |  $\mathbf{G}^{\prime}$ |  83.83
MoE |  $G$ |  85.59
$\delta$-social |  |  n/a |  87.6
str |  $\mathbf{G}^{\prime}$ |  86.75
str |  $G$ |  87.24
gcn |  $\mathbf{G}^{\prime}$ |  87.92
gcn |  $G$ |  88.12
MoE |  $\mathbf{G}^{\prime}$ |  88.45
MoE |  $G$ |  88.62
Table 14: Results for different combinations of graph encoder, graph type.

##  Appendix J Graph-augmented defeasible reasoning algorithm

In Algorithm 1, we outline our graph-augmented defeasible learning process.

Given: A language model $\mathcal{L}$, defeasible query with graph
$(\mathbf{x},\mathbf{G})$.

Result: Result for the query.

// Encode query

$\mathbf{h}_{\mathbf{Q}}\leftarrow\mathcal{L}(\mathbf{x})$;

// encode nodes of $\mathbf{G}$

$\mathbf{h}_{\mathbf{V}}\leftarrow\mathcal{L}(\mathbf{v}\in\mathbf{G})$ ;

// MOE1: Combine nodes

$\mathbf{h}_{\mathbf{G}}\leftarrow$ Equation 3;

// MOE2: Combine $Q$, $G$

$\mathbf{h}_{\mathbf{y}}\leftarrow$ Equation 4;

return _softmax(MLP($\mathbf{h}_{\mathbf{y}}$))_

Algorithm 1 Graph-augmented defeasible reasoning using MoE. Figure 15:
Overview of the gcn encoder.