# Counterfactual Multihop QA: A Cause-Effect Approach for Reducing
Disconnected Reasoning

Wangzhen Guo Qinkang Gong Hanjiang Lai
Sun Yat-Sen University
<EMAIL_ADDRESS><EMAIL_ADDRESS>*Corresponding
Author

###### Abstract

Multi-hop QA requires reasoning over multiple supporting facts to answer the
question. However, the existing QA models always rely on shortcuts, e.g.,
providing the true answer by only one fact, rather than multi-hop reasoning,
which is referred as disconnected reasoning problem. To alleviate this issue,
we propose a novel counterfactual multihop QA, a causal-effect approach that
enables to reduce the disconnected reasoning. It builds upon explicitly
modeling of causality: 1) the direct causal effects of disconnected reasoning
and 2) the causal effect of true multi-hop reasoning from the total causal
effect. With the causal graph, a counterfactual inference is proposed to
disentangle the disconnected reasoning from the total causal effect, which
provides us a new perspective and technology to learn a QA model that exploits
the true multi-hop reasoning instead of shortcuts. Extensive experiments have
conducted on the benchmark HotpotQA dataset, which demonstrate that the
proposed method can achieve notable improvement on reducing disconnected
reasoning. For example, our method achieves 5.8% higher points of its Supps
score on HotpotQA through true multihop reasoning. The code is available at
supplementary material.

##  1 Introduction

Multi-hop question answering (QA) Groeneveld et al. (2020); Ding et al.
(2019); Asai et al. (2019); Shao et al. (2020) requires the model to reason
over multiple supporting facts to correctly answer a complex question. It is a
challenging task, and many datasets, e.g., HotpotQA Yang et al. (2018) and
approaches Fang et al. (2019); Zhu et al. (2021b) have been proposed for this
reasoning task.

One of the main problems of multihop QA models is disconnected reasoning
Trivedi et al. (2020), which allows the models to exploit the reasoning
shortcuts (Jiang and Bansal, 2019; Lee et al., 2021) instead of multi-hop
reasoning to cheat and obtain the right answer. Taking Fig. 1 as an example,
to answer the question “until when in the U.S. Senate", we should consider two
supporting facts to infer the answer “Devorah Adler $\stackrel{{\scriptstyle
Director\ of\ Research\ for}}{{\Longrightarrow}}$ Barack Obama
$\stackrel{{\scriptstyle served\ from\ 2005\ to\ 2008}}{{\Longrightarrow}}$
2008". However, one may also infer the correct answer by just utilizing the
types of problems, e.g., we can find the corresponding fact “from 2005 to
2008" in the contexts without reasoning to answer this type of question “until
when".

One possible solution for reducing the disconnected reasoning is to strengthen
the training dataset via extra annotations or adversarial examples, which make
it cannot find the correct answers by only one supporting fact. For example,
Jiang and Bansal (2019) constructed the adversarial examples to generate
better distractor facts. (Trivedi et al., 2020) firstly defined a evaluate
measure, DiRe in short, to measure how much the QA model can cheat via
disconnected reasoning. Then, a transformed dataset is constructed to reduce
disconnected reasoning. Besides, counterfactual intervention (Lee et al.,
2021; Ye et al., 2021) had also been explored to change the distribution of
the training dataset. These methods improve the generalizability and
interpretability of the multi-hop reasoning QA model via balancing the train
data, which is noted as debiased training in QA model (Niu et al., 2021).
However, when the existing approaches decrease the disconnected reasoning, the
original performance also drops significantly. It is still challenging to
reduce disconnected reasoning while maintaining the same accuracy on the
original test set.

Motivated by causal inference (Pearl and Mackenzie, 2018; Pearl, 2022; Niu et
al., 2021), we utilize the counterfactual reasoning to reduce the disconnected
reasoning in multi-hop QA and also obtain the robust performance on the
original dataset. We formalize a causal graph to reflect the causal
relationships between question ($Q$), contexts and answer ($Y$). To evaluate
the disconnected reasoning, contexts are further divided into two subsets: $S$
is a supporting fact and $C$ are the remaining supporting facts. Hence, we can
formulate the disconnected reasoning as two natural direct causal effects of
$(Q,S)$ and $(Q,C)$ on $Y$ as shown in Fig. 1. With the proposed causal graph,
we can relieve the disconnected reasoning by disentangling the two natural
direct effects and the true multi-hop reasoning from the total causal effect.
A novel counterfactual multihop QA is proposed to disentangle them from the
total causal effect. We utilize the generated probing dataset proposed by
(Trivedi et al., 2020) and DiRe to measures how much the proposed multi-hop QA
model can reduce the disconnected reasoning. Experiment results show that our
approach can substantially decrease the disconnected reasoning while guarantee
the strong performance on the original test set. The results indicate that the
proposed approach can reduce the disconnected reasoning and improve the true
multi-hop reasoning capability.

The main contribution of this paper is threefold. Firstly, our counterfactual
multi-hop QA model formulates disconnected reasoning as two direct causal
effects on answer, which is a new perspective and technology to learn the true
multi-hop reasoning. Secondly, our approach achieves notable improvement on
reducing disconnected reasoning compared to various state-of-the-arts.
Thirdly, our causal-effect approach is model-agnostic and can be used for
reducing disconnected reasoning in many multi-hop QA architectures.

##  2 Related Work

Multi-hop question answering (QA) requires the model to retrieve the
supporting facts to predict the answer. Many approaches and datasets have been
proposed to train QA systems. For example, HotpotQA (Yang et al., 2018)
dataset is a widely used dataset for multi-hop QA, which consists of fullwiki
setting (Das et al., 2019; Nie et al., 2019; Qi et al., 2019; Chen et al.,
2019; Li et al., 2021; Xiong et al., 2020) and distractor setting (Min et al.,
2019b; Nishida et al., 2019; Qiu et al., 2019; Jiang and Bansal, 2019; Trivedi
et al., 2020).

In fullwiki setting, it firstly finds relevant facts from all Wikipedia
articles, and then finish the multi-hop QA task with the found facts. The
retrieval model is important in this setting. For instance, SMRS (Nie et al.,
2019) and DPR (Karpukhin et al., 2020) found the implicit importance of
retrieving relevant information in the semantic space. Entity-centric (Das et
al., 2019), CogQA (Ding et al., 2019) and Golden Retriever (Qi et al., 2019)
explicitly used the entity that is mentioned or reformed in query key words to
retrieve next hop document. Furthermore, PathRetriever (Asai et al., 2019) and
HopRetriever (Li et al., 2021) can iteratively select the documents to form a
paragraph-level reason path using RNN. MDPR (Xiong et al., 2020) retrieved
passages only using dense query vector in many times. These methods hardly
discuss the QA model’s disconnected reasoning problem.

In distractor setting, 10 paragraphs, two gold paragraphs and eight
distractors, are given. Many methods have been proposed to strengthen the
model’s capability of multi-hop reasoning, using graph neural network (Qiu et
al., 2019; Fang et al., 2019; Shao et al., 2020) or adversarial examples or
counterfactual examples (Jiang and Bansal, 2019; Lee et al., 2021) or the
sufficiency of the supporting evidences (Trivedi et al., 2020) or make use of
the pretrained language models (Zhao et al., 2020; Zaheer et al., 2020).

However, Min et al. (2019a) demonstrated that many compositional questions in
HotpotQA can be answered with a single hop. It means that QA models can take
shortcuts instead of multi-hop reasoning to produce the corrected answer. To
relieve the issue, Jiang and Bansal (2019) added adversarial examples as hard
distractors during training. Recently, (Trivedi et al., 2020) proposed an
approach, DiRe, to measure the model’s disconnected reasoning behavior and use
the supporting sufficiency label to reduce the disconnected reasoning. Lee et
al. (2021) selected the supporting evidence according to the sentence
causality to the predicted answer, which guarantees the explainability of the
behavior of the model. While, the original performance also drops when
reducing the disconnected reasoning.

Figure 1: Illustration of disconnected reasoning in multi-hop QA, where red
node denotes question $Q$, blue node is a supporting fact $S$, and orange and
green nodes denote the remaining facts $C$. Deep gray nodes mean their
variables are reference values instead of the given values, (e.g., $S=s^{*}$
instead of $S=s$). (a): Causal graph of multi-hop QA model; (b): is a possible
scenario of disconnected reasoning, which uses only one fact $s$ to answer the
question. (c): is another possibility of disconnected reasoning, e.g., the
exclusive method to find whether $s$ is a supporting fact by a process of
elimination of other facts $c$. (d): is the true multi-hop reasoning. All
facts $s$ and $c$ are taken into considered to produce the answer.

Causal Inference. Recently, causal inference (Pearl and Mackenzie, 2018;
Pearl, 2022) has been applied to many tasks of natural language processing,
and it shows promising results and provides strong interpretability and
generalizability. The representative works include counterfactual intervention
for visual attention (Rao et al., 2021), causal effect disentanglement for VQA
(Niu et al., 2021), the back-door and front-door adjustments (Zhu et al.,
2021a; Wang et al., 2021; Yang et al., 2021). Our method can be viewed as a
complement of the recent approaches that utilize the counterfactual inference
(Lee et al., 2021; Ye et al., 2021) to identify the supporting facts and
predict answer.

##  3 Preliminaries

In this section, we use the theory of causal inference (Pearl and Mackenzie,
2018; Pearl, 2022) to formalize our multi-hop reasoning method. Support that
we have a multi-hop dataset $D$ and each instance has the form of $(Q,P;Y)$,
where $Q$ is a question and $P=\\{s_{1},s_{2},\cdots,s_{n}\\}$ is a context
consisting of a set of $n$ paragraphs. And $Y$ is the ground-truth label.
Given a question $Q$ with multiple paragraphs as a context $P$, the multi-hop
QA models are required to identify which paragraphs are the supporting facts
and predict an answer using the supporting facts.

Causal graph. In multi-hop QA, multiple supporting facts are required to
predict the answer. While QA models may use only one fact to give the answer,
which is referred as disconnected reasoning. For example, given a question $q$
and only a paragraph $s$, the disconnected reasoning model can predict the
correct answer or correctly determine whether the paragraph $s$ is the
supporting fact. Hence, to define the causal graph of disconnected reasoning,
the context $P$ is further divided into a paragraph $S$ and the remaining
paragraphs $C$. Now the $(Q,P;Y)$ becomes $(Q,S,C;Y)$. That is each instance
$(Q,P;Y)$ is converted into $n$ examples, i.e.,
$(q,s_{1},C=\\{s_{2},\cdots,s_{n}\\};Y(s_{1}))$, $\cdots$,
$(q,s_{n},C=\\{s_{1},\cdots,s_{n-1}\\};Y(s_{n}))$, where
$Y(s_{i})=\\{F_{s_{i}};A\\}$ includes the supporting fact and answer, where
$A$ is the answer and $F_{s_{i}}=1$ means the paragraph $s$ is the supporting
fact otherwise it is not. For each example, we consider the disconnected
reasoning with the fact $S$ (not $C$).

The causal graph for multi-hop QA is shown in Figure 1 (a), where nodes denote
the variables and directed edges represents the causal-and-effect
relationships between variables. The paths in Figure 1 (a) are as follows.

$(Q,S)\to K_{1}\to Y$: $(Q,S)\to K_{1}$ denotes that the feature/knowledge
$K_{1}$ is extracted from the question $(Q)$ and the paragraph $(S)$ via the
QA model backbone, e.g., BERT. $K_{1}\to Y$ represents the process that the
label $Y$ is predicted by only using the $K_{1}$.

$(Q,C)\to K_{2}\to Y$: Similarly, the feature/knowledge $K_{2}$ extracted from
the question $(Q)$ and the remaining paragraphs $(C)$ is used to predict the
label $Y$.

$(Q,S,C)\to(K_{1},K_{2})\to K\to Y$: This path indicates that the QA model
predicts the label $Y$ based on both the $K_{1}$ and $K_{2}$.

Based on the above, the effect of $Q,S,C$ on Y can be divide into: 1) shortcut
impacts, e.g., $(Q,S)\to K_{1}\to Y$ and $(Q,C)\to K_{2}\to Y$, and 2)
reasoning impact, e.g., $(Q,S,C)\to(K_{1},K_{2})\to K\to Y$. The shortcut
impacts capture the direct effect of $(Q,S)$ or $(Q,C)$ on $Y$ via
$K_{1}\rightarrow Y$ or $K_{2}\rightarrow Y$. The reasoning impact captures
the indirect effect of $(Q,S,C)$ on $Y$ via $K\rightarrow Y$.

Hence, to reduce the multi-hop QA model’s disconnected reasoning proposed in
(Trivedi et al., 2020), we should exclude shortcut impacts ($K_{1}\rightarrow
Y$ and $K_{2}\rightarrow Y$) from the total effect.

Counterfactual definitions. Figure 1 (a) shows the causal graph. From causal
graph to formula, we denote the value of $Y$, i.e., the answer $A$ (e.g.,
2008) or the supporting paragraph $F_{s}$ (e.g., is the paragraph $s$
supporting fact?), would be obtained when question $Q$ is set to $q$, the
paragraph $S$ is set to $s$ and the remaining paragraphs $c=P-s$ are used as
the context $C$, which is defined as

$\displaystyle Y_{q,s,c}(A)=Y(A\ |\ Q=q,S=s,C=c),$ $\displaystyle
Y_{q,s,c}(s)=Y(s\ |\ Q=q,S=s,C=c).$

For simplicity, we omit $A,s$ and unify both equations as
$Y_{q,s,c}=Y_{q,s,c}(A)$ or $Y_{q,s,c}=Y_{q,s,c}(s)$. Since the causal effect
of $q,s,c$ on $Y$ via $K_{1},K_{2},K$ on $Y$, we have

$Y_{k_{1},k_{2},k}=Y_{q,s,c}$ (1)

in the following discussion.

To disentangle the shortcut impacts from the total causal effect, we use the
counterfactual causal inference to block other effects. To model the $K_{1}\to
Y$, the counterfactual formulation

$K_{1}\to Y:Y_{k_{1},k_{2}^{*},k^{*}},$ (2)

which describes the situation where $K_{1}$ is set to the original value
$k_{1}$ and $K$ and $K_{2}$ are blocked. The $k^{*}$ and $k^{*}_{2}$ are the
counterfactual notations. The $k_{1}$ and $k^{*},k^{*}_{2}$ represent the the
two situations where the $k_{1}$ is under treatment in the factual scenario
and $k^{*},k^{*}_{2}$ are not under treatment Pearl and Mackenzie (2018) in
the counterfactual scenario. The same definitions for other two effects as

$K_{2}\to Y:Y_{k_{1}^{*},k_{2},k^{*}},$ (3)

and

$K\to Y:Y_{k_{1}^{*},k_{2}^{*},k}.$ (4)

Causal effects. According the counterfactual definitions, total effect of
$q,s,c$ on $Y$ can be decomposed into the natural direct effects of
$K_{1},K_{2}$ on $Y$ and the effect of $K$ on $Y$ as discussed before. The two
natural direct effects cause the disconnected reasoning problem, and the
effect of $K\to Y$ is the desired multi-hop reasoning.

As shown in Figure 1 (b), the effect of $K_{1}$ on $Y$ with $K_{2},K$ blocked
and the effect of $K_{2}$ on $Y$ with $K_{1},K$ blocked can be easily obtained
by setting the $S$ or $C$ to counterfactual values (please refer to Section 4
for more details). While the effect of $K$ on $Y$ can not be obtained by
changing the values $S$/$C$ to $S^{*}$/$C^{*}$. We follow Niu et al. (2021)
and total indirect effect (TIE) is used to express the effect of $K$ on $Y$,
which is formulated as

$Y_{k^{*}_{1},k^{*}_{2},k}=Y_{k_{1},k_{2},k}-Y_{k_{1},k_{2},k^{*}}.$ (5)

##  4 Counterfactual Multihop QA

Following the former formulations, we propose to construct the counterfactual
examples to estimate the natural direct effect of $K_{1}$ and $K_{2}$, as well
as using parameters to estimate the total indirect effect of $K$. And our
calculation of $Y$ in Eq. (2), (3) and (5) is parametrized by a neural multi-
hop QA model $\mathcal{F}$. Please note that $\mathcal{F}$ can be any multi-
hop QA models and our method is model-agnostic.

###  4.1 Disentanglement of causal effect

$K_{1}\to Y$. Specifically, in Eq. (2), the $Y_{k_{1},k_{2}^{*},k^{*}}$
describes the situation where $K_{1}$ is set to the factual value with
$Q=q,S=s$ as inputs, and $K_{2}$/$K$ are set to the counterfactual values.
Taking Figure 1 as an example, the QA model only consider the interaction
between question $q$ and a given paragraph $s$. The remaining paragraphs $c$
are not given. It is the disconnected reasoning (Trivedi et al., 2020). To
obtain the counterfactual values of $K_{2}$ and $K$, we can set the context
$C$ as its counterfactual sample, and we have

$Y_{k_{1},k_{2}^{*},k^{*}}=Y_{q,s,c^{*}}=\mathcal{F}(q,s,c^{*}).$ (6)

In this paper, we randomly sample the remaining contexts from the training set
to construct the counterfactual $c^{*}$. It represents that the no-treatment
or the prior knowledge of the remaining context is unknown. In the
implementation, we randomly sample the remaining contexts in a mini-batch to
replace the $c$ in original triple example ($q,s,c$) and obtain the
corresponding counterfactual triple example ($q,s,c^{*}$). With that, we can
feed it into the QA model to get $Y_{q,s,c^{*}}$.

$K_{2}\to Y$. Similarly, in Eq. (3), the $Y_{k_{1}^{*},k_{2},k^{*}}$ describes
the situation where $K_{2}$ is set to the factual value with the inputs $Q=q$
and $C=c$. The $K_{1}$ and $K$ are set to the counterfactual values with the
counterfactual sample $S=s^{*}$ as input, which is defined as

$Y_{k_{1}^{*},k_{2},k^{*}}=Y_{q,s^{*},c}=\mathcal{F}(q,s^{*},c).$ (7)

One may argue that how to predict the label when $S$ is set to the
counterfactual values? As the example shown in Figure 1, even without the
paragraph $s$ as input, the QA model still can infer the paragraph $s$ is
supporting fact via wrong reasoning: since all paragraphs in $C$ do not
include the words about time range, the rest paragraph $s$ should be the
supporting fact to answer the “until when" question. It is the exclusive
method. The wrong reasoning is caused by an incorrect interaction between the
paragraph $s$ and the remaining context $c$. Hence, the $S$ is set to the
counterfactual values can correct such incorrect interactions.

Hence, in the implementation, we use the adversarial examples as suggested in
Jiang and Bansal (2019) to construct the counterfactual $s^{*}$, which aims to
remove the incorrect interaction and perform multi-hop reasoning.
Specifically, we randomly perturb the 15% tokens of the paragraph $s$, 80% of
which will be replaced by other random tokens, 10% of which will be replaced
by the mask token (e.g. $[MASK]$) of the tokenizer, and 10% of which will keep
unchanged. After that, we obtain another counterfactual triple example
($q,s^{*},c$), we can feed it into QA model to get $Y_{q,s^{*},c}$.

$K\to Y$. In Eq. (5), $Y_{q,s,c}$ indicates that the question $s$, paragraph
$s$ and remaining context $c$ are visible to the QA model $\mathcal{F}$:

$Y_{k_{1},k_{2},k}=Y_{q,s,c}=\mathcal{F}(q,s,c).$ (8)

The main problem is that $Y_{k_{1},k_{2},k^{*}}$ is unknown. It is also hard
to use the counterfactual samples of $Q,S,C$ to obtain its value since the
$q,s$ and $c$ should be the factual values for $k_{1},k_{2}$. In this paper,
we follow the work Niu et al. (2021) and also assume the model will guess the
output probability under the no-treatment condition of $k^{*}$, which is
represented as

$Y_{k_{1},k_{2},k^{*}}=\mathcal{C},$ (9)

where $\mathcal{C}$ is the output and a learnable parameter. Similar to
Counterfactual VQA (Niu et al., 2021), we guarantee a safe estimation of
$Y_{k_{1},k_{2},k^{*}}$ in expectation.

Training objective. From the above discussion, we can estimate the total
causal effect:

$\displaystyle Y\leftarrow Y_{q,s,c^{*}}+Y_{q,s^{*},c}+Y_{q,s,c}-\mathcal{C}.$
(10)

Note that $Y$ can be any ground-truth labels of answer span prediction,
supporting facts identification or answer type prediction. See Appendix A.1
for further implementation details.

###  4.2 Training and Inference

| $\rm Ans$ | $\rm Supp_{p}$ | $\rm Supp_{s}$ | $\rm Ans+Supp_{p}$ | $\rm Ans+Supp_{s}$
---|---|---|---|---|---
| original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$
BERT | 74.2 | 47.5 | 94.1 | 73.9 | 83.8 | 64.1 | 71.0 | 36.5 | 64.2 | 32.5
+ours | 74.0 | 45.5 | 95.4 | 67.5 | 82.8 | 54.6 | 71.6 | 31.8 | 63.9 | 26.7
XLNET | 76.2 | 50.3 | 96.5 | 75.0 | 86.6 | 64.8 | 74.4 | 39.1 | 68.0 | 34.6
+ours | 75.9 | 49.8 | 96.6 | 74.1 | 86.6 | 63.6 | 74.1 | 38.0 | 67.9 | 33.6
DFGN | 71.7 | 44.5 | 94.4 | 73.8 | 83.8 | 64.0 | 68.7 | 34.2 | 62.1 | 30.4
+ours | 73.3 | 48.3 | 96.1 | 72.1 | 85.4 | 61.6 | 71.5 | 36.0 | 65.1 | 31.7
HGN | 73.3 | 47.0 | 91.1 | 67.4 | 81.4 | 59.0 | 68.3 | 33.6 | 62.0 | 30.2
+ours | 70.9 | 41.1 | 93.4 | 67.6 | 83.5 | 58.1 | 67.6 | 28.9 | 61.8 | 25.5
Table 1: F1 scores. The "original" denotes the model’s performance on the development set of HotpotQA in the distractor setting, and the "dire" indicates that the model scores on the corresponding probing set, which measures how much disconnected reasoning the model can achieve. The smaller score of "dire" is better. We can see that the proposed method can reduce disconnected reasoning while maintaining the same accuracy on the original dataset.  | $\rm Ans$ | $\rm Supp_{p}$ | $\rm Supp_{s}$ | $\rm Ans+Supp_{p}$ | $\rm Ans+Supp_{s}$
---|---|---|---|---|---
| original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$
BERT | 59.7 | 35.3 | 86.1 | 17.3 | 54.8 | 10.1 | 53.6 | 7.0 | 35.7 | 4.4
+ours | 60.2 | 33.7 | 87.9 | 3.7 | 53.5 | 2.0 | 55.2 | 1.4 | 36.6 | 0.8
XLNET | 62.0 | 38.0 | 91.8 | 14.5 | 58.9 | 8.5 | 58.6 | 6.5 | 40.1 | 4.2
+ours | 61.7 | 37.6 | 92.1 | 6.9 | 59.3 | 3.9 | 58.5 | 2.6 | 40.9 | 1.6
DFGN | 57.4 | 44.5 | 85.9 | 19.5 | 53.4 | 11.7 | 51.3 | 6.9 | 33.6 | 4.6
+ours | 59.6 | 35.9 | 90.9 | 6.2 | 57.7 | 3.3 | 55.9 | 2.3 | 38.4 | 1.4
HGN | 58.9 | 35.2 | 79.5 | 16.9 | 52.3 | 10.4 | 49.5 | 6.8 | 34.2 | 4.4
+ours | 57.1 | 30.1 | 85.1 | 6.4 | 56.1 | 3.3 | 51.3 | 2.2 | 36.0 | 1.2
Table 2: EM scores on the development set and probing set of HotpotQA in the
distractor setting.

Training. The model $\mathcal{F}$ is expected to disentangle the two natural
direct effect and the true multi-hop effect from the total causal effect. To
achieve this goal, we apply the Eq. $(\ref{newY})$ to train the QA model
$\mathcal{F}$. Our training strategy follows the HGN (Fang et al., 2019):

$\displaystyle\mathcal{L}$
$\displaystyle=\mathcal{L}_{start}+\mathcal{L}_{end}+\lambda\mathcal{L}_{sent}$
(11)
$\displaystyle+\mathcal{L}_{para}+\mathcal{L}_{type}+\mathcal{L}_{entity},$

where $\lambda$ is a hyper-parameter and each term of $\mathcal{L}$ is cross-
entropy loss function. Specifically, for answer prediction, we utilize the Eq.
(10) to obtain the predicted logits of the start and end position of the
answer span, and respectively calculate the cross-entropy loss
$\mathcal{L}_{start}$ and $\mathcal{L}_{end}$ with corresponding ground truth
labels. As for supporting facts prediction, similarly, we use the Eq. (10) to
calculate the predicted logits in sentence level and paragraph level, and then
calculate $\mathcal{L}_{sent}$ and $\mathcal{L}_{para}$. We also apply our
counterfactual reasoning method to identify the answer type (Qiu et al., 2019;
Fang et al., 2019), which consists of yes, no, span and entity. We use the
$[CLS]$ token as the global representation to predict the answer type under
the Eq. (10) and calculate $\mathcal{L}_{type}$ with the ground truth label.
Entity prediction ($\mathcal{L}_{entity}$) (Fang et al., 2019) is only a
regularization term and the Eq. (10) is not applied to this term.

Inference. As illustrated in Section 3, our goal is to exclude the natural
direct effect ($K_{1}\rightarrow Y,K_{2}\rightarrow Y$) and use the true
multi-hop effect ($K\rightarrow Y$) to reduce the multi-hop QA model’s
disconnected reasoning, so we use Eq. (5) for inference:

$\mathcal{F}(q,s,c)-\mathcal{C}.$ (12)

The time consumption of our approach is equal to the existing methods.

##  5 Experiments

We extensively conduct the experiments on the HotpotQA (Yang et al., 2018)
dataset. The compared results show that our proposed algorithm can reduce the
amount of the disconnected reasoning while obtaining the strong performance on
the original dev set. And it is general and suitable for other multi-hop QA
architectures.

Datasets: We utilize the distractor setting of HotpotQA, where each question
is equipped with two ground truth supporting paragraphs and eight distractor
paragraphs as the context. And the answer of the question is annotated as a
span in one of the supporting paragraphs. To measure the disconnected
reasoning (dire) in HotpotQA models, we generate the probing dataset following
(Trivedi et al., 2020), which is only used in the test phase. Specifically,
the probing dataset for HotpotQA in the distractor setting divides each
example of the original dataset into two instances, both of which only contain
one of two ground truth supporting paragraphs respectively. If the multi-hop
QA model can arrive at the correct test output on two instances, it means that
the model performs disconnected reasoning on the original example. Please
refer to Trivedi et al. (2020) for more details.

Baselines: First, we simply use the BERT (Devlin et al., 2018) to predict the
answer, supporting sentences and supporting paragraphs as the baseline.
$BERT+ours$ denotes that we apply our counterfactual multi-hop reasoning
method based on BERT as the backbone. The proposed approach is model-agnostic
and we also implement it on several multi-hop QA architectures, including DFGN
(Qiu et al., 2019), HGN (Fang et al., 2019) and XLNet in Dire (Trivedi et al.,
2020; Yang et al., 2019). Our proposed algorithm also can be implemented on
other baselines.

| $\rm Ans$ | $\rm Supp_{p}$ | $\rm Supp_{s}$ | $\rm Ans+Supp_{p}$ | $\rm Ans+Supp_{s}$
---|---|---|---|---|---
| original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$
BERT | 74.2 | 47.5 | 94.1 | 73.9 | 83.8 | 64.1 | 71.0 | 36.5 | 64.2 | 32.5
+$K_{1}\to Y$ | 74.2 | 50.2 | 96.6 | 73.5 | 85.7 | 61.9 | 72.5 | 37.8 | 65.7 | 32.8
+$K_{2}\to Y$ | 74.6 | 48.8 | 96.4 | 68.2 | 85.6 | 57.7 | 72.8 | 34.5 | 66.0 | 30.1
+ours(full) | 74.0 | 45.5 | 95.4 | 67.5 | 82.8 | 54.6 | 71.6 | 31.8 | 63.9 | 26.7
Table 3: F1 scores of ablation study on the development set and probing set of HotpotQA in the Distractor setting. The "+ $K_{1}\to Y$" denotes that we only utilize the counterfactual examples ($q,s,c^{*}$) to estimate the shortcut impact of $K_{1}\to Y$, and similarly the "+ $K_{2}\to Y$" represents that only the counterfactual examples ($q,s^{*},c$) are used to estimate the shortcut impact of $K_{2}\to Y$. | $\rm Ans$ | $\rm Supp_{p}$ | $\rm Supp_{s}$ | $\rm Ans+Supp_{p}$ | $\rm Ans+Supp_{s}$
---|---|---|---|---|---
| original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$ | original | dire$\downarrow$
random | 74.0 | 45.5 | 95.4 | 67.5 | 82.8 | 54.6 | 71.6 | 31.8 | 63.9 | 26.7
uniform | 73.8 | 45.9 | 92.1 | 67.4 | 76.6 | 46.3 | 69.2 | 31.8 | 59.6 | 23.2
Table 4: F1 scores of ablation study of assumptions for counterfactual outputs
$\mathcal{C}$ on the development set and probing set of HotpotQA in the
distractor setting.

Metrics: Following the Dire (Trivedi et al., 2020), we report the metrics for
HotpotQA: answer span ($\rm Ans$), supporting paragraphs ($\rm Supp_{p}$),
supporting sentences ($\rm Supp_{s}$), joint metrics ($\rm
Ans+Supp_{p},Ans+Supp_{s}$). We show both EM scores and F1 scores to compare
the performance between baselines and our counterfactual multi-hop reasoning
method.

###  5.1 Quantitative Results

For fairness, we conduct the experiments under the same preprocessing of the
dataset following HGN (Fang et al., 2019), which select top K relevant
paragraphs corresponding to each question example. And the experimental
results are shown in Table 1 and Table 2. The main observation can be made as
follows:

Our method can reduce disconnected reasoning. Compared to BERT baseline, our
proposed counterfactual multi-hop reasoning method can reduce the disconnected
reasoning of answer prediction and supporting facts identification in both the
paragraph level and sentence level. In particular, we can see big drops of 9.5
F1 points on $\rm Supp_{s}$ (from 64.1 to 54.6) and 13.6 EM points on $\rm
Supp_{p}$ (from 17.3 to 3.7) in disconnected reasoning (dire). Our method are
better at reducing disconnected reasoning on the Exact Match (EM) evaluation
metric. This is because EM is stricter evaluation metric. For example, EM
requires both of the supporting facts should be predicted correctly while it
has F1 scores even when only one supporting fact is predicted correctly. For
dire evaluation where only one supporting fact is provided, our approach
punishes this situation and achieves lower scores on EM metric of disconnected
reasoning. It demonstrates that our method effectively reduce disconnected
reasoning when there are without sufficient supporting facts.

Our method still guarantees the comparable performance on the original dev
set. As seen from Table 1 and Table 2, the proposed method also maintain the
same accuracy on the original set. It even shows the better performance on the
supporting facts prediction in the paragraph level.

Our method is model-agnostic and it demonstrates effectiveness in several
multi-hop QA models.  Based on our causal-effect insight, our proposed
approach can easily be applied to other multi-hop QA architectures including
XLNET, DFGN, HGN (Trivedi et al., 2020; Qiu et al., 2019; Fang et al., 2019).
As shown in Table 1 and Table 2, our proposed counterfactual reasoning method
achieves better performance. Our method can reduce the disconnected reasoning
by introducing the proposed counterfactual approach in the training procedure.
The dire scores of HGN (Fang et al., 2019) and XLNET (Trivedi et al., 2020) in
$\rm Ans,Supp_{p},Supp_{s}$ all drop to some extent. Besides, the performances
on the original dev set are comparable simultaneously. It indicates that our
proposed method is more stable and general.

In summary, reducing the disconnected reasoning and guarantee the strong
performance on original development set indicate that the most progress of the
model is attributed to the multi-top reasoning ($K\to Y$) capability. For
intuitiveness, we also show the real multi-hop reasoning promoted by our
proposed counterfactual reasoning approach, as shown in Fig. 2.

Figure 2: F1 scores of real multi-hop reasoning, which is denoted as the
original scores minus the dire scores. We compare BERT, DFGN, HGN and ours
method except XLNET, as they utilize the same pretrained language model (i.e.
bert-base-uncased).

###  5.2 Ablation Study

As illustrated in Section 4, our goal is to exclude the shortcut impacts
($K_{1}\to Y,K_{2}\to Y$) to reduce the disconnected reasoning. Hence, we
study the ablation experiments of excluding one of shortcuts impacts. We
explore to remove $K_{1}\to Y$ or $K_{2}\to Y$ that reduce the disconnected
reasoning, as shown in Table 3. We can see that exclude one of them can
decrease the amount of disconnected reasoning to some extent on supporting
facts identification except the answer span prediction. However, relieving the
both impacts of $K_{1}\to Y$ and $K_{2}\to Y$ can achieve better performance
on decreasing disconnected reasoning. Because the model can always exploit
another shortcut if only one of the shortcuts is blocked.

We further conduct ablation studies to validate the distribution assumption
for the counterfactual output of the parameter $\mathcal{C}$. Similar to CF-
VQA (Niu et al., 2021), we empirically validate the two distribution
assumptions, as shown in Table 4. The "random" denotes that $\mathcal{C}$ are
learned without constraint and it means that
$\mathcal{C}_{Ans}\in\mathbb{R}^{n},\mathcal{C}_{supp}\in\mathbb{R}^{2},\mathcal{C}_{type}\in\mathbb{R}^{4}$
respectively, and $n$ represents the length of the context. The "uniform"
denotes that $\mathcal{C}$ should satisfy uniform distribution and it means
that $\mathcal{C}_{Ans},\mathcal{C}_{supp}$ and $\mathcal{C}_{type}$ are
scalar. As shown in Table 4, the random distribution assumption performs
better than the uniform distribution assumption.

##  6 Conclusion

In this work, we proposed a novel counterfactual reasoning approach to reduce
the disconnected reasoning in multi-hop QA. We used the causal graph to
explain the existing multi-hop QA approaches’ behaviors, which consists of the
shortcut impacts and reasoning impacts. The shortcut impacts capture the
disconnected reasoning. Thus the disconnected reasoning is formulated as
natural direct causal effect and we construct the counterfactual examples
during the training phase to estimate the both natural direct effects of
question and context on answer prediction as well as supporting facts
identification. The reasoning impact represents the multi-hop reasoning and is
estimated via introducing learnable parameters.

During the test phase, we excluded the natural direct effect and utilize the
true multi-hop effect to decrease the disconnected reasoning. Experimental
results demonstrate that our proposed counterfactual reasoning method can
significantly drop the disconnected reasoning on probing dataset and guarantee
the strong performance on original dataset, which indicates the most profress
of the multi-hop QA model is attributed to the true multi-hop reasoning.
Besides, our approach is model-agnostic, and can be applied to other multi-hop
QA architectures to prevent the model from exploiting the shortcuts.

Overall, our insights of reducing disconnected reasoning and learning the true
multi-hop reasoning can motivate the development of new methods which
contributes to promoting the multi-hop question answering task.

## Limitations

Our proposed method needs to construct counterfactual examples to estimate the
natural direct effect of disconnected reasoning during the training phase,
thus we need a little more GPU resources and computational time. However, the
need of resource occupancy and time consumption of our approach does not
increase during inference. Another limitation is that we use the learnable
parameters to approximate the $Y_{k_{1},k_{2},k^{*}}$. In our future work, we
will explore better approach to model it.

## References

  * Asai et al. (2019) Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. 2019.  Learning to retrieve reasoning paths over wikipedia graph for question answering.  _arXiv preprint arXiv:1911.10470_.
  * Chen et al. (2019) Jifan Chen, Shih-ting Lin, and Greg Durrett. 2019.  Multi-hop question answering via reasoning chains.  _arXiv preprint arXiv:1910.02610_.
  * Das et al. (2019) Rajarshi Das, Ameya Godbole, Dilip Kavarthapu, Zhiyu Gong, Abhishek Singhal, Mo Yu, Xiaoxiao Guo, Tian Gao, Hamed Zamani, Manzil Zaheer, et al. 2019.  Multi-step entity-centric information retrieval for multi-hop question answering.  In _Proceedings of the 2nd Workshop on Machine Reading for Question Answering_ , pages 113–118.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_.
  * Ding et al. (2019) Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. 2019.  Cognitive graph for multi-hop reading comprehension at scale.  _arXiv preprint arXiv:1905.05460_.
  * Fang et al. (2019) Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, and Jingjing Liu. 2019\.  Hierarchical graph network for multi-hop question answering.  _arXiv preprint arXiv:1911.03631_.
  * Groeneveld et al. (2020) Dirk Groeneveld, Tushar Khot, Ashish Sabharwal, et al. 2020.  A simple yet strong pipeline for hotpotqa.  _arXiv preprint arXiv:2004.06753_.
  * Jiang and Bansal (2019) Yichen Jiang and Mohit Bansal. 2019.  Avoiding reasoning shortcuts: Adversarial evaluation, training, and model development for multi-hop qa.  _arXiv preprint arXiv:1906.07132_.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  _arXiv preprint arXiv:2004.04906_.
  * Lee et al. (2021) Kyungjae Lee, Seung-won Hwang, Sang-eun Han, and Dohyeon Lee. 2021.  Robustifying multi-hop qa through pseudo-evidentiality training.  _arXiv preprint arXiv:2107.03242_.
  * Li et al. (2021) Shaobo Li, Xiaoguang Li, Lifeng Shang, Xin Jiang, Qun Liu, Chengjie Sun, Zhenzhou Ji, and Bingquan Liu. 2021.  Hopretriever: Retrieve hops over wikipedia to answer complex questions.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 13279–13287.
  * Min et al. (2019a) Sewon Min, Eric Wallace, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019a.  Compositional questions do not necessitate multi-hop reasoning.  _arXiv preprint arXiv:1906.02900_.
  * Min et al. (2019b) Sewon Min, Victor Zhong, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019b.  Multi-hop reading comprehension through question decomposition and rescoring.  _arXiv preprint arXiv:1906.02916_.
  * Nie et al. (2019) Yixin Nie, Songhe Wang, and Mohit Bansal. 2019.  Revealing the importance of semantic retrieval for machine reading at scale.  _arXiv preprint arXiv:1909.08041_.
  * Nishida et al. (2019) Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, and Junji Tomita. 2019.  Answering while summarizing: Multi-task learning for multi-hop qa with evidence extraction.  _arXiv preprint arXiv:1905.08511_.
  * Niu et al. (2021) Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen. 2021.  Counterfactual vqa: A cause-effect look at language bias.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 12700–12710.
  * Pearl (2022) Judea Pearl. 2022.  Direct and indirect effects.  In _Probabilistic and Causal Inference: The Works of Judea Pearl_ , pages 373–392.
  * Pearl and Mackenzie (2018) Judea Pearl and Dana Mackenzie. 2018.  _The book of why: the new science of cause and effect_.  Basic books.
  * Qi et al. (2019) Peng Qi, Xiaowen Lin, Leo Mehr, Zijian Wang, and Christopher D Manning. 2019.  Answering complex open-domain questions through iterative query generation.  _arXiv preprint arXiv:1910.07000_.
  * Qiu et al. (2019) Lin Qiu, Yunxuan Xiao, Yanru Qu, Hao Zhou, Lei Li, Weinan Zhang, and Yong Yu. 2019\.  Dynamically fused graph network for multi-hop reasoning.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 6140–6150.
  * Rao et al. (2021) Yongming Rao, Guangyi Chen, Jiwen Lu, and Jie Zhou. 2021.  Counterfactual attention learning for fine-grained visual categorization and re-identification.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1025–1034.
  * Shao et al. (2020) Nan Shao, Yiming Cui, Ting Liu, Shijin Wang, and Guoping Hu. 2020.  Is graph structure necessary for multi-hop question answering?  _arXiv preprint arXiv:2004.03096_.
  * Trivedi et al. (2020) Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2020\.  Is multihop qa in dire condition? measuring and reducing disconnected reasoning.  _arXiv preprint arXiv:2005.00789_.
  * Wang et al. (2021) Tan Wang, Chang Zhou, Qianru Sun, and Hanwang Zhang. 2021.  Causal attention for unbiased visual recognition.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 3091–3100.
  * Xiong et al. (2020) Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick Lewis, William Yang Wang, Yashar Mehdad, Wen-tau Yih, Sebastian Riedel, Douwe Kiela, et al. 2020.  Answering complex open-domain questions with multi-hop dense retrieval.  _arXiv preprint arXiv:2009.12756_.
  * Yang et al. (2021) Xu Yang, Hanwang Zhang, Guojun Qi, and Jianfei Cai. 2021.  Causal attention for vision-language tasks.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 9847–9857.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32.
  * Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018.  Hotpotqa: A dataset for diverse, explainable multi-hop question answering.  _arXiv preprint arXiv:1809.09600_.
  * Ye et al. (2021) Xi Ye, Rohan Nair, and Greg Durrett. 2021.  Connecting attributions and qa model behavior on realistic counterfactuals.  _arXiv preprint arXiv:2104.04515_.
  * Zaheer et al. (2020) Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020.  Big bird: Transformers for longer sequences.  _Advances in Neural Information Processing Systems_ , 33:17283–17297.
  * Zhao et al. (2020) Chen Zhao, Chenyan Xiong, Corby Rosset, Xia Song, Paul Bennett, and Saurabh Tiwary. 2020.  Transformer-xh: Multi-evidence reasoning with extra hop attention.
  * Zhu et al. (2021a) Beier Zhu, Yulei Niu, Xian-Sheng Hua, and Hanwang Zhang. 2021a.  Cross-domain empirical risk minimization for unbiased long-tailed classification.  _arXiv preprint arXiv:2112.14380_.
  * Zhu et al. (2021b) Yunchang Zhu, Liang Pang, Yanyan Lan, Huawei Shen, and Xueqi Cheng. 2021b.  Adaptive information seeking for open-domain question answering.  _arXiv preprint arXiv:2109.06747_.

##  Appendix A Appendix

###  A.1 Implementation Details

Specifically, given the question $Q=q$, and context
$P=\\{s_{1},s_{2},...,s_{m}\\}$ where $m$ is the number of paragraphs, we
denote the remaining context $c_{i}=P-\\{s_{i}\\}$, $1\leq i\leq m$. To
distinguish whether $s_{i}$ is supporting fact and get the answer distribution
on $s_{i}$, we construct the $s_{i}^{*}$ and $c_{i}^{*}$ as illustrated in the
subsection 4.1. We respectively encode $(q,s_{i},c_{i})$,$(q,s_{i}^{*},c_{i})$
and $(q,s_{i},c_{i}^{*})$ to get the contextualized representation $O\in
R^{n\times d},M_{i}\in^{n\times d},G_{i}\in R^{n\times d}$, where $n$ is the
length of the question and context.

For supporting facts identification, similar to Dire (Trivedi et al., 2020),
we use the start token $s_{i}^{start}$ of the paragraph as its representation
and obtain their predicted logits under factual and counterfactual scenario:

$\displaystyle Y_{q,s_{i},c_{i}}(s_{i})=g(O[s_{i}^{start}])$ (13)
$\displaystyle Y_{q,s_{i}^{*},c_{i}}(s_{i})=g(M_{i}[s_{i}^{start}])$
$\displaystyle Y_{q,s_{i},c_{i}^{*}}(s_{i})=g(G_{i}[s_{i}^{start}]),$

where $g$ is a classifier instantiated as $MLP$ layer in practice. And
$s_{i}^{start}$ and $s_{i}^{end}$ are denoted as the start and end position of
the paragraph $s_{i}$ respectively. In sentence level, we use the their start
positions in paragraph $s_{i}$ and operate in the same way.

As for answer span prediction, we concatenate the representation of $s_{i}$ in
$M_{i}$ or $G_{i}$ $(1\leq i\leq m)$ to construct the entire answer span
prediction on the whole context:

$\displaystyle\bar{M}=[M_{1}[s_{1}^{start}:s_{1}^{end}];...;M_{m}[s_{m}^{start}:s_{m}^{end}]]$
(14)
$\displaystyle\bar{G}=[G_{1}[s_{1}^{start}:s_{1}^{end}];...;G_{m}[s_{m}^{start}:s_{m}^{end}]]$
$\displaystyle Y_{q,s,c}(start)=g(O)$ $\displaystyle
Y_{q,s^{*},c}(start)=g(\bar{M})$ $\displaystyle
Y_{q,s,c^{*}}(start)=g(\bar{G}),$

where $[;]$ denotes the operation of concatenation. And we can predict the end
position of the answer in the same way.

We also apply our counterfactual reasoning method to identify the answer type,
consisting of yes, no, span and entity:

$\displaystyle Y_{q,s,c}(type)=g(O[0])$ (15) $\displaystyle
Y_{q,s^{*},c}(type)=\sum_{i=1}^{m}g(M_{i}[0])$ $\displaystyle
Y_{q,s,c^{*}}(type)=\sum_{i=1}^{m}g(G_{i}[0]).$

We use the $[CLS]$ token as the global representation to predict the answer
type, following previous work (Qiu et al., 2019; Fang et al., 2019).