# Bumps, chimera states, and Turing patterns in systems of coupled active
rotators

Igor Franović<EMAIL_ADDRESS>Scientific Computing Laboratory, Center for
the Study of Complex Systems, Institute of Physics Belgrade, University of
Belgrade, Pregrevica 118, 11080 Belgrade, Serbia  Oleh E. Omel’chenko
<EMAIL_ADDRESS>University of Potsdam, Institute of Physics and
Astronomy, Karl-Liebknecht-Str. 24/25, 14476 Potsdam-Golm, Germany  Matthias
Wolfrum<EMAIL_ADDRESS>Weierstrass Institute, Mohrenstrasse 39, 10117
Berlin, Germany

###### Abstract

Self-organized coherence-incoherence patterns, called chimera states, have
first been reported in systems of Kuramoto oscillators. For coupled excitable
units, similar patterns where coherent units are at rest, are called bump
states. Here, we study bumps in an array of active rotators coupled by non-
local attraction and global repulsion. We demonstrate how they can emerge in a
_supercritical_ scenario from completely coherent Turing patterns: a single
incoherent unit appears in a homoclinic bifurcation, undergoing subsequent
transitions to quasiperiodic and chaotic behavior, which eventually transforms
into extensive chaos with many incoherent units. We present different types of
transitions and explain the formation of coherence-incoherence patterns
according to the classical paradigm of short-range activation and long-range
inhibition.

Since their discovery in 2002 by Kuramoto & Battogtokh [1] chimera states have
attracted remarkable attention. They represent a new type of self-organization
phenomenon where identical units in a system with symmetric couplings develop
a stable pattern with regions of qualitatively different behavior. In their
original work, Kuramoto and Battogtokh found such patterns with self-organized
domains of synchronized (coherent) and non-synchronized (incoherent)
oscillators in systems of phase oscillators in a one-dimensional array with
nonlocal coupling. After the term chimera state was coined by Abrams &
Strogatz [2], it has been used for similar phenomena in a large variety of
theoretical models [3, 4] and has also been demonstrated in various
experiments [5, 6, 7, 8]. In spite of the abundance of examples of chimera
states [9, 10, 11, 12, 13, 14, 15, 16], sometimes even only loosely related to
the original phenomenon from phase oscillator systems, an understanding of a
principal mechanism leading to their formation is still missing [17]. In [2],
it has been pointed out as an intriguing property of chimera states that they
”cannot be ascribed to a supercritical instability” since they always stably
coexist with the uniform locked state. Since then, no supercritical scenario
leading to the emergence of chimera states has been presented. Some recent
progress has been made only for the case of Stuart-Landau oscillators with
global nonlinear coupling, where clustering has been identified as a
prerequisite for chimera states in such systems [18].

In this Letter, we show that the solution to this outstanding problem can be
found by applying the classical paradigm [19, 20] of short-range activation
and long-range inhibition to the synchrony in an array of coupled
excitable/oscillatory units. We demonstrate that the coherence-incoherence
patterns can emerge in a _supercritical_ scenario via a Turing instability of
completely coherent states and a secondary homoclinic bifurcation, creating a
_single_ incoherent oscillator, a state which can be seen as a _weak chimera_
in the sense of [21]. Subsequent transitions via periodic, quasiperiodic and
chaotic states with an increasing number of incoherent oscillators finally
lead to a fully developed coherence-incoherence pattern with localized
extensive chaos in the incoherent region. Remarkably, this scenario is
essentially independent on the system size. We achieve this by introducing two
modifications to the original phase oscillator system. In addition to the
nonlocal attractive coupling, we introduce a global repulsive coupling, and
the uniformly rotating phase oscillators are replaced by so-called active
rotators, which can be in an oscillatory or excitable regime. Coupled units of
this type can be seen as a simplified version of neuronal oscillators similar
to theta neurons [22, 23], and under excitatory and/or inhibitory coupling
they are known to display various types of localized or propagating spiking
patterns. In particular, they can display localized states of activity, so-
called bump states [24, 25, 26, 28, 27], which have also been extensively
studied in continuum models for neuronal mean-field activity [29, 30, 31, 32,
33, 34, 35, 36, 37]. With these additions, our system shows a variety of self-
organized patterns, see Fig. 1. Already completely coherent states, where all
active rotators have identical average frequencies $\omega_{k}$, can be locked
($\omega_{k}=0$) or unlocked ($\omega_{k}\neq 0$), and in both cases they can
be spatially uniform or spatially modulated, see Fig. 1(a) and (b),
respectively. Incoherent regions, where average frequencies $\omega_{k}$ are
gradually varying, can occur interspersed with locked coherent regions, as in
bump states, see Fig. 1(c)-(d), or with unlocked coherent regions, as observed
in chimera states. The point that bumps and chimeras are related by a
collective unlocking of the coherent region has previously been described in a
system of active rotators even without global repulsion, having a classical
chimera subjected to a global periodic forcing [38].

Along with a mechanism of emergence, a related puzzling aspect is that in
their original form, chimeras cannot be observed in small systems. In [39], it
has been shown that even for large system size, they are in fact chaotic
transients collapsing to the completely coherent state after a lifetime that
is exponentially increasing with the system size. There is of course no reason
to believe that both these properties are necessarily true for any chimera-
like phenomenon in systems other than Kuramoto’s original phase oscillators.
Indeed, Kuramoto’s simple phase oscillator system allows for variations only
in the phase lag parameter and the shape of the non-local coupling.
Introducing of a more general coupling function has led to a discovery of weak
chimera states [21] which can also occur in rather small systems. However,
they share only some of the properties of the classical chimeras, and it
remained unclear to which extent the mechanisms of their emergence could serve
as a general explanation of the original chimera phenomenon. In case of bump
states, our results demonstrate that they can stably exist in small systems
without eventually collapsing to a completely coherent state.

Figure 1: Dynamical regimes of (1) with $N=100$, $P=35$, $\alpha=0.6$ and
different choices of $K_{1},\,K_{2},\,a$. All the solutions were obtained from
coherent spatially modulated initial conditions, and are plotted centered at
$k=50$ using translational invariance. Snapshots of phases $\theta_{k}$
(colored symbols) and average frequencies $\omega_{k}$ (black) in (a)–(d).
(a)–completely coherent locked states for $a=1.2$: homogeneous (green) at
$K_{1}=1.4,K_{2}=1.8$, spatially modulated (red, blue) at
$K_{1}=1.4,K_{2}=1.983$ and $K_{1}=2,K_{2}=2.495$. (b)–time-periodic
(unlocked) completely coherent states for $a=0.5,K_{2}=1.8$: homogeneous
(blue) at $K_{1}=3.4$, spatially modulated (red) at $K_{1}=3.3$; different
symbols of the same color indicate snapshots at different time moments. Bump
states: (c)–single-headed for $K_{1}=1.4,K_{2}=2,a=1.2$, (d)–two-headed for
$K_{1}=2,K_{2}=2.52,a=1.2$. Corresponding space-time plots of phase velocities
$\dot{\theta_{k}}(t)$ in (e) and (f).

We start with an array of $N$ oscillators where the dynamics of phases
$\theta_{j}\in S^{1}$, $j=1,\dots,N$ is given by

$\displaystyle\frac{\displaystyle d{\theta}_{j}}{\displaystyle dt}$
$\displaystyle=$ $\displaystyle 1-a\cos\theta_{j}-\frac{\displaystyle
K_{1}}{\displaystyle
2P+1}\sum\limits_{k=j-P}^{j+P}\sin(\theta_{j}-\theta_{k}+\alpha)$ (1)
$\displaystyle+$ $\displaystyle\frac{\displaystyle K_{2}}{\displaystyle
N}\sum\limits_{k=1}^{N}\sin(\theta_{j}-\theta_{k}),\qquad j=1,\dots,N,$

where $K_{1}>0$ denotes the strength of the nonlocal attractive coupling and
$K_{2}>0$ the global repulsive coupling. For $a=0$, $K_{2}=0$, and an
appropriate choice of the phase lag parameter $0<\alpha<\pi/2$ and the
coupling range $1<P<N$, this system is known to give rise to chimera states,
see e.g. [41, 42, 40]. Below $|a|=1$ the dynamics of individual units changes
from oscillatory to excitable.

Figure 2: Instabilities of the homogeneous locked state: fold (4) with
$\kappa=0$ (black), existence region (gray); Turing instability (4) for the
mode with wave number $\kappa=1$ (red), stability region (orange); dashed
parts of the curves lie on the unstable sheet. (a)– locking cone in the
$(a,K_{1})$ plane for fixed $K_{2}=1.4$; (b)– locking region in the
$(K_{1},K_{2})$ plane for fixed $a=1.2$. Other parameters:
$\alpha=0.6,N=100,P=35$. Vertical lines in (b) indicate choices of $K_{1}$ in
Fig. 3.

The system (1) admits completely coherent homogeneous locked states

$\theta_{j}(t)\equiv\theta^{\pm}=\pm\arccos\left[\frac{1-K_{1}\sin\alpha}{a}\right],\quad
1\leq j\leq N,$ (2)

which come in pairs within a locking cone

$(K_{1}\sin\alpha-1)^{2}<a^{2},$ (3)

with its tip located at $a=0$, $K_{1}=1/\sin\alpha$, cf. Fig. 2(a). Note that
the locking cone does not depend on the global coupling $K_{2}$, since there
is no phase lag in the corresponding coupling function. However, $K_{2}$
strongly affects the stability of the homogeneous locked states. Their
Jacobian is a symmetric circulant matrix with real spectrum and discrete
Fourier modes as eigenfunctions. The bifurcation condition for the mode with
wave number $\kappa$ is given by

$a^{2}=(K_{1}(1-R_{\kappa})\cos\alpha+(\delta_{\kappa
0}-1)K_{2})^{2}+(1-K_{1}\sin\alpha)^{2},$ (4)

where

$R_{\kappa}=\frac{1}{2P+1}\sum_{m=-P}^{P}\cos(2\pi\kappa m/N)$

is the corresponding discrete Fourier component of the nonlocal coupling term.
Note that inserting $\kappa=0$ into (4) we recover the fold bifurcations
outlining the locking cone (3), while $\kappa=1,\dots N$ leads to a discrete
Turing instability [43, 44] with wave number $\kappa$. In Fig. 2 we show the
regions of existence and stability of the coherent uniform locked states. For
$K_{2}=0$, the homogeneous locked state $\theta^{-}$ is stable within the
whole locking region. Increasing $K_{2}$, the system undergoes a discrete
Turing instability with the leading mode $\kappa=1$. If this bifurcation is
supercritical, we obtain a stable spatially modulated completely coherent
state (Turing pattern), see also [46, 45]. We analyze now in detail four
different destabilization scenarios of the homogeneous locked state induced by
increasing the repulsive coupling $K_{2}$ along the vertical lines in Fig.
2(b), which all finally lead to the onset of a bump state.

## Sub- and Supercritical transitions to bump states

Figure 3: Colored regions indicate increasing width of incoherent regions of
bump states for varying $K_{2}$ and $K_{1}\in\\{0.7,1,1.4,2.5\\}$. Spatial
parameter $x=k/N$ ($k$–oscillator index). Other parameters: $a=1.2$,
$\alpha=0.6$. Turing instability of the homogeneous locked state (horizontal
red line), where stable branches of modulated coherent locked states emerge
(hatched region), ending at a saddle-node (black dashed line). If the saddle-
node is a SNIC (cases (c) and (d)), there is a supercritical transition to
bump states. In case (b) a saddle node of the modulated coherent state induces
a subcritical transition to a fully developed bump. In case (a), the Turing
instability is subcritical (red dashed line) and the solution jumps from the
homogeneous coherent state to a bump.

In Fig. 3 are illustrated different scenarios for the emergence of bump states
showing how the incoherent region grows with increasing global repulsion
$K_{2}$ for different choices of $K_{1}$. For larger values of $K_{1}$, see
panels (b)-(d), the Turing bifurcation is supercritical and a branch of stable
spatially modulated coherent states appears (hatched region). In (c) and (d),
the stable branch of modulated coherent states extends to a SNIC (saddle-node
on invariant circle) bifurcation. This instability represents the
supercritical transition from a classical Turing pattern to a coherence-
incoherence pattern. Remarkably, it is characterized by unlocking of single
localized oscillators, independent on the system size $N$. Further increasing
$K_{2}$ leads to the subsequent unlocking of neighboring oscillators and the
coherence-incoherence pattern gradually attains temporal and spatial
complexity. In Fig. 3(d) is shown a scenario where the modulated coherent
state develops two maxima, such that two incoherent regions emerge
simultaneously. Increasing $K_{2}$ further, the two incoherent regions merge
into a single one. During this process, the branch folds over and a region of
coexistence of two different bump solutions appears. Fixing $K_{1}=1.4$, we
observe a single monotonically growing incoherent region, as shown in Fig.
3(c). This transition will be investigated in more detail below. For
$K_{1}=1.0$, shown in panel (b), the saddle-node of the stable branch of
modulated coherent states is a classical saddle-node bifurcation, which does
not involve an invariant circle and is not localized to a single unlocking
oscillator. Such a _collective instability_ can induce a subcritical
transition to a coexisting fully developed bump state, with incoherent region
of finite size. This transition shows a hysteretic behavior when the coupling
parameter $K_{2}$ is reduced again, whereby the bump state disappears in a
chaotic saddle before the size of the incoherent region completely vanishes.
For small values of $K_{1}$ the Turing instability becomes subcritical, and
there is a direct transition from the homogeneous coherent state to a fully
developed bump, shown in panel (a), displaying the same hysteretic behavior as
described above.

Figure 4: (a)-(b) Bifurcation diagrams in $K_{2}$: time intervals $\Delta
t_{n}$ between successive velocity peaks. Space-time plots of phase
velocities: periodic patterns in (c), (d), (f) and (g); chaotic patterns
without (e) and with drift instability (h) for $K_{2}$ values indicated by
dash-dotted lines in (a) and (b). Other parameters are
$K_{1}=1.4,a=1.2,\alpha=0.6,N=20,P=7$.

## Microscopic structure of the supercritical transition to bump states.

Directly after the SNIC bifurcation, when the number of incoherent oscillators
in the bump states is small, one can observe an intricate scenario of
increasing spatial and temporal complexity, which finally leads to high
dimensional extensive chaos. For large $N$, this transition is confined to a
small parameter interval and one observes the almost immediate emergence of a
small region of extensive chaos. In Figure 4 we chose $N=20$ such that we can
study in detail an example of such a transition. The resulting dynamics can be
characterized by the spatio-temporal pattern of the single excitation events
of the individual oscillators, which manifest themselves as localized peaks in
the phase velocity. A selection of such patterns is given in panels (c)-(h) of
Figure 4, while in panels (a) and (b) we show a full parameter scan with
respect to $K_{2}$ where we sampled the return times $\Delta t_{n}$ between
two consecutive peaks performed by any of the oscillators [42]. Starting from
the simple periodic pattern with one incoherent oscillator that emerges from
the SNIC of the modulated coherent state, we see multiple transitions between
regular and chaotic states of increasing complexity. The transitions to chaos
are mostly of intermittency type, but also torus breakup, illustrated in Fig.
5, and period-doubling cascades can be observed. The shadings of different
color in panels (a) and (b) indicates the increasing number of incoherent
oscillators. Note that for $K_{2}\approx 2.0255$ the chaotic lateral motion of
the incoherent region, which was described in [39] for classical chimera
states, sets in. Obviously, the specific shape of the transition scenario
depends crucially on even small variations of the system parameters, in
particular the number of oscillators $N$. However, a similar global scenario
has been reported in [42], where the classical chimera system of [1] has been
extended by a control term, such that also chimera states with a small number
of incoherent oscillators became visible.

Figure 5: Emergence of chaos under increasing $K_{2}$. Torus bifurcation from
periodic to quasiperiodic pattern at $K_{2}\approx 1.9394$, onset of chaos via
torus breakup at $K_{2}\approx 1.93955$. Inset: space-time plots of phase
velocities for quasiperiodic pattern at $K_{2}=1.9395$. Remaining parameters:
$K_{1}=1.3,a=1.2,\alpha=0.6,N=20,P=7$.

## Outlook and discussion

Our system of excitable/oscillatory units with attractive and repulsive
coupling, as given in (1), shows an extremely rich variety of dynamics. Here
we have focused on the emergence of coherence-incoherence patterns and
demonstrated how the classical paradigm of pattern formation by A. Turing [19]
and Gierer & Meinhardt [20] in terms of local activation and long-range
inhibition leads to the formation of coherence-incoherence patterns in a
_supercritical_ transition scenario. In the spatially extended discrete medium
of active rotators the attractive and repulsive coupling with different
spatial ranges does not activate/inhibit the local activity, as in neural
field models, but acts on the local synchrony and in this way induces a
pattern of qualitatively different behavior, rather than inducing
quantitatively different levels of local activity as in the classical examples
of pattern formation in neural field models. In this way, also the classical
chimera states, which are related to the bump states discussed here by a
simple collective unlocking of the coherent region as described in [38], are
no longer an isolated phenomenon in the family of patterns, as stated in [2],
but can be seen as a specific type of a Turing pattern, where a spatial
modulation results in a self-localized unlocking that emerges gradually from a
smooth coherent profile. We have further explained how this transition depends
on the coupling strengths $K_{1,2}$. Moreover, we have shown that the
coherence-incoherence patterns can be found for a large range of other
parameters: in particular, the fine tuning of the phase lag $\alpha$ slightly
below $\pi/2$ that was necessary to obtain chimera states in the classical
setting [40, 4] is no longer needed. Also, another puzzling aspect of chimera
states in their original form could be resolved in our modified system. In
[39], it was shown that chimeras cannot be observed in small systems, and that
even for large system size, they behave as chaotic transients which collapse
to the completely coherent state. With our extension of Kuramoto’s simple
phase oscillator system, coherence-incoherence patterns no longer need to
coexist with the stable homogeneous state but, as we demonstrated, can be
found as stable attractors even for small system size. In this way,
sophisticated control schemes, which have been constructed for their
observation [47, 48], become no longer necessary. Instead, our model is
universal in the sense that the transition between the classical subcritical
scenario and the new supercritical scenario for the onset of coherence-
incoherence patterns is achieved by the coupling parameters.

###### Acknowledgements.

I.F. acknowledges funding from the Institute of Physics Belgrade through the
grant by the Ministry of Education, Science and Technological Development of
the Republic of Serbia. The work of O.O. was supported by the Deutsche
Forschungsgemeinschaft under grant OM 99/2-1. The work of M.W. was supported
by the Deutsche Forschungsgemeinschaft (DFG, German Research
Foundation)–Projektnummer 163436311–SFB 910.

## References

  * [1] Y. Kuramoto, and D. Battogtokh, Nonlinear Phenom. Complex Syst. 5,380 (2002).
  * [2] D. M. Abrams, and S. H. Strogatz, Phys. Rev. Lett. 93, 174102 (2004).
  * [3] F. Parastesh, S. Jafari, H. Azarnoush, Z. Shahriari, Z. Wang, S. Boccaletti, and M. Perc, Phys. Rep. 898, 1 (2021).
  * [4] M. J. Panaggio and D. M. Abrams, Nonlinearity 28, R67 (2015).
  * [5] J. F. Totz, J. Rode, M. R. Tinsley, K. Showalter, and H. Engel, Nature Phys. 14, 282 (2018).
  * [6] A. M. Hagerstrom, T. E. Murphy, R. Roy, P. Hövel, I. Omelchenko, and E. Schöll, Nature Phys. 8, 658 (2012).
  * [7] C. Lainscsek, N. Rungratsameetaweemana, S. S. Cash, and T. J. Sejnowski, Chaos 29, 121106 (2019).
  * [8] R. G. Andrzejak, C. Rummel, F. Mormann, and K. Schindler, Sci. Rep. 6, 23000 (2016).
  * [9] O. E. Omel’chenko and E. Knobloch, New J. Phys. 21, 093034 (2019).
  * [10] D. M. Abrams, R. Mirollo, S. H. Strogatz, and D. A. Wiley, Phys. Rev. Lett. 101, 084103 (2008).
  * [11] E. A. Martens, C. Bick, and M. J. Panaggio, Chaos 26, 094819 (2016).
  * [12] M. Mikhaylenko, L. Ramlow, S. Jalan, A. Zakharova, Chaos 29, 023122 (2019).
  * [13] A. Zakharova, _Chimera Patterns in Networks: Interplay Between Dynamics, Structure, Noise, and Delay_ (Springer International Publishing, 2020).
  * [14] I. Omelchenko, O. E. Omel’chenko, P. Hövel, E. Schöll, Phys. Rev. Lett. 110, 224101 (2013).
  * [15] N. Semenova, A. Zakharova, V. S. Anishchenko, E. Schöll, Phys. Rev. Lett. 117, 014102 (2016).
  * [16] K. Höhlein, F. P. Kemeth, and K. Krischer, Phys. Rev. E 100, 022217 (2019).
  * [17] Y. Zhang and A. E. Motter, Phys. Rev. Lett. 126, 094101 (2021).
  * [18] L. Schmidt, and K. Krischer, Phys. Rev. Lett. 114, 034101 (2015).
  * [19] A. M. Turing, Philos. Trans. R. Soc. B 237, 37 (1952).
  * [20] A. Gierer and H. Meinhardt, Kybernetik 12, 30 (1972).
  * [21] P. Ashwin, and O. Burylko, Chaos 25, 013106 (2015).
  * [22] C. R. Laing, Phys. Rev. E 90, 010901(R) (2014).
  * [23] T. B. Luke, E. Barreto, and P. So, Neural Comput. 25, 3207 (2013).
  * [24] M. Owen, C. Laing, and S. Coombes, New J. Phys. 9, 378 (2007).
  * [25] C. R. Laing, Physica D 240, 1960 (2011).
  * [26] C. R. Laing, Frontiers Comput. Neurosci. 10, 53 (2016).
  * [27] C. R. Laing, and O. E. Omel’chenko, Chaos 30, 043117 (2020).
  * [28] H. Schmidt and D. Avitabile, Chaos 30, 033133 (2020).
  * [29] C. Bick, M. Goodfellow, C. R. Laing, and E. A. Martens, J. Math. Neurosci. 10, 9 (2020).
  * [30] Á. Byrne, D. Avitabile, and S. Coombes, Phys. Rev. E 99, 012313 (2019).
  * [31] S. Coombes, Biol. Cybern. 93, 91 (2005).
  * [32] P. C. Bressloff, J. Phys. A Math. Theor. 45, 033001 (2012).
  * [33] C. R. Laing, W. C. Troy, B. Gutkin, and G. B. Ermentrout, SIAM J. Appl. Math. 63, 62 (2002).
  * [34] C. R. Laing, and C. Chow, Neural Comput. 13, 1473 (2001).
  * [35] K. Wimmer, D. Q. Nykamp, C. Constantinidis, and A. Compte, Nat. Neurosci. 17, 431 (2014).
  * [36] E. Montbrió, D. Pazó, and A. Roxin, Phys. Rev. X 5, 021028 (2015).
  * [37] J. M. Esnaola-Acebes, A. Roxin, D. Avitabile and E. Montbrió, Phys. Rev. E 96, 052407 (2017).
  * [38] M. I. Bolotov, L. A. Smirnov, G. V. Osipov, and A. Pikovsky, Phys. Rev. E 102, 042218 (2020).
  * [39] M. Wolfrum, and O. E. Omel’chenko, Phys. Rev. E 84, 015201(R) (2011).
  * [40] O. E. Omel’chenko, Nonlinearity 31, R121 (2018).
  * [41] O. E. Omel’chenko, Nonlinearity 26, 2469 (2013).
  * [42] M. Wolfrum, O. E. Omel’chenko, and J. Sieber, Chaos 25, 053113 (2015).
  * [43] M. Wolfrum, Physica D: Nonlinear Phenomena 241, 1351 (2012).
  * [44] T. Carletti and H. Nakao Phys. Rev. E 101, 022203 (2020).
  * [45] C.-U. Choe, R.-S. Kim, and J.-S. Ri, Phys. Rev. E 98, 012210 (2018).
  * [46] R.-S. Kim, and C.-U. Choe, Phys. Rev. E 98, 042207 (2018).
  * [47] J. Sieber, O. E. Omel’chenko, and M. Wolfrum, Phys. Rev. Lett. 112, 054102 (2014).
  * [48] I. Omelchenko, O. E. Omel’chenko, A. Zakharova, and E. Schöll, Phys. Rev. E 97, 012216 (2018).


# Divide & Conquer for Entailment-aware Multi-hop Evidence Retrieval

Fan Luo
University of Arizona
Tucson, AZ, USA
<EMAIL_ADDRESS>
&Mihai Surdeanu
University of Arizona
Tucson, AZ, USA
<EMAIL_ADDRESS>

###### Abstract

Lexical and semantic matches are commonly used as relevance measurements for
information retrieval. Together they estimate the semantic equivalence between
the query and the candidates. However, semantic equivalence is not the only
relevance signal that needs to be considered when retrieving evidences for
multi-hop questions. In this work, we demonstrate that textual entailment
relation is another important relevance dimension that should be considered.
To retrieve evidences that are either semantically equivalent to or entailed
by the question simultaneously, we divide the task of evidence retrieval for
multi-hop question answering (QA) into two sub-tasks, i.e., semantic textual
similarity and inference similarity retrieval. We propose two ensemble models,
EAR and EARnest, which tackle each of the sub-tasks separately and then
jointly re-rank sentences with the consideration of the diverse relevance
signals. Experimental results on HotpotQA verify that our models not only
significantly outperform all the single retrieval models it is based on, but
is also more effective than two intuitive ensemble baseline models.

## 1\. Introduction

Widely adopted QA approaches use a two-stage pipeline, i.e., a retriever
module followed by a reader module (Chen et al., 2017). The retriever is
responsible for collecting relevant evidences, then the reader module combines
the relevant information from the retriever module to infer the answer.
Evidence retrieval is a ranking task, which usually maps questions and
candidates to vectors and scores the semantic relationship between them with
respect to relevance. The goal is to rank the most relevant pieces of evidence
(supporting facts) at the top of the list among all the candidates.

According to formal semantic notions, the semantic relationship between two
text fragments includes semantic equivalence, referential equality, and
textual entailment. While referential equality can be mostly solved by
coreference resolution and entity linking, semantic similarity and textual
entailment require deep semantic understanding between question and context
(Lin et al., 2021). Textual entailment is a framework that captures semantic
inference. Textual Entailment (TE) of two text fragments can be defined as the
task of deciding whether the meaning of one text fragment can be inferred from
another text fragment. That is, a premise T entails a hypothesis H if,
typically, a human reading T would infer that H is most likely true. For
example, “T: Jack sold the house to Peter. H: Peter owns the house. Here H can
be inferred from T, so T entails H. While semantic equivalence capture “Do
these two texts mean the same thing?”, textual entailment is a framework that
captures semantic inference, deciding whether the meaning of one text can be
inferred from another.

Question: What nationality was James Henry Miller’s wife? Answer: American
Supporting Evidences Ewan MacColl (1) James Henry Miller (25 January 1915 – 22
October 1989), better known by James Henry Miller stage name Ewan MacColl, was
an English folk singer, songwriter, communist, labour activist, actor, poet,
playwright and record producer . Peggy Seeger (2) Margaret "Peggy" Seeger
(born June 17 , 1935) is an American folksinger. (3) She is also well known in
Britain, where she has lived for more than 30 years, and was married to the
singer and songwriter Ewan MacColl until his death in 1989. Semantic Equality
James Henry Miller (Q) $\approx$ James Henry Miller (25 January 1915 – 22
October 1989), better known by James Henry Miller stage name Ewan MacColl (1)
Textual Entailment
American (2) $\mkern 2.0mu\vdash$ nationality (Q) was married to (3) $\mkern
2.0mu\vdash$ wife (Q)

Figure 1:  An example from the HotpotQA dataset showing the two different
dimensions of relevance between the question and its supporting evidences.

It is common to utilize an inference model as a reader to infer the correct
answer from a subset of retrieved context. However, most works of the evidence
ranking only measure the semantic textual similarity between the question and
candidate corpus to determine the relevance , and ignore the inference
signals. This works for single-hop questions, in which relevant information
usually share the same entity mentions with the question. However, it is not
sufficient for multi-hop questions. The relevance between multi-hop question
and its evidence(s) is beyond the lexical or semantic similarity that is
targeted by most retrievers, especially for secondary hops. As the example in
Figure 1 shows, relevance is multi-dimensional. The first hop evidence “James
Henry Miller (25 January 1915 – 22 October 1989), better known by James Henry
Miller stage name Ewan MacColl, was an English folk singer, songwriter,
communist, labour activist, actor, poet, playwright and record producer" has
lexical overlap with the question and thus get high scores from token overlap
match and semantic similarity comparison. However, the subsequent hops in the
paragraph ‘Peggy Seeger’ failed to be retrieved by both lexical match and
semantic similarity comparison, as they would not be considered as “mention or
mean the same thing” as the question. Instead, they have an entailment
relation with the question. “Margaret ‘Pegg’ Seeger (born June 17, 1935) is an
American folksinger.” entails a nationality relationship, and similarly ‘James
Henry Miller’s wife’ entails a marriage relationship. Thus, we argue that
semantic similarity is not the only relevance signal that needs to be
considered when ranking evidences for multi-hop questions.

To consider the fine-grained aspects of relevance for the multi-hop QA
evidence retrieval task, we divide the task into separate retrieval subtasks,
where each aims to retrieve a subset of sentences that score highly on one of
the relevance dimension (i.e., semantic equivalence or textual entailment)
respectively, and then combine them to output the final ranking with an
ensemble model.

Our contributions are: 1\. We call attention that fine-grained aspects of a
model’s capability in performing types of relevance signals should be
considered. Especially, textual entailment should be explicitly taken into
consideration when measure relevance for complex question answering evidence
retrieval, so that the retriever covers a more accurate relevant context that
not only lexical match based but also require inference to identify. 2\. we
propose two ensemble models that combine diverse relevance signals captured by
different base models. Our experimental results demonstrate that not only are
the individual base retrieval model necessary in evidence retrieval but
cooperate advantageously to produce a better ranking for multi-hop QA evidence
retrieval when used together . 3\. We empirically show the effectiveness of
the proposed ensemble retrieval models by evaluating on multi-hop HotpotQA
dataset and show they not only outperform all the base models, and also
several ensemble baselines.

## 2\. Related Work

### 2.1. Text Retrieval

Traditional retrieval models such as TF-IDF and BM25 (Trotman et al., 2014)
use sparse bag-of-words representations to collect lexical matching signals
(e.g., term frequency). Such sparse retrieval models are mostly limited to
exact matches. Dense Retrieval models move away from sparse signals to dense
representations, which help address the vocabulary mismatch problem. These
models can be categorized into two types according to their model
architecture, representation-based (Huang et al., 2013; Shen et al., 2014) and
interaction-based models (Pang et al., 2016; Lu and Li, 2013; Guo et al.,
2016; Mitra et al., 2017). Hybrid methods (Lin et al., 2021a; Gao et al.,
2020; Karpukhin et al., 2020; Shan et al., 2020) aggregate the dense retrieval
with sparse retrieval methods to better model relevance. The entailment-aware
retrieval models we propose are also hybrid methods that combine sparse and
dense retrieval methods, but our method is unsupervised. Further, we combine a
sparse model with multiple dense models to consider diverse relevance signals,
i.e., textual entailment in addition to semantic equivalence.

### 2.2. Multi-hop Evidence Retrieval

Research works on multi-hop evidence retrieval can be broadly categorized into
two directions: (1) Question decomposition: (Min et al., 2019; Jiang and
Bansal, 2019; Fu et al., 2021; Perez et al., 2020; Talmor and Berant, 2018)
decompose multi-hop questions into multiple single-hop sub-questions. The
question decomposition works that relies on a decomposer model trained via
supervised labels to be able to decompose the multi-hop questions into single-
hop questions accurately. In contrast, our entailment-aware retrieval models
tackle this task in the opposite direction. That is, instead of decompose the
question, we assemble candidate sentences pairs that carrying different
relevance signals to match against the question. (2) Iterative evidence
retrieval: Feldman and El-Yaniv (2019) proposed a method to iteratively
retrieve supporting paragraphs, using the paragraphs retrieved in previous
iteration to reformulate the search vector. (Asai et al., 2020) iteratively
retrieve a subsequent passage in the reasoning chain with an RNN. Qi et al.
(2019) trained a retriever to generate a query from the question and the
available context at each step. Das et al. (2019) proposed a multi-step
reasoner, which reformulates the question into its latent space with respect
to its current value and the state of the reader. While iterative retrieval
considers evidence retrieval as a sequence process, so that the accuracy of
subsequent retrieval steps highly depends on previous decisions, our method
jointly consider high potential evidence pairs simultaneously.

## 3\. Methodology

In this section, we introduce two ensemble models for entailment-aware multi-
hop QA evidence retrieval. At a high-level, we model diverse relevance
relationships with three base models, and combine the relevance signals they
capture to jointly retrieve candidate evidences for multi-hop questions.

### 3.1. Task

The evidence ranking for the multi-hop QA task broadly involves two different
dimensions of relevance to measuring: semantic equivalence and textual
entailment. Figure 1 shows a multi-hop question example from the HotpotQA
dataset. Beside the semantic equivalence relation between the question and its
first hop evidence, the textual entailment relation between the question and
its secondary hop evidence, such as American and nationality, is another
important relevance dimension that should be considered. To identify evidences
that are either semantically equivalent to or entailed by the question
simultaneously, we divide the task of evidence ranking for multi-hop QA into
two separate ranking tasks, i.e., semantic textual similarity and inference
similarity based ranking. _Semantic textual similarity_ is a commonly used
relevance measurement for in evidence ranking. _Inference Similarity_ (IS)
measures the textual entailment between the question and its candidate
evidences. For example, ‘where’ questions would have high IS with evidence
about locations or directions. Each of the two sub-tasks aims to rank
candidates that score highly on one of the relevance dimensions respectively,
and then combine them to output the final ranking with an ensemble model.

Given a multi-hop question Q and a corpus $C=\\{P_{1},P_{2},\dots,P_{m}\\}$
containing a set of documents or paragraphs, a retrieval system aims to select
a small set of relevant supporting facts for the reader module to infer the
answer. An efficient retriever selects a small subset $D$ of documents that
are likely relevant to the question. That is, a retriever $F:(q,C)\rightarrow
D$ is a function that takes as input a question $q$ and a corpus $C$ and
returns a much smaller filtered set $D\subset C$. It firstly performs sentence
segmentation (sentence as the “unit of indexing") over each of the documents
in the retrieved subset $D$ to get $M$ total candidate sentences $S$, and then
estimates the relevance of a question and candidate evidence sentence pairs
$r(q,s)$, ranks the candidate sentences in $S$, and outputs the top-ranked $N$
relevant sentences as supporting facts for answering $q$. Formally, given the
input of a question q and candidate sentences $S$, a evidence ranking function
of $R:q,S\rightarrow S^{\prime}$ takes the candidate sentences in $S$, and
returns a subset $S^{\prime}$ with $r(q,s_{h})>r(q,s_{l})$ if $h<l$, where
$r(q,S)=(s_{1},...,s_{k})$. Relevance estimation of each candidate sentence to
the question is clearly the key to the task. Two dimensions of relevance
(i.e., semantic equivalence and entailment) need to be considered in order to
provide a more accurate estimation of the relevance of each candidate sentence
for the multi-hop questions.

We divide this multi-criteria task into two separate ranking subtasks:
semantic equivalence as well as textual entailment.111For the focus of this
work, we conduct coreference resolution within each paragraph in advance. Both
tasks require comparing information between the question and candidates, but
the objectives of the comparison are different. In this work, we capture the
entailment relations in parallel with the semantic equivalence with separate
models, which produce different and potentially conflicting rankings. The goal
is to combine them to figure out an aggregated ranking that better estimates
overall relevance and promotes gold evidence sentences to the top of the list.

### 3.2. Base Models

In order to build a ranking method that does not rely on a large training set
with evidence annotations, we chose three off-the-shelf base models to capture
diverse relevance patterns. To better estimate semantic equivalence, we use
both a sparse model (i.e., BM25) and a dense model to examine exact match and
semantic match respectively. BM25 relies on lexical match, which looks for
literal matches of the question words in the document collection. In addition,
we utilized another dense model for capturing entailment relations. While
making use of BM25 as a relevance signal relies on the lexical overlap, dense
models address a variety of linguistic phenomena beyond exact term matching,
including synonyms, paraphrases, term variation, and different expressions of
similar intents. For example, a dense model would be able to better match
"password reset" with "account unlock" and fetch the relevant context, while a
term-based system would have difficulty connecting them without exact token
matching. Dense models using semantic search overcome the shortcomings of
lexical search and can recognize synonyms and acronyms, and thus are
complementary to sparse vector models.

For the dense models, we choose two Pre-trained Cross-Encoders (CE) instead of
Bi-Encoders. While by definition, CE supports more extensive query–document
interactions than bi-encoders and thus can exploit richer relevance signals
and achieve better performances (Reimers and Gurevych, 2019). CEs pass the
query and a possible document simultaneously to the transformer network, which
then outputs a single score between 0 and 1 indicating how relevant the
document is for the given query. The advantage of CE is the higher
performance, as they perform attention across the query and the document.

The two Pre-trained CEs we choose are:

##### MSMARCO Passage Cross-Encoder

is trained on the MS Marco Passage Ranking dataset (Bajaj et al., 2016) for
retrieval task. MS MARCO is a large scale corpus consists of about 500k real
search queries from the Bing search engine with 1000 most relevant passages.
The model is trained to rank the most relevant passages that answer the query
as high as possible. It is a strong base model commonly used for semantic
search to identify relevant information for the given search query.

##### QNLI Cross-Encoder

is a Pre-trained model obtained using the Question Natural Language Inference
(QNLI) dataset introduced by GLUE Benchmark (Wang et al., 2018). Given a
passage from Wikipedia, annotators created questions that are answerable by
that passage. The positive examples are (question, sentence) pairs that
contain the correct answer, and the negative examples are pairs with sentence
from the same paragraph that do not contain the answer. QNLI was automatically
derived from SQuAD, the Stanford Question Answering Dataset v1.1 with the
processing target of question-answer entailment.

### 3.3. Ensemble Models

Ensemble modeling is a process where multiple diverse base models are created
to predict an outcome, and then the ensemble model aggregates the prediction
of each base model and results in one final prediction. Since the three base
models aforementioned independently capture diverse relevance signals and are
complementing each other as shown in table 7, an ensemble model that seeks the
wisdom of crowds to take advantage of all base models should potentially
improve the final retrieval performance if an appropriate aggregation strategy
is designed to combine them.

In this section, we explore several ensemble techniques to combine the base
models. We first introduce two simple and intuitive ranking aggregation
strategies as baselines, and then present two ensemble models and explain why
they are better than the baselines. We will show the benefits of our ensemble
models in the experimental results (Section 3.3.3).

#### 3.3.1. Ensemble Baselines

##### Average ranking

(AR) is a simple ensemble ranking model which combines the ranking outputs
from the multiple base models that ranks all candidate sentences
independently. It simply sums up all the ranks from base models for individual
candidate evidence, and rank all the candidates according to the summation of
the ranks. A rank of a candidate sentence obtained by each base model is with
respect to the relevance signal the base model targets. Thus, each sentence
has M ranks (where M is the number of base models). The final ranking is
obtained by sorting the sum of all M rankings that each sentence received.
Table 1 illustrates the AR method with a simple example.

Sents | R${}_{\scaleto{\mathcal{{\cal BM\text{25}}}}{3pt}}$ | R${}_{\scaleto{\mathcal{MSMARCO}}{3pt}}$ | R${}_{\scaleto{\mathcal{QNLI}}{3pt}}$ | Sum(R) | AR
---|---|---|---|---|---
_S 1_ | 1 | 1 | 5 | 7 | 1
_S 2_ | 4 | 3 | 2 | 9 | 3
_S 3_ | 3 | 4 | 6 | 13 | 5
_S 4_ | 5 | 6 | 3 | 14 | 6
_S 5_ | 6 | 5 | 1 | 12 | 4
_S 6_ | 2 | 2 | 4 | 8 | 2
Table 1: A pseudo-example of an average ranking method. Each of the 6
candidate sentences is ranked by the three base models according to its degree
of relevance to the query with respect to the relevance signal each model
captures. When aggregating the ranking results of the various models, the
average ranking method simply sums all the ranks for each sentence, and rank
all the sentences according to the summation of the ranks of each sentence.

##### Similarity Combination

(SimCom) calculates hybrid retrieval scores through a linear combination of
sparse and dense scores. For a given question, we aggregate the scores
produced by the base models through a weighted average for each candidate
sentence, called Question Evidence Relevance (QER) (see the equation in
Appendix B). QER are then used to rank the candidate evidence sentences.

#### 3.3.2. Entailment-Aware Retrieval

In this work, we propose an entailment-aware retrieval (EAR) method to jointly
consider pairs of candidate sentences that potentially contain complementary
relevance signals. We form such pairs using the Cartesian product of two sets
of top-ranked candidate sentences as for semantic equivalence and textual
entailment respectively.

While BM25 and MSMARCO Cross-Encoder capture exact and semantic matches,
respectively, they both aim for estimating STS. Thus, we take the union of
top-ranked sentences by BM25 and MSMARCO Cross-Encoder as a unified set
$\mathcal{A}=\\{S_{a_{1}},S_{a_{2}},S_{a_{3}},S_{a_{4}}\\}$, and top-ranked
sentences by QNLI Cross-Encoder as another set
$\mathcal{B}=\\{S_{b_{1}},S_{b_{2}},S_{b_{3}}\\}$. The pairs we consider are
$\mathcal{P}=\mathcal{A}\times\mathcal{B}=\\{(\mathbf{a,b})\,|\,\mathbf{a}\in\mathcal{A}\wedge\mathbf{b}\in\mathcal{B}\\}$.
We then concatenate the two sentences of each pair as a sequence to score
against the question with a initial ranker222We use the MSMARCO Cross-Encoder
as the ranker since it is the best performing base model., such that the top-
scored sentence pair ($S_{a_{i}},S_{b_{j}}$) is most likely to form a
compositional relevant context covering both semantic equivalence and
entailment relevance signals. When $S_{a_{i}}$ and $S_{b_{j}}$ are examined
individually, there is a high chance that $S_{a_{i}}$ receives a low IS score
from the QNLI cross-encoder and is ranked down to the list, $S_{b_{j}}$ can be
scored and ranked low by BM25 and MSMARCO cross-encoder. Thus, either using
individual base models or aggregating ranking or scores with the ensemble
baseline models, $S_{a_{i}}$ and $S_{b_{j}}$ are unlikely to be both promoted
to the top of the ranking list. Finding the best combination from the top-
ranked subsets with respect to both semantic equivalence and textual
entailment efficiently takes the compositional requirement into consideration.
In case there are additional relevant sentences besides the highly relevant
sentence pair for a question, we further concatenate the question q with the
pair $S_{a_{i}}$ and $S_{b_{j}}$ as a new query to rank the rest of the
candidate sentences.

#### 3.3.3. EARnest

Evidences for a multi-hop question should be intuitively related, and often
logically connected via a shared named entity that would allow a human reader
to connect the information they contain. Thus, the presence of a shared named
entity between two candidate sentences often indicates the likelihood that the
sentence pairs relate to each other and, so that they can be connected to form
a coherent context for the question. To leverage such connection as an
additional cue, we add a named entity similarity term ($NEST$) to the scoring
function of the reranker in EAR when estimate the top-scored sentence pairs
as:

$\mathcal{QER}_{\scaleto{Earnest}{4pt}}\;=\;(1\;+\;\mathcal{NEST})\;*\;Sim(q,\;s_{i}\mathbin{\|}s_{j})$

where $Sim()$ is the scoring function of the reranker, which scores the
concatenation of sentence pair $S_{i}$ and $S_{j}$ against the question.
$NEST$ is a binary switch, that is, if the two sentences share one or more
named entity, the promotion mechanism is activated; otherwise it is
deactivated.

Named Entity Similarity Term  Besides using SpaCy (Honnibal et al., 2020) to
recognize named entities with common entity types (such as names of people,
places, and organizations), we also consider titles of documents and phrases
between a pair of single or double quotes. When comparing whether two
sentences share an entity, we apply basic normalization (i.e., lower case,
removing articles and special punctuations) and fuzzy match to tolerate typos,
variations, and inclusive match.

## 4\. Evaluation and Results

### 4.1. Dataset

We conduct the evaluation using HotpotQA dataset (Yang et al., 2018). HotpotQA
contains two question categories: bridge-type and comparison-type questions.
Given the focus of this work, we use solely the 5918 bridge-type questions out
of the 7,405 examples from the development partition in the distractor setting
in our evaluation.333On average, comparison-type questions are easier to
answer because the necessary information (i.e., the two entities to be
compared) tends to be present in the question. Each question in HotpotQA is
supported by two documents, and provided with ground-truth supporting
sentences, which enables us to evaluate the evidence retrieval performance of
the various models.

### 4.2. Results

To evaluate and compare the performance of the base models and various
ensemble methods, we adopt the standard performance evaluation metrics for
retrieval tasks, including precision at different cut-off points ($P@k$), mean
average precision ($MAP$), and recall at different cut-off points ($R@k$).

Table 2 reports the evidence retrieval performance of all models discussed.
Three base models that target either semantic equivalence or inference do not
yield optimal performance. As expected, the MSMARCO CE achieves the highest
performance among the base models, as it is a strong baseline that is commonly
used for retrieval tasks. However, it only considers the semantic matching
between question and individual candidate sentences, ignoring other important
relevance matching characteristics such as exact matching, textual entailment,
and relatedness between candidate evidence sentences.

For the baseline ensemble models, AR performs worse than the MSMARCO CE, while
being slightly better than BM25 and the QNLI CE. Its retrieval performance is
essentially a compromise among the performances of the three base models,
because it directly averages the individual ranking results. In contrast,
SimCom 444The result of the SimCom uses equation in Appendix B with
$\alpha\\!=\\!3$ and $\beta\\!=\\!1$, which achieves the highest performance
according to the grid search results on 10% of full dataset.  does take
advantage of complementary relevance signals from the base models, so to
perform better than any of the individual base model. This suggests that with
an appropriate aggregation, the base models cooperate advantageously to
produce a better ranking. However, it fails to deliver the best overall
performance because it simply combines the final output scores from the base
models without exploiting the interactions between the relevance signals
behind.

Lastly, our approaches (i.e., EAR and EARnest) not only outperform the base
models, but also exceed the order-based and score-based ensemble models on all
metrics. They both jointly consider diverse relevance signals simultaneously,
and therefore achieve greater improvements on the retrieval performances.
EARnest further considers the relatedness between evidence sentences, becoming
our best model. It achieves the highest MAP, and higher than the MSMARCO CE by
10%555One drawback of $P@n$ and $R@n$ is that they fail to take into account
the positions of the relevant sentences among the top n. And the average
number of ground-truth evidence sentences is less than 3. For n that is larger
than 3, even a perfect system will have a $P@n$ score less than 1, and
decreasing with higher n. Thus, $MAP$ is a relative more importance metric to
exam..

Models | P@3 | P@5 | MAP | R@3 | R@5 |  R@10
---|---|---|---|---|---|---
Base models
BM25 | 0.43 | 0.31 | 0.59 | 0.54 | 0.65 |  0.78
MSmarco | 0.47 | 0.33 | 0.64 | 0.59 | 0.69 |  0.81
QNLI | 0.33 | 0.25 | 0.46 | 0.43 | 0.52 |  0.65
Ensemble Baselines
AR | 0.43 | 0.31 | 0.61 | 0.55 | 0.66 |  0.83
SimCom | 0.5 | 0.36 | 0.68 | 0.63 | 0.74 |  0.86
Our Approach
EAR | 0.53 | 0.36 | 0.71 | 0.66 | 0.76 |  0.86
EARnest | 0.55 | 0.38 | 0.74 | 0.7 | 0.78 |  0.87
Table 2: Evidence retrieval results of base models, baseline ensembles, and
our methods on HotpotQA. The result shows that our proposed ensemble methods
(EAR and EARnest) are effective for improving the retrieval performance in
terms of all the metrics. Our best model EARnest achieves the highest MAP
performance, outperforming all the base models and ensemble baselines.

## 5\. Conclusion

In this work, we have shown that our ensemble models, which capture different
dimensions of relevance and combine them to jointly retrieve candidate
evidences, are effective for improving the retrieval performance for multi-hop
questions when compared to all the single retrieval models they are based on,
as well as the order-based and score-based ensemble baseline models.

## References

  * Asai et al. (2020) Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. 2020.  Learning to retrieve reasoning paths over wikipedia graph for question answering.
  * Bajaj et al. (2016) Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016.  Ms marco: A human generated machine reading comprehension dataset.  _arXiv preprint arXiv:1611.09268_.
  * Chen et al. (2017) Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017.  Reading wikipedia to answer open-domain questions.
  * Min et al. (2019b) Min, S., V. Zhong, L. Zettlemoyer, and H. Hajishirzi (2019b).  Multi-hop Reading Comprehension through Question Decomposition and Rescoring.
  * Molino et al. (2016) Molino, P., L. M. Aiello, and P. Lops (2016).  Social question answering: Textual, user, and network features for best answer prediction.  _ACM Transactions on Information Systems (TOIS)_ , 35(1), pp. 1–40.
  * Nakade et al. (2018) Nakade, V., A. Musaev, and T. Atkison (2018).  Preliminary research on thesaurus-based query expansion for Twitter data extraction.  In _Proceedings of the ACMSE 2018 Conference_ , pp. 1–4.
  * Nallapati (2004) Nallapati, R. (2004).  Discriminative models for information retrieval.  In _Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval_ , pp. 64–71.
  * Ng et al. (2000) Ng, H. T., L. H. Teo, and J. L. P. Kwan (2000).  A machine learning approach to answering questions for reading comprehension tests.  In _2000 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora_ , pp. 124–132.
  * Nguyen et al. (2018) Nguyen, D. T., S. Sharma, H. Schulz, and L. E. Asri (2018).  From film to video: Multi-turn question answering with multi-modal context.  _arXiv preprint arXiv:1812.07023_.
  * Nguyen et al. (2016) Nguyen, T., M. Rosenberg, X. Song, J. Gao, S. Tiwary, R. Majumder, and L. Deng (2016).  MS MARCO: A human generated machine reading comprehension dataset.  In _CoCo@ NIPS_.
  * Nie et al. (2019) Nie, Y., S. Wang, and M. Bansal (2019).  Revealing the importance of semantic retrieval for machine reading at scale.  _arXiv preprint arXiv:1909.08041_.
  * Das et al. (2019) Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. 2019.  Multi-step retriever-reader interaction for scalable open-domain question answering.
  * Feldman and El-Yaniv (2019) Yair Feldman and Ran El-Yaniv. 2019.  Multi-hop paragraph retrieval for open-domain question answering.
  * Fu et al. (2021) Ruiliu Fu, Han Wang, Xuejun Zhang, Jun Zhou, and Yonghong Yan. 2021.  Decomposing complex questions makes multi-hop qa easier and more interpretable.  _arXiv preprint arXiv:2110.13472_.
  * Gao et al. (2020) Luyu Gao, Zhuyun Dai, Zhen Fan, and Jamie Callan. 2020.  Complementing lexical retrieval with semantic residual embedding. corr abs/2004.13969 (2020).  _arXiv preprint arXiv:2004.13969_.
  * Guo et al. (2016) Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft. 2016.  A deep relevance matching model for ad-hoc retrieval.  In _Proceedings of the 25th ACM international on conference on information and knowledge management_ , pages 55–64.
  * Honnibal et al. (2020) Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020.  spaCy: Industrial-strength Natural Language Processing in Python.
  * Huang et al. (2013) Po-Sen Huang, Xiaodong He, Jianfeng Gao, Li Deng, Alex Acero, and Larry Heck. 2013.  Learning deep structured semantic models for web search using clickthrough data.  In _Proceedings of the 22nd ACM international conference on Information & Knowledge Management_, pages 2333–2338.
  * Jiang and Bansal (2019) Yichen Jiang and Mohit Bansal. 2019.  Self-assembling modular networks for interpretable multi-hop reasoning.  _arXiv preprint arXiv:1909.05803_.
  * Karpukhin et al. (2020) Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020.  Dense passage retrieval for open-domain question answering.  _arXiv preprint arXiv:2004.04906_.
  * Lin et al. (2021a) Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021a.  Pyserini: An easy-to-use python toolkit to support replicable ir research with sparse and dense representations.  _arXiv preprint arXiv:2102.10073_.
  * Lin et al. (2021b) Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. 2021b.  Pretrained transformers for text ranking: Bert and beyond.  _Synthesis Lectures on Human Language Technologies_ , 14(4):1–325.
  * Lu and Li (2013) Zhengdong Lu and Hang Li. 2013.  A deep architecture for matching short texts.  _Advances in neural information processing systems_ , 26.
  * Min et al. (2019) Sewon Min, Victor Zhong, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2019.  Multi-hop reading comprehension through question decomposition and rescoring.
  * Mitra et al. (2017) Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017.  Learning to match using local and distributed representations of text for web search.  In _Proceedings of the 26th International Conference on World Wide Web_ , pages 1291–1299.
  * Pang et al. (2016) Liang Pang, Yanyan Lan, Jiafeng Guo, Jun Xu, and Xueqi Cheng. 2016.  A study of matchpyramid models on ad-hoc retrieval.  _arXiv preprint arXiv:1606.04648_.
  * Perez et al. (2020) Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. 2020.  Unsupervised question decomposition for question answering.  _arXiv preprint arXiv:2002.09758_.
  * Qi et al. (2019) Peng Qi, Xiaowen Lin, Leo Mehr, Zijian Wang, and Christopher D Manning. 2019.  Answering complex open-domain questions through iterative query generation.  _arXiv preprint arXiv:1910.07000_.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  _arXiv preprint arXiv:1908.10084_.
  * Shan et al. (2020) Xuan Shan, Chuanjie Liu, Yiqian Xia, Qi Chen, Yusi Zhang, Angen Luo, and Yuxiang Luo. 2020.  Bison: Bm25-weighted selfattention framework for multi-fields document search.  _arXiv preprint arXiv:2007.05186_.
  * Shen et al. (2014) Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. 2014.  Learning semantic representations using convolutional neural networks for web search.  In _Proceedings of the 23rd international conference on world wide web_ , pages 373–374.
  * Rolínek et al. (2020) Rolínek, M., V. Musil, A. Paulus, M. Vlastelica, C. Michaelis, and G. Martius (2020).  Optimizing rank-based metrics with blackbox differentiation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 7620–7630.
  * Roy and McCallum (2001) Roy, N. and A. McCallum (2001).  Toward optimal active learning through sampling estimation of error reduction. Int. Conf. on Machine Learning.
  * Ru et al. (2020) Ru, D., J. Feng, L. Qiu, H. Zhou, M. Wang, W. Zhang, Y. Yu, and L. Li (2020).  Active sentence learning by adversarial uncertainty sampling in discrete space.  _arXiv preprint arXiv:2004.08046_.
  * Sachan et al. (2021) Sachan, D. S., M. Patwary, M. Shoeybi, N. Kant, W. Ping, W. L. Hamilton, and B. Catanzaro (2021).  End-to-end training of neural retrievers for open-domain question answering.  _arXiv preprint arXiv:2101.00408_.
  * Salant and Berant (2017) Salant, S. and J. Berant (2017).  Contextualized word representations for reading comprehension.  _arXiv preprint arXiv:1712.03609_.
  * Samek et al. (2017) Samek, W., T. Wiegand, and K.-R. Müller (2017).  Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models.  _arXiv preprint arXiv:1708.08296_.
  * Sarker (2021) Sarker, I. H. (2021).  Deep learning: a comprehensive overview on techniques, taxonomy, applications and research directions.  _SN Computer Science_ , 2(6), pp. 1–20.
  * Schröder et al. (2021) Schröder, C., A. Niekler, and M. Potthast (2021).  Uncertainty-based query strategies for active learning with transformers.  _arXiv preprint arXiv:2107.05687_.
  * Schröder et al. (2022) Schröder, C., A. Niekler, and M. Potthast (2022).  Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers.  In _Findings of the Association for Computational Linguistics: ACL 2022_ , pp. 2194–2203.
  * Schubotz et al. (2018) Schubotz, M., P. Scharpf, K. Dudhat, Y. Nagar, F. Hamborg, and B. Gipp (2018).  Introducing mathqa: a math-aware question answering system.  _Information Discovery and Delivery_.
  * Seo et al. (2016) Seo, M., A. Kembhavi, A. Farhadi, and H. Hajishirzi (2016).  Bidirectional attention flow for machine comprehension.  _arXiv preprint arXiv:1611.01603_.
  * Settles (2009) Settles, B. (2009).  Active learning literature survey.
  * Shan et al. (2020) Shan, X., C. Liu, Y. Xia, Q. Chen, Y. Zhang, A. Luo, and Y. Luo (2020).  BISON: BM25-weighted selfattention framework for multi-fields document search.  _arXiv preprint arXiv:2007.05186_.
  * Shashua and Levin (2002) Shashua, A. and A. Levin (2002).  Ranking with large margin principle: Two approaches.  _Advances in neural information processing systems_ , 15.
  * Shekarpour et al. (2013) Shekarpour, S., K. Höffner, J. Lehmann, and S. Auer (2013).  Keyword query expansion on linked data using linguistic and semantic features.  In _2013 IEEE Seventh International Conference on Semantic Computing_ , pp. 191–197. IEEE.
  * Chen et al. (2017) Chen, D., A. Fisch, J. Weston, and A. Bordes (2017).  Reading wikipedia to answer open-domain questions.  _arXiv preprint arXiv:1704.00051_.
  * Lin et al. (2021) Lin, J., R. Nogueira, and A. Yates (2021).  Pretrained transformers for text ranking: Bert and beyond.  _Synthesis Lectures on Human Language Technologies_ , 14(4), pp. 1–325.
  * Shen et al. (2004) Shen, D., J. Zhang, J. Su, G. Zhou, and C. L. Tan (2004).  Multi-criteria-based active learning for named entity recognition.  In _Proceedings of the 42nd annual meeting of the Association for Computational Linguistics (ACL-04)_ , pp. 589–596.
  * Talmor and Berant (2018) Alon Talmor and Jonathan Berant. 2018.  The web as a knowledge-base for answering complex questions.  _arXiv preprint arXiv:1803.06643_.
  * Trotman et al. (2014) Andrew Trotman, Antti Puurula, and Blake Burgess. 2014.  Improvements to bm25 and language models examined.  In _Proceedings of the 2014 Australasian Document Computing Symposium_ , pages 58–65.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  _arXiv preprint arXiv:1804.07461_.
  * Yang et al. (2018) Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018.  Hotpotqa: A dataset for diverse, explainable multi-hop question answering.

##  Appendix A Comparison of Base models

We compare the base models via checking what percentage of questions from
HotpotQA development dataset with at least one evidence are ranked within
top-$k$ by a base model or not. Table 7 also shows that any one of the three
base models cannot capture all the relevant information for answering multi-
hop QA, and they capture complementary relevance signals.

BM25 | MSMARCO | QNLI | % Ques | % Ques
---|---|---|---|---
| CE | CE | (k=3) | (k=5)
✓ | ✗ | ✗ | 14 | 10
✗ | ✗ | ✓ | 25 | 22
✗ | ✓ | ✗ | 20 | 16
✗ | ✓ |  | 33 | 30
✗ |  | ✓ | 38 | 35
| ✓ | ✗ | 64 | 62
| ✗ | ✓ | 35 | 33
✗ | ✗ | ✗ | 44 | 29
Table 3:  Each line shows the percentage of questions that have at least one
evidence ranked within top-$k$ by the model marked with a ‘✓’ but beyond
top-$k$ by model(s) marked with a ‘✗’. For example, there are 14% questions
with at least one evidence sentence is ranked within top-3 by
BM25777https://pypi.org/project/rank-bm25 , but ranked beyond top-3 by MSMARCO
CE and QNLI CE; 35% questions with at least one evidence sentence ranked
within top-3 by QNLI CE but ranked beyond top-3 by MSMARCO CE.

##  Appendix B QER of Similarity Combination

$\scriptscriptstyle\mathcal{QER}_{\scaleto{q,s_{j}}{3.5pt}}=\begin{cases}\dfrac{{\scriptscriptstyle\eta(\mathcal{{\cal
BM\text{25}}}_{\scaleto{q,s_{j}}{3.5pt}})+\alpha\cdot\eta(\mathcal{STS}_{\scaleto{q,s_{j}}{3.5pt}})+\beta\cdot\eta(\mathcal{IS}_{\scaleto{q,s_{j}}{3.5pt}})}}{{\scriptscriptstyle
3}}&\ {\scriptscriptstyle\text{if }\mathcal{{\cal
BM\text{25}}}_{\scaleto{q,s_{j}}{3.5pt}}>0}\\\ \\\
\dfrac{{\scriptscriptstyle\alpha\cdot\eta(\mathcal{STS}_{\scaleto{q,s_{j}}{3.5pt}})+\beta\cdot\eta(\mathcal{IS}_{\scaleto{q,s_{j}}{3.5pt}})}}{{\scriptscriptstyle
2}}&\ {\scriptscriptstyle\text{Otherwise}}\\\ \end{cases}$

where semantic textual similarity ($STS$) and inference similarity ($IS$) are
scores from MSMARCO CE and QNLI CE. It first normalizes the scores with
$\eta$888$\eta$ performs normalization to scale inputs to unit norms with
Scikit-learn’s normalizer:https://scikit-
learn.org/stable/modules/generated/sklearn.preprocessing.Normalizer.html, and
then combines the normalized scores using the weights $\alpha$ and $\beta$.

##  Appendix C Impact of $K$

EAR and EARnest both jointly consider pairs of candidate sentences top ranked
by the base models. The cut-off parameter $K$ is used to partition sentences
considered as top-ranked by individual base model or not. The larger $K$ is,
the more exhaustive combination of candidate sentence pairs would be
considered. However, the number of pairs is quadratic in the number of $K$, so
it becomes much more computational costly when $K$ is too large. Thus, we test
on 600 randomly sampled questions (about 10% of full dataset) to compare the
impact when changing the value of $K$. The resulting retrieval performance is
exact same when changing $K$ from 3 to 5, while the number of pairs compared
increases from 12 to 33.5 on average. This is expected, because we only
consider the top pair to scored against the question, and sentences in the
pairs are often more likely to be ranked closer to the top of lists by base
models respectfully since they contain stronger relevance signals.

##  Appendix D Necessity of Inference model

To further demonstrate the benefits brought by the inference model, we conduct
an ablation experiment by replacing the QNLI CE in EARnest with MSMARCO CE
while keep everything else the same. We also compare the difference on
retrieval performance with the randomly sampled 600 questions. The result is
shown in table 4. Without the QNLI CE capturing the entailment relation to
promote evidences that are can be inferred by the questions to the top, BM25
and MSMARCO CE might miss them according to lexical and semantic matches.
Therefore, the result is significantly lower than the full EARnest model,
which confirms that textual entailment is a very important relevance signal to
the multi-hop QA evidence retrieval task and should be considered along with
the semantic equivalence.

Models |  P@3 |  P@5 |  MAP |  R@3 |  R@5 |  R@10
---|---|---|---|---|---|---
BM25 |  0.44 |  0.32 |  0.6 |  0.55 |  0.66 |  0.79
MSmarco |  0.47 |  0.34 |  0.65 |  0.59 |  0.71 |  0.82
QNLI |  0.34 |  0.24 |  0.45 |  0.43 |  0.52 |  0.64
EARnest |  0.56 |  0.38 |  0.75 |  0.71 |  0.8 |  0.88
EARnest - QNLI |  0.52 |  0.37 |  0.7 |  0.66 |  0.77 |  0.86
Table 4: With the EARnest ensemble model framework, we replace QNLI CE with Ms
Marco CE, and the retrieval performance significantly decreased. Comparing to
the full EARnest model, MAP drops 5% without exploiting the QNLI CE model to
capture the textual entailment relevance signal.