# Sparse and Dense Approaches for the Full-rank Retrieval of Responses for
Dialogues

Gustavo Penha  Delft University of TechnologyDelftNetherlands
<EMAIL_ADDRESS>and  Claudia Hauff  Delft University of
TechnologyDelftNetherlands<EMAIL_ADDRESS>
###### Abstract.

Ranking responses for a given dialogue context is a popular benchmark in which
the setup is to re-rank the ground-truth response over a limited set of $n$
responses, where $n$ is typically 10. The predominance of this setup in
conversation response ranking has lead to a great deal of attention to
building neural _re_ -rankers, while the first-stage retrieval step has been
overlooked. Since the correct answer is always available in the candidate list
of $n$ responses, this artificial evaluation setup assumes that there is a
first-stage retrieval step which is always able to rank the correct response
in its top-$n$ list. In this paper we focus on the more realistic task of
_full-rank retrieval of responses_ , where $n$ can be up to millions of
responses. We investigate both dialogue context and response expansion
techniques for sparse retrieval, as well as zero-shot and fine-tuned dense
retrieval approaches. Our findings—based on three different information-
seeking dialogue datasets—reveal that a learned response expansion technique
is a solid baseline for sparse retrieval. We find the best performing method
overall to be dense retrieval with intermediate training—a step after the
language model pre-training where sentence representations are
learned—followed by fine-tuning on the target conversational data. We also
investigate the intriguing phenomena that harder negatives sampling techniques
lead to worse results for the fine-tuned dense retrieval models. The code and
datasets are available at
https://github.com/Guzpenha/transformer_rankers/tree/full_rank_retrieval_dialogues.

††copyright: none

##  1\. Introduction

_Conversational search_ is concerned with creating agents that satisfy an
information need by means of a _mixed-initiative_ conversation through natural
language interaction, rather than through the traditional search engine
results page. It is an active area of research—as evident for instance in the
SCAI111https://scai.info/ workshop series and two recent and independent
surveys about the topic (Zamani et al., 2022; Gao et al., 2022)—due to the
widespread deployment of voice-based agents, such as Google Assistant and
Microsoft Cortana. A popular approach to conversational search is retrieval-
based (Anand et al., 2020): given an ongoing conversation and a large corpus
of historic conversations, retrieve the response that is best suited from the
corpus (i.e., conversation response ranking (Wu et al., 2017; Yang et al.,
2018; Penha and Hauff, 2019; Yang et al., 2020b; Han et al., 2021)). This
retrieval-based approach does not require task-specific semantics by domain
experts (Henderson et al., 2019), and it avoids the difficult task of dialogue
generation, which often suffers from uninformative, generic responses (Li et
al., 2016a) or responses that are incoherent given the dialogue context (Li et
al., 2016b).

Early neural models for response ranking were based on the interaction of the
concatenated dialogue context and the response in a single-turn manner (Lowe
et al., 2015; Kadlec et al., 2015), with CNN and LSTM architectures.
Researchers later explored using multiple interactions, where each utterance
in the dialogue context is matched with the response, known as multi-turn
matching neural networks (Yan et al., 2016; Wu et al., 2016; Zhou et al.,
2018; Gu et al., 2019; Lin et al., 2020). More recently, due to the
effectiveness of heavily pre-trained transformer-based language models such as
BERT (Devlin et al., 2018) across NLP tasks, they have become the predominant
approach for response ranking (Penha and Hauff, 2019; Xu et al., 2020; Whang
et al., 2021; Zhang and Zhao, 2021b; Su et al., 2020; Gu et al., 2020; Whang
et al., 2019). This is currently the most successful family of methods for
retrieval-based chatbots (Zhang and Zhao,
2021a)222https://github.com/JasonForJoy/Leaderboards-for-Multi-Turn-Response-
Selection.

The offline evaluation of such neural ranking models is to rank the ground-
truth response over a limited set of $n$ responses and measure the number of
relevant responses found in the first $K$ positions— $Recall_{n}@K$ (Zhang and
Zhao, 2021a). Since the entire collection of available responses is typically
way bigger333While for most benchmarks (Zhang and Zhao, 2021a) we have only
10–100 candidate responses, a working system with the Reddit dataset from
PolyAI https://github.com/PolyAI-LDN/conversational-datasets for example would
need to retrieve from 3.7 billion responses. than such set of candidates, this
setup is in fact a re-ranking problem, where we have to select the best
response out of a few options. Additionally, in existing benchmarks the
correct response is traditionally amongst the $n$ responses to re-rank (Penha
and Hauff, 2020). This is thus an artificial evaluation that overlooks the
first-stage retrieval step, which needs to retrieve the $n$ responses that
will be later re-ranked. If the first-stage model, e.g. BM25, fails to
retrieve relevant responses, the _retrieve then re-rank_ pipeline will also
fail even if the re-ranker is an oracle model.

In this paper we offer a novel and comprehensive comparison of supervised and
unsupervised, dense and sparse retrieval models444Although we evaluate them as
standalone methods for the full-rank retrieval problem, they can also be
employed as first-stage retrievers followed by a re-ranking step. for the
overlooked problem of _full-rank retrieval of responses for dialogues_. We
adapt prominent techniques for the problem, i.e. effective in other ranking
tasks such as passage retrieval, including document expansion for the task of
ranking responses for dialogue contexts.

We provide here empirical evidence to the following open questions when
setting up a full-rank retrieval system for conversation response ranking.
What is the effectiveness of sparse and dense retrieval when ranking responses
from the _entire collection_? How do dense models compare with strong sparse
baselines? What is their effectiveness in a zero-shot setup? What is the
effect of adding an intermediate representation learning step between the
language model pre-training and the training with conversational data?

We also shed light on the important problem of selecting negative samples when
training dense retrieval models, which has been shown to have a great effect
on the final effectiveness in different retrieval tasks (Xiong et al., 2020;
Zhan et al., 2021). Unlike previous work that study sampling out of a few
random conversational responses in the re-ranking setup through modifications
to a cross-encoder model (Li et al., 2019), we study the harder problem of
sampling negative responses from the entire collection. We are the first to
investigate different hypotheses in the context of negative sampling of
responses for dialogues that can explain difficulties in using harder
negatives in the training of dense retrievers.

Our main findings in building retrieval models of responses for dialogues in
the full-rank setting are:

  * •

While dialogue context expansion is not successful for sparse retrieval,
supervised response expansion through the proposed $resp2ctxt_{lu}$ is a
strong baseline for full-rank retrieval of responses for dialogues.

  * •

Dense retrieval without access to the target dialogue data, i.e. the zero-shot
scenario, is able to beat a strong sparse baseline only when it has access to
a large amount of out of domain supervision data.

  * •

Dense retrieval models that have intermediate training followed by fine-tuning
with the target data are the best performing models, even with a simple random
sampling approach for obtaining negative responses.

  * •

Harder negative sampling techniques lead to worse effectiveness. We found
evidence for the hypothesis that false positives strongly contribute to this
phenomena. Denoising is an effective approach to take advantage of harder
negative samples.

##  2\. Related Work

In this section we analyze previous work pertinent to this paper by first
discussing current research in the domain of retrieval-based chatbots,
followed by reviewing (un)supervised dense and sparse retrieval approaches. We
conclude this section with an overview of findings on the topic of selecting
negative samples to train neural ranking models.

###  2.1. Ranking Responses for Dialogues

Early neural models for response ranking were based on matching the
representations of the concatenated dialogue context and the representation of
a response in a single-turn manner with architectures such as CNN and LSTM
(Lowe et al., 2015; Kadlec et al., 2015). Researchers later explored matching
each utterance in the dialogue context with the response with more complex
neural architectures (Yan et al., 2016; Wu et al., 2016; Zhou et al., 2018; Gu
et al., 2019; Lin et al., 2020).

Using heavily pre-trained language models for ranking was first shown to be
effective by (Nogueira and Cho, 2019). They used a BERT model to re-rank the
responses of a first stage retrieval system on the MSMarco passage retrieval
task and showed significant improvements in effectiveness. Such language
models for ranking have quickly became a predominant approach in information
retrieval (Lin et al., 2021b). This was also shown to be effective for ranking
responses in conversations (Whang et al., 2019; Penha and Hauff, 2019). Penha
and Hauff (2019) showed a way of using a BERT-based re-ranking model for the
dialogues domain, which is improved when notions of difficulty are taken into
account in a curriculum learning training procedure.

One limitation of transformer-based language models is that they do not take
into account the structure of a dialogue. Gu et al. (2020) proposed adding
another embedding layer to BERT that takes into account the speaker of the
dialogue. Dialogue-aware training has also been further explored, for example
both by Han et al. (2021) and Whang et al. (2021) who proposed different
modifications to the conversational data to improve the fine-tuning of
language models. Building better re-ranking models for dialogue tasks is still
a very active research field as seen by recent surveys on the topic (Tao et
al., 2021b; Zhang and Zhao, 2021a).

In contrast, full-rank retrieval of responses has been under explored (Penha
and Hauff, 2020). Lan et al. (2021) showed that a BERT-based dense retrieval
model outperforms BM25 on the full-rank task. Tao et al. (2021a) later
proposed a mutual learning model that trains both the dense retrieval bi-
encoder model and the cross-encoder re-ranker model at the same time. They
also showed that such dense model is more effective than BM25 without
expansion techniques for the full-rank problem of retrieving responses for
dialogues.

A limitation of previous work is that a strong sparse retrieval baseline
model, e.g. BM25+dialogue context expansion or BM25+response expansion, was
not compared. Such expansion methods are capable of combating the problem of
vocabulary mismatch and thus the question if dense models are able to
outperform sparse ones when using expansion techniques is still unanswered. We
expand on the analysis of previous work (Lan et al., 2021; Tao et al., 2021a)
by looking into stronger sparse baselines, evaluating the effect of
intermediate training, testing zero-shot effectiveness of dense models and
studying the effect of other negative sampling methods besides random
sampling.

###  2.2. Dense and Sparse Retrieval

The proposed conceptual framework by (Lin, 2021) argues for a categorization
of retrieval models into two dimensions: supervised vs. unsupervised and dense
vs. sparse representations555A distinction can also be made of cross-encoders
and bi-encoders, where the first encode the query and document jointly as
opposed to separately (Thakur et al., 2020). Cross-encoders are typically
applied in a re-ranking step due to their inefficiency and thus are not the
focus of this paper.. An _unsupervised_ sparse representation model such as
BM25 (Robertson and Walker, 1994) and TF-IDF (Jones, 1972) represents each
document and query with a sparse vector with the dimension of the collection’s
vocabulary, having many zero weights due to non-occurring terms. Since the
weights of each term are calculated using term statistics they are considered
unsupervised methods.

A _supervised_ sparse retrieval model such as COIL (Gao et al., 2021a), SPLADE
(Formal et al., 2021), TILDE (Zhuang and Zuccon, 2021) and DeepImpact (Mallia
et al., 2021) can take advantage of the effectiveness of transformer-based
language models by changing the terms’ weights from collection statistics to
something that is learned. DeepCT (Dai and Callan, 2019) for example learns
term weights with a transformer-based regression model from the supervision of
the MSMarco dataset. Approaches that only modify non-zero weights however are
not able to address the vocabulary mismatch problem (Furnas et al., 1987), as
non-zero terms will not be affected. One way to address such problem in sparse
retrieval is by using query expansion methods. RM3 (Abdul-Jaleel et al., 2004)
has been shown to be a competitive query expansion technique that uses pseudo-
relevance feedback to add new terms to the queries followed by another final
retrieval step using the modified query.

Document expansion has also been shown to be an effective technique to improve
sparse retrieval, which is able to address the vocabulary mismatch problem.
The core idea is to create pseudo documents that have expanded terms and use
them instead when doing retrieval. Doc2query (Nogueira et al., 2019) is an
effective approach to document expansion that uses a language model to predict
the queries which might be issued to find the document. The predictions of
this model are used to create the augmented pseudo documents. Expansion
techniques are able to modify non-zero weights by adding terms that did not
exist in the query or document.

Supervised dense retrieval models, such as ANCE (Xiong et al., 2020), RocketQA
(Qu et al., 2020), PAIR (Ren et al., 2021) and coCodenser (Gao and Callan,
2021), represent query and documents in a smaller fixed-length space, for
example of 768 dimensions, which can naturally capture semantics. In this
manner they are able to address the vocabulary mismatch problem. While dense
retrieval models have shown to consistently outperform BM25 in multiple
datasets, this is not so easily the case when dense retrieval models do not
have access to training data from the target task, known as the _zero-shot
scenario_. The BEIR benchmark (Thakur et al., 2021) showed that BM25 was
superior to dense retrieval from 9–18 (depending on the model) out of the 18
datasets under this evaluation scheme. While the zero-shot scenario offers a
fairer comparison of dense models with unsupervised sparse models, learned
dense retrieval models should also be compared with learned sparse models,
e.g. BM25+doc2query.

Unlike previous work that compares supervised and unsupervised, dense and
sparse retrieval models for other tasks such as passage ranking, we provide a
novel and comprehensive comparison for the problem of _full-rank retrieval of
responses for dialogues_.

###  2.3. Negative Sampling for Ranking

Neural ranking models are known to require a reasonable amount of labeled
queries and documents for training. The TREC-DL dataset (Craswell et al.,
2020)—created from MSMarco (Nguyen et al., 2016)—offers a large training set,
367k queries, with _no negative labels_ and often only one positive label per
query. This is also the case for conversation response ranking datasets, such
as MANtIS (Penha et al., 2019) and UDCDSTC8 (Kummerfeld et al., 2019), where
we have a large number of training data consisting of relevant responses for
dialogue contexts and no explicitly negative labelled responses.

In order to train neural ranking models, negative candidates are also
necessary since it is prohibitively expensive to use every other document in
the collection as negative sample for a query, and manual relevance judgements
are expensive to obtain for a large scale dataset. This motivates
automatically finding non-relevant documents for a combination of query and
relevant documents, known as _negative sampling_.

This problem is also present for other machine learning techniques that do
self-supervision through contrastive learning in different domains such as
computer vision, natural language and graphs (Robinson et al., 2020; Yang et
al., 2020a; Jiang et al., 2021). For example the well-known word2vec (Mikolov
et al., 2013) word embedding technique randomly samples words that are not
relevant for the context (other words in the sentence) to distinguish from the
actual word that is part of the context.

In information retrieval, since most of the documents in a collection are not
relevant for a given query, a simple approach is to obtain negative candidates
through randomly selecting documents (the same way word2vec selects random
words), with the exception of the documents that were labeled relevant. A
popular technique is to use in-batch negative samples, which are in essence
random and make the training procedure efficient (Humeau et al., 2019; Mazaré
et al., 2018; Lan et al., 2021). However, a limitation of random samples is
that the documents might be too easy for the ranking model to discriminate
from relevant ones, while for negative documents that are harder to
distinguish the model might still struggle.

For this reason, another popular approach has been to use a ranking model to
retrieve negative documents using the given query with a classical retrieval
technique such as BM25. This leads to finding negative documents that are
closer to the query in the sparse representation space, and thus _harder_
negatives.

Since dense retrieval models have been outperforming sparse retrieval in a
number of cases with available training data, more complex negative sampling
techniques taking advantage of dense retrieval models’ effectiveness have been
proposed. The ANCE (Xiong et al., 2020) model uses the dense model itself to
find negatives (ANCE negatives), which is asynchronously updated in
checkpoints. This effectively makes the model find harder and harder negatives
throughout training. Gao et al. (2021b) proposed to localize the training
procedure with the same negative samples from the target distribution of the
target set used for evaluation. Hofstätter et al. (2021) trained dense models
using balanced batches, where an equal number of easy and difficult
negative—according to the distillation margins—documents can be found.

Negative sampling has also been given attention in the training of cross-
encoder re-ranking for conversation response ranking. Li et al. (2019) showed
that if different selection methods to obtain negatives from a pool of
randomly sampled responses are employed, re-rankers can be trained more
effectively. Note that this is quite different to negative sampling from the
entire collection to train bi-encoders which is what we are focused on here.
Qiu et al. (2021) showed that with negatives generated with a pre-trained
language model such as DialogGPT (Zhang et al., 2019), which can also be seen
as a data augmentation technique, one can obtain effectiveness improvements
for re-ranking responses to dialogues. The combination of augmented negatives
through different negative sampling techniques has also been shown to be
effective for re-ranking responses when using a multi-level ranking loss (Lin
et al., 2020).

Building on these prior works, we are the first to investigate different
hypotheses in the context of negative sampling of responses for dialogues that
might explain difficulties in using harder negatives in the training of dense
retrievers for the full-rank task.

##  3\. Full-rank Retrieval for Dialogues

Figure 1. Pipelines for sparse and dense retrieval for the task of full-rank
retrieval of responses for dialogues.

In this section we first describe the problem of full-rank retrieval of
responses, followed by the proposed sparse and then dense approaches for the
problem. A summary of both pipelines can be seen in Figure 1.

###  3.1. Problem Definition

The task of full-rank retrieval of responses for dialogues, concerns
retrieving the best response out of the entire collection given the dialogue
context. Formally, let
$\mathcal{D}=\\{(\mathcal{U}_{i},\mathcal{R}_{i},\mathcal{Y}_{i})\\}_{i=1}^{M}$
be a data set consisting of $M$ triplets: dialogue context, response
candidates and response relevance labels. The dialogue context
$\mathcal{U}_{i}$ is composed of the previous utterances
$\\{u^{1},u^{2},...,u^{\tau}\\}$ at the turn $\tau$ of the dialogue. The
candidate responses $\mathcal{R}_{i}=\\{r^{1},r^{2},...,r^{n}\\}$ are either
ground-truth responses $r^{+}$ or negative sampled candidates $r^{-}$,
indicated by the relevance labels
$\mathcal{Y}_{i}=\\{y^{1},y^{2},...,y^{n}\\}$.

In previous work, the number of candidates is limited, typically $n=10$. Since
we are concerned with the full-rank task and not the re-ranking setting, in
all of our experiments we have $n$ as the size of the responses available in
the collection. By design the number of ground-truth responses is usually one,
the observed response in the conversational data666One limitation of current
benchmarks is that it is likely to exist many semantically equivalent
responses as well as different relevant responses to the same dialogue
context, while the ground-truth ones are considered to be single responses
given by other users..

The task is then to learn a ranking function $f(.)$ that is able to generate a
ranked list for the set of candidate responses $\mathcal{R}_{i}$ based on
their predicted relevance scores $f(\mathcal{U},r)$.

To evaluate the effectiveness of the retrieval systems we use $R@K$, and thus
we evaluate the models effectiveness in finding the correct response out of
the whole possible set of responses at position $K$.

###  3.2. Sparse Retrieval

In order to do sparse retrieval of responses we rely on classical retrieval
methods with query and document expansion techniques. One of the limitations
of sparse retrieval is that, since it represents each dialogue context and
response using the existing terms in a bag-of-words manner, the vocabulary
mismatch problem might occur. Such expansion techniques are able to overcome
this problem if they append new words to the dialogue contexts and responses.

For this reason we propose here to do _dialogue context expansion_ with RM3
(Abdul-Jaleel et al., 2004), which is a very competitive unsupervised method
that assumes that the top-ranked responses by the sparse retrieval model are
relevant. From such pseudo-relevant responses words are selected and an
expanded dialogue context is generated, which will then be used by the sparse
retrieval method to generate the final ranked list.

In order to expand the responses to be retrieved, we propose here $resp2ctxt$.
This is an adaptation of the popular and effective doc2query (Nogueira et al.,
2019) approach for dialogues. Formally, we fine-tune a generative transformer
model for the task of generating the dialogue context $\mathcal{U}_{i}$ from
the ground-truth response $r^{+}$. This model is then used to generate
expansions for all responses in the collection. They are appended to the
responses and the sparse retrieval method itself is not modified.

This way, $resp2ctxt$ allows for two things: term re-weighting (adding terms
that already exist in the document) and addition of new terms (to deal with
the vocabulary mismatch problem).

Unlike most adhoc retrieval problems where the queries are smaller than the
documents, full-rank retrieval of responses for dialogues is the exact
opposite. For example, while the TREC-DL 2020 passage and document retrieval
tasks the queries have between 5–6 terms on average and the passages and
documents have over 50 and 1000 terms respectively, the dialogue contexts
(queries) have between 70 and 474 terms on average depending on the dataset
while the responses (documents) have between 11 and 71 terms on average, as
seen in the first two rows of Table 2. This is a challenge for the generative
model, since generating larger pieces of text is a more difficult problem than
smaller ones, e.g. more room for error.

Motivated by this, we also explored an adaptation of $resp2ctxt$ that aims to
generate only the last utterance of the dialogue context: $resp2ctxt_{lu}$.
This model is trained to generate $u^{\tau}$ from $r+$. The underlying premise
is that the part that needs to be answered by the dialogue context is the last
utterance, and if this is correctly generated by $resp2ctxt_{lu}$, the sparse
retrieval method will be able to find the correct response from the
collection.

###  3.3. Dense Retrieval

In order to do dense retrieval of responses we rely on methods that learn to
represent the dialogue context and the responses separately in a dense
embedding space. This is later used to rank responses by their similarity to
the dialogue context. We rely here on heavily pre-trained language transformer
models, such as BERT (Devlin et al., 2018), RoBERTa (Liu et al., 2019) or
MPNet (Song et al., 2020), to obtain such representations of the dialogue
context and response. This approach is generally referred as a _bi-encoder_
model (Lin et al., 2021b).

###  3.4. Intermediate Training

The first step of the pipeline is to train the representations of the language
model with intermediate777We differentiate this intermediate step to a pre-
training step due to the fact that the transformer based language models were
already pre-trained for their respective language modelling tasks. For
example, BERT is pre-trained for next sentence prediction and masked language
modeling and can be later trained to represent queries and documents. data
that does not contain the target domain data. Such intermediate data contains
triplets of query, relevant document and negative document and can include
multiple datasets. The main advantage of adding this step before fine-tuning
the bi-encoder for the target conversational data is to reduce the gap between
the pre-training, often including language modelling, and downstream task at
hand.

The intermediate training step learns to represent texts (query and documents)
by doing a mean pooling function over the transformer final layer, which is
then used to calculate the dot-product similarity. The relevant document
representation is used to contrast with the representations of the document
that is not relevant. Such procedure learns better text representations than a
naive approach of simply using the $[CLS]$ token representation of BERT for
the dialogue contexts and responses (Reimers and Gurevych, 2019; Aghajanyan et
al., 2021).

The loss function employs multiple negative texts to learn the representations
in a constrastive manner, also known as in-batch negative sampling. This model
is then able to do zero-shot retrieval for the full-rank retrieval of
responses to dialogue contexts, since it does not have access to the target
domain data.

The function $f(\mathcal{U},r)$ can be defined as
$dot(\eta(concat(\mathcal{U})),\eta(r))$, where $\eta$ is the representation
obtained with the mean pooling of all the output vectors of the transformers
language model, and
$concat(\mathcal{U})=u^{1}\;|\;[U]\;|\;u^{2}\;|\;[T]\;|\;...\;|\;u^{\tau}\;$,
where $|$ indicates the concatenation operation. The utterances from the
context $\mathcal{U}$ are concatenated with special separator tokens $[U]$ and
$[T]$ indicating end of utterances and turns888The special tokens $[U]$ and
$[T]$ will not have any meaningful representation in the zero-shot setting,
but they can be learned on the fine-tuning step..

###  3.5. Fine-tuning

The second step in the pipeline is to fine-tune the model with data from the
target domain, in our case dialogue contexts and responses. Since we do not
have labeled negative responses, and only relevant ones, the remaining
responses can be thought of as non relevant for the dialogue context.
Computing the probability of the correct response over all other responses in
the dataset would give us
$P(r\mid\mathcal{U})=\frac{P(\mathcal{U},r)}{\sum_{k}P\left(\mathcal{U},r_{k}\right)}$.
Since this computation is prohibitively expensive to calculate, we approximate
it using only a few negative samples retrieved by a negative sampling
approach.

The _negative sampling_ task is then to: given the dialogue context
$\mathcal{U}$ find challenging responses that are not relevant. This can be
seen as a retrieval task as well, where one can use a retrieval model to find
negatives by applying $f(\mathcal{U},r)$ for every $r$ in the collection,
sorting, and removing $r^{+}$ from the resulting top negatives.

In our experiments we test a number of different approaches to obtain hard
negative samples: random sampling, BM25 sampling, bi-encoder sampling and
generative transformer models999The generative approach goes directly from the
dialogue context to the negative candidate: $\mathcal{U}\xrightarrow{}r^{-}$..

With such dataset at hand, we continue the training—after the intermediate
step—in the same manner as done by the intermediate training step, with the
following cross-entropy loss function101010We refer to this loss as
MultipleNegativesRankingLoss. for a batch with size $B$:

$\begin{aligned}
\mathcal{J}(\mathbf{\mathcal{U}},\mathbf{r},\theta)&=-\frac{1}{B}\sum_{i=1}^{B}\left[f\left(\mathcal{U}_{i},r_{i}\right)-\log\sum_{j=1,j!=i}^{B}e^{f\left(\mathcal{U}_{i},r_{j}\right)}\right]\end{aligned},$

where $f(\mathcal{U},r)$ is the dot-product of the mean pooled representation
of the transformer model.

##  4\. Experimental Setup

In order to compare the different sparse and dense approaches we consider
three large-scale information-seeking conversation datasets111111MSDialog is
available at https://ciir.cs.umass.edu/downloads/msdialog/; MANtIS is
available at https://guzpenha.github.io/MANtIS/; UDCDSTC8 is available at
https://github.com/dstc8-track2/NOESIS-II. that allow the training of neural
ranking models for conversation response ranking: MSDialog (Qu et al., 2018)
contains 246K context-response pairs, built from 35.5K information seeking
conversations from the Microsoft Answer community, a QA forum for several
Microsoft products; MANtIS (Penha et al., 2019) contains 1.3 million context-
response pairs built from conversations of 14 Stack Exchange sites, such as
askubuntu and travel; UDCDSTC8 (Kummerfeld et al., 2019) contains 184k
context-response pairs of disentangled Ubuntu IRC dialogues.

###  4.1. Implementation Details

For BM25 and BM25+RM3 we rely on the pyserini implementations (Lin et al.,
2021a). In order to train $resp2ctxt$ expansion methods we rely on the
Huggingface transformers library (Wolf et al., 2019), using the t5-base model.
We fine-tune the T5 model for 2 epochs, with learning rate of 2e-5, weight
decay of 0.01 and batch size of 5. When augmenting the responses with
$resp2ctxt$ we follow docT5query (Nogueira et al., 2019) and append three
different context predictions, using sampling and keeping the top-10 highest
probability vocabulary tokens.

For the zero-shot dense retrieval models, we rely on the SentenceTransformers
(Reimers and Gurevych, 2019) model releases121212The pre-trained models can be
found here https://www.sbert.net/docs/pretrained_models.html. The library uses
Hugginface transformers for the pre-trained models such as BERT (Devlin et
al., 2018), RoBERTa (Liu et al., 2019), MPNet (Song et al., 2020). When fine-
tuning the dense retrieval models, we rely on the
MultipleNegativesRankingLoss, which accepts a number of hard negatives, and
also uses the remaining in-batch random negatives to train the model. We use a
total of 10 negative samples for dialogue context.

We fine-tune the dense retrieval models for a total of 10k steps, and every
100 steps we evaluate the models on a re-ranking task that selects the
relevant response out of 10 responses. We use the re-ranking validation MAP to
select the best model from the whole training to use in evaluation. We use a
batch size of 5, with 10% of the training steps as warmup-steps. The learning
rate is set to 2e-5 and weight decay of 0.01. We use the dot-product of the
mean pooled representations for both training and also for test (using FAISS
(Johnson et al., 2019)).

On the follow-up experiments to investigate negative sampling approaches, we
denoise negatives on E2 using lists of 100 responses and keeping the bottom 10
as negatives. We expand the collection with an external corpus for E5 using
ConvoKit (Chang et al., 2020). We choose datasets which have similar topics to
the information-seeking datasets we use131313They are namely movie-corpus,
wiki-corpus, subreddit-Ubuntu, subreddit-microsoft, subreddit-apple,
subreddit-Database, subreddit-DIY, subreddit-electronics, subreddit-ENGLISH,
subreddit-gis, subreddit-Physics, subreddit-scifi, subreddit-statistics,
subreddit-travel and subreddit-worldbuilding, amounting to a total of 17M non-
empty candidate responses. For experiment E6 we generate the negative
candidates using Hugginface (Wolf et al., 2019) conversational pipelines, with
the pre-trained models DialoGPT-large and blenderbot-400M-distill.

###  4.2. Evaluation

To evaluate the effectiveness of the retrieval systems, instead of resorting
to the standard evaluation metric in conversation response ranking (Yuan et
al., 2019; Gu et al., 2020; Tao et al., 2019) which is recall at position $K$
with $n$ candidates141414For example $R_{10}@1$ indicates the number of
relevant responses found at the first position when the model has to rank 10
candidate responses. $R_{n}@K$, we set $n$ to be the entire collection of
answers, and thus we evaluate the models effectiveness in finding the correct
response out of the whole possible set of responses: $R@K$. We perform
Students t-tests at confidence level of 0.95 with Bonferroni correction to
compare statistical significance of methods across different dialogue
contexts.

Table 1. Effectiveness of sparse and dense retrieval methods for the full-rank retrieval of responses for dialogues. Bold values indicate the highest recall values for each group (type of approach). Superscripts indicate statistically significant improvements using Students t-test with Bonferroni correction. $\dagger$=other methods from the same group; $1$=best from unsupervised sparse retrieval ; $2$=best from supervised sparse retrieval; $3$=best from zero-shot dense retrieval. We only do comparisons between the best approaches of each category, and within categories. |  |  | MANtIS | MSDialog | UDCDSTC8
---|---|---|---|---|---
|  |  | R@1 | R@10 | R@1 | R@10 | R@1 | R@10
(0) | Random | 0.000 | 0.000 | 0.000 | 0.001 | 0.000 | 0.001
| Unsupervised sparse retrieval |  |  |  |  |  |
(1a) | BM25 | 0.133† | 0.299† | 0.064† | 0.177† | 0.027† | 0.070†
(1b) | BM25 + RM3 | 0.073 | 0.206 | 0.035 | 0.127 | 0.011 | 0.049
| Supervised sparse retrieval |  |  |  |  |  |
(2a) | BM25 + $resp2ctxt$ | 0.135 | 0.309 | 0.074 | 0.208 | 0.028 | 0.067
(2b) | BM25 + $resp2ctxt_{lu}$ | 0.147†1 | 0.325†1 | 0.0751 | 0.2021 | 0.029 | 0.076
| Zero-shot dense retrieval |  |  |  |  |  |
| ModelLanguageModel | Intermediate data |  |  |  |  |  |
(3a) | ANCEroberta-base | 600K MSMarco-PR | 0.048 | 0.111 | 0.050 | 0.124 | 0.010 | 0.028
(3b) | TAS-Bdistillbert-base | 400K MSMarco-PR | 0.062 | 0.143 | 0.060 | 0.157 | 0.019 | 0.050
(3c) | Bi-encoderbert-base | 500K MSMarco-QA | 0.038 | 0.098 | 0.043 | 0.113 | 0.014 | 0.040
(3d) | Bi-encodermpnet-base | 215M mul. sources | 0.138 | 0.297 | 0.108 | 0.277 | 0.023 | 0.076
(3e) | Bi-encodermpnet-base | 1.17B mul. sources | 0.155†1 | 0.341†12 | 0.147†12 | 0.339†12 | 0.041† | 0.097†12
| Fine-tuned dense retrieval |  |  |  |  |  |
| ModelLanguageModel | Negative sampler |  |  |  |  |  |
(4a) | Bi-encodermpnet-base (3e) | Random (0) | 0.130† | 0.307† | 0.168†123 | 0.387†123 | 0.050†12 | 0.128†123
(4b) | BM25 (1a) | 0.112 | 0.271 | 0.128 | 0.316 | 0.027 | 0.087
(4c) | Bi-encoder (3e) | 0.065 | 0.146 | 0.144 | 0.306 | 0.018 | 0.051

##  5\. Results

In this section we first report on both dense and sparse retrieval results.
Then we analyze the negative sampling procedure used to train the dense
retrieval models.

###  5.1. Sparse Retrieval

In order to compare supervised and unsupervised sparse retrieval methods as
well as zero-shot and fine-tuned dense retrieval models, we divided then into
four categories as shown in Table 1. Each row is a retrieval approach,
containing the effectiveness in terms of R@1 and R@10 for each of the three
datasets.

#### Does dialogue context expansion via RM3 lead to improvements over no
expansion for sparse retrieval?

BM25+RM3 (row 1b) does not improve over BM25 (1a) on any of the three
conversational datasets analyzed. A thorough hyperparameter fine-tuning was
performed and no combination of the RM3 hyperparameters outperformed
BM25151515Review appendix A for all hyperparameter combinations tested..

A manual analysis of the new terms appended to a sample of 60 dialogue
contexts reveals that only 18% of them have at least one relevant term added
based on our best judgement. Unlike web search where the query is often
incomplete, under-specified, and ambiguous, in the information-seeking
datasets employed in our work we observe that the dialogue context (query) is
often times quite detailed and has more terms than the responses (documents).
We hypothesize that because the dialogue contexts are already quite
descriptive the task of expansion is trickier in this domain and thus we
observe many dialogues for which the terms added are just noise.

#### Does response expansion, i.e. $resp2ctxt$, lead to improvements over no
expansion for sparse retrieval?

We find that response expansion161616See augmentation examples on Appendix B.
helped in two of the three datasets tested. BM25+$resp2ctxt$ (2a) outperforms
BM25 (1a) in two of the three datasets. Predicting only the last utterance of
the dialogue ($resp2ctxt_{lu}$) performs better than predicting the whole
utterance, as shown by BM25+$resp2ctxt_{lu}$’s (2b) higher recall values. For
example in the MANtIS dataset the R@10 goes from 0.309, when using the model
trained to predict the dialogue context, to 0.325 when using the one trained
to predict only the last utterance of the dialogue context.

In order to understand what the response expansion methods are doing most—term
re-weighting or adding novel terms—we present the percentage of novel terms
added by both methods in Table 2. The table shows that $resp2ctxt_{lu}$ does
more term re-weighting than adding new words when compared to $resp2ctxt$ (53%
and 70% on average are new words respectively and thus 47% vs 30% are changing
the weights by adding existing words), generating overall smaller
augmentations (115.45 vs 431.17 on average respectively).

In terms of sparse retrieval, the experiments so far reveal that using a
response augmentation technique is a far better baseline than using BM25,
which has been claimed to be a strong and proper baseline for comparison with
dense models in conversational benchmarks (Tao et al., 2021a; Lan et al.,
2021).

Table 2. Statistics of the augmentations for the response (document) expansion methods $resp2ctxt$ and $resp2ctxt_{lu}$. | MANtIS | MSDialog | UDCDSTC8
---|---|---|---
Context avg length | 474.12 | 426.08 | 76.95
Response avg length | 42.58 | 71.38 | 11.06
Aug. avg length - $resp2ctxt$ | 494.23 | 596.99 | 202.3
Aug. avg length - $resp2ctxt_{lu}$ | 138.5 | 135.29 | 72.57
% new words - $resp2ctxt$ | 71% | 69% | 71%
% new words - $resp2ctxt_{lu}$ | 59% | 37% | 63%

###  5.2. Dense Retrieval

#### Can zero-shot dense retrieval outperform a strong sparse baseline?

Zero-shot dense retrieval, i.e. no access to target data, beats the strong
sparse baseline BM25+$resp2ctxt$ (2b) _only when it is fine-tuned on large
datasets containing diverse data including dialogues_ , as we see by comparing
rows (3a–c) and (3e–d) with row (2b) in Table 1. For example, while the zero-
shot dense retrieval models based only on the MSMarco dataset (3a–c) perform
on average 35% worse than the strong sparse baseline (2b) in terms of R@10 for
the MSDialog dataset, the zero-shot model trained with 1.17B instances on
diverse data (3e) is 68% better than the strong sparse baseline (2b).

If the intermediate training data for the dense retrieval model is MSMarco
(3a–c), none of the zero-shot retrieval models is able to beat the strong
sparse baseline (2b).

When using however a bigger amount of intermediate training data, which
includes Reddit and Stack Exchange responses171717For the full description of
the intermediate data see https://huggingface.co/sentence-transformers/all-
mpnet-base-v2., we see that the zero-shot dense retrieval model (3e) is able
to outperform the sparse retrieval baseline by margins of 33% of R@10 on
average across the datasets.

As expected, the closer the intermediate training data distribution is to the
target domain, the better the dense retrieval model performs. The results
indicate that a good zero-shot retrieval model needs to be trained for
representation learning on a large set of datasets to outperform strong sparse
retrieval baselines. Our results match previous empirical evidence on the
effect of the intermediate training step on dense retrieval for different
retrieval tasks (Oğuz et al., 2021).

#### Is intermediate training of dense retrieval models helpful or is it
sufficient to fine-tune a dense model on the target data?

Intermediate training on a large set of training instances is quite important
for learning dense representations. Table 3 compares the dense models using
either different pre-trained language models with and without using the
intermediate data, with a different number of negative sampling procedures.

We see that if we fine-tune mpnet-base directly on the target data, and do not
do any intermediate training step the effectiveness drops are significant and
substantial as shown when comparing results of 1.17B mul.sources (rows 1—3) vs
no intermediate data (rows 4–6) in Table 3. For example in the MANtIS dataset
the R@10 goes from 0.307 to 0.172 when using random negative sampling. This
also happens for other language models and intermediate datasets, e.g. for
bert-base and MSMarco the R@10 goes from 0.205 to 0.092 the MANtIS dataset.

#### What is the effect of fine-tuning the dense retrieval model after the
intermediate training?

First, we see that simply using random sampling to find negatives and then
fine-tune the dense retrieval model that had already gone through intermediate
training— row (4a) in Table 1—achieves the best overall effectiveness we
obtain in two of the three datasets. Having access to the target
conversational data as opposed to only a diverse set of question and answers
means that the representations learned by the model are closer to the true
distribution of the data.

We hypothesize that fine-tuning Bi-encodermpnet-base (3e) for MANtIS (4a) is
harmful because the intermediate data contains multiple Stack Exchange
responses. In this way, the subset of dialogues of Stack Exchange that MANtIS
encompasses might be serving only to overfit the intermediate representations.
As evidence for this hypothesis, we found that (I) the learning curves flatten
quickly (as opposed to other datasets) and (II) fine-tuning another language
model that does not have Stack Exchange data (MSMarco) in their fine-tuning,
Bi-encoderbert-base (3c), improves the effectiveness with statistical
significance from 0.092 R@10 to 0.205 R@10, as shown in Table 3.

#### Do harder negative samples lead to more effective fine-tuning of dense
retrieval models?

Surprisingly we found that using more effective models to select negative
candidates is detrimental to the effectiveness of the dense retrieval model
(rows 4a–c). We observe this phenomena when using different different language
models and whether using intermediate training or not for all datasets tested,
as shown in Table 3.

Figure 2 shows validation curves during training for the
MultipleNegativesRankingLoss and an alternative contrastive loss (Hadsell et
al., 2006) (ContrastiveLoss) that does employ in-batch negative sampling. The
experiment tries to isolate the effect the specific loss function might be
having on the negative sampling phenomena, and we observe that the same
behaviour regardless of the loss function181818Other loss functions besides
the two plotted were also tested and resulted in the the same effectiveness
for the negative samplers: $Random>>BM25{}>>Bi-encoder$..

Based on brainstorm sessions and discussions the authors had with other IR
researchers a set of hypotheses were formed that could explain why this
phenomena might be happening. Next we explore the three resulting hypotheses
with six additional experiments.

Table 3. Effectiveness of fine-tuned dense retrieval models when using different language models and intermediate training for each negative sampling procedures from Table 1. Bold indicates the highest value within different negative sampling methods for the same setting. We observe the same phenomena of decreasing effectiveness for better negative sampling methods when using different language models and whether using intermediate training or not. |  |  | MANtIS | MSDialog | UDCDSTC8
---|---|---|---|---|---
Model | Intermediate data | Neg. Sampler | R@1 | R@10 | R@1 | R@10 | R@1 | R@10
Bi-encodermpnet-base | 1.17B mul.sources | Random (0) | 0.130 | 0.307 | 0.168 | 0.387 | 0.050 | 0.128
BM25 (1a) | 0.112 | 0.271 | 0.128 | 0.316 | 0.027 | 0.087
Bi-encoder (3e) | 0.065 | 0.146 | 0.144 | 0.306 | 0.018 | 0.051
- | Random (0) | 0.070 | 0.172 | 0.114 | 0.308 | 0.021 | 0.063
BM25 (1a) | 0.043 | 0.118 | 0.091 | 0.256 | 0.009 | 0.027
Bi-encoder (3e) | 0.032 | 0.087 | 0.083 | 0.205 | 0.002 | 0.019
Bi-encoderbert-base | 500K MSMarco-QA | Random (0) | 0.085 | 0.205 | 0.138 | 0.339 | 0.030 | 0.079
BM25 (1a) | 0.051 | 0.130 | 0.116 | 0.287 | 0.007 | 0.022
Bi-encoder (3e) | 0.043 | 0.106 | 0.107 | 0.242 | 0.008 | 0.030
- | Random (0) | 0.029 | 0.092 | 0.063 | 0.200 | 0.012 | 0.038
BM25 (1a) | 0.017 | 0.057 | 0.040 | 0.144 | 0.002 | 0.006
Bi-encoder (3e) | 0.011 | 0.041 | 0.034 | 0.119 | 0.000 | 0.009
Figure 2. Validation effectiveness training curves for the re-ranking task (10
candidates obtained using random sampling). The plot displays the dense
retrieval Bi-encodermpnet-base model when using different loss functions for
each negative sampling procedures (the right column is the training curves for
rows 4a–c on Table 1). We observe the phenomena of decreasing effectiveness
for better negative sampling methods when using different loss functions.

###  5.3. Dense Retrieval: Negative Sampling

We investigated the following hypotheses that could explain the observed
phenomena of decreasing effectiveness for better negative sampling functions:

  * H1

False negative samples increase when using better negative sampling methods.
False negative are responses that are potentially valid for the context. Such
relevant responses sampled will lead to unlearning relevant matches between
context and responses as they receive negative labels. Example retrieved by
the Bi-encoder model (line 3e of Table 1):

Dialogue context ($\mathcal{U}$):  hey… how long until dapper comes out? $[U]$
14 days […] $[U]$ i thought it was coming out tonight Correct response
($r^{+}$): just kidding couple hours Negative sample ($r^{-}$): there is a
possibility dapper will be delayed […] meanwhile, dapper discussions should
occur in ubuntu+1

  * H2

Confusing negative samples increase when using better negative sampling
methods. They are not relevant, i.e. a valid response to the context, but they
are semantically or lexically identical to (or exact matches or part of) the
context. Such negative samples will lead to representations of similar
sentences to be far apart in the embedding space. Example of a partial match
retrieved by BM25 (line 1a of Table 1):

Dialogue context ($\mathcal{U}$): can any one help me im trying to install
some thing and i get this error GTK… configure: error: Package requirements
(gtk+-2.0 $[U]$ perhaps sudo apt-get install libgtk2.0-dev $[U]$ any way to
tell it to install all dependencies too Correct response ($r^{+}$): what do
you mean, it won’t compile if you don’t have the dependencies Negative sample
($r^{-}$): sudo apt-get install libgtk2.0-dev

  * H3

There is a lack of informative negative samples, i.e. responses that are more
informative than random negative responses for training, for the dialogue
contexts within each dataset. Informative negative samples are ideally the
ones that (I) have lexical matches with the dialogue context but are not
subsets of continuous parts of the context and are not semantically relevant
or (II) are potential responses to the dialogue (gives the impression that it
is a natural and fluent response to the last utterance of the dialogue
context) but are not semantically relevant.

Unlike a web collection where we can likely find the same term used in
different contexts, the dialogue collections used in this work have only a few
domains, and a single term is less likely then to appear within different
contexts. Moreover, different to common adhoc retrieval tasks such as web
search and passage ranking, in response ranking the response (negative sample)
has to be a likely continuation of the last utterance of the dialogue context.
Examples of potentially informative negative samples:

Dialogue context ($\mathcal{U}$): I had my iPhone swapped out by Apple and
after reinstalling my apps, signing in, etc, I noticed my OneDrive app was
saying ””Be sure you’re connected to cellular or wifi””…. and it is. I’ve
signed out and back in… removed and re added the app… etc no dice. Anyone have
any suggestions? Correct response ($r^{+}$): Hi PERSON_PLACEHOLDER, I realized
the inconvenience you are experiencing. I certainly help you. Is the issue
specific to OneDrive app or with other apps as well? First, update iOS on your
device. Then, make sure you’ve installed any available updates to the app. […]
Different collection negative sample ($r^{-}$)191919Response selected from
reddit/r/onedrive dialogues.: I love OneDrive, have used it for years with no
issues. I believe a lot of people have issues because they don’t understand
how it works, they don’t read the instructions and when they install it they
just click ’yes’ to everything. Generated negative sample
($r^{-}$)202020Response generated by DialoGPT-large for the dialogue context.:
I had the same problem. I had to uninstall and reinstall the app.

In order to test our hypotheses we perform the following experiments, each one
geared towards investigating one hypothesis:

  * E1

Manually annotate a subset of negative samples in terms of their relevance, to
check whether the number of false negatives increases with better negative
sampling functions (H1).

  * E2

Instead of using the top ranked responses as negative responses, we use the
bottom responses of the top ranked responses as negatives. As an example, when
we retrieve $k=100$ responses, instead of using responses ranked 1 to 10 we
use responses ranked 91 to 100. This decreases the chances of obtaining false
positives and if we set $k$ to small values such as 100, it will not render
the sampling procedure to be essentially random (H1).

  * E3

Remove negative samples that are subsets of the context when training dense
models and compare its effectiveness with the original negative samples (H2).

  * E4

Use only the last utterance to retrieve negative samples, this will make it
less likely that a response is an exact match with the entire dialogue context
(H2).

  * E5

Compare the effectiveness of dense retrieval models when using a corpus of
responses for negative sampling which has additional responses from external
corpora, that are potentially more informative than the ones from the original
dataset (H3).

  * E6

Generate negative samples for the contexts using a generative language model
and compare the effectiveness of this model against using retrieved negative
samples (H3).

Table 4. Experiments to examine why better negatives sampling procedures lead
to worse dense retrieval results. Bold indicates positive evidence for the
corresponding hypothesis. We present the R@10 gains between the condition
presented and the absence of the condition (4a–c on Table 1) on E2–E5. E1
---
NS | Count | MANtIS | MSDialog | UDCDSTC8
Random (0) | false $r^{-}$ | 0 | 0 | 0
BM25 (1a) | 0 | 4 | 2
Bi-encoder (3e) |  | 11 | 4 | 15
E2
|  | MANtIS | MSDialog | UDCDSTC8
NS | Condition |  R@10$\Delta$ |  R@10$\Delta$ |  R@10$\Delta$
BM25 (1a) | denoising | -0.014 | +0.042 | +0.034
Bi-encoder (3e) | +0.170 | +0.091 | +0.056
E3
|  | MANtIS | MSDialog | UDCDSTC8
NS | Condition |  R@10$\Delta$ |  R@10$\Delta$ |  R@10$\Delta$
BM25 (1a) | $r^{-}$ subset of $\mathcal{U}$ | -0.032 | +0.015 | -0.010
Bi-encoder (3e) | +0.034 | -0.040 | -0.003
E4
|  | MANtIS | MSDialog | UDCDSTC8
NS | Condition |  R@10$\Delta$ |  R@10$\Delta$ |  R@10$\Delta$
BM25 (1a) | Ulu as query | -0.002 | +0.044 | -0.008
Bi-encoder (3e) | +0.173 | +0.042 | +0.047
E5
|  | MANtIS | MSDialog | UDCDSTC8
NS | Corpus |  R@10$\Delta$ |  R@10$\Delta$ |  R@10$\Delta$
BM25 (1a) | expanded | -0.014 | +0.031 | +0.023
Bi-encoder (3e) | +0.113 | +0.028 | +0.057
E6
|  | MANtIS | MSDialog | UDCDSTC8
NS | R@10 | R@10 | R@10
Random (0) | 0.307 | 0.387 | 0.128
GenNegativesBlenderbot | 0.267 | 0.348 | 0.134
GenNegativesDialoGPT | 0.260 | 0.363 | 0.123
Table 5. Summary of the findings of our follow-up experiments to explain why better negative samplers lead to worse dense retrieval results. Experiment | Hypothesis | Evidence for H
---|---|---
E1 | H1 | ✓
E2 | ✓
E3 | H2 | ✗
E4 | ✓
E5 | H3 | ✓
E6 | ✗

Our findings for the six experiments (E1–E6) are displayed in Table 4. Bold
values indicate positive evidence for their respective hypothesis.

In the first experiment (E1), we manually annotated the relevance of 270 pairs
of dialogue context and negative samples (3 datasets $\times$ 3 dialogue
contexts $\times$ 10 negative samples $\times$ 3 negative sampling method). We
found that indeed the number of false positives increases when using better
negative sampling approaches, providing positive evidence for the hypothesis
that false positives are detrimental to the training of the dense retrieval
models. For the second experiment (E2) we employ a denoising technique that
uses the bottom negative samples from the top-k list instead of the first. We
found that the effectiveness improves by large margins when using dense model
to find negatives in all three datasets. In two datasets (MANtIS and MSDialog)
we find that the denoised negative sampling of the Bi-encoder yields
statistically significant improvements over Random (0.316 R@10 vs 0.307 R@10
for MANtIS and 0.397 R@10 vs 0.387 for MSDialog). The results for the second
experiment are thus additional positive evidence for the hypothesis that false
positives are detrimental.

In the third experiment (E3), by allowing the negative samples to be subsets
of the dialogue context, we expect the effectiveness of the model to drop by
large margins since the number of confusing negative samples increase. This
was not the case. The results indicates that possibly confusing negative
samples with exact matches with the dialogue context were not detrimental. For
the fourth experiment (E4) we expected that when using only the last utterance
of the dialogue to find negatives, we would decrease the number of confusing
negatives. This was the case for training the model with the bi-encoder as the
negative sampler.

In the final two experiments we tested whether we could find more informative
samples by using an expanded corpus of responses (E5) and by using generated
negative responses (E6). We found that using the larger corpus was beneficial
when using the bi-encoder negative sampler, showing that we can possibly find
more informative negative samples when using larger data. We found however
that the generated negative responses from both models were not effective, as
random samples from lead to better effectiveness when training the dense
retrieval model.

A summary interpretation of the experiments can be found in Table 5, where for
each experiment we display whether we found evidence or not for its
hypothesis. Overall we see that we have the most evidence for the first
hypothesis (H1) of false negatives degrading the training procedure. The
problems of false negatives when using harder negatives has been discussed
before for other retrieval tasks (Qu et al., 2020; Gao and Callan, 2021), and
we find evidence here on the conversational task that matches prior works on
denoising the hard negatives. Other hypothesis (H2 and H3) had partial
positive evidence, which suggests that they could also be a potential source
of difficulty when training dense models with harder negatives.

In conclusion, we demonstrate that a denoising strategy to remove false
negative samples is required to train dense models for ranking responses for
conversations, when taking into account hard negative samples.

##  6\. Conclusion

We study here the problem of full-rank retrieval of responses for dialogues.
We explore sparse and dense techniques that retrieve responses out of the
entire collection available. The expansion of responses, i.e. $resp2ctxt_{lu}$
showed to be a strong baseline for sparse retrieval. We also find that dense
retrieval needs large datasets in order to beat a strong sparse retrieval
baseline in the zero-shot setting. Our findings also suggest that fine-tuning
a bi-encoder dense retrieval model after intermediate training to be the best
performing method for the task of full-rank retrieval of responses for
dialogues. We finish our experiments with a thorough analysis of negative
sampling methods, exploring hypothesis that could explain why harder negatives
lead to worse effectiveness for the dense methods.

As future work we believe important directions include: taking advantage of
language-model based term re-weighting for sparse retrieval, intermediate
training strategies in order to improve their generalization power as well as
dense-sparse hybridization techniques for the full-rank retrieval of responses
for dialogues.

###### Acknowledgements.

This research has been supported by NWO projects SearchX (639.022.722) and NWO
Aspasia (015.013.027).

## References

  * (1)
  * Abdul-Jaleel et al. (2004) Nasreen Abdul-Jaleel, James Allan, W Bruce Croft, Fernando Diaz, Leah Larkey, Xiaoyan Li, Mark D Smucker, and Courtney Wade. 2004.  UMass at TREC 2004: Novelty and HARD.  _Computer Science Department Faculty Publication Series_ (2004), 189.
  * Aghajanyan et al. (2021) Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, and Sonal Gupta. 2021\.  Muppet: Massive multi-task representations with pre-finetuning.  _arXiv preprint arXiv:2101.11038_ (2021).
  * Anand et al. (2020) Avishek Anand, Lawrence Cavedon, Hideo Joho, Mark Sanderson, and Benno Stein. 2020.  Conversational search (dagstuhl seminar 19461). In _Dagstuhl Reports_ , Vol. 9. Schloss Dagstuhl-Leibniz-Zentrum für Informatik.
  * Chang et al. (2020) Jonathan P Chang, Caleb Chiam, Liye Fu, Andrew Z Wang, Justine Zhang, and Cristian Danescu-Niculescu-Mizil. 2020.  Convokit: A toolkit for the analysis of conversations.  _arXiv preprint arXiv:2005.04246_ (2020).
  * Craswell et al. (2020) Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M Voorhees. 2020.  Overview of the TREC 2019 deep learning track.  _arXiv preprint arXiv:2003.07820_ (2020).
  * Dai and Callan (2019) Zhuyun Dai and Jamie Callan. 2019.  Context-aware sentence/passage term importance estimation for first stage retrieval.  _arXiv preprint arXiv:1910.10687_ (2019).
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Formal et al. (2021) Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. 2021.  SPLADE v2: Sparse lexical and expansion model for information retrieval.  _arXiv preprint arXiv:2109.10086_ (2021).
  * Furnas et al. (1987) George W. Furnas, Thomas K. Landauer, Louis M. Gomez, and Susan T. Dumais. 1987.  The vocabulary problem in human-system communication.  _Commun. ACM_ 30, 11 (1987), 964–971.
  * Gao et al. (2022) Jianfeng Gao, Chenyan Xiong, Paul Bennett, and Nick Craswell. 2022.  Neural Approaches to Conversational Information Retrieval.  _CoRR_ abs/2201.05176 (2022).  https://arxiv.org/abs/2201.05176
  * Gao and Callan (2021) Luyu Gao and Jamie Callan. 2021.  Unsupervised corpus aware language model pre-training for dense passage retrieval.  _arXiv preprint arXiv:2108.05540_ (2021).
  * Gao et al. (2021a) Luyu Gao, Zhuyun Dai, and Jamie Callan. 2021a.  COIL: Revisit exact lexical match in information retrieval with contextualized inverted list.  _arXiv preprint arXiv:2104.07186_ (2021).
  * Gao et al. (2021b) Luyu Gao, Zhuyun Dai, and Jamie Callan. 2021b.  Rethink training of BERT rerankers in multi-stage retrieval pipeline. In _European Conference on Information Retrieval_. Springer, 280–286.
  * Gu et al. (2020) Jia-Chen Gu, Tianda Li, Quan Liu, Zhen-Hua Ling, Zhiming Su, Si Wei, and Xiaodan Zhu. 2020.  Speaker-aware BERT for multi-turn response selection in retrieval-based chatbots. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 2041–2044.
  * Gu et al. (2019) Jia-Chen Gu, Zhen-Hua Ling, and Quan Liu. 2019.  Interactive matching network for multi-turn response selection in retrieval-based chatbots. In _proceedings of the 28th ACM international conference on information and knowledge management_. 2321–2324.
  * Hadsell et al. (2006) Raia Hadsell, Sumit Chopra, and Yann LeCun. 2006\.  Dimensionality reduction by learning an invariant mapping. In _2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06)_ , Vol. 2. IEEE, 1735–1742.
  * Han et al. (2021) Janghoon Han, Taesuk Hong, Byoungjae Kim, Youngjoong Ko, and Jungyun Seo. 2021.  Fine-grained Post-training for Improving Retrieval-based Dialogue Systems. In _Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, Online, 1549–1558.  https://doi.org/10.18653/v1/2021.naacl-main.122
  * Henderson et al. (2019) Matthew Henderson, Iñigo Casanueva, Nikola Mrkšić, Pei-Hao Su, Ivan Vulić, et al. 2019\.  ConveRT: Efficient and Accurate Conversational Representations from Transformers.  _arXiv preprint arXiv:1911.03688_ (2019).
  * Hofstätter et al. (2021) Sebastian Hofstätter, Sheng-Chieh Lin, Jheng-Hong Yang, Jimmy Lin, and Allan Hanbury. 2021\.  Efficiently teaching an effective dense retriever with balanced topic aware sampling. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 113–122.
  * Humeau et al. (2019) Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019.  Poly-encoders: Transformer architectures and pre-training strategies for fast and accurate multi-sentence scoring.  _arXiv preprint arXiv:1905.01969_ (2019).
  * Jiang et al. (2021) Ting Jiang, Deqing Wang, Leilei Sun, Huayi Yang, Zhengyang Zhao, and Fuzhen Zhuang. 2021\.  Lightxml: Transformer with dynamic negative sampling for high-performance extreme multi-label text classification.  _arXiv preprint arXiv:2101.03305_ (2021).
  * Johnson et al. (2019) Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019\.  Billion-scale similarity search with GPUs.  _IEEE Transactions on Big Data_ 7, 3 (2019), 535–547.
  * Jones (1972) Karen Sparck Jones. 1972\.  A statistical interpretation of term specificity and its application in retrieval.  _Journal of documentation_ (1972).
  * Kadlec et al. (2015) Rudolf Kadlec, Martin Schmid, and Jan Kleindienst. 2015\.  Improved deep learning baselines for ubuntu corpus dialogs.  _arXiv preprint arXiv:1510.03753_ (2015).
  * Kummerfeld et al. (2019) Jonathan K. Kummerfeld, Sai R. Gouravajhala, Joseph J. Peper, Vignesh Athreya, Chulaka Gunasekara, Jatin Ganhotra, Siva Sankalp Patel, Lazaros C Polymenakos, and Walter Lasecki. 2019.  A Large-Scale Corpus for Conversation Disentanglement.  _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ (2019).  https://doi.org/10.18653/v1/p19-1374
  * Lan et al. (2021) Tian Lan, Deng Cai, Yan Wang, Yixuan Su, Xian-Ling Mao, and Heyan Huang. 2021\.  Exploring Dense Retrieval for Dialogue Response Selection.  _arXiv preprint arXiv:2110.06612_ (2021).
  * Li et al. (2016a) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a.  A Diversity-Promoting Objective Function for Neural Conversation Models. In _NAACL_. 110–119.
  * Li et al. (2016b) Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016b.  A Persona-Based Neural Conversation Model. In _ACL_. 994–1003.
  * Li et al. (2019) Jia Li, Chongyang Tao, Wei Wu, Yansong Feng, Dongyan Zhao, and Rui Yan. 2019\.  Sampling matters! an empirical study of negative sampling strategies for learning of matching models in retrieval-based dialogue systems. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_. 1291–1296.
  * Lin (2021) Jimmy Lin. 2021\.  A proposed conceptual framework for a representational approach to information retrieval.  _arXiv preprint arXiv:2110.01529_ (2021).
  * Lin et al. (2021a) Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021a.  Pyserini: A Python Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations. In _Proceedings of the 44th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)_. 2356–2362.
  * Lin et al. (2021b) Jimmy Lin, Rodrigo Nogueira, and Andrew Yates. 2021b.  Pretrained transformers for text ranking: Bert and beyond.  _Synthesis Lectures on Human Language Technologies_ 14, 4 (2021), 1–325.
  * Lin et al. (2020) Zibo Lin, Deng Cai, Yan Wang, Xiaojiang Liu, Hai-Tao Zheng, and Shuming Shi. 2020\.  The world is not binary: Learning to rank with grayscale data for dialogue response selection.  _arXiv preprint arXiv:2004.02421_ (2020).
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Lowe et al. (2015) Ryan Lowe, Nissan Pow, Iulian Serban, and Joelle Pineau. 2015\.  The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems.  _arXiv preprint arXiv:1506.08909_ (2015).
  * Mallia et al. (2021) Antonio Mallia, Omar Khattab, Torsten Suel, and Nicola Tonellotto. 2021.  Learning passage impacts for inverted indexes. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1723–1727.
  * Mazaré et al. (2018) Pierre-Emmanuel Mazaré, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018.  Training millions of personalized dialogue agents.  _arXiv preprint arXiv:1809.01984_ (2018).
  * Mikolov et al. (2013) Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013\.  Efficient estimation of word representations in vector space.  _arXiv preprint arXiv:1301.3781_ (2013).
  * Nguyen et al. (2016) Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016.  MS MARCO: A human generated machine reading comprehension dataset. In _CoCo@ NIPS_.
  * Nogueira and Cho (2019) Rodrigo Nogueira and Kyunghyun Cho. 2019.  Passage Re-ranking with BERT.  _arXiv preprint arXiv:1901.04085_ (2019).
  * Nogueira et al. (2019) Rodrigo Nogueira, Jimmy Lin, and AI Epistemic. 2019.  From doc2query to docTTTTTquery.  _Online preprint_ 6 (2019).
  * Oğuz et al. (2021) Barlas Oğuz, Kushal Lakhotia, Anchit Gupta, Patrick Lewis, Vladimir Karpukhin, Aleksandra Piktus, Xilun Chen, Sebastian Riedel, Wen-tau Yih, Sonal Gupta, et al. 2021\.  Domain-matched pre-training tasks for dense retrieval.  _arXiv preprint arXiv:2107.13602_ (2021).
  * Penha et al. (2019) Gustavo Penha, Alexandru Balan, and Claudia Hauff. 2019\.  Introducing MANtIS: a novel multi-domain information seeking dialogues dataset.  _arXiv preprint arXiv:1912.04639_ (2019).
  * Penha and Hauff (2019) Gustavo Penha and Claudia Hauff. 2019.  Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking.  _arXiv preprint arXiv:1912.08555_ (2019).
  * Penha and Hauff (2020) Gustavo Penha and Claudia Hauff. 2020.  Challenges in the Evaluation of Conversational Search Systems.. In _Converse@ KDD_.
  * Qiu et al. (2021) Yao Qiu, Jinchao Zhang, Huiying Ren, and Jie Zhou. 2021\.  Challenging Instances are Worth Learning: Generating Valuable Negative Samples for Response Selection Training.  _arXiv preprint arXiv:2109.06538_ (2021).
  * Qu et al. (2018) Chen Qu, Liu Yang, W Bruce Croft, Johanne R Trippas, Yongfeng Zhang, and Minghui Qiu. 2018\.  Analyzing and characterizing user intent in information-seeking conversations. In _The 41st international acm sigir conference on research & development in information retrieval_. 989–992.
  * Qu et al. (2020) Yingqi Qu, Yuchen Ding, Jing Liu, Kai Liu, Ruiyang Ren, Wayne Xin Zhao, Daxiang Dong, Hua Wu, and Haifeng Wang. 2020.  RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering.  _arXiv preprint arXiv:2010.08191_ (2020).
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics.  https://arxiv.org/abs/1908.10084
  * Ren et al. (2021) Ruiyang Ren, Shangwen Lv, Yingqi Qu, Jing Liu, Wayne Xin Zhao, QiaoQiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021.  PAIR: Leveraging passage-centric similarity relation for improving dense passage retrieval.  _arXiv preprint arXiv:2108.06027_ (2021).
  * Robertson and Walker (1994) Stephen E Robertson and Steve Walker. 1994.  Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In _SIGIR’94_. Springer, 232–241.
  * Robinson et al. (2020) Joshua Robinson, Ching-Yao Chuang, Suvrit Sra, and Stefanie Jegelka. 2020.  Contrastive learning with hard negative samples.  _arXiv preprint arXiv:2010.04592_ (2020).
  * Song et al. (2020) Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020.  Mpnet: Masked and permuted pre-training for language understanding.  _Advances in Neural Information Processing Systems_ 33 (2020), 16857–16867.
  * Su et al. (2020) Yixuan Su, Deng Cai, Qingyu Zhou, Zibo Lin, Simon Baker, Yunbo Cao, Shuming Shi, Nigel Collier, and Yan Wang. 2020.  Dialogue response selection with hierarchical curriculum learning.  _arXiv preprint arXiv:2012.14756_ (2020).
  * Tao et al. (2021a) Chongyang Tao, Jiazhan Feng, Chang Liu, Juntao Li, Xiubo Geng, and Daxin Jiang. 2021a.  Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning.  _arXiv preprint arXiv:2110.00159_ (2021).
  * Tao et al. (2021b) Chongyang Tao, Jiazhan Feng, Rui Yan, Wei Wu, and Daxin Jiang. 2021b.  A survey on response selection for retrieval-based dialogues. In _Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI_ , Vol. 21. 4619–4626.
  * Tao et al. (2019) Chongyang Tao, Wei Wu, Can Xu, Wenpeng Hu, Dongyan Zhao, and Rui Yan. 2019\.  Multi-Representation Fusion Network for Multi-Turn Response Selection in Retrieval-Based Chatbots. In _WSDM_. 267–275.
  * Thakur et al. (2020) Nandan Thakur, Nils Reimers, Johannes Daxenberger, and Iryna Gurevych. 2020.  Augmented sbert: Data augmentation method for improving bi-encoders for pairwise sentence scoring tasks.  _arXiv preprint arXiv:2010.08240_ (2020).
  * Thakur et al. (2021) Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021\.  BEIR: A heterogenous benchmark for zero-shot evaluation of information retrieval models.  _arXiv preprint arXiv:2104.08663_ (2021).
  * Whang et al. (2019) Taesun Whang, Dongyub Lee, Chanhee Lee, Kisu Yang, Dongsuk Oh, and Heuiseok Lim. 2019\.  An effective domain adaptive post-training method for bert in response selection.  _arXiv preprint arXiv:1908.04812_ (2019).
  * Whang et al. (2021) Taesun Whang, Dongyub Lee, Dongsuk Oh, Chanhee Lee, Kijong Han, Dong-hun Lee, and Saebyeok Lee. 2021.  Do response selection models really know what’s next? utterance manipulation strategies for multi-turn response selection. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 35. 14041–14049.
  * Wolf et al. (2019) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019\.  Huggingface’s transformers: State-of-the-art natural language processing.  _arXiv preprint arXiv:1910.03771_ (2019).
  * Wu et al. (2016) Yu Wu, Wei Wu, Chen Xing, Ming Zhou, and Zhoujun Li. 2016.  Sequential matching network: A new architecture for multi-turn response selection in retrieval-based chatbots.  _arXiv preprint arXiv:1612.01627_ (2016).
  * Wu et al. (2017) Yu Wu, Wei Wu, Chen Xing, Ming Zhou, and Zhoujun Li. 2017.  Sequential Matching Network: A New Architecture for Multi-turn Response Selection in Retrieval-Based Chatbots. In _ACL_. 496–505.
  * Xiong et al. (2020) Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020\.  Approximate nearest neighbor negative contrastive learning for dense text retrieval.  _arXiv preprint arXiv:2007.00808_ (2020).
  * Xu et al. (2020) Ruijian Xu, Chongyang Tao, Daxin Jiang, Xueliang Zhao, Dongyan Zhao, and Rui Yan. 2020\.  Learning an effective context-response matching model with self-supervised tasks for retrieval-based dialogues.  _arXiv preprint arXiv:2009.06265_ (2020).
  * Yan et al. (2016) Rui Yan, Yiping Song, and Hua Wu. 2016.  Learning to respond with deep neural networks for retrieval-based human-computer conversation system. In _Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval_. 55–64.
  * Yang et al. (2020b) Liu Yang, Minghui Qiu, Chen Qu, Cen Chen, Jiafeng Guo, Yongfeng Zhang, W Bruce Croft, and Haiqing Chen. 2020b.  IART: Intent-aware Response Ranking with Transformers in Information-seeking Conversation Systems.  _arXiv preprint arXiv:2002.00571_ (2020).
  * Yang et al. (2018) Liu Yang, Minghui Qiu, Chen Qu, Jiafeng Guo, Yongfeng Zhang, W Bruce Croft, Jun Huang, and Haiqing Chen. 2018\.  Response ranking with deep matching networks and external knowledge in information-seeking conversation systems. In _SIGIR_. 245–254.
  * Yang et al. (2020a) Zhen Yang, Ming Ding, Chang Zhou, Hongxia Yang, Jingren Zhou, and Jie Tang. 2020a.  Understanding negative sampling in graph representation learning. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1666–1676.
  * Yuan et al. (2019) Chunyuan Yuan, Wei Zhou, Mingming Li, Shangwen Lv, Fuqing Zhu, Jizhong Han, and Songlin Hu. 2019.  Multi-hop Selector Network for Multi-turn Response Selection in Retrieval-based Chatbots. In _EMNLP_. 111–120.
  * Zamani et al. (2022) Hamed Zamani, Johanne R. Trippas, Jeff Dalton, and Filip Radlinski. 2022.  Conversational Information Seeking.  _CoRR_ abs/2201.08808 (2022).
  * Zhan et al. (2021) Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. 2021\.  Optimizing dense retrieval model training with hard negatives. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1503–1512.
  * Zhang et al. (2019) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019.  Dialogpt: Large-scale generative pre-training for conversational response generation.  _arXiv preprint arXiv:1911.00536_ (2019).
  * Zhang and Zhao (2021a) Zhuosheng Zhang and Hai Zhao. 2021a.  Advances in multi-turn dialogue comprehension: A survey.  _arXiv preprint arXiv:2110.04984_ (2021).
  * Zhang and Zhao (2021b) Zhuosheng Zhang and Hai Zhao. 2021b.  Structural pre-training for dialogue comprehension.  _arXiv preprint arXiv:2105.10956_ (2021).
  * Zhou et al. (2018) Xiangyang Zhou, Lu Li, Daxiang Dong, Yi Liu, Ying Chen, Wayne Xin Zhao, Dianhai Yu, and Hua Wu. 2018\.  Multi-turn response selection for chatbots with deep attention matching network. In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_. 1118–1127.
  * Zhuang and Zuccon (2021) Shengyao Zhuang and Guido Zuccon. 2021.  Fast passage re-ranking with contextualized exact term matching and efficient passage expansion.  _arXiv preprint arXiv:2108.08513_ (2021).

##  Appendix A RM3 hyperparameters tested

Table 6. Appendix A. Hyperparameters tested for RM3. The hyperparameters are: number of expansion terms - number of expansion documents - weight to assign to the original query. We see that when we use more of the original query we get higher effectiveness and other hyperparameters do not help much either decreasing or increasing. | MANtIS | MSDialog | UDCDSTC8
---|---|---|---
| R@1 | R@10 | R@1 | R@10 | R@1 | R@10
BM25 | 0.133 | 0.299 | 0.064 | 0.177 | 0.027 | 0.070
+RM3 (5-5-0.5) | 0.089 | 0.214 | 0.042 | 0.133 | 0.014 | 0.042
+RM3 (5-5-0.7) | 0.097 | 0.238 | 0.048 | 0.150 | 0.017 | 0.051
+RM3 (5-10-0.5) | 0.072 | 0.190 | 0.036 | 0.123 | 0.010 | 0.037
+RM3 (5-10-0.7) | 0.086 | 0.218 | 0.043 | 0.145 | 0.012 | 0.047
+RM3 (5-15-0.5) | 0.066 | 0.175 | 0.032 | 0.115 | 0.008 | 0.038
+RM3 (5-15-0.7) | 0.080 | 0.206 | 0.039 | 0.135 | 0.010 | 0.047
+RM3 (10-5-0.5) | 0.094 | 0.248 | 0.042 | 0.150 | 0.016 | 0.053
+RM3 (10-5-0.7) | 0.103 | 0.266 | 0.048 | 0.161 | 0.019 | 0.058
+RM3 (10–10-0.5) | 0.081 | 0.221 | 0.038 | 0.136 | 0.014 | 0.049
+RM3 (10-10-0.7) | 0.092 | 0.246 | 0.046 | 0.153 | 0.015 | 0.057
+RM3 (10-15-0.5) | 0.073 | 0.206 | 0.035 | 0.127 | 0.011 | 0.049
+RM3 (10-15-0.7) | 0.085 | 0.232 | 0.044 | 0.142 | 0.017 | 0.056
+RM3 (15-5-0.5) | 0.099 | 0.261 | 0.045 | 0.152 | 0.019 | 0.057
+RM3 (15-5-0.7) | 0.107 | 0.275 | 0.050 | 0.162 | 0.022 | 0.061
+RM3 (15–10-0.5) | 0.085 | 0.238 | 0.036 | 0.145 | 0.013 | 0.054
+RM3 (15-10-0.7) | 0.097 | 0.258 | 0.048 | 0.156 | 0.016 | 0.059
+RM3 (15-15-0.5) | 0.077 | 0.219 | 0.036 | 0.131 | 0.014 | 0.054
+RM3 (15-15-0.7) | 0.091 | 0.245 | 0.042 | 0.148 | 0.017 | 0.060

##  Appendix B Examples of $resp2ctxt$ augmentations

Table 7. Appendix B. Random examples of augmentations by the $resp2ctxt$ methods. Dataset |  Context |  Response |  Augmentation $resp2ctxt$ |  Augmentation $resp2ctxt_{lu}$
---|---|---|---|---
MANtIS |  On macOs High Sierra I am trying to set a default icon for .pdf files: when I used Adobe Reader all the pdf files showed its icon, then I uninstalled it and replaced it with Foxit Reader as default app to open the same file. However all the pdf files now show a white icon and I would like them to show the Foxit logo instead.I tried to copy-paste the logo from Foxit to a random pdf and it does work, but this cannot be applied to the whole configuration (this means that I should copy-paste everytime the logo in every single pdf file).Is there a way to set the default icon? [UTT_SEP] ”Once you have changed the default application using Get Info, you may need to clear the icon cache.To get started, close all open applications and then launch Terminal from Macintosh HD ¿ Applications ¿ Utilities. Copy and paste the following command at the Terminal prompt and press Return to execute it: [code removed for paper] Restart, job done.From [URL] Safari download bar rar file icon show as VLC icon,which associate to The Unarchiver should shows its icon” [UTT_SEP] I did what you explained but the problem still exist. I have updated to Mojave since the last time. In Finder (choosing the list view for my files) I still see .pdf icon represented as a blank page. |  It appears Foxit Reader is missing the readerpdf.icns file. As a quick fix until the developer releases a solution you could borrow the PDF icon from another app or find an icon from a previous version. |  The newest version of Foxit Reader is , which comes with Firefox, and has an icon for Adobe PDF Reader. is a version that was released by Foxit in May of 2014. However, this is not an update, it has been updated to a different version. is in a separate file that the reader PDF icon can be found in a new window and it is a PDF icon a new icon is found on the new window and not the app store. I have not been […] |  I tried it and it worked. I just reinstalled the software and the app works fine. I have to say that i can’t find a PDF icon. I’ve renamed and downloaded the app.
MSDialog |  KF-UK you are not alone. What I did, is I uninstalled all new updates and paused the download feature. Hopefully someone has a workaround before serious problems occur. As you know Microsoft is a security conscious organization looking out for the well being of it customers. In other words we need your money but if the matter hits the fan we have no knowledge of your existence but would you recommend our products to your friends. [UTT_SEP] Even after KB4040724 (build 15063.632) update, the issue persists (in my case). There are two workaround for this issue reguarding my opinion at the moment: 1 - at login or later, the combination of shift-ctrl-win + B restores the graphics card and this ””awaken”” the monitor in standby 2 - Turn off Fast Boot by disabling it via Control Panel Power Options Choose What the Power Buttons do Change Settings that are Currently Unavailable untick ””Turn on fast startup””. |  I too am having problems with a dual monitor setup. My system had KB4038788 installed this month. I will attempt an uninstall and see if the dual monitor sleep issue goes away…. Complete symptom details for me are below. I am subscribing to this thread hoping Microsoft provides an answer. |  My 870 has been working great and my 810 is having some problems (like the aforementioned one). Is this an update that is supposed to fix the 870, which I’m not sure I’m missing - I’m not sure what happened and if I’m not a 870, I’m not a 870. I don’t know what happened and it just hangs up in bed […] |  for your reply. I have the same problem as you, if I am on my iMac. When I wake up, it will go to sleep but when I get up to do that the same thing happens again. It’s also not the same. I’m using the 2nd monitor. I’ve also slept well on this monitor for 2 years, so it’
UDCDSTC8 |  anyone notices there is something wrong with man page of netstat….. [UTT_SEP] wondering if its only me… or there is something wrong [UTT_SEP] it looks fine to me [UTT_SEP] my man page is not showing argument options, like -an etc |  mine shows both -a and -n : |  Hello, a lot of ubuntu users have asked me this on the forum, and I just can’t seem to figure this out, but when I open my terminal, it asks me ””Can I do anything with my ubuntu”?” [UTT_SEP] how […] |  how to change the name of a pc in gnome-server?


#  : Automatic Issue-Solution Pair Extraction from Community Live Chats

(2018)

TBD

††copyright: acmcopyright††journalyear: 2018††doi:
10.1145/1122445.1122456††conference: Woodstock ’18: ACM Symposium on Neural
Gaze Detection; June 03–05, 2018; Woodstock, NY††booktitle: Woodstock ’18: ACM
Symposium on Neural Gaze Detection, June 03–05, 2018, Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††ccs: Software and its engineering
Software maintenance tools††ccs: Software and its engineering Programming
teams††ccs: Software and its engineering Documentation