as the reward signal of an environment. They modeled a stack-like user agenda
to iteratively change the user goal and thus shifting the dialogue states.
While using a user simulator for environment modeling seems to be promising
for that it involves less human interaction, Zhang et al. (2019c) argued that
training a user simulator required a large amount of annotated data. Takanobu
et al. (2020) proposed Multi-Agent Dialog Policy Learning, where they have two
agents interact with each other, performing both user and agent, learning
policy simultaneously. Furthermore, they incorporated a role-specific reward
to facilitate role-based response generation and here both agents also acted
as the environment of the other one.

Response consistency in NLG is a challenging problem since it cannot be solved
by simply augmenting the training samples. Instead, additional corrections or
regulations should be designed. Wen et al. (2015b) proposed the Semantically
Controlled LSTM (SC-LSTM) which used a semantic planning gate to control the
retention or abandonment of dialogue actions thus ensuring the response
consistency. Likewise, Tran and Nguyen (2017) also applied a gating mechanism
to jointly perform sentence planning and surface realization where dialogue
action features were gated before entering GRU cells. Li et al. (2020b)
proposed Iterative Rectification Network (IRN), which combined a slot
inconsistency reward into the reinforcement learning framework. Thus, the
model iteratively checked the correctness of slots and corresponding values.

End-to-end systems are usually fully data-driven, which contributes to their
robust and natural responses. However, because of the finiteness of annotated
training samples, a hot research topic is figuring out how to increase the
response quality of end-to-end task-oriented dialogue systems with limited
data. Using rule-based methods to constrain response generation is a way to
improve response quality. Balakrishnan et al. (2019) used linearized tree-
structured representation as input to obtain control over discourse-level and
sentence-level semantic concepts. Kale and Rastogi (2020) used templates to
improve the semantic correctness of generated responses. They broke down the
response generation into a two-stage process: first generating semantically
correct but possibly incoherent responses based on the slots, with the
constraint of templates; then in the second stage, pretrained language models
were applied to re-organize the generated utterances into coherent ones.
Training the network with reinforcement learning was another strategy to
alleviate the reliance on annotated data. He et al. (2020a) trained two
teacher networks using a reinforcement learning framework with the objectives
of knowledge retrieval and response generation respectively. Then the student
network learns to produce responses by mimicking the output of teacher
networks. Training the network in a supervised way, Dai et al. (2020)
alternatively tried to optimize the learning strategy to improve the learning
efficiency of models given limited data. They combined the meta-learning
algorithm with human-machine interaction and achieved significant improvement
compared with strong baselines not trained with the meta-learning algorithms.
A more direct way to solve the data finiteness problem in supervised learning
was augmenting the dataset (Elder et al., 2020), which also improved the
response quality to some extent. Additionally, pretraining large-scale models
on common corpus and then applying them in a domain that lacks annotated data
is a popular approach in recent years (Henderson et al., 2019b; Mehri et al.,
2019; Bao et al., 2019b).

Retrieval-based methods are rare in task-oriented systems for the
insufficiency of data to cover all possible responses. However, Henderson et
al. (2019b) argued that in some situations not relating with specific
knowledge facts, retrieval-based methods were more precise and effective. They
first pretrained the response selection model on general domain corpora and
then finetuned on small target domain data. Experiments on six datasets from
different domains proved the effectiveness of the pretrained response
selection model. Lu et al. (2019) constructed Spatio-temporal context features
to facilitate response selection, and achieved significant improvements on the
Ubuntu IRC dataset.

##  4 Open-Domain Dialogue Systems

This section discusses open-domain dialogue systems, which are also called
chit-chat dialogue systems or non-task-oriented dialogue systems. Almost all
state-of-the-art open-domain dialogue systems are based on neural methods. We
organize this section by first briefly introducing the concepts of different
branches of open-domain dialogue systems, and then we focus on different
research challenges and hot topics. We view these challenges and hot topics as
different research directions in open-domain dialogue systems.

Instead of managing to complete tasks, open-domain dialogue systems aim to
perform chit-chat with users without the task and domain restriction (Ritter
et al., 2011) and are usually fully data-driven. Open-domain dialogue systems
are generally divided into three categories: generative systems, retrieval-
based systems, and ensemble systems. Generative systems apply sequence-to-
sequence models to map the user message and dialogue history into a response
sequence that may not appear in the training corpus. By contrast, retrieval-
based systems try to find a pre-existing response from a certain response set.
Ensemble systems combine generative methods and retrieval-based methods in two
ways: retrieved responses can be compared with generated responses to choose
the best among them; generative models can also be used to refine the
retrieved responses (Zhu et al., 2018; Song et al., 2016; Qiu et al., 2017;
Serban et al., 2017b). Generative systems can produce flexible and dialogue
context-related responses while sometimes they lack coherence and tend to make
dull responses. Retrieval-based systems select responses from human response
sets and thus are able to achieve better coherence in surface-level language.
However, retrieval systems are restricted by the finiteness of the response
sets and sometimes the responses retrieved show a weak correlation with the
dialogue context (Zhu et al., 2018).

In the next few subsections, we discuss some research challenges and hot
topics in open-domain dialogue systems.

###  4.1 Context Awareness

Dialogue context consists of user and system messages and is an important
source of information for dialogue agents to generate responses because
dialogue context decides the conversation topic and user goal (Serban et al.,
2017a). A context-aware dialogue agent responds not only depending on the
current message but also based on the conversation history. The earlier deep
learning based systems added up all word representations in dialogue history
or used a fixed-size window to focus on the recent context (Sordoni et al.,
2015b; Li et al., 2015). Serban et al. (2016) proposed Hierarchical Recurrent
Encoder-Decoder (HRED), which was ground-breaking in building context-
awareness dialogue systems. They built a word-level encoder to encode
utterances and a turn-level encoder to further summarize and deliver the topic
information over past turns. Xing et al. (2018) augmented the hierarchical
neural networks with the attention mechanism to help the model focus on more
meaningful parts of dialogue history.

Both generative and retrieval-based systems rely heavily on dialogue context
modeling. Shen et al. (2019) proposed Conversational Semantic Relationship RNN
(CSRR) to model the dialogue context in three levels: utterance-level, pair-
level, and discourse-level, capturing content information, user-system topic,
and global topic respectively. Zhang et al. (2019a) argued that the
hierarchical encoder-decoder does not lay enough emphasis on certain parts
when the decoder interacted with dialogue contexts. Also, they claimed that
attention-based HRED models also suffered from position bias and relevance
assumption insufficiency problems. Therefore, they proposed ReCoSa, whose
architecture was inspired by the transformer. The model first used a word-
level LSTM to encode dialogue contexts, and then self-attention was applied to
update the utterance representations. In the final stage, an encoder-decoder
attention was computed to facilitate the response generation process.
Additionally, Mehri et al. (2019) examined several applications of large-scale
pretrained models in dialogue context learning, providing guidance for large-
scale network selection in context modeling.

Dialogue rewriting problem aims to convert several messages into a single
message conveying the same information and dialogue context awareness is very
crucial to this task (Xu et al., 2020b). Su et al. (2019a) modeled multi-turn
dialogues via dialogue rewriting and benefited from the conciseness of
rewritten utterances.

Some works propose structured attention to improve context-awareness. Qiu et
al. (2020) learned structured dialogue context by combining structured
attention with a Variational Recurrent Neural Network (VRNN). Comparatively,
Ferracane et al. (2019) examined the RST discourse tree model proposed by Liu
and Lapata (2018) and observed little or even no discourse structures in the
learned latent tree. Thus, they argued that structured attention did not
benefit dialogue modeling and sometimes might even harm the performance.

Uniquely, Feng et al. (2020a) not only utilized dialogue history, but also
future conversations. Considering that in real inference situations dialogue
agents cannot be explicitly aware of future information, they first trained a
scenario-based model jointly on past and future context and then used an
imitation framework to transfer the scenario knowledge to a target network.

Tao et al. (2019), Jia et al. (2020) and Lin et al. (2020b) built retrieval-
based dialogue systems and tried to improve the response selection performance
via better context modeling. Tao et al. (2019) proposed Interaction-over-
Interaction network (IoI), which consisted of multiple interaction blocks to
perform deeper interactions between dialogue context and candidate responses.
Jia et al. (2020) organized the dialogue history into conversation threads by
performing classifications on their dependency relations. They further used a
pretrained Transformer model to encode the threads and candidate responses to
compute the matching score. Lin et al. (2020b) argued that response-retrieval
datasets should not only be annotated with relevant or irrelevant responses.
Instead, a greyscale metric should be used to measure the relevance degree of
a response given the dialogue context, thus increasing the context-awareness
ability of retrieval models.

###  4.2 Response Coherence

Coherence is one of the qualities that a good generator seeks (Stent et al.,
2005). Coherence means maintaining logic and consistency in a dialogue, which
is essential in an interaction process for that a response with weak
consistency in logic and grammar is hard to understand. Coherence is a hot
topic in generative systems but not in retrieval-based systems because
candidate responses in retrieval methods are usually human responses, which
are naturally coherent.

Wu et al. (2019b) improved the response coherence via the task of inconsistent
order detection. The dialogue systems learned response generation and order
detection jointly, which was self-supervised multi-task learning. Bao et al.
(2019a) proposed a Generation-Evaluation framework that evaluated the
qualities, including coherence, of the generated response. The feedback was
further seen as a reward signal in the reinforcement learning framework and
guided to a better dialogue strategy via policy gradient, thus improving the
response quality. Xu et al. (2019) presented the concept of meta-words. Meta-
words were diverse attributes describing the response. Learning dialogue based
on meta-words helped promote response generation in a more controllable way.
Liu et al. (2019a) used three granularities of encoders to encode raw words,
low-level clusters, and high-level clusters. The architecture was called
Vocabulary Pyramid Network (VPN), which performed a multi-pass encoding and
decoding process on hierarchical vocabularies to generate coherent responses.
Shen et al. (2019) also built a three-level hierarchical dialogue model to
capture richer features and improved the response quality. Ji et al. (2020)
built Cross Copy Networks (CCN), which used a copy mechanism to copy from
similar dialogues based on the current dialogue context. Thus, the system
benefited from the pre-existing coherent responses, which alleviated the need
of performing the reasoning process from scratch.

Bi et al. (2019) proposed to annotate sentence functions in existing
conversation datasets to improve the sentence logic and coherence of generated
responses. Akama et al. (2020) focused on data effectiveness as well. They
filtered out low-quality utterance pairs by scoring the relatedness and
connectivity, which was proved to be effective in improving the response
coherence.

Li et al. (2019a) improved the logical consistency of generated utterances by
incorporating an unlikelihood loss to control the distribution mismatches. Gao
et al. (2020b) raised response quality by ranking generated responses based on
user feedbacks like upvotes, downvotes, and comments on social networks. Zhu
et al. (2018) built a retrieval-enhanced generation model, which enhanced the
generated responses in two ways. First, a discriminator was trained with the
help of a retrieval system, and then the generator was trained in a GAN
framework under the supervision signal of a discriminator. Second, retrieved
responses were also used as a part of the generator input to provide a
coherent example for the generator. Xu et al. (2020a) achieved a global
coherent dialogue by constructing a knowledge graph from corpora. They further
performed graph walks to decide "what to say" and "how to say", thus improving
the dialogue flow coherence. Mesgar et al. (2019) proposed an assessment
approach for dialogue coherence evaluation by combining the dialogue act
prediction in a multi-task learning framework and learned rich dialogue
representations.

###  4.3 Response Diversity

The Bland and generic response is a long-existing problem in generative
dialogue systems. Because of the high frequency of generic responses like "I
don’t know" in training samples and the beam search decoding scheme of neural
sequence-to-sequence models, generative dialogue systems tend to respond with
universally acceptable but meaningless utterances (Serban et al., 2016;
Vinyals and Le, 2015; Sordoni et al., 2015b). For example, to respond to the
user message "I really want to have a meal", the agent tends to choose simple
responses like "It’s OK" instead of responding with more complicated sentences
like recommendations and suggestions.

Earlier works solve this challenge by modifying the decoding objective or
adding a reranking process. Li et al. (2015) replaced the traditional
likelihood objective $p(R|C)$ with mutual information. The optimization of
mutual information objective aims to achieve a Maximum Mutual Information
(MMI). Specifically, the task is to find a best response $R$ based on the
dialogue context $C$, in order to maximize their mutual information:

$\begin{split}\hat{R}&=arg\max_{R}{log\frac{P(C,R)}{P(C)P(R)}}\\\
&=arg\max_{R}{logP(R|C)-logP(R)}\end{split}$ (49)

The objective $p(R|C)$ causes the model to choose responses with high
probability even if the response is unconditionally frequent in the dataset,
thus causing it to ignore the content of $C$. Maximizing the mutual
information as Equation (49) solves this issue by achieving a trade-off
between safety and relativity.

Vijayakumar et al. (2016) combined a dissimilarity term into the beam search
objective and proposed Diverse Beam Search (DBS) to promote diversity.
Similarly, Shao et al. (2017) proposed a stochastic beam search algorithm by
performing stochastic sampling when choosing top-B responses. In the beam
search algorithm, siblings sharing the same parent nodes tended to guide to
similar sequences. Inspired by this, Li et al. (2016c) penalized siblings
sharing the same parent nodes using an additional term in the beam search
objective. This encouraged the algorithm to search more diverse paths by
expanding from different parent nodes. Some works further added a reranking
stage to select more diverse responses in the generated N-best list (Li et
al., 2015; Sordoni et al., 2015b; Shao et al., 2017).

A user message can be mapped into multiple acceptable responses, which is also
known as the one-to-many mapping problem. Qiu et al. (2019) considered the
one-to-many mapping problem in open-domain dialogue systems and proposed a
two-stage generation model to increase response diversity - the first stage
extracting common features of multiple ground truth responses and the second
stage extracting the distinctive ones. Ko et al. (2020) solved the one-to-many
mapping problem via a classification task to learn latent semantic
representations. So that given one example response, different ones could be
generated by exploring the semantically close vectors in the latent space.

Xu et al. (2019) proposed meta-words as the description of responses based on
which the memory-augmented generators could produce more diverse responses.
Bao et al. (2019a) used human instinct or pre-defined objective as a reward
signal in a reinforcement learning setting to prompt the agent to avoid
generating dull responses. Still, in a reinforcement learning framework, Zhu
et al. (2020) performed counterfactual reasoning to explore the potential
response space. Given a pre-existing response, the model inferred another
policy, which represented another possible response, thus increasing the
response diversity.

Utilizing external knowledge sources is another way to improve the diversity
of generated responses because it can enrich the content. Wu et al. (2020b)
built a common-sense dialogue generation model which seeks highly related
knowledge facts based on the dialogue history. Likewise, Su et al. (2020)
incorporated external knowledge sources to diversify the response generation,
but the difference was that they utilized non-conversational texts like news
articles as relevant knowledge facts, which were obviously easier to obtain.
Tian et al. (2019) used a memory module to abstract and store useful
information in the training corpus for generating diverse responses.

He and Glass (2019) used a negative training method to minimize the generation
of bland responses. They first collected negative samples and then gave
negative training signals based on these samples to fine-tune the model,
impeding the model to generate bland responses. To achieve a better
performance, Du and Black (2019) synthesized different dialogue models
designed for response diversity based on boosting. The ensemble model
significantly outperformed each of its base models.

Another approach to diversify the response generation is to make modifications
to the training corpus. Csáky et al. (2019) solved the challenge by filtering
out the generic responses in the dataset using an entropy-based algorithm,
which was simple but effective. Augmented with human feedback data, Gao et al.
(2020b) proposed that the generated responses could be reranked via a response
ranking framework trained on the human feedback data and responses with higher
quality including diversity were selected. Stasaski et al. (2020) proposed to
change the data collection pipeline by iteratively computing the diversity of
responses from different human participants in dataset construction and
selected those participants who tend to generate informative and diverse
responses.

###  4.4 Speaker Consistency and Personality-based Response

In open-domain dialogue systems, one big issue is that the responses are
totally learned from sample distributions of training data. The inconsistent
response may be received when asking the system about some personal facts
(e.g., age, hobbies). If the dataset contains multiple utterance pairs about
the query of age, then the response generated tends to be shifting, which is
unacceptable because personal facts are usually not random. Thus, for a data-
driven chatbot, it is necessary to be aware of its role and respond based on a
fixed persona.

Liu et al. (2020b) proposed a persona-based dialogue generator consisting of a
Receiver and a Transmitter. The receiver was responsible for modeling the
interlocutor’s persona through several turns’ chat while Transmitter generated
utterances based on the persona of agent and interlocutor, together with
conversation content. The proposed model supported conversations between two
persona-based chatbots by modeling each other’s persona. Without training with
additional Natural Language Inference labels, Kim et al. (2020) built an
imaginary listener following a normal generator, which reasoned over the
tokens generated by the generator and predicted a posterior distribution over
the personas in a certain space. After that, a self-conscious speaker
generated tokens aligned with the predicted persona. Likewise, Boyd et al.
(2020) used an augmented GPT-2 to reason over the past conversations and model
the target actor’s persona, conditioning on which persona consistency was
achieved.

Responding with personas needs to condition on some persona descriptions. For
example, to build a generous agent, descriptions like "I am a generous person"
are needed as a part of the model input. However, these descriptions require
hand-crafted feature design, which is labor intensive. Madotto et al. (2019)
proposed to use Model-Agnostic Meta-Learning (MAML) to adapt to new personas
with only a few training samples and needed no persona description. Majumder
et al. (2020a) relied on external knowledge sources to expand current persona
descriptions so that richer persona descriptions were obtained, and the model
could associate current descriptions with some commonsense facts.

Song et al. (2020a) argued that traditional persona-based systems were one-
stage systems and the responses they generated still contain many persona
inconsistent words. To tackle this issue, they proposed a three-stage
architecture to ensure persona consistency. A generate-delete-rewrite
mechanism was implemented to remove the unacceptable words generated in
prototype responses and rewrite them. Wang et al. (2019b) annotated a persona
dataset that contained the labels about each participant’s backgrounds. The
dataset consisted of dialogues persuading people for good behaviors and they
further analyzed the persuasion strategies of different personas.

###  4.5 Empathetic Response

Empathy means being able to sense other people’s feelings (Ma et al., 2020b).
An empathetic dialogue system can sense the user’s emotional changes and
produce appropriate responses with a certain sentiment. This is an essential
topic in chit-chat systems because it directly affects the user’s feeling and
to some extent decides the response quality. Industry systems such as
Microsoft’s Cortana, Facebook M, Google Assistant, and Amazon’s Alexa are all
equipped with empathy modules.

There are two ways to generate utterances with emotion: one is to use explicit
sentiment words as a part of input; another is to implicitly combine neural
words (Song et al., 2019). Song et al. (2019) proposed a unified framework
that uses a lexicon-based attention to explicitly plugin emotional words and a
sequence-level emotion classifier to classify the output sequence, implicitly
guiding the generator to generate emotional responses through backpropagation.
Zhong et al. (2020) used CoBERT for persona-based empathetic response
selection and further investigated the impact of persona on empathetic
responses. Smith et al. (2020) blended the skills of being knowledgeable,
empathetic, and role-aware in one open-domain conversation model and overcame
the bias issue when blending these skills.

Since the available datasets for empathetic conversations are scarce, Rashkin
et al. (2018) provided a new benchmark and dataset for empathetic dialogue
systems. Oraby et al. (2019) constructed a dialogue dataset with rich
emotional markups from user reviews and further proposed a novel way to
generate similar datasets with rich markups.

###  4.6 Conversation Topic

Daily chats of people usually involve a topic or goal. Actually, a topic or
goal is the key to keep each participant engaged in conversations and thus
being essential to a chatbot. In real applications, a good topic model helps
to retrieve related knowledge and guide the conversation instead of passively
responding to the user’s message (Xing et al., 2017). For example, if the user
mentions "I like sunny days", a topic-aware system may reason over relevant
external knowledge and produce responses like "I know there is a nice park
near the seaside, have you ever been there before?". Thus, the agent pushes
the conversation to a more engaging stage and enriches the dialogue content.

Wu et al. (2019c) tried to change the traditional passive response fashion and
radically pursue active guidance of conversation. The dialogue agent consists
of a leader and a follower, where the leader reasons over a knowledge graph
and decides the conversation topic. Likewise, a common-sense knowledge graph
was used by Liu et al. (2020c) to lead the conversation topic and make
recommendations. Tang et al. (2019) built a topic-aware retrieval-based
chatbot. It aimed to guide the conversation topic to the target one step by
step. It used a keyword predictor to predict turn-level keywords and selected
the discourse-level keyword based on that. The discourse-level keyword was
further fed into the retrieval model to retrieve responses regarding a certain
topic. Chen and Yang (2020) built a multi-view sequence-to-sequence model to
learn dialogue topics by first extracting dialogue structures of unstructured
chit-chat dialogues, then generating topic summaries using BART decoder.

Zhang and Danescu-Niculescu-Mizil (2020) studied the topic-aware chatbot in
counseling conversations. In counseling conversations, the agent led the
dialogue topic by deciding between empathetically addressing a situation
within the current range and moving on to a new target resolution. Cao et al.
(2019) studied chatbots in the psychotherapy treatment area and built a topic
prediction model to forecast the behavior codes for upcoming conversations,
thus guiding the dialogue.

###  4.7 Knowledge-Grounded System

External knowledge such as common-sense knowledge is a significant source of
information when organizing an utterance. Humans associate current
conversation context with their experiences and memories and produce
meaningful related responses, such capability results in the gap between human
and machine chit-chat systems. As discussed, the earlier chit-chat systems are
simply variants of machine translation systems, which can be viewed as
sequence-to-sequence language models. However, dialogue generation is much
more complicated than machine translation because of the higher freedom and
vaguer constraints. Thus, chit-chat systems cannot simply consist of a
sequence-to-sequence mapping since appropriate and informative responses are
always related to some external common-sense knowledge. Instead, there must be
a module incorporating world knowledge.

Many researchers devoted their research efforts to building knowledge-grounded
dialogue systems. A representative model is memory networks introduced in
Section 2.4. Knowledge grounded systems use Memory Networks to store external
knowledge and the generator retrieves relevant knowledge facts from it at the
generation stage (Ghazvininejad et al., 2018; Vougiouklis et al., 2016; Yin et
al., 2015).

Tian et al. (2019) built a memory-augmented conversation model. The proposed
model abstracted from the training samples and stored useful ones in the
memory module. Bao et al. (2019a) proposed a generation-evaluation framework
which aimed to make both conversation participants know more about each other
and develop a better knowledge selection strategy. An informativeness metric
was proposed as a reward signal for strategy learning of knowledge selection
in the reinforcement learning framework. Zhao et al. (2020b) built a
knowledge-grounded dialogue generation system based on GPT-2. They combined a
knowledge selection module into the language model and learned knowledge
selection and response generation simultaneously.

Li et al. (2019c) built a multi-turn document-grounded system. They used an
incremental transformer to encode multi-turns’ dialogue context and respective
documents retrieved. In the generation phase, they designed a two-stage
generation scheme. The first stage took dialogue context as input and
generated coherent responses; the second stage utilized both the utterance
from the first stage and the document retrieved for the current turn for
response generation. In this case, selecting knowledge based on both dialogue
context and generated response was called posterior knowledge selection, while
selecting knowledge with only dialogue context was called prior knowledge
selection, which only utilized prior information. Chen et al. (2020b) argued
that the gap between prior and posterior knowledge selection was large. Thus,
they designed a Posterior Information Prediction Module (PIPM) to enhance the
prior selection module and a Knowledge Distillation Based Training Strategy
(KDBTS) to alleviate the exposure bias in the generation phase.

Lin et al. (2020a) proposed Knowledge-Interaction and knowledge Copy (KIC).
They performed recurrent knowledge interactions during the decoding phase to
compute an attention distribution. Then they performed knowledge copy using a
knowledge-aware pointer network to copy knowledge words according to the
attention distribution computed. Wu et al. (2020b) built a common-sense
knowledge-aware model. It used a Felicitous Fact mechanism to raise the
knowledge relevance. Flexible Mode Fusion and Context-Knowledge Fusion were
proposed to achieve better knowledge integration. Majumder et al. (2020b)
proposed a multi-task learning framework consisting of two sub-tasks:
question-attribute prediction and look-ahead dialog structure prediction. Both
knowledge selection and utterance generation benefited from the learning of
subtasks and the model proposed outperformed strong baselines of media
dialogue generation. Wang et al. (2020c) built a quotation model in online
conversations and investigated the consistency between quoted sentences and
latent dialogue topics.

Knowledge-grounded datasets containing context-knowledge-response triples are
scarce and hard to obtain. Cho and May (2020) collected a large dataset
consisting of more than 26000 turns of improvised dialogues which were further
grounded with a larger movie corpus as external knowledge. Also tackling the
data insufficiency problem, Li et al. (2020a) proposed a method that did not
require context-knowledge-response triples for training. They viewed knowledge
as a latent variable to bridge the context and response. The variational
approach learned the parameters of the generator from both a knowledge corpus
and a dialogue corpus which were independent of each other.

The Knowledge graph is another source of external information, which is more
and more popular in knowledge-grounded systems. Jung et al. (2020) proposed a
dialogue-conditioned graph traversal model for knowledge-grounded dialogue
systems. The proposed model leveraged attention flows of two directions and
fully made use of the structured information of knowledge graph to flexibly
decide the expanding range of nodes and edges. Likewise, Zhang et al. (2019b)
applied graph attention to traverse the concept space, which was a common-
sense knowledge graph. The graph attention helped to move to more meaningful
nodes conditioning on dialogue context. Xu et al. (2020a) applied knowledge
graphs as an external source to control a coarse-level utterance generation.
Thus, the conversation was supported by common-sense knowledge, and the agent
guided the dialogue topic in a more reasonable way. Moon et al. (2019) built a
retrieval system retrieving responses based on the graph reasoning task. They
used a graph walker to traverse the graph conditioning on symbolic transitions
of the dialogue context. Huang et al. (2020a) proposed Graph-enhanced
Representations for Automatic Dialogue Evaluation (GRADE), a novel evaluation
metric for open-domain dialogue systems. This metric considered both
contextualized representations and topic-level graph representations. The main
idea was to use an external knowledge graph to model the conversation logic
flow as a part of the evaluation criteria.

###  4.8 Interactive Training

Interactive training, also called human-in-loop training, is a unique training
method for dialogue systems. Annotated data is fixed and limited, not being
able to cover all situations. Also, it takes a long time to train a good
system. But in some industrial products, the dialogue systems need not be
perfect when accomplishing their tasks. Thus, interactive training is
desirable because the dialogue systems can improve themselves via interactions
with users anywhere and anytime, which is a more flexible and cheap way to
finetune the parameters.

Li et al. (2016a) introduced a reinforcement learning-based online learning
framework. The agent interacted with a human dialogue partner and the partner
provided feedback as a reward signal. Asghar et al. (2016) first trained the
agent with two-stage supervised learning, and then used an interaction-based
reinforcement learning to finetune. Every time the user chose the best one
from K responses generated by the pretrained model and then responded to this
selected response. Instead of learning through being passively graded, Li et
al. (2016b) proposed a model that actively asked questions to seek
improvement. Active learning was applicable to both offline and online
learning settings. Hancock et al. (2019) argued that most conversation samples
an agent saw happened after it was pretrained and deployed. Thus, they
proposed a framework to train the agent from the real conversations it
participated in. The agent evaluated the satisfaction score of the user from
the user’s response of each turn and explicitly requested the user feedback
when it thought that a mistake has been made. The user feedback was further
used for learning. Bouchacourt and Baroni (2019) placed the interactive
learning in a cooperative game and tried to learn a long-term implicit
strategy via Reinforce algorithm.

###  4.9 Visual Dialogue

More and more researchers cast their eyes to a broader space and are not only
restricted to NLP. The combination of CV and NLP giving rise to tasks like
visual question answering attracted lots of interest. The VQA task is to
answer a question based on the content of a picture or video. Recently, this
has evolved into a more challenging task: visual dialogue, which conditions a
dialogue on the visual information and dialogue history. The dialogue can
consist of a series of questions, which is why it is more complicated than
VQA.

Figure 16: The architecture of VD-BERT, a state-of-the-art visual dialogue
system (Wang et al., 2020f) Figure 17:  Three samples from the IMAGE-CHAT
dataset (Shuster et al., 2020)

Gan et al. (2019) viewed visual dialogue as a multi-step reasoning process
over a series of questions. They learned semantic representation of the
question based on dialogue history and a given image, and recurrently updated
the representation. Shuster et al. (2019) proposed a set of image-based tasks
and provided strong baselines. Wang et al. (2020f) employed R-CNN as an image
encoder and fused the visual and dialogue modality with a VD-BERT (Figure 16).
The proposed architecture achieved sufficient interactions between multi-turn
dialogue and images.

Compared with image-grounded dialogue systems, video-grounded systems are more
interesting but also more challenging. There are two main challenges of video
dialogue, as claimed by Le and Hoi (2020). One is that both spatial and
temporal features exist in the video, which increases the difficulty of
feature extraction. Another is that video dialogue features span across
multiple conversation turns and thus are more complicated. A GPT-2 model was
applied by Le and Hoi (2020), being able to fuse multi-modality information
over different levels. Likewise, Le et al. (2019) built a multi-modal
transformer network to incorporate information from different modalities and
further applied a query-aware attention to extract context-related features
from non-text modalities. Le et al. (2020a) proposed a Bi-directional Spatio-
Temporal Learning (BiST) leveraging temporal-to-spatial and spatial-to-
temporal reasoning process and could adapt to the dynamically evolving
semantics in the video.

Some researchers hold different opinions on the effectiveness of dialogue
history in visual dialogue. Takmaz et al. (2020) proposed that many
expressions were already mentioned in previous turns and they built a visual
dialogue model grounded on both image and conversation history. They further
proved that better performance was achieved when grounding the model on
dialogue context. However, Agarwal et al. (2020) argued that though with
dialogue history the visual dialogue model could achieve better results, in
fact only a small proportion of cases benefited from the history. Furthermore,
they proved that existing evaluation metrics for visual dialogue promoted
generic responses.

The annotation of visual dialogue is laborious and thus the datasets are
scarce. Recently, some researchers have tried to tackle the data insufficiency
problem. Shuster et al. (2020) collected a dataset (IMAGE-CHAT, shown in
Figure 17) of image-grounded human-human conversations in which speakers are
asked to perform role-playing based on an emotional mood or style offered,
since the usage of such characteristics is also a significant factor in
engagingness. Kamezawa et al. (2020) constructed a visual-grounded dialogue
dataset. Uniquely, it additionally annotated the eye-gaze locations of the
interlocutor in the image to provide information on what the interlocutor was
paying attention to. Cogswell et al. (2020) proposed a method to utilize the
VQA data when adapting to a new task, minimizing the requirement of dialogue
data which is expensive to annotate.

##  5 Evaluation Approaches

Evaluation is an essential part of research in dialogue systems. It is not
only a way to assess the performance of agents, but it can also be a part of
the learning framework which provides signals to facilitate the learning (Bao
et al., 2019a). This section discusses the evaluation methods in task-oriented
and open-domain dialogue systems.

###  5.1 Evaluation Methods for Task-oriented Dialogue Systems

Task-oriented systems aim to accomplish tasks and thus have more direct
metrics evaluating their performance such as task completion rate and task
completion cost. Some evaluation methods also involve metrics like BLEU to
compare system responses with human responses, which will be discussed later.
In addition, human-based evaluation and user simulators are able to provide
real conversation samples.

Task Completion Rate is the rate of successful events in all task completion
attempts. It measures the task completion ability of a dialogue system. For
example, in movie ticket booking tasks, the Task Completion Rate is the
fraction of dialogues that meet all requirements specified by the user, such
as movie time, cinema location, movie genre, etc. The task completion rate was
applied in many task-oriented dialogue systems (Walker et al., 1997; Williams,
2007; Peng et al., 2017). Additionally, some works (Singh et al., 2002; Yih et
al., 2015) used partial success rate.

Task Completion Cost is the resources required when completing a task. Time
efficiency is a significant metric belonging to Task Completion Cost. In
dialogue-related tasks, the number of conversation turns is usually used to
measure the time efficiency and dialogue with fewer turns is preferred when
accomplishing the same task.

Human-based Evaluation provides user dialogues and user satisfaction scores
for system evaluation. There are two main streams of human-based evaluation.
One is to recruit human labor via crowdsourcing platforms to test-use a
dialogue system. The crowdsource workers converse with the dialogue systems
about predefined tasks and then metrics like Task Completion Rate and Task
Completion Cost can be calculated. Another is computing the evaluation metrics
in real user interactions, which means that evaluation is done after the
system is deployed in real use.

User Simulator provides simulated user dialogues based on pre-defined rules or
models. Since recruiting human labor is expensive and real user interactions
are not available until a mature system is deployed, user simulators are able
to provide task-oriented dialogues at a lower cost. There are two kinds of
user simulators. One is agenda-based simulators (Schatzmann and Young, 2009;
Li et al., 2016e; Ultes et al., 2017), which only feed dialogue systems with
the pre-defined user goal as a user message, without surface realization.
Another is model-based simulators (Chandramohan et al., 2011; Asri et al.,
2016), which generate user utterances using language models given constraint
information.

###  5.2 Evaluation Methods for Open-domain Dialogue Systems

Evaluation of open-domain dialogue systems has long been a challenging
problem. Unlike task-oriented systems, there is no clear metric like task
completion rate or task completion cost. Human evaluation has been adopted by
many works to converse with and rate dialogue agents. However, human
evaluation is not an ideal approach for that human labor is expensive and the
evaluation results are highly subjective, varying from person to person. Thus,
automatic and objective metrics are desirable. In general, there are two
categories of automatic metrics in recent research: word-overlap metrics and
neural metrics.

Word-overlap Metrics are widely used in Machine Translation and Summarization
tasks, which calculate the similarity between the generated sequence and the
ground truth sequence. Representative metrics like BLEU (Papineni et al.,
2002) and ROUGE (Lin, 2004) are n-gram matching metrics. METEOR (Banerjee and
Lavie, 2005) was further proposed with an improvement based on BLEU. It
identified the paraphrases and synonyms between the generated sequence and the
ground truth. Galley et al. (2015) extended the BLEU by exploiting numerical
ratings of responses. Liu et al. (2016) argued that word-overlap metrics were
not correlated well with human evaluation. These metrics are effective in
Machine Translation because each source sentence has a ground truth to compare
with, whereas in dialogues there may be many possible responses corresponding
with one user message, and thus an acceptable response may receive a low score
if simply computing word-overlap metrics.

Neural Metrics are metrics computed by neural models. Neural methods improve
the evaluation effectiveness in terms of adaptability compared with word-
overlap metrics, but they require an additional training process. Su et al.
(2015) used an RNN and a CNN model to extract turn-level features in a
sequence and give the score. Tao et al. (2018) proposed Ruber, which was an
automatic metric combining referenced and unreferenced components. The
referenced one computed the similarity between generated response
representations and ground truth representations, while the unreferenced one
learned a scoring model to rate the query-response pairs. Lowe et al. (2017)
learned representations of dialogue utterances using an RNN and then computed
the dot-product between generated response and ground truth response as an
evaluation score. Kannan and Vinyals (2017) and Bruni and Fernandez (2017)
used the discriminator of a GAN framework to distinguish the generated
responses from human responses. If a generated response achieved a high
confidence score, this was indicative of a human-like response, thus
desirable.

Evaluation of open-domain dialogue systems is a hot topic at present and many
researchers cast their eyes on this task recently. Bao et al. (2019a)
evaluated the generated responses by designing two metrics. One was the
informativeness metric calculating information utilization over turns. Another
was the coherence metric, which was predicted by GRUs, given the response,
context, and background as input. Likewise, Akama et al. (2020) designed
scoring functions to compute connectivity of utterance pairs and content
relatedness as two evaluation metrics and used another fusion function to
combine the metrics. Pang et al. (2020) combined four metrics in their
automatic evaluation framework: the context coherence metric based on GPT-2;
phrase fluency metric based on GPT-2; diversity metric based on n-grams;
logical self-consistency metric based on textual-entailment-inference. Mehri
and Eskenazi (2020) proposed a reference-free evaluation metric. They
annotated responses considering the following qualities: Understandable (0-1),
Maintains Context (1-3), Natural (1-3), Uses Knowledge (0-1), Interesting
(1-3), Overall Quality (1-5). Furthermore, a transformer was trained on these
annotated dialogues to compute the score of quality.

Gao et al. (2020b) leveraged large-scale human feedback data such as upvotes,
downvotes, and replies to learn a GPT-2-based response ranker. Thus, responses
were evaluated by their rankings given by the ranker. Deriu et al. (2020) also
evaluated the dialogue systems by ranking. They proposed a low-cost human-
involved evaluation framework, in which different conversational agents
conversed with each other and the human’s responsibility was to annotate
whether the generated utterance was human-like or not. The systems were
evaluated by comparing the number of turns their responses were judged as
human-like responses.

Liang et al. (2020) proposed a three-stage framework to denoise the self-
rating process. They first performed dialogue flow anomaly detection via self-
supervised representation learning, and then the model was fine-tuned with
smoothed self-reported user ratings. Finally, they performed a denoising
procedure by calculating the Shapley value and removed the samples with
negative values. Zhao et al. (2020a) trained RoBERTa as a response scorer to
achieve reference-free and semi-supervised evaluation. Sato et al. (2020)
constructed a test set by first generating several responses based on one user
message and then human evaluation was performed to annotate each response with
a score, where the response with the highest score was taken as a true
response and the remainder taken as false responses. Dialogue systems were
further evaluated by comparing the response selection accuracy on the test
set, where a cross-entropy loss was calculated between the generated response
and candidate responses to perform the selection operation. Likewise, Sinha et
al. (2020) trained a BERT-based model to discriminate between true and false
responses, where false responses were automatically generated. The model was
further used to predict the evaluation score of a response based on dialogue
context. Huang et al. (2020a) argued that responses should not be simply
evaluated based on their surface-level features, and instead the topic-level
features were more essential. They incorporated a common-sense graph in their
evaluation framework to obtain topic-level graph representations. The topic-
level graph representation and utterance-level representation were jointly
considered to evaluate the coherence of responses generated by open-domain
dialogue systems.

##  6 Datasets

The dataset is always one of the most essential components in dialogue systems
study. Nowadays the datasets are far from enough no matter for task-oriented
or open-domain dialogue systems, especially for those tasks requiring
additional annotations. For task-oriented dialogue systems, data can be
collected via two main methods. One is to recruit human labor via
crowdsourcing platforms to produce dialogues in a given task. Another is to
collect dialogues in real task completions like film ticket booking. For open-
domain dialogue systems, apart from dialogues collected in real interactions,
social media is also a significant source of data. Some social media companies
such as Twitter and Reddit provide API access to a small proportion of posts,
but these services are restricted by many legal terms which affect the
reproducibility of research. As a result, many recent works in dialogue
systems collect their own datasets for train and test.

In this section, we review and categorize these datasets and make a
comprehensive summary. To our best knowledge, Table LABEL:Datasets_for_Task-
oriented_dialogue_systems and LABEL:Datasets_for_Open-domain_dialogue_systems
cover almost all available datasets used in recent task-oriented or open-
domain dialogue systems.

###  6.1 Datasets for Task-oriented Dialogue Systems

Table 2: Datasets for Task-oriented dialogue systems |  |  |
---|---|---|---
Name |  Description |  Task |  Origin
Schema |  A dataset mainly for dialogue state tracking. |  Dialogue State Tracking |  Rastogi et al. (2020)
MetaLWOZ |  Collected by crowdsourcing platforms, spanning over 227 tasks and 47 domains. This dataset is designed for learning in unseen domains. |  Domain Transfer |  Lee et al. (2019)
E2E |  A dataset for end-to-end dialogue generation in restaurant domain. Data is collected in crowdsourced fashion. |  End-to-end Task-oriented Dialogue Systems |  Novikova et al. (2017)
MSR-E2E |  Contain dialogues spanning over 3 domains: movie-ticket booking, restaurant reservation, and taxi booking. |  End-to-end Task-oriented Dialogue Systems |  Li et al. (2018b)
YELPNLG |  A corpus consisting of utterances spanning over different restaurant attributes. |  Natural Language Generation |  Oraby et al. (2019)
Clinical Conversation data set |  It consists of conversations between physicians and participants. |  Natural Language Understanding |  Du et al. (2019)
OOS |  A large-scale dataset for intent detection. |  Natural Language Understanding |  Larson et al. (2019)
ATIS |  A dataset consisting of voice calls from people who intend to make flight reservations. |  Natural Language Understanding; Dialogue State Tracking |  Tur et al. (2010)
MultiWOZ |  Human-human written conversations with rich annotations spanning over multi-domains. |  Task-oriented Dialogue |  Budzianowski et al. (2018)
SNIPS-NLU |  Task-oriented dialogue dataset colleted in a crowdsourced fashion. It was used to train voice assistant agents. |  Task-oriented Dialogue |  https://github.com/snipsco/nlubenchmark
bAbI |  Restaurant table reservation dialogues. |  Task-oriented Dialogue |  Bordes et al. (2016)
JDC |  A Chinese customer service dataset, consisting of context-response pairs. |  Task-oriented Dialogue |  https://www.jddc.jd.com
UbuntuV2 |  It consists of dialogues collected via Ubuntu question-answering forum. |  Task-oriented Dialogue |  Lowe et al. (2015)
MICROSOFT DIALOGUE CHALLENGE data set |  A task-oriented dataset collected via Amazon Mechanical Turk. |  Task-oriented Dialogue |  Li et al. (2018b)
WOZ |  Task-oriented data collected in crowdsourced fashion. |  Task-oriented Dialogue |  Wen et al. (2016c)
DSTC series |  Multi-domain task-oriented dataset. |  Task-oriented Dialogue |  https://www.microsoft.com/en-us/research/event/dialog-state-tracking-challenge/
SimDial |  Simulated conversations spanning over multiple domains. |  Task-oriented Dialogue |  Zhao and Eskenazi (2018)
SMD |  Human-human dialogues in weather, navigation and scheduling domain. |  Task-oriented Dialogue |  Eric and Manning (2017)
BANKING |  Question-answer pairs with 77 categories in e-banking domain. |  Task-oriented Dialogue |  Henderson et al. (2019b)
Weather forecast |  A task-oriented dataset in the weather domain. |  Task-oriented Dialogue |  Balakrishnan et al. (2019)
MedDialog-(EN,CN) |  Large scale dataset in medical domain consisting of conversations between doctors and patients |  Task-oriented Dialogue |  He et al. (2020b)
CamRest |  It consists of human-human multi-turn dialogues in restaurant domain. |  Task-oriented Dialogue |  Wen et al. (2016a)
Taskmaster |  Contain dialogues spanning over 6 domains. It has 22.9 average length of conversational turns. |  Task-oriented Dialogue |  Byrne et al. (2019)
Frames |  Conversational dataset with annotations of semantic frame tracking. |  Task-oriented Dialogue |  Asri et al. (2017)
JDDC |  A Chinese customer service dataset, consisting of context-response pairs. |  Task-oriented Dialogue |  Chen et al. (2019a)
Court Debate Dataset |  A task-oriented dataset in judicial field containing court debate conversations. |  Task-oriented Dialogue |  Ji et al. (2020)
TreeDST |  A task-oriented dataset annotated with tree structured dialogue states and agent acts. |  Task-oriented Dialogue |  Cheng et al. (2020)
RiSAWOZ |  Contain utterances for 12 domains, annotated with rich semantic information. |  Task-oriented Dialogue |  Quan et al. (2020)
Cambridge Restaurant |  A task-oriented dataset in restaurant booking field. |  Task-oriented Dialogue |  Wen et al. (2016c)
SB-TOP |  A task-oriented dataset with semantic parsing annotation. It spans over 4 domains: Reminder, Weather, Calling and Music. |  Task-oriented Dialogue |  Aghajanyan et al. (2020)
GSIM |  A machine-machine task-oriented dataset. It covers two domains: restaurant table booking and movie ticket booking. |  Task-oriented Dialogue |  Shah et al. (2018)
SGD |  A schema-guided dataset spanning over multiple domains. |  Task-oriented Dialogue |  Rastogi et al. (2020)
cite-8K |  A task-oriented dataset collected in restaurant booking calls. |  Task-oriented Dialogue |  Coope et al. (2020)

###  6.2 Datasets for Open-domain Dialogue Systems

Table 3: Datasets for Open-domain dialogue systems |  |  |
---|---|---|---
Name |  Description |  Task |  Origin
Large-Scale Corpus for Conversation Disentanglement |  A dataset consisting of messages annotated with reply-structure graphs for dialogue disentanglement. |  Conversation Disentaglement |  Kummerfeld et al. (2018)
DuConv |  Collected in conversations between a conversation leader and a conversation follower. |  Conversation Topic |  Wu et al. (2019c)
PERSUASION FOR GOOD |  A topic-oriented dataset annotated with persuasion strategies. |  Conversation Topic |  Wang et al. (2019b)
MutualFriends |  A topic-oriented dataset based on bot-bot stratigical conversations. |  Conversation Topic |  He et al. (2017)
SAMSum |  A large-scale dialogue summary dataset. |  Conversation Topic |  Gliwa et al. (2019)
OpenDialKG |  It consists conversations between two agents and each dialogue corresponds with a knowledge graph path annotation. |  Conversation Topic; Dialogue Reasoning |  Moon et al. (2019)
doc2dial |  A dataset consisting of conversations annotated with goals and accociated documents. |  Conversation Topic; Knowledge-Grounded System |  Feng et al. (2020b)
DialEdit |  A dataset constructed for image editing via conversational language instructions. |  Conversational Image Editing |  Manuvina-kurike et al. (2018)
CHART DIALOGS |  A dataset containing dialogues describing matplotlib plot features. |  Conversational Plotting |  Shao and Nakashole (2020)
CONAN |  A multilingual dataset for hate speech tackling. |  Dialogue Classification |  Chung et al. (2019)
Dialogue NLI |  A NLI dataset with sentences annotated with entailment (E), neutral (N), or contradiction (C). |  Dialogue Inference |  Welleck et al. (2018)
MuTual |  A dialogue reasoning dataset containing English listening comprehension exams. |  Dialogue Reasoning |  Cui et al. (2020)
RST-DT |  It consists of samples from 385 news articles annotated with dialogue features. |  Discourse Parsing |  Carlson et al. (2002)
NLPCC |  A dataset consisting of emotional classification data. |  Empathetic Response |  http://tcci.ccf.org.cn/nlpcc.php
MELD |  A multi-party conversational dataset with emotion annotations. |  Empathetic Response |  Poria et al. (2019)
EMPATHETIC DIALOGUES |  A dataset containing conversations annotated with emotion labels. |  Empathetic Response |  Rashkin et al. (2018)
IEMOCAP |  Contain multi-party dialogues. Each dialogue is annotated with an emotion label. |  Empathetic Response |  Busso et al. (2008)
EmoryNLP |  Collected from Friends’ TV series, annotated with emotion labels. |  Empathetic Response |  Zahiri and Choi (2017)
MojiTalk |  A largescale dataset collected from Twitter, including emojis. |  Empathetic Response |  Zhou and Wang (2017)
CBET |  A dialogue dataset annotated with nine emotion labels: surprise, anger, love, sadness, joy, fear, guilt, disgust and thankfulness |  Empathetic Response |  Yadollahi et al. (2017)
Stanford Politeness Corpus |  A conversational dataset annotated with politeness labels. |  Empathetic Response |  Danescu-Niculescu-Mizil et al. (2013)
AIT-2018 |  Collected in SemEval-2018 Task 1: Affect in Tweets. |  Empathetic Response |  Mohammad et al. (2018)
EMOTyDA |  A dataset containing short videos about multi-party conversations, each annotated with respective emotion. |  Empathetic Response; Visual Dialogue |  Saha et al. (2020)
Wizard of Wikipedia |  A large-sclale dataset consisting of conversations grounded with Wikipedia knowledge. |  Knowledge-Grounded System |  Dinan et al. (2018)
CMU DoG |  A dataset consisting of conversations grounded with Wikipedia articles about popular movies. |  Knowledge-Grounded System |  Zhou et al. (2018)
Holl-E |  Contain dialogues grounded with documents. |  Knowledge-Grounded System |  Moghe et al. (2018)
Interview |  A dataset containing multi-party conversations in the form of interviews. |  Knowledge-Grounded System |  Majumder et al. (2020b)
Curiosity |  An open-domain dataset annotated with pre-existing user knowledge and dialogue acts, also grounding in Wikipedia. |  Knowledge-Grounded System |  Rodriguez et al. (2020)
KdConv |  A chinese knowledge-grounded dialogue dataset. |  Knowledge-Grounded System |  Zhou et al. (2020)
ELI5 |  A QA dataset grounded with retrieved documents. |  Knowledge-Grounded System |  Fan et al. (2019)
Topical Chat |  A knowledge-grounded dataset where the knowledge spans over eight different topics. |  Knowledge-Grounded System; Conversation Topic |  Gopalakrishnan et al. (2019)
WHERE ARE YOU? |  A dialogue dataset annotated with localization information. |  Localization Dialogue |  Hahn et al. (2020)
MMD |  A multi-modal dataset consisting of dialogues between sales agents and shoppers. |  Multi-modal Dialogue |  Saha et al. (2018)
OpenSubtitles |  A multilingual dataset made up of movie captions, containing about 8 billion words. |  Open-domain Dialogue |  Tiedemann (2012)
NTCIR |  A social media dataset collected from Sina Weibo. |  Open-domain Dialogue |  http://research.nii.ac.jp/ntcir/data/data-en.html
Twitter |  A social media dataset collected from Twitter. |  Open-domain Dialogue |  https://github.com/Marsan-Ma-zz/chatcorpus
Douban Conversation Corpus |  A social media dataset collected from Douban. |  Open-domain Dialogue |  Zhang et al. (2018c)
E-commerce Dialogue Corpus |  It consists of conversations between customers and customer service staff on Taobao. |  Open-domain Dialogue |  Zhang et al. (2018c)
REDDIT |  A social media dataset collected from REDDIT. |  Open-domain Dialogue |  Henderson et al. (2019a)
STC-SeFun |  A social media dataset collected from Tieba, Zhidao, Douban and Weibo. |  Open-domain Dialogue |  Bi et al. (2019)
DailyDialog |  A dataset consisting of daily dialogues, annotated with conversation intention and emotion information. |  Open-domain Dialogue |  Li et al. (2017c)
PDTB |  Dialogue dataset annotated with discourse relations. |  Open-domain Dialogue |  Prasad et al. (2008)
Luna |  Dialogue dataset with Italian relation annotations. |  Open-domain Dialogue |  Tonelli et al. (2010)
Edina-DR |  Dialogue dataset with English relation annotations, which is based on Luna data set. |  Open-domain Dialogue |  Ma et al. (2019)
Cornell Movie Dialog Corpus |  A dialogue dataset collected via IMDB database. |  Open-domain Dialogue |  Danescu-Niculescu-Mizil and Lee (2011)
Reddit Movie Dialogue Dataset |  A movie dialogue dataset collected from Reddit. |  Open-domain Dialogue |  Liu et al. (2020a)
LIGHT |  A dialogue dataset with configurable text adventure environment. |  Open-domain Dialogue |  Urbanek et al. (2019)
This American Life |  A media dialogue dataset collected in long-form expository podcast episodes. |  Open-domain Dialogue |  Mao et al. (2020)
RadioTalk |  A media dialogue dataset collected from radio transcripts. |  Open-domain Dialogue |  Beeferman et al. (2019)
French EPAC |  A media dialogue dataset collected from news. |  Open-domain Dialogue |  Esteve et al. (2010)
TREC Conversational Assistance |  An open-domain dataset spanning 30 conversation topics. |  Open-domain Dialogue |  Dalton et al. (2020)
Search as a Conversation |  A dataset for conversations with search engines. |  Open-domain Dialogue |  Ren et al. (2020)
Amazon Alexa Prize Competition |  A dataset containing real-world conversations between Amazon Alexa customers and Gunrock, which is a champion chatbot. |  Open-domain Dialogue |  Ram et al. (2018)
SwitchBoard |  An open-domain dataset containing English phone conversations. |  Open-domain Dialogue |  Jurafsky (1997)
Zhihu |  A Chinese social media dataset with posts and comments. |  Open-domain Dialogue |  https://www.zhihu.com
SPOLIN |  A dataset containing yes-and conversations. |  Open-domain Dialogue |  Cho and May (2020)
CRD3 |  A dataset collected in the role-playing game Dungeons and Dragons. |  Open-domain Dialogue |  Rameshkumar and Bailey (2020)
Baidu Zhidao |  A Chinese social media dataset with posts and comments. |  Open-domain Dialogue |  https://zhidao.baidu.com/
Webis Gmane Email Corpus 2019 |  A conversational dataset collected from 153M emails. |  Open-domain Dialogue |  Bevendorff et al. (2020)
LibreSpeech Corpus |  Contain 500 hours’ speech produced by 1252 participants. |  Open-domain Dialogue |  Panayotov et al. (2015)
Motivational Interviewing |  A dialogue dataset about conversational psychotherapy. |  Open-domain Dialogue |  Tanana et al. (2016)
SubTle Corpus |  Contact Ameixa for data. |  Open-domain Dialogue |  Lubis et al. (2018)
TED-LIUM |  TED-talk monologues. |  Open-domain Dialogue |  Fung et al. (2016)
ECG NLPCC 2017 Data |  Conversational dataset extracted from Weibo. |  Open-domain Dialogue |  Huang et al. (2018)
SEMEVAL15 |  QA dataset with answer quality annotations via Amazon Mechanical Turk. |  Question Answering |  Nakov et al. (2019)
AMAZONQA |  A QA dataset solving one-to-many problems. |  Question Answering |  Wan and McAuley (2016)
TGIF-QA |  A video-grounded QA dataset. |  Question Answering |  Jang et al. (2017)
QuAC |  A QA dataset with 14K QA dialogues. |  Question Answering |  Choi et al. (2018)
SQuAD |  A question-answering dataset collected in crowdsourced fashion. |  Question Answering |  Rajpurkar et al. (2018)
LIF |  A dataset constructed based on QuAC. |  Question Answering |  Kundu et al. (2020)
Yelp |  It consists of customer reviews from Yelp Dataset Challenge |  Response Retrieval |  Tang et al. (2015)
Debates |  The dataset consists of debates on Congerssional bills. |  Response Retrieval |  Thomas et al. (2006)
PERSONACHAT |  It provides profile information of the agents and background of users. |  Speaker Consistency and Personality Response |  Zhang et al. (2018b)
KvPI |  Contain consistency annotations between response and corresponding key-value profiles. |  Speaker Consistency and Personality Response |  Song et al. (2020b)
ConvAI2 |  A dataset constructed on the base of Persona-Chat, each conversation having profiles from a set containing persona candidates. |  Speaker Consistency and Personality Response |  Dinan et al. (2019)
PEC |  An open-domain dataset annotated with persona labels. |  Speaker Consistency and Personality Response; Empathetic Response |  Zhong et al. (2020)
GuessWhat?! |  A visual dialogue dataset for a two-player game about object recognition. |  Visual Dialogue |  De Vries et al. (2017)
VisDial |  A visual dialogue dataset whose images are obtained from COCO data set. |  Visual Dialogue |  https://visualdialog.org/data
AVSD |  A video-grounded dialogue dataset. |  Visual Dialogue |  Yoshino et al. (2018)
VFD |  A visual dialogue dataset annotated with unique eye-gaze locations. |  Visual Dialogue |  Kamezawa et al. (2020)
PhotoBook |  A dataset for task-oriented visual dialogues. |  Visual Dialogue |  Haber et al. (2019)
IGC |  A dataset containing conversations discussing a given image. |  Visual Dialogue |  Mostafazadeh et al. (2017)
Image-Chat |  Contain conversations grounded with images. The conversations are also annotated with personality. |  Visual Dialogue; Speaker Consistency and Personality Response |  Shuster et al. (2018)

##  7 Conclusions and Trends

More and more researchers are investigating conversational tasks. One factor
contributing to the popularity of conversational tasks is the increasing
demand for chatbots in industry and daily life. Industry agents like Apple’s
Siri, Microsoft’s Cortana, Facebook M, Google Assistant, and Amazon’s Alexa
have brought huge convenience to people’s lives. Another reason is that a
considerable amount of natural language data is in the form of dialogues,
which contributes to the efforts in dialogue research.

In this paper we discuss dialogue systems from two perspectives: model and
system type. Dialogue systems are a complicated but promising task because it
involves the whole process of communication between agent and human. The works
of recent years show an overwhelming preference towards neural methods, no
matter in task-oriented or open-domain dialogue systems. Neural methods
outperform traditional rule-based methods, statistical methods and machine
learning methods for that neural models have the stronger fitting ability and
require less hand-crafted feature engineering.

We mainly summarized and categorized the latest works in dialogue systems, but
also included other dialogue-related tasks. We hope these summaries and
insights provide a comprehensive picture of the state-of-the-art in this area
and pave the way for further research. Finally, we discuss some possible
research trends arising from the works reviewed:

Multimodal dialogue systems. The world is multimodal and humans observe it via
multiple senses such as vision, hearing, smell, taste, and touch. In a
conversational interaction, humans tend to make responses not only based on
text, but also on what they see and hear. Thus, some researchers argue that
chatbots should also have such abilities to blend information from different
modalities. There are some recent works trying to build multimodal dialogue
systems (Le et al., 2019; Chauhan et al., 2019; Saha et al., 2020; Singla et
al., 2020; Young et al., 2020), but these systems are still far from mature.

Multitask dialogue systems. Dialogue systems are categorized into task-
oriented and open-domain systems. Such a research boundary has existed for a
long time because task-oriented dialogue systems involve dialogue states,
which constrain the decoding process. However, works in end-to-end task-
oriented dialogue systems and knowledge-grounded open-domain systems provide a
possibility of blending these two categories into a single framework, or even
a single model. Such blended dialogue systems perform as assistants and
chatbots simultaneously.

Corpus exploration on Internet. In Section 6 we reviewed many datasets for
dialogue systems training. However, data is still far from enough to train a
perfect dialogue system. Many learning techniques are designed to alleviate
this problem, such as reinforcement learning, meta-learning, transfer
learning, and active learning. But many works ignore a significant source of
information, which is the dialogue corpus on the Internet. There is a large
volume of conversational corpus on the Internet but people have no access to
the raw corpus because much of it is in a messy condition. In the future,
dialogue agents should be able to explore useful corpus on the Internet in
real-time for training. This can be achieved by standardizing online corpus
access and their related legal terms. Moreover, real-time conversational
corpus exploration can be an independent task that deserves study.

User modeling. User modeling is a hot topic in both dialogue generation (Gür
et al., 2018; Serras et al., 2019) and dialogue systems evaluation (Kannan and
Vinyals, 2017). Basically, the user modeling module tries to simulate the real
decisions and actions of a human user. It makes decisions based on the
dialogue state or dialogue history. In dialogue generation tasks, modeling the
user helps the agent converse more coherently, based on the background
information or even speaking habits. Besides that, a mature user simulator can
provide an interactive training environment, which reduces the reliance on
annotated training samples when training a dialogue system. In dialogue
systems evaluation tasks, a user simulator provides user messages to test a
dialogue agent. More recent user simulators also give feedback concerning the
responses generated by the dialogue agent. However, user modeling is a
challenging task since no matter explicit user simulation or implicit user
modeling is actually the same in difficulty as response generation. Since
response generation systems are not perfect yet, user modeling can still be a
topic worthy of study.

Dialogue generation with a long-term goal. Most of our daily conversations are
chitchats without any purpose. However, there are quite a few conditions when
we purposely guide the conversation content to achieve a specific goal.
Current open-domain dialogue systems tend to model the conversation without a
long-term goal, which does not exhibit enough intelligence. There are some
recent works that apply reinforcement policy learning to model a long-term
reward which encourages the agent to converse with a long-term goal, such as
the work of Xu et al. (2020a). This topic will lead to strong artificial
intelligence, which is useful in some future applications such as negotiation
or story-telling chatbots.

## Acknowledgements

This research/project is supported by A*STAR under its Industry Alignment Fund
(LOA Award I1901E0046).

## References

  * Agarwal et al. (2020) Agarwal S, Bui T, Lee JY, Konstas I, Rieser V (2020) History for visual dialog: Do we really need it? arXiv preprint arXiv:200507493
  * Aghajanyan et al. (2020) Aghajanyan A, Maillard J, Shrivastava A, Diedrick K, Haeger M, Li H, Mehdad Y, Stoyanov V, Kumar A, Lewis M, et al. (2020) Conversational semantic parsing. arXiv preprint arXiv:200913655
  * Akama et al. (2020) Akama R, Yokoi S, Suzuki J, Inui K (2020) Filtering noisy dialogue corpora by connectivity and content relatedness. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 941–958
  * Aloysius and Geetha (2017) Aloysius N, Geetha M (2017) A review on deep convolutional neural networks. In: 2017 International Conference on Communication and Signal Processing (ICCSP), IEEE, pp 0588–0592
  * Annervaz et al. (2018) Annervaz K, Chowdhury SBR, Dukkipati A (2018) Learning beyond datasets: Knowledge graph augmented neural networks for natural language processing. arXiv preprint arXiv:180205930
  * Arora et al. (2013) Arora S, Batra K, Singh S (2013) Dialogue system: A brief review. arXiv preprint arXiv:13064134
  * Asghar et al. (2016) Asghar N, Poupart P, Jiang X, Li H (2016) Deep active learning for dialogue generation. arXiv preprint arXiv:161203929
  * Asri et al. (2016) Asri LE, He J, Suleman K (2016) A sequence-to-sequence model for user simulation in spoken dialogue systems. arXiv preprint arXiv:160700070
  * Asri et al. (2017) Asri LE, Schulz H, Sharma S, Zumer J, Harris J, Fine E, Mehrotra R, Suleman K (2017) Frames: a corpus for adding memory to goal-oriented dialogue systems. arXiv preprint arXiv:170400057
  * Aubert et al. (1994) Aubert X, Dugast C, Ney H, Steinbiss V (1994) Large vocabulary continuous speech recognition of wall street journal data. In: Proceedings of ICASSP’94. IEEE International Conference on Acoustics, Speech and Signal Processing, IEEE, vol 2, pp II–129
  * Bahdanau et al. (2014) Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:14090473
  * Baheti et al. (2020) Baheti A, Ritter A, Small K (2020) Fluent response generation for conversational question answering. arXiv preprint arXiv:200510464
  * Balakrishnan et al. (2019) Balakrishnan A, Rao J, Upasani K, White M, Subba R (2019) Constrained decoding for neural nlg from compositional representations in task-oriented dialogue. arXiv preprint arXiv:190607220
  * Banerjee and Lavie (2005) Banerjee S, Lavie A (2005) Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In: Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp 65–72
  * Bao et al. (2019a) Bao S, He H, Wang F, Lian R, Wu H (2019a) Know more about each other: Evolving dialogue strategy via compound assessment. arXiv preprint arXiv:190600549
  * Bao et al. (2019b) Bao S, He H, Wang F, Wu H, Wang H (2019b) Plato: Pre-trained dialogue generation model with discrete latent variable. arXiv preprint arXiv:191007931
  * Bapna et al. (2017) Bapna A, Tur G, Hakkani-Tur D, Heck L (2017) Towards zero-shot frame semantic parsing for domain scaling. arXiv preprint arXiv:170702363
  * Beeferman et al. (2019) Beeferman D, Brannon W, Roy D (2019) Radiotalk: A large-scale corpus of talk radio transcripts. arXiv preprint arXiv:190707073
  * Bengio et al. (1994) Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks 5(2):157–166
  * Bevendorff et al. (2020) Bevendorff J, Al Khatib K, Potthast M, Stein B (2020) Crawling and preprocessing mailing lists at scale for dialog analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 1151–1158
  * Bi et al. (2019) Bi W, Gao J, Liu X, Shi S (2019) Fine-grained sentence functions for short-text conversation. arXiv preprint arXiv:190710302
  * Bordes et al. (2013) Bordes A, Usunier N, Garcia-Duran A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Neural Information Processing Systems (NIPS), pp 1–9
  * Bordes et al. (2014) Bordes A, Glorot X, Weston J, Bengio Y (2014) A semantic matching energy function for learning with multi-relational data. Machine Learning 94(2):233–259
  * Bordes et al. (2016) Bordes A, Boureau YL, Weston J (2016) Learning end-to-end goal-oriented dialog. arXiv preprint arXiv:160507683
  * Bosselut et al. (2019) Bosselut A, Rashkin H, Sap M, Malaviya C, Celikyilmaz A, Choi Y (2019) Comet: Commonsense transformers for automatic knowledge graph construction. arXiv preprint arXiv:190605317
  * Bouchacourt and Baroni (2019) Bouchacourt D, Baroni M (2019) Miss tools and mr fruit: Emergent communication in agents learning about object affordances. arXiv preprint arXiv:190511871
  * Boyd et al. (2020) Boyd A, Puri R, Shoeybi M, Patwary M, Catanzaro B (2020) Large scale multi-actor generative dialog modeling. arXiv preprint arXiv:200506114
  * Bruni and Fernandez (2017) Bruni E, Fernandez R (2017) Adversarial evaluation for open-domain dialogue generation. In: Proceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, pp 284–288
  * Budzianowski et al. (2018) Budzianowski P, Wen TH, Tseng BH, Casanueva I, Ultes S, Ramadan O, Gašić M (2018) Multiwoz–a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv preprint arXiv:181000278
  * Busso et al. (2008) Busso C, Bulut M, Lee CC, Kazemzadeh A, Mower E, Kim S, Chang JN, Lee S, Narayanan SS (2008) Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation 42(4):335–359
  * Byrne et al. (2019) Byrne B, Krishnamoorthi K, Sankar C, Neelakantan A, Duckworth D, Yavuz S, Goodrich B, Dubey A, Cedilnik A, Kim KY (2019) Taskmaster-1: Toward a realistic and diverse dialog dataset. arXiv preprint arXiv:190905358
  * Cahill et al. (1999) Cahill L, Doran C, Evans R, Mellish C, Paiva D, Reape M, Scott D, Tipper N (1999) In search of a reference architecture for nlg systems. In: Proceedings of the 7th european workshop on natural language generation, Citeseer, pp 77–85
  * Campagna et al. (2020) Campagna G, Foryciarz A, Moradshahi M, Lam MS (2020) Zero-shot transfer learning with synthesized data for multi-domain dialogue state tracking. arXiv preprint arXiv:200500891
  * Cao et al. (2019) Cao J, Tanana M, Imel ZE, Poitras E, Atkins DC, Srikumar V (2019) Observing dialogue in therapy: Categorizing and forecasting behavioral codes. arXiv preprint arXiv:190700326
  * Carlson et al. (2002) Carlson L, Okurowski ME, Marcu D (2002) RST discourse treebank. Linguistic Data Consortium, University of Pennsylvania
  * Chandramohan et al. (2011) Chandramohan S, Geist M, Lefevre F, Pietquin O (2011) User simulation in dialogue systems using inverse reinforcement learning. In: Twelfth Annual Conference of the International Speech Communication Association
  * Chauhan et al. (2019) Chauhan H, Firdaus M, Ekbal A, Bhattacharyya P (2019) Ordinal and attribute aware response generation in a multimodal dialogue system. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 5437–5447
  * Chen et al. (2017a) Chen H, Liu X, Yin D, Tang J (2017a) A survey on dialogue systems: Recent advances and new frontiers. Acm Sigkdd Explorations Newsletter 19(2):25–35
  * Chen and Yang (2020) Chen J, Yang D (2020) Multi-view sequence-to-sequence models with conversational structure for abstractive dialogue summarization. arXiv preprint arXiv:201001672
  * Chen et al. (2020a) Chen J, Zhang R, Mao Y, Xu J (2020a) Parallel interactive networks for multi-domain dialogue state generation. arXiv preprint arXiv:200907616
  * Chen et al. (2017b) Chen L, Zhou X, Chang C, Yang R, Yu K (2017b) Agent-aware dropout dqn for safe and efficient on-line dialogue policy learning. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp 2454–2464
  * Chen et al. (2019a) Chen M, Liu R, Shen L, Yuan S, Zhou J, Wu Y, He X, Zhou B (2019a) The jddc corpus: A large-scale multi-turn chinese dialogue dataset for e-commerce customer service. arXiv preprint arXiv:191109969
  * Chen et al. (2019b) Chen W, Chen J, Qin P, Yan X, Wang WY (2019b) Semantically conditioned dialog response generation via hierarchical disentangled self-attention. arXiv preprint arXiv:190512866
  * Chen et al. (2019c) Chen X, Xu J, Xu B (2019c) A working memory model for task-oriented dialog response generation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 2687–2693
  * Chen et al. (2020b) Chen X, Meng F, Li P, Chen F, Xu S, Xu B, Zhou J (2020b) Bridging the gap between prior and posterior knowledge selection for knowledge-grounded dialogue generation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 3426–3437
  * Chen et al. (2016) Chen YN, Hakkani-Tür D, Tür G, Gao J, Deng L (2016) End-to-end memory networks with knowledge carryover for multi-turn spoken language understanding. In: Interspeech, pp 3245–3249
  * Cheng et al. (2020) Cheng J, Agrawal D, Alonso HM, Bhargava S, Driesen J, Flego F, Kaplan D, Kartsaklis D, Li L, Piraviperumal D, et al. (2020) Conversational semantic parsing for dialog state tracking. arXiv preprint arXiv:201012770
  * Cho and May (2020) Cho H, May J (2020) Grounding conversations with improvised dialogues. arXiv preprint arXiv:200409544
  * Cho et al. (2014a) Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014a) On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:14091259
  * Cho et al. (2014b) Cho K, Van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y (2014b) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:14061078
  * Choi et al. (2018) Choi E, He H, Iyyer M, Yatskar M, Yih Wt, Choi Y, Liang P, Zettlemoyer L (2018) Quac: Question answering in context. arXiv preprint arXiv:180807036
  * Chung et al. (2014) Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:14123555
  * Chung et al. (2019) Chung YL, Kuzmenko E, Tekiroglu SS, Guerini M (2019) Conan–counter narratives through nichesourcing: a multilingual dataset of responses to fight online hate speech. arXiv preprint arXiv:191003270
  * Cogswell et al. (2020) Cogswell M, Lu J, Jain R, Lee S, Parikh D, Batra D (2020) Dialog without dialog data: Learning visual dialog agents from vqa data. arXiv preprint arXiv:200712750
  * Conneau et al. (2016) Conneau A, Schwenk H, Barrault L, Lecun Y (2016) Very deep convolutional networks for text classification. arXiv preprint arXiv:160601781
  * Coope et al. (2020) Coope S, Farghly T, Gerz D, Vulić I, Henderson M (2020) Span-convert: Few-shot span extraction for dialog with pretrained conversational representations. arXiv preprint arXiv:200508866
  * Csáky et al. (2019) Csáky R, Purgai P, Recski G (2019) Improving neural conversational models with entropy-based data filtering. arXiv preprint arXiv:190505471
  * Cui et al. (2020) Cui L, Wu Y, Liu S, Zhang Y, Zhou M (2020) Mutual: A dataset for multi-turn dialogue reasoning. arXiv preprint arXiv:200404494
  * Dai et al. (2020) Dai Y, Li H, Tang C, Li Y, Sun J, Zhu X (2020) Learning low-resource end-to-end goal-oriented dialog for fast and reliable system deployment. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 609–618
  * Dalton et al. (2020) Dalton J, Xiong C, Callan J (2020) Trec cast 2019: The conversational assistance track overview. arXiv preprint arXiv:200313624
  * Danescu-Niculescu-Mizil and Lee (2011) Danescu-Niculescu-Mizil C, Lee L (2011) Chameleons in imagined conversations: A new approach to understanding coordination of linguistic style in dialogs. arXiv preprint arXiv:11063077
  * Danescu-Niculescu-Mizil et al. (2013) Danescu-Niculescu-Mizil C, Sudhof M, Jurafsky D, Leskovec J, Potts C (2013) A computational approach to politeness with application to social factors. arXiv preprint arXiv:13066078
  * De Vries et al. (2017) De Vries H, Strub F, Chandar S, Pietquin O, Larochelle H, Courville A (2017) Guesswhat?! visual object discovery through multi-modal dialogue. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 5503–5512
  * Deng et al. (2012) Deng L, Tur G, He X, Hakkani-Tur D (2012) Use of kernel deep convex networks and end-to-end learning for spoken language understanding. In: 2012 IEEE Spoken Language Technology Workshop (SLT), IEEE, pp 210–215
  * Deoras and Sarikaya (2013) Deoras A, Sarikaya R (2013) Deep belief network based semantic taggers for spoken language understanding. In: Interspeech, pp 2713–2717
  * Deriu et al. (2020) Deriu J, Tuggener D, von Däniken P, Campos JA, Rodrigo A, Belkacem T, Soroa A, Agirre E, Cieliebak M (2020) Spot the bot: A robust and efficient framework for the evaluation of conversational dialogue systems. arXiv preprint arXiv:201002140
  * Devlin et al. (2018) Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805
  * Dhingra et al. (2016) Dhingra B, Li L, Li X, Gao J, Chen YN, Ahmed F, Deng L (2016) Towards end-to-end reinforcement learning of dialogue agents for information access. arXiv preprint arXiv:160900777
  * Dinan et al. (2018) Dinan E, Roller S, Shuster K, Fan A, Auli M, Weston J (2018) Wizard of wikipedia: Knowledge-powered conversational agents. arXiv preprint arXiv:181101241
  * Dinan et al. (2019) Dinan E, Logacheva V, Malykh V, Miller A, Shuster K, Urbanek J, Kiela D, Szlam A, Serban I, Lowe R, et al. (2019) The second conversational intelligence challenge (convai2). arXiv preprint arXiv:190200098
  * Du et al. (2019) Du N, Chen K, Kannan A, Tran L, Chen Y, Shafran I (2019) Extracting symptoms and their status from clinical conversations. arXiv preprint arXiv:190602239
  * Du and Black (2019) Du W, Black AW (2019) Boosting dialog response generation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 38–43
  * Dušek and Jurčíček (2016a) Dušek O, Jurčíček F (2016a) A context-aware natural language generator for dialogue systems. arXiv preprint arXiv:160807076
  * Dušek and Jurčíček (2016b) Dušek O, Jurčíček F (2016b) Sequence-to-sequence generation for spoken dialogue via deep syntax trees and strings. arXiv preprint arXiv:160605491
  * Elder et al. (2020) Elder H, O’Connor A, Foster J (2020) How to make neural natural language generation as reliable as templates in task-oriented dialogue. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 2877–2888
  * Elman (1990) Elman JL (1990) Finding structure in time. Cognitive science 14(2):179–211
  * Eric and Manning (2017) Eric M, Manning CD (2017) Key-value retrieval networks for task-oriented dialogue. arXiv preprint arXiv:170505414
  * Esteve et al. (2010) Esteve Y, Bazillon T, Antoine JY, Béchet F, Farinas J (2010) The epac corpus: Manual and automatic annotations of conversational speech in french broadcast news. In: LREC, Citeseer
  * Fan et al. (2019) Fan A, Jernite Y, Perez E, Grangier D, Weston J, Auli M (2019) Eli5: Long form question answering. arXiv preprint arXiv:190709190
  * Fan et al. (2014) Fan M, Zhou Q, Chang E, Zheng F (2014) Transition-based knowledge graph embedding with relational mapping properties. In: Proceedings of the 28th Pacific Asia conference on language, information and computing, pp 328–337
  * Feldman and El-Yaniv (2019) Feldman Y, El-Yaniv R (2019) Multi-hop paragraph retrieval for open-domain question answering. arXiv preprint arXiv:190606606
  * Feng et al. (2019) Feng J, Tao C, Wu W, Feng Y, Zhao D, Yan R (2019) Learning a matching model with co-teaching for multi-turn response selection in retrieval-based dialogue systems. arXiv preprint arXiv:190604413
  * Feng et al. (2020a) Feng S, Ren X, Chen H, Sun B, Li K, Sun X (2020a) Regularizing dialogue generation by imitating implicit scenarios. arXiv preprint arXiv:201001893
  * Feng et al. (2020b) Feng S, Wan H, Gunasekara C, Patel SS, Joshi S, Lastras LA (2020b) doc2dial: A goal-oriented document-grounded dialogue dataset. arXiv preprint arXiv:201106623
  * Ferracane et al. (2019) Ferracane E, Durrett G, Li JJ, Erk K (2019) Evaluating discourse in structured text representations. arXiv preprint arXiv:190601472
  * Finn et al. (2017) Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, PMLR, pp 1126–1135
  * Fung et al. (2016) Fung P, Dey A, Siddique FB, Lin R, Yang Y, Bertero D, Wan Y, Chan RHY, Wu CS (2016) Zara: a virtual interactive dialogue system incorporating emotion, sentiment and personality recognition. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: System Demonstrations, pp 278–281
  * Galley et al. (2015) Galley M, Brockett C, Sordoni A, Ji Y, Auli M, Quirk C, Mitchell M, Gao J, Dolan B (2015) deltableu: A discriminative metric for generation tasks with intrinsically diverse targets. arXiv preprint arXiv:150606863
  * Gan et al. (2019) Gan Z, Cheng Y, Kholy AE, Li L, Liu J, Gao J (2019) Multi-step reasoning via recurrent dual attention for visual dialog. arXiv preprint arXiv:190200579
  * Gangadharaiah and Narayanaswamy (2020) Gangadharaiah R, Narayanaswamy B (2020) Recursive template-based frame generation for task oriented dialog. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 2059–2064
  * Gao et al. (2018) Gao J, Galley M, Li L (2018) Neural approaches to conversational ai. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp 1371–1374
  * Gao et al. (2020a) Gao S, Zhang Y, Ou Z, Yu Z (2020a) Paraphrase augmented task-oriented dialog generation. arXiv preprint arXiv:200407462
  * Gao et al. (2020b) Gao X, Zhang Y, Galley M, Brockett C, Dolan B (2020b) Dialogue response ranking training with large-scale human feedback data. arXiv preprint arXiv:200906978
  * Gao et al. (2020c) Gao Y, Wu CS, Joty S, Xiong C, Socher R, King I, Lyu M, Hoi SC (2020c) Explicit memory tracker with coarse-to-fine reasoning for conversational machine reading. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 935–945
  * Ghazvininejad et al. (2018) Ghazvininejad M, Brockett C, Chang MW, Dolan B, Gao J, Yih Wt, Galley M (2018) A knowledge-grounded neural conversation model. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32
  * Gliwa et al. (2019) Gliwa B, Mochol I, Biesek M, Wawer A (2019) Samsum corpus: A human-annotated dialogue dataset for abstractive summarization. arXiv preprint arXiv:191112237
  * Goddeau et al. (1996) Goddeau D, Meng H, Polifroni J, Seneff S, Busayapongchai S (1996) A form-based dialogue manager for spoken language applications. In: Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP’96, IEEE, vol 2, pp 701–704
  * Golovanov et al. (2019) Golovanov S, Kurbanov R, Nikolenko S, Truskovskyi K, Tselousov A, Wolf T (2019) Large-scale transfer learning for natural language generation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 6053–6058
  * Goo et al. (2018) Goo CW, Gao G, Hsu YK, Huo CL, Chen TC, Hsu KW, Chen YN (2018) Slot-gated modeling for joint slot filling and intent prediction. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp 753–757
  * Goodfellow et al. (2014) Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial networks. arXiv preprint arXiv:14062661
  * Gopalakrishnan et al. (2019) Gopalakrishnan K, Hedayatnia B, Chen Q, Gottardi A, Kwatra S, Venkatesh A, Gabriel R, Hakkani-Tür D, AI AA (2019) Topical-chat: Towards knowledge-grounded open-domain conversations. In: INTERSPEECH, pp 1891–1895
  * Gordon-Hall et al. (2020) Gordon-Hall G, Gorinski PJ, Cohen SB (2020) Learning dialog policies from weak demonstrations. arXiv preprint arXiv:200411054
  * Graves et al. (2014) Graves A, Wayne G, Danihelka I (2014) Neural turing machines. arXiv preprint arXiv:14105401
  * Graves et al. (2016) Graves A, Wayne G, Reynolds M, Harley T, Danihelka I, Grabska-Barwińska A, Colmenarejo SG, Grefenstette E, Ramalho T, Agapiou J, et al. (2016) Hybrid computing using a neural network with dynamic external memory. Nature 538(7626):471–476
  * Gruber and Jockisch (2020) Gruber N, Jockisch A (2020) Are gru cells more specific and lstm cells more sensitive in motive classification of text? Frontiers in Artificial Intelligence 3(40):1–6
  * Gu et al. (2016) Gu J, Lu Z, Li H, Li VO (2016) Incorporating copying mechanism in sequence-to-sequence learning. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Berlin, Germany, pp 1631–1640, DOI 10.18653/v1/P16-1154, URL https://www.aclweb.org/anthology/P16-1154
  * Guo et al. (2020) Guo X, Yu M, Gao Y, Gan C, Campbell M, Chang S (2020) Interactive fiction game playing as multi-paragraph reading comprehension with reinforcement learning. arXiv preprint arXiv:201002386
  * Gür et al. (2018) Gür I, Hakkani-Tür D, Tür G, Shah P (2018) User modeling for task oriented dialogues. In: 2018 IEEE Spoken Language Technology Workshop (SLT), IEEE, pp 900–906
  * Haber et al. (2019) Haber J, Baumgärtner T, Takmaz E, Gelderloos L, Bruni E, Fernández R (2019) The photobook dataset: Building common ground through visually-grounded dialogue. arXiv preprint arXiv:190601530
  * Hahn et al. (2020) Hahn M, Krantz J, Batra D, Parikh D, Rehg JM, Lee S, Anderson P (2020) Where are you? localization from embodied dialog. arXiv preprint arXiv:201108277
  * Hakkani-Tür et al. (2016) Hakkani-Tür D, Tür G, Celikyilmaz A, Chen YN, Gao J, Deng L, Wang YY (2016) Multi-domain joint semantic frame parsing using bi-directional rnn-lstm. In: Interspeech, pp 715–719
  * Ham et al. (2020) Ham D, Lee JG, Jang Y, Kim KE (2020) End-to-end neural pipeline for goal-oriented dialogue systems using gpt-2. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 583–592
  * Han et al. (2019) Han M, Kang M, Jung H, Hwang SJ (2019) Episodic memory reader: Learning what to remember for question answering from streaming data. arXiv preprint arXiv:190306164
  * Hancock et al. (2019) Hancock B, Bordes A, Mazare PE, Weston J (2019) Learning from dialogue after deployment: Feed yourself, chatbot! arXiv preprint arXiv:190105415
  * Hashemi et al. (2016) Hashemi HB, Asiaee A, Kraft R (2016) Query intent detection using convolutional neural networks. In: International Conference on Web Search and Data Mining, Workshop on Query Understanding
  * He et al. (2017) He H, Balakrishnan A, Eric M, Liang P (2017) Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. arXiv preprint arXiv:170407130
  * He et al. (2016) He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
  * He and Glass (2019) He T, Glass J (2019) Negative training for neural dialogue response generation. arXiv preprint arXiv:190302134
  * He et al. (2020a) He W, Yang M, Yan R, Li C, Shen Y, Xu R (2020a) Amalgamating knowledge from two teachers for task-oriented dialogue system with adversarial training. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 3498–3507
  * He et al. (2020b) He X, Chen S, Ju Z, Dong X, Fang H, Wang S, Yang Y, Zeng J, Zhang R, Zhang R, et al. (2020b) Meddialog: Two large-scale medical dialogue datasets. arXiv e-prints pp arXiv–2004
  * Henderson et al. (2008) Henderson J, Lemon O, Georgila K (2008) Hybrid reinforcement/supervised learning of dialogue policies from fixed data sets. Computational Linguistics 34(4):487–511
  * Henderson (2015) Henderson M (2015) Machine learning for dialog state tracking: A review. In: Proceedings of The First International Workshop on Machine Learning in Spoken Language Processing
  * Henderson et al. (2013) Henderson M, Thomson B, Young S (2013) Deep neural network approach for the dialog state tracking challenge. In: Proceedings of the SIGDIAL 2013 Conference, pp 467–471
  * Henderson et al. (2014a) Henderson M, Thomson B, Williams JD (2014a) The second dialog state tracking challenge. In: Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), pp 263–272
  * Henderson et al. (2014b) Henderson M, Thomson B, Williams JD (2014b) The third dialog state tracking challenge. In: 2014 IEEE Spoken Language Technology Workshop (SLT), IEEE, pp 324–329
  * Henderson et al. (2019a) Henderson M, Budzianowski P, Casanueva I, Coope S, Gerz D, Kumar G, Mrkšić N, Spithourakis G, Su PH, Vulić I, et al. (2019a) A repository of conversational datasets. arXiv preprint arXiv:190406472
  * Henderson et al. (2019b) Henderson M, Vulić I, Gerz D, Casanueva I, Budzianowski P, Coope S, Spithourakis G, Wen TH, Mrkšić N, Su PH (2019b) Training neural response selection for task-oriented dialogue systems. arXiv preprint arXiv:190601543
  * Hochreiter and Schmidhuber (1997) Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural computation 9(8):1735–1780
  * Hochreiter et al. (2001) Hochreiter S, Bengio Y, Frasconi P, Schmidhuber J, et al. (2001) Gradient flow in recurrent nets: the difficulty of learning long-term dependencies
  * Hopfield (1982) Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences 79(8):2554–2558
  * Hori et al. (2015) Hori C, Hori T, Watanabe S, Hershey JR (2015) Context sensitive spoken language understanding using role dependent lstm layers. In: Proceedings of the NIPS 2015 Workshop on Machine Learning for Spoken Language Understanding and Interaction, Montreal, QC, Canada, vol 11
  * Hosseini-Asl et al. (2020) Hosseini-Asl E, McCann B, Wu CS, Yavuz S, Socher R (2020) A simple language model for task-oriented dialogue. arXiv preprint arXiv:200500796
  * Hu et al. (2020) Hu J, Yang Y, Chen C, Yu Z, et al. (2020) Sas: Dialogue state tracking via slot attention and slot information sharing. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 6366–6375
  * Hua et al. (2020) Hua Y, Li YF, Haffari G, Qi G, Wu T (2020) Few-shot complex knowledge base question answering via meta reinforcement learning. arXiv preprint arXiv:201015877
  * Huang et al. (2020a) Huang L, Ye Z, Qin J, Lin L, Liang X (2020a) Grade: Automatic graph-enhanced coherence metric for evaluating open-domain dialogue systems. arXiv preprint arXiv:201003994
  * Huang et al. (2018) Huang X, Jiang J, Zhao D, Feng Y, Hong Y (2018) Natural Language Processing and Chinese Computing: 6th CCF International Conference, NLPCC 2017, Dalian, China, November 8–12, 2017, Proceedings, vol 10619. Springer
  * Huang et al. (2020b) Huang X, Qi J, Sun Y, Zhang R (2020b) Semi-supervised dialogue policy learning via stochastic reward estimation. arXiv preprint arXiv:200504379
  * Huang et al. (2020c) Huang Y, Feng J, Hu M, Wu X, Du X, Ma S (2020c) Meta-reinforced multi-domain state generator for dialogue systems. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 7109–7118
  * Jaderberg et al. (2016) Jaderberg M, Mnih V, Czarnecki WM, Schaul T, Leibo JZ, Silver D, Kavukcuoglu K (2016) Reinforcement learning with unsupervised auxiliary tasks. arXiv preprint arXiv:161105397
  * Jang et al. (2017) Jang Y, Song Y, Yu Y, Kim Y, Kim G (2017) Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2758–2766
  * Jaques et al. (2020) Jaques N, Shen JH, Ghandeharioun A, Ferguson C, Lapedriza A, Jones N, Gu SS, Picard R (2020) Human-centric dialog training via offline reinforcement learning. arXiv preprint arXiv:201005848
  * Ji et al. (2020) Ji C, Zhou X, Zhang Y, Liu X, Sun C, Zhu C, Zhao T (2020) Cross copy network for dialogue generation. arXiv preprint arXiv:201011539
  * Ji et al. (2015) Ji G, He S, Xu L, Liu K, Zhao J (2015) Knowledge graph embedding via dynamic mapping matrix. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: Long papers), pp 687–696
  * Ji et al. (2021) Ji S, Pan S, Cambria E, Marttinen P, Yu PS (2021) A survey on knowledge graphs: Representation, acquisition and applications. IEEE Transactions on Neural Networks and Learning Systems 32
  * Jia et al. (2020) Jia Q, Liu Y, Ren S, Zhu KQ, Tang H (2020) Multi-turn response selection using dialogue dependency relations. arXiv preprint arXiv:201001502
  * Jordan (1986) Jordan M (1986) Serial order: a parallel distributed processing approach. technical report, june 1985-march 1986. Tech. rep., California Univ., San Diego, La Jolla (USA). Inst. for Cognitive Science
  * Jung et al. (2020) Jung J, Son B, Lyu S (2020) Attnio: Knowledge graph exploration with in-and-out attention flow for knowledge-grounded dialogue. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 3484–3497
  * Jurafsky (1997) Jurafsky D (1997) Switchboard swbd-damsl shallow-discourse-function annotation coders manual. Institute of Cognitive Science Technical Report
  * Kale and Rastogi (2020) Kale M, Rastogi A (2020) Template guided text generation for task-oriented dialogue. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics, Online, pp 6505–6520, DOI 10.18653/v1/2020.emnlp-main.527, URL https://www.aclweb.org/anthology/2020.emnlp-main.527
  * Kamezawa et al. (2020) Kamezawa H, Nishida N, Shimizu N, Miyazaki T, Nakayama H (2020) A visually-grounded first-person dialogue dataset with verbal and non-verbal responses. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 3299–3310
  * Kannan and Vinyals (2017) Kannan A, Vinyals O (2017) Adversarial evaluation of dialogue models. arXiv preprint arXiv:170108198
  * Kim et al. (2018) Kim A, Song HJ, Park SB, et al. (2018) A two-step neural dialog state tracker for task-oriented dialog processing. Computational intelligence and neuroscience 2018
  * Kim et al. (2020) Kim H, Kim B, Kim G (2020) Will i sound like me? improving persona consistency in dialogues through pragmatic self-consciousness. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 904–916
  * Kim et al. (2016) Kim S, D’Haro LF, Banchs RE, Williams JD, Henderson M, Yoshino K (2016) The fifth dialog state tracking challenge. In: 2016 IEEE Spoken Language Technology Workshop (SLT), IEEE, pp 511–517
  * Kim et al. (2017) Kim S, D’Haro LF, Banchs RE, Williams JD, Henderson M (2017) The fourth dialog state tracking challenge. In: Dialogues with Social Robots, Springer, pp 435–449
  * Kim et al. (2019) Kim S, Yang S, Kim G, Lee SW (2019) Efficient dialogue state tracking by selectively overwriting memory. arXiv preprint arXiv:191103906
  * Ko et al. (2020) Ko WJ, Ray A, Shen Y, Jin H (2020) Generating dialogue responses from a semantic latent space. arXiv preprint arXiv:201001658
  * Konda and Tsitsiklis (2000) Konda VR, Tsitsiklis JN (2000) Actor-critic algorithms. In: Advances in neural information processing systems, Citeseer, pp 1008–1014
  * Krizhevsky et al. (2012) Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25:1097–1105
  * Kummerfeld et al. (2018) Kummerfeld JK, Gouravajhala SR, Peper J, Athreya V, Gunasekara C, Ganhotra J, Patel SS, Polymenakos L, Lasecki WS (2018) A large-scale corpus for conversation disentanglement. arXiv preprint arXiv:181011118
  * Kundu et al. (2020) Kundu S, Lin Q, Ng HT (2020) Learning to identify follow-up questions in conversational question answering. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 959–968
  * Kurach et al. (2015) Kurach K, Andrychowicz M, Sutskever I (2015) Neural random-access machines. arXiv preprint arXiv:151106392
  * Larson et al. (2019) Larson S, Mahendran A, Peper JJ, Clarke C, Lee A, Hill P, Kummerfeld JK, Leach K, Laurenzano MA, Tang L, et al. (2019) An evaluation dataset for intent classification and out-of-scope prediction. arXiv preprint arXiv:190902027
  * Le and Hoi (2020) Le H, Hoi SC (2020) Video-grounded dialogues with pretrained generation language models. arXiv preprint arXiv:200615319
  * Le et al. (2019) Le H, Sahoo D, Chen NF, Hoi SC (2019) Multimodal transformer networks for end-to-end video-grounded dialogue systems. arXiv preprint arXiv:190701166
  * Le et al. (2020a) Le H, Sahoo D, Chen NF, Hoi SC (2020a) Bist: Bi-directional spatio-temporal reasoning for video-grounded dialogues. arXiv preprint arXiv:201010095
  * Le et al. (2020b) Le H, Sahoo D, Liu C, Chen NF, Hoi SC (2020b) Uniconv: A unified conversational neural architecture for multi-domain task-oriented dialogues. arXiv preprint arXiv:200414307
  * LeCun et al. (1998) LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11):2278–2324
  * Lee and Dernoncourt (2016) Lee JY, Dernoncourt F (2016) Sequential short-text classification with recurrent and convolutional neural networks. arXiv preprint arXiv:160303827
  * Lee (2013) Lee S (2013) Structured discriminative model for dialog state tracking. In: Proceedings of the SIGDIAL 2013 Conference, pp 442–451
  * Lee and Eskenazi (2013) Lee S, Eskenazi M (2013) Recipe for building robust spoken dialog state trackers: Dialog state tracking challenge system description. In: Proceedings of the SIGDIAL 2013 Conference, pp 414–422
  * Lee and Jha (2019) Lee S, Jha R (2019) Zero-shot adaptive transfer for conversational language understanding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 6642–6649
  * Lee et al. (2019) Lee S, Schulz H, Atkinson A, Gao J, Suleman K, El Asri L, Adada M, Huang M, Sharma S, Tay W, et al. (2019) Multi-domain task-completion dialog challenge. Dialog system technology challenges 8:9
  * Lei et al. (2018) Lei W, Jin X, Kan MY, Ren Z, He X, Yin D (2018) Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 1437–1447
  * Lemon and Pietquin (2007) Lemon O, Pietquin O (2007) Machine learning for spoken dialogue systems. In: Eighth Annual Conference of the International Speech Communication Association
  * Li et al. (2018a) Li C, Li L, Qi J (2018a) A self-attentive model with gate mechanism for spoken language understanding. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 3824–3833
  * Li et al. (2015) Li J, Galley M, Brockett C, Gao J, Dolan B (2015) A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:151003055
  * Li et al. (2016a) Li J, Miller AH, Chopra S, Ranzato M, Weston J (2016a) Dialogue learning with human-in-the-loop. arXiv preprint arXiv:161109823
  * Li et al. (2016b) Li J, Miller AH, Chopra S, Ranzato M, Weston J (2016b) Learning through dialogue interactions by asking questions. arXiv preprint arXiv:161204936
  * Li et al. (2016c) Li J, Monroe W, Jurafsky D (2016c) A simple, fast diverse decoding algorithm for neural generation. arXiv preprint arXiv:161108562
  * Li et al. (2016d) Li J, Monroe W, Ritter A, Galley M, Gao J, Jurafsky D (2016d) Deep reinforcement learning for dialogue generation. arXiv preprint arXiv:160601541
  * Li et al. (2017a) Li J, Monroe W, Shi T, Jean S, Ritter A, Jurafsky D (2017a) Adversarial learning for neural dialogue generation. arXiv preprint arXiv:170106547
  * Li et al. (2020a) Li L, Xu C, Wu W, Zhao Y, Zhao X, Tao C (2020a) Zero-resource knowledge-grounded dialogue generation. arXiv preprint arXiv:200812918
  * Li et al. (2019a) Li M, Roller S, Kulikov I, Welleck S, Boureau YL, Cho K, Weston J (2019a) Don’t say that! making inconsistent dialogue unlikely with unlikelihood training. arXiv preprint arXiv:191103860
  * Li et al. (2016e) Li X, Lipton ZC, Dhingra B, Li L, Gao J, Chen YN (2016e) A user simulator for task-completion dialogues. arXiv preprint arXiv:161205688
  * Li et al. (2017b) Li X, Chen YN, Li L, Gao J, Celikyilmaz A (2017b) End-to-end task-completion neural dialogue systems. arXiv preprint arXiv:170301008
  * Li et al. (2018b) Li X, Wang Y, Sun S, Panda S, Liu J, Gao J (2018b) Microsoft dialogue challenge: Building end-to-end task-completion dialogue systems. arXiv preprint arXiv:180711125
  * Li et al. (2019b) Li X, Yin F, Sun Z, Li X, Yuan A, Chai D, Zhou M, Li J (2019b) Entity-relation extraction as multi-turn question answering. arXiv preprint arXiv:190505529
  * Li (2017) Li Y (2017) Deep reinforcement learning: An overview. arXiv preprint arXiv:170107274
  * Li et al. (2017c) Li Y, Su H, Shen X, Li W, Cao Z, Niu S (2017c) Dailydialog: A manually labelled multi-turn dialogue dataset. arXiv preprint arXiv:171003957
  * Li et al. (2020b) Li Y, Yao K, Qin L, Che W, Li X, Liu T (2020b) Slot-consistent nlg for task-oriented dialogue systems with iterative rectification network. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 97–106
  * Li et al. (2019c) Li Z, Niu C, Meng F, Feng Y, Li Q, Zhou J (2019c) Incremental transformer with deliberation decoder for document grounded conversations. arXiv preprint arXiv:190708854
  * Liang et al. (2020) Liang W, Zou J, Yu Z (2020) Beyond user self-reported likert scale ratings: A comparison model for automatic dialog evaluation. arXiv preprint arXiv:200510716
  * Lin (2004) Lin CY (2004) Rouge: A package for automatic evaluation of summaries. In: Text summarization branches out, pp 74–81
  * Lin (1992) Lin LJ (1992) Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning 8(3-4):293–321
  * Lin et al. (2019) Lin X, Joty S, Jwalapuram P, Bari MS (2019) A unified linear-time framework for sentence-level discourse parsing. arXiv preprint arXiv:190505682
  * Lin et al. (2020a) Lin X, Jian W, He J, Wang T, Chu W (2020a) Generating informative conversational response using recurrent knowledge-interaction and knowledge-copy. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 41–52
  * Lin et al. (2015) Lin Y, Liu Z, Sun M, Liu Y, Zhu X (2015) Learning entity and relation embeddings for knowledge graph completion. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 29
  * Lin et al. (2020b) Lin Z, Cai D, Wang Y, Liu X, Zheng H, Shi S (2020b) The world is not binary: Learning to rank with grayscale data for dialogue response selection. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 9220–9229
  * Lin et al. (2020c) Lin Z, Madotto A, Winata GI, Fung P (2020c) Mintl: Minimalist transfer learning for task-oriented dialogue systems. arXiv preprint arXiv:200912005
  * Lipton et al. (2015) Lipton ZC, Berkowitz J, Elkan C (2015) A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:150600019
  * Liu and Lane (2016) Liu B, Lane I (2016) Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint arXiv:160901454
  * Liu and Lane (2017) Liu B, Lane I (2017) Iterative policy learning in end-to-end trainable task-oriented neural dialog models. In: 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), IEEE, pp 482–489
  * Liu et al. (2019a) Liu C, He S, Liu K, Zhao J (2019a) Vocabulary pyramid network: Multi-pass encoding and decoding with multi-level vocabularies for response generation. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 3774–3783
  * Liu et al. (2016) Liu CW, Lowe R, Serban IV, Noseworthy M, Charlin L, Pineau J (2016) How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:160308023
  * Liu et al. (2020a) Liu H, Wang W, Wang Y, Liu H, Liu Z, Tang J (2020a) Mitigating gender bias for neural dialogue generation with adversarial learning. arXiv preprint arXiv:200913028
  * Liu et al. (2020b) Liu Q, Chen Y, Chen B, Lou JG, Chen Z, Zhou B, Zhang D (2020b) You impress me: Dialogue generation via mutual persona perception. arXiv preprint arXiv:200405388
  * Liu and Lapata (2018) Liu Y, Lapata M (2018) Learning structured text representations. Transactions of the Association for Computational Linguistics 6:63–75
  * Liu et al. (2019b) Liu Y, Meng F, Zhang J, Zhou J, Chen Y, Xu J (2019b) Cm-net: A novel collaborative memory network for spoken language understanding. arXiv preprint arXiv:190906937
  * Liu et al. (2020c) Liu Z, Wang H, Niu ZY, Wu H, Che W, Liu T (2020c) Towards conversational recommendation over multi-type dialogs. arXiv preprint arXiv:200503954
  * Liu et al. (2020d) Liu Z, Winata GI, Lin Z, Xu P, Fung P (2020d) Attention-informed mixed-language training for zero-shot cross-lingual task-oriented dialogue systems. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 8433–8440
  * Lowe et al. (2015) Lowe R, Pow N, Serban I, Pineau J (2015) The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems. arXiv preprint arXiv:150608909
  * Lowe et al. (2017) Lowe R, Noseworthy M, Serban IV, Angelard-Gontier N, Bengio Y, Pineau J (2017) Towards an automatic turing test: Learning to evaluate dialogue responses. arXiv preprint arXiv:170807149
  * Lu et al. (2019) Lu J, Zhang C, Xie Z, Ling G, Zhou TC, Xu Z (2019) Constructing interpretive spatio-temporal features for multi-turn responses selection. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 44–50
  * Lubis et al. (2018) Lubis N, Sakti S, Yoshino K, Nakamura S (2018) Eliciting positive emotion through affect-sensitive dialogue response generation: A neural network approach. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32, no 1
  * Ma et al. (2019) Ma MD, Bowden KK, Wu J, Cui W, Walker M (2019) Implicit discourse relation identification for open-domain dialogues. arXiv preprint arXiv:190703975
  * Ma et al. (2020a) Ma W, Cui Y, Liu T, Wang D, Wang S, Hu G (2020a) Conversational word embedding for retrieval-based dialog system. arXiv preprint arXiv:200413249
  * Ma et al. (2020b) Ma Y, Nguyen KL, Xing FZ, Cambria E (2020b) A survey on empathetic dialogue systems. Information Fusion 64:50–70
  * Madotto et al. (2019) Madotto A, Lin Z, Wu CS, Fung P (2019) Personalizing dialogue agents via meta-learning. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 5454–5459
  * Majumder et al. (2020a) Majumder BP, Jhamtani H, Berg-Kirkpatrick T, McAuley J (2020a) Like hiking? you probably enjoy nature: Persona-grounded dialog with commonsense expansions. arXiv preprint arXiv:201003205
  * Majumder et al. (2020b) Majumder BP, Li S, Ni J, McAuley J (2020b) Interview: Large-scale modeling of media dialog with discourse patterns and knowledge grounding. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 8129–8141
  * Mallios and Bourbakis (2016) Mallios S, Bourbakis N (2016) A survey on human machine dialogue systems. In: 2016 7th international conference on information, intelligence, systems & applications (iisa), IEEE, pp 1–7
  * Manuvina-kurike et al. (2018) Manuvina-kurike R, Brixey J, Bui T, Chang W, Artstein R, Georgila K (2018) Dialedit: Annotations for spoken conversational image editing. In: Proceedings 14th Joint ACL-ISO Workshop on Interoperable Semantic Annotation, pp 1–9
  * Mao et al. (2020) Mao HH, Li S, McAuley J, Cottrell G (2020) Speech recognition and multi-speaker diarization of long conversations. arXiv preprint arXiv:200508072
  * Mehri and Eskenazi (2020) Mehri S, Eskenazi M (2020) Usr: An unsupervised and reference free evaluation metric for dialog generation. arXiv preprint arXiv:200500456
  * Mehri et al. (2019) Mehri S, Razumovskaia E, Zhao T, Eskenazi M (2019) Pretraining methods for dialog context representation learning. arXiv preprint arXiv:190600414
  * Mesgar et al. (2019) Mesgar M, Bücker S, Gurevych I (2019) Dialogue coherence assessment without explicit dialogue act labels. arXiv preprint arXiv:190808486
  * Mesnil et al. (2013) Mesnil G, He X, Deng L, Bengio Y (2013) Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding. In: Interspeech, pp 3771–3775
  * Mesnil et al. (2014) Mesnil G, Dauphin Y, Yao K, Bengio Y, Deng L, Hakkani-Tur D, He X, Heck L, Tur G, Yu D, et al. (2014) Using recurrent neural networks for slot filling in spoken language understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing 23(3):530–539
  * Miller et al. (2016) Miller A, Fisch A, Dodge J, Karimi AH, Bordes A, Weston J (2016) Key-value memory networks for directly reading documents. arXiv preprint arXiv:160603126
  * Mirowski et al. (2016) Mirowski P, Pascanu R, Viola F, Soyer H, Ballard AJ, Banino A, Denil M, Goroshin R, Sifre L, Kavukcuoglu K, et al. (2016) Learning to navigate in complex environments. arXiv preprint arXiv:161103673
  * Mnih et al. (2015) Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al. (2015) Human-level control through deep reinforcement learning. nature 518(7540):529–533
  * Mnih et al. (2016) Mnih V, Badia AP, Mirza M, Graves A, Lillicrap T, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. In: International conference on machine learning, PMLR, pp 1928–1937
  * Mo et al. (2018) Mo K, Zhang Y, Li S, Li J, Yang Q (2018) Personalizing a dialogue system with transfer reinforcement learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32, no 1
  * Moghe et al. (2018) Moghe N, Arora S, Banerjee S, Khapra MM (2018) Towards exploiting background knowledge for building conversation systems. arXiv preprint arXiv:180908205
  * Mohammad et al. (2018) Mohammad S, Bravo-Marquez F, Salameh M, Kiritchenko S (2018) Semeval-2018 task 1: Affect in tweets. In: Proceedings of the 12th international workshop on semantic evaluation, pp 1–17
  * Moon et al. (2019) Moon S, Shah P, Kumar A, Subba R (2019) Opendialkg: Explainable conversational reasoning with attention-based walks over knowledge graphs. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 845–854
  * Mostafazadeh et al. (2017) Mostafazadeh N, Brockett C, Dolan B, Galley M, Gao J, Spithourakis GP, Vanderwende L (2017) Image-grounded conversations: Multimodal context for natural question and response generation. arXiv preprint arXiv:170108251
  * Mrkšić et al. (2015) Mrkšić N, Séaghdha DO, Thomson B, Gašić M, Su PH, Vandyke D, Wen TH, Young S (2015) Multi-domain dialog state tracking using recurrent neural networks. arXiv preprint arXiv:150607190
  * Mrkšić et al. (2016) Mrkšić N, Séaghdha DO, Wen TH, Thomson B, Young S (2016) Neural belief tracker: Data-driven dialogue state tracking. arXiv preprint arXiv:160603777
  * Nakov et al. (2019) Nakov P, Màrquez L, Magdy W, Moschitti A, Glass J, Randeree B (2019) Semeval-2015 task 3: Answer selection in community question answering. arXiv preprint arXiv:191111403
  * Nickel et al. (2016) Nickel M, Rosasco L, Poggio T (2016) Holographic embeddings of knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 30, no 1
  * Niu et al. (2019) Niu P, Chen Z, Song M, et al. (2019) A novel bi-directional interrelated model for joint intent detection and slot filling. arXiv preprint arXiv:190700390
  * Novikova et al. (2017) Novikova J, Dušek O, Rieser V (2017) The e2e dataset: New challenges for end-to-end generation. arXiv preprint arXiv:170609254
  * Oraby et al. (2019) Oraby S, Harrison V, Ebrahimi A, Walker M (2019) Curate and generate: A corpus and method for joint control of semantics and style in neural nlg. arXiv preprint arXiv:190601334
  * Oriol et al. (2015) Oriol V, Meire F, Navdeep J (2015) Pointer networks. Advances in neural information processing systems 28:2692–2700
  * Ouyang et al. (2020) Ouyang Y, Chen M, Dai X, Zhao Y, Huang S, Jiajun C (2020) Dialogue state tracking with explicit slot connection modeling. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 34–40
  * Panayotov et al. (2015) Panayotov V, Chen G, Povey D, Khudanpur S (2015) Librispeech: an asr corpus based on public domain audio books. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 5206–5210
  * Pang et al. (2020) Pang B, Nijkamp E, Han W, Zhou L, Liu Y, Tu K (2020) Towards holistic and automatic evaluation of open-domain dialogue generation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 3619–3629
  * Papineni et al. (2002) Papineni K, Roukos S, Ward T, Zhu WJ (2002) Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp 311–318
  * Parikh et al. (2016) Parikh AP, Täckström O, Das D, Uszkoreit J (2016) A decomposable attention model for natural language inference. arXiv preprint arXiv:160601933
  * Pascanu et al. (2013) Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: International conference on machine learning, PMLR, pp 1310–1318
  * Peng et al. (2017) Peng B, Li X, Li L, Gao J, Celikyilmaz A, Lee S, Wong KF (2017) Composite task-completion dialogue policy learning via hierarchical deep reinforcement learning. arXiv preprint arXiv:170403084
  * Peters et al. (2018) Peters ME, Neumann M, Iyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. arXiv preprint arXiv:180205365
  * Pfau and Vinyals (2016) Pfau D, Vinyals O (2016) Connecting generative adversarial networks and actor-critic methods. arXiv preprint arXiv:161001945
  * Poria et al. (2019) Poria S, Hazarika D, Majumder N, Naik G, Cambria E, Mihalcea R (2019) MELD: A multimodal multi-party dataset for emotion recognition in conversations. In: ACL, pp 527–536
  * Prasad et al. (2008) Prasad R, Dinesh N, Lee A, Miltsakaki E, Robaldo L, Joshi AK, Webber BL (2008) The penn discourse treebank 2.0. In: LREC, Citeseer
  * Puterman (2014) Puterman ML (2014) Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons
  * Qian and Yu (2019) Qian K, Yu Z (2019) Domain adaptive dialog generation via meta learning. arXiv preprint arXiv:190603520
  * Qin et al. (2019a) Qin L, Che W, Li Y, Wen H, Liu T (2019a) A stack-propagation framework with token-level intent detection for spoken language understanding. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computational Linguistics, Hong Kong, China, pp 2078–2087, DOI 10.18653/v1/D19-1214, URL https://www.aclweb.org/anthology/D19-1214
  * Qin et al. (2019b) Qin L, Liu Y, Che W, Wen H, Li Y, Liu T (2019b) Entity-consistent end-to-end task-oriented dialogue system with KB retriever. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Association for Computational Linguistics, Hong Kong, China, pp 133–142, DOI 10.18653/v1/D19-1013, URL https://www.aclweb.org/anthology/D19-1013
  * Qin et al. (2020a) Qin L, Ni M, Zhang Y, Che W (2020a) Cosda-ml: Multi-lingual code-switching data augmentation for zero-shot cross-lingual nlp. In: Bessiere C (ed) Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, International Joint Conferences on Artificial Intelligence Organization, pp 3853–3860, DOI 10.24963/ijcai.2020/533, URL https://doi.org/10.24963/ijcai.2020/533, main track
  * Qin et al. (2020b) Qin L, Xu X, Che W, Liu T (2020b) Towards fine-grained transfer: An adaptive graph-interactive framework for joint multiple intent detection and slot filling. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp 1807–1816
  * Qin et al. (2020c) Qin L, Xu X, Che W, Zhang Y, Liu T (2020c) Dynamic fusion network for multi-domain end-to-end task-oriented dialog. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Association for Computational Linguistics, Online, pp 6344–6354, DOI 10.18653/v1/2020.acl-main.565, URL https://www.aclweb.org/anthology/2020.acl-main.565
  * Qin et al. (2021a) Qin L, Liu T, Che W, Kang B, Zhao S, Liu T (2021a) A co-interactive transformer for joint slot filling and intent detection. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp 8193–8197
  * Qin et al. (2021b) Qin L, Xie T, Che W, Liu T (2021b) A survey on spoken language understanding: Recent advances and new frontiers. arXiv preprint arXiv:210303095
  * Qiu et al. (2019) Qiu L, Li J, Bi W, Zhao D, Yan R (2019) Are training samples correlated? learning to generate dialogue responses with multiple references. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 3826–3835
  * Qiu et al. (2020) Qiu L, Zhao Y, Shi W, Liang Y, Shi F, Yuan T, Yu Z, Zhu SC (2020) Structured attention for unsupervised dialogue structure induction. arXiv preprint arXiv:200908552
  * Qiu et al. (2017) Qiu M, Li FL, Wang S, Gao X, Chen Y, Zhao W, Chen H, Huang J, Chu W (2017) Alime chat: A sequence to sequence and rerank based chatbot engine. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp 498–503
  * Quan and Xiong (2020) Quan J, Xiong D (2020) Modeling long context for task-oriented dialogue state generation. arXiv preprint arXiv:200414080
  * Quan et al. (2020) Quan J, Zhang S, Cao Q, Li Z, Xiong D (2020) Risawoz: A large-scale multi-domain wizard-of-oz dataset with rich semantic annotations for task-oriented dialogue modeling. arXiv preprint arXiv:201008738
  * Rajpurkar et al. (2018) Rajpurkar P, Jia R, Liang P (2018) Know what you don’t know: Unanswerable questions for squad. arXiv preprint arXiv:180603822