# A Cooperative Memory Network for Personalized Task-oriented Dialogue Systems
with Incomplete User Profiles

Jiahuan Pei  University of AmsterdamAmsterdamThe Netherlands<EMAIL_ADDRESS>,
Pengjie Ren  Shandong UniversityQingdaoChina<EMAIL_ADDRESS>and
Maarten de Rijke  University of Amsterdam & Ahold DelhaizeAmsterdamThe
Netherlands<EMAIL_ADDRESS>
(2021)

###### Abstract.

There is increasing interest in developing personalized Task-oriented Dialogue
Systems. Previous work on personalized TDSs often assumes that complete user
profiles are available for most or even all users. This is unrealistic because
(1) not everyone is willing to expose their profiles due to privacy concerns;
and  (2) rich user profiles may involve a large number of attributes (e.g.,
gender, age, tastes, …).  In this paper, we study personalized TDSs without
assuming that user profiles are complete. We propose a Cooperative Memory
Network (CoMemNN) that has a novel mechanism to gradually enrich user profiles
as dialogues progress and to simultaneously improve response selection based
on the enriched profiles. CoMemNN consists of two core modules: User Profile
Enrichment (UPE) and Dialogue Response Selection (DRS). The former enriches
incomplete user profiles by utilizing collaborative information from neighbor
users as well as current dialogues. The latter uses the enriched profiles to
update the current user query so as to encode more useful information, based
on which a personalized response to a user request is selected.

We conduct extensive experiments on the personalized bAbI dialogue benchmark
datasets. We find that CoMemNN is able to enrich user profiles effectively,
which results in an improvement of 3.06% in terms of response selection
accuracy compared to state-of-the-art methods. We also test the robustness of
CoMemNN against incompleteness of user profiles by randomly discarding
attribute values from user profiles. Even when discarding 50% of the attribute
values, CoMemNN is able to match the performance of the best performing
baseline without discarding user profiles, showing the robustness of CoMemNN.

Dialogue systems, personalization, neural networks, collaborative agents

††journalyear: 2021††copyright: iw3c2w3††conference: Proceedings of the Web
Conference 2021; April 19–23, 2021; Ljubljana, Slovenia††booktitle:
Proceedings of the Web Conference 2021 (WWW ’21), April 19–23, 2021,
Ljubljana, Slovenia††doi: 10.1145/3442381.3449843††isbn:
978-1-4503-8312-7/21/04††ccs: Computing methodologies Discourse, dialogue and
pragmatics††ccs: Information systems Personalization††submissionid: fp5177732

##  1\. Introduction

The use of Task-oriented Dialogue Systems (TDSs) is becoming increasingly
widespread. Unlike Open-ended Dialogue Systems (Li et al., 2016; Zhang et al.,
2018), TDSs are meant to help users achieve specific goals during multiple-
turn dialogues (Chen et al., 2017). Applications include booking restaurants,
planning trips, grocery shopping, customer service (e.g., Young et al., 2013;
Bordes and Weston, 2017; Williams et al., 2017; Madotto et al., 2018; Pei et
al., 2019a, 2020).

Considerable progress has been made in improving the performance of TDSs
(e.g., Bordes and Weston, 2017; Liu and Perez, 2017; Wu et al., 2018;
Henderson et al., 2019; Pei et al., 2019b; Lu et al., 2019; Lin et al., 2020).
Human-human dialogues reflect diverse personalized preferences in terms of,
e.g., modes of expression habits (Ficler and Goldberg, 2017; Zeng et al.,
2019), individual needs and related to specific goals (Joshi et al., 2017; Mo
et al., 2018; Luo et al., 2019). Recent work has begun to explore how to
improve the user experience by personalizing TDSs in similar ways. Several
personalized TDS models have been proposed and have achieved good performance
(Joshi et al., 2017; Luo et al., 2019; Zhang et al., 2020). Personalized TDS
models use user profiles in order to be able to capture, and optimize for,
users’ personal preferences. Those user profiles may not always be available
or complete. While profiles may be obtained by asking users to fill in
personal profiles with all predefined attributes (Joshi et al., 2017; Luo et
al., 2019; Zhang et al., 2020), more often than not, they are incomplete and
have missing values for some of the attributes of interest:  (1) not all users
are willing to expose their profiles due to privacy concerns (Wang et al.,
2015); Tigunova et al. (2019) have shown that users rarely reveal their
personal information in dialogues explicitly; and  (2) user profiles may
involve many attributes (such as, e.g., gender, age, tastes), which makes it
hard to collect values for all of them. For example, even if we know a user’s
favorite food is “fish and chips,” this does not mean the user does not like
“hamburgers.”

In this paper, we study the problem of personalized TDSs with _incomplete user
profiles_. This problem comes with two key challenges:  (1) how to infer
missing attribute values of incomplete user profiles; and  (2) how to use
enriched profiles so as to enhance personalized TDSs.  There have been
previous attempts to extract user profiles from open-ended dialogues (Tigunova
et al., 2019; Tigunova, 2020; Wu et al., 2020; Li et al., 2020, 2014) but to
the best of our knowledge the problem of inferring and using missing attribute
values has not been studied yet in the context of TDSs.

We address the problem of personalized TDSs with _incomplete user profiles_ by
proposing an end-to-end Cooperative Memory Network (CoMemNN) in which profiles
and dialogues are used to mutually improve each other. See Figure 1 for an
intuitive sketch.

Figure 1. Cooperative interaction between user profiles and dialogues.

The intuition behind CoMemNN is that user profiles can be gradually improved
(i.e., missing values can be added) by leveraging useful information from each
dialogue turn, and, simultaneously, the performance of DRS can be improved
based on enriched profiles for later turns. For example, when user $u_{1}$
produces the utterance “Does it have ‘decent’ french fries?” and the user
reveals his like of “french fries,” the attribute ‘favorite food’ in his user
profile can be enriched with the value of “french fries.” In addition, we want
to consider collaborative information from similar users, assuming that
similar users have similar preferences as reflected in their user profiles.
For example, a young male non-vegetarian who is a big fan of “pizza” might
also love “fish and chips” if there are several users with similar profiles
stating “fish and chips” as their favorite food. In turn, knowledge of these
preferences can affect the choice of the response selected by a TDS in case
there are multiple candidate responses. In other words, users with similar
profiles may expect the same or a similar response given a certain dialogue
context (Luo et al., 2019). CoMemNN operationalizes these intuitions with two
key modules: User Profile Enrichment (UPE) and Dialogue Response Selection
(DRS). The former enriches incomplete user profiles by utilizing useful
information from the current dialogue as well as collaborative information
from similar users. The latter uses the enriched profiles to update the query
representing all requested information, based on which a personalized response
is selected to reply to user requests.

To verify the effectiveness of CoMemNN, we conduct extensive experiments on
the personalized bAbI dialogue (PbAbI) benchmark dataset, which comes in two
flavors, a small version which has 1,000 dialogues, and a large version, which
has 12,000 dialogues. First, we find that CoMemNN improves over the best
baseline by 3.06%/2.80% on the small/large dataset, respectively, when using
all available user profiles. Second, to assess the performance of CoMemNN in
the presence of incomplete user profiles, we randomly discard values of
attributes with varying probabilities and find that even when it discards 50%
of the attribute values, the performance of CoMemNN matches the performance of
the best performing baseline without discarding user profiles. In contrast,
the best performing baseline decreases 2.12%/1.97% in performance on the
small/large dataset with the same amount of discarded values.

The main contributions of this paper are as follows:

  * •

We consider the task personalized TDSs with incomplete user profiles, which
has not been investigated so far, to the best of our knowledge.

  * •

We devise a CoMemNN model with dedicated modules to gradually enrich user
profiles as a dialogue progresses and to improve response selection based on
enriched profiles at the same time.

  * •

We carry out extensive experiments to show the robustness of CoMemNN in the
presence of incomplete user profiles.

##  2\. Related work

In this section, we briefly present an overview of related work on
personalized Open-ended Dialogue Systems (ODSs) and personalized Task-oriented
Dialogue Systems (TDSs).

###  2.1. Personalized Open-ended Dialogue Systems

Previous studies on personalized ODSs mainly fuse unstructured persona
information (Zhang et al., 2018; Mazare et al., 2018). Li et al. (2016) first
attempt to incorporate a persona into the Seq2Seq framework (Sutskever et al.,
2014) to generate personalized responses. Ficler and Goldberg (2017) apply an
RNN language model conditioned on a persona to control response generation
with linguistic style. Zhang et al. (2018) find that selection models based on
Memory Networks (Sukhbaatar et al., 2015) are more promising than recurrent
generation models based on Seq2Seq (Sutskever et al., 2014). Mazare et al.
(2018) develop a response selection model based on MemNN and model persona to
improve the performance of an ODS. Song et al. (2019) explore how to generate
diverse personalized responses using a variational autoencoder conditioned on
a persona memory. Liu et al. (2020) make use of persona interaction between
two interlocutors. Xu et al. (2020) further exploit topical information to
extend persona.

Prior attempts to address data sparsity problems in order to enhance
personalized ODSs have considered pretraining (Herzig et al., 2017; Zheng et
al., 2020), sketch generation and filling (Shum et al., 2019), multiple-stage
decoding (Song et al., 2020), multi-task learning (Luan et al., 2017),
transfer learning (Yang et al., 2017; Zhang et al., 2019; Wolf et al., 2019),
and meta-learning (Madotto et al., 2019). Only few studies have explored
structured user profiles for ODSs (Qian et al., 2018; Zheng et al., 2019; Zhou
et al., 2020).

Most of the methods listed above focus on unstructured persona information
while we target structured user profiles. Importantly, they focus on ODSs, so
they cannot be applied to TDSs directly.

###  2.2. Personalized Task-oriented Dialogue Systems

Unlike ODSs, personalized TDSs have not been investigated extensively so far.
Joshi et al. (2017) release the first and, so far, only benchmark dataset for
personalized TDSs, to the best of our knowledge. They propose a memory network
based model, MemNN, to encode user profiles and conduct personalized response
selection. They also propose an extension of MemNN, Split MemNN, which splits
a memory into a profile memory followed by a dialogue memory. Zhang et al.
(2020) introduce Retrieval MemNN by incorporating a retrieval module into
memory network, which enhances the performance by retrieving the relevant
responses from other users. Luo et al. (2019) present Personalized MemNN which
learns distributed embeddings for user profiles, dialogue history, and the
dialogue history from users with the same gender and age, and shows better
performance by using the idea user bias towards Knowledge Base (KB) entries
over candidate responses. Mo et al. (2018) introduce a transfer reinforce
learning paradigm to alleviate data scarcity, which uses a collection of
multiple users as a source domain and an individual user as a target domain.

The methods above all assume that complete user profiles can be obtained by
urging users to fill in all blanks in user profiles, which is unrealistic in
practice. Thus, it remains unexplored how the methods above perform when
incomplete user profiles are provided, and whether we can bridge the gap in
performance if their performance is negatively affected. An alternative is to
first infer missing user profiles, e.g., by mining query logs or previous
conversations (Li et al., 2014; Tigunova et al., 2019; Tigunova, 2020; Li et
al., 2020), and then apply the model with the above methods. But to do so, we
need to train a model to infer missing user profiles asynchronously. Besides,
it will likely bring cumulative errors to downstream TDS tasks. Instead, we
propose to enrich user profiles _and_ achieve a TDS simultaneously with an
end-to-end model.

##  3\. Method

###  3.1. Task

In this work, we follow previous studies and model a personalized TDS as a
response selection task, which selects a response from predefined candidates
given a dialogue context (Joshi et al., 2017; Luo et al., 2019; Zhang et al.,
2020; Eric et al., 2017; Wen et al., 2017; Rajendran et al., 2018; Pei et al.,
2019b). Table 1 summarizes the main notation used in this paper.

Table 1. Summary of main notation used in the paper. $X^{u}_{t}$ | User utterance at turn $t$.
---|---
$X^{s}_{t}$ | System response at turn $t$.
$D_{t}$ | Dialogue history at turn $t$.
$\mathbf{h}_{t}$ | Hidden representation of $D_{t}$.
$u$ |  | A user profile in the form of $\\{(a_{i},v_{i})\\}_{i=1}^{m}$, $v_{i}$ is a candidate
---
value of $i$-th attribute $a_{i}$.
$\mathbf{p}$ | One-hot representation of $u$.
$\mathbf{q}_{t}$ |  | A query representation that represents the user’s current
---
request at turn $t$.
$\mathbf{M}_{t}^{P}$ |  | Profile memory that contains user profile presentations of $u$
---
and his/her neighbors at turn $t$.
$\mathbf{M}_{t}^{D}$ |  | Dialogue memory that contains dialogue history presentation
---
of $u$ and his/her neighbors at turn $t$.

Given a dialogue context $(u,D_{t},X^{u}_{t})$ at the $t$-th dialogue turn,
our goal is to select an appropriate response $y_{t}=X^{s}_{t}$ from candidate
responses $Y=\\{X_{j}^{s}\\}|_{j=1}^{|Y|}$. Here, $u$ is the user profile,
which consists of $m$ attribute-value pairs $\\{(a_{i},v_{i})\\}_{i=1}^{m}$,
where $a_{i}$ is the $i$-th attribute and $v_{i}$ is a candidate value of
$a_{i}$. For example, in Fig. 1, the user profile is denoted as {(Gender,
Male), (Age, Young), (Dietary, Non-vegetarian), (Favorite food, Fish and
Chips)}. $D_{t}=X_{1:t-1}$ is the dialogue history. Similar to (Joshi et al.,
2017; Luo et al., 2019; Zhang et al., 2020), $D_{t}$ is represented as a
sequence of words that are aggregated from historical utterances
$[X^{u}_{1},X^{s}_{1},\ldots,X^{u}_{t-1},X^{s}_{t-1}]$, alternating between
the user $u$ or system $s$. $X^{u}_{t}$ denotes the current user utterance,
representing the user’s current request.

###  3.2. Overview of CoMemNN

An overview of the proposed architecture, CoMemNN, is shown in Fig. 2.

Figure 2. An overview of the CoMemNN architecture, which consists of two
cooperative modules: UPE and DRS. Figure 3. An overview of the dynamic
pipeline of the CoMemNN model. The UPE modules captures the interaction
between user profiles and dialogues by three submodules: MI, MU and MR. The
DRS module and the UPE module cooperate so as to select better responses.
Section 3 contains a walkthrough of the model.

A key aspect of the architecture is that it aims to capture all useful
information from the given dialogue context $(u,D_{t},X^{u}_{t})$, based on
which we learn a query representation $\mathbf{q}_{t}$ to represent the user’s
current request. $\mathbf{q}_{t}$ is usually initialized with the current user
utterance $X^{u}_{t}$ (Joshi et al., 2017; Luo et al., 2019; Zhang et al.,
2020). Then, $\mathbf{q}_{t}$ is updated by the User Profile Enrichment (UPE)
module by incorporating dialogue and personal information from dialogues and
user profiles, respectively. Specifically, UPE captures the interaction
between user profiles and dialogues with three submodules: Memory
Initialization (MI), Memory Updating (MU) and Memory Reading (MR). MI searches
neighbors of the current user to initialize the profile memory
$\mathbf{M}_{t}^{P}$, which contains profiles from both the current user and
his/her neighbors. MI also initializes the dialogue memory
$\mathbf{M}_{t}^{D}$ with the dialogue history of both the current user and
his/her neighbors, each of which is represented by addressing dialogue
historical utterance representations with $\mathbf{q}_{t}$. MU updates the
profile memory $\mathbf{M}_{t}^{P}$ and the dialogue memory
$\mathbf{M}_{t}^{D}$ by considering their interaction, after which the user
profiles are enriched by inferring missing values based on the dialogue and
personal information from the current user and his/her neighbors. Afterwards,
MR updates the query representation $\mathbf{q}_{t}$ by reading from the
enriched profile memory as well as dialogue memory. Finally, the Dialogue
Response Selection (DRS) module uses the updated query to match candidate
responses so as to select an appropriate response. Next, we introduce each of
the modules MI, MU and MR, one by one.

###  3.3.  Memory Initialization (MI)

#### Profile Memory Initialization.

To model user-profile relations, we initialize the profile memory as:
$\mathbf{M}_{t}^{P}=[\Psi(u_{1}),\dots,\Psi(u_{k})]\in\mathbb{R}^{k\times d}$,
where $u_{1}$ is the Current Profile (CP) from the current user. The others
are Neighbor Profiles from neighbor users. For each user profile, the $i$-th
attribute can be represented as an one-hot vector
$\mathbf{\tilde{p}}_{i}\in\mathbb{R}^{C(p_{i})}$, where there are $C(a_{i})$
candidate values for $p_{i}$. Then, each user profile can be initialized as an
one-hot vector
$\mathbf{p}=\text{Concat}(\mathbf{\tilde{p}}_{1},\dots,\mathbf{\tilde{p}}_{m})\in\mathbb{R}^{n}(n=\sum_{i=1}^{m}(C(p_{i}))$,
which is the concatenation of one-hot representations of attributes. $k$ is
the number of users, $d$ is the embedding dimension, and $\Psi$ is a linear
transformation function. Given any user profile $u$, we find his/her $(k-1)$
nearest neighbors based on dot product similarity.

#### Dialogue Memory Initialization.

To model user-dialogue relations, we initialize a dialogue memory
$\mathbf{M}_{t}^{D}=[\mathbf{h}_{t}^{1},\dots,\mathbf{h}_{t}^{k}]\in\mathbb{R}^{k\times
d}$, where $\mathbf{h}_{t}^{1}$ is the representation of the Current Dialogue
(CD) from the current user. The others are the Neighbor Dialogues from
neighbor users. For each user, the dialogue history can be computed as:

(1)
$\begin{split}\mathbf{h}_{t}&=\sum_{i=1}^{2(t-1)}\lambda_{t}^{i}\mathbf{H}_{t}^{i}\in\mathbb{R}^{d}\\\
\lambda_{t}^{i}&=(\mathbf{\tilde{q}}_{t})^{T}\cdot\mathbf{H}_{t}^{i}\in\mathbb{R}^{1},\end{split}$

where we use the updated query $\mathbf{\tilde{q}}_{t}$ to address the
aggregated dialogue history $\mathbf{H}_{t}$, the addressing weight
$\lambda_{t}^{i}$ is computed by the dot product of query
$\mathbf{\tilde{q}}_{t}$ and the $i$-th utterance representation
$\mathbf{H}_{t}^{i}$.

Following (Dodge et al., 2015; Luo et al., 2019), we represent each utterance
as a bag-of-words using the embedding matrix $\mathbf{E}\in\mathbb{R}^{d\times
V}$, where $d$ is the embedding dimension, $V$ is the vocabulary size,
$\Phi(\cdot)$ maps the utterance to a bag of dimension $V$. At the beginning
of turn $t$, the updated query $\mathbf{\tilde{q}}_{t}$ is initialized as:

(2) $\mathbf{\tilde{q}}_{t}=\mathbf{E}\Phi(X_{t}^{u})\in\mathbb{R}^{d}.$

Similarly, the aggregated dialogue history $\mathbf{H}_{t}$ of the current
user $u_{1}$ can be embedded as:

(3) $\begin{split}\mathbf{H}_{t}&={}\\\
&[\mathbf{E}\Phi(X^{u}_{1}),\mathbf{E}\Phi(X^{s}_{1}),\dots,\mathbf{E}\Phi(X^{u}_{t-1}),\mathbf{E}\Phi(X^{s}_{t-1})]\in\mathbb{R}^{2(t-1)\times
d}.\end{split}$

###  3.4.  Memory Updating (MU)

#### Dialogue Memory Updating.

To obtain an intermediate dialogue memory $\mathbf{\tilde{M}}_{t}^{D}$, we
update the $i$-th dialogue memory slot $\mathbf{\tilde{M}}_{t}^{D}[:,i]$ using
the newest updated query $\mathbf{\tilde{q}}_{t}$ to address initial dialogue
memory $\mathbf{M}_{t}^{D}$ as:

(4)
$\begin{split}\mathbf{\tilde{M}}_{t}^{D}[:,i]&=\sum_{j=1}^{k}\beta_{t}^{j}\mathbf{M}_{t}^{D}[:,j]\in\mathbb{R}^{d}\\\
\beta_{t}^{j}&=(\mathbf{\tilde{q}}_{t})^{T}\cdot\mathbf{M}_{t}^{D}[:,j]\in\mathbb{R}^{1}.\end{split}$

Next, the initial dialogue memory is updated by assigning
$\mathbf{M}_{t}^{D}=\mathbf{\tilde{M}}_{t}^{D}$. As the dialogue evolves, the
profile memory will gradually improve the dialogue memory because
$\mathbf{\tilde{q}}_{t}$ contains information from the previous profile
memory, so addressing with $\mathbf{\tilde{q}}_{t}$ links profile-dialogue
relations to the dialogue memory.

#### Profile Memory Updating.

Similarly, we can obtain an intermediate profile memory
$\mathbf{\tilde{M}}_{t}^{P}$ with the following steps:

(5)
$\begin{split}\mathbf{\tilde{M}}_{t}^{P}[:,i]&=\sum_{j=1}^{k}\alpha_{t}^{j}\mathbf{M}_{t}^{P}[:,j]\in\mathbb{R}^{d}\\\
\alpha_{t}^{j}&=(\mathbf{M}_{t}^{P}[:,i])^{T}\cdot\mathbf{M}_{t}^{P}[:,j]\in\mathbb{R}^{1}.\end{split}$

Next, the profile memory slot $\mathbf{M}_{t}^{P}[:,i]$ is updated by a
function $\Gamma(\cdot)$ using the intermediate profile memory slot
$\mathbf{\tilde{M}}_{t}^{P}[:,i]$ and the newest updated dialogue memory slot
$\mathbf{\tilde{M}}_{t}^{D}[:,i]$:

(6)
$\mathbf{M}_{t}^{P}[:,i]=\Gamma(\mathbf{\tilde{M}}_{t}^{P}[:,i],\mathbf{\tilde{M}}_{t}^{D}[:,i])\in\mathbb{R}^{d},$

where $\Gamma(\cdot)$ is a mapping function that is implemented by a Multiple
Layer Perceptron (MLP) in this work. In this process, the dialogue memory
helps to improve the profile memory because $\Gamma(\cdot)$ links dialogue-
profile relations to the profile memory.

###  3.5.  Memory Reading (MR)

#### Dialogue Memory Reading.

Since the first memory slot corresponds to the current user, we compute
$\mathbf{m}_{t}^{D}$ by hard addressing and use it to update the query
$\mathbf{\tilde{q}}_{t}$ as follows:

(7)
$\begin{split}\mathbf{\tilde{q}}_{t}&=\mathbf{\tilde{q}}_{t}+\mathbf{m}_{t}^{D}\in\mathbb{R}^{d}\\\
\mathbf{m}_{t}^{D}&=\mathbf{\tilde{M}}_{t}^{D}[:,1]\in\mathbb{R}^{d}.\end{split}$

#### Profile Memory Reading.

Similarly, we obtain $\mathbf{m}_{t}^{P}$ by hard addressing and use it to
update the query $\mathbf{\tilde{q}}_{t}$ as follows:

(8)
$\begin{split}\mathbf{\tilde{q}}_{t}&=\mathbf{\tilde{q}}_{t}+\mathbf{m}_{t}^{P}\in\mathbb{R}^{d}\\\
\mathbf{m}_{t}^{P}&=\mathbf{M}_{t}^{P}[:,1]\in\mathbb{R}^{d}.\end{split}$

###  3.6. Dialogue Response Selection

We use the latest updated query $\mathbf{\tilde{q}}_{t}$ to match with
candidate dialogue responses and the predicted response distribution is
computed as follows:

(9)
$\begin{split}\mathbf{\tilde{y}}_{t}&=\mathrm{Softmax}(\mathbf{\tilde{q}}_{t}^{T}\mathbf{r}_{1}+\mathbf{b}_{1},\dots,\mathbf{\tilde{q}}_{t}^{T}\mathbf{r}_{|Y|}+\mathbf{b}_{|Y|})\in\mathbb{R}^{|Y|}\\\
\mathbf{b}_{j}&=\begin{cases}\mathbf{f}_{i}\in\mathbb{R}^{1}&\text{if
$\mathbf{r}_{j}$ mentions $i$-{th} attribute of a \acs{KB} entry}\\\
0&\text{otherwise}\end{cases}\\\
\mathbf{f}&=\operatorname{ReLU}(\mathbf{F}\mathbf{p}_{1})\in\mathbb{R}^{kb},\end{split}$

where $\mathbf{r}_{j}$ is the representation of the $j$-th candidate response,
$|Y|$ is the number of all candidate responses. We follow Luo et al. (2019) to
model the user bias towards KB entries over the $j$-th candidate response by a
term $\mathbf{b}_{j}$, where the dimension $kb$ is the number of attributes of
a KB entry. $\mathbf{p}_{1}\in\mathbb{R}^{n}$ is the one-hot representation of
the current user profile. $\mathbf{F}\in\mathbb{R}^{kb\times n}$ maps user
profiles into a KB entry.

###  3.7. Learning of CoMemNN

Multiple-hop reading or updating has been shown to help improve performance of
MemNN by reading or updating the memory multiple times (Sukhbaatar et al.,
2015; Joshi et al., 2017; Luo et al., 2019). To enhance CoMemNN, we devise a
learning algorithm to update the query and memories with multiple hops, and
further differentiate the specific losses of the UPE and DRS modules. The
learning procedure is shown in Algorithm 1. First, MI searches neighbors
$\\{u_{2},\dots,u_{k}\\}$ of the current user $u_{1}$ to initialize the
profile memory $\mathbf{M}_{t}^{P}$ and dialogue memory $\mathbf{M}_{t}^{D}$.
Second, MU and MR are conducted $HopN$ times, and for each time: MU updates
the dialogue memory $\mathbf{M}_{t}^{D}$ and the profile memory
$\mathbf{M}_{t}^{P}$ by considering their cooperative interaction. After that,
MR updates the query representation $\mathbf{q}_{t}$ by reading from the
enriched dialogue memory followed by profile memory. Last, the Dialogue
Response Selection (DRS) module uses the newest updated query
$\mathbf{\tilde{q}}_{t}$ to match candidate responses so as to predict a
response distribution $\mathbf{\tilde{y}}_{t}$.

To evaluate the performance of DRS and UPE, we define two mapping functions to
get prediction labels:

  * •

$\mathrm{Argmax}(\cdot)$: it outputs the index $y_{t}$ with the highest
probability in a predicted response distribution $\tilde{\mathbf{y}}_{t}$;

  * •

$\mathrm{PiecewiseArgmax}(\cdot)$: it generates a 1-0 vector from the
predicted enriched profile $\mathbf{m}_{t}^{P}$, where
$\tilde{\mathbf{p}}_{t}^{1}[i]=1$ only if $\mathbf{m}_{t}^{P}[i]$ achieves the
highest probability among the values that belong to the same attribute.

To optimize DRS, we use a standard cross-entropy loss between the prediction
$\mathbf{\tilde{y}}$ and the one-hot encoded true label $\mathbf{y}$:

(10)
$\mathcal{L}_{\acs{DRS}}(\theta)=-\frac{1}{N_{1}}\sum_{i=1}^{N_{1}}\sum_{j=1}^{|Y|}\mathbf{y}_{j}\log\mathbf{\tilde{y}}_{j},$

where $\theta$ are all parameters in the model and $N_{1}$ is the number of
training samples.

To control the learning of UPE, we introduce the element-wise mean squared
loss between the sampled profile $\mathbf{p}=\\{p_{1},\dots,p_{N_{2}}\\}$ and
its corresponding enriched profile
$\tilde{\mathbf{p}}=\\{\tilde{p}_{1},\dots,\tilde{p}_{N_{2}}\\}$:

(11)
$\mathcal{L}_{\acs{UPE}}(\theta)=-\frac{1}{N_{2}}\sum_{i=1}^{N_{2}}(p_{i}-\tilde{p}_{i}),$

where $\theta$ are all parameters in the model and $N_{2}$ is the number of
sampled values.

Finally, the final loss is a linear combination:

(12)
$\mathcal{L}(\theta)=\mu\mathcal{L}_{\acs{DRS}}(\theta)+(1-\mu)\mathcal{L}_{\acs{UPE}}(\theta),$

where $\mu$ is a hyper-parameter to balance the relative importance of the
constituent losses.

Input: turn $t$, user $u_{1}$, profile $\mathbf{p}_{1}$, dialogue history
$\mathbf{H}_{t}$, query $\mathbf{q}_{t}$, response candidates
$\\{\mathbf{r}_{1},\dots,\mathbf{r}_{|Y|}\\}$, max hop $HopN$, $(k-1)$
neighbors

Output: A index $\mathbf{y}_{t}$ of next response; An one-hot vector
$\tilde{\mathbf{p}}_{t}^{1}$ presenting the enriched profile.

$\\{u_{2},\dots,u_{k}\\}\leftarrow\text{Search}(\mathbf{p}_{1},k-1)$;

$\triangleright$ MI

1 $\mathbf{M}_{t}^{P}\leftarrow[\mathbf{p}_{1},\dots,\mathbf{p}_{k}]$;

2
$\mathbf{M}_{t}^{D}\leftarrow[\mathbf{h}_{t}^{1},\dots,\mathbf{h}_{t}^{k}];\mathbf{h}_{t}^{i}\leftarrow(\tilde{q}_{t},\mathbf{H}_{t}^{i}),i\in[1,k];\mathbf{\tilde{q}}_{t}\leftarrow\mathbf{q}_{t}$;

3 while  _hop $\leq$ HopN_ do

$\mathbf{\tilde{M}}_{t}^{D}\leftarrow\mathbf{M}_{t}^{D}$;
$\mathbf{\tilde{M}}_{t}^{P}\leftarrow\mathbf{M}_{t}^{P}$ ;

$\triangleright$ MU

4 $\mathbf{M}_{t}^{D}\leftarrow\mathbf{\tilde{M}}_{t}^{D}$;

5
$\mathbf{M}_{t}^{P}\leftarrow\Gamma(\mathbf{\tilde{M}}_{t}^{P},\mathbf{\tilde{M}}_{t}^{D})$;

$\mathbf{m}_{t}^{D}\leftarrow\mathbf{M}_{t}^{D}$;
$\mathbf{\tilde{q}}_{t}\leftarrow\mathbf{\tilde{q}}_{t}+\mathbf{m}_{t}^{D}$ ;

$\triangleright$ MR

6 $\mathbf{m}_{t}^{P}\leftarrow\mathbf{M}_{t}^{P}$;
$\mathbf{\tilde{q}}\leftarrow\mathbf{\tilde{q}}_{t}+\mathbf{m}_{t}^{P}$;

7

8 end while

$\mathbf{\tilde{y}}_{t}\leftarrow\mathrm{Softmax}(\mathbf{\tilde{q}}_{t}^{T}\mathbf{r}_{1}+\mathbf{b}_{1},\dots,\mathbf{\tilde{q}}_{t}^{T}\mathbf{r}_{|Y|}+\mathbf{b}_{|Y|})$
;

$\triangleright$ DRS

9 $y_{t}\leftarrow\mathrm{Argmax}_{j}(\mathbf{\tilde{y}}_{t})$;

$\tilde{\mathbf{p}}_{t}^{1}\leftarrow\mathrm{PiecewiseArgmax}(\mathbf{m}_{t}^{P})$

Algorithm 1 Multiple hop CoMemNN.

##  4\. Experimental Setup

###  4.1. Research questions

We seek to answer the following questions in our experiments:  (Q1) How well
does CoMemNN perform? Does it significantly and continuously outperform state-
of-the-art methods?  (Q2) What are the effects of different components in
CoMemNN?  (Q3) Do different profile attributes contribute differently? and
(Q4) How well does CoMemNN perform in terms of robustness?

###  4.2. Dataset and evaluation

We use the personalized bAbI dialogue (PbAbI) dataset (Joshi et al., 2017) for
our experiments; this is an extension of the bAbI dialogue (bAbI) dataset that
incorporates personalization (Bordes and Weston, 2017). To the best of our
knowledge, this is the only available open dataset for personalized TDSs.
There are two versions: a large version with around 12,000 dialogues and a
small version with 1,000 dialogues. These two datasets share the same
vocabulary with $14,819$ tokens and candidate response set with $43,863$
responses. It defines four user profile attributes (gender, age, dietary
preference, and favorite food) and composes corresponding attribute-value
pairs to a user profile. Each conversation is provided with all of the above
user profile attributes, e.g., {(Gender, Male), (Age, Young), (Dietary, Non-
vegetarian), (Favorite: Fish and Chips)}. But this does not mean the given
user profile is complete because the user may also like “Paella”, although
“Fish and Chips” is his/her favorite food. To simulate incomplete profiles
with various degrees of incompleteness, we randomly discard attribute values
from a user profile with probabilities of [0%, 10%, 30%, 50%, 70%, 90%, 100%]
and obtain 7 alternative datasets, respectively.

We evaluate the performance of the full dialogue task using the following two
metrics (Joshi et al., 2017):

  * •

Response Selection Accuracy (RSA): the fraction of correct responses out of
all candidate responses (Joshi et al., 2017; Luo et al., 2019); and

  * •

Profile Enrichment Accuracy (PEA): we define this metric as the fraction of
correct profile values out of all discarded profile values.

We use a paired t-test to measure statistical significance ($p<0.01$) of
relative improvements.

To compare model stability, we propose a statistic $\sigma$, namely stability
coefficient, which is defined as the standard deviation of a list of
performance results. Formally, given a list of evaluation values
$[z_{1},\dots,z_{N+1}]$, either RSA or PEA scores, $\sigma$ is computed as
follows:

(13)
$\begin{split}\sigma(\mathbf{z})&=\sqrt{\frac{1}{N}\sum_{i=1}^{N}{(\mathbf{z}_{i}-\bar{\mathbf{z}})^{2}}}\\\
\mathbf{z}&=[z_{2}-z_{1},\dots,z_{N+1}-z_{N}],\end{split}$

where $\bar{z}$ is the mean of the values in performance difference list
$\mathbf{z}$.

###  4.3. Baselines

We compare with all the methods that have reported results on the PbAbI
dataset (Joshi et al., 2017).

  * •

Memory Network (MemNN). It regards the profile information as the first user
utterance ahead of each dialogue and achieves personalization by modeling
dialogue context using the standard MemNN model (Bordes et al., 2016).

  * •

Split Memory Network (SMemNN). It splits memory into a proﬁle memory and a
dialogue memory. The former encodes user profile attributes as separate
entries and the latter operates the same as the MemNN. The element-wise sum of
both memories are used for ﬁnal decision (Joshi et al., 2017).

  * •

Retrieval Memory Network (RMemNN). It features an encoder-encoder memory
network with a retrieval module that employs the user utterances and user
profiles to collect relevant information from similar users’ conversations
(Zhang et al., 2020).

  * •

Personalized Memory Network (PMemNN). It uses MemNN to model the current user
profile, the current dialogue history, as well as the dialogue history of all
users with the same gender and age. It also models user bias towards different
KB entries (Luo et al., 2019).

  * •

Neighbor-based Personalized Memory Network (NPMemNN). Our implementation of
PMemNN is based on Pytorch. Unlike PMemNN, we use the dialogue history from
the nearest $(k-1)$ neighbors instead of all users with the same gender and
age.

###  4.4. Implementation details

We follow the experimental settings detailed in (Luo et al., 2019). The
embedding size of word/profile is 128. The size of memory is 250. The mini-
batch size is 64. The maximum number of training epoch is 250, and the number
of hops is 3 (see Algorithm 1). The K-Nearest Neighbors (KNN) algorithm is
implemented based on faiss111https://github.com/facebookresearch/faiss with
the inner product measurement and the number of collaborative users $k=100$.
We implement NPMemNN and CoMemNN in PyTorch.222https://pytorch.org/ And the
code of the other models is taken from the original papers. We use Adam
(Kingma and Ba, 2015) as our optimization algorithm with learning rate of
$0.01$ and initialize the learnable parameters with the Xavier initializer. We
also apply gradient clipping (Pascanu et al., 2013) with range $[-10,10]$
during training. We use $l2$ regularization to alleviate overfitting, the
weight of which is set to $10^{-5}$. We treat the importance of losses of DRS
and UPE equally, i.e., $\mu=0.5$. The code is available
online.333https://github.com/Jiahuan-Pei/CoMemNN

##  5\. Results (Q1)

###  5.1. Results without discarding user profiles

We show the overall response selection performance of all methods in Table 2.

Table 2. Overall performance in terms of the RSA metric. Bold face indicates leading results. Significant improvements over NPMemNN are marked with ∗ (paired t-test, $p<0.01$). | Small set (%) | Large set (%)
---|---|---
MemNN (Joshi et al., 2017) | 77.74 | 85.10
SMemNN (Joshi et al., 2017) | 78.10 | 87.28
RMemNN (Zhang et al., 2020) | 83.94 | 87.33
PMemNN (Luo et al., 2019) | 88.07 | 95.33
NPMemNN | 87.91 | 97.49
CoMemNN | 91.13∗ | 98.13∗

First, CoMemNN outperforms all baselines on both the small and large datasets
by a large margin. It significantly outperforms the best baseline PMemNN by
3.06% on the small dataset and 2.80% on the large dataset. The improvements
demonstrate the effectiveness of CoMemNN. We believe the main reason is that
the proposed cooperative mechanism is able to enrich the incomplete profiles
gradually as dialogues progress and the enriched profiles improve help to
response selection simultaneously. We will analyze this in more depth in the
next session.

Second, the performance of NPMemNN is comparable to that of PMemNN on the
small dataset and achieves 2.16% higher RSA on the large dataset. Recall that
NPMemNN is our implementation of PMemNN using Pytorch; the only difference is
the KNN algorithm used for neighbor searching, so the result shows that our
new neighbor searching method is more effective. Since our CoMemNN is built
upon NPMemNN, for the remaining experiments, we will use NPMemNN for further
comparison and analysis.

Third, the results on the small and large datasets mostly show consistent
trends. For the remaining analysis experiments in the next section (Section
6), we will report results on the small dataset only. The findings on the
large dataset are qualitatively similar.

###  5.2. Results with different profile discard ratios

We compare CoMemNN and NPMemNN under different profile discard ratios. The
results are shown in Table 3.

Table 3. Comparison of CoMemNN and NPMemNN in terms of the RSA metric w.r.t. different profile discard ratios. Bold face indicates leading results. Significant improvements over NPMemNN are marked with ∗ (paired t-test, $p<0.01$). The values of Diff. are computed by absolute difference of RSA (%) between CoMemNN and NPMemNN. Discard Ratio | 0% | 10% | 30% | 50% | 70% | 90% | 100%
---|---|---|---|---|---|---|---
NPMemNN | 87.91 | 86.11 | 86.56 | 85.79 | 83.93 | 84.08 | 84.83
CoMemNN | 91.13∗ | 89.90∗ | 88.69∗ | 87.80∗ | 86.35∗ | 84.83∗ | 82.85
Small Set/Diff. |  03.22 |  03.79 |  02.13 |  02.01 |  02.42 |  00.75 |  $-$1.98
NPMemNN | 97.49 | 97.01 | 96.05 | 95.52 | 95.40 | 90.96 | 90.50
CoMemNN | 98.13∗ | 97.94∗ | 97.68∗ | 97.53∗ | 96.98∗ | 96.63∗ | 92.73∗
Large Set/Diff. |  00.64 |  00.93 |  01.63 |  02.01 |  01.58 |  05.67 |  02.23

First, CoMemNN significantly outperforms NPMemNN on both the small and large
datasets when the profile discard ratios range from 0% to 90%. Specifically,
it gains an improvement of 0.75%–3.79% on the small dataset and 0.64%–5.67% on
the large dataset, respectively. Without discarding profile attribute values,
CoMemNN achieves 3.22% / 0.64% of improvement compared with NPMemNN. Unlike
the raw profiles where each attribute has only one value, the enriched profile
generated by CoMemNN is able to represent a distribution over all possible
values, which can better capture users’ preference. For example, a user may
label “Fish and Chips” as his favorite food, but this does not mean he does
not like “Paella.” With the raw profile, this is not addressed.

Second, the performance of CoMemNN steadily decreases with the increase of the
profile discard ratio, as is to be expected. This is reasonable as it becomes
more and more challenging for CoMemNN to find back missing values of user
profiles. Interestingly, the performance difference between CoMemNN and
NPMemNN first increases and then decreases with the increase of the profile
discard ratio. A possible reason is that CoMemNN is able to infer the missing
values of user profiles effectively with lower profile discard ratios.
However, the profile enrichment ability decreases due to the lack of too many
profile values. This hypothesis can be verified by the results that the
increase trend lasts longer on the large dataset. Because even with the same
profile discard ratio, there are more values of user profiles left on the
large dataset for CoMemNN to infer the missing ones. We note that NPMemNN
outperforms CoMemNN when all user profiles are discarded on the small dataset.
The reason is that UPE cannot enrich user profiles properly in this case,
which results in a negative impact on DRS. But this is not the case on the
large dataset where UPE can still enrich user profiles properly when the model
can find enough personal information clues from more dialogue history.

Third, to answer Q4, we compute the statistic $\sigma$ (Eq. 13) to compare the
model stability. The $\sigma$ values for CoMemNN and NPMemNN are 0.3357/1.0407
on the small dataset and 1.3479/1.4849 on the large dataset, respectively.
Thus, NPMemNN has higher deviations, which shows that CoMemNN is more stable
than NPMemNN with various profile discard ratios.

##  6\. Analysis

We analyze the performance of the following variants of CoMemNN:

  * •

CoMemNN. The full model.

  * •

CoMemNN-PEL. CoMemNN without Profile Enrichment Loss (PEL), defined in Eq. 11.

  * •

CoMemNN-PEL-UPE. CoMemNN without PEL or UPE. This is exactly NPMemNN.

  * •

CoMemNN-NP. CoMemNN without the Neighbor Profile (NP) as input for UPE.

  * •

CoMemNN-NP-CP. CoMemNN without NP or the Current Profile (CP) as input for
UPE.

  * •

CoMemNN-ND. CoMemNN without the Neighbor Dialogue (ND) of dialogues as input
for UPE.

  * •

CoMemNN-ND-CD. CoMemNN without ND or the Current Dialogue (CD) of dialogues as
input for UPE.

  * •

CoMemNN-ND-NP. CoMemNN without ND or NP of dialogues as input for UPE.

###  6.1. Ablation study on PEA (Q2)

We study the Profile Enrichment Accuracy (PEA) performance of different
variants in Table 4.

Table 4. Performance of UPE evaluated in terms of Profile Enrichment Accuracy (PEA). In each cell, the ﬁrst number represents the PEA (%), and the number in parentheses shows the difference compared with CoMemNN. $\downarrow$ and $||$ denote a decrease and no change compared to CoMemNN, respectively.  Discard Ratio | 10% | 30% | 50% | 70% | 90% | 100%
---|---|---|---|---|---|---
CoMemNN | 99.99 | 99.93 | 99.82 | 99.83 | 99.38 | 98.98
CoMemNN-PEL | 85.71 ($\downarrow$14.28) | 87.85 ($\downarrow$12.08) | 91.34 ($\downarrow$8.48) | 89.19 ($\downarrow$10.64) | 90.04 ($\downarrow$9.34) | 90.60 ($\downarrow$8.38)
CoMemNN-NP | 99.87 ($\downarrow$0.12) | 99.85 ($\downarrow$0.08) | 99.24 ($\downarrow$0.58) | 99.15 ($\downarrow$0.68) | 99.13 ($\downarrow$0.25) | 98.86 ($\downarrow$0.12)
CoMemNN-NP-CP | 98.89 ($\downarrow$1.10) | 99.09 ($\downarrow$0.84) | 99.16 ($\downarrow$0.66) | 99.20 ($\downarrow$0.63) | 99.14 ($\downarrow$0.23) | 98.92 ($\downarrow$0.06)
CoMemNN-ND | 99.72 ($\downarrow$0.26) | 99.87 ($\downarrow$0.06) | 99.80 ($\downarrow$0.02) | 99.46 ($\downarrow$0.37) | 98.72 ($\downarrow$0.66) | 97.23 ($\downarrow$1.75)
CoMemNN-ND-CD | 99.99 ($||$0.00) | 99.86 ($\downarrow$0.07) | 99.68 ($\downarrow$0.14) | 99.69 ($\downarrow$0.14) | 99.19 ($\downarrow$0.19) | 34.78 ($\downarrow$64.2)
CoMemNN-ND-NP | 99.09 ($\downarrow$0.90) | 98.98 ($\downarrow$0.95) | 97.95 ($\downarrow$1.87) | 97.69 ($\downarrow$2.14) | 97.06 ($\downarrow$2.32) | 97.23 ($\downarrow$1.75)
Table 5. Ablation study on DRS evaluated in terms Response Selection Accuracy (RSA). In each cell, the ﬁrst number represents the RSA (%), and the number in parentheses shows the difference compared with CoMemNN. $\downarrow$ and $\uparrow$ denote decrease and increase, respectively. Underlining marks results that are $\geq$1.0% higher than those of CoMemNN. Discard Ratio | 0% | 10% | 30% | 50% | 70% | 90% | 100%
---|---|---|---|---|---|---|---
CoMemNN | 91.13 | 89.90 | 88.69 | 87.80 | 86.35 | 84.83 | 82.85
CoMemNN-PEL | 90.84 ($\downarrow$0.29) | 90.29 ($\uparrow$0.39) | 89.07 ($\uparrow$0.38) | 87.18 ($\downarrow$0.62) | 85.42 ($\downarrow$0.93) | 80.54 ($\downarrow$4.29) | 81.23 ($\downarrow$1.62)
CoMemNN-PEL-UPE | 87.91 ($\downarrow$3.22) | 86.11 ($\downarrow$3.79) | 86.56 ($\downarrow$2.13) | 85.79 ($\downarrow$2.01) | 83.93 ($\downarrow$2.42) | 84.08 ($\downarrow$0.75) |  84.83 ($\uparrow$1.98)
CoMemNN-NP | 91.06 ($\downarrow$0.07) |  91.23 ($\uparrow$1.33) | 89.17 ($\uparrow$0.48) | 85.26 ($\downarrow$2.54) | 83.30 ($\downarrow$3.05) | 82.10 ($\downarrow$2.73) | 82.83 ($\downarrow$0.02)
CoMemNN-NP-CP | 86.60 ($\downarrow$4.53) | 86.10 ($\downarrow$3.80) | 84.56 ($\downarrow$4.13) | 83.53 ($\downarrow$4.27) | 82.48 ($\downarrow$3.87) | 81.95 ($\downarrow$2.88) | 81.35 ($\downarrow$1.50)
CoMemNN-ND | 90.91 ($\downarrow$0.22) | 87.33 ($\downarrow$2.57) | 89.06 ($\uparrow$0.37) | 87.49 ($\downarrow$0.31) | 86.59 ($\uparrow$0.24) | 85.38 ($\uparrow$0.55) |  85.41 ($\uparrow$2.56)
CoMemNN-ND-CD | 87.70 ($\downarrow$3.43) | 90.44 ($\uparrow$0.54) | 85.79 ($\downarrow$2.90) | 84.90 ($\downarrow$2.90) | 83.56 ($\downarrow$2.79) | 82.57 ($\downarrow$2.26) |  85.38 ($\uparrow$2.53)
CoMemNN-ND-NP | 90.04 ($\downarrow$1.09) | 91.08 ($\uparrow$1.18) | 89.23 ($\uparrow$0.54) | 87.38 ($\downarrow$0.42) | 85.76 ($\downarrow$0.59) | 85.46 ($\uparrow$0.63) |  85.41 ($\uparrow$2.56)

First, CoMemNN can effectively enrich user profiles by inferring the missing
values. It is able to correctly predict more than 98.98% of missing values in
user profiles under different profile discard ratios. We believe UPE benefits
a lot from modeling the interaction between user profiles and dialogues. UPE
is able to capture more personal information from dialogue history with
dialogues gradually going on. The PEA scores are all very high, because the
PbAbI dataset is simulated, which makes it relatively easy to predict missing
attribute values of user profiles.

Second, we can see that each component of UPE generally has a positive effect
on the performance since most PEA scores of most variants decrease.
Specifically, CoMemNN-PEL decreases by 8.38%–14.20% compared with CoMemNN.
This means that it is important to add the UPE loss (Eq. 11), rather than only
optimizing the DRS loss (Eq. 10). We also show how the four components of UPE
(i.e., NP, CP, ND, and CD as defined in Section 3.3) affect its performance.
We find that:  (1) CoMemNN-ND-NP continuously decreases 0.90%–2.32% with the
increase of the profile discard ratio. This means that neighbor users play an
important role.  (2) CoMemNN-ND-CD (with 100% profile discard ratio) decreases
dramatically, which is as expected, because CoMemNN cannot infer the missing
values without any dialogue history and profiles. This also explains the
increase of the corresponding RSA score in Table 5.  (3) The decrease is
mostly less than 2.32% except that the decrease of CoMemNN-ND-CD (with 100%
profile discard ratio, i.e., no NP or CP as well) is 64.2%. This reveals that
different information sources are complementary to each other. The performance
will not be affected largely unless all the four inputs (i.e., NP, CP, ND, CD)
are removed.

Lastly, we compute the stability coefficient $\sigma$ (Eq. 13) of the variants
in Table 4 which are 0.1867, 1.8781, 0.2236, 0.1402, 25.6845, 0.1867, 0.4182,
respectively. This shows that all variants are robust in terms of the
performance of UPE with small stability coefficient, except for CoMemNN-ND-CD.

###  6.2. Ablation study on RSA (Q2)

We investigate the RSA performance of different variants in Table 5.

First, the performance decreases generally by removing any component of UPE.
In particular, CoMemNN-PEL has a greater effect on RSA when the profile
discard ratios get larger. This is reasonable because the larger the profile
discard ratio, the more space for improvement the proposed model has compared
with NPMemNN. CoMemNN-PEL-UPE is inferior to CoMemNN-PEL generally, which
means that the UPE module helps as it implicitly impact the DRS loss (Eq. 10).
But this ability weakens when the profile discard ratio is larger than 90%.

Second, we observe that the four information sources (i.e., NP, CP, ND, CD)
have different effects under different profile discard ratios. Particularly,
the profiles of the current users and their neighbors generally contribute
most to the RSA performance. We can see that CoMemNN-NP-CP drops 1.50%–4.53%
under all profile discard ratios. The reason is that user profiles directly
store personal information; it is easier to infer missing values from
collaborative user profiles than from dialogues.

Third, we find that NP and ND are complementary to each other. CoMemNN-NP
either has a massive drop (2.54%–3.05%) or small changes ($\leq$0.48%) with
the most profile discard ratios, except for one obvious rise (1.33%) under the
10% profile discard ratio. In contrast, CoMemNN-ND works fine under the 10%
profile discard ratio, but it performs poorly for the rest. Thus, the
performance of CoMemNN is influence strongly by a drop in attribute values
unless we remove both NP and ND under 100% profile discard ratios.

Lastly, the dialogue history also contributes to the RSA performance in most
cases. CoMemNN-ND-CD shows decrease (2.26%–3.43%) or a small change (0.54%)
for most of cases, except for an obvious increase under the 100% profile
discard ratio. We think that the reason is that some of the predicted profiles
are not even in provided profiles, which leads to a very limited PEA score of
34.78% under the 100% profile discard ratio (see Table 4). But these predicted
values happen to be useful for selecting an appropriate response in DRS.

###  6.3. Effect of multiple-hop mechanism (Q2)

We compare the RSA performance of CoMemNN and NPMemNN with different numbers
of hops. The results are shown in Table 6.

Table 6. Analysis of the effect of hop number on DRS. Bold face indicates leading results. Significant improvements over NPMemNN are marked with ∗ (paired t-test, $p<0.01$). The values of Diff. are computed by absolute difference of RSA (%) between CoMemNN and NPMemNN. #Hop | 1 | 2 | 3 | 4
---|---|---|---|---
NPMemNN | 88.11 | 87.22 | 87.91 | 87.61
CoMemNN | 90.07∗ | 90.78∗ | 91.13∗ | 90.77∗
Diff. |  01.96 |  03.56 |  03.22 |  03.16

We see that CoMemNN greatly outperforms NPMemNN by a large margin
(1.96%–3.56%) with all number of hops. This further confirms the non-trivial
improvement of CoMemNN. Besides, CoMemNN improves by 1.06% when the number of
hop changes from 1 to 3 and slightly decreases with 4. This means that CoMemNN
benefits from a multiple-hop mechanism.

###  6.4. Effect of different profile attributes (Q3)

We explore how the four types of profile attributes (i.e., gender, age,
dietary preference, and favorite food) affect the RSA performance. The results
are shown in Table 7.

Table 7. Analysis of profile attribute importance to DRS. Discard attribute table shows we discard all values of a specific attribute or a combination of two specific attributes. Retain attribute table shows we retain all values of a specific attribute and discard all values for the rest. Underline indicates the lower bound baseline that retains no attributes. Bold face indicates the upper bound baseline that retains all attributes. Discarded attribute | none | gender | age | dietary | favorite | all
---|---|---|---|---|---|---
gender | / | 93.05 | 91.94 | 88.86 | 91.95 | /
age | / | / | 92.26 | 89.37 | 91.04 | /
dietary | / | / | / | 86.74 | 86.42 | /
favorite | / | / | / | / | 90.25 | /
Retained attribute | 82.85 | 87.46 | 87.93 | 90.57 | 87.37 | 91.13

First, each attribute works well in isolation. Specifically, when we only
retain the values of each single attribute, we obtain the results in the last
row as 87.46%, 87.93%, 90.57%, 87.37% for gender, age, dietary, favorite,
respectively. The attribute “dietary” contributes most followed by “age”,
“gender” and “favorite.”

Second, different types of attributes depend on each other and influence the
RSA performance differently. If we only remove the values of one attribute, we
get the results on the diagonal: 93.05%, 92.26%, 86.74%, 90.25%, respectively.
Removing “dietary” drops most followed by “favorite.” Thus, “dietary”
contributes more than the rest.

An exception is that the RSA performance increases when discarding “gender”
and “age.” We believe this is the effect of the neighbors. To show this, we
further investigate the effect of “gender” and “age” without using neighbor
information. The results are shown in Table 8.

Table 8. Analysis of profile attribute importance to DRS without the effect of neighbors. Bold face indicates the baseline of CoMemNN without neighbors. In each cell, the ﬁrst number represents the RSA (%), and the number in parenthesis shows the difference values, and $\downarrow$ denotes decrease compared with the baseline. |  RSA (Diff.)
---|---
CoMemNN w/o neighbors |  90.34
CoMemNN w/o neighbors - gender |  88.25 ($\downarrow$2.09)
CoMemNN w/o neighbors - age |  85.62 ($\downarrow$4.72)
CoMemNN w/o neighbors - gender - age |  83.73 ($\downarrow$6.61)

We can see that removing “gender” and “age” decreases the performance in this
case. Thus, the different effects of “gender” and “age” are due to the
neighbors.

##  7\. Conclusion

In this paper, we have studied personalized TDSs without assuming that we have
complete user profiles. We have proposed Cooperative Memory Network (CoMemNN),
which introduces a cooperative mechanism to enrich user profiles gradually as
dialogues progress, and to improve response selection based on enriched
profiles simultaneously. We also devise a learning algorithm to effectively
learn CoMemNN with multiple hops.

Extensive experiments on the personalized bAbI dialogue (PbAbI) dataset
demonstrate that CoMemNN significantly outperforms state-of-the-art baselines.
Further analysis experiments confirm the effectiveness of CoMemNN by analyzing
the performance and contribution of each component.

A limitation of our work is that we tested the performance of CoMemNN on the
only open available personalized TDSs dataset PbAbI. We encourage the
community to work on creating additional resources for this task.

As to future work, we hope to experiment on more datasets and investigate how
the performance varies on different datasets and whether we can further
improve the performance by leveraging non-personalized TDS datasets.

###### Acknowledgements.

This research was partially supported by the China Scholarship Council (CSC).
All content represents the opinion of the authors, which is not necessarily
shared or endorsed by their respective employers and/or sponsors.

## References

  * (1)
  * Bordes et al. (2016) Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2016\.  Learning end-to-end goal-oriented dialog.  _arXiv preprint arXiv:1605.07683_ (2016).
  * Bordes and Weston (2017) Antoine Bordes and Jason Weston. 2017.  Learning end-to-end goal-oriented dialog. In _ICLR_.
  * Chen et al. (2017) Hongshen Chen, Xiaorui Liu, Dawei Yin, and Jiliang Tang. 2017\.  A survey on dialogue systems: Recent advances and new frontiers.  _ACM SIGKDD Explorations Newsletter_ 19, 2 (2017), 25–35.
  * Dodge et al. (2015) Jesse Dodge, Andreea Gane, Xiang Zhang, Antoine Bordes, Sumit Chopra, Alexander Miller, Arthur Szlam, and Jason Weston. 2015\.  Evaluating prerequisite qualities for learning end-to-end dialog systems.  _arXiv preprint arXiv:1511.06931_ (2015).
  * Eric et al. (2017) Mihail Eric, Lakshmi Krishnan, Francois Charette, and Christopher D Manning. 2017.  Key-value retrieval networks for task-oriented dialogue. In _SIGDIAL_. 37–49.
  * Ficler and Goldberg (2017) Jessica Ficler and Yoav Goldberg. 2017.  Controlling linguistic style aspects in neural language generation. In _Proceedings of the Workshop on Stylistic Variation_. 94–104.
  * Henderson et al. (2019) Matthew Henderson, Ivan Vulić, Daniela Gerz, Iñigo Casanueva, Paweł Budzianowski, Sam Coope, Georgios Spithourakis, Tsung-Hsien Wen, Nikola Mrkšić, and Pei-Hao Su. 2019.  Training neural response selection for task-oriented dialogue systems. In _ACL_. 5392–5404.
  * Herzig et al. (2017) Jonathan Herzig, Michal Shmueli-Scheuer, Tommy Sandbank, and David Konopnicki. 2017.  Neural response generation for customer service based on personality traits. In _Proceedings of the 10th International Conference on Natural Language Generation_. 252–256.
  * Joshi et al. (2017) Chaitanya K Joshi, Fei Mi, and Boi Faltings. 2017.  Personalization in goal-oriented dialog. In _NIPS Conversational AI Workshop_.
  * Kingma and Ba (2015) Diederik Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization. In _ICLR_.
  * Li et al. (2020) Aaron W Li, Veronica Jiang, Steven Y Feng, Julia Sprague, Wei Zhou, and Jesse Hoey. 2020\.  ALOHA: Artificial learning of human attributes for dialogue agents. In _AAAI_. 8155–8163.
  * Li et al. (2016) Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016\.  A persona-based neural conversation model. In _ACL_. 994–1003.
  * Li et al. (2014) Xiang Li, Gokhan Tur, Dilek Hakkani-Tür, and Qi Li. 2014\.  Personal knowledge graph population from user utterances in conversational understanding. In _2014 IEEE Spoken Language Technology Workshop (SLT)_. IEEE, 224–229.
  * Lin et al. (2020) Zibo Lin, Deng Cai, Yan Wang, Xiaojiang Liu, Hai-Tao Zheng, and Shuming Shi. 2020\.  Grayscale data construction and multi-level ranking objective for dialogue response selection.  _arXiv preprint arXiv:2004.02421_ (2020).
  * Liu and Perez (2017) Fei Liu and Julien Perez. 2017.  Gated end-to-end memory networks. In _EACL_. 1–10.
  * Liu et al. (2020) Qian Liu, Yihong Chen, Bei Chen, Jian-Guang Lou, Zixuan Chen, Bin Zhou, and Dongmei Zhang. 2020.  You impress me: Dialogue generation via mutual persona perception.  _arXiv preprint arXiv:2004.05388_ (2020).
  * Lu et al. (2019) Yichao Lu, Manisha Srivastava, Jared Kramer, Heba Elfardy, Andrea Kahn, Song Wang, and Vikas Bhardwaj. 2019.  Goal-oriented end-to-end conversational models with profile features in a real-world setting. In _NAACL-HLT_. 48–55.
  * Luan et al. (2017) Yi Luan, Chris Brockett, Bill Dolan, Jianfeng Gao, and Michel Galley. 2017.  Multi-task learning for speaker-role adaptation in neural conversation models. In _IJCNLP_. 605–614.
  * Luo et al. (2019) Liangchen Luo, Wenhao Huang, Qi Zeng, Zaiqing Nie, and Xu Sun. 2019.  Learning personalized end-to-end goal-oriented dialog. In _AAAI_. 6794–6801.
  * Madotto et al. (2019) Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019.  Personalizing dialogue agents via meta-learning. In _ACL_. 5454–5459.
  * Madotto et al. (2018) Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2018\.  Mem2Seq: Effectively incorporating knowledge bases into end-to-end task-oriented dialog systems. In _ACL_. 1468–1478.
  * Mazare et al. (2018) Pierre-Emmanuel Mazare, Samuel Humeau, Martin Raison, and Antoine Bordes. 2018.  Training millions of personalized dialogue agents. In _EMNLP_. 2775–2779.
  * Mo et al. (2018) Kaixiang Mo, Yu Zhang, Shuangyin Li, Jiajun Li, and Qiang Yang. 2018.  Personalizing a dialogue system with transfer reinforcement learning. In _AAAI_. 5317–5324.
  * Pascanu et al. (2013) Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013\.  On the difficulty of training recurrent neural networks. In _ICML_. 1310–1318.
  * Pei et al. (2019a) Jiahuan Pei, Pengjie Ren, and Maarten de Rijke. 2019a.  A modular task-oriented dialogue system using a neural mixture-of-experts. In _SIGIR Workshop on Conversational Interaction Systems_.
  * Pei et al. (2020) Jiahuan Pei, Pengjie Ren, Christof Monz, and Maarten de Rijke. 2020\.  Retrospective and prospective mixture-of-generators for task-oriented dialogue response generation. In _ECAI_. 2148–2155.
  * Pei et al. (2019b) Jiahuan Pei, Arent Stienstra, Julia Kiseleva, and Maarten de Rijke. 2019b.  SEntNet: Source-aware recurrent entity network for dialogue response selection. In _IJCAI Workshop SCAI_.
  * Qian et al. (2018) Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018.  Assigning personality/profile to a chatting machine for coherent conversation generation.. In _IJCAI_. 4279–4285.
  * Rajendran et al. (2018) Janarthanan Rajendran, Jatin Ganhotra, Satinder Singh, and Lazaros Polymenakos. 2018.  Learning end-to-end goal-oriented dialog with multiple answers. In _EMNLP_. 3834–3843.
  * Shum et al. (2019) Michael Shum, Stephan Zheng, Wojciech Kryściński, Caiming Xiong, and Richard Socher. 2019\.  Sketch-Fill-AR: A persona-grounded chit-chat generation framework.  _arXiv preprint arXiv:1910.13008_ (2019).
  * Song et al. (2020) Haoyu Song, Yan Wang, Wei-Nan Zhang, Xiaojiang Liu, and Ting Liu. 2020.  Generate, delete and rewrite: A three-stage framework for improving persona consistency of dialogue generation.  _arXiv preprint arXiv:2004.07672_ (2020).
  * Song et al. (2019) Haoyu Song, Wei-Nan Zhang, Yiming Cui, Dong Wang, and Ting Liu. 2019.  Exploiting persona information for diverse generation of conversational responses. In _IJCAI_. 5190–5196.
  * Sukhbaatar et al. (2015) Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 2015\.  End-to-end memory networks. In _Advances in neural information processing systems_. 2440–2448.
  * Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014\.  Sequence to sequence learning with neural networks. In _NeurIPS_. 3104–3112.
  * Tigunova (2020) Anna Tigunova. 2020\.  Extracting personal information from conversations. In _Companion Proceedings of The Web Conference 2020_. 284–288.
  * Tigunova et al. (2019) Anna Tigunova, Andrew Yates, Paramita Mirza, and Gerhard Weikum. 2019.  Listening between the lines: learning personal attributes from conversations. In _The Web Conference_. ACM, 1818–1828.
  * Wang et al. (2015) Hao Wang, Naiyan Wang, and Dit-Yan Yeung. 2015.  Collaborative deep learning for recommender systems. In _SIGKDD_. ACM, 1235–1244.
  * Wen et al. (2017) Tsung-Hsien Wen, David Vandyke, Nikola Mrkšić, Milica Gasic, Lina M Rojas Barahona, Pei-Hao Su, Stefan Ultes, and Steve Young. 2017\.  A network-based end-to-end trainable task-oriented dialogue system. In _EACL_. 438–449.
  * Williams et al. (2017) Jason D Williams, Kavosh Asadi, and Geoffrey Zweig. 2017\.  Hybrid code networks: Practical and efficient end-to-end dialog control with supervised and reinforcement learning. In _ACL_. 665–677.
  * Wolf et al. (2019) Thomas Wolf, Victor Sanh, Julien Chaumond, and Clement Delangue. 2019\.  Transfertransfo: A transfer learning approach for neural network based conversational agents.  _arXiv preprint arXiv:1901.08149_ (2019).
  * Wu et al. (2020) Chien-Sheng Wu, Andrea Madotto, Zhaojiang Lin, Peng Xu, and Pascale Fung. 2020.  Getting to know you: User attribute extraction from dialogues. In _Proceedings of the 12th Language Resources and Evaluation Conference_. 581–589.
  * Wu et al. (2018) Chien-Sheng Wu, Andrea Madotto, Genta Indra Winata, and Pascale Fung. 2018.  End-to-end dynamic query memory network for entity-value independent task-oriented dialog. In _IEEE-ICASSP_. IEEE, 6154–6158.
  * Xu et al. (2020) Minghong Xu, Piji Li, Haoran Yang, Pengjie Ren, Zhaochun Ren, Zhumin Chen, and Jun Ma. 2020.  A neural topical expansion framework for unstructured persona-oriented dialogue generation. In _ECAI_. –.
  * Yang et al. (2017) Min Yang, Zhou Zhao, Wei Zhao, Xiaojun Chen, Jia Zhu, Lianqiang Zhou, and Zigang Cao. 2017.  Personalized response generation via domain adaptation. In _SIGIR_. 1021–1024.
  * Young et al. (2013) Steve Young, Milica Gašić, Blaise Thomson, and Jason D Williams. 2013.  POMDP-based statistical spoken dialog systems: A review.  _Proc. IEEE_ 101, 5 (2013), 1160–1179.
  * Zeng et al. (2019) Wenhuan Zeng, Abulikemu Abuduweili, Lei Li, and Pengcheng Yang. 2019.  Automatic generation of personalized comment based on user profile. In _ACL: Student Research Workshop_. 229–235.
  * Zhang et al. (2020) Bowen Zhang, Xiaofei Xu, Xutao Li, Yunming Ye, Xiaojun Chen, and Zhongjie Wang. 2020\.  A memory network based end-to-end personalized task-oriented dialogue generation.  _Knowledge-Based Systems_ (2020), 106398.
  * Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018\.  Personalizing dialogue agents: I have a dog, do you have pets too?. In _ACL_. 2204–2213.
  * Zhang et al. (2019) Wei-Nan Zhang, Qingfu Zhu, Yifa Wang, Yanyan Zhao, and Ting Liu. 2019.  Neural personalized response generation as domain adaptation.  _World Wide Web_ 22, 4 (2019), 1427–1446.
  * Zheng et al. (2019) Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu. 2019.  Personalized dialogue generation with diversified traits.  _arXiv preprint arXiv:1901.09672_ (2019).
  * Zheng et al. (2020) Yinhe Zheng, Rongsheng Zhang, Minlie Huang, and Xiaoxi Mao. 2020\.  A Pre-training based personalized dialogue generation model with persona-sparse data. In _AAAI_. 9693–9700.
  * Zhou et al. (2020) Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020\.  The design and implementation of xiaoice, an empathetic social chatbot.  _Computational Linguistics_ 46, 1 (2020), 53–93.

  *[TDSs]: Task-oriented Dialogue System
  *[CoMemNN]: Cooperative Memory Network
  *[UPE]: User Profile Enrichment
  *[DRS]: Dialogue Response Selection
  *[TDS]: Task-oriented Dialogue System
  *[PbAbI]: personalized bAbI dialogue
  *[ODSs]: Open-ended Dialogue System
  *[MemNN]: Memory Network
  *[ODS]: Open-ended Dialogue System
  *[KB]: Knowledge Base
  *[MI]: Memory Initialization
  *[MU]: Memory Updating
  *[MR]: Memory Reading
  *[CP]: Current Profile
  *[CD]: Current Dialogue
  *[MLP]: Multiple Layer Perceptron
  *[bAbI]: bAbI dialogue
  *[RSA]: Response Selection Accuracy
  *[PEA]: Profile Enrichment Accuracy
  *[SMemNN]: Split Memory Network
  *[RMemNN]: Retrieval Memory Network
  *[PMemNN]: Personalized Memory Network
  *[KNN]: K-Nearest Neighbors
  *[NPMemNN]: Neighbor-based Personalized Memory Network
  *[PEL]: Profile Enrichment Loss
  *[NP]: Neighbor Profile
  *[ND]: Neighbor Dialogue