# Enhancing User Personalization in Conversational Recommenders

Allen Lin  Texas A&M UniversityCollege Station, TexasUSA<EMAIL_ADDRESS>,
Ziwei Zhu  George Mason UniversityFairfax, VirginiaUSA<EMAIL_ADDRESS>,
Jianling Wang  Texas A&M UniversityCollege Station, TexasUSA<EMAIL_ADDRESS>and  James Caverlee  Texas A&M UniversityCollege Station, TexasUSA
<EMAIL_ADDRESS>
(2023)

###### Abstract.

Conversational recommenders are emerging as a powerful tool to personalize a
user’s recommendation experience. Through a back-and-forth dialogue, users can
quickly hone in on just the right items. Many approaches to conversational
recommendation, however, only partially explore the user preference space and
make limiting assumptions about how user feedback can be best incorporated,
resulting in long dialogues and poor recommendation performance. In this
paper, we propose a novel conversational recommendation framework with two
unique features: (i) a greedy NDCG attribute selector, to enhance user
personalization in the interactive preference elicitation process by
prioritizing attributes that most effectively represent the actual preference
space of the user; and (ii) a user representation refiner, to effectively fuse
together the user preferences collected from the interactive elicitation
process to obtain a more personalized understanding of the user. Through
extensive experiments on four frequently used datasets, we find the proposed
framework not only outperforms all the state-of-the-art conversational
recommenders (in terms of both recommendation performance and conversation
efficiency), but also provides a more personalized experience for the user
under the proposed multi-groundtruth multi-round conversational recommendation
setting.

Conversational Recommender System, User Personalization

††journalyear: 2023††copyright: acmlicensed††conference: Proceedings of the
ACM Web Conference 2023; May 1–5, 2023; Austin, TX, USA††booktitle:
Proceedings of the ACM Web Conference 2023 (WWW ’23), May 1–5, 2023, Austin,
TX, USA††price: 15.00††doi: 10.1145/3543507.3583192††isbn:
978-1-4503-9416-1/23/04††ccs: Information systems Recommender systems

##  1\. Introduction

Conversational recommender systems (CRSs) have attracted increasing attention
in the research community due to their wide range of potential applications
and promising functionality (Xu et al., 2021; Deng et al., 2021; Lei et al.,
2020a; Sun and Zhang, 2018). Rather than inferring from past user-item
interactions, a CRS directly engages with the user through a conversation to
learn the user’s current and fine-grained preferences. Such a conversational
approach can provide recommendations that are both highly personalized and
justifiable. While several settings have been proposed to formulate the task
of conversational recommendation, in this work, we focus on the multi-round
conversational recommendation (MCR) setting, which is widely perceived to be
the most realistic setting so far (Zhang et al., 2018; Xu et al., 2021; Deng
et al., 2021; Lei et al., 2020a; Lin et al., 2022b). Under the MCR setting, a
conversational recommender system starts with the interactive preference
elicitation process in which the system explores the preference space of the
user, via asking the user’s preferences on certain attributes (e.g., are you
looking for country music?), to obtain an accurate estimation of the user’s
current interests. Based on the learned preferences of the user, the system
recommends the top-$k$ items that best match the user’s current interests.

The MCR setting has shown promising results across many recommendation tasks
(e.g., music recommendations (Sun and Zhang, 2018; Xu et al., 2021), business
recommendations (Lei et al., 2020b; Lei et al., 2020a), and E-Commerce (Deng
et al., 2021)), yet it makes a strict assumption – that a user only has one
target-item in mind when interacting with the system. In reality, users often
have diverse preferences which makes them simultaneously interested in a wide
variety of items (Qiu et al., 2019; Qin et al., 2014; Tan et al., 2021; Zhang
et al., 2022). Take music recommendations as an example, while Alice likes the
pop style song ‘Shape of You’, she also finds a country style song – such as
‘Somebody like you’ – enjoyable to listen. In this situation, designing and
evaluating a CRS with such a strong single-groundtruth assumption greatly
limits the generalizability of the system.

Hence, we extend the MCR setting to the multi-groundtruth multi-round
conversational recommendation (MGMCR) setting, under which the CRS is designed
and evaluated with the realistic formulation that a user could be
simultaneously interested in a variety of items. This MGMCR setting is more
challenging than the traditional MCR setting, as the CRS now aims to discover
all the groundtruth items of the user. The key in achieving promising
recommendation performance lies in the effectiveness of user personalization
in the system. Specifically, we identify two key challenges that existing
works face under the MGMCR setting:
(1) lack of user personalization in interactive preference elicitation: During
the interactive preference elicitation process, a CRS aims to explore a user’s
preference space by asking the user’s preference on a set of domain specific
attributes. Since every user has unique preferences, a CRS must prioritize on
asking the attributes that the user would consider relevant in expressing her
actual preference space. Although there are some differences among how
existing CRSs decide on which attribute(s) to prioritize for a specific user,
most of them rely heavily on the use of entropy, which does not provide
sufficient granularity for the system to identify the most relevant
attribute(s) for the user. Such an issue could even be exacerbated under the
MGMCR setting given the user now has a broader preference space. In turn, the
asked attributes often result in a sub-optimal representation of the user’s
actual preference space, resulting in poor recommendation performance.
(2) lack of user personalization in preference fusion: To achieve personalized
recommendations for the user, a CRS must fuse together the user’s preferences
– collected in the interactive preference elicitation process – to form a
personalized understanding of the user. To do that, many existing works (Deng
et al., 2021; Xu et al., 2021; Lei et al., 2020b; Lei et al., 2020a) have
adopted the use of attribute-filtering in which the system treats every item
that does not align perfectly with the collected preferences as a mismatch for
the user. This approach has two limitations under the MGMCR setting. First,
many groundtruth items could be mistakenly considered as mismatches. For
instance, Alice is interested in both the action movie, Top Gun: Maverick, and
the comedy movie, Despicable Me. So when the system asks Alice if she is
looking for a comedy movie, Alice will respond with a confirmation. However,
with the attribute comedy added to her set of collected preference, Top Gun:
Maverick would now be considered as a mismatch for Alice since it does not
contain the attribute comedy. Second, it does not take the user’s past
interests into consideration, overlooking the scenario where the same
preference could indicate different signals for different users. For a user
who has rarely watched horror movies, a confirmation on the attribute horror
could indicate a stronger shift in preference compared to a user who has
frequently watched horror movies.

To address the aforementioned challenges, we propose a novel framework for
enhancing user personalization in conversational recommendation systems
(UpsRec). To enhance user personalization in the interactive preference
elicitation process, the framework adopts an NDCG-oriented greedy attribute
selection strategy in which the system directly prioritizes on asking the
attributes that most effectively personalize the recommended items for each
user. And to enhance user personalization in fusing the collected user
preferences, the framework deploys a user representation refiner to
iteratively refine the initial user representation learned from past
interactions. With the refined representation being a more accurate reflection
of the user’s current interests, items no longer need to be filtered to ease
the difficulty of ranking, avoiding the scenario of mistakenly excluding
groundtruth items. In summary, the main contributions of this work are as
follows:

  * •

We are the first to comprehensively study the MGMCR scenario for
conversational recommendation, which is more realistic and generalizable than
the existing MCR setting.

  * •

We propose the UpsRec framework with unique features designed to provide
personalized experiences for the users and effectively guide them to their
items of interest.

  * •

We show the proposed UpsRec framework significantly outperforms state-of-the-
art CRS methods by an average of 23%, while requiring fewer interactions with
the user.

##  2\. Preliminaries

In this section, we provide a brief introduction on multi-round conversational
recommendation (MCR), which is the most widely adopted setting for
conversational recommendation (Lei et al., 2020a; Xu et al., 2021; Deng et
al., 2021; Lei et al., 2020b).

Formally, let $V$ denote the itemset, and $P=(p_{0},p_{1},...,p_{m})$ denote a
set of $m$ domain-specific attributes that describe an item $v\in V$. During
each conversation session, a CRS aims to first learn the user’s fine-grained
preferences on the asked attributes, then provides personalized
recommendations. A conversation session starts by the user specifying an
attribute $p_{0}$ contained in the target item(s) that she is asking the
system to find (e.g., Can you find me some country music?). Next, a CRS calls
upon its conversation component to decide whether to ask the user more
attributes or make a recommendation. If the policy agent decides not enough
preference evidence has been collected, it will pick one attribute $p$ from
the set of unasked attributes $P_{cand}$ to prompt the user. If the user likes
the asked attribute, then policy agent updates the set of collected
preferences of the user, $P_{u}$, by adding $p$ to $P_{u}$; otherwise $P_{u}$
remains unchanged. If the policy agent decides enough information has been
collected after $t$ turns of interaction, the CRS then calls upon its
recommender component to make a list of recommendations. The recommender
component would then rank all the items in the candidate itemset $V_{cand}$ –
the itemset that contains only items with attributes perfectly matching all
specified preferences in $P_{u}$ – then recommends the top-K ranked items to
display to the user. If the user accepts the recommendation, then the system
quits. If the user rejects all the recommended items, the system first removes
the rejected item $V_{rej}$ from $V_{cand}$ then repeats the cycle by calling
upon its conversational component to decide the next action to take. This
process continues until the user quits due to impatience or a predefined
maximum number of turns has been reached.

Multi-round Multi-Groundtruth Setting. Although MCR is currently perceived as
one of the most realistic settings for conversational recommendation, it makes
the limiting assumption that a user only has one item of interest when
interacting with the system. Therefore, we propose the new multi-round multi-
groundtruth (MGMCR) setting, which makes the following changes: (1). when
being asked by the system on the preference towards an attribute $p$, the user
is now assumed to answer according to the attributes associated with all her
groundtruth items (i.e., can be one or multiple), which more realistically
mimics how a user would interact with a CRS; and (2). the system now only
generates one conversation session for each user $u$, instead of a new
conversation session for each observed user-item interaction pair of the user.

##  3\. Proposed Framework

In this section, we introduce a novel framework for enhancing user
personalization in conversational recommendation systems, namely UpsRec.
UpsRec is designed to tackle two key challenges: (i) how to bring user
personalization to interactive preference elicitation; and (ii) how to bring
user personalization to preference fusion. Specifically, we design a greedy
NDCG attribute selector that personalizes the interactive preference
elicitation process by utilizing an NDCG-oriented attribute selection strategy
to prioritize on attributes that most effectively represent the actual
preference space of the user. Then, to effectively fuse together the user
preferences collected from the interactive preference elicitation process, a
user representation refiner is deployed to iteratively refine the current
representation of the user.

The framework starts with the latent interest estimation stage where the
system builds an initial predictive model to estimate the interest of users by
learning a set of latent representations from the historical user-item
interactions. Then a reinforcement learning based policy agent, integrated
with both the greedy NDCG attribute selector and the user representation
refiner, is deployed to decide how to interact with the user. Together, the
entire flow of the proposed UpsRec framework is detailed in Algorithm 1.

Input : the user-item interaction matrix M; the item-attribute information
Dictionary $\textbf{D}_{p}$; user $u$; a matrix containing the adjacent
attributes of all users $\textbf{P}_{adj}$ (see Section 3.2 for details); all
items $V$; the number of items to recommend $k$; the maximum number of turns
$T$

Output : k recommended items

1 Set: $\textbf{R}_{U}$, $\textbf{R}_{V}$, $\textbf{R}_{P}$ = Latent Interest
Estimation (M, $\textbf{D}_{p}$)

2 Initialize: $\textbf{r}_{u_{0}}$ = $\textbf{R}_{U}[u]$; $\textbf{r}_{u_{t}}$
= $\textbf{R}_{U}[u]$

3 Initialize: $P_{u}$ = $\\{\\}$; $P_{cand}$ = $\textbf{P}_{adj}[u]$;
$V_{cand}=V$

4 for  _turn $t=1,2,3...T$_ do

5 Policy Agent selects an action $a$
if  _$a==a_{ask}$_ then

6 $p_{ask}$ = Greedy NDCG Attribute Selector ($\textbf{r}_{u_{0}}$,
$\textbf{r}_{u_{t}}$, $P_{u}$,
$P_{cand}$)
Update: $P_{cand}$ = $P_{cand}$ $\backslash$ $\textit{p}_{ask}$
if  _$u$ accepts $p_{ask}$_ then

7 Update: $P_{u}=P_{u}\cup\ \textit{p}_{ask}$
Set: $\textbf{r}_{u_{t}}$ = User Representation Refiner ($\textbf{r}_{u_{0}}$,
$P_{u}$)

8

9 else

10 Rank all items in $V_{cand}$

11 Recommends the top $k$ ranked items, $V_{rec}$, to $u$;

12 if  _$u$ accepts $V_{rec}$_ then

13 Return $V_{rec}$; Exit.

14 else

15 Update $:V_{cand}=V_{cand}\backslash\ V_{rec}$

16 end if

17

18 end if

19

20 end for

Algorithm 1 UpsRec

###  3.1. Latent Interest Estimation

The goal of the Latent Interest Estimation stage is to learn an initial set of
latent representations for all users, items, and attributes – denoted
respectively as $\textbf{R}_{U},\textbf{R}_{V},\textbf{R}_{P}$ – that will
later be used throughout the framework. Following (Lei et al., 2020b; Lei et
al., 2020a; Xian et al., 2019), we adopt a factorization machine (Rendle,
2010) (FM) as the predictive model. Given user $u$, the user’s collected
preferences $P_{u}$, and an item $v$, $u$’s predicted interest in $v$ is
computed as:

(1) $y(u,v,P_{u})=\textbf{r}_{u}^{T}\textbf{r}_{v}+\sum_{p_{i}\in
P_{u}}\textbf{r}_{v}^{T}\textbf{r}_{p_{i}}$

where $\textbf{r}_{u}$ and $\textbf{r}_{v}$ respectively denote the latent
representation of user $u$ and item $v$, and $\textbf{r}_{p_{i}}$ denotes the
latent representation for attribute $p_{i}\in P_{u}$. As shown in (Lin et al.,
2022a), re-weighting an item’s relevance based on its frequency in the
training set results in more effective learned latent representations.
Therefore, we modify the standard pairwise Bayesian Personalized Ranking (BPR)
objective function (Rendle et al., 2012) to a weighted variation as follows:

(2) $\displaystyle
L=\frac{1}{e^{n_{1}f_{i}}}[-ln\sigma(y(u,v,P_{u}))-\sigma(y(u,v^{\prime},P_{u}))]+e^{n_{2}f_{i}}||\textbf{v}||^{2}$
$\displaystyle+\lambda_{\hat{\theta}}(||\hat{\theta}||^{2})$

where $e$ denotes the natural log; $f_{i}$ denotes the interaction frequency
of the item $v$; $\frac{1}{e^{n_{1}f_{i}}}$ controls the relevance of the item
$v$ to the loss function; and $e^{n_{2}f_{i}}$ controls the scale of the
regularization on the item’s learned latent representation v. And $n_{1}$ and
$n_{2}$ denote hyperparameters that control the impact of the interaction
frequency on $v$’s relevance and regularization scale, and $\lambda_{\theta}$
denotes the regularization parameter to prevent overfitting.

###  3.2. Greedy NDCG Attribute Selector

Once the conversation begins, how do we personalize the elicitation of a
user’s preferences? To achieve this, we propose a greedy NDCG attribute
selector which utilizes an NDCG-oriented attribute selection strategy to help
the CRS identify the attributes that most effectively represent the current
preference space of the user. Since the eventual goal of a CRS is to make
accurate recommendations for the user, we consider the effectiveness of an
attribute equivalent to how much it can increase the recommendation
performance. That is, after being asked by the system, how much can knowing
the user’s preference on that particular attribute contribute to the
refinement of the user’s recommendations? To this end, the NDCG-oriented
greedy attribute selection strategy identifies the attribute with the highest
expected increase in recommendation performance (measured in NDCG), from the
user’s unasked adjacent attributes $P_{cand}$, to ask the user. The workflow
of the proposed greedy NDCG attribute selector is illustrated in Algorithm 2.

Input : the initial representation of the user $\textbf{r}_{u_{0}}$; the
current representation of the user at the $t$-th turn $\textbf{r}_{u_{t}}$;
collected preferences of the user $P_{u}$; the unasked adjacent attributes of
the user $P_{cand}$

Output : the attribute to ask next $p_{ask}$

1 Initialize: $p_{ask}$ = $\phi$; $max\\_ndcg=0$

2

3for  _$p$ in $P_{cand}$_ do

4 Set: $\hat{P}_{u}$ = $P_{u}$ $\cup$ $p$

5 Refine the initial user representation with $\hat{P}_{u}$:

6 ${\textbf{r}}_{u_{t+1}}$ = User Representation Refiner
($\textbf{r}_{u_{0}}$, $\hat{P}_{u}$)

7 Set: $p\\_ndcg$ = NDCG(${\textbf{r}}_{u_{t+1}}$) -
NDCG($\textbf{r}_{u_{t}}$)

8 if  _$p\\_ndcg$ ¿ $max\\_ndcg$_ then

9 $max\\_ndcg$ = $p\\_ndcg$

10 $p_{ask}$ = $p$

11

12

13 end for

return _$p_{ask}$_

Algorithm 2 Greedy NDCG Attribute Selector

It is worth noting $P_{cand}$ contains only the unasked adjacent attributes of
the user – attributes that are contained in the user’s interacted items and
have not been asked before – which significantly reduces the search space (Lei
et al., 2020b). To start, the attribute selector takes in the initial
representation of the user learned from past interactions
$\textbf{r}_{u_{0}}$, the current representation of the user
$\textbf{r}_{u_{t}}$, the set of collected preferences of the user $P_{u}$,
and the set of candidate attributes $P_{cand}$ as inputs. Then for every
unasked candidate attribute $p$ in $P_{cand}$, we compute its expected NDCG
gain. First, we add $p$ to $P_{u}$ to simulate the scenario that the asked
attribute, $p$, will be liked by the user. Then based on the updated $P_{u}$,
denoted as $\hat{P}_{u}$, we use the user representation refiner to compute
the refined representation $\textbf{r}_{u_{t+1}}$ which reflects the system’s
understanding of the user after knowing the user’s preference on $p$. Then, we
use $\textbf{r}_{u_{t+1}}$ and $\textbf{r}_{u_{t}}$ to compute the NDCGs of
the user, both before and after knowing the user’s preference on $p$, and use
the difference to serve as the expected NDCG gain for $p$. Once we have
computed the expected NDCG gains for all the candidate attributes in
$P_{cand}$, we pick the attribute with the highest expected NDCG gain to be
asked next, $p_{ask}$. It is important to note that the NDCG of a user is
computed on the validation set, instead of the testing set. Specifically, we
first rank all the items that are not included the training samples of the
user by taking the dot product of the item’s representation (learned in
Section 3.1) with the input user representation. Then we use the ranked items
of the user to compute the DCG and the groundtruth items of the user (in the
validation set) to compute the Ideal DCG, which together give us the NDCG of
the user.

###  3.3. User Representation Refiner

After eliciting a user’s preferences, a natural next question is how do we
personalize the system’s model of the user? To provide a personalized fusion
for the collected preferences from the interactive preference elicitation
process, we deploy a user representation refiner to refine the initial latent
representation of the user, $\textbf{r}_{u_{0}}$, to be a more accurate
reflection of the user’s current interests, by considering the collected
preferences of the user, $P_{u}$. Formally, we have:

$\mathcal{F}(\textbf{r}_{u_{0}},P_{u})=\hat{\textbf{r}}_{u}$

where $\mathcal{F}$ denotes the user representation refiner which maps the
initial representation of the user, $\textbf{r}_{u_{0}}$, to the refined
representation, $\hat{\textbf{r}}_{u}$, based on the collected preferences of
the user, $P_{u}$.

Input Layer: To fuse the collected preferences in a personalized manner, the
user representation refiner takes both the past and the current interests of
the user, respectively encoded in $\textbf{r}_{u_{0}}$ and $P_{cand}$, into
consideration. First to construct the inputs, we fetch, from $\textbf{R}_{P}$
(learned in Section 3.1), the latent representations of all specified
preferences in $P_{u}$, denoted as $\\{\textbf{r}_{p_{i}}|1\leq
i\leq|P_{u}|\\}$, and stack them together into a matrix
$\textbf{R}_{p_{u}}\in\mathbb{R}^{|P_{u}|\times d}$, where $d$ is the
dimensionality of the latent representation.

Preference Aggregation Layer: Then to account for the contextual consistency
of the neighboring preferences, an attention-based preference aggregation
layer is applied to $\textbf{R}_{p_{u}}$. Following (Vaswani et al., 2017), we
have:

$\hat{\textbf{R}}_{p_{u}}=softmax(\frac{\textbf{R}_{p_{u}}\textbf{W}^{q}\>\textbf{R}_{p_{u}}^{T}\textbf{W}^{k}}{\sqrt{d}})\>\textbf{R}_{p_{u}}\textbf{W}^{v}$

where $\hat{\textbf{R}}_{p_{u}}\in\mathbb{R}^{|P_{u}|\times d}$ denotes a set
of preference representations refined by incorporating the contextual
information of the neighboring preferences. And $\textbf{W}^{q}$,
$\textbf{W}^{k}$, and $\textbf{W}^{v}$ denote the projection matrices. Next,
we apply the weighted-sum to aggregate all the refined preference
representations to construct a new representation that comprehensively
represents the collected preferences of the user as an entity:

$\textbf{z}_{p_{u}}=\sum_{i\in|P_{u}|}\alpha_{i}\cdot\hat{\textbf{r}}_{p_{i}}$

where $\textbf{z}_{p_{u}}$ denotes the aggregated representation that
comprehensively represents all the specified preferences in $P_{u}$ as an
entity; $\hat{\textbf{r}}_{p_{i}}$ denotes a particular updated preference
representation in $\hat{\textbf{R}}_{p_{u}}$. And $\alpha_{i}$ is the weight
learned by the vanilla attention network:

$\alpha_{i}=\textbf{r}_{u_{0}}^{T}\
tanh(\textbf{W}^{c}\textbf{r}_{p_{i}}+\textbf{b}_{1})\quad\alpha_{i}=\frac{exp(\alpha_{i})}{\sum_{n\in
M}exp(\alpha_{n})}$

where $\alpha_{i}$ is a scalar that represents the relevance of the
representation of a particular attribute, $\hat{\textbf{r}}_{p_{i}}$,
$\textbf{W}^{c}$ is the projection matrix, and $\textbf{b}_{1}$ is the bias
vector. Note that the initial latent representation of the user,
$\textbf{r}_{u_{0}}$, is directly used as the ‘query’ in calculation of
$\alpha_{i}$ to enhance personalization in the calculation of the relevance of
a particular attribute, $\alpha_{i}$, to the user. In this way, $\alpha_{i}$
is learned by taking into account – (1) the intrinsic properties of individual
attributes; (2) the contextual consistency of the neighboring attributes; and
(3) the user’s general past interests – so that $\textbf{z}_{p_{u}}$ is a
comprehensive representation of $P_{u}$.

Update Layer: Once $\textbf{z}_{p_{u}}$ has been computed, we concatenate it
with the user’s initial latent representation, $\textbf{r}_{u_{0}}$, to obtain
a concatenated representation $\textbf{r}_{u_{cat}}$ that contains information
of the user’s past and current interests:

$\textbf{r}_{u_{cat}}=\textbf{z}_{p_{u}}\oplus\textbf{r}_{u_{0}}$

Then, a one-layer feed-forward neural network is applied to obtain the refined
representation of the user $\hat{\textbf{r}}_{u}$:

$\hat{\textbf{r}}_{u}=\textbf{W}\textbf{r}_{u_{cat}}+\textbf{b}_{2}$

where W is the weight matrix of the neural network and $\textbf{b}_{2}$ is the
bias vector.

Objective Function: Since the intention of the user representation refiner is
to enhance personalization in fusing the collected preferences of the user
($P_{u}$), the refined representation of the user, $\hat{\textbf{r}}_{u}$,
should have higher predicted affinity scores (calculated using dot-product)
with items that align well with $P_{u}$. To this end, we adopt the standard
pairwise Bayesian Personalized Ranking (BPR) objective function for optimizing
the user representation refiner as a personalized ranking algorithm. Note we
did not use the weighted variation of the BPR loss, Equation 2, since it is
not applicable in this situation. Formally, we have:

$L_{bpr}=\sum_{(v,v^{\prime},P_{u})\in\mathcal{D}}-ln\sigma(\hat{y}(u,v,P_{u})-\hat{y}(u,v^{\prime},P_{u}))+\lambda_{\theta}||\theta||^{2}$

with $\hat{y}(u,v,P_{u})$ is computed as:

(3) $\hat{y}(u,v,P_{u})=\mathcal{F}(\textbf{r}_{u_{0}},P_{u})^{T}\>\textbf{v}$

Here, $\mathcal{D}$ is a set of pairwise instances for the BPR training, $u$
is the user whose embedding is being updated, $\sigma$ is the sigmoid
function, $\lambda_{\theta}$ is the regularization parameter to prevent
overfitting, $\textbf{r}_{u_{0}}$ is the initial user representation, and
$\mathcal{F}$ is the user representation refiner. Note, Equation 3 involves
the use of $\mathcal{F}$ in its computation, which allows $\mathcal{F}$ to be
trained through back-propagation. In addition, Equation 3 drops the second
item-attribute affinity term used in Equation 1. This is because the attribute
information has already been fused into the refined user presentation,
$\hat{\textbf{r}}_{u}$, making the item-attribute affinity term redundant.

Pairwise Instances Sampling: In this section, we introduce the pairwise
instances sampling strategy used to construct $\mathcal{D}$, for training the
user representation refiner. For each user $u$, we randomly sample one of
$u$’s interacted items to serve the positive sample, $v$. Then, we randomly
keep $n$, where $T-1\geq n\geq 1$ (T being the maximum allowed turns of
interaction), attribute(s) from the $v$’s list of attributes to serve as
$P_{u}$. Lastly, we randomly sample one of $u$’s non-interacted items, whose
attributes share a low Jaccard Similarity with $P_{u}$, to serve as the
negative sample, $v^{\prime}$. Compared to the standard pairwise instance
sampling where $P_{u}$ would be $v$’s entire list of attributes, we randomly
select n attributes to keep the user representation refiner robust to a wide
variety of attribute combinations. Note, $v^{\prime}$ is specifically chosen
to be an item that share few attribute(s) with $v$ to emphasize the importance
of $P_{u}$.

###  3.4. Action Selection Policy Learning

Following (Lei et al., 2020b), we adopt a two-layer feed forward neural
network as the policy network and use the standard Deep Q-Learning (Mnih et
al., 2015) for optimization. As shown by previous works (Deng et al., 2021;
Lei et al., 2020b), massive action space hinders both the training and the
performance of RL-based policy networks; thus, we define our action space to
be $\mathcal{A}=\\{a_{ask},\>a_{rec}\\}$. During each interaction with the
user, the policy network takes in the state vector $\textbf{s}_{t}$ and
outputs the Q-values for both $a_{ask}$ and $a_{rec}$ to be their estimated
rewards. The CRS will then choose the action with higher reward and interact
with the user accordingly. The state vector $\textbf{s}_{t}$ is a
concatenation of two vectors:

$\textbf{s}_{t}=\textbf{r}_{u_{0}}\oplus\textbf{s}_{t}^{his}$

where $\textbf{r}_{u_{0}}$ denotes the initial user representation, which is
expected to add personalization into the policy network, and
$\textbf{s}_{t}^{his}$ denotes the conversation history up until turn $t$,
which is expected to guide the system to act more wisely – e.g., if multiple
asked attributes have been accepted, the system might be ready to recommend.
Extending from (Lei et al., 2020b), we define the five kinds of rewards used
for training the policy network: (1) $r_{rec\\_suc}$, a NDCG-oriented positive
reward when recommendation succeeds; (2) $r_{rec\\_fail}$, a negative reward
when the user rejects the recommendation; (3) $r_{ask\\_suc}$, a slightly
positive reward when the user confirms the asked attribute; and (4)
$r_{ask\\_fail}$, a negative reward when the user rejects the asked attribute,
and (5). $r_{stop}$, a strongly negative reward when reaching the maximum
number of turns.

##  4\. Experiments

In this section, we showcase experiments over multiple datasets to evaluate
the performance of the proposed UpsRec approach to answer three key research
questions: RQ1. How does the recommendation performance of the proposed
framework compare to existing CRS approaches? RQ2. Does the proposed greedy
NDCG attribute selector enhance user personalization in the interactive
preference elicitation process? RQ3. Does the proposed user representation
refiner enhance user personalization in the preference fusion process?

###  4.1. Experiments Setup

Dataset. We evaluate the proposed framework on four benchmark datasets that
are widely adopted to evaluate multi-round conversational recommender systems
(Sun and Zhang, 2018; Lei et al., 2020a; Lei et al., 2020b; Xu et al., 2021;
Deng et al., 2021).

  * $\bullet$

Lastfm and Yelp. Lastfm is a dataset used to evaluate music artist
recommendation and Yelp is used to evaluate business recommendation. In (Lei
et al., 2020a), Lei et al. manually categorize the original 8,432 attributes
in Lastfm into 33 coarse-grained categories, and construct a 2-layer taxonomy
to reduce the original 590 attributes to 29 first-layer categories for Yelp.

  * $\bullet$

Lastfm*111https://grouplens.org/datasets/hetrec-2011/ and
Yelp*222https://www.yelp.com/dataset/. In (Lei et al., 2020b), Lei et al.
propose it is unrealistic to manually merge attributes for practical
applications, so they adopt the original attributes for these two datasets.
For a fair comparison, we conduct experiments on both versions.

All datasets contain only the implicit (binary) feedback from the user (e.g.
whether a user has interacted with the item or not). Following previous works
(He et al., 2017; Lei et al., 2020a), we keep users with at least 10
interactions to alleviate data sparsity. Statistics of all the datasets are
summarized in Table 1.

Table 1. Dataset statistics Dataset | #users | #items | #interactions | #attributes
---|---|---|---|---
Lastfm | 1,801 | 7,432 | 76,693 | 33
Lastfm* | 1,801 | 7,432 | 76,693 | 8,432
Yelp | 27,675 | 70,311 | 1,368,606 | 29
Yelp* | 27,675 | 70,311 | 1,368,606 | 590

Baselines. To examine the proposed framework, we compare its performance with
the following start-of-the-art baseline conversational recommendation
approaches.

  * $\bullet$

Matrix Factorization (MF): MF is a collaborative filtering algorithm that is
widely used for many static recommendation tasks. We report its performance to
demonstrate the general effectiveness of CRSs.

  * $\bullet$

Max Entropy (MaxEnt): MaxEnt is a rule-based attribute selection strategy. At
each turn, the policy agent selects the attribute with the highest entropy to
ask. Recommendation is made either when the candidate space is small enough,
or the policy agent runs out of attributes.

  * $\bullet$

CRM (Sun and Zhang, 2018): CRM is a CRS that uses a belief tracker to record a
user’s preference conveyed, and trains a policy network via reinforcement
learning to decide how to interact with the user. We follow (Lei et al.,
2020a) to adapt it to the multi-round conversational setting for fair
comparison.

  * $\bullet$

EAR (Lei et al., 2020a): Similar to CRM, EAR also learns a predictive model to
estimate a user’s preference and trains a policy network to determine whether
to ask more attributes or make recommendations. In addition, it also considers
the feedback from the user to further fine-tune the learned predictive model.

  * $\bullet$

SCPR (Lei et al., 2020b): Extending from EAR, SCPR leverages the concept of
adjacent attributes to construct a knowledge graph to reduce the search space
of attributes and uses Deep Q-Learning (Mnih et al., 2015) to learn a more
efficient policy network.

  * $\bullet$

UNICORN (Deng et al., 2021): Unicorn is the current state-of-the-art CRS. It
applies a dynamic-weighted-graph based RL approach to integrate the
conversation and the recommendation components and deploys an weighted-entropy
action selection strategy to reduce the candidate action space.

Since all baselines are initially proposed under the MCR setting, for a fair
comparison, we retrain the conversational components of all baseline models to
adapt to the proposed MGMCR setting as introduced in Section 2 and adopt a
NDCG-oriented reward function to optimize the policy networks of all baseline
models.

Metrics. To evaluate a CRS under the MGMCR setting, we adopt the Top-K
metrics: Hit Rate (HT@K) and Normalized Discounted Cumulative Gain (NDCG@K).
HT@K could be considered as an extension of the widely adopted (Sun and Zhang,
2018; Deng et al., 2021; Lei et al., 2020a; Xu et al., 2021) Success Rate (SR)
metric under the MCR setting. Instead of measuring if the one groundtruth item
exists in the K recommended items, HT@K measures what fraction of the K
recommended items are of the groundtruth items. To provide a more
comprehensive evaluation, NDCG@K is also adopted to account for the rank order
in recommendations. In addition, we also report Average Turns (AT) to evaluate
the efficiency of the system. If the conversation session reaches the
predefined maximum turn T, then the turn count for the session is T.

Training Details. Following (Lei et al., 2020a), we split the user-item
interactions in the ratio of 7:2:1 for training, validation, and testing, and
set the size of the recommendation list $k$ to 10 and the maximum turn $T$ to
15. The training process is divided into offline and online stages. The
offline training is intended for building: (i) the predictive model (FM) for
estimating the latent interests of the user; and (ii) the user representation
refiner for incorporating the collected preferences to iteratively refine the
representation of the user. Afterwards, we follow the user simulation process,
introduced in Section 2, to conduct online training to optimize the policy
agent (network) on how to best interact with the user via reinforcement
learning. We tune all hyper-parameters on the validation set and set them as
follows: the dimensionality, $d$, of all the user, item, attribute
representations are set to 64. SGD is used for optimizing the weighted BPR
loss adopted in the predictive model used in Section 3.1, with $n_{1}$ set to
7, $n_{2}$ set to 8, $\lambda$ set to .001. Adam is used for optimizing the
standard BPR loss adopted in the user representation refiner, with $\lambda$
set to .002. For constructing the pairwise instance $\mathcal{D}$, we collect
15,000 pairwise samples for each user with the Jaccard Similarity threshold
set to 0.33. For training the policy network, we set the experience replay
memory size to 50,000, the sample batch size to 256, and the discount factor
$\gamma$ to 0.95.

Table 2. Overall Recommendation Performance Comparison, with NDCG@10
$\uparrow$, HT@10 $\uparrow$, and AT $\downarrow$ used as evaluation metrics.
Boldface indicates statistically significant improvement ($\rho$ ¡ 0.01) over
all baselines.

| Lastfm | Lastfm* | Yelp | Yelp*
---|---|---|---|---
| NDCG@10 | HT@10 | AT | NDCG@10 | HT@10 | AT | NDCG@10 | HT@10 | AT | NDCG@10 | HT@10 | AT
MF | .0415 | .0684 | NA | .0392 | .0679 | NA | .0199 | .0419 | NA | .0165 | .0305 | NA
MaxEnt | .1033 | .1289 | 12.80 | .1819 | .2253 | 8.36 | .2668 | .2870 | 8.44 | .1694 | .2041 | 13.10
CRM | .1359 | .2006 | 11.13 | .1710 | .2237 | 7.60 | .2422 | .2920 | 7.24 | .1072 | .1289 | 13.69
EAR | .1455 | .2111 | 10.75 | .1765 | .2283 | 7.55 | .2503 | .3012 | 7.11 | .1095 | .1307 | 13.71
SCPR | .1671 | .2407 | 9.94 | .2079 | .2752 | 6.90 | .2529 | .3016 | 7.09 | .3031 | .2924 | 12.77
Unicorn | .1982 | .2808 | 8.95 | .2239 | .2818 | 6.84 | .2557 | .3041 | 6.99 | .3118 | .2977 | 12.92
UpsRec | .3144 | .3358 | 5.99 | .3278 | .3437 | 5.89 | .3631 | .3805 | 5.72 | .3853 | .3754 | 6.57

###  4.2. Performance Comparison (RQ1)

As shown in Table 2, UpsRec significantly outperforms all the baseline models
in both recommendation performance (measured in both NDCG@10 and HT@10) and
conversation efficiency (measured in AT). This validates our hypothesis that
enhancing user personalization, in both the interactive preference elicitation
process and the preference incorporation process, is an effective strategy to
build a CRS. We also make the following observations:
(1). All conversational recommendation approaches significantly outperform the
static matrix factorization (MF) model, showing the promising functionality of
CRSs in general.
(2). For all baseline models, we observe a significant gap between the two
recommendation performance metrics, especially on the Lastfm and the Lastfm*
datasets. In general, the measured NDCG@10 is significantly lower than the
measured HT@10. This is because NDCG@10 is measured by taking the rank order
of the groundtruth items into account. Since baseline models only rely on
attribute-filtering to fuse the collected preferences of the user, the user
representation – used for ranking the candidate items to calculate NDCG – is
often not an accurate reflection of the user’s current preference space, since
it has never been updated after learned from the past user-item interactions.
As a result, even when the groundtruth items can be successfully recommended,
they are usually ranked low on the recommendation list. Compared to the
baseline models, UpsRec show similar recommendation performance measured in
both NDCG@10 and HT@10. This is because UpsRec integrates the user
representation refiner to iteratively update a user’s representation whenever
a new preference has been collected. In turn, the user representation used for
ranking the candidate items will be a more realistic reflection of the user’s
current interests, resulting in groundtruth items being ranked higher on the
recommendation list when they are successfully recommended.
(3). Comparing to all other baseline CRSs, UpsRec shows consistently higher
conversational efficiency (measured by AT) and recommendation performance
(measured by both NDCG@10 and HT@10) across all four tested datasets,
especially on the Yelp* dataset. This indicates that UpsRec can scale up to
both large attribute spaces (8,432 attributes in Lastfm* v.s. 29 attributes in
Yelp) and larger number of user-item interactions (1,368,606 interactions in
Yelp* v.s. 76,693 interactions in Lastfm), which is critical for deploying
CRSs in practice.

Figure 1. CRS Performance vs Conversation Turns

Figure 2. Asking Likelihood of the Top 40 Most Frequently Asked Attributes in
UpsRec and UNICORN

###  4.3. Enhancing User Personalization in Interactive Preference Elicitation
(RQ2)

To determine whether the proposed greedy NDCG attribute selector enhances user
personalization in the interactive preference elicitation process, we examine
both the effectiveness and the efficiency of the asked attributes in
representing user’s actual preference space.

Since the goal of a CRS is to make highly personalized recommendations, we
consider the effectiveness of an asked attribute equivalent to how much it can
increase the recommendation performance. As shown in Figure 1, knowing the
user’s preferences on the attributes – selected by the proposed greedy NDCG
attribute selector – brings significantly higher increments in recommendation
performance than knowing the user’s preferences on the attributes selected by
all the baseline CRSs. This indicates that the proposed greedy NDCG attribute
selector selects attributes that are more relevant to the user, such that
knowing the user’s preferences on these attributes helps the system gain a
more comprehensive understanding of the user’s current interests. Next, we
consider the efficiency of the attributes selected by the greedy NDCG
attribute selector. From Figure 1, for UpsRec, we observe a strong diminishing
increase in recommendation performance after asking five attributes. This
indicates that the asked attributes are highly efficient in representing the
user’s actual preference space, such that knowing the user’s preferences on
four asked attributes helps the system to achieve roughly 80% of its best
recommendation performance on the Lastfm* dataset or nearly 90% on the Yelp*
dataset. Compared to UpsRec, all other baselines show marginal increases in
recommendation performance after knowing the user’s preference on the first
few asked attributes (Figure 1). This is due to the lack of user
personalization in the interactive preference elicitation. Compared to the
proposed NDCG-oriented attribute selection strategy, the entropy-based
attribute selection strategy adopted by existing works cannot precisely
identify the set of attributes that most efficiently represent the user’s
actual preference space. To better illustrate, we compare the averaged asking
likelihood of the top-40 most frequently asked attributes in both UNICORN and
UpsRec (Figure 2). The asking likelihood of an attribute can be interpreted as
the percentage of users that the system predicts will find the asked attribute
relevant. For instance, the top-three most frequently asked attributes in
UNICORN have an average asking likelihood of 0.7. That is, UNICORN thinks that
70% of the users will consider these three attributes important in expressing
their preferences. However, due to the lack of user personalization in the
weighted-entropy attribute selection, such prediction is often inaccurate. As
can be observed in Figure 2, UNICORN exhibits a higher asking likelihood for
almost all the top-40 most frequently asked attributes, on both the LastFM*
and the Yelp* dataset (Table 2). This indicates that many of the asked
attributes in UNICORN are inefficient in representing the user’s actual
preference space. A sample conversation of such, on the Yelp* business
recommendation dataset, is shown in Figure 4. While the attributes Restaurant
and affordable have high calculated weighted-entropy, knowing the user
preference on them has virtually no contribution to the recommendation
performance of the system. Compared to the weighted-entropy attribute
selection strategy used in UNICORN, the proposed NDCG-oriented attribute
selection strategy identifies attributes that more efficiently represent the
preference space of the user.

###  4.4. Enhancing User Personalization in Fusing Preferences (RQ3)

A personalized preference fusing process aims to consistently increase the
system’s understanding of the user by providing a personalized modeling based
on the user’s collected preferences. Therefore, to determine the effectiveness
of the proposed user representation refiner in enhancing user personalization
in the collected-preference incorporation process, we examine whether the
refined representation of the user more precisely reflects the user’s actual
items of interest and attributes of interest, compared to the current state-
of-the-art CRS – UNICORN. To examine if the refined user representation more
precisely reflects the user’s actual items of interest, we compare the AUC
Scores on groundtruth item prediction (computed following (Rendle et al.,
2012)) between UpsRec and UNICORN. As shown in Figure 3, by iteratively
refining the representation of the user once a new preference of the user has
been collected, UpsRec’s AUC gradually increases as the conversation goes on.
Note UNICORN adopts a different strategy for learning the initial user
representations which results in different AUC scores at turn 0; however,
since UNICORN (and all baselines) performs no refinement on the learned
initial user representations, its AUC remains constant and is surpassed by
UpsRec’s in one turn. This indicates that compared to the attribute-filtering
approach used in UNICORN, the proposed user representation refiner is able to
help the system obtain a uniquely personalized modeling of the user, which
more precisely reflects the user’s items of interests. A similar pattern is
also observed in (Figure 3). As the conversation goes on, UpsRec’s AUC on
groundtruth attributes prediction also gradually surpasses the AUC of UNICORN.
Note, the AUC score on groundtruth attributes prediction is computed by
treating all the attributes that exist in any of the groundtruth items as the
positive attributes and the attributes that do not as the negative attributes.
A higher AUC score indicates that the proposed user representation refiner is
able to help the system establish a more personalized modeling of the user,
which facilitates the process of identifying the attributes that the user
would find relevant in expressing her preference space.

Figure 3. AUC Scores on GroundTruth(GT) Item Prediction and GT Attribute
Prediction at Different Conversation Turns

##  5\. Related Work

Traditional recommender systems challenges of (i) how to capture a user’s
precise preferences; and (ii) how to provide a human-interpretable
justification for their recommendations (Jannach et al., 2021; Gao et al.,
2021). Many existing works have approached these challenges with auxiliary
data to better interpret user intentions (Xian et al., 2019; He et al., 2015;
Cheng et al., 2019; Zhang et al., 2020a, 2014). In another direction, the
emergence of conversational recommenders that dynamically interact with users
through real-time interactions are able to elicit current and detailed
preferences directly from users (Jannach et al., 2021). In this way, a CRS can
precisely learn a user’s interests along with a series of actions that provide
justification evidence.

In general, a CRS is composed of two components – the conversation component
and the recommendation component. The conversation component decides which
action (further collecting user preference or calling upon the recommender
component to make recommendations) to take during each interaction with the
user. The recommendation component learns a set of user, item, and attribute
embeddings for ranking all the candidate items. While early works on CRSs
primarily rely on template or choice-based conversation components for
collecting user preferences (Graus and Willemsen, 2015; Jiang et al., 2014;
Zou et al., 2020b), recent works have explored the possibilities of more
effective preference elicitation methods and conversational strategies with
the help of reinforcement learning. For example, CRM (Sun and Zhang, 2018)
proposes a deep reinforcement-based conversation component that builds a
personalized preference elicitation process. Inspired by CRM, EAR (Lei et al.,
2020a) proposes a novel optimization function to enhance the richness of the
embeddings learned by the recommendation component, then designs a three-stage
framework that further strengthens the interaction between the recommender
component and the conversation component. While sharing the same recommender
component, SCPR (Lei et al., 2020b) integrates knowledge graphs to improve the
reasoning ability of the conversation component. Further utilizing the
informativeness of knowledge graphs, UNICORN (Deng et al., 2021) proposes a
dynamic weighted graph based reinforcement approach that systematically
integrate the recommendation and the conversation components. However, the
above-introduced approaches (Sun and Zhang, 2018; Lei et al., 2020a; Lei et
al., 2020b; Deng et al., 2021; Xu et al., 2021) all adopt an entropy-based
attribute-selection strategy which limits user personalization in the
preference elicitation process. In addition, these approaches only make coarse
uses of the obtained user preferences, refraining the system from gaining a
deeper understanding of the user. In this work, we propose a novel attribute-
selection and a preference fusion strategy that jointly address these two
issues. Note that while there are other research directions in CRSs including
dialogue understanding (Zheng et al., 2020; Jin et al., 2018; Li et al., 2016;
Zou et al., 2020a; Wang et al., 2022), response generation (Zhou et al.,
2020b; Lin, 2004; Liu et al., 2020; Papineni et al., 2002), and exploration-
exploitation trade-offs (Zhang et al., 2020b; Yu et al., 2019; Zhou et al.,
2020a; Lee et al., 2019; Iovine et al., 2022) those are not the focus of this
work.

##  6\. Conclusion

In this work, we propose a novel framework, named UpsRec, with two unique
features aiming to enhance user personalization in both the interactive
preference elicitation process and the preference fusion process. The
experimental results on four frequently adopted conversational recommendation
datasets show that the proposed framework not only outperforms all baseline
CRS methods but also brings a more personalized experience for the user under
the more realistic and generalizable MGMCR setting.

Figure 4. Sample Conversations by UNICORN and UpsRec

## References

  * (1)
  * Cheng et al. (2019) Zhiyong Cheng, Xiaojun Chang, Lei Zhu, Rose C Kanjirathinkal, and Mohan Kankanhalli. 2019.  MMALFM: Explainable recommendation by leveraging reviews and images.  _ACM Transactions on Information Systems (TOIS)_ 37, 2 (2019), 1–28.
  * Deng et al. (2021) Yang Deng, Yaliang Li, Fei Sun, Bolin Ding, and Wai Lam. 2021.  Unified conversational recommendation policy learning via graph-based reinforcement learning. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1431–1441.
  * Gao et al. (2021) Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2021.  Advances and challenges in conversational recommender systems: A survey.  _AI Open_ 2 (2021), 100–126.
  * Graus and Willemsen (2015) Mark P. Graus and Martijn C. Willemsen. 2015.  Improving the User Experience during Cold Start through Choice-Based Preference Elicitation. In _RecSys_.
  * He et al. (2015) Xiangnan He, Tao Chen, Min-Yen Kan, and Xiao Chen. 2015\.  Trirank: Review-aware explainable recommendation by modeling aspects. In _Proceedings of the 24th ACM international on conference on information and knowledge management_. 1661–1670.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _Proceedings of the 26th international conference on world wide web_. 173–182.
  * Iovine et al. (2022) Andrea Iovine, Pasquale Lops, Fedelucio Narducci, Marco de Gemmis, and Giovanni Semeraro. 2022\.  An empirical evaluation of active learning strategies for profile elicitation in a conversational recommender system.  _Journal of Intelligent Information Systems_ (2022), 1–26.
  * Jannach et al. (2021) Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2021\.  A survey on conversational recommender systems.  _ACM Computing Surveys (CSUR)_ 54, 5 (2021), 1–36.
  * Jiang et al. (2014) Hai Jiang, Xin Qi, and He Sun. 2014.  Choice-based recommender systems: a unified approach to achieving relevancy and diversity. In _Operations Research_.
  * Jin et al. (2018) Xisen Jin, Wenqiang Lei, Zhaochun Ren, Hongshen Chen, Shangsong Liang, Yihong Zhao, and Dawei Yin. 2018.  Explicit state tracking with semi-supervisionfor neural dialogue generation. In _Proceedings of the 27th ACM International Conference on Information and Knowledge Management_. 1403–1412.
  * Lee et al. (2019) Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. 2019.  Melu: Meta-learned user preference estimator for cold-start recommendation. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1073–1082.
  * Lei et al. (2020a) Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020a.  Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In _WSDM_.
  * Lei et al. (2020b) Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat-Seng Chua. 2020b.  Interactive path reasoning on graph for conversational recommendation. In _KDD_.
  * Li et al. (2016) Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. 2016\.  Deep reinforcement learning for dialogue generation.  _arXiv preprint arXiv:1606.01541_ (2016).
  * Lin et al. (2022a) Allen Lin, Jianling Wang, Ziwei Zhu, and James Caverlee. 2022a.  Quantifying and mitigating popularity bias in conversational recommender systems. In _Proceedings of the 31st ACM International Conference on Information & Knowledge Management_. 1238–1247.
  * Lin (2004) Chin-Yew Lin. 2004\.  Rouge: A package for automatic evaluation of summaries. In _Text summarization branches out_. 74–81.
  * Lin et al. (2022b) Shuo Lin, Ziwei Zhu, Jianling Wang, and James Caverlee. 2022b.  Towards Fair Conversational Recommender Systems.  _arXiv preprint arXiv:2208.03854_ (2022).
  * Liu et al. (2020) Zeming Liu, Haifeng Wang, Zheng-Yu Niu, Hua Wu, Wanxiang Che, and Ting Liu. 2020\.  Towards conversational recommendation over multi-type dialogs.  _arXiv preprint arXiv:2005.03954_ (2020).
  * Mnih et al. (2015) Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015\.  Human-level control through deep reinforcement learning.  _nature_ 518, 7540 (2015), 529–533.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002\.  Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th annual meeting of the Association for Computational Linguistics_. 311–318.
  * Qin et al. (2014) Lijing Qin, Shouyuan Chen, and Xiaoyan Zhu. 2014.  Contextual combinatorial bandit and its application on diversified online recommendation. In _Proceedings of the 2014 SIAM International Conference on Data Mining_. SIAM, 461–469.
  * Qiu et al. (2019) Tian Qiu, Chi Wan, Xiao-Fan Wang, and Zi-Ke Zhang. 2019\.  User interest dynamics on personalized recommendation.  _Physica A: Statistical Mechanics and its Applications_ 525 (2019), 965–977.
  * Rendle (2010) Steffen Rendle. 2010\.  Factorization machines. In _2010 IEEE International conference on data mining_. IEEE, 995–1000.
  * Rendle et al. (2012) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012.  BPR: Bayesian personalized ranking from implicit feedback.  _arXiv preprint arXiv:1205.2618_ (2012).
  * Sun and Zhang (2018) Yueming Sun and Yi Zhang. 2018.  Conversational recommender system. In _SIGIR_.
  * Tan et al. (2021) Qiaoyu Tan, Jianwei Zhang, Jiangchao Yao, Ninghao Liu, Jingren Zhou, Hongxia Yang, and Xia Hu. 2021.  Sparse-interest network for sequential recommendation. In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_. 598–606.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wang et al. (2022) Jianling Wang, Ya Le, Bo Chang, Yuyan Wang, Ed H Chi, and Minmin Chen. 2022\.  Learning to Augment for Casual User Recommendation. In _Proceedings of the ACM Web Conference 2022_. 2183–2194.
  * Xian et al. (2019) Yikun Xian, Zuohui Fu, Shan Muthukrishnan, Gerard De Melo, and Yongfeng Zhang. 2019.  Reinforcement knowledge graph reasoning for explainable recommendation. In _Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval_. 285–294.
  * Xu et al. (2021) Kerui Xu, Jingxuan Yang, Jun Xu, Sheng Gao, Jun Guo, and Ji-Rong Wen. 2021.  Adapting user preference to online feedback in multi-round conversational recommendation. In _Proceedings of the 14th ACM international conference on web search and data mining_. 364–372.
  * Yu et al. (2019) Tong Yu, Yilin Shen, and Hongxia Jin. 2019.  A visual dialog augmented interactive recommender system. In _Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining_. 157–165.
  * Zhang et al. (2020b) Xiaoying Zhang, Hong Xie, Hang Li, and John CS Lui. 2020b.  Conversational contextual bandit: Algorithm and application. In _Proceedings of the web conference 2020_. 662–672.
  * Zhang et al. (2020a) Yongfeng Zhang, Xu Chen, et al. 2020a.  Explainable recommendation: A survey and new perspectives.  _Foundations and Trends® in Information Retrieval_ 14, 1 (2020), 1–101.
  * Zhang et al. (2018) Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018.  Towards Conversational Search and Recommendation: System Ask, User Respond. In _CIKM_.
  * Zhang et al. (2014) Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014\.  Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In _Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval_. 83–92.
  * Zhang et al. (2022) Yiming Zhang, Lingfei Wu, Qi Shen, Yitong Pang, Zhihua Wei, Fangli Xu, Bo Long, and Jian Pei. 2022\.  Multiple Choice Questions based Multi-Interest Policy Learning for Conversational Recommendation. In _Proceedings of the ACM Web Conference 2022_. 2153–2162.
  * Zheng et al. (2020) Yinhe Zheng, Rongsheng Zhang, Minlie Huang, and Xiaoxi Mao. 2020\.  A pre-training based personalized dialogue generation model with persona-sparse data. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 9693–9700.
  * Zhou et al. (2020a) Chunyi Zhou, Yuanyuan Jin, Xiaoling Wang, and Yingjie Zhang. 2020a.  Conversational music recommendation based on bandits. In _2020 IEEE International Conference on Knowledge Graph (ICKG)_. IEEE, 41–48.
  * Zhou et al. (2020b) Kun Zhou, Wayne Xin Zhao, Shuqing Bian, Yuanhang Zhou, Ji-Rong Wen, and Jingsong Yu. 2020b.  Improving conversational recommender systems via knowledge graph based semantic fusion. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1006–1014.
  * Zou et al. (2020a) Jie Zou, Yifan Chen, and Evangelos Kanoulas. 2020a.  Towards question-based recommender systems. In _Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval_. 881–890.
  * Zou et al. (2020b) Lixin Zou, Long Xia, Yulong Gu, Xiangyu Zhao, Weidong Liu, Jimmy Xiangji Huang, and Dawei Yin. 2020b.  Neural interactive collaborative filtering. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 749–758.