# Improving Transformer-based Sequential Recommenders
through Preference Editing

Muyang Ma  Shandong UniversityQingdaoChina<EMAIL_ADDRESS>,  Pengjie
Ren∗ Shandong UniversityQingdaoChina<EMAIL_ADDRESS>,  Zhumin Chen
Shandong UniversityQingdaoChina<EMAIL_ADDRESS>,  Zhaochun Ren
Shandong UniversityQingdaoChina<EMAIL_ADDRESS>,  Huasheng Liang
WeChat, TencentShenzhenChina<EMAIL_ADDRESS>,  Jun Ma  Shandong
UniversityQingdaoChina<EMAIL_ADDRESS>and  Maarten de Rijke
0000-0002-1086-0202 University of Amsterdam and Ahold DelhaizeAmsterdam and
ZaandamThe Netherlands<EMAIL_ADDRESS>
(2021)

###### Abstract.

One of the key challenges in sequential recommendation (SR) is how to extract
and represent user preferences. Traditional SR methods rely on the next item
as the supervision signal to guide preference extraction and representation.
We propose a novel learning strategy, named preference editing. The idea is to
force the SR model to discriminate the common and unique preferences in
different sequences of interactions between users and the recommender system.
By doing so, the SR model is able to learn how to identify common and unique
user preferences, and thereby do better user preference extraction and
representation. We propose a transformer based SR model, named MrTransformer
(Multi-preference Transformer), that concatenates some special tokens in front
of the sequence to represent multiple user preferences and makes sure they
capture different aspects through a preference coverage mechanism. Then, we
devise a preference editing-based self-supervised learning mechanism for
training MrTransformer that contains two main operations: preference
separation and preference recombination. The former separates the common and
unique user preferences for a given pair of sequences. The latter swaps the
common preferences to obtain recombined user preferences for each sequence.
Based on the preference separation and preference recombination operations, we
define two types of self-supervised learning loss that require that the
recombined preferences are similar to the original ones, and that the common
preferences are close to each other.

We carry out extensive experiments on two benchmark datasets. MrTransformer
with preference editing significantly outperforms state-of-the-art SR methods
in terms of Recall, MRR and NDCG. We find that long sequences whose user
preferences are harder to extract and represent benefit most from preference
editing.

Transformer-based sequential recommendation, Self-supervised learning, User
preference extraction and representation

∗Corresponding author.

††copyright: acmcopyright††journalyear: 2021††doi:
xx.xxxx/xxxxxxx.xxxxxxx††price: 15.00††isbn: 978-1-4503-9999-9/18/06††journal:
TOIS††ccs: Information systems Recommender systems

##  1\. Introduction

Sequential recommendation (SR) methods aim to predict the next item that the
user is most likely to interact with based on his/her past interactions, such
as clicking on products or watching movies. One of the key challenges faced by
SR approaches is extract and represent user preferences historical interaction
sequences (Yuan et al., 2020b; Sun et al., 2020a). Traditional methods for SR
rely solely on predicting the next item to guide user preference extraction
and representation (Donkers et al., 2017; Zhuang et al., 2017; Wu et al.,
2019b; Pan et al., 2020b, a; Ren et al., 2020b). So far, correlations between
interactions have not been well investigated as a supervision signal that can
guide user preference extraction and representation.

Self-supervised learning (SSL) can automatically generate a supervision signal
to learn representations of the data or to automatically label a dataset.
self-supervised learning (SSL) has made great progress in computer vision (CV)
(Zhai et al., 2019; Sun et al., 2019b; van den Oord et al., 2018) and neural
language processing (NLP) (Devlin et al., 2019; Lan et al., 2020; Lewis et
al., 2020; Felbo et al., 2017; Wang et al., 2020c). It has been introduced to
SR as well. SSL methods can enhance the representations of learned users/items
and in a way to alleviate the data sparsity of cold-start users/items through
exploring the intrinsic correlation among sequences of interactions (Yao et
al., 2020; Wu et al., 2020c; Liu et al., 2021b; Yu et al., 2021). Existing SSL
methods for SR can be divided into two categories: masked item prediction
(MIP) and item contrastive learning (ICL). masked item prediction (MIP) is
inspired by masked language modeling in NLP (Devlin et al., 2019), which
randomly masks items from the sequence and tries to predict the masked items
based on the remaining information (Yuan et al., 2020a; Yao et al., 2020; Sun
et al., 2019a). Different from natural language, there is no natural
grammatical constraint that a sequence of items satisfies. Given the current
sequence, item contrastive learning (ICL) samples negative target items to
guide the model to learn the difference between the positive and negative
target items (Zhou et al., 2020a; Ma et al., 2020; Zhou et al., 2020b). The
performance largely depends on the negative sampling strategy used (He et al.,
2020). These studies propose new SSL signals by exploring a current
interaction sequence itself but neglect the relation between different
sequences. Moreover, so far SSL methods have not made a direct connection to
user preference extraction and representation.

Figure 1. Schematic comparison of a traditional recommender system and
MrTransformer (PE). Traditional recommender systems models the user preference
as a single vector. MrTransformer (PE) identifies common and unique
preferences among different interaction sequences.

In this paper, we propose a novel learning strategy, named preference editing,
that focuses on discriminating the common and unique preference
representations between different interaction sequences. For example, in
Figure 1, users $U_{i}$ and $U_{j}$ have interacted in different ways with a
recommender system in the beauty domain, giving rise to two interaction
sequences. Their common preference is lipstick. User $U_{i}$ also prefers
perfume and this brand, and user $U_{j}$ prefers facial care products. Our
proposed sequential recommender MrTransformer (with Preference Editing) is
able to identify common and unique preferences so as to get better preference
representations.

Preference editing involves two operations to learn to edit the user
preferences in a given pair of sequences: preference separation and preference
recombination. The former separates the common and unique preference
representations for a pair of sequences. The latter swaps the common user
preference representations so as to get recombined user preference
representations for each interaction sequence. Specifically, given a pair of
interaction sequences, we first obtain their preference representations
$P_{1}$ and $P_{2}$. Then, we extract the common preference representations
$c_{1}$ and $c_{2}$, and the unique preference representations $u_{1}$ and
$u_{2}$, respectively, through the preference separation operation. After
that, we combine $c_{2}$ and $u_{1}$ to get the recombined preferences
$P^{\prime}_{1}$, and we combine $c_{1}$ and $u_{2}$ to get the recombined
preferences $P^{\prime}_{2}$ through the preference recombination operation.
Based on the above process, we can devise two types of SSL signals:  (1) We
use the recombined preferences to predict the next item in both sequences; and
(2) We require that the recombined preferences (e.g., $P^{\prime}_{1}$) are as
similar as possible to the original preferences (e.g., $P_{1}$), and that the
common preference representations are close to each other (i.e.,
$c_{1}=c_{2}$).  By doing so, we force the preference extraction model to
learn how to identify and edit user preferences so as to do better user
preference extraction and representation.

To extract multiple user preferences, we propose a transformer-based network,
named MrTransformer. Most studies on SR assume that the user preference is
centralized (Ye et al., 2020; Liu et al., 2021a; Ren et al., 2020a), i.e.,
they assume that there is only one main user preference for each sequence.
Consequently, they usually model the user preference by a single vector. For
example, in Figure 1, the traditional methods only obtain a mixed preference
representation for each sequence while they ignore relations between other
sequences. Recent research has investigated how to identify multiple
preferences through a multi-head attention mechanism on top of RNN-based
methods (Ashkan et al., 2015; Chen et al., 2020; Guo et al., 2020). But to the
best of our knowledge, this has not been explored in transformer-based
methods. Therefore, we incorporate a preference identification module into a
transformer-based model, BERT4Rec (Sun et al., 2019a), which extracts multiple
user preferences by concatenating some special tokens at the start of the each
sequence. When feeding an interaction sequence to the MrTransformer, we add
some special tokens in the beginning with each capturing a particular of
preference by attending to different items in the sequence. The outputs
corresponding to the special tokens are considered as preference
representations. To make sure that the preference representations differ from
one another and capture different user preferences from the sequence, we
devise an extra regularization terms. Specifically, we maintain an attention
vector for each preference so as to keep track of attention distributions for
all items in the sequences being considered, which encourages coverage of the
entire sequence.

To assess MrTransformer (PE), we carry out extensive experiments on two
benchmark datasets: Amazon-Beauty and ML-100k. The results show that
MrTransformer with preference editing significantly outperforms state-of-the-
art baselines on both datasets in terms of Recall, MRR and NDCG. We also find
that long sequences whose user preferences are harder to extract and represent
benefit the most from preference editing. Especially, when the sequence length
is between 20 and 30, preference editing achieves the biggest improvements.

To sum up, the main contributions of this work are as follows:

  * •

We devise a novel self-supervised learning method, preference editing, for
SRs.

  * •

We propose a multi-preference transformer-based model, MrTransformer, for SRs.

  * •

We demonstrate the effectiveness of MrTransformer and preference editing
through extensive experiments on two benchmark datasets.

##  2\. Related Work

In this section, we survey related work from two categories: transformer-based
SR and self-supervised learning for SR.

###  2.1. Transformer-based sequential recommendation

Various neural architectures or mechanisms have successfully been applied to
SRs. These include recurrent neural networks (Hidasi et al., 2016a; Li et al.,
2017; Ma et al., 2019; Sun et al., 2018), convolutional neural networks (Tang
and Wang, 2018; Sun et al., 2020b), graph neural networks (Wang et al., 2020d;
Qiu et al., 2020; Yu et al., 2020), reinforcement learning (RL) (Xin et al.,
2020; Wang et al., 2020b), copy mechanism (Ren et al., 2019), memory networks
(Huang et al., 2018; Chen et al., 2018; Wang et al., 2019). Transformer-based
methods have recently been proven to be effective (Kang and McAuley, 2018; Sun
et al., 2019a; Xie et al., 2020).

Kang and McAuley (2018) propose SASRec which introduces a self-attention
mechanism (the most important component in the transformer) to SRs to identify
important items from interactions. Several variants have been proposed to
improve upon SASRec. Mi et al. (2020) argue that traditional SRs use SASRec as
a basic sequence representation extractor, and propose a continual learning
setup with an adaptive distillation loss to update the recommender
periodically as new data streams in. Li et al. (2020) propose TiSASRec, a
variant of SASRec, to explicitly model the timestamps of items in sequences to
explore the influence of different time intervals on next item prediction. Luo
et al. (2020) argue that even the same item can be represented differently for
different users at the same time step; they propose a collaborative self-
attention network to learn sequence representations and predict the
preferences of the current sequence by investigating neighborhood sequences.
Sun et al. (2019a) adopt a bidirectional transformer to predict masked items
in sequences based on the surrounding items. Wang et al. (2020a) equip the
transformer with hyper-graph neural networks to capture dynamic
representations of items across time and users. Xie et al. (2020) propose
three data augmentation approaches (crop/mask/reorder) to pre-train a
transformer-based model to get user and sequence representations, and then
fine-tune it on the SR task.

Previous has also investigated how to combine auxiliary tasks or information
with SR based on transformer. Cho et al. (2020) introduce multiple types of
position embeddings by considering timestamp information, and propose a self-
attention based model, in which each attention head uses a different position
embedding. Wu et al. (2020a) point out that previous studies ignore the
temporal and context information when modeling the influence of a historical
item to the current prediction; they propose a contextualized temporal
attention mechanism to weigh the influence of historical interactions not only
on what items to interact with, but also when and how the interactions took
place. Lin et al. (2020) argue that modeling users’ global preferences only
based on their historical interactions is imperfect and the users’ preference
is uncertain; they propose a FISSA solution, which fuses item similarity
models with self-attention networks to balance the local and global user
preference representations by taking the information of the candidate items
into account. Wu et al. (2020b) argue propose a personalized transformer model
with a recent regularization technique called stochastic shared embeddings
(SSE) (Wu et al., 2019a) to overcome overfitting caused by simply adding user
embeddings.

Although the studies listed above have proposed various transformer-based SR
models, they all assume that the user preference is centralized, i.e., they
assume that there is a single main user preference for each sequence, and use
a single vector to model the main user preference. No previous work has
considered how to identify users’ multiple preferences behind the interaction
sequence. In contrast, we extract multiple user preferences and represent them
using distributed vectors.

###  2.2. Self-supervised learning for SR

Previous approaches to SR are typically trained by predicting the next
interaction, which is prone to suffer from data sparsity (Yao et al., 2020;
Xie et al., 2020). To mitigate this, some work explores SSL and derives self-
supervised signals to enhance the learning of item and sequence
representations.

One category of SSL approaches is masked item prediction (MIP), which randomly
masks some items in an interaction sequence and tries to predict these masked
items based on the remaining information. Sun et al. (2019a) adopt a BERT-like
training scheme, which predicts the masked items in the sequence based on
surrounding items. Instead of masking items, Yao et al. (2020) propose to mask
sparse categorical features of items; they mask or dropout some categorical
feature embeddings to learn internal relations between two sets of categorical
features. Yuan et al. (2020a) represent and transfer user representation
models for serving downstream tasks where only limited data exists through a
pretrain-finetune strategy; in the pretraining stage, they randomly mask a
certain percentage of items in the sequence and then predict the masked items
to get user preference representations.

Another category of SSL approaches to SR is item contrastive learning (ICL),
which samples negative items to guide the model to learn the difference
between positive and negative target items. Zhou et al. (2020a) propose a
fixed-size queue to store items’ representations computed in previous batches,
and use the queue to sample negative examples for each sequence. Ma et al.
(2020) propose a sequence-to-sequence training strategy to mine extra
supervision by looking at the longer-term future; they first use a
disentangled encoder to obtain multiple representations of a given sequence
and predict the representation of the future sub-sequence given the
representation of the earlier sequence; sequence representations that are not
from the same sequence as the earlier ones are considered as negative samples.
Zhou et al. (2020b) devise four auxiliary self-supervised objectives to learn
correlations among four types of data (item attributes, items, sub-sequences,
and sequences), respectively, by utilizing the mutual information maximization
principle. Xie et al. (2020) propose three data augmentation methods
(crop/mask/reorder); then they encode the sequence representation by
maximizing the agreement between different augmented methods of the same
sequence in the latent space. Xia et al. (2021) model sequences as a
hypergraph and propose a dual channel hypergraph convolutional network to
capture higher-order relations among items within sequences; during training,
they maximize the mutual information between the sequence representations
learned via the two channels; negative sampling SSL ensures that different
channels of the same sample are similar.

The SSL strategies proposed in the studies listed above are mostly based on
the current sequence itself. In contrast, we focus on how to devise self-
supervision signals by investigating the correlation of different sequences,
i.e., forcing the SR model to learn better representations by identifying the
common and unique preferences for any pair of interaction sequences.

##  3\. Method

We first formulate the SR task. Then, we introduce our basic transformer-based
multi-preference extraction model MrTransformer. Next, we describe our new SSL
method preference editing. Together, preference editing and MrTransformer
constitute our complete model MrTransformer (PE).

###  3.1. Task definition

Let $\mathbb{I}=\\{i_{1},i_{2},\ldots,i_{|\mathbb{I}|}\\}$ denote the item
set, and $\mathbb{S}=\\{S_{1},S_{2},\ldots,S_{|\mathbb{S}|}\\}$ denote the
interaction sequence set. Each interaction sequence $S\in\mathbb{S}$ can be
denoted as $S=\\{i_{1},i_{2},\ldots,i_{\tau},\ldots,i_{t}\\}$ where $i_{\tau}$
refers to the item interacted with at timestep $\tau$. Given $S$, SR aims to
predict the next item that the user will interact with at timestep $t+1$ by
computing the recommendation probabilities over all candidate items as
follows:

(1) $\begin{split}P(i_{t+1}|S)&\sim f(S),\end{split}$

where $P(i_{t+1}|S)$ denotes the probability of recommending the next item
$i_{t+1}$, and $f(S)$ is the model or function to estimate $P(i_{t+1}|S)$.

###  3.2. MrTransformer

In this section, we will introduce our proposed transformer-based multi-
preference extraction model MrTransformer. Unlike the basic BERT4Rec model,
the core idea of MrTransformer is to mine multiple preferences behind this
interaction sequence, and then generate predictions based on the multiple
preference representation.

MrTransformer’s network structure is based on the transformer, and it consists
of three main components:  (1) a sequence encoder,  (2) a preference
identification module, and  (3) a sequence decoder.  The preference
identification module identifies multiple preference behind the current
interaction sequence and represents it as distributed vectors. Unlike RNN-
based methods and traditional transformer-based methods, we concatenate $K$
special tokens ($\left[P_{1}\right],\\\
\left[P_{2}\right],\ldots,\left[P_{K}\right]$) at the start of each sequence,
where each special token represents a particular user preference. Next, we
introduce these modules in detail.

#### Sequence encoder

Here, we define the processed sequence $S^{\prime}=\\{\left[P_{1}\right],$
$\left[P_{2}\right],\ldots,\left[P_{K}\right],i_{1},i_{2},$
$\ldots,i_{\tau},\ldots,i_{t}\\}$ which concatenates $K$ special tokens at the
start of the sequence $S$. It is worth noting that $K$ represents the number
of latent preferences for the whole dataset, not for a particular sequence. In
this module, we encode the processed sequence $S^{{}^{\prime}}$ into hidden
representations.

First, we initialize the embedding matrix $\mathbf{E}$ of $S^{{}^{\prime}}$,
where $\mathbf{e_{i_{\tau}}}\in R^{d}$ represents the embedding for item
$i_{\tau}$, $d$ is the embedding size. Then we add the position embedding
matrix $\mathbf{P}$ of $S^{{}^{\prime}}$ it, which can be defined as
$\mathbf{E}=\mathbf{E}+\mathbf{P}$. After that, we feed the sequence of items
into a stack of $L$ bidirectional transformer layers. Each layer iteratively
revises the representation of all positions by exchanging information across
some specific positions, which are controlled by the masking matrix at
previous layers. This process is defined as follows:

(2)
$\begin{split}\mathbf{E^{l}}=\operatorname{Trm}(\mathbf{E^{l-1},Mask_{e}}),\\\
\end{split}$

where $\operatorname{Trm}$ refers to the transformer layer, $\mathbf{E^{l}}\in
R^{(K+t)*h}$ is the representation matrix of $S^{{}^{\prime}}$ at the $l$-th
layer, $h$ is the hidden size, and $\mathbf{Mask_{e}}$ is the masking matrix.
Specifically, each special token $\left[P_{k}\right]$ can obtain information
from all positions because it aims to capture the user’s multiple preference
behind the whole sequence. So the receptive field is all positions, and the
masking vector of each special token is all one. For each item, the special
tokens are masked out. Therefore, masking vector of each item is composed of
$K$ zeros and $t$ ones. From the top layer, we can obtain the representation
$\mathbf{E^{L}}$ of $S^{{}^{\prime}}$.

#### Preference identification

In this module, we calculate the distributed attention scores over all items
for each special token. The process is defined as follows:

(3)
$\begin{split}\mathbf{P,A}=\operatorname{Ident}(\mathbf{E^{L},Mask_{I}}),\\\
\end{split}$

where $\operatorname{Ident}$ is implemented by a transformer layer, and
$\mathbf{Mask_{I}}$ is the masking matrix for preference identification.
Different from $\mathbf{Mask_{e}}$, each special token only calculates
attention weights over all items, so it masks out other special tokens. And
for each item, the masking vector is the same as $\mathbf{Mask_{e}}$.
$\mathbf{P}\in R^{K*h}$ is the representation matrix for the multiple
preference corresponding to the first $K$ special tokens. We can also get the
attention matrix $\mathbf{A}$ over all items for the special tokens.

To avoid that different special tokens focus on the same items and thus learn
similar representations, we introduce a _preference coverage mechanism_. We
maintain $K$ coverage vectors $c_{\tau}^{k}|_{k=1}^{K}$. Vector $c_{\tau}^{k}$
is the sum of attention distributions over items at previous timesteps by the
special token $\left[P_{k}\right]$, which represents the degree of coverage
that those items have received from the attention mechanism by
$\left[P_{k}\right]$ so far:

(4) $\begin{split}c_{\tau}^{k}=\sum_{j=1}^{\tau-1}a_{j}^{k},\\\ \end{split}$

where $a^{k}\in\mathbf{A}$ is a distributed attention vector over all items by
$\left[P_{k}\right]$ and $\sum_{i}a_{i}^{k}=1$. And $c_{0}^{k}$ is a zero
vector, which denotes that, at the first timestep, none of the items have been
covered.

#### Sequence decoder

After the preference identification module, we get the representation for the
multiple preference. Different from existing methods that use item
representations to predict the next item (Huang et al., 2019; Li et al., 2019;
Zheng et al., 2020; Song et al., 2019), we use the learned multiple preference
representation to do recommendation:

(5) $\begin{split}P(i_{t+1}|S)=\operatorname{softmax}(\mathbf{p}W+b),\\\
\end{split}$

where $\mathbf{p}$ is the sum of the multiple preference $\mathbf{P}$; $W$ is
the embedding matrix of all items, and $b$ is the bias term.

#### Objective functions

As with traditional SR methods (Hidasi et al., 2016b; Wang et al., 2019;
Quadrana et al., 2017; Liu et al., 2018), our first objective is to predict
the next item for each position in the input sequence. We employ the negative
log-likelihood loss to define the recommendation loss as follows:

(6)
$\begin{split}L_{rec}(\theta)&=-\frac{1}{|\mathbb{S}|}\sum_{S\in\mathbb{S}}\sum_{i_{\tau}\in
S}\log P(i_{\tau+1}|S),\\\ \end{split}$

where $\theta$ are all parameters of MrTransformer.

Apart from this, we also define a coverage loss to penalize repeatedly
attending to the same items by different preferences:

(7)
$\begin{split}L_{cov}(\theta)&=\sum_{k=1}^{K}\sum_{\tau}\min(a_{\tau}^{k},c_{\tau}^{k}).\\\
\end{split}$

Finally, the coverage loss function, weighted by the hyperparameter $\alpha$,
is added to the recommendation loss function to yield the total loss function:

(8) $\begin{split}L(\theta)&=L_{rec}(\theta)+\alpha L_{cov}(\theta),\\\
\end{split}$

where $\alpha$ controls the ratio of the coverage loss.

Figure 2. Overview of preference editing. Section 3.3 contains a walkthrough
of this figure.

###  3.3. Preference editing

In this subsection, we detail the preference editing learning strategy to mine
correlations among items between different interaction sequences.

As illustrated in Figure 2, we first sample two sequences $S_{x}$ and $S_{y}$
from the sequence set $\mathbb{S}$ with the requirement that the two sequences
share some common items while also have their unique items. Through the
preference identification module, we get the multiple preference
representation for each sequence (e.g., $P_{1}^{x}$, $P_{2}^{x}$ and
$P_{3}^{x}$ for sequence $S_{x}$ and $P_{1}^{y}$, $P_{2}^{y}$ and $P_{3}^{y}$
for sequence $S_{y}$). Then, the preference separation module forces the model
to separate common ($P_{1}^{x}$, $P_{2}^{x}$ and $P_{1}^{y}$, $P_{2}^{y}$) and
unique ($P_{3}^{x}$ and $P_{3}^{y}$) preference representations for the paired
sequences. After that, the preference recombination module swaps the common
preference representation so as to obtain the recombined preference
representations for each sequence (e.g., $P_{1}^{y}$, $P_{2}^{y}$, $P_{3}^{x}$
for sequence $S_{x}$ and $P_{1}^{x}$, $P_{2}^{x}$, $P_{3}^{y}$ for sequence
$S_{y}$). Next, we explain the above process in detail.

#### Preference separation

For each pair of sequences $S_{x}$ and $S_{y}$, we get the multiple preference
representation matrices $\mathbf{P_{x}}$ and $\mathbf{P_{y}}$ where
$\mathbf{P_{x}},\mathbf{P_{y}}\in R^{K*h}$. In order to measure the degree of
similarity between these two representations, we calculate the similarity
matrix $\mathbf{I}\in R^{K*K}$ to consider their relation. Each element
$I_{ij}\in R$ is calculated as:

(9) $\begin{split}I_{ij}=W_{I}^{\mathrm{T}}\left[p_{i};p_{j};p_{i}\odot
p_{j}\right],\\\ \end{split}$

where $p_{i}$ and $p_{j}$ are vectors in the multiple preference
representation matrices $\mathbf{P_{x}}$ and $\mathbf{P_{y}}$, respectively;
$\odot$ is the element-wise calculation, and $W_{I}\in R^{3h*1}$ are the
parameters. Based on this similarity matrix, we calculate the attention
matrices $\mathbf{A_{x}}$ and $\mathbf{B_{y}}$, which reflect the attention
distribution of the preference representations of one sequence to the
preference representation in another sequence as follows:

(10)
$\begin{split}\mathbf{A_{x}}&{}=\operatorname{softmax}_{row}(\mathbf{I})\\\
\mathbf{B_{y}}&{}=\operatorname{softmax}_{col}(\mathbf{I}),\\\ \end{split}$

where $\operatorname{softmax}_{row}$ and $\operatorname{softmax}_{col}$ refer
to performing a $\operatorname{softmax}$ calculation in rows and columns,
respectively. After that, the common and unique preference representation of
each sequence are calculated as follows:

(11)
$\begin{split}\mathbf{C_{x}}&{}=\mathbf{B_{y}}\odot\mathbf{P_{x}},\quad\mathbf{U_{x}}=(1-\mathbf{B_{y}})\odot\mathbf{P_{x}}\\\
\mathbf{C_{y}}&{}=\mathbf{A_{x}}\odot\mathbf{P_{y}},\quad\mathbf{U_{y}}=(1-\mathbf{A_{x}})\odot\mathbf{P_{y}},\\\
\end{split}$

where $C_{x}$ and $C_{y}$ refer to the common preference representation of
each sequence respectively. And $U_{x}$ and $U_{y}$ are the unique preference
representation respectively. Here, $A_{x}$ and $B_{y}$ play the role of the
gate functions which filter the common and unique preference representation.

#### Preference recombination

Through the preference separation module, we get the common and unique
preference representation of the two input sequences $S_{x}$ and $S_{y}$.
Then, we swap the common preference representation to form a recombined
representation of each sequence as follows:

(12)
$\begin{split}\mathbf{P_{x}^{{}^{\prime}}}&{}=\operatorname{combine}[\mathbf{C_{y}};\mathbf{U_{x}};\mathbf{C_{y}}\odot\mathbf{U_{x}}]\\\
\mathbf{P_{y}^{{}^{\prime}}}&{}=\operatorname{combine}[\mathbf{C_{x}};\mathbf{U_{y}};\mathbf{C_{x}}\odot\mathbf{U_{y}}].\\\
\end{split}$

Based on the recombined representation, we define two types of supervision
signals for learning:

(13) $\begin{split}L_{SSL}(\theta)&{}=L_{pred}(\theta)+L_{reg}(\theta).\\\
\end{split}$

$L_{pred}(\theta)$ is to do next item prediction based on the recombined
representation $P_{x}^{{}^{\prime}}$ and $P_{y}^{{}^{\prime}}$:

(14) $\begin{split}L_{pred}(\theta)&=L_{rec}^{x}+L_{rec}^{y},\\\ \end{split}$

where $L_{rec}^{x}$ and $L_{rec}^{y}$ are negative log-likelihood loss, which
are the same as calculation in Eq. 6. And $L_{rec}^{x}$ and $L_{rec}^{y}$ are
used to predict the next item of each sequence using the recombined
representation.

$L_{reg}(\theta)$ is a regularization term, which consists of three parts:

(15)
$\begin{split}L_{reg}(\theta)^{x}&{}=\frac{1}{K*h}\sum_{S_{x},S_{y}\in\mathbb{S}}(\mathbf{P_{x}}-\mathbf{P_{x}^{{}^{\prime}}})^{2}\\\
L_{reg}(\theta)^{y}&{}=\frac{1}{K*h}\sum_{S_{x},S_{y}\in\mathbb{S}}(\mathbf{P_{y}}-\mathbf{P_{y}^{{}^{\prime}}})^{2}\\\
L_{reg}(\theta)^{c}&{}=\frac{1}{K*h}\sum_{S_{x},S_{y}\in\mathbb{S}}(\mathbf{C_{x}}-\mathbf{C_{y}})^{2}\\\
L_{reg}(\theta)&{}=L_{reg}(\theta)^{x}+L_{reg}(\theta)^{y}+L_{reg}(\theta)^{c}.\\\
\end{split}$

$L_{reg}(\theta)^{x}$ and $L_{reg}(\theta)^{y}$ make sure that the recombined
representation is close enough to the original preference representation,
respectively. $L_{reg}(\theta)^{c}$ requires that the learned common
representations are also close to each other.

The final training loss for MrTransformer is as follows:

(16) $\begin{split}L_{all}(\theta)&{}=L(\theta)+L_{SSL}(\theta),\\\
\end{split}$

where $L(\theta)$ is the loss for preference extraction and recommendation
(Eq. 8), $L_{SSL}(\theta)$ is the loss for preference editing (Eq. 13).

We use MrTransformer to refer to our basic model that models a users’
preferences based on transformer layers by using the preference identification
module as detailed in Section 3.2. We write MrTransformer (PE) for
MrTransformer pretrained with the preference editing learning strategy
described in this section.

##  4\. Experimental Setup

We seek to answer the following questions in our experiments:

  1. (RQ1)

What is the performance of MrTransformer (PE) compared to other methods? Does
it outperform the state-of-the-art methods in terms of Recall, MRR and NDCG on
all datasets?

  2. (RQ2)

What is the effect of the preference editing learning strategy on the
performance of MrTransformer (PE)? And how does it affect sequences of
different lengths?

  3. (RQ3)

What is the effect of the preference coverage mechanism on the performance of
MrTransformer (PE)?

  4. (RQ4)

How does the hyperparameter $K$ (the number of assumed preferences) affect the
performance of MrTransformer (PE)?

###  4.1. Datasets

We conduct experiments on five datasets in different domains:

  * •

Amazon Beauty, Sports, Toys: These datasets are obtained from Amazon product
review datasets crawled by McAuley et al. (2015). We select three
subcategories of Beauty (40,226 users and 54,542 items), Sports (25,598 users
and 18,357 items) and Toys (19,412 users and 11,924
items).111http://jmcauley.ucsd.edu/data/amazon/

  * •

MovieLens: This is a popular benchmark dataset for recommendation evaluation.
We adopt a well-established version, ML-100k (943 users and 1,349
items).222https://grouplens.org/datasets/movielens/

  * •

Yelp: This is a popular dataset for business recommendation (30,431 users and
20,033 items). 333https://www.yelp.com/dataset As it is very large, we only
use the transaction records after January 1st, 2019 like (Zhou et al., 2020b).

For data processing, we follow (Sun et al., 2019a; Kang and McAuley, 2018). We
convert the rating scores of reviews to the implicit feedback of 1 if the user
has interacted with this item. Then we group the interactions by the same user
and sort them according to the timesteps. Like (Zhang et al., 2019; Li et al.,
2019; Meng et al., 2020), since we do not target cold-start recommendation, we
filter out cold-start users who have less than 5 interactions and cold-start
items with less than 5 interactions. Because each user has a lot of
interactions in a long time, we limit the maximum length of each sequence to
be 50. And the statistics of datasets are shown in Table 1.

Table 1. Statistics of datasets. Datasets | #users | #items | #actions | #min(len) | #max(len) | #avg(len)
---|---|---|---|---|---|---
Beauty | 40,226 | 54,542 | 353,962 | 3 | 291 | 8.8
Sports | 25,598 | 18,357 | 296,337 | 3 | 294 | 6.3
Toys | 19,412 | 11,924 | 167,597 | 3 | 548 | 6.6
ML-100k | 943 | 1,349 | 99,287 | 17 | 646 | 103.2
Yelp | 30,431 | 20,033 | 316,354 | 3 | 348 | 8.4

###  4.2. Evaluation metrics

We adopt the leave-one-out recommendation evaluation scheme (Sun et al.,
2019a). We use the last item and the second last item as the test set and
validation set, and the rest as training set. We concatenate the training and
validation data to predict the last item in the test set. Following (Li et
al., 2019; McAuley et al., 2015), we pair each ground truth item with 100
randomly sampled negative items according to their popularities.

We report results using Recall@k, MRR@k and NDCG@k with $k=5,10,20$. Assuming
that the tested item of user $u$ is ranked $r_{u}$ based on the predicted
scores, the metrics are calculated as follows:

  * •

Recall@k: The proportion of cases when the ground-truth item is amongst the
top-$k$ ranked items.

  * •

MRR@k: The average of reciprocal ranks of the ground-truth items, i.e.,
$MRR_{u}@k=\frac{1}{r_{u}}$ if $r_{u}\leq k$ and $MRR_{u}@k=0$ otherwise.

  * •

NDCG@k: A position-aware metric which assigns larger weights on higher ranks,
i.e., $NDCG_{u}@k=\frac{1}{\log_{2}(1+r_{u})}$ if $r_{u}\leq k$ and
$NDCG_{u}@k=0$ otherwise.

Table 2. Experimental results on five datasets. Bold face indicates the best results in terms of the corresponding metrics. Significant improvements over the best baseline results are marked with ∗ (t-test, $p<.05$).  | Metric | POP | BPR-MF | Item-Knn | SASRec | SSE-PT | FISSA | BERT4Rec | S3-Rec | MrTransformer | MrTransformer (PE)
---|---|---|---|---|---|---|---|---|---|---|---
Beauty | Recall@5 | 0.0315 | 0.1390 | 0.1660 | 0.1934 | 0.1959 | 0.2116 | 0.2137 | 0.2173 | 0.2220 | 0.2286
Recall@10 | 0.0628 | 0.2152 | 0.2029 | 0.2653 | 0.2813 | 0.3079 | 0.2964 | 0.2909 | 0.2989 | 0.3126
Recall@20 | 0.1280 | 0.3292 | 0.2695 | 0.3724 | 0.3901 | 0.4289 | 0.3971 | 0.3926 | 0.4070 | 0.4302∗
MRR@5 | 0.0147 | 0.0755 | 0.1259 | 0.1112 | 0.1199 | 0.1198 | 0.1299 | 0.1348 | 0.1438 | 0.1444∗
MRR@10 | 0.0187 | 0.0855 | 0.1308 | 0.1220 | 0.1286 | 0.1325 | 0.1408 | 0.1445 | 0.1540 | 0.1555∗
MRR@20 | 0.0230 | 0.0933 | 0.1352 | 0.1291 | 0.1327 | 0.1408 | 0.1477 | 0.1514 | 0.1614 | 0.1635∗
NDCG@5 | 0.0188 | 0.0912 | 0.1359 | 0.1436 | 0.1491 | 0.1424 | 0.1506 | 0.1552 | 0.1632 | 0.1653
NDCG@10 | 0.0288 | 0.1157 | 0.1477 | 0.1633 | 0.1683 | 0.1734 | 0.1773 | 0.1789 | 0.1879 | 0.1923∗
NDCG@20 | 0.0449 | 0.1443 | 0.1644 | 0.1823 | 0.1944 | 0.2039 | 0.2026 | 0.2044 | 0.2151 | 0.2218∗
Sports | Recall@5 | 0.0313 | 0.0376 | 0.1512 | 0.1622 | 0.2102 | 0.2133 | 0.2124 | 0.2072 | 0.2069 | 0.2320∗
Recall@10 | 0.0624 | 0.0819 | 0.1927 | 0.2788 | 0.3212 | 0.3180 | 0.3105 | 0.3250 | 0.3225 | 0.3456∗
Recall@20 | 0.1364 | 0.1735 | 0.2260 | 0.3908 | 0.4492 | 0.4592 | 0.4464 | 0.4817 | 0.4829 | 0.5000∗
MRR@5 | 0.0139 | 0.0167 | 0.0449 | 0.0913 | 0.1180 | 0.1262 | 0.1227 | 0.1122 | 0.1146 | 0.1354
MRR@10 | 0.0178 | 0.0224 | 0.0547 | 0.1042 | 0.1344 | 0.1367 | 0.1356 | 0.1276 | 0.1297 | 0.1504∗
MRR@20 | 0.0227 | 0.0285 | 0.1224 | 0.1105 | 0.1464 | 0.1486 | 0.1449 | 0.1384 | 0.1408 | 0.1616∗
NDCG@5 | 0.0181 | 0.0218 | 0.0783 | 0.1102 | 0.1407 | 0.1325 | 0.1449 | 0.1356 | 0.1374 | 0.1593∗
NDCG@10 | 0.0279 | 0.0359 | 0.1132 | 0.1532 | 0.1780 | 0.1726 | 0.1764 | 0.1734 | 0.1744 | 0.1958∗
NDCG@20 | 0.0464 | 0.0587 | 0.1550 | 0.1704 | 0.2126 | 0.2157 | 0.2106 | 0.2128 | 0.2149 | 0.2347∗
Toys | Recall@5 | 0.0348 | 0.0487 | 0.0986 | 0.2411 | 0.2940 | 0.3127 | 0.2980 | 0.3110 | 0.3023 | 0.3285∗
Recall@10 | 0.0697 | 0.0970 | 0.2344 | 0.3417 | 0.4001 | 0.4062 | 0.3878 | 0.4094 | 0.3955 | 0.4218
Recall@20 | 0.1443 | 0.1995 | 0.2360 | 0.4407 | 0.5344 | 0.5134 | 0.5058 | 0.5282 | 0.5224 | 0.5445∗
MRR@5 | 0.0161 | 0.0226 | 0.0388 | 0.1530 | 0.1753 | 0.1934 | 0.1932 | 0.1964 | 0.1969 | 0.2208
MRR@10 | 0.0206 | 0.0288 | 0.0562 | 0.1630 | 0.1882 | 0.2098 | 0.2050 | 0.2096 | 0.2092 | 0.2332∗
MRR@20 | 0.0256 | 0.0356 | 0.0665 | 0.1707 | 0.1960 | 0.2200 | 0.2131 | 0.2177 | 0.2178 | 0.2415∗
NDCG@5 | 0.0207 | 0.0290 | 0.0534 | 0.1749 | 0.2047 | 0.2254 | 0.2192 | 0.2249 | 0.2231 | 0.2476∗
NDCG@10 | 0.0318 | 0.0443 | 0.0965 | 0.2177 | 0.2380 | 0.2653 | 0.2481 | 0.2567 | 0.2530 | 0.2776∗
NDCG@20 | 0.0505 | 0.0699 | 0.1325 | 0.2274 | 0.2708 | 0.3025 | 0.2778 | 0.2866 | 0.2849 | 0.3084∗
ML-100k | Recall@5 | 0.0965 | 0.1866 | 0.1845 | 0.2948 | 0.3017 | 0.2585 | 0.3003 | 0.3012 | 0.3436 | 0.3601∗
Recall@10 | 0.1431 | 0.3138 | 0.3276 | 0.4746 | 0.4688 | 0.4492 | 0.4662 | 0.4761 | 0.4920 | 0.5005
Recall@20 | 0.2396 | 0.4655 | 0.4835 | 0.6548 | 0.6723 | 0.6409 | 0.6506 | 0.6755 | 0.6648 | 0.6808∗
MRR@5 | 0.0484 | 0.0989 | 0.0958 | 0.1753 | 0.1748 | 0.1311 | 0.1707 | 0.1513 | 0.1929 | 0.2031
MRR@10 | 0.0549 | 0.1154 | 0.1152 | 0.1984 | 0.2046 | 0.1555 | 0.1926 | 0.1748 | 0.2119 | 0.2211∗
MRR@20 | 0.0613 | 0.1258 | 0.1257 | 0.2117 | 0.2183 | 0.1690 | 0.2058 | 0.1897 | 0.2230 | 0.2336∗
NDCG@5 | 0.0602 | 0.1203 | 0.1176 | 0.2035 | 0.2106 | 0.1624 | 0.2028 | 0.1881 | 0.2315 | 0.2416∗
NDCG@10 | 0.0755 | 0.1610 | 0.1642 | 0.2655 | 0.2599 | 0.2230 | 0.2562 | 0.2448 | 0.2784 | 0.2866∗
NDCG@20 | 0.0996 | 0.1992 | 0.2032 | 0.3088 | 0.2993 | 0.2718 | 0.3033 | 0.2985 | 0.3209 | 0.3321∗
Yelp | Recall@5 | 0.0474 | 0.0500 | 0.0499 | 0.4070 | 0.3884 | 0.4520 | 0.4561 | 0.4224 | 0.4403 | 0.4640∗
Recall@10 | 0.0960 | 0.0983 | 0.1080 | 0.5691 | 0.5609 | 0.6153 | 0.6138 | 0.5926 | 0.6086 | 0.6312
Recall@20 | 0.1897 | 0.1970 | 0.2278 | 0.7414 | 0.7684 | 0.7757 | 0.7621 | 0.7625 | 0.7771 | 0.7972∗
MRR@5 | 0.0227 | 0.0230 | 0.0224 | 0.2265 | 0.2261 | 0.2690 | 0.2703 | 0.2439 | 0.2609 | 0.2742
MRR@10 | 0.0290 | 0.0307 | 0.0298 | 0.2476 | 0.2492 | 0.2974 | 0.2914 | 0.2666 | 0.2834 | 0.2965
MRR@20 | 0.0353 | 0.0365 | 0.0380 | 0.2592 | 0.2686 | 0.3002 | 0.3017 | 0.2783 | 0.2951 | 0.3080∗
NDCG@5 | 0.0288 | 0.0291 | 0.0291 | 0.2699 | 0.2663 | 0.3140 | 0.3163 | 0.2880 | 0.3053 | 0.3211∗
NDCG@10 | 0.0443 | 0.0447 | 0.0476 | 0.3242 | 0.3224 | 0.3631 | 0.3673 | 0.3430 | 0.3597 | 0.3752∗
NDCG@20 | 0.0677 | 0.0691 | 0.0777 | 0.3632 | 0.3789 | 0.4089 | 0.4048 | 0.3860 | 0.4023 | 0.4172∗

###  4.3. Methods used for comparison

To assess the effectiveness of MrTransformer and MrTransformer (PE), we
compare it with the following methods:

  * •

POP ranks items in the training set based on their popularities, and always
recommends the most popular items (He et al., 2017).

  * •

BPR-MF is a commonly used matrix factorization method. We apply it for SR by
representing a new sequence with the average latent factors of items appearing
in the sequence so far (Hidasi et al., 2016a).

  * •

Item-KNN computes an item-to-item similarity matrix and recommends items that
are similar to the actual item. Regularization is included to avoid
coincidental high similarities (Linden et al., 2003).

  * •

SASRec (Kang and McAuley, 2018) uses a left-to-right self-attention based
model to capture users’ sequential behavior.

  * •

SSE-PT (Wu et al., 2020b) is a personalized transformer-based model with a
regularization technique called SSE (Wu et al., 2019a) to overcome overfitting
caused by simply adding user embeddings.

  * •

FISSA (Lin et al., 2020) uses the SASRec model to get the user preference
representations, with a transformer-based layer that fuses item similarity to
model the global user preference representation; a gating module balances the
local and global representations.

  * •

BERT4Rec (Sun et al., 2019a) models the user preference representation with a
bidirectional transformer network; during training, it randomly masks some
items in the sequence and predicts these items jointly conditioned on their
left and right context; during testing, it only masks the last item to do
recommendations.

  * •

S3-rec (Zhou et al., 2020b) adopts SASRec as the base model and devises four
auxiliary self-supervised objectives to pretrain the sequential recommender
model using attribute, item, subsequence, and sequence by utilizing mutual
information maximization; it fine-tunes the parameters according to the SR
task; we do not consider attribute characteristics and only use the MIP loss
function as they are not supported by the datasets.

Other SSL based methods (e.g., Yuan et al., 2020a; Yao et al., 2020; Zhou et
al., 2020a; Wu et al., 2020c; Yu et al., 2021) have only been proposed for
general, cross-domain or social recommendation tasks; hence, we omit
comparisons against them. We also exclude the SSL based method in (Ma et al.,
2020), as the authors found flaws in their experimental setup. As for the SSL
based method in (Xia et al., 2021), they adopt a different experimental
setting of next-session recommendation to model the sequence graph
representation, instead, we use the next-item recommendation setting which
cannot meet their premise of hypergraph construction.

###  4.4. Implementation details

We implement POP, BPR-MF and Item-KNN using Tensorflow. For
SASRec,444https://github.com/kang205/SASRec SSE-
PT,555https://github.com/wuliwei9278/SSE-PT
FISSA,666https://github.com/RUCAIBox/CIKM2020-S3Rec
BERT4Rec,777https://github.com/FeiSun/BERT4Rec and
S3-rec,888https://github.com/RUCAIBox/CIKM2020-S3Rec we use the code provided
by the authors. We use hyperparameters as reported or suggested in the
original papers.

We implement MrTransformer with Tensorflow. All parameters are initialized
using a truncated normal distribution in the range $\left[-0.02,0.02\right]$.
We set the hidden size to 64 and drop ratio to 0.5. We set the number of
transformer layers to 2 and the number of attention heads to 2. For
MrTransformer (PE), the training phase contains two stages (i.e., pre-training
and fine-tuning stage). The learned parameters in the pre-training stage are
used to initialize the embedding layers and transformer layers in the fine-
tuning stage. In the pre-training stage, we only use the SSL loss (as defined
in Eq. 13) to train MrTransformer. In the fine-tune stage, we use the
recommendation loss (as defined in Eq. 15) to train MrTransformer. We try
different settings for the preference size $K$, the analysis of which can be
found in Section 5.4. We train the model using Adam optimizer, we set the
learning rate as 1e-4, $\beta_{1}=0.99,\beta_{2}=0.999$, $l_{2}$ weight decay
of 0.01, with linear decay of the learning rate. We also apply gradient
clipping with range $\left[-5,5\right]$ during training. To speed up training
and converge, we use a mini-batch size of 256. We test the model performance
on the validation set for every epoch. All the models are trained on a GeForce
GTX TitanX GPU.

For sampling sequence pairs in the learning of preference editing, we sample
20 sequences for each sequence according to its similarities with all other
sequences. Specifically, we first build a 0-1 vector for each sequence. If the
item appears in the sequence, the corresponding element is 1, otherwise it is
0. Then we use the FAISS library999https://github.com/facebookresearch/faiss
to search the nearest neighbors for each sequence. We choose the 20 samples
according to their item frequency similarity, and make sure that each pair of
sequences shares some common items but also have some unique items.

##  5\. Results and Analysis

###  5.1. Overall performance of MrTransformer (PE)

To answer RQ1, we report on the performance of MrTransformer (PE) and the
baseline methods in terms of Recall, MRR, NDCG. See Table 2. We obtain the
following insights from the results.

First, MrTransformer (PE) achieves the best results on all datasets in terms
of all metrics. It outperforms strong Transformer-based SR methods (SSE-PT,
FISSA) and SSL-based methods (BERT4Rec and S3-Rec). Specifically, concerning
the transformer-based SR methods on the “Beauty” dataset, the increase over
FISSA is 8.03%, 20.53% and 16.08% in terms of Recall@5, MRR@5, and NDCG@5. As
for the SSL-based SR methods, the increase over S3-Rec is 9.58%, 7.99% and
8.51% in terms of Recall@5, MRR@5, and NDCG@5. On the “ML-100k” dataset, the
increase over SSE-PT is 19.35%, 15.96% and 14.71% in terms of Recall@5, MRR@5,
and NDCG@5, while the increase over S3-Rec is 19.55%, 33.97% and 28.44% in
terms of Recall@5, MRR@5, and NDCG@5. On the “Yelp” dataset, the increase over
the transformer-based SR method FISSA is 2.77%, 2.59% and 3.33% in terms of
Recall@20, MRR@20, and NDCG@10. And the increase over SSL-based SR method
BERT4Rec is 4.60%, 2.08%and 3.06% in terms of Recall@20, MRR@20, and NDCG@20.
To sum up, our method consistently outperforms all the compared methods on
these datasets.

The improvements are mainly because that MrTransformer (PE) achieves better
preference extraction and representation by discriminating the common and
unique items of multiple preferences between sequences with the help of
preference editing. We will analyze the preference editing learning strategy
in more depth in Section 5.2.

Second, the improvements of MrTransformer (PE) on “ML-100k” are larger than
those on other datasets. This is related to the characteristics of different
datasets. For example, the “Beauty” dataset is collected from cosmetic
purchases. For a particular sequence, it usually has a clear preference and
specific buying needs, although the performance on the whole dataset can be
very extensive and diverse. This is also true for the “Sports”, “Toys” and
“Yelp” datasets, which are collected from the outdoor sports, game purchase
records and business reviews, respectively. Under these scenarios, users’
preferences are usually fixed and clear within the same sequence. However, the
“ML-100k” dataset is collected from movie watching. Users’ watching
preferences usually change from time to time with the change of context such
as time, place or mood, even within the same sequence. Since MrTransformer
(PE) is especially adept at extracting multiple preferences and capturing
their commonness and uniqueness through preference editing, its advantages are
more obvious on “ML-100k”.

Third, the improvements of NDCG and MRR on most datasets are larger than
Recall generally. On the one hand, this demonstrates that MrTransformer (PE)
is beneficial to the ranking of the recommendation list. On the other hand,
this also reveals that Recall is generally more difficult to improve. Compared
with the improvements of the best baseline over the second best, i.e., the
largest increase of S3-Rec over FISSA on the “Beauty” dataset achieves 2.69%
in terms of Recall@5. On the “ML-100k” dataset, the largest increase of SSE-PT
over BERT4Rec achieves the improvement of 3.33% in terms of Recall@20, the
improvements of MrTransformer (PE) on Recall is already considered large.
Generally, SSL-based SR methods outperform transformer-based SR methods, which
indicates that SSL is an effective direction for SRs by introducing more
supervision signals to improve representation learning. What’s more, the SSL-
based method S3-Rec is better than the other SSL-based methods, e.g.,
BERT4Rec, on almost all datasets. This is because S3-Rec also leverages
sequence-level self-supervised signals besides the item-level signals, which
further confirms that the self-supervised signals are very helpful for
improving the recommendation performance.

###  5.2. Effect of preference editing

To answer RQ2, we conduct an ablation study to analyze the effects of the
preference editing learning strategy. We compare MrTransformer (PE) with
MrTransformer where MrTransformer is only trained with the recommendation loss
(Eq. 6) and the preference coverage loss (Eq. 7). The results are shown in
Table 2.

The results decrease when removing preference editing on all datasets. By
comparison, the results drop by more than 5.70%pt, 1.30%pt and 3.11%pt in
terms of Recall@20, MRR@20 and NDCG@20 on the “Beauty” dataset. And the
results drop by more than 4.80%pt, 5.08%pt and 4.36%pt in terms of Recall@5,
MRR@5 and MRR@5 on the “ML-100k” dataset. On the “Yelp” dataset, the results
drop by more than 2.58%pt, 4.37%pt and 3.70%pt in terms of Recall@20, MRR@20
and NDCG@20, respectively. This demonstrates that preference editing is
effective by forcing MrTransformer to learn common and unique items between
sequences. At the same time, we see that the MRR and NDCG of MrTransformer are
slightly higher than the strong baselines like BERT4Rec and FISSA while the
Recall values are comparable on most datasets. This is reasonable because
MrTransformer is also a transformer-base method, and without preference
editing, its improvements are limited.

We also carry out analysis experiments on the three datasets (i.e., “Sports”,
“ML-100k”, “Yelp”) to analyze the effects of preference editing on sequences
with different lengths. Specifically, we divide the sequences into four groups
according to their length. The results are shown in Table 3–8.

Table 3. Recall, MRR and NDCG of MrTransformer without preference editing on the “Sports” dataset. The best results are denoted in underline fonts. Length | MrTransformer
---|---
Recall | MRR | NDCG
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
$\left[20,30\right)$ | 0.2176 | 0.3387 | 0.4846 | 0.1267 | 0.1424 | 0.1525 | 0.1492 | 0.1879 | 0.2246
$\left[30,40\right)$ | 0.2172 | 0.3252 | 0.4521 | 0.1317 | 0.1455 | 0.1543 | 0.1528 | 0.1871 | 0.2192
$\left[40,50\right]$ | 0.2018 | 0.3211 | 0.4678 | 0.1233 | 0.1392 | 0.1497 | 0.1425 | 0.1810 | 0.2185
$\geq 50$ | 0.2125 | 0.3375 | 0.4560 | 0.1295 | 0.1430 | 0.1520 | 0.1486 | 0.1822 | 0.2153
Table 4. Recall, MRR and NDCG of MrTransformer with preference editing on the “Sports” dataset. The best results are denoted in underline fonts. Length | MrTransformer (PE)
---|---
Recall | MRR | NDCG
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
$\left[20,30\right)$ | 0.2263 | 0.3413 | 0.4963 | 0.1347 | 0.1495 | 0.1602 | 0.1573 | 0.1940 | 0.2331
$\left[30,40\right)$ | 0.2496 | 0.3589 | 0.4958 | 0.1445 | 0.1586 | 0.1683 | 0.1705 | 0.2089 | 0.2457
$\left[40,50\right]$ | 0.2247 | 0.3394 | 0.4908 | 0.1376 | 0.1523 | 0.1625 | 0.1591 | 0.1956 | 0.2334
$\geq 50$ | 0.2313 | 0.3438 | 0.4938 | 0.1523 | 0.1665 | 0.1770 | 0.1718 | 0.2073 | 0.2403
Table 5. Recall, MRR and NDCG of MrTransformer without preference editing on the “ML-100k” dataset. The best results are denoted in underline fonts. Length | MrTransformer
---|---
Recall | MRR | NDCG
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
$\left[20,30\right)$ | 0.3872 | 0.6011 | 0.7645 | 0.1949 | 0.2232 | 0.2354 | 0.2424 | 0.3113 | 0.3554
$\left[30,40\right)$ | 0.4220 | 0.5871 | 0.7706 | 0.2457 | 0.2670 | 0.2798 | 0.2888 | 0.3414 | 0.3880
$\left[40,50\right]$ | 0.4545 | 0.6590 | 0.7840 | 0.2464 | 0.2713 | 0.2827 | 0.3032 | 0.3628 | 0.4034
$\geq 50$ | 0.2772 | 0.4176 | 0.5857 | 0.1529 | 0.1712 | 0.1827 | 0.1835 | 0.2284 | 0.2708
Table 6. Recall, MRR and NDCG of MrTransformer with preference editing on the “ML-100k” dataset. The best results are denoted in underline fonts. Length | MrTransformer (PE)
---|---
Recall | MRR | NDCG
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
$\left[20,30\right)$ | 0.4335 | 0.6127 | 0.7861 | 0.2430 | 0.2668 | 0.2788 | 0.2927 | 0.3516 | 0.3954
$\left[30,40\right)$ | 0.4403 | 0.6238 | 0.7798 | 0.2441 | 0.2694 | 0.2793 | 0.2927 | 0.3528 | 0.3896
$\left[40,50\right]$ | 0.4785 | 0.6785 | 0.8095 | 0.2333 | 0.2544 | 0.2668 | 0.2924 | 0.3462 | 0.3953
$\geq 50$ | 0.3032 | 0.4228 | 0.5875 | 0.1604 | 0.1772 | 0.1887 | 0.1955 | 0.2350 | 0.2767
Table 7. Recall, MRR and NDCG of MrTransformer without preference editing on the “Yelp” dataset. The best results are denoted in underline fonts. Length | MrTransformer
---|---
Recall | MRR | NDCG
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
$\left[20,30\right)$ | 0.4730 | 0.6405 | 0.8035 | 0.2842 | 0.3065 | 0.3178 | 0.3310 | 0.3851 | 0.4263
$\left[30,40\right)$ | 0.4326 | 0.5879 | 0.7606 | 0.2482 | 0.2693 | 0.2814 | 0.2938 | 0.3444 | 0.3882
$\left[40,50\right]$ | 0.4512 | 0.6017 | 0.7694 | 0.2653 | 0.2845 | 0.2871 | 0.3104 | 0.3562 | 0.3971
$\geq 50$ | 0.3756 | 0.5405 | 0.7135 | 0.2171 | 0.2393 | 0.2512 | 0.2563 | 0.3098 | 0.3534
Table 8. Recall, MRR and NDCG of MrTransformer with preference editing on the “Yelp” dataset. The best results are denoted in underline fonts. Length | MrTransformer (PE)
---|---
Recall | MRR | NDCG
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
$\left[20,30\right)$ | 0.4840 | 0.6637 | 0.8139 | 0.2950 | 0.3193 | 0.3300 | 0.3418 | 0.4001 | 0.4385
$\left[30,40\right)$ | 0.4437 | 0.6371 | 0.8035 | 0.2569 | 0.2832 | 0.2947 | 0.3028 | 0.3658 | 0.4078
$\left[40,50\right]$ | 0.4675 | 0.6234 | 0.7792 | 0.2661 | 0.2864 | 0.2948 | 0.3155 | 0.3655 | 0.4045
$\geq 50$ | 0.3946 | 0.5757 | 0.7703 | 0.2309 | 0.2547 | 0.2679 | 0.2714 | 0.3297 | 0.3786

From these tables, we see that MrTransformer (PE) outperforms MrTransformer in
terms of most metrics over all sequence length groups on all three datasets.
This verifies the effectiveness of preference editing. On the “Sports”
dataset, MrTransformer (PE) achieves the largest increase when the sequence
length is between 30 and 40. It gains 14.91%, 9.71% and 12.08% in terms of
Recall@5, MRR@5 and NDCG@20, respectively. On the “ML-100k” dataset, when the
sequence length is between 20 and 30, MrTransformer (PE) achieves the largest
increase. It gains 11.95%, 24.67% and 20.75% in terms of Recall@5, MRR@5 and
NDCG@5, respectively. This indicates that preference editing is more effective
for sequences which are less than 50 in length. We also see that the best
performances are achieved in different length groups on different datasets. It
achieves the best performances in the groups of [30, 40), [40, 50), [20, 30)
on the “Sports”, “ML-100k”, “Yelp” datasets, respectively. We believe that
this is because of the varied length distributions on different datasets. From
Table 1, we can see that the average length of “Sports” and “Yelp” datasets
are shorter than that of the “ML-100k” dataset. Hence the best improvements
are achieved for longer sequences with a length of no more than 50 on the
“ML-100k” dataset. Note that when the sequence length is beyond 50, the
performance drops sharply on all three datasets. This reveals the limitation
of MrTransformer for very long sequences.

###  5.3. Effect of preference coverage

To answer RQ3, we remove the coverage mechanism from MrTransformer (PE) and
keep other settings unchanged. The results of MrTransformer (PE) with/without
coverage mechanism are shown in Table 9.

Table 9. Recall, MRR and NDCG of MrTransformer (PE) with and without coverage mechanism on five datasets. Bold face indicates the better results.  MrTransformer (PE) | Recall | MRR | NDCG
---|---|---|---
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
Beauty | -cover | 0.2169 | 0.2913 | 0.4012 | 0.1421 | 0.1520 | 0.1591 | 0.1606 | 0.1846 | 0.2111
+cover | 0.2286 | 0.3126 | 0.4302 | 0.1444 | 0.1555 | 0.1635 | 0.1653 | 0.1923 | 0.2218
Sports | -cover | 0.2263 | 0.3379 | 0.4897 | 0.1309 | 0.1457 | 0.1560 | 0.1545 | 0.1904 | 0.2286
+cover | 0.2320 | 0.3456 | 0.5000 | 0.1354 | 0.1504 | 0.1616 | 0.1593 | 0.1958 | 0.2347
Toys | -cover | 0.3251 | 0.4191 | 0.5428 | 0.2176 | 0.2301 | 0.2385 | 0.2443 | 0.2746 | 0.3057
+cover | 0.3285 | 0.4218 | 0.5445 | 0.2208 | 0.2332 | 0.2415 | 0.2476 | 0.2776 | 0.3084
ML-100k | -cover | 0.3351 | 0.4984 | 0.6702 | 0.1916 | 0.2131 | 0.2248 | 0.2270 | 0.2796 | 0.3227
+cover | 0.3601 | 0.5005 | 0.6808 | 0.2031 | 0.2211 | 0.2336 | 0.2416 | 0.2866 | 0.3221
Yelp | -cover | 0.4579 | 0.6268 | 0.7901 | 0.2729 | 0.2955 | 0.3069 | 0.3187 | 0.3733 | 0.4147
+cover | 0.4640 | 0.6312 | 0.7972 | 0.2742 | 0.2965 | 0.3080 | 0.3211 | 0.3752 | 0.4172

The results of MrTransformer (PE) with preference coverage are significantly
higher than MrTransformer (PE) without preference coverage in terms of most
metrics on all datasets. On the “Beauty” dataset, adding preference coverage
results in an increase of 7.31%, 2.76% and 4.17% in terms of Recall@10, MRR@20
and NDCG@10, respectively. And on the “ML-100k” dataset, adding preference
coverage achieves an increase of 7.46%, 6.00% and 6.43% in terms of Recall@5,
MRR@5 and NDCG@5, through NDCG@20 drops a bit. This is because, without the
preference coverage, different preferences might attend to the same items,
resulting in redundant preferences and low coverage of the whole sequence.
Besides, the effects of the preference coverage vary on different datasets.
For example, compared with the other datasets, on the “Toys” dataset, adding
preference coverage results in minor increase of 0.64%, 1.47% and 1.35% in
terms of Recall@10, MRR@5 and NDCG@5, respectively. One possible reason is
that the characteristics of the datasets are different, where the number of
items interacted by users are smaller and the data is more sparse on “Toys”
dataset than other datasets. This may result in little similarity between
items. In this case, the attention distribution in Transformer itself is not
very concentrated, so the increase caused by the coverage mechanism is not so
large. What’s more, MrTransformer (PE) without preference coverage performs
worse than some of the baselines on the “Beauty” dataset. This demonstrates
the importance and complementarity of preference coverage for preference
modeling. Comparing the results of MrTransformer in Table 9 and Table 2, we
can obtain that removing the preference editing strategy results in worse
performances in general than removing the preference coverage mechanism. It
demonstrates that the preference editing strategy plays a leading role in
improving recommendation performances.

###  5.4. Impact of the hyperparameter $K$

MrTransformer (PE) concatenates $K$ special tokens to represent user
preferences. Note that $K$ represents latent user preferences for the whole
dataset, not for a particular sequence. To answer RQ4, and study how $K$
affects the recommendation performance of MrTransformer (PE), we compare
different values of $K$ while keeping other settings unchanged. We consider
$K=1,3,5,7,9,11,15,20$.

Table 10. Results of MrTransformer (PE) on the “Beauty” dataset with different values of $K$. $K$ | Recall | MRR | NDCG
---|---|---|---
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
1 | 0.2267 | 0.3068 | 0.4185 | 0.1430 | 0.1566 | 0.1642 | 0.1660 | 0.1918 | 0.2198
3 | 0.2286 | 0.3126 | 0.4302 | 0.1444 | 0.1555 | 0.1635 | 0.1653 | 0.1923 | 0.2218
5 | 0.2263 | 0.3057 | 0.4217 | 0.1432 | 0.1536 | 0.1615 | 0.1638 | 0.1893 | 0.2184
7 | 0.2250 | 0.3043 | 0.4176 | 0.1422 | 0.1567 | 0.1644 | 0.1657 | 0.1913 | 0.2197
9 | 0.2294 | 0.3013 | 0.4170 | 0.1450 | 0.1570 | 0.1637 | 0.1638 | 0.1917 | 0.2215
11 | 0.2237 | 0.3051 | 0.4217 | 0.1441 | 0.1523 | 0.1633 | 0.1642 | 0.1904 | 0.2197
15 | 0.2248 | 0.3021 | 0.4115 | 0.1438 | 0.1550 | 0.1625 | 0.1648 | 0.1895 | 0.2170
20 | 0.2234 | 0.3071 | 0.4173 | 0.1437 | 0.1546 | 0.1621 | 0.1634 | 0.1903 | 0.2180
Table 11. Results of MrTransformer (PE) on the “Sports” dataset with different values of $K$. $K$ | Recall | MRR | NDCG
---|---|---|---
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
1 | 0.2339 | 0.3401 | 0.4812 | 0.1375 | 0.1514 | 0.1610 | 0.1613 | 0.1954 | 0.2309
3 | 0.2238 | 0.3334 | 0.4840 | 0.1307 | 0.1451 | 0.1554 | 0.1537 | 0.1889 | 0.2268
5 | 0.2283 | 0.3375 | 0.4905 | 0.1324 | 0.1468 | 0.1573 | 0.1560 | 0.1912 | 0.2297
7 | 0.2284 | 0.3397 | 0.4956 | 0.1334 | 0.1480 | 0.1586 | 0.1569 | 0.1926 | 0.2318
9 | 0.2320 | 0.3456 | 0.5000 | 0.1354 | 0.1504 | 0.1616 | 0.1593 | 0.1958 | 0.2347
11 | 0.2270 | 0.3349 | 0.4865 | 0.1335 | 0.1476 | 0.1579 | 0.1566 | 0.1912 | 0.2293
15 | 0.2285 | 0.3427 | 0.4965 | 0.1334 | 0.1485 | 0.1589 | 0.1618 | 0.1936 | 0.2322
20 | 0.2175 | 0.3317 | 0.4845 | 0.1259 | 0.1410 | 0.1514 | 0.1485 | 0.1853 | 0.2236
Table 12. Results of MrTransformer (PE) on the “Toys” dataset with different values of $K$. $K$ | Recall | MRR | NDCG
---|---|---|---
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
1 | 0.3215 | 0.4166 | 0.5356 | 0.2156 | 0.2282 | 0.2363 | 0.2419 | 0.2725 | 0.3025
3 | 0.3230 | 0.4177 | 0.5389 | 0.2171 | 0.2296 | 0.2379 | 0.2434 | 0.2739 | 0.3044
5 | 0.3202 | 0.4162 | 0.5446 | 0.2142 | 0.2269 | 0.2357 | 0.2405 | 0.2715 | 0.3038
7 | 0.3257 | 0.4243 | 0.5466 | 0.2160 | 0.2290 | 0.2374 | 0.2432 | 0.2750 | 0.3037
9 | 0.3123 | 0.4069 | 0.5305 | 0.2044 | 0.2169 | 0.2254 | 0.2312 | 0.2616 | 0.2928
11 | 0.3285 | 0.4218 | 0.5445 | 0.2208 | 0.2332 | 0.2415 | 0.2476 | 0.2776 | 0.3084
15 | 0.2997 | 0.3982 | 0.5317 | 0.1913 | 0.2043 | 0.2143 | 0.2182 | 0.2499 | 0.2835
20 | 0.3006 | 0.3968 | 0.5208 | 0.1981 | 0.2108 | 0.2193 | 0.2236 | 0.2546 | 0.2857
Table 13. Results of MrTransformer (PE) on the “ML-100k” dataset with different values of $K$. $K$ | Recall | MRR | NDCG
---|---|---|---
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
1 | 0.3595 | 0.5143 | 0.6808 | 0.1975 | 0.2188 | 0.2297 | 0.2375 | 0.2896 | 0.3297
3 | 0.3601 | 0.5005 | 0.6808 | 0.2031 | 0.2211 | 0.2336 | 0.2416 | 0.2866 | 0.3321
5 | 0.3521 | 0.5068 | 0.6903 | 0.1991 | 0.2193 | 0.2319 | 0.2367 | 0.2862 | 0.3325
7 | 0.3383 | 0.4973 | 0.6787 | 0.1948 | 0.2160 | 0.2289 | 0.2303 | 0.2817 | 0.3280
9 | 0.3521 | 0.4942 | 0.6861 | 0.1961 | 0.2154 | 0.2289 | 0.2346 | 0.2809 | 0.3297
11 | 0.3351 | 0.4942 | 0.6585 | 0.1940 | 0.2150 | 0.2265 | 0.2287 | 0.2799 | 0.3216
15 | 0.3298 | 0.4835 | 0.6702 | 0.2030 | 0.2211 | 0.2341 | 0.2322 | 0.2823 | 0.3294
20 | 0.3552 | 0.5121 | 0.6935 | 0.2010 | 0.2214 | 0.2338 | 0.2308 | 0.2790 | 0.3217
Table 14. Results of MrTransformer (PE) on the “Yelp” dataset with different values of $K$. $K$ | Recall | MRR | NDCG
---|---|---|---
@5 | @10 | @20 | @5 | @10 | @20 | @5 | @10 | @20
1 | 0.4640 | 0.6312 | 0.7972 | 0.2742 | 0.2965 | 0.3080 | 0.3211 | 0.3752 | 0.4172
3 | 0.4500 | 0.6284 | 0.8039 | 0.2787 | 0.2918 | 0.3040 | 0.3154 | 0.3630 | 0.4075
5 | 0.4447 | 0.6173 | 0.7877 | 0.2541 | 0.2771 | 0.2889 | 0.3012 | 0.3570 | 0.4000
7 | 0.4514 | 0.6300 | 0.8012 | 0.2602 | 0.2841 | 0.2960 | 0.3075 | 0.3652 | 0.4086
9 | 0.4385 | 0.6148 | 0.7885 | 0.2490 | 0.2725 | 0.2846 | 0.2958 | 0.3528 | 0.3968
11 | 0.4587 | 0.6361 | 0.8055 | 0.2592 | 0.2830 | 0.2948 | 0.3086 | 0.3660 | 0.4089
15 | 0.4125 | 0.5870 | 0.7634 | 0.2295 | 0.2528 | 0.2651 | 0.2747 | 0.3312 | 0.3758
20 | 0.4354 | 0.6147 | 0.7918 | 0.2488 | 0.2727 | 0.2851 | 0.2949 | 0.3529 | 0.3978

Table 10-14 show that the best values of Recall, MRR and NDCG are achieved
when $K=3$ on the “Beauty” and “ML-100k” datasets, $K=9$ on the “Sports”
dataset, $K=11$ on the “Toys” dataset, and $K=1$ on the “Yelp” dataset. This
reflects that user preferences on the “Beauty”, “ML-100k” and “Yelp” datasets
are less extensive than those on the “Sports” and “Toys” datasets. Note that
this does not conflict with our findings in Section 5.1 (i.e., the
improvements of MrTransformer (PE) on “ML-100k” dataset are larger than those
on other datasets and the advantages of MrTransformer (PE) are more obvious on
“ML-100k”), as $K$ represents the number of latent preferences for all
sequences in the whole dataset in general, not for a particular sequence. It
is also in line with the number of preferences that users reflect in an
interaction sequence in most cases. The lowest scores are achieved when $K$ is
set to a very large value (e.g., $K=15,20$) on most datasets. Although the
value of $K$ can affect recommendation performance, the influence is limited.
On the “Beauty” dataset, the largest gap between the best and the worst
performance is 4.54%, 3.08% and 2.21% in terms of Recall@20, MRR@10 and
NDCG@20, respectively.

##  6\. Conclusions and Future Work

We have proposed a transformer-based model named MrTransformer and introduced
a novel SSL strategy, i.e., preference editing. We have conducted extensive
experiments and analysis on two benchmark datasets to show the effectiveness
of MrTransformer and preference editing.

Our experimental results demonstrate that MrTransformer with preference
editing significantly outperforms state-of-the-art methods in terms of Recall,
MRR and NDCG.

We have designed SSL signals between interaction sequences for the SR
scenario, that we believe have further uses in the context of recommender
systems, including cold-start scenarios and privacy preserving
recommendations.

At the same time, MrTransformer (PE) also has limitations. Importantly, it is
not able to effectively explore user multiple preference from very long
sequences. MrTransformer can be advanced and extended in several directions.
First, rich side information can be taken into consideration for user
preference extraction and representation, as well as for SSL. Second, variants
of MrTransformer can be applied to other recommendation tasks by introducing
other information, such as conversational recommendations.

## Reproducibility

To facilitate the reproducibility of the results, we share the datasets, code
and parameter files used in this paper at
[https://github.com/mamuyang/MrTransformer].

###### Acknowledgements.

This research was partially supported by the National Key R&D Program of China
with grant No. 2020YFB1406704, the Natural Science Foundation of China
(61972234, 61902219, 62072279), the Key Scientific and Technological
Innovation Program of Shandong Province (2019JZZY010129), the Tencent WeChat
Rhino-Bird Focused Research Program (JR-WXG-2021411), the Fundamental Research
Funds of Shandong University, and the Hybrid Intelligence Center, a 10-year
programme funded by the Dutch Ministry of Education, Culture and Science
through the Netherlands Organisation for Scientific Research, https://hybrid-
intelligence-centre.nl. All content represents the opinion of the authors,
which is not necessarily shared or endorsed by their respective employers
and/or sponsors.

## References

  * (1)
  * Ashkan et al. (2015) Azin Ashkan, Branislav Kveton, Shlomo Berkovsky, and Zheng Wen. 2015.  Optimal greedy diversity for recommendation. In _The 24th International Joint Conference on Artificial Intelligence_. 1742–1748.
  * Chen et al. (2020) Wanyu Chen, Pengjie Ren, Fei Cai, Fei Sun, and Maarten de Rijke. 2020.  Improving end-to-end sequential recommendations with intent-aware diversification. In _The 29th Conference on Information and Knowledge Management_. 175–184.
  * Chen et al. (2018) Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2018.  Sequential recommendation with user memory networks. In _The 11th Conferences on Web-inspired research involving Search and Data Mining_. 108–116.
  * Cho et al. (2020) Sung Min Cho, Eunhyeok Park, and Sungjoo Yoo. 2020.  MEANTIME: Mixture of attention mechanisms with multi-temporal embeddings for sequential recommendation. In _The 14th ACM Conference on Recommender Systems_. 515–520.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional Transformers for language understanding. In _The North American Chapter of the Association for Computational Linguistics_. 4171–4186.
  * Donkers et al. (2017) Tim Donkers, Benedikt Loepp, and Jürgen Ziegler. 2017\.  Sequential user-based recurrent neural network recommendations. In _The 11th ACM Conference on Recommender Systems_. 152–160.
  * Felbo et al. (2017) Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017.  Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In _The Conference on Empirical Methods in Natural Language Processing_. 1615–1625.
  * Guo et al. (2020) Cheng Guo, Mengfei Zhang, Jinyun Fang, Jiaqi Jin, and Mao Pan. 2020.  Session-based recommendation with hierarchical leaping networks. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1705–1708.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020.  Momentum contrast for unsupervised visual representation learning. In _Conference on Computer Vision and Pattern Recognition_. 9729–9738.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _The 26th Web Conference_. 173–182.
  * Hidasi et al. (2016a) Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016a.  Session-based recommendations with recurrent neural networks. In _The 4th International Conference on Learning Representations_.
  * Hidasi et al. (2016b) Balázs Hidasi, Massimo Quadrana, Alexandros Karatzoglou, and Domonkos Tikk. 2016b.  Parallel recurrent neural network architectures for feature-rich session-based recommendations. In _The 10th ACM Conference on Recommender Systems_. 241–248.
  * Huang et al. (2019) Jin Huang, Zhaochun Ren, Wayne Xin Zhao, Gaole He, Ji-Rong Wen, and Daxiang Dong. 2019\.  Taxonomy-aware multi-hop reasoning networks for sequential recommendation. In _The 12th Conferences on Web-inspired research involving Search and Data Mining_. 573–581.
  * Huang et al. (2018) Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y Chang. 2018.  Improving sequential recommendation with knowledge-enhanced memory networks. In _The 41st International ACM Conference on Research and Development in Information Retrieval_. 505–514.
  * Kang and McAuley (2018) Wangcheng Kang and Julian McAuley. 2018.  Self-attentive sequential recommendation. In _The 18th International Conference on Data Mining_. 197–206.
  * Lan et al. (2020) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020\.  ALBERT: A lite BERT for self-supervised learning of language representations. In _The 8th International Conference on Learning Representations_.
  * Lewis et al. (2020) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2020\.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In _The 58th Association for Computational Linguistics_. 7871–7880.
  * Li et al. (2019) Chenliang Li, Xichuan Niu, Xiangyang Luo, Zhenzhong Chen, and Cong Quan. 2019.  A review-driven neural model for sequential recommendation. In _The 28th International Joint Conference on Artificial Intelligence_. 2866–2872.
  * Li et al. (2017) Jing Li, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Tao Lian, and Jun Ma. 2017\.  Neural attentive session-based recommendation. In _The 26th Conference on Information and Knowledge Management_. 1419–1428.
  * Li et al. (2020) Jiacheng Li, Yujie Wang, and Julian McAuley. 2020.  Time interval aware self-attention for sequential recommendation. In _The 13th Conferences on Web-inspired research involving Search and Data Mining_. 322–330.
  * Lin et al. (2020) Jing Lin, Weike Pan, and Zhong Ming. 2020.  FISSA: Fusing item similarity models with self-attention networks for sequential recommendation. In _The 14th ACM Conference on Recommender Systems_. 130–139.
  * Linden et al. (2003) Greg Linden, Brent Smith, and Jeremy York. 2003.  Amazon.com recommendations: item-to-item collaborative filtering.  _IEEE Internet Computing_ 1 (2003), 76–80.
  * Liu et al. (2021a) Chang Liu, Xiaoguang Li, Guohao Cai, Zhenhua Dong, Lifeng Shang, and Hong Zhu. 2021a.  Noninvasive self-attention for side information fusion in sequential recommendation. In _The 35th Conference on Artificial Intelligence_.
  * Liu et al. (2018) Qiao Liu, Yifu Zeng, Refuoe Mokhosi, and Haibin Zhang. 2018\.  STAMP: Short-term attention/memory priority model for session-based recommendation. In _The 24th ACM International Conference on Knowledge discovery and data mining_. 1831–1839.
  * Liu et al. (2021b) Zhuang Liu, Yunpu Ma, Yuanxin Ouyang, and Zhang Xiong. 2021b.  Contrastive learning for recommender system. In _arXiv preprint arXiv:2101.01317_.
  * Luo et al. (2020) Anjing Luo, Pengpeng Zhao, Yanchi Liu, Fuzhen Zhuang, Deqing Wang, Jiajie Xu, Junhua Fang, and Victor S Sheng. 2020\.  Collaborative Self-Attention Network for Session-based Recommendation. In _The 29th International Joint Conference on Artificial Intelligence_. 2591–2597.
  * Ma et al. (2020) Jianxin Ma, Chang Zhou, Hongxia Yang, Peng Cui, Xin Wang, and Wenwu Zhu. 2020\.  Disentangled self-Supervision in sequential recommenders. In _The 26th ACM International Conference on Knowledge discovery and data mining_. 483–491.
  * Ma et al. (2019) Muyang Ma, Pengjie Ren, Yujie Lin, Zhumin Chen, Jun Ma, and Maarten de Rijke. 2019\.  $\pi$-Net: A Parallel information-sharing network for shared-account cross-domain sequential recommendations. In _The 42nd International ACM Conference on Research and Development in Information Retrieval_. 685–694.
  * McAuley et al. (2015) Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015.  Image-based recommendations on styles and substitutes. In _The 38th International ACM Conference on Research and Development in Information Retrieval_. 43–52.
  * Meng et al. (2020) Wenjing Meng, Deqing Yang, and Yanghua Xiao. 2020\.  Incorporating user micro-behaviors and item knowledge into multi-task learning for session-based recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1091–1100.
  * Mi et al. (2020) Fei Mi, Xiaoyu Lin, and Boi Faltings. 2020.  Ader: Adaptively distilled exemplar replay towards continual learning for session-based recommendation. In _The 14th ACM Conference on Recommender Systems_. 408–413.
  * Pan et al. (2020a) Zhiqiang Pan, Fei Cai, Yanxiang Ling, and Maarten de Rijke. 2020a.  An intent-guided collaborative machine for session-based recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1833–1836.
  * Pan et al. (2020b) Zhiqiang Pan, Fei Cai, Yanxiang Ling, and Maarten de Rijke. 2020b.  Rethinking item importance in session-based recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1837–1840.
  * Qiu et al. (2020) Ruihong Qiu, Hongzhi Yin, Zi Huang, and Tong Chen. 2020\.  GAG: Global attributed graph neural network for streaming session-based recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 669–678.
  * Quadrana et al. (2017) Massimo Quadrana, Alexandros Karatzoglou, Balzs Hidasi, and Paolo Cremonesi. 2017.  Personalizing session-based recommendations with hierarchical recurrent neural networks. In _The 11th ACM Conference on Recommender Systems_. 130–137.
  * Ren et al. (2019) Pengjie Ren, Zhumin Chen, Jing Li, Zhaochun Ren, Jun Ma, and Maarten de Rijke. 2019\.  RepeatNet: A repeat aware neural recommendation machine for session-based recommendation. In _The 33rd Conference on Artificial Intelligence_. 4806–4813.
  * Ren et al. (2020b) Pengjie Ren, Zhaochun Ren, Fei Sun, Xiangnan He, Dawei Yin, and Maarten de Rijke. 2020b.  NLP4REC: The WSDM 2020 workshop on natural language processing for recommendations. In _The 13th Conferences on Web-inspired research involving Search and Data Mining_. 907–908.
  * Ren et al. (2020a) Ruiyang Ren, Zhaoyang Liu, Yaliang Li, Wayne Xin Zhao, Hui Wang, Bolin Ding, and Ji-Rong Wen. 2020a.  Sequential recommendation with self-attentive multi-adversarial network. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 89–98.
  * Song et al. (2019) Weiping Song, Zhiping Xiao, Yifan Wang, Laurent Charlin, Ming Zhang, and Jian Tang. 2019\.  Session-based social recommendation via dynamic graph attention networks. In _The 12th Conferences on Web-inspired research involving Search and Data Mining_. 555–563.
  * Sun et al. (2019a) Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019a.  BERT4Rec: Sequential recommendation with bidirectional encoder representations from Transformer. In _The 28th Conference on Information and Knowledge Management_. 1441–1450.
  * Sun et al. (2018) Peijie Sun, Le Wu, and Meng Wang. 2018.  Attentive recurrent social recommendation. In _The 41st International ACM Conference on Research and Development in Information Retrieval_. 185–194.
  * Sun et al. (2020a) Peijie Sun, Le Wu, Kun Zhang, Yanjie Fu, Richang Hong, and Meng Wang. 2020a.  Dual learning for explainable recommendation: Towards unifying user preference prediction and review generation. In _The 29th Web Conference_. 837–847.
  * Sun et al. (2019b) Yu Sun, Eric Tzeng, Trevor Darrell, and Alexei A. Efros. 2019b.  Unsupervised domain adaptation through self-supervision. In _arXiv preprint arXiv:1909.11825_.
  * Sun et al. (2020b) Yang Sun, Fajie Yuan, Min Yang, Guoao Wei, Zhou Zhao, and Duo Liu. 2020b.  A generic network compression framework for sequential recommender systems. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1299–1308.
  * Tang and Wang (2018) Jiaxi Tang and Ke Wang. 2018.  Personalized top-n sequential recommendation via convolutional sequence embedding. In _The 11th Conferences on Web-inspired research involving Search and Data Mining_. 565–573.
  * van den Oord et al. (2018) Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018.  Representation learning with contrastive predictive coding. In _arXiv preprint arXiv:1807.03748_.
  * Wang et al. (2020a) Jianling Wang, Kaize Ding, Liangjie Hong, Huan Liu, and James Caverlee. 2020a.  Next-item recommendation with sequential hypergraphs. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1101–1110.
  * Wang et al. (2020c) Meng Wang, Weijie Fu, Xiangnan He, Shijie Hao, and Xindong Wu. 2020c.  A Survey on Large-scale Machine Learning.  (2020).
  * Wang et al. (2019) Meirui Wang, Pengjie Ren, Lei Mei, Zhumin Chen, Jun Ma, and Maarten de Rijke. 2019\.  A collaborative session-based recommendation approach with parallel memory modules. In _The 42nd International ACM Conference on Research and Development in Information Retrieval_. 345–354.
  * Wang et al. (2020b) Pengfei Wang, Yu Fan, Long Xia, Wayne Xin Zhao, Shaozhang Niu, and Jimmy Huang. 2020b.  KERL: A knowledge-guided reinforcement learning model for sequential recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 209–218.
  * Wang et al. (2020d) Ziyang Wang, Wei Wei, Gao Cong, Xiao-Li Li, Xian-Ling Mao, and Minghui Qiu. 2020d.  Global context enhanced graph neural networks for session-based recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 169–178.
  * Wu et al. (2020a) Jibang Wu, Renqin Cai, and Hongning Wang. 2020a.  Déjà vu: A contextualized temporal attention mechanism for sequential recommendation. In _The 29th Web Conference_. 2199–2209.
  * Wu et al. (2020c) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2020c.  Self-supervised graph learning for recommendation. In _arXiv preprint arXiv:2010.10783_.
  * Wu et al. (2020b) Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. 2020b.  SSE-PT: Sequential recommendation via personalized transformer. In _The 14th ACM Conference on Recommender Systems_. 328–337.
  * Wu et al. (2019a) Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James L Sharpnack. 2019a.  Stochastic shared embeddings: data-driven regularization of embedding layers. In _The 33rd Conference on Neural Information Processing Systems_. 24–34.
  * Wu et al. (2019b) Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019b.  Session-based recommendation with graph neural networks. In _The 33rd Conference on Artificial Intelligence_. 346–353.
  * Xia et al. (2021) Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, and Xiangliang Zhang. 2021\.  Self-supervised hypergraph convolutional networks for session-based recommendation. In _The 35th Conference on Artificial Intelligence_.
  * Xie et al. (2020) Xu Xie, Fei Sun, Zhaoyang Liu, Jinyang Gao, Bolin Ding, and Bin Cui. 2020\.  Contrastive Learning for Sequential Recommendation.  _arXiv preprint arXiv:2010.14395_ (2020).
  * Xin et al. (2020) Xin Xin, Alexandros Karatzoglou, Ioannis Arapakis, and Joemon M Jose. 2020.  Self-supervised reinforcement learning for recommender systems. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 931–940.
  * Yao et al. (2020) Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix X. Yu, Aditya Krishna Menon, Lichan Hong, Ed H. Chi, Steve Tjoa, Jieqi Kang, and Evan Ettinger. 2020\.  Self-supervised learning for deep models in recommendations. In _arXiv preprint arXiv:2007.12865_.
  * Ye et al. (2020) Wenwen Ye, Shuaiqiang Wang, Xu Chen, Xuepeng Wang, Zheng Qin, and Dawei Yin. 2020\.  Time matters: Sequential recommendation with complex temporal information. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1459–1468.
  * Yu et al. (2020) Feng Yu, Yanqiao Zhu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2020.  TAGNN: Target attentive graph neural networks for session-based recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1921–1924.
  * Yu et al. (2021) Junliang Yu, Hongzhi Yin, Jundong Li, Qinyong Wang, Nguyen Quoc Viet Hung, and Xiangliang Zhang. 2021.  Self-supervised multi-channel hypergraph convolutional network for social recommendation. In _arXiv preprint arXiv:2101.06448_.
  * Yuan et al. (2020a) Fajie Yuan, Xiangnan He, Karatzoglou Alexandros, and Liguang Zhang. 2020a.  Parameter-efficient transfer from sequential behaviors for user modeling and recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1469–1478.
  * Yuan et al. (2020b) Fajie Yuan, Xiangnan He, Haochuan Jiang, Guibing Guo, Jian Xiong, Zhezhao Xu, and Yilin Xiong. 2020b.  Future data helps training: modeling future contexts for session-based recommendation. In _The 29th Web Conference_. 303–313.
  * Zhai et al. (2019) Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, and Lucas Beyer. 2019.  S4L: Self-supervised semi-supervised learning. In _International Conference on Computer Vision_. 1476–1485.
  * Zhang et al. (2019) Tingting Zhang, Pengpeng Zhao, Yanchi Liu, Victor Sheng, Jiajie Xu, Deqing Wang, Guanfeng Liu, and Xiaofang Zhou. 2019\.  Feature-level deeper self-attention network for sequential recommendation. In _The 28th International Joint Conference on Artificial Intelligence_. 4320–4326.
  * Zheng et al. (2020) Lin Zheng, Naicheng Guo, Weihao Chen, Jin Yu, and Dazhi Jiang. 2020.  Sentiment-guided sequential recommendation. In _The 43rd International ACM Conference on Research and Development in Information Retrieval_. 1957–1960.
  * Zhou et al. (2020a) Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, and Hongxia Yang. 2020a.  Contrastive learning for debiased candidate generation in large-scale recommender systems. In _arXiv preprint arXiv:2005.12964_.
  * Zhou et al. (2020b) Kun Zhou, Hui Wang, WayneXin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020b.  S3-Rec: Self-supervised learning for sequential recommendation with mutual information maximization. In _The 29th Conference on Information and Knowledge Management_. 1893–1902.
  * Zhuang et al. (2017) Fuzhen Zhuang, Yingmin Zhou, Fuzheng Zhang, Xiang Ao, Xing Xie, and Qing He. 2017\.  Cross-domain novelty seeking trait mining for sequential recommendation. In _The 26th Web Conference_. 881–882.

  *[SR]: sequential recommendation
  *[SSL]: self-supervised learning
  *[CV]: computer vision
  *[NLP]: neural language processing
  *[MIP]: masked item prediction
  *[ICL]: item contrastive learning
  *[RNN]: recurrent neural network
  *[SRs]: sequential recommendation
  *[RL]: reinforcement learning
  *[SSE]: stochastic shared embeddings