# Contrastive Self-supervised Sequential Recommendation with Robust
Augmentation

Zhiwei Liu  University of Illinois at Chicago<EMAIL_ADDRESS>,  Yongjun
Chen  Salesforce Research<EMAIL_ADDRESS>,  Jia Li  Salesforce
Research<EMAIL_ADDRESS>,  Philip S. Yu  University of Illinois at
Chicago<EMAIL_ADDRESS>,  Julian McAuley  UC San Diego<EMAIL_ADDRESS>and
Caiming Xiong  Salesforce Research<EMAIL_ADDRESS>
(2021)

###### Abstract.

_Sequential Recommendation_ describes a set of techniques to model dynamic
user behavior in order to predict future interactions in sequential user data.
At their core, such approaches model transition probabilities between items in
a sequence, whether through Markov chains, recurrent networks, or more
recently, Transformers. However both old and new issues remain, including
data-sparsity and noisy data; such issues can impair performance, especially
in complex, parameter-hungry models. In this paper, we investigate the
application of contrastive Self-Supervised Learning (SSL) to sequential
recommendation, as a way to alleviate some of these issues. Contrastive SSL
constructs augmentations from unlabelled instances, where agreements among
positive pairs are maximized. It is challenging to devise a contrastive SSL
framework for sequential recommendation, due to its discrete nature,
correlations among items, and skewness of length distributions. To this end,
we propose a novel framework, Contrastive Self-supervised Learning for
Sequential Recommendation (CoSeRec). We introduce two informative augmentation
operators leveraging item correlations to create high quality views for
contrastive learning. Experimental results on three real-world datasets
demonstrate the effectiveness of the proposed method on improving model
performance, and the robustness against sparse and noisy data. Our
implementation is available: https://github.com/YChen1993/CoSeRec

Contrastive Learning, Robustness, Sequential Recommendation

††copyright: acmcopyright††journalyear: 2021††doi:
nn.nnnn/nnnnnnn.nnnnnnn††conference: CIKM ’21: The 30th ACM International
Conference on Information and Knowledge Management; November 1-5, 2021;
Online††booktitle: CIKM ’21: The 30th ACM International Conference on
Information and Knowledge Management, November 1-5, 2021, Online††price:
15.00††isbn: XXX-X-XXXX-XXXX-X/YY/MM

##  1\. Introduction

Recommender systems aim at predicting potential interests of users toward
unseen items (Rendle et al., 2009; Liu et al., 2020; Wang et al., 2019) by
leveraging historical interaction records. In order to model dynamic user
behavior (and especially _short-term_ dynamics), Sequential Recommendation
(SR) (Rendle et al., 2010; Hidasi et al., 2015; Kang and McAuley, 2018; Sun et
al., 2019; Wu et al., 2020b) arguably represents the current state-of-the-art.
SR models interactions between users and items as temporally-ordered
sequences, with the goal of predicting the next interaction conditioned on
recent observations.

The core idea to model SR problems is to capture relationships among items in
sequences (Rendle et al., 2010; Wu et al., 2017; Kang and McAuley, 2018;
Hidasi et al., 2015). Pioneering works (Rendle et al., 2010; He and McAuley,
2016) employ Markov chains to learn pair-wise transition relationships. Later,
RNN-based (Hidasi et al., 2015; Wu et al., 2017; Yu et al., 2016) models were
proposed to infer sequence-wise correlations. More recently, the powerful
ability of Transformers (Vaswani et al., 2017; Devlin et al., 2018) in
encoding sequences has been adopted for applications in SR (Kang and McAuley,
2018; Sun et al., 2019; Li et al., 2020; Ye et al., 2020). Transformer-based
SR models (Kang and McAuley, 2018; Sun et al., 2019; Wu et al., 2020b) encode
sequences by learning item importances via self-attention mechanisms.

Despite the effectiveness of existing approaches, several issues remain less
explored: (1) Data-sparsity. Transformers are motivated by NLP tasks (Vaswani
et al., 2017), where large corpora are available to train complex models
(Devlin et al., 2018). However, SR tasks usually involve rather sparse
datasets (Liu et al., 2021; McAuley et al., 2015), which undermines the
capability of a Transformer to model item correlations in sequences, e.g.
performing poorly on short sequences (Liu et al., 2021). (2) Noisy
interactions. Occasionally, items in sequences may not reflect true item
correlations, e.g. a user consumes an item because of promotion ads,
ultimately resulting in negative feedback (Wang et al., 2021). Inference from
those noisy sequences spoils the performance of a model in revealing item
transition correlations, thus producing less satisfying recommendation.

Inspired by recent developments of Self-Supervised Learning (SSL) (Misra and
Maaten, 2020; Lan et al., 2019; Devlin et al., 2018; Fang et al., 2020; Jiao
et al., 2020; Wu et al., 2020c), this paper studies the possibility of
contrastive self-supervised learning (Chen et al., 2020a; Fang et al., 2020;
Wu et al., 2020c) for alleviating the aforementioned issues in SR.
Intuitively, SSL constructs augmentations from unlabelled data, and enhances
the discrimination ability of encoders by employing a contrastive loss (Zhou
et al., 2020a; Chen et al., 2020a; He et al., 2020; You et al., 2020). The
contrastive loss maximizes agreements between positive pairs, which are
different augmentations (i.e. views) from one instance. It is worth noting
that this learning scheme is different from widely used contrastive losses in
recommender systems, e.g. BPR (Rendle et al., 2009) and NCE (Gutmann and
Hyvärinen, 2010) losses, because contrastive SSL involves no prediction
targets between two different entities but only the optimization of the
consistency between two views of one instance. Nevertheless, it is challenging
to devise a contrastive SSL framework for SR due to the following reasons:

  * •

Discreteness: Items in sequences are represented by discrete IDs, which pose a
challenge for popular augmentation methods designed for continuous feature
spaces (Chen et al., 2020a; He et al., 2020). Augmentation techniques for
sequence data are still under-explored and require investigation.

  * •

Item Correlations: Items in sequences are correlated with each other. However,
existing techniques (Chen et al., 2020a; Fang et al., 2020; Wu et al., 2020c;
Xie et al., 2020) augment sequences based on random item perturbations. Those
methods tend to destroy the original item relationships in sequences when
constructing augmentations, thus impairing the confidence of positive pairs.

  * •

Length Skewness: According to (Liu et al., 2021), the length distribution of
sequences is skewed and suffers from the long-tail patterns. However, current
contrastive SSL (Chen et al., 2020a; Xie et al., 2020; Wu et al., 2020c)
frameworks employ identical augmentation methods across instances. This leads
to less confident contrastive pairs in SR, e.g., short sequences are more
sensitive to random item perturbations.

To this end, we propose a new framework, Contrastive Self-Supervised learning
for Sequential Recommendation (CoSeRec). It contains three key components: (1)
robust data augmentation that characterizes item correlations and sequence
length skewness; (2) a contrastive SSL objective to maximize the agreement of
positive views of sequences; and (3) a multi-task training strategy to jointly
optimize the SR objective and the contrastive SSL objective effectively. To be
more specific, we first propose two informative augmentation methods,
_substitute_ and _insert_ , which advances current random augmentations (Xie
et al., 2020; Zhou et al., 2020b), such as _crop_ , _reorder_ , and _mask_.
Random augmentations may yield less confident positive pairs, as those random
perturbations of items break item relationships in sequences. Random
augmentations can also hinder the quality of learned representations, because
they (e.g. crop) often lead to fewer items in sequences, which exaggerates the
cold-start issue. Unlike random augmentations, our informative augmentations
leverage item correlations when constructing positive views, thus being more
robust. Additionally, both ‘substitute’ and ‘insert’ operations expand user
interaction records, which alleviates the cold-start issue. CoSeRec adopts a
multi-task training strategy rather than a pre-training scheme (Chen et al.,
2020a; Wu et al., 2020d), because next-item prediction and the contrastive SSL
share similar targets, i.e. modeling item relationships in sequences.
Experiments on three real-world datasets verify the efficacy of CoSeRec. We
summarize our contributions as follows:

  * •

We devise a novel learning framework (CoSeRec), which addresses data-sparsity
and noisy interaction issues by unifying contrastive SSL with sequential
recommendation.

  * •

We propose novel and robust sequence augmentation methods that exploit item
correlations and alleviate length skewness problems in sequential
recommendation.

  * •

We conduct extensive experiments on three benchmark datasets with detailed
analyses of the proposed paradigm and the optimal combinations of
augmentations.

##  2\. Related Work

###  2.1. Sequential Recommendation

SR predicts future items in user sequences by modeling item transition
correlations. Pioneering works (Rendle et al., 2010; He and McAuley, 2016)
adopt Markov chains to model the pair-wise item transition correlations. FPMC
(Rendle et al., 2010) directly factorizes item transition matrices. Fossil (He
and McAuley, 2016) extends this idea by leveraging similarities between items,
which alleviates sparsity issues. Later, Recurrent Neural Network (RNN) have
been adapted to solve SR (Hidasi et al., 2015; Wu et al., 2017; Yu et al.,
2016), ostensibly modeling sequence-level correlations among transitions.
Hierarchical RNNs (Quadrana et al., 2017) enhance RNNs using personalization
information. Wu, et al. (Wu et al., 2017) apply LSTMs to explore both long-
term and short-term item transition correlations. The major drawback of Markov
chain and RNN models is that their receptive fields of the transition function
are limited. Therefore, they can only incorporate shallow item correlations,
which limits their power to encode sequences (Chen et al., 2018).

Recently, owing to the success of self-attention models (Vaswani et al., 2017;
Devlin et al., 2018) in NLP tasks, a series of Transformer-based SR models
have been proposed (Kang and McAuley, 2018; Sun et al., 2019; Ma et al., 2020;
Wu et al., 2020a). SASRec (Kang and McAuley, 2018) applies Transformer layer
to learn item importance in sequences, which characterize complex item
transition correlations. Later, inspired by BERT (Devlin et al., 2018) model,
BERT4Rec (Sun et al., 2019) is proposed with a bidirectional Transformer
layer. Other works (Ma et al., 2020; Wu et al., 2020b; Li et al., 2020; Fan et
al., 2021b, a) also extend Transformer to incorporate complex signals in
sequences, which verifies the efficacy of Transformer in solving SR. However,
a recent work (Liu et al., 2021) argues Transformer is vulnerable to severe
user cold-start issues in SR, where existing Transformer-based models yield
unsatisfying results for short sequences. Therefore, augmentation for short
sequences is desirable. S3-Rec (Zhou et al., 2020b) and CLS4Rec (Xie et al.,
2020) also investigate contrastive learning in SR. However, both investigate
weak self-supervised signals with random augmentations in sequences, but do
not yet address the length skewness and item correlation challenges.

###  2.2. Contrastive Self-supervised Learning

Contrastive learning has recently achieved remarkable successes when coupled
with the Self-Supervised Learning (SSL) framework, in areas ranging from
Computer Vision (CV) (Chen et al., 2020a; He et al., 2020), Natural Language
Understanding (NLU) (Fang et al., 2020; Gao et al., 2021; Wu et al., 2020d),
graph embedding (You et al., 2020; Jiao et al., 2020; Zhu et al., 2020), as
well as recommender systems (Wu et al., 2020c; Zhou et al., 2020b; Xie et al.,
2020). Contrastive SSL trains encoders by maximizing agreement between two
augmented views of one instance, i.e., positive pairs. As such, it encourages
self-supervised training from the space of unlabelled data. The core of
Contrastive SSL is to exploit strong augmentations (Xiao et al., 2020) of data
instances, which are diverse with respect to the data domains and objective
tasks.

SimCLR (Chen et al., 2020a) proposes a simple contrastive learning framework
for visual representations, which studies stochastic image augmentations.
Following this, (Tian et al., 2020; Xiao et al., 2020) argues that the optimal
choice of augmentations is critically task-oriented and exists with balanced
mutual information. As for NLU tasks, the augmentation (Wei and Zou, 2019) for
text is of a discrete nature. Deleting (Wu et al., 2020d; Giorgi et al.,
2020), reordering (Wu et al., 2020d) and substituting (Fang et al., 2020; Wu
et al., 2020d) words in a sentence leads to positive augmentation pairs, which
also inspires this work. The augmentation in graph contrastive SSL differs
from other domains in its structural characteristics. Thus, node-drop (You et
al., 2020; Zhu et al., 2020), edge-drop (You et al., 2020; Zhu et al., 2020),
and random walk (Qiu et al., 2020; Veličković et al., 2018) are adopted as the
augmentation operations.

In recommendation scenarios, contrastive learning is not a new technology,
e.g., the widely used BPR loss (Rendle et al., 2012) and NCE loss (Gutmann and
Hyvärinen, 2010), both adopt pair-wise contrastive losses on positive and
negative samples. Their recent union with SSL shows promise in improving
recommender systems (Wu et al., 2020c). S3-Rec (Zhou et al., 2020b) devises
contrastive SSL to maximize the mutual information over attributes and
sequence augmentations, which adopts random masks of attributes and items.
(Yao et al., 2020) proposes two-stage augmentations by masking item embedding
layers and dropping categorical features, which enables SSL for item encoders.
SGL (Wu et al., 2020c) advances graph-based recommender systems with SSL by
employing graph structure augmentations. A recent work, CLS4Rec (Xie et al.,
2020) proposes a contrastive SSL framework for improving SR. Although close to
our paper, CLS4Rec adopt random augmentation methods, ignoring crucial item
correlations and length skewness for sequence augmentations.

##  3\. Preliminaries

Figure 1. Overall framework. (a) illustrates the structure of Transformer. (b)
presents the sequence encoding and prediction of Transformer; $v_{i}$ denotes
an item at position $i$ of the sequence $s_{u}$. (c) demonstrates the
contrastive SSL of a sequence. It first augments a sequence as two views with
two augmentation methods. Then, it encodes the sequence by concatenating
embedding outputs from Transformer. Finally, it maximizes the agreement
between the two views.

###  3.1. Problem Formulation

We denote user and item sets as $\mathcal{U}$ and $\mathcal{V}$ respectively.
Each user $u\in\mathcal{U}$ is associated with a sequence of items in
chronological order $s_{u}=[v_{1},\dots,v_{t},\dots,v_{|s_{u}|}]$, where
$v_{t}\in\mathcal{V}$ denotes the item that $u$ has interacted with at time
$t$ and $|s_{u}|$ is the total number of items. SR tasks seek to predict the
next item $v_{|s_{u}|+1}$, which is formulated as follows:

(1)
$\underset{v_{i}\in\mathcal{V}}{\mathrm{arg\,max}}~{}P(v_{|s_{u}|+1}=v_{i}\left|s_{u}\right.),$

which is interpreted as calculating the probability of all candidate items and
selecting the highest one for recommendation.

###  3.2. Transformer for SR

Transformer (Vaswani et al., 2017) architecture is a powerful way to encode
sequences, leading to successful application in SR (Kang and McAuley, 2018; Li
et al., 2020; Zhou et al., 2020b). It consists of two key components: a multi-
head self-attention module and a position-wise Feed-Forward Network (FFN). We
illustrate the structure of a Transformer in Figure 1(a). Multi-head self-
attention models the item correlations in sequence. Position-wise FFN outputs
a bag of embeddings, where the embedding at each position predicts the
corresponding next item in the sequence. We present next-item prediction in
Figure 1(b). To keep generality, we denote a Transformer encoder as
$\mathsf{SeqEnc}(\cdot)$, which can be an arbitrary sequence encoder. We
formulate the encoding process as:

(2) $\mathbf{h}_{u}=\mathsf{SeqEnc}(s_{u}),$

where $\mathbf{h}_{u}$ denotes the sequence embedding of $s_{u}$.
$\mathbf{h}_{u}$ is a bag of embeddings in Transformer. For each position $t$,
$\mathbf{h}_{u}^{t}$, represents a predicted next-item (Kang and McAuley,
2018). We adopt the log-likelihood loss function to optimize the encoder for
next-item prediction as follows:

(3)
$\mathcal{L}_{\text{rec}}(u,t)=-\log(\sigma(\mathbf{h}_{u}^{t}\cdot\mathbf{e}_{v_{t+1}}))-\sum_{v_{j}\not\in
s_{u}}\log(1-\sigma(\mathbf{h}_{u}^{t}\cdot\mathbf{e}_{v_{j}})),$

where $\mathcal{L}_{\text{rec}}(u,t)$ denotes the loss score for the
prediction at position $t$ in sequence $s_{u}$, $\sigma$ is the non-linear
activation function, $\mathbf{e}_{v_{t+1}}$ denotes the embedding for item
$v_{t+1}$, and $v_{j}$ is the sampled negative item for $s_{u}$. The
embeddings of items are retrieved from the embedding table in
$\mathsf{SeqEnc}$, which is jointly optimized with Transformer.

##  4\. Methodology

We first introduce data augmentations adopted in this paper, including both
random and informative augmentations. Then we describe contrastive SSL with
those augmentations. Finally, we present the overall training algorithm.

###  4.1. Robust Sequence Augmentation

We first review and formulate existing random augmentation methods (Wu et al.,
2020d; Xie et al., 2020; Zhou et al., 2020b), and then introduce two novel
informative augmentation methods. We also describe how to augment sequences
based on their length. This section assumes the original sequence being
$s_{u}=[v_{1},v_{2},\dots,v_{n}]$. Toy examples of those methods are
illustrated in Figure 2, where $n=4$.

(a) Random

(b) Informative

Figure 2. Five augmentations adopted in this paper. The original sequence is
$[v_{1},v_{2},v_{3},v_{4}]$. Informative augmentations incorporate
correlations among items.

####  4.1.1. Random Augmentation

Three random operators introduced by (Zhou et al., 2020b; Xie et al., 2020)
are included in this paper to create various views of original sequences:

  * •

Crop ($\mathsf{C}$). Randomly select a continuous sub-sequence of the original
sequence starting from position $i$:

(4) $s_{u}^{\mathsf{C}}=\mathsf{C}(s_{u})=[v_{i},v_{i+1},...,v_{i+c-1}],$

where $c=\lceil\eta n\rceil$ is the sub-sequence length that is controlled by
a hyperparameter $\eta$ and $0\leq\eta\leq 1$. $\lceil\cdot\rceil$ is the
ceiling function.

  * •

Mask ($\mathsf{M}$). Randomly mask $l=\lceil\mu n\rceil$ items in a sequence:

(5)
$s_{u}^{\mathcal{M}}=\mathsf{M}(s_{u})=[v_{1}^{\prime},v_{2}^{\prime},...,v_{n}^{\prime}],$

where $v_{i}^{\prime}$ is the ‘mask’ if $v_{i}$ is a selected item, otherwise
$v_{i}^{\prime}=v_{i}$. $l$ is controlled by a hyper-parameter $\mu$, where
$0\leq\mu\leq 1$.

  * •

Reorder ($\mathsf{R}$). Randomly shuffle a sub-sequence
$\left[v_{i},\cdots,v_{i+r-1}\right]$ of $s_{u}$ as
$[{v}^{\prime}_{i},\ldots,{v}^{\prime}_{i+r-1}]$:

(6)
$s_{u}^{\mathsf{R}}=\mathsf{R}(s_{u})=[v_{1},v_{2},\cdots,{v}^{\prime}_{i},\cdots,{v}^{\prime}_{i+r-1},\cdots,v_{n}],$

where $r=\lceil\omega n\rceil$ is the sub-sequence length and $0\leq\omega\leq
1$.

In general, good augmentations for one instance should give views sharing
task-oriented information (Tian et al., 2020), thus being positive pairs. In
our case, views from one sequence are supposed to maintain the original
sequential correlations, which otherwise could yield less confident positive
pairs. However, those random augmentations break the item correlations in
sequences, especially for short sequences. For example, reordering a sequence
leads to changed item correlations in the sub-sequence. This impact is
enlarged when fewer items are present in sequences.

Moreover, random augmentations exaggerate cold-start issues in sequences, e.g.
masking a short sequence induces few items in a sequence. As such, the
sequence encoder may fail to learn high quality representations, and thus fail
to characterize item relationships in sequences. Contrasting on low quality
sequence representations further deteriorates the robustness of SSL.

Next, we will introduce two novel informative augmentation methods, which
advance random augmentations regarding the mentioned concerns.

####  4.1.2. Informative Augmentation

We propose two informative augmentation operators leveraging item correlations
to generate robust augmented sequences, as illustrated in Figure 2(b).

  * •

Substitute ($\mathsf{S}$). The substitution augmentation is motivated by the
real case (Chen et al., 2020b) where recommending substitutable items to users
expands the chance of discovering their actual interests. Correspondingly,
substituting items in sequences with highly correlated item injects less
corruption to the original sequential information, which thus yields confident
positive pairs of views. Formally, we randomly select $k$ different indices
$\\{\mathrm{idx}_{1},\mathrm{idx}_{2},\dots,\mathrm{idx}_{k}\\}$ in the
sequence $s_{u}$, where $k=\lceil\alpha n\rceil$ and
$\mathrm{idx}_{i}\in[1,2,\dots,n]$. $\alpha\in[0,1]$ is substitution ratio.
Then we replace each with an correlated item based on the selected indices.
The sequence after substitution will be:

(7)
$s_{u}^{\mathsf{S}}=\mathsf{S}(s_{u})=[v_{1},v_{2},...,\bar{v}_{\mathrm{idx}_{i}},\dots,v_{|s_{u}|}],$

where $\bar{v}_{\mathrm{idx}_{i}}$ is a correlated item with
$v_{\mathrm{idx}_{i}}$. We will describe how to choose the correlated items in
Section 4.1.3.

  * •

Insert ($\mathsf{I}$). In practice, recorded interactions are less than the
complete consumption behavior of users, e.g. transactions from other sources
are missing (Liu et al., 2019). Consequently, training sequences may fail to
track comprehensive user dynamics and item correlations. To this end, we
construct augmentations by inserting items into sequences, in order to
‘complete’ the sequence. The number of inserted items is controlled by the
ratio $\beta\in[0,1]$. We first randomly select $k$ different indices
$\\{\mathrm{idx}_{1},\mathrm{idx}_{2},\dots,\mathrm{idx}_{k}\\}$ in the
sequence $s_{u}$, where $k=\lceil\beta n\rceil$ and
$\mathrm{idx}_{i}\in[1,2,\dots,n]$. We then insert correlated items at those
indices. After insertion, the sequence is:

(8)
$s_{u}^{\mathsf{I}}=\mathsf{I}(s_{u})=[v_{1},v_{2},...,\bar{v}_{\mathrm{idx}_{i}},v_{\mathrm{idx}_{i}},\dots,v_{n}],$

where $i\in\\{1,2,\dots,k\\}$, and $\bar{v}_{\mathrm{idx}_{i}}$ is the most
correlated item with $v_{\mathrm{idx}_{i}}$. The length of the augmented
sequence $s_{u}^{\mathsf{I}}$ is $k+n$.

Both augmentations are informative as they perturb the sequence with
correlated items. As such, augmented sequences have higher confidence of being
positive pairs. Moreover, adding correlated items to sequences extends the
number of user interactions, which alleviates user cold-start issues. Next, we
introduce how to choose correlated items for informative augmentations.

####  4.1.3. Item Correlations

Informative augmentations require inferring correlations among items to
perturb sequences. Thus, it is necessary to design a simple yet effective
method for calculating item correlations. We first introduce two
straightforward methods and then demonstrate a hybrid method adopted in this
work.

The first is memory-based correlation. Collaborative signals are crucial in
recommender systems. Therefore, we employ item-based collaborative filtering
considering inverse user frequency (ItemCF-IUF (Breese et al., 2013)) to
measure item correlations due to its simplicity and effectiveness. Formally,
the memory-based correlation score between item $i$ and $j$ is defined as:

(9)
$\mathrm{{Cor}_{o}}(i,j)=\frac{1}{\sqrt{|\mathcal{N}(i)||\mathcal{N}(j)|}}\sum_{u\in\mathcal{N}(i)\cap\mathcal{N}(j)}\frac{1}{\log(1+|\mathcal{N}(u)|)},$

where $u$ is a user, $|\mathcal{N}(i)|$, and $|\mathcal{N}(j)|$ are the number
of users who have interacted with $i$ and $j$, respectively. Memory-based
correlation treats two items as being more correlated if they share a higher
ratio of common users.

The other is model-based correlation. This correlation directly infers
correlations by measuring the similarity between item representations. Since
item representations are jointly learned with the encoder, this method is thus
model-based. In this work, we adopt the dot-product as the similarity metric.
Given the representations of items $i$ and $j$ as $\mathbf{e}_{i}$ and
$\mathbf{e}_{i}$, the model-based correlation score is defined as:

(10) $\mathrm{Cor_{e}}(i,j)=\mathbf{e}_{i}\cdot\mathbf{e}_{j}.$

In this paper, we fuse the memory-based and model-based correlations as a
hybrid correlation $\mathrm{Cor_{h}}$, which is defined as the highest value
between memory-based and model-based correlations as:

(11)
$\mathrm{Cor_{h}}(i,j)=\max(\mathrm{\overline{Cor}_{o}}(i,j),\mathrm{\overline{Cor}_{e}}(i,j)),$

where $\mathrm{\overline{Cor}_{o}}$ and $\mathrm{\overline{Cor}_{e}}$ are the
normalized score of memory-based and model-based correlations, respectively.
We normalize (min-max normalization) them to be comparable. Additionally,
since item representations learned at early epochs are not informative, we
initially use $\mathrm{Cor_{o}}$ to infer correlated items. Then, after
training with $E$ epochs, we switch to $\mathrm{Cor_{h}}$, where $E$ is a
hyper-parameter.

####  4.1.4. Augment w.r.t. Sequence Length

The length of interaction sequences tends to follow a long-tail distribution
(Liu et al., 2021), in which most sequences are short. Considering that short
sequences are more sensitive to random item perturbations, we should carefully
choose augmentation operations on short sequences. In this work, we employ
different augmentation operators sets to sequences with respect to their
lengths. We use a hyper-parameter $K$ to decide whether a sequence is short or
long, and then apply data augmentation as follows:

(12)
$s_{u}^{{a}}=\left\\{\begin{array}[]{lr}{{a}}(s_{u}),~{}{{a}}\sim\\{\mathsf{S},\mathsf{I},\mathsf{M}\\},&|s_{u}|\leq
K\\\
{{a}}(s_{u}),~{}{a}\sim\\{\mathsf{S},\mathsf{I},\mathsf{M},\mathsf{C},\mathsf{R}\\},&|s_{u}|>K\end{array}\right.$

where ${a}$ is an augmentation operator selecting from the corresponding
augmentation set. Though $\mathsf{M}$ is a random augmentation, we also
include it in the augmentation set for short sequences. The reason are
twofold: Firstly, mask augmentation implicitly models item relationships in
sequence, which is similar to the next-item prediction target. Secondly,
masking operation encourages items near masked items being closer, which
models high-order item relationships in the original sequence.

###  4.2. Contrastive Self-Supervision

Contrastive SSL optimizes encoders by maximizing the agreements between
‘positive’ pairs, which are two augmentations from one sequence. To be
specific, given a minibatch of sequences $\\{s_{u}\\}_{u=1}^{N}$, we sample
two augmentation operators for each sequence $s_{u}$, which obtains $2N$
augmented sequences as:

(13)
$\\{\tilde{s}_{1},\tilde{s}_{2},\dots,\tilde{s}_{2u-1},\tilde{s}_{2u},\dots,\tilde{s}_{2N-1},\tilde{s}_{2N}\\},$

where $u\in\\{1,2,\dots,N\\}$. Following (Chen et al., 2020a), each pair
$(\tilde{s}_{2u-1},\tilde{s}_{2u})$ is treated as a positive pair, and the
other $2(N-1)$ augmented views are considered as negative samples for this
pair. The augmented sequences are then encoded as in Eq. (2) with a shared
sequence encoder. For each sequence pair $(\tilde{s}_{2u-1},\tilde{s}_{2u})$,
their representations are
$(\tilde{\mathbf{h}}_{2u-1},\tilde{\mathbf{h}}_{2u})$. We adopt the NT-Xent
loss (Chen et al., 2020a) for optimization as follows:

(14)
$\mathcal{L}_{\mathrm{ssl}}(\mathbf{\tilde{h}}_{2u-1},\mathbf{\tilde{h}}_{2u})=-\log\frac{\exp(\text{sim}(\mathbf{\tilde{h}}_{2u-1},\mathbf{\tilde{h}}_{2u}))}{\sum_{m=1}^{2N}\mathbbm{1}_{m\neq
2u-1}\exp(\text{sim}(\mathbf{\tilde{h}}_{2u-1},\mathbf{\tilde{h}}_{m}))},$

where $\mathrm{sim}(\cdot)$ is a dot product to measure the similarity between
two augmented views, and $\mathbbm{1}_{|m\neq 2u-1|}\in\\{0,1\\}$ is an
indicator function. Since the output embeddings from a Transformer encoder are
position-wise, we thus concatenate representations at all positions as the
sequence representation. We illustrate the contrastive SSL framework in Figure
1(c).

###  4.3. Multi-Task Training

Because next-item prediction and contrastive SSL both model item relationships
in sequences, to boost sequential recommendation performance with the
contrastive SSL objective, we leverage a multi-task strategy to optimize them
jointly as follows:

Input: hyparameters $\alpha$, $\beta$, $K$, $E$, and $\lambda$, max training
epochs $B$, batch size $N$, and a memory-based correlation matrix
$\mathrm{Cor_{o}}$.

Output: $\mathsf{SeqEnc}(\cdot)$.

1 while  _ $epoch\leq B$_ do

// Item correlation selection (Sec. 4.1.3)

2

3 if  _$\mathit{epoch}\leq E$_ then

4 Leverage $\mathrm{Cor_{o}}$ for informative augmentations.

5 else

6 Leverage $\mathrm{Cor_{h}}$ for informative augmentations.

// End-to-end optimization

7 for  _sampled minibatch $\\{s_{u}\\}_{u=1}^{N}$_ do

8 for  _$u\in\\{1,2,\cdots,N\\}$_ do

// Construct two augmentations.

9 if  _$|s_{u}|\leq K$_ then

10 sample two operators:
$a_{1}^{u},a_{2}^{u}\sim\\{\mathsf{I},\mathsf{S},\mathsf{M}\\}$

11 else

12 sample two operators:
$a_{1}^{u},a_{2}^{u}\sim\\{\mathsf{I},\mathsf{S},\mathsf{M},\mathsf{C},\mathsf{R}\\}$

13 $\tilde{s}_{2u-1}=a_{1}^{u}(s_{u}),\tilde{s}_{2u}=a_{2}^{u}(s_{u})$

// Representation learning via $\mathsf{SeqEnc}$

14

15 $\tilde{\mathbf{h}}_{u}=\mathsf{SeqEnc}(s_{u})$

16
$\tilde{\mathbf{h}}_{2u-1},\tilde{\mathbf{h}}_{2u}=\mathsf{SeqEnc}(\tilde{s}_{2u-1}),\mathsf{SeqEnc}(\tilde{s}_{2u})$

17

// Multi-Task optimization

18
$\mathcal{L}=\frac{1}{N}\sum_{u=1}^{N}\mathcal{L}_{\mathrm{rec}}(\tilde{\mathbf{h}}_{u})+\lambda\frac{1}{2N}\sum_{u=1}^{N}[\mathcal{L}_{\mathrm{ssl}}(\tilde{\mathbf{h}}_{2u-1},\tilde{\mathbf{h}}_{2u})+\mathcal{L}_{\mathit{ssl}}(\tilde{\mathbf{h}}_{2u},\tilde{\mathbf{h}}_{2u-1})]$

19

20 update network $\mathsf{SeqEnc}(\cdot)$ to minimize $\mathcal{L}$

21

22

Algorithm 1 CoSeRec training.

(15)
$\mathcal{L}=\mathcal{L}_{\mathrm{rec}}+\lambda\mathcal{L}_{\mathrm{ssl}},$

where $\lambda$ is a hyper-parameter to control the intensity of contrastive
SSL. Algorithm 1 summarizes the training process of CoSeRec. Alternatively, a
two-stage optimization, i.e. first pre-training the encoder with SSL and then
fine-tuning with next-item prediction, can be adopted. We provide comparisons
in Section 5.5.3, which indicates joint training is better than two-stage
training.

##  5\. Experiments

In this section, we conduct experiments on three public datasets and answer
the following Research Questions (RQs):

  * •

RQ1: How does CoSeRec perform on sequential recommendations compared with
existing methods?

  * •

RQ2: What are the optimal augmentation methods for sequential recommendation?

  * •

RQ3: Can CoSeRec perform robustly against data-sparsity and noisy interactions
issues?

  * •

RQ4: How do different settings influence CoSeRec’s performance?

###  5.1. Experimental Setting

####  5.1.1. Datasets

We conduct experiments on three public datasets collected from two real-word
platforms. _Beauty_ and _Sports_ are two subcategories of Amazon review data
introduced in (McAuley et al., 2015). The Yelp111https://www.yelp.com/dataset
dataset is a dataset for business recommendation.

We follow common practice in (Zhou et al., 2020b; Xie et al., 2020) to
preprocess the datasets. The numeric ratings or the presence of a review are
treated as positive instances while others as negative instances. We only keep
the ‘5-core’ datasets, i.e. every user bought in total at least 5 items and
vice versa each item was bought by at least 5 users. Table 1 shows the
statistics of the datasets after preprocessing.

Table 1. Dataset information. Dataset | # Users | # Items | # Actions | Avg. length | Sparsity
---|---|---|---|---|---
Beauty | 22,363 | 12,101 | 198,502 | 8.9 | 99.73%
Sports | 35,598 | 18,357 | 296,337 | 8.3 | 99.95%
Yelp | 30,431 | 20,033 | 316,354 | 10.3 | 99.95%

####  5.1.2. Evaluation Metrics

We follow (Kang and McAuley, 2018; Sun et al., 2019) to evaluate the model as
next-item prediction. We rank the prediction on whole item set without
negative sampling, which otherwise leads biased discoveries (Krichene and
Rendle, 2020). Performance is evaluated on top-$k$ ranking metrics including
Hit Ratio$@k$ ($\mathrm{HR}@k$), and Normalized Discounted Cumulative Gain$@k$
($\mathrm{NDCG}@k$). We report $\mathrm{HR}$ and $\mathrm{NDCG}$ with
$k\in\\{5,10,20\\}$.

####  5.1.3. Baseline Methods

We include three groups of baseline methods for comparison.

  * •

Non-Sequential models: PopRec is based on the popularity of items; BPR-MF
(Rendle et al., 2012) is a matrix factorization model with a pairwise Bayesian
Personalized Ranking loss.

  * •

Transformer-based SR models: SASRec (Kang and McAuley, 2018) is one of the
state-of-the-art Transformer-based baselines for SR, Bert4Rec (Sun et al.,
2019) extends SASRec with bidirectional self-attention modules. S3-Rec (Zhou
et al., 2020b) uses self-supervised learning to capture relationships among
item and associated attributes. We remove its modules for mining on attributes
as we have no attributes for items, namely S${}^{3}\text{-Rec}_{m}$. CL4SRec
(Xie et al., 2020) fuses contrastive SSL with Transformer-based SR model. It
only has random augmentation methods for SSL.

  * •

Other SR models: GRU4Rec (Hidasi et al., 2015) is an RNN-based approach, and
Caser (Tang and Wang, 2018) is a CNN-based model.

####  5.1.4. Implementation Details

Table 2. Performance comparisons of different methods. The best score is bolded in each row, and the second best is underlined. The last two columns are the relative improvements compared with SASRec and the best baseline results. Dataset | Metric | PopRec | BPR | GRU4Rec | Caser | SASRec | BERT4Rec | S${}^{3}\text{-Rec}_{m}$ | CL4SRec | CoSeRec | Improv. v.s.
---|---|---|---|---|---|---|---|---|---|---|---
SASRec | All
Beauty | HR@5 | 0.0080 | 0.0212 | 0.0111 | 0.0251 | 0.0374 | 0.0351 | 0.0189 | 0.0401 | 0.0537 | 43.58% | 33.92%
HR@10 | 0.0152 | 0.0372 | 0.0162 | 0.0342 | 0.0575 | 0.0601 | 0.0307 | 0.0642 | 0.0752 | 30.78% | 17.13%
HR@20 | 0.0217 | 0.0589 | 0.0478 | 0.0643 | 0.0901 | 0.0942 | 0.0487 | 0.0974 | 0.1041 | 15.54% | 6.88%
NDCG@5 | 0.0044 | 0.0130 | 0.0058 | 0.0145 | 0.0241 | 0.0219 | 0.0115 | 0.0268 | 0.0361 | 49.79% | 34.7%
NDCG@10 | 0.0068 | 0.0181 | 0.0075 | 0.0226 | 0.0305 | 0.0300 | 0.0153 | 0.0345 | 0.0430 | 40.98% | 24.64%
NDCG@20 | 0.0084 | 0.0236 | 0.0104 | 0.0298 | 0.0387 | 0.0386 | 0.0198 | 0.0428 | 0.0503 | 29.97% | 17.52%
Sports | HR@5 | 0.0056 | 0.0141 | 0.0162 | 0.0154 | 0.0206 | 0.0217 | 0.0121 | 0.0231 | 0.0287 | 39.32% | 24.24%
HR@10 | 0.0094 | 0.0216 | 0.0258 | 0.0261 | 0.0320 | 0.0359 | 0.0205 | 0.0369 | 0.0437 | 36.56% | 18.43%
HR@20 | 0.0192 | 0.0323 | 0.0421 | 0.0399 | 0.0497 | 0.0604 | 0.0344 | 0.0557 | 0.0635 | 27.77% | 8.60%
NDCG@5 | 0.0041 | 0.0091 | 0.0103 | 0.0114 | 0.0135 | 0.0143 | 0.0084 | 0.0146 | 0.0196 | 45.19% | 34.25%
NDCG@10 | 0.0053 | 0.0115 | 0.0142 | 0.0135 | 0.0172 | 0.0190 | 0.0111 | 0.0191 | 0.0242 | 40.7% | 26.7%
NDCG@20 | 0.0078 | 0.0142 | 0.0186 | 0.0178 | 0.0216 | 0.0251 | 0.0146 | 0.0238 | 0.0292 | 35.19% | 15.14%
Yelp | HR@5 | 0.0057 | 0.0127 | 0.0152 | 0.0142 | 0.0160 | 0.0196 | 0.0101 | 0.0227 | 0.0241 | 50.63% | 6.17%
HR@10 | 0.0099 | 0.0216 | 0.0248 | 0.0254 | 0.0260 | 0.0339 | 0.0176 | 0.0384 | 0.0395 | 51.92% | 2.86%
HR@20 | 0.0164 | 0.0346 | 0.0371 | 0.0406 | 0.0443 | 0.0564 | 0.0314 | 0.0623 | 0.0649 | 46.5% | 4.17%
NDCG@5 | 0.0037 | 0.0082 | 0.0091 | 0.008 | 0.0101 | 0.0121 | 0.0068 | 0.0143 | 0.0151 | 49.5% | 5.59%
NDCG@10 | 0.0051 | 0.0111 | 0.0124 | 0.0113 | 0.0133 | 0.0167 | 0.0092 | 0.0194 | 0.0205 | 54.14% | 5.67%
NDCG@20 | 0.0067 | 0.0143 | 0.0145 | 0.0156 | 0.0179 | 0.0223 | 0.0127 | 0.0254 | 0.0263 | 46.93% | 3.54%

We use code provided by the authors for
S3-Rec222https://github.com/RUCAIBox/CIKM2020-S3Rec,
Bert4Rec333https://github.com/FeiSun/BERT4Rec and
Caser444https://github.com/graytowne/caser_pytorch methods. We implement
GRU4Rec555https://github.com/slientGe/Sequential_Recommendation_Tensorflow and
BPRMF666https://github.com/xiangwang1223/neural_graph_collaborative_filtering
methods based on public resources. We tune the mask proportion of BERT4Recc
with $\\{0.2,0.4,0.6,0.8\\}$. We set all other hyper-parameters of each method
as reported in the papers.

We implement our method in PyTorch. For common hyparparameters, we follow (Xie
et al., 2020) to set the number of the self-attention blocks and attention
heads as $2$, the embedding dimension as 64, and the maximum sequence length
as $50$. We tune $\alpha$, $\beta$, $K$, $E$, and $\lambda$ within the ranges
of $[0.1,0.9]$, $[0.1,0.9]$, $\\{4,12,20\\}$, $\\{0,20,40,60,80,100\\}$, and
$\\{0.1,0.2,0.3,0.4,0.5\\}$ respectively. For hyperparameters in random
augmentation operators $\eta$, $\mu$, and $\omega$, we set the optimal values
reported by (Xie et al., 2020). We adopt early stopping on the validation set
if the performance does not improve for 40 epochs, and report results on test
set. The model is optimized by Adam optimizer (Kingma and Ba, 2014) with a
learning rate of 0.001, $\beta_{1}=0.9$, $\beta_{2}=0.999$, and batch size of
256.

###  5.2. Overall Performance Comparison (RQ1)

Table 2 shows the performance of all methods on three datasets. We make the
follow observations:

  * •

Overall Comparison.  (1) Non-sequential models perform worse than sequential
recommendation methods. This suggests the importance of mining sequential
patterns for next-item prediction. For sequential models, Transformer-based
methods achieve better performance than other type of SR models, which
indicates that self-attention mechanisms are more effective at capturing
sequential patterns than CNN and RNNs. (2) S${}^{3}\mathit{Rec}_{m}$, though
leveraging self-supervision signals by masking items in sequences, performs
worse than SASRec. We hypothesize the poor performance results from two
reasons: the two-stage training preventing information sharing between SSL and
next-item prediction targets, and the weak supervision signals without
informative augmentations. (3) CL4SRec consistently performs better than other
baselines, which verifies the efficacy of employing contrastive SSL in
sequential recommendation. However, it still performs worse than CoSeRec,
since it has no informative augmentations. (4) The proposed CoSeRec
consistently outperforms other models on all datasets in all evaluation
metrics. The improvements comparing with the best baseline range from 3.54% to
30.88% in terms of $\mathrm{HR}$ and $\mathrm{NDCG}$. Different from
baselines, our proposed method leverages contrastive SSL signals with
informative augmentation methods, incorporating length skewness of sequences,
and optimize the model with multi-task strategy.

  * •

Comparison with SASRec and CL4SRec. (1) We observe that compared with SASRec,
the performance of CoSeRec is significantly better, ranging from $14.54\%$ to
$54.14\%$ in $\mathrm{HR}$ and $\mathrm{NDCG}$ on three datasets. Since both
CoSeRec and SASRec adopt the same sequence encoder, those results demonstrate
the necessity of leveraging the constrastive SSL signals for SR. (2) Compared
with CL4SRec which employs random augmentations for contrastive SSL task,
CoSeRec acheives 22.47%, 23.39% and 4.67% average relative improvement on
Beauty, Sports, and Yelp datasets respectively. The performance gains result
from the informative augmentation, which is first proposed in this work. (3)
We also observe larger average improvements on Sports and Beauty compared with
Yelp. We hypothesis the difference is because the average sequence length of
Sports and Beauty is shorter than that of Yelp, thus suffering severer ‘cold-
start’ issues. Therefore, SSL contributes more benefits to the sequence
encoding on the Sports and Beauty datasets.

###  5.3. Augmentation Analysis (RQ2)

In this paper, we include five augmentation operators for the contrastive SSL
task. To study impacts of these augmentation methods and find the optimal
choice, we conduct three groups of ablation studies on the Sports and Beauty
datasets. The first is a ‘leave-one-out’ setting in which only one
augmentation operator is removed from the augmentation set
$\\{\mathsf{M},\mathsf{C},\mathsf{R},\mathsf{S},\mathsf{I}\\}$ at a time. The
second is ‘pair-wise’ where we only apply two augmentations to create two
types of views for contrastive SSL tasks in each model. Lastly, we investigate
the optimal augmentations for short sequences.

####  5.3.1. Leave-One-Out Comparison

Figure 3. Performance comparison (in NDCG@5) w.r.t. different augmentation
sets on Sports and Beauty datasets. ‘w/o’ indicates one type is removed from
the full augmentation set
$\\{\mathsf{M},\mathsf{C},\mathsf{R},\mathsf{S},\mathsf{I}\\}$. CoSeRec adopts
the full set.

Figure 3 shows the comparison results under the ‘leave-one-out’ setting. We
included CL4SRec for comparison since it utilizes random augmentations
$\\{\mathsf{M},\mathsf{C},\mathsf{R}\\}$. We observe that without the ‘Insert’
or ‘Substitute’ operator, the performance consistently decreases on both
datasets. Without random operators such as ‘Crop’ or ‘Reorder’, performance
even significantly increases on the Beauty dataset. This phenomenon implies
that existing random augmentations, such as randomly cropping or reordering a
sequence without considering item correlations, demolish item relationships in
sequences, which leads to less confident positive pairs and thus impairs the
efficacy of the contrastive SSL learning.

####  5.3.2. Pair-Wise Comparison

Figure 4. Performance comparison (in $\mathrm{NDCG}@5$) w.r.t. different
augmentation pairs on Sports and Beauty datasets.

Figure 4 shows the comparison results on the Sports and Beauty datasets under
the ‘pair-wise’ setting. We observe that: (1) For pair of the same operator
(diagonal values), the best performance is $\mathsf{I}$ and $\mathsf{S}$ on
Sports and Beauty dataset, respectively. This demonstrates our proposed
informative operators are better than those random augmentations for
contrastive SSL. (2) A pair of different operators performs better than two
which are the same in general. For example in Beauty,
$(\mathsf{I},\mathsf{S})$ pair achieves 0.0307 while $(\mathsf{I},\mathsf{I})$
and $(\mathsf{S},\mathsf{S})$ pairs achieves 0.0255 and 0.0262 in
$\mathrm{NDCG}@5$ respectively. Comparing with Figure 3(b), we observe that
having multiple augmentation operators perform better than one or two
augmentation operators. For example, with
$\\{\mathsf{M},\mathsf{R},\mathsf{S},\mathsf{I}\\}$ augmentation set, the
model achieves 0.0334 in $\mathrm{NDCG}@5$ on Beauty which is higher than all
pair-wise based model results. These observations imply that having multiple
views for contrastive SSL is beneficial as it allows the model to capture more
mutual information from different views. (3) ‘$\mathsf{S}$’ performs well on
Beauty but not on Sports under the pair-wise setting. However, in Figure 3(a),
model without ‘S’ augmentation has the largest performance drop compared to
others. This indicates that the ‘S’ operator is more suitable for providing a
complementary view for contrastive SSL learning.

Figure 5. Performance comparison w.r.t. different augmentation sets for short
sequences on Sports and Beauty datasets.

####  5.3.3. Augmentation Set for Short Sequences

As described in Sec. 4.1.4, short sequences are more sensitive to random
augmentations, for which we only augmented by ‘$\mathsf{M}$’, ‘$\mathsf{S}$’,
and ‘$\mathsf{I}$’ augmentation operators. In this section, we explore the
optimal augmentation combinations for short sequences to verify our claims.

Figure 5 shows the performance with different combinations of augmentations
for short sequences. For example, ‘$\\{\mathsf{S},\mathsf{I},\mathsf{M}\\}$’
denotes adopting all ‘$\mathsf{S}$’, ‘$\mathsf{I}$’, and ‘$\mathsf{M}$’
augmentations for short sequences. We observe that all combinations which
include informative augmentations outperform CL4SRec, which demonstrates the
necessity of using informative operators for short sequences. With the
additional ‘$\mathsf{M}$’ augmentation, the performance improves most. This is
because ‘$\mathsf{M}$’ operator shares the same information as the next-item
prediction target, and it encourages the encoder to capture high-order item
relationships. Additionally, the model with
‘$\\{\mathsf{S},\mathsf{I},\mathsf{M}\\}$’ operators outperforms
‘$\\{\mathsf{S},\mathsf{I},\mathsf{M},\mathsf{R},\mathsf{C}\\}$’ indicating
that applying different augmentation operators to short and long sequences is
meaningful as short sequences can be more sensitive to randomness.

###  5.4. Robustness Analysis (RQ3)

Figure 6. Performance comparison w.r.t. sparsity ratio.

Recommender systems usually suffer data-sparsity issues where there are
limited historical records. To simulate this scenario, we train a model with
only partial training data (25%, 50%, 75%, and 100%) and keep the test data
unchanged. We compare the proposed method with the best baseline (CL4SRec) on
the Sports and Beauty datasets, which are presented in Figures 6 (a) and (b),
respectively. We observe that performance substantially degrades when less
training data is used, but CoSeRec consistently performs better than CL4SRec
and the performance degradation is slower than CL4SRec. For example, on
Sports, CoSeRec achieves similar performance with only 75% of training data as
of CL4SRec with full training data. Moreover, CL4SRec drops 76.19% of its
original performance while CoSeRec only drop 34%, when both with 50% training
data. These observations demonstrate that informative augmentations in CoSeRec
can the alleviate data-sparsity issue. We also observe that the impact of data
sparsity varies on different datasets. With 50% training data, CoSeRec’s
performance drops 34% on Sports while dropping 52.14% on Beauty.

Figure 7. Model performance comparisons w.r.t. noise ratio.

We also evaluate the robustness of CoSeRec to noisy interactions in the
inference phrase. Specifically, we train a model with the original training
data, and randomly add a certain proportion (10%, 20%, 30%, 40%, and 50%) of
negative user-item interactions to every test sequence. Figures 6 (c)-(d) show
the results on the Sports and Beauty datasets. We see that the performance of
both CoSeRec and CL4SRec decreases. CoSeRec can consistently perform better
than CL4SRec under any noise ratio. This implies that with additional high
quality augmentation operators (‘Insert‘ and ‘Substitute‘), CoSeRec creates
higher confident positive views for the contrastive SSL objective to maximize
agreements, which thus endows the encoder with more robustness against the
noisy interactions during the inference stage.

###  5.5. Study of CoSeRec (RQ4)

Figure 8. Performance comparison of CoSeRec w.r.t. different $\alpha$,
$\beta$, $K$, and $\lambda$ on Sports and Beauty datasets.

In this section, we first explore the effects of hyperparamters $\alpha$,
$\beta$, $K$, and $\lambda$ in CoSeRec, which control the ‘Substitute’ ratio,
‘Insert’ ratio, threshold of deciding whether a sequence to be short, and the
intensity of contrastive SSL, respectively. Then we compare different item
correlations. Lastly, we investigate an alternative two-stages optimization
strategy, which first pre-trains from SSL and then fine-tunes on next-item
prediction targets.

####  5.5.1. Hyperparameter Study.

We study the hyperparameters one by one meaning that all other hyperparameters
are assigned the optimal tuned values when studying a certain hyperparameter.
Figure 8 shows overall comparisons.

We observe that: (1) A general pattern for $\alpha$ and $\beta$ is that the
performance reaches to a peak as the ratios increasing and then start to
deteriorate. The reason is that there is no informative augmentation when the
ratio is $0$ and increasing the ratios too high leads heavier corruptions,
thus constructing false positive samples to impair the SSL. Specifically,
$\beta=0.4$ performs the best while $\alpha=0.1$ performs the best in the
Sports and Beauty datasets, respectively. The larger $\beta$ indicates that
due to the long-tail distribution of short sequences, the dataset requires
more ‘Insert’ operator to alleviate the cold-start problem. (2) The best
values of $K$ are $4$ and $12$ on Sports and Beauty, respectively. We
hypothesis that the difference between the optimal $K$ of two datasets is
because the length skewness is different. (3) We can see that having the
contrastive SSL signal ($\lambda>0$) is important as it significantly improves
the performance 32.88% and 12.82% in $\mathrm{NDCG}@5$ on the Sports and
Beauty datasets. We also observe that performance starts to deteriorate when
$\lambda>0.1$. This indicates that the contrastive SSL signal complements but
should not dominate the learning goal.

####  5.5.2. Effect of Item Correlations

We study the effect of different correlations introduced in Section 4.1.3
including memory-based, model-based and hybrid correlations. Table 3 shows
comparison results. We can see that using any type of item correlations helps
improving performance compared with CL4SRec. Among these correlations, model-
based correlation performs worse than memory-based and hybrid correlations.
This might be because item representations learned from the model at early
epochs are not informative. Hybrid correlations performs best with $E\geq
160$. This indicates that it is beneficial to incorporate both model-based and
memory-based correlations.

Table 3. Performance comparison w.r.t. different item correlations for informative augmentations. Correlation Type | Sports | Beauty
---|---|---
HR@5 | NDCG@5 | HR@5 | NDCG@5
None (CL4SRec) | 0.0231 | 0.0146 | 0.0401 | 0.0268
Memory-based | 0.0284 | 0.0194 | 0.0528 | 0.0357
Model-based | 0.0251 | 0.0164 | 0.0497 | 0.0338
Hybrid ($E=80$) | 0.0270 | 0.0179 | 0.0503 | 0.0342
Hybrid ($E=160$) | 0.0287 | 0.0196 | 0.0537 | 0.0361
Hybrid ($E=200$) | 0.0290 | 0.0194 | 0.0530 | 0.0357

####  5.5.3. Effect of pre-training

We have demonstrated that jointly optimizing the recommendation objective with
the contrastive learning objective leads to an effective training of CoSeRec.
However, S3-Rec claims that a two-stage training, i.e. pre-training with
contrastive SSL and fine-tuning with next-item prediction, is also effective.
This two-stage training is also widely adopted in other areas (Chen et al.,
2020a; Devlin et al., 2018). Therefore, we conduct experiments to compare the
performance of our multi-task training strategy and an alternative two-stage
one. Table 4 shows the result comparisons between multi-task training and two-
stage training on the Sports and Beauty datasets. We observe that a two-stage
training performs worse compared with the multi-task strategy. The better
performance of the multi-task strategy of CoSeRec reflects that contrastive
learning objectives and recommendation objectives can benefit from each other
when jointly training, while two stage training can lead to the pre-trained
self-supervision information being forgotten in the fine-tuning stage.

Table 4. Performance comparison under multi-task and two-stage training strategies on Sports and Beauty datasets. Strategy | Sports | Beauty
---|---|---
HR@5 | NDCG@5 | HR@5 | NDCG@5
Multi-task | 0.0287 | 0.0196 | 0.0537 | 0.0361
Two-stage | 0.0242 | 0.0388 | 0.0435 | 0.0283

##  6\. Conclusion and Future Work

In this work, we studied the contrastive SSL in sequential recommendation,
which alleviate the data-sparsity and noisy interaction issues We proposed a
novel learning framework, CoSeRec, which jointly optimizes the contrastive SSL
objective and the next-item prediction objective. We proposed two novel
informative augmentation methods, i.e. substitute and insert, which advances
existing random augmentations by leveraging item correlations. Additionally,
we endowed augmentations with sequence length awareness, which addresses the
length skewness in sequential dataset. We conducted extensive experiments on
three benchmark datasets and justified the effectiveness and the robustness of
CoSeRec. Moreover, we investigated the optimal augmentation for sequential
recommendation, which verifies the efficacy of informative augmentations.

In the future, we plan to investigate more advanced model-based item
correlation functions. For example, searching for correlated items under a
reinforcement learning framework so that more accurate correlation information
can be leveraged. Additionally, we will explore finer-grained exploration of
the relation between augmentations and the sequence length. Moreover, we plan
to study the effects of different contrastive learning functions in the self-
supervised learning of sequences.

## References

  * (1)
  * Breese et al. (2013) John S Breese, David Heckerman, and Carl Kadie. 2013\.  Empirical analysis of predictive algorithms for collaborative filtering.  _arXiv preprint arXiv:1301.7363_ (2013).
  * Chen et al. (2020a) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020a.  A simple framework for contrastive learning of visual representations. In _International conference on machine learning_. PMLR, 1597–1607.
  * Chen et al. (2020b) Tong Chen, Hongzhi Yin, Guanhua Ye, Zi Huang, Yang Wang, and Meng Wang. 2020b.  Try this instead: Personalized and interpretable substitute recommendation. In _Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval_. 891–900.
  * Chen et al. (2018) Xu Chen, Hongteng Xu, Yongfeng Zhang, Jiaxi Tang, Yixin Cao, Zheng Qin, and Hongyuan Zha. 2018.  Sequential recommendation with user memory networks. In _WSDM_. 108–116.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Fan et al. (2021a) Ziwei Fan, Zhiwei Liu, Jiawei Zhang, Yun Xiong, Lei Zheng, and Philip S. Yu. 2021a.  Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer. In _Proceedings of the 30th ACM International Conference on Information and Knowledge Management_. ACM.
  * Fan et al. (2021b) Ziwei Fan, Zhiwei Liu, Lei Zheng, Shen Wang, and Philip S. Yu. 2021b.  Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation. In _Proceedings of the 30th ACM International Conference on Information and Knowledge Management_. ACM.
  * Fang et al. (2020) Hongchao Fang, Sicheng Wang, Meng Zhou, Jiayuan Ding, and Pengtao Xie. 2020.  Cert: Contrastive self-supervised learning for language understanding.  _arXiv preprint arXiv:2005.12766_ (2020).
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple Contrastive Learning of Sentence Embeddings.  _arXiv preprint arXiv:2104.08821_ (2021).
  * Giorgi et al. (2020) John M Giorgi, Osvald Nitski, Gary D Bader, and Bo Wang. 2020\.  Declutr: Deep contrastive learning for unsupervised textual representations.  _arXiv preprint arXiv:2006.03659_ (2020).
  * Gutmann and Hyvärinen (2010) Michael Gutmann and Aapo Hyvärinen. 2010.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In _Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics_. JMLR Workshop and Conference Proceedings, 297–304.
  * He et al. (2020) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020.  Momentum contrast for unsupervised visual representation learning. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9729–9738.
  * He and McAuley (2016) Ruining He and Julian McAuley. 2016.  Fusing similarity models with markov chains for sparse sequential recommendation. In _2016 IEEE 16th International Conference on Data Mining (ICDM)_. IEEE, 191–200.
  * Hidasi et al. (2015) Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015.  Session-based recommendations with recurrent neural networks.  _arXiv preprint arXiv:1511.06939_ (2015).
  * Jiao et al. (2020) Yizhu Jiao, Yun Xiong, Jiawei Zhang, Yao Zhang, Tianqi Zhang, and Yangyong Zhu. 2020\.  Sub-graph Contrast for Scalable Self-Supervised Graph Representation Learning.  _arXiv preprint arXiv:2009.10273_ (2020).
  * Kang and McAuley (2018) Wang-Cheng Kang and Julian McAuley. 2018.  Self-attentive sequential recommendation. In _ICDM_. IEEE, 197–206.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization.  _arXiv preprint arXiv:1412.6980_ (2014).
  * Krichene and Rendle (2020) Walid Krichene and Steffen Rendle. 2020.  On Sampled Metrics for Item Recommendation. In _SIGKDD_. 1748–1757.
  * Lan et al. (2019) Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019\.  Albert: A lite bert for self-supervised learning of language representations.  _arXiv preprint arXiv:1909.11942_ (2019).
  * Li et al. (2020) Jiacheng Li, Yujie Wang, and Julian McAuley. 2020.  Time Interval Aware Self-Attention for Sequential Recommendation. In _WSDM_. 322–330.
  * Liu et al. (2021) Zhiwei Liu, Ziwei Fan, Yu Wang, and Philip S. Yu. 2021\.  Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer.  _Proceedings of the 44th international ACM SIGIR conference on Research and development in information retrieval_.
  * Liu et al. (2020) Zhiwei Liu, Mengting Wan, Stephen Guo, Kannan Achan, and Philip S Yu. 2020.  Basconv: aggregating heterogeneous interactions for basket recommendation with graph convolutional neural network. In _Proceedings of the 2020 SIAM International Conference on Data Mining_. SIAM, 64–72.
  * Liu et al. (2019) Zhiwei Liu, Lei Zheng, Jiawei Zhang, Jiayu Han, and S Yu Philip. 2019.  JSCN: Joint spectral convolutional network for cross domain recommendation. In _2019 IEEE International Conference on Big Data (Big Data)_. IEEE, 850–859.
  * Ma et al. (2020) Jianxin Ma, Chang Zhou, Hongxia Yang, Peng Cui, Xin Wang, and Wenwu Zhu. 2020\.  Disentangled Self-Supervision in Sequential Recommenders. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 483–491.
  * McAuley et al. (2015) Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015.  Image-based recommendations on styles and substitutes. In _SIGIR_. 43–52.
  * Misra and Maaten (2020) Ishan Misra and Laurens van der Maaten. 2020.  Self-supervised learning of pretext-invariant representations. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 6707–6717.
  * Qiu et al. (2020) Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. 2020\.  Gcc: Graph contrastive coding for graph neural network pre-training. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 1150–1160.
  * Quadrana et al. (2017) Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, and Paolo Cremonesi. 2017\.  Personalizing session-based recommendations with hierarchical recurrent neural networks. In _RecSys_. 130–137.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian Personalized Ranking from Implicit Feedback. In _Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence_ (Montreal, Quebec, Canada) _(UAI ’09)_. AUAI Press, Arlington, Virginia, USA, 452–461.
  * Rendle et al. (2012) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2012.  BPR: Bayesian personalized ranking from implicit feedback.  _arXiv preprint arXiv:1205.2618_ (2012).
  * Rendle et al. (2010) Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010\.  Factorizing personalized markov chains for next-basket recommendation. In _WWW_. 811–820.
  * Sun et al. (2019) Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019.  BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In _CIKM_. 1441–1450.
  * Tang and Wang (2018) Jiaxi Tang and Ke Wang. 2018.  Personalized top-n sequential recommendation via convolutional sequence embedding. In _WSDM_. 565–573.
  * Tian et al. (2020) Yonglong Tian, Chen Sun, Ben Poole, Dilip Krishnan, Cordelia Schmid, and Phillip Isola. 2020\.  What makes for good views for contrastive learning.  _arXiv preprint arXiv:2005.10243_ (2020).
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In _NIPS_. 5998–6008.
  * Veličković et al. (2018) Petar Veličković, William Fedus, William L Hamilton, Pietro Liò, Yoshua Bengio, and R Devon Hjelm. 2018.  Deep graph infomax.  _arXiv preprint arXiv:1809.10341_ (2018).
  * Wang et al. (2021) Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021.  Denoising implicit feedback for recommendation. In _Proceedings of the 14th ACM International Conference on Web Search and Data Mining_. 373–381.
  * Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019.  Neural Graph Collaborative Filtering. In _SIGIR_. 165–174.
  * Wei and Zou (2019) Jason Wei and Kai Zou. 2019\.  Eda: Easy data augmentation techniques for boosting performance on text classification tasks.  _arXiv preprint arXiv:1901.11196_ (2019).
  * Wu et al. (2017) Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J Smola, and How Jing. 2017.  Recurrent recommender networks. In _WSDM_. 495–503.
  * Wu et al. (2020a) Jibang Wu, Renqin Cai, and Hongning Wang. 2020a.  Déjà vu: A Contextualized Temporal Attention Mechanism for Sequential Recommendation. In _The Web Conference_. 2199–2209.
  * Wu et al. (2020c) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2020c.  Self-supervised Graph Learning for Recommendation.  _arXiv preprint arXiv:2010.10783_ (2020).
  * Wu et al. (2020b) Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharpnack. 2020b.  SSE-PT: Sequential Recommendation Via Personalized Transformer. In _RecSys_. ACM, 328–337.
  * Wu et al. (2020d) Zhuofeng Wu, Sinong Wang, Jiatao Gu, Madian Khabsa, Fei Sun, and Hao Ma. 2020d.  CLEAR: Contrastive Learning for Sentence Representation.  _arXiv preprint arXiv:2012.15466_ (2020).
  * Xiao et al. (2020) Tete Xiao, Xiaolong Wang, Alexei A Efros, and Trevor Darrell. 2020\.  What should not be contrastive in contrastive learning.  _arXiv preprint arXiv:2008.05659_ (2020).
  * Xie et al. (2020) Xu Xie, Fei Sun, Zhaoyang Liu, Jinyang Gao, Bolin Ding, and Bin Cui. 2020\.  Contrastive Pre-training for Sequential Recommendation.  _arXiv preprint arXiv:2010.14395_ (2020).
  * Yao et al. (2020) Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix Yu, Aditya Menon, Lichan Hong, Ed H Chi, Steve Tjoa, Evan Ettinger, et al. 2020\.  Self-supervised Learning for Deep Models in Recommendations.  _arXiv preprint arXiv:2007.12865_ (2020).
  * Ye et al. (2020) Wenwen Ye, Shuaiqiang Wang, Xu Chen, Xuepeng Wang, Zheng Qin, and Dawei Yin. 2020\.  Time Matters: Sequential Recommendation with Complex Temporal Information. In _SIGIR_. 1459–1468.
  * You et al. (2020) Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020\.  Graph contrastive learning with augmentations.  _Advances in Neural Information Processing Systems_ 33 (2020).
  * Yu et al. (2016) Feng Yu, Qiang Liu, Shu Wu, Liang Wang, and Tieniu Tan. 2016.  A dynamic recurrent model for next basket recommendation. In _SIGIR_. 729–732.
  * Zhou et al. (2020a) Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, and Hongxia Yang. 2020a.  Contrastive Learning for Debiased Candidate Generation at Scale.  _arXiv preprint arXiv:2005.12964_ (2020).
  * Zhou et al. (2020b) Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020b.  S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 1893–1902.
  * Zhu et al. (2020) Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2020.  Graph Contrastive Learning with Adaptive Augmentation.  _arXiv preprint arXiv:2010.14945_ (2020).