# Self-Guided Learning to Denoise for Robust Recommendation

Yunjun Gao, Yuntao Du, Yujia Hu,
Lu Chen, Xinjun Zhu, Ziquan Fang  College of Computer Science, Zhejiang
University, China gaoyj, ytdu, yjhu, luchen, xjzhu<EMAIL_ADDRESS>and
Baihua Zheng  School of Computing and Information Systems,
Singapore Management University, Singapore<EMAIL_ADDRESS>
(2022)

###### Abstract.

The ubiquity of implicit feedback makes them the default choice to build
modern recommender systems. Generally speaking, observed interactions are
considered as positive samples, while unobserved interactions are considered
as negative ones. However, implicit feedback is inherently noisy because of
the ubiquitous presence of noisy-positive and noisy-negative interactions.
Recently, some studies have noticed the importance of denoising implicit
feedback for recommendations, and enhanced the robustness of recommendation
models to some extent. Nonetheless, they typically fail to (1) capture the
hard yet clean interactions for learning comprehensive user preference, and
(2) provide a universal denoising solution that can be applied to various
kinds of recommendation models.

In this paper, we thoroughly investigate the memorization effect of
recommendation models, and propose a new denoising paradigm, _i.e.,_ Self-
Guided Denoising Learning (SGDL), which is able to collect memorized
interactions at the early stage of the training (_i.e.,_ “noise-resistant”
period), and leverage those data as denoising signals to guide the following
training (_i.e.,_ “noise-sensitive” period) of the model in a meta-learning
manner. Besides, our method can automatically switch its learning phase at the
memorization point from memorization to self-guided learning, and select clean
and informative memorized data via a novel adaptive denoising scheduler to
improve the robustness. We incorporate SGDL with four representative
recommendation models (_i.e.,_ NeuMF, CDAE, NGCF and LightGCN) and different
loss functions (_i.e.,_ binary cross-entropy and BPR loss). The experimental
results on three benchmark datasets demonstrate the effectiveness of SGDL over
the state-of-the-art denoising methods like T-CE, IR, DeCA, and even state-of-
the-art robust graph-based methods like SGCN and SGL.

Recommender System; Denoising Recommendation; Implicit Feedback; Robust
Learning

††journalyear: 2022††copyright: acmcopyright††conference: Proceedings of the
45th International ACM SIGIR Conference on Research and Development in
Information Retrieval; July 11–15, 2022; Madrid, Spain††booktitle: Proceedings
of the 45th International ACM SIGIR Conference on Research and Development in
Information Retrieval (SIGIR ’22), July 11–15, 2022, Madrid, Spain††price:
15.00††doi: 10.1145/3477495.3532059††isbn: 978-1-4503-8732-3/22/07††ccs:
Information systems Recommender systems††ccs: Computing methodologies Learning
from implicit feedback

##  1\. Introduction

Figure 1. Key idea of SGDL: (a) and (c) show the memory rate when training
NeuMF and LightGCN on the MovieLens dataset, respetively; (b) and (d) show the
memory rate when training the corresponding model with SGDL. The memory rate
is the proportion of memorized data (see Section 3.1.1 for details) in the
clean and noisy interactions. NR denotes “noise-resistant” period and NS
represents “noise-sensitive” period, respectively.

Recommender systems have been widely deployed to perform personalized
information filtering, especially for various online services such as
E-commerce (Lin et al., 2019; Gharibshah et al., 2020), social media (Ren et
al., 2017), and news portals (Wang et al., 2018). Most of existing recommender
systems use implicit feedback (_e.g.,_ view and click) to develop machine
learning models, due to its large volume and availability (Rendle et al.,
2009; He et al., 2016; Du et al., 2022a, b). Specifically, the observed
interactions between users and items are viewed as positive instances, while
unobserved interactions are viewed as negative instances. However, implicit
feedback is inherently noisy because of the ubiquitous presence of noisy-
positive and noisy-negative interactions (Hu et al., 2008; Bian et al., 2021;
Wang et al., 2021b; Lee et al., 2021). Take the E-commerce scenario as an
example. A large portion of click behaviors are triggered by the curiosity of
users, which cannot directly indicate the users’ positive views of the
products. On the other hand, unobserved interactions may attribute to the
unawareness of users because the items are simply not exposed to them. Hence,
blindly fitting the implicit feedback to recommender systems without
considering the inherent noise would fail to understand users’ true
preferences, and eventually harm user experiences and degrade recommendation
performance (Xie et al., 2020; Wang et al., 2021a).

Considering the widespread use of implicit feedback and its large impact on
the recommendation model, some recent studies have noticed the importance of
denoising implicit feedback for recommendations. Existing efforts on tackling
this problem can be roughly divided into two categories: _sample selection
methods_ (Gantner et al., 2012; Ding et al., 2019b; Yu and Qin, 2020; Wang et
al., 2021b) and _sample re-weighting methods_ (Wang et al., 2021a; Hu et al.,
2021; Wang et al., 2022). Sample selection methods focus on designing more
effective samplers to collect clean samples for learning users’ preferences,
while their performance suffers from high variance since they heavily depend
on the sampling distribution (Yuan et al., 2018). On the other hand, sample
re-weighting methods aim to distinguish noisy interactions from clean data in
terms of the loss values, and assign lower weights to noisy interactions with
high loss values during training. Their key idea is consistent with
memorization effect (Arpit et al., 2017): models tend to initially learn easy
and clean patterns (_i.e.,_ user preferences) in the early stage of their
learning, and eventually memorize all training interactions. Benefiting from
this principle, sample re-weighting methods are able to successfully identify
noisy interactions.

Although sample re-weighting methods can achieve promising performance for
denoising implicit feedback, we want to highlight that they commonly suffer
from following two problems:

  * •

Abandon of Hard Clean Interactions. These methods heavily rely on the loss
values. They simply assume that interactions with large loss values are noisy
and penalize them with small weights. However, it has been reported that some
clean interactions (_aka._ hard yet clean interactions) may also have high
loss values at the beginning of the training, and those interactions play an
important role in understanding users’ behaviors and preferences (Ding et al.,
2019a; Hu et al., 2021; Wang et al., 2022). Nonetheless, these interactions
are simply discarded by existing methods due to high loss values, incurring
insufficient understanding of users’ true preferences.

  * •

Lack of Adaptivity and Universality. Existing sample re-weighting methods are
able to achieve good performance _only_ when the reweighting configurations
(e.g., weights and thresholds) are properly specified. However, obtaining
proper configurations normally requires time-consuming procedures (_e.g.,_
grid search), and the optimal configurations for one dataset typically are not
applicable for other datasets because of the different data distributions.
Meanwhile, these methods are only applicable to the recommender models with
predefined pointwise loss function (_e.g.,_ cross-entropy loss in (Wang et
al., 2021a, 2022)), which makes it hard to incorporate other popular ranking
loss functions (_e.g.,_ BPR loss (Rendle et al., 2009)) and limits their
applications.

In this regard, we have thoroughly explored the memorization of the
representative recommendation models (_e.g.,_ NeuMF (He et al., 2017) and
LightGCN (He et al., 2020)) on implicit feedback, including the clean
interactions and the noisy interactions. By analyzing the learning processes
of different models, we have observed the existence of two learning periods.
As shown in Figure 1, a typical learning process consists of the “noise-
resistant” (NR) period and the “noise-sensitive” (NS) period. The former is
the duration when the memorization of noisy interactions is insignificant
because the models focus on memorizing easy and clean patterns at their early
stage of training; while the latter is the duration when the memorization of
noisy interactions rapidly increases since models eventually begin memorizing
all the implicit feedback at the late stage of training. The timestamp in the
training process that can best differentiate above two periods is defined as
memorization point of the model training, which are represented by dotted
lines in Figure 1.

These observations motivate us to design a new approach that better leverages
the memorization nature of recommendation models. Specifically, we propose a
new denoising paradigm, namely, Self-Guided Denoising Learning (SGDL), which
is able to collect memorized interactions at the early stage of training
(_i.e.,_ “noise-resistant” period), and leverage those data as denoising
signals to guide the subsequent training (_i.e.,_ “noise-sensitive” period) of
the model in a meta-learning manner. Besides, SGDL can automatically transit
from noise-resistant period to noise-sensitive period in order to stop
accumulating memorized interactions at the memorization point, since the noisy
interactions are gradually memorized with the training process. In a nutshell,
corresponding to the two observed periods commonly existing in the training
process of different models, SGDL contains two key phases, _i.e.,_
memorization and self-guided learning:

  * •

Memorization. Owing to the negligible memorization of noisy interactions
during “noise-resistant” period, the model is initially trained with all the
implicit feedback. To better reveal the underlying memorization nature of the
model during training, we design the new memorization-based metrics to define
the memorization states of data. As the memorized data are mostly easy and
clean interactions at the early stage of training, we collect them as
denoising signals to guide the following denoising training process. Moreover,
SGDL is able to automatically estimate the best memorization point, from which
the learning moves from the “noise-resistant” period to “noise-sensitive”
period, to stop accumulating memorized data without any supervision.

  * •

Self-Guided Learning. To avoid the memorization of noisy interactions in the
“noise-sensitive” period, we leverage the memorized data collected from the
“noise-resistant” period to represent user preferences. Specifically, a
denoising module is proposed to learn a parameterized weighting function for
the implicit feedback, which is guided by memorized data and updated with the
learning process of the model. Moreover, since some of the memorized data can
also be noisy, we further develop a novel adaptive denoising scheduler to
prevent the denoising module from being corrupted by noisy yet memorized
samples. Technically, the adaptive denoising scheduler characterizes the
contribution of each memorized data to the denoising performance, and decides
whether to use the sample by predicting its probability being sampled. The
scheduler is also simultaneously optimized together with the learning process
to enhance the robustness of the model.

Through the above two phases, SGDL can denoise implicit feedback with the help
of memorized data, which are naturally collected by exploiting the noise-
resistant period of training. Compared with standard training, SGDL can
dramatically reduce the memory ratio of noisy samples in the noise-sensitive
period, and enhance the robustness of recommendation models, as shown in
Figure 1. Moreover, since the model is constantly trained with all implicit
feedback data, our method can help the model learn users’ true preferences
with hard yet clean samples, leading to better recommendation performance.
Last but not the least, our method does not need any thresholds or predefined
weighting functions, and is easy to be applied to any learning-based
recommendation models. We conduct extensive experiments on three real-world
datasets with four representative recommendation models (_i.e.,_ NeuMF (He et
al., 2017), CDAE (Wu et al., 2016), NGCF (Wang et al., 2019), and LightGCN (He
et al., 2020)) and two loss functions (_i.e.,_ binary cross-entropy loss and
BPR loss (Rendle et al., 2009)). Experimental results show that our SGDL
significantly outperforms all state-of-the-art denoising methods, and it
achieves comparable (in many cases even better) performance to the state-of-
the-art robust graph-based methods like SGCN (Chen et al., 2021b) and SGL (Wu
et al., 2021).

In summary, we make three key contributions in this paper, as listed below.

  * •

We develop a new denoising paradigm, _i.e.,__self-guided denoising learning
(SGDL)_ , which leverages the self-labeled memorized data as guidance to offer
denoising signals for robust recommendation without defining any weighting
functions or requiring any auxiliary information.

  * •

We carefully exploit the memorization effect of recommendation models, and
design two training phases that can collect memorized data and utilize them as
guidance to denoise implicit feedback, respectively. Besides, a novel adaptive
denoising scheduler is introduced to further improve the robustness.

  * •

We incorporate SGDL with four representative recommendation models, and
conduct extensive experiments on three public benchmark datasets with various
state-of-the-art methods to demonstrate the superiority and universality of
SGDL.

##  2\. Problem Formulation

We first introduce the common paradigm of user preferences learning from
implicit feedback, and then formulate our task.

Preference learning from implicit feedback. In this paper, we focus on
learning the user preferences from the implicit feedback (Rendle et al.,
2009). Specifically, the behavior data (_e.g.,_ click and review)
$\mathcal{D}=\\{u,i,y_{ui}|u\in\mathcal{U},i\in\mathcal{I}\\}$ involves a set
of users $\mathcal{U}=\\{u\\}$ and a set of items $\mathcal{I}=\\{i\\}$, as
well as the interaction $y_{ui}=\\{0,1\\}$ that indicates whether user $u$ has
interacted with item $i$. Most of the state-of-the-art recommendation methods
(_e.g.,_ NeuMF (He et al., 2017) and LightGCN (He et al., 2020)) assume that
the interaction $y_{ui}$ could represent the users’ true preferences, and
directly learn the model $f$ with parameters $\theta$ by minimizing a ranking
loss function over $\mathcal{D}$.

Denoising implicit feedback for recommendations. However, due to the existence
of inherent noise in implicit feedback, recommendation models might fail to
learn the users’ true preferences with typical training process, resulting in
suboptimal performance. Thus, the task of the paper is, given the noisy
implicit feedback $\mathcal{D}$ that contains both noisy-positive and noisy-
negative feedback, to infer users’ true preferences with optimal model
parameters $\theta^{*}$.

Figure 2. The overall framework of SGDL. During Phase I (memorization),
memorized interactions are collected as denoising signals; During Phase II
(self-guided learning), the weighting function is simultaneously learned with
the recommender model, which is guided by the memorization data in a meta-
learning manner (see Section 3.2.1 for details).

##  3\. Methodology

In this section, we detail SGDL that comprises the following two phases: (i)
_memorization_ , which exploits the memorization effect of models to collect
memorized data during the noise-resistant period and estimates the best
memorization point to automatically transit to phase II; and (ii) _self-guided
learning_ , which leverages the memorized samples as denoise signals to guide
the model learning during the noise-sensitive period, and discards the
potential noisy interactions in memorized data with a novel adaptive denoising
scheduler for robustness. We detail the two phases as follows.

###  3.1. Phase I: Memorization

Initially, Phase I tries to train the recommendation model in a conventional
way during the noise-resistant period, where the memorization of noisy
interactions is assumed to be suppressed, as we discover in Figure 1. Since
most of the memorized interactions until the memorization point are clean, we
collect them to form the memorized data, which can be used as the denoising
signals to guide the training in Phase II. Therefore, the major challenges in
Phase I include 1) how to define the memorization of interactions and 2) how
to estimate the memorization point.

####  3.1.1. Memorized Interactions.

Previous studies (Wang et al., 2021a, 2022) mainly use loss values of training
data to demonstrate the memorization effect of recommendation models. However,
we argue that loss values are insufficient to reflect the learning process,
since they are inconsistent with the optimization target of recommendation
models (_i.e.,_ personalized ranking), and unable to distinguish hard
interactions from noisy ones. Thus, it is necessary to design a new
memorization-based metric, which takes the learning process of recommendation
models into consideration.

Inspired by the widely used hit ratio metric (Rendle et al., 2009; Cremonesi
et al., 2010), we define that an interaction $(u,i)$ is memorized at epoch $t$
by the model $\theta$ if item $i$ is in the ranking list of user $u$, denoted
as $m_{t}(u,i)$. To ensure the reliability of ranking results, for each user
$u$, we include the top-$N$ of the ranking items into its ranking list, where
$N$ is the length of observed interactions. However, simply calculating the
memorization of interactions of a single epoch could lead to unstable results,
since the model is not well trained during the noise-resistant period. Hence,
we trace the memorization states of interactions for the most recent $h$
epochs, and define the final memorization of interaction $(u,i)$ as follows:

(1) $\displaystyle
m_{t}^{h}(u,i)=\frac{1}{{|\mathcal{P}_{t}^{h}(u,i)|}}\sum\nolimits_{m_{j}(u,i)\in\mathcal{P}_{t}^{h}(u,i)}m_{i}(u,i)$

where $\mathcal{P}_{t}^{h}(u,i)=\\{m_{t-h+1}(u,i),\cdots,m_{t}(u,i)\\}$
captures the most recent $h$ memorization histories of interaction $(u,i)$. We
define an interaction $(u,i)$ memorized by a model $\theta$ if the majority of
the recent histories $\mathcal{P}_{t}^{h}(u,i)$ coincides with the
memorization state (_i.e.,_ $m_{t}^{h}(u,i)$ is larger than $0.5$). It is
worth mentioning that the definition of memorized interactions dose not need
any labels for supervision, and is able to effectively indicate the underlying
memorization effect of recommendation models.

Figure 3. The monotonicity of MP and MR when training NeuMF and LightGCN on
the MovieLens dataset.

####  3.1.2. Memorization Point Estimation.

As shown in Figure 1, the model predominantly learns clean interactions until
the noise-sensitive period begins. In others words, during noise-resistant
period, the recommendation model (1) not only learns sufficient information
from the clean interactions, (2) but also accumulates some noise from the
noisy implicit feedback. Therefore, we aim to design two metrics to reflect
the above two memorization characteristics of the model. Formally, we use
$\mathcal{M}_{t}$ to denote a set of memorized interactions at epoch $t$, and
use $y_{ui}^{*}$ to represent the true label of interaction $(u,i)$, which is
not available because of the noise in implicit feedback. Inspired by recent
advances in robust learning (Shu et al., 2019; Song et al., 2021) and
recommendation evaluation metrics, we propose two memorization-based metrics,
namely, _memorization precision_ ($MP$) and _memorization recall_ ($MR$), to
address the memorization effect of recommendation models respectively:

(2) $\displaystyle MP_{t}=\frac{|\mathcal{R}_{t}|}{|\mathcal{M}_{t}|},\quad
MR_{t}=\frac{|\mathcal{R}_{t}|}{|\mathcal{G}|}$

where $\mathcal{R}_{t}=\\{(u,i)\in\mathcal{M}_{t}:y_{ui}=y_{ui}^{*}\\}$
denotes the set of memorized data whose true labels are consistent with
predictions, and $\mathcal{G}=\\{(u,i)\in\mathcal{D}:y_{ui}=y_{ui}^{*}\\}$ is
the set of true labeled data in implicit feedback. According to the definition
of $MP$ and $MR$, we can conclude that: $MP$ monotonically decreases since the
model tends to memorize clean data first and then gradually memorizes all the
noisy interactions as the training progresses; and $MR$ monotonically
increases because the model eventually memorizes all clean interactions as the
training progresses, as depicted in Figure 3 (please refer to Section 3.3 for
the theoretical analysis of the monotonicity of the two metrics). Thus, the
best memorization point $t_{m}$ is naturally the best trade-off epoch when
$MP$ and $MR$ share the same value, _i.e.,_ $MP_{t}=MR_{t}$. By substituting
it into Equation (2), the best memorization point $t_{m}$ can be calculated
as:

(3)
$\displaystyle\mathcal{M}_{t_{m}}=|\\{(u,i)\in\mathcal{D}_{t}:y_{ui}=y_{ui}^{*}\\}|=(1-\sigma)|\mathcal{D}|$

where $\sigma$ is the noise rate of the implicit feedback. Since $\sigma$ is
typically unknown, we leverage the difference of loss distributions in clean
and noisy data to estimate $\sigma$, as shown in Figure 4.

Figure 4. Loss distribution of MovieLens dataset.

Specifically, we first normalize the loss values of all training interactions
and then fit them into a two-component Gaussian Mixture Model (GMM) to model
the bi-modal distribution (Arazo et al., 2019; Li et al., 2020) of true- and
false-labed samples. The GMM model can be easily trained by using the
Expectation-Maximization (EM) algorithm. Hence, we could obtain the
probability of an interaction $(u,i)$ being noisy through the posterior
probability of loss distributions. Accordingly, the noise rate $\sigma$ is
estimated as:

(4)
$\displaystyle\hat{\sigma}=\mathbb{E}_{(u,i)\in\mathcal{D}}[p(\mu|L_{(u,i)}(\theta)]$

where $L_{(u,i)}(\theta)$ is the loss of interaction $(u,i)$ of recommendation
model $\theta$, and $\mu$ is the Gaussian component with a larger mean, since
noisy data have typically larger loss values. Note that there are many
approaches (Liu and Tao, 2015; Chen et al., 2019) available to estimate the
noise ratio, we choose GMM because it is easy to apply and has stable
performance on different noisy datasets.

Therefore, SGDL transits to phase II when the number of memorized interactions
reaches the estimated clean data size (_i.e.,_
$|\mathcal{M}_{t}|\geq(1-\hat{\sigma})\mathcal{D}$). Note that, SGDL is able
to estimate the memorization point $t_{m}$, and collect memorized data
$\mathcal{M}_{t_{m}}$ with nearly zero computational overhead and no
additional supervision.

###  3.2. Phase II: Self-Guided Learning

Phase II aims to robustly learn the recommendation model with the denoising
signals of memorized data, which are collected from the training process of
Phase I. Previous studies (Wang et al., 2021a, 2022) need to pre-specify
weighting functions (_e.g.,_ exponential function), and require additional
hyperparameters (_e.g.,_ threshold) to denoise implicit feedback during
training. However, we argue that these methods are fairly hard to be generally
applied in real-world recommendation systems due to two issues: (1) The proper
weighting schemes heavily rely on the training data, which limits their
adaptivity. (2) These methods simply abandon the hard yet clean interactions
because the fixed weighting functions fail to distinguish the interactions
from noisy data, incurring suboptimal recommendation performance. Thus, we aim
to propose a self-guided method that is capable of learning an adaptive
weighting function automatically from the collected memorized data to tackle
the above two issues.

The above denoising scheme assumes that the memorized data is clean and useful
to provide denoising signals for learning user preferences. Nevertheless, as
the memorized data is collected by leveraging the memorization effect of
recommendation models, it inevitably contains some noise. To prevent the model
from being corrupted by such detrimental interactions, we further devise a
novel adaptive denoising scheduler to select and use proper memorized data for
self-guided learning.

####  3.2.1. Denoising Learning with Memorized Data.

During the noise-sensitive period, we aim to enhance the robustness of
training by imposing a weight on each sample loss. Here, we consider the
memorized data that is able to provide denoising signals to learn the weight
for each sample, since it is mostly clean thanks to the memorization effect of
the model. Specifically, let $L_{k}(\theta)$ be the $k$-th sample loss111The
loss can be represented as the pointwise or pairwise loss, according to the
optimization target of models. We demonstrate the effectiveness of our method
on both kinds of loss functions in Section 4.2. of model $\theta$, and
$\mathcal{D}_{T}$ be the training data. The optimal recommendation model
parameter $\theta$ is calculated by minimizing the following weighted loss:

(5)
$\displaystyle\theta^{*}(\psi)=\mathop{\mathrm{argmin}}_{\theta}\frac{1}{|\mathcal{D}_{T}|}\sum\nolimits_{k}^{|\mathcal{D}_{T}|}g(L_{k}(\theta);\psi)L_{k}(\theta)$

where $g(L_{k}(\theta);\psi)$ is the weight on the $k$-th sample loss, and
$\psi$ represents the current parameters of weighting function $g(\cdot)$.
Here, we formulate $g(\cdot)$ as a simple Multi-Layer Perceptron (MLP) with
only one hidden layer, since it is known as a universal approximator for
almost any continuous function (Csáji, 2001; Shu et al., 2019). To adaptively
learn the weight for each sample from memorized data, we optimize the
weighting parameters $\psi$ given current optimal $\theta^{*}(\psi)$:222Notice
that, here $\psi$ is a variable instead of a quantity, which makes
$\theta^{*}(\psi)$ a function of $\psi$, and the gradient in Equation (6) can
be computed.

(6)
$\displaystyle\psi^{*}=\mathop{\mathrm{argmin}}_{w}\frac{1}{|\mathcal{M}_{t_{m}}|}\sum\nolimits_{m}^{|\mathcal{M}_{t_{m}}|}L_{m}(\theta^{*}(\psi))$

where $L_{m}(\theta^{*}(\psi))$ is the $m$-th memorized sample loss given
optimal model parameters $\theta^{*}(\psi)$. Note that the searching for the
optimal $\theta^{*}$ and optimal $\psi^{*}$ requires two nested loops of
optimization (_i.e.,_ bi-level optimization) (Shu et al., 2019; Chen et al.,
2021a), which is costly. Hence, we adopt the idea of meta learning (Finn et
al., 2017; Wu et al., 2018), and update $\theta$ and $\psi$ alternately in a
single loop to guarantee the efficiency of the algorithm. Specifically, as
illustrated in Figure 2, we perform the following training procedure in each
iteration:

  * •

Assumed update of $\theta$. As shown by the blue arrows in Figure 2, we first
make an assumed update of $\theta$ with current weight $\psi$:

(7)
$\displaystyle\hat{\theta}(\psi)=\theta-\eta_{1}\frac{1}{|\mathcal{D}_{T}|}\sum\nolimits_{k}^{|\mathcal{D}_{T}|}g(L_{k}(\theta);\psi)\nabla_{\theta}L_{k}(\theta)$

where we update $\theta$ using gradient descent with learning rate $\eta_{1}$.

  * •

Update of $\psi$. As indicated by the brown arrow in Figure 2, the updates of
the weighting parameters $\psi$ can be guided by gradient of the memorized
data on the updated model:

(8)
$\displaystyle\psi\leftarrow\psi-\eta_{2}\frac{1}{|\mathcal{M}_{t_{m}}|}\sum\nolimits_{m}^{|\mathcal{M}_{t_{m}}|}\nabla_{\psi}L_{m}(\hat{\theta}(\psi))$

where $\eta_{2}$ is the learning rate of weighting parameters $\psi$.

  * •

Actual update of $\theta$. After receiving the denoising signals (_i.e.,_
updated parameters $\psi$) from memorized data, we use them to update the
model:

(9)
$\displaystyle\theta\leftarrow\theta-\eta_{1}\frac{1}{|\mathcal{D}_{T}|}\sum\nolimits_{k}^{|\mathcal{D}_{T}|}g(L_{k}(\theta);\psi)\nabla_{\theta}L_{k}(\theta)$

We use this alternative strategy to optimize the recommendation model and
weighting function during the noise-sensitive period, as illustrated in Figure
2. Although this strategy does not guarantee to find the global optimum, it
empirically works well in many bi-level optimization problems (Finn et al.,
2017; Shu et al., 2019; Chen et al., 2021a).

####  3.2.2. Adaptive Denoising Scheduler.

Above denoising scheme relies on the memorized data to provide denoising
signals for training. However, as memorized data inevitably exhibit noise,
integrating the noise of them would degrade the denoising performance.

Thus, we propose an adaptive denoising scheduler to select _only_ the clean
and informative memorized data for denoising learning. Specifically, we define
the scheduler as $s$ with parameter $\phi$, and choose two representative
factors to quantify the contribution of each memorized data for denoising: (1)
the loss $L_{m}(\theta)$ of $m$-th memorized sample, where $\theta$ is the
actual updated parameters of the model; and (2) the gradient similarity of
$m$-th memorized sample on assumed updated model parameters $\hat{\theta}$ and
actual updated model parameters $\theta$, _i.e.,_
$\cos\big{(}\nabla_{\hat{\theta}}L_{m}(\hat{\theta}),\nabla_{\theta}L_{m}(\theta)\big{)}$.
Here, we use cosine function as the similarity measurement, and other metrics
like inner product can also be applied in practice. The two factors are
associated with the learning outcome and learning process of the $m$-th
memorized sample, respectively. Specifically, the gradient similarity
characterizes the contribution of the memorized sample in the model training.
A large loss value may represent a crucial memorized sample if the gradient
similarity is also large (_i.e.,_ the gradient direction of memorized sample
is consistent with the optimization of the model); and a large loss value with
small gradient similarity may indicate a noisy memorized sample. Considering
the two factors simultaneously, we formulate the sampling probability of
$m$-th memorized data:

(10) $\displaystyle
o_{m}=s\big{(}L_{m}(\theta),\cos(\nabla_{\hat{\theta}}L_{m}(\hat{\theta}),\nabla_{\theta}L_{m}(\theta));\phi\big{)}$
(11)
$\displaystyle\pi_{m}=\frac{\exp(o_{m};\phi)}{\sum_{i\in\mathcal{M}_{t_{m}}}\exp(o_{i};\phi)}$

where $o_{m}$ is the output of the scheduler, and $\pi_{m}$ is the predicted
sampling probability of $m$-th sample. We choose LSTM network (Hochreiter and
Schmidhuber, 1997) as the scheduler, and feed it with the training factors in
each iteration. The intuition behind is that the LSTM could leverage the
historical information to capture the prediction variance (Chang et al., 2017;
Yao et al., 2021), which shows stable performance in our experiments
(different strategies are compared in Section 4.3.3). However, it is
intractable to directly optimize the scheduler since the sampling process is
not differentiable. To make this procedure differentiable and to jointly
optimize the scheduler and the model, we apply the Gumbel-Softmax
reparameterization trick (Jang et al., 2017) to generate differentiable
samples:

(12) $\displaystyle
y_{m}=\frac{\exp(\log(\pi_{m})+\epsilon_{m})/\tau}{\sum_{i\in\mathcal{M}_{t_{m}}}\exp(\log(\pi_{i})+\epsilon_{i})/\tau}$

where $\epsilon_{m}$ is randomly drawn from uniform distribution between 0 and
1, and $\tau$ is the temperature that controls the interpolation between the
discrete distribution and continuous categorical densities (we set $\tau=0.05$
for all experiments). Thus, the scheduler is able to decide which memorized
data to use according to its contribution to denoising implicit feedback, and
adpatively adjust the sampling probability for more informative guided
learning.

###  3.3. Model Analysis

####  3.3.1. Analysis on the monotonicity of $MP$ and $MR$.

We now prove that $MP$ and $MR$ change monotonically over the training time
$t$. Let $\mathcal{N}_{t}$ ($=\mathcal{M}_{t}\setminus\mathcal{R}_{t}$) be the
set of memorized data that are falsely predicted by models. Then,
$|\mathcal{N}_{t+1}|/|\mathcal{N}_{t}|\geq|\mathcal{R}_{t+1}|/|\mathcal{R}_{t}|$
typically holds because the noisy samples are memorized faster than clean
samples after the model stabilizes, as depicted in Figure 3. Hence, it is easy
to conclude:

(13) $\displaystyle|\mathcal{N}_{t+1}|/|\mathcal{N}_{t}|$
$\displaystyle\geq|\mathcal{R}_{t+1}|/|\mathcal{R}_{t}|$ (14)
$\displaystyle\implies|\mathcal{N}_{t+1}||\mathcal{R}_{t}|+|\mathcal{R}_{t+1}||\mathcal{R}_{t}|$
$\displaystyle\geq|\mathcal{R}_{t+1}||\mathcal{N}_{t}|+|\mathcal{R}_{t+1}||\mathcal{R}_{t}|$
(15)
$\displaystyle\implies(|\mathcal{N}_{t+1}|+|\mathcal{R}_{t+1}|)|\mathcal{R}_{t}|$
$\displaystyle\geq(|\mathcal{N}_{t}|+|\mathcal{R}_{t}|)|\mathcal{R}_{t+1}|$

Then, $MP_{t+1}\geq MP_{t}$ can be directly derived from Inequation (15).
Besides, considering that the model eventually would memorize all training
data (Song et al., 2021), we assume that $\mathcal{M}$ would gradually include
more observed interactions, including true predicted ones, _i.e.,_
$|\mathcal{R}_{t+1}|\geq|\mathcal{R}_{t}|$. Consequently, $MR$ increases
monotonically since
$|\mathcal{R}_{t+1}|/|\mathcal{G}|\geq|\mathcal{R}_{t}|/|\mathcal{G}|$.

####  3.3.2. Analysis of the Self-Guide Learning Scheme.

We then focus on the guidance strategy to explain how memorized data benefit
the denoising training. Formally, we follow (Shu et al., 2019), and utilize
chain rule to derive the update function of $\psi$:

(16)
$\displaystyle\psi\leftarrow\psi+\frac{\eta_{1}\eta_{2}}{|\mathcal{D}_{T}|}\sum_{k}^{|\mathcal{D}_{T}|}\big{(}\frac{1}{|\mathcal{M}_{t_{m}}|}\sum_{m}^{|\mathcal{M}_{t_{m}}|}G_{mk}\big{)}\nabla_{\psi}g(L_{k}(\hat{\theta});\psi)$

where $G_{mk}$
($=\nabla_{\theta}L_{m}(\hat{\theta})^{T}\nabla_{\theta}L_{k}(\theta)$)
measures the gradient similarity between $m$-th memorized data and $k$-th
training data. Thus, for each $k$-th training sample, if its gradient is
similar to the average gradient of memorized data, it would be considered as a
beneficial one for learning, and its weight tends to be increased. Conversely,
the weight of the sample inclines to be suppressed. Therefore, memorized data
is able to offer a proper weight for each training sample in terms of gradient
similarities under the self-guide learning scheme.

####  3.3.3. Model Size.

The additional parameters of SGDL come from two parts: (1) the parameters
$2d_{w}$ of weighting function, where $d_{w}$ is the number of the hidden
neurons in one layer MLP; and (2) the $4d_{l}^{2}+12d_{l}$ parameters of LSTM
unit in the adaptive denoising scheduler, where $d_{l}$ is the dimension of
hidden size. Overall, the additional cost of SDGL is negligible, compared with
the tremendous parameters of modern recommendation models.

####  3.3.4. Time Complexity.

Assume that the time complexity of the base model is $O(T)$, the additional
complexity of SGDL mainly comes from phase II, which consists of two denoising
components: (1) the cost of self-guided learning is also $O(T)$, as the
alternative optimization scheme takes no more than three times compared with
the normal training; and (2) the computational complexity of denoising
adaptive scheduler is $O(|\mathcal{D}|d_{l})$. Therefore, the additional time
complexity of SGDL is $O(T+|\mathcal{D}|d_{l})$. Under the same experimental
settings (_i.e.,_ same base model and same embedding size), SGDL achieves
better trade-off between efficiency and effectiveness compared with various
state-of-the-art models: i) For sample re-weighting and sample selection
methods, although most of them are more time-saving than SGDL, they suffer
from difficult/expensive hyperparameters tuning and unstable performance, as
discussed in Section 4.2. ii) For robust graph-based methods, SGDL has a
complexity that is comparable with them, since graph-based methods typically
leverage on extra graph structure to enhance the robustness of the model.

##  4\. Experiments

We provide empirical results to demonstrate the effectiveness of our proposed
SGDL. The experiments are designed to answer the following research questions:

  * •

RQ1: How does SGDL perform, compared with the state-of-the-art denoising
methods as well as the state-of-the-art robust recommender methods?

  * •

RQ2: How does each component of SGDL (_i.e.,_ memorization point estimation,
denoising learning strategy, and adaptive denoising scheduler) affect SGDL?

  * •

RQ3: Is SGDL able to distinguish hard yet clean interactions from noisy
interactions?

###  4.1. Experimental Settings

####  4.1.1. Dataset Description.

We select three real world benchmark datasets to evaluate and compare the
performance of SGDL and its competitors. Table 1 lists their statistics.

  * •

Adressa333https://www.adressa.no/ is a news reading dataset from Adressavisen,
including user click behaviors and the dwell time for each click. Following
previous work (Wang et al., 2021a), clicks with dwell time less than 10
seconds are viewed as noisy interactions.

  * •

Yelp444https://www.yelp.com/dataset/challenge is an open recommendation
dataset released by the Yelp challenge. We use the 2018 version in our
experiments. We follow (Wang et al., 2021a) to mark ratings below 3 as noisy
interactions.

  * •

MovieLens is a widely used dataset for recommendation, which contains 100,000
movie ratings ranging from 1 to 5. Ratings below 3 are regarded as noisy
interactions.

Table 1. Statistics of the datasets used in our experiments.

Dataset | #Users | #Items | #Interactions | Sparsity
---|---|---|---|---
Adressa | 212,231 | 6,596 | 419,491 | 99.97%
MovieLens | 943 | 1,683 | 100,000 | 93.70%
Yelp | 45,548 | 57,396 | 1,672,520 | 99.94%

Table 2. Performance comparison of different denoising methods on robust
recommendation. “${\dagger}$” indicates the improvement of the SGDL over the
baseline is significant at the level of 0.05. The highest scores are in Bold.
R and N refer to Recall and NDCG, respectively.

Database | Adressa | MovieLens | Yelp
---|---|---|---
Base Model | Method | R@5 | R@20 | N@5 | N@20 | R@5 | R@20 | N@5 | N@20 | R@5 | R@20 | N@5 | N@20
NeuMF | Normal | 0.1533† | 0.3208† | 0.1224† | 0.1808† | 0.1023† | 0.2687† | 0.2890† | 0.2765† | 0.0129† | 0.0393† | 0.0129† | 0.0215†
WBPR | 0.1538† | 0.3207† | 0.1225† | 0.1809† | 0.1025† | 0.2689† | 0.2891† | 0.2769† | 0.0128† | 0.0392† | 0.0127† | 0.0214†
IR | 0.1541† | 0.3212† | 0.1229† | 0.1830† | 0.1054† | 0.2704† | 0.2928† | 0.2758† | 0.0132† | 0.0407† | 0.0131† | 0.0229†
T-CE | 0.1537† | 0.3220† | 0.1267† | 0.1839† | 0.1025† | 0.2821† | 0.2923† | 0.2845† | 0.0119† | 0.0396† | 0.0119† | 0.0211†
DeCA | 0.1597 | 0.3205† | 0.1226† | 0.1799† | 0.1024† | 0.2723† | 0.2904† | 0.2801† | 0.0129† | 0.0394† | 0.0129† | 0.0216†
SGDL | 0.1598 | 0.3291 | 0.1272 | 0.1853 | 0.1135 | 0.2844 | 0.3279 | 0.3032 | 0.0155 | 0.0469 | 0.0158 | 0.0260
CDAE | Normal | 0.1445† | 0.3159† | 0.0987† | 0.1886† | 0.0904† | 0.2185† | 0.2617† | 0.2356† | 0.0145† | 0.0436† | 0.0149† | 0.0277†
WBPR | 0.1443† | 0.3158† | 0.0987† | 0.1890† | 0.0908† | 0.2184† | 0.2619† | 0.2346† | 0.0148† | 0.0437† | 0.0151† | 0.0278†
IR | 0.1444 | 0.3152† | 0.0981† | 0.1893† | 0.0909† | 0.2186† | 0.2612† | 0.2358† | 0.0153† | 0.0438 | 0.0152† | 0.0278†
T-CE | 0.1415† | 0.3106† | 0.0991 | 0.1840† | 0.0912† | 0.2158† | 0.2642 | 0.2386† | 0.0147† | 0.0439 | 0.0151† | 0.0279†
DeCA | 0.1447† | 0.3159† | 0.0991 | 0.1888† | 0.0917† | 0.2189† | 0.2641 | 0.2378† | 0.0158† | 0.0438 | 0.0154† | 0.0292†
SGDL | 0.1450 | 0.3181 | 0.0993 | 0.1956 | 0.0921 | 0.2220 | 0.2643 | 0.2404 | 0.0162 | 0.0439 | 0.0172 | 0.0296
NGCF | Normal | 0.0769† | 0.1322† | 0.0571† | 0.0769† | 0.1285† | 0.3103† | 0.3694† | 0.3392† | 0.0267† | 0.0736† | 0.0262† | 0.0417†
WBPR | 0.0770† | 0.1324† | 0.0572† | 0.0769† | 0.1287† | 0.3105† | 0.3692† | 0.3395† | 0.0265† | 0.0739† | 0.0265† | 0.0417†
IR | 0.0772† | 0.1337† | 0.0570† | 0.0768† | 0.1280† | 0.3104† | 0.3701† | 0.3395† | 0.0269† | 0.0737† | 0.0261† | 0.0412†
DeCA | 0.0760† | 0.1326† | 0.0571† | 0.0766† | 0.1304† | 0.3113† | 0.3729† | 0.3401 | 0.0277 | 0.0739† | 0.0262 | 0.0418
SGCN | 0.0773† | 0.1336† | 0.0543† | 0.0770 | 0.1288† | 0.3112† | 0.3768 | 0.3401 | 0.0267† | 0.0734† | 0.0265 | 0.0443
SGL | 0.0775† | 0.1345 | 0.0576 | 0.0768† | 0.1303† | 0.3141† | 0.3763† | 0.3360† | 0.0279 | 0.0750 | 0.0264† | 0.0409†
SGDL | 0.0788 | 0.1347 | 0.0579 | 0.0771 | 0.1309 | 0.3186 | 0.3745 | 0.3404 | 0.0273 | 0.0746 | 0.0267 | 0.0420
LightGCN | Normal | 0.0951† | 0.1817† | 0.0713† | 0.0994† | 0.1258† | 0.3173† | 0.3678† | 0.3358† | 0.0334† | 0.0912† | 0.0332† | 0.0515†
WBPR | 0.0958† | 0.1845† | 0.0733† | 0.1006† | 0.1262† | 0.3189† | 0.3701† | 0.3510 | 0.0333† | 0.0911† | 0.0331† | 0.0512†
IR | 0.0953† | 0.1822† | 0.0726† | 0.1003† | 0.1285† | 0.3194† | 0.3681† | 0.3361† | 0.0305† | 0.0909† | 0.0326† | 0.0510†
DeCA | 0.0974† | 0.1855† | 0.0758† | 0.1162† | 0.1293† | 0.3076† | 0.3575† | 0.3270† | 0.0337 | 0.0911† | 0.0332† | 0.0524
SGCN | 0.0941† | 0.1899† | 0.0765† | 0.1160† | 0.1282† | 0.3210† | 0.3602† | 0.3318† | 0.0335† | 0.0916 | 0.0346 | 0.0528
SGL | 0.0980† | 0.1770† | 0.0741† | 0.0999† | 0.1299† | 0.3156† | 0.3638† | 0.3343† | 0.0341 | 0.0915 | 0.0344 | 0.0526
SGDL | 0.1134 | 0.2105 | 0.0844 | 0.1178 | 0.1378 | 0.3335 | 0.3844 | 0.3513 | 0.0339 | 0.0918 | 0.0341 | 0.0525

####  4.1.2. Evaluation Metrics.

We adopt cross-validation to verify the performance. Specifically, we follow
(Wang et al., 2021a, 2022) to split the interactions into the training set,
validation set, and clean test set with the ratio of 8:1:1. The performance is
measured by two widely used valuation protocols (Wang et al., 2019; He et al.,
2020): Recall@$K$ and NDCG@$K$, where $K$ is set as 5 and 20 by default. We
report the average metrics for all users in the test set.

####  4.1.3. Baselines.

We select four state-of-the-art recommendation methods as the base model $f$
of SGDL:

  * •

NeuMF (He et al., 2017) is a state-of-the-art model, which generalizes
Factorization Machines (FM) with a Multi-Layer Perceptron (MLP).

  * •

CDAE (Wu et al., 2016) is denoising auto-encoder model, which corrupts the
interactions with random noises, and then employs a MLP model to reconstruct
the original input.

  * •

NGCF (Wang et al., 2019) is a graph model, which applies graph convolution
network (GCN) to encode high-order collaborative signals in user-item
bipartite graph.

  * •

LightGCN (He et al., 2020) is a state-of-the-art graph model, which simplifies
the design of GCN by discarding the nonlinear feature transformations for
recommendation.

We train the base models with different ranking loss functions to demonstrate
the universality of SGDL. Specifically, we train NeuMF and CDAE with binary
cross-entropy (BCE) loss, and train NGCF and LightGCN with BPR loss (Rendle et
al., 2009). Each model is trained with the following denoising approaches:

  * •

Normal is trained with the original architecture design, without any denoising
consideration.

  * •

WBPR (Gantner et al., 2012) is a sample selection method, which considers the
popular but uninteracted items are likely to be real negative ones.

  * •

IR (Wang et al., 2021b) is the state-of-the-art sample selection method, which
interactively relabels uncertain samples to mitigate the noise in both
observed and unobserved interactions.

  * •

T-CE (Wang et al., 2021a) is the state-of-the-art sample re-weighting method,
which uses the Truncated BCE to assign zero weights to large-loss examples
with a dynamic threshold. Note that, this denoising approach can only be used
for BCE loss, and thus, we implement it with NeuMF and CDAE for comparison.

  * •

DeCA (Wang et al., 2022) is a newly proposed sample re-weighting method, which
considers the disagreement predictions of noisy samples across different
models, and minimizes KL-divergence between the two models’ predictions to
enhance the robustness of models.

In addition, we also compare SGDL with the state-of-the-art robust graph-based
methods to further confirm the effectiveness of our model. Note that the
methods can only be applied to graph-based recommenders (_i.e.,_ NGCF and
LightGCN in our experiments), since they regard the noisy interactions as
noisy edges, and devise enhanced graph learning methods for robust
recommendation.

  * •

SGCN (Chen et al., 2021b) is the state-of-the-art graph structure enhanced
method, which attaches the GCN layers with a trainable stochastic binary mask
to prune noisy edges in user-item bipartite graph.

  * •

SGL (Wu et al., 2021) is the state-of-the-art self-supervised graph method,
which designs different graph views to mine hard negatives and denoise noise
in implicit feedback. We choose the Edge Dropout (ED) view as auxiliary
supervision signal as it performs the best for most datasets.

####  4.1.4. Parameter Settings.

We implement SGDL in Pytorch, and have released our
implementations555https://github.com/ZJU-DAILY/SGDL (codes, parameter
settings, andtraining logs) to facilitate reproducibility. We use recommended
parameter settings for all models, and optimize them with Adam (Kingma and Ba,
2014) optimizer. The Xavier initializer (Glorot and Bengio, 2010) is used to
initialize the model parameters. We set the batch size as 128 for MovieLens,
1024 for Adressa, and 2048 for Yelp due to the different sizes of each
dataset. The learning rate of CDAE, NeuMF, and LightGCN is tuned as 0.001; and
for NGCF, the learning rate is set as 0.0001. For NeuMF, the embedding size is
32, and the number of layers is 3. For CDAE, the hidden size is 100, and the
dropout ratio is set to 0.5. For LightGCN, we set the embedding size to 64,
the number of layers to 3, and train it without dropout. For NGCF, the
embedding size and layers are the same as LightGCN; and the node dropout rate
is set to 0.1. For SGDL, we set $\eta_{1}=\eta_{2}$ for denoising learning,
and tune the learning rate amongst $\\{10^{-4},10^{-3},10^{-2}\\}$ for two
phases, respectively. The hidden size of MLP and LSTM unit is 64; the length
of memorization history $h$ is tuned among $\\{2,5,10,20\\}$. Note that, the
hyperparameters of base models keep exactly the same across all training
methods for a fair comparison.

###  4.2. Performance Comparison (RQ1)

We begin with the performance comparison _w.r.t._ recall@$K$ and NDCG@$K$,
where we test two values (5 and 20) of $K$. The experimental results are
reported in Table 2, and we find that:

  * •

The proposed SGDL can effectively improve the performance of all base models,
and outperform all denoising methods over three datasets. Besides, even if the
base model is designed to be robust against noisy implicit feedback (_i.e.,_
CDAE), our method can still boost its performance by a large margin. We
attribute these improvement to the memorization-based denoising schemes of
SGDL: (1) By tracing the memorization states of data in the noise-resistant
period, SGDL is able to collect memorized interactions during training process
to provide valuable denoising signals without any supervision. In contrast,
none of the baselines considers explicitly characterizing data from the
memorization perspective. (2) Benefiting from our learning to weight strategy
and adaptive denoising scheduler, SGDL can adaptively select clean and
informative samples from memorized interactions, and use them to guide the
learning process of the model. However, other re-weighting baselines (_e.g.,_
T-CE and DeCA) are insufficient to provide proper weight for each interaction
since they do not have memorized data as guidance.

  * •

Jointly analyzing the performance of SGDL across three datasets, we find that
the improvement on MovieLens dataset are less significant than that on other
datasets. One possible reason is that the sparsity of MovieLens dataset is
denser than the sparsity of Yelp and Adressa. Accordingly, there are
sufficient interactions to identify user behavior patterns, which offsets the
impact of noisy implicit feedback.

  * •

Jointly analyzing the performance of SGDL across the recommenders, we observe
that the relative improvement (_i.e.,_ the performance of SGDL over the
strongest baselines) on NeuMF and CDAE are more substantial than that on
graph-based methods (_i.e.,_ NGCF and LightGCN). This is because our method is
model-agnostic, which does not take the graph structure into consideration. In
comparison, both SGCN and SGL are designed for graph-based recommendations,
and thus, they can leverage the graph structure information to yield better
performance. Nonetheless, our method still achieves the best results on most
cases, which demonstrate the superior robustness of SGDL.

  * •

All the denoising approaches show better results compared with normal training
in most cases, which indicate the necessities of denoising implicit feedback
for recommendations. The results are consistent with prior studies (Wang et
al., 2021a, 2022).

  * •

The performance of sample selection methods (_i.e.,_ WBPR and IR) is rather
unstable across different recommenders, compared with sample re-weighting
methods. This is reasonable since sample selection methods highly depend on
the sampling distribution, which makes their performance unstable. For
instance, although IR achieves some improvement on Yelp dataset, it performs
worse than the base model (_i.e.,_ CDAE) on Adressa dataset.

  * •

Robust Graph-based methods (_i.e.,_ SGCN and SGL) achieve competitive or even
the best performance against other methods. We attribute such improvement to
their carefully designed graph structures, which is able to prune noisy and
insignificant edges for clean information propagation. However, the methods
can only be applied to graph-based models, while SGDL is able to be easily
integrated with any learning-based recommender systems, and achieves
comparable or even better performance.

Table 3. Impact of denoising Learning and adaptive denoising scheduler.

Database | Adressa | MovieLens | Yelp
---|---|---|---
Base Model | Method | R@5 | R@20 | N@5 | N@20 | R@5 | R@20 | N@5 | N@20 | R@5 | R@20 | N@5 | N@20
NeuMF | w/o DLS | 0.1528 | 0.3107 | 0.1211 | 0.1794 | 0.1055 | 0.2690 | 0.2911 | 0.2774 | 0.0136 | 0.0397 | 0.0131 | 0.0218
w/o ADS | 0.1576 | 0.3285 | 0.1255 | 0.1801 | 0.1097 | 0.2801 | 0.3210 | 0.3008 | 0.0146 | 0.0438 | 0.0146 | 0.0259
LightGCN | w/o DLS | 0.0964 | 0.1810 | 0.0702 | 0.0985 | 0.1244 | 0.3159 | 0.3688 | 0.3349 | 0.0330 | 0.0909 | 0.0331 | 0.0513
w/o ADS | 0.1013 | 0.1995 | 0.0811 | 0.1007 | 0.1316 | 0.3328 | 0.3824 | 0.3502 | 0.0338 | 0.0914 | 0.0340 | 0.0521

###  4.3. Study of SDGL (RQ2)

As the memorization and self-guided denoising learning are the core of SGDL,
we also conduct ablation studies to investigate their effectiveness.
Specifically, how the presence of denoising learning and adaptive denoising
scheduler, the estimation of memorization point, and the design of scheduler
affect our model.

####  4.3.1. Impact of Denoising Learning & Scheduler.

We first evaluate the effectiveness of the denoising learning scheme and
adaptive denoising scheduler. To this end, two variants of SGDL are
constructed by (1) discarding the denoising learning strategy in noise-
sensitive period, called SGDL${}_{\text{w/o DLS}}$; and (2) removing the
scheduler and directly using all memorized data for denoising learning, named
SGDL${}_{\text{w/o ADS}}$. We summarize the results of NeuMF and LightGCN in
Table 3, while skip the results of other models (_i.e.,_ CDAE and NGCF)
because they demonstrate similar trends, and we have space limitation.

Obviously, compared with SGDL in Table 2, removing the denoising learning
scheme (_i.e.,_ SGDL${}_{\text{w/o DLS}}$) dramatically reduces the predictive
accuracy, indicating the necessity of self-guided learning. To be more
specific, SGDL${}_{\text{w/o DLS}}$ only trains with memorized data in the
noise-sensitive period, and thus, it is insufficient to learn user true
preference with hard yet clean interactions. Besides, directly leveraging all
memorized data as denoising signals would inevitably introduce some noise, and
hence, the SGDL${}_{\text{w/o ADS}}$ also underperforms the complete model.

####  4.3.2. Estimation of Memorization Point.

We then verify the estimation of memorization point, since it plays an
important role in our model to transit from noise-resistant period to noise-
sensitive period. Specifically, we explore the performance change of SGDL by
increasing or decreasing the estimated noisy ratio $\hat{\sigma}$ to force
early or late transition. The results of SGDL on NeuMF and LightGCN are
presented in Table 4. We observe that:

  * •

Generally speaking, the best performance is achieved at the estimated
memorization point. When the memorization point more deviates from the
estimated one, the performance tends to be worse. This means the existence of
the best memorization point and the effectiveness of our estimation.

  * •

It is worth mentioning that, when the memorization point is slightly earlier
than the estimated value, it also shows good performance. We attribute this to
the clean memorized data: when we transit the model to noise-sensitive period
earlier, it is more likely that most of the memorized interactions are clean,
which benefits the following self-guided learning. On the contrary, when we
delay the memorization point, the memorized data tends to contain more noisy
samples, incurring suboptimal performance.

Table 4. Impact of estimation of memorization point. R@20 is used to evaluate
the performance, and Est. denotes estimated memorization point.

Memorization Point | Early | Est. | Late
---|---|---|---
Base Model | Database | +10% | +5% | +0% | -5% | -10%
NeuMF | Adressa | 0.3221 | 0.3275 | 0.3291 | 0.3256 | 0.3203
MovieLens | 0.2810 | 0.2851 | 0.2844 | 0.2757 | 0.2704
Yelp | 0.458 | 0.4420 | 0.0469 | 0.4430 | 0.4370
LightGCN | Adressa | 0.2006 | 0.2114 | 0.2105 | 0.2017 | 0.1990
MovieLens | 0.3321 | 0.3325 | 0.3335 | 0.3262 | 0.3198
Yelp | 0.0895 | 0.0912 | 0.0918 | 0.0904 | 0.0887

####  4.3.3. The Design of Adaptive Denoising Scheduler.

We also investigate the design of adaptive denoising scheduler. Specifically,
we propose three different approaches to evaluate the denoising contributions
of each memorized data: i) Rank memorized data according to their sums of two
factors (_i.e.,_ normalized gradient similarity and loss value), and simply
pick top-$F$ memorized data as informative denoising signals. To keep the
picked data clean, we set $F$ as the half size of memorized data. ii) Choose a
one-layer MLP as the scheduler, and train the scheduler with the strategy
presented in Section 3.2.2. iii) Select the LSTM as the scheduler, and
leverage the historical factors to predict the sampling probabilities. We
report the performance of three sampling approaches with Recall@20 and NDCG@20
in Table 5, while Recall@5 and NDCG@5 are omitted due to limited space. We
have the following observations:

  * •

By simply choosing top-$F$ memorized data with the highest sums as denoising
signals, the performance drops and becomes even worse than the normal
training. We attribute such degradation to the non-linear correlation of two
factors: the large loss value may not necessarily mean the memorized sample is
beneficial to denoising, as it can also be noisy if the gradient similarity is
small. Directly summing up the two factors is unable to properly capture the
characteristic of memorized data, and thus fails to provide reliable denoising
signals for self-guided learning.

  * •

The scheduler with LSTM consistently achieves the best performance cross the
datasets. This is because LSTM can leverage the historical factors information
to predict more accurate and stable sampling probability for memorized data.
The results are consistent with previous robust learning studies (Chang et
al., 2017; Yao et al., 2021).

Table 5. Impact of the design of adaptive denoising scheduler.

Database | Yelp | MovieLens | Adressa
---|---|---|---
Base Model | ADS | R@20 | N@20 | R@20 | N@20 | R@20 | N@20
NeuMF | top-$F$ | 0.3106 | 0.1766 | 0.2590 | 0.2705 | 0.0386 | 0.0212
MLP | 0.3257 | 0.1842 | 0.2749 | 0.2841 | 0.0392 | 0.0225
LSTM | 0.3291 | 0.1853 | 0.2844 | 0.3032 | 0.0469 | 0.0260
LightGCN | top-$F$ | 0.1810 | 0.0985 | 0.3159 | 0.3349 | 0.0909 | 0.0513
MLP | 0.2066 | 0.0103 | 0.3284 | 0.3427 | 0.0912 | 0.0512
LSTM | 0.2105 | 0.1178 | 0.3335 | 0.3513 | 0.0918 | 0.0525

Figure 5. (a) Sample weight distribution on MovieLens dataset. (b) Learned
weighting function on MovieLens dataset.

###  4.4. Learned Weights of SGDL (RQ3)

In this section, we visualize the learned weights of SGDL to offer an
intuitive impression of our denoising performance. Specifically, we train
NeuMF with SGDL on the MovieLens dataset, and plot the learned weights
distribution _w.r.t._ clean and noisy interactions as well as their loss
values in Figure 5. We find that:

  * •

The left of Figure 5 indicates that almost all large weights belong to clean
interactions, and the weights of noisy interactions are much smaller than that
of clean ones, meaning that the SGDL can differentiate clean implicit feedback
from the noisy one.

  * •

The learned weighting function of SGDL in the right of Figure 5 demonstrates
that when the loss has relatively small values, the weighting function
inclines to increase the weight together with loss, indicating that it tends
to emphasize more on clean data for learning user preference; while as the
loss gradually becomes larger, the weighting function first remains unchanged
and then begins to dramatically decrease its weight, implying that it tends to
highlight the hard yet clean interactions with large weights first and then
suppress noise interactions. Therefore, SGDL is able to automatically locate
hard interactions for better learning user preferences.

##  5\. Related Work

Existing recommender systems are typically trained with implicit feedback.
Recently, some studies (Jagerman et al., 2019; Wang et al., 2021a, b; Lee et
al., 2021) have noticed that implicit feedback could be easily corrupted by
different factors (_e.g.,_ popularity bias (Chen et al., 2020) and unawareness
of users’ behaviors (Hu et al., 2008)), and the inevitable noise would
dramatically degrade the recommendation performance (Wang et al., 2021b, a,
2022; Tan et al., 2022). As a result, some efforts have been dedicated to
solving the noisy implicit feedback problem, which can be categorized into
sample selection methods (Gantner et al., 2012; Park and Chang, 2019; Ding et
al., 2019b; Yu and Qin, 2020; Wang et al., 2021b) and sample re-weighting
methods (Wang et al., 2021a; Hu et al., 2021; Wang et al., 2022).

Sample Selection. A simple idea to denoise implicit feedback is to select
clean and informative samples only, and train the recommendation model with
them. For example, WBPR (Gantner et al., 2012) considers that the missing
interactions of popular items are highly likely to be real negative examples,
and hence assigns higher sampling probabilities to them. IR (Wang et al.,
2021b) interactively generates pseudo-labels for user preferences based on the
difference between labels and predictions, to discover the noisy-positive and
noisy-negative examples. Nonetheless, their performance has high variance
since they heavily depend on the sampling distribution (Yuan et al., 2018).

Sample Re-weighting. On the other hand, loss-based methods focus on the
learning process of models (_e.g.,_ loss values and predictions) to
distinguish noisy interactions from clean data. For instance, T-CE (Wang et
al., 2021a) dynamically assigns lower weights to high-loss samples since it
has been shown that noisy examples would have larger loss values. DeCA (Wang
et al., 2022) develops an ensemble method to minimize the KG-divergence
between the two models’ predictions, under the assumption that different
models make relatively similar predictions on clean examples. Although these
methods achieve promising results without additional data, they heavily rely
on the predefined loss function and hyperparameters, incurring poor
generalization for different recommendation models.

Other Directions. There are some recent studies that consider using additional
information (Yi et al., 2014; Xie et al., 2020; Bian et al., 2021) or
designing model-specific structures (Yang et al., 2021; Chen et al., 2021b; Wu
et al., 2021) to improve the robustness of recommender systems. For instance,
DFN (Xie et al., 2020) proposes a feedback interaction component to extract
clean and useful information from noisy feedback with additional explicit
feedback (_e.g.,_ like and dislike). SGCN (Chen et al., 2021b) treat user-item
interactions as a bipartite graph, and devise a learnable regularization
module to preserve the sparsity and low rank of graph structure. SGL (Wu et
al., 2021) advances graph-based recommender systems with self-supervised
learning by employing graph structure augmentations. However, these methods
suffer from poor generalization, since they either need additional information
(_e.g.,_ explicit feedback) as guidance to denoise implicit feedback (Xie et
al., 2020; Bian et al., 2021), or are only applicable to specific data
structures (_e.g.,_ user-item bipartite graph) (Chen et al., 2021b; Yang et
al., 2021; Wu et al., 2021).

Difference from Existing Work. Our work can be seen as a variant of self-
training (Zoph et al., 2020), which leverages self-labeled memorized data to
enhance the robustness of training process. Compared with existing work, SGDL
is in an orthogonal direction, opening up a new research line of denoising
implicit feedback for recommendations. The most relevant work with ours is
MORPH (Song et al., 2021). However, it tackles the classification problem in
computer vision, and only uses the memorized data for training, which is not
feasible in ranking-based recommendations since the hard yet clean data may
not be memorized without clear guidance. Compared with previous methods, SGDL
is a truly model-agnostic framework, which can be easily applied to any
learning-based recommendation models with any ranking loss functions, and does
not need to define any weighting functions.

##  6\. Conclusion and Future Work

In this paper, we present a new denoising paradigm, called SGDL, which
leverages the memorization effect of recommendation models and designs two
training phases to exploit the self-labeled memorization data as guidance for
denoising learning. Extensive experiments conducted on three real-world
datasets with four representative recommendation models demonstrate the
superiority and universality of SGDL. In the future, we plan to jointly
explore the noise and bias existing in implicit feedback to develop a
universal denoising and debiasing solution.

###### Acknowledgements.

This work was supported by the NSFC under Grants No. (62025206, 61972338, and
62102351). Lu Chen is the corresponding author of the work.

## References

  * (1)
  * Arazo et al. (2019) Eric Arazo, Diego Ortego, Paul Albert, Noel O’Connor, and Kevin McGuinness. 2019.  Unsupervised label noise modeling and loss correction. In _ICML_. 312–321.
  * Arpit et al. (2017) Devansh Arpit, Stanisław Jastrzundefinedbski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S. Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, and Simon Lacoste-Julien. 2017.  A Closer Look at Memorization in Deep Networks. In _ICML_. 233–242.
  * Bian et al. (2021) Zhi Bian, Shaojun Zhou, Hao Fu, Qihong Yang, Zhenqi Sun, Junjie Tang, Guiquan Liu, Kaikui Liu, and Xiaolong Li. 2021.  Denoising User-aware Memory Network for Recommendation. In _RecSys_. 400–410.
  * Chang et al. (2017) Haw-Shiuan Chang, Erik Learned-Miller, and Andrew McCallum. 2017\.  Active bias: Training more accurate neural networks by emphasizing high variance samples. In _NeurIPS_. 1002–1012.
  * Chen et al. (2021b) Huiyuan Chen, Lan Wang, Yusan Lin, Chin-Chia Michael Yeh, Fei Wang, and Hao Yang. 2021b.  Structured Graph Convolutional Networks with Stochastic Masks for Recommender Systems. In _SIGIR_. 614–623.
  * Chen et al. (2021a) Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. 2021a.  AutoDebias: Learning to Debias for Recommendation. In _SIGIR_. 21–30.
  * Chen et al. (2020) Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020\.  Bias and debias in recommender system: A survey and future directions. In _arXiv preprint arXiv:2010.03240_.
  * Chen et al. (2019) Pengfei Chen, Ben Ben Liao, Guangyong Chen, and Shengyu Zhang. 2019.  Understanding and utilizing deep neural networks trained with noisy labels. In _ICML_. 1062–1070.
  * Cremonesi et al. (2010) Paolo Cremonesi, Yehuda Koren, and Roberto Turrin. 2010\.  Performance of recommender algorithms on top-n recommendation tasks. In _RecSys_. 39–46.
  * Csáji (2001) Balázs Csanád Csáji. 2001\.  Approximation with artificial neural networks.  _Faculty of Sciences, Etvs Lornd University, Hungary_ 24, 48 (2001).
  * Ding et al. (2019a) Jingtao Ding, Yuhan Quan, Xiangnan He, Yong Li, and Depeng Jin. 2019a.  Reinforced Negative Sampling for Recommendation with Exposure Data.. In _IJCAI_. 2230–2236.
  * Ding et al. (2019b) Jingtao Ding, Guanghui Yu, Xiangnan He, Fuli Feng, Yong Li, and Depeng Jin. 2019b.  Sampler design for bayesian personalized ranking by leveraging view data.  _TKDE_ 33 (2019), 667–681.
  * Du et al. (2022a) Yuntao Du, Xinjun Zhu, Lu Chen, Ziquan Fang, and Yunjun Gao. 2022a.  MetaKG: Meta-learning on Knowledge Graph for Cold-start Recommendation.  _TKDE_ (2022).
  * Du et al. (2022b) Yuntao Du, Xinjun Zhu, Lu Chen, Baihua Zheng, and Yunjun Gao. 2022b.  HAKG: Hierarchy-Aware Knowledge Gated Network for Recommendation. In _SIGIR_.
  * Finn et al. (2017) Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017\.  Model-agnostic meta-learning for fast adaptation of deep networks. In _ICML_. 1126–1135.
  * Gantner et al. (2012) Zeno Gantner, Lucas Drumond, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2012.  Personalized ranking for non-uniformly sampled items. In _KDD Cup_. 231–247.
  * Gharibshah et al. (2020) Zhabiz Gharibshah, Xingquan Zhu, Arthur Hainline, and Michael Conway. 2020.  Deep learning for user interest and response prediction in online display advertising.  _Data Science and Engineering_ 5, 1 (2020), 12–26.
  * Glorot and Bengio (2010) Xavier Glorot and Yoshua Bengio. 2010.  Understanding the difficulty of training deep feedforward neural networks. In _AISTATS_. 249–256.
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, YongDong Zhang, and Meng Wang. 2020\.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In _SIGIR_. 639–648.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _WWW_. 173–182.
  * He et al. (2016) Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016.  Fast matrix factorization for online recommendation with implicit feedback. In _SIGIR_. 549–558.
  * Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jürgen Schmidhuber. 1997.  Long short-term memory.  _Neural computation_ 9 (1997), 1735–1780.
  * Hu et al. (2021) Kaixi Hu, Lin Li, Qing Xie, Jianquan Liu, and Xiaohui Tao. 2021.  What is Next When Sequential Prediction Meets Implicitly Hard Interaction?. In _CIKM_. 710–719.
  * Hu et al. (2008) Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008.  Collaborative filtering for implicit feedback datasets. In _ICDM_. 263–272.
  * Jagerman et al. (2019) Rolf Jagerman, Harrie Oosterhuis, and Maarten de Rijke. 2019\.  To model or to intervene: A comparison of counterfactual and online learning to rank from user interactions. In _SIGIR_. 15–24.
  * Jang et al. (2017) Eric Jang, Shixiang Gu, and Ben Poole. 2017.  Categorical reparameterization with gumbel-softmax. In _ICLR_.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization. In _ICLR_.
  * Lee et al. (2021) Dongha Lee, SeongKu Kang, Hyunjun Ju, Chanyoung Park, and Hwanjo Yu. 2021.  Bootstrapping User and Item Representations for One-Class Collaborative Filtering. In _SIGIR_. 317–326.
  * Li et al. (2020) Junnan Li, Richard Socher, and Steven CH Hoi. 2020\.  DivideMix: Learning with noisy labels as semi-supervised learning. In _ICLR_.
  * Lin et al. (2019) Tzu-Heng Lin, Chen Gao, and Yong Li. 2019.  Cross: Cross-platform recommendation for social e-commerce. In _SIGIR_. 515–524.
  * Liu and Tao (2015) Tongliang Liu and Dacheng Tao. 2015.  Classification with noisy labels by importance reweighting.  _PAMI_ 38, 3 (2015), 447–461.
  * Park and Chang (2019) Dae Hoon Park and Yi Chang. 2019.  Adversarial sampling and training for semi-supervised information retrieval. In _WWW_. 1443–1453.
  * Ren et al. (2017) Zhaochun Ren, Shangsong Liang, Piji Li, Shuaiqiang Wang, and Maarten de Rijke. 2017.  Social Collaborative Viewpoint Regression with Explainable Recommendations. In _WSDM_. 485–494.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian Personalized Ranking from Implicit Feedback. In _UAI_. 452–461.
  * Shu et al. (2019) Jun Shu, Qi Xie, Lixuan Yi, Qian Zhao, Sanping Zhou, Zongben Xu, and Deyu Meng. 2019.  Meta-Weight-Net: Learning an Explicit Mapping For Sample Weighting. In _NeurIPS_. 1919–1930.
  * Song et al. (2021) Hwanjun Song, Minseok Kim, Dongmin Park, Yooju Shin, and Jae-Gil Lee. 2021.  Robust Learning by Self-Transition for Handling Noisy Labels. In _KDD_. 1490–1500.
  * Tan et al. (2022) Yanchao Tan, Carl Yang, Xiangyu Wei, Ziyue Wu, and Xiaolin Zheng. 2022.  Partial Relaxed Optimal Transport for Denoised Recommendation. In _arXiv preprint arXiv:2204.08619_.
  * Wang et al. (2018) Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018\.  DKN: Deep knowledge-aware network for news recommendation. In _WWW_. 1835––1844.
  * Wang et al. (2021a) Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021a.  Denoising implicit feedback for recommendation. In _WSDM_. 373–381.
  * Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019.  Neural graph collaborative filtering. In _SIGIR_. 165–174.
  * Wang et al. (2022) Yu Wang, Xin Xin, Zaiqiao Meng, Xiangnan He, Joemon Jose, and Fuli Feng. 2022\.  Learning Robust Recommenders through Cross-Model Agreement. In _WWW_.
  * Wang et al. (2021b) Zitai Wang, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, and Qingming Huang. 2021b.  Implicit Feedbacks are Not Always Favorable: Iterative Relabeled One-Class Collaborative Filtering against Noisy Interactions. In _MM_. 3070–3078.
  * Wu et al. (2021) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021.  Self-supervised graph learning for recommendation. In _SIGIR_. 726–735.
  * Wu et al. (2018) Lijun Wu, Fei Tian, Yingce Xia, Yang Fan, Tao Qin, Lai Jian-Huang, and Tie-Yan Liu. 2018.  Learning to teach with dynamic loss functions. In _NeurIPS_. 6466–6477.
  * Wu et al. (2016) Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. 2016.  Collaborative denoising auto-encoders for top-n recommender systems. In _WSDM_. 153–162.
  * Xie et al. (2020) Ruobing Xie, Cheng Ling, Yalong Wang, Rui Wang, Feng Xia, and Leyu Lin. 2020\.  Deep Feedback Network for Recommendation.. In _IJCAI_. 2519–2525.
  * Yang et al. (2021) Yonghui Yang, Le Wu, Richang Hong, Kun Zhang, and Meng Wang. 2021.  Enhanced Graph Learning for Collaborative Filtering via Mutual Information Maximization. In _SIGIR_. 71–80.
  * Yao et al. (2021) Huaxiu Yao, Yu Wang, Ying Wei, Peilin Zhao, Mehrdad Mahdavi, Defu Lian, and Chelsea Finn. 2021.  Meta-learning with an Adaptive Task Scheduler. In _NeurIPS_.
  * Yi et al. (2014) Xing Yi, Liangjie Hong, Erheng Zhong, Nanthan Nan Liu, and Suju Rajan. 2014.  Beyond clicks: dwell time for personalization. In _RecSys_. 113–120.
  * Yu and Qin (2020) Wenhui Yu and Zheng Qin. 2020.  Sampler design for implicit feedback data by noisy-label robust learning. In _SIGIR_. 861–870.
  * Yuan et al. (2018) Fajie Yuan, Xin Xin, Xiangnan He, Guibing Guo, Weinan Zhang, Tat-Seng Chua, and Joemon M Jose. 2018.  fBGD: Learning Embeddings From Positive Unlabeled Data with BGD. In _UAI_.
  * Zoph et al. (2020) Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, and Quoc Le. 2020.  Rethinking Pre-training and Self-training. In _NeurIPS_. 3833–3845.


# SMCL: SALIENCY MASKED CONTRASTIVE LEARNING FOR
LONG-TAILED VISUAL RECOGNITION

###### Abstract

Real-world data often follow a long-tailed distribution with a high imbalance
in the number of samples between classes. The problem with training from
imbalanced data is that some background features, common to all classes, can
be unobserved in classes with scarce samples. As a result, this background
correlates to biased predictions into “major” classes. In this paper, we
propose saliency masked contrastive learning, a new method that uses saliency
masking and contrastive learning to mitigate the problem and improve the
generalizability of a model. Our key idea is to mask the important part of an
image using saliency detection and use contrastive learning to move the masked
image towards minor classes in the feature space, so that background features
present in the masked image are no longer correlated with the original class.
Experiment results show that our method achieves state-of-the-art level
performance on benchmark long-tailed datasets.

Index Terms—  Long-tailed learning, contrastive learning, saliency, masking,
vision recognition, data augmentation

##  1 Introduction

(a) major (bird)

(b) minor (horse)

(c) saliency masked contrastive learning.

Fig. 1: (a): The image and its CAM of a major class sample (bird). (b): The
image and its CAMs of a minor class sample (horse), misclassified as “bird” by
the classifier. The middle image is the CAM for the predicted label (bird),
and the right image is the CAM for the true label (horse). (c): Illustration
of saliency masked contrastive learning.

Deep neural networks have shown remarkable performance across many computer
vision tasks such as image recognition and object detection. It is well known
that outstanding performance results from training with a large amount of
labeled data. However, many real-world datasets follow a long-tailed
distribution, where the number of samples in each class differs greatly. In
these datasets, the “major” classes have a large number of samples while the
“minor” classes have only a few samples in each class. When a model is trained
with a long-tailed dataset, the imbalance between classes hinders the model
from correctly learning discriminative features.

We focus on the problem where a background feature that should be common to
all classes is mistakenly correlated with a particular class. While this
problem also occurs in balanced datasets, its effect is amplified in long-
tailed datasets due to the lack of diversity in minor classes. For example,
background features such as seas and trees should not affect class prediction,
but may not be observed in minor classes, but only in some major classes. This
biased observation mistakenly biases some background to a major class
prediction.

Examples in Fig.1a and 1b show the “biased background” problem of a model
trained from imbalanced data. The image of a horse (a minor class) in Fig.1b
is misclassified as a bird (a major class) by the model. Fig.1b shows two
class activation maps (CAMs) obtained from the model. The left CAM is
associated with the predicted label “bird”, in which the grass area is
highlighted. On the other hand, the right CAM is associated with the true
label “horse”, where the body of the animal is given focus. In other words,
the model thinks the body looks like a horse, but the grass part is a clue
that this image is a bird. Since the background feature (grass) is strongly
attached to the bird class, the model outputs “bird” as the final decision.
The bias is also observed in the CAM image of a bird in Fig.1a, in which the
green area around the bird is highlighted, although the area is not related to
the bird itself.

Our goal is to move the representation of background features presented in
major classes towards minor classes in feature space so that the features are
shared with minor classes and no longer biased towards major classes. To this
end, we propose a new method called saliency masked contrastive learning
(SMCL). As illustrated in Fig.1c, the idea is to mask the salient part of an
image so that only the background part remains, and use contrastive learning
to pull the masked sample towards minor classes in feature space. While the
source image is selected from the original long-tailed distribution, the
target class where the masked image moves to is selected from a distribution
that gives higher weights to minor classes.

Previous methods exist where data augmentation is used to transfer the rich
context features from major classes to minor classes [1]. Our key difference
is that we use contrastive learning with masked data in order to move
background features towards minor classes in feature space, compared to
previous practices where augmented data are assigned labels and trained with
cross-entropy-only loss. As discussed in experiments and ablation studies,
applying contrastive learning is more effective than training them with cross-
entropy-only loss. Experiment results show that the proposed method achieves
comparable performance with the current state-of-the-art on the benchmark
long-tailed datasets.

##  2 Related Work

#### Long-tailed Recognition.

Long-tailed recognition is an important task since many practical datasets are
imbalanced. Initial approaches for long-tailed learning include re-sampling
[1] where more samples are selected from minor classes to achieve a balance
between classes, and re-weighting [2] where the loss function is adjusted to
give more weight to minor class losses. Many variations of re-sampling and re-
weighting techniques have been proposed [3, 4, 5, 6], but they do not address
the problem of biased background features. Mixup methods [7, 8, 1, 9, 10] can
help improve long-tailed learning by mixing two images of different classes
and assigning mixed labels. However, they do not apply contrastive learning
with mixed samples in order to obtain a better representation.

#### Supervised Contrastive Learning.

Supervised contrastive learning (SCL) [11, 12] pulls together samples
belonging to the same class and pushes apart samples from different classes in
the feature space. Recently, SCL is applied to long-tailed learning to
maintain uniform distribution in the feature space and improve class
boundaries [13, 14]. For example, TSC [14] generates uniformly distributed
targets on a hypersphere and uses contrastive learning to make the features of
different classes converge to the targets. Different from previous schemes,
our proposed method uses supervised contrastive learning so that the masked
data containing background features are pulled towards minor class samples in
the feature space.

##  3 Proposed Method

Overview of the proposed method SMCL is shown in Fig.2. SMCL involves saliency
masking to produce background images, weighted sampling to select target
labels giving priority to minor classes, and saliency masked contrastive
learning to pull the background image towards minor classes in the feature
space.

###  3.1 Saliency Masking

In order to mask discriminative features that belong to a certain class, we
use saliency detection methods. Similarly [8], we use the saliency function
$S(\cdot)$ and find the pixel $i$, $j$ that has the maximum value:

$\begin{array}[]{cc}i,j=\mathrm{argmax}(S(x)),\end{array}$ (1)

where $x$ is a sample in the training set $\mathcal{D}$. Then, following the
convention [7], we mask a region of the size generated following a beta
distribution $\mathrm{Beta}(\alpha,\alpha)$ centered at ($i$, $j$).

###  3.2 Weighted Sampling

For each data in the training set, we sample target data to be used in masked
contrastive learning. In order to increase the chance of selecting a minor
class label, we follow the sampling strategy [1] based on the effective number
calculated from class size [2]:

$E_{k}=\frac{(1-\beta^{n_{k}})}{(1-\beta)},$ (2)

where $n_{k}$ is the number of samples in $k$th class, $N=\sum_{k}{n_{k}}$,
and $\beta=(N-1)/N$. Based on $E_{k}$, the sampling probability of class $k$
is:

$p_{k}=\frac{1/E_{k}}{\sum_{k}{1/E_{k}}},$ (3)

which leads to a higher sampling probability for the minor classes. The “minor
weighted” distribution $\tilde{\mathcal{D}}$ is a distribution resulting from
sampling based on $p_{k}$.

###  3.3 Saliency Masked Contrastive Learning

The key idea of SMCL is to transfer a background feature from a major class to
a minor class. For an image $x$ in a batch, we sample a target image
$\tilde{x}$ from the minor weighted distribution $\tilde{\mathcal{D}}$. Then
we use basic augmentation schemes such as random crop and random flip to
create two versions of each sample, $x_{1}$, $x_{2}$, $\tilde{x_{1}}$,
$\tilde{x_{2}}$. Finally, we use saliency masking to generate the masked image
$x_{m}$. Inserting the five samples into the model, we obtain their logits $O$
and feature vectors $F$

$\begin{array}[]{cc}O,F=J([x,\tilde{x},x_{m}]),\end{array}$ (4)

where $O=[o_{1},o_{2},\tilde{o}_{1},\tilde{o}_{2},o_{m}]$ and
$F=[f_{1},f_{2},\tilde{f}_{1},\tilde{f}_{2},f_{m}]$. For the loss function, we
use a combination of the cross-entropy loss and the contrastive loss. The
cross-entropy loss $\mathcal{L}_{MCE}$ is calculated as:

$\mathcal{L}_{MCE}=(1-A)\cdot\mathcal{L}_{CE}(o,y)+A\cdot\mathcal{L}_{CE}(\tilde{o},\tilde{y}),$
(5)

where $o=[o_{1},o_{2},o_{m}]$,
$\tilde{o}=[\tilde{o}_{1},\tilde{o}_{2},o_{m}]$, $\mathcal{L}_{CE}(o,y)$ is
the cross-entropy between the logits $o$ and their true labels $y$, and $A$
indicates the proportion of the masked region. Next, we define the supervised
contrastive loss $\mathcal{L}_{SC}$ as:

$\mathcal{L}_{SC}=-\frac{1}{|B_{y}|-1}\sum_{p\in
B_{y}\setminus\\{z\\}}\log{\frac{\exp(f_{z}\cdot f_{p}/\tau)}{\sum_{k\in
B\setminus\\{z\\}}\exp(f_{z}\cdot f_{k}/\tau)}},$ (6)

where $B$ is the batch, $B_{y}$ is the set of samples in the batch that has
the label $y$, $z$ is the source image, $f$ is the feature vector, and $\tau$
is a temperature hyperparameter. We use a mixed contrastive loss
$\mathcal{L}_{MSC}$:

$\mathcal{L}_{MSC}=(1-A)\cdot\mathcal{L}_{SC}(f,y)+A\cdot\mathcal{L}_{SC}(\tilde{f},\tilde{y}),$
(7)

where $f=[f_{1},f_{2},f^{m}]$,
$\tilde{f}=[\tilde{f}_{1},\tilde{f}_{2},f^{m}]$, and $\mathcal{L}_{SC}(f,y)$
is the supervised contrastive loss when $f$ is the set of feature vectors and
$y$ is the set of true labels associated with the feature vectors. Finally,
our loss function combines the cross-entropy loss and the contrastive loss,
where $\lambda$ and $\mu$ are hyperparameters.

$\mathcal{L}=\lambda\cdot\mathcal{L}_{MCE}+\mu\cdot\mathcal{L}_{MSC}.$ (8)

Fig. 2: Overview of the proposed framework.

##  4 Experiments

###  4.1 Datasets

#### CIFAR-10-LT and CIFAR-100-LT.

CIFAR-LT datasets are constructed from CIFAR [15] by adjusting the dataset
into a long-tailed shape. In our evaluations, we consider three different
imbalance ratios $\rho\in\\{10,50,100\\}$ for CIFAR-10 and CIFAR-100
respectively.

#### ImageNet-LT.

ImageNet-LT is generated by selecting the long-tailed subset from
ImageNet-2012 [16]. The imbalance ratio $\rho$ is 256 and has 115.8K images
from 1,000 categories.

###  4.2 Implementation Settings

#### CIFAR-10-LT and CIFAR-100-LT.

We train ResNet-32 [17] for 200 epochs using stochastic gradient descent (SGD)
with a batch size of 256, a momentum of 0.9, and a weight decay of 2e-4. The
initial learning rate is 0.1 and we decay the learning rate at epochs 160 and
180 by 0.1 [3]. We use CutOut [18] and SimAugment [11] as augmentation
strategies. saliency masking is applied with a probability of 0.2 starting
from epoch 160. The hyperparameters $\lambda$ and $\mu$ are empirically tuned
to $\lambda=1.0$ and $\mu=0.3$.

#### ImageNet-LT.

We train ResNet-50 for 100 epochs using SGD with a batch size of 256, a
momentum of 0.9, and a weight decay of 5e-4. The initial learning rate is set
to 0.1 and follows cosine scheduling. We used RandAug and SimAugment as the
augmentation strategies. Saliency masking is applied with a probability of
0.4. Both augmentation and re-weighting are applied from epoch 80 [3]. The
hyperparameters $\lambda$ and $\mu$ are empirically tuned to $\lambda=1.0$ and
$\mu=0.35$.

The results from Mixup and CutMix are obtained following the same setup from
CMO [1].

###  4.3 Experimental Results

We compare the performance of SMCL with previous methods [3, 2, 19, 6, 20],
including mixup augmentation based methods [9, 7, 10, 1], and supervised
contrastive learning based methods [12, 14, 13].

Results on CIFAR-10/100-LT using different imbalance ratios are shown in Table
1. SMCL outperforms previous methods on CIFAR-10/100-LT when applied with
deferred re-weighting (DRW) [3]. Compared with baseline ERM and Supervised
Contrastive Learning (SCL), SMCL achieves 6.2% and 5.1% improvements in
CIFAR-100-LT (100), respectively. Furthermore, SMCL significantly outperforms
similar augmentation methods, such as Remix [10] and CMO [1]. SMCL also shows
consistent improvements even when compared with SCL based methods.

Dataset | CIFAR-100-LT | CIFAR-10-LT
---|---|---
Imbalance Ratio | 100 | 50 | 10 | 100 | 50 | 10
ERM† | 38.3 | 43.9 | 55.7 | 70.4 | 74.8 | 86.4
DRW† | 41.5 | 45.3 | 58.1 | 76.3 | 80.0 | 87.6
LDAM-DRW† [3] | 42.0 | 46.6 | 58.7 | 77.0 | 81.0 | 88.2
CB Focal [2] | 39.6 | 45.3 | 58.0 | 74.6 | 79.3 | 87.5
BBN† [19] | 42.6 | 47.0 | 59.1 | 79.8 | 82.2 | 88.3
MiSLAS$\ast$ [6] | 47.0 | 52.3 | 63.2 | 82.1 | 85.7 | 90.0
ResLT$\ast$ [20] | 48.2 | 52.7 | 62.0 | 82.4 | 85.2 | 89.7
DRW+Mixup [9] | 45.3 | 49.7 | 60.3 | 81.1 | 83.8 | 89.1
DRW+CutMix [7] | 46.4 | 51.2 | 62.1 | 81.8 | 85.1 | 89.7
ReMix$\ast$ [10] | 46.8 | - | 61.2 | 79.8 | - | 89.0
DRW+CMO [1] | 46.4 | 51.5 | 61.7 | 81.7 | 84.9 | 89.6
SCL | 39.4 | 44.4 | 56.7 | 72.2 | 76.8 | 87.1
SCL-DRW | 41.7 | 46.8 | 57.9 | 75.4 | 79.3 | 87.9
TSC$\ast$ [14] | 43.8 | 47.4 | 59.0 | 79.7 | 82.9 | 88.7
Hybrid-SC$\ast$ [13] | 46.7 | 51.9 | 63.1 | 81.4 | 85.4 | 91.1
SMCL (Ours) | 44.5 | 49.8 | 61.8 | 80.2 | 84.3 | 90.3
DRW+SMCL (Ours) | 50.1 | 54.8 | 64.1 | 84.2 | 86.8 | 91.0

Table 1: Comparison of classification accuracy(%) on the CIFAR-10/100-LT
datasets using ResNet-32. $\ast$ and † are results from the original papers
and [19], respectively.

Methods | All | Many | Med | Few
---|---|---|---|---
ERM† | 41.6 | 64.0 | 33.8 | 5.8
Decouple-cRT† [4] | 47.3 | 58.8 | 44.0 | 26.1
Decouple-LWS† [4] | 47.7 | 57.1 | 45.2 | 29.3
DRW | 49.9 | 61.2 | 47.1 | 29.2
LDAM-DRW | 50.3 | 61.7 | 47.0 | 30.6
Balanced Softmax [5] | 50.6 | 60.1 | 48.4 | 32.6
DRW+Mixup | 49.3 | 60.7 | 46.4 | 28.7
DRW+CutMix | 50.3 | 60.7 | 47.6 | 31.7
DRW+CMO | 50.8 | 61.2 | 48.7 | 30.1
SCL | 45.9 | 69.0 | 38.7 | 8.2
SCL-DRW | 49.4 | 61.6 | 47.2 | 24.1
TSC$\ast$ | 52.4 | 63.5 | 49.7 | 30.4
SMCL (Ours) | 46.0 | 67.9 | 39.1 | 10.3
DRW+SMCL (Ours) | 51.4 | 60.4 | 49.7 | 32.8

Table 2: Comparison of classification accuracy(%) on the ImageNet-LT dataset
using ResNet-50. $\ast$ and † are results from the original papers and [4],
respectively.

Results for ImageNet-LT are reported in Table 2. The table shows the accuracy
of three subsets: Many shots ($>$100 samples), medium shots (20-100 samples),
and few shots ($<$20 samples). SMCL shows comparable performance with the
state-of-the-art [14], showing 2.4% better performance on a few shots. Since
ImageNet-LT is composed of 38.5% of many, 46.9% of medium, and 14.6% of few
classes, improved performance in few has relatively little impact on overall
performance. However, it is noteworthy that applying SMCL with DRW outperforms
most of the existing methods, especially achieving considerable gains in the
accuracy of a few shots.

###  4.4 Ablation Study

#### Impact of Saliency Masking.

To show the impact of saliency masking, we tried masking images with two other
methods: masking random area and masking the center. As shown in Table 3,
covering the center of an image achieved 0.3% higher than the random. Since
many images have their core feature at the center, masking the center area is
more useful than masking a random area. Saliency masking showed the highest
result, approximately 0.3% higher than center masking. This result shows that
saliency masking is indeed effective in selecting background features from an
image.

| Random | Center | Saliency
---|---|---|---
Acc. (%) | 49.56 | 49.82 | 50.14

Table 3: Performance of saliency masking compared with other masking methods
on CIFAR-100-LT ($\rho=100$).

#### Effect of Contrastive Learning.

To evaluate the effect of saliency masked contrastive learning, we compared
the performance of models with and without contrastive learning. In Table 4,
“Cross Entropy” indicates the accuracy when the model is trained with only the
cross-entropy loss. The results show that applying contrastive learning is
effective in improving model performance.

| SMCL | DRW+SMCL
---|---|---
Cross Entropy | 43.57 | 49.12
| Supervised
---
Contrastive Learning
44.49 | 50.14

Table 4: Performance of applying saliency masked contrastive learning compared
with cross-entropy-only learning on CIFAR-100-LT ($\rho=100$).

##  5 Conclusion

We proposed SMCL, a saliency masked contrastive learning method that achieves
high accuracy on long-tailed datasets. SMCL uses saliency masking to obtain
background features from images and applies contrastive learning to move their
embeddings towards minor classes so that they are detached from major classes.
The proposed method is simple to implement, yet achieves state-of-the-art
level performance on benchmark long-tailed datasets such as CIFAR-10/100-LT
and ImageNet-LT.

## References

  * [1] Seulki Park, Youngkyu Hong, Byeongho Heo, Sangdoo Yun, and Jin Young Choi,  “The majority can help the minority: Context-rich minority oversampling for long-tailed classification,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2022.
  * [2] Yin Cui, Menglin Jia, Tsung-Yi Lin, Yang Song, and Serge Belongie,  “Class-balanced loss based on effective number of samples,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9268–9277.
  * [3] Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma,  “Learning imbalanced datasets with label-distribution-aware margin loss,”  Advances in Neural Information Processing Systems, vol. 32, pp. 1567–1578, 2019.
  * [4] Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalantidis,  “Decoupling representation and classifier for long-tailed recognition,”  in International Conference on Learning Representations, 2020.
  * [5] Jiawei Ren, Cunjun Yu, Xiao Ma, Haiyu Zhao, Shuai Yi, et al.,  “Balanced meta-softmax for long-tailed visual recognition,”  Advances in neural information processing systems, vol. 33, pp. 4175–4186, 2020.
  * [6] Zhisheng Zhong, Jiequan Cui, Shu Liu, and Jiaya Jia,  “Improving calibration for long-tailed recognition,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 16489–16498.
  * [7] Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo,  “Cutmix: Regularization strategy to train strong classifiers with localizable features,”  in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019.
  * [8] A F M Shahab Uddin, Mst. Sirazam Monira, Wheemyung Shin, TaeChoong Chung, and Sung-Ho Bae,  “Saliencymix: A saliency guided data augmentation strategy for better regularization,”  in International Conference on Learning Representations, 2021.
  * [9] Hongyi Zhang, Moustapha Cisse, Yann N. Dauphin, and David Lopez-Paz,  “mixup: Beyond empirical risk minimization,”  in International Conference on Learning Representations, 2018.
  * [10] Hsin-Ping Chou, Shih-Chieh Chang, Jia-Yu Pan, Wei Wei, and Da-Cheng Juan,  “Remix: rebalanced mixup,”  in European Conference on Computer Vision. Springer, 2020, pp. 95–110.
  * [11] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton,  “A simple framework for contrastive learning of visual representations,”  in International conference on machine learning. PMLR, 2020, pp. 1597–1607.
  * [12] Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan,  “Supervised contrastive learning,”  Advances in Neural Information Processing Systems, vol. 33, pp. 18661–18673, 2020.
  * [13] Peng Wang, Kai Han, Xiu-Shen Wei, Lei Zhang, and Lei Wang,  “Contrastive learning based hybrid networks for long-tailed image classification,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 943–952.
  * [14] Tianhong Li, Peng Cao, Yuan Yuan, Lijie Fan, Yuzhe Yang, Rogerio S Feris, Piotr Indyk, and Dina Katabi,  “Targeted supervised contrastive learning for long-tailed recognition,”  in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 6918–6928.
  * [15] Alex Krizhevsky, Geoffrey Hinton, et al.,  “Learning multiple layers of features from tiny images,”  University of Toronto, 2009.
  * [16] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei,  “ImageNet Large Scale Visual Recognition Challenge,”  International Journal of Computer Vision (IJCV), vol. 115, no. 3, pp. 211–252, 2015.
  * [17] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun,  “Deep residual learning for image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [18] Terrance Devries and Graham W. Taylor,  “Improved regularization of convolutional neural networks with cutout,”  CoRR, vol. abs/1708.04552, 2017.
  * [19] Boyan Zhou, Quan Cui, Xiu-Shen Wei, and Zhao-Min Chen,  “Bbn: Bilateral-branch network with cumulative learning for long-tailed visual recognition,”  in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 9719–9728.
  * [20] Jiequan Cui, Shu Liu, Zhuotao Tian, Zhisheng Zhong, and Jiaya Jia,  “Reslt: Residual learning for long-tailed recognition,”  IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.