# Plug-In Diffusion Model for Embedding Denoising in Recommendation System

Jujia Zhao<EMAIL_ADDRESS>National University of Singapore ,
Wenjie Wang<EMAIL_ADDRESS>National University of Singapore ,  Yiyan
Xu<EMAIL_ADDRESS>University of Science and Technology of China ,  Teng
Sun<EMAIL_ADDRESS>Shandong University and  Fuli Feng
<EMAIL_ADDRESS>University of Science and Technology of China

###### Abstract.

In the realm of recommender systems, handling noisy implicit feedback is a
prevalent challenge. While most research efforts focus on mitigating noise
through data cleaning methods like resampling and reweighting, these
approaches often rely on heuristic assumptions. Alternatively, model
perspective denoising strategies actively incorporate noise into user-item
interactions, aiming to bolster the model’s inherent denoising capabilities.
Nonetheless, this type of denoising method presents substantial challenges to
the capacity of the recommender model to accurately identify and represent
noise patterns.

To overcome these hurdles, we introduce a plug-in diffusion model for
embedding denoising in recommendation systems, which employs a multi-step
denoising approach based on diffusion models to foster robust representation
learning of embeddings. Our model operates by introducing controlled Gaussian
noise into user and item embeddings derived from various recommender systems
during the forward phase. Subsequently, it iteratively eliminates this noise
in the reverse denoising phase, thereby augmenting the embeddings’ resilience
to noisy feedback. The primary challenge in this process is determining
direction and an optimal starting point for the denoising process. To address
this, we incorporate a specialized denoising module that utilizes
collaborative data as a guide for the denoising process. Furthermore, during
the inference phase, we employ the average of item embeddings previously
favored by users as the starting point to facilitate ideal item generation.
Our thorough evaluations across three datasets and in conjunction with three
classic backend models confirm its superior performance.

Denoising Recommendation, Diffusion Model

††ccs: Information systems Recommender systems

##  1\. Introduction

Figure 1. Illustration of resampling, reweighting, model perspective
denoising, and denoising diffusion methods.

Recommender systems play a pivotal role in personalized information delivery
across a wide range of Web applications (Yuan et al., 2023; Dong et al.,
2023). Typically, recommender models learn personalized user preferences from
user feedback (Quan et al., 2023; Zhang et al., 2023a). Due to the ease of
collecting implicit feedback (_e.g.,_ click and purchase) in large volume, it
has become indispensable for user preference learning (Tian et al., 2022;
Zhang et al., 2023b). Unfortunately, implicit feedback inevitably contains
noises (Chen et al., 2021; Wang et al., 2021b; Lin et al., 2023). For
instance, clicks on micro-videos may not indicate users’ actual satisfaction
due to various interference factors (Wang et al., 2021a). Such noisy feedback
misguides recommender models in interpreting user preferences, subsequently
hampering the recommendation performance (Wang et al., 2023a; Fan et al.,
2023; Lin et al., 2023). As such, denoising implicit feedback for
recommendation becomes an imperative task (Chen et al., 2022).

Previous work primarily mitigates the impact of noisy feedback from the
perspective of data cleaning, including resampling and reweighting user-item
interactions. Specifically, 1) resampling methods (Ding et al., 2018, 2019; Yu
and Qin, 2020) aim to identify noisy interactions and sample more clean
interactions for training (Figure 1(a)). For instance, WBPR (Gantner et al.,
2011) believes that non-interacted popular items are more likely to be true
negative items and allocates higher sampling probabilities. 2) Reweighting
methods (Wang et al., 2023a, 2021a, 2022) utilize all training interactions
yet assign lower weights to potential noisy ones (Figure 1(b)). For instance,
reweighted loss (Wang et al., 2021a) assigns lower weights to the large-loss
interactions since it assumes large-loss interactions are more likely to be
noisy. Notably, these data cleaning methods depend on certain heuristic
assumptions, such as the large-loss assumption (Wang et al., 2021a) and cross-
model agreement (Wang et al., 2022). As their assumptions rely heavily on the
distribution of noisy interactions, these data cleaning methods suffer from
limited adaptability, requiring substantial configuration tuning to adapt to
different backend models and datasets.

For denoising implicit feedback, another research line is from model
perspective, seeking to bolster recommender models’ inherent noise resistance
capabilities. These model perspective methods usually add random noises to
user-item interactions (Wu et al., 2016) or drop positive interactions as
augmented data (Wu et al., 2021; Gao et al., 2022), and then regulate
recommender models to learn robust representations from the augmented data
(Wang and Chen, 2023). For example, CDAE corrupts users’ interactions randomly
with a noise ratio and subsequently optimizes recommender models to recover
the original clean interactions. Nevertheless, as illustrated in Figure 1(c),
these model perspective methods solely rely on a single model to directly
convert noisy data into clean data, imposing substantial demands on the
model’s representation capacity to efficiently capture noise patterns.

Diffusion models, as a kind of powerful generative model, inherently possess a
denoising aptitude to enhance existing model-perspective methods (Ho et al.,
2020). Diffusion models have already revealed remarkable effectiveness across
various domains like image generation and molecule generation, demonstrating
their superior representation capabilities (Vignac et al., 2022; Croitoru et
al., 2022). To adopt diffusion models for denoising recommendations, the
potential benefits lie in two aspects: 1) during the forward process,
diffusion models enhance noise diversity by continuously injecting noises with
controllable noise scales and steps; and 2) in the reverse denoising process,
diffusion models decompose the complex denoising problem into multiple steps,
thereby reducing the denoising difficulty at each step (Figure 1(d)). In light
of these, it is promising to incorporate diffusion models to enhance robust
representation learning of recommender models.

To this end, we propose a plug-in denoising model for existing recommender
models called Denoising Diffusion Recommender Model (DDRM). Given user and
item embeddings from any recommender models, DDRM improves their robustness
against noisy feedback via two processes. In the forward process, we
proactively inject Gaussian noises into user and item embeddings with
adjustable scales and steps, yielding noisy embeddings. The reverse denoising
process then iteratively removes noises via a learnable neural network.
However, the denoising process hinges critically on two elements: the
direction and starting point of denoising, which need to be carefully designed
in the recommendation scenario. In detail, 1) effective denoising direction
can ensure that the denoising path leads to the clean embeddings of users (or
items). In some fields like image generation where diffusion models are
commonly used, they utilize high-level semantic instructions as guidance for
the denoising direction (Croitoru et al., 2022), but the recommendation domain
has yet to explore how to extract good representation from weak collaborative
information to guide the denoising process. 2) A well-chosen starting point
can verify the feasibility and reliability of the denoising process. While
traditional diffusion models typically generate images from pure noises
conditional on abundant textual instructions (Li et al., 2022), generating
high-quality denoised embeddings from pure noises for users and items is
particularly challenging (_cf._ Table 3) since the recommendation data lacks
adequate guidance for conditional generation.

To tackle these obstacles, we design a specialized denoising module for the
reverse process of DDRM. Given noisy user (or item) embeddings, the denoising
module devises strategies to encode the collaborative information, _e.g.,_
users’ liked items, to guide the reverse denoising process. For the inference
phase, to generate an ideal item as the recommendation for a user, we take the
average embeddings of the user’s historically liked items as the starting
point, instead of reverting from pure noises (_cf._ Section 3.3 for details).
Given the generated item embedding, we present a rounding function to ground
the generated item to existing item candidates by the embeddings’ similarity.
As an extension, we also consider adding a reweighted loss to supplement DDRM
from the perspective of data cleaning. We implement DDRM on three
representative recommender models and conduct comprehensive experiments to
validate its effectiveness against other baselines on three public datasets.

The main contributions of this work are threefold:

  * •

We propose a model-agnostic denoising diffusion recommender model, aiming at
enhancing robust representation learning of existing recommender models
against noisy feedback.

  * •

We utilize collaborative information to guide the reverse denoising process of
DDRM and revise the inference phase to alleviate the challenge of generating
item embeddings from pure noises.

  * •

We instantiate DDRM on three backend models and execute extensive experiments
under various settings, confirming its efficacy across three public datasets.

##  2\. Preliminary

Diffusion models have already demonstrated proficient performance in domains
like computer vision and molecular generation (Ruiz et al., 2023; Huang et
al., 2023). Typically, diffusion models encompass two components: the forward
and reverse processes (Ho et al., 2020).

Figure 2. Structure of DDRM. The left part is the backend recommender model.
DDRM accepts both user and item embeddings as inputs and subsequently produces
denoised embeddings that are fed back into the model to do the recommendation
task.

$\bullet$ Forward process aims to inject Gaussian noises into the original
data. Given a data sample $\bm{x_{0}}$, diffusion models continuously add
different scale Gaussian noises to it in $T$ steps until get $\bm{x_{T}}$.
Specifically, for adding noise from $\bm{x_{t-1}}$ to $\bm{x_{t}}$, we have:

(1) $\displaystyle
q(\bm{x_{t}}|\bm{x_{t-1}})=\mathcal{N}(\bm{x_{t}},\sqrt{1-\beta_{t}}\bm{x_{t-1}},\beta_{t}\mathbf{I}),$

where $t\in\\{{1,2,...,T}\\}$ is the current step, $\beta_{t}\in(0,1)$ is the
noise scale in step $t$, $\bm{I}$ is the identity matrix, and $\mathcal{N}$ is
the Gaussian distribution which means $\bm{x_{t}}$ is sampled from this
distribution. According to the additivity of independent Gaussian noises and
reparameterization trick (Ho et al., 2020; Hoogeboom et al., 2021),
$\bm{x_{t}}$ can be directly obtained from $\bm{x_{0}}$ in the calculation:

(2) $\displaystyle
q(\bm{x_{t}}|\bm{x_{0}})=\mathcal{N}(\bm{x_{t}},\sqrt{\bar{\alpha_{t}}}\bm{x_{0}},(1-\bar{\alpha_{t}})\mathbf{I}),$

where $\bar{\alpha_{t}}=\prod_{s=1}^{t}\alpha_{s}$, $\alpha_{s}=1-\beta_{s}$.

$\bullet$ Reverse process is designed to iteratively denoise the noisy data
$\bm{x_{T}}$, following the sequence
($\bm{x_{T}}\rightarrow\bm{x_{T-1}}\rightarrow\bm{x_{T-2}}\rightarrow...\rightarrow\bm{x_{0}}$).
According to (Yang et al., 2022), under the conditions where
$q(\bm{x_{t}}|\bm{x_{t-1}})$ conforms to a Gaussian distribution and
$\beta_{t}$ remains sufficiently small, the distribution
$p(\bm{x_{t-1}}|\bm{x_{t}})$ also exhibits Gaussian properties. As such, a
neural network can be utilized to predict this reverse distribution:

(3) $\displaystyle
p(\bm{x_{t-1}}|\bm{x_{t}})=\mathcal{N}(\bm{x_{t-1}};\bm{\mu}_{\theta}(\bm{x_{t}},t),\bm{\Sigma}_{\theta}(\bm{x_{t}},t)),$

where $\theta$ is the parameters of the neural network, and
$\bm{\mu}_{\theta}(\bm{x}_{t},t)$ and $\bm{\Sigma}_{\theta}(\bm{x}_{t},t)$ are
the mean and covariance of this Gaussian distribution.

$\bullet$ Training. For training the diffusion models, the key focus is
obtaining reliable values for $\bm{\mu}_{\theta}(\bm{x}_{t},t)$ and
$\bm{\Sigma}_{\theta}(\bm{x}_{t},t))$ to guide the reverse process towards
accurate denoising. To achieve this, it is important to optimize the
variational lower bound of the negative log-likelihood of the model’s
predictive denoising distribution $p_{\theta}(\bm{x_{0}})$ :

(4) $\displaystyle\mathcal{L}$ $\displaystyle=\mathbb{E}_{q(x_{0})}\left[-\log
p_{\theta}(\bm{x}_{0})\right]$
$\displaystyle\leq\mathbb{E}_{q}\left[L_{T}+L_{T-1}+...+L_{0}\right],\rm
where$

(5) $\left\\{\begin{aligned}
&L_{T}=D_{\text{KL}}(q(\bm{x}_{T}|\bm{x}_{0})\parallel
p_{\theta}(\bm{x}_{T}))\\\
&L_{t}=D_{\text{KL}}(q(\bm{x}_{t}|\bm{x}_{t+1},\bm{x}_{0})\parallel
p_{\theta}(\bm{x}_{t}|\bm{x}_{t+1}))\\\ &L_{0}=-\log
p_{\theta}(\bm{x}_{0}|\bm{x}_{1}),\end{aligned}\right.$

where $t\in\\{{1,2,...,T-1}\\}$. While $L_{T}$ can be disregarded during
training due to the absence of learnable parameters in the forward process,
$L_{0}$ represents the negative log probability of the original data sample
$x_{0}$ given the first-step noisy data $x_{1}$, and $L_{t}$ aims to align the
distribution $p_{\theta}(\bm{x}_{t}|\bm{x}_{t+1})$ with the tractable
posterior distribution $q(\bm{x}_{t}|\bm{x}_{t+1},\bm{x}_{0})$ in the reverse
process (Luo, 2022).

##  3\. Method

To mitigate the effect of noisy feedback, we propose DDRM to denoise the user
and item embeddings. Refer to Figure 2, DDRM mainly contains three parts:
forward process, reverse process, and denoising modules. Intuitively, given
pre-trained user and item embeddings, DDRM continuously injects Gaussian
noises and then denoises these noisy embeddings iteratively to attain the
final clean embeddings. Through these processes, DDRM captures the noise
distribution in the noisy embeddings and mitigates such noises.

###  3.1. DDRM Framework

$\bullet$ Forward process. Given pre-trained user embeddings $\bm{e_{u}}$ of
user $u$ and item embeddings $\bm{e_{i}}$ of item $i$ from a backend
recommender model, we begin the forward process by setting
$\bm{e_{0}^{u}}=\bm{e_{u}}$ and $\bm{e_{0}^{i}}=\bm{e_{i}}$. Subsequently, we
continuously incorporate Gaussian noises into $\bm{e_{0}^{u}}$ and
$\bm{e_{0}^{i}}$ separately with adjustable scales and steps:

(6) $\displaystyle
q(\bm{e_{t}^{u}}|\bm{e_{0}^{u}})=\mathcal{N}(\bm{e_{t}^{u}},\sqrt{\bar{\alpha_{t}}}\bm{e_{0}^{u}},(1-\bar{\alpha_{t}})\mathbf{I}),$
(7) $\displaystyle
q(\bm{e_{t}^{i}}|\bm{e_{0}^{i}})=\mathcal{N}(\bm{e_{t}^{i}},\sqrt{\bar{\alpha_{t}}}\bm{e_{0}^{i}},(1-\bar{\alpha_{t}})\mathbf{I}),$

where $\bar{\alpha_{t}}=\prod_{s=1}^{t}\alpha_{s}$, $\alpha_{s}=1-\beta_{s}$,
$\beta_{s}\in(0,1)$ controls the noise scale added to the embedding in the
current step $s$, and $e_{t}^{(\cdot)}$ denotes the user or item embeddings in
the forward step $t$. To regulate the noise level in each step, we follow
(Wang et al., 2023c) employing a linear noise schedule in the forward process:

(8)
$1-\bar{\alpha}_{t}=s\cdot\left[\alpha_{\min}+\dfrac{t-1}{T-1}(\alpha_{\max}-\alpha_{\min})\right],$

where $\alpha_{min}$ and $\alpha_{max}$ are the minimum and maximum of the
noise correspondingly, $t$ is the current forward step, $T$ is the total
forward step, and $s\in(0,1)$ controls the noise scale.

$\bullet$ Reverse process. After getting noisy user embeddings
$\bm{e_{T}^{u}}$ and noisy item embeddings $\bm{e_{T}^{i}}$ in the forward
process, we denoise these embeddings iteratively in the reverse process. In
each reverse step, we design the user denoising module and the item denoising
module to denoise user embeddings and item embeddings separately since the
noise distribution is different for users and items:

(9) $\displaystyle
p_{\theta}(\bm{\hat{e}_{t-1}^{u}}|\bm{\hat{e}_{t}^{u}})=\mathcal{N}(\bm{\hat{e}_{t-1}^{u}};\bm{\mu}_{\theta}(\bm{\hat{e}_{t}^{u}},t),\bm{\Sigma}_{\theta}(\bm{\hat{e}_{t}^{u}},t)),$
(10) $\displaystyle
p_{\psi}(\bm{\hat{e}_{t-1}^{i}}|\bm{\hat{e}_{t}^{i}})=\mathcal{N}(\bm{\hat{e}_{t-1}^{i}};\bm{\mu}_{\psi}(\bm{\hat{e}_{t}^{i}},t),\bm{\Sigma}_{\psi}(\bm{\hat{e}_{t}^{i}},t)),$

where $\bm{\hat{e}_{t}^{u}}$ and $\bm{\hat{e}_{t}^{i}}$ are the denoised
embeddings in the reverse step $t$, $\theta$ and $\psi$ are the learnable
parameters of the user denoising module and the item denoising module
correspondingly. These denoising module are executed iteratively in the
reverse process until the generation of final clean embeddings
$\bm{\hat{e}_{0}^{u}}$ and $\bm{\hat{e}_{0}^{i}}$.

$\bullet$ Denoising module. Denoising module aims to denoise the noisy
embedding in each reverse step. Since the user denoising module and the item
denoising module have the same structure, we mainly focus on explaining the
user denoising module which is formulated as Eq. (9). As illustrated in Eq.
(5), the diffusion training aims to align the distribution
$p_{\theta}(\bm{\hat{e}_{t-1}^{u}}|\bm{\hat{e}_{t}^{u}})$ with the tractable
posterior distribution $q(\bm{e_{t-1}^{u}}|\bm{e_{t}^{u}},\bm{e_{0}^{u}})$ in
the reverse process, thus we can use
$q(\bm{e_{t-1}^{u}}|\bm{e_{t}^{u}},\bm{e_{0}^{u}})$ to constrain
$p_{\theta}(\bm{\hat{e}_{t-1}^{u}}|\bm{\hat{e}_{t}^{u}})$. Through Bayes’
theorem, we can derive:

(11) $\displaystyle
q(\bm{e_{t-1}^{u}}|\bm{e_{t}^{u}},\bm{e_{0}^{u}})=\mathcal{N}(\bm{e_{t-1}^{u}},\tilde{\bm{\mu_{t}}}(\bm{e_{t}^{u}},\bm{e_{0}^{u}}),\tilde{\beta}_{t}\bm{I}),\quad\rm
where$ (12) $\small\left\\{\begin{aligned}
&\tilde{\bm{\mu_{t}}}(\bm{e_{t}^{u}},\bm{e_{0}^{u}})=\dfrac{\sqrt{\alpha_{t}}(1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_{t}}\bm{e_{t}^{u}}+\dfrac{\sqrt{\bar{\alpha}_{t-1}}(1-\alpha_{t})}{1-\bar{\alpha}_{t}}\bm{e_{0}^{u}},\\\
&\tilde{\beta}_{t}=\dfrac{(1-\alpha_{t})(1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_{t}}.\end{aligned}\right.$

$\tilde{\bm{\mu_{t}}}(\bm{e_{t}^{u}},\bm{e_{0}^{u}})$ and
$\tilde{\beta}_{t}\bm{I}$ are the mean and covariance of
$q(\bm{e_{t-1}^{u}}|\bm{e_{t}^{u}},\bm{e_{0}^{u}})$. Following (Wang et al.,
2023c), we can similarity factorize
$p_{\theta}(\bm{\hat{e}_{t-1}^{u}}|\bm{\hat{e}_{t}^{u}})$:

(13) $\displaystyle
p_{\theta}(\bm{\hat{e}_{t-1}^{u}}|\bm{\hat{e}_{t}^{u}})=\mathcal{N}(\bm{\hat{e}_{t-1}^{u}};\bm{\mu}_{\theta}(\bm{\hat{e}_{t}^{u}},t),\tilde{\beta}_{t}\bm{I})),\quad\rm
where$ (14)
$\displaystyle{\bm{\mu_{\theta}}}(\bm{\hat{e}_{t}^{u}},t)=\dfrac{\sqrt{\alpha_{t}}(1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_{t}}\bm{\hat{e}_{t}^{u}}+\dfrac{\sqrt{\bar{\alpha}_{t-1}}(1-\alpha_{t})}{1-\bar{\alpha}_{t}}\bm{\tilde{e}_{0}^{u}}.$

$\bm{\tilde{e}_{0}^{u}}$ is the predicted $\bm{e_{0}^{u}}$ since the
distribution of $\bm{e_{0}^{u}}$ is unknown in the reverse process. We employ
the multi-layer perceptron (MLP) to reconstruct $\bm{e_{0}^{u}}$ in the
denoising module.

To ensure tractability in the embedding denoising process, it is paramount to
infuse collaborative information as guidance. Collaborative information, which
is based on user interaction behaviors, can enable the denoising module to
recognize user preferences, subsequently assisting to identify and mitigate
noise. Additionally, the step information also affects the denoising
performance. It provides insight into the current noise level in the
embeddings, offering a gauge on the extent of denoising required at each step.
As such, we use collaborative information $\bm{c_{u}}$ and step embedding as
condition elements, to guide the reconstruction process in the denoising
modules (_cf._ Section 3.2 for the calculation of $\bm{c_{u}}$ in detail).
Specifically, for user embedding $\bm{\hat{e}_{t}^{u}}$ in the reverse step
$t$, the user reconstruction MLP yields:

(15)
$\displaystyle\bm{\tilde{e}_{0}^{u}}={\bm{f_{\theta}}}(\bm{\hat{e}_{t}^{u}},\bm{c_{u}},t),$

where $\bm{\tilde{e}_{0}^{u}}$ is the predicted $\bm{e_{0}^{u}}$ by the
reconstruction MLP with the parameter $\bm{\theta}$. It is noteworthy that the
step information is encoded through sinusoidal positional encoding (Ho et al.,
2020), and these three inputs are concatenated together to feed into the MLP.

Similarly, for the item denoising module, given item embedding
$\bm{\hat{e}_{t}^{i}}$ in the reverse step $t$, the other item reconstruction
MLP outputs
$\bm{\tilde{e}_{0}^{i}}={\bm{f_{\psi}}}(\bm{\hat{e}_{t}^{i}},\bm{c_{i}},t)$,
where $\bm{c_{i}}$ represent the collaborative information of item $i$, and
$\bm{\tilde{e}_{0}^{i}}$ is the predicted $\bm{e_{0}^{i}}$ by the item
reconstruction MLP with the parameter $\bm{\psi}$.

Generally, in the denoising module, we get predicted original embedding
$\bm{\tilde{e}_{0}^{u}}$ and $\bm{\tilde{e}_{0}^{i}}$ from the user
reconstruction MLP and the item reconstruction MLP respectively, and then
utilize Eq. (13) to get the denoised embeddings for the current step.

Algorithm 1 DDRM Training

1:interaction data $\bm{\bar{D}}$, pre-trained user embedding
$\bm{e_{0}^{u}}$, pre-trained item embedding $\bm{e_{0}^{i}}$, diffusion step
$T$, user reconstruction MLP $\bm{f_{\theta}}$, item reconstruction MLP
$\bm{f_{\psi}}$

2:repeat

3: Sample a batch of interactions $\bm{D}\subset\bm{\bar{D}}$.

4: for all $(u,i,j)\in\bm{D}$ do

5: Sample $t\sim\mathcal{U}(1,T)$,
$\bm{\epsilon_{u}}\sim\mathcal{N}(\bm{0},\bm{I})$,
$\bm{\epsilon_{i}}\sim\mathcal{N}(\bm{0},\bm{I})$;

6: Compute $\bm{e}_{t}^{u}$ given $\bm{e}_{0}^{u}$, $t$, and
$\bm{\epsilon_{u}}$ via $q(\bm{e}_{t}^{u}|\bm{e}_{0}^{u})$ in Eq. (6);

7: Compute $\bm{e}_{t}^{i}$ given $\bm{e}_{0}^{i}$, $t$, and
$\bm{\epsilon_{u}}$ via $q(\bm{e}_{t}^{i}|\bm{e}_{0}^{i})$ in Eq. (7);

8: Reconstruct $\bm{\tilde{e}_{0}^{u}}$ and $\bm{\tilde{e}_{0}^{i}}$ through
$\bm{f_{\theta}}$ and $\bm{f_{\psi}}$;

9: Calculate $\mathcal{L}_{\text{final}}$ by Eq. (22);

10: Take gradient descent step on
$\nabla_{\theta}(\mathcal{L}_{\text{final}})$ to optimize $\theta$;

11: Take gradient descent step on$\nabla_{\psi}(\mathcal{L}_{\text{final}})$
to optimize $\psi$;

12:until converged

13:optimized $\theta$, $\psi$.

###  3.2. Optimization

$\bullet$ DDRM training. The optimization of DDRM is under the BPR training
setting: given recommendation data with the triplet $(u,i,j)$, item $i$ and
item $j$ are the positive item and negative item of user $u$, respectively.
Please note that we only conduct denoising for user $u$ and positive
interacted item $i$ since denoising negative item $j$ is rendered relatively
insignificant due to the random negative sampling mechanism.

To optimize the embedding denoising process, it is essential to minimize the
variational lower bound of the predicted user and item embeddings. According
to the KL divergence based on the multivariate Gaussian distribution in Eq.
(5), the reconstruction loss of the denoising process within a training
iteration is expressed as:

(16) $\displaystyle\mathcal{L}_{\text{re}}(u,i)$
$\displaystyle=\mathbb{E}_{q}\left[-\log
p_{\bm{\theta}}(\bm{\hat{e}}_{0}^{u})-\log
p_{\bm{\psi}}(\bm{\hat{e}}_{0}^{i})\right]$
$\displaystyle\leq\mathcal{L}^{u}+\mathcal{L}^{i}+\mathcal{L}_{0}^{u}+\mathcal{L}_{0}^{i},\quad\rm
where$ (17) $\small\left\\{\begin{aligned}
\mathcal{L}^{u}&=\textstyle\sum_{t=2}^{T}\mathbb{E}_{q}\left[\dfrac{1}{2}\left(\dfrac{\bar{\alpha}_{t-1}}{1-\bar{\alpha}_{t-1}}-\dfrac{\bar{\alpha}_{t}}{1-\bar{\alpha}_{t}}\right)||\bm{e_{0}^{u}}-\hat{\bm{f_{\theta}}}(\bm{e_{t}^{u}},\bm{e_{i}},t)||^{2}_{2}\right],\\\
\mathcal{L}^{i}&=\textstyle\sum_{t=2}^{T}\mathbb{E}_{q}\left[\dfrac{1}{2}\left(\dfrac{\bar{\alpha}_{t-1}}{1-\bar{\alpha}_{t-1}}-\dfrac{\bar{\alpha}_{t}}{1-\bar{\alpha}_{t}}\right)||\bm{e_{0}^{i}}-\hat{\bm{f_{\psi}}}(\bm{e_{t}^{i}},\bm{e_{u}},t)||^{2}_{2}\right],\\\
\mathcal{L}_{0}^{u}&=\mathbb{E}_{q}\left[\parallel\bm{e_{0}^{u}}-\hat{\bm{f_{\theta}}}(\bm{e_{1}^{u}},\bm{e_{i}},1)\parallel_{2}^{2}\right],\\\
\mathcal{L}_{0}^{i}&=\mathbb{E}_{q}\left[\parallel\bm{e_{0}^{i}}-\hat{\bm{f_{\theta}}}(\bm{e_{1}^{i}},\bm{e_{u}},1)\parallel_{2}^{2}\right],\\\
\end{aligned}\right.$

where $\bm{e_{i}}$ and $\bm{e_{u}}$ are the original embeddings of user $u$
and item $i$, which serve as the collaborative information $\bm{c_{u}}$ and
$\bm{c_{i}}$, respectively. $\mathcal{L}^{u}$ and $\mathcal{L}^{i}$ are the
user and item reconstruction loss in the reverse process,
$\mathcal{L}_{0}^{u}$ and $\mathcal{L}_{0}^{i}$ are the final prediction loss
correspondingly. From Eq. (18), it is clear that the essence of the DDRM
training lies in optimizing the distance between the reconstructed embedding
derived from MLP and the original embedding.

To reduce the computational cost in the implementation, we simplify Eq. (17)
by uniformly sampling $t$ from $\\{{1,2,...,T}\\}$ instead of summing $T$
steps and removing the weight before the MSE terms:

(18)
$\displaystyle\mathcal{L}_{\text{re}}(u,i)=(\mathcal{L}_{\text{simple}}^{u}+\mathcal{L}_{\text{simple}}^{i})/2,\quad\rm
where$ (19) $\small\left\\{\begin{aligned}
\mathcal{L}_{\text{simple}}^{u}&=\mathbb{E}_{t\sim\mathcal{U}(1,T)}\mathbb{E}_{q}\left[||\bm{e_{0}^{u}}-\hat{\bm{e_{\theta}}}(\bm{e_{t}^{u}},\bm{e_{i}},t)||^{2}_{2}\right],\\\
\mathcal{L}_{\text{simple}}^{i}&=\mathbb{E}_{t\sim\mathcal{U}(1,T)}\mathbb{E}_{q}\left[||\bm{e_{0}^{i}}-\hat{\bm{e_{\psi}}}(\bm{e_{t}^{i}},\bm{e_{u}},t)||^{2}_{2}\right].\\\
\end{aligned}\right.$

$\bullet$ Loss function. The final loss function of DDRM comprises two parts:
a reconstruction loss for the denoising process and a BPR loss for the
recommendation task. The reconstruction loss $\mathcal{L}_{\text{re}}(u,i)$ is
derived from Eq. (18), which regulates the denoising of user and item
embeddings.

After obtaining the denoised user and positive item embeddings via DDRM, these
embeddings contribute to the computation of BPR loss
$\mathcal{L}_{\text{bpr}}$ (He et al., 2020). We design a loss balance factor
$\lambda$ to adjust the weight of these two losses:

(20)
$\displaystyle\mathcal{L}(u,i,j)=\lambda\mathcal{L}_{\text{bpr}}(u,i,j)+(1-\lambda)\mathcal{L}_{\text{re}}(u,i),$

where $i$ and $j$ are positive and negative items for user $u$ in the BPR
training setting. As an extension, we also consider adding a reweighted loss
to supplement DDRM from the perspective of data cleaning (see empirical
evidence of its effectiveness in Section 4.3.2). Specifically, inspired by
(Wang et al., 2021a), we dynamically allocate lower weights to instances with
relatively lower positive scores since they are more likely to be noisy data.

(21) $\displaystyle w(u,i,j)=\text{sigmoid}(s(u,i))^{\gamma},$ (22)
$\displaystyle\mathcal{L}_{\text{final}}(u,i,j)=w(u,i,j)\mathcal{L}(u,i,j),$

where $s(u,i)$ quantifies the score between users $u$ and positive items $i$,
and $\gamma$ is the reweighted factor which controls the range of weights. The
training step of DDRM is illustrated in Algorithm 1.

Algorithm 2 DDRM Inference

1:all users $\bm{\bar{U}}$, diffusion step $T$, inference step $T^{\prime}$,
item reconstruction MLP $\bm{f_{\psi}}$

2:Sample a batch of users $\bm{U}\subset\bm{\bar{U}}$.

3:for all $u\in\bm{U}$ do

4: Sample $\bm{\epsilon_{i}}\sim\mathcal{N}(\bm{0},\bm{I})$;

5: Compute average embeddings of users’ historically liked items
$\bm{\bar{e}_{i}}$ via Eq. (23);

6: Compute $\bm{\bar{e}_{T}^{i}}$ given $\bm{\bar{e}_{0}^{i}}$, $T$, and
$\bm{\epsilon_{i}}$ via $q(\bm{\bar{e}_{T}^{i}}|\bm{\bar{e}_{0}^{i}})$ in Eq.
(7);

7: for $t=T^{\prime},\dots,1$ do

8: Reconstruct $\bm{\hat{e}_{0}^{i}}$ through $\bm{f_{\psi}}$;

9: Compute $\bm{\hat{e}_{t-1}^{i}}$ from $\bm{\hat{e}_{t}^{i}}$ and
$\bm{\hat{e}_{0}^{i}}$ via Eq. (13);

10: Rounding via Eq. (24) to get the ideal item $\bm{e}$;

11:the ideal item $\bm{e}$.

###  3.3. Inference

In the inference phase, we need to utilize trained DDRM to generate ideal
items to do the recommendation task for each user. Typically, the default
method used in image generation entails generating an item from pure noise in
the reverse process, conditional on textual instructions (Croitoru et al.,
2022). However, recommendation data lacks sufficient guidance signals for
conditional generation. Consequently, instead of generating from pure noise,
we take the average embeddings of users’ historically liked items as input
since the interaction information reflects the preferences of users, thereby
guiding the diffusion process. Specifically, we first get the average item
embedding $\bm{\bar{e}_{i}}$:

(23)
$\displaystyle\bm{\bar{e}_{i}}=\frac{1}{n}\textstyle\sum\limits_{i\in\mathcal{I}_{u}}\bm{e_{i}},$

where $n$ is the number of historical interacted items of user $u$, and
$\mathcal{I}_{u}$ is the historical interacted items. Subsequently, we
introduce noise into $\bm{\bar{e}_{i}}$ continuously following the sequence
$\bm{\bar{e}_{0}^{i}}\rightarrow\bm{\bar{e}_{1}^{i}}\rightarrow\cdots\rightarrow\bm{\bar{e}_{T}^{i}}$
in the forward process. And then, we set
$\bm{\hat{e}_{T^{\prime}}^{i}}=\bm{\bar{e}_{T}^{i}}$ to execute the reverse
process by
$\bm{\hat{e}_{T^{\prime}}^{i}}\rightarrow\bm{\hat{e}_{T^{\prime}-1}^{i}}\rightarrow\cdots\rightarrow\bm{\hat{e}_{0}^{i}}$
to generate a new item embedding $\bm{\hat{e}_{0}^{i}}$ conditioned on current
step embedding and user original embedding $\bm{e_{u}}$. It is important to
note that we enhance the denoising process by increasing the reverse step in
comparison to the training phase. This is crucial because the average
embedding of users’ historically liked items tends to contain more noise,
resulting from the aggregation of numerous item embeddings. Following this, to
obtain ideal items for the recommendation task, we develop a rounding function
$s(\bm{\hat{e}_{0}^{i}},\bm{e_{i}})$ that calculates the inner product between
the generated item embeddings $\bm{\hat{e}_{0}^{i}}$ and the candidate item
embeddings to get a similarity score:

(24) $\displaystyle
s(\bm{\hat{e}_{0}^{i}},\bm{e_{i}})=\bm{\hat{e}_{0}^{i}}\cdot{\bm{e_{i}^{T}}},\quad
i\in\mathcal{I}$

where $\mathcal{I}$ is the candidate item pool. Subsequently, we rank the
similarity score and select the top-k candidate items for recommendation. The
inference procedure of DDRM is stated in Algorithm 2.

##  4\. Experiments

In this section, we conduct a comprehensive experimental study to address the
following research questions:

  1. -

RQ1: How does the performance of DDRM compare with other baselines across the
datasets in different experiment settings?

  2. -

RQ2: What is the impact of different components within the DDRM on overall
performance?

  3. -

RQ3: How do variations in DDRM’s backbone design impact its efficacy?

Table 1. Statistics of three datasets under two distinct settings. “#Int.”
denotes interactions numbers. “N” and “R” represent natural noise setting and
random noise setting, respectively.

| #User | #Item (N) | #Int. (N) | #Item (R) | #Int. (R)
---|---|---|---|---|---
Yelp | 54,574 | 77,405 | 1,471,675 | 34,395 | 1,402,736
Amazon-book | 108,822 | 178,181 | 3,145,223 | 94,949 | 3,146,256
ML-1M | 5,949 | 3,494 | 618,297 | 2,810 | 571,531

Table 2. Overall performance of DDRM and other baselines under natural noise
setting. Bold signifies the best performance among the backend models, model-
agnostic methods and DDRM. underline represents the best results in all
compared models. * denotes statistically significant improvements of DDRM over
the backend models, according to the t-tests with a significance level of $p$
¡ 0.01.

| Yelp | Amazon-book | ML-1M
---|---|---|---
Methods | R@10 | R@20 | N@10 | N@20 | R@10 | R@20 | N@10 | N@20 | R@10 | R@20 | N@10 | N@20
CDAE | 0.0305 | 0.0530 | 0.0178 | 0.0246 | 0.0219 | 0.0399 | 0.0122 | 0.0175 | 0.0355 | 0.0675 | 0.0272 | 0.0391
AdaGCL | 0.0464 | 0.0774 | 0.0277 | 0.0371 | 0.0282 | 0.0464 | 0.0166 | 0.0220 | 0.0630 | 0.1200 | 0.0453 | 0.0659
MultiVAE | 0.0484 | 0.0823 | 0.0289 | 0.0391 | 0.0508 | 0.0771 | 0.0300 | 0.0379 | 0.0636 | 0.1229 | 0.0450 | 0.0667
DiffRec | 0.0501 | 0.0847 | 0.0307 | 0.0412 | 0.0537 | 0.0806 | 0.0329 | 0.0411 | 0.0658 | 0.1236 | 0.0488 | 0.0703
MFBPR | 0.0286 | 0.0503 | 0.0176 | 0.0242 | 0.0217 | 0.0379 | 0.0131 | 0.0179 | 0.0445 | 0.0890 | 0.0429 | 0.0582
+T-CE | 0.0316 | 0.0547 | 0.0191 | 0.0261 | 0.0230 | 0.0393 | 0.0136 | 0.0185 | 0.0460 | 0.0866 | 0.0432 | 0.0573
+R-CE | 0.0324 | 0.0554 | 0.0195 | 0.0265 | 0.0227 | 0.0394 | 0.0134 | 0.0184 | 0.0472 | 0.0901 | 0.0434 | 0.0587
+DeCA | 0.0295 | 0.0494 | 0.0181 | 0.0241 | 0.0118 | 0.0188 | 0.0071 | 0.0092 | 0.0451 | 0.0863 | 0.0428 | 0.0851
+DDRM | 0.0354* | 0.0578* | 0.0215* | 0.0282* | 0.0249* | 0.0406* | 0.0148* | 0.0196* | 0.0477* | 0.0916* | 0.0450* | 0.0601*
LightGCN | 0.0502 | 0.0858 | 0.0295 | 0.0403 | 0.0432 | 0.0710 | 0.0251 | 0.0333 | 0.0618 | 0.1193 | 0.0444 | 0.0652
+T-CE | 0.0504 | 0.0856 | 0.0294 | 0.0400 | 0.0421 | 0.0691 | 0.0242 | 0.0323 | 0.0625 | 0.1191 | 0.0457 | 0.0661
+R-CE | 0.0516 | 0.0877 | 0.0304 | 0.0412 | 0.0439 | 0.0723 | 0.0253 | 0.0337 | 0.0623 | 0.1208 | 0.0457 | 0.0668
+DeCA | 0.0486 | 0.0832 | 0.0286 | 0.0390 | 0.0419 | 0.0688 | 0.0242 | 0.0321 | 0.0616 | 0.1202 | 0.0446 | 0.0659
+DDRM | 0.0516* | 0.0870* | 0.0305* | 0.0412* | 0.0468* | 0.0742* | 0.0273* | 0.0355* | 0.0667* | 0.1221* | 0.0508* | 0.0710*
SGL | 0.0485 | 0.0835 | 0.0287 | 0.0393 | 0.0467 | 0.0758 | 0.0267 | 0.0353 | 0.0620 | 0.1164 | 0.0448 | 0.0648
+T-CE | 0.0493 | 0.0840 | 0.0293 | 0.0398 | 0.0483 | 0.0765 | 0.0276 | 0.0361 | 0.0647 | 0.1184 | 0.0470 | 0.0667
+R-CE | 0.0488 | 0.0831 | 0.0289 | 0.0393 | 0.0498 | 0.0772 | 0.0283 | 0.0367 | 0.0651 | 0.1165 | 0.0479 | 0.0670
+DeCA | 0.0476 | 0.0801 | 0.0282 | 0.0380 | 0.0489 | 0.0764 | 0.0285 | 0.0368 | 0.0641 | 0.1183 | 0.0475 | 0.0673
+DDRM | 0.0517* | 0.0860* | 0.0312* | 0.0415* | 0.0535* | 0.0813* | 0.0313* | 0.0396* | 0.0698* | 0.1261* | 0.0530* | 0.0739*

###  4.1. Experimental Settings

####  4.1.1. Datasets.

We evaluate our proposed DDRM on three publicly accessible datasets in
different experiment settings. 1) Yelp111https://www.yelp.com/dataset/. is an
open dataset for recommendation, which contains a large collection of user
reviews and ratings for different restaurants. 2) Amazon-book is one of the
Amazon product review datasets222https://jmcauley.ucsd.edu/data/amazon/.,
covering users’ purchase history and rating scores over books. 3)
ML-1M333https://grouplens.org/datasets/movielens/1m/. is a well-established
benchmark dataset that compiles movie ratings submitted by users. For each
dataset, the interactions with ratings ¡ 4 are regarded as false-positive
interactions.

Following (Wang et al., 2023c), we first arrange the user-item interactions
chronologically based on the timestamps, and then split true-positive
interactions (ratings $\geq$ 4) into training, validation and testing sets
with a ratio of 7:1:2. To evaluate the effectiveness of denoising implicit
feedback, we train and validate the framework on noisy interactions (both
true-positive and false-positive interactions), and test the framework only on
true-positive interactions. Specifically, we explore two types of noisy
settings: natural noise and random noise. While keeping the testing set
containing only true-positive interactions, 1) Natural noise setting
introduces false-positive interactions (ratings ¡ 4) into the original
training and validation sets; 2) Random noise setting randomly samples
unobserved interactions into the original training and validation sets.
Moreover, we ensure that the training and validation sets under the two noisy
settings are at the same scale as the original dataset partition. The
statistics of datasets are shown in Table 1.

####  4.1.2. Baselines.

To demonstrate the efficacy of our proposed DDRM in denoising implicit
feedback, we compare DDRM with the state-of-the-art model-agnostic denoising
methods. In particular,

  * •

R-CE (Wang et al., 2021a) employs a reweight-based denoising strategy that
assumes large-loss interactions are more likely to be noisy and adaptively
allocates lower weights to them.

  * •

T-CE (Wang et al., 2021a), guided by the same assumption as R-CE, directly
eliminates large-loss interactions using a dynamic threshold during training.

  * •

DeCA (Wang et al., 2022) leverages predictions from different models as
denoising signals, under the assumption that different models tend to provide
more consistent predictions for clean data in contrast to noisy ones.

Furthermore, we also compare DDRM with other competitive baselines including
model perspective denoising methods and generative methods:

  * •

AdaGCL (Jiang et al., 2023) is a graph collaborative filtering-based denoising
method that incorporates data augmentation through two adaptive contrastive
view generators.

  * •

CDAE (Wu et al., 2016) introduces random noises to users’ interactions during
training and employs an auto-encoder for denoising.

  * •

MultiVAE (Liang et al., 2018) utilizes variational auto-encoders with
multinomial likelihood to model implicit feedback.

  * •

DiffRec (Wang et al., 2023c) is a diffusion-based generative recommender model
that infers users’ preferences by modeling the interaction probabilities in a
denoising manner.

We implement DDRM and the aforementioned model-agnostic baselines to three
representative backend models.

  * •

MFBPR (Rendle et al., 2009) is a collaborative filtering method based on
matrix factorization with BPR ranking loss.

  * •

LightGCN (He et al., 2020) employs a simple yet effective neighborhood
aggregation scheme on graph convolutional networks for representation
learning.

  * •

SGL (Wu et al., 2021) is a self-supervised learning method for graph
collaborative filtering, which conducts graph data augmentation for robust
representation learning.

Evaluation Metrics. We adopt the full-ranking protocol to evaluate the top-K
recommendation performance using two widely used metrics: Recall@K and NDCG@K
with $K=\\{10,20\\}$.

####  4.1.3. Hyper-parameter Settings.

We fix the embedding size at 64 to maintain fairness during the evaluation of
different methods. For model-agnostic methods, we initially determine the
optimal hyper-parameters of the three backend models on each dataset according
to their default settings. Subsequently, we maintain the backend models’
hyper-parameters at their optimal and adjust only the specific denoising
parameters, as per the original papers. For non-model-agnostic baselines,
hyper-parameters are finely tuned within their default ranges.

Regarding our proposed DDRM, we have seven hyper-parameters in total: the
diffusion steps $T$, the inference steps $T^{\prime}=nT$, the noise lower
bound $\alpha_{\min}$, the noise upper bound $\alpha_{\max}$, the noise scale
$s$ and loss balance factor $\lambda$ and denoising weight factor $\sigma$. In
detail, $T$ and $T^{\prime}=nT$ are tuned within $T=\\{10,20,\dots,60\\}$ and
$n=\\{1,1.25,1.5\\}$. As for the noise-related parameters $\alpha_{\min}$,
$\alpha_{\max}$ and $s$, we explore the combinations in $\\{1e-4,1e-3\\}$,
$\\{1e-3,1e-2\\}$ and $\\{1e-4,1e-3\\}$, respectively. For the loss-related
parameters, loss balance factor $\lambda$ and denoising weight factor $\gamma$
are tuned within $\\{0.1,0.2,\dots,0.6\\}$ and
$\\{0,0.05,0.1,0.2,\dots,0.9\\}$, respectively.

###  4.2. Overall Performance (RQ1)

We conduct comprehensive experiments in natural noise setting to compare
DDRM’s performance with other referenced baselines. The results, illustrated
in Table 2, yield several key observations:

  * •

DDRM mostly outperforms backend models and other model-agnostic denoising
methods across all three datasets. This superior performance can be attributed
to DDRM’s denoising diffusion process, which enhances robust representation
learning through multi-step denoising.

  * •

The performance of DeCA is not consistently better than the backend model. Two
potential reasons emerge: 1) DeCA operates under the presumption that distinct
models yield analogous predictions on clean data but deviate on noisy data.
This assumption may not consistently hold true across our datasets. 2) DeCA’s
training process involves four models optimized concurrently, which
potentially induces instability.

  * •

DiffRec consistently exhibits commendable performance across all three
datasets, thereby highlighting the adeptness of diffusion models in denoising.
DDRM, more flexible than DiffRec owing to its model-agnostic identity, can be
deployed on any recommender model with user and item embeddings. What’s more,
as for DiffRec, it requires to perform prediction tasks on all candidate items
for a given user, resulting in high computational costs. DDRM only
necessitates generating one single ideal item at the embedding level, and
calculating the score between generated item embeddings and candidate item
embeddings, which is more efficient. Furthermore, DDRM bolsters suboptimal
models to perform comparably with, or even surpass, DiffRec, thereby denoting
its tangible enhancements upon the backend model.

###  4.3. In-depth Analysis

Figure 3. Performance comparison of noisy training with random noises on Yelp.

####  4.3.1. Random Noisy Training (RQ1).

We conduct random noisy training to evaluate the noise resistance capability
of DDRM in comparison to both the backend model and R-CE, which is the most
competitive model-agnostic method. The proportion of noise in our training
settings spanned from 0% to 60%. We report the results in Figure 3. The
results show that: 1) As the noise ratio increases, the performance of the
backend model, R-CE, and DDRM all tend to deteriorate. This decline is
attributed to the intensifying corruption of data due to the escalating noise
level, making it challenging to discern genuine user preferences. 2) DDRM
consistently outperforms both the backend model and R-CE in different noise
ratio settings. This emphasizes DDRM’s commendable noise resistance, which can
be attributed to its robust representation learned through the diffusion
process. Similar observations are noted on Amazon-book and ML-1M but the
figures are omitted for brevity.

Figure 4. Contributions of reconstruction loss and reweighted loss to DDRM
compared with backend models.

Figure 5. Contributions of item denoising module to DDRM compared with backend
models.

Figure 6. Hyper-parameter analysis of diffusion step $T$, noise scale $s$,
loss balance factor $\lambda$, and reweighted factor $\gamma$ in MFBPR-based
DDRM for Yelp dataset.

####  4.3.2. Ablation Study (RQ2).

We execute ablation study to analyze DDRM from two distinct angles: loss
perspective and module perspective.

$\bullet$ Loss Perspective. We assess the distinct contributions of the
reconstruction loss from the reverse process and the reweighted loss from the
extension, with outcomes depicted in Figure 4 for Yelp and Amazon-book
(omitting ML-1M due to similar trends). We select the classic model MF and the
graph model SGL which has uniformly commendable performance as our backend
models. Based on these results, we can find that: 1) DDRM with reconstruction
loss consistently outperforms the backend model, underscoring the efficacy of
embedding denoising in DDRM. 2) The performance of DDRM with reweighted loss
varies. In certain scenarios, it underperforms relative to the backend model.
Even when it does show improvements, they are not as pronounced as those from
DDRM with reconstruction loss. This suggests that the underlying assumptions
of data cleaning methods (_cf._ Sec 1) may not always hold true. 3) Overall,
while reconstruction and reweighted losses jointly amplify DDRM’s performance,
the latter’s contributions are notably milder than those of the former.

$\bullet$ Module Perspective. To explore the impact of user denoising module
and item denoising module, we conduct additional experiments on the Yelp and
Amazon-book dataset using only the item denoising module of DDRM, with
performance shown in Figure 5. The results indicate a performance decrease
compared to the full DDRM approach, suggesting that noise is present in user
embeddings and necessitates user denoising. However, the performance with only
item denoising still surpasses that of the backend models alone, which implies
that the item denoising module contributes effectively to noise reduction.
Therefore, it can be inferred that noise exists in both user and item
representations, which prove the effectiveness and necessity of both user
denoising module and item denoising module. It’s important to note that
experiments focusing solely on user denoising were not feasible in our work
since we need to denoise item embeddings to generate ideal items for users in
the inference phase.

Table 3. Performance of different backbone designs in Yelp. Bold signifies the
best performance among the listed models.

Backbone Model | R@10 | R@20 | N@10 | N@20
---|---|---|---|---
SGL | 0.0488 | 0.0841 | 0.0290 | 0.0397
DDRM | 0.0517 | 0.0860 | 0.0312 | 0.0415
DDRM (Transformer) | 0.0469 | 0.0786 | 0.0282 | 0.0378
DDRM (Noise Inference) | 0.0213 | 0.0364 | 0.0133 | 0.0179
DDRM (Dynamic Scale) | 0.0508 | 0.0847 | 0.0305 | 0.0407

####  4.3.3. Backbone Design (RQ3).

We explore various backbone design methods to validate the performance of our
proposed DDRM. The performance of different backbone designs is shown in Table
3, noting that the default backend model is SGL since it has uniformly
competitive performance across three datasets. The detailed analyses of each
backbone design are provided below.

$\bullet$ Transformer. In this variation, we substitute the reconstruction MLP
within the denoising module with the standard Transformer architecture. Our
objective was to investigate whether the inclusion of a more intricate
attention mechanism could bolster performance. Specifically, the collaborative
information and step embedding were designated as the query and key in the
Transformer, respectively, while the embedding requiring denoising was
assigned as the value (Vaswani et al., 2017). However, the results revealed
the Transformer’s inferiority to the MLP in this context. A likely rationale
for this outcome lies in the inductive bias introduced by the additional model
structures in the Transformer, which is inappropriate in the denoising
context, thus rendering suboptimal performance in reconstructing these
embeddings. In contrast, the simplicity of the MLP proves to be highly
efficient for the task at hand.

$\bullet$ Noise Inference. As inspired by (Li et al., 2023), another approach
we explored is denoising the embedding exclusively from pure noise instead of
noisy average embeddings $\bm{\bar{e}_{T}^{i}}$ (_cf._ Section 3.3) during the
inference phase. This method highly depends on diffusion’s impressive
generation capabilities. Nonetheless, this leads to a substantial decline in
performance. A potential explanation is that recommendation tasks necessitate
the guidance of interaction data to steer the reverse generation process.
Starting with pure noise seems to make this process unregulated and undermine
the ability to learn effective representations.

$\bullet$ Dynamic Scale. We further implement dynamic noise scales $s$ for
distinct user and item embeddings during the forward process. The primary
objective behind this maneuver is to evaluate whether incorporating variable
noise scales might enhance the diversity of the data’s noise profile, thereby
facilitating the model to derive more resilient representations in the reverse
phase. However, we observe a performance decline, suggesting that the dynamic
scales will potentially destabilize the model’s learning process. As
previously noted, while the step information $t$ signifies the current noise
level of embeddings, the introduction of a dynamic noise scale disrupts this
consistency; thus, it becomes intricate for the denoising module to utilize
the step information effectively in assessing the current noise level for
denoising execution.

####  4.3.4. Hyper-parameters Analysis (RQ2).

For a more nuanced understanding, we select certain sensitive hyper-
parameters, adjusting them within the ranges delineated in Section 4.1. The
outcomes of these experiments are visually represented in Figure 6. From our
observations: 1) With an increase in the diffusion step $T$ and noise scale
$s$, DDRM’s performance initially rises. This is attributed to an enhanced
noise diversity in the data, enabling the model to foster more robust
representations. Nonetheless, overly extending the diffusion steps and noise
scale adversely affects performance, compromising the model’s personalization
capabilities. Hence, it becomes imperative to judiciously determine the
optimal diffusion step and noise scale to harness peak performance. 2) The
loss balance factor $\lambda$ influences DDRM performance by mediating the
focus between recommendation and reconstruction tasks. While an increase in
$\lambda$ initially bolsters performance by prioritizing embedding denoising,
too high a value risks neglecting the core recommendation task, undermining
overall performance. 3) Appropriate selection of $\gamma$ is crucial for
effective denoising as higher values may filter out clean samples, while lower
values may not comprehensively filter noisy samples.

##  5\. Related Work

$\bullet$ Denoising Implicit Feedback. Addressing the noise in implicit
feedback, caused by false-positive interactions, has seen approaches primarily
from data cleaning perspective (Wang et al., 2021a, 2022; Gantner et al.,
2011) and model perspective (Wu et al., 2016; Jiang et al., 2023). Data
cleaning methods often rely on specific assumptions to directly eliminate
noisy interactions(_e.g.,_ WBPR (Gantner et al., 2011)) or assign lower
weights to such samples(_e.g.,_ T-CE (Wang et al., 2021a), R-CE (Wang et al.,
2021a), DeCA (Wang et al., 2022)), resulting in limited adaptability across
different backend models and datasets. Model perspective methods, on the other
hand, focus on enhancing the noise resistance of recommender model.
Specifically, some graph-based models (_e.g.,_ SGL (Wu et al., 2021), AdaGCL
(Jiang et al., 2023)) employ data augmentation to enhance data diversity and
then regulate models to learn more robust representation. Additionally, some
auto-encoder based models (_e.g.,_ CDAE (Wu et al., 2016)) intentionally
corrupt users’ interactions by introducing different types of noise during
training, and then attempt to reconstruct the original clean data using simple
auto-encoders. However, these model perspective methods expect models to
capture the complex noise distribution in the real world, which might pose a
considerable challenge for the neural networks. Diffusion models have excelled
in various fields, particularly in high-quality generation and multi-step
denoising, revealing substantial robust representation learning potential.
Thus, incorporating diffusion models to denoise implicit feedback is valuable
to further strengthen representation learning robustness.

$\bullet$ Generative Recommendation. Generative models, notably Generative
Adversarial Networks (GANs) (Wang et al., 2017; Gao et al., 2021; Jin et al.,
2020) and Variational Autoencoders (VAEs) (Ma et al., 2019; Liang et al.,
2018; Zhang et al., 2017), are pivotal for personalized recommendations but
face structural limitations (Sohl-Dickstein et al., 2015; Kingma et al.,
2016). Emergent diffusion models, providing enhanced stability and
representation compared with GANs and VAEs, have been recently explored in
recommendation contexts (Wu et al., 2019, 2022). While models like CODIGEM
(Walker et al., 2022) and DiffRec (Wang et al., 2023c) utilize diffusion
models for inferring user preferences by modeling the distribution of users’
interaction probabilities, other research (Li et al., 2023; Wang et al.,
2023b; Du et al., 2023; Liu et al., 2023) targets content generation at the
embedding level, akin to our DDRM. For example, DiffuRec (Li et al., 2023) and
CDDRec (Wang et al., 2023b) corrupt target item representations into pure
noise in the forward process, subsequently reconstructing them condition on
users’ historical interaction sequences. DiffuASR (Liu et al., 2023) uses
diffusion models to generate new item sequences, mitigating data sparsity
issues. However, these methods primarily concentrate on sequential
recommendation and tend to overlook the presence of natural false-positive
interactions in implicit feedback. In contrast, our proposed DDRM employs
diffusion models to denoise implicit feedback, contributing to more robust
representation learning.

##  6\. Conclusion and Future Work

In this work, we proposed the Denoising Diffusion Recommender Model (DDRM), a
plug-in model to bolster robust representation learning amidst noisy feedback
for existing recommendation models. Given user and item embeddings from any
recommender models, DDRM proactively injects Gaussian noises into the
embeddings and then iteratively removes noise in the reverse process. To guide
the reverse process in a tractable way, we design a denoising module to encode
collaborative information as guidance. During the inference phase, we utilize
the average embeddings of users’ historically liked items to facilitate ideal
item generation. Extensive experiment results demonstrate the superiority of
DDRM compared with other competitive baselines.

This work offers a novel diffusion-based method to denoise representations at
the embedding level. Future enhancements for DDRM may involve: 1) enriching
the denoising module with more complex yet well-designed neural networks, 2)
incorporating various noise types to enhance generative capabilities, and 3)
exploring adaptations of DDRM for the sequential recommendation.

## References

  * (1)
  * Chen et al. (2022) Huiyuan Chen, Yusan Lin, Menghai Pan, Lan Wang, Chin-Chia Michael Yeh, Xiaoting Li, Yan Zheng, Fei Wang, and Hao Yang. 2022.  Denoising self-attentive sequential recommendation. In _RecSys_. ACM, 92–101.
  * Chen et al. (2021) Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. 2021.  AutoDebias: Learning to debias for recommendation. In _SIGIR_. ACM, 21–30.
  * Croitoru et al. (2022) Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah. 2022.  Diffusion models in vision: A survey.  _arXiv:2209.04747_ (2022).
  * Ding et al. (2018) Jingtao Ding, Fuli Feng, Xiangnan He, Guanghui Yu, Yong Li, and Depeng Jin. 2018.  An improved sampler for bayesian personalized ranking by leveraging view data. In _WWW_. ACM, 13–14.
  * Ding et al. (2019) Jingtao Ding, Guanghui Yu, Xiangnan He, Fuli Feng, Yong Li, and Depeng Jin. 2019.  Sampler design for bayesian personalized ranking by leveraging view data.  _TKDE_ (2019), 667–681.
  * Dong et al. (2023) Yushun Dong, Jundong Li, and Tobias Schnabel. 2023.  When Newer is Not Better: Does Deep Learning Really Benefit Recommendation From Implicit Feedback?  , 942–952 pages.
  * Du et al. (2023) Hanwen Du, Huanhuan Yuan, Zhen Huang, Pengpeng Zhao, and Xiaofang Zhou. 2023.  Sequential Recommendation with Diffusion Models.  _arXiv:2304.04541_.
  * Fan et al. (2023) Ziwei Fan, Ke Xu, Zhang Dong, Hao Peng, Jiawei Zhang, and Philip S Yu. 2023.  Graph Collaborative Signals Denoising and Augmentation for Recommendation. In _SIGIR_. ACM, 2037–2041.
  * Gantner et al. (2011) Zeno Gantner, Lucas Drumond, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2011.  Personalized Ranking for Non-Uniformly Sampled Items. In _KDDCUP_. JMLR, 231–247.
  * Gao et al. (2021) Min Gao, Junwei Zhang, Junliang Yu, Jundong Li, Junhao Wen, and Qingyu Xiong. 2021.  Recommender systems based on generative adversarial networks: A problem-driven perspective.  _Inf. Sci._ (2021), 1166–1185.
  * Gao et al. (2022) Yunjun Gao, Yuntao Du, Yujia Hu, Lu Chen, Xinjun Zhu, Ziquan Fang, and Baihua Zheng. 2022.  Self-guided learning to denoise for robust recommendation. In _SIGIR_. ACM, 1412–1422.
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020.  Lightgcn: Simplifying and powering graph convolution network for recommendation. In _SIGIR_. 639–648.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising diffusion probabilistic models. In _NeurIPS_. Curran Associates, Inc., 6840–6851.
  * Hoogeboom et al. (2021) Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. 2021.  Argmax flows and multinomial diffusion: Learning categorical distributions. Curran Associates, Inc., 12454–12465.
  * Huang et al. (2023) Lei Huang, Hengtong Zhang, Tingyang Xu, and Ka-Chun Wong. 2023.  Mdm: Molecular diffusion model for 3d molecule generation. In _AAAI_. AAAI press, 5105–5112.
  * Jiang et al. (2023) Yangqin Jiang, Chao Huang, and Lianghao Huang. 2023.  Adaptive Graph Contrastive Learning for Recommendation. In _KDD_. ACM, 4252–4261.
  * Jin et al. (2020) Binbin Jin, Defu Lian, Zheng Liu, Qi Liu, Jianhui Ma, Xing Xie, and Enhong Chen. 2020.  Sampling-decomposable generative adversarial recommender. In _NeurIPS_. Curran Associates, Inc., 22629–22639.
  * Kingma et al. (2016) Durk P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. 2016.  Improved variational inference with inverse autoregressive flow. In _NeurIPS_. Curran Associates, Inc., 4743–4751.
  * Li et al. (2022) Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B Hashimoto. 2022.  Diffusion-lm improves controllable text generation.  _arXiv:2205.14217_.
  * Li et al. (2023) Zihao Li, Aixin Sun, and Chenliang Li. 2023.  DiffuRec: A Diffusion Model for Sequential Recommendation.  _arXiv:2304.00686_.
  * Liang et al. (2018) Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018.  Variational Autoencoders for Collaborative Filtering. In _WWW_. ACM, 689–698.
  * Lin et al. (2023) Weilin Lin, Xiangyu Zhao, Yejing Wang, Yuanshao Zhu, and Wanyu Wang. 2023.  AutoDenoise: Automatic Data Instance Denoising for Recommendations. In _WWW_. ACM, 1003–1011.
  * Liu et al. (2023) Qidong Liu, Fan Yan, Xiangyu Zhao, Zhaocheng Du, Huifeng Guo, Ruiming Tang, and Feng Tian. 2023.  Diffusion Augmentation for Sequential Recommendation.  _arXiv:2309.12858_.
  * Luo (2022) Calvin Luo. 2022.  Understanding diffusion models: A unified perspective.  _arXiv:2208.11970_.
  * Ma et al. (2019) Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019.  Learning Disentangled Representations for Recommendation. In _NeurIPS_. Curran Associates, Inc., 5712–5723.
  * Quan et al. (2023) Yuhan Quan, Jingtao Ding, Chen Gao, Lingling Yi, Depeng Jin, and Yong Li. 2023.  Robust Preference-Guided Denoising for Graph based Social Recommendation. In _WWW_. ACM, 1097–1108.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian personalized ranking from implicit feedback. In _UAI_. AUAI Press, 452–461.
  * Ruiz et al. (2023) Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. 2023.  Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In _CVPR_. IEEE, 22500–22510.
  * Sohl-Dickstein et al. (2015) Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015.  Deep unsupervised learning using nonequilibrium thermodynamics. In _ICML_. PMLR, 2256–2265.
  * Tian et al. (2022) Changxin Tian, Yuexiang Xie, Yaliang Li, Nan Yang, and Wayne Xin Zhao. 2022.  Learning to denoise unreliable interactions for graph collaborative filtering. In _SIGIR_. ACM, 122–132.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Vignac et al. (2022) Clement Vignac, Igor Krawczuk, Antoine Siraudin, Bohan Wang, Volkan Cevher, and Pascal Frossard. 2022.  Digress: Discrete denoising diffusion for graph generation. In _ICLR_.
  * Walker et al. (2022) Joojo Walker, Ting Zhong, Fengli Zhang, Qiang Gao, and Fan Zhou. 2022.  Recommendation via Collaborative Diffusion Generative Model. In _KSEM_. Springer, 593–605.
  * Wang et al. (2017) Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017.  Irgan: A minimax game for unifying generative and discriminative information retrieval models. In _SIGIR_. ACM, 515–524.
  * Wang et al. (2021a) Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021a.  Denoising implicit feedback for recommendation. In _WSDM_. ACM, 373–381.
  * Wang et al. (2023c) Wenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, and Tat-Seng Chua. 2023c.  Diffusion Recommender Model. In _SIGIR_. ACM, 832–841.
  * Wang et al. (2023b) Yu Wang, Zhiwei Liu, Liangwei Yang, and Philip S. Yu. 2023b.  Conditional Denoising Diffusion for Sequential Recommendation.  _arXiv:2304.11433_.
  * Wang et al. (2022) Yu Wang, Xin Xin, Zaiqiao Meng, Joemon M Jose, Fuli Feng, and Xiangnan He. 2022.  Learning Robust Recommenders through Cross-Model Agreement. In _WWW_. ACM, 2015–2025.
  * Wang and Chen (2023) Zhenlei Wang and Xu Chen. 2023.  Robust Recommendation with Adversarial Gaussian Data Augmentation. In _WWW_. ACM, 897–905.
  * Wang et al. (2023a) Zongwei Wang, Min Gao, Wentao Li, Junliang Yu, Linxin Guo, and Hongzhi Yin. 2023a.  Efficient Bi-Level Optimization for Recommendation Denoising. In _SIGKDD_. ACM, 2502–2511.
  * Wang et al. (2021b) Zitai Wang, Qianqian Xu, Zhiyong Yang, Xiaochun Cao, and Qingming Huang. 2021b.  Implicit feedbacks are not always favorable: Iterative relabeled one-class collaborative filtering against noisy interactions. In _MM_. ACM, 3070–3078.
  * Wu et al. (2021) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021.  Self-Supervised Graph Learning for Recommendation. In _SIGIR_. ACM, 726–735.
  * Wu et al. (2022) Le Wu, Junwei Li, Peijie Sun, Richang Hong, Yong Ge, and Meng Wang. 2022.  DiffNet++: A Neural Influence and Interest Diffusion Network for Social Recommendation.  _TKDE_ 34, 10 (2022), 4753–4766.
  * Wu et al. (2019) Le Wu, Peijie Sun, Yanjie Fu, Richang Hong, Xiting Wang, and Meng Wang. 2019.  A neural influence diffusion model for social recommendation. In _SIGIR_. 235–244.
  * Wu et al. (2016) Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. 2016.  Collaborative denoising auto-encoders for top-n recommender systems. In _WSDM_. ACM, 153–162.
  * Yang et al. (2022) Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. 2022.  Diffusion models: A comprehensive survey of methods and applications.  _Comput. Surveys_ (2022).
  * Yu and Qin (2020) Wenhui Yu and Zheng Qin. 2020.  Sampler design for implicit feedback data by noisy-label robust learning. In _SIGIR_. ACM, 861–870.
  * Yuan et al. (2023) Wei Yuan, Chaoqun Yang, Quoc Viet Hung Nguyen, Lizhen Cui, Tieke He, and Hongzhi Yin. 2023.  Interaction-level membership inference attack against federated recommender systems.  (2023), 1053–1062.
  * Zhang et al. (2023a) Chi Zhang, Rui Chen, Xiangyu Zhao, Qilong Han, and Li Li. 2023a.  Denoising and Prompt-Tuning for Multi-Behavior Recommendation. In _WWW_. ACM, 1355–1363.
  * Zhang et al. (2023b) Shengyu Zhang, Tan Jiang, Kun Kuang, Fuli Feng, Jin Yu, Jianxin Ma, Zhou Zhao, Jianke Zhu, Hongxia Yang, Tat-Seng Chua, et al. 2023b.  SLED: Structure Learning based Denoising for Recommendation.  _TOIS_ (2023).
  * Zhang et al. (2017) Shuai Zhang, Lina Yao, and Xiwei Xu. 2017.  Autosvd++ an efficient hybrid collaborative filtering model via contractive auto-encoders. In _SIGIR_. ACM, 957–960.