# Collaborative Filtering Based on Diffusion Models:
Unveiling the Potential of High-Order Connectivity

Yu Hou  1234-5678-9012 Yonsei UniversitySeoulRepublic of Korea
<EMAIL_ADDRESS>,  Jin-Duk Park  Yonsei UniversitySeoulRepublic of Korea
<EMAIL_ADDRESS>and  Won-Yong Shin  Yonsei UniversitySeoulRepublic of
Korea<EMAIL_ADDRESS>
(2024)

###### Abstract.

A recent study has shown that diffusion models are well-suited for modeling
the generative process of user–item interactions in recommender systems due to
their denoising nature. However, existing diffusion model-based recommender
systems do not explicitly leverage high-order connectivities that contain
crucial collaborative signals for accurate recommendations. Addressing this
gap, we propose CF-Diff, a new diffusion model-based collaborative filtering
(CF) method, which is capable of making full use of collaborative signals
along with multi-hop neighbors. Specifically, the forward-diffusion process
adds random noise to user–item interactions, while the reverse-denoising
process accommodates our own learning model, named cross-attention-guided
multi-hop autoencoder (CAM-AE), to gradually recover the original user–item
interactions. CAM-AE consists of two core modules: 1) the attention-aided AE
module, responsible for precisely learning latent representations of user–item
interactions while preserving the model’s complexity at manageable levels, and
2) the multi-hop cross-attention module, which judiciously harnesses high-
order connectivity information to capture enhanced collaborative signals.
Through comprehensive experiments on three real-world datasets, we demonstrate
that CF-Diff is (a) Superior: outperforming benchmark recommendation methods,
achieving remarkable gains up to 7.29% compared to the best competitor, (b)
Theoretically-validated: reducing computations while ensuring that the
embeddings generated by our model closely approximate those from the original
cross-attention, and (c) Scalable: proving the computational efficiency that
scales linearly with the number of users or items.

Collaborative filtering; cross-attention; diffusion model; high-order
connectivity; recommender system.

††journalyear: 2024††copyright: rightsretained††conference: Proceedings of the
47th International ACM SIGIR Conference on Research and Development in
Information Retrieval; July 14–18, 2024; Washington, DC, USA††booktitle:
Proceedings of the 47th International ACM SIGIR Conference on Research and
Development in Information Retrieval (SIGIR ’24), July 14–18, 2024,
Washington, DC, USA††doi: 10.1145/3626772.3657742††isbn:
979-8-4007-0431-4/24/07††ccs: Information systems Recommender systems

##  1\. Introduction

Diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020) have become
one of recent emerging topics thanks to their state-of-the-art performance in
various domains, including computer vision (Ho et al., 2020; Rombach et al.,
2022; Dhariwal and Nichol, 2021), natural language processing (Austin et al.,
2021; Savinov et al., 2022), and multi-modal deep learning (Avrahami et al.,
2022; Saharia et al., 2022). Diffusion models, categorized as deep generative
models, gradually perturb the input data by adding random noise in the
forward-diffusion process and then recover the original input data by learning
in the reverse-denoising process, step by step. Due to their denoising nature,
diffusion models align well with recommender systems, which can be viewed as a
denoising process because user–item historical interactions are naturally
noisy and diffusion models can learn to recover the original interactions
based on corrupted ones (Wu et al., 2016; Li and She, 2017; Wang et al.,
2023). Recent efforts have verified the effectiveness of diffusion models for
sequential recommendations (Yang et al., 2023; Liu et al., 2023; Wu et al.,
2023; Li et al., 2023), where the process of modeling sequential item
recommendations mirrors the step-wise process of diffusion models. However,
the application of diffusion models to recommender systems has yet been
largely underexplored.

On one hand, one of the dominant techniques used in recommender systems is
collaborative filtering (CF), where attention has been paid to model-based
approaches including matrix factorization (MF) (Koren et al., 2009; Yang et
al., 2018) and deep learning (He et al., 2017; Liang et al., 2018; Zhou et
al., 2018; Wang et al., 2019; He et al., 2020; Park et al., 2023) (e.g., graph
neural networks (GNNs) (Wang et al., 2019; He et al., 2020; Park et al.,
2023)). CF-based recommender systems have achieved great success in many real-
world applications, due to their simplicity, efficiency, and effectiveness,
while aiming to learn multi-hop relationships among users and items. For
example, the message passing mechanism in GNNs, being increasingly used in the
tasks of recommendation, captures collaborative signals in high-order
connectivities by aggregating features of neighbors. Figure 1a illustrates the
multi-hop neighbors used for CF with an example involving two users. It is
seen that, although User 1 and User 3 have different direct interactions, they
share similar 2-hop (User 2) and 3-hop (Item 2, Item 5) neighbors, which
implies that User 1 (resp. User 3) is highly likely to prefer Item 4 consumed
by User 3 (resp. Item 1 and Item 3 consumed by User 1).

On the other hand, unlike the existing CF techniques using MF and GNNs, it is
not straightforward to grasp how to exploit such high-order connectivity
information from a diffusion model’s perspective, as shown in Figure 1b.
Recent studies on diffusion model-

(a) Multi-hop neighbors

(b) New challenges

Figure 1. Illustration showing (a) neighbors of User 1 and User 3 up to 3 hops
and (b) how such high-order connectivity information can be potentially
encoded and infused into the diffusion model-based learning system. Here,
$\left\\{{{\bf{u}}_{0},\cdots,{\bf{u}}_{T}}\right\\}$ are the encoded
information of direct user–item interactions at each step, and ${\bf
u}^{\prime}$ is the encoded high-order connectivity information.

based recommender systems (Walker et al., 2022; Wang et al., 2023; Yang et
al., 2023; Liu et al., 2023; Wu et al., 2023; Li et al., 2023) often
overlooked the exploration of multi-hop similarity/proximity among nodes,
albeit the core mechanism of CF in achieving satisfactory performance. In this
context, even with recent attempts to develop recommender systems via
diffusion models (Walker et al., 2022; Wang et al., 2023; Yang et al., 2023;
Liu et al., 2023; Wu et al., 2023; Li et al., 2023), a natural question
arising is: “how can high-order connectivity information be efficiently and
effectively incorporated into recommender systems based on diffusion models?”.
To answer this question, we would like to outline the following two design
challenges:

  * •

C1. how to ensure the complexity of the learning model (to be designed) at an
acceptable level even when including high-order connectivity information;

  * •

C2. how to judiciously link the high-order connectivity information with the
direct user–item interactions under a diffusion-model framework.

It is worth noting that leveraging direct user–item interactions (i.e., direct
neighbors) of each individual is rather straightforward so that diffusion
models can learn the distribution of these interactions (see, e.g., (Wang et
al., 2023) for such an attempt). However, the exploration of high-order
collaborative signals among users and items inevitably poses technical
challenges. First, the infusion of high-order connectivity information may
lead to an increased memory and computational burden, as training diffusion
models is known to be quite expensive in terms of space and time (Sohl-
Dickstein et al., 2015; Ho et al., 2020). This complexity issue will be severe
with an increasing number of users and items. Second, injecting high-order
connectivities in an explicit manner into a learning system within a
diffusion-model framework is technically abstruse. As shown in Figure 1b,
while direct user–item interactions can be readily fed to the diffusion model-
based learning system, the accommodation of high-order collaborative signals
necessitates a complex and challenging integration task.

To address these aforementioned challenges, we make the first attempt towards
developing a lightweight CF method based on diffusion models, named CF-Diff.

(Idea 1) The proposed CF-Diff method naturally involves two distinct
processes, the forward-diffusion process and the reverse-denoising process.
The forward-diffusion process gradually adds random noise to the individual
user–item interactions, while the reverse-denoising process aims to gradually
recover these interactions by infusing high-order connectivities, achieved
through our proposed learning model to be specified later.

(Idea 2) As one of our main contributions, we next design an efficient yet
effective learning model for the reverse-denoising process, dubbed cross-
attention-guided multi-hop autoencoder (CAM-AE), which is capable of infusing
and learning high-order connectivities without incurring additional
computational costs and scalability issues. Our CAM-AE model consist of three
primary parts: a high-order connectivity encoder, an attention-aided AE
module, and a multi-hop cross-attention module. First, we initially pre-
process the user–item interactions in the sense of extracting and encoding
‘per-user’ connectivity information from pre-defined multi-hop neighboring
nodes. Next, we incorporate the attention-aided AE module into CAM-AE to
precisely learn latent representations of the noisy user–item interactions
while preserving the model’s complexity at manageable levels by controlling
the dimension of latent representations (solving the challenge C1). Lastly,
inspired by conditional diffusion models (Rombach et al., 2022), we
incorporate the multi-hop cross-attention module into CAM-AE since high-order
connectivity information can be seen as a condition for denoising the original
user–item interactions. This module takes advantages of the conditional nature
of these connectivities while connecting with the direct user–item
interactions in the reverse-denoising process, thereby enriching the
collaborative signal (solving the challenge C2).

Our main contributions are summarized as follows:

  * •

Novel methodology: We propose CF-Diff, a novel diffusion model-based CF method
featuring our specially designed learning model, CAM-AE. This model is
composed of 1) the encoder of high-order connectivity information, 2) the
attention-aided AE module primarily designed for preserving the model’s
complexity at manageable levels, and 3) the multi-hop cross-attention module
for accommodating high-order connectivity information.

  * •

Extensive evaluations: Through comprehensive experimental evaluations on three
real-world benchmark datasets, including two large-scale datasets, we
demonstrate (a) the superiority of CF-Diff, showing substantial gains up to
7.29% in terms of NDCG@10 compared to the best competitor, (b) the
effectiveness of core components in CAM-AE, and (c) the impact of multi-hop
neighbors in CF-Diff.

  * •

Theoretical findings: We theoretically prove that (a) our learning model’s
embeddings closely approximate those from the (computationally more expensive)
original cross-attention, and (b) the model’s computational complexity scales
linearly with the maximum between the number of users and the number of items.
This is further supported by empirical verifications, confirming the
scalability of CF-Diff.

##  2\. Methodology

###  2.1. Notations

Let $u\in\mathcal{U}$ and $i\in\mathcal{I}$ denote a user and an item,
respectively, where $\mathcal{U}$ and $\mathcal{I}$ denote the sets of all
users and all items, respectively. Historical interactions of a user
$u\in\mathcal{U}$ with items are represented as a binary vector ${\bf
u}\in\left\\{{0,1}\right\\}^{\left|\mathcal{I}\right|}$ whose $i$-th entry is
1 if there exists implicit feedback (such as a click or a view) between user
$u$ and item $i\in\mathcal{I}$, and 0 otherwise.111The unbolded $u$ represents
a user, while the bolded ${\bf u}$ represents a certain user’s interaction
vector as utilized in the proposed method.

###  2.2. Overview of CF-Diff

We describe the methodology of CF-Diff, a new diffusion model-based CF method
that is capable of reflecting high-order connectivity information, revealing
co-preference patterns between users and items, for accurate recommendations.
We recall that recent recommendation methods using diffusion models (Walker et
al., 2022; Wang et al., 2023; Yang et al., 2023; Liu et al., 2023; Wu et al.,
2023; Li et al., 2023) focus primarily on leveraging only the direct user–item
interactions and overlook the collaborative signal in high-order
connectivities during training. Our study aims to fill this gap by infusing
high-order connectivity information into the proposed method, which poses two
main design challenges that we mentioned earlier: preserving the learning
model’s complexity at an acceptable level (C1) and learning complex high-order
connectivities at a fine-grained level (C2).

Figure 2. The schematic overview of CF-Diff when both $2$-hop and $3$-hop
neighboring nodes are taken into account.

To tackle these challenges, as a core module of CF-Diff, we develop an
innovative learning model, CAM-AE. In the CAM-AE model, we propose to use a
multi-hop cross-attention mechanism to infuse multi-hop neighborhood
information from the target user during training, thereby enriching the
collaborative signal, which however causes additional computational costs. To
counter this, we next employ an attention-aided AE module, enabling to
preserving the model’s complexity at manageable levels.

Note that diffusion models can be viewed as partitioning the denoising process
of an AE into a series of finer sub-processes (Deja et al., 2022; Huang et
al., 2021), which can capture more delicate recovery details. Since CF-Diff is
built upon such diffusion models, it naturally involves two distinct
processes, namely the forward-diffusion process and the reverse-denoising
process, achieved with a tailored neural network architecture in CAM-AE. The
schematic overview of the CF-Diff method is illustrated in Figure 2, and each
process in CF-Diff is summarized as follows.

  1. (1)

Forward-diffusion process (Section 2.3): The forward diffusion, aligning with
standard diffusion models, gradually adds Gaussian noise to the user–item
historical interactions, as shown in the upper left part of Figure 2.

  2. (2)

Reverse-denoising process (Section 2.4): We aim to gradually recover the
original user–item interactions from noisy ones. This is achieved by using the
proposed learning model, CAM-AE (to be specified Section 3), which infuses
high-order connectivities to iteratively guide the reverse-denoising process.
To bridge the historical one-hop interactions and multi-hop neighbors, our
CAM-AE model integrates an attention-aided AE with a cross-attention
architecture, progressively recovering user–item interactions by leveraging
high-order connectivity information (see the right part of Figure 2).

###  2.3. Forward-Diffusion Process

We denote the initial state of a specific user $u\in\mathcal{U}$ as
${\bf{u}}_{0}={\bf u}$.222For notational convenience, since each user $u$
experiences the forward-diffusion and reverse-denoising processes
independently, we do not use the user index in $u$ unless it causes any
confusion. In the forward-diffusion process, we gradually insert Gaussian
noise in the initial user–item interactions ${\bf u}_{0}$ over $T$ steps,
producing a sequence of noisy samples ${\bf u}_{1},\ldots,{\bf u}_{T}$,
denoted as ${\bf u}_{1:T}$ (see Figure 2), which can be modeled as

(1)
$q\left({{\bf{u}}_{1:T}\left|{{\bf{u}}_{0}}\right.}\right)=\prod\limits_{t=1}^{T}q\left({{\bf{u}}_{t}\left|{{\bf{u}}_{t-1}}\right.}\right),$

where

(2) $q\left({{\bf u}_{t}\left|{{\bf
u}_{t-1}}\right.}\right)={\mathcal{N}}\left({{\bf
u}_{t};\sqrt{1-\beta_{t}}{\bf u}_{t-1},\beta_{t}{\bf I}}\right)$

represents the transition of adding noise from states ${\bf u}_{t-1}$ to ${\bf
u}_{t}$ via a Gaussian distribution (Sohl-Dickstein et al., 2015; Ho et al.,
2020). Here, $t\in\left\\{{1,\ldots,T}\right\\}$ refers to the diffusion step;
$\mathcal{N}$ denotes the Gaussian distribution; and
$\beta_{t}\in\left({0,1}\right)$ controls the Gaussian noise scales added at
each time step $t$. To generate the noisy sample ${\bf u}_{t}$ from
$q\left({{\bf u}_{t}\left|{{\bf u}_{t-1}}\right.}\right)$, we employ the
reparameterization trick (Kingma and Welling, 2014), expressed as ${\bf
u}_{t}=\sqrt{1-\beta_{t}}{\bf u}_{t-1}+\sqrt{\beta_{t}}\varepsilon_{t-1}$,
where $\varepsilon_{t-1}\sim\mathcal{N}\left({\bf{0},\bf{I}}\right)$. This
process is iteratively applied until we obtain the final sample ${\bf u}_{T}$
at time step $T$.

It is noteworthy that, in contrast to existing diffusion models, our approach
focuses on adding noise to user–item interactions from a single user’s
perspective, which originates from the nature of the denoising process in
variational AE (VAE)-based CF (Li and She, 2017).

###  2.4. Reverse-Denoising Process

In the reverse-denoising process, the estimation of the distribution
$q\left({{\bf u}_{t-1}\left|{{\bf u}_{t}}\right.}\right)$ is technically not
easy as it requires using the entire dataset. Therefore, a neural network
model $p_{\theta}$ is employed to approximate such conditional probabilities
(Ho et al., 2020). Starting from ${\bf u}_{T}$, the reverse-denoising process
gradually recovers ${\bf u}_{t-1}$ from ${\bf u}_{t}$ via the denoising
transition step. However, only relying on user–item interactions do not ensure
the high-quality recovery for CF-based recommendations, as high-order
connectivity information plays an important role in guaranteeing state-of-the-
art performance of CF, as shown in Figure 1a.

To address this, we integrate multi-hop neighbors of the target user $u$
(denoted as ${\bf u}^{\prime}$) into our learning model, thereby enhancing
recommendation accuracies. This differs from original diffusion models, which
focus on denoising solely from noisy samples (i.e., $p_{\theta}\left({{\bf
u}_{t-1}\left|{{\bf u}_{t}}\right.}\right)$ in (Ho et al., 2020)). In other
words, our approach not only denoises from noisy samples but also enriches the
denoising process by exploiting high-order connectivities. The denoising
transition via the Gaussian distribution is formulated as follows (Ho et al.,
2020; Rombach et al., 2022):

(3)
$p_{\theta}\left({{\bf{u}}_{0:T}}\right)=p\left({{\bf{u}}_{T}}\right)\prod\limits_{t=1}^{T}p_{\theta}\left({{\bf{u}}_{t-1}\left|{{\bf{u}}_{t},{\bf{u}}^{\prime}}\right.}\right),$

where

(4) $p_{\theta}\left({{\bf u}_{t-1}\left|{{\bf u}_{t},{\bf
u}^{\prime}}\right.}\right)=\mathcal{N}\left({{\bf
u}_{t-1};\bm{\mu}_{\theta}\left({{\bf u}_{t},{\bf
u}^{\prime},t}\right),\bm{\Sigma}_{\theta}{\left({{\bf u}_{t},{\bf
u}^{\prime},t}\right)}}\right).$

Here, ${\bm{\mu}_{\theta}\left({{\bf u}_{t},{\bf u}^{\prime},t}\right)}$ and
${\bm{\Sigma}_{\theta}{\left({{\bf u}_{t},{\bf u}^{\prime},t}\right)}}$ are
the mean and covariance of the Gaussian distribution predicted by the neural
network with learnable parameters $\theta$. Besides, to maintain training
stability and simplify calculations, we ignore learning of
${\bm{\Sigma}_{\theta}{\left({{\bf u}_{t},{\bf u}^{\prime},t}\right)}}$ in Eq.
(4) and set $\bm{\Sigma}_{\theta}{\left({{\bf u}_{t},{\bf
u}^{\prime},t}\right)}=\beta_{t}{\bf I}$ by following (Ho et al., 2020). After
leaning the mean ${\bm{\mu}_{\theta}\left({{\bf u}_{t},{\bf
u}^{\prime},t}\right)}$ in the model, we can obtain the recovered ${\bf
u}_{t-1}$ by sampling from $p_{\theta}\left({{\bf u}_{t-1}\left|{{\bf
u}_{t},{\bf u}^{\prime}}\right.}\right)$. This process is iteratively applied
until we obtain an estimate of the original sample ${\bf u}_{0}$.

The neural network architecture of CAM-AE is designed in the sense of
judiciously infusing high-order connectivities in the reverse-denoising
process. To this end, CAM-AE consists of two key components: 1) an attention-
aided AE module precisely learns latent representations of the noisy user–item
interactions, helping preserve the complexity manageable (solving the
challenge C1), and 2) a multi-hop cross-attention module, which accommodates
high-order connectivity information to facilitate the reverse-denoising
process, thus capturing the enriched collaborative signal (solving the
challenge C2).

##  3\. Learning Model: CAM-AE

In this section, we elaborate on the proposed CAM-AE model, comprising an
attention-aided AE module and a multi-hop cross-attention module. After
showing how to extract and encode multi-hop neighborhood information for a
given bipartite graph, we describe implementation details of each module in
CAM-AE. We then explain how to optimize our learning model. Finally, we
provide analytical findings, which theoretically validate the efficiency of
CAM-AE.

###  3.1. High-Order Connectivity Encoder

To extract multi-hop neighbors of a given user, we may use a bipartite graph
constructed by establishing edges based on all user–item interactions.
However, using such a bipartite graph will result in a huge memory and
computational burden during training. To solve this practical issue, we pre-
process the user–item interactions in such a way of initially extracting
multi-hop neighbors of a user. This extracted ‘per-user’ connectivity
information is then made available in the reverse-denoising process to assist
recovery of the original user–item interactions (see Figure 2).

Given a target user’s historical interactions u, we explain how to explore
multi-hop neighbors along paths within the user–item bipartite graph. In our
study, we encode high-order connectivity information (i.e., high-order
collaborative signals) up to $H$-hop neighbors as in the following form:

(5)
${\bf{u}}^{\prime}=\left[{{\bf{u}}^{\left(2\right)},\ldots,{\bf{u}}^{\left(H\right)}}\right],$

where

(6) ${\bf u}^{\left(h\right)}=\frac{1}{N_{h-1,h}}{\bf
r}\left({\mathcal{G}\left({u,h}\right),{\bf c}^{(h)}}\right)$

for $h=2,\cdots,H$.333If $h$ is even, then
$\left|{{\bf{u}}^{\left(h\right)}}\right|=\left|\mathcal{U}\right|$.
Otherwise,
$\left|{{\bf{u}}^{\left(h\right)}}\right|=\left|\mathcal{I}\right|$. However,
to tractably handle ${\bf u}^{\prime}$, we can set the dimensionality of each
${\bf u}^{(h)}$ to
$\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}$.
Here, ${\bf r}(\cdot,\cdot)$ is the vector-valued function returning a multi-
hot encoded vector where one is assigned only to the elements corresponding to
$h$-hop neighbors of user $u$; $\mathcal{G}\left({u,h}\right)$ indicates the
set of $h$-hop neighbors of user $u$; ${\bf
c}^{(h)}\in\mathbb{R}^{|\mathcal{G}(u,h)|\times 1}$ is the integer vector,
each of which represents the number of incoming links from $(h-1)$-hop
neighbors of user $u$ to each of $h$-hop neighbors; and $N_{h-1,h}$ is the
total number of interactions between $(h-1)$-hop and $h$-hop neighbors of of
user $u$. Now, let us show an explicit form of encoded $h$-hop neighborhood
information ${\bf u}^{(h)}$ along with the following example.

###### Example 1.

Consider the target user (User 1) in the user–item bipartite graph consisting
of 3 users and 5 items, as illustrated in Figure 3. Here, it follows that
${\bf u}=\left[{1\;1\;0\;0\;1}\right]^{T}$ as User 1 has interacted with Item
1, Item 2, and Item 5. Since the 2-hop neighbors of User 1 are User 2, User 3
and User 3 has two incoming links, we have
${\bf{u}}^{(2)}=[0\;\frac{1}{3}\;\frac{2}{3}]^{T}$ normalized to the total
number of interactions at the second hop. Similarly, we obtain
${\bf{u}}^{(3)}=[0\;0\;\frac{1}{3}\;\frac{2}{3}\;0]^{T}$.

Figure 3. Extraction and encoding of 2-hop and 3-hop neighbors of the target
user (User 1) as well as direct neighbors for a given bipartite graph.

###  3.2. Attention-Aided AE Module

VAE-based CF (Li and She, 2017) shows great potential in capturing underlying
patterns by encoding user–item interactions into a latent space. Similarly, in
the CAM-AE model, we would like to design lightweight encoders to project the
user–item interactions into a latent space, aiming to capture high-level
patterns while keeping the computations manageable by controlling the latent
dimension. This design principle enables us to solve the challenge C1.

In CAM-AE, the attention-added AE module involves hop-specific encoders. As
illustrated in Figure 2, an encoder $\mathcal{E}_{1}\left(\cdot\right)$ is
adopted to project user $u$’s noisy interactions ${\bf u}_{t}$ into a latent
space, represented by the latent embedding ${\bf
z}_{t}\in\mathbb{R}^{{}^{k\times 1}}$ with its dimensionality $k$. Likewise,
another hop-specific encoder $\mathcal{E}_{h}\left(\cdot\right)$ generates
embeddings for the encoded information of $h$-hop neighbors of user $u$,
${{\bf{u}}^{\left(h\right)}}$, yielding ${\bf
z}^{\left(h\right)}\in\mathbb{R}^{{}^{k\times 1}}$. Similarly as in (Wang et
al., 2020), these two encoders $\mathcal{E}_{1}\left(\cdot\right)$ and
$\mathcal{E}_{h}\left(\cdot\right)$ are implemented as linear transformations,
which are formally expressed as

(7) ${{\bf{z}}_{t}={\mathcal{E}}_{1}\left({{\bf{u}}_{t}}\right)={\bf
E}_{1}{{\bf{u}}_{t}},}$ (8)
${{\bf{z}}^{\left(h\right)}={\mathcal{E}}_{h}\left({{\bf{u}}^{\left(h\right)}}\right)={\bf
E}_{h}{{\bf{u}}^{\left(h\right)}},}$

where ${\bf E}_{1}\in\mathbb{R}^{{}^{k\times{\left|\mathcal{I}\right|}}}$ and
${\bf
E}_{h}\in\mathbb{R}^{{}^{k\times{\left|{\bf{u}}^{\left(h\right)}\right|}}}$
represents the transformation matrices. Figure 2 illustrates the case where
the embeddings ${\bf z}^{(2)}$ and ${\bf z}^{(3)}$ of both 2-hop and 3-hop
neighbors of a target user are generated.444Although the example in Figure 2
deals with up to 3-hop neighbors, it is straightforward to extend our module
to the case of leveraging general $h$-hop neighbors. We can preserve the CAM-
AE model’s complexity at manageable levels through these linear
transformations that reduce the dimension of latent representations.

We turn to addressing a decoder $\mathcal{D}\left(\cdot\right)$, which is
adopted to recover the mean value of $p_{\theta}\left({{\bf
u}_{t-1}\left|{{\bf u}_{t},{\bf u}^{\prime}}\right.}\right)$ using the
embedding, denoted as $\bar{\bf z}_{t}$, as input that is returned by the
multi-hop cross-attention module (to be specified in Section 3.3), as depicted
in Figure 2. The decoder is formulated as follows:

(9) $\hat{\bm{\mu}}_{\theta}=\mathcal{D}\left({\bar{\bf z}_{t}}\right)={\bf
D}\bar{\bf z}_{t},$

where ${\bf D}\in\mathbb{R}^{{}^{{\left|\mathcal{I}\right|}\times k}}$ is the
transformation matrix in the decoder. Then, ${\bf u}_{t-1}$ can be sampled
from $p_{\theta}\left({{\bf u}_{t-1}\left|{{\bf u}_{t},{\bf
u}^{\prime}}\right.}\right)=\mathcal{N}\left({{\bf
u}_{t-1};\hat{\bm{\mu}}_{\theta},\beta_{t}{\bf I}}\right)$.

###  3.3. Multi-Hop Cross-Attention Module

The CAM-AE model is enlightened by conditional diffusion models (Rombach et
al., 2022), which achieved impressive success in various fields by using the
cross-attention mechanism (Vaswani et al., 2017) to integrate additional
conditions. In CAM-AE, high-order connectivity information in Eq. (5) can be
regarded as a condition for denoising the original user–item interactions
${\bf u}_{0}$, following the principle of conditional diffusion models
(Rombach et al., 2022). In this study, to effectively infuse high-order
connectivities into our learning model, we propose the multi-hop cross-
attention module. This module judiciously harnesses the conditional nature of
these connectivities while connecting with the direct user–item interactions
in the reverse-denoising process. This design principle is established to
fundamentally solve the challenge C2.

In the multi-hop cross-attention module, we start by expanding the dimension
of ${\bf z}_{t}\in\mathbb{R}^{{}^{k\times 1}}$ and ${\bf
z}^{\left(h\right)}\in\mathbb{R}^{{}^{k\times 1}}$ (i.e., the output
embeddings of encoders $\mathcal{E}_{1}$ and $\mathcal{E}_{h}$) to obtain
${\bf v}_{t}\in\mathbb{R}^{{}^{k\times d}}$ and ${\bf
q}^{\left(h\right)}\in\mathbb{R}^{{}^{k\times d}}$ for improving the
expressiveness. This expansion can be implemented as ${\bf
v}_{t}={\bf{z}}_{t}{\bf E}^{v}$ and ${\bf
q}^{\left(h\right)}={\bf{z}}^{\left(h\right)}{\bf E}^{q}$, where ${\bf
E}^{v}\in\mathbb{R}^{1\times d}$ and ${\bf E}^{q}\in\mathbb{R}^{1\times d}$
are the transformation matrices with $d$ being the expended dimensionality.
Then, the resulting embedding of $h$-hop neighbors of user $u$, ${\bf
q}^{\left(h\right)}$, is integrated into ${\bf v}_{t}$ using the multi-hop
cross-attention module:

(10) $Attention_{h}\left({{\bf Q}^{\left(h\right)},{\bf K}_{t},{\bf
V}_{t}}\right):=softmax\left({\frac{{{\bf Q}^{\left(h\right)}{\bf
K}_{t}^{T}}}{{\sqrt{d}}}}\right){\bf V}_{t},$

where ${\bf Q}^{\left(h\right)}\\!=\\!{\bf q}^{\left(h\right)}{\bf
W}_{\theta}^{Q}$, ${\bf K}_{t}\\!=\\!{\bf v}_{t}{\bf W}_{\theta}^{K}$, and
${\bf V}_{t}\\!=\\!{\bf v}_{t}{\bf W}_{\theta}^{V}$; and
$softmax\left(\cdot\right)$ is the softmax function. Here, $\left\\{{{\bf
W}_{\theta}^{Q},{\bf W}_{\theta}^{K},{\bf
W}_{\theta}^{V}}\right\\}\in\mathbb{R}^{{}^{d\times d}}$ are trainable
parameters. Figure 2 includes the multi-hop cross-attention module (see the
light red blocks in the reverse-denoising process) when 2-hop and 3-hop
neighbors of the target user are taken into account.

Due to the fact that the aforementioned process is basically built upon linear
transformations that lack the ability to capture the intrinsic data
complexity, a per-hop forward operation $f_{h}\left(\cdot\right)$ using non-
linear transformations is applied to $Attention_{h}$ (Vaswani et al., 2017).
We stack $N$ identical layers, each consisting of cross-attention and non-
linear transformation, with the output from the last layer aggregated to form
${\bar{\bf{z}}}_{t}\in\mathbb{R}^{k\times d}$, calculated as

(11) ${\bar{\bf
z}}_{t}=\sum\limits_{h=2}^{H}{\alpha_{h}f_{h}\left({Attention_{h}}\right)},$

where $\sum\limits_{h=2}^{H}{\alpha_{h}=1}$; $\alpha_{h}$ is the weight
balancing among different $f_{h}(\cdot)$’s specific to hop $h$; and $H$ is the
number of hops. Finally, $\bar{\bf{z}}_{t}$ is the input of the decoder
$\mathcal{D}\left(\cdot\right)$ in Eq. (9).

It is worthwhile to note that both ${\bf v}_{t}$ and ${\bf
q}^{\left(h\right)}$ originate from the same user, offering two different
perspectives of the same data. This dual perspective is beneficial for
precisely capturing the collaborative signal. In other words, through the
cross-attention mechanism in CAM-AE, high-order connectivities can
significantly improve the reverse-denoising process, thereby ultimately
enhancing recommendation accuracies.

###  3.4. Optimization

In our learning model, the denoising transition
$p_{\theta}\left({{\bf{u}}_{t-1}\left|{{\bf{u}}_{t},{\bf{u}}^{\prime}}\right.}\right)=\mathcal{N}\left({{\bf{u}}_{t-1};}\right.$
$\left.{{\bm{\mu}}_{\theta}\left({{\bf{u}}_{t},{\bf{u}}^{\prime},t}\right),\beta_{t}{\bf{I}}}\right)$
is forced to approximate the tractable distribution
$q\left({{\bf{u}}_{t-1}\left|{{\bf{u}}_{t},{\bf{u}}_{0}}\right.}\right)=\mathcal{N}\left({{\bf
u}_{t-1};\tilde{\bm{\mu}}\left({{\bf u}_{t},{\bf u}_{0}}\right),\beta_{t}{\bf
I}}\right)$ (note that the mean $\tilde{\bm{\mu}}\left({{\bf u}_{t},{\bf
u}_{0}}\right)$ can be computed via Bayes’ rule as shown in (Ho et al., 2020):
$q\left({{\bf{u}}_{t-1}\left|{{\bf{u}}_{t},{\bf{u}}_{0}}\right.}\right)=q\left({{\bf{u}}_{t}\left|{{\bf{u}}_{t-1},{\bf{u}}_{0}}\right.}\right)\frac{{q\left({{\bf{u}}_{t-1}\left|{{\bf{u}}_{0}}\right.}\right)}}{{q\left({{\bf{u}}_{t}\left|{{\bf{u}}_{0}}\right.}\right)}}$).
Following this approximation, we can generate ${\bf u}_{t-1}$ from ${\bf
u}_{t}$ progressively until ${\bf u}_{0}$ is reconstructed. Figure 2
visualizes a single denoising step from ${\bf u}_{T}$ to ${\bf u}_{T-1}$,
which is repeated $T$ times to obtain ${\bf u}_{0}$.

To optimize the parameter $\theta$, our model aims at minimizing the
variational lower bound (VLB) (Kingma and Welling, 2014; Ho et al., 2020) for
the observed user–item interactions ${\bf u}_{0}$ alongside the following
loss:

(12)
$\mathcal{L}_{\text{VLB}}=\mathcal{L}_{0}+\sum\nolimits_{t=2}^{T}{\mathcal{L}_{t-1}},$

where $\mathcal{L}_{0}=\mathbb{E}_{q}\left[{-\log
p_{\theta}\left({{\bf{u}}_{0}\left|{{\bf{u}}_{1},{\bf{u}}^{\prime}}\right.}\right)}\right]$
is the reconstruction term to recover the original interactions ${\bf u}_{0}$;
and $\mathcal{L}_{t-1}$ is the denoising matching term, regulating
$p_{\theta}\left({{\bf u}_{t-1}\left|{{\bf u}_{t},{\bf
u}^{\prime}}\right.}\right)$ to align with the tractable distribution
$q\left({{\bf{u}}_{t-1}\left|{{\bf{u}}_{t},{\bf{u}}_{0}}\right.}\right)$,
served as the ground truth, and is given by

(13)
$\begin{array}[]{l}\mathcal{L}_{t-1}=\mathbb{E}_{q}\left[{D_{\text{KL}}\left({q\left({{\bf
u}_{t-1}\left|{{\bf u}_{t},{\bf
u}_{0}}\right.}\right)\left\|{p_{\theta}\left({{\bf u}_{t-1}\left|{{\bf
u}_{t},{\bf u}^{\prime}}\right.}\right)}\right.}\right)}\right]\\\
\;\;\;\;\;\;\;\;=\mathbb{E}_{q}\left[{\frac{1}{{2\beta_{t}}}\left[{\left\|{{\bm{\mu}}_{\theta}\left({{\bf
u}_{t},{\bf u}^{\prime},t}\right)-\tilde{\bm{\mu}}\left({{\bf u}_{t},{\bf
u}_{0}}\right)}\right\|^{2}}\right]}\right],\\\ \end{array}$

where $D_{\text{KL}}(\cdot\|\cdot)$ denotes the Kullback–Leibler (KL)
divergence between two distributions.

###  3.5. Theoretical Analyses

In this subsection, we are interested in theoretically showing the efficiency
of the CAM-AE model. In CAM-AE, we use an AE to generate embeddings, reducing
computations to an acceptable level by controlling the embedding dimension
$k$. We first establish the following theorem, which analyzes that the
potential difference incurred by using our low-complexity modules in CAM-AE is
negligibly small compared to the (computationally more expensive) original
cross-attention (Vaswani et al., 2017), defined as

(14) $Attention\left({{\bf Q},{\bf K},{\bf
V}}\right):=softmax\left({\frac{{{\bf Q}{\bf K}^{T}}}{{\sqrt{d}}}}\right){\bf
V},$

where ${\bf Q}={\bf q}{\tilde{\bf W}}_{\theta}^{Q}$, ${\bf K}={\bf
k}{\tilde{\bf W}}_{\theta}^{K}$, and ${\bf V}={\bf v}{\tilde{\bf
W}}_{\theta}^{V}$. Here, ${\bf
q}\\!\in\\!\mathbb{R}^{\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}\\!\times\\!d}$
and $\left\\{{{\bf k},{\bf
v}}\right\\}\\!\in\\!\mathbb{R}^{\left|\mathcal{I}\right|\times d}$ are the
embedding matrices and $\left\\{{{\tilde{\bf W}}_{\theta}^{Q},\\!{\tilde{\bf
W}}_{\theta}^{K},\\!\tilde{{\bf
W}}_{\theta}^{V}}\right\\}\\!\\!\in\\!\mathbb{R}^{{}^{d\times d}}$ are
trainable parameters of $Attention$ in Eq. (14).

###### Theorem 0.

Suppose that
$\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}$ is
sufficiently large. If
$k\geq{\rm{}}{{5\\!\ln\left({\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}}\right)}\mathord{\left/{\vphantom{{5\ln\left({\left|I\right|}\right)}\\!{\left({\varepsilon^{2}-\varepsilon^{3}}\right)}}}\right.\kern-1.2pt}{\left({\varepsilon^{2}-\varepsilon^{3}}\right)}}{\rm{}}$,
then there exist matrices ${\bf
E}_{Q}\in\mathbb{R}^{k\times\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}}$,
${\bf E}_{K},{\bf E}_{V}\in\mathbb{R}^{k\times\left|\mathcal{I}\right|}$ and
${\bf D}\in\mathbb{R}^{\left|\mathcal{I}\right|\times k}$ such that

(15) $\Pr\left({\left|{\sqrt{\left\|{\frac{{{\bf D}\cdot softmax\left({{\bf
E}_{Q}{\bf A}{\bf E}_{K}^{T}}\right){\bf E}_{V}{\bf V}}}{{softmax\left({\bf
A}\right){\bf
V}}}}\right\|}-1}\right|\leq\varepsilon}\right)>1-o\left(1\right),$

where ${\bf
Q}\in\mathbb{R}^{\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}\times
d}$ and $\left\\{{{\bf K},{\bf
V}}\right\\}\in\mathbb{R}^{\left|\mathcal{I}\right|\times d}$ are the
embedding matrices in the original cross-attention; ${\bf A}=\frac{{{\bf
Q}{\bf K}^{T}}}{{\sqrt{d}}}$; and $\varepsilon>0$ is an arbitrarily small
constant.

The proof of Theorem 1 is omitted due to page limitations. Theorem 1 implies
that the probability that the two terms ${{softmax\left({\bf A}\right){\bf
V}}}$ and ${{\bf D}\\!\cdot\\!softmax\left({{\bf E}_{Q}{\bf A}{\bf
E}_{K}^{T}}\right){\bf E}_{V}{\bf V}}$ are approximately equal approaches one
asymptotically when the maximum value of $|\mathcal{U}|$ and $|\mathcal{I}|$
is sufficiently large. We are capable of bridging this theorem and our CAM-AE
model by setting ${\bf E}_{V}{\bf V}={\bf V}_{t}$, ${\bf E}_{K}{\bf K}={\bf
K}_{t}$, ${\bf E}_{Q}{\bf Q}={\bf Q}^{\left(h\right)}$, and ${\bf D}$ as in
Eq. (9), where ${\bf E}_{Q}={\bf E}_{h}$, ${\bf E}_{K}={\bf E}_{1}$, and ${\bf
E}_{V}={\bf E}_{1}$, which leads to the conclusion that the term ${\bf
D}\cdot{softmax\left({{\bf E}_{Q}{\bf A}{\bf E}_{K}^{T}}\right)\\!{\bf
E}_{V}\\!{\bf V}}$ in Eq. (15) is equivalent to ${\bf D}\cdot
Attention_{h}\left({{\bf Q}^{\left(h\right)},{\bf K}_{t},{\bf V}_{t}}\right)$.
From Theorem 1, one can see that the original cross-attention can be
effectively approximated by our low-complexity modules in CAM-AE, which
combines the cross-attention mechanism with linear transformations, thus
significantly reducing the computational complexity (which is to be
empirically validated later). In other words, we can control $k$ to maintain
the amount of computations manageable while ensuring that the embeddings
generated by our model closely approximate those from the original cross-
attention, especially for large
$\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}$.

Additionally, to validate the scalability of the CAM-AE model, we analytically
show its computational complexity during training by establishing the
following theorem.

###### Theorem 0.

The computational complexity of CF-Diff training, including both the
computation time of the forward-diffusion process and the training time of the
reverse-denoising process, is given by
$\mathcal{O}\left({\max\left\\{{\left|\mathcal{U}\right|,\left|\mathcal{I}\right|}\right\\}}\right)$.

The proof of Theorem 2 is omitted due to page limitations. From Theorem 2, one
can see that the computational complexity required to train CF-Diff scales
linearly with the maximum between the number of users and the number of items.
This is because we are capable of considerably reducing the computation of Eq.
(10) (corresponding to the cross-attention part in Figure 2) by controlling
the embedding dimension $k$.

##  4\. Experimental Evaluation

In this section, we systematically conduct extensive experiments to answer the
following five key research questions (RQs):

  * •

RQ1: How much does CF-Diff improve the top-$K$ recommendation over benchmark
recommendation methods?

  * •

RQ2: How does each component in CAM-AE contribute to the recommendation
accuracy?

  * •

RQ3: How many hops in CF-Diff benefit for the recommendation accuracy?

  * •

RQ4: How do key parameters of CAM-AE affect the performance of CF-Diff?

  * •

RQ5: How scalable is CF-Diff when the size of datasets increases?

###  4.1. Experimental Settings

Datasets. We conduct our experiments on three real-world datasets widely
adopted for evaluating the performance of recommender systems, which include
MovieLens-1M (ML-1M)555https://grouplens.org/datasets/movielens/1m/., and two
larger datasets, Yelp666https://www.yelp.com/dataset/. and
Anime777https://www.kaggle.com/datasets/CooperUnion/anime-recommendations-
database.. Table 1 provides a summary of the statistics for each dataset.

Competitors To comprehensively demonstrate the superiority of CF-Diff, we
present nine recommendation methods, including five general benchmark CF
methods (NGCF (Wang et al., 2019), LightGCN (He et al., 2020), SGL (Wu et al.,
2021), NCL (Lin et al., 2022), and BSPM (Choi et al., 2023)) and four
generative model-based recommendation methods (CFGAN (Chae et al., 2018),
MultiDAE (Wu et al., 2016), RecVAE (Shenbin et al., 2020), and DiffRec (Wang
et al., 2023)).

Performance metrics. We follow the full-ranking protocol (He et al., 2020) by
ranking all the non-interacted items for each user. In our study, we adopt two
widely used ranking metrics, Recall@$K$ (R@$K$) and NDCG@$K$ (N@$K$), where
$K\in\left\\{{10,20}\right\\}$.

Implementation details. We use the best hyperparameters of competitors and CF-
Diff obtained by extensive hyperparameter tuning on the validation set. We use
the Adam optimizer (Kingma and Ba, 2015), where the batch size is selected in
the range of $\left\\{{32,64,128,256}\right\\}$. In CF-Diff, the
hyperparameters used in the diffusion model (e.g., the noise scale $\beta_{t}$
and the diffusion step $T$) essentially follow the settings in (Wang et al.,
2023). We choose the best hyperparameters in the following ranges:
$\left\\{{1,2,3,4}\right\\}$ for the number of hops, $H$;
$\left\\{{512,1024,2048}\right\\}$ for the latent dimension $k$ in the
attention-aided AE module; and $\left\\{{16,32,64,128}\right\\}$ for the
expanded dimension $d$, $\left\\{{1,2,3,4}\right\\}$ for the number of layers,
$N$, and $\left\\{{0.3,0.5,0.7}\right\\}$ for $\alpha_{h}$’s in the multi-hop
cross-attention module. All experiments are carried out with Intel (R) 12-Core
(TM) E5-1650 v4 CPUs @ 3.60 GHz and GPU of NVIDIA GeForce RTX 3080. The code
of CF-Diff is available at https://github.com/jackfrost168/CF_Diff.

Table 1. The statistics of three datasets. Dataset | # of users | # of items | # of interactions
---|---|---|---
MovieLens-1M | 5,949 | 2,810 | 571,531
Yelp | 54,574 | 34,395 | 1,402,736
Anime | 73,515 | 11,200 | 7,813,737
Table 2. Performance comparison among CF-Diff and nine recommendation competitors for the three benchmark datasets. Here, the best and second-best performers are highlighted by bold and underline, respectively. | ML-1M | Yelp | Anime
---|---|---|---
Method | R@10 | R@20 | N@10 | N@20 | R@10 | R@20 | N@10 | N@20 | R@10 | R@20 | N@10 | N@20
NGCF | 0.0864 | 0.1484 | 0.0805 | 0.1008 | 0.0428 | 0.0726 | 0.0255 | 0.0345 | 0.1924 | 0.2888 | 0.3515 | 0.3485
LightGCN | 0.0824 | 0.1419 | 0.0793 | 0.0982 | 0.0505 | 0.0858 | 0.0312 | 0.0417 | 0.2071 | 0.3043 | 0.3937 | 0.3824
SGL | 0.0806 | 0.1355 | 0.0799 | 0.0968 | 0.0564 | 0.0944 | 0.0346 | 0.0462 | 0.1994 | 0.2918 | 0.3748 | 0.3652
NCL | 0.0878 | 0.1471 | 0.0819 | 0.1011 | 0.0535 | 0.0906 | 0.0326 | 0.0438 | 0.2063 | 0.3047 | 0.3915 | 0.3819
BSPM | 0.0884 | 0.1494 | 0.0750 | 0.0957 | 0.0565 | 0.0932 | 0.0331 | 0.0439 | 0.2054 | 0.3103 | 0.4355 | 0.4231
CFGAN | 0.0684 | 0.1181 | 0.0663 | 0.0828 | 0.0206 | 0.0347 | 0.0129 | 0.0172 | 0.1946 | 0.2889 | 0.4601 | 0.4289
MultiDAE | 0.0769 | 0.1335 | 0.0737 | 0.0919 | 0.0531 | 0.0876 | 0.0316 | 0.0421 | 0.2142 | 0.3085 | 0.4177 | 0.4125
RecVAE | 0.0835 | 0.1422 | 0.0769 | 0.0963 | 0.0493 | 0.0824 | 0.0303 | 0.0403 | 0.2137 | 0.3068 | 0.4105 | 0.4068
DiffRec | 0.1021 | 0.1763 | 0.0877 | 0.1131 | 0.0554 | 0.0914 | 0.0343 | 0.0452 | 0.2104 | 0.3012 | 0.5047 | 0.4649
CF-Diff | 0.1077 | 0.1843 | 0.0912 | 0.1176 | 0.0585 | 0.0962 | 0.0368 | 0.0480 | 0.2191 | 0.3155 | 0.5152 | 0.4748

###  4.2. Results and Analyses

In RQ1–RQ3, we provide experimental results on all datasets. For RQ4, we show
here only the results on ML-1M in terms of N@$K$ due to space limitations,
since the results on other datasets and metrics showed similar tendencies to
those on ML-1M. Additionally, we highlight the best and second-best performers
in each case of the following tables in bold and underline, respectively.

####  4.2.1. Comparison with nine recommendation competitors (RQ1)

We validate the superiority of CF-Diff over nine recommendation competitors
through extensive experiments on the three datasets. Table 2 summarizes the
results, and we make the following insightful observations.

  1. (1)

Our CF-Diff consistently and significantly outperforms all recommendation
competitors regardless of the datasets and the performance metrics.

  2. (2)

The second-best performer tends to be DiffRec. Its superior performance among
other generative model-based methods can be attributed to the use of diffusion
models, known for their state-of-the-art performance in various fields. This
enables DiffRec to more intricately recover user–item interactions for
recommendations compared to VAE-based CF methods. However, DiffRec is
consistently inferior to CF-Diff, primarily because it overlooks the high-
order connectivity information, which is essential for capturing crucial
collaborative signals.

  3. (3)

The performance gap between CF-Diff ($X$) and Diffrec ($Y$) is the largest
when the Yelp dataset is used; the maximum improvement rate of 7.29% is
achieved in terms of N@10, where the improvement rate (%) is given by
$\frac{{X-Y}}{Y}\times 100$.

  4. (4)

Compared with GNN-based methods (NGCF, LightGCN, SGL, and NCL) that exploit
high-order connectivity information through the message passing mechanism, our
CF-Diff method exhibits remarkable gains. This superiority basically stems
from the ability of inherently powerful diffusion models and avoiding the
over-smoothing issue when integrating high-order connectivities.

  5. (5)

CFGAN shows relatively lower accuracies compared to other generative model-
based methods. This performance degradation is caused by mode collapse during
GAN training, resulting in inferior recommendation outcomes.

####  4.2.2. Impact of components in CAM-AE (RQ2)

To discover what role each component plays in the success of our learning
model, CAM-AE, we conduct an ablation study by removing or replacing each
component in CAM-AE.

  * •

CAM-AE: corresponds to the original CAM-AE model.

  * •

CAM-AE-att: removes the multi-hop cross-attention module in CAM-AE.

  * •

CAM-AE-ae: removes the attention-aided AE in CAM-AE.

  * •

CAM-AE-self: replaces the multi-hop cross-attention module in CAM-AE with the
multi-hop self-attention module, which ignores the high-order connectivity
information (by replacing ${\bf q}^{\left(h\right)}$ with ${\bf v}_{t}$).

The performance comparison among the original CAM-AE and its three variants is
presented in Table 3 with respect to R@$K$ and N@$K$ on the three datasets.
Our findings are as follows:

Table 3. Performance comparison among CAM-AE and its three variants. Here, the best and second-best performers are highlighted by bold and underline, respectively. Dataset | Method | R@10 | R@20 | N@10 | N@20
---|---|---|---|---|---
ML-1M | CAM-AE-att | 0.1016 | 0.1751 | 0.0873 | 0.1123
CAM-AE-ae | 0.1024 | 0.1732 | 0.0871 | 0.1117
CAM-AE-self | 0.1057 | 0.1794 | 0.0891 | 0.1144
CAM-AE | 0.1077 | 0.1843 | 0.0912 | 0.1176
Yelp | CAM-AE-att | 0.0553 | 0.0905 | 0.0342 | 0.0448
CAM-AE-ae | OOM | OOM | OOM | OOM
CAM-AE-self | 0.0574 | 0.0952 | 0.0355 | 0.0469
CAM-AE | 0.0585 | 0.0962 | 0.0368 | 0.0480
Anime | CAM-AE-att | 0.2091 | 0.3024 | 0.5023 | 0.4623
CAM-AE-ae | OOM | OOM | OOM | OOM
CAM-AE-self | 0.2112 | 0.3094 | 0.5079 | 0.4678
CAM-AE | 0.2191 | 0.3155 | 0.5152 | 0.4748

  1. (1)

The original CAM-AE always exhibits substantial gains over the other variants,
which demonstrates that each component in CAM-AE plays a crucial role in
enhancing the recommendation accuracy.

  2. (2)

CAM-AE outperforms CAM-AE-att, which can be attributed to the fact that the
multi-hop cross-attention module is capable of infusing high-order
connectivities into the proposed method to improve the performance of
recommendations via CF.

  3. (3)

The performance gain of CAM-AE over CAM-AE-ae is relatively higher than that
of the other variants for the ML-1M dataset. Additionally, the attention-aided
AE’s removal leads to out-of-memory (OOM) issues on the Yelp and Anime
datasets, signifying its crucial role not only in extracting representations
that can precisely capture the underlying patterns of user–item interactions
but also in maintaining the computational complexity at acceptable levels.

  4. (4)

CAM-AE is superior to CAM-AE-self. This confirms that infusing high-order
connectivity information enriches the collaborative signal and thus results in
performance enhancement even under a diffusion-model framework.

####  4.2.3. The impact of multi-hop neighbors (RQ3)

To investigate how many hop neighbors in the CF-Diff method are informative,
we present a variant of CF-Diff, CF-Diff-$H$, which considers up to $H$-hop
neighbors constantly instead of optimally searching for the value of $H$
depending on a given dataset. The results are shown in Table 4 and our
observations are as follows:

  1. (1)

CF-Diff-3 outperforms CF-Diff-2 on ML-1M and Yelp, indicating that
incorporating a wider range of neighboring nodes into the CAM-AE model can
positively influence the recommendation results through CF.

  2. (2)

CF-Diff-2 shows the highest recommendation accuracy on Anime. This means that
2-hop neighbors sufficiently capture the collaborative signal, and there is no
need for exploiting higher-order connectivity information in this dataset.

  3. (3)

Notably, there is a decline in the performance of CF-Diff-4, because infusing
4-hop neighbors introduces an excess of global connectivity information. This
surplus information potentially acts as noise, thereby interfering the
personalized recommendations.

Table 4. Performance comparison according to different values of H. Here, the best and second-best performers are highlighted by bold and underline, respectively. Dataset | Method | R@10 | R@20 | N@10 | N@20
---|---|---|---|---|---
ML-1M | CF-Diff-2 | 0.1062 | 0.1786 | 0.0907 | 0.1164
CF-Diff-3 | 0.1077 | 0.1843 | 0.0912 | 0.1176
CF-Diff-4 | 0.1055 | 0.1764 | 0.0883 | 0.1134
Yelp | CF-Diff-2 | 0.0572 | 0.0935 | 0.0351 | 0.0462
CF-Diff-3 | 0.0585 | 0.0962 | 0.0368 | 0.0480
CF-Diff-4 | 0.0561 | 0.0917 | 0.0347 | 0.0455
Anime | CF-Diff-2 | 0.2191 | 0.3155 | 0.5152 | 0.4748
CF-Diff-3 | 0.2082 | 0.3021 | 0.4998 | 0.4586
CF-Diff-4 | 0.1938 | 0.2824 | 0.4605 | 0.4236

####  4.2.4. The effect of hyperparameters (RQ4)

We analyze the impact of key parameters of CAM-AE, including $k$, $d$, $N$,
and $\alpha_{h}$, on the recommendation accuracy for the ML-1M dataset. In
this experiment, we consider 3-hop neighbors (i.e., $H=3$). For notational
convenience, we denote $\alpha_{2}=\alpha$ and $\alpha_{3}=1-\alpha$, which
signify the importance of 2-hop and 3-hop neighbors, respectively. When a
hyperparameter varies so that its effect is clearly revealed, other parameters
are set to the following pivot values: $k=500,d=16,N=2,\alpha=0.7$. Our
findings are as follows:

(Effect of $k$) From Figure 4a, the maximum N@10 and N@20 are achieved at
$k=500$ on ML-1M. It reveals that high values of $k$ degrade the performance
since the resulting embeddings contain more noise and low values of $k$ result
in insufficient information during training. Hence, it is crucial to suitably
determine the value of $k$ in guaranteeing satisfactory performance.

(Effect of $d$) From Figure 4b, the maximum N@10 and N@20 are achieved at
$d=16$ on ML-1M. Using values of $d$ that are too high and too low has a
negative impact on the model’s expressiveness. Thus, it is important to
appropriately determine the value of $d$ depending on the datasets.

(Effect of $N$) From Figure 5a, the maximum N@10 and N@20 are achieved at
$N=2$ on ML-1M. A higher $N$ rather degrades the performance, possibly due to
the over-fitting problem. Thus, the value of $N$ should be carefully chosen
based on given datasets.

(Effect of $\alpha$) Figure 5b shows that the maximum N@10 and N@20 are
achieved at $\alpha=0.7$ on ML-1M. Tuning $\alpha$ is crucial since it
directly determines the model’s ability while balancing between neighbors that
are different hops away from the target user, which in turn affects the
recommendation performance.

$350$$500$$650$$800$$950$$0.08$$0.1$$0.12$(a) Effect of $k$NDCG

$8$$16$$24$$32$$0.08$$0.1$$0.12$(b) Effect of $d$NDCGN@10N@20

Figure 4. The effect of hyperparameters $k$ and $d$ on N@K for the ML-1M
dataset.

$1$$2$$3$$4$$0.06$$0.08$$0.1$$0.12$(a) Effect of $N$NDCG

$0.3$$0.5$$0.7$$0.9$$0.06$$0.08$$0.1$$0.12$(b) Effect of $\alpha$NDCGN@10N@20

Figure 5. The effect of hyperparameters $N$ and $\alpha$ on N@K for the ML-1M
dataset.

$1\cdot 10^{4}$$3\cdot 10^{4}$$6\cdot 10^{4}$$9\cdot
10^{4}$$0$$20$$40$$60$$80$$100$(a) $\left|\mathcal{U}\right|$Execution time
(s)CF-Diff$\mathcal{O}\left({\left|\mathcal{U}\right|}\right)$

$1\cdot 10^{4}$$6\cdot 10^{4}$$13\cdot 10^{4}$$20\cdot
10^{4}$$10$$12$$14$$16$$18$$20$(b) $\left|\mathcal{I}\right|$Execution time
(s)CF-Diff$\mathcal{O}\left({\left|\mathcal{I}\right|}\right)$

Figure 6. The computational complexity of CF-Diff, where the plots of the
execution time versus $\left|\mathcal{U}\right|$ in Figure 6a and the
execution time versus $\left|\mathcal{I}\right|$ in Figure 6b are shown.

####  4.2.5. Computational complexity (RQ5)

To empirically validate the scalability of our CF-Diff method, we measure the
execution time during training on synthetic datasets having user–item
interactions. These interactions are generated purely at random, simulating a
sparsity level of 0.99, analogous to that observed on Yelp and Anime. By
setting different $\left|\mathcal{U}\right|$’s and
$\left|\mathcal{I}\right|$’s, we can create user–item interactions of various
sizes. More specifically, we generate two sets of user–item interactions: in
the first set, we generate a set of interactions with
$\left|\mathcal{I}\right|=1e^{4}$ and
$\left|\mathcal{U}\right|=\left\\{{1e^{4},3e^{4},4e^{4},6e^{4},7e^{4},8e^{4},9e^{4}}\right\\}$;
and in the second set, we generate another set of interactions with
$\left|\mathcal{U}\right|=1e^{4}$ and
$\left|\mathcal{I}\right|=\left\\{{1e^{4},4e^{4},6e^{4},8e^{4},12e^{4},16e^{4},20e^{4}}\right\\}$.
Figure 6a (resp. Figure 6b) illustrates the execution time (in seconds) per
iteration of CF-Diff, including the forward-diffusion process and the reverse-
denoising process, as the number of users (resp. the number of items)
increases. The dashed line indicates a linear scaling in
$\left|\mathcal{U}\right|$ and $\left|\mathcal{I}\right|$, derived from
Theorem 2. It can be seen that our empirical evaluation concurs with the
theoretical analysis.

##  5\. Related Work

In this section, we review some representative methods in two broad fields of
research, including 1) benchmark CF methods and 2) generative model-based
recommendation methods.

###  5.1. General Benchmark CF

The most common paradigm of CF is to factorize the user–item interaction
matrix into lower-dimensional matrices (Mnih and Salakhutdinov, 2007; Koren et
al., 2009; Rendle et al., 2009). The dot product in MF can be replaced with a
multi-layer perceptron (MLP) to capture the non-linearities in the complex
behavior of such interactions (He et al., 2017; Chen et al., 2020). To analyze
beyond direct user connections to items, high-order connectivities are
essential for understanding the user preferences, leading to the rise of GNNs
in CF for modeling these complex relationships (Berg et al., 2018). GC-MC
(Berg et al., 2018) first proposed a graph AE framework for recommendations
using message passing on the user–item bipartite graph. NGCF (Wang et al.,
2019) employed GNNs to propagate user and item embeddings on the bipartite
graph capturing the collaborative signal in complex high-order connectivities.
NIA-GCN (Sun et al., 2020) was developed by taking into account both the
relational information between neighboring nodes and the heterogeneous nature
of the user–item bipartite graph. LightGCN (He et al., 2020) improved the
performance by lightweight message passing, omitting feature transformation
and nonlinear activation. UltraGCN (Mao et al., 2021) advanced efficiency by
skipping infinite layers of explicit message passing and directly
approximating graph convolution limits with a constraint loss. BSPM (Choi et
al., 2023) made a connection between the concept of blurring-sharpening
process models and graph filtering (Shen et al., 2021), utilizing ordinary
differential equations to model the perturbation and recovery of user–item
interactions. Additionally, contrastive learning was used to further improve
the recommendation accuracy by taking node self-discrimination into account
(Wu et al., 2021; Lin et al., 2022).

###  5.2. Generative Model-Based Recommendation

GAN-based methods. Generative adversarial network (GAN)-based models in CF
employ a generator to estimate user–item interaction probabilities, optimized
through adversarial training (Goodfellow et al., 2014; Wang et al., 2017; Guo
et al., 2020; Wu et al., 2019). RecGAN (Bharadhwaj et al., 2018) combined
recurrent neural network (RNN) with GAN for capturing complex user–item
interaction patterns, while CFGAN (Chae et al., 2018) enhanced the
recommendation accuracy with real-valued vector-wise adversarial learning.
Nevertheless, adversarial training is often associated with training
instability and mode collapse, potentially leading to suboptimal performance
(Arjovsky and Bottou, 2017; Metz et al., 2017).

VAE-based methods. The denoising AE (DAE) was firstly used for top-$K$
recommendations, learning latent representations from corrupted user
preferences (Wu et al., 2016). CVAE (Li and She, 2017) extended this by using
a VAE to learn latent representations of items from ratings and multimedia
content for multimedia recommendations. A series of VAE-based methods (Liang
et al., 2018; Ma et al., 2019; Shenbin et al., 2020) were further developed
for CF with implicit feedback, enhancing the accuracy, interpretability, and
robustness by incorporating a multinomial likelihood and a Bayesian approach
for user preference modeling. However, VAE-based models struggle to balance
between simplicity and representations of complex data, with simpler models
possibly failing to capture diverse user preferences and more complex models
potentially being computationally intractable (Sohl-Dickstein et al., 2015).

Diffusion model-based methods. Recently, diffusion models have achieved state-
of-the-art performance in image generation by decomposing the image generation
process into a series of DAEs. CODIGEM (Walker et al., 2022) extended this
with the denoising diffusion probabilistic model (DDPM) in (Ho et al., 2020)
to recommender systems, leveraging the intricate and non-linear patterns in
the user–item interaction matrix. Additionally, diffusion models have been
successfully applied to sequential recommendations (Yang et al., 2023; Liu et
al., 2023; Wu et al., 2023; Li et al., 2023). Inspired by score-based
generative models (Song et al., 2020), DiffRec (Wang et al., 2023)
accommodated diffusion models to predict unknown user–item interactions in a
denoising manner by gradually corrupting interaction histories with scheduled
Gaussian noise and then recovering the original interactions iteratively
through a neural network.

Discussion. Despite the impressive performance of current diffusion model-
based recommender systems, existing models overlook high-order user–item
connectivities that reveal co-preference patterns between users and items.
These high-order connectivities among users and items are crucial in CF
performed with limited direct user–item interactions, aiding in delivering
more precise and personalized recommendations. However, effectively
incorporating such high-order connectivity information remains a significant
challenge in diffusion model-based CF.

##  6\. Conclusions

In this paper, we explored an open yet fundamental problem of how to empower
CF-based recommender systems when diffusion models are employed as a core
framework for training. To tackle this challenge, we proposed CF-Diff, a
diffusion model-based approach for generative recommender systems, designed to
infuse high-order connectivity information into our own learning model, CAM-
AE, while preserving the model’s complexity at manageable levels. Through
extensive experiments on three real-world benchmark datasets, we demonstrated
(a) the superiority of CF-Diff over nine state-of-the-art recommendation
methods while showing dramatic gains up to 7.29% in terms of NDCG@10 compared
to the best competitor, (b) the theoretical findings that analytically confirm
the computational tractability and scalability of CF-Diff, (c) the
effectiveness of core components in CAM-AE, and (d) the impact of tuning key
hyperparameters in CAM-AE.

###### Acknowledgements.

This work was supported by the National Re- search Foundation of Korea (NRF),
Republic of Korea Grant by the Korean Government through MSIT under Grants
2021R1A2C3004345 and RS-2023-00220762 and by the Institute of Information and
Communications Technology Planning and Evaluation (IITP), Republic of Korea
Grant by the Korean Government through MSIT (6G Post-MAC–POsitioning and
Spectrum-Aware intelligenT MAC for Computing and Communication Convergence)
under Grant 2021-0-00347.

## References

  * (1)
  * Arjovsky and Bottou (2017) Martin Arjovsky and Léon Bottou. 2017.  Towards principled methods for training generative adversarial networks. In _ICLR_.
  * Austin et al. (2021) Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. 2021\.  Structured denoising diffusion models in discrete state-spaces. In _NeurIPS_. 17981–17993.
  * Avrahami et al. (2022) Omri Avrahami, Dani Lischinski, and Ohad Fried. 2022\.  Blended diffusion for text-driven editing of natural images. In _CVPR_. 18208–18218.
  * Berg et al. (2018) Rianne van den Berg, Thomas N Kipf, and Max Welling. 2018\.  Graph convolutional matrix completion. In _KDD (Deep Learning Day)_.
  * Bharadhwaj et al. (2018) Homanga Bharadhwaj, Homin Park, and Brian Y Lim. 2018.  RecGAN: Recurrent generative adversarial networks for recommendation systems. In _RecSys_. 372–376.
  * Chae et al. (2018) Dong-Kyu Chae, Jin-Soo Kang, Sang-Wook Kim, and Jung-Tae Lee. 2018.  CFGAN: A generic collaborative filtering framework based on generative adversarial networks. In _CIKM_. 137–146.
  * Chen et al. (2020) Chong Chen, Min Zhang, Yongfeng Zhang, Yiqun Liu, and Shaoping Ma. 2020.  Efficient neural matrix factorization without sampling for recommendation.  _ACM Transactions on Information Systems (TOIS)_ 38, 2 (2020), 1–28.
  * Choi et al. (2023) Jeongwhan Choi, Seoyoung Hong, Noseong Park, and Sung-Bae Cho. 2023\.  Blurring-sharpening process models for collaborative filtering. In _SIGIR_. 1096–1106.
  * Deja et al. (2022) Kamil Deja, Anna Kuzina, Tomasz Trzcinski, and Jakub Tomczak. 2022\.  On analyzing generative and denoising capabilities of diffusion-based deep generative models. In _NeurIPS_. 26218–26229.
  * Dhariwal and Nichol (2021) Prafulla Dhariwal and Alexander Nichol. 2021.  Diffusion models beat gans on image synthesis. In _NeurIPS_. 8780–8794.
  * Goodfellow et al. (2014) Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014\.  Generative adversarial nets. In _NeurIPS_. 2672–2680.
  * Guo et al. (2020) Guibing Guo, Huan Zhou, Bowei Chen, Zhirong Liu, Xiao Xu, Xu Chen, Zhenhua Dong, and Xiuqiang He. 2020\.  IPGAN: Generating informative item pairs by adversarial sampling.  _IEEE Transactions on Neural Networks and Learning Systems_ 33, 2 (2020), 694–706.
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020\.  LightGCN: Simplifying and powering graph convolution network for recommendation. In _SIGIR_. 639–648.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _WWW_. 173–182.
  * Ho et al. (2020) Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020.  Denoising diffusion probabilistic models. In _NeurIPS_. 6840–6851.
  * Huang et al. (2021) Chin-Wei Huang, Jae Hyun Lim, and Aaron C Courville. 2021\.  A variational perspective on diffusion-based generative models and score matching. In _NeurIPS_. 22863–22876.
  * Kingma and Ba (2015) Diederik P Kingma and Jimmy Ba. 2015.  Adam: A method for stochastic optimization. In _ICLR (Poster)_.
  * Kingma and Welling (2014) Diederik P Kingma and Max Welling. 2014.  Auto-encoding variational bayes. In _ICLR_.
  * Koren et al. (2009) Yehuda Koren, Robert Bell, and Chris Volinsky. 2009\.  Matrix factorization techniques for recommender systems.  _Computer_ 42, 8 (2009), 30–37.
  * Li and She (2017) Xiaopeng Li and James She. 2017.  Collaborative variational autoencoder for recommender systems. In _KDD_. 305–314.
  * Li et al. (2023) Zihao Li, Aixin Sun, and Chenliang Li. 2023.  DiffuRec: A diffusion model for sequential recommendation.  _ACM Transactions on Information Systems_ 42, 3 (2023), 1–28.
  * Liang et al. (2018) Dawen Liang, Rahul G Krishnan, Matthew D Hoffman, and Tony Jebara. 2018.  Variational autoencoders for collaborative filtering. In _WWW_. 689–698.
  * Lin et al. (2022) Zihan Lin, Changxin Tian, Yupeng Hou, and Wayne Xin Zhao. 2022\.  Improving graph collaborative filtering with neighborhood-enriched contrastive learning. In _WWW_. 2320–2329.
  * Liu et al. (2023) Qidong Liu, Fan Yan, Xiangyu Zhao, Zhaocheng Du, Huifeng Guo, Ruiming Tang, and Feng Tian. 2023.  Diffusion augmentation for sequential recommendation. In _CIKM_. 1576–1586.
  * Ma et al. (2019) Jianxin Ma, Chang Zhou, Peng Cui, Hongxia Yang, and Wenwu Zhu. 2019.  Learning disentangled representations for recommendation. In _NeurIPS_. 5712–5723.
  * Mao et al. (2021) Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, and Xiuqiang He. 2021\.  UltraGCN: Ultra simplification of graph convolutional networks for recommendation. In _CIKM_. 1253–1262.
  * Metz et al. (2017) Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. 2017\.  Unrolled generative adversarial networks. In _ICLR (Poster)_.
  * Mnih and Salakhutdinov (2007) Andriy Mnih and Russ R Salakhutdinov. 2007.  Probabilistic matrix factorization. In _NeurIPS_. 1257–1264.
  * Park et al. (2023) Jin-Duk Park, Siqing Li, Xin Cao, and Won-Yong Shin. 2023\.  Criteria tell you more than ratings: Criteria preference-aware light graph convolution for effective multi-criteria recommendation. In _KDD_. 1808–1819.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian personalized ranking from implicit feedback. In _UAI_. 452–461.
  * Rombach et al. (2022) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022\.  High-resolution image synthesis with latent diffusion models. In _CVPR_. 10684–10695.
  * Saharia et al. (2022) Chitwan Saharia, William Chan, and et al. 2022.  Photorealistic text-to-image diffusion models with deep language understanding. In _NeurIPS_. 36479–36494.
  * Savinov et al. (2022) Nikolay Savinov, Junyoung Chung, Mikolaj Binkowski, Erich Elsen, and Aaron van den Oord. 2022.  Step-unrolled denoising autoencoders for text generation. In _ICLR_.
  * Shen et al. (2021) Yifei Shen, Yongji Wu, Yao Zhang, Caihua Shan, Jun Zhang, B Khaled Letaief, and Dongsheng Li. 2021.  How powerful is graph convolution for recommendation?. In _CIKM_. 1619–1629.
  * Shenbin et al. (2020) Ilya Shenbin, Anton Alekseev, Elena Tutubalina, Valentin Malykh, and Sergey I Nikolenko. 2020\.  RecVAE: A new variational autoencoder for top-N recommendations with implicit feedback. In _WSDM_. 528–536.
  * Sohl-Dickstein et al. (2015) Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015.  Deep unsupervised learning using nonequilibrium thermodynamics. In _ICML_. 2256–2265.
  * Song et al. (2020) Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2020.  Score-based generative modeling through stochastic differential equations. In _ICLR_.
  * Sun et al. (2020) Jianing Sun, Yingxue Zhang, Wei Guo, Huifeng Guo, Ruiming Tang, Xiuqiang He, Chen Ma, and Mark Coates. 2020\.  Neighbor interaction aware graph convolution networks for recommendation. In _SIGIR_. 1289–1298.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need. In _NeurIPS_. 6000–6010.
  * Walker et al. (2022) Joojo Walker, Ting Zhong, Fengli Zhang, Qiang Gao, and Fan Zhou. 2022.  Recommendation via collaborative diffusion generative model. In _KSEM_. Springer, 593–605.
  * Wang et al. (2017) Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017\.  IRGAN: A minimax game for unifying generative and discriminative information retrieval models. In _SIGIR_. 515–524.
  * Wang et al. (2020) Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020.  Linformer: Self-attention with linear complexity.  _arXiv preprint arXiv:2006.04768_ (2020).
  * Wang et al. (2023) Wenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, and Tat-Seng Chua. 2023\.  Diffusion recommender model. In _SIGIR_. 832–841.
  * Wang et al. (2019) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019.  Neural graph collaborative filtering. In _SIGIR_. 165–174.
  * Wu et al. (2021) Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021.  Self-supervised graph learning for recommendation. In _SIGIR_. 726–735.
  * Wu et al. (2019) Qiong Wu, Yong Liu, Chunyan Miao, Binqiang Zhao, Yin Zhao, and Lu Guan. 2019\.  PD-GAN: Adversarial learning for personalized diversity-promoting recommendation. In _IJCAI_. 3870–3876.
  * Wu et al. (2016) Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. 2016.  Collaborative denoising auto-encoders for top-N recommender systems. In _WSDM_. 153–162.
  * Wu et al. (2023) Zihao Wu, Xin Wang, Hong Chen, Kaidong Li, Yi Han, Lifeng Sun, and Wenwu Zhu. 2023.  Diff4Rec: Sequential recommendation with curriculum-scheduled diffusion augmentation. In _MM_. 9329–9335.
  * Yang et al. (2018) Jheng-Hong Yang, Chih-Ming Chen, Chuan-Ju Wang, and Ming-Feng Tsai. 2018.  HOP-Rec: High-order proximity for implicit recommendation. In _RecSys_. 140–144.
  * Yang et al. (2023) Zhengyi Yang, Jiancan Wu, Zhicai Wang, Xiang Wang, Yancheng Yuan, and Xiangnan He. 2023\.  Generate what you prefer: Reshaping sequential recommendation via guided diffusion. In _NeurIPS_.
  * Zhou et al. (2018) Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018\.  Deep interest network for click-through rate prediction. In _KDD_. 1059–1068.