: Sign-Aware Recommendation
Using Graph Neural Networks

Changwon Seo, Kyeong-Joong Jeong, Sungsu Lim, Member, IEEE, and Won-Yong Shin, Senior Member, IEEE
C. Seo, K.-J. Jeong, and W.-Y. Shin are with the School of Mathematics and Computing (Computational Science and Engineering), Yonsei University, Seoul 03722, Republic of Korea (e-mail: {changwoni, jeongkj, wy.shin}@yonsei.ac.kr).
S. Lim is with the Department of Computer Science and Engineering, Chungnam National University, Daejeon 34134, Republic of Korea (e-mail: sungsu@cnu.ac.kr).

(Corresponding author: Won-Yong Shin.)

In recent years, many recommender systems using network embedding (NE) such as graph neural networks (GNNs) have been extensively studied in the sense of improving recommendation accuracy. However, such attempts have focused mostly on utilizing only the information of positive user–item interactions with high ratings. Thus, there is a challenge on how to make use of low rating scores for representing users' preferences since low ratings can be still informative in designing NE-based recommender systems. In this study, we present , a new Sign-aware Recommender system based on GNN models. Specifically,  has three key components: 1) constructing a signed bipartite graph for more precisely representing users' preferences, which is split into two edge-disjoint graphs with positive and negative edges each, 2) generating two embeddings for the partitioned graphs with positive and negative edges via a GNN model and a multi-layer perceptron (MLP), respectively, and then using an attention model to obtain the final embeddings, and 3) establishing a sign-aware Bayesian personalized ranking (BPR) loss function in the process of optimization. Through comprehensive experiments, we empirically demonstrate that  consistently outperforms state-of-the-art NE-aided recommendation methods.

Bayesian personalized ranking (BPR) loss, graph neural network, network embedding, recommender system, signed bipartite graph.

§ INTRODUCTION

§.§ Background and Motivation

Recommender systems have been widely advocated as a way of providing suitable recommendation solutions to customers in various fields such as e-commerce, advertising, and social media sites. One of the most important and popular techniques in recommender systems is collaborative filtering (CF), which computes similarities between users and items from historical interactions (e.g., clicks and purchases) to suggest relevant items to users by assuming that users who have behaved similarly with each other exhibit similar preferences for items <cit.>. Moreover, following up the great success of network embedding (NE), also known as network representation learning, considerable research attention has been paid to NE-based recommender systems that attempt to model high-order connectivity information from user–item interactions viewed as a bipartite graph <cit.>.[In the following, we use the terms “graph" and “network" interchangeably.] In recent years, graph neural networks (GNNs) <cit.> have emerged as a powerful neural architecture to learn vector representations of nodes and graphs. By virtue of great prowess in solving various downstream machine learning problems, GNN-based recommender systems <cit.> have also been developed for improving the recommendation accuracy.

GNN models are basically trained by aggregating the information of direct neighbor nodes via message passing under the homophily (or assortativity) assumption that a target node and its neighbors are similar to each other <cit.>.  Due to the neighborhood aggregation mechanism, existing literature posits that high homophily of the underlying graph is a necessity for GNNs to achieve good performance especially on node classification <cit.>. On the other hand, in recommender systems, while users' feedback on many online websites (e.g., likes/dislikes on YouTube and high/low ratings on Amazon) can be positive and negative, existing GNN-based recommender systems overlook the existence of negative feedback (e.g., dislikes on YouTube and low ratings on Amazon) due to their ease of modeling. Precisely, most GNN-based approaches utilize only positive feedback by removing the negative interactions in order to exhibit strong homophily in neighbors (see Fig. <ref>).[A graph is usually referred to as homophilous (or assortative) if connected nodes are much more likely to have the same class label than if edges are independent of labels. In our study, a homophilous bipartite graph assumes that user–item relations are significantly more likely to be positive.] It is worthwhile to note that, despite the remarkable performance boost by existing GNN-based recommender systems, the low ratings can be still informative. This is because such information expresses signs of what users dislike. In other words, full exploitation of two types of feedback in GNNs may have the potential to further improve the recommendation performance, which remains a new design challenge.

New challenges in GNN-based recommender systems.

§.§ Main Contributions

CF in recommender systems has been often studied by regarding low ratings as implicit negative feedback (see <cit.>). However, such interpretations may not be straightforwardly extended to GNN-based recommender systems due to their inherent architecture including neighborhood aggregation. In this context, even with the wide applications of GNNs to recommender systems <cit.>, a question that arises naturally is: “How can we make use of implicit  negative feedback (i.e., low rating scores) for representing users' preferences via GNNs?". In this paper, to answer this question, we introduce , a Sign-aware Recommender system based on GNN models. To this end, as illustrated in Fig. <ref>, we design and optimize our new GNN-aided learning model while distinguishing users' positive and negative feedback.

Our proposed  method includes three key components: 1) signed graph construction and partitioning, 2) model architecture design, and 3) model optimization. First, to overcome the primary problem of existing GNN-based recommender systems <cit.> that fail to learn both positive and negative relations between users and items, we start by constructing a signed bipartite graph, which is split into two edge-disjoint graphs with positive and negative edges each. This signed graph construction and partitioning process enables us to more distinctly identify users' preferences for observed items. Second, we show how to design our model architecture for discovering embeddings of all users and items in the signed bipartite graph. Although several GNN models were introduced in <cit.> for signed unipartite graphs, simply applying them to recommender systems, corresponding to user–item bipartite graphs, may not be desirable. This is because such GNN models were built upon the assumption of balance theory <cit.>, which implies that “the friend of my friend is my friend" and “the enemy of my enemy is my friend". However, the balance theory no longer holds in recommender systems since users' preferences cannot be dichotomous.  In other words, users are likely to have dissimilar preferences even if they dislike the same item(s). This motivates us to design our own model architecture for each partitioned graph, rather than employing existing GNN approaches based on the balance theory. Concretely,  contains three learning models. For the graph with positive edges, we employ a GNN model suit for recommender systems. For the graph with negative edges, we adopt a multi-layer perceptron (MLP) due to the fact that negative edges can weaken the homophily and thus message passing to such dissimilar nodes would not be feasible. To obtain the final embeddings, we then use an attention model that learns the importance of two embeddings generated from GNN and MLP models. Third, as our objective function in the process of optimization, we present a sign-aware Bayesian personalized ranking (BPR) loss function, which is built upon the original BPR loss <cit.> widely used in recommender systems. More specifically, unlike the original BPR loss, our objective function takes into account two types of observed items, including both positive and negative relations between users and items, as well as unobserved ones.

To validate the superiority of our  method, we comprehensively perform empirical evaluations using various real-world datasets. Experimental results show that our method consistently outperforms state-of-the-art GNN methods for top-$K$ recommendation in terms of several recommendation accuracy metrics. Such a gain is possible owing to the use of low rating information along with judicious model design and optimization. We also empirically validate the effectiveness of MLP in comparison with other model architectures used for the graph with negative edges. Additionally, our experimental results demonstrate the robustness of our  method to more challenging interaction sparsity levels.

It is worth noting that our method is GNN-model-agnostic and thus any competitive GNN architectures can be appropriately chosen for potentially better performance. The main technical contributions of this paper are four-fold and summarized as follows:

* We propose , a novel GNN-aided recommender system that makes full use of the user–item interaction information after signed graph construction and partitioning;

* We design our model architecture using three learning models in the sense of utilizing sign awareness so as to generate embeddings of users and items;

* We establish the sign-aware BPR loss as our objective function;

* We validate  through extensive experiments using three real-world datasets while showing the superiority of our method over state-of-the-art NE-aided methods under diverse conditions.

§.§ Organization and Notation

The remainder of this paper is organized as follows. In Section II, we present prior studies related to our work. In Section III, we explain the methodology of our study, including basic settings and an overview of our  method. Section IV describes technical details of the proposed method. Comprehensive experimental results are shown in Section V. Finally, we provide a summary and concluding remarks in Section VI.

§ RELATED WORK

The method that we proposed in this study is related to three  broader research lines, namely standard CF approaches,  NE-based recommendation approaches, and NE approaches for signed graphs.

§.§ Standard CF Approaches

As one of the most popular techniques, CF in recommender systems aims to capture the relationships between users and items from historical interactions (e.g., ratings and purchases) by discovering learnable vector representations for users and items based on a rating matrix <cit.>. Matrix factorization (MF) <cit.> decomposes the user–item interaction matrix into a product of two low-dimensional matrices and models their similarities with the dot product of two matrices. ANLF <cit.> was designed based on a non-negative MF (NMF) model <cit.> using the alternating direction method. In DMF <cit.> and NCF <cit.>, MF models with neural network architectures were proposed to project users and items into a latent structured space. Moreover, SLIM <cit.> proposed a sparse linear model by directly reconstructing the low-density user–item interaction matrix. To capture complex relationships between items in sparse datasets, FISM <cit.> and NPE <cit.> showed how to learn item–item similarities as the product of two latent factor matrices. While MF-based models such as <cit.> rely on the dot product of user and item latent vectors as a similarity measure, CML <cit.> showed how to learn a joint metric space to encode not only users' preferences but also user–user and item–item similarities in the Euclidean distance. In NAIS <cit.>, the attention mechanism was incorporated to obtain the importance of each item from user–item interactions for preference prediction. DLMF <cit.> designed a trust-aware recommender system that leverages deep learning to determine the initialization in MF by synthesizing the users' interests and their trusted friends' interests in addition to the user–item interactions.

§.§ NE-Based Recommendation Approaches

Recently, it has been comprehensively studied how to develop recommender systems using NE. While standard CF approaches are capable of only modeling the first-order connectivity between users and items, NE-based approaches aim to exploit high-order proximity among users and items through the user–item bipartite graph structure <cit.>. BiNE <cit.> and CSE <cit.> were developed based on random walks in order to infer similar users (or items) in the underlying bipartite graph (i.e., user–item interactions).

Moreover, encouraged by the success of GNNs in solving many graph mining tasks <cit.>, GNN-based recommender systems have more recently emerged as promising techniques <cit.>. Existing GNN models for top-$K$ recommendation were generally developed using implicit feedback, which treats observed user–item interactions as positive relations. GC-MC <cit.> presented a graph autoencoder including graph convolutions for rating matrix completion. PinSage <cit.> showed a scalable GNN framework developed in production at Pinterest by improving upon GraphSAGE <cit.> along with several aggregation functions. Spectral CF <cit.> presented a spectral convolution operation to learn the rich information of connectivity between users and items in the spectral domain. As state-of-the-art NE-based recommender systems using only user–item interactions as input, NGCF <cit.>, LR-GCCF <cit.>, and LightGCN <cit.> were developed based on GCN <cit.>, the first attempt to apply convolutions to graph domains, while performing layer aggregation to solve the oversmoothing problem. IGMC <cit.> proposed an inductive GNN-based matrix completion model that learns local subgraphs in the underlying user–item interaction matrix. To explore users' latent purchasing motivations (e.g., cost effectiveness and appearance), MCCF <cit.> presented a CF approach based on GNNs for generating multiple user/item embeddings and then combining them via attention mechanisms. AGCN <cit.> proposed a GCN approach for joint item recommendation and attribute inference in an attributed user–item bipartite graph with incomplete attribute values.

In addition to various GNN architectures in <cit.> for recommender systems, to  alleviate the sparsity of user–item interactions, each user's local neighbors' preferences in social networks were also utilized in <cit.> for better user embedding modeling, thus enabling us to improve the recommendation accuracy. An approach for formulating feature-aware recommendation from the review information via a signed hypergraph convolutional network was also presented in <cit.>.

§.§ NE Approaches for Signed Networks

The design of NE in signed networks has garnered considerable attention while primarily solving the problem of link sign prediction <cit.>, which is to predict unobserved signs of existing edges <cit.>. Specifically, NE methods in <cit.>, <cit.> for signed networks were developed by formulating their own likelihood functions based on generated random walk sequences. Moreover, GNN-aided NE approaches for signed networks were presented in <cit.> while being guided by the structural balance theory <cit.>. SGCN <cit.> generalized GCN <cit.> to signed networks by generating both balanced and unbalanced embeddings. SiGAT <cit.> was built upon GAT <cit.> along with motif-based GNNs. SNEA <cit.> presented a universal way of leveraging the graph attention mechanism to aggregate more important information through both positive and negative edges. By adopting the k-group theory beyond the balance theory, GS-GNN <cit.> designed a dual GNN architecture to learn global and local representations.

§.§ Discussion

Despite the aforementioned contributions, leveraging explicit feedback data (i.e., user–item interactions with ratings) in NE-based approaches has been largely underexplored in the literature. To learn vector representations for users and items based on the NE, it is common to utilize only positive user–item interactions as observed data when explicit feedback data are given (see <cit.> and references therein). However, negative interactions with low ratings can be still informative since such information shows signs of what users dislike. It remains open how to make use of low ratings in designing NE-aided recommender systems.

On the other hand, as aforementioned, several GNN models <cit.> have been actively developed to learn vector representations of nodes in signed (unipartite) graphs with both positive and negative edges based on the balance theory or its variant. However, it is worth noting that these models focus primarily on such networks that exhibit friend/foe (or trust/distrust) relationships. This implies that they do not always hold in recommender systems due to the fact that users' preferences cannot be dichotomous, i.e., a behavior of users disliking similar items does not alway imply the same degree of user preferences about items. Therefore, adopting such GNN methods designed for signed graphs would not be desirable to capture different levels of user preferences in recommender systems.

The schematic overview of our  method.

§ METHODOLOGY

In this section, we describe our network model with basic settings. Next, we explain an overview of the proposed  method as a solution to the problem of making full use of low ratings in GNN models.

§.§ Network Model and Basic Settings

In recommender systems, the basic input is the historical user–item interactions with ratings, which is represented as a weighted bipartite graph. Let us denote the underlying bipartite graph as $G =( \mathcal{U}, \mathcal{V}, \mathcal{E})$, where $\mathcal{U}$ and $\mathcal{V}$ are the set of $M$ users and the set of $N$ items, respectively, and $\mathcal{E}$ is the set of weighted edges between $\mathcal{U}$ and $\mathcal{V}$. A weighted edge $(u,v,w_{uv})\in \mathcal{E}$ can be interpreted as the rating $w_{uv}$ with which the user $u\in \mathcal{U}$ has given to the item $v\in \mathcal{V}$. We assume $G$ to be a static network without repeated edges, where ratings (i.e., user preferences) do not change over time.

In our study, we aim at designing a new GNN-aided recommender system for improving the accuracy of top-$K$ recommendation by making full use of the user–item rating information in $G$, including low ratings that have not been explored by conventional GNN-based recommender systems <cit.>, without any side information.

§.§ Overview of

In this subsection, we explain our methodology along with the overview of the proposed  method. We recall that our study is motivated by the fact that recent recommender systems built upon GNN models such as <cit.> take advantage of only high rating scores as observed data by deleting some edges, corresponding to low ratings (e.g., the rating scores of 1 and 2 in the 1–5 rating scale), in the set $\mathcal{E}$ over the weighted bipartite graph $G$ <cit.>. This is because such a removal of low ratings from $G$ enables us to aggregate the positively connected neighbors via message passing in GNNs. However, the set of negative interactions indicates what users dislike and thus is still quite informative. In other words, it remains open how to fully exploit the rating information in building GNN-based recommender systems as recently developed models fail to capture the effect of low rating scores.

To tackle this challenge, we present , a new sign-aware recommender system using GNNs, which is basically composed of the following three core components (refer to Fig. <ref>):

* signed bipartite graph construction and partitioning

* embedding generation for each partitioned graph

* optimization via a sign-aware BPR loss.

First, we describe how to construct a signed bipartite graph $G^s$ that enables us to more distinctly identify users' preferences based on all the user–item interactions. More specifically, we construct $G^s=(\mathcal{U}, \mathcal{V}, \mathcal{E}^s)$ with a parameter $w_o>0$, representing a criterion for dividing high and low ratings, where

\begin{align}
\label{align_signededges}
\mathcal{E}^s= \big\{ (u,v,w_{uv}^{s})\big|w_{uv}^{s}=w_{uv}-w_{o},~(u,v,w_{uv})\in \mathcal{E} \big\}.
\end{align}

Here, $w_o$ can be determined according to characteristics (e.g., the rating scale and the popularity distribution of items) of a given dataset; from an algorithm design perspective, we assume that a user $u$ likes an item $v$ if $w_{uv}^s>0$ and he/she dislikes $v$ otherwise, where $w_{uv}^s=w_{uv}-w_o$ corresponds to the edge weight in the signed bipartite graph $G^s$.  Note that this signed graph construction is not necessary if we deal with originally signed bipartite graphs (e.g., like/dislike rating systems on YouTube).

Basically, GNN models are trained by aggregating the information of neighbor nodes under the homophily assumption <cit.>. However, due to the fact that the signed graph $G^s$ includes negative edges (i.e., interactions with low ratings), aggregating the information of such negatively connected neighbors may not be desirable. As illustrated in Fig. <ref>, to more delicately capture each relation of positively and negatively connected neighbors, we then partition the signed bipartite graph $G^s$ into two edge-disjoint graphs $G^p=(\mathcal{U}, \mathcal{V}, \mathcal{E}^p)$ and $G^n=(\mathcal{U}, \mathcal{V}, \mathcal{E}^n)$, consisting of the set of positive edges and negative edges, respectively. Here, it follows that $\mathcal{E}^s=\mathcal{E}^p\cup \mathcal{E}^n$, where

\begin{align}
\label{graph_split}
\mathcal{E}^p&= \big\{ (u,v,w_{uv}^{s})\big|~w_{uv}^s>0,~(u, v, w_{uv}^{s})\in \mathcal{E}^s \big\}\\
\mathcal{E}^n&= \big\{ (u,v,w_{uv}^{s})\big|~w_{uv}^s<0,~(u, v, w_{uv}^{s})\in \mathcal{E}^s \big\}.
\end{align}

The purpose of this graph partitioning is to make the graphs $G^p$ and $G^n$, respectively, assortative and disassortative so that each partitioned graph is used as input to the most appropriate learning model.

Second, we describe how to generate embeddings of $M$ users and $N$ items along with three learning models in our  method. Using the graph $G^p$ having positive edges, we adopt a GNN model suit for recommender systems (e.g., <cit.>) to calculate embedding vectors ${\bf Z}^p\in \mathbb{R}^{(M+N)\times d}$ for the nodes in $\mathcal{U}\cup \mathcal{V}$:

\begin{align}
\label{align_GNN_pos}
{\bf Z}^p=\text{GNN}_{\theta_1}(G^p),
\end{align}

where $d$ is the dimension of the embedding space and $\theta_1$ is the learned model parameters of GNN. On the other hand, we adopt an MLP for the graph $G^n$ to calculate embedding vectors ${\bf Z}^n\in \mathbb{R}^{(M+N)\times d}$ for the same nodes as those in (<ref>):

\begin{align}
\label{align_MLP_neg}
{\bf Z}^n=\text{MLP}_{\theta_2}(G^n),
\end{align}

where $\theta_2$ is the learned model parameters of MLP. We would like to state the following two remarks to explain the model selection according to types of graphs.

It is worth noting that existing GNN models, built upon message passing architectures, work on the basic assumption of homophily <cit.>. We recall that users who dislike similar items may not have similar preferences (i.e., the tendency in rating items) with each other. Negative edges in $G^n$ can undermine the effect of homophily and thus message passing to such dissimilar nodes would not be feasible. For these reasons, adopting GNNs in the graph with negative edges may not be desirable, based on the fact that many GNNs fail to generalize to disassortative graphs, i.e., graphs with low levels of homophily <cit.>.

Additionally, note that the MLP architecture itself does not exploit the topological information. However, it does not imply that the connectivity information in the graph $G^n$ is not used at all. In the optimization step, we update the embedding vectors in (<ref>) and (<ref>) by fully taking advantage of the set $\mathcal{E}^n$ in $G^n$ as well as the set $\mathcal{E}^p$ in $G^p$, which will be specified later.

Next, let us mention another training model in , the so-called attention model. To get the importance of two embeddings ${\bf Z}^p$ and ${\bf Z}^n$, we use the attention mechanism <cit.> that learns the corresponding importance $({\bf \alpha}_p,{\bf \alpha}_n)$ as follows:

\begin{align}
\label{align_attn}
({\bf \alpha}^p,{\bf \alpha}^n)=\text{ATTENTION}_{\theta_3} ({\bf Z}^p,{\bf Z}^n),
\end{align}

which results in the final embeddings:

\begin{align}
\label{attn_final_emb}
{\bf Z}=({\bf \alpha}^p{\bf 1}_{\text{attn}}) \odot {\bf Z}^p + ({\bf \alpha}^n{\bf 1}_{\text{attn}}) \odot {\bf Z}^n,
\end{align}

where ${\bf \alpha}^p,{\bf \alpha}^n\in \mathbb{R}^{(M+N)\times 1}$; ${\bf 1}_{\text{attn}}\in\mathbb{R}^{1\times d}$ is the all-ones vector; $\odot$ denotes the Hadamard (element-wise) product; $\theta_3$ is the learned model parameters of ATTENTION; and each row of ${\bf Z}\in\mathbb{R}^{(M+N)\times d}$ indicates the embedding vector of each node in $\mathcal{U}\cup\mathcal{V}$.

Third, we turn to the optimization of model parameters $\{\theta_1,\theta_2,\theta_3\}$, which updates the embeddings ${\bf Z}$ accordingly. In our study, we adopt the BPR loss <cit.>, which has been widely used in recommender systems to comprehensively learn what users prefer from the historical user–item interactions. Nevertheless, simply applying the existing BPR loss to our setting does not precisely capture the relations of negatively connected neighbors; thus, we establish a sign-aware BPR loss, which is a new BPR-based loss that takes into account both positive and negative relations in the signed bipartite graph $G^s$, while accommodating the sign of edges in $G^s$ as an indicator of what users like and dislike.

In the next section, we shall describe implementation details of the proposed  method.

[1]

$G$, $w_o$, $\Theta \triangleq \{\theta_1, \theta_2, \theta_3\}$, $N_{\text{neg}}$ (number of negative samples), $\lambda_{\text{reg}}$ (regularization coefficient)
Initialization: $\Theta \leftarrow $ random initialization
/* Graph partitioning */
Construction of $G^s$ from $G$ along with $w_o$
Splitting $G^s$ into $G^p$ and $G^n$

not converged
Create a training set  $\mathcal{D}_S$ from $G^s$ with $N_{\text{neg}}$ negative samples
each mini-batch  $\mathcal{D}_S' \subset \mathcal{D}_S$

/* Generation of embeddings */
${\bf Z}^p\leftarrow \text{GNN}_{\theta_1}(G^p)$
${\bf Z}^n\leftarrow \text{MLP}_{\theta_2}(G^n)$
$({\bf \alpha}^p,{\bf \alpha}^n)\leftarrow \text{ATTENTION}_{\theta_3} ({\bf Z}^p,{\bf Z}^n)$
${\bf Z}\leftarrow ({\bf \alpha}^p{\bf 1}_{\text{attn}})\odot {\bf Z}^p + ({\bf \alpha}^n{\bf 1}_{\text{attn}})\odot{\bf Z}^n$
/* Optimization */
$\mathcal{L}_0 \leftarrow \text{\textit{sign-aware} BPR loss}$
$\mathcal{L} \leftarrow {\mathcal{L}}_0+\lambda_{\text{reg}}\norm{\bf \Theta}^2$
Update $\Theta$ by taking one step of gradient descent
${\bf Z}$

§ PROPOSED  METHOD

In this section, we elaborate on our  method, designed for top-$K$ recommendation.  has the following three key components: 1) constructing a signed bipartite graph $G^s$ for more precisely representing users' preferences, which is split into two graphs $G^p$ and $G^n$ with positive and negative edges each, 2) generating two embeddings ${\bf Z}^p$ and ${\bf Z}^n$ for the partitioned graphs with positive and negative edges via a GNN model and an MLP, respectively, and then using the attention model to learn the importance of ${\bf Z}^p$ and ${\bf Z}^n$, and 3) establishing a sign-aware BPR loss function in the process of optimization. The overall procedure of the proposed  method is summarized in Algorithm <ref>.

As one of main contributions to the design of our method, we start by constructing the signed bipartite graph $G^s$ and then partitioning $G^s$ into two edge-disjoint graphs $G^p$ and $G^n$ for exploiting the relation of positively and negatively connected neighbors.

In the following subsections, we explain how we discover embeddings of all nodes (i.e., users and items) and optimize our model via the sign-aware BPR loss during the training phase.

§.§ Network Architecture

In this subsection, we describe how to generate the embeddings of $M$ users and $N$ items in our  method. As stated in Section III-B,  basically contains three learning models, i.e., $\text{GNN}_{\theta_1}$, $\text{MLP}_{\theta_2}$, and $\text{ATTENTION}_{\theta_3}$. To generally indicate either users or items interchangeably, we denote a node in the graphs $G^p$ and $G^n$, which can be in either $\mathcal{U}$ or $\mathcal{V}$, by $x$.

First, we describe the GNN model, which is designed to be model-agnostic. To this end, we show a general form of the message passing mechanism <cit.> in which we iteratively update the representation of each node by aggregating representations of its neighbors using two functions, namely $\text{AGGREGATE}_x^{\ell}$ and $\text{UPDATE}_x^{\ell}$, along with model parameters of $\text{GNN}_{\theta_1}$ in (<ref>). Formally, at the $\ell$-th layer of a GNN, $\text{AGGREGATE}^{\ell}_{x}$ aggregates (latent) feature information from the local neighborhood of node $x$ in $G^p$ at the $(\ell-1)$-th GNN layer as follows:

\begin{align}
\label{aggregate_}
{\bf m}_{x}^{\ell} \leftarrow \text{AGGREGATE}^{\ell}_{x}\big(\big\{ {\bf h}_{y}^{\ell-1}\big|y \in \mathcal{N}_{x} \cup \{x\}\big\}\big),
\end{align}

where ${\bf h}_{x}^{\ell-1}\in\mathbb{R}^{1\times d_{\text{GNN}}^{\ell-1}}$ denotes the $d_\text{GNN}^{\ell-1}$-dimensional latent representation vector of node $x$ at the $(\ell-1)$-th GNN layer, $\mathcal{N}_{x}$ is the set of neighbor nodes of $x$ in $G^p$, and ${\bf m}_{x}^{\ell}\in\mathbb{R}^{1\times d_\text{GNN}^{\ell-1}}$ is the aggregated information for node $x$ at the $\ell$-th GNN layer. Since $x$ belongs to a node in either $\mathcal{U}$ or $\mathcal{V}$, $\text{AGGREGATE}_{x}^{\ell}$ aggregates feature information of connected items if $x$ is a user node, and vice versa. In the update step, we use $\text{UPDATE}_{x}^{\ell}$ to obtain the $\ell$-th embedding vector ${\bf h}_{x}^{\ell}$ from the aggregated information ${\bf m}_{x}^{\ell}$ as follows:

\begin{align}
\label{update_}
{\bf h}_{x}^{\ell} \leftarrow \text{UPDATE}_{x}^{\ell}\big(x,{\bf m}_{x}^{\ell}\big).
\end{align}

We note that, for each node $x$, we randomly initialize the learnable 0-th embeddings (i.e., ${\bf h}^0_{x}$) due to the fact that we have no side information for users and items in our setting as in <cit.>. Additionally, we present another function in our GNN model, namely $\text{LAYER-AGG}_x^{L_{\text{GNN}}}$, which performs layer aggregation similarly as in <cit.>. This operation is motivated by the argument that oversmoothing tends to occur in GNN-based recommender systems if the last GNN layer's embedding vectors are used as the final embedding ${\bf Z}^p$ <cit.>. To alleviate the oversmoothing problem, we calculate the embedding vector ${\bf z}_x^p\in \mathbb{R}^{1\times d}$ of node $x$ via layer aggregation as follows:

\begin{align}
\label{layer_agg}
{\bf z}^p_x \leftarrow {\text{LAYER-AGG}}_x^{L_{\text{GNN}}} \left( {\big\{{\bf h}_x^{\ell}\big\}}_{\ell=0}^{\ell=L_{\text{GNN}}}\right),
\end{align}

which results in the embeddings ${\bf Z}^p$ for the graph $G^p$ where $L_\text{GNN}$ is the number of GNN layers. The GNN architecture of our  method including the above three functions $\text{AGGREGATE}_x^{\ell}$, $\text{UPDATE}_x^{\ell}$, and $\text{LAYER-AGG}_x^{L_{\text{GNN}}}$ is illustrated in Fig. <ref>.

The GNN architecture in our  method, composed of three functions in (<ref>)–(<ref>).

Now, let us state how the above three functions in (<ref>)–(<ref>) can be specified by several types of GNN-based recommender systems. As one state-of-the-art method,  NGCF <cit.> can be implemented by using

\begin{align}
\text{AGGREGATE}_{1,x}^{\ell} = &\text{ }W_{\text{GNN};1}^{\ell} {\bf h}_{x}^{\ell -1}\\
% \nonumber
\text{AGGREGATE}_{2,x}^{\ell}=&\sum_{y \in \mathcal{N}_{x}}\frac{1}{\sqrt{|\mathcal{N}_x||\mathcal{N}_y|}}\Big( W_{\text{GNN};1}^{\ell}{\bf h}_y^{\ell -1}\\
\nonumber
&+W_{\text{GNN};2}^{\ell}({\bf h}_x^{\ell -1}\odot {\bf h}_y^{\ell -1}) \Big) \\
\text{UPDATE}_x^{\ell} = &\text{ }\text{LeakyReLU}({\bf m}_{1,x}^{\ell}+{\bf m}_{2,x}^{\ell}) \\
{\text{LAYER-AGG}}_x^{L_{\text{GNN}}} = &\text{ }{\bf h}_x^0 {\big| \big|} {\bf h}_x^1 {\big| \big|} \cdot \cdot \cdot {\big| \big|}  {\bf h}_x^{L_{\text{GNN}}},
\end{align}

where $W_{\text{GNN};1}^{\ell},~ W_{\text{GNN};2}^{\ell}\in\mathbb{R}^{d_\text{GNN}^{\ell-1} \times d_\text{GNN}^{\ell}}$ are learnable weight transformation matrices, ${\big| \big|}$ is the concatenation operator, $L_{\text{GNN}}$ is the number of GNN layers, and $\text{LeakyReLU}$ is an activation function with a parameter $\alpha >0$:

\begin{align}
\text{LeakyReLU}(x)=\begin{cases}
x \text{ if $x>0$} \\
\alpha x \text{ otherwise}.
\end{cases}
\end{align}

In addition, as another state-of-the-art method for recommendation, LR-GCCF <cit.> can be implemented by using

\begin{align}
%		\text{AGGREGATE}_x^{\ell}&=\dfrac{1}{|\mathcal{N}_x|+1}{\bf h}_x^{\ell-1}+\sum_{y\in \mathcal{N}_x}\dfrac{1}{\sqrt{|\mathcal{N}_x|+1}\sqrt{|\mathcal{N}_y|+1}}{\bf h}_y^{\ell-1} \\
\text{AGGREGATE}_x^{\ell}&=\sum_{y\in \mathcal{N}_x\cup \{x\}}\dfrac{1}{\sqrt{|\mathcal{N}_x|+1}\sqrt{|\mathcal{N}_y|+1}}{\bf h}_y^{\ell-1} \\
\text{UPDATE}_x^{\ell}&={\bf m}^\ell_x\cdot W^{\ell}_{\text{GNN}} \\
{\text{LAYER-AGG}}_x^{L_{\text{GNN}}} &= {\bf h}_x^0 {\big| \big|} {\bf h}_x^1 {\big| \big|} \cdot \cdot \cdot {\big| \big|}  {\bf h}_x^{L_{\text{GNN}}},
\end{align}

where $W_{\text{GNN}}^{\ell}\in\mathbb{R}^{d_\text{GNN}^{\ell-1} \times d_\text{GNN}^{\ell}}$ is a learnable weight transformation matrix. As the most recently developed state-of-the-art method, LightGCN <cit.> can be specified according to the following function setting:

\begin{align}
\text{AGGREGATE}_x^{\ell}&=\sum_{y\in \mathcal{N}_x}\dfrac{1}{\sqrt{|\mathcal{N}_x|}\sqrt{|\mathcal{N}_y|}}{\bf h}^{\ell-1}_y \\
\text{UPDATE}_x^{\ell}&={\bf m}^\ell_x\\
\text{LAYER-AGG}_x^{L_{\text{GNN}}} &=\dfrac{1}{L_{\text{GNN}}+1}\sum_{\ell=0}^{L_{\text{GNN}}}{{\bf h}_x^{\ell}}.
\end{align}

Second, we pay our attention to the MLP architecture designed for the graph $G^n$ having negative edges. We calculate the embeddings ${\bf Z}^n$ using the MLP as follows:

\begin{align}
{\bf Z}^n_{\ell} &= \text{ReLU}\big( {\bf Z}_{\ell-1}^nW^{\ell}_{\text{MLP}}+{\bf 1}_{\text{MLP}}{\bf b}^{\ell}_{\text{MLP}}  \big) \\
{\bf Z}^n &=  {\bf Z}^n_{L_{\text{MLP}}},
% {\bf Z}^n = \text{ReLU} \Big(\big(\cdots \text{ReLU} (W_{\text{MLP}}^1  {\bf Z}_0^n+{\bf 1}_{\text{MLP}}{\bf b}_{\text{MLP}}^1)\big)W_{\text{MLP}}^{L_{\text{MLP}}}+{\bf 1}_{\text{MLP}}{\bf b}_{\text{MLP}}^{L_{\text{MLP}}}\Big),
% {\bf Z}^n = \text{ReLU} \Big(W_{\text{MLP}}^{L_{\text{MLP}}}\big(\cdots \text{ReLU} (W_{\text{MLP}}^1  {\bf Z}_0^n+{\bf b}_{\text{MLP}}^1)+{\bf b}_{\text{MLP}}^{L_{\text{MLP}}}\big)\Big),
\end{align}

where $\ell=1,2,\cdots,L_{\text{MLP}}$; $L_{\text{MLP}}$ is the number of MLP layers; $\text{ReLU}(x)=\max{(0,x)}$; $W_{\text{MLP}}^{\ell}\in\mathbb{R}^{d_\text{MLP}^{\ell-1} \times d_\text{MLP}^{\ell}}$ is a learnable weight transformation matrix; ${\bf b}_{\text{MLP}}^{\ell}\in \mathbb{R}^{1 \times d_\text{MLP}^{\ell}}$ is a bias vector; $d_\text{MLP}^{\ell}$ is the dimension of the latent representation vector ${\bf Z}_{\ell}^n$ at the $\ell$-th MLP layer; ${\bf 1}_{\text{MLP}}\in \mathbb{R}^{(M+N)\times 1}$ is the all-ones vector; and ${\bf Z}_0^n\in \mathbb{R}^{(M+N)\times d_\text{MLP}^{0}}$ is the learnable 0-th layer's embedding matrix for all nodes in the set $\mathcal{U}\cup \mathcal{V}$, which is randomly initialized. That is, ${\big\{W_{\text{MLP}}^{\ell}\big\}}_{\ell=1}^{\ell=L_{\text{MLP}}}$, ${\big\{{\bf b}_{\text{MLP}}^{\ell}\big\}}_{\ell=1}^{\ell=L_{\text{MLP}}}$, and ${\bf Z}_0^n$ correspond to the model parameters of $\text{MLP}_{\theta_2}$ in (<ref>).

Third, we turn to describing the attention model. The importance $({\bf \alpha}^p, {\bf \alpha}^n)$ in (<ref>) represents the attention values of two embeddings ${\bf Z}^p$ and ${\bf Z}^n$ for all nodes in $\mathcal{U}\cup \mathcal{V}$. Let us focus on node $x\in \mathcal{U}\cup \mathcal{V}$ whose embedding vectors calculated for the graphs $G^p$ and $G^n$ are given by ${\bf z}^p_{x},{\bf z}^n_{x}\in \mathbb{R}^{1\times d}$, respectively. Let $w_x^p$ and $w_x^n$ denote attention values of the two embeddings ${\bf z}^p_x$ and ${\bf z}^n_x$, respectively, for node $x$. Then, our attention model learns a weight transformation matrix $W_{attn}\in \mathbb{R}^{d' \times d}$, an attention vector ${\bf q}\in \mathbb{R}^{d' \times 1}$, and a bias vector ${\bf b}\in \mathbb{R}^{d' \times 1}$ with a dimension $d'$, corresponding to the model parameters of $\text{ATTENTION}_{\theta_3}$ in (<ref>), as follows:

\begin{align}
\label{weight_attn1}
w_x^p&={\bf q}^T tanh(W_{attn} {{\bf z}^p_x}^T + {\bf b}) \\
\label{weight_attn2}
w_x^n&={\bf q}^T tanh(W_{attn} {{\bf z}^n_x}^T + {\bf b}),
\end{align}

where $tanh(x)=\frac{\exp(x)-\exp(-x)}{\exp(x)+\exp(-x)}$ is the hyperbolic tangent activation function. By normalizing the attention values in (<ref>) and (<ref>) according to the softmax function, we have

\begin{align}
\alpha_{x}^p&=\frac{\text{exp}({w_x^p})}{\text{exp}(w_x^p)+\text{exp}(w_x^n)}\\
\alpha_{x}^n&=\frac{\text{exp}({w_x^n})}{\text{exp}(w_x^p)+\text{exp}(w_x^n)},
\end{align}

where $\alpha_{x}^p$ and $\alpha^n_{x}$ are the resulting importance of two embeddings ${\bf z}_x^p$ and ${\bf z}_x^n$, respectively, which thus yields the final embedding ${\bf z}_x = \alpha_{x}^p{\bf z}_x^p + \alpha_{x}^n{\bf z}_x^n$ for each node $x$ in (<ref>).

§.§ Optimization

In this subsection, we explain the optimization of  method in the training phase via our proposed loss function. We start by randomly initializing the learnable model parameters $\Theta=\{\theta_1, \theta_2, \theta_3\}$ in (<ref>)–(<ref>) (refer to line 1 in Algorithm <ref>). To train our learning models (i.e., the GNN, MLP, and attention models), we use a training set  $\mathcal{D}_S$ consisting of multiple samples of a triplet $(u,i,j)$, where $(u,i,w_{ui}^s)\in\mathcal{E}^s$ and $j\in\mathcal{V}$ is a negative sample (i.e., an unobserved item), which is not in the set of direct neighbors of user $u$ in the signed bipartite graph $G^s$ (refer to line 6). For sampling $j\in\mathcal{V}$, we use the degree-based noise distribution $P(j)\propto d_j^{3/4}$ <cit.>, where $d_j$ is the degree of item $j$. More specifically, when $N_{\text{neg}}$ denotes the number of negative samples, we first acquire the set of edges, $\mathcal{E}^s$, in $G^s$ and then sample $N_{\text{neg}}$ negative samples $\{j_n\}_{n=1}^{N_{\text{neg}}}$ for each $(u,i,w_{ui}^s)\in\mathcal{E}^s$  using the distribution $P(j_n)$ in order to create new samples of a node triplet $(u,i,j_n)$ for all $n\in\{1,\cdots,N_{\text{neg}}\}$, which yields the training set  $\mathcal{D}_S$ as follows:

\begin{align}
\label{data_batch}
\mathcal{D}_S=\big\{(u,i,j_n)\big|(u,i,w_{ui}^s)\in\mathcal{E}^s,j_n\notin \mathcal{N}_u,n\in\{1,\cdots,N_{\text{neg}}\}  \big\},
\end{align}

where $\mathcal{N}_u$ is the set of neighbor nodes of user $u$ in $G^s$. We further subsample mini-batches $\mathcal{D}_S'\subset\mathcal{D}_S$ to efficiently train our learning models (refer to line 7). The sampled triplets are fed into the loss function in the training loop along with the calculated embeddings $\bf Z$ for all nodes in $\mathcal{U}\cup \mathcal{V}$ (refer to lines 9–12).

Now, we present our sign-aware BPR loss function, which is built upon the original BPR loss <cit.> widely used in recommender systems (see <cit.> and references therein). To this end, we define a user $u$'s predicted preference for an item $i$ as the inner product of user and item final embeddings:

\begin{align}
\hat{r}_{ui} \triangleq {\bf z}_u{\bf z}_i^T,
\end{align}

which is used for establishing our loss function.

However, simply employing the original BPR loss is not desirable in our setting since it is a pairwise loss based on the relative order between observed and unobserved items by basically assuming that high ratings are more reflective of a user's preferences with higher prediction values of $\hat{r}_{ui}$ than the case of unobserved ones. On the other hand, our objective function should account for two types of observed items, which include both positive and negative relations between users and items, as well as unobserved ones. The proposed sign-aware BPR loss function is designed in such a way that the predicted preference for an observed item is higher than its unobserved counterparts along with the induced difference between high and low ratings (i.e., strong and weak user–item bonds).

To this end, more formally, we define a ternary relation $>_u(i,j,w)\triangleq \{(i,j,w)|\hat{r}_{ui}>\hat{r}_{uj}~\text{if}~w>0~\text{and}~\hat{r}_{ui}>{1 \over c}\hat{r}_{uj}~\text{otherwise}\} \subset\mathcal{V}\times \mathcal{V} \times (\mathbb{R}\setminus \{0\})$, where $c>1$ is a hyperparameter used for adjusting two levels of user preferences for observed items during training. In the relation, a higher value of $c$ indicates a larger gap between the degrees of user preferences for observed items. Based on the relation $>_u$, we aim at minimizing the following loss function $\mathcal{L}$ for a given mini-batch $\mathcal{D}_S'$ with the $L_2$ regularization:

\begin{align}
\label{lossfunction}
\mathcal{L}=\mathcal{L}_0+\lambda_{\text{reg}}\norm{\Theta}^2,
\end{align}

where $\lambda_{\text{reg}}$ is a hyperparameter that controls the $L_2$ regularization strength; $\Theta$ represents the model parameters; and $\mathcal{L}_0$ is the sign-aware BPR loss term realized by

\begin{align}
\label{sign_BPRloss}
\mathcal{L}_0=-\sum_{(u,i,j)\in\mathcal{D}_S'}{\log{p(>_u(i,j,w^s_{ui})\big| \Theta)}}.
\end{align}

Here, to capture the above relation $>_u(i,j,w)$ for each $(u,i,j)\in\mathcal{D}_S'$, we model the likelihood in (<ref>) as

\begin{align}
\label{likelihood} % cw
p(>_u(i,j,w^s_{ui})\big| \Theta)=\begin{cases}
\sigma \big(\hat{r}_{ui}-\hat{r}_{uj}\big) \text{ if $w^s_{ui}>0$} \\
\sigma \big(\text{c}\cdot\hat{r}_{ui}-\hat{r}_{uj}\big) \text{ otherwise,}
\end{cases}
\end{align}

where $\sigma(x)=\frac{1}{1+\exp(-x)}$ is the sigmoid function and $c>1$. Our model is trained through the loss function in (<ref>) towards the objective of (i) $\hat{r}_{ui}>\hat{r}_{uj}$ for pairs $(u,i)$ with high ratings and (ii) $\hat{r}_{ui}>\frac{1}{c}\hat{r}_{uj}$ for pairs $(u,i)$ with low ratings. In consequence, it is possible to more elaborately learn representations of nodes depending on both positive and negative relations.

§ EXPERIMENTAL EVALUATION

In this section, we first describe real-world datasets used in the evaluation. We also present five competing methods including two baseline MF methods and three state-of-the-art GNN-based methods for comparison. After describing performance metrics and our experimental settings, we comprehensively evaluate the performance of our  method and five benchmark methods. The source code for  can be accessed via https://github.com/woni-seo/SiReN-reco.

§.§ Datasets

We conduct experiments on three real-world datasets, which are widely adopted for evaluating recommender systems. For all experiments, we use user–item interactions with ratings in each dataset as the input. The main statistics of each dataset, including the number of users, the number of items, the number of ratings, the density, and the rating scale, are summarized in Table <ref>. In the following, we explain important characteristics of the datasets briefly.

MovieLens-1M (ML-1M)[https://grouplens.org/datasets/movielens/1m/.]. This is the most popular dataset in movie recommender systems, which consists of 5-star ratings (i.e., integer values from 1 to 5) of movies given by users <cit.>.

Amazon-Book[https://jmcauley.ucsd.edu/data/amazon/index.html.]. Among the Amazon-Review dataset containing product reviews and metadata, we select the Amazon-Book dataset, which consists of 5-star ratings <cit.>. We remove users/items that have less than 20 interactions similarly as in <cit.>.

Yelp[https://www.yelp.com/dataset.]. This dataset is a local business review data consisting of 5-star ratings. As in the Amazon-Book dataset, we remove users/items that have less than 20 interactions.

Dataset     ML-1M    Amazon-Book     Yelp

# of users ($M$)     6,040     35,736     41,772

# of items ($N$)     3,952     38,121     30,037

# of ratings     1,000,209     1,960,674   2,116,215

Density (%)    4.19   0.14   0.16

Rating scale    1–5   1–5   1–5

Statistics of three real-world datasets.

§.§ Benchmark Methods

In this subsection, we present two baseline MF methods and four state-of-the-art GNN methods for comparison.

BPRMF <cit.>. This baseline method is a MF model optimized by the BPR loss, which assumes that each user prefers the items with which he/she has interacted to items with no interaction.

NeuMF <cit.>. As another popular baseline, NeuMF is a neural CF model, which generalizes standard MF and uses multiple hidden layers to generate user and item embeddings.

NGCF <cit.>. This state-of-the-art GNN-based approach follows basic operations inherited from the standard GCN <cit.> to explore the high-order connectivity information. More specifically, NGCF stacks embedding layers and concatenates embeddings obtained in all layers to constitute the final embeddings.

LR-GCCF <cit.>. LR-GCCF is a state-of-the-art GCN-based CF model. As two main characteristics, this model uses only linear transformation without nonlinear activation and concatenates all layers' embeddings to alleviate oversmoothing at deeper layers <cit.>.

LightGCN <cit.>. LightGCN simplifies the design of GCN <cit.> to make the model more appropriate for recommendation by including only the most essential component such as neighborhood aggregation without nonlinear activation and weight transformation operations. Similarly as in LR-GCCF, this approach uses the weighted sum of embeddings learned at all layers as the final embedding.

SGCN <cit.>. SGCN is a generalized version of GCN <cit.> and harnesses the balance theory while capturing both positive and negative edges coherently in the aggregation process. Although SGCN was primarily aimed at conducting the link sign prediction task in signed unipartite graphs, it can also be applied to signed bipartite graphs.

enlarge x limits=0.25,

legend columns=-1,
legend pos=south east,
symbolic x coords=$P@10$,$R@10$,$nDCG@10$,

ymin=0.1, ymax=0.4, width=0.35,
nodes near coords align=vertical,
group style=
group size=3 by 1,
xlabels at=edge bottom,
ylabels at=edge left,
xticklabels at=edge bottom]

[bar width=0.3cm, xlabel=(a) ML-1M,legend style =  column sep = 5pt, legend columns = -1, legend to name = grouplegend,
legend to name=grouplegend,
legend entries=
-LightGCN, -LRGCCF, -NGCF
[style=black,fill=bblue,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.2774)+-(0.0037,0.0037) ($R@10$,0.2016)+-(0.0033,0.0033) ($nDCG@10$,0.3423)+-(0.0047,0.0047);
[style=black,fill=ggreen,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.2642)+-(0.0027,0.0027) ($R@10$,0.1944)+-(0.0017,0.0017) ($nDCG@10$,0.3244)+-(0.0037,0.0037);
[style=black,fill=ppurple,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.261)+-(0.0081,0.0081) ($R@10$,0.191)+-(0.0038,0.0038) ($nDCG@10$,0.3212)+-(0.0093,0.0093);
[bar width=0.3cm, xlabel=(b) Amazon-Book,ymax=0.09,ymin=0.01]
[style=black,fill=bblue,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0565)+-(0.002,0.002) ($R@10$,0.0775)+-(0.0008,0.0008) ($nDCG@10$,0.0823)+-(0.0013,0.0013);
[style=black,fill=ggreen,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0538)+-(0.001,0.001) ($R@10$,0.0742)+-(0.0022,0.0022) ($nDCG@10$,0.0771)+-(0.0022,0.0022);
[style=black,fill=ppurple,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0519)+-(0.0018,0.0018) ($R@10$,0.072)+-(0.001,0.001) ($nDCG@10$,0.0753)+-(0.0017,0.0017);
[bar width=0.3cm, xlabel=(c) Yelp,ymax=0.06,ymin=0.01]
[style=black,fill=bblue,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0352)+-(0.0007,0.0007) ($R@10$,0.0554)+-(0.0011,0.0011) ($nDCG@10$,0.0539)+-(0.0007,0.0007);
[style=black,fill=ggreen,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.034)+-(0.0011,0.0011) ($R@10$,0.0534)+-(0.0003,0.0003) ($nDCG@10$,0.0513)+-(0.0007,0.0007);
[style=black,fill=ppurple,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0333)+-(0.0009,0.0009) ($R@10$,0.0519)+-(0.0004,0.0004) ($nDCG@10$,0.05)+-(0.0009,0.0009);

at ($(group c2r1) + (0, 3cm) $) <ref>;

Performance comparison according to different GNN models  in our  method for each dataset.

§.§ Performance Metrics

To validate the performance of the proposed  method and the six benchmark methods, we adopt three metrics, which are widely used to evaluate the accuracy of top-$K$ recommendation. Let $Te_u^+$ and $R_u (K)$ denote the ground truth set (i.e., the set of items rated by user $u$ in the test set) and the top-$K$ recommendation list for user $u$, respectively. In the following, we describe each of metrics for recommendation accuracy.

The precision $P@K$ is defined as the ratio of relevant items to the set of recommended items and is expressed as

\begin{align}
P@K=\dfrac{1}{M}\sum_{u\in \mathcal{U}}{\dfrac{\big|Te_u^+\cap R_u(K) \big|}{K}}.
\end{align}

The recall $R@K$ is defined as the ratio of relevant items to the ground truth set and is expressed as

\begin{align}
R@K=\dfrac{1}{M}\sum_{u\in \mathcal{U}}{\dfrac{\big| Te_u^+ \cap R_u(K) \big|}{\big| Te_u^+ \big|}}.
\end{align}

The normalized discounted cumulative gain $nDCG@K$ <cit.> measures a ranking quality of the recommendation list by assigning higher scores to relevant items at top-$K$ ranking positions in the list:

\begin{align}
nDCG@K = \dfrac{1}{M}\sum_{u\in \mathcal{U}}{nDCG_u@K}.
\end{align}

Let $y_k$ be the binary relevance of the $k$-th item $i_k$ in $R_u(K)$ for each user $u$: $y_k=1$ if $i_k \in Te_u^+$ and $0$ otherwise. Then, $nDCG_u@K$ can be computed as

\begin{align}
\end{align}

where $DCG_u@K$ is

\begin{align}
\end{align}

and $IDCG_u@K$ indicates the ideal case of $DCG_u@K$ (i.e., all relevant items are at the top rank in $R_u(K)$). Note that all metrics are in a range of $[0,1]$, and higher values represent better performance.

§.§ Experimental Setup

In this subsection, we describe the experimental settings of neural networks in our  method. We implement  via PyTorch Geometric <cit.>, which is a geometric deep learning extension library in PyTorch. In our experiments, we adopt GNN models for the graph $G^p$ with positive edges and the 2-layer MLP architecture for the graph $G^n$ with negative edges. We use the Xavier initializer <cit.> to initialize the model parameters $\Theta = \{\theta_1, \theta_2, \theta_3\}$. We use dropout regularization <cit.> with the probability of 0.5 for the MLP and attention models in (<ref>) and (<ref>). We set the dimension of the embedding space and all hidden latent spaces to 64; the number of negative samples, $N_{\text{neg}}$, to 40;  the hyperparameter $c$ in (<ref>) to 2;[It was empirically found that the case of $c>2$ does not lead to high performance regardless of datasets even if it is not presented in this article for the sake of brevity.] and the strength of $L_2$ regularization, $\lambda_{\text{reg}}$, to 0.1 for the ML-1M dataset and 0.05 for the Amazon-Book and Yelp datasets. We train our model using the Adam optimizer <cit.> with a learning rate of 0.005.

For each dataset, we conduct 5-fold cross-validation by splitting it into two subsets: 80% of the ratings (i.e., user–item interactions) as the training set and 20% of the ratings as the test set. In the training set, when we implement five benchmark methods, we regard only the items with the rating scores of 4 and 5 as observed interactions by removing the ratings whose scores are lower than 4 as in <cit.>; however, when we implement our  method, we utilize all user–item interactions including low ratings in the training set while the parameter $w_o$, indicating the design criterion for signed bipartite graph construction, is set to 3.5.[Our empirical findings reveal that such a setting in  consistently leads to superior performance to that of other values of $w_o$ for all datasets having the 1–5 rating scale. This is obvious due to the fact that, in our experiments, the rating scores of 4 and 5 correspond to positive user–item interactions as well as the ground truth set.] It is worthwhile to note that, for fair comparison, the test set consists of only the ratings of 4 and 5 as the ground truth set for all the methods including .

§.§ Experimental Results

In this subsection, our empirical study is designed to answer the following four key research questions.

RQ1. How do underlying GNN models affect the performance of the  method?

RQ2. Which model architecture is appropriate to the graph $G^n$ with negative edges?

RQ3. How much does the  method improve the top-$K$ recommendation over baseline and state-of-the-art methods?

RQ4. How robust is our  method with respect to interaction sparsity levels?

To answer these research questions, we comprehensively carry out experiments in the following.

Comparative Study Among GNN Models Used for $G^p$ (RQ1)
In Fig. <ref>, for all datasets, we evaluate the accuracy of top-$K$ recommendation in terms of $P@K$, $R@K$, and $nDCG@K$ when $K$ is set to 10 while using various GNN models used for the graph $G^p$ in our  method. Since our method is GNN-model-agnostic, any existing GNN models can be adopted; however, in our experiments, we adopt three state-of-the-art GNN models that exhibit superior performance in recommender systems from the literature, namely NGCF <cit.> (-NGCF), LR-GCCF <cit.> (-LRGCCF), and LightGCN <cit.> (-LightGCN). From Fig. <ref>, we observe that -LightGCN consistently outperforms other models for all performance metrics. As discussed in <cit.>, this is because nonlinear activation and weight transformation operations in GNNs rather tend to degrade the recommendation accuracy; LightGCN thus attempted to simplify the design of GCN by removing such operations. It turns out that such a gain achieved by LightGCN is also possible in our  model that contains three learning models including the GNN, MLP, and attention models.

From these findings, we use -LightGCN in our subsequent experiments unless otherwise stated.

enlarge x limits=0.25,

legend columns=-1,
legend pos=south east,
symbolic x coords=$P@10$,$R@10$,$nDCG@10$,

ymin=0.1, ymax=0.4, width=0.35,
nodes near coords align=vertical,
group style=
group size=3 by 1,
xlabels at=edge bottom,
ylabels at=edge left,
xticklabels at=edge bottom]

[bar width=0.3cm, xlabel=(a) ML-1M,legend style =  column sep = 5pt, legend columns = -1, legend to name = grouplegend2,
legend to name=grouplegend2,
legend entries=$\textsf{SiReN}_{\text{MLP-}G^n}$,$\textsf{SiReN}_{\text{GNN-}G^n}$,$\textsf{SiReN}_{\text{No-}G^n}$
[style=black,fill=bblue,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.2774)+-(0.0037,0.0037) ($R@10$,0.2016)+-(0.0033,0.0033) ($nDCG@10$,0.3423)+-(0.0047,0.0047);
[style=black,fill=ggreen,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.2712)+-(0.0025,0.0025) ($R@10$,0.1985)+-(0.0038,0.0038) ($nDCG@10$,0.335)+-(0.0034,0.0034);
[style=black,fill=ppurple,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.2735)+-(0.0016,0.0016) ($R@10$,0.1986)+-(0.0039,0.0039) ($nDCG@10$,0.3375)+-(0.0028,0.0028);
[style=black,fill=ppink,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.2746)+-(0.0019,0.0019) ($R@10$,0.2005)+-(0.0039,0.0039) ($nDCG@10$,0.3389)+-(0.0031,0.0031);
[bar width=0.3cm, xlabel=(b) Amazon-Book,ymax=0.09,ymin=0.01]
[style=black,fill=bblue,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0565)+-(0.002,0.002) ($R@10$,0.0775)+-(0.0008,0.0008) ($nDCG@10$,0.0823)+-(0.0013,0.0013);
[style=black,fill=ggreen,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0508)+-(0.0018,0.0018) ($R@10$,0.0696)+-(0.0003,0.0003) ($nDCG@10$,0.0733)+-(0.0011,0.0011);
[style=black,fill=ppurple,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0523)+-(0.0018,0.0018) ($R@10$,0.0719)+-(0.0009,0.0009) ($nDCG@10$,0.0759)+-(0.0016,0.0016);
[style=black,fill=ppink,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0518)+-(0.0017,0.0017) ($R@10$,0.0713)+-(0.0007,0.0007) ($nDCG@10$,0.0754)+-(0.0012,0.0012);
[bar width=0.3cm, xlabel=(c) Yelp,ymax=0.06,ymin=0.01]
[style=black,fill=bblue,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0352)+-(0.0007,0.0007) ($R@10$,0.0554)+-(0.0011,0.0011) ($nDCG@10$,0.0539)+-(0.0007,0.0007);
[style=black,fill=ggreen,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0296)+-(0.0009,0.0009) ($R@10$,0.0467)+-(0.0009,0.0009) ($nDCG@10$,0.0447)+-(0.001,0.001);
[style=black,fill=ppurple,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.0297)+-(0.0012,0.0012) ($R@10$,0.0471)+-(0.0004,0.0004) ($nDCG@10$,0.0451)+-(0.0011,0.0011);
[style=black,fill=ppink,mark=none, error bars/.cd,
y dir=both,y explicit]
coordinates ($P@10$, 0.03)+-(0.0009,0.0009) ($R@10$,0.0473)+-(0.0008,0.0008) ($nDCG@10$,0.0456)+-(0.0009,0.0009);

at ($(group c2r1) + (0, 3cm) $) <ref>;

Performance comparison according to several model architectures used for the graph $G^n$ in our  method for each dataset.

Comparative Study Among Model Architectures Used for $G^n$ (RQ2)
We perform another comparative study four among model architectures used for the graph $G^n$ with negative edges. We adopted the MLP for $G^n$ since negative edges in $G^n$ can undermine the assortativity and thus message passing to such dissimilar nodes would not be feasible. In this experiment, we empirically validate this claim by taking into account three other design scenarios. We evaluate the accuracy of top-$K$ recommendation when $K$ is set to 10 for all datasets. First, we recall the original  method employing MLP for $G^n$, dubbed $\textsf{SiReN}_{\text{MLP-}G^n}$.

Second, instead of employing MLP, we introduce $\textsf{SiReN}_{\text{GNN-}G^n}$ that uses an additional GNN model for the graph $G^n$ to calculate the embedding vectors ${\bf Z}^n$:

\begin{align}
{\bf Z}^n = \text{GNN}_{\theta_1'}(G^n),
\end{align}

where $\theta_1'$ is the learned model parameters of $\text{GNN}$ for the graph $G^n$. In this experiment, we adopt LightGCN <cit.> among GNN models.

Third, as an ablation study, we introduce $\textsf{SiReN}_{\text{No-}G^n}$ that does not calculate the embedding vectors ${\bf Z}^n$. In other words, we only use the embedding vectors ${\bf Z}^p$ in (<ref>) as the final embeddings ${\bf Z}$ in (<ref>) (i.e., ${\bf Z} = {\bf Z}^p$). Note that $\textsf{SiReN}_{\text{No-}G^n}$ is identical to the model architecture of LightGCN <cit.>, which generates embedding vectors by aggregating only the information of positively connected neighbors. However, unlike LightGCN, $\textsf{SiReN}_{\text{No-}G^n}$ utilizes the sign-aware BPR loss in (<ref>)–(<ref>) as our objective function in the process of optimization.

Fourth, we present $\textsf{SiReN}_\text{No-split}$, a variant of  that does not partition the signed bipartite graph $G^s$ into the two edge-disjoint graphs. That is, $\textsf{SiReN}_\text{No-split}$ generates the embeddings ${\bf Z}$ in (<ref>) based on the model architecture of LightGCN <cit.> without distinguishing positive and negative edges. However, unlike LightGCN, $\textsf{SiReN}_\text{No-split}$ adopts the sign-aware BPR loss in the optimization.

From Fig. <ref>, our findings are as follows:

* $\textsf{SiReN}_{\text{MLP-}G^n}$ is always superior to $\textsf{SiReN}_{\text{GNN-}G^n}$ regardless of datasets and performance metrics, which indeed validates our claim addressed in Remark <ref>.

* $\textsf{SiReN}_{\text{MLP-}G^n}$ is always superior to $\textsf{SiReN}_\text{No-split}$ regardless of datasets and performance metrics. This implies that partitioning the signed bipartite graph into two edge-disjoint graphs can capture the user preferences more precisely.

* $\textsf{SiReN}_{\text{GNN-}G^n}$ exhibits the worst performance among the four model architectures used for $G^n$ for all cases. This implies that message passing to dissimilar nodes in $G^n$ is not effective.

* Although all the models are  trained via our sign-aware BPR loss, both positive and negative relations in the signed bipartite graph $G^s$ are not precisely captured during training unless an appropriate model architecture is designed for the graph $G^n$.

From these findings, we use $\textsf{SiReN}_{\text{MLP-}G^n}$ in our subsequent experiments unless otherwise stated.

1c           3c|$K=5$     3c|$K=10$      3c$K=15$

Dataset    Method     $P@K$     $R@K$     $nDCG@K$     $P@K$     $R@K$     $nDCG@K$    $P@K$     $R@K$     $nDCG@K$

BPRMF      0.2360$\pm$0.0058       0.0746$\pm$0.0017       0.2536$\pm$0.0081       0.1999$\pm$0.0044       0.1227$\pm$0.0019       0.2363$\pm$0.0063       0.1772$\pm$0.0033       0.1608$\pm$0.0016       0.2314$\pm$0.0053

NeuMF                0.2785$\pm$0.0054       0.0966$\pm$0.0011       0.299$\pm$0.0035       0.2397$\pm$0.0030       0.1599$\pm$0.0011       0.2847$\pm$0.0035       0.2138$\pm$0.0027       0.2098$\pm$0.0023       0.2827$\pm$0.0032

NGCF                 0.2973$\pm$0.0043       0.1099$\pm$0.0026       0.3238$\pm$0.0045       0.2477$\pm$0.0023       0.1748$\pm$0.0025       0.3031$\pm$0.0033       0.2174$\pm$0.0022       0.2229$\pm$0.0027       0.2985$\pm$0.0029

LR-GCCF             0.3052$\pm$0.0033       0.114$\pm$0.0023       0.333$\pm$0.0044       0.2539$\pm$0.0027       0.1802$\pm$0.0031       0.3117$\pm$0.0039       0.2220$\pm$0.0025       0.2292$\pm$0.0046       0.3066$\pm$0.0042

LightGCN            0.3218$\pm$0.0022       0.1206$\pm$0.0011       0.3519$\pm$0.0023       0.2679$\pm$0.0013       0.1909$\pm$0.0016       0.3297$\pm$0.0018       0.2349$\pm$0.0016       0.2432$\pm$0.0029       0.3249$\pm$0.0022

0.2484$\pm$0.0033       0.091$\pm$0.0017       0.2683$\pm$0.0035
0.1873$\pm$0.002       0.1492$\pm$0.0026       0.2547$\pm$0.0024      0.1873$\pm$0.002       0.1932$\pm$0.0037       0.253$\pm$0.0026

0.3328$\pm$0.0054       0.1279$\pm$0.0027       0.3637$\pm$0.0055
0.2774$\pm$0.0037       0.2016$\pm$0.0033       0.3423$\pm$0.0047       0.2444$\pm$0.0025       0.2569$\pm$0.0029       0.3388$\pm$0.004

BPRMF               0.0298$\pm$0.0039       0.0209$\pm$0.0035       0.0331$\pm$0.0046       0.0263$\pm$0.0033       0.0365$\pm$0.0057       0.0371$\pm$0.0053       0.0243$\pm$0.0029       0.0500$\pm$0.0074       0.0419$\pm$0.0058

NeuMF               0.0402$\pm$0.0021       0.0271$\pm$0.0007       0.0448$\pm$0.002       0.0339$\pm$0.0018       0.0452$\pm$0.0010       0.0483$\pm$0.0018       0.0303$\pm$0.0014       0.0597$\pm$0.0012       0.0530$\pm$0.0017

NGCF              0.0463$\pm$0.0014       0.032$\pm$0.0007       0.0518$\pm$0.0011         0.0391$\pm$0.0014       0.0532$\pm$0.0008       0.0562$\pm$0.0010       0.0403$\pm$0.0127       0.0706$\pm$0.0007       0.0618$\pm$0.0009

LR-GCCF             0.0469$\pm$0.0016       0.0324$\pm$0.0002       0.0527$\pm$0.0012      0.0399$\pm$0.0014       0.0544$\pm$0.0005       0.0574$\pm$0.0009       0.0357$\pm$0.0013       0.0721$\pm$0.0004       0.0631$\pm$0.0008

LightGCN            0.0529$\pm$0.0015       0.0362$\pm$0.0007       0.0596$\pm$0.0011      0.0443$\pm$0.0013       0.0595$\pm$0.0008       0.0638$\pm$0.0007       0.0393$\pm$0.0011       0.0781$\pm$0.0008       0.0698$\pm$0.0007

0.039$\pm$0.0023       0.0267$\pm$0.0012       0.0433$\pm$0.0024
0.0336$\pm$0.0018       0.0454$\pm$0.0018       0.0475$\pm$0.0023      0.0304$\pm$0.0016       0.0609$\pm$0.0021       0.0527$\pm$0.0024

0.0678$\pm$0.0025       0.0474$\pm$0.0005       0.0766$\pm$0.0002       0.0565$\pm$0.0025       0.0775$\pm$0.0008       0.0823$\pm$0.0013       0.0497$\pm$0.0018       0.1009$\pm$0.0011       0.0897$\pm$0.0011

BPRMF               0.0124$\pm$0.0013       0.0096$\pm$0.0009       0.0137$\pm$0.0013     0.0116$\pm$0.0014       0.0180$\pm$0.0020       0.0165$\pm$0.0017       0.0111$\pm$0.0013       0.0257$\pm$0.0026       0.0194$\pm$0.0021

NeuMF               0.0199$\pm$0.0011       0.015$\pm$0.0012       0.0226$\pm$0.0015      0.0174$\pm$0.0009       0.0262$\pm$0.0017       0.0254$\pm$0.0015       0.0159$\pm$0.0007       0.0358$\pm$0.0019       0.0288$\pm$0.0016

NGCF                0.0285$\pm$0.0012       0.0226$\pm$0.0007       0.0329$\pm$0.001      0.0243$\pm$0.0009       0.0383$\pm$0.0010       0.0368$\pm$0.0010       0.0219$\pm$0.0007       0.0515$\pm$0.0008       0.0413$\pm$0.0009

LR-GCCF           0.0303$\pm$0.0014       0.024$\pm$0.0006       0.0351$\pm$0.0013       0.0258$\pm$0.0010       0.0405$\pm$0.0009       0.0392$\pm$0.0011       0.0232$\pm$0.0008       0.0543$\pm$0.0010       0.0439$\pm$0.0012

LightGCN          0.0333$\pm$0.0011       0.0259$\pm$0.0004       0.0386$\pm$0.0009       0.0281$\pm$0.0009       0.0435$\pm$0.0007       0.0427$\pm$0.0008       0.0251$\pm$0.0008       0.0582$\pm$0.0010       0.0476$\pm$0.0008

0.0293$\pm$0.001       0.0226$\pm$0.0006       0.0332$\pm$0.001
0.0256$\pm$0.0008       0.0395$\pm$0.0011       0.0377$\pm$0.001      0.0232$\pm$0.0007       0.0538$\pm$0.0019       0.0426$\pm$0.0012

0.042$\pm$0.009       0.0332$\pm$0.0005       0.0488$\pm$0.0007       0.0352$\pm$0.0007       0.0554$\pm$0.0011       0.0539$\pm$0.0007       0.0314$\pm$0.0006       0.0737$\pm$0.0012       0.06$\pm$0.0006

Performance comparison among  and six benchmark methods in terms of three performance metrics (average $\pm$ standard deviation) when $K\in\{5,10,15\}$. Here, the best method for each case is highlighted using underlines.

1c             3c|$K=5$     3c|$K=10$      3c$K=15$

Dataset   Group    Method     $P@K$     $R@K$     $nDCG@K$     $P@K$     $R@K$     $nDCG@K$    $P@K$     $R@K$     $nDCG@K$

BPRMF             0.0625$\pm$0.0065       0.0882$\pm$0.0149       0.0858$\pm$0.0131       0.0485$\pm$0.0056       0.1348$\pm$0.0200       0.1046$\pm$0.0148       0.0433$\pm$0.0031       0.1810$\pm$0.0145       0.1223$\pm$0.0128

NeuMF               0.0842$\pm$0.0063       0.1185$\pm$0.0122       0.1158$\pm$0.0122       0.0675$\pm$0.0050       0.1873$\pm$0.0182       0.1449$\pm$0.0147       0.0576$\pm$0.0036       0.2397$\pm$0.0204       0.1653$\pm$0.0149

NGCF               0.0987$\pm$0.0077       0.1384$\pm$0.0157       0.1376$\pm$0.0142       0.0760$\pm$0.0040       0.2141$\pm$0.0126       0.1679$\pm$0.0129       0.0638$\pm$0.0015       0.2660$\pm$0.0100       0.1884$\pm$0.0101

LR-GCCF             0.1035$\pm$0.0062       0.1457$\pm$0.0131       0.1457$\pm$0.0153       0.0776$\pm$0.0037       0.2148$\pm$0.0145       0.1736$\pm$0.0162       0.0643$\pm$0.0025       0.2670$\pm$0.0139       0.1938$\pm$0.0147

LightGCN          0.1070$\pm$0.0047       0.1508$\pm$0.0107       0.1515$\pm$0.0104       0.0832$\pm$0.0029       0.2326$\pm$0.0116       0.1847$\pm$0.0108       0.069$\pm$0.0028       0.2878$\pm$0.0146       0.2063$\pm$0.0108

SGCN      0.0823$\pm$0.0079       0.1151$\pm$0.0084       0.1126$\pm$0.0096       0.0649$\pm$0.0065       0.1789$\pm$0.0158       0.1393$\pm$0.0122       0.0541$\pm$0.0051       0.224$\pm$0.0177       0.1566$\pm$0.0123

0.123$\pm$0.008       0.1755$\pm$0.0152       0.1756$\pm$0.0168
0.0933$\pm$0.0053       0.2606$\pm$0.0129       0.2106$\pm$0.0149      0.077$\pm$0.002       0.3223$\pm$0.0051       0.2345$\pm$0.0121

BPRMF              0.0877$\pm$0.0041       0.0825$\pm$0.0029       0.1052$\pm$0.0057       0.0731$\pm$0.0018       0.1358$\pm$0.0026       0.1194$\pm$0.0041       0.0635$\pm$0.0013       0.1773$\pm$0.0046       0.1364$\pm$0.0037

NeuMF                0.1245$\pm$0.0032       0.1196$\pm$0.0047       0.1488$\pm$0.004     0.1010$\pm$0.0020       0.1916$\pm$0.0052       0.1679$\pm$0.0038       0.0877$\pm$0.0014       0.2485$\pm$0.0061       0.1913$\pm$0.0041

NGCF                 0.1460$\pm$0.0027       0.1428$\pm$0.0052       0.1768$\pm$0.0059     0.1149$\pm$0.0021       0.2198$\pm$0.0073       0.1961$\pm$0.0060       0.0959$\pm$0.0015       0.2727$\pm$0.0086       0.2177$\pm$0.0061

LR-GCCF             0.1548$\pm$0.0036       0.1504$\pm$0.0041       0.1885$\pm$0.0047      0.1193$\pm$0.0026       0.2298$\pm$0.0050       0.2062$\pm$0.0048       0.0993$\pm$0.0022       0.2844$\pm$0.0077       0.2284$\pm$0.0055

LightGCN           0.1633$\pm$0.0027       0.1596$\pm$0.0037       0.2003$\pm$0.0037      0.1262$\pm$0.0020       0.2424$\pm$0.0044       0.2194$\pm$0.0037       0.1055$\pm$0.0015       0.3013$\pm$0.0055       0.2434$\pm$0.0036

SGCN       0.1204$\pm$0.0029       0.1179$\pm$0.0028       0.1451$\pm$0.0031       0.0985$\pm$0.0022       0.1915$\pm$0.0056       0.1657$\pm$0.0032       0.084$\pm$0.0026       0.2421$\pm$0.0084       0.1865$\pm$0.0043

0.1728$\pm$0.0036       0.1722$\pm$0.0044       0.2147$\pm$0.0046
0.1342$\pm$0.0026       0.2613$\pm$0.0053       0.2361$\pm$0.0044      0.1116$\pm$0.0018       0.322$\pm$0.0052       0.261$\pm$0.0044

BPRMF               0.3201$\pm$0.0079       0.0698$\pm$0.0019       0.3372$\pm$0.0099     0.2722$\pm$0.0068       0.1156$\pm$0.0035       0.3021$\pm$0.0083       0.2418$\pm$0.0052       0.1514$\pm$0.0037       0.2852$\pm$0.0071

NeuMF               0.3673$\pm$0.0091       0.0841$\pm$0.0028       0.385$\pm$0.0089      0.3194$\pm$0.0055       0.1430$\pm$0.0033       0.3514$\pm$0.0066       0.2862$\pm$0.0043       0.1893$\pm$0.0037       0.3355$\pm$0.0059

NGCF               0.3853$\pm$0.0061       0.0922$\pm$0.0025       0.4085$\pm$0.006      0.3247$\pm$0.0040       0.1505$\pm$0.0031       0.3649$\pm$0.0052       0.2875$\pm$0.0039       0.1962$\pm$0.0037       0.3459$\pm$0.0052

LR-GCCF           0.3931$\pm$0.0038       0.0944$\pm$0.0019       0.4167$\pm$0.0052      0.3321$\pm$0.0033       0.1544$\pm$0.0030       0.3730$\pm$0.0046       0.2930$\pm$0.0031       0.2006$\pm$0.0032       0.3531$\pm$0.0043

LightGCN         0.4146$\pm$0.0044       0.1$\pm$0.0024       0.4402$\pm$0.0052     0.3502$\pm$0.0026       0.1635$\pm$0.0029       0.3898$\pm$0.0109       0.3098$\pm$0.0024       0.2125$\pm$0.0034       0.3736$\pm$0.0043

SGCN      0.3227$\pm$0.0055       0.0766$\pm$0.0017       0.3393$\pm$0.0064       0.2763$\pm$0.0029       0.1273$\pm$0.0023       0.3393$\pm$0.0064       0.2472$\pm$0.0033       0.1682$\pm$0.003       0.2926$\pm$0.0048

0.4258$\pm$0.0075       0.1033$\pm$0.0029       0.4497$\pm$0.0079       0.3604$\pm$0.005       0.1688$\pm$0.0035       0.3896$\pm$0.0278       0.321$\pm$0.0033       0.2211$\pm$0.0035       0.3844$\pm$0.0055

21*90Amazon-Book   6*90$[0,20)$
BPRMF             0.0181$\pm$0.0027       0.0263$\pm$0.0052       0.0239$\pm$0.0042       0.0157$\pm$0.0021       0.0455$\pm$0.008       0.033$\pm$0.0054       0.0142$\pm$0.0019       0.0619$\pm$0.0103       0.0393$\pm$0.0062

NeuMF               0.0247$\pm$0.0022       0.0333$\pm$0.0014       0.032$\pm$0.0018       0.0199$\pm$0.0016       0.0542$\pm$0.0013       0.042$\pm$0.002       0.0171$\pm$0.0012       0.0699$\pm$0.0017       0.0482$\pm$0.0021

NGCF               0.0288$\pm$0.0016       0.0398$\pm$0.0014       0.0377$\pm$0.0012       0.0235$\pm$0.0012       0.0649$\pm$0.0019       0.0497$\pm$0.0014       0.0204$\pm$0.0011       0.0848$\pm$0.0015       0.0576$\pm$0.0013

LR-GCCF             0.0292$\pm$0.0017       0.0402$\pm$0.0007       0.0382$\pm$0.001       0.0241$\pm$0.0013       0.0665$\pm$0.0013       0.0508$\pm$0.0009       0.0208$\pm$0.0012       0.0863$\pm$0.001       0.0586$\pm$0.0009

LightGCN           0.0318$\pm$0.0015       0.0438$\pm$0.0012       0.042$\pm$0.0008       0.0258$\pm$0.0011       0.0712$\pm$0.0021       0.0551$\pm$0.0008       0.0224$\pm$0.0011       0.0923$\pm$0.0022       0.0635$\pm$0.0009

SGCN      0.0267$\pm$0.0069       0.0317$\pm$0.0038       0.0335$\pm$0.0054       0.0225$\pm$0.0063       0.053$\pm$0.0054       0.0427$\pm$0.003       0.0199$\pm$0.0058       0.0701$\pm$0.0063       0.0492$\pm$0.0025

0.0435$\pm$0.0028       0.06$\pm$0.0013       0.0575$\pm$0.001       0.0344$\pm$0.0024       0.0946$\pm$0.0013       0.0742$\pm$0.0014       0.0292$\pm$0.0019       0.1208$\pm$0.0018       0.0845$\pm$0.0013

BPRMF              0.0242$\pm$0.0031       0.0214$\pm$0.0035       0.0268$\pm$0.0038       0.0211$\pm$0.0026       0.0371$\pm$0.0058       0.032$\pm$0.0046       0.0194$\pm$0.0023       0.0508$\pm$0.0077       0.0383$\pm$0.0055

NeuMF                0.0337$\pm$0.0021       0.0281$\pm$0.0004       0.0374$\pm$0.002       0.0281$\pm$0.0019       0.0468$\pm$0.0008       0.0426$\pm$0.0015       0.0249$\pm$0.0016       0.0622$\pm$0.0010       0.0497$\pm$0.0015

NGCF                 0.0386$\pm$0.0019       0.033$\pm$0.0006       0.043$\pm$0.0014       0.0322$\pm$0.0016       0.0547$\pm$0.0007       0.0494$\pm$0.0008       0.0286$\pm$0.0014       0.0727$\pm$0.0009       0.0576$\pm$0.0009

LR-GCCF             0.0394$\pm$0.0016       0.0336$\pm$0.0005       0.0442$\pm$0.0011       0.033$\pm$0.0016       0.056$\pm$0.0007       0.0507$\pm$0.0007       0.0265$\pm$0.0013       0.0744$\pm$0.0006       0.0591$\pm$0.0007

LightGCN           0.0441$\pm$0.0017       0.0375$\pm$0.001       0.0496$\pm$0.0012       0.0364$\pm$0.0016       0.0615$\pm$0.0009       0.0562$\pm$0.0005       0.0319$\pm$0.0014       0.0807$\pm$0.0009       0.065$\pm$0.0005

SGCN      0.0324$\pm$0.0024       0.0275$\pm$0.0015       0.0358$\pm$0.0025       0.0277$\pm$0.0019       0.0469$\pm$0.002       0.0417$\pm$0.0022       0.0249$\pm$0.0016       0.0632$\pm$0.0025       0.0491$\pm$0.0024

0.0576$\pm$0.0024       0.0489$\pm$0.0011       0.0647$\pm$0.0018       0.0476$\pm$0.0022       0.0805$\pm$0.0016       0.0735$\pm$0.0006       0.0414$\pm$0.002       0.1049$\pm$0.0019       0.0846$\pm$0.0006

BPRMF               0.0551$\pm$0.0067       0.0143$\pm$0.0019       0.0574$\pm$0.007       0.0497$\pm$0.0058       0.0255$\pm$0.0034       0.0533$\pm$0.0063       0.0466$\pm$0.0052       0.0357$\pm$0.0046       0.0527$\pm$0.0062

NeuMF               0.0717$\pm$0.0034       0.0183$\pm$0.0007       0.0756$\pm$0.0036       0.0625$\pm$0.0026       0.0318$\pm$0.0007       0.0683$\pm$0.003       0.057$\pm$0.0023       0.0433$\pm$0.0008       0.0658$\pm$0.0025

NGCF               0.0827$\pm$0.0016       0.0215$\pm$0.0003       0.0875$\pm$0.0019       0.0718$\pm$0.002       0.0371$\pm$0.0003       0.0789$\pm$0.0019       0.0652$\pm$0.0014       0.0505$\pm$0.0004       0.0761$\pm$0.0012

LR-GCCF            0.0834$\pm$0.0031       0.0216$\pm$0.0004       0.0882$\pm$0.0033       0.0729$\pm$0.0022       0.0378$\pm$0.0002       0.0799$\pm$0.0026       0.0665$\pm$0.0019       0.0515$\pm$0.0004       0.0774$\pm$0.002

LightGCN         0.0958$\pm$0.0026       0.0249$\pm$0.0003       0.1019$\pm$0.0032       0.0823$\pm$0.0021       0.0423$\pm$0.0003       0.0911$\pm$0.0025       0.0747$\pm$0.0018       0.0574$\pm$0.0004       0.0877$\pm$0.0018

SGCN       0.0708$\pm$0.0036       0.0185$\pm$0.0007       0.0742$\pm$0.0038       0.0624$\pm$0.0035       0.0324$\pm$0.0015       0.0677$\pm$0.0037       0.0571$\pm$0.003       0.0442$\pm$0.002       0.0657$\pm$0.0033

0.1176$\pm$0.0046       0.0309$\pm$0.0006       0.1249$\pm$0.0055       0.1008$\pm$0.0032       0.0525$\pm$0.0008       0.1117$\pm$0.004       0.0909$\pm$0.0028       0.0706$\pm$0.0011       0.1071$\pm$0.0033

21*90Yelp   6*90$[0,20)$
BPRMF            0.0066$\pm$0.0009       0.0107$\pm$0.0012       0.0091$\pm$0.0013      0.0061$\pm$0.0009       0.0197$\pm$0.0024         0.0132$\pm$0.0018       0.0057$\pm$0.0008       0.0276$\pm$0.0031       0.0161$\pm$0.0022

NeuMF              0.0097$\pm$0.0011       0.0155$\pm$0.0018       0.0138$\pm$0.0016      0.0083$\pm$0.0009       0.0268$\pm$0.0024       0.0189$\pm$0.0020       0.0075$\pm$0.0007       0.0364$\pm$0.0026       0.0224$\pm$0.0021

NGCF               0.0152$\pm$0.0013       0.0244$\pm$0.0012       0.0219$\pm$0.0014      0.0128$\pm$0.0010       0.0413$\pm$0.0017       0.0295$\pm$0.0016       0.0114$\pm$0.0009       0.0552$\pm$0.0014       0.0347$\pm$0.0016

LR-GCCF             0.0162$\pm$0.0018       0.0259$\pm$0.0014       0.0232$\pm$0.0017     0.0134$\pm$0.0013       0.0434$\pm$0.0023       0.0310$\pm$0.0022       0.0120$\pm$0.0001       0.0584$\pm$0.0022       0.0366$\pm$0.0021

LightGCN           0.0171$\pm$0.0012       0.0274$\pm$0.0009       0.0248$\pm$0.0012     0.0144$\pm$0.0010       0.0462$\pm$0.0012       0.0332$\pm$0.0015       0.0127$\pm$0.0009       0.0615$\pm$0.0014       0.0389$\pm$0.0016

SGCN       0.0152$\pm$0.0012       0.0243$\pm$0.0014       0.0213$\pm$0.0013       0.0131$\pm$0.001       0.0422$\pm$0.0027       0.0293$\pm$0.0018       0.0119$\pm$0.0009       0.0574$\pm$0.0031       0.035$\pm$0.002

0.0224$\pm$0.0016       0.0359$\pm$0.001       0.0321$\pm$0.0012       0.0184$\pm$0.0012       0.0596$\pm$0.0016       0.0427$\pm$0.0014       0.0163$\pm$0.0011       0.079$\pm$0.002       0.05$\pm$0.0016

BPRMF             0.01$\pm$0.0013       0.0097$\pm$0.0008       0.0111$\pm$0.0013      0.0094$\pm$0.0014       0.0183$\pm$0.0019       0.0146$\pm$0.0016       0.0090$\pm$0.0012       0.0263$\pm$0.0025       0.0181$\pm$0.0019

NeuMF                0.0157$\pm$0.001       0.0152$\pm$0.0011       0.018$\pm$0.0011     0.0137$\pm$0.0008       0.0267$\pm$0.0017       0.0222$\pm$0.0013       0.0124$\pm$0.0008       0.0365$\pm$0.0020       0.0265$\pm$0.0015

NGCF                 0.0235$\pm$0.0012       0.0232$\pm$0.0008       0.0271$\pm$0.0009      0.0198$\pm$0.0010       0.0392$\pm$0.0011       0.0329$\pm$0.0009       0.0177$\pm$0.0009       0.0526$\pm$0.0010       0.0388$\pm$0.0009

LR-GCCF             0.0245$\pm$0.0015       0.0244$\pm$0.0005       0.0286$\pm$0.0012      0.0208$\pm$0.0011       0.0413$\pm$0.0007       0.0348$\pm$0.0008       0.0186$\pm$0.0009       0.0552$\pm$0.0010       0.0409$\pm$0.0009

LightGCN          0.0269$\pm$0.0014       0.0266$\pm$0.0005       0.0314$\pm$0.0011       0.0224$\pm$0.0012       0.0443$\pm$0.0008       0.0377$\pm$0.0009       0.0200$\pm$0.0010       0.0592$\pm$0.0013       0.0442$\pm$0.0009

SGCN       0.0237$\pm$0.0015       0.0231$\pm$0.0006       0.027$\pm$0.0012       0.0207$\pm$0.0012       0.0405$\pm$0.0009       0.0335$\pm$0.001       0.0187$\pm$0.0009       0.0549$\pm$0.0019       0.0398$\pm$0.0011

0.0344$\pm$0.0012       0.0339$\pm$0.0007       0.0402$\pm$0.0007       0.0287$\pm$0.0009       0.0567$\pm$0.0015       0.0482$\pm$0.0006       0.0254$\pm$0.0008       0.0753$\pm$0.0016       0.0564$\pm$0.0004

BPRMF               0.0256$\pm$0.0021       0.008$\pm$0.0005       0.0263$\pm$0.0017     0.0241$\pm$0.0024       0.0150$\pm$0.0015       0.0256$\pm$0.0020       0.0236$\pm$0.0025       0.0221$\pm$0.0024       0.0269$\pm$0.0024

NeuMF              0.0435$\pm$0.0024       0.0137$\pm$0.0008       0.0456$\pm$0.0029      0.0384$\pm$0.0018       0.0240$\pm$0.0014       0.0420$\pm$0.0023       0.0352$\pm$0.0013       0.0331$\pm$0.0014       0.0423$\pm$0.0020

NGCF              0.0584$\pm$0.0023       0.0187$\pm$0.0004       0.0617$\pm$0.0023      0.0506$\pm$0.0016       0.0323$\pm$0.0004       0.0562$\pm$0.0017       0.0458$\pm$0.0012       0.0439$\pm$0.0003       0.0562$\pm$0.0012

LR-GCCF         0.0629$\pm$0.0018       0.0201$\pm$0.0004       0.0671$\pm$0.002        0.0542$\pm$0.0012       0.0346$\pm$0.0003       0.0608$\pm$0.0014      0.0492$\pm$0.0013       0.047$\pm$0.0006       0.0606$\pm$0.0013

LightGCN         0.0699$\pm$0.0015       0.0222$\pm$0.0004       0.0748$\pm$0.0015     0.0599$\pm$0.0013       0.0380$\pm$0.0004       0.0675$\pm$0.0012       0.0540$\pm$0.0013       0.0514$\pm$0.0004       0.0670$\pm$0.0008

SGCN        0.0616$\pm$0.0008       0.0194$\pm$0.0004       0.0645$\pm$0.0011     0.054$\pm$0.0008       0.0341$\pm$0.0008       0.0593$\pm$0.0009       0.0491$\pm$0.0009       0.0465$\pm$0.0011       0.0593$\pm$0.0009

0.0864$\pm$0.001       0.0277$\pm$0.0004       0.0924$\pm$0.0009       0.0732$\pm$0.0011       0.047$\pm$0.001       0.0829$\pm$0.0009       0.0656$\pm$0.0008       0.0629$\pm$0.0013       0.082$\pm$0.0008

Performance comparison among  and six benchmark methods in terms of three performance metrics (average $\pm$ standard deviation) according to three different user groups  when $K\in \{5, 10, 15\}$. Here, the best method for each case is highlighted using underlines.

Comparison With Benchmark Methods (RQ3)
The performance comparison between our  method and four state-of-the-art GNN methods, including NGCF <cit.>, LR-GCCF <cit.>, LightGCN <cit.>, and SGCN <cit.>  as well as two baseline MF methods, including BPRMF <cit.> and NeuMF <cit.>, for top-$K$ recommendation is comprehensively presented in Table <ref> with respect to three performance metrics using three real-world datasets, where  $K\in\{5,10,15\}$.  We note that the hyperparameters in all the aforementioned benchmark methods are tuned differently according to each dataset so as to provide the best performance. In Table <ref>, the value with an underline indicates the best performer for each case. We would like to make the following insightful observations:

* Our  method consistently outperforms five benchmark methods for all datasets regardless of the performance metrics and the values of $K$. The superiority of our method comes from the fact that we are capable of more precisely representing users' preferences without any information loss. This implies that low ratings are indeed informative as long as the low rating information is well exploited through judicious model design and optimization.

* The second best performer is LightGCN for all the cases. The performance gap between our  method ($X$) and the second best performer ($Y$) is the largest when the Amazon-Book dataset is used; the maximum improvement rates of 28.16%, 30.93%, and 28.52% are achieved in terms of $P@5$, $R@5$, and $nDCG@5$, respectively, where the improvement rate ($\%$) is given by $\frac{X-Y}{Y}\times 100$. We recall that the Amazon-Book dataset has the lowest density (i.e., the highest sparsity) out of three datasets (refer to Table <ref>). Thus, from the above empirical finding, it is seen that exploiting negative user–item interactions in sparser datasets would be more beneficial and effective in improving the recommendation accuracy.

* SGCN is far inferior to some of state-of-the-art GNN methods (e.g., LR-GCCF and LightGCN). This implies that performance of the GNN model built upon the balance theory is not effective compared to other competing GNN-based recommender systems.

* Two baseline MF methods reveal worse performance than that of three state-of-the-art GNN methods including NGCF, LR-GCCF, and LightGCN. This indicates that exploring the high-order connectivity information via GNNs indeed significantly improves the recommendation accuracy.

* The gain of LightGCN over LR-GCCF and NGCF is consistently observed. We note that, in contrast to LR-GCCF and NGCF, LightGCN aggregates the information of neighbors without both nonlinear activation and weight transformation operations. Our experimental results coincide with the argument in <cit.> that a simple aggregator of the information of neighbors using a weighted sum is the most effective as long as GNN-based recommender systems are associated.

Robustness to Interaction Sparsity Levels (RQ4)
Needless to say, the sparsity issue is one of crucial challenges on designing recommender systems since few user–item interactions are insufficient to generate high-quality embeddings <cit.>. In this experiment, we demonstrate that making use of low rating scores for better representing users’ preferences enables us to alleviate this sparsity issue. To this end, we partition the set of users in the test set into three groups according to the number of interactions in the training set as in <cit.>. More precisely, for each dataset, we split the users into three groups, each of which is composed of the users whose number of interactions in the training set ranges between $[0,20)$, $[20,50)$, and $[50,\infty)$, respectively. In Table  <ref>, we comprehensively carry out the performance comparison between our  method and six benchmark methods with respect to three performance metrics of top-$K$ recommendation using the ML-1M, Amazon-Book, and Yelp datasets, where experimental results are shown according to three interaction sparsity levels for each dataset. Our findings can be summarized as follows:

* Our  method consistently outperforms all the benchmark method. This demonstrates that exploiting the set of negative interactions in designing GNN-based recommender systems is useful in improving recommendation accuracy regardless of interaction sparsity levels.

* The performance gap between our  method and the second best performer is the largest for the user group having $[0,20)$ interactions in the Amazon-Book dataset; the maximum improvement rates of 36.79%, 36.98%, and 36.9% are achieved in terms of $P@5$, $R@5$, and $nDCG@5$, respectively. As stated above, the performance improvement of  over competing methods is significant when sparse datasets are used.

* As the number of interactions per user increases, the performance is likely to be enhanced for all the methods regardless of types of datasets. It is obvious that more user–item interactions yield higher recommendation accuracy.

§ CONCLUDING REMARKS

In this paper, we explored a fundamentally important problem of how to take advantage of both high and low rating scores in developing GNN-based recommender systems. To tackle this challenge, we introduced a novel method, termed , that is designed based on sign-aware learning and optimization models along with a GNN architecture. Specifically, we presented an approach to 1) constructing a signed bipartite graph $G^s$ to distinguish users' positive and negative feedback and then partitioning $G^s$ into two edge-disjoint graphs $G^p$ and $G^n$ with positive and negative edges each, respectively, 2) generating two embeddings for $G^p$ and $G^n$ via a GNN model and an MLP, respectively, and then using an attention model to discover the final embeddings, and 3) training our learning models by establishing a sign-aware BPR loss function that captures each relation of positively and negatively connected neighbors. Using three real-world datasets, we demonstrated that our  method remarkably outperforms  four  state-of-the-art GNN methods as well as two baseline MF methods while showing gains over the second best performer (i.e., LightGCN) by up to  30.93%  in terms of the recommendation accuracy.  We also demonstrated that our proposed method is robust to more challenging situations according to interaction sparsity levels by investigating that the performance improvement of  over state-of-the-art methods is significant when sparse datasets are used. Additionally, we empirically showed the effectiveness of MLP used for the graph $G^n$ with negative edges.

Potential avenues of future research include the design of a more sophisticated GNN model that fits well into $G^n$ in signed bipartite graphs. Here, the challenges lie in developing a new information aggregation and propagation mechanism. Furthermore, the design of multi-criteria recommender systems utilizing sign awareness remains for future work.

§ ACKNOWLEDGMENTS

The work of W.-Y. Shin was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2021R1A2C3004345) and by the Yonsei University Research Fund of 2021 (2021-22-0083). The work of S. Lim was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government (MSIT) (No. 2020-0-01441, Artifical Intelligence Convergence Research Center (Chungnam National University)). The authors would like to thank Dr. Hajoon Ko from Harvard University for his helpful comments.