# HAKG: Hierarchy-Aware Knowledge Gated Network for Recommendation

Yuntao Du1, Xinjun Zhu2, Lu Chen1, Baihua Zheng3, and Yunjun Gao1∗ 1College of
Computer Science, Zhejiang University, Hangzhou, China
2School of Software, Zhejiang University, Ningbo, China
3School of Computing and Information Systems, Singapore Management University,
Singapore<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
(2022)

###### Abstract.

Knowledge graph (KG) plays an increasingly important role to improve the
recommendation performance and interpretability. A recent technical trend is
to design end-to-end models based on information propagation schemes. However,
existing propagation-based methods fail to (1) model the underlying
hierarchical structures and relations, and (2) capture the high-order
collaborative signals of items for learning high-quality user and item
representations.

In this paper, we propose a new model, called Hierarchy-Aware Knowledge Gated
Network (HAKG), to tackle the aforementioned problems. Technically, we model
users and items (that are captured by a user-item graph), as well as entities
and relations (that are captured in a KG) in hyperbolic space, and design a
hyperbolic aggregation scheme to gather relational contexts over KG.
Meanwhile, we introduce a novel angle constraint to preserve characteristics
of items in the embedding space. Furthermore, we propose a dual item
embeddings design to represent and propagate collaborative signals and
knowledge associations separately, and leverage the gated aggregation to
distill discriminative information for better capturing user behavior
patterns. Experimental results on three benchmark datasets show that, HAKG
achieves significant improvement over the state-of-the-art methods like CKAN,
Hyper-Know, and KGIN. Further analyses on the learned hyperbolic embeddings
confirm that HAKG offers meaningful insights into the hierarchies of data.

Recommendation, Graph Neural Network, Knowledge Graph

∗Yunjun Gao is the corresponding author.

††journalyear: 2022††copyright: acmcopyright††conference: Proceedings of the
45th International ACM SIGIR Conference on Research and Development in
Information Retrieval; July 11–15, 2022; Madrid, Spain.††booktitle:
Proceedings of the 45th International ACM SIGIR Conference on Research and
Development in Information Retrieval (SIGIR ’22), July 11–15, 2022, Madrid,
Spain††price: 15.00††isbn: 978-1-4503-8732-3/22/07††doi:
10.1145/XXXXXX.XXXXXX††ccs: Information systems Recommender systems

##  1\. Introduction

(a) Knowledge-aware recommendation

(b) Distribution of interactions

Figure 1. (a) An example of knowledge-aware recommendation. The blue lines
indicate the hierarchical relations, while the black lines denote the non-
hierarchical relations. (b) Degree distribution of Alibaba-iFashion dataset.

In the era of information explosion, recommender systems have become an
essential part of Internet applications to provide personalized information
services. Traditional recommender systems that are based on collaborative
filtering (Koren, 2008; Hu et al., 2008; Wang et al., 2019b; Gharibshah et
al., 2020) usually suffer from data sparsity and cold-start problems.
Recently, knowledge graph (KG), which provides various real-world facts
related to items via relations, has demonstrated an impressive ability to
alleviate cold-start issue and improve the explainability of recommendations.

Learning user and item representations from auxiliary KG has become the term
of knowledge-aware recommendation. Early studies (Zhang et al., 2016; Wang et
al., 2018b; Ai et al., 2018) directly integrate knowledge graph embeddings
with items to enhance their representations. Some subsequent studies
(Catherine and Cohen, 2016; Hu et al., 2018; Wang et al., 2019c; Jin et al.,
2020) enrich the interactions via meta-paths from users to items for better
identifying user-item connectivities. However, in order to obtain informative
paths, these methods suffer from labor-intensive process (Wang et al., 2019c),
poor generalization (Hu et al., 2018; Zhu et al., 2018), and unstable
performance (Xian et al., 2019). Recently, the success of Graph Neural
Networks (GNNs) (Kipf and Welling, 2017; Hamilton et al., 2017; Veličković et
al., 2018) has inspired the community to develop end-to-end models based on
the information aggregation schemes (Wang et al., 2018a, 2019a, 2019d, 2020a,
2021; Tu et al., 2021; Chen et al., 2022; Du et al., 2022). The key idea is to
iteratively propagate high-order information over KG, which could effectively
integrate multi-hop neighbors into representations and hence improve the
recommendation performance.

Although existing propagation-based methods are able to achieve good
performance, we would like to highlight that they all fail to model the
following two important factors.

  * •

Hierarchical Structures and Relations. Existing methods model both user-item
interactions and KG in Euclidean space, while both data structures exhibit a
highly non-Euclidean latent anatomy. Specifically, the user-item interactions
typically follow the power-law distribution (as shown in Figure 1(b)),
indicating the underlying hierarchical structures (Vinh Tran et al., 2020;
Zhang et al., 2021; Chen et al., 2022). Meanwhile, hierarchical information is
ubiquitous in real-world KGs, since human knowledge is organized
hierarchically (Chami et al., 2020; Bai et al., 2021). Euclidean-based methods
are insufficient to capture the intrinsic hierarchical structures of the data,
since they suffer from a high distortion when embedding hierarchical data
(Nickel and Kiela, 2017; Ganea et al., 2018a; Chami et al., 2019). Moreover,
none of them considers KG relations at a finer-grained level of hierarchies.
They all conveniently ignore an important fact, _i.e.,_ hierarchical relations
(blue lines in Figure 1(a)) and non-hierarchical relations (black lines in
Figure 1(a)) are not of equal importance for characterizing items. Taking the
outfit recommendation in Figure 1(a) as an example. Hierarchical relation
$r_{1}$ offers complementary information to the cloth $i_{1}$ in the aspect
that $i_{1}$ is the jeans $e_{1}$ while non-hierarchical relation $r_{5}$
means that cloth $i_{2}$ and cloth $i_{3}$ can compose an outfit. Thus,
hierarchical relations can indicate the attributes of items while non-
hierarchical relations only reveal the relatedness between entities. Ignoring
the hierarchical structures and relations limits model’s expressive power.

  * •

High-order Collaborative Signals of Items. In existing studies, the item
aggregation schemes are mostly KG-oriented, that is, recursively collecting
items’ knowledge associations from KG without considering the collaborative
signals from users (Wang et al., 2020a, 2021), or blindly mixing the
heterogeneous information from neighboring nodes (users or entities) in the
Unified Knowledge Graph (UKG) (Wang et al., 2019a; Tu et al., 2021). They fail
to explicitly encode the crucial high-order collaborative signals of items ,
which are latent in user-item interactions and play an important role in
learning user preference from behavior aspect. Take Figure 1(a) as an example.
The path $i_{1}\rightarrow u_{1}\rightarrow i_{3}\rightarrow u_{2}\rightarrow
i_{2}$ indicates the long-range connectivity between item $i_{1}$ and item
$i_{2}$: since users $u_{1}$ and $u_{2}$ share the same interest (_i.e.,_ they
both like item $i_{3}$), the items $i_{1}$ and $i_{2}$ which are favored by
user $u_{1}$ and user $u_{2}$ respectively may be similar to some extent.
Therefore, such KG-oriented aggregation schemes are insufficient to explicitly
capture the high-order collaborative signals for comprehensive item
representations.

To tackle the aforementioned challenges, a new latent space with a smaller
embedding distortion is required. Hyperbolic geometry offers an ideal
alternative, as it not only enables embeddings with much smaller distortion
(Ganea et al., 2018a; Chami et al., 2019) but also naturally preserves the
data hierarchy (Chami et al., 2020; Zhang et al., 2021). The key property of
hyperbolic space is that, unlike Euclidean space, it expands exponentially
rather than polynomially. This allows hyperbolic space to have much larger
data capacity than Euclidean space, and hence, it has “sufficient” room to
better preserve the distance between hierarchical data, as shown in the left
of Figure 2. Besides, hyperbolic space can be viewed as a continuous version
of trees, which makes it perfect to model hierarchical tree-like data due to
their analogous structure. Thus, we propose Hierarchy-Aware Knowledge Gated
Network (HAKG), a new hyperbolic knowledge-aware model with two key
components, which can effectively capture and model the aforementioned two
important factors that are not modeled by any existing methods.

  * •

Hierarchy-Aware Modeling. To better capture the underlying hierarchical
structures, we map user and item embeddings as well as entity and relation
embeddings to hyperbolic space. We also design a new hyperbolic relation-
transitive aggregation mechanism to capture the relation dependencies carried
by neighbors in hyperbolic space. Moreover, a novel angle constraint is
introduced for hierarchical relations, which is able to better preserve items’
attributes information in the embedding space and thus improve the
representational capacity and expressiveness.

  * •

Gated Aggregation with Dual Embeddings. Unlike previous KG-oriented
aggregation strategies, we view user-item interactions and KG as two different
information channels, and develop different aggregation strategies for the two
channels. Besides, since items can serve as a natural bridge to connect the
two information channels, we use dual embedding instead of single embedding
for items to represent and propagate each channel’s information separately, so
as to better capture the holistic semantics of items. Moreover, an information
gated mechanism is introduced to adaptively fuse the two types of semantics
for better identifying user behavior patterns.

Figure 2. (a) Hyperbolic space expands exponentially. (b) Illustration of
Hyperbolic cones.

To this end, the newly proposed HAKG framework is designed to i) leverage the
expressiveness of hyperbolic geometry for better hierarchical modeling, and
ii) effectively capture the holistic semantics of items. We conduct extensive
experiments on three real-world datasets to evaluate the performance of HAKG
and existing methods. Experimental results show that our HAKG significantly
outperforms all the start-of-the-art methods such as CKAN (Wang et al.,
2020a), Hyper-Know (Ma et al., 2021), and KGIN (Wang et al., 2021).
Furthermore, HAKG is able to reveal the underlying hierarchical structures and
relations of data in the embedding space for better model expressiveness.

In summary, our contributions are as follows:

  * •

We present knowledge-aware recommendation from a new perspective by taking the
hierarchy and high-order items’ collaborative signals into consideration.

  * •

We embed users and items, as well as entities and relations in hyperbolic
space, and design a new hyperbolic aggregation in KG to preserve the relation
dependencies among neighbors. Besides, an angle constraint is introduced for
hierarchical relations to profile items with the attributes in the embedding
space.

  * •

We devise the dual item embeddings design to i) better represent and propagate
collaborative signals and knowledge associations simultaneously, and ii) more
effectively leverage the information gate mechanism to control discriminative
signals towards the users’ preference from both behavior and attribute
aspects.

  * •

We conduct extensive experiments on three public benchmark datasets to
demonstrate the superiority of HAKG.

##  2\. Problem Formulation

We first introduce the data structures related to our studied problem, and
then formulate our task.

User-Item Bipartite Graph. In this paper, we focus on learning the user
preference from implicit feedback (Rendle et al., 2009). To be more specific,
the behavior data (_e.g.,_ click and review) involves a set of users
$\mathcal{U}=\\{u\\}$ and a set of items $\mathcal{I}=\\{i\\}$. We model user-
item interactions as a bipartite graph
$\mathcal{G}_{b}=\\{(u,i)|u\in\mathcal{U},i\in\mathcal{I}\\}$ where each
$(u,i)$ pair indicates that user $u$ has interacted with item $i$.

Knowledge Graph (KG). KGs are collections of real-world facts, such as item
attributes, concepts, or external commonsense. Let $\mathcal{T}$ be the
triplet set, $\mathcal{E}$ be a set of entities, and $\mathcal{R}$ be the
relation set, which involves relations in both canonical and inverse
directions (_e.g.,_ compose and composed-of). Let KG be a heterogeneous graph
$\mathcal{G}_{k}=\\{(h,r,t)|h,t\in\mathcal{E},r\in\mathcal{R}\\}$, where each
triplet $(h,r,t)\in\mathcal{T}$ means that there is a relation $r$ between
head entity $h$ and tail entity $t$. For example, a triplet (jeans, brand,
Levi’s) indicates that the brand of the jeans is Levi’s. As we assume all the
items appear in KG as entities (i.e., $\mathcal{I}\subset\mathcal{E}$), a
common assumption made by all existing knowledge-aware recommendation systems
(Wang et al., 2019e, a, 2021), we can link items in user-item graph with
entities in KG to offer auxiliary semantics to interactions.

Task Description. Given a user-item graph $\mathcal{G}_{b}$ and a KG
$\mathcal{G}_{k}$, our task of knowledge-aware recommendation is to predict
how likely that a user would adopt an item that she has never engaged with.

##  3\. Methodology

Figure 3. Illustration of the proposed HAKG framework.

In this section, we first introduce the basic concepts about hyperbolic space,
and then detail two key components of HAKG, _i.e.,_ hierarchy-aware modeling
and gated aggregation with dual embeddings. The former embeds users and items
as well as entities and relations in hyperbolic space. It adopts a novel
hyperbolic relation-aware aggregation over KG, and also introduces an angle
constraint for learning attribute semantics of item in the embedding space.
The latter leverages dual item embeddings to separately encode the high-order
collaborative signals from user-item graph and the knowledge associations from
KG, and distills useful semantics via the gate mechanism for learning high-
quality user representations.

###  3.1. Preliminaries

Hyperbolic geometry (Cannon et al., 1997) is a non-Euclidean geometry with
constant negative curvature. Compared with the Euclidean spaces, the amount of
space covered by the hyperbolic geometry increases exponentially rather than
polynomially _w.r.t._ the radius. This property allows us to effectively
capture underlying hierarchical structures of user-item interactions and KG in
hyperbolic space. In this work, we use the Poincaré ball model with constant
curvature $-c=-1$ for learning hyperbolic embeddings due to its feasibility
for gradient optimization (Nickel and Kiela, 2017; Zhang et al., 2021). In the
following, we first introduce the necessary mathematical basis for the
Poincaré ball model.

Poincaré Ball & Tangent Space. The definition domain of the Poincaré ball
model is:

(1)
$\displaystyle\mathbb{B}=\\{(x_{1},\ldots,x_{n}):x_{1}^{2}+\cdots+x_{n}^{2}<\frac{1}{c}\\}$

in $\mathbb{R}^{n}$. The tangent space $\mathcal{T}_{\textbf{z}}\mathbb{B}$ at
point $z$ on $\mathbb{B}$ is a $n$-dimensional Euclidean space that best
approximates $\mathbb{B}$ around $z$, which is useful to perform aggregation
operations in hyperbolic space (Chami et al., 2019).

Exponential Map & Logarithmic Map. Hyperbolic space and tangent space can be
bridged by exponential and logarithmic mappings. Specifically, the exponential
map can map the tangent space $\mathcal{T}_{\textbf{z}}\mathbb{B}$ to the
hyperbolic space $\mathbb{B}$, and the logarithmic map maps $\mathbb{B}$ to
$\mathcal{T}_{\textbf{z}}\mathbb{B}$ conversely. In particular, the closed-
form expressions for the two maps at point $z$ are defined below.

(2)
$\displaystyle\exp_{\textbf{z}}(\textbf{x})=\textbf{z}\oplus\tanh(\frac{\|\textbf{x}\|}{1-\|\textbf{z}\|^{2}})\frac{\textbf{x}}{\|\textbf{x}\|}$
(3)
$\displaystyle\log_{\textbf{z}}(\textbf{y})=(1-\|\textbf{z}\|^{2})\cdot\tanh^{-1}(\|-\textbf{z}\oplus\textbf{y}\|)\frac{-\textbf{z}\oplus\textbf{y}}{\|-\textbf{z}\oplus\textbf{y}\|}$

where $\oplus$ represents Möbius addition (Ganea et al., 2018a):

(4)
$\displaystyle\textbf{x}\oplus\textbf{y}=\frac{(1+2\langle\textbf{x},\textbf{y}\rangle+\|\textbf{y}\|^{2})\textbf{x}+(1-\|\textbf{x}\|^{2})\textbf{y}}{1+2\langle\textbf{x},\textbf{y}\rangle+\|\textbf{x}\|^{2}\|\textbf{y}\|^{2}}$

###  3.2. Hierarchy-Aware Modeling

Unlike the previous propagation-based studies (Wang et al., 2019a, d; Tu et
al., 2021; Wang et al., 2021) that simply embed user-item graph and KG in
Euclidean space, we aim to capture the non-Euclidean latent anatomy of data.
Taking the Alibaba-iFishion dataset shown in Figure 1(b) as an example, the
degree distribution of user-item interactions reveals an underlying
hierarchical tree-like structure with power-law distribution. Meanwhile, KG
also exhibits hierarchical patterns (Chami et al., 2020; Bai et al., 2021).
However, the hierarchical property of data cannot be properly captured in
Euclidean space because of the high-distortion (Nickel and Kiela, 2017; Ganea
et al., 2018a). Thus, we embed users and items as well as entities and
relations in hyperbolic space, where the hierarchical structures can be
naturally preserved. We also design a new aggregation scheme in KG since
previous Euclidean-based methods are not feasible in hyperbolic space.

Meanwhile, we argue that previous methods are unable to characterize items
properly because they do not differentiate hierarchical relations from the
rest, and only model KG relations at a coarse granularity. Specifically,
hierarchical relations in KG often carry entities’ attributes information
(_e.g.,_ brand and fabric), while non-hierarchical relations can only indicate
the relatedness between entities (_e.g.,_ match and similar to). This
motivates us to introduce an additional constraint for hierarchical triplets
in the embedding space to improve the representational capacity and
expressiveness.

####  3.2.1. Hyperbolic Relation-Transitive Aggregation.

We first consider the aggregation scheme of KG in hyperbolic space. As an item
$i$ can be involved in multiple KG triplets, its neighborhood reflects the
relational similarity between $i$ and its connected entities to a certain
degree. Formally, given a target item $i$ in KG, aggregating local information
from $i$’s neighbors in KG can reveal related knowledge associations for $i$.
Thus, we use $\mathcal{N}_{i}=\\{(r,t)|(i,r,t)\in\mathcal{G}_{k}\\}$ to
represent the neighborhood entities and the first-order relations of item $i$
in KG, and propose to integrate the relational context from neighborhood
entities to generate the _knowledge representation_ of item $i$:

(5)
$\displaystyle\textbf{e}^{(1)}_{i}=f_{\mathbb{B}}\Big{(}\\{(\textbf{e}^{(0)}_{i},\textbf{e}_{r},\textbf{e}^{(0)}_{t})|(r,t)\in\mathcal{N}_{i}\\}\Big{)}$

where $\textbf{e}^{(1)}_{i}\in\mathbb{B}^{d}$ is the hyperbolic knowledge
representation that collects the contextual information of item $i$ from the
first-order connectivity, and $f_{\mathbb{B}}(\cdot)$ is the hyperbolic
aggregation function to extract and integrate information related to item $i$
from each connection $(i,r,e)$ in hyperbolic space. Previous studies (Wang et
al., 2021) have shown that node-based aggregation cannot differentiate
relational paths, which fails to preserve the relation dependencies carried by
paths. Consequently, it is necessary to integrate relational paths into
representations. Besides, Euclidean propagation-based methods mainly use mean
aggregation to average the neighbor information. An analog of mean aggregation
in hyperbolic space is the Fréchet mean (Fréchet, 1948), which, however, has
no closed form solution. Instead, we model relational context in the mean
aggregator as:

(6)
$\displaystyle\textbf{e}^{(1)}_{i}=\exp_{\textbf{0}}\Big{(}\frac{1}{|\mathcal{N}_{i}|}\sum\nolimits_{(r,t)\in\mathcal{N}_{i}}\log_{\textbf{e}_{i}^{(0)}}(\textbf{e}_{t}^{(0)}\oplus\textbf{e}_{r})\Big{)}$

where $\oplus$ is the Möbius addition operation, $\log(\cdot)$ and
$\exp(\cdot)$ are the aforementioned exponential map and logarithmic map
respectively, and $\textbf{e}^{(0)}_{t}$ is the ID embedding of entity $t$ in
hyperbolic space. For each triplet $(i,r,t)$ in KG, we devise a relational
transition $\textbf{e}_{t}^{(0)}\oplus\textbf{e}_{r}$ by modeling the relation
$r$ as the vector translation of $\textbf{e}^{(0)}_{t}$. To avoid the complex
mean operation in hyperbolic space, we first map each relational context to
the tangent space, as this is where the Euclidean approximation performs the
best (_cf._ Figure 3); we then leverage the exponential map to map it back to
hyperbolic space to obtain the representation $\textbf{e}^{(1)}_{i}$. As a
result, the aggregator is able to integrate relational message into target
representations in hyperbolic space, and avoid intricate hyperbolic mean
pooling via map operations. Analogously, we can obtain the representation
$\textbf{e}^{(1)}_{e}$ of each KG entity $e\in\mathcal{E}$.

We further stack more aggregation layers to explore the high-order knowledge
associations for items. Technically, we recursively formulate the knowledge
representations of item $i$ after $l$ layers as:

(7)
$\displaystyle\textbf{e}^{(l)}_{i}=\exp_{\textbf{0}}\Big{(}\frac{1}{|\mathcal{N}_{i}|}\sum\nolimits_{(r,t)\in\mathcal{N}_{i}}\log_{\textbf{e}_{i}^{(l-1)}}(\textbf{e}_{t}^{(l-1)}\oplus\textbf{e}_{r})\Big{)}$

####  3.2.2. Angle Constraint of Hierarchical triplets.

Next, we focus on modeling the attributes information of items with
hierarchical relations. To ease our discussion, we assume that all the
relations in KG are clustered into two types (_i.e.,_ hierarchical and non-
hierarchical), and the types of relations are given a priori. In fact, the
types of relations are explicitly available in most KGs (Chami et al., 2020;
Bai et al., 2021). When such information is not available, there are two
alternative options. One is to infer the “hierarchicalness” of a relation by
widely used Krackhardt scores criteria (Krackhardt, 2014); the other is to
simply assume that item-connected entities are acted as the attributes of
items, and thus, these relations are hierarchical. Our experiments to be
reported in Section 4.3.2 show that the difference between two options is
negligible.

As mentioned earlier, hierarchical relations can reveal the attributes of
items. However, above aggregation scheme cannot preserve this semantic in the
embedding space since it simply aggregates all neighboring information into
representations. To explicitly model items’ characteristic information, we
consider the hyperbolic entailment cone (Ganea et al., 2018b), which is a
family of convex cones that can model the hierarchies with nested angular
cones. Technically, we use $\mathcal{C}_{\textbf{x}}$ to denote the cone at
apex x, and define the cone width function (_i.e.,_ half
aperture111https://en.wikipedia.org/wiki/Cone of cone
$\mathcal{C}_{\textbf{x}}$) as
$\psi_{\textbf{x}}=\arcsin(K\frac{1-\|\textbf{x}\|^{2}}{\|\textbf{x}\|})$,
where $K\in\mathbb{R}$ is a hyperparameter (we follow (Bai et al., 2021) and
set $K=0.1$). Also, for $\textbf{x},\textbf{y}\in\mathbb{B}^{d}$, we define
the angle of y at x to be the angle between the half-lines
$\overrightarrow{\textbf{ox}}$ and $\overrightarrow{\textbf{xy}}$, and denote
it as $\angle_{\textbf{x}}\textbf{y}$:

(8)
$\displaystyle\angle_{\textbf{x}}\textbf{y}=\arccos(\frac{\langle\textbf{x},\textbf{y}\rangle(1+\|\textbf{x}\|^{2})-\|\textbf{x}\|^{2}(1+\|\textbf{y}\|^{2})}{\|\textbf{x}\|\|\textbf{x}-\textbf{y}\|\sqrt{1+\|\textbf{x}\|^{2}\|\textbf{y}\|^{2}-2\langle\textbf{x},\textbf{y}\rangle}})$

The transitivity property of angular cones (Ganea et al., 2018b) guarantees
that if y is in $\mathcal{C}_{\textbf{x}}$, $\mathcal{C}_{\textbf{y}}$ is also
in $\mathcal{C}_{\textbf{x}}$. In other words, hyperbolic entailment cone
forms a nested structure in embedding space, and the width (_aka._ angle) of
the cone can naturally indicate the “attribute” semantics of embeddings, as
shown in the right of Figure 2. To leverage the expressive power of hyperbolic
entailment cone geometry, we first use
$\mathcal{H}=\\{(i,t)|(i,r,t)\in\mathcal{G}_{k}\land r\text{ is
hierarchical}\\}$ to represent the set of hierarchical entities in terms of
hierarchical relations $r$ in KG, and use Eq. (9) to define the angle
constraint in embedding space:

(9)
$\displaystyle\mathbf{\mathop{\mathcal{L}}}_{\text{H}}=\sum\nolimits_{(i,t)\in\mathcal{H}}\max(\angle_{\textbf{e}_{t}}\textbf{e}_{i}-\psi_{\textbf{e}_{t}},0)$

where $\textbf{e}_{t}$ and $\textbf{e}_{i}$ are the final knowledge
representations of entity $i$ and entity $t$ respectively, defined in Eq.
(15). The hinge loss on angle constraint suggests the angle of
$\angle_{\textbf{e}_{t}}\textbf{e}_{i}$ to be smaller than the width of
entailment cone $\mathcal{C}_{\textbf{t}}$, meaning that the embedding of item
$\textbf{e}_{i}$ should be “contained” inside of the embedding of entity
$\textbf{e}_{t}$ in the entailment cone geometry.

While such a constraint can ensure the attributes semantic with geometric
angles, it also limits the representational capacity. For example, let item
$i$ be a jeans with multiple hierarchical relations (_e.g.,_ brand and
fabric). As each of $i$’s connected entity defines an entailment cone in
hyperbolic space, the embedding of $i$ can only be in their intersection area.
Since items often have dozens of relations in KG, this intersection area could
be extremely small, leading to possible collapse in the embedding space. Thus,
we relax it by randomly assigning a corresponding subspace for the embedding
of item $i$ that satisfies the following constraint:

(10)
$\displaystyle\mathbf{\mathop{\mathcal{L}}}_{\text{H}}^{\prime}=\sum\nolimits_{(i,t)\in\mathcal{H}}\max(\angle_{\textbf{e}_{t}^{\prime}}\textbf{e}_{i}^{\prime}-\psi_{\textbf{e}_{t}^{\prime}},0)$

where $\textbf{e}_{i}^{\prime}$ and $\textbf{e}_{t}^{\prime}$ are the subspace
embeddings of $\textbf{e}_{i}$ and $\textbf{e}_{t}$ respectively by randomly
masking some dimensions. In our current implementation, each dimension has a
probability of 0.5 to be masked off, while this probability can be easily
adjusted to cater for the application needs. By doing so, we only enforce the
angle constraint in a subset of $d$ hyperbolic planes, leaving room for
expressing relatedness and other knowledge semantics of items.

###  3.3. Gated Aggregation with Dual embeddings

The second key component of HAKG framework is designed mainly to capture the
high-order collaborative signals of items. Existing methods utilize KG-
oriented aggregation strategies to collect the information related to an item
from its neighboring nodes. Unfortunately, this approach either mixes the
collaborative signals with knowledge associations such that they become
indistinguishable (Wang et al., 2019a; Chen et al., 2022), or completely
neglects the collaborative signals of items (Wang et al., 2020a, 2021) that
results in suboptimal item representations. Thus, we propose to separately
aggregate the two types of information with dual item embeddings to solve the
limitation: (1) Knowledge item embeddings, which are used to characterize
items with knowledge information from KG, as demonstrated in Section 3.2. (2)
Collaborative item embeddings, which encode items’ high-order collaborative
signals such as co-occurrence relationship in user-item graph. Neighbored with
the holistic representations of items, we further develop an information gate
mechanism to adaptively distill useful information for users’ aggregation. We
illustrate our approach in Figure 3.

####  3.3.1. Collaborative Aggregation for Items.

To explicitly encode the collaborative signals of items, we initialize new
item representations $\tilde{\textbf{e}}^{(0)}$ in hyperbolic space to items’
collaborative information, which are latent in user-item interactions. We also
use the neighborhood aggregation scheme to integrate an item’s multi-hop
neighbors into its representation for capturing the high-order connectivities.
Given an item $i$ in the user-item graph, we use
$\tilde{\mathcal{N}_{i}}=\\{u|(u,i)\in\mathcal{G}_{b}\\}$ to represent all the
users who have interacted with the item. To generate the $l$-th layer
representation of item $i$, we recursively integrate the collaborative
information from item $i$’s neighbor users:

(11)
$\displaystyle\tilde{\textbf{e}}^{(l)}_{i}=\exp_{\textbf{0}}\Big{(}\frac{1}{|\tilde{\mathcal{N}_{i}}|}\sum\nolimits_{(u)\in\tilde{\mathcal{N}_{i}}}\log_{\textbf{0}}(\textbf{e}_{u}^{(l-1)})\Big{)}$

where $\tilde{\textbf{e}}^{(l)}_{i}$ is the collaborative embedding of item
$i$ after $l$ layers, $\log(\cdot)$ and $\exp(\cdot)$ are the exponential map
and logarithmic map that are used to map between Euclidean space and
hyperbolic space, and $\textbf{e}_{u}^{(l-1)}$ is the hyperbolic embedding of
user $u$ after $l-1$ layers in hyperbolic space, which we will detail below.

####  3.3.2. Information Gated Aggregation for Users.

To distill useful information from neighbor items, we design a gated module
which is able to generate discriminative signals for users’ aggregation.
Specifically, inspired by the design of GRU (Chung et al., 2014) that learns
gating signals to control the update of hidden states, we propose to learn a
fusion gate to adaptively control the combination of two different types of
semantic item representations. For an item $i$, given its knowledge
representation ${\textbf{e}}^{(l)}_{i}$ and collaborative representation
$\tilde{\textbf{e}}^{(l)}_{i}$ in the $l$-th layer, we balance the
contributions of the two different types of information with a learnable
gating fusion unit:

(12)
$\displaystyle\textbf{g}^{(l)}_{i}=\sigma(W_{1}\log_{\textbf{0}}({\textbf{e}}^{(l)}_{i})+W_{2}\log_{\textbf{0}}(\tilde{\textbf{e}}^{(l)}_{i}))$
(13)
$\displaystyle\hat{\textbf{e}}^{(l)}_{i}=\exp_{\textbf{0}}\Big{(}\textbf{g}^{(l)}_{i}\cdot{\textbf{e}}^{(l)}_{i}+(1-\textbf{g}^{(l)}_{i})\cdot\tilde{\textbf{e}}^{(l)}_{i}\Big{)}$

where $W_{1},W_{2}\in\mathbb{R}^{d\times d}$ are learnable transformation
parameters, and $\sigma(\cdot)$ is the Sigmoid function. Notation
$\textbf{g}^{(l)}_{i}\in\mathbb{R}^{d}$ denotes the learned gate signals to
balance the contributions of collaborative signals and knowledge association
of item $i$, as shown in Figure 3. A high value of $\textbf{g}^{(l)}_{i}$
indicates that the users interact with item $i$ mainly because of the
attributes of item $i$ (_e.g.,_ $i$ is the jeans), rather than the behavior
similarities among items (_e.g.,_ the jeans $i$ liked by user $u$ is
collaboratively relevant to other bottoms that user $u$ likes). Let
$\mathcal{N}_{u}=\\{i|(u,i)\in\mathcal{G}_{b}\\}$ denote the set of items that
user $u$ has interacted with. Then, we can formulate the representation of
user $u$ with distilled information:

(14)
$\displaystyle\textbf{e}^{(l)}_{u}=\exp_{\textbf{0}}\Big{(}\frac{1}{|\mathcal{N}_{u}|}\sum\nolimits_{(i)\in\mathcal{N}_{u}}\log_{\textbf{0}}(\hat{\textbf{e}}_{i}^{(l-1)})\Big{)}$

###  3.4. Model Prediction

After $L$ layers, we obtain the representations of item $i$ and user $u$ at
different layers, and then sum them up as the final representation:

(15)
$\displaystyle\textbf{e}_{i}=\sum\nolimits_{l=0}^{L}\textbf{e}_{i}^{(l)},\qquad\tilde{\textbf{e}}_{i}=\sum\nolimits_{l=0}^{L}\tilde{\textbf{e}}_{i}^{(l)},\qquad\textbf{e}_{u}=\sum\nolimits_{l=0}^{L}\textbf{e}_{u}^{(l)}$

By doing so, the complementary information of collaboration and knowledge is
separately encoded in the final representations. Different from previous work,
we employ the consine similarity to separately predict how likely the user
$u_{i}$ would adopt an item $i$ from item’ behavior and attributes aspects,
and then adopt the sum of the two predictions as the final prediction score
$\hat{y}_{ui}$:

(16) $\displaystyle\hat{y}_{ui}=\cos(e_{u},\tilde{e}_{i})+\cos(e_{u},e_{i})$

###  3.5. Model Optimization

We opt for the contrastive loss (Mao et al., 2021) to optimize HAKG. Compared
with widely used BPR loss (Rendle et al., 2009), it alleviates the convergence
problem by introducing more negative samples and penalizing uninformative
ones. Specifically, for each interaction $(u,i)$ captured by user-item graph
(_i.e.,_ positive pair), we randomly sample —$\mathcal{M}_{u}$— unobserved
items to form the negative pairs together with user $u$, denoted as
$\mathcal{M}_{u}$, and maximize the similarity between the positive pair and
meanwhile minimize the similarity of negative pairs with a margin $m$:

(17)
$\displaystyle\mathbf{\mathop{\mathcal{L}}}=\sum\nolimits_{(u,i)\in\mathcal{D}}\big{[}2-\hat{y}_{ui}+\frac{1}{|\mathcal{M}_{u}|}\sum\nolimits_{j\in\mathcal{M}_{u}}(\hat{y}_{uj}-m)_{+}\big{]}$

where $\mathcal{D}$ is the interaction data, and $(\cdot)_{+}$ is the ramp
function $\max(0,\cdot)$. By combing the prediction loss and angle loss, we
minimize the following objective function to learn the model parameters:

(18)
$\displaystyle\mathbf{\mathop{\mathcal{L}}}=\mathbf{\mathop{\mathcal{L}}}+\lambda\mathbf{\mathop{\mathcal{L}}}_{\text{H}}^{\prime}$

where $\lambda$ is a hyperparameter to control the weight of angle loss
defined in Eq. (9).

###  3.6. Model Analysis

####  3.6.1. Model Size.

Previous studies (Wang et al., 2021) have confirmed that discarding the
nonlinear feature transformations not only yields better performance but also
reduces redundant parameters. Hence, in the aggregation schemes of HAKG, we
discard the nonlinear activation function and the learnable transformation
metrics. The model parameters of HAKG consist of (1) ID embedding of users,
items, KG entities and relations
$\\{\textbf{e}_{u}^{(0)},\tilde{\textbf{e}}_{i}^{(0)},\textbf{e}_{e}^{(0)},\textbf{e}_{r}|u\in\mathcal{U},i\in\mathcal{I},e\in\mathcal{E},r\in\mathcal{R}\\}$;
and (2) transformation parameters $W_{1},W_{2}$ for information gated
aggregation used in Eq. (12).

####  3.6.2. Time Complexity.

The time cost of HAKG mainly comes from the aggregation schemes. In the
aggregations over KG, the computational complexity of updating knowledge item
embeddings in hyperbolic space is $O(|\mathcal{G}_{k}|dL)$, where
$|\mathcal{G}_{k}|$, $d$, and $L$ denote the number of KG triplets, the
embeddings size, and the number of layers. In the aggregation over user-item
graph, the computational complexity of collaborative item embeddings and user
embeddings is $O(|\mathcal{G}_{b}|dL)$, where $|\mathcal{G}_{b}|$ is the
number of interactions. For the hierarchy-aware modeling, the cost of angle
constraint is $O(|\mathcal{I}|)$, where $|\mathcal{I}|$ is the number of
items. Besides, the cost of mappings between Euclidean space and hyperbolic
space is $O(|\mathcal{G}_{b}|+|\mathcal{G}_{k}|)$. Thus, the time complexity
of the whole training epoch is $O(|\mathcal{G}_{k}|dL+|\mathcal{G}_{b}|dL)$.
Under the same experimental settings (_i.e.,_ same embeddings size and
propagation layer), HAKG has comparable complexity to KGAT, CKAN, and KGIN,
three representative propagation-based methods.

##  4\. Experiments

We present empirical results to demonstrate the effectiveness of our proposed
HAKG framework. The experiments are designed to answer the following three
research questions:

  * •

RQ1: How does HAKG perform, compared with the state-of-the-art knowledge-aware
recommendation models?

  * •

RQ2: How do different components of HAKG (_i.e.,_ hierarchical modeling, gated
propagation, dual embeddings, and the depth of propagation layers) affect the
performance of HAKG?

  * •

RQ3: Can HAKG provide meaningful insights on the hierarchical modelling both
from structures and relations?

###  4.1. Experimental Settings

####  4.1.1. Dataset Description.

We utilize three benchmark datasets to evaluate the performance of HAKG:
Alibaba-iFashion, Yelp2018, and Last-FM. These three datasets are widely
adopted in the state-of-the-art methods (Wang et al., 2019a, 2021; Ma et al.,
2021; Chen et al., 2022), and vary in terms of domain, size, and sparsity.
Following previous work (Wang et al., 2019a), we collect the two-hop neighbor
entities of items in KG to construct the item knowledge graph for each
dataset. Besides, for the hierarchical types of relations in KG, without loss
of generality, we simply treat the item-connected relations are hierarchical,
since they are able to reveal the attributes of items. Moreover, in order to
ensure the data quality, we adopt the 10-core setting (Wang et al., 2019a),
_i.e.,_ retaining users and items with at least ten interactions and filtering
out KG entities involved in less than ten triplets. We use the same data
partition with previous studies (Wang et al., 2019a, 2021) for comparison
(_i.e.,_ the proportions of training, validation, and testing set are 80%,
10%, and 10% for all datasets). Table 1 summarizes the overall statistics of
the three datasets used in our experiments.

####  4.1.2. Evaluation Metrics.

We adopt the all-ranking strategy (Krichene and Rendle, 2020; Wang et al.,
2021) to evaluate the performance. Specifically, we treat all the items that
user has not adopted before as negative, and treat the relevant items in the
testing set as positive. To evaluate the effectiveness of top-$K$
recommendation, we adopt two widely-used evaluation protocols (He et al.,
2017; Krichene and Rendle, 2020) recall@$K$ and ndcg@$K$, where $K=20$ by
default. We report the average metrics for all the users in the testing set.

Table 1. Statistics of the datasets used in our experiments.

|  | Alibaba-iFashion | Yelp2018 | Last-FM
---|---|---|---|---
User-Item Interaction | #Users | 114,737 | 45,919 | 23,566
#Items | 30,040 | 45,538 | 48,123
#Interactions | 1,781,093 | 1,185,068 | 3,034,796
Knowledge Graph | #Entities | 59,156 | 90,961 | 58,266
#Relations | 51 | 42 | 9
#Triplets | 279,155 | 1,853,704 | 464,567

####  4.1.3. Baselines.

In order to demonstrate the effectiveness of HAKG, we compare it with the
state-of-the-art methods, including KG-free method (MF), embedding-based
methods (CKE and UGRec), propagation-based methods (KGNN-LS, KGAT, CKAN, and
KGIN), and hyperbolic-based methods (Hyper-Know and LKGR):

  * •

MF (Rendle et al., 2009) is a benchmark factorization model, which only
considers the user-item interactions, leaving KG untouched. Here, we use ID
embeddings of users and items to perform the prediction.

  * •

CKE (Zhang et al., 2016) is a representative embedding method, which utilizes
TransR (Lin et al., 2015) to learn item structural representations from KG,
and feeds learned embeddings to MF in an integrated framework.

  * •

UGRec (Zhao et al., 2021) is a state-of-the-art embedding method that models
directed and undirected relations from KG and co-occurrence behavior data.
While such undirected relations are inaccessible for other methods and
unavailable for the three datasets, we add the connectivities between items
which are co-interacted by the same user, and treat them as the co-occurrence
relationships.

  * •

KGNN-LS (Wang et al., 2019d) is a propagation-based model, which transforms KG
into user-specific graphs, and then considers user preference on KG relations
and label smoothness in the information aggregation phase, so as to generate
personalized item representations.

  * •

KGAT (Wang et al., 2019a) is a propagation-based recommender model. It applies
a unified relation-aware attentive aggregation mechanism in UKG to generate
user and item representations.

  * •

CKAN (Wang et al., 2020a) is based on KGNN-LS, which utilizes different
aggregation schemes on the user-item graph and KG respectively, to encode
knowledge association and collaborative signals.

  * •

KGIN (Wang et al., 2021) is a state-of-the-art propagation-based method, which
models user interaction behaviors with latent intents, and proposes a
relation-aware information aggregation scheme to capture long-range
connectivity in KG.

  * •

Hyper-Know (Ma et al., 2021) is a state-of-the-art hyperbolic method that
embeds KG in hyperbolic space. An adaptive regularization mechanism is also
proposed to adaptively regularize items and their neighboring representations.

  * •

LKGR (Chen et al., 2022) is a state-of-the-art hyperbolic GNN method with
Lorentz model, which employs different information propagation strategies in
the hyperbolic space to encode heterogeneous information from historical
interaction and KG.

####  4.1.4. Parameter Settings.

We implement HAKG in Pytorch, and have released our
implementations222https://github.com/Scottdyt/HAKG (codes, parameter settings,
and training logs) to facilitate reproducibility. For a fair comparison, we
fix the size of ID embeddings as 64, and optimize all models with Adam (Kingma
and Ba, 2014) optimizer, where the batch size is fixed at 4096. The Xavier
initializer (Glorot and Bengio, 2010) is used to initialize the model
parameters. We apply a grid search for hyperparameters: the learning rate is
tuned amongst $\\{10^{-4},10^{-3},10^{-2}\\}$, the weight of angle loss
$\lambda$ is searched in $\\{10^{-5},10^{-4},\cdots,10^{-2},10^{-1}\\}$, and
tune the GNN layers $L$ in $\\{1,2,3\\}$ for propagation-based methods. For
the number of negative samples $|\mathcal{M}_{u}|$ per user and the margin $m$
of contrastive loss, we set to $\\{200,400,400\\}$ and $\\{0.6,0.8,0.7\\}$ for
Alibaba-iFashion, Yelp2018, and Last-FM datasets, respectively. Besides, since
optimization in hyperbolic space is practically challenging, we instead define
all parameters in the tangent space at the origin, optimize embeddings using
standard Euclidean techniques, and use the exponential map to recover the
hyperbolic parameters (Chami et al., 2020). The parameters for all baseline
methods are carefully tuned to achieve optimal performance. Specifically, for
KGAT, we set the depth to three with the hidden size $\\{64,32,16\\}$, and use
the pre-trained ID embeddings of MF as initialization; for UGRec, we set the
margin value for hinge loss to 1.5; for CKAN, KGNN-LS, and LKGR, we set the
size of neighborhood to 16; for KGIN, we fix the number of intents to 4; and
for Hyper-Know, we set the curvature to -1. Moreover, early stopping strategy
is performed for all methods, _i.e.,_ premature stopping if recall@20 on the
test set does not increase for 10 successive epochs.

###  4.2. Performance Comparison (RQ1)

We begin with the performance comparison _w.r.t._ recall@20 and ndcg@20. The
experimental results are reported in Table 2, where %Imp. denotes the relative
improvement of the best performing method (starred) over the strongest
baselines (underlined). We have the following observations:

  * •

HAKG consistently yields the best performance on all the datasets. In
particular, it achieves significant improvement even over the strongest
baselines _w.r.t._ ndcg@20 by 15.43%, 8.21%, and 9.79% in Alibaba-iFashion,
Yelp2018, and Last-FM, respectively. We attribute these improvement to the
hierarchical modeling and holistic information propagation of HAKG: (1) By
embedding user-item interactions and KG in hyperbolic space and introducing
auxiliary angle constraint for hierarchical relations, HAKG is able to capture
the underlying hierarchical structure of data, and better characterize items
in the embedding space. In contrast, all baselines ignore the importance of
hierarchical relations, and simply use KG-oriented aggregation schemes to
capture KG information. (2) Benefited from our gated aggregation with dual
item embeddings, HAKG can explicitly preserve the holistic semantics of items
to better encode user behavior patterns, while other propagation-based
baselines (_e.g.,_ CKAN, KGIN, and LKGR) fail to capture the high-order
collaborative signals of items.

  * •

Jointly analyzing the performance of HAKG across the three datasets, we find
that the improvement on Alibaba-iFashion dataset is more significant than that
on other datasets. One possible reason is that the size of KG on Alibaba-
iFashion is much smaller than that on Yelp2018 and Last-FM, and most of the
relations are hierarchical, thus it is more important to i) preserve the
valuable attribute information of items, and ii) capture the crucial
collaborative signals of items for learning user preference. This confirms
that HAKG can better leverage the user-item interactions and KG for
comprehensive item representations.

  * •

The side information of KG is important for recommendations. Compared with
vanilla MF, CKE outperforms MF by simply incorporating KG embeddings into MF.
The results are consistent with prior studies (Wang et al., 2021).

  * •

Propagation-based methods outperform embedding methods on most datasets,
indicating the importance of modeling long-range connectivities. We attribute
their success to the information aggregation schemes. Specifically,
recursively collecting information from neighboring nodes is able to capture
the high-order latent semantics for high-quality representation.

  * •

Hyperbolic-based methods (_i.e.,_ Hyper-Know and LKGR) achieve competitive
performance compared with Euclidean methods. However, they still slightly
underperform state-of-the-art propagation methods (_i.e.,_ KGIN). This is
because they fail to fully exploit the expressive power of hyperbolic space
for comprehensive representations. Specifically, Hyper-Know embeds KG in
hyperbolic space and directly optimizes the entities’ embeddings with
triplets, without taking the high-order knowledge associations into
consideration. LKGR designs an Euclidean attention mechanism for hyperbolic
entities’ aggregation, which cannot preserve the local similarities of
embeddings in hyperbolic space.

Table 2. Overall performance comparison.

| Alibaba-iFashion | Yelp2018 | Last-FM
---|---|---|---
recall | ndcg | recall | ndcg | recall | ndcg
MF | 0.1095 | 0.0670 | 0.0627 | 0.0413 | 0.0724 | 0.0617
CKE | 0.1103 | 0.0676 | 0.0653 | 0.0423 | 0.0732 | 0.0630
UGRec | 0.1006 | 0.0621 | 0.0651 | 0.0419 | 0.0730 | 0.0624
KGNN-LS | 0.1039 | 0.0557 | 0.0671 | 0.0422 | 0.0880 | 0.0642
KGAT | 0.1030 | 0.0627 | 0.0705 | 0.0463 | 0.0873 | 0.0744
CKAN | 0.0970 | 0.0509 | 0.0646 | 0.0441 | 0.0812 | 0.0660
KGIN | 0.1147 | 0.0716 | 0.0698 | 0.0451 | 0.0978 | 0.0848
Hyper-know | 0.1057 | 0.0648 | 0.0685 | 0.0447 | 0.0948 | 0.0812
LKGR | 0.1033 | 0.0612 | 0.0679 | 0.0438 | 0.0883 | 0.0675
HAKG | 0.1319∗ | 0.0848∗ | 0.0778∗ | 0.0501∗ | 0.1008∗ | 0.0931∗
%Imp. | 14.99% | 15.43% | 10.35% | 8.21% | 3.07% | 9.79%

###  4.3. Study of HAKG (RQ2)

As the hierarchical modeling is at the core of HAKG, we also conduct ablation
studies to investigate the effectiveness. Specifically, how the presence of
angle loss and gated aggregation, the hyperbolic embeddings and hierarchical
relations, the dual item embeddings, and the number of propagation layers
affect our model.

####  4.3.1. Impact of Angle Loss & Gated Aggregation.

We first verify the effectiveness of the angle loss and gated aggregation. To
this end, three variants of HAKG are constructed by (1) discarding the angle
constraint and gated aggregation scheme, termed as HAKG${}_{\text{w/o A\&G}}$,
(2) removing the angle loss for hierarhical triplets, called
HAKG${}_{\text{w/o A}}$, and (3) replacing the gated aggregation with simple
point-wise addition, named HAKG${}_{\text{w/o G}}$. We summarize the results
in Table 3.

Table 3. Impact of angle loss and gated aggregation.

| Alibaba-iFashion | Yelp2018 | Last-FM
---|---|---|---
recall | ndcg | recall | ndcg | recall | ndcg
w/o A&G | 0.1218 | 0.0799 | 0.0737 | 0.0458 | 0.0946 | 0.0872
w/o A | 0.1272 | 0.0825 | 0.0763 | 0.0485 | 0.0963 | 0.0907
w/o G | 0.1253 | 0.0817 | 0.0758 | 0.0471 | 0.0959 | 0.0894

Table 4. Impact of hierarchical modeling.

| Alibaba-iFashion | Yelp2018 | Last-FM
---|---|---|---
recall | ndcg | recall | ndcg | recall | ndcg
Euclidean | 0.1231 | 0.0798 | 0.0756 | 0.0484 | 0.0981 | 0.0916
PH-Relation | 0.1317 | 0.0845 | 0.0772 | 0.0494 | 0.1001 | 0.0928
GH-Relation | 0.1320 | 0.0845 | 0.0776 | 0.0498 | 0.1005 | 0.0929

Compared with the complete model of HAKG in Table 2, the absence of the angle
constraint and gated aggregation dramatically degrades the performance,
indicating the necessity of modeling hierarchical relation and collaborative
signals. Specifically, HAKG${}_{\text{w/o A\&G}}$ directly fuses the dual item
embeddings for user aggregation, and ignores the hierarchical relations in KG,
and thus, it fails to profile item properly and propagate comprehensive
information for learning use. Analogously, leaving the hierarchical relations
unexplored (_i.e.,_ HAKG${}_{\text{w/o A}}$) also downgrades the performance.
Although HAKG${}_{\text{w/o G}}$ retains the modeling of hierarchical
relations for characterizing items, it is unable to provide discriminative
signals for identifying user behavior patterns, incurring suboptimal user
representations.

####  4.3.2. Impact of Hierarchical Modeling.

We then evaluate the influence of hierarchical modeling by considering both
hierarchical structure and relations. To be more specific, we propose three
alternative models, which are modified by: i) replacing all hyperbolic
operations with Euclidean alternatives and retaining the computation logic of
HAKG (called Euclidean), ii) predicting the hierarchical types of relations by
widely used Krackhardt scores criteria (Krackhardt, 2014; Bai et al., 2021) in
order to compensate for the unavailability of relation types (named PH-
Relation), and iii) leveraging the given hierarchical types of relations to
characterize items (termed as GH-Relation). The results of three alternative
models are listed in Table 4. We observe that:

  * •

The performance degrades for all three datasets when we remove the hyperbolic
geometry for HAKG, meaning that modeling user-item interactions and KG in
hyperbolic space could enhance the model’s expressive power and yield better
representations for recommendation.

  * •

When the hierarchical types of KG relations are not explicitly available, the
performance of alternative approaches (_i.e.,_ PH-Relation) approximates to
that of model with ground-truth hierarchical relations (_i.e.,_ GH-Relation),
which empirically shows that our model is robust for different datatsets even
when the KG’s hierarchical information is not available.

Table 5. Impact of dual item embedding.

| Alibaba-iFashion | Yelp2018 | Last-FM
---|---|---|---
recall | ndcg | recall | ndcg | recall | ndcg
Single | 0.1186 | 0.0755 | 0.0769 | 0.0492 | 0.0989 | 0.0904
Dual | 0.1319 | 0.0847 | 0.0778 | 0.0501 | 0.1008 | 0.0931

Table 6. Impact of the number of layers $L$.

| Alibaba-iFashion | Yelp2018 | Last-FM
---|---|---|---
recall | ndcg | recall | ndcg | recall | ndcg
HAKG-1 | 0.1313 | 0.0845 | 0.0766 | 0.0489 | 0.0972 | 0.0897
HAKG-2 | 0.1306 | 0.0831 | 0.0778 | 0.0501 | 0.0988 | 0.0913
HAKG-3 | 0.1319 | 0.0848 | 0.0774 | 0.0498 | 0.1008 | 0.0931

####  4.3.3. Impact of Dual Item Embeddings.

To analyze the effectiveness of the design of dual item embeddings, we compare
it with the conventional single item embeddings, which removes the additional
collaborative item embeddings and directly leverages the knowledge embeddings
of items in Eq. (7) for users’ aggregation. We compare their performance on
three datasets, and present the results in Table 5. We have the following
findings:

  * •

Discarding the collaborative item embeddings would consistently degrade the
performance cross three datasets. This is because if we model items with
single representations, they inevitably suffer from information loss when
performing the neighbor aggregation. In other words, since items are
inherently exhibit two kinds of semantics (collaboration and knowledge), it is
unlikely to simultaneously incorporate all the high-order information of items
with single embeddings.

  * •

The performance of single item embeddings decrease more dramatically on
Aliabab-iFashion dataset, compared with it on other two datasets. One possible
reason is that the amount of auxiliary information provided by KG is the
smallest, which makes it more important to capture the collaborative signals
of items with additional collaborative item embeddings.

####  4.3.4. Impact of Model Depth.

We also explore the impact of the number of aggregation layers. Stacking more
layers is able to collect the high-order collaborative signals and knowledge
associations for better capturing of the latent user behavior patterns but at
a higher cost. Here, we search $L$ in the range of $\\{1,2,3\\}$, and report
the results in Table 6. We have the following observations:

  * •

Generally speaking, increasing the aggregation layers can enhance the
performance, especially for Last-FM datasets. We attribute such improvement to
two reasons: (1) Gathering more relevant collaborative signals and knowledge
association could provide informative semantics for learning high-quality
representations, deepening the understanding of user interest. (2) The dual
item embeddings explicitly encode both items’ behavior and attribute
similarities, which profile item at a finer-grained way than single item
embeddings used in other methods.

  * •

It is worth mentioning that our model is less sensitive to the model depth,
compared with other propagation-based methods (Wang et al., 2019a, 2020a,
2021). Specifically, HAKG could achieve competitive performance even when
$L=1$. This is because our separate aggregation schemes and gate mechanism can
directly capture useful patterns from both user-item interactions and KG,
while the KG-oriented aggregation schemes need more layers to encode the
latent semantics from the mixed and obscure information.

Figure 4. (a) Distance from node (user or item) embedding to origin vs node
popularity. (b) Hyperbolic embeddings of entities in KG.

###  4.4. Hierarchies Visualization (RQ3)

In this section, we visualize the learned embeddings to give an intuitive
impression of our hierarchical modeling. Specifically, we first train HAKG
with two-dimensional embeddings on the Alibaba-iFashion dataset, and
separately analyze the hierarchies that exhibit in $\mathcal{G}_{b}$ and
$\mathcal{G}_{k}$: (1) For the underlying hierarchical structure in
interactions, we plot the distance from the origin to hyperbolic embeddings
(_e.g.,_ user embeddings or collaborative item embeddings) versus its
popularity. (2) For the hierarchical relations in KG, we select representative
entities and demonstrate their connectivities with triplets. As shown in
Figure 4, we find that:

  * •

The left of Figure 4 indicates a clear exponential trend that distance to the
origin increases exponentially for less popular items, which is consistent
with the degree distribution of interactions in Figure 1(b). This confirms
that our model takes advantage of the exponentially growing volume in
hyperbolic space, and uses it to naturally represent the users and items.

  * •

The connectivities in the right of Figure 4 show clear hierarchical relations
between entities. For instance, the entity bottoms is categorized into pants,
jeans, and skirt, and the attribute information of items (_e.g.,_ T-shirt is a
type of tops) can be naturally preserved with the hierarchical relations.
Thus, HAKG is able to capture the items’ attribute semantics in the embedding
space, and yield better item representations for learning user preferences.

##  5\. Related Work

Existing recommendation systems incorporated with KG information can be mainly
categorized into three clusters, viz., _embedding-based methods_ , _path-based
methods_ , and _propagation-based methods_. We briefly review them in the
following.

  * •

Embedding-based methods (Zhang et al., 2016; Wang et al., 2018b; Ai et al.,
2018; Huang et al., 2018; Cao et al., 2019; Wang et al., 2020b; Zhao et al.,
2021) directly embed entities and relations in KG via knowledge graph
embedding (KGE) methods (_e.g.,_ TransR (Lin et al., 2015) and TransD (Ji et
al., 2015)) to serve as item embedding in recommendation. For example, CKE
(Zhang et al., 2016) utilizes TransR to learn item structural representations
from knowledge graph, and feeds the learned embeddings to matrix factorization
(MF) (Rendle et al., 2009) in an integrated framework. Hyper-Know (Ma et al.,
2021) embeds knowledge graph in Poincaré Ball and then designs an adaptive
regularization mechanism to regularize item representations. Although these
methods benefit from the simplicity and expressiveness of KGE, they fail to
capture high-order dependence of user-item relations for user preference
learning.

  * •

Path-based methods (Catherine and Cohen, 2016; Hu et al., 2018; Zhu et al.,
2018; Wang et al., 2019c; Ma et al., 2019; Jin et al., 2020) aim to find
semantic paths in KG, and then connect items and users to discover long-range
connectivity for recommendation. Those paths can be used to predict user
preference with recurrent neural networks (Wang et al., 2019c; Zhu et al.,
2018) or attention mechanism (Hu et al., 2018). For instance, KPRN (Wang et
al., 2019c) captures the sequential dependence within a knowledge-aware path
to infer the underlying high-order relation of a user-item interaction.
However, defining proper meta-path patterns requires domain knowledge, which
can be extremely time-consuming for complicated KG with various types of
entities and relations. Moreover, domain-specific meta-paths inevitably lead
to poor generalization for different recommendation scenarios (Jin et al.,
2020; Lu et al., 2020).

  * •

Propagation-based methods (Wang et al., 2018a, 2019a, 2019d, 2019e, 2020a,
2021; Tu et al., 2021) are inspired by the recent advances of graph neural
networks (GNNs) (Kipf and Welling, 2017; Hamilton et al., 2017; Veličković et
al., 2018; Wu et al., 2019; Wang et al., 2019b; He et al., 2020), which
iteratively perform information aggregation mechanism from neighborhood nodes.
As such, these methods are able to discover high-order relations in an end-to-
end fashion. For example, KGAT (Wang et al., 2019a) creates unified knowledge
graph (UKG) to combine user-item interactions and KG, and then performs
knowledge-aware attention on it. CKAN (Wang et al., 2020a) employs the
heterogeneous propagation strategy to encode the user preference with
knowledge associations to further improve the performance. Most recently, KGIN
(Wang et al., 2021) considers a new aggregation mechanism to integrate long-
range relation paths, and disentangles user preference with intents for better
interpretability.

How to leverage hyperbolic geometry and hyperbolic embeddings in recommender
systems has become a recent trend (Vinh Tran et al., 2020; Ma et al., 2021;
Sun et al., 2021; Zhang et al., 2021; Chen et al., 2022). For instance,
HyperML (Vinh Tran et al., 2020) is the first work to leverage hyperbolic
geometry for recommendation through metric learning approach; and HGCF (Sun et
al., 2021) proposes a hyperbolic GCN model for collaborative filtering.
However, to the best of our knowledge, none of existing methods considers
modeling KG relations at a finer-grained level of hierarchies. Moreover, they
fail to preserve the crucial high-order collaborative signals of items. Our
work differs from them in hierarchical modeling and aggregation, i.e., we aim
at addressing the importance of hierarchical relations for profiling items,
and explicitly propagate two kinds of information with dual item embeddings
for better identifying user behaviors.

##  6\. Conclusion and Future Work

In this paper, we propose a new knowledge-aware recommendation model, called
HAKG, which captures the underlying hierarchical structure of data in
hyperbolic space, and characterize items with hierarchical relations in KG.
Besides, HAKG employs dual item embeddings to separately encode items’
collaborative signals and knowledge associations, and develops a gated
mechanism to control discriminative signals towards the users’ behavior
patterns. Extensive experiments conducted on three real-world datasets
demonstrate the superiority of HAKG. In the future, we plan to investigate the
problem of numerical errors in hyperbolic space (Yu and De Sa, 2021), and
examine all operations (_e.g.,_ addition and multiplication) in hyperbolic
space to fully exploit the power of hyperbolic geometry.

###### Acknowledgements.

This work was supported by the NSFC under Grants No. (62025206, 61972338, and
62102351).

## References

  * (1)
  * Ai et al. (2018) Qingyao Ai, Vahid Azizi, Xu Chen, and Yongfeng Zhang. 2018\.  Learning Heterogeneous Knowledge Base Embeddings for Explainable Recommendation.  _Algorithms_ 11, 9 (2018), 137–146.
  * Bai et al. (2021) Yushi Bai, Rex Ying, Hongyu Ren, and Jure Leskovec. 2021\.  Modeling Heterogeneous Hierarchies with Relation-specific Hyperbolic Cones. In _NeurIPS_.
  * Cannon et al. (1997) James W. Cannon, William J. Floyd, Richard Kenyon, Walter, and R. Parry. 1997\.  Hyperbolic geometry. In _In Flavors of geometry_. 59–115.
  * Cao et al. (2019) Yixin Cao, Xiang Wang, Xiangnan He, Zikun hu, and Tat-Seng Chua. 2019.  Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences. In _WWW_. 151–161.
  * Catherine and Cohen (2016) Rose Catherine and William Cohen. 2016.  Personalized Recommendations Using Knowledge Graphs: A Probabilistic Logic Programming Approach. In _RecSys_. 325–332.
  * Chami et al. (2020) Ines Chami, Adva Wolf, Da-Cheng Juan, Frederic Sala, Sujith Ravi, and Christopher Ré. 2020\.  Low-Dimensional Hyperbolic Knowledge Graph Embeddings. In _ACL_. 6901–6914.
  * Chami et al. (2019) Ines Chami, Zhitao Ying, Christopher Ré, and Jure Leskovec. 2019\.  Hyperbolic graph convolutional neural networks.  (2019), 4868–4879.
  * Chen et al. (2022) Yankai Chen, Menglin Yang, Yingxue Zhang, Mengchen Zhao, Ziqiao Meng, Jian Hao, and Irwin King. 2022.  Modeling Scale-free Graphs for Knowledge-aware Recommendation. In _WSDM_. 94–102.
  * Chung et al. (2014) Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014.  Empirical evaluation of gated recurrent neural networks on sequence modeling.  _arXiv preprint arXiv:1412.3555_.
  * Du et al. (2022) Yuntao Du, Xinjun Zhu, Lu Chen, Ziquan Fang, and Yunjun Gao. 2022.  MetaKG: Meta-learning on Knowledge Graph for Cold-start Recommendation. In _arXiv preprint arXiv:2202.03851_.
  * Fréchet (1948) Maurice Fréchet. 1948\.  Les éléments aléatoires de nature quelconque dans un espace distancié. In _Annales de l’institut Henri Poincaré_. 215–310.
  * Ganea et al. (2018a) Octavian Ganea, Gary Becigneul, and Thomas Hofmann. 2018a.  Hyperbolic Neural Networks. In _NeurIPS_. 5350–5360.
  * Ganea et al. (2018b) Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. 2018b.  Hyperbolic Entailment Cones for Learning Hierarchical Embeddings. In _ICML_. 1646–1655.
  * Gharibshah et al. (2020) Zhabiz Gharibshah, Xingquan Zhu, Arthur Hainline, and Michael Conway. 2020.  Deep learning for user interest and response prediction in online display advertising.  _Data Science and Engineering_ 5, 1 (2020), 12–26.
  * Glorot and Bengio (2010) Xavier Glorot and Yoshua Bengio. 2010.  Understanding the difficulty of training deep feedforward neural networks. In _AISTATS_. 249–256.
  * Hamilton et al. (2017) William L. Hamilton, Rex Ying, and Jure Leskovec. 2017\.  Inductive Representation Learning on Large Graphs. In _NeurIPS_. 1025–1035.
  * He et al. (2020) Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, YongDong Zhang, and Meng Wang. 2020\.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. In _SIGIR_. 639–648.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural Collaborative Filtering. In _WWW_. 173–182.
  * Hu et al. (2018) Binbin Hu, Chuan Shi, Wayne Xin Zhao, and Philip S. Yu. 2018\.  Leveraging Meta-Path Based Context for Top- N Recommendation with A Neural Co-Attention Model. In _KDD_. 1531–1540.
  * Hu et al. (2008) Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008.  Collaborative Filtering for Implicit Feedback Datasets. In _ICDM_. 263–272.
  * Huang et al. (2018) Jin Huang, Wayne Xin Zhao, Hongjian Dou, Ji-Rong Wen, and Edward Y. Chang. 2018.  Improving Sequential Recommendation with Knowledge-Enhanced Memory Networks. In _SIGIR_. 505–514.
  * Ji et al. (2015) Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. 2015.  Knowledge Graph Embedding via Dynamic Mapping Matrix. In _ACL_. 687–696.
  * Jin et al. (2020) Jiarui Jin, Jiarui Qin, Yuchen Fang, Kounianhua Du, Weinan Zhang, Yong Yu, Zheng Zhang, and Alexander J. Smola. 2020\.  An Efficient Neighborhood-Based Interaction Model for Recommendation on Heterogeneous Graph. In _KDD_. 75–84.
  * Kingma and Ba (2014) Diederik P Kingma and Jimmy Ba. 2014.  Adam: A method for stochastic optimization. In _ICLR_.
  * Kipf and Welling (2017) Thomas N. Kipf and Max Welling. 2017.  Semi-Supervised Classification with Graph Convolutional Networks. In _ICLR_.
  * Koren (2008) Yehuda Koren. 2008\.  Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model. In _KDD_. 426–434.
  * Krackhardt (2014) David Krackhardt. 2014\.  Graph theoretical dimensions of informal organizations.  In _Computational organization theory_. 107–130.
  * Krichene and Rendle (2020) Walid Krichene and Steffen Rendle. 2020.  On sampled metrics for item recommendation. In _KDD_. 1748–1757.
  * Lin et al. (2015) Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015.  Learning Entity and Relation Embeddings for Knowledge Graph Completion. In _AAAI_. 2181–2187.
  * Lu et al. (2020) Yuanfu Lu, Yuan Fang, and Chuan Shi. 2020.  Meta-Learning on Heterogeneous Information Networks for Cold-Start Recommendation. In _KDD_. 1563–1573.
  * Ma et al. (2021) Chen Ma, Liheng Ma, Yingxue Zhang, Haolun Wu, Xue Liu, and Mark Coates. 2021\.  Knowledge-Enhanced Top-K Recommendation in Poincaré Ball. In _AAAI_. 4285–4293.
  * Ma et al. (2019) Weizhi Ma, Min Zhang, Yue Cao, Woojeong Jin, Chenyang Wang, Yiqun Liu, Shaoping Ma, and Xiang Ren. 2019\.  Jointly Learning Explainable Rules for Recommendation with Knowledge Graph. In _WWW_. 1210–1221.
  * Mao et al. (2021) Kelong Mao, Jieming Zhu, Jinpeng Wang, Quanyu Dai, Zhenhua Dong, Xi Xiao, and Xiuqiang He. 2021.  SimpleX: A Simple and Strong Baseline for Collaborative Filtering. In _CIKM_. 1243–1252.
  * Nickel and Kiela (2017) Maximillian Nickel and Douwe Kiela. 2017.  Poincaré embeddings for learning hierarchical representations. In _NeurIPS_. 6338–6347.
  * Rendle et al. (2009) Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009.  BPR: Bayesian Personalized Ranking from Implicit Feedback. In _UAI_. 452–461.
  * Sun et al. (2021) Jianing Sun, Zhaoyue Cheng, Saba Zuberi, Felipe Perez, and Maksims Volkovs. 2021.  HGCF: Hyperbolic Graph Convolution Networks for Collaborative Filtering. In _WWW_. 593–601.
  * Tu et al. (2021) Ke Tu, Peng Cui, Daixin Wang, Zhiqiang Zhang, Jun Zhou, Yuan Qi, and Wenwu Zhu. 2021.  Conditional Graph Attention Networks for Distilling and Refining Knowledge Graphs in Recommendation. In _CIKM_. 1834–1843.
  * Veličković et al. (2018) Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018.  Graph Attention Networks. In _ICLR_.
  * Vinh Tran et al. (2020) Lucas Vinh Tran, Yi Tay, Shuai Zhang, Gao Cong, and Xiaoli Li. 2020.  Hyperml: A boosting metric learning approach in hyperbolic space for recommender systems. In _WSDM_. 609–617.
  * Wang et al. (2020b) Chenyang Wang, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020b.  Make It a Chorus: Knowledge- and Time-Aware Item Modeling for Sequential Recommendation. In _SIGIR_. 109–118.
  * Wang et al. (2018a) Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, and Minyi Guo. 2018a.  RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems. In _CIKM_. 417–426.
  * Wang et al. (2018b) Hongwei Wang, Fuzheng Zhang, Xing Xie, and Minyi Guo. 2018b.  DKN: Deep knowledge-aware network for news recommendation. In _WWW_. 1835–1844.
  * Wang et al. (2019d) Hongwei Wang, Fuzheng Zhang, Mengdi Zhang, Jure Leskovec, Miao Zhao, Wenjie Li, and Zhongyuan Wang. 2019d.  Knowledge-Aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems. In _KDD_. 968–977.
  * Wang et al. (2019e) Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019e.  Knowledge Graph Convolutional Networks for Recommender Systems. In _WWW_. 3307–3313.
  * Wang et al. (2019a) Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019a.  KGAT: Knowledge Graph Attention Network for Recommendation. In _KDD_. 950–958.
  * Wang et al. (2019b) Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019b.  Neural Graph Collaborative Filtering. In _SIGIR_. 165–174.
  * Wang et al. (2021) Xiang Wang, Tinglin Huang, Dingxian Wang, Yancheng Yuan, Zhenguang Liu, Xiangnan He, and Tat-Seng Chua. 2021.  Learning Intents behind Interactions with Knowledge Graph for Recommendation. In _WWW_. 878–887.
  * Wang et al. (2019c) Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, and Tat-Seng Chua. 2019c.  Explainable Reasoning over Knowledge Graphs for Recommendation. In _AAAI_. 5329–5336.
  * Wang et al. (2020a) Ze Wang, Guangyan Lin, Huobin Tan, Qinghong Chen, and Xiyang Liu. 2020a.  CKAN: Collaborative Knowledge-Aware Attentive Network for Recommender Systems. In _SIGIR_. 219–228.
  * Wu et al. (2019) Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. 2019\.  Simplifying Graph Convolutional Networks. In _ICML_. 6861–6871.
  * Xian et al. (2019) Yikun Xian, Zuohui Fu, Shan Muthukrishnan, Gerard De Melo, and Yongfeng Zhang. 2019.  Reinforcement knowledge graph reasoning for explainable recommendation. In _SIGIR_. 285–294.
  * Yu and De Sa (2021) Tao Yu and Christopher M De Sa. 2021.  Representing Hyperbolic Space Accurately using Multi-Component Floats. In _NeurIPS_. 15570–15581.
  * Zhang et al. (2016) Fuzheng Zhang, Nicholas Jing Yuan, Defu Lian, Xing Xie, and Wei-Ying Ma. 2016\.  Collaborative Knowledge Base Embedding for Recommender Systems. In _KDD_. 353–362.
  * Zhang et al. (2021) Sixiao Zhang, Hongxu Chen, Xiao Ming, Lizhen Cui, Hongzhi Yin, and Guandong Xu. 2021\.  Where Are We in Embedding Spaces? A Comprehensive Analysis on Network Embedding Approaches for Recommender Systems. In _KDD_. 2223–2231.
  * Zhao et al. (2021) Xinxiao Zhao, Zhiyong Cheng, Lei Zhu, Jiecai Zheng, and Xueqing Li. 2021.  UGRec: Modeling Directed and Undirected Relations for Recommendation. In _SIGIR_. 193–202.
  * Zhu et al. (2018) Sun Zhu, Yang Jie, Zhang Jie, Bozzon Alessandro, Huang Long-Kai, and Xu Chi. 2018\.  Recurrent Knowledge Graph Embedding for Effective Recommendation. In _RecSys_. 297–305.


[a]Toru Nishimura

#  Dilepton production rate near the critical temperature of color
superconductivity
J-PARC-TH-0251

Masakiyo Kitazawa  Teiji Kunihiro

###### Abstract

We investigate modification of the dilepton production rate by the diquark
fluctuations that form well-developed collective modes near the critical
temperature of color superconductivity. Through the analysis of the photon
self-energy called the Aslamasov-Larkin, Maki-Thompson and density of states
terms in the theory of metalic superconductivity, it is shown that the
collective mode in the diquark channel affects the photon self-energy
significantly and thereby gives rise to an anomalous enhacement of the
dilepton production rate in the low invariant-mass region.

##  1 Introduction

Experimental programs in relativistic heavy-ion collisions (HIC) such as the
beam-energy scan program at RHIC, HADES and NA61/SHINE, as well as the future
plans at FAIR, NICA and J-PARC-HI, are aimed at revealing rich physics in high
baryon-density matter at finite temperature. In this report, we theoretically
explore the possibility to observe precursory phenomena of the color
superconductivity (CSC) [1] in these experiments through the analysis of the
dilepton production rate, on the basis of the observation [2, 3] that the
diquark fluctuations are developed in the temperature higher than but near the
critical temperature $T_{c}$ of the CSC. We show that the diquark fluctuations
modify the photon self-energy and thereby affect the dilepton production rates
near $T_{c}$ by extending the theory of the paraconductivity in metals [4].

Figure 1:  The phase diagram obtained in the massless 2-flavor NJL model Eq.
(1). The dashed lines show the second-order phase transitions.

##  2 Model and phase diagram

We focus on the 2-flavor color-superconductivity (2SC), which is expected to
be realized at relatively low densities, and employ the massless 2-flavor and
3-color NJL model [2, 3],

$\displaystyle\mathcal{L}=\bar{\psi}i\gamma_{\mu}\partial^{\mu}\psi\ +\
G_{S}[(\bar{\psi}\psi)^{2}+(\bar{\psi}i\gamma_{5}\vec{\tau}\psi)^{2}]\ +\
G_{C}(\bar{\psi}i\gamma_{5}\tau_{2}\lambda_{A}\psi^{C})(\bar{\psi}^{C}i\gamma_{5}\tau_{2}\lambda_{A}\psi),$
(1)

where the second and third terms represent the $qq$ and $q\bar{q}$
interactions, respectively, and $\psi^{C}(x)\equiv C\bar{\psi}^{T}(x)$ with
$C=i\gamma_{2}\gamma_{0}$. $\tau_{2}$ and $\gamma_{A}$ $(A=2,5,7)$ are the
antisymmetric components of the Pauli and Gell-mann matrices for the flavor
$SU(2)_{f}$ and color $SU(3)_{c}$, respectively. The scalar coupling constant
$G_{S}=5.01\rm{MeV^{-2}}$ and the three-momentum cutoff $\Lambda=650$ MeV are
determined so as to reproduce the pion decay constant $f_{\pi}=93\rm{MeV}$ and
the chiral condensate $\langle\bar{\psi}\psi\rangle=(-250\rm{MeV})^{3}$ in the
chiral limit. The diquark coupling constant is set to $G_{C}=0.6G_{S}$. We
show the phase diagram obtained in the mean-field approximation (MFA) with the
mean fields $\langle\bar{\psi}\psi\rangle$ and
$\langle\bar{\psi}^{C}i\gamma_{5}\tau_{2}\lambda_{A}\psi\rangle$ in Fig. 1.
The 2SC phase is realized at low temperature and high density region. In the
following, we focus on the medium near but above the critical temperature of
2SC.

##  3 Photon self-energy due to the diquark fluctuations

The dilepton production rate is given in terms of the retarded photon self-
energy $\Pi^{R\mu\nu}(k)$ as

$\displaystyle\frac{d^{4}\Gamma}{d^{4}k}=\frac{\alpha}{12\pi^{4}}\frac{1}{k^{2}}\frac{1}{e^{\beta\omega}-1}g_{\mu\nu}{\rm
Im}\Pi^{R\mu\nu}(k),$ (2)

where $k=(\bm{k},\omega)$ is the four momentum of the photon and $\alpha$ is
the fine structure constant.

In Refs. [2, 3], it has been pointed out that the diquark fluctuations develop
the collectivity at temperatures above but near the critical temperature of
the CSC. In the present study we investigate the modification of the photon
self-energy due to the diquark fluctuations. The photon self-energy is derived
so that it satisfies the Ward-Takahashi (WT) identity
$k_{\mu}\Pi^{R\mu\nu}(k)=0$ utilizing the thermodynamic potential.

###  3.1 Diquark fluctuation mode

Figure 2: Diagrammatic representation for the propagator Eq. (3).

The propagator of the diquark fluctuations in the random-phase approximation
(Fig. 2) is given by

$\displaystyle\Xi(\bm{q},i\nu_{n})=\frac{G_{C}}{1+G_{C}\mathcal{Q}(\bm{q},i\nu_{n})},$
(3)

where $\mathcal{Q}(\bm{q},i\nu_{n})$ is the one-loop quark-quark correlation

$\displaystyle\mathcal{Q}(\bm{q},i\nu_{n})=-2N_{f}(N_{c}-1)T\sum_{m}\int\frac{d^{3}p}{(2\pi)^{3}}{\rm
Tr}[C\Gamma\mathcal{G}_{0}(\bm{q-p},i\nu_{n}-i\omega_{m})\Gamma
C\mathcal{G}_{0}^{T}(\bm{p},i\omega_{m})],$ (4)

$\omega_{m}$ ($\nu_{n}$) is the Matsubara frequency for fermions (bosons),
${\rm Tr}$ is the trace over the Dirac indices and
$\mathcal{G}_{0}(\bm{p},i\omega_{m})=1/[(i\omega_{m}+\mu)\gamma_{0}-\bm{q}\cdot\bm{\gamma}]$
is the free quark propagator. By taking the analytic continuation
$\Xi^{R}(\bm{q},\omega)=\Xi(\bm{q},i\nu_{n}\rightarrow\omega+i\eta)$ one
obtains the retarded propagator. We remark that $[\Xi^{R}(\bm{0},0)]^{-1}=0$
is satisfied at $T=T_{c}$ determined by the MFA, which is nothing but the
Thouless criterion for determining the critical temperature of the second-
order phase transition. The Thouless criterion shows that the diquark
propagator $\Xi^{R}(\bm{q},\omega)$ has a pole at the origin at $T=T_{c}$, and
hence the diquark fluctuations have the properties of the soft mode [2, 3].

###  3.2 Photon self-energy

Figure 3: Contribution of the diquark fluctuations to the thermodynamic
potential.

To incorporate the effects of the diquark fluctuations into the photon self-
energy in a form that satisfies the WT identity, we start from the one-loop
diagram of $\Xi(\bm{q},i\nu_{n})$ shown in Fig. 3, i.e. the lowest
contribution of diquark fluctuations to the thermodynamic potential. The
photon self-energy is then constructed by attaching electromagnetic vertices
at two points of quark lines in Fig. 3. One then obtains four types of
diagrams shown in Fig. 4. These diagrams are called the Aslamasov-Larkin (AL)
(Fig. 4(a)) [5], Maki-Thompson (MT) (Fig. 4(b)) [6] and density of states
(DOS) (Fig. 4(c, d)) [7] terms in the theory of metallic superconductivity.
Each contribution to the photon self-energy, $\Pi_{\rm AL}^{\mu\nu}(k)$,
$\Pi_{\rm MT}^{\mu\nu}(k)$ and $\Pi_{\rm DOS}^{\mu\nu}(k)$, respectively, is
given by

$\displaystyle\Pi_{\rm AL}^{\mu\nu}(k)$ $\displaystyle=4N_{c}\
T\sum_{n}\int\frac{d^{3}q}{(2\pi)^{3}}\Gamma^{\mu}(q,q+k)\Xi(q+k)\Gamma^{\nu}(q+k,q)\Xi(q),$
(5) $\displaystyle\Pi_{\rm MT}^{\mu\nu}(k)$ $\displaystyle=2N_{c}\
T\sum_{n}\int\frac{d^{3}q}{(2\pi)^{3}}\Xi(q)R_{\rm MT}^{\mu\nu}(q,k),$ (6)
$\displaystyle\Pi_{\rm DOS}^{\mu\nu}(k)$ $\displaystyle=2N_{c}\
T\sum_{n}\int\frac{d^{3}q}{(2\pi)^{3}}\Xi(q)R_{\rm DOS}^{\mu\nu}(q,k),$ (7)

with

$\displaystyle\Gamma^{\mu}(q,q+k)$ $\displaystyle=2N_{f}(N_{c}-1)\
T\sum_{m}\int\frac{d^{3}p}{(2\pi)^{3}}{\rm
Tr}[\mathcal{G}_{0}(p)\gamma^{\mu}\mathcal{G}_{0}(p+k)\mathcal{G}_{0}(q-p)],$
$\displaystyle R_{\rm MT}^{\mu\nu}(q,k)$ $\displaystyle=2N_{f}(N_{c}-1)\
T\sum_{m}\int\frac{d^{3}p}{(2\pi)^{3}}{\rm
Tr}[\mathcal{G}_{0}(p)\gamma^{\mu}\mathcal{G}_{0}(p+k)\mathcal{G}_{0}(q-p-k)\gamma^{\nu}\mathcal{G}_{0}(q-p)],$
$\displaystyle R_{\rm DOS}^{\mu\nu}(q,k)$ $\displaystyle=2N_{f}(N_{c}-1)\
T\sum_{m}\int\frac{d^{3}p}{(2\pi)^{3}}\Big{\\{}{\rm
Tr}[\mathcal{G}_{0}(p)\gamma^{\mu}\mathcal{G}_{0}(p+k)\gamma^{\nu}\mathcal{G}_{0}(p)\mathcal{G}_{0}(q-p)]$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\quad+{\rm
Tr}[\mathcal{G}_{0}(p)\gamma^{\mu}\mathcal{G}_{0}(p-k)\gamma^{\nu}\mathcal{G}_{0}(p)\mathcal{G}_{0}(q-p)]\Big{\\}}.$

The total photon self-energy is then given by

$\displaystyle\Pi^{\mu\nu}(k)=\Pi_{\rm free}^{\mu\nu}(k)+\Pi_{\rm
fluc}^{\mu\nu}(k),\qquad\Pi_{\rm fluc}^{\mu\nu}(k)=\Pi_{\rm
AL}^{\mu\nu}(k)+\Pi_{\rm MT}^{\mu\nu}(k)+\Pi_{\rm DOS}^{\mu\nu}(k)$ (8)

where $\Pi^{\mu\nu}_{\rm free}(k)$ is the self-energy of the free quark system
and $\Pi^{\mu\nu}_{\rm fluc}(k)$ denotes the modification of the self-energy
due to the diquark fluctuations. One can explicitly check that Eq. (8)
satisfies the WT identity.

(a) |  (b) |  (c) |  (d)
---|---|---|---
Figure 4: Diagrammatic representations of the Aslamasov-Larkin (a), Maki-
Thompson (b) and density of states (c,d) terms in Eqs. (5)–(7). The wavy lines
represent photons.

###  3.3 Time-dependent Ginzburg-Landau (TDGL) approximation

The diagrams in Fig. 4 involve three-loop momentum integrals, which are
cumbersome to compute. Therefore, we employ an approximation that incorporates
essential effects of the diquark fluctuations near $T_{c}$ but, at the same
time, allows us to evaluate the diagrams with a relative ease.

Since $[\Xi^{R}(\bm{0},0)]^{-1}=0$ at $T=T_{c}$ by the Thouless criterion,
$\Xi^{R}(\bm{q},\omega)$ in the low energy-momentum region may be well
approximated near but above $T_{c}$ as follows,

$\displaystyle\Xi^{R}(\bm{q},\omega)=\frac{G_{C}}{c_{0}\omega+c_{1}\bm{q}^{2}+a},$
(9)

where the coefficients $c_{0}$, $c_{1}$ and $a$ can have $T$ dependence and
$a=0$ at $T=T_{c}$ from the Thouless criterion. We determine these
coefficients as $a=G_{C}[\Xi^{R}(\bm{0},0)]^{-1}$,
$c_{0}=G_{C}(\partial/\partial\omega)[\Xi^{R}(\bm{0},0)]^{-1}$ and
$c_{1}=G_{C}(\partial/\partial\bm{q}^{2})[\Xi^{R}(\bm{0},0)]^{-1}$ from
$\Xi^{R}(\bm{q},\omega)$ obtained in the NJL model. It is found that $c_{0}$
is complex while $c_{1}$ and $a$ real numbers. The approximation Eq. (9) is
called the time-dependent Ginzburg-Landau (TDGL) approximation in literature.
In Ref. [3], it has been shown that Eq. (9) reproduces the behavior of
$\Xi^{R}(\bm{q},\omega)$ over wide ranges of $\omega$, $\bm{q}^{2}$ and
$T(>T_{c})$.

Next we consider similar approximations for the vertex functions
$\Gamma^{\mu}(q,q+k)$ and $R^{\mu\nu}(q,k)=R_{\rm MT}^{\mu\nu}(q,k)+R_{\rm
DOS}^{\mu\nu}(q,k)$. Here, we consider such approximations only for the
spatial components of these vertices because Eq. (2) can be obtained only with
the spatial components of $\Pi^{R\mu\nu}(k)$; although Eq. (2) contains
$\Pi^{00}(k)$, this term is expressed in terms of the longitudinal part as
$\Pi^{R00}(k)=\bm{k}^{2}\Pi^{R11}(k)/k_{0}^{2}$ with $k=(k_{0},|\bm{k}|,0,0)$
from the WT identity. To approximate the spatial components
$\Gamma^{i}(q,q+k)$ and $R^{ij}(q,k)$ to be consistent with Eq. (9), we
substitute Eq. (9) into the WT identities for these vertices

$\displaystyle k_{\mu}\Gamma^{\mu}(q,q+k)=\Xi^{-1}(q+k)-\Xi^{-1}(q),$ (10)
$\displaystyle
k_{\mu}R^{\mu\nu}(q,k)=2[\Gamma^{\nu}(q-k,q)-\Gamma^{\nu}(q,q+k)].$ (11)

Then, by comparing the lowest order terms of $\bm{q}$ and $\bm{k}$ in Eqs.
(10) and (11) we obtain

$\displaystyle\Gamma^{i}(q,q+k)=\frac{\partial\Xi^{-1}(q+k)}{\partial
k_{i}}=\frac{c_{1}}{G_{C}}(2q+k)^{i},\quad
R^{ij}(q,k)=2\frac{\partial\Gamma^{j}(q,q-k)}{\partial
k_{i}}=-\frac{4c_{1}}{G_{C}}\frac{k^{i}k^{j}}{\bm{k}^{2}}.$ (12)

Each vertex in Eq. (12) is real, and this fact simplifies the analytic
continuation to obtain the retarded self-energy. One also finds that the
imaginary part of $\Pi^{Rij}_{\rm MT}(q)+\Pi^{Rij}_{\rm DOS}(q)$ calculated
with Eqs. (9) and (11) vanishes. Therefore, the MT and DOS terms do not
contribute to the dilepton production rate. This is in accordance with the
case of the metallic superconductivity [7].

##  4 Numerical results of dilepton production rate

|
---|---
Figure 5: The dilepton production rates per unit energy $\omega$ at $\bm{k}=0$
(left) and per invariant mass $M$ (right). In each panel, the thick (red)
lines are the contribution of diquark fluctuations $\Pi^{\mu\nu}_{\rm
fluc}(k)$, and the thin (green) lines are the results for the free quark gas.
The solid, dashed and dash-dotted lines are the results for $T=1.05T_{c}$,
$1.2T_{c}$ and $2T_{c}$.

In Fig. 5, we show the dilepton production rate Eq. (2) calculated from the
photon self-energy Eq. (8) at the quark chemical potential $\mu=350$ MeV for
several values of $T$ above $T_{c}$. Shown in the left panel is the production
rate per unit energy at $\bm{k}=0$. The thick (red) lines show the
contribution of diquark fluctuations $\Pi_{\rm fluc}^{\mu\nu}(k)$, while the
thin (green) lines are the results for free quarks obtained from $\Pi_{\rm
free}^{\mu\nu}(k)$. The total rate is given by the sum of these two
contributions. The figure shows that the production rate from the diquark
fluctuations is greatly enhanced in the low energy region compared with the
free quark gas for $T\lesssim 2T_{c}$, and this enhancement is more pronounced
as the system is closer to the critical temperature $T_{c}$. This is not
unexpected because the diquark fluctuations are the soft mode which acquires
more concentrated strength in the vicinity of the critical point. In the right
panel, we show the invariant-mass ($M$) spectrum

$\displaystyle\frac{d\Gamma}{dM^{2}}=\frac{1}{2\omega}\int
d^{3}k\frac{d^{4}\Gamma}{d^{4}k},$ (13)

which is more convenient for a comparison with experimental data. One sees
that the enhancement of the production rate at small $M$ is observed in the
invariant-mass spectrum for a similar temperature range, while the $T$
dependence of the enhancement is milder than the left panel.

##  5 Summary and concluding remarks

In this study, we investigated the effect of diquark fluctuations on the
dilepton production rate near but above the critical temperature of the 2SC.
The contribution of the diquark fluctuations was taken into account so as to
satisfy the WT identity in the TDGL approximation. It was found that the
dilepton production rate is greatly enhanced in comparison to the free-quark
gas case in the low energy and low invariant-mass regions near $T_{c}$. This
result suggests that such an enhancement can be used for the experimental
signal for the existence of the CSC phases. In particular, the fact that the
enhancement is seen even at $T\simeq 2T_{c}$ would allow us to detect the
signal in the HIC experiments even when $T_{c}$ is so small that the
realization of the CSC phase itself in the HIC is impossible.

## References

  * [1] M. G. Alford, A. Schmitt, K. Rajagopal and T. Schäfer, Rev. Mod. Phys. 80 (2008), 1455.
  * [2] M. Kitazawa, T. Koide, T. Kunihiro and Y. Nemoto, Phys. Rev. D 65 (2002), 091504.
  * [3] M. Kitazawa, T. Koide, T. Kunihiro and Y. Nemoto, Prog. Theor. Phys. 114 (2005), 117.
  * [4] T. Kunihiro, M. Kitazawa and Y. Nemoto, PoS CPOD07 (2007), 041.
  * [5] L. G. Aslamasov and A. L. Larkin, Sov. Phys. -Solid State 10 (1968), 875.
  * [6] K. Maki, Prog. Theor. Phys. 40 (1968), 193; R. S. Thompson, Phys. Rev. B 1 (1970), 327.
  * [7] A. I. Larkin and A. A. Varlamov, Fluctuation Phenomena in Superconductors (Springer Berlin Heidelberg, 2008).