# Knowledge Graph Completion based on Tensor Decomposition for Disease Gene
Prediction

Xinyan Wang1 Ting Jia1 Chongyu Wang1 Kuan Xu1 Zixin Shu1 Jian Yu1 Kuo
Yang1∗&Xuezhong Zhou1∗ 1Institute of Medical Intelligence, School of Computer
and Information Technology, Beijing Jiaotong University, Beijing 100044, China
{xinyan_wang<EMAIL_ADDRESS><EMAIL_ADDRESS>{xukuan,
18112032, jianyu, yangkuo<EMAIL_ADDRESS>
###### Abstract

Accurate identification of disease genes has consistently been one of the keys
to decoding a disease’s molecular mechanism. Most current approaches focus on
constructing biological networks and utilizing machine learning, especially,
deep learning to identify disease genes, but ignore the complex relations
between entities in the biological knowledge graph. In this paper, we
construct a biological knowledge graph centered on diseases and genes, and
develop an end-to-end Knowledge graph completion model for Disease Gene
Prediction using interactional tensor decomposition (called KDGene). KDGene
introduces an interaction module between the embeddings of entities and
relations to tensor decomposition, which can effectively enhance the
information interaction in biological knowledge. Experimental results show
that KDGene significantly outperforms state-of-the-art algorithms.
Furthermore, the comprehensive biological analysis of the case of diabetes
mellitus confirms KDGene’s ability for identifying new and accurate candidate
genes. This work proposes a scalable knowledge graph completion framework to
identify disease candidate genes, from which the results are promising to
provide valuable references for further wet experiments.

##  1 Introduction

Unraveling the molecular mechanism of disease is essential for realizing
precision medicine Ashley (2016). One of the main goals is to identify the
causing genes of disease. Traditional methods of identifying disease-causing
genes (e.g., Genome-wide association study Hirschhorn (2009)) are mainly
obtained through experiments, which are extremely time-consuming and labor-
intensive Calvo et al. (2007).

With the completion of the Human Genome Project and the maturity of high-
throughput sequencing technology Lander (2011), a growing body of computing-
based disease gene prediction methods have been developed, which are proven
effective Luo et al. (2021). Compared with traditional experiments, computing-
based methods can significantly save resources and reduce experimental errors
Wu et al. (2008a); Guo et al. (2019). Typical studies include network
propagation methods (Picart-Armada et al. (2019)), clustering or
classification methods (Care et al. (2009)), network features (Wu et al.
(2008b); Jalilvand et al. (2019)), and network embedding methods (Yang et al.
(2018a, b)). Neural network methods have also been applied to disease gene
prediction and obtained high performance Yang et al. (2020); Wu et al.
(2019b); Zhang et al. (2020).

In recent years, Knowledge Graphs (KGs) have been successfully applied to life
science research Lin et al. (2020). KG is a semantic network that reveals the
relations between entities, which can formally describe things and their
relations in the real world. In KGs, nodes represent entities or concepts, and
edges are composed of attributes or relations. Knowledge exists in the form of
triples Rossi et al. (2021). Inferring unknown facts from those already in KG
is called KG Completion (KGC). Performing better in existing KGC models, KG
Embedding (KGE) based methods learn the latent representation of entities and
relations in continuous vector space Wang et al. (2017). Network Embedding
(NE) assigns nodes in a network to low-dimensional representations and
preserves the network structure effectively Grover and Leskovec (2016). The
main difference between KGE and NE is that the latter focuses on the topology
of the network, while KGE focuses on the internal information of different
relations and the semantic connotation of facts.

A few studies have explored KGE-based methods individually for disease gene
prediction. They tend to adopt existing KGE models from the general domain
Nunes et al. (2021); Vilela et al. (2022) or use external information, such as
the textual description of biological entities Choi and Lee (2021). Although
the conventional KGE models have been proven to be useful for inferring new
biological relations, their performance with biological data is not as
satisfactory as that of general-domain KGs Choi and Lee (2019). One of the key
points is how to model KGE in the process of disease gene prediction to
accurately capture the interaction between biological entities (such as
Protein-Protein Interactions) Huang and Li (2010); Zhu et al. (2022), so that
diseases and genes can be learned with more comprehensive biological features.

To address these issues, we first integrated multiple relations centered on
diseases and genes from biomedical knowledge bases to construct a large-scale
biological KG, and develop an end-to-end Knowledge graph completion model
using an interactional tensor decomposition to identify Disease-Gene
associations, called KDGene. KDGene introduces a gating mechanism-based
interaction module between the embeddings of entities and relations to tensor
decomposition, which can effectively enhance the information interaction in
biological knowledge. Perceiving related knowledge, the model is capable of
learning the connotation of different relations and endows biological entities
and relations with more comprehensive and precise representations, which is
beneficial to disease gene prediction. Experimental results show that KDGene
performs optimally among existing disease gene prediction methods. In
particular, compared with conventional KGE methods, KDGene realizes an average
improvement of over 20% on HR and MAP metrics. Meanwhile, we evaluate the
impacts of KGs composed of knowledge with different relation types and degrees
of confidence on KDGene’s performance. In summary, the main contributions of
our work are three-fold:

  1. 1.

We construct a biological knowledge graph centered on diseases and genes, then
adopt a scalable end-to-end KGC framework to predict disease genes.

  2. 2.

We propose a novel KGC model, called KDGene, specifically for disease gene
prediction. The model introduces an interaction module to tensor
decomposition, which effectively enhances the information interaction between
biological knowledge.

  3. 3.

Our KDGene achieves state-of-the-art on disease gene prediction. The
biological analysis of diabetes mellitus also confirms KDGene’s ability to
identify new and accurate candidate genes.

##  2 Related Work

###  2.1 Disease Genes Prediction Models

Researchers have proposed various computing-based Disease Gene Prediction
(DGP) methods, mainly divided into four categories: (1) Network Propagation
methods. The network propagation models are based on the classic random walk
algorithm for the most part, and they are common in early disease gene
prediction tasks Vanunu et al. (2010); Erten et al. (2011). (2) Methods based
on Network Features. These methods usually use the constructed network to
obtain the topological feature information of nodes, then calculate the
correlation between a query disease and candidate genes, completing the
prediction by sorting the gene list Lage et al. (2007). (3) Supervised
Learning methods such as classification Yang et al. (2021).

And (4) Network Embedding and Deep Learning methods. These methods have gained
wide attention in recent years Alshahrani and Hoehndorf (2018); Lei et al.
(2021). HerGePred Yang et al. (2018b) is a heterogeneous disease-gene-related
network embedding representation framework for disease gene prediction which
can realize similarity prediction and a random walk with restart on a
reconstructed heterogeneous disease-gene network. GLIM Hou et al. (2022) can
systematically mine the potential relationships between multilevel elements by
embedding the features of the human multilevel network through contrastive
learning. With the development of deep learning technology, researchers have
tried to build specific neural network models to predict disease genes Yang et
al. (2020).

###  2.2 KGE Models

Recently, growing amounts of work have been proposed to learn distributed
representations for entities and relations in KGs, which fall into three major
categories Rossi et al. (2021): (1) Geometric Models. They utilize distance-
based scoring functions to measure the plausibility of facts, which interpret
the relation as a geometric transformation in the latent space Sun et al.
(2019). The most representative model, TransE Bordes et al. (2013), regards
the relation r in each triple (h, r, t) as a translation from the head entity
h to the tail entity t, by requiring the tail entity embedding lies close to
the sum of the head entity and relation embeddings. (2) Deep Learning Models.
These methods learn the embedding vectors of entity and relation using multi-
layer neural networks Dettmers et al. (2018).

And (3) Tensor Decomposition Models. In Canonical Polyadic (CP) decomposition
Hitchcock (1927), a tensor can be decomposed into a set of matrices, where
each row in the matrix represents an embedding vector of entity or relation.
DistMult Yang et al. (2014), a special case of CP decomposition, forces all
relation embeddings as diagonal matrices, which reduces the space of
parameters and makes the model easier to train. ComplEx Trouillon et al.
(2016) introduces asymmetry into tensor decomposition by adding complex-valued
embeddings so that it can simulate asymmetric relations. Since current
implementations of CP are lagging behind their competitors, CP-N3 Lacroix et
al. (2018) uses a tensor nuclear p-norm as a regularizer to break through the
limitations of CP and obtain good performance.

###  2.3 DGP models combined with KGE

At present, the DGP methods combined with KGE have not been fully exploited.
KGED Choi and Lee (2021) is a convolutional neural network-based KGE model,
which is based on a biological KG with entity descriptions to infer
relationships between biological entities. Since KGED is used to predict gene-
gene relations to generate gene interaction networks for diseases, it is not
an end-to-end model for DGP. And it requires textual descriptions of entities,
which may introduce noise and are not simple to obtain. In Nunes et al.
(2021), KGE is adopted to predict disease genes directly. However, this work
applies conventional methods in the KGC task without comparison with other DGP
models.

Figure 1:  Visualization of the KDGene architecture. After constructing the
biological KG, a triple is represented as $(h,r,t)$, with two entities $h,t$
and a relation $r$. We use $\bm{e_{h}},\bm{e_{t}}\in\mathbb{R}^{d_{e}}$ to
denote the embeddings of head and tail entities and
$\bm{e_{r}}\in\mathbb{R}^{d_{r}}$ to represent the relation embeddings. When
the embeddings $\bm{e_{h}},\bm{e_{r}},\bm{e_{t}}$ are trained, taking the
relation embedding $\bm{e_{r}}$ as the input, the head entity embedding
$\bm{e_{h}}$ as the hidden layer, we use the interaction module to obtain the
updated relation embedding $\bm{e_{r}^{\prime}}\in\mathbb{R}^{d_{e}}$. Then
the scoring function of triple $(h,r,t)$ is calculated by $\bm{e_{h}}$,
$\bm{e_{r}^{\prime}}$ and $\bm{e_{t}}$. After training, for a query disease,
score all candidate genes and rank by descending as the prediction results.

##  3 Preliminaries

Knowledge Graph $\mathcal{G}$. A knowledge graph can be denoted as
$\mathcal{G}=\\{\mathcal{E},\mathcal{R},\mathcal{T}\\}$ where $\mathcal{E}$
and $\mathcal{R}$ are the entity set and relation set, respectively. And
$\mathcal{T}=\\{(h,r,t)\in\mathcal{E}\times\mathcal{R}\times\mathcal{E}\\}$
denotes the triple set which consists of all the triple facts in
$\mathcal{G}$. When constructing a biomedical KG, we integrate knowledge from
different biological databases in the form of triples and add them to KG.
Correspondingly, the associated entity set and relation set are generated.

Knowledge Graph Completion. The task of KGC, also known as Link Prediction, is
to either predict unseen relations r between two existing entities: (h, ?, t),
or predict entities when the triple’s relation and another entity are given:
(h, r, ?) or (?, r, t). For Disease Gene Prediction, since triples of this
kind of facts are in the form (disease, disease_gene, gene), we focus on the
second mode to predict the tail entity (gene) given the head entity (disease)
and relation (disease_gene).

In this paper, we adopt the improved tensor decomposition-based model under
the framework of KGC, in which the triple (h, r, t) can be represented as an
element of the binary third-order entity-relation tensor
$\mathcal{X}^{\textit{N}\times\textit{M}\times\textit{N}}$, where
$\textit{N}=|\mathcal{E}|$ is the total number of entities and
$\textit{M}=|\mathcal{R}|$ the number of relations. In the entity-relation
tensor $\mathcal{X}$, $\mathcal{X}_{ikj}$ denotes that there is a k-th
relation between the i-th entity and the j-th one, which is:

$\displaystyle\mathcal{X}_{ikj}=\begin{cases}1,&\text{ if
}\left(h_{i},r_{k},t_{j}\right)\in\mathcal{G}\\\ 0,&\text{ if
}\left(h_{i},r_{k},t_{j}\right)\notin\mathcal{G}\end{cases}$ (1)

Therefore, tensor decomposition-based algorithms can infer a predicted tensor
$\mathcal{\widehat{X}}$ that approximates $\mathcal{X}$. To predict the
candidate genes of a disease, queries like (i, k, ?) are answered by ordering
gene entities $j^{\prime}$ by decreasing scoring values of
$\widehat{\mathcal{X}}_{ikj^{\prime}}$. Note that we propose a scalable KGC
framework for disease gene prediction which means the KGE model can be
replaced by others.

##  4 Methodology

We present KDGene, an end-to-end knowledge graph completion model based on
interactional tensor decomposition,

Entity Type | quantity | Relation Type | quantity
---|---|---|---
Disease | 22,697 | Disease-Protein | 117,738
Protein | 21,616 | Protein-Protein | 841,068
Symptom | 2,504 | Disease-Symptom | 184,831
GO | 1,207 | GO-Protein | 61,634
Pathway | 316 | Pathway-Protein | 25,813
Total | 48,340 | Total | 1,231,084
Table 1: The scale of our constructed biological knowledge graph related to
diseases and genes.

and formulate the problem of disease gene prediction as an LP task in KG.
Figure 1 shows an overview of KDGene, and the whole framework consists of the
following three parts:

  1. 1.

Biological KG Construction. We collected and integrated multiplex relations,
e.g., disease-gene, disease-symptom, protein-protein, protein-GO, and protein-
pathway from well-known biomedical databases and construct a biological KG
centered on diseases and genes.

  2. 2.

Representation learning of entities and relations in the biological KG. We do
not directly utilize the existing KGE methods but improve the tensor
decomposition model by introducing an interaction module, which can enhance
the information interaction in biological knowledge and learn the more
dedicated representation of relations.

  3. 3.

Scoring and ranking of disease candidate genes. Based on the representation
features learned in the second step, we score and sort all the candidate genes
of a disease according to Equation 6, and obtain predicted results of
candidate genes.

###  4.1 Biological KG Construction

To learn more comprehensive representations of diseases and genes, we
introduce the knowledge of different relation types to construct a biological
KG. Regarding diseases, the disease-symptom relations from SymMap Wu et al.
(2019a) are introduced into the KG. Regarding genes, we introduce the Protein-
Protein Interactions (PPI) from STRING Von Mering et al. (2005), the Protein-
GO relations from GO (2021), and the Protein-Pathway relations from
KEGGKanehisa and Goto (2000). Table 1 shows the scale of the biological KG.

In our framework, there are no restrictions on the entity type and relation
type which means the construction of the KG is flexible. When others use it,
the disease-gene relation facts can be added or subtracted from the KG to
complete the training according to the demand. As the amount of knowledge in
biomedical databases grows, abundant facts about new types of relations can be
continuously added to the KG.

###  4.2 CP-N3

The KGC task can be regarded as a 3D binary tensor completion problem, where
each slice is the adjacency matrix of one relation type in the KG. It is a
natural solution to apply tensor decomposition to the KGC task, which is
simple, expressive, and can achieve state-of-the-art results in general-domain
KGs. Here, we take the typical CP-N3 model as an example and further introduce
the interaction module on this basis to predict disease-gene associations.

CP-N3 Lacroix et al. (2018) is based on CP decomposition Hitchcock (1927),
which decomposes a high-order tensor $\mathcal{X}\in\mathbb{R}^{n_{1}\times
n_{2}\times n_{3}}$ into several $r$ rank one tensors
$u_{i}\in\mathbb{R}^{n_{1}},v_{i}\in\mathbb{R}^{n_{2}},w_{i}\in\mathbb{R}^{n_{3}}$
($\otimes$ denotes the tensor product):

$\displaystyle\mathcal{X}\approx\sum_{i=1}^{r}u_{i}\otimes v_{i}\otimes
w_{i}.$ (2)

###  4.3 Interaction Module

Introducing the interaction module aims to equip KGE models, TD-based methods
in particular, with better biomedical knowledge perception. That is, the model
should learn more precise representations of entities and relations. To deal
with the problem of long-term dependencies, Hochreiter and Schmidhuber
proposed long short-term memory (LSTM) Hochreiter and Schmidhuber (1997). They
improved the remembering capacity of the standard recurrent cell by
introducing a “gate” into the cell in which the gate mechanism can choose
which information enters the next cell Yu et al. (2019). We adopt the vanilla
LSTM cell Gers et al. (2000) consisting of an input gate, an output gate, and
a forget gate. The activation process of LSTM is as follows:

First, the forget gate $f$ and the input gate $i$ at the time step $t$ are
computed by

$\displaystyle f_{t}$
$\displaystyle=\sigma\left(W_{fh}h_{t-1}+W_{fx}x_{t}+b_{f}\right),$ (3)
$\displaystyle i_{t}$
$\displaystyle=\sigma\left(W_{ih}h_{t-1}+W_{ix}x_{t}+b_{i}\right),$
$\displaystyle\tilde{c}_{t}$
$\displaystyle=\tanh\left(W_{\tilde{c}h}h_{t-1}+W_{\tilde{c}x}x_{t}+b_{\tilde{c}}\right),$

where $\sigma$ is the logistic sigmoid function, and $x_{t}$ is the current
input. For the forget gate, the LSTM unit determines which information should
be removed from its previous cell states $h_{t-1}$. The candidate memory cell
is also added to the cell state through a TanH Layer. All the $W$ are weights
that need to be learned, while $b$ represents the bias vector associated with
this component. Then, the cell state is updated by

$\displaystyle c_{t}$ $\displaystyle=f_{t}\circ
c_{t-1}+i_{t}\circ\tilde{c}_{t},$ (4) $\displaystyle o_{t}$
$\displaystyle=\sigma\left(W_{oh}h_{t-1}+W_{ox}x_{t}+b_{o}\right),$
$\displaystyle h_{t}$ $\displaystyle=o_{t}\circ\tanh\left(c_{t}\right).$

$o_{t},h_{t}$ are the outputs at the current time, and $\circ$ is the Hadamard
product. In this intuitionistic structure, the control of the forget gate can
save the previous information, and the control of the input gate can prevent
the current irrelevant information from being added to the cell. The
information in each part sufficiently interacts with others, so we utilize
this simple and effective structure as our interaction module.

###  4.4 KDGene

We present KDGene, a knowledge graph completion model that introduces the
interaction module into CP-N3, which applies to disease gene prediction. In
the following, a triple is represented as $(h,r,t)$, with two entities $h,t\in
E$ (the set of entities) and a relation $r\in R$ (the set of relations). We
use $\bm{e_{h}},\bm{e_{t}}\in\mathbb{R}^{d_{e}}$ to denote the embeddings of
head and tail entities and $\bm{e_{r}}\in\mathbb{R}^{d_{r}}$ to represent the
relation embeddings.

Instead of adopting the translation-based principle $\bm{h}+\bm{r}=\bm{t}$ in
TransE Bordes et al. (2013), we use the gating mechanism as the entity-to-
relation translation. When the embeddings $\bm{e_{h}},\bm{e_{r}},\bm{e_{t}}$
are trained, taking the relation embedding $\bm{e_{r}}$ as the input, and the
head entity embedding $\bm{e_{h}}$ as the hidden layer, we use an LSTM cell to
obtain the updated relation embedding
$\bm{e_{r}^{\prime}}\in\mathbb{R}^{d_{e}}$. The calculation process is as
follows:

$\displaystyle f$
$\displaystyle=\sigma\left(W_{fh}\bm{e_{h}}+W_{fx}\bm{e_{r}}+b_{f}\right),$
(5) $\displaystyle i$
$\displaystyle=\sigma\left(W_{ih}\bm{e_{h}}+W_{ix}\bm{e_{r}}+b_{i}\right),$
$\displaystyle\tilde{c}$
$\displaystyle=\tanh\left(W_{\tilde{c}h}\bm{e_{h}}+W_{\tilde{c}x}\bm{e_{r}}+b_{\tilde{c}}\right),$
$\displaystyle c$ $\displaystyle=f\circ c_{0}+i\circ\tilde{c},$
$\displaystyle\bm{e_{r}^{\prime}}$
$\displaystyle=\sigma\left(W_{oh}\bm{e_{h}}+W_{ox}\bm{e_{r}}+b_{o}\right),$

where all $W$ are weight matrices and $b$ are bias vectors learned in the
training process. The initial input of the cell state is set to 0.

After getting the updated relation embedding $\bm{e_{r}^{\prime}}$, we define
the scoring function of a triple $(h,r,t)$ for KDGene as:

$\displaystyle\phi(h,r,t)=\sum_{i=1}^{d_{e}}e_{hi}\otimes
e_{ri}^{\prime}\otimes e_{ti}.$ (6)

In CP-N3, the embedding dimensions of entities and relations must be the same,
resulting in a lot of parameter redundancy for those datasets with very
different numbers of entities and relations. After introducing the interaction
module, the dimensions of entities and relations can be different, which
significantly improves the operability and flexibility of KDGene. More
importantly, through the gating mechanism of LSTM, entities, and relations are
learned with more precise representations, which will benefit disease gene
prediction.

###  4.5 Training and Prediction

We use the standard data augmentation techniques Lacroix et al. (2018) of
adding reciprocal predicates in the original training set $S$ and get
$S^{\prime}$, i.e. add $(t,r^{-1},h)$ for every $(h,r,t)$. Besides, we follow
the 1-N scoring introduced by Dettmers et al. (2018), that is, we take one
$(h,r)$ pair and score it against all entities $t^{\prime}\in E$
simultaneously. We train our model with the full multiclass log-loss:

$\displaystyle\mathcal{L}=\sum_{(h,r,t)\in
S^{\prime}}(-\phi(h,r,t)+log(\sum_{t^{\prime}\in
E}exp(\phi(h,r,t^{\prime})))).$ (7)

where $\mathcal{L}$ is the loss function that should be minimized. For KDGene,
we follow the N3 regularization used in CP-N3 Lacroix et al. (2018), and the
loss function for KDGene is as follows:

$\displaystyle\mathcal{L}=$ $\displaystyle\sum_{(h,r,t)\in
S^{\prime}}(-\phi(h,r,t)+log(\sum_{t^{\prime}\in E}exp(\phi(h,r,t^{\prime})))$
$\displaystyle+$
$\displaystyle\lambda\sum_{i}^{d_{e}}(|e_{hi}|^{3}+|e_{ri}^{\prime}|^{3}+|e_{ti}|^{3})).$
(8)

After training, for disease gene prediction, we take $(h,r)$ pairs, where the
head entity is the query disease, and the relation is disease-gene, and then
score all candidate genes that are not in the training set. The list of genes
with scores from high to low is the prediction result of the candidate genes.

DGP Models | HR@1 | HR@3 | HR@10 | HR@50 | MAP@1 | MAP@3 | MAP@10 | MAP@50
---|---|---|---|---|---|---|---|---
DADA | 0.012 | 0.025 | 0.047 | 0.107 | 0.045 | 0.044 | 0.049 | 0.053
GUILD | 0.023 | 0.032 | 0.049 | 0.107 | 0.073 | 0.076 | 0.080 | 0.084
RWRH | 0.082 | 0.153 | 0.269 | 0.486 | 0.297 | 0.268 | 0.272 | 0.286
PDGNet | 0.020 | 0.031 | 0.045 | 0.068 | 0.094 | 0.056 | 0.044 | 0.043
PRINCE | 0.006 | 0.011 | 0.024 | 0.074 | 0.025 | 0.026 | 0.028 | 0.031
RWR_PPI | 0.070 | 0.148 | 0.271 | 0.474 | 0.257 | 0.241 | 0.255 | 0.270
RWR_HMLN | 0.094 | 0.180 | 0.304 | 0.502 | 0.342 | 0.303 | 0.306 | 0.320
GLIM_DG | 0.105 | 0.194 | 0.312 | 0.508 | 0.383 | 0.335 | 0.329 | 0.342
KDGene (ours) | 0.126 | 0.243 | 0.416 | 0.620 | 0.406 | 0.365 | 0.361 | 0.370
Improvement | +19.96% | +25.16% | +33.25% | +22.04% | +5.82% | +8.91% | +9.48% | +8.12%

KGC Models | HR@1 | HR@3 | HR@10 | HR@50 | MAP@1 | MAP@3 | MAP@10 | MAP@50
---|---|---|---|---|---|---|---|---
TransE | 0.086 | 0.160 | 0.272 | 0.472 | 0.278 | 0.243 | 0.241 | 0.252
RotatE | 0.085 | 0.159 | 0.272 | 0.477 | 0.275 | 0.241 | 0.241 | 0.252
DistMult | 0.107 | 0.200 | 0.309 | 0.406 | 0.346 | 0.301 | 0.292 | 0.299
ComplEx | 0.103 | 0.193 | 0.317 | 0.515 | 0.331 | 0.288 | 0.281 | 0.291
TuckER | 0.096 | 0.182 | 0.288 | 0.394 | 0.308 | 0.269 | 0.261 | 0.269
CP-N3 | 0.090 | 0.165 | 0.273 | 0.471 | 0.290 | 0.249 | 0.244 | 0.254
KDGene (ours) | 0.126 | 0.243 | 0.416 | 0.620 | 0.406 | 0.365 | 0.361 | 0.370
Improvement | +17.24% | +21.44% | +31.15% | +20.35% | +17.23% | +21.12% | +23.52% | +23.59%

Table 2: Disease gene prediction results on DisGeNet. The table consists of
two parts, the upper part is about the baselines for typical Disease Gene
Prediction (DGP) models, and the lower is about the baselines of KGC models.
Bold numbers are the best results of all and underline numbers are the best
results of baseline models. Improved results of the last row compare the
performance of KDGene with the best of baseline models.

##  5 Experiments

###  5.1 Experimental setting

Dataset. We select curated disease-gene associations from the DisGeNet
database Piñero et al. (2016) as a benchmark dataset and apply the
conventional 10-fold cross validation to evaluate the disease gene prediction
algorithms. For each fold, there are 117,738 disease-gene associations in the
training set and 13,082 in the testing set.

Baselines. For baselines, comparisons with existing disease gene prediction
algorithms are essential. Typical models including DADA Erten et al. (2011),
GUILD Guney and Oliva (2012), RWRH Li and Patra (2010), PDGNet Yang et al.
(2020), PRINCE Vanunu et al. (2010), GLIM Hou et al. (2022) are our baseline.
In addition, since we formulate disease gene prediction as the KGC task, and
propose a novel KGC method, KDGene should also be compared with existing KGC
models. We experiment with six popular KGE baselines: TransE Bordes et al.
(2013), RotatE Sun et al. (2019), DistMult Yang et al. (2014), ComplEx
Trouillon et al. (2016), TuckER Balažević et al. (2019) and CP-N3 Lacroix et
al. (2018).

Evaluation Metrics. Following GLIM Hou et al. (2022), we select the hit ratio
(HR@N) and mean average precision (MAP@N) as evaluation metrics (where
N=1,3,10,50). For both HR and MAP, higher values indicate higher predictive
performance.

Implementation Details. We implement KDGene with PyTorch and have made our
source code available on GitHub.111https://github.com/sienna-wxy/KDGene In our
experiments, we carried out extensive grid search, over the following ranges
of hyperparameter values: batch size in {128, 256, 512, 1024}, learning rate
in {0.01, 0.03, 0.05, 0.1}, regularization coefficient in {0.001, 0.01, 0.05,
0.1, 0.2, 0.5}, the entity dimension in {1000, 1500, 2000, 2500} and the
relation dimension in {500, 1000, 1500, 2000}. The optimal parameters used in
KDGene can be seen on Github.1 Adagrad algorithm Duchi et al. (2011) is
adopted to optimize all trainable parameters. For all the baselines, we follow
the best hyperparameters they provide and obtain disease gene prediction
results on the DisGeNet dataset.

###  5.2 Results and Analysis

Table 2 reports the evaluation results of disease gene prediction on the
DisGeNet dataset. It can be seen that KDgene outperforms all the baselines
consistently and significantly. Specifically, compared with DGP baselines,
KDGene realizes an average improvement of 16.59% and over 25% improvement on
HR metric in particular. Compared with KGC baselines, in terms of HR@1, HR@3,
HR@10, HR@50, MAP@1, MAP@3, MAP@10, MAP@50, the performance gains achieved by
our model are 17.24%, 21.44%, 31.15%, 20.35%, 17.23%, 21.12%, 23.52%, 23.59%,
with an average improvement of 21.96%. These results illustrate the
effectiveness of KDGene for disease gene prediction.

Our results also suggest that the success of KDGene does not mean that KGC-
based models are the best in all DGP models (e.g., the MAP metric of GLIM_DG
is better than all existing KGC methods), but they can still outperform most
DPG models. It’s very promising for KGC-based methods to become the best in
terms of the HR metric, showing a powerful recall ability, which indicates
that combining complex relations in biological knowledge is beneficial for
predicting candidate genes comprehensively.

Existing DGP models combined with KGE Nunes et al. (2021) usually adopt
conventional KGC methods. Our experiments confirm that existing KGC-based
models are effective but not necessarily optimal. Among these typical KGC
models, methods based on tensor decomposition perform better. Adopting tensor
decomposition, we further introduce an interaction module based on the gating
mechanism. It is worth noting that the result of CP-N3 can be regarded as an
ablation study. Compared with CP-N3, KDGene realizes an increase by 39.86%,
47.35%, 52.12%, 31.81%, 39.85%, 46.30%, 47.52%, and 45.39% on HR@1, HR@3,
HR@10, HR@50, MAP@1, MAP@3, MAP@10, and MAP@50, respectively, with an anveage
improvement of 43.77%. The impressive improvement compared with CP-N3
demonstrates the significance of our proposed interaction module, which
enables KDGene with more precise representations of entities and relations to
predict disease genes.

###  5.3 Comparison of Different KGs

To evaluate the impact of KGs composed of different relations on KDGene, we
use different combinations of relations to construct biological KGs from
$KG_{1}$ to $KG_{6}$ and evaluate the performance of KDGene. The results are
shown in Figure 2. $KG_{1}$ consists only of the disease-gene facts in the
training set, with no external relations introduced. Based on $KG_{1}$,
$KG_{2}$ and $KG_{3}$ introduce the facts about Disease-Symptom and Protein-
Protein Interaction, respectively. $KG_{2}$ achieves the best performance,
indicating that disease-symptom associations are beneficial for candidate gene
prediction, while PPI has little effect. $KG_{4}$, which jointly introduces
the two relations, fails to achieve the cumulative effect. Referring Vilela et
al. (2022), $KG_{5}$ introduces GO and Pathway associations of genes. $KG_{6}$
reintroduces the disease-symptom relation based on $KG_{5}$. From the results,
the former gets no obvious improvement, and the latter has improved, but still
has not reached the performance of $KG_{2}$. The possible reason for the
performance degradation of $KG_{3}$ and $KG_{5}$ that further introduce
Protein-Protein or GO/Pathway-Protein associations is that the relations about
proteins add more similar entities into KG, which could be noise.

Figure 2:  Results of KDGene with different biological KGs. We use five
relations associated with disease and gene to evaluate the effect of six
combinations on KDGene performance. Except for the different KGs used, other
experimental settings are kept the same.

###  5.4 Comparison of Different PPI Score

To analyze the impact of knowledge with different confidence levels on KDGene,
we consider scores in Protein-Protein Interaction facts. This score is often
higher than the individual sub-scores, expressing increased confidence when an
association is supported by several types of evidence Von Mering et al.
(2005). We select three grades of scores for evaluation, that is, the
interaction scores $\geq 700$, $\geq 850$, and $\geq 950$, respectively. For a
fair comparison, all three evaluations are performed on the KG with the same
disease-gene facts, with no differences other than the PPI facts. In Figure
3a, as the score threshold increases, the performance of KDGene gradually
improves, which indicates the introduction of reliable biological knowledge
into the KG is more beneficial for KDGene to learn the representations of
entities and relations.

Figure 3:  a. Results of KDGene with facts in different confidence levels of
PPI. b. Results of KDGene with different interaction modules.

###  5.5 Comparison of Different interaction modules

To evaluate the impact of different interaction modules on the performance of
KDGene, we conduct experiments with similar structures such as RNNCell and
GRUCell, and the results are shown in Figure 3b. Among the three gating
mechanisms, the relation embedding is used as the input, and the head entity
embedding as the hidden layer. The results of introducing different
interaction structures are all better than the model without the gating
mechanism (here we compare with CP-N3). The results of introducing different
interaction structures are all better than the model without the gating
mechanism (here we compare with CP-N3), illustrating the significance of the
interaction module for tensor decomposition models. Among the three, the
result of LSTMCell is slightly better than the remaining two. The possible
reason is that the setting of the forget gate makes it more parameters to
learn more details.

Figure 4:  PPI Network of the known and candidate genes of Diabetes Mellitus.
The observed number of network links is significantly larger than the random
control (P = 2.05E-88, binomial test).

###  5.6 Case study

We take Diabetes Mellitus as a case example to illustrate the high network
closeness and functional relevance between genes in the training set and the
candidate genes predicted by KDGene. For the disease, we keep all 42 genes in
the training set and 6 genes in the testing set of the DisGeNet dataset and
take out the top 50 candidate genes predicted by KDGene. The dense links (221
real links vs. 41.23 expected links, P = 2.05E-88, binomial test) that hold in
the PPI network indicate that those two kinds of genes would tend to have
closer interactions than expectation and rely on the same functional module in
the PPI network. The top ten gene prediction results of KDGene hit three
diabetes mellitus-associated genes in the testing set. Meanwhile, we find that
more than half of the predicted genes have corresponding literature evidence
for association with diabetes. For example, interleukin-6 (IL-6, the top
predicted gene), is not in the testing set, but Ururahy et al. (2015)
indicates pro-inflammatory cytokines, such as interleukin-6 (IL-6), have been
considered as key factors in type 1 diabetes mellitus (T1DM) and diabetic
nephropathy. The results illustrate the accuracy and reliability of KDGene’s
prediction, which are promising to provide valuable references for further wet
experiments.

##  6 Conclusion

In this paper, we first utilize the biological knowledge bases to create KGs
and develop a scalable end-to-end knowledge graph completion model using an
interactional tensor decomposition to identify disease-gene associations.
KDGene introduces a gating mechanism-based interaction module between the
embeddings of entities and relations to tensor decomposition, which can
effectively enhance the information interaction in biological knowledge.
Experimental results show that KDGene performs optimally among existing
disease gene prediction methods. We also evaluate the impacts of KGs composed
of knowledge with different relation types and degrees of confidence on
KDGene’s performance. Furthermore, the comprehensive biological analysis of
the case of diabetes mellitus confirms KDGene’s ability to identify new and
accurate candidate genes. Future work will explore the capability of
attention-based interaction modules in disease gene prediction. We will also
extend this kind of module to other KGC models.

## References

  * Alshahrani and Hoehndorf [2018] Mona Alshahrani and Robert Hoehndorf.  Semantic disease gene embeddings (smudge): phenotype-based disease gene prioritization without phenotypes.  Bioinformatics, 34(17):i901–i907, 2018.
  * Ashley [2016] Euan A Ashley.  Towards precision medicine.  Nature Reviews Genetics, 17(9):507–522, 2016.
  * Balažević et al. [2019] Ivana Balažević, Carl Allen, and Timothy M Hospedales.  Tucker: Tensor factorization for knowledge graph completion.  arXiv preprint arXiv:1901.09590, 2019.
  * Bordes et al. [2013] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko.  Translating embeddings for modeling multi-relational data.  Advances in neural information processing systems, 26, 2013.
  * Calvo et al. [2007] Borja Calvo, Núria López-Bigas, Simon J Furney, Pedro Larrañaga, and Jose A Lozano.  A partially supervised classification approach to dominant and recessive human disease gene prediction.  Computer methods and programs in biomedicine, 85(3):229–237, 2007\.
  * Care et al. [2009] MA Care, JR Bradford, CJ Needham, AJ Bulpitt, and DR Westhead.  Combining the interactome and deleterious snp predictions to improve disease gene identification.  Human mutation, 30(3):485–492, 2009.
  * Choi and Lee [2019] Wonjun Choi and Hyunju Lee.  Inference of biomedical relations among chemicals, genes, diseases, and symptoms using knowledge representation learning.  IEEE Access, 7:179373–179384, 2019.
  * Choi and Lee [2021] Wonjun Choi and Hyunju Lee.  Identifying disease-gene associations using a convolutional neural network-based model by embedding a biological knowledge graph with entity descriptions.  Plos one, 16(10):e0258626, 2021.
  * Dettmers et al. [2018] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel.  Convolutional 2d knowledge graph embeddings.  In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  * Duchi et al. [2011] John Duchi, Elad Hazan, and Yoram Singer.  Adaptive subgradient methods for online learning and stochastic optimization.  Journal of Machine Learning Research, 12:2121–2159, 2011.
  * Erten et al. [2011] Sinan Erten, Gurkan Bebek, Rob M Ewing, and Mehmet Koyutürk.  Da da: degree-aware algorithms for network-based disease gene prioritization.  BioData mining, 4(1):1–20, 2011.
  * Gers et al. [2000] Felix A Gers, Jürgen Schmidhuber, and Fred Cummins.  Learning to forget: Continual prediction with lstm.  Neural computation, 12(10):2451–2471, 2000.
  * GO [2021] The gene ontology resource: enriching a gold mine.  Nucleic acids research, 49(D1):D325–D334, 2021.
  * Grover and Leskovec [2016] Aditya Grover and Jure Leskovec.  node2vec: Scalable feature learning for networks.  In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855–864, 2016.
  * Guney and Oliva [2012] Emre Guney and Baldo Oliva.  Exploiting protein-protein interaction networks for genome-wide disease-gene prioritization.  2012\.
  * Guo et al. [2019] Yucheng Guo, Chen Bao, Dacheng Ma, Yubing Cao, Yanda Li, Zhen Xie, and Shao Li.  Network-based combinatorial crispr-cas9 screens identify synergistic modules in human cells.  ACS synthetic biology, 8(3):482–490, 2019.
  * Hirschhorn [2009] Joel N Hirschhorn.  Genomewide association studies—illuminating biologic pathways.  New England journal of medicine, 360(17):1699–1701, 2009.
  * Hitchcock [1927] Frank L Hitchcock.  The expression of a tensor or a polyadic as a sum of products.  Journal of Mathematics and Physics, 6(1-4):164–189, 1927.
  * Hochreiter and Schmidhuber [1997] Sepp Hochreiter and Jürgen Schmidhuber.  Long short-term memory.  Neural computation, 9(8):1735–1780, 1997.
  * Hou et al. [2022] Siyu Hou, Peng Zhang, Kuo Yang, Lan Wang, Changzheng Ma, Yanda Li, and Shao Li.  Decoding multilevel relationships with the human tissue-cell-molecule network.  Briefings in Bioinformatics, 2022.
  * Huang and Li [2010] Yezhou Huang and Shao Li.  Detection of characteristic sub pathway network for angiogenesis based on the comprehensive pathway network.  BMC bioinformatics, 11(1):1–9, 2010.
  * Jalilvand et al. [2019] Ali Jalilvand, Behzad Akbari, Fatemeh Zare Mirakabad, and Foad Ghaderi.  Disease gene prioritization using network topological analysis from a sequence based human functional linkage network.  arXiv preprint arXiv:1904.06973, 2019.
  * Kanehisa and Goto [2000] Minoru Kanehisa and Susumu Goto.  Kegg: kyoto encyclopedia of genes and genomes.  Nucleic acids research, 28(1):27–30, 2000.
  * Lacroix et al. [2018] Timothée Lacroix, Nicolas Usunier, and Guillaume Obozinski.  Canonical tensor decomposition for knowledge base completion.  In International Conference on Machine Learning, pages 2863–2872. PMLR, 2018.
  * Lage et al. [2007] Kasper Lage, E Olof Karlberg, Zenia M Størling, Páll I Olason, Anders G Pedersen, Olga Rigina, Anders M Hinsby, Zeynep Tümer, Flemming Pociot, Niels Tommerup, et al.  A human phenome-interactome network of protein complexes implicated in genetic disorders.  Nature biotechnology, 25(3):309–316, 2007.
  * Lander [2011] Eric S Lander.  Initial impact of the sequencing of the human genome.  Nature, 470(7333):187–197, 2011.
  * Lei et al. [2021] Yipin Lei, Shuya Li, Ziyi Liu, Fangping Wan, Tingzhong Tian, Shao Li, Dan Zhao, and Jianyang Zeng.  A deep-learning framework for multi-level peptide–protein interaction prediction.  Nature Communications, 12(1):5465, 2021.
  * Li and Patra [2010] Yongjin Li and Jagdish C Patra.  Genome-wide inferring gene–phenotype relationship by walking on the heterogeneous network.  Bioinformatics, 26(9):1219–1224, 2010.
  * Lin et al. [2020] Xuan Lin, Zhe Quan, Zhi-Jie Wang, Tengfei Ma, and Xiangxiang Zeng.  Kgnn: Knowledge graph neural network for drug-drug interaction prediction.  In IJCAI, volume 380, pages 2739–2745, 2020.
  * Luo et al. [2021] Ping Luo, Bolin Chen, Bo Liao, and Fang-Xiang Wu.  Predicting disease-associated genes: Computational methods, databases, and evaluations.  Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11(2):e1383, 2021.
  * Nunes et al. [2021] Susana Nunes, Rita T Sousa, and Catia Pesquita.  Predicting gene-disease associations with knowledge graph embeddings over multiple ontologies.  arXiv preprint arXiv:2105.04944, 2021.
  * Picart-Armada et al. [2019] Sergio Picart-Armada, Steven J Barrett, David R Willé, Alexandre Perera-Lluna, Alex Gutteridge, and Benoit H Dessailly.  Benchmarking network propagation methods for disease gene identification.  PLoS computational biology, 15(9):e1007276, 2019.
  * Piñero et al. [2016] Janet Piñero, Àlex Bravo, Núria Queralt-Rosinach, Alba Gutiérrez-Sacristán, Jordi Deu-Pons, Emilio Centeno, Javier García-García, Ferran Sanz, and Laura I Furlong.  Disgenet: a comprehensive platform integrating information on human disease-associated genes and variants.  Nucleic acids research, page gkw943, 2016.
  * Rossi et al. [2021] Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo.  Knowledge graph embedding for link prediction: A comparative analysis.  ACM Transactions on Knowledge Discovery from Data (TKDD), 15(2):1–49, 2021.
  * Sun et al. [2019] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang.  Rotate: Knowledge graph embedding by relational rotation in complex space.  arXiv preprint arXiv:1902.10197, 2019.
  * Trouillon et al. [2016] Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard.  Complex embeddings for simple link prediction.  In International conference on machine learning, pages 2071–2080. PMLR, 2016.
  * Ururahy et al. [2015] Marcela Abbott Galvão Ururahy, Karla Simone Costa de Souza, Yonara Monique da Costa Oliveira, Melina Bezerra Loureiro, Heglayne Pereira Vital da Silva, Francisco Paulo Freire-Neto, Joao Felipe Bezerra, Andre Ducati Luchessi, Sonia Quateli Doi, Rosario Dominguez Crespo Hirata, et al.  Association of polymorphisms in il6 gene promoter region with type 1 diabetes and increased albumin-to-creatinine ratio.  Diabetes/metabolism research and reviews, 31(5):500–506, 2015.
  * Vanunu et al. [2010] Oron Vanunu, Oded Magger, Eytan Ruppin, Tomer Shlomi, and Roded Sharan.  Associating genes and protein complexes with disease via network propagation.  PLoS computational biology, 6(1):e1000641, 2010.
  * Vilela et al. [2022] Joana Vilela, Muhammad Asif, Ana Rita Marques, João Xavier Santos, Célia Rasga, Astrid Vicente, and Hugo Martiniano.  Biomedical knowledge graph embeddings for personalized medicine: Predicting disease-gene associations.  Expert Systems, page e13181, 2022.
  * Von Mering et al. [2005] Christian Von Mering, Lars J Jensen, Berend Snel, Sean D Hooper, Markus Krupp, Mathilde Foglierini, Nelly Jouffre, Martijn A Huynen, and Peer Bork.  String: known and predicted protein–protein associations, integrated and transferred across organisms.  Nucleic acids research, 33(suppl_1):D433–D437, 2005.
  * Wang et al. [2017] Quan Wang, Zhendong Mao, Bin Wang, and Li Guo.  Knowledge graph embedding: A survey of approaches and applications.  IEEE Transactions on Knowledge and Data Engineering, 29(12):2724–2743, 2017.
  * Wu et al. [2008a] Xuebing Wu, Rui Jiang, Michael Q Zhang, and Shao Li.  Network-based global inference of human disease genes.  Molecular systems biology, 4(1):189, 2008.
  * Wu et al. [2008b] Xuebing Wu, Rui Jiang, Michael Q Zhang, and Shao Li.  Network-based global inference of human disease genes.  Molecular systems biology, 4(1):189, 2008.
  * Wu et al. [2019a] Yang Wu, Feilong Zhang, Kuo Yang, Shuangsang Fang, Dechao Bu, Hui Li, Liang Sun, Hairuo Hu, Kuo Gao, Wei Wang, et al.  Symmap: an integrative database of traditional chinese medicine enhanced by symptom mapping.  Nucleic acids research, 47(D1):D1110–D1117, 2019.
  * Wu et al. [2019b] Ye Wu, Ruibang Luo, Henry Leung, Hing-Fung Ting, and Tak-Wah Lam.  Renet: A deep learning approach for extracting gene-disease associations from literature.  In International Conference on Research in Computational Molecular Biology, pages 272–284. Springer, 2019.
  * Yang et al. [2014] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng.  Embedding entities and relations for learning and inference in knowledge bases.  arXiv preprint arXiv:1412.6575, 2014.
  * Yang et al. [2018a] Kuo Yang, Ning Wang, Guangming Liu, Ruyu Wang, Jian Yu, Runshun Zhang, Jianxin Chen, and Xuezhong Zhou.  Heterogeneous network embedding for identifying symptom candidate genes.  Journal of the American Medical Informatics Association, 25(11):1452–1459, 2018.
  * Yang et al. [2018b] Kuo Yang, Ruyu Wang, Guangming Liu, Zixin Shu, Ning Wang, Runshun Zhang, Jian Yu, Jianxin Chen, Xiaodong Li, and Xuezhong Zhou.  Hergepred: heterogeneous network embedding representation for disease gene prediction.  IEEE journal of biomedical and health informatics, 23(4):1805–1815, 2018.
  * Yang et al. [2020] Kuo Yang, Yi Zheng, Kezhi Lu, Kai Chang, Ning Wang, Zixin Shu, Jian Yu, Baoyan Liu, Zhuye Gao, and Xuezhong Zhou.  Pdgnet: Predicting disease genes using a deep neural network with multi-view features.  IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2020.
  * Yang et al. [2021] Kuo Yang, Kezhi Lu, Yang Wu, Jian Yu, Baoyan Liu, Yi Zhao, Jianxin Chen, and Xuezhong Zhou.  A network-based machine-learning framework to identify both functional modules and disease genes.  Human Genetics, 140(6):897–913, 2021.
  * Yu et al. [2019] Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang.  A review of recurrent neural networks: Lstm cells and network architectures.  Neural computation, 31(7):1235–1270, 2019.
  * Zhang et al. [2020] Yiding Zhang, Lyujie Chen, and Shao Li.  Cipher-sc: disease-gene association inference using graph convolution on a context-aware network with single-cell data.  IEEE/ACM Transactions on Computational Biology and Bioinformatics, 19(2):819–829, 2020.
  * Zhu et al. [2022] Chaoyu Zhu, Zhihao Yang, Xiaoqiong Xia, Nan Li, Fan Zhong, and Lei Liu.  Multimodal reasoning based on knowledge graph embedding for specific diseases.  Bioinformatics, 38(8):2235–2245, 2022.