# Group extensions preserve almost finiteness

Petr Naryshkin  Petr Naryshkin, Mathematisches Institut, WWU Münster,
Einsteinstr. 62, 48149 Münster, Germany<EMAIL_ADDRESS>
###### Abstract.

We show that a free action $G\curvearrowright X$ is almost finite if its
restriction to some infinite normal subgroup of $G$ is almost finite. Consider
the class of groups which contains all infinite groups of locally
subexponential growth and is closed under taking direct limits and extensions
on the right by any amenable group. It follows that all free actions of a
group from this class on finite-dimensional spaces are almost finite and
therefore that minimal such actions give rise to classifiable crossed
products. In particular, that gives a much easier proof for the recent result
of Kerr and the author on elementary amenable groups.

##  1\. Introduction

When it comes to actions of discrete amenable groups, the Ornstein-Weiss
tiling machinery is one of the most useful tools. Being able to approximate
the space with a finite collection of Rokhlin towers provides a lot of
information on the structure of the action. In the context of ergodic theory
some of the applications are to the theory of orbit equivalence (originally
these techniques were used by Ornstein and Weiss to extend Dye’s theorem to
all amenable groups) and the structure of the von Neumann crossed products.
However, the classic tiling technique allows one only to approximate most of
the space. In the context of topological dynamics, this is often not enough:
instead, one would like to find an _exact_ partition of the space. In the
Borel setting, this is tightly linked to the question of hyperfinite
equivalence relations (see [1]). In the zero-dimensional setting, this leads
to almost finiteness.

Suppose $G\curvearrowright X$ is an action of a discrete countable amenable
group on a zero-dimensional compact metrizable space. We say that it is
_almost finite_ if for every finite subset $K$ of the group and every
$\varepsilon>0$ there is a finite collection $\\{(S_{i},V_{i})\\}_{i=1}^{n}$
such that $V_{i}$ are clopen subsets of $X$, $S_{i}$ are
$(K,\varepsilon)$-invariant subsets (meaning that $\left|S_{i}\Delta
KS_{i}\right|<\varepsilon\left|S_{i}\right|$) of the group and

$X=\bigsqcup_{i=1}^{n}\bigsqcup_{g\in S_{i}}gV_{i}.$

This was originally introduced by Matui [8] (in fact, for a more general case
of groupoids) and later extended in an appropriate way by Kerr [4] to actions
on spaces of arbitrary (including infinite) dimension (see Definition 2.2).

Currently, the main application of this notion is to the classification theory
of C∗-algebras. Specifically, to every topological dynamical system
$G\curvearrowright X$ one can associate a C∗-algebra $C(X)\rtimes G$, which is
called the crossed product of the system. The algebraic invariants of such
C∗-algebras have been of interest for a long time (starting with the works
such as [10] and [3]) and with the advent of the Elliott classification
program, much attention (see, for instance, [7], [12] and [11]) has been
devoted to understanding which crossed products fit into its scope (“are
classifiable”). In the same paper [4], Kerr proved that under some natural
assumptions almost finiteness is a sufficient criterion for the crossed
product to be classifiable. This allowed for the treatment of many systems
which were inaccessible with the prior methods. In fact, it is believed that
the following conjecture is true.

######  Conjecture.

Every free action of a countably infinite discrete amenable group on a finite-
dimensional compact metrizable space is almost finite.

By now this has been verified for a large class of amenable groups. Kerr and
Szabo showed in [6] (based on [2]) that the conjecture holds for groups of
locally subexponential growth. Later it was established for groups which admit
normal series of a certain form in [1], and, more recently, for all elementary
amenable groups in [5]. The proof of the latter goes as follows. It is clear
from the definition that the class of groups satisfying the conjecture is
closed under taking direct limits and it is also not hard to show that it is
closed under taking extensions on the right by finite groups. The bulk of work
goes into showing that this class is also closed under extensions on the right
by ${\mathbb{Z}}$. Finally, by the result of Osin [9], every elementary
amenable group can be obtained from the trivial group using these three
operation, which concludes the proof.

The main result of this paper is the following.

######  Theorem A.

Suppose $G\curvearrowright X$ is a free action of a countable discrete
amenable group on a compact metrizable space, $H\lhd G$ is an infinite normal
subgroup and the restricted action $H\curvearrowright X$ is almost finite.
Then $G\curvearrowright X$ is almost finite.

Note that in particular this covers both extensions on the right by finite
groups and by ${\mathbb{Z}}$ (unless $H$ is finite, in which case the
conclusion follows from [6]) and therefore gives a simple new proof of the
result in [5]. However, it also immediately provides a striking class of
examples not covered by the prior methods.

######  Corollary B.

Let $G$ be an amenable group. Then every free action of the group
$G\times{\mathbb{Z}}$ on a finite-dimensional space is almost finite.

More generally, we can describe the largest class of groups that are known to
satisfy the conjecture as follows.

######  Corollary C.

Consider the smallest class of groups which contains all infinite groups of
locally subexponential growth and is closed under taking countable direct
limits and extensions on the right by an amenable group. Then every free
action of a group from this class on a finite-dimensional space is almost
finite.

Acknowledgements. The author is grateful to Robin Tucker-Drob for pointing out
Remark 3.2 and to the anonymous reviewer for helpful comments. The research
was partially funded by the Deutsche Forschungsgemeinschaft (DFG, German
Research Foundation) under Germany’s Excellence Strategy – EXC 2044 –
390685587, Mathematics Münster – Dynamics – Geometry – Structure; the Deutsche
Forschungsgemeinschaft (DFG, German Research Foundation) – Project-ID
427320536 – SFB 1442, and ERC Advanced Grant 834267 - AMAREC.

##  2\. Preliminaries

It would be convenient for us to introduce the following framework.

######  Definition 2.1.

Let $G\curvearrowright X$ be an action of a countable discrete group on a
compact metrizable space. The _type semigroup_ is the abelian semigroup
generated by the open sets $\\{\left[U\right]\colon U\subset
X\,\mbox{open}\\}$ subject to the relations

(2.1) $\left[U\sqcup
V\right]=\left[U\right]+\left[V\right]\quad\mbox{and}\quad\left[gU\right]=\left[U\right]$

for every $g\in G$.

For open sets $U$ and $V$ we say that $U$ is _dynamically subequivalent_ to
$V$ and write $U\precsim_{G}V$ if for every compact subset $K\subset U$
there’s an open cover $K\subset\bigcup_{i=1}^{n}U_{i}$ and elements
$g_{1},g_{2},\ldots,g_{n}\in G$ such that $\\{g_{i}U_{i}\\}_{i=1}^{n}$ are
pairwise disjoint subsets of $V$. It is easily seen that $\precsim_{G}$ is
transitive and respects the relations (2.1). Thus, it extends to a partial
order on the type semigroup which we again denote by $\precsim_{G}$.

We say that the action $G\curvearrowright X$ _has comparison_ if whenever $U$
and $V$ are open sets such that $\mu(U)<\mu(V)$ for every $G$-invariant
measure $\mu$, then

$U\precsim_{G}V.$

Similarly, we’ll say that the action _has comparison on multisets_ if whenever
$\sum_{i=1}^{n}\left[U_{i}\right]$ and $\sum_{j=1}^{m}\left[V_{j}\right]$ are
elements in the type semigroup such that
$\sum_{i=1}^{n}\mu(U_{i})<\sum_{j=1}^{m}\mu(V_{j})$ for every $G$-invariant
measure $\mu$, then we have

$\sum_{i=1}^{n}\left[U_{i}\right]\precsim_{G}\sum_{j=1}^{m}\left[V_{j}\right].$

We recall the general definition of almost finiteness, due to Kerr.

######  Definition 2.2 ([4, Definition 8.2]).

Let $G\curvearrowright X$ be an action of a countable discrete amenable group
on a compact metrizable space. Fix a (not necessarily invariant) compatible
metric on $X$. We say that the action is _almost finite_ , if for every finite
subset $K\subset G$ and every $\varepsilon>0$ there exist a finite collection
$\\{(S_{i},V_{i})\\}_{i=1}^{n}$ (called a _castle_), where

  1. (1)

for every $i=1,2,\ldots,n$ the set $S_{i}\subset G$ (called _shape_) is finite
and $(K,\varepsilon)$-invariant, that is $\left|S_{i}\Delta
KS_{i}\right|<\varepsilon\left|S_{i}\right|$,

  2. (2)

for every $i=1,2,\ldots,n$ the set $V_{i}\subset X$ (called _base_) is open,

  3. (3)

the open sets $sV_{i}$ for all possible choices of $i=1,\ldots,n$ and $s\in
S_{i}$ are pairwise disjoint and have diameter smaller than $\varepsilon$,

  4. (4)

there are subsets $S_{i}^{\prime}\subset S_{i}$ with
$\left|S_{i}^{\prime}\right|<\varepsilon\left|S_{i}\right|$ such that

$X\setminus\bigsqcup_{i=1}^{n}S_{i}V_{i}\precsim_{G}\bigsqcup_{i=1}^{n}S_{i}^{\prime}V_{i}.$

The connection between almost finiteness and comparison has been studied by
Kerr and Szabó and it’s not hard to check that the same results hold for
comparison on multisets, which we record as follows.

######  Remark 2.3.

If the action $G\curvearrowright X$ is almost finite then it has comparison on
multisets. In particular, under the small boundary property (meaning that
there’s a base for the topology on $X$ with the boundaries being $\mu$-null
for every invariant measure $\mu$) comparison, comparison on multisets, and
almost finiteness are all equivalent. The proofs are exactly the same as in
[4, Theorem 9.2] and [6, Theorem A].

##  3\. Main results

We are now ready to prove the main result of this paper.

###### Proof of Theorem A.

Since $H\curvearrowright X$ is almost finite it has the small boundary
property ([6, Theorem A]). That implies that $G\curvearrowright X$ also has
the small boundary property. Thus, it is sufficient to prove that
$G\curvearrowright X$ has comparison. Let $A$ and $B$ be a open sets such that

(3.1) $\mu(A)<\mu(B)$

for every $G$-invariant measure $\mu$. It follows that there’s a sufficiently
big Følner set $F\subset G$ and $\gamma>0$ such that

(3.2) $\frac{1}{\left|F\right|}\sum_{g\in
F}\nu(gA)+\gamma<\frac{1}{\left|F\right|}\sum_{g\in F}\nu(gB)$

for all probability measures $\nu\in M(X)$. In particular, that is the case
for all $\nu\in M_{H}(X)$ (the space of all $H$-invariant probability measures
on $X$). Denote by $n$ the cardinality $\left|F\right|$.

Since $H\curvearrowright X$ is almost finite and $H$ is infinite, we can find
an open castle $\\{(V_{j},S_{j})\\}_{j=1}^{m}$ such that all the shapes
$S_{i}$ have cardinality at least $\frac{4n}{\gamma}$ and

$\nu\left(X\setminus\bigsqcup_{j=1}^{m}S_{j}V_{j}\right)<\frac{\gamma}{4}$

for every $H$-invariant measure $\nu$. Additionally, by the small boundary
property we may assume that $\nu(\partial V_{j})=0$, $\nu(\partial A)=0$, and
$\nu(\partial B)=0$ for every $\nu\in M_{H}(X)$. It follows that we can refine
each tower in the castle according to the pattern of intersections with $A$
and $B$ and that will not change the $\nu$-measure of the castle for any
$H$-invariant measure $\nu$. Thus, we may further assume that every level
$hV_{j}$, $h\in S_{j}$ is either fully contained in $A$ (respectively $B$) or
is disjoint from it. For $j=1,2,\ldots m$ denote by $a_{j}$ (respectively
$b_{j}$) the number of levels from the tower $(V_{j},S_{j})$ that are
contained in $A$ (respectively $B$). Denote by $R$ the remainder
$X\setminus\bigsqcup_{j=1}^{m}S_{j}V_{j}$ and use [6, Proposition 3.4] to find
an $\varepsilon$-neighbourhood $R^{\varepsilon}$ such that

$\nu\left(R^{\varepsilon}\right)<\frac{\gamma}{4}$

for all $\nu\in M_{H}(X)$.

Now for the action $G\curvearrowright X$ we have

$A\precsim_{G}\sum_{j=1}^{m}a_{j}\left[V_{j}\right]+\left[R^{\varepsilon}\right]\precsim_{G}\sum_{j=1}^{m}n\left\lceil\frac{a_{j}}{n}\right\rceil\left[V_{j}\right]+\left[R^{\varepsilon}\right]=\sum_{j=1}^{m}\sum_{g\in
F}\left\lceil\frac{a_{j}}{n}\right\rceil\left[gV_{j}\right]+\left[R^{\varepsilon}\right].$

Moreover, as $H$ is a normal subgroup, the action $G\curvearrowright X$
induces an action $G\curvearrowright M_{H}(X)$ by
$\nu\mapsto\nu\left(g\cdot\right)$. We shall denote the latter by $g_{*}\nu$.
Thus, for every $H$-invariant measure $\nu$,

$\nu\left(\sum_{j=1}^{m}\sum_{g\in
F}\left\lceil\frac{a_{j}}{n}\right\rceil\left[gV_{j}\right]+\left[R^{\varepsilon}\right]\right)-\frac{1}{n}\sum_{g\in
F}\nu(gA)<\sum_{j=1}^{m}\sum_{g\in
F}\left(\left\lceil\frac{a_{j}}{n}\right\rceil-\frac{a_{j}}{n}\right)g_{*}\nu(V_{j})+\gamma/4<\gamma/2$

since for every $g\in G$ and every $\nu\in M_{H}(X)$

$\sum_{j=1}^{m}g_{*}\nu(V_{i})\leq\sup\limits_{\nu\in
M_{H}(X)}\sum_{j=1}^{m}\nu(V_{i})\leq\frac{1}{\min\limits_{1\leq j\leq
m}\left|S_{j}\right|}\leq\frac{\gamma}{4n}.$

Similarly,

$\sum_{j=1}^{m}\sum_{g\in
F}\left\lfloor\frac{b_{j}}{n}\right\rfloor\left[gV_{i}\right]\precsim_{G}B$

and

$\nu\left(\sum_{j=1}^{m}\sum_{g\in
F}\left\lfloor\frac{b_{j}}{n}\right\rfloor\left[gV_{i}\right]\right)>\frac{1}{n}\sum_{g\in
F}\nu(gB)-\gamma/2.$

for every $\nu\in M^{H}(X)$. It follows from (3.2) that

$\nu\left(\sum_{j=1}^{m}\sum_{g\in
F}\left\lceil\frac{a_{j}}{n}\right\rceil\left[gV_{j}\right]+\left[R^{\varepsilon}\right]\right)<\nu\left(\sum_{j=1}^{m}\sum_{g\in
F}\left\lfloor\frac{b_{j}}{n}\right\rfloor\left[gV_{i}\right]\right)$

for every $\nu\in M^{H}(X)$ and since the action of $H$ has comparison of
multisets we have

$\sum_{j=1}^{m}\sum_{g\in
F}\left\lceil\frac{a_{j}}{n}\right\rceil\left[gV_{j}\right]+\left[R^{\varepsilon}\right]\precsim_{H}\sum_{j=1}^{m}\sum_{g\in
F}\left\lfloor\frac{b_{j}}{n}\right\rfloor\left[gV_{i}\right],$

and therefore

$A\precsim_{G}B.$

∎

######  Remark 3.1.

Note that under the same assumptions as in Theorem A except that the action
$G\curvearrowright X$ is free the proof still yields that it has comparison.

###### Proof of Corollaries B and C.

It is known from [6] (based on the result of [2]) that all free actions of
infinite groups of subexponential growth on finite-dimensional spaces are
almost finite. Clearly, direct limits preserve this property and from Theorem
A extensions on the right do as well. In particular, this applies to the
extensions of the form ${\mathbb{Z}}\lhd G\times{\mathbb{Z}}$. ∎

######  Remark 3.2.

When the space $X$ is zero-dimensional every free action of a finite group is
almost finite. It follows that all free actions of infinite groups which
contain finite normal subgroups of arbitrary large cardinality on zero-
dimensional spaces are almost finite (and thus also on finite-dimensional
spaces by [6, Theorem B]). Indeed, one can proceed as in the proof of Theorem
A and choose a finite normal subgroup $H$ of sufficiently large cardinality
based on the sets $A$ and $B$.

## References

  * [1] C. Conley, S. Jackson, A. Marks, B. Seward, and R. Tucker-Drob. Borel asymptotic dimension and hyperfinite equivalence relations. To appear in Duke Math. J.
  * [2] T. Downarowicz and G. Zhang. Symbolic extensions of amenable group actions and the comparison property. Mem. Am. Math. Soc. 281 (2023).
  * [3] G. A. Elliott and D. E. Evans. The structure of the irrational rotation C∗-algebra. Ann. of Math. (2) 138 (1993), 477–501.
  * [4] D. Kerr. Dimension, comparison, and almost finiteness. J. Eur. Math. Soc. 22 (2020), 3697–3745.
  * [5] D. Kerr and P. Naryshkin. Elementary amenability and almost finiteness. arXiv:2107.05273.
  * [6] D. Kerr and G. Szabó. Almost finiteness and the small boundary property. Comm. Math. Phys. 374 (2020), 1–31.
  * [7] H. Lin and N. C. Phillips. Crossed products by minimal homeomorphisms. J. reine angew. Math.  641 (2010), 95–122.
  * [8] H. Matui. Homology and topological full groups of étale groupoids on totally disconnected spaces. Proc. Lond. Math. Soc. (3) 104 (2012), 27–-56.
  * [9] D. V. Osin. Elementary classes of groups. Math. Notes 72 (2002), 75–82.
  * [10] I. F. Putnam. On the topological stable rank of certain transformation group C∗-algebras. Ergodic Theory Dynam. Systems 10 (1990), 197–207.
  * [11] G. Szabó. The Rokhlin dimension of topological ${\mathbb{Z}}^{m}$-actions. Proc. Lond. Math. Soc. (3) 110 (2015), 673–694.
  * [12] A. S. Toms and W. Winter. Minimal dynamics and K-theoretic rigidity: Elliott’s conjecture. Geom. Funct. Anal. 23 (2013), 467–481.


Journal \- Preprint

Gema et al.

[$\dagger$]Joint first author. [$\ddagger$]Joint senior authorship.
[$\ast$]Corresponding authors<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
# Knowledge Graph Embeddings in the Biomedical Domain: Are They Useful?
A Look at Link Prediction, Rule Learning, and Downstream Polypharmacy Tasks

Aryo Pradipta Gema0009-0007-1163-3531  Dominik Grabarczyk0009-0001-4340-430X
Wolf De Wulf0000-0002-0219-3120  Piyush Borole0000-0003-3327-5847  Javier
Antonio Alfaro0000-0002-5553-6991  Pasquale Minervini0000-0002-8442-602X
Antonio Vergari0000-0003-0036-5678  Ajitha Rajan0000-0003-3765-3075  School of
Informatics, University of Edinburgh, Edinburgh, United Kingdom  International
Centre for Cancer Vaccine Science, University of Gdańsk, Gdańsk, Poland
Department of Biochemistry and Microbiology, University of Victoria, British
Columbia, Canada

(2023; 2023)

###### Abstract

Motivation: Knowledge graphs are powerful tools for representing and
organising complex biomedical data. Several knowledge graph embedding
algorithms have been proposed to learn from and complete knowledge graphs.
However, a recent study demonstrates the limited efficacy of these embedding
algorithms when applied to biomedical knowledge graphs, raising the question
of whether knowledge graph embeddings have limitations in biomedical settings.
This study aims to apply state-of-the-art knowledge graph embedding models in
the context of a recent biomedical knowledge graph, BioKG, and evaluate their
performance and potential downstream uses.
Results: We achieve a three-fold improvement in terms of performance based on
the HITS@10 score over previous work on the same biomedical knowledge graph.
Additionally, we provide interpretable predictions through a rule-based
method. We demonstrate that knowledge graph embedding models are applicable in
practice by evaluating the best-performing model on four tasks that represent
real-life polypharmacy situations. Results suggest that knowledge learnt from
large biomedical knowledge graphs can be transferred to such downstream use
cases.
Availability and implementation: Our code is available at
https://github.com/aryopg/biokge.

###### keywords:

Knowledge Graphs, Knowledge Graph Embeddings, Polypharmacy, Rule-Based
Learning, Transfer Learning

##  1 Introduction

Knowledge Graphs (KGs) are increasingly utilised for knowledge representation
in the biomedical domain. Recent studies show that KGs can be utilised to aid
drug repurposing research schultzMethodRationalSelection2021  and to predict
the side effects of drug combinations zitnikModelingPolypharmacySide2018 ;
carlettiPredictingPolypharmacySide2021 . To maximise the utility of KGs in
this domain, comprehensive coverage of entities and links is essential. A
novel biomedical KG, called BioKG walshBioKGKnowledgeGraph2020 , has been
developed to be the first in the domain that attempts to agglomerate a wide
range of entity and link types. However, accurately predicting links between
entities in KGs can be challenging.

Knowledge Graph Embeddings (KGEs) offer a solution by representing KGs in a
low-dimensional space ferrariComprehensiveAnalysisKnowledge2022 . However, the
potential utility of KGEs in the biomedical field remains underexplored. A
recent study reports the limited success of KGEs for a biomedical KG
bonnerUnderstandingPerformanceKnowledge2022 , raising the question of whether
KGE methods have reached their maximum efficacy in this domain.

In this study, we show that it is possible to learn to accurately predict
links in BioKG. By taking into account recent studies outlining the best
practices for KGE algorithms ruffinelliYouCANTeach2020 , previously determined
limitations can be overcome. Furthermore, the pretrained KGE models are also
transferable to four downstream polypharmacy tasks, suggesting that a transfer
learning paradigm where KGE models trained on large KGs are adapted for
solving downstream tasks is feasible. In addition, we investigate the efficacy
of a rule-based model, called Anytime Bottom-Up Rule Learning (AnyBURL;
meilickeAnytimeBottomUpRule2019, ), for BioKG. Such a rule-based model offers
some degree of interpretability, which is important in the biomedical domain.

In summary, this study presents the following contributions:

  * •

A comprehensive evaluation of KGEs for BioKG using recent training best
practices, which reveals significant HITS@10 and mean reciprocal rank (MRR)
improvement compared to previous results
bonnerUnderstandingPerformanceKnowledge2022 . The best-performing KGE model
(ComplEx) reaches 0.793 HITS@10, compared to 0.286 in
bonnerUnderstandingPerformanceKnowledge2022 .

  * •

An investigation of the interpretability of a rule-based model for BioKG.
AnyBURL achieves a competitive HITS@10 score of 0.677 while providing
interpretable rules.

  * •

An investigation of applying KGE models in real-world tasks. The best-
performing pretrained KGE model can easily be adapted to four downstream
polypharmacy tasks in a transfer learning paradigm.

##  2 Background

DB00860D000544MIM:104300P04150D019970Q16678P34980Drug
TargetDPIDrDiAPDiAPPIGeDiDrDiA Figure 1: An extract of BioKG
walshBioKGKnowledgeGraph2020 . The nodes represent entities in the KG, edges
between them are links. The variety in identifier structure shows that BioKG
is a combination of multiple smaller KGs. In this extract, the centre node
(DB00860) represents the drug prednisolone, which targets the _Glucocorticoid
receptor_ (P04150). This receptor is associated with disorders related to or
resulting from the use of cocaine (D019970), indicated by the Protein-Disease-
Association relation (PDiA). Hence, _prednisolone_ is connected to said
disorders through the Drug-Disease-Association relation (DrDiA). The right-
most node (D000544) represents _Alzheimer’s disease_ , a genetic disorder
(GeDi). One possible application that uses the information in the KG would be
to train a model to predict missing links. Such a model could consider
information from, for example, the Drug-Drug Interaction (DDI) and Protein-
Protein Interaction (PPI) relations starting from prednisolone to predict that
prednisolone could also be used to treat _Alzheimer’s disease_ , as indicated
by the dashed DrDiA relation between them.

###  2.1 Link Prediction in Knowledge Graphs

KGs are a knowledge representation formalism in which knowledge about the
world is modelled as relationships between entities hoganKnowledgeGraphs2022 .
A KG can be represented as a set of subject-predicate-object triples, where
each $(s,p,o)$ triple represents a relationship of type $p$ between the
subject $s$ and the object $o$. Link Prediction (LP) is the task of
identifying missing triples, i.e. triples encoding true facts that are missing
from the KG. Consider, for example, the extract of BioKG presented in Figure
1. It contains information about prednisolone (DB00860), a drug that targets
the Glucocorticoid receptor. An LP model that is trained on BioKG could be
used to fill in blanks in triples such as $(\text{DB00860},\text{DrDiA},\\_)$,
effectively predicting other disorders that prednisolone could treat.
Similarly, such a model could be used to fill in blanks in triples such as
$(\\_,\text{DrDiA},\text{D000544})$, where D000544 is Alzheimer’s disease,
predicting drugs that could be repurposed to treat Alzheimer’s. Recent work
does this using a different biomedical KG, finding prednisolone likely to be
associated with Alzheimer’s disease nianMiningAlzheimerDiseases2022 .
Moreover, early-stage investigations have confirmed that high doses of
prednisolone can result in some delay of cognitive decline
ricciarelliAmyloidCascadeHypothesis2017 .

Before considering such real-life applications, an LP model should be
sufficiently evaluated using adequate baselines to avoid wasting resources in
failed pharmaceutical trials. An LP model’s generalisation capabilities are
evaluated using rank-based metrics. To do so, the KG is partitioned into
training, validation, and test triples. For each test triple $(s,p,o)$,
trained models are used to predict the subject or the tail, i.e. fill in
blanks in $(\\_,p,o)$ or $(s,p,\\_)$, respectively. The resulting triples are
then ranked based on how the model scores them. Subsequently, triples aside
from $(s,p,o)$ that exist in the training, validation, or test sets are
filtered out such that other triples that are known to be true do not
influence the ranking. The resulting ranking is ultimately used to calculate
metrics such as the MRR or the average HITS$@k$ (see section 9).

###  2.2 Knowledge Graph Embeddings

A prevalent class of LP models come in the form of KGEs, which represent
entities and relations as low-dimensional vectors and use a scoring function
to indicate the plausibility of a triple. Many models and training paradigms
for embedding knowledge graphs have been proposed
wangKnowledgeGraphEmbedding2017 ; ferrariComprehensiveAnalysisKnowledge2022 .
Models usually differ in how entities and relation representations are used to
compute the likelihood of a link in the KG. Examples are translational models
such as TransE bordesTranslatingEmbeddingsModeling2013 , TransH
wangKnowledgeGraphEmbedding2014 , and RotatE sunRotatEKnowledgeGraph2019 ,
factorisation models such as DistMult yangEmbeddingEntitiesRelations2015  and
ComplEx trouillonComplexEmbeddingsSimple2016 , and neural-network models such
as ConvE dettmersConvolutional2DKnowledge2018 . Table 1 provides a summary of
the domains in which these KGEs embed and the scoring functions they use.

Table 1:  An overview of KGE models, with the domain they embed in ($d$
corresponds to the embedding size) and their scoring function. Here, $*$
denotes the convolution operation, $\text{Re}(x)$ is the real part of
$x\in\mathbb{C}$,
$\langle\bm{x},\bm{y},\bm{z}\rangle=\sum_{i}\bm{x}_{i}\bm{y}_{i}\bm{z}_{i}$
denotes the tri-linear dot product, $g(x)$ is a nonlinear function,
$\text{vec}(x)$ is the flattening operator, $w$ denotes the convolutional
filter, and $W$ denotes a linear transformation matrix.

Model | Domain | Scoring function $f(\bm{e_{s}},\bm{r_{p}},\bm{e_{o}})$
---|---|---
TransE bordesTranslatingEmbeddingsModeling2013  | $\mathbb{R}^{d}$ | $-\|\bm{e_{s}}+\bm{r_{p}}-\bm{e_{o}}\|$
TransH wangKnowledgeGraphEmbedding2014  | $\mathbb{R}^{d}$ | $-\|(\bm{e_{s}}-\bm{w_{p}}^{\top}\bm{e_{s}}\bm{w_{p}})+\bm{r_{p}}-(\bm{e_{o}}-\bm{w_{p}}^{\top}\bm{e_{o}}\bm{w_{p}})\|$
RotatE sunRotatEKnowledgeGraph2019  | $\mathbb{C}^{d}$ | $-\|\bm{e_{s}}\circ\bm{r_{p}}-\bm{e_{o}}\|$
DistMult yangEmbeddingEntitiesRelations2015  | $\mathbb{R}^{d}$ | $\langle\bm{e_{s}},\bm{r_{p}},\bm{e_{o}}\rangle$
ComplEx trouillonComplexEmbeddingsSimple2016  | $\mathbb{C}^{d}$ | $\text{Re}(\langle\bm{e_{s}},\bm{r_{p}},\bm{e_{o}}\rangle)$
ConvE dettmersConvolutional2DKnowledge2018  | $\mathbb{R}^{d}$ | $g(\text{vec}(g([\bm{e_{s}},\bm{r_{p}}]*w))W)\bm{e_{o}}$
|  |

The paradigms wherein these models are usually trained can vary in several
ways, with free variables such as the loss function, regularisation,
initialisation, and data augmentation strategies (ruffinelliYouCANTeach2020,
). Furthermore, KGs generally do not explicitly contain negative triples.
However, for a KGE to be trained in a way that allows it to differentiate
between true and false triples, negative triples do need to be generated
explicitly at training time. Different approaches of generating negative
samples consist of randomly corrupting certain selections of triples, possibly
filtering out corrupted triples that already exist in the KG
bordesTranslatingEmbeddingsModeling2013 ;
lacroixCanonicalTensorDecomposition2018 ; ruffinelliYouCANTeach2020 .
Investigating the appropriate settings of these hyperparameters is important.
While certain combinations of settings have been found to often significantly
outperform others, the KG itself still dictates which settings are best
ruffinelliYouCANTeach2020 .

###  2.3 Rule Learning

A different class of algorithms, called rule learning algorithms, predict
links through logical rules extracted from the KG
meilickeAnytimeBottomUpRule2019 . As an example, take the following rule:

$\displaystyle(\text{DB00860},\text{DrDiA},\text{D019970})\rightarrow(\text{DB00860},\text{DrDiA},\text{D000544}),$

which states that if prednosolone is a drug associated with cocaine-related
disorders, it can also be a drug associated with Alzheimer’s disease,
simulating the LP example in a rule learning context. Such a rule may arise
from frequently observed occurrences of drugs associated with both diseases
and a lack of occurrences of drugs that only affect one of the two diseases.

A major advantage of rule learning algorithms is that, since predictions are
based on concrete rules, one can provide explanations for the predictions.
Even if such rules are not easily understandable by humans, the effort to
assess the likelihood that a prediction is sound might still be useful. For
example, in scenarios like drug discovery, where experimental confirmation is
costly. An argument often used against rule-based algorithms is that, when
applied to large KGs, these systems would struggle with the exponentially
increasing search space for possible rules, leading to a large computational
overhead or incomplete rule-bases zhangIterativelyLearningEmbeddings2019 .
Furthermore, rule learning models do not produce a latent representation of
entities and relations. Within the confines of a singular task, this is not a
problem, but it means it is impossible to use these models as foundational
models for downstream tasks.

##  3 Evaluation Setup

###  3.1 Dataset

####  3.1.1 BioKG

Though a great number of biological databases exist, most are highly
specialised in the kind of data they describe, focusing, for example, only on
proteins auerDBpediaNucleusWeb2007 ; szklarczykSTRINGDatabase20112011  or
drugs wishartDrugBankKnowledgebaseDrugs2008 . Most attempts to adapt such data
sources into KGs tailor them to specific projects. This limits possible
downstream tasks compared to more generalist KGs. BioKG minimises these
limitations by combining knowledge from various databases, such as UniProt
auerDBpediaNucleusWeb2007 , KEGG kanehisaKEGGReferenceResource2016 , and
Reactome croftReactomeDatabaseReactions2011 . BioKG utilises metadata to
provide valuable insights and predictions, focusing on a high-level analysis
based on entity relations rather than on in-depth information, such as
sequence data. While it also includes structural information, specifically
through the MEMBER_OF_COMPLEX relation, its core strength lies in it providing
a macroscopic view of complex biomedical data.

Figure 2: Frequencies of relationship types in BioKG on a logarithmic scale.

In this work, we use BioKG version 1.0.0111https://github.com/dsi-
bdi/biokg/releases/tag/v1.0.0 which contains 2,067,998 entries across 17
relations. The number of entries per relationship varies extremely, as can be
seen in Figure 2. An important metric of a KG is the degree of its nodes. One
recent study reasons that the in-degree has a large impact on how complex a
model has to be to perform well on a KG dettmersConvolutional2DKnowledge2018 .
A summary of these relationship statistics can be found in section 10.

A caveat of BioKG is that it does not explicitly track the directionality of
relationships. Defining whether a relationship should be directed, however, is
not a trivial task in the context of BioKG and requires a closer look at the
data in question. For example, drug-drug interaction (DDI) is a relationship
between two entities of the same class, and trivially, if drug A interacts
with B, then the reverse should hold and be explicitly modelled. With
relationships denoting class membership, such as
$(\text{cat},\text{is\\_a},\text{mammal})$, the reverse does not hold.
However, for associative relationships between members of different
categories, for example, drug-protein interaction (DPI), the matter is less
clear. While the relationship is undirected, as the same protein will interact
with the drug as well, these entries in BioKG always appear in the same order
(drug before protein). Modelling this as a directed edge might be sufficient,
and adding the reverse might further imbalance the dataset. Therefore,
reverses are only explicitly added in the cases of DDI and protein-protein
interaction (PPI) since these are the only categories where the same entity
class appears on both sides of the relationship, and all other categories have
a consistent ordering in their subject and object classes.

####  3.1.2 BioKG Polypharmacy Tasks

Four smaller KGs that centre around the discovery of drug targets and the
study of DDIs are published alongside BioKG. These KGs were constructed by
building on pre-existing benchmark datasets and leveraging larger and more
current KGs. The KGs are called DDI-Mineral, DDI-Efficacy, DPI-FDA, and DPI-
FDA-EXP.

DDI-Mineral is centred around the study of DDIs and their association with
abnormal mineral levels in the human body, with a particular emphasis on
potassium, calcium, sodium, and glucose. The KG is based on a previous KG
zitnikModelingPolypharmacySide2018 , which employs a dated TWOSIDES dataset
tatonettiDataDrivenPredictionDrug2012 . DDI-Mineral improves on this by
integrating the more current and comprehensive DrugBank dataset. It comprises
56,017 DDI triples across eight undirected relation types pertaining to 922
distinct drugs and their correlation with an increased or decreased likelihood
of developing an abnormal mineral level.

DDI-Efficacy focuses on the relationship between DDIs and the therapeutic
efficacy of the interacting drugs. It is similar to DDI-Mineral but with a
specific focus on the polypharmacy side effects in relation to the efficacy of
interacting drugs. It comprises 136,127 DDI triples across two undirected
relation types that involve 3,342 unique drugs and their effect on the
therapeutic efficacy of the interacting drugs, whether an increase or a
decrease. Similar to DDI-Mineral, DDI-Efficacy is based on the DrugBank
dataset.

DPI-FDA focuses on drug target protein interactions of FDA-approved drugs,
compiled from the KEGG kanehisaKEGGReferenceResource2016  and DrugBank
databases. It contains 18,928 drug-protein interactions that involve 2,277
drugs and 2,654 proteins. DPI-FDA is an extension of the previously published
DrugBank_FDA wishartDrugBankKnowledgebaseDrugs2008  and Yamanishi09
yamanishiPredictionDrugTarget2008  KGS, with 9,881 triples and 5,127 triples,
respectively. In contrast, DPI-FDA is derived from more modern versions of the
databases, leading to a more comprehensive and up-to-date representation of
the drug target protein interactions of FDA-approved drugs.

DPI-FDA-EXP focuses on the effects of FDA-approved drugs on the protein
expression levels in living systems. It comprises 903,429 triples with two
directed relations describing the effects of 1,291 drugs on the expression of
55,196 proteins.

###  3.2 Models

Figure 3:  A visualisation of the KGE transfer learning framework from a large
and all-encompassing KG, such as BioKG, to specific and purpose-driven
downstream KGs. The KGE model is first pretrained using BioKG framed as an LP
task. The pretrained LP model can then be utilised in more specific downstream
KGs. In our study, we use the pretrained LP entity embedding and initialise
the relation embedding from scratch as the downstream tasks introduce new
types of relation. However, theoretically, one may use both pretrained LP
entity and relation embedding should the downstream use cases contain a subset
of the relation types in the larger KG.

####  3.2.1 Link Prediction

We evaluate the six KGE models listed in Table 1, as well as one rule-based
model called AnyBURL. These specific models were selected because they are
widely recognised and commonly used in comparable studies
bonnerUnderstandingPerformanceKnowledge2022 ; ruffinelliYouCANTeach2020 .
Additionally, AnyBURL is included to provide a rule-based baseline for
comparison against the KGE models.

All evaluations concerning the KGE models are performed using the LIBKGE
Python package libkge . The hyperparameters of the KGE models are optimised on
the validation split through 30 quasi-random hyperparameter optimisation (HPO)
trials. The negative sampling method and loss function are always fixed.
Previous work on KGs with statistics similar to those of BioKG has shown that
1vsAll and CE loss are often the best choices for the selection of models that
are implemented in this work ruffinelliYouCANTeach2020 . However, this
combination is computationally infeasible for RotatE and TransH. Whenever
1vsAll was found to be infeasible, negative sampling is used. A table that
contains all the hyperparameters and the ranges over which they are optimised
can be found in section 11. Seeded LIBKGE configuration files and the
training, validation, and test splits (80/10/10) that allow reproducing all
the HPO runs can be found on the accompanied code repository.

To assess the value of transfer learning from BioKG to downstream
applications, the best-performing LP model is also evaluated on the BioKG
polypharmacy tasks, referred to as the pretrained LP model. We compare this
pretrained LP model to one trained from scratch in terms of the HITS@10 and
MRR scores as well as the number of epochs required to reach the peak
performance. Figure 3 visualises this transfer learning framework.

AnyBURL is trained using the original Java
implementation222https://web.informatik.uni-mannheim.de/AnyBURL/ at version
23-1. One key difference with the KGE models is that no negative sampling is
necessary and that there is no loss function. Instead, the hyperparameters are
limited to the training time, the confidence threshold to retain rules, and
the maximum rule length. Optimal training time depends significantly on the
number of allocated CPUs; in the case of this work, 76 CPUs were used. HPO is
performed by grid search, with training times between 100 and 1000 in
intervals of 100, thresholds of 0.1, 1 and 10 and rule lengths of 1, 2, 3, and
4. The source code for the AnyBURL evaluation can be found on a separate
repository333https://github.com/Dominko/biokg_anyburl

####  3.2.2 Relation Prediction

In addition to LP, the BioKG polypharmacy tasks can be further analysed as
multiclass relation classification tasks, where the focus specifically lies on
predicting the correct relations given pairs of entities as inputs. In such a
context, the task KGs require an additional relation type that denotes no
interaction between the entities to make the tasks more realistic. In
practice, the model receives a pair of entities, such as ‘Nebivolol’ and
‘Lofexidin’, and predicts the corresponding relation. For instance, in DDI-
Efficacy, the model should predict either ‘decrease therapeutic efficacy”,
‘increase therapeutic efficacy’, or ‘no effect’.

The objective of this study is to evaluate the performance of a classification
model that uses a pretrained LP embedding taken from the best-performing
pretrained LP model in BioKG compared to a model that is initialised from
scratch. To do so, an entity embedding layer is extracted from the pretrained
LP model that performs best on BioKG, with the addition of a softmax output
layer. The baseline setup has an identical network architecture and uses the
same hyperparameters but does not have the pretrained LP embedding weights.
Figure 3 visualises this transfer learning framework to downstream tasks
framed as relation prediction tasks. The goal is to verify the hypothesis that
the model with the pretrained LP embedding outperforms the baseline model.
Additionally, the performance of the same model with pretrained and frozen LP
embedding is compared to the same model with pretrained and fine-tuned LP
embedding. During training, all the models are configured to use an embedding
vector of size 512, a batch size of 512, and a learning rate of $110-4$. The
model performance is evaluated with standard classification metrics, i.e. Area
Under the Receiver Operating Characteristic curve (AUROC), Area Under the
Precision-Recall curve (AUPRC), and Mean Average Precision (MAP).

##  4 Results & Analyses

###  4.1 Link Prediction in BioKG

subsection 4.1 presents the LP results for BioKG for each of the best-found
configurations of the models. The complete configurations of the KGE models
can be found in Table 7 in section 12. In terms of MRR and HITS@10, ComplEx
performs best. Generally, the results suggest that factorisation models such
as ComplEx and DistMult outperform other classes of models. Translational
models perform worst. This indicates that similarity-based scoring functions
might be a good fit for biomedical KGs such as BioKG. An interesting
observation is that the best DistMult configuration is able to achieve a
relatively high HITS@10 of 0.667 and MRR of 0.471 in a relatively low number
of epochs (14) and with a relatively small embedding size (128). More
extensive HPO might allow DistMult to achieve scores similar to those of
ComplEx. If so, this would suggest that embedding in complex space instead of
in Euclidean space is not a requirement for performing well on BioKG. ConvE
performs similarly to ComplEx, indicating that a substantial amount of
information can be captured from a triple’s local neighbourhood
dettmersConvolutional2DKnowledge2018 .

As a baseline for the KGE models, a comparison is made to results in the
literature. Specifically, those of a recent study that performs similar
evaluations on the same version and splits of BioKG
bonnerUnderstandingPerformanceKnowledge2022 . The authors perform HPO runs for
all KGE models except for ConvE. The results presented in subsection 4.1 are
all significantly better than what is reported in the previous study
bonnerUnderstandingPerformanceKnowledge2022 . For ComplEx, the aforementioned
study reports a HITS@10 of $0.012$, compared to $0.793$ here. For DistMult,
this is $0.082$, compared to $0.667$ here. The best-performing model in the
previous study is RotatE, which achieved a HITS@10 of $0.286$, compared to
$0.618$ here, and compared to $0.793$ of the best-performing model here
(ComplEx). TransE and TransH also achieve higher scores, with HITS@10 of
$0.239$ and $0.080$ compared to $0.474$ and $0.574$ here, respectively. Since
the data and splits are all the same, an explanation for these results must
lie elsewhere. The only differences with the evaluations in
bonnerUnderstandingPerformanceKnowledge2022  are the way negative samples are
generated and the loss function. In
bonnerUnderstandingPerformanceKnowledge2022 , negative sampling and margin
ranking loss are implemented for all models. Based on the evaluations on KGs
with statistics similar to those of BioKG in another work
ruffinelliYouCANTeach2020 , 1vsAll (where feasible, negative sampling
otherwise, see Section 3.2.1) and CE loss seemed like better choices. The
presented results confirm that this is the case, which underlines the
importance of optimal training setup.

Table 2: LP performance of KGE models and one rule-based model on BioKG. Each
entry corresponds to the best configuration found in 30 quasi-random HPO
trials.

Model | Epochs | Emb. Size | MRR | HITS@10
---|---|---|---|---
This work |  bonnerUnderstandingPerformanceKnowledge2022 ∗∗
ComplEx | 184 | 512 | 0.629 | 0.793 | 0.012
DistMult | 14 | 128 | 0.471 | 0.667 | 0.082
TransE | 124 | 256 | 0.273 | 0.474 | 0.239
TransH | 200 | 256 | 0.280 | 0.574 | 0.080
RotatE | 200 | 1024 | 0.422 | 0.618 | 0.286
ConvE | 94 | 1024 | 0.599 | 0.765 | –
AnyBURL | 1000s* | – | 0.130 | 0.677 | –
∗ AnyBURL training time is in seconds.
∗∗ Average over their five best HPO trials.

###  4.2 Rule Learning

With the standard setup, AnyBURL achieves HITS@10 similar to that of DistMult;
however, its MRR lies significantly lower than most models. This indicates
that AnyBURL often ranks triples at the tail-end of 10 predictions.
Furthermore, due to its rule-based nature, AnyBURL sometimes does not generate
the considered triple (see Section 2.3 on AnyBURL). Consequently, some triples
may not contribute to the MRR of AnyBURL. The percentage of triples for which
this is the case is around 13%. This does not happen with the KGE models,
which are always able to generate a score for a triple. This phenomenon
influences HITS@10 less compared to MRR, which contributes to the discrepancy
between the two. Particularly because counting a zero towards HITS@10 carries
less weight than counting a zero towards MRR.

The hyperparameter optimisation of AnyBURL does not yield significant
differences. The threshold has no notable impact on the Hits@10 and MRR.
Increasing rule length by 1, makes the runtime, memory, and storage
requirements prohibitively high. It should also be noted that AnyBURL
explicitly separates learning and inference. Hits@10 is close to the peak
after as little as 400 seconds, but inference takes up to 12 hours with 76
CPUs.

Nevertheless, AnyBURL predictions can be connected back to the rules that
generate them, and thus provide explanations for why they have been made.
Consider the examples generated using AnyBURL below:

$\displaystyle(x,\text{DrDiA},\text{D006099})$
$\displaystyle\rightarrow(x,\text{DrDiA},\text{D006973})$ (1)
$\displaystyle(x,\text{DrDiA},\text{D013610})$
$\displaystyle\rightarrow(x,\text{DrDiA},\text{D007022})$ (2)

Rule (1) states that if drug $x$ is associated with granuloma (D006099), then
it will also be related to hypertension (D006973). Similarly, rule (2) states
that if drug $x$ interacts with Tachycardia (D013610), it will also interact
with hypotension (D007022.) AnyBURL also supplies a confidence score in the
form of the total number of occurrences of the right-hand side and co-
occurrences of both. For example, for the rule (2), there were 238 occurrences
and 150 co-occurrences in the training data, resulting in a confidence score
of 63%. These might be interesting in contexts such as drug discovery, where
specialists can interpret such statistical connections to assess their
likelihood in the real world.

###  4.3 Adapting to Downstream Polypharmacy Tasks

We use the best-performing model for BioKG, ComplEx, and evaluate it on the
BioKG polypharmacy KGs. Table 3 presents the results for all four polypharmacy
KGs using ComplEx in two different configurations - (1) when CompleEx is
trained from scratch, and (2) when it is initialised with the embeddings of
the configuration of ComplEx that performed best on BioKG (denoted as
ComplEx-P). The generation of negative triples and the loss function are kept
fixed as 1vsAll and CE, respectively. To use the pretrained LP embeddings, the
embedding size is fixed. To improve comparability, this is also done for the
instance of ComplEx trained from scratch.

Table 3: Performance of ComplEx and ComplEx-P on the BioKG polypharmacy KGs as LP tasks, where “-P” indicates that ComplEx was initialised with pretrained LP embeddings. Each entry corresponds to the best configuration found in 30 quasi-random HPO trials. The generation of negative samples and the loss function are fixed as 1vsAll and CE, respectively. The embedding size is fixed at 512, such that the learnt embeddings of the best ComplEx configuration from subsection 4.1 can be used as initialisation. Benchmark | Model | Epochs | HITS@10 | MRR
---|---|---|---|---
DDI-Efficacy | ComplEx | 196 | 0.962 | 0.847
| ComplEx-P | 52 | 0.975 | 0.865
DDI-Minerals | ComplEx | 106 | 0.976 | 0.861
| ComplEx-P | 54 | 0.987 | 0.884
DPI-FDA | ComplEx | 76 | 0.549 | 0.386
| ComplEx-P | 4 | 0.742 | 0.542
DEP-FDA-EXP | ComplEx | 72 | 0.287 | 0.171
| ComplEx-P | 22 | 0.304 | 0.185
|  |  |  |

Both models perform very well for DDI-Efficacy and DDI-Minerals. A HITS@10 in
the 90s and MRR in the 80s indicates that ComplEx is able to accurately rank
general triples and specific triples. Performance for the DPI-FDA and DEP-FDA-
EXP KGs lies lower. It must be noted that DPI-FDA is a significantly smaller
KG compared to the others, whereas DEP-FDA-EXP is a significantly larger KG.
Hence, overfitting and underfitting, respectively, must be considered as
causes of these results. In general, these two benchmarks seem to be
relatively harder.

When comparing ComplEx to ComplEx-P, two observations can be made. Firstly,
ComplEx-P’s best-found configuration requires significantly fewer epochs to
reach similar or better performance for all four benchmarks. Secondly, for
DPI-FDA and DEP-FDA-EXP, the best configuration for ComplEx-P produces more
accurate results in comparison with the best configuration for ComplEx. The
obvious decrease in epochs required for ComplEx-P to reach performance similar
to that of ComplEx strongly suggests that the pretrained LP embeddings, in the
least, allow for a positive transfer of knowledge in the form of a good
initialisation for these tasks. The case of DPI-FDA is particularly
interesting because it resembles a common scenario in the biomedical world
where there is little training data. The results show that performance and
training time in these scenarios can be improved upon by pretraining using
larger KGs.

Table 4 presents the results for all four tasks when the focus lies on
predicting the relation only. For DDI-Efficacy, DDI-Minerals, and DPI-FDA, all
models demonstrate near-perfect performance in all classification metrics.
This is not the case for DEP-FDA-EXP, indicating that this task is more
complex than the others. The pretrained and fine-tuned model shows the best
AUROC and MAP scores, with 0.908 and 0.759, respectively. The pretrained but
frozen model shows the best AUPRC score of 0.812. These results suggest that
the polypharmacy tasks are trivial when framed as relation classification
tasks. Consequently, no conclusions can be drawn regarding knowledge transfer
in this setting. Further study should look into more difficult tasks to verify
whether models with pretrained LP embedding weights can perform better than
models that are trained from scratch.

Table 4: Classifier performance on the BioKG polypharmacy KGs as relation prediction tasks. In the model column, “NN” denotes a fully-connected feed-forward neural network, “-PF” denotes a pretrained and frozen embedding, and “-P” denotes a pretrained and fine-tuned embedding. Benchmark | Model | AUROC | AUPRC | MAP
---|---|---|---|---
DDI-Efficacy | NN | 0.970 | 0.936 | 0.935
| NN-PF | 0.968 | 0.933 | 0.933
| NN-P | 0.970 | 0.944 | 0.938
DDI-Minerals | NN | 0.998 | 0.995 | 0.995
| NN-PF | 0.997 | 0.986 | 0.986
| NN-P | 0.998 | 0.995 | 0.995
DPI-FDA | NN | 1 | 1 | 1
| NN-PF | 1 | 1 | 1
| NN-P | 1 | 1 | 1
DEP-FDA-EXP | NN | 0.889 | 0.791 | 0.738
| NN-PF | 0.908 | 0.761 | 0.759
| NN-P | 0.891 | 0.812 | 0.739
|  |  |  |

##  5 Conclusion

Various models are evaluated for performing LP and relation classification on
a biomedical KG, i.e. BioKG walshBioKGKnowledgeGraph2020 . The presented
results show that the performance of the models are superior compared to those
reported in a recent similar study bonnerUnderstandingPerformanceKnowledge2022
. The results suggest that factorisation models such as ComplEx and DistMult
and neural network models such as ConvE generally perform better than the
other classes of models. The relative success of ConvE suggests that further
investigation into neural network models could be fruitful. Graph Neural
Networks (GNN), also known to capture local information
zhouGraphNeuralNetworks2020 , could be interesting to evaluate.

ComplEx achieves HITS@10 of 0.793 and MRR of 0.629 on BioKG, followed by ConvE
with 0.765 HITS@10 and 0.599 MRR. As the best-performing model on BioKG,
ComplEx is evaluated on the BioKG benchmarks. The evaluations compare ComplEx
when trained from scratch with ComplEx when initialised with the embeddings of
the configuration of ComplEx that performed best on BioKG. The comparison
shows that pretrained ComplEx requires significantly fewer epochs to
outperform ComplEx without pretraining. Particularly, in a task with little
training data (DPI-FDA), pretrained ComplEx significantly outperforms its non-
pretrained counterpart, with HITS@10 of 0.742 compared to 0.549, respectively.

Though the rule learning model AnyBURL does not achieve as high a performance
as the embedding models, its extremely short training time and explainable
rules make it an interesting avenue for future research. Particularly,
increasing the rule length might yield a more competitive performance, though
performance and memory bottlenecks need to be overcome, by, for example,
creating approaches using GPUs rather than CPUs or better rule aggregation
methods, such as SAFRAN ottSAFRANInterpretableRulebased2021 .

An additional experiment frames the BioKG benchmark tasks as relation
classification tasks. Similar to the LP benchmarking setting, the best-
performing ComplEx configuration is utilised further in this experiment. The
entity embedding that ComplEx learns for BioKG is extracted to be used as
features in classification tasks. The results show that the BioKG benchmark
tasks are trivial when framed as relation classification tasks. A simple feed-
forward NN model can achieve near-perfect results, and pretrained LP
embeddings provide only marginal improvements in performance in the DEP-FDA-
EXP task, with 0.908 AUROC compared to 0.889. Future studies should analyse
the triviality of the downstream polypharmacy tasks more closely. Although we
reach near-perfect performance here, the task of predicting the relation
between these entities is generally considered to be hard
bansalCommunityComputationalChallenge2014 ;
tatonettiDataDrivenPredictionDrug2012 . Future work should evaluate the
usefulness of pretrained LP embedding weights on challenging tasks that
require multi-hop reasoning with different entity types.

In summary, this study emphasises the potential utility of KGEs for predicting
yet-to-be-known interactions between biomedical entities. This capability has
the potential to reduce costs in biomedical research, such as optimising the
selection of candidate compounds in drug research. This study also
investigates an interpretable rule-based model which performs comparably and
may be of interest in biomedical research that prioritises explainability.
Furthermore, this study demonstrates that knowledge in KGEs is transferable
from large and comprehensive KGs such as BioKG walshBioKGKnowledgeGraph2020
to data-sparse domains such as polypharmacy prediction. The results from four
downstream polypharmacy tasks highlight the feasibility of implementing such
approaches in scenarios where data collection is expensive, which are exactly
the domains in which effective candidate selection carries the largest
benefits.

##  6 Competing interests

No competing interest is declared.

##  7 Author contributions statement

A.P.G. conceived the general experiments planning. A.P.G. worked on relation
prediction. D.G. worked on the rule-learning experiments. W.D.W. worked on the
knowledge graph embedding models for the link prediction experiments. A.P.G.,
D.G., and W.D.W. analysed and interpreted the results and wrote the manuscript
jointly. P.B. and J.A.A. helped with understanding the KG entities and
relations. A.R., A.V. and P.M. supervised the project and reviewed the
manuscript.

##  8 Acknowledgements

This work was supported by the United Kingdom Research and Innovation (grant
EP/S02431X/1), UKRI Centre for Doctoral Training in Biomedical AI at the
University of Edinburgh, School of Informatics. For the purpose of open
access, the author has applied a creative commons attribution (CC BY) licence
to any author-accepted manuscript version arising.

This work made use of the resources provided by the Edinburgh Compute and Data
Facility (ECDF, www.ecdf.ed.ac.uk/), as well as those provided by the
Cambridge Service for Data-Driven Discovery (CSD3) operated by the University
of Cambridge Research Computing Service (www.csd3.cam.ac.uk), provided by Dell
EMC and Intel using Tier-2 funding from the Engineering and Physical Sciences
Research Council (capital grant EP/T022159/1), and DiRAC funding from the
Science and Technology Facilities Council (www.dirac.ac.uk).

P.M. was partially funded by the European Union’s Horizon 2020 research and
innovation programme under grant agreement no. 875160, the Edinburgh
Laboratory for Integrated Artificial Intelligence (ELIAI,
web.inf.ed.ac.uk/eliai/) EPSRC (grant no. EP/W002876/1), an industry grant
from Cisco, and a donation from Accenture LLP.

A.R., J.A.A., and P.B. are funded by European Union’s Horizon 2020 research
and innovation programme under grant agreement No. 101017453, with A.R. also
being funded by a Royal Society Industry Fellowship.

## References

  * [1] Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary G. Ives.  Dbpedia: A nucleus for a web of open data.  In Karl Aberer, Key-Sun Choi, Natasha Fridman Noy, Dean Allemang, Kyung-Il Lee, Lyndon J. B. Nixon, Jennifer Golbeck, Peter Mika, Diana Maynard, Riichiro Mizoguchi, Guus Schreiber, and Philippe Cudré-Mauroux, editors, The Semantic Web, 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007 + ASWC 2007, Busan, Korea, November 11-15, 2007, volume 4825 of Lecture Notes in Computer Science, pages 722–735. Springer, 2007.
  * [2] Mukesh Bansal, Jichen Yang, Charles Karan, Michael P Menden, James C Costello, Hao Tang, Guanghua Xiao, Yajuan Li, Jeffrey Allen, Rui Zhong, et al.  A community computational challenge to predict the activity of pairs of compounds.  Nature Biotechnology, 32(12):1213–1222, 2014.
  * [3] Stephen Bonner, Ian P. Barrett, Cheng Ye, Rowan Swiers, Ola Engkvist, Charles Tapley Hoyt, and William L. Hamilton.  Understanding the performance of knowledge graph embeddings in drug discovery.  Artificial Intelligence in the Life Sciences, 2:100036, December 2022.
  * [4] Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko.  Translating Embeddings for Modeling Multi-relational Data.  In Advances in Neural Information Processing Systems, volume 26. Curran Associates, Inc., 2013.
  * [5] Samuel Broscheit, Daniel Ruffinelli, Adrian Kochsiek, Patrick Betz, and Rainer Gemulla.  LibKGE - a knowledge graph embedding library for reproducible research.  In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 165–174, Online, October 2020. Association for Computational Linguistics.
  * [6] Vincenzo Carletti, Pasquale Foggia, Antonio Greco, Antonio Roberto, and Mario Vento.  Predicting polypharmacy side effects through a relation-wise graph attention network.  In Andrea Torsello, Luca Rossi, Marcello Pelillo, Battista Biggio, and Antonio Robles-Kelly, editors, Structural, Syntactic, and Statistical Pattern Recognition - Joint IAPR International Workshops, S+SSPR 2020, Padua, Italy, January 21-22, 2021, Proceedings, volume 12644 of Lecture Notes in Computer Science, pages 119–128. Springer, 2020.
  * [7] David Croft, Gavin O’Kelly, Guanming Wu, Robin Haw, Marc Gillespie, Lisa Matthews, Michael Caudy, Phani V. Garapati, Gopal Gopinath, Bijay Jassal, Steven Jupe, Irina Kalatskaya, Shahana Mahajan, Bruce May, Nelson Ndegwa, Esther Schmidt, Veronica Shamovsky, Christina K. Yung, Ewan Birney, Henning Hermjakob, Peter D’Eustachio, and Lincoln Stein.  Reactome: a database of reactions, pathways and biological processes.  Nucleic Acids Res., 39(Database-Issue):691–697, 2011.
  * [8] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel.  Convolutional 2d knowledge graph embeddings.  In Sheila A. McIlraith and Kilian Q. Weinberger, editors, Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 1811–1818. AAAI Press, 2018.
  * [9] Ilaria Ferrari, Giacomo Frisoni, Paolo Italiani, Gianluca Moro, and Claudio Sartori.  Comprehensive Analysis of Knowledge Graph Embedding Techniques Benchmarked on Link Prediction.  Electronics, 11(23):3866, January 2022.  Number: 23 Publisher: Multidisciplinary Digital Publishing Institute.
  * [10] Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard de Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, Axel-Cyrille Ngonga Ngomo, Axel Polleres, Sabbir M. Rashid, Anisa Rula, Lukas Schmelzeisen, Juan F. Sequeda, Steffen Staab, and Antoine Zimmermann.  Knowledge graphs.  ACM Comput. Surv., 54(4):71:1–71:37, 2022.
  * [11] Minoru Kanehisa, Yoko Sato, Masayuki Kawashima, Miho Furumichi, and Mao Tanabe.  KEGG as a reference resource for gene and protein annotation.  Nucleic Acids Res., 44(Database-Issue):457–462, 2016.
  * [12] Timothée Lacroix, Nicolas Usunier, and Guillaume Obozinski.  Canonical tensor decomposition for knowledge base completion.  In Jennifer G. Dy and Andreas Krause, editors, Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, volume 80 of Proceedings of Machine Learning Research, pages 2869–2878. PMLR, 2018\.
  * [13] Christian Meilicke, Melisachew Wudage Chekol, Daniel Ruffinelli, and Heiner Stuckenschmidt.  Anytime bottom-up rule learning for knowledge graph completion.  In Sarit Kraus, editor, Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, pages 3137–3143. ijcai.org, 2019.
  * [14] Yi Nian, Xinyue Hu, Rui Zhang, Jingna Feng, Jingcheng Du, Fang Li, Larry Bu, Yuji Zhang, Yong Chen, and Cui Tao.  Mining on Alzheimer’s diseases related knowledge graph to identity potential AD-related semantic triples for drug repurposing.  BMC Bioinformatics, 23(6):407, September 2022.
  * [15] Simon Ott, Christian Meilicke, and Matthias Samwald.  SAFRAN: an interpretable, rule-based link prediction method outperforming embedding models.  In Danqi Chen, Jonathan Berant, Andrew McCallum, and Sameer Singh, editors, 3rd Conference on Automated Knowledge Base Construction, AKBC 2021, Virtual, October 4-8, 2021, 2021.
  * [16] Roberta Ricciarelli and Ernesto Fedele.  The Amyloid Cascade Hypothesis in Alzheimer’s Disease: It’s Time to Change Our Mind.  Current Neuropharmacology, 15(6):926–935, August 2017.
  * [17] Daniel Ruffinelli, Samuel Broscheit, and Rainer Gemulla.  You CAN teach an old dog new tricks! on training knowledge graph embeddings.  In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
  * [18] Bruce Schultz, Andrea Zaliani, Christian Ebeling, Jeanette Reinshagen, Denisa Bojkova, Vanessa Lage-Rupprecht, Reagon Karki, Sören Lukassen, Yojana Gadiya, Neal G. Ravindra, Sayoni Das, Shounak Baksi, Daniel Domingo-Fernández, Manuel Lentzen, Mark Strivens, Tamara Raschka, Jindrich Cinatl, Lauren Nicole DeLong, Phil Gribbon, Gerd Geisslinger, Sandra Ciesek, David van Dijk, Steve Gardner, Alpha Tom Kodamullil, Holger Fröhlich, Manuel Peitsch, Marc Jacobs, Julia Hoeng, Roland Eils, Carsten Claussen, and Martin Hofmann-Apitius.  A method for the rational selection of drug repurposing candidates from multimodal knowledge harmonization.  Scientific Reports, 11(1):11049, May 2021.  Number: 1 Publisher: Nature Publishing Group.
  * [19] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang.  Rotate: Knowledge graph embedding by relational rotation in complex space.  In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019.
  * [20] Damian Szklarczyk, Andrea Franceschini, Michael Kuhn, Milan Simonovic, Alexander Roth, Pablo Minguez, Tobias Doerks, Manuel Stark, Jean Muller, Peer Bork, Lars J. Jensen, and Christian von Mering.  The STRING database in 2011: functional interaction networks of proteins, globally integrated and scored.  Nucleic Acids Research, 39(Database issue):D561–568, January 2011\.
  * [21] Nicholas P. Tatonetti, Patrick P. Ye, Roxana Daneshjou, and Russ B. Altman.  Data-Driven Prediction of Drug Effects and Interactions.  Science Translational Medicine, 4(125), March 2012.
  * [22] Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard.  Complex embeddings for simple link prediction.  In Maria-Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pages 2071–2080. JMLR.org, 2016.
  * [23] Brian Walsh, Sameh K. Mohamed, and Vít Nováček.  BioKG: A Knowledge Graph for Relational Learning On Biological Data.  In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, CIKM ’20, pages 3173–3180, New York, NY, USA, October 2020. Association for Computing Machinery.
  * [24] Quan Wang, Zhendong Mao, Bin Wang, and Li Guo.  Knowledge Graph Embedding: A Survey of Approaches and Applications.  IEEE Transactions on Knowledge and Data Engineering, 29(12):2724–2743, December 2017.
  * [25] Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen.  Knowledge Graph Embedding by Translating on Hyperplanes.  Proceedings of the AAAI Conference on Artificial Intelligence, 28(1), June 2014.
  * [26] David S. Wishart, Craig Knox, Anchi Guo, Dean Cheng, Savita Shrivastava, Dan Tzur, Bijaya Gautam, and Murtaza Hassanali.  Drugbank: a knowledgebase for drugs, drug actions and drug targets.  Nucleic Acids Res., 36(Database-Issue):901–906, 2008.
  * [27] Yoshihiro Yamanishi, Michihiro Araki, Alex Gutteridge, Wataru Honda, and Minoru Kanehisa.  Prediction of drug-target interaction networks from the integration of chemical and genomic spaces.  In Proceedings 16th International Conference on Intelligent Systems for Molecular Biology (ISMB), Toronto, Canada, July 19-23, 2008, pages 232–240, 2008.
  * [28] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng.  Embedding entities and relations for learning and inference in knowledge bases.  In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
  * [29] Wen Zhang, Bibek Paudel, Liang Wang, Jiaoyan Chen, Hai Zhu, Wei Zhang, Abraham Bernstein, and Huajun Chen.  Iteratively learning embeddings and rules for knowledge graph reasoning.  In Ling Liu, Ryen W. White, Amin Mantrach, Fabrizio Silvestri, Julian J. McAuley, Ricardo Baeza-Yates, and Leila Zia, editors, The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019, pages 2366–2377. ACM, 2019.
  * [30] Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun.  Graph neural networks: A review of methods and applications.  AI Open, 1:57–81, January 2020.
  * [31] Marinka Zitnik, Monica Agrawal, and Jure Leskovec.  Modeling polypharmacy side effects with graph convolutional networks.  Bioinformatics, 34(13):i457–i466, July 2018.

##  9 Evaluation Metrics

We give a formal definition of the evaluation metrics used in this work. If
$\mathcal{E}$ is the set of all entities in the KG and
$\mathcal{K}^{\text{test}}$ is the test set of triples, denote for a given
triple $(s,p,o)$:

  * •

The set of filtered pseudo-negative triples:

$\displaystyle\\{(s,p,o^{\prime}):t^{\prime}\in\mathcal{E}\text{ and
}(s,p,o^{\prime})\text{ does not appear in the training, validation or test
triples}\\}$

  * •

score$(s,p,o)$ the model’s score for $(s,p,o)$.

  * •

rank$(o\mid s,p)$ (and symmetrically rank$(s\mid p,o)$) the filtered rank of
entity $t$, i.e. the rank of score$(s,p,o)$ in the scores of all triples in
the set of filtered pseudo-negative triples. For ties in scores, we take the
mean rank of all triples with score$(s,p,o)$.

Then the evaluation metrics are defined as:

MRR
$\displaystyle=\frac{1}{2|\mathcal{K}^{\text{test}}|}\sum\limits_{(s,p,o)\in\mathcal{K}^{\text{test}}}\left(\frac{1}{\text{rank}(s\mid
p,o)}+\frac{1}{\text{rank}(o\mid s,p)}\right),$ HITS@k
$\displaystyle=\frac{1}{2|\mathcal{K}^{\text{test}}|}\sum\limits_{(s,p,o)\in\mathcal{K}^{\text{test}}}\left(\mathds{1}(\text{rank}(s\mid
p,o)\leq k)+\mathds{1}(\text{rank}(o\mid s,p)\leq k)\right),$

with $\mathds{1}(C)$ an indicator function that is 1 if the condition $C$ is
true and $0$ otherwise.

##  10 BioKG Node Degree Statistics

Table 5: Complete node degree statistics for BioKG | Mean degree | Median Degree | Degree std | Max Degree | Min degree
---|---|---|---|---|---
Total | 39.19 | 5 | 171.07 | 2872 | 1
DDI | 624.86 | 581.5 | 525.85 | 2477 | 1
Protein Pathway Association | 8.46 | 3 | 20.80 | 1471 | 1
PPI | 10.28 | 4 | 19.61 | 478 | 1
Protein Disease Association | 7.72 | 2 | 38.11 | 2779 | 1
Member of Complex | 8.21 | 3 | 17.95 | 421 | 1
Drug Disease Association | 20.94 | 5 | 44.49 | 724 | 1
DPI | 5.35 | 2 | 17.80 | 968 | 1
Complex in Pathway | 2.88 | 1 | 6.43 | 158 | 1
Complex Top Level Pathway | 2.40 | 1 | 41.58 | 2787 | 1
Drug Target | 3.23 | 1 | 8.01 | 297 | 1
Drug Pathway Association | 4.54 | 2 | 6.56 | 54 | 1
Drug Enzyme | 4.85 | 2 | 27.67 | 958 | 1
Disease Genetic Disorder | 3.02 | 1 | 6.26 | 46 | 1
Related Genetic Disorder | 1.03 | 1 | 0.32 | 14 | 1
Disease Pathway Association | 4.74 | 3 | 5.14 | 52 | 1
Drug Transporter | 4.83 | 2 | 18.39 | 495 | 1
Drug Carrier | 2.49 | 1 | 16.08 | 389 | 1
|  |  |  |  |

##  11 Hyperparameters

Table 6: All LP hyperparameters and corresponding possible values. Hyperparameter | Values
---|---
Embedding size | [128, 256, 512, 1024]
Training type | [NegSamp, 1vsAll]
NegSamp neg. subjects | [1, 100]
NegSamp neg. objects | [1, 100]
Max epochs | 200
Reciprocal | [True, False]
Loss | CE
Optimiser | [Adam, Adagrad]
Batch size | [128, 256, 512, 1024]
Learning rate | [$0.0003$, 1.0]
Scheduler patience | [0, 10]
Lp regularisation | [None, L1, F2, N3]
Entity emb. weight | $[1.0e-20,1.0e-1]$
Relation emb. weight | $[1.0e-20,1.0e-1]$
Frequency weighting | [True, False]
Dropout |
Entity embedding | [-0.5, 0.5]
Relation embedding | [-0.5, 0.5]
Embedding initialisation | [Uniform, Normal, XavierUniform, XavierNormal]
Normal mean | 0.0
Normal std. | [0.00001, 1.0]
Uniform lower bound | [-1.0, -0.00001]
XavierUniform gain | 1.0
XavierNormal gain | 1.0
|

##  12 Best-found configurations

Table 7: Hyperparameters and MRR of the LP models on BioKG as a whole. Best from 30 quasi-random HPO trials. | BioKG
---|---
| ComplEx | DistMult | TransE | TransH | RotatE | ConvE
MRR (valid) | 0.630 | 0.165 | 0.274 | 0.281 | 0.421 | 0.599
MRR (test) | 0.629 | 0.471 | 0.273 | 0.281 | 0.422 | 0.599
Embedding size | 512 | 128 | 256 | 256 | 1024 | 1024
Training type | 1vsAll | 1vsAll | 1vsAll | NegSamp | NegSamp | 1vsAll
NegSamp neg. objects | – | – | – | 51 | 1 | –
NegSamp neg. subjects | – | – | – | 1 | 3 | –
Epochs | 184 | 14 | 124 | 200 | 200 | 94
Reciprocal | Yes | No | Yes | Yes | Yes | Yes
Loss | CE | CE | CE | CE | CE | CE
Optimiser | Adag. | Adag. | Adag. | Adag. | Adag. | Adag.
Batch size | 128 | 128 | 256 | 256 | 128 | 512
Learning rate | 0.417 | 0.417 | $9.8510-3$ | $8.6110-3$ | $3.2110-3$ | $2.0210-3$
Scheduler patience | 10 | 10 | 10 | 10 | 10 | 10
Lp regularisation | F2 | F2 | None | None | None | None
Entity emb. weight | $6.6310-7$ | $6.6310-7$ | – | – | – | –
Relation emb. weight | $2.5710-15$ | $2.5710-15$ | – | – | – | –
Frequency weighting | No | No | – | – | – | –
Dropout |  |  |  |  |  |
Entity embedding | 0.407 | 0.407 | 0.117 | 0.0208 | 0.000 | 0.000
Relation embedding | 0.0370 | 0.0370 | 0.000 | 0.000 | 0.0392 | 0.394
Embedding initialisation |  XN($\sigma=0.0866$) |  XN($\sigma=0.0866$) |  N($\sigma=$$4.1210-4$) |  N($\sigma=$$1.3310-3$) |  XU($\pm 0.199$) |  N($\sigma=$$1.2610-3$)
|  |  |  |  |  |

Table 8: Hyperparameters and MRR of ComplEx on the BioKG polypharmacy KGs. Best from 30 quasi-random HPO trials. | DDI-EFFICACY | DDI-MINERAL | DPI-FDA | DEP-FDA-EXP
---|---|---|---|---
| ComplEx | ComplEx-P | ComplEx | ComplEx-P | ComplEx | ComplEx-P | ComplEx | ComplEx-P
MRR (valid) | 0.838 | 0.859 | 0.861 | 0.885 | 0.383 | 0.540 | 0.171 | 0.184
MRR (test) | 0.847 | 0.865 | 0.861 | 0.884 | 0.386 | 0.542 | 0.171 | 0.185
Embedding size | 512 | 512 | 512 | 512 | 512 | 512 | 512 | 512
Training type | 1vsAll | 1vsAll | 1vsAll | 1vsAll | 1vsAll | 1vsAll | 1vsAll | 1vsAll
Epochs | 196 | 52 | 106 | 54 | 76 | 4 | 72 | 22
Reciprocal | No | No | No | No | Yes | Yes | Yes | Yes
Loss | CE | CE | CE | CE | CE | CE | CE | CE
Optimiser | Adag. | Adag. | Adag. | Adag. | Adam | Adag. | Adag. | Adam
Batch size | 128 | 256 | 128 | 256 | 128 | 1024 | 1024 | 1024
Learning rate | 0.0918 | 0.0229 | 0.0918 | 0.0170 | 0.0114 | $1.5010-3$ | 0.156 | $5.8110-3$
Scheduler patience | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 10
Lp regularisation | N3 | N3 | N3 | L1 | None | None | None | None
Entity emb. weight | $3.2910-18$ | $9.3610-9$ | $3.2910-18$ | $3.1510-13$ | – | – | – | –
Relation emb. weight | $1.2710-5$ | $3.7310-4$ | $1.2710-5$ | $2.0310-6$ | – | – | – | –
Frequency weighting | No | Yes | No | Yes | Yes | Yes | Yes | Yes
Dropout |  |  |  |  |  |  |  |
Entity embedding | 0.451 | 0.189 | 0.451 | 0.000 | 0.000 | 0.345 | 0.000 | 0.000
Relation embedding | 0.000 | 0.135 | 0.000 | 0.0863 | 0.159 | 0.462 | 0.259 | 0.0563
Embedding initialisation |  N($\sigma=$$1.6310-3$) |  U($\pm 0.700$) |  N($\sigma=$$1.6310-3$) |  U($\pm$$4.0910-4$) |  N($\sigma=$$1.7810-5$) |  N($\sigma=$$3.0210-3$) |  XN($\sigma=0.832$) |  U($\pm 0.329$)
|  |  |  |  |  |  |  |