# ADRNet: A Generalized Collaborative Filtering Framework Combining Clinical
and Non-Clinical Data for Adverse Drug Reaction Prediction

Haoxuan Li<EMAIL_ADDRESS>Peking UniversityBeijingChina ,  Taojun Hu
Peking UniversityBeijingChina<EMAIL_ADDRESS>,  Zetong Xiong  Yale
UniversityNew HavenUSA ,  Chunyuan Zheng  University of California, San
DiegoSan DiegoUSA ,  Fuli Feng  University of Science and Technology of China
HefeiChina ,  Xiangnan He  University of Science and Technology of China 8600
Datapoint DriveHefeiChina and  Xiao-Hua Zhou  Peking UniversityBeijingChina
<EMAIL_ADDRESS>
(2023)

###### Abstract.

Adverse drug reaction (ADR) prediction plays a crucial role in both health
care and drug discovery for reducing patient mortality and enhancing drug
safety. Recently, many studies have been devoted to effectively predict the
drug-ADRs incidence rates. However, these methods either did not effectively
utilize non-clinical data, i.e., physical, chemical, and biological
information about the drug, or did little to establish a link between content-
based and pure collaborative filtering during the training phase. In this
paper, we first formulate the prediction of multi-label ADRs as a drug-ADR
collaborative filtering problem, and to the best of our knowledge, this is the
first work to provide extensive benchmark results of previous collaborative
filtering methods on two large publicly available clinical datasets. Then, by
exploiting the easy accessible drug characteristics from non-clinical data, we
propose ADRNet, a generalized collaborative filtering framework combining
clinical and non-clinical data for drug-ADR prediction. Specifically, ADRNet
has a shallow collaborative filtering module and a deep drug representation
module, which can exploit the high-dimensional drug descriptors to further
guide the learning of low-dimensional ADR latent embeddings, which
incorporates both the benefits of collaborative filtering and representation
learning. Extensive experiments are conducted on two publicly available real-
world drug-ADR clinical datasets and two non-clinical datasets to demonstrate
the accuracy and efficiency of the proposed ADRNet. The code is available at
https://github.com/haoxuanli-pku/ADRnet.

adverse drug reaction; sided effect; drug-ADR prediction

††journalyear: 2023††copyright: acmlicensed††conference: Seventeenth ACM
Conference on Recommender Systems; September 18–22, 2023; Singapore,
Singapore††booktitle: Seventeenth ACM Conference on Recommender Systems
(RecSys ’23), September 18–22, 2023, Singapore, Singapore††price: 15.00††doi:
10.1145/3604915.3608813††isbn: 979-8-4007-0241-9/23/09††ccs: Applied computing
Bioinformatics††ccs: Information systems Collaborative filtering

##  1\. Introduction

Adverse drug reactions (ADRs, also known as side effects) refer to harmful
effects produced by drugs that are detrimental to the patient’s treatment in
normal therapy (Edwards and Aronson, 2000; Karimi et al., 2015). Until now,
thousands types of ADRs have been reported, many of which have led to serious
unwanted and harmful consequences (Wu et al., 2010). Timely and effective
warning of ADRs can help regulate as well as guide the production of drugs
with limited side effects. Therefore, the accurate prediction of ADRs is
meaningful in both health care (Houck and Kavlock, 2008) and drug discovery
(Arrowsmith and Miller, 2013) for improving drug safety and reducing patient
mortality.

Many previous approaches are based on association rule mining and statistical
significance tests (Agresti, 1992) to identify the significance of drug-ADR
associations (Chen and Ung, 2001; Vedani et al., 2005; Fliri et al., 2005;
Tatonetti et al., 2009; Hammann et al., 2010; Montastruc et al., 2011), but
with limited performance on prediction tasks. An alternative class of deep
learning-based methods uses pharmacovigilance networks (Cami et al., 2011;
Davazdahemami and Delen, 2018), ensemble approaches (Jahid and Ruan, 2013),
and deep neural models (Wang et al., 2019; Sedhain et al., 2015), and has
achieved promising predictive performance for specific adverse reactions, as
shown in Figure 1(a). However, as drugs may cause multiple ADRs
simultaneously, the direct repeat use of the single-label prediction methods
may lead to time-consuming and suboptimal prediction performance.

To tackle this issue, collaborative filtering methods can efficiently and
accurately predict the relationship between drugs and multiple ADRs. The
underlying motivation is that drugs with similar interactions in clinical data
tend to have similar ADRs. For example, matrix factorization (MF) (Koren et
al., 2009; Pauwels et al., 2011) and neural matrix factorization (NMF) (He et
al., 2017) first learn a latent embedding for each drug and ADR, and then
perform drug-ADR prediction. However, as shown in Figure 1(b), the pure
collaborative filtering methods ignore the non-clinical data (e.g., drug
descriptors (Kim et al., 2016)), which are readily available and can
significantly improve the drug-ADR prediction performance. Recently, content-
based collaborative filtering methods have achieved impressive performance in
recommendation, as shown in Figure 1(c), by incorporating user-side and item-
side features, the prediction performance can be further improved. _However,
unlike traditional recommender systems where users and items have symmetric
information and status, data collected for drugs and ADRs usually contain non-
clinical drug characteristics but lack of ADR characteristics (e.g., cough and
fever), due to the inconvenience of the ADR characteristics to be obtained and
processed, which poses a great challenge for developing effective drug-ADR
predictions using both clinical and non-clinical data._

Figure 1. Comparisons of (a) previous multi-label prediction methods; (b)
previous pure collaborative filtering methods; (c) previous content-based
collaborative filtering methods; (d) proposed ADRNet as a generalized
collaborative filtering framework combining clinical and non-clinical data for
drug-ADR prediction, contains (i) a deep drug representation module; (ii) a
shallow latent collaborative filtering module; and (iii) a drug collaborative
filtering module.

In this paper, we first formulate the prediction of multi-label ADRs as a
drug-ADR collaborative filtering problem, and to the best of our knowledge,
this is the first study to provide extensive benchmark results of previous
collaborative filtering methods on two large publicly available clinical
datasets. Next, by noting the easy accessibility of drug characteristics and
the asymmetric information of drugs and ADRs that differs from previous
collaborative filtering, we propose ADRNet, a generalized collaborative
filtering framework combining clinical and non-clinical data for drug-ADR
prediction. Specifically,ADRNet has a shallow collaborative filtering module
and a deep drug representation module, which can exploit the high-dimensional
drug descriptors to further guide the learning of low-dimensional ADR latent
embeddings, which incorporates both the benefits of collaborative filtering
and representation learning.

Notably, the proposed ADRNet effectively exploits both the advantages of deep
content-based and latent collaborative filtering-based approaches. On one
hand, compared with the deep drug feature-based networks, our approach has
fewer parameters and is convenient. Previous studies mapped drugs to ADR space
(Dimitri and Lió, 2017; Xiao et al., 2017b; Lorberbaum et al., 2016) or to
metabolic reaction space (Shaked et al., 2016) and applied flux variability
analysis to represent drug-protein/gene interactions (Mahadevan and Schilling,
2003), whereas our approach further utilize the learned ADR latent embedding
from collaborative filtering to guide the deep representations learning of
drugs. On the other hand, compared with the latent collaborative filtering-
based approaches, the proposed approach has the clear advantage of exploiting
the chemical, physical and biological characteristics of drugs contained in
non-clinical data (Belleau et al., 2008) to help reveal the mechanisms of
drug-ADR and improve the prediction performance.

The contributions of this paper are summarized as follows.

  * $\bullet$

We formulate the prediction of multi-label ADRs as a drug-ADR collaborative
filtering problem, and to the best of our knowledge, this is the first study
to provide extensive benchmark results of previous collaborative filtering
methods on two large publicly available clinical datasets.

  * $\bullet$

By exploiting the easy accessible drug characteristics from non-clinical data,
we propose ADRNet, a generalized collaborative filtering framework combining
clinical and non-clinical data for drug-ADR prediction.

  * $\bullet$

We further propose a sharing mechanism for ADR latent vectors that is easily
applicable to previous pure collaborative filtering methods, and a learning
approach to jointly train a deep drug representation and a shallow
collaborative filtering network to better trade-off the predictions of each
sub-network.

  * $\bullet$

Extensive experiments are conducted on two publicly available real-world drug-
ADR clinical datasets and two non-clinical datasets to demonstrate the
accuracy and efficiency of the proposed ADRNet.

##  2\. Preliminaries and Problem Setup

Let $M$ and $N$ be the number of drugs and ADRs, respectively. Based on the
clinical data containing drug-ADR records from electronic health or adverse
reporting systems (Liu et al., 2012; Banda et al., 2016), we define the drug-
ADR interaction matrix $\mathbf{Y}\in\mathbb{R}^{M\times N}$ as

$y_{i,j}=\begin{cases}1,&\text{ if interaction ( drug }i,\text{ ADR }j\text{ )
is observed; }\\\ 0,&\text{ otherwise. }\end{cases}$

The multi-label ADR prediction problem is formalized as a collaborative
filtering problem to accurately predict drug-ADR incidence rate. Specifically,
the model-based approach is abstracted as learning
$\hat{y}_{i,j}=f(i,j\mid\Theta)$, where $\hat{y}_{i,j}$ is the predicted
incidence rate of drug $i$ to ADR $j$, and $f$ is a parametric prediction
model, $\Theta$ is the model parameters.

To accurately estimate parameters $\Theta$ in pure collaborative filtering, we
associate each drug $i$ and ADR $j$ with a real-valued vector of hidden
features, denoted as $\mathbf{p}_{i}$ and $\mathbf{q}_{j}$, respectively. Then
many standard collaborative filtering methods can be adopted, e.g., matrix
factorization (MF) (Koren et al., 2009; Pauwels et al., 2011) using the inner
product of $\mathbf{p}_{i}$ and $\mathbf{q}_{j}$ for drug-ADR interaction
prediction as
$\hat{y}_{i,j}=f\left(i,j\mid\mathbf{p}_{i},\mathbf{q}_{j}\right)=\mathbf{p}_{i}^{\top}\mathbf{q}_{j}=\sum_{k=1}^{K}p_{ik}q_{jk},$
where $K$ denotes the dimension of the latent space. More generally, the
backbone can be substituted by neural networks
$\hat{y}_{i,j}=f\left(\mathbf{P},\mathbf{Q}\mid\Theta\right),$ where
$\mathbf{P}\in\mathbb{R}^{M\times K}$ and $\mathbf{Q}\in\mathbb{R}^{N\times
K}$, denoting the latent factor matrix for drugs and ADRs, respectively.
However, these methods do not exploit drug features in easily available non-
clinical data, leading to suboptimal performance in practice.

Content-based collaborative filtering further utilizes non-clinical data that
contain chemical, physical and biological information about the drugs, which
can be beneficial for the prediction of drug-ADR relations (Testa and Kier,
1991; Consonni, 2009; Grisoni et al., 2018; Yamanishi et al., 2012).
Specifically, drug descriptors in the non-clinical data can be categorized
into two types: physical, chemical descriptors (PC-descriptors), and
biological descriptors (BIO-descriptors) (Nguyen et al., 2021). To enhance the
prediction accuracy of drug-ADR, we use PubChem (Kim et al., 2016) as the PC-
descriptors and Bio2RDF (Belleau et al., 2008) as the BIO-descriptors for
drugs, and denote them with $\mathbf{x}^{PC}_{i}$ and $\mathbf{x}^{BIO}_{i}$,
respectively.

Unfortunately, unlike drug descriptors, ADRs in non-clinical data always
appear in a descriptive manner (e.g., cough and fever), and these features are
difficult to be processed and utilized directly to improve the prediction
performance. Such drug-ADR information asymmetry poses a serious challenge to
the previous collaborative filtering approaches. In order to simultaneously
take the advantage of the latent collaborative filtering and exploit the
informative drug descriptors, we propose a general asymmetric collaborative
filtering framework for accurate drug-ADR prediction next.

##  3\. Proposed ADRNet: A Generalized Collaborative Filtering Framework

In this section, we propose ADRNet, a generalized collaborative filtering
framework combining clinical and non-clinical data for drug-ADR prediction. By
learning the latent embeddings from clinical data and leveraging high-
dimensional drug descriptors from non-clinical data, the proposed method
combines the benefits of collaborative filtering and deep representation
learning. We introduce the structure of ADRNet as shown in Figure 1(d) in the
following subsections, including (i) a deep drug representation module in
Section 3.1; (ii) a shallow latent collaborative filtering module in Section
3.2; and (iii) a drug collaborative filtering module in Section 3.3. We
discuss the training of ADRNet and summarize its advantages in Section 3.4.

###  3.1. Deep Drug Representation Module

The deep drug representation module first concatenates and maps sparse and
high-dimensional drug PC-descriptors and BIO-descriptors to a low-dimensional
and dense real-valued vector, also known as the drug embedding vector. These
embedding vectors are randomly initialized, then fed into a feed-forward
neural network, and obtain a drug representation _with the same dimensions_ as
the drug/ADR latent space in latent collaborative filtering. Given a learned
drug representation, as in Figure 1(d), the representation is fed into a fully
connected (FC) layer to obtain drug-level predictions
$\hat{y}_{i,j}^{D}=\hat{y}_{i}^{D}$. In addition, this representation is
shared to make predictions together with the ADR latent vector obtained from
the collaborative filtering part. Specifically, the deep drug representation
layers under our ADRNet framework are defined as

$\displaystyle\mathbf{z}_{1}^{D}=\phi_{1}(\mathbf{x}^{PC}_{i},\mathbf{x}^{BIO}_{i})=\left[\begin{array}[]{l}\mathbf{x}^{PC}_{i}\\\
\mathbf{x}^{BIO}_{i}\end{array}\right],\quad\mathbf{z}_{2}^{D}=\phi_{2}^{D}(\mathbf{z}_{1}^{D})=a_{2}\left((\mathbf{W}_{2}^{D})^{\top}\mathbf{z}_{1}^{D}+\mathbf{b}_{2}^{D}\right),\quad\ldots,$
$\displaystyle\mathbf{z}_{L}^{D}=\phi_{L}^{D}(\mathbf{z}_{L-1}^{D})=a_{L}\left((\mathbf{W}_{L}^{D})^{\top}\mathbf{z}_{L-1}^{D}+\mathbf{b}_{L}^{D}\right),\quad\hat{y}_{i,j}^{D}=\hat{y}_{i}^{D}=\sigma\left((\mathbf{h}^{D})^{\top}\mathbf{z}_{L}^{D}\right),$

where $\mathbf{W}_{l},\mathbf{b}_{l}$, and $a_{l}(\cdot)$ denote the weight
matrix, bias vector, and activation function of the $l$-th layer’s perceptron,
and $\sigma(\cdot)$ denote the sigmoid function.

###  3.2. Shallow Latent Collaborative Filtering Module

The shallow latent collaborative filtering module concatenates the drug and
ADR latent vectors and performs pure collaborative filtering using a feed-
forward neural network, which is able to capture the non-linearity of
interactions between $\mathbf{p}_{i}$ and $\mathbf{q}_{j}$. Compared with the
drug representation module, the collaborative filtering module does not
directly use the drug features, and therefore has shallower layers. However,
in contrast to neural collaborative filtering (NCF) (He et al., 2017), the
learned drug representations can further guide the training of ADR latent
vectors in Figure 1(d), which provides a stronger supervised signal for latent
vectors compared with collaborative filtering using only clinical
interactions, and thus leads to more accurate predictions. Another advantage
is induced from the joint learning of the deep network in Section 3.1 and the
shallow collaborative filtering. As suggested by the previous studies, jointly
learning a shallow and a deep network can further improve the model’s ”memory”
and ”generalization” abilities (Cheng et al., 2016; Wang et al., 2017).
Specifically, the ”memory” ability means that the model directly learns and
utilizes the co-occurrence frequency of drugs and ADRs from historical data,
and the ”generalization” ability refers to letting the model mine the drug-ADR
associations, expecting the model to be able to accurately predict sparse or
even unobserved drug or ADR features. Formally, the shallow latent
collaborative filtering module is formulated as

$\displaystyle\mathbf{z}_{1}^{CF}=\phi_{1}(\mathbf{p}_{i},\mathbf{q}_{j})=\left[\begin{array}[]{l}\mathbf{p}_{i}\\\
\mathbf{q}_{j}\end{array}\right],\quad\mathbf{z}_{2}^{CF}=\phi_{2}^{CF}(\mathbf{z}_{1}^{CF})=a_{2}\left((\mathbf{W}_{2}^{CF})^{\top}\mathbf{z}_{1}^{CF}+\mathbf{b}_{2}^{CF}\right),\quad\ldots,$
$\displaystyle\mathbf{z}_{S}^{CF}=\phi_{S}^{CF}(\mathbf{z}_{S-1}^{CF})=a_{S}\left((\mathbf{W}_{S}^{CF})^{\top}\mathbf{z}_{S-1}^{CF}+\mathbf{b}_{S}^{CF}\right),\quad\hat{y}_{i,j}^{CF}=\sigma\left((\mathbf{h}^{CF})^{\top}\mathbf{z}_{S}^{CF}\right),$

where $\mathbf{W}_{s},\mathbf{b}_{s}$, and $a_{s}$ denote the weight matrix,
bias vector, and activation function for the $s$-th layer’s perceptron.

###  3.3. Drug Collaborative Filtering Module

The drug collaborative filtering module performs element-wise product of
learned drug representations from the deep network and ADR latent vectors from
the shallow network, and then feeds the element-wise product into an FC layer
to obtain $\hat{y}_{i,j}^{DCF}$ as in Figure 1(d) that

$\hat{y}_{i,j}^{DCF}=\sigma\left((\mathbf{h}^{DCF})^{\top}\left(\phi_{L}^{D}(\mathbf{z}_{L-1}^{D})\odot\mathbf{q}_{j}\right)\right),$

which has the same form as generalized matrix factorization (GMF) (He et al.,
2017), but should be considered as an _asymmetric_ collaborative filtering.
The key observation is that the learned drug representations are derived from
PC-descriptors and BIO-descriptors, which inherently contain extensive
structural, chemical, physical and biological features about the drug. Also,
the drug representations are fed to an FC layer for drug-ADR prediction in
Section 3.1, so that these representations can retain a strong association to
ADR. Notably, the ADR latent vectors are _shared_ between deep representations
and shallow collaborative filtering, which is further interpreted as drug
representations can guide the training of ADR latent vectors. We also
empirically validate this in our experiments.

###  3.4. Joint Training of ADRNet

We jointly train the deep drug representation network and the shallow latent
collaborative filtering network in ADRNet. Specifically, the drug
representation $\phi^{D}$, its element-wise product with the ADR latent vector
$\phi^{DCF}$, and the collaborative filtering vector $\phi^{CF}$ are
concatenated and fed into an FC layer for drug-ADR prediction that

$\displaystyle\phi^{D}_{i,j}$
$\displaystyle=\phi_{L}^{D}(\dots(\phi_{2}^{D}(\phi_{1}(\mathbf{x}^{PC}_{i},\mathbf{x}^{BIO}_{i}))\dots),\quad\phi^{CF}_{i,j}=\phi_{S}^{CF}(\dots(\phi_{2}^{CF}(\phi_{1}(\mathbf{p}_{i},\mathbf{q}_{j}))\dots),\quad\phi^{DCF}_{i,j}=\phi^{D}\odot\mathbf{q}_{j},$
$\displaystyle\hat{y}_{i,j}$
$\displaystyle=\sigma\left(\mathbf{h}^{\top}\left[\begin{array}[]{l}\phi^{D}_{i,j}\\\
\phi^{CF}_{i,j}\\\
\phi^{DCF}_{i,j}\end{array}\right]\right)=\sigma\left(\sigma^{-1}(\hat{y}_{i,j}^{D})+\sigma^{-1}(\hat{y}_{i,j}^{CF})+\sigma^{-1}(\hat{y}_{i,j}^{DCF})\right),$

where
$\mathbf{h}=[(\mathbf{h}^{D})^{\top},(\mathbf{h}^{CF})^{\top},(\mathbf{h}^{DCF})^{\top}]^{\top}$
denote the edge weights of the output layer. We use mini-batch stochastic
optimization to back-propagate the gradients in both the deep and shallow
networks, and train the prediction model by minimizing the training loss

$L(\theta)=-\sum_{i\in\mathcal{I}}\sum_{j\in\mathcal{J}}y_{i,j}\log\hat{y}_{i,j}+\left(1-y_{i,j}\right)\log\left(1-\hat{y}_{i,j}\right).$

We summarize the advantages of ADRNet over previous works as follows: (a)
provides a convenient and flexible framework to combine drug descriptor-based
models with latent-based collaborative filtering; (b) has better ”memory” and
”generalization” ability due to joint learning of shallow and deep models; (c)
learns more informative drug and ADR latent vectors compared with pure
collaborative filtering.

##  4\. Experiments

In this section, we aim to answer the following research questions on real-
world clinical and non-clinical datasets:

  * $\bullet$

How does the proposed method perform compared with the existing models in
terms of drug-ADR prediction? Does non-clinical data help enhance the previous
collaborative filtering methods?

  * $\bullet$

What factors (shallow collaborative filtering, deep drug representation,
latent embedding, drug descriptor, sharing mechanism) influence the validity
of our method?

  * $\bullet$

Does our method stably outperform the previous models at varying latent
embedding sizes?

  * $\bullet$

Are deeper layers in collaborative filtering and drug representation helpful
for drug–ADR prediction?

###  4.1. Experimental Setup

####  4.1.1. Dataset and Preprocessing

We follow the previous studies (Zhang et al., 2015; Nguyen et al., 2021) to
use two widely used real-world clinical datasets Liu’s (Liu et al., 2012) and
AEOLUS (Banda et al., 2016) that contain drug-ADR interactions. We then select
the drugs that appear in DrugBank, as well as require ADRs to occur on more
than 50 drugs. Specifically, the Liu’s dataset contains 58,810 drug-ADR
interactions occurring on 828 drugs and 1,385 ADRs, and the AEOLUS dataset
contains 605,121 interactions occurring on 1,358 drugs and 2,707 ADRs. In
addition, to verify the effectiveness of the proposed generalized
collaborative filtering framework, we also adopt two real-world non-clinical
data PubChem (Kim et al., 2016) and Bio2RDF (Belleau et al., 2008).
Specifically, for each drug, PubChem provides pre-defined 881 bits of
structural PC-descriptors, and Bio2RDF provides 6,712 bits of chemical,
physical, and biological BIO-descriptors. We concatenate the PC-descriptors
and BIO-descriptors into 7,593 bits and feed them into the deep drug
representation layers.

####  4.1.2. Baselines

To comprehensively evaluate the proposed method, we compare with the following
multi-label prediction methods: LNSM (Zhang et al., 2015; Muñoz et al., 2016),
SVM (Jahid and Ruan, 2013), RF (Davazdahemami and Delen, 2018; Liu et al.,
2012), LR (Cami et al., 2011; Liu et al., 2012), CCA (Cami et al., 2011;
Yamanishi et al., 2012). We also compare with the following collaborative
filtering methods: DrugCF111DrugCF is implemented through drug similarity-
based collaborative filtering., MF (Koren et al., 2009; Pauwels et al., 2011),
NCF (He et al., 2017), NMF (He et al., 2017), PNN (Qu et al., 2016), FNN
(Zhang et al., 2016), Deep Crossing (Shan et al., 2016), Wide&Deep (Cheng et
al., 2016), Deep&Cross (Wang et al., 2017), DeepFM (Guo et al., 2017), NFM (He
et al., 2017), AFM (Xiao et al., 2017a), and UltraGCN (Mao et al., 2021).

####  4.1.3. Experimental Protocols and Details

Table 1. Performance comparison of drug–ADR prediction models on Liu’s. We
bold the best model, and underline the best single-label prediction model and
collaborative filtering model.

Dataset | Liu’s | AEOLUS
---|---|---
Method |  AUC $(\times 10^{-2})$ |  AUPR $(\times 10^{-2})$ | Time (s) |  AUC $(\times 10^{-2})$ |  AUPR $(\times 10^{-2})$ | Time (s)
LNSM |  90.31 $\pm$ 0.25 |  46.64 $\pm$ 0.69 | 32.6 |  84.98 $\pm$ 0.64 |  57.40 $\pm$ 1.63 | 85.7
SVM | 90.98 $\pm$ 0.22 | 48.79 $\pm$ 0.71 |  8863.3 | 89.25 $\pm$ 0.46 | 67.54 $\pm$ 1.60 | 20378.0
RF |  87.77 $\pm$ 0.32 |  44.00 $\pm$ 0.59 | 7.7 |  88.56 $\pm$ 0.40 |  66.56 $\pm$ 1.49 | 22.5
LR |  89.25 $\pm$ 0.23 |  46.54 $\pm$ 0.83 | 9.5 |  89.02 $\pm$ 1.02 |  66.89 $\pm$ 2.29 | 13.8
CCA |  63.48 $\pm$ 0.83 |  19.50 $\pm$ 0.94 | 72.0 |  58.59 $\pm$ 1.51 |  26.69 $\pm$ 1.80 | 141.4
DrugCF |  87.88 $\pm$ 0.35 |  45.53 $\pm$ 0.75 | 2.2 |  84.79 $\pm$ 0.58 |  58.40 $\pm$ 1.06 | 5.9
MF |  90.89 $\pm$ 0.26 |  45.00 $\pm$ 0.65 | 10.9 |  87.15 $\pm$ 0.37 |  61.03 $\pm$ 1.11 | 15.7
NCF |  89.84 $\pm$ 0.36 |  44.00 $\pm$ 0.67 | 17.2 |  84.68 $\pm$ 0.41 |  59.35 $\pm$ 1.13 | 50.0
NMF |  91.21 $\pm$ 0.37 |  46.73 $\pm$ 0.67 | 19.3 |  87.16 $\pm$ 0.32 |  61.03 $\pm$ 1.11 | 43.5
PNN |  91.10 $\pm$ 0.29 |  46.48 $\pm$ 1.06 | 10.1 |  90.22 $\pm$ 0.35 |  69.85 $\pm$ 0.96 | 12.7
FNN |  91.05 $\pm$ 0.43 |  45.17 $\pm$ 0.86 | 17.6 |  89.64 $\pm$ 0.42 |  66.72 $\pm$ 0.82 | 19.4
Deep Crossing |  90.79 $\pm$ 0.38 |  48.13 $\pm$ 1.09 | 17.2 |  88.44 $\pm$ 0.47 |  69.43 $\pm$ 1.23 | 13.5
Wide&Deep | 91.42 $\pm$ 0.25 | 50.23 $\pm$ 0.85 | 4.6 | 90.50 $\pm$ 0.45 | 72.38 $\pm$ 1.11 | 9.2
Deep&Cross |  90.03 $\pm$ 0.27 |  46.34 $\pm$ 1.12 | 11.7 |  87.95 $\pm$ 0.57 |  60.40 $\pm$ 2.91 | 43.2
DeepFM |  91.35 $\pm$ 0.20 |  50.22 $\pm$ 0.75 | 18.9 |  90.46 $\pm$ 0.44 |  72.35 $\pm$ 1.04 | 34.2
NFM |  91.41 $\pm$ 0.25 |  50.20 $\pm$ 0.99 | 20.3 |  90.40 $\pm$ 0.50 |  71.86 $\pm$ 0.90 | 30.1
AFM |  88.76 $\pm$ 0.53 |  46.05 $\pm$ 1.16 | 3.3 |  86.71 $\pm$ 0.57 |  65.63 $\pm$ 1.41 | 5.0
UltraGCN |  91.29 $\pm$ 0.28 |  47.01 $\pm$ 0.69 | 5.0 |  89.54 $\pm$ 0.40 |  71.25 $\pm$ 1.14 | 14.6
ADRNet (ours) | 92.23 $\pm$ 0.21* | 51.72 $\pm$ 0.89* | 5.9 | 91.20 $\pm$ 0.42* | 74.21 $\pm$ 1.04* | 15.2

  * •

Note: * means statistically significant results ($\text{p-value}\leq 0.01$)
using the paired-t-test compared with the best baseline.

Table 2. Ablation studies on shallow _versus_ deep layers, and latent
collaborative filtering _versus_ content-based methods.

| Model architecture | Liu’s | AEOLUS
---|---|---|---
Method | Shallow layers | Deep layers | Latent vector | Drug descriptor |  AUC $(\times 10^{-2})$ |  AUPR $(\times 10^{-2})$ |  AUC $(\times 10^{-2})$ |  AUPR $(\times 10^{-2})$
MF | ✓ |  | ✓ |  |  90.89 $\pm$ 0.26 |  45.00 $\pm$ 0.65 |  87.15 $\pm$ 0.37 |  61.03 $\pm$ 1.11
NCF |  | ✓ | ✓ |  |  89.84 $\pm$ 0.36 |  44.00 $\pm$ 0.67 |  84.68 $\pm$ 0.41 |  59.35 $\pm$ 1.13
NMF | ✓ | ✓ | ✓ |  |  91.21 $\pm$ 0.37 |  46.73 $\pm$ 0.67 |  87.16 $\pm$ 0.32 |  61.03 $\pm$ 1.11
LR | ✓ |  |  | ✓ |  89.25 $\pm$ 0.23 |  46.54 $\pm$ 0.83 |  89.02 $\pm$ 1.02 |  66.89 $\pm$ 2.29
MLP |  | ✓ |  | ✓ |  90.23 $\pm$ 0.29 |  45.64 $\pm$ 0.79 |  89.48 $\pm$ 0.08 |  68.65 $\pm$ 1.19
Wide&Deep | ✓ | ✓ |  | ✓ |  91.42 $\pm$ 0.25 |  50.23 $\pm$ 0.85 |  90.50 $\pm$ 0.45 |  72.38 $\pm$ 1.11
ADRNet w/o share | ✓ | ✓ | ✓ | ✓ | 92.16 $\pm$ 0.21* | 51.71 $\pm$ 0.93* | 91.14 $\pm$ 0.40* | 73.91 $\pm$ 1.09*
ADRNet | ✓ | ✓ | ✓ | ✓ | 92.23 $\pm$ 0.21* | 51.72 $\pm$ 0.89* | 91.20 $\pm$ 0.42* | 74.21 $\pm$ 1.04*

  * •

Note: * means statistically significant results ($\text{p-value}\leq 0.01$)
using the paired-t-test compared with the best baseline.

Following previous studies (Yamanishi et al., 2012; Zhang et al., 2015;
Pauwels et al., 2011; Dimitri and Lió, 2017), we use 10-fold cross-validation
for our experiments. Two common metrics are used to evaluate the prediction
performance, i.e., area under the ROC curve (AUC) and area under the
precision-recall curve (AUPR). All the experiments are implemented on PyTorch
with Adam as the optimizer. We tune the learning rate in
$\\{0.001,0.005,0.01,0.05\\}$, weight decay in $\\{1e-6,1e-5,1e-4,1e-3\\}$,
and embedding size in $\\{16,32,64,128,256,512,1024\\}$ for both Liu’s and
AEOLUS.

###  4.2. Performance Comparison

_Overall Performance._ We compare the proposed ADRNet with a wide range of
baseline methods on two real-world datasets Liu’s and AEOLUS, and the results
are shown in Table 1. We find the proposed ADRNet outperforms the best
baseline in terms of AUC and AUPR at 0.01 statistical significance. We also
compare the running times of the various methods in Table 1. Compared with the
content-based collaborative filtering, ADRNet has a lower running time due to
the faster convergence resulting from the joint training of the deep network
with latent collaborative filtering.
_Analysis of Convergence._ To further explore the convergence performance of
ADRNet, we compare AUCs and AUPRs with respect to the number of iterations in
Figure 2. We find that ADRNet using joint training has the fastest convergence
speed on both datasets, while NMF converges the slowest. This indicates the
guiding effect of drug descriptor-based representations on the latent vectors
in collaborative filtering.

###  4.3. Ablation Studies

_Shallow vs. Deep Layers._ ADRNet has both shallow latent collaborative
filtering and deep drug representation layers. In Table 2, we conduct ablation
experiments to explore the effects of using the shallow network, using the
deep network, and using both on drug-ADR prediction performance. We find that
using both shallow and deep networks helps to improve the prediction
performance in terms of AUC and AUPR, which is consistent with the discussion
in Section 3.2.
_Latent Vector vs. Drug Descriptor._ We also explore the effect of latent
vectors versus drug descriptors on prediction performance, and the results are
presented in Table 2. We find that the high-dimensional drug descriptor-based
models outperform the pure collaborative filtering models significantly. This
is due to the structural, chemical, physical, and biological information about
the drug contained in the drug descriptors.
_Embedding Size._ We next investigate how embedding size affects drug-ADR
prediction performance. Specifically, we compare ADRNet with the best pure
(NMF) and the best content-based (Wide&Deep) collaborative filtering that uses
both shallow and deep networks, with the results shown in Figure 3. Notably,
ADRNet stably outperforms NMF and Wide&Deep at all embedding sizes and
achieves the optimal performance when embedding sizes are around 128-256.

(a) AUC on Liu’s

(b) AUPR on Liu’s

(c) AUC on AEOLUS

(d) AUPR on AEOLUS

Figure 2. Prediction performance of the ADRNet, compared with the best single-
label prediction model (SVM), multi-label prediction model (Wide&Deep), latent
collabrative filtering model (NMF), w.r.t. the number of iterations on Liu’s
and AEOLUS.

(a) AUC on Liu’s

(b) AUPR on Liu’s

(c) AUC on AEOLUS

(d) AUPR on AEOLUS

Figure 3. Effects of the embedding size on Liu’s and AEOLUS.

##  5\. Conclusion

We formulate the prediction of multi-label ADRs as a drug-ADR collaborative
filtering problem, and to the best of our knowledge, this is the first study
to provide extensive benchmark results of previous collaborative filtering
methods on two large publicly available clinical datasets. Then, by noting the
gap between previous studies on collaborative filtering with symmetric
information and the asymmetry of available drug and ADR information for multi-
label ADR predictions, we propose ADRNet as a generalized collaborative
filtering framework combining clinical and non-clinical data for drug-ADR
prediction. We describe and discuss in detail the three modules of ADRNet: a
deep drug representation module, a shallow latent collaborative filtering
module, and a drug collaborative filtering module. Notably, ADRNet inherits
the ”generalization” ability of the deep model and the ”memory” ability of the
latent collaborative filtering, and further shares the ADR latent vector to
establish stronger connection. Finally, we jointly train the deep drug
representation and the shallow collaborative filtering network to better
trade-off the predictions of each sub-network. Extensive experiments are
conducted on two publicly available real-world drug-ADR clinical datasets and
two non-clinical datas, and the results show that the proposed ADRNet can
effectively combine the advantages of drug descriptors and latent
collaborative filtering to improve drug-ADR prediction performance.

## References

  * (1)
  * Agresti (1992) Alan Agresti. 1992\.  A survey of exact inference for contingency tables.  _Statistical science_ 7, 1 (1992), 131–153.
  * Arrowsmith and Miller (2013) John Arrowsmith and Philip Miller. 2013.  Trial watch: phase II and phase III attrition rates 2011-2012.  _Nature reviews. Drug discovery_ 12, 8 (2013), 569\.
  * Banda et al. (2016) Juan M Banda, Lee Evans, Rami S Vanguri, Nicholas P Tatonetti, Patrick B Ryan, and Nigam H Shah. 2016\.  A curated and standardized adverse drug event resource to accelerate drug safety research.  _Scientific data_ 3, 1 (2016), 1–11.
  * Belleau et al. (2008) François Belleau, Marc-Alexandre Nolin, Nicole Tourigny, Philippe Rigault, and Jean Morissette. 2008\.  Bio2RDF: towards a mashup to build bioinformatics knowledge systems.  _Journal of biomedical informatics_ 41, 5 (2008), 706–716.
  * Cami et al. (2011) Aurel Cami, Alana Arnold, Shannon Manzi, and Ben Reis. 2011\.  Predicting adverse drug events using pharmacological network models.  _Science translational medicine_ 3, 114 (2011), 114ra127–114ra127.
  * Chen and Ung (2001) YZ Chen and CY Ung. 2001\.  Prediction of potential toxicity and side effect protein targets of a small molecule by a ligand–protein inverse docking approach.  _Journal of Molecular Graphics and Modelling_ 20, 3 (2001), 199–218.
  * Cheng et al. (2016) Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016\.  Wide & deep learning for recommender systems. In _Proceedings of the 1st workshop on deep learning for recommender systems_. 7–10.
  * Consonni (2009) R Todeschiniand V Consonni. 2009\.  Handbook of Molecular Descriptors, vol. 11.
  * Davazdahemami and Delen (2018) Behrooz Davazdahemami and Dursun Delen. 2018.  A chronological pharmacovigilance network analytics approach for predicting adverse drug events.  _Journal of the American Medical Informatics Association_ 25, 10 (2018), 1311–1321.
  * Dimitri and Lió (2017) Giovanna Maria Dimitri and Pietro Lió. 2017.  DrugClust: a machine learning approach for drugs side effects prediction.  _Computational biology and chemistry_ 68 (2017), 204–210.
  * Edwards and Aronson (2000) I Ralph Edwards and Jeffrey K Aronson. 2000.  Adverse drug reactions: definitions, diagnosis, and management.  _The lancet_ 356, 9237 (2000), 1255–1259.
  * Fliri et al. (2005) Anton F Fliri, William T Loging, Peter F Thadeio, and Robert A Volkmann. 2005.  Analysis of drug-induced effect patterns to link structure and side effects of medicines.  _Nature chemical biology_ 1, 7 (2005), 389–397.
  * Grisoni et al. (2018) Francesca Grisoni, Davide Ballabio, Roberto Todeschini, and Viviana Consonni. 2018.  Molecular descriptors for structure–activity applications: a hands-on approach.  In _Computational Toxicology_. Springer, 3–53.
  * Guo et al. (2017) Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017.  DeepFM: a factorization-machine based neural network for CTR prediction.  _arXiv preprint arXiv:1703.04247_ (2017).
  * Hammann et al. (2010) Felix Hammann, Heike Gutmann, Nadine Vogt, Christoph Helma, and Juergen Drewe. 2010.  Prediction of adverse drug reactions using decision tree modeling.  _Clinical Pharmacology & Therapeutics_ 88, 1 (2010), 52–59.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017\.  Neural collaborative filtering. In _Proceedings of the 26th international conference on world wide web_. 173–182.
  * Houck and Kavlock (2008) Keith A Houck and Robert J Kavlock. 2008.  Understanding mechanisms of toxicity: insights from drug discovery research.  _Toxicology and applied pharmacology_ 227, 2 (2008), 163–178.
  * Jahid and Ruan (2013) Md Jamiul Jahid and Jianhua Ruan. 2013.  An ensemble approach for drug side effect prediction. In _2013 IEEE International Conference on Bioinformatics and Biomedicine_. IEEE, 440–445.
  * Karimi et al. (2015) Sarvnaz Karimi, Chen Wang, Alejandro Metke-Jimenez, Raj Gaire, and Cecile Paris. 2015\.  Text and data mining techniques in adverse drug reaction detection.  _ACM Computing Surveys (CSUR)_ 47, 4 (2015), 1–39.
  * Kim et al. (2016) Sunghwan Kim, Paul A Thiessen, Evan E Bolton, Jie Chen, Gang Fu, Asta Gindulyte, Lianyi Han, Jane He, Siqian He, Benjamin A Shoemaker, et al. 2016\.  PubChem substance and compound databases.  _Nucleic acids research_ 44, D1 (2016), D1202–D1213.
  * Koren et al. (2009) Yehuda Koren, Robert Bell, and Chris Volinsky. 2009\.  Matrix factorization techniques for recommender systems.  _Computer_ 42, 8 (2009), 30–37.
  * Liu et al. (2012) Mei Liu, Yonghui Wu, Yukun Chen, Jingchun Sun, Zhongming Zhao, Xue-wen Chen, Michael Edwin Matheny, and Hua Xu. 2012\.  Large-scale prediction of adverse drug reactions using chemical, biological, and phenotypic properties of drugs.  _Journal of the American Medical Informatics Association_ 19, e1 (2012), e28–e35.
  * Lorberbaum et al. (2016) Tal Lorberbaum, Kevin J Sampson, Raymond L Woosley, Robert S Kass, and Nicholas P Tatonetti. 2016\.  An integrative data science pipeline to identify novel drug interactions that prolong the QT interval.  _Drug safety_ 39, 5 (2016), 433–441.
  * Mahadevan and Schilling (2003) Radhakrishnan Mahadevan and Chrisophe H Schilling. 2003.  The effects of alternate optimal solutions in constraint-based genome-scale metabolic models.  _Metabolic engineering_ 5, 4 (2003), 264–276.
  * Mao et al. (2021) Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, and Xiuqiang He. 2021\.  UltraGCN: ultra simplification of graph convolutional networks for recommendation. In _Proceedings of the 30th ACM International Conference on Information & Knowledge Management_. 1253–1262.
  * Montastruc et al. (2011) Jean-Louis Montastruc, Agnès Sommet, Haleh Bagheri, and Maryse Lapeyre-Mestre. 2011.  Benefits and strengths of the disproportionality analysis for identification of adverse drug reactions in a pharmacovigilance database.  _British journal of clinical pharmacology_ 72, 6 (2011), 905\.
  * Muñoz et al. (2016) Emir Muñoz, Vít Nováček, and Pierre-Yves Vandenbussche. 2016\.  Using drug similarities for discovery of possible adverse reactions. In _AMIA Annual Symposium Proceedings_ , Vol. 2016. American Medical Informatics Association, 924.
  * Nguyen et al. (2021) Duc Anh Nguyen, Canh Hao Nguyen, and Hiroshi Mamitsuka. 2021\.  A survey on adverse drug reaction studies: data, tasks and machine learning methods.  _Briefings in Bioinformatics_ 22, 1 (2021), 164–177.
  * Pauwels et al. (2011) Edouard Pauwels, Véronique Stoven, and Yoshihiro Yamanishi. 2011.  Predicting drug side-effect profiles: a chemical fragment-based approach.  _BMC bioinformatics_ 12, 1 (2011), 1–13.
  * Qu et al. (2016) Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016.  Product-based neural networks for user response prediction. In _2016 IEEE 16th International Conference on Data Mining (ICDM)_. IEEE, 1149–1154.
  * Sedhain et al. (2015) Suvash Sedhain, Aditya Krishna Menon, Scott Sanner, and Lexing Xie. 2015.  Autorec: Autoencoders meet collaborative filtering. In _Proceedings of the 24th international conference on World Wide Web_. 111–112.
  * Shaked et al. (2016) Itay Shaked, Matthew A Oberhardt, Nir Atias, Roded Sharan, and Eytan Ruppin. 2016.  Metabolic network prediction of drug side effects.  _Cell systems_ 2, 3 (2016), 209–213.
  * Shan et al. (2016) Ying Shan, T Ryan Hoens, Jian Jiao, Haijing Wang, Dong Yu, and JC Mao. 2016\.  Deep crossing: Web-scale modeling without manually crafted combinatorial features. In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_. 255–262.
  * Tatonetti et al. (2009) Nicholas P Tatonetti, Tianyun Liu, and Russ B Altman. 2009\.  Predicting drug side-effects by chemical systems biology.  _Genome biology_ 10, 9 (2009), 1–4.
  * Testa and Kier (1991) Bernard Testa and Lemont B Kier. 1991.  The concept of molecular structure in structure–activity relationship studies and drug design.  _Medicinal research reviews_ 11, 1 (1991), 35–48.
  * Vedani et al. (2005) Angelo Vedani, Max Dobler, and Markus A Lill. 2005\.  In silico prediction of harmful effects triggered by drugs and chemicals.  _Toxicology and Applied Pharmacology_ 207, 2 (2005), 398–407.
  * Wang et al. (2019) Chi-Shiang Wang, Pei-Ju Lin, Ching-Lan Cheng, Shu-Hua Tai, Yea-Huei Kao Yang, Jung-Hsien Chiang, et al. 2019\.  Detecting potential adverse drug reactions using a deep neural network model.  _Journal of medical Internet research_ 21, 2 (2019), e11016.
  * Wang et al. (2017) Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017\.  Deep & cross network for ad click predictions.  In _Proceedings of the ADKDD’17_. 1–7.
  * Wu et al. (2010) Tai-Yin Wu, Min-Hua Jen, Alex Bottle, Mariam Molokhia, Paul Aylin, Derek Bell, and Azeem Majeed. 2010.  Ten-year trends in hospital admissions for adverse drug reactions in England 1999–2009.  _Journal of the Royal Society of Medicine_ 103, 6 (2010), 239–250.
  * Xiao et al. (2017b) Cao Xiao, Ping Zhang, W Chaovalitwongse, Jianying Hu, and Fei Wang. 2017b.  Adverse drug reaction prediction with symbolic latent dirichlet allocation. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 31.
  * Xiao et al. (2017a) Jun Xiao, Hao Ye, Xiangnan He, Hanwang Zhang, Fei Wu, and Tat-Seng Chua. 2017a.  Attentional factorization machines: Learning the weight of feature interactions via attention networks.  _arXiv preprint arXiv:1708.04617_ (2017).
  * Yamanishi et al. (2012) Yoshihiro Yamanishi, Edouard Pauwels, and Masaaki Kotera. 2012\.  Drug side-effect prediction based on the integration of chemical and biological spaces.  _Journal of chemical information and modeling_ 52, 12 (2012), 3284–3292.
  * Zhang et al. (2016) Weinan Zhang, Tianming Du, and Jun Wang. 2016.  Deep learning over multi-field categorical data. In _European conference on information retrieval_. Springer, 45–57.
  * Zhang et al. (2015) Wen Zhang, Feng Liu, Longqiang Luo, and Jingxia Zhang. 2015\.  Predicting drug side effects by multi-label learning and ensemble learning.  _BMC bioinformatics_ 16, 1 (2015), 1–11.