#  Change Matters: Medication Change Prediction with Recurrent Residual
Networks

Chaoqi Yang1 Cao Xiao2 Lucas Glass2&Jimeng Sun1∗
1Department of Computer Science, University of Illinois Urbana Champaign
2Analytics Center of Excellence, IQVIA
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Deep learning is revolutionizing predictive healthcare, including recommending
medications to patients with complex health conditions. Existing approaches
focus on predicting all medications for the current visit, which often
overlaps with medications from previous visits. A more clinically relevant
task is to identify medication changes.

In this paper, we propose a new recurrent residual networks, named MICRON, for
medication change prediction. MICRON takes the changes in patient health
records as input and learns to update a hidden medication vector and the
medication set recurrently with a reconstruction design. The medication vector
is like the memory cell that encodes longitudinal information of medications.
Unlike traditional methods that require the entire patient history for
prediction, MICRON has a residual-based inference that allows for sequential
updating based only on new patient features (e.g., new diagnoses in the recent
visit), which is efficient.

We evaluated MICRON on real inpatient and outpatient datasets. MICRON achieves
$3.5$% and $7.8$% relative improvements over the best baseline in F1 score,
respectively. MICRON also requires fewer parameters, which significantly
reduces the training time to $38.3$s per epoch with $1.5\times$ speed-up.

##  1 Introduction

Recently years, deep learning has demonstrated initial success in potentially
assisting clinical decision-making Almirall et al. (2012); Choi et al. (2017);
Xiao et al. (2018); Mao et al. (2019). Among others, the medication
recommendation task has drawn lots of research interest Wang et al. (2017,
2019); Shang et al. (2019a, b); Zhang et al. (2017); Killian et al. (2019);
Wang et al. (2018). The common strategy of medication recommendation learns
representations for medical entities (e.g., diagnoses, medications) from
electronic health records, and use the learned representations to predict
medications that fit the patient’s health condition while avoiding adverse
drug interactions.

Many existing works focus on recommending the full set of medications in a
visit Zhang et al. (2017); Shang et al. (2019a, b); Xiao et al. (2018), which
can be quite redundant from previous visits. Because the medications often
remain stable over time with large overlaps between consecutive visits. For
example, we investigate the Jaccard coefficient over consecutive visits on
MIMIC-III data Johnson et al. (2016) (Fig. 1). Among patients with multiple
visits, although most of them are diagnosed with different conditions during
consecutive visits (mean Jaccard below 0.2), the sets of medications remain
stable (mean Jaccard around 0.5).

Figure 1: Histogram of Jaccard Coefficients between consecutive visits. We
observe relatively weaker overlaps in diagnoses but much stronger overlaps in
medications over consecutive visits. It implies that the medication change is
potentially more meaningful to predict.

However, such medication patterns were rarely explored and leveraged to
augment medication recommendation tasks. Challenges mainly arise from (1) how
to accurately characterize the changes of patient health condition for each
time step, and (2) how to correctly identify the medication changes based on
health changes.

To fill in the gap, we propose a new recurrent residual learning approach,
named MedicatIon Change pRedictiON (MICRON) to predict medication changes and
simultaneously model longitudinal medical history. MICRON is enabled by the
following technical contributions.

  * •

Efficient representation of changing health conditions. MICRON uses a residual
health representation to sequentially update changes in patient health
conditions, which provides more efficient model inference than RNN-based
alternatives.

  * •

Explicit change set prediction: MICRON decomposes the medication change
prediction task into predicting two sets: (1) the removal set that removes
previous medicines that are no longer needed; and (2) the addition set that
brings in new medicines for newly developed diseases. Two sets are modeled by
a recurrently updated medication vector and addition and removal thresholds
selected at the high-confidence region from validation ROC-curve and thus
could provide reliable inclusion-exclusion criterion.

We evaluated MICRON against state-of-the-art models on two real-world patient
datasets: one inpatient MIMIC-III data and one outpatient dataset. MICRON
outperforms the best baseline GAMENet Shang et al. (2019b) with $3.5$% and
$7.8$% relative improvement in F1 measure, respectively. In addition, MICRON
achieves $1.5\times$ speed-up in training and inference compared with GAMENet.

##  2 Related Works

Rule-based models Almirall et al. (2012); Chen et al. (2016) typically rely on
human-designed clinical guidelines, which require huge efforts from
clinicians. For example, Lakkaraju and Rudin (2017) optimizes a sequence of
if-then-else rules, which maps the patient status into the prescription
decision.

Instance-based methods extract patient features only from current visits.
Zhang et al. (2017) formulated the medication recommendation as a multi-
instance multi-label (MIML) task and proposed a content-attention mechanism-
based sequence-to-sequence model. Wang et al. (2017) jointly embedded
diseases, medicines, patients, and their corresponding relations into a shared
space by the knowledge graph, which requires multiple external data sources.

Longitudinal approach Wang et al. (2018, 2019); Xiao et al. (2018); Bhoi et
al. (2020) is a popular approach that captures the sequential dependency in
patient treatment history. Choi et al. (2016); Bajor and Lasko (2017) modeled
the longitudinal patient history by RNNs for various clinical predictive
tasks. Shang et al. (2019b) and Le et al. (2018) adopted memory-based networks
with RNNs to handle the dependency among longitudinal medical codes.

Compared with existing works, MICRON is based on a new perspective that
focuses on predicting the changes. This is more realistic since clinicians
usually update patient prescriptions by only a small proportion for a
patient’s new visit.

##  3 Method

###  3.1 Problem Formulation

######  Definition 1 (Patient EHR Records)

Patient EHR records are usually represented by an ordered sequence of tuples.
For a patient $j$, we denote his/her clinical documentaries as
$\mathbf{X}_{j}=[\mathbf{x}_{j}^{(1)},\mathbf{x}_{j}^{(2)},\mathbf{x}^{(3)}_{j},\dots]$,
where the $t_{th}$ entry, $\mathbf{x}^{(t)}_{j}$, records the information of
the $t_{th}$ visit, such as diagnoses, procedures and prescription
information. In the paper,
$\mathbf{x}^{(t)}_{j}=[\mathbf{d}^{(t)}_{j},\mathbf{p}^{(t)}_{j},\mathcal{M}^{(t)}_{j}]$,
where $\mathbf{d}^{(t)}_{j}\in\\{0,1\\}^{|\mathcal{D}|}$ and
$\mathbf{p}^{(t)}_{j}\in\\{0,1\\}^{|\mathcal{P}|}$ are multi-hot diagnoses and
procedure vectors, while $\mathcal{D}$ and $\mathcal{P}$ are the overall
diagnosis and procedure sets. $\mathcal{M}^{(t)}_{j}\subset\mathcal{M}$ is the
$t_{th}$ medication set and $\mathcal{M}$ is a set for all possible medicines.
We denote the visit-wise medication addition (new) and removal (old) sets as
$\mathcal{N}^{(t)}_{target},\mathcal{O}^{(t)}_{target}\subset\mathcal{M}$,
separately, which naturally follows the equality,
${\mathcal{M}}^{(t)}=({\mathcal{M}}^{(t-1)}\cup\mathcal{N}^{(t)}_{target})\setminus\mathcal{O}^{(t)}_{target}$.

######  Problem 1 (Medication Change Prediction)

Medication change prediction aims at determining the medication addition set
$\mathcal{N}^{(t)}$ and the removal set $\mathcal{O}^{(t)}$ at visit $t$,
given last prescription, $\tilde{\mathcal{M}}^{(t-1)}$ and patient health
history $[\mathbf{d}^{(1)},\dots,\mathbf{d}^{(t)}]$ and
$[\mathbf{p}^{(1)},\dots,\mathbf{p}^{(t)}]$. The model aims to minimize the
gap between current estimation
$\tilde{\mathcal{M}}^{(t)}=(\tilde{\mathcal{M}}^{(t-1)}\cup\mathcal{N}^{(t)})\setminus\mathcal{O}^{(t)}$
and real prescriptions ${\mathcal{M}}^{(t)}$, and also control the incidence
of DDIs as denoted by
$\mathbf{A}\in\\{0,1\\}^{|\mathcal{M}|\times|\mathcal{M}|}$, where
$\mathbf{A}_{ij}=1$ implies that medicine $i$ and $j$ could interact.

###  3.2 MICRON Method

Figure 2: MICRON Framework. To represent a patient health condition, the model
first feeds on diagnosis and procedure information and then generates a
compact patient health representation by health representation network, an
affine function. During training, the model uses a feed-forward network for
prescription network and learns the residual representation under a novel
reconstruction loss. In the inference, our model inputs the heath update to
the same prescription network and then generates addition/removal sets to
update the current prescription.

Overview As shown in Fig. 2, MICRON has three modules: (1) a patient
representation module that embeds diagnosis and procedure codes into latent
health representation; (2) a prescription reconstruction module (training
phase), where MICRON trains on consecutive pairs of visits and learns residual
medication representations under a new reconstruction design; (3) a medication
updating module (inference phase) for model inference, where MICRON
initializes with previous medication information. For each subsequent visit,
MICRON only requires an update of patient health status and then will predict
the changes in the existing medications.

The key difference between MICRON and existing medication recommendation
models Shang et al. (2019b); Choi et al. (2016) is that while these models
learn global sequential patterns using RNNs, MICRON learns sequential
information locally (by every two consecutive visits) and propagates them
visit-by-visit to preserve the longitudinal patient information.

###  3.3 Patient Representation

Patient representation aims to learn a compact and indicative vector to
represent a patient’s status. In a clinical visit, doctors will recommend
medications based on diagnosis and procedure information. Our module also
feeds on these two features. Since MICRON is proposed for generic patients, we
leave the subscript notation in the following.

Diagnosis and Procedure Encoders. For the $t_{th}$ visit, the input features,
$\mathbf{d}^{(t)}\in\mathbb{R}^{|\mathcal{D}|}$ and
$\mathbf{p}^{(t)}\in\mathbb{R}^{|\mathcal{P}|}$, can be extracted from
clinical documentary. $\mathbf{d}^{(t)}$ is the multi-hot diagnosis vector,
while $\mathbf{p}^{(t)}$ is the procedure vector. Following the similar
strategy in Zhang et al. (2017); Shang et al. (2019b), we transform these two
vectors into the embedding space using mapping matrices
$\mathbf{E}_{d}\in\mathbb{R}^{s\times|\mathcal{D}|}$ and
$\mathbf{E}_{p}\in\mathbb{R}^{s\times|\mathcal{P}|}$ ($s$ is the size of
embedding space),

$\displaystyle\mathbf{d}^{(t)}_{e}=\mathbf{E}_{d}\mathbf{d}^{(t)}~{}~{}~{}~{}\mbox{and}~{}~{}~{}~{}\mathbf{p}^{(t)}_{e}=\mathbf{E}_{p}\mathbf{p}^{(t)}.$
(1)

During training, these two tables are shared among all visits and patients.
The results, $\mathbf{d}^{(t)}_{e}$ and $\mathbf{p}^{(t)}_{e}$, are of the
same dimension $\mathbb{R}^{s}$.

Patient Hidden Representation. To achieve one compact health representation,
$\mathbf{d}^{(t)}_{e}$ and $\mathbf{p}^{(t)}_{e}$ are further concatenated and
parametrized by a health representation network, $\mbox{NET}_{health}$,

$\mathbf{h}^{(t)}=\mbox{NET}_{health}~{}\left(\left[\mathbf{d}^{(t)}_{e}~{}\|~{}\mathbf{p}^{(t)}_{e}\right]\right),$
(2)

which outputs an integrated health representation
$\mathbf{h}^{(t)}\in\mathbb{R}^{s}$. In this paper, we use affine function
(one layer neural network without activation) for $\mbox{NET}_{health}$.

Unlike previous works Shang et al. (2019b); Zhang et al. (2017), this paper
does not use recurrent neural networks (RNN) to capture a patient’s health
history. Starting from $\mathbf{h}^{(t)}$, the model architecture differs in
training and inference. Next, we elaborate on these two phases.

###  3.4 Training: Prescription Reconstruction Module

Our MICRON trains on two consecutive visits, e.g., the $(t-1)_{th}$ and the
$t_{th}$ visits. Given the health representations, i.e., $\mathbf{h}^{(t-1)}$
and $\mathbf{h}^{(t)}$, a straightforward way for recommending medications is
to learn a mapping, i.e., a prescription network,
$\mbox{NET}_{med}:\mathbb{R}^{s}\mapsto\mathbb{R}^{|\mathcal{M}|}$, from
hidden embedding space to medication space for two visits, separately.

$\displaystyle\hat{\mathbf{m}}^{(t-1)}$
$\displaystyle=\mbox{NET}_{med}~{}(\mathbf{h}^{(t-1)}),$ (3)
$\displaystyle\hat{\mathbf{m}}^{(t)}$
$\displaystyle=\mbox{NET}_{med}~{}(\mathbf{h}^{(t)}).$ (4)

where
$\hat{\mathbf{m}}^{(t-1)},\hat{\mathbf{m}}^{(t)}\in\mathbb{R}^{|\mathcal{M}|}$
are medication representations and each entry quantifies a real value for the
corresponding medicine. In the paper, $\mbox{NET}_{med}$ is implemented as a
fully connected neural network. To obtain the actual recommendations, a
trivial way Shang et al. (2019a, b) is to apply a medication output layer,
which consists of a Sigmoid function $\sigma(\cdot)$, followed by a pre-
defined threshold $\delta$, picking up medicines with larger activation value.
However, in this paper, we hope to utilize and emphasize the dependency usage
between $\mathbf{h}^{(t-1)}$ and $\mathbf{h}^{(t)}$ in the model.

Residual Medication Representation. Formally, the difference between
$\mathbf{h}^{(t-1)}$ and $\mathbf{h}^{(t)}$, i.e.,
$\mathbf{r}^{(t)}=\mathbf{h}^{(t)}-\mathbf{h}^{(t-1)}$, is denoted as residual
health representation, which encodes the changes in clinical health
measurements, indicating an update in patient’s health condition. Naturally,
the health update $\mathbf{r}^{(t)}$ will cause an update in the resulting
medication representation $\mathbf{u}^{(t)}$. Our motivation is that if
$\mbox{NET}_{med}$ can map a complete health representation (e.g.,
$\mathbf{h}^{(t)}$) into a complete medication representation (e.g.,
$\hat{\mathbf{m}}^{(t)}$), then a residual health representation should also
be mapped into an update in the same representation space through
$\mbox{NET}_{med}$. In other words, $\mathbf{r}^{(t)}$ and $\mathbf{u}^{(t)}$
shall also follow the same mapping function, $\mbox{NET}_{med}$,

$\mathbf{u}^{(t)}=\mbox{NET}_{med}~{}(\mathbf{r}^{(t)}).$ (5)

To learn Eqn. (3) and (4), we could use the medication combinations in the
dataset as supervision, however, it is hard to formulate direct supervision
for Eqn. (5). A simple idea is to model the addition and the removal
medication sets separately (as we show in the experiment that separate
modeling DualNN does not work well). Therefore, we consider reconstructing
${\mathbf{u}}^{(t)}$ from $\hat{\mathbf{m}}^{(t-1)}$ and
$\hat{\mathbf{m}}^{(t)}$ by both unsupervised and supervised regularization.

Unsupervised Residual Reconstruction. To model the medication changes, we
design a reconstruction loss. For Eqn. (3), (4) and (5), the inputs follow a
residual relation: $\mathbf{h}^{(t-1)}+\mathbf{r}^{(t)}=\mathbf{h}^{(t)}$.
Naturally, we also impose a similar relation in the medication output layer by
introducing an unsupervised reconstruction loss ($\sigma(\cdot)$ is a Sigmoid
function),

$L^{(t)}_{rec}=\|\sigma(\hat{\mathbf{m}}^{(t-1)}+{\mathbf{u}}^{(t)})-\sigma(\hat{\mathbf{m}}^{(t)})\|_{2},$
(6)

which is calculated with $L_{2}$ norm. This reconstruction loss enforces the
reconstructed recommendations from $\hat{\mathbf{m}}^{(t-1)}$ and the residual
${\mathbf{u}}^{(t)}$ to be close to the recommendations given by
$\hat{\mathbf{m}}^{(t)}$. We show in the experiment that $L^{(t)}_{rec}$ is
essential for learning the residual.

Supervised Multi-label Classification. To jointly modeling a low DDI output,
we introduce three differentiable loss functions to improve
$\hat{\mathbf{m}}^{(t-1)}$ and $\hat{\mathbf{m}}^{(t)}$, so as to achieve a
better reconstruction ${\mathbf{u}}^{(t)}$.

  * •

Drug-Drug Interaction Loss. Since adverse drug-drug interaction (DDI) is a
leading cause of morbidity and mortality in clinical treatments Percha and
Altman (2013), we penalize the presence of DDIs in the output medication
representation, $\hat{\mathbf{m}}^{(t)}$. First, we transform it by Sigmoid
function, $\hat{\mathbf{o}}^{(t)}=\sigma(\hat{\mathbf{m}}^{(t)})$, and then
design the DDI loss as,

$L^{(t)}_{ddi}=\sum_{i=1}\sum_{j=1}\mathbf{A}_{ij}\cdot\hat{\mathbf{o}}^{(t)}_{i}\cdot\hat{\mathbf{o}}^{(t)}_{j},$
(7)

where $\mathbf{A}$ is the binary DDI matrix, extracted externally Tatonetti et
al. (2012) and $\mathbf{A}_{ij}$ indicates that medicine $i$ and $j$ have
interaction or not. The term
$\mathbf{A}_{ij}\cdot\hat{\mathbf{o}}^{(t)}_{i}\cdot\hat{\mathbf{o}}^{(t)}_{j}$
is the a scalar product, which is the interaction penalty for medicine $i$ and
$j$, and $\hat{\mathbf{o}}^{(t)}_{i}$ is the $i$-th element of the vector.
Since we care about the DDI rate in the reconstructed representation, this
loss only applies to the current visit $t$.

  * •

Binary Cross-entropy Loss. In addition, we also extract real medication set as
supervision. Assume a multi-hot vector
${\mathbf{m}}^{(t)}\in\\{0,1\\}^{|\mathcal{M}|}$ is the vectorization of the
target medication set $\mathcal{M}^{(t)}$. We adopt binary cross entropy (BCE)
loss,

$\footnotesize
L^{(t)}_{bce}=-\sum_{i=1}{\mathbf{m}}^{(t)}_{i}log(\hat{\mathbf{o}}^{(t)}_{i})+(1-{\mathbf{m}}^{(t)}_{i})log(1-\hat{\mathbf{o}}^{(t)}_{i}),$
(8)

where subscript $i$ indicates each element of the vectors. For this loss
function, we compute on both $\hat{\mathbf{m}}^{(t-1)}$ and
$\hat{\mathbf{m}}^{(t)}$.

  * •

Multi-Label Margin Loss. Then, we employ margin-based loss to enlarge the gap
between the recommended medications and the unselected ones. Since
$\hat{\mathbf{o}}^{(t)}_{i}\in(0,1)$, the margin is set 1 in our paper.

$\vspace{-1mm}\footnotesize
L^{(t)}_{multi}=\sum_{i,j:~{}\mathbf{m}^{(t)}_{i}=1,\mathbf{m}^{(t)}_{j}\neq
1}\frac{\mbox{max}(0,1-(\hat{\mathbf{o}}^{(t)}_{i}-\hat{\mathbf{o}}^{(t)}_{j}))}{|\mathcal{M}|}.$
(9)

We also consider to penalize both of the visits, i.e., calculating
$L^{(t-1)}_{multi}$ and $L^{(t)}_{multi}$ using this loss.

These three losses use external supervision to optimize the prescription
network, so that during inference, our MICRON would predict medication changes
more accurately.

Overall Loss Function. In the training process, we hope to find optimal values
for embedding tables, $\mathbf{E}_{d}$ and $\mathbf{E}_{p}$, parameter
matrices in $\mbox{NET}_{health}$ and $\mbox{NET}_{med}$. The loss functions
are combined by weighted sum,

$\displaystyle L_{total}$
$\displaystyle=\lambda_{1}L^{(t)}_{rec}+\lambda_{2}L^{(t)}_{ddi}+\lambda_{3}\left(\gamma
L^{(t)}_{bce}+(1-\gamma)L^{(t-1)}_{bce}\right)$
$\displaystyle+\lambda_{4}\left(\gamma
L^{(t)}_{multi}+(1-\gamma)L^{(t-1)}_{multi}\right),$ (10)

where $\lambda_{i},~{}i=1,2,3,4$, are different weights for four types of loss
functions, and $\gamma$ is introduced to balance two consecutive visits.
During the training, one batch contains all visits of one patient, and the
loss is back-propagated after each batch. In the paper, we treat the weights
as hyperparameters. In Appendix, we also prototype a momentum-based method to
select the weights automatically.

###  3.5 Inference: Medication Updating Module

To predict medication changes, it is essential to maintain a medication
combination. We hope that for the subsequent visit, it would be enough to
derive an update in the combination based on new diagnosis or procedure
information. However, like Risperdal for treating schizophrenia and bipolar
disorder, some medicines will not be prescribed based on only one or two
visits, and it might need long-term clinical observation. We therefore also
maintain a medication vector, where each element quantifies the cumulative
effect of a medicine. After clinical visits, each element in the vector will
increase or decrease based on the updates of patient health status.
Essentially, the medication vector is like the memory cell in RNNs, which is
refreshed visit-by-visit. Once it is above or below certain thresholds, the
medicine will be added or removed from the current sets. More concretely, the
medicine change prediction follows three steps.

Step 1: Medication Vector Update. Specifically, for the $t_{th}$ visit of a
patient, the medication changes start from a medication vector,
$\tilde{\mathbf{m}}^{(t-1)}\in\mathbb{R}^{|\mathcal{M}|}$, and a medication
set, $\tilde{\mathcal{M}}^{(t-1)}\subset\mathcal{M}$. The model first updates
the vector based on a residual health representation, $\mathbf{r}^{(t)}$,

$\displaystyle\tilde{\mathbf{m}}^{(t)}$
$\displaystyle=\tilde{\mathbf{m}}^{(t-1)}+{\mathbf{u}}^{(t)}$
$\displaystyle=\tilde{\mathbf{m}}^{(t-1)}+\mbox{NET}_{med}~{}(\mathbf{r}^{(t)}),$
(11)

where $\mathbf{r}^{(t)}$ is calculated by
$\mathbf{h}^{(t)}-\mathbf{h}^{(t-1)}$ (defined in Eqn. (2),
$\mbox{NET}_{health}$), which is implemented as an affine function. We use an
efficient smart inference module to calculate $\mathbf{r}^{(t)}$, in case only
the updates in medical codes (e.g., diagnosis and procedure) are accessible.
We specify it in Appendix.

Step 2: Addition and Removal. Then, based on the updated medication vector,
$\tilde{\mathbf{m}}^{(t)}\in\mathbb{R}^{|\mathcal{M}|}$, we identify which
medicines are ready to add or remove. We design two thresholds ($\delta_{1}$
is for the addition set, while $\delta_{2}$ is for the removal set, where
$1\geq\delta_{1}\geq\delta_{2}\geq 0$) to control the size of changes.
Specifically, we first apply a Sigmoid function $\sigma(\cdot)$, and then the
addition and removal sets are generated by applying the thresholds
$\delta_{1}$ and $\delta_{2}$ element-wise,

$\displaystyle\mathcal{N}^{(t)}$
$\displaystyle=\\{i\mid\sigma(\tilde{\mathbf{m}}^{(t)}_{i})\geq\delta_{1}\\},$
(12) $\displaystyle\mathcal{O}^{(t)}$
$\displaystyle=\\{i\mid\sigma(\tilde{\mathbf{m}}^{(t)}_{i})\leq\delta_{2}\\},$
(13)

where $\mathcal{N}^{(t)}$ ($\mathcal{O}^{(t)}$) is for addition (removal) set,
and subscript $i$ enumerates the index of $\tilde{\mathbf{m}}^{(t)}$. Note
that, $\mathcal{N}^{(t)}\cap\mathcal{O}^{(t)}=\varnothing$. For two
thresholds, if $\delta_{1}=1$ and $\delta_{2}=0$, then
$\mathcal{N}^{(t)}=\mathcal{O}^{(t)}=\varnothing$; in another case,
$\delta_{1}=\delta_{2}$, then “medication change prediction” becomes “full
medication prediction”.

The thresholds $\delta_{1}$ and $\delta_{2}$ are selected based on the
receiver operating characteristic (ROC) of each medicine. Specifically, we
load the pre-trained MICRON on the validation set. For each medicine, we
collect cut-off thresholds of the ROC curve in descending order. $\delta_{1}$
is the average of $5$-percentile over all medications, while $\delta_{2}$ is
based on $95$-percentile. Essentially, $\delta_{1}$ will provide a low false
negative (FN) rate, while $\delta_{2}$ ensures a low false positive (FP) rate.

Step 3: Medication Set Update. Next, we apply the changes (addition and
removal) in the existing medication set. The generation of a new combination
is given by set operations,

$\tilde{\mathcal{M}}^{(t)}=(\tilde{\mathcal{M}}^{(t-1)}\cup{\mathcal{N}}^{(t)})\setminus{\mathcal{O}}^{(t)},$
(14)

where we use set union and subtraction operation. ${\mathcal{N}}^{(t)}$ and
$\tilde{\mathcal{M}}^{(t-1)}$ could have overlaps, while ${\mathcal{O}}^{(t)}$
could also contain medications that are not in $\tilde{\mathcal{M}}^{(t-1)}$.
The overlaps will not affect the final recommendation results due to set
operations.

To sum up, the model begins with a medication vector,
$\tilde{\mathbf{m}}^{(t-1)}$, and a medication set,
$\tilde{\mathcal{M}}^{(t-1)}$, which are provided by the previous visit.
During the current visit, MICRON uses the update of patient status as input
and walks through the above three steps to finish one round of medication
change, as well as to update $\tilde{\mathbf{m}}^{(t)}$ and
$\tilde{\mathcal{M}}^{(t)}$ for the next visit.

##  4 Experiments

We evaluate MICRON against several baselines in both inpatient and outpatient
datasets. We focus on answering the following questions:

  * •

How does MICRON perform against the baselines in medication and change
prediction?

  * •

How does MICRON perform in model efficiency?

  * •

How do different components in MICRON contribute to accurate recommendations?

###  4.1 Experimental Setup

Dataset. We consider a benchmark inpatient dataset: MIMIC-III Johnson et al.
(2016), and a private outpatient dataset: IQVIA PharMetrics Plus (see
processed statistics in Table 1). Details of dataset descriptions,
preprocessing, hyperparameter selections can be found in Appendix.

Table 1: Statistics of Datasets Items | MIMIC-III | IQVIA
---|---|---
# of visits | 14,960 | 30,794
# of patients | 6,335 | 3,023
# of diagnosis codes | 1,958 | 1,744
# of procedure codes | 1,430 | 1,250
# of medication codes | 131 | 155

Baselines. We consider the following baselines (SimNN and DualNN are designed
by ourselves).

  * •

SimNN use the same patient representation $\mathbf{h}^{(t)}$ as MICRON and
then learns a simple 3-way classifier for each medicine (add, remove, and
remain) with the cross-entropy loss.

  * •

DualNN also starts from patient representation $\mathbf{h}^{(t)}$ and then
diverges to two different neural networks. The first one is for addition and
the second for removal. Each neural network classifier uses the binary cross-
entropy loss.

  * •

LEAP Zhang et al. (2017) is an instance-based approach that uses a sequence to
sequence model with reinforcement aftermath fine-tuning. This method generates
a list of medications based on the diagnoses in the same visit.

  * •

RETAIN Choi et al. (2016) is a longitudinal predictive model, which designs a
specialized attention model over RNN. It learns the temporal dependencies
between clinical visits and makes medication recommendations.

  * •

GAMENet Shang et al. (2019b) is also a longitudinal model, which uses RNN,
memory network, and graph neural network. It predicts medications using
historical prescriptions as reference.

Evaluation Strategy and Metrics. We use evaluation metrics such as DDI rate,
Jaccard Similarity, F1-score similar to evaluate the overall recommended
medications, as other related works Shang et al. (2019b); Zhang et al. (2017).
Also, we design new error metrics to evaluate the accuracy of medication
changes: Err(add) and Err(remove).

Err(add) computes the sum of false positive part and false negative part
(white region in Fig. 3) between the predicted addition set
$\mathcal{N}^{(t)}$ and the target addition set $\mathcal{N}_{target}^{(t)}$.
The target addition set is calculated by

$\mathcal{N}_{target}^{(t)}=\mathcal{M}^{(t)}\setminus\tilde{\mathcal{M}}^{(t-1)},$

where $\mathcal{M}^{(t)}$ is the target medication set in the $t_{th}$ visit,
and $\tilde{\mathcal{M}}^{(t-1)}$ is the predicted medication set at the
$(t-1)_{th}$ visit. For a particular patient $j$, this metric is calculated
from the second visit, where the medication changes start,

$\displaystyle\mbox{Err(add)}_{j}$
$\displaystyle=\frac{1}{V(j)}\sum_{t=2}^{V(j)}\left(|\mathcal{N}_{target}^{(t)}\setminus\mathcal{N}^{(t)}|+|\mathcal{N}^{(t)}\setminus\mathcal{N}_{target}^{(t)}|\right),$

where $\mathcal{A}\setminus\mathcal{B}$ is the set subtraction, and $V(j)$
means total number of visits of patient $j$. Finally, we average over all
patients and get Err(add). Similarly, we design Err(remove) to denote errors
for removal. Also, we report model size and training/inference time.

Figure 3: Illustration of Err(add) Metric at the $t$-th Visit

Since the evaluation is on medication change prediction, we assume the
earliest visit appeared in the window as the “first” visit of a patient $j$,
where we could extract the initial medication set,
$\tilde{\mathcal{M}}_{j}^{(1)}=\mathcal{M}_{j}^{(1)}$, and the initial
medication vector, $\tilde{\mathbf{m}}_{j}^{(1)}=\hat{\mathbf{m}}_{j}^{(1)}$.
For a fair comparison, the evaluation of all models starts from the “second”
visit. The definition of other metrics can be found in Appendix.

###  4.2 Experimental Results

Table 2: Performance Comparison (on MIMIC-III / IQVIA)

Metrics | DDI | Jaccard | F1 | Err(add) | Err(remove) | Model Size | Train(epoch) / Test
---|---|---|---|---|---|---|---
SimNN |  0.0837 / 0.0152 | 0.4658 / 0.1920 | 0.6235 / 0.2383 | 6.856 / 2.906 | 7.329 / 1.679 | 1.5MB (376,009 params) | 27.53s / 6.67s
DualNN | 0.0925 / 0.0156 | 0.4880 / 0.1120 | 0.6447 / 0.1783 |  6.261 / 3.406 | 7.987 / 2.579 | 1.3MB (325,702 params) | 27.90s / 5.76s
RETAIN | 0.0932 / 0.0279 | 0.4796 / 0.3320 | 0.6389 / 0.4215 | 8.552 / 2.353 | 6.338 / 0.927 | 1.2MB (287,940 params) | 34.78s / 6.97s
LEAP | 0.0880 / 0.0134 | 0.4331 / 0.1871 | 0.5953 / 0.2742 | 9.105 / 3.663 | 5.939 / 1.286 | 1.7MB (433,286 params) | 199.94s / 26.45s
GAMENet | 0.0928 / 0.0166 |  0.4980 / 0.2025 |  0.6549 / 0.3016 | 8.810 / 3.016 |  5.854 / 2.179 | 1.8MB (449,092 params) | 55.31s / 8.84s
MICRON |  0.0695 / 0.0143 | 0.5234 / 0.3634 | 0.6778 / 0.4544 |  6.090 / 2.088 |  5.853 / 1.213 | 1.1MB (275,395 params) | 38.83s / 4.41s
$\Delta$ Improve. |  $\downarrow 17.0$% / $\uparrow 6.7$% |  $\uparrow 5.1$% / $\uparrow 9.5$% |  $\uparrow 3.5$% / $\uparrow 7.8$% |  $\downarrow 2.7$% / $\downarrow 11.3$% |  $\downarrow 0.02$% / $\uparrow 30.9$% | — | —

*The result of MIMIC-III and DS2 are reported together and separated by “/”. The last two metrics, Model Size and Train(epoch)/Test, are based on MIMIC-III dataset. For the other five metrics, we select the best results in bond font and use underscore to select the best baseline. We also report the improvement $\Delta$ of our MICRON over the best baseline.

Table 3: Ablation Study for Different Model Components (on MIMIC-III)

Metrics | DDI | Jaccard | F1 | Err(add) | Err(remove)
---|---|---|---|---|---
MICRON w/o $L_{rec}$ | 0.0618 $\pm$ 0.0002 | 0.4449 $\pm$ 0.0138 | 0.6050 $\pm$ 0.0014 | 7.143 $\pm$ 0.3753 | 6.224 $\pm$ 0.0598
MICRON w/o $\tilde{\mathbf{m}}^{(t)}$ | 0.0696 $\pm$ 0.0004 | 0.4509 $\pm$ 0.0046 | 0.6096 $\pm$ 0.0368 | 8.496 $\pm$ 0.1401 | 6.463 $\pm$ 0.1468
MICRON w/o $L_{multi}$ | 0.0780 $\pm$ 0.0015 | 0.5020 $\pm$ 0.0072 | 0.6590 $\pm$ 0.0288 | 6.544 $\pm$ 0.2172 | 5.509 $\pm$ 0.0732
MICRON w/o $L_{ddi}$ | 0.0931 $\pm$ 0.0005 | 0.5248 $\pm$ 0.0006 | 0.6793 $\pm$ 0.0081 | 6.402 $\pm$ 0.3020 | 5.897 $\pm$ 0.0397
MICRON w/o $\delta_{1},\delta_{2}$ | 0.0628 $\pm$ 0.0018 | 0.5074 $\pm$ 0.0016 | 0.6635 $\pm$ 0.0026 | 7.216 $\pm$ 0.1335 | 5.084 $\pm$ 0.2706
MICRON | 0.0695 $\pm$ 0.0004 | 0.5234 $\pm$ 0.0008 | 0.6778 $\pm$ 0.0007 | 6.090 $\pm$ 0.0189 | 5.853 $\pm$ 0.0219

For DDI, Err(add), Err(remove) metrics, the lower the better, while for
Jaccard and F1 metrics, the higher the better.

We conduct experimental comparison based on five different random seeds and
show the mean metric values in Table 2. MICRON outperforms all baselines in
both inpatient and outpatient settings, especially for Jaccard and F1 metrics.
LEAP gives a relatively good DDI measure on two datasets; however, its
performance is weaker than other baselines in terms of accuracy. Although
SimNN, DualNN, and RETAIN are implemented from very different perspectives,
the former two are instance-based while the latter uses sequence modeling.
They show neck-to-neck performance on MIMIC-III. For outpatient medication
change prediction (on IQVIA), RETAIN shows strong performance while some
recent state of the art baselines failed, such as GAMENet. We hypothesize that
time spans between two visits can be much longer for outpatients, and thus the
stored memory can be less trustworthy in GAMENet. By learning an effective
residual representation, MICRON provides more accurate and safe medication
recommendations for inpatient or outpatient settings. Also, MICRON requires
much fewer parameters than the state-of-the-art approaches, which is more
efficient.

Due to space limitation, the standard deviation results are reported in the
Appendix. We also test the model’s stability and do a T-hypothesis testing of
MICRON on each metric. As a summary, most of the $p$-values are less than
$0.001$ (mean $p$-value at 6.2e-5), except in two cases on IQVIA: the DDI rate
compared to LEAP and the Err(remove) compared to RETAIN.

###  4.3 Ablation Study on Model Components

In this section, we verify the effectiveness of different components in
MICRON. Specifically, we conduct ablation studies on MIMIC-III and test on the
following variants:

  * •

(i) MICRON w/o $L_{rec}$. We remove the unsupervised loss during training and
solely trained on the supervised loss.

  * •

(ii) MICRON w/o $\tilde{\mathbf{m}}^{(t)}$. We do not maintain the medication
vector, $\tilde{\mathbf{m}}^{(t)}$, and only utilizes the update feature
information, $\mathbf{r}^{(t)}$, between two visits;

  * •

(iii) MICRON w/o $L_{multi}$. We remove $L_{multi}$, and it will be less
confident to use thresholds, $\delta_{1}$ and $\delta_{2}$;

  * •

(iv) MICRON w/o $L_{ddi}$. We remove DDI loss, and the model probably would
provide high-DDI combinations;

  * •

(v) MICRON w/o $\delta_{1},\delta_{2}$. We set $\delta_{1}=\delta_{2}=\delta$,
which implies medications with score above or equal $\delta$ being added, and
medications with score less than $\delta$ being removed. This is a common
strategy used in previous works: $\delta=0.5$ in Shang et al. (2019b) and
$\delta=0.3$ in Shang et al. (2019a) (this model requires ontology
information, so it is not included as baseline). We use $\delta=0.5$ for this
model variant.

The comparison results with variances (after $\pm$) are shown in Table 3.
Overall, all other variations perform better than variant (i) and (ii),
highlighting that the reconstruction design and the initial medication vector
are essential in the model. Without medication vector
$\tilde{\mathbf{m}}^{(t)}$, the model cannot retain the longitudinal
information, thus variant (ii) provides poor results. We also notice that
without DDI loss, variant (iv) outputs a significantly higher DDI rate, and
MICRON shows slightly better results than model variant (iii) without
$L_{multi}$. By integrating all components, MICRON achieves a more balanced
and stable performance in all metrics.

##  5 Conclusion

This paper tackles the medication change prediction problem and proposes a
recurrent residual learning model, named MICRON, for predicting medication
changes. We compare our model with state of the art approaches and show its
effectiveness and efficiency on inpatient MIMIC-III dataset and a proprietary
outpatient IQVIA dataset.

Like previous medication recommendation works, this paper uses the existing
prescriptions as a gold standard. The efficacy of the recommendation is
evaluated by comparing it with the prescriptions given by the dataset, which
might be a limitation. In the future, we will perform a clinical user study to
evaluate our results further. Also, we would like to consider three practical
extensions: (i) enhance the model by utilizing relations between medications,
e.g., similarities; (ii) consider medicine dosages during recommendation;
(iii) penalize different medications based on patient-specific conditions.

## References

  * Almirall et al. [2012] Daniel Almirall, Scott N Compton, Meredith Gunlicks-Stoessel, Naihua Duan, and Susan A Murphy.  Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy.  Statistics in medicine, 31(17):1887–1902, 2012.
  * Bajor and Lasko [2017] Jacek M Bajor and Thomas A Lasko.  Predicting medications from diagnostic codes with recurrent neural networks.  In International Conference on Learning Representation, 2017.
  * Bhoi et al. [2020] Suman Bhoi, Lee Mong Li, and Wynne Hsu.  Premier: Personalized recommendation for medical prescriptions from electronic records.  arXiv preprint arXiv:2008.13569, 2020.
  * Chen et al. [2016] Zhuo Chen, Kyle Marple, Elmer Salazar, Gopal Gupta, and Lakshman Tamil.  A physician advisory system for chronic heart failure management based on knowledge patterns.  Theory and Practice of Logic Programming, 16(5-6):604–618, 2016\.
  * Choi et al. [2016] Edward Choi, Mohammad Taha Bahadori, Jimeng Sun, Joshua Kulas, Andy Schuetz, and Walter Stewart.  Retain: An interpretable predictive model for healthcare using reverse time attention mechanism.  In Advances in Neural Information Processing Systems, pages 3504–3512, 2016.
  * Choi et al. [2017] Edward Choi, Andy Schuetz, Walter F Stewart, and Jimeng Sun.  Using recurrent neural network models for early detection of heart failure onset.  Journal of the American Medical Informatics Association, 24(2):361–370, 2017.
  * Johnson et al. [2016] Alistair EW Johnson, Tom J Pollard, Lu Shen, H Lehman Li-Wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark.  Mimic-iii, a freely accessible critical care database.  Scientific data, 3(1):1–9, 2016.
  * Killian et al. [2019] Jackson A Killian, Bryan Wilder, Amit Sharma, Vinod Choudhary, Bistra Dilkina, and Milind Tambe.  Learning to prescribe interventions for tuberculosis patients using digital adherence data.  In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2430–2438, 2019.
  * Lakkaraju and Rudin [2017] Himabindu Lakkaraju and Cynthia Rudin.  Learning cost-effective and interpretable treatment regimes.  In Artificial Intelligence and Statistics, pages 166–175, 2017\.
  * Le et al. [2018] Hung Le, Truyen Tran, and Svetha Venkatesh.  Dual memory neural computer for asynchronous two-view sequential learning.  In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1637–1645, 2018.
  * Mao et al. [2019] Chengsheng Mao, Liang Yao, and Yuan Luo.  Medgcn: Graph convolutional networks for multiple medical tasks.  arXiv preprint arXiv:1904.00326, 2019.
  * Percha and Altman [2013] Bethany Percha and Russ B Altman.  Informatics confronts drug–drug interactions.  Trends in pharmacological sciences, 34(3):178–184, 2013.
  * Shang et al. [2019a] Junyuan Shang, Tengfei Ma, Cao Xiao, and Jimeng Sun.  Pre-training of graph augmented transformers for medication recommendation.  arXiv preprint arXiv:1906.00346, 2019.
  * Shang et al. [2019b] Junyuan Shang, Cao Xiao, Tengfei Ma, Hongyan Li, and Jimeng Sun.  Gamenet: Graph augmented memory networks for recommending medication combination.  In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1126–1133, 2019.
  * Tatonetti et al. [2012] Nicholas P Tatonetti, P Ye Patrick, Roxana Daneshjou, and Russ B Altman.  Data-driven prediction of drug effects and interactions.  Science translational medicine, 4(125):125ra31–125ra31, 2012.
  * Wang et al. [2017] Meng Wang, Mengyue Liu, Jun Liu, Sen Wang, Guodong Long, and Buyue Qian.  Safe medicine recommendation via medical knowledge graph embedding.  arXiv preprint arXiv:1710.05980, 2017.
  * Wang et al. [2018] Lu Wang, Wei Zhang, Xiaofeng He, and Hongyuan Zha.  Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation.  In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2447–2456, 2018.
  * Wang et al. [2019] Shanshan Wang, Pengjie Ren, Zhumin Chen, Zhaochun Ren, Jun Ma, and Maarten de Rijke.  Order-free medicine combination prediction with graph convolutional reinforcement learning.  In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1623–1632, 2019.
  * Xiao et al. [2018] Cao Xiao, Edward Choi, and Jimeng Sun.  Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review.  Journal of the American Medical Informatics Association, 25(10):1419–1428, 2018.
  * Zhang et al. [2017] Yutao Zhang, Robert Chen, Jie Tang, Walter F Stewart, and Jimeng Sun.  Leap: Learning to prescribe effective and safe treatment combinations for multimorbidity.  In proceedings of the 23rd ACM SIGKDD international conference on knowledge Discovery and data Mining, pages 1315–1324, 2017.

##  Appendix A Momentum-based Loss (MBL) Functions

As stated in the main text, the weights for different loss,
$\lambda_{i},~{}i=1,2,3,4$, are treated as hyperparameters. To automate the
procedure of hyperparameter search, we design momentum-based weights, which
are adjusted adaptively during the training process. This module is primarily
proposed for automatic selection of four hyperparameters, not for performance
improvement.

Momentum-based Weights. The key advantage of using momentum is to combine
current exploration with historical information. For each training epoch, we
maintain a mean value for each type of loss, $\bar{L}_{i}^{(k)}$, which is
calculated over the first $k$ training samples ($i=1,2,3,4$ stand for rec,
ddi, bce, multi parts, separately). We use $L_{i}^{(k)}$ to denote the actual
loss at the $k_{th}$ training point, and the momentum-based weights,
$\lambda_{i}^{(k)}$, are calculated by the relative difference from the mean,

$\displaystyle\vspace{-1mm}\mbox{diff}_{i}^{(k)}$
$\displaystyle=\frac{L_{i}^{(k)}-\bar{L}_{i}^{(k-1)}}{\bar{L}_{i}^{(k-1)}},$
(15) $\displaystyle\vspace{-1mm}\lambda_{i}^{(k)}$
$\displaystyle=\gamma\cdot\frac{exp(\mbox{diff}_{i}^{(k)})}{\sum_{i}exp(\mbox{diff}_{i}^{(k)})}+(1-\gamma)\cdot\lambda_{i}^{(k-1)},\vspace{-1mm}$
(16)

where we use the same $\gamma$ (in Eqn. (10)) to balance the current
exploration and historical inertia. Meanwhile, the momentum values are also
updated by,

$\footnotesize\bar{L}_{i}^{(k)}=\frac{L_{i}^{(k)}+(k-1)\cdot\bar{L}_{i}^{(k-1)}}{k}.$
(17)

The momentum weights are in the same spirit as the negative control. The
momentum weight would enable our model to automatically identify and focus on
the under-optimized parts in the loss as well as utilizing historical loss
information. It empirically provides stable results. Note that in model
implementation, we further employ a DDI threshold, $\eta\in(0,1)$. If the
result DDI rate of the $k_{th}$ sample is below the threshold, we will
directly set $\lambda_{2}^{(k)}=0$. We provide an ablation study on the
momentum-based loss in Table 4.

Table 4: Ablation Study on Momentum-based Loss (for MIMIC-III / DS2) Metrics | MICRON |  MICRON with MBL
---|---|---
DDI | 0.0695 / 0.0143 | 0.0690 / 0.0147
Jaccard | 0.5234 / 0.3634 | 0.5252 / 0.3479
F1 | 0.6778 / 0.4544 | 0.6807 / 0.4536
Err(add) | 6.090 / 2.088 | 6.255 / 2.115
Err(remove) | 5.853 / 1.213 | 5.623 / 1.157

##  Appendix B Smart Inference

This module will potentially improve the efficiency and interpretation of our
model. Our model can reflect when each medicine is added or dropped and why
(based on the appearance or disappearance of what diagnoses or procedures).
Typically, during the inference phase, $\mathbf{r}^{(t)}$ is calculated by

$\displaystyle\mathbf{r}^{(t)}$
$\displaystyle=\mathbf{h}^{(t)}-\mathbf{h}^{(t-1)}$
$\displaystyle{=\mbox{NET}_{health}\left(\left[\mathbf{d}^{(t)}_{e}~{}\|~{}\mathbf{p}^{(t)}_{e}\right]\right)-\mbox{NET}_{health}\left(\left[\mathbf{d}^{(t-1)}_{e}~{}\|~{}\mathbf{p}^{(t-1)}_{e}\right]\right)}.\vspace{-2mm}$
(18)

However, we can trace the cause back to the very beginning if
$\mbox{NET}_{health}$ is affine,

$\mbox{NET}_{health}(\mathbf{x}-\mathbf{y})=\mbox{NET}_{health}(\mathbf{x})-\mbox{NET}_{health}(\mathbf{y}),~{}\forall\mathbf{x},\mathbf{y}$

where Eqn. (18) could be written as

$\displaystyle\mathbf{r}^{(t)}$
$\displaystyle={\mbox{NET}_{health}\left(\left[\mathbf{d}^{(t)}_{e}-\mathbf{d}^{(t-1)}_{e}~{}\|~{}\mathbf{p}^{(t)}_{e}-\mathbf{p}^{(t-1)}_{e}\right]\right)}$
$\displaystyle={\mbox{NET}_{health}\left(\left[(\mathbf{d}^{(t)}-\mathbf{d}^{(t-1)})\cdot\mathbf{E}_{d}~{}\|~{}(\mathbf{p}^{(t)}-\mathbf{p}^{(t-1)})\cdot\mathbf{E}_{p}\right]\right)}.$
(19)

In fact, this equation reveals the causality between the update in diagnosis
and procedure measurements and the update in patient health representation.
For example, a patient get extra X-ray scanning at this visit while other
diagnosis and procedure measurements are the same as the last time. Then, the
model input is barely the code of X-ray, and the input will hopefully
stimulate some of the medicines while discourage others, based solely on the
X-ray information.

Eqn. (19) is practically efficient, since the clinical update
$\mathbf{d}^{(t)}-\mathbf{d}^{(t-1)}$ (or
$\mathbf{p}^{(t)}-\mathbf{p}^{(t-1)}$) is much sparser than original diagnosis
documentary $\mathbf{d}^{(t-1)}$ and $\mathbf{d}^{(t)}$ (or procedure
documentary $\mathbf{p}^{(t-1)}$ and $\mathbf{p}^{(t)}$). Also, it only
requires one-time computation of $\mbox{NET}_{health}$ for the residual health
representation.

##  Appendix C Datasets and Hyperparameter Settings

Here are the details about two datasets used in the paper.

  * •

(i) MIMIC-III Johnson et al. [2016] is a benchmark inpatient dataset, which
collects electronic ICU records of Beth Israel Deaconess Medical Center
between 2001 and 2012. We utilize the diagnosis, procedure, medication data
and filter out patients with only one visit, so many patients will have 2 and
more visits (up to 29 visits) in this data. From the original files in MIMIC-
III “DIAGNOSES_ICD.csv”, “PROCEDURES_ICD.csv” and “PRESCRIPTIONS.csv”, we
extract diagnosis, procedure and medication code list, separately, then. These
three files are then merged by the patient id and “HADM_ID”, which is also
used as visit id. After the merging, diagnosis and procedure are ICD-9 coded.

  * •

(ii) IQVIA PharMetrics Plus is a private outpatient dataset. Patients in this
database is generally representative of the under 65 commercially insured
population in the U.S., who receives treatment without being admitted to a
hospital from 2015 to 2019. We keep visit records with at least $3$
medications. In this dataset, patients usually have around 10 visits. Similar
to Figure 1 in the main text, we provide a Jaccard coefficient distribution
for IQVIA dataset in Figure 4. Again, we observe that medications have larger
overlaps than diagnosis.

Figure 4: Frequency Histogram of Jaccard Coefficients on IQVIA.

The experimental settings are largely adopted from GAMENet Shang et al.
[2019b] with minor adjustments. For both of the datasets, we extract the DDI
information from Top-40 severity types from TWOSIDES Tatonetti et al. [2012].
We use ATC-3 Level drug coding to process all medications, use ICD-9 codes for
diagnosis and procedures in MIMIC-III, and for IQVIA, we use ICD-10-CM codes
for diagnoses and CPT/HCPCS codes for procedures. In MIMIC, “visit” is
identified by the field “HADM_ID” in raw data file. When running the
experiments, we use multi-hot encodings for the diagnosis and procedure
vectors.

Two datasets are randomly split into training, validation and test by
$60\%:20\%:20\%$. For MIMIC-III, we set embedding size $s=64$ for diagnosis
and procedure tables, $\mathbf{E}_{d},\mathbf{E}_{p}$. The health
representation network, $\mbox{NET}_{health}$, is an affine transformation, we
implement it as one linear layer without activation function. The prescription
network, $\mbox{NET}_{med}$, is a two-layer feed-forward neural network with
$256$ hidden units. The hyperparameters, $\gamma=0.75,\eta=0.08$, are selected
from validation set. We choose RMSprop as the optimizer with learning rate at
$2e^{-4}$ and weight decay at $1e^{-5}$. For IQVIA, we set $s=32$, choose a
three-layer $\mbox{NET}_{med}$ with $128$ hidden units, the hyperparameters
$\gamma=0.75,\eta=0.03$, while the learning rate is $5e^{-4}$. For two
datasets, the hyperparameters for the loss function are
$\lambda_{i}=0.25,~{}i=1,2,3,4$.

The implementation for baseline methods can be referred to
RETAIN111https://github.com/mp2893/retain, LEAP and
GAMENet222https://github.com/sjy1203/GAMENet. For MIMIC-III dataset, the
hyperparameters of the baselines are mainly selected by grid search.

  * •

For RETAIN, We choose two 64-dim GRUs as the implementation of two-level RNN
with dropout rate at 0.5 for the output embedding. RMSprop is used as the
optimizer with learning rate at $5\times 10^{-4}$ for RETAIN and other deep
learning baselines.

  * •

LEAP is also implemented by 64-dim GRU as the feature encoder. However, we
choose droput rate at 0.3 between two layers, as 0.3 works better than 0.5 in
the validation set. Particularly, since LEAP are sequence-based models, we set
20 as the max drug combination size.

  * •

For GAMENet, we use the same suit of hyperparameters reported from the
original paper Shang et al. [2019b]: expected DDI rate at 0.05, initial
annealing temperature at 0.85, mixture weights $\pi=[0.9,0.1]$ and also use
64-dim embedding tables and 64-dim GRU as RNN. We find the hyperparameter
works well in validation set, so we keep it for test.

For IQVIA, the embedding size and hidden layers are reduced to 32-dim. In both
of the datasets, we could only observe a window of patient’s visits, instead
of a complete visit history. Since the evaluation is on medication change
prediction, we assume the earliest visit appeared in the window as the “first”
visit of a patient $j$, and the evaluation of all models starts from the
“second” visit. All experiments are implemented by PyTorch 1.4.0 on a Ubuntu
server with 64GB memory, 32 CPUs and two GTX 2080Ti GPU. The reported results
are averaged over 5 independent experiments with different random seeds.

##  Appendix D Details about Metrics

This section provides the definition of each metric used in the experiment
section.

We already present the calculation of Err(add) in the main text, in this
section, we show the calculation of Err(remove), DDI, Jaccard coefficient
(Jaccard), F1-score (F1). All the metrics are calculated from the observed
“second” visits.

#### Err(remove).

The Err(remove) metrics for a particular patient $j$ is calculated by

$\displaystyle\small\mathcal{O}_{target,j}^{(t)}$
$\displaystyle=\tilde{\mathcal{M}}_{j}^{(t-1)}\setminus\mathcal{M}^{(t)}_{j}$
$\displaystyle\mbox{Err(remove)}_{j}$
$\displaystyle=\frac{1}{V(j)}\sum_{t=2}^{V(j)}|\mathcal{O}_{target,j}^{(t)}\setminus\mathcal{O}_{j}^{(t)}|+|\mathcal{O}_{j}^{(t)}\setminus\mathcal{O}_{target,j}^{(t)}|$

where $\mathcal{O}_{target}^{(t)}$ is the target removal set,
$\mathcal{N}^{(t)}$ is the predicted removal set,
$\mathcal{A}\setminus\mathcal{B}$ is set minus operation, $V(j)$ means total
number of visits of patient $j$, $|\cdot|$ is for set cardinality,
$|\mathcal{O}_{target,j}^{(t)}\setminus\mathcal{O}_{j}^{(t)}|$ and
$|\mathcal{O}_{j}^{(t)}\setminus\mathcal{O}_{target,j}^{(t)}|$ are false
negative and false positive numbers in terms of removal for the $t_{th}$
visit. The final Err(remove) is calculated by taking the average of all
patients.

#### DDI rate.

This metric is used to reflect the safety of drug combinations. As is stated
in the main text, $\mathcal{M}_{j}^{(t)}$ is the target medication set and
$\tilde{\mathcal{M}}_{j}^{(t)}$ is the estimated medication set for patient
$j$ at the $t_{th}$ visit. The DDI rate for patient $j$ is calculated by,

$DDI_{j}=\frac{\sum^{V(j)}_{t=2}\sum_{m,n\in\tilde{\mathcal{M}}_{j}^{(t)}}\mathbf{1}\\{\mathbf{A}_{mn}=1\\}}{\sum^{V(j)}_{t=2}\sum_{m,n\in\tilde{\mathcal{M}}_{j}^{(t)}}1},$

where $V(j)$ is the total number of visits of the $j$-th patient, $\mathbf{A}$
is the DDI matrix and $\mathbf{1}\\{\cdots\\}$ is an indicator function, which
returns 1 when the expression in $\\{\cdots\\}$ is true, otherwise 0. The
reported DDI rate is by taking average of all patients.

#### Jaccard.

The Jaccard coefficient for patient $j$ is calculated by

$Jaccard_{j}=\frac{1}{V(j)}\sum_{t=2}^{V(j)}\frac{|\tilde{\mathcal{M}}_{j}^{(t)}\cap{\mathcal{M}}_{j}^{(t)}|}{|\tilde{\mathcal{M}}_{j}^{(t)}\cup{\mathcal{M}}_{j}^{(t)}|},$

while the overall Jaccard coefficient of the test data is by taking further
average over all patients. In the main text, we report only the overall
Jaccard coefficient.

#### F1.

The F1 score is calculated by the harmonic mean of precision and recall. For
patient $j$ at the $t$-th visit, the Precision, Recall, F1 are calculated by

$Precision_{j}^{(t)}=\frac{|\tilde{\mathcal{M}}_{j}^{(t)}\cap{\mathcal{M}}_{j}^{(t)}|}{|\tilde{\mathcal{M}}_{j}^{(t)}|}$
$Recall_{j}^{(t)}=\frac{|\tilde{\mathcal{M}}_{j}^{(t)}\cap{\mathcal{M}}_{j}^{(t)}|}{|{\mathcal{M}}_{j}^{(t)}|}$
$F1_{j}^{(t)}=\frac{2}{\frac{1}{Precision_{j}^{(t)}}+\frac{1}{Recall_{j}^{(t)}}}$

The F1 score for one patient $j$ is by taking the average over his visits,

$F1_{j}=\frac{1}{V(j)}\sum_{t=2}^{V(j)}F1_{j}^{(t)},$

while the overall F1-score of the test data is by taking further average over
all patients. In the main text, we report only the overall F1-score.

##  Appendix E Standard Deviation and $p$-value

We provide the standard deviation and $p$-value results for performance
comparison in MIMIC-III (Table 5) and IQVIA (Table 6).

Table 5: Performance Comparison on MIMIC-III

Metrics | DDI | Jaccard | F1 | Err(add) | Err(remove)
---|---|---|---|---|---
SimNN | 0.0837 $\pm$ 0.0005 (2.5518e-11) | 0.4658 $\pm$ 0.0004 (6.6613e-16) | 0.6235 $\pm$ 0.0004 (4.4408e-16) | 6.856 $\pm$ 0.0054 (9.3702e-14) | 7.329 $\pm$ 0.0123 (1.5543e-15)
DualNN | 0.0925 $\pm$ 0.0006 (6.9011e-13) | 0.4880 $\pm$ 0.0004 (8.0380e-14) | 0.6447 $\pm$ 0.0004 (5.5733e-14) | 6.261 $\pm$ 0.0119 (7.6331e-07) | 7.987 $\pm$ 0.0112 (0.0)
RETAIN | 0.0932 $\pm$ 0.0016 (1.8352e-09) | 0.4796 $\pm$ 0.0006 (2.9309e-14) | 0.6389 $\pm$ 0.0005 (2.1316e-14) | 8.552 $\pm$ 0.0396 (2.4424e-15) | 6.338 $\pm$ 0.0369 (1.8950e-08)
LEAP | 0.0880 $\pm$ 0.0028 (3.2323e-06) | 0.4331 $\pm$ 0.0011 (4.4408e-16) | 0.5953 $\pm$ 0.0010 (4.4408e-16) | 9.105 $\pm$ 0.0356 (2.2204e-16) | 5.939 $\pm$ 0.0335 (0.0124)
GAMENet | 0.0928 $\pm$ 0.0007 (1.8409e-12) | 0.4980 $\pm$ 0.0012 (2.4894e-10) | 0.6549 $\pm$ 0.0011 (2.5575e-10) | 8.810 $\pm$ 0.1021 (4.8889e-12) | 5.854 $\pm$ 0.0450 (0.9780)
MICRON | 0.0695 $\pm$ 0.0004 | 0.5234 $\pm$ 0.0008 | 0.6778 $\pm$ 0.0007 | 6.090 $\pm$ 0.0189 | 5.853 $\pm$ 0.0219

* $p$-values are at the parenthesis.

Table 6: Performance Comparison on IQVIA

Metrics | DDI | Jaccard | F1 | Err(add) | Err(remove)
---|---|---|---|---|---
SimNN | 0.0152 $\pm$ 0.0002 (0.0054) | 0.192 $\pm$ 0.0003 (0.0) | 0.2383 $\pm$ 0.0005 (0.0) | 2.906 $\pm$ 0.0031 (0.0) | 1.679 $\pm$ 0.0056 (1.0043e-12)
DualNN | 0.0156 $\pm$ 0.0003 (0.0015) | 0.112 $\pm$ 0.0003 (0.0) | 0.1783 $\pm$ 0.0003 (0.0) | 3.406 $\pm$ 0.0069 (0.0) | 2.579 $\pm$ 0.0051 (0.0)
RETAIN | 0.0279 $\pm$ 0.0008 (6.7339e-10) | 0.332 $\pm$ 0.0002 (1.5543e-15) | 0.4215 $\pm$ 0.0003 (4.4408e-16) | 2.353 $\pm$ 0.0229 (3.7266e-08) | 0.927 $\pm$ 0.0169 (better)
LEAP | 0.0134 $\pm$ 0.0013 (better) | 0.1871 $\pm$ 0.0007 (0.0) | 0.2742 $\pm$ 0.0008 (0.0) | 3.663 $\pm$ 0.0205 (2.2204e-16) | 1.286 $\pm$ 0.0153 (0.0005)
GAMENet | 0.0166 $\pm$ 0.0003 (1.7060e-05) | 0.2025 $\pm$ 0.0008 (0.0) | 0.3016 $\pm$ 0.0007 (0.0) | 3.016 $\pm$ 0.0589 (8.6569e-10) | 2.179 $\pm$ 0.0205 (9.7255e-14)
MICRON | 0.0143 $\pm$ 0.0003 | 0.3634 $\pm$ 0.0005 | 0.4544 $\pm$ 0.0004 | 2.088 $\pm$ 0.0104 | 1.213 $\pm$ 0.0141

* $p$-values are at the parenthesis.
* In this dataset, for Err(remove) metric, RETAIN is better than our MICRON. For DDI, LEAP is better than our MICRON.