supplementary

# ODD: A Benchmark Dataset for the NLP-based
Opioid Related Aberrant Behavior Detection

Sunjae Kwon1, Xun Wang2, Weisong Liu3, Emily Druhl4, Minhee L. Sung5,
Joel I. Reisman4, Wenjun Li3, Robert D. Kerns5, William Becker5, Hong
Yu1,3,4,6
1UMass Amherst, 2Microsoft, 3UMass Lowell, 4U.S. Department of Veteran
Affairs,
5Yale University, 6UMass Chan Medical School
<EMAIL_ADDRESS><EMAIL_ADDRESS>
{weisong_liu, wenjun_li<EMAIL_ADDRESS>{emily.druhl<EMAIL_ADDRESS>
{minhee.sung, robert.kerns<EMAIL_ADDRESS>

###### Abstract

Opioid related aberrant behaviors (ORAB) present novel risk factors for opioid
overdose. Previously, ORAB have been mainly assessed by survey results and by
monitoring drug administrations. Such methods however, cannot scale up and do
not cover the entire spectrum of aberrant behaviors. On the other hand, ORAB
are widely documented in electronic health record notes. This paper introduces
a novel biomedical natural language processing benchmark dataset named ODD,
for ORAB Detection Dataset. ODD is an expert-annotated dataset comprising of
more than 750 publicly available EHR notes. ODD has been designed to identify
ORAB from patients’ EHR notes and classify them into nine categories; 1)
Confirmed Aberrant Behavior, 2) Suggested Aberrant Behavior, 3) Opioids, 4)
Indication, 5) Diagnosed opioid dependency, 6) Benzodiapines, 7) Medication
Changes, 8) Central Nervous System-related, and 9) Social Determinants of
Health. We explored two state-of-the-art natural language processing (NLP)
models (finetuning pretrained language models and prompt-tuning approaches) to
identify ORAB. Experimental results show that the prompt-tuning models
outperformed the finetuning models in most cateogories and the gains were
especially higher among uncommon categories (Suggested aberrant behavior,
Diagnosed opioid dependency and Medication change). Although the best model
achieved the highest 83.92% on area under precision recall curve, uncommon
classes (Suggested Aberrant Behavior, Diagnosed Opioid Dependence, and
Medication Change) still have a large room for performance improvement.

##  1 Introduction

The opioid overdose (OOD) crisis has had a striking impact on the United
States, not only threatening citizens’ health [3] but also bringing about a
substantial financial burden [12]. According to a report by the Centers for
Disease Control and Prevention [7], OOD accounted for 110,236 deaths in a
single year in 2022. In addition, fatal OOD and opioid use disorder (OUD) cost
the United States $1.04 trillion in 2017 and that figure rose sharply to $1.5
trillion in 2021 [5]. Identifying patients at risk of OOD could help prevent
serious consequences [25].

Opioid-related aberrant behaviors (ORABs) are patient behaviors that may
indicate prescription medication abuse [11]. ORABs can be categorized into
confirmed aberrant behavior and suggested aberrant behavior [30, 20, 27].
Herein, confirmed aberrant behaviors have a clear evidence of medication abuse
and addiction while suggested aberrant behaviors do not have a clear evidence
[27]. Table 1 presents examples of such categories. Since ORABs have shown to
be associated with patients with drug abuse problems [42], assessment of ORABs
has been recognized as beneficial in evaluating the risk associated with
opioid abuse [26] and OOD [39].

Table 1: ORAB Examples [27] ORAB Type | Example
---|---
Confirmed Aberrant Behavior | Resistance to changing medications
Falsification of prescription—forgery or alteration
Injecting medications meant for oral use
Suggested Aberrant Behavior | Asking for or even demanding, more medication
Asking for specific medications
Reluctance to decrease opioid dosing once stable

Previously, ORABs have been detected by monitoring opioid administration
(e.g., frequency and dosage) [31] or self-reported questionnaires [1, 42].
However such measurements do not include the full spectrum of ORABs (e.g.,
medication sharing, denying medication changing). In addition, patients can
obtain opioids from multiple resources (e.g. illegal purchase and medication
sharing), which are not captured in the structured data. It has been known
that ORABs are widely described in EHR notes and natural language processing
(NLP) techniques can be used to identify ORABs Lingeman et al. [23]. However,
the previous study relied on a small amount of annotated notes, which were not
publicly available. Moreover, the previous work only considered ORABs as a
binary classification (present or not) and only explored traditional machine
learning models (e.g., support vector machine (SVM)).

This paper proposes ORAB detection that is a novel Biomedical NLP (BioNLP)
task. We also introduce an ORAB Detection Dataset (ODD) which is large-size,
expert-annotated, and multi-label classification benchmark dataset
corresponding to the task. For this, we first designed a robust and
comprehensive annotation guideline that labels text into nine categories which
encompass two types of ORABs (Confirmed Aberrant Behavior and Suggested
Aberrant Behavior) and seven types of auxiliary opioid-related information
(Opioids, Indication, Diagnosed Opioid Dependency, Benzodiapines, Medication
Change, Central Nervous System Related, Social Determinant of Health). Using
the guideline, domain experts annotated 750 EHR notes of 500 opioid-treated
patients extracted from MIMIC-IV database [15]. Overall, we annotated 2,519
instances with 157 ORABs instances (113 for Confirmed Aberrant Behavior and 44
for Suggested Aberrant Behavior).

Experiments conducted on two ORAB detection models based on state-of-the-art
(SOTA) natural language processing (NLP) models; traditional finetuning [9]
and prompt-based tuning [41] approaches. The experimental results on MIMIC
showed that prompt-based tuning models surpass finetuning models in almost all
categories (eight out of nine). When the numbers of instances were less than
100 ($uncommon\leavevmode\nobreak\ categories$: Suggest Aberrant Behavior,
Diagnosed Opioid Dependency, and Medication Change), the performance
improvement was greater, in particular, the Medication Change and Suggest
Aberrant Behavior classes achieve performance improvements of over 20%p and
7%p respectively. Experimental codes are available at
https://github.com/soon91jae/ORAB_MIMIC. ODD will be published as soon as
possible via PhysioNet 111https://physionet.org.

The main contributions of this paper can be organized as follows:

  * •

This paper introduces a new Biomedical NLP (BioNLP) task ORAB detection for
extracting information related to a patient’s risk of opioid addiction and
abuse from EHR notes. We also curate a corresponding benchmark dataset, named
ODD, an expert-annotated dataset for the ORAB detection task.

  * •

In this paper, we present the experimental results of two state-of-the-art NLP
models as baseline performances for the benchmark dataset. Moreover, we report
comprehensive data and error analyses to guide future studies in constructing
improved models.

##  2 Related Work

##### NLP-based Opioid Abuse Analysis

Recently, with the development of NLP technology, studies have been actively
conducted to analyze information relevant to opioid abuse and OOD from text
(e.g. EHR notes, social media) [32, 6, 14, 48, 34]. Studies have explored a
broad range of NLP techniques to identify OUD [48]. Zhu et al. [48] developed
a keyword-based OUD detection model for patients who have been treated with
chronic opioid therapy. Their NLP models were able to uncover OUD cases that
would be missed using the International Classification of Diseases (ICD) codes
alone. Singleton et al. [34] proposed a multiple-phase OUD detection approach
using a combination of dictionary and rule-based approaches. Blackley et al.
[6] developed feature engineering-based machine learning models. Herein, the
authors demonstrated that the machine learning models outperformed a rule-
based one that utilizes keywords.

Other works adopted NLP to study factors associated with opioid abuse.
Goodman-Meza et al. [14] utilized text features such as term frequency–inverse
document frequency (TF-IDF), concept unique identifier (CUI) embeddings, and
word embeddings to analyze substances that contribute to opioid overdose
deaths. Sarker et al. [32] conducted a geospatial and temporal analysis of
opioid-related mentions in Twitter posts. They found a positive correlation
between the rate of opioid abuse-indicating posts and opioid misuse rates and
county-level overdose death rates.

The ORAB detection task is similar to the studies above in that it analyzes
drug abuse-related information using NLP approaches. However, different from
the previous studies that mainly depend on keywords such as drug mentioning,
the ORAB detection is a more challenging NLP task considering that it needs to
identify various and complex linguistic patterns such as trying to deceive
physicians [29] and emotional reaction on opioid prescription [23].

##### ORAB Risk Assessment and Detection

Webster and Webster [42] introduced a risk management tool that monitors ORABs
by scoring a patient’s self-reports on risk factors (history of family and
personal substance abuse, history of preadolescent sexual abuse, and
psychological illness) related to substance abuse. Then, each patient is
categorized into three risk levels (low risk, moderate risk, and high risk)
according to the sum of the scores. Other studies [33, 36, 18, 38, 31] suggest
detecting ORAB by relying on diagnostic criteria based on structured
information such as the frequency of opioid dosage, the number of opioid
prescribers, and the number of pharmacies. Although the above methodologies
can detect patients at risk of ORABs with high precision, the recall was low
[31].

The most relevant work is Lingeman et al. [23]. However, as described earlier,
Lingeman et al. [23]’s work relied on a small scaled EHR notes which is not
publicly available. In contrast, ODD consists of a larger size of EHR notes
(750 EHR notes, six times larger than Lingeman et al. [23]’s dataset of 121
EHR notes) which is publicly available. Furthermore, ODD’s annotation scheme
provides rich sub-categorized aberrant behaviors (suggested and confirmed) and
additional opioid-related information. In contrast, Lingeman et al. [23]’s
study was designed as a binary classification task to detect ORABs. Finally,
we leverage the SOTA deep learning models that the previous work Lingeman et
al. [23] did not explore.

##  3 ORAB Detection Dataset

###  3.1 Data Collection

The source of the first dataset is made up of publicly available fully de-
identified EHR notes of the MIMIC-IV database [16]. ORABs are uncommon events.
To increase the likelihood that our annotated data incorporate ORABs, we
sorted out patients at risk of opioid misuse based on repetitive opioid use
and diagnosis related to opioid misuse. Specifically, we first extracted EHR
notes mentioning opioids with the generic and brand name of opioid
medications. In addition, we selected patients diagnosed based on their ICD
codes. Detailed information on opioid medications (and their generic names),
and ICD codes utilized for filtering EHR notes are presented in Appendix A.

Figure 1: Socio-demographic statistics of the sampled patients.

Among 331,794 EHR notes of 299,712 patients in MIMIC-IV database, we found
that approximately 57% of patients were prescribed opioids during their
hospitalization. Then, we selected patients who were repeatedly prescribed
(more than twice) opioids. In addition, we chose patients who were diagnosed
with drug poisoning and drug dependence based on the ICD codes. Overall, there
are 3,904 patients who are satisfied the aforementioned conditions. Among
them, we randomly select 750 notes from a randomly sampled 500 patients for
annotation. Figure 1 presents the socio-demographic statistics of the sampled
patients. Herein, we can notice that the number of male and female is
balanced.

###  3.2 Data Annotation

Table 2: The definitions and examples of the categories of ODD. Category | Definition | Example
---|---|---
Confirmed Aberrant Behavior |  Evidence confirming the loss of control of opioid use, specifically aberrant usage of opioid medications. |  [Patient] admits that he has been sharing his Percocet with his wife, and that is why he has run out early.
Suggested Aberrant Behavior |  Evidence suggesting loss of control of opioid use or compulsive/inappropriate use of opioids. |  [Patient] states that ‘that [drug] won’t work; only [X drug] will and I won’t take any other’
Opioids |  The mention or listing of the name(s) of the opioid medication(s) that the patient is currently prescribed or has just been newly prescribed. |  Oxycodone has been known to make [the patient] sleepy at 5 mg.
Indication |  Patients are using opioids under instructions. |  [The patient] is in a daze.
Diagnosed Opioid Dependency |  Patients have the condition of being dependent on opioids, have chronic opioid use, or is undergoing opioid titration |  [The patient] is in severe pain and has been taking [opioid drug] for [time].[HY1]
Benzodiazepines |  Patients are co-prescribed benzodiazepines. |  Valium has been listed in patient medication list.
Medicine Changes |  Change in opioid medicine, dosage, and prescription since the last visit. |  [Patient] reports that his previous PCP just recently changed his pain regimen, adding oxycodone.
Central Nervous System Related |  CNS-related terms/terms suggesting altered sensorium. |  [Patient] reported to have nausea after taking [drug].
Social Determinants of Health |  The nonmedical factors that influence health outcomes |  [Patient] divorced a years ago.

We first designed an annotation guideline in Section 3.2.1 to label each
sentence in an EHR note into nine categories as shown in Table 2. Herein, the
categories contain two ORABs (confirmed aberrant behavior and suggested
aberrant behavior) and seven additional information that relevant to opioid
usage.

EHR notes were annotated independently by two domain experts by following the
annotation guidelines. Herein, the primary annotator annotated all EHR notes
with eHOST [10] annotation tool. The other annotator coded 100 of the EHRs of
the primary annotator with the same environment to compute inter-rater
reliability with Cohen’s kappa [40]. As a result, the inter-rater reliability
shows strong agreement ($\kappa=0.87$) between the annotators.

####  3.2.1 Annotation Guideline

##### Confirmed aberrant behavior:

This class refers to behavior that is more likely to lead to a catastrophic
adverse event. It is defined as evidence confirming the loss of control of
opioid use, specifically aberrant usage of opioid medications, including: 1)
Aberrant use of opioids, such as administration/consumption in a way other
than described or self-escalating doses. 2) Evidence suggesting or proving
that the patient has been selling or giving away opioids to others, including
family members. 3) Use of opioids for a different indication other than the
indication intended by the prescriber. 4) Phrases suggesting current use of
illicit or illicitly obtained substances or misuse of legal substances (e.g.
alcohol) other than prescription opioid medications.

##### Suggested aberrant behavior:

This class refers to behavior implying patient distress related to their
opioid treatment. SAB includes three kinds of behavior that suggest potential
misuse of opioids. 1) Patient attempt to get extra opioid medicine like
requesting for early refill, asking for increasing dosage, or reporting
missing/stolen opioid medication. 2) Patient emotions toward opioids like a
request for a certain opioid medication use/change/increase. 3) Physician
concerns.

##### Opioids:

This class refers to the mention or listing of the name(s) of the opioid
medication(s) that the patient is currently prescribed or has just been newly
prescribed.

##### Indication:

This class indicates that patients are using opioids under instructions, such
as using opioids for pain, for treatment of opioid use disorder, etc.

##### Diagnosed Opioid Dependence:

It refers to patients who have the condition of being dependent on opioids,
has chronic opioid use, or are undergoing opioid titration.

##### Benzodiazepines:

This class refers to co-prescribed benzodiazepines (a risk factor for
accidental opioid overdose [37]). In this case, the patient is simply being
co-prescribed benzodiazepines (with no noted evidence of abuse).

##### Medication Change:

This class indicates that the physician makes changes to the patient’s opioid
regimen during this current encounter or that the patient’s opioid regimen has
been changed since the patient’s last encounter with the provider writing the
note.

##### Central Nervous System Related:

This is defined as central nervous system-related terms or terms suggesting
altered sensorium, including cognitive impairment, sedation, lightheadedness,
intoxication, and general term suggesting altered sensorium (e.g. “altered
mental status”).

##### Social Determinants of Health:

This class refers to the factors in the surroundings which impact their well-
being. Our dataset captured the following attributes:

  * •

Marital status (single, married …)

  * •

Cohabitation status (live alone, lives with others …)

  * •

Educational level (college degree, high school diploma, no high school diploma
…)

  * •

Socioeconomic status (retired, disabled, pension, working …)

  * •

Homelessness (past, present …)

Table 3: The distribution of categories of the annotated data. Categories | Instances
---|---
Confirmed Aberrant Behavior | 113 (4.24%)
Suggested Aberrant Behavior | 44 (1.65%)
Opioids | 1,232 (46.18%)
Indication | 212 (7.95%)
Diagnosed Opioid Dependency | 60 (2.25%)
Benzodiazepines | 248 (9.30%)
Medication Change | 64 (2.49%)
Central Nervous System Related | 546 (20.46%)
Social Determinants of Health | 149 (5.58%)
Total | 2,519 (100%)

###  3.3 Annotation Statistics

Table 3 shows the statistics of the annotated categories and instances.
Herein, MIMIC dataset consist of 2,519 instances annotated from the EHRs.
Especially, we can notice that ‘confirmed aberrant behavior’ and ‘suggested
aberrant behavior’ in EHRs are relatively rare events only accounting for 157
(5.89%); 113 (4.24%) for confirmed aberrant behavior and 44 (1.65%) for
suggested aberrant behavior. The ‘Opioid’ and ‘CNS-related’ are majority
classes accounting for over 77% of overall instances while the other
categories are less than 10% each.

##  4 Task Definition and Evaluation Criterion

##### Task Definition

We can formulate ORAB detection as a multi-label classification task that
identifies whether an input text contains ORABs (Confirmed, and Suggested
aberrant behaviors) and information relevant to opioid usage. This is because
all labels can co-occurred together in a sentence.

##### Evaluation Criterion

Previous study on NLP-based ORAB detection [23] utilizes accuracy as an
evaluation criterion. However, since the labels in the dataset are highly
imbalanced (in Table 3), the accuracy may mislead performance on rare labels
since it can overestimate true negative cases [4]. Thus, as a main evaluation
criterion, we adopt the Area Under Precision-Recall Curve (AUPRC) that have
widely utilized for the performance evaluation of the binary classifiers on
highly biased labels [28].

##  5 ORAB Detection Models

Figure 2: The figures illustrate the conceptual architectures of our ORAB
detection models. (a) demonstrates a finetuning model and (b) depicts a
promtuning model. Herein, x, y, and p indicate input text, output labels, and
prompt text respectively. $\mathbf{h_{i}}$ is the hidden vector representation
of the $i^{th}$ input token. EHR text input to ‘{text placeholder}’. The name
of each category ($c_{1...n}$) in Table 2 is input at ‘{$c_{1...n}$
placeholder}’.

This section demonstrates pretrained Language Model (LM) based ORAB detections
models; traditional fine-tuning model [47] and prompt-tuning model. The
prompt-based finetuning model has shown advantages in rare category
classification (e.g. zero-shot or few-shot classification) [46]. Figure 2
demonstrates illustrative concepts of the ORAB detection models.

###  5.1 Finetuning Models

The most common way to construct classification models using a pretrained
language model (LM) is to employ finetuning, as illustrated in Figure 2(a). In
this approach, the input text $\mathbf{x}$ is passed through the fine-tuning
model. The hidden representation vector of the first token ‘[CLS]’
($\mathbf{h_{0}}$) is then used as input for the classifier. Here, $W_{c}$ and
$\mathbf{b_{c}}$ represent the weight matrix and bias, respectively. The
classifier calculates the probability distribution over output labels
$\mathbf{y}$ using the following formula:

$P(\mathbf{y}|\mathbf{x})=sigmoid(W_{c}\cdot\mathbf{h_{0}}+\mathbf{b_{c}})$

###  5.2 Prompt-based Finetuning Models

Although finetuning on pretrained LMs has been successfully applied to most of
NLP tasks [9], it is still known that finetuning still requires considerable
annotated examples to achieve a high performance [41, 46]. Thus, uncommon
categories in ODD may be a performance bottleneck.

The widely recognized technique of prompt-based finetuning, as demonstrated in
studies by Gao et al. [13] and Yang et al. [45], utilizes a template to
transform a downstream task into a language modeling problem by incorporating
masked language modeling and a predefined set of label words, effectively
enabling effective few-shot learning capabilities.

In this study, we utilize the full name of each class to curate the prompt
text $p$. Specifically, the prompts for each class are arranged in the same
order as Table 1, following the template “[$c_{i}$ placeholder]? [MASK]” where
$c_{i}$ represents the name of the $i^{th}$ class. The prompt text is then
concatenated with $\mathbf{x}$, distinguished by a separator token “[SEP],”
and fed into a prompt-based tuning model.

Next, we calculate the probability that the language model (LM) output of the
masked token corresponding to each class would be a positive word or a
negative word. Following the approach of Gao et al. [13], we define the
positive word as ‘yes’ and the negative word as ‘no’. Thus, the probability of
‘yes’ for the $i^{th}$ class $c_{i}$
($P(y_{c_{i}}=$‘yes’$|\mathbf{x},\mathbf{p})$) can be interpreted as the
probability that $c_{i}$ is included in the input text $\mathbf{x}$, and vice
versa.

##  6 Experiment

###  6.1 Experimental Environment

##### Experimental Models

To verify the generalizability of experimental results, we prepared two
different LMs pretrained on Biomedical literacy; BioBERT [22] and
BioClinicalBERT [2] 222Note that, BioBERT is additionally trained BERT [9] on
biomedical corpora and BioClinicalBERT is further trained BioBERT using MIMIC-
III [17] which is the previous version of MIMIC-IV.. Herein, ‘Finetune’ and
‘Prompt’ indicate an LM trained on ODD via finetuning (in Section 5.1) and
prompt-based finetuning (in Section 5.2) respectively.

##### Experimental Setting

For the experiments, we conducted 5-fold cross-validation and report the
average performance and standard deviation. We adopted a loss function as a
binary cross entropy for finetuning models and categorical cross entropy for
prompt-based finetuning models [21]. Moreover, we selected the optimizer as
AdamW [24].

##### Hyper-parameter Setting

We conducted the grid search with the following range of possible values for
each hyper-parameter: {2e-5, 3e-5, 5e-5} for learning rate, {4, 8, 16} for
batch size, {2,3,4} for the number of epoch. Herein, we choose the hyper-
parameters that achieved the best performance on the first fold of the
BioClinicalBERT finetune environment with the grid search. Finally, we chose
3e-5 for learning rate, 8 for batch size, and 3 for the number of epochs.

##### Others

To evaluation the statistical significance in performance between models, we
adopted student’s t-test [35]. In all of the experiments, we keep the random
seed as 0. Finally, all experiments were performed on an NVIDIA P40 GPU with
CentOS 7 version.

Table 4: This table presents the experimental results of ODD on BioClinicalBERT and BioBERT. Note that, ‘Finetune’ and ‘Prompt’ indicate models are trained with the finetuning and prompt-based finetuning respectively. Each value stands for the average and the standard deviation of five-fold cross-validation results and average scores with higher values are bolded. Otherwise, $\Delta$ means the performance difference between a prompt-based finetuning model and that of the corresponding finetuning model. Finally, ∗ stands for the statistical significance ($p<.05$) of performance improvement. Categories | BioClinicalBERT | BioBERT
---|---|---
Finetune | Prompt | $\Delta$ | Finetune | Prompt | $\Delta$
Confirmed Aberrant Behavior |  83.42 $\pm$ 3.39 | 84.28 $\pm$ 4.88 | 0.86 |  74.72 $\pm$ 8.49 | 84.60 $\pm$ 9.18 | 9.89
Suggested Aberrant Behavior |  44.74 $\pm$ 16.58 | 51.78 $\pm$ 12.55∗ | 7.04 |  33.38 $\pm$ 22.38 | 50.92 $\pm$ 9.99 | 17.54
Opioids |  97.09 $\pm$ 0.60 | 97.25 $\pm$ 0.37 | 0.16 |  95.86 $\pm$ 1.21 | 97.07 $\pm$ 0.59 | 1.21
Indication |  93.79 $\pm$ 2.56 | 95.97 $\pm$ 1.48∗ | 2.18 |  88.75 $\pm$ 7.84 | 95.17 $\pm$ 1.37 | 6.43
Diagnosed Opioid Dependency |  69.85 $\pm$ 7.72 | 73.03 $\pm$ 2.96 | 3.18 |  53.80 $\pm$ 22.59 | 70.56 $\pm$ 16.41 | 16.75
Benzodiapines |  92.61 $\pm$ 5.84 | 93.09 $\pm$ 2.33 | 0.48 |  86.37 $\pm$ 8.82 | 93.42 $\pm$ 2.37 | 7.05
Medication Change |  44.45 $\pm$ 9.64 | 66.58 $\pm$ 12.56∗ | 22.13 |  37.70 $\pm$ 24.70 | 57.87 $\pm$ 10.92∗ | 20.17
Central Nervous System Related | 98.21 $\pm$ 0.54 |  98.18 $\pm$ 0.29 | -0.03 |  97.12 $\pm$ 1.75 | 98.07 $\pm$ 0.50 | 0.95
Social Determinant of Health |  95.15 $\pm$ 3.17 | 95.16 $\pm$ 3.14 | 0.01 |  89.49 $\pm$ 11.67 | 93.93 $\pm$ 4.58 | 4.44
Macro Average |  79.92 $\pm$ 21.86 | 83.92 $\pm$ 16.54 | 4.00 |  73.02 $\pm$ 24.98 | 82.40 $\pm$ 18.08∗ | 9.38

###  6.2 Experimental Results

Table 4 shows the experimental results of the five-fold cross-validation on
the experimental models. To sum up, the performance range shows [73.02-83.92]
based on the macro average AUPRC over the categories. Especially, prompt-based
finetuning models outperformed the finetuning models in both BioClinicalBERT
and BioBERT with large margins of 4.00%p and 9.38%p respectively. Herein,
BioClinicalBERT-based models achieved a higher performance compared to
BioBERT-based models. These results are not surprising because the pre-
training BioClinicalBERT’s corpora contain EHR notes from MIMIC-III [17] that
is the previous version of our target database MIMIC-IV and both databases
were collected from the same hospital.

On the other hand, the performance among the classes has a large gap. For
example, in the BioClinicalBERT finetuned model, the class with the highest
performance (Central Nervous System Related) is 98.21, which is more than
double the performance gap compared to 44.45 of the lowest class (Medication
Change). Herein, the performance gap between these classes is related to the
number of instances. For example, the dominant classes, Opioids, and Central
Nervous System Related show very high performance with scores of 97.09 and
98.21, respectively. However, it can be seen that the detection performance of
the uncommon categories is inferior showing 44.74 for Suggested Aberrant
Behavior, 69.85 for Diagnosed Opioid Dependency, and 44.45 for Medication
Change. Moreover, we can notice that the performance results show the same
trend in BioBERT.

Overall, prompt-based finetuning contributes to enhanced performance in nearly
all environments (17 out of 18 cases), with the sole exception being Central
Nervous System Related on BioClinicalBERT, where the performance difference
was negligible (-0.03%p). The introduction of prompt-based finetuning resulted
in significant improvements, particularly in uncommon categories. The
performance of prompt-based finetuning on BioClinicalBERT and BioBERT
increased by 7.04%p and 17.54%p respectively in the Suggested Aberrant
Behavior class. In the Diagnosed Opioid Dependence class, the performance of
prompt-based finetuning on BioClinicalBERT and BioBERT improved by 3.18%p and
16.75%p, respectively. Lastly, in the Medication Change class, the performance
saw a rise of more than 20%p. Despite these advancements, further performance
improvements are still needed for uncommon categories.

##  7 Discussion

###  7.1 Error Analysis

We conducted an error analysis on the first fold of the BioClinicalBERT
prompt-based finetuning model. Especially, this paper focuses on error cases
of the three uncommon categories: Suggested Aberrant Behavior, Diagnosed
Opioid Dependency, and Medication Change.

Firstly, regarding Suggested Aberrant Behavior, we identified a problem with
insufficient data on specific abnormal behavior patterns. For instance,
consider the sentence “He is requesting IV morphine for his chest pain.” This
is a clear example of suggested aberrant behavior as the patient is asking for
a specific medication (IV morphine). However, due to a lack of similar pattern
sentences in the data, the model finds it challenging to learn these patterns.

Likewise, in the case of Medication Change, the sentence “The only exception
being that his home dilaudid 4mg was increased from every 6h to every 4h”
represents a medication change due to alterations in the drug administration
time interval. In this instance, the ML model might overlook the significance
of the change in time intervals due to the scarcity of similar patterns.

Furthermore, when dealing with Diagnosed Opioid Dependency, we noticed that a
model heavily relies on specific keywords. For example, the sentence “Insulin
Dependent DM c/b has peripheral neuropathy…” was classified as opioid
dependence, which is a misclassification. This error occurred due to the
reliance on the keyword ‘dependent’, despite the fact that insulin is not an
opioid.

Finally, we observed some text requires commonsense to correctly predict the
label. For example, the text “3 pitcher sized cocktail daily,” indicates the
patient is addicted to alcohol which is a confirmed aberrant behavior.
However, the prediction probability for this sentence is 0.31%, so the
training model totally fails to identify that this sentence stands for
confirmed aberrant behavior. This is because, different from other examples
where keywords such as alcohol addiction and alcohol abuse are presented, in
order to understand the above example, it is understood that the 3 pitcher
cocktail is an excessive dose and daily consumption is clear evidence of
alcohol abuse.

###  7.2 Merits & Demerits

Our research can have the following positive impacts. Firstly, the information
extracted by ORAB detection models can be utilized for various studies and
systems aimed at addressing opioid abuse. For instance, since ORABs serve as
important evidence of OUD, they can be used as key features in opioid risk
monitoring systems. Improved OOD prediction can save people’s lives.

Additionally, this information can be leveraged to detect a patient’s risk of
OOD or opioid addiction at an earlier stage, thereby assisting in the
prevention of fatal OOD cases. Consequently, by supporting efforts to mitigate
future opioid overdoses, our research would contribute to maintaining people’s
health.

However, it is important to acknowledge that our work may have certain
negative social impacts. As previously mentioned, ORAB detection can be
utilized to strengthen opioid monitoring systems, but this may unintentionally
encroach upon the autonomy of doctors [8]. Consequently, patients who
genuinely require opioids to manage their pain may encounter obstacles and
challenges in obtaining the necessary medications.

###  7.3 Limitation & Future Work

The ORAB detection task relies on EHR notes. Thus, if health providers do not
recognize the patient’s abnormal signs, they may not describe aberrant
behaviors in a note. In this case, our approach cannot detect ORABs. In the
future, we will develop an algorithm that detects a wider spectrum of ORABs by
combining them with previous structured information-based methods.

Moreover, ORAB detection models still have limited performance in the uncommon
categories. It is necessary to improve performance through advanced NLP
approaches data augmentation [43], medical knowledge injection [45], or
leveraging knowledge extracted from large language models [19].

##  8 Conclusion

This paper introduces a novel BioNLP task called ORAB detection, which aims to
identify two ORAB categories and seven categories relevant to opioid usage
from EHR notes. We also present the associated benchmark dataset, ODD. The
paper provides baseline models and their performances on ODD. To this end, we
trained two SOTA pretrained LMs using a fine-tuning approach and prompt-based
fine-tuning. Experimental results demonstrate that the performance in three
uncommon categories was notably lower compared to the other categories.
However, we also discovered that prompt-based fine-tuning can help mitigate
this issue. Additionally, we provide various error analysis results to guide
future studies.

## References

  * Adams et al. [2004] L. L. Adams, R. J. Gatchel, R. C. Robinson, P. Polatin, N. Gajraj, M. Deschner, and C. Noe.  Development of a self-report screening instrument for assessing potential opioid medication misuse in chronic pain patients.  _Journal of pain and symptom management_ , 27(5):440–459, 2004.
  * Alsentzer et al. [2019] E. Alsentzer, J. Murphy, W. Boag, W.-H. Weng, D. Jindi, T. Naumann, and M. McDermott.  Publicly available clinical bert embeddings.  In _Proceedings of the 2nd Clinical Natural Language Processing Workshop_ , pages 72–78, 2019.
  * Azadfard et al. [2022] M. Azadfard, M. R. Huecker, and J. M. Leaming.  Opioid addiction.  In _StatPearls [Internet]_ , 2022.  URL https://www.ncbi.nlm.nih.gov/books/NBK448203/.
  * Bekkar et al. [2013] M. Bekkar, H. K. Djemaa, and T. A. Alitouche.  Evaluation measures for models assessment over imbalanced data sets.  _J Inf Eng Appl_ , 3(10), 2013.
  * Beyer [2022] D. Beyer.  The economic toll of the opioid crisis reached nearly $1.5 trillion in 2020.  shorturl.at/kovz3, 2022.  Accessed: 2023-03-08.
  * Blackley et al. [2020] S. V. Blackley, E. MacPhaul, B. Martin, W. Song, J. Suzuki, and L. Zhou.  Using natural language processing and machine learning to identify hospitalized patients with opioid use disorder.  In _AMIA Annual Symposium Proceedings_ , volume 2020, page 233. American Medical Informatics Association, 2020.
  * Centers for Disease Control and Prevention [2023] Centers for Disease Control and Prevention.  Provisional drug overdose death counts.  https://www.ncbi.nlm.nih.gov/books/NBK448203/, 2023.  Accessed: 2023-03-08.
  * Clark et al. [2012] T. Clark, J. Eadie, P. Kreiner, and G. Strickler.  Prescription drug monitoring programs: an assessment of the evidence for best practices.  _The Prescription Drug Monitoring Program Center of Excellence_ , 2012\.
  * Devlin et al. [2018] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ , 2018.
  * eHOST [2011] eHOST.  ehost: The extensible human oracle suite of tools.  https://code.google.com/archive/p/ehost/, 2011.  Accessed: 2023-4-10.
  * Fleming et al. [2008] M. F. Fleming, J. Davis, and S. D. Passik.  Reported lifetime aberrant drug-taking behaviors are predictive of current substance use and mental health problems in primary care patients.  _Pain Medicine_ , 9(8):1098–1106, 2008.
  * Florence et al. [2021] C. Florence, F. Luo, and K. Rice.  The economic burden of opioid use disorder and fatal opioid overdose in the united states, 2017.  _Drug and alcohol dependence_ , 218:108350, 2021.
  * Gao et al. [2021] T. Gao, A. Fisch, and D. Chen.  Making pre-trained language models better few-shot learners.  In _Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP 2021_ , pages 3816–3830. Association for Computational Linguistics (ACL), 2021.
  * Goodman-Meza et al. [2022] D. Goodman-Meza, C. L. Shover, J. A. Medina, A. B. Tang, S. Shoptaw, and A. A. Bui.  Development and validation of machine models using natural language processing to classify substances involved in overdose deaths.  _JAMA Network Open_ , 5(8):e2225593–e2225593, 2022.
  * Johnson et al. [2021] A. Johnson, L. Bulgarelli, T. Pollard, L. A. Celi, R. Mark, and S. Horng IV.  Mimic-iv-ed.  _PhysioNet_ , 2021.
  * Johnson et al. [2023] A. Johnson, L. Bulgarelli, T. Pollard, S. Horng, L. A. Celi, and R. Mark.  MIMIC-IV, 2023.
  * Johnson et al. [2016] A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. Anthony Celi, and R. G. Mark.  Mimic-iii, a freely accessible critical care database.  _Scientific data_ , 3(1):1–9, 2016.
  * Katz et al. [2010] N. Katz, L. Panas, M. Kim, A. D. Audet, A. Bilansky, J. Eadie, P. Kreiner, F. C. Paillard, C. Thomas, and G. Carrow.  Usefulness of prescription monitoring programs for surveillance—analysis of schedule ii opioid prescription data in massachusetts, 1996–2006.  _Pharmacoepidemiology and drug safety_ , 19(2):115–123, 2010.
  * Kwon et al. [2023] S. Kwon, R. Garodia, M. Lee, Z. Yang, and H. Yu.  Vision meets definitions: Unsupervised visual word sense disambiguation incorporating gloss information.  _arXiv preprint arXiv:2305.01788_ , 2023.
  * Laxmaiah Manchikanti et al. [2008] M. Laxmaiah Manchikanti, M. Sairam Atluri, and M. Andrea Trescot.  Monitoring opioid adherence in chronic pain patients: Tools, techniques and utility.  _Pain Physician_ , 11:1, 2008.
  * LeCun et al. [2015] Y. LeCun, Y. Bengio, and G. Hinton.  Deep learning.  _nature_ , 521(7553):436–444, 2015.
  * Lee et al. [2020] J. Lee, W. Yoon, S. Kim, D. Kim, S. Kim, C. H. So, and J. Kang.  Biobert: a pre-trained biomedical language representation model for biomedical text mining.  _Bioinformatics_ , 36(4):1234–1240, 2020.
  * Lingeman et al. [2017] J. M. Lingeman, P. Wang, W. Becker, and H. Yu.  Detecting opioid-related aberrant behavior using natural language processing.  In _AMIA Annual Symposium Proceedings_ , volume 2017, page 1179. American Medical Informatics Association, 2017.
  * Loshchilov and Hutter [2017] I. Loshchilov and F. Hutter.  Decoupled weight decay regularization.  _arXiv preprint arXiv:1711.05101_ , 2017.
  * Marks et al. [2021] C. Marks, D. Abramovitz, C. A. Donnelly, G. Carrasco-Escobar, R. Carrasco-Hernández, D. Ciccarone, A. González-Izquierdo, N. K. Martin, S. A. Strathdee, D. M. Smith, et al.  Identifying counties at risk of high overdose mortality burden during the emerging fentanyl epidemic in the usa: a predictive statistical modelling study.  _The Lancet Public Health_ , 6(10):e720–e728, 2021.
  * Maumus et al. [2020] M. Maumus, R. Mancini, D. M. Zumsteg, and D. K. Mandali.  Aberrant drug-related behavior monitoring.  _The Ochsner Journal_ , 20(4):358, 2020.
  * National Institute on Drug Abuse [2023] National Institute on Drug Abuse.  Aberrant drug taking behaviors information sheet.  https://nida.nih.gov/sites/default/files/AberrantDrugTakingBehaviors.pdf, 2023\.  Accessed: 2023-04-14.
  * Ozenne et al. [2015] B. Ozenne, F. Subtil, and D. Maucort-Boulch.  The precision–recall curve overcame the optimism of the receiver operating characteristic curve in rare diseases.  _Journal of clinical epidemiology_ , 68(8):855–859, 2015.
  * Passik and Kirsh [2007] S. D. Passik and K. L. Kirsh.  Commentary on jung and reidenberg’s “physicians being deceived”: aberrant drug-taking behaviors: what pain physicians can know (or should know).  _Pain Medicine_ , 8(5):442–444, 2007.
  * Portenoy [1996] R. K. Portenoy.  Opioid therapy for chronic nonmalignant pain: a review of the critical issues.  _Journal of pain and symptom management_ , 11(4):203–217, 1996.
  * Rough et al. [2019] K. Rough, K. F. Huybrechts, S. Hernandez-Diaz, R. J. Desai, E. Patorno, and B. T. Bateman.  Using prescription claims to detect aberrant behaviors with opioids: comparison and validation of 5 algorithms.  _Pharmacoepidemiology and drug safety_ , 28(1):62–69, 2019.
  * Sarker et al. [2019] A. Sarker, G. Gonzalez-Hernandez, Y. Ruan, and J. Perrone.  Machine learning and natural language processing for geolocation-centric monitoring and characterization of opioid-related social media chatter.  _JAMA network open_ , 2(11):e1914672–e1914672, 2019.
  * Schloff et al. [2004] L. A. Schloff, T. S. Rector, R. Seifeldin, and J. D. Haddox.  Identifying controlled substance patterns of utilization requiring evaluation using administrative claims data.  _The American journal of managed care_ , 10:783–790, 2004\.
  * Singleton et al. [2023] J. Singleton, C. Li, P. D. Akpunonu, E. L. Abner, and A. M. Kucharska-Newton.  Using natural language processing to identify opioid use disorder in electronic health record data.  _International Journal of Medical Informatics_ , 170:104963, 2023.
  * Student [1908] Student.  The probable error of a mean.  _Biometrika_ , pages 1–25, 1908.
  * Sullivan et al. [2010] M. D. Sullivan, M. J. Edlund, M.-Y. Fan, A. DeVries, J. B. Braden, and B. C. Martin.  Risks for possible and probable opioid misuse among recipients of chronic opioid therapy in commercial and medicaid insurance plans: The troup study.  _Pain_ , 150(2):332–339, 2010.
  * Sun et al. [2017] E. C. Sun, A. Dixit, K. Humphreys, B. D. Darnall, L. C. Baker, and S. Mackey.  Association between concurrent use of prescription opioids and benzodiazepines and overdose: retrospective analysis.  _bmj_ , 356, 2017.
  * Tudor [2013] C. Tudor.  Memorandum: Medicare part d overutilization monitoring system.  _Centers for Medicare and Medicaid Services_ , 2013.
  * Wang [2022] X. Wang.  Using natural language processing to detect opioid-related aberrant behaviors from electronic health records.  In _APHA 2022 Annual Meeting and Expo_. APHA, 2022.
  * Warrens [2015] M. J. Warrens.  Five ways to look at cohen’s kappa.  _Journal of Psychology & Psychotherapy_, 5(4):1, 2015.
  * Webson and Pavlick [2022] A. Webson and E. Pavlick.  Do prompt-based models really understand the meaning of their prompts?  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2300–2344, 2022.
  * Webster and Webster [2005] L. R. Webster and R. M. Webster.  Predicting aberrant behaviors in opioid-treated patients: preliminary validation of the opioid risk tool.  _Pain medicine_ , 6(6):432–442, 2005.
  * Wei and Zou [2019] J. Wei and K. Zou.  Eda: Easy data augmentation techniques for boosting performance on text classification tasks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6382–6388, 2019.
  * Weiss et al. [2020] A. J. Weiss, K. C. Heslin, C. Stocks, and P. L. Owens.  Hospital inpatient stays related to opioid use disorder and endocarditis, 2016: statistical brief# 256.  2020\.
  * Yang et al. [2022] Z. Yang, S. Wang, B. P. S. Rawat, A. Mitra, and H. Yu.  Knowledge injected prompt based fine-tuning for multi-label few-shot icd coding.  In _Findings of Empirical Methods in Natural Language Processing (EMNLP)_ , 2022.
  * Yang et al. [2023] Z. Yang, S. Kwon, Z. Yao, and H. Yu.  Multi-label few-shot icd coding as autoregressive generation with prompt.  In _AAAI 2023_ , 2023.
  * Zahera et al. [2019] H. M. Zahera, I. A. Elgendy, R. Jalota, M. A. Sherif, and E. Voorhees.  Fine-tuned bert model for multi-label tweets classification.  In _TREC_ , pages 1–7, 2019.
  * Zhu et al. [2022] V. J. Zhu, L. A. Lenert, K. S. Barth, K. N. Simpson, H. Li, M. Kopscik, and K. T. Brady.  Automatically identifying opioid use disorder in non-cancer patients on chronic opioid therapy.  _Health Informatics Journal_ , 28(2):14604582221107808, 2022.

## Data Sharing

We have shared the source code via supplementary data to the anonymous
reviewers. We are working on making the ODD publicly accessible via open
credentialized access, PhysioNet333https://physionet.org.

## Checklist

For all authors…

(a) Do the main claims made in the abstract and introduction accurately
reflect the paper’s contributions and scope? (Y)

(b) Have you read the ethics review guidelines and ensured that your paper
conforms to them? (Y)

(c) Did you discuss any potential negative societal impacts of your work? (Y)

(d) Did you describe the limitations of your work? (Y)

If you ran experiments…

(a) Did you include the code, data, and instructions needed to reproduce the
main experimental results (either in the supplemental material or as a URL)?
(Y)

(b) Did you specify all the training details (e.g., data splits,
hyperparameters, how they were chosen)? (Y)

(c) Did you report error bars (e.g., with respect to the random seed after
running experiments multiple times)? (Y)

(d) Did you include the amount of compute and the type of resources used
(e.g., type of GPUs, internal cluster, or cloud provider)? (Y)

If you are using existing assets (e.g., code, data, models) or
curating/releasing new assets…

(a) If your work uses existing assets, did you cite the creators? (Y)

(b) Did you mention the license of the assets? (N) We are doing contact with
PhysioNet to publicly open the dataset. Thus, in the current step, license is
not available yet.

(c) Did you include any new assets either in the supplemental material or as a
URL? (N) We are working with PhysioNet to make data publicly available. We
will response as soon as possible when the dataset accepted by PhysioNet.

(d) Did you discuss whether and how consent was obtained from people whose
data you’re using/curating?

(Y) Our dataset is based on a fully de-identified EHR notes and all of our
authors have valid lisence to work on the dataset.

(a) Did you include the full text of instructions given to participants and
screenshots, if applicable?

(Y) We described the dataset annotation environment in the supplementary.

(b) Did you describe any potential participant risks, with links to
Institutional Review Board (IRB) approvals, if applicable? (NA)

(c) Did you include the estimated hourly wage paid to participants and the
total amount spent on participant compensation?

(NA) All annotators are full employed domain experts. Thus, it is difficult to
estimate hourly wage.

##  Appendix A Details on Data Construction

###  A.1 Details on Data Collection

Table 5: Opioids and their generic naming that used for filtering. Medication Names | Generic Names
---|---
Ascomp with Codeine | aspirin/butalbital/caffeine/codeine
B & O Supprettes | belladonna/opium
Darvon Compound-65 | aspirin/caffeine/propoxyphene
Lorcet | acetaminophen/hydrocodone
Maxidone | acetaminophen/hydrocodone
Fiorinal with Codeine III | aspirin/butalbital/caffeine/codeine
Magnacet | acetaminophen/oxycodone
Meprozine | meperidine/promethazine
Fiorinal with Codeine | aspirin/butalbital/caffeine/codeine
Fioricet with Codeine | acetaminophen/butalbital/caffeine/codeine
Lorcet Plus | acetaminophen/hydrocodone
Percocet 10 / 325 | acetaminophen/oxycodone
Primlev | acetaminophen/oxycodone
Suboxone | buprenorphine/naloxone
Ibudone | hydrocodone/ibuprofen
Lorcet 10 / 650 | acetaminophen/hydrocodone
Panlor DC | acetaminophen/caffeine/dihydrocodeine
Reprexain | hydrocodone/ibuprofen
Percocet | acetaminophen/oxycodone
Combunox | ibuprofen/oxycodone
Hydrocet | acetaminophen/hydrocodone
Roxicet | acetaminophen/oxycodone
Tylox | acetaminophen/oxycodone
Xolox | acetaminophen/oxycodone
Vicodin ES | acetaminophen/hydrocodone
Hycet | acetaminophen/hydrocodone
Talacen | acetaminophen/pentazocine
Vicodin HP | acetaminophen/hydrocodone
Vicoprofen | hydrocodone/ibuprofen
Percocet 7.5 / 325 | acetaminophen/oxycodone
Lortab | acetaminophen/hydrocodone
Norco | acetaminophen/hydrocodone
Vicodin | acetaminophen/hydrocodone
Percocet 5 / 325 | acetaminophen/oxycodone
Stagesic | acetaminophen/hydrocodone
Targiniq ER | naloxone/oxycodone
Xodol | acetaminophen/hydrocodone
Endocet | acetaminophen/oxycodone
Ultracet | acetaminophen/tramadol
Panlor SS | acetaminophen/caffeine/dihydrocodeine
Zubsolv | buprenorphine/naloxone
Xartemis XR | acetaminophen/oxycodone
Talwin Nx | naloxone/pentazocine
Tylenol with Codeine | acetaminophen/codeine
Anexsia | acetaminophen/hydrocodone
Darvocet-N 50 | acetaminophen/propoxyphene
Liquicet | acetaminophen/hydrocodone
Darvocet-N 100 | acetaminophen/propoxyphene
Trezix | acetaminophen/caffeine/dihydrocodeine
Percodan | aspirin/oxycodone
Darvocet A500 | acetaminophen/propoxyphene
Percocet 2.5 / 325 | acetaminophen/oxycodone
Balacet | acetaminophen/propoxyphene
Aceta w/ Codeine | acetaminophen/codeine
Zamicet | acetaminophen/hydrocodone
Embeda | morphine/naltrexone
Bunavail | buprenorphine/naloxone
Tylenol with Codeine #3 | acetaminophen/codeine
Narvox | acetaminophen/oxycodone
Zydone | acetaminophen/hydrocodone
Tylenol with Codeine #4 | acetaminophen/codeine
Capital w/ Codeine | acetaminophen/codeine
Co-Gesic | acetaminophen/hydrocodone
Cocet Plus | acetaminophen/codeine
Codrix | acetaminophen/codeine
Dolacet | acetaminophen/hydrocodone
Dolagesic | acetaminophen/hydrocodone
Endodan | aspirin/oxycodone
Perloxx | acetaminophen/oxycodone
Phrenilin with Caffeine and Codeine | acetaminophen/butalbital/caffeine/codeine
Roxilox | acetaminophen/oxycodone
Synalgos-DC | aspirin/caffeine/dihydrocodeine
Theracodophen Low 90 | acetaminophen/hydrocodone
Tramapap | acetaminophen/tramadol
Trycet | acetaminophen/propoxyphene
Verdrocet | acetaminophen/hydrocodone
Zolvit | acetaminophen/hydrocodone
Astramorph PF | morphine
Ionsys | fentanyl
Lazanda | fentanyl
Levo-Dromoran | levorphanol
Numorphan | oxymorphone
Onsolis | fentanyl
Oxyfast | oxycodone
Palladone | hydromorphone
Roxanol | morphine
Roxanol-T | morphine
Roxicodone Intensol | oxycodone
Meperitab | meperidine
Methadone Diskets | methadone
Actiq | fentanyl
Fentora | fentanyl
Subutex | buprenorphine
Demerol | meperidine
Dolophine | methadone
Roxicodone | oxycodone
Duragesic-25 | fentanyl
Infumorph | morphine
Methadose | methadone
Ultram ODT | tramadol
Dilaudid | hydromorphone
Subsys | fentanyl
MSIR | morphine
OxyContin | oxycodone
Paregoric | opium
Duragesic-100 | fentanyl
Abstral | fentanyl
Oxydose | oxycodone
Stadol | butorphanol
Duragesic | fentanyl
Duragesic-50 | fentanyl
Buprenex | buprenorphine
Zohydro ER | hydrocodone
Duragesic-75 | fentanyl
MS Contin | morphine
Kadian | morphine
Opana | oxymorphone
Opana ER | oxymorphone
Sublimaze | fentanyl
Exalgo | hydromorphone
Opium Deodorized | opium
Oxaydo | oxycodone
Avinza | morphine
Nucynta ER | tapentadol
Darvon-N | propoxyphene
OxyIR | oxycodone
Nubain | nalbuphine
Dilaudid-HP | hydromorphone
Rybix ODT | tramadol
Ultram ER | tramadol
Butrans | buprenorphine
Darvon | propoxyphene
Oramorph SR | morphine
Nucynta | tapentadol
Ultram | tramadol
Duramorph | morphine
Ryzolt | tramadol
Talwin | pentazocine
Duragesic-12 | fentanyl
Alfenta | alfentanil
ConZip | tramadol
Hysingla ER | hydrocodone
Belbuca | buprenorphine
Dazidox | oxycodone
DepoDur | morphine liposomal
ETH-Oxydose | oxycodone
Oxecta | oxycodone
Probuphine | buprenorphine
RMS | morphine
Sufenta | sufentanil
Ultiva | remifentanil
Continued on next page

Table 6: ICD 9 and ICD 10 diagnosis codes relevant to OUD. Note that, all of these codes defined by Weiss et al. [44]. ICD code | ICD Description
---|---
ICD 9 diagnosis codes
304 | Opioid type dependence, unspecified
304.01 | Opioid type dependence, continuous
304.02 | Opioid type dependence, episodic
304.03 | Opioid type dependence, in remission
304.7 | Combinations of opioid type drug with any other drug dependence, unspecified
304.71 | Combinations of opioid type drug with any other drug dependence, continuous
304.72 | Combinations of opioid type drug with any other drug dependence, episodic
304.73 | Combinations of opioid type drug with any other drug dependence, in remission
305.5 | Opioid abuse, unspecified
305.51 | Opioid abuse, continuous
305.52 | Opioid abuse, episodic
305.53 | Opioid abuse, in remission
965 | Poisoning by opium (alkaloids), unspecified
965.01 | Poisoning by heroin
965.02 | Poisoning by methadone
965.09 | Poisoning by other opiates and related narcotics
970.1 | Poisoning by opiate antagonists
E850.0 | Accidental poisoning by heroin
E850.1 | Accidental poisoning by methadone
E850.2 | Accidental poisoning by other opiates and related narcotics
E935.0 | Heroin causing adverse effects in therapeutic use
E935.1 | Methadone causing adverse effects in therapeutic use
E935.2 | Other opiates and related narcotics causing adverse effects in therapeutic use
E940.1 | Adverse effects of opiate antagonists
ICD 10 diagnosis codes
Opioid abuse/dependence
F11.10 | Opioid abuse, uncomplicated
F11.120 | Opioid abuse with intoxication, uncomplicated
F11.121 | Opioid abuse with intoxication, delirium
F11.122 | Opioid abuse with intoxication, with perceptual disturbance
F11.129 | Opioid abuse with intoxication, unspecified
F11.14 | Opioid abuse with opioid-induced mood disorder
F11.150 | Opioid abuse with opioid-induced psychotic disorder, with delusions
F11.151 | Opioid abuse with opioid-induced psychotic disorder, with hallucinations
F11.159 | Opioid abuse with opioid-induced psychotic disorder, unspecified
F11.181 | Opioid abuse with opioid-induced sexual dysfunction
F11.182 | Opioid abuse with opioid-induced sleep disorder
F11.188 | Opioid abuse with other opioid-induced disorder
F11.19 | Opioid abuse with unspecified opioid-induced disorder
F11.20 | Opioid dependence, uncomplicated
F11.21 | Opioid dependence, in remission
F11.220 | Opioid dependence with intoxication, uncomplicated
F11.221 | Opioid dependence with intoxication, delirium
F11.222 | Opioid dependence with intoxication, with perceptual disturbance
F11.229 | Opioid dependence with intoxication, unspecified
F11.23 | Opioid dependence with withdrawal
F11.24 | Opioid dependence with opioid-induced mood disorder
F11.250 | Opioid dependence with opioid-induced psychotic disorder, with delusions
F11.251 | Opioid dependence with opioid-induced psychotic disorder, with hallucinations
F11.259 | Opioid dependence with opioid-induced psychotic disorder, unspecified
F11.281 | Opioid dependence with opioid-induced sexual dysfunction
F11.282 | Opioid dependence with opioid-induced sleep disorder
F11.288 | Opioid dependence with other opioid-induced disorder
F11.29 | Opioid dependence with unspecified opioid-induced disorder
Opioid use
F11.90 | Opioid use, unspecified, uncomplicated
F11.920 | Opioid use, unspecified with intoxication, uncomplicated
F11.921 | Opioid use, unspecified with intoxication delirium
F11.922 | Opioid use, unspecified with intoxication, with perceptual disturbance
F11.929 | Opioid use, unspecified with intoxication, unspecified
F11.93 | Opioid use, unspecified, with withdrawal
F11.94 | Opioid use, unspecified, with opioid-induced mood disorder
F11.950 | Opioid use, unspecified with opioid-induced psychotic disorder, with delusions
F11.951 | Opioid use, unspecified with opioid-induced psychotic disorder, with hallucinations
F11.959 | Opioid use, unspecified with opioid-induced psychotic disorder, unspecified
F11.981 | Opioid use, unspecified with opioid-induced sexual dysfunction
F11.982 | Opioid use, unspecified with opioid-induced sleep disorder
F11.988 | Opioid use, unspecified with other opioid-induced disorder
F11.99 | Opioid use, unspecified, with unspecified opioid-induced disorder
Poisoning
T40.0X1A | Poisoning by opium, accidental (unintentional), initial encounter
T40.0X1D | Poisoning by opium, accidental (unintentional), subsequent encounter
T40.0X2A | Poisoning by opium, intentional self-harm, initial encounter
T40.0X2D | Poisoning by opium, intentional self-harm, subsequent encounter
T40.0X3A | Poisoning by opium, assault, initial encounter
T40.0X3D | Poisoning by opium, assault, subsequent encounter
T40.0X4A | Poisoning by opium, undetermined, initial encounter
T40.0X4D | Poisoning by opium, undetermined, subsequent encounter
T40.1X1A | Poisoning by heroin, accidental (unintentional), initial encounter
T40.1X1D | Poisoning by heroin, accidental (unintentional), subsequent encounter
T40.1X2A | Poisoning by heroin, intentional self-harm, initial encounter
T40.1X2D | Poisoning by heroin, intentional self-harm, subsequent encounter
T40.1X3A | Poisoning by heroin, assault, initial encounter
T40.1X3D | Poisoning by heroin, assault, subsequent encounter
T40.1X4A | Poisoning by heroin, undetermined, initial encounter
T40.1X4D | Poisoning by heroin, undetermined, subsequent encounter
T40.2X1A | Poisoning by other opioids, accidental (unintentional), initial encounter
T40.2X1D | Poisoning by other opioids, accidental (unintentional), subsequent encounter
T40.2X2A | Poisoning by other opioids, intentional self-harm, initial encounter
T40.2X2D | Poisoning by other opioids, intentional self-harm, subsequent encounter
T40.2X3A | Poisoning by other opioids, assault, initial encounter
T40.2X3D | Poisoning by other opioids, assault, subsequent encounter
T40.2X4A | Poisoning by other opioids, undetermined, initial encounter
T40.2X4D | Poisoning by other opioids, undetermined, subsequent encounter
T40.3X1A | Poisoning by methadone, accidental (unintentional), initial encounter
T40.3X1D | Poisoning by methadone, accidental (unintentional), subsequent encounter
T40.3X2A | Poisoning by methadone, intentional self-harm, initial encounter
T40.3X2D | Poisoning by methadone, intentional self-harm, subsequent encounter
T40.3X3A | Poisoning by methadone, assault, initial encounter
T40.3X3D | Poisoning by methadone, assault, subsequent encounter
T40.3X4A | Poisoning by methadone, undetermined, initial encounter
T40.3X4D | Poisoning by methadone, undetermined, subsequent encounter
T40.4X1A | Poisoning by synthetic narcotics, accidental (unintentional), initial encounter
T40.4X1D | Poisoning by synthetic narcotics, accidental (unintentional), subsequent encounter
T40.4X2A | Poisoning by other synthetic narcotics, intentional self-harm, initial encounter
T40.4X2D | Poisoning by other synthetic narcotics, intentional self-harm, subsequent encounter
T40.4X3A | Poisoning by other synthetic narcotics, assault, initial encounter
T40.4X3D | Poisoning by other synthetic narcotics, assault, subsequent encounter
T40.4X4A | Poisoning by synthetic narcotics, undetermined, initial encounter
T40.4X4D | Poisoning by synthetic narcotics, undetermined, subsequent encounter
T40.601A | Poisoning by unspecified narcotics, accidental (unintentional), initial encounter
T40.601D | Poisoning by unspecified narcotics, accidental (unintentional), subsequent encounter
T40.602A | Poisoning by unspecified narcotics, intentional self-harm, initial encounter
T40.602D | Poisoning by unspecified narcotics, intentional self-harm, subsequent encounter
T40.603A | Poisoning by unspecified narcotics, assault, initial encounter
T40.603D | Poisoning by unspecified narcotics, assault, subsequent encounter
T40.604A | Poisoning by unspecified narcotics, undetermined, initial encounter
T40.604D | Poisoning by unspecified narcotics, undetermined, subsequent encounter
T40.691A | Poisoning by other narcotics, accidental (unintentional), initial encounter
T40.691D | Poisoning by other narcotics, accidental (unintentional), subsequent encounter
T40.692A | Poisoning by other narcotics, intentional self-harm, initial encounter
T40.692D | Poisoning by other narcotics, intentional self-harm, subsequent encounter
T40.693A | Poisoning by other narcotics, assault, initial encounter
T40.693D | Poisoning by other narcotics, assault, subsequent encounter
T40.694A | Poisoning by other narcotics, undetermined, initial encounter
T40.694D | Poisoning by other narcotics, undetermined, subsequent encounter
Adverse effects
T40.0X5A | Adverse effect of opium, initial encounter
T40.0X5D | Adverse effect of opium, subsequent encounter
T40.2X5A | Adverse effect of other opioids, initial encounter
T40.2X5D | Adverse effect of other opioids, subsequent encounter
T40.3X5A | Adverse effect of methadone, initial encounter
T40.3X5D | Adverse effect of methadone, subsequent encounter
T40.4X5A | Adverse effect of synthetic narcotics, initial encounter
T40.4X5D | Adverse effect of synthetic narcotic, subsequent encounter
T40.605A | Adverse effect of unspecified narcotics, initial encounter
T40.605D | Adverse effect of unspecified narcotics, subsequent encounter
T40.695A | Adverse effect of other narcotics, initial encounter
T40.695D | Adverse effect of other narcotics, subsequent encounter
Long-term use of opiates
Z79.891 | Long-term (current) use of opiate analgesic
Continued on next page