# L12pt

# Item-Level Heterogeneous Treatment Effects of Selective Serotonin Reuptake
Inhibitors (SSRIs) on Depression: Implications for Inference,
Generalizability, and Identification

Joshua B. Gilbert  Harvard Graduate School of Education  Fredrik Hieronymus
University of Gothenburg  Elias Eriksson  University of Gothenburg  Benjamin
W. Domingue  Stanford Graduate School of Education

Objectives: In analysis of randomized controlled trials (RCTs) with patient-
reported outcome measures (PROMs), Item Response Theory (IRT) models that
allow for heterogeneity in the treatment effect at the item level merit
consideration. These models for “item-level heterogeneous treatment effects”
(IL-HTE) can provide more accurate statistical inference, allow researchers to
better generalize their results, and resolve critical identification problems
in the estimation of interaction effects. In this study, we extend the IL-HTE
model to polytomous data and apply the model to determine how the effect of
selective serotonin reuptake inhibitors (SSRIs) on depression varies across
the items on a depression rating scale.

Methods: We first conduct a Monte Carlo simulation study to assess the
performance of the polytomous IL-HTE model under a range of conditions. We
then apply the IL-HTE model to item-level data from 28 RCTs measuring the
effect of SSRIs on depression using the 17-item Hamilton Depression Rating
Scale (HDRS-17) and estimate potential heterogeneity by subscale (HDRS-6).

Results: Our results show that the IL-HTE model provides more accurate
statistical inference, allows for generalizability of results to out-of-sample
items, and resolves identification problems in the estimation of interaction
effects. Our empirical application shows that while the average effect of
SSRIs on depression is beneficial (i.e., negative) and statistically
significant, there is substantial IL-HTE, with estimates of the standard
deviation of item-level effects nearly as large as the average effect. We show
that this substantial IL-HTE is driven primarily by systematically larger
effects on the HDRS-6 subscale items.

Conclusions: The IL-HTE model has the potential to provide new insights for
the inference, generalizability, and identification of treatment effects in
clinical trials using patient reported outcome measures.

Keywords: causal inference, heterogeneous treatment effects, item response
theory, depression, SSRIs

##  1 Introduction

Heterogeneous treatment effects (HTE) are crucial for epidemiological research
and public health policy because understanding for whom and under what
conditions a medical treatment works allows policy makers to best target
treatments to populations or subgroups that would most benefit [85, 94, 124,
150, 140, 129]. One limitation of standard statistical methods for HTE
analysis is that they focus on person characteristics (e.g., age, gender,
etc.) and may ignore the potential HTE that exists among the items used to
measure a latent variable of interest. That is, many outcomes relevant to
clinical trials or epidemiological research can only be assessed indirectly
through multi-item surveys or psychometric instruments, such as patient-
reported outcome measures (PROMs) for well-being [131], depression [120, 143],
perceptions of hearing loss [121], pain [101], or recovery after childbirth
[146, 147], in contrast to simple biometric measures that can be measured to
arbitrary levels of precision and captured in a single number (e.g., height,
weight, blood pressure, mortality, etc.). Therefore, when we use a PROM to
construct a sum or factor score to serve as an outcome measure in clinical
trials or epidemiological studies, we may be ignoring potential HTE that
exists among the individual items of the PROM. As a result, our understanding
of the consistency or generalizability of treatment effects may be limited
[111, 79, 144], and—when there is interest in treatment by baseline covariate
interactions—create causal identification challenges that can only be resolved
by leveraging item-level data [112].

To address these limitations, recent work applies techniques from Item
Response Theory (IRT; [149]) to allow for the assessment of treatment effects
that vary at the outcome item level [144, 79, 111]. These techniques allow us
to determine whether treatment effects are consistent and impact all PROM
items equally or, alternatively, vary across the items within the PROM. To our
knowledge, however, the “item-level heterogeneous treatment effects” (IL-HTE)
model has not yet been applied beyond standardized tests in education
research; clinical trial or epidemiological studies that similarly rely on
latent variable outcomes such as PROMs may also benefit from such approaches.
Our work therefore builds on the previous interest in the affordances of item-
level analysis in clinical trials and epidemiological research [120, 123, 114,
82, 91, 80].

Our study pursues two aims. First, we apply the IL-HTE model to polytomous
item response data, thereby extending past analyses limited to dichotomous
(i.e., correct vs. incorrect) item responses. Polytomous data is of interest
in epidemiological settings given the widespread use of, for example, Likert
scales [90, 102]. Second, we apply the IL-HTE model to a clinical trial
context using item-level data from a set of 28 randomized controlled trials
(RCTs) evaluating the effect of SSRIs on the 17-item Hamilton Depression
Rating Scale (HDRS-17). The study is organized as follows. In Section 1.1, we
contextualize the IL-HTE model within the broader context of HTE analysis, for
both dichotomous and polytomous responses. In Section 2, we describe our Monte
Carlo simulation design and our empirical data. In Section 3, we present the
results of the simulation and the results of our models fit to the empirical
data. We conclude in Section 4 with a discussion of the implications of our
findings for HTE analysis in clinical trials and epidemiology.

###  1.1 Estimating Heterogeneous Treatment Effects

Consider the following standard regression model for HTE:

$\displaystyle Y_{j}$
$\displaystyle=\beta_{0}+\beta_{1}T_{j}+\beta_{2}X_{j}+\beta_{3}T_{j}\times
X_{j}+e_{j}$ (1) $\displaystyle e_{j}$ $\displaystyle\sim
N(0,\sigma_{\theta}),$ (2)

in which $Y_{j}$ is the outcome variable for individual $j$, $T_{j}$ is a
dichotomous indicator for randomized treatment assignment, and $X_{j}$ is a
person characteristic (e.g., age, gender, etc.). $\beta_{1}$ is the
conditional average treatment effect (CATE) when $X=0$, $\beta_{2}$ is the
main effect of $X$ when $T=0$, and $\beta_{3}$ is our HTE parameter, capturing
how the CATE depends on the level of $X$. When $\beta_{3}=0$, the ATE is
constant across all level of $X$; when $\beta_{3}>0$, the ATE is larger at
higher levels of $X$. A concrete example of such a model in epidemiology would
be how the effect of COVID vaccinations ($T$) on mortality ($Y$) varies by
patient age ($X$) [93, 103].

When $Y_{j}$ is a latent outcome such as a PROM, such as a depression rating
scale with multiple items reflecting various symptoms of depression, the
treatment may differentially affects individual symptoms or symptom clusters
represented by the individual items of the PROM. The standard HTE model above
represents one extreme based on analysis of a single number such as the sum
score as the outcome (a common practice; [105, 132]) that ignores all such
differentiation. On the other extreme, researchers could analyze treatment
effects on each item separately, but this approach is difficult to interpret
and suffers from multiple comparisons problems, particularly when the number
of items is large. As a compromise, researchers examine effects on subscales
or item clusters, but this approach requires an a priori specification of
which subscales to evaluate and the assumption that within each subscale, the
item effects are constant [109], and as such is somewhat ad hoc.

An elegant solution that leverages the PROM item responses directly and thus
makes use of all available information without the need to compute total or
subscale scores in a separate step of the analysis is the Explanatory Item
Response Model (EIRM) [153, 152, 138, 97]. For example, when items are
dichotomous (e.g., 0 = symptom absence, 1 = symptom presence), we can use a
cross-classified logistic regression model with a main effect for treatment,
such as,

$\displaystyle\text{logit}(Y_{ij}=1)=\eta_{ij}$
$\displaystyle=\theta_{j}+b_{i}$ (3) $\displaystyle\theta_{j}$
$\displaystyle=\beta_{0}+\beta_{1}T_{j}+e_{j}$ (4) $\displaystyle b_{i}$
$\displaystyle=b_{i}$ (5) $\displaystyle b_{i}$ $\displaystyle\sim
N(0,\sigma_{b})$ (6) $\displaystyle e_{j}$ $\displaystyle\sim
N(0,\sigma_{\theta}),$ (7)

where $Y_{ij}$ is the response of person $j$ to item $i$, $\theta_{j}$ is the
unobserved or latent person trait (e.g., depression), and $b_{i}$ is item
location (i.e., “easiness” in educational measurement). The EIRM is equivalent
to a one-parameter logistic (1PL) or Rasch IRT model when the item location
parameters are considered fixed [95]. $\beta_{1}$ represents the ATE, but
estimated directly on the latent trait $\theta_{j}$ without the need to
compute a sum or factor score to be used as an outcome in a two-step analysis
[109, 111, 110, 108, 92, 156]. Even without considering the possibility of IL-
HTE, the EIRM still provides some benefits over a sum score analysis as it can
be more robust to violations of model assumptions such as heteroskedasticity
or missing data [110] and can provide unbiased estimates of standardized
effect sizes that are attenuated by measurement error [108, 117].

We can allow for IL-HTE by introducing a random interaction between treatment
and item in a random slope term in the equation for $b_{ij}$, where the added
subscript $j$ allows for separate $b$ parameters for each treatment group
[111, 109, 79, 144, 112]:

$\displaystyle\text{logit}(Y_{ij}=1)=\eta_{ij}$
$\displaystyle=\theta_{j}+b_{ij}$ (8) $\displaystyle\theta_{j}$
$\displaystyle=\beta_{0}+\beta_{1}T_{j}+e_{j}$ (9) $\displaystyle b_{ij}$
$\displaystyle=b_{i}+\zeta_{i}T_{j}$ (10)
$\displaystyle\begin{bmatrix}b_{i}\\\ \zeta_{i}\end{bmatrix}$
$\displaystyle\sim N(0,\begin{bmatrix}\sigma_{b}&\rho\\\
\rho&\sigma_{\zeta}\end{bmatrix})$ (11) $\displaystyle e_{j}$
$\displaystyle\sim N(0,\sigma_{\theta}).$ (12)

Here, $\beta_{1}$ still represents the ATE on the latent trait $\theta_{j}$,
but now for the average item on the scale. Item-specific residual treatment
effects are represented by $\zeta_{i}$, the item-specific deviation from
$\beta_{1}$. If $\zeta_{i}>0$, then item $i$ is more affected by treatment
than the average item. The $\zeta_{i}$ are equivalent to uniform differential
item function (DIF) caused by the treatment [111, 134]. $\sigma_{\zeta}$
provides a direct parameter estimate for the degree of IL-HTE in the data by
providing the standard deviation (SD) of the item-specific treatment effects
around the ATE $\beta_{1}$ and $\rho$ indexes the correlation between item
location and item-specific treatment effect size. That is, if $\rho>0$, then
items representing more commonly endorsed symptoms show systematically larger
or smaller treatment effects. $\rho$ may be of interest in itself [111, pp.
893-894], but is perhaps most critical in that $\rho\neq 0$ can create a
causal identification problem. That is, $\rho\neq 0$ can induce spurious
treatment by baseline covariate interaction effects that can only be resolved
with item-level analysis [112], a result with critical implications for the
analysis of HTE that we will return to in our simulations, particularly given
the great interest in HTE by baseline severity in depression treatments [127,
106]. A directed acyclic graph (DAG) [115, 122, 113, 148] representation of
the IL-HTE model is presented in Figure 1.

Figure 1: Directed Acyclic Graph of the IL-HTE Model

Notes: Squares indicate observed variables, hollow circles indicate latent
variables, and solid circles represent cross product interaction terms.
$I_{n}$ are item responses and $T_{j}$ is the treatment indicator. $\beta_{1}$
represents the average treatment effect. $\rho$ represents the correlation
between item location and item-specific treatment effect size. Path
coefficients are fixed at 1 unless otherwise indicated.

A large value of $\sigma_{\zeta}$ suggests that there is substantial IL-HTE in
the data. We can test whether item features can explain the IL-HTE by
interacting item properties (e.g., item subscale, item type, item modality,
etc.) with the treatment indicator to explain some of the IL-HTE. For example,
we can extend Eqn. 10 as follows:

$\displaystyle\theta_{j}$ $\displaystyle=\beta_{0}+\beta_{1}T_{j}+e_{j}$ (13)
$\displaystyle b_{ij}$
$\displaystyle=b_{i}+\gamma_{1}S_{i}+\gamma_{2}S_{i}\times
T_{j}+\zeta_{i}T_{j}.$ (14)

Here, $S_{i}$ is an indicator for whether item $i$ is part of subscale $S$.
Accordingly, $\beta_{1}$ represents the CATE for items in the reference set
(assumed here to be impacting $\theta_{j}$ directly), $\gamma_{1}$ is a main
effect for item location, and $\gamma_{2}$ provides the difference in the CATE
for items on subscale $S$. For example, education researchers have used the
subscale effects IL-HTE model to look at treatment effects on items related to
different reading comprehension passages [111, 109, 125].

Application of the EIRM to polytomous responses is less common, but it is
straightforward to extend the IL-HTE model to polytomous data. In short, we
can leverage the computational machinery of binary logistic regression to fit
polytomous models by reshaping the data to represent pairwise contrasts
between the ordered response categories. We then fit a logistic regression
model that includes fixed effects for item threshold parameters representing
the boundaries between each cut point on the scale [88]. If we assume that the
distances between thresholds are equal across items, the result is a Rating
Scale Model (RSM). In contrast, if we assume that each item has a unique
distance between each threshold, the result is a Partial Credit Model (PCM).
While such models are typically implemented with fixed item and threshold
parameters, they can be extended to the random item case that better allows
for IL-HTE modeling [126]. For readers interested in applying these models to
their own data sets, our references provide various tutorials in the R
programming language: the general EIRM for dichotomous items [96], extending
the EIRM to polytomous items [88], the IL-HTE model for dichotomous items
[109], and a Bayesian approach that allows for extensions such as 2PL models
or the Graded Response Model (GRM) [89, 109].

##  2 Methods

###  2.1 Monte Carlo Simulation

We use Monte Carlo simulation to examine the performance of the IL-HTE model
applied to polytomous item responses. Previous simulation studies of the IL-
HTE model using dichotomous item responses have demonstrated two key results.
First, in terms of statistical inference, they show that substantial IL-HTE
inflates the standard error of the ATE ($\text{SE}(\hat{\beta}_{1})$) but does
not cause bias [111]. Second, in terms of causal identification, they show
that the correlation $\rho$ between item location $b_{i}$ and item-specific
treatment effect size $\zeta_{i}$ induces spurious treatment by covariate
interaction effects, which is critical because it suggests that standard HTE
analysis of composite outcomes (e.g., Eqn. 1) can be misleading [112]. We
hypothesize that same pattern of results will apply to polytomous data,
because ordered logit models are approximately invariant to the collapsing of
response categories [145, p. 13].

We simulate polytomous item responses from the RSM and fit each model with the
glmer function from the lme4 package in R with fixed item thresholds and
random item location parameters [109, 88, 96, 126]. We fit two sets of
simulations, one to examine the effect of IL-HTE on ATEs and associated SEs
(i.e., replicating and extending [111]), and the other to examine the effect
of $\rho$ on treatment by covariate interaction effects (i.e., replicating and
extending [112]). We fix the following parameters across our simulations: 500
patients, 20 items, an ATE of .20 on the logit scale, $\sigma_{b}=1$,
$\sigma_{\theta}=.5$, and a baseline covariate with a coefficient of 1.

In our first set of simulations, we vary the following factors in a fully
crossed 3x3 design: the number of categories $k$ at 3, 5, 7 and
$\sigma_{\zeta}$ at 0, .2, .4 SDs representing no, moderate, and large IL-HTE.
We fit two models to each simulated data set, one assuming constant treatment
effects, the other allowing for IL-HTE. In our second set of simulations, we
fix $k=3$ and $\sigma_{\zeta}=.4$ and vary $\rho$ from -1 to 1 in increments
of .25 to determine how an estimated treatment by baseline covariate
interaction becomes biased when we assume a constant effects model. We fit two
models to each simulated data set, one allowing for a treatment by baseline
covariate interaction but constant item effects, the other allowing for both
the interaction and IL-HTE. We repeat the process 200 times per condition.

###  2.2 Empirical Application

####  2.2.1 The Hamilton Depression Rating Scale (HDRS)

The Hamilton Depression Rating Scale (HDRS) has been the de facto“gold
standard” in antidepressant research for over half a century [141]. The HDRS
has its origins in the 1950s in England and has since been applied worldwide
[116, 136]. The most commonly used version of the HDRS (which we use here)
consists of 17 polytomous (3- and 5-category) items [151] (HDRS-17).

There is a long history of psychometric analysis of the HDRS-17 and derivative
measures, including questioning its psychometric properties [81] or focusing
on the dimensionality and applying various techniques aimed at distinguishing
an HDRS subscale which can function as a unidimensional measure of depressive
severity across different populations and treatments [83, 107, 84]. Prior
analyses of item-level HDRS data include the reliability of subscales or items
[130], treatment heterogeneity by baseline depression severity [120], the
properties of the unidimensional HDRS-6 subscale (also known as the Bech or
melancholia subscale; items 1, 2, 7, 8, 10, 13) [84, 142, 137], and patterns
of treatment effects when individual items or subscales are analyzed
separately [118]. We emphasize that our analysis of the HDRS-17 data is
intended to be illustrative of the affordances of the IL-HTE model rather than
a definitive analysis of the measurement properties of the HDRS-17 scale. The
HDRS-17 items are summarized in Table 1.

Item Number | Subject | HDRS-6 Item | Range
---|---|---|---
1 | Depressed Mood | Yes | 0-4
2 | Feelings of Guilt | Yes | 0-4
3 | Suicide |  | 0-4
4 | Insomnia: Early Night |  | 0-2
5 | Insomnia: Middle Night |  | 0-2
6 | Insomnia: Early Morning |  | 0-2
7 | Work and Activities | Yes | 0-4
8 | Retardation | Yes | 0-4
9 | Agitation |  | 0-4
10 | Anxiety: Psychic | Yes | 0-4
11 | Anxiety: Somatic |  | 0-4
12 | Gastro-Intestinal Symptoms |  | 0-2
13 | General Somatic Symptoms | Yes | 0-2
14 | Genital Symptoms |  | 0-2
15 | Hypochondriasis |  | 0-4
16 | Loss of Weight |  | 0-2
17 | Insight |  | 0-2
Table 1: HDRS-17 Items

####  2.2.2 Data

For our empirical application, we use a subset of data from a prior study that
evaluated the effects of SSRIs on depression measured with the HDRS-17 [120].
Our sample is comprised of data from 28 RCTs comparing acute-phase SSRI
treatment to placebo in patients diagnosed with depression. 8262 participants
were included in the studies and in this analysis we focus on those 5313
patients who had HDRS-17 data available after six weeks of treatment (i.e.,
excluding patients who dropped out of treatment due to e.g., adverse events or
lack of efficacy; for further details see [118, 120]). In total 90313 person-
item combinations were included. There were 8 missing item responses; one
affordance of the EIRM approach is that it employs Maximum Likelihood for
missing item response data [110]. We included baseline HDRS-17 sum scores as a
covariate to improve the precision of our estimates.

####  2.2.3 Model Building Strategy

We fit a taxonomy of six primary models, summarized in Table 2. Models 1A, 1B,
and 1C use dichotomized item responses in a 1PL or Rasch model, where 0
indicates symptom absence and 1 indicates symptom presence. We begin with this
simpler dichotomous model to provide a baseline for interpretation and build
on past work of the IL-HTE model of dichotomous items, and to determine how
sensitive the results are to dichotomization. Models 2A, 2B, and 2C are RSMs
that account for the polytomous data, using fixed effects for average item
thresholds with random uniform item location shifts. Models 1A and 2A allow
for a constant treatment effect and provide a baseline for comparison (Eqn.
3). Models 1B and 2B allow for randomly varying IL-HTE by including a random
slope for treatment at the item level (Eqn. 8). Models 1C and 2C add a main
effect and interaction for the HDRS-6 subscale items (Eqn. 13) to determine
whether treatment effects systematically vary by subscale.

Label | Model | Treatment Effect
---|---|---
1A | 1PL | Constant TE
1B | 1PL | Randomly Varying IL-HTE
1C | 1PL | Subscale Effects IL-HTE
2A | RSM | Constant TE
2B | RSM | Randomly Varying IL-HTE
2C | RSM | Subscale Effects IL-HTE

Table 2: Taxonomy of EIRMs fit to the SSRI Data

Notes: 1PL = One-parameter logistic. RSM = Rating Scale Model. EIRM =
Explanatory Item Response Model.

We also examined two additional modeling strategies to probe the sensitivity
of our results. First, to examine how $\rho$ can induce spurious interaction
effects, we fit an additional set of models interacting treatment status with
baseline depression to determine how sensitive the model is to the inclusion
or exclusion of $\rho$. Second, while the RSM is not typically applied when
items have different numbers of response categories, we determined it would be
appropriate in our case because item responses with ratings of 4 or 5 were (a)
quite rare in our sample (see online supplemental materials) and (b)
represented a qualitatively more extreme range of symptoms than the 3-category
items. In our online supplement, we fit the more flexible PCM that allows for
separate thresholds for each item and show that the results of our analysis
are unchanged.

##  3 Results

###  3.1 Monte Carlo Simulation

Our simulation results replicate past findings from the dichotomous IL-HTE
model in the polytomous setting [111]. That is, in terms of statistical
inference, IL-HTE does not cause bias in treatment effect point estimates but
IL-HTE vastly inflates the SE of the ATE $\beta_{1}$. Figure 2 shows the
distribution of bias by condition and we see that the degree of bias is small
across all conditions and does not vary by constant effects or IL-HTE model.
While there is a slight negative bias in both models and at all levels of IL-
HTE as the number of response categories increases, the absolute magnitude of
the bias is small, and additional simulations in our online supplement show
that the magnitude of the bias approaches 0 as the number of items increases.
Figure 3 shows the calibration of the SEs of the model by plotting the mean
model-based SE against the observed SD of the treatment effect point estimates
(i.e., the empirical SE). A ratio of 100% indicates the mean model-based SE is
equivalent to the empirical SE. We clearly see that as IL-HTE increases, the
constant TE model SEs are systematically far too low, at about 50% of their
true values. In contrast, the SEs of the IL-HTE model are much better
calibrated regardless of the level of IL-HTE in the data, an important result
for accurate statistical inference.

The inflation of SEs occurs because IL-HTE accounts for an additional source
of uncertainty in the estimation of the ATE. That is, if there is IL-HTE in
the population of items from which a PROM is constructed, any finite draw of
items for a PROM administration will have a sample ATE that varies from the
population ATE due to sampling error. The random slope variance of the IL-HTE
model ($\sigma_{\zeta}$) accounts for this source of uncertainty and provides
adjusted estimates of the SE of the treatment effect, whereas the constant
effects model provides uncertainty consistent with the set of realized items
treated as fixed [111, 133]. The implication of this result is that
researchers may be vastly overestimating their precision when they assume a
constant effect model and intend to generalize their results to a latent trait
that could have been measured with an alternative set of items.

Figure 2: Estimated Bias of Treatment Main Effect by Simulation Condition

Notes: Each panel presents a different number of response categories $k$.

Figure 3: Estimated Standard Error Calibration of Treatment Main Effect by
Simulation Condition

Notes: Each panel presents a different number of response categories $k$. A
value of 100% indicates that the model-based SE was equivalent to the
empirical SE, on average.

Our second set of simulations examines the impact of $\rho$ on treatment by
baseline covariate interactions also confirms previous simulation findings
from dichotomous items [112]. That is, $\rho$ induces a spurious interaction
between treatment and baseline variables. Figure 4 shows the bias in an
estimated baseline by treatment interaction term (the true data-generating
value is zero), and we see that high magnitudes of $\rho$, both positive and
negative, induce a substantial bias in the estimated treatment by baseline
interaction term in the constant effects model. In contrast, the IL-HTE model
that allows for $\rho$ successfully eliminates this bias. This result suggests
that treatment by baseline covariate interaction effects estimated on PROMs
should be interpreted cautiously when item-level data is not available [99,
112].

Figure 4: Estimated Bias in Treatment by Baseline Interaction Term

###  3.2 Empirical Application

Model results are presented in Table 3. In Models 1A, 1B, 2A, and 2B, we see a
negative ATE of SSRIs on depression. That is, patients randomly assigned to
the SSRI condition reported substantial reductions in depression symptoms
compared to patients assigned to the placebo condition, at about -.25 units on
the logit scale ($p<.001$). The 1PL and RSM provide similar results in terms
of magnitude and statistical significance, though the z-statistics are
slightly larger in the RSM, consistent with the polytomous items providing
more information than the dichotomized items [145].

| M1A | M1B | M1C | M2A | M2B | M2C
---|---|---|---|---|---|---
1 = SSRI | $-.272^{***}$ | $-.274^{***}$ | $-.136^{*}$ | $-.224^{***}$ | $-.204^{***}$ | $-.111^{*}$
| $(.036)$ | $(.071)$ | $(.062)$ | $(.027)$ | $(.052)$ | $(.050)$
Baseline Depression (Std.) | $.279^{***}$ | $.280^{***}$ | $.280^{***}$ | $.200^{***}$ | $.201^{***}$ | $.201^{***}$
| $(.017)$ | $(.017)$ | $(.017)$ | $(.012)$ | $(.012)$ | $(.012)$
1 = HDRS6 Item |  |  | $1.840^{***}$ |  |  | $1.328^{***}$
|  |  | $(.489)$ |  |  | $(.376)$
SSRI x HDRS6 |  |  | $-.398^{***}$ |  |  | $-.265^{***}$
|  |  | $(.091)$ |  |  | $(.072)$
Threshold 2 |  |  |  | $-.764^{***}$ | $-.772^{***}$ | $-.772^{***}$
|  |  |  | $(.016)$ | $(.016)$ | $(.016)$
Threshold 3 |  |  |  | $-2.618^{***}$ | $-2.642^{***}$ | $-2.642^{***}$
|  |  |  | $(.029)$ | $(.029)$ | $(.029)$
Threshold 4 |  |  |  | $-3.818^{***}$ | $-3.870^{***}$ | $-3.872^{***}$
|  |  |  | $(.075)$ | $(.076)$ | $(.076)$
AIC | $99389.193$ | $99251.523$ | $99237.694$ | $146967.819$ | $146838.173$ | $146827.259$
BIC | $99436.249$ | $99317.400$ | $99322.393$ | $147046.163$ | $146936.103$ | $146944.775$
Log Likelihood | $-49689.597$ | $-49618.762$ | $-49609.847$ | $-73475.910$ | $-73409.086$ | $-73401.629$
Num. obs. | $90313$ | $90313$ | $90313$ | $132326$ | $132326$ | $132326$
Num. groups: PID | $5313$ | $5313$ | $5313$ | $5313$ | $5313$ | $5313$
Num. groups: itemID | $17$ | $17$ | $17$ | $17$ | $17$ | $17$
Var: PID (Intercept) | $1.119$ | $1.127$ | $1.128$ | $.562$ | $.571$ | $.570$
Var: itemID (Intercept) | $1.445$ | $1.678$ | $.898$ | $.867$ | $.985$ | $.549$
Var: itemID SSRI |  | $.064$ | $.026$ |  | $.034$ | $.017$
Cov: itemID (Intercept) SSRI |  | $-.183$ | $-.010$ |  | $-.090$ | $-.004$
${}^{***}p<0.001$; ${}^{**}p<0.01$; ${}^{*}p<0.05$

Table 3: EIRMs fit to the SSRI Data

Notes: PID = Person identifier. itemID = item identifier.

Comparing the constant treatment effect models (1A, 2A) to the IL-HTE models
(1B, 2B), we see substantial IL-HTE in the data, with large increases to the
log likelihood for the IL-HTE models and $\sigma_{\zeta}=.25$ for the 1PL
model and $.18$ for the RSM, SDs nearly as large as the point estimates for
the ATEs themselves. Accordingly, we can see that $\text{SE}(\hat{\beta_{1}})$
nearly doubles in both cases, capturing the added uncertainty of which items
were selected for the HDRS-17 from the population of potential items that
could have been selected. In other words, the estimated treatment effect of
SSRIs on depression measured with a different set of items from those that
could have been selected might be quite different than what was observed on
these items. The approximate doubling of $\text{SE}(\hat{\beta_{1}})$ is
equivalent to reducing the effective sample size by a factor of four.

An intuitive way to interpret the IL-HTE parameter $\sigma_{\zeta}$ is as a
measure of generalizability. That is, we can calculate a 95% prediction
interval (PI) for a range of item-specific treatment effects on out-of-sample
depression items (i.e., items that are similar to those on the HDRS-17), using
the formula $\text{PI}=\hat{\beta}_{1}\pm
1.96\sqrt{\sigma^{2}_{\zeta}+Var(\hat{\beta}_{1})}$ [86, p. 130]. Applied to
Models 1B and 2B, we see that,

$\displaystyle\text{PI}_{\text{M1B}}$ $\displaystyle=-.274\pm
1.96\sqrt{.064+.005}$ (15) $\displaystyle=-.274\pm.515$ (16)
$\displaystyle=[-.789,.241]$ (17) $\displaystyle\text{PI}_{\text{M2B}}$
$\displaystyle=-.204\pm 1.96\sqrt{.034+.003}$ (18)
$\displaystyle=-.204\pm.377$ (19) $\displaystyle=[-.581,.173],$ (20)

which suggests that treatment effects on items measuring other symptoms that
could reasonably be included in a depression rating scale could show anywhere
from large negative effects of SSRIs to moderate positive effects (i.e., be
made worse by SSRIs), a key finding for the generalizability of SSRI effects
on PROMs.

The correlation between item location and treatment effect size of about
$\rho=-.5$ in Models 1B and 2B suggests that the most commonly endorsed
symptoms saw the largest (negative) effects, as displayed in Figure 5, which
plots empirical Bayes estimates of item-specific treatment effects
($\beta_{1}+\zeta_{i}$, y-axis) against item location in the control group
($b_{i}$, x-axis). As discussed earlier, $\rho$ can create causal
identification challenges by inducing spurious interaction effects when not
appropriately modeled, as shown in our simulation results in Figure 4. We
examine this phenomenon in our online supplement by fitting additional models
that allow for a baseline depression by SSRI interaction, and see that the
inclusion of $\rho$ in the model shifts the point estimate of the interaction
effect, in line with our simulation results and prior research. However, the
shift in the estimated interaction term in our data set is not large in
magnitude, changing from -.015 in the constant effects model to -.024 in the
IL-HTE model, and neither is statistically significant. Figure 5 also
highlights the HDRS-6 items, and it appears that the treatment effects on the
HDRS-6 subscale are systematically larger than treatment effects on the 11
remaining items, and that the negative correlation may be partially or
entirely driven by the larger effects on the HDRS-6 subscale.

Figure 5: Correlation between Item Location and Item-Specific Treatment Effect
Size

Notes: Empirical Bayes estimates for item-specific treatment effect
($\beta_{1}+\zeta_{i}$) on item location ($b_{i}$) derived from Model 2B.

Models 1B and 2B show substantial IL-HTE and make appropriate adjustments to
our SEs, and as such are preferable to Models 1A and 2A. However, with such a
large value of $\sigma_{\zeta}$, a natural question is what item
characteristics might explain this IL-HTE. Accordingly, Models 1C and 2C
include HDRS-6 subscale by SSRI interaction effects to allow the treatment
effect to systematically differ between the HDRS-6 items and the remaining 11
items [111, 109, 125]. Both models confirm that treatment effects on HDRS-6
items are systematically larger than the other 11 items
($\beta_{\text{RSM}}=-.265,p<.001$). The main effect of treatment on the
remaining 11 items is negative and statistically significant, though much
reduced in magnitude ($\beta_{\text{RSM}}=-.111,p<.05$). These results are in
line with prior analysis of differential effects when each subscale is
considered separately [120]; one advantage of the IL-HTE model is that we
obtain a direct hypothesis test of differences in effect size by subscale in a
single model. The results of Model 2C are displayed in Figure 6, which shows
the fitted log-odds of exceeding the average category on an average item in
each subscale (y-axis) by baseline depression (x-axis) and treatment status
(color). The CATE is represented by the vertical distance between the fitted
lines, and we can see that it is much larger for the HDRS-6 items.

Figure 6: Conditional Average Treatment Effects of SSRIs on HDRS-6 Items and
the Remaining 11 Items

Notes: Estimates derived from Model 2C. Group 1 is the SSRI treatment, Group 0
is placebo.

The larger effects on the HDRS-6 subscale have important policy implications
for clinical trial evaluation and interpretation, because, “if researchers
could somehow a priori select those items known to be more sensitive to the
treatment, they would obtain a larger measured treatment impact as an artifact
of the selected items, rather than a truly more effective treatment” [111, p.
895]. In the context considered here, evaluations of SSRIs using HDRS-6 might
appear to be more effective than those using the full HDRS-17, not because the
treatment is more effective on depression as a whole, but because the specific
symptoms of depression assessed on the HDRS-6 subscale are more sensitive to
SSRIs. Conversely, it is also the case that some symptoms that are common in
depression covary also with other psychiatric and somatic conditions, as well
as with age and sex; insomnia being more common in old age, for example. In
such cases one might expect some “depressive symptoms” to persist after
remission of the depressive episode because in such a case they are not
causally related to the depression, but to an alternative explanation (e.g.,
added exogeneous causes of the item responses over and above $\eta_{ij}$ in
Figure 1). That is, such symptoms may reflect some construct-irrelevant
variance [100]. Indeed, it is well known that symptomatic remission as
measured by the HDRS-17 does not map particularly well to patient-defined
remission (in either direction) [155, 154]; such an inference is similarly
supported by our IL-HTE analysis.

Finally, we can calculate an approximate $R^{2}$ of the proportion of IL-HTE
explained by the SSRI by HDRS-6 interaction effect comparing
$\sigma^{2}_{\zeta}$ between models B and C. We see that the SSRI by HDRS-6
interaction explains 59% percent of the IL-HTE in the 1PL model and 50% in the
RSM, and that after accounting for the differential effects by subscale, our
estimate of $\rho$ goes nearly to 0, in line with Figure 5, suggesting that
the majority of the IL-HTE in the data is explained by the differential
subscale effects.

##  4 Discussion

Models for heterogeneous treatment effects (HTE) have typically focused on
person characteristics as moderators of treatment efficacy. While valuable,
such approaches may generate false positives given the inference,
generalizability, and identification challenges that arise when the outcome of
interest is a latent variable such as a PROM constructed from a set of items.
In this study, we extend novel Item Response Theory methods to the estimation
of item-level HTE (IL-HTE) using an analysis of clinical trial data of the
effects of SSRIs on depression as measured by the 17-item Hamilton Depression
Rating Scale as an illustration. As such, our study represents an important
contribution to epidemiological methodology, from both statistical and
substantive perspectives.

When IL-HTE is present in the data but ignored in the model, the standard
errors of treatment effects are underestimated, leading to spurious estimates
of precision and potential false positives. For example, the near doubling of
the SE of the treatment effect we observed in our empirical analysis is
equivalent to a reduction in the sample size by a factor of four, a finding
with implications for both statistical inference generally and prospective
power analysis. The IL-HTE model better accounts for the uncertainty of which
items were chosen for the PROM and provides a metric for how generalizable our
results might be if we were to use different items to assess treatment impact.
Similarly, the ability to model subscale effects can provide insight as to
which symptom clusters are most sensitive to treatment, allowing researchers
to test explicit hypotheses about differential subscale effects. Furthermore,
correlation between item location and item-specific treatment effect sizes can
induce spurious interactions between treatment and baseline characteristics
that only item-level analysis can appropriately identify. While spurious
interactions did not emerge in our empirical illustration, it would be prudent
to complement analyses that examine HTE by baseline covariates by the kind of
item-level analysis advocated here to ensure accurate identification of
interaction effects. Even when HTE by baseline covariates is not of primary
interest, non-zero correlations will also create bias in the main effects of
covariates, because the main effect in a model without interaction is an
average of the effects in each subgroup, weighted by subgroup sample size.

Substantively, the application of the IL-HTE model to the placebo-controlled
SSRI trials showed substantial IL-HTE in the HDRS-17. Estimates of the
standard deviation of item-specific treatment effects were nearly as large as
the point estimates for the average treatment effect themselves and standard
errors nearly doubled when we accounted for IL-HTE in the model. Prediction
intervals for out-of-sample items suggested that the impact of SSRIs could be
anywhere from slightly harmful to strongly beneficial. Such a fine-grained
level of insight on the extent of SSRI impact on depression would be masked by
a traditional analysis of a single number summary outcome, such as a sum
score.

Furthermore, our analysis of subscale effects on the HDRS-6 items showed
systematically higher treatment effects on HDRS-6 than the other eleven items,
a finding that has potentially critical policy implications for how we judge
the effectiveness of treatments using different outcome measures for the same
underlying construct such as depression. The HDRS-6, which was designed to be
a unidimensional measure of depressive severity, was developed well before
modern SSRIs came to market and it is therefore unlikely that researchers
developing the HDRS-6 would have chanced upon a collection of items that would
in the future make SSRIs appear particularly effective [83, 84], especially
considering that all efforts aimed at developing unidimensional subscales have
arrived at subscales that closely resemble the HDRS-6 (i.e., always including
the items depressed mood, feelings of guilt, work and activities, and psychic
anxiety) [81]. Nevertheless, the HTE by subscale has important implications
for the interpretation of HDRS-6 or other subscales as outcome measures given
that it demonstrates the extent to which efficacy estimates are conditional on
which rating instrument is applied [141, 130].

In the same vein, SSRIs—the antidepressant class examined in this study—have
well-known side effects related to sexual dysfunction, decreased appetite,
gastrointestinal complaints and insomnia [104]. These side effects, which are
present also in healthy volunteers [128], have been shown to correlate with
ratings on corresponding HDRS items thereby introducing another way in which
ratings of individual depressive symptoms may diverge from the latent variable
that they are intended to measure [119]. Notably, other antidepressants such
as amitriptyline and mirtazapine have the opposite effects in that they are
hypnotics and increase appetite and could therefore potentially introduce the
opposite biases, that is, making them appear as more effective antidepressants
when measured on a scale including many such items.

###  4.1 Limitations

Clearly, IL-HTE has both statistical and substantive implications for the
analysis of treatment effects in clinical trials and epidemiology, but we
acknowledge the following four limitations of the IL-HTE, building on the
limitations described in prior research [111, 112]. First, it is unknown how
common IL-HTE is in clinical trial or epidemiological PROMs data, and we leave
this largely as an open question. Some systematic evidence comes from
education research, where an analysis of 15 RCTs showed substantial IL-HTE,
including a case where 40% of the overall treatment effect was driven by a
single test item [79]. Therefore, future research using the IL-HTE model in
other clinical trial or epidemiological contexts has the potential to shed
light on how widespread the IL-HTE phenomenon is in this field.

Second, estimating the IL-HTE model necessitates the availability of item-
level data, which may not be available, particularly in secondary analyses. As
such, one practical implication of our results is that researchers should heed
calls to share item-level outcome data [98], so that researchers and secondary
analysts can evaluate the extent of IL-HTE in a data set given the statistical
and substantive insights allowed by IL-HTE analysis.

Third, the use of latent variable models such as the EIRM may be more
difficult to interpret and justify to practitioners, particularly when
coefficients are on the logit scale [87, 135]. One approach to improve
communicability is to convert the logit coefficients to standardized effect
sizes by dividing the regression coefficient by the estimated standard
deviation of the latent variable $\theta_{j}$, a value that can be derived
from model results [111, pp. 907-908].

Finally, the EIRM can be substantially more computationally demanding than
alternative models such as OLS regression because the cross-classified
multilevel structure of the item response data requires numerical integration
approaches [139] or Markov chain Monte Carlo (MCMC) methods in Bayesian
applications [89, 109]. Thus, when the number of items and persons is large,
IL-HTE methods may become computationally prohibitive.

###  4.2 Conclusion

In sum, applying measurement and psychometric principles to causal inference
in epidemiological and public health research provides a powerful opportunity
anywhere multi-item patient-reported outcome measures are used to assess
treatment impact. By using the IL-HTE model, both clinical and epidemiological
researchers can gain more accurate statistical inference, better estimates of
generalizability to new symptoms, and unbiased estimates of interaction
effects, all essential qualities for clinical trial and epidemiological
research that aims to inform public health policy.

##  5 Declarations

Research Ethics

Not applicable.

Informed Consent

Not applicable.

Author Contributions

Conceptualization: Author 1, Author 4

Methodology: Author 1, Author 4

Software: Author 1

Formal Analysis: Author 1, Author 2

Writing—original draft preparation: Author 1

Writing—review and editing: Author 1, Author 2, Author 3, Author 4

Competing Interests

Authors 1 and 4 report no conflicts of interest.

Author 2 has received speaker’s fees from Janssen Pharmaceuticals in the last
five years and is a board member of the Swedish Serotonin Society.

Author 3 has received speaker’s fees from Janssen Pharmaceuticals in the last
five years.

Research Funding

This work was funded in part by the Jacobs Foundation.

Data Availability

The data analyzed in this study are proprietary. However, we share a simulated
version of the data set based on the results of our models in our online
supplemental file. The code for this article is available as an online
supplemental file (https://researchbox.org/2494&PEER_REVIEW_passcode=HEOZYC).

The full HDRS-17 is available at the following URL:
https://dcf.psychiatry.ufl.edu/files/2011/05/HAMILTON-DEPRESSION.pdf

## References

  * [1] Ishita Ahmed et al.  “Heterogeneity of item-treatment interactions masks complexity and generalizability in randomized controlled trials”  In _Edworkingpapers. com_ , 2023
  * [2] Elena M Andresen et al.  “Performance of the 10-item Center for Epidemiologic Studies Depression scale for caregiving research”  In _SAGE Open Medicine_ 1 SAGE Publications Sage UK: London, England, 2013, pp. 2050312113514576
  * [3] R. Bagby, Andrew G. Ryder, Deborah R. Schuller and Margarita B. Marshall  “The Hamilton Depression Rating Scale: Has the Gold Standard Become a Lead Weight?”  In _American Journal of Psychiatry_ 161.12 American Psychiatric Association Publishing, 2004, pp. 2163–2177  DOI: 10.1176/appi.ajp.161.12.2163
  * [4] Brian Barger  “Epidemiology with psychometric spirit: MoBa leads autism’s interdisciplinary future—a commentary on Havdahl et al.(2023)”  In _Journal of Child Psychology and Psychiatry_ Wiley Online Library, 2023
  * [5] P. Bech et al.  “QUANTITATIVE RATING OF DEPRESSIVE STATES: Correlation Between Clinical Assessment, Beck’s Self‐Rating Scale and Hamilton’s Objective Rating Scale”  In _Acta Psychiatrica Scandinavica_ 51.3 Wiley, 1975, pp. 161–170  DOI: 10.1111/j.1600-0447.1975.tb00002.x
  * [6] Per Bech et al.  “The Hamilton Depression Scale: evaluation of objectivity using logistic models”  In _Acta Psychiatrica Scandinavica_ 63.3 Wiley Online Library, 1981, pp. 290–299
  * [7] Ettore Beghi et al.  “The epidemiology and treatment of ALS: focus on the heterogeneity of the disease and critical appraisal of therapeutic trials”  In _Amyotrophic Lateral Sclerosis_ 12.1 Taylor & Francis, 2011, pp. 1–10
  * [8] Michael Borenstein, Larry V Hedges, Julian PT Higgins and Hannah R Rothstein  “Introduction to meta-analysis”  John Wiley & Sons, 2009
  * [9] Richard Breen, Kristian Bernt Karlson and Anders Holm  “Interpreting and understanding logits, probits, and other nonlinear probability models”  In _annual review of sociology_ 44 Annual Reviews, 2018, pp. 39–54
  * [10] Okan Bulut, Guher Gorgun and Seyma Nur Yildirim-Erbasli  “Estimating explanatory extensions of dichotomous and polytomous Rasch models: The eirm package in R”  In _Psych_ 3.3 MDPI, 2021, pp. 308–321
  * [11] Paul-Christian Bürkner  “Bayesian Item Response Modeling in R with brms and Stan”  In _Journal of Statistical Software_ 100.5, 2021, pp. 1–54  DOI: 10.18637/jss.v100.i05
  * [12] Ana W Capuano et al.  “Modeling Likert scale outcomes with trend-proportional odds with and without cluster data”  In _Methodology_ Hogrefe Publishing, 2016
  * [13] Kitty S Chan et al.  “The interview mode effect on the Center for Epidemiological Studies Depression (CES-D) scale: an item response theory analysis”  In _Medical care_ JSTOR, 2004, pp. 281–289
  * [14] Karl Bang Christensen  “From Rasch scores to regression”  In _Journal of Applied Measurement_ 7.2, 2006, pp. 184
  * [15] Dami A Collier et al.  “Age-related immune response heterogeneity to SARS-CoV-2 vaccine BNT162b2”  In _Nature_ 596.7872 Nature Publishing Group UK London, 2021, pp. 417–422
  * [16] Cynthia P Cordero and Antonio L Dans  “Key concepts in clinical epidemiology: detecting and dealing with heterogeneity in meta-analyses”  In _Journal of Clinical Epidemiology_ 130 Elsevier, 2021, pp. 149–151
  * [17] Paul De Boeck  “Random item IRT models”  In _Psychometrika_ 73 Springer, 2008, pp. 533–559
  * [18] Paul De Boeck et al.  “The estimation of item response models with the lmer function from the lme4 package in R”  In _Journal of Statistical Software_ 39, 2011, pp. 1–28
  * [19] Paul De Boeck, Sun-Joo Cho and Mark Wilson  “Explanatory item response models”  In _The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications_ Wiley Online Library, 2016, pp. 247–266
  * [20] Ben Domingue and Klint Kanopka  “The Item Response Warehouse (IRW)”  PsyArXiv, 2023
  * [21] Benjamin W Domingue et al.  “Ubiquitous bias and false discovery due to model misspecification in analysis of statistical interactions: The role of the outcome’s distribution and metric properties.”  In _Psychological Methods_ , 2022
  * [22] Steven M Downing  “Threats to the validity of locally developed multiple-choice tests in medical education: Construct-irrelevant variance and construct underrepresentation”  In _Advances in Health Sciences Education_ 7 Springer, 2002, pp. 235–241
  * [23] Robert H Dworkin et al.  “Development and initial validation of an expanded and revised version of the Short-form McGill Pain Questionnaire (SF-MPQ-2)”  In _Pain®_ 144.1-2 Elsevier, 2009, pp. 35–42
  * [24] A Esterman  “The Likert scale”  In _Australasian epidemiologist_ 10.2, 2003
  * [25] João Faro-Viana et al.  “Population homogeneity for the antibody response to COVID-19 BNT162b2/Comirnaty vaccine is only reached after the second dose across all adult age ranges”  In _Nature Communications_ 13.1 Nature Publishing Group UK London, 2022, pp. 140
  * [26] James M. Ferguson  “SSRI Antidepressant Medications: Adverse Effects and Tolerability”  In _The Primary Care Companion For CNS Disorders_ 3.1 Physicians Postgraduate Press, Inc, 2001  DOI: 10.4088/pcc.v03n0105
  * [27] Jessica K Flake, Jolynn Pek and Eric Hehman  “Construct validation in social and personality research: Current practice and recommendations”  In _Social Psychological and Personality Science_ 8.4 Sage Publications Sage CA: Los Angeles, CA, 2017, pp. 370–378
  * [28] Jay C Fournier et al.  “Antidepressant drug effects and depression severity: a patient-level meta-analysis”  In _Jama_ 303.1 American Medical Association, 2010, pp. 47–53
  * [29] Robert D Gibbons, David C Clark and David J Kupfer  “Exactly what does the Hamilton depression rating scale measure?”  In _Journal of psychiatric research_ 27.3 Elsevier, 1993, pp. 259–273
  * [30] Joshua B Gilbert  “How measurement affects causal inference: Attenuation bias is (usually) more important than scoring weights”  In _Edworkingpapers. com_ , 2023
  * [31] Joshua B Gilbert  “Modeling item-level heterogeneous treatment effects: A tutorial with the glmer function from the lme4 package in R”  In _Behavior Research Methods_ Springer, 2023, pp. 1–13
  * [32] Joshua B Gilbert  “Estimating treatment effects with the explanatory item response model”  In _Journal of Research on Educational Effectiveness_ Taylor & Francis, 2024, pp. 1–19
  * [33] Joshua B Gilbert, James S Kim and Luke W Miratrix  “Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions”  In _Journal of Educational and Behavioral Statistics_ SAGE Publications Sage CA: Los Angeles, CA, 2023, pp. 10769986231171710
  * [34] Joshua B Gilbert, Luke W Miratrix, Mridul Joshi and Benjamin W Domingue  “Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity”, 2023  URL: https://doi.org/10.26300/6b7w-vp07
  * [35] M Maria Glymour  “Using causal diagrams to understand common problems in social epidemiology”  In _Methods in social epidemiology_ Jossey-Bass San Francisco, CA, 2006, pp. 393–428
  * [36] David A Grayson et al.  “Item bias in the Center for Epidemiologic Studies Depression Scale: effects of physical disorders and disability in an elderly community sample”  In _The Journals of Gerontology Series B: Psychological Sciences and Social Sciences_ 55.5 Oxford University Press, 2000, pp. P273–P282
  * [37] Sander Greenland, Judea Pearl and James M Robins  “Causal diagrams for epidemiologic research”  In _Epidemiology_ JSTOR, 1999, pp. 37–48
  * [38] M. Hamilton  “A RATING SCALE FOR DEPRESSION”  In _Journal of Neurology, Neurosurgery & Psychiatry_ 23.1 BMJ, 1960, pp. 56–62  DOI: 10.1136/jnnp.23.1.56
  * [39] Larry V Hedges  “Distribution theory for Glass’s estimator of effect size and related estimators”  In _journal of Educational Statistics_ 6.2 Sage Publications Sage CA: Thousand Oaks, CA, 1981, pp. 107–128
  * [40] F Hieronymus, J F Emilsson, S Nilsson and E Eriksson  “Consistent superiority of selective serotonin reuptake inhibitors over placebo in reducing depressed mood in patients with major depression”  In _Molecular Psychiatry_ 21.4 Springer ScienceBusiness Media LLC, 2015, pp. 523–530  DOI: 10.1038/mp.2015.53
  * [41] Fredrik Hieronymus, Alexander Lisinski, Staffan Nilsson and Elias Eriksson  “Influence of baseline severity on the effects of SSRIs in depression: an item-based, patient-level post-hoc analysis”  In _The Lancet Psychiatry_ 6.9 Elsevier, 2019, pp. 745–752
  * [42] Fredrik Hieronymus, Alexander Lisinski, Elias Eriksson and Søren Dinesen Østergaard  “Correction: Do side effects of antidepressants impact efficacy estimates based on the Hamilton Depression Rating Scale? A pooled patient-level analysis”  In _Translational Psychiatry_ 11.1 Springer ScienceBusiness Media LLC, 2021  DOI: 10.1038/s41398-021-01403-w
  * [43] Annika Jessen et al.  “Improving measurement efficiency of the Inner EAR scale with item response theory”  In _Otolaryngology–Head and Neck Surgery_ 158.6 SAGE Publications Sage CA: Los Angeles, CA, 2018, pp. 1093–1100
  * [44] Michael Joffe, Manoj Gambhir, Marc Chadeau-Hyam and Paolo Vineis  “Causal diagrams in systems epidemiology”  In _Emerging themes in epidemiology_ 9 Springer, 2012, pp. 1–18
  * [45] Richard N Jones  “Differential item functioning and its relevance to epidemiology”  In _Current epidemiology reports_ 6 Springer, 2019, pp. 174–183
  * [46] David M Kent et al.  “Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials”  In _International journal of epidemiology_ 45.6 Oxford University Press, 2016, pp. 2075–2088
  * [47] James S Kim et al.  “A longitudinal randomized trial of a sustained content literacy intervention from first to second grade: Transfer effects on students’ reading comprehension.”  In _Journal of Educational Psychology_ 115.1 American Psychological Association, 2023, pp. 73
  * [48] Jinho Kim and Mark Wilson  “Polytomous item explanatory item response theory models”  In _Educational and psychological measurement_ 80.4 Sage Publications Sage CA: Los Angeles, CA, 2020, pp. 726–755
  * [49] Irving Kirsch et al.  “Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration”  In _PLoS medicine_ 5.2 Public Library of Science San Francisco, USA, 2008, pp. e45
  * [50] Ulla Knorr, Jasmine Melissa Madsen and Lars Vedel Kessing  “The effect of selective serotonin reuptake inhibitors in healthy subjects revisited: A systematic review of the literature.”  In _Experimental and Clinical Psychopharmacology_ 27.5 American Psychological Association (APA), 2019, pp. 413–432  DOI: 10.1037/pha0000264
  * [51] Catherine R Lesko, Nicholas C Henderson and Ravi Varadhan  “Considerations when assessing heterogeneity of treatment effect in patient-centered outcomes research”  In _Journal of clinical epidemiology_ 100 Elsevier, 2018, pp. 22–31
  * [52] David A Luckenbaugh, Rezvan Ameli, Nancy E Brutsche and Carlos A Zarate Jr  “Rating depression over brief time intervals with the Hamilton Depression Rating Scale: standard vs. abbreviated scales”  In _Journal of psychiatric research_ 61 Elsevier, 2015, pp. 40–45
  * [53] Peter M McEvoy, Rachel Grove and Tim Slade  “Epidemiology of anxiety disorders in the Australian general population: findings of the 2007 Australian National Survey of Mental Health and Wellbeing”  In _Australian & New Zealand Journal of Psychiatry_ 45.11 Sage Publications Sage UK: London, England, 2011, pp. 957–967
  * [54] Daniel McNeish and Melissa Gordon Wolf  “Thinking twice about sum scores”  In _Behavior research methods_ 52 Springer, 2020, pp. 2287–2305
  * [55] Luke W Miratrix, Michael J Weiss and Brit Henderson  “An applied researcher’s guide to estimating effects from multisite individually randomized trials: Estimands, estimators, and estimates”  In _Journal of Research on Educational Effectiveness_ 14.1 Taylor & Francis, 2021, pp. 270–308
  * [56] Amanda K Montoya and Minjeong Jeon  “MIMIC models for uniform and nonuniform DIF as moderated mediation models”  In _Applied psychological measurement_ 44.2 SAGE Publications Sage CA: Los Angeles, CA, 2020, pp. 118–136
  * [57] Carina Mood  “Logistic regression: Why we cannot do what we think we can do, and what we can do about it”  In _European sociological review_ 26.1 Oxford University Press, 2010, pp. 67–82
  * [58] S Obeid et al.  “Validation of the Hamilton Depression Rating Scale (HDRS) and sociodemographic factors associated with Lebanese depressed patients”  In _L’encephale_ 44.5 Elsevier, 2018, pp. 397–402
  * [59] Seon-Cheol Park et al.  “Clinical validation of the Psychotic Depression Assessment Scale, Hamilton Depression Rating Scale-6, and Brief Psychiatric Rating Scale-5: results from the clinical research center for depression study”  In _Psychiatry Investigation_ 14.5 Korean Neuropsychiatric Association, 2017, pp. 568
  * [60] Yaacov Petscher, Donald L Compton, Laura Steacy and Hannah Kinnon  “Past perspectives and new opportunities for the explanatory item response model”  In _Annals of dyslexia_ 70 Springer, 2020, pp. 160–179
  * [61] Sophia Rabe-Hesketh and Anders Skrondal  “Multilevel and longitudinal modeling using Stata”  STATA press, 2022
  * [62] Sarah E Robertson, Andrew Leith, Christopher H Schmid and Issa J Dahabreh  “Assessing heterogeneity of treatment effects in observational studies”  In _American Journal of Epidemiology_ 190.6 Oxford University Press, 2021, pp. 1088–1100
  * [63] Henricus G Ruhé et al.  “Clinical use of the Hamilton depression rating scale: is increased efficiency possible? A post hoc comparison of Hamilton depression rating scale, Maier and Bech subscales, clinical global impression, and symptom Checklist-90 scores”  In _Comprehensive Psychiatry_ 46.6 Elsevier, 2005, pp. 417–427
  * [64] Augustus John Rush et al.  “Clinically significant changes in the 17-and 6-Item Hamilton Rating Scales for Depression: a STAR* D report”  In _Neuropsychiatric Disease and Treatment_ Taylor & Francis, 2021, pp. 2333–2345
  * [65] Tolulope T Sajobi et al.  “Unsupervised item response theory models for assessing sample heterogeneity in patient-reported outcomes measures”  In _Quality of Life Research_ Springer, 2023, pp. 1–12
  * [66] Adam Sales, Ethan Prihar, Neil Heffernan and John F Pane  “The Effect of an Intelligent Tutor on Performance on Specific Posttest Problems.”  In _International Educational Data Mining Society_ ERIC, 2021
  * [67] Fiona Steele  “Module 9: Single-level and multilevel models for ordinal responses concepts”  In _Centre for Multilevel Modelling, University of Bristol_ , 2011
  * [68] Pervez Sultan et al.  “Evaluation of domains of patient-reported outcome measures for recovery after childbirth: a scoping and systematic review”  In _JAMA network open_ 3.5 American Medical Association, 2020, pp. e205540–e205540
  * [69] Pervez Sultan et al.  “Use of patient-reported outcome measures to assess outpatient postpartum recovery: a systematic review”  In _JAMA Network Open_ 4.5 American Medical Association, 2021, pp. e2111600–e2111600
  * [70] Peter WG Tennant et al.  “Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations”  In _International journal of epidemiology_ 50.2 Oxford University Press, 2021, pp. 620–632
  * [71] Wim J Van der Linden  “Handbook of Item Response Theory: Volume 1: Models”  CRC press, 2017
  * [72] Ravi Varadhan et al.  “A framework for the analysis of heterogeneity of treatment effect in patient-centered outcomes research”  In _Journal of clinical epidemiology_ 66.8 Elsevier, 2013, pp. 818–825
  * [73] Janet B.. Williams  “Standardizing the Hamilton Depression Rating Scale: past, present, and future”  In _European Archives of Psychiatry and Clinical Neuroscience_ 251.S2 Springer ScienceBusiness Media LLC, 2001, pp. 6–12  DOI: 10.1007/bf03035120
  * [74] Mark Wilson and Paul De Boeck  “Descriptive and explanatory item response models”  In _Explanatory item response models: A generalized linear and nonlinear approach_ Springer, 2004, pp. 43–74
  * [75] Mark Wilson, Paul De Boeck and Claus H Carstensen  “Explanatory item response models: A brief introduction”  In _Assessment of competencies in educational contexts_ , 2008, pp. 91–120
  * [76] Mark Zimmerman et al.  “WHY DO SOME DEPRESSED OUTPATIENTS WHO ARE NOT IN REMISSION ACCORDING TO THE HAMILTON DEPRESSION RATING SCALE NONETHELESS CONSIDER THEMSELVES TO BE IN REMISSION?: Research Article: Why Do Some Depressed Outpatients Consider Themselves to be in Remission?”  In _Depression and Anxiety_ 29.10 Hindawi Limited, 2012, pp. 891–895  DOI: 10.1002/da.21987
  * [77] Mark Zimmerman et al.  “Why Do Some Depressed Outpatients Who Are in Remission According to the Hamilton Depression Rating Scale Not Consider Themselves to Be in Remission?”  In _The Journal of Clinical Psychiatry_ 73.06 Physicians Postgraduate Press, Inc, 2012, pp. 790–795  DOI: 10.4088/jcp.11m07203
  * [78] Aeilko H Zwinderman  “A generalized Rasch model for manifest predictors”  In _Psychometrika_ 56 Springer, 1991, pp. 589–600

## References

  * [79] Ishita Ahmed et al.  “Heterogeneity of item-treatment interactions masks complexity and generalizability in randomized controlled trials”  In _Edworkingpapers. com_ , 2023
  * [80] Elena M Andresen et al.  “Performance of the 10-item Center for Epidemiologic Studies Depression scale for caregiving research”  In _SAGE Open Medicine_ 1 SAGE Publications Sage UK: London, England, 2013, pp. 2050312113514576
  * [81] R. Bagby, Andrew G. Ryder, Deborah R. Schuller and Margarita B. Marshall  “The Hamilton Depression Rating Scale: Has the Gold Standard Become a Lead Weight?”  In _American Journal of Psychiatry_ 161.12 American Psychiatric Association Publishing, 2004, pp. 2163–2177  DOI: 10.1176/appi.ajp.161.12.2163
  * [82] Brian Barger  “Epidemiology with psychometric spirit: MoBa leads autism’s interdisciplinary future—a commentary on Havdahl et al.(2023)”  In _Journal of Child Psychology and Psychiatry_ Wiley Online Library, 2023
  * [83] P. Bech et al.  “QUANTITATIVE RATING OF DEPRESSIVE STATES: Correlation Between Clinical Assessment, Beck’s Self‐Rating Scale and Hamilton’s Objective Rating Scale”  In _Acta Psychiatrica Scandinavica_ 51.3 Wiley, 1975, pp. 161–170  DOI: 10.1111/j.1600-0447.1975.tb00002.x
  * [84] Per Bech et al.  “The Hamilton Depression Scale: evaluation of objectivity using logistic models”  In _Acta Psychiatrica Scandinavica_ 63.3 Wiley Online Library, 1981, pp. 290–299
  * [85] Ettore Beghi et al.  “The epidemiology and treatment of ALS: focus on the heterogeneity of the disease and critical appraisal of therapeutic trials”  In _Amyotrophic Lateral Sclerosis_ 12.1 Taylor & Francis, 2011, pp. 1–10
  * [86] Michael Borenstein, Larry V Hedges, Julian PT Higgins and Hannah R Rothstein  “Introduction to meta-analysis”  John Wiley & Sons, 2009
  * [87] Richard Breen, Kristian Bernt Karlson and Anders Holm  “Interpreting and understanding logits, probits, and other nonlinear probability models”  In _annual review of sociology_ 44 Annual Reviews, 2018, pp. 39–54
  * [88] Okan Bulut, Guher Gorgun and Seyma Nur Yildirim-Erbasli  “Estimating explanatory extensions of dichotomous and polytomous Rasch models: The eirm package in R”  In _Psych_ 3.3 MDPI, 2021, pp. 308–321
  * [89] Paul-Christian Bürkner  “Bayesian Item Response Modeling in R with brms and Stan”  In _Journal of Statistical Software_ 100.5, 2021, pp. 1–54  DOI: 10.18637/jss.v100.i05
  * [90] Ana W Capuano et al.  “Modeling Likert scale outcomes with trend-proportional odds with and without cluster data”  In _Methodology_ Hogrefe Publishing, 2016
  * [91] Kitty S Chan et al.  “The interview mode effect on the Center for Epidemiological Studies Depression (CES-D) scale: an item response theory analysis”  In _Medical care_ JSTOR, 2004, pp. 281–289
  * [92] Karl Bang Christensen  “From Rasch scores to regression”  In _Journal of Applied Measurement_ 7.2, 2006, pp. 184
  * [93] Dami A Collier et al.  “Age-related immune response heterogeneity to SARS-CoV-2 vaccine BNT162b2”  In _Nature_ 596.7872 Nature Publishing Group UK London, 2021, pp. 417–422
  * [94] Cynthia P Cordero and Antonio L Dans  “Key concepts in clinical epidemiology: detecting and dealing with heterogeneity in meta-analyses”  In _Journal of Clinical Epidemiology_ 130 Elsevier, 2021, pp. 149–151
  * [95] Paul De Boeck  “Random item IRT models”  In _Psychometrika_ 73 Springer, 2008, pp. 533–559
  * [96] Paul De Boeck et al.  “The estimation of item response models with the lmer function from the lme4 package in R”  In _Journal of Statistical Software_ 39, 2011, pp. 1–28
  * [97] Paul De Boeck, Sun-Joo Cho and Mark Wilson  “Explanatory item response models”  In _The Wiley handbook of cognition and assessment: Frameworks, methodologies, and applications_ Wiley Online Library, 2016, pp. 247–266
  * [98] Ben Domingue and Klint Kanopka  “The Item Response Warehouse (IRW)”  PsyArXiv, 2023
  * [99] Benjamin W Domingue et al.  “Ubiquitous bias and false discovery due to model misspecification in analysis of statistical interactions: The role of the outcome’s distribution and metric properties.”  In _Psychological Methods_ , 2022
  * [100] Steven M Downing  “Threats to the validity of locally developed multiple-choice tests in medical education: Construct-irrelevant variance and construct underrepresentation”  In _Advances in Health Sciences Education_ 7 Springer, 2002, pp. 235–241
  * [101] Robert H Dworkin et al.  “Development and initial validation of an expanded and revised version of the Short-form McGill Pain Questionnaire (SF-MPQ-2)”  In _Pain®_ 144.1-2 Elsevier, 2009, pp. 35–42
  * [102] A Esterman  “The Likert scale”  In _Australasian epidemiologist_ 10.2, 2003
  * [103] João Faro-Viana et al.  “Population homogeneity for the antibody response to COVID-19 BNT162b2/Comirnaty vaccine is only reached after the second dose across all adult age ranges”  In _Nature Communications_ 13.1 Nature Publishing Group UK London, 2022, pp. 140
  * [104] James M. Ferguson  “SSRI Antidepressant Medications: Adverse Effects and Tolerability”  In _The Primary Care Companion For CNS Disorders_ 3.1 Physicians Postgraduate Press, Inc, 2001  DOI: 10.4088/pcc.v03n0105
  * [105] Jessica K Flake, Jolynn Pek and Eric Hehman  “Construct validation in social and personality research: Current practice and recommendations”  In _Social Psychological and Personality Science_ 8.4 Sage Publications Sage CA: Los Angeles, CA, 2017, pp. 370–378
  * [106] Jay C Fournier et al.  “Antidepressant drug effects and depression severity: a patient-level meta-analysis”  In _Jama_ 303.1 American Medical Association, 2010, pp. 47–53
  * [107] Robert D Gibbons, David C Clark and David J Kupfer  “Exactly what does the Hamilton depression rating scale measure?”  In _Journal of psychiatric research_ 27.3 Elsevier, 1993, pp. 259–273
  * [108] Joshua B Gilbert  “How measurement affects causal inference: Attenuation bias is (usually) more important than scoring weights”  In _Edworkingpapers. com_ , 2023
  * [109] Joshua B Gilbert  “Modeling item-level heterogeneous treatment effects: A tutorial with the glmer function from the lme4 package in R”  In _Behavior Research Methods_ Springer, 2023, pp. 1–13
  * [110] Joshua B Gilbert  “Estimating treatment effects with the explanatory item response model”  In _Journal of Research on Educational Effectiveness_ Taylor & Francis, 2024, pp. 1–19
  * [111] Joshua B Gilbert, James S Kim and Luke W Miratrix  “Modeling Item-Level Heterogeneous Treatment Effects With the Explanatory Item Response Model: Leveraging Large-Scale Online Assessments to Pinpoint the Impact of Educational Interventions”  In _Journal of Educational and Behavioral Statistics_ SAGE Publications Sage CA: Los Angeles, CA, 2023, pp. 10769986231171710
  * [112] Joshua B Gilbert, Luke W Miratrix, Mridul Joshi and Benjamin W Domingue  “Disentangling Person-Dependent and Item-Dependent Causal Effects: Applications of Item Response Theory to the Estimation of Treatment Effect Heterogeneity”, 2023  URL: https://doi.org/10.26300/6b7w-vp07
  * [113] M Maria Glymour  “Using causal diagrams to understand common problems in social epidemiology”  In _Methods in social epidemiology_ Jossey-Bass San Francisco, CA, 2006, pp. 393–428
  * [114] David A Grayson et al.  “Item bias in the Center for Epidemiologic Studies Depression Scale: effects of physical disorders and disability in an elderly community sample”  In _The Journals of Gerontology Series B: Psychological Sciences and Social Sciences_ 55.5 Oxford University Press, 2000, pp. P273–P282
  * [115] Sander Greenland, Judea Pearl and James M Robins  “Causal diagrams for epidemiologic research”  In _Epidemiology_ JSTOR, 1999, pp. 37–48
  * [116] M. Hamilton  “A RATING SCALE FOR DEPRESSION”  In _Journal of Neurology, Neurosurgery & Psychiatry_ 23.1 BMJ, 1960, pp. 56–62  DOI: 10.1136/jnnp.23.1.56
  * [117] Larry V Hedges  “Distribution theory for Glass’s estimator of effect size and related estimators”  In _journal of Educational Statistics_ 6.2 Sage Publications Sage CA: Thousand Oaks, CA, 1981, pp. 107–128
  * [118] F Hieronymus, J F Emilsson, S Nilsson and E Eriksson  “Consistent superiority of selective serotonin reuptake inhibitors over placebo in reducing depressed mood in patients with major depression”  In _Molecular Psychiatry_ 21.4 Springer ScienceBusiness Media LLC, 2015, pp. 523–530  DOI: 10.1038/mp.2015.53
  * [119] Fredrik Hieronymus, Alexander Lisinski, Elias Eriksson and Søren Dinesen Østergaard  “Correction: Do side effects of antidepressants impact efficacy estimates based on the Hamilton Depression Rating Scale? A pooled patient-level analysis”  In _Translational Psychiatry_ 11.1 Springer ScienceBusiness Media LLC, 2021  DOI: 10.1038/s41398-021-01403-w
  * [120] Fredrik Hieronymus, Alexander Lisinski, Staffan Nilsson and Elias Eriksson  “Influence of baseline severity on the effects of SSRIs in depression: an item-based, patient-level post-hoc analysis”  In _The Lancet Psychiatry_ 6.9 Elsevier, 2019, pp. 745–752
  * [121] Annika Jessen et al.  “Improving measurement efficiency of the Inner EAR scale with item response theory”  In _Otolaryngology–Head and Neck Surgery_ 158.6 SAGE Publications Sage CA: Los Angeles, CA, 2018, pp. 1093–1100
  * [122] Michael Joffe, Manoj Gambhir, Marc Chadeau-Hyam and Paolo Vineis  “Causal diagrams in systems epidemiology”  In _Emerging themes in epidemiology_ 9 Springer, 2012, pp. 1–18
  * [123] Richard N Jones  “Differential item functioning and its relevance to epidemiology”  In _Current epidemiology reports_ 6 Springer, 2019, pp. 174–183
  * [124] David M Kent et al.  “Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials”  In _International journal of epidemiology_ 45.6 Oxford University Press, 2016, pp. 2075–2088
  * [125] James S Kim et al.  “A longitudinal randomized trial of a sustained content literacy intervention from first to second grade: Transfer effects on students’ reading comprehension.”  In _Journal of Educational Psychology_ 115.1 American Psychological Association, 2023, pp. 73
  * [126] Jinho Kim and Mark Wilson  “Polytomous item explanatory item response theory models”  In _Educational and psychological measurement_ 80.4 Sage Publications Sage CA: Los Angeles, CA, 2020, pp. 726–755
  * [127] Irving Kirsch et al.  “Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration”  In _PLoS medicine_ 5.2 Public Library of Science San Francisco, USA, 2008, pp. e45
  * [128] Ulla Knorr, Jasmine Melissa Madsen and Lars Vedel Kessing  “The effect of selective serotonin reuptake inhibitors in healthy subjects revisited: A systematic review of the literature.”  In _Experimental and Clinical Psychopharmacology_ 27.5 American Psychological Association (APA), 2019, pp. 413–432  DOI: 10.1037/pha0000264
  * [129] Catherine R Lesko, Nicholas C Henderson and Ravi Varadhan  “Considerations when assessing heterogeneity of treatment effect in patient-centered outcomes research”  In _Journal of clinical epidemiology_ 100 Elsevier, 2018, pp. 22–31
  * [130] David A Luckenbaugh, Rezvan Ameli, Nancy E Brutsche and Carlos A Zarate Jr  “Rating depression over brief time intervals with the Hamilton Depression Rating Scale: standard vs. abbreviated scales”  In _Journal of psychiatric research_ 61 Elsevier, 2015, pp. 40–45
  * [131] Peter M McEvoy, Rachel Grove and Tim Slade  “Epidemiology of anxiety disorders in the Australian general population: findings of the 2007 Australian National Survey of Mental Health and Wellbeing”  In _Australian & New Zealand Journal of Psychiatry_ 45.11 Sage Publications Sage UK: London, England, 2011, pp. 957–967
  * [132] Daniel McNeish and Melissa Gordon Wolf  “Thinking twice about sum scores”  In _Behavior research methods_ 52 Springer, 2020, pp. 2287–2305
  * [133] Luke W Miratrix, Michael J Weiss and Brit Henderson  “An applied researcher’s guide to estimating effects from multisite individually randomized trials: Estimands, estimators, and estimates”  In _Journal of Research on Educational Effectiveness_ 14.1 Taylor & Francis, 2021, pp. 270–308
  * [134] Amanda K Montoya and Minjeong Jeon  “MIMIC models for uniform and nonuniform DIF as moderated mediation models”  In _Applied psychological measurement_ 44.2 SAGE Publications Sage CA: Los Angeles, CA, 2020, pp. 118–136
  * [135] Carina Mood  “Logistic regression: Why we cannot do what we think we can do, and what we can do about it”  In _European sociological review_ 26.1 Oxford University Press, 2010, pp. 67–82
  * [136] S Obeid et al.  “Validation of the Hamilton Depression Rating Scale (HDRS) and sociodemographic factors associated with Lebanese depressed patients”  In _L’encephale_ 44.5 Elsevier, 2018, pp. 397–402
  * [137] Seon-Cheol Park et al.  “Clinical validation of the Psychotic Depression Assessment Scale, Hamilton Depression Rating Scale-6, and Brief Psychiatric Rating Scale-5: results from the clinical research center for depression study”  In _Psychiatry Investigation_ 14.5 Korean Neuropsychiatric Association, 2017, pp. 568
  * [138] Yaacov Petscher, Donald L Compton, Laura Steacy and Hannah Kinnon  “Past perspectives and new opportunities for the explanatory item response model”  In _Annals of dyslexia_ 70 Springer, 2020, pp. 160–179
  * [139] Sophia Rabe-Hesketh and Anders Skrondal  “Multilevel and longitudinal modeling using Stata”  STATA press, 2022
  * [140] Sarah E Robertson, Andrew Leith, Christopher H Schmid and Issa J Dahabreh  “Assessing heterogeneity of treatment effects in observational studies”  In _American Journal of Epidemiology_ 190.6 Oxford University Press, 2021, pp. 1088–1100
  * [141] Henricus G Ruhé et al.  “Clinical use of the Hamilton depression rating scale: is increased efficiency possible? A post hoc comparison of Hamilton depression rating scale, Maier and Bech subscales, clinical global impression, and symptom Checklist-90 scores”  In _Comprehensive Psychiatry_ 46.6 Elsevier, 2005, pp. 417–427
  * [142] Augustus John Rush et al.  “Clinically significant changes in the 17-and 6-Item Hamilton Rating Scales for Depression: a STAR* D report”  In _Neuropsychiatric Disease and Treatment_ Taylor & Francis, 2021, pp. 2333–2345
  * [143] Tolulope T Sajobi et al.  “Unsupervised item response theory models for assessing sample heterogeneity in patient-reported outcomes measures”  In _Quality of Life Research_ Springer, 2023, pp. 1–12
  * [144] Adam Sales, Ethan Prihar, Neil Heffernan and John F Pane  “The Effect of an Intelligent Tutor on Performance on Specific Posttest Problems.”  In _International Educational Data Mining Society_ ERIC, 2021
  * [145] Fiona Steele  “Module 9: Single-level and multilevel models for ordinal responses concepts”  In _Centre for Multilevel Modelling, University of Bristol_ , 2011
  * [146] Pervez Sultan et al.  “Evaluation of domains of patient-reported outcome measures for recovery after childbirth: a scoping and systematic review”  In _JAMA network open_ 3.5 American Medical Association, 2020, pp. e205540–e205540
  * [147] Pervez Sultan et al.  “Use of patient-reported outcome measures to assess outpatient postpartum recovery: a systematic review”  In _JAMA Network Open_ 4.5 American Medical Association, 2021, pp. e2111600–e2111600
  * [148] Peter WG Tennant et al.  “Use of directed acyclic graphs (DAGs) to identify confounders in applied health research: review and recommendations”  In _International journal of epidemiology_ 50.2 Oxford University Press, 2021, pp. 620–632
  * [149] Wim J Van der Linden  “Handbook of Item Response Theory: Volume 1: Models”  CRC press, 2017
  * [150] Ravi Varadhan et al.  “A framework for the analysis of heterogeneity of treatment effect in patient-centered outcomes research”  In _Journal of clinical epidemiology_ 66.8 Elsevier, 2013, pp. 818–825
  * [151] Janet B.. Williams  “Standardizing the Hamilton Depression Rating Scale: past, present, and future”  In _European Archives of Psychiatry and Clinical Neuroscience_ 251.S2 Springer ScienceBusiness Media LLC, 2001, pp. 6–12  DOI: 10.1007/bf03035120
  * [152] Mark Wilson and Paul De Boeck  “Descriptive and explanatory item response models”  In _Explanatory item response models: A generalized linear and nonlinear approach_ Springer, 2004, pp. 43–74
  * [153] Mark Wilson, Paul De Boeck and Claus H Carstensen  “Explanatory item response models: A brief introduction”  In _Assessment of competencies in educational contexts_ , 2008, pp. 91–120
  * [154] Mark Zimmerman et al.  “WHY DO SOME DEPRESSED OUTPATIENTS WHO ARE NOT IN REMISSION ACCORDING TO THE HAMILTON DEPRESSION RATING SCALE NONETHELESS CONSIDER THEMSELVES TO BE IN REMISSION?: Research Article: Why Do Some Depressed Outpatients Consider Themselves to be in Remission?”  In _Depression and Anxiety_ 29.10 Hindawi Limited, 2012, pp. 891–895  DOI: 10.1002/da.21987
  * [155] Mark Zimmerman et al.  “Why Do Some Depressed Outpatients Who Are in Remission According to the Hamilton Depression Rating Scale Not Consider Themselves to Be in Remission?”  In _The Journal of Clinical Psychiatry_ 73.06 Physicians Postgraduate Press, Inc, 2012, pp. 790–795  DOI: 10.4088/jcp.11m07203
  * [156] Aeilko H Zwinderman  “A generalized Rasch model for manifest predictors”  In _Psychometrika_ 56 Springer, 1991, pp. 589–600