# Perspectives on ‘harm’ in personalized medicine

Aaron L. Sarvet  Mats J. Stensrud  Department of Mathematics, École
Polytechnique Fédérale de Lausanne, Switzerland

###### Abstract

Avoiding harm is an uncontroversial aim of personalized medicine and other
epidemiologic initiatives. However, the precise mathematical translation of
“harm” is disputable. Here we use a formal causal language to study common,
but distinct, definitions of “harm”. We clarify that commitment to a
definition of harm has important practical and philosophical implications for
decision making. We relate our practical and philosophical considerations to
ideas from medical ethics and legal practice.

##  1 Introduction

Personalized medicine aims to leverage observed data to personalize treatment
assignment in the future. When treatments are personalized, their assignment
depends on patient characteristics. Individualizing treatments is justifiable
if it results in better (expected) utility in the population. In some cases,
personalized treatments are (biologically) engineered to be compatible with
known patient features, for example with immunologically-targeted cancer
therapies. In other cases, several treatments already exist, and the optimal
treatment for each patient can be learned from data. Finding the optimal
treatment requires causal inference: we need to identify and subsequently
estimate the patient’s expected utilities under each of the treatment options
that are available (Robins, 1986; Richardson and Robins, 2013; Murphy, 2003;
Robins, 2004; Tsiatis et al., 2019; Kosorok et al., 2021).

Specifying utility functions – the mapping of a patient’s outcomes and other
features to some number – is a fundamental step in posing this task, because
it forms the basis for determining preferences among treatment options.
However, this step is often avoided by epidemiologists and statisticians,
perhaps because of its philosophical and subjective reputation. Yet, decision
makers and methodologists will usually agree, at least superficially, with the
classical Hippocratic maxim: “First, do no harm”.

Consideration of “harm” is the focus of this article. We anchor results in an
emerging literature offering an operationalization of “harm” that promises to
possibly ‘provide the key for next-generation personalized decision making.’
(Mueller and Pearl, 2022). Their methods propose strategies for supplementing
experimental data with confounded non-experimental data that are often
disregarded in classical approaches (Tian and Pearl, 2000). We demonstrate
that their proposals take for granted a particular operationalization of
“harm”, and we contrast it with notions of harm defined by parameters
identified by experimental data. To bring clarity, we articulate two
alternative approaches, which we refer to as counterfactual and
interventionist, respectively. These two approaches are distinguished by
utility functions of fundamentally different natures. Furthermore, they each
correspond to different notions of “harm” and to different interpretations of
the Hippocratic principle.

##  2 Preliminaries

Suppose we observe a random sample of patients from a large population. We let
$Y$ be the outcome, $A$ be a binary treatment, and $L$ be baseline covariates.

Throughout, we will use potential outcomes notation to denote the outcome that
would occur under an intervention that fixes treatment to a specific value
(Rubin, 1974) (see Web Appendix 1 for additional remarks). For example,
$Y^{a=1}$ denotes a patient’s potential outcome that would occur had $A$ been
fixed, possibly counter to fact, to $a=1$.

We consider $\\{L,A,Y,Y^{a=1},Y^{a=0}\\}$ to be the full data, of which only
$\\{L,A,Y\\}$ are observed. When discussing the counterfactual approach we use
$S$ to denote the joint principal stratum variable $\\{Y^{a=1},Y^{a=0}\\}$
(Robins, 1986; Frangakis and Rubin, 2002), where the events
$(S=1,S=2,S=3,S=4)$ correspond to the events
$(\\{Y^{a=1}=1,Y^{a=0}=0\\},\\{Y^{a=1}=0,Y^{a=0}=1\\},\\{Y^{a=1}=1,Y^{a=0}=1\\},\\{Y^{a=1}=0,Y^{a=0}=0\\})$.

###  2.1 Defining utility functions

Utilities are crucial to theories about decisions (Von Neumann and
Morgenstern, 2007). The simplest utility function is one that is equal to a
patient’s outcome under a treatment $a$, $Y^{a}$. A more elaborate utility
function might depend further on the treatment actually received. We will
consider one such utility $U^{a}_{1}$ under $a$,

$U_{1}^{a}=\mu(Y^{a},a).$

This utility is defined solely by variables that are directly observable under
treatment $a$. We will refer to these utilities as interventionist utilities.

In principle, however, utilities may be functions of any subset of well-
defined patient variables, for example, the potential outcome corresponding to
a treatment that was not assigned to that patient. In contrast to an
interventionist utility, this utility is defined in part by counterfactual
variables that would not be directly observable under treatment $a$. We will
thus refer to these utilities as counterfactual utilities. A special case is a
$U_{2}^{a}$ that depends simultaneously on the interventionist utilities that
would arise under either treatment level:

$\displaystyle U_{2}^{a}=$
$\displaystyle\gamma^{*}(\mu(Y^{a},a),\mu(Y^{1-a},1-a),a)$ (1)
$\displaystyle=$ $\displaystyle\gamma(S,a).$ (2)

$U_{2}^{a}$ may alternatively be understood as a function of a patient’s
principal strata $S=\\{Y^{a=1},Y^{a=0}\\}$ and their assigned treatment $a$. A
patient or investigator specifying a counterfactual utility may either specify
$\mu$ and $\gamma^{*}$, as in (1), or alternatively may specify $\gamma$
directly, as in (2).

We refer to the counterfactual and interventionist approaches as those that
use counterfactual and interventionist utility functions, respectively.

##  3 Twin definitions of harm in personalized medicine

Recital of the Hippocratic Oath “first, do no harm” is a common ritual for
clinical trainees (Orr et al., 1997). However, there is broad consensus that
the Oath is not a sufficiently rich formulation of a modern medical ethics,
nor does its meaning unambiguously speak for itself (Beauchamp and Childress,
1994). Nevertheless, this has not prevented some contemporary authors in
causal inference from claiming a singular mathematical translation: for
example Ben-Michael et al. (2022) identify the “first, do no harm” principle
with a particular specification of a counterfactual utility function,
$\gamma$, such that

$\displaystyle\gamma(s=1,a=0)-\gamma(s=1,a=1)>\gamma(s=2,a=1)-\gamma(s=2,a=0).$
(3)

When $Y$ indicates patient death, then $S=1$ indicates a patient that would
only die under treatment (the “harmed”), and $S=2$ indicates a patient who
would only survive under treatment (the “saved”). Thus, Ben-Michael et al.
(2022) consider a $\gamma$ such that the utility gain upon withholding
treatment from a patient who would be “harmed”,
$\gamma(s=1,a=0)-\gamma(s=1,a=1)$, is greater than that gain upon giving
treatment to a patient who would be “saved”,
$\gamma(s=2,a=1)-\gamma(s=2,a=0)$.

These appear to be symmetric actions and thus should be assigned equal gain:
both decisions presumably save a life, by selecting the only decision
resulting in a favorable outcome. However, according to the translation of
harm adopted by (Ben-Michael et al., 2022), treatment $a=1$ is privileged with
the capacity to “do”. Thus, the first gain apparently avoids an error of
comission, whereas the second gain avoids an error of omission. The asymmetry
apparently derives from the refrain to “first, do no harm”, as in “first” make
no errors of comission. Ben-Michael et al. (2022, pg. 8) assert that this
particular type of asymmetry is “encoded by the principle to ’do no harm’ in
the Hippocratic oath”. Their translation motivates the following definition of
counterfactual harm:

######  Definition 1 (Counterfactual harm).

Suppose a patient is assigned treatment $a=1$ (relative to no treatment,
$a=0$) and that their interventionist utility under treatment is less than
their utility under no treatment, $\mu(Y^{a=1},a=1)<\mu(Y^{a=0},a=0)$. This is
counterfactual harm.

In other words, counterfactual harm occurs when a patient takes treatment
$a=1$ but their personal interventionist utility is not maximized by this
treatment value. We call this counterfactual because its definition depends on
at least one potential outcome for a patient that is in principle impossible
to observe. We associate this notion of counterfactual harm with the proposals
of Ben-Michael et al. (2022) and Li and Pearl (2019); Mueller and Pearl
(2022), because their proposals also require consideration of a patient’s
principal stratum.

While Definition 1 is an ostensibly natural interpretation of “harm” in the
Hippocratic sense, examination of modern medical ethics suggests that an
alternative formulation may operate in practice. Beauchamp and Childress
(1994) noted that the principle of Nonmaleficence is treated identically to
the principle of “first, do no harm”. However, they distinguish between cases
where negative outcomes are presumably caused almost surely, and cases where
merely the risk of harm is caused by an agent. For example, Beauchamp and
Childress (1994, pg. 154) state that “Obligations of nonmaleficence include
not only obligations not to inflict harms, but also obligations not to impose
risks of harm.” In such cases where certainty is forgone, maleficence is
identified with a deviation from a “standard of due care”. But in modern
evidence-based medicine, several standards of care are often available, i.e.,
whenever a second treatment $a=1$ is accepted as non-inferior to a first
treatment $a=0$. These determinations are often based on clinical trials, in
which $\mathbb{E}[Y^{a=1}]$ and $\mathbb{E}[Y^{a=0}]$ are identified.

Consider then the following example: suppose $U^{a}_{1}=(1-Y^{a})$ defined a
particular interventionist utility and thus
$\\{Y^{a=1}>Y^{a=0}\\}\equiv\\{S=1\\}$ constitutes counterfactual harm. It is
possible that both $P(S=1)>0$ and $\mathbb{E}[Y^{a=1}]=\mathbb{E}[Y^{a=0}]$;
some patients are counterfactually harmed by treatment in the sense of
Definition 1, yet a physician could still be considered to have followed “due
standard of care” when assigning $a=1$. This has an immediate, important
implication: if upholding a “due standard of care” is equivalent to upholding
the principle of nonmaleficence, and if nonmaleficence is equivalent to the
principle of “first, do no harm”, then we cannot simply equate “harm” in the
Hippocratic sense with counterfactual harm in Definition 1. This argument
motivates an alternative definition more concordant with nonmaleficence in
modern bioethics when outcomes are uncertain, which is nearly always the case.
To state this definition of harm, we first define the patient features that a
provider must consider in order to uphold “due standard of care”.

######  Definition 2 (Relevant features).

Let $W$ denote the features taking well-defined values prior to treatment. Let
$X$ denote the subset of $W$ that is measured prior to treatment and suppose a
patient has a realization $X=x$. Let $X^{\prime}$ be the largest subset of $X$
such that, for $x^{\prime}\subset x$,

$\text{sign}\Big{\\{}\mathbb{E}[Y^{a=1}\mid
X^{\prime}=x^{\prime}]-\mathbb{E}[Y^{a=0}\mid X^{\prime}=x^{\prime}]\Big{\\}}$

is taken to be fixed and known by the investigator for the purpose of the
decision task. Then $X^{\prime}$ are the relevant features for that patient.

Definition 2 identifies a patient’s features that can be practically used to
make a personalized comparison of expected outcomes under either treatment.
For practical purposes, the sign of the ATE for patients with relevant
features $x^{\prime}$ might be taken as fixed and known upon consideration of
subject matter knowledge, the results of sufficiently large clinical trials,
or other empirical evidence (e.g., consistent estimates of conditional causal
effects from the data on hand.) Now we can give a second definition of “harm”
that we refer to as interventionist harm.

######  Definition 3 (Interventionist harm).

Suppose $L$ represents the relevant features for a patient. Suppose a patient
is assigned treatment $a=1$ (relative to no treatment, $a=0$) and that the
expected potential utility under treatment, given $L=l$, is less than that
expected utility under no treatment,

$\mathbb{E}[\mu(Y^{a=1},a=1)\mid L=l]<\mathbb{E}[\mu(Y^{a=0},a=0)\mid L=l].$

This is interventionist harm.

Thus, interventionist harm occurs when a patient takes treatment $a=1$ but
their expected interventionist utility conditional on their relevant features
is not maximized. This is an interventionist notion because the parameters
needed to determine its realization are observable in experiments.
Furthermore, it motivates utilities based on factual responses observable
post-hoc for a given patient.

##  4 Decision theory

It is classically assumed that rational agents in a future decision setting
will select the treatment $a$ maximizing their expected utility (Von Neumann
and Morgenstern, 2007), $\mathbb{E}[U^{a}]$. A patient would leverage both
their knowledge of their own features at the decision point, and knowledge of
the distribution of the full data; that is, they will base their decisions on
the parameters of $P$ corresponding to their relevant features. For example,
when a patient observes the value of their baseline relevant features $L=l$,
then they would choose the treatment $a$ equal to

$\text{argmax}_{a}\mathbb{E}[U^{a}\mid L=l].$

Suppose a patient with relevant features $X^{\prime}=x^{\prime}$ subscribes to
a counterfactual notion of harm (Definition 1). Thus, they specify a
counterfactual utility function $\gamma$ via specification of an
interventionist utility function $\mu$ and its modifying function
$\gamma^{*}$. Let $U^{a}_{2}$ define this utility. If this patient is a
rational agent, they will choose the treatment $a$ maximizing

$\mathbb{E}[U_{2}^{a}\mid X^{\prime}=x^{\prime}]=\sum_{s}\gamma(s,a)P(S=s\mid
X^{\prime}=x^{\prime}).$

Alternatively, suppose this patient rather subscribes to an interventionist
notion of harm (Definition 3). Thus, they merely specify an interventionist
utility function $\mu$. Let $U^{a}_{1}$ denote this interventionist utility
under $\mu$. If this patient is a rational agent, they will choose the
treatment $a$ maximizing

$\mathbb{E}[U_{1}^{a}\mid X^{\prime}=x^{\prime}]=\sum_{y}\mu(y,a)P(Y^{a}=y\mid
X^{\prime}=x^{\prime}).$

##  5 Practical considerations

###  5.1 Utility maximization with experimental data

Consider the practical task of selecting a treatment using parameters of the
observed data. For a patient maximizing interventionist utility, this is
reducible to identifying $\mathbb{E}[Y^{a}\mid L=l]$ for each $a$ in terms of
parameters of those observed data. When data arise from an ideal experiment,
we identify $\mathbb{E}[Y^{a}\mid L=l]$ by

$\displaystyle\mathbb{E}[Y^{a}\mid L=l]=$ $\displaystyle\mathbb{E}[Y\mid
L=l,A=a].$ (4)

This identity follows from assumptions enforced by design of the experiment
(see Web Appendix 2 for explicit arguments). With these parameters and a
specified interventionist utility function $\mu$, then $L$ would constitute
each patient’s relevant covariate. A patient with $L=l$ can readily choose the
treatment $a$ maximizing expected utility via

$\text{argmax}_{a}\mathbb{E}[Y\mid L=l,A=a].$

For a patient maximizing counterfactual utility, this task is reducible to
identifying $P(S=s\mid L=l)$ for each $s$, but is more challenging than
maximizing interventionist utility; identification is generally not possible
under only those assumptions enforced by an ideal randomized trial.

Alternatively, the values of $P(S=s\mid L=l)$ may be partially determined
under such assumptions by leveraging constraints imposed by identified
quantities. Specifically these constraints can be used to construct logical
upper and lower limits, i.e., bounds, on the principal stratum probabilities,
which fully leverage these assumptions. When upper and lower limits coincide,
probabilities are identified. Otherwise, the true value of the probabilities
within the bounds will depend ultimately on a parameter whose value is
completely uninformed by the observed data in this setting.

We review two cases when bounds are sufficient for identifying the treatment
$a$ maximizing $\mathbb{E}[U_{2}^{a}\mid L=l]$. A first case is when the upper
and lower limits coincide, so that the joint distribution of
$\\{Y^{a=1},Y^{a=0}\\}$ is fully determined by the experimental data, which
occurs only in the case that, for at least one treatment level $a$, the
resulting outcome is predicted with certainty. Thus, the conditional
distribution of $S$ is point identified, and $\mathbb{E}[U_{2}^{a}\mid L=l]$
is evaluated directly. This is a condition that is verifiable with
experimental data, i.e. when $P(Y^{a}=y\mid L=l)$ is identified.

Second, certain specifications of $\gamma$ permit maximization even when only
bounds for $P(S=s\mid L=l)$ are available. If we take
$\delta(s)=\gamma(s,a=1)-\gamma(s,a=0)$ to be the counterfactual utility gain
for choosing treatment $a=1$ among patients with principal stratum $s$, we can
re-express $\mathbb{E}[U_{2}^{a=1}\mid L=l]-\mathbb{E}[U_{2}^{a=0}\mid L=l]$
as

$\displaystyle\sum_{s}\delta(s)P(S=s\mid L=l).$ (5)

Particularly, when $\gamma$ is such that

$\displaystyle\delta(1)+\delta(2)=\delta(3)+\delta(4),$ (6)

then $\mathbb{E}[U_{2}^{a=1}\mid L=l]-\mathbb{E}[U_{2}^{a=0}\mid L=l]$ can be
re-expressed as

$\displaystyle\Bigg{\\{}\Big{(}\delta(1)-\delta(4)\Big{)}P(Y^{a=1}=1\mid
L=l)\Bigg{\\}}+\Bigg{\\{}\Big{(}\delta(3)-\delta(1)\Big{)}P(Y^{a=0}=1\mid
L=l)\Bigg{\\}}+\delta(4).$ (7)

Expression (7) is only a function of the investigator-specified $\delta$, and
terms like $P(Y^{a}=y\mid L=l)$, which are identified in an experiment. The
constraint on $\gamma$ in expression (6) has been called “gain equality” by
some authors (Li and Pearl, 2019, 2022b). Ben-Michael et al. (2022) consider a
special class of $\gamma$, called “symmetric” utility functions, which also
satisfy “gain equality”. Such functions weigh equally errors of “comission”
and errors of “omission” and thus would not satisfy their interpretation of
the “first, do no harm principle“.

In general, however, patients who would like to choose the treatment $a$
maximizing their counterfactual utility, using only assumptions that could be
experimentally enforced, will have to proceed with only the knowledge that
$P(S=s\mid L=l)$ must be contained within an interval. Building on results in
Cui (2021), Ben-Michael et al. (2022) illustrate that, while the treatment
value $a^{*}$ that maximizes $\mathbb{E}[U_{2}^{a}\mid L=l]$ is not in general
identified, one can identify the value of treatment $a^{\prime}$ that
minimizes the worst case regret, relative to $a^{*}$ (Savage, 1951), and
suggest that such a selection procedure be used by a patient. Li and Pearl
(2019) heuristically suggest an identical procedure, based on the results of
two simulated examples. Although regret minimization allows decisions to be
made under uncertainty, we note that its use represents a departure from
classical decision theory.

###  5.2 On the fusion of experimental & non-experimental data

For illustrative purposes, we will suppose that no baseline covariates have
been measured, i.e. that $L\equiv\emptyset$, but the arguments can
straightforwardly be extended to settings with measured baseline covariates.

####  5.2.1 Counterfactual approach

Tian and Pearl (2000) illustrate how bounds for $P(S=s)$ identified by
experimental data can be narrowed by supplementing with confounded non-
experimental data. To provide some intuition for this result, consider an
arbitrary treatment regime $g$. The average treatment effect (ATE) comparing
“always treat”, $a=1$, and $g$, that is

$\mathbb{E}[Y^{a=1}]-\mathbb{E}[Y^{g}]$

is a valid lower bound for $P(S=1)$ (See Proposition 2 in Web Appendix 2). As
an example consider the special case where $g$ is “never treat”, $a=0$.
Greenland and Robins (1986) classically illustrated that, for a binary
$Y\in\\{0,1\\}$,

$\mathbb{E}[Y^{a=1}]-\mathbb{E}[Y^{a=0}]=P(S=1)-P(S=2).$

Thus, $P(S=1)$ can be no less than this ATE, and it is only greater than this
ATE by an amount equal to $P(S=2)$. Among all the average treatment effects
identified by the experimental data, it is this special case which maximizes
the lower bound (See Proposition 3 in Web Appendix 2). For other regimes $g$,
which possibly depend on unmeasured features, an improved lower bound could be
obtained if not for the fact that no such regime is identified by such data.
Thus, it is this canonical ATE that is typically used to compute sharp
nonparametric bounds. However, when confounded non-experimental data are
additionally used, the expected outcome under one such regime is indeed
identified: the factual (but unknown) regime that was used to generate that
non-experimental data, which is $\mathbb{E}[Y]$. This parameter then may be
used as one additional chance to improve the lower bound on $P(S=1)$, as
illustrated by Tian and Pearl (2000). We provide an articulation of these
improved bounds in terms of our notation and conventional assumptions in Web
Appendix 3.

While this result has been known for decades, it is currently emphasized by Li
and Pearl (2022a) and also Mueller and Pearl (2022) as an important practical
tool for implementing a counterfactual approach.

####  5.2.2 Interventionist approach

The allocation of treatment to patients in practice can be decomposed into two
steps: 1) treatment intention; and 2) treatment assignment. In this first
step, the patient, or provider, decides on the desired treatment. This
decision will typically be related to health status and other information that
could predict the outcome, so that those who do and do not intend to take
treatment are not exchangeable groups. This is confounding. We introduce a
binary “intention to treat” (ITT) variable $A^{*}$ to denote the treatment
desired. The assigned treatment $A$ is the treatment actually received. In
non-experimental (confounded) data, the intended and assigned treatment may
always coincide, because patients and clinicians are free to exercise their
intentions. In experimental data, one would in general not assume that $A^{*}$
will coincide with $A$ because in an experiment, a patient’s treatment
intentions are subverted by the random treatment assignment mechanism. In both
cases the observed data will not in general include measurements of $A^{*}$.
However, we might leverage $A^{*}$ through auxiliary assumptions that would
justify the fusion of experimental and non-experimental data, which might hold
if experimental and non-experimental patients were recruited randomly from the
same source population. We define such assumptions in Web Appendix 2.

Importantly, when $A^{*}$ is measured in a future decision setting, and
knowledge of the sign of the average treatment effect conditional on $A^{*}$
is known or identified, then $A^{*}$ will be a relevant feature as in
Definition 2. Indeed, these effects are identified when properly fused
experimental and non-experimental data are available (see Web Appendix 2).
Intuitively, the challenging quantity to identify is the expected outcome
under assigned treatment level $a$ among patients whose own treatment
intention $a^{\prime}$ – unmeasured in the experiment – disagreed with their
assignment, $a\neq a^{\prime}$. But we also know that the marginal expected
outcome under assigned treatment level $a$ – identified in the experiment – is
simply an average of this conditional quantity and the corresponding expected
outcome among those whose own treatment intention actually agreed with their
assignment, $a=a^{\prime}$, weighted by the probabilities of treatment
intentions. Since this second set of quantities are observed directly in the
non-experimental data, then the challenging quantity is the sole unknown, and
its value can be simply deduced.

As illustrated in Stensrud and Sarvet (2022), a patient will do as well or
better, in terms of expected interventionist utility, when the ITT variable
$A^{*}$ is additionally used as a relevant covariate. Furthermore, Proposition
4 in Web Appendix 2 shows that non-experimental data will improve an
interventionist’s expected utilities if and only if these non-experimental
data improve the lower bound on the probability of “harm”, that is $P(S=1\mid
L=l)$. Thus, the counterfactual and interventionist uses of confounded non-
experimental data are connected.

##  6 Philosophical considerations

We ground philosophical considerations by assuming a simple interventionist
utility $U^{a}_{1}=\mu(Y^{a},a)=1-Y^{a}$ and suppose that $Y=1$ indicates
patient death. We also consider a counterfactual utility $U^{a}_{2}$ induced
by $\gamma^{*}$ and $\mu$. We emphasize that these considerations are not
exhaustive, but are prompted by claims from contemporary advocates of a
counterfactual approach. In contrast to these advocates, we do not here
promote either approach.

###  6.1 Excess death under a counterfactual utility

Ben-Michael et al. (2022) show that the task of selecting a treatment value
$a$ under a counterfactual approach may be reformulated as the following
constrained optimization task:

$\displaystyle\text{argmin}_{a}$ $\displaystyle\mathbb{E}[Y^{a}]\text{ subject
to the constraints that}$ $\displaystyle P(S=1)<\chi_{1}\text{ and}$
$\displaystyle P(S=2)<\chi_{2},$

where $\chi_{1}$ and $\chi_{2}$ are some numbers in $[0,1]$ implied by
$\gamma$ under a particular distribution of the full data. In contrast, the
interventionist’s task will simply have the optimization task
$\text{argmin}_{a}\mathbb{E}[Y^{a}]$, subject to no constraints.

This formulation has the following simple implication: when policy-makers
optimize counterfactual utilities, then, in general, more people will die.

Proponents of a counterfactual approach may argue: but all deaths are not
considered equal; they may argue that the true utility, given by $\gamma$, is
one that uses the possibly asymmetric counterfactual utility function on the
principal strata that is appropriately formulated to reflect notions of
counterfactual harm. However, a serious problem is that we have no direct
evidence that these principal strata exist. Even if one makes a metaphysical
commitment to their existence, a patient will never know their true principal
stratum, except under extreme circumstances; thus, no patient will ever know
their post-hoc utility, and no policy may ever be evaluated, or compared to an
alternative policy, using direct observations.

In other words, the counterfactual approach requires a belief in metaphysical
objects whose existence can neither be confirmed nor denied. While patients
may be free to hold such beliefs, policy-makers should know the implications:
when a counterfactual framework is deployed to determine social policies and
regulations, it coerces conformity to an unverifiable metaphysics and a
corresponding logic that deals in those terms. In contrast, when an
interventionist framework is deployed in such a setting, no such coercion is
made, and patient and group outcomes are observable and thus transparent.

###  6.2 Medical liability in United States Law

Proponents of a counterfactual approach, see e.g. Tian and Pearl (2000), have
justified its use by appealing to discourse in modern law and medicine.
Indeed, the modern medico-legal system in the United States appears to parlay
in terms of counterfactual logic when determining liability for patient
outcomes. Legal case precedent has established a “but for” doctrine in medical
malpractice determination. One interpretation asserts that a provider cannot
be considered liable for a negative patient outcome except when it could have
been determined at the time of treatment that the negative outcome would not
occur “but for” the provider’s actions. However, although such counterfactual
terms are apparently invoked, the exact same legal phenomena might be
understood through an interventionist lens. If liability requires a
determination that $P(S=1\mid L=l)=1$, merely a high probability of “but-for
causation” is not sufficient, see Becker et al. (2018) for a review of
relevant case law. But, $P(S=1\mid L=l)=1$ occurs precisely when
$P(Y^{a=0}=0\mid L=l)=1$ and $P(Y^{a=1}=1\mid L=l)=1$. Thus, despite the use
of counterfactual language, the actual phenomena produced by the current
medico-legal framework could equivalently be explained by an interventionist
logic: a provider is held liable only when $P(Y^{a=0}=0\mid L=l)=1$ and
$P(Y^{a=1}=1\mid L=l)=1$.

###  6.3 Issues with partial identification

An investigator adopting minimal assumptions will in general only partially
identify expected counterfactual utilities (see Section 5). This is not
resolved through supplementation with non-experimental data, since these data
will in general only result in narrower bounds. Thus, the treatment that truly
maximizes counterfactual utility will still be ambiguous. Ben-Michael et al.
(2022) proposes to resolve this ambiguity through a regret-minimization
approach (Savage, 1951). However, regret minimization is only one of many
well-understood approaches for resolving such ambiguity. Alternatives include
choosing the rule that maximizes the minimum possible utility, or placing a
distribution over the remaining possible values of an unidentified nuisance
parameter (see Stoye (2012) and Cui (2021) for reviews). These different
approaches will in general converge to different decisions at increasing
sample sizes whenever counterfactual utilities are not point identified, and
so an investigator’s choice among them is one additional subjective researcher
degree of freedom that does not burden an interventionist.

##  7 Conclusion

We have defined and contrasted counterfactual and interventionist approaches
to decision making. Contrary to claims of some authors (Li and Pearl, 2019;
Mueller and Pearl, 2022), a counterfactual approach should not necessarily
portend a revolution in personalized medicine. Specifying utility functions
and their domains, as well as a framework for decision-making, remains highly
subjective. In tension with proponents of a counterfactual approach, we have
reviewed several practical and philosophical considerations that seem to
problematize its use and challenge some of its core premises, e.g. that it
somehow naturally corresponds with prevailing medical ethics and legal
practice. Perhaps most problematic from a population policy-maker’s
perspective: when the outcome is death and a counterfactual approach is used,
in general, more people will die under the identified optimal policy compared
to that identified by an interventionist approach. A strong critique of the
counterfactual approach calls it “dangerously misguided” and warns that “real
harm will ensue if it is applied in practice” (Dawid and Senn, ). We take the
following stance: as causal inference become increasingly embedded in the
development of personalized medicine, it is important that researchers,
practitioners and other decision makers clearly understand the different
approaches to decision making and their practical and philosophical
consequences.

## Acknowledgements

We thank Philip Dawid and Stephen Senn for many insightful discussions and
comments. This work has been supported by the Swiss National Science
Foundation (Grant Number $200021_{2}07436$: Causal Inference when resources
are limited).

## References

  * Beauchamp and Childress (1994) Tom L Beauchamp and James F Childress.  _Principles of biomedical ethics_.  Edicoes Loyola, 1994.
  * Becker et al. (2018) Charles L. Becker, Shanin Specter, and Thomas R. Kline.  The supreme court and medical malpractice law.  In _The Supreme Court of Pennsylvania: Life and Law in the Commonwealth, 1684-2017_ , pages 241–258. Penn State Press, 2018.
  * Ben-Michael et al. (2022) Eli Ben-Michael, Kosuke Imai, and Zhichao Jiang.  Policy learning with asymmetric utilities.  _arXiv preprint arXiv:2206.10479v1_ , 2022.
  * Cui (2021) Yifan Cui.  Individualized decision-making under partial identification: Three perspectives, two optimality results, and one paradox.  _arXiv preprint arXiv:2110.10961_ , 2021.
  * Dawid (2021) A. Philip Dawid.  Decision-theoretic foundations for statistical causality.  _Journal of Causal Inference_ , 9:39–77, 2021.
DOI:10.1515/jci-2020-0008.

  * Dawid and Musio (2022) A. Philip Dawid and Monica Musio.  What can group level data tell us about individual causality?  In A. Carriquiry, J. Tanur, and W. Eddy, editors, _Statistics in the Public Interest: In Memory of Stephen E. Fienberg_ , pages 235–256. Springer International Publishing, 2022.
DOI: 10.1007/978-3-030-75460-0_13.

  * (7) Philip A Dawid and Stephen Senn.  Personalised decision-making without counterfactuals.  _Journal of Causal Inference_.
  * Frangakis and Rubin (2002) Constantine E Frangakis and Donald B Rubin.  Principal stratification in causal inference.  _Biometrics_ , 58(1):21--29, 2002.
  * Geneletti and Dawid (2011) Sara G. Geneletti and A. Philip Dawid.  Defining and identifying the effect of treatment on the treated.  In Phyllis M. Illari, Federica Russo, and Jon Williamson, editors, _Causality in the Sciences_ , pages 728--749. Oxford University Press, 2011\.
  * Greenland and Robins (1986) Sander Greenland and James M Robins.  Identifiability, exchangeability, and epidemiological confounding.  _International journal of epidemiology_ , 15(3):413--419, 1986.
  * Kosorok et al. (2021) Michael R Kosorok, Eric B Laber, Dylan S Small, and Donglin Zeng.  Introduction to the theory and methods special issue on precision medicine and individualized policy discovery, 2021.
  * Li and Pearl (2019) Ang Li and Judea Pearl.  Unit selection based on counterfactual logic.  In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19_ , pages 1793--1799. International Joint Conferences on Artificial Intelligence Organization, 7 2019\.  DOI:10.24963/ijcai.2019/248.
  * Li and Pearl (2022a) Ang Li and Judea Pearl.  Probabilities of causation: Role of observational data.  _arXiv preprint arXiv:2210.08874_ , 2022a.
  * Li and Pearl (2022b) Ang Li and Judea Pearl.  Unit selection: Case study and comparison with a/b test heuristic.  _arXiv preprint arXiv:2210.05030_ , 2022b.
  * Mueller and Pearl (2022) Scott Mueller and Judea Pearl.  Personalized decision making--a conceptual introduction.  _arXiv preprint arXiv:2208.09558_ , 2022.
  * Murphy (2003) Susan A Murphy.  Optimal dynamic treatment regimes.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 65(2):331--355, 2003.
  * Orr et al. (1997) Robert D Orr, Norman Pang, Edmund D Pellegrino, and Mark Siegler.  Use of the hippocratic oath: a review of twentieth century practice and a content analysis of oaths administered in medical schools in the us and canada in 1993.  _Journal of Clinical Ethics_ , 8(4):377--388, 1997\.
  * Richardson and Robins (2013) Thomas S Richardson and James M Robins.  Single world intervention graphs (swigs): A unification of the counterfactual and graphical approaches to causality.  _Center for the Statistics and the Social Sciences, University of Washington Series. Working Paper_ , 128(30):2013, 2013.
  * Robins (1986) James M Robins.  A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect.  _Mathematical modelling_ , 7(9-12):1393--1512, 1986.
  * Robins (2004) James M Robins.  Optimal structural nested models for optimal sequential decisions.  In _Proceedings of the second seattle Symposium in Biostatistics_ , pages 189--326. Springer, 2004.
  * Robins et al. (2007) James M. Robins, Tyler J. Vanderweele, and Thomas S. Richardson.  Comment on ‘‘Causal effects in the presence of non compliance: A latent variable interpretation’’ by Antonio Forcina.  _Metron_ , LXIV:288--298, 2007.
  * Rubin (1974) Donald B. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies.  _Journal of Educational Psychology_ , 66:688--701, 1974\.
  * Savage (1951) Leonard J Savage.  The theory of statistical decision.  _Journal of the American Statistical association_ , 46(253):55--67, 1951.
  * Stensrud and Sarvet (2022) Mats J Stensrud and Aaron L Sarvet.  Optimal regimes for algorithm-assisted human decision-making.  _to come_ , 2022.
  * Stoye (2012) Jörg Stoye.  New perspectives on statistical decisions under ambiguity.  _Annu. Rev. Econ._ , 4(1):257--282, 2012.
  * Tian and Pearl (2000) Jin Tian and Judea Pearl.  Probabilities of causation: Bounds and identification.  _Annals of Mathematics and Artificial Intelligence_ , 28:287--313, 2000.
  * Tsiatis et al. (2019) Anastasios A Tsiatis, Marie Davidian, Shannon T Holloway, and Eric B Laber.  _Dynamic treatment regimes: Statistical methods for precision medicine_.  Chapman and Hall/CRC, 2019.
  * Von Neumann and Morgenstern (2007) John Von Neumann and Oskar Morgenstern.  Theory of games and economic behavior.  In _Theory of games and economic behavior_. Princeton university press, 2007.

##  Appendix A Additional Propositions and Proofs

In this appendix we provide formal justification for results discussed in the
main text, including the fusion of experimental and non-experimental data, and
the use of current study data for future decision problems.

###  A.1 Preliminaries

In order to distinguish between experimental and non-experimental data, we
introduce an additional variable $R$, a binary indicator of participation
within a controlled experiment on $A$ within levels of $L$. We let
$O_{F}\equiv\\{R,L,A,Y,Y^{a=1},Y^{a=0}\\}$ denote the full data, and
$O\equiv\\{R,L,A,Y\\}$ denote the observed data.

Additionally, we will distinguish between probability measures on two
dimensions: 1) whether the measure describes a probability distribution for
the full counterfactual data or describes a distribution for the observed
data; and 2) whether the measure describes a probability distribution in the
current study – used for data analysis– or in a future setting – used for
decision-making. While the first distinction allows differences between a
directly observable distribution and a partially observed causal distribution,
the latter allows consideration of possible differences between the present
study setting and the future decision setting.

As such, we let $P^{F}$ denote an arbitrary joint distribution for the full
data, and we let $P\equiv P(P^{F})$ denote the subdistribution of $P^{F}$ for
the observed data. Furthermore, we let $P^{F}_{0}$ and $P_{0}$ denote the true
such distributions at the time of the study, and further let $P^{F}_{1}$ and
$P_{1}$ be the analogous such distributions at the time of a future decision.
Note that Definitions 2 and 3 logically involve expectations evaluated at
$P^{F}_{1}$, the joint distribution for the full data at the time of a future
decision. Likewise, the expectations used in utility maximization, reviewed in
Section 4 (Decision theory), are likewise evaluated at $P^{F}_{1}$.
Intuitively, $P^{F}_{1}$ is the law of the data by which a future decision
maker will find themselves governed, and so is the relevant measure for this
individual.

Let $g$ be an arbitrary regime (treatment assignment rule) for $A$ and
consider a binary outcome $Y$ taking values in $\\{0,1\\}$. Suppose the
factual data is generated by a set of non parametric structural equations and
that all counterfactuals are generated via recursive substitution (for
example, as in Richardson and Robins (2013)).

Let $A^{g+}$ represent the assigned value of treatment under regime $g$, where
$A$ is that treatment under the regime that generates the factual data. Let
$\tau_{P^{F}}^{g}$ denote the average treatment effect (ATE) comparing $a=1$
and $g$, at the law $P^{F}$:

$\mathbb{E}_{P^{F}}[Y^{a=1}]-\mathbb{E}_{P^{F}}[Y^{g}].$

Let $\tau_{P^{F}}^{0}$ denote the special ATE where $g$ is the rule where no
one is treated, $a=0$,

$\mathbb{E}_{P^{F}}[Y^{a=1}]-\mathbb{E}_{P^{F}}[Y^{a=0}]$

and let $\tau_{P^{F}}$ denote the special case where $g$ is that rule
implemented in the factual data

$\mathbb{E}_{P^{F}}[Y^{a=1}]-\mathbb{E}_{P^{F}}[Y].$

Likewise, let $\beta_{P^{F}}^{g}$ denote the average treatment effect (ATE)
comparing $g$ and $a=0$, and let $\beta_{P^{F}}$ denote the special case where
$g$ is the rule implemented in the factual data.

Let $\mathcal{L}_{P^{F}}$ be the sharp lower bound for $P^{F}(S=1)$ when only
the parameters $\mathbb{E}_{P^{F}}[Y^{a=1}]$ and $\mathbb{E}_{P^{F}}[Y^{a=1}]$
are known. Let $\mathcal{L}_{P^{F}}^{*}$ be that sharp lower bound when the
parameters $P(Y=y,A=a)$, for all $a$, $y$, are additionally known.

###  A.2 Identifying conditional expected potential outcomes under model
$\mathcal{M}$

We define a model $\mathcal{M}$ based on the following assumptions.

  * A1.

$\mathbb{E}_{P^{F}_{0}}[Y^{a}\mid A=a,L=l,R=r]=\mathbb{E}_{P^{F}_{0}}[Y\mid
A=a,L=l,R=r]$ for all $r,l,a$.

  * A2.

$Y^{a}\perp\mkern-9.5mu\perp_{P_{0}^{F}}A\mid L,R=1$ for all $a$.

  * A3.

$(Y^{a=1},Y^{a=0})\perp\mkern-9.5mu\perp_{P_{0}^{F}}R\mid L$.

  * A4.

$P^{F}_{0}=P^{F}_{1}$.

Assumption A1 states that the expected outcome among patients with treatment
$a$, covariate $l$, and trial participation indicator $r$ is equal to that
expectation for similar such patients in a trial in which treatment is fixed
to $a$, also known as “distributional consistency” (Dawid, 2021). Assumption
A2 states that patients are exchangeable across levels of treatment within
strata of $L$ in the trial ($R=1$). Assumption A3 states that patients in the
trial are exchangeable with patients outside of the trial, within strata of
$L$. Assumption A4 states that the true law of the full data is stable between
the time of data collection ($P^{F}_{0}$) and a future decision setting
($P^{F}_{1}$).

We consider $\mathcal{M}$ to be minimal because it comprises only the basic
model motifs that are used to: 1) link observed and causal parameters (A1); 2)
distinguish experimental and non-experimental data (A2); 3) justify the fusion
of experimental and non-experimental data (A3); and 4) justify the use of
observed data for future decision settings (A4).

We may obtain $\mathbb{E}_{P^{F}_{1}}[Y^{a}\mid L=l]$ from parameters of
$P_{0}$ under $\mathcal{M}$ as follows:

$\displaystyle\mathbb{E}_{P^{F}_{1}}[Y^{a}\mid L=l]=$
$\displaystyle\mathbb{E}_{P^{F}_{0}}[Y^{a}\mid L=l]$ (8) $\displaystyle=$
$\displaystyle\mathbb{E}_{P^{F}_{0}}[Y^{a}\mid L=l,R=1]$ $\displaystyle=$
$\displaystyle\mathbb{E}_{P^{F}_{0}}[Y^{a}\mid L=l,A=a,R=1]$ $\displaystyle=$
$\displaystyle\mathbb{E}_{P_{0}}[Y\mid L=l,A=a,R=1],$

assuming all terms are well defined. The first equality follows from
equivalence of the full law at the time of data collection $P_{0}^{F}$ and the
full law at the future decision time point $P_{1}^{F}$ (A4.), the second
equality follows from the exchangeability of trial and non-trial participants
given $L$ (A3.), the third follows from the exchangeability of treated and
untreated participants in the trial given $L$ (A2.), and the fourth follows
from consistency (A1.).

###  A.3 On the Interventionist use of treatment intention as a relevant
covariate

Here we define an elaborated model $\mathcal{M}^{*}$ whose assumptions permit
the fusion of experimental and non-experimental data for improved decision
making under an Interventionist approach. We let $\mathcal{M}^{*}$ denote the
model defined by $\mathcal{M}$ supplemented with assumptions B1. to B3. We let
$P^{F*}$ denote an arbitrary law of the full data including the intention to
treat variable $A^{*}$.

  * B1.

$P^{*}_{1}(A=A^{*}\mid R=0)=1$.

  * B2.

$(Y^{a=1},Y^{a=0},A^{*})\perp\mkern-9.5mu\perp_{P_{0}^{F}}R\mid L$.

  * B3.

$P^{F*}_{0}=P^{F*}_{1}$.

Assumption B1. asserts that a patient’s intention to treat variable ($A^{*}$)
agrees almost surely with the treatment actually received ($A$) in the non-
experimental setting ($R=0$), and thus the intention to treat variable is
indirectly measured. Assumption B2. differs from A3. by asserting that trial
participation is additionally independent of a patient’s intention to take
treatment $A^{*}$, conditional on covariates. Assumption B3. differs from A4.
in asserting that the elaborated law of the full data (that includes $A^{*}$)
is also stable between the time of data collection ($P^{F*}_{0}$) and a future
decision setting ($P^{F*}_{1}$).

The following proposition asserts that these parameters are indeed identified
under $\mathcal{M}^{*}$.

######  Proposition 1.

Under $\mathcal{M}^{*}$, $P^{F*}_{1}(Y^{a}=y\mid A^{*}=a^{*},L=l)$ is
identified for all $(a,a^{*}=0,1)$. Specifically,

$\displaystyle P^{F*}_{1}(Y^{a}=y\mid A^{*}=a^{*},L=l)=\begin{cases}P(Y=y\mid
A=a,L=l,R=1)&\text{for }a=a^{*}\\\ \frac{P(Y=y\mid A=a,L=l,R=1)-P(Y=y,A=a\mid
L=l,R=0)}{P(A=a^{*}\mid L=l,R=0)}&\text{for }a\neq a^{*}.\end{cases}$

Related propositions have appeared in Stensrud and Sarvet (2022); Robins et
al. (2007); Geneletti and Dawid (2011); Dawid and Musio (2022).

###### Proof.

When $a=a^{*}$, arguments proceed identically as in (8). When $a\neq a^{*}$,
we have by the law of total probability that $P^{F*}_{1}(Y^{a}=y\mid
A^{*}=a^{*},L=l)$ equals

$\frac{P^{F*}_{1}(Y^{a}=y\mid L=l)-P^{F*}_{1}(Y^{a}=y,A^{*}=a\mid
L=l)}{P^{F*}_{1}(A^{*}=a^{*}\mid L=l)}.$

Then, the left hand term in the numerator is identified by the experimental
data under the assumptions in $\mathcal{M}$ alone, and the remaining terms are
identified by the non-experimental data using assumptions unique to
$\mathcal{M}^{*}$. For example, we have :

$\displaystyle P^{F*}_{1}(Y^{a}=y,A^{*}=a\mid L=l)=$ $\displaystyle
P^{F*}_{0}(Y^{a}=y,A^{*}=a\mid L=l),$ $\displaystyle=$ $\displaystyle
P^{F*}_{0}(Y^{a}=y,A^{*}=a\mid L=l,R=0),$ $\displaystyle=$ $\displaystyle
P^{F*}_{0}(Y^{a}=y,A=a\mid L=l,R=0),$ $\displaystyle=$ $\displaystyle
P_{0}(Y=y,A=a\mid L=l,R=0).$

In the above, the first equality follows from B3., the second by B2., the
third by B1., and the fourth by laws of probability and A3. ∎

The above proof relies on $A$ (and so $A^{*}$) being binary, but $Y$ need not
be.

###  A.4 Propositions 2 and 3

######  Proposition 2.

The average treatment effect comparing $a=1$ and $g$ is a valid lower bound
for the probability of harm. That is, for all laws $P^{F}$:

$\displaystyle P^{F}(S=1)\geq\tau_{P^{F}}^{g}.$

###### Proof.

$\displaystyle P^{F}(Y^{g}=1)=$ $\displaystyle
P^{F}(Y^{g}=1,A^{g+}=1)+P(Y^{g}=1,A^{g+}=0)$ $\displaystyle=$ $\displaystyle
P^{F}(Y^{a=1}=1,A^{g+}=1)+P^{F}(Y^{a=0},A^{g+}=0)$ $\displaystyle=$
$\displaystyle\Big{\\{}P^{F}(S=1,A^{g+}=1)+P^{F}(S=2,A^{g+}=0)\Big{\\}}+P^{F}(S=3).$

The first equality follows by laws of probability, the second by recursive
substitution (consistency), and and third follows by laws of probability. Let
$\epsilon$ denote the term $\\{P^{F}(S=1,A^{g+}=1)+P^{F}(S=2,A^{g+}=0)\\}$.
Then

$\displaystyle\tau_{P^{F}}^{g}=$ $\displaystyle
P^{F}(S=1)-\\{P^{F}(S=1,A^{g+}=1)+P^{F}(S=2,A^{g+}=0)\\}$ $\displaystyle=$
$\displaystyle P^{F}(S=1)-\epsilon.$

The first equality follows by definition (and cancellation of $P^{F}(S=3)$),
and the final by definition. Because $\epsilon$ must be positive for all laws
$P^{F}$, then $\tau_{P^{F}}^{g}$ is a valid lower bound for $P^{F}(S=1)$. ∎

######  Proposition 3.

When strong exchangeability holds under $P^{F}$ relative to regime $g$,

$\\{Y^{a=1},Y^{a=0}\\}\perp\mkern-9.5mu\perp_{P^{F}}A^{g+}$

then $\tau_{P^{F}}^{0}\geq\tau_{P^{F}}^{g}$.

###### Proof.

$\displaystyle\tau_{P^{F}}^{g}=$ $\displaystyle
P^{F}(S=1)-\\{P^{F}(S=1,A^{g+}=1)+P^{F}(S=2,A^{g+}=0)\\}$ $\displaystyle=$
$\displaystyle
P^{F}(S=1)-\\{P^{F}(S=1)P^{F}(A^{g+}=1)+P^{F}(S=2)P^{F}(A^{g+}=0)\\}$
$\displaystyle=$ $\displaystyle P^{F}(A^{g+}=0)\\{P^{F}(S=1)-P^{F}(S=2)\\}$
$\displaystyle=$ $\displaystyle\tau_{P^{F}}^{0}P^{F}(A^{g+}=0).$

The first equality follows by Proposition 2, the second by strong
exchangeability, and the final two by laws of probability. Because
$P^{F}(A^{g+}=0)$ is bounded to the unit interval for all laws $P^{F}$, then
the claim of Proposition 3 follows immediately. ∎

Arguments are identical when all terms additionally condition on the event
$L=l$.

###  A.5 Proposition 4

Tian and Pearl (2000) showed that the lower bound for $P^{F}(S=1)$ is improved
by the addition of non-experimental data, that is,

$\mathcal{L}^{*}_{P^{F}}>\mathcal{L}_{P^{F}}$

whenever:

$\displaystyle\text{max}\begin{Bmatrix}&\tau_{P^{F}},\\\
&\beta_{P^{F}}\end{Bmatrix}>\text{max}\begin{Bmatrix}&0,\\\
&\tau_{P^{F}}^{0}\end{Bmatrix}.$ (9)

Let $\tau_{P^{F},A=1}$ denote the average treatment effect among the treated,

$\tau_{P^{F},A=1}\coloneqq\mathbb{E}_{P^{F}}(Y^{a=1}\mid
A=1)-\mathbb{E}_{P^{F}}(Y^{a=0}\mid A=1),$

and let $\tau_{P^{F},A=0}$ denote the average treatment effect among the
untreated, $A=0$.

######  Proposition 4.

Condition (9) holds if and only if the following condition holds:

$\displaystyle\text{sign}\Big{\\{}\tau_{P^{F},A=1}\Big{\\}}\not\equiv\text{sign}\Big{\\{}\tau_{P^{F},A=0}\Big{\\}}.$
(10)

###### Proof.

Note that (10) holds if either $\tau_{P^{F}}>\\{0,\tau_{P^{F}}^{0}\\}$ or
$\beta_{P^{F}}>\\{0,\tau_{P^{F}}^{0}\\}$. Consider the following implications:

$\displaystyle\tau_{P^{F}}>\tau_{P^{F}}^{0}$ $\displaystyle\iff$
$\displaystyle
P^{F}(S=1)-\\{P^{F}(S=1,A=1)+P^{F}(S=2,A=0)\\}>P^{F}(S=1)-P^{F}(S=2)$
$\displaystyle\iff$ $\displaystyle P^{F}(S=2)>P^{F}(S=1,A=1)+P^{F}(S=2,A=0)$
$\displaystyle\iff$ $\displaystyle P^{F}(S=2,A=1)>P^{F}(S=1,A=1)$
$\displaystyle\iff$ $\displaystyle P^{F}(S=2\mid A=1)>P^{F}(S=1\mid A=1)$
$\displaystyle\iff$ $\displaystyle\tau_{P^{F},A=1}<0.$

Suppose also that $\tau_{P^{F}}>0$.

$\displaystyle\tau_{P^{F}}>0$ $\displaystyle\iff$ $\displaystyle
P^{F}(S=1)>P^{F}(S=1,A=1)+P^{F}(S=2,A=0)$ $\displaystyle\iff$ $\displaystyle
P^{F}(S=1,A=0)>P^{F}(S=2,A=0)$ $\displaystyle\iff$ $\displaystyle
P^{F}(S=1\mid A=0)>P^{F}(S=2\mid A=0)$ $\displaystyle\iff$
$\displaystyle\tau_{P^{F},A=0}>0.$

Thus, $\tau_{P^{F}}>\\{0,\tau_{P^{F}}^{0}\\}$ if and only if both
$\tau_{P^{F},A=1}<0$ and $\tau_{P^{F},A=0}>0$. By isomorphic arguments, we can
show that $\beta_{P^{F}}>\\{0,\tau_{P^{F}}^{0}\\}$ if and only if both
$\tau_{P^{F},A=1}>0$ and $\tau_{P^{F},A=0}<0$. Then the claim of proposition 4
follows immediately.

∎

Arguments are identical when all terms additionally condition on the event
$L=l$. Thus, it follows that, when $A=A^{*}$ with probability 1 (as under
$\mathcal{M}^{*}$) and (9) holds, the expected outcome for the optimal regime
that is a function of $L$ will be inferior to that for the optimal regime that
is additionally a function of $A^{*}$. Thus, the argument in the main text
follows: under $\mathcal{M}^{*}$, non-experimental data will improve the lower
bound on the probability of counterfactual harm if and only if an
interventionist would obtain a greater expected utility through the use of
such data.

##  Appendix B Review of principal strata bounds with experimental and non-
experimental data

Tian and Pearl (2000) note that, for an arbitrary conditioning event $X=x$,
constraints of the type

$P^{F}_{1}(S=1,A=1\mid X=x)+P^{F}_{1}(S=3,A=1\mid X=x)=P^{F}_{1}(Y=1,A=1\mid
X=x)$

will impose constraints on the principal strata probabilities, for example,
that $P^{F}_{1}(S=1\mid X=x)$ must be greater than $P_{1}(Y=1\mid
X=x)-P_{1}^{F}(Y^{a=0}=1\mid X=x)$. Leveraging experimental data under model
$\mathcal{M}$, we likewise have that $P^{F}_{1}(S=1\mid L=l)=P^{F}_{1}(S=1\mid
L=l,R=1)$ must be greater than $P_{1}(Y=1\mid L=l,R=1)-P_{1}^{F}(Y^{a=0}=1\mid
L=l)$. We also have $P^{F}_{1}(S=1\mid L=l)=P^{F}_{1}(S=1\mid L=l,R=0)$ must
be greater than $P_{1}(Y=1\mid L=l,R=0)-P_{1}^{F}(Y^{a=0}=1\mid L=l)$. The
results of Tian and Pearl (2000) motivate the following lower bound on
$P^{F}_{1}(S=s\mid L=l)$:

$\displaystyle P^{F}_{1}(S=1\mid L=l)\geq\text{max}\begin{Bmatrix}&0\\\
&P_{1}^{F}(Y^{a=1}=1\mid L=l)-P_{1}^{F}(Y^{a=0}=1\mid L=l),\\\ &P_{1}(Y=1\mid
L=l,R=1)-P_{1}^{F}(Y^{a=0}=1\mid L=l),\\\ &P_{1}^{F}(Y^{a=1}=1\mid
L=l)-P_{1}(Y=1\mid L=l,R=1),\\\ &P_{1}(Y=1\mid
L=l,R=0)-P_{1}^{F}(Y^{a=0}=1\mid L=l),\\\ &P_{1}^{F}(Y^{a=1}=1\mid
L=l)-P_{1}(Y=1\mid L=l,R=0).\\\ \end{Bmatrix}$ (11)

Using identification results from the experimental data, we express these
constraints fully in terms of observed data:

$\displaystyle P^{F}_{1}(S=1\mid L=l)\geq\text{max}\begin{Bmatrix}&0,\\\
&P_{0}(Y=1\mid A=1,L=l,R=1)-P_{0}(Y=1\mid A=0,L=l,R=1),\\\ &P_{0}(Y=1\mid
L=l,R=1)-P_{0}(Y=1\mid A=0,L=l,R=1),\\\ &P_{0}(Y=1\mid
A=1,L=l,R=1)-P_{0}(Y=1\mid L=l,R=1),\\\ &P_{0}(Y=1\mid L=l,R=0)-P_{0}(Y=1\mid
A=0,L=l,R=1),\\\ &P_{0}(Y=1\mid A=1,L=l,R=1)-P_{0}(Y=1\mid
L=l,R=0).\end{Bmatrix}$ (12)

However, because of the constraint that $P_{0}(Y=1\mid L=l,R=1)$ is a convex
combination of $P_{0}(Y=1\mid A=0,L=l,R=1)$ and $P_{0}(Y=1\mid A=1,L=l,R=1)$,
then the third and fourth terms in the set are guaranteed to be less or equal
to than the second term, and so they do not effectively sharpen the bounds.
However, when $Y^{a}\not\perp\mkern-9.5mu\perp_{P_{0}^{F}}A\mid L,R=0$ for
some $a$, as would be expected in non-experimental data, then no such
constraint on $P_{0}(Y=1\mid L=l,R=0)$ is imposed and these additional
constraints (the fifth and sixth terms) may indeed sharpen the bounds. Of
course, such constraints may only be leveraged when investigators additionally
take advantage of (confounded) non-experimental data. We also emphasize that
these bounds will only hold under models like $\mathcal{M}$, in which the
CATEs are preserved between both experimental and non-experimental data, but
that the conditional joint distribution of treatment $A$ and potential
outcomes differ.

##  Appendix C A note on potential outcomes notation

Although we use a notation that often corresponds to assumptions about the
existence of counterfactual variables, we strongly emphasize that such
assertions are unnecessary for the interventionist approach. We use this
notation in the first place because the counterfactual approach depends
inextricably on the existence of counterfactuals. We use them throughout to
reduce notational burden, but we encourage readers to interpret any parameter
defined by a single intervention as simply the analogous parameter that would
be observed in the arm of a hypothetical controlled experiment corresponding
to that intervention. For example, the expected potential outcome under
treatment $a$, $Y^{a}$, among patients with covariates $L=l$ should be
interpreted as the expected outcome among patients with $L=l$ in a
hypothetical controlled experiment where $A$ is fixed to $a$. As a second
example, claims of (conditional) independencies between single-intervention
counterfactuals and treatments, for example $Y^{a=1}\perp\mkern-9.5mu\perp A$,
should simply be interpreted as claims that patients with different treatments
in the observed data are (conditionally) exchangeable with respect to the
outcome $Y$.