Testing identification in mediation and dynamic treatment models

Martin Huber*, Kevin Kloiber+, Lukáš Lafférs**

*University of Fribourg, Department of Economics
+University of Munich, Department of Economics
**Matej Bel University, Department of Mathematics
**Norwegian School of Economics, Department of Economics

###### Abstract

We propose a test for the identification of causal effects in mediation and
dynamic treatment models that is based on two sets of observed variables,
namely covariates to be controlled for and suspected instruments, building on
the test by Huber and Kueck (2022) for single treatment models. We consider
models with a sequential assignment of a treatment and a mediator to assess
the direct treatment effect (net of the mediator), the indirect treatment
effect (via the mediator), or the joint effect of both treatment and mediator.
We establish testable conditions for identifying such effects in observational
data. These conditions jointly imply (1) the exogeneity of the treatment and
the mediator conditional on covariates and (2) the validity of distinct
instruments for the treatment and the mediator, meaning that the instruments
do not directly affect the outcome (other than through the treatment or
mediator) and are unconfounded given the covariates. Our framework extends to
post-treatment sample selection or attrition problems when replacing the
mediator by a selection indicator for observing the outcome, enabling joint
testing of the selectivity of treatment and attrition. We propose a machine
learning-based test to control for covariates in a data-driven manner and
analyze its finite sample performance in a simulation study. Additionally, we
apply our method to Slovak labor market data and find that our testable
implications are not rejected for a sequence of training programs typically
considered in dynamic treatment evaluations.
JEL Classification: C12, C21, C26
Keywords: mediation, dynamic treatment effects, causality, conditional
independence, sequential exogeneity, instrument, covariates, hypothesis test.

We have benefited from comments of seminar participants in Neuchâtel,
Düsseldorf, and Kassel, as well as conference participants at the European
Causal Inference Meeting 2024 in Copenhagen. Addresses for correspondence:
Martin Huber, University of Fribourg, Bd. de Pérolles 90, 1700 Fribourg,
Switzerland<EMAIL_ADDRESS>Kevin Kloiber, University of Munich,
Ludwigstrasse 33, 80539 München, Germany<EMAIL_ADDRESS>Lukáš
Lafférs, Department of Mathematics, Matej Bel University, Tajovského 40, 974
01 Banská Bystrica, Slovakia; Department of Economics, Norwegian School of
Economics, Helleveien 30, 5045 Bergen, Norway<EMAIL_ADDRESS>Lafférs acknowledges support provided by the Slovak Research and Development
Agency under contracts VEGA 1/0398/23 and APVV-21-0360.

##  1 Introduction

Causal mediation concerns the evaluation of the direct effect of a treatment
on an outcome, as well as the indirect effect mediated by an intermediate
outcome referred to as a mediator. For instance, when assessing the effect of
two sequential training programs (such as job application training and an IT
course) on employment, the direct effect corresponds to the effect of the
first program, net of participation in the second program. Conversely, the
evaluation of dynamic treatment effects pertains to assessing the effects of
specific sequences of treatment and mediator values, such as the impact of
participating in two sequential training programs versus not participating in
any program. Effect identification in numerous studies within the fields of
causal mediation and dynamic treatment evaluation relies on a sequential
selection-on-observables or ignorability assumption, see for instance Robins
(1986), Robins, Hernan, and Brumback (2000), Lechner (2009), Flores and
Flores-Lagunes (2009), Imai, Keele, and Yamamoto (2010), Hong (2010), Tchetgen
Tchetgen and Shpitser (2012), and Huber (2014). This assumption postulates
that, conditional on observed covariates, the treatment and the mediator are
exogenous, such that their causal effects are not confounded by unobservables.

Related identification issues also arise when assessing causal effects in the
context of sample selection or post-treatment outcome attrition, which implies
that the outcome is only observed for a selectively non-random subpopulation.
For example, wages are only observed for employed individuals. In such cases,
sequential ignorability means that both the treatment and a selection
indicator for the observability of the outcome (rather than a mediator) are
exogenous conditional on observed characteristics. This allows for the
identification of the total effect of the treatment on the outcome, as
discussed, for instance, in Negi (2020) and Bia, Huber, and Lafférs (2024).
Whether the observed covariates plausibly satisfy the assumption of sequential
ignorability is conventionally motivated based on theory, intuition, domain
knowledge, or previous empirical findings. However, it is typically also
subject to controversy, as unobserved confounders can rarely be completely
ruled out in empirical applications relying on observational data.

This study introduces a test for conditions that imply sequential
ignorability, facilitating the verification of identification in observational
data. Testing relies on two types of observed variables for each of the
treatment and mediator (or selection indicator): covariates to be controlled
for and additional variables that are presumed to be separate instruments for
the treatment and the mediator. The testable condition arises within a
specific causal structure that rules out certain forms of reverse causality,
such as those from the outcome to the mediator, treatment, covariates, or the
suspected instruments and imposes that the suspected instruments have a (first
stage) association with the treatment or the mediator on other observed
variables. If, under these circumstances, the supposed instrument for the
treatment is conditionally independent of the outcome, given the treatment and
the covariates, and the supposed instrument for the mediator is conditionally
independent of the outcome, given the treatment, mediator, and covariates,
then the following conditions hold: (A) The instruments are valid, meaning
they do not directly influence the outcome, except through their impact on the
treatment or mediator, and are not associated with unobservable factors
affecting the outcome, given the observed variables. (B) The treatment and the
mediator are exogenous, implying that they are not associated with
unobservable factors that affect the outcome, conditional on the observed
variables.

Therefore, the testable conditional independence assumption implies that the
treatment and the mediator satisfy a sequential ignorability assumption, which
permits identifying dynamic treatment effects or the controlled direct effect
discussed, for instance, in Pearl (2001). The controlled direct effect
corresponds to the net effect of the treatment when holding the mediator fixed
at a specific value (e.g., one). In sample selection models, this assumption
allows for the evaluation of the total treatment effect. Furthermore, if it
additionally holds that the supposed instrument for the treatment is
conditionally independent of the mediator given the treatment and the
covariates, the effect of the treatment on the mediator is identifiable. This
is also a precondition for the evaluation of natural direct and indirect
effects as discussed in Robins and Greenland (1992) and Pearl (2001), albeit
their identification rests on somewhat stronger conditions than those implied
by our testable implications. Natural effects are defined based on setting the
mediator to its potential value under a specific treatment (rather than fixing
it at a constant value like one, as is the case when evaluating the controlled
direct effect).

Our study is, to the best of our knowledge, the first to propose an
identification test in the contexts of causal mediation, dynamic treatment
effects, or sample selection, building on the work of Huber and Kueck (2022)
for testing the identification of the (total) effect of a single treatment
variable. We extend the machine learning-based testing approach presented
therein to enable testing conditional independence assumptions involving
multiple variables, such as the treatment, mediator, and selection indicator.
Additionally, our extension allows for the consideration of multivalued
instruments for the treatment, mediator, or selection indicator, as opposed to
binary instruments. The utilization of machine learning methods offers the
advantage of data-driven control for important observed confounders during
testing, which is particularly valuable in high-dimensional contexts where
numerous potential control variables are available. More concisely, we
consider a test that is based on the expectation of squared differences of
regression functions including and excluding the respective instruments. The
squared difference satisfies the Neyman (1959)-orthogonality condition,
implying that we may account for covariates in the regression functions by
machine learning without compromising on desirable asymptotic properties,
given that specific regularity conditions hold, see Chernozhukov, Chetverikov,
Demirer, Duflo, Hansen, Newey, and Robins (2018).

We investigate the finite sample performance of our test in a simulation study
and find that it performs very decently in terms of empirical size and power
in our simulation designs when the sample size consists of several thousand
observations (or more). Furthermore, we apply our method to large
administrative data from Slovakia to test the identification of the effects of
sequential labor market programs for jobseekers on employment. We use the
availability in public employment service centers as instruments and also
control for a rich set of jobseeker characteristics. Our findings indicate
that our testable implications are not rejected for the specific sequences of
programs tested, namely a graduate practice program followed by employment
incentives (consisting of hiring incentives and subsidized employment).

Our paper contributes to a growing literature on testing identifying
assumptions. For instance, de Luna and Johansson (2014) and Black, Joo,
LaLonde, Smith, and Taylor (2015) consider the same testable implication as
Huber and Kueck (2022) to assess the conditional exogeneity of a treatment
based on a valid instrument, rather than jointly testing the treatment
exogeneity and instrument validity. Peters, Buhlmann, and Meinshausen (2015)
employ instruments to learn in a data-driven way which variables are
treatments, in the sense that they directly influence an outcome of interest,
under the assumption that they satisfy conditional exogeneity. The approach
utilizes instruments in a way that enables the rejection of treatments
violating conditional exogeneity, thereby providing the power to detect
identification failures. Angrist and Rokkanen (2015) test an analogous
implication as in de Luna and Johansson (2014), Black, Joo, LaLonde, Smith,
and Taylor (2015), and Huber and Kueck (2022), but when the treatment is
unconfounded, in order to test the validity of an instrument within the
framework of the regression discontinuity design (RDD). In a sharp RDD, the
treatment is a deterministic function of a cutoff in a running variable and,
therefore, satisfies conditional exogeneity by design. This allows for testing
whether the running variable is a valid instrument, i.e., whether it is not
associated with the outcome conditional on the treatment. If this holds true,
causal effects can also be identified away from the cutoff, which is otherwise
not feasible due to the lack of common support in the treatment across
different values of the running variable.

The remainder of this study is organized as follows. Section 2 presents the
identifying assumptions and testable implications in scenarios involving only
pre-treatment covariates as control variables. Section 3 considers a modified
causal framework that allows the first instrument (for the treatment) to
directly influence the second instrument (for the mediator), which requires
controlling for the second instrument in a specific way when testing. Section
4 discusses a setup with dynamic confounding, which requires controlling for
post-treatment covariates in addition to pre-treatment covariates when
testing. Section 5 outlines the machine learning-based test, which permits
controlling for high-dimensional covariates in a data-driven manner. Section 6
presents a simulation study that investigates the finite sample performance of
our test. Section 7 provides an empirical application to Slovak labor market
data. Section 8 concludes. The proofs of the identification results are
provided in the Appendix.

##  2 Assumptions and implications with pre-treatment covariates

In causal mediation analysis, the objective is to dissect the overall causal
impact of a treatment variable $D$ on an outcome variable $Y$ into two
distinct components: the direct effect of the treatment on the outcome and the
indirect effect that operates through the mediator variable $M$. A
conceptually related framework is dynamic treatment evaluation, where $D$
signifies an initial treatment and $M$ a subsequent one, aiming to evaluate
the effectiveness of different treatment sequences involving $D$ and $M$. In
both mediation and dynamic treatment models, the variables $D$, $M$, and $Y$
can exhibit either discrete or continuous distributions. However, in models
featuring sample selection or post-treatment outcome attrition, $M$ represents
a binary selection indicator determining the observability of outcome $Y$,
while $D$ represents the treatment. Despite this distinction, the
identification challenges encountered in sample selection models are related
to those in dynamic treatment models.

To formalize the assumptions required for identifying causal effects in
mediation, dynamic treatment, and sample selection models, we make us of the
potential outcome framework (e.g., Neyman (1923) and Rubin (1974)) and denote
random variables by capital letters and their specific values by lowercase
letters. Specifically, we denote by $M(d)$ the potential mediator under
treatment value $d\in\mathcal{D}$, where $\mathcal{D}$ is the support of the
treatment. $Y(d,m)$ denotes the potential outcomes as a function of both the
treatment and some mediator value $m\in\mathcal{M}$, where $\mathcal{M}$ is
the support of the mediator. Our definition of $Y(d,m)$ and $M(d)$ as a
function of an individual’s treatment status $D=d$ and mediator value $M=m$
assumes that (i) an individual’s potential outcomes are not affected by the
treatment or mediator status of others, and (ii) there are no multiple
versions of any treatment level $d$ and mediator level $d$ across individuals.
This is known as Stable Unit Treatment Value Assumption (SUTVA), see e.g.
Rubin (1980) and Cox (1958). Furthermore, we denote observed pre-treatment
covariates as $X$, one or more observed instrumental variables for the
treatment $D$ as $Z_{1}$, and one or more observed instrumental variables for
the mediator $M$ as $Z_{2}$. The properties of these instrumental variables
are yet to be established. Finally, let
$\mathcal{X},\mathcal{Z}_{1},\mathcal{Z}_{2}$, and $\mathcal{Y}$ denote the
support of $X,Z_{1},Z_{2}$, and $Y$.

We subsequently discuss a range of identifying assumptions for causal
mediation analysis, which permit identifying direct and indirect effects
conditional on the covariates and testing identification based on specific
assumptions on the instruments. Our first assumption establishes a specific
causal structure between variables within our framework, positing that only
certain variables exert a causal influence on others. We formalise this causal
structure using the previously mentioned potential outcome notation, by
applying the latter also to other variables than the outcome and the mediator.
To this end, let $A(b)$ and $A(b,c,...)$ correspond to the potential value of
variable $A$ when setting variable $B$ to $b$, or variables $B$, $C$,… to $b$,
$c$,…, respectively. Moreover, we assert that if there exists a causal
relation between two variables (potentially conditional on other variables),
then a statistical dependence necessarily exists between them, aligning with
the principle of causal faithfulness.

######  Assumption 1 (Causal structure and faithfulness).

$\displaystyle
M(y)=M,D(m,y,z_{2})=D,X(d,m,y,z_{2})=X,Z_{1}(d,m,y,z_{2})=Z_{1},\;\textrm{ and
}Z_{2}(m,y)=Z_{2},$ $\displaystyle\forall
d\in\mathcal{D},m\in\mathcal{M},z_{2}\in\mathcal{Z}_{2}\textrm{ and
}y\in\mathcal{Y},$

only variables which are d-separated in any causal model are statistically
independent.

The first line of Assumption 1 rules out a reverse causal effect of outcome
$Y$ on $D$, $X$, $M$, $Z_{1}$, or $Z_{2}$. In addition, the treatment $D$ must
not casually affect $X$, $Z_{1}$, the mediator $M$ must not causally affect
$X$, $D$, $Z_{1}$, $Z_{2}$, while $X$ might affect $D$, $M$, $Z_{1}$, $Z_{2}$
or $Y$, $Z_{1}$ might affect $D$. $Z_{2}$ might affect $M$. The second line of
Assumption 1 establishes causal faithfulness, which ensures that only
variables that are d-separated, i.e., not linked via any causal paths, are
statistically independent or conditionally independent, see e.g. Pearl (2000).
To be more precise, we employ the d-separation criterion of Pearl (1988),
which relies on blocking causal paths between variables. A path between two
(sets of) variables $A$ and $B$ is blocked when conditioning on a (set of)
control variable(s) $C$ if

  1. 1.

the path between $A$ and $B$ is a causal chain, implying that $A\rightarrow
M\rightarrow B$ or $A\leftarrow M\leftarrow B$, or a confounding association,
implying that $A\leftarrow M\rightarrow B$, and variable (set) $M$ is among
control variables $C$ (i.e. controlled for),

  2. 2.

the path between $A$ and $B$ contains a collider, implying that $A\rightarrow
S\leftarrow B$, and variable (set) $S$ or any variable (set) causally affected
by $S$ is not among control variables $C$ (i.e. not controlled for).

According to the d-separation criterion, $A$ and $B$ are d-separated when
conditioning on a set of control variables $C$ if, and only if, $C$ blocks
every path between $A$ and $B$. d-separation is sufficient for the
(conditional) independence of two variables, providing a vital component for
the proof of our Theorems. Causal faithfulness imposes that d-separation is
also a necessary condition, such that two variables are statistically
independent if and only if d-separation holds. A scenario in which
faithfulness fails is that one variable affects another one via several causal
paths (or mechanisms) which exactly cancel out such that the variables are
independent, see e.g. the discussion in Spirtes, Glymour, Scheines, and
Heckerman (2000).

Next, we introduce two common support assumptions which are required for
nonparametric identification and testing. To this end, let $f(A=a|B=b)$ denote
the conditional density of variable $A$ given $B$ at values $A=a$ and $B=b$.
We note that if $A$ is discrete rather than continuous, then $f(A=a|B=b)$ is a
conditional probability rather than a density.

######  Assumption 2 (Common support for $D$ and $Z_{1}$).

$\displaystyle f(D=d,Z_{1}=z_{1}|M=m,X=x)>0\quad\forall
d\in\mathcal{D}\textrm{,}z_{1}\in\mathcal{Z}_{1},m\in\mathcal{M},\textrm{ and
}x\in\mathcal{X}$

######  Assumption 3 (Common support for $M$ and $Z_{2}$).

$\displaystyle f(M=m,Z_{2}=z_{2}|D=d,X=x)>0\quad\forall
m\in\mathcal{M},z_{2}\in\mathcal{Z}_{2},d\in\mathcal{D},\textrm{ and
}x\in\mathcal{X}$

Assumption 2 requires that conditional on $M$ and $X$, their exist
observations with any values of $D$ and $Z_{1}$ that occur in the total
population. Likewise, Assumption 3 implies that conditional on $D$ and $X$,
their exist observations with any values of $M$ and $Z_{2}$ that occur in the
total population. In case of a violation of these common support conditions,
effects can only be evaluated and/or tested over a subset of the support of
$X$, $D$, and/or $M$.

Our next assumptions impose that the instruments are relevant in the sense
that the instrument for treatment $Z_{1}$ is statistically associated with the
treatment $D$ conditional covariates $X$, while the instrument for the
mediator $Z_{2}$ is associated with the mediator $M$, conditional on the
treatment $D$ and covariates $X$.

######  Assumption 4 (Conditional dependence between the treatment and
treatment instrument).

$\displaystyle D\not\\!\perp\\!\\!\\!\perp Z_{1}|X=x\quad\forall
x\in\mathcal{X}$

######  Assumption 5 (Conditional dependence between the mediator and mediator
instrument).

$\displaystyle M\not\\!\perp\\!\\!\\!\perp Z_{2}|D=d,X=x\quad\forall
d\in\mathcal{D}\textrm{ and }x\in\mathcal{X}$

We note that $\not\\!\perp\\!\\!\\!\perp$ denotes statistical dependence.
Assumptions 4 and 5 can be directly tested in the data by investigating the
conditional associations of $D$ and $Z_{1}$ as well as $M$ and $Z_{2}$. If
these assumptions do not hold, the testing approach lacks power in detecting
violations of the identifying assumptions, as discussed in more detail in
Huber and Kueck (2022).

The next assumptions invoke the conditional independence of the treatment on
the one hand and the potential outcomes and potential mediators on the other
hand conditional on the covariates, with ${\perp\\!\\!\\!\perp}$ denoting
statistical independence. These kind of assumptions are known as treatment
exogeneity, selection-on-observables, or unconfoundedness, as for instance
discussed in Imbens (2004) and Imbens and Wooldridge (2009). The imply that
given the covariates, the treatment is as good as random when aiming to assess
its effect on the outcome or the mediator, respectively.

######  Assumption 6a (Conditional independence of the treatment and potential
outcomes).

$\displaystyle Y(d,m){\perp\\!\\!\\!\perp}D|X=x\quad\forall
d\in\mathcal{D},m\in\mathcal{M},\textrm{ and }x\in\mathcal{X}$

######  Assumption 6b (Conditional independence of the treatment and potential
mediators).

$\displaystyle M(d){\perp\\!\\!\\!\perp}D|X=x\quad\forall
d\in\mathcal{D},m\in\mathcal{M},\textrm{ and }x\in\mathcal{X}$

Assumptions 6a and 6b require that conditional on covariates $X$, there exist
no confounders jointly affecting the treatment $D$ on the one hand and the
outcome $Y$ or the mediator $M$ on the other hand. This permits identifying
the causal effect of $D$ on $M$ or $Y$ when controlling for $X$.

Next, we impose a related conditional independence assumption also w.r.t. to
the mediator, requiring that the latter is conditionally independent of the
potential outcomes conditional on the treatment and the pre-treatment
covariates.

######  Assumption 7 (Conditional independence of the mediator).

$\displaystyle Y(d,m){\perp\\!\\!\\!\perp}M|D=d,X=x\quad\forall
d\in\mathcal{D},m\in\mathcal{M},\textrm{ and }x\in\mathcal{X}$

Assumption 7 requires that conditional on $D$ and $X$, there exist no
confounders jointly affecting the mediator $M$ and the outcome $Y$. This
permits identifying the causal effect of $M$ on $Y$ when controlling for $D$
and $X$.

Our next assumptions concern the validity of the first instrument $Z_{1}$ and
require that the latter is conditionally independent of the potential outcomes
and the potential mediators given the covariates $X$.

######  Assumption 8a (Conditional independence of the treatment instrument
and potential outcomes).

$\displaystyle Y(d,m){\perp\\!\\!\\!\perp}Z_{1}|X=x\quad\forall
d\in\mathcal{D},m\in\mathcal{M},\textrm{ and }x\in\mathcal{X}$

######  Assumption 8b (Conditional independence of the treatment instrument
and potential mediators).

$\displaystyle M(d){\perp\\!\\!\\!\perp}Z_{1}|X=x\quad\forall
d\in\mathcal{D}\textrm{ and }x\in\mathcal{X}$

Assumptions 8a and 8b rule out confounders jointly affecting $Z_{1}$ on the
one hand and the outcome $Y$ or the mediator $M$ on the other hand when
controlling for $X$, which is similar to Assumptions 6a and 6b, which imposed
such an exogeneity assumption w.r.t. the treatment. In addition, Assumptions
8a and 8b require that conditional on $X$, instrument $Z_{1}$ does not
directly affect $M$ or $Y$ other than through $D$. This exclusion restriction
implies that $M(d,z_{1})=M(d)$ and $Y(d,m,z_{1})=Y(d,m)$ for any value $z_{1}$
of instrument $Z_{1}$, otherwise the respective conditional independence
assumptions are violated. Assumptions 8a and 8b are not sufficient for the
nonparametric identification of the causal effects of $D$ on $Y$ or $M$ based
on the instrument $Z_{1}$, which would require further assumptions like
treatment monotonicity as discussed in Imbens and Angrist (1994) and Angrist,
Imbens, and Rubin (1996). However, Assumptions 8a and 8b are useful for
testing the identification of the causal effects of $D$ on $M$ and $Y$ as
outlined further below.

Lastly, we also invoke an instrument validity assumption on the second
instrument $Z_{2}$, requiring it to be conditionally independent of the
potential outcomes conditional on $D$ and $X$.

######  Assumption 9 (Conditional independence of the mediator instrument).

$\displaystyle Y(d^{\prime},m){\perp\\!\\!\\!\perp}Z_{2}|D=d,X=x\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M},\textrm{ and }x\in\mathcal{X}$

Assumption 9 rules out confounders jointly affecting $Z_{2}$ and $Y$
conditional on $D$ and $X$, and that $Z_{2}$ does not directly affect $Y$
(other than through $M$) such that the exclusion restriction
$Y(d,m,z_{2})=Y(d,m)$ holds for any value $z_{2}$ of $Z_{2}$. This assumption
is useful for testing the identification of the causal effects of $M$ on $Y$.

If one is willing to impose the causal structure and faithfulness postulated
in Assumption 1 along with the (testable) conditional dependencies of the
respective instruments and the treatment or the mediator as invoked in
Assumptions 4 and Assumption 5, then the joint satisfaction of the conditional
independence assumptions on the treatment, mediators, and instruments imposed
in Assumptions 6a, 6b, 7, 8a, 8b, and 9 implies the following testable
implications, which under the common support assumptions 2 and 3 are
verifiable for all values in the support of the respective conditioning set:

$\displaystyle Y$ $\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x\quad$
$\displaystyle\forall d\in\mathcal{D}\textrm{ and }x\in\mathcal{X},$ (TIa)
$\displaystyle M$ $\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x\quad$
$\displaystyle\forall d\in\mathcal{D}\textrm{ and }x\in\mathcal{X},$ (TIb)
$\displaystyle Y$ $\displaystyle{\perp\\!\\!\\!\perp}Z_{2}|D=d,M=m,X=x\quad$
$\displaystyle\forall d\in\mathcal{D},m\in\mathcal{M}\textrm{ and
}x\in\mathcal{X}.$ (TIc)

Furthermore, conditional on Assumptions 1, 4, 5, the joint satisfaction of the
testable implications (TIa), (TIb), (TIc) is not only implied by, but also
imply the joint satisfaction of the conditional independence assumptions 6a,
6b, 7, 8a, 8b, 9, as formalized in the following theorem, which is proven in
the appendix.

######  Theorem 1.

$\text{Under Assumptions }\ref{A1},\ref{A4},\ref{A5}:\ \text{Assumptions
}\ref{A6a},\ref{A6b},\ref{A7},\ref{A8a},\ref{A8b},\ref{A9}\iff(\ref{TIa}),(\ref{TIb}),(\ref{TIc}).$

In practice, we may use the result in Theorem 1 to test the satisfaction of
the conditional independence assumptions for factual values of mediators and
outcomes, i.e., for the potential outcomes $Y(d,m)$ and potential mediators
$M(d)$ among subjects actually receiving $D=d$ and $M=d$, but not for
counterfactual values $d^{\prime}\neq d$ or $m^{\prime}\neq m$ for those
subjects with $D=d$ and $M=d$. If violations of the conditional independence
assumptions 6a, 6b, 7, 8a, 8b, 9 exclusively concern counterfactual outcomes
and mediators, but never (f)actual outcomes or mediators, testing cannot
detect such violations. Therefore, our approach permits testing necessary, but
not sufficient conditions for the identification of various causal effects
discussed below. From a practical perspective, however, it seems unlikely that
violations exclusively occur among counterfactual, but not among factual
outcomes and mediators, because this would imply very particular, not to say
implausible modeling constraints. Therefore, we would expect our test to have
power in empirical applications, as violations of conditional independence
should typically not affect only counterfactual outcomes or mediators, but
also (at least some) factual ones.

We note that if the conditions (TIa),(TIb),(TIc) hold for both factual and
counterfactual mediators or outcomes, the following causal effects on the
distributions of the mediator or the outcome are identified:

  * •

The effect of treatment $D$ on outcome $Y$, e.g., the ATE of treatment levels
$d\neq d^{\prime}$, $E[Y(d,M(d))-Y(d^{\prime},M(d^{\prime}))]$, where
identification follows from the conditional independence of the treatment
invoked in Assumption 6a.

  * •

The effect of treatment $D$ on mediator $M$, e.g., the ATE
$E[M(d)-M(d^{\prime})]$, where identification follows from the conditional
independence of the treatment invoked in Assumption 6b.

  * •

The effect of mediator $M$ on outcome $Y$, e.g., the ATE of mediator levels
$m\neq m^{\prime}$, $E[Y(D,m)-Y(D,m^{\prime})]$, where identification follows
from the conditional independence of the mediator invoked in Assumption 7.

  * •

The dynamic effect of specific sequences of treatment $D$ and mediator $M$ on
outcome $Y$, e.g., the ATE $E[Y(d,m)-Y(d^{\prime},m^{\prime})]$, including the
average controlled direct effect of the treatment on the outcome conditional
on fixing the mediator at some value $M=m$, $E[Y(d,m)-Y(d^{\prime},m)]$, where
identification follows from Assumptions 6a and 7, see for instance the
discussions in Lechner and Miquel (2010) and Huber (2023), Chapters 4.9 and
4.10.

  * •

The effect of treatment $D$ on outcome $Y$ in sample selection models, where
$M$ represents a binary indicator for the observability of $Y$ (but not a
mediator, because $M$ does not affect $Y$ in sample selection models).
Identification follows from Assumptions 6a and 7 when replacing $Y(d,m)$ by
$Y(d)$ (as $M$ does not affect $Y$ in sample selection models), see for
instance the discussions in Bia, Huber, and Lafférs (2024) and Huber (2023),
Chapter 4.11.

In causal mediation analysis, the controlled direct effect, which is based on
forcing or prescribing the mediator to take a specific value $M=m$, is
typically not the only causal parameter of interest. A large literature also
focuses on natural direct and indirect effects defined in terms of potential
(rather than prescribed) mediator states, as for instance discussed in Robins
and Greenland (1992), Pearl (2001), Imai, Keele, and Yamamoto (2010), and
Huber (2014). For instance, the average natural direct treatment effect of
$d\neq d^{\prime}$ on $Y$ conditional the potential mediator under treatment
value $d$, $M(d)$, corresponds to $E[Y(d,M(d))-Y(d^{\prime},M(d))]$, while the
average natural indirect effect of $D$ on $Y$ via $M$ when fixing the
treatment at $D=d^{\prime}$ is $E[Y(d,M(d))-Y(d,M(d^{\prime}))]$.

It is worth noting that the identification of natural effects not only hinges
on the satisfaction of Assumptions 6a, 6a, and 7 for both factual and
counterfactual outcomes, but even on additional requirements. One additional
condition that yields identification and has been suggested in Pearl (2001) is
invoking conditional independence of potential mediators and potential
outcomes across treatment states $d\neq d^{\prime}$:

$\displaystyle Y(d,m)$
$\displaystyle{\perp\\!\\!\\!\perp}M(d^{\prime})|X=x\quad\forall
d,d^{\prime}\in\mathcal{D},m\in\mathcal{M},\textrm{ and }x\in\mathcal{X}.$
(2.1)

It is obvious that this conditional independence is an inherently
counterfactual assumption, as either only $d$ or $d^{\prime}$ can be observed
for any subject in the sample. For this reason, we cannot test this assumption
in the data. Furthermore, it is worth noting that the conditional independence
in (2.1) as well as Assumptions 6a and 6b are implied by the following, joint
conditional independence assumption of Imai, Keele, and Yamamoto (2010), which
they impose in addition to Assumption 7 for the identification of natural
direct and indirect effects:

$\displaystyle\\{Y(d,m),M(d^{\prime})\\}$
$\displaystyle{\perp\\!\\!\\!\perp}D|X=x\quad\forall
d,d^{\prime}\in\mathcal{D},m\in\mathcal{M},\textrm{ and }x\in\mathcal{X}.$
(2.2)

The conditional independence assumption in expression (2.2) implies that the
joint distribution of potential outcomes and potential mediators is
conditionally independent of treatment assignment. In contrast, Assumptions 6a
and 6b only impose conditional independence w.r.t. the marginal distributions
of the potential outcomes and potential mediators. Yet, we argue that testing
Assumptions 6a and 6b for actual outcomes typically also has nontrivial power
against violations of the counterfactual condition (2.2). The reason is that
only under quite specific mediator and outcome models, it can be the case that
Assumptions 6a and 6b always hold for factual mediators and outcomes, while we
have at the same time that the conditional independence (2.2) is violated for
certain counterfactual outcomes and mediators. For instance, if the treatment
is randomly assigned given $X$, then both the joint conditional independence
assumption (2.2) as well as Assumptions 6a and 6b on the marginal
distributions are satisfied. However, if conditional randomization of $D$
fails, then we would suspect violations of all assumptions (2.2), 6a, 6b, and
even so for at least some factual outcomes and mediators.

Figure 1 presents a causal model in which the identifying assumptions of the
various causal effects discussed above as well as the testable implications
given in Theorem 1 are satisfied, based on a causal graph in which causal
relations between variables are represented by arrows2, see e.g. Pearl (2000).
Treatment $D$ affects outcome $Y$ both directly and via the mediator $M$ and
is conditionally independent of potential mediators and outcomes, because no
unobserved variables, whose effects are depicted by dashed lines (to indicate
their non-observability), jointly affect $D$ and the post-treatment variables
$M$ and $Y$ conditional on covariates $X$. Analogously, there are no
unobservables jointly affecting the mediator $M$ and the outcome $Y$
conditional on $X$ and $D$. Furthermore, there are no confounders of the
instrument for the treatment, $Z_{1}$, and the $M$ or $Y$ conditional on $X$,
and $Z_{1}$ does not directly affect $M$ or $Y$ other than through $D$.
Moreover, there are no confounders of the instrument for the mediator,
$Z_{2}$, and $Y$ conditional on $X$ and $D$, and $Z_{2}$ does not directly
affect $Y$ other than through $M$.

Figure 1: Causal models satisfying the identification results in Theorem
1$Z1$$D$$M$$Y$$Z2$$X$$U1$$U2$$U4$$U5$$U3$

It is worth noting that if $Z_{2}$ has a direct causal effect on $M$ as in
Figure 1, then the satisfaction of the identifying assumptions underlying
Theorem 1 requires that $Z_{1}$ and $Z_{2}$ are conditionally independent of
each other given $D$ and $X$. To see this, note that if e.g. $Z_{1}$ has a
direct effect on $Z_{2}$ conditional on $D$ and $X$ and $Z_{2}$ has a direct
effect on $M$, $Z_{1}$ is a confounder that jointly affects treatment $D$ (by
Assumption 4) on the one hand and $M$ and $Y$ on the other hand. Related
issues also occur if unobserved confounders affect both $Z_{1}$ and $Z_{2}$
and $Z_{2}$ directly affects $M$. The left graph of Figure 2 provides such a
causal model in which Assumptions 6a and 6b are violated.

If, on the contrary, $Z_{2}$ is associated with $M$ solely through unobserved
confounders of both $Z_{2}$ and $M$ (rather than through a direct effect),
then all identifying assumptions in Theorem 1 hold even if $Z_{1}$ has a
direct causal effect on $Z_{2}$ given $D$ and $X$, and/or unobserved
confounders affect both $Z_{1}$ and $Z_{2}$, as illustrated in the right graph
of Figure 2. The rationale behind this is that $Z_{2}$ does not lie on any
causal pathway through which the treatment $D$ influences the mediator $M$, as
$Z_{2}$ does not affect $M$. Here, adjusting for $Z_{2}$ would even be harmful
as it would introduce so-called M-bias, a specific form of collider or
selection bias, see e.g. Pearl (2000). The reason is that by conditioning on
$Z_{2}$, one introduces a spurious association between the treatment $D$ and
the unobservable $U_{2}$ which also affects $M$ (and $Y$ via $M$). Such a
spurious association comes from the fact that $D$ either directly affects the
collider variable $Z_{2}$ or is associated with the latter through $U_{1}$
(which both affects $Z_{2}$ and $D$ via $Z_{1}$).

Figure 2:  Causal models with conditional associations between the instruments

$Z1$$D$$M$$Y$$Z2$$X$$U1$$U2$$U4$$U5$$U3$
$Z1$$D$$M$$Y$$Z2$$X$$U1$$U2$$U4$$U3$$U5$

##  3 Modifications when controlling for the mediator instrument

In causal models in which the first instrument $Z_{1}$ affects the second
instrument $Z_{2}$ and $Z_{2}$ affects the mediator $M$, controlling for
$Z_{2}$ is required to block any causal effects of $Z_{1}$ on $M$ or $Y$ that
do not operate via the treatment $D$, conditional on covariates $X$. This
scenario is illustrated in the left graph of Figure 2, where controlling for
$Z_{2}$ does not introduce collider bias, very much in contrast to the
scenario depicted in the right graph of Figure 2. Consequently, there may
exist an alternative testing approach for identification that includes $Z_{2}$
in some of the conditioning sets when verifying the conditional independence
of $Z_{1}$ and $Y$. For this reason, we subsequently modify the conditional
independence assumptions imposed on the treatment and the first instrument to
hold when controlling for both $X$ and $Z_{2}$, rather than $X$ alone. To this
end, we replace Assumptions 6a, 6b, 8a, and 8b by the following assumptions:

######  Assumption 6am (Conditional independence of the treatment and
potential outcomes).

$\displaystyle Y(d,m){\perp\\!\\!\\!\perp}D|X=x,Z_{2}=z_{2}\quad\forall
d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X},\textrm{ and
}z_{2}\in\mathcal{Z_{2}}$

######  Assumption 6bm (Conditional independence of the treatment and
potential mediators).

$\displaystyle M(d){\perp\\!\\!\\!\perp}D|X=x,Z_{2}=z_{2}\quad\forall
d\in\mathcal{D},x\in\mathcal{X},\textrm{ and }z_{2}\in\mathcal{Z_{2}}$

######  Assumption 8am (Conditional independence of the treatment instrument
and potential outcomes).

$\displaystyle Y(d,m){\perp\\!\\!\\!\perp}Z_{1}|X=x,Z_{2}=z_{2}\quad\forall
d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X},\textrm{ and
}z_{2}\in\mathcal{Z_{2}}$

######  Assumption 8bm (Conditional independence of the treatment instrument
and potential mediators).

$\displaystyle M(d){\perp\\!\\!\\!\perp}Z_{1}|X=x,Z_{2}=z_{2}\quad\forall
d\in\mathcal{D},x\in\mathcal{X},\textrm{ and }z_{2}\in\mathcal{Z_{2}}$

This adjustment in the identifying assumptions yields the following testable
implications instead of the previous ones (TIa) and (TIb), while the previous
implication (TIc) remains unaffected:

$\displaystyle Y{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x,Z_{2}=z_{2}\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{2}\in\mathcal{Z_{2}},$ (TIam)
$\displaystyle M{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x,Z_{2}=z_{2}\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{2}\in\mathcal{Z_{2}}.$ (TIbm)

We note that there exists even a further testable implication, which follows
from the fact that any causal association of $Z_{1}$ on $Y$ via $Z_{2}$
necessarily operates through $M$:

$\displaystyle Y{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x,M=m\quad\forall
d\in\mathcal{D},x\in\mathcal{X},\textrm{ and }m\in\mathcal{M}.$ (TId)

This testable implication (TId), however, does not add any new information and
thus, no additional power in testing when compared to the previous testable
implications. This is formalized in the following lemma, which states that
given Assumption 1 and implications (TIbm), (TIc), (TIam) implies (TId) and
vice versa:

######  Lemma 1.

$\displaystyle\textrm{Under Assumption
}\ref{A1}:(\ref{TIam}),(\ref{TIbm}),(\ref{TIc})\implies(\ref{TId})$
$\displaystyle\textrm{Under Assumption
}\ref{A1}:(\ref{TIbm}),(\ref{TIc}),(\ref{TId})\implies(\ref{TIam})$

The following theorem formalizes that our modified set of identifying
assumptions, which involve conditioning on $Z_{2}$, implies and is implied by
the respective testable implications:

######  Theorem 2.

$\displaystyle\text{Under Assumptions }\ref{A1},\ref{A4},\ref{A5}:\
\text{Assumptions
}\ref{A6am},\ref{A6bm},\ref{A7},\ref{A8am},\ref{A8bm},\ref{A9}$
$\displaystyle\iff(\ref{TIam}),(\ref{TIbm}),(\ref{TIc})$
$\displaystyle\iff(\ref{TIbm}),(\ref{TIc}),(\ref{TId}).$

The proofs of Lemma 1 and Theorem 2 are provided in the appendix.

It is important to note even though the testable implications involve
conditioning on $Z_{2}$, this does not automatically imply that controlling
for $Z_{2}$ is also appropriate for the identification of causal effects. In
fact, if the testable implications in Theorem 2 hold and $Z_{2}$ is a
confounder of $D$ and $M$, or unobserved confounders affect both $D$ and
$Z_{2}$ while $Z_{2}$ affects $M$, then controlling for $Z_{2}$ (along with
$X$) is required when evaluating the causal effect of $D$ on $M$ or $Y$.
However, if $D$ affects $Z_{2}$ and no unobserved confounders affect $D$ and
$Z_{2}$, then controlling for $Z_{2}$ is only required for testing, but not
for evaluating the causal effects of $D$. In fact, conditioning on $Z_{2}$
would block any impact of $D$ on $M$ and $Y$ that operates via $Z_{2}$, such
that only a partial effect could be identified.

##  4 Pre- and post-treatment covariates

In many empirical applications, it may not hold that pre-treatment covariates
are sufficiently informative to fully control for confounders jointly
affecting the mediator $M$ and outcome $Y$, implying a violation of the
conditional independence of the mediator as imposed by Assumption 7. This
issue appears particularly relevant if the mediator is measured at a
substantially later point in time than the treatment. Just as the control
variables of the treatment are typically measured shortly prior to treatment
assignment, it then seems reasonable to also control for possible confounders
of the mediator-outcome relation shortly prior to selection into the mediator.
In such a scenario, several control variables for the mediator might be
affected by the treatment. For instance, when assessing the treatment effect
of education ($D$) on earnings ($Y$) when considering work experience as
mediator ($M$), post-treatment covariates like health or mental well-being
($W$) might affect both $M$ and $Y$. For this reason, the subsequent
discussion considers the case that the treatment can influence observed post-
treatment confounders of the mediator-outcome relation to be controlled for,
which are denoted by $W$, while $\mathcal{W}$ denotes their support. Figure 3
depicts a causal graph in which both pre-treatment covariates $X$ and post-
treatment covariates $W$ are present. $W$ may be affected by treatment $D$,
pre-treatment covariates $X$, and the first instrument $Z_{1}$, and may itself
affect mediator $M$, outcome $Y$, and the second instrument $Z_{2}$.

Figure 3:  Pre- and post-treatment covariates
$Z1$$D$$M$$Y$$W$$Z2$$U1$$X$$U4$$U5$$U3$$U2$

For our testing approach, this implies that the post-treatment covariates $W$
now are required to enter the conditioning set when imposing the conditional
independence assumptions with respect to the second instrument and the
mediator. To ensure that the test has power, there must be a first-stage
association between instrument $Z_{2}$ and treatment $M$ when also controlling
for $W$ in addition to $D$ and $X$. We also rule out that $M$ has a direct
effect on $W$, based on the fact that $W$ are pre-mediator confounders and
thus cannot be causally affected by the mediator. For these reasons, we
replace Assumptions (1), (5), (7), and (9) in Section 2 by the following four
assumptions.

######  Assumption 1m (Causal structure and faithfulness).

$\displaystyle
M(y)=M,D(m,y,z_{2})=D,X(d,m,y,z_{2})=X,Z_{1}(d,m,y,z_{2})=Z_{1},Z_{2}(m,y)=Z_{2},$
$\displaystyle\;\textrm{ and }W(m)=W,$ $\displaystyle\forall
d\in\mathcal{D},m\in\mathcal{M},z_{2}\in\mathcal{Z}_{2}\textrm{ and
}y\in\mathcal{Y},$

only variables which are d-separated in any causal model are statistically
independent.

######  Assumption 5m (Conditional dependence between the mediator and
mediator instrument).

$\displaystyle M\not\\!\perp\\!\\!\\!\perp Z_{2}|D=d,X=x,W=w\quad\forall
d\in\mathcal{D},x\in\mathcal{X},\textrm{ and }w\in\mathcal{W}$

######  Assumption 7m (Conditional independence of the mediator).

$\displaystyle Y(d^{\prime},m){\perp\\!\\!\\!\perp}M|D=d,X=x,W=w\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X},\textrm{ and
}w\in\mathcal{W}$

######  Assumption 9m (Conditional independence of the mediator instrument).

$\displaystyle
Y(d^{\prime},m){\perp\\!\\!\\!\perp}Z_{2}|D=d,X=x,W=w\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X},\textrm{ and
}w\in\mathcal{W}$

Under these modifications of also including $W$ in the conditioning set, the
testable implication (TIc) is substituted by the following implication, which
states that the second instrument is conditionally independent of the outcome
given the treatment, the mediator, as well as the pre- and post-treatment
covariates:

$\displaystyle Y{\perp\\!\\!\\!\perp}Z_{2}|D=d,M=m,X=x,W=w\quad\forall
d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X},\textrm{ and
}w\in\mathcal{W}.$ (TIe)

Similar to the previously analyzed causal scenarios, our revised set of
identifying assumptions, which includes conditioning on $W$, implies and is
implied by a corresponding set of testable implications. This result is
formally stated in the following theorem.

######  Theorem 3.

$\displaystyle\text{Under Assumptions }\ref{A1m},\ref{A4},\ref{A5m}:\
\text{Assumptions
}\ref{A6a},\ref{A6b},\ref{A7m},\ref{A8a},\ref{A8b},\ref{A9m}$
$\displaystyle\iff(\ref{TIa}),(\ref{TIb}),(\ref{TIe})$

The proof is provided in the appendix.

We note that if the conditional independence assumptions 6a, 6b, 7m, 8a, 8b,
and 9m hold for both factual and counterfactual mediators or outcomes, the
effect of treatment $D$ on outcome $Y$ and mediator $M$ (by Assumptions 6a and
6b), the effect of the mediator $M$ on outcome $Y$ (by Assumption 7m), and the
dynamic effect of specific sequences of $D$ and $M$, including the controlled
direct effect, (by Assumptions 6a and 7m) are identified, as well as the
effect of treatment $D$ on outcome $Y$ in sample selection models, where $M$
represents a binary indicator for the observability of $Y$ (by Assumptions 6a
and 7m). See, for instance, Lechner and Miquel (2010), Bia, Huber, and Lafférs
(2024), and Huber (2023) for discussions of identification in dynamic
treatment and sample selection models with post-treatment control variables.
Concerning natural direct and indirect effects, identification is not attained
under post-treatment confounders even when imposing the joint conditional
independence assumption in expression (2.2) along with Assumption 7m and
analogous conditional independence assumption on $W$. In contrast to the case
when all confounders are pre-treatment, identification under post-treatment
confounders is infeasible without specific parametric conditions. See, for
instance, the discussions in Avin, Shpitser, and Pearl (2005), Robins (2003),
who imposes the absence of treatment-mediator interaction effects on the
outcome to obtain identification, and Imai and Yamamoto (2013), who assume
that any treatment-mediator interaction effects are homogeneous across
subjects.

##  5 Testing

This section introduces our testing approach, focusing primarily on the
assumptions and testable implications discussed in Section 2. However, we note
that the causal scenarios outlined in Sections 3 and 4 can also be
accommodated by appropriately adjusting the conditioning sets during testing.
Moreover, instead of verifying complete statistical independence as stipulated
in the testable implications (TIa), (TIb), and (TIc), we test the conditional
mean independence of the outcome or mediator and the instruments, which is
sufficient for identifying (conditional or unconditional) average causal
effects or the treatment and the mediator. To formalize the testable
implications in terms of conditional mean independence, we denote by
$\mu_{B}(a)=E(B|A=a)$ the conditional mean of random variable $B$, where $A$
represents a set of conditioning variables taking values $a$. Then, testing
whether implications (TIa), (TIb), (TIc) hold on average involves verifying
the following null hypothesis.

$\displaystyle H_{0}:$
$\displaystyle\mu_{Y}(d,x)-\mu_{Y}(d,x,z_{1})=0\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{1}\in\mathcal{Z}_{1},$
$\displaystyle\mu_{M}(d,x)-\mu_{M}(d,x,z_{1})=0\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{1}\in\mathcal{Z}_{1},$
$\displaystyle\mu_{Y}(d,m,x)-\mu_{Y}(d,m,x,z_{2})=0\quad\forall
d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X}\textrm{ and
}z_{2}\in\mathcal{Z}_{2},$ (H0)

The intuition behind hypothesis (5) is that if $Y$ and $M$ are conditionally
mean independent of the instruments, then whether the instruments are included
or excluded from the respective conditioning sets should not alter the
conditional means of the outcome and mediator, respectively. Following Huber
and Kueck (2022), our test relies on a quadratic formulation of the null
hypothesis, which is a common approach in specification tests for
nonparametric regression, see e.g. Racine (1997), Racine, Hart, and Li (2006),
Hong and White (1995) and Wooldridge (1992). To this end, let us denote by

$\displaystyle\theta=E\begin{pmatrix}{\left(\mu_{Y}(D,X)-\mu_{Y}(D,X,Z_{1})\right)}^{2}\\\
{\left(\mu_{M}(D,X)-\mu_{M}(D,X,Z_{1})\right)}^{2}\\\
{\left(\mu_{Y}(D,M,X)-\mu_{Y}(D,M,X,Z_{2})\right)}^{2}\end{pmatrix},$ (5.1)

which implies the null hypothesis

$\displaystyle H_{0}:\theta=0.$ (5.2)

We note that testing based on $\theta$ can accommodate both discrete and
continuous variables for the instruments, treatment, mediator, outcome, and
covariates. In contrast, Huber and Kueck (2022) focused on a binary
instrument, testing the squared difference in conditional mean outcomes with
instrument values of zero versus one, rather than comparing the inclusion
versus exclusion of (possibly non-binary) instruments, as proposed here.
However, akin to Huber and Kueck (2022), we test the null hypothesis (5.2)
using a moment condition which makes use of the following score function to
verify whether it is mean zero:

$\displaystyle\phi(V,\theta,\eta)={(\eta_{1}(V)-\eta_{2}(V))}^{2}-\theta+\zeta,$
(5.3)

where $V=(Y,D,M,X,Z_{1},Z_{2})$ represents the random variables, while
$\eta_{1}(V)=(\mu_{Y}(D,X)$, $\mu_{M}(D,X)$, $\mu_{Y}(D,M,X))^{\prime}$,
$\eta_{2}(V)=(\mu_{Y}(D,X,Z_{1}),\mu_{M}(D,X,Z_{1}),\mu_{Y}(D,M,X,Z_{2}))^{\prime}$
are the respective conditional means. Additionally, $\zeta$ is an independent
mean-zero random variable with $\|\zeta\|_{P,q}<C$, where $C$ is a positive
constant, for $q>2$ and variance $\sigma_{\zeta}^{2}>0$.111For any random
vector $R=(R_{1},...,R_{l}),$ we have $||R||_{q}=\max_{1\leq j\leq
l}||R_{l}||_{q},$ where $||R_{l}||_{q}=(E[|R_{l}|^{q}])^{\frac{1}{q}}.$

Introducing the random perturbation $\zeta$ serves to prevent our testing
approach from having a degenerate variance under the null hypothesis $H_{0}$.
The variance $\sigma_{\zeta}^{2}$ acts as a tuning parameter that should be
adjusted based on the sample size $n$. This adjustment involves a trade-off: a
higher variance brings the testing based on $\phi$ closer to the nominal size
in smaller samples, but it may diminish the test’s power by potentially
obscuring genuine deviations from the null hypothesis. As shown in Huber and
Kueck (2022), score functions of this quadratic type satisfy Neyman
orthogonality under the null hypothesis, which implies that estimators and
tests based on such score functions are relatively robust to misspecifications
of the conditional mean outcomes, see Chernozhukov, Chetverikov, Demirer,
Duflo, Hansen, Newey, and Robins (2018). This robustness is particularly
advantageous when estimating the conditional mean outcomes using machine
learning techniques, which often entail regularization bias in estimation.

For estimating the target parameter $\theta$, we employ cross-fitting as
described in Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, Newey, and
Robins (2018). This approach involves estimating the models for the
conditional mean outcomes and the score function (5.3) underlying the test
using distinct observations from the sample to mitigate overfitting. We assume
an i.i.d. sample with $n$ subjects, where $i$ ranges from 1 to $n$,
representing the index of subjects in the sample. The subjects are randomly
divided into $K$ subsamples or folds of size $n/K$, which is assumed to be an
integer for simplicity. Let $I_{k_{i}}$ with $k_{i}$ ranging from 1 to $K$
denote the fold containing subject $i$, and $I_{k_{i}}^{c}$ denote its
complement, consisting of all remaining folds excluding subject $i$. For any
subject $i$ in fold $k_{i}$, we obtain predictions
$\hat{\eta}_{1}^{k_{i}}(V_{i})$ and $\hat{\eta}_{2}^{k_{i}}(V_{i})$ of the
true conditional means $\eta_{1}(V_{i})$ and $\eta_{2}(V_{i})$ respectively,
based on estimating the model parameters (e.g., coefficients in a lasso
regression) of the conditional means in the complement set $I_{k_{i}}^{c}$.
The cross-fitted estimator of $\theta$ then corresponds to

$\displaystyle\hat{\theta}=\frac{1}{3n}\sum_{i=1}^{n}\begin{pmatrix}\left(\hat{\mu}_{Y}^{k_{i}}(D_{i},X_{i})-\hat{\mu}_{Y}^{k_{i}}(D_{i},X_{i},Z_{1,i})\right)^{2}+\zeta_{i}\\\
+\left(\hat{\mu}_{M}^{k_{i}}(D_{i},X_{i})-\hat{\mu}_{M}^{k_{i}}(D_{i},X_{i},Z_{1,i})\right)^{2}+\zeta_{i}\\\
+\left(\hat{\mu}_{Y}^{k_{i}}(D_{i},M_{i},X_{i})-\hat{\mu}_{Y}^{k_{i}}(D_{i},M_{i},X_{i},Z_{2,i})\right)^{2}+\zeta_{i}\end{pmatrix},$
(5.4)

with
$(\hat{\mu}_{Y}^{k_{i}}(D_{i},X_{i}),\hat{\mu}_{M}^{k_{i}}(D_{i},X_{i}),\hat{\mu}_{Y}^{k_{i}}(D_{i},M_{i},X_{i}))^{\prime}=\hat{\eta}_{1}^{k_{i}}(V_{i})$
and
$(\hat{\mu}_{Y}^{k_{i}}(D_{i},X_{i},Z_{1,i}),\hat{\mu}_{M}^{k_{i}}(D_{i},X_{i},,Z_{1,i}),\hat{\mu}_{Y}^{k_{i}}(D_{i},M_{i},X_{i},Z_{2,i}))^{\prime}=\hat{\eta}_{2}^{k_{i}}(V_{i})$.

The results in Huber and Kueck (2022), although focusing on a binary
instrument, imply that under the null hypothesis $H_{0}$, the estimator
$\hat{\theta}$ follows an asymptotically normal distribution, provided certain
regularity conditions are met, such as a convergence rate of the estimators
$\hat{\eta}_{1}^{k_{i}}(V_{i})$ and $\hat{\eta}_{2}^{k_{i}}(V_{i})$ satisfying
$o(n^{-1/4})$. Appendix A.2 provides a formal proof for the Neyman
orthogonality of score function $\phi(V,\theta,\eta)$ and the asymptotic
normality of testing under the null hypothesis, along with the required
regularity conditions. However, we point out that under the alternative
hypothesis, the estimator is non-normal. Regarding the variance of
$\hat{\theta}$, we note that for any subject $i$, the three squared
differences adjusted by $\zeta_{i}$ within the brackets of expression (5.4)
are not independent of each other. Consequently, the conventional variance
formula of the score function (5.3),
$E[(\eta_{1}(V)-\eta_{2}(V))^{4}]+\sigma_{\zeta}^{2}$, is inappropriate for
inference under the null hypothesis. For this reason, we apply cluster-robust
variance estimation by clustering at the subject-level to address this issue.

##  6 Simulations

This section presents a simulation study aimed at evaluating the finite sample
performance of our testing approach suggested in the previous section. Our
first simulation design is based on the following model, which is related to
the the causal framework considered in Section 2:

$\displaystyle D$ $\displaystyle=$ $\displaystyle
I\\{X^{\prime}\beta+0.5Z_{1}+U_{1}>0\\},$ (6.1) $\displaystyle M$
$\displaystyle=$ $\displaystyle 0.5D+0.5Z_{2}+X^{\prime}\beta+\delta
U_{1}+U_{2},$ $\displaystyle Y$ $\displaystyle=$ $\displaystyle
D+0.5M+X^{\prime}\beta+\gamma Z_{1}+\gamma Z_{2}+\delta U_{1}+U_{3},$
$\displaystyle X$ $\displaystyle\sim$
$\displaystyle\mathcal{N}(0,\sigma^{2}_{X}),Z_{1}\sim\mathcal{N}(0,1),Z_{2}\sim\mathcal{N}(0,1),$
$\displaystyle U_{1}$ $\displaystyle\sim$
$\displaystyle\mathcal{N}(0,1),U_{2}\sim\mathcal{N}(0,1),U_{3}\sim\mathcal{N}(0,1),$

with $X$, $Z_{1}$, $Z_{2}$, $U_{1}$, $U_{2}$, and $U_{3}$ being independent of
each other. The treatment variable $D$ is a binary variable modeled by an
indicator function $I\\{...\\}$ whose value is determined by pre-treatment
covariates $X$, given that the coefficient vector $\beta$ is nonzero, the
first instrument $Z_{1}$, and the unobservable $U_{1}$. The mediator $M$ is a
linear function of $X$ (if coefficient vector $\beta\neq 0$), treatment $D$,
instrument $Z_{2}$, unobservable $U_{2}$, and unobservable $U_{1}$ (if
coefficient $\delta\neq 0$). The outcome variable $Y$ is a linear function of
treatment $D$, mediator $M$, covariates $X$ (if $\beta\neq 0)$, instruments
$Z_{1}$ and $Z_{2}$ (if coefficient $\gamma\neq 0$), unobservable $U_{3}$, and
unobservable $U_{1}$ (if $\delta\neq 0$). The causal model implies that the
(total) treatment effect is $1+0.5\cdot 0.5=1.25$, the direct effect (not
operating via the mediator $M$) is $1$, and the indirect (of $D$ on $Y$ via
$M$) is $0.5\cdot 0.5=0.25$. The unobserved terms $U_{1},U_{2},U_{3}$ and the
instruments $Z_{1},Z_{2}$ are standard normally distributed and independent of
each other and of $X$. $X$ is a vector of normally distributed covariates with
zero means and a covariance matrix $\sigma^{2}_{X}$, where the covariance of
the $i$th and $j$th covariate in $X$ corresponds to $0.5^{|i-j|}$. The
coefficient vector $\beta$ gauges the effects of the covariates on $Y$, $M$,
and $D$, quantifying the degree of confounding due to observables. The $i$th
element of $\beta$ is set to $0.5/i^{2}$ for $i=1,\ldots,p$, implying a
quadratic decay in the relevance of any additional covariate $i$ for
confounding.

Our study comprises 1000 simulations for each of two distinct sample sizes $n$
consisting of 1000 and 4000 observations respectively, with the number of
covariates $X$ set to 200. To test the null hypothesis (5.2), we estimate the
conditional means involved in the score function (5.3) using lasso regression
and 2-fold cross-fitting. We generate the random variable $\zeta$ from a mean-
zero normal distribution, $\mathcal{N}(0,\sigma_{\zeta}^{2})$, with the
standard deviation $\sigma_{\zeta}$ being inversely proportional to the sample
size, $\sigma_{\zeta}=500/n$. This choice ensures that the influence of
$\zeta$ on the estimated violations becomes asymptotically negligible.
Furthermore, we estimate the total as well as the natural direct and indirect
treatment effects using Double Machine Learning (DML) with cross-fitting,
employing the default options of the medDML() command in the R package
causalweight by Bodory and Huber (2018). This method applies lasso regression
to estimate the outcome, mediator, and treatment equations, and requires that
selection-on-observables holds with respect to the treatment and the mediator
(given $X$ and given $D,X$ respectively), as imposed in expression (2.2) and
Assumption 7. The procedure drops observations with conditional treatment
probabilities, known as propensity scores, close to zero or one (i.e., smaller
than a threshold of 0.01 or 1% or larger than 0.99 or 99%) from the estimation
to prevent an inflation of the propensity score-based weights, which could
lead to increased variance in effect estimation.

We note that setting the coefficients $\delta=0$ and $\gamma=0$ in the
simulations implies the satisfaction of any conditional independence
assumptions imposed on the treatment, the mediator, or the instruments, such
that the testable implications in Theorem 1 all hold. In contrast, when
$\delta\neq 0$, the variable $U_{1}$ becomes an unobserved confounder jointly
affecting the treatment, mediator, and outcome variables. This violates the
selection-on-observables assumptions on the treatment and the mediator.
Furthermore, when $\gamma\neq 0$, the instruments exhibit direct effects on
the outcome, violating instrument validity. Table 1 presents the results of
our simulations under various choices of $\delta$ and $\gamma$. It includes
the test’s rejection rate (rej. rate) at the 5% level of statistical
significance and its average p-value (mean pval). Additionally, we report the
absolute biases (bias) and root mean squared errors (RMSE) of the DML
estimator of the total, direct, and indirect effects to assess how violations
of the testable implications impact effect estimation performance. These
statistics are provided for the direct effects under both treatment and
control (dir.(1), dir.(0)), defined as $E[Y(1,M(1))-Y(0,M(1))]$ and
$E[Y(1,M(0))-Y(0,M(0))]$, as well as for the indirect effects under both
treatment and control (indir.(1), indir.(0)), defined as
$E[Y(1,M(1))-Y(1,M(0))]$ and $E[Y(0,M(1))-Y(0,M(0))]$. However, it is worth
noting that in our simulation design without treatment-mediator interaction
effects, the respective effects are equivalent under treatment and control.

bias RMSE sample size rej. rate mean pval total dir.(1) dir.(0) indir.(1)
indir.(0) total dir.(1) dir.(0) indir.(1) indir.(0) $\delta$ = 0 & $\gamma$ =
0 1000 0.044 0.513 0.005 0.004 0.004 0.001 0.001 0.084 0.079 0.079 0.052 0.051
4000 0.047 0.510 0.001 0.001 0.000 0.001 0.002 0.041 0.037 0.037 0.022 0.023
$\delta=$ 1& $\gamma$ = 0 1000 0.688 0.122 1.318 0.928 0.929 0.389 0.390 1.324
0.937 0.937 0.403 0.404 4000 1.000 0.000 1.376 0.916 0.914 0.461 0.459 1.377
0.918 0.916 0.464 0.462 $\delta$ = 0 & $\gamma$ = 0.2 1000 0.086 0.447 0.143
0.104 0.104 0.039 0.039 0.168 0.132 0.132 0.069 0.069 4000 1.000 0.000 0.117
0.123 0.122 0.005 0.006 0.124 0.129 0.128 0.022 0.023 • Notes: Column ‘rej.
rate’ gives the empirical rejection rate when setting the level of statistical
significance to 0.05. ‘mean pval’ gives the average p-value across
simulations. Columns ‘total’, ‘dir.(1)’, ‘dir.(0)’, ‘indir.(1)’, ‘indir.(0)’
provide the average absolute bias or RMSE across all samples for the total
effect, the direct effects under treatment (1) and control (0), as well as the
indirect effects under treatment (1) and control (0), respectively.

Table 1: Simulations

The top panel of Table 1 shows the results for $\delta=0$ and $\gamma=0$,
ensuring satisfaction of all testable implications. The empirical rejection
rate of the test is close to the nominal rate of 5%, with values of 0.044 (or
4%) for the smaller sample size $n=1000$ and 0.047 (or 4.7%) for $n=4000$.
Correspondingly, the average p-value across the 1000 simulations is relatively
high, slightly exceeding 50%. Regarding effect estimation, we find that the
absolute biases of the total, direct, and indirect effects are generally close
to zero and decrease with increasing sample size. Additionally, the RMSE
decreases as the sample size increases from $n=1000$ to $n=4000$, decaying
roughly at a rate proportional to $\sqrt{n}$. The intermediate panel reports
the results for $\delta=1$ and $\gamma=0$, indicating a violation of the
selection-on-observables assumptions. Accordingly, the rejection rate of our
test increases to 68.8% under the smaller sample size and reaches 100% under
$n=4000$, while the average p-value decreases from 12.2% to 0%, which points
to a very decent power of our test in the scenario considered. As expected,
the DML estimator of the total, direct, and indirect effects performs poorly
under both sample sizes, with absolute biases and RMSEs substantially
different from zero.

In the bottom panel, we consider $\delta=0$ and $\gamma=0.2$, implying a
violation of instrument validity due to moderate direct effects of the
instruments on the outcome. We observe that the test’s statistical power
increases sharply with sample size, from 8.6% under $n=1000$ to 100% under
$n=4000$. Regarding the estimated effects, we note that the absolute biases
are generally not substantial but not negligible either, and they only
diminish slightly as the sample size increases. This phenomenon occurs because
the instruments confound the treatment-outcome and mediator-outcome
associations due to their direct effects on the outcome, which is not
accounted for in the effect estimations that only consider $X$ as covariates
to be controlled for. Overall, the simulation results suggest that the test
exhibits satisfactory size and power to detect violations of the testable
implications, which increase with sample size.

Our second simulation design explores a scenario where the first instrument,
$Z_{1}$, exerts a direct influence on the second instrument, $Z_{2}$. This
structural adjustment is reflected by redefining $Z_{2}$ in model (6.1) as
follows, while the remaining variable definitions are the same as before:

$\displaystyle Z_{2}=U_{4}+0.5Z_{1},\quad U_{4}\sim\mathcal{N}(0,1).$

Table 2 presents the results of 1000 simulations when testing the implications
of Theorem 2, which now partly contain $Z_{2}$ in the conditioning set. In the
scenario where all testable implications are met ($\delta$ = 0 & $\gamma$ =
0), as depicted in the top panel, the empirical size of the test closely
mirrors the nominal size of 5%, while the average p-value is larger than 50%.
Notably, the absolute biases of effect estimates remain close to zero across
both sample sizes of $n=1,000$ and $n=4,000$ and the RMSE follows a decay rate
roughly proportional to $\sqrt{n}$. Transitioning to the intermediate panel
with $\delta=1$ and $\gamma=0$, implying a violation of the selection-on-
observables assumptions, testing power quickly increases in the sample size,
while the effect estimators are substantially biased. In the bottom panel with
$\delta=0$ and $\gamma=0.2$, implying that the instruments directly affect the
outcome, the test promptly gains power in the sample size, while the effect
estimators are moderately biased. These findings are qualitatively similar to
those of the initial simulation design discussed above.

Bias RMSE sample size rej. rate mean pval total dir.(1) dir.(0) indir.(1)
indir.(0) total dir.(1) dir.(0) indir.(1) indir.(0) $\delta$ = 0 & $\gamma$ =
0 1000 0.042 0.514 0.012 0.005 0.005 0.007 0.007 0.085 0.081 0.081 0.051 0.052
4000 0.049 0.510 0.004 0.001 0.000 0.005 0.006 0.040 0.039 0.039 0.022 0.023
$\delta=$ 1& $\gamma$ = 0 1000 0.297 0.286 1.244 0.914 0.917 0.327 0.330 1.250
0.925 0.927 0.344 0.347 4000 1.000 0.000 1.268 0.864 0.864 0.404 0.403 1.269
0.867 0.866 0.407 0.407 $\delta$ = 0 & $\gamma$ = 0.2 1000 0.234 0.318 0.112
0.124 0.123 0.010 0.011 0.140 0.149 0.148 0.056 0.057 4000 1.000 0.000 0.117
0.123 0.122 0.005 0.006 0.124 0.129 0.128 0.022 0.023 • Notes: Column ‘rej.
rate’ gives the empirical rejection rate when setting the level of statistical
significance to 0.05. ‘mean pval’ gives the average p-value across
simulations. Columns ‘total’, ‘dir.(1)’, ‘dir.(0)’, ‘indir.(1)’, ‘indir.(0)’
provide the average absolute bias or RMSE across all samples for the total
effect, the direct effects under treatment (1) and control (0), as well as the
indirect effects under treatment (1) and control (0), respectively.

Table 2: Simulations: Alternative Identifying Assumptions

##  7 Empirical application

This section presents an empirical application of our test to rich
administrative data from Slovakia previously analyzed by Lafférs and Štefánik
(2024) to evaluate a Slovak active labor market intervention known as the
‘Youth Guarantee’. The dataset contains comprehensive information from
administrative records maintained by the Slovak public employment service
centers, which are part of the Central Office for Labour, Social Affairs, and
Family of the Slovak Republic (COLSAF), including the unemployment history,
socio-economic characteristics, participation in labor market programs, and
regional characteristics.

We focus on the dynamic effects of a specific sequential labor market
intervention aimed at unemployed youth for job seekers registering as
unemployed at the beginning of 2016. The program sequence starts with a three
to six-month training called graduate practice (GP), designed to foster early-
career workplace experience, albeit with less substantial monetary
contributions than the subsequent interventions. The time lag between
unemployment registration and the start of GP must be at least one month by
law. After completing GP, participants could advance to a second program that
is typically twelve months long and provides employment incentives (EI) that
combine hiring incentives with subsidized employment, by offsetting up to 75%
of the costs for employing unemployed youth over twelve months, which is then
followed by a compulsory employment term of six months. Svabova and Kramarova
(2021) and Stefanik, Karasova, and Studena (2020) find these interventions to
have moderate but statistically significant positive effects on employment,
but negative effects on earnings. In our analysis, the first intervention,
$D$, is defined as receiving GP within the first 6 months after the
unemployment registration, while the second intervention, $M$, is defined as
receiving employment incentives within months seven to twelve since
unemployment registration. The timeline of the interventions is depicted in
Figure 4.

Figure 4:  Timeline of interventions and variable measurements

Central to our testing approach are two instrumental variables, $Z_{1}$ and
$Z_{2}$, designed to capture the local availability of programs $D$ and $M$,
respectively, at public employment service (PES) centers. The first instrument
$Z_{1}$ is computed based on the ratio of jobseekers enrolled in intervention
$D$ in the previous year (2015) to the total influx of new jobseekers at the
respective PES office during the previous two years (2014 and 2015). An
analogous methodology is applied to compute the second instrument $Z_{2}$
related to intervention $M$. Our approach is in line with a range of studies
that leverage regional variations in the availability of interventions across
employment agencies, particularly to construct instruments for participation
in specific labor market interventions. Examples include Frölich and Lechner
(2010), Lechner, Wunsch, and Scioch (2013), Boockmann, L. Thomsen, and Walter
(2014), Markussen and Røed (2014), Frölich and Lechner (2015), and, in
particular, Caliendo, Künn, and Mahlstedt (2017), Dauth (2020), and Lang
(2022), who use a similar approach as ours by comparing the number of
participants in a program to the overall number of eligible jobseekers. By
applying this approach in the Slovak context, our instruments aim to quantify
the local propensity for receiving a specific labor market intervention. It is
worth noting that the application to a particular intervention needs to be
submitted by the jobseeker themselves, while it is the regional PES management
that makes the final decision of acceptance, as a function of budgetary
restrictions and caseworker recommendations. As such, program enrollment
variability is significantly shaped by regional factors, such as unemployment
rates, proximity to PES offices, and demographic considerations, including the
proportion of the Roma population in the area, which we can observe and
control for.

In addition to regional control variables, our data set contains rich
individual-level information. The pre-treatment covariates ($X$) are measured
prior to the GP intervention ($D$) and consist of 264 variables that include
besides the regional information for instance a jobseeker’s marital status,
the presence of dependents, comprehensive measures of education and skills,
employment histories, prior claims of unemployment benefits, willingness to
relocate for work, health information, and caseworker assessments of
employability prospects. We also include five post-treatment covariates ($W$)
in our analysis, which are measured after (and may be influenced by) $D$ but
prior to the start of the EI intervention ($M$) and might potentially affect
both EI participation and the outcome variable. The variables $W$ include
information on a jobseeker’s participation in other labor market programs than
GP ($D$) during the first intervention period and whether the jobseeker was
absent from the unemployment register in months four to six (e.g., due to
being employed). Finally, our outcome variable ($Y$) is a binary indicator for
employment three years after the initial unemployment registration. The causal
framework considered for testing relies on the assumptions and testable
implications discussed in Section 4, related to the illustration in Figure 3.

Our evaluation sample comprises 12,436 individuals aged 15 to 29 who
registered as unemployed in 2016, remained jobless for a minimum of three
months, and were tracked for up to 48 months after registration. Among them,
2,491 participated solely in GP ($D$), 1,597 solely in EI ($M$), 2,025 in both
programs, and 6,323 in neither. In line with suggestions by Chernozhukov,
Chetverikov, Demirer, Duflo, Hansen, Newey, and Robins (2018), Table 3
displays the median p-value (pval) from 21 runs of the testing procedure
outlined in Section 5, which applies lasso regression and 10-fold cross-
fitting to estimate conditional means as simulated in Section 6. This
procedure tests the implications of Theorem 3 detailed in Section 4. The
median p-value for these runs is 24.19%, the related test statistics 0.000421,
and the standard error 0.000360, such that we cannot reject the null
hypothesis of the joint satisfaction of the testable implications at
conventional significance levels. Furthermore, we note that the instruments
$Z_{1}$ and $Z_{2}$ exhibit highly statistically significant first-stage
associations with $D$ and $M$, respectively. When utilizing the DoubleMLData()
command (with default options) in the DoubleML package by Bach, Kurz,
Chernozhukov, Spindler, and Klaassen (2024) to estimate the first-stage
associations of $D$ and $Z_{1}$ conditional on $X$ and of $M$ and $Z_{2}$
conditional on $(D,X,W)$ based on DML with lasso regression models for the
instruments and interventions, the p-values are below 1%. This suggests the
satisfaction of Assumptions 4 and 5m, which are important for the power of the
test.

teststat se pval effect effect_se effect_pval effect_ntrimmed 0.00042 0.00036
0.24189 0.0855 0.0249 0.0006 6,288 • Notes: Columns ‘teststat’, ‘se’, and
‘pval’ provide the median value of the p-value and their corresponding test
statistic and standard error, across 21 runs using 21 different seeds.
‘effect’, ‘effect_se’, ‘effect_pval’ provide the dynamic treatment effect
estimate, its standard error, and its p-value, respectively. ‘effect_ntrimmed’
gives the number of discarded observations, due to extreme propensity scores.

Table 3: Empirical Application

Due to the insignificant test result, we suspect that the selection-on-
observables assumptions regarding $D$ and $M$ are (close to being) satisfied.
For this reason, Table 3 also presents an estimate of the ATE (effect) of
participating in both GP and EI versus not participating in any program,
$E[Y(1,1)-Y(0,0)]$, along with the standard error of the effect estimate
(effect_se) and its p-value (effect_pval). Estimation is based on the
dyntreatDML command (with default options) of the causalweight package, which
employs cross-fitted DML and lasso regression for the estimation of the models
for $D$, $M$, and $Y$, as discussed in Bodory, Huber, and Lafférs (2022). The
ATE estimate suggests that participating in both programs increases the
employment probability by 8.55 percentage points, and this effect is highly
statistically significant, with a p-value of less than 0.1%.

Table 3 also reports the number of discarded observations due to extreme
propensity scores related to $D$ and/or $M$ (effect_ntrimmed), indicating that
the product of the propensities to receive $D$ and $M$ is either smaller than
1% or larger than 99%. Specifically, 6,280 observations, or 50.5% of the
original sample, were discarded, implying that only roughly half of the
individuals in the sample are sufficiently similar in terms of their
covariates across different interventions to be considered for effect
estimation. It is noteworthy that various propensity score thresholds for
discarding observations, such as smaller than 0.5% and larger than 99.5%, or
smaller than 5% and larger than 95%, yield very similar effect estimates.

A further analysis was conducted to examine the robustness of our test results
to changes in the set of control variables. Specifically, the number of
covariates included in the model was reduced by excluding post-treatment
variables $W$ and substantially reducing $X$ to only include age and four
binary indicators representing the highest level of educational attainment.
This revised specification yielded a median test statistic of 0.00066, with a
corresponding standard error of 0.00036 and a p-value of 6.92%. Therefore, our
test rejects the null hypothesis at the 10% significance level, which is in
line with our impression that the assumption of sequential ignorability does
not appear plausible under such a limited set of control variables.

##  8 Conclusion

This paper introduces a new method for testing the identification of causal
effects in mediation, dynamic treatment, and sample selection models, by
making use of covariates to be controlled for and suspected instruments. It
establishes sufficient testable conditions for identifying such effects in
observational data. These conditions jointly imply the exogeneity of the
treatment and the mediator conditional on observed variables, as well as the
validity of the instruments for the treatment and mediator (or sample
selection indicator). We explore various causal frameworks and testable
implications, contingent upon whether the covariates to be controlled for are
solely pre-treatment or also include post-treatment variables, and depending
on the causal association between the instruments for the treatment and the
mediator. We suggest a machine learning-based testing approach that handles
high-dimensional covariates in a data-adaptive manner and demonstrates a
decent performance in our simulation study. Furthermore, we apply the test to
evaluate a sequence of active labor market programs in Slovakia, incorporating
both pre- and post-treatment control variables, and do not reject the testable
implications for the identification of dynamic treatments when employing the
local availability of labor market programs as instruments.

## References

  * (1)
  * Angrist, Imbens, and Rubin (1996) Angrist, J., G. W. Imbens, and D. Rubin (1996): “Identification of Causal Effects using Instrumental Variables,” _Journal of American Statistical Association_ , 91, 444–472 (with discussion).
  * Angrist and Rokkanen (2015) Angrist, J. D., and M. Rokkanen (2015): “Wanna get away? Regression discontinuity estimation of exam school effects away from the cutoff,” _Journal of the American Statistical Association_ , 110, 1331–1344.
  * Avin, Shpitser, and Pearl (2005) Avin, C., I. Shpitser, and J. Pearl (2005): “Identifiability of path-specific effects,” in _IJCAI-05, Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence_ , pp. 357–363, Edinburgh, UK.
  * Bach, Kurz, Chernozhukov, Spindler, and Klaassen (2024) Bach, P., M. S. Kurz, V. Chernozhukov, M. Spindler, and S. Klaassen (2024): “DoubleML: An Object-Oriented Implementation of Double Machine Learning in R,” _Journal of Statistical Software_ , 108(3), 1–56.
  * Bia, Huber, and Lafférs (2024) Bia, M., M. Huber, and L. Lafférs (2024): “Double machine learning for sample selection models,” _Journal of Business & Economic Statistics_, 42, 958–969.
  * Black, Joo, LaLonde, Smith, and Taylor (2015) Black, D. A., J. Joo, R. J. LaLonde, J. A. Smith, and E. J. Taylor (2015): “Simple Tests for Selection Bias: Learning More from Instrumental Variables,” _IZA Discussion Paper No 9346_.
  * Bodory and Huber (2018) Bodory, H., and M. Huber (2018): “The causalweight package for causal inference in R,” _SES Working Paper 493, University of Fribourg_.
  * Bodory, Huber, and Lafférs (2022) Bodory, H., M. Huber, and L. Lafférs (2022): “Evaluating (weighted) dynamic treatment effects by double machine learning,” _The Econometrics Journal_ , 25(3), 628–648.
  * Boockmann, L. Thomsen, and Walter (2014) Boockmann, B., S. L. Thomsen, and T. Walter (2014): “Intensifying the Use of Benefit Sanctions: An Effective Tool to Increase Employment?,” _IZA Journal of Labor Policy_ , 3(1), 21.
  * Caliendo, Künn, and Mahlstedt (2017) Caliendo, M., S. Künn, and R. Mahlstedt (2017): “The Return to Labor Market Mobility: An Evaluation of Relocation Assistance for the Unemployed,” _Journal of Public Economics_ , 148, 136–151.
  * Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, Newey, and Robins (2018) Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, and J. Robins (2018): “Double/debiased machine learning for treatment and structural parameters,” _The Econometrics Journal_ , 21(1), C1–C68.
  * Cox (1958) Cox, D. (1958): _Planning of Experiments_. Wiley, New York.
  * Dauth (2020) Dauth, C. (2020): “Regional Discontinuities and the Effectiveness of Further Training Subsidies for Low-Skilled Employees,” _ILR Review_ , 73(5), 1147–1184.
  * de Luna and Johansson (2014) de Luna, X., and P. Johansson (2014): “Testing for the unconfoundedness assumption using an instrumental assumption,” _Journal of Causal Inference_ , 2, 187–199.
  * Flores and Flores-Lagunes (2009) Flores, C. A., and A. Flores-Lagunes (2009): “Identification and Estimation of Causal Mechanisms and Net Effects of a Treatment under Unconfoundedness,” _IZA DP No. 4237_.
  * Frölich and Lechner (2010) Frölich, M., and M. Lechner (2010): “Exploiting Regional Treatment Intensity for the Evaluation of Labor Market Policies,” _Journal of the American Statistical Association_ , 105(491), 1014–1029.
  * Frölich and Lechner (2015) Frölich, M., and M. Lechner (2015): “Combining Matching and Nonparametric Instrumental Variable Estimation: Theory and An Application to the Evaluation of Active Labour Market Policies,” _Journal of Applied Econometrics_ , 30, 718–738.
  * Hong (2010) Hong, G. (2010): “Ratio of mediator probability weighting for estimating natural direct and indirect effects,” in _Proceedings of the American Statistical Association_ , pp. 2401–2415, Alexandria, VA. American Statistical Association.
  * Hong and White (1995) Hong, Y., and H. White (1995): “Consistent Specification Testing via Nonparametric Series Regression,” _Econometrica_ , 63(5), 1133–59.
  * Huber (2014) Huber, M. (2014): “Identifying causal mechanisms (primarily) based on inverse probability weighting,” _Journal of Applied Econometrics_ , 29, 920–943.
  * Huber (2023) Huber, M. (2023): _Causal analysis: Impact evaluation and Causal Machine Learning with applications in R_. MIT Press.
  * Huber and Kueck (2022) Huber, M., and J. Kueck (2022): “Testing the identification of causal effects in observational data,” _arXiv preprint arXiv:2203.15890_.
  * Imai, Keele, and Yamamoto (2010) Imai, K., L. Keele, and T. Yamamoto (2010): “Identification, Inference and Sensitivity Analysis for Causal Mediation Effects,” _Statistical Science_ , 25, 51 71.
  * Imai and Yamamoto (2013) Imai, K., and T. Yamamoto (2013): “Identification and Sensitivity Analysis for Multiple Causal Mechanisms: Revisiting Evidence from Framing Experiments,” _Political Analysis_ , 21, 141–171.
  * Imbens (2004) Imbens, G. W. (2004): “Nonparametric estimation of average treatment effects under exogeneity: a review,” _The Review of Economics and Statistics_ , 86, 4–29.
  * Imbens and Angrist (1994) Imbens, G. W., and J. Angrist (1994): “Identification and Estimation of Local Average Treatment Effects,” _Econometrica_ , 62, 467–475.
  * Imbens and Wooldridge (2009) Imbens, G. W., and J. M. Wooldridge (2009): “Recent Developments in the Econometrics of Program Evaluation,” _Journal of Economic Literature_ , 47, 5–86.
  * Kalisch, Mächler, Colombo, Maathuis, and Bühlmann (2012) Kalisch, M., M. Mächler, D. Colombo, M. H. Maathuis, and P. Bühlmann (2012): “Causal Inference Using Graphical Models with the R Package pcalg,” _Journal of Statistical Software_ , 47(11), 1â€“26.
  * Lafférs and Štefánik (2024) Lafférs, L., and M. Štefánik (2024): “Locking-in or Pushing-out: The Caseworker Dilemma,” _Working papers : Pracovné listy, Issue 115_.
  * Lang (2022) Lang, J. (2022): “Employment Effects of Language Training for Unemployed Immigrants,” _Journal of Population Economics_ , 35(2), 719–754.
  * Lechner (2009) Lechner, M. (2009): “Sequential Causal Models for the Evaluation of Labor Market Programs,” _Journal of Business and Economic Statistics_ , 27, 71–83.
  * Lechner and Miquel (2010) Lechner, M., and R. Miquel (2010): “Identification of the effects of dynamic treatments by sequential conditional independence assumptions,” _Empirical Economics_ , 39, 111–137.
  * Lechner, Wunsch, and Scioch (2013) Lechner, M., C. Wunsch, and P. Scioch (2013): “Do Firms Benefit from Active Labour Market Policies?,” _IZA Discussion Papers No. 7614_.
  * Markussen and Røed (2014) Markussen, S., and K. Røed (2014): “The Impacts of Vocational Rehabilitation,” _Labour Economics_ , 31, 1–13.
  * Negi (2020) Negi, A. (2020): “Doubly weighted M-estimation for nonrandom assignment and missing outcomes,” _arXiv preprint arXiv:2011.11485_.
  * Neyman (1923) Neyman, J. (1923): “On the Application of Probability Theory to Agricultural Experiments. Essay on Principles.,” _Statistical Science_ , Reprint, 5, 463–480.
  * Neyman (1959) Neyman, J. (1959): _Optimal asymptotic tests of composite statistical hypotheses_ pp. 416–444. Wiley.
  * Pearl (1988) Pearl, J. (1988): _Probabilistic reasoning in intelligent systems: networks of plausible inference_. Morgan Kaufmann, San Mateo.
  * Pearl (2000) Pearl, J. (2000): _Causality: Models, Reasoning, and Inference_. Cambridge University Press, Cambridge.
  * Pearl (2001) Pearl, J. (2001): “Direct and indirect effects,” in _Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence_ , pp. 411–420, San Francisco. Morgan Kaufman.
  * Peters, Buhlmann, and Meinshausen (2015) Peters, J., P. Buhlmann, and N. Meinshausen (2015): “Causal inference using invariant prediction: identification and confidence intervals,” _arXiv preprint 1501.01332_.
  * Racine (1997) Racine, J. (1997): “Consistent significance testing for nonparametric regression,” _Journal of Business and Economic Statistics_ , 15, 369–379.
  * Racine, Hart, and Li (2006) Racine, J., J. Hart, and Q. Li (2006): “Testing the Significance of Categorical Predictor Variables in Nonparametric Regression Models,” _Econometric Reviews_ , 25, 523–544.
  * Richardson and Robins (2013) Richardson, T. S., and J. M. Robins (2013): “Single world intervention graphs (SWIGs): A unification of the counterfactual and graphical approaches to causality,” _Center for the Statistics and the Social Sciences, University of Washington Series. Working Paper_ , 128(30), 2013.
  * Robins (1986) Robins, J. (1986): “A new approach to causal inference in mortality studies with sustained exposure periods - application to control of the healthy worker survivor effect,” _Mathematical Modelling_ , 7, 1393–1512.
  * Robins (2003) Robins, J. M. (2003): “Semantics of causal DAG models and the identification of direct and indirect effects,” in _In Highly Structured Stochastic Systems_ , ed. by P. Green, N. Hjort, and S. Richardson, pp. 70–81, Oxford. Oxford University Press.
  * Robins and Greenland (1992) Robins, J. M., and S. Greenland (1992): “Identifiability and Exchangeability for Direct and Indirect Effects,” _Epidemiology_ , 3, 143–155.
  * Robins, Hernan, and Brumback (2000) Robins, J. M., M. A. Hernan, and B. Brumback (2000): “Marginal Structural Models and Causal Inference in Epidemiology,” _Epidemiology_ , 11, 550–560.
  * Rubin (1980) Rubin, D. (1980): “Comment on ’Randomization Analysis of Experimental Data: The Fisher Randomization Test’ by D. Basu,” _Journal of American Statistical Association_ , 75, 591–593.
  * Rubin (1974) Rubin, D. B. (1974): “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies,” _Journal of Educational Psychology_ , 66, 688–701.
  * Spirtes, Glymour, Scheines, and Heckerman (2000) Spirtes, P., C. N. Glymour, R. Scheines, and D. Heckerman (2000): _Causation, prediction, and search_. Springer.
  * Stefanik, Karasova, and Studena (2020) Stefanik, M., K. Karasova, and I. Studena (2020): “Can supporting workplace insertions of unemployed recent graduates improve their long-term employability?: Evidence on the treatment effects of the Contribution for the graduate practice in Slovakia,” _Empirica_ , 47(2), 245–265.
  * Svabova and Kramarova (2021) Svabova, L., and K. Kramarova (2021): “An analysis of participation factors and effects of the active labour market measure Graduate practice in Slovakia - Counterfactual approach,” _Evaluation and Program Planning_ , 86, 101917.
  * Tchetgen Tchetgen and Shpitser (2012) Tchetgen Tchetgen, E. J., and I. Shpitser (2012): “Semiparametric theory for causal mediation analysis: Efficiency bounds, multiple robustness, and sensitivity analysis,” _The Annals of Statistics_ , 40, 1816–1845.
  * Textor, van der Zander, Gilthorpe, Liśkiewicz, and Ellison (2016) Textor, J., B. van der Zander, M. S. Gilthorpe, M. Liśkiewicz, and G. T. Ellison (2016): “Robust causal inference using directed acyclic graphs: the R package ‘dagitty’,” _International Journal of Epidemiology_ , 45(6), 1887–1894.
  * Wooldridge (1992) Wooldridge, J. M. (1992): “A test for functional form against nonparametric alternatives,” _Econometric Theory_ , 8(4), 452–475.

##  Appendix A Appendix

###  A.1 Proofs of identification results

We provide a compact overview of all the identifying assumptions, testable
restrictions and results that are in the paper.

Assumptions:

$\displaystyle M(y)$
$\displaystyle=M,D(m,y,z_{2})=D,X(d,m,y,z_{2})=X,Z_{1}(d,m,y,z_{2})=Z_{1},$
$\displaystyle Z_{2}(m,y)$ $\displaystyle=Z_{2},\ \ \forall
d\in\mathcal{D},m\in\mathcal{M},z_{2}\in\mathcal{Z}_{2}\textrm{ and
}y\in\mathcal{Y},$ (Causal faithfulness) + (SEM representation).
$\displaystyle M(y)$
$\displaystyle=M,D(m,y,z_{2})=D,X(d,m,y,z_{2})=X,Z_{1}(d,m,y,z_{2})=Z_{1},$
$\displaystyle Z_{2}(m,y)$ $\displaystyle=Z_{2},W(m)=W,\ \ \forall
d\in\mathcal{D},m\in\mathcal{M},z_{2}\in\mathcal{Z}_{2}\textrm{ and
}y\in\mathcal{Y},$ (Causal faithfulness) + (SEM representation).
$\displaystyle D$ $\displaystyle\not\\!\perp\\!\\!\\!\perp
Z_{1}|X=x\quad\forall x\in\mathcal{X}$ (4) $\displaystyle M$
$\displaystyle\not\\!\perp\\!\\!\\!\perp Z_{2}|D=d,X=x\quad\forall
d\in\mathcal{D},x\in\mathcal{X}$ (5) $\displaystyle M$
$\displaystyle\not\\!\perp\\!\\!\\!\perp Z_{2}|D=d,X=x,W=w\quad\forall
d\in\mathcal{D},x\in\mathcal{X},w\in\mathcal{W}$ (5m) $\displaystyle
Y(d^{\prime},m)$ $\displaystyle{\perp\\!\\!\\!\perp}D|X=x\quad\forall
d^{\prime}\in\mathcal{D},m\in\mathcal{M}\textrm{ and }x\in\mathcal{X}$ (6a)
$\displaystyle M(d)$ $\displaystyle{\perp\\!\\!\\!\perp}D|X=x\quad\forall
d\in\mathcal{D},m\in\mathcal{M}\textrm{ and }x\in\mathcal{X}$ (6b)
$\displaystyle Y(d^{\prime},m)$
$\displaystyle{\perp\\!\\!\\!\perp}D|X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}=z_{2}}\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X}\textrm{ and
}z_{2}\in\mathcal{Z}_{2}$ (6am) $\displaystyle M(d)$
$\displaystyle{\perp\\!\\!\\!\perp}D|X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}=z_{2}}\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{2}\in\mathcal{Z}_{2}$ (6bm)
$\displaystyle Y(d^{\prime},m)$
$\displaystyle{\perp\\!\\!\\!\perp}M|D=d,X=x\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M}\textrm{ and }x\in\mathcal{X}$ (7)
$\displaystyle Y(d^{\prime},m)$
$\displaystyle{\perp\\!\\!\\!\perp}M|D=d,X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}W=w}\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X}\textrm{ and
}w\in\mathcal{W}$ (7m) $\displaystyle Y(d^{\prime},m)$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|X=x\quad\forall
d^{\prime}\in\mathcal{D},m\in\mathcal{M}\textrm{ and }x\in\mathcal{X}$ (8a)
$\displaystyle M(d)$ $\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|X=x\quad\forall
d\in\mathcal{D}\textrm{ and }x\in\mathcal{X}$ (8b) $\displaystyle
Y(d^{\prime},m)$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}=z_{2}}\quad\forall
d^{\prime}\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X}\textrm{ and
}z_{2}\in\mathcal{Z}_{2}$ (8am) $\displaystyle M(d)$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}=z_{2}}\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{2}\in\mathcal{Z}_{2}$ (8bm)
$\displaystyle Y(d^{\prime},m)$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{2}|D=d,X=x\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M}\textrm{ and }x\in\mathcal{X}$ (9)
$\displaystyle Y(d^{\prime},m)$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{2}|D=d,X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}W=w}\quad\forall
d^{\prime},d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X}\textrm{ and
}w\in\mathcal{W}$ (9m)

Testable implications:

$\displaystyle Y$ $\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x\quad\forall
d\in\mathcal{D}\textrm{ and }x\in\mathcal{X}.$ (TIa) $\displaystyle Y$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}=z_{2}}\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{2}\in\mathcal{Z}_{2}.$ (TIam)
$\displaystyle M$ $\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x\quad\forall
d\in\mathcal{D}\textrm{ and }x\in\mathcal{X}.$ (TIb) $\displaystyle M$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|D=d,X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}=z_{2}}\quad\forall
d\in\mathcal{D},x\in\mathcal{X}\textrm{ and }z_{2}\in\mathcal{Z}_{2}.$ (TIbm)
$\displaystyle Y$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{2}|D=d,M=m,X=x\quad\forall
d\in\mathcal{D},m\in\mathcal{M}\textrm{ and }x\in\mathcal{X}$ (TIc)
$\displaystyle Y$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{1}|D=d,M=m,X=x\quad\forall
d\in\mathcal{D},m\in\mathcal{M}\textrm{ and }x\in\mathcal{X}$ (TId)
$\displaystyle Y$
$\displaystyle{\perp\\!\\!\\!\perp}Z_{2}|D=d,M=m,X=x,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}W=w}\quad\forall
d\in\mathcal{D},m\in\mathcal{M},x\in\mathcal{X}\textrm{ and }w\in\mathcal{W}$
(TIe)

Propositions:

Theorem 1 (Main setup)

$\text{Under }(\ref{A1}),(\ref{A4}),(\ref{A5}):\
(\ref{A6a}),(\ref{A6b}),(\ref{A8a}),(\ref{A8b}),(\ref{A7}),(\ref{A9})\iff(\ref{TIa}),(\ref{TIb}),(\ref{TIc})$

Lemma 1 (Alternative setup)

$\text{Under }(\ref{A1}):\
(\ref{TIa}),(\ref{TIbm}),(\ref{TIc})\implies(\ref{TId}),$ $\text{Under
}(\ref{A1}):\ (\ref{TIbm}),(\ref{TIc}),(\ref{TId})\implies(\ref{TIa}),$

Theorem 2 (Alternative setup)

$\text{Under }(\ref{A1}),(\ref{A4}),(\ref{A5}):\
(\ref{A6am}),(\ref{A6bm}),(\ref{A8am}),(\ref{A8bm}),(\ref{A7}),(\ref{A9})\iff(\ref{TIam}),(\ref{TIbm}),(\ref{TIc})\iff(\ref{TIbm}),(\ref{TIc}),(\ref{TId})$

Theorem 3 (Dynamic setup)

$\text{Under }(\ref{A1m}),(\ref{A4}),(\ref{A5m}):\
(\ref{A6a}),(\ref{A6b}),(\ref{A8a}),(\ref{A8b}),(\ref{A7m}),(\ref{A9m})\iff(\ref{TIa}),(\ref{TIb}),(\ref{TIe})$

Proofs (computational approach):

The proofs of all these results can be done analytically following the steps
similar to Huber and Kueck (2022).

These theorems as well as all the steps in the proof can also be verified
computationally. We may consider all the possible DAGs with observed variables
$Y,D,X,Z_{1},Z_{2}$ (and $W$ for the Dynamic setup) and with unobserved
confounders for possibly all the pairs of the observed variables. We do not
need to consider

  * (i)

unobserved colliders, as these paths are closed anyway,

  * (ii)

unobserved mediators, as these can be interpreted as direct paths,

  * (iii)

confounders for more than two observed variables, as these are equivalent to
the existence of multiple pair-wise confounders from the point of view of
existence of open paths and hence identification.

First of all, we translate all the assumptions into the DAG semantics.

Let $G$ denote the original graph and let $G_{D}$ and $G_{DM}$ denote the
interventional graph in all the arrows starting from $D$ or $\\{D,M\\}$ are
removed. This follows from the Single World Intervention Graphs (SWIGs)
framework of Richardson and Robins (2013).

Assumptions (DAG semantics):

There are no directed paths in the following directions: (1) $\displaystyle
Y\rightarrow M,Y\rightarrow X,Y\rightarrow Z_{1},Y\rightarrow
Z_{2},M\rightarrow D,M\rightarrow X,M\rightarrow Z_{1},M\rightarrow Z_{2},$
$\displaystyle Z_{2}\rightarrow D,Z_{2}\rightarrow X,Z_{2}\rightarrow
Z_{1},Z_{2}\rightarrow D,D\rightarrow X,D\rightarrow Z_{1}\textrm{ in graph
}G$ There are no directed paths in the following directions: (1m)
$\displaystyle Y\rightarrow M,Y\rightarrow X,Y\rightarrow Z_{1},Y\rightarrow
Z_{2},M\rightarrow D,M\rightarrow X,M\rightarrow Z_{1},M\rightarrow Z_{2},$
$\displaystyle Z_{2}\rightarrow D,Z_{2}\rightarrow X,Z_{2}\rightarrow
Z_{1},Z_{2}\rightarrow D,D\rightarrow X,D\rightarrow Z_{1},M\rightarrow
W\textrm{ in graph }G$ $\displaystyle D\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-connected with conditioning set }\\{X\\}\textrm{
in graph }G$ (4) $\displaystyle M\textrm{ and }Z_{2}$ $\displaystyle\textrm{
are d-connected with conditioning set }\\{D,X\\}\textrm{ in graph }G$ (5)
$\displaystyle M\textrm{ and }Z_{2}$ $\displaystyle\textrm{ are d-connected
with conditioning set }\\{D,X,W\\}\textrm{ in graph }G$ (5m) $\displaystyle
Y\textrm{ and }D$ $\displaystyle\textrm{ are d-separated with conditioning set
}\\{X\\}\textrm{ in graph }G_{DM}$ (6a) $\displaystyle M\textrm{ and }D$
$\displaystyle\textrm{ are d-separated with conditioning set }\\{X\\}\textrm{
in graph }G_{D}$ (6b) $\displaystyle Y\textrm{ and }D$ $\displaystyle\textrm{
are d-separated with conditioning set
}\\{X,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}}\\}\textrm{
in graph }G_{DM}$ (6am) $\displaystyle M\textrm{ and }D$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}}\\}\textrm{
in graph }G_{D}$ (6bm) $\displaystyle Y\textrm{ and }M$ $\displaystyle\textrm{
are d-separated with conditioning set }\\{D,X\\}\textrm{ in graph }G_{DM}$ (7)
$\displaystyle Y\textrm{ and }M$ $\displaystyle\textrm{ are d-separated with
conditioning set
}\\{D,X,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}W}\\}\textrm{
in graph }G_{DM}$ (7m) $\displaystyle Y\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-separated with conditioning set }\\{X\\}\textrm{
in graph }G_{DM}$ (8a) $\displaystyle M\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-separated with conditioning set }\\{X\\}\textrm{
in graph }G_{D}$ (8b) $\displaystyle Y\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}}\\}\textrm{
in graph }G_{DM}$ (8am) $\displaystyle M\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}}\\}\textrm{
in graph }G_{D}$ (8bm) $\displaystyle Y\textrm{ and }Z_{2}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{D,X\\}\textrm{ in graph }G_{DM}$ (9) $\displaystyle Y\textrm{ and }Z_{2}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{D,X,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}W}\\}\textrm{
in graph }G_{DM}$ (9m)

Testable implications :

$\displaystyle Y\textrm{ and }Z_{1}$ $\displaystyle\textrm{ are d-separated
with conditioning set }\\{X,D\\}\textrm{ in graph }G$ (TIa) $\displaystyle
Y\textrm{ and }Z_{1}$ $\displaystyle\textrm{ are d-separated with conditioning
set
}\\{X,D,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}}\\}\textrm{
in graph }G$ (TIam) $\displaystyle M\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,D\\}\textrm{ in graph }G$ (TIb) $\displaystyle M\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,D,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}Z_{2}}\\}\textrm{
in graph }G$ (TIbm) $\displaystyle Y\textrm{ and }Z_{2}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,D,M\\}\textrm{ in graph }G$ (TIc) $\displaystyle Y\textrm{ and }Z_{1}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,D,M\\}\textrm{ in graph }G$ (TId) $\displaystyle Y\textrm{ and }Z_{2}$
$\displaystyle\textrm{ are d-separated with conditioning set
}\\{X,D,M,{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}W}\\}\textrm{
in graph }G$ (TIe)

We conduct an exhaustive search across all the possible DAGs. We restrict
ourselves only to those for which Assumption 1 holds, which immediately
removes the possibility of many of the arrows. For instance, there cannot be
any direct arrow between $Y\rightarrow M.$ Then, given that all the
assumptions are conditional on $X$, we do not have to consider $X$ explicitly
in the DAGs but keep it implicit. This reduces the dimension of the graphs
significantly and makes them computationally tractable. There are 10 different
arrows between the observed variables to consider that are not ruled out by
the assumption (1). We also consider confounders for all the pairs of observed
variables, hence $\binom{5}{2}=10$ different unobserved confounders.
Altogether we have $2^{10}2^{10}=1048576$ DAGs to search through. We utilized
the R packages daggity by Textor, van der Zander, Gilthorpe, Liśkiewicz, and
Ellison (2016) and pcalg by Kalisch, Mächler, Colombo, Maathuis, and Bühlmann
(2012) to check which of the assumptions
(1),(1m),(4),(5),(6a),(6b),(6am),(6bm),(7),(7m),(8a),(8b),(8am),(8bm),(9),(9m)
and testable implications (TIa),(TIam),(TIb),(TIbm),(TIc),(TId),(TIe) are
valid.

We used pcalg2daggity() to translate an adjacency matrix into a DAG and
dconnected()/dseparated() for the existence/non-existence of a path in a
certain graph. Interventional graphs $G_{D}$ and $G_{DM}$ were obtained by
removing all the arrows pointing from $D$ and from $\\{D,M\\}$, respectively.

This computational approach provides us with a direct and principled way of
proving theorems related to the testable implications regardless of the type
of assumptions imposed on causal structure. As an example, consider Theorem 1.
There are 735232 DAGs that satisfy assumptions (1), (4), (5). Out of these

  * (i)

480 DAGs satisfy (6a), (6b), (8a), (8b), (7), (9) and at the same time,
satisfy (TIa), (TIb), (TIc),

  * (ii)

73043 (=73523-480) DAGs that do not satisfy at least one of these assumptions:
(6a), (6b), (8a), (8b), (7), (9) and at the same time, do not satisfy at least
one of these testable implications: (TIa), (TIb), (TIc).

By the mechanical exhaustive search we establish the equivalence directly as
we have complete information on which specific DAGs satisfy which
assumptions/testable implications.222All the computations take about 10 hours
on a standard desktop computer (mid 2023, M2 Pro processor, 32GB RAM).

We have verified Theorems 1 and 2 computationally. While the space of all the
DAGs in Theorem 3 is too large for an exhaustive search, the proof fortunately
requires only a simple modification of the analytical (rather than
computational) proof of Theorem 1, as provided below.

Proofs (analytical approach):

We subsequently present the proofs of Theorems 1 and 3 based on the analytical
(rather than computational) approach. The proof for the Theorem 2 follows in
analogous manner and is motted for the sake of brevity.

Proof of Theorem 1

“$\implies$ (TIa)”

By contradiction.

Let (TIa)’: there exists an open path $Y-Z_{1}$ in $G$ conditional on
$\\{D,X\\}$.

By (8a) there is no open path $Y-Z_{1}$ in $G_{D,M}$ conditional on $\\{X\\}$.

We may have one of the two situations:

  * (i)

the conditioning on $D$ that opens up a path in $G$ conditional on $\\{X\\}$.

So it has to be a collider (or a descendant of a collider).

Thus we need to have $Y-...\rightarrow D\leftarrow...-Z_{1}$ in $G$ where
$Y-...\rightarrow D$ has to be open in $G$ conditional on $\\{X\\}.$

But $Y-D$ is closed in $G_{DM}$ conditional on $\\{X\\}$ by (6a).

So it is the removing of arrows from $\\{D,M\\}$ that makes $Y$ and $Z_{1}$ no
longer connected. Given that there is an arrow to $D$, so it has to be
removing the arrow from $M$ that makes $Y$ and $Z_{1}$ no longer connected.
Therefore we need to have $Y-...-M-...\rightarrow D$, where both $Y-...-M$ and
$M-...\rightarrow D$ are open in $G$ conditional on $\\{X\\}$. But according
to (6b), $M-D$ is closed in $G_{D}$ conditional on $\\{X\\}.$
$\text{\Lightning}.$

  * (ii)

removing arrows from $\\{D,M\\}$ closes the $Y-Z_{1}$ path what would
otherwise be open in $G_{D}$.

Given (i), we know that $D$ cannot be a collider on the path $Y-Z_{1}$ in $G$,
in any other case conditioning on it would block the path hence (TIa)’ would
not hold. It therefore has to be an arrow from $M$ that is removed that causes
$Y$ and $Z_{1}$ to be no longer connected. So we need to have
$Y-...-M-...\rightarrow Z_{1}$ which is open in $G$ and there is no $D$ on
this path. So $M-Z_{1}$ is open and also $Y-M$ is open. But (8b) says
$M-Z_{1}$ is closed in $G_{D}$ conditional on $\\{X\\}$. $\text{\Lightning}.$

“$\implies$ (TIb)”

By contradiction.

Let (TIb)’: there exists an open path $M-Z_{1}$ in $G$ conditional on
$\\{D,X\\}$.

By (8b) there is no open path $M-Z_{1}$ in $G_{D}$ conditional on $\\{X\\}$.

We may have one of the two situations:

  * (i)

the conditioning on $D$ opens up a path in $G$ conditional on $\\{X\\}$.

So it has to be a collider (or a descendant of a collider).

Thus we need to have $M-...\rightarrow D\leftarrow...-Z_{1}$ in $G$ where
$M-...\rightarrow D$ has to be open in $G$ conditional on $\\{X\\}.$

But $M-D$ is closed in $G_{D}$ conditional on $\\{X\\}$ by (6b).
$\text{\Lightning}.$

  * (ii)

removing arrows from $D$ closes the $M-Z_{1}$ path what would otherwise be
open in $G$.

Given (i), we know that $D$ cannot be a collider on the path $M-Z_{1}$ in $G$.
But given (TIb)’ conditioning on $D$ would block this path.
$\text{\Lightning}.$

“$\implies$ (TIc)”

By contradiction.

Let (TIc)’: there exists an open path $Y-Z_{2}$ in $G$ conditional on
$\\{D,M,X\\}$.

By (9) there is no open path $Y-Z_{2}$ in $G_{D,M}$ conditional on
$\\{D,X\\}$.

We may have one of the two situations:

  * (i)

the conditioning on $M$ opens up a path $Y-Z_{2}$ in $G$ conditional on
$\\{D,X\\}$.

So it has to be a collider (or a descendant of a collider).

Thus we need to have $Y-...\rightarrow M\leftarrow...-Z_{2}$ in $G$ where
$Y-...\rightarrow M$ has to be open in $G$ conditional on $\\{D,X\\}.$

But $Y-M$ is closed in $G_{DM}$ conditional on $\\{D,X\\}$ by (7).
$\text{\Lightning}.$

  * (ii)

removing arrows from $\\{D,M\\}$ closes the $Y-Z_{2}$ path what would
otherwise be open in $G$.

Given (i), we know that $M$ cannot be a collider on the path $Y-Z_{2}$ in $G$.
But given (TIc)’ conditioning on $M$ would block this path. It therefore has
to be an arrow from $D$ that is removed that causes $Y$ and $Z_{2}$ to be no
longer connected. So we need to have $Y-...-D-...\rightarrow Z_{2}$ which is
open in $G$ and there is no $M$ on this path. So $Y-D$ is open and also
$D-Z_{2}$ is open. But (6a) says $Y-D$ is closed in $G_{DM}$ conditional on
$\\{X\\}$. $\text{\Lightning}.$

“(8a) $\impliedby$ ”

By contradiction.

Let (8a)’: there exists an open path $Y-Z_{1}$ in $G_{DM}$ conditional on
$\\{X\\}$.

By (TIa) there is no open path $Y-Z_{1}$ in $G$ conditional on $\\{D,X\\}$ and
hence not even in $G_{DM}$ (removing arrow cannot create a new path).

Given that (8a)’ and (TIa) hold simultaneously, we need to have that
conditioning on $D$ closes otherwise open path $Y-Z_{1}$ in $G_{DM}.$ But
there are no arrows from $D$ in $G_{DM},$ so $D$ can only be a collider on the
path $Y-Z_{1},$ but this would violate (TIa). $\text{\Lightning}.$

“(6a) $\impliedby$ ”

By contradiction.

Let (6a)’: there exists an open path $Y-D$ in $G_{DM}$ conditional on
$\\{X\\}$.

By (TIa) there is no open path $Y-Z_{1}$ in $G$ conditional on $\\{D,X\\}$

By (4) there is an open path $D-Z_{1}$ in $G$ conditional on $\\{X\\}$

Thus we need to have that conditioning on $D$ blocks the $Y-D-Z_{1}$ path.

We have one of the following situations.

  * (i)

$Y-...\rightarrow D\leftarrow...-Z_{1}$: ruled out, as $D$ cannot be a
collider.

  * (ii)

$Y-...\leftarrow D\rightarrow...-Z_{1}$: ruled out, because $Y-...\leftarrow
D$ is open in $G_{DM}$ and there are not arrows from $D$ in $G_{DM}$.

  * (iii)

$Y-...\leftarrow D\leftarrow...-Z_{1}$: ruled out, because $Y-...\leftarrow D$
is open in $G_{DM}$ and there are not arrows from $D$ in $G_{DM}$,

  * (iv)

$Y-...\rightarrow D\rightarrow...-Z_{1}$.

Consider the path $Y-...\rightarrow D$. Because of (1) there is no directed
path from $Y$ to $D$ and hence there has to be a confounder on this path,
because any collider would block the path and violate (6a)’. The only possible
confounder compatible with (1) is an unobserved variable. Therefore we could
only have a path of type $Y\leftarrow U\rightarrow D\rightarrow...-Z_{1}$. But
because of $D\rightarrow...-Z_{1}$ (1) there cannot be a directed path from
$D$ to $Z_{1}$, thus there must exist a collider on this path
$D\rightarrow...-Z_{1}$. The only possible observable collider compatible with
(4) is $X$ but a directed $X\leftarrow Z_{1}$ path would violate
(1).$\text{\Lightning}.$

“(8b) $\impliedby$ ”

By contradiction.

Let (8b)’: there exists an open path $M-Z_{1}$ in $G_{D}$ conditional on
$\\{X\\}$.

By (TIb) there is no open path $M-Z_{1}$ in $G$ conditional on $\\{D,X\\}$ and
hence not even in $G_{D}$ (removing arrow cannot create a new path).

Given that (8b)’ and (TIb) hold simultaneously, we need to have that
conditioning on $D$ closes otherwise open path $M-Z_{1}$ in $G_{D}.$ But there
are no arrows from $D$ in $G_{D},$ so $D$ can only be a collider on the path
$M-Z_{1},$ but this would violate (TIb). $\text{\Lightning}.$

“(6b) $\impliedby$ ”

By contradiction.

Let (6b)’: there exists an open path $M-D$ in $G_{D}$ conditional on
$\\{X\\}$.

By (TIb) there is no open path $M-Z_{1}$ in $G$ conditional on $\\{D,X\\}$

By (4) there is an open path $D-Z_{1}$ in $G$ conditional on $\\{X\\}$

Thus we need to have that conditioning on $D$ blocks the $M-D-Z_{1}$ path.

We have one of the following situations.

  * (i)

$M-...\rightarrow D\leftarrow...-Z_{1}$:ruled out, as $D$ cannot be a
collider.

  * (ii)

$M-...\leftarrow D\rightarrow...-Z_{1}$: ruled out, because $Y-...\leftarrow
D$ is open in $G_{D}$ and there are not arrows from $D$ in $G_{D}$.

  * (iii)

$M-...\leftarrow D\leftarrow...-Z_{1}$: ruled out, because $Y-...\leftarrow D$
is open in $G_{D}$ and there are not arrows from $D$ in $G_{D}$,

  * (iv)

$M-...\rightarrow D\rightarrow...-Z_{1}$.

Consider the path $M-...\rightarrow D$. Because of (1) there is no directed
path from $M$ to $D$ and hence there has to be a confounder on this path,
because any collider would block the path and violate (6b)’. The only possible
confounder compatible with (1) is an unobserved variable. Therefore we could
only have a path of type $M\leftarrow U\rightarrow D\rightarrow...-Z_{1}$. But
because of $D\rightarrow...-Z_{1}$ (1) there cannot be a directed path from
$D$ to $Z_{1}$, thus there must be a collider on this path
$D\rightarrow...-Z_{1}$. The only possible observable collider compatible with
(4) is $X$ but directed $D\rightarrow X$ path would violate
(1).$\text{\Lightning}.$

“(9) $\impliedby$ ”

By contradiction.

Let (9)’: there exists an open path $Y-Z_{2}$ in $G_{DM}$ conditional on
$\\{D,X\\}$.

By (TIc) there is no open path $Y-Z_{2}$ in $G$ conditional on $\\{D,X,M\\}$
and hence not even in $G_{DM}$ (removing arrow cannot create a new path).

Given that (9)’ and (TIc) hold simultaneously, we need to have that
conditioning on $M$ closes otherwise open path $Y-Z_{2}$ in $G_{DM}.$ But
there are no arrows from $M$ in $G_{DM},$ so $M$ can only be a collider on the
path $Y-Z_{2},$ but this would violate (TIc). $\text{\Lightning}.$

“(7) $\impliedby$ ”

By contradiction.

Let (7)’: there exists an open path $Y-M$ in $G_{DM}$ conditional on
$\\{D,X\\}$.

By (TIc) there is no open path $Y-Z_{2}$ in $G$ conditional on $\\{D,X,M\\}$

By (5) there is an open path $M-Z_{2}$ in $G$ conditional on $\\{D,X\\}$

Thus we need to have that conditioning on $M$ blocks the $Y-M-Z_{2}$ path.

We have one of the following situations.

  * (i)

$Y-...\rightarrow M\leftarrow...-Z_{2}$: ruled out, as $M$ cannot be a
collider.

  * (ii)

$Y-...\leftarrow M\rightarrow...-Z_{2}$: ruled out, because $Y-...\leftarrow
M$ is open in $G_{DM}$ and there are no arrows from $M$ in $G_{DM}$.

  * (iii)

$Y-...\leftarrow M\leftarrow...-Z_{2}$: ruled out, because $Y-...\leftarrow M$
is open in $G_{DM}$ and there are no arrows from $M$ in $G_{DM}$,

  * (iv)

$Y-...\rightarrow M\rightarrow...-Z_{2}$.

Consider the path $Y-...\rightarrow M$. Because of (1) there is no directed
path from $Y$ to $M$ and hence there needs to be a confounder on this path,
because any collider would block the path and violate (6b)’. The only possible
confounder compatible with (1) is an unobserved variable. Therefore we could
only have a path of type $Y\leftarrow U\rightarrow M\rightarrow...-Z_{2}$. But
because of (1) there cannot be a directed path from $M$ to $Z_{2}$, thus we
need to have a collider on this path $M\rightarrow...-Z_{2}$. The only
possible observable collider compatible with (5) is $X$ or $D$ but directed
$X\leftarrow Z_{2}$ or $D\leftarrow Z_{2}$ path would violate
(1).$\text{\Lightning}.$

This completes the proof. $\blacksquare$

Proof of Lemma 1 (a)

Let (TId)’, then the path $Y-Z_{1}$ is open given $\\{D,X,M\\}$ in $G.$

By (TIam), there is no open path $Y-Z_{1}$ given $\\{D,X,Z_{2}\\}$ in $G.$

We may have one of the two situations:

  * (i)

Path $Y-Z_{1}$ is open given $\\{D,X\\}$ in $G$, then (i) $M$ does not close
it and (ii) $Z_{2}$ closes it So $Y-Z_{2}-Z_{1}$ has to be closed by $Z_{2}$,
but open $Y-Z_{2}$ path contradicts (TIc) $\text{\Lightning}.$

  * (ii)

Path $Y-Z_{1}$ is closed given $\\{D,X\\}$ in $G$ then (i) $M$ is a collider
and (ii) $Z_{2}$ is not a collider we have to have $Y-...\rightarrow
M\leftarrow...-Z_{1}$ where $M-Z_{1}$ is open a this contradicts (TIb)
$\text{\Lightning}.$

Proof of Lemma 1 (b)

Let (TIam)’, then the path $Y-Z_{1}$ is open given $\\{D,X,Z_{2}\\}$ in $G.$

By (TId), there is no open path $Y-Z_{1}$ given $\\{D,X,M\\}$ in $G.$

We may have one of the two situations:

  * (i)

Path $Y-Z_{1}$ is open given $\\{D,X\\}$ in $G$, then (i) $Z_{2}$ does not
close it and (ii) $M$ closes it. So $Y-M-Z_{1}$ has to be closed by $M$, but
the open $M-Z_{1}$ path contradicts (TIb) $\text{\Lightning}.$

  * (ii)

Path $Y-Z_{1}$ is closed given $\\{D,X\\}$ in $G$, then (i) $Z_{2}$ is a
collider and (ii) $M$ is not a collider; it must hold that $Y-\rightarrow
Z_{2}\leftarrow...-Z_{1}$, where $Y-Z_{2}$ is open a this contradicts (TIc)
$\text{\Lightning}.$

Proof of Theorem 3

Several results in the proof of Theorem 1 are unaffected by the inclusion of
post-treatment covariates $W,$ and thus remain unchanged. This applies to the
following assertions: “$\implies$ (TIa)”, “$\implies$ (TIb)”, “(8a)
$\impliedby$ ”, “(6a) $\impliedby$ ”, “(8b) $\impliedby$ ”, “(6b) $\impliedby$
”.

The implications that remain to be proven are: “$\implies$ (TIe)”, “(7m)
$\impliedby$ ”, “(9m) $\impliedby$ ”. These results can be shown by
modifications of the previous analytical proof, as outlined below.

“$\implies$ (TIe)”

By contradiction.

Let (TIe)’: there exists an open path $Y-Z_{2}$ in $G$ conditional on
$\\{D,M,X,W\\}$.

By (9m) there is no open path $Y-Z_{2}$ in $G_{DM}$ conditional on
$\\{D,X,W\\}$.

We may have one of the two situations:

  * (i)

The conditioning on $M$ opens up a path $Y-Z_{2}$ in $G$ conditional on
$\\{D,X,W\\}$, such that it is a collider (or a descendant of a collider).

Thus it must hold that $Y-...\rightarrow M\leftarrow...-Z_{2}$ in $G$, where
$Y-...\rightarrow M$ has to be open in $G$ conditional on $\\{D,X,W\\}.$

But $Y-M$ is closed in $G_{DM}$ conditional on $\\{D,X,W\\}$ by (7m).
$\text{\Lightning}.$

  * (ii)

Removing arrows from $\\{D,M\\}$ closes the $Y-Z_{2}$ path what would
otherwise be open in $G$.

Given (i), $M$ cannot be a collider on the path $Y-Z_{2}$ in $G$. Furthermore,
given (TIe)’, conditioning on $M$ would block any path $Y-Z_{2}$ operating via
$M$. For this reason, there has to be an arrow from $D$ that is removed such
that $Y$ and $Z_{2}$ are no longer connected. So we need to have
$Y-...-D-...\rightarrow Z_{2}$ which is open in $G$ and there is no $M$ on
this path. So $Y-D$ is open and also $D-Z_{2}$ is open. But (6a) says $Y-D$ is
closed in $G_{DM}$ conditional on $\\{X\\}$. $\text{\Lightning}.$

“(9m) $\impliedby$ ”

By contradiction.

Let (9m)’: there exists an open path $Y-Z_{2}$ in $G_{DM}$ conditional on
$\\{D,X,W\\}$.

By (TIe) there is no open path $Y-Z_{2}$ in $G$ conditional on $\\{D,X,M,X\\}$
and hence not even in $G_{DM}$ (removing arrow cannot create a new path).

Given that (9m)’ and (TIe) hold simultaneously, it must hold that conditioning
on $M$ closes the otherwise open path $Y-Z_{2}$ in $G_{DM}.$ But there are no
arrows from $M$ in $G_{DM},$ so $M$ can only be a collider on the path
$Y-Z_{2},$ which would violate (TIe). $\text{\Lightning}.$

“(7m) $\impliedby$ ”

By contradiction.

Let (7m)’: there exists an open path $Y-M$ in $G_{DM}$ conditional on
$\\{D,X,W\\}$.

By (TIe) there is no open path $Y-Z_{2}$ in $G$ conditional on $\\{D,X,M,W\\}$

By (5m) there is an open path $M-Z_{2}$ in $G$ conditional on $\\{D,X,W\\}$

Thus, it must hold that conditioning on $M$ blocks the $Y-M-Z_{2}$ path.

We have one of the following situations.

  * (i)

$Y-...\rightarrow M\leftarrow...-Z_{2}$: ruled out, as $M$ cannot be a
collider.

  * (ii)

$Y-...\leftarrow M\rightarrow...-Z_{2}$: ruled out, because $Y-...\leftarrow
M$ is open in $G_{DM}$ and there are no arrows from $M$ in $G_{DM}$.

  * (iii)

$Y-...\leftarrow M\leftarrow...-Z_{2}$: ruled out, because $Y-...\leftarrow M$
is open in $G_{DM}$ and there are no arrows from $M$ in $G_{DM}$,

  * (iv)

$Y-...\rightarrow M\rightarrow...-Z_{2}$.

Consider the path $Y-...\rightarrow M$. Because of (1m), there is no directed
path from $Y$ to $M$ and hence there has to be a confounder on this path,
because any collider would block the path and violate (6b)’. The only possible
confounder compatible with (1m) is an unobserved variable. Therefore, the only
possible path would be one of the type $Y\leftarrow U\rightarrow
M\rightarrow...-Z_{2}$. But because of (1m), any directed path from $M$ to
$Z_{2}$ is ruled out, which necessarily implies a collider on the path
$M\rightarrow...-Z_{2}$. The only possible observable colliders compatible
with (5m) would be $X$, $D$, or $W$. However, directed paths $X\leftarrow
Z_{2}$, $D\leftarrow Z_{2}$, or $M\rightarrow W$ would violate (1m).
$\text{\Lightning}.$

This completes the proof. $\blacksquare$

###  A.2 Proof of Neyman orthogonality and asymptotic normality

To prove the Neyman orthogonality of score function (5.3) in Section 5 as well
as the asymptotic normality of the testing approach under the null hypothesis
5.2, we closely follow Huber and Kueck (2022) and apply Theorem 3.1 in
Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, Newey, and Robins (2018).
All bounds in the proof hold uniformly over $P\in\mathcal{P}$, but we omit
this qualifier for the sake brevity. We use $C$ to denote a strictly positive
constant that is independent of $n$ and $P\in\mathcal{P}$. The value of $C$
may change at each appearance.

First, we note that the score in (5.3) is linear

$\displaystyle\phi_{2}(V,\theta,\eta)=\phi_{2}^{a}(V,\eta)\theta+\phi_{2}^{b}(V,\eta)$

with $\phi_{2}^{a}(V,\eta)=-1$ and
$\phi_{2}^{b}(V,\eta)=(\eta_{1}(V)-\eta_{2}(V))^{2}+\zeta$. With a slight
abuse of notation, we henceforth denote by $\theta_{0}$ and
$\eta_{0}=(\eta_{1,0},\eta_{2,0})$ the true values of expression 5.1 and the
conditional means including and excluding the instruments, respectively,
defined in Section 5, while $\theta$ and $\eta=(\eta_{1},\eta_{2})$ denote
possibly distinct values of those parameters. We note that the score satisfies
by the construction the following moment condition for the true values
$\theta_{0}$ and $\eta_{0}$,

$\displaystyle
E[\phi_{2}(V,\theta_{0},\eta_{0})]=E[(\eta_{1,0}(V)-\eta_{2,0}(V))^{2}-\theta_{0}+\zeta]=0.$

Furthermore, under the null hypothesis $H_{0}$, it holds

$\displaystyle\partial_{r}E[\phi_{2}(V,\theta_{0},\eta_{0}+r(\eta-\eta_{0}))]\big{|}_{r=0}$
$\displaystyle=E\left[\partial_{r}\phi_{2}(V,\theta_{0},\eta_{0}+r(\eta-\eta_{0}))\big{|}_{r=0}\right]$
$\displaystyle=E\left[\partial_{r}\left(\eta_{1,0}(V)+r(\eta_{1}-\eta_{1,0})(V)-(\eta_{2,0}(V)+r(\eta_{2}-\eta_{2,0})(V))\right)^{2}\big{|}_{r=0}\right]$
$\displaystyle=E\left[2\underbrace{(\eta_{1,0}(V)-\eta_{2,0}(V))}_{=0\text{\
a.s.}}((\eta_{1}-\eta_{1,0})(V)-(\eta_{2}-\eta_{2,0})(V))\right]=0.$

Therefore, the score function satisfies Neyman orthogonality.

Furthermore, the identification condition $J_{0}:=E[\phi_{2}^{a}(W,\eta)]=-1$
holds, such that Assumption 3.1 in Chernozhukov, Chetverikov, Demirer, Duflo,
Hansen, Newey, and Robins (2018) is satisfied. Next, we demonstrate the
satisfaction of Assumption 3.2 in Chernozhukov, Chetverikov, Demirer, Duflo,
Hansen, Newey, and Robins (2018) to complete the proof of asymptotic
normality. We define the realization set $\mathcal{T}_{n}$ of the conditional
means as the set of all P-square-integrable functions $\eta$ such that

$\displaystyle\|\eta_{0}-\eta\|_{P,2q}$ $\displaystyle\leq C,$
$\displaystyle\|\eta_{0}-\eta\|_{P,4}$ $\displaystyle\leq\delta_{n},$
$\displaystyle\|\eta_{0}-\eta\|_{P,2}$
$\displaystyle\leq\delta_{n}^{1/2}n^{-1/4},$

for $\delta_{n}=o(1)$ and a constant $q>2$. Furthermore, we impose the
following regularity conditions for all $n\geq 3$, $P\in\mathcal{P}$ and
$q>2$: Given a random subset $I$ of $[n]$ of size $n/K$ with $I^{c}$ denoting
its complement, the conditional mean estimator
$\hat{\eta}=\hat{\eta}((W_{i})_{i\in I^{c}})$ satisfies
$\|\hat{\eta}-\eta_{0}\|_{P,2q}\leq C$,
$\|\hat{\eta}-\eta_{0}\|_{P,4}\leq\delta_{n}$, and
$\|\hat{\eta}-\eta_{0}\|_{P,2}\leq\delta_{n}^{1/2}n^{-1/4}$ with
$P$-probability not less than $1-o(1)$.

Assumption 3.2(a) of Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, Newey,
and Robins (2018) holds by construction of the set $\mathcal{T}_{n}$ and our
regularity conditions. We proceed by verifying Assumption 3.2(b). For $q>2$,
we have

$\displaystyle\sup_{\eta\in\mathcal{T}_{n}}E[|\phi_{2}(V,\theta_{0},\eta)|^{q}]^{1/q}$
$\displaystyle=\sup_{\eta\in\mathcal{T}_{n}}E\left[\left((\eta_{1}(V)-\eta_{2}(V))^{2}-\theta_{0}+\zeta\right)^{q}\right]^{1/q}$
$\displaystyle\leq\sup_{\eta\in\mathcal{T}_{n}}\|(\eta_{1}(V)-\eta_{2}(V))^{2}\|_{P,q}+|\theta_{0}|+\|\zeta\|_{P,q}$
$\displaystyle\lesssim\sup_{\eta\in\mathcal{T}_{n}}\|(\eta_{1}(V)-\eta_{2}(V))^{2}\|_{P,q}+C$

due to $\|\zeta\|_{P,q}<C$ with

$\displaystyle\quad\sup_{\eta\in\mathcal{T}_{n}}\|(\eta_{1}(V)-\eta_{2}(V))^{2}\|_{P,q}$
$\displaystyle=\sup_{\eta\in\mathcal{T}_{n}}\|\eta_{1}(V)-\eta_{2}(V)\|_{P,2q}^{2}$
$\displaystyle\leq\sup_{\eta\in\mathcal{T}_{n}}\|(\eta_{1}-\eta_{1,0})(V)+(\eta_{1,0})(V)-\eta_{2,0})(V))+(\eta_{2,0}-\eta_{2})(V)\|_{P,2q}^{2}$
$\displaystyle\leq\sup_{\eta\in\mathcal{T}_{n}}\left(\|(\eta_{1}-\eta_{1,0})(V)\|_{P,2q}+\|(\eta_{2,0}-\eta_{2})(V)\|_{P,2q}\right)^{2}$
$\displaystyle\leq C$

by construction of set $\mathcal{T}_{n}$. Furthermore, we have

$\displaystyle\sup_{\eta\in\mathcal{T}_{n}}E[|\phi_{2}^{a}(V,\theta_{0},\eta)|^{q}]^{1/q}$
$\displaystyle=1.$

Next, we verify Assumption 3.2(c). It holds that

$\displaystyle\sup_{\eta\in\mathcal{T}_{n}}|E[\phi_{2}^{a}(V,\theta_{0},\eta)-\phi_{2}^{a}(V,\theta_{0},\eta_{0})]|=|(-1)-(-1)|=0.$

In addition, we have

$\displaystyle\quad\sup_{\eta\in\mathcal{T}_{n}}E[|\phi_{2}(V,\theta_{0},\eta)-\phi_{2}(V,\theta_{0},\eta_{0})|^{2}]^{1/2}$
$\displaystyle=\sup_{\eta\in\mathcal{T}_{n}}E\left[\left((\eta_{1}(V)-\eta_{2}(V))^{2}-(\eta_{1,0}(V)-\eta_{2,0}(V))^{2}\right)^{2}\right]^{1/2}$
$\displaystyle=\sup_{\eta\in\mathcal{T}_{n}}E\bigg{[}\left((\eta_{1,0}(V)-\eta_{2,0}(V))+(\eta_{1}(V)-\eta_{2}(V))\right)^{2}$
$\displaystyle\quad\left((\eta_{1,0}(V)-\eta_{2,0}(V))-(\eta_{1}(V)-\eta_{2}(V))\right)^{2}\bigg{]}^{1/2}$
$\displaystyle\leq\sup_{\eta\in\mathcal{T}_{n}}E\left[((\eta_{1,0}(V)-\eta_{2,0}(V))+(\eta_{1}(V)-\eta_{2}(V)))^{4}\right]^{1/4}$
$\displaystyle\quad
E\left[((\eta_{1,0}(V)-\eta_{2,0}(V))-(\eta_{1}(V)-\eta_{2}(V)))^{4}\right]^{1/4}$
$\displaystyle\leq
C\sup_{\eta\in\mathcal{T}_{n}}(E\left[((\eta_{1,0}(V)-\eta_{1}(V))+(\eta_{2}(V)-\eta_{2,0}(V)))^{4}\right]^{1/4}$
$\displaystyle\leq
2C\sup_{\eta\in\mathcal{T}_{n}}\|\eta_{0}-\eta\|_{P,4}\lesssim\delta_{n}.$

Moreover, for all $r\in(0,1)$, it holds

$\displaystyle\quad\sup_{\eta\in\mathcal{T}_{n}}\partial_{r}^{2}E[\phi_{2}(V,\theta_{0},\eta_{0}+r(\eta-\eta_{0})]$
$\displaystyle=\sup_{\eta\in\mathcal{T}_{n}}E\left[\partial_{r}^{2}\phi_{2}(V,\theta_{0},\eta_{0}+r(\eta-\eta_{0}))\right]$
$\displaystyle=\sup_{\eta\in\mathcal{T}_{n}}E\left[\partial_{r}^{2}(\eta_{1,0}(V)+r(\eta_{1}(V)-\eta_{1,0}(V))-(\eta_{2,0}(V)+r(\eta_{2}(V)-\eta_{2,0}(V))))^{2}\right]$