# Instrumental Variable Estimation of Distributional Causal Effects

Lucas Kook1 and Niklas Pfister2 E-mail<EMAIL_ADDRESS>LK
carried out part of this work at the Department of Mathematical Sciences,
University of Copenhagen, Denmark.

(1Institute for Statistics and Mathematics, Vienna University of Economics and
Business, Austria
2Department of Mathematical Sciences, University of Copenhagen, Denmark )

###### Abstract

Estimating the causal effect of a treatment on the entire response
distribution is an important yet challenging task. For instance, one might be
interested in how a pension plan affects not only the average savings among
all individuals but also how it affects the entire savings distribution. While
sufficiently large randomized studies can be used to estimate such
distributional causal effects, they are often either not feasible in practice
or involve non-compliance. A well-established class of methods for estimating
average causal effects from either observational studies with unmeasured
confounding or randomized studies with non-compliance are instrumental
variable (IV) methods. In this work, we develop an IV-based approach for
identifying and estimating distributional causal effects. We introduce a
distributional IV model with corresponding assumptions, which leads to a novel
identification result for the interventional cumulative distribution function
(CDF) under a binary treatment. We then use this identification to construct a
nonparametric estimator, called DIVE, for estimating the interventional CDFs
under both treatments. We empirically assess the performance of DIVE in a
simulation experiment and illustrate the usefulness of distributional causal
effects on two real-data applications.

##  1 Introduction

Unlike average causal effects, which only capture how a given treatment
affects the response on average, distributional causal effects capture the
effect on the entire response distribution, including the mean, variance and
higher moments of the response (Kneib et al., 2023). Estimands beyond the
conditional mean are of interest in many applications, such as electricity
price forecasting (Marcjasz et al., 2023), poverty research (Hohberg et al.,
2021), epidemiology (Lohse et al., 2017), and the analysis of survival times
(Cox, 1972). Estimating distributional causal effects is feasible when data
from randomized experiments are available, for instance, using quantile
(Koenker et al., 2017) or distributional regression (Kneib, 2013). However, in
the presence of unobserved confounding, the task of estimating distributional
causal effects is considerably more challenging and the available methods are
limited.

Observational data, despite the presence of hidden confounding, may, under
appropriate assumptions, be used to draw causal conclusions (Pearl, 2009;
Wooldridge, 2015). Often, the estimand is the average causal effect, i.e., the
difference in the average response between treated and untreated experimental
units. One way to estimate average causal effects from observational data is
via instrumental variables (IVs), which induce heterogeneity in the treatment
assignment that is independent from both the hidden confounders and only
affects the response via the treatment (Haavelmo, 1943; Angrist et al., 1996;
Imbens and Rubin, 1997). IV approaches for estimating average causal effects
have been proposed for non-linear additive noise models (Newey, 1990),
censored and truncated outcomes (Newey, 2001), the nonparametric additive
noise model (Newey and Powell, 2003; Newey, 2013), semi-parametric cases
(Hansen et al., 2010), and high-dimensional data and sparse causal effects
(Pfister and Peters, 2022). For the case of non-linear additive noise models,
several kernel- (Singh et al., 2019; Muandet et al., 2020; Zhang et al., 2020)
and neural network-based (Bennett et al., 2019) estimators have been
developed. Most existing IV-based methods make use of conditional moment
restrictions, however, recent work (e.g., Poirier, 2017; Dunker, 2021;
Saengkyongam et al., 2022), has proposed to use a stronger identifiablity
condition based on independence between the instruments and the residuals. Our
work expands on this line of work and also proposes an independence based
identification strategy, which is sufficiently strong to ensure
identifiability even of distributional causal effects. The work closest to
ours is the instrumental variable quantile regression (IVQR) framework due to
Chernozhukov and Hansen (2005), which, similarly to us, considers a binary
treatment and absolutely continuous response. For continuous treatments,
Imbens and Newey (2009) propose a control function approach and, for arbitrary
treatments but binary instruments, Chernozhukov et al. (2024) propose an
approach based on distributional regression via copulas.

In this work, we present a novel identification result for distributional
causal effects in settings with an absolutely continuous response $Y$ under a
binary treatment $D$. We introduce a IV model based on structural causal
models (SCMs, Pearl, 2009), which we term IV-SCM and provide assumptions under
which the distributional causal effect is identifiable. The resulting
identification result (Theorem 1) is based on the fact that the interventional
probability integral transform (iPIT) residuals $F^{*}_{D}(Y)$ (i.e., the
interventional CDFs under the observed treatment and evaluated at the observed
response) are uniformly distributed and independent of the instrument. Based
on this result, we propose the distributional instrumental variable estimator
(DIVE) which trades off uniformity of the iPIT residuals (via the Cramér–von
Mises criterion Cram’er, 1928) with its independence from the instrument (via
the Hilbert-Schmidt independence criterion, Gretton et al., 2007). DIVE is
proven to be consistent, which is empirically validated in a simulation study.
In two real-data applications, DIVE leads to similar conclusions as IVQR
(based on linear quantile regression) but is more stable and avoids quantile
crossing. Finally, we also theoretically compare the proposed IV-SCM with the
IVQR model. Importantly, if the quantile causal effect is identifiable in an
IVQR, then there exists a corresponding IV-SCM for which our identification
result holds (Proposition 2).

The remainder of the paper is structured as follows. Section 2 introduces IV-
SCMs and defines distributional causal effects. Section 3.2 then turns to the
assumptions required for identifying distributional causal effects and
contains our identification result. Section 3.3 gives a theoretical comparison
between IV-SCM and the IVQR model. Section 4 formally introduces DIVE and
provides the explicit algorithm for hot to implement it Section 5 contains the
simulation study and two real-data applications. Finally, in Section 6, we
provide a discussion and outlook.

##  2 Distributional causal effects

We consider distributional causal effects as contrasts between the
interventional CDF of treated and untreated populations (assuming a binary
treatment). Throughout this paper, we consider an absolutely continuous
response $Y\in\mathcal{Y}\subseteq\mathbb{R}$ together with a binary treatment
indicator $D\in\\{0,1\\}$ in the presence of hidden confounders
$H\in\mathcal{H}$. Further, we assume access to instrumental variables
$Z\in\mathcal{Z}$. We use SCMs, to define the interventional CDF, but our
results can also be phrased using other causal models, such as potential
outcome models (see Section 3.3).

######  Setting 1 (IV-SCM).

Let $\mathcal{Z}\subseteq\mathbb{R}^{d_{Z}}$,
$\mathcal{H}\subseteq\mathbb{R}^{d_{H}}$, $\mathcal{Y}\subseteq\mathbb{R}$ be
measurable sets and let
$(Z,H,D,Y)\in\mathcal{Z}\times\mathcal{H}\times\\{0,1\\}\times\mathcal{Y}$ be
random variables satisfying the following SCM:

$\displaystyle\begin{cases}Z\coloneqq N_{Z}\\\ H\coloneqq N_{H}\\\ D\coloneqq
f(Z,H,N_{D})\\\ Y\coloneqq g_{D}(H,N_{Y})\end{cases}$

$Z$$D$$H$$Y$

where $N_{Z},N_{H},N_{D},N_{Y}$ are noise variables satisfying
$(N_{Z},N_{D})\perp\nolinebreak\perp N_{Y}\mid N_{H}$ and
$N_{Z}\perp\nolinebreak\perp N_{H}$.111For simplicity one can also assume that
$N_{Z},N_{H},N_{D},N_{Y}$ are jointly independent. This is, however, slightly
more restrictive than necessary here. We denote by
$\mathcal{D}_{n}=\\{(Z_{1},D_{1},Y_{1}),\ldots,(Z_{n},D_{n},Y_{n})\\}$ a
sample of $n$ independent copies of $(Z,D,Y)$ from the above SCM.

We only use the IV-SCMs to model the observed distribution and the two do-
interventions $\operatorname{do}(D=0)$ and $\operatorname{do}(D=1)$.
Importantly, we do not use any of the other induced interventional and
counterfactual distributions and in fact do not even require them to be
correctly specified.

######  Definition 1 (Interventional quantities).

Under Setting 1, for all $d\in\\{0,1\\}$, we define the _interventional
cumulative distribution function (interventional CDF) under treatment $d$_
$F^{*}_{d}:\mathbb{R}\to[0,1]$, for all $y\in\mathbb{R}$ by

$F^{*}_{d}(y)\coloneqq\mathbb{E}[\mathds{1}(Y\leq
y)\mid\operatorname{do}(D=d)].$

We refer to the random variable $F_{D}^{*}(Y)$ as the (population)
_interventional probability integral transform (iPIT) residual_ (after the
observational PIT, see Rosenblatt, 1952). We further define for all
$d\in\\{0,1\\}$ the _interventional quantile function under treatment $d$,_
$Q_{d}^{*}:(0,1)\to\mathbb{R}$ for all $\tau\in(0,1)$ by
$Q_{d}^{*}(\tau)\coloneqq(F_{d}^{*})^{-1}(\tau)$.

###  2.1 Distributional causal effects on different scales

Average causal effects can only describe effects in the population average and
are unable to capture whether specific parts of the distribution of $Y$ are
affected differently. To capture such effects, one can use distributional
causal effects, which can be expressed on various different scales, with the
difference in CDFs being an intuitive example that can be interpreted as a
risk difference varying with the response (see Figure 1A and the first panel
of B). However, other intepretational scales, often implied by parametric
modelling assumptions (such as differences on the log-odds scale in logistic
regression), may be more familiar to practitioners. We define several
transformed distributional causal effect that capture, on different scales,
how the response distributions differ across the interventions
$\operatorname{do}(D=0)$ and $\operatorname{do}(D=1)$.

######  Definition 2 (Transformed distributional causal effects).

Assume Setting 1. We define the following transformed distributional causal
effects:

  * •

The _distributional causal effect_
$\operatorname{DCE}:\mathcal{Y}\rightarrow\mathbb{R}$ for all
$y\in\mathcal{Y}$ by

$\displaystyle\operatorname{DCE}(y)\coloneqq F^{*}_{1}(y)-F^{*}_{0}(y).$ (1)

  * •

The _quantile causal effect_ $\operatorname{QCE}:(0,1)\rightarrow\mathcal{Y}$
for all $\tau\in(0,1)$ by

$\displaystyle\operatorname{QCE}(\tau)\coloneqq
Q_{1}^{*}(\tau)-Q_{0}^{*}(\tau).$ (2)

  * •

The _Doksum causal effect_
$\operatorname{DOK}:\mathcal{Y}\rightarrow\mathbb{R}$ for all
$y\in\mathcal{Y}$ by

$\displaystyle\operatorname{DOK}(y)\coloneqq Q_{0}^{*}(F_{1}^{*}(y))-y.$ (3)

  * •

The _logit distributional causal effect_
$\operatorname{LogitCE}:\mathcal{Y}\rightarrow\mathbb{R}$ for all
$y\in\mathcal{Y}$ by

$\displaystyle\operatorname{LogitCE}(y)\coloneqq\log\left(\frac{F_{1}^{*}(y)}{1-F_{1}^{*}(y)}\right)-\log\left(\frac{F_{0}^{*}(y)}{1-F_{0}^{*}(y)}\right).$
(4)

The Doksum causal effect Doksum (1974) corresponds to taking vertical
differences between the interventional CDF for treated and untreated
observations as a function of the response. Assuming rank invariance (see
Section 3.1), DOK can be interpreted as the difference in potential outcomes
under treatment and control. Depending on the application at hand other
versions of transformed distributional effects could also be considered.

Our proposed estimator introduced in Section 4 is constructed to estimate
distributional causal effects, but it also directly entails plug-in estimators
for any of the other transformed distributional causal effects. Figure 1
illustrates the different interpretational scales for distributional causal
effects on simulated data. In this example, the typically made structural
assumption of proportional odds (see, e.g., Lohse et al., 2017) is not tenable
and a nonparametric approach is needed to capture the correct distributional
causal effect.

Figure 1: Illustration of transformations of the distributional causal effect
with simulated (observational and interventional) data. A: Interventional and
observational CDFs of the response for the two treatment groups. B:
Distributional causal effects on different scales and computed both based on
the estimated interventional and observational CDFs, leading to correct and
biased effects, respectively. $\operatorname{DCE}$ captures the difference in
CDFs (vertical), while $\operatorname{DOK}$ is a difference on the original
scale of the response (horizontal). $\operatorname{LogitCE}$ shows that the
treatment effect is non-constant, which violates the proportional odds
assumption in a simple logistic regression model.

##  3 Identifying distributional causal effects

In order to identify distributional causal effects, we require additional
assumptions on the IV-SCM. We introduce several assumptions in Section 3.1 and
show that they are sufficient for identifiablity in Theorem 1. Finally, in
Section 3.3 we compare these assumptions with the IVQR model and its
assumptions and show that the assumptions provided here are slightly weaker.

###  3.1 Identification assumptions

We begin with an assumption that constrains how the confounding can influence
the treatment assignment.

######  Assumption 1 (Propensity-conditioned unconfoundedness).

In Setting 1, let $\pi(H)\coloneqq\mathbb{E}[D\mid H]$ denote the treatment
propensity and assume

$\displaystyle(Z,D)\perp\nolinebreak\perp H\mid\pi(H).$ (5)

Intuitively, this assumptions limits the treatment assignment to only depend
on $\pi(H)$ and $Z$. For instance, Assumption 1 is fulfilled for an IV-SCM
with jointly independent noise variables when there exists a measurable $h$
such that $f(Z,H,N_{D})=h(Z,\pi(H),N_{D})$.222By joint independence of
$(Z,H,N_{D})$ and weak union (Lemma 2), we have
$(Z,N_{D})\perp\nolinebreak\perp(H,\pi(H))\implies(Z,N_{D})\perp\nolinebreak\perp
H\mid\pi(H)$. Further, by Dawid (1979, Lemma 4.1), we have
$(Z,N_{D},\pi(H))\perp\nolinebreak\perp H\mid\pi(H)$. We conclude using the
existence and measurability of $h$ that
$h(Z,\pi(H),N_{D})\perp\nolinebreak\perp H\mid\pi(H)$. The assumption can also
be interpreted in terms adding a node $\pi(H)$ that $d$-separates $H$ from $D$
in the graph from Setting 1 and is closely related to the “balance equation”
(see Lemma 4) in propensity score analysis (Rosenbaum and Rubin, 1983).

Next, we consider assumptions on the ranks of individual observations among
the treated and untreated group.

######  Assumption 2 (Rank assumptions).

In Setting 1, let $\pi(H)\coloneqq\mathbb{E}[D\mid H]$ denote the treatment
propensity conditional on $H$. We then assume one of the following conditions.

  1. (RI)[asmp:crs:rinv] _Rank invariance_ :

$\displaystyle F^{*}_{D}(Y)\perp\nolinebreak\perp D\mid H,N_{Y}.$ (6)

  2. (RS)[asmp:crs:rsim] _Rank similarity_ :

$\displaystyle F^{*}_{D}(Y)\perp\nolinebreak\perp D\mid H.$ (7)

  3. (CRS)[asmp:crs:crs] _Conditional rank similarity_ :

$\displaystyle F^{*}_{D}(Y)\perp\nolinebreak\perp D\mid\pi(H).$ (8)

Rank invariance is equivalent to
$F^{*}_{1}(g_{1}(H,N_{Y}))=F^{*}_{0}(g_{0}(H,N_{Y}))$ being true almost
surely. Rank similarity and conditional rank similarity assume that, after
taking into account the hidden confounding or treatment propensity,
respectively, the distribution of the rank of an individual is the same
regardless of whether this individual is treated or untreated. The following
result shows that conditional rank similarity is strictly weaker than rank
similarity and rank invariance. A proof is given in Appendix D.2.

######  Proposition 1 (Rank assumptions).

Under Setting 1, Assumption 2 LABEL:asmp:crs:rinv implies LABEL:asmp:crs:rsim
and LABEL:asmp:crs:rsim implies LABEL:asmp:crs:crs. None of the reverse
implications hold.

Example 1 below, provides an example IV-SCM in which rank invariance is
violated, but rank similarity is not. The remaining examples prove that none
of the reverse implications hold are provided in Examples 4 and 5 in Appendix
B. They may help the reader to gain additional intuition on the rank
assumptions.

######  Example 1 (Only rank invariance violated).

Consider the IV-SCM given by

$\displaystyle H$ $\displaystyle\coloneqq N_{H}$ $\displaystyle D$
$\displaystyle\coloneqq\mathds{1}(N_{D}\geq 0.2+0.6\mathds{1}(H\geq 0))$
$\displaystyle Y$ $\displaystyle\coloneqq H-D+DN_{Y}-(1-D)N_{Y},$

where $N_{H},N_{Y}\sim\operatorname{Normal}(0,1)$,
$N_{D}\sim\operatorname{Uniform}(0,1)$ jointly independent. Here, it holds
that $F_{0}:y\mapsto\Phi(y/\sqrt{2})$ and
$F_{1}:y\mapsto\Phi((1+y)/\sqrt{2})$. We now discuss the three types of rank
assumptions mentioned above and summarize the example in Figure 2.

Figure 2: Overview of the rank assumptions discussed in Example 1. A: Rank
invariance does not hold. B: Rank similarity holds. C: Conditional rank
similarity holds. D: The iPIT residuals are uniform.

Since $F_{0}(g_{0}(H,N_{Y}))=\Phi((H+N_{Y})/\sqrt{2})$ and
$F_{1}(g_{1}(H,N_{Y}))=\Phi((H-N_{Y})/\sqrt{2})$, rank invariance is violated.
Figure 2A shows the (non-degenerate) joint distribution of the ranks
$F_{0}^{*}(g_{0}(H,N_{Y}))$ and $F_{1}^{*}(g_{1}(H,N_{Y}))$. Rank similarity
is fulfilled, since for all $h\in\mathbb{R}$,
$\Phi((h+N_{Y})/\sqrt{2})\stackrel{{\scriptstyle
d}}{{=}}\Phi((h-N_{Y})/\sqrt{2})$. This is illustrated in Figure 2B, which
plots the conditional distributions of the iPIT residuals given the hidden
confounder $H$. Conditional rank invariance holds by Proposition 1 and is
illustrated in Figure 2C. In Figure 2D, we see that the iPIT residuals
$DF_{1}^{*}(Y)+(1-D)F_{0}^{*}(Y)=D\Phi((H+N_{Y})/\sqrt{2})+(1-D)\Phi((H+2N_{Y})/\sqrt{3})$
are uniformly distributed.

Finally, we consider an assumption on the relationship between treatment and
instrument that slightly strengthens the “relevance” assumption (Angrist et
al., 1996), which assumes that $D\not\perp\nolinebreak\perp Z$ and is used for
nonparametric identification of average causal effects.

######  Assumption 3 (Relevance).

In Setting 1, assume that for all measurable and bounded
$m,k:\mathcal{H}\rightarrow\mathbb{R}$ it holds that

$m(H)+k(H)D\perp\nolinebreak\perp Z\quad\Longrightarrow\quad
k(H)=0\quad\mathbb{P}_{H}\mbox{-almost surely.}$

Assumption 3 requires that the instrument affects the joint distribution of
$D$ and $Y$ in a specific way. Under no hidden confounding, the assumption is
trivially fulfilled. If hidden confounding is present, Assumption 3 is
stronger than the conventional notion of relevance (i.e.,
$D\not\perp\nolinebreak\perp Z$) and additionally enforces a type of
monotonicity (see Example 2 and (11) below). Although Assumption 3 is not
testable, it is falsifiable because Assumption 3 implies the classical and
testable notion of relevance, $D\not\perp\nolinebreak\perp Z$.333Assume
relevance is violated, i.e., $D\perp\nolinebreak\perp Z$ holds. Then, for
$m\equiv 0$ and $k\equiv 1$, $m(H)+k(H)D\perp\nolinebreak\perp Z$ but $k(H)=1$
almost surely. Thus, Assumption 3 is violated.We show in Example 2 that this
assumption is satisfied in simple IV-SCMs.

######  Example 2.

Consider an IV-SCM which fulfills
$N_{Z},N_{H}\sim\operatorname{Bernoulli}(0.5)$,
$N_{D}\sim\operatorname{Uniform}(0,1)$ which are jointly independent and
$D\coloneqq f(Z,H,N_{D})=\mathds{1}(N_{D}\leq p(H,Z))$. In this setup,
Assumption 3 is satisfied if $p(0,0)-p(0,1)\neq p(1,0)-p(1,1)$ and for all
$z,h\in\\{0,1\\}$, $p(z,h)\neq 1$. This type of condition on the probabilities
is similar in spirit to the monotonicity assumption in binary IV models, as it
avoids cancellations in how the effect of $Z$ propagates via $D$. This can be
seen by considering the conditional distribution of $H$ and $D$ given $Z$. A
formal proof is given in Appendix D.5.

###  3.2 Identification result

We are now ready to present the main identification result which also serves
as a basis for the estimator we propose in Section 4. A proof is given in
Appendix D.

######  Theorem 1 (Identification).

Assume Setting 1. Then, under Assumption 3 and either Assumptions 1 and 2
LABEL:asmp:crs:crs or Assumption 2 LABEL:asmp:crs:rsim, it holds for all
$F_{0},F_{1}\in\mathcal{F}\coloneqq\\{F:\mathbb{R}\to[0,1]\mid F\mbox{ is a
continuous CDF}\\}$ that

$\displaystyle F_{D}(Y)\perp\nolinebreak\perp Z\mbox{ and
}F_{D}(Y)\sim\operatorname{Uniform}(0,1)\iff\forall d\in\\{0,1\\}:\
F_{d}=F_{d}^{*}.$ (9)

Thus, for all $d\in\\{0,1\\}$, the interventional CDF under treatment $d$ is
identified under either a weaker rank similarity assumption (Assumption 2
LABEL:asmp:crs:crs) but a stronger assumption on the confounding mechanism
(Assumption 1), or a stronger rank similarity assumption (Assumption 2
LABEL:asmp:crs:rsim) but without restricting the confounding mechanism. In
both cases, relevance is required for identification.

###  3.3 Relation to potential outcomes and the IVQR model

Estimating distributional causal effects using IVs has been considered before
using a formulation based on quantile functions and potential outcomes, called
IVQR (Chernozhukov and Hansen, 2004, 2005, 2006, 2008; Chernozhukov et al.,
2007). However, the identification strategy employed in IVQR is fundamentally
different to the one we propose and use in Theorem 1. To illustrate this, we
first introduce the IVQR model and its identification strategy and then show
that if an IVQR model has an identifiable quantile causal effect, the model
can be expressed in terms of an IV-SCM (Setting 1) in which our
identifiability conditions hold. The IVQR model (Chernozhukov and Hansen,
2005) is formulated in terms of potential outcomes (Rubin, 1974) and relies on
the conditions outlined in Setting 2.444We have adapted the notation/setting
used in Chernozhukov and Hansen (2005) to be more in line with ours. In
particular, we use the slightly stronger independence assumption
$(U(0),U(1),V)\perp\nolinebreak\perp Z$ instead of
$(U(0),U(1))\perp\nolinebreak\perp Z$ and express the rank similarity
condition in terms of conditional independence rather than conditional
distributions.

######  Setting 2 (IVQR model).

There exists a distribution over the variables

$(Z,D,U(0),U(1),Y(0),Y(1))\in\mathcal{Z}\times\\{0,1\\}\times[0,1]\times[0,1]\times\mathcal{Y}\times\mathcal{Y},$
(10)

satisfying the following conditions:

  1. 1.

(Potential outcomes). For all $d\in\\{0,1\\}$,
$U(d)\sim\operatorname{Uniform}(0,1)$ and there exists a strictly increasing
function $q_{d}:[0,1]\to\mathbb{R}$, such that $Y(d)=q_{d}(U(d))$.

  2. 2.

(Selection). There exists a measurable
$\delta:\mathbb{R}^{d_{Z}}\times\mathbb{R}^{d_{V}}\to\\{0,1\\}$ and random
vector $V\in\mathbb{R}^{d_{V}}$, such that $D=\delta(Z,V)$.

  3. 3.

(Independence). It holds that $(U(0),U(1),V)\perp\nolinebreak\perp Z$.

  4. 4.

(Rank similarity). It holds that $U(D)\perp\nolinebreak\perp D\mid V,Z$.

  5. 5.

(Consistency, no interference, i.i.d.). The observed data
$\mathcal{D}_{n}\coloneqq\\{(Z_{1},D_{1},Y_{1}),\ldots,(Z_{n},D_{n},Y_{n})\\}$
are i.i.d. copies of $(Z,D,Y(D))$.

Condition 1 ensure the existence of a structural quantile function under
treatment $d$, for all $d\in\\{0,1\\}$, that takes the endogenous noise $U(d)$
as input. Since for all $d\in\\{0,1\\}$, $U(d)$ is uniform, it can be thought
of as a rank of an individual under treatment $d$. Rank similarity corresponds
to the conditional distribution of $U(d)$ given $V$ and $Z$ to be the same for
all $d$. For our identification strategy, we employed conditional rank
similarity (Assumption 2 LABEL:asmp:crs:crs), which was shown to be strictly
weaker than both rank invariance and rank similarity (Proposition 1).
Condition 2 assumes the existence of a structural equation for the treatment
assignment, where $\nu$ plays the role of $(H,N_{D})$. Condition 5 postulates
consistency, no interference and that the data consist of i.i.d. copies of the
observable $(Z,D,Y(D))$.

The identification strategy used for IVQR models (Chernozhukov and Hansen,
2005) is based on the fact that under Setting 2, it holds for all
$\tau\in(0,1)$ that

$\mathbb{P}(Y\leq q_{D}(\tau)\mid Z)=\tau,$

that is, the conditional CDF of the observed $Y$ given $Z$ evaluated at the
structural quantile functions under the observed treatment $D$, and any $\tau$
does not depend on $Z$ and is equal to $\tau$. This is fundamentally different
from our condition in which the population iPIT residual $F^{*}_{D}(Y)$ is
shown to be uniform and independent of $Z$.

Proposition 2 below establishes that for each IVQR model there exists an IV-
SCM from Setting 1 which fulfills rank similarity as defined in Assumption 2
LABEL:asmp:crs:rsim with equivalent observational and interventional
distributions.

######  Proposition 2.

Let $M$ be an IVQR model as in Setting 2 and denote by $P^{M}$ the
distribution over the potential outcomes (10). Then, there exists an IV-SCM
$\mathcal{G}(M)$ as in Setting 1 satisfying Assumption 2 LABEL:asmp:crs:rsim,
such that

$P^{M}_{Z,D,Y(D)}=P^{\mathcal{G}(M)}_{Z,D,Y},\quad
P^{M}_{Y(0)}=P^{\mathcal{G}(M);\operatorname{do}(D=0)}_{Y}\quad\text{and}\quad
P^{M}_{Y(1)}=P^{\mathcal{G}(M);\operatorname{do}(D=1)}_{Y}.$

A proof is given in Appendix D.3. Proposition 2 in particular implies, given
appropriate relevance conditions, that, for all $d\in\\{0,1\\}$, whenever the
structural quantile function under treatment $d$ in an IVQR model is
identifiable, there exists a corresponding IV-SCM in which the interventional
CDF under treatment $d$ is identified as well. The assumption of rank
similarity can even be weakened slightly to conditional rank similarity in the
IV-SCM setting (see Assumption 2). Chernozhukov and Hansen (2005) state an
explicit relevance condition for point identification when $Z,D\in\\{0,1\\}$
based on the impact of the instrument on the joint density of $D$ and $Y$
conditional on $Z$, denoted by $f_{Y,D\mid
Z}:\mathbb{R}\times\\{0,1\\}\times\\{0,1\\}\to[0,\infty)$. Namely, it is
assumed that for all $y\in\mathcal{Y}$,

$\displaystyle\frac{f_{Y,D\mid Z}(y,1\mid 1)}{f_{Y,D\mid Z}(y,0\mid
1)}>\frac{f_{Y,D\mid Z}(y,1\mid 0)}{f_{Y,D\mid Z}(y,0\mid 0)},$ (11)

which is referred to as a “monotone likelihood ratio condition”. This notion
of relevance is distinct from the independence condition in Assumption 3.

##  4 Distributional IV estimator

Using the identification strategy (Theorem 1), we propose an estimation
procedure based on uniformity of the empirical iPIT residuals and their
independence of the instrument. In Definitions 3 and 4 we present a loss
function and an estimator which, for all $d\in\\{0,1\\}$, consistently
estimates the interventional CDF under treatment $d$ (Proposition 3).

######  Definition 3 (Distributional IV loss).

Let $\lambda>0$ be a constant,
$\beta:\mathbb{R}_{+}\times\mathbb{R}_{+}\to\mathbb{R}_{+}$ a Lipschitz-
continuous aggregation function such that $\beta(a,b)=0$ if and only if
$(a,b)=(0,0)$ and $k_{R}:[0,1]\times[0,1]\rightarrow\mathbb{R}$ and
$k_{Z}:\mathcal{Z}\times\mathcal{Z}\rightarrow\mathbb{R}$ two positive
definite kernels. Moreover, let
$\mathcal{D}_{n}=\\{(Y_{i},D_{i},Z_{i})\\}_{i=1}^{n}$ be data from Setting 1.
Then, for all pairs of CDFs $(F_{0},F_{1})\in\\{F:\mathbb{R}\to[0,1]\mid
F\mbox{ continuous CDF}\\}$, we define the distributional IV loss

$\ell_{\lambda}(F_{0},F_{1};\mathcal{D}_{n})\coloneqq\beta\bigg{(}\widehat{\operatorname{CvM}}((F_{D_{i}}(Y_{i}))_{i=1}^{n}),\lambda\widehat{\operatorname{HSIC}}((F_{D_{i}}(Y_{i}))_{i=1}^{n},(Z_{i})_{i=1}^{n};k_{R},k_{Z})\bigg{)},$

where $\widehat{\operatorname{CvM}}((F_{D_{i}}(Y_{i}))_{i=1}^{n})$ denotes the
empirical Cramér–von Mises criterion w.r.t. the standard uniform distribution
(see Definition 6 in Appendix E, Cram’er, 1928) and
$\widehat{\operatorname{HSIC}}(\boldsymbol{\cdot},\boldsymbol{\cdot};k_{R},k_{Z})$
denotes the empirical Hilbert-Schmidt Independence criterion (see Definition 8
in Appendix E, Gretton et al., 2007) based on kernels $k_{R}$ and $k_{Z}$.

The CvM and HSIC terms entering the distributional IV loss enforce uniformity
of the empirical iPIT residuals and their independence of the instruments,
respectively: Firstly, the population CvM (Definition 5 in Appendix E) is zero
if and only if it’s argument is uniformly distributed on $(0,1)$ (Lemma 8).
The empirical Cramér–von Mises criterion can be used as a nonparametric test
of whether $(X_{i})_{i=1}^{n}$ follow a standard uniform distribution
(Cram’er, 1928). Secondly, the population HSIC (see Definition 7 in Appendix
E) using characteristic kernels is zero if and only if the arguments are
independent. The choice of kernel can be made depending on the sample space of
$Z$. For continuous $Z$, we use a Gaussian kernel and for discrete $Z$ a
discrete kernel. For the ranks, we use a Gaussian kernel after transforming
the ranks with the standard normal quantile function $\Phi^{-1}$. The
empirical HSIC is formally defined in Definition 8 in Appendix E.

In the following we use the aggregation function $\beta(a,b)=a+b$. Other
options are also possible, however the choice did not strongly affect our
empirical results so we do not consider this further (see Appendix C.2 for
results using $\beta(a,b)=\max\\{a,b\\}$). For the case of normal
interventional CDFs under both treatment $0$ and $1$ and with equal variances,
the loss landscape is shown for different values of $\lambda$ and
interventional means $\mu_{0},\mu_{1}$ in Figure 3.

Figure 3: Loss-landscape for the linear IV model in terms of the
interventional means $\mu_{d}$, $d\in\\{0,1\\}$ and for
$\lambda\in\\{0.01,0.1,1\\}$.

We parametrize the CDFs $F_{0}$ and $F_{1}$, using Bernstein polynomials of
order $M$. More formally, define $a_{M}:[0,1]\to\mathbb{R}^{M+1}$ such that
for all $j\in\\{1,\ldots,M+1\\}$ and all $y\in[0,1]$ it holds that
$(a_{M})_{j}(y)=\binom{M}{j}y^{j-1}(1-y)^{M-j+1}$ and define the parameter
space
$\Theta_{M}\coloneqq\\{\vartheta\in\mathbb{R}^{M+1}\mid\vartheta_{1}\leq\vartheta_{2}\leq\dots\leq\vartheta_{M+1}\\}$.
Then, due to the linear constraints in $\Theta_{M}$, the basis expansion
$a_{M}(y)^{\top}\vartheta$ for any $\vartheta\in\Theta_{M}$ is monotone
increasing (Farouki, 2012). Since Bernstein polynomials can approximate any
continuous function on the interval $[0,1]$, they can be used to parameterize
the CDF of any bounded absolutely continuous real-valued random random
variable (Farouki, 2012). We can therefore parameterize the interventional CDF
under treatment $d$, for all $d\in\\{0,1\\}$ and $\vartheta_{d}\in\Theta_{M}$,
as $F_{d}:y\mapsto\Phi(a_{M}(y)^{\top}\vartheta_{d})$, where $\Phi$ denotes
the standard normal CDF. Using this parametrization together with the
distributional IV loss we can define our proposed estimator.

######  Definition 4 (DIVE).

Let $\mathcal{D}_{n}$ be data as in Setting 1, $M\in\mathbb{N}$ and $B>0$
fixed constants and assume $\mathcal{Y}=[L,U]$ for some $L,U\in\mathbb{R}$.
Then, for
$\mathcal{F}_{B}\coloneqq\\{F:\mathbb{R}\rightarrow[0,1]\mid\exists\vartheta\in\Theta_{M}\text{
with }\left\lVert\vartheta\right\rVert_{\infty}<B\text{ s.t.\ }\forall
y\in\mathbb{R}:\,F(y)=\Phi(a_{M}(\tfrac{y-L}{U-L})^{\top}\vartheta)\\}$, we
define the distributional IV estimator (DIVE) as

$\displaystyle(\widehat{F}_{0}^{n,\lambda},\widehat{F}_{1}^{n,\lambda})\coloneqq\operatorname*{{arg\,min}}_{(F_{0},F_{1})\in\mathcal{F}_{B}^{2}}\ell_{\lambda}(F_{0},F_{1};\mathcal{D}_{n}).$
(12)

The choice of $\Phi$ in our parameterization influences the tail behavior of
DIVE. In our software, we implement several alternative choices to $\Phi$,
including the standard logistic, minimum and maximimum extreme value CDFs. In
practice, we suggest to cross-validate the choice. While we choose $\Phi$ and
Bernstein polynomials to parameterize CDFs for computational convenience,
other approaches based on nonparametric shape-constrained regression could be
used as well (Groeneboom and Jongbloed, 2014).

As long as we assume that, for all $d\in\\{0,1\\}$, the true interventional
CDF under treatment $d$ can be parametrized using the same Bernstein
polynomials, DIVE converges with increasing sample size to the interventional
CDF. A proof is given in Appendix D.4

######  Proposition 3 (Consistency).

Assume Setting 1, Assumption 3 and either Assumptions 1 and 2
LABEL:asmp:crs:crs or Assumption 2 LABEL:asmp:crs:rsim. Furthermore assume
there exists $M\in\mathbb{N}$, $B>0$ and
$\vartheta_{0}^{*},\vartheta_{1}^{*}\in\Theta_{M}$ such that for all
$y\in\mathcal{Y}$ and $d\in\\{0,1\\}$ it holds that
$F_{d}^{*}(y)=\Phi(a_{M}(\tfrac{y-L}{U-L})^{\top}\vartheta_{d}^{*})$ and
$F_{d}^{*}\in\mathcal{F}_{B}$. Then, it holds for all $\lambda>0$ and
$d\in\\{0,1\\}$ that

$\displaystyle\sup_{y\in\mathcal{Y}}\,\lvert\widehat{F}_{d}^{n,\lambda}(y)-F_{d}^{*}(y)\rvert\xrightarrow{\,p\,}0\quad\text{as
$n\rightarrow\infty$},$ (13)

where $\widehat{F}_{d}^{n,\lambda}$ denotes the DIVE for $\mathcal{F}_{B}$
(Definition 4) using fixed bounded, continuously differentiable,
characteristic kernels $k_{R}$ and $k_{Z}$ with bounded derivatives.

The procedure we use to estimate the interventional CDF under treatment $d$
for $d\in\\{0,1\\}$, is given in Algorithm 1. The idea is to estimate the CDFs
as in Definition 4 for a fixed penalty parameter $\lambda$, then test whether
the resulting iPIT residuals satisfy uniformity and independence from $Z$ and
finally either terminate the algorithm if neither test rejects or repeat the
same steps but with an updated value for $\lambda$. More formally, we
initialize $\lambda$ (initialize_lambda routine) such that the two terms
entering the loss function in Definition 3 are of the same size for the
conditional distribution functions (i.e., $\widehat{\mathbb{P}}(Y\leq y\mid
D=d)$) estimated via maximum likelihood and using the same Bernstein basis as
for DIVE. We then estimate the interventional CDFs under treatment $d$, for
all $d\in\\{0,1\\}$, for $\lambda$ as in Definition 4, evaluate the iPIT
residuals and compute $p$-values for both a Cramér–von Mises (CvM) test
(Cram’er, 1928), which tests for uniformity of the iPIT residuals, and an HSIC
independence test (Gretton et al., 2007), which tests for independence between
$Z$ and the iPIT residuals. If both $p$-values exceed a pre-specified level
$\alpha$, the procedure stops and the interventional CDFs are returned.
Otherwise, if the $p$-value of the HSIC test is lower (higher) than that of
the CvM test, $\lambda$ is increased (decreased) with a heuristic and the same
procedure is run again. In the case that the procedure does not terminate
after a fixed number of $T$ iterations, the last estimated CDFs and a warning
of non-convergence at level $\alpha$ are returned. Failure to converge, could
either be due to a type-I error in the tests, exceeding the computational
budget (maximum number of updates), in which case the algorithm could be re-
started with a different value for $\lambda$ and a larger maximum number of
updates, or at least one of the identifying assumptions (Theorem 1) are
violated (see Section 6).

Algorithm 1 Distributional Instrumental Variables Estimation.

1:Data $\mathcal{D}_{n}$, maximum number of restarts $T\in\mathbb{N}$,
significance level $\alpha\in(0,1)$, sequence of step sizes
$\nu_{1},\ldots,\nu_{T}>0$.

2:$t\leftarrow 1$

3:$\lambda\leftarrow$ initialize_lambda($\mathcal{D}_{n}$)

4:while $t\leq T$ do

5: Obtain $(\widehat{F}_{0}^{n,\lambda},\widehat{F}_{1}^{n,\lambda})$ as in
Definition 4 using SGD

6: $R_{n}\leftarrow(\widehat{F}^{n,\lambda}_{D_{i}}(Y_{i}))_{i=1}^{n}$

7: $p_{U}\leftarrow$ $p$-value of CvM test for uniformity of $R_{n}$

8: $p_{I}\leftarrow$ $p$-value of HSIC test for independence of $R_{n}$ and
$(Z_{i})_{i=1}^{n}$ with kernels $k_{R}$, $k_{Z}$

9: if $\min(p_{U},p_{I})>\alpha$ then

10: return $(\widehat{F}_{0}^{n,\lambda},\widehat{F}_{1}^{n,\lambda})$

11: end if

12: $\lambda\leftarrow\lambda(1+\nu_{t})^{2\mathds{1}(p_{I}\leq p_{U})-1}$

13: $t\leftarrow t+1$

14:end while

15:return $(\widehat{F}_{0}^{n,\lambda},\widehat{F}_{1}^{n,\lambda})$ and
‘Warning: DIVE did not converge.’

If Algorithm 1 fails to terminate within the specified number of restarts, it
could indicate that some of the identifying assumptions in Theorem 1 are not
satisfied. For instance, if $Z$ is not a valid instrument
($N_{Z}\not\perp\nolinebreak\perp N_{Y}\mid N_{H}$ or
$N_{Z}\not\perp\nolinebreak\perp N_{H}$ in Setting 1) or if (conditional) rank
similarity (Assumption 2) is violated, even the population iPIT residuals
could either be dependent on the instrument or not uniformly distributed,
leading the test to reject. Therefore, if the algorithm does not terminate
within the specified number of restarts, one should treat the resulting
estimate with care.

###  4.1 Computational details

DIVE is implemented in the R language for statistical computing (R Core Team,
2020) and publicly available at https://github.com/LucasKook/dive. The
implementation of DIVE relies on Bernstein and nonparametric transformation
models trainable via gradient descent implemented in deeptrafo (Kook et al.,
2024), using tensorflow (Allaire and Tang, 2022) and the Adam optimizer
(Kingma and Ba, 2015). The HSIC criterion and test are implemented in dHSIC
(Pfister and Peters, 2019) and the CvM test is implemented in goftest (Faraway
et al., 2021). We use linear IVQR implemented in the IVQR R package available
at https://github.com/yuchang0321/IVQR.

##  5 Numerical experiments

###  5.1 Simulation study

We investigate the following simulation scenarios within Setting 1 and
describe comparator methods and evaluation metrics below. We consider sample
sizes $n\in\\{100,200,400,800,1600\\}$ with $M=50$ and, to run Algorithm 1, we
use $\alpha=0.1$, $T=10$, gradient descent using Adam with learning rate
$0.1$, callbacks to reduce the learning rate (patience 20, tolerance $0.001$),
stop early (patience 60, tolerance $0.0001$) and step sizes $\nu_{t}=\nu/t$
for $\nu=5$ and $t\in\\{1,\dots,10\\}$. The observational and interventional
CDFs under treatment $d$, for all $d\in\\{0,1\\}$, for each scenario are
depicted in Figure C1 in Appendix C. All scenarios fulfill Assumptions 2
LABEL:asmp:crs:rsim.

######  Scenario 1 (Linear additive noise).

Consider an IV-SCM with $N_{Z},N_{H},N_{D}\sim\operatorname{Logistic}(0,1)$
mutually independent, $f(H,Z,N_{D})\coloneqq\mathds{1}(4Z+4H\geq N_{D})$,
$g_{0}(H,N_{Y})\coloneqq-10+6H$ and $g_{1}(H,N_{Y})\coloneqq-2+6H$.

######  Scenario 2 (Linear additive heteroscedastic noise).

Consider an IV-SCM with
$N_{Z},N_{H},N_{D},N_{Y}\sim\operatorname{Logistic}(0,1)$ mutually
independent, $f(H,Z,N_{D})\coloneqq\mathds{1}(4Z+4H\geq N_{D})$,
$g_{0}(H,N_{Y})\coloneqq 6H+HN_{Y}$ and $g_{1}(H,N_{Y})\coloneqq
16+6H+HN_{Y}$.

######  Scenario 3 (Non-linear non-additive noise).

Consider Setting 1 with $N_{Z},N_{H},N_{D}\sim\operatorname{Logistic}(0,1)$
mutually independent, $f(H,Z,N_{D})\coloneqq\mathds{1}(4Z+4H\geq N_{D})$,
$g_{0}(H,N_{Y})\coloneqq\log(1+\exp(18+6H))$ and
$g_{1}(H,N_{Y})\coloneqq\log(1+\exp(26+6H))$.

######  Scenario 4 (Linear additive noise with crossing).

Consider an IV-SCM with $N_{Z},N_{H},N_{D}\sim\operatorname{Logistic}(0,1)$
mutually independent, $f(H,Z,N_{D})\coloneqq\mathds{1}(4Z+4H\geq N_{D})$,
$g_{0}(H,N_{Y})\coloneqq 6H$ and $g_{1}(H,N_{Y})\coloneqq 9H-6$.

We use a Bernstein polynomial of the same order as DIVE to estimate the
conditional CDF (CCDF) of $Y$ given $D$ to compare against the observational
conditional distribution. Due to the unobserved confounding this method is
biased.

Given a sample $\\{(Z_{i},D_{i},Y_{i})\\}_{i=1}^{n}$ from one of Scenarios 1–4
above, to quantify the discrepancy between the true interventional CDFs
$(F^{*}_{0},F_{1}^{*})$ and DIVE
$(\widehat{F}_{0}^{n,\lambda},\widehat{F}_{1}^{n,\lambda})$, we consider the
the mean squared error, which is given by

$\displaystyle\operatorname{MSE}(\widehat{F}_{0}^{n,\lambda},\widehat{F}_{1}^{n,\lambda})\coloneqq\frac{1}{n}\sum_{i=1}^{n}(F_{D_{i}}^{*}(Y_{i})-\widehat{F}_{D_{i}}^{n,\lambda}(Y_{i}))^{2},$
(14)

and the maximum absolute error,

$\displaystyle\operatorname{MAE}(\widehat{F}^{n,\lambda}_{0},\widehat{F}^{n,\lambda}_{1})\coloneqq\max_{i\in\\{1,\dots,n\\}}\lvert
F_{D_{i}}^{*}(Y_{i})-\widehat{F}^{n,\lambda}_{D_{i}}(Y_{i})\rvert,$ (15)

which measures the largest distance between the true interventional CDFs and
the estimated CDFs.

Figure 4 summarizes the performance of DIVE and CCDF in terms of MSE and MAE
for increasing sample sizes and the scenarios described above. In all
scenarios, the estimation error for DIVE decreases linearly (on a double-
logarithmic scale). In contrast, for the CCDF the estimation error converges
to a constant, indicating that it converges to the true conditional CDFs which
are different from the interventional CDFs due to the unobserved confounding.

Figure 4: Results for the simulation scenarios described in Section 5.1 for
DIVE and CCDF in terms of mean squared error and maximum absolute error
compared to the true interventional CDFs.

###  5.2 Distributional causal effect of residing in a metropolitan area on
wage

We apply DIVE to the SchoolingReturns dataset contained in the R package
ivreg. We model the distribution of the log-transformed wage depending on
whether an individual resided in a standard metropolitan statistical area in
1976. The instrument is the binary indicator of whether or not an individual
grew up near a four-year college. We use the same settings for DIVE as in
Section 5.1 but with $M=20$. Figure 5 summarizes the results in terms of the
estimated CDFs (A), uniformity of the empirical iPIT residuals and their
independence of the instrument (B), the estimated distributional causal effect
(C) and a comparison with two-stage least squares (2SLS) and a linear model
for estimating the average causal effect (D).

Figure 5: Results for the schooling return dataset from Section 5.2 for 50
subsamples of the data of size 1000. A: Estimated CDFs of log-transformed wage
by whether an individual grew up in a metropolitan area or not for DIVE and
CCDF. B: ECDFs of the empirical (interventional) PIT residuals stratified by
the instrument (whether a person grew up close to a four-year college or not).
C: Distributional causal effect estimated via DIVE and CCDF. D: Comparison of
the estimated average causal effect for DIVE, CCDF, OLS and 2SLS (obtained via
numerical integration for DIVE and CCDF). Standard errors are based on 50
subsamples.

DIVE estimates a larger distributional causal effect for all wages compared to
the observational conditional distribution, which also translates in a larger
estimated average causal effect. The iPIT residuals based on the fitted DIVE
satisfy that both a CvM test for uniformity and an HSIC test for independence
to the instrument (growing up near a four-year college) are not rejected.
Moreover, while DIVE and 2SLS agree in terms of direction and magnitude of the
average causal effect, the distributional causal effect estimates shows that
the effect is larger for lower wages. This illustrates how distributional
causal effects can provide further insights into the effect of living in a
metropolitan area on wages that cannot be captured by considering the average
causal effect.

The estimated interventional CDFs can be inverted to obtain an estimate of the
quantile causal effect, which allows the comparison of DIVE and IVQR. Figure 6
(a) shows the estimated QCEs for DIVE and IVQR. Overall, there is large
agreement between IVQR and DIVE over 50 subsamples of the data of sample size
1000. Since IVQR requires a grid search over both coefficients and quantiles,
the resulting effect estimates may lead to quantile crossing (Bassett and
Koenker, 1982), which cannot appear using DIVE. The IVQR estimates show larger
variation, especially for more extreme quantiles (at 0.05 and 0.95), compared
to DIVE.

(a) Schooling data

(b) 401(k) data

Figure 6: Estimated quantile treatment effects for 50 subsamples of the
schooling and 401(k) datasets of size 1000 using DIVE and IVQR with linear
quantile regression for $\tau\in(0.1,0.9)$.

###  5.3 Distributional causal effect of 401(k) participation on net financial
assets

We re-analyze the 401(k) dataset (Chernozhukov and Hansen, 2004) using DIVE.
We model the distribution of log-transformed net total financial assets
depending on whether an individual participated in a 401(k) plan or not. The
instrument is the binary indicator of whether or not an individual was
eligible to participate in a 401(k) plan. Figure 7 summarizes the results. We
use the same settings for DIVE as in Section 5.1 but with $M=10$.

On the 401(k) data, DIVE estimates a larger distributional causal effect for
all wages compared to the observational conditional distribution, which again
translates in a larger estimated average causal effect. DIVE and 2SLS agree
closely in terms of direction and magnitude of the average causal effect. Also
in this application, the distributional causal effect estimates show that the
effect is larger for lower net total financial assets.

Figure 7: Results for the 401(k) dataset from Section 5.3 for 50 subsamples of
the data of size 1000. A: Estimated CDFs of net total financial assets by
whether an individual participates in a 401(k) plan or not for DIVE and CCDF.
B: ECDFs of the empirical (interventional) PIT residuals stratified by the
instrument (whether a person is eligible to participate in a 401(k) plan). C:
Distributional causal effect estimated via DIVE and CCDF. D: Comparison of the
estimated average causal effect for DIVE, CCDF, OLS and 2SLS (obtained via
numerical integration for DIVE and CCDF). Standard errors are based on 50
subsamples.

Figure 6 (b) shows the estimated QCE for both DIVE and IVQR. For both methods,
the estimated quantile causal effect decreases with increasing quantiles.
Similar to the schooling dataset, the variability of the IVQR estimates is
larger than that of DIVE over 50 subsamples of the dataset of size 1000.

##  6 Discussion and outlook

We propose a method for identifying and estimating distributional causal
effects using IV based on two properties of the iPIT residuals: Under the
assumptions stated in Theorem 1, only the interventional CDFs given the random
observed treatment $D$ produces iPIT residuals that are both uniformly
distributed and independent of the instrument. As shown both in simulations
and in real-data applications, the resulting estimator, DIVE, can be estimated
consistently and provides valuable information beyond the average causal
effect. Further, DIVE circumvents the issue of quantile crossing (Bassett and
Koenker, 1982) by employing basis expansions instead of estimation over a grid
of quantiles.

IVQR is the most similar method that applies in our setting (binary treatment
and absolutely continuous response). In contrast to our proposed IV-SCM, it is
based on the potential outcome model and uses a different identification
strategy. However, as shown in Proposition 2 the two models are related: For
any IVQR model in which the structural quantile function is identified, there
exists a corresponding IV-SCM with the same observational and interventional
distributions (w.r.t. interventions on the treatment). Thus, under appropriate
relevance conditions, whenever IVQR is applicable, DIVE is too. While
available implementations of IVQR are limited to linear quantile regression,
they provide inference on the estimated quantile treatment effect (for a fixed
quantile) based on asymptotic normality established in Chernozhukov et al.
(2007).

In this work, we did not consider adjusting for additional observed
covariates. A possible extension of DIVE would allow for this by estimating
the conditional interventional CDFs under treatment and control of the
response given some features, for which normalizing flows (Papamakarios et
al., 2021) or other nonparametric distributional regression methods could be
used. When considering conditional IV, the choice between a distributional or
quantile-based estimand becomes important. This is because, in general, an
average quantile function and the corresponding inverse average CDF need not
agree. Both theoretical and computational advantages and disadvantages have
been discussed in Leorato and Peracchi (2015). Furthermore, it is an open
problem whether our identification strategy can be extended to continuous-
valued treatments and whether the control function approach proposed in Imbens
and Newey (2009) or the recently proposed copula-based approach (Chernozhukov
et al., 2024) can be adapted to our setting. We further discuss potential
extensions of DIVE for censored and discrete responses in Appendix A. Whether
asymptotic confidence intervals, analogously to IVQR, for the distributional
causal effect at pre-specified values of the response can be obtained is also
left for future work.

### Acknowledgments

We thank Anton Rask Lundborg for fruitful discussions. LK was supported by the
Swiss National Science Foundation (Grant number 214457). NP was supported by a
research grant (0069071) from Novo Nordisk Fonden.

## References

  * Allaire and Tang [2022] J. Allaire and Y. Tang.  _tensorflow : R Interface to ‘TensorFlow’_, 2022\.  URL https://CRAN.R-project.org/package=tensorflow.  R package version 2.9.0.
  * Angrist et al. [1996] J. D. Angrist, G. W. Imbens, and D. B. Rubin.  Identification of Causal Effects Using Instrumental Variables.  _Journal of the American Statistical Association_ , 91(434):444–455, 1996.  doi:10.1080/01621459.1996.10476902.
  * Bassett and Koenker [1982] G. Bassett and R. Koenker.  An empirical quantile function for linear models with —operatornameiid errors.  _Journal of the American Statistical Association_ , 77(378):407–415, 1982.  doi:10.2307/2287261.
  * Bennett et al. [2019] A. Bennett, N. Kallus, and T. Schnabel.  Deep Generalized Method of Moments for Instrumental Variable Analysis.  In _Advances in Neural Information Processing Systems_ , volume 32, pages 3564–3574, 2019.
  * Chernozhukov and Hansen [2004] V. Chernozhukov and C. Hansen.  The effects of 401 (k) participation on the wealth distribution: An instrumental quantile regression analysis.  _Review of Economics and Statistics_ , 86(3):735–751, 2004.  doi:10.1162/0034653041811734.
  * Chernozhukov and Hansen [2005] V. Chernozhukov and C. Hansen.  An IV Model of Quantile Treatment Effects.  _Econometrica_ , 73(1):245–261, 2005.  doi:10.1111/j.1468-0262.2005.00570.x.
  * Chernozhukov and Hansen [2006] V. Chernozhukov and C. Hansen.  Instrumental Quantile Regression Inference for Structural and Treatment Effect Models.  _Journal of Econometrics_ , 132(2):491–525, 2006\.  doi:10.1016/j.jeconom.2005.02.009.
  * Chernozhukov and Hansen [2008] V. Chernozhukov and C. Hansen.  Instrumental Variable Quantile Regression: A Robust Inference Approach.  _Journal of Econometrics_ , 142(1):379–398, 2008\.  doi:10.1016/j.jeconom.2007.06.005.
  * Chernozhukov et al. [2007] V. Chernozhukov, C. Hansen, and M. Jansson.  Inference Approaches for Instrumental Variable Quantile Regression.  _Economics Letters_ , 95(2):272–277, 2007.  doi:10.1016/j.econlet.2006.10.016.
  * Chernozhukov et al. [2024] V. Chernozhukov, I. Fern’andez-Val, S. Han, and K. Wüthrich.  Estimating causal effects of discrete and continuous treatments with binary instruments.  _arXiv Preprint arXiv:2403.05850_ , 2024.  doi:10.48550/arxiv.2403.05850.
  * Cox [1972] D. R. Cox.  Regression Models and Life-Tables.  _Journal of the Royal Statistical Society B_ , 34(2):187–202, 1972.  doi:10.1111/j.2517-6161.1972.tb00899.x.
  * Cram’er [1928] H. Cram’er.  On the composition of elementary errors.  _Scandinavian Actuarial Journal_ , 1928(1):13–74, 1928.  doi:10.1080/03461238.1928.10416862.
  * Dawid [1979] A. P. Dawid.  Conditional Independence in Statistical Theory.  _Journal of the Royal Statistical Society B_ , 41(1):1–15, 1979.  doi:10.1111/j.2517-6161.1979.tb01052.x.
  * Dawid [1980] A. P. Dawid.  Conditional independence for statistical operations.  _The Annals of Statistics_ , 8(3):598–617, 1980\.  doi:10.1214/aos/1176345011.
  * Doksum [1974] K. Doksum.  Empirical probability plots and statistical inference for nonlinear models in the two-sample case.  _The Annals of Statistics_ , 2(2):267–277, 1974\.  doi:10.1214/aos/1176342662.
  * Dunker [2021] F. Dunker.  Adaptive estimation for some nonparametric instrumental variable models with full independence.  _Electronic Journal of Statistics_ , 15:6151–6190, 2021\.  doi:10.1214/21-ejs1938.
  * Faraway et al. [2021] J. Faraway, G. Marsaglia, J. Marsaglia, and A. Baddeley.  _goftest : Classical Goodness-of-Fit Tests for Univariate Distributions_, 2021.  URL https://CRAN.R-project.org/package=goftest.  R package version 1.2-3.
  * Farouki [2012] R. T. Farouki.  The Bernstein Polynomial Basis: A Centennial Retrospective.  _Computer Aided Geometric Design_ , 29(6):379–419, 2012.  doi:10.1016/j.cagd.2012.03.001.
  * Gretton et al. [2007] A. Gretton, K. Fukumizu, C. Teo, L. Song, B. Schölkopf, and A. Smola.  A kernel statistical test of independence.  In _Advances in Neural Information Processing Systems_ , volume 20, 2007.
  * Groeneboom and Jongbloed [2014] P. Groeneboom and G. Jongbloed.  _Nonparametric Estimation under Shape Constraints_.  Cambridge University Press, 2014.
  * Haavelmo [1943] T. Haavelmo.  The Statistical Implications of a System of Simultaneous Equations.  _Econometrica_ , 11(1):1, 1943.  doi:10.2307/1905714.
  * Hansen et al. [2010] C. Hansen, J. B. McDonald, and W. K. Newey.  Instrumental Variables Estimation With Flexible Distributions.  _Journal of Business & Economic Statistics_, 28(1):13–25, 2010.  doi:10.1198/jbes.2009.06161.
  * Hohberg et al. [2021] M. Hohberg, F. Donat, G. Marra, and T. Kneib.  Beyond Unidimensional Poverty Analysis Using Distributional Copula Models for Mixed Ordered-Continuous Outcomes.  _Journal of the Royal Statistical Society C_ , 70(5):1365–1390, 2021.  doi:10.1111/rssc.12517.
  * Imbens and Newey [2009] G. W. Imbens and W. K. Newey.  Identification and estimation of triangular simultaneous equations models without additivity.  _Econometrica_ , 77(5):1481–1512, 2009.  doi:10.3982/ecta7108.
  * Imbens and Rubin [1997] G. W. Imbens and D. B. Rubin.  Estimating Outcome Distributions for Compliers in Instrumental Variables Models.  _The Review of Economic Studies_ , 64(4):555–574, 1997.  doi:10.2307/2971731.
  * Kingma and Ba [2015] D. P. Kingma and J. L. Ba.  Adam: A Method for Stochastic Optimization.  In _3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings_. International Conference on Learning Representations, ICLR, 2015.  doi:10.48550/arxiv.1412.6980.
  * Kneib [2013] T. Kneib.  Beyond mean regression.  _Statistical Modelling_ , 13(4):275–303, 2013\.  doi:10.1177/1471082X13494159.
  * Kneib et al. [2023] T. Kneib, A. Silbersdorff, and B. Säfken.  Rage against the mean – a review of distributional regression approaches.  _Econometrics and Statistics_ , 26:99–123, 2023.  doi:10.1016/j.ecosta.2021.07.006.
  * Koenker et al. [2017] R. Koenker, V. Chernozhukov, X. He, and L. Peng.  _Handbook of Quantile Regression_.  CRC press, 2017.  doi:10.1201/9781315120256.
  * Kook et al. [2024] L. Kook, P. F. M. Baumann, O. Dürr, B. Sick, and D. Rügamer.  Estimating conditional distributions with neural networks using R package deeptrafo.  _Journal of Statistical Software_ , 2024.  doi:10.48550/arxiv.2211.13665.  Forthcoming.
  * Leorato and Peracchi [2015] S. Leorato and F. Peracchi.  Comparing Distribution and Quantile Regression.  EIEF Working Papers Series 1511, Einaudi Institute for Economics and Finance (EIEF), 2015.  URL https://ideas.repec.org/p/eie/wpaper/1511.html.
  * Lohse et al. [2017] T. Lohse, S. Rohrmann, D. Faeh, and T. Hothorn.  Continuous Outcome Logistic Regression for Analyzing Body Mass Index Distributions.  _F1000Research_ , 6:1933, 2017.  doi:10.12688/f1000research.12934.1.
  * Marcjasz et al. [2023] G. Marcjasz, M. Narajewski, R. Weron, and F. Ziel.  Distributional neural networks for electricity price forecasting.  _Energy Economics_ , 125:106843, 2023.  doi:10.1016/j.eneco.2023.106843.
  * Mooij et al. [2016] J. M. Mooij, J. Peters, D. Janzing, J. Zscheischler, and B. Schölkopf.  Distinguishing cause from effect using observational data: Methods and benchmarks.  _Journal of Machine Learning Research_ , 17(32):1–102, 2016.
  * Muandet et al. [2020] K. Muandet, A. Mehrjou, S. K. Lee, and A. Raj.  Dual Instrumental Variable Regression.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 2710–2721, 2020.
  * Newey [1990] W. K. Newey.  Efficient Instrumental Variables Estimation of Nonlinear Models.  _Econometrica_ , 58(4):809, 1990.  doi:10.2307/2938351.
  * Newey [1991] W. K. Newey.  Uniform convergence in probability and stochastic equicontinuity.  _Econometrica_ , 59(4):1161–1167, 1991.  doi:10.2307/2938179.
  * Newey [2001] W. K. Newey.  conditional Moment Restrictions in Censored and Truncated Regression Models.  _Econometric Theory_ , 17(5):863–888, 2001.  doi:10.1017/s0266466601175018.
  * Newey [2013] W. K. Newey.  Nonparametric Instrumental Variables Estimation.  _American Economic Review_ , 103(3):550–556, 2013\.  doi:10.1257/aer.103.3.550.
  * Newey and Powell [2003] W. K. Newey and J. L. Powell.  Instrumental Variable Estimation of Nonparametric Models.  _Econometrica_ , 71(5):1565–1578, 2003.  doi:10.1111/1468-0262.00459.
  * Papamakarios et al. [2021] G. Papamakarios, E. Nalisnick, D. J. Rezende, S. Mohamed, and B. Lakshminarayanan.  Normalizing flows for probabilistic modeling and inference.  _Journal of Machine Learning Research_ , 22(57):1–64, 2021.
  * Pearl [2009] J. Pearl.  _Causality_.  Cambridge university press, 2009.
  * Pfister and Peters [2019] N. Pfister and J. Peters.  _dHSIC : Independence Testing via Hilbert Schmidt Independence Criterion_, 2019.  URL https://CRAN.R-project.org/package=dHSIC.  R package version 2.1.
  * Pfister and Peters [2022] N. Pfister and J. Peters.  Identifiability of sparse causal effects using instrumental variables.  In _Uncertainty in Artificial Intelligence_ , pages 1613–1622. PMLR, 2022.  doi:10.48550/arxiv.2203.09380.
  * Pfister et al. [2018] N. Pfister, P. Bühlmann, B. Schölkopf, and J. Peters.  Kernel-based tests for joint independence.  _Journal of the Royal Statistical Society B_ , 80(1):5–31, 2018.  doi:10.1111/rssb.12235.
  * Poirier [2017] A. Poirier.  Efficient estimation of models with independence restrictions.  _Journal of Econometrics_ , 196(1):1–22, 2017\.  doi:10.1016/j.jeconom.2016.07.007.
  * R Core Team [2020] R Core Team.  _R: A Language and Environment for Statistical Computing_.  R Foundation for Statistical Computing, Vienna, Austria, 2020.  URL https://www.R-project.org/.
  * Rosenbaum and Rubin [1983] P. R. Rosenbaum and D. B. Rubin.  The central role of the propensity score in observational studies for causal effects.  _Biometrika_ , 70(1):41–55, 1983.  doi:10.1093/biomet/70.1.41.
  * Rosenblatt [1952] M. Rosenblatt.  Remarks on a Multivariate Transformation.  _The Annals of Mathematical Statistics_ , 23(3):470 – 472, 1952.  doi:10.1214/aoms/1177729394.
  * Rubin [1974] D. B. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies.  _Journal of Educational Psychology_ , 66(5):688–701, 1974.  doi:10.1037/h0037350.
  * Saengkyongam et al. [2022] S. Saengkyongam, L. Henckel, N. Pfister, and J. Peters.  Exploiting Independent Instruments: Identification and Distribution Generalization.  In _International Conference on Machine Learning_ , pages 18935–18958. PMLR, 2022.  doi:10.48550/arxiv.2202.01864.
  * Singh et al. [2019] R. Singh, M. Sahani, and A. Gretton.  Kernel Instrumental Variable Regression.  _Advances in Neural Information Processing Systems_ , 32, 2019.  doi:10.7551/mitpress/7503.003.0069.
  * Wooldridge [2015] J. M. Wooldridge.  _Introductory Econometrics: A Modern Approach_.  Cengage learning, 2015.
  * Zhang et al. [2020] R. Zhang, M. Imaizumi, B. Schölkopf, and K. Muandet.  Maximum Moment Restriction for Instrumental Variable Regression.  _arXiv Preprint arXiv:2010.07684_ , 2020.  doi:10.48550/arxiv.2010.07684.

##  Appendix A DIVE for discrete responses

In Setting 1, absolute continuity of $Y$ is a sufficient condition to ensure
that the iPIT residuals are uniformly distributed and indepdent of $Z$. In
Example 3 below, we present a setting with a binary response $Y$ depending on
a binary treatment $D$ without unmeasured confounding and a binary instrument
$Z$, in which $Y$ given $D$ is correctly specified by a logistic regression.
However, the iPIT residuals based on the correctly specified model are neither
uniform nor independent of $Z$. We leave extensions to discrete and censored
cases for future work.

######  Example 3.

Consider the following IV-SCM

$\displaystyle Z$ $\displaystyle\coloneqq 2\mathds{1}(N_{Z}\geq 0)-1$ (16)
$\displaystyle D$ $\displaystyle\coloneqq\mathds{1}(Z+N_{D}\geq 0)$ (17)
$\displaystyle Y$ $\displaystyle\coloneqq\mathds{1}(-1+D+N_{Y}\geq 0),$ (18)

where $N_{Z},N_{D},N_{Y}$ are jointly independent and standard logistically
distributed. Here, for all $d\in\\{0,1\\}$ it holds that
$F_{d}^{*}(0)=\mathbb{P}(-1+d+N_{Y}\leq 0)$ and $F_{d}^{*}(1)=1$. The iPIT
residuals are therefore given by

$\displaystyle(1-Y)F_{D}^{*}(Y)+Y=\begin{cases}\operatorname{expit}(1)&\mbox{if
}D=0,Y=0,\\\ 0.5&\mbox{if }D=1,Y=0,\\\ 1&\mbox{if
}D\in\\{0,1\\},Y=1.\end{cases}$ (19)

This further implies that $(1-Y)F_{D}^{*}(Y)+Y\geq 0.5$ almost surely. We can
now compute

$\displaystyle\mathbb{E}[Z((1-Y)F_{D}^{*}(Y)+Y)]$
$\displaystyle=\mathbb{P}(Z=1)\mathbb{E}[(1-Y)F_{D}^{*}(Y)+Y\mid Z=1]$ (20)
$\displaystyle=0.5^{2}\mathbb{E}[(1-Y)F_{D}^{*}(Y)+Y\mid D=1,Z=1]\ +$ (21)
$\displaystyle\quad\quad 0.5^{2}\mathbb{E}[(1-Y)F_{D}^{*}(Y)+Y\mid D=0,Z=1]$
(22) $\displaystyle=0.5^{2}\mathbb{E}[(1-Y)F_{D}^{*}(Y)+Y\mid D=1]\ +$ (23)
$\displaystyle\quad\quad 0.5^{2}\mathbb{E}[(1-Y)F_{D}^{*}(Y)+Y\mid D=0]>0.$
(24)

Due to the discrete nature of the response, $F_{D}^{*}(Y)$ is also not
uniformly distributed. Further, contrary to the results obtained for Settings
1 and 2, $F_{D}^{*}(Y)\not\perp\nolinebreak\perp Z$.

##  Appendix B Rank assumptions: Examples

######  Example 4 (Violation of rank invariance and rank similarity).

Consider the following IV-SCM

$\displaystyle H$ $\displaystyle\coloneqq N_{H}$ (25) $\displaystyle D$
$\displaystyle\coloneqq\mathds{1}(N_{D}\geq 0.2+0.6\mathds{1}(H>0))$ (26)
$\displaystyle Y$ $\displaystyle\coloneqq-DH^{2}+(1-D)H^{2},$ (27)

where $N_{H}\sim\operatorname{Normal}(0,1)$ and
$N_{D}\sim\operatorname{Uniform}(0,1)$. We now discuss the three types of rank
assumptions mentioned above and summarize the example in Figure B1.

Figure B1: Overview of the rank assumptions discussed in Example 4. A: Rank
invariance does not hold. B: Rank similarity does not hold. C: Conditional
rank similarity holds. D: The iPIT residuals are uniform.

Here, we have for all $d\in\\{0,1\\}$ and $h\in\mathbb{R}$ that

$g_{d}(h)=-dh^{2}+(1-d)h^{2}$

and for all $y\in\mathcal{Y}$ that

$F_{0}^{*}(y)=F_{\chi^{2}_{1}}(y)\mbox{ and
}F_{1}^{*}(y)=1-F_{\chi^{2}_{1}}(-y),$

where $F_{\chi^{2}_{1}}$ is the CDF of a chi-squared distribution with one
degree of freedom. We thus have

$\displaystyle F_{0}^{*}(g_{0}(H))=1-F_{1}^{*}(g_{1}(H))\mbox{ almost
surely,}$ (28)

and

$F_{D}^{*}(g_{D}(H))=F_{0}^{*}(g_{0}(H))+D(F_{1}^{*}(g_{1}(H))-F_{0}^{*}(g_{0}(H))).$

Now, define $W=F_{1}^{*}(g_{1}(H))-F_{0}^{*}(g_{0}(H))$ and
$M=F_{0}^{*}(g_{0}(H))$. Then,

$\displaystyle E[WD+M\mid D=0,H]$ $\displaystyle=E[M\mid D=0,H]=M,\mbox{ and}$
(29) $\displaystyle E[WD+M\mid D=1,H]$ $\displaystyle=E[W+M\mid D=0,H]=W+M,$
(30)

since both $W$ and $M$ are $H$-measurable. By (28), we conclude (29) and (30)
are unequal almost surely and, thus, rank invariance is violated. By
Proposition 1, rank invariance is violated too. We now turn to showing that
conditional rank similarity is fulfilled. The treatment propensity is given by

$\pi(H)=0.2+0.6\mathds{1}(H>0).$

In this specific example, it holds that

$\displaystyle H^{2}\perp\nolinebreak\perp\pi(H).$ (31)

To see this, observe that for all $h>0$,

$\mathbb{P}(H^{2}\leq h\mid\pi(H)=0.2)=\mathbb{P}(H^{2}\leq h\mid H\leq
0)=\mathbb{P}(H^{2}\leq h)=F_{\chi^{2}_{1}}(h),$

and

$\mathbb{P}(H^{2}\leq h\mid\pi(H)=0.8)=\mathbb{P}(H^{2}\leq h\mid
H>0)=\mathbb{P}(H^{2}\leq h)=F_{\chi^{2}_{1}}(h).$

Moreover, since $D\perp\nolinebreak\perp H\mid\pi(H)$ (Lemma 4), it also holds
that

$\displaystyle D\perp\nolinebreak\perp H^{2}\mid\pi(H).$ (32)

Together, (31) and (32) imply $(D,\pi(H))\perp\nolinebreak\perp H^{2}$ by
contraction (Lemma 2). Thus, for all $t\in(0,1)$,

$\displaystyle\mathbb{P}(DW+M\leq t\mid\pi(H),D=0)$
$\displaystyle=\mathbb{P}(M\leq t\mid\pi(H),D=0)$
$\displaystyle=\mathbb{P}(F_{\chi^{2}_{1}}(H^{2})\leq t\mid\pi(H),D=0)$
$\displaystyle=\mathbb{P}(F_{\chi^{2}_{1}}(H^{2})\leq t)=t,$

and

$\displaystyle\mathbb{P}(DW+M\leq t\mid\pi(H),D=1)$
$\displaystyle=\mathbb{P}(W+M\leq t\mid\pi(H),D=1)$
$\displaystyle=\mathbb{P}(1-F_{\chi^{2}_{1}}(H^{2})\leq t\mid\pi(H),D=1)$
$\displaystyle=\mathbb{P}(F_{\chi^{2}_{1}}(H^{2})\leq t)=t,$

where we used that since $U\sim\operatorname{Uniform}(0,1)$ it also holds that
$1-U\sim\operatorname{Uniform}(0,1)$. Thus, conditional rank invariance,
$F_{D}^{*}(g_{D}(H))\perp\nolinebreak\perp D\mid\pi(H)$, holds.

######  Example 5 (Violation of all rank assumptions).

Consider the following IV-SCM

$\displaystyle H$ $\displaystyle\coloneqq N_{H}$ $\displaystyle D$
$\displaystyle\coloneqq\mathds{1}(N_{D}\geq H)$ $\displaystyle Y$
$\displaystyle\coloneqq 2DH-H,$

where $N_{H}\sim\operatorname{Normal}(0,1)$ and $N_{D}$ follows a standard
logistic distribution. We now discuss the three types of rank assumptions
mentioned above and summarize the example in Figure B2.

Figure B2: Overview of the rank assumptions discussed in Example 5. A: Rank
invariance does not hold. B: Rank similarity does not hold. C: Conditional
rank similarity does not hold. D: The iPIT residuals are not uniform.

Here, we have for all $d\in\\{0,1\\}$ and $h\in\mathcal{H}$ that
$g_{d}(h)=2dh-h$ and for all $y\in\mathcal{Y}$ that
$F_{0}^{*}(y)=\Phi(-h)=1-\Phi(h)$ and $F_{1}^{*}(y)=\Phi(h)$. Conditional rank
similarity is not fulfilled, because we have $\pi(H)=\operatorname{expit}(H)$
and thus $\mathbb{P}(F^{*}_{0}(g_{0}(H))\leq
t\mid\pi(H))=\mathbb{P}(F^{*}_{0}(g_{0}(H))\leq t\mid
H)=\mathbb{P}[\Phi(-H)\leq t\mid H)=1-\mathbb{P}(\Phi(H)\leq t\mid H)$ and
$\mathbb{P}(F^{*}_{1}(g_{1}(H))\leq t\mid H)=\mathbb{P}(\Phi(H)\leq t\mid H)$
By the implications in Proposition 1, rank similarity and rank invariance are
violated as well.

##  Appendix C Additional numerical results

###  C.1 Simulation scenarios

Figure C1 depicts the observational and interventional CDFs under treatments
$0$ and $1$ for the simulation scenarios described in Section 5.1.

Figure C1: Observational and interventional CDFs for the simulation scenarios
from Section 5.1. In the first scenario, the interventional CDFs under both
treatments are logistic. In the second, the interventional CDFs under both
treatments are an average of logistic CDFs. In the third scenario, a non-
linear transformation bounds the outcome from below at zero. The fourth
scenario is considerably more challenging as the observational CDFs cross at
lower values of the response whereas the interventional CDFs cross at larger
values of the response.

###  C.2 Combining uniformity and independence losses

Figure C2 shows the results when using $\beta(a,b)\coloneqq\max\\{a,b\\}$ in
the DIV loss (Definition 3) instead of the weighted sum used in the main text.
There is no indication that the choice of $\beta$ changes the interpretation
of the results.

(a) Schooling data

(b) 401(k) data

Figure C2: Results for the schooling return dataset from Section 5.2 and
401(k) dataset from Section 5.3 for 50 subsamples of the data of size 1000
using $\beta(a,b)\coloneqq\max\\{a,b\\}$.

##  Appendix D Proofs

###  D.1 Proof of Theorem 1

###### Proof.

We begin with the backward direction. Using the adjustment formula [e.g.,
Pearl, 2009], for all $d\in\\{0,1\\}$, the interventional CDF under treatment
$d$ of $Y$ can be expressed, for all $y\in\mathcal{Y}$, as

$\displaystyle F^{*}_{d}(y)=\mathbb{E}[\mathds{1}(g_{d}(H,N_{Y})\leq y)].$
(33)

We fist show the case where Assumptions 1 and 2 LABEL:asmp:crs:crs are
satisfied. For all $t\in(0,1)$, we have

$\displaystyle\mathbb{P}(F_{D}^{*}(Y)\leq t)$ (34)
$\displaystyle=\mathbb{E}[\mathds{1}(F^{*}_{D}(Y)\leq t)]$ (35)
$\displaystyle=\mathbb{E}[\mathds{1}(F^{*}_{D}(g_{D}(H,N_{Y}))\leq t)]$ (36)
$\displaystyle=\mathbb{E}[\mathbb{E}[D\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq
t)+(1-D)\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\mid H,N_{Y}]]$ (37)
$\displaystyle=\mathbb{E}[\mathbb{E}[D\mid
H,N_{Y}]\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq t)+\mathbb{E}[(1-D)\mid
H,N_{Y}]\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq t)]$ (38)
$\displaystyle=\mathbb{E}[\mathbb{E}[\pi(H)\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq
t)+(1-\pi(H))\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq t)\mid\pi(H)]]$ (39)
$\displaystyle=\mathbb{E}[\pi(H)\mathbb{E}[\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq
t)\mid\pi(H)]+(1-\pi(H))\mathbb{E}[\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq
t)\mid\pi(H)]]$ (40)
$\displaystyle=\mathbb{E}[\pi(H)\mathbb{E}[\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq
t)\mid\pi(H)]+(1-\pi(H))\mathbb{E}[\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq
t)\mid\pi(H)]]$ (41) $\displaystyle=\mathbb{P}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq
t)$ (42) $\displaystyle=\mathbb{P}(g_{0}(H,N_{Y})\leq(F_{0}^{*})^{-1}(t))$
(43) $\displaystyle=t.$ (44)

For these equalities, we used the following arguments: (35) follows by
expressing the probability as an expectation; (36) follows from substituting
the structural equation of $Y$; (37) follows by the law of total expectation
and since $D$ is binary; (38) follows since the arguments in the indicator
function are $(H,N_{Y})$-measurable; (39) follows by the law of total
expectation; (40) follows since $\pi(H)$ is $\pi(H)$-measurable; (41) follows
from Assumption 2 LABEL:asmp:crs:crs; (42) follows by direct computation; (43)
follows by taking the inverse of $F^{*}$ (which exists since $Y$ is assumed to
be absolutely continuous) on both sides of the inequality. We can hence
conclude that $F_{D}^{*}(Y)$ is uniformly distributed.

To establish that $F_{D}^{*}(Y)$ is independent of $Z$, consider the
interventional distribution of $Y$ conditional on the instrument $Z$. For all
$t\in(0,1)$, we have

$\displaystyle\mathbb{P}(F_{D}^{*}(Y)\leq t\mid Z)$ (45)
$\displaystyle=\mathbb{E}[\mathds{1}(F^{*}_{D}(Y)\leq t)\mid Z]$ (46)
$\displaystyle=\mathbb{E}[\mathds{1}(F^{*}_{D}(g_{D}(H,N_{Y}))\leq t)\mid Z]$
(47) $\displaystyle=\mathbb{E}[D\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq
t)+(1-D)\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\mid Z]$ (48)

Here, (46) follows again by expressing the probability as an expectation; (47)
follows from substituting the structural equation of $Y$; (48) follows by the
law of total expectation and because $D$ is binary. For all $t\in(0,1)$, the
first term in (48) can be expressed as

$\displaystyle\mathbb{E}[D\mathds{1}(F_{1}^{*}(g_{1}(H,N_{Y}))\leq t)\mid Z]$
(49)
$\displaystyle=\mathbb{E}[D\mathbb{E}[\mathds{1}(F_{1}^{*}(g_{1}(H,N_{Y}))\leq
t)\mid D,Z,\pi(H)]\mid Z]$ (50)
$\displaystyle=\mathbb{E}[D\mathbb{E}[\mathds{1}(F_{1}^{*}(g_{1}(H,N_{Y}))\leq
t)\mid\pi(H)]\mid Z],$ (51)

where we used the law of total expectation for (50) and that
$(Z,D)\perp\nolinebreak\perp(H,N_{Y})\mid\pi(H)$ for (51). The conditional
independence $(Z,D)\perp\nolinebreak\perp(H,N_{Y})\mid\pi(H)$ follows by
applying contraction (Lemma 2) to the conditional independencies

$(Z,D)\perp\nolinebreak\perp N_{Y}\mid H,\pi(H)\mbox{ and
}(Z,D)\perp\nolinebreak\perp H\mid\pi(H),$

which themselves follow from Assumption 1 and Lemma 3. The same argument
applies to the second term in (48), yielding

$\displaystyle\mathbb{E}[(1-D)\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq t)\mid
Z]=\mathbb{E}[(1-D)\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq
t)\mid\pi(H)]\mid Z].$ (52)

By Assumption 2 LABEL:asmp:crs:crs, $(H,N_{Y})\perp\nolinebreak\perp Z$ (which
follows by contraction (Lemma 2) with $N_{Z}\perp\nolinebreak\perp N_{H}$ and
$N_{Z}\perp\nolinebreak\perp N_{Y}\mid N_{H}$ from Setting 1) and combining
(51) and (52), we obtain

$\displaystyle\mathbb{P}(F_{D}^{*}(Y)\leq t\mid Z)$ (53)
$\displaystyle=\mathbb{E}[D\mathbb{E}[\mathds{1}(F_{1}^{*}(g_{1}(H,N_{Y}))\leq
t)\mid\pi(H)]\mid
Z]+\mathbb{E}[(1-D)\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq
t)\mid\pi(H)]\mid Z]$ (54)
$\displaystyle=\mathbb{E}[\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq
t)\mid\pi(H)]\mid Z]$ (55)
$\displaystyle=\mathbb{E}[\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq
t)]]=t,$ (56)

from which we conclude $F_{D}^{*}(Y)\perp\nolinebreak\perp Z$, which completes
the backward direction.

Next we show the same result when assuming Assumption 2 LABEL:asmp:crs:rsim
instead of Assumptions 1 and 2 LABEL:asmp:crs:crs. A similar line of argument
as (35)–(43) to show uniformity of $F_{D}^{*}(Y)$ holds: For all $t\in(0,1)$,
we have

$\displaystyle\mathbb{P}(F_{D}^{*}(Y)\leq t)$ (57)
$\displaystyle=\mathbb{E}[\mathbb{E}[D\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq
t)+(1-D)\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq t)\mid H]]$ (58)
$\displaystyle=\mathbb{E}[\pi(H)\mathbb{E}[\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq
t)\mid H]+(1-\pi(H))\mathbb{E}[\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq t)\mid
H]]$ (59)
$\displaystyle=\mathbb{E}[\pi(H)\mathbb{E}[\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq
t)\mid H]+(1-\pi(H))\mathbb{E}[\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq t)\mid
H]]$ (60) $\displaystyle=\mathbb{P}(F^{*}_{0}(g_{0}(H,N_{Y}))\leq
t)=\mathbb{P}(g_{0}(H,N_{Y})\leq(F_{0}^{*})^{-1}(t))=t.$ (61)

To establish independence of $Z$ and $F_{D}^{*}(Y)$ under Assumption 2
LABEL:asmp:crs:rsim, we proceed as in (48) to get for all $t\in[0,1]$ that

$\displaystyle\mathbb{P}(F_{D}^{*}(Y)\leq t\mid Z)$ (62)
$\displaystyle=\mathbb{E}[D\mathds{1}(F^{*}_{1}(g_{1}(H,N_{Y}))\leq
t)+(1-D)\mathds{1}(F^{*}_{0}(g_{0}(H,N_{Y}))\mid Z]$ (63)
$\displaystyle=\mathbb{E}[D\mathbb{E}[\mathds{1}(F_{1}^{*}(g_{1}(H,N_{Y}))\leq
t)\mid D,Z,H]+(1-D)\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq t)\mid
D,Z,H]\mid Z]$ (64)
$\displaystyle=\mathbb{E}[D\mathbb{E}[\mathds{1}(F_{1}^{*}(g_{1}(H,N_{Y}))\leq
t)\mid H]+(1-D)\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq t)\mid
H]\mid Z],$ (65)

where we used the law of total expectation and that
$(Z,D)\perp\nolinebreak\perp(H,N_{Y})\mid H$ by Lemma 3, for the second and
third equality, respectively. Then, combining this with Assumption 2
LABEL:asmp:crs:rsim, we furthermore get that

$\displaystyle\mathbb{P}(F_{D}^{*}(Y)\leq t\mid Z)$ (66)
$\displaystyle=\mathbb{E}[D\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq
t)\mid H]\mid
Z]+\mathbb{E}[(1-D)\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq t)\mid
H]\mid Z]$ (67)
$\displaystyle=\mathbb{E}[\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq
t)\mid H]\mid Z]$ (68)
$\displaystyle=\mathbb{E}[\mathbb{E}[\mathds{1}(F_{0}^{*}(g_{0}(H,N_{Y}))\leq
t)\mid H]]=t,$ (69)

where for the third equality we used that $(H,N_{Y})\perp\nolinebreak\perp Z$
(as shown above). This completes the proof of the backward direction given
Assumption 2 LABEL:asmp:crs:rsim instead of Assumptions 1 and 2
LABEL:asmp:crs:crs.

Next, consider the forward direction and fix arbitrary
$F_{0},F_{1}\in\mathcal{F}$ such that $F_{D}(Y)\perp\nolinebreak\perp Z$ and
$F_{D}(Y)\sim\operatorname{Uniform}(0,1)$. We use the structural equation of
$Y$ to get

$\displaystyle
F_{D}(Y)=DF_{1}(Y)+(1-D)F_{0}(Y)=F_{0}(g_{0}(H,N_{Y}))+D(F_{1}(g_{1}(H,N_{Y}))-F_{0}(g_{0}(H,N_{Y}))).$
(70)

Since $F_{D}(Y)\perp\nolinebreak\perp Z$ we have by Assumption 3, that
$\mathbb{P}_{(H,N_{Y})}$-almost surely,

$\displaystyle F_{1}(g_{1}(H,N_{Y}))=F_{0}(g_{0}(H,N_{Y})).$ (71)

By the above, the premise $F_{D}(Y)\sim\operatorname{Uniform}(0,1)$ implies
that $F_{0}(g_{0}(H,N_{Y}))\sim\operatorname{Uniform}(0,1)$. For all
$d\in\\{0,1\\}$, $Y$ is equal to $g_{d}(H,N_{Y})$ under
$\operatorname{do}(D=d)$, which, together with the uniqueness of the CDF of
absolutely continuous random variables (Lemma 1), implies $F_{0}=F_{0}^{*}$.
By the same argument, we have
$F_{1}(g_{1}(H,N_{Y}))\sim\operatorname{Uniform}(0,1)$ and $F_{1}=F_{1}^{*}$,
which concludes the proof. ∎

###  D.2 Proof of Proposition 1

###### Proof.

Let $\phi:\mathbb{R}\rightarrow\mathbb{R}$ be an arbitrary bounded continuous
function. Then, it holds that

$\displaystyle\mathbb{E}[\phi(F^{*}_{D}(g_{D}(H,N_{Y})))\mid H]$
$\displaystyle\quad=\mathbb{E}[D\phi(F^{*}_{1}(g_{1}(H,N_{Y})))\mid
H]+\mathbb{E}[(1-D)\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid H]$
$\displaystyle\quad=\mathbb{E}[D\mid
H]\mathbb{E}[\phi(F^{*}_{1}(g_{1}(H,N_{Y})))\mid H]+\mathbb{E}[(1-D)\mid
H]\mathbb{E}[\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid H]$
$\displaystyle\quad=\pi(H)\mathbb{E}[\phi(F^{*}_{1}(g_{1}(H,N_{Y})))\mid
H]+(1-\pi(H))\mathbb{E}[\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid H],$

where for the second equality we used that
$D\perp\nolinebreak\perp(H,N_{Y})\mid H$ (which follows directly from Lemma 3
using decomposition from Lemma 2). Similarly, we get

$\displaystyle\mathbb{E}[\phi(F^{*}_{D}(g_{D}(H,N_{Y})))\mid D,H]$
$\displaystyle\quad=\mathbb{E}[\mathbb{E}[\phi(F^{*}_{D}(g_{D}(H,N_{Y})))\mid
D,H,N_{Y}]\mid D,H]$
$\displaystyle\quad=\mathbb{E}[\mathbb{E}[\phi(F^{*}_{D}(g_{D}(H,N_{Y})))\mid
H,N_{Y}]\mid D,H]$
$\displaystyle\quad=\mathbb{E}[\mathbb{E}[D\phi(F^{*}_{1}(g_{1}(H,N_{Y})))\mid
H,N_{Y}]+\mathbb{E}[(1-D)\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid H,N_{Y}]\mid
D,H]$ $\displaystyle\quad=\mathbb{E}[\mathbb{E}[D\mid
H,N_{Y}]\phi(F^{*}_{1}(g_{1}(H,N_{Y})))+\mathbb{E}[(1-D)\mid
H,N_{Y}]\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid D,H]$
$\displaystyle\quad=\mathbb{E}[\mathbb{E}[D\mid
H]\phi(F^{*}_{1}(g_{1}(H,N_{Y})))+\mathbb{E}[(1-D)\mid
H]\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid D,H]$
$\displaystyle\quad=\mathbb{E}[\pi(H)\phi(F^{*}_{1}(g_{1}(H,N_{Y})))+(1-\pi(H))\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid
D,H]$ $\displaystyle\quad=\pi(H)\mathbb{E}[\phi(F^{*}_{1}(g_{1}(H,N_{Y})))\mid
H,D]+(1-\pi(H))\mathbb{E}[\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid D,H]$
$\displaystyle\quad=\pi(H)\mathbb{E}[\phi(F^{*}_{1}(g_{1}(H,N_{Y})))\mid
H]+(1-\pi(H))\mathbb{E}[\phi(F^{*}_{0}(g_{0}(H,N_{Y})))\mid H],$

where we used Assumption 2 LABEL:asmp:crs:rinv for the second equality and
$D\perp\nolinebreak\perp(N_{Y},H)\mid H$ (as above) for the last equality.
Since $\phi$ was arbitrary and using the definition of conditional
independence (Definition 9), we conclude that Assumption 2 LABEL:asmp:crs:rinv
implies Assumption 2 LABEL:asmp:crs:rsim.

To show the second implication, we use

$\displaystyle F_{D}^{*}(g_{D}(H,N_{Y}))\perp\nolinebreak\perp D\mid H\iff
F_{D}^{*}(g_{D}(H,N_{Y}))\perp\nolinebreak\perp D\mid H,\pi(H)$ (72)

which holds by the definition of conditional independence (since the sigma
algebras of $H$ and $(H,\pi(H))$ are identical as $\pi(H)$ is $H$-measurable),
together with $D\perp\nolinebreak\perp H\mid\pi(H)$ (which holds by Lemma 4),
to conclude

$F_{D}^{*}(g_{D}(H,N_{Y}))\perp\nolinebreak\perp D\mid\pi(H)$

by contraction (Lemma 2). Counterexamples showing that the reverse direction
of the first and second implications in Proposition 1 are given in Examples 1
and 4, respectively. ∎

###  D.3 Proof of Proposition 2

###### Proof.

We begin by fully specifying the parametrization of the IVQR models defined in
Setting 2.

Let $\mathcal{P}$ denote the set of all distributions on
$[0,1]\times[0,1]\times\mathbb{R}^{d_{V}}\times\mathcal{Z}$ and define the
sets
$\mathcal{D}\coloneqq\\{\delta:\mathcal{Z}\times\mathbb{R}^{d_{V}}\to\\{0,1\\}\mid\delta\mbox{
measurable}\\}$ and $\mathcal{Q}\coloneqq\\{q:[0,1]\to\mathcal{Y}\mid q\mbox{
measurable and strictly increasing}\\}$. All IVQR models in Setting 2 can then
be parametrized by the following set

$\displaystyle\mathcal{M}_{\rm
IVQR}\coloneqq\Big{\\{}(\nu,\delta,q_{0},q_{1})\in\mathcal{P}\times\mathcal{D}\times\mathcal{Q}\times\mathcal{Q}\,\big{|}\,(U(0),U(1),V,Z)\sim\nu:\,\text{satisfying
Setting~{}\ref{setting:ivqr}}\Big{\\}}.$

Similarly, we now parametrize a subset of the IV-SCMs in Setting 1 for
$\mathcal{H}\coloneqq\mathbb{R}^{d_{V}}$ and $N_{D}=0$. Let
$\mathcal{P}^{\prime}$ denote the set of all distributions on
$\mathbb{R}\times[0,1]^{2}\times\mathbb{R}^{d_{V}}\times\mathbb{R}^{d_{Z}}$
and define the sets
$\mathcal{D}^{\prime}\coloneqq\\{f:\mathcal{Z}\times\mathcal{H}\times\mathbb{R}\to\\{0,1\\}\mid
f\mbox{ measurable}\\}$ and
$\mathcal{Q}^{\prime}\coloneqq\\{g:\mathcal{H}\times[0,1]^{2}\to\mathcal{Y}\mid
g\mbox{ measurable}\\}$. Then, we define the set,

$\displaystyle\mathcal{M}_{\rm IVSCM}\coloneqq\Big{\\{}$
$\displaystyle(\mu,f,g_{0},g_{1})\in\mathcal{P}^{\prime}\times\mathcal{D}^{\prime}\times\mathcal{Q}^{\prime}\times\mathcal{Q}^{\prime}\,\big{|}\,(N_{D},N_{Y},N_{H},N_{Z})\sim\mu,\,$
$\displaystyle\quad\text{with }N_{D}=0\mbox{ a.s.}\text{ and
}(N_{Z},N_{D})\perp\nolinebreak\perp N_{Y}\mid N_{H}\mbox{ and
}N_{H}\perp\nolinebreak\perp N_{Z}\Big{\\}}.$

We now explicitly define a map

$\mathcal{G}:\mathcal{M}_{\rm IVQR}\to\mathcal{M}_{\rm IVSCM}$

and then show that it satisfies the desired properties. To this end, we define
for all $M=(\nu,\delta,q_{0},q_{1})\in\mathcal{M}_{\rm IVQR}$,

$\mathcal{G}(M)\coloneqq(\mu^{M},f^{M},g_{0}^{M},g_{1}^{M}),$

where

  * •

$\mu^{M}\in\mathcal{P}^{\prime}$ is defined by

$\mu^{M}\coloneqq(\Delta_{0}\otimes\nu_{U(0),U(1),V,Z},f,g_{0},g_{1}),$

where $\Delta_{0}$ the Dirac measure at zero on $\mathbb{R}$.

  * •

$f^{M}\in\mathcal{D}^{\prime}$ for all
$(z,h,n)\in\mathcal{Z}\times\mathcal{H}\times\mathbb{R}$ is defined by

$f^{M}(z,h,n)\coloneqq\delta(z,h).$

  * •

$g_{0}^{M}\in\mathcal{Q}^{\prime}$ and $g_{1}^{M}\in\mathcal{Q}^{\prime}$ for
all $(h,n)\in\mathcal{H}\times[0,1]^{2}$ are defined by

$\displaystyle g_{0}(h,n)\coloneqq q_{0}(n_{1})\quad\text{and}\quad
g_{1}(h,n)\coloneqq q_{1}(n_{2}).$ (73)

By construction of $\mu^{M}$ it holds for
$(N_{D},N_{Y},N_{H},N_{Z})\sim\mu^{M}$ that $N_{D}=0$. Moreover, since by
assumption $Z\perp\nolinebreak\perp(U(0),U(1),V)$ for $(U(0),U(1),V,Z)\sim\nu$
we also get (using Lemma 2) that $(N_{Z},N_{D})\perp\nolinebreak\perp
N_{Y}\mid N_{H}$ and $N_{H}\perp\nolinebreak\perp N_{Z}$ for
$(N_{D},N_{Y},N_{H},N_{Z})\sim\mu^{M}$, hence
$\mathcal{G}(M)\in\mathcal{M}_{\rm IVSCM}$ as desired.

We are now ready to prove the main result. To this end, fix
$M=(\nu,\delta,q_{0},q_{1})\in\mathcal{M}_{\rm IVQR}$. Then, this induces a
unique distribution $P^{M}$ over

$(Z,D,U(0),U(1),Y(0),Y(1))$

by first sampling $(U(0),U(1),V,Z)\sim\nu$ and then setting $D=\delta(Z,V)$,
$Y(0)=q_{0}(U(0))$ and $Y(1)=q_{1}(U(1))$. By construction $P^{M}$ satisfies
the conditions of an IVQR model as in Setting 2. We now prove (1) the
equivalence of the observational distributions, (2) equivalence of the
interventional distributions and (3) that $\mathcal{G}(M)$ satisfies rank
similarity (Assumption 2 LABEL:asmp:crs:rsim).

Step 1: Equivalence of observational distribution:
Based on the construction of $P^{M}$ it holds for $(U(0),U(1),V,Z)\sim\nu$
that

$\left(Z,\delta(Z,V),q_{\delta(Z,V)}(U(\delta(Z,V)))\right)\sim
P^{M}_{Z,D,Y(D)}.$ (74)

For the IV-SCM $\mathcal{G}(M)$ we can use the definition of the function
$f^{M}$ to express the structural equation of $D$ in terms of the noise
variables $(N_{D},N_{Y},N_{H},N_{Z})$ as follows

$D=f^{M}(Z,H,N_{D})=\delta(Z,H)=\delta(N_{Z},N_{H}).$ (75)

Similarly, using this expansion and the definitions of $g_{0}^{M}$ and
$g_{1}^{M}$ we get for the structural equation of $Y$ that

$Y=g_{D}(H,N_{Y})=q_{D}((N_{Y})_{1+D})=q_{\delta(N_{Z},N_{H})}((N_{Y})_{1+\delta(N_{Z},N_{H})}).$
(76)

Together with $Z=N_{Z}$ it therefore holds for
$(N_{D},N_{Y},N_{H},N_{Z})\sim\mu^{M}$ that

$(N_{Z},\delta(N_{Z},N_{H}),q_{\delta(N_{Z},N_{H})}((N_{Y})_{1+\delta(N_{Z},N_{H})}))\sim
P^{\mathcal{G}(M)}_{Z,D,Y}.$ (77)

By definition of $\mu^{M}$ it holds for $(N_{D},N_{Y},N_{H},N_{Z})\sim\mu^{M}$
that $(N_{Y},N_{H},N_{Z})\sim\nu$. Hence, (74) and (77) together imply that

$P^{M}_{Z,D,Y(D)}=P^{\mathcal{G}(M)}_{Z,D,Y}.$

Step 2: Equivalence of interventional distributions:
Again by the construction of $P^{M}$ we get for $(U(0),U(1),V,Z)\sim\nu$ that

$q_{0}(U(0))\sim P^{M}_{Y(0)}\quad\text{and}\quad q_{1}(U(1))\sim
P^{M}_{Y(1)}.$ (78)

Furthermore, performing $\operatorname{do}(D=0)$ (respectively,
$\operatorname{do}(D=1)$) and then using the definition of $g_{0}^{M}$ and
$g_{1}^{M}$ together with the intervened structural equations show for
$(N_{D},N_{Y},N_{H},N_{Z})\sim\mu^{M}$ that

$q_{0}((N_{Y})_{1})\sim
P^{\mathcal{G}(M);\operatorname{do}(D=0)}_{Y}\quad\text{and}\quad
q_{1}((N_{Y})_{2})\sim P^{\mathcal{G}(M);\operatorname{do}(D=1)}_{Y}.$ (79)

Hence, using again the definition of $\mu^{M}$, (78) and (79) imply that

$P^{M}_{Y(0)}=P^{\mathcal{G}(M);\operatorname{do}(D=0)}_{Y}\quad\text{and}\quad
P^{M}_{Y(1)}=P^{\mathcal{G}(M);\operatorname{do}(D=1)}_{Y}.$

Step 2: Rank similarity:
By (79) and since $(N_{Y})_{1}$ and $(N_{Y})_{2}$ are both distributed
uniformly on $[0,1]$ (using the definition of $\mu^{M}$, assumptions on $\nu$
and that $q_{0}$ and $q_{1}$ are strictly increasing) it holds for all
$d\in\\{0,1\\}$ that $F^{*}_{d}=q_{d}^{-1}$. Moreover, using (75) and (76) it
follows for $(N_{D},N_{Y},N_{H},N_{Z})\sim\mu^{M}$ that

$F^{*}_{D}(g_{D}(H,N_{Y}))=(N_{Y})_{1+\delta(N_{Z},N_{H})},\quad
D=\delta(N_{Z},N_{H})\quad\text{and}\quad H=N_{H}.$ (80)

Moreover, by the connection between $\mu^{M}$ and $\nu$ it holds that

$(N_{D},N_{Y},N_{H},N_{Z})\sim\mu^{M}:\quad(N_{Y})_{1+\delta(N_{Z},N_{H})}\perp\nolinebreak\perp\delta(N_{Z},N_{H})\mid
N_{H}$ (81)

if and only if

$(U(0),U(1),V,Z)\sim\nu:\quad
U(\delta(Z,V))\perp\nolinebreak\perp\delta(Z,V)\mid V.$ (82)

Now rank similarity in the IVQR model (i.e., $U(D)\perp\nolinebreak\perp D\mid
V,Z$) together with the independence assumption
$(U(0),U(1))\perp\nolinebreak\perp Z\mid V$ imply that (82) and therefore also
(81) is true. Hence, using (80), it holds that $\mathcal{G}(M)$ satisfies
Assumption 2 LABEL:asmp:crs:rsim.

This completes the proof of Proposition 2. ∎

###  D.4 Proof of Proposition 3

###### Proof.

Fix an arbitrary $\lambda>0$. Define $s:\mathcal{Y}\to\mathbb{R}$ for all
$y\in\mathcal{Y}$ by $s(y)\coloneqq\frac{y-L}{U-L}$,
$\Theta\coloneqq\Theta_{M}\times\Theta_{M}$,
$\vartheta^{*}\coloneqq\left((\vartheta^{*}_{0})^{\top},(\vartheta^{*}_{1})^{\top}\right)^{\top}$,
and, for all $(d,y)\in\\{0,1\\}\times\mathcal{Y}$ define

$a_{d}(y)\coloneqq\left((1-d)a_{M}(s(y))^{\top},da_{M}(s(y))^{\top}\right)^{\top}.$

Furthermore, for all $(d,y)\in\\{0,1\\}\times\mathcal{Y}$ and
$\vartheta\in\Theta$, define

$F_{d}(y;\vartheta)\coloneqq\Phi\left(a_{d}(y)^{\top}\vartheta\right),$

and the following sample and population quantities:

$\displaystyle{H}_{n}(\vartheta)$
$\displaystyle\coloneqq\widehat{\operatorname{HSIC}}((F_{D_{i}}(Y_{i};\vartheta))_{i=1}^{n},(Z_{i})_{i=1}^{n};k_{R},k_{Z}),$
(83) $\displaystyle H(\vartheta)$
$\displaystyle\coloneqq\operatorname{HSIC}(F_{D}(Y;\vartheta),Z;k_{R},k_{Z}),$
(84) $\displaystyle U_{n}(\vartheta)$
$\displaystyle\coloneqq\int_{0}^{1}\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(F_{D_{i}}(Y_{i};\vartheta)\leq
u)-u\right)^{2}\,\mathrm{d}u$ (85) $\displaystyle U(\vartheta)$
$\displaystyle\coloneqq\int_{0}^{1}(\mathbb{P}(F_{D}(Y;\vartheta)\leq
u)-u)^{2}\,\mathrm{d}u,$ (86) $\displaystyle\ell_{\lambda,n}(\vartheta)$
$\displaystyle\coloneqq\beta(\lambda H_{n}(\vartheta),U_{n}(\vartheta)),$ (87)
$\displaystyle\ell_{\lambda}(\vartheta)$ $\displaystyle\coloneqq\beta(\lambda
H(\vartheta),U(\vartheta)),$ (88)

where the population HSIC is defined in Definition 7 in Appendix E and
$\widehat{\operatorname{HSIC}}$ is defined in Definition 8 in Appendix E.

The proof proceeds in four steps. In the first two steps, we show uniform
convergence of (83) to (84) and (85) to (86) and conclude that (87) converges
uniformly to (88). In the third step, we show convergence in probability of
the estimated basis coefficients to the true $\vartheta^{*}$. In the final
step, we prove uniform convergence of the interventional CDFs.

#### Step 1.

To show uniform convergence of (83) to (84), we can follow the argument in
Saengkyongam et al. [2022, Appendix A.3.] to conclude:

  * •

Since $k_{R}$ and $k_{Z}$ are assumed to be non-negative and bounded, it
follows by Mooij et al. [2016, Corollary 15], that for all
$\vartheta\in\Theta$ and $\epsilon>0$,

$\lim_{n\to\infty}\mathbb{P}(\lvert
H_{n}(\vartheta)-H(\vartheta)\rvert>\epsilon)=0;$

  * •

By Mooij et al. [2016, Lemma 16] it follows that for all
$\vartheta,\vartheta^{\prime}\in\Theta$ and $n\geq 2$,

$\displaystyle\left\lvert
H_{n}(\vartheta)-H_{n}(\vartheta^{\prime})\right\rvert$
$\displaystyle\leq\frac{32LC}{\sqrt{n}}\left\lVert(a_{D_{i}}(Y_{i})^{\top}(\vartheta-\vartheta^{\prime}))_{i=1}^{n}\right\rVert$
$\displaystyle\leq\frac{32LC}{\sqrt{n}}\left\lVert(a_{D_{i}}(Y_{i})^{\top})_{i=1}^{n}\right\rVert\cdot\left\lVert\vartheta-\vartheta^{\prime}\right\rVert$
$\displaystyle\leq\frac{32LC}{\sqrt{n}}\sqrt{nA^{2}}\left\lVert\vartheta-\vartheta^{\prime}\right\rVert$
$\displaystyle=32LCA\left\lVert\vartheta-\vartheta^{\prime}\right\rVert;$

where $L$ is the Lipschitz constant for
$k_{R}(\Phi^{-1}(\boldsymbol{\cdot}),\Phi^{-1}(\boldsymbol{\cdot}))$, $C$ is
an upper bound for $k_{Z}$ and $A$ satisfies
$\sup_{d\in\\{0,1\\}}\sup_{y\in\mathcal{Y}}\lVert a_{d}(y)\rVert<A$, which
exists by boundedness of $a_{d}$ for $d\in\\{0,1\\}$,

  * •

$k_{R}$ and $\vartheta\mapsto F_{D}(Y;\vartheta)$ are (almost surely)
continuously differentiable and $k_{R}$ and $k_{Z}$ have a bounded derivative
by assumption. Thus, by dominated convergence, the chain rule, the population
representation of HSIC as in Pfister et al. [2018, Proposition 2.5] and
compactness of $\Theta$, it follows that $H(\vartheta)$ is Lipschitz on
$\Theta$.

With the three points above, we can apply Newey [1991, Corollary 2.2] to
conclude that

$\displaystyle\lim_{n\to\infty}\mathbb{P}\left(\max_{\vartheta\in\Theta}\lvert
H_{n}(\vartheta)-H(\vartheta)\rvert>\epsilon\right)=0,$ (89)

where we additionally used that, by continuity of
$\vartheta\mapsto|H_{n}(\vartheta)-H(\vartheta)|$ and compactness of $\Theta$
the function attains its maximum.

#### Step 2.

It holds that:

  * •

For all $\vartheta\in\Theta$ and all $\epsilon>0$,
$\lim_{n\to\infty}\mathbb{P}(\lvert
U_{n}(\vartheta)-U(\vartheta)\rvert>\epsilon)=0$ (Lemma 6);

  * •

$U_{n}$ and $U$ are Lipschitz (Lemma 7).

We can conclude by Newey [1991, Corollary 2.2] that, for all $\epsilon>0$,

$\displaystyle\lim_{n\to\infty}\mathbb{P}\left(\max_{\vartheta\in\Theta}\lvert
U_{n}(\vartheta)-U(\vartheta)\rvert>\epsilon\right)=0.$ (90)

Next, using that $\beta$ is Lipschitz continuous with constant $B$, we get

$\displaystyle\lim_{n\to\infty}\mathbb{P}\left(\max_{\vartheta\in\Theta}\lvert\ell_{\lambda,n}(\vartheta)-\ell_{\lambda}(\vartheta)\rvert>\epsilon\right)$
(91)
$\displaystyle\quad=\lim_{n\to\infty}\mathbb{P}\left(\max_{\vartheta\in\Theta}\lvert\beta(U_{n}(\vartheta),\lambda
H_{n}(\vartheta))-\beta(U(\vartheta),\lambda
H(\vartheta))\rvert>\epsilon\right)$ (92)
$\displaystyle\quad\leq\lim_{n\to\infty}\mathbb{P}\left(\max_{\vartheta\in\Theta}\max\\{\left\lvert
U_{n}(\vartheta)-U(\vartheta)\right\rvert,\lambda\left\lvert
H_{n}(\vartheta)-H(\vartheta)\right\rvert\\}>\epsilon/B\right)$ (93)
$\displaystyle\quad\leq\lim_{n\to\infty}\mathbb{P}\left(\\{\max_{\vartheta\in\Theta}\left\lvert
U_{n}(\vartheta)-U(\vartheta)\right\rvert>\epsilon/B\\}\cup\\{\max_{\vartheta\in\Theta}\lambda\left\lvert
H_{n}(\vartheta)-H(\vartheta)\right\rvert>\epsilon/B\\}\right)$ (94)
$\displaystyle\quad\leq\lim_{n\to\infty}\mathbb{P}\left(\max_{\vartheta\in\Theta}\left\lvert
U_{n}(\vartheta)-U(\vartheta)\right\rvert>\epsilon/B\right)+\mathbb{P}\left(\max_{\vartheta\in\Theta}\lambda\left\lvert
H_{n}(\vartheta)-H(\vartheta)\right\rvert>\epsilon/B\right)$ (95)
$\displaystyle\quad=0,$ (96)

where for the last step we used (89) and (90).

#### Step 3.

Now consider any sequence
$\widehat{\vartheta}^{n,\lambda}\in\operatorname*{{arg\,min}}_{\vartheta\in\Theta}\ell_{\lambda,n}(\vartheta)$.
By Theorem 1 (and since we assumed that Assumption 3 and either Assumptions 1
and 2 LABEL:asmp:crs:crs or Assumption 2 LABEL:asmp:crs:rsim are satisfied) it
holds that $\vartheta^{*}$ is the unique parameter for which
$F_{D}(Y;\vartheta^{*})\perp\nolinebreak\perp Z$ and
$F_{D}(Y;\vartheta^{*})\sim\operatorname{Uniform}(0,1)$. Moreover, by Pfister
et al. [2018], $H(\vartheta)=0\iff F_{D}(Y;\vartheta)\perp\nolinebreak\perp Z$
and by Lemma 8, $U(\vartheta)=0\iff
F_{D}(Y;\vartheta)\sim\operatorname{Uniform}(0,1)$. Hence, we get that
$(U(\vartheta),H(\vartheta))=(0,0)\iff\vartheta=\vartheta^{*}$, which together
with $\beta(a,b)=0\iff(a,b)=(0,0)$ implies that $\vartheta^{*}$ is the unique
minimizer of $\ell_{\lambda}$ on $\Theta$. Therefore, for all $\epsilon>0$,
there exists $\zeta(\epsilon)$ such that for all $\vartheta\notin
B_{\epsilon}(\vartheta^{*})\coloneqq\\{\vartheta\in\Theta:\lVert\vartheta-\vartheta^{*}\rVert<\epsilon\\}$
it holds that
$\ell_{\lambda}(\vartheta)-\ell_{\lambda}(\vartheta^{*})>\zeta(\epsilon)$.

Fix $\epsilon>0$ and $\delta>0$, then by (96) there exists
$n^{\prime}\in\mathbb{N}$, such that for all $n\in\mathbb{N}$ with $n\geq
n^{\prime}$,

$\displaystyle\mathbb{P}\left(\max_{\vartheta\in\Theta}\lvert\ell_{\lambda,n}(\vartheta)-\ell_{\lambda}(\vartheta)\rvert>\frac{\zeta(\epsilon)}{2}\right)\leq\frac{\delta}{2}.$
(97)

Then, for all $n\in\mathbb{N}$ with $n\geq n^{\prime}$,

$\displaystyle\mathbb{P}(\lVert\widehat{\vartheta}^{n,\lambda}-\vartheta^{*}\rVert>\epsilon)$
$\displaystyle\leq\mathbb{P}\left(\min_{\vartheta\in\Theta\setminus
B_{\epsilon}(\vartheta^{*})}\\{\ell_{\lambda,n}(\vartheta)-\ell_{\lambda,n}(\vartheta^{*})\\}\leq
0\right)$ $\displaystyle\leq\mathbb{P}\left(\max_{\vartheta\in\Theta\setminus
B_{\epsilon}(\vartheta^{*})}\lvert\ell_{\lambda,n}(\vartheta)-\ell_{\lambda}(\vartheta)\rvert+\lvert\ell_{\lambda,n}(\vartheta^{*})-\ell_{\lambda}(\vartheta^{*})\rvert>\zeta(\epsilon)\right)$
$\displaystyle\leq\mathbb{P}\left(\left\\{\max_{\vartheta\in\Theta\setminus
B_{\epsilon}(\vartheta^{*})}\lvert\ell_{\lambda,n}(\vartheta)-\ell_{\lambda}(\vartheta)\rvert>\frac{\zeta(\epsilon)}{2}\right\\}\cup\left\\{\lvert\ell_{\lambda,n}(\vartheta^{*})-\ell_{\lambda}(\vartheta^{*})\rvert>\frac{\zeta(\epsilon)}{2}\right\\}\right)$
$\displaystyle\leq\delta,$

where

  * •

the first inequality follows because
$\\{\lVert\widehat{\vartheta}^{n,\lambda}-\vartheta^{*}\rVert>\epsilon\\}$
occurs only if there exists a $\vartheta\in\Theta\setminus
B_{\epsilon}(\vartheta^{*})$ such that
$\\{\ell_{\lambda,n}(\vartheta)-\ell_{\lambda,n}(\vartheta^{*})\leq 0\\}$
occurs;

  * •

the second inequality follows because $\zeta(\epsilon)$ is such that for all
$\vartheta\in\Theta\setminus B_{\epsilon}(\vartheta^{*})$ we have
$\ell_{\lambda}(\vartheta)-\ell_{\lambda}(\vartheta^{*})>\zeta(\epsilon)$.
Hence, if there exists $\vartheta\in\Theta$ such that
$\\{\ell_{\lambda,n}(\vartheta)-\ell_{\lambda,n}(\vartheta^{*})\leq 0\\}$
occurs, then there must exists $\vartheta\in\Theta$ such that
$\\{\lvert\ell_{\lambda,n}(\vartheta)-\ell_{\lambda}(\vartheta)\rvert+\lvert\ell_{\lambda}(\vartheta^{*};\mathcal{D}_{n})-\ell_{\lambda}(\vartheta^{*})\rvert>\zeta(\epsilon)\\}$
occurs;

  * •

the third inequality follows because for $\\{\max_{\vartheta\in\Theta\setminus
B_{\epsilon}(\vartheta^{*})}\lvert\ell_{\lambda,n}(\vartheta)-\ell_{\lambda}(\vartheta)\rvert+\lvert\ell_{\lambda,n}(\vartheta^{*})-\ell_{\lambda}(\vartheta^{*})\rvert>\zeta(\epsilon)\\}$
to occur, either $\left\\{\max_{\vartheta\in\Theta\setminus
B_{\epsilon}(\vartheta^{*})}\lvert\ell_{\lambda,n}(\vartheta)-\ell_{\lambda}(\vartheta)\rvert>\frac{\zeta(\epsilon)}{2}\right\\}$
or
$\left\\{\lvert\ell_{\lambda,n}(\vartheta^{*})-\ell_{\lambda}(\vartheta^{*})\rvert>\frac{\zeta(\epsilon)}{2}\right\\}$
must occur;

  * •

and the last inequality follows by the union bound together with (97).

Since $\delta>0$ was arbitrary, this implies
$\lim_{n\to\infty}\mathbb{P}(\lVert\widehat{\vartheta}^{n,\lambda}-\vartheta^{*}\rVert>\epsilon)=0$.

#### Step 4.

Finally, for all $d\in\\{0,1\\}$, by $\frac{1}{\sqrt{2\pi}}$-Lipschitz
continuity of $\Phi$ (which follows since $\Phi$ is differentiable everywhere
and its derivative is a density bounded above by $\tfrac{1}{\sqrt{2\pi}}$) and
boundedness of $a_{d}$, $d\in\\{0,1\\}$ by $A$, we obtain

$\displaystyle\sup_{y\in\mathcal{Y}}\left\lvert\widehat{F}_{d}^{n,\lambda}(y)-F_{d}^{*}(y)\right\rvert$
$\displaystyle=\sup_{y\in\mathcal{Y}}\left\lvert\Phi(a_{d}(y)^{\top}\widehat{\vartheta}^{n,\lambda})-\Phi(a_{d}(y)^{\top}\vartheta^{*})\right\rvert$
$\displaystyle\leq\tfrac{1}{\sqrt{2\pi}}\sup_{y\in\mathcal{Y}}\left\lvert
a_{d}(y)^{\top}\widehat{\vartheta}^{n,\lambda}-a_{d}(y)^{\top}\vartheta^{*}\right\rvert$
$\displaystyle\leq\tfrac{1}{\sqrt{2\pi}}\sup_{y\in\mathcal{Y}}\left\lVert
a_{d}(y)\right\rVert\cdot\left\lVert\widehat{\vartheta}^{n,\lambda}-\vartheta^{*}\right\rVert$
$\displaystyle\leq\tfrac{A}{\sqrt{2\pi}}\left\lVert\widehat{\vartheta}^{n,\lambda}-\vartheta^{*}\right\rVert,$

which concludes the proof. ∎

###  D.5 Proof of the statement in Example 2

###### Proof.

Fix arbitrary functions $m:\\{0,1\\}\rightarrow\mathbb{R}$ and
$k:\\{0,1\\}\rightarrow\mathbb{R}$. For all $h,d\in\\{0,1\\}$ define
$\gamma(h,d)=m(h)+k(h)d$. Then, for $W=w(H,D)$ and
$\mathcal{W}\coloneqq\\{m(0),m(1),m(0)+k(0),m(1)+k(1)\\}$ it holds that

$\displaystyle W\perp\nolinebreak\perp Z\iff\forall$ $\displaystyle
w\in\mathcal{W}:\,\mathbb{P}(W=w\mid Z=0)=\mathbb{P}(W=w\mid Z=1).$ (98)

Furthermore, explicitly computing the probabilities we get that for all
$w\in\mathcal{W}$ and all $z\in\\{0,1\\}$ it holds that

$\displaystyle\mathbb{P}(W=w\mid Z=z)$ $\displaystyle=0.5\mathbb{P}(W=w\mid
Z=z,H=0)+0.5\mathbb{P}(W=w\mid Z=z,H=1)$
$\displaystyle=0.5\big{(}p(0,z)\mathds{1}(\gamma(0,1)=w)+(1-p(0,z))\mathds{1}(\gamma(0,0)=w)\big{)}$
$\displaystyle\quad\quad+0.5\big{(}p(1,z)\mathds{1}(\gamma(1,1)=w)+(1-p(1,z))\mathds{1}(\gamma(1,0)=w)\big{)}.$

Next, define for all $w\in\mathcal{W}$,
$l_{1}(w)\coloneqq\mathds{1}(\gamma(0,1)=w)-\mathds{1}(\gamma(0,0)=w)$ and
$l_{2}(w)\coloneqq\mathds{1}(\gamma(1,1)=w)-\mathds{1}(\gamma(1,0)=w)$. Then,
by (98), we get that $W\perp\nolinebreak\perp Z$ is equivalent to: for all
$w\in\mathcal{W}$

$\displaystyle
p(0,0)l_{1}(w)+\mathds{1}(\gamma(0,0)=w)+p(1,0)l_{2}(w)+\mathds{1}(\gamma(1,0)=w)$
(99)
$\displaystyle\qquad=p(0,1)l_{1}(w)+\mathds{1}(\gamma(0,0)=w)+p(1,1)l_{2}(w)+\mathds{1}(\gamma(1,0)=w).$
(100)

Simplifying this expression we get that $W\perp\nolinebreak\perp Z$ is
equivalent to for all $w\in\mathcal{W}$

$\displaystyle(p(0,0)-p(0,1))l_{1}(w)+(p(1,0)-p(1,1))l_{2}(w)=0.$

Now, the functions $l_{1}$ and $l_{2}$ both take values in $\\{-1,0,1\\}$,
hence since by assumption $p(0,0)-p(0,1)\neq p(1,0)-p(1,1)$ and for all
$z,h\in\\{0,1\\}$, $p(z,h)\neq 1$ it holds that

$\\{(p(0,0)-p(0,1))l_{1}(w)\mid
w\in\mathcal{W}\\}\cap\\{-(p(1,0)-p(1,1))l_{2}(w)\mid
w\in\mathcal{W}\\}\subseteq\\{0\\}.$

Hence, the only way for (99) to be satisfied is if for all $w\in\mathcal{W}$
it holds that $l_{1}(w)=l_{2}(w)=0$. This is however equivalent to
$\gamma(0,1)=\gamma(0,0)$ and $\gamma(1,0)=\gamma(1,1)$, which is equivalent
to $k(0)=k(1)=0$. Therefore, we have shown that

$Z\perp\nolinebreak\perp W\iff k\equiv 0,$

which completes the proof. ∎

##  Appendix E Auxiliary definitions and results

######  Definition 5 (Population CvM criterion).

The population Cramér–von Mises criterion w.r.t. the standard uniform
distribution for a real-valued random variable $X$ is defined as

$\operatorname{CvM}(X)\coloneqq\int_{0}^{1}\left(\mathbb{P}(X\leq
u)-u\right)^{2}\,\mathrm{d}u.$

######  Definition 6 (Empirical CvM criterion).

The empirical Cramér–von Mises criterion w.r.t. the standard uniform
distribution for $n$ i.i.d. real-valued random variables $(X_{i})_{i=1}^{n}$
is defined as

$\widehat{\operatorname{CvM}}((X_{i})_{i=1}^{n})\coloneqq\int_{0}^{1}\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(X_{i}\leq
u)-u\right)^{2}\,\mathrm{d}u=\frac{1}{12n^{2}}+\frac{1}{n}\sum_{i=1}^{n}\left(\frac{i-0.5}{n}-X_{(i)}\right)^{2},$

where, for all $i\in\\{1,\dots,n\\}$, $X_{(i)}$ denotes the $i$th order
statistic and the equality is shown in Lemma 5.

######  Definition 7 (Population HSIC).

Let $(Y,X)$ be random variables taking values in
$\mathcal{Y}\times\mathcal{X}\subseteq\mathbb{R}^{d_{Y}}\times\mathbb{R}^{d_{X}}$
and $k:\mathcal{Y}\times\mathcal{Y}\to\mathbb{R}_{+}$,
$l:\mathcal{X}\times\mathcal{X}\to\mathbb{R}_{+}$ be two positive definite
kernels. Let $(Y^{\prime},X^{\prime})$ and
$(Y^{\prime\prime},X^{\prime\prime})$ denote independent copies of $(Y,X)$.
Then, the population HSIC is defined as

$\displaystyle\operatorname{HSIC}(Y,X;k,l)\coloneqq\mathbb{E}[k(Y,Y^{\prime})l(X,X^{\prime})]+\mathbb{E}[k(Y,Y^{\prime})]\mathbb{E}[l(X,X^{\prime})]-2\mathbb{E}[k(Y,Y^{\prime})l(X,X^{\prime\prime})].$
(101)

######  Definition 8 (Empirical HSIC).

Let $(Y_{i},X_{i})_{i=1}^{n}$ be independent and identically distributed
random variables taking values in
$\mathcal{Y}\times\mathcal{X}\subseteq\mathbb{R}^{d_{Y}}\times\mathbb{R}^{d_{X}}$
and $k:\mathcal{Y}\times\mathcal{Y}\to\mathbb{R}_{+}$,
$l:\mathcal{X}\times\mathcal{X}\to\mathbb{R}_{+}$ be two positive definite
kernels. Then, the empirical HSIC is defined as

$\displaystyle\widehat{\operatorname{HSIC}}((Y_{i})_{i=1}^{n},(X_{i})_{i=1}^{n};k,l)$
(102) $\displaystyle\quad\coloneqq\
\frac{1}{n^{2}}\sum_{i,j}^{n}k(Y_{i},Y_{j})l(X_{i},X_{j})+\frac{1}{n^{4}}\sum_{i,j,q,r}^{n}k(Y_{i},Y_{j})l(X_{q},X_{r})-\frac{2}{n^{3}}\sum_{i,j,q}^{n}k(Y_{i},Y_{j})l(X_{i},X_{q}).$
(103)

######  Lemma 1.

Let $Y$ be an absolutely continuous random variable and
$F:\mathbb{R}\rightarrow[0,1]$ be a CDF. Then, it holds that

$F(Y)\sim\operatorname{Uniform}(0,1)\iff Y\sim F.$

###### Proof.

First, assume $Y\sim F$. For all $t\in(0,1)$ define the generalized inverse
CDF by $F^{-1}(t)\coloneqq\inf\\{y\in\mathbb{R}\mid F(y)\geq t\\}$. Then, it
holds for all $t\in(0,1)$ that

$\mathbb{P}(F(Y)\leq t)=\mathbb{P}(Y\leq F^{-1}(t))=F(F^{-1}(t)).$ (104)

Next, since $Y$ is absolutely continuous, we know that $F$ is continuous. For
all $t\in(0,1)$, by the intermediate value theorem there must exist
$y^{*}_{t}\in\mathbb{R}$ such that $F(y^{*}_{t})=t$. Hence we get by the
definition of the generalized inverse that $F^{-1}(t)\leq y^{*}_{t}$, which
further implies that $F(F^{-1}(t))\leq F(y^{*}_{t})=t$. Since it also holds
that $F(F^{-1}(t))\geq t$, we get that $F(F^{-1}(t))=t$. Together with (104)
this implies that $F(Y)\sim\operatorname{Uniform}(0,1)$.

Second, assume that $F(Y)\sim\operatorname{Uniform}(0,1)$. Then, it holds for
all $y\in\mathbb{R}$ that

$\mathbb{P}(Y\leq y)=\mathbb{P}(F(Y)\leq F(y))=F(y)$

and hence $Y\sim F$. ∎

######  Definition 9 (Conditional independence, Dawid, 1979).

Let $X,Y,Z$ be random vectors. We say $X$ is conditionally independent of $Y$
given $Z$ and write $Y\perp\nolinebreak\perp X\mid Z$ if and only if, for all
bounded measurable $\phi$,

$\mathbb{E}[\phi(Y)\mid X,Z]=\mathbb{E}[\phi(Y)\mid Z].$

######  Lemma 2 (Properties of conditional independence).

Let $(A,B,C)\sim P$. Then, the following statements hold [Dawid, 1980]:

  * Decomposition:

$A\perp\nolinebreak\perp(B,C)\implies(A\perp\nolinebreak\perp
B)\land(A\perp\nolinebreak\perp C).$

  * Weak union:

$A\perp\nolinebreak\perp(B,C)\implies(A\perp\nolinebreak\perp B\mid
C)\land(A\perp\nolinebreak\perp C\mid B).$

  * Contraction:

$(A\perp\nolinebreak\perp B\mid C)\land(A\perp\nolinebreak\perp C)\implies
A\perp\nolinebreak\perp(B,C).$

######  Lemma 3.

Under Setting 1, it holds that $(N_{Y},H)\perp\nolinebreak\perp(D,Z)\mid H$.

###### Proof.

By Setting 1, we have

$(N_{Z},N_{D})\perp\nolinebreak\perp N_{Y}\mid N_{H}.$

By Dawid [1979, Lemma 4.1], it holds

$(N_{Z},N_{D})\perp\nolinebreak\perp N_{Y}\mid
N_{H}\iff(N_{Z},N_{D},N_{H})\perp\nolinebreak\perp(N_{Y},N_{H})\mid N_{H}.$

Now, since $f$ from Setting 1 is measurable, $H\coloneqq N_{H}$, $Z\coloneqq
N_{Z}$ and $D\coloneqq f(N_{Z},H,N_{D})$, we have

$\displaystyle(N_{Z},N_{D},N_{H})\perp\nolinebreak\perp(N_{Y},N_{H})\mid
N_{H}$
$\displaystyle\implies(N_{Z},f(N_{Z},N_{D},N_{H}))\perp\nolinebreak\perp(N_{Y},N_{H})\mid
N_{H}$ (105) $\displaystyle\implies(Z,D)\perp\nolinebreak\perp(N_{Y},H)\mid
H,$ (106)

which concludes the proof. ∎

######  Lemma 4 (Balance equation, Rosenbaum and Rubin, 1983).

In Setting 1, it holds that

$D\perp\nolinebreak\perp H\mid\pi(H),$

where $\pi(H)\coloneqq\mathbb{E}[D\mid H]$.

###### Proof.

By

$\mathbb{P}(D=1\mid H,\pi(H))=\mathbb{P}(D=1\mid H)=\pi(H),$

and

$\mathbb{P}(D=1\mid\pi(H))=\mathbb{E}[\mathbb{E}[\mathds{1}(D=1)\mid
H,\pi(H)]\mid\pi(H)]=\mathbb{E}[\pi(H)\mid\pi(H)]=\pi(H),$

we arrive at the desired conclusion. ∎

######  Lemma 5.

Let $X_{1},\dots,X_{n}$ be independent and identically standard uniformly
distributed random variables and let $X_{(1)},\dots,X_{(n)}$ denote their
order statistic. Then, it holds that

$\int_{0}^{1}\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(X_{i}\leq
u)-u\right)^{2}\,\mathrm{d}u=\frac{1}{12n^{2}}+\frac{1}{n}\sum_{i=1}^{n}\left(\frac{i-0.5}{n}-X_{(i)}\right)^{2}.$

###### Proof.

The result follows from the following computation

$\displaystyle\int_{0}^{1}\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(X_{i}\leq
u)-u\right)^{2}\,\mathrm{d}u$
$\displaystyle=\sum_{k=0}^{n}\int_{X_{(k)}}^{X_{(k+1)}}\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(X_{i}\leq
u)-u\right)^{2}\,\mathrm{d}u$
$\displaystyle=\sum_{k=0}^{n}\int_{X_{(k)}}^{X_{(k+1)}}\left(\tfrac{k}{n}-u\right)^{2}\,\mathrm{d}u$
$\displaystyle=\sum_{k=0}^{n}\left[-\tfrac{1}{3}(\tfrac{k}{n}-X_{(k+1)})^{3}+\tfrac{1}{3}(\tfrac{k}{n}-X_{(k)})^{3}\right]$
$\displaystyle=\sum_{k=1}^{n}\tfrac{1}{3}\left[(\tfrac{k}{n}-X_{(k)})^{3}-(\tfrac{k-1}{n}-X_{(k)})^{3}\right]$
$\displaystyle=\sum_{k=1}^{n}\tfrac{1}{3}\left[(\tfrac{k}{n})^{3}-(\tfrac{k-1}{n})^{3}\right]+\left[\tfrac{1}{n}X_{(k)}^{2}-\tfrac{2k-1}{n}X_{(k)}\right]$
$\displaystyle=\sum_{k=1}^{n}\tfrac{1}{12n^{3}}+\tfrac{1}{n}\left(\tfrac{2k-1}{2n}-X_{(k)}\right)^{2}$
$\displaystyle=\tfrac{1}{12n^{2}}+\frac{1}{n}\sum_{k=1}^{n}\left(\tfrac{k-0.5}{n}-X_{(k)}\right)^{2}.$

This completes the proof of Lemma 5. ∎

######  Lemma 6.

Let $X_{1},\dots,X_{n}$ be independent and identically copies of $X$ taking
values in $\mathbb{R}$. Then, it holds that

$\int_{0}^{1}\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(X_{i}\leq
u)-u\right)^{2}\,\mathrm{d}u\;\xrightarrow{\text{a.s.}}\;\int_{0}^{1}(\mathbb{P}(U\leq
u)-u)^{2}\,\mathrm{d}u\quad\text{as $n\rightarrow\infty$}.$

###### Proof.

For all $u\in(0,1)$ and $n\in\mathbb{N}$, define
$\widehat{F}_{n}(u)\coloneqq\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(X_{i}\leq u)$
and $F(u)\coloneqq\mathbb{P}(X\leq u)$. We then have,

$\displaystyle\quad\,\,\int_{0}^{1}\left(\widehat{F}_{n}(u)-u\right)^{2}\,\mathrm{d}u$
(107)
$\displaystyle=\int_{0}^{1}\left(\widehat{F}_{n}(u)-F(u)+F(u)-u\right)^{2}\,\mathrm{d}u$
(108)
$\displaystyle=\int_{0}^{1}(\widehat{F}_{n}(u)-F(u))^{2}+2(\widehat{F}_{n}(u)-F(u))(F(u)-u)+(F(u)-u)^{2}\,\mathrm{d}u$
(109)
$\displaystyle=\int_{0}^{1}(\widehat{F}_{n}(u)-F(u))^{2}\,\mathrm{d}u+2\int_{0}^{1}(\widehat{F}_{n}(u)-F(u))(F(u)-u)\,\mathrm{d}u$
(110) $\displaystyle\quad\quad\,\,+\ \int_{0}^{1}(F(u)-u)^{2}\,\mathrm{d}u$
(111)
$\displaystyle\leq\left(\sup_{u\in(0,1)}\left\lvert\widehat{F}_{n}(u)-F(u)\right\rvert\right)^{2}+\sup_{u\in(0,1)}\left\lvert\widehat{F}_{n}(u)-F(u)\right\rvert\left(\int_{0}^{1}(F(u)-u)^{2}\,\mathrm{d}u\right)^{1/2}$
(112) $\displaystyle\quad\quad\,\,+\ \int_{0}^{1}(F(u)-u)^{2}\,\mathrm{d}u.$
(113) $\displaystyle\leq
2\sup_{u\in(0,1)}\left\lvert\widehat{F}_{n}(u)-F(u)\right\rvert+\int_{0}^{1}(F(u)-u)^{2}\,\mathrm{d}u,$
(114)

where the last inequality uses that for all $u\in(0,1)$, $F(u)\in[0,1]$. By
the Glivenko–Cantelli theorem, it holds that,
$\sup_{u\in(0,1)}\left\lvert\widehat{F}_{n}(u)-F(u)\right\rvert\xrightarrow{\text{a.s.}}0,$
which concludes the proof of Lemma 6. ∎

######  Lemma 7.

Given the setting outlined in the proof of Proposition 3, there exists $K>0$,
such that, for all $\vartheta,\vartheta^{\prime}\in\Theta$ and all
$n\in\mathbb{N}$,

$\left\lvert U_{n}(\vartheta)-U_{n}(\vartheta^{\prime})\right\rvert\leq
K\left\lVert\vartheta-\vartheta^{\prime}\right\rVert\mbox{ and }\left\lvert
U(\vartheta)-U(\vartheta^{\prime})\right\rvert\leq
K\left\lVert\vartheta-\vartheta^{\prime}\right\rVert.$

###### Proof.

For all $\vartheta\in\Theta$ and all $n\in\mathbb{N}$, it holds that

$\displaystyle\left\lvert
U_{n}(\vartheta)-U_{n}(\vartheta^{\prime})\right\rvert$
$\displaystyle=\left\lvert\int_{0}^{1}\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(F_{D_{i}}(Y_{i};\vartheta)\leq
u)-u\right)^{2}-\left(\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(F_{D_{i}}(Y_{i};\vartheta^{\prime})\leq
u)-u\right)^{2}\,\mathrm{d}u\right\rvert$ (115) $\displaystyle\leq
2\int_{0}^{1}\left\lvert\frac{1}{n}\sum_{i=1}^{n}\mathds{1}(F_{D_{i}}(Y_{i};\vartheta)\leq
u)-\mathds{1}(F_{D_{i}}(Y_{i};\vartheta^{\prime})\leq
u)\right\rvert\,\mathrm{d}u$ (116) $\displaystyle\leq
2\int_{0}^{1}\frac{1}{n}\sum_{i=1}^{n}\left\lvert\mathds{1}(F_{D_{i}}(Y_{i};\vartheta)\leq
u)-\mathds{1}(F_{D_{i}}(Y_{i};\vartheta^{\prime})\leq
u)\right\rvert\,\mathrm{d}u$ (117)
$\displaystyle\leq\frac{2}{n}\sum_{i=1}^{n}\int_{0}^{1}\left\lvert\mathds{1}(F_{D_{i}}(Y_{i};\vartheta)\leq
u)-\mathds{1}(F_{D_{i}}(Y_{i};\vartheta^{\prime})\leq
u)\right\rvert\,\mathrm{d}u$ (118)
$\displaystyle\leq\frac{2}{n}\sum_{i=1}^{n}\left\lvert
F_{D_{i}}(Y_{i};\vartheta)-F_{D_{i}}(Y_{i};\vartheta^{\prime})\right\rvert$
(119)
$\displaystyle=\frac{2}{n}\sum_{i=1}^{n}\left\lvert\Phi(a_{D_{i}}(Y_{i})^{\top}\vartheta)-\Phi(a_{D_{i}}(Y_{i})^{\top}\vartheta^{\prime})\right\rvert$
(120) $\displaystyle\leq\frac{2}{\sqrt{2\pi}n}\sum_{i=1}^{n}\left\lvert
a_{D_{i}}(Y_{i})^{\top}(\vartheta-\vartheta^{\prime})\right\rvert$ (121)
$\displaystyle\leq\frac{2}{\sqrt{2\pi}n}\sum_{i=1}^{n}\left\lVert
a_{D_{i}}(Y_{i})^{\top}\right\rVert\cdot\left\lVert(\vartheta-\vartheta^{\prime})\right\rVert$
(122)
$\displaystyle\leq\frac{2A}{\sqrt{2\pi}}\left\lVert(\vartheta-\vartheta^{\prime})\right\rVert,$
(123)

where the first inequality follows because the integrand is positive and for
all $x,y\in[0,1]$ it holds that $|x^{2}-y^{2}|<2|x-y|$; the second inequality
follows by the triangle inequality; the fourth inequality follows by
evaluating the integral; the fifth inequality follows from
$\tfrac{1}{\sqrt{2\pi}}$-Lipschitz continuity of $\Phi$; the sixth inequality
follows from the Cauchy–Schwarz inequality; and, the seventh inequality
follows by $a_{D}(Y)$ being almost surely upper bounded by $A$.

Simiarly, for the popoulation $U$, it holds that

$\displaystyle\left\lvert U(\vartheta)-U(\vartheta^{\prime})\right\rvert$
$\displaystyle=\left\lvert\int_{0}^{1}(\mathbb{P}(F_{D}(Y;\vartheta)\leq
u)-u)^{2}\,\mathrm{d}u-\int_{0}^{1}(\mathbb{P}(F_{D}(Y;\vartheta^{\prime})\leq
u)-u)^{2}\,\mathrm{d}u\right\rvert$ (124) $\displaystyle\leq
2\int_{0}^{1}\left\lvert\mathbb{P}(F_{D}(Y;\vartheta)\leq
u)-\mathbb{P}(F_{D}(Y;\vartheta^{\prime})\leq u)\right\rvert\,\mathrm{d}u$
(125)
$\displaystyle=2\int_{0}^{1}\left\lvert\mathbb{E}[\mathds{1}(F_{D}(Y;\vartheta)\leq
u)-\mathds{1}(F_{D}(Y;\vartheta^{\prime})\leq u)]\right\rvert\,\mathrm{d}u$
(126) $\displaystyle\leq
2\mathbb{E}\left[\int_{0}^{1}\left\lvert\mathds{1}(F_{D}(Y;\vartheta)\leq
u)-\mathds{1}(F_{D}(Y;\vartheta^{\prime})\leq
u)\right\rvert\,\mathrm{d}u\right]$ (127)
$\displaystyle\leq\frac{2A}{\sqrt{2\pi}}\left\lVert(\vartheta-\vartheta^{\prime})\right\rVert,$
(128)

where the first inequality follows since for all $x,y\in[0,1]$ it holds that
$|x^{2}-y^{2}|<2|x-y|$; the second inequality follows by Jensen’s inequality
and the last inequality follows analogously to (119) and onward.

This completes the proof of Lemma 7. ∎

######  Lemma 8.

Let $X$ be an absolutely continuous real-valued random variable. Then, it
holds that

$\int_{0}^{1}(\mathbb{P}(X\leq u)-u)^{2}\,\mathrm{d}u=0\iff
X\sim\operatorname{Uniform}(0,1).$

###### Proof.

The backward direction is immediate by $\mathbb{P}(X\leq u)=u$. For the
forward direction,

$\int_{0}^{1}(\mathbb{P}(X\leq
u)-u)^{2}\,\mathrm{d}u=0\implies\mathbb{P}(X\leq u)-u=0\mbox{ a.e.}\implies
X\sim\operatorname{Uniform}(0,1),$

where the first implication follows because the $L_{2}$ norm of a function is
zero if and only if the function zero almost everywhere and the second
implication follows by right-continuity of
$\mathbb{P}(X\leq\boldsymbol{\cdot})$, absolute continuity of $X$ and the
uniqueness of the CDF. ∎