# Invariant Representation Learning for Treatment Effect Estimation

Claudia Shi  Columbia University  Victor Veitch  Google Research  The
University of Chicago  David M. Blei  Columbia University

###### Abstract

The defining challenge for causal inference from observational data is the
presence of ‘confounders’, covariates that affect both treatment assignment
and the outcome. To address this challenge, practitioners collect and adjust
for the covariates, hoping that they adequately correct for confounding.
However, including every observed covariate in the adjustment runs the risk of
including ‘bad controls’, variables that _induce_ bias when they are
conditioned on. The problem is that we do not always know which variables in
the covariate set are safe to adjust for and which are not. To address this
problem, we develop Nearly Invariant Causal Estimation (NICE). NICE uses
invariant risk minimization (IRM) [1] to learn a representation of the
covariates that, under some assumptions, strips out bad controls but preserves
sufficient information to adjust for confounding. Adjusting for the learned
representation, rather than the covariates themselves, avoids the induced bias
and provides valid causal inferences. We evaluate NICE on both synthetic and
semi-synthetic data. When the covariates contain unknown collider variables
and other bad controls, NICE performs better than adjusting for all the
covariates. Code is available at github.com/claudiashi57/nice.

##  1 Introduction

Consider the following causal inference problem.

We want to estimate the effect of sleeping pills on lung disease using
electronic health records, collected from multiple hospitals around the world.
For each hospital $e$ and patient $i$, we observe whether the drug was
administered $T^{e}_{i}$, the patient’s outcome $Y^{e}_{i}$, and their
covariates $X_{i}^{e}$, which includes comprehensive health and socioeconomic
information. The different hospitals serve different populations, so the
distribution of the covariate $X^{e}$ is different across the datasets. But
the causal mechanism between sleeping pills $T^{e}$ and lung disease $Y^{e}$
remains the same across hospitals.

The data in this example are observational. One challenge to causal inference
from observational data is the presence of confounding variables that
influence both $T$ and $Y$ [35, 27]. To account for confounding, we try to
find them among the covariates $X$ and then adjust for them, e.g., using a
method like G-computation [32], backdoor adjustment [28], or inverse
propensity score weighting [3]. The selected covariates are called the
adjustment set.

To ensure that we have adjusted for all confounding variables, we might
include every covariate in the adjustment set. However, naively adjusting for
all covariates runs the risk of including “bad controls” [5, 28, 7], variables
that _induce_ bias when they are adjusted for. In the example, a health
condition caused by lung disease would be a bad control. It is causally
affected by the outcome.

How can we exclude bad controls from the adjustment set? One approach is to
select confounders through a causal graph [28]. We ask a domain expert to
construct a causal graph or a class of equivalent graphs. We then select the
confounders for the causal adjustment. However, in practice, we may have
thousands of covariates in the dataset. It may be too difficult to construct a
graph with thousands of nodes.

Another approach is to restrict the adjustment set to those that are known to
be pre-treatment covariates [34, 37]. However, this approach can lead us to
include covariates that are predictive of treatment assignment but not the
outcome. If the record is sufficiently rich, this information can lead to
near-perfect prediction of treatment, which is a problem for causal inference.
Specifically, this creates an apparent violation of overlap, the requirement
that each unit had a non-zero probability of receiving treatment [9].
Practically, near-violations of overlap can lead to unstable or high-variance
estimates of treatment effects [10].

But these methods, and their challenges, suggest a new approach for causal
estimation — we want a representation of the covariates that contains
sufficient information for causal adjustment, excludes bad controls, and helps
provide low-variance causal estimates. This paper presents a method to find
such a representation.

##### Problem.

We now state the problem plainly. We want to do causal inference with data
collected from multiple environments, as in the hospitals’ example above. The
observed covariates are rich — including all the causal parents of the
outcome. There are no unobserved confounders, but identifiability [27] or
strong ignorability [35] is _not_ guaranteed, due to the possible existence of
bad controls. We do not know which covariates are safe to adjust for. The main
question is: how can we use the multiple environments to find a representation
of the covariates for valid causal estimation?

To address this question, we develop nearly invariant causal estimation
(NICE), an estimation procedure for causal inference from observational data
where the data comes from multiple datasets. The datasets are drawn from
distinct environments, corresponding to distinct distributions of the
covariates.

NICE applies Invariant Risk Minimization (IRM) [1] for causal adjustment. IRM
is a framework for solving prediction problems. The goal is produce a
predictor that is robust to changes in the deployment domain. The IRM
procedure uses data from multiple environments to learn an invariant
representation $\Phi(T,X)$, a function such that the outcome $Y$ and the
representation of the treatment and covariates $\Phi(T,X)$ have the same
relationship in each environment. Predictors built on top of this
representation will have the desired robustness.

The main insight that enables NICE is that the IRM invariant representation
also suffices for causal adjustment. Informally, a representation is invariant
if and only if it is informationally equivalent to the causal parents of the
outcome $Y$ [1]. For example, an invariant representation of the medical
records will isolate the causal parents of lung disease. Assuming _no
mediators_ — variables on the causal path between the treatment and outcome —
in the covariate set, the causal parents of $Y$ constitute an adjustment set
that suffices for causal adjustment, minimally impacts overlap, and that
excludes all bad controls. Hence, adjusting for an invariant representation is
a safe way to estimate the causal effect.111To keep the exposition simple, we
defer the discussion of mediators to the appendix.

##### Contributions.

This paper develops NICE, an estimation procedure that leverages data from
multiple environments to do causal inference. It articulates the theoretical
conditions under which NICE provides unbiased causal estimates and evaluates
the method on synthetic and semi-synthetic causal estimation problems.

##  2 Related Work

Estimating the treatment effect from observational data consists of
identification and estimation. The motivating problem is related to
identification — we do not know what covariates to adjust for.

In the introduction, we discussed two widely applied adjustment approaches:
selecting covariates basing on a causal graph [27] and restricting to
covariates that are known to be pre-treatment [34, 37]. Another approach to
select the adjustment set is through causal discovery.

Causal discovery methods aim to recover causal relationships or causal
direction from data [25, 42, 40, 13, 41, 30, 24, 17]. In particular, NICE
shares the same setup as invariance based causal discovery methods. [30, 17,
31] leverage multiple environments to find the causal predictors of the target
variable in the linear, non-linear, and time series settings.

Causal discovery assumes that the observed covariates correspond to well-
defined variables in the causal graph (e.g., no measurement issues). The
representation learning approach of NICE does not require this assumption.
Further, even in the case where this assumption holds, causal discovery
methods are designed to conservatively select parents of $Y$. In practice,
they often fail to select many actual parents. In Section 5.2, we show that
while the causal discovery method [30] is better at stripping out bad
controls, it also discards confounders, which leads to poor estimation
quality.

With identification, we can then estimate the treatment effect. There is
extensive literature on different statistical estimators [3, 14, 43, 12] and
machine learning methods adapted for causal inference [18, 2, 4, 16, 38, 22,
44, 39]. All these estimators and methods assume identification and focus on
improving the finite sample estimation quality. In contrast, NICE considers a
setting where identification is not guaranteed.

NICE uses the principle of invariance to solve a causal inference problem. A
thread of related work uses the same principle to tackle different problems.

The principle of invariance is: if a relationship between $X$ and $Y$ is
causal, then it is invariant to perturbations that changes the distributions
of $X$. Conversely, if a relationship is invariant to many different
perturbations, it’s likely be causal [15, 6]. This principle inspired a line
of causality-based domain adaptation and robust prediction work.

[33] apply the idea for causal transfer learning, assuming the conditional
distribution of the target variable given some subset of covariates is the
same across domains. [23] relax that assumption. [30, 17] apply this principle
for causal variable selection from multiple environments. [45] recast the
problem of domain adaptation as a problem of Bayesian inference on the
graphical models. [1] advocate a new generalizable statistical learning
principle that is based on the invariant principle. [36] critically examined
the generalizability of the proposed principle and its implementations.

These works focus of _robust prediction_. NICE focuses _causal estimation_.
NICE is complementary as it studies the idea of applying domain adaptation
methods for causal estimation. In particular, we focus on the application of
IRM for treatment effect estimation.

##  3 Nearly Invariant Causal Estimation

$X^{e}_{cn}$$X^{e}_{an}$$X^{e}_{pa}$$T^{e}$$Y^{e}$$X^{e}_{cl}$$X^{e}_{de}$
Figure 1: If the composition of $X^{e}$ is unknown, the treatment effect
cannot be identified. (cn=confounder, cl=collider, pa=parents, an=ancestors,
de=descendants)

We observe multiple datasets. Each dataset is from an environment $e$, in
which we observe a treatment $T^{e}$, an outcome $Y^{e}$, and other variables
$X^{e}$, called covariates. Assume each environment involves the same causal
mechanism between the causal parents of $Y^{e}$ and $Y^{e}$, but otherwise
might be different from the others, e.g., in the distribution of $X^{e}$.
Assume we have enough information in $X^{e}$ to estimate the causal effect,
i.e., it contains a set of variables sufficient for adjustment. But we do not
know the status of each covariate in the causal graph. A covariate might be an
ancestor, confounder, collider, parent, or descendant. Figure 1 shows an
example graph that defines these terms.

Each environment is a data generating process (DGP) with a causal graph and an
associated probability distribution $P^{e}$. The data from each environment is
drawn i.i.d., $\\{X^{e}_{i},T^{e}_{i},Y^{e}_{i}\\}\overset{\mathrm{iid}}{\
\sim\ }P^{e}$. The causal mechanism relating $Y$ to $T$ and $X$ is assumed to
be the same in each environment. In the example from the introduction,
different hospitals constitute different environments. All the hospitals share
the same causal mechanism for lung disease, but they vary in the population
distribution of who they serve, their propensity to prescribe sleeping pills,
and other aspects of the distribution.

The goal is to estimate the _average treatment effect on the treated_
(ATT)222For simple exposition, we focus on the ATT estimation. The method can
also be applied to conditional average treatment effect or average treatment
estimations. in each environment,

$\displaystyle\begin{split}\psi^{e}\triangleq\mathbb{E}\left[Y^{e}\,|\,\mathrm{do}(T^{e}=1),T^{e}=1\right]\\\
-\mathbb{E}\left[Y^{e}\,|\,\mathrm{do}(T^{e}=0),T^{e}=1\right].\end{split}$
(3.1)

The use of do notation [27] indicates that the estimand is causal. The ATT is
the difference between _intervening_ by assigning the treatment and
intervening to prevent the treatment, averaged over the people who were
actually assigned the treatment. The causal effect for any given individual
does not depend on the environment. However, the ATT does depend on the
environment because it averages over different populations of individuals.

###  3.1 Causal estimation

For the moment, consider one environment. In theory, we can estimate the
effect by adjusting for the confounding variables that influence both $T$ and
$Y$ [35]. Let $Z(X)$ be an admissible subset of $X$—it contains no descendants
of $Y$ and blocks all “backdoor paths” between $Y$ and $T$ [29]. An admissible
subset in Figure 1 is any that includes $X_{cn}$ but excludes $X_{cl}$ and
$X_{de}$. Using $Z(X)$, the causal effect can be expressed as a function of
the observational distribution,

$\displaystyle\begin{split}\psi&=\mathbb{E}_{X}[\mathbb{E}_{Y}\left[Y\,|\,T=1,Z(X)\right]\\\
&-\mathbb{E}_{Y}\left[Y\,|\,T=0,Z(X)\right]\,|\,T=1].\end{split}$ (3.2)

We estimate $\psi$ in two stages. First, we fit a model $\hat{Q}$ for the
conditional expectation $Q(T,Z(X))=\mathbb{E}_{Y}\left[Y\,|\,T,Z(X)\right]$.
Second, we use Monte Carlo to approximate the expectation over $X$,

$\displaystyle\hat{\psi}=\frac{1}{\sum_{i}t_{i}}\sum_{i:t_{i}=1}\left(\hat{Q}(1,Z(X_{i}))-\hat{Q}(0,Z(X_{i}))\right),$
(3.3)

The function $\hat{Q}$ can come from any model that predicts $Y$ from
$\\{T,Z(X)\\}$.

If the causal graph is known then the admissible set $Z(X)$ can be easily
selected and the estimation in Eq. 3.2 is straightforward. But here we do not
know the status of each covariate—if we inadvertently include bad controls in
$Z(X)$ then we will bias the estimate. To solve this problem, we develop a
method for learning an admissible representation $\Phi(T,X)$, which is learned
from datasets from multiple environments. An admissible representation is a
function of the full set of covariates but one that captures the confounding
factors and excludes the bad controls, i.e., the descendants of the outcome
that can induce bias.333An admissible representation is analogous to an
‘admissible set’ [27], which is a valid adjustment set. Given the
representation, we estimate the conditional expectations
$\mathbb{E}_{Y}\left[Y\,|\,\Phi(T,X)\right]$ and proceed to estimate the
causal effect.

###  3.2 Invariant Risk Minimization

To learn an admissible representation, we use IRM. IRM is a framework for
learning predictors that perform well across many environments. We first
review the main ideas of IRM and then adapt it to causal estimation.

Each environment is a causal structure and probability distribution.
Informally, for an environment to be valid, it must preserve the causal
mechanism relating the outcome and the other variables.

######  Definition 3.1 (Valid environment [1]).

Consider a causal graph $\mathcal{G}$ and a distribution $P(X,T,Y)$ respecting
$\mathcal{G}$. Let $\mathcal{G}_{e}$ denote the graph under an intervention
and $P^{e}=P(X^{e},T^{e},Y^{e})$ be the distribution induced by the
intervention. The intervention can be either atomic or stochastic. An
intervention is valid with respect to $(\mathcal{G},P)$ if (i)
$\mathbb{E}_{P^{e}}\left[Y^{e}|Pa(Y)\right]=\mathbb{E}_{P}\left[Y|Pa(Y)\right]$,
and (ii) $V(Y^{e}|Pa(Y))$ is finite. An environment is valid with respect to
$(\mathcal{G},P)$ if it can be created by a valid intervention.

Given this definition, a natural notion of an invariant representation is one
where the conditional expectation of the outcome is the same regardless of the
environment.

######  Definition 3.2 (Invariant representation).

A representation $\Phi(T,X)$ is invariant with respect to environments
$\mathcal{E}$ if and only if
$\mathbb{E}\left[Y^{e_{1}}|\Phi(T^{e_{1}},X^{e_{1}})=\pi\right]=\mathbb{E}\left[Y^{e_{2}}|\Phi(T^{e_{2}},X^{e_{2}})=\pi\right]$
for all $e_{1},e_{2}\in\mathcal{E}$.

arjovsky2019invariant recast the problem of finding an invariant
representation as one about prediction. In this context, the goal of IRM is to
learn a representation such that there is a single classifier $w$ that is
optimal in all environments. Thus IRM seeks a composition
$w\circ\Phi(T^{e},X^{e})$ that is a good estimate of $Y^{e}$ in the given set
of environments. This estimate is composed of a representation $\Phi(T,X)$ and
a classifier $w$ that estimates $Y$ from the representation.

######  Definition 3.3 (Invariant representation via predictor [1]).

A data representation $\Phi:\mathcal{X}\rightarrow\mathcal{H}$ elicits an
invariant predictor across environments $\mathcal{E}$ if there is a classifier
$w:\mathcal{H}\rightarrow\mathcal{Y}$ that is simultaneously optimal for all
environments. That is,

$\displaystyle
w\in\operatorname*{arg\,min}_{\bar{w}:\mathcal{H}\rightarrow\mathcal{Y}}R^{e}(\bar{w}\circ\Phi)\quad\textrm{for
all \,}e\in\mathcal{E},$ (3.4)

where $R^{e}$ is the the training objective’s risk in environment $e$.

The invariant representations in Definitions 3.2 and 3.3 align if we choose a
loss function for which the minimizer of the associated risk in Eq. 3.4 is a
conditional expectation. (Examples include squared loss and cross entropy
loss.) In this case, we can find an invariant predictor
$Q^{\mathrm{inv}}=w\circ\Phi(T^{e},X^{e})=\mathbb{E}\left[Y\,|\,\Phi(T,X)\right]$
by solving Eq. 3.4 for both $w$ and $\Phi$.

However, the general formulation of Eq. 3.4 is computationally intractable,
[1] introduce IRMv1 as a practical alternative.

######  Definition 3.4 (IRMv1[1] ).

IRMv1 is:

$\hat{\Phi}=\operatorname*{arg\,min}_{\Phi}\sum_{e\in\mathcal{E}}R^{e}(1.0\cdot\Phi)+\lambda\parallel\nabla_{w|w=1.0}R^{e}(w\cdot\Phi)\parallel^{2}.$
(3.5)

Notice here, IRMv1 fixes the classifier to the simplest possible choice:
multiplication by the scalar constant $w=1.0$. The task is then to learn a
representation $\Phi$ such that $w=1.0$ is the optimal classifier in all
environments. In effect, $\Phi$ becomes the invariant predictor, as
$Q^{\mathrm{inv}}=1.0\cdot\Phi$. The gradient norm penalizes model deviations
from the optimal classifier in each environment $e$, enforcing the invariance.
The hyperparameter $\lambda$ controls the trade-off between invariance and
predictive accuracy.444For details on IRMv1, see [1, section 3.1]

In practice, we parameterize $\Phi$ with a neural network that takes
$\\{t^{e}_{i},x^{e}_{i}\\}$ as input and outputs a real number. Let $\ell$ be
a loss function, such as squared error or cross entropy, and $n_{e}$ be the
number of units sampled in environment $e$. Then, we learn $\hat{\Phi}$ by
solving IRMv1 where each environment risk is replaced with the corresponding
empirical risk:

$\hat{R}^{e}(Q)=\frac{1}{n_{e}}\sum_{i}\ell(y^{e}_{i},Q(t^{e}_{i},x^{e}_{i})).$
(3.6)

$\hat{Q}^{\mathrm{inv}}=1.0\cdot\hat{\Phi}$ is an empirical estimate of
$\mathbb{E}\left[Y|\Phi(T,X)\right]$.

###  3.3 Nearly Invariant Causal Estimation

We now introduce nearly invariant causal estimation (NICE). NICE is a causal
estimation procedure that uses data collected from multiple environments. NICE
exploits invariance across the environments to perform causal adjustment
without detailed knowledge of which covariates are bad controls.

Informally, the key connection between causality and invariance is that if a
representation is invariant across all valid environments then the information
in that representation is the information in the causal parents of $Y$. Since
the causal structure relevant to the outcome is invariant across environments,
a representation capturing only the causal parents will also be invariant. We
can see that $\mathrm{Pa}(Y)$ is the minimal information required for
invariance. A representation that is invariant over all valid environments
will be minimal; hence, an invariant representation must capture only the
parents of $Y$.

NICE is based on two insights. First, as just explained, if $\Phi(T,X)$ is
invariant over all valid environments, then
$\mathbb{E}\left[Y|T,\mathrm{Pa}(Y)\setminus\\{T\\}\right]=\mathbb{E}\left[Y|\Phi(T,X)\right]$.
Second, $\mathrm{Pa}(Y)\setminus\\{T\\}$ suffices for causal adjustment. That
is, $\mathrm{Pa}(Y)\setminus\\{T\\}$ blocks any backdoor paths and does not
include bad controls. Following Eq. 3.2,

$\displaystyle\psi$ $\displaystyle=\mathbb{E}[\mathbb{E}\left[Y\mid
T=1,\mathrm{Pa}(Y)\setminus\\{T\\}\right]$ (3.7)
$\displaystyle-\mathbb{E}\left[Y\mid
T=0,\mathrm{Pa}(Y)\setminus\\{T\\})\right]\,|\,T=1]$

Since
$\mathbb{E}\left[Y|T,\mathrm{Pa}(Y)\setminus\\{T\\}\right]=\mathbb{E}\left[Y|\Phi(T,X)\right]$,

$\displaystyle\psi=\mathbb{E}\left[\mathbb{E}\left[Y\mid\Phi(1,X)\right]-\mathbb{E}\left[Y\mid\Phi(0,X)\right]\,|\,T=1\right].$
(3.8)

Recall the invariant predictor
$Q^{\mathrm{inv}}(T,X)=\mathbb{E}\left[Y\mid\Phi(T,X)\right]$. The NICE
procedure is

  1. 1.

Input: multiple datasets
$\mathcal{D}_{e}:=\\{(X_{i}^{e},Y_{i}^{e},T_{i}^{e})\\}^{n_{e}}_{i=1}$.

  2. 2.

Estimate the invariant predictor $\hat{Q}^{\mathrm{inv}}=1.0\cdot\hat{\Phi}$
using an invariant objective, such as IRMv1.

  3. 3.

Compute
$\hat{\psi}^{e}=\frac{1}{\sum_{i}t^{e}_{i}}\sum\limits_{i:t^{e}_{i}=1}\hat{Q}^{\mathrm{inv}}(1,x^{e}_{i})-\hat{Q}^{\mathrm{inv}}(0,x^{e}_{i})$
for each environment $e$.

Similar to the function $\hat{Q}$ in Eq. 3.2, $\hat{Q}^{\mathrm{inv}}$ can
come from any prediction model that uses an invariant objective. In Section 5,
we use linear regression, TARNet [38], and Dragonnet [39].

We call the procedure ‘nearly’ invariant as we only ever have access to a
limited number of environments, so we cannot be certain that we’ll achieve
invariance across all valid environments.

##  4 Justification of NICE

We now establish the validity of NICE as a causal estimation procedure. All
proofs are in the appendix.

First consider the case where we observe data from a sufficiently diverse set
of environments that the learned representation is invariant across all valid
environments. We prove that conditioning on a fully invariant representation
is the same as conditioning on the parents of $Y$.

######  Lemma 4.1.

Suppose that
$\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)=a\right]\neq\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)=a^{\prime}\right]$
whenever $a\neq a^{\prime}$.Then a representation $\Phi$ is invariant across
all valid environments if and only if
$\mathbb{E}\left[Y^{e}|\Phi(T^{e},X^{e})\right]=\mathbb{E}\left[Y|\mathrm{Pa}(Y)\right]$
for all valid environments.

Lemma 4.1 helps show that a representation that elicits an invariant predictor
suffices for adjustment.

######  Theorem 4.2.

Let $L$ be a loss function such that the minimizer of the associated risk is a
conditional expectation, and let $\Phi$ be a representation that elicits a
predictor $Q^{\mathrm{inv}}$ that is invariant for all valid environments.
Assuming $X^{e}$ does not contain mediators between the treatment and the
outcome, then
$\psi^{e}=\mathbb{E}\left[Q^{\mathrm{inv}}(1,X^{e})-Q^{\mathrm{inv}}(0,X^{e})|T^{e}=1\right]$.

Theorem 4.2 shows that the NICE estimand is equal to the ATT as long as the
predictor $Q^{\mathrm{inv}}$ is invariant across all valid environments.

In practice, if a predictor is invariant across a limited set of diverse
environments, it may generalize to all valid environments. Assuming a linear
data generating process, [1] establish sufficient conditions on the number and
diversity of the training environments such that the learned representation
generalizes to all valid environments. In the non-linear case, there are no
known sufficiency results. However, [1] give empirical evidence that access to
even a few environments may suffice.555Establishing the sufficiency result of
IRM is an open question.

In addition to identifiability, non-parametric estimation of treatment effects
with finite data, i.e., Eq. 3.3, requires ‘positivity’ or ‘overlap’ – both
treatment and non-treatment have a non-zero probability for all levels of the
confounders [35, 19]. Let $\Phi(X^{e})$ be the covariate representation, i.e.,
$\Phi(X^{e})=\\{\Phi(T^{e}=1,X^{e}),\Phi(T^{e}=0,X^{e})\\}$, in the following
theorem, we establish that if the covariate set $X$ is sufficient for overlap,
then $\Phi(X^{e})$ is sufficient for overlap.

######  Theorem 4.3.

Suppose $\epsilon\leq P(T^{e}=1|X^{e})\leq 1-\epsilon$ with probability 1,
then $\epsilon\leq P(T^{e}=1|\Phi(X^{e}))\leq 1-\epsilon$ with probability 1.

$X_{1}^{e}$$T^{e}$$Y^{e}$$X_{2}^{e}$ (a) Noise

$X_{1}^{e}$$T^{e}$$Y^{e}$$X_{2}^{e}$ (b) Descendant

$X_{1}^{e}$$T^{e}$$Y^{e}$$X_{2}^{e}$ (c) Collider

Figure 2: We observe $\\{X_{1},X_{2}\\}$, but do not know its composition. In
(a) $\\{X_{1},X_{2}\\}$ is a valid adjustment. In (b) and (c), $X_{2}$ is
downstream of $Y$, so $\\{X_{1},X_{2}\\}$ is not a valid adjustment.

The intuition is that the richer the covariate set is, the more likely it is
to accurately predict the treatment assignment [9]. The covariate
representation $\Phi(X^{e})$, by definition, contains less information than
$X^{e}$, therefore $\Phi(X^{e})$ satisfies overlap if $X^{e}$ satisfies
overlap.

Even when invariance across all valid environments is not guaranteed, NICE may
still improve the estimation quality when there are possible colliders in the
adjustment set. If the observed environments are induced by valid
interventions, either atomic or stochastic, on the bad controls, an invariant
representation over these environments can also exclude bad controls. Even
when the representation does not exclude the bad controls, invariance may
remove at least some (if not all) collider dependence. Intuitively,
conditioning on a subset of collider information should reduce bias in the
resulting estimate. Theorem 7.1 in the appendix shows that this intuition
holds for at least one illustrative causal structure. A fully general
statement remains open.

##### The case of mediators.

So far, we assumed the observed covariate set $X$ does not contain mediators
between $T$ and $Y$. What happens to the interpretation of the learned
parameter, $\hat{\psi}^{e}$, if the adjustment set contains mediators?

Intuitively, NICE captures the information in the direct link between $T$ and
$Y$. Concretely, if there are no mediators, the parameter reduces to ATT. If
there are mediators but no confounders, the parameter reduces to the Natural
Direct Effect [27]. If there are mediators and confounders, we define the
parameters as the natural direct effect on the treated (NDET). The mathematics
definitions are in the appendix.

##  5 Empirical Studies

We study the performance of NICE with three experiments. We are interested in
three empirical questions: (1) Does NICE strip out bad controls in practice?
(2) Is NICE "cost-less" when there are no bad controls? (3) What is the effect
of different amounts of environmental variation on NICE’s performance.

We find that (1) when there are bad controls in the adjustment set, NICE can
reduce bias induced by the bad controls. (2) When there are no bad controls in
the adjustment set, NICE does not hurt the estimation quality. (3) Whether
NICE can strip out bad controls depends on the diversity of the environments.
The more diverse the environments, the more likely it is that NICE can strip
out the bad controls.

###  5.1 Experimental Setup

We construct three experiments corresponding to different settings. We first
consider the setting where NICE is theoretically guaranteed to strip out bad
controls. In Section 5.2, the data are collected from diverse environments,
and the DGPs are linear. In the non-linear setting, there are no known
sufficiency results for the generalizability of IRM. Therefore, there is no
theoretical guarantee that NICE can strip out bad controls.

To study whether NICE can reduce bias from bad controls empirically, we
validate NICE using non-linear semi-synthetic benchmark datasets in Section
5.3. Furthermore, we study the effect of different amounts of environmental
variation on NICE’s performance in Section 5.4.

##### Causal Estimands & Evaluation metrics.

We consider two estimands: the sample average treatment effect on the treated
(SATT),
$\psi_{s}=\frac{1}{\sum_{i}t_{i}}\sum_{i:t_{i}=1}\left(Q(1,Z(x_{i}))-Q(0,Z(x_{i}))\right)$
and the conditional average treatment effect (CATE),
$\tau(x_{i})=Q(1,Z(x_{i}))-Q(0,Z(x_{i}))$ [19]. For the SATT, the evaluation
metric is the mean absolute error (MAE),
$\epsilon_{att}=|\hat{\psi}_{s}-\psi_{s}|$. For the CATE, the metric is the
Precision in Estimation of Heterogeneous Effect (PEHE)
$\mathbf{\epsilon_{\text{PEHE}}}=\frac{1}{n}\sum_{0}^{n}(\hat{\tau}(x_{i})-\tau(x_{i}))^{2}$
[18]. PEHE reflects the ability to capture individual variation in treatment
effects. The main paper shows the MAE of the SATT averaged across
environments. For the evaluation of CATE, see the appendix.

Figure 3: NICE strips out bad controls, which leads to better downstream
treatment effect estimation. ICP (causal discovery) strips out bad controls,
but also useful confounders (see fig. 4). The non-causal error is measured by
the mean square error of the weights on $X_{2}$. Lower is better.

##### Predictor Choices.

Under the NICE procedure, the invariant predictor $\hat{Q}^{inv}$ can be any
class of predictor trained with an IRMv1 objective. In the linear settings, we
use OLS-2 as the predictor. OLS-2 is linear regression with two separate
regressors for the treated and the control population.

In the nonlinear settings, we consider two neural network models similar to
the structure of TARNet [38] and Dragonnet [39]. TARNet is a two-headed model
with a shared representation $Z(X)\in R^{p}$, and two heads for the treated
and control representation. The network has 4 layers for the shared
representation and 3 layers for each expected outcome head. The hidden layer
size is 250 for the shared representation layers and 100 for the expected
outcome layers. We use Adam [21] as the optimizer, set the learning rate as
0.001, and an l2 regularization rate of 0.0001. For Dragonnet, there’s an
additional treatment head, which makes treatment prediction from the shared
representation. For the hyper-parameter $\lambda$ used in IRMv1, we use
$\lambda=10$ in the linear settings and $\lambda=100$ in the non-linear
settings.

Since we use the same predictor across different DGPs, the hyper-parameters
are chosen arbitrarily. We use data from all environments to train and
evaluate the predictor. The main paper presents results using TARnet. Results
derived from Dragonnet are in the appendix.

Figure 4: NICE reduces bias when the adjustment set contains bad controls and
does not hurt if the adjustment set is valid. We use ICP for the causal
discovery method, which is often too conservative. When ICP returns an empty
set, estimated causal effect is zero. The figure reports average MAE and
standard error of the SATT over 10 simulations.

##### Adjustment Schemes.

We compare estimation quality produced by the following adjustment schemes:
(1) adjusting for all covariates, (2) NICE, and (3) causal variable selection.
Under (1), we pool the data across environments to fit a predictor $\hat{Q}$
and compute SATT using Eq. 3.2. Under (3), we first use Invariant Causal
Prediction (ICP) [30] to select an adjustment set. ICP is a variable selection
method that identifies the target variable’s causal parents by leveraging data
from multiple environments. We then pool data across environments, use the
adjustment set to fit a predictor and compute SATT using Eq. 3.2. The
estimation procedure of NICE is described in Section 3.3.

###  5.2 NICE in linear settings

We simulate data with the three causal graphs in Figure 2. With a slight abuse
of notation, each intervention $e$ generates a new environment $e$ with
interventional distribution $P(X^{e},T^{e},Y^{e})$. $T^{e}$ is the binary
treatment and $Y^{e}$ is the outcome. $X^{e}$ is a 10-dimensional covariate
set that differs across DGPs. $X^{e}=(X^{e}_{1},X^{e}_{2})$, where $X^{e}_{1}$
is a five-dimensional confounder. $X^{e}_{2}$ is either noise, a descendant,
or a collider in each DGP. The DGPs are:

$\displaystyle X_{1}^{e}$ $\displaystyle\leftarrow\mathcal{N}(0,e^{2})$
$\displaystyle T^{e}$ $\displaystyle\leftarrow
Bern(\text{sigmoid}(X_{1}^{e}\cdot w_{xt^{e}}+\mathcal{N}(0,1)))$
$\displaystyle\tau$ $\displaystyle\leftarrow 5+\mathcal{N}(0,\sigma^{2})$
$\displaystyle Y^{e}$ $\displaystyle\leftarrow X_{1}^{e}\cdot
w_{xy^{e}}+T^{e}\cdot\tau+\mathcal{N}(0,e^{2})$

In (a) $X_{2}^{e}\leftarrow\mathcal{N}(0,1)$, in (b) $X_{2}^{e}\leftarrow
e*Y^{e}+\mathcal{N}(0,1)$, and in (c) $X_{2}^{e}\leftarrow
e*Y^{e}+T^{e}+\mathcal{N}(0,1)$.

For evaluation, following [1], we create three environments
$\mathcal{E}=\\{0.2,2,5\\}$. We ran 10 simulations. In each simulation, we
draw 1000 samples from each environment. We consider two types of variations:
(1) whether the observed covariates $S(X)$ are scrambled versions of the true
covariates $X$. If scrambled, $S$ is an orthogonal matrix. If not scrambled,
$S$ is an identity matrix. (2) whether the treatment effects are
heteroskedastic across environments. In the heteroskedastic setting
$\tau\leftarrow 5+\mathcal{N}(0,e^{2})$. In the environment-level
homoskedastic setting $\tau\leftarrow 5+\mathcal{N}(0,1)$.

We compare the estimation quality produced by four different adjustment
approaches: (1) adjusting for all covariates, (2) causal variable selection,
(3) NICE, and (4) No adjustment. The results in Figure 4 and Figure 3 are
under the unscrambled and heteroskedastic variant. The results of the other
variants are in the appendix.

##### Analysis.

Figure 4 reports the average of the MAE of SATT estimates over all three
environments. We observe that when the covariate set does not include bad
controls—simulation setting (a)— NICE performs as well as adjusting for all
covariates. When the covariate set includes bad controls that are closely
related to the outcome, that is (b) and (c), NICE can help reduce the
estimation bias.

To understand why NICE reduces the estimation bias, we look at the weights of
the control predictor. Ideally, the weights that correspond to the bad
controls should be 0. As shown in Figure 3, the predictor trained an IRMv1
objective places less weight on the bad controls than the predictor using
empirical risk minimization. We observe ICP successfully strips out most of
the bad controls. However, it produces worse causal estimates as it also
strips out confounders. We believe that this is because (1) the amount of
noise in the DGP is non-trivial, and (2) in some settings, the observed
covariates are scrambled versions of the true covariates. The result suggests
that while ICP is a robust causal discovery method, it should not be used for
downstream estimation. A similar observation is made in [46], where slight
perturbations on ICP’s assumptions might lead to poor performance.

###  5.3 NICE in non-linear settings

We validate NICE for the non-linear case on a benchmark dataset, SpeedDating.
SpeedDating was collected to study the gender difference in mate selection
[11]. The study recruited university students to participate in speed dating,
and collected objective and subjective information such as ‘undergraduate
institution’ and ‘perceived attractiveness’. It has 8378 entries and 185
covariates. ACIC 2019’s simulation samples subsets of the covariates to
simulate binary treatment $T$ and binary outcome $Y$. Specifically, it
provides four modified DGPs: Mod1: parametric models; Mod2: complex models;
Mod3: parametric models with poor overlap; Mod4: complex models with treatment
heterogeneity. Each modification includes three versions: low, med, high,
indicating an increasing number of covariates included in the models for $T$
and $Y$.

Table 1:  If the adjustment set is valid, NICE does not hurt the estimation
performance. The table reports average MAE and bootstrap standard deviations
of the SATT estimation.

Valid Adjustment | $\epsilon_{att}$
---|---
|  | Mod1 | Mod2 | Mod3 | Mod4
Low | Adjust All | $.04\pm.08$ | $.05\pm.09$ | $.07\pm.09$ | $.01\pm.01$
| NICE | $.07\pm.03$ | $.02\pm.01$ | $.09\pm.03$ | $.04\pm.02$
Med | Adjust All | $.07\pm.10$ | $.05\pm.05$ | $.04\pm.04$ | $.07\pm.08$
| NICE | $.05\pm.02$ | $.04\pm.03$ | $.05\pm.03$ | $.03\pm.02$
High | Adjust All | $.07\pm.07$ | $.06\pm.05$ | $.06\pm.07$ | $.04\pm.04$
| NICE | $.02\pm.01$ | $.06\pm.03$ | $.04\pm.02$ | $.07\pm.04$

Table 2: NICE reduces estimation bias in the presence of bad controls. The
table reports the average MAE and bootstrap standard deviation of SATT.

Bad Controls in Adjustment Set | $\epsilon_{att}$
---|---
|  | Mod1 | Mod2 | Mod3 | Mod4
low | Adjust All | $.26\pm.09$ | $.42\pm.03$ | $.34\pm.08$ | $.46\pm.09$
| NICE | $.09\pm.07$ | $.03\pm.01$ | $.11\pm.04$ | $.08\pm.04$
med | Adjust All | $.38\pm.10$ | $.35\pm.06$ | $.40\pm.17$ | $.3\pm.09$
| NICE | $.06\pm.03$ | $.06\pm.03$ | $.06\pm.02$ | $.03\pm.03$
high | Adjust All | $.32\pm.14$ | $.38\pm.09$ | $.42\pm.05$ | $.28\pm.05$
| NICE | $.05\pm.03$ | $.11\pm.03$ | $.16\pm.05$ | $.11\pm.05$

The ACIC simulations are designed to assess the estimation quality of
predictors and estimators. They do not come in multiple environments, nor do
the covariates include bad controls. To create multiple environments, we draw
6000 samples and select a covariate $x$ that’s not the causal parent of $Y$.
We sort the samples based on $x$ and divide them into three equal sized
environments. For each DGP, we draw 10 bootstrap samples. To simulate bad
controls, we included 20 copies of a collider in the adjustment set:
$X^{e}_{co}=T^{e}+Y^{e}+\mathcal{N}(0,e^{2})$, where $e\in\\{0.01,0.2,1\\}$.

##### Analysis.

We compare two adjustment schemes: adjusting for all covariates and NICE. We
first consider the setting where there are no bad controls. Table 1 reports
the average SATT MAE and standard deviations over 10 bootstraps under two
adjustment schemes. We observe that NICE does not hurt the estimation quality
in comparison to adjusting for all covariates.

We also consider the setting where there is a strong collider. As shown in
Table 2, NICE reduces collider bias across simulation setups. However, we also
observe that while it reduces the collider bias, it does not eliminate it
completely. One potential reason is that the predictor is not optimal.

###  5.4 The effect of environment variations on NICE’s performance

$A_{t}$$A_{y}$$A_{1}$$X_{t}$$X_{y}$$T$$Y$$Z$ Figure 5: The DGP for Section
5.4. The adjustment set {X, A} is valid. Adjustment set {X, A, Z} is not
valid.  Figure 6: NICE mitigates bad controls more with access to more diverse
environments. The x-axis is the environmental diversity. The y-axis is the
average MAE of the SATT.

In this experiment, we examine the effect of environment variations on NICE’s
performance. We simulate non-linear data using the causal graph illustrated in
Figure 5. The details of the data simulation are in the appendix.

We first draw three source environments $\\{P^{e_{1}},P^{e_{2}},P^{e_{3}}\\}$
that are diverse. To control the level of environment variation, we construct
three new environments
$\\{P^{e^{\prime}_{1}},P^{e^{\prime}_{2}},P^{e^{\prime}_{3}}\\}$ that are
mixtures of the three source environments. Respectively,
$P^{e^{\prime}_{1}},P^{e^{\prime}_{2}},P^{e^{\prime}_{3}}$ draw ($p_{1}$,
$p_{2}$, $p_{3}$) proportions from $P^{e_{1}}$, ($p_{2}$, $p_{3}$, $p_{1}$)
proportions from $P^{e_{2}}$, and ($p_{2}$, $p_{3}$, $p_{1}$) proportions from
$P^{e_{3}}$. The proportions $(p_{1}$, $p_{2}$, $p_{3})$ sum to one.

We approximate the diversity of the environments by the diversity of the
proportions. The diversity measure is: $\frac{1}{{3}}\sum_{ij}|p_{i}-p_{j}|$.
We consider 14 set of new environments, induced by different combination of
the mixture probabilities. We compare the estimation quality of NICE when
given a covariate set that include bad controls $\\{X,A,Z\\}$ against
adjusting for a valid covariate set $\\{X,A\\}$.

As shown in Figure 6, the more diverse the environments, the more likely that
NICE can strip out bad controls and reduce bias. When environments are
sufficiently diverse, the learned representation is equivalent to a valid
adjustment set.

##  6 Discussion

NICE lives at the intersection of representation learning and causal
inference, demonstrating how representation learning ideas can be harnessed to
improve causal estimation. Here we have examined the causal setup where it’s
unknown which covariates are safe to adjust for. One important direction for
future work is to expand this setting to one where we combine partial causal
knowledge with representation learning for estimating effects in more general
scenarios.

###### Acknowledgements.

This work is funded by ONR N00014-17-1-2131, ONR N00014-15-1-2209, NIH
1U01MH115727-01, NSF CCF-1740833, DARPA SD2 FA8750-18-C-0130, Amazon, and
Simons Foundation. We thank Ryan Carey, Gemma Moran, Kevin Murphy, Dhanya
Sridhar, and anonymous reviewers for helpful comments and feedback.

#### References

## References

  * [1] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani and David Lopez-Paz  “Invariant Risk Minimization”  In _arXiv preprint arXiv:1907.02893_ , 2019
  * [2] Susan Athey and Guido Imbens  “Recursive partitioning for heterogeneous causal effects”  In _Proceedings of the National Academy of Sciences_ 113.27 National Acad Sciences, 2016, pp. 7353–7360
  * [3] Peter C Austin  “An introduction to propensity score methods for reducing the effects of confounding in observational studies”  In _Multivariate behavioral research_ 46.3 Taylor & Francis, 2011, pp. 399–424
  * [4] Nathaniel Beck, Gary King and Langche Zeng  “Improving quantitative studies of international conflict: A conjecture”  In _American Political science review_ JSTOR, 2000, pp. 21–35
  * [5] Jay Bhattacharya and William B Vogt  “Do instrumental variables belong in propensity scores?”, 2007
  * [6] Peter Bühlmann  “Invariance, causality and robustness”  In _arXiv preprint arXiv:1812.08233_ , 2018
  * [7] Carlos Cinelli and Chad Hazlett  “Making sense of sensitivity: Extending omitted variable bias”  In _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ 82.1 Wiley Online Library, 2020, pp. 39–67
  * [8] Juan Correa and Elias Bareinboim  “A calculus for stochastic interventions: Causal effect identification and surrogate experiments”  In _Proceedings of the AAAI Conference on Artificial Intelligence_ 34.06, 2020, pp. 10093–10100
  * [9] Alex D’Amour et al.  “Overlap in observational studies with high-dimensional covariates”  In _Journal of Econometrics_ , 2020
  * [10] Peng Ding, TJ VanderWeele and James M Robins  “Instrumental variables as bias amplifiers with general outcome and confounding”  In _Biometrika_ 104.2 Oxford University Press, 2017, pp. 291–302
  * [11] Raymond Fisman, Sheena S Iyengar, Emir Kamenica and Itamar Simonson  “Gender differences in mate selection: Evidence from a speed dating experiment”  In _The Quarterly Journal of Economics_ 121.2 MIT Press, 2006, pp. 673–697
  * [12] Michele Jonsson Funk et al.  “Doubly robust estimation of causal effects”  In _American journal of epidemiology_ 173.7 Oxford University Press, 2011, pp. 761–767
  * [13] Clark Glymour, Kun Zhang and Peter Spirtes  “Review of causal discovery methods based on graphical models”  In _Frontiers in genetics_ 10 Frontiers, 2019, pp. 524
  * [14] Adam N Glynn and Kevin M Quinn  “An introduction to the augmented inverse propensity weighted estimator”  In _Political analysis_ JSTOR, 2010, pp. 36–56
  * [15] Trygve Haavelmo  “The statistical implications of a system of simultaneous equations”  In _Econometrica, Journal of the Econometric Society_ JSTOR, 1943, pp. 1–12
  * [16] Jason Hartford, Greg Lewis, Kevin Leyton-Brown and Matt Taddy  “Deep IV: A flexible approach for counterfactual prediction”  In _International Conference on Machine Learning_ , 2017, pp. 1414–1423  PMLR
  * [17] Christina Heinze-Deml, Jonas Peters and Nicolai Meinshausen  “Invariant causal prediction for nonlinear models”  In _Journal of Causal Inference_ 6.2 De Gruyter, 2018
  * [18] Jennifer Hill  “Bayesian Nonparametric Modeling for Causal Inference”  In _Journal of Computational and Graphical Statistics_ 20, 2011, pp. 217–240  DOI: 10.1198/jcgs.2010.08162
  * [19] Guido W Imbens  “Nonparametric estimation of average treatment effects under exogeneity: A review”  In _Review of Economics and statistics_ 86.1 MIT Press, 2004, pp. 4–29
  * [20] Dominik Janzing  “Causal regularization”  In _arXiv preprint arXiv:1906.12179_ , 2019
  * [21] Diederik P Kingma and Jimmy Ba  “Adam: A method for stochastic optimization”  In _arXiv preprint arXiv:1412.6980_ , 2014
  * [22] Christos Louizos et al.  “Causal effect inference with deep latent-variable models”  In _NEURIPS_ , 2017, pp. 6449–6459
  * [23] Sara Magliacane et al.  “Domain adaptation by using causal inference to predict invariant conditional distributions”  In _Advances in Neural Information Processing Systems_ , 2018, pp. 10846–10856
  * [24] Joris M Mooij et al.  “Distinguishing cause from effect using observational data: methods and benchmarks”  In _The Journal of Machine Learning Research_ 17.1 JMLR. org, 2016, pp. 1103–1204
  * [25] Kevin Murphy and Saira Mian  “Modelling gene expression data using dynamic Bayesian networks”, 1999
  * [26] Trang Quynh Nguyen, Allan Dafoe and Elizabeth L Ogburn  “The magnitude and direction of collider bias for binary variables”  In _Epidemiologic Methods_ 8.1 De Gruyter, 2019
  * [27] Judea Pearl  “Causality: models, reasoning and inference”  Springer, 2000
  * [28] Judea Pearl  “Causality”  Cambridge university press, 2009
  * [29] Judea Pearl and Azaria Paz  “Confounding equivalence in causal inference”  In _Journal of Causal Inference J. Causal Infer._ 2.1 De Gruyter, 2014, pp. 75–93
  * [30] Jonas Peters, Peter Bühlmann and Nicolai Meinshausen  “Causal inference by using invariant prediction: identification and confidence intervals”  In _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ Wiley Online Library, 2016
  * [31] Niklas Pfister, Peter Bühlmann and Jonas Peters  “Invariant causal prediction for sequential data”  In _Journal of the American Statistical Association_ 114.527 Taylor & Francis, 2019, pp. 1264–1276
  * [32] James Robins  “A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect”  In _Mathematical modelling_ 7.9-12 Elsevier, 1986, pp. 1393–1512
  * [33] Mateo Rojas-Carulla, Bernhard Schölkopf, Richard Turner and Jonas Peters  “Invariant models for causal transfer learning”  In _The Journal of Machine Learning Research_ 19.1 JMLR. org, 2018, pp. 1309–1342
  * [34] Paul R Rosenbaum  “Overt bias in observational studies”  In _Observational studies_ Springer, 2002, pp. 71–104
  * [35] Paul R Rosenbaum and Donald B Rubin  “The central role of the propensity score in observational studies for causal effects”  In _Biometrika_ Oxford University Press, 1983
  * [36] Elan Rosenfeld, Pradeep Ravikumar and Andrej Risteski  “The Risks of Invariant Risk Minimization”  In _arXiv preprint arXiv:2010.05761_ , 2020
  * [37] Donald B Rubin  “Should observational studies be designed to allow lack of balance in covariate distributions across treatment groups?”  In _Statistics in Medicine_ 28.9 Wiley Online Library, 2009, pp. 1420–1423
  * [38] Uri Shalit, Fredrik D. Johansson and David Sontag  “Estimating individual treatment effect: generalization bounds and algorithms”  In _arXiv e-prints arXiv:1606.03976_ , 2016
  * [39] Claudia Shi, David M Blei and Victor Veitch  “Adapting Neural Networks for the Estimation of Treatment Effects”  In _Advances in neural information processing systems_ , 2019
  * [40] Shohei Shimizu, Patrik O Hoyer, Aapo Hyvärinen and Antti Kerminen  “A linear non-Gaussian acyclic model for causal discovery”  In _Journal of Machine Learning Research_ 7.Oct, 2006, pp. 2003–2030
  * [41] Susan M Shortreed and Ashkan Ertefaie  “Outcome-adaptive lasso: Variable selection for causal inference”  In _Biometrics_ 73.4 Wiley Online Library, 2017, pp. 1111–1122
  * [42] Pater Spirtes et al.  “Constructing Bayesian network models of gene expression networks from microarray data”  Carnegie Mellon University, 2000
  * [43] Tyler J VanderWeele and Ilya Shpitser  “A new criterion for confounder selection”  In _Biometrics_ 67.4 Wiley Online Library, 2011, pp. 1406–1413
  * [44] Jinsung Yoon, James Jordon and Mihaela Schaar  “GANITE: Estimation of individualized treatment effects using generative adversarial nets”, 2018
  * [45] Kun Zhang et al.  “Domain Adaptation As a Problem of Inference on Graphical Models”  In _arXiv preprint arXiv:2002.03278_ , 2020
  * [46] Qingyuan Zhao, Charles Zheng, Trevor Hastie and Robert Tibshirani  “Comment on Causal inference using invariant prediction”  In _arXiv preprint arXiv: 1501.01332_ , 2016

##  7 Appendix

###  7.1 Proofs of theorems

######  Lemma 4.1.

Suppose that
$\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)=a\right]\neq\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)=a^{\prime}\right]$
whenever $a\neq a^{\prime}$. Then a representation $\Phi$ is invariant across
all valid environments if and only if
$\mathbb{E}\left[Y^{e}|\Phi(T^{e},X^{e})\right]=\mathbb{E}\left[Y|\mathrm{Pa}(Y)\right]$
for all valid environments.

###### Proof.

The if direction is immediate.

To establish the only if direction, we first show that $\Phi$ must contain at
least $\mathrm{Pa}(Y)$, in the sense
$\mathbb{E}\left[Y\,|\,\Phi(X)\right]=\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\cup
Z\right]$ for some set $Z$. We proceed with proof by contradiction. Suppose
that conditioning on $\Phi$ is equivalent to conditioning on only
$\mathrm{Pa}(Y)\setminus\\{P\\}\cup Z$, where $P$ is a parent of $Y$. We now
create two environments by setting $P=p$ and $P=p^{\prime}$. Since $P$ is a
parent of $Y$ this follows from the second rule of do calculus [27],

$\displaystyle\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z;do(P=p)\right]$
$\displaystyle=\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z,P=p\right]$

and

$\displaystyle\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z;do(P=p^{\prime})\right]$
$\displaystyle=\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z,P=p^{\prime}\right].$

The equality $\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z,P=p\right]=\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z,P=p^{\prime}\right]$ holds only if $P$ is conditionally independent of $Y$
given $\mathrm{Pa}(Y)\setminus\\{P\\}\cup Z$. Since $P$ is a parent of $Y$, by
the first assumption of the lemma, the equality does not hold. It follows that
$\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z\,;\,do(P=p)\right]\neq\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\setminus\\{P\\}\cup
Z\,;\,do(P=p^{\prime})\right]$. That is, if conditioning on $\Phi$ was
equivalent to conditioning on less information than $\mathrm{Pa}(Y)\cup Z$,
then $\Phi$ would not be invariant across all valid environments.

It remains to show that $\Phi$ does not contain any more information than
$\mathrm{Pa}(Y)$.

$\Phi$ cannot contain any descendants of the outcome. Suppose that $\Phi$
depends on some descendant $D$ of $Y$ in the sense that there is at least one
environment and $d\neq d^{\prime}$ where
$\mathbb{E}\left[Y\,|\,\Phi(X\setminus
D,D=d)\right]\neq\mathbb{E}\left[Y\,|\,\Phi(X\setminus
D,D=d^{\prime})\right]$. Then, construct a new environment $e$ by randomly
intervening and setting $\mathrm{do}(D=d)$ or $\mathrm{do}(D=d^{\prime})$,
each with probability $0.5$. In this new environment, there is no relationship
between $Y$ and $D$. Accordingly,
$\mathbb{E}\left[Y^{e}\,|\,\Phi(X^{e}\setminus
D^{e},D^{e}=d)\right]=\mathbb{E}\left[Y^{e}\,|\,\Phi(X^{e}\setminus
D^{e},D^{e}=d^{\prime})\right]$. Thus, the conditional expectations are not
equal (as functions of $d$) in the two environments—a contradiction.

Next, we show that, $\Phi$ needs not contain the non-parent ancestors $A$ of
the outcome, because
$\mathbb{E}\left[Y\,|\,\\{A\\}\cup\mathrm{Pa}(Y)\right]=\mathbb{E}\left[Y\,|\,\mathrm{Pa}(Y)\right]$
by the Markov property of the causal graph, where $A$ is any non-ancestor
variables. Since $\Phi$ contains $\mathrm{Pa}(Y)$, it follows that $\Phi$ does
not depend on any non-parent ancestor $A$.

For expository purposes, the proof is done with $\mathrm{do}$ calculus [27]
for atomic interventions. If the environments are generated with stochastic
interventions, we can use the same proof strategy with $\sigma$ calculus [8].

∎

######  Theorem 4.2.

Let $L$ be a loss function such that the minimizer of the associated risk is a
conditional expectation, and let $\Phi$ be a representation that elicits a
predictor $Q^{\mathrm{inv}}$ that is invariant for all valid distributions.
Assuming there is no mediators between the treatment and the outcome, then
$\psi^{e}=\mathbb{E}\left[Q^{\mathrm{inv}}(1,X^{e})-Q^{\mathrm{inv}}(0,X^{e})|T^{e}=1\right]$.

###### Proof.

We assume the technical condition of Lemma 4.1, that
$\mathbb{E}\left[Y^{e}\,|\,\mathrm{Pa}(Y^{e})=a\right]\neq\mathbb{E}\left[Y^{e}\,|\,\mathrm{Pa}(Y^{e})=a^{\prime}\right]$
whenever $a\neq a^{\prime}$. This is without loss of generality because
violations of this condition will not lead to different causal effects.

By the assumption on the loss function, the elicited invariant predictor is
$\mathbb{E}\left[Y^{e}\mid\Phi(T^{e},X^{e})\right]$. Lemma 4.1 shows that
$\mathbb{E}\left[Y^{e}\mid\Phi(T^{e},X^{e})\right]=\mathbb{E}\left[Y^{e}\mid\mathrm{Pa}(Y^{e})\right]$.
We further observe that the non-treatment parents of $Y^{e}$ are sufficient to
block backdoor paths. It follows the ATT can be expressed as the following.

$\displaystyle\psi^{e}$ $\displaystyle=\mathbb{E}[\mathbb{E}\left[Y^{e}\mid
T^{e}=1,\mathrm{Pa}(Y^{e})\setminus\\{T^{e}\\}\right]$
$\displaystyle-\mathbb{E}\left[Y^{e}\mid
T^{e}=0,\mathrm{Pa}(Y^{e})\setminus\\{T^{e}\\})\right]\,|\,T^{e}=1]$
$\displaystyle=\mathbb{E}\left[\mathbb{E}\left[Y^{e}\mid\Phi(1,X^{e})\right]-\mathbb{E}\left[Y^{e}\mid\Phi(0,X^{e})\right]\,|\,T^{e}=1\right]$

∎

######  Theorem 4.3.

Suppose $\epsilon\leq P(T^{e}=1|X^{e})\leq 1-\epsilon$ with probability 1,
then $\epsilon\leq P(T^{e}=1|\Phi(X^{e}))\leq 1-\epsilon$ with probability 1.

Proof.  The proof follows directly from Theorem 1 in [9]. The intuition is
that the richer the covariate set is, the more likely it is to predict the
treatment assignment accurately [9]. The covariate representation
$\Phi(X^{e})$ by definition contains less information than $X^{e}$, therefore
$\Phi(X^{e})$ satisfies overlap if $X^{e}$ satisfies overlap.

###  7.2 The case of Colliders

Consider the DGP with binary variables $\\{X,Y,T\\}$ illustrated in Figure 7,
where $X$ is causally influence by $Y$ and $T$.

######  Theorem 7.1.

Let $cov$ denote the covariance between two variables, we define collider bias
at $X=c$ as $\Delta(X=c)=cov(T,Y|X=c)-cov(T,Y)$, and collider bias of $X$ as
$\Delta(X)=|P(X=1)\Delta(X=1)+P(X=0)\Delta(X=0)|$. Let $\Phi(T,X)$ be a random
variable, where $P(\Phi(T,X)=X)\geq 0.5$. Suppose $P(X=1)=0.5$, and
$\Delta(X=1)$ has the same sign as $\Delta(X=0)$, conditioning on $X$ induce
more collider bias than conditioning its coarsening $\Phi(T,X)$:

$\Delta(\Phi(T,X))\leq\Delta(X)$

$X$$T$$Y$ Figure 7: V-structure graph. We denote the bias induced by
conditioning on $X$ as V-bias.  $C$$D$$T$$Y$ Figure 8: Y-structure graph. We
denote the bias induced by conditioning on $D$ as Y-bias.

Proof.  The proof follows corollary 2.1 in [26].

######  Corollary 2.1.

We refer to collider bias in the $V$ substructure embedded in the Y structure
as ‘embedded V-bias’ and denote it as $\Delta(C=c)$. For the covariance effect
scale, Y-bias $\Delta(D=d)$ relates to embedded V-bias through the following
formula:

$\displaystyle\Delta(D=d)$
$\displaystyle=\frac{p(D=d\,|\,C=1)-p(D=d\,|\,C=0)}{\\{\mathrm{P}(D=d)\\}^{2}}$
$\displaystyle\cdot\begin{bmatrix}p(D=d\,|\,C=1)\\{\mathrm{P}(C=1)\\}^{2}\cdot\Delta(C=1)-\\\
p(D=d\,|\,C=0)\\{\mathrm{P}(C=0)\\}^{2}\cdot\Delta(C=0)\end{bmatrix}.$

With the corollary above, let $D$ denote $\Phi(T,X)$, let $C$ denote the
collider $X$ in Figure 7. The bias induced by conditioning on $D$ is less than
the bias induced by conditioning on $C$.

$\displaystyle\Delta(D=1)$
$\displaystyle=\frac{2\alpha-1}{0.25}(0.25\alpha\cdot\Delta(C=1)$
$\displaystyle-0.25(1-\alpha)\cdot\Delta(C=0))$
$\displaystyle=(2\alpha-1)(\alpha\cdot\Delta(C=1)$
$\displaystyle-(1-\alpha)\cdot\Delta(C=0))$ $\displaystyle\Delta(D=0)$
$\displaystyle=\frac{1-2\alpha}{0.25}(0.25(1-\alpha)\cdot\Delta(C=1)$
$\displaystyle-0.25\alpha\cdot\Delta(C=0))$
$\displaystyle=(1-2\alpha)((1-\alpha)\cdot\Delta(C=1)$
$\displaystyle-\alpha\cdot\Delta(C=0))$ $\displaystyle\Delta(C)$
$\displaystyle=|0.5\cdot\Delta(C=0)+0.5\cdot\Delta(C=1)|$
$\displaystyle\Delta(D)$
$\displaystyle=|0.5\cdot\Delta(D=0)+0.5\cdot\Delta(D=1)|$
$\displaystyle\Delta(D)$ $\displaystyle=|0.5(2\alpha-1)^{2}\cdot\Delta(C=1)$
$\displaystyle+0.5(2\alpha-1)^{2}\cdot\Delta(C=0)|$
$\displaystyle\leq\Delta(C)$

###  7.3 The Case of Mediators

In the main paper, we assumed the covariate set $X$ contains no mediators
between treatment and outcome. What happens to the interpretation of the
learned parameter if the adjustment set contains mediators? Intuitively, NICE
retains the direct link between the treatment and the outcome. Specifically,
if there are no mediators, the parameter reduces to ATT. If there are
mediators but no confounders, the parameter reduces to the Natural Direct
Effect [27]. If there are mediators and confounders, the NICE estimand is a
non-standard causal target that we call the natural direct effect on the
treated (NDET).

Conceptually, NDET describes the expected change in outcome $Y$ for the
treated population, induced by changing the value of $T$, while keeping all
mediating factors $M$, constant at whatever value they would have obtained
under $\mathrm{do}(t)$. The main point is that NDET provides answers to
questions such as, “does this treatment have a substantial direct effect on
this outcome?”. Substantively, NDET is the natural direct effect, adjusted for
confounders.

Formally, NDET for environment $e$ is

$\displaystyle\psi^{e}$
$\displaystyle=\mathbb{E}_{M^{e}\,|\,T^{e}=1}[\mathbb{E}\left[Y^{e}\,|\,M^{e}\,;\,\mathrm{do}(T^{e}=1)\right]$
(7.1)
$\displaystyle-\mathbb{E}\left[Y^{e}\,|\,M^{e}\,;\,\mathrm{do}(T^{e}=0)\right]|T^{e}=1].$

With adjustment set $W^{e}$, the causal effect can be expressed through a
parameter of the observational distribution:

$\displaystyle\psi^{e}$
$\displaystyle=\mathbb{E}_{M^{e},W^{e}}[\mathbb{E}\left[Y^{e}\,|\,T^{e}=1,M^{e},W^{e}\right]$
(7.2)
$\displaystyle-\mathbb{E}\left[Y^{e}\,|\,T^{e}=0,M^{e},W^{e}\right]\,|\,T^{e}=1].$

Importantly, the mediators $M^{e}$ and the confounders $W^{e}$ show up in the
same way in Eq. 7.2. Accordingly, we don’t need to know which observed
variables are mediators and which are confounders to compute the parameter.
Under the NICE procedure, we condition on all parents of $Y^{e}$, including
possible mediators. Thus, the NICE estimand is the NDET in each environment.

###  7.4 Details of the experiments

####  7.4.1 Experiment 1

We evaluate the treatment effect estimation of various adjustment schemes
using four variants of the data simulations. The variants are generated
according to the following: 1) The observed covariates $S(X)$ are scrambled
versions of the true covariates $X$. If scrambled, $S$ is an orthogonal
matrix. If not scrambled, $S$ is an identity matrix. 2) In the heteroskedastic
setting $\tau\leftarrow 5+\mathcal{N}(0,e^{2})$. In the environment-level
homoskedastic setting $\tau\leftarrow 5+\mathcal{N}(0,1)$. The results are
illustrated in Figure 9, Figure 10, Figure 11, and Figure 12.

Figure 9: models performance under the scrambled and heteroskedastic setting
Figure 10: models performance under the scrambled and homoscedastic setting
Figure 11: models performance under the unscrambled and heteroskedastic
setting  Figure 12: models performance under the unscrambled and homoscedastic
setting

To understand why NICE reduces the estimation bias, we measure the weight of
the control predictor. The non-causal error is measured by the mean square
error of the weight on $X_{2}$. The results are illustrated Figure 13, Figure
14, Figure 15, and Figure 16.

Figure 13: non causal error under the scrambled and heteroskedastic setting
Figure 14: non causal error under the scrambled and homoscedastic setting
Figure 15: non causal error under the unscrambled and heteroskedastic setting
Figure 16: non causal error under the scrambled and homoscedastic setting

####  7.4.2 Experiment 2

We validate NICE for the non-linear case on a benchmark dataset, SpeedDating.
SpeedDating was collected to study the gender difference in mate selection
[11]. The study recruited university students to participate in speed dating,
and collected objective and subjective information such as ‘undergraduate
institution’ and ‘perceived attractiveness’. It has 8378 entries and 185
covariates. ACIC 2019’s simulation samples subsets of the covariates to
simulate binary treatment $T$ and binary outcome $Y$. Specifically, it
provides four modified DGPs: Mod1: parametric models; Mod2: complex models;
Mod3: parametric models with poor overlap; Mod4: complex models with treatment
heterogeneity. Each modification includes three versions: low, med, high,
indicating an increasing number of covariates included in the models for $T$
and $Y$.

We compare the estimation quality of the SATT and CATE over 10 bootstraps. We
use two predictors: TARnet and Dragonnet. The main paper report the estimation
quality of SATT using TARnet. We now report the estimation quality of CATE and
SATT using Dragonnet, as well as CATE using TARnet.

The DGPs are recorded in the R file under SpeedDating folder. The original
ACIC DGPs can be downloaded using the following link.

    drive.google.com/file/d/
    1Qqgmb3R9Vt9KTx6t8i_5IbFenylsPfrK/view

We made several modifications to the original DGPs. (1) ACIC competitions are
usually designed to evaluate model performances using ATE or ATT. The
simulation study provided ATE values but not ITE. In the SpeedDating DGPs, the
treatment is binary, and the outcome is also binary. To calculate ITE, we take
the difference between the propensity of the outcome of the treated predictor
and the propensity of the outcome of the control predictor. (2) Unfortunately,
ACIC datasets do not come in multiple environments, nor do the covariates
include bad controls. To create multiple environments, we draw 6000 samples
and select a covariate $x$ that’s not the causal parent of $Y$. We sort the
samples based on the the covariates value and divide them into three equal
sized environments. For each DGP, we draw 10 bootstrap samples.

Table 5 and Table 6 show the average of the MAE of SATT estimates over three
environments. The predictor uses the architecture of Dragonnet. We observe
that NICE does not hurt the estimation quality in comparison to adjusting for
all the covariates. When there is a strong collider in the adjustment set,
NICE reduces collider bias across simulation setups.

Table 3 and Table 7 show the PEHE of the CATE, with TARnet and dragonnet,
respectively. We find that NICE improves the CATE estimates in Mod1, Mod2, and
Mod3. This is surprising, as we expect NICE to perform equally well as
adjusting for all covariates when the adjustment set is valid. To understand
this phenomenon better, we examine the CATE estimates across simulation
settings.

In Figure 17, Figure 18, Figure 19, and Figure 20 illustrate the estimation
quality of CATE under the “med" simulation setting. we compare the ground
truth, the CATE estimates using NICE and Adjusting for all covariates across
settings. Recall NICE uses an predictor trained with IRMv1 objective,
adjusting for all uses a predictor that’s trained with ERM objective. In Mod1,
Mod2, and Mod3, there are little heterogeneity of the treatment effects. In
Mod4, the treatment effects are more spread out and heterogeneous. We observe
that in Mod1, Mod2, and Mod3, the ERM predictors produce extreme CATE
estimates. In contrast, using an IRMv1 objective, the CATE estimates are less
extreme. In Mod4, where the CATE varies drastically, both IRMv1 and ERM
predictors were able to capture the heterogeneity.

To examine whether the difference is due to over-fitting, we use two
environments as training and one environment as testing. Figure 21 and Figure
22 show the corresponding training and testing accuracy across experiment
setups. We observe the ERM predictors have similar training and testing
accuracy. This suggests the model is not overfitting in the robust prediction
sense. We suspect that the IRMv1 penalty term becomes a regularization term
that restrict the model to simpler solutions. Prior work [20] has shown that
regularizing terms in linear regression settings not only help against over-
fitting finite data, but sometimes also produce better causal models in the
infinite sample settings. There are no known results in the non-linear
settings.

Note that the ACIC competitions are _not_ designed for evaluating CATE
performance. Estimating CATE when the outcomes are binary are difficult,
especially given a flexible neural network model. The analysis above is about
model specification for estimation, in settings _without_ bad controls. In
this paper, we consider the problem of causal adjustment. We focus on finding
a causal representation that strips out bad controls. Does an invariant
predictor produce better causal estimate, even when there are no bad controls?
We defer this question to future work.

valid adjustment | $\epsilon_{pehe}$
---|---
| TARnet | Mod1 | Mod2 | Mod3 | Mod4
low | Adjust All | $.16\pm.06$ | $.13\pm.05$ | $.22\pm.18$ | $.05\pm.01$
| NICE | $.06\pm.02$ | $.05\pm.02$ | $.06\pm.02$ | $.05\pm.01$
med | Adjust All | $.14\pm.06$ | $.15\pm.02$ | $.12\pm.02$ | $.06\pm.02$
| NICE | $.05\pm.01$ | $.05\pm.01$ | $.07\pm.03$ | $.04\pm.01$
high | Adjust All | $.13\pm.07$ | $.12\pm.02$ | $.16\pm.09$ | $.06\pm.01$
| NICE | $.05\pm.01$ | $.05\pm.01$ | $.07\pm.02$ | $.04\pm.01$

Table 3:

bad controls in adjustment | $\epsilon_{pehe}$
---|---
| TARnet | Mod1 | Mod2 | Mod3 | Mod4
low | Adjust All | $.35\pm.12$ | $.17\pm.02$ | $.30\pm.04$ | $.24\pm.04$
| NICE | $.08\pm.04$ | $.05\pm.01$ | $.07\pm.02$ | $.07\pm.01$
med | Adjust All | $.26\pm.02$ | $.27\pm.05$ | $.39\pm.07$ | $.13\pm.01$
| NICE | $.04\pm.01$ | $.05\pm.02$ | $.06\pm.01$ | $.04\pm.01$
high | Adjust All | $.31\pm.04$ | $.23\pm.06$ | $.31\pm.07$ | $.12\pm.01$
| NICE | $.07\pm.02$ | $.07\pm.03$ | $.14\pm.07$ | $.06\pm.03$

Table 4:

valid adjustment | $\epsilon_{att}$
---|---
| Dragonnet | Mod1 | Mod2 | Mod3 | Mod4
low | Adjust All | $.09\pm.08$ | $.05\pm.03$ | $.04\pm.04$ | $.03\pm.01$
| NICE | $.06\pm.03$ | $.03\pm.01$ | $.09\pm.02$ | $.02\pm.02$
med | Adjust All | $.06\pm.09$ | $.06\pm.05$ | $.11\pm.11$ | $.07\pm.05$
| NICE | $.07\pm.02$ | $.06\pm.03$ | $.08\pm.03$ | $.04\pm.03$
high | Adjust All | $.04\pm.04$ | $.05\pm.08$ | $.03\pm.02$ | $.02\pm.02$
| NICE | $.02\pm.01$ | $.07\pm.02$ | $.06\pm.02$ | $.08\pm.05$

Table 5:

bad controls in adjustment | $\epsilon_{att}$
---|---
| Dragonnet | Mod1 | Mod2 | Mod3 | Mod4
low | Adjust All | $.31\pm.12$ | $.52\pm.16$ | $.39\pm.06$ | $.75\pm.11$
| NICE | $.17\pm.10$ | $.03\pm.02$ | $.24\pm.03$ | $.11\pm.06$
med | Adjust All | $.54\pm.09$ | $.47\pm.15$ | $.57\pm.24$ | $.31\pm.13$
| NICE | $.13\pm.10$ | $.25\pm.05$ | $.15\pm.08$ | $.05\pm.03$
high | Adjust All | $.50\pm.06$ | $.58\pm.05$ | $.54\pm.05$ | $.43\pm.13$
| NICE | $.07\pm.04$ | $.21\pm.06$ | $.17\pm.05$ | $.08\pm.03$

Table 6:

valid adjustment | $\epsilon_{pehe}$
---|---
| Dragonnet | Mod1 | Mod2 | Mod3 | Mod4
low | Adjust All | $.20\pm.06$ | $.12\pm.02$ | $.19\pm.02$ | $.06\pm.01$
| NICE | $.06\pm.01$ | $.04\pm.01$ | $.06\pm.01$ | $.04\pm.01$
med | Adjust All | $.15\pm.06$ | $.16\pm.01$ | $.16\pm.07$ | $.06\pm.01$
| NICE | $.04\pm.01$ | $.03\pm.01$ | $.07\pm.02$ | $.03\pm.01$
high | Adjust All | $.12\pm.02$ | $.14\pm.05$ | $.13\pm.02$ | $.06\pm.00$
| NICE | $.05\pm.01$ | $.06\pm.01$ | $.07\pm.02$ | $.04\pm.02$

Table 7:

bad controls in adjustment | $\epsilon_{pehe}$
---|---
| Dragonnet | Mod1 | Mod2 | Mod3 | Mod4
low | Adjust All | $.47\pm.05$ | $.23\pm.05$ | $.37\pm.05$ | $.46\pm.14$
| NICE | $.16\pm.08$ | $.05\pm.02$ | $.18\pm.04$ | $.05\pm.02$
med | Adjust All | $.32\pm.05$ | $.37\pm.06$ | $.55\pm.18$ | $.15\pm.02$
| NICE | $.07\pm.03$ | $.13\pm.04$ | $.11\pm.05$ | $.04\pm.01$
high | Adjust All | $.38\pm.06$ | $.29\pm.05$ | $.39\pm.05$ | $.14\pm.02$
| NICE | $.09\pm.03$ | $.11\pm.04$ | $.13\pm.05$ | $.04\pm.01$

Table 8:  Figure 17:  Mod1: parametric models  Figure 18: Mod2: complex models
Figure 19: Mod1: parametric models with poor overlap Figure 20: Mod4: complex
models with treatment heterogeneity. Figure 21: The training accuracy of the
predictors Figure 22: The testing accuracy of the predictors

####  7.4.3 Experiment 3

In the third experiment, we examine the effect of environment variations on
NICE’s performance. We simulate non-linear data using the causal graph
illustrated in fig. 5. We draw three source environments
$\\{P^{e_{1}},P^{e_{2}},P^{e_{3}}\\}$, where $e_{1}=0.2,e_{2}=1,e_{3}=5$. In
each source environments, we draw 900 samples. Figure 6 reports the average
MAE of SATT over 5 simulations.

$\displaystyle A^{e}$ $\displaystyle\leftarrow\mathcal{N}(0,e^{2})$
$\displaystyle X^{e}$ $\displaystyle\leftarrow A^{e}\cdot w_{ax^{e}}$
$\displaystyle X^{e}_{t}$ $\displaystyle\leftarrow X^{e}_{\\{1...12\\}}$
$\displaystyle X^{e}_{y}$ $\displaystyle\leftarrow X^{e}_{\\{13...30\\}}$
$\displaystyle p^{e}_{t}$
$\displaystyle\leftarrow\text{sigmoid}(f(X^{e}_{t}))$ $\displaystyle T^{e}$
$\displaystyle\leftarrow Bern(p^{e}_{t})$ $\displaystyle p^{e}_{y}$
$\displaystyle\leftarrow\text{sigmoid}(g(X^{e}_{t},X^{e}_{y},T^{e}))$
$\displaystyle Y^{e}$ $\displaystyle\leftarrow B(n,p^{e}_{y})$ $\displaystyle
Z^{e}$ $\displaystyle\leftarrow Y^{e}+T^{e}+\mathcal{N}(0,1)$

$f(X^{e}_{t})=X^{e}_{t}\cdot w_{xt^{e}}+h(X_{t}^{e})\cdot
w_{xt^{e^{\prime}}}$, where $h(X_{t}^{e})$ is implemented as

      [X_t[:, :1] * X_t[:, 1:2],
      X_t[:, 1:2] * X_t[:, 2:4],
      X_t[:, 2:3] * X_t[:, 3:]
      / np.square(X_t).mean()]

$g(X^{e}_{t},X^{e}_{y},T^{e})=1.25*T^{e}+X_{t}\cdot
w_{xy^{e}}+2*p^{e}_{t}+m(x^{e}_{y})\cdot w_{xy^{e^{\prime}}}$

Here $m(x^{e}_{y})$ is implemented as

        [X_y[:, :1] * X_y[:, 4:5],
        X_y[:, 1:2] * X_y[:, 3:4],
        X_y[:, 1:2] * X_y[:, 2:]
        / np.square(X_y).mean()]

The complete data generating code is under

    diverse_environments/gen_dat.py

New environments $P^{e^{\prime}_{1}},P^{e^{\prime}_{2}},P^{e^{\prime}_{3}}$
are mixtures of the three source environments $P^{e_{1}},P^{e_{2}},P^{e_{3}}$
. Respectively, $P^{e^{\prime}_{1}},P^{e^{\prime}_{2}},P^{e^{\prime}_{3}}$
draw ($p_{1}$, $p_{2}$, $p_{3}$) proportions from from $P^{e_{1}}$, ($p_{2}$,
$p_{3}$, $p_{1}$) proportions from from $P^{e_{2}}$, and ($p_{2}$, $p_{3}$,
$p_{1}$) proportions from from $P^{e_{3}}$.The proportions $(p_{1}$, $p_{2}$,
$p_{3})$ sum to one.

The mixing proportions we considered are: $(0,0,1)$, $(0,0.1,0.9)$,
$(0,0.2,0.8)$, $(0,0.3,0.7)$, $(0,0.4,0.6)$,
$(0,0.5,0.5)$,$(0.1,0.1,0.8)$,$(0.1,0.2,0.7)$,$(0.1,0.3,0.6)$,
$(0.1,0.4,0.5)$,$(0.2,0.2,0.6)$, $(0.2,0.3,0.5)$,$(0.2,0.4,0.4)$,
$(0.3,0.3,0.4)$.