# Spillover Detection for Donor Selection in Synthetic Control Models

Michael O’Riordan Spotify  Ciarán M. Gilligan-Lee Spotify  University College
London

###### Abstract

Synthetic control (SC) models are widely used to estimate causal effects in
settings with observational time-series data. To identify the causal effect on
a target unit, SC requires the existence of correlated units that are not
impacted by the intervention. Given one of these potential donor units, how
can we decide whether it is in fact a _valid_ donor—that is, one not subject
to spillover effects from the intervention? Such a decision typically requires
appealing to strong _a priori_ domain knowledge specifying the units, which
becomes infeasible in situations with large pools of potential donors. In this
paper, we introduce a practical, theoretically-grounded donor selection
procedure, aiming to weaken this domain knowledge requirement. Our main result
is a Theorem that yields the assumptions required to identify donor values at
post-intervention time points using only pre-intervention data. We show how
this Theorem—and the assumptions underpinning it—can be turned into a
practical method for detecting potential spillover effects and excluding
invalid donors when constructing SCs. Importantly, we employ sensitivity
analysis to formally bound the bias in our SC causal estimate in situations
where an excluded donor was indeed valid, or where a selected donor was
invalid. Using ideas from the proximal causal inference and instrumental
variables literature, we show that the excluded donors can nevertheless be
leveraged to further debias causal effect estimates. Finally, we illustrate
our donor selection procedure on both simulated and real-world datasets.

##  1 Introduction

The ability to estimate causal effects is of fundamental importance in many
domains, including medicine, economics, and industry [see e.g. Lee and
Spekkens, 2017, Gilligan-Lee, 2020, Richens et al., 2020, Dhir and Lee, 2020,
Perov et al., 2020, Vlontzos et al., 2021, Gilligan-Lee et al., 2022, Jeunen
et al., 2022, Reynaud et al., 2022, Van Goffrier et al., 2023]. In the absence
of experimental data from randomised controlled trials or A/B tests,
practitioners are faced with observational (non-randomised) data, and must
rely on the assumptions, tools, and techniques of causal inference to estimate
the impact of interventions. Synthetic control (SC) models, first introduced
more than 20 years ago by Abadie and Gardeazabal [2003], are widely used to
estimate treatment effects in settings with observational _time-series_
(panel) data [e.g. Abadie et al., 2010, 2015, Brodersen et al., 2015, Kreif et
al., 2016], and have been described by Athey and Imbens [2017] as “arguably
the most important innovation in the policy evaluation literature in the last
15 years”.

To estimate the impact of an intervention on a target unit, SC requires time-
series data for the target unit as well as time-series data for other units
correlated with the target, often called donors. Crucially, the donors must
not be impacted by the intervention. That is, there must no spillover effects
from the intervention to the donors. The SC method uses pre-intervention data
to construct a _synthetic control unit_ that matches the pre-intervention
target as closely as possible. The post-intervention evolution of this SC unit
estimates the evolution of the target unit in a counterfactual world where the
intervention did not occur, all else being equal. Therefore, the causal impact
of the intervention can be estimated by comparing the observed, factual, post-
intervention target to the counterfactual one.

Given time-series data for a potential donor, how can we determine that it is
_not_ impacted by the intervention, and hence constitutes a _valid_ donor that
can be used in the construction of SCs? Such a decision typically requires
appealing to strong _a priori_ domain knowledge about the nature of the
intervention and donors. Usually, we must already know (or assume) that the
entire pool of potential donors is not subject to spillover effects from the
intervention. However, in many real-world applications the pool of potential
donors can be very large, such as when estimating the impact of a new feature
on a large online platform [Lin et al., 2023, Section 5], and domain knowledge
alone is unlikely to be adequate for donor selection due to the scale of the
problem.

In this work, we aim to relax the domain knowledge requirements by introducing
a practical, theoretically-grounded donor selection procedure. This procedure
can be used to augment partial knowledge about invalid donors, thereby
allowing us to rely on a weaker form of domain knowledge than usually required
to select valid donors. We also relate the failure modes of our selection
procedure to recent advances in the sensitivity analysis frameworks for SC
[Zeitler et al., 2023] and negative control [Miao et al., 2020], giving formal
bounds on the bias for a given selection of donors, and further reducing the
burden on domain knowledge to select a perfect set of valid donors.

Our main result is a Theorem—based on techniques from proximal causal
inference—that provides the assumptions required to identify and forecast
values of specific donors at post-intervention time points using pre-
intervention donor data only. This is in contrast to the standard SC
identifiability, which additionally uses post-intervention donor values to
predict the post-intervention counterfactual for the target unit. The main
assumptions required for a given donor’s post-intervention values to be
identified are that the donor is not impacted by the intervention, and the
distribution of the underlying data generating mechanism in the past is
representative of future data points. Therefore, if we use pre-intervention
data to predict post-intervention data and get the wrong answer, either the
underlying latent distribution has changed, the donor has been impacted by
spillover effects from the intervention, or both.

We use this result to detect potential spillover effects and exclude donors
when constructing SCs. Importantly, we formally bound the potential bias
introduced by this selection procedure due to false positives (_excluding_
donors _not impacted_ by spillover effects) and false negatives (_including_
donors _impacted_ by spillover effects) using sensitivity analysis. While the
excluded donors aren’t used in constructing the SC, we show how they can still
be leveraged to further debias causal effect estimates in situations where the
donors are noisy proxies of the latent dynamics [Shi et al., 2023].

We conclude the paper by providing an empirical demonstration of our donor
selection procedure on both simulated and real-world datasets.

In summary, our main contributions are as follows:

  1. 1.

We prove a Theorem that yields the assumptions required to identify donor
values at post-intervention time points using only pre-intervention data.

  2. 2.

We introduce a practical donor selection procedure using this Theorem, that
detects potential spillover effects and excludes invalid donors when
constructing SCs.

  3. 3.

We use sensitivity analysis to formally bound the bias in our SC causal
estimate for a given selection of donors in situations where an excluded donor
was indeed valid, or where a selected donor was invalid.

  4. 4.

We provide a two-stage method that uses the excluded donors to further debias
causal effect estimates.

  5. 5.

We illustrate the performance of our donor selection procedure on both
simulated and real-world datasets.

##  2 Related work

##### Identifiability of Synthetic Controls

Historically, SC identifiability relied on assuming that the data generating
process can be modelled as a latent linear factor model. With this assumption,
the counterfactual is identified as a linear combination of valid donors. For
instance, see Abadie and Gardeazabal [2003], Abadie et al. [2010, 2015], and
extensions of these approaches utilising Bayesian structural time-series by
Brodersen et al. [2015].

More recently, Shi et al. [2022a] argued that linearity emerges in a non-
parametric manner if the target and donor units are in fact aggregations of
“smaller” units (e.g. country-level data are aggregates of individual-level
behaviours). The need for this “aggregate unit” assumption was subsequently
removed by Shi et al. [2023] and Zeitler et al. [2023], who leveraged proximal
causal inference to prove that the counterfactual can be non-parametrically
identified as a (potentially non-linear) function of valid donors.

##### Proximal Causal Inference

Proximal causal inference was initially investigated by Kuroki and Pearl
[2014], Miao et al. [2018], and has been further developed by Tchetgen et al.
[2020]. It has been used, for instance, in long-term causal effect estimation
by Imbens et al. [2022], and formulated in terms of the graphical causal
inference framework by Shpitser et al. [2021]. In the context of SC models,
Shi et al. [2023] and Zeitler et al. [2023] employed proximal causal inference
to prove non-parametric identifiability.

Recently, Liu et al. [2023] proposed a novel SC model leveraging special
donors called “surrogates”, which are correlates of the causal effect itself,
and can include potentially invalid donors. Assuming the existence of such
surrogates (as well as the requisite domain knowledge to identify them), Liu
et al. [2023] demonstrated SC estimation based on post-intervention data
alone.

##### Sensitivity Analysis

Later in the paper, we discuss scenarios where our donor selection procedure
fails, and relate these to recent works on sensitivity analysis in SC [Zeitler
et al., 2023] and negative control [Miao et al., 2020]. Sensitivity analysis
in the causal inference literature has mainly been focused on investigating
omitted variable bias in propensity-based models. This line of work originated
in Rosenbaum and Rubin [1983] and Imbens [2003], with modern formulations
provided by Veitch and Zaveri [2020] and Cinelli and Hazlett [2020], Cinelli
et al. [2019].

In the context of SC models, sensitivity analysis has been investigated by
Zeitler et al. [2023] and Nazaret et al. [2023]. The work by Zeitler et al.
[2023] explored a formal framework for sensitivity to violations of
identifiability of the full SC model, in particular, sensitivity to the
existence of relevant latent variables with no observed proxies. Nazaret et
al. [2023], on the other hand, explored mis-specifications to the standard
linearity assumption in SC models. Both of these works assumed valid donors
(i.e. no donors impacted by spillover effects from the intervention).

Miao et al. [2020] investigated sensitivity analysis in the context of
negative control [widely used in epidemiological research, negative control
variables can be viewed as specific proxy types within the proximal causal
inference framework Tchetgen et al., 2020, Shi et al., 2022b]. In particular,
Miao et al. [2020] introduced _positive_ control outcomes (which are
reminiscent of invalid donors in SC models), and discussed treating the
(unknown) spillover effects as sensitivity parameters to evaluate the
plausibility of causal estimates.

##  3 Methods

###  3.1 Synthetic Control Structural Causal Model

(a)

(b)

Figure 1: (a) SC DAG. Grey nodes are observed variables, white latent. The
intervention $I$ is applied at time point $t$, and is taken to be $0$ for all
time points before $t$, and $1$ for all time points from $t$ onwards. The
noise terms for the target $Y$, donors $X$, and latents $U$ have been
suppressed for ease of exposition. Note that autocorrelation in the target and
donor time-series arises due to the causal links between the latents at
different time points. As discussed in Appendix A, the model and results can
be readily generalised to the case where the evolution depends on $T$ time
points. (b) Donor forecast. At time point $t-1$, the donors
$x_{1}^{t-1},\dots,x_{N}^{t-1}$ are proxies for the latents
$u_{1}^{t-1},\dots,u_{M}^{t-1}$, which allows us to write $x_{i}^{t}$ as a
function of $x_{1}^{t-1},\dots,x_{N}^{t-1}$ and the noise terms at time $t$.
The noise terms at time $t-1$ have been suppressed for ease of exposition.

In Def. 3.1 [first provided by Zeitler et al., 2023], we formally define SC
models in the structural causal model (SCM) framework of Pearl [2009].

######  Definition 3.1 (Synthetic Control Structural Causal Model).

SC structural causal models (SCSCMs) consist of a set of $\mathbf{M}$ latent
variables $\mathbf{U}$ and their distributions, a set of $\mathbf{N}$ observed
variables $\mathbf{X}$ representing the donor units, a set of observed
variables $\mathbf{Y,I}$ representing the target unit, and the intervention,
and a set of deterministic functions mapping parents to their children in the
causal structure in Figure 1(a), represented as a directed acyclic graph
(DAG), each indexed by a specific time point $t$, such that

  1. 1.

$u_{i}^{t}=m_{i}^{t}(u_{i}^{t-1},\epsilon^{t}_{u_{i}})$

  2. 2.

$x^{t}_{i}=f_{i}^{t}(u_{1}^{t},\dots,u_{M}^{t},\epsilon^{t}_{x_{i}})$

  3. 3.

$y^{t}=g^{t}(u_{1}^{t},\dots,u_{M}^{t},I^{t},\epsilon^{t}_{y})$

where $\epsilon^{t}_{u_{i}}\sim P(\epsilon^{t}_{u_{i}})$,
$\epsilon^{t}_{x_{i}}\sim P(\epsilon^{t}_{x_{i}})$, and $\epsilon^{t}_{y}\sim
P(\epsilon^{t}_{y})$ are independent, exogenous error terms.

For simplicity, we sometimes follow Zhang and Bareinboim [2022] and suppress
the functional dependence on the exogenous error terms for the latents
$u_{1},\dots,u_{M}$. We also often drop the indices on the donors and latents
as follows: $x^{t}:=x_{1}^{t},\dots,x_{N}^{t}$,
$u^{t}:=u_{1}^{t},\dots,u_{M}^{t}$,
$\epsilon_{x}^{t}:=\epsilon_{x_{1}}^{t},\dots,\epsilon_{x_{N}}^{t}$, and
$\epsilon_{u}^{t}:=\epsilon_{u_{1}}^{t},\dots,\epsilon_{u_{M}}^{t}$.

This definition of SCSCMs generalises the standard latent linear factor model
formulation of SCs Abadie et al. [2010]. In particular, notice that the target
$y^{t}$ and donors $x^{t}$ can be arbitrary functions of the latents $u^{t}$.
We sometimes also use $y^{t}$ and $x^{t}$ to denote the _values_ these
variables take, however, the difference will be clear from the context. Note
that autocorrelation in the target and donor time-series arises due to the
causal links between the latents at different time points. As discussed in
Appendix A, the definition of SCSCMs and our results can be readily
generalised to the case where evolution depends on $T$ time points.

We finish this section by defining the proxy variable completeness condition
(Def. 3.2), and invariant causal mechanisms (Def. 3.3), which are necessary
for SC non-parametric identifiability, and form the basis for our Theorem 3.1.

The following completeness condition formally defines when a set of donors can
be considered _proxies_ for a set of latent variables [Tchetgen et al., 2020,
Zeitler et al., 2023].

######  Definition 3.2 (Completeness Condition).

For any square-integrable function $f$, if
$\operatorname{\mathbb{E}}\left(f(x^{t}_{1},\dots,x^{t}_{N})\mid
u^{t}_{1},\dots,u^{t}_{M}\right)=0,$ then $f(x^{t}_{1},\dots,x^{t}_{N})=0$ for
any $t$.

At a given time point, the completeness condition characterises how much
“information” the donors have about the latent variables, in the sense that
any variation in the $u$’s is captured by variation in $x$’s. For the rest of
the paper, we assume that the set of donors $x^{t}_{1},\dots,x^{t}_{N}$ can be
treated as proxies for the latents $u^{t}_{1},\dots,u^{t}_{M}$.

A _causal mechanism_ is the deterministic function that uniquely specifies a
variable from its parents in the causal graph, and is equivalent to the
conditional distribution of that variable given its (latent and observed)
parents. We assume that causal mechanisms are _invariant_ (Def. 3.3).

######  Definition 3.3 (Invariant Causal Mechanism).

A causal mechanism is invariant if it doesn’t depend on time point $t$.

The SCM formulation allows us to define (strong) interventions via the _do-
operator_ , disconnecting the intervened variable from its parents in the
causal graph, and assigning to it a specific value [Pearl, 2009]. Thus, we
quantify the impact of an intervention $I$ on the target $y$ at time $t$ as

$\tau=\underbrace{\operatorname{\mathbb{E}}\left(y^{t}\mid\text{do}(I^{t}=1),I^{t}=1\right)}_{\text{Observed}}\\\
-\,\underbrace{\operatorname{\mathbb{E}}\left(y^{t}\mid\text{do}(I^{t}=0),I^{t}=1\right)}_{\text{Counterfactual}}$
(1)

The first term is the observed, factual, post-intervention target, whereas the
second term is the unobserved, counterfactual one. For SCSCMs, Zeitler et al.
[2023] showed that if causal mechanisms are invariant, and if the donors are
proxies for the latents, then the counterfactual is identified via a unique
function $h$ of the (valid) donors such that
$\operatorname{\mathbb{E}}\left(y^{t}\mid\text{do}(I^{t}=0),I^{t}=1\right)=\operatorname{\mathbb{E}}\left(h(x^{t},I^{t}=0)\right)$.

###  3.2 Spillover Detection for Selecting Valid Donors

The power of SC lies in the fact that the counterfactual in Eq. 1 can be
identified even when there are post-intervention shifts in the exogenous
errors $P(\epsilon_{u}^{t})$ for the latents $u$. However, to identify the
counterfactual we still need to specify donor units $x$ that are _valid_. A
donor is valid if it adheres to the DAG depicted in Figure 1(a), and remains a
proxy for the latents (Def. 3.2) at all time points. In particular, it must
not be impacted by spillover effects from the intervention. Given an SCSCM
from Def. 3.1, spillover effects manifest as post-intervention shifts in the
donor errors $P(\epsilon_{x}^{t})$.

Usually, one must appeal to strong _a priori_ domain knowledge in order to
select valid donors. Can we leverage data from the donor pool itself,
augmenting incomplete domain knowledge, to gain confidence that a potential
donor is valid? Ideally, for a given donor candidate $x_{i}$, we would like to
be able to use pre-intervention data to test for shifts in
$P(\epsilon_{x_{i}}^{t})$ that would rule out $x_{i}$ as a valid donor. Such a
procedure would allow us to select donors based on a weaker form of domain
knowledge than that required to know exactly which donors adhere to the DAG.
This issue of selecting _valid_ donors is distinct from the usual
considerations of selecting donors based on pre-treatment fit [see e.g., Ben-
Michael et al., 2021], and is in fact a prerequisite.

In Section 3.2.1, we provide a Theorem showing that the assumptions necessary
for SC non-parametric identifiability—the proxy variable completeness
condition (Def. 3.2), and invariant causal mechanisms (Def. 3.3)—also
facilitate forecasting donor values given additional assumptions. We use this
Theorem to introduce a practical, theoretically-grounded donor selection
procedure based on detecting potential spillover effects to identify invalid
donors.

####  3.2.1 Donor Selection Procedure: Theory

######  Theorem 3.1.

If causal mechanisms are invariant, and the donors
$x^{t-1}_{1},\dots,x^{t-1}_{N}$ are proxies for the latents
$u^{t-1}_{1},\dots,u^{t-1}_{M}$ then, for each donor $x_{i}$, there exists a
unique function $h_{i}$ such that for all time points $t$ we have:

$\operatorname{\mathbb{E}}\left(x^{t}_{i}\right)=\operatorname{\mathbb{E}}\left(h_{i}(x^{t-1}_{1},\dots,x^{t-1}_{N},P(\epsilon^{t}_{x_{i}},\epsilon^{t}_{u}))\right)$
(2)

The proof of this Theorem can be found in Appendix A. Intuitively, because
$x_{i}^{t}$ is a function of the latents at time point $t-1$, and because the
donors at $t-1$ are proxies for those latents, we can swap the latents for the
donors and write $x_{i}^{t}$ as a function of $x_{1}^{t-1},\dots,x_{N}^{t-1}$
and the exogenous error terms at time $t$ (see also Figure 1(b)).

We now use Theorem 3.1 to identify and exclude potentially invalid donors.
Under conditions that a) donor $x_{i}$ is valid, and b) the exogenous error
distributions $P\bigl{(}\epsilon_{u}^{t}\bigr{)}$ for the latents have not
shifted at time $t$ relative to their pre-intervention values, then Theorem
3.1 implies that we can forecast post-intervention values for $x_{i}$ based on
pre-intervention donor data alone. Conversely, failing to forecast $x_{i}^{t}$
implies the violation of condition a), b), or both. This observation forms the
basis of our spillover detection method.

To use this theoretical result to get a practical method for flagging invalid
donors, we assume that forecast errors are due to the donor unit error terms
(due to being impacted by the intervention) and not the latent error terms.
Crucially, we assume that failing to forecast $x_{i}^{t}$ means that condition
a) is violated—that $P\bigl{(}\epsilon_{x_{i}}^{t}\bigr{)}$ has shifted111For
example,
$\epsilon_{x_{i}}^{t-1}\sim\mathcal{N}\bigl{(}0,\sigma_{x_{i}}\bigr{)}$, and
$\epsilon_{x_{i}}^{t}\sim\mathcal{N}\bigl{(}\tau_{x_{i}},\sigma_{x_{i}}\bigr{)}$,
where $\tau_{x_{i}}$ is the spillover effect on donor $x_{i}$. relative to
pre-intervention values—and $x_{i}$ is not a valid donor.

In general, without additional domain knowledge, we can’t be sure that the
unpredictability of $x_{i}$ is due to spillover effects and not changes in
$u_{i}$. However, in Section 3.4 we employ sensitivity analysis to formally
bound the bias in our SC causal estimate violations to this assumption – ie.
false positives, when we exclude a donor $x_{i}$ even though it was the latent
$u_{i}$ that changed, and false negatives, when we include a donor $x_{i}$
even though it was impacted by spillover effects. In particular, when our
spillover detection incorrectly flags a donor $x_{i}$ (which is a false
positive), then excluding this donor will only introduce bias if the remaining
selected donors don’t satisfy the completeness condition (i.e. there is
omitted variable bias). This sensitivity analysis is the bridge that brings us
from the theory of Theorem 3.1 to our practical donor selection method in
Algorithm 1.

As we demonstrate in Section 4, violations of condition b)—shifts in the
latents—are only an issue for the selection method if they occur at the same
time point as the donor forecast. If the latents shift later in the post-
intervention period, this does not bias donor selection, as this later post-
intervention data is not used in our selection procedure. Lags between the
intervention and spillover effects can be dealt with by forecasting on
coarsened, time-averaged donor data222For example, weekly averages instead of
daily. Note that the time averaging should not combine pre- and post-
intervention data.. Furthermore, time averaging can reduce false negatives in
cases with very noisy donors (see Figure 3). However, the longer time windows
also increase the risk of false positives due to potential shifts in the
latent distributions.

For the rest of the paper, we restrict our attention to _linear_ SC models,
both because of their ubiquity, and also, more practically, to be able to
discuss concrete sensitivity analysis bounds [Miao et al., 2020, Zeitler et
al., 2023]. We make no distinction, however, between linear SCs as a
consequence of the target and donors being linear functions of the latents,
and linearity emerging non-parametrically due to the “aggregate unit”
assumption of Shi et al. [2022a]. For demonstration purposes, we also focus on
simple linear models for estimating $h_{i}$ as part of the donor selection
procedure, although this is _not_ a strict requirement (even when restricting
to linear SC models, and linear sensitivity analysis). In principle, we can
swap the linear model in this procedure for any flexible machine-learning
model, using e.g. conformal inference for constructing calibrated prediction
intervals, perhaps improving performance [see Chernozhukov et al., 2021, Shi
et al., 2023, for applications of conformal inference in the context of SC
models].

####  3.2.2 Donor Selection Procedure: Practical Method

In Algorithm 1, we present pseudocode for our spillover detection procedure.
The normalisation step is important for ensuring that the procedure remains
invariant to the scale of the donors.

Algorithm 1 Spillover Detection For Candidate Donor $x_{i}$

Inputs:
• Training data and labels
$\\{x_{1}^{t^{\prime}-1},\dots,x_{N}^{t^{\prime}-1};\,\,x_{i}^{t^{\prime}}\\}$
for all pre-intervention time points $1<t^{\prime}<t$
• Test data and label $\\{x_{1}^{t-1},\dots,x_{N}^{t-1};\,\,x_{i}^{t}\\}$
• Posterior predictive interval (PPI) bound $\phi$ (e.g. 80%)
Outputs:
• Procedure $S1$: Prediction absolute error
$\bigl{|}x^{t}_{i}-\widehat{x}^{t}_{i}\bigr{|}$
• Procedure $S2$: $0$ if $x^{t}_{i}$ is inside $\phi$ PPI, else $1$

1:Normalise the data and labels

2:Regress $x_{i}^{t^{\prime}}$ on
$x_{1}^{t^{\prime}-1},\dots,x_{N}^{t^{\prime}-1}$ $\,\forall\,t^{\prime}<t$ to
obtain $\widehat{h}_{i}$

3:Predict
$\widehat{x}_{i}^{t},\,[\widehat{x}_{i,-}^{t},\,\widehat{x}_{i,+}^{t}]=\widehat{h}_{i}\left(x_{1}^{t-1},\dots,x_{N}^{t-1};\,\phi\right)$

4:Set $A=\bigl{|}x^{t}_{i}-\widehat{x}^{t}_{i}\bigr{|}$

5:Set $B=0$ if $\widehat{x}_{i,-}^{t}<x_{i}^{t}<\widehat{x}_{i,+}^{t}$, else
$B=1$

6:Output $A$ for selection procedure $S1$

7:Output $B$ for selection procedure $S2$

We assume the following linear model for the forecast

$x_{i}^{t^{\prime}}\sim\mathcal{N}\left(\alpha_{i}+\beta_{ij}\,x_{j}^{t^{\prime}-1},\,\sigma_{x_{i}}\right)$
(3)

where $1<t^{\prime}<t$, the index $j$ runs from $1$ to $N$, and we use the
Einstein summation convention such that a repeated index implies summation
$\beta_{ij}\,x_{j}^{t^{\prime}-1}:=\sum_{j}\beta_{ij}\,x_{j}^{t^{\prime}-1}$.
Note that the coefficients $\alpha_{i}$ and $\beta_{ij}$ are the same for
different time points, encoding the fact that $h_{i}$ should be independent of
time. In Section 4.1, we demonstrate the following two approaches, $S1$ and
$S2$, for selecting donors based on Algorithm 1, using the regression model in
Eq. 3:

  * •

$S1$: Select donors with the smallest difference between their actual and
predicted values at the time of intervention, i.e.
$\min_{x_{i}}\bigl{(}\bigl{|}x^{t}_{i}-\beta_{ij}\,x_{j}^{t-1}\bigr{|}\bigr{)}$.

  * •

$S2$: Select donors with values $x^{t}_{i}$ falling within some specified
posterior predictive intervals (e.g. 80%).

###  3.3 Using Excluded Donors To Debias Linear Synthetic Control Models

While the donors excluded by Algorithm 1 aren’t used in constructing the SC
counterfactual, they can still be used to debias SC causal effect estimates,
as we now show. The experiments in Section 4 use the following linear model
for constructing SCs based on pre-intervention time points.

$y^{t}\sim\mathcal{N}\left(\alpha+\beta_{i}\,x_{i}^{t}\,,\,\sigma_{y}\right)$
(4)

As discussed by Shi et al. [2023], in situations where the donors are noisy
(imperfect) proxies of the latents, the SC model in Eq. 4 might fail to
estimate consistent donor weights $\beta_{i}$. Using proximal causal inference
[Tchetgen et al., 2020, Shi et al., 2023], we can leverage proxies not used in
the construction of the SC to debias the estimates. In the proximal causal
inference literature, this debiasing is typically accomplished with two-stage
least squares estimation, or the generalised method of moments. For the
experiment in Figure 4, we opt to jointly model the target and donors as

$\begin{pmatrix}y^{t}\\\ x^{t}_{1}\\\ \vdots\\\
x^{t}_{N}\end{pmatrix}\sim\mathcal{N}\left(\begin{pmatrix}\alpha+\beta_{i}\,x^{t}_{i}\\\
\gamma_{1}+\lambda_{1i}\,z^{t}_{i}\\\ \vdots\\\
\gamma_{N}+\lambda_{Ni}\,z^{t}_{i}\end{pmatrix},\,\Sigma\right)$ (5)

where $z_{i}$ are the (potentially invalid) donors excluded by our selection
procedure. Such a model can be readily estimated using a probabilistic
programming language.

How is it possible that we can use excluded donors $z_{i}$, potentially
impacted by spillover effects from the intervention, when constructing
debiased SC estimates? The key point is that we only ever use pre-intervention
data from these excluded donors, and so the SC estimates are unaffected by the
post-intervention dynamics of the donors $z_{i}$.

To build intuition about the above procedure, consider the following model.
$Y$ is analogous to the target variable, $U$ the latent, and $X$ the donor
(ie. a noisy proxy of the latent $U$). We would like to estimate the donor
weight $\gamma$. Because $U$ is latent, we can’t estimate the expectation on
the right hand side. If we regress $Y$ on $X$ we won’t get a consistent
estimate of $\gamma$ because the errors are correlated with $X$.

$\displaystyle\mathbb{E}(Y|U)=\alpha\,U,\quad\mathbb{E}(X|U)=\beta\,U,$
$\displaystyle\implies\mathbb{E}(Y|U)=\gamma\,\mathbb{E}(X|U)$

Instead, if we have an additional proxy variable $Z$, we can rewrite the model
as

$\displaystyle\mathbb{E}(Y|Z)=\alpha\,\mathbb{E}(U|Z),\quad\mathbb{E}(X|Z)=\beta\,\mathbb{E}(U|Z),$
$\displaystyle\implies\mathbb{E}(Y|Z)=\gamma\,\mathbb{E}(X|Z)$

In this case, $Z$ is observed and so we can estimate the expectation on the
right hand side, and then get a consistent estimate for the donor weight
$\gamma$. One approach for estimation (often used for estimating instrumental
variables) is a two-stage estimator where we first regress $X$ on $Z$ to
estimate $\hat{X}$ (sometimes referred to as a proximal control variable), and
then regress $Y$ on $\hat{X}$. The multivariate model in equation 5 above
effectively combines this two-stage process into a single model [similar
models can be used for estimating instrumental variables, see e.g. Gelman and
Hill, 2006, Section 23.4].

Figure 2: Bias $\widehat{\tau}-\tau$ and 95% Monte Carlo confidence intervals
for 2000 simulated datasets. The data generating process is described in
Appendix B. To construct SCs, we must identify the subset of donors that are
valid. The horizontal axis shows the procedure used to identify this
potentially valid set. For comparison, the case labelled _All_ shows the
expected bias of $1.6$ when we assume that all donors are valid, and the case
labelled _Valid_ shows the bias when we have perfect knowledge about which
donors are valid. Our $S1$ and $S2$ donor selection procedures are described
in Section 3.2.1, and Algorithm 1. The standard deviation of the donor noise
term, $\epsilon_{x}^{t}\sim\mathcal{N}\left(0,\sigma\right)$, increases from
left to right. As the noise increases, the spillover detection procedure is
more likely to return false negatives, which increases the bias due to invalid
donors. Note that even with optimal selection of valid donors, the estimates
can still be biased due to donor noise.  Figure 3: The left panel shows the
bias in the high noise case of Figure 2. Our selection procedures $S1$ and
$S2$ are severely biased due to invalid donors. The right panel shows where we
time average the donor data, in buckets of 5 time points, before passing
through the spillover detection. Note that we do not average pre- and post-
intervention data in the same bucket, and we use the original, non-averaged
data in the SC model. The averaging reduces false negatives such that the
performance is on par with the optimal selection of valid donors. We address
the residual bias in Figure 4.  Figure 4: The left panel shows the bias in the
medium noise case of Figure 2. The estimated causal effects are biased even
with optimal selection of valid donors. This is because the donors are noisy
(imperfect) proxies of the latents, and so conditioning fails to fully close
backdoor paths. As described in Section 3.3, we can debias these estimates
using proximal causal inference to leverage donors excluded by the selection
procedure. The right panel shows that the _Valid_ and $S1$ selection
procedures now give unbiased estimates of the causal effect, although there is
still some bias with procedure $S2$ due to invalid donors. As in Figure 3,
time averaging would reduce the bias in $S2$.  Figure 5: The left panel shows
the bias when the error term for one of the latents shifts _at the same time_
as the intervention. The error term for $U_{1}$ shifts from
$\epsilon_{u_{1}}\sim\mathcal{N}\left(0,1\right)$ pre-intervention, to
$\epsilon_{u_{1}}\sim\mathcal{N}\left(0.5,1\right)$ post-intervention. Our
selection procedure incorrectly flags this as a spillover effect (false
positive), thereby excluding donors that depend on $U_{1}$. The right panel
shows where this latent shift occurs just _after_ the intervention. In this
case our donor selection procedure recovers an unbiased estimate of the causal
effect.

###  3.4 Sensitivity Analysis

In this Section, we discuss the false positive (excluding donors not impacted
by spillover effects), and false negative (including donors impacted by
spillover effects) failure modes of our donor selection procedure. Zeitler et
al. [2023] provide a general framework for sensitivity analysis in non-
parametric SC models, bounding the bias when there are relevant latent
variables with no observed donors as proxies (omitted variable bias). They
also give a formula for calculating these bounds in the case of linear SC
models, which we make use of in Sections 3.4.1, and 3.4.2. In the context of
negative control, Miao et al. [2020] introduce positive control outcome
variables (which are similar to invalid donors), and discuss treating the
spillover effect as a sensitivity parameter to investigate the plausibility of
causal effect estimates. We make use of this approach in Section 3.4.3.

####  3.4.1 Relevant Latents With No Observed Donors

Firstly, we discuss omitted variable bias due to the absence of observed
donors. When there are latent confounding variables with no observed donors to
act as proxies, we cannot close all backdoor paths and so our estimated causal
effect will be biased. For linear SC models, this bias can be bounded using
Eq. 3 from Zeitler et al. [2023]. In particular, let $x_{i}$ be the observed
donors selected to construct the SC, and $\beta_{x_{i}}$ the corresponding
donor weights. The potential omitted variable bias due to unobserved donors is
then

$\text{OV Bias}\leq
N\times\max_{x_{i}}\left(\left|\beta_{x_{i}}\right|\right)\\\
\times\max_{x_{i}}\left(\left|\operatorname{\mathbb{E}}\left(x^{\text{pre}}_{i}\right)-\operatorname{\mathbb{E}}\left(x^{\text{post}}_{i}\right)\right|\right)$
(6)

Note that for this to be a _valid_ bound, we must assume that the observed
donors are at least as important as the unobserved ones [in the sense that the
maximum weight and post-intervention shift for the observed donors is larger
than that of the unobserved donors Zeitler et al., 2023]. This is similar to
assumptions used in propensity-based sensitivity analysis, where to obtain
valid bounds one requires that unobserved confounders are at most as important
to estimation as the observed confounders [Veitch and Zaveri, 2020].

####  3.4.2 Relevant Latents With No Selected Donors

A false positive result from the spillover detection procedure potentially
introduces a related form of omitted variable bias. In particular, if _all_
donors acting as proxies for a relevant latent variable are excluded by the
selection procedure, then the resulting SC causal estimate will be biased, in
a similar manner to Section 3.4.1. However, in this case we can bound the bias
using weaker assumptions than necessary for Eq. 6, because the excluded donors
are actually observed. Let $x_{i}$ be the donors selected to construct the SC,
and $z_{j}$ be the donors excluded by the spillover detection procedure. The
potential omitted variable bias due to false positives is then

$\text{FP Bias}\leq
N\times\max_{x_{i}}\left(\left|\beta_{x_{i}}\right|\right)\\\
\times\max_{z_{j}}\left(\left|\operatorname{\mathbb{E}}\left(z^{\text{pre}}_{j}\right)-\operatorname{\mathbb{E}}\left(z^{\text{post}}_{j}\right)\right|\right)$
(7)

####  3.4.3 Selected Donors Impacted By The Intervention

Finally, we discuss the potential bias introduced by false negatives. The
approach taken here is similar to Section 6 of Miao et al. [2020]. Let $x_{i}$
be the donors selected to construct the SC, and $\tau_{x_{i}}$ the
corresponding spillover effects from the intervention. The potential bias due
to false negatives is then

$\text{FN Bias}\leq
N\times\max_{x_{i}}\left(\left|\beta_{x_{i}}\right|\right)\times\max_{x_{i}}\left(\left|\tau_{x_{i}}\right|\right)$
(8)

We don’t know $\tau_{x_{i}}$, but [following Miao et al., 2020] we can treat
it as a sensitivity parameter to gauge the plausibility of our causal effect
estimate. For example, if we have domain knowledge bounding $\tau_{x_{i}}$
then this can be used in Eq. 8 to bound the bias. We can also judge how large
the spillover effect would have to be in order for the estimated causal effect
to change sign.

##  4 Experiments

In this Section, we illustrate the performance of our donor selection
procedure in different scenarios using simulated data. We also demonstrate the
procedure on real-world data by applying it to semi-synthetic variants of the
German Reunification [Abadie et al., 2015], and California Tobacco Control
[Abadie et al., 2010] datasets.

###  4.1 Simulated Data

We construct 2000 simulated datasets according to the data generating process
described in Appendix B, with large pools of potential donors, most of which
are impacted by spillover effects from the intervention. Each dataset consists
of a target timeseries, 10 latents, and a pool of 1000 potential donors. In
each pool, a random set of $80\%$ of the donors are invalid, impacted by a
spillover effect of $-2$.

For a given dataset, we must first attempt to identify the subset of valid
donors. We refer to the set of donors returned by a selection procedure as the
_potentially valid donors_ (PVDs). Next, in order to simplify comparisons
between different selection procedures, we estimate the SC using a sample of
10 donors from the PVDs. This sampling step ensures that the resulting SC
models all have the same number of parameters, regardless of the number of
PVDs identified by the different selection procedures333 For example, if
procedure $S1$ returns 10 PVDs, and procedure $S2$ returns 50 PVDs, we sample
10 donors from $S2$’s PVDs such that the final SC models both have 10 donors.
. In Appendix C, we compare this approach to one where we estimate SCs using
the full set of PVDs, and employ regularization to enforce sparsity in the
donor weights. For the data generating process considered here, the expected
bias is the same with both approaches. We use the sampling approach to focus
the comparisons on the selection of _valid_ donors, rather than also having to
consider possible differences due to selecting donors based on pre-treatment
fit.

In Figure 2, we show the bias after applying our donor selection procedures
$S1$444 For selection procedure $S1$, we select the 10 donors with the
smallest forecast errors, and so the sampling step discussed previously is not
relevant for this procedure.  and $S2$. For comparison, the case labelled
_All_ shows the bias if we assume that all the donors are valid, and the case
labelled _Valid_ shows the optimal scenario if we had perfect knowledge about
which donors are valid. Our selection procedures are very close to optimal in
both the low and medium donor noise cases. However, the performance degrades
as the donor noise becomes comparable in magnitude to the spillover effect. In
Figure 3, we address this issue and show how time averaging improves
performance when the donors are very noisy. In Figure 4, we leverage excluded
donors to further debias effect estimates in situations with noisy donors, as
discussed in Section 3.3. Finally, in Figure 5, we show how contemporaneous
shifts in the latent distributions can bias our selection procedure, and that
these latent shifts are not an issue if they occur later in the post-
intervention period.

(a)

(b)

Figure 6: (a) Estimated causal effect of German reunification. The
intervention time is indicated by the vertical dotted line, and the shaded
area gives the 95% uncertainty interval. In the top panel, the pool of
potential donors includes a semi-synthetic unit that is a noisy proxy of West
Germany. This invalid donor receives a large weight in the SC model, which
biases the estimate towards zero. The bottom panel shows the effect estimated
after applying our donor selection procedure. The invalid donor is correctly
flagged and excluded from the SC model. In this case, the results are very
similar to Figure 3 from Abadie et al. [2015]. (b) Estimated causal effect of
the California tobacco tax. The top panel includes a semi-synthetic unit that
is a noisy proxy of California, biasing the estimate towards zero. In the
bottom panel, this invalid donor has been excluded by our selection procedure,
and the resulting effect estimate is very similar to Figure 3 from Abadie et
al. [2010].

###  4.2 Semi-Synthetic Data

In this Section, we further validate our donor selection procedure with using
real-world data. In particular, we consider semi-synthetic variants of the
1990 reunification of West and East Germany [Abadie et al., 2015], and the 25
cents tobacco tax increase in California in 1988 [Abadie et al., 2010]. For
each dataset, we introduce a semi-synthetic unit to the pool of potential
donors, constructed to be a noisy proxy of the target as
$x_{\text{syn}}^{t}\sim\mathcal{N}\left(y^{t},\sigma\right)$. Being predictive
of the targets, these invalid donors receive large weights in the SC models
and bias the effect estimates towards zero. This also highlights the critical
distinction between selecting _valid_ donors, and the usual considerations of
selecting donors based on pre-treatment fit.

In Figure 6(a), we show the estimates for the effect of German reunification
on Germany’s per-capita GDP, and in Figure 6(b), the estimates for the effect
of California’s tobacco tax increase on per-capita pack sales. The SC models
in the top panels include the semi-synthetic, invalid donors, which results in
causal effect estimates much closer to zero than the original findings of
Abadie et al. [2010, 2015]. The bottom panels show the effect estimates after
applying our donor selection procedure. In this case, the semi-synthetic,
invalid donors are correctly flagged and excluded from the SC models,
resulting in causal effect estimates that are consistent with Abadie et al.
[2010, 2015].

##  5 Conclusion

In this paper, we presented a practical, theoretically-grounded donor
selection procedure for SC models, aimed at weakening the domain knowledge
requirements for selecting valid donors. This procedure augments partial
knowledge about invalid donors, thereby reducing the burden on the
practitioner to explicitly know that a (potentially very large) pool of donors
is not impacted by spillover effects from the intervention. Working in the
structural causal model framework, we utilised techniques from proximal causal
inference to show that the assumptions necessary for SC identifiability also
facilitate forecasting post-intervention donor values from pre-intervention
data. We used this result to detect potential spillover effects, and exclude
invalid donors when constructing SCs. Furthermore, in the context of recent
works on sensitivity analysis, we discussed bounding the bias due to false
positive and false negative selection errors. We concluded by providing an
empirical demonstration of our selection procedure on both simulated and real-
world datasets.

## References

  * Abadie and Gardeazabal [2003] Alberto Abadie and Javier Gardeazabal.  The economic costs of conflict: A case study of the basque country.  _American economic review_ , 93(1):113–132, 2003.
  * Abadie et al. [2010] Alberto Abadie, Alexis Diamond, and Jens Hainmueller.  Synthetic control methods for comparative case studies: Estimating the effect of california’s tobacco control program.  _Journal of the American statistical Association_ , 105(490):493–505, 2010.
  * Abadie et al. [2015] Alberto Abadie, Alexis Diamond, and Jens Hainmueller.  Comparative politics and the synthetic control method.  _American Journal of Political Science_ , 59(2):495–510, 2015.
  * Athey and Imbens [2017] Susan Athey and Guido W Imbens.  The state of applied econometrics: Causality and policy evaluation.  _Journal of Economic perspectives_ , 31(2):3–32, 2017.
  * Ben-Michael et al. [2021] Eli Ben-Michael, Avi Feller, and Jesse Rothstein.  The augmented synthetic control method.  _Journal of the American Statistical Association_ , 116(536):1789–1803, 2021.  10.1080/01621459.2021.1929245.
  * Betancourt [2021] Michael Betancourt.  Modelling sparsity, 2021.  URL https://betanalpha.github.io/assets/case_studies/modeling_sparsity.html.
  * Brodersen et al. [2015] Kay H Brodersen, Fabian Gallusser, Jim Koehler, Nicolas Remy, Steven L Scott, et al.  Inferring causal impact using bayesian structural time-series models.  _Annals of Applied Statistics_ , 9(1):247–274, 2015.
  * Chernozhukov et al. [2021] Victor Chernozhukov, Kaspar Wüthrich, and Yinchu Zhu.  An exact and robust conformal inference method for counterfactual and synthetic controls.  _Journal of the American Statistical Association_ , 116(536):1849–1864, 2021.
  * Cinelli and Hazlett [2020] Carlos Cinelli and Chad Hazlett.  Making sense of sensitivity: Extending omitted variable bias.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 82(1):39–67, 2020.
  * Cinelli et al. [2019] Carlos Cinelli, Daniel Kumor, Bryant Chen, Judea Pearl, and Elias Bareinboim.  Sensitivity analysis of linear structural causal models.  In _International conference on machine learning_ , pages 1252–1261. PMLR, 2019.
  * Dhir and Lee [2020] Anish Dhir and Ciarán M Lee.  Integrating overlapping datasets using bivariate causal discovery.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 34, pages 3781–3790, 2020.
  * Gelman and Hill [2006] Andrew Gelman and Jennifer Hill.  _Data Analysis Using Regression and Multilevel/Hierarchical Models_.  Analytical Methods for Social Research. Cambridge University Press, 2006.
  * Gilligan-Lee [2020] Ciarán Gilligan-Lee.  Causing trouble.  _New Scientist_ , 246(3279):32–35, 2020.
  * Gilligan-Lee et al. [2022] Ciarán M Gilligan-Lee, Christopher Hart, Jonathan Richens, and Saurabh Johri.  Leveraging directed causal discovery to detect latent common causes in cause-effect pairs.  _IEEE Transactions on Neural Networks and Learning Systems_ , 2022.
  * Imbens et al. [2022] Guido Imbens, Nathan Kallus, Xiaojie Mao, and Yuhao Wang.  Long-term causal inference under persistent confounding via data combination.  _arXiv preprint arXiv:2202.07234_ , 2022.
  * Imbens [2003] Guido W Imbens.  Sensitivity to exogeneity assumptions in program evaluation.  _American Economic Review_ , 93(2):126–132, 2003.
  * Jeunen et al. [2022] Olivier Jeunen, Ciarán M Gilligan-Lee, Rishabh Mehrotra, and Mounia Lalmas.  Disentangling causal effects from sets of interventions in the presence of unobserved confounders.  _arXiv preprint arXiv:2210.05446_ , 2022.
  * Kreif et al. [2016] Noémi Kreif, Richard Grieve, Dominik Hangartner, Alex James Turner, Silviya Nikolova, and Matt Sutton.  Examination of the synthetic control method for evaluating health policies with multiple treated units.  _Health economics_ , 25(12):1514–1528, 2016.
  * Kuroki and Pearl [2014] Manabu Kuroki and Judea Pearl.  Measurement bias and effect restoration in causal inference.  _Biometrika_ , 101(2):423–437, 2014.
  * Lee and Spekkens [2017] Ciarán M Lee and Robert W Spekkens.  Causal inference via algebraic geometry: feasibility tests for functional causal structures with two binary observed variables.  _Journal of Causal Inference_ , 5(2), 2017.
  * Lin et al. [2023] Sicheng Lin, Meng Xu, Xi Zhang, Shih-Kang Chao, Ying-Kai Huang, and Xiaolin Shi.  Balancing approach for causal inference at scale.  _arXiv preprint arXiv:2302.05549_ , 2023.
  * Liu et al. [2023] Jizhou Liu, Eric J. Tchetgen Tchetgen, and Carlos Varjão.  Proximal causal inference for synthetic control with surrogates.  _arXiv preprint arXiv:2023.09527_ , 2023.
  * Miao et al. [2018] Wang Miao, Zhi Geng, and Eric J Tchetgen Tchetgen.  Identifying causal effects with proxy variables of an unmeasured confounder.  _Biometrika_ , 105(4):987–993, 2018.
  * Miao et al. [2020] Wang Miao, Xu Shi, and Eric Tchetgen Tchetgen.  A confounding bridge approach for double negative control inference on causal effects.  _arXiv preprint arXiv:1808.04945_ , 2020.
  * Mitchell and Beauchamp [1988] T. J. Mitchell and J. J. Beauchamp.  Bayesian variable selection in linear regression.  _Journal of the American Statistical Association_ , 83(404):1023–1032, 1988.
  * Nazaret et al. [2023] Achille Nazaret, Claudia Shi, and David M Blei.  On the misspecification of linear assumptions in synthetic control.  _arXiv preprint arXiv:2302.12777_ , 2023.
  * Pearl [2009] Judea Pearl.  _Causality_.  Cambridge university press, 2009.
  * Perov et al. [2020] Yura Perov, Logan Graham, Kostis Gourgoulias, Jonathan Richens, Ciaran Lee, Adam Baker, and Saurabh Johri.  Multiverse: causal reasoning using importance sampling in probabilistic programming.  In _Symposium on advances in approximate bayesian inference_ , pages 1–36. PMLR, 2020.
  * Reynaud et al. [2022] Hadrien Reynaud, Athanasios Vlontzos, Mischa Dombrowski, Ciarán Lee, Arian Beqiri, Paul Leeson, and Bernhard Kainz.  D’artagnan: Counterfactual video generation.  _arXiv preprint arXiv:2206.01651_ , 2022.
  * Richens et al. [2020] Jonathan G Richens, Ciarán M Lee, and Saurabh Johri.  Improving the accuracy of medical diagnosis with causal machine learning.  _Nature communications_ , 11(1):1–9, 2020.
  * Rosenbaum and Rubin [1983] Paul R Rosenbaum and Donald B Rubin.  Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 45(2):212–218, 1983.
  * Shi et al. [2022a] Claudia Shi, Dhanya Sridhar, Vishal Misra, and David Blei.  On the assumptions of synthetic control methods.  In _International Conference on Artificial Intelligence and Statistics_ , pages 7163–7175. PMLR, 2022a.
  * Shi et al. [2022b] Xu Shi, Wang Miao, and Eric Tchetgen Tchetgen.  A selective review of negative control methods in epidemiology.  _arXiv preprint arXiv:2009.05641_ , 2022b.
  * Shi et al. [2023] Xu Shi, Kendrick Li, Wang Miao, Mengtong Hu, and Eric Tchetgen Tchetgen.  Theory for identification and inference with synthetic controls: A proximal causal inference framework.  _arXiv preprint arXiv:2108.13935_ , 2023.
  * Shpitser et al. [2021] Ilya Shpitser, Zach Wood-Doughty, and Eric J Tchetgen Tchetgen.  The proximal id algorithm.  _arXiv preprint arXiv:2108.06818_ , 2021.
  * Tchetgen et al. [2020] Eric J Tchetgen Tchetgen, Andrew Ying, Yifan Cui, Xu Shi, and Wang Miao.  An introduction to proximal causal learning.  _arXiv preprint arXiv:2009.10982_ , 2020.
  * Van Goffrier et al. [2023] Graham Van Goffrier, Lucas Maystre, and Ciarán Gilligan-Lee.  Estimating long-term causal effects from short-term experiments and long-term observational data with unobserved confounding.  _arXiv preprint arXiv:2302.10625_ , 2023.
  * Veitch and Zaveri [2020] Victor Veitch and Anisha Zaveri.  Sense and sensitivity analysis: Simple post-hoc analysis of bias due to unobserved confounding.  _Advances in Neural Information Processing Systems_ , 33:10999–11009, 2020.
  * Vlontzos et al. [2021] Athanasios Vlontzos, Bernhard Kainz, and Ciaran M Gilligan-Lee.  Estimating the probabilities of causation via deep monotonic twin networks.  _arXiv preprint arXiv:2109.01904_ , 2021.
  * Zeitler et al. [2023] Jakob Zeitler, Athanasios Vlontzos, and Ciarán Mark Gilligan-Lee.  Non-parametric identifiability and sensitivity analysis of synthetic control models.  In _2nd Conference on Causal Learning and Reasoning_ , 2023.
  * Zhang and Bareinboim [2022] Junzhe Zhang and Elias Bareinboim.  Can humans be out of the loop?  In _Conference on Causal Learning and Reasoning_ , pages 1010–1025. PMLR, 2022.

## Appendix

##  Appendix A Proof of Theorem 3.1

######  Theorem.

If causal mechanisms are invariant, and the donors
$x^{t-1}_{1},\dots,x^{t-1}_{N}$ are proxies for the latents
$u^{t-1}_{1},\dots,u^{t-1}_{M}$ then, for each donor $x_{i}$, there exists a
unique function $h_{i}$ such that for all time points $t$ we have:

$\operatorname{\mathbb{E}}\left(x^{t}_{i}\right)=\operatorname{\mathbb{E}}\left(h_{i}(x^{t-1}_{1},\dots,x^{t-1}_{N},P(\epsilon^{t}_{x_{i}},\epsilon^{t}_{u}))\right)$
(9)

###### Proof.

Define $x^{t-1}:=x^{t-1}_{1},\dots,x^{t-1}_{N}$,
$u^{t-1}:=u^{t-1}_{1},\dots,u^{t-1}_{M}$, and
$\epsilon^{t}_{i}:=(\epsilon^{t}_{x_{i}},\epsilon^{t}_{u})$ for ease of
exposition. First, note that we can write the causal mechanism for $x^{t}_{i}$
as

$P^{t}(x^{t}_{i}\mid u^{t},\epsilon_{x_{i}}^{t})=P^{t}(x^{t}_{i}\mid
u^{t-1},\epsilon_{u}^{t},\epsilon_{x_{i}}^{t})=P^{t}(x_{i}^{t}\mid
u^{t-1},\epsilon^{t}_{i})$ (10)

where the first equality follows from the fact $u^{t}$ is a deterministic
function of $u^{t-1}$, and $\epsilon_{u}^{t}$. Now, using the proxy variable
completeness condition (Def. 3.2), we can relate the causal mechanism for
$x_{i}^{t}$ to $x^{t-1}$ via a function555A proof for the existence of
$H_{i}^{t}$ can be found in section F of the supplementary material in Shi et
al. 2023, as well as appendix A of Zeitler et al. 2023. A simple example of
such a function is: $P(Y)=\int P(Y\mid X)P(X)dx=\int H(Y,X)P(X)dx.$
$H^{t}_{i}$ as

$\displaystyle P^{t}(x_{i}^{t}\mid u^{t-1},\epsilon^{t}_{i})$
$\displaystyle=\int
H_{i}^{t}(x_{i}^{t},x^{t-1},\epsilon^{t}_{i})\underbrace{P^{t}(x^{t-1}\mid
u^{t-1},\epsilon^{t}_{i})}_{P^{t}(x^{t-1}\mid u^{t-1})}dx^{t-1}$ (11)

This implies that

$\displaystyle\operatorname{\mathbb{E}}\left(x_{i}^{t}\mid u^{t-1}\right)$
$\displaystyle=\iiint
x_{i}^{t}\,H_{i}^{t}(x_{i}^{t},x^{t-1},\epsilon^{t}_{i})P^{t}(x^{t-1}\mid
u^{t-1})P(\epsilon^{t}_{i})\,d\epsilon^{t}_{i}\,dx_{i}^{t}\,dx^{t-1}$
$\displaystyle=\int{P^{t}(x^{t-1}\mid
u^{t-1})}\int\underbrace{\left[\int{x_{i}^{t}\,H_{i}^{t}(x_{i}^{t},x^{t-1},\epsilon^{t}_{i})}dx_{i}^{t}\right]}_{g_{i}^{t}(x^{t-1},\epsilon^{t}_{i})}P(\epsilon^{t}_{i})\,d\epsilon^{t}_{i}\,dx^{t-1}$
$\displaystyle=\int\underbrace{\operatorname{\mathbb{E}}_{P(\epsilon^{t}_{i})}\left(g_{i}^{t}(x^{t-1},\epsilon^{t}_{i})\right)}_{h_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))}P^{t}(x^{t-1}\mid
u^{t-1})\,dx^{t-1}$
$\displaystyle=\operatorname{\mathbb{E}}\left(h_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))\mid
u^{t-1}\right)$

Marginalising over $u^{t-1}$ yields the result that

$\operatorname{\mathbb{E}}\left(x_{i}^{t}\right)=\operatorname{\mathbb{E}}\left(h_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))\right)$

Next, we prove that $h^{t}_{i}$ doesn’t depend on time, by showing that the
solution to the integral equation in Eq. 11 for time point $t$ is also a
solution for any other time point $t^{\prime}$. Remember that causal
mechanisms are invariant and so don’t depend on $t$. Consider the left hand
side of Eq. 11:

$P^{t}(x_{i}^{t}\mid u^{t-1},\epsilon^{t}_{i})=\int
H_{i}^{t}(x_{i}^{t},x^{t-1},\epsilon^{t}_{i})P^{t}(x^{t-1}\mid
u^{t-1})\,dx^{t-1}.$

By Eq. 10, $P^{t}(x_{i}^{t}\mid u^{t-1},\epsilon^{t}_{i})$ is a causal
mechanism and so doesn’t depend on $t$. Now consider the $P^{t}(x^{t-1}\mid
u^{t-1})$ term under the integral on the right hand side. This distribution
Markov factorises according to the structure of the DAG in Figure 1(a),
resulting in products of causal mechanisms. Hence, a solution to the integral
equation for one time point $t$ is a solution for any other time point.

Finally, we prove uniqueness of $h_{i}^{t}$ for a given time point, as this
implies there exists a unique function $h_{i}$ for all time points. Suppose
that $h_{i}^{t}$ and $\widetilde{h}_{i}^{t}$ are both solutions at time $t$,
such that

$\operatorname{\mathbb{E}}\left(h_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))\mid
u^{t-1}\right)=\operatorname{\mathbb{E}}\left(\widetilde{h}_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))\mid
u^{t-1}\right)$

Therefore,
$\operatorname{\mathbb{E}}\left(h_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))-\widetilde{h}_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))\mid
u^{t-1}\right)=0$, and the proxy variable completeness condition (Def. 3.2)
implies that
$h_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))=\widetilde{h}_{i}^{t}(x^{t-1},P(\epsilon^{t}_{i}))$,
completing the proof. ∎

The result can be generalised to the case where the evolution depends on $T$
time points by modifying the SCSCM such that

  1. 1.

$u_{i}^{t}=m_{i}^{t}(u_{i}^{t-1},\dots,u_{i}^{t-1-T},\epsilon^{t}_{u_{i}})$

  2. 2.

$x^{t}_{i}=f_{i}^{t}(u_{1}^{t},\dots,u_{M}^{t},\dots,u_{1}^{t-T},\dots,u_{M}^{t-T},\epsilon^{t}_{x_{i}})$

  3. 3.

$y^{t}=g^{t}(u_{1}^{t},\dots,u_{M}^{t},\dots,u_{1}^{t-T},\dots,u_{M}^{t-T},I^{t},\epsilon^{t}_{y})$

and updating the proxy variable completeness condition (Def. 3.2) accordingly.

Figure 7: Example of a target variable $y$ from a simulated dataset. The
intervention happens at time 0, increasing $y$ by $\tau=2$.

##  Appendix B Simulated Data

We construct simulated datasets in Section 4.1 according to the following data
generating process [similar to the local linear trend model with long-term
slope from Brodersen et al., 2015]. As mentioned in the main text, we use the
Einstein summation convention such that a repeated index implies summation,
e.g. $\beta_{ij}\,u_{j}^{t}:=\sum_{j}\beta_{ij}\,u_{j}^{t}$.

$\displaystyle u_{i}^{t+1}$
$\displaystyle\sim\mathcal{N}\left(u_{i}^{t}+\delta_{i}^{t}\,,\,\sigma_{u}\right)$
$\displaystyle\delta_{i}^{t+1}$
$\displaystyle\sim\mathcal{N}\left(S_{i}+\rho_{i}\left(\delta_{i}^{t}-S_{i}\right),\,\sigma_{\delta}\right)$
$\displaystyle y^{t}$
$\displaystyle\sim\mathcal{N}\left(\alpha_{i}\,u_{i}^{t}+\tau
I^{t}\,,\,\sigma_{y}\right)$ $\displaystyle x_{i}^{t}$
$\displaystyle\sim\mathcal{N}\left(\beta_{ij}\,u_{j}^{t}+\tau_{x_{i}}I^{t}\,,\,\sigma_{x}\right)$

We generate 2000 datasets for each panel in Figure 2. A dataset consists of 1
target $y$, 10 latents $u_{i}$, and 1000 potential donors $x_{i}$ (out of
which we select 10 donors for constructing SCs). The time-series data have 100
time points pre-intervention, and 30 time points post-intervention. The causal
effect of the intervention on the target is $\tau=2$, and 80% of the potential
donors are invalid, with spillover effects $\tau_{x_{i}}=-2$ (the remaining
20% are valid, with $\tau_{x_{i}}=0$). In the above, $I^{t}$ is an indicator
variable for the intervention that is 0 pre-intervention and 1 post-
intervention. The long-term slope is sampled as
$S_{i}\sim\mathcal{N}(0.1,0.1)$, with
$\rho_{i}\sim\mathcal{U}\left(0,1\right)$ interpolating between this slope and
a random walk. We set $\sigma_{u}=1$, $\sigma_{\delta}=\sigma_{y}=0.1$, and
$\sigma_{x}\in\\{0.1,0.5,1.0\\}$ for the low, medium, and high donor noise
levels (the high noise level is comparable in magnitude to $\tau$). For
simplicity, we set the coefficients as $\alpha_{i}=1$, and $\beta_{ij}=1$
(except for the datasets demonstrating the latent shift in Figure 5, where we
set $\beta_{i1}=0$ for a random subset of valid donors). Figure 7 provides an
illustrated example of a target variable generated by the process described
above.

##  Appendix C Sparse Donor Weights

Figure 8: Low noise case of Figure 2. The filled circles show the results
discussed in the main text, where each SC model is estimated using 10 donors
sampled from the set of PVDs. The open circles show the alternative sparse
regularization approach, where each SC model is estimated using the full set
of PVDs, and we employ sparsity-inducing priors (Eq. 12) to control the number
of non-zero weights. The expected bias is identical for both approaches. The
variance in the _All_ case is slightly lower. This is because the number of
donors with non-zero weight typically ends up being a bit larger than 10, and
so the fraction of invalid donors is more concentrated around 80%.

As described in Section 4.1, for each of the 2000 simulated datasets, we
construct SCs by sampling 10 donors from the set of potentially valid donors
(PVDs). The set of PVDs is a subset of the pool of 1000 potential donors, and
depends on the specific selection procedure used to identify valid donors (ie.
_All_ , _Valid_ , _S1_ , or _S2_). As an alternative to sampling 10 donors, we
can construct SCs using the full set of PVDs, and employ regularization to
select approximately 10 donors to have non-zero weights based on the pre-
treatment fit [e.g., Brodersen et al., 2015, Ben-Michael et al., 2021]. With
this sparse regularization approach, the total number of parameters is equal
to the number of PVDs identified by the selection procedure. For the
simulations described in Appendix B, SC models estimated using the full set of
PVDs have 1000 parameters in the _All_ case, 200 parameters in the _Valid_
case, 10 parameters in the _S1_ case, and approximately 200 parameters in the
_S2_ case (but the exact number varies across datasets).

To enforce sparsity in the donor weights, we set the prior to be a discrete
mixture of normal distributions [Betancourt, 2021].

$\beta_{i}\sim\eta\,\mathcal{N}\left(0,\sigma_{1}\right)+\left(1-\eta\right)\mathcal{N}\left(0,\sigma_{2}\right),\qquad
0<\eta<1,\qquad\sigma_{1}\ll\sigma_{2}$ (12)

The parameter $\eta$ controls how weights tend to cluster close to zero via
the narrow distribution, and can be interpreted as the expected fraction of
donors with effectively zero weight. This is similar to the spike and slab
prior of Mitchell and Beauchamp [1988]. However, replacing the exact zero
values with a narrow distribution of “irrelevant” values facilitates
straightforward estimation with standard probabilistic programming languages.

In Figure 8, we compare this sparse regularization approach to the sampling
approach from the main text. For this data generating process, the expected
bias is identical. The variance in the _All_ case is slightly lower. This is
because the number of donors with non-zero weight typically ends up being a
bit larger than 10, and so the fraction of invalid donors is more concentrated
around 80%.