#  Same Root Different Leaves: Time Series and Cross-Sectional Methods in
Panel Data

Dennis<EMAIL_ADDRESS>[ Peng
<EMAIL_ADDRESS>[ Jasjeet
<EMAIL_ADDRESS>[ Bin<EMAIL_ADDRESS>[
Simons Institute for the Theory of Computing, University of California,
Berkeley  Department of Statistics, University of California, Berkeley
Departments of Statistics & Data Science and Political Science, Yale
University  Departments of Statistics and EECS, University of California,
Berkeley

###### Abstract

A central goal in social science is to evaluate the causal effect of a policy.
One dominant approach is through panel data analysis in which the behaviors of
multiple units are observed over time. The information across time and space
motivates two general approaches: (i) horizontal regression (i.e.,
unconfoundedness), which exploits time series patterns, and (ii) vertical
regression (e.g., synthetic controls), which exploits cross-sectional
patterns. Conventional wisdom states that the two approaches are fundamentally
different. We establish this position to be partly false for estimation but
generally true for inference. In particular, we prove that both approaches
yield identical point estimates under several standard settings. For the same
point estimate, however, each approach quantifies uncertainty with respect to
a distinct estimand. The confidence interval developed for one estimand may
have incorrect coverage for another. This emphasizes that the source of
randomness that researchers assume has direct implications for the accuracy of
inference.

horizontal regression,

vertical regression,

unconfoundedness,

synthetic controls,

causal inference,

minimum norm estimators,

###### keywords:

We sincerely thank Alberto Abadie, Avi Feller, Guido Imbens, and Devavrat Shah
for their thoughtful comments and insightful feedback. We gratefully
acknowledge support from NSF grants 1945136, 1953191, 2022448, 2023505 on
Collaborative Research: Foundations of Data Science Institute (FODSI), and ONR
grant N00014-17-1-2176. The data and code to reproduce the results in this
article are available at https://github.com/deshen24/panel-data-regressions.

##  1 Introduction

In a seminal paper, [Abadie and Gardeazabal (2003)] set out to investigate the
economic impact of terrorism in Basque Country. Prior to the outset of
terrorist activity in the early 1970’s, Basque Country was considered to be
one of the wealthiest regions in Spain. After thirty years of turmoil,
however, its economic activity dropped substantially relative to its
neighboring regions. Although intuition affirms that Basque Country’s economic
downturn can be attributed, at least partially, to its political and civil
unrest, it is difficult to quantitatively isolate the economic costs of
conflict. In response to this challenge, [Abadie and Gardeazabal (2003)]
introduced the synthetic controls framework. At its core, synthetic controls
constructs a synthetic Basque Country from a weighted composition of control
regions that are largely unaffected by the instability to estimate Basque
Country’s economic evolution in the absence of terrorism. This novel concept
has inspired an entire subliterature within econometrics that is “arguably the
most important innovation in the policy evaluation literature in the last 15
years” (Athey and Imbens, 2017).

Researchers have historically tackled problems of this flavor using repeated
observations of units across time, i.e., panel data, where a subset of units
are exposed to a treatment during some time periods while the other units are
unaffected. In the study above, the per capita gross domestic product (GDP) of
$17$ Spanish regions are measured from 1955–1998. Basque Country is the sole
treated unit and the remaining regions are the control units; the pre- and
post-treatment periods are defined as the time horizons before and after the
first wave of terrorist activity, respectively.

Synthetic controls has become a cornerstone for panel studies in recent years
and across numerous fields. Beforehand, the unconfoundedness approach
(Rosenbaum and Rubin, 1983; Imbens and Wooldridge, 2009) served as a common
workhorse. Whereas synthetic controls posits a relation between treated and
control units that is stable across time, unconfoundedness posits a relation
between treated and pretreatment periods that is stable across units.
Accordingly, synthetic controls exploits cross-sectional correlation patterns
while unconfoundedness exploits time series correlation patterns. Considering
the panel data format, unconfoundedness and synthetic controls based methods
are commonly referred to as horizontal (HZ) and vertical (VT) regressions,
respectively. Given their conceptual and computational distinctions, the two
approaches are considered to be fundamentally different (Athey et al., 2021).

Yet, contrary to conventional wisdom, it turns out that HZ and VT regressions
can yield identical point estimates. As Figure 1(a) shows, when the regression
models are learned via ordinary least squares (OLS) or principal component
regression (PCR), then the two approaches produce the same economic evolution
for Basque Country in the absence of terrorism. Figure 1(b), by contrast,
shows that when the regression models are learned via lasso or lie within the
simplex—as proposed by Abadie and Gardeazabal (2003) for VT regression—then
the two approaches output contrasting economic trajectories.

(a) Symmetric regressions.

(b) Asymmetric regressions.

Figure 1:  1(a): Estimates of OLS with minimum $\ell_{2}$-norm and PCR. 1(b):
Estimates of lasso and simplex regression. HZ and VT estimates correspond to
colored solid and dashed-dotted lines, respectively. The outset of terrorism
is the vertical line and Basque Country’s observed GDP is in solid black.

Curiously, Figure 2 indicates that even when the two regressions arrive at the
same point estimate, the confidence intervals can be markedly different under
different sources of randomness.

(a) OLS under HZ model.

(b) OLS under VT model.

(c) PCR under HZ model.

(d) PCR under VT model.

Figure 2:  Confidence intervals for OLS (top) and PCR (bottom) constructed
from HZ-based (left) and VT-based (right) generative models, which are
precisely defined in Section 4.1.1.

The juxtaposition of these figures beg two questions:

Q1: “When are HZ and VT point estimates identical?” Q2: “When the point
estimates are identical, how does the source of randomness impact inference?”

Contribution. This article tackles Q1–Q2 from first principles. In this
endeavor, we begin by classifying several widely studied regression
formulations into (i) a symmetric class that yields identical point estimates
and (ii) an asymmetric class that yields contrasting point estimates. Within
the symmetric class, we study properties of the estimator with randomness
stemming from (i) time series patterns, (ii) cross-sectional patterns, and
(iii) both patterns. We conduct our analysis from a (i) model-based
perspective, which attributes randomness to the potential outcomes, and a (ii)
design-based perspective, which attributes randomness to the treatment
assignment mechanism. In both frameworks, we find that the source of
randomness has large implications for the estimand and inference. Under the
model-based framework, we construct confidence intervals for each source of
randomness. Through data-inspired simulations and empirical applications, we
demonstrate that the confidence interval developed for one estimand often has
incorrect coverage for another estimand. Taken together, our results emphasize
that the source of randomness that researchers assume has direct implications
for the accuracy of the inference that can be conducted.

Organization. Section 2 overviews the panel data framework. Sections 3–4
provide one set of answers for Q1–Q2. Section 5 illustrates concepts developed
in this article. Section 6 summarizes our findings. Details of simulations and
empirical applications, select discussions, and mathematical proofs are
relegated to the appendix.

Notation. Let $\boldsymbol{I}$ be the identity matrix. Let $\boldsymbol{1}$
and $\boldsymbol{0}$ be the vectors of ones and zeros, respectively. The
curled inequality denotes $\succeq$ the generalized inequality, i.e.,
componentwise inequality between vectors and matrix inequality between
symmetric matrices. Let $\circ$ denote the componentwise product. For vectors
$\boldsymbol{a}$ and $\boldsymbol{b}$, let $\langle a,b\rangle=a^{\prime}b$
denote the inner product. Let $\operatorname{tr}(\boldsymbol{A})$ denote the
trace of $\boldsymbol{A}$. We define $0/0=0$ when applicable.

##  2 The Panel Data Framework

We anchor on the Basque study to introduce the panel data framework. Panel
data contains observations of $N$ units over $T$ time periods. The Basque
study, for instance, consists of per capita GDP across $N=17$ Spanish regions
over $T=43$ years. In each time period $t$, each unit $i$ is characterized by
two potential outcomes, $Y_{it}(0)$ and $Y_{it}(1)$, which correspond to its
outcome in the absence and presence of a binary treatment, respectively. The
potential outcomes framework posits that each region possesses two possible
levels of economic activity each year, one that is immune to terrorism and
another that is affected by terrorism. In reality, however, we can only
observe one economic state, $Y_{it}(0)$ or $Y_{it}(1)$—this is the fundamental
challenge of causal inference.

Let $Y_{it}$ be the observed outcome. Often, we observe all $N$ units without
treatment (control) for $T_{0}$ time periods, i.e., $Y_{it}=Y_{it}(0)$ for all
$i\leq N$ and $t\leq T_{0}$. For the remaining $T_{1}=T-T_{0}$ time periods,
$N_{1}$ units receive treatment while the remaining $N_{0}=N-N_{1}$ units
remain under control, i.e., if we arbitrarily label the first $N_{0}$ units as
the control group, then $Y_{it}=Y_{it}(1)$ for all $i>N_{0}$ and $t>T_{0}$,
and $Y_{it}=Y_{it}(0)$ for all $i\leq N_{0}$ and $t>T_{0}$. In our study,
Basque Country is the single treated unit, thus $N_{1}=1$ and $N_{0}=16$. The
first wave of terrorist activity partitions the time horizon into pre- and
post-treatment periods of lengths $T_{0}=15$ and $T_{1}=28$ years,
respectively.

For ease of exposition, this article considers a single treated unit and
single treated period indexed by the $N$th unit and $T$th time period,
respectively. However, our results hold for any $(i,t)$ pair where $i>N_{0}$
is a treated unit and $t>T_{0}$ is a treated period. We organize our observed
control data into an $N\times T$ matrix, $\boldsymbol{Y}=[Y_{it}]$, as shown
in Figure 3. In our example, $\boldsymbol{y}_{N}=[Y_{Nt}:t\leq
T_{0}]\in\mathbb{R}^{T_{0}}$ represents Basque Country’s economic evolution
prior to the outset of terrorism; $\boldsymbol{Y}_{0}=[Y_{it}:i\leq
N_{0},t\leq T_{0}]\in\mathbb{R}^{N_{0}\times T_{0}}$ represents the control
regions’ economic evolution prior to the outset of terrorism; and
$\boldsymbol{y}_{T}=[Y_{iT}:i\leq N_{0}]\in\mathbb{R}^{N_{0}}$ represents the
control regions’ economic evolution after the outset of terrorism. Our object
of interest is Basque Country’s counterfactual GDP in the absence of
terrorism, $Y_{NT}(0)$.

Figure 3:  Panel data format with rows and columns indexed by units and time,
respectively.

###  2.1 Time Series Versus Cross-Sectional Based Regressions

The information across time and space motivates two natural ways to impute the
missing $(N,T)$th entry. These perspectives are explored in two large and
mostly separate bodies of work (Athey et al., 2021).

####  2.1.1 Horizontal Regression and Unconfoundedness

The unconfoundedness literature operates on the concept that “history is a
guide to the future”. As such, unconfoundedness methods express outcomes in
the treated period as a weighted composition of outcomes in the pretreatment
periods. This is carried out by regressing the control units’ treated period
outcomes $\boldsymbol{y}_{T}$ on its lagged outcomes $\boldsymbol{Y}_{0}$ and
applying the learned regression coefficients to the treated unit’s lagged
outcomes $\boldsymbol{y}_{N}$ to predict the missing $(N,T)$th outcome.
Following Athey et al. (2021), we refer to such methods as horizontal (HZ)
regression.

####  2.1.2 Vertical Regression and Synthetic Controls

The synthetic controls literature is built on the concept that “similar units
behave similarly”. Therefore, synthetic controls methods express the treated
unit’s outcomes as a weighted composition of control units’ outcomes. This is
carried out by regressing the treated unit’s lagged outcomes
$\boldsymbol{y}_{N}$ on the control units’ lagged outcomes
$\boldsymbol{Y}^{\prime}_{0}$ and applying the learned regression coefficients
to the control units’ treated period outcomes $\boldsymbol{y}_{T}$ to predict
the missing $(N,T)$th outcome. Following Athey et al. (2021), we refer to such
methods as vertical (VT) regression.

####  2.1.3 Conventional Wisdom

The asymmetry between HZ and VT regressions has created the conception that
they are fundamentally different approaches (Athey et al., 2021). In fact, the
unregularized forms of HZ and VT regressions are cautioned against when $T>N$
and $N>T$, respectively (Abadie et al., 2015; Doudchenko and Imbens, 2016; Li
and Bell, 2017; Athey et al., 2021). With regularization, however, Athey et
al. (2021) argues the two approaches can be applied to the same setting. In
turn, this allows the two approaches to be systematically compared through
methods such as cross-validation.

In parallel, the growth rates of the two literatures have also exhibited
asymmetry. While the development of the unconfoundedness literature has
seemingly plateaued, the synthetic controls literature continues to rapidly
expand. Across many domains, synthetic controls based methods are arguably the
de facto approach for panel studies.

##  3 Point Estimation

Q1: “When are HZ and VT point estimates identical?”

We tackle Q1 by studying the finite-sample estimation properties of HZ and VT
regressions. We denote the singular value decomposition of
$\boldsymbol{Y}_{0}$ as
$\boldsymbol{Y}_{0}=\sum_{\ell=1}^{R}s_{\ell}\boldsymbol{u}_{\ell}\boldsymbol{v}^{\prime}_{\ell}=\boldsymbol{U}\boldsymbol{S}\boldsymbol{V}^{\prime}$,
where $\boldsymbol{u}_{\ell}\in\mathbb{R}^{N_{0}}$ and
$\boldsymbol{v}_{\ell}\in\mathbb{R}^{T_{0}}$ are the left and right singular
vectors, respectively, $s_{\ell}\in\mathbb{R}$ are the ordered singular
values, and $R=\text{rank}(\boldsymbol{Y}_{0})\leq\min\\{N_{0},T_{0}\\}$.
$\boldsymbol{U}\in\mathbb{R}^{N_{0}\times R}$ and
$\boldsymbol{V}\in\mathbb{R}^{T_{0}\times R}$ denote the matrices formed by
the left and right singular vectors, respectively, and
$\boldsymbol{S}\in\mathbb{R}^{R\times R}$ is the diagonal matrix of singular
values. The Moore-Penrose pseudoinverse of $\boldsymbol{Y}_{0}$ is
$\boldsymbol{Y}_{0}^{\dagger}=\sum_{\ell=1}^{R}(1/s_{\ell})\boldsymbol{v}_{\ell}\boldsymbol{u}^{\prime}_{\ell}=\boldsymbol{V}\boldsymbol{S}^{-1}\boldsymbol{U}^{\prime}$.
Critically, we do not place any assumptions on the relative magnitudes of $N$
and $T$.

###  3.1 Classifying Notable Regression Formulations

We present several of the most widely studied regression formulations in the
HZ and VT literatures. This list is far from exhaustive given the vastness of
these literatures.

####  3.1.1 Description of Estimation Strategies

Penalized regression. A large class of penalized regressions are expressed as
follows:

  * (a)

HZ regression: for $\lambda_{1},\lambda_{2}\geq 0$,

$\displaystyle\widehat{\boldsymbol{\alpha}}=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}}~{}\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}\|_{2}^{2}+\lambda_{1}\|\boldsymbol{\alpha}\|_{1}+\lambda_{2}\|\boldsymbol{\alpha}\|_{2}^{2}$
(1)
$\displaystyle\widehat{Y}_{NT}^{\text{hz}}(0)=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle.$
(2)

  * (b)

VT regression: for $\lambda_{1},\lambda_{2}\geq 0$,

$\displaystyle\widehat{\boldsymbol{\beta}}=\operatorname*{\arg\\!\min}_{\boldsymbol{\beta}}~{}\|\boldsymbol{y}_{N}-\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}\|_{2}^{2}+\lambda_{1}\|\boldsymbol{\beta}\|_{1}+\lambda_{2}\|\boldsymbol{\beta}\|_{2}^{2}$
(3)
$\displaystyle\widehat{Y}_{NT}^{\text{vt}}(0)=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle.$
(4)

We overview common choices for $(\lambda_{1},\lambda_{2})$ and describe the
corresponding strategy.

I: Ordinary least squares (OLS). Arguably, the mother of all regressions is
OLS, where $\lambda_{1}=\lambda_{2}=0$. OLS is an unconstrained problem with
possibly infinitely many solutions. OLS has been analyzed in numerous works on
panel studies, including Hsiao et al. (2012), Li and Bell (2017), and Li
(2020).

II: Principal component regression (PCR). To formalize PCR, let

$\displaystyle\boldsymbol{Y}_{0}^{(k)}=\sum_{\ell=1}^{k}s_{\ell}\boldsymbol{u}_{\ell}\boldsymbol{v}^{\prime}_{\ell}$
(5)

denote the rank $k<R$ approximation of $\boldsymbol{Y}_{0}$ that retains the
top $k$ principal components. HZ and VT PCR corresponds to replacing
$\boldsymbol{Y}_{0}$ with $\boldsymbol{Y}_{0}^{(k)}$ within (1) and (3),
respectively, with $\lambda_{1}=\lambda_{2}=0$. In words, PCR first finds a
$k$ dimensional representation of the covariate matrix via principal component
analysis; then, PCR performs OLS with the compressed $k$ dimensional
covariates. Within the synthetic controls literature, Amjad et al. (2018,
2019) and Agarwal et al. (2021) utilize PCR.

III: Ridge regression. Consider ridge regression, where $\lambda_{1}=0$ and
$\lambda_{2}>0$. When $\boldsymbol{Y}_{0}$ is rank deficient, the gram matrix,
i.e., $\boldsymbol{Y}^{\prime}_{0}\boldsymbol{Y}_{0}$ for HZ regression and
$\boldsymbol{Y}_{0}\boldsymbol{Y}^{\prime}_{0}$ for VT regression, is ill-
conditioned. This often discourages the usage of OLS. In these settings, ridge
provides a remedy by adding a ridge on the diagonal of the gram matrix, which
increases all eigenvalues by $\lambda_{2}$, thus removing the singularity
problem. Ben-Michael et al. (2021) explores the properties of a doubly robust
estimator that utilizes HZ ridge regression.

IV: Lasso regression. Consider lasso regression, where $\lambda_{1}>0$ and
$\lambda_{2}=0$. Lasso has become a popular tool for estimating sparse linear
coefficients in high-dimensional regimes. Because the lasso criterion not
strictly convex, there are possibly infinitely many solutions. Thus, for our
analysis of lasso only, we make the mild assumption that the entries of
$\boldsymbol{Y}_{0}$ are drawn from a continuous distribution. As established
in Tibshirani (2013), this guarantees the lasso solution to be unique. Several
notable works in the synthetic controls literature, e.g., Li and Bell (2017),
Carvalho et al. (2018), and Chernozhukov et al. (2021), analyze the lasso.

V: Elastic net regression. Mixing both $\ell_{1}$ and $\ell_{2}$-penalties,
i.e., $\lambda_{1},\lambda_{2}>0$, is known as elastic net. At a high level,
elastic net selects variables similar to the lasso, but deals with correlated
variables more gracefully as with ridge. When $\lambda_{2}>0$, the criterion
is strictly convex so the solution is unique. Doudchenko and Imbens (2016)
propose an elastic net synthetic controls variant.

Constrained regression.

VI: Simplex regression. The next formulation constrains the regression weights
to lie within the simplex, i.e., the weights are nonnegative and sum to one:

  * (a)

HZ regression: for $\lambda\geq 0$,

$\displaystyle\widehat{\boldsymbol{\alpha}}=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}}~{}\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}\|_{2}^{2}+\lambda\|\boldsymbol{\alpha}\|_{2}^{2}\,\,\text{~{}subject
to~{}}\,\boldsymbol{\alpha}^{\prime}\boldsymbol{1}=1,\boldsymbol{\alpha}\succeq\boldsymbol{0}$
(6)
$\displaystyle\widehat{Y}_{NT}^{\text{hz}}(0)=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle.$
(7)

  * (b)

VT regression: for $\lambda\geq 0$,

$\displaystyle\widehat{\boldsymbol{\beta}}=\operatorname*{\arg\\!\min}_{\boldsymbol{\beta}}~{}\|\boldsymbol{y}_{N}-\boldsymbol{Y}_{0}^{\prime}\boldsymbol{\beta}\|_{2}^{2}+\lambda\|\boldsymbol{\beta}\|_{2}^{2}\,\,\text{~{}subject
to~{}}\,\boldsymbol{\beta}^{\prime}\boldsymbol{1}=1,\boldsymbol{\beta}\succeq\boldsymbol{0}$
(8)
$\displaystyle\widehat{Y}_{NT}^{\text{vt}}(0)=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle.$
(9)

We consider a vanishing $\ell_{2}$ penalty since $\lambda=0$ (standard
formulation) can induce multiple minima (Abadie and L’Hour, 2021). When
$\lambda>0$, the criterion becomes strictly convex and the solution is unique.
Simplex regression is the original formulation set forth in the pioneering
works of Abadie and Gardeazabal (2003); Abadie et al. (2010, 2015), and its
properties continue to be actively studied today. Attractive aspects of
simplex regression include interpretability, sparsity, and transparency
(Abadie, 2021).

####  3.1.2 Classification Results

To answer Q1, we classify the regression formulations into (i) a symmetric
class, where HZ and VT point estimates agree, and (ii) an asymmetric class,
where HZ and VT point estimates disagree. We use the shorthand
$\textsf{HZ}=\textsf{VT}$ if the two approaches produce identical point
estimates and $\textsf{HZ}\neq\textsf{VT}$ otherwise.

I: Symmetric class. We first state the symmetric formulations.

######  Theorem 1.

$\emph{{HZ}}=\emph{{VT}}$ for (i) OLS with
$(\widehat{\boldsymbol{\alpha}},\widehat{\boldsymbol{\beta}})$ as the minimum
$\ell_{2}$-norm solutions:

$\displaystyle\widehat{Y}_{NT}^{\emph{hz}}(0)$
$\displaystyle=\widehat{Y}_{NT}^{\emph{vt}}(0)=\langle\boldsymbol{y}_{N},\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{y}_{T}\rangle=\sum_{\ell=1}^{R}(1/s_{\ell})\langle\boldsymbol{y}_{N},\boldsymbol{v}_{\ell}\rangle\langle\boldsymbol{u}_{\ell},\boldsymbol{y}_{T}\rangle;$
(10)

(ii) PCR with the same choice of $k<R$:

$\displaystyle\widehat{Y}_{NT}^{\emph{hz}}(0)$
$\displaystyle=\widehat{Y}_{NT}^{\emph{vt}}(0)=\langle\boldsymbol{y}_{N},(\boldsymbol{Y}_{0}^{(k)})^{\dagger}\boldsymbol{y}_{T}\rangle=\sum_{\ell=1}^{k}(1/s_{\ell})\langle\boldsymbol{y}_{N},\boldsymbol{v}_{\ell}\rangle\langle\boldsymbol{u}_{\ell},\boldsymbol{y}_{T}\rangle;$
(11)

(iii) ridge regression with the same choice of $\lambda_{2}>0$:

$\displaystyle\widehat{Y}_{NT}^{\emph{hz}}(0)$
$\displaystyle=\widehat{Y}_{NT}^{\emph{vt}}(0)=\langle\boldsymbol{y}_{N},(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda_{2}\boldsymbol{I})^{-1}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}\rangle=\sum_{\ell=1}^{R}\frac{s_{\ell}}{s_{\ell}^{2}+\lambda_{2}}\langle\boldsymbol{y}_{N},\boldsymbol{v}_{\ell}\rangle\langle\boldsymbol{u}_{\ell},\boldsymbol{y}_{T}\rangle.$
(12)

Theorem 1 might seem familiar at first glance. As observed in Abadie et al.
(2015) and Ben-Michael et al. (2021, Lemma 2), the point estimates associated
with HZ OLS and HZ ridge can be written as linear combinations of the elements
in $\boldsymbol{y}_{T}$, which take the same linear forms as the corresponding
VT point estimates. However, their results stop short of establishing
numerical equivalence as in Theorem 1. From this perspective, Theorem 1 is
perhaps surprising as it takes the next step forward in contradicting the
notion that the two regressions are fundamentally different. In particular,
Theorem 1 proves the same point estimate is derived from both approaches
whenever the regression model belongs to the symmetric class, namely (i) OLS
with minimum $\ell_{2}$-norm, (ii) PCR for the same choice of $k$, and (iii)
ridge for the same choice of $\lambda_{2}$. In this view, HZ and VT
regressions are not perspectives at duel—they are dual perspectives.

We emphasize that Theorem 1 holds for any data configuration. As such, it
clarifies that HZ and VT OLS are not invalid when $T>N$ and $N>T$,
respectively, as previously believed. In fact, the OLS estimate can even be
written as
$\langle\widehat{\boldsymbol{\alpha}},\boldsymbol{Y}^{\prime}_{0}\widehat{\boldsymbol{\beta}}\rangle$,
which incorporates both regression models. The origin of the prior
misconception may have come from the fact that infinitely many solutions exist
when $\boldsymbol{Y}_{0}$ is rank deficient. Among these solutions, however,
is the unique minimum $\ell_{2}$-norm model, which is arguably sufficient for
inference (Shao and Deng, 2012). This is also the solution when the problem is
optimized via gradient descent, a ubiquitous optimizer in practice. Phenomena
of this form are known as “implicit regularization”, where the optimization
algorithm is biased towards a particular solution even though the bias is not
explicit in the objective function (Neyshabur et al., 2015; Gunasekar et al.,
2017).

Through its connection to the $\ell_{2}$-penalty, the minimum $\ell_{2}$-norm
also offers a high-level intuition for the root of symmetry. More
specifically, observe that the ridge model converges to the OLS model with
minimum $\ell_{2}$-norm as $\lambda_{2}\rightarrow 0$. Since the PCR model is
precisely the OLS minimum $\ell_{2}$-norm model that is restricted to the
space spanned by the top $k$ principal components, we conjecture that the
geometry of the $\ell_{2}$-ball is a likely source for HZ and VT estimation
symmetry.

II: Asymmetric class. Next, we state the class of formulations that fracture
symmetry.

######  Theorem 2.

$\emph{{HZ}}\neq\emph{{VT}}$ for (i) lasso, (ii) elastic net, and (iii)
simplex regression.

To examine the implications of Theorem 2, we observe that the common thread
between the objective functions in the asymmetric class is a penalty or
constraint that promotes sparse models. Such regularizers are noticeably
absent in the symmetric formulations. This highlights an interesting trade-
off—while sparsity is widely considered to be a salient feature, the
geometries of the $\ell_{1}$-ball and simplex that encourage sparsity are also
the likely sources of HZ and VT estimation asymmetry.

###  3.2 Doubly Robust Estimators

In recent years, there has been a surge of interest in doubly robust
estimators. Within panel data, we discuss two prominent works that are rising
in popularity.

####  3.2.1 Synthetic Difference-in-Differences

An important approach that continues to dominate empirical work in panel data
is the difference-in-differences (DID) estimator (Ashenfelter, 1978). At a
high level, DID posits an additive outcome model with unit- and time-specific
fixed effects, known more colloquially as the “parallel trends” assumption.
The recent work of Arkhangelsky et al. (2021) anchors on the DID principle and
brings in concepts from the unconfoundedness and synthetic controls
literatures to derive a doubly robust estimator called synthetic difference-
in-differences (SDID). In our setting, the SDID prediction for the missing
$(N,T)$th potential outcome can be written as

$\displaystyle\widehat{Y}^{\text{sdid}}_{NT}(0)$ $\displaystyle=\sum_{i\leq
N_{0}}\widehat{\beta}_{i}Y_{iT}+\sum_{t\leq
T_{0}}\widehat{\alpha}_{t}Y_{Nt}-\sum_{i\leq N_{0}}\sum_{t\leq
T_{0}}\widehat{\beta}_{i}\widehat{\alpha}_{t}Y_{it}$ (13)
$\displaystyle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle+\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle-\langle\widehat{\boldsymbol{\beta}},\boldsymbol{Y}_{0}\widehat{\boldsymbol{\alpha}}\rangle,$
(14)

where $\widehat{\boldsymbol{\alpha}}$ and $\widehat{\boldsymbol{\beta}}$
represent general HZ and VT models, respectively. The authors note that
$\widehat{\boldsymbol{\alpha}}=(1/T_{0})\boldsymbol{1}$ and
$\widehat{\boldsymbol{\beta}}=(1/N_{0})\boldsymbol{1}$ recovers DID. Moving
beyond simple DID to performing a weighted two-way bias removal, the authors
propose to learn $\widehat{\boldsymbol{\alpha}}$ via simplex regression and
$\widehat{\boldsymbol{\beta}}$ via simplex regression with an
$\ell_{2}$-penalty.

####  3.2.2 Augmented Synthetic Controls

Another notable work is that of Ben-Michael et al. (2021). The authors
introduce the augmented synthetic control (ASC) estimator, which uses an
outcome model to correct the bias induced by the classical synthetic controls
estimator.111See Abadie and L’Hour (2021) for a bias correction of synthetic
controls through matching. Concretely, the ASC estimator predicts the missing
$(N,T)$th potential outcome as

$\displaystyle\widehat{Y}^{\text{asc}}_{NT}(0)$
$\displaystyle=\widehat{M}_{NT}(0)+\sum_{i\leq
N_{0}}\widehat{\beta}_{i}(Y_{iT}-\widehat{M}_{iT}(0)),$ (15)

where $\widehat{M}_{iT}(0)$ is the estimator for the $(i,T)$th entry. The
authors instantiate

$\displaystyle\widehat{M}_{iT}(0)$ $\displaystyle=\sum_{t\leq
T_{0}}\widehat{\alpha}_{t}Y_{it}.$ (16)

Plugging the HZ outcome model in (16) into (15) then gives

$\displaystyle\widehat{Y}^{\text{asc}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle+\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle-\langle\widehat{\boldsymbol{\beta}},\boldsymbol{Y}_{0}\widehat{\boldsymbol{\alpha}}\rangle.$
(17)

We consider this particular variant of ASC since it takes the same form as
SDID, as seen in (14). In contrast to Arkhangelsky et al. (2021), Ben-Michael
et al. (2021) learns $\widehat{\boldsymbol{\alpha}}$ via ridge regression and
$\widehat{\boldsymbol{\beta}}$ via simplex regression.

####  3.2.3 When Doubly Robust Estimators are No Longer Doubly Robust

We leverage Theorem 1 to study the estimation properties of SDID and ASC when
$(\widehat{\boldsymbol{\alpha}},\widehat{\boldsymbol{\beta}})$ are learned via
OLS and PCR.

######  Corollary 1.

$\emph{{SDID}}=\emph{{ASC}}=\emph{{HZ}}=\emph{{VT}}$ for (i)
$(\widehat{\boldsymbol{\alpha}},\widehat{\boldsymbol{\beta}})$ as the OLS
minimum $\ell_{2}$-norm solutions and (ii)
$(\widehat{\boldsymbol{\alpha}},\widehat{\boldsymbol{\beta}})$ as the PCR
solutions with the same choice of $k<R$.

Corollary 1 states that a researcher who uses the implicitly regularized (but
explicitly unconstrained) variants of SDID and ASC inevitably arrive at the
same point estimate as their colleague who simply uses HZ or VT OLS. The same
phenomena occurs for PCR. From this, we observe that SDID and ASC can lose
their weighted double-differencing effects under certain formulations.
However, Corollary 1 is not to discredit either approach. The fact remains
that both methods allow researchers to naturally and simultaneously encode
their knowledge of time and unit specific structures into the estimator. SDID
and ASC warrant further studies and careful consideration.

###  3.3 Intercepts

Intercepts can be included in the HZ regression model by modifying the
$\ell_{2}$-errors in (1) and (6) as
$\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}-\alpha_{0}\boldsymbol{1}\|_{2}^{2}$;
similarly, they can be included in the VT regression model by modifying
$\ell_{2}$-errors in (3) and (8) as
$\|\boldsymbol{y}_{N}-\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}-\beta_{0}\boldsymbol{1}\|_{2}^{2}$.
To date, there is still a lack of general consensus on the inclusion of
intercepts within the synthetic controls literature. We attempt to shed light
on the role of intercepts.

######  Corollary 2.

$\emph{{HZ}}\neq\emph{{VT}}$ for (i) OLS, (ii) PCR, and (iii) ridge with
intercepts.

We develop an intuition for Proposition 2 by interpreting intercepts in panel
studies. A nonzero time intercept, $\alpha_{0}$, imposes a permanent constant
difference between the treated and pretreatment periods; a nonzero unit
intercept, $\beta_{0}$, imposes a permanent constant difference between the
treated and control units. These systematic structures then create an
asymmetry between the two regressions. Below, we propose a methodology based
on centering the data that allows for intercepts yet retains symmetry.

####  3.3.1 Including Intercepts and Retaining Symmetry through Data Centering

Let $\boldsymbol{Y}_{0}$ be twice centered, i.e., the rows and columns of
$\boldsymbol{Y}_{0}$ are mean zero. This can be satisfied by applying
$\boldsymbol{I}-(1/N_{0})\boldsymbol{1}\boldsymbol{1}^{\prime}$ and
$\boldsymbol{I}-(1/T_{0})\boldsymbol{1}\boldsymbol{1}^{\prime}$ to the left
and right, respectively, of $\boldsymbol{Y}_{0}$. Consider the following
modified formulations.

  * (a)

HZ regression: for $\lambda\geq 0$,

$\displaystyle(\widehat{\alpha}_{0},\widehat{\alpha}_{1},\widehat{\boldsymbol{\alpha}})=\operatorname*{\arg\\!\min}_{(\alpha_{0},\alpha_{1},\boldsymbol{\alpha})}~{}\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}-\alpha_{0}\boldsymbol{1}\|_{2}^{2}+\|\boldsymbol{y}_{N}-\alpha_{1}\boldsymbol{1}\|_{2}^{2}+\lambda\|\boldsymbol{\alpha}\|_{2}^{2}$
(18)
$\displaystyle\widehat{Y}_{NT}^{\text{hz}}(0)=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle+\widehat{\alpha}_{0}+\widehat{\alpha}_{1}.$
(19)

  * (b)

VT regression: for $\lambda\geq 0$,

$\displaystyle(\widehat{\beta}_{0},\widehat{\beta}_{1},\widehat{\boldsymbol{\beta}})=\operatorname*{\arg\\!\min}_{(\beta_{0},\beta_{1},\boldsymbol{\beta})}~{}\|\boldsymbol{y}_{N}-\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}-\beta_{0}\boldsymbol{1}\|_{2}^{2}+\|\boldsymbol{y}_{T}-\beta_{1}\boldsymbol{1}\|_{2}^{2}+\lambda\|\boldsymbol{\beta}\|_{2}^{2}$
(20)
$\displaystyle\widehat{Y}_{NT}^{\text{vt}}(0)=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle+\widehat{\beta}_{0}+\widehat{\beta}_{1}.$
(21)

Similar to before, OLS corresponds to $\lambda=0$, PCR corresponds to OLS with
$\boldsymbol{Y}_{0}^{(k)}$ for $k<R$ in place of $\boldsymbol{Y}_{0}$, and
ridge regression corresponds to any $\lambda>0$.

######  Corollary 3.

$\emph{{HZ}}=\emph{{VT}}$ for the symmetric estimators in Theorem 1 under the
formulations set in (18) and (20) with $\boldsymbol{Y}_{0}$ being twice
centered.

We inspect (19) and (21) to understand the implications of Corollary 3. First,
we recall Theorem 1, which establishes that the HZ and VT estimates share the
same “base” estimate, i.e.,
$\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle$.
Next, we note that
$\widehat{\alpha}_{0}=\widehat{\beta}_{1}=(1/N_{0})\boldsymbol{1}^{\prime}\boldsymbol{y}_{T}$
and
$\widehat{\beta}_{0}=\widehat{\alpha}_{1}=(1/T_{0})\boldsymbol{1}^{\prime}\boldsymbol{y}_{N}$,
which correspond to the time and unit fixed effects, respectively.
Intuitively, the modified point estimates in (19) and (21) include both fixed
effect models to compensate for $\boldsymbol{Y}_{0}$ being twice centered.
Putting everything together, the modified HZ and VT point estimates are
identical.

##  4 Inference

Q2: “When the HZ and VT point estimates are identical, how does the source of
randomness impact inference?”

To answer Q2, we study the inferential properties of the counterfactual
prediction. Formal discussions for classical inference require an explicit
postulation of where the randomness arises from. This article takes both a (i)
model-based approach, which makes assumptions about the distribution of the
potential outcomes, and a (ii) design-based approach, which makes assumptions
about the assignment mechanism of treatment. We start by taking a model-based
route that is similar in parts to the recent works of Li (2020), Cattaneo et
al. (2021), and Chernozhukov et al. (2021, 2022) from the synthetic controls
literature. We then transition to a design-based route that is paved by the
ideas introduced in Bottmer et al. (2021).

For ease of exposition, we focus on OLS and its minimum $\ell_{2}$-norm
solutions, i.e.,
$\widehat{\boldsymbol{\alpha}}=\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{y}_{T}$
and
$\widehat{\boldsymbol{\beta}}=(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{y}_{N}$.
As such, we define
$\widehat{Y}_{NT}(0)=\widehat{Y}^{\text{hz}}_{NT}(0)=\widehat{Y}^{\text{vt}}_{NT}(0)$,
which is justified under Theorem 1. To improve readability, we present several
results informally and provide their precise versions in Appendix A.

###  4.1 Model-Based Inference

Within the model-based framework, we consider a classical regression model.
This postulation is not always plausible but it is useful to dissect its
implications for the role of randomness in conducting inference.

####  4.1.1 Generative Models

We now study properties of $\widehat{Y}_{NT}(0)$ from three different sources
of randomness.

I: Horizontal model. The HZ model focuses on time series correlation patterns.

######  Assumption 1.

Conditional on $(\boldsymbol{y}_{N},\boldsymbol{Y}_{0})$, we have

$\displaystyle Y_{iT}=\sum_{t\leq
T_{0}}\alpha^{*}_{t}Y_{it}+\varepsilon_{iT},\quad i=1,\dots,N_{0},$ (22)

where $\boldsymbol{\alpha}^{*}$ is a vector of unknown coefficients and
$\varepsilon_{iT}$ is an idiosyncratic error term.

Assumption 1 motivates the HZ approach, which models time series patterns as
the source of randomness. Under the HZ model, the statistical uncertainty of
$\widehat{Y}_{NT}(0)$ is governed by the construction of
$\widehat{\boldsymbol{\alpha}}$ from
$(\boldsymbol{y}_{T},\boldsymbol{Y}_{0})$, i.e., the in-sample uncertainty.

######  Assumption 2.

$\\{\varepsilon_{iT}\\}_{i=1}^{N_{0}}$ has zero mean and is independent over
$i=1,\dots,N_{0}$, conditional on $(\boldsymbol{y}_{N},\boldsymbol{Y}_{0})$.

Assumption 2 states that the errors have zero mean and thus the regressors,
i.e., lagged outcomes, are uncorrelated with the errors; this is known in the
literature as strict exogeneity. The errors are also conditionally independent
across space.

II: Vertical model. The VT model focuses on cross-sectional correlation
patterns.

######  Assumption 3.

Conditional on $(\boldsymbol{y}_{T},\boldsymbol{Y}_{0})$, we have

$\displaystyle Y_{Nt}=\sum_{i\leq
N_{0}}\beta^{*}_{i}Y_{it}+\varepsilon_{Nt},\quad t=1,\dots,T_{0},$ (23)

where $\boldsymbol{\beta}^{*}$ is a vector of unknown coefficients and
$\varepsilon_{Nt}$ is an idiosyncratic error term.

Assumption 3 is analogous to Assumption 1 with the source of randomness here
emanating from cross-sectional patterns. Hence, the statistical uncertainty of
$\widehat{Y}_{NT}(0)$ under the VT model is governed by the construction of
$\widehat{\boldsymbol{\beta}}$ from $(\boldsymbol{y}_{N},\boldsymbol{Y}_{0})$.

######  Assumption 4.

$\\{\varepsilon_{Nt}\\}_{t=1}^{T_{0}}$ has zero mean and is independent over
$t=1,\dots,T_{0}$, conditional on $(\boldsymbol{y}_{T},\boldsymbol{Y}_{0})$.

Assumption 4 is analogous to Assumption 2 with the errors here being
conditionally independent across time.

III: Mixed model. We introduce a new model that mixes aspects of the HZ and VT
models. At a high level, the mixed model accounts for randomness along both
dimensions of the data. This comes at the price of placing additional
constraints on the stochastic properties of the errors.

######  Assumption 5.

Conditional on $\boldsymbol{Y}_{0}$, we have

$\displaystyle Y_{iT}=\sum_{t\leq
T_{0}}\alpha^{*}_{t}Y_{it}+\varepsilon_{iT},\quad i=1,\dots,N_{0},$ (24)
$\displaystyle Y_{Nt}=\sum_{i\leq
N_{0}}\beta^{*}_{i}Y_{it}+\varepsilon_{Nt},\quad t=1,\dots,T_{0},$ (25)

where $(\boldsymbol{\alpha}^{*},\varepsilon_{iT})$ and
$(\boldsymbol{\beta}^{*},\varepsilon_{Nt})$ are defined as in Assumptions 1
and 3, respectively.

(24) and (25) correspond to Assumptions 1 and 3, respectively. Collectively,
they model time series and cross-sectional patterns as two distinct sources of
randomness. Thus, the statistical uncertainty of $\widehat{Y}_{NT}(0)$ under
the mixed model is governed by the constructions of both
$\widehat{\boldsymbol{\alpha}}$ and $\widehat{\boldsymbol{\beta}}$.

######  Assumption 6.

$\\{\varepsilon_{iT}\\}_{i=1}^{N_{0}}$ and
$\\{\varepsilon_{Nt}\\}_{t=1}^{T_{0}}$ have zero mean and are independent over
$i=1,\dots,N_{0}$ and $t=1,\dots,T_{0}$, conditional on $\boldsymbol{Y}_{0}$.

Assumption 6 combines Assumptions 2 and 4. In words, it states that
$\boldsymbol{Y}_{0}$ contains all measured confounders and the errors are
independent across both time and space. We leave a proper analysis under
dependent errors as future work.

####  4.1.2 Inferential Properties

Equipped with our three models, we are ready to provide one set of answers to
Q2. In what follows, we define the error covariance matrices as (i)
$\boldsymbol{\Sigma}^{\text{hz}}_{T}=\operatorname{Cov}(\boldsymbol{\varepsilon}_{T}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0})$,
(ii)
$\boldsymbol{\Sigma}^{\text{vt}}_{N}=\operatorname{Cov}(\boldsymbol{\varepsilon}_{N}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0})$,
(iii)
$\boldsymbol{\Sigma}^{\text{mix}}_{T}=\operatorname{Cov}(\boldsymbol{\varepsilon}_{T}|\boldsymbol{Y}_{0})$,
and (iv)
$\boldsymbol{\Sigma}^{\text{mix}}_{N}=\operatorname{Cov}(\boldsymbol{\varepsilon}_{N}|\boldsymbol{Y}_{0})$,
where $\boldsymbol{\varepsilon}_{T}=[\varepsilon_{iT}:i\leq N_{0}]$ and
$\boldsymbol{\varepsilon}_{N}=[\varepsilon_{Nt}:t\leq T_{0}]$.

######  Theorem 3.

(i) _[ HZ model]_ Under Assumptions 1–2 and suitable moment conditions, we
have

$\displaystyle\frac{\widehat{Y}_{NT}(0)-\mu_{0}^{\emph{hz}}}{\sqrt{v_{0}^{\emph{hz}}}}\xrightarrow{d}\mathcal{N}(0,1),$
(26)

where
$\mu_{0}^{\emph{hz}}=\langle\boldsymbol{y}_{N},\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*}\rangle$
and
$v_{0}^{\emph{hz}}=\widehat{\boldsymbol{\beta}}^{\prime}\boldsymbol{\Sigma}^{\emph{hz}}_{T}\widehat{\boldsymbol{\beta}}$.
(ii) _[ VT model]_ Under Assumptions 3–4 and suitable moment conditions, we
have

$\displaystyle\frac{\widehat{Y}_{NT}(0)-\mu_{0}^{\emph{vt}}}{\sqrt{v^{\emph{vt}}_{0}}}\xrightarrow{d}\mathcal{N}(0,1),$
(27)

where
$\mu_{0}^{\emph{vt}}=\langle\boldsymbol{y}_{T},\boldsymbol{H}^{u}\boldsymbol{\beta}^{*}\rangle$
and
$v^{\emph{vt}}_{0}=\widehat{\boldsymbol{\alpha}}^{\prime}\boldsymbol{\Sigma}^{\emph{vt}}_{N}\widehat{\boldsymbol{\alpha}}$.
(iii) _[Mixed model]_ Under Assumptions 5–6 and suitable moment conditions, we
have

$\displaystyle\frac{\widehat{Y}_{NT}(0)-\mu^{\emph{mix}}_{0}}{\sqrt{v^{\emph{mix}}_{0}}}\xrightarrow{d}\mathcal{N}(0,1),$
(28)

where
$\mu^{\emph{mix}}_{0}=\langle\boldsymbol{\alpha}^{*},\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}^{*}\rangle$
and

$v^{\emph{mix}}_{0}=(\boldsymbol{H}^{u}\boldsymbol{\beta}^{*})^{\prime}\boldsymbol{\Sigma}^{\emph{mix}}_{T}(\boldsymbol{H}^{u}\boldsymbol{\beta}^{*})+(\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*})^{\prime}\boldsymbol{\Sigma}^{\emph{mix}}_{N}(\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*})+\operatorname{tr}(\boldsymbol{Y}^{\dagger}_{0}\boldsymbol{\Sigma}^{\emph{mix}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{\Sigma}^{\emph{mix}}_{N}).$

Theorem 3 highlights that each model measures uncertainty with respect to a
different estimand. In particular, Theorem 3 states that the asymptotic
variance is controlled by time series patterns under the HZ model, cross-
sectional patterns under the VT model, and both correlation patterns under the
mixed model. This clarifies that the source of randomness has substantive
implications for the estimand and inference.

####  4.1.3 Confidence Intervals

Theorem 3 motivates separate HZ, VT, and mixed confidence intervals: for
$\theta\in(0,1)$,

$\displaystyle\mu_{0}^{\text{hz}}$
$\displaystyle\in\left[\widehat{Y}_{NT}(0)~{}\pm~{}z_{\frac{\theta}{2}}\sqrt{\widehat{v}^{\text{hz}}_{0}}\right],$
(29) $\displaystyle\mu_{0}^{\text{vt}}$
$\displaystyle\in\left[\widehat{Y}_{NT}(0)~{}\pm~{}z_{\frac{\theta}{2}}\sqrt{\widehat{v}^{\text{vt}}_{0}}\right],$
(30) $\displaystyle\mu_{0}^{\text{mix}}$
$\displaystyle\in\left[\widehat{Y}_{NT}(0)~{}\pm~{}z_{\frac{\theta}{2}}\sqrt{\widehat{v}^{\text{mix}}_{0}}\right],$
(31)

where $z_{\frac{\theta}{2}}$ is the upper $\theta/2$ quantile of
$\mathcal{N}(0,1)$, and
$(\widehat{v}^{\text{hz}}_{0},\widehat{v}^{\text{vt}}_{0},\widehat{v}^{\text{mix}}_{0})$
are the estimators of
$(v^{\text{hz}}_{0},v^{\text{vt}}_{0},v^{\text{mix}}_{0})$. We construct

$\displaystyle\widehat{v}^{\text{hz}}_{0}=\widehat{\boldsymbol{\beta}}^{\prime}\widehat{\boldsymbol{\Sigma}}_{T}\widehat{\boldsymbol{\beta}},\quad\widehat{v}^{\text{vt}}_{0}=\widehat{\boldsymbol{\alpha}}^{\prime}\widehat{\boldsymbol{\Sigma}}_{N}\widehat{\boldsymbol{\alpha}},\quad\widehat{v}^{\text{mix}}_{0}=\widehat{v}_{0}^{\text{hz}}+\widehat{v}_{0}^{\text{vt}}-\operatorname{tr}(\boldsymbol{Y}_{0}^{\dagger}\widehat{\boldsymbol{\Sigma}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\widehat{\boldsymbol{\Sigma}}_{N}),$
(32)

where $\widehat{\boldsymbol{\Sigma}}_{T}$ and
$\widehat{\boldsymbol{\Sigma}}_{N}$ are the estimators of
$(\boldsymbol{\Sigma}^{\text{hz}}_{T},\boldsymbol{\Sigma}^{\text{mix}}_{T})$
and
$(\boldsymbol{\Sigma}^{\text{vt}}_{N},\boldsymbol{\Sigma}^{\text{mix}}_{N})$,
respectively. We precisely define them under homoskedastic and heteroskedastic
errors below. To reduce ambiguity, we index
$(\widehat{v}^{\text{hz}}_{0},\widehat{v}^{\text{vt}}_{0},\widehat{v}^{\text{mix}}_{0})$
by the covariance estimator. We also denote
$\boldsymbol{H}^{u}=\boldsymbol{U}\boldsymbol{U}^{\prime}$ and
$\boldsymbol{H}^{u}_{\perp}=\boldsymbol{I}-\boldsymbol{H}^{u}$. With this
notation, the HZ and VT in-sample errors can be written as
$\boldsymbol{H}^{u}_{\perp}\boldsymbol{y}_{T}=\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\widehat{\boldsymbol{\alpha}}$
and
$\boldsymbol{H}^{v}_{\perp}\boldsymbol{y}_{N}=\boldsymbol{y}_{N}-\boldsymbol{Y}^{\prime}_{0}\widehat{\boldsymbol{\beta}}$,
respectively.

It is clear from (32) that
$(\widehat{v}^{\text{hz}}_{0},\widehat{v}^{\text{vt}}_{0})$ are plug-in
estimators for $(v^{\text{hz}}_{0},v^{\text{vt}}_{0})$. As such, we discuss
$\widehat{v}^{\text{mix}}_{0}$ with respect to $v^{\text{mix}}_{0}$. Recall
$\widehat{\boldsymbol{\alpha}}=\boldsymbol{H}^{v}\widehat{\boldsymbol{\alpha}}$
and
$\widehat{\boldsymbol{\beta}}=\boldsymbol{H}^{u}\widehat{\boldsymbol{\beta}}$
by construction. To justify the negative trace in
$\widehat{v}^{\text{mix}}_{0}$, note that $\widehat{v}_{0}^{\text{hz}}$ is a
quadratic involving $(\boldsymbol{y}_{N},\boldsymbol{y}_{T})$. Since both
quantities are random, the expectation of $\widehat{v}_{0}^{\text{hz}}$
induces an additional term that precisely corresponds to the trace term in
$v_{0}^{\text{mix}}$. The same property holds for
$\widehat{v}_{0}^{\text{vt}}$. Thus, $\widehat{v}^{\text{mix}}_{0}$ corrects
for this bias via the negative trace.

Homoskedastic errors. Consider $\boldsymbol{\Sigma}^{\text{hz}}_{T}$ with
identical diagonal elements, i.e.,
$\boldsymbol{\Sigma}^{\text{hz}}_{T}=(\sigma^{\text{hz}}_{T})^{2}\boldsymbol{I}$,
where
$(\sigma^{\text{hz}}_{T})^{2}=\operatorname{Var}(\varepsilon_{iT}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0})$
for $i=1,\dots,N_{0}$. Let
$(\boldsymbol{\Sigma}^{\text{vt}}_{N},\boldsymbol{\Sigma}^{\text{mix}}_{T},\boldsymbol{\Sigma}^{\text{mix}}_{N})$
be defined analogously. We use the standard variance estimators

$\displaystyle\widehat{\boldsymbol{\Sigma}}_{T}^{\text{homo}}=\frac{1}{N_{0}-R}\|\boldsymbol{H}^{u}_{\perp}\boldsymbol{y}_{T}\|_{2}^{2}\boldsymbol{I},$
(33)
$\displaystyle\widehat{\boldsymbol{\Sigma}}_{N}^{\text{homo}}=\frac{1}{T_{0}-R}\|\boldsymbol{H}^{v}_{\perp}\boldsymbol{y}_{N}\|_{2}^{2}\boldsymbol{I},$
(34)

where $R=\text{rank}(\boldsymbol{Y}_{0})$, which can be computed as
$R=\operatorname{tr}(\boldsymbol{H}^{u})=\operatorname{tr}(\boldsymbol{H}^{v})$.

######  Lemma 1.

Consider homoskedastic errors. (i) _[ HZ model]_ Under Assumptions 1–2, we
have

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{T}^{\emph{homo}}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{hz}}_{T}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{hz},\emph{homo}}_{0}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]=v^{\emph{hz}}_{0}.$
(35)

(ii) _[ VT model]_ Under Assumptions 3–4, we have

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{homo}}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{vt}}_{N}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{vt},\emph{homo}}_{0}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]=v^{\emph{vt}}_{0}.$
(36)

(iii) _[Mixed model]_ Under Assumptions 5–6, we have

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{T}^{\emph{homo}}|\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{mix}}_{T},\quad\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{homo}}|\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{mix}}_{N},\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{mix},\emph{homo}}_{0}|\boldsymbol{Y}_{0}]=v^{\emph{mix}}_{0}.$
(37)

Lemma 1 is a well known result within the OLS literature, albeit it is
typically formalized under the stricter full column rank assumption.

Heteroskedastic errors. We adopt two strategies for the heteroskedastic
setting.

I: Jackknife. The first estimator is based on the jackknife. Traditionally,
the jackknife estimates the covariance of the regression coefficients
$(\widehat{\boldsymbol{\alpha}},\widehat{\boldsymbol{\beta}})$. By analyzing
said estimates, we derive the following:

$\displaystyle\widehat{\boldsymbol{\Sigma}}_{T}^{\text{jack}}$
$\displaystyle=\text{diag}\left(\left[\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{I}\right]^{\dagger}\left[\boldsymbol{H}^{u}_{\perp}\boldsymbol{y}_{T}\circ\boldsymbol{H}^{u}_{\perp}\boldsymbol{y}_{T}\right]\right)$
(38) $\displaystyle\widehat{\boldsymbol{\Sigma}}_{N}^{\text{jack}}$
$\displaystyle=\text{diag}\left(\left[\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{I}\right]^{\dagger}\left[\boldsymbol{H}^{v}_{\perp}\boldsymbol{y}_{N}\circ\boldsymbol{H}^{v}_{\perp}\boldsymbol{y}_{N}\right]\right).$
(39)

######  Lemma 2.

Consider heteroskedastic errors. (i) _[ HZ model]_ Let Assumptions 1–2 hold.
If
$(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{I})$
is nonsingular, then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}^{\emph{jack}}_{T}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]\succeq\boldsymbol{\Sigma}^{\emph{hz}}_{T}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{hz},\emph{jack}}_{0}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]\geq
v^{\emph{hz}}_{0}.$ (40)

(ii) _[ VT model]_ Let Assumptions 3–4 hold. If
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{I})$
is nonsingular, then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{jack}}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]\succeq\boldsymbol{\Sigma}^{\emph{vt}}_{N}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{vt},\emph{jack}}_{0}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]\geq
v^{\emph{vt}}_{0}.$ (41)

(iii) _[Mixed model]_ Let Assumptions 5–6 hold. If
$(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{I})$
and
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{I})$
are nonsingular, then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{T}^{\emph{jack}}|\boldsymbol{Y}_{0}]\succeq\boldsymbol{\Sigma}^{\emph{mix}}_{T},\quad\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{jack}}|\boldsymbol{Y}_{0}]\succeq\boldsymbol{\Sigma}^{\emph{mix}}_{N},\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{mix},\emph{jack}}_{0}|\boldsymbol{Y}_{0}]\geq
v^{\emph{mix}}_{0}.$ (42)

Lemma 2 establishes that the jackknife is conservative, provided
$(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{I})$
and
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{I})$
are nonsingular. Strictly speaking, the jackknife is well defined if these
quantities are singular, as seen through the pseudoinverse in (38) and (39).
Lemma 2 considers the nonsingular case for simplicity. We remark that
$\max_{\ell}H^{u}_{\ell\ell}<1$ and $\max_{\ell}H^{v}_{\ell\ell}<1$ are
sufficient conditions for invertibility.

II: HRK-estimator. Next, we consider the covariance estimator proposed by
Hartley et al. (1969). We index this estimator by the authors, Hartley-Rao-
Kiefer:

$\displaystyle\widehat{\boldsymbol{\Sigma}}_{T}^{\text{HRK}}$
$\displaystyle=\text{diag}\left(\left[\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp}\right]^{-1}\left[\boldsymbol{H}^{u}_{\perp}\boldsymbol{y}_{T}\circ\boldsymbol{H}^{u}_{\perp}\boldsymbol{y}_{T}\right]\right)$
(43) $\displaystyle\widehat{\boldsymbol{\Sigma}}_{N}^{\text{HRK}}$
$\displaystyle=\text{diag}\left(\left[\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp}\right]^{-1}\left[\boldsymbol{H}^{v}_{\perp}\boldsymbol{y}_{N}\circ\boldsymbol{H}^{v}_{\perp}\boldsymbol{y}_{N}\right]\right).$
(44)

######  Lemma 3.

Consider heteroskedastic errors. (i) _[ HZ model]_ Let Assumptions 1–2 hold.
If $(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp})$ is
nonsingular, then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}^{\emph{HRK}}_{T}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{hz}}_{T}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{hz},\emph{HRK}}_{0}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]=v^{\emph{hz}}_{0}.$
(45)

(ii) _[ VT model]_ Let Assumptions 3–4 hold. If
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp})$ is nonsingular,
then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{HRK}}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{vt}}_{N}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{vt},\emph{HRK}}_{0}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]=v^{\emph{vt}}_{0}.$
(46)

(iii) _[Mixed model]_ Let Assumptions 5–6 hold. If
$(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp})$ and
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp})$ are nonsingular,
then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{T}^{\emph{HRK}}|\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{mix}}_{T},\quad\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{HRK}}|\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{mix}}_{N},\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{mix},\emph{HRK}}_{0}|\boldsymbol{Y}_{0}]=v^{\emph{mix}}_{0}.$
(47)

Lemma 3 establishes that the HRK estimator is unbiased, provided
$(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp})$ and
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp})$ are invertible.
For the former quantity, $\max_{\ell}H^{u}_{\ell\ell}<1/2$ is a sufficient
condition for invertibility. Since $\operatorname{tr}(\boldsymbol{H}^{u})=R$,
this restricts $R<N_{0}/2$. A similar conclusion is drawn for VT regression.

###  4.2 Design-Based Inference

This section studies the counterfactual prediction from a design-based
perspective, whereby the potential outcomes are considered fixed and the
treatment assignments are considered stochastic.

####  4.2.1 Assumptions

In congruence with the article thus far, we focus on a single treated unit and
treated period. We will find it useful to separate the assignment mechanism
into the selection of each component. Accordingly, let
$\boldsymbol{A}\in\\{0,1\\}^{T}$ with
$\boldsymbol{1}^{\prime}\boldsymbol{A}=1$ and $\boldsymbol{B}\in\\{0,1\\}^{N}$
with $\boldsymbol{1}^{\prime}\boldsymbol{B}=1$ be the indicator vectors for
the treated time period and treated unit, respectively, e.g., $B_{N}=1$ and
$A_{T}=1$ if unit $N$ is treated at time $T$. With this notation, we denote
the realized outcome as $Y_{it}=B_{i}A_{t}Y_{it}(1)+(1-B_{i}A_{t})Y_{it}(0)$.
Following Bottmer et al. (2021), we consider the following assignment
mechanisms:

######  Assumption 7 (Random assignment of time period).

$\displaystyle\mathbb{P}(\boldsymbol{A}=\boldsymbol{a})=\begin{cases}&1/T,\quad\text{if
}a_{t}\in\\{0,1\\}~{}\forall t,~{}\boldsymbol{1}^{\prime}\boldsymbol{a}=1\\\
&0,\quad\text{otherwise}.\end{cases}$ (48)

######  Assumption 8 (Random assignment of unit).

$\displaystyle\mathbb{P}(\boldsymbol{B}=\boldsymbol{b})=\begin{cases}&1/N,\quad\text{if
}b_{i}\in\\{0,1\\}~{}\forall i,~{}\boldsymbol{1}^{\prime}\boldsymbol{b}=1\\\
&0,\quad\text{otherwise}.\end{cases}$ (49)

Assumption 7 considers the treated period to be randomly selected while
Assumption 8 considers the treated unit to be randomly selected. As Bottmer et
al. (2021) notes, these assumptions are not always plausible, but they
underlie the placebo tests that are commonly used in synthetic controls
applications.

####  4.2.2 Estimator

To conduct design-based analysis, we consider all possible treatment
assignments, not only the realized assignment. Let $Y^{*}_{it}(0)$ be the OLS
fit of $Y_{it}(0)$ based on $[Y_{j\tau}:j\neq i,\tau<t]$,
$[Y_{i\tau}:\tau<t]$, and $[Y_{jt}:j\neq i]$. We define the design-based
estimator as

$\displaystyle\widehat{Y}(0)=\sum_{i\leq N}\sum_{t\leq
T}B_{i}A_{t}Y^{*}_{it}(0).$ (50)

In words, (50) predicts the mean counterfactual outcome under control for unit
$i$ at time $t$ if unit $i$ is treated at time period $t$, i.e., $B_{i}=1$ and
$A_{t}=1$. We reemphasize that the stochasticity of $\widehat{Y}(0)$ stems
from the treatment assignment mechanism since $Y^{*}_{it}(0)$ is a fixed
quantity. As such, while the model-and design-based estimators share the same
point estimate for the realized assignment, they differ in their formulations
and attributions of randomness.

####  4.2.3 Inferential Properties

In Table 1, we summarize the estimands associated with the model-based and
design-based estimators under three sources of randomness: (i) time, (ii)
unit, and (iii) time and unit. Within the model-based framework, mechanisms
(i)–(iii) correspond to the HZ model (Assumptions 1–2), VT model (Assumptions
3–4), and mixed model (Assumptions 5–6), respectively. Within the design-based
framework, mechanisms (i)–(iii) correspond to Assumption 7, Assumption 8, and
both Assumptions 7 and 8, respectively. Accordingly, the model-based and
design-based expectations are taken over different probability measures.

Table 1:  Model-based and design-based estimands under different sources of randomness. We use the shorthand $\tilde{\boldsymbol{\alpha}}^{*}=\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*}$ and $\tilde{\boldsymbol{\beta}}^{*}=\boldsymbol{H}^{u}\boldsymbol{\beta}^{*}$ and consider the realized $(N,T)$th assignment. Source of Randomness | Model-Based Estimand | Design-Based Estimand
---|---|---
Time | $\mathbb{E}[\widehat{Y}_{NT}(0)|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]=\sum_{t\leq T_{0}}\tilde{\alpha}^{*}_{t}Y_{Nt}$ | $\mathbb{E}[\widehat{Y}(0)|\boldsymbol{B}]=\frac{1}{T}\sum_{t\leq T}Y^{*}_{Nt}(0)$
Unit | $\mathbb{E}[\widehat{Y}_{NT}(0)|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]=\sum_{i\leq N_{0}}\tilde{\beta}^{*}_{i}Y_{iT}$ | $\mathbb{E}[\widehat{Y}(0)|\boldsymbol{A}]=\frac{1}{N}\sum_{i\leq N}Y^{*}_{iT}(0)$
Time and Unit | $\mathbb{E}[\widehat{Y}_{NT}(0)|\boldsymbol{Y}_{0}]=\sum_{i\leq N_{0}}\sum_{t\leq T_{0}}\alpha^{*}_{t}\beta^{*}_{i}Y_{it}$ | $\mathbb{E}[\widehat{Y}(0)]=\frac{1}{NT}\sum_{i\leq N}\sum_{t\leq T}Y^{*}_{it}(0)$

Let us compare the estimands in Table 1. In words,
$\mathbb{E}[\widehat{Y}_{NT}(0)|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]$ is a
weighted combination of outcomes under control for the treated unit across all
$T_{0}$ pretreatment periods; $\mathbb{E}[\widehat{Y}(0)|\boldsymbol{B}]$ is a
simple average of fitted outcomes under control for the treated unit across
all $T$ periods, which Bottmer et al. (2021) calls the “HZ” effect. Similarly,
$\mathbb{E}[\widehat{Y}_{NT}(0)|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]$ is a
weighted combination of outcomes under control for the treated period across
all $N_{0}$ control units; $\mathbb{E}[\widehat{Y}(0)|\boldsymbol{A}]$ is a
simple average of fitted outcomes under control for the treated period across
all $N$ units, also called the “VT” effect. Finally,
$\mathbb{E}[\widehat{Y}_{NT}(0)|\boldsymbol{Y}_{0}]$ is a weighted combination
of outcomes under control across all $N_{0}$ control units and $T_{0}$
pretreatment periods; $\mathbb{E}[\widehat{Y}(0)]$ is a simple average of
fitted outcomes under control across all unit and time period pairs, which we
coin the “mixed” effect.

Though our design-based analysis is brief, there two important takeaways: (i)
the model-based and design-based estimators recover similar estimands for each
source of randomness; and (ii) different sources of randomness lead to
different estimands, which is consistent with our model-based insights. The
connection between assumptions of randomness and resulting estimand has been
previously noticed in related contexts, e.g., Abadie et al. (2020); Bottmer et
al. (2021) and Sekhon and Shem-Tov (2021).

###  4.3 Discussion

Several remarks on this section’s results and extensions are in order.

######  Remark 1 (Correct Specification).

For expositional convenience, we assume correct specification of the outcome
model, as in Li (2020), to explain the panel data intuition and main
theoretical results in a simple and transparent fashion. Alternatively, we can
interpret Assumptions 1, 3, and 5 as linear prediction models that are
detached from structural meanings a la Cattaneo et al. (2021) and Chernozhukov
et al. (2021, 2022). For instance, within the mixed framework, we can redefine
$\boldsymbol{\alpha}^{*}=\operatorname*{\arg\\!\min}\mathbb{E}[\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}\|_{2}^{2}|\boldsymbol{Y}_{0}]$
as the best linear approximation of $\boldsymbol{y}_{T}$ based on
$\boldsymbol{Y}_{0}$, conditional on $\boldsymbol{Y}_{0}$. Such models can be
justified via factor models and vector autoregressive models (Chernozhukov et
al., 2022).

######  Remark 2 (Mixed Model).

Any estimator, whether it falls within the symmetric or asymmetric class, can
be studied under the mixed model (Assumptions 5–6). There are also numerous
ways to encode the mixed perspective beyond our postulation, e.g., random
assignment of periods and units (Assumptions 7–8) of Bottmer et al. (2021).

######  Remark 3 (Mixed Variance Estimators).

We highlight that Lemmas 1–3 only hold in expectation. For any particular
realization, $\widehat{v}^{\text{mix}}_{0}$ may exhibit unexpected properties.
For instance, if
$\operatorname{tr}(\boldsymbol{Y}_{0}^{\dagger}\widehat{\boldsymbol{\Sigma}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\widehat{\boldsymbol{\Sigma}}_{N})>\max\\{\widehat{v}_{0}^{\text{hz}},\widehat{v}_{0}^{\text{vt}}\\}$,
then
$\widehat{v}^{\text{mix}}_{0}<\min\\{\widehat{v}_{0}^{\text{hz}},\widehat{v}_{0}^{\text{vt}}\\}$;
thus, the mixed coverage will be smaller than both HZ and VT coverages. In
fact, $\widehat{v}^{\text{mix}}_{0}$ can be negative if
$\operatorname{tr}(\boldsymbol{Y}_{0}^{\dagger}\widehat{\boldsymbol{\Sigma}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\widehat{\boldsymbol{\Sigma}}_{N})>\widehat{v}_{0}^{\text{hz}}+\widehat{v}_{0}^{\text{vt}}$,
which may occur if both HZ and VT in-sample errors are “too large”. For these
scenarios, one naïve solution is to modify $\widehat{v}^{\text{mix}}_{0}$ as
$\widehat{v}_{0}^{\text{mix}}\leftarrow\widehat{v}_{0}^{\text{hz}}+\widehat{v}_{0}^{\text{vt}}$,
which is conservative by Lemmas 1–3. However, this case is arguably better
resolved with a different point estimator altogether.

######  Remark 4 (Bridging Model-Based and Design-Based Inferences).

The takeaways from the model-based and design-based analyses are consistent
with one another. However, a formal and complete connection between the two
perspectives on inference remains to be established. Towards this, we
highlight that Lin (2013) demonstrates that one set of model-based confidence
intervals (i.e., the Huber-White sandwich estimator) are justifiable under
design-based arguments. In this view, a fascinating line of future inquiry is
to analyze whether similar arguments hold for the heteroskedastic confidence
intervals presented in Section 4.1.3 or other related model-based
constructions.

######  Remark 5 (On the Role of Randomness).

This section underscores that the source of randomness plays a pivotal role
for conducting inference. Translated to practice, our results stress that
researchers should be scrupulous in reasoning through where the randomness in
their data comes from. For instance, in the Basque study, some researchers may
find it more plausible that the randomness is over time rather than over
space, e.g., it is more conceivable that the onset of terrorism could have
occurred in a different year but less conceivable that it could have occurred
in a different region of Spain but not in the Basque Country. We do not take a
substantive view on the matter, but we do stress that these decisions are
meaningful for proper analysis as they have immediate implications for the
resulting estimand and inferential procedure.

######  Remark 6 (Model Checking).

It is possible in an application that substantive knowledge does not make
clear on the source of randomness and hence which model to use, i.e., HZ, VT,
or mixed. In such scenarios, the “in-space” and “in-time” placebo tests (a la
cross-validation) proposed in Abadie et al. (2010, 2015) are attractive tools
to analyze the prediction properties of the various estimators under
consideration. More generally, practices established within Yu and Kumbier
(2020) provide an organized framework based on the principles of
predictability, computability, and stability (PCS) to conduct rigorous
comparison analyses.

######  Remark 7 (Extension to PCR).

The previous results immediately extend to PCR by replacing
$\boldsymbol{Y}_{0}$ with $\boldsymbol{Y}_{0}^{(k)}$, as defined in (5), for
any $k<R$. Intuitively, PCR-based models operate under the belief that the
data is inherently low-dimensional. We comment on several benefits of PCR over
OLS. To begin, the HZ and VT OLS variance estimators constructed in Section
4.1.3 can suffer from degeneracy when $N$ and $T$ are of different sizes. That
is, if $N<T$, then the HZ in-sample error is likely zero (otherwise known as
overfitting), which causes the HZ coverage to collapse on the point estimate;
analogous statements hold for the VT coverage when $N>T$. The PCR-based
variance estimators, on the other hand, can avoid degeneracy through the
number of chosen principal components $k$ (regularization). On a related note,
the nonsingularity conditions required for the jackknife and HRK variance
estimators can also be by controlled by $k$.

##  5 Illustrations

This section illustrates key concepts developed in this article. Our report is
based on three canonical synthetic controls studies: (i) terrorism in Basque
Country, (ii) California’s Proposition 99 (Abadie et al., 2010), and (iii) the
reunification of West Germany (Abadie et al., 2015). In particular, we will
conduct a model-based analysis using the confidence intervals developed in
Section 4.1.3. We provide an overview of the results and relegate details
(e.g., implementation) to Appendix B.

###  5.1 Background on Case Studies

Basque study. See Sections 1–2 for details.

California study. This study examines the effect of California’s Proposition
99, an anti-tobacco legislation, on its tobacco consumption. The panel data
contains per capita cigarette sales of $N=39$ U.S. states over $T=31$ years.
There are $T_{0}=18$ pretreatment observations and $N_{0}=38$ control units.
Our interest is to estimate California’s cigarette sales in the absence of
Proposition 99.

West Germany study. This study examines the economic impact of the 1990
reunification in West Germany. The panel data contains per capita GDP of
$N=17$ countries over $T=44$ years. There are $T_{0}=30$ pretreatment
observations and $N_{0}=16$ control units. Our interest is to estimate West
Germany’s GDP in the absence of reunification.

###  5.2 Data-Inspired Simulation Studies

We look to better understand the trade-offs in conducting inference under
different sources of randomness. In an attempt to document our analysis in a
realistic environment, we calibrate our simulations to our three studies.

####  5.2.1 Data Generating Process

We consider the single treated unit and time period setting. Specifically, we
consider the actual treated unit, e.g., Basque Country, and focus on the first
post-treatment period, e.g., one year after the outset of terrorism; hence,
$T\leftarrow T_{0}+1$. Using the actual data, we generate the underlying
regression models as

$\displaystyle\boldsymbol{\alpha}^{*}=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}}\|\boldsymbol{y}^{*}_{T}-\boldsymbol{Y}^{*}_{0}\boldsymbol{\alpha}\|_{2}^{2}\quad\text{and}\quad\boldsymbol{\beta}^{*}=\operatorname*{\arg\\!\min}_{\boldsymbol{\beta}}\|\boldsymbol{y}^{*}_{N}-(\boldsymbol{Y}^{*}_{0})^{\prime}\boldsymbol{\beta}\|_{2}^{2},$
(51)

where $\boldsymbol{y}^{*}_{N}=[Y_{Nt}:t\leq T_{0}]$,
$\boldsymbol{y}^{*}_{T}=[Y_{iT}:i\leq N_{0}]$, and
$\boldsymbol{Y}^{*}_{0}=[Y_{it}:i\leq N_{0},t\leq T_{0}]$.

Observationally, we have access to the following quantities. Let
$\boldsymbol{Y}_{0}$ be the rank $r$ approximation of
$\boldsymbol{Y}^{*}_{0}$, where $r$ is chosen as the minimum number of
singular values needed to capture at least $99.9\%$ of
$\boldsymbol{Y}^{*}_{0}$’s spectral energy. Next, we sample
$\boldsymbol{y}_{T}\sim\mathcal{N}(\boldsymbol{Y}_{0}\boldsymbol{\alpha}^{*},(N_{0}-r)^{-1}\|\boldsymbol{y}^{*}_{T}-\boldsymbol{Y}^{*}_{0}\boldsymbol{\alpha}^{*}\|_{2}^{2}\boldsymbol{I})$
and
$\boldsymbol{y}_{N}\sim\mathcal{N}(\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}^{*},(T_{0}-r)^{-1}\|\boldsymbol{y}^{*}_{N}-(\boldsymbol{Y}^{*}_{0})^{\prime}\boldsymbol{\beta}^{*}\|_{2}^{2}\boldsymbol{I})$.
We then define three estimands: (i)
$\mu_{0}^{\text{hz}}=\langle\boldsymbol{y}_{N},\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*}\rangle$,
(ii)
$\mu_{0}^{\text{vt}}=\langle\boldsymbol{y}_{T},\boldsymbol{H}^{u}\boldsymbol{\beta}^{*}\rangle$,
and (iii)
$\mu_{0}^{\text{mix}}=\langle\boldsymbol{\alpha}^{*},\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}^{*}\rangle$,
where $(\boldsymbol{H}^{u},\boldsymbol{H}^{v})$ are computed from
$\boldsymbol{Y}_{0}$.

####  5.2.2 Simulation Results

For the purposes of stability, we conduct 500 replications of the above DGP
for each study. In the $\ell$th simulation repeat, we learn the regression
coefficients as

$\displaystyle\widehat{\boldsymbol{\alpha}}^{(\ell)}=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}}\|\boldsymbol{y}^{(\ell)}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}\|_{2}^{2}\quad\text{and}\quad\widehat{\boldsymbol{\beta}}^{(\ell)}=\operatorname*{\arg\\!\min}_{\boldsymbol{\beta}}\|\boldsymbol{y}^{(\ell)}_{N}-\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}\|_{2}^{2}.$
(52)

The corresponding point estimate is defined as
$\widehat{Y}^{(\ell)}_{NT}(0)=\langle\boldsymbol{y}^{(\ell)}_{N},\widehat{\boldsymbol{\alpha}}^{(\ell)}\rangle=\langle\boldsymbol{y}_{T}^{(\ell)},\widehat{\boldsymbol{\beta}}^{(\ell)}\rangle$.
We then construct separate HZ, VT, and mixed homoskedastic confidence
intervals based on (33) and (34) around the point estimate.

In Table 2, we report the coverage probabilities (CP) and average lengths (AL)
for each confidence interval with respect to each estimand at the $95\%$
nominal mark. With respect to $\mu_{0}^{\text{hz}}$, the coverage of the HZ
confidence interval is closer to the nominal coverage than that of the VT and
mixed intervals as the latter two can substantially under- or over-cover. This
storyline is consistent for the VT interval with respect to
$\mu_{0}^{\text{vt}}$ and the mixed interval with respect to
$\mu_{0}^{\text{mix}}$.

Collectively, our formal results and simulations demonstrate that (i) the
choice of estimand directly affects the accuracy of the inference; and (ii)
the variance formulas developed for one estimand may not have the correct
coverage for another estimand. Accordingly, researchers should carefully
consider the source of randomness in their data as it can have a significant
influence over their ability to conduct valid inference. We comment that these
conclusions are in line with those drawn in Sekhon and Shem-Tov (2021), which
analyzes the classical difference-in-means estimator with respect to standard
estimands for randomized control trials.

Table 2: Coverage for nominal 95% confidence intervals across 500 replications. The coverage length is normalized by the magnitude of the corresponding point estimate. Case study | $\widehat{v}_{0}^{\text{hz}}$ | $\widehat{v}_{0}^{\text{vt}}$ | $\widehat{v}_{0}^{\text{mix}}$
---|---|---|---
$\mu_{0}^{\text{hz}}$ | $\mu_{0}^{\text{vt}}$ | $\mu_{0}^{\text{mix}}$ | $\mu_{0}^{\text{hz}}$ | $\mu_{0}^{\text{vt}}$ | $\mu_{0}^{\text{mix}}$ | $\mu_{0}^{\text{hz}}$ | $\mu_{0}^{\text{vt}}$ | $\mu_{0}^{\text{mix}}$
Basque (CP) | $0.92$ | $0.74$ | $0.63$ | $0.99$ | $0.93$ | $0.88$ | $1.00$ | $0.97$ | $0.94$
Basque (AL) | $0.02$ | $0.02$ | $0.02$ | $0.03$ | $0.03$ | $0.03$ | $0.04$ | $0.04$ | $0.04$
California (CP) | $0.95$ | $1.00$ | $0.92$ | $0.64$ | $0.93$ | $0.60$ | $0.98$ | $1.00$ | $0.95$
California (AL) | $0.07$ | $0.07$ | $0.07$ | $0.03$ | $0.03$ | $0.03$ | $0.08$ | $0.08$ | $0.08$
W. Germany (CP) | $0.94$ | $1.00$ | $0.93$ | $0.49$ | $0.94$ | $0.49$ | $0.96$ | $1.00$ | $0.95$
W. Germany (AL) | $0.03$ | $0.03$ | $0.03$ | $0.01$ | $0.01$ | $0.01$ | $0.03$ | $0.03$ | $0.03$

###  5.3 Empirical Applications

Next, we analyze our three case studies of interest. All regression models are
built on pretreatment data only, and the point and variance estimation
formulas are separately applied for the treated unit at each post-treatment
period $t>T_{0}$.

####  5.3.1 Point Estimation

Figure 4 visualizes the counterfactual trajectories generated by the
estimators in Section 3.1. Our findings reinforce Theorems 1 and 2. On a
separate note, we observe that, within the Basque study, the OLS estimates are
wildly different from the other estimates and HZ simplex regression reduces to
the last observation carried forward (LOCF) estimator. In the California and
West Germany studies, the estimates are all qualitatively similar with the
exception of the HZ simplex regression, which again reduces to LOCF. In fact,
the OLS and ridge estimates appear to overlap, as well as the lasso and
elastic net estimates.

Figure 4:  Top and bottom figures correspond to symmetric and asymmetric
estimators, respectively. From left to right, the figures are indexed by the
Basque, California, and West Germany studies. Across all figures, the treated
year is the dotted vertical line; the observed trajectory is in solid black;
and the HZ and VT counterfactual trajectories are colored solid and dashed-
dotted lines, respectively.

####  5.3.2 Inference

To reduce visual redudancy, we only present the figures associated with the
jackknife-based intervals for OLS and PCR in Figures 5 and 6, respectively.
Consider the Basque study. The top row of plots in both figures demonstrates
that $\mu_{0}^{\text{vt}}$ is more accurately estimated than both
$\mu_{0}^{\text{hz}}$ and $\mu_{0}^{\text{mix}}$. Put differently, there is
less uncertainty about conducting inference on $\mu_{0}^{\text{vt}}$ relative
to the other estimands. At the same time, these plots indicate that if
$\mu_{0}^{\text{hz}}$ or $\mu_{0}^{\text{mix}}$ are the estimands of interest,
then the VT confidence interval will undercover in both settings. Analogous
statements can be made for the remaining subfigures. As with our simulations,
the large potential differences in coverage reinforce the importance of
properly reasoning through the source of randomness in the data.

(a) HZ model.

(b) VT model.

(c) Mixed model.

Figure 5:  OLS estimates with jackknife confidence intervals. From top to
bottom, the rows are indexed by the Basque, California, and West Germany
studies. From left to right, the columns are indexed by the HZ, VT, and mixed
models. Figure 5 visualizes the problem of degeneracy for OLS-based intervals
that is discussed in Remark 7.

(a) HZ model.

(b) VT model.

(c) Mixed model.

Figure 6:  PCR estimates with jackknife confidence intervals. From top to
bottom, the rows are indexed by the Basque, California, and West Germany
studies. From left to right, the columns are indexed by the HZ, VT, and mixed
models.

##  6 Conclusion

This article rewrites the conventional wisdom on HZ and VT regressions for
panel data analysis. Contrary to standard notions, we show that the two
regressions yield identical point estimates under several standard settings.
At the same time, we articulate that the source of randomness directly affects
the accuracy of the inference that can be conducted. From a practical
standpoint, this stresses that researchers should carefully consider where the
randomness in their data stems from as this decision will then guide their
choice of estimand and inferential procedure.

## References

  * Abadie (2021) Abadie, Alberto (2021): “Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects,” _Journal of Economic Literature_ , 59, 391–425.
  * Abadie et al. (2020) Abadie, Alberto, Susan Athey, Guido W. Imbens, and Jeffrey M. Wooldridge (2020): “Sampling-Based versus Design-Based Uncertainty in Regression Analysis,” _Econometrica_ , 88, 265–296.
  * Abadie et al. (2010) Abadie, Alberto, Alexis Diamond, and Jens Hainmueller (2010): “Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of Californiaâs Tobacco Control Program,” _Journal of the American Statistical Association_ , 105.
  * Abadie et al. (2015) ——— (2015): “Comparative Politics and the Synthetic Control Method,” _American Journal of Political Science_ , 59, 495–510.
  * Abadie and Gardeazabal (2003) Abadie, A. and J. Gardeazabal (2003): “The Economic Costs of Conflict: A Case Study of the Basque Country,” _American Economic Review_ , 93, 113–132.
  * Abadie and L’Hour (2021) Abadie, Alberto and Jérémy L’Hour (2021): “A Penalized Synthetic Control Estimator for Disaggregated Data,” _Journal of the American Statistical Association_ , 116, 1817–1834.
  * Agarwal et al. (2021) Agarwal, Anish, Devavrat Shah, and Dennis Shen (2021): “Synthetic Interventions,” _arXiv preprint arXiv:2006.07691_.
  * Amjad et al. (2019) Amjad, Muhammad, Vishal Misra, Devavrat Shah, and Dennis Shen (2019): “MRSC: Multi-Dimensional Robust Synthetic Control,” _Proc. ACM Meas. Anal. Comput. Syst._ , 3.
  * Amjad et al. (2018) Amjad, Muhammad, Devavrat Shah, and Dennis Shen (2018): “Robust Synthetic Control,” _Journal of Machine Learning Research_ , 19, 1–51.
  * Arkhangelsky et al. (2021) Arkhangelsky, Dmitry, Susan Athey, David A. Hirshberg, Guido W. Imbens, and Stefan Wager (2021): “Synthetic Difference-in-Differences,” _American Economic Review_ , 111, 4088–4118.
  * Ashenfelter (1978) Ashenfelter, Orley (1978): “Estimating the Effect of Training Programs on Earnings,” _The Review of Economics and Statistics_ , 60, 47–57.
  * Athey et al. (2021) Athey, Susan, Mohsen Bayati, Nikolay Doudchenko, Guido Imbens, and Khashayar Khosravi (2021): “Matrix Completion Methods for Causal Panel Data Models,” _Journal of the American Statistical Association_ , 116, 1716–1730.
  * Athey and Imbens (2017) Athey, Susan and Guido W. Imbens (2017): “The State of Applied Econometrics: Causality and Policy Evaluation,” _Journal of Economic Perspectives_ , 31, 3–32.
  * Ben-Michael et al. (2021) Ben-Michael, Eli, Avi Feller, and Jesse Rothstein (2021): “The Augmented Synthetic Control Method,” _Journal of the American Statistical Association_ , 116, 1789–1803.
  * Bottmer et al. (2021) Bottmer, Lea, Guido Imbens, Jann Spiess, and Merrill Warnick (2021): “A Design-Based Perspective on Synthetic Control Methods,” .
  * Carvalho et al. (2018) Carvalho, Carlos, Ricardo Masini, and Marcelo C. Medeiros (2018): “ArCo: An artificial counterfactual approach for high-dimensional panel time-series data,” _Journal of Econometrics_ , 207, 352–380.
  * Cattaneo et al. (2021) Cattaneo, Matias D., Yingjie Feng, and Rocio Titiunik (2021): “Prediction Intervals for Synthetic Control Methods,” _Journal of the American Statistical Association_ , 116, 1865–1880.
  * Chernozhukov et al. (2022) Chernozhukov, Victor, Kaspar Wuthrich, and Yinchu Zhu (2022): “A $t$-test for synthetic controls,” _arXiv preprint arXiv:2006.07691_.
  * Chernozhukov et al. (2021) Chernozhukov, Victor, Kaspar Wüthrich, and Yinchu Zhu (2021): “An Exact and Robust Conformal Inference Method for Counterfactual and Synthetic Controls,” _Journal of the American Statistical Association_ , 116, 1849–1864.
  * Cline (1965) Cline, Randall E. (1965): “Representations for the Generalized Inverse of Sums of Matrices,” _Journal of the Society for Industrial and Applied Mathematics: Series B, Numerical Analysis_ , 2, 99–114.
  * Doudchenko and Imbens (2016) Doudchenko, Nikolay and Guido W Imbens (2016): “Balancing, Regression, Difference-In-Differences and Synthetic Control Methods: A Synthesis,” Working Paper 22791, National Bureau of Economic Research.
  * Gunasekar et al. (2017) Gunasekar, Suriya, Blake Woodworth, Srinadh Bhojanapalli, Behnam Neyshabur, and Nathan Srebro (2017): “Implicit Regularization in Matrix Factorization,” in _Advances in Neural Information Processing Systems_.
  * Hartley et al. (1969) Hartley, H. O., J. N. K. Rao, and Grace Kiefer (1969): “Variance Estimation with One Unit per Stratum,” _Journal of the American Statistical Association_ , 64, 841–851.
  * Hoff (2017) Hoff, Peter D. (2017): “Lasso, fractional norm and structured sparse estimation using a Hadamard product parametrization,” _Computational Statistics & Data Analysis_, 115, 186–198.
  * Hsiao et al. (2012) Hsiao, Cheng, H. Steve Ching, and Shui Ki Wan (2012): “A Panel Data Approach for Program Evaluation: Measuring the Benefits of Political and Economic Integration of Hong Kong with Mainland China,” _Journal of Applied Econometrics_ , 27, 705–740.
  * Imbens and Wooldridge (2009) Imbens, Guido W. and Jeffrey M. Wooldridge (2009): “Recent Developments in the Econometrics of Program Evaluation,” _Journal of Economic Literature_ , 47, 5–86.
  * Lehmann (2000) Lehmann, E.L. (2000): “Elements of Large-Sample Theory,” .
  * Li (2020) Li, Kathleen T. (2020): “Statistical Inference for Average Treatment Effects Estimated by Synthetic Control Methods,” _Journal of the American Statistical Association_ , 115, 2068–2083.
  * Li and Bell (2017) Li, Kathleen T. and David R. Bell (2017): “Estimation of average treatment effects with panel data: Asymptotic theory and implementation,” _Journal of Econometrics_ , 197, 65–75.
  * Lin (2013) Lin, Winston (2013): “Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique,” _The Annals of Applied Statistics_ , 7, 295 – 318.
  * Meyer (1973) Meyer, Carl D. (1973): “Generalized Inversion of Modified Matrices,” _SIAM Journal on Applied Mathematics_ , 24, 315–323.
  * Neyshabur et al. (2015) Neyshabur, Behnam, Ryota Tomioka, and Nathan Srebro (2015): “In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning,” in _International Conference on Learning Representations_.
  * Rosenbaum and Rubin (1983) Rosenbaum, PAUL and Donald Rubin (1983): “The Central Role of the Propensity Score in Observational Studies For Causal Effects,” _Biometrika_ , 70, 41–55.
  * Sekhon and Shem-Tov (2021) Sekhon, Jasjeet S. and Yotam Shem-Tov (2021): “Inference on a New Class of Sample Average Treatment Effects,” _Journal of the American Statistical Association_ , 116, 798–804.
  * Shao and Deng (2012) Shao, Jun and Xinwei Deng (2012): “Estimation in high-dimensional linear models with deterministic design matrices,” _The Annals of Statistics_ , 40, 812 – 831.
  * Styan (1973) Styan, George P.H. (1973): “Hadamard products and multivariate statistical analysis,” _Linear Algebra and its Applications_ , 6, 217–240.
  * Tibshirani (2013) Tibshirani, Ryan J. (2013): “The lasso problem and uniqueness,” _Electronic Journal of Statistics_ , 7, 1456 – 1490.
  * Varga (1962) Varga, Richard S. (1962): _Matrix Iterative Analysis_ , Prentice-Hall Series in Automatic Computation, Englewood Cliffs: Prentice-Hall.
  * Yu and Kumbier (2020) Yu, Bin and Karl Kumbier (2020): “Veridical data science,” _Proceedings of the National Academy of Sciences_ , 117, 3920–3929.

##  Appendix A Inference

###  A.1 Model-Based Inference: Asymptotic Properties

First, we state the precise form of Theorem 3. Towards this, let
$(\sigma^{\text{hz}}_{iT})^{2}=\operatorname{Var}(\varepsilon_{iT}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0})$
for $i=1,\dots,N_{0}$. We define
$((\sigma^{\text{vt}}_{Nt})^{2},(\sigma^{\text{mix}}_{iT})^{2},(\sigma^{\text{mix}}_{Nt})^{2})$
for $i=1,\dots,N_{0}$ and $t=1,\dots,T_{0}$ with respect to
$(\boldsymbol{\Sigma}^{\text{vt}}_{N},\boldsymbol{\Sigma}^{\text{mix}}_{T},\boldsymbol{\Sigma}^{\text{mix}}_{N})$
analogously.

######  Theorem 4.

(i) _[ HZ model]_ Let Assumptions 1–2 hold. If

$\displaystyle\left(\sum_{i\leq
N_{0}}\mathbb{E}\left[|\widehat{\beta}_{i}\varepsilon_{iT}|^{3}|~{}\boldsymbol{y}_{N},\boldsymbol{Y}_{0}\right]\right)^{2}$
$\displaystyle=o\left(\Big{(}\sum_{i\leq
N_{0}}\widehat{\beta}^{2}_{i}(\sigma^{\emph{hz}}_{iT})^{2}\Big{)}^{3}\right),$
(53)

then

$\displaystyle\frac{\widehat{Y}_{NT}(0)-\mu_{0}^{\emph{hz}}}{\sqrt{v^{\emph{hz}}_{0}}}\xrightarrow{d}\mathcal{N}(0,1),$
(54)

where
$\mu_{0}^{\emph{hz}}=\langle\boldsymbol{y}_{N},\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*}\rangle$
and
$v_{0}^{\emph{hz}}=\widehat{\boldsymbol{\beta}}^{\prime}\boldsymbol{\Sigma}^{\emph{hz}}_{T}\widehat{\boldsymbol{\beta}}$.
(ii) _[ VT model]_ Let Assumptions 3–4 hold. If

$\displaystyle\left(\sum_{t\leq
T_{0}}\mathbb{E}\left[|\widehat{\alpha}_{t}\varepsilon_{Nt}|^{3}|~{}\boldsymbol{y}_{T},\boldsymbol{Y}_{0}\right]\right)^{2}$
$\displaystyle=o\left(\Big{(}\sum_{t\leq
T_{0}}\widehat{\alpha}^{2}_{t}(\sigma_{Nt}^{\emph{vt}})^{2}\Big{)}^{3}\right),$
(55)

then

$\displaystyle\frac{\widehat{Y}_{NT}(0)-\mu_{0}^{\emph{vt}}}{\sqrt{v_{0}^{\emph{vt}}}}\xrightarrow{d}\mathcal{N}(0,1),$
(56)

where
$\mu^{\emph{vt}}_{0}=\langle\boldsymbol{y}_{T},\boldsymbol{H}^{u}\boldsymbol{\beta}^{*}\rangle$
and
$v^{\emph{vt}}_{0}=\widehat{\boldsymbol{\alpha}}^{\prime}\boldsymbol{\Sigma}^{\emph{vt}}_{N}\widehat{\boldsymbol{\alpha}}$.
(iii) _[Mixed model]_ Let Assumptions 5–6 hold. If

$\displaystyle\left(\sum_{i\leq N_{0}}\sum_{t\leq
T_{0}}\mathbb{E}\Big{[}|(\boldsymbol{Y}_{0}^{\dagger})_{it}\left\\{\mathbb{E}[Y_{iT}|\boldsymbol{Y}_{0}]\varepsilon_{Nt}+\mathbb{E}[Y_{Nt}|\boldsymbol{Y}_{0}]\varepsilon_{iT}+\varepsilon_{iT}\varepsilon_{Nt}\right\\}|^{3}~{}|~{}\boldsymbol{Y}_{0}\Big{]}\right)^{2}$
(57) $\displaystyle=o\left(\left(\sum_{i\leq N_{0}}\sum_{t\leq
T_{0}}(\boldsymbol{Y}_{0}^{\dagger})^{2}_{it}\left\\{\mathbb{E}[Y_{iT}|\boldsymbol{Y}_{0}]^{2}(\sigma^{\emph{mix}}_{Nt})^{2}+\mathbb{E}[Y_{Nt}|\boldsymbol{Y}_{0}]^{2}(\sigma^{\emph{mix}}_{iT})^{2}+(\sigma^{\emph{mix}}_{iT})^{2}(\sigma^{\emph{mix}}_{Nt})^{2}\right\\}\right)^{3}\right),$
(58)

then

$\displaystyle\frac{\widehat{Y}_{NT}(0)-\mu^{\emph{mix}}_{0}}{\sqrt{v^{\emph{mix}}_{0}}}\xrightarrow{d}\mathcal{N}(0,1),$
(59)

where
$\mu^{\emph{mix}}_{0}=\langle\boldsymbol{\alpha}^{*},\boldsymbol{Y}^{\prime}_{0}\boldsymbol{\beta}^{*}\rangle$
and

$\displaystyle
v^{\emph{mix}}_{0}=(\boldsymbol{H}^{u}\boldsymbol{\beta}^{*})^{\prime}\boldsymbol{\Sigma}^{\emph{mix}}_{T}(\boldsymbol{H}^{u}\boldsymbol{\beta}^{*})+(\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*})^{\prime}\boldsymbol{\Sigma}^{\emph{mix}}_{N}(\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*})+\operatorname{tr}(\boldsymbol{Y}^{\dagger}_{0}\boldsymbol{\Sigma}^{\emph{mix}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{\Sigma}^{\emph{mix}}_{N}).$
(60)

If $(\varepsilon_{iT},(\sigma^{\text{hz}}_{iT})^{2})$ are bounded, then (53)
translates to $\sum_{i\leq
N_{0}}|\widehat{\beta}_{i}|^{3}=o(\|\widehat{\boldsymbol{\beta}}\|_{2}^{3})$,
which rules out outlier coefficients; a similar interpretation can be derived
for (55). Similarly, if $(\varepsilon_{iT},\varepsilon_{Nt})$ and
$(\sigma^{2}_{iT},\sigma^{2}_{Nt})$ are bounded for all $(i,t)$, then (58)
loosely translates to

$\sum_{i\leq N_{0}}|\widehat{\beta}_{i}|^{3}+\sum_{t\leq
T_{0}}|\widehat{\alpha}_{t}|^{3}+\sum_{i\leq N_{0}}\sum_{t\leq
T_{0}}|(\boldsymbol{Y}^{\dagger}_{0})_{it}|^{3}=o\left(\|\widehat{\boldsymbol{\beta}}\|_{2}^{3}+\|\widehat{\boldsymbol{\alpha}}\|_{2}^{3}+\|\boldsymbol{Y}_{0}^{\dagger}\|_{F}^{3}\right),$

which effectively bounds the magnitudes of the HZ and VT OLS coefficients and
pseudoinverse matrix entries. We note that (53)–(58) are known as Lyapunov’s
condition, and we refer the interested reader to Lehmann (2000) for details.

Next, we provide a bound on the trace term in $v^{\text{mix}}_{0}$. Beginning
with the upper bound, notice that

$\displaystyle\operatorname{tr}(\boldsymbol{Y}^{\dagger}_{0}\boldsymbol{\Sigma}^{\text{mix}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{\Sigma}^{\text{mix}}_{N})\leq\max_{i\leq
N_{0}}(\sigma^{\text{mix}}_{iT})^{2}\max_{t\leq
T_{0}}(\sigma^{\text{mix}}_{Nt})^{2}\operatorname{tr}(\boldsymbol{Y}^{\dagger}_{0}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}).$
(61)

By the cyclic property of the trace operator,

$\displaystyle\operatorname{tr}(\boldsymbol{Y}^{\dagger}_{0}(\boldsymbol{Y}^{\prime}_{0})^{\dagger})=\operatorname{tr}(\boldsymbol{V}\boldsymbol{S}^{-2}\boldsymbol{V}^{\prime})=\operatorname{tr}(\boldsymbol{S}^{-2}\boldsymbol{V}^{\prime}\boldsymbol{V})=\operatorname{tr}(\boldsymbol{S}^{-2}).$
(62)

Putting everything together, we obtain

$\displaystyle\operatorname{tr}(\boldsymbol{Y}^{\dagger}_{0}\boldsymbol{\Sigma}^{\text{mix}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{\Sigma}^{\text{mix}}_{N})\leq\max_{i\leq
N_{0}}(\sigma^{\text{mix}}_{iT})^{2}\max_{t\leq
T_{0}}(\sigma^{\text{mix}}_{Nt})^{2}\operatorname{tr}(\boldsymbol{S}^{-2}).$
(63)

The same arguments can be applied to derive the lower bound, which yields

$\displaystyle\operatorname{tr}(\boldsymbol{Y}^{\dagger}_{0}\boldsymbol{\Sigma}^{\text{mix}}_{T}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{\Sigma}^{\text{mix}}_{N})$
(64)
$\displaystyle\qquad\in\operatorname{tr}(\boldsymbol{S}^{-2})\left[\min_{i\leq
N_{0}}(\sigma^{\text{mix}}_{iT})^{2}\min_{t\leq
T_{0}}(\sigma^{\text{mix}}_{Nt})^{2},~{}\max_{i\leq
N_{0}}(\sigma^{\text{mix}}_{iT})^{2}\max_{t\leq
T_{0}}(\sigma^{\text{mix}}_{Nt})^{2}\right].$ (65)

From (65), we see that the lower and upper bounds match under
homoskedasticity.

###  A.2 Model-Based Inference: Confidence Intervals

Next, we discuss technical aspects of the variance estimators in Section
4.1.3.

Homoskedastic errors. We take note of the recent work of Agarwal et al. (2021)
in the synthetic controls literature. Agarwal et al. (2021) propose a VT PCR
estimator under the homoskedastic setting and provide a similar confidence
interval to that of (34) via large-sample approximations. Under a closely
related VT model, they propose
$\widehat{\boldsymbol{\beta}}^{\prime}\widehat{\boldsymbol{\Sigma}}_{N}^{\text{homo}}\widehat{\boldsymbol{\beta}}$
in place of
$\widehat{\boldsymbol{\alpha}}^{\prime}\widehat{\boldsymbol{\Sigma}}_{N}^{\text{homo}}\widehat{\boldsymbol{\alpha}}$.
While the point estimate of Agarwal et al. (2021) also takes the form
$\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle$, their
variance estimator only depends on $(\boldsymbol{y}_{N},\boldsymbol{Y}_{0})$;
in comparison, ours depends on
$(\boldsymbol{y}_{N},\boldsymbol{y}_{T},\boldsymbol{Y}_{0})$. Therefore, the
confidence interval as per Agarwal et al. (2021) is numerically identical for
every post-treatment point estimate while ours can vary across the post-
treatment periods, which may be favorable.

Heteroskedastic errors. Consider the heteroskedastic setting.

I: Jackknife. In the following lemma, we quantify the bias in Lemma 2.

######  Lemma 4 (Detailed restatement of Lemma 2).

(i) _[ HZ model]_ Let Assumptions 1–2 hold. If
$(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{I})$
is nonsingular, then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}^{\emph{jack}}_{T}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{hz}}_{T}+\boldsymbol{\Delta}^{\emph{hz}}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{hz},\emph{jack}}_{0}|\boldsymbol{y}_{N},\boldsymbol{Y}_{0}]=v^{\emph{hz}}_{0}+\widehat{\boldsymbol{\alpha}}^{\prime}\boldsymbol{\Delta}^{\emph{hz}}\widehat{\boldsymbol{\alpha}},$
(66)

where
$\Delta^{\emph{hz}}_{\ell\ell}=\sum_{j\neq\ell}(\sigma^{\emph{hz}}_{jT})^{2}(H^{u}_{\ell
j})^{2}(1-H^{u}_{\ell\ell})^{-2}$ for $\ell=1,\dots,N_{0}$. (ii) _[ VT model]_
Let Assumptions 3–4 hold. If
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{I})$
is nonsingular, then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{jack}}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{vt}}_{N}+\boldsymbol{\Gamma}^{\emph{vt}}\quad\text{and}\quad\mathbb{E}[\widehat{v}^{\emph{vt},\emph{jack}}_{0}|\boldsymbol{y}_{T},\boldsymbol{Y}_{0}]=v^{\emph{vt}}_{0}+\widehat{\boldsymbol{\beta}}^{\prime}\boldsymbol{\Gamma}^{\emph{vt}}\widehat{\boldsymbol{\beta}},$
(67)

where
$\Gamma^{\emph{vt}}_{\ell\ell}=\sum_{j\neq\ell}(\sigma^{\emph{vt}}_{Nj})^{2}(H^{v}_{\ell
j})^{2}(1-H^{v}_{\ell\ell})^{-2}$ for $\ell=1,\dots,T_{0}$. (iii) _[Mixed
model]_ Let Assumptions 5–6 hold. If
$(\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{H}^{u}_{\perp}\circ\boldsymbol{I})$
and
$(\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{H}^{v}_{\perp}\circ\boldsymbol{I})$
are nonsingular, then

$\displaystyle\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{T}^{\emph{jack}}|\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{mix}}_{T}+\boldsymbol{\Delta}^{\emph{mix}},\quad\mathbb{E}[\widehat{\boldsymbol{\Sigma}}_{N}^{\emph{jack}}|\boldsymbol{Y}_{0}]=\boldsymbol{\Sigma}^{\emph{mix}}_{N}+\boldsymbol{\Gamma}^{\emph{mix}},$
(68)
$\displaystyle\mathbb{E}[\widehat{v}^{\emph{mix},\emph{jack}}(\boldsymbol{Y}_{0})|\boldsymbol{Y}_{0}]$
(69)
$\displaystyle\qquad=v^{\emph{mix}}_{0}+(\boldsymbol{H}^{u}\boldsymbol{\beta}^{*})^{\prime}\boldsymbol{\Delta}^{\emph{mix}}(\boldsymbol{H}^{u}\boldsymbol{\beta}^{*})+(\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*})^{\prime}\boldsymbol{\Gamma}^{\emph{mix}}(\boldsymbol{H}^{v}\boldsymbol{\alpha}^{*})+\operatorname{tr}(\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{\Delta}^{\emph{mix}}(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{\Gamma}^{\emph{mix}}),$
(70)

where $\Delta^{\emph{mix}}_{\ell\ell}$ and $\Gamma^{\emph{mix}}_{\ell\ell}$
are defined analogously to $\Delta^{\emph{hz}}_{\ell\ell}$ and
$\Gamma^{\emph{vt}}_{\ell\ell}$, respectively, with
$(\sigma^{\emph{mix}}_{jT})^{2}$ and $(\sigma^{\emph{mix}}_{Nj})^{2}$ in place
of $(\sigma^{\emph{hz}}_{jT})^{2}$ and $(\sigma^{\emph{vt}}_{Nj})^{2}$,
respectively.

Without loss of generality, we consider the magnitudes of
$(\Delta^{\text{mix}}_{\ell\ell},\Gamma^{\text{mix}}_{\ell\ell})$. Towards
bounding the former quantity, notice that $\boldsymbol{H}^{u}$ is an
orthogonal projector and is thus idempotent, i.e.,
$(\boldsymbol{H}^{u})^{2}=\boldsymbol{H}^{u}$, and symmetric. Therefore,

$\displaystyle
H^{u}_{\ell\ell}=(H^{u}_{\ell\ell})^{2}+\sum_{j\neq\ell}(H^{u}_{\ell
j})^{2}\implies\sum_{j\neq\ell}(H^{u}_{\ell
j})^{2}=H^{u}_{\ell\ell}(1-H^{u}_{\ell\ell}).$ (71)

This yields $\Delta^{\text{mix}}_{\ell\ell}\in
H^{u}_{\ell\ell}(1-H^{u}_{\ell\ell})^{-1}[\min_{j\neq\ell}(\sigma^{\text{mix}}_{jT})^{2},\max_{j\neq\ell}(\sigma^{\text{mix}}_{jT})^{2}]$.
Since $\sum_{j\neq\ell}(H_{\ell j}^{u})^{2}\geq 0$, (71) implies
$H^{u}_{\ell\ell}\in[0,1]$. Thus, $\Delta^{\text{mix}}_{\ell\ell}=0$ if
$H^{u}_{\ell\ell}=0$ and diverges if $H^{u}_{\ell\ell}=1$. If
$H^{u}_{\ell\ell}$ takes the average value
$N_{0}^{-1}\sum_{\ell}H^{u}_{\ell\ell}=RN_{0}^{-1}$, then it follows that
$\Delta^{\text{mix}}_{\ell\ell}\in
R(N_{0}-R)^{-1}[\min_{j\neq\ell}(\sigma^{\text{mix}}_{jT})^{2},\max_{j\neq\ell}(\sigma^{\text{mix}}_{jT})^{2}]$.
A similar result is derived for $\Gamma^{\text{mix}}_{\ell\ell}$.

II: HRK-estimator. Recall Lemma 3. Consider the HZ estimator and the
invertibility of $(\boldsymbol{H}^{u}\circ\boldsymbol{H}^{u})$. A sufficient
condition is strict diagonal dominance (Varga, 1962):
$(1-H^{u}_{\ell\ell})^{2}>\sum_{j\neq\ell}(H^{u}_{\ell j})^{2}$. Using (71),
we simplify this condition as
$(1-H^{u}_{\ell\ell})^{2}>H^{u}_{\ell\ell}-(H^{u}_{\ell\ell})^{2}$. Thus,
$\max_{\ell}H^{u}_{\ell\ell}<1/2$ is a sufficient condition for invertibility.
The same arguments apply for VT regression.

Mixed variance estimator. Let us bound $\widehat{v}^{\text{mix}}_{0}$. From
(65), it follows that
$\widehat{v}^{\text{mix}}_{0}\in[\widehat{v}^{\text{mix}}_{0,\min},\widehat{v}^{\text{mix}}_{0,\max}]$,
where

$\displaystyle\widehat{v}^{\text{mix}}_{0,\min}=\min_{i\leq
N_{0}}\widehat{\sigma}^{2}_{iT}\|\widehat{\boldsymbol{\beta}}\|_{2}^{2}~{}+\min_{t\leq
T_{0}}\widehat{\sigma}^{2}_{Nt}\|\widehat{\boldsymbol{\alpha}}\|_{2}^{2}~{}-\max_{i\leq
N_{0}}\widehat{\sigma}^{2}_{iT}\max_{t\leq
T_{0}}\widehat{\sigma}^{2}_{Nt}\operatorname{tr}(\boldsymbol{S}^{-2}),$ (72)
$\displaystyle\widehat{v}^{\text{mix}}_{0,\max}=\max_{i\leq
N_{0}}\widehat{\sigma}^{2}_{iT}\|\widehat{\boldsymbol{\beta}}\|_{2}^{2}~{}+\max_{t\leq
T_{0}}\widehat{\sigma}^{2}_{Nt}\|\widehat{\boldsymbol{\alpha}}\|_{2}^{2}~{}-\min_{i\leq
N_{0}}\widehat{\sigma}^{2}_{iT}\min_{t\leq
T_{0}}\widehat{\sigma}^{2}_{Nt}\operatorname{tr}(\boldsymbol{S}^{-2}).$ (73)

Here, $\widehat{\sigma}^{2}_{iT}$ and $\widehat{\sigma}^{2}_{Nt}$ are the
$i$th and $t$th diagonal elements of $\widehat{\boldsymbol{\Sigma}}_{T}$ and
$\widehat{\boldsymbol{\Sigma}}_{N}$, respectively. Observe that
$\widehat{v}^{\text{mix}}_{0,\min}=\widehat{v}^{\text{mix}}_{0,\max}$ for
homoskedastic errors.

##  Appendix B Illustrations

This section provides details on the simulations that were absent in the main
article.

###  B.1 Implementation Details

For ridge, lasso, and elastic net regressions, we use the default scikit-learn
hyperparameters ($\lambda_{1},\lambda_{2}$). For PCR, we choose the number of
principal components $k$ via the approach described in Section 5.2. This
yields $k=$ for the Basque study, $k=3$ for the California study, and $k=4$
for the West Germany study. We implement simplex regression using the code
made available at https://matheusfacure.github.io/python-causality-
handbook/15-Synthetic-Control.html.

###  B.2 Data-Inspired Simulation Studies

Formally, we define average length (AL) as

$\displaystyle\text{AL}=\frac{1}{m}\sum_{\ell=1}^{m}\frac{(2\cdot
1.96)\sqrt{\widehat{v}^{(\ell)}_{0}}}{|\widehat{Y}^{(\ell)}_{NT}(0)|},$ (74)

where $\widehat{v}^{(\ell)}_{0}$ and $\widehat{Y}^{(\ell)}_{NT}(0)$ are the
variance and point estimates for the $\ell$th repeat.

##  Appendix C Proofs for Point Estimation

Helper lemmas. To establish Theorems 1 and 2, we first state the following
useful lemmas for the collection of regression formulations presented in
Section 3. We provide their proofs in Appendix C.6.

######  Lemma 5 (OLS).

$\emph{{HZ}}=\emph{{VT}}$ for OLS with
$(\widehat{\boldsymbol{\alpha}},\widehat{\boldsymbol{\beta}})$ as the minimum
$\ell_{2}$-norm solutions:

$\displaystyle\widehat{Y}_{NT}^{\emph{hz}}(0)$
$\displaystyle=\widehat{Y}_{NT}^{\emph{vt}}(0)=\langle\boldsymbol{y}_{N},\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{y}_{T}\rangle=\sum_{\ell=1}^{R}(1/s_{\ell})\langle\boldsymbol{y}_{N},\boldsymbol{v}_{\ell}\rangle\langle\boldsymbol{u}_{\ell},\boldsymbol{y}_{T}\rangle.$
(75)

######  Lemma 6 (PCR).

$\emph{{HZ}}=\emph{{VT}}$ for PCR with the same choice of $k<R$:

$\displaystyle\widehat{Y}_{NT}^{\emph{hz}}(0)$
$\displaystyle=\widehat{Y}_{NT}^{\emph{vt}}(0)=\langle\boldsymbol{y}_{N},(\boldsymbol{Y}_{0}^{(k)})^{\dagger}\boldsymbol{y}_{T}\rangle=\sum_{\ell=1}^{k}(1/s_{\ell})\langle\boldsymbol{y}_{N},\boldsymbol{v}_{\ell}\rangle\langle\boldsymbol{u}_{\ell},\boldsymbol{y}_{T}\rangle.$
(76)

######  Lemma 7 (Ridge).

$\emph{{HZ}}=\emph{{VT}}$ for ridge regression with the same choice of
$\lambda_{2}>0$:

$\displaystyle\widehat{Y}_{NT}^{\emph{hz}}(0)$
$\displaystyle=\widehat{Y}_{NT}^{\emph{vt}}(0)=\langle\boldsymbol{y}_{N},(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda_{2}\boldsymbol{I})^{-1}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}\rangle=\sum_{\ell=1}^{R}\frac{s_{\ell}}{s_{\ell}^{2}+\lambda_{2}}\langle\boldsymbol{y}_{N},\boldsymbol{v}_{\ell}\rangle\langle\boldsymbol{u}_{\ell},\boldsymbol{y}_{T}\rangle.$
(77)

######  Lemma 8 (Lasso).

$\emph{{HZ}}\neq\emph{{VT}}$ for lasso regression.

######  Lemma 9 (Elastic net).

$\emph{{HZ}}\neq\emph{{VT}}$ for elastic net regression.

######  Lemma 10 (Simplex regression).

$\emph{{HZ}}\neq\emph{{VT}}$ for simplex regression.

###  C.1 Proof of Theorem 1

###### Proof.

The proof is immediate from Lemmas 5–7. ∎

###  C.2 Proof of Theorem 2

###### Proof.

The proof is immediate from Lemmas 8–10. ∎

###  C.3 Proof of Corollary 1

###### Proof.

We begin with the OLS. Recall that
$\widehat{Y}^{\text{hz}}_{NT}(0)=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle$
with
$\widehat{\boldsymbol{\alpha}}=\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{y}_{T}$
and
$\widehat{Y}^{\text{vt}}_{NT}(0)=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle$
with
$\widehat{\boldsymbol{\beta}}=(\boldsymbol{Y}_{0}^{\prime})^{\dagger}\boldsymbol{y}_{N}$.
By Theorem 1, we have

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)=\widehat{Y}^{\text{vt}}_{NT}(0)=\langle\boldsymbol{y}_{N},\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{y}_{T}\rangle.$
(78)

Returning to (14), we obtain

$\displaystyle\widehat{Y}^{\text{sdid}}_{NT}(0)$
$\displaystyle=\widehat{Y}^{\text{asc}}_{NT}(0)=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle+\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle-\langle\widehat{\boldsymbol{\alpha}},\boldsymbol{Y}_{0}^{\prime}\widehat{\boldsymbol{\beta}}\rangle=2\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle-\langle\widehat{\boldsymbol{\alpha}},\boldsymbol{Y}_{0}^{\prime}\widehat{\boldsymbol{\beta}}\rangle.$
(79)

Recall
$(\boldsymbol{Y}_{0}^{\prime})^{\dagger}=(\boldsymbol{Y}_{0}^{\dagger})^{\prime}$.
Therefore,

$\displaystyle\langle\widehat{\boldsymbol{\alpha}},\boldsymbol{Y}_{0}^{\prime}\widehat{\boldsymbol{\beta}}\rangle$
$\displaystyle=\boldsymbol{y}^{\prime}_{T}(\boldsymbol{Y}_{0}^{\prime})^{\dagger}\boldsymbol{Y}_{0}^{\prime}(\boldsymbol{Y}_{0}^{\prime})^{\dagger}\boldsymbol{y}_{N}=\boldsymbol{y}^{\prime}_{T}(\boldsymbol{Y}_{0}^{\prime})^{\dagger}\boldsymbol{y}_{N}=\boldsymbol{y}^{\prime}_{T}\widehat{\boldsymbol{\beta}}.$
(80)

Plugging (80) into (79), we conclude

$\displaystyle\widehat{Y}^{\text{sdid}}_{NT}(0)$
$\displaystyle=\widehat{Y}^{\text{asc}}_{NT}(0)=2\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle-\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle=\widehat{Y}^{\text{vt}}_{NT}(0)=\widehat{Y}^{\text{hz}}_{NT}(0).$
(81)

Now, observe that the same arguments above hold when
$\boldsymbol{Y}_{0}^{(k)}$ takes the place of $\boldsymbol{Y}_{0}$ for any
$k<R$. Therefore, the same reduction can be derived for PCR. ∎

###  C.4 Proof of Corollary 2

###### Proof.

Let $\boldsymbol{Y}^{\text{hz}}_{0}=[\boldsymbol{1},\boldsymbol{Y}_{0}]$ and
$\boldsymbol{Y}^{\text{vt}}_{0}=[\boldsymbol{1},\boldsymbol{Y}^{\prime}_{0}]$.
The proof is immediate from Theorem 1 by noting that
$(\boldsymbol{Y}^{\text{hz}}_{0})^{\prime}\neq\boldsymbol{Y}^{\text{vt}}_{0}$.
∎

###  C.5 Proof of Corollary 3

###### Proof.

Consider HZ ridge regression. To begin, the optimality conditions give

$\displaystyle\nabla_{(\alpha_{0},\alpha_{1},\boldsymbol{\alpha})}\left\\{\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}-\alpha_{0}\boldsymbol{1}\|_{2}^{2}+\|\boldsymbol{y}_{N}-\boldsymbol{\alpha}_{1}\boldsymbol{1}\|_{2}^{2}+\lambda\|\boldsymbol{\alpha}\|_{2}^{2}\right\\}$
$\displaystyle=0.$ (82)

Solving for $\alpha_{0}$, we have
$\widehat{\alpha}_{0}=(1/N_{0})\langle\boldsymbol{y}_{T},\boldsymbol{1}\rangle$,
where we have used the fact that
$\boldsymbol{Y}^{\prime}_{0}\boldsymbol{1}=\boldsymbol{0}$. Solving for
$\alpha_{1}$, we have
$\widehat{\alpha}_{1}=(1/T_{0})\langle\boldsymbol{y}_{N},\boldsymbol{1}\rangle$.
Finally, solving for $\boldsymbol{\alpha}$, we have
$\widehat{\boldsymbol{\alpha}}=(\boldsymbol{Y}^{\prime}_{0}\boldsymbol{Y}_{0}+\lambda\boldsymbol{I})^{-1}\boldsymbol{Y}^{\prime}_{0}\boldsymbol{y}_{T}$.

Switching gears to VT ridge regression, similar arguments yield
$\widehat{\beta}_{0}=(1/T_{0})\langle\boldsymbol{y}_{N},\boldsymbol{1}\rangle$,
$\widehat{\beta}_{1}=(1/N_{0})\langle\boldsymbol{y}_{T},\boldsymbol{1}\rangle$,
and
$\widehat{\boldsymbol{\beta}}=(\boldsymbol{Y}_{0}\boldsymbol{Y}^{\prime}_{0}+\lambda\boldsymbol{I})\boldsymbol{Y}_{0}\boldsymbol{y}_{N}$.

We establish our desired result by invoking Theorem 1 to obtain
$\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle$.
Next, we observe that $\widehat{\alpha}_{0}=\widehat{\beta}_{1}$ and
$\widehat{\beta}_{0}=\widehat{\alpha}_{1}$. This proves that HZ and VT ridge
yield numerically identical point estimates. Setting $\lambda=0$ and using the
pseudoinverse, we have our result for OLS. The result for PCR is then
established from the OLS result by substituting $\boldsymbol{Y}_{0}^{(k)}$ for
$\boldsymbol{Y}_{0}$. This completes the proof. ∎

###  C.6 Proofs of Point Estimation Lemmas

####  C.6.1 Proof of Lemma 5: OLS

###### Proof.

Consider HZ regression. By the optimality conditions,

$\displaystyle\nabla_{\boldsymbol{\alpha}}\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}\|_{2}^{2}$
$\displaystyle=0.$ (83)

Solving for $\boldsymbol{\alpha}$, we derive the well-known “normal equations”

$\displaystyle\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}\boldsymbol{\alpha}=\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}.$
(84)

Using the pseudoinverse, we obtain

$\displaystyle\widehat{\boldsymbol{\alpha}}$
$\displaystyle=(\boldsymbol{Y}^{\prime}_{0}\boldsymbol{Y}_{0})^{\dagger}\boldsymbol{Y}^{\prime}_{0}\boldsymbol{y}_{T}=\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{y}_{T}$
(85)

Observe that this corresponds to the unique minimum $\ell_{2}$-norm solution
that lies within the rowspace of $\boldsymbol{Y}_{0}$. Therefore, the HZ
prediction is given by

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle=\langle\boldsymbol{y}_{N},\boldsymbol{Y}^{\dagger}_{0}\boldsymbol{y}_{T}\rangle.$
(86)

Following the arguments above for VT regression, it follows that

$\displaystyle\widehat{\boldsymbol{\beta}}$
$\displaystyle=(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime})^{\dagger}\boldsymbol{Y}_{0}\boldsymbol{y}_{N}=(\boldsymbol{Y}_{0}^{\prime})^{\dagger}\boldsymbol{y}_{N},$
(87)

which corresponds to the unique minimum $\ell_{2}$-norm solution that lies
within the columnspace of $\boldsymbol{Y}_{0}$. Therefore,

$\displaystyle\widehat{Y}^{\text{vt}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle=\langle\boldsymbol{y}_{T},(\boldsymbol{Y}^{\prime}_{0})^{\dagger}\boldsymbol{y}_{N}\rangle.$
(88)

Given that
$(\boldsymbol{Y}_{0}^{\prime})^{\dagger}=(\boldsymbol{Y}_{0}^{\dagger})^{\prime}$,
we conclude

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{N},\boldsymbol{Y}_{0}^{\dagger}\boldsymbol{y}_{T}\rangle=\langle\boldsymbol{y}_{T},(\boldsymbol{Y}_{0}^{\prime})^{\dagger}\boldsymbol{y}_{N}\rangle=\widehat{Y}^{\text{vt}}_{NT}(0).$
(89)

∎

####  C.6.2 Proof of Lemma 6: PCR

###### Proof.

Consider HZ regression with any $k<R$. Let
$\boldsymbol{U}_{k}\in\mathbb{R}^{N_{0}\times k}$ and
$\boldsymbol{V}_{k}\in\mathbb{R}^{T_{0}\times k}$ denote the matrices formed
by the top $k$ left and right singular vectors, respectively, and
$\boldsymbol{S}_{k}\in\mathbb{R}^{k\times k}$ denote the matrix of top $k$
singular values. Observe that

$\displaystyle\left((\boldsymbol{Y}_{0}^{(k)})^{\prime}\boldsymbol{Y}_{0}^{(k)}\right)^{\dagger}(\boldsymbol{Y}_{0}^{(k)})^{\prime}$
$\displaystyle=(\boldsymbol{V}_{k}\boldsymbol{S}_{k}^{-2}\boldsymbol{V}^{\prime}_{k})\boldsymbol{V}_{k}\boldsymbol{S}_{k}\boldsymbol{U}^{\prime}_{k}=\boldsymbol{V}_{k}\boldsymbol{S}_{k}^{-1}\boldsymbol{U}^{\prime}_{k}=(\boldsymbol{Y}_{0}^{(k)})^{\dagger}.$
(90)

Therefore,
$\widehat{\boldsymbol{\alpha}}=(\boldsymbol{Y}_{0}^{(k)})^{\dagger}\boldsymbol{y}_{T}$,
which corresponds to the unique minimum $\ell_{2}$-norm solution that lies
within the rowspace of $\boldsymbol{Y}^{(k)}_{0}$. Following the proof of
Lemma 5, we conclude that

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle=\langle\boldsymbol{y}_{N},(\boldsymbol{Y}_{0}^{(k)})^{\dagger}\boldsymbol{y}_{T}\rangle.$
(91)

Similarly, for VT regression, we note that

$\displaystyle\left(\boldsymbol{Y}_{0}^{(k)}(\boldsymbol{Y}_{0}^{(k)})^{\prime}\right)^{\dagger}\boldsymbol{Y}_{0}^{(k)}$
$\displaystyle=(\boldsymbol{U}_{k}\boldsymbol{S}_{k}^{-2}\boldsymbol{U}^{\prime}_{k})\boldsymbol{U}_{k}\boldsymbol{S}_{k}\boldsymbol{V}^{\prime}_{k}=\boldsymbol{U}_{k}\boldsymbol{S}_{k}^{-1}\boldsymbol{V}^{\prime}_{k}=((\boldsymbol{Y}_{0}^{(k)})^{\prime})^{\dagger}.$
(92)

In turn, we have
$\widehat{\boldsymbol{\beta}}=((\boldsymbol{Y}_{0}^{(k)})^{\prime})^{\dagger}\boldsymbol{y}_{N}$,
which corresponds to the unique minimum $\ell_{2}$-norm solution that lies
within the columnspace of $\boldsymbol{Y}^{(k)}_{0}$. Moreover,

$\displaystyle\widehat{Y}^{\text{vt}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle=\langle\boldsymbol{y}_{T},((\boldsymbol{Y}_{0}^{(k)})^{\prime})^{\dagger}\boldsymbol{y}_{N}\rangle.$
(93)

We finish by establishing

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)=\langle\boldsymbol{y}_{N},(\boldsymbol{Y}_{0}^{(k)})^{\dagger}\boldsymbol{y}_{T}\rangle=\langle\boldsymbol{y}_{T},((\boldsymbol{Y}_{0}^{(k)})^{\prime})^{\dagger}\boldsymbol{y}_{N}\rangle=\widehat{Y}^{\text{vt}}_{NT}(0).$
(94)

∎

Helper lemma. To establish Lemmas 7–9, we first establish a general result in
Lemma 11 for $\ell_{p}$-penalties, where $p=2/K$ and $K$ is an integer $\geq
1$, based on the contributions of Hoff (2017). More formally, consider

  * (a)

HZ regression: for $K\geq 1$ and $\lambda>0$,

$\displaystyle\widehat{\boldsymbol{\alpha}}=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}}\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}\|_{2}^{2}~{}+~{}\lambda\|\boldsymbol{\alpha}\|_{p}^{p}$
(95)
$\displaystyle\widehat{Y}_{NT}^{\text{hz}}(0)=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}\rangle.$
(96)

  * (b)

VT regression: for $K\geq 1$ and $\lambda>0$,

$\displaystyle\widehat{\boldsymbol{\beta}}=\operatorname*{\arg\\!\min}_{\boldsymbol{\beta}}\|\boldsymbol{y}_{N}-\boldsymbol{Y}_{0}^{\prime}\boldsymbol{\beta}\|_{2}^{2}~{}+~{}\lambda\|\boldsymbol{\beta}\|_{p}^{p}$
(97)
$\displaystyle\widehat{Y}_{NT}^{\text{vt}}(0)=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}\rangle.$
(98)

We remark that $K=1$ and $K=2$ yield ridge and lasso regression, respectively,
while $K>2$ yields non-convex penalties. We relegate the proof of Lemma 11 to
Appendix C.6.7.

######  Lemma 11.

For any $K\geq 1$ and $\lambda>0$, a _HZ_ and _VT_ regression solution is

$\displaystyle\widehat{Y}^{\emph{hz}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}_{1}\circ\dots\circ\widehat{\boldsymbol{\alpha}}_{K}\rangle$
(99) $\displaystyle\widehat{Y}^{\emph{vt}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}_{1}\circ\dots\circ\widehat{\boldsymbol{\beta}}_{K}\rangle,$
(100)

where for every $k\leq K$,

$\displaystyle\widehat{\boldsymbol{\alpha}}_{k}$
$\displaystyle=\left(\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}_{\sim
k})\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}_{\sim
k})+\frac{\lambda}{K}\boldsymbol{I}\right)^{-1}\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}_{\sim
k})\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T},$ (101)
$\displaystyle\widehat{\boldsymbol{\beta}}_{k}$
$\displaystyle=\left(\boldsymbol{D}(\widehat{\boldsymbol{\beta}}_{\sim
k})\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{D}(\widehat{\boldsymbol{\beta}}_{\sim
k})+\frac{\lambda}{K}\boldsymbol{I}\right)^{-1}\boldsymbol{D}(\widehat{\boldsymbol{\beta}}_{\sim
k})\boldsymbol{Y}_{0}\boldsymbol{y}_{N},$ (102)

$\widehat{\boldsymbol{\alpha}}_{\sim
k}=\widehat{\boldsymbol{\alpha}}_{1}\circ\dots\circ\widehat{\boldsymbol{\alpha}}_{k-1}\circ\widehat{\boldsymbol{\alpha}}_{k+1}\circ\dots\circ\widehat{\boldsymbol{\alpha}}_{K}$,
$\widehat{\boldsymbol{\beta}}_{\sim
k}=\widehat{\boldsymbol{\beta}}_{1}\circ\dots\circ\widehat{\boldsymbol{\beta}}_{k-1}\circ\widehat{\boldsymbol{\beta}}_{k+1}\circ\dots\circ\widehat{\boldsymbol{\beta}}_{K}$,
and $\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}_{\sim k})$ and
$\boldsymbol{D}(\widehat{\boldsymbol{\beta}}{\sim k})$ are diagonal matrices
formed from $\widehat{\boldsymbol{\alpha}}_{\sim k}$ and
$\widehat{\boldsymbol{\beta}}_{\sim k}$, respectively.

####  C.6.3 Proof of Lemma 7: Ridge Regression

###### Proof.

By Lemma 11 for $K=1$ and $\lambda=\lambda_{2}>0$, the HZ regression solution
is

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{N},(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda_{2}\boldsymbol{I})^{-1}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}\rangle.$
(103)

Similarly, the VT regression solution is given by

$\displaystyle\widehat{Y}^{\text{vt}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{T},(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}+\lambda_{2}\boldsymbol{I})^{-1}\boldsymbol{Y}_{0}\boldsymbol{y}_{N}\rangle.$
(104)

Since
$(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda\boldsymbol{I})^{-1}\boldsymbol{Y}_{0}^{\prime}=\boldsymbol{Y}_{0}^{\prime}(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}+\lambda\boldsymbol{I})^{-1}$,
it follows that
$\widehat{Y}^{\text{hz}}_{NT}(0)=\widehat{Y}^{\text{vt}}_{NT}(0)$. ∎

####  C.6.4 Proof of Lemma 8: Lasso Regression

###### Proof.

By Lemma 11 for $K=2$ and $\lambda=\lambda_{1}>0$, a HZ regression solution is

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}_{1}\circ\widehat{\boldsymbol{\alpha}}_{2}\rangle,$
(105)

where

$\displaystyle\widehat{\boldsymbol{\alpha}}_{1+k}$
$\displaystyle=\left(\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}_{2-k})\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}_{2-k})+\frac{\lambda_{1}}{2}\boldsymbol{I}\right)^{-1}\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}_{2-k})\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}$
(106)

for $k\in\\{0,1\\}$. Similarly, a VT regression solution is given by

$\displaystyle\widehat{Y}^{\text{vt}}_{NT}(0)$
$\displaystyle=\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}_{1}\circ\widehat{\boldsymbol{\beta}}_{2}\rangle,$
(107)

where

$\displaystyle\widehat{\boldsymbol{\beta}}_{1+k}$
$\displaystyle=\left(\boldsymbol{D}(\widehat{\boldsymbol{\beta}}_{2-k})\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{D}(\widehat{\boldsymbol{\beta}}_{2-k})+\frac{\lambda_{1}}{2}\boldsymbol{I}\right)^{-1}\boldsymbol{D}(\widehat{\boldsymbol{\beta}}_{2-k})\boldsymbol{Y}_{0}\boldsymbol{y}_{N}$
(108)

for $k\in\\{0,1\\}$. Leveraging (105) and (107), we find that the HZ
regression solution can be linear in $y$ and at least quadratic in $q$. On the
other hand, the VT regression solution can be linear in $q$ and at least
quadratic in $y$. Since the lasso solution is unique under the assumption the
entries of $\boldsymbol{Y}_{0}$ are drawn from a continuous distribution, this
implies that HZ and VT regressions do not yield matching solutions in general.
∎

####  C.6.5 Proof of Lemma 9: Elastic Net Regression

###### Proof.

Consider HZ regression. We rewrite (1) in a lasso formulation:

$\displaystyle\widehat{\boldsymbol{\alpha}}^{*}$
$\displaystyle=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}^{*}}\|\boldsymbol{y}_{T}^{*}-\boldsymbol{Y}_{0}^{*}\boldsymbol{\alpha}^{*}\|_{2}^{2}~{}+~{}\lambda^{*}\|\boldsymbol{\alpha}^{*}\|_{1},$
(109)

where

$\displaystyle q^{*}=\begin{pmatrix}\boldsymbol{y}_{T}\\\
\boldsymbol{0}\end{pmatrix},\quad\boldsymbol{Y}_{0}^{*}=\frac{1}{\sqrt{1+\lambda_{2}}}\begin{pmatrix}\boldsymbol{Y}_{0}\\\
\sqrt{\lambda_{2}}\boldsymbol{I}\end{pmatrix},\quad\lambda^{*}=\frac{\lambda_{1}}{\sqrt{1+\lambda_{2}}},\quad\boldsymbol{\alpha}^{*}=(\sqrt{1+\lambda_{2}})\boldsymbol{\alpha}.$
(110)

We apply Lemma 11 to (109) with $K=2$ and $\lambda=\lambda^{*}>0$ to obtain

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\frac{\langle\boldsymbol{y}_{N},\widehat{\boldsymbol{\alpha}}^{*}_{1}\circ\widehat{\boldsymbol{\alpha}}^{*}_{2}\rangle}{\sqrt{1+\lambda_{2}}},$
(111)

where

$\displaystyle\widehat{\boldsymbol{\alpha}}^{*}_{1+k}$
$\displaystyle=\left(\frac{1}{\sqrt{1+\lambda_{2}}}\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}^{*}_{2-k})(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda_{2}\boldsymbol{I})\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}^{*}_{2-k})+\frac{\lambda_{1}}{2}\boldsymbol{I}\right)^{-1}\boldsymbol{D}(\widehat{\boldsymbol{\alpha}}^{*}_{2-k})\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}$
(112)

for $k\in\\{0,1\\}$. Similarly, for VT regression, we proceed as above to
obtain

$\displaystyle\widehat{Y}^{\text{vt}}_{NT}(0)$
$\displaystyle=\frac{\langle\boldsymbol{y}_{T},\widehat{\boldsymbol{\beta}}^{*}_{1}\circ\widehat{\boldsymbol{\beta}}^{*}_{2}\rangle}{\sqrt{1+\lambda_{2}}},$
(113)

where

$\displaystyle\widehat{\boldsymbol{\beta}}^{*}_{1+k}$
$\displaystyle=\left(\frac{1}{\sqrt{1+\lambda_{2}}}\boldsymbol{D}(\widehat{\boldsymbol{\beta}}^{*}_{2-k})(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}+\lambda_{2}\boldsymbol{I})\boldsymbol{D}(\widehat{\boldsymbol{\beta}}^{*}_{2-k})+\frac{\lambda_{1}}{2}\boldsymbol{I}\right)^{-1}\boldsymbol{D}(\widehat{\boldsymbol{\beta}}^{*}_{2-k})\boldsymbol{Y}_{0}\boldsymbol{y}_{N}$
(114)

for $k\in\\{0,1\\}$. Leveraging (111) and (113), we find that the HZ
regression solution can be linear in $y$ and at least quadratic in $q$. On the
other hand, the VT regression solution can be linear in $q$ and at least
quadratic in $y$. Since the elastic net regression solution is unique,
provided $\lambda_{2}>0$, this implies that HZ and VT regressions do not yield
matching solutions in general. ∎

####  C.6.6 Proof of Lemma 10: Simplex Regression

###### Proof.

Consider HZ regression. We write the Lagrangian of (6) as

$\displaystyle\widehat{\boldsymbol{\alpha}}$
$\displaystyle=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}}~{}\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}\boldsymbol{\alpha}\|_{2}^{2}~{}+~{}\lambda\|\boldsymbol{\alpha}\|_{2}^{2}~{}-~{}(\boldsymbol{\theta}^{\text{hz}})^{\prime}\boldsymbol{\alpha}+\nu^{\text{hz}}(\boldsymbol{1}^{\prime}\boldsymbol{\alpha}-1),$
(115)

where $\boldsymbol{\theta}^{\text{hz}}\in\mathbb{R}^{T_{0}}$ and
$\nu^{\text{hz}}\in\mathbb{R}$. By the Karush-Kuhn-Tucker (KKT) conditions,
optimality is achieved if the following are satisfied:

$\displaystyle\widehat{\boldsymbol{\alpha}}\succeq\boldsymbol{0},\quad\boldsymbol{1}^{\prime}\widehat{\boldsymbol{\alpha}}=1,$
(116)
$\displaystyle\widehat{\boldsymbol{\theta}}^{\text{hz}}\succeq\boldsymbol{0},$
(117)
$\displaystyle\widehat{\theta}^{\text{hz}}_{i}\widehat{\alpha}_{i}=0\quad\text{for
}i=1,\dots,T_{0},$ (118)
$\displaystyle\widehat{\boldsymbol{\alpha}}=(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda\boldsymbol{I})^{-1}\left(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}+\frac{1}{2}\widehat{\boldsymbol{\theta}}^{\text{hz}}-\frac{\widehat{\nu}^{\text{hz}}}{2}\boldsymbol{1}\right).$
(119)

Therefore, given primal and dual feasible variables
$(\widehat{\boldsymbol{\alpha}},\widehat{\boldsymbol{\theta}}^{\text{hz}},\widehat{\nu}^{\text{hz}})$,
we can write the final HZ prediction as

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\widehat{Y}^{\text{hz},\text{ols}}_{NT}(0)+(1/2)\boldsymbol{y}^{\prime}_{N}(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda\boldsymbol{I})^{-1}(\widehat{\boldsymbol{\theta}}^{\text{hz}}-\widehat{\nu}^{\text{hz}}\boldsymbol{1}),$
(120)

where
$\widehat{Y}^{\text{hz},\text{ols}}_{NT}(0)=\boldsymbol{y}^{\prime}_{N}(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda\boldsymbol{I})^{-1}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}$
converges to the prediction corresponding to the OLS solution with minimum
$\ell_{2}$-norm as $\lambda\rightarrow 0^{+}$. Similarly, for VT regression,
the KKT conditions are

$\displaystyle\widehat{\boldsymbol{\beta}}\succeq\boldsymbol{0},\quad\boldsymbol{1}^{\prime}\widehat{\boldsymbol{\beta}}=1,$
(121)
$\displaystyle\widehat{\boldsymbol{\theta}}^{\text{vt}}\succeq\boldsymbol{0},$
(122)
$\displaystyle\widehat{\theta}^{\text{vt}}_{i}\widehat{\beta}_{i}=0\quad\text{for
}i=1,\dots,N_{0},$ (123)
$\displaystyle\widehat{\boldsymbol{\beta}}=(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}+\lambda\boldsymbol{I})^{-1}\left(\boldsymbol{Y}_{0}\boldsymbol{y}_{N}+\frac{1}{2}\widehat{\boldsymbol{\theta}}^{\text{vt}}-\frac{\widehat{\nu}^{\text{vt}}}{2}\boldsymbol{1}\right).$
(124)

For primal and dual feasible variables
$(\widehat{\boldsymbol{\beta}},\widehat{\boldsymbol{\theta}}^{\text{vt}},\widehat{\nu}^{\text{vt}})$,
this yields

$\displaystyle\widehat{Y}^{\text{vt}}_{NT}(0)$
$\displaystyle=\widehat{Y}^{\text{vt},\text{ols}}_{NT}(0)+(1/2)\boldsymbol{y}^{\prime}_{T}(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}+\lambda\boldsymbol{I})^{-1}(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1}),$
(125)

where
$\widehat{Y}^{\text{vt},\text{ols}}_{NT}(0)=\boldsymbol{y}^{\prime}_{T}(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}+\lambda\boldsymbol{I})^{-1}\boldsymbol{Y}_{0}\boldsymbol{y}_{N}$
converges to the prediction corresponding to the OLS solution with minimum
$\ell_{2}$-norm as $\lambda\rightarrow 0^{+}$. Notably, as per Theorem 1,
$\widehat{Y}^{\text{hz},\text{ols}}_{NT}(0)=\widehat{Y}^{\text{vt},\text{ols}}_{NT}(0)=\widehat{Y}^{\text{ols}}_{NT}(0)$
for any $\lambda\geq 0$. As a result,

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=\widehat{Y}^{\text{ols}}_{NT}(0)+(1/2)\boldsymbol{y}^{\prime}_{N}(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}+\lambda\boldsymbol{I})^{-1}(\widehat{\boldsymbol{\theta}}^{\text{hz}}-\widehat{\nu}^{\text{hz}}\boldsymbol{1})$
(126) $\displaystyle\widehat{Y}^{\text{vt}}_{NT}(0)$
$\displaystyle=\widehat{Y}^{\text{ols}}_{NT}(0)+(1/2)\boldsymbol{y}^{\prime}_{T}(\boldsymbol{Y}_{0}\boldsymbol{Y}_{0}^{\prime}+\lambda\boldsymbol{I})^{-1}(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1}).$
(127)

As seen from (126) and (127), the leading terms in the HZ and VT simplex
regression predictions are identical. The remaining terms, however, can differ
from one another. As an example, consider $N=T$ with

$\displaystyle\boldsymbol{Y}_{0}=\boldsymbol{I},\quad\boldsymbol{y}_{N}=\boldsymbol{0},\quad\boldsymbol{y}_{T}=(1+\lambda)(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1}).$
(128)

By construction, observe that

$\displaystyle\widehat{\boldsymbol{\beta}}=\frac{1}{2(1+\lambda)}(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1}).$
(129)

Recall from the KKT conditions for VT regression that
$\widehat{\boldsymbol{\beta}}\succeq\boldsymbol{0}$ and
$\boldsymbol{1}^{\prime}\widehat{\boldsymbol{\beta}}=1$. Therefore, at least
one entry of
$(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1})$
must be strictly positive. This yields

$\displaystyle(1+\lambda)^{-1}\boldsymbol{y}^{\prime}_{T}(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1})$
$\displaystyle=(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1})^{\prime}(\widehat{\boldsymbol{\theta}}^{\text{vt}}-\widehat{\nu}^{\text{vt}}\boldsymbol{1})>0.$
(130)

Plugging (128) and (130) into (126) and (127), we obtain

$\displaystyle\widehat{Y}^{\text{hz}}_{NT}(0)$
$\displaystyle=0\quad\text{and}\quad\widehat{Y}^{\text{vt}}_{NT}(0)>0,$ (131)

which concludes our proof. ∎

####  C.6.7 Proof of Lemma 11: $\ell_{p}$-penalties

###### Proof.

We recall the Hadamard product parametrization (HPP): for any vector
$\boldsymbol{z}$ and integer $K\geq 1$,

$\displaystyle\|\boldsymbol{z}\|_{p}^{p}$
$\displaystyle=\min_{\boldsymbol{z}_{1}\circ\dots\circ\boldsymbol{z}_{K}=\boldsymbol{z}}\frac{1}{K}\sum_{k=1}^{K}\|\boldsymbol{z}_{k}\|_{2}^{2},$
(132)

where $\circ$ denotes the Hadamard (componentwise) product. We rewrite our
subclass of $\ell_{p}$-penalties, i.e., (95) and (97), as sums of
$\ell_{2}$-penalties via the HPP technique:

$\displaystyle(\widehat{\boldsymbol{\alpha}}_{1},\dots,\widehat{\boldsymbol{\alpha}}_{K})$
$\displaystyle=\operatorname*{\arg\\!\min}_{\boldsymbol{\alpha}_{1},\dots,\boldsymbol{\alpha}_{K}}\|\boldsymbol{y}_{T}-\boldsymbol{Y}_{0}(\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{K})\|_{2}^{2}~{}+~{}\frac{\lambda}{K}\sum_{k=1}^{K}\|\boldsymbol{\alpha}_{k}\|_{2}^{2}$
(133)
$\displaystyle(\widehat{\boldsymbol{\beta}}_{1},\dots,\widehat{\boldsymbol{\beta}}_{K})$
$\displaystyle=\operatorname*{\arg\\!\min}_{\boldsymbol{\beta}_{1},\dots,\boldsymbol{\beta}_{K}}\|\boldsymbol{y}_{N}-\boldsymbol{Y}_{0}^{\prime}(\boldsymbol{\beta}_{1}\circ\dots\circ\boldsymbol{\beta}_{K})\|_{2}^{2}~{}+~{}\frac{\lambda}{K}\sum_{k=1}^{K}\|\boldsymbol{\beta}_{k}\|_{2}^{2},$
(134)

where
$\widehat{\boldsymbol{\alpha}}=\widehat{\boldsymbol{\alpha}}_{1}\circ\dots\circ\widehat{\boldsymbol{\alpha}}_{K}$
and
$\widehat{\boldsymbol{\beta}}=\widehat{\boldsymbol{\beta}}_{1}\circ\dots\circ\widehat{\boldsymbol{\beta}}_{K}$.
Below, we leverage the results of Hoff (2017), which provides an alternating
ridge regression algorithm to solve for (133)–(134).

Consider HZ regression. Let us solve for $\boldsymbol{\alpha}_{k}$ for
$k\in[K]$ by fixing $\boldsymbol{\alpha}_{k^{\prime}}$ for $k^{\prime}\neq k$.
By the optimality conditions,

$\displaystyle\nabla_{\boldsymbol{\alpha}_{k}}\left\\{(\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{K})^{\prime}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}(\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{K})-2(\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{K})^{\prime}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}+\frac{\lambda}{K}\boldsymbol{\alpha}_{k}^{\prime}\boldsymbol{\alpha}_{k}\right\\}=0.$
(135)

In order to solve for (135), observe that

$\displaystyle(\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{K})^{\prime}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}(\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{K})=\boldsymbol{\alpha}_{k}^{\prime}(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}\circ\boldsymbol{\alpha}_{\sim
k}\boldsymbol{\alpha}_{\sim k}^{\prime})\boldsymbol{\alpha}_{k}$ (136)
$\displaystyle(\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{K})^{\prime}\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}=\boldsymbol{\alpha}_{k}^{\prime}(\boldsymbol{\alpha}_{\sim
k}\circ\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T}),$ (137)

where $\boldsymbol{\alpha}_{\sim
k}=\boldsymbol{\alpha}_{1}\circ\dots\circ\boldsymbol{\alpha}_{k-1}\circ\boldsymbol{\alpha}_{k+1}\circ\dots\circ\boldsymbol{\alpha}_{K}$.
This allows us to rewrite (135) as

$\displaystyle\nabla_{\boldsymbol{\alpha}_{k}}\left\\{\boldsymbol{\alpha}_{k}^{\prime}\left(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}\circ\boldsymbol{\alpha}_{\sim
k}\boldsymbol{\alpha}_{\sim
k}^{\prime}+\frac{\lambda}{K}\boldsymbol{I}\right)\boldsymbol{\alpha}_{k}-2\boldsymbol{\alpha}_{k}^{\prime}(\boldsymbol{\alpha}_{\sim
k}\circ\boldsymbol{Y}_{0}^{\prime}\boldsymbol{y}_{T})\right\\}=0.$ (138)

This is quadratic in $\boldsymbol{\alpha}_{k}$ for fixed
$\boldsymbol{\alpha}_{\sim k}$. Thus, the unique minimizer at convergence is

$\displaystyle\widehat{\boldsymbol{\alpha}}_{k}$
$\displaystyle=\left(\boldsymbol{Y}_{0}^{\prime}\boldsymbol{Y}_{0}\circ\widehat{\boldsymbol{\alpha}}_{\sim
k}\widehat{\boldsymbol{\alpha}}_{\sim