# Targeted Function Balancing††thanks: I thank Chad Hazlett for advising me
throughout the course of this project. I also thank Erin Hartman, Onyebuchi
Arah, Mark Handcock, Avi Feller, Yixin Wang, and the UCLA Causality Reading
Group for their valuable comments and suggestions.

Leonard Wainstein Ph.D. in Statistics, University of California, Los Angeles.
Email<EMAIL_ADDRESS>
(January 5, 2022)

###### Abstract

This paper introduces Targeted Function Balancing (TFB), a covariate balancing
framework for estimating the causal effect of a binary intervention on an
outcome. TFB first regresses the outcome on observed covariates, and then
selects weights that balance functions (of the covariates) that are
probabilistically near the resulting regression function. This yields balance
in the predicted values of the regression function and the covariates, with
the regression function’s estimated variance determining how much balance in
the covariates is sufficient. Notably, TFB demonstrates that intentionally
leaving imbalance in some covariates can increase efficiency without
introducing bias, challenging traditions that warn against imbalance in any
variable. TFB is shown here to be useful in high-dimensional settings, with
particular focus given to kernel regularized least squares and the LASSO as
potential regression estimators. With the former, TFB contributes to the
literature of kernel-based weights. As for the LASSO, TFB uses the regression
function’s estimated variance to prioritize balance in certain dimensions of
the covariates, a feature that can be exploited to great effects by choosing a
sparse regression estimator. Most appealing is that TFB is entirely defined by
the choice of regression estimator and an estimator for the resulting
function’s variance, turning the problem of how best to balance the covariates
into how best to model the outcome. Additionally, this paper introduces a
balance diagnostic, Targeted Function Imbalance, which TFB minimizes subject
to constraints on its weights’ variance, and may have other applications.

Keywords: Augmented estimator, average treatment effect, balancing weight,
causal inference, kernel, propensity score

##  1 Introduction

Weighting is commonly employed to estimate the causal effect of an
intervention, $D_{i}$, on an outcome of interest, $Y_{i}$. When the average
treatment effect on the treated is the target estimand, and given covariates,
$X_{i}$, that make up all confounders, investigators seek weights that render
the control (i.e., untreated) group more similar to the treated group on
$X_{i}$. Weights based on the probability of being treated given $X_{i}$,
$\pi(X_{i})$ or the propensity score (Rosenbaum and Rubin,, 1983), equate the
distribution of $X_{i}$ in the control group to that in the treated group in
expectation, and yield a weighted difference in means,
$\hat{\tau}_{\mathrm{wdim}}$, that is unbiased. But these weights require
$\pi$ to be known or consistently estimated, which is unrealistic, and
misspecifying $\pi$ can bring large biases (e.g., Kang and Schafer,, 2007).
Another approach is to find weights that make the means of $X_{i}$ more
similar between the groups, referred to here as “balancing”, or seeking
“balance” in, $X_{i}$. This can be easily extended to weights that instead, or
additionally, balance a transformation of $X_{i}$ (e.g., its squares and pair-
wise interactions) by replacing $X_{i}$ with, or adding to it, said
transformation.

One family of balancing methods includes those that equate the means of
$X_{i}$ in both groups, or balance $X_{i}$ “exactly” (e.g., Hainmueller,,
2012; Chan et al.,, 2016). However, these methods have two limitations — one
of feasibility, and one of specification. On feasibility, such weights may not
exist (e.g., when $X_{i}$ is of large dimension) or have high variance,
resulting in a high variance $\hat{\tau}_{\mathrm{wdim}}$. On specification,
such weights only guarantee an unbiased $\hat{\tau}_{\mathrm{wdim}}$ when the
conditional expectation function of $Y_{i}$ given $X_{i}$ and without
treatment, $f_{0}(X_{i})=\mathbbm{E}(Y_{i}\ |\ D_{i}=0,X_{i})$, is linear in
$X_{i}$. Further, these limitations compound each other — exactly balancing a
high-dimensional transformation of $X_{i}$ (e.g., high-degree polynomial
terms) would yield an unbiased estimator for a wider range of $f_{0}$, but may
not be possible. Thus, methods that balance $X_{i}$ “approximately”, or reduce
the “imbalance” (i.e., differences in the means) to a level within some small
threshold, have entered the literature in recent years (e.g, Zubizarreta,,
2015; Wang and Zubizarreta,, 2020). This approach allows weights that instead
balance high-dimensional transformations of $X_{i}$, such as basis functions
from a reproducing kernel Hilbert space (e.g, Kallus,, 2020; Hirshberg et
al.,, 2021; Wong and Chan,, 2018; Hazlett,, 2020; Tarr and Imai,, 2021),
linear functions of which can approximate any smooth $f_{0}$. Approximate
balancing weights may thus avoid bias stemming from misspecifying $f_{0}$.
However, feasibility remains an issue — $\hat{\tau}_{\mathrm{wdim}}$ is still
susceptible to bias because these weights leave imbalance in $X_{i}$, with the
average imbalance across $X_{i}$ typically increasing as does its dimension.
Further, imbalance in certain elements of $X_{i}$ can be more damaging than in
others, depending on their influence on $Y_{i}$. These concerns require one to
proceed cautiously when exchanging $X_{i}$ with its high-dimensional
transformation or prioritizing balance in select dimensions of $X_{i}$, if at
all.

But while balance in $X_{i}$, or a transformation of it, may be sufficient for
an unbiased $\hat{\tau}_{\mathrm{wdim}}$, it is not required — in fact, all
that is required is exact balance in $f_{0}(X_{i})$, i.e., equality of the
means of $f_{0}(X_{i})$ in the treated and control groups after weighting.
From this perspective, seeking balance in $X_{i}$ is excessive. To illustrate,
were $f_{0}$ linear in $X_{i}$, balance in one linear function of $X_{i}$
(i.e., $f_{0}$) would be sufficient, but weights that balance $X_{i}$ also
balance all linear functions of it — an inefficient strategy at best, and can
hinder proper balance in $f_{0}(X_{i})$. This perspective motivates Targeted
Function Balancing (TFB), which more directly targets balance in
$f_{0}(X_{i})$ by estimating it with $\hat{f}_{0}(X_{i})$, and choosing
weights that balance functions near $\hat{f}_{0}$, in hopes that $f_{0}$ is
among them. In short, this yields approximate balance in $X_{i}$ and
$\hat{f}_{0}(X_{i})$, with the estimated variance of $\hat{f}_{0}$ determining
how much balance in $X_{i}$ is sufficient.

More specifically, TFB has several notable characteristics. First, it is an
approximate balancing method, so it may exchange $X_{i}$ with a high-
dimensional transformation of $X_{i}$. This includes basis functions from a
reproducing kernel Hilbert space, building on the developing literature of
weights that draw on them (e.g., Kallus,, 2020; Hazlett,, 2020; Wong and
Chan,, 2018; Hirshberg et al.,, 2021; Huang et al.,, 2006; Zhao,, 2019; Tarr
and Imai,, 2021), by estimating $f_{0}$ with kernel regularized least squares
(Hainmueller and Hazlett,, 2014). However, TFB offers a solution to
feasibility concerns by narrowing the function space that the weights seek
balance in, that is, from all linear functions of $X_{i}$, to those
probabilistically near $\hat{f}_{0}$ per its estimated variance. Second, and
relatedly, TFB primarily seeks balance in $\hat{f}_{0}(X_{i})$, but
additionally seeks balance in $X_{i}$ to safeguard against uncertainty in
$\hat{f}_{0}$, and to a degree deemed necessary by the estimated variance of
$\hat{f}_{0}$. Consequently, and in contrast to traditions that advocate for
good balance on every covariate (e.g., Stuart,, 2010), TFB may intentionally
leave imbalance in dimensions of $X_{i}$ because they have little to no
influence on $Y_{i}$, or to offset imbalances in other dimensions. This can
improve feasibility and yield efficiency gains without introducing bias, as is
demonstrated below. Third, the estimated variance of $\hat{f}_{0}$ determines
how to prioritize balance in each dimension of $X_{i}$, a feature that can be
exploited to great effect by estimating $f_{0}$ with sparsity (e.g., the
LASSO). Fourth, and finally, TFB’s weights are entirely defined by the choice
of regression estimator (i.e., $\hat{f}_{0}$) and an estimator for its
variance, thus turning the problem of how best to balance the covariates into
how best to model the outcome.

Combining a $\hat{f}_{0}$ and weights to estimate treatment effects is a well-
studied research area. It is essential to the augmented (weighted) estimator,
the typical form of doubly-robust estimators (e.g., Robins et al.,, 1994;
Robins and Rotnitzky,, 1995; van der Laan and Rubin,, 2006; Tan,, 2010;
Rotnitzky et al.,, 2012; Chernozhukov et al.,, 2018), which are consistent in
a propensity score weights setting when the investigator has correctly
specified $f_{0}$ or $\pi$. Augmented estimators have also been applied with
approximate balancing weights (e.g. Athey et al.,, 2018, Hirshberg and Wager,,
2020). In fact, TFB performs similarly to an augmented estimator in practice,
so this estimator is given particular attention below. Other methods that use
a $\hat{f}_{0}$ in conjunction with weights include those that allow a
$\hat{f}_{0}$ to inform which dimensions of $X_{i}$ to prioritize balance in,
or to use to estimate $\pi$ for propensity score weights (e.g., Shortreed and
Ertefaie,, 2017; Kuang et al.,, 2017; Ning et al.,, 2020), and methods that
expand on prognostic score theory from Hansen, (2008) (e.g., Leacy and
Stuart,, 2014; Antonelli et al.,, 2018), which supports seeking balance in
$\hat{f}_{0}(X_{i})$. TFB contributes to this literature by letting the
estimated variance of $\hat{f}_{0}$ determine how to prioritize balance in
$\hat{f}_{0}(X_{i})$ over $X_{i}$, and in certain dimensions of $X_{i}$ over
others. Additionally, this paper introduces Targeted Function Imbalance (TFI),
a balance diagnostic that TFB minimizes subject to constraints on the weights’
variance. As will be elaborated on, TFI may have applications to other methods
(e.g., Nikolaev et al.,, 2013; Hazlett,, 2020; Zhao,, 2019; Pirracchio and
Carone,, 2018; Imai and Ratkovic,, 2014), though these will not be fully
explored here.

To outline, Section 2 details notation and assumptions, and reviews
reproducing kernel Hilbert spaces, which appear throughout, and Section 3
expands on the motivation for TFB. Section 4 introduces TFB (and TFI),
demonstrates its merits through simulation, and details its uncertainty
estimates. Section 4 also illuminates the connection between TFB and augmented
estimation. Finally, Section 5 applies TFB to the National Supported Work
Demonstration (LaLonde,, 1986) data, and Section 6 concludes. Further, the
main text frames TFB as an estimator of the average treatment effect on the
treated, but Appendix A extends it to other estimands.

##  2 Background

###  2.1 Notation and assumptions

#### Notation

Let $i\in\\{1,\dots,n\\}$ index the units of observation and let $p(\cdot)$ be
the density function of an arbitrary random variable. Define
$D_{i}\in\\{0,1\\}$ to be an indicator variable for receiving the treatment,
with $n_{t}$ the number of treated units and $n_{c}=n-n_{t}$ the number of
control units. Additionally, let $D=[D_{1}\ \dots\ D_{n}]^{\top}$ be the
vector of treatment statuses for the sample, and let the first $n_{c}$ units
make up the control group (i.e., $1\leq i\leq n_{c}$ are the indices for the
control units and $n_{c}+1\leq i\leq n$ are the indices for the treated
units). Then, let $X_{i}$ be a $P$-dimensional vector of covariates and let
$X$ be the matrix of $X_{i}$ for the sample.

$\displaystyle X_{i}$ $\displaystyle=\begin{bmatrix}X_{i}^{(1)}\\\ \vdots\\\
X_{i}^{(P)}\end{bmatrix}\in\mathbbm{R}^{P}\ ,\
X=\begin{bmatrix}X^{\top}_{1}\\\ \vdots\\\
X_{n}^{\top}\end{bmatrix}\in\mathbbm{R}^{n\times P}$ (1)

In accordance with the potential outcomes framework (Splawa-Neyman et al.,,
1990; Rubin,, 1974), $Y_{i}(0)$ is the outcome if control and $Y_{i}(1)$ is
the outcome if treated, so $Y_{i}=Y_{i}(D_{i})$ is observed. Implicit here is
the stable unit treatment value assumption (SUTVA), that the potential
outcomes for unit $i$ are not functions of the treatment statuses of other
units, and that each treatment status is administered the same across the
units. Further, the tuples $(X_{i},D_{i},Y_{i}(0),Y_{i}(1))$ are assumed
independent and identically distributed (iid).

Of particular importance to TFB is the conditional expectation function of
$Y_{i}(d)$ given $X_{i}$, defined here as $f_{d}(X_{i})=\mathbbm{E}[Y_{i}(d)\
|\ X_{i}]$, where $\mathbbm{E}(\cdot)$ is the expectation over $p(\cdot)$
(i.e., the super-population). This allows the model:111Notice that
$\epsilon_{i}$ here is the same for $Y_{i}(0)$ and $Y_{i}(1)$. This is done to
avoid extra notation, and the results in this paper do not change if the
$\epsilon_{i}$ are allowed to vary by potential outcome.

$\displaystyle Y_{i}(d)=f_{d}(X_{i})+\epsilon_{i}\ \ \text{where}\ \
\mathbbm{E}(\epsilon_{i}\ |\ X_{i})=0\ \ \text{and}\ \
\operatorname{var}(\epsilon_{i}\ |\ X_{i})=\sigma^{2}_{i}$ (2)

where $\epsilon_{i}$ is an error term. TFB will apply the working assumption
that $f_{d}$ is linear in $X_{i}$,

$\displaystyle f_{d}(X_{i})=X_{i}^{\top}\beta_{d}\ \ \text{where}\ \
\beta_{d}=[\beta_{d}^{(1)}\ \cdots\ \beta_{d}^{(P)}]^{\top}\in\mathbbm{R}^{P}$
(3)

(3) appears restrictive because $X_{i}$ has been defined thus far as a vector
of untransformed covariates. However, exchanging $X_{i}$ with, or allowing it
to include, its nonlinear transformations (e.g., polynomial terms, or basis
functions) makes this representation of $f_{d}$ quite general, and is the
setting in which TFB largely operates.

#### Assumptions

This paper assumes throughout the absence of unobserved confounding in the
relationship between $D_{i}$ and $Y_{i}$, otherwise known as conditional
ignorability:

######  Assumption 1 (Conditional Ignorability)

$(Y_{i}(0),Y_{i}(1))\ \rotatebox[origin={c}]{90.0}{$\models$}\ D_{i}\ |\
X_{i}$

Note that the conditional independence in Assumption 1 extends to
$\epsilon_{i}$ and $D_{i}$ (i.e., $\epsilon_{i}\
\rotatebox[origin={c}]{90.0}{$\models$}\ D_{i}\ |\ X_{i}$), as (2) implies
that $\epsilon_{i}=Y_{i}(d)-f_{d}(X_{i})$.

#### Estimands and estimators

This paper primarily considers the average treatment effect on the treated
(ATT),

$\displaystyle\mathrm{ATT}=\mathbbm{E}[Y_{i}(1)-Y_{i}(0)\ |\ D_{i}=1]$ (4)

However, Appendix A adapts TFB to estimate the average treatment effect,
$\mathrm{ATE}=\mathbbm{E}[Y_{i}(1)-Y_{i}(0)]$, and the average treatment
effect on the controls, $\mathrm{ATC}=\mathbbm{E}[Y_{i}(1)-Y_{i}(0)\ |\
D_{i}=0]$.

Were it observed, the sample average treatment effect on the treated (SATT),

$\displaystyle\mathrm{SATT}$
$\displaystyle=\frac{1}{n_{t}}\sum_{i:D_{i}=1}Y_{i}(1)-\frac{1}{n_{t}}\sum_{i:D_{i}=1}Y_{i}(0)$
(5)

would be the ideal estimator for the ATT, but the $Y_{i}(0)$ are unobserved
for treated units without strong assumptions. The class of estimators studied
here thus replaces $\frac{1}{n_{t}}\sum_{i:D_{i}=1}Y_{i}(0)$ in SATT with a
weighted average of the $Y_{i}(0)$ for the control units,
$\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}Y_{i}(0)$ for nonnegative weights
$w_{i}$. The resulting weighted difference in means estimator has the form

$\displaystyle\hat{\tau}_{\mathrm{wdim}}=\frac{1}{n_{t}}\sum_{i:D_{i}=1}Y_{i}(1)-\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}Y_{i}(0)$
(6)

Section 3 discusses the conditions the $w_{i}$ must satisfy for
$\hat{\tau}_{\mathrm{wdim}}$ to be unbiased. Finally, let $w$ be an arbitrary
vector of nonnegative weights for control units,

$\displaystyle w=[w_{1}\ \cdots\ w_{n_{c}}]\in\mathbb{R}^{n_{c}}$ (7)

These weights will have mean 1 (i.e.,
$\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}=1$) in most settings considered here.
However, this condition will not be required, unless otherwise noted.

###  2.2 Background on reproducing kernel Hilbert spaces

The unbiasedness of many $\hat{\tau}_{\mathrm{wdim}}$ in the literature hinges
on the specification of $f_{0}$. Thus, some, including this paper, have turned
to reproducing kernel Hilbert spaces (RKHS) to characterize a large space of
functions from which $f_{0}$, or an arbitrarily close approximation, could
come. This section therefore provides a brief review of the relevant points in
RKHS.222For a more extensive review, see Wainwright, (2019).

Background on “positive semi-definite kernel functions” is first required. A
positive semi-definite kernel function,
$\mathcal{K}:\mathbb{R}^{P}\times\mathbb{R}^{P}\rightarrow\mathbb{R}$, is a
symmetric function such that for all possible choices of
$\\{x_{1},\dots,x_{m}\\}\subseteq\mathbb{R}^{P}$ for arbitrary $m$, the
$m\times m$ matrix with $ij$th element equal to $\mathcal{K}(x_{i},x_{j})$ is
positive semi-definite. Per Hazlett, (2020), it is helpful to interpret
$\mathcal{K}$ as a similarity score. For example, the $\mathcal{K}$ primarily
employed here is the Gaussian kernel,
$\mathcal{K}(x_{i},x_{j})=e^{-\frac{||x_{i}-x_{j}||_{2}^{2}}{b}}$ for some
bandwidth parameter $b$. This kernel’s range is 0 to 1, with
$\mathcal{K}(x_{i},x_{j})=1$ only when $x_{i}=x_{j}$ (i.e., when $x_{i}$ and
$x_{j}$ are as similar as can be), and $\mathcal{K}(x_{i},x_{j})$ approaches 0
as $||x_{i}-x_{j}||_{2}\rightarrow\infty$ (i.e., as $x_{i}$ and $x_{j}$
diverge).

A RKHS, $H(\mathcal{K})$, is then the unique Hilbert space of functions
$h:\mathbb{R}^{P}\rightarrow\mathbb{R}$ that contains $\mathcal{K}(\cdot,x)$
for $\forall x\in\mathbb{R}^{P}$, and has the inner product
$\langle\cdot,\cdot\rangle_{H(\mathcal{K})}:H(\mathcal{K})\times
H(\mathcal{K})\rightarrow\mathbb{R}$ satisfying the Reproducing Property:

$\displaystyle\langle\mathcal{K}(\cdot,x),h\rangle_{H(\mathcal{K})}=h(x)\ \ \
\text{for}\ \ \ \forall{h}\in H(\mathcal{K}),\ \forall x\in\mathbb{R}^{P}$
(Reproducing Property)

and norm $||h||_{H(\mathcal{K})}=\sqrt{\langle h,h\rangle_{H(\mathcal{K})}}$.
If $f_{0}\in H(\mathcal{K})$, then the Reproducing Property allows the
representation $f_{0}(X_{i})=K_{i}\alpha_{0}$ for
$K_{i}=[\mathcal{K}(X_{i},X_{1})\ \cdots\ \mathcal{K}(X_{i},X_{n})]$ and some
$\alpha_{0}\in\mathbb{R}^{n}$ (see Wainwright,, 2019 for proof). Thus, within
the sample, any function in $H(\mathcal{K})$ can be expressed as a linear
function of $K_{i}$. This representation can be quite powerful, depending on
the choice of $\mathcal{K}$. For example, for some choices (e.g., the Gaussian
kernel for any $b$), a function in $H(\mathcal{K})$ can approximate any smooth
function to an arbitrarily close degree.333$H(\mathcal{K})$ can be represented
in several ways. The first is the closure of the function space
$\\{\sum_{j=1}^{m}\alpha^{(j)}\mathcal{K}(x_{j},\cdot)\ |\ \forall
m\in\mathbb{Z}^{\geq 0},\forall
x_{j}\in\mathbb{R}^{P},\forall\alpha^{(j)}\in\mathbb{R}\\}$, i.e., arbitrary
linear combinations of $\mathcal{K}(x_{j},\cdot)$ for arbitrarily many
$x_{j}$. This representation illuminates why $f_{0}(X_{i})=K_{i}\alpha_{0}$,
but does not explain why, for some $\mathcal{K}$, functions in
$H(\mathcal{K})$ can approximate any smooth function. This is more easily
understood from the alternate representation of
$H(\mathcal{K})=\\{\sum_{\ell=1}^{\infty}\beta^{(\ell)}\phi_{\mathcal{K}}^{(\ell)}\
|\ \sum_{\ell=1}^{\infty}(\beta^{(\ell)})^{2}<\infty\ \text{and}\
\sum_{\ell=1}^{\infty}(\beta^{(\ell)})^{2}/\mu_{\mathcal{K}}^{(\ell)}<\infty\\}$
where $\phi^{(\ell)}_{\mathcal{K}}$ and $\mu^{(\ell)}_{\mathcal{K}}$ are
eigenfunctions and eigenvalues, respectively, associated with $\mathcal{K}$,
i.e., $H(\mathcal{K})$ contains linear functions of (potentially) infinite
dimensional $\phi_{\mathcal{K}}=[\phi^{(1)}_{\mathcal{K}}\
\phi^{(2)}_{\mathcal{K}}\ \dots]^{\top}$. If $\phi_{\mathcal{K}}$ is flexible,
then it is plausible that linear functions of it could approximate any smooth
function, just like a high degree polynomial might.

Finally, extra notation is needed. Let $K$ be the $n\times n$ matrix with
$ij$th element $\mathcal{K}(X_{i},X_{j})$, otherwise known as the “gram”
matrix for the full sample, and let $K_{i}$ be the $i$th row of $K$.

$\displaystyle
K=\begin{bmatrix}\mathcal{K}(X_{1},X_{1})&\dots&\mathcal{K}(X_{1},X_{n})\\\
\vdots&\ddots&\vdots\\\
\mathcal{K}(X_{n},X_{1})&\dots&\mathcal{K}(X_{n},X_{n})\end{bmatrix}=\begin{bmatrix}K_{1}\\\
\vdots\\\ K_{n}\end{bmatrix}\in\mathbb{R}^{n\times n}$ (8)

From a similarity score perspective, $K_{i}$ holds unit $i$’s similarity
scores to each unit in the data. Then, letting $K_{c}$ be the first $n_{c}$
columns of $K$ with rows $K_{ic}=[\mathcal{K}(X_{i},X_{1})\ \cdots\
\mathcal{K}(X_{i},X_{n_{c}})]$, the $K_{ic}$ may be thought of as holding unit
$i$’s similarity scores to each control unit in the data. Accordingly, let
$K_{t}$ be the last $n_{t}$ columns of $K$ with rows
$K_{it}=[\mathcal{K}(X_{i},X_{n_{c}+1})\ \cdots\ \mathcal{K}(X_{i},X_{n})]$.

$\displaystyle K=\begin{bmatrix}K_{c}&K_{t}\end{bmatrix}\ ,\
K_{c}=\begin{bmatrix}K_{1c}\\\ \vdots\\\
K_{nc}\end{bmatrix}\in\mathbb{R}^{n\times n_{c}}\ ,\
K_{t}=\begin{bmatrix}K_{1t}\\\ \vdots\\\
K_{nt}\end{bmatrix}\in\mathbb{R}^{n\times n_{t}}$ (9)

Finally, let $K_{cc}$ be the first $n_{c}$ rows of $K_{c}$ and let $K_{tc}$
the last $n_{t}$ rows of $K_{c}$. $K_{cc}$ is also the gram matrix for the
control group.

$\displaystyle K_{c}=\begin{bmatrix}K_{cc}\in\mathbb{R}^{n_{c}\times n_{c}}\\\
K_{tc}\in\mathbb{R}^{n_{t}\times n_{c}}\end{bmatrix}$ (10)

##  3 Motivation for TFB

This section motivates TFB by reframing existing approaches in the weighting
literature as decisions on a set of potential $f_{0}$ to seek balance in. TFB
builds on this idea by letting a $\hat{f}_{0}$, an estimate of $f_{0}$, and
its estimated variance inform this decision. Section 3.1 first reviews the
role that $f_{0}$ plays in the conditions the weights must satisfy for
$\hat{\tau}_{\mathrm{wdim}}$ to be unbiased.

###  3.1 Desirable properties of the weights

Letting $w_{i}=w(X_{i},D_{i})$ for a weight function $w(\cdot)$,

$\displaystyle\hat{\tau}_{\mathrm{wdim}}\overset{p}{\rightarrow}\mathbbm{E}[Y_{i}(1)\
|\ D_{i}=1]-\mathbbm{E}[w_{i}Y_{i}(0)\ |\ D_{i}=0]$ (11)

Therefore, obtaining a consistent estimate of the ATT using
$\hat{\tau}_{\mathrm{wdim}}$ requires that

$\displaystyle\mathbbm{E}[w_{i}Y_{i}(0)\ |\ D_{i}=0]=\mathbbm{E}[Y_{i}(0)\ |\
D_{i}=1]$ (12)

Note that the expectation on the left-hand side of (12) is over
$p(X_{i},Y_{i}(0)|D_{i}=0)$, and the expectation on the right-hand side is
over $p(Y_{i}(0)|D_{i}=1)$, the empirical analog of which is unobserved.
Deriving desirable conditions for the weights involves rewriting (12) in terms
of the observed data.

Applying the law of iterative expectations, (12) becomes

$\displaystyle\mathbbm{E}\biggr{(}w_{i}\mathbbm{E}[Y_{i}(0)\ |\
X_{i},D_{i}=0]\ \biggr{|}\
D_{i}=0\biggr{)}=\mathbbm{E}\biggr{(}\mathbbm{E}[Y_{i}(0)\ |\ X_{i},D_{i}=1]\
\biggr{|}\ D_{i}=1\biggr{)}$ (13)

Assumption 1 then allows one to remove the conditioning on $D_{i}$ in the
inner expectations of both sides of (13), giving

$\displaystyle\mathbbm{E}\biggr{(}w_{i}\mathbbm{E}[Y_{i}(0)\ |\ X_{i}]\
\biggr{|}\ D_{i}=0\biggr{)}=\mathbbm{E}\biggr{(}\mathbbm{E}[Y_{i}(0)\ |\
X_{i}]\ \biggr{|}\ D_{i}=1\biggr{)}$ (14)

Note that the outer expectations in (14) are now over $p(X_{i}|D_{i}=0)$ on
the left-hand-side, and $p(X_{i}|D_{i}=1)$ on the right-hand side, the
empirical analogs of which are both observed. This is seen more clearly by
applying the definition of $f_{0}$ to (14), which also yields a desired
condition for the weights:

$\displaystyle\mathbbm{E}[w_{i}f_{0}(X_{i})\ |\
D_{i}=0]=\mathbbm{E}[f_{0}(X_{i})\ |\ D_{i}=1]$ (WC)

which will be referred to as the “weighting condition” (WC). Empirically, WC
becomes

$\displaystyle\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}f_{0}(X_{i})=\frac{1}{n_{t}}\sum_{i:D_{i}=1}f_{0}(X_{i})$
(EWC)

which will be referred to as the “empirical weighting condition” (EWC). In
words, WC and EWC require that the means of $f_{0}(X_{i})$ be equated in
expectation or empirically.444Note also that neither WC nor EWC requires
complete overlap in the conditional distributions of $X_{i}$, or
$\displaystyle 0<p(D_{i}=1|X_{i}=x)<1\ \ \text{for}\ \ \forall x$ which is
traditionally assumed in settings with propensity score weights (Rosenbaum and
Rubin,, 1983). I therefore forgo this assumption throughout.

EWC is given particular focus below, and is thus worthwhile to discuss
further. First, define the imbalance, or difference in the means, of a
function of $X_{i}$ as

$\displaystyle\mathrm{imbal}(w,g(X),D)=\frac{1}{n_{t}}\sum_{i:D_{i}=1}g(X_{i})-\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}g(X_{i})$
(15)

where $g$ is an arbitrary vector-valued function. How far the weights are from
satisfying EWC, to be referred to as the “EWC Bias”, then has an intuitive
form as the imbalance in $f_{0}(X_{i})$:

$\displaystyle\mathrm{EWC\ Bias}=\mathrm{imbal}(w,f_{0}(X),D)$ (16)

Furthermore, the expectation of the EWC Bias is exactly the bias of
$\hat{\tau}_{\mathrm{wdim}}$. Decomposing the difference between
$\hat{\tau}_{\mathrm{wdim}}$ and SATT reveals this:

$\displaystyle\hat{\tau}_{\mathrm{wdim}}-\mathrm{SATT}=\mathrm{EWC\ Bias}\ +\
\biggr{(}\frac{1}{n_{t}}\sum_{i:D_{i}=1}\epsilon_{i}-\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}\epsilon_{i}\biggr{)}$
(17)

First, SATT is unbiased for the ATT. Then, assuming that each pair of $w_{i}$
and $Y_{i}$ (and thus $\epsilon_{i}$) are independent given $X$ and $D$,
taking the expectation of the both sides of (17) proves the result, as the
terms with $\epsilon_{i}$ are then mean-zero under Assumption 1.555Many
methods in the literature (e.g., Hainmueller,, 2012; Zubizarreta,, 2015;
Hazlett,, 2020) choose weights that are “honest”, or entirely defined by $X$
and $D$, which leaves all of the $w_{i}$ independent of all of the $Y_{i}$
given $X$ and $D$. As will be discussed in later sections, TFB’s weights
depend on the $Y_{i}$, but in such a way that each pair of $w_{i}$ and $Y_{i}$
are independent given $X$ and $D$.  Putting these pieces together, the bias of
$\hat{\tau}_{\mathrm{wdim}}$ is driven by the remaining imbalance in
$f_{0}(X_{i})$ after weighting.

###  3.2 Building blocks

This section reviews several weighting procedures in the literature and
categorizes them into three families: propensity score, exact balancing, and
approximate balancing weights, the last of which includes TFB. Within each
family, particularly focus is given to the tension between the assumptions on
$f_{0}$ required for unbiasedness and the resulting estimators’ feasibility.

#### Propensity score weights

WC is satisfied by

$\displaystyle w_{i}=\frac{p(X_{i}\ |\ D_{i}=1)}{p(X_{i}\ |\
D_{i}=0)}=\biggr{[}\frac{\pi(X_{i})}{1-\pi(X_{i})}\biggr{]}\biggr{[}\frac{p(D_{i}=0)}{p(D_{i}=1)}\biggr{]}$
(18)

where $\pi(X_{i})=p(D_{i}=1\ |\ X_{i})$ is the propensity score. This offers
motivation for propensity score weights (Rosenbaum and Rubin,, 1983) — were
$\hat{\pi}(X_{i})$ to consistently estimate $\pi(X_{i})$, then
$w_{i}=\frac{n_{c}}{n_{t}}\frac{\hat{\pi}(X_{i})}{1-\hat{\pi}(X_{i})}$ yields
a consistent $\hat{\tau}_{\mathrm{wdim}}$, where $\frac{n_{c}}{n}$ and
$\frac{n_{t}}{n}$ estimate $p(D_{i}=0)$ and $p(D_{i}=1)$, respectively.666A
$\hat{\tau}_{\mathrm{wdim}}$ with
$w_{i}=n_{c}(\sum_{i:D_{i}=0}\frac{\hat{\pi}(X_{i})}{1-\hat{\pi}(X_{i})})^{-1}\frac{\hat{\pi}(X_{i})}{1-\hat{\pi}(X_{i})}$
is also consistent, allowing $\sum_{i:D_{i}=0}w_{i}=n_{c}$. Of note is that
propensity score weights satisfy WC for any $f_{0}$. However, this robustness
comes at the price of a specification of $\pi$. An incorrect specification may
lead to biased $\hat{\tau}_{\mathrm{wdim}}$ (e.g., Kang and Schafer,,
2007).777In hopes of avoiding this pitfall, some have turned to machine
learning (e.g., tree-based methods, neural networks, and ensemble methods) to
estimate the propensity score in matching and weighting settings (e.g.,
Setoguchi et al.,, 2008; Lee et al.,, 2010; Pirracchio et al.,, 2014;
Pirracchio and Carone,, 2018).

#### Exact balancing weights

Satisfying WC for any $f_{0}$ is unnecessary if one has prior knowledge about
$f_{0}$. For example, if $f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$ then weights
that achieve exact balance in $X_{i}$,

$\displaystyle\mathrm{imbal}(w,X,D)=0$ (19)

satisfy EWC. The problem then becomes how to choose such weights when several
exist. A common approach, originating from calibration in the survey
literature (e.g., Deming and Stephan,, 1940; Deville and Särndal,, 1992), is
to minimize a function of the weights, $B(w)$, that controls their variance or
distance from base weights, subject to the exact balance conditions in (19),
i.e.,

$\displaystyle\underset{w}{\text{argmin}}\ B(w)\ \ \text{where}\ \
\mathrm{imbal}(w,X,D)=0$ (BAL)

where the constraints $\sum_{i:D_{i}=0}w_{i}=n_{c}$ and $w_{i}\geq 0$ are
typically added. Examples of BAL include Hainmueller, (2012) and Chan et al.,
(2016), the former of which chooses $B(w)$ to be the (negative) entropy,
$\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}\mathrm{log}(w_{i})$, calling the
procedure Entropy Balancing.888Also of note is the Covariate Balancing
Propensity Score (Imai and Ratkovic,, 2014), which seeks propensity score
weights that satisfy the balance conditions in (19). In fact, Zhao and
Percival, (2017) prove an equivalence between Entropy Balancing and the
Covariate Balancing Propensity Score — Entropy Balancing weights are
propensity score weights with $\pi$ estimated by logistic regression and
parameters that minimize a loss function different from the traditional
negative log-likelihood from maximum likelihood estimation.

Exact balancing weights exhibit two shortcomings, however — one of
specification and one of feasibility. On specification, the resulting
$\hat{\tau}_{\mathrm{wdim}}$ is only assured to be unbiased when
$f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$. On feasibility, weights that achieve the
exact balance in (19) may not exist (e.g., when $X_{i}$ is of large dimension)
or may be extreme, resulting in a high variance $\hat{\tau}_{\mathrm{wdim}}$.
Further, these issues compound each other. One may expand $X_{i}$ to include
its nonlinear transformations, finding exact balance has become infeasible.
This is the precisely the setting that TFB explores.

#### Approximate balancing weights

Concerns with exact balancing weights have motivated approximate balancing
weights. One subfamily, found in Zubizarreta, (2015) and generalized by Wang
and Zubizarreta, (2020), relaxes the balance conditions in BAL:

$\displaystyle\underset{w}{\text{argmin}}\ B(w)\ \ \text{where}\ \
|\mathrm{imbal}(w,X,D)|\leq\delta$ (ABAL 1)

for some $\delta=[\delta^{(1)}\ \dots\ \delta^{(P)}]\geq 0$, and the
constraints $\sum_{i:D_{i}=0}w_{i}=n_{c}$ and $w_{i}\geq 0$ are usually
added.999Wang and Zubizarreta, (2020) refer to ABAL 1 as “minimal dispersion
approximately balancing weights” (or “minimal weights”) and, building off
Zhao, (2019), prove the resulting $w_{i}$ are, for certain $B(w)$, equivalent
to propensity score weights where $\pi$ has been estimated with $L_{1}$
shrinkage and the appropriate link and loss functions. This connection has
also been pointed out by Chattopadhyay et al., (2020).  If
$f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$, then $\delta$ presents a bias-variance
tradeoff: smaller $\delta^{(\ell)}$ reduce the bias of the resulting
$\hat{\tau}_{\mathrm{wdim}}$, but also shrink the space of solutions for the
weights, which may increase the variance of $\hat{\tau}_{\mathrm{wdim}}$. The
choice of $\delta$ is thus paramount to the method’s performance.

Additionally, if $f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$, weights found by ABAL 1
bound the EWC Bias for the “worst-case” $\beta_{0}$ , or the $\beta_{0}$ that
yields the most bias, as

$\displaystyle|\text{EWC Bias}|$
$\displaystyle=|\mathrm{imbal}(w,X,D)^{\top}\beta_{0}|\leq||\beta_{0}||_{2}||\mathrm{imbal}(w,X,D)||_{2}\leq||\beta_{0}||_{2}||\delta||_{2}$
(20)

where the first inequality in (20) becomes an equality when $\beta_{0}$ is in
the direction of the leftover imbalance. Thus, $X_{i}$ and $\delta$ can be
thought of as defining a space of functions from which the worst-case $f_{0}$
may come (i.e., $\mathcal{F}=\\{x^{\top}\beta\ |\
|\beta^{(\ell)}|\leq\delta^{(\ell)}\frac{||\beta_{0}||_{2}}{||\delta||_{2}}\\}$).
Another subfamily of procedures incorporates this idea directly, finding low
variance weights that minimize the EWC Bias for the worst-case $f_{0}$ over
some space of functions, $\mathcal{F}$,

$\displaystyle\underset{w}{\text{argmin}}\biggr{[}\underset{f_{0}\in\mathcal{F}}{\text{max}}(\text{EWC
Bias})^{2}+B(w)\biggr{]}$ (ABAL 2)

where the constraints $\sum_{i:D_{i}=0}w_{i}=n_{c}$ and $w_{i}\geq 0$ are
usually added.101010Hirshberg and Wager, (2020) and Hirshberg et al., (2021)
refer to ABAL 2 as “minimax” weights. According to Wang and Zubizarreta,
(2020), various weights that can be framed as ABAL 2 (e.g., Kallus,, 2020;
Hirshberg and Wager,, 2020; Wong and Chan,, 2018) approximate ABAL 1 by using
a Lagrange relaxation of the balance constraints. Analogous to $X_{i}$ and
$\delta$ in ABAL 1, the parametrization of $\mathcal{F}$ determines the
complexity of $f_{0}$ that ABAL 2 aims to balance, and the size of
$\mathcal{F}$ determines how to prioritize minimizing the squared EWC Bias
relative to $B(w)$.111111For example, if $\mathcal{F}=\\{x^{\top}\beta\ |\
||\beta||_{2}\leq C\\}$, the objective function in ABAL 2 becomes
$\displaystyle C^{2}||\mathrm{imbal}(w,X,D)||_{2}^{2}+B(w)$ The resulting
procedure attempts to balance the worst-case linear function of $X_{i}$, which
is determined by overall balance in $X_{i}$. $C$ determines how much balance
in $X_{i}$ is prioritized over $B(w)$.

By obviating the requirement of exact balance, approximate balancing weights
allow a high-dimensional $X_{i}$ or, analogously in ABAL 2, a flexible
$\mathcal{F}$. These methods can thus be made to appear robust to a wide range
of $f_{0}$. For example, Kallus, (2020), Hazlett, (2020), Wong and Chan,
(2018), Hirshberg et al., (2021), Zhao, (2019), and Tarr and Imai, (2021) may
assume $f_{0}$ to be of a RKHS, applying the representation
$f_{0}(X_{i})=K_{i}\alpha_{0}$, and approximately balance $K_{i}$. The weights
in Kallus, (2020), Wong and Chan, (2018), Hirshberg et al., (2021), and Tarr
and Imai, (2021) can be framed as ABAL 2, where $\mathcal{F}$ is chosen as a
subspace of a RKHS. Hazlett, (2020) exactly balances (via Entropy Balancing)
an $r$-dimensional approximation (i.e., using the first $r$
eigenvectors/eigenvalues) of the gram matrix, $K$, choosing the $r$ that
minimizes the worst-case EWC Bias.

However, the optimal ways to choose $\delta$ in ABAL 1 or $\mathcal{F}$ in
ABAL 2 are unsolved problems, despite data-driven proposals available (see
Wang and Zubizarreta,, 2020 and Chattopadhyay et al.,, 2020 for options in
ABAL 1; see Kallus,, 2020 and Wong and Chan,, 2018 for options in ABAL 2).
Furthermore, ABAL 2 estimators that use a data-driven $\mathcal{F}$ often
center it at the zero function, $f(x)=0$. Kallus, (2020), for example, uses a
ball in a RKHS: $\mathcal{F}=\\{h\in H(\mathcal{K})\ |\
||h||_{H(\mathcal{K})}\leq C\\}$ for a data-driven $C$.121212Kallus, (2020)
gives strong justification for this choice, proving that the choices of
$\mathcal{F}$ and $B$ yield an objective function for the weights that is
equal (up to a constant shift) to the conditional mean squared error (given
$X$ and $D$) of $\hat{\tau}_{\mathrm{wdim}}$ as an estimator of SATT under the
Bayesian model that $f_{0}$ follows a Gaussian process.  The worst-case
function in $\mathcal{F}$ may then be in the opposite direction of, or
orthogonal to, $f_{0}$ in $H(\mathcal{K})$ (i.e.,
$\langle\cdot,f_{0}\rangle_{H(\mathcal{K})}=0$).131313An orthogonal function
when $f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$ is perhaps illustrative. If
$X_{i}=[X_{i}^{(1)}\ X_{i}^{(2)}]^{\top}$ and $\beta_{0}=[1\ 0]^{\top}$, a
function orthogonal to $f_{0}$ would be
$f_{\perp}(X_{i})=X_{i}^{\top}\beta_{\perp}$ where $\beta_{\perp}=[0\
1]^{\top}$. Therefore, while only $X_{i}^{(1)}$ has any marginal effect on
$Y_{i}(0)$, balancing the worst-case linear function of $X_{i}$ would involve
balancing functions that disregard any marginal effect of $X_{i}^{(1)}$ on
$Y_{i}(0)$. Balance on these orthogonal functions would not grant further bias
reduction, were balance in $f_{0}(X_{i})$ to remain constant. However,
requiring good balance on these functions would, at least, increase the
variance of the resulting estimator, and could, at worst, distract the weights
from obtaining good balance on $f_{0}(X_{i})$. Further, as $\mathcal{F}$ is
allowed to be larger and more flexible, this inefficiency could be
exacerbated. This concern provides the primary motivation for TFB, which can
be framed as ABAL 2 and seeks a $\mathcal{F}$ that contains only functions
close to $f_{0}$ with a user-chosen probability (e.g., 0.95).

##  4 Framework for TFB

###  4.1 A new balance diagnostic

This section derives Targeted Function Imbalance (TFI), the balance diagnostic
that TFB minimizes over the $w_{i}$, and approximates the EWC Bias using a
$\hat{f}_{0}$ and its estimated variance.

If $f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$, the magnitude of the EWC Bias
simplifies to

$\displaystyle|\text{EWC Bias}|=|\mathrm{imbal}(w,X,D)^{\top}\beta_{0}|$ (21)

Then, letting $\hat{\beta}_{0}$ be an arbitrary estimate of $\beta_{0}$, (21)
can be rewritten as

$\displaystyle|\mathrm{imbal}(w,X,D)^{\top}(\beta_{0}-\hat{\beta}_{0})+\mathrm{imbal}(w,X,D)^{\top}\hat{\beta}_{0}|$
(22)

Rewriting $|$EWC Bias$|$ as (22) may appear unhelpful, as $\beta_{0}$ is still
unknown, but it reveals a useful approximation — consider (22) for the worst-
case $\beta_{0}$ close to $\hat{\beta}_{0}$ with some high level of
probability (e.g., 0.95):

$\displaystyle\underset{\beta_{0}\in(\hat{\beta}_{0}-S_{q})}{\text{max}}|\mathrm{imbal}(w,X,D)^{\top}(\beta_{0}-\hat{\beta}_{0})+\mathrm{imbal}(w,X,D)^{\top}\hat{\beta}_{0}|$
(23)

where $S_{q}\subseteq\mathbb{R}^{P}$ is a set that contains 0 and
$p(\hat{\beta}_{0}-\beta_{0}\in S_{q})\approx q$ for some $q$ ($q$ will be
left arbitrary here, but Section 4.4 studies it more closely). In other words,
$S_{q}$ is set of small deviations such that, with probability approximately
$q$, $\beta_{0}$ can be recovered by subtracting one of these small deviations
from $\hat{\beta}_{0}$. Clearly, $S_{q}$ will be defined by the distribution
of $\hat{\beta}_{0}$.

Proceeding further thus requires a choice of $\hat{\beta}_{0}$. For
simplicity, assume that $\hat{\beta}_{0}$ is the OLS estimate using only the
control units. $\hat{\beta}_{0}$ then has an easily computed estimated
variance in the sandwich variance estimator (White,, 1980),

$\displaystyle\hat{V}_{\beta_{0}}$
$\displaystyle=\frac{1}{n_{c}}\biggr{(}\frac{1}{n_{c}}\sum_{i:D_{i}=0}X_{i}X_{i}^{\top}\biggr{)}^{-1}\biggr{(}\frac{1}{n_{c}}\sum_{i:D_{i}=0}(Y_{i}-X_{i}^{\top}\hat{\beta}_{0})^{2}X_{i}X_{i}^{\top}\biggr{)}\biggr{(}\frac{1}{n_{c}}\sum_{i:D_{i}=0}X_{i}X_{i}^{\top}\biggr{)}^{-1}$
(24)

$\hat{V}_{\beta_{0}}^{-\frac{1}{2}}(\hat{\beta}_{0}-\beta_{0})\overset{d}{\rightarrow}\mathcal{N}(0,I_{P})$
under iid data. Thus, a logical $S_{q}$ is one that, after being transformed
by $\hat{V}_{\beta_{0}}^{-\frac{1}{2}}$, becomes a ball centered at the origin
in which a $\mathcal{N}(0,I_{P})$ falls with probability $q$. This yields
$S_{q}=\\{\hat{V}_{\beta_{0}}^{\frac{1}{2}}x\in\mathbb{R}^{P}\ |\
||x||_{2}\leq\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\\}$, where
$Q_{q}(\mathcal{X}^{2}_{P})$ is the $q$th quantile of a chi-squared random
variable with $P$ degrees of freedom, simplifying (23) to an example of TFI:

$\displaystyle\mathrm{TFI}(w,X,D,\hat{\beta}_{0},\hat{V}_{\beta_{0}},q)=\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\times||\hat{V}_{\beta_{0}}^{\frac{1}{2}}\mathrm{imbal}(w,X,D)||_{2}+|\mathrm{imbal}(w,X,D)^{\top}\hat{\beta}_{0}|$
(25)

TFI easily generalizes to non-OLS regressions by replacing $\hat{\beta}_{0}$
with the new estimate, and $\hat{V}_{\beta_{0}}$ with a suitable variance
estimator. To preview, I primarily consider two such forms of TFI. The first
form estimates $f_{0}$ with kernel regularized least squares, setting
$X_{i}=K_{ic}^{\top}$ and $\hat{\beta}_{0}=\hat{\alpha}_{0}$ where

$\displaystyle\hat{\alpha}_{0}=\underset{\alpha}{\text{argmin}}\biggr{(}\sum_{i:D_{i}=0}(Y_{i}-K_{ic}\alpha)^{2}+\lambda\alpha^{\top}K_{cc}\alpha\biggr{)}$
(26)

with $\lambda$ chosen by cross-validation. Hainmueller and Hazlett, (2014)
prove a suitable $\hat{V}_{\alpha_{0}}$ is141414Hainmueller and Hazlett,
(2014) show this under the additional assumption that $\epsilon_{i}$ have
constant variance given $X_{i}$. While this is unlikely, TFB does not require
$\hat{V}_{\alpha_{0}}$ for standard errors, but uses it as a rough estimate of
the variation in $\hat{\alpha}_{0}$. Therefore, I suspect that a violation to
this assumption is not of major consequence.

$\displaystyle\hat{V}_{\alpha_{0}}=\frac{1}{n_{c}}\sum_{i:D_{i}=0}(Y_{i}-K_{ic}\hat{\alpha}_{0})^{2}\times(K_{cc}+\lambda
I_{n_{c}})^{-2}$ (27)

The second form I focus on estimates $f_{0}$ with LASSO regression, and
obtains $\hat{V}_{\beta_{0}}$ by residual bootstrapping.151515Knight and Fu,
(2000) and Chatterjee and Lahiri, (2010) prove the residual bootstrap is
inconsistent under mild conditions for the LASSO’s asymptotic variance when
one of more components of $\beta_{0}$ are zero, while Chatterjee and Lahiri,
(2011) show that the residual bootstrap with a modified LASSO can be
consistent. However, because TFB only uses $\hat{V}_{\beta_{0}}$ as a rough
estimate of variation in $\hat{\beta}_{0}$ and not for confidence intervals,
and any inconsistency in a residual bootstrap does not keep TFB from
performing well in later demonstrations, I forgo this modified LASSO.

###  4.2 Choosing weights in TFB

This section introduces TFB, which finds low variance weights that minimize
TFI. TFB, like Kallus, (2020), considers minimizing the conditional mean
squared error of $\hat{\tau}_{\mathrm{wdim}}$ as an estimate of the SATT,

$\displaystyle\mathbbm{E}\biggr{(}(\hat{\tau}_{\mathrm{wdim}}-\mathrm{SATT})^{2}\
\biggr{|}\ X,D\biggr{)}=(\text{EWC
Bias})^{2}+\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma^{2}_{i}+\text{Constant}$
(28)

where the constant does not depend on the weights. Define TFB then as

$\displaystyle\underset{w}{\text{argmin}}\biggr{[}[\mathrm{TFI}(w,X,D,\hat{\beta}_{0},\hat{V}_{\beta_{0}},q)]^{2}+\frac{\hat{\sigma}^{2}}{n_{c}^{2}}||w||_{2}^{2}\biggr{]}\
\text{where}\ w_{i}\geq 0\ \text{and}\ \sum_{i:D_{i}=0}w_{i}=n_{c}$ (TFB)

where the objective function in the above approximates the EWC Bias in (28)
with TFI, and substitutes $\sigma^{2}_{i}$ with
$\hat{\sigma}^{2}=\frac{1}{n_{c}}\sum_{i:D_{i}=0}(Y_{i}-X_{i}^{\top}\hat{\beta}_{0})^{2}$.

Dissecting TFB’s objective function when $f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$
is illustrative for envisioning its strengths. The
$|\mathrm{imbal}(w,X,D)^{\top}\hat{\beta}_{0}|$ term penalizes imbalance in
$\hat{f}_{0}(X_{i})=X_{i}^{\top}\hat{\beta}_{0}$, which equals the EWC Bias if
$\hat{\beta}_{0}=\beta_{0}$. However, $\hat{\beta}_{0}=\beta_{0}$ is unlikely,
so the
$\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\times||\hat{V}_{\beta_{0}}^{\frac{1}{2}}\mathrm{imbal}(w,X,D)||_{2}$
term penalizes imbalance in $X_{i}$, with the variance of each
$\hat{\beta}_{0}^{(\ell)}$ determining how to prioritize balance in the
$X_{i}^{(\ell)}$.161616Note that the coefficients’ covariance also plays a
role, adding penalties and rewards for the signs of the leftover imbalances.
To illustrate, let $X_{i}$ be two-dimensional. Then,
$\displaystyle||\hat{V}_{\beta_{0}}^{\frac{1}{2}}\mathrm{imbal}(w,X,D)||_{2}$
$\displaystyle=\biggr{(}\widehat{\operatorname{var}}(\hat{\beta}_{0}^{(1)})[\mathrm{imbal}(w,X^{(1)},D)]^{2}+\widehat{\operatorname{var}}(\hat{\beta}_{0}^{(2)})[\mathrm{imbal}(w,X^{(2)},D)]^{2}$
$\displaystyle+2\widehat{\operatorname{cov}}(\hat{\beta}_{0}^{(1)},\hat{\beta}_{0}^{(2)})\mathrm{imbal}(w,X^{(1)},D)\mathrm{imbal}(w,X^{(2)},D)\biggr{)}^{1/2}$
Therefore, if
$\widehat{\operatorname{cov}}(\hat{\beta}_{0}^{(1)},\hat{\beta}_{0}^{(2)})>0$,
then the imbalances in the dimensions of $X_{i}$ are penalized more if they
are of the same sign, and less if they are of the opposite sign. If
$\widehat{\operatorname{cov}}(\hat{\beta}_{0}^{(1)},\hat{\beta}_{0}^{(2)})<0$,
then imbalances are penalized more if they are of the opposite sign, and less
if they are of the same sign. This yields two desirable properties. First, TFB
balances $X_{i}$ to safeguard against uncertainty in $\hat{\beta}_{0}$ and
only to a degree deemed necessary by its estimated distribution. Section 4.3
also demonstrates that this, while simultaneously seeking balance in
$\hat{f}_{0}(X_{i})$, can yield intentional imbalance in certain dimensions of
$X_{i}$ that offsets imbalance in others, granting efficiency gains without
inviting bias.171717To illustrate, let
$f_{0}(X_{i})=5X_{i}^{(1)}+X_{i}^{(2)}$. While the way to balance
$f_{0}(X_{i})$ that yields the most efficient estimator may be to obtain
$\mathrm{imbal}(w,X,D)=0$, there also might be minimal overlap in
$X_{i}^{(2)}$. In this case, exact balance in $X_{i}^{(2)}$ would require high
variance weights. Meanwhile, it may require lower variance weights to obtain
$\mathrm{imbal}(w,X^{(1)},D)=-0.2$ and $\mathrm{imbal}(w,X^{(2)},D)=1$, which
also achieves balance in $f_{0}(X_{i})$:
$\displaystyle\mathrm{imbal}(w,f_{0}(X),D)=5\times\mathrm{imbal}(w,X^{(1)},D)+\mathrm{imbal}(w,X^{(2)},D)=5(-0.2)+1=0$
Second, the variance of each $\hat{\beta}_{0}^{(\ell)}$ determines a hierarchy
in balancing the $X_{i}^{(\ell)}$, meaning TFB incorporates variable
selection. Section 4.3 shows how estimating $\beta_{0}$ with sparsity (e.g.,
the LASSO) can exploit this. Finally,
$\frac{\hat{\sigma}^{2}}{n_{c}^{2}}||w||_{2}^{2}$ controls the resulting
estimator’s variance.

These qualities are not all unique to TFB. Prognostic score theory, developed
by Hansen, (2008), also supports seeking balance in $\hat{f}_{0}(X_{i})$,
inspiring some to incorporate $\hat{f}_{0}$ in propensity score matching
schemes (e.g., Leacy and Stuart,, 2014; Antonelli et al.,, 2018). Both Kuang
et al., (2017) and Ning et al., (2020) emphasize balance in
$\hat{f}_{0}(X_{i})$, and, along with Shortreed and Ertefaie, (2017), develop
methods that use $\hat{f}_{0}$ to inform variable selection, either in which
dimensions of $X_{i}$ to prioritize balance in, or which dimensions to use to
estimate $\pi$ for propensity score weights.181818Kuang et al., (2017), Ning
et al., (2020), and Shortreed and Ertefaie, (2017) are consistent with
longstanding research (e.g., Brookhart et al.,, 2006; Patrick et al.,, 2011;
De Luna et al.,, 2011) that suggests disregarding the dimensions of $X_{i}$
that are unassociated with $Y_{i}$ given the other dimensions. TFB’s
contribution to this literature is its allowing $\hat{V}_{\beta_{0}}$ to guide
how to prioritize balance in $\hat{f}_{0}(X_{i})$ over $X_{i}$, and in certain
dimensions of $X_{i}$ over others. Additionally, ABAL 1 and ABAL 2 could
integrate TFB’s properties by carefully choosing $\delta$ and $\mathcal{F}$,
respectively — in fact, TFB is a special case of ABAL 2.191919The equivalence
holds if $B(w)=\frac{\hat{\sigma}^{2}}{n_{c}^{2}}||w||_{2}^{2}$ and
$\mathcal{F}=\\{x^{\top}\beta\ |\
||\hat{V}_{\beta_{0}}^{-\frac{1}{2}}(\hat{\beta}_{0}-\beta)||_{2}\leq\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\\}$.
In real-data scenarios, investigators could apply content knowledge to tune
$\delta$ and $\mathcal{F}$, potentially closing any performance gap between
choosing arbitrary values and applying TFB. Nevertheless, TFB’s advantage is
that it is defined by $\hat{f}_{0}$ and $\hat{V}_{\beta_{0}}$, turning the
problem of how best to balance $X_{i}$ into how best to model $Y_{i}(0)$.
Lastly, the augmented (weighted) estimator is a general form of estimators
(e.g., Abadie and Imbens,, 2011; Chernozhukov et al.,, 2018; Athey et al.,,
2018; Hirshberg and Wager,, 2020) that bias-correct
$\hat{\tau}_{\mathrm{wdim}}$ with a $\hat{f}_{0}$. Thorough discussion of this
estimator is left to Section 4.6, but TFB’s seeking balance in
$\hat{f}_{0}(X_{i})$ leads to its performing similarly to an augmented
estimator in practice.

Finally, although it provided a motivating example, TFB is not recommended
when $X_{i}$ is low-dimensional and OLS estimates $f_{0}$. This is because the
term penalizing imbalance in $X_{i}$,
$\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\times||\hat{V}_{\beta_{0}}^{\frac{1}{2}}\mathrm{imbal}(w,X,D)||_{2}$,
goes to 0 as $n$ increases quite generally, as
$\hat{V}_{\beta_{0}}\overset{p}{\rightarrow}0$ under mild conditions (e.g.,
$\hat{V}_{\beta_{0}}$ in (24)). TFB thus allows more imbalance in $X_{i}$ as
$n$ grows, assuming balance in
$\hat{f}_{0}(X_{i})=X_{i}^{\top}\hat{\beta}_{0}$ is sufficient. If $f_{0}$ is
linear in $X_{i}$, then this is a favorable result. However, if $f_{0}$ is not
linear in $X_{i}$, then this could induce bias that would otherwise not be
there were the weights to prioritize balance in $X_{i}$, as balance in $X_{i}$
could yield better balance in other functions of $X_{i}$. TFB is thus
recommended when $X_{i}$ is high-dimensional, either to start or because it is
a high-dimensional transformation of a smaller set of variables, and a
regularized regression estimates $f_{0}$. Balance in every dimension $X_{i}$
is likely infeasible in this case, while $f_{0}(X_{i})=X_{i}^{\top}\beta_{0}$
is believable. As such, the form of TFB where kernel regularized least squares
estimates $f_{0}$ (TFB-K) and that where LASSO regression estimates $f_{0}$
(TFB-L) are considered below. For future reference, Table 1 describes these
forms.

Table 1: Descriptions of TFB-K and TFB-L Form of TFB | Regression Estimator ($\hat{f}_{0}$) | Variance Estimator ($\hat{V}_{\beta_{0}}$)
---|---|---
TFB-K | Kernel regularized least squares | Given in (27)
TFB-L | LASSO | Residual bootstrap

###  4.3 Demonstrations

This section demonstrates TFB’s merits in two data generating processes
(DGPs).

#### DGP 1

The first demonstration compares TFB-K and the kernel-based weights of
Hazlett, (2020) and Kallus, (2020), which balance $K_{i}$. By compromising
between seeking balance in $f_{0}$ through balance in $\hat{f}_{0}$, and
through balance in $K_{ic}$, TFB-K strategically leaves imbalances in certain
confounders that offset imbalances in others, and achieves minimal bias and
high efficiency. This is despite little overlap in some confounders, and
approximating them with $K_{ic}$.

Consider a DGP in which $X_{i}$ is drawn from a bivariate normal distribution
around $C_{i}$, one of four equally likely (cluster)
centers,202020Specifically, $c_{1}=(0,0)$, $c_{2}=(0,5)$, $c_{3}=(5,0)$, and
$c_{4}=(5,5)$.

$\displaystyle X_{i}\overset{iid}{\sim}\mathcal{N}(C_{i},I_{2})\ \
\text{where}\ \ C_{i}\in\\{c_{1},c_{2},c_{3},c_{4}\\}\ \ \text{with}\ \
P(C_{i}=c_{\ell})=\frac{1}{4}\ \ \text{for}\ \ \ell=1,\dots,4$ (29)

Unknown to the researcher, however, $D_{i}$ and $Y_{i}$ depend on the
proximity of $X_{i}$ to each center, given by $Z_{i}^{(\ell)}$:

$\displaystyle Z_{i}^{(\ell)}=\frac{1}{||X_{i}-c_{\ell}||_{2}+1}$ (30)

Each unit’s proximity to its own center then determines the probability of
treatment:

$\displaystyle\mathrm{log}\frac{\pi(X_{i})}{1-\pi(X_{i})}$
$\displaystyle=\mathbbm{1}\\{C_{i}=c_{1}\\}(4)(Z_{i}^{(1)}-0.47)+\sum_{\ell=2}^{4}\mathbbm{1}\\{C_{i}=c_{\ell}\\}(20)(Z_{i}^{(\ell)}-0.47)$
(31)

where units closer to their center are more likely to be treated.212121The
$Z_{i}^{(\ell)}$ are centered by 0.47 in (31) because it allows
$P(D_{i}=1)\approx 0.50$. Figure 1 shows that units with centers $c_{2}$,
$c_{3}$, and $c_{4}$ show far more imbalance in the proximity to their center,
and therefore their respective $Z_{i}^{(\ell)}$, than are units with center
$c_{1}$. Units with center $c_{1}$ will thus be referred to as those in the
“easy-to-balance” cluster, and the other units will be referred to as those in
the “hard-to-balance” clusters.

Figure 1: Illustration of overlap in $X_{i}$ in clusters from DGP 1

(a) Easy-to-balance ($C_{i}=c_{1}$)

(b) Hard-to-balance ($C_{i}\in\\{c_{2},c_{3},c_{4}\\}$)

(c) Note: Plots of $X_{i}$ for (a) the easy-to-balance cluster and (b) one of
the three hard-to-balance clusters from a single draw from DGP 1 with
$n=1000$. Treated units in the hard-to-balance cluster tend to be closer to
their center, and control units further, than they are in the easy-to-balance
cluster.

Finally, the outcome is linear in the $Z_{i}^{(\ell)}$,

$\displaystyle
Y_{i}=10Z_{i}^{(1)}+\sum_{\ell=2}^{4}Z_{i}^{(\ell)}+\epsilon_{i},\ \ \
\epsilon_{i}\overset{iid}{\sim}N(0,1.5)$ (32)

where the proximity to $c_{1}$, captured by $Z_{i}^{(1)}$, is by far the most
important contributor to $Y_{i}$, and there is no treatment effect (i.e., the
ATT is 0). Note also that choice of $\operatorname{var}(\epsilon_{i})=1.5$
results in a true
$R^{2}=\frac{\operatorname{var}(Y_{i}-\epsilon_{i})}{\operatorname{var}(Y_{i})}$
of approximately 0.60.

Balancing $Z_{i}^{(2)}$, $Z_{i}^{(3)}$, and $Z_{i}^{(4)}$ is potentially
challenging in finite samples. The task thus becomes more difficult when
instead working with high-dimensional transformations of $X_{i}$ that contain
or approximate the $Z_{i}^{(\ell)}$, such as $K_{i}$ in the kernel-based
approaches to be applied here. However, balance in $f_{0}(X_{i})$ is
obtainable, as there is substantial overlap in the easy-to-balance cluster,
and $Z_{i}^{(1)}$ has large influence on $Y_{i}$. Allowing imbalance in
$Z_{i}^{(1)}$ of the opposite sign of the original imbalance, i.e. allowing
the weighted average control unit to be closer to $c_{1}$ than the average
treated unit, would relieve bias stemming from imbalance in the other
$Z_{i}^{(\ell)}$.222222For example, note that the original imbalance in
$Z_{i}$ is positive (see Figure 2(b)). However, weights such that
$\mathrm{imbal}(w,Z^{(1)},D)=-\frac{1}{100}$ and
$\mathrm{imbal}(w,Z^{(\ell)},D)=\frac{1}{30}$ for $2\leq\ell\leq 4$ also
satisfy EWC, as
$\displaystyle\mathrm{imbal}(w,f_{0}(X),D)=10(-\frac{1}{100})+1(\frac{1}{30})+1(\frac{1}{30})+1(\frac{1}{30})=0$
In theory, TFB-K should be capable of discerning this by consulting a
$\hat{f}_{0}$, while estimators that neglect $Y_{i}$ may increase their
variance by balancing the hard-to-balance clusters, and be too distracted by
them to properly balance $f_{0}(X_{i})$. At the same time, however, leaving
imbalance in $Z_{i}^{(1)}$ of the opposite sign is risky with $f_{0}$
uncertain, as it is highly impactful on the outcome. In order to correctly
offset this imbalance, the imbalance in the other $Z_{i}^{(\ell)}$ must be
tuned according to each variable’s coefficient in $f_{0}$.232323For example,
were the leftover imbalance in $Z_{i}^{(1)}$ to be allowed to equal
$-\frac{1}{100}$, the leftover imbalance in the other $Z_{i}^{(\ell)}$ would
need to equal $\frac{1}{30}$ on average in order for $f_{0}(X_{i})$ to be
exactly balanced.  However, even were TFB-K to know to use the
$Z_{i}^{(\ell)}$ instead of approximating them with $K_{ic}$, it could only
tune these imbalances according to a $\hat{f}_{0}$, meaning that they will
induce bias in each realization of the data that would not exist were all the
$Z_{i}^{(\ell)}$ balanced. Nevertheless, TFB-K should, in theory, inhibit
these imbalances from becoming too extreme by letting the variance of
$\hat{f}_{0}$ dictate how confidently it leaves imbalance in $K_{ic}$.

Figure 2 confirms these hypotheses, comparing TFB-K to Kernel Balancing (KBAL;
Hazlett,, 2020) and Kernel Optimal Matching (KOM; Kallus,, 2020).242424Each
method uses the Gaussian kernel,
$\mathcal{K}(X_{i},X_{j})=e^{-\frac{||X_{i}-X_{j}||_{2}^{2}}{b}}$, with
standardized $X_{i}$ and $b=2$. I forgo a kernel-selection procedure, such as
that proposed by Kallus, (2020), for simplicity. Additionally, kernel
regularized least squares with the Gaussian kernel and $b=2$ performs well
here ($\hat{f}_{0}(X_{i})$ and $f_{0}(X_{i})$ are correlated at 0.937 on
average), and $b=P$ has been found to perform well empirically (e.g.,
Hainmueller and Hazlett,, 2014; Schölkopf and Smola,, 2002).

Figure 2: Performance of TFB in DGP 1

(a) Bias, RMSE, and RESS

(b) Leftover imbalance

(c) Note: Comparison of TFB with $f_{0}$ estimated by kernel regularized least
squares (TFB-K; see Table 1), Kernel Optimal Matching (KOM), Kernel Balancing
(KBAL), and a difference in means (DIM) across 1000 draws from DGP 1 with
$n=1000$. (a) Distributions of the estimates from each method, as well as
their bias, RMSE, and RESS. (b) Leftover imbalance from each method after
weighting, where imbalance for $Z_{i}^{(2)}$, $Z_{i}^{(3)}$, and $Z_{i}^{(4)}$
is averaged at each iteration.

Define bias, root mean squared error (RMSE), and relative effective sample
size (RESS) as

$\displaystyle\text{Bias}=\frac{1}{M}\sum_{m=1}^{M}(\hat{\tau}_{\mathrm{wdim}}^{(m)}-\mathrm{ATT}),\
\ \
\text{RMSE}=\sqrt{\frac{1}{M}\sum_{m=1}^{M}(\hat{\tau}_{\mathrm{wdim}}^{(m)}-\mathrm{ATT})^{2}},$
$\displaystyle\text{RESS}=\frac{1}{M}\sum_{m=1}^{M}n_{c}^{-1}\biggr{(}\frac{(\sum_{i:D_{i}=0}w^{(m)}_{i})^{2}}{\sum_{i:D_{i}=0}(w^{(m)}_{i})^{2}}\biggr{)}$

where $m$ indexes the iteration from $1$ to $M$, and
$\hat{\tau}_{\mathrm{wdim}}^{(m)}$ and $w^{(m)}_{i}$ are the estimate and
weight for unit $i$, respectively, from the $m$th iteration. In Figure 2(a),
TFB-K shows minimal bias, unlike KOM, and though KBAL also shows little bias,
TFB-K’s RMSE is nearly two-thirds that of KBAL and KOM. Figure 2(b) reveals
why this is — TFB-K leaves imbalance in $Z_{i}^{(1)}$ of the opposite sign of
the original imbalance to offset imbalance in the other $Z_{i}^{(\ell)}$.
Consequently, although KBAL obtains better balance in the $Z_{i}^{(\ell)}$,
its weights are higher variance than are TFB-K’s, as evidenced by TFB-K’s
higher RESS (see Figure 2(a)). TFB-K’s strategy here notably challenges
traditional advice that one’s weights should attain the best balance in
$X_{i}$ possible given the setting (e.g., Stuart,, 2010). However, a more
universal guiding principle is that, given equal bias, a more efficient
estimator is preferable. Furthermore, balance remains a priority for TFB-K, as
it successfully reduces the imbalance in each variable. Additionally, it
obtains better balance in the $Z_{i}^{(\ell)}$ than does KOM because, tasked
with minimal overlap in the hard-to-balance clusters, KOM instead prioritizes
lower variance weights, as seen in its higher RESS.252525Further discussion on
KOM’s performance is warranted. Both TFB-K and KOM fit into the ABAL 2
framework with $B(w)=\lambda||w||_{2}^{2}$ for some $\lambda$, but differ on
their choices of $\mathcal{F}$. Presented here for KOM are the results after
using the data-driven procedure suggested by Kallus, (2020) to select
$\lambda$. However, I have found that decreasing $\lambda$ by several factors,
giving KOM more freedom to increase the variance of its weights in order to
further reduce the imbalance in the $K_{i}$, results in KOM performing on par
with TFB-K in terms of bias and RMSE. Thus, TFB-K may only outperform KOM in
the presented results because it happens to more effectively compromise
imbalance reduction and variance in the specific DGP, rather than because it
has chosen a $\mathcal{F}$ that allows it to more effectively target balance
in $f_{0}(X_{i})$, which is TFB-K’s central motivation.

#### DGP 2

Next, TFB can prioritize balance in the dimensions of $X_{i}$ that are most
influential on $Y_{i}$ by estimating $f_{0}$ with sparsity. This section
demonstrates this by applying TFB-L, which estimates $f_{0}$ with LASSO
regression, in a high-dimensional setting.

Consider a DGP in which $X_{i}$ contains three types of variables: (i)
“confounders”, $Z_{i}$, that are related to both $D_{i}$ and $Y_{i}$; (ii)
“distractors”, $A_{i}$, that are related to $D_{i}$, and are difficult to
balance, but are unrelated to $Y_{i}$; and (iii) “extraneous” variables,
$U_{i}$, that are independent of both $D_{i}$ and $Y_{i}$. The outcome and the
log odds of treatment are both linear in $Z_{i}=[Z_{i}^{(1)}\ \cdots\
Z_{i}^{(4)}]$,

$\displaystyle\mathrm{log}\frac{\pi(X_{i})}{1-\pi(X_{i})}$
$\displaystyle=\frac{1}{5}(Z_{i}^{(1)}+Z_{i}^{(2)}+Z_{i}^{(3)}+Z_{i}^{(4)})$
(33) $\displaystyle Y_{i}$
$\displaystyle=8Z_{i}^{(1)}+4Z_{i}^{(2)}+2Z_{i}^{(3)}+1Z_{i}^{(4)}+\epsilon_{i},\
\epsilon_{i}\overset{iid}{\sim}N(0,9.21^{2})$ (34)

where the $Z_{i}^{(\ell)}$ are equally imbalanced, and have varying levels of
influence on $Y_{i}$. Furthermore, the ATT is 0, and
$\epsilon_{i}\overset{iid}{\sim}N(0,9.21^{2})$ allows the true $R^{2}$ to be
approximately 0.50. The $Z_{i}$, the distractors ($A_{i}$), and the extraneous
variables ($U_{i}$) are then generated as

$\displaystyle Z_{i}\overset{iid}{\sim}\mathcal{N}(0,I_{4})\ ,\ A_{i}\ |\
D_{i}\overset{iid}{\sim}\mathcal{N}(D_{i}\vec{\mathbbm{1}}_{5},I_{5})\ ,\
\text{and}\ U_{i}\overset{iid}{\sim}\mathcal{N}(0,I_{10})$ (35)

Balance in $Z_{i}$ is sufficient to unbiasedly estimate the ATT. However,
$A_{i}$ and $U_{i}$, particularly $A_{i}$ which is significantly more
imbalanced than is $Z_{i}$, complicate balancing the entirety of $X_{i}$.
However, TFB-L performs well by finding $\hat{\beta}_{0}$ by LASSO regression.
Figure 3 compares TFB-L with ABAL 1 where the $\delta^{(\ell)}$ are all
equal.262626Specifically, ABAL 1 here is the method from Zubizarreta, (2015)
with $\delta$ implied by the approx.balance function in R (from the balanceHD
package) with default parameters. Further, to make the task of balancing
$Z_{i}$ more challenging for each method, TFB-L and ABAL 1 expand on $X_{i}$,
additionally (and needlessly) seeking balance in its squares and pairwise
interactions, a total of 209 variables. Figure 3 also includes TFB-K to
further illustrate the effect of TFB-L’s estimating $f_{0}$ with
sparsity.272727As in DGP 1, TFB-K uses the Gaussian kernel with bandwidth
$b=P=19$.

Figure 3: Performance of TFB in DGP 2

(a) Bias, RMSE, and RESS

(b) Leftover imbalance

(c) Note: Comparison of TFB with $f_{0}$ estimated by LASSO regression (TFB-L;
see Table 1), TFB with $f_{0}$ estimated by kernel regularized least squares
(TFB-K; see Table 1), ABAL 1 where the $\delta^{(\ell)}$ are all equal, and a
difference in means (DIM) across 1000 draws from DGP 2 with $n=1000$. (a)
Distributions of the estimates from each method, as well as their bias, RMSE,
and RESS. (b) Leftover imbalance from each method after weighting, where
imbalance in (i) $Z_{i}^{(2)}$, $Z_{i}^{(3)}$, and $Z_{i}^{(4)}$ and (ii) the
distractors is averaged at each iteration.

TFB-L shows minimal bias, while ABAL 1 is more comparable to a difference in
means. TFB-L also boasts a lower RMSE than ABAL 1 by over a factor of two, and
a higher RESS. TFB-K also performs well here, and is a marked improvement over
ABAL 1 in both bias and RMSE, but falls short of TFB-L.

As in DGP 1, the leftover imbalance in Figure 3(b) reveals why TFB-L excels.
Balancing the distractors in $A_{i}$ keeps ABAL 1 from properly balancing
$Z_{i}$ and reduces its RESS. Meanwhile, TFB-L largely disregards balance in
$A_{i}$, and obtains good balance in the $Z_{i}$, particularly in
$Z_{i}^{(1)}$, the most influential variable on $Y_{i}$. This is a byproduct
of how TFB-L models $f_{0}$ — LASSO regression forces coefficients in
$\hat{\beta}_{0}$ to 0 when their respective variables are weakly correlated
with $Y_{i}$, deflating their variances. Accordingly, Figure 4 shows that the
distractors’, extraneous variables’, and polynomial terms’ diagonal elements
in $\hat{V}_{\beta_{0}}$ are small, illuminating why TFB-L devalues balance in
those variables.

Figure 4: Diagonal elements of $\hat{V}_{\beta_{0}}^{\frac{1}{2}}$ for TFB-L
(see Table 1) in DGP 2 (a) Note: Each variable’s diagonal entry in
$\hat{V}_{\beta_{0}}^{\frac{1}{2}}$ is averaged across 1000 draws of DGP 2
with $n=1000$. The average entries of (i) the distractors, (ii) the extraneous
variables, and (iii) the squares and pairwise interactions of $X_{i}$ are then
averaged across their 5, 10, and 190 dimensions, respectively.

Additionally, the diagonal elements for the $Z_{i}^{(\ell)}$ in
$\hat{V}_{\beta_{0}}$ increase with the size of their coefficients in the
model for $Y_{i}$. Meanwhile, this variable selection does not carry over to
TFB-K — it obtains balance on the distractors at almost the level of ABAL 1,
while falling short of TFB-L’s balance in $Z_{i}^{(1)}$. This is because
kernel regularized least squares does not estimate coefficients for $K_{ic}$
with sparsity, so the diagonal entries of $\hat{V}_{\alpha_{0}}$, given by
(27), depend more on the variance of the elements of $K_{ic}$ than on their
contribution to $Y_{i}$. Nevertheless, TFB-K still obtains far better balance
on $Z_{i}$ than does ABAL 1, which explains its improved performance.

###  4.4 Choosing $q$

It is now worthwhile to examine $q$, the probability threshold in TFB. In
fact, the forms of TFB explored so far, TFB-K and TFB-L, offer two scenarios
that deserve separate discussions.

Starting with TFB-K, notice that $P\rightarrow\infty$ as $n\rightarrow\infty$.
This is because $P=n_{c}$ when $K_{ic}$ replaces $X_{i}$. Notice then that

$\displaystyle Q_{q}(\mathcal{X}^{2}_{P})$
$\displaystyle=\sqrt{2P}Q_{q}\biggr{[}\sqrt{\frac{P}{2}}\biggr{(}\frac{\sum_{\ell=1}^{P}Z_{\ell}^{2}}{P}-1\biggr{)}\biggr{]}+P\sim\sqrt{2P}\Phi^{-1}(q)+P$
(36)

where $Z_{\ell}$ are iid standard normal variables, $\Phi^{-1}(q)$ is the
$q$th quantile of a $N(0,1)$, and
$\sqrt{\frac{P}{2}}(\frac{\sum_{\ell=1}^{P}Z_{\ell}^{2}}{P}-1)\overset{d}{\rightarrow}N(0,1)$
by the Central Limit Theorem. By (36) above,
$\frac{\sqrt{Q_{q_{1}}(\mathcal{X}^{2}_{P})}}{\sqrt{Q_{q_{2}}(\mathcal{X}^{2}_{P})}}\rightarrow
1$ for fixed $q_{1}$ and $q_{2}$ and as $P\rightarrow\infty$, meaning the
specific choice of $q$ is inconsequential when $n$, and thus $P=n_{c}$, is
large. Figure 5 shows this empirically — TFB-K’s bias and RMSE in DGP 1 become
stable across $q$ as $n$ increases. However, with smaller $n$ (i.e., $n=500$),
there is a bias-variance tradeoff — bias decreases as $q$ increases, but RMSE
is mostly unchanged.

Figure 5: Demonstration of TFB-K (see Table 1) in DGP 1 for varying $q$

(a) $|$Bias$|$

(b) RMSE

(c) Note: Results across 1000 draws of DGP 1 with $n=1000$. $q$ takes values
from 0.05 to 0.95 in intervals of 0.05. TFB-K, where kernel regularized least
squares estimates $f_{0}$, is applied.

Moving to TFB-L, $P$ will be fixed in most applications. This means that
$\hat{V}_{\beta_{0}}\overset{p}{\rightarrow}0$ quite generally, so TFB-L will
devalue overall balance in $X_{i}$ as $n$ grows. A $q$ near 1 may thus be
preferable in this case, because it will yield better balance on $X_{i}$,
which may be valuable if $f_{0}$ has been misspecified (i.e., is not linear in
$X_{i}$). Therefore, because $q$ is inconsequential as $n$ increases for
TFB-K, and $q$ near 1 is advisable for TFB-L, $q=0.95$ is a reasonable choice
generally, and is used in all demonstrations here.

###  4.5 Variance estimation and confidence intervals

Moving towards standard errors and confidence intervals for TFB, Theorem 1
first describes sufficient conditions for the asymptotic normality of a
general $\hat{\tau}_{\mathrm{wdim}}$.

######  Theorem 1 (Conditions for asymptotic normality of a general
$\hat{\tau}_{\mathrm{wdim}}$)

Given Assumption 1, further assume the following conditions: (i) the $w_{i}$
are “honest”, or $w=w_{\mathrm{vec}}(X,D)$ for a function
$w_{\mathrm{vec}}(\cdot)$; (ii) $\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}<\infty$; (iii)
$\exists\underline{C},\overline{C}>0$ such that
$\underline{C}\leq\sigma_{i}^{2}\leq\overline{C}$; (iv)
$\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\ |\ D_{i}=1)<\infty$;
(v) $\exists\delta>0$ such that

$\displaystyle\frac{\sum_{i:D_{i}=0}w_{i}^{2+\delta}}{(\sum_{i:D_{i}=0}w_{i}^{2})^{\frac{2+\delta}{2}}}\overset{a.s.}{\rightarrow}0\
\ \ \text{and}\ \ \ \mathbbm{E}(|\epsilon_{i}|^{2+\delta}\ |\
X)\leq\overline{C}_{\delta}\ \ \ \text{for \ some}\ \ \
\overline{C}_{\delta}>0\text{;}$

and (vi) $\text{EWC Bias}=o_{p}(n^{-1/2})$. Then,
$V_{\mathrm{ATT}}^{-\frac{1}{2}}(\hat{\tau}_{\mathrm{wdim}}-\mathrm{ATT})\overset{d}{\rightarrow}N(0,1)$
where

$\displaystyle
V_{\mathrm{ATT}}=\frac{1}{n_{t}}\mathbbm{E}\biggr{(}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
\biggr{|}\
D_{i}=1\biggr{)}+\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$

Proof is given in Appendix B.1. Conditions (iii) and (v) restrict the moments
of $\epsilon_{i}$, and condition (iv) restricts the variance of the
individual-specific treatment effects (i.e. $\mathbbm{E}[Y_{i}(1)-Y_{i}(0)\ |\
X_{i}]$). Conditions (ii) and (v) then restrict how extreme the $w_{i}$ can be
in asymptopia. Condition (vi) is key to the $\sqrt{n}$-consistency of
$\hat{\tau}_{\mathrm{wdim}}$, requiring that the bias of
$\hat{\tau}_{\mathrm{wdim}}$ converges in probability to 0 at a fast enough
rate, and the assumptions that allow it to hold will differ by the weighting
method.282828For example, Abadie and Imbens, (2006) show it does not hold
generally for the matching estimators they examine if $P>1$. Finally,
condition (i) requires that $w_{i}$ not depend on the $Y_{i}$.

Condition (i) is violated by TFB as defined. However, sample splitting, as in
Chernozhukov et al., (2018), remedies this, and is done in all demonstrations
of TFB here. Sample splitting with TFB requires randomly splitting the full
sample into (at least) two non-overlapping subsamples, using one subsample to
obtain $(\hat{\beta}_{0},\hat{V}_{\beta_{0}})$ and the other to form the
$w_{i}$ and $\hat{\tau}_{\mathrm{wdim}}$. Next, switch the roles of the two
subsamples, and the final estimate is the average of the
$\hat{\tau}_{\mathrm{wdim}}$ from each subsample. With sample splitting,
condition (i) holds for the $\hat{\tau}_{\mathrm{wdim}}$ from each subsample,
say $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ and $\hat{\tau}_{\mathrm{wdim}}^{(2)}$,
as well as potentially Theorem 1, with respective variances
$V_{\mathrm{ATT}}^{(1)}$ and $V_{\mathrm{ATT}}^{(2)}$. However,
$\hat{\tau}_{\mathrm{wdim}}^{(1)}$ and $\hat{\tau}_{\mathrm{wdim}}^{(2)}$ may
covary because the $Y_{i}$ in $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ contribute to
the $w_{i}$ in $\hat{\tau}_{\mathrm{wdim}}^{(2)}$, and vice versa, meaning
$(\frac{1}{4}V_{\mathrm{ATT}}^{(1)}+\frac{1}{4}V_{\mathrm{ATT}}^{(2)})$ may
not be consistent for the asymptotic variance of
$\frac{\hat{\tau}_{\mathrm{wdim}}^{(1)}+\hat{\tau}_{\mathrm{wdim}}^{(2)}}{2}$.
Table 2 shows evidence of such correlation, albeit at low levels, particularly
as $n$ increases. Nevertheless, if $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ and
$\hat{\tau}_{\mathrm{wdim}}^{(2)}$ are asymptotically uncorrelated,

$\displaystyle\frac{\hat{\tau}_{\mathrm{wdim}}^{(1)}+\hat{\tau}_{\mathrm{wdim}}^{(2)}}{2}\pm\Phi^{-1}\biggr{(}\frac{1+\gamma}{2}\biggr{)}\times\sqrt{\frac{1}{4}V_{\mathrm{ATT}}^{(1)}+\frac{1}{4}V_{\mathrm{ATT}}^{(2)}}$
(37)

yields asymptotically valid $(100\times\gamma)$% confidence intervals for the
ATT.

Table 2: Correlation of $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ and $\hat{\tau}_{\mathrm{wdim}}^{(2)}$ from TFB in DGP 1 and DGP 2 | $n=500$ | $n=1000$ | $n=2000$
---|---|---|---
DGP 1 (TFB-K) | 0.075* | 0.057 | 0.004
DGP 2 (TFB-L) | 0.171* | 0.057 | 0.058
DGP 2 (TFB-K) | 0.257* | 0.137* | 0.103*
(a) Note: Results are across 1000 draws of DGP 1 and DGP 2. Starred (*) values
indicate significance at the 0.05 level of the coefficient for
$\hat{\tau}_{\mathrm{wdim}}^{(1)}$ in an OLS regression that predicts
$\hat{\tau}_{\mathrm{wdim}}^{(2)}$, with robust standard errors (White,,
1980). See Table 1 for descriptions of TFB-K and TFB-L.

Turning to variance estimation, assuming that $\hat{f}_{0}$ is a consistent
estimator of $f_{0}$ suggests the plug-in estimator

$\displaystyle\hat{V}_{\mathrm{ATT}}=\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}[Y_{i}(1)-\hat{f}_{0}(X_{i})-\hat{\tau}_{\mathrm{wdim}}]^{2}+\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\hat{e}_{i}^{2}$
(38)

where $\hat{e}_{i}=Y_{i}(0)-\hat{f}_{0}(X_{i})$. Theorem 2 describes
conditions under which
$\hat{V}_{\mathrm{ATT}}\overset{p}{\rightarrow}V_{\mathrm{ATT}}$.

######  Theorem 2 (Consistency of $\hat{V}_{\mathrm{ATT}}$)

Given Assumption 1, and conditions (i)-(vi) for Theorem 1, further assume
that: (vii)
$\frac{1}{n}\sum_{i=1}^{n}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]^{2}=o_{p}(1)$;
(viii)
$\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\epsilon_{i}^{2}\overset{p}{\rightarrow}\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$;
and (ix)
$\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]^{2}=o_{p}(1)$.
Then, $\hat{V}_{\mathrm{ATT}}\overset{p}{\rightarrow}V_{\mathrm{ATT}}$.

See Appendix B.2 for proof. Conditions (vii) and (ix) quantify how accurate an
estimator of $f_{0}$ that $\hat{f}_{0}$ must be in terms of unweighted and
$w^{2}$-weighted mean squared error, respectively. Additionally, condition
(viii) and (ix) allow $\hat{V}_{\mathrm{ATT}}$ to be robust to heteroscedastic
$\epsilon_{i}$.

Figures 6 and 7 demonstrate that $\hat{V}_{\mathrm{ATT}}$ performs well with
TFB-L in DGP 2, and with TFB-K in DGP 1 as $n$ increases.

Figure 6: Coverage of 95% confidence intervals for
$\frac{\hat{\tau}_{\mathrm{wdim}}^{(1)}+\hat{\tau}_{\mathrm{wdim}}^{(2)}}{2}$
and $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ from TFB-K (see Table 1) in DGP 1

(a)
$\frac{\hat{\tau}_{\mathrm{wdim}}^{(1)}+\hat{\tau}_{\mathrm{wdim}}^{(2)}}{2}$
with
$\widehat{\operatorname{var}}=\frac{1}{4}V^{(1)}_{\mathrm{ATT}}+\frac{1}{4}V^{(2)}_{\mathrm{ATT}}$

(b) $\hat{\tau}^{(1)}_{\mathrm{wdim}}$ with
$\widehat{\operatorname{var}}=V_{\mathrm{ATT}}^{(1)}$

(c) Note: Results across 1000 draws from DGP 1. The black line indicates the
coverage rate with $V_{\mathrm{ATT}}$ (i.e., the true $f_{0}$,
$\sigma_{i}^{2}$, and ATT are used). The blue line indicates the coverage rate
with $\hat{V}_{\mathrm{ATT}}$. Figure 7: Coverage of 95% confidence intervals
for
$\frac{\hat{\tau}_{\mathrm{wdim}}^{(1)}+\hat{\tau}_{\mathrm{wdim}}^{(2)}}{2}$
and $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ from TFB-L (see Table 1) in DGP 2

(a)
$\frac{\hat{\tau}_{\mathrm{wdim}}^{(1)}+\hat{\tau}_{\mathrm{wdim}}^{(2)}}{2}$
with
$\widehat{\operatorname{var}}=\frac{1}{4}V^{(1)}_{\mathrm{ATT}}+\frac{1}{4}V^{(2)}_{\mathrm{ATT}}$

(b) $\hat{\tau}^{(1)}_{\mathrm{wdim}}$ with
$\widehat{\operatorname{var}}=V_{\mathrm{ATT}}^{(1)}$

(c) Note: Results across 1000 draws from DGP 2. The black line indicates the
coverage rate with $V_{\mathrm{ATT}}$ (i.e., the true $f_{0}$,
$\sigma_{i}^{2}$, and ATT are used). The blue line indicates the coverage rate
with $\hat{V}_{\mathrm{ATT}}$.

The undercoverage for TFB-K when $n=500$ in DGP 1 is likely because TFB-K
still shows bias at that sample size (see Figure 5). Figures 6 and 7 also show
that potential correlation between $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ and
$\hat{\tau}_{\mathrm{wdim}}^{(2)}$ has little consequence here, as the
coverage rate for their average is only meaningfully below the coverage rate
for $\hat{\tau}_{\mathrm{wdim}}^{(1)}$ when $n=500$ in DGP 1. Finally,
$\hat{V}_{\mathrm{ATT}}$ overestimates $V_{\mathrm{ATT}}$ here, but this need
not be true generally.

#### Reporting TFB estimates after sample splitting

When sample-splitting in real-data applications, it is prudent to examine
estimates from many random splits of the sample. However, if a single estimate
is required for reporting, Chernozhukov et al., (2018) suggest the
median.292929Chernozhukov et al., (2018) also prove that the mean is valid.
Let $\hat{\tau}_{\mathrm{wdim},s}$ be the TFB estimate from the $s$th split of
the sample and let $\hat{V}_{\mathrm{ATT},s}$ consistently estimate its
variance. Then, let $\hat{\tau}_{\mathrm{wdim},\mathrm{med}}$ be the median of
the $\hat{\tau}_{\mathrm{wdim},s}$. Results from Chernozhukov et al., (2018)
imply the variance estimator

$\displaystyle\widehat{\operatorname{var}}(\hat{\tau}_{\mathrm{wdim},\mathrm{med}})=\mathrm{median}\biggr{\\{}\hat{V}_{\mathrm{ATT},s}+\frac{(\hat{\tau}_{\mathrm{wdim},s}-\hat{\tau}_{\mathrm{wdim},\mathrm{med}})^{2}}{n}\biggr{\\}}$
(39)

with $(100\times\gamma)$% confidence intervals

$\displaystyle\hat{\tau}_{\mathrm{wdim},\mathrm{med}}\pm\Phi^{-1}\biggr{(}\frac{1+\gamma}{2}\biggr{)}\times\widehat{\operatorname{var}}(\hat{\tau}_{\mathrm{wdim},\mathrm{med}})$
(40)

###  4.6 Connection to augmented estimation

That TFB allows $\hat{f}_{0}$ to inform its weights is key to its merits.
However, using estimates of $f_{d}$ in conjunction with weights is not novel —
it is essential for the augmented (weighted) estimator, the typical form of
estimators that do so (e.g., Abadie and Imbens,, 2011; Chernozhukov et al.,,
2018; Athey et al.,, 2018; Hirshberg and Wager,, 2020).303030Augmented
estimators do not necessitate weights that depend on $\hat{f}_{d}$, as in TFB.
However, this is not novel either. Examples include Tan, (2010), Rotnitzky et
al., (2012), Shortreed and Ertefaie, (2017), Ning et al., (2020) and Kuang et
al., (2017). ,313131Disregarding weights and simply using a $\hat{f}_{d}$ to
impute the counterfactual potential outcomes has also recently been revisited
(e.g., Ratkovic and Tingley,, 2017; Xu,, 2017; Liu et al.,, 2021). This
section thus explores the connection between TFB and the augmented estimator.

Recall from Section 3.1 that the expectation of the EWC Bias is exactly the
bias of $\hat{\tau}_{\mathrm{wdim}}$. Consider then bias-correcting
$\hat{\tau}_{\mathrm{wdim}}$ by estimating the EWC Bias with a $\hat{f}_{0}$,
and subtracting the resulting estimate from $\hat{\tau}_{\mathrm{wdim}}$:

$\displaystyle\hat{\tau}_{\mathrm{aug}}$
$\displaystyle=\hat{\tau}_{\mathrm{wdim}}-\underbrace{\mathrm{imbal}(w,\hat{f}_{0}(X),D)}_{\widehat{\text{EWC
Bias}}}$ (41)

$\hat{\tau}_{\mathrm{aug}}$ is the augmented estimator, and the common form of
doubly-robust estimators (Robins et al.,, 1994; Robins and Rotnitzky,, 1995;
van der Laan and Rubin,, 2006; Chernozhukov et al.,, 2018), which employ
propensity score weights, and are consistent when $f_{0}$ or $\pi$ has been
correctly specified.323232Tan, (2010) provides an illuminating review of
doubly robust estimators that can be expressed as $\hat{\tau}_{\mathrm{aug}}$.
Augmented estimators have also been applied with approximate balancing weights
(e.g. Athey et al.,, 2018, Hirshberg and Wager,, 2020), and in related methods
such as synthetic control (Ben-Michael et al.,, 2021). In fact, TFB is similar
in philosophy to the augmented estimator explored by Hirshberg and Wager,
(2020), which specifies a function space that contains the difference between
a $\hat{f}_{0}$ and $f_{0}$ (i.e., $f_{0}-\hat{f}_{0}$), and chooses weights
that balance the functions in that space.

Noting that $\hat{f}_{0}(X_{i})=X_{i}^{\top}\hat{\beta}_{0}$ in TFB reveals
its connection to augmented estimators, as the estimated EWC Bias subtracted
from $\hat{\tau}_{\mathrm{wdim}}$ to form $\hat{\tau}_{\mathrm{aug}}$ is

$\displaystyle\widehat{\text{EWC
Bias}}=\mathrm{imbal}(w,\hat{f}_{0}(X),D)=\mathrm{imbal}(w,X,D)^{\top}\hat{\beta}_{0}$
(42)

the magnitude of which TFB penalizes. Therefore, often, $\widehat{\text{EWC
Bias}}\approx 0$ in TFB, implying
$\hat{\tau}_{\mathrm{wdim}}\approx\hat{\tau}_{\mathrm{aug}}$. Figure 8
confirms this suspicion in DGPs 1 and 2 — TFB’s augmented and weighted
difference in means estimators are numerically identical.

Figure 8: Performance of TFB and augmented estimators in DGPs 1 and 2

(a) DGP 1

(b) DGP 2

(c) Note: Results across 1000 draws of DGPs 1 and 2. (a) Comparison of TFB
where $f_{0}$ is estimated by kernel regularized least squares (TFB-K; see
Table 1), its augmented form (augTFB-K), and augmented forms of Kernel
Balancing (augKBAL) and Kernel Optimal Matching (augKOM). $\hat{f}_{0}$ is
found by kernel regularized least squares for augKBAL and augKOM. augKOM is a
special case of the estimator studied by Hirshberg and Wager, (2020). (b)
Comparison of TFB with $f_{0}$ estimated by LASSO regression (TFB-L; see Table
1) and its augmented form (augTFB-L), TFB with $f_{0}$ estimated by kernel
regularized least squares (TFB-K; see Table 1) and its augmented form
(augTFB-K), and an augmented form of ABAL 1 (augABAL 1) where the
$\delta^{(\ell)}$ are all equal. $\hat{f}_{0}$ is found by LASSO regression
for augABAL 1. TFB-L and augABAL 1 include the squares and pairwise
interactions of $X_{i}$ as regressors. augABAL 1 is a special case of the
estimator studied by Athey et al., (2018).

Figure 8 also shows that TFB still achieves lower RMSE than do augmented forms
of the estimators examined in Figure 2(a) and 3(a), among them special cases
of the estimators studied by Athey et al., (2018) and Hirshberg and Wager,
(2020), though the gap in performance has shrunk.

##  5 Application

This section assesses TFB’s performance on the National Supported Work
Demonstration (NSW) data. NSW was a labor training program in the mid-1970s
that randomly selected its participants from a group of candidates. In
addition to experimental treated and control groups drawn from the program
candidates, data on comparison groups of non-candidates are available.
LaLonde, (1986) and Dehejia and Wahba, (1999) use these data to compare non-
experimental estimates of NSW’s impact on 1978 earnings (the outcome here) to
the unbiased experimental estimate. Defining the treated (185 units),
experimental control (260), and non-candidate comparison (2,490) groups as in
Dehejia and Wahba, (1999) yields an experimental estimate of $1,794 and a
naive observational estimate (i.e., the unweighted difference in means between
the experimental treated and non-candidate comparison groups) of -$15,205.

The available covariates are age, years of education, an indicator of having a
high school degree, 1974 and 1975 earnings, indicators of unemployment in 1974
and 1975, marital status, and several indicators of race/ethnicity. Figure 9
compares TFB-L, where $f_{0}$ is estimated by LASSO regression, and TFB-K,
where $f_{0}$ is estimated by kernel regularized least squares, to seven other
estimators.

Figure 9: Performance on NSW data

(a) Comparison of estimates

(b) TFB estimates across 100 sample splits

(c) Note: (a) Comparison of TFB-K (see Table 1), TFB-L (see Table 1), EBAL,
(aug)ABAL 1, (aug)KBAL, and (aug)KOM. TFB estimates are the median estimates
from 100 random splits of the sample. 95% confidence intervals are provided
for TFB-K and TFB-L, with confidence intervals for the median estimate per
Section 4.5. (b) Boxplots of TFB estimates over 100 splits of the sample. The
dashed line indicates the experimental benchmark ($1,794).

The first of these estimators is Entropy Balancing (EBAL; Hainmueller,, 2012),
which is an exact balancing method (see Section 3.2). The next three
estimators are the same approximate balancing methods that were used in the
demonstrations from Section 4.3: ABAL 1 where all the $\delta^{(\ell)}$ are
equal (see Section 3.2), Kernel Balancing (KBAL; Hazlett,, 2020), and Kernel
Optimal Matching (KOM; Kallus,, 2020). The final three estimators are
augmented forms (see Section 4.6) of the three approximate balancing
methods.333333Each method’s specifics are consistent with Sections 4.3 and
4.6. The kernel-based approximate balancing methods (i.e., TFB-K, KBAL, and
KOM) use the Gaussian kernel with bandwidth $b=P=10$, and their corresponding
augmented forms use $\hat{f}_{0}$ from kernel regularized least squares.
augABAL uses $\hat{f}_{0}$ from LASSO regression. TFB-K uses
$\hat{V}_{\alpha_{0}}$ from (27) and TFB-L finds $\hat{V}_{\beta_{0}}$ by
residual bootstrapping. Both TFB-K and TFB-L set $q=0.95$, per Section 4.4.
TFB-K, EBAL, (aug)KBAL, and (aug)KOM are limited to the 10 untransformed
covariates, while (aug)ABAL 1 and TFB-L additionally use the covariates’
squared terms and pairwise interactions, a total of 52
covariates.343434Following Hainmueller, (2012), I exclude various squares and
interactions. These are the interaction between the indicator for high school
degree and years of education, the interactions of the race/ethnicity
indicators, and the squares and interactions of real earnings in 1974 and
1975.  Because TFB-L and TFB-K both employ sample splitting (see Section 4.5),
estimates across 100 random splits of the sample are provided in Figure 9(b),
and Figure 9(a) reports the median estimate.

As are the other estimators, TFB-K and TFB-L are large improvements over the
naive estimate of -$15,205, and are the second and seventh closest (of nine),
respectively, to the experimental benchmark. However, as this is just one
realization of the data, drawing conclusions from these minute differences is
challenging. It is more affirming that TFB performs comparably to other well-
known estimators and that the experimental estimate falls within TFB-K’s and
TFB-L’s 95% confidence intervals (as do the other estimators excluding augABAL
1).

Finally, Table 3(a) displays the standardized imbalance in the covariates for
each weighting method.

Table 3: Leftover imbalance in NSW data | Partial Corr. | Initial Imbal. | TFB-K | TFB-L | EBAL | ABAL 1 | KBAL | KOM
---|---|---|---|---|---|---|---|---
Age | -0.08 | -0.86 | -0.12 | -0.34 | 0 | -0.03 | -0.13 | -0.11
Years of Education | 0.11 | -0.58 | -0.05 | 0.04 | 0 | -0.05 | 0.03 | -0.03
Ethnicity (Black) | -0.02 | 1.3 | 0.07 | 0.18 | 0 | 0.05 | 0.01 | 0.05
Ethnicity (Hispanic) | 0.04 | 0.15 | 0.04 | 0.06 | 0 | 0.05 | -0.02 | 0.02
Married | 0.04 | -1.76 | -0.08 | -0.26 | 0 | -0.06 | -0.06 | -0.06
No HS Degree | 0.02 | 0.85 | 0.06 | 0.06 | 0 | 0.06 | 0.07 | 0.05
Earnings (1974) | 0.18 | -1.26 | -0.01 | 0 | 0 | -0.02 | -0.04 | -0.05
Earnings (1975) | 0.32 | -1.26 | -0.03 | -0.01 | 0 | -0.06 | -0.05 | -0.07
Unemployed (1974) | 0.02 | 1.85 | 0.02 | 0.09 | 0 | 0.06 | 0.01 | 0.08
Unemployed (1975) | -0.03 | 1.46 | -0.01 | -0.06 | 0 | 0.06 | -0.01 | 0.02
(a) Note: The “Partial Corr.” column reports the partial correlation (given
the other variables) of each variable and real earnings in 1978 (i.e., the
outcome) within the non-candidate comparison group, and the “Initial Imbal.”
column reports the imbalance before weighting. The remaining columns report
the imbalance after weighting for each method. All covariates were
standardized prior to calculating imbalances. Imbalances for TFB are averaged
across 100 splits of the sample. See Table 1 for descriptions of TFB-K and
TFB-L.

TFB shows similar tendencies here as in previous demonstrations. TFB obtains
better balance than the other approximate balancing methods in 1974 and 1975
earnings, which have the highest partial correlations with the outcome, though
the differences are slight. Furthermore, because TFB-K does not estimate
$f_{0}$ with sparsity, TFB-K obtains good balance in all of the covariates,
even those with low partial correlation with the outcome. Meanwhile, TFB-L
leaves imbalance in age, the indicator of being Black, and the indicator of
being married, the latter two of which show low partial correlation with the
outcome.

##  6 Conclusion

This paper has introduced TFB, an approximate balancing framework that first
regresses the outcome on the observed covariates, and then uses the resulting
regression function and its estimated variance to determine how to balance the
covariates. As a result, TFB seeks balance in the predicted values of the
regression function and in the covariates, allowing the regression function’s
variance to determine how much of the latter is necessary. In simulated
examples, TFB showed promising results when compared to applications of
existing approximate balancing methods that disregarded the outcome and any
potential background knowledge (e.g., which covariates are most influential on
the outcome) when balancing the covariates. In real-data scenarios,
investigators could apply background knowledge to tune these methods,
potentially closing any performance gap that might exist between these methods
and TFB. Nevertheless, TFB’s advantage is that the choice of regression
estimator and an estimator for its variance entirely determines how it
balances the covariates. This turns the problem of how best to balance the
covariates into how best the model the outcome. TFB does, however, require
different weights for different outcomes. This is a practical disadvantage in
that it complicates analyses, and can be time-consuming. But it is also a
potential advantage, particularly in settings with minimal overlap in the
covariates — while it may be impossible to find a single set of weights that
is appropriate for all outcomes, TFB may be able to find a different sets of
weights that are individually appropriate for some (or all) of the outcomes.

One consequence of TFB is that it may intentionally leave imbalance in the
covariates even when exact balance is feasible, either because certain
dimensions appear conditionally unrelated to the outcome, or even to use
imbalances in some dimensions to offset imbalances in others. Intentional
imbalance is not a novel idea — several existing methods also leave imbalance
in the name of variance reduction (e.g., Zubizarreta,, 2015) or to prioritize
balance in more predictive dimensions (e.g., Ning et al.,, 2020). However, the
motivation behind this characteristic of TFB does challenge traditional advice
(e.g., Stuart,, 2010) that the ultimate goal of weighting is to equate the
distributions of the covariates in the treated and control groups, with a
particular focus on mean balance in the covariates as a benchmark for success.
TFB instead argues that any balance in the covariates should only be a
byproduct of balance in $f_{0}(X_{i})=\mathbbm{E}[Y_{i}(0)\ |\ X_{i}]$, which
is all that is required for unbiased estimation. It is understandable to be
suspicious of this perspective in practice, given TFB’s reliance on a
$\hat{f}_{0}$, not $f_{0}$. However, note that it is theoretically sound, and
is shown here to potentially yield efficiency gains. Further, recall that TFB
still does value overall balance in the covariates, using it to safeguard
against an over-reliance on $\hat{f}_{0}$.

As for implementation, TFB is recommended in high-dimensional settings,
particularly the forms that use kernel regularized least squares to regress
the outcome, or an estimator that induces sparsity, such as the LASSO.
However, I do not recommend a form of TFB that regresses the outcome with OLS
in a low-dimensional setting. Furthermore, when applying TFB, a probability
threshold ($q$) close to 1 is recommended (e.g., $q=0.95$), and I suggest
calculating estimates across multiple random splits of the sample, and
ultimately reporting the median. Note also that while this paper has framed
TFB as an estimator for the average treatment effect on the treated (ATT),
Appendix A adapts it to estimate the average treatment effect (ATE) and the
average treatment effect on the controls (ATC).

Finally, this paper has also introduced TFI, a balance diagnostic that TFB
minimizes over weights, subject to constraints on their variance. However,
this is not TFI’s only possible application. It could incorporated into best
subset matching (e.g., Nikolaev et al.,, 2013), where TFI (and other
components) would be minimized over $w_{i}\in\\{0,\frac{1}{M}\\}$ for some
whole number $M\leq n_{c}$. TFI could also be integrated into a loss function
used to estimate the parameters for a propensity score (e.g., Zhao,, 2019;
Pirracchio and Carone,, 2018). The resulting propensity score weighted
estimator might then be doubly robust. TFI might also have uses in other
methods, such as a diagnostic for choosing the dimension of the approximation
of the gram matrix in Kernel Balancing (Hazlett,, 2020). However, I leave such
ideas to future research.

## References

  * Abadie and Imbens, (2006) Abadie, A. and Imbens, G. W. (2006).  Large sample properties of matching estimators for average treatment effects.  Econometrica, 74(1):235–267.
  * Abadie and Imbens, (2011) Abadie, A. and Imbens, G. W. (2011).  Bias-corrected matching estimators for average treatment effects.  Journal of Business & Economic Statistics, 29(1):1–11.
  * Antonelli et al., (2018) Antonelli, J., Cefalu, M., Palmer, N., and Agniel, D. (2018).  Doubly robust matching estimators for high dimensional confounding adjustment.  Biometrics, 74(4):1171–1179.
  * Athey et al., (2018) Athey, S., Imbens, G. W., and Wager, S. (2018).  Approximate residual balancing: Debiased inference of average treatment effects in high dimensions.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 80(4):597–623.
  * Ben-Michael et al., (2021) Ben-Michael, E., Feller, A., and Rothstein, J. (2021).  The augmented synthetic control method.  Journal of the American Statistical Association, 0(0):1–15.
  * Brookhart et al., (2006) Brookhart, M. A., Schneeweiss, S., Rothman, K. J., Glynn, R. J., Avorn, J., and Stürmer, T. (2006).  Variable selection for propensity score models.  American Journal of Epidemiology, 163(12):1149–1156.
  * Chan et al., (2016) Chan, K. C. G., Yam, S. C. P., and Zhang, Z. (2016).  Globally efficient non-parametric inference of average treatment effects by empirical balancing calibration weighting.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78(3):673–700.
  * Chatterjee and Lahiri, (2010) Chatterjee, A. and Lahiri, S. N. (2010).  Asymptotic properties of the residual bootstrap for lasso estimators.  Proceedings of the American Mathematical Society, 138(12):4497–4509.
  * Chatterjee and Lahiri, (2011) Chatterjee, A. and Lahiri, S. N. (2011).  Bootstrapping lasso estimators.  Journal of the American Statistical Association, 106(494):608–625.
  * Chattopadhyay et al., (2020) Chattopadhyay, A., Hase, C. H., and Zubizarreta, J. R. (2020).  Balancing versus modeling approaches to weighting in practice.  Statistics in Medicine, 39(24):3227–3254.
  * Chernozhukov et al., (2018) Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018).  Double/debiased machine learning for treatment and structural parameters.  Econometrics Journal, 21(1):C1–C68.
  * De Luna et al., (2011) De Luna, X., Waernbaum, I., and Richardson, T. S. (2011).  Covariate selection for the nonparametric estimation of an average treatment effect.  Biometrika, 98(4):861–875.
  * Dehejia and Wahba, (1999) Dehejia, R. H. and Wahba, S. (1999).  Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs.  Journal of the American Statistical Association, 94(448):1053–1062.
  * Deming and Stephan, (1940) Deming, W. E. and Stephan, F. F. (1940).  On a least squares adjustment of a sampled frequency table when the expected marginal totals are known.  The Annals of Mathematical Statistics, 11(4):427–444.
  * Deville and Särndal, (1992) Deville, J.-C. and Särndal, C.-E. (1992).  Calibration estimators in survey sampling.  Journal of the American Statistical Association, 87(418):376–382.
  * Hainmueller, (2012) Hainmueller, J. (2012).  Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies.  Political Analysis, 20(1):25–46.
  * Hainmueller and Hazlett, (2014) Hainmueller, J. and Hazlett, C. (2014).  Kernel regularized least squares: Reducing misspecification bias with a flexible and interpretable machine learning approach.  Political Analysis, 22(2):143–168.
  * Hansen, (2008) Hansen, B. B. (2008).  The prognostic analogue of the propensity score.  Biometrika, 95(2):481–488.
  * Hazlett, (2020) Hazlett, C. (2020).  Kernel balancing: A flexible non-parametric weighting procedure for estimating causal effects.  Statistica Sinica, 30(1):1155–1189.
  * Hirshberg et al., (2021) Hirshberg, D. A., Maleki, A., and Zubizarreta, J. R. (2021).  Minimax linear estimation of the retargeted mean.  arXiv preprint arXiv:1901.10296v2.
  * Hirshberg and Wager, (2020) Hirshberg, D. A. and Wager, S. (2020).  Augmented minimax linear estimation.  arXiv preprint arXiv:1712.00038v6.
  * Huang et al., (2006) Huang, J., Smola, A. J., Gretton, A., Borgwardt, K. M., and Schölkopf, B. (2006).  Correcting sample selection bias by unlabeled data.  Advances in Neural Information Processing Systems, 19:601–608.
  * Imai and Ratkovic, (2014) Imai, K. and Ratkovic, M. (2014).  Covariate balancing propensity score.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(1):243–263.
  * Kallus, (2020) Kallus, N. (2020).  Generalized optimal matching methods for causal inference.  Journal of Machine Learning Research, 21:1–54.
  * Kang and Schafer, (2007) Kang, J. D. Y. and Schafer, J. L. (2007).  Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data.  Statistical Science, 22(4):523–539.
  * Knight and Fu, (2000) Knight, K. and Fu, W. (2000).  Asymptotics for lasso-type estimators.  The Annals of Statistics, 28(5):1356–1378.
  * Kuang et al., (2017) Kuang, K., Cui, P., Li, B., Jiang, M., and Yang, S. (2017).  Estimating treatment effect in the wild via differentiated confounder balancing.  In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 265–274.
  * LaLonde, (1986) LaLonde, R. J. (1986).  Evaluating the econometric evaluations of training programs with experimental data.  The American Economic Review, 76(4):604–620.
  * Leacy and Stuart, (2014) Leacy, F. P. and Stuart, E. A. (2014).  On the joint use of propensity and prognostic scores in estimation of the average treatment effect on the treated: A simulation study.  Statistics in Medicine, 33(20):3488–3508.
  * Lee et al., (2010) Lee, B. K., Lessler, J., and Stuart, E. A. (2010).  Improving propensity score weighting using machine learning.  Statistics in Medicine, 29(3):337–346.
  * Liu et al., (2021) Liu, L., Wang, Y., and Xu, Y. (2021).  A practical guide to counterfactual estimators for causal inference with time-series cross-sectional data.  arXiv preprint arXiv:2107.00856v1.
  * Nikolaev et al., (2013) Nikolaev, A. G., Jacobson, S. H., Cho, W. K. T., Sauppe, J. J., and Sewell, E. C. (2013).  Balance optimization subset selection (BOSS): An alternative approach for causal inference with observational data.  Operations Research, 61(2):398–412.
  * Ning et al., (2020) Ning, Y., Peng, S., and Imai, K. (2020).  Robust estimation of causal effects via a high-dimensional covariate balancing propensity score.  Biometrika, 107(3):533–554.
  * Patrick et al., (2011) Patrick, A. R., Schneeweiss, S., Brookhart, M. A., Glynn, R. J., Rothman, K. J., Avorn, J., and Stürmer, T. (2011).  The implications of propensity score variable selection strategies in pharmacoepidemiology: An empirical illustration.  Pharmacoepidemiology and Drug Safety, 20(6):551–559.
  * Pirracchio and Carone, (2018) Pirracchio, R. and Carone, M. (2018).  The balance super learner: A robust adaptation of the super learner to improve estimation of the average treatment effect in the treated based on propensity score matching.  Statistical Methods in Medical Research, 27(8):2504–2518.
  * Pirracchio et al., (2014) Pirracchio, R., Petersen, M. L., and van der Laan, M. (2014).  Improving propensity score estimators’ robustness to model misspecification using super learner.  American Journal of Epidemiology, 181(2):108–119.
  * Ratkovic and Tingley, (2017) Ratkovic, M. and Tingley, D. (2017).  Causal inference through the method of direct estimation.  https://csdp.princeton.edu/publications/causal-inference-through-method-direct-estimation.
  * Robins and Rotnitzky, (1995) Robins, J. M. and Rotnitzky, A. (1995).  Semiparametric efficiency in multivariate regression models with missing data.  Journal of the American Statistical Association, 90(429):122–129.
  * Robins et al., (1994) Robins, J. M., Rotnitzky, A., and Zhao, L. P. (1994).  Estimation of regression coefficients when some regressors are not always observed.  Journal of the American Statistical Association, 89(427):846–866.
  * Rosenbaum and Rubin, (1983) Rosenbaum, P. R. and Rubin, D. B. (1983).  The central role of the propensity score in observational studies for causal effects.  Biometrika, 70(1):41–55.
  * Rotnitzky et al., (2012) Rotnitzky, A., Lei, Q., Sued, M., and Robins, J. M. (2012).  Improved double-robust estimation in missing data and causal inference models.  Biometrika, 99(2):439–456.
  * Rubin, (1974) Rubin, D. B. (1974).  Estimating causal effects of treatments in randomized and nonrandomized studies.  Journal of Educational Psychology, 66(5):688–701.
  * Schölkopf and Smola, (2002) Schölkopf, B. and Smola, A. J. (2002).  Learning with kernels: Support vector machines, regularization, optimization, and beyond.  MIT Press.
  * Setoguchi et al., (2008) Setoguchi, S., Schneeweiss, S., Brookhart, M. A., Glynn, R. J., and Cook, E. F. (2008).  Evaluating uses of data mining techniques in propensity score estimation: A simulation study.  Pharmacoepidemiology and Drug Safety, 17(6):546–555.
  * Shortreed and Ertefaie, (2017) Shortreed, S. M. and Ertefaie, A. (2017).  Outcome-adaptive lasso: Variable selection for causal inference.  Biometrics, 73(4):1111–1122.
  * Splawa-Neyman et al., (1990) Splawa-Neyman, J., Dabrowska, D. M., and Speed, T. (1990).  On the application of probability theory to agricultural experiments. Essay on principles. Section 9.  Statistical Science, 5(4):465–472.
  * Stuart, (2010) Stuart, E. A. (2010).  Matching methods for causal inference: A review and a look forward.  Statistical Science, 25(1):1–21.
  * Tan, (2010) Tan, Z. (2010).  Bounded, efficient and doubly robust estimation with inverse weighting.  Biometrika, 97(3):661–682.
  * Tarr and Imai, (2021) Tarr, A. and Imai, K. (2021).  Estimating average treatment effects with support vector machines.  arXiv preprint arXiv:2102.11926v2.
  * van der Laan and Rubin, (2006) van der Laan, M. J. and Rubin, D. (2006).  Targeted maximum likelihood learning.  The International Journal of Biostatistics, 2(1):1–38.
  * Wainwright, (2019) Wainwright, M. J. (2019).  High-dimensional statistics: A non-asymptotic viewpoint.  Cambridge University Press, first edition.
  * Wang and Zubizarreta, (2020) Wang, Y. and Zubizarreta, J. R. (2020).  Minimal dispersion approximately balancing weights: Asymptotic properties and practical considerations.  Biometrika, 107(1):93–105.
  * White, (1980) White, H. (1980).  A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity.  Econometrica, 48(4):817–838.
  * Wong and Chan, (2018) Wong, R. K. W. and Chan, K. C. G. (2018).  Kernel-based covariate functional balancing for observational studies.  Biometrika, 105(1):199–213.
  * Xu, (2017) Xu, Y. (2017).  Generalized synthetic control method: Causal inference with interactive fixed effects models.  Political Analysis, 25(1):57–76.
  * Zhao, (2019) Zhao, Q. (2019).  Covariate balancing propensity score by tailored loss functions.  The Annals of Statistics, 47(2):965–993.
  * Zhao and Percival, (2017) Zhao, Q. and Percival, D. (2017).  Entropy balancing is doubly robust.  Journal of Causal Inference, 5(1):20160010.
  * Zubizarreta, (2015) Zubizarreta, J. R. (2015).  Stable weights that balance covariates for estimation with incomplete outcome data.  Journal of the American Statistical Association, 110(511):910–922.

##  Appendix A Estimating the ATC and the ATE

Although this paper describes TFI and TFB in the context of estimating the
ATT, they are both easily extended to estimate the ATC and the ATE. This
appendix details how to do so.

###  A.1 Weighted difference in mean estimators for the ATC and the ATE

I first define the weighted difference in means estimators that will be used
to estimate the ATC and the ATE.

#### ATC

A weighted difference in means estimator for the ATC takes the form

$\displaystyle\hat{\tau}_{\mathrm{wdim}}^{(\mathrm{ATC})}=\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}Y_{i}-\frac{1}{n_{c}}\sum_{i:D_{i}=0}Y_{i}$
(43)

where all $w_{i}\geq 0$ and, often, $\sum_{D_{i}=1}w_{i}=n_{t}$.

#### ATE

A weighted difference in means estimator for the ATE takes the form

$\displaystyle\hat{\tau}_{\mathrm{wdim}}^{(\mathrm{ATE})}=\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}Y_{i}-\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}Y_{i}$
(44)

where all $w_{i}\geq 0$ and, often, $\sum_{D_{i}=0}w_{i}=n_{c}$ and
$\sum_{D_{i}=1}w_{i}=n_{t}$.

###  A.2 TFI for the ATC and the ATE

This section extends TFI to weighted difference in means estimators of the ATC
and the ATE. Like when estimating the ATT, these extensions of TFI are
approximations of the bias of $\hat{\tau}_{\mathrm{wdim}}^{(\mathrm{ATC})}$
and $\hat{\tau}_{\mathrm{wdim}}^{(\mathrm{ATE})}$ in estimating the ATC and
ATE, respectively.

First, however, we require extra notation. Define

$\displaystyle\mathrm{imbal}^{(\mathrm{ATC})}(w,g(X),D)=\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}g(X_{i})-\frac{1}{n_{c}}\sum_{i:D_{i}=0}g(X_{i})$
(45)

where $g$ is an arbitrary vector-valued function. In words,
$\mathrm{imbal}^{(\mathrm{ATC})}(w,g(X),D)$ is the imbalance in $g(X_{i})$
between the unweighted control group and the weighted treated group. Then,
define

$\displaystyle\mathrm{imbal}^{(\mathrm{ATE},0)}(w,g(X),D)=\frac{1}{n}\sum_{i=1}^{n}g(X_{i})-\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}g(X_{i})$
and
$\displaystyle\mathrm{imbal}^{(\mathrm{ATE},1)}(w,g(X),D)=\frac{1}{n}\sum_{i=1}^{n}g(X_{i})-\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}g(X_{i})$
(46)

In words, $\mathrm{imbal}^{(\mathrm{ATE},d)}(w,g(X),D)$ is the imbalance in
$g(X_{i})$ between the full sample and the weighted group with treatment
status $d\in\\{0,1\\}$. Finally, let $\hat{\beta}_{d}$ for $d\in\\{0,1\\}$ be
the coefficients from regression estimators
$\hat{f}_{d}(X_{i})=X_{i}^{\top}\hat{\beta}_{d}$, and let
$\hat{V}_{\hat{\beta}_{d}}$ be estimates of their variances.

#### ATC

TFI for a $\hat{\tau}_{\mathrm{wdim}}^{(\mathrm{ATC})}$ is defined as

$\displaystyle\mathrm{TFI}^{(\mathrm{ATC})}(w,X,D,\hat{\beta}_{1},\hat{V}_{\beta_{1}},q)=$
$\displaystyle\ \ \ \ \ \ \ \ \ \ \ \ \ \
\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\times||\hat{V}_{\beta_{1}}^{\frac{1}{2}}\mathrm{imbal}^{(\mathrm{ATC})}(w,X,D)||_{2}+|\mathrm{imbal}^{(\mathrm{ATC})}(w,X,D)^{\top}\hat{\beta}_{1}|$
(47)

As before, $q$ is a probability threshold chosen by the researcher.

#### ATE

Define TFI for a $\hat{\tau}_{\mathrm{wdim}}^{(\mathrm{ATE})}$ as

$\displaystyle\mathrm{TFI}^{(\mathrm{ATE})}(w,X,D,\hat{\beta}_{0},\hat{V}_{\beta_{0}},\hat{\beta}_{1},\hat{V}_{\beta_{1}},q)=$
$\displaystyle\ \ \ \ \ \ \ \ \ \ \ \ \ \
\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\times||\hat{V}_{\beta_{0}}^{\frac{1}{2}}\mathrm{imbal}^{(\mathrm{ATE,0})}(w,X,D)||_{2}+|\mathrm{imbal}^{(\mathrm{ATE,0})}(w,X,D)^{\top}\hat{\beta}_{0}|\
+$ $\displaystyle\ \ \ \ \ \ \ \ \ \ \ \ \ \
\sqrt{Q_{q}(\mathcal{X}^{2}_{P})}\times||\hat{V}_{\beta_{1}}^{\frac{1}{2}}\mathrm{imbal}^{(\mathrm{ATE,1})}(w,X,D)||_{2}+|\mathrm{imbal}^{(\mathrm{ATE,1})}(w,X,D)^{\top}\hat{\beta}_{1}|$
(48)

Again, $q$ is a probability threshold chosen by the user.

###  A.3 TFB for the ATC and the ATE

This section details how TFB finds weights to estimate the ATC and the ATE.

#### ATC

Similar to when estimating the ATT, TFB chooses weights to estimate the ATC
that minimize the corresponding TFI subject to constraints on the weights’
variance. In other words,

$\displaystyle\underset{w}{\text{argmin}}\biggr{[}[\mathrm{TFI}^{(\mathrm{ATC})}(w,X,D,\hat{\beta}_{1},\hat{V}_{\beta_{1}},q)]^{2}+\frac{\hat{\sigma}_{1}^{2}}{n_{t}^{2}}\sum_{i:D_{i}=1}w_{i}^{2}\biggr{]}$
where $\displaystyle w_{i}\geq 0\ \text{and}\ \sum_{i:D_{i}=1}w_{i}=n_{t}$
(49)

where
$\hat{\sigma}_{1}^{2}=\frac{1}{n_{t}}\sum_{i:D_{i}=1}(Y_{i}-X_{i}^{\top}\hat{\beta}_{1})^{2}$.

#### ATE

To estimate the ATE, TFB chooses

$\displaystyle\underset{w}{\text{argmin}}\biggr{[}[\mathrm{TFI}^{(\mathrm{ATE})}(w,X,D,\hat{\beta}_{0},\hat{V}_{\beta_{0}},\hat{\beta}_{1},\hat{V}_{\beta_{1}},q)]^{2}+\frac{\hat{\sigma}_{0}^{2}}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}+\frac{\hat{\sigma}_{1}^{2}}{n_{t}^{2}}\sum_{i:D_{i}=1}w_{i}^{2}\biggr{]}$
where $\displaystyle w_{i}\geq 0,\ \sum_{i:D_{i}=0}w_{i}=n_{c},\ \text{and}\
\sum_{i:D_{i}=1}w_{i}=n_{t}$ (50)

where
$\hat{\sigma}_{0}^{2}=\frac{1}{n_{c}}\sum_{i:D_{i}=0}(Y_{i}-X_{i}^{\top}\hat{\beta}_{0})^{2}$
and
$\hat{\sigma}_{1}^{2}=\frac{1}{n_{t}}\sum_{i:D_{i}=1}(Y_{i}-X_{i}^{\top}\hat{\beta}_{1})^{2}$.

###  A.4 Asymptotics for the ATC and the ATE

Finally, this section details how to estimate the variance of
$\hat{\tau}^{(\mathrm{ATC})}_{\mathrm{wdim}}$ and
$\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}$.

#### ATC

Theorem 3 describes sufficient conditions to determine the asymptotic
normality of a general $\hat{\tau}^{(\mathrm{ATC})}_{\mathrm{wdim}}$. Then,
Theorem 4 introduces an estimator of the asymptotic variance of
$\hat{\tau}^{(\mathrm{ATC})}_{\mathrm{wdim}}$ and describes sufficient
conditions for the estimator’s consistency.

######  Theorem 3 (Conditions for asymptotic normality of a general
$\hat{\tau}^{(\mathrm{ATC})}_{\mathrm{wdim}}$)

Given Assumption 1, further assume the following conditions: (i) the $w_{i}$
are “honest”, or $[w_{n_{c}+1}\ \cdots\
w_{n}]=w_{\mathrm{vec}}^{(\mathrm{ATC})}(X,D)$ for a function
$w_{\mathrm{vec}}^{(\mathrm{ATC})}(\cdot)$; (ii)
$\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}\sigma_{i}^{2}<\infty$; (iii)
$\exists\underline{C},\overline{C}>0$ such that
$\underline{C}\leq\sigma_{i}^{2}\leq\overline{C}$; (iv)
$\mathbbm{E}([f_{1}(X_{i})-Y_{i}(0)-\mathrm{ATC}]^{2}\ |\ D_{i}=0)<\infty$;
(v) $\exists\delta>0$ such that

$\displaystyle\frac{\sum_{i:D_{i}=1}w_{i}^{2+\delta}}{(\sum_{i:D_{i}=1}w_{i}^{2})^{\frac{2+\delta}{2}}}\overset{a.s.}{\rightarrow}0\
\ \ \text{and}\ \ \ \mathbbm{E}(|\epsilon_{i}|^{2+\delta}\ |\
X)\leq\overline{C}_{\delta}\ \ \ \text{for \ some}\ \ \
\overline{C}_{\delta}>0\text{;}$

and (vi) $\mathrm{imbal}^{(\mathrm{ATC})}(w,f_{1}(X),D)=o_{p}(n^{-1/2})$.
Then,
$V_{\mathrm{ATC}}^{-\frac{1}{2}}(\hat{\tau}^{(\mathrm{ATC})}_{\mathrm{wdim}}-\mathrm{ATC})\rightarrow
N(0,1)$ where

$\displaystyle
V_{\mathrm{ATC}}=\frac{1}{n_{c}}\mathbbm{E}\biggr{(}[f_{1}(X_{i})-Y_{i}(0)-\mathrm{ATC}]^{2}\
\biggr{|}\
D_{i}=0\biggr{)}+\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}w_{i}^{2}\sigma_{i}^{2}$

######  Theorem 4 (Estimating $V_{\mathrm{ATC}}$)

Given Assumption 1 and conditions (i)-(vi) for Theorem 3, further assume that:
(vii)
$\frac{1}{n}\sum_{i=1}^{n}[f_{1}(X_{i})-\hat{f}_{1}(X_{i})]^{2}=o_{p}(1)$;
(viii)
$\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}\epsilon_{i}^{2}\overset{p}{\rightarrow}\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}\sigma_{i}^{2}$;
and (ix)
$\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}[f_{1}(X_{i})-\hat{f}_{1}(X_{i})]^{2}=o_{p}(1)$.
Then, $\hat{V}_{\mathrm{ATC}}\overset{p}{\rightarrow}V_{\mathrm{ATC}}$ where

$\displaystyle\hat{V}_{\mathrm{ATC}}=\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}[\hat{f}_{1}(X_{i})-Y_{i}(0)-\hat{\tau}^{(\mathrm{ATC})}_{\mathrm{wdim}}]^{2}+\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}w_{i}^{2}[Y_{i}(1)-\hat{f}_{1}(X_{i})]^{2}$

The proofs for Theorems 3 and 4 are omitted here because the ATC is simply the
negative of the ATT for the reversed treatment, $\tilde{D}_{i}=1-D_{i}$, and
potential outcomes, $\tilde{Y}_{i}(\tilde{d})=Y_{i}(1-\tilde{d})$ (i.e.,
$\mathbbm{E}[Y_{i}(1)-Y_{i}(0)\ |\
D_{i}=0]=-\mathbbm{E}[\tilde{Y}_{i}(1)-\tilde{Y}_{i}(0)\ |\
\tilde{D}_{i}=1])$. Therefore, the proofs for Theorems 3 and 4 follow nearly
identical steps to those for Theorems 1 and 2 (in Appendices B.1 and B.2),
respectively.

#### ATE

Theorem 5 describes sufficient conditions to determine the asymptotic
normality of a general $\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}$. Then,
Theorem 6 introduces an estimator of the asymptotic variance of
$\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}$ and describes sufficient
conditions for the estimator’s consistency.

######  Theorem 5 (Conditions for asymptotic normality of a general
$\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}$)

Given Assumption 1, further assume the following conditions: (i) the $w_{i}$
are “honest”, or $[w_{1}\ \cdots\
w_{n}]=w_{\mathrm{vec}}^{(\mathrm{ATE})}(X,D)$ for a function
$w_{\mathrm{vec}}^{(\mathrm{ATE})}(\cdot)$; (ii)
$\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}<\infty$ and
$\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}\sigma_{i}^{2}<\infty$; (iii)
$\exists\underline{C},\overline{C}>0$ such that
$\underline{C}\leq\sigma_{i}^{2}\leq\overline{C}$; (iv)
$\mathbbm{E}([f_{1}(X_{i})-f_{0}(X_{i})-\mathrm{ATE}]^{2})<\infty$; (v)
$\exists\delta>0$ such that

$\displaystyle\frac{\sum_{i:D_{i}=0}w_{i}^{2+\delta}}{(\sum_{i:D_{i}=0}w_{i}^{2})^{\frac{2+\delta}{2}}}\overset{a.s.}{\rightarrow}0,\
\
\frac{\sum_{i:D_{i}=1}w_{i}^{2+\delta}}{(\sum_{i:D_{i}=1}w_{i}^{2})^{\frac{2+\delta}{2}}}\overset{a.s.}{\rightarrow}0,\
\ \text{and}\ \ \mathbbm{E}(|\epsilon_{i}|^{2+\delta}\ |\
X)\leq\overline{C}_{\delta}\ \ \text{for some}\ \
\overline{C}_{\delta}>0\text{;}$

and (vi) $\mathrm{imbal}^{(\mathrm{ATE,0})}(w,f_{0}(X),D)=o_{p}(n^{-1/2})$ and
$\mathrm{imbal}^{(\mathrm{ATE,1})}(w,f_{1}(X),D)=o_{p}(n^{-1/2})$. Then,
$V_{\mathrm{ATE}}^{-\frac{1}{2}}(\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}-\mathrm{ATE})\rightarrow
N(0,1)$ where

$\displaystyle
V_{\mathrm{ATE}}=\frac{1}{n}\mathbbm{E}\biggr{(}[f_{1}(X_{i})-f_{0}(X_{i})-\mathrm{ATE}]^{2}\biggr{)}+\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}+\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}w_{i}^{2}\sigma_{i}^{2}$

######  Theorem 6 (Estimating $V_{\mathrm{ATE}}$)

Given Assumption 1 and conditions (i)-(vi) for Theorem 5, further assume that:
(vii)
$\frac{1}{n}\sum_{i=1}^{n}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]^{2}=o_{p}(1)$ and
$\frac{1}{n}\sum_{i=1}^{n}[f_{1}(X_{i})-\hat{f}_{1}(X_{i})]^{2}=o_{p}(1)$;
(viii)
$\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\epsilon_{i}^{2}\overset{p}{\rightarrow}\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$
and
$\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}\epsilon_{i}^{2}\overset{p}{\rightarrow}\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}\sigma_{i}^{2}$;
and (ix)
$\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]^{2}=o_{p}(1)$
and
$\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}^{2}[f_{1}(X_{i})-\hat{f}_{1}(X_{i})]^{2}=o_{p}(1)$.
Then, $\hat{V}_{\mathrm{ATE}}\overset{p}{\rightarrow}V_{\mathrm{ATE}}$ where

$\displaystyle\hat{V}_{\mathrm{ATE}}$
$\displaystyle=\frac{1}{n^{2}}\sum_{i=1}^{n}[\hat{f}_{1}(X_{i})-\hat{f}_{0}(X_{i})-\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}]^{2}$
$\displaystyle+\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}[Y_{i}(0)-\hat{f}_{0}(X_{i})]^{2}$
$\displaystyle+\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}w_{i}^{2}[Y_{i}(1)-\hat{f}_{1}(X_{i})]^{2}$

Proof for Theorem 5 is given in Appendix B.3 and proof for Theorem 6 is given
in Appendix B.4.

##  Appendix B Proofs

###  B.1 Proof of Theorem 1

$\hat{\tau}_{\mathrm{wdim}}-\mathrm{ATT}$ can be written as the sum of three
terms that I analyze separately:

$\displaystyle\hat{\tau}_{\mathrm{wdim}}-\mathrm{ATT}=\underbrace{\biggr{(}\frac{1}{n_{t}}\sum_{i:D_{i}=1}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]\biggr{)}}_{(a)}\
-\ \underbrace{\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}\epsilon_{i}}_{(b)}\ +\
\text{EWC Bias}$ (51)

For $(a)$, first notice that

$\displaystyle\mathbbm{E}(\epsilon_{i}\ |\ D_{i}=1)$
$\displaystyle=\mathbbm{E}\biggr{(}\mathbbm{E}(\epsilon_{i}\ |\
D_{i}=1,X_{i})\ \biggr{|}\ D_{i}=1\biggr{)}$
$\displaystyle=\mathbbm{E}\biggr{(}\mathbbm{E}(\epsilon_{i}\ |\ X_{i})\
\biggr{|}\ D_{i}=1\biggr{)}$ $\displaystyle=0$ (52)

where Assumption 1 allows the second line in (B.1) above. Therefore,

$\displaystyle\mathbbm{E}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}\ |\ D_{i}=1]$
$\displaystyle=\mathbbm{E}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}\ |\
D_{i}=1]-\mathbbm{E}(\epsilon_{i}\ |\ D_{i}=1)$
$\displaystyle=\mathbbm{E}[Y_{i}(1)-Y_{i}(0)-\mathrm{ATT}\ |\ D_{i}=1]$
$\displaystyle=\mathbbm{E}[Y_{i}(1)-Y_{i}(0)\ |\ D_{i}=1]-\mathrm{ATT}$
$\displaystyle=0$ (53)

Thus, the Central Limit Theorem applies to $(a)$ by (B.1) and iid data,
meaning that

$\displaystyle\sqrt{n_{t}}\times(a)\overset{d}{\rightarrow}N\biggr{(}0,\mathbbm{E}\biggr{[}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
\biggr{|}\ D_{i}=1\biggr{]}\biggr{)}$ (54)

where $\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\ |\
D_{i}=1)<\infty$ by condition (iv) for the theorem. Some manipulation then
gives that

$\displaystyle\sqrt{n}\times(a)\overset{d}{\rightarrow}N\biggr{(}0,\frac{\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
|\ D_{i}=1)}{p(D_{i}=1)}\biggr{)}$ (55)

Moving to $(b)$, I first consider its distribution given $X$ and $D$. To
begin, under Assumption 1, iid data, and condition (i) for the theorem,

$\displaystyle\operatorname{var}\biggr{(}(b)\ \biggr{|}\
X,D\biggr{)}=\operatorname{var}(\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}\epsilon_{i}\
|\ X,D)=\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$ (56)

Additionally, under conditions (iii) and (v) for the theorem,
$\exists\delta>0$ such that

$\displaystyle\frac{\sum_{i:D_{i}=0}\mathbbm{E}(|\frac{w_{i}}{n_{c}}\epsilon_{i}|^{2+\delta}\
|\ X,D)}{\operatorname{var}(\sum_{i:D_{i}=0}\frac{w_{i}}{n_{c}}\epsilon_{i}\
|\ X,D)^{\frac{2+\delta}{2}}}\rightarrow 0$ (57)

To see this, Assumption 1, iid data, and conditions (i) and (iii) give

$\displaystyle\frac{\sum_{i:D_{i}=0}\mathbbm{E}(|\frac{w_{i}}{n_{c}}\epsilon_{i}|^{2+\delta}\
|\ X,D)}{\operatorname{var}(\sum_{i:D_{i}=0}\frac{w_{i}}{n_{c}}\epsilon_{i}\
|\ X,D)^{\frac{2+\delta}{2}}}$
$\displaystyle=\frac{\sum_{i:D_{i}=0}\mathbbm{E}(|w_{i}\epsilon_{i}|^{2+\delta}\
|\ X,D)}{\operatorname{var}(\sum_{i:D_{i}=0}w_{i}\epsilon_{i}\ |\
X,D)^{\frac{2+\delta}{2}}}$
$\displaystyle=\frac{\sum_{i:D_{i}=0}w_{i}^{2+\delta}\mathbbm{E}(|\epsilon_{i}|^{2+\delta}\
|\ X)}{(\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2})^{\frac{2+\delta}{2}}}$
$\displaystyle\leq\biggr{(}\frac{1}{\underline{C}^{\frac{2+\delta}{2}}}\biggr{)}\biggr{(}\frac{\sum_{i:D_{i}=0}w_{i}^{2+\delta}\mathbbm{E}(|\epsilon_{i}|^{2+\delta}\
|\ X)}{(\sum_{i:D_{i}=0}w_{i}^{2})^{\frac{2+\delta}{2}}}\biggr{)}$ (58)

Then, by condition (v) for the theorem,
$\exists\delta,\overline{C}_{\delta}>0$ that allow (B.1) to continue as

$\displaystyle\frac{\sum_{i:D_{i}=0}\mathbbm{E}(|\frac{w_{i}}{n_{c}}\epsilon_{i}|^{2+\delta}\
|\ X,D)}{\operatorname{var}(\sum_{i:D_{i}=0}\frac{w_{i}}{n_{c}}\epsilon_{i}\
|\
X,D)^{\frac{2+\delta}{2}}}\leq\biggr{(}\frac{\overline{C}_{\delta}}{\underline{C}^{\frac{2+\delta}{2}}}\biggr{)}\biggr{(}\frac{\sum_{i:D_{i}=0}w_{i}^{2+\delta}}{(\sum_{i:D_{i}=0}w_{i}^{2})^{\frac{2+\delta}{2}}}\biggr{)}\rightarrow
0$ (59)

Therefore, $(b)$ satisfies the Lyapunov condition conditional on $X$ and $D$,
meaning that, by the Lindberg-Feller theorem,

$\displaystyle\frac{(b)}{\sqrt{\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}}\
\biggr{|}\ X,D\overset{d}{\rightarrow}N(0,1)$ (60)

which also implies here that

$\displaystyle\frac{(b)}{\sqrt{\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}}\overset{d}{\rightarrow}N(0,1)$
(61)

or, after some manipulation, that

$\displaystyle\sqrt{n}\times(b)\overset{d}{\rightarrow}N\biggr{(}0,\
\frac{\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}{p(D_{i}=0)}\biggr{)}$
(62)

where $\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}<\infty$ by condition
(ii).

Putting these pieces together, $\sqrt{n}\times(a)$ and $\sqrt{n}\times(b)$ are
asymptotically independent because they are independent given $X$ and $D$ and
(60) holds given $X$ and $D$. Therefore, by (51), (55), and (62)

$\displaystyle\sqrt{n}(\hat{\tau}_{\mathrm{wdim}}-\mathrm{ATT}-\mathrm{EWC\
Bias})$ $\displaystyle\ \ \ \ \ =\sqrt{n}\biggr{(}(a)-(b)\biggr{)}$
$\displaystyle\ \ \ \ \
\overset{d}{\rightarrow}N\biggr{(}0,\frac{\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
|\ D_{i}=1)}{p(D_{i}=1)}+\frac{\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}{p(D_{i}=0)}\biggr{)}$
(63)

Further, $\sqrt{n}\times\mathrm{EWC\ Bias}\overset{p}{\rightarrow}0$ by
condition (vi) for the theorem. Thus, (B.1) can be rewritten as

$\displaystyle\sqrt{n}(\hat{\tau}_{\mathrm{wdim}}-\mathrm{ATT})$
$\displaystyle\ \ \ \ \
\overset{d}{\rightarrow}N\biggr{(}0,\frac{\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
|\ D_{i}=1)}{p(D_{i}=1)}+\frac{\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}{p(D_{i}=0)}\biggr{)}$
(64)

Finally, examining $V_{\mathrm{ATT}}$, it is then evident that

$\displaystyle n\times V_{\mathrm{ATT}}$
$\displaystyle=\frac{\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\ |\
D_{i}=1)}{\frac{n_{t}}{n}}+\frac{\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}{\frac{n_{c}}{n}}$
$\displaystyle\overset{p}{\rightarrow}\frac{\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
|\ D_{i}=1)}{p(D_{i}=1)}+\frac{\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}{p(D_{i}=0)}$ (65)

Thus, applying (B.1) to (B.1) completes the proof.

$\square$

###  B.2 Proof of Theorem 2

$\hat{V}_{\mathrm{ATT}}$ can be rewritten as:

$\displaystyle\hat{V}_{\mathrm{ATT}}$
$\displaystyle=\underbrace{\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}\biggr{(}[Y_{i}-f_{0}(X_{i})-\mathrm{ATT}]+[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]+[\mathrm{ATT}-\hat{\tau}_{\mathrm{wdim}}]\biggr{)}^{2}}_{(a)}$
$\displaystyle+\underbrace{\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\biggr{(}\epsilon_{i}+[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]\biggr{)}^{2}}_{(b)}$
(66)

I prove the theorem by showing that
$(a)\overset{p}{\rightarrow}\frac{1}{n_{t}}\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
|\ D_{i}=1)$ and that
$(b)\overset{p}{\rightarrow}\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$.

Starting with $(a)$,

$\displaystyle(a)$
$\displaystyle=\underbrace{\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}[Y_{i}-f_{0}(X_{i})-\mathrm{ATT}]^{2}}_{(a.1)}$
$\displaystyle+\underbrace{\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]^{2}}_{(a.2)}$
$\displaystyle+\underbrace{\frac{1}{n_{t}^{2}}\sum_{i:D_{i}=1}[\mathrm{ATT}-\hat{\tau}_{\mathrm{wdim}}]^{2}}_{(a.3)}$
$\displaystyle+\underbrace{\frac{2}{n_{t}^{2}}\sum_{i:D_{i}=1}[Y_{i}-f_{0}(X_{i})-\mathrm{ATT}][f_{0}(X_{i})-\hat{f}_{0}(X_{i})]}_{(a.4)}$
$\displaystyle+\underbrace{\frac{2}{n_{t}^{2}}\sum_{i:D_{i}=1}[Y_{i}-f_{0}(X_{i})-\mathrm{ATT}][\mathrm{ATT}-\hat{\tau}_{\mathrm{wdim}}]}_{(a.5)}$
$\displaystyle+\underbrace{\frac{2}{n_{t}^{2}}\sum_{i:D_{i}=1}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})][\mathrm{ATT}-\hat{\tau}_{\mathrm{wdim}}]}_{(a.6)}$
(67)

Given iid data, and thus by the Law of Large Numbers,

$\displaystyle(a.1)\overset{p}{\rightarrow}\frac{1}{n_{t}}\mathbbm{E}\biggr{[}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
\biggr{|}\ D_{i}=1\biggr{]}$ (68)

where $\mathbbm{E}([Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\ |\
D_{i}=1)<\infty$ by condition (iv) for Theorem 1. By condition (vii) of the
theorem,

$\displaystyle(a.2)=o_{p}(n^{-1})$ (69)

and, by the Cauchy-Schwartz Inequality, along with (68) and (69),

$\displaystyle|(a.4)|\leq
2\sqrt{(a.1)}\sqrt{(a.2)}=2\sqrt{O_{p}(n^{-1})}\sqrt{o_{p}(n^{-1})}=o_{p}(n^{-1})$
(70)

Then, under the conditions for Theorem 1,
$\hat{\tau}_{\mathrm{wdim}}\overset{p}{\rightarrow}\mathrm{ATT}$, so

$\displaystyle(a.3)=\frac{1}{n_{t}}(\mathrm{ATT}-\hat{\tau}_{\mathrm{wdim}})^{2}=o_{p}(n^{-1})$
(71)

and, by iid data, and thus by the Law of Large Numbers,

$\displaystyle(a.5)\overset{p}{\rightarrow}2\biggr{(}\frac{\mathrm{ATT}-\hat{\tau}_{\mathrm{wdim}}}{n_{t}}\biggr{)}\mathbbm{E}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}\
|\ D_{i}=1]=o_{p}(n^{-1})$ (72)
$\displaystyle(a.6)\overset{p}{\rightarrow}2\biggr{(}\frac{\mathrm{ATT}-\hat{\tau}_{\mathrm{wdim}}}{n_{t}}\biggr{)}\mathbbm{E}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})\
|\ D_{i}=1]=o_{p}(n^{-1})$ (73)

Therefore, applying (68), (69), (70), (71), (72), and (73) to (B.2) yields

$\displaystyle(a)\overset{p}{\rightarrow}\frac{1}{n_{t}}\mathbbm{E}\biggr{[}[Y_{i}(1)-f_{0}(X_{i})-\mathrm{ATT}]^{2}\
\biggr{|}\ D_{i}=1\biggr{]}$ (74)

Moving to $(b)$ in (B.2),

$\displaystyle(b)$
$\displaystyle=\underbrace{\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\epsilon_{i}^{2}}_{(b.1)}$
$\displaystyle+\underbrace{\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]^{2}}_{(b.2)}$
$\displaystyle+\underbrace{\frac{2}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\epsilon_{i}[f_{0}(X_{i})-\hat{f}_{0}(X_{i})]}_{(b.3)}$
(75)

By condition (viii) for the theorem,

$\displaystyle(b.1)\overset{p}{\rightarrow}\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$
(76)

Then, by condition (ix) for the theorem,

$\displaystyle(b.2)=o_{p}(n^{-1})$ (77)

and using the Cauchy-Schwartz Inequality and (76),

$\displaystyle|(b.3)|$ $\displaystyle\leq 2\sqrt{(b.1)}\sqrt{(b.2)}$
$\displaystyle\overset{p}{\rightarrow}2\sqrt{\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}}\sqrt{(b.2)}$
(78)

Then, because $\underset{n\rightarrow\infty}{\mathrm{plim}}\
\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$ is finite by condition
(ii) for Theorem 1, we find that
$\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$ in (B.2) is
$O_{p}(n^{-1})$. Additionally using (77) allows (B.2) to continue as

$\displaystyle|(b.3)|\leq
2\sqrt{O_{p}(n^{-1})}\sqrt{o_{p}(n^{-1})}=o_{p}(n^{-1})$ (79)

Therefore, applying (76), (77), and (79) to (B.2) gives

$\displaystyle(b)\overset{p}{\rightarrow}\frac{1}{n_{c}^{2}}\sum_{i:D_{i}=0}w_{i}^{2}\sigma_{i}^{2}$
(80)

Finally, applying (74) and (80) to (B.2) completes the proof.

$\square$

###  B.3 Proof of Theorem 5

$\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}-\mathrm{ATE}$ can be written as
the sum of four terms that I analyze separately:

$\displaystyle\hat{\tau}^{(\mathrm{ATE})}_{\mathrm{wdim}}-\mathrm{ATE}$
$\displaystyle=\underbrace{\biggr{(}\frac{1}{n}\sum_{i=1}^{n}[f_{1}(X_{i})-f_{0}(X_{i})-\mathrm{ATE}]\biggr{)}}_{(a)}$
$\displaystyle-\underbrace{\frac{1}{n_{c}}\sum_{i:D_{i}=0}w_{i}\epsilon_{i}}_{(b)}$
$\displaystyle+\underbrace{\frac{1}{n_{t}}\sum_{i:D_{i}=1}w_{i}\epsilon_{i}}_{(c)}$
$\displaystyle+\underbrace{\biggr{(}\mathrm{imbal}^{(\mathrm{ATE,0})}(w,f_{0}(X),D)-\mathrm{imbal}^{(\mathrm{ATE,1})}(w,f_{1}(X),D)\biggr{)}}_{(d)}$
(81)

For $(a)$, first notice that

$\displaystyle\mathbbm{E}(\epsilon_{i})$
$\displaystyle=\mathbbm{E}[\mathbbm{E}(\epsilon_{i}\ |\ X_{i})]=0$ (82)

Therefore,

$\displaystyle\mathbbm{E}[f_{1}(X_{i})-f_{0}(X_{i})-\mathrm{ATE}]$
$\displaystyle=\mathbbm{E}[f_{1}(X_{i})-f_{0}(X_{i})-\mathrm{ATE}]+\mathbbm{E}(\epsilon_{i})-\mathbbm{E}(\epsilon_{i})$
$\displaystyle=\mathbbm{E}[Y_{i}(1)-Y_{i}(0)-\mathrm{ATE}]$
$\displaystyle=\mathbbm{E}[Y_{i}(1)-Y_{i}(0)]-\mathrm{ATE}$ $\displaystyle=0$
(83)

Thus, the Central Limit Theorem applies to $(a)$ by (B.3) and iid data,
meaning that

$\displaystyle\sqrt{n}\times(a)\overset{d}{\rightarrow}N\biggr{(}0,\mathbbm{E}\biggr{[}[f_{1}(X_{i})-f_{0}(X_{i})-\mathrm{ATE}]^{2}\biggr{]}\biggr{)}$
(84)

where $\mathbbm{E}([f_{1}(X_{i})-f_{0}(X_{i})-\mathrm{ATE}]^{2})<\infty$ by
condition (iv) for the theorem.