# Notes on Causation, Comparison, and Regression††thanks: We thank Isaiah
Andrews, Ronald Kessler, Russell Localio, Joseph Newhouse, Bijan Niknam, and
Paul Rosenbaum for helpful comments and conversations. For graphics, we thank
Xavier Alemañy. This work was supported by the Alfred P. Sloan Foundation
(G-2020-13946) and the Patient Centered Outcomes Research Initiative (PCORI,
ME-2022C1-25648).

Ambarish Chattopadhyay Stanford Data Science, Stanford University, 450 Jane
Stanford Way Wallenberg, Stanford, CA 94305; email<EMAIL_ADDRESS>José R. Zubizarreta Departments of Health Care Policy, Biostatistics, and
Statistics, Harvard University, 180 Longwood Avenue, Office 307-D, Boston, MA
02115; email<EMAIL_ADDRESS>
###### Abstract

Comparison and contrast are the basic means to unveil causation and learn
which treatments work. To build good comparisons that isolate the average
effect of treatment from confounding factors, randomization is key, yet often
infeasible. In such non-experimental settings, we illustrate and discuss how
well the common linear regression approach to causal inference approximates
features of randomized experiments, such as covariate balance, study
representativeness, sample-grounded estimation, and unweighted analyses. We
also discuss alternative regression modeling, weighting, and matching
approaches. We argue they should be given strong consideration in empirical
work.

Keywords: Causal Inference; Randomized Experiments; Observational Studies;
Regression Modeling; Weighting Adjustments; Multivariate Matching

###### Contents

  1. 1 Unweighted comparisons work well in randomized experiments
    1. 1.1 Learning which treatments work
    2. 1.2 Randomized comparisons
    3. 1.3 Running example
    4. 1.4 Weighted contrasts
    5. 1.5 Observational studies
  2. 2 Regression models implicitly use unequal weights for estimating treatment effects
    1. 2.1 Implied adjustments
    2. 2.2 Approximating experiments
  3. 3 Regression adjustments in nonexperimental comparisons
  4. 4 A regression adjustment is implicitly a weighted comparison
  5. 5 The implied regression weights create a limited form of covariate balance
    1. 5.1 Exact covariate balance.
    2. 5.2 A hidden population.
    3. 5.3 Estimation beyond the support of the data and negative weights.
    4. 5.4 Differential weighting of minimum variance.
  6. 6 Alternative regression modeling, weighting, and matching approaches
    1. 6.1 Multi regression.
    2. 6.2 Weighting.
    3. 6.3 Matching.
  7. 7 It’s through contrast that we see

##  1 Unweighted comparisons work well in randomized experiments

###  1.1 Learning which treatments work

1.1 _Learning which treatments work._ Comparison and contrast are the basic
means to learn the effects of causes — which treatments work — in a complex
world of which we only know so much. Particularly in the medical and social
sciences, where the units in which the treatments are applied or withheld are
highly heterogeneous (humans), these comparisons must be made with special
care. To make useful contrasts and “see” the effect of treatment, it is
crucial to build similar groups of subjects, exposed and not exposed to the
treatment, in order to isolate its effect from observable and unobservable
confounding factors.

###  1.2 Randomized comparisons

1.2 _Randomized comparisons._ The randomized experiment is the method par
excellence to build such comparable groups. Randomization is impartial: it
produces groups with similar pre-treatment observed and unobserved
characteristics that differ only on the receipt of treatment, and possibly, if
the treatment is effective, on their subsequent outcomes. Randomization also
provides a factual basis to quantify whether differences in the outcomes are
due to chance or system and conduct statistical tests for the effect of
treatment (Fisher 1935). To see this, consider the following toy example
inspired by the famous RAND Health Insurance Experiment (Newhouse et al.
1993).111The data and code of this toy example are available upon request.

###  1.3 Running example

1.3 _Running example._ The goal is to learn the causal effect of a new health
insurance plan (the _treatment_ variable) on health expenditures (the
_outcome_ variable). The purpose of this new plan is to protect patients
against high out-of-pocket medical costs while reducing health spending among
those who enroll. The basic parameter that we wish to learn is the average
reduction in health expenditure caused by the insurance plan; that is the
average effect of treatment on the treated subjects, henceforth denoted ATT.

To this end, a randomized experiment is conducted on a random sample of 200
individuals from a population. Half of these individuals are randomly assigned
to enroll in the plan — the treatment group —, and the other 100 are similarly
assigned to control. For each individual, there is information on two pre-
treatment characteristics, or covariates; namely, household per-capita income
and number of hospital visits in the previous year. Figure 1 depicts this
situation for 10 typical individuals in each group.

Figure 1: Randomized experiment.

In the figure the covariate values for each individual are shown over their
left shoulder. For example, the treated individual in the top-left corner has
a household per-capita income of $37K and 1 hospital visit in the previous
year. The _diagnostic dashboard_ on the rightmost panel tabulates the means of
these two covariates in the sample of 100 treated subjects, 100 control
subjects, and the target (here, treated) population as represented by the
treated sample. It also tabulates the _effective sample size_ (i.e., the
number of subjects effectively used for estimation) and the _nominal sample
size_ (i.e., the original number of subjects) in each group.

###  1.4 Weighted contrasts

1.4 _Weighted contrasts._ After randomization, because the groups tend to have
similar observed and unobserved characteristics, we can estimate the average
effect of treatment by simply taking the mean difference in outcome values y
between the treated and the control groups,

$\overbrace{\frac{y_{t,1}+y_{t,2}+...+y_{t,10}}{10}}^{\text{Treated
group}}-\overbrace{\frac{y_{c,1}+y_{c,2}+...+y_{c,10}}{10}}^{\text{Control
group}}.$

In this expression, each observation contributes equally to the total effect
estimate, so we can say that the treated and control groups are unweighted (as
each observation has a constant weight equal to one) and equivalently write

$\overbrace{\frac{1\cdot y_{t,1}+1\cdot y_{t,2}+...+1\cdot
y_{t,10}}{10}}^{\text{Treated group}}-\overbrace{\frac{1\cdot y_{c,1}+1\cdot
y_{c,2}+...+1\cdot y_{c,10}}{10}}^{\text{Control group}}.$

In randomized experiments, without weights the treated and control groups are
balanced, so in principle, without weights, we can estimate the average effect
of treatment by contrasting the simple averages in the treated and control
groups. This observation is important as we move forward to more complex
settings and methods.

###  1.5 Observational studies

1.5 _Observational studies._ The power and appeal of randomized experiments
stem from its treatment assignment mechanism — which is controlled by the
investigator — and by its resulting data structure, which is transparent and
simple: by design, we can clearly see how randomization works, whether the
randomly formed groups are comparable or not, and we can talk about it,
fostering collaboration and the elaboration of scientific theories. However,
due to ethical or practical constraints, we cannot always carry out
experiments. In these situations, we must fall back on observational studies,
where by means of a process unbeknownst to the investigator, subjects are
selected into treatment or control, typically in an unbalanced way.

Following our running example, we now consider a parallel observational study
comprised of a sample of 300 subjects from a population where, through an
unknown process, some have enrolled in the new health insurance plan and the
rest have not. As before, the goal is to learn the average treatment effect on
the treated or ATT. In the sample, there are 100 treated subjects and 200
controls. Figure 2 summarizes this for 10 and 20 typical subjects from the
treated and control groups, respectively. The average difference in health
expenditures between the treatment and control group is roughly -$5K,
suggesting that the insurance plan reduces the health expenditure by that
amount. But how trustworthy is this conclusion?

Figure 2: Observational study, before adjustments.

As seen in the diagnostic dashboard of Figure 2, the treatment groups differ
in their observed characteristics, and likely in the unobserved ones. For
instance, the average income in the treated group is $27.2K, which is markedly
different from the $45K average income among the controls, likely implying
that the observed differences in health expenditure are driven by this pre-
treatment imbalance. Therefore, some form of adjustments must be made to make
the groups comparable, at least in terms of the observed characteristics. To
this end, the randomized experiment is the conceptual benchmark that guides
most methods of adjustment in an observational study (Dorn 1953; Cochran 1965;
Rosenbaum 2010; Imbens and Rubin 2015; Hernán and Robins 2020).

##  2 Regression models implicitly use unequal weights for estimating
treatment effects

###  2.1 Implied adjustments

2.1 _Implied adjustments._ In observational studies we seek to approximate or
emulate as closely as possible randomized experiments in order to estimate the
effect of treatments (Rubin 2007). However, observational studies often rely
on regression models to adjust for covariate imbalance and estimate such
effects. In these analyses, how well does regression mimic key features of a
randomized experiment? Specifically, how does linear regression recover (i)
covariate balance, (ii) study representativeness, (iii) sample-grounded
estimation, and (iv) unweighted samples that facilitate transparent analyses?

###  2.2 Approximating experiments

2.2 _Approximating experiments._ In this question, (i) covariate balance
refers to making the observed characteristics between the treatment and
control groups similar. The influence of likely imbalances in unobserved
characteristics must be assessed with a sensitivity analysis (e.g., Cornfield
et al. 1959; Rosenbaum 1987b, 2005; VanderWeele and Ding 2017; Zhao et al.
2019). (ii) Study representativeness is requiring that the characteristics of
each group be similar to (and hence, representative of) a target population.
(iii) Sample-grounded estimation or sample-boundedness means estimating
effects that are an interpolation of the observed data rather than an
extrapolation beyond the range or support of the actual data (Robins et al.
2007). (iv) Finally, an unweighted or self-weighted sample refers to a simple
data structure where the observations have homogeneous weights and which does
not require further adjustments that can decrease the efficiency and
transparency of the study. A recent paper provides answers to this question by
deriving and analyzing the implied weights of linear regression (Chattopadhyay
and Zubizarreta 2023).

##  3 Regression adjustments in nonexperimental comparisons

It is common to fit regression models to nonexperimental data, including
linear regression, logistic regression, and proportional hazards regression.
Consider the following linear regression model where the outcome of interest y
is regressed on observed covariates or characteristics (e.g., age, gender, and
risk factors) and the treatment assignment indicator,222The treatment
assignment indicator of a subject equals one if the subject receives treatment
and equals zero otherwise.

$\text{Outcome variable}=\alpha+\beta\cdot\text{Observed
covariates}+\tau\cdot\text{Treatment indicator}+\epsilon.$

In the model, $\varepsilon$ is a random error term with mean zero, distributed
independently of the covariates, and $\alpha$, $\beta$ and $\tau$ are the
unknown regression coefficients. Under the assumption that the observed
covariates encompass all relevant confounders, the objective again is to
estimate the parameter $\tau$ for the average treatment effect on those
treated (ATT).

Arguably, this is the traditional regression approach to causal inference. By
standard regression theory, if the assumed model is correct, then the
traditional ordinary least squares (OLS) estimator is the most efficient,
linear unbiased estimator for the ATT. But where is the experiment? Does the
above regression approach approximate the above features of randomized
experiments? And what happens if the model is used when it is incorrect?

##  4 A regression adjustment is implicitly a weighted comparison

To answer these questions, we must first ask a crucial intermediate question.
How does the above regression approach implicitly weight the individual
observations in the data at hand? Building on important work by Imbens (2015;
see also Abadie et al. 2015, Gelman and Imbens 2018), Chattopadhyay and
Zubizarreta (2023) show that the traditional regression estimator is
equivalent to the following weighted contrast

$\overbrace{\frac{w_{t,1}\cdot y_{t,1}+w_{t,2}\cdot y_{t,2}+...+w_{t,10}\cdot
y_{t,10}}{10}}^{\text{Treated group}}-\overbrace{\frac{w_{c,1}\cdot
y_{c,1}+w_{c,2}\cdot y_{c,2}+...+w_{c,10}\cdot y_{c,10}}{10}}^{\text{Control
group}}.$

where the weights $w$ in each group sum to the corresponding group size and
can be expressed in closed form; i.e., without approximations.333See Angrist
and Krueger (1999) and Aronow and Samii (2016) for related work. If, in the
control group, $w_{c,1}=2$, then the first control counts as if she were two
controls, correcting for the fact that people like this control are
underrepresented in the control group and overrepresented in the treated group
because of the absence of randomization. In other words, linear regression
implicitly creates and contrasts weighted samples of treated and control
observations, where the contribution or weight of each observation to the
aggregate average treatment effect estimate can be computed and scrutinized.
By analyzing these weights, we can transparently assess how linear regression
attempts to mimic features of randomized experiments, as we discuss next. In
Figure 3, we illustrate the structure of the weighted groups after regression
in our running example.444The implied regression weights are computed using
the R package lmw; see the associated paper Chattopadhyay et al. (2023) for
details. Here, the size of people is proportional to their corresponding
weights.

##  5 The implied regression weights create a limited form of covariate
balance

###  5.1 Exact covariate balance.

5.1 _Exact covariate balance._ Does linear regression balance covariates
between the treated and control groups? As it turns out, there is a specific
form of covariate balance concealed within the linear regression model. The
implied weights of linear regression exactly balance the means of the
covariates included in the model, such that the means of those observed
characteristics are identical between the weighted treated and weighted
control groups. See the diagnostic dashboard in Figure 3: the mean pre-
treatment income and hospital visits are the same after regression
adjustments. Furthermore, if one includes transformations of the covariates in
the regression model, such as squares or interactions between them, then the
means of these transformations will also be exactly balanced. However, there
is no guarantee of balance for covariate transformations that are not included
in the model.555Unless the true propensity score model is inverse linear on
the covariates included in the model (Robins et al. 2007; Chattopadhyay and
Zubizarreta 2023).

Figure 3: Observational study, after adjustments.

###  5.2 A hidden population.

5.2 _A hidden population._ Does linear regression create treated and control
samples that are representative of the target population? While the above
linear regression model exactly balances the means of the covariates in the
treated and control samples, it may not balance them at the target population.
See the diagnostic dashboard in Figure 3. After regression, the average income
in both groups is $29.6K, whereas the average income in the target (i.e.,
treated) population is $27.2K. Similarly, after regression, the average number
of hospital visits in both groups is 4.3, which differs from 4.6, the average
in the target.

In general, linear regression may not balance the samples relative to the
natural populations characterized by the overall subjects, the treated
subjects, or the control subjects. Implicitly, it balances them elsewhere, at
a different covariate profile (see Chattopadhyay and Zubizarreta 2023, for an
exact formula). In this example, the profile is ($29.6K, 4.3), which does not
correspond to any of these natural populations. This profile may in fact
characterize a population that does not exist. In other words, here linear
regression ends up estimating the average treatment effect on a hypothetical
population of individuals whose average income is $29.6K and who has 4.3
hospital visits on average.

Taking a step back, the above linear regression model assumes that the
treatment effect is constant everywhere. If this assumption is correct, then
it does not really matter which population regression ends up targeting: the
average treatment effect is the same as that in any other population. But if
this assumption is incorrect, then effect estimates will be biased, especially
if the implied weights take large negative values, as we explain next.

###  5.3 Estimation beyond the support of the data and negative weights.

5.3 _Estimation beyond the support of the data and negative weights._ Does
linear regression produce sample-grounded or, to be precise, sample-bounded
estimates (Robins et al. 2007) of the treatment effects? To answer this
question, we first note that the implied weights of the regression model can
take negative values. In Figure 3, subjects with negative weights are upside
down. The most notable of them is a control subject with an income of $264K
and 4 hospital visits, who is assigned a very large negative weight.

From a practical standpoint, negative weights are hard to interpret: while a
person with a weight value of 1/2 in the sample means that it has half its
weight in the population, and a weight of zero removes the person from the
population, what is the meaning of a person with a weight value of -1? From an
analytic perspective, these negative weights translate into forming effect
estimates that may lie outside the support or range of the observed data, and
thus that are not sample-bounded. In other words, one can get estimates that
are very distant from any of the measurements in the actual data. As a result,
one may end up obtaining rather silly estimates of the treatment effect. In
our example, the effect estimate under regression is $557, suggesting that
health insurance increases health expenditure. If the assumed regression model
is correct, then the degree of extrapolation is negligible in sufficiently
large samples. However, if the model is incorrect, then this problem may arise
even in large samples.

###  5.4 Differential weighting of minimum variance.

5.4 _Differential weighting of minimum variance._ Does linear regression
produce an evenly weighted or self-weighted sample? In general, unless the
covariate means are exactly balanced between the treated and control groups
before adjustment, regression ensures such mean balance by enforcing
differential weighting of individuals. In our example, disparate weighting is
evident in the control group and is also present in the treatment group. As
shown in the diagnostic dashboard in Figure 3, this differential weighting
results in effective sample sizes that are smaller than the nominal sample
sizes.

Further, an additional fundamental property is that the implied regression
weights have minimum variance. Thus, although the sample individuals are
differentially weighted, the weights are “as stable as possible” subject to
their mean balancing property.666In other words, the OLS estimator for $\tau$
can be equivalently obtained solving an optimization problem that minimizes
the variance of the weights that exactly balance the means of the covariates
between the treatment and control groups, with weights that add up to one and
that can take negative values. See Chattopadhyay and Zubizarreta (2023) for a
general expression of this optimization problem that also encompasses weighted
least squares (WLS), augmented inverse probability weighting (AIPW), and ridge
regression estimation, in connection to the stable balancing weights (SBW;
Zubizarreta 2015). Hence, the OLS estimator of $\tau$ is the best (minimum
variance) linear unbiased estimator if the assumed regression model is
correct. The issue, again, is that the true model that generates the data is
unknown. If it were known, we wouldn’t need the data. If the assumed model is
incorrect, then this stability property of the regression weights offers
limited value, because in that case, regression may end up balancing the wrong
functions, targeting the wrong population, and extrapolating beyond the range
of the data.

##  6 Alternative regression modeling, weighting, and matching approaches

###  6.1 Multi regression.

6.1 _Multi regression._ An alternative regression approach extends the
previous model by including interactions between the observed covariates and
the treatment assignment indicator as follows,

Outcome variable $\displaystyle=\alpha+\beta\cdot\text{Observed
covariates}+\tau\cdot\text{Treatment indicator}$ $\displaystyle\hskip
11.38092pt+\gamma\cdot\text{Observed covariates}\cdot\text{Treatment
indicator}+\epsilon.$

This approach can be used to estimate the average treatment effect in the
overall population, which corresponds to the parameter $\tau$. See Peters
(1941) and Belson (1956) for early discussions of this method. By including
the treatment-covariates interactions, this regression model specification now
allows for effect modification or heterogeneity. As discussed by Chattopadhyay
and Zubizarreta (2023), this approach is equivalent to fitting separate linear
regression models of the outcome on the covariates in the treated and control
groups, and then taking the mean difference in predicted outcomes between
groups under each of the fitted models.

More specifically, for estimating the ATT the multi regression approach
proceeds as follows. First fit the regression model

$\text{Outcome variable}=\alpha+\beta\cdot\text{Observed covariates}+\epsilon$

in the _contro_ l group. Then, for each unit in the treatment group, predict
its outcome in the absence of treatment, by using the previously fitted
regression model. Finally, over all the treated units, compute the mean of
their observed outcome minus the mean of the predicted outcomes.

To what extent does this approach approximate an experiment? Chattopadhyay and
Zubizarreta (2023) show that, under this model specification, the estimator of
the ATT can also be written as a weighted contrast,

$\overbrace{\frac{1\cdot y_{t,1}+1\cdot y_{t,2}+...+1\cdot
y_{t,10}}{10}}^{\text{Treated group}}-\overbrace{\frac{w_{c,1}\cdot
y_{c,1}+w_{c,2}\cdot y_{c,2}+...+w_{c,10}\cdot y_{c,10}}{10}}^{\text{Control
group}}.$

with implied weights $w$ that satisfy the following properties. (i) Like the
previous approach, the means of the observed covariates are exactly balanced
between the treatment and control groups. (ii) The covariate means of each
group are centered at the treated population (the target population for the
ATT), addressing the representativeness problem of the previous approach.
(iii) An issue that persists is that the weights can take negative values,
leaving room for biases due to extrapolation from an incorrectly specified
model. (iv) As before, regression induces differential weighting on the
control subjects such that the weights have the minimum variance among those
that exactly balance the means of the covariates at the target population. See
Figure 4 for an illustration of these properties.

Figure 4: Observational study, after multi regression.

Properties (i) and (ii) are manifest because the covariate profiles of both
groups are ($27.2K, 4.6), the same as that of the target. Also, the
differential weighting in property (iv) is evident in the control group, while
by construction, subjects are weighted uniformly in the treatment group.
Finally, similar to the previous regression approach, property (iii) is
present in the control group, with the same control subject having an income
of $264K and 4 hospital visits getting a very large negative weight. The
effect estimate turns out to be $763 which, like the previous approach, is
contrary to substantive knowledge.

Through the lens of the above properties, this multi regression approach comes
closer to approximating the ideal randomized experiment than the previous
approach. Thus, unless there is a high degree of certainty that treatment
effects are homogeneous, the multi regression approach may be preferred.
However, a potential drawback that persists is bias due to extrapolating an
incorrect model and, under certain interpretative and aesthetic canons, not
having an unweighted sample.

Arguably, the performance of both regression methods discussed can be improved
by conducting careful exploratory analyses of the data before adjustment and
estimation. For instance, in our example, diagnostics may suggest that the
control unit with profile ($264K, 4) is an outlier (in terms of income) and
should be discarded before adjustment. However, such checks can be overlooked
in routine use of regression in practice, and moreover, with multiple
covariates, these checks may not be straightforward. Ideally, one would use
methods that have some of these checks built-in, and hence, that force the
investigator to carefully inspect the data. Along these lines, we now discuss
weighting and matching methods for adjustments in observational studies.

###  6.2 Weighting.

6.2 _Weighting._ In the two aforementioned regression methods, we started with
an outcome model and then considered its implied weights to understand how
these methods attempt to mimic a randomized experiment. Alternatively,
acknowledging that the true outcome model is unknown, we can instead consider
directly weighting the treatment and control groups without specifying an
outcome model (Rosenbaum 1987a). The most commonly used weighting method is
based on modeling the probability of treatment assignment, or propensity
score, using the observed covariates (Rosenbaum and Rubin 1983). If the model
for the propensity score is correct, then in large samples we can unbiasedly
estimate the average treatment effect by simply weighting the observations by
the inverse of their propensity score estimates.

Does the above weighting procedure create a sample that looks as if it came
from a randomized experiment? For a correctly specified propensity score
model, (i) the weights balance, in large samples, the entire joint
distribution of the observed covariates between the two groups. (ii) They also
balance each group relative to the target (i.e., treated) population. With an
incorrect model, however, large imbalances between the groups and relative to
the target may exist for some covariates or their transformations. (iii) The
propensity score is, by definition, non-negative and hence, for any reasonable
propensity score model (correct or incorrect) the resulting effect estimates
are sample-bounded. (iv) The resulting weights can be extreme and highly
variable across individuals, especially when there is a lack of overlap in
covariate distributions between the treatment and control groups.

Overall, weighting is a powerful and flexible method to adjust for covariates.
However, in view of the experimental ideal, care must be taken to adequately
balance the covariates toward the right target population, with weights that
produce stable estimators. Recent weighting methods that emphasize these
features are Hainmueller (2012), Imai and Ratkovic (2014), and Zubizarreta
(2015), among others. See Austin and Stuart (2015) and Chattopadhyay et al.
(2020) for reviews.

In our example, we implement the stable balancing weights (SBW) method of
Zubizarreta (2015). Figure 5 shows the structure of the resulting weighted
sample. Overall, the covariates are reasonably well-balanced relative to the
target population, although not exactly (see the balance diagnostic for
income). By construction, for the ATT, this method puts differential weights
only on the control units, but with weights that are non-negative and less
extreme than the regression methods. Notice that it puts zero weight on the
outlying control subject with profile ($264K, 4). With this method, the
estimated treatment is -$985 in the example.

Figure 5: Observational study, after weighting (SBW).

###  6.3 Matching.

6.3 _Matching._ Matching is a common approach that aims to find the randomized
experiment that is “hidden inside” (Hansen 2004) the observational study
(Stuart 2010; Imbens 2015; Rosenbaum 2020). In its simplest form, matching
forms pairs of similar treated and control subjects, such that the matched
groups are balanced on aggregate. After matching, the structure of the data is
similar to that of a randomized experiment: (i) covariates — the observed ones
— are transparently balanced, while the unobserved ones, like with any other
method for adjustment, must be dealt with a sensitivity analysis. (ii)
Representativeness is overt and easily verifiable, as the target population
can simply be compared to the matched sample in balance tables. (iii) By
achieving covariate balance, effect estimates mostly are an interpolation
grounded in the actual data and not an extrapolation of a potentially
incorrect model. (iv) Each observation in the matched sample has the same
weight, so the simplest test statistics and graphical displays can be used to
analyze the outcomes. Finally, conducting a sensitivity analysis after
matching is straightforward using Rosenbaum bounds (Rosenbaum 1987b; Rosenbaum
2002, chapter 4). Popular matching methods include Rosenbaum (1989), Iacus et
al. (2011), and Diamond and Sekhon (2013). Recent optimal matching methods
that capitalize on modern optimization using network flows include Hansen and
Klopfer (2006), Pimentel et al. (2015), Yu et al. (2020), and Yu and Rosenbaum
(2022). For recent optimization matching methods that leverage modern
optimization using generic integer programming, see Zubizarreta (2012),
Zubizarreta et al. (2014), and Cohn and Zubizarreta (2022).

In our example, we implement two matching methods, optimal pair matching as
proposed by Rosenbaum (1989) and profile matching by Cohn and Zubizarreta
(2022). The corresponding structures of the matched samples are depicted in
Figures 6 and 7. Overall, both methods produce well-balanced and uniformly
weighted samples relative to the target population. Finally, the effect
estimates under pair matching and profile matching are -$1114 and -$989,
respectively, close to the estimate provided by SBW.

Figure 6: Observational study, after pair matching. Figure 7: Observational
study, after profile matching.

##  7 It’s through contrast that we see

Comparison and contrast are the basic means for understanding whether a
treatment works; and randomization, the ideal mechanism to build similar
groups where the effect of treatment is isolated. In the absence of
randomization, covariate imbalance adjustments must be made, and regression,
weighting, and matching are three fundamental methods for building comparable
groups. These methods can be enhanced using non-parametric (machine) learning
and combined in doubly robust estimators. The extent to which these three
methods approximate the experimental ideal varies and is not always overt.
With actual data, the implied weights of linear regression can be calculated
using the new lmw package for R. The story of regression is about a correctly
specified model, in which case the approach is optimal. But true models are
elusive and a more realistic and useful approach is to build good contrasts,
approximating an experiment with guarantees of robustness, interpretability,
and ease to assess generalizability to target populations and sensitivity to
hidden biases. In this sense, the traditional regression method is dominated
by the multi regression approach, and matching and weighting approaches should
be given strong consideration.

## References

  * Abadie et al. (2015) Abadie, A., Diamond, A., and Hainmueller, J. (2015), “Comparative politics and the synthetic control method,” American Journal of Political Science, 59, 495–510.
  * Angrist and Krueger (1999) Angrist, J. D. and Krueger, A. B. (1999), “Empirical strategies in labor economics,” in Handbook of Labor Economics, Elsevier, vol. 3, pp. 1277–1366.
  * Aronow and Samii (2016) Aronow, P. M. and Samii, C. (2016), “Does regression produce representative estimates of causal effects?” American Journal of Political Science, 60, 250–267.
  * Austin and Stuart (2015) Austin, P. C. and Stuart, E. A. (2015), “Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies,” Statistics in Medicine, 34, 3661–3679.
  * Belson (1956) Belson, W. A. (1956), “A technique for studying the effects of a television broadcast,” Journal of the Royal Statistical Society: Series C (Applied Statistics), 5, 195–202.
  * Chattopadhyay et al. (2023) Chattopadhyay, A., Greifer, N., and Zubizarreta, J. R. (2023), “lmw: Linear Model Weights for Causal Inference,” arXiv preprint arXiv:2303.08790.
  * Chattopadhyay et al. (2020) Chattopadhyay, A., Hase, C. H., and Zubizarreta, J. R. (2020), “Balancing vs modeling approaches to weighting in practice,” Statistics in Medicine, 39, 3227–3254.
  * Chattopadhyay and Zubizarreta (2023) Chattopadhyay, A. and Zubizarreta, J. R. (2023), “On the implied weights of linear regression for causal inference,” Biometrika, in press.
  * Cochran (1965) Cochran, W. G. (1965), “The planning of observational studies of human populations,” Journal of the Royal Statistical Society, 128, 234–266.
  * Cohn and Zubizarreta (2022) Cohn, E. R. and Zubizarreta, J. R. (2022), “Profile Matching for the Generalization and Personalization of Causal Inferences,” Epidemiology, 33, 678–688.
  * Cornfield et al. (1959) Cornfield, J., Haenszel, W., Hammond, E., Lilienfeld, A., Shimkin, M., and Wynder, E. (1959), “Smoking and lung cancer: Recent evidence and a discussion of some questions,” Journal of the National Cancer Institute.
  * Diamond and Sekhon (2013) Diamond, A. and Sekhon, J. S. (2013), “Genetic matching for estimating causal effects: A general multivariate matching method for achieving balance in observational studies,” Review of Economics and Statistics, 95, 932–945.
  * Dorn (1953) Dorn, H. F. (1953), “Philosophy of inferences from retrospective studies,” American Journal of Public Health and the Nations Health, 43, 677–683.
  * Fisher (1935) Fisher, R. A. (1935), The Design of Experiments, London: Oliver & Boyd.
  * Gelman and Imbens (2018) Gelman, A. and Imbens, G. (2018), “Why high-order polynomials should not be used in regression discontinuity designs,” Journal of Business & Economic Statistics, 1–10.
  * Hainmueller (2012) Hainmueller, J. (2012), “Entropy balancing for causal effects: a multivariate reweighting method to produce balanced samples in observational studies,” Political Analysis, 20, 25–46.
  * Hansen (2004) Hansen, B. B. (2004), “Full matching in an observational study of coaching for the SAT,” Journal of the American Statistical Association, 99, 609–618.
  * Hansen and Klopfer (2006) Hansen, B. B. and Klopfer, S. O. (2006), “Optimal full matching and related designs via network flows,” Journal of Computational and Graphical Statistics, 15, 609–627.
  * Hernán and Robins (2020) Hernán, M. A. and Robins, J. M. (2020), Causal Inference: What if, CRC Boca Raton.
  * Iacus et al. (2011) Iacus, S. M., King, G., and Porro, G. (2011), “Causal inference without balance checking: coarsened exact matching,” Political Analysis, mpr013.
  * Imai and Ratkovic (2014) Imai, K. and Ratkovic, M. (2014), “Covariate balancing propensity score,” Journal of the Royal Statistical Society: Series B, 76, 243–263.
  * Imbens (2015) Imbens, G. W. (2015), “Matching methods in practice: three examples,” Journal of Human Resources, 50, 373–419.
  * Imbens and Rubin (2015) Imbens, G. W. and Rubin, D. B. (2015), Causal Inference in Statistics, Social, and Biomedical Sciences, Cambridge University Press.
  * Newhouse et al. (1993) Newhouse, J. P., Group, R. C. I. E., and Staff, I. E. G. (1993), Free for All?: Lessons from the RAND Health Insurance Experiment, Harvard University Press.
  * Peters (1941) Peters, C. C. (1941), “A method of matching groups for experiment with no loss of population,” The Journal of Educational Research, 34, 606–612.
  * Pimentel et al. (2015) Pimentel, S. D., Kelz, R. R., Silber, J. H., and Rosenbaum, P. R. (2015), “Large, sparse optimal matching with refined covariate balance in an observational study of the health outcomes produced by new surgeons,” Journal of the American Statistical Association, 110, 515–527.
  * Robins et al. (2007) Robins, J., Sued, M., Lei-Gomez, Q., and Rotnitzky, A. (2007), “Comment: performance of double-robust estimators when “inverse probability” weights are highly variable,” Statistical Science, 22, 544–559.
  * Rosenbaum (1987a) Rosenbaum, P. R. (1987a), “Model-based direct adjustment,” Journal of the American Statistical Association, 82, 387–394.
  * Rosenbaum (1987b) — (1987b), “Sensitivity analysis for certain permutation inferences in matched observational studies,” Biometrika, 74, 13–26.
  * Rosenbaum (1989) — (1989), “Optimal matching for observational studies,” Journal of the American Statistical Association, 84, 1024–1032.
  * Rosenbaum (2002) — (2002), Observational Studies, Springer.
  * Rosenbaum (2005) — (2005), “Sensitivity analysis in observational studies,” Encyclopedia of statistics in behavioral science.
  * Rosenbaum (2010) — (2010), Design of Observational Studies, Springer.
  * Rosenbaum (2020) — (2020), “Modern algorithms for matching in observational studies,” Annual Review of Statistics and Its Application, 7, 143–176.
  * Rosenbaum and Rubin (1983) Rosenbaum, P. R. and Rubin, D. B. (1983), “The central role of the propensity score in observational studies for causal effects,” Biometrika, 70, 41–55.
  * Rubin (2007) Rubin, D. B. (2007), “The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials,” Statistics in Medicine, 26, 20–36.
  * Stuart (2010) Stuart, E. A. (2010), “Matching methods for causal inference: a review and a look forward,” Statistical Science, 25, 1–21.
  * VanderWeele and Ding (2017) VanderWeele, T. J. and Ding, P. (2017), “Sensitivity analysis in observational research: introducing the E-value,” Annals of Internal Medicine, 167, 268–274.
  * Yu and Rosenbaum (2022) Yu, R. and Rosenbaum, P. R. (2022), “Graded Matching for Large Observational Studies,” Journal of Computational and Graphical Statistics, 31, 1406–1415.
  * Yu et al. (2020) Yu, R., Silber, J. H., and Rosenbaum, P. R. (2020), “Matching methods for observational studies derived from large administrative databases,” Statistical Science, 35, 338–355.
  * Zhao et al. (2019) Zhao, Q., Small, D. S., and Bhattacharya, B. B. (2019), “Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap,” Journal of the Royal Statistical Society Series B: Statistical Methodology, 81, 735–761.
  * Zubizarreta (2012) Zubizarreta, J. R. (2012), “Using mixed integer programming for matching in an observational study of kidney failure after surgery,” Journal of the American Statistical Association, 107, 1360–1371.
  * Zubizarreta (2015) — (2015), “Stable weights that balance covariates for estimation with incomplete outcome data,” Journal of the American Statistical Association, 110, 910–922.
  * Zubizarreta et al. (2014) Zubizarreta, J. R., Paredes, R. D., and Rosenbaum, P. R. (2014), “Matching for balance, pairing for heterogeneity in an observational study of the effectiveness of for-profit and not-for-profit high schools in Chile,” Annals of Applied Statistics, 8, 204–231.