plain ##1 ##2 ##1 ##2 (##3)

# Variance-based sensitivity analysis for weighting estimators results in more
informative bounds111The authors would like to thank Erin Hartman, Kevin Guo,
Alex Franks, Avi Feller, Peng Ding, Nicole Pashley, and Cyrus Samii for
helpful comments. Melody Huang is supported by the National Science Foundation
Graduate Research Fellowship under Grant No. 2146752, and Samuel D. Pimentel
is supported by the National Science Foundation under Grant No. 2142146. Any
opinion, findings, and conclusions or recommendations expressed in this
material are those of the authors(s) and do not necessarily reflect the views
of the National Science Foundation.

Melody Huang and Samuel D. Pimentel

###### Abstract

Weighting methods are popular tools for estimating causal effects; assessing
their robustness under unobserved confounding is important in practice. In the
following paper, we introduce a new set of sensitivity models called the
“variance-based sensitivity model”. The variance-based sensitivity model
characterizes the bias from omitting a confounder by bounding distributional
differences that arise in the weights from omitting a confounder, with several
notable innovations over existing approaches. First, the variance-based
sensitivity model can be parameterized by an $R^{2}$ parameter that is both
standardized and bounded. We introduce a formal benchmarking procedure that
allows researchers to use observed covariates to reason about plausible
parameter values in an interpretable and transparent way. Second, we show that
researchers can estimate valid confidence intervals under the variance-based
sensitivity model, and provide extensions for incorporating substantive
knowledge about the confounder to help tighten the intervals. Last, we
demonstrate, both empirically and theoretically, that the variance-based
sensitivity model can provide improvements on both the stability and tightness
of the estimated confidence intervals over existing methods. We illustrate our
proposed approach on a study examining blood mercury levels using the National
Health and Nutrition Examination Survey (NHANES).

##  1 Introduction

In observational studies of causal effects, researchers must address possible
confounding effects from non-random treatment assignment. Typically, one
relies on pre-treatment covariates either to re-weight units based on
propensity of treatment, or to model the outcome of interest. In practice,
researchers have no way of knowing whether the included covariates fully
capture the confounding effects. When confounders are omitted, the resulting
estimates will be biased. Sensitivity analyses speak to this concern by
allowing researchers to assess the robustness of their point estimates to
omitted confounders. In a sensitivity analysis, a researcher introduces a
parameter describing the amount of unobserved confounding present and redoes
the analysis under different values of this parameter, determining the set of
values for which the results of the study will be reversed. The robustness of
the study may then be evaluated by reasoning about the plausibility of these
values.

In contrast to typical estimands, parameters in sensitivity analysis are
inherently unindentifiable, because they are designed to describe an omitted
variable. Thus, there exists a trade-off between how complex the sensitivity
analysis is, and how informative the sensitivity analysis can be. For example,
Dahabreh et al. (2019) proposed a sensitivity analysis in which researchers
can obtain both an adjusted point estimate and the associated uncertainty from
omitting a confounder. However, the sensitivity analysis requires researchers
to directly model the bias that arises from omitting a confounder. In
contrast, Zhao et al. (2019) introduced a sensitivity analysis that only
requires one parameter and allows researchers to estimate confidence intervals
that account for the unobserved confounder. However, the resulting intervals
are often extremely wide, making it difficult to reason about whether or not
there is sensitivity from omitting a confounder.

In the following paper, we introduce a new set of sensitivity models known as
the variance-based sensitivity model that provide the flexibility and
generality of existing sensitivity analyses, while simultaneously allowing
researchers to estimate narrower, more informative bounds for weighted
estimators. The proposed sensitivity models constrain distributional
differences in the weights that arise from omitting a confounder, and do not
rely on additional assumptions on the outcome, confounder, or treatment
assignment mechanism.

The paper provides three primary contributions. First, the proposed variance-
based sensitivity model can be parameterized by a single sensitivity
parameter, an $R^{2}$ measure. Unlike previously proposed sensitivity
analyses, our parameter is both bounded and standardized on an interval of 0
to 1. We develop formal benchmarking approaches that allow researchers to use
observed covariates to reason about plausible values for the $R^{2}$ value,
providing much-needed interpretability.

Second, we introduce a method for estimation of valid confidence intervals
under the variance-based sensitivity model. We give a closed-form solution for
the maximum bias that can occur for a fixed set of sensitivity models, which
we denote the optimal bias bound. We also provide extensions for incorporating
additional substantive knowledge about the confounder into the optimal bias
bound, further restricting the range of plausible bias.

Finally, we show that a variance-based sensitivity analysis can be formulated
as a bias maximization problem, with a constraint on the weighted average
error. We formalize the relationship between the variance-based sensitivity
model and alternative approaches, which rely on constraining a worst-case
error. By moving away from characterizing bias from the perspective of a
worst-case error, variance-based sensitivity analysis is able to estimate more
informative and stable bounds.

The paper is organized as follows. Section 2 gives set-up and notation.
Section 3 introduces the variance-based sensitivity model. Section 4, compares
the proposed sensitivity models to alternative sensitivity approaches. Proofs
and extended discussion are provided in the Appendix.

##  2 Background

###  2.1 Set-Up and Notation

To begin, we consider an observational study with $n$ individuals. Define
$Z_{i}$ as a binary treatment assignment variable, where $Z_{i}=1$ when unit
$i$ is assigned to treatment, and 0 otherwise, $Y_{i}(1)$ and $Y_{i}(0)$ are
potential outcomes, and $\mathcal{X}_{i}$ is a vector of pre-treatment
covariates. Let the tuple $(Y_{i}(1),Y_{i}(0),X_{i},Z_{i})$ for all
$i\in\\{1,…,n\\}$ be independently and identically distributed from an
arbitrary joint distribution, where $Y(1),Y(0)\in\mathbbm{R}^{n}$,
$Z_{i}\in\\{0,1\\}^{n}$ and $\mathcal{X}\in\mathbbm{R}^{n\times p}$.

Throughout, we invoke the standard SUTVA assumption—i.e., no interference,
with treatments identically administered across all units (Rubin, 1980). Thus
observed outcomes $Y_{i}$ can be written as $Y_{i}:=Y_{i}(1)\cdot
Z_{i}+Y_{i}(0)\cdot(1-Z_{i})$. Because treatments are not randomly assigned in
an observational study, we must also invoke an additional assumption, known as
the conditional ignorability of treatment assignment:

######  Assumption 1 (Conditional Ignorability of Treatment Assignment).

$Y_{i}(1),Y_{i}(0)\
\raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\
Z_{i}\mid\mathcal{X}_{i}$

Assumption 1 states that conditional on a set of pre-treatment covariates
$\mathcal{X}$, treatment assignment is independent of potential outcomes (i.e.
no further confounding remains).

In addition to Assumption 1, we also assume overlap, such that conditional on
some set of pre-treatment covariates, the probability of being assigned
treatment is non-zero (Rosenbaum and Rubin, 1983):

######  Assumption 2 (Overlap).

For all $x\in\mathcal{X}$, $0<P(Z_{i}=1\mid\mathcal{X}=x)<1$.

Our primary estimand is the average treatment effect for the treated (ATT):

$\tau:=\mathbb{E}(Y_{i}(1)-Y_{i}(0)\mid Z_{i}=1).$

However, the proposed method can easily be extended for estimating the average
treatment effect (ATE). Furthermore, all proofs and derivations are done with
respect to a general missingness indicator, such that the results can be
applied to general, missing data settings, such as weighting for external
validity or survey weighting (see Appendix A.1 for more discussion and
details).

Weighted estimators, which are popular for estimating causal effects in
observational setting, adjust for distributional differences in the pre-
treatment covariates $\mathcal{X}$ across the treatment and control groups:

$\hat{\tau}_{W}=\frac{1}{\sum_{i=1}^{n}Z_{i}}\sum_{i=1}^{n}Z_{i}Y_{i}-\underbrace{\frac{\sum_{i=1}^{n}(1-Z_{i})Y_{i}w_{i}}{\sum_{i=1}^{n}(1-Z_{i})w_{i}}}_{\text{Weighted
Control Mean}}.$

A common choice of weights is inverse propensity weights, where
$w_{i}=P(Z_{i}=1\mid\mathcal{X})/P(Z_{i}=0\mid\mathcal{X})$. These weights are
often constructed using a logistic regression to predict the probability of
treatment assignment. Recent balancing approaches provide a semi-parametric
option for researchers to estimate weights by minimizing the distributional
difference between the treatment and control groups, without modeling the
underlying probabilities (see Ben-Michael et al. (2020) for a recent review on
balancing weights).

Under the correct specification of the weights, if the correct set of
covariates are included, the weighted estimator will be a consistent and
unbiased estimate of the true treatment effect. However, in practice, there is
no way of knowing whether Assumption 1 holds. We propose a set of sensitivity
models that characterize the bias of a weighted estimator when researchers
omit a variable from the set $\mathcal{X}$. More specifically, we will assume
that $\mathcal{X}=\\{\mathbf{X},\mathbf{U}\\}$, such that both $\mathbf{X}$
and $\mathbf{U}$ are necessary for Assumption 1 to hold. We assume that
researchers have otherwise correctly specified the weights.222For example, the
framework does not explicitly account for cases in which researchers are using
a probit model, when the true underlying data generating process is logistic.
However, mis-specification concerns can also be addressed with the proposed
framework, if researchers can write the mis-specification error as an omitted
variable problem. A simple example of this is if a linear probability model is
used, $\mathbf{U}$ can include non-linear functions of $\mathbf{X}$ that
matter for modeling selection. We provide more discussion in Appendix A.3. We
define $w$ as the weights that include only $\mathbf{X}$, and $w^{*}$ as the
ideal weights that include both $\mathbf{X}$ and $\mathbf{U}$. We refer to the
omitted variables, $\mathbf{U}$, as confounders. We note that the sensitivity
framework will not explicitly account for settings in which (1) researchers
estimate uniform weights (i.e., no adjustment for confounding), or (2) the
true ideal weights are uniform. Finally, we will assume, without loss of
generality, that both $w$ and $w^{*}$ are centered at mean 1.

###  2.2 Related Literature

A popular approach for assessing the robustness of weighted estimates to
omitted confounders uses the marginal sensitivity model (Tan, 2006), in which
researchers posit a bound, $\Lambda$, on the individual-level error in the
weights that can arise under unobserved confounding:

$\Lambda^{-1}\leq\frac{w_{i}^{*}}{w_{i}}\leq\Lambda,\ \ \ \ \text{ for
}i=1,...,n,$

where $\Lambda\geq 1$. $\Lambda$ represents the largest possible error that
can arise from omitting a confounder. Researchers can bound the maximum and
minimum bias that arises under a fixed $\Lambda$, and use a percentile
bootstrap to estimate valid confidence intervals (Zhao et al., 2019).

In practice, the true $\Lambda$ is unknown, so to conduct the sensitivity
analysis, researchers posit increasing values of $\Lambda$ until the estimated
confidence intervals contain zero. The minimum $\Lambda$ value for which the
estimated intervals cross zero is denoted as $\Lambda^{*}$. If $\Lambda^{*}$
is close to 1, even a small amount of error from omitting a confounder could
result in an estimated effect becoming insignificant. On the other hand, if
$\Lambda^{*}$ is much larger than 1, estimated effects are only sensitive to
very strong unmeasured confounders.

While the marginal sensitivity model guarantees valid intervals
asymptotically, in practice, the intervals tend to be extremely conservative.
This means that the estimated intervals often include the null estimate, even
under low amounts of confounding. As such, when researchers’ estimated bounds
imply an estimated effect is no longer statistically significant, it is
difficult to distinguish if this is a sign that there is sensitivity to an
omitted confounder, or if the sensitivity model is overly pessimistic.
Tightening these intervals often requires researchers to invoke additional
constraints in the models, or parametrically model the outcomes in some way
(Dorn and Guo, 2021; Nie et al., 2021). Furthermore, the underlying
sensitivity parameter in the marginal sensitivity model is dependent on the
worst-case error that arises from omitting a confounder. This is inherently
difficult to reason about in practice, as the true value of the parameter will
depend on outliers and, in asymptotic settings, can be infinitely large.

We now propose a new set of sensitivity models, the variance-based sensitivity
model. The proposed framework can be viewed as a one parameter generalization
of existing sensitivity frameworks in the literature that rely on bounding the
error in the weights from omitting a confounder (i.e., Hartman and Huang
(2022), Hong et al. (2021), Shen et al. (2011)). We provide several key
contributions. First, unlike the frameworks proposed by Hong et al. (2021) and
Shen et al. (2011), the variance-based sensitivity model introduce a
standardized and bounded parameterization of the confounding strength, which
can help improve transparency and interpretability for applied researchers.
Second, the aforementioned sensitivity analyses do not engage with how
potential confounders may affect their inference, and are limited to
discussions about movements in the point estimate. In contrast, the variance-
based sensitivity model provides a method for researchers to estimate valid
asymptotic confidence intervals for fixed level of confounding.

Furthermore, we formalize the connection between these variance-based
approaches to alternative sensitivity approaches, which formulate sensitivity
models as optimization problems. In particular, we demonstrate that the
variance-based sensitivity model can be viewed as a constrained weighted
$L_{2}$ norm problem, which provides a framework to compare the proposed
sensitivity models with the marginal sensitivity model. Moving away from a
worst-case error parameterization of the error allows researchers to obtain
more informative and stable bounds under the variance-based sensitivity model.
The benefits of constraining a weighted $L_{2}$ norm instead of a worst-case
error is conceptually similar to the advantages highlighted in Zhang and Zhao
(2022), in which authors consider a constraint on $L_{2}$ norms, and Kallus
and Zhou (2018), which introduces an $L_{1}$ norm, with the added benefit of
having an interpretable sensitivity parameter in the form of an $R^{2}$ value.

###  2.3 Running Example: NHANES

Throughout the paper, we perform a re-analysis of a study presented in Zhao et
al. (2018) (as well as Zhao et al. (2019) and Soriano et al. (2021)),
analyzing the effects of fish consumption on blood mercury levels. More
specifically, we use data from the 2013-2014 National Health and Nutrition
Examination Survey (NHANES).

Following the original study, we define the outcome of interest as the total
blood mercury (in $\log_{2}$), measured in micrograms per liter. As such, an
estimated treated-control outcome difference of 1 implies that a treated
person’s total blood mercury is twice that of an individual in control’s total
blood mercury. The treatment is defined by whether or not individuals consumed
more than 12 servings of fish or shellfish in the preceding month. There are
234 total treated units and 873 control units. To account for the non-random
treatment assignment, we use the available demographic data for the
individuals in the survey, which include variables like gender, age, income,
race, education, and smoking history to estimate entropy balancing weights
(Hainmueller, 2012).

The unweighted estimate is 2.37; after accounting for pre-treatment
covariates, we obtain a weighted ATT estimate of 2.15 (see Table 1 for a
summary). Therefore, from our estimate, we expect that on average, a treated
individual who consumes more fish will have around 4 times as much total blood
mercury than an individual in control.

| Unweighted (DiM) | IPW
---|---|---
Estimated Effect (ATT) | 2.37 (0.10) | 2.15 (0.11)
Table 1: Estimated effect of fish consumption on blood mercury levels.
Standard errors are reported in parentheses.

##  3 The Variance-Based Sensitivity Model

We now introduce the variance-based sensitivity model. Section 3.1 defines the
sensitivity models and shows that the set of sensitivity models lends itself
naturally to an $R^{2}$ parameterization. Section 3.2 shows that there exists
a closed-form solution for the maximum bias under the variance-based
sensitivity model that can be directly estimated from the data. Section 3.3
lays out a method to estimate asymptotically valid confidence intervals under
the proposed set of sensitivity models. Sections 3.4 and 3.5 provide formal
benchmarking tools to help researchers conduct their sensitivity analyses, and
illustrate the proposed approach on the running example respectively. Appendix
A.4 provides an extension for researchers to impose constraints on the
strength of the relationship between the confounder and the outcome within the
variance-based sensitivity model.

###  3.1 Defining a New Sensitivity Model

Instead of constraining the worst-case, individual-level multiplicative error
across the weights, we constrain the difference in the distribution of the
estimated weights and the distribution of the true weights. We refer to this
set as the “variance-based sensitivity model.”

######  Definition 3.1 (Variance-Based Sensitivity Model).

Let $R^{2}$ be the residual variation in the true weights $w^{*}$, not
explained by $w$:

$R^{2}:=1-\underbrace{\frac{\text{var}(w_{i}\mid
Z_{i}=0)}{\text{var}(w^{*}_{i}\mid
Z_{i}=0)}}_{{\begin{subarray}{c}{\scriptsize{\text{Variation in }w^{*}}}\\\
{\scriptsize\text{explained by }w}\end{subarray}}}$

Then, for a fixed $R^{2}\in[0,1)$, we define the variance-based sensitivity
model $\sigma(R^{2})$:

$\sigma(R^{2})\equiv\left\\{w^{*}_{i}\in\mathbbm{R}^{n}:1\leq\frac{\text{var}(w^{*}_{i}\mid
Z_{i}=0)}{\text{var}(w_{i}\mid Z_{i}=0)}\leq\frac{1}{1-R^{2}}\right\\}.$

The variance-based sensitivity model constrains how different the true weights
$w^{*}$ can be from the estimated weights $w$. This serves as a constraint on
the residual imbalance in the omitted variable. More formally, we decompose
the true weight $w^{*}$ into two components: (1) the weight $w$, and (2) the
residual imbalance in $\mathbf{U}$:

$\displaystyle w^{*}$
$\displaystyle=\frac{P(Z=1\mid\mathbf{X},\mathbf{U})}{1-P(Z=1\mid\mathbf{X},\mathbf{U})}=\underbrace{\frac{P(Z=1\mid\mathbf{X})}{1-P(Z=1\mid\mathbf{X})}}_{\begin{subarray}{c}\text{Population
Propensity}\\\ {\text{Weights
}(w)}\end{subarray}}\cdot\underbrace{\frac{P(\mathbf{U}\mid\mathbf{X},Z=1)}{P(\mathbf{U}\mid\mathbf{X},Z=0)}}_{\text{Imbalance
in $\mathbf{U}$}},$ (1)

where the imbalance term is a ratio of the conditional probability density
function of the omitted variable across the treatment and control groups. The
distributional difference between the weights $w$ and the ideal weights
$w^{*}$ will be driven by the imbalance term. Intuitively, if the omitted
variable $\mathbf{U}$ is very imbalanced, then accounting for the omitted
variable results in very different values for $w^{*}$ and $w$. Alternatively,
if $\mathbf{U}$ is not very imbalanced, then including it will result in
weights $w^{*}$ that are very similar to $w$. Limiting the distributional
difference between the true weights and estimated weights effectively
restricts the amount of residual imbalance in the omitted confounder. In
Section 4, we show that this is equivalent to constraining a weighted $L_{2}$
norm of the errors $w^{*}_{i}/w_{i}$, in contrast with the marginal
sensitivity model, which constrains an $L_{\infty}$ norm.

The distributional difference between the estimated weights and the true
weights can be written as a function of an $R^{2}$ parameter. The $R^{2}$
parameter represents the residual variation in the true weights, not explained
by the estimated weights, and is naturally bounded on an interval of $[0,1]$.

###  3.2 Optimal Bias Bounds

Valid confidence intervals for a set of sensitivity models must account for
two factors: (1) the bias that arises from omitting a confounder, and (2) the
uncertainty associated with estimation. In the following subsection, we
introduce optimal bias bounds that researchers can estimate for the variance-
based sensitivity model, under a fixed $R^{2}$ value. Section 3.3 introduces a
percentile bootstrap approach for researchers to simultaneously account for
uncertainty in estimation.

In the following theorem, we show that for a fixed $R^{2}$, we can estimate
the possible range of bias values. We refer to the minimum and maximum values
of these potential bias values as the optimal bias bounds. However, unlike the
marginal sensitivity models, in which researchers must solve a linear
programming problem to identify the extrema, the variance-based sensitivity
model admits a closed-form solution for the optimal bias. More specifically,
the optimal bias bounds are a function of three different components: (1) a
correlation bound, which represents the maximum correlation an omitted
confounder can have with the outcome of interest; (2) the imbalance
(represented by the $R^{2}$); and (3) a scaling factor.

######  Theorem 3.1 (Optimal Bias Bounds).

For a fixed $R^{2}\in[0,1)$, the maximum bias under $\sigma(R^{2})$ (denoted
as $\max_{\tilde{w}\in\sigma(R^{2})}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})$)
can be written as a function of the following components:

$\displaystyle\max_{\tilde{w}\in\sigma(R^{2})}\text{
Bias}(\hat{\tau}_{W}\mid\tilde{w})$
$\displaystyle=\underbrace{\sqrt{1-\text{cor}(w_{i},Y_{i}\mid
Z_{i}=0)^{2}}}_{\text{(a) Correlation
Bound}}\sqrt{\phantom{\leavevmode\hbox{\set@color$\displaystyle\frac{R^{2}}{1-R^{2}}\cdot\text{var}(Y_{i}|Z_{i}=0)\cdot\text{var}(w_{i}|Z_{i}=0)$}}}{\underbrace{\frac{R^{2}}{1-R^{2}}}_{\makebox[0.0pt]{\scriptsize\text{(b)
Imbalance}}}}\cdot{\underbrace{\text{var}(Y_{i}|Z_{i}=0)\cdot\text{var}(w_{i}|Z_{i}=0)}_{\makebox[0.0pt]{\scriptsize}\text{(c)
Scaling Factor}}},$ (2)

with the minimum bias given as the negative of Equation (2). The optimal bias
bounds are given by the minimum and maximum biases.

Theorem 3.1 highlights the different components that affect the magnitude of
the bias bounds. We provide more details about each component below.

##### Correlation Bound.

The correlation bound, given by Equation (2)-(a), represents the maximum
correlation between imbalance in an omitted confounder and outcome.
Intuitively, this is similar to the marginal sensitivity model, in which Dorn
and Guo (2021) demonstrated that the optimal bias bounds are obtained when the
imbalance from the omitted confounder is maximally correlated with the
outcome. From Equation (2)-(a), we see that the bound is a function of the
correlation between the estimated weights and the outcome. If the estimated
weights are highly correlated with the outcome, then the degree to which the
residual imbalance in the omitted confounder can be correlated to the outcome
is limited, and this bound is lower. However, if the correlation between the
estimated weights and the outcome is relatively low, then the possible
correlation between omitted-confounder imbalance and outcome has a much larger
range. In the worst case, the estimated weights and the outcome are not
correlated at all (i.e., $\text{cor}(w_{i},Y_{i}\mid Z_{i}=0)=0$); then, the
correlation bound will simply equal 1.

##### Residual Imbalance.

The second component of the bias bound is the residual imbalance in an omitted
confounder, and is a function of the $R^{2}$ parameter (Equation 2-(b)).
Similar to a point made in Cinelli and Hazlett (2020), there exists an
asymmetry in the drivers of bias. More specifically, as the correlation
between the imbalance term and the outcome increases towards 1 (i.e., the
correlation bound approaches 1), the overall impact on the bias bound is
bounded at 1. In contrast, as the level of imbalance in the omitted confounder
increases, the effect on the bias bounds is unbounded. In other words, as
$R^{2}\to 1$, the corresponding bias bounds will increase towards infinity.

##### Scaling Factor.

The last factor in the bias bound is a scaling factor (represented by Equation
(2)-(c)). The scaling factor comprises of the variance of the outcomes across
the control units (i.e., $\text{var}(Y_{i}\mid Z_{i}=0)$) and the variance of
the estimated weights (i.e., $\text{var}(w_{i}\mid Z_{i}=0)$). This represents
the overall heterogeneity that is present in the data. More specifically, as
the variance in the estimated weights increases, there is more imbalance
between the treatment and control groups that the weights are accounting for.
Similarly, as the variance in the outcomes increases, there is more potential
for heterogeneity to be related to the selection into treatment, making it
more difficult to recover the true estimated effect.333We note that in
settings when $\text{var}(Y_{i}\mid Z_{i}=0)=0$, there is no variation in the
outcomes across the control group. As such, no amount of weighting will alter
our estimate. The scaling factor is a function of the observed data, and is
not related to the omitted confounder. However, the scaling factor serves as
an amplification of any bias that would arise from omitting a confounder.

The key takeaway from Theorem 3.1 is that given a researcher-chosen $R^{2}$
value, the optimal bias bound in Theorem 3.1 is directly estimable from the
data. As such, for a fixed $R^{2}$ value, researchers can directly calculate
the range of possible bias values.

###  3.3 Constructing Confidence Intervals

We now introduce a method to construct valid asymptotic confidence intervals
for the variance-based sensitivity models. Our method builds on the work of
Zhao et al. (2019) and uses a percentile bootstrap to simultaneously accounts
for the bias due to omitting a confounder and the uncertainty associated with
estimation. Our approach is distinct from those in the partial identification
literature that require known asymptotic distributions of the boundaries of
the partially identified region (Imbens and Manski, 2004; Aronow and Lee,
2013); similar to the discussion provided in Zhao et al. (2019), it can
difficult to characterize these distributions analytically in our sensitivity
framework. Instead, the proposed bootstrap approach allows researchers to
account for sampling uncertainty without explicitly characterizing the
asymptotic distributions of the boundary estimates.

To begin, for a fixed $R^{2}$, we can define $\hat{\tau}(\tilde{w})$ as the
weighted estimate, using weights $\tilde{w}\in\sigma(R^{2})$. We can
equivalently view $\hat{\tau}(\tilde{w})$ as an adjusted weighted estimate for
some $\tilde{w}\in\sigma(R^{2})$:

$\hat{\tau}(\tilde{w}):=\hat{\tau}_{W}-\text{Bias}(\hat{\tau}_{W}\mid\tilde{w}),$
(3)

where $\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})$ is the bias of
$\hat{\tau}_{W}$, assuming $\tilde{w}$ were the true weights (i.e.,
$\text{Bias}(\hat{\tau}_{W}\mid\tilde{w}):=\hat{\tau}_{W}-\hat{\tau}(\tilde{w})$),
assuming the true weights are equal to $\tilde{w}$.

Applying the results from Zhao et al. (2019), for every
$\tilde{w}\in\sigma(R^{2})$, we can construct a confidence interval for
$\tau(\tilde{w})$ using a percentile bootstrap:

$[L(\tilde{w}),U(\tilde{w})]=\left[Q_{\alpha/2}\big{(}\hat{\tau}^{(b)}(\tilde{w})\big{)},Q_{1-\alpha/2}\big{(}\hat{\tau}^{(b)}(\tilde{w})\big{)}\right],$
(4)

where $\hat{\tau}^{(b)}(\tilde{w})$ is the adjusted weighted estimator in
bootstrap sample $b\in\\{1,...,B\\}$, and $Q_{\alpha}(\cdot)$ denotes the
$\alpha$-th percentile in the bootstrap distribution. By the following
theorem, $[L(\tilde{w}),U(\tilde{w})]$ will be an asymptotically valid
(1-$\alpha$) confidence interval for $\tau(\tilde{w})$:

######  Theorem 3.2 (Validity of Percentile Bootstrap).

Under mild regularity conditions (see Assumption 4 in the Appendix), for every
$\tilde{w}\in\sigma(R^{2})$:

$\limsup_{n\to\infty}P(\tau(\tilde{w})<L(\tilde{w}))\leq\frac{\alpha}{2}\text{
and }\limsup_{n\to\infty}P(\tau(\tilde{w})>U(\tilde{w}))\leq\frac{\alpha}{2},$

where $L(\tilde{w})$ and $U(\tilde{w})$ are defined as the $\alpha/2$ and
$1-\alpha/2$-th quantiles of the bootstrapped estimates (i.e., Equation (4)).

Theorem 3.2 states that for any set of weights $\tilde{w}$, the percentile
bootstrap can be applied to estimate valid confidence intervals for an
adjusted, weighted estimate $\tau(\tilde{w})$. However, as discussed in the
previous section, for a given $R^{2}$ value, there exists many different sets
of weights $\tilde{w}$ that can be defined from $\sigma(R^{2})$. As such, we
apply the union method to estimate a conservative $(1-\alpha)$% confidence
interval $\text{CI}(\alpha)$ for $\tau(\tilde{w})$:

$\text{CI}(\alpha)=\left[Q_{\alpha/2}\left(\inf_{\tilde{w}\in\sigma(R^{2})}\hat{\tau}^{(b)}(\tilde{w})\right),\
Q_{1-\alpha/2}\left(\sup_{\tilde{w}\in\sigma(R^{2})}\hat{\tau}^{(b)}(\tilde{w})\right)\right].$
(5)

We can estimate $\text{CI}(\alpha)$ directly from the bootstrap samples by
calculating the minimum and maximum adjusted weighted estimate for each
bootstrap iteration, and then estimating the $\alpha/2$ and $1-\alpha/2$-th
percentiles across the bootstrap distributions. The extrema of the adjusted
weighted estimate follow directly from the results of Theorem 3.1:

$\inf_{\tilde{w}\in\sigma(R^{2})}\hat{\tau}(\tilde{w})=\hat{\tau}_{W}-\max_{\tilde{w}\in\sigma(R^{2})}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})\
\ \ \ \
\sup_{\tilde{w}\in\sigma(R^{2})}\hat{\tau}(\tilde{w})=\hat{\tau}_{W}+\max_{\tilde{w}\in\sigma(R^{2})}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})$

As such, to estimate valid confidence intervals, researchers can use a
percentile bootstrap, estimating the bias bound and calculate an adjusted
point estimate in each bootstrap sample. We summarize the steps in Figure 1.

Valid Confidence Intervals for the Variance-Based Sensitivity Model Step 1.
Fix $R^{2}\in[0,1)$ and generate $B$ bootstrap samples of the data. Step 2.
For each bootstrap sample $b=1,...,B$: 1. Estimate weights $\hat{w}_{i}^{(b)}$
and the point estimate $\hat{\tau}_{W}^{(b)}$. 2. Calculate
$\widehat{\text{var}}_{b}(Y_{i})$,
$\widehat{\text{cor}}_{b}(\hat{w}_{i}^{(b)},Y_{i})$, and
$\widehat{\text{var}}_{b}(\hat{w}_{i}^{(b)})$, where the subscript $b$ denotes
the quantity calculated over the $b$-th bootstrap sample. 3. Use the optimal
bias bounds (i.e., Equation (2)) to calculate the range of potential point
estimates:
$\hat{\tau}^{(b)}(\tilde{w})\in\Bigg{[}\hat{\tau}^{(b)}_{W}-\max_{\tilde{w}\in\sigma(R^{2})}\text{Bias}(\hat{\tau}^{(b)}_{W}\mid\tilde{w}),\
\
\hat{\tau}^{(b)}_{W}+\max_{\tilde{w}\in\sigma(R^{2})}\text{Bias}(\hat{\tau}^{(b)}_{W}\mid\tilde{w})\Bigg{]}$
Step 3. From the $B$ bootstrapped optimal bounds, estimate the $\alpha/2$ and
$1-\alpha/2$-th percentiles of the minima and maxima values respectively to
obtain valid confidence intervals (i.e., Equation (5)).

Figure 1: Summary of percentile bootstrap procedure for estimating valid
confidence intervals.

###  3.4 Conducting the Sensitivity Analysis

To conduct the sensitivity analysis, researchers estimate confidence intervals
for increasing $R^{2}$ values until an estimated confidence interval just
contains the null estimate; the corresponding $R^{2}$ is denoted as
$R^{2}_{*}$. Because the $R^{2}$ is bounded on an interval $[0,1]$,
researchers are restricted by the range of values that they can posit for the
different $R^{2}$ values. However, it can nonetheless be difficult to reason
about the plausibility of a given $R^{2}$ value and the strength or weakness
of confounders on the $R^{2}$ scale.

Previous papers have suggested the use of benchmarking to assess what may be
plausible sensitivity parameters (Huang, 2022; Hartman and Huang, 2022;
Cinelli and Hazlett, 2020; Hong et al., 2021; Carnegie et al., 2016; Hsu and
Small, 2013). To perform benchmarking, researchers sequentially omit different
observed covariates and re-estimate the weights. They can then directly
calculate the error that arises from omitting each covariate and directly
estimate the sensitivity parameters (or bounds for the sensitivity
parameters). The estimated sensitivity parameters from omitting each covariate
are then interpreted as the sensitivity parameters for omitted confounders
with equivalent confounding strength to an observed covariate. If researchers
have a strong substantive understanding of covariates that explain a lot of
the variation in the treatment assignment mechanism or outcome, then
benchmarking can be a useful tool for understanding the strength of
hypothetical omitted variables.

We propose a formal benchmarking procedure for the variance-based sensitivity
model. To begin, let there be $p$ total observed covariates (i.e.,
$\mathbf{X}\in\mathbbm{R}^{n\times p}$). Then for the $j$-th covariate, where
$j\in\\{1,...,p\\}$, we define the benchmarked weights $w^{-(j)}$ as the
estimated weights, containing all covariates, except for the $j$-th covariate.
Using $w^{-(j)}$, we can estimate the benchmarked $R^{2}$ value for an omitted
confounder that is equivalently imbalanced as the $j$-th covariate:

$\hat{R}^{2}_{(j)}=\frac{\hat{R}^{2}_{-(j)}}{1+\hat{R}^{2}_{-(j)}},\ \ \ \ \
\text{where
}\hat{R}^{2}_{-(j)}:=1-\frac{\text{var}(w_{i}^{-(j)})}{\text{var}(w_{i})}.$
(6)

$\hat{R}^{2}_{(j)}$ represents the $R^{2}$ value of an omitted variable that
has the same amount of residual imbalance as the $j$-th covariate.444The
reason we cannot directly use $\hat{R}^{2}_{-(j)}$ as an estimate for the
benchmarked $R^{2}$ value comes from the fact that we must adjust for changes
in the baseline variation of the weights between $w$ and $w^{*}$. We refer
readers to Cinelli and Hazlett (2020) and Huang (2022) for more discussion on
this point. More specifically, $R^{2}_{(j)}$ corresponds to an omitted
variable with the same amount of imbalance, after controlling for
$\mathbf{X}$, as the $j$-th covariate, after controlling for
$\mathbf{X}^{-(j)}$.

When interpreting the benchmarking results, it is important to consider that
the magnitude of the benchmarked $R^{2}$ values is determined by the residual
imbalance. More concretely, we consider the variables income and education in
the running example. We expect that both income and education will be
predictive of individuals’ propensity for fish consumption. However, omitting
just income may not result in a very large $R^{2}$ value, because by balancing
education, we have implicitly controlled for some of the imbalance in income.
The benchmarked $R^{2}$ parameter thus represents the setting in which
researchers have omitted a variable that, when controlling for all the other
observed variables, has the same amount of residual imbalance as income after
controlling for education. In cases when researchers wish to consider omitting
a variable similar to a set of collinear variables, they can omit subsets of
variables and perform the same benchmarking exercise.

Formal benchmarking can also be used to assess the plausibility of the event
$R^{2}\geq R^{2}_{*}$. More specifically, we can directly compare the
benchmarked $\hat{R}^{2}_{(j)}$ values for $j\in\\{1,...,p\\}$ with the
estimated $R^{2}_{*}$ to see how much more or less imbalanced an omitted
confounder must be, relative to an observed covariate, in order to result in
an $R^{2}$ value equal to $R^{2}_{*}$. We refer to this as the minimum
relative imbalance (MRI):

$\text{MRI}(j)=\frac{R^{2}_{*}}{\hat{R}^{2}_{(j)}}.$

If the MRI is small (i.e., $\text{MRI}(j)<1$), the omitted confounder need not
be very imbalanced, relative to the $j$-th covariate, in order to make a null
result plausible. In contrast, if the MRI is large (i.e., $\text{MRI}(j)>1$),
then the omitted confounder must be more imbalanced than the $j$-th observed
covariate to make a null result plausible.

Formal benchmarking offers an opportunity for researchers to incorporate their
substantive understanding into the sensitivity analysis and provides much-
needed interpretability for the sensitivity framework. In particular, when
researchers have strong priors about which underlying observed variables
control the treatment assignment mechanism, formal benchmarking is very useful
for reasoning about the plausibility of an omitted confounder strong enough to
explain observed results in the absence of a true effect.

###  3.5 Illustration on NHANES

In our running example, we begin by varying the $R^{2}$ parameter across the
range $[0,1)$, and estimate the corresponding the 95% confidence intervals. We
estimate $R^{2}_{*}=0.57$, such that if $R^{2}\geq 0.57$, the intervals
contain the null estimate. This implies that if the omitted confounder
explains 57% or more of the variation in the true weights, our estimated
effect of fish consumption on blood mercury levels is no longer significantly
different from the expected distribution under the null.

To assess the plausibility of an omitted confounder resulting in an $R^{2}$
value of 0.57, we perform formal benchmarking and estimate benchmarked
$\hat{R}^{2}$ values for each covariate. Omitting a confounder like race,
education, or income results in the largest $R^{2}$ values. More specifically,
omitting a confounder with equivalent confounding strength to race results in
an $R^{2}$ of 0.19, while omitting a confounder with equivalent confounding
strength to education or income results in an $R^{2}$ of 0.17 and 0.14,
respectively. From these results, we see that an omitted confounder would have
to explain around 3 times the variation in true weights as the strongest
observed covariate, race, in order for the $R^{2}$ value to equal the cutoff
value. We argue that while mathematically possible, the plausibility of a
confounder resulting in the threshold $R^{2}_{*}=0.57$ value is low.

Figure 2: Results from the sensitivity analysis under the variance-based
sensitivity model. We vary the $R^{2}$ measure across the $x$-axis and plot
the range of estimated ATT values on the $y$-axis. The solid bar denotes the
point estimate bounds for a specified $R^{2}$ value, estimated as the point
estimate plus and minus the optimal bias bounds (Theorem 3.1). The lighter
intervals represent the 95% confidence intervals. We also plot the
benchmarking results for the observed covariates, where the lines represent
the corresponding benchmarked $R^{2}$ values.

###  3.6 Extension: Bounding a Confounder’s Relationship with the Outcome

Previous literature has highlighted two characteristics of the imbalance term
in Equation (1) that affect the bias from omitting a variable: (1) the overall
magnitude of the the imbalance term, and (2) the relationship between the
imbalance term to the outcomes (e.g., Huang (2022); Hong et al. (2021);
Cinelli and Hazlett (2020); Shen et al. (2011)). Like the marginal sensitivity
model, the variance-based sensitivity model constrain the overall magnitude of
the imbalance term, and implicitly assume that the imbalance is maximally
correlated with the outcome. In settings when researchers wish to account for
this additional characteristic of the imbalance term, the variance-based
sensitivity model can be easily extended to allow researchers to bound the
relationship between the imbalance and the outcome. In particular, unlike the
marginal sensitivity model, in which researchers must solve a linear
programming problem to identify the extrema, there exists a closed-form
solution for the optimal bias bounds under the variance-based sensitivity
model. As such, researchers can choose to evaluate the optimal bias bounds and
associated confidence intervals using less conservative values of the
correlation bound.

While amplification approaches for examining the relationship between the
outcomes and the confounder for a fixed level of imbalance exist for
alternative sensitivity models, many of these methods require introducing
additional complexities. (For example, Rosenbaum and Silber (2009) requires
invoking parametric assumptions on the outcomes.) In contrast, the variance-
based sensitivity model allow researchers to easily incorporate additional
information about the confounder to directly bound the relationship between
the outcome and the imbalance in an omitted confounder. We provide
recommendations for alternative bounds that researchers can use in Appendix
A.4, as well as benchmarking procedure that allows researchers to use observed
covariate data to estimate plausible correlation bounds.

##  4 Relationship to the Marginal Sensitivity Model

We now examine the relationship between the variance-based sensitivity model
and the marginal sensitivity model. We first show that both sets of
sensitivity models can be written as norm-constrained optimization problems.
The variance-based sensitivity model implicitly constrains a weighted $L_{2}$
norm, while the marginal sensitivity model constrains an $L_{\infty}$ norm. We
demonstrate that by moving away from a worst-case characterization of the
error from an omitted variable, the variance-based sensitivity model can
obtain narrower, more informative bounds. We illustrate the potential for
narrower bounds using benchmarked results from the running example.

###  4.1 Sensitivity Models as an Optimization Problem

Re-formulating the variance-based sensitivity model as an optimization problem
under a bounded norm enables comparison with the marginal sensitivity model,
which can be written as bias maximization problems under a constrained
$L_{\infty}$ norm. We argue that constraining the weighted $L_{2}$ norm can
produce less conservative estimated bounds.

To begin, we show that the variance-based sensitivity model is a bias
maximization problem, given a fixed constraint on a weighted $L_{2}$ norm.

######  Theorem 4.1 (Weighted $L_{2}$ Norm Constraint).

Define the individual-level error in the weights as
$\lambda_{i}:=w_{i}^{*}/w_{i}$. Define the $L_{2,w}$ norm as follows:

$||\lambda||^{2}_{2,w}:=\begin{cases}\displaystyle\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}\cdot\nu(w_{i})&\text{if
}\text{var}(w_{i})>0,\\\ \infty&\text{else}\end{cases},$

where $\nu(w_{i})$ is a function of the estimated weights. Then, the variance-
based sensitivity model can equivalently be written as a norm-constrained
optimization problem:

$\max_{\tilde{w}\in\sigma(R^{2})}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})\iff\begin{cases}\displaystyle\max_{\tilde{w}}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})\\\
\displaystyle\text{s.t.
}||\lambda||_{2,w}\leq\sqrt{\frac{k}{1-R^{2}}},\end{cases}$

where $k:=1-R^{2}/\mathbb{E}(w_{i}^{2})$. See Appendix B for proof and
details.

Theorem 4.1 is especially in concert with the following result from Zhao et
al. (2019) showing that the marginal sensitivity model is constrained
$L_{\infty}$ problems:

$\max_{\tilde{w}\in\varepsilon(\Lambda)}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})\iff\begin{cases}\displaystyle\max_{\tilde{w}}\
\ \text{Bias}(\hat{\tau}_{W}\mid\tilde{w})\\\ \displaystyle\text{s.t.
}\Lambda^{-1}\leq||\lambda||_{\infty}\leq\Lambda.\end{cases}$

These constrained-norm representations provide insight into the benefits
expected from the variance-based sensitivity model. Because the marginal
sensitivity model optimize over the set of weights defined by a worst-case
error, the estimated bounds on the bias always correspond to cases in which
all units are exposed to this worst-case error. However, in settings when one
or two subjects are subject to much larger levels of confounding than others,
this can result in an overly pessimistic view of the potential bias (Fogarty
and Hasegawa, 2019; Zhao et al., 2019). In contrast, the variance-based
sensitivity model is optimizing over a set of weights defined by average
weighted error, and thus allow a small number of weights to be exposed to
large amounts of error, even at moderate levels of overall confounding.

###  4.2 Comparison of Estimated Bounds

While the constrained-norm representation provides intuition for why the
variance-based sensitivity model may obtain narrower bounds than the marginal
sensitivity model, in practice it is difficult to directly compare the bounds
estimated under the two families of models. This is because the two approaches
are using two different parameters and are fundamentally characterizing the
error from omitting a confounder in a different manner. In the following
subsection, we consider a setting in which researchers can estimate bounds
using the true sensitivity parameter and compare the size of the associated
confidence intervals. While in practice, researchers do not have access to the
true sensitivity parameters, this approach provides intuition for the relative
performances of the two sensitivity models.

First, we formalize a condition under which the variance-based sensitivity
model will result in narrower bounds than the marginal sensitivity model. In
general, we expect the variance-based sensitivity model to result in narrower
bounds if the worst-case error ( $\Lambda$) is much larger than the true
average weighted error (proxied by $R^{2}$). If the difference in the worst-
case error and average weighted error is not very large, then there will not
be much improvement in the estimated bounds from using the variance-based
sensitivity model. Theorem 4.2 provides a maximum threshold for the size of
$R^{2}$ relative to the $\Lambda$ value sufficient for strictly narrower
bounds under the variance-based sensitivity model.

######  Theorem 4.2 (Narrower Bounds under the Variance-Based Sensitivity
Model).

Let $\psi(\Lambda)$ represent the difference in the estimated point estimate
bounds under the marginal sensitivity model $\varepsilon(\Lambda)$ for a given
$\Lambda\geq 1$:

$\psi(\Lambda):=\max_{\tilde{w}\in\varepsilon(\Lambda)}\frac{\sum_{i:Z_{i}=0}Y_{i}Z_{i}\tilde{w}_{i}}{\sum_{i:Z_{i}=0}Z_{i}\tilde{w}_{i}}-\min_{\tilde{w}\in\varepsilon(\Lambda)}\frac{\sum_{i:Z_{i}=0}Y_{i}Z_{i}\tilde{w}_{i}}{\sum_{i:Z_{i}=0}Z_{i}\tilde{w}_{i}}.$

Then if the true $R^{2}$ parameter is lower than the following threshold,

$R^{2}\leq\frac{\psi(\Lambda)^{2}}{4\underbrace{(1-\text{cor}(w_{i},Y_{i}\mid
Z_{i}=0)^{2})}_{\text{Correlation Bound}}\cdot\underbrace{\text{var}(w_{i}\mid
Z_{i}=0)\text{var}(Y_{i}\mid Z_{i}=0)}_{\text{Scaling
Factor}}+\psi(\Lambda)^{2}},$ (7)

the bounds under the variance-based sensitivity model will be narrower than
the bounds for the marginal sensitivity model.

Besides the worst-case error $\Lambda$, the $R^{2}$ threshold is determined by
the correlation between the estimated weights and the outcome,
$\text{cor}(w_{i},Y_{i}\mid Z_{i}=0)$, and the scaling factor,
$\text{var}(w_{i}\mid Z_{i}=0)\cdot\text{var}(Y_{i}\mid Z_{i}=0)$. These
components affect the estimated bounds under both sets of sensitivity models.
Both $\text{cor}(w_{i},Y_{i}\mid Z_{i}=0)$ and the scaling factor are direct
inputs into the optimal bias bounds under the variance-based sensitivity
model. In addition, increases in these quantities either lead to larger
outcome values, or more extreme weights; because the optimal bounds under the
marginal sensitivity model are estimated by scaling the weights and outcomes
by $\Lambda$ (or $\Lambda^{-1}$), $\psi(\Lambda)$ will also increase.

Theorem 4.2 does not guarantee that the variance-based sensitivity model will
always result in narrower bounds than the marginal sensitivity model, but we
consider two specific scenarios that highlight the practical advantages from
using variance-based sensitivity model. First, we consider an asymptotic
setting. We show that in many cases, the worst-case error $\Lambda$ will
diverge to infinity, regardless of the omitted variable’s confounding
strength. In contrast, the $R^{2}$ parameter is a direct function of the
confounding strength and retains a more stable interpretation across different
data scales.

Second, we consider a finite-sample setting, in which the outcomes and
probability of treatment are highly correlated (which we refer to as limited
outcome overlap). In this setting, the marginal sensitivity model may produce
narrower intervals, but these intervals can be misleadingly narrow and fail to
provide nominal coverage. In contrast, while the intervals under the variance-
based sensitivity model will be wider in this setting, the intervals
adequately account for the limited outcome overlap and continue to provide
nominal coverage.

##### Remark.

Several recent extensions of the marginal sensitivity model allow researchers
to mitigate some the conservative nature of the method by adding in additional
constraints beyond bounding the worst-case error (Kallus and Zhou, 2018; Dorn
and Guo, 2021; Dorn et al., 2021). However, these methods usually require
adding additional sensitivity parameters or performing some form of outcome
modeling. The variance-based sensitivity model offers stable and informative
bounds via a one-parameter sensitivity analysis without additional
assumptions, constraints, or complexities.

####  4.2.1 Infinite Worst-Case Error in Asymptotic Settings

We begin by considering the asymptotic setting. We show that when the omitted
confounder results in an error that can be arbitrarily small or large and the
outcomes $Y_{i}$ are unbounded, the asymptotic confidence intervals for the
variance-based sensitivity model will necessarily be narrower than the
confidence intervals estimated under the marginal sensitivity model. Consider
the following instructive example.

######  Example 4.1 (Behavior of $\Lambda$ for a Logit Model).

Assume researchers use a logit model to estimate the weights using
$\mathbf{X}_{i}$, but omit a confounder $U_{i}$. The estimated and ideal
weights take on the following forms:

$\hat{w}_{i}=\exp(\hat{\gamma}^{\top}\mathbf{X}_{i})\ \ \ \ \ \ \ \ \ \
\hat{w}_{i}^{*}=\exp(\hat{\gamma}^{*\top}\mathbf{X}_{i}+\hat{\beta}U_{i})$

Then let $\hat{\Lambda}$ be the maximum error across our observed sample
(i.e., $\hat{\Lambda}:=\max_{1\leq i\leq
n}\\{\hat{w}_{i}^{*}/\hat{w}_{i},\hat{w}_{i}/\hat{w}_{i}^{*}\\}$). Assume
$\begin{bmatrix}\mathbf{X}_{i},U_{i}\end{bmatrix}\stackrel{{\scriptstyle
iid}}{{\sim}}MVN(0,I)$. Then $\mathbb{E}(\hat{\Lambda})\to\infty$ as
$n\to\infty$:

$\lim_{n\to\infty}\frac{\mathbb{E}(\hat{\Lambda})}{\exp(\sqrt{2\nu^{2}\log(n)})}\geq
1,$

where $\nu^{2}=(\gamma^{*}-\gamma)^{\top}(\gamma^{*}-\gamma)+\beta^{2}$ (where
$\gamma^{*}$, $\gamma$, and $\beta$ represent the population counterparts to
the estimated coefficients), and the results follow immediately from
Wainwright (2019).

Example 4.1 highlights that the worst-case error will increase towards
infinity as the sample size grows, regardless of the confounding strength of
the omitted variable (represented by $\hat{\beta}$). The omitted confounder
could explain little of the treatment assignment process, but the worst-case
bound on the error would still be infinitely large. This means that
researchers would have to specify an infinitely large $\Lambda$ value for the
marginal sensitivity model to be valid. In other words, while we can find a
$\hat{\Lambda}$ in an observed sample that provides an upper bound on the
difference between the realized and the ideal weights, the marginal
sensitivity model will not hold for any value of $\Lambda$ in the population
(Jin et al., 2022). Intuitively, this occurs because $\hat{\Lambda}$ is a
function of the largest $U_{i}$ value in the sample: as the sample size
increases, the probability of observing an extreme $U_{i}$ value increases.

A secondary issue arises from the decoupling of the magnitude of the
sensitivity parameter and the underlying confounding strength of the omitted
variable. In particular, reasoning about whether or not $\Lambda$ values is
large or small is can be challenging. Even with the aid of benchmarking,
researchers can at best estimate the worst-case error that arises from
omitting different covariates, but reasoning about whether or not it is
plausible for such an error to arise from an omitted variable amounts to
reasoning about whether or not it is plausible for potential outliers to
occur. Furthermore, as the sample size increases, researchers must take into
account the potential for more outliers to occur.

In contrast, we can calculate the $R^{2}$ value for the variance-based
sensitivity model under the same setting as Example 4.1. It is a function of
$\hat{\gamma}$, $\hat{\gamma}^{*}$, and $\hat{\beta}$ that does not depend on
the sample size.

######  Example 4.2 (Behavior of $R^{2}$ for a Logit Model).

Consider the same setting as Example 4.1. Then, the $R^{2}$ value can be
written as follows:

$R^{2}=1-\frac{\exp(\hat{\gamma}^{\top}\hat{\gamma})-1}{\exp(\hat{\gamma}^{*\top}\hat{\gamma}^{*}+\hat{\beta}^{2})-1}\cdot\frac{\exp(\hat{\gamma}^{\top}\hat{\gamma})}{\exp(\hat{\gamma}^{*\top}\hat{\gamma}^{*}+\hat{\beta}^{2})}.$

Example 4.1 gives one setting in which $\Lambda$ will be infinitely large,
regardless of the confounding strength of the omitted variable. More
generally, the following corollary to Theorem 4.2 shows that under any setting
when the error from omitting a confounder can take on values that are
arbitrarily small or large, the variance-based sensitivity model necessarily
produces narrower bounds in sufficiently large samples.

######  Corollary 4.1 (Narrower Bounds under the Variance-Based Sensitivity
Model).

Consider the set of confounders, in which for all $\delta>0$,
$P(w^{*}_{i}/w_{i}<\delta)>0$, or $P(w^{*}_{i}/w_{i}>\delta)>0$. Then, if the
outcomes are unbounded, $\psi(\Lambda)$ will diverge in probability to
infinity, and the threshold from Theorem 4.2 will converge in probability to
1:

$\frac{\psi(\Lambda)^{2}}{4(1-\text{cor}(w_{i},Y_{i})^{2})\cdot\text{var}(w_{i})\text{var}(Y_{i})+\psi(\Lambda)^{2}}\stackrel{{\scriptstyle
p}}{{\rightarrow}}1$

Because $R^{2}<1$ by definition, for sufficiently large $n$, the variance-
based sensitivity model will produce narrower bounds.

Corollary 4.1 highlights that in certain asymptotic settings, if the outcomes
are unbounded, the marginal sensitivity models will result in infinitely large
bias bounds. This will occur, regardless of whether the omitted confounder is
strong or weak. In contrast, because the variance-based sensitivity model is
not using a worst-case characterization of error, the resulting bias bounds
will be less susceptible to extreme values and be narrower by construction.

####  4.2.2 Limited Overlap in Finite-Samples

We now consider a finite-sample setting. We show that paradoxically, in
certain finite-samples, the marginal sensitivity model can result in narrower
bounds than the variance-based sensitivity model even when $\Lambda$ is very
large. However, the narrower bounds come with a risk of substantial
undercoverage, especially when the sample size is small or there is limited
outcome overlap. In these settings, the variance-based sensitivity model will
tend to return wider intervals, but maintain nominal coverage.

The key to this phenomenon is a property of the marginal sensitivity model,
referred to as sample boundedness. Sample boundedness implies that even at
infinitely large $\Lambda$ values, the worst-case bounds under the marginal
sensitivity model approach, but cannot exceed, the range of the observed
control outcomes (i.e.,
$\lim_{\Lambda\to\infty}\psi(\Lambda)\leq\max_{i:Z_{i}=0}Y_{i}-\min_{i:Z_{i}=0}Y_{i}$).

In contrast, the variance-based sensitivity model is not inherently sample
bounded. In settings with relatively large amounts of confounding, the
marginal sensitivity model will have narrower intervals than the variance-
based sensitivity model, since as $R^{2}$ increases towards 1, the estimated
bounds under the variance-based sensitivity model will be adequately wide.
However, sample boundedness may prohibit the construction of valid confidence
intervals unless in the absence of a key implicit assumption on the
distribution of the unobserved potential outcomes. Consider the following toy
example:

######  Example 4.3 (Misleading Optimism from Sample Boundedness).

Consider the following population of 4 units, with the following potential
outcomes, treatment assignment, and the estimated probability of treatments
for each unit:

$i$ | $Y_{i}(0)$ | $Y_{i}(1)$ | $\hat{P}(Z_{i}=1)$ | $Z_{i}$
---|---|---|---|---
1 | -10 | -10 | 0.1 | 0
2 | 5 | 5 | 0.2 | 0
3 | 10 | 10 | 0.9 | 1
4 | 20 | 20 | 0.95 | 1

The true ATT is zero, but the estimated ATT is equal to 14.6, so substantial
confounding is present. However, since the sample bounds for the ATT are the
interval $[10,25]$, no value of $\Lambda$ can produce an estimated interval
(under the marginal sensitivity model) containing zero, erroneously suggesting
the presence of a true effect highly robust to substantial confounding.

While this example is somewhat contrived, it highlights the problems with
sample boundedness if the potential outcome ranges in the two groups have
limited overlap, which may occur when potential outcomes are strongly
correlated with the probability of treatment. For a formal characterization of
this outcome overlap condition, see Appendix A.5.

When there exists limited outcome overlap, estimated intervals from the
marginal sensitivity model may be misleadingly optimistic, especially for
dramatic levels of potential confounding, but intervals constructed under the
variance-based sensitivity model, which are not sample bounded, are not
affected.

Figure 3 illustrates the behavior and coverage rates of both sets of
sensitivity models under varying amounts of outcome overlap and sample sizes
in an empirical example, described in greater detail in Appendix A.5.

Figure 3: Coverage rates for the marginal sensitivity model and the variance-
based sensitivity model, assuming an oracle bias setting when researchers have
full knowledge of the true underlying sensitivity parameter. The $x$-axis
represents the amount of outcome overlap between the treatment and control
groups (i.e., as $\sigma^{2}_{v}$, there is more outcome overlap).

##### Remark.

We note that sample boundedness is not necessarily a negative feature in the
context of estimation. The bias-variance tradeoff of using a stabilized
weighted estimator has been extensively studied (e.g., Robins et al. (2007)).
However, in the context of a sensitivity analysis, in which we are explicitly
interested in examining the potential bias that can arise under varying levels
of confounding, requiring sample boundedness can lead to misleading
conclusions, and potential issues with outcome overlap should be considered
carefully when interpreting results.

###  4.3 Illustration on NHANES

We now conduct formal benchmarking for the variance-based models and the
marginal sensitivity model in our running example. We then estimate the
corresponding bounds and intervals under both approaches. See Figure 4 for a
visualization. We see that for each of the covariates, omitting a confounder
like any of the observed covariates would result in wider bounds under the
marginal sensitivity model than the variance-based models.

Notably, under the marginal sensitivity model, omitting a confounder like
education would be sufficient to explain the entire observed effect under the
null hypothesis of no effect. In contrast, under the variance-based
sensitivity model, omitting a confounder with equivalent confounding strength
to any of the observed covariates would not be sufficient to explain the
observed data under the null. While the average error from omitting a variable
like education is relatively low, the maximum error that occurs is relatively
large. The marginal sensitivity model, which assumes such a maximal error
could occur in the unobserved confounder for most or all data points, thus
produces much wider intervals than the variance-based model, which is much
less responsive to individual outliers.

Figure 4: Estimated intervals for both the marginal sensitivity model (in
yellow) and the variance-based sensitivity model (in light blue), and the
variance-based sensitivity model, using a less conservative correlation bound
(in dark blue). The darker intervals represent the point estimate bounds,
while the lighter intervals represent the 95% confidence intervals. The
intervals are estimated using the benchmarked $\Lambda$ and $R^{2}$ values for
each covariate, and are interpreted as the resulting intervals for an omitted
confounder with equivalent confounding strength to the observed covariate.

We also estimate intervals (and bounds) under the variance-based sensitivity
model using a relaxed correlation bound. In particular, we choose the
correlation bound by benchmarking an optional correlation parameter, giving
the correlation between the outcome and the imbalance in an omitted
confounder, to the observed correlation between the outcome and each observed
covariate. (See Appendix A.4 for more details.) By accounting for the
relationship between the confounder and the outcome, we are able to obtain
much narrower intervals. In particular, we see that even in cases where a
potential omitted confounder is highly imbalanced (e.g., omitting a confounder
like age results in an $R^{2}$ value of 0.12, and $\Lambda$ value of 2.1), the
overall bias that occurs from omitting it may be relatively low if the
imbalance is largely unrelated to the outcome. By considering this additional
dimension of the bias–which can be easily done using the variance-based
sensitivity model– researchers are able to better characterize the types of
confounders that may lead to large amounts of bias and obtain a more holistic
understanding of the sensitivity in their estimated effects.

##  5 Conclusion

We have introduced a novel sensitivity model, the variance-based sensitivity
model, which characterizes the error from omitting a confounder by using the
distributional differences between the estimated weights and true weights. We
show that the variance-based sensitivity model can be parameterized using an
$R^{2}$ measure that represents the degree of residual imbalance in an omitted
confounder, and provide methods for benchmarking the $R^{2}$ value of an
omitted confounder against residual imbalances for observed covariates. We
also derive a closed-form solution for the maximum possible bias and introduce
a method for estimation of asymptotically valid confidence intervals under the
sensitivity model.

The variance-based sensitivity model has several notable advantages over the
existing marginal sensitivity model. First, by characterizing bias in terms of
distributional differences instead of a worst-case error bound, variance-based
sensitivity model can estimate narrower, less conservative bounds. Second, we
show empirically that the variance-based sensitivity model obtains nominal
coverage even in finite sample settings where the standard marginal
sensitivity model dramatically undercovers due to issues with outcome overlap.
Finally, because the variance based sensitivity model admits a closed-form
solution for the optimal bias, we can introduce a natural two-parameter
extension that uses constraints on the relationship between the omitted
confounder and the outcome to produce narrower bounds.

We suggest several directions for future work. First, we demonstrated that
variance-based sensitivity model, like the marginal sensitivity model, can be
written as a norm-constrained optimization problem. Exploring other possible
norms under which to constrain unobserved confounding could lead to a broad
unified sensitivity framework, helping contextualize a wider variety of
different sensitivity methods with their own strengths and weaknesses.

Second, it is natural to ask what factors under a researcher’s control at the
design stage may influence the degree of robustness to unmeasured bias
exhibited under the variance-based sensitivity analysis. While the closed form
for the optimal bias bound already provides insights in this direction,
developing a metric akin to design sensitivity for matched studies (Rosenbaum,
2004, 2010) would provide valuable further guidance about how to design
weighting estimators for maximum robustness.

Finally, while we focused on a choice between bounding a weighted average
error and bounding a worst-case error, future work could incorporate both
constraints in the same study. We anticipate that this would result in further
narrowing of sensitivity bounds.

## References

  * Aronow and Lee (2013) Aronow, P. M. and D. K. Lee (2013).  Interval estimation of population means under unknown but bounded probabilities of sample selection.  Biometrika 100(1), 235–240.
  * Ben-Michael et al. (2020) Ben-Michael, E., D. Hirschberg, A. Feller, and J. Zubizarreta (2020).  The balancing act for causal inference.
  * Carnegie et al. (2016) Carnegie, N. B., M. Harada, and J. L. Hill (2016).  Assessing sensitivity to unmeasured confounding using a simulated potential confounder.  Journal of Research on Educational Effectiveness 9(3), 395–420.
  * Cinelli and Hazlett (2020) Cinelli, C. and C. Hazlett (2020).  Making sense of sensitivity: Extending omitted variable bias.  Journal of the Royal Statistical Society: Series B (Statistical Methodology) 82(1), 39–67.
  * Dahabreh et al. (2019) Dahabreh, I. J., J. M. Robins, S. J. Haneuse, I. Saeed, S. E. Robertson, E. A. Stuart, and M. A. Hernán (2019).  Sensitivity analysis using bias functions for studies extending inferences from a randomized trial to a target population.  arXiv preprint arXiv:1905.10684.
  * Dorn and Guo (2021) Dorn, J. and K. Guo (2021).  Sharp sensitivity analysis for inverse propensity weighting via quantile balancing.  arXiv preprint arXiv:2102.04543.
  * Dorn et al. (2021) Dorn, J., K. Guo, and N. Kallus (2021).  Doubly-valid/doubly-sharp sensitivity analysis for causal inference with unmeasured confounding.  arXiv preprint arXiv:2112.11449.
  * Fogarty and Hasegawa (2019) Fogarty, C. B. and R. B. Hasegawa (2019).  Extended sensitivity analysis for heterogeneous unmeasured confounding with an application to sibling studies of returns to education.  The Annals of Applied Statistics 13(2), 767–796.
  * Hainmueller (2012) Hainmueller, J. (2012).  Entropy balancing for causal effects: A multivariate reweighting method to produce balanced samples in observational studies.  Political analysis 20(1), 25–46.
  * Hartman et al. (2021) Hartman, E., C. Hazlett, and C. Sterbenz (2021).  Kpop: A kernel balancing approach for reducing specification assumptions in survey weighting.  arXiv preprint arXiv:2107.08075.
  * Hartman and Huang (2022) Hartman, E. and M. Huang (2022).  Sensitivity analysis for survey weights.  arXiv preprint arXiv:2206.07119.
  * Hong et al. (2021) Hong, G., F. Yang, and X. Qin (2021).  Did you conduct a sensitivity analysis? a new weighting-based approach for evaluations of the average treatment effect for the treated.  Journal of the Royal Statistical Society: Series A (Statistics in Society) 184(1), 227–254.
  * Hsu and Small (2013) Hsu, J. Y. and D. S. Small (2013).  Calibrating sensitivity analyses to observed covariates in observational studies.  Biometrics 69(4), 803–811.
  * Huang (2022) Huang, M. (2022).  Sensitivity analysis in the generalization of experimental results.  arXiv preprint arXiv:2202.03408.
  * Huang et al. (2022) Huang, M. Y., B. G. Vegetabile, L. F. Burgette, C. Setodji, and B. A. Griffin (2022).  Higher moments matter for optimal balance weighting in causal estimation.  Epidemiology (Cambridge, Mass.).
  * Imbens and Manski (2004) Imbens, G. W. and C. F. Manski (2004).  Confidence intervals for partially identified parameters.  Econometrica 72(6), 1845–1857.
  * Jin et al. (2022) Jin, Y., Z. Ren, and Z. Zhou (2022).  Sensitivity analysis under the $f$-sensitivity models: Definition, estimation and inference.  arXiv preprint arXiv:2203.04373.
  * Kallus and Zhou (2018) Kallus, N. and A. Zhou (2018).  Confounding-robust policy improvement.  Advances in neural information processing systems 31.
  * Kosorok (2008) Kosorok, M. R. (2008).  Introduction to empirical processes and semiparametric inference. Springer.
  * Nie et al. (2021) Nie, X., G. Imbens, and S. Wager (2021).  Covariate balancing sensitivity analysis for extrapolating randomized trials across locations.  arXiv preprint arXiv:2112.04723.
  * Olkin (1981) Olkin, I. (1981).  Range restrictions for product-moment correlation matrices.  Psychometrika 46(4), 469–472.
  * Resnick (2008) Resnick, S. I. (2008).  Extreme values, regular variation, and point processes, Volume 4.  Springer Science & Business Media.
  * Robins et al. (2007) Robins, J., M. Sued, Q. Lei-Gomez, and A. Rotnitzky (2007).  Comment: Performance of double-robust estimators when “inverse probability” weights are highly variable.  Statistical Science 22(4), 544–559.
  * Rosenbaum (2004) Rosenbaum, P. R. (2004).  Design sensitivity in observational studies.  Biometrika 91(1), 153–164.
  * Rosenbaum (2010) Rosenbaum, P. R. (2010).  Design sensitivity and efficiency in observational studies.  Journal of the American Statistical Association 105(490), 692–702.
  * Rosenbaum and Rubin (1983) Rosenbaum, P. R. and D. B. Rubin (1983).  Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome.  Journal of the Royal Statistical Society: Series B (Methodological) 45(2), 212–218.
  * Rosenbaum and Silber (2009) Rosenbaum, P. R. and J. H. Silber (2009).  Amplification of sensitivity analysis in matched observational studies.  Journal of the American Statistical Association 104(488), 1398–1405.
  * Rubin (1980) Rubin, D. B. (1980).  Discussion of ‘Randomization analysis of experimental data: The Fisher randomization test comment’ by Basu.  Journal of the American Statistical Association 75(371), 591–593.
  * Shen et al. (2011) Shen, C., X. Li, L. Li, and M. C. Were (2011).  Sensitivity analysis for causal inference using inverse probability weighting.  Biometrical Journal 53(5), 822–837.
  * Soriano et al. (2021) Soriano, D., E. Ben-Michael, P. J. Bickel, A. Feller, and S. D. Pimentel (2021).  Interpretable sensitivity analysis for balancing weights.  arXiv preprint arXiv:2102.13218.
  * Tan (2006) Tan, Z. (2006).  A distributional approach for causal inference using propensity scores.  Journal of the American Statistical Association 101(476), 1619–1637.
  * Wainwright (2019) Wainwright, M. J. (2019).  High-dimensional statistics: A non-asymptotic viewpoint, Volume 48.  Cambridge University Press.
  * Zhang and Zhao (2022) Zhang, Y. and Q. Zhao (2022).  Bounds and semiparametric inference in $l_{\infty}$ and $l_{2}$-sensitivity analysis for observational studies.  arXiv preprint arXiv:2211.04697.
  * Zhao et al. (2019) Zhao, Q., D. S. Small, and B. B. Bhattacharya (2019).  Sensitivity analysis for inverse probability weighting estimators via the percentile bootstrap.  Journal of the Royal Statistical Society: Series B (Statistical Methodology) 81(4), 735–761.
  * Zhao et al. (2018) Zhao, Q., D. S. Small, and P. R. Rosenbaum (2018).  Cross-screening in observational studies that test many hypotheses.  Journal of the American Statistical Association 113(523), 1070–1084.

Supplementary Materials:
Variance-based Sensitivity Analysis for Weighting Estimators Result in More
Informative Bounds

##  Appendix A Additional Discussion

###  A.1 Missingness

In the man manuscript, the estimand of interest is the average treatment
effect, across the treated. However, we note that the sensitivity framework
introduced can be applied to more general settings, in which we consider
missingness conditionally at random:

$Y_{i}\ \raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\
A_{i}\mid\mathcal{X}$

This provides a very flexible framework to consider many settings of interest.
Table 2 summarizes several settings of interest, along with the associated
conditional ignorability assumption to be relaxed by sensitivity analysis.

Setting | Missingness Indicator | Ignorability Statement
---|---|---
Survey Response |  $R_{i}$ (Response) | $Y_{i}\ \raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\ R_{i}\mid\mathcal{X}$
Internal Validity |  $Z_{i}$ (Treatment Assignment) | $Y_{i}(1),Y_{i}(0)\ \raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\ Z_{i}\mid\mathcal{X}$
External Validity |  $S_{i}$ (Inclusion in Experimental Sample) | $Y_{i}(1)-Y_{i}(0)\ \raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\ S_{i}\mid\mathcal{X}$
Table 2: Summary of different common missingness settings.

###  A.2 Relationship with Extensions for Sharper Bounds

Recently, several papers have demonstrated that the worst-case bounds derived
under the marginal sensitivity model result in $w^{*}$ that fail to recover
the causal estimand. Thus, these worst-case bounds tend to be unnecessarily
conservative, and may not necessarily be sharp. Both Dorn and Guo (2021) and
Nie et al. (2021) introduce additional optimization constraints for slightly
tighter (and sometimes sharp) bounds. However, these additional optimization
constraints come at a cost. In particular, the approach in Dorn and Guo (2021)
require imposing parametric assumptions of the conditional quantiles of the
outcomes; furthermore, the method does not accommodate discrete outcome
variables. Nie et al. (2021) include additional balancing constraints when
solving for the bounds; however, Dorn and Guo (2021) show that doing so can
result in unstable performance in finite-sample settings. For this paper, we
restrict our discussion to the marginal sensitivity model, but similar
comparisons could be made to extensions of these models. Because these methods
are all extensions of the marginal sensitivity model, the shortcomings and
drawbacks that are discussed about the marginal sensitivity model similarly
apply to these approaches as well.

###  A.3 Parametric Assumption of Conditional Ignorability

In practice, when researchers estimate weights, they are implicitly assuming a
parametric version of Assumption 1. Following Hartman et al. (2021), we
formalize the parametric version of Assumption 1:

######  Assumption 3 (Linear ignorability in $\phi(\mathbf{X})$).

Let $\phi(\cdot)$ be a feature mapping of $\mathbf{X}_{i}$. Then, write the
outcome $Y_{i}$ as follows:

$Y_{i}=\phi(\mathbf{X}_{i})^{\top}\beta+\delta_{i}$

Similarly, write $P(Z_{i}=1\mid\mathbf{X}_{i})$ as follows:

$\displaystyle
Pr(Z_{i}=1|\mathbf{X}_{i})=g(\phi(\mathbf{X}_{i})^{\top}\theta+\eta_{i}),$

where $g(\cdot):\mathcal{R}\mapsto[0,1]$. Then, linear ignorability holds when
$\delta_{i}\raisebox{0.50003pt}{\rotatebox[origin={c}]{90.0}{$\models$}}\eta_{i}$.

Linear ignorability in $\phi(\mathbf{X}_{i})$ implies that the part of the
outcome that is orthogonal to $\phi(\mathbf{X}_{i})$ is independent to the
part of the treatment assignment process that is orthogonal to
$\phi(\mathbf{X}_{i})$.

The distinction between the non-parametric version of conditional ignorability
(i.e., Assumption 1) and the parametric version (i.e., Assumption 3) arises
from the types of violations that matter for omitted variable bias. Under the
non-parametric version of conditional ignorability, only variables that are
fully unobserved (or omitted) will result in bias. However, under Assumption
3, in addition to including all of the correct variables, the choice of
feature mapping also matters. For example, if researchers only include first-
order moments in their weights estimation, then
$\phi(\mathbf{X}_{i})=\mathbf{X}_{i}$. However, if the true feature map
necessary for linear ignorability to hold also includes higher-order terms or
non-linear interactions between covariates, then using only the first-order
moments will result in bias (Huang et al., 2022). As such, omitted variables
in such a setting would also include any transformations of existing
covariates that have not been explicitly accounted for in the estimated
weights. We refer readers to Hartman and Huang (2022) for more discussion
about the two assumptions in the context of sensitivity analysis. We note that
the proposed sensitivity framework is valid, regardless of which version of
conditional ignorability researchers are interested in using.

###  A.4 Moving Away from Worst-Case Correlation Bounds

Theorem 3.1 allows researchers to calculate the maximum bias that can occur
for a fixed $R^{2}$. This is done by assuming the correlation between the
imbalance in the omitted confounder is maximally correlated with the outcome.
This can be conservative in practice. We provide several recommendations for
researchers who may wish to relax this bound. Doing so can result in narrower
bounds, at the cost of having to reason about an additional parameter.
Throughout this section, we will refer to the correlation bound as $\rho^{*}$,
such that the maximum bias is written as:

$\rho^{*}\cdot\sqrt{\frac{R^{2}}{1-R^{2}}\cdot\text{var}(Y_{i}\mid
A_{i}=1)\cdot\text{var}(w_{i}\mid A_{i}=1)}$

We suggest several different approaches for researchers to estimate less
conservative bounds.

##### Estimating Bounds using Relative Correlation

Applying the results from Huang (2022), we can decompose the correlation
between the imbalance and the outcome into a function of the $R^{2}$ value,
the correlation between the estimated weights and the outcomes, and the
correlation between the true weights and the outcomes:

$\text{cor}(w_{i},Y_{i})\sqrt{\frac{1-R^{2}}{R^{2}}}-\text{cor}(w_{i}^{*},Y_{i})\cdot\sqrt{\frac{1}{R^{2}}}$
(8)

As such, an intuitive way to evaluate bounds for the correlation term is to
posit a bound for the correlation between the true weights and the outcomes by
a relative scaling constant $k$:

$k:=\frac{\text{cor}(w^{*}_{i},Y_{i})}{\text{cor}(w_{i},Y_{i})},$

where $k$ represents how many more times correlated the true weights are to
the outcomes, relative to the estimated weights. $k$ will be naturally upper-
bounded at $1/\text{cor}(w_{i},Y_{i})$. Using Equation (8), researchers can
then obtain a new upper bound for $\rho^{*}$:

$\rho^{*}\leq\frac{\text{cor}(w_{i},Y_{i})}{\sqrt{R^{2}}}\left(\sqrt{1-R^{2}}-k\right)$

It is worth noting that the correlation bound will change, depending on the
$R^{2}$ parameter.

##### Benchmarking the Correlation Term

In practice, researchers may also perform formal benchmarking to estimate what
may be plausible correlation values. More specifically, researchers can
calculate the error from omitting the $j$-th covariate and evaluate the
correlation between the residual imbalance in the $j$-th covariate and the
outcome, using this as the upper bound for $\rho^{*}$:

$\rho^{*}_{(j)}\leq\widehat{\text{cor}}(w_{i}-w_{i}^{-(j)},Y_{i}\mid A_{i}=1)$

Evaluating the bias at $\rho^{*}_{(j)}$ and $\hat{R}^{2}_{(j)}$ provides
researchers with an estimate of the bias if they omitted a confounder with
residual imbalance that is (1) equivalent in magnitude as the residual
imbalance of the $j$-th covariate, and (2) equivalently as correlated with the
outcome as the residual imbalance of the $j$-th covariate. Researchers can
then estimate the associated confidence intervals by fixing both the
correlation term and $R^{2}$.

###  A.5 Extended discussion for sample boundedness

######  Proposition A.1 (Necessary Condition for Validity of Sample Bounds).

Define $\mathcal{A}$ as the set of all observed $Y_{i}$ values across the
sample $A_{i}=1$. For sample boundedness to be true (i.e.,
$\mathbb{E}(Y_{i}\mid
A_{i}=0)\in[\min_{i:A_{i}=1}Y_{i},\max_{i:A_{i}=1}Y_{i}])$, the expectation of
the outcomes not contained in the sample range must be constrained by the
following:

$\displaystyle\mathbb{E}(Y_{i}$ $\displaystyle\mid
A_{i}=0,Y_{i}\not\in\mathcal{A})\in$
$\displaystyle\left[\frac{1}{1-p_{\mathcal{A}}}\min_{i:A_{i}=1}Y_{i}-\frac{p_{\mathcal{A}}}{1-p_{\mathcal{A}}}\max_{i:A_{i}=1}Y_{i},\
\frac{1}{1-p_{\mathcal{A}}}\max_{i:A_{i}=1}Y_{i}-\frac{p_{\mathcal{A}}}{1-p_{\mathcal{A}}}\min_{i:A_{i}=1}Y_{i}\right],$

where $p_{\mathcal{A}}:=P(Y_{i}\in\mathcal{A}\mid A_{i}=0)$ represents the
proportion of unobserved outcomes that fall within the observed sample range.

The bound specified above represents how much overlap there must exist in the
observed and unobserved potential outcomes. The bound is a function of (1) the
proportion of unobserved units with outcomes that are outside the range of
outcomes across the observed sample units (i.e.,
$1-p_{\mathcal{A}}=P(Y_{i}\not\in\mathcal{A}\mid A_{i}=0)$), and (2) the
sample bounds. If a small proportion of the outcomes in the unobserved
population fall outside the sample bounds, then the bound will be relatively
wide. However, if a large proportion of outcomes in the unobserved population
fall outside the sample bounds, then the bound will be more narrow.

We also simulate the behavior of both sensitivity models under varying amounts
of overlap.

######  Example A.1 (Coverage Rates in Limited Outcome Overlap Settings).

Define the treatment assignment mechanism as a logit model, and the outcome
model as a linear model:

$P(Z_{i}=1\mid\mathcal{X})\propto\frac{\exp(\gamma_{1}X_{i,1}+\gamma_{2}X_{i,2}+\beta
U_{i})}{1+\exp(\gamma_{1}X_{i,1}+\gamma_{2}X_{i,2}+\beta U_{i})}\ \ \ \ \ \ \
\ Y_{i}=\gamma_{1}X_{i,1}+\gamma_{2}X_{i,2}+\beta U_{i}+v_{i},$

where $X_{i,1},X_{i,2}$ and $U_{i}$ are standard normal random variables, and
$v_{i}\sim N(0,\sigma_{v}^{2})$. $v_{i}$ represents a noise parameter that
controls for how much outcome overlap there is. When $\sigma_{v}^{2}$ is
large, then there is increased overlap between the treatment and control
groups, as the treatment probability is less correlated with the outcome.

We vary $\sigma^{2}_{v}\in\\{0,0.1,0.25,1,2,2.5\\}$, and set $\gamma_{1}=2.5$,
$\gamma_{2}=5$, and $\beta=1$. For each iteration of the simulation, we assume
that researchers omit $U_{i}$, and estimate confidence intervals using both
the marginal sensitivity model and the variance-based sensitivity model, using
the true sensitivity parameters. We visualize the coverage rates across
simulations in Figure 3. We see that even in low overlap scenarios and small
sample sizes, the variance-based sensitivity model have nominal coverage.
However, the marginal sensitivity model struggles to achieve nominal coverage
in limited overlap settings.

Example A.1 highlights that in small sample settings and limited overlap, the
marginal sensitivity model fails to obtain nominal coverage, even with the
true $\Lambda$ value. In contrast, the variance-based sensitivity model
consistently has nominal coverage.

We see that within finite-sample settings, the marginal sensitivity model may
obtain narrower bounds than the variance-based sensitivity model, due to their
inherent sample boundedness. However, these narrower bounds risk not being
valid in settings with smaller sample size and limited outcome overlap, and
can risk large amounts of under-coverage. Thus, the estimated confidence
intervals under the variance-based sensitivity model are technically wider,
but appropriately so, providing at least nominal coverage, even in cases with
severely limited outcome overlap.

##  Appendix B Proofs and Derivations

###  B.1 Theorem 3.1

For a fixed $R^{2}\in[0,1)$, then the maximum bias under $\sigma(R^{2})$ can
be written as a function of the following components:
$\displaystyle\max_{\tilde{w}\in\sigma(R^{2})}\text{
Bias}(\hat{\tau}_{W}\mid\tilde{w})$
$\displaystyle=\underbrace{\sqrt{1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2}}}_{\text{(a) Correlation
Bound}}\sqrt{\phantom{\leavevmode\hbox{\set@color$\displaystyle\frac{R^{2}}{1-R^{2}}\cdot\text{var}(Y_{i}|A_{i}=1)\text{var}(w_{i}|A_{i}=1)$}}}{\underbrace{\frac{R^{2}}{1-R^{2}}}_{\makebox[0.0pt]{\scriptsize\text{(b)
Imbalance}}}}\cdot{\underbrace{\text{var}(Y_{i}|A_{i}=1)\text{var}(w_{i}|A_{i}=1)}_{\makebox[0.0pt]{\scriptsize}\text{(c)
Scaling Factor}}},$ with the minimum bias given as the negative of Equation
(2). The optimal bias bounds are thus given by the minimum and maximum biases.

###### Proof.

We will start by deriving the optimal bounds. To begin, we can decompose the
bias of a weighted estimator as follows:

$\displaystyle\text{Bias}(\hat{\tau}_{W})$
$\displaystyle=\mathbb{E}\left(\hat{\tau}_{W}\right)-\tau$ By conditional
ignorability:
$\displaystyle=\mathbb{E}\left(\sum_{i\in\mathcal{A}}w_{i}Y_{i}\right)-\mathbb{E}\left(\sum_{i\in\mathcal{A}}w_{i}^{*}Y_{i}\right)$
$\displaystyle=\mathbb{E}(w_{i}Y_{i}\mid
A_{i}=1)-\mathbb{E}(w_{i}^{*}Y_{i}\mid A_{i}=1)$
$\displaystyle=\mathbb{E}((w_{i}-w_{i}^{*})\cdot Y_{i}\mid A_{i}=1)$ By
construction, $\mathbb{E}(w_{i}\mid A_{i}=1)=\mathbb{E}(w_{i}^{*}\mid
A_{i}=1)$: $\displaystyle=\mathbb{E}((w_{i}-w_{i}^{*})\cdot Y_{i}\mid
A_{i}=1)-\mathbb{E}(w_{i}-w_{i}^{*}\mid A_{i}=1)\cdot\mathbb{E}(Y_{i}\mid
A_{i}=1)$ $\displaystyle=\text{cov}(w_{i}-w_{i}^{*},Y_{i}\mid A_{i}=1)$
$\displaystyle=\text{cor}(w_{i}-w_{i}^{*},Y_{i}\mid
A_{i}=1)\cdot\sqrt{\text{var}(w_{i}-w_{i}^{*}\mid
A_{i}=1)\cdot\text{var}(Y_{i}\mid A_{i}=1)}$ (9)

This is similar to the derivation provided in Shen et al. (2011) and Hong et
al. (2021). However, we will go a step further to amplify the term,
$\text{var}(w_{i}-w_{i}^{*}\mid A_{i}=1)$, into an $R^{2}$ value and the
variance of the estimated weights. To do so, we extend the results from Huang
(2022), which examined the bias in the context of an external validity
setting, and thus, focused on re-weighting an individual-level treatment
effect $\tau_{i}$. We instead apply the results to a general missingness
setting, in which we are re-weighting outcomes $Y_{i}$. We re-write the
variance of the error in the weights in Equation (9) as a function of the
$R^{2}$ parameter and the variance of the estimated weights, providing the
following bias decomposition:

$\text{Bias}(\hat{\tau}_{W})=\text{cor}(w_{i}-w_{i}^{*},Y_{i}\mid
A_{i}=1)\cdot\sqrt{\frac{R^{2}}{1-R^{2}}\cdot\text{var}(Y_{i}\mid
A_{i}=1)\cdot\text{var}(w_{i}\mid A_{i}=1)},$

where $R^{2}$ is defined in Definition 3.1. Because we are fixing
$R^{2}\in[0,1)$,555In settings when $R^{2}=1$, this implies that researchers
have effectively explained none of the variation in the true weights–i.e., in
settings when researchers use uniform weights. However, if researchers have at
least included one covariate that is at least correlated with a variable in
the separating set $\mathcal{X}$, then $R^{2}<1$. and $\text{var}(Y_{i}\mid
A_{i}=1)\cdot\text{var}(w_{i}\mid A_{i}=1)$ are directly estimable from the
data, to maximize the bias, we must maximize the correlation term.

Applying results from Huang (2022), we note that the error in the weights
(i.e., $w_{i}-w_{i}^{*}$) is orthogonal to the estimated weights $w_{i}$
(i.e., $\text{cov}(w_{i}-w_{i}^{*},w_{i}\mid A_{i}=1)=0$). Then, applying the
recursive formula of partial correlation, we obtain the following bounds for
the correlation:666This follows from results in Olkin (1981), which shows that
for any three random vectors $a$, $b$, and $c$, the correlation of $a$ and $c$
can be bounded by:
$\text{cor}(a,b)\text{cor}(c,b)-\sqrt{1-\text{cor}^{2}(a,b)}\cdot\sqrt{1-\text{cor}^{2}(b,c)}\leq\text{cor}(a,c)\leq\text{cor}(a,b)\text{cor}(c,b)+\sqrt{1-\text{cor}^{2}(a,b)}\cdot\sqrt{1-\text{cor}^{2}(b,c)}.$

$-\sqrt{1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2}}\leq\text{cor}(w_{i}-w_{i}^{*},Y_{i}\mid
A_{i}=1)\leq\sqrt{1-\text{cor}(w_{i},Y_{i}\mid A_{i}=1)^{2}}$

Thus, Equation 2 in Theorem 3.1 directly follows.

###  B.2 Theorem 3.2

For every $\tilde{w}\in\sigma(R^{2})$:
$\limsup_{n\to\infty}P(\tau(\tilde{w})<L(\tilde{w}))\leq\frac{\alpha}{2}\text{
and }\limsup_{n\to\infty}P(\tau(\tilde{w})>U(\tilde{w}))\leq\frac{\alpha}{2},$
where $L(\tilde{w})$ and $U(\tilde{w})$ are defined as the $\alpha/2$ and
$1-\alpha/2$-th quantiles of the bootstrapped estimates (i.e., Equation (4)).

###### Proof.

We may re-write our bootstrapped estimate $\hat{\tau}^{(b)}(\tilde{w})$ as:

$\displaystyle\hat{\tau}^{(b)}(\tilde{w})$
$\displaystyle=\hat{\tau}_{W}^{(b)}-\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})$
$\displaystyle=\hat{\tau}_{W}^{(b)}-\rho\cdot\sqrt{\text{var}(\hat{w}_{i}^{(b)})\frac{R^{2}}{1-R^{2}}\cdot\text{var}(Y_{i}^{(b)})}$

Because $\rho$ and $R^{2}$ are fixed (across bootstrap samples), the
components that drive variation across bootstrap samples are:
$\hat{\tau}_{W}^{(b)}$, $\text{var}(\hat{w}_{i}^{(b)})$, and
$\text{var}(Y_{i}^{(b)})$.

An overview of the proof is as follows. Similar to Zhao et al. (2019), we will
use a $Z$-estimation framework. In particular, we will add in three additional
parameters: $\hat{\mu}_{w}^{2}$, $\hat{\mu}_{Y}$, $\hat{\mu}_{Y}^{2}$, which
represent the second order moment of the weights, the average of the outcomes,
and the second order moment of the outcomes, respectively. Then, we will
invoke the asymptotic normality of bootstrapped $Z$-estimators. In the
following proof, we will show the validity of the percentile bootstrap in the
case that researchers are using inverse propensity score weights; however, we
note that researchers can invoke the results in Soriano et al. (2021) to show
validity of the results for balancing weights.

To begin, define $\mu_{w}$ as the expectation of the weights:

$\mu_{w}=\mathbb{E}(Aw)\equiv\mathbb{E}(A\cdot(1+\exp(-\beta X))).$

Then, we define $\mu$ as:

$\mu=\frac{\mathbb{E}(AY(1+\exp(-\beta^{\top}X)))}{\mu_{w}}.$

Define $\mu^{2}_{w}=\mathbb{E}(Aw^{2})$ and
$\sigma^{2}_{Y}=\mathbb{E}(AY^{2})$ as the second moment of the weights and
the outcomes, respectively. Then, we define the vector
$\theta=\left(\mu,\mu_{w},\beta,\mu^{2}_{w},\mu_{Y},\mu^{2}_{Y}\right)^{\top}\in\Theta$.
Define the function
$Q:{0,1}\times\mathbbm{R}^{d}\times\mathbbm{R}\to\mathbbm{R}^{d+5}$, where for
$t=(a,x^{\top},y)\in\\{0,1\\}\times\mathbbm{R}^{d}\times\mathbbm{R}$:

$\displaystyle Q(t\mid\theta)=\begin{pmatrix}Q_{1}(t|\theta)\\\
Q_{2}(t|\theta)\\\ Q_{3}(t|\theta)\\\ Q_{4}(t|\theta)\\\ Q_{5}(t|\theta)\\\
Q_{6}(t|\theta)\end{pmatrix}:=\begin{pmatrix}\left(a-\frac{\exp(\beta^{\top}x)}{1+\exp(\beta^{\top}x)}\right)x\\\
\mu_{w}-a\left(1+\exp(-\beta^{\top}x)\right)\\\ \mu_{w}\mu-
ay\left(1+\exp(-\beta^{\top}x)\right)\\\
\mu_{w}^{2}-a\left(1+\exp(-\beta^{\top}x)\right)^{2}\\\ \mu_{y}-ay\\\
\mu_{y}^{2}-ay^{2}\end{pmatrix}$ (10)

Finally, we define $\Phi(\theta)$ as:

$\Phi(\theta)=\int Q(t|\theta)d\mathbb{P}(t),$

where $T=(A,X^{\top},AY)^{\top}\sim\mathbb{P}$, where $\mathbb{P}$ represents
the true distribution generating the data. It is simple to see that
$\Phi(\theta^{*})=0$, when $\theta^{*}$ is equal to the true parameter values.
Then, the $Z$-estimates $\hat{\theta}$ :

$\displaystyle\Phi_{n}(\hat{\theta}):$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}Q(T_{i}|\hat{\theta})$ (11)
$\displaystyle=\begin{pmatrix}\left(\frac{1}{n}\sum_{i=1}^{n}A_{i}-\frac{\exp(\hat{\beta}^{\top}\mathbf{X}_{i})}{1+\exp(\hat{\beta}^{\top}\mathbf{X}_{i})}\right)\mathbf{X}_{i}\\\
\hat{\mu}_{w}-\frac{1}{n}\sum_{i=1}^{n}A_{i}\left(1+\exp(-\hat{\beta}^{\top}\mathbf{X}_{i})\right)\\\
\hat{\mu}_{w}\mu-\frac{1}{n}\sum_{i=1}^{n}A_{i}Y_{i}\left(1+\exp(-\hat{\beta}^{\top}\mathbf{X}_{i})\right)\\\
\hat{\mu}^{2}_{w}-\frac{1}{n}\sum_{i=1}^{n}A_{i}\left(1+\exp(-\hat{\beta}^{\top}\mathbf{X}_{i})\right)^{2}\\\
\hat{\mu}_{y}-\frac{1}{n}\sum_{i=1}^{n}A_{i}Y_{i}\\\
\hat{\mu}_{y}^{2}-\frac{1}{n}\sum_{i=1}^{n}(A_{i}Y_{i}^{2})\end{pmatrix}=0$
(12)

We define $\Sigma:=\mathbb{E}(Q(t\mid\theta)Q(t\mid\theta)^{\top})$. We will
invoke the following regularity conditions, consistent with Zhao et al.
(2019).

######  Assumption 4 (Regularity Conditions).

Assume that the parameter space $\Theta$ is compact, and that $\theta$ is in
the interior of $\Theta$. Furthermore, $(Y,\mathbf{X})$ satisfies the
following:

  1. 1.

$\mathbb{E}(Y^{4})<\infty$

  2. 2.

$\det\left(\mathbb{E}\left(\frac{\exp(\beta^{\top}\mathbf{X})}{(1+\exp(\beta^{\top}\mathbf{X}))^{2}}\mathbf{X}\mathbf{X}^{\top}\right)\right)>0$

  3. 3.

$\forall$ compact subsets $S\subset\mathbbm{R}^{d}$,
$\mathbb{E}(\sup_{\beta\in S}\exp(\beta^{\top}\mathbf{X}))<\infty$

To show asymptotic normality of bootstrapped $Z$-estimators, we must first
verify that $\dot{\Phi}_{0}$ and $\Sigma$ are well-defined.

$\displaystyle\dot{\Phi}_{0}$
$\displaystyle=\mathbb{E}\left(\nabla_{\theta=\theta_{0}}Q(T|\theta)\right)$
$\displaystyle=\begin{pmatrix}0&0&-\mathbb{E}\left(\frac{\exp(\beta_{0}^{\top}\mathbf{X})}{1+\exp(\beta_{0}^{\top}\mathbf{X})^{2}}\mathbf{X}\mathbf{X}^{\top}\right)&0&0&0\\\
0&1&\mathbb{E}(A\mathbf{X}^{\top}\exp(-\beta_{0}^{\top}\mathbf{X})&0&0&0\\\
\mu_{w}&\mu&\mathbb{E}(AY\mathbf{X}^{\top}(\exp(\beta_{0}^{\top}\mathbf{X})&0&0&0\\\
0&0&\mathbb{E}(A\mathbf{X}^{\top}(\exp(\beta_{0}^{\top}\mathbf{X})+\exp(-2\beta_{0}^{\top}\mathbf{X})))&1&0&0\\\
0&0&0&0&1&0\\\ 0&0&0&0&0&1\end{pmatrix}$

By Leibniz Formula:

$\displaystyle\left|\det(\dot{\Phi}_{0})\right|$
$\displaystyle=\left|\det\begin{pmatrix}0&0&-\mathbb{E}\left(\frac{\exp(\beta_{0}^{\top}\mathbf{X})}{1+\exp(\beta_{0}^{\top}\mathbf{X})^{2}}\mathbf{X}\mathbf{X}^{\top}\right)\\\
0&1&\mathbb{E}(A\mathbf{X}^{\top}\exp(-\beta_{0}^{\top}\mathbf{X})\\\
\mu_{w}&\mu&\mathbb{E}(AY\mathbf{X}^{\top}(\exp(\beta_{0}^{\top}\mathbf{X})\end{pmatrix}\det\begin{pmatrix}1&0&0\\\
0&1&0\\\ 0&0&1\end{pmatrix}\right|$
$\displaystyle=\mu_{w}\left|\det\mathbb{E}\left(\frac{\exp(\beta_{0}^{\top}\mathbf{X})}{(1+\exp(\beta_{0}^{\top}\mathbf{X}))^{2}}\mathbf{X}\mathbf{X}^{\top}\right)\right|>0,$

which follows by regularity condition (2). As such, $\dot{\Phi}_{0}$ is
invertible. Furthermore, by regularity condition (1), $\Sigma<\infty$.

As such, we simply need to verify the three conditions for asymptotic
normality of bootstrapped $Z$-estimators:

  1. 1.

The class of functions ${t\to Q(t|\theta):\theta\in\Theta}$ is
$\mathbb{P}$-Glivenko-Cantelli.

  2. 2.

$||\Phi(\theta)||_{1}$ is strictly positive outside every open neighborhood of
$\theta_{0}$.

  3. 3.

The class of functions is $\mathbb{P}$-Donsker, and
$\mathbb{E}((Q(T|\theta_{n})-Q(T|\theta_{0}))^{2})\to 0$ whenever
$||\theta_{n}-\theta_{0}||_{1}\to 0$.

It is worth noting that the first three parameters ($\mu$, $\mu_{w}$, $\beta$)
are special cases from Zhao et al. (2019), in which we do not perform any
shifting in the weights (i.e., $h(x,y)=0$). We will then show that the three
conditions still hold after additionally accounting for the last three
parameters. The proof for each condition is provided below.

Condition 1: The class of functions ${t\to Q(t|\theta):\theta\in\Theta}$ is
$\mathbb{P}$-Glivenko-Cantelli.

$\displaystyle||Q(t|\theta)\leq||Q_{1}(t|\theta)||_{1}+\sum_{b=2}^{5}|Q_{b}(t|\theta)|$

Zhao et al. (2019) show that
$||Q_{1}(t|\theta)||_{1}+|Q_{2}(t|\theta)|+|Q_{3}(t|v)|$ is bounded as a
function of $x$, $y$, and some absolute constant $M_{1}$:

$||Q_{1}(t|\theta)||_{1}+|Q_{2}(t|\theta)|+|Q_{3}(t|v)|\leq||x||_{1}+|y|+\exp(-\beta^{\top}x)(1+|y|)+M_{1}.$

As such, all that is left to show is to show that
$|Q_{4}(t|\theta)|+|Q_{5}(t|\theta)|+|Q_{6}(t|\theta)|$ is finite. To begin:

$\displaystyle|Q_{4}(t|\theta)|$
$\displaystyle=|\mu^{2}_{w}-(a(1+\exp(-\beta^{\top}x)^{2}|$
$\displaystyle\leq\mu_{w}^{2}+(1+\exp(-\beta^{\top}x))^{2}$
$\displaystyle|Q_{5}(t|\theta)|$ $\displaystyle=|\mu_{y}-ay|$
$\displaystyle\leq|\mu_{y}|+|y|$ $\displaystyle|Q_{6}(t|\theta)|$
$\displaystyle=|\mu^{2}_{y}-ay^{2}|$ $\displaystyle\leq\mu^{2}_{y}+|y^{2}|$

As such:

$\displaystyle|Q_{4}(t|\theta)|+|Q_{5}(t|\theta)|+|Q_{6}(t|\theta)|\leq
M_{2}+(1+\exp(-\beta^{\top}x))^{2}+|y|+|y^{2}|,$

where $M_{2}$ is some absolute constant. As such, where $M$ is an absolute
constant:

$||Q(t|\theta||_{1}\leq||x||_{1}+2|y|+|y^{2}|+\exp(-\beta^{\top}x)(1+|y|)+(1+\exp(-\beta^{\top}x))^{2}+M,$

where $M<\infty$ by regularity condition (1). Therefore,
$\mathbb{E}(\sup_{\theta\in\Theta}||Q(t|\theta)||_{1})<\infty$, and $\\{t\to
Q(t|\theta):\theta\in\Theta\\}$ is $\mathbb{P}$-Glivenko-Cantelli.

Condition 2: $||\Phi(\theta)||_{1}$ is strictly positive outside every open
neighborhood of $\theta_{0}$.
Following Zhao et al. (2019), we fix some $\varepsilon>0$. If
$||\beta-\beta_{0}||_{1}>\varepsilon/M$, then it is trivial to show that
$||\Phi(\theta)||_{1}>0$. Zhao et al. (2019) show that when
$||\beta-\beta_{0}||_{1}\leq\varepsilon/M$, if
$|\mu_{w}-\mu_{w,0}|>\varepsilon/4K$, where
$K=\sup_{\theta\in\Theta}|\mu|\in(0,\infty)$, then $||\Phi(\theta)||_{1}>0$.
Furthermore, when $||\beta-\beta_{0}||_{1}\leq\varepsilon/M$ and
$|\mu_{w}-\mu_{w,0}|\leq\varepsilon/4K$ and
$|\mu-\mu_{0}|>\varepsilon/2\mu_{w}$, then $||\Phi(\theta)||_{1}>0$.

Thus, we must show for the remaining 3 parameters that when
$||\mu^{2}_{w}-\mu^{2}_{w,0}||$, $||\mu_{y}-\mu_{y,0}||$, or
$||\mu^{2}_{y}-\mu^{2}_{y,0}||_{1}$ are greater than some $\varepsilon$,
$||\Phi(\theta)||_{1}>0$. Assume $|||\beta-\beta_{0}||_{1}\leq\varepsilon/M$.
Then:

$\displaystyle\left|\mathbb{E}\left[A\exp(-\beta^{\top}\mathbf{X})^{2}+A\exp(-\beta_{0}^{\top}\mathbf{X})^{2}\right]\right|$
$\displaystyle=\left|\mathbb{E}\left[A\exp(-2\beta^{\top}\mathbf{X})+A\exp(-2\beta_{0}^{\top}\mathbf{X})\right]\right|$
$\displaystyle\leq\left|\mathbb{E}\left(\exp(-2\beta^{\top}\mathbf{X}-2\beta_{0}^{\top}\mathbf{X})\right)\right|$
$\displaystyle\leq
2||\beta-\beta_{0}||_{\infty}\mathbb{E}\left(||\mathbf{X}||_{1}\exp(-t^{*})\right)\text{
for }t^{*}\in[\beta_{0}^{\top}\mathbf{X},\beta^{\top}\mathbf{X}]$
$\displaystyle\leq 2\cdot\frac{\varepsilon}{64K}=\frac{\varepsilon}{32K}$

As such, if $||\mu^{2}_{w}-\mu^{2}_{w,0}||>\varepsilon/32K$:

$\displaystyle||\Phi(\theta)||_{1}\geq\left|\mu^{2}_{w}-\mu^{2}_{w,0}+\mathbb{E}\left[A\exp(-\beta^{\top}\mathbf{X})^{2}+A\exp(-\beta_{0}^{\top}\mathbf{X})^{2}\right]\right|>0$
(13)

For the final two parameters, it is worth noting that there is no dependency
on the other parameter estimates. As such, regardless of whether the other
parameters are smaller than some $\epsilon$, if
$||\mu_{y}-\mu_{y,0}||_{1}>\varepsilon$:

$\displaystyle||\Phi(\theta)||_{1}$
$\displaystyle\geq\left|\mu_{y}-\mathbb{E}(AY)-(\mu_{y,0}-\mathbb{E}(AY))\right|$
$\displaystyle=\left|\mu_{y}-\mu_{y,0}\right|>0$ (14)

Similarly, if $||\mu^{2}_{y}-\mu^{2}_{y,0}||>\varepsilon$

$\displaystyle||\Phi(\theta)||_{1}$
$\displaystyle\geq\left|\mu^{2}_{y}-\mathbb{E}(AY^{2})-(\mu^{2}_{y,0}-\mathbb{E}(AY^{2}))\right|$
$\displaystyle=\left|\mu^{2}_{y,0}-\mu_{y}\right|>0$ (15)

As such, combining Equation (13), (14), (15), as well as the results from Zhao
et al. (2019), we have shown that for all $\delta>0$,
$\inf\\{||\Phi(\theta)||^{2}:||\theta-\theta_{0}||_{1}>\delta\\}>0$.

Condition 3: The class of functions is $\mathbb{P}$-Donsker, and
$\mathbb{E}((Q(T|\theta_{n})-Q(T|\theta_{0}))^{2})\to 0$ whenever
$||\theta_{n}-\theta_{0}||_{1}\to 0$.
From Zhao et al. (2019), we obtain a bound for the first three terms (i.e.,
$Q_{1}(t|\theta)$, $Q_{2}(t|\theta)$, and $Q_{3}(t|\theta)$). Then, for the
4th term:777Consistent with Zhao et al. (2019), for some $a,b\in\mathbbm{R}$,
and some constant $C>0$, if $a\leq C\cdot b$, then we write $a\lesssim b$.

$\displaystyle|Q_{4}$ $\displaystyle(t|\theta_{2})-Q_{4}(t|\theta_{1})|$
$\displaystyle=|\mu^{2}_{w,2}-(a(1+\exp(-\beta_{2}^{\top}x))^{2}-(\mu^{2}_{w,1}-(a(1+\exp(-\beta_{1}^{\top}x))^{2}|$
$\displaystyle\leq|\mu^{2}_{w,2}-\mu^{2}_{w,1}|+|(1+\exp(-\beta_{2}^{\top}x))^{2}-(1+\exp(-\beta_{1}^{\top}x))^{2}|$
$\displaystyle=|\mu^{2}_{w,2}-\mu^{2}_{w,1}|+\left|2\big{(}\exp(-\beta_{2}^{\top}x)-\exp(-\beta_{1}^{\top}x)\big{)}+\exp(-2\beta_{2}^{\top}x)-\exp(-2\beta_{1}^{\top}x)\right|$
$\displaystyle\leq|\mu^{2}_{w,2}-\mu^{2}_{w,1}|+\left|2\big{(}\exp(-\beta_{2}^{\top}x)-\exp(-\beta_{1}^{\top}x)\big{)}\right|+\left|\exp(-2\beta_{2}^{\top}x)-\exp(-2\beta_{1}^{\top}x)\right|$
Applying the Mean Value Theorem (equivalently, results from Zhao et al.
(2019)):
$\displaystyle\lesssim|\mu^{2}_{w,2}-\mu^{2}_{w,1}|+2||\beta_{2}-\beta_{1}||_{2}||x||_{2}\sup_{\beta\in\Theta}\exp(-\beta^{\top}x)+||2\beta_{2}-2\beta_{1}||_{2}||x||_{2}\sup_{\beta\in\Theta}\exp(-2\beta^{\top}x)$
$\displaystyle=|\mu^{2}_{w,2}-\mu^{2}_{w,1}|+2||\beta_{2}-\beta_{1}||_{2}||x||_{2}\sup_{\beta\in\Theta}\exp(-\beta^{\top}x)+2||\beta_{2}-\beta_{1}||_{2}||x||_{2}\sup_{\beta\in\Theta}\exp(-2\beta^{\top}x)$
$\displaystyle=|\mu^{2}_{w,2}-\mu^{2}_{w,1}|+2||\beta_{2}-\beta_{1}||_{2}||x||_{2}\sup_{\beta\in\Theta}\exp(-\beta^{\top}x)\left(1+\sup_{\beta\in\Theta}\exp(-\beta^{\top}x)\right)$
$\displaystyle\lesssim
M_{4}(x)\left(|\mu^{2}_{w,2}-\mu^{2}_{w,1}|+||\beta_{2}-\beta_{1}||_{1}\right)$

Finally, for the 5th and 6th terms:

$\displaystyle|Q_{5}$ $\displaystyle(t|\theta_{2})-Q_{5}(t|\theta_{1})|$
$\displaystyle=|\mu_{y,2}-ay-(\mu_{y,1}-ay)|$
$\displaystyle=|\mu_{y,2}-\mu_{y,1}|$ $\displaystyle|Q_{6}$
$\displaystyle(t|\theta_{2})-Q_{6}(t|\theta_{1})|$
$\displaystyle=|\mu^{2}_{y,2}-ay^{2}-(\mu^{2}_{y,1}-ay^{2})|$
$\displaystyle\leq|\mu^{2}_{y,2}-\mu^{2}_{y,1}|$

Combining results with Zhao et al. (2019), we see that:

$\displaystyle||Q(t|\theta_{2})-Q(t|\theta_{1})||_{1}=\sum_{b=1}^{6}||Q_{b}(t|\theta_{2})-Q_{b}(t|\theta_{1})||\lesssim
M(x,y)||\theta_{2}-\theta_{1}||_{1}$

Since $\mathbb{E}(M(X,Y)^{2})<\infty$, we have shown that the class of
functions is $\mathbb{P}$-Donsker, and furthermore, that whenever
$||\theta_{n}-\theta_{0}||_{1}\to 0$,
$\mathbb{E}\big{[}(Q(t|\theta_{n})-Q(t|\theta_{0}))^{2}\big{]}\to 0$.

Then, by invoking Kosorok (2008), Theorem 10.16:

$\sqrt{n}(\hat{\theta}-\theta)\stackrel{{\scriptstyle
d}}{{\rightarrow}}N\left(0,\dot{\Phi}^{-1}_{0}\Sigma\dot{\Phi}_{0}\right),\ \
\text{ and }\ \ \sqrt{n}(\hat{\theta}^{(b)}-\theta)\stackrel{{\scriptstyle
d}}{{\rightarrow}}N\left(0,\dot{\Phi}^{-1}_{0}\Sigma\dot{\Phi}_{0}\right),$
(16)

As such, applying Delta Method and results from Appendix C3 in Zhao et al.
(2019) concludes the proof.

###  B.3 Theorem 4.1 (Weighted $L_{2}$ Analog)

Define the individual-level error in the weights as
$\lambda_{i}:=w_{i}^{*}/w_{i}$. Define the $L_{2,w}$ norm as follows:
$||\lambda||^{2}_{2,w}:=\begin{cases}\displaystyle\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}\cdot\nu(w_{i})&\text{if
}\text{var}(w_{i})>0,\\\ \infty&\text{else}\end{cases},$ where $\nu(w_{i})$ is
a function of the estimated weights. Then, the variance-based sensitivity
model can equivalently be written as a norm-constrained optimization problem:
$\max_{\tilde{w}\in\sigma(R^{2})}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})\iff\begin{cases}\displaystyle\max_{\tilde{w}}\text{Bias}(\hat{\tau}_{W}\mid\tilde{w})\\\
\displaystyle\text{s.t.
}||\lambda||_{2,w}\leq\sqrt{\frac{k}{1-R^{2}}},\end{cases}$ where
$k:=1-R^{2}/\mathbb{E}(w_{i}^{2})$.

###### Proof.

Define $\lambda_{i}:=w_{i}^{*}/w_{i}$. Then, following results from Huang
(2022):

$\displaystyle\text{var}(w_{i}-w_{i}^{*})$
$\displaystyle=\text{var}(w_{i}^{*})-\text{var}(w_{i})$ We can then substitute
in $\lambda_{i}$: $\displaystyle=\text{var}(\lambda_{i}\cdot
w_{i})-\text{var}(w_{i})$ $\displaystyle=\mathbb{E}(\lambda_{i}^{2}\cdot
w_{i}^{2})-\underbrace{\mathbb{E}(\lambda_{i}\cdot
w_{i})^{2}}_{\equiv\mathbb{E}(w_{i}^{*})^{2}=1}-\text{var}(w_{i})$
$\displaystyle=\text{cov}(\lambda_{i}^{2},w_{i}^{2})+\mathbb{E}(\lambda_{i}^{2})\mathbb{E}(w_{i}^{2})-1-\text{var}(w_{i})$
$\displaystyle=\text{cov}(\lambda_{i}^{2},w_{i}^{2})+\mathbb{E}(\lambda_{i}^{2})\text{var}(w_{i})+\mathbb{E}(\lambda_{i}^{2})-1-\text{var}(w_{i})$
$\displaystyle=\text{cov}(\lambda_{i}^{2},w_{i}^{2})+(\mathbb{E}(\lambda_{i}^{2})-1)\cdot\left(\text{var}(w_{i})+1\right)$
$\displaystyle=\text{cov}(\lambda_{i}^{2},w_{i}^{2})+(\mathbb{E}(\lambda_{i}^{2})-1)\cdot\mathbb{E}(w_{i}^{2})$
$\displaystyle\implies\frac{\text{var}(w_{i}-w_{i}^{*})}{\mathbb{E}(w_{i}^{2})}$
$\displaystyle=\frac{\text{cov}(\lambda_{i}^{2},w_{i}^{2})}{\mathbb{E}(w_{i}^{2})}+(\mathbb{E}(\lambda_{i}^{2})-1)$

Re-arranging the terms:

$\displaystyle\frac{\text{cov}(\lambda_{i}^{2},w_{i}^{2})}{\mathbb{E}(w_{i}^{2})}+\mathbb{E}(\lambda_{i}^{2})$
$\displaystyle=1+\frac{\text{var}(w_{i}-w_{i}^{*})}{\mathbb{E}(w_{i}^{2})}$
$\displaystyle=1+\frac{\mathbb{E}(w_{i}^{2})-\mathbb{E}(w_{i})^{2}}{\mathbb{E}(w_{i}^{2})}\cdot\frac{R^{2}}{1-R^{2}}$
$\displaystyle=1+\frac{R^{2}}{1-R^{2}}-\frac{\mathbb{E}(w_{i})^{2}}{\mathbb{E}(w_{i}^{2})}\cdot\frac{R^{2}}{1-R^{2}}$
$\displaystyle=\frac{1}{1-R^{2}}-\underbrace{\frac{\mathbb{E}(w_{i})^{2}}{\mathbb{E}(w_{i}^{2})}}_{1/\mathbb{E}(w_{i}^{2})}\cdot\frac{R^{2}}{1-R^{2}}$
$\displaystyle=\frac{1}{1-R^{2}}\underbrace{\left(1-\frac{R^{2}}{\mathbb{E}(w_{i}^{2})}\right)}_{:=k}$

By setting $R^{2}$, we are also setting the value for
$\frac{\text{cov}(\lambda_{i}^{2},w_{i}^{2})}{\mathbb{E}(w^{2}_{i})}+\mathbb{E}(\lambda_{i}^{2})$.

We now re-write
$\frac{\text{cov}(\lambda_{i}^{2},w_{i}^{2})}{\mathbb{E}(w^{2}_{i})}+\mathbb{E}(\lambda_{i}^{2})$
as a weighted sum:

$\displaystyle\mathbb{E}(\lambda_{i}^{2})+\frac{\text{cov}(\lambda_{i}^{2},w_{i}^{2})}{\mathbb{E}(w_{i}^{2})}$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}+\frac{1}{\mathbb{E}(w_{i}^{2})}\cdot\frac{1}{n}\sum_{i=1}^{n}(\lambda_{i}^{2}-\mathbb{E}(\lambda_{i}^{2}))(w_{i}^{2}-\mathbb{E}(w_{i}^{2}))$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}+\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}\cdot\frac{w_{i}^{2}-\mathbb{E}(w_{i}^{2})}{\mathbb{E}(w_{i}^{2})}-\frac{1}{n}\sum_{i=1}^{n}\mathbb{E}(\lambda_{i}^{2})\cdot\frac{w_{i}^{2}-\mathbb{E}(w_{i}^{2})}{\mathbb{E}(w_{i}^{2})}$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}\cdot\underbrace{\left(1+\frac{w_{i}^{2}-\mathbb{E}(w_{i}^{2})}{\mathbb{E}(w_{i}^{2})}\right)}_{:=\nu(w_{i})}+\mathbb{E}(\lambda_{i}^{2})\underbrace{\frac{1}{n}\sum_{i=1}^{n}\frac{w_{i}^{2}-\mathbb{E}(w_{i}^{2})}{\mathbb{E}(w^{2}_{i})}}_{:=0}$
$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}\cdot\nu(w_{i})$

As such, we can define the $L_{2,w}$ norm as follows:

$||\lambda||^{2}_{2,w}:=\begin{cases}\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}^{2}\cdot\nu(w_{i})&\text{if
}\text{var}(w_{i})>0\\\ \infty&\text{else}\end{cases}$

We will show that $L_{2,w}$ meets the criteria for being a semi-norm.

  1. 1.

Triangle Inequality:
$||\lambda_{1}+\lambda_{2}||_{2,w}\leq||\lambda_{1}||_{2,w}+||\lambda_{2}||_{2,w}$

$\displaystyle||\lambda_{1}+\lambda_{2}||^{2}_{2,w}$
$\displaystyle=\sum_{i=1}^{n}(\lambda_{i,1}+\lambda_{i,2})^{2}\cdot\nu(w_{i})$
$\displaystyle=\sum_{i=1}^{n}\lambda^{2}_{i,1}\nu(w_{i})+\sum_{i=1}^{n}\lambda^{2}_{i,2}\nu(w_{i})+2\sum_{i=1}^{n}\lambda_{i,1}\lambda_{i,2}\nu(w_{i})$
Applying Cauchy-Schwarz:
$\left(\sum_{i=1}^{n}\lambda_{i,1}\lambda_{i,2}\nu(w_{i})\right)^{2}\leq\left(\sum_{i=1}^{n}\lambda_{i,1}\lambda_{i,1}^{2}\nu(w_{i})\right)^{2}\left(\sum_{i=1}^{n}\lambda_{i,1}\lambda_{i,2}^{2}\nu(w_{i})\right)^{2}$.
Then:
$\displaystyle\leq\sum_{i=1}^{n}\lambda^{2}_{i,1}\nu(w_{i})+\sum_{i=1}^{n}\lambda^{2}_{i,2}\nu(w_{i})+2\left(\sum_{i=1}^{n}\lambda_{i,1}\lambda_{i,1}^{2}\nu(w_{i})\right)\left(\sum_{i=1}^{n}\lambda_{i,1}\lambda_{i,2}^{2}\nu(w_{i})\right)$
$\displaystyle=\left(||\lambda_{1}||_{2,w}+||\lambda_{2}||_{2,w}\right)^{2}$

  2. 2.

Absolute homogeneity:

$\displaystyle||k\cdot\lambda||_{2,w}$
$\displaystyle=\sqrt{\sum_{i=1}^{n}(k\cdot\lambda_{i})^{2}\cdot\nu(w_{i})}$
$\displaystyle=k\sqrt{\sum_{i=1}^{n}\lambda_{i}^{2}\cdot\nu(w_{i})}$
$\displaystyle=k\cdot||\lambda||_{2,w}$

###  B.4 Theorem 4.2

Let $\psi(\Lambda)$ represent the difference in the estimated point estimate
bounds under the marginal sensitivity model $\varepsilon(\Lambda)$ for a given
$\Lambda\geq 1$:
$\psi(\Lambda):=\max_{\tilde{w}\in\varepsilon(\Lambda)}\frac{\sum_{i:Z_{i}=0}Y_{i}Z_{i}\tilde{w}_{i}}{\sum_{i:Z_{i}=0}Z_{i}\tilde{w}_{i}}-\min_{\tilde{w}\in\varepsilon(\Lambda)}\frac{\sum_{i:Z_{i}=0}Y_{i}Z_{i}\tilde{w}_{i}}{\sum_{i:Z_{i}=0}Z_{i}\tilde{w}_{i}}.$
Then if the true $R^{2}$ parameter is lower than the following threshold,
$R^{2}\leq\frac{\psi(\Lambda)^{2}}{4\underbrace{(1-\text{cor}(w_{i},Y_{i})^{2})}_{\text{Correlation
Bound}}\cdot\underbrace{\text{var}(w_{i})\text{var}(Y_{i})}_{\text{Scaling
Factor}}+\psi(\Lambda)^{2}},$ the bounds under the variance-based sensitivity
model will be narrower than the bounds for the marginal sensitivity model.

###### Proof.

The length of the point estimate bounds under the variance-based sensitivity
model $\sigma(R^{2})$ is equal to two times the maximum bias bound:

$\displaystyle\max_{\tilde{w}\in\sigma(R^{2})}\tau(\tilde{w})-\min_{\tilde{w}\in\sigma(R^{2})}\tau(\tilde{w})=2\cdot\sqrt{1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2}}\cdot\sqrt{\frac{R^{2}}{1-R^{2}}\cdot\text{var}(w_{i})\cdot\text{var}(Y_{i}\mid
A_{i}=1)}$

By definition, the length of the estimated point estimate bounds under the
marginal sensitivity model is represented by $\psi(\Lambda)$. Thus, we want to
solve for the $R^{2}$ value such that the following inequality holds:

$\displaystyle 2\cdot\sqrt{1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2}}\cdot\sqrt{\frac{R^{2}}{1-R^{2}}\cdot\text{var}(w_{i}\mid
A_{i}=1)\cdot\text{var}(Y_{i}\mid A_{i}=1)}\leq\psi(\Lambda)$

Solving for the $R^{2}$ value:

$\displaystyle\frac{R^{2}}{1-R^{2}}$
$\displaystyle\leq\frac{\psi(\Lambda)^{2}/4}{(1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2})\cdot\text{var}(w_{i}\mid A_{i}=1)\cdot\text{var}(Y_{i}\mid
A_{i}=1)}$ $\displaystyle R^{2}$
$\displaystyle\leq\frac{\psi(\Lambda)^{2}/4(1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2})\text{var}(w_{i}\mid A_{i}=1)\text{var}(Y_{i}\mid
A_{i}=1)}{1+\psi(\Lambda)^{2}/4(1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2})\text{var}(w_{i}\mid A_{i}=1)\text{var}(Y_{i}\mid A_{i}=1)}$
$\displaystyle=\frac{\psi(\Lambda)^{2}}{4(1-\text{cor}(w_{i},Y_{i}\mid
A_{i}=1)^{2})\text{var}(w_{i}\mid A_{i}=1)\text{var}(Y_{i}\mid
A_{i}=1)+\psi(\Lambda)^{2}},$

The results from the corollary directly follow.

###  B.5 Example 4.1

Assume researchers use a logit model to estimate the weights using
$\mathbf{X}_{i}$, but omit a confounder $U_{i}$. The estimated and ideal
weights take on the following forms:
$\hat{w}_{i}=\exp(\hat{\gamma}^{\top}\mathbf{X}_{i})\ \ \ \ \ \ \ \ \ \
\hat{w}_{i}^{*}=\exp(\hat{\gamma}^{*\top}\mathbf{X}_{i}+\hat{\beta}U_{i})$
Assume
$\begin{bmatrix}\mathbf{X}_{i},U_{i}\end{bmatrix}\stackrel{{\scriptstyle
iid}}{{\sim}}MVN(0,I)$. Then $\mathbb{E}(\hat{\Lambda})\to\infty$ as
$n\to\infty$:
$\lim_{n\to\infty}\frac{\mathbb{E}(\hat{\Lambda})}{\exp(\sqrt{2\nu^{2}\log(n)})}\geq
1,$ where $\nu^{2}=(\gamma^{*}-\gamma)^{\top}(\gamma^{*}-\gamma)+\beta^{2}$,
and the results follow immediately from Wainwright (2019).

###### Proof.

The multiplicative error between $w_{i}^{*}$ and $w_{i}$ is written as:

$\displaystyle\frac{\hat{w}_{i}^{*}}{\hat{w}_{i}}=\frac{\exp(\hat{\gamma}^{*\top}\mathbf{X}_{i}+\hat{\beta}U_{i})}{\exp(\hat{\gamma}^{\top}\mathbf{X}_{i})}=\exp((\hat{\gamma}^{*}-\hat{\gamma})^{\top}\mathbf{X}_{i}+\hat{\beta}U_{i}),$

and $\hat{\Lambda}$ is defined as the maximum:

$\hat{\Lambda}=\max_{1\leq i\leq
n}\exp(|(\hat{\gamma}^{*}-\hat{\gamma})^{\top}\mathbf{X}_{i}+\hat{\beta}U_{i}|)$

We will show that $\mathbb{E}(\hat{\Lambda})\to\infty$, as $n\to\infty$. To
begin, define $V_{i}$ as:

$V_{i}:=(\hat{\gamma}^{*}-\hat{\gamma})^{\top}\mathbf{X}_{i}+\hat{\beta}U_{i}$

Because $\mathbf{X}_{i}$ and $U_{i}$ are normally distributed, $V_{i}$ will be
normally distributed, with mean 0, and variance
$\nu^{2}:=(\hat{\gamma}^{*}-\hat{\gamma})^{\top}+\hat{\beta}^{2}$. Let
$V^{(1)},...,V^{(n)}$ be the ordered set of $V$ such that $V^{(1)}\leq...\leq
V^{(n)}$. Without loss of generality, assume $|V^{(n)}|\geq|V^{(1)}|$. Then,
$\mathbb{E}(\hat{\Lambda})=\mathbb{E}(\exp(|V^{(n)}|)$. Using Jensen’s
inequality, the expectation of $\hat{\Lambda}$ may be lower bounded:

$\mathbb{E}(\hat{\Lambda})=\mathbb{E}(\exp(|V^{(n)}|)\geq\exp(\mathbb{E}(|V^{(n)}|))$

Then, we may invoke a well-studied result that for any set of $n$ normally
distributed random variables (Wainwright (2019)):

$\lim_{n\to\infty}\frac{\mathbb{E}(V^{(n)})}{\sqrt{2\nu^{2}\log(n)}}=1$

Because $\mathbb{E}(|V^{(n)}|)\geq\mathbb{E}(V^{(n)})$

$\lim_{n\to\infty}\frac{\mathbb{E}(V^{(n)})}{\sqrt{2\nu^{2}\log(n)}}=1$

As such, as $n\to\infty$, $\mathbb{E}(\hat{\Lambda})\to\infty$.

###  B.6 Example 4.2

Consider the same setting as Example 4.1. Then, the $R^{2}$ value can be
written as follows:
$R^{2}=1-\frac{\exp(\hat{\gamma}^{\top}\hat{\gamma})-1}{\exp(\hat{\gamma}^{*\top}\hat{\gamma}^{*}+\hat{\beta}^{2})-1}\cdot\frac{\exp(\hat{\gamma}^{\top}\hat{\gamma})}{\exp(\hat{\gamma}^{*\top}\hat{\gamma}^{*}+\hat{\beta}^{2})}.$

###### Proof.

Because $[\mathbf{X}_{i},U_{i}]\stackrel{{\scriptstyle iid}}{{\sim}}MVN(0,I)$,
$\hat{w}_{i}$ and $\hat{w}_{i}^{*}$ both are lognormal random variables, by
definition, the variance of $\hat{w}_{i}^{*}$ is:
$(\exp(\hat{\gamma}^{*\top}\gamma^{*}+\hat{\beta}^{2})-1)\cdot\exp(\hat{\gamma}^{*\top}\hat{\gamma}+\hat{\beta}^{2}),$
and similarly, the variance of $\hat{w}_{i}$ is:
$(\exp(\hat{\gamma}^{\top}\hat{\gamma})-1)\cdot\exp(\hat{\gamma}^{\top}\hat{\gamma})$.
Then, the result of the example immediately follows, using
$R^{2}:=1-\text{var}(w_{i}\mid A_{i}=1)/\text{var}(w_{i}^{*}\mid A_{i}=1).$

###  B.7 Corollary 4.1

Consider the set of confounders, in which for all $\delta>0$,
$P(w^{*}_{i}/w_{i}<\delta)>0$, or $P(w^{*}_{i}/w_{i}>\delta)>0$. Then, if the
outcomes are unbounded, the threshold from Theorem 4.2 will converge in
probability to 1:
$\frac{\psi(\Lambda)^{2}}{4(1-\text{cor}(w_{i},Y_{i})^{2})\cdot\text{var}(w_{i})\text{var}(Y_{i})+\psi(\Lambda)^{2}}\stackrel{{\scriptstyle
p}}{{\rightarrow}}1$

###### Proof.

To begin, for simplicity of notation, we define $g(\psi(\Lambda);Y_{i},w_{i})$
as the threshold from Theorem 4.2:

$g(\psi(\Lambda);Y_{i},w_{i}):=\frac{\psi(\Lambda)^{2}}{4(1-\text{cor}(w_{i},Y_{i})^{2})\cdot\text{var}(w_{i})\text{var}(Y_{i})+\psi(\Lambda)^{2}}$

We will use results from Resnick (2008), who show that for a sequence of
random variables drawn i.i.d., the maximum of the sequence will converge in
probability towards the upper bound of the support. We provide the derivation
for completeness. Let $\\{W_{i}\\}_{i=1}^{n}$ be drawn i.i.d. from a
distribution $F$. Then by i.i.d.:

$P(W^{(n)}\leq w)=P\left(\bigcap_{i=1}^{n}\\{W_{i}\leq
w\\}\right)=F_{Y}^{n}(w),$

where $W^{(1)}\leq...\leq W^{(n)}$. Then define
$w_{0}=\sup\\{w:F_{W}(w)<1\\}$. Then for some $w^{\prime}<w_{0}$,
$P(W^{(n)}\leq w^{\prime})=F_{W}^{n}(w^{\prime})=0$, since
$F_{W}(w^{\prime})<1$. As such, $W^{(n)}$ converges almost surely, and by
extension, in probability, to $w_{0}$. We note that the same result can be
applied for the minima of $\\{W_{i}\\}_{i=1}^{n}$ by using
$-\\{W_{i}\\}_{i=1}^{n}$.

Now, define $\lambda_{i}:=w_{i}^{*}/w_{i}$. We have restricted the set of
plausible $\lambda_{i}$ such that
$\lim\inf\\{\lambda:1-F_{\lambda}(\lambda)<1\\}=0$, or
$\lim\sup\\{\lambda:F_{\lambda}(\lambda)<1\\}\to\infty$. First consider the
setting for $\lim\inf\\{\lambda:1-F_{\lambda}(\lambda)<1\\}=0$. We can apply
the results from above to show that for a sequence of random
$\lambda_{1},...,\lambda_{n}$, the minimum of the sequence will converge in
probability towards zero Because $\Lambda=\max_{1\leq i\leq
n}\left\\{\lambda_{i},1/\lambda_{i}\right\\}$, this implies that $\Lambda$
will diverge in probability towards infinity. Similarly, for
$\lim\sup\\{\lambda:F_{\lambda}(\lambda)<1\\}\to\infty$, the maximum of the
sequence will diverge in probability towards infinity, which implies that
$\Lambda$ will diverge in probability towards infinity.

Applying Continuous Mapping Theorem and sample boundedness, the length of the
point estimate bounds under the marginal sensitivity model ($\psi(\Lambda)$)
will be equal to the range of the observed control outcomes. Thus, if the
outcomes $Y_{i}$ are unbounded as well (i.e., $F_{Y}(y)<1$ for all
$y\in\mathbbm{R}$.), $\psi(\Lambda)$ will diverge in probability to infinity.

Thus, we have shown that $\psi(\Lambda)$ will diverge in probability to
infinity. Applying Continuous Mapping Theorem again,
$g(\psi(\Lambda);Y_{i},w_{i})\stackrel{{\scriptstyle p}}{{\rightarrow}}1$,
which concludes the proof.

##  Appendix C Extended Tables

Benchmarking Results
Covariate $\hat{\Lambda}$ MSM $\hat{R}^{2}$ $\hat{\rho}$ VBM VBM, w/ Corr.
Gender 1.1 [1.71, 2.57] 0.00 0.01 [1.81, 2.48] [1.81, 2.48] Age 2.1 [0.98,
3.10] 0.12 -0.01 [1.24, 3.02] [1.87, 2.43] Income 2.9 [0.56, 3.32] 0.14 -0.09
[1.18, 3.07] [1.87, 2.43] Income (Missing) 1.2 [1.62, 2.64] 0.00 -0.05 [1.81,
2.48] [1.81, 2.48] Education 4.7 [-0.06, 3.55] 0.17 0.06 [1.10, 3.15] [1.77,
2.52] Cig. Smoked 2.2 [0.92, 3.13] 0.01 -0.01 [1.66, 2.62] [1.81, 2.48]
Smoking History 1.5 [1.38, 2.83] 0.04 -0.02 [1.50, 2.77] [1.82, 2.47] Race 3.5
[0.31, 3.43] 0.19 -0.09 [1.05, 3.20] [1.88, 2.42]

Table 3: Benchmarking results for both the marginal sensitivity model and the
variance-based sensitivity model. We include both the benchmarked parameter
values, as well as the estimated 95% confidence intervals.