12 YYYY

# Identification and Inference with Invalid Instruments

Hyunseung Kang  1 Zijian Guo  2 Zhonghua Liu  3 and Dylan Small4 1Department
of Statistics, University of Wisconsin-Madison, Madison WI, United States,
53706; email<EMAIL_ADDRESS>2Department of Statistics, Rutgers
University, Piscataway NJ, United States, 08854; email:
<EMAIL_ADDRESS>3Department of Biostatistics, Columbia University,
New York NY, United States, 10032; email<EMAIL_ADDRESS>4Department of Statistics and Data Science, The Wharton School, University of
Pennsylvania, Philadelphia PA, United States, 19104; email:
<EMAIL_ADDRESS>
###### Abstract

Instrumental variables (IVs) are widely used to study the causal effect of an
exposure on an outcome in the presence of unmeasured confounding. IVs require
an instrument, a variable that is (A1) associated with the exposure, (A2) has
no direct effect on the outcome except through the exposure, and (A3) is not
related to unmeasured confounders. Unfortunately, finding variables that
satisfy conditions (A2) or (A3) can be challenging in practice. This paper
reviews works where instruments may not satisfy conditions (A2) or (A3), which
we refer to as invalid instruments. We review identification and inference
under different violations of (A2) or (A3), specifically under linear models,
non-linear models, and heteroskedatic models. We conclude with an empirical
comparison of various methods by re-analyzing the effect of body mass index on
systolic blood pressure from the UK Biobank.

###### doi:

10.1146/((please add article doi))

###### keywords:

Heteroskedasticity, instrumental variables, invalid instruments, Mendelian
randomization,

††journal: Annu. Rev. Stat. Appl.

###### Contents

  1. 1 INTRODUCTION
  2. 2 LINEAR MODELS
    1. 2.1 Model and Definition of Invalid Instruments
    2. 2.2 Identification of the Causal Effect of the Exposure with Invalid Instruments
    3. 2.3 Estimation and Inference of the Causal Effect of the Exposure
      1. 2.3.1 Consistent Estimators of $\beta$
      2. 2.3.2 Pointwise Inference of $\beta$
      3. 2.3.3 Uniformly Valid Inference of $\beta$
    4. 2.4 Connection to Two-Sample Summary Data Design in Mendelian Randomization
  3. 3 NONLINEAR MODELS
  4. 4 HETEROSKEDASTIC MODELS
  5. 5 ILLUSTRATION WITH REAL DATA
    1. 5.1 Background and Setup
    2. 5.2 Results
  6. 6 DISCUSSION

##  1 INTRODUCTION

Instrumental variables (IVs) are a popular tool to estimate the causal effect
of an exposure on an outcome in the presence of unmeasured confounders, which
are unmeasured variables that affect both the exposure and the outcome.
Briefly, IVs require finding a variable called an instrument that satisfies
three core conditions:

  1. (A1)

The instrument is related to the exposure;

  2. (A2)

The instrument has no direct pathway to the outcome except through the
exposure;

  3. (A3)

The instrument is not related to unmeasured confounders that affect the
exposure and the outcome.

See Hernán & Robins (2006), Baiocchi et al. (2014) and Section 2.1 for
details. We remark that condition (A2) is often referred to as the exclusion
restriction (Imbens & Angrist, 1994, Angrist et al., 1996). IVs can be a
valuable tool in settings where a randomized trial, which is the gold standard
for dealing with unmeasured confounders, is impractical.

Oftentimes there is uncertainty about whether candidate instruments, in fact,
satisfy (A1)-(A3), especially conditions (A2) and (A3); this problem is
loosely referred to as the invalid instruments problem (Murray, 2006, Conley
et al., 2012). For example, if the instrument is a genetic marker, which is
the case in Mendelian randomization (MR) (Davey Smith & Ebrahim, 2003, 2004),
satisfying (A2) would imply that the genetic marker’s only biological function
is to affect the exposure. However, this assumption is untenable for many
genetic markers as they often have multiple biological functions, a phenomenon
broadly known as pleiotropy (Solovieff et al., 2013, Hemani et al., 2018).
More generally, without a complete understanding of the instruments’
downstream effects on the outcome, IVs are plagued by possible violations of
condition (A2). Also, when condition (A1) is weakly satisfied in that the
instrument’s association to the exposure is “small” (i.e. weak instruments;
see Staiger & Stock (1997),Stock et al. (2002) and Andrews et al. (2019)), a
slight violation of condition (A2) can lead to dramatically biased estimates
of the causal effect of the exposure (Bound et al., 1995, Hahn & Hausman,
2005, Small & Rosenbaum, 2008).

Recent works have developed some promising frameworks to study the
identification and inference of the causal effect of the exposure when some
candidate instruments are, in fact, invalid. In this paper, we organize these
works into the following categories:

  1. 1.

_Linear Models_ : One of the earliest works in this literature is to
generalize a popular linear, IV model to parametrize the violations of
conditions (A2) and (A3). Roughly speaking, instruments are invalid under
linear models if they have non-zero, partial effects on the outcome after
conditioning on the exposure. Some works in this area include Han (2008),
Kolesár et al. (2015), Kang et al. (2016b), Guo et al. (2018), Windmeijer et
al. (2019, 2021), Fan & Wu (2022), and Guo (2023). Given that the invalid
instruments violate conditions (A2) and (A3) in a linear fashion, these works
generate simple conditions for identification, most notably the majority rule.
The majority rule states that a majority of the instruments are valid, without
knowing which instruments are valid a priori. Also, some of these works have
relatively straightforward methods for estimation, such as the median
estimator. However, if the linear outcome model is mis-specified, these
methods can lead to wrong conclusions about the effect of the exposure.

  2. 2.

_Nonlinear Models_ : Recent works utilize the non-linearities in the exposure
model when instruments do not satisfy conditions (A2) and (A3).For example,
Guo et al. (2022) proposed finding non-linear trends in the exposure model via
machine learning methods to identify and infer the causal effect of the
exposure. Sun et al. (2023) proposed using higher-order interactions of
instruments and G estimation (Robins et al., 1992, Robins, 1994) to identify
and infer the causal effect of the exposure. Critically, both of these methods
rely on the exposure model’s nonlinearity for identification, and it’s
important to check the required nonlinearity conditions to ensure that the
identification conditions are plausible with the data (Lewbel, 2019).

  3. 3.

_Heteroskedastic Models_ : These works utilize heteroskedsaticity in the
exposure model or the outcome model to identify the causal effect with invalid
instruments. Some works in this area include Lewbel (2012), Tchetgen Tchetgen
et al. (2021), Sun et al. (2022), Ye et al. (2024) and Liu et al. (2023).
Similar to the nonlinear modeling framework above, these methods require
heteroskedacity and the required heteroskedascity condition should be checked
in practice.

The rest of the paper goes into the details behind each category above. We
also empirically compare the methods discussed in the paper by re-analyzing
the causal effect of body mass index on blood pressure from the UK Biobank. We
remark that the review does not discuss, in detail, an important interplay
between weak instruments and invalid instruments and the challenges that they
pose. The intersection between weak instruments and invalid instruments have
recently gained considerable attention and we provide a summary of the recent
works in this area plus other remaining questions in the study of invalid
instruments at the end of the paper.

##  2 LINEAR MODELS

###  2.1 Model and Definition of Invalid Instruments

Let $Y\in\mathbb{R}$ denote the outcome, $D\in\mathbb{R}$ denote the exposure,
and $\mathbf{Z}\in\mathbb{R}^{p}$ denote the $p$ instruments. Consider the
following model for $Y,D$ and $\mathbf{Z}$ where, without loss of generality,
we assume that $Y$, $D$, and $\mathbf{Z}$ are centered to mean zero:

$\displaystyle D$ $\displaystyle=\mathbf{Z}^{\intercal}\bm{\gamma}+\delta,$
$\displaystyle\mathbb{E}(\delta\mid\mathbf{Z})=0,$ (1) $\displaystyle Y$
$\displaystyle=D\beta+\mathbf{Z}^{\intercal}\bm{\pi}+\epsilon,$
$\displaystyle\mathbb{E}(\epsilon\mid\mathbf{Z})=0.$ (2)

In Equation 1, the parameter $\bm{\gamma}\in\mathbb{R}^{p\times 1}$ represents
the instruments’ relevance to the exposure. For this review, we only consider
the case where all instruments are relevant (i.e., $\gamma_{j}\neq 0$ for all
$j=1,\ldots,p$) in order to focus on invalid instruments. For extensions to
identifying and inferring the causal effect when some instruments are not
relevant and some instruments are invalid, see Guo et al. (2018) and Fan & Wu
(2022).

In Equation 2, the parameter $\beta\in\mathbb{R}$ represents the effect of the
exposure on the outcome and is the main parameter of interest in IVs. The
parameter $\beta$ is sometimes referred to as a “structural” parameter
(Goldberger, 1972, Wooldridge, 2010, Angrist & Pischke, 2009), to distinguish
it from the usual regression coefficient or a “reduced-form” parameter, which
can often be estimated consistently with ordinary least squares. Specifically,
the estimate of the regression coefficients from running an ordinary least
squares regression of $Y$ on $D$ and $\mathbf{Z}$, denoted as
$\hat{\beta}_{\rm ols}$, is generally inconsistent for $\beta$ because the
variable $D$ is not independent of the error term $\epsilon$. In contrast, for
the parameters in Equation 1, we can run a ordinary least squares regression
of $D$ on $\mathbf{Z}$ to obtain a consistent estimator of $\bm{\gamma}$.

The parameter $\bm{\pi}\in\mathbb{R}^{p\times 1}$ represents the effects of
the instruments on the outcome after adjusting for the exposure. If
$\bm{\pi}=\bm{0}$, Equation 2 reduces to a well-studied, linear IV model
(Angrist & Pischke, 2009, Wooldridge, 2010) and all $p$ instruments are said
to be valid instruments. If $\bm{\pi}\neq\bm{0}$, some of the $p$ instruments
are not valid and the non-zero elements of $\bm{\pi}$ encode which of the
instruments are invalid.

######  Definition 1 (Valid Instrument).

Suppose Equation 1 and Equation 2 hold. Instrument $j\in\\{1,\ldots,p\\}$ is
valid if $\pi_{j}=0$ and invalid if $\pi_{j}\neq 0$. Let
$\mathcal{V}=\\{j:\pi_{j}=0\\}$ be the set of valid instruments.

If the set of valid instruments $\mathcal{V}$ is known a priori and there is
at least one valid instrument (i.e., the size of the set, denoted as
$|\mathcal{V}|$, is greater than or equal to $1$), Equation 2 again reduces to
a well-studied linear IV model where the complement of $\mathcal{V}$ serves as
the control variables; see Chapter 5 of Wooldridge (2010) for an example. But,
in practice, the knowledge about $\mathcal{V}$ is unknown, and the central
goal under the invalid IV framework is to study identification and inference
of the causal effect when there is no a priori knowledge about which
instruments among the $p$ candidate instruments are valid.

Definition 1 of a valid instrument is closely related to the definition of a
valid instrument under an additive, linear, constant effects (ALICE) potential
outcomes model (Holland, 1988). Let $Y(d,\mathbf{z})\in\mathbb{R}$ be the
potential outcome if an individual were to have exposure $d\in\mathbb{R}$ and
instruments $\mathbf{z}\in\mathbb{R}^{p}$. For $d,d^{\prime}\in\mathbb{R}$ and
$\mathbf{z},\mathbf{z}^{\prime}\in\mathbb{R}^{p}$, the ALICE model states

$Y(d^{\prime},\mathbf{z}^{\prime})-Y(d,\mathbf{z})=(d^{\prime}-d)\tilde{\beta}+(\mathbf{z}^{\prime}-\mathbf{z})^{\intercal}\tilde{\bm{\psi}},\quad{}\mathbb{E}[Y(0,\mathbf{0})\mid\mathbf{Z}]=\mathbf{Z}^{\intercal}\tilde{\bm{\phi}}.$
(3)

The tilde notation in Equation 3 emphasizes that the model parameters are from
the potential outcomes model. The parameter $\tilde{\beta}$ represents the
causal effect of changing the exposure by one unit on the outcome. Each $j$th
element of $\tilde{\bm{\psi}}\in\mathbb{R}^{p\times 1}$ represents the causal
effect of changing the $j$th instrument by one unit on the outcome. The term
$\mathbb{E}[Y(0,\mathbf{0})\mid\mathbf{Z}]=\mathbf{Z}^{\intercal}\tilde{\bm{\phi}}$
where $\tilde{\bm{\phi}}\in\mathbb{R}^{p\times 1}$ represents the effect from
unmeasured confounding. If the no direct effect condition (A2) is formalized
as $Y(d,\mathbf{z})=Y(d,\mathbf{z}^{\prime})$ for all
$d,\mathbf{z},\mathbf{z}^{\prime}$, we have $\tilde{\bm{\psi}}=\mathbf{0}$. If
the no instrument confounding condition (A3) is formalized as $\mathbf{Z}\perp
Y(d,\mathbf{z})$ where $\perp$ stands for independence between two random
variables, we have $\mathbf{Z}^{\intercal}\tilde{\bm{\phi}}=\mathbf{0}$. Also,
if we assume the stable unit treatment value assumption (Rubin, 1980) or
causal consistency where $Y=Y(D,\mathbf{Z})$, Equation 3 simplifies to
Equation 2 where $\tilde{\beta}=\beta$,
$\bm{\pi}=\tilde{\bm{\psi}}+\tilde{\bm{\phi}}$, and
$\epsilon=Y(0,\mathbf{0})-\mathbb{E}[Y(0,\mathbf{0})\mid\mathbf{Z}]$. In other
words, under the ALICE model for potential outcomes and causal consistency,
the violations of conditions (A2) and (A3) are summarized with the parameter
$\bm{\pi}$.

We make some other important remarks about the definition of a valid
instrument. First, the validity of instrument $j$ depends on the candidate set
of instruments. Instrument $j$ may be valid with one set of instruments but
may not be valid with another set of instruments. Second, if we have
covariates $\mathbf{X}$ that are independent of the error terms in Equation 1
and Equation 2, we can adjust for $\mathbf{X}$ by first fitting a linear
regression of $Y$ on $\mathbf{X}$, $D$ on $\mathbf{Z}$, and $\mathbf{Z}$ on
$\mathbf{X}$. Then we replace $Y$, $D$, and $\mathbf{Z}$ in Equation 1 and
Equation 2 with the residuals of the linear regressions from the first step.
This procedure is justified by the Frisch-Waugh-Lovell theorem (Davidson &
MacKinnon, 1993). Third, several works (Hahn & Hausman, 2005, Berkowitz et
al., 2008, 2012, Guggenberger, 2012, Conley et al., 2012, Armstrong & Kolesár,
2021) have studied the properties of existing estimators and tests of $\beta$
when there is a near-violation of instrument validity. Roughly speaking, a
near-violation of instrument validity is characterized as
$\tilde{\bm{\psi}}=\mathbf{0},\tilde{\bm{\phi}}=\mathbf{C}/\sqrt{n}$ or as
$\bm{\pi}=\mathbf{C}/\sqrt{n}$ for some constant vector
$\mathbf{C}\in\mathbb{R}^{p}$ and $n$ is the sample size. These works showed
that existing estimators will be biased and tests of $\beta$ will have an
inflated Type I error rate. Fourth, Liao (2013), Cheng & Liao (2015),
DiTraglia (2016) and Patel et al. (2024) considered estimating $\beta$ when
there is a known set of valid instruments and another set of potentially
invalid instruments. Finally, we remark that there are works on constructing
bounds of $\beta$ (Small, 2007, Baiocchi et al., 2010, Kang et al., 2016a,
Ashley & Parmeter, 2015, Fogarty et al., 2021) under various assumptions about
the magnitude of $\bm{\pi}$.

###  2.2 Identification of the Causal Effect of the Exposure with Invalid
Instruments

To better understand how $\beta$ can be identified when the set of valid
instruments $\mathcal{V}$ is unknown, it’s useful to consider a model of $Y$
that only depends on $\mathbf{Z}$, often referred to as a reduced-form model:

$\displaystyle Y$ $\displaystyle=\mathbf{Z}^{\intercal}\bm{\Gamma}+e,$
$\displaystyle\bm{\Gamma}=\beta\bm{\gamma}+\bm{\pi},\ e=\beta\delta+\epsilon,\
\mathbb{E}(e\mid\mathbf{Z})=0.$ (4)

With the observed data $Y,D$ and $\mathbf{Z}$, we can identify the parameters
$\bm{\gamma}\in\mathbb{R}^{p\times 1}$ and $\bm{\Gamma}\in\mathbb{R}^{p\times
1}$ based on the following relationships from ordinary least squares:

$\bm{\Gamma}=\mathbb{E}(\mathbf{Z}\mathbf{Z}^{\intercal})^{-1}\mathbb{E}(\mathbf{Z}Y),\quad{}\bm{\gamma}=\mathbb{E}(\mathbf{Z}\mathbf{Z}^{\intercal})^{-1}\mathbb{E}(\mathbf{Z}D).$

With the parameters $\bm{\Gamma}$ and $\bm{\gamma}$ identified from the data,
we can reframe identifying $\beta$ as finding a unique value of
$(\beta,\bm{\pi})$ from $(\bm{\gamma},\bm{\Gamma})$ based on the system of
linear equations in Equation 4 (i.e.,
$\bm{\Gamma}=\beta\bm{\gamma}+\bm{\pi}$). Also, the system of linear equations
reveals the role that $\mathcal{V}$ plays in identifying $\beta$. For example,
suppose $\bm{\pi}=\mathbf{0}$ so that all instruments are valid and
$\mathcal{V}=\\{1,\ldots,p\\}$. Then, the linear system of equations
simplifies to $\bm{\Gamma}=\beta\bm{\gamma}$ and we can identify $\beta$ given
$\bm{\Gamma}$ and $\bm{\gamma}$ by setting $\beta=\Gamma_{j}/\gamma_{j}$ for
any $j=1,\ldots,p$. Or, suppose the set of valid instruments $\mathcal{V}$ is
known a priori and there is at least one valid instrument (i.e.
$|\mathcal{V}|\geq 1$). Then, for $j\in\mathcal{V}$, the system of linear
equations simplifies to $\Gamma_{j}=\beta\gamma_{j}$ and we can again identify
$\beta$. Finally, if there are no restrictions on $\bm{\pi}$, there are no
unique values of $(\beta,\bm{\pi})$ given $(\bm{\gamma},\bm{\Gamma})$ since
there are $p$ linear equations and $p+1$ unknown variables.

When $\mathcal{V}$ is unknown, Han (2008) and Kang et al. (2016b) proposed
what’s now called the majority rule to identify $\beta$:

$|\mathcal{V}|>\frac{p}{2}.$ (5)

Simply put, the majority rule states that the number of valid instruments is
more than 50% of the instruments. Critically, we do not have to know a priori
which instruments are valid; we simply have to know that the majority of the
instruments are valid.

We briefly illustrate how the majority rule in Equation 5 places a constraint
on the linear system of equations above to identify $\beta$; for the full
proof, see Kang et al. (2016b) where they prove a necessary and sufficient
condition to have a unique solution of $(\beta,\bm{\pi})$ given
$(\bm{\gamma},\bm{\Gamma})$ and a lower bound on $|\mathcal{V}|$. Given
$(\bm{\gamma},\bm{\Gamma})$, let $(\beta,\bm{\pi})$ and
$(\beta^{\prime},\bm{\pi}^{\prime})$ be the solutions to the system of
equations, i.e., $\bm{\Gamma}=\beta\bm{\gamma}+\bm{\pi}$ and
$\bm{\Gamma}=\beta^{\prime}\bm{\gamma}+\bm{\pi}^{\prime}$. Let $\mathcal{V}$
and $\mathcal{V}^{\prime}$ denote the sets of valid instruments defined by
$\bm{\pi}$ and $\bm{\pi}^{\prime}$, respectively. By the majority rule,
$\mathcal{V}\cap\mathcal{V}^{\prime}\neq\emptyset$ and we can always pick
$j\in\mathcal{V}\cap\mathcal{V}^{\prime}$ where $\pi_{j}=\pi_{j}^{\prime}=0$.
For instrument $j$, the system of linear equations simplifies to
$\Gamma_{j}=\beta\gamma_{j}$ and $\Gamma_{j}=\beta^{\prime}\gamma_{j}$, which
imply that $\beta=\beta^{\prime}$. Furthermore, we have
$\bm{\pi}=\bm{\Gamma}-\beta\bm{\gamma}=\bm{\Gamma}-\beta^{\prime}\bm{\gamma}=\bm{\pi}^{\prime}$
and thus, the solution to the system of linear equations is unique.

We can also construct a falsification test of the majority rule where the
“null hypothesis” assumes that the majority rule holds. For example, suppose
the majority rule holds, and the non-zero elements of $\bm{\pi}$ are far away
from $0$. In this setting, there should only be one large cluster of
instruments with the same $\Gamma_{j}/\gamma_{j}$, and the size of this
cluster should be greater than $p/2$. If we do not see such a cluster, this
indicates a violation of the majority rule. For additional details on how to
conduct this test, see Guo et al. (2023).

Next, Guo et al. (2018) proposed the plurality rule to identify $\beta$:

$|\mathcal{V}|>\max_{c\neq
0}|\mathcal{I}_{c}|\quad\text{with}\quad\mathcal{I}_{c}=\left\\{j\in\\{1,\ldots,p\\}:\frac{\pi_{j}}{\gamma_{j}}=c\right\\}.$
(6)

In words, $\mathcal{I}_{c}$ denotes a subset of instruments that have a common
value of $\Gamma_{j}/\gamma_{j}$, specifically
$\Gamma_{j}/\gamma_{j}=\beta+c$. Note that the set of valid instruments
$\mathcal{V}$ equals to $\mathcal{I}_{0}$ where $c=0$. The plurality condition
requires that the set of valid instruments $\mathcal{V}$ is the largest among
all subsets of instruments with a common value of $\Gamma_{j}/\gamma_{j}$ that
is not equal to $\beta$. Also, if the majority rule holds, the plurality rule
holds since the size of the set $\mathcal{I}_{c}$ for any $c\neq 0$ cannot be
greater than $p/2$, i.e., $p/2>\max_{c\neq 0}|\mathcal{I}_{c}|$. In other
words, the majority rule is a sufficient condition for the plurality rule.
Finally, we can also falsify the plurality rule, albeit under more stringent
conditions; see Guo et al. (2023) for details.

We conclude with a remark about the work by Andrews (1999). Andrews (1999)
primarily focused on developing a model selection procedure to consistently
estimate $\mathcal{V}$. The model selection procedure relies on using a test
statistic called the $J$ test (Hansen, 1982) to distinguish between valid
instruments and invalid instruments, and a couple of the works discussed below
use this procedure to tune relevant tuning parameters. Also, when
characterizing the properties of the proposed model selection procedure,
Andrews (1999) proposed a condition for identifying $\beta$ that is a version
of the plurality rule. Specifically, for any subset
$\mathcal{C}\subseteq\\{1,\ldots,p\\}$ and a vector
$\mathbf{v}\in\mathbb{R}^{p}$, let
$\mathbf{v}_{\mathcal{C}}\in\mathbb{R}^{|\mathcal{C}|}$ be the vector with
elements defined by the subset $\mathcal{C}$. Then, Andrews (1999) stated that
$\beta$ is identified if

$\forall\mathcal{C}\text{ where }|\mathcal{C}|\geq|\mathcal{V}|\text{ and
}\mathcal{V}\neq\mathcal{C},\quad\bm{\pi}_{\mathcal{C}}\neq
q\bm{\gamma}_{\mathcal{C}}$

for any $q\neq 0$.

###  2.3 Estimation and Inference of the Causal Effect of the Exposure

####  2.3.1 Consistent Estimators of $\beta$

We lay out the following notations to describe different estimators of
$\beta$. For each study unit $i=1,\ldots,n$, let
$(Y_{i},D_{i},\mathbf{Z}_{i})\in\mathbb{R}\otimes\mathbb{R}\otimes\mathbb{R}^{p}$
be the observed outcome, exposure, and $p$ instruments, respectively. Let
$\mathbf{Y}=(Y_{1},\ldots,Y_{n})\in\mathbb{R}^{n\times 1}$,
$\mathbf{D}=(D_{1},\ldots,D_{n})\in\mathbb{R}^{n\times 1}$, and
$\mathbf{Z}=(\mathbf{Z}_{1},\ldots,\mathbf{Z}_{n})\in\mathbb{R}^{n\times p}$.
As before, without loss of generality, we assume that the vectors $\mathbf{Y}$
and $\mathbf{D}$ as well as the columns of the matrix $\mathbf{Z}$ are
centered to mean zero. Let
$\mathbf{P}_{\mathbf{Z}}=\mathbf{Z}(\mathbf{Z}^{\intercal}\mathbf{Z})^{-1}\mathbf{Z}^{\intercal}\in\mathbb{R}^{n\times
n}$ be the projection matrix onto the column space of $\mathbf{Z}$ and let
$\mathbf{P}_{\mathbf{Z}}^{\perp}=\mathbf{I}-\mathbf{\mathbf{P}}_{\mathbf{Z}}\in\mathbb{R}^{n\times
n}$ be the residual projection matrix where $\mathbf{I}\in\mathbb{R}^{n\times
n}$ is the identity matrix. For any vector $\mathbf{v}\in\mathbb{R}^{p}$ and
$q\geq 1$, let $\|\mathbf{v}\|_{q}=(\sum_{j=1}^{p}v_{j}^{q})^{1/q}$ be its $q$
norm and let $v_{j}$ denote the $j$th element of $\mathbf{v}$. Finally, for a
set $\mathcal{V}\subseteq\\{1,\ldots,p\\}$, let $\mathcal{V}^{c}$ denote its
complement and $\mathbf{Z}_{\mathcal{V}}\in\mathbb{R}^{n\times|\mathcal{V}|}$
denote the matrix $\mathbf{Z}$ with the columns specified by $\mathcal{V}$.

We start by describing the two-stage least squares (TSLS) estimator, a popular
estimator of $\beta$ when $\mathcal{V}$ is known a priori and
$|\mathcal{V}|\geq 1$. The two-stage least squares estimator first fits a
linear regression model of $\mathbf{D}$ on $\mathbf{Z}$ and obtains the
predicted values of $\mathbf{D}$, i.e., $\mathbf{P}_{\mathbf{Z}}\mathbf{D}$.
Second, it fits a linear regression model of $\mathbf{Y}$ on
$\mathbf{P}_{\mathbf{Z}}\mathbf{D}$ and $\mathbf{Z}_{\mathcal{V}^{c}}$. If
$\mathcal{V}^{c}$ is an empty set (i.e., all of the instruments are valid), we
drop the $\mathbf{Z}_{\mathcal{V}^{c}}$ term in the second regression model.
The two-stage least squares estimator of $\beta$ is the estimated regression
coefficient in front of $\mathbf{P}_{\mathbf{Z}}\mathbf{D}$ from the second
regression model. More succinctly, given a set of valid IVs $\mathcal{V}$ and
$|\mathcal{V}|\geq 1$, the two-stage least squares is the solution to the
following optimization problem:

$\bigl{[}\hat{\beta}_{\rm tsls}(\mathcal{V}),\hat{\bm{\pi}}_{\rm
tsls}(\mathcal{V})\bigr{]}={\rm
argmin}_{\beta,\bm{\pi}_{\mathcal{V}^{c}}}\frac{1}{2}\|\mathbf{P}_{\mathbf{Z}}(\mathbf{Y}-\mathbf{D}\beta-\mathbf{Z}_{\mathcal{V}^{c}}\bm{\pi}_{\mathcal{V}^{c}})\|_{2}^{2}.$
(7)

Some works on invalid instruments compare their proposed estimators of
$\beta$, which do not know $\mathcal{V}$ a priori, with the two-stage least
squares estimator with a known $\mathcal{V}$; if used in this context, the
two-stage least squares estimator is sometimes referred to as the oracle
estimator (Guo et al., 2018, Windmeijer et al., 2021). If their proposed
estimator of $\beta$ is asymptotically equivalent to the oracle estimator, the
proposed estimator is said to be oracle-optimal in the literature.

One of the first methods to estimate $\beta$ when $\mathcal{V}$ is unknown a
priori is the median estimator of Han (2008). Consider the ordinary least
squares estimators of $\bm{\Gamma}$ and $\bm{\gamma}$:

$\hat{\bm{\Gamma}}=(\mathbf{Z}^{\intercal}\mathbf{Z})^{-1}\mathbf{Z}^{\intercal}\mathbf{Y},\quad{}\hat{\bm{\gamma}}=(\mathbf{Z}^{\intercal}\mathbf{Z})^{-1}\mathbf{Z}^{\intercal}\mathbf{D}.$

Under mild assumptions (see Chapter 4 of Wooldridge (2010)),
$\hat{\bm{\Gamma}}$ and $\hat{\bm{\gamma}}$ are asymptotically normal:

$\sqrt{n}\left[\begin{pmatrix}\hat{\bm{\Gamma}}\\\
\hat{\bm{\gamma}}\end{pmatrix}-\begin{pmatrix}\bm{\Gamma}\\\
\bm{\gamma}\end{pmatrix}\right]\xrightarrow{d}N\left[\begin{pmatrix}\bm{0}\\\
\bm{0}\end{pmatrix},\begin{pmatrix}\bm{\Omega}_{\Gamma}&\bm{\Omega}_{\Gamma\gamma}\\\
\bm{\Omega}_{\Gamma\gamma}^{\intercal}&\bm{\Omega}_{\gamma}\end{pmatrix}\right].$
(8)

We remark that the covariance matrices
$\bm{\Omega}_{\Gamma}\in\mathbb{R}^{p\times
p},\bm{\Omega}_{\Gamma\gamma}\in\mathbb{R}^{p\times p}$ and
$\bm{\Omega}_{\gamma}\in\mathbb{R}^{p\times p}$ can be consistently estimated.
The median estimator of $\beta$, denoted as $\hat{\beta}_{\rm med}$, can be
written as the median of $p$ ratios of $\hat{\Gamma}_{j}/\hat{\gamma}_{j}$,
$j=1,\ldots,p$:

$\displaystyle\hat{\beta}_{\rm med}$ $\displaystyle={\rm
median}\\{\tilde{\beta}_{1},\ldots,\tilde{\beta}_{p}\\},\quad{}\tilde{\beta}_{j}=\frac{\hat{\Gamma}_{j}}{\hat{\gamma}_{j}}.$
(9)

Han (2008) established that $\hat{\beta}_{\rm med}$ is consistent if the
majority rule in Equation 5 holds. Later, Windmeijer et al. (2019) established
that the limiting distribution of the median estimator is an order statistic
of a normal distribution. However, inference based on the median estimator is
challenging due to the non-negligible bias of the order statistic (Windmeijer
et al., 2019, Guo et al., 2023).

Kang et al. (2016b) proposed a Lasso-based estimator of $\beta$, which the
authors referred to as SISVIVE (Some Invalid, Some Valid Instrumental
Variables Estimator). SISVIVE is inspired by the two-stage least squares
estimator in Equation 7 and directly solves for the model parameters in
Equation 2 with a penalty term on $\bm{\pi}$:

$\bigl{[}\hat{\beta}_{\rm sisvive},\hat{\bm{\pi}}_{\rm
sisvive}\bigr{]}=\underset{\beta,\bm{\pi}}{\rm argmin}\
\frac{1}{2}\|\mathbf{P}_{\mathbf{Z}}(\mathbf{Y}-\mathbf{D}\beta-\mathbf{Z}\bm{\pi})\|_{2}^{2}+\lambda\|\bm{\pi}\|_{1},\quad{}\lambda>0.$
(10)

This optimization problem can be solved with existing penalized regression
software by reformulating Equation 10 as follows:

$\displaystyle\hat{\bm{\pi}}_{\rm sisvive}$
$\displaystyle=\underset{\bm{\pi}}{\rm argmin}\
\frac{1}{2}\|(\mathbf{P}_{\mathbf{Z}}-\mathbf{P}_{\mathbf{P}_{\mathbf{Z}}\mathbf{D}})(\mathbf{Y}-\mathbf{Z}\bm{\pi})\|_{2}^{2}+\lambda\|\bm{\pi}\|_{1},$
$\displaystyle\hat{\beta}_{\rm sisvive}$
$\displaystyle=\frac{(\mathbf{P}_{\mathbf{Z}}\mathbf{D})^{\intercal}(\mathbf{Y}-\mathbf{Z}\hat{\bm{\pi}}_{\rm
sisvive})}{\|\mathbf{P}_{\mathbf{Z}}\mathbf{D}\|_{2}^{2}}.$

The first step of the two-step procedure estimates $\bm{\pi}$ by using
existing software for the Lasso (e.g. Efrom et al. (2004)). The second step is
a dot product between two $n$ dimensional vectors. Kang et al. (2016b)
established conditions when $\hat{\beta}_{\rm sisvive}$ is consistent for
$\beta$ and recommended choosing the tuning parameter $\lambda$ by cross-
validation. Later, Bao et al. (2019) studied the finite sample properties of
$\hat{\beta}_{\rm sisvive}$ through a simulation study.

Finally, Kolesár et al. (2015) showed that the $k$-class estimator from
Anatolyev (2013), denoted as $\hat{\beta}_{\rm kclass}$ and formalized as

$\hat{\beta}_{\rm
kclass}=\frac{\mathbf{D}^{\intercal}(\mathbf{I}-k\mathbf{P}_{\mathbf{Z}}^{\perp})\mathbf{Y}}{\mathbf{D}^{\intercal}(\mathbf{I}-k\mathbf{P}_{\mathbf{Z}}^{\perp})\mathbf{D}},\quad{}k=\frac{1-\frac{1}{n}}{1-\frac{p}{n}-\frac{1}{n}},$

is consistent for $\beta$ when both the number of instruments $p$ and the
number of samples $n$ grow to infinity and the following orthogonality
condition between $\bm{\pi}$ and $\bm{\gamma}$ hold:

$\frac{1}{n}\bm{\pi}^{\intercal}\mathbf{Z}^{\intercal}\mathbf{Z}\bm{\gamma}\to
0,\quad{}\frac{p}{n}\to c\in[0,1).$ (11)

In words, the orthogonality condition states that the effect of the
instruments on the exposure (i.e., $\bm{\gamma}$) is orthogonal to the direct
effect of the instruments on the outcome (i.e., $\bm{\pi}$). If all the
instruments are mutually independent of each other, the orthogonality
condition roughly translates to $\bm{\pi}^{\intercal}\bm{\gamma}\approx 0$.
Under this case, the system of linear equations in Equation 4 can be rewritten
as
$\bm{\Gamma}^{\intercal}\bm{\gamma}\approx\beta\bm{\gamma}^{\intercal}\bm{\gamma}$
and the parameter $\beta$ can be identified given $(\bm{\gamma},\bm{\Gamma})$.

Unfortunately, beyond consistency, $\hat{\beta}_{\rm med}$ and
$\hat{\beta}_{\rm sisvive}$ have no inferential guarantees, such as having a
limiting normal distribution to enable testing $H_{0}:\beta=0$ or constructing
a confidence interval for $\beta$. Also, to establish asymptotic normality of
$\hat{\beta}_{\rm kclass}$, Kolesár et al. (2015) further assumed that the
parameters $\bm{\gamma}$ and $\bm{\pi}$ are random and follow a multivariate
normal distribution. The next two sections discuss some progress on conducting
inference about $\beta$.

####  2.3.2 Pointwise Inference of $\beta$

We start with Windmeijer et al. (2019), who proposed to use an adaptive
version of the SISVIVE estimator. Specifically, consider the adaptive Lasso
(Zou, 2006) version of the SISVIVE estimator in Equation 10 where the initial
estimator is the median estimator in Equation 9:

$\displaystyle\hat{\bm{\pi}}_{\rm med}$
$\displaystyle=\hat{\bm{\Gamma}}-\hat{\bm{\gamma}}\hat{\beta}_{\rm med},$
$\displaystyle\hat{\bm{\pi}}_{\rm adlasso}$
$\displaystyle=\underset{\bm{\pi}}{\rm argmin}\
\frac{1}{2}\|(\mathbf{P}_{\mathbf{Z}}-\mathbf{P}_{\mathbf{P}_{\mathbf{Z}}\mathbf{D}})(\mathbf{Y}-\mathbf{Z}\bm{\pi})\|_{2}^{2}+\lambda\sum_{j=1}^{p}\frac{1}{|\hat{\pi}_{{\rm
med},j}|}|\pi_{j}|,\quad{}\lambda>0$ $\displaystyle\hat{\beta}_{\rm adlasso}$
$\displaystyle=\frac{(\mathbf{P}_{\mathbf{Z}}\mathbf{D})^{\intercal}(\mathbf{Y}-\mathbf{Z}\hat{\bm{\pi}}_{\rm
adp})}{\|\mathbf{P}_{\mathbf{Z}}\mathbf{D}\|_{2}^{2}}.$

Windmeijer et al. (2019) showed that $\hat{\beta}_{adlasso}$ is consistent and
asymptotically normal if the majority rule holds. Windmeijer et al. (2019)
also proposed a method to choose $\lambda$ by using a downward testing
procedure of Andrews (1999).

Guo et al. (2018) proposed a different approach, called two-stage hard
thresholding (TSHT), to conduct inference on $\beta$. Broadly speaking, TSHT
treats each instrument $j$ as a voter and uses a plurality voting procedure to
estimate $\beta$. Specifically, consider a voting matrix
$\mathbf{H}\in\mathbb{R}^{p\times p}$ where $H_{j,k}=1$ if the pair of
instruments $j$ and $k$ yield similar estimates of $\beta$ and $H_{j,k}=0$ if
the pair yield different estimates of $\beta$, i.e.,

$H_{j,k}=\begin{cases}1&\text{ if
}\left|\frac{\hat{\Gamma}_{j}}{\hat{\gamma}_{j}}-\frac{\hat{\Gamma}_{k}}{\hat{\gamma}_{k}}\right|\leq
z_{1-\frac{\alpha}{2p}}\cdot\hat{\rm
se}\left(\frac{\hat{\Gamma}_{j}}{\hat{\gamma}_{j}}-\frac{\hat{\Gamma}_{k}}{\hat{\gamma}_{k}}\right),\\\
0&\text{otherwise}.\end{cases}$ (12)

The term $\alpha\in(0,1)$ is the pre-specified significance level,
$z_{1-\alpha/2p}$ is the $1-\alpha/2p$ quantile of the standard normal
distribution, and $\hat{\rm
se}(\hat{\Gamma}_{j}/\hat{\gamma}_{j}-\hat{\Gamma}_{k}/\hat{\gamma}_{k})$ is
the estimated standard error of the difference between
$\hat{\Gamma}_{j}/\hat{\gamma}_{j}$ and $\hat{\Gamma}_{k}/\hat{\gamma}_{k}$.
For each $j=1,\ldots,p$, let
$\|\mathbf{H}_{j,\cdot}\|_{0}=\sum_{k=1}^{p}H_{j,k}$ denote the number of non-
zero elements in the $j$th row of $\mathbf{H}$ (i.e. $\mathbf{H}_{j,\cdot}$).
Roughly speaking, $\|\mathbf{H}_{j,\cdot}\|_{0}$ measures the number of
instruments that are close to the $j$th instrument’s ratio
$\hat{\Gamma}_{j}/\hat{\gamma}_{j}$. $\|\mathbf{H}_{j,\cdot}\|_{0}$ can also
be thought of as the number of votes that instrument $j$ received on being the
valid instrument. From $\|\mathbf{H}_{j,\cdot}\|_{0}$, we can estimate the set
of valid instruments by picking instruments that received a majority or a
plurality of votes:

$\hat{\mathcal{V}}_{\rm tsht}:=\left\\{j\in\\{1,\ldots,p\\}\ \bigl{|}\
\|\mathbf{H}_{j,\cdot}\|_{0}>\frac{p}{2}\right\\}\cup\left\\{j\in\\{1,\ldots,p\\}\
\bigl{|}\
\|\mathbf{H}_{j,\cdot}\|_{0}=\max_{k}\|\mathbf{H}_{k,\cdot}\|_{0}\right\\}$

After estimating $\mathcal{V}$, we can use the two-stage least squares
estimator with the set $\hat{\mathcal{V}}_{\rm tsht}$ to estimate $\beta$.
Under the plurality rule, Guo et al. (2018) showed that this estimator,
denoted as $\hat{\beta}_{\rm tsht}$, is consistent, asymptotically normal and
oracle-optimal for $\beta$. Zhang et al. (2022) recently proposed an
improvement of TSHT that prevents choosing a large number of irrelevant
instruments through a resampling method. They showed that their procedure is
effective in regimes where the number of instruments $p$ is very large.

Windmeijer et al. (2021) proposed the confidence interval method (CIM) for
conducting inference on $\beta$. Roughly speaking, CIM uses “working”
confidence intervals of $\beta$ to cluster instruments and picks the largest
cluster of instruments with overlapping confidence intervals. Specifically,
for each instrument $j$, CIM first constructs $p$ working confidence interval
${\rm CI}_{j}(q_{n})$ of $\beta$:

$\displaystyle{\rm CI}_{j}(q_{n})=[\hat{\beta}_{j}-q_{n}\cdot\hat{\rm
se}(\hat{\beta}_{j}),\hat{\beta}_{j}+q_{n}\cdot\hat{\rm
se}(\hat{\beta}_{j})],\quad{}\tilde{\beta}_{j}$
$\displaystyle=\frac{\hat{\Gamma}_{j}}{\hat{\gamma}_{j}}.$

The term $\hat{\rm se}(\hat{\beta}_{j})$ is the standard error of
$\tilde{\beta}_{j}$ based on applying the delta method to Equation 8. The
parameter $q_{n}$, which depends on the sample size $n$, is set to measure the
similarity between confidence intervals. Note that $q_{n}$ is not equal to
$z_{1-\alpha/2}$, the $1-\alpha/2$ quantile of the standard normal
distribution. Instead, Windmeijer et al. (2021) proposed an adaptive approach
to set $q_{n}$ based on a downward testing procedure of Andrews (1999).
Second, the procedure constructs $K\leq p$ subgroups of instruments where all
instruments in the subgroup have overlapping working confidence intervals:

$V_{k}=\left\\{j\mid{\rm CI}_{j}(q)\cap{\rm CI}_{j^{\prime}}(q)\neq\emptyset\
\forall j,j^{\prime}\in\\{1,\ldots,p\\}\right\\},\quad{}k=1,\ldots,K.$

The CIM estimator of $\mathcal{V}$ is the largest subset of instruments where
all the confidence intervals in the subset overlap with each other:

$\hat{\mathcal{V}}_{\rm
cim}=\\{V_{k}\mid|V_{k}|=\max_{k^{\prime}=1,\ldots,K}|V_{k^{\prime}}|\\}.$

Windmeijer et al. (2021) show that if we construct a two-stage least squares
estimator of $\beta$ with $\hat{\mathcal{V}}_{\rm cim}$, the estimator,
denoted as $\hat{\beta}_{\rm cim}$, is consistent, asymptotically normal, and
oracle-optimal.

While both TSHT and CIM produce confidence intervals for $\beta$ and are
oracle-optimal, a major downside of both procedures is that they rely on
correctly estimating the set of valid instruments asymptotically, i.e.,
$\lim_{n\to\infty}P(\hat{\mathcal{V}}=\mathcal{V})=1$; this property is
sometimes referred to as selection consistency. As noted in the post-selection
inference literature (e.g. Leeb & Pötscher (2005) and Berk et al. (2013)),
relying on selection consistency to enable inference on $\beta$ can lead to
poor finite-sample properties, such as inflated Type I errors. This phenomenon
is exacerbated when the true $\bm{\pi}$ is close to $\mathbf{0}$ so that
selection consistency is not guaranteed. The next section highlights some
progress in this area by constructing uniformly valid confidence intervals of
$\beta$.

####  2.3.3 Uniformly Valid Inference of $\beta$

We discuss two procedures that construct uniformly valid confidence intervals
of $\beta$. Kang et al. (2022) proposed to take a union of several confidence
intervals of $\beta$ constructed from subsets of instruments that pass the $J$
test (Hansen, 1982). Specifically, suppose the investigator believes that at
least $v\geq 1$ instruments are valid (i.e., $|\mathcal{V}|\geq v$) and wants
to construct a $1-\alpha$ confidence interval of $\beta$. The union confidence
interval, denoted as ${\rm CI}_{\rm union}$, takes a union of $1-\alpha_{t}$
confidence intervals of $\beta$ that use $v$ instruments and the $J$ test does
not reject the null with the $v$ instruments at level $\alpha_{s}$:

${\rm CI}_{\rm
union}^{[v]}=\cup_{\mathcal{V}^{\prime},|\mathcal{V}^{\prime}|=v}\left\\{{\rm
CI}_{1-\alpha_{t}}(\mathcal{V}^{\prime})\mid J(\mathcal{V}^{\prime})\leq
q_{1-\alpha_{s}}\right\\},\quad{}\alpha=\alpha_{s}+\alpha_{t},\quad{}|\mathcal{V}|\geq
v\geq 1.$

Here, ${\rm CI}_{1-\alpha_{t}}(\mathcal{V}^{\prime})$ is the $1-\alpha_{t}$
confidence interval of $\beta$ using $\mathcal{V}^{\prime}$ as valid
instruments, $J(\mathcal{V}^{\prime})$ is the $J$ test using
$\mathcal{V}^{\prime}$ as valid instruments, and $q_{1-\alpha_{s}}$ is the
$1-\alpha_{s}$ quantile of the $J$ test under its null hypothesis. The
confidence interval ${\rm CI}_{1-\alpha_{t}}$ can be any confidence interval
of $\beta$ that has the desired $1-\alpha_{t}$ coverage if valid instruments
are used. For example, ${\rm CI}_{1-\alpha_{t}}$ can be the Wald confidence
interval from the two-stage least squares estimator in Equation 7, the
Anderson-Rubin confidence interval (Anderson & Rubin, 1949), or the confidence
interval based on inverting the conditional likelihood ratio test (Moreira,
2003). Also, the terms $\alpha_{s}$ and $\alpha_{t}$ satisfy the constraint
$\alpha_{s}+\alpha_{t}=\alpha$. For instance, choosing $\alpha_{s}=0.01$ and
$\alpha_{t}=0.04$ would lead to a $95$% confidence interval for ${\rm CI}_{\rm
union}^{[v]}$. A main advantage of the union confidence interval is that it
does not rely on selection consistency and is guaranteed to yield a $1-\alpha$
confidence interval of $\beta$. However, the procedure requires an exponential
number of computations and is generally infeasible for a moderate to a large
number of instruments.

Guo (2023) proposed another approach to construct a uniformly valid confidence
interval of $\beta$. To illustrate the main idea, we focus on the case where
the majority rule holds; for the case under the plurality rule, see Guo
(2023). Guo (2023) proposed the searching confidence interval of $\beta$ based
on the relationship between $\Gamma$ and $\gamma$ in Equation 4:

${\rm CI}_{\rm
search}=\left\\{\beta\in\mathbb{R}:L(\beta)>\frac{p}{2}\right\\},\quad{}L(\beta)=\sum_{j=1}^{p}I\left[|\hat{\Gamma}_{j}-\beta\hat{\gamma}_{j}|\leq
z_{1-\frac{\alpha}{2p}}\cdot\hat{\rm
se}(\hat{\Gamma}_{j}-\beta\hat{\gamma}_{j})\right].$ (13)

The term $\hat{\rm se}(\hat{\Gamma}_{j}-\beta\hat{\gamma}_{j})$ is the
estimated standard error of $\hat{\Gamma}_{j}-\beta\hat{\gamma}_{j}$ using the
delta method, and $I[\cdot]$ is the indicator function. The term $L(\beta)$
measures the number of valid instruments for a particular value of $\beta$,
and the interval ${\rm CI}_{\rm search}$ collects all $\beta$ values that lead
to more than $50\%$ of instruments being valid. Also, to reduce the length of
${\rm CI}_{\rm search}$, Guo (2023) proposed a sampling version of ${\rm
CI}_{\rm search}$, denoted as ${\rm CI}_{\rm ss}$. The sampling confidence
interval starts by re-sampling $(\hat{\bm{\Gamma}},\hat{\bm{\gamma}})$ $M$
number of times based on Equation 8:

$\begin{pmatrix}\hat{\bm{\Gamma}}^{[m]}\\\
\hat{\bm{\gamma}}^{[m]}\end{pmatrix}\overset{iid}{\sim}N\left[\begin{pmatrix}\hat{\bm{\Gamma}}\\\
\hat{\bm{\gamma}}\end{pmatrix},\frac{1}{n}\begin{pmatrix}\hat{\bm{\Omega}}_{\Gamma}&\hat{\bm{\Omega}}_{\Gamma\gamma}\\\
\hat{\bm{\Omega}}_{\Gamma\gamma}^{\intercal}&\hat{\bm{\Omega}}_{\gamma}\end{pmatrix}\right],\quad\quad
1\leq m\leq M,$

where the terms $\hat{\bm{\Omega}}_{\Gamma},\hat{\bm{\Omega}}_{\Gamma\gamma}$,
and $\hat{\bm{\Omega}}_{\gamma}$ denote consistent estimators of
${\bm{\Omega}}_{\Gamma},{\bm{\Omega}}_{\Gamma\gamma}$, and
${\bm{\Omega}}_{\gamma}$, respectively. We then recompute the searching
confidence interval in Equation 13 from the resampled
$(\hat{\bm{\Gamma}}^{[m]},\hat{\bm{\gamma}}^{[m]})$:

${\rm
CI}^{[m]}=\left\\{\beta\in\mathbb{R}:L^{[m]}(\beta)>\frac{p}{2}\right\\},\;{L}^{[m]}(\beta)=\sum_{j=1}^{p}I\left[|\hat{\Gamma}_{j}^{[m]}-\beta\hat{\gamma}_{j}^{[m]}|\leq\lambda\cdot
z_{1-\frac{\alpha}{2p}}\cdot\hat{\rm
se}(\hat{\Gamma}_{j}-\beta\hat{\gamma}_{j})\right].$ (14)

The main difference between Equation 13 and Equation 14 is the shrinkage
parameter $\lambda$ in Equation 14 with $\lambda=c_{n}\left(M^{-1}\log
n\right)^{\frac{1}{2p}}$ where $c_{n}$ is a data-dependent parameter; see Guo
(2023) for details. The sampling confidence interval aggregates non-empty
intervals of ${\rm CI}^{[m]}$ by taking the lower and the upper limit of the
interval ${\rm CI}^{[m]}$, denoted as $l^{[m]}$ and $u^{[m]}$, respectively:

${\rm CI}_{\rm
ss}=\left(\min_{m\in\mathcal{M}}l^{[m]},\max_{m\in\mathcal{M}}u^{[m]}\right),\quad\mathcal{M}=\left\\{1\leq
m\leq M:{\rm CI}_{\rm search}^{[m]}\neq\emptyset\right\\}.$

Guo (2023) showed that both ${\rm CI}_{\rm search}$ and ${\rm CI}_{\rm ss}$
achieve the desired coverage level even in the presence of instrumental
variable selection error. Also, the length of both intervals are of the order
$1/\sqrt{n}.$ In finite samples, we observe that ${\rm CI}_{\rm search}$ and
${\rm CI}_{\rm ss}$ are longer than the confidence intervals from TSHT and
CIM, which is to be expected since ${\rm CI}_{\rm search}$ and ${\rm CI}_{\rm
ss}$ guarantee uniform coverage of $\beta$.

###  2.4 Connection to Two-Sample Summary Data Design in Mendelian
Randomization

Linear models often serve as the data generating model for a popular study
design in Mendelian randomization (MR) called the two-sample, summary data
design (Pierce & Burgess, 2013, Burgess et al., 2013). In this section, we
briefly discuss the connection between the assumptions underlying two-sample,
summary data designs in Mendelian randomization and the assumptions discussed
in the prior sections. For a full review of assumptions underlying MR, see
Bowden et al. (2017), Slob & Burgess (2020) and Sanderson et al. (2022).

Briefly, two-sample, summary data designs assume that the data is generated
from two independent samples and only summary statistics, usually estimates of
$\hat{\bm{\Gamma}}$ and $\hat{\bm{\gamma}}$ along with their corresponding
standard errors, are available. The summary statistics are assumed to follow a
multivariate normal distribution with diagonal covariance matrices. These
assumptions are formalized below:

$\bm{\Gamma}=\beta\bm{\gamma}+\bm{\pi},\quad{}\quad{}\hat{\Gamma}_{j}\overset{\text{ind}}{\sim}N(\Gamma_{j},\sigma_{j}^{2}),\
\hat{\gamma}_{j}\overset{\text{ind}}{\sim}N(\gamma_{j},\omega_{j}^{2}),\
\hat{\bm{\Gamma}}\perp\hat{\bm{\gamma}}.$ (15)

See Bowden et al. (2017), Zhao et al. (2020) and Ye et al. (2021) for
additional details. In terms of identification, two-sample summary data
designs assume the same relationship $\bm{\Gamma}=\beta\bm{\gamma}+\bm{\pi}$
in Equation 4. Also, many works in two-sample summary data Mendelian
randomization make similar assumptions about $\bm{\pi}$ as those in Section
2.2. For example, Bowden et al. (2016) proposed the weighted median estimator
of $\beta$, which is consistent whenever the majority rule in Equation 5
holds. Hartwig et al. (2017) proposed the Zero Modal Pleiotropy Assumption
(ZEMPA), which is the MR version of the plurality rule in Equation 6, and
showed that their proposed modal estimator of $\beta$ is consistent. An
improvement of the modal estimator was proposed by Burgess et al. (2018). Yao
et al. (2024) proposed MR-SPI as a modified version of the methods proposed in
Guo et al. (2018) and Guo (2023). Bowden et al. (2015) proposed the Instrument
Strength Independent of Direct Effect (InSIDE) assumption, which is the MR
version of the orthogonality condition in Equation 11. We remark that the
InSIDE assumption is similar to balanced horizontal pleiotropy (Bowden et al.,
2017, Hemani et al., 2018, Zhao et al., 2020), which states that
$\pi_{j}\overset{\text{ind}}{\sim}N(0,\tau^{2})$ and $\pi_{j}$’s are
independent of $\hat{\bm{\Gamma}}$ and $\hat{\bm{\gamma}}$.

In terms of inference, two-sample summary data designs in Equation 15 place
stronger assumptions on $\hat{\bm{\Gamma}}$ and $\hat{\bm{\gamma}}$ than prior
sections. Specifically, Equation 15 assumes that $\hat{\bm{\Gamma}}$ and
$\hat{\bm{\gamma}}$ are exactly normal and entries of the vector
$(\hat{\bm{\Gamma}},\hat{\bm{\gamma}})$ are independent of each other. This is
stronger than Equation 8 where there can be dependence among
$(\hat{\bm{\Gamma}},\hat{\bm{\gamma}})$, and
$(\hat{\bm{\Gamma}},\hat{\bm{\gamma}})$ only have to be asymptotically normal.

##  3 NONLINEAR MODELS

Recent works have utilized non-linearities in the exposure model to identify
and estimate $\beta$ in the presence of invalid instruments. The main idea is
to leverage non-linear trends in the exposure model to create new instruments,
which are then used to identify and estimate the causal effect of the
exposure. Compared to linear methods, nonlinear treatment methods enable
causal identification even when the plurality or majority assumptions are
violated. But, as mentioned in Section 1, investigators should verify that the
exposure model is indeed non-linear to ensure that these methods yield valid
results. In this section, we review two recent works in this area.

First, Guo et al. (2022) considered the following modifications of Equation 1:

$\displaystyle D$ $\displaystyle=f(\mathbf{Z})+\delta,$
$\displaystyle\mathbb{E}(\delta\mid\mathbf{Z})=0,{\rm
Var}[\mathcal{P}_{\mathbf{Z}}^{\perp}f(\mathbf{Z})]>0,$
$\displaystyle\mathbb{E}[f(\mathbf{Z})]=0,\mathbb{E}(\mathbf{Z})=\mathbf{0}.$

The term $\mathbf{Z}$ is a $p$ dimensional random variable and the term
$\mathcal{P}_{\mathbf{Z}}^{\perp}f(\mathbf{Z})=f(\mathbf{Z})-\mathbf{Z}\tilde{\bm{\gamma}}$,
$\tilde{\bm{\gamma}}={\rm
argmin}_{\bm{\gamma}}\mathbb{E}[f(\mathbf{Z})-\mathbf{Z}{\bm{\gamma}}]^{2}$,
is the residual from the best linear approximation of $f(\mathbf{Z})$. The
positive variance assumption ${\rm
Var}[\mathcal{P}_{\mathbf{Z}}^{\perp}f(\mathbf{Z})]>0$ ensures that
$f(\mathbf{Z})$ is a non-linear function of $\mathbf{Z}$. More generally, the
positive variance assumption states that the exposure $D$ can be explained
through linear and non-linear functions of $\mathbf{Z}$. In contrast, the
outcome’s relationship with the instruments in Equation 2 is linear.
Critically, this discrepancy allows opportunities to create a “non-linear”
instrument and identify $\beta$ with invalid instruments. To put it
differently, the nonlinearity assumption on $f$ guarantees that the
association between the treatment and the instrument, which is also
characterized by the function $f(\cdot)$, is more complicated than the
functional form of the violation, which is linear. We can formalize this
observation by noticing that at the true value of $\beta$, the following
equation holds:

$\mathbb{E}\\{[\mathcal{P}_{\mathbf{Z}}^{\perp}{f}(\mathbf{Z})]\cdot(\mathbf{Y}-\mathbf{D}\beta)\\}=\mathbb{E}\\{[\mathcal{P}_{\mathbf{Z}}^{\perp}{f}(\mathbf{Z})]\cdot(\mathbf{Z}^{\intercal}\bm{\pi})\\}=0.$

The first equality follows from $\mathbb{E}[\epsilon\mid\mathbf{Z}]=0$ in
Equation 2, and the second equality follows from the property of the
orthogonal projection $\mathcal{P}_{\mathbf{Z}}^{\perp}$. Guo et al. (2022)
also discusses a generalization of the above observation to the case where the
instruments’ direct effect on the outcome is non-linear.

For estimation, Guo et al. (2022) proposed the following two-step method with
sample splitting. Suppose we randomly split the data into two parts where the
data in the first part is indexed as $i=1,\ldots,n_{1}$ and the data in the
second part is indexed as $i=n_{1}+1,\ldots,n$. In the first step,
$f(\mathbf{Z})$ is estimated using a nonparametric estimator or a machine
learning algorithm (e.g., random forests) based on the data from the second
part. In the second step, the fitted function $f$ is evaluated in the data
from the first part and Guo et al. (2022) showed that this fit can be
represented as

$\begin{bmatrix}\hat{f}(\mathbf{Z}_{1})\\\ \vdots\\\
\hat{f}(\mathbf{Z}_{n_{1}})\end{bmatrix}=\mathbf{Q}\tilde{\mathbf{D}},\quad{}\tilde{\mathbf{D}}=\begin{pmatrix}D_{1}\\\
\vdots\\\ D_{n_{1}}\end{pmatrix}\in\mathbb{R}^{n_{1}\times
1},\quad{}\mathbf{Q}\in\mathbb{R}^{n_{1}\times n_{1}}.$

The matrix $\mathbf{Q}$ can be thought of as a matrix representation of the
nonparametric estimator used in the second part of the data. For example, if
$f(\mathbf{Z})$ is estimated via split random forests, each row of the matrix
$\mathbf{Q}$ represents a $n_{1}$-dimensional aggregation weight (Lin & Jeon,
2006, Meinshausen, 2006, Wager & Athey, 2018) . Let

$\tilde{\mathbf{Y}}=\mathbf{Q}\begin{pmatrix}Y_{1}\\\ \vdots\\\
Y_{n_{1}}\end{pmatrix}\in\mathbb{R}^{n_{1}\times
1},\quad{}\tilde{\mathbf{Z}}=\mathbf{Q}\begin{pmatrix}\mathbf{Z}_{1}\\\
\vdots\\\ \mathbf{Z}_{n_{1}}\end{pmatrix}\in\mathbb{R}^{n_{1}\times
p},\quad{}\mathbf{M}=\mathbf{Q}^{\intercal}\mathbf{P}_{\tilde{\mathbf{Z}}}^{\perp}\mathbf{Q}\in\mathbb{R}^{n_{1}\times
n_{1}}.$

Then, we introduce the following bias-corrected estimator of $\beta$,

$\hat{\beta}_{\rm
tsci}=\tilde{\beta}-\frac{\sum_{i=1}^{n_{1}}M_{i,i}\hat{\delta}_{i}\hat{\epsilon}_{i}}{\tilde{\mathbf{D}}^{\intercal}\mathbf{M}\tilde{\mathbf{D}}},\quad{}\tilde{\beta}=\frac{{\mathbf{Y}}^{\intercal}\mathbf{M}\mathbf{D}}{{\mathbf{D}}^{\intercal}\mathbf{M}{\mathbf{D}}},$
(16)

where $\hat{\delta}_{i}=D_{i}-\hat{f}(\mathbf{Z}_{i})$ and
$\hat{\epsilon}_{i}$ is the $i$th element of the vector
$\mathbf{P}_{\mathbf{{Z}}}^{\perp}({\mathbf{Y}}-{\mathbf{D}}\tilde{\beta})$.
Because identification of $\beta$ relies on the nonlinear curvature
$\mathcal{P}_{\mathbf{Z}}^{\perp}{f}(\mathbf{Z})$ and the estimation of
$\beta$ uses a two-step procedure, the estimator $\hat{\beta}_{\rm tsci}$ is
referred to as “Two-Stage Curvature Identification” (TSCI) estimator.

Second, Sun et al. (2023) proposed to use higher-order interactions of $p$
instruments in the exposure model to identify $\beta$. Similar to Guo et al.
(2022), Sun et al. (2023) generated “new” instruments from the $p$ instruments
and the new instruments have non-linear effects on the exposure. However, Guo
et al. (2022) used non-parametric estimators to create these new instruments
whereas Sun et al. (2023) used higher-order interactions to create them. A bit
more formally, suppose all $p$ instruments are mutually independent and there
is at least $v$ valid instruments (i.e., $|\mathcal{V}|\geq v$ and $v\geq 1$);
see Sun et al. (2023) when the instruments are dependent. Then, using the G
estimation framework (Robins et al., 1992, Robins, 1994), Sun et al. (2023)
showed that there is a function
$\mathbf{h}^{[v]}(\mathbf{Z})\in\mathbb{R}^{d}$ with
$d=\sum_{j=0}^{v-1}{p\choose j}$ such that $\beta$ is the unique solution to

$\mathbb{E}[\mathbf{h}^{[v]}(\mathbf{Z})(Y-\beta
D)]=\mathbf{0},\quad{}|\mathcal{V}|\geq v,$ (17)

if $\mathbf{h}^{[v]}(\mathbf{Z})$ is associated with the exposure $D$. The
function $\mathbf{h}^{[v]}(\mathbf{Z})$ represents all higher-order
interactions of $p$ instruments of order greater than or equal to $v$.
Specifically, for each $j=0,\ldots,v-1$, we create all possible subsets of $p$
instruments of size $p-j$ and denote this set as $\mathcal{C}_{j}$:

$\displaystyle\mathcal{C}_{0}$ $\displaystyle=\\{(1,\ldots,p)\\},$
$\displaystyle\mathcal{C}_{1}$
$\displaystyle=\\{(1,\ldots,p-1),(1,\ldots,p-2,p),...,(2,\ldots,p)\\},$
$\displaystyle\mathcal{C}_{j}$
$\displaystyle=\\{C\subseteq\\{1,\ldots,p\\}\mid|C|=p-j\\},\quad{}j=0,\ldots,v-1.$

Then, for each element $C\in\mathcal{C}_{j}$, we create the interaction
instrument $\prod_{k\in C}[Z_{j}-\mathbb{E}(Z_{j})]$. For example, for
$\mathcal{C}_{0}$ and $\mathcal{C}_{1}$, we have the following interaction
instruments:

$\displaystyle\mathcal{C}_{0}$
$\displaystyle\Rightarrow\prod_{k=1}^{p}[Z_{k}-\mathbb{E}(Z_{k})],$
$\displaystyle\mathcal{C}_{1}$ $\displaystyle\Rightarrow\prod_{k\neq
p}[Z_{k}-\mathbb{E}(Z_{k})],\prod_{k\neq
p-1}[Z_{k}-\mathbb{E}(Z_{k})],\ldots,\prod_{k\neq
1}[Z_{k}-\mathbb{E}(Z_{k})].$

Stacking all the interaction instruments generated by every
$\mathcal{C}_{j},j=0,\ldots,v-1$ into a vector defines the function
$\mathbf{h}^{[v]}(\mathbf{Z})$. Or, in other words,
$\mathbf{h}^{[v]}(\mathbf{Z})$ creates $d$ interaction instruments.

As an illustrative example, suppose we have $p=2$ instruments
$\mathbf{Z}=(Z,Z^{\prime})$, $Z$ and $Z^{\prime}$ are mutually independent,
and at least one of them is valid (i.e. $v=1$). Then,
$d=\sum_{j=0}^{v-1}{p\choose j}=1$, $\mathcal{C}_{0}=\\{\\{1,2\\}\\}$, and
$h^{[1]}(\mathbf{Z})=[Z-\mathbb{E}(Z)][Z^{\prime}-\mathbb{E}(Z^{\prime})]$. As
long as $[Z-\mathbb{E}(Z)][Z^{\prime}-\mathbb{E}(Z^{\prime})]$ is associated
with $D$, $\beta$ is the unique solution of Equation 17 because

$\mathbb{E}[(Z-\mathbb{E}[Z])(Z^{\prime}-\mathbb{E}[Z^{\prime}])(Y-\beta
D)]=\mathbb{E}[(Z-\mathbb{E}[Z])(Z^{\prime}-\mathbb{E}[Z^{\prime}])Z_{\mathcal{V}^{C}}\pi_{\mathcal{V}^{C}}]=0.$

From the law of total expectations, the last equality holds for any
$\mathcal{V}$ so long as $|\mathcal{V}|\geq v=1$ and the two instruments are
independent of each other. Also, the last equality continues to hold for any
$\mathcal{V}$ with $|\mathcal{V}|\geq 1$ even if the term
$Z_{\mathcal{V}^{C}}\pi_{\mathcal{V}^{c}}$ in Equation 2 is non-linear, for
instance if $Z_{\mathcal{V}^{C}}\pi_{\mathcal{V}^{c}}$ is replaced by an
unknown function $\pi(Z_{\mathcal{V}^{c}})$. Or more loosely stated, in
addition to $Z$ and $Z^{\prime}$, $h^{[1]}(\mathbf{Z})$ serve as the “new”,
interaction instrument and so long as one of these instruments are valid, we
can still identify the parameters in Equation 2.

For estimation, we can replace the terms in Equation 17 with their empirical
counterparts. Or, we can run two-stage least squares with the interaction
instruments $\mathbf{h}^{[v]}(\mathbf{Z})$ and the term $\mathbb{E}(Z_{j})$ in
$\mathbf{h}^{[v]}(\mathbf{Z})$ is replaced by the $j$th column mean of the
instrument matrix $\mathbf{Z}\in\mathbf{R}^{n\times p}$. In the example above
where we have two instruments and $v=1$, the estimator simplifies to

$\hat{\beta}_{\rm
g}^{[1]}=\frac{\sum_{i=1}^{n}(Z_{i}-\bar{Z})(Z_{i}^{\prime}-\bar{Z}^{\prime})Y_{i}}{\sum_{i=1}^{n}(Z_{i}-\bar{Z})(Z_{i}^{\prime}-\bar{Z}^{\prime})D_{i}}$

The terms $\bar{Z}$ and $\bar{Z}^{\prime}$ represent the mean of $Z$ and $Z$’,
respectively. The statistical properties of $\hat{\beta}_{\rm g}^{[v]}$ can be
established by using the M-estimation framework. We remarked that Sun et al.
(2023) also proposed a new multiply robust identification framework, and a
semiparametric efficient estimator of $\beta$.

##  4 HETEROSKEDASTIC MODELS

Another approach to study the causal effect of the exposure with invalid
instruments is by leveraging heteroskedasticity of the observed data (Lewbel,
2012, Tchetgen Tchetgen et al., 2021, Sun et al., 2022, Ye et al., 2024).
Specifically, consider the following variations of Equations 1 and 2:

$\displaystyle\mathbb{E}(Y\mid D,\mathbf{Z},U)=\beta
D+\mathbf{Z}\bm{\pi}+\xi_{y}(U),$ (18)
$\displaystyle\mathbb{E}(D\mid\mathbf{Z},U)=\mathbf{Z}\bm{\gamma}+\xi_{d}(U),\quad{}\mathbf{Z}\perp
U$ (19)

where $\xi_{y},\xi_{d}$ are unspecified functions. The variable $U$ represents
an unmeasured variable that affects both the outcome $Y$ and the exposure $D$.
Tchetgen Tchetgen et al. (2021) showed that under Equations 18 and 19, $\beta$
can be identified as the unique solution to the following estimating equation

$\mathbb{E}\\{[\mathbf{Z}-\mathbb{E}(\mathbf{Z})][D-\mathbb{E}(D|\mathbf{Z})](Y-\beta
D)\\}=\mathbf{0}$ (20)

as long as $D$ is heteroscedastic, i.e., ${\rm Var}(D|\mathbf{Z})$ varies as a
function of $\mathbf{Z}$. Specifically, under Equations 18 and 19, the left-
hand side of Equation 20 simplifies to

$\displaystyle\mathbb{E}\\{[\mathbf{Z}-\mathbb{E}(\mathbf{Z})][D-\mathbb{E}(D|\mathbf{Z})][\mathbb{E}(Y\mid
D,\mathbf{Z},U)-\beta D]\\}$ $\displaystyle=$
$\displaystyle\mathbb{E}\\{[\mathbf{Z}-\mathbb{E}(\mathbf{Z})][D-\mathbb{E}(D|\mathbf{Z})][\mathbf{Z}\bm{\pi}+\xi_{y}(U)]\\}$
$\displaystyle=$
$\displaystyle\mathbb{E}\\{[\mathbf{Z}-\mathbb{E}(\mathbf{Z})][D-\mathbb{E}(D|\mathbf{Z})]\xi_{y}(U)\\}$
$\displaystyle=$
$\displaystyle\mathbb{E}\\{[\mathbf{Z}-\mathbb{E}(\mathbf{Z})][\xi_{d}(U)-\mathbb{E}[\xi_{d}(U)]]\xi_{y}(U)\\}$
$\displaystyle=$
$\displaystyle\mathbb{E}[\mathbf{Z}-\mathbb{E}(\mathbf{Z})]\mathbb{E}\\{[\xi_{d}(U)-\mathbb{E}[\xi_{d}(U)]]\xi_{y}(U)\\}$
$\displaystyle=$ $\displaystyle\mathbf{0}$

We remark that the above argument holds more broadly even if we replace
$\mathbf{Z}\bm{\pi}$ in Equation 18 with any function $\mathbf{Z}$ or if we
replace $\mathbf{Z}\bm{\gamma}$ in Equation 19 with any function of
$\mathbf{Z}$. Tchetgen Tchetgen et al. (2021) provides more general conditions
under which Equation 20 holds and refers to this framework as “G-Estimation
under No Interaction with Unmeasured Selection” (GENIUS).

Intuitively, Equation 20 constructs $p$ new interaction instruments of the
form $[\mathbf{Z}-\mathbb{E}(\mathbf{Z})][D-\mathbb{E}(D\mid\mathbf{Z})]$,
which is the product of the original candidate set of instruments $\mathbf{Z}$
and the residual $D-\mathbb{E}(D\mid\mathbf{Z})$. From the independence of $U$
and $\mathbf{Z}$, the residual $D-\mathbb{E}(D\mid\mathbf{Z})$ is a proxy for
$\xi_{d}(U)$. Then, the interaction instruments are valid instruments in that
since there are no interactions between $U$ and $\mathbf{Z}$ in Equation 18,
the interaction instruments satisfy the no direct effect assumption in
condition (A2). Also, the interaction instruments are correlated with the
exposure where for any $j=1,\ldots,p$, we have

$\displaystyle{\rm
Cov}\\{[Z_{j}-\mathbb{E}(Z_{j})][D-\mathbb{E}(D\mid\mathbf{Z})],D\\}$
$\displaystyle=$
$\displaystyle\mathbb{E}\\{[Z_{j}-\mathbb{E}(Z_{j})][D-\mathbb{E}(D\mid\mathbf{Z})]D\\}$
$\displaystyle=$ $\displaystyle\mathbb{E}\\{[Z_{j}-\mathbb{E}(Z_{j})]{\rm
Var}(D\mid\mathbf{Z})\\}+\mathbb{E}\\{[Z_{j}-\mathbb{E}(Z_{j})][D-\mathbb{E}(D\mid\mathbf{Z})]\mathbb{E}(D\mid\mathbf{Z})\\}$
$\displaystyle=$ $\displaystyle\mathbb{E}\\{[Z_{j}-\mathbb{E}(Z_{j})]{\rm
Var}(D\mid\mathbf{Z})\\}\neq 0.$

The first equality uses the definition of covariance and the second equality
uses $D=D-\mathbb{E}[D\mid\mathbf{Z}]+\mathbb{E}[D\mid\mathbf{Z}]$ along with
the definition of conditional variance. The third equality uses the law of
total expectation that conditions on $\mathbf{Z}$ and the final inequality is
due to heteroskedasticity of $D$. Combined, the interaction instruments
$[\mathbf{Z}-\mathbb{E}(\mathbf{Z})][D-\mathbb{E}(D\mid\mathbf{Z})]$ are “new”
instruments to identify $\beta$ in Equation 18. Note that this approach is
similar to Section 3 where higher-order interactions of $\mathbf{Z}$ are used
to create new instruments and identify $\beta$. Also, because the above
identification strategy relies on heteroskedasticity of the exposure $D$ to
create the new interaction instruments, it’s possible to identify $\beta$ in
Equation 18 even if all instruments have direct effects on the outcome, i.e.,
if $\pi_{j}\neq 0$ for all $j$.

For estimation, one simple approach is to solve the sample equivalent version
of Equation 20:

$\hat{\beta}_{\rm genius}=\underset{\beta}{\rm
argmin}\sum_{i=1}^{n}[(\mathbf{Z}_{i}-\bar{\mathbf{Z}})(D_{i}-\mathbf{Z}_{i}\hat{\bm{\gamma}})(Y_{i}-\beta
D_{i})]^{\intercal}[(\mathbf{Z}_{i}-\bar{\mathbf{Z}})(D_{i}-\mathbf{Z}_{i}\hat{\bm{\gamma}})(Y_{i}-\beta
D_{i})].$

The estimator $\hat{\beta}_{\rm genius}$ replaces
$\mathbb{E}[D\mid\mathbf{Z}]$ with an estimate from a linear regression model
where we regress $D$ on $\mathbf{Z}$. Under some moment assumptions, Tchetgen
Tchetgen et al. (2021) show that $\hat{\beta}_{\rm genius}$ is consistent and
asymptotically normal for $\beta$. Ye et al. (2024) presents an extension of
this estimator that is robust to many weak invalid instruments and Sun et al.
(2022) proposes multiply robust estimator of $\beta$ where they use machine
learning estimators for estimating relevant nuisance functions (e.g.,
$E[D\mid\mathbf{Z}]$).

Finally, we discuss another idea based based on heteroskedastcity by Liu et
al. (2023). Following previous notations, the authors considered the following
variation of Equation 2:

$Y=\beta_{0}+\beta D+\mathbf{Z}\bm{\pi}+\alpha
D\exp(\eta_{0}+\mathbf{Z}\bm{\eta})+\xi,\quad{}\xi\mid D,\mathbf{Z}\sim
N\left[0,\exp(\eta_{0}+\mathbf{Z}\bm{\eta})\right],\bm{\eta}\neq\mathbf{0}.$
(21)

Here, the parameter $\beta$ represents the average treatment effect on the
treated and is the target parameter of interest. Equation 21 is the observable
implication of the following identification assumptions: (a) no additive
interaction between $D$ and $\mathbf{Z}$ (i.e.,
$\mathbb{E}[Y(d,\mathbf{z})-Y(0,\mathbf{z})\mid
D=d,\mathbf{Z}=\mathbf{z}]=d\beta$ for all $\mathbf{z}$), (b) homogeneous
confounding of $D$; and (c) the outcome $Y$ among $D=0$ following a normal
distribution with variance $\exp(\eta_{0}+\mathbf{Z}\bm{\eta})$ and
$\bm{\eta}\neq\mathbf{0}$. Condition (a) is weaker than the constant effect
assumption where $Y(d,\mathbf{z})-Y(0,\mathbf{z})=d\beta$ and is satisfied if
$\mathbf{Z}$ does not modify the average treatment effect on the treated.
Condition (b) is defined on the odds ratio scale and encodes the assumption
that confounding on $D$ does not depend on $\mathbf{Z}$; see Liu et al. (2023)
for additional discussions. Conditions (b) and (c) give rise to the term
$\alpha D\exp(\eta_{0}+\mathbf{Z}\bm{\eta})+\xi$ in Equation 21. The
distributional assumption can be relaxed to a mixture of normal distributions
or to a distribution of $Y$ given $D=0$ that is heteroskedsatic in
$\mathbf{Z}$. Finally, estimation of $\beta$ is based on the likelihood
principle where we maximize the log likelihood of $\mathbb{P}(Y_{i}\mid
D_{i},\mathbf{Z}_{i})$ in Equation 21; we denote this estimator
$\hat{\beta}_{\rm misteri}$ where MiSTERI stands for “Mixed-Scale Treatment
Effect Robust Identification.” Another estimation approach based on the method
of moments is discussed in Liu et al. (2023).

##  5 ILLUSTRATION WITH REAL DATA

###  5.1 Background and Setup

We demonstrate the methods introduced above by re-analyzing the effect of body
mass index (BMI) on systolic blood pressure (SBP) from the UK Biobank; see
Section 5 of Sun et al. (2023) for details. Briefly, the UK Biobank is a
large-scale prospective cohort study that recruited roughly 500,000
participants between 2006 and 2010 in the United Kingdom (Sudlow et al.,
2015). In the dataset, BMI was measured in units of kilograms per meter
squared and SBP was measured in units of millimeters of mercury. Following Sun
et al. (2023), we restrict our analysis to people of genetically verified
white British descent (Tyrrell et al., 2016) and who are not taking anti-
hypertensive medication based on self reporting. The sample size for the final
analysis is $n=292,757$. We use the top $p=10$ single nucleotide polymorphisms
(SNPs) ranked by their $p$-values, each of which were derived from testing the
effect of a SNP on BMI with simple linear regression. The 10 p-values reach
genome-wide significance level of $5\times 10^{-8}$ (Locke et al., 2015) and
have pairwise correlation coefficients that are less than 0.01. The 10 SNPs
are rs1558902, rs6567160, rs543874, rs13021737, rs10182181, rs2207139,
rs11030104, rs10938397, rs13107325, and rs3888190. The overall F-statistic for
the first-stage model (i.e., Equation 1 with 10 instruments) is 146.1, with a
p-value that is less than 10-8. To focus on problems caused by invalid
instruments, we chose the top 10 SNPs here to minimize effects from weak
instruments; see Sun et al. (2023) for further details on how these
instruments were chosen.

We compare the following methods for estimating $\beta$: $\hat{\beta}_{\rm
med}$, $\hat{\beta}_{\rm sisvive}$, $\hat{\beta}_{\rm kclass}$
$\hat{\beta}_{\rm adlasso}$, $\hat{\beta}_{\rm tsht}$, $\hat{\beta}_{\rm
cim}$, $\hat{\beta}_{\rm tsci}$, $\hat{\beta}_{\rm g}^{[v]}$,
$\hat{\beta}_{\rm genius}$, and $\hat{\beta}_{\rm misteri}$. For
$\hat{\beta}_{\rm g}^{[v]}$, we set the minimum number of valid instruments to
be $6$ and $8$ and they are denoted as $\hat{\beta}_{\rm g}^{[6]}$ and
$\hat{\beta}_{\rm g}^{[8]}$, respectively. We also compute he union confidence
interval ${\rm CI}_{\rm union}^{[v]}$ and the search and sampling confidence
interval ${\rm CI}_{\rm ss}$. For ${\rm CI}_{\rm union}^{[v]}$, we set the
minimum number of valid instruments to be $v=6$ and $v=8$ and use the
conditional likelihood ratio test (Moreira, 2003). Finally, we include two
baseline analyses of the causal effect. The first baseline analysis is the
ordinary least squares estimator of $\beta$ that fits a linear regression of
SBP on BMI. The second baseline analysis is the TSLS estimator of $\beta$ from
Section 2.3.1 that sets $\mathcal{V}$ to be all 10 SNPs; in other words, this
estimator assumes that all 10 SNPs are valid.

We use the following software to compute the estimates or confidence intervals
of $\beta$. To compute $\hat{\beta}_{\rm sisvive}$, we use the R package
sisVIVE (Kang, 2017). To compute $\hat{\beta}_{\rm kclass}$, we use the R
package ivmodel (Kang et al., 2021). To compute $\hat{\beta}_{\rm med}$ and
$\hat{\beta}_{\rm adlasso}$, we use the code provided in Windmeijer et al.
(2019). To compute $\hat{\beta}_{\rm tsht}$ and the search and sampling
confidence interval ${\rm CI}_{\rm ss}$, we use the R package RobustIV (Koo et
al., 2023). To compute the union confidence interval ${\rm CI}_{\rm
union}^{[v]}$, we use the code provided in Kang et al. (2022). To compute
$\hat{\beta}_{\rm cim}$, we use the code provided in Windmeijer et al. (2021).
To compute $\hat{\beta}_{\rm tsci}$, we use R package TSCI (Carl et al.,
2023). To compute $\hat{\beta}_{\rm g}^{[v]}$, we use the code in the R
package MRSquare. Finally, to compute $\hat{\beta}_{\rm genius}$ and
$\hat{\beta}_{\rm misteri}$, we use the code provided in Tchetgen Tchetgen et
al. (2021) and Liu et al. (2023), respectively.

###  5.2 Results

The point estimates and 95% confidence intervals for different methods are
summarized in Figure 1. Except for $\hat{\beta}_{\rm g}^{[6]}$, ${\rm CI}_{\rm
union}^{[6]}$, and ${\rm CI}_{\rm union}^{[8]}$, all methods, including the
TSLS estimator that assumes all instruments are valid and the OLS estimator
that does not use any instruments, suggest that there is a positive effect of
increasing BMI on SBP. The largest and smallest values of the causal effect
are from the union confidence interval that assumes at least 6 instruments are
valid; its upper confidence limit of the causal effect is $1.2710$ and its
lower confidence limit is $-0.799$. The G-estimator that assumes at least six
instruments are valid (i.e. $\hat{\beta}_{\rm g}^{[6]}$) also gives wide
confidence intervals, ranging from $-0.5740$ to $0.6860$. But after we
increase the number of valid IVs to 8, the confidence interval is
$(0.174,0.646)$ and no longer covers $0$. In general, the G-estimator and the
union confidence interval allow users to conduct sensitivity analysis by
varying the number of valid IVs, and can be used to reflect the uncertainty
about the validity of instruments.

Among methods that generate confidence intervals, their 95% confidence
intervals overlap with each other. In other words, after accounting for
sampling uncertainty, these methods, despite making different assumptions
about instrument invalidity, do not differ from each other with respect to
their conclusions about the causal effect. Excluding the OLS estimator, the
narrowest 95% confidence interval is generated from $\hat{\beta}_{\rm TSCI}$
and the widest confidence interval is generated from the union confidence
interval that assumes at least six instruments are valid.

Despite almost all methods suggesting that the effect of BMI on SBP is
positive, we do see that the methods roughly cluster into two types. The first
cluster of methods (i.e., SISVIVE, the adaptive Lasso, the confidence interval
method, MiSTERI, and the search and sampling confidence interval) roughly
estimates the causal effect to be around $0.68$ and they are close to the OLS
estimator that does not use any instruments. The second cluster of methods
(i.e., the median estimator, the k-class estimator, TSHT, TSCI, the G
estimators, GENIUS, and the union confidence intervals) roughly estimates the
causal effect to be around $0.40$ and they are close to the TSLS estimator
that assumes all 10 instruments are valid.

Among methods that select valid instruments, CIM selected rs543874 and
rs10182181 as invalid instruments. TSHT selected rs10182181 as an invalid
instrument. The adaptive Lasso selected rs543874, rs10182181, and rs13107325
as invalid instruments. Across all methods that can select valid instruments,
instrument rs10182181 was selected as an invalid instrument.

Figure 1: Point estimates (denoted as points) or 95% confidence intervals
(denoted as an interval) for the effect of body mass index (BMI) on systolic
blood pressure (SBP) from the UK Biobank. Sample size is $n=292,757$ and there
are $p=10$ genetic instruments. OLS refers to the ordinary least squares
estimator of $\beta$ without using instruments. TSLS refers to
$\hat{\beta}_{\rm tsls}$ that assumes all 10 instruments are valid. Median
refers to $\hat{\beta}_{\rm med}$, SISVIVE refers to $\hat{\beta}_{\rm
sisvive}$, k-class refers to $\hat{\beta}_{\rm kclass}$, AdLasso refers to
$\hat{\beta}_{\rm adlasso}$, TSHT refers to $\hat{\beta}_{\rm tsht}$, CIM
refers to $\hat{\beta}_{\rm cim}$, TSCI refers to $\hat{\beta}_{\rm tsci}$,
GENIUS refers to $\hat{\beta}_{\rm genius}$, and MiSTERI refers to
$\hat{\beta}_{\rm misteri}$. G(6) and G(8) refer to $\hat{\beta}_{\rm
g}^{[6]}$ and $\hat{\beta}_{\rm g}^{[8]}$, respectively. SS refers to ${\rm
CI}_{\rm ss}$. Also, ${\rm CI}_{\rm union}^{[6]}$ and ${\rm CI}_{\rm
union}^{[8]}$ refer to union confidence intervals which assume at least 6 and
8 instruments are valid, respectively.

##  6 DISCUSSION

This paper provides a review of identification and inference of the causal
effect of the exposure on the outcome when there are invalid instruments. We
start with the linear model framework where the parameter $\bm{\pi}$ in
Equation 2 encodes the violations of the IV assumptions. Broadly speaking,
works in this framework require either that the majority of instruments or a
plurality of instruments are valid to obtain identification and inference of
$\beta$. Subsequent works have leveraged non-linearities or
heteroskedastcities in the models to identify and infer $\beta$. In our data
analysis, we find that all methods yield similar conclusions about the effect
of body mass index on systolic blood pressure.

Despite significant progress in identifying and inferring the causal effect of
the exposure in the presence of invalid instruments, several challenges remain
and we highlight a couple of them. First, as illustrated throughout the paper,
there are different ways to define a valid (or an invalid) instrument based on
how the instrument deviates from the assumptions (A2) and (A3). Roughly
speaking, Section 2.1 defines an invalid instrument through a “linear
deviation” from the IV assumptions (A2) and (A3). In contrast, Sections 3 and
4 defines an invalid instrument through both linear and “non-linear”
deviations from the IV assumption. Because the latter sections allow for
broader types of invalid instruments, they typically require additional
conditions on the data-generating model for identification and inference, such
as a non-linear exposure model or a heteroskedastic exposure or outcome model.
Second, while methods for uniform inference exist, there is still room for
improvement, especially compared to the oracle TSLS estimator that knows which
instruments are valid a priori. Third, only a handful of works have explored
how to conduct valid inference when instruments are both invalid and are
weakly associated with the exposure; these instruments are common in Mendelian
randomization where the genetic variants only explain a fraction of the
variance in the exposure and most of them are suspected to be pleiotropic. Guo
et al. (2018) proposes a thresholding procedure in TSHT to select instruments
that are strongly correlated with the exposure. Lin et al. (2024) considers
many weak instruments under the plurality rule where instead of only using
instruments that are strongly correlated with the exposure, they use both
strong and weak instruments. Zhang et al. (2022) considered another approach
to select strong instruments that also prevents the mis-selection of valid
instruments. Ye et al. (2024) proposes an improved version of the GENIUS
estimator that allows for many weak invalid instruments discussed in Newey &
Windmeijer (2009). The method in Kang et al. (2022) allows for uniformly valid
inference in the presence of weak instruments defined by Staiger & Stock
(1997) and invalid instruments defined in Definition 1. Fourth, there is still
an open question about how to select the optimal set of instruments for bias
reduction and/or efficiency improvement when invalid instruments are present.
This problem is especially challenging when faced with many weak instruments
(Liu et al., 2023, Ye et al., 2024, Lin et al., 2024).

## DISCLOSURE STATEMENT

The authors are not aware of any affiliations, memberships, funding, or
financial holdings that might be perceived as affecting the objectivity of
this review.

## ACKNOWLEDGMENTS

Dr. Zijian Guo’s research was supported in part by NIH grants R01-GM140463 and
R01-LM013614. Dr. Zhonghua Liu’s research was supported in part by NIH grant
1R01AG086379-01. Dr. Dylan Small’s research was supported in part by NIH grant
5R01AG065276-02.

## References

  * Anatolyev (2013) Anatolyev S. 2013. Instrumental variables estimation and inference in the presence of many exogenous regressors. The Econometrics Journal 16(1):27–72
  * Anderson & Rubin (1949) Anderson TW, Rubin H. 1949. Estimation of the parameters of a single equation in a complete system of stochastic equations. The Annals of Mathematical Statistics 20:46–63
  * Andrews (1999) Andrews DWK. 1999. Consistent moment selection procedures for generalized method of moments estimation. Econometrica 67(3):543–563
  * Andrews et al. (2019) Andrews I, Stock JH, Sun L. 2019. Weak instruments in instrumental variables regression: Theory and practice. Annual Review of Economics 11(1):727–753
  * Angrist et al. (1996) Angrist JD, Imbens GW, Rubin DB. 1996. Identification of causal effects using instrumental variables. Journal of the American Statistical Association 91(434):444–455
  * Angrist & Pischke (2009) Angrist JD, Pischke JS. 2009. Mostly harmless econometrics: An empiricist’s companion. Princeton University Press
  * Armstrong & Kolesár (2021) Armstrong TB, Kolesár M. 2021. Sensitivity analysis using approximate moment condition models. Quantitative Economics 12(1):77–108
  * Ashley & Parmeter (2015) Ashley RA, Parmeter CF. 2015. Sensitivity analysis for inference in 2sls/gmm estimation with possibly flawed instruments. Empirical Economics 49(4):1153–1171
  * Baiocchi et al. (2014) Baiocchi M, Cheng J, Small DS. 2014. Instrumental variable methods for causal inference. Statistics in Medicine 33(13):2297–2340
  * Baiocchi et al. (2010) Baiocchi M, Small DS, Lorch S, Rosenbaum PR. 2010. Building a stronger instrument in an observational study of perinatal care for premature infants. Journal of the American Statistical Association 105(492):1285–1296
  * Bao et al. (2019) Bao Y, Clarke PS, Smart M, Kumari M. 2019. Assessing the robustness of sisvive in a mendelian randomization study to estimate the causal effect of body mass index on income using multiple snps from understanding society. Statistics in Medicine 38(9):1529–1542
  * Berk et al. (2013) Berk R, Brown L, Buja A, Zhang K, Zhao L. 2013. Valid post-selection inference. The Annals of Statistics :802–837
  * Berkowitz et al. (2008) Berkowitz D, Caner M, Fang Y. 2008. Are “nearly exogenous instruments” reliable? Economics Letters 101(1):20–23
  * Berkowitz et al. (2012) Berkowitz D, Caner M, Fang Y. 2012. The validity of instruments revisited. Journal of Econometrics 166(2):255–266
  * Bound et al. (1995) Bound J, Jaeger DA, Baker RM. 1995. Problems with instrumental variables estimation when the correlation between instruments and the endogenous variable is weak. Journal of the American Statistical Association 90:443–450
  * Bowden et al. (2015) Bowden J, Davey Smith G, Burgess S. 2015. Mendelian randomization with invalid instruments: effect estimation and bias detection through Egger regression. International Journal of Epidemiology 44(2):512–525
  * Bowden et al. (2016) Bowden J, Davey Smith G, Haycock PC, Burgess S. 2016. Consistent estimation in Mendelian randomization with some invalid instruments using a weighted median estimator. Genetic Epidemiology 40(4):304–314
  * Bowden et al. (2017) Bowden J, Del Greco M F, Minelli C, Davey Smith G, Sheehan N, Thompson J. 2017. A framework for the investigation of pleiotropy in two-sample summary data mendelian randomization. Statistics in Medicine 36(11):1783–1802
  * Burgess et al. (2013) Burgess S, Butterworth A, Thompson SG. 2013. Mendelian randomization analysis with multiple genetic variants using summarized data. Genetic Epidemiology 37(7):658–665
  * Burgess et al. (2018) Burgess S, Zuber V, Gkatzionis A, Foley CN. 2018. Modal-based estimation via heterogeneity-penalized weighting: model averaging for consistent and efficient estimation in mendelian randomization when a plurality of candidate instruments are valid. International journal of epidemiology 47(4):1242–1254
  * Carl et al. (2023) Carl D, Emmenegger C, Bühlmann P, Guo Z. 2023. Tsci: two stage curvature identification for causal inference with invalid instruments. arXiv preprint arXiv:2304.00513
  * Cheng & Liao (2015) Cheng X, Liao Z. 2015. Select the valid and relevant moments: An information-based lasso for gmm with many moments. Journal of Econometrics 186(2):443–464
  * Conley et al. (2012) Conley TG, Hansen CB, Rossi PE. 2012. Plausibly exogenous. Review of Economics and Statistics 94(1):260–272
  * Davey Smith & Ebrahim (2003) Davey Smith G, Ebrahim S. 2003. Mendelian randomization: can genetic epidemiology contribute to understanding environmental determinants of disease? International Journal of Epidemiology 32(1):1–22
  * Davey Smith & Ebrahim (2004) Davey Smith G, Ebrahim S. 2004. Mendelian randomization: prospects, potentials, and limitations. International Journal of Epidemiology 33(1):30–42
  * Davidson & MacKinnon (1993) Davidson R, MacKinnon JG. 1993. Estimation and inference in econometrics. New York: Oxford University Press
  * DiTraglia (2016) DiTraglia FJ. 2016. Using invalid instruments on purpose: Focused moment selection and averaging for gmm. Journal of Econometrics 195(2):187–208
  * Efrom et al. (2004) Efrom B, Hastie T, Johnstone I, Tibshirani R. 2004. Least angle regression. The Annals of Statistics 32(2):407–499
  * Fan & Wu (2022) Fan Q, Wu Y. 2022. Endogenous treatment effect estimation with a large and mixed set of instruments and control variables. Review of Economics and Statistics :1–45
  * Fogarty et al. (2021) Fogarty CB, Lee K, Kelz RR, Keele LJ. 2021. Biased encouragements and heterogeneous effects in an instrumental variable study of emergency general surgical outcomes. Journal of the American Statistical Association 116(536):1625–1636
  * Goldberger (1972) Goldberger AS. 1972. Structural equation methods in the social sciences. Econometrica: Journal of the Econometric Society :979–1001
  * Guggenberger (2012) Guggenberger P. 2012. On the asymptotic size distortion of tests when instruments locally violate the exogeneity assumption. Econometric Theory 28(2):387–421
  * Guo (2023) Guo Z. 2023. Causal inference with invalid instruments: post-selection problems and a solution using searching and sampling. Journal of the Royal Statistical Society Series B: Statistical Methodology 85(3):959–985
  * Guo et al. (2018) Guo Z, Kang H, Tony Cai T, Small DS. 2018. Confidence intervals for causal effects with invalid instruments by using two-stage hard thresholding with voting. Journal of the Royal Statistical Society Series B: Statistical Methodology 80(4):793–815
  * Guo et al. (2023) Guo Z, Li X, Han L, Cai T. 2023. Robust inference for federated meta-learning. arXiv preprint arXiv:2301.00718
  * Guo et al. (2022) Guo Z, Zheng M, Bühlmann P. 2022. Robustness against weak or invalid instruments: Exploring nonlinear treatment models with machine learning. arXiv preprint arXiv:2203.12808
  * Hahn & Hausman (2005) Hahn J, Hausman J. 2005. Estimation with valid and invalid instruments. Annales d’Économie et de Statistique (79/80):25–57
  * Han (2008) Han C. 2008. Detecting invalid instruments using l1-gmm. Economics Letters 101(3):285–287
  * Hansen (1982) Hansen LP. 1982. Large sample properties of generalized method of moments estimators. Econometrica :1029–1054
  * Hartwig et al. (2017) Hartwig FP, Davey Smith G, Bowden J. 2017. Robust inference in summary data Mendelian randomization via the zero modal pleiotropy assumption. International Journal of Epidemiology 46(6):1985–1998
  * Hemani et al. (2018) Hemani G, Bowden J, Davey Smith G. 2018. Evaluating the potential role of pleiotropy in mendelian randomization studies. Human Molecular Genetics 27(R2):R195–R208
  * Hernán & Robins (2006) Hernán MA, Robins JM. 2006. Instruments for causal inference: An epidemiologist’s dream? Epidemiology 17(4):360–372
  * Holland (1988) Holland PW. 1988. Causal inference, path analysis, and recursive structural equations models. Sociological Methodology 18(1):449–484
  * Imbens & Angrist (1994) Imbens GW, Angrist JD. 1994. Identification and estimation of local average treatment effects. Econometrica: Journal of the Econometric Society :467–475
  * Kang (2017) Kang H. 2017. sisvive: Some invalid some valid instrumental variables estimator. R package version 1.4
  * Kang et al. (2021) Kang H, Jiang Y, Zhao Q, Small DS. 2021. Ivmodel: an r package for inference and sensitivity analysis of instrumental variables models with one endogenous variable. Observational Studies 7(2):1–24
  * Kang et al. (2016a) Kang H, Kreuels B, May J, Small DS. 2016a. Full matching approach to instrumental variables estimation with application to the effect of malaria on stunting. Annals of Applied Statistics 10(1):335–364
  * Kang et al. (2022) Kang H, Lee Y, Cai TT, Small DS. 2022. Two robust tools for inference about causal effects with invalid instruments. Biometrics 78(1):24–34
  * Kang et al. (2016b) Kang H, Zhang A, Cai TT, Small DS. 2016b. Instrumental variables estimation with some invalid instruments and its application to mendelian randomization. Journal of the American Statistical Association 111(513):132–144
  * Kolesár et al. (2015) Kolesár M, Chetty R, Friedman J, Glaeser E, Imbens GW. 2015. Identification and inference with many invalid instruments. Journal of Business & Economic Statistics 33(4):474–484
  * Koo et al. (2023) Koo T, Lee Y, Small DS, Guo Z. 2023. Robustiv and controlfunctioniv: Causal inference for linear and nonlinear models with invalid instrumental variables. Observational Studies 9(4):97–120
  * Leeb & Pötscher (2005) Leeb H, Pötscher BM. 2005. Model selection and inference: Facts and fiction. Econometric Theory 21(1):21–59
  * Lewbel (2012) Lewbel A. 2012. Using heteroscedasticity to identify and estimate mismeasured and endogenous regressor models. Journal of Business & Economic Statistics 30(1):67–80
  * Lewbel (2019) Lewbel A. 2019. The identification zoo: Meanings of identification in econometrics. Journal of Economic Literature 57(4):835–903
  * Liao (2013) Liao Z. 2013. Adaptive gmm shrinkage estimation with consistent moment selection. Econometric Theory 29(5):857–904
  * Lin & Jeon (2006) Lin Y, Jeon Y. 2006. Random forests and adaptive nearest neighbors. Journal of the American Statistical Association 101(474):578–590
  * Lin et al. (2024) Lin Y, Windmeijer F, Song X, Fan Q. 2024. On the instrumental variable estimation with many weak and invalid instruments. Journal of The Royal Statistical Society Series B-statistical Methodology
  * Liu et al. (2023) Liu Z, Ye T, Sun B, Schooling M, Tchetgen ET. 2023. Mendelian randomization mixed-scale treatment effect robust identification and estimation for causal inference. Biometrics 79(3):2208–2219
  * Locke et al. (2015) Locke AE, Kahali B, Berndt SI, et al. 2015. Genetic studies of body mass index yield new insights for obesity biology. Nature 518(7538):197–206
  * Meinshausen (2006) Meinshausen N. 2006. Quantile regression forests. Journal of Machine Learning Research 7(Jun):983–999
  * Moreira (2003) Moreira MJ. 2003. A conditional likelihood ratio test for structural models. Econometrica 71(4):1027–1048
  * Murray (2006) Murray MP. 2006. Avoiding invalid instruments and coping with weak instruments. The Journal of Economic Perspectives 20(4):111–132
  * Newey & Windmeijer (2009) Newey WK, Windmeijer F. 2009. Generalized method of moments with many weak moment conditions. Econometrica 77(3):687–719
  * Patel et al. (2024) Patel A, Ditraglia F, Zuber V, Burgess S. 2024. Selection of invalid instruments can improve estimation in mendelian randomization. Annals of Applied Statistics
  * Pierce & Burgess (2013) Pierce BL, Burgess S. 2013. Efficient design for mendelian randomization studies: subsample and 2-sample instrumental variable estimators. American Journal of Epidemiology 178(7):1177–1184
  * Robins (1994) Robins JM. 1994. Correcting for non-compliance in randomized trials using structural nested mean models. Communnications in Statistics 23(8):2379–2412
  * Robins et al. (1992) Robins JM, Mark SD, Newey WK. 1992. Estimating exposure effects by modelling the expectation of exposure conditional on confounders. Biometrics 48(2):479–495
  * Rubin (1980) Rubin DB. 1980. Comment on “randomized analysis of experimental data: The fisher randomization test”. Journal of the American Statistical Association 75(371):591–593
  * Sanderson et al. (2022) Sanderson E, Glymour MM, Holmes MV, Kang H, Morrison J, et al. 2022. Mendelian randomization. Nature Reviews Methods Primers 2(1):6
  * Slob & Burgess (2020) Slob EA, Burgess S. 2020. A comparison of robust Mendelian randomization methods using summary data. Genetic Epidemiology 44(4):313–329
  * Small (2007) Small DS. 2007. Sensitivity analysis for instrumental variables regression with overidentifying restrictions. Journal of the American Statistical Association 102(479):1049–1058
  * Small & Rosenbaum (2008) Small DS, Rosenbaum PR. 2008. War and wages: the strength of instrumental variables and their sensitivity to unobserved biases. Journal of the American Statistical Association 103(483):924–933
  * Solovieff et al. (2013) Solovieff N, Cotsapas C, Lee PH, Purcell SM, Smoller JW. 2013. Pleiotropy in complex traits: challenges and strategies. Nature Reviews Genetics 14(7):483–495
  * Staiger & Stock (1997) Staiger D, Stock JH. 1997. Instrumental variables regression with weak instruments. Econometrica 65(3):557–586
  * Stock et al. (2002) Stock JH, Wright JH, Yogo M. 2002. A survey of weak instruments and weak identification in generalized method of moments. Journal of Business & Economic Statistics 20(4):518–529
  * Sudlow et al. (2015) Sudlow C, Gallacher J, Allen N, Beral V, Burton P, et al. 2015. UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Medicine 12(3):e1001779
  * Sun et al. (2022) Sun B, Cui Y, Tchetgen ET. 2022. Selective machine learning of the average treatment effect with an invalid instrumental variable. Journal of Machine Learning Research 23(204):1–40
  * Sun et al. (2023) Sun B, Liu Z, Tchetgen Tchetgen EJ. 2023. Semiparametric efficient G-estimation with invalid instrumental variables. Biometrika 110(4):953–971
  * Tchetgen Tchetgen et al. (2021) Tchetgen Tchetgen E, Sun B, Walter S. 2021. The genius approach to robust mendelian randomization inference. Statistical Science 36(3):443–464
  * Tyrrell et al. (2016) Tyrrell J, Jones SE, Beaumont R, Astley CM, Lovell R, et al. 2016. Height, body mass index, and socioeconomic status: mendelian randomisation study in UK Biobank. British Medical Journal 352:i582
  * Wager & Athey (2018) Wager S, Athey S. 2018. Estimation and inference of heterogeneous treatment effects using random forests. J. Am. Stat. Assoc. 113(523):1228–1242
  * Windmeijer et al. (2019) Windmeijer F, Farbmacher H, Davies N, Davey Smith G. 2019. On the use of the lasso for instrumental variables estimation with some invalid instruments. Journal of the American Statistical Association 114(527):1339–1350
  * Windmeijer et al. (2021) Windmeijer F, Liang X, Hartwig FP, Bowden J. 2021. The confidence interval method for selecting valid instrumental variables. Journal of the Royal Statistical Society Series B: Statistical Methodology 83(4):752–776
  * Wooldridge (2010) Wooldridge JM. 2010. Econometric analysis of cross section and panel data. MIT press, 2nd ed.
  * Yao et al. (2024) Yao M, Miller G, Vardarajan B, Baccarelli A, Guo Z, Liu Z. 2024. Robust mendelian randomization analysis by automatically selecting valid genetic instruments with applications to identify plasma protein biomarkers for Alzheimer’s disease. medRxiv
  * Ye et al. (2024) Ye T, Liu Z, Sun B, Tchetgen Tchetgen E. 2024. GENIUS-MAWII: for robust Mendelian randomization with many weak invalid instruments. Journal of the Royal Statistical Society Series B: Statistical Methodology :qkae024
  * Ye et al. (2021) Ye T, Shao J, Kang H. 2021. Debiased inverse-variance weighted estimator in two-sample summary-data mendelian randomization. The Annals of Statistics 49(4):2079–2100
  * Zhang et al. (2022) Zhang X, Wang L, Volgushev S, Kong D. 2022. Fighting noise with noise: Causal inference with many candidate instruments. arXiv preprint arXiv:2203.09330
  * Zhao et al. (2020) Zhao Q, Wang J, Hemani G, Bowden J, Small DS. 2020. Statistical inference in two-sample summary-data Mendelian randomization using robust adjusted profile score. The Annals of Statistics 48(3):1742 – 1769
  * Zou (2006) Zou H. 2006. The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101(476):1418–1429