ones obtained by the proposed method using full data, especially when
$\sigma_{D}^{2}$ is small. And estimates by “Naive” are biased. When the
variance of the random error in the exposure $\epsilon_{D}$ increases, the
performance of “Proposed (full)” remains similar, which indicates that our
approach using full data works even when the variance of unobserved
confounding is relatively small compared to that of the random error
$\epsilon_{D}$. For the sample splitting method, one can see the RMSE
increases and the coverage probability decreases further below the nominal
level when $\epsilon_{D}$ increases.

Table 4:  Average number of invalid, valid IVs and irrelevant variables in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ under simulation 1 respectively across 1000 Monte
Carlo runs. Note that “Naive” is applied without including pseudo variables.
The results are based on $n=500$ and $p=100,000$. “$-$” stands for
“inapplicable”

|  | $\check{\mathcal{S}}_{2}$ | $\check{\mathcal{S}}_{3}$ | $\check{\mathcal{S}}_{4}$
---|---|---|---|---
| Method | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant
| Proposed (full) | 2.00 | 7.00 | 32.41 |  | 1.99 | 6.99 | 1.97 |  | 0.00 | 6.67 | 0.52
$\sigma_{D}^{2}=0$ | Proposed (split) | 1.89 | 7.00 | 51.38 |  | 1.80 | 6.65 | 2.16 |  | 0.21 | 6.64 | 0.11
| Naive | 2.00 | 7.00 | 15.66 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.03 | 15.65
| Proposed (full) | 2.00 | 7.00 | 37.76 |  | 1.97 | 6.98 | 2.07 |  | 0.00 | 6.85 | 1.07
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.88 | 7.00 | 56.76 |  | 1.78 | 6.36 | 2.07 |  | 0.28 | 6.33 | 0.12
| Naive | 2.00 | 7.00 | 18.44 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.48 | 18.44
| Proposed (full) | 2.00 | 7.00 | 42.17 |  | 1.98 | 6.68 | 2.08 |  | 0.04 | 6.57 | 1.21
$\sigma_{D}^{2}=4$ | Proposed (split) | 1.87 | 7.00 | 62.37 |  | 1.78 | 5.33 | 2.13 |  | 0.36 | 5.25 | 0.17
| Naive | 2.00 | 7.00 | 21.37 |  | $-$ | $-$ | $-$ |  | 0.00 | 2.41 | 21.36

The results in Table 4 show that “Naive” misidentifies spurious instruments as
valid and miss many or almost all the valid instruments. The sample splitting
procedure can find most valid instruments when the variance $\sigma_{D}^{2}$
is small. However, the estimated set of valid instruments will include some
invalid ones since we only use part of the data and the estimates have larger
variation than those obtained using the full sample. Consequently, the
resulting causal effect estimate has moderate bias and the empirical coverage
deviates from the nominal level. In comparison, “Proposed (full)” can locate
spurious instruments with the help of pseudo variables and correctly identify
most valid instruments even if $\sigma_{D}^{2}$ is large.

(a) $\check{\mathcal{S}}_{2}$ (b) $\check{\mathcal{S}}_{3}$: Remove spurious
IVs (c) $\check{\mathcal{S}}_{4}$ (d) $\check{\mathcal{S}}_{4}$ (refit)

Figure 7: Plots (a)-(c) correspond to the simulation 1 for higher dimension in
Section F.1. Plots (a)-(c) are histograms of causal effect estimates from
valid, invalid and spurious instruments in $\check{\mathcal{S}}_{2}$,
$\check{\mathcal{S}}_{3}$ and $\check{\mathcal{S}}_{4}$ respectively obtained
from Algorithm 2–Proposed (full) across 1000 Monte Carlo runs. The blue pile
represents causal estimates from valid IVs; the green pile refers to invalid
IVs; the orange pile is causal estimates from irrelevant variables, including
both irrelevant variables from the initial candidate set and pseudo ones. The
red dashed line is the true causal effect $\beta^{*}=2$. The results are based
on $(n,p,\sigma_{D}^{2})=(500,50,000,0)$. Plot (d) gives the distribution of
effect estimates from refitted model.

###  F.2 Some irrelevant variables are correlated with valid instruments –
simulation 2

To further evaluate the performance of our method, we consider the scenario
where noise variables can be correlated with valid instruments. In the
simulations, we set $p=50,000$ and there are 9 relevant instruments generated
from independent standard normal, two of which are invalid that have direct
effect on the outcome. For each $j=3,\ldots,7$, we generate 10 irrelevant
variables which follow $N(0,1)$ and are correlated with valid instrument
$Z_{j}$. In our numerical experiments, we consider weak and moderate
correlations, which are 0.2 and 0.6 respectively. All the other irrelevant
variables, the unobserved confounding $U$ and random error $\epsilon_{Y}$ are
independently generated from $N(0,1)$. Random error $\epsilon_{D}$ follows
$N(0,\sigma_{D}^{2})$ with $\sigma_{D}^{2}=0,2,4$. We set the true causal
effect $\beta^{*}=2$. Coefficient parameters of the true models are given by

$\displaystyle\bm{\gamma}^{*}=(3,3,3,3,3,3,3,3,3,0,\ldots,0)^{\mathrm{\scriptstyle
T}}$ $\displaystyle\bm{\pi}^{*}=(3.5,3.5,0,0,\dots,0)^{\mathrm{\scriptstyle
T}}$ $\displaystyle\alpha_{D}^{*}=4;\alpha_{Y}^{*}=-3.$

Our comparison includes the proposed method using full data–Proposed (full),
sample splitting—Proposed (split), the naive Algorithm 1–Naive, and the oracle
method (Oracle) by assuming that we know the valid instruments a priori. We
evaluate the performance in terms of bias, efficiency and validity of
inference, and summarize the simulation results in Tables 5 and 6. Histograms
of causal effect estimates for weak and moderate correlation are displayed in
Figure 8 and Figure 9 respectively.

Table 5: Performance summary for various methods and $\sigma_{D}^{2}$ under
simulation 2 across 1000 Monte Carlo runs with $n=500$ and $p=50,000$. We make
the numbers bold for the method with the best performance except the oracle
estimate. Standard errors are presented in the bracket if applicable. For
$\text{SE}<0.005$, we report as $\text{SE}=0.00$

| Method |  $\text{Bias}\times 10$ ($\text{SE}\times 10$) | $\text{RMSE}\times 10$ | Coverage (Nominal = 95%)
---|---|---|---|---
|  | Weak Correlation |
| Proposed (full) | 0.01(0.01) | 0.39 | 0.95
$\sigma_{D}^{2}=0$ | Proposed (split) | 0.52(0.06) | 2.10 | 0.84
| Naive |  $-$7.61(0.01) | 7.62 | 0.00
| Oracle | 0.02(0.01) | 0.33 | 0.96
| Proposed (full) | 0.04(0.01) | 0.46 | 0.95
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.49(0.10) | 3.53 | 0.69
| Naive |  $-$6.77(0.03) | 6.83 | 0.00
| Oracle | 0.01(0.01) | 0.33 | 0.95
| Proposed (full) | 0.33(0.06) | 1.80 | 0.92
$\sigma_{D}^{2}=4$ | Proposed (split) | 3.49(0.14) | 5.60 | 0.43
| Naive |  $-$4.86(0.07) | 5.33 | 0.02
| Oracle | 0.01(0.01) | 0.33 | 0.95
|  | Moderate Correlation |
| Proposed (full) | 0.02(0.01) | 0.43 | 0.95
$\sigma_{D}^{2}=0$ | Proposed (split) | 0.53(0.06) | 1.91 | 0.82
| Naive |  $-$7.56(0.01) | 7.57 | 0.00
| Oracle | 0.02(0.01) | 0.33 | 0.96
| Proposed (full) | 0.04(0.02) | 0.52 | 0.95
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.27(0.09) | 3.09 | 0.68
| Naive |  $-$6.49(0.04) | 6.62 | 0.00
| Oracle | 0.01(0.01) | 0.33 | 0.95
| Proposed (full) | 0.25(0.05) | 1.47 | 0.93
$\sigma_{D}^{2}=4$ | Proposed (split) | 3.15(0.13) | 5.15 | 0.45
| Naive |  $-$3.88(0.07) | 4.48 | 0.05
| Oracle | 0.01(0.01) | 0.33 | 0.95

Table 6:  Average number of invalid, valid IVs and irrelevant variables in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ under simulation 2 respectively across 1000 Monte
Carlo runs. Note that “Naive” is applied without including pseudo variables.
The results are based on $n=500$ and $p=50,000$. “$-$” stands for
“inapplicable”

|  | $\check{\mathcal{S}}_{2}$ | $\check{\mathcal{S}}_{3}$ | $\check{\mathcal{S}}_{4}$
---|---|---|---|---
| Method | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant
|  |  |  | Weak Correlation |  |  |  |  |  |
| Proposed (full) | 2.00 | 7.00 | 39.55 |  | 2.00 | 7.00 | 2.04 |  | 0.00 | 6.75 | 0.57
$\sigma_{D}^{2}=0$ | Proposed (split) | 2.00 | 6.96 | 77.75 |  | 1.99 | 6.69 | 2.07 |  | 0.22 | 6.67 | 0.04
| Naive | 2.00 | 7.00 | 31.49 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.00 | 31.48
| Proposed (full) | 2.00 | 7.00 | 45.25 |  | 2.00 | 6.96 | 2.01 |  | 0.00 | 6.89 | 1.01
$\sigma_{D}^{2}=2$ | Proposed (split) | 2.00 | 6.95 | 83.21 |  | 1.99 | 5.79 | 1.88 |  | 0.46 | 5.66 | 0.03
| Naive | 2.00 | 7.00 | 36.09 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.04 | 36.06
| Proposed (full) | 2.00 | 7.00 | 50.49 |  | 2.00 | 6.61 | 2.07 |  | 0.06 | 6.55 | 1.17
$\sigma_{D}^{2}=4$ | Proposed (split) | 2.00 | 6.95 | 89.15 |  | 1.99 | 4.03 | 2.02 |  | 0.84 | 3.72 | 0.12
| Naive | 2.00 | 7.00 | 40.09 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.74 | 40.06
|  |  |  | Moderate Correlation |  |  |  |  |  |
| Proposed (full) | 2.00 | 7.00 | 37.39 |  | 2.00 | 7.00 | 2.17 |  | 0.00 | 6.61 | 0.71
$\sigma_{D}^{2}=0$ | Proposed (split) | 2.00 | 6.96 | 76.13 |  | 1.99 | 6.72 | 2.28 |  | 0.25 | 6.70 | 0.03
| Naive | 2.00 | 7.00 | 31.06 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.00 | 31.04
| Proposed (full) | 2.00 | 7.00 | 42.15 |  | 2.00 | 6.96 | 2.08 |  | 0.00 | 6.89 | 1.08
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.99 | 6.95 | 82.50 |  | 1.99 | 5.88 | 2.13 |  | 0.48 | 5.72 | 0.04
| Naive | 2.00 | 7.00 | 35.26 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.13 | 35.24
| Proposed (full) | 2.00 | 7.00 | 47.33 |  | 2.00 | 6.57 | 2.25 |  | 0.04 | 6.42 | 1.26
$\sigma_{D}^{2}=4$ | Proposed (split) | 1.99 | 6.94 | 88.12 |  | 1.99 | 4.18 | 2.18 |  | 0.83 | 3.76 | 0.09
| Naive | 2.00 | 7.00 | 38.96 |  | $-$ | $-$ | $-$ |  | 0.00 | 1.52 | 38.93

In the presence of irrelevant variables correlated with valid instruments,
“Proposed (full)” is comparable with the oracle estimate. However, “Proposed
(split)” and “Naive” perform badly. Comparing the number of valid, invalid and
spurious instruments in the estimated set of valid instruments
$\check{\mathcal{S}}_{4}$ as shown in Table 6, one can see that “Proposed
(full)” can correctly identify most valid instruments. The sample splitting
fails to identify invalid IVs due to the fact that only part of the data is
used and the associated estimates have larger variances. For “Naive”, it
misses most valid IVs and misuses many spurious instruments as valid ones.

(a) $\check{\mathcal{S}}_{2}$ (b) $\check{\mathcal{S}}_{3}$: Remove spurious
IVs (c) $\check{\mathcal{S}}_{4}$ (d) $\check{\mathcal{S}}_{4}$ (refit)

Figure 8: Plots (a)-(c) correspond to the simulation 2 for weak correlation in
Section F.2. Plots (a)-(c) are histograms of causal effect estimates from
valid, invalid and spurious instruments in $\check{\mathcal{S}}_{2}$,
$\check{\mathcal{S}}_{3}$ and $\check{\mathcal{S}}_{4}$ respectively obtained
from Algorithm 2–Proposed (full) across 1000 Monte Carlo runs. The blue pile
represents causal estimates from valid IVs; the green pile refers to invalid
IVs; the orange pile is causal estimates from irrelevant variables, including
both irrelevant variables from the initial candidate set and pseudo ones. The
red dashed line is the true causal effect $\beta^{*}=2$. The results are based
on $(n,p,\sigma_{D}^{2})=(500,50,000,0)$. Plot (d) gives the distribution of
effect estimates from refitted model.

(a) $\check{\mathcal{S}}_{2}$ (b) $\check{\mathcal{S}}_{3}$: Remove spurious
IVs (c) $\check{\mathcal{S}}_{4}$ (d) $\check{\mathcal{S}}_{4}$ (refit)

Figure 9: Plots (a)-(c) correspond to the simulation 2 for moderate
correlation in Section F.2. Plots (a)-(c) are histograms of causal effect
estimates from valid, invalid and spurious instruments in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ respectively obtained from Algorithm 2–Proposed
(full) across 1000 Monte Carlo runs. The blue pile represents causal estimates
from valid IVs; the green pile refers to invalid IVs; the orange pile is
causal estimates from irrelevant variables, including both irrelevant
variables from the initial candidate set and pseudo ones. The red dashed line
is the true causal effect $\beta^{*}=2$. The results are based on
$(n,p,\sigma_{D}^{2})=(500,50,000,0)$. Plot (d) gives the distribution of
effect estimates from refitted model.

###  F.3 Correlated irrelevant variables – simulation 3

In practice, irrelevant variables might also be correlated with each other.
For example, in genetic data, neighbouring SNPs may have large correlation,
formally called haplotypes. If a SNP has nonnegligible sample correlation with
the risk factor due to pure chance and is estimated to be relevant, then the
SNPs that are highly correlated with this one have more chance to enter the
model. To assess how the proposed method performs under this setting, we mimic
the aforementioned data structure by generating correlated irrelevant
variables having different correlation magnitude: low, medium and high.
Specifically, we generate $p=50,000$ candidate instruments from $N(0,1)$,
where $Z_{1},Z_{2}$ have direct effects on the outcome, $Z_{3},\ldots,Z_{9}$
are valid instruments and $Z_{j}$’s are noise variables for $j\geq 10$. We
group irrelevant variables into 200 blocks. In the first block, irrelevant
variables $Z_{10},\ldots,Z_{250}$ are independent. While for the remaining 199
blocks, each block consists of 250 irrelevant variables, where the first half
are weakly correlated with correlation 0.2 and the second half have moderate
correlation equal to 0.5. Same as before, the unmeasured confounder $U$ and
random error $\epsilon_{Y}$ follow $N(0,1)$. The random error $\epsilon_{D}$
is generated from $N(0,\sigma_{D}^{2})$ with $\sigma_{D}^{2}=0,2,4$. We set
the true causal estimate $\beta^{*}=2$. The other parameters in the risk
factor and the outcome models (2.1) and (2.2) are given by

$\displaystyle\bm{\gamma}^{*}=(3,3,3,3,3,3,3,3,3,0,\ldots,0)^{\mathrm{\scriptstyle
T}}$ $\displaystyle\bm{\pi}^{*}=(3.5,3.5,0,0,\dots,0)^{\mathrm{\scriptstyle
T}}$ $\displaystyle\alpha_{D}^{*}=4;\alpha_{Y}^{*}=-3.$

Our comparison includes the proposed method using full data–Proposed (full),
sample splitting–Proposed (split), the naive Algorithm 1–Naive, and the oracle
method (Oracle) by assuming that we know the valid instruments a priori. We
evaluate the performance in terms of bias, efficiency and validity of
inference, and summarize the simulation results in Table 7. We present in 8
the average number of valid, invalid and spurious instruments passing joint
thresholding, after spurious IVs removal, and also in the estimated set of
valid instruments, respectively. See Figure 10 for the histogram of causal
effect estimates under $\sigma_{D}^{2}=0$.

Table 7: Performance summary for various methods and $\sigma_{D}^{2}$ under
simulation 3 across 1000 Monte Carlo runs with $n=500$ and $p=50,000$. We make
the numbers bold for the method with the best performance except the oracle
estimate. Standard errors are presented in the bracket if applicable. For
$\text{SE}<0.005$, we report as $\text{SE}=0.00$

| Method |  $\text{Bias}\times 10$ ($\text{SE}\times 10$) | $\text{RMSE}\times 10$ | Coverage (Nominal = 95%)
---|---|---|---|---
| Proposed (full) | 0.00(0.01) | 0.39 | 0.95
$\sigma_{D}^{2}=0$ | Proposed (split) | 0.40(0.06) | 2.01 | 0.86
| Naive |  $-$7.55(0.01) | 7.56 | 0.00
| Oracle | 0.02(0.01) | 0.33 | 0.96
| Proposed (full) | 0.04(0.02) | 0.60 | 0.95
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.19(0.09) | 3.17 | 0.71
| Naive |  $-$6.64(0.03) | 6.71 | 0.00
| Oracle | 0.01(0.01) | 0.33 | 0.95
| Proposed (full) | 0.27(0.05) | 1.66 | 0.93
$\sigma_{D}^{2}=4$ | Proposed (split) | 1.98(0.10) | 3.59 | 0.52
| Naive |  $-$4.39(0.08) | 5.06 | 0.06
| Oracle | 0.01(0.01) | 0.33 | 0.95

The “Proposed (full)” still performs well under the setting that some
irrelevant variables are correlated. The sample splitting only uses part of
the samples and some invalid instruments are estimated as valid due to the
loss of efficiency and its performance is getting worse as $\sigma_{D}^{2}$
increases. Consequently, the resulting causal effect estimate has moderate
bias and the empirical coverage deviates from the nominal level. The “Naive”
gives biased causal estimate as it has misidentified valid as invalid and
misused many spurious instruments as valid ones. As a result, the empirical
coverage of “Naive” is far from the nominal level and average bias of causal
estimate is large in comparison with other methods.

Table 8:  Average number of invalid, valid IVs and irrelevant variables in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ under simulation 3 respectively across 1000 Monte
Carlo runs. Note that “Naive” is applied without including pseudo variables.
The results are based on $n=500$ and $p=50,000$. “$-$” stands for
“inapplicable”

|  | $\check{\mathcal{S}}_{2}$ | $\check{\mathcal{S}}_{3}$ | $\check{\mathcal{S}}_{4}$
---|---|---|---|---
| Method | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant
| Proposed (full) | 2.00 | 7.00 | 33.81 |  | 2.00 | 7.00 | 2.00 |  | 0.00 | 6.73 | 0.49
$\sigma_{D}^{2}=0$ | Proposed (split) | 1.99 | 6.97 | 70.16 |  | 1.98 | 6.72 | 1.90 |  | 0.19 | 6.71 | 0.05
| Naive | 2.00 | 7.00 | 26.17 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.00 | 26.17
| Proposed (full) | 2.00 | 7.00 | 38.43 |  | 2.00 | 6.97 | 1.98 |  | 0.00 | 6.92 | 1.00
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.99 | 6.96 | 76.54 |  | 1.99 | 6.02 | 2.25 |  | 0.42 | 5.92 | 0.04
| Naive | 2.00 | 7.00 | 29.64 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.04 | 29.63
| Proposed (full) | 2.00 | 7.00 | 42.72 |  | 2.00 | 6.69 | 1.94 |  | 0.05 | 6.60 | 1.01
$\sigma_{D}^{2}=4$ | Proposed (split) | 1.99 | 6.96 | 81.84 |  | 1.98 | 4.32 | 2.19 |  | 0.82 | 4.05 | 0.11
| Naive | 2.00 | 7.00 | 33.05 |  | $-$ | $-$ | $-$ |  | 0.00 | 1.00 | 33.03

(a) $\check{\mathcal{S}}_{2}$ (b) $\check{\mathcal{S}}_{3}$: Remove spurious
IVs (c) $\check{\mathcal{S}}_{4}$ (d) $\check{\mathcal{S}}_{4}$ (refit)

Figure 10: Plots (a)-(d) correspond to the simulation 3 where some irrelevant
variables are correlated in Section F.3. Plots (a)-(c) are histograms of
causal effect estimates from valid, invalid and spurious instruments in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ respectively obtained from Algorithm 2–Proposed
(full) across 1000 Monte Carlo runs. The blue pile represents causal estimates
from valid IVs; the green pile refers to invalid IVs; the orange pile is
causal estimates from irrelevant variables, including both irrelevant
variables from the initial candidate set and pseudo ones. The red dashed line
is the true causal effect $\beta^{*}=2$. The results are based on
$(n,p,\sigma_{D}^{2})=(500,50,000,0)$. The last figure (d) gives the
distribution of causal estimates from refitted model.

###  F.4 Numerical experiment under multivariate unmeasured confounding –
simulation 4

In this section, we consider multivariate unmeasured confounders
$\bm{U}\in\mathbb{R}^{2}$. The direct effects of $\bm{U}$ on the exposure and
the outcome are set to

$\displaystyle\bm{\alpha}_{D}^{*}=(2,2)^{\mathrm{\scriptstyle
T}};\bm{\alpha}_{Y}^{*}=(-2,-2)^{\mathrm{\scriptstyle T}}.$

All the other settings remain the same as the simulation study in the main
paper. Our comparison includes the proposed method using full data–Proposed
(full), sample splitting—Proposed (split), the naive Algorithm 1–Naive, and
the oracle method (Oracle) by assuming that we know the valid instruments a
priori. We evaluate the performance in terms of bias, efficiency and validity
of inference, and summarize the simulation results in Tables 9 and 10. The
histogram of causal effect estimates under $\sigma_{D}^{2}=0$ is displayed in
Figure 11.

Table 9: Performance summary for various methods and $\sigma_{D}^{2}$ under
simulation 4 across 1000 Monte Carlo runs with $n=500$ and $p=50,000$. We make
the numbers bold for the method with the best performance except the oracle
estimate. Standard errors are presented in the bracket if applicable. For
$\text{SE}<0.005$, we report as $\text{SE}=0.00$

| Method |  $\text{Bias}\times 10$ ($\text{SE}\times 10$) | $\text{RMSE}\times 10$ | Coverage (Nominal = 95%)
---|---|---|---|---
| Proposed (full) |  $-$0.13(0.02) | 0.80 | 0.91
$\sigma_{D}^{2}=0$ | Proposed (split) | 0.07(0.07) | 2.09 | 0.79
| Naive |  $-$1.61(0.06) | 2.41 | 0.50
| Oracle |  $-$0.02(0.01) | 0.27 | 0.94
| Proposed (full) |  $-$0.08(0.02) | 0.54 | 0.93
$\sigma_{D}^{2}=2$ | Proposed (split) | 0.08(0.06) | 1.88 | 0.81
| Naive |  $-$1.23(0.05) | 1.97 | 0.50
| Oracle |  $-$0.02(0.01) | 0.26 | 0.94
| Proposed (full) |  $-$0.13(0.04) | 1.22 | 0.90
$\sigma_{D}^{2}=4$ | Proposed (split) | 0.35(0.11) | 3.45 | 0.78
| Naive |  $-$0.53(0.02) | 0.86 | 0.59
| Oracle |  $-$0.02(0.01) | 0.26 | 0.95

The simulation results in Table 9 suggest that “Proposed (full)” performs well
in terms of bias, efficiency and validity of inference under multivariate
unmeasured confounders. For the sample splitting–Proposed (split), although
the bias is close to “Proposed (full)” when $\sigma_{D}^{2}=0,2$, but the RMSE
is much larger and the empirical coverage deviates from the nominal level
compared to the ones obtained by the proposed method using full data. When the
variance of the random error in the exposure $\epsilon_{D}$ increases, the
performance of “Proposed (full)” remains similar, which indicates that our
approach using full data works even when the variance of unobserved
confounding is relatively small compared to that of the random error
$\epsilon_{D}$. For the sample splitting method, one can see the RMSE
increases and the coverage probability decreases further below the nominal
level when $\epsilon_{D}$ increases. The causal estimate by “Naive” is biased
and the empirical coverage is far from the nominal level.

Table 10:  Average number of invalid, valid IVs and irrelevant variables in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ under simulation 4 respectively across 1000 Monte
Carlo runs. Note that “Naive” is applied without including pseudo variables.
The results are based on $n=500$ and $p=50,000$. “$-$” stands for
“inapplicable”

|  | $\check{\mathcal{S}}_{2}$ | $\check{\mathcal{S}}_{3}$ | $\check{\mathcal{S}}_{4}$
---|---|---|---|---
| Method | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant
| Proposed (full) | 2.00 | 7.00 | 11.95 |  | 1.53 | 6.99 | 1.89 |  | 0.03 | 6.83 | 0.31
$\sigma_{D}^{2}=0$ | Proposed (split) | 1.94 | 7.00 | 23.85 |  | 1.26 | 6.80 | 2.02 |  | 0.45 | 6.80 | 0.04
| Naive | 2.00 | 7.00 | 4.92 |  | $-$ | $-$ | $-$ |  | 0.12 | 2.85 | 4.84
| Proposed (full) | 2.00 | 7.00 | 16.28 |  | 1.73 | 6.99 | 1.88 |  | 0.02 | 6.89 | 0.90
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.94 | 7.00 | 30.75 |  | 1.38 | 6.67 | 2.02 |  | 0.43 | 6.67 | 0.04
| Naive | 2.00 | 7.00 | 7.29 |  | $-$ | $-$ | $-$ |  | 0.34 | 4.24 | 7.25
| Proposed (full) | 2.00 | 7.00 | 20.82 |  | 1.85 | 6.61 | 2.06 |  | 0.06 | 6.52 | 1.26
$\sigma_{D}^{2}=4$ | Proposed (split) | 1.94 | 7.00 | 37.03 |  | 1.55 | 5.47 | 2.03 |  | 0.41 | 5.44 | 0.09
| Naive | 2.00 | 7.00 | 9.78 |  | $-$ | $-$ | $-$ |  | 0.29 | 6.32 | 9.76

The results in Table 10 show that “Naive” misidentifies spurious instruments
and some invalid instruments as valid ones, and also misses some valid
instruments. The sample splitting procedure can find most valid instruments
when the variance $\sigma_{D}^{2}$ is small. However, the estimated set of
valid instruments will include some invalid ones since we only use part of the
data and the estimates have larger variation than those obtained using the
full sample. In comparison, “Proposed (full)” can locate spurious instruments
with the help of pseudo variables and correctly identify most valid
instruments even if $\sigma_{D}^{2}$ is large.

(a) $\check{\mathcal{S}}_{2}$ (b) $\check{\mathcal{S}}_{3}$: Remove spurious
IVs (c) $\check{\mathcal{S}}_{4}$ (d) $\check{\mathcal{S}}_{4}$ (refit)

Figure 11: Plots (a)-(d) correspond to the simulation 4 where some irrelevant
variables are correlated in Section F.4. Plots (a)-(c) are histograms of
causal effect estimates from valid, invalid and spurious instruments in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ respectively obtained from Algorithm 2–Proposed
(full) across 1000 Monte Carlo runs. The blue pile represents causal estimates
from valid IVs; the green pile refers to invalid IVs; the orange pile is
causal estimates from irrelevant variables, including both irrelevant
variables from the initial candidate set and pseudo ones. The red dashed line
is the true causal effect $\beta^{*}=2$. The results are based on
$(n,p,\sigma_{D}^{2})=(500,50,000,0)$. The last figure (d) gives the
distribution of causal estimates from refitted model.

###  F.5 Weak instruments – simulation 5

In this simulation, we consider a more general setting in which there are 7
strong valid instruments, 50 weak valid instruments and 2 invalid IVs.
Candidate instruments $Z_{j}$, the unobserved confounding $U$ and random error
$\epsilon_{Y}$ are generated from independent standard normal distributions
for $1\leq j\leq p$. The random error $\epsilon_{D}$ follows
$N(0,\sigma_{D}^{2})$ with $\sigma_{D}^{2}=0,2,4$.

For the weak IVs $Z_{10},\ldots,Z_{59}$, true parameter values of the
relevance strength to the exposure in absolute value are randomly generated
from uniform distribution on the interval $[0.01,0.3]$. The other parameters
are given by

$\displaystyle\bm{\gamma}_{1:9}^{*}=(3,3,3,3,3,3,3,3,3)^{\mathrm{\scriptstyle
T}}$
$\displaystyle\bm{\pi}_{1:2}^{*}=(3.5,3.5,0,0,\dots,0)^{\mathrm{\scriptstyle
T}}$ $\displaystyle\bm{\psi}^{*}=\bm{\phi}^{*}=\bm{0}$
$\displaystyle\alpha_{D}^{*}=4;\alpha_{Y}^{*}=-3.$

Here $\bm{\gamma}_{1:9}^{*}$ denotes the first nine elements in
$\bm{\gamma}^{*}$ and analogously $\bm{\pi}_{1:2}^{*}$ represents the first
two components in $\bm{\pi}^{*}$. The dimension of candidate instruments
$p=50,000$. Our comparison includes the proposed method using full
data–Proposed (full), sample splitting–Proposed (split), the naive Algorithm
1–Naive, and the oracle method by assuming that we know the valid instruments
a priori. Simulation results are presented in Tables 11 and 12. In the tables,
we use “Oracle (All IVs)” and “Oracle (Strong IVs)” to distinct the oracle
estimate calculated using all valid instruments (strong and weak) and the
oracle estimate obtained using strong valid instruments only. Additionally,
histograms of causal effect estimates under $\sigma_{D}^{2}=0$ are presented
in Figure 12.

Table 11: Performance summary for various methods and $\sigma_{D}^{2}$ under
simulation 5 across 1000 Monte Carlo runs with $n=500$ and $p=50,000$. We make
the numbers bold for the method with the best performance except the oracle
estimate. Standard errors are presented in the bracket if applicable. For
$\text{SE}<0.005$, we report as $\text{SE}=0.00$

| Method |  $\text{Bias}\times 10$ ($\text{SE}\times 10$) | $\text{RMSE}\times 10$ | Coverage (Nominal = 95%)
---|---|---|---|---
| Proposed (full) | 0.02(0.01) | 0.38 | 0.94
| Proposed (split) | 0.74(0.07) | 2.34 | 0.76
$\sigma_{D}^{2}=0$ | Naive |  $-$7.11(0.03) | 7.19 | 0.00
| Oracle(All IVs) | 0.14(0.01) | 0.35 | 0.91
| Oracle(Strong IVs) | 0.01(0.01) | 0.33 | 0.95
| Proposed (full) | 0.11(0.03) | 0.90 | 0.94
| Proposed (split) | 2.43(0.13) | 4.75 | 0.55
$\sigma_{D}^{2}=2$ | Naive |  $-$4.90(0.09) | 5.65 | 0.07
| Oracle(All IVs) | 0.14(0.01) | 0.34 | 0.93
| Oracle(Strong IVs) | 0.01(0.01) | 0.33 | 0.95
| Proposed (full) | 0.71(0.08) | 2.70 | 0.88
| Proposed (split) | 4.69(0.15) | 6.63 | 0.33
$\sigma_{D}^{2}=4$ | Naive |  $-$2.08(0.08) | 3.31 | 0.17
| Oracle(All IVs) | 0.14(0.01) | 0.34 | 0.93
| Oracle(Strong IVs) | 0.01(0.01) | 0.33 | 0.96

Performance of “Proposed (full)” is similar to the oracle estimate that only
uses strong valid instruments when $\sigma_{D}^{2}=0,2$. But the performance
becomes undesirable as $\sigma_{D}^{2}$ further increases, this is possibly
due to the fact that effect estimates have larger variances and more invalid
IVs are estimated as valid ones and some valid IVs are misidentified as
spurious ones as shown in Table 12. The sample splitting only uses part of the
samples and some invalid instruments are estimated as valid due to the loss of
efficiency and its performance is getting worse as $\sigma_{D}^{2}$ increases.
The “Naive” fails as it misidentifies many spurious instruments as valid
instruments. The performance of “Naive” starts to improve a bit for large
$\sigma_{D}^{2}$ since some valid instruments are correctly identified. This
is because causal estimates have larger variation when $\sigma_{D}^{2}$
further increases and some valid instruments may have similar effect estimates
to those for spurious instruments. Hence part of valid instruments are not
distinguishable from the spurious ones. The difference between “Oracle(All
IVs)” and “Oracle(Strong IVs)” is attributable to the bias introduced by weak
instruments, which is consistent with existing literature (Nelson & Startz,
1990a; Stock & Yogo, 2005; Burgess et al., 2015).

Table 12:  Average number of invalid, valid IVs and irrelevant variables in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ under simulation 5 respectively across 1000 Monte
Carlo runs. Note that “Naive” is applied without including pseudo variables.
The results are based on $n=500$ and $p=50,000$. “$-$” stands for
“inapplicable”

|  | $\check{\mathcal{S}}_{2}$ | $\check{\mathcal{S}}_{3}$ | $\check{\mathcal{S}}_{4}$
---|---|---|---|---
| Method | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant
| Proposed (full) | 2.00 | 7.09 | 42.68 |  | 2.00 | 7.03 | 2.02 |  | 0.00 | 6.88 | 0.95
$\sigma_{D}^{2}=0$ | Proposed (split) | 1.99 | 7.06 | 81.10 |  | 1.99 | 6.45 | 2.25 |  | 0.34 | 6.38 | 0.03
| Naive | 2.00 | 7.09 | 21.05 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.14 | 21.05
| Proposed (full) | 2.00 | 7.08 | 48.32 |  | 2.00 | 6.85 | 2.02 |  | 0.02 | 6.81 | 1.08
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.99 | 7.03 | 87.24 |  | 1.98 | 4.88 | 2.21 |  | 0.69 | 4.63 | 0.11
| Naive | 2.00 | 7.08 | 24.08 |  | $-$ | $-$ | $-$ |  | 0.00 | 0.94 | 24.08
| Proposed (full) | 2.00 | 7.07 | 53.30 |  | 2.00 | 6.13 | 1.99 |  | 0.13 | 6.03 | 1.15
$\sigma_{D}^{2}=4$ | Proposed (split) | 1.99 | 7.03 | 93.18 |  | 1.98 | 3.17 | 2.19 |  | 1.03 | 2.78 | 0.22
| Naive | 2.00 | 7.08 | 27.68 |  | $-$ | $-$ | $-$ |  | 0.00 | 3.00 | 27.68

(a) $\check{\mathcal{S}}_{2}$ (b) $\check{\mathcal{S}}_{3}$: Remove spurious
IVs (c) $\check{\mathcal{S}}_{4}$ (d) $\check{\mathcal{S}}_{4}$ (refit)

Figure 12: Plots (a)-(d) correspond to the simulation 5 when some instruments
are weak in Section F.5. Plots (a)-(c) are histograms of causal effect
estimates from valid, invalid and spurious instruments in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ respectively obtained from Algorithm 2–Proposed
(full) across 1000 Monte Carlo runs. The blue pile represents causal estimates
from valid IVs; the green pile refers to invalid IVs; the orange pile is
causal estimates from irrelevant variables, including both irrelevant
variables from the initial candidate set and pseudo ones. The red dashed line
is the true causal effect $\beta^{*}=2$. The results are based on
$(n,p,\sigma_{D}^{2})=(500,50,000,0)$. The last figure (d) gives the
distribution of causal estimates from refitted model.

###  F.6 Violation of independence assumption (I3) – simulation 6

In this simulation, to examine the robustness our method, in addition to the
violation of no direct effect assumption (I2), we also consider the violation
of independence assumption (I3), that is, instrument can be correlated with
the unobserved confounding $U$. Suppose we have 9 relevant instruments, among
which one has direct effect on the outcome, one is correlated with $U$ and the
others are valid. Instrument $Z_{2}$ and $U$ are bivariate normal with
correlation 0.7 and variance 1. The other candidates instruments and
$\epsilon_{Y}$ are independent standard normal. Random error $\epsilon_{D}$
follows $N(0,\sigma_{D}^{2})$ with $\sigma_{D}^{2}=0,2,4$ and the true causal
effect $\beta^{*}$ is set to 2. The dimension of candidate instruments
$p=50,000$. The true parameters are given as follows.

$\displaystyle\bm{\gamma}^{*}=(3,3,3,3,3,3,3,3,3,0,\ldots,0)^{\mathrm{\scriptstyle
T}}$ $\displaystyle\bm{\pi}^{*}=(3.5,0,0,0,\dots,0)^{\mathrm{\scriptstyle T}}$
$\displaystyle\alpha_{D}^{*}=4;\alpha_{Y}^{*}=-3.$

Our comparison includes the proposed method using full data–Proposed (full),
sample splitting–Proposed (split), the naive Algorithm 1–Naive, and the oracle
method (Oracle) by assuming that we know the valid instruments a priori.
Empirical results are presented in Tables 13 and 14. Histograms of causal
effect estimates under $\sigma_{D}^{2}=0$ are presented in Figure 13.

Table 13: Performance summary for various methods and $\sigma_{D}^{2}$ under
simulation 6 across 1000 Monte Carlo runs with $n=500$ and $p=50,000$. We make
the numbers bold for the method with the best performance except for the
oracle estimate. Standard errors are presented in the bracket if applicable

| Method |  $\text{Bias}\times 10$ ($\text{SE}\times 10$) | $\text{RMSE}\times 10$ | Coverage (Nominal = 95%)
---|---|---|---|---
| Proposed (full) |  $-$0.05(0.02) | 0.54 | 0.92
$\sigma_{D}^{2}=0$ | Proposed (split) | 1.26(0.11) | 3.82 | 0.71
| Naive |  $-$5.38(0.10) | 6.30 | 0.03
| Oracle |  $-$0.01(0.01) | 0.26 | 0.96
| Proposed (full) | 0.09(0.04) | 1.20 | 0.90
$\sigma_{D}^{2}=2$ | Proposed (split) | 3.51(0.16) | 6.25 | 0.42
| Naive |  $-$1.51(0.03) | 1.85 | 0.01
| Oracle |  $-$0.01(0.01) | 0.26 | 0.96
| Proposed (full) | 0.95(0.10) | 3.34 | 0.81
$\sigma_{D}^{2}=4$ | Proposed (split) | 6.63(0.19) | 8.96 | 0.35
| Naive |  $-$1.40(0.01) | 1.46 | 0.00
| Oracle |  $-$0.01(0.01) | 0.26 | 0.96

Table 14:  Average number of invalid, valid IVs and irrelevant variables in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ under simulation 6 respectively across 1000 Monte
Carlo runs. Note that “Naive” is applied without including pseudo variables.
The results are based on $n=500$ and $p=50,000$. “$-$” stands for
“inapplicable”

|  | $\check{\mathcal{S}}_{2}$ | $\check{\mathcal{S}}_{3}$ | $\check{\mathcal{S}}_{4}$
---|---|---|---|---
| Method | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant |  | Invalid | Valid | Irrelevant
| Proposed (full) | 2.00 | 7.00 | 15.19 |  | 1.97 | 6.99 | 1.94 |  | 0.00 | 6.82 | 0.60
$\sigma_{D}^{2}=0$ | Proposed (split) | 1.98 | 6.89 | 43.37 |  | 1.51 | 6.24 | 2.13 |  | 0.36 | 5.93 | 0.18
| Naive | 2.00 | 7.00 | 11.71 |  | $-$ | $-$ | $-$ |  | 0.22 | 0.90 | 11.69
| Proposed (full) | 2.00 | 7.00 | 20.36 |  | 1.29 | 6.75 | 1.95 |  | 0.03 | 6.71 | 1.07
$\sigma_{D}^{2}=2$ | Proposed (split) | 1.98 | 6.88 | 51.86 |  | 1.03 | 4.43 | 2.13 |  | 0.67 | 3.61 | 0.38
| Naive | 2.00 | 7.00 | 16.01 |  | $-$ | $-$ | $-$ |  | 0.96 | 5.24 | 16.00
| Proposed (full) | 2.00 | 7.00 | 25.79 |  | 1.03 | 5.57 | 1.96 |  | 0.12 | 5.53 | 1.27
$\sigma_{D}^{2}=4$ | Proposed (split) | 1.98 | 6.87 | 59.66 |  | 0.99 | 2.09 | 2.13 |  | 0.79 | 1.54 | 0.91
| Naive | 2.00 | 7.00 | 20.25 |  | $-$ | $-$ | $-$ |  | 1.00 | 6.77 | 20.25

The simulation results suggest that the proposed procedure using full data
achieves the best performance among the other competing methods even if the
independence assumption (I3) is violated, but the results become undesirable
when $\sigma_{D}^{2}$ further increases. It was shown in Table 14 that the
“Proposed (full)” would miss some valid instruments and also misidentify some
invalid instruments as valid ones for larger $\sigma_{D}^{2}$. The sample
splitting only uses part of the samples and some invalid instruments are
estimated as valid due to the loss of efficiency and its performance is
getting worse as $\sigma_{D}^{2}$ increases. The “Naive” fails since it has
misidentified some invalid instruments and many spurious instruments as valid
ones.

(a) $\check{\mathcal{S}}_{2}$ (b) $\check{\mathcal{S}}_{3}$: Remove spurious
IVs (c) $\check{\mathcal{S}}_{4}$ (d) $\check{\mathcal{S}}_{4}$

Figure 13: Plots (a)-(c) correspond to the simulation 6 where the independence
assumption is violated in Section F.6. Plots (a)-(c) are histograms of causal
effect estimates from valid, invalid and spurious instruments in
$\check{\mathcal{S}}_{2}$, $\check{\mathcal{S}}_{3}$ and
$\check{\mathcal{S}}_{4}$ respectively obtained from Algorithm 2–Proposed
(full) across 1000 Monte Carlo runs. The blue pile represents causal estimates
from valid IVs; the green pile refers to invalid IVs; the orange pile is
causal estimates from irrelevant variables, including both irrelevant
variables from the initial candidate set and pseudo ones. The red dashed line
is the true causal effect $\beta^{*}=2$. The results are based on
$(n,p,\sigma_{D}^{2})=(500,50,000,0)$. Plot (d) gives the distribution of
effect estimates from refitted model.

###  F.7 Competing methods in econometric literature – simulation 7

In this section, we consider some competing approaches which deal with many
instruments and allow for weak IVs in the econometric literature to lend
further support of our proposed method. Donald & Newey (2001) provided
explicit forms for asymptotic mean squared error (MSE) of two-stage least
squares (2SLS) and limited-information maximum likelihood (LIML) estimators,
which is a function of the number of instruments. They found that selecting
the number of instruments by minimizing MSE can improve performance in the
presence weak instruments. “D2SLS” and “DLIML” refer to the refined 2SLS and
LIML estimators by Donald & Newey (2001) respectively. We also include
regularized 2SLS (Carrasco, 2012) and regularized LIML (Carrasco & Tchuente,
2015) based on Tikhonov (ridge) regularization, which aim to solve the problem
of many instruments efficiently when some are irrelevant. We use the same
notation “T2SLS” and “TLIML” as Carrasco & Tchuente (2015) to denote these
methods.

In the simulation, we assume no invalid instruments, that is, candidate
instrument is either valid or irrelevant. The candidate instruments, the
unobserved confounding $U$, the random error $\epsilon_{Y}$ are independently
generated from independent $N(0,1)$. The random error $\epsilon_{D}$ follows
$N(0,\sigma_{D}^{2})$ with $\sigma_{D}^{2}=0,2,4$. The true parameters are set
as follows.

$\displaystyle\bm{\gamma}^{*}=(2.5,2.5,2.5,2.5,2.5,2.5,2.5,0,\ldots,0)^{\mathrm{\scriptstyle
T}}$ $\displaystyle\pi_{j}^{*}=0,\text{ for }j=1,\ldots p$
$\displaystyle\alpha_{D}^{*}=4;\alpha_{Y}^{*}=-3.$

Evaluation of the performance is carried out across 1000 Monte Carlo runs.
Same as before, suppose there are 50,000 candidate instruments at the
beginning and we first reduce the dimension of this initial set of candidate
instruments to $s=500$ via marginal screening and then apply D2SLS, DLIML,
T2SLS and TLIML to the selected candidates. Then we implement Algorithm 2, the
proposed method using full data. We report bias, root-mean-square error (RMSE)
and emprical coverage for each method, and the oracle estimate which uses
valid instruments only. We summarize the results in Table 15.

Table 15: Performance summary for the comparison with competing methods in
econometric literature-simulation 7. Results are obtained from 1000 Monte
Carlo runs with $p=50,000$. We make the numbers bold for the method with the
best performance except the oracle estimate. Standard errors are presented in
the bracket if applicable. For $\text{SE}<0.005$, we report as
$\text{SE}=0.00$

| Method |  $\text{Bias}\times 10$ ($\text{SE}\times 10$) | $\text{RMSE}\times 10$ | Coverage (Nominal = 95%)
---|---|---|---|---
| Proposed (full) |  $-$0.09(0.01) | 0.39 | 0.92
| T2LS |  $-$1.89(0.01) | 1.90 | 0.00
$\sigma_{D}^{2}=0$ | TLIML |  $-$1.72(0.01) | 1.73 | 0.00
| D2SLS |  $-$0.12(0.01) | 0.26 | 0.88
| DLIML |  $-$0.24(0.01) | 0.31 | 0.98
| Oracle |  $-$0.03(0.01) | 0.21 | 0.95
| Proposed (full) |  $-$0.07(0.01) | 0.27 | 0.93
| T2LS |  $-$1.83(0.00) | 1.84 | 0.00
$\sigma_{D}^{2}=2$ | TLIML |  $-$1.68(0.01) | 1.69 | 0.00
| D2SLS |  $-$0.19(0.01) | 0.31 | 0.79
| DLIML |  $-$0.45(0.01) | 0.50 | 0.77
| Oracle |  $-$0.03(0.01) | 0.21 | 0.95
| Proposed (full) |  $-$0.09(0.02) | 0.54 | 0.91
| T2LS |  $-$1.78(0.00) | 1.79 | 0.00
$\sigma_{D}^{2}=4$ | TLIML |  $-$1.64(0.01) | 1.65 | 0.00
| D2SLS |  $-$0.27(0.01) | 0.38 | 0.68
| DLIML |  $-$0.61(0.01) | 0.65 | 0.46
| Oracle |  $-$0.03(0.01) | 0.21 | 0.95

In summary, the “Proposed (full)” outperforms the other approaches for
estimating the causal effect and its performance is similar to the oracle
estimate when $\sigma_{D}^{2}$ is not too large.

##  Appendix G Estimate of precision matrix

Following van de Geer et al. (2014), to obtain the matrix
$\bm{M}\in\mathbb{R}^{s\times s}$ that estimates the precision matrix
$\bm{\Sigma}_{\widehat{\mathcal{S}}_{1}}^{-1}$, we first solve the node-wise
lasso problems below:

$\widehat{\bm{\theta}}_{j}^{(\widehat{\mathcal{S}}_{1})}=\mathop{\mathrm{argmin}}_{\bm{\theta}\in\mathbb{R}^{s}}\bigg{(}\frac{1}{n}\|\bm{Z}_{\widehat{\mathcal{S}}_{1},j}-\bm{\mathbb{Z}}_{\widehat{\mathcal{S}}_{1},-j}\bm{\theta}\|^{2}+2\lambda_{j}\|\bm{\theta}\|_{1}\bigg{)}$
(G.1)

for $j=1,\ldots,s$, where
$\bm{Z}_{\widehat{\mathcal{S}}_{1},j}\in\mathbb{R}^{n}$ denotes the $j$th
column of the design matrix
$\bm{\mathbb{Z}}_{\widehat{\mathcal{S}}_{1}}\in\mathbb{R}^{n\times s}$ and
$\bm{\mathbb{Z}}_{\widehat{\mathcal{S}}_{1},-j}\in\mathbb{R}^{n\times(s-1)}$
is formed by columns of $\bm{\mathbb{Z}}_{\widehat{\mathcal{S}}_{1}}$ indexed
by $\\{1,\ldots,s\\}\setminus\\{j\\}$. The $k$th component of
$\widehat{\bm{\theta}}_{j}^{(\widehat{\mathcal{S}}_{1})}$ is denoted by
$\\{\widehat{\theta}_{j,k}^{(\widehat{\mathcal{S}}_{1})}:k=1,\ldots,s,k\neq
j\\}$. Define matrix $\widehat{\bm{N}}$ as

$\widehat{\bm{N}}=\begin{pmatrix}1&-\widehat{\theta}_{1,2}^{(\widehat{\mathcal{S}}_{1})}&\cdots&-\widehat{\theta}_{1,s}^{(\widehat{\mathcal{S}}_{1})}\\\
-\widehat{\theta}_{2,1}^{(\widehat{\mathcal{S}}_{1})}&1&\cdots&-\widehat{\theta}_{2,s}^{(\widehat{\mathcal{S}}_{1})}\\\
\vdots&\vdots&\ddots&\vdots\\\
-\widehat{\theta}_{s,1}^{(\widehat{\mathcal{S}}_{1})}&-\widehat{\theta}_{s,2}^{(\widehat{\mathcal{S}}_{1})}&\cdots&1\end{pmatrix}.$
(G.2)

Let
$\widehat{\bm{T}}^{2}=\text{diag}\\{\widehat{\tau}_{\widehat{\mathcal{S}}_{1},1}^{2},\ldots,\widehat{\tau}_{\widehat{\mathcal{S}}_{1},s}^{2}\\}$,
where
$\widehat{\tau}_{\widehat{\mathcal{S}}_{1},j}^{2}=\|\bm{Z}_{\widehat{\mathcal{S}}_{1},j}-\bm{\mathbb{Z}}_{\widehat{\mathcal{S}}_{1},-j}\widehat{\bm{\theta}}_{j}^{(\widehat{\mathcal{S}}_{1})}\|^{2}/n+\lambda_{j}\|\widehat{\bm{\theta}}_{j}^{(\widehat{\mathcal{S}}_{1})}\|_{1}$.
Then $\bm{M}$ takes the following form

$\bm{M}=\widehat{\bm{T}}^{-2}\widehat{\bm{N}}.$ (G.3)

##  Appendix H Standard error of bias estimate

The standard error of bias estimate is given by

$\text{SE}\Big{(}\widehat{b}^{(j,l)}\Big{)}=\sqrt{(\widehat{v}_{jj}-2\widehat{v}_{jl}+\widehat{v}_{ll})/n},$
(H.1)

where

$\displaystyle\widehat{v}_{jj}=\frac{\bm{M}_{jj}}{\big{\\{}\widehat{\gamma}_{\check{\mathcal{S}}_{1},j}\big{\\}}^{2}}\bigg{\\{}\widehat{\Theta}_{11}-\frac{2\widehat{\Gamma}_{\check{\mathcal{S}}_{1},j}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},j}}\widehat{\Theta}_{12}+\bigg{(}\frac{\widehat{\Gamma}_{\check{\mathcal{S}}_{1},j}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},j}}\bigg{)}^{2}\widehat{\Theta}_{22}\bigg{\\}}$
(H.2)
$\displaystyle\widehat{v}_{ll}=\frac{\bm{M}_{ll}}{\big{\\{}\widehat{\gamma}_{\check{\mathcal{S}}_{1},l}\big{\\}}^{2}}\bigg{\\{}\widehat{\Theta}_{11}-\frac{2\widehat{\Gamma}_{\check{\mathcal{S}}_{1},l}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},l}}\widehat{\Theta}_{12}+\bigg{(}\frac{\widehat{\Gamma}_{\check{\mathcal{S}}_{1},l}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},l}}\bigg{)}^{2}\widehat{\Theta}_{22}\bigg{\\}}$
(H.3)
$\displaystyle\widehat{v}_{jl}=\frac{\bm{M}_{jl}}{\big{\\{}\widehat{\gamma}_{\check{\mathcal{S}}_{1},j}\big{\\}}\big{\\{}\widehat{\gamma}_{\check{\mathcal{S}}_{1},l}\big{\\}}}\bigg{\\{}\widehat{\Theta}_{11}-\bigg{(}\frac{\widehat{\Gamma}_{\check{\mathcal{S}}_{1},j}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},j}}+\frac{\widehat{\Gamma}_{\check{\mathcal{S}}_{1},l}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},l}}\bigg{)}\widehat{\Theta}_{12}+\bigg{(}\frac{\widehat{\Gamma}_{\check{\mathcal{S}}_{1},j}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},j}}\bigg{)}\bigg{(}\frac{\widehat{\Gamma}_{\check{\mathcal{S}}_{1},l}}{\widehat{\gamma}_{\check{\mathcal{S}}_{1},l}}\bigg{)}\widehat{\Theta}_{22}\bigg{\\}}$
(H.4)

and $\widehat{\Theta}_{11},\widehat{\Theta}_{12},\widehat{\Theta}_{22}$ take
the following form

$\widehat{\Theta}_{11}=\frac{1}{n}\Big{\|}\bm{\mathrm{Y}}-\widetilde{\bm{\mathbb{Z}}}_{\check{\mathcal{S}}_{1}}\widetilde{\bm{\Gamma}}_{\check{\mathcal{S}}_{1}}\Big{\|}_{2}^{2},\quad\widehat{\Theta}_{22}=\frac{1}{n}\Big{\|}\bm{\mathrm{D}}-\widetilde{\bm{\mathbb{Z}}}_{\check{\mathcal{S}}_{1}}\widetilde{\bm{\gamma}}_{\check{\mathcal{S}}_{1}}\Big{\|}_{2}^{2},\quad\widehat{\Theta}_{12}=\frac{1}{n}(\bm{\mathrm{Y}}-\widetilde{\bm{\mathbb{Z}}}_{\check{\mathcal{S}}_{1}}\widetilde{\bm{\Gamma}}_{\check{\mathcal{S}}_{1}})^{\mathrm{\scriptstyle
T}}(\bm{\mathrm{D}}-\widetilde{\bm{\mathbb{Z}}}_{\check{\mathcal{S}}_{1}}\widetilde{\bm{\gamma}}_{\check{\mathcal{S}}_{1}}).$