primal program is feasible for all 5 folds of the data. We call it the “min-
feas” rule, which would lead to the most conservative confidence interval for
our debiasing method.

We recommend the final value of $\gamma$ to be chosen as any value between the
“min-feas” and “1SE” rules, depending how conservative that we need for the
subsequent inference from our proposed debiasing program; recall from Section
4.2 for our simulation results.

##  Appendix B Sufficient Conditions for Assumption 6

We present several sufficient conditions in Lemma 1 under which Assumption 6
on the sparse approximation to the population dual solution
$\ell_{0}(x)\in\mathbb{R}^{d}$ in Definition 1 holds and illuminate these
sufficient conditions through some motivating examples.

######  Lemma 1 (Sufficient conditions for the sparse approximation to
$\ell_{0}(x)$).

Under Assumption 4, we let
$\left[\mathrm{E}(RXX^{T})\right]^{-1}:=M=(\bm{m}_{1},...,\bm{m}_{d})\in\mathbb{R}^{d\times
d}$, where $\bm{m}_{k},k=1,...,d$ are columns of the matrix
$M\in\mathbb{R}^{d\times d}$. We also denote by
$M_{S,T}\in\mathbb{R}^{|S|\times|T|}$ the submatrix of $M$ indexed by
$S,T\subset\\{1,...,d\\}$ and let $\sigma_{\min}(M_{S,T})$ be its smallest
singular value.

  1. (a)

If each column of $M$ has at most $m\geq 1$ nonzero entries and the query
vector $x\in\mathbb{R}^{d}$ has at most $s_{x}\geq 1$ nonzero entries, then
Assumption 6 holds with $s_{\ell}(x)=m\cdot s_{x}$ and $r_{\ell}=0$ for all
$n\geq 1$.

  2. (b)

Suppose that each column $\bm{m}_{k}$ of $M$ lies in
$\mathcal{C}_{1}(J_{k},\vartheta)$ for some $J_{k}\subset\\{1,...,d\\}$ and
$\vartheta\in[0,\infty)$. If the query vector $x\in\mathbb{R}^{d}$ has its
support set as $\mathrm{support}(x)=S_{x}$ of size at most $s_{x}>0$ and an
index set $J\subset\\{1,...,d\\}$ satisfies $\sigma_{\min}(M_{J,S_{x}})>0$,
then Assumption 6 holds with $s_{\ell}(x)=|J|\cdot R_{\ell}(n)$ and
$r_{\ell}=O\left(\frac{(1+\vartheta)\cdot K_{J,x}}{\sqrt{R_{\ell}(n)}}\right)$
for some $R_{\ell}(n)\to\infty$ as $n\to\infty$, where $K_{J,x}=\max_{k\in
S_{x}}\frac{\sqrt{s_{x}}\left|\left|M_{J,k}\right|\right|_{1}}{\sigma_{\min}(M_{J,S_{x}})}$.

  3. (c)

Suppose that each column $\bm{m}_{k}$ of $M$ lies in
$\mathcal{C}_{1}(J_{k},\vartheta)$ for some $J_{k}\subset\\{1,...,d\\}$ and
$\vartheta\in[0,\infty)$. If the query vector $x\in\mathbb{R}^{d}$ lies in
$\mathcal{C}_{1}(J,\vartheta_{x})$ with the index set $J\subset\\{1,...,d\\}$
satisfying that $\kappa_{\min}(J,\vartheta_{x})>0$ , then Assumption 6 holds
with $s_{\ell}(x)=|J|\cdot R_{\ell}(n)$ and
$r_{\ell}=O\left(\frac{(1+\vartheta)K_{J,x}}{\sqrt{R_{\ell}(n)}}\right)$ for
some $R_{\ell}(n)\to\infty$ as $n\to\infty$, where
$K_{J,x}=(1+\vartheta_{x})\max\limits_{1\leq k\leq
d}\frac{\sqrt{|J|}\cdot\left|\left|M_{J,k}\right|\right|_{1}}{\kappa_{\min}(J,\vartheta_{x})}$
with
$\kappa_{\min}(J,\vartheta_{x})=\inf\limits_{u\in\mathcal{C}_{1}(J,\vartheta_{x})}\frac{\sqrt{|J|}\cdot\left|\left|M_{J}u\right|\right|_{2}}{\left|\left|u_{J}\right|\right|_{1}}$
being the compatibility constant of the submatrix $M_{J}$.

###### Proof of Lemma 1.

The proof of Lemma 1 are inspired and modified from Lemma 4 in Giessing and
Wang (2021).

(a) Let $S_{x}=\mathrm{support}(x)\subset\\{1,...,d\\}$ for the fixed query
vector $x\in\mathbb{R}^{d}$. Given that $\max_{1\leq k\leq
d}\left|\left|M_{k}\right|\right|_{0}\leq m$ and
$\ell_{0}(x)=-2\left[\mathrm{E}\left(RXX^{T}\right)\right]^{-1}x\in\mathbb{R}^{d}$,
we compute that

$\displaystyle\left|\left|\ell_{0}(x)\right|\right|_{0}=\left|\left|Mx\right|\right|_{0}=\left|\left|\sum_{k=1}^{d}\bm{m}_{k}x_{k}\right|\right|_{0}=\left|\left|\sum_{k\in
S_{x}}\bm{m}_{k}x_{k}\right|\right|_{0}\leq\sum_{k\in
S_{x}}\left|\left|\bm{m}_{k}\right|\right|_{0}\leq m\cdot s_{x}.$

Thus, $\ell_{0}(x)$ by itself is sparse and we can take
$\widetilde{\ell}(x)=\ell_{0}(x)$ in Assumption 6 with $s_{\ell}(x)=m\cdot
s_{x}$ and $r_{\ell}=0$ for all $n\geq 1$.

(b) Since each column of $M$ lies in $\mathcal{C}_{1}(J_{k},\vartheta)$ for
some $J_{k}\subset\\{1,...,d\\}$ with $|J_{k}|=|J|$ and
$\vartheta\in[0,\infty)$, we know that
$\left|\left|M_{S_{x}^{c},j}\right|\right|_{1}\leq(1+\vartheta)\left|\left|M_{S_{x},j}\right|\right|_{1}$
for all $j\in S_{x}$. Moreover,
$\sigma_{\min}(M_{J,S_{x}})=\inf_{\mathrm{support}(u)=S_{x}}\frac{\left|\left|M_{J}u\right|\right|_{2}}{\left|\left|u\right|\right|_{2}}>0$
by our assumption. Now, we compute that

$\displaystyle\left|\left|\left[\ell_{0}(x)\right]_{J^{c}}\right|\right|_{1}$
$\displaystyle=2\left|\left|\left[Mx\right]_{J^{c}}\right|\right|_{1}$
$\displaystyle=2\left|\left|\sum_{k=1}^{d}M_{J^{c},k}\cdot
x_{k}\right|\right|_{1}$ $\displaystyle\leq 2\sum_{k\in
S_{x}}\left|\left|M_{J^{c},k}\right|\right|_{1}|x_{k}|$ $\displaystyle\leq
2\sum_{k\in S_{x}}\left(\left|\left|M_{J^{c}\cap
J_{k}^{c},k}\right|\right|_{1}+\left|\left|M_{J^{c}\cap
J_{k},k}\right|\right|_{1}\right)|x_{k}|$ $\displaystyle\leq 2\sum_{k\in
S_{x}}\left(\left|\left|M_{J_{k}^{c},k}\right|\right|_{1}+\left|\left|M_{J_{k},k}\right|\right|_{1}\right)|x_{k}|$
$\displaystyle\leq 2(1+\vartheta)\sum_{k\in
S_{x}}\left|\left|M_{J_{k},k}\right|\right|_{1}|x_{k}|$ $\displaystyle\leq
2(1+\vartheta)\sqrt{s_{x}}\cdot\max_{k\in
S_{x}}\left|\left|M_{J_{k},k}\right|\right|_{1}\left|\left|x\right|\right|_{2}$
$\displaystyle\leq 2(1+\vartheta)\sqrt{s_{x}}\cdot\max_{k\in
S_{x}}\left|\left|M_{J_{k},k}\right|\right|_{1}\left|\left|(Mx)_{J}\right|\right|_{2}\sup_{\mathrm{support}(u)=S_{x}}\frac{\left|\left|u\right|\right|_{2}}{\left|\left|(Mu)_{J}\right|\right|_{2}}$
$\displaystyle\leq\frac{2(1+\vartheta)\sqrt{s_{x}}\max_{k\in
S_{x}}\left|\left|M_{J,k}\right|\right|_{1}}{\sigma_{\min}(M_{J,S_{x}})}\cdot\left|\left|(Mx)_{J}\right|\right|_{1}$
$\displaystyle\equiv(1+\vartheta)K_{J,x}\left|\left|\left[\ell_{0}(x)\right]_{S_{x}}\right|\right|_{1},$

showing that
$\ell_{0}(x)\in\mathcal{C}_{1}\left(J,(1+\vartheta)K_{J,x}\right)$. By Theorem
2.5 in Foucart and Rauhut (2013), there exists a $s_{\ell}(x)$-sparse
approximation $\widetilde{\ell}(x)\in\mathbb{R}^{d}$ to the population dual
solution $\ell_{0}(x)$ such that

$\left|\left|\widetilde{\ell}(x)-\ell_{0}(x)\right|\right|_{2}\leq\frac{1}{2\sqrt{s_{\ell}(x)}}\left|\left|\ell_{0}(x)\right|\right|_{1}.$

Combining with
$\ell_{0}(x)\in\mathcal{C}_{1}\left(J,(1+\vartheta)K_{J,x}\right)$, we have
that

$\displaystyle\left|\left|\widetilde{\ell}(x)-\ell_{0}(x)\right|\right|_{2}$
$\displaystyle\leq\frac{1}{2\sqrt{s_{\ell}(x)}}\left[\left|\left|\left[\ell_{0}(x)\right]_{J}\right|\right|_{1}+\left|\left|\left[\ell_{0}(x)\right]_{J^{c}}\right|\right|_{1}\right]$
$\displaystyle\leq\frac{1+(1+\vartheta)K_{J,x}}{2\sqrt{s_{\ell}(x)}}\cdot\left|\left|\left[\ell_{0}(x)\right]_{J}\right|\right|_{1}$
$\displaystyle\leq\frac{1+(1+\vartheta)K_{J,x}}{2}\sqrt{\frac{|J|}{s_{\ell}(x)}}\cdot\left|\left|\ell_{0}(x)\right|\right|_{2}.$

Thus, when $s_{\ell}(x)=|J|\cdot R_{\ell}(n)$, it follows that

$\left|\left|\widetilde{\ell}(x)-\ell_{0}(x)\right|\right|_{2}\leq\frac{1+(1+\vartheta)K_{J,x}}{2\sqrt{R_{\ell}(n)}}\cdot\left|\left|\ell_{0}(x)\right|\right|_{2},$

so we take
$r_{\ell}=O\left(\frac{(1+\vartheta)K(J,x)}{\sqrt{R_{\ell}(n)}}\right)$ to
fulfill Assumption 6.

(c) Since $x\in\mathcal{C}_{1}(J,\vartheta_{x})$, it follows that
$\left|\left|x_{J^{c}}\right|\right|_{1}\leq\vartheta_{x}\left|\left|x_{J}\right|\right|_{1}$.
Now, we compute that

$\displaystyle\left|\left|\left[\ell_{0}(x)\right]_{J^{c}}\right|\right|_{1}$
$\displaystyle=2\left|\left|\left(Mx\right)_{J^{c}}\right|\right|_{1}$
$\displaystyle=2\left|\left|\sum_{k=1}^{d}M_{J^{c},k}x_{k}\right|\right|_{1}$
$\displaystyle\leq 2\max_{1\leq k\leq
d}\left|\left|M_{J^{c},k}\right|\right|_{1}\left|\left|x\right|\right|_{1}$
$\displaystyle\leq 2(1+\vartheta_{x})\max_{1\leq k\leq
d}\left|\left|M_{J^{c},k}\right|\right|_{1}\left|\left|x_{J}\right|\right|_{1}$
$\displaystyle\leq 2(1+\vartheta_{x})\max_{1\leq k\leq
d}\left|\left|M_{J^{c},k}\right|\right|_{1}\left|\left|(Mx)_{J}\right|\right|_{2}\sup_{u\in\mathcal{C}_{1}(J,a_{0})}\frac{\left|\left|u_{J}\right|\right|_{1}}{\left|\left|(Mu)_{J}\right|\right|_{2}}$
$\displaystyle\leq 2(1+\vartheta)(1+\vartheta_{x})\max_{1\leq k\leq
d}\left|\left|M_{J_{k},k}\right|\right|_{1}\left|\left|(Mx)_{J}\right|\right|_{2}\sup_{u\in\mathcal{C}_{1}(J,\vartheta_{x})}\frac{\left|\left|u_{J}\right|\right|_{1}}{\left|\left|M_{J}u\right|\right|_{2}}$
$\displaystyle=2(1+\vartheta)(1+\vartheta_{x})\max_{1\leq k\leq
d}\frac{\sqrt{|J|}\cdot\left|\left|M_{J_{k},k}\right|\right|_{1}}{\kappa_{\min}(J,\vartheta_{x})}\cdot\left|\left|\left(Mx\right)_{J}\right|\right|_{1}$
$\displaystyle\equiv(1+\vartheta)K_{J,x}\left|\left|\left[\ell_{0}(x)\right]_{J}\right|\right|_{1},$

showing that
$\ell_{0}(x)\in\mathcal{C}_{1}\left(J,(1+\vartheta)K_{J,x}\right)$. Therefore,
as in the proof of Part (b), we conclude that Assumption 6 holds with
$s_{\ell}(x)=|J|\cdot R_{\ell}(n)$ and
$r_{\ell}=O\left(\frac{(1+\vartheta)K_{J,x}}{\sqrt{R_{\ell}(n)}}\right)$. ∎

######  Remark 1.

The constants $\kappa_{\min}(M_{S_{x},S_{x}})$ and
$\kappa_{\min}(J,\vartheta_{x})$ in (b) and (c) of Lemma 1 are well-known in
the statistical literature. Sun and Zhang (2012, 2013) introduced the
eigenvalue version of $\sigma_{\min}(M_{J,S_{x}})>0$ to study the least-square
and precision matrix estimations after the scaled lasso selection. The
compatibility constant $\kappa_{\min}(J,\vartheta_{x})$ appears even more
commonly in the literature (see Section 6.2.2 in Bühlmann and van De Geer
2011), given that it is also related to our restricted eigenvalue condition
(16) as:

$\kappa_{\min}(J,a_{0})=\inf\limits_{u\in\mathcal{C}_{1}(J,\vartheta_{x})}\frac{\sqrt{|J|}\cdot\left|\left|M_{J}u\right|\right|_{2}}{\left|\left|u_{J}\right|\right|_{1}}\geq\inf\limits_{u\in\mathcal{C}_{1}(J,\vartheta_{x})}\frac{\left|\left|M_{J}u\right|\right|_{2}}{\left|\left|u\right|\right|_{2}}.$

Now, we illustrate these sufficient conditions in Lemma 1 through the linear
regression model (1) under both MCAR and MAR data settings and connect the
sparsity structure of
$\left[\mathrm{E}(RXX^{T})\right]^{-1}\in\mathbb{R}^{d\times d}$ with the
well-studied covariance matrix $\Sigma\in\mathbb{R}^{d\times d}$ of the
covariate vector $X\in\mathbb{R}^{d}$ in the statistical literature.

######  Example 1 (Linear regression model under the MCAR mechanism).

Recall the high-dimensional sparse linear regression model (1) in Assumption 1
as:

$Y=X^{T}\beta+\epsilon\quad\text{ with
}\quad\mathrm{E}(\epsilon|X)=0\quad\text{ and
}\quad\mathrm{E}\left(\epsilon^{2}|X\right)=\sigma_{\epsilon}^{2}.$

Furthermore, we suppose, without loss of generality, that the covariate vector
$X\in\mathbb{R}^{d}$ is centered in this example, _i.e._ , $\mathrm{E}(X)=0$.
In order to study the sparsity structure of

$\ell_{0}(x)=-2\left[\mathrm{E}(RXX^{T})\right]^{-1}x:=-2Mx,$

it suffices to exploit the (weak) sparsity of
$M=\left[\mathrm{E}(RXX^{T})\right]^{-1}\in\mathbb{R}^{d\times d}$ by Lemma 1
when the query point $x\in\mathbb{R}^{d}$ is sparse or lie in a cone of
dominant coordinates $\mathcal{C}_{1}\left(J,\vartheta_{x}\right)$. In
particular, under the MCAR setting, we know that
$R{\perp\\!\\!\\!\perp}(Y,X)$, and the propensity score satisfies
$\pi(X,Y)=\mathrm{P}(R=1|X,Y)=\mathrm{P}(R=1)$ so that

$M=\left[\mathrm{E}(RXX^{T})\right]^{-1}=\frac{1}{\mathrm{P}(R=1)}\cdot\Sigma^{-1}$

will be sparse if and only if the precision matrix $\Sigma^{-1}$ is sparse.
Examples of sparse precision matrices include the auto-regressive model of
order $q\geq 1$ (whose precision matrix is banded; see, _e.g._ , Section 9.2
in Lindsey 2004) and solutions to the graphical Lasso model (Friedman et al.,
2008) or other $L_{1}$-minimization programs (Cai et al., 2016). In addition,
as the sufficient conditions in Lemma 1(b,c) only require each column of
$\Sigma^{-1}$ to lie in $\mathcal{C}_{1}(J,\vartheta)$ for some
$J\subset\\{1,...,d\\}$, any moving-average process of order $q\geq 1$ (whose
precision matrix is Toeplitz; recall $\Sigma^{\mathrm{ar}}$ in Section 4.1)
and other short-range dependent processes (Rosenblatt, 2015) will have their
precision matrices fulfilling the requirements as well.

######  Example 2 (Linear regression model under the MAR mechanism).

Under the same linear model setup in Example 1, we consider the MAR setting
$R{\perp\\!\\!\\!\perp}Y|X$, in which the propensity score becomes
$\pi(X)=\mathrm{P}(R=1|X)$. We consider several scenarios that relate the
sparsity structures of
$M=\left[\mathrm{E}(RXX^{T})\right]^{-1}\in\mathbb{R}^{d\times d}$ to the
covariance matrix $\Sigma=\mathrm{E}(XX^{T})\in\mathbb{R}^{d\times d}$.

  1. 1.

Suppose that the propensity score $\pi(X)$ depends on at most $1\leq
s_{\pi}\ll d$ covariates in $X\in\mathbb{R}^{d}$, _i.e._ , without loss of
generality,

$\pi(x)=\mathrm{P}\left(R=1|X=(x_{1},...,x_{d})^{T}\right)=\pi(x_{1},...,x_{s_{\pi}}).$
(25)

We further assume that the first few covariates $X^{(1)},...,X^{(d_{1})}$ with
$d_{1}\geq s_{\pi}$ are independent of the rest covariates
$X^{(d_{1}+1)},...,X^{(d_{1}+d_{2})}$ in the covariate vector
$X=\left(X^{(1)},...,X^{(d)}\right)^{T}\in\mathbb{R}^{d}$ with
$d=d_{1}+d_{2}$. Then, the covariance matrix $\Sigma$ is block diagonal as
$\Sigma=\begin{pmatrix}\Sigma_{1}&\bm{0}\\\ \bm{0}&\Sigma_{2}\end{pmatrix}$,
where $\Sigma_{1}\in\mathbb{R}^{d_{1}\times d_{1}}$ and
$\Sigma_{2}\in\mathbb{R}^{d_{2}\times d_{2}}$. Under the sparsity assumption
(25) on the propensity score $\pi(X)$, we know that

$\mathrm{E}(RXX^{T})=\mathrm{E}\left(\pi(X)XX^{T}\right)=\begin{pmatrix}\widetilde{\Sigma}_{1}&\bm{0}\\\
\bm{0}&\Sigma_{2}\end{pmatrix}\quad\text{ and }\quad
M=\left[\mathrm{E}(RXX^{T})\right]^{-1}=\begin{pmatrix}\widetilde{\Sigma}_{1}^{-1}&\bm{0}\\\
\bm{0}&\Sigma_{2}^{-1}\end{pmatrix}.$

Thus, the sufficient condition (a) in Lemma 1 holds.

  2. 2.

More generally, the complete-case gram matrix
$\mathrm{E}(RXX^{T})=\begin{pmatrix}\widetilde{\Sigma}_{11}&\widetilde{\Sigma}_{12}\\\
\widetilde{\Sigma}_{21}&\widetilde{\Sigma}_{22}\end{pmatrix}$ has its inverse
as:

$\displaystyle M$ $\displaystyle=\left[\mathrm{E}(RXX^{T})\right]^{-1}$
$\displaystyle=\begin{pmatrix}\widetilde{\Sigma}_{11}^{-1}+\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{12}\left(\widetilde{\Sigma}_{22}-\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{22}\right)^{-1}\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}&-\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{12}\left(\widetilde{\Sigma}_{22}-\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{22}\right)^{-1}\\\
-\left(\widetilde{\Sigma}_{22}-\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{22}\right)^{-1}\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}&\left(\widetilde{\Sigma}_{22}-\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{22}\right)^{-1}\end{pmatrix}.$

If the entries of $\widetilde{\Sigma}_{21}=\widetilde{\Sigma}_{12}^{T}$ are
relatively small in their absolute values, which happens when, _e.g._ ,
$\Sigma=\mathrm{E}(XX^{T})$ is diagonally dominant (such as banded, Toeplitz,
or other sparse covariance matrices; recall $\Sigma^{\mathrm{cs}}$ and
$\Sigma^{\mathrm{ar}}$ in Section 4.1) together with (25), then the off-
diagonal block matrices of $M=\left[\mathrm{E}(RXX^{T})\right]^{-1}$ given by

$-\left(\widetilde{\Sigma}_{22}-\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{22}\right)^{-1}\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}=-\left[\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{12}\left(\widetilde{\Sigma}_{22}-\widetilde{\Sigma}_{21}\widetilde{\Sigma}_{11}^{-1}\widetilde{\Sigma}_{22}\right)^{-1}\right]^{T}$

are also small in their absolute values compared with the diagonal block
matrices of $M$ and therefore, the sufficient conditions (b) and (c) in Lemma
1 apply.

##  Appendix C Generalized Linear Model with Lasso Regularization for
Propensity Score Estimation

Recall that the consistency and asymptotic normality of our proposed debiasing
method holds as long as the propensity scores are consistently estimated under
a mild rate of convergence (20). While it is not necessary to estimate the
propensity scores via parametric models for our debiasing method (see Section
4.4 for our simulation results), the generalized linear model always serves as
a baseline approach and can approximate many complex nonparametric models by
including as many (well-designed) covariates as possible. As a result, the
covariate vector is often high-dimensional with $d\gg n$ in practice, and we
consider estimating the propensity score $\pi(x)=\mathrm{P}(R=1|X=x)$ under
the MAR assumption (Assumption 1) with the data
$\\{(X_{i},R_{i})\\}_{i=1}^{n}\subset\mathbb{R}^{d}\times\\{0,1\\}$ by the
Lasso-type generalized linear regression defined as:

$\displaystyle\begin{split}\widehat{\theta}&=\operatorname*{arg\,min}_{\theta\in\mathbb{R}^{d}}\left\\{-\frac{1}{n}\sum_{i=1}^{n}R_{i}\log\left[\frac{\varphi(X_{i}^{T}\theta)}{1-\varphi(X_{i}^{T}\theta)}\right]-\frac{1}{n}\sum_{i=1}^{n}\log\left[1-\varphi(X_{i}^{T}\theta)\right]+\zeta\left|\left|\theta\right|\right|_{1}\right\\}\\\
&:=\operatorname*{arg\,min}_{\theta\in\mathbb{R}^{d}}\left[\mathcal{L}_{\varphi}(\theta)+\zeta\left|\left|\theta\right|\right|_{1}\right],\end{split}$
(26)

where $\mathcal{L}_{\varphi}:\mathbb{R}^{d}\to\mathbb{R}$ is the negative log-
likelihood function associated with a known link function
$\varphi:\mathbb{R}\to(0,1)$ and $\zeta>0$ is a regularization parameter. In
particular, when the logistic link function
$\varphi_{\mathrm{log}}(u)=\frac{1}{1+e^{-u}}$ is applied, the above
regression problem (26) reduces to the Lasso-type logistic regression as:

$\widehat{\theta}=\operatorname*{arg\,min}_{\theta\in\mathbb{R}^{d}}\left\\{-\frac{1}{n}\sum_{i=1}^{n}\left[R_{i}X_{i}^{T}\theta-\log\left(1+e^{X_{i}^{T}\theta}\right)\right]+\zeta\left|\left|\theta\right|\right|_{1}\right\\}.$
(27)

With the estimate $\widehat{\theta}$ from the Lasso-type generalized linear
regression (26), it is also feasible to derive an explicit formula of
$r_{\pi}=r_{\pi}(n,\delta)$ in Assumption 5 for the estimated propensity
scores $\widehat{\pi}_{i},i=1,...,n$ under the following regularity
conditions.

######  Assumption 7 (Sparse generalized linear propensity score model).

Under Assumption 1, the missingness variable $R\in\\{0,1\\}$ is generated from
the following model as:

$R|X\sim\mathrm{Bernoulli}\left(\varphi(X^{T}\theta_{0})\right),$ (28)

where $\varphi:\mathbb{R}\to(0,1)$ is a known link function and
$\theta_{0}\in\mathbb{R}^{d}$ is a high-dimensional sparse vector with support
$S_{\theta}$ and
$|S_{\theta}|=\left|\left|\theta_{0}\right|\right|_{0}=s_{\theta}\ll d$.

######  Assumption 8 ($\mathcal{C}_{1}(S_{\theta},3)$-restricted eigenvalue
condition on $\mathrm{E}(RXX^{T})$).

The $\mathcal{C}_{1}(S_{\theta},3)$-restricted minimum eigenvalue of the
complete-case population gram matrix
$\mathrm{E}\left(RXX^{T}\right)\in\mathbb{R}^{d\times d}$ is bounded away from
zero, _i.e._ , there exists an absolute constant $\rho>0$ such that

$\mathrm{E}\left[R(X^{T}v)^{2}\right]>\rho\left|\left|v\right|\right|_{2}^{2}\quad\text{
for any }\quad v\in\mathcal{C}_{1}(S_{\theta},3),$

where $S_{\theta}$ is the support of the $s_{\theta}$-sparse vector
$\theta_{0}\in\mathbb{R}^{d}$.

######  Assumption 9 (Regular link function).

The link function $\varphi$ satisfies the following regularity conditions.

  1. (a)

$\varphi:\mathbb{R}\to[0,1]$ is twice differentiable, nondecreasing, and
$L_{\varphi}$-Lipschitz on $\mathbb{R}$ as well as concave on $\mathbb{R}^{+}$
for some absolute constant $L_{\varphi}>0$; also, for any $u\in\mathbb{R}$, it
holds that $\varphi(x)+\varphi(-x)=1$.

  2. (b)

There exists an absolute constant $A_{\varphi}>0$ such that
$\frac{\varphi^{\prime}(u)}{1-\varphi(u)}\leq A_{\varphi}$ for all $u\geq 0$.

  3. (c)

For $\mathcal{L}_{\varphi}(\theta)$ defined in (26), there exists an absolute
constant $A_{H}>1$ such that the Hessian matrix
$\nabla\nabla\mathcal{L}_{\varphi}(\theta)$ can be expressed as
$\nabla\nabla\mathcal{L}_{\varphi}(\theta)=\frac{1}{n}\sum_{i=1}^{n}H(\theta;R_{i},X_{i})X_{i}X_{i}^{T}$
for some $H(\theta;R_{i},X_{i})>0$ satisfying

$\left|\log H(\theta+b;R_{i},X_{i})-\log H(\theta;R_{i},X_{i})\right|\leq
A_{H}\left(|X_{i}^{T}\theta|^{2}+|X_{i}^{T}b|^{2}+|X_{i}^{T}b|\right)$

for any $\theta,b\in\mathbb{R}^{d}$ and $i=1,...,n$.

The sparse generalized linear regression setup in Assumption 7 is commonly
imposed in the statistical literature to handle high-dimensional data (van de
Geer, 2008; Huang and Zhang, 2012; Cai et al., 2021). Assumption 8 is the
restricted eigenvalue condition over the cone $\mathcal{C}_{1}(S_{\theta},3)$;
recall (16). In Assumption 9, the constants $A_{\varphi},A_{H}>0$ only depend
on the link function $\varphi$. A similar condition is also imposed in Cai et
al. (2021) to establish the consistency of $\widehat{\theta}$ in (26) and
derive the asymptotic normality of their debiased estimator for
$\widehat{\theta}$. One can verify by direct calculations and Example 8 in
Huang and Zhang (2012) that the logistic link function
$\varphi_{\mathrm{log}}(u)=\frac{1}{1+e^{-u}}$ satisfies Assumption 9.
Furthermore, Section 3 of the supplement in Cai et al. (2021) also verifies
Assumption 9 for the probit link function
$\varphi_{\mathrm{probit}}(u)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{u}e^{-\frac{t^{2}}{2}}dt$,
the class of generalized logistic functions, as well as the CDFs of Student’s
$t_{\nu}$-distributions with any $\nu\in\mathbb{N}$ in their latent variable
model (Example 3 in their supplement).

######  Remark 1.

Assumption 9(b) implies that
$\max\left\\{\frac{\varphi^{\prime}(u)}{\varphi(u)},\frac{\varphi^{\prime}(u)}{1-\varphi(u)}\right\\}\leq
A_{\varphi}$ for the same constant $A_{\varphi}>0$. This is because if $u\geq
0$, then by Assumption 9(a), $\varphi(u)\geq\frac{1}{2}$ and consequently,

$\frac{\varphi^{\prime}(u)}{\varphi(u)}\leq\frac{\varphi^{\prime}(u)}{1-\varphi(u)}\leq
A_{\varphi}.$

On the other hand, if $u<0$, then Assumption 9(a) also shows that
$\varphi(u)\leq\frac{1}{2}$ and therefore,

$\frac{\varphi^{\prime}(u)}{1-\varphi(u)}\leq\frac{\varphi^{\prime}(u)}{\varphi(u)}=\frac{\varphi^{\prime}(-u)}{1-\varphi(-u)}\leq
A_{\varphi},$

where we use the condition that $\varphi(x)+\varphi(-x)=1$ and
$\varphi^{\prime}(x)-\varphi^{\prime}(-x)=0$ in Assumption 9(a).

As for Assumption 9(c), some algebra show that

$H(\theta;R_{i},X_{i})=-\frac{R_{i}\left[\varphi^{\prime\prime}(X_{i}^{T}\theta)\varphi(X_{i}^{T}\theta)-\left(\varphi^{\prime}(X_{i}^{T}\theta)\right)^{2}\right]}{\left[\varphi(X_{i}^{T}\theta)\right]^{2}}+\frac{(1-R_{i})\left[\varphi^{\prime\prime}(X_{i}^{T}\theta)\left(1-\varphi(X_{i}^{T}\theta)\right)+\left(\varphi^{\prime}(X_{i}^{T}\theta)\right)^{2}\right]}{\left[1-\varphi(X_{i}^{T}\theta)\right]^{2}}.$

Hence, a sufficient condition for $H(\theta;R_{i},X_{i})>0$ is that

$\left[\varphi^{\prime}(X_{i}^{T}\theta)\right]^{2}>\max\Big{\\{}\varphi^{\prime\prime}(X_{i}^{T}\theta)\varphi(X_{i}^{T}\theta),\varphi^{\prime\prime}(X_{i}^{T}\theta)\left[\varphi(X_{i}^{T}\theta)-1\right]\Big{\\}},$
(29)

which is also equivalent to
$\left[\varphi^{\prime}(X_{i}^{T}\theta)\right]^{2}>\varphi^{\prime\prime}(X_{i}^{T}\theta)\varphi(X_{i}^{T}\theta)$
for all $X_{i}^{T}\theta<0$. This condition can be easily verified for the
logistic link function $\varphi_{\mathrm{log}}(u)=\frac{1}{1+e^{-u}}$ and the
probit link function
$\varphi_{\mathrm{probit}}(u)=\frac{1}{\sqrt{2\pi}}\int_{-\infty}^{u}e^{-\frac{t^{2}}{2}}dt$.

Under the above regularity conditions, we derive the explicit formula for
$r_{\pi}=r_{\pi}(n,\delta)$ as follows.

######  Proposition 1.

Let $\delta\in(0,1)$ and $A_{1},A_{2},A_{3},A_{4}>0$ be some absolute
constants. Suppose that Assumptions 2, 7, 8, and 9 hold as well as

$d\geq s_{\theta}+2,\quad
A_{1}\phi_{s_{\theta}}\log\left(\frac{\phi_{s_{\theta}}}{\rho}\right)\sqrt{\frac{\phi_{1}s_{\theta}\log
d}{n}}\leq\frac{\rho}{8},\;\text{ and
}\;A_{2}\exp\left[-\frac{n\rho^{2}}{\phi_{s_{\theta}}^{2}\log^{2}\left(\frac{\phi_{s_{\theta}}}{\rho}\right)}-\phi_{1}s_{\theta}\log
d\right]\leq\delta.$

If $\zeta=A_{3}A_{\varphi}\sqrt{\frac{\phi_{1}\log(d/\delta)}{n}}$, then the
rate of convergence $r_{\pi}=r_{\pi}(n,\delta)$ in Assumption 5 can be taken
as:

$r_{\pi}(n,\delta)=\frac{A_{4}L_{\varphi}A_{\varphi}}{\rho}\sqrt{\frac{\phi_{1}\phi_{s_{\theta}}s_{\theta}\log(d/\delta)}{n}\left[\log\left(\frac{n}{\delta}\right)+s_{\theta}\log\left(\frac{ed}{s_{\theta}}\right)\right]}.$

The proof of Proposition 1 is in Section E.8, which relies on the consistency
of $\left|\left|\widehat{\theta}-\theta_{0}\right|\right|_{2}$ obtained from
the Lasso-type generalized linear regression (26) as in Lemma 2. It suggests
that the rate of convergence for $r_{\pi}$ is of the order
$O_{P}\left(\frac{s_{\theta}\log d}{\sqrt{n}}\right)$, provided that all other
model parameters $L_{\varphi},A_{\varphi},\rho,\phi_{1},\phi_{s_{\theta}}>0$
are independent of $d$ and $n$ with $\delta=\frac{1}{d}$ and $d>n$.

##  Appendix D Additional Simulation Results

We have carried out simulations in Section 4 of the main paper that compares
our proposed debiasing inference method with those standard methods in the
literature under various data generating processes and the MAR settings. Here,
we supplement the full records of those simulation results mentioned in the
main paper. Additionally, we conduct comparative experiments under the simpler
MCAR setting for completeness and interpretations of Example 1.

###  D.1 Full Comparative Simulation Results

Here is an outline for all the comparative simulation results in this section.

  * •

Table 1 presents the simulation results under the circulant symmetric
covariance matrix $\Sigma^{\mathrm{cs}}$, Gaussian noise $\mathcal{N}(0,1)$,
and the MAR setting (22) in Section 4.1.

  * •

Table 2 presents the simulation results under the Toeplitz (or auto-
regressive) covariance matrix $\Sigma^{\mathrm{ar}}$, Gaussian noise
$\mathcal{N}(0,1)$, and the MAR setting (22).

  * •

Table 3 presents the simulation results under the circulant symmetric
covariance matrix $\Sigma^{\mathrm{cs}}$,
$\mathrm{Laplace}\left(0,\frac{1}{\sqrt{2}}\right)$ distributed noise, and the
MAR setting (22).

  * •

Table 4 presents the simulation results under the circulant symmetric
covariance matrix $\Sigma^{\mathrm{cs}}$, $t_{2}$ distributed noise, and the
MAR setting (22).

  * •

Table 5 presents the simulation results under the Toeplitz (or auto-
regressive) covariance matrix $\Sigma^{\mathrm{ar}}$,
$\mathrm{Laplace}\left(0,\frac{1}{\sqrt{2}}\right)$ distributed noise, and the
MAR setting (22).

  * •

Table 6 presents the simulation results under the Toeplitz (or auto-
regressive) covariance matrix $\Sigma^{\mathrm{ar}}$, $t_{2}$ distributed
noise, and the MAR setting (22).

  * •

Table 7 presents the simulation results for our proposed debiasing method when
the propensity scores are estimated by various machine learning methods under
the circulant symmetric covariance matrix $\Sigma^{\mathrm{cs}}$, Gaussian
noises $\mathcal{N}(0,1)$, and the new MAR setting (23) in Section 4.4.

  * •

Figure 9 supplements the “QQ-plots” corresponding to the asymptotic normality
plots for our proposed debiasing method in Figure 6 and Figure 7 when the
propensity scores are estimated by various machine learning methods under the
circulant symmetric covariance matrix $\Sigma^{\mathrm{cs}}$, Gaussian noises
$\mathcal{N}(0,1)$, and the new MAR setting (23).

(a) Simulation results with dense $\beta_{0}^{de}$ and sparse $x^{(2)}$.

(b) Simulation results with sparse $\beta_{0}^{sp}$ and dense $x^{(4)}$.

Figure 9: “QQ-plots” of the transformed studentized debiased estimators
$\Phi\left(\sqrt{n}\cdot\widehat{\sigma}_{n}^{-1}(x)\left[\widehat{m}(x)-m_{0}(x)\right]\right)$
obtained from our debiasing method when the propensity scores are estimated by
various machine learning methods under the new MAR setting (23), where
$\Phi(\cdot)$ is the CDF of $\mathcal{N}(0,1)$.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| DL-vdG
---
(CC)
| DL-vdG
---
(IPW)
| DL-vdG
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.033 | 0.035 | 0.028 | 0.122 | 0.058 | 0.304 | 0.249 | 0.198 | 0.315 | 0.032 | 0.038 | 0.028 | 0.042 | 0.039 | 0.038
Avg-Cov | 0.966 | 0.967 | 0.962 | 0.882 | 0.882 | 0.894 | 0.962 | 0.958 | 0.940 | 0.946 | 0.889 | 0.948 | 0.852 | 0.898 | 0.914
Avg-Len | 0.176 | 0.184 | 0.145 | 0.267 | 0.181 | 0.200 | 0.369 | 0.305 | 0.638 | 0.162 | 0.156 | 0.133 | 0.151 | 0.161 | 0.168
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.144 | 0.150 | 0.108 | 0.180 | 0.157 | 0.405 | 0.098 | 0.104 | 0.207 | 0.175 | 0.187 | 0.139 | 0.224 | 0.191 | 0.172
Avg-Cov | 0.994 | 0.989 | 0.999 | 0.902 | 0.864 | 0.908 | 1.00 | 0.998 | 0.996 | 0.925 | 0.835 | 0.916 | 0.859 | 0.931 | 0.969
Avg-Len | 0.980 | 0.993 | 0.844 | 0.759 | 0.595 | 0.618 | 1.253 | 1.042 | 1.784 | 0.789 | 0.663 | 0.608 | 0.819 | 0.871 | 0.908
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.044 | 0.048 | 0.035 | 0.069 | 0.052 | 0.045 | 0.068 | 0.057 | 0.088 | 0.048 | 0.050 | 0.037 | 0.066 | 0.059 | 0.055
Avg-Cov | 0.985 | 0.982 | 0.996 | 0.874 | 0.840 | 0.898 | 0.987 | 0.983 | 0.985 | 0.937 | 0.888 | 0.939 | 0.857 | 0.913 | 0.949
Avg-Len | 0.284 | 0.291 | 0.237 | 0.229 | 0.190 | 0.179 | 0.404 | 0.351 | 0.527 | 0.222 | 0.199 | 0.171 | 0.241 | 0.256 | 0.267
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.042 | 0.042 | 0.037 | 0.212 | 0.090 | 0.119 | NA | NA | NA | 0.036 | 0.043 | 0.030 | 0.039 | 0.039 | 0.042
Avg-Cov | 0.982 | 0.995 | 0.988 | 0.808 | 0.848 | 0.806 | NA | NA | NA | 0.956 | 0.892 | 0.955 | 0.929 | 0.937 | 0.938
Avg-Len | 0.263 | 0.312 | 0.236 | 0.243 | 0.241 | 0.193 | NA | NA | NA | 0.175 | 0.170 | 0.144 | 0.174 | 0.183 | 0.201
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.170 | 0.184 | 0.132 | 0.356 | 0.224 | 0.247 | NA | NA | NA | 0.245 | 0.280 | 0.179 | 0.217 | 0.207 | 0.183
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.750 | 0.830 | 0.810 | NA | NA | NA | 0.870 | 0.719 | 0.896 | 0.921 | 0.940 | 0.990
Avg-Len | 1.659 | 1.853 | 1.514 | 1.011 | 0.792 | 0.772 | NA | NA | NA | 0.931 | 0.787 | 0.725 | 0.939 | 0.989 | 1.089
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.054 | 0.056 | 0.046 | 0.108 | 0.078 | 0.080 | NA | NA | NA | 0.059 | 0.065 | 0.046 | 0.067 | 0.065 | 0.060
Avg-Cov | 0.999 | 1.00 | 1.00 | 0.730 | 0.810 | 0.788 | NA | NA | NA | 0.920 | 0.858 | 0.917 | 0.904 | 0.927 | 0.965
Avg-Len | 0.467 | 0.530 | 0.414 | 0.305 | 0.253 | 0.233 | NA | NA | NA | 0.266 | 0.239 | 0.205 | 0.277 | 0.291 | 0.321
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.025 | 0.025 | 0.022 | 0.753 | 0.603 | 0.349 | 0.740 | 0.556 | 0.739 | 0.00 | 0.001 | 0.00 | 0.031 | 0.033 | 0.035
Avg-Cov | 0.992 | 0.994 | 0.989 | 0.940 | 0.924 | 0.912 | 0.960 | 0.963 | 0.937 | 1.00 | 0.993 | 1.00 | 0.949 | 0.944 | 0.944
Avg-Len | 0.176 | 0.185 | 0.145 | 4.899 | 8.281 | 0.589 | 6.590 | 1.483 | 1.177 | 0.00 | 0.001 | 0.00 | 0.151 | 0.161 | 0.168
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.103 | 0.104 | 0.090 | 0.173 | 0.221 | 0.202 | 0.093 | 0.096 | 0.084 | 0.188 | 0.192 | 0.192 | 0.191 | 0.174 | 0.161
Avg-Cov | 0.998 | 0.998 | 0.999 | 0.916 | 0.886 | 0.910 | 1.00 | 0.997 | 0.997 | 0.001 | 0.056 | 0.007 | 0.901 | 0.948 | 0.972
Avg-Len | 0.981 | 0.994 | 0.844 | 0.760 | 0.865 | 0.596 | 1.256 | 1.073 | 1.728 | 0.013 | 0.057 | 0.025 | 0.819 | 0.870 | 0.910
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.031 | 0.031 | 0.028 | 0.056 | 0.590 | 0.095 | 0.122 | 0.561 | 0.316 | 0.039 | 0.041 | 0.040 | 0.053 | 0.052 | 0.051
Avg-Cov | 0.995 | 0.996 | 0.998 | 0.940 | 0.918 | 0.934 | 0.996 | 0.987 | 0.987 | 0.00 | 0.016 | 0.00 | 0.929 | 0.948 | 0.966
Avg-Len | 0.284 | 0.291 | 0.237 | 0.247 | 5.597 | 1.688 | 1.567 | 1.857 | 28.765 | 0.001 | 0.007 | 0.002 | 0.241 | 0.256 | 0.268
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.033 | 0.035 | 0.029 | 0.095 | 0.066 | 0.317 | NA | NA | NA | 0.031 | 0.037 | 0.025 | 0.034 | 0.034 | 0.034
Avg-Cov | 0.977 | 0.991 | 0.981 | 0.842 | 0.870 | 0.852 | NA | NA | NA | 0.955 | 0.906 | 0.963 | 0.918 | 0.936 | 0.945
Avg-Len | 0.197 | 0.234 | 0.180 | 0.198 | 0.195 | 0.212 | NA | NA | NA | 0.162 | 0.156 | 0.133 | 0.152 | 0.162 | 0.168
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.143 | 0.152 | 0.109 | 0.235 | 0.166 | 0.195 | NA | NA | NA | 0.184 | 0.220 | 0.147 | 0.198 | 0.177 | 0.165
Avg-Cov | 1.00 | 0.998 | 1.00 | 0.840 | 0.862 | 0.876 | NA | NA | NA | 0.905 | 0.783 | 0.893 | 0.900 | 0.939 | 0.968
Avg-Len | 1.212 | 1.350 | 1.122 | 0.817 | 0.643 | 0.634 | NA | NA | NA | 0.790 | 0.664 | 0.608 | 0.821 | 0.875 | 0.910
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.045 | 0.047 | 0.036 | 0.079 | 0.058 | 0.057 | NA | NA | NA | 0.048 | 0.052 | 0.037 | 0.063 | 0.057 | 0.055
Avg-Cov | 0.997 | 0.998 | 0.999 | 0.814 | 0.836 | 0.844 | NA | NA | NA | 0.928 | 0.874 | 0.934 | 0.880 | 0.927 | 0.949
Avg-Len | 0.343 | 0.389 | 0.309 | 0.246 | 0.205 | 0.191 | NA | NA | NA | 0.222 | 0.199 | 0.171 | 0.242 | 0.258 | 0.268

Table 1: Simulation results under the circulant symmetric covariance matrix
$\Sigma^{\mathrm{cs}}$, Gaussian noise $\mathcal{N}(0,1)$, and the MAR setting
(22). Nearly all the standard errors for the above average quantities are less
than 0.01 except for those associated with “DL-vdG” or “R-Proj” and thus
omitted.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| DL-vdG
---
(CC)
| DL-vdG
---
(IPW)
| DL-vdG
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.071 | 0.072 | 0.062 | 0.814 | 0.750 | 0.361 | 0.579 | 0.481 | 0.608 | 0.073 | 0.074 | 0.060 | 0.074 | 0.075 | 0.089
Avg-Cov | 0.888 | 0.889 | 0.884 | 0.932 | 0.914 | 0.920 | 0.953 | 0.950 | 0.950 | 0.946 | 0.944 | 0.956 | 0.889 | 0.913 | 0.939
Avg-Len | 0.279 | 0.281 | 0.248 | 0.734 | 0.744 | 0.643 | 1.196 | 1.539 | 1.917 | 0.354 | 0.352 | 0.301 | 0.296 | 0.326 | 0.413
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.284 | 0.291 | 0.157 | 0.293 | 0.287 | 0.952 | 0.297 | 0.219 | 0.372 | 0.428 | 0.257 | 0.178 | 0.468 | 0.405 | 0.358
Avg-Cov | 0.747 | 0.748 | 0.913 | 0.552 | 0.414 | 0.538 | 0.943 | 0.800 | 0.945 | 0.781 | 0.748 | 0.824 | 0.845 | 0.915 | 0.987
Avg-Len | 0.828 | 0.843 | 0.673 | 0.558 | 0.334 | 0.689 | 1.008 | 0.617 | 0.833 | 1.157 | 0.758 | 0.593 | 1.651 | 1.819 | 2.306
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.092 | 0.093 | 0.074 | 0.145 | 0.402 | 0.179 | 0.118 | 0.181 | 0.265 | 0.098 | 0.095 | 0.075 | 0.103 | 0.096 | 0.101
Avg-Cov | 0.861 | 0.863 | 0.894 | 0.828 | 0.814 | 0.854 | 0.948 | 0.935 | 0.925 | 0.912 | 0.903 | 0.921 | 0.874 | 0.926 | 0.961
Avg-Len | 0.341 | 0.342 | 0.301 | 0.391 | 0.309 | 0.611 | 0.611 | 0.593 | 1.490 | 0.416 | 0.388 | 0.335 | 0.390 | 0.429 | 0.544
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.058 | 0.058 | 0.052 | 0.107 | 0.399 | 0.168 | NA | NA | NA | 0.048 | 0.049 | 0.041 | 0.056 | 0.060 | 0.070
Avg-Cov | 0.989 | 0.993 | 0.994 | 0.900 | 0.892 | 0.878 | NA | NA | NA | 0.948 | 0.948 | 0.952 | 0.934 | 0.942 | 0.956
Avg-Len | 0.354 | 0.388 | 0.319 | 0.322 | 0.602 | 0.300 | NA | NA | NA | 0.237 | 0.237 | 0.202 | 0.255 | 0.287 | 0.348
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.241 | 0.244 | 0.135 | 0.254 | 0.205 | 0.325 | NA | NA | NA | 0.376 | 0.244 | 0.177 | 0.386 | 0.336 | 0.310
Avg-Cov | 0.977 | 0.982 | 1.00 | 0.578 | 0.420 | 0.514 | NA | NA | NA | 0.698 | 0.693 | 0.750 | 0.852 | 0.939 | 0.981
Avg-Len | 1.367 | 1.457 | 1.074 | 0.524 | 0.313 | 0.624 | NA | NA | NA | 0.996 | 0.650 | 0.519 | 1.426 | 1.603 | 1.943
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.083 | 0.083 | 0.069 | 0.102 | 0.254 | 0.312 | NA | NA | NA | 0.086 | 0.083 | 0.067 | 0.091 | 0.085 | 0.087
Avg-Cov | 0.991 | 0.993 | 0.996 | 0.818 | 0.806 | 0.846 | NA | NA | NA | 0.918 | 0.906 | 0.921 | 0.872 | 0.933 | 0.962
Avg-Len | 0.523 | 0.557 | 0.456 | 0.370 | 0.289 | 0.564 | NA | NA | NA | 0.375 | 0.349 | 0.301 | 0.336 | 0.378 | 0.458
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.057 | 0.057 | 0.051 | 0.448 | 1.176 | 1.228 | 0.597 | 0.926 | 0.733 | 0.00 | 0.000 | 0.00 | 0.080 | 0.091 | 0.101
Avg-Cov | 0.989 | 0.989 | 0.981 | 0.962 | 0.958 | 0.950 | 0.971 | 0.970 | 0.958 | 1.00 | 0.999 | 1.00 | 0.947 | 0.947 | 0.943
Avg-Len | 0.348 | 0.351 | 0.312 | 0.949 | 20.869 | 355.74 | 1.667 | 11.133 | 8.075 | 0.00 | 0.000 | 0.00 | 0.373 | 0.428 | 0.480
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.167 | 0.166 | 0.128 | 0.383 | 0.292 | 0.818 | 0.309 | 0.239 | 0.279 | 0.260 | 0.223 | 0.200 | 0.399 | 0.372 | 0.390
Avg-Cov | 0.978 | 0.982 | 0.994 | 0.758 | 0.542 | 0.584 | 0.989 | 0.888 | 0.954 | 0.251 | 0.349 | 0.391 | 0.960 | 0.986 | 0.992
Avg-Len | 1.055 | 1.073 | 0.858 | 3.623 | 1.893 | 0.833 | 1.304 | 0.792 | 1.097 | 0.411 | 0.376 | 0.331 | 2.084 | 2.392 | 2.681
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.068 | 0.068 | 0.062 | 0.664 | 0.403 | 0.387 | 0.421 | 0.244 | 0.394 | 0.059 | 0.058 | 0.056 | 0.103 | 0.107 | 0.115
Avg-Cov | 0.989 | 0.989 | 0.986 | 0.922 | 0.926 | 0.926 | 0.962 | 0.946 | 0.963 | 0.302 | 0.341 | 0.377 | 0.943 | 0.960 | 0.969
Avg-Len | 0.432 | 0.433 | 0.382 | 27.188 | 1.820 | 2.489 | 2.658 | 0.774 | 1.953 | 0.136 | 0.146 | 0.143 | 0.492 | 0.564 | 0.633
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.050 | 0.050 | 0.042 | 0.117 | 0.241 | 0.159 | NA | NA | NA | 0.044 | 0.044 | 0.037 | 0.044 | 0.045 | 0.047
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.864 | 0.882 | 0.836 | NA | NA | NA | 0.955 | 0.952 | 0.960 | 0.906 | 0.932 | 0.941
Avg-Len | 0.720 | 0.753 | 0.570 | 0.270 | 0.444 | 0.245 | NA | NA | NA | 0.214 | 0.213 | 0.182 | 0.180 | 0.208 | 0.222
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.172 | 0.177 | 0.095 | 0.213 | 0.137 | 0.201 | NA | NA | NA | 0.343 | 0.218 | 0.159 | 0.408 | 0.295 | 0.254
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.690 | 0.572 | 0.676 | NA | NA | NA | 0.626 | 0.583 | 0.613 | 0.664 | 0.865 | 0.945
Avg-Len | 3.586 | 3.684 | 2.537 | 0.440 | 0.257 | 0.302 | NA | NA | NA | 0.728 | 0.460 | 0.359 | 1.004 | 1.161 | 1.238
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.065 | 0.064 | 0.054 | 0.158 | 0.127 | 0.173 | NA | NA | NA | 0.064 | 0.061 | 0.050 | 0.075 | 0.062 | 0.059
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.486 | 0.618 | 0.508 | NA | NA | NA | 0.887 | 0.880 | 0.886 | 0.793 | 0.906 | 0.941
Avg-Len | 1.295 | 1.327 | 1.021 | 0.310 | 0.245 | 0.244 | NA | NA | NA | 0.252 | 0.235 | 0.203 | 0.237 | 0.274 | 0.292

Table 2: Simulation results under the Toeplitz (or auto-regressive) covariance
matrix $\Sigma^{\mathrm{ar}}$, Gaussian noise $\mathcal{N}(0,1)$ and the MAR
setting (22). Nearly all the standard errors for the above average quantities
are less than 0.01 except for those associated with “R-Proj” and thus omitted.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.033 | 0.035 | 0.028 | 0.070 | 0.063 | 0.492 | 0.033 | 0.038 | 0.027 | 0.043 | 0.040 | 0.039
Avg-Cov | 0.965 | 0.965 | 0.959 | 0.969 | 0.969 | 0.928 | 0.944 | 0.897 | 0.953 | 0.859 | 0.901 | 0.911
Avg-Len | 0.176 | 0.185 | 0.145 | 0.334 | 0.329 | 0.926 | 0.161 | 0.156 | 0.133 | 0.151 | 0.161 | 0.167
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.148 | 0.157 | 0.108 | 0.093 | 0.207 | 0.088 | 0.183 | 0.189 | 0.141 | 0.227 | 0.198 | 0.179
Avg-Cov | 0.992 | 0.989 | 0.999 | 0.998 | 0.999 | 0.996 | 0.916 | 0.840 | 0.919 | 0.846 | 0.917 | 0.958
Avg-Len | 0.980 | 0.994 | 0.843 | 1.250 | 1.304 | 1.729 | 0.790 | 0.664 | 0.608 | 0.821 | 0.873 | 0.908
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.045 | 0.048 | 0.035 | 0.056 | 0.078 | 0.083 | 0.049 | 0.051 | 0.036 | 0.063 | 0.058 | 0.055
Avg-Cov | 0.987 | 0.985 | 0.993 | 0.992 | 0.981 | 0.980 | 0.936 | 0.880 | 0.932 | 0.868 | 0.918 | 0.945
Avg-Len | 0.284 | 0.291 | 0.236 | 0.387 | 0.362 | 0.543 | 0.222 | 0.199 | 0.171 | 0.241 | 0.256 | 0.267
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.041 | 0.041 | 0.035 | NA | NA | NA | 0.036 | 0.041 | 0.029 | 0.041 | 0.041 | 0.045
Avg-Cov | 0.986 | 0.995 | 0.987 | NA | NA | NA | 0.951 | 0.907 | 0.957 | 0.910 | 0.922 | 0.930
Avg-Len | 0.262 | 0.312 | 0.235 | NA | NA | NA | 0.175 | 0.169 | 0.144 | 0.173 | 0.183 | 0.201
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.177 | 0.191 | 0.130 | NA | NA | NA | 0.245 | 0.271 | 0.180 | 0.226 | 0.217 | 0.195
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.869 | 0.746 | 0.894 | 0.903 | 0.932 | 0.978
Avg-Len | 1.657 | 1.853 | 1.512 | NA | NA | NA | 0.932 | 0.789 | 0.725 | 0.941 | 0.992 | 1.091
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.054 | 0.056 | 0.045 | NA | NA | NA | 0.062 | 0.068 | 0.046 | 0.071 | 0.068 | 0.065
Avg-Cov | 0.999 | 0.999 | 0.999 | NA | NA | NA | 0.906 | 0.838 | 0.932 | 0.889 | 0.909 | 0.956
Avg-Len | 0.467 | 0.530 | 0.413 | NA | NA | NA | 0.266 | 0.238 | 0.205 | 0.277 | 0.291 | 0.321
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.025 | 0.025 | 0.021 | 0.430 | 1.097 | 0.407 | 0.00 | 0.000 | 0.000 | 0.033 | 0.034 | 0.036
Avg-Cov | 0.989 | 0.993 | 0.986 | 0.960 | 0.958 | 0.930 | 1.00 | 0.998 | 0.999 | 0.952 | 0.951 | 0.948
Avg-Len | 0.176 | 0.185 | 0.145 | 1.138 | 14.143 | 0.509 | 0.00 | 0.000 | 0.000 | 0.151 | 0.161 | 0.168
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.105 | 0.104 | 0.087 | 0.134 | 0.108 | 0.226 | 0.190 | 0.194 | 0.193 | 0.200 | 0.180 | 0.168
Avg-Cov | 0.997 | 0.997 | 0.999 | 0.998 | 0.999 | 0.995 | 0.001 | 0.060 | 0.005 | 0.900 | 0.935 | 0.964
Avg-Len | 0.981 | 0.993 | 0.843 | 1.638 | 52.948 | 2.199 | 0.015 | 0.063 | 0.028 | 0.820 | 0.872 | 0.910
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.033 | 0.032 | 0.027 | 0.433 | 0.159 | 0.350 | 0.040 | 0.042 | 0.040 | 0.055 | 0.053 | 0.052
Avg-Cov | 0.994 | 0.995 | 0.997 | 0.998 | 0.989 | 0.984 | 0.001 | 0.011 | 0.001 | 0.915 | 0.934 | 0.953
Avg-Len | 0.284 | 0.291 | 0.236 | 5.491 | 1.625 | 0.909 | 0.001 | 0.007 | 0.003 | 0.241 | 0.256 | 0.267
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.034 | 0.035 | 0.028 | NA | NA | NA | 0.034 | 0.038 | 0.027 | 0.036 | 0.036 | 0.036
Avg-Cov | 0.976 | 0.990 | 0.985 | NA | NA | NA | 0.948 | 0.892 | 0.948 | 0.897 | 0.927 | 0.933
Avg-Len | 0.197 | 0.234 | 0.180 | NA | NA | NA | 0.161 | 0.156 | 0.133 | 0.152 | 0.161 | 0.168
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.146 | 0.157 | 0.108 | NA | NA | NA | 0.194 | 0.217 | 0.148 | 0.211 | 0.190 | 0.177
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.912 | 0.770 | 0.905 | 0.886 | 0.937 | 0.963
Avg-Len | 1.212 | 1.349 | 1.121 | NA | NA | NA | 0.789 | 0.662 | 0.608 | 0.823 | 0.976 | 0.911
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.044 | 0.047 | 0.036 | NA | NA | NA | 0.051 | 0.053 | 0.038 | 0.067 | 0.061 | 0.058
Avg-Cov | 0.998 | 0.999 | 0.998 | NA | NA | NA | 0.920 | 0.858 | 0.928 | 0.853 | 0.924 | 0.941
Avg-Len | 0.344 | 0.389 | 0.309 | NA | NA | NA | 0.222 | 0.199 | 0.171 | 0.242 | 0.257 | 0.268

Table 3: Simulation results under the circulant symmetric covariance matrix
$\Sigma^{\mathrm{cs}}$, $\mathrm{Laplace}\left(0,\frac{1}{\sqrt{2}}\right)$
distributed noise, and the MAR setting (22). Nearly all the standard errors
for the above average quantities are less than 0.01 except for those
associated with “R-Proj” and thus omitted.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.099 | 0.103 | 0.086 | 0.189 | 0.198 | 0.452 | 0.099 | 0.110 | 0.087 | 0.125 | 0.117 | 0.110
Avg-Cov | 0.946 | 0.960 | 0.955 | 0.948 | 0.945 | 0.923 | 0.957 | 0.916 | 0.949 | 0.867 | 0.918 | 0.938
Avg-Len | 0.517 | 0.532 | 0.440 | 0.858 | 1.144 | 1.629 | 0.477 | 0.459 | 0.402 | 0.471 | 0.501 | 0.527
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.181 | 0.187 | 0.141 | 0.177 | 0.177 | 0.471 | 0.215 | 0.229 | 0.175 | 0.259 | 0.227 | 0.206
Avg-Cov | 0.993 | 0.992 | 0.996 | 0.999 | 0.993 | 0.993 | 0.923 | 0.845 | 0.911 | 0.874 | 0.930 | 0.957
Avg-Len | 1.195 | 1.205 | 1.050 | 1.487 | 1.301 | 2.202 | 0.932 | 0.815 | 0.743 | 0.965 | 1.026 | 1.075
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.109 | 0.112 | 0.092 | 0.168 | 0.174 | 0.298 | 0.113 | 0.121 | 0.097 | 0.140 | 0.129 | 0.121
Avg-Cov | 0.981 | 0.977 | 0.982 | 0.970 | 0.960 | 0.960 | 0.942 | 0.891 | 0.944 | 0.865 | 0.918 | 0.945
Avg-Len | 0.628 | 0.642 | 0.538 | 0.905 | 0.873 | 1.366 | 0.523 | 0.500 | 0.439 | 0.537 | 0.571 | 0.601
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.129 | 0.129 | 0.111 | NA | NA | NA | 0.111 | 0.125 | 0.101 | 0.122 | 0.122 | 0.128
Avg-Cov | 0.981 | 0.996 | 0.991 | NA | NA | NA | 0.946 | 0.906 | 0.934 | 0.928 | 0.934 | 0.943
Avg-Len | 0.767 | 0.893 | 0.714 | NA | NA | NA | 0.516 | 0.499 | 0.435 | 0.540 | 0.570 | 0.626
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.225 | 0.233 | 0.178 | NA | NA | NA | 0.304 | 0.337 | 0.228 | 0.257 | 0.250 | 0.233
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.862 | 0.748 | 0.879 | 0.913 | 0.943 | 0.968
Avg-Len | 1.971 | 2.201 | 1.852 | NA | NA | NA | 1.086 | 0.956 | 0.878 | 1.107 | 1.167 | 1.284
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.149 | 0.148 | 0.127 | NA | NA | NA | 0.144 | 0.156 | 0.123 | 0.143 | 0.141 | 0.144
Avg-Cov | 0.997 | 0.999 | 0.996 | NA | NA | NA | 0.932 | 0.858 | 0.920 | 0.903 | 0.931 | 0.951
Avg-Len | 0.984 | 1.123 | 0.911 | NA | NA | NA | 0.620 | 0.592 | 0.520 | 0.617 | 0.651 | 0.715
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.076 | 0.075 | 0.067 | 0.810 | 0.566 | 0.628 | 0.00 | 0.002 | 0.00 | 0.097 | 0.102 | 0.106
Avg-Cov | 0.987 | 0.989 | 0.986 | 0.959 | 0.957 | 0.940 | 1.00 | 0.996 | 1.00 | 0.955 | 0.953 | 0.957
Avg-Len | 0.517 | 0.531 | 0.440 | 5.314 | 2.157 | 2.126 | 0.00 | 0.003 | 0.00 | 0.471 | 0.502 | 0.522
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.133 | 0.131 | 0.115 | 0.174 | 0.196 | 0.396 | 0.187 | 0.196 | 0.192 | 0.227 | 0.211 | 0.204
Avg-Cov | 0.998 | 0.997 | 0.996 | 0.999 | 0.996 | 0.995 | 0.001 | 0.034 | 0.002 | 0.908 | 0.951 | 0.972
Avg-Len | 1.194 | 1.203 | 1.050 | 1.757 | 2.160 | 2.170 | 0.008 | 0.053 | 0.021 | 0.966 | 1.028 | 1.070
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.083 | 0.081 | 0.072 | 0.271 | 0.523 | 0.269 | 0.039 | 0.042 | 0.039 | 0.114 | 0.117 | 0.120
Avg-Cov | 0.996 | 0.997 | 0.994 | 0.974 | 0.969 | 0.963 | 0.00 | 0.001 | 0.00 | 0.943 | 0.954 | 0.963
Avg-Len | 0.628 | 0.641 | 0.538 | 7.329 | 6.314 | 1.360 | 0.000 | 0.005 | 0.000 | 0.538 | 0.573 | 0.596
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.102 | 0.105 | 0.089 | NA | NA | NA | 0.105 | 0.118 | 0.089 | 0.111 | 0.108 | 0.108
Avg-Cov | 0.970 | 0.991 | 0.989 | NA | NA | NA | 0.952 | 0.912 | 0.950 | 0.907 | 0.933 | 0.939
Avg-Len | 0.575 | 0.671 | 0.545 | NA | NA | NA | 0.476 | 0.457 | 0.402 | 0.472 | 0.504 | 0.522
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.183 | 0.189 | 0.145 | NA | NA | NA | 0.224 | 0.256 | 0.179 | 0.237 | 0.218 | 0.207
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.914 | 0.807 | 0.904 | 0.889 | 0.947 | 0.965
Avg-Len | 1.448 | 1.613 | 1.380 | NA | NA | NA | 0.931 | 0.813 | 0.739 | 0.968 | 1.033 | 1.071
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.113 | 0.114 | 0.098 | NA | NA | NA | 0.118 | 0.129 | 0.098 | 0.136 | 0.128 | 0.125
Avg-Cov | 0.989 | 0.997 | 0.996 | NA | NA | NA | 0.940 | 0.879 | 0.939 | 0.889 | 0.928 | 0.943
Avg-Len | 0.730 | 0.834 | 0.686 | NA | NA | NA | 0.527 | 0.503 | 0.438 | 0.539 | 0.575 | 0.596

Table 4: Simulation results under the circulant symmetric covariance matrix
$\Sigma^{\mathrm{cs}}$, $t_{2}$ distributed noise, and the MAR setting (22).
Nearly all the standard errors for the above average quantities are less than
0.01 except for those associated with “R-Proj” and thus omitted.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.075 | 0.075 | 0.064 | 0.545 | 0.738 | 0.250 | 0.073 | 0.074 | 0.061 | 0.074 | 0.076 | 0.088
Avg-Cov | 0.859 | 0.869 | 0.877 | 0.939 | 0.946 | 0.944 | 0.945 | 0.940 | 0.947 | 0.881 | 0.914 | 0.942
Avg-Len | 0.279 | 0.281 | 0.247 | 0.967 | 1.735 | 2.550 | 0.353 | 0.351 | 0.300 | 0.295 | 0.325 | 0.412
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.288 | 0.289 | 0.157 | 0.205 | 0.236 | 0.377 | 0.411 | 0.246 | 0.176 | 0.457 | 0.394 | 0.363
Avg-Cov | 0.749 | 0.751 | 0.908 | 0.937 | 0.774 | 0.953 | 0.781 | 0.782 | 0.815 | 0.845 | 0.924 | 0.982
Avg-Len | 0.828 | 0.842 | 0.673 | 0.996 | 0.625 | 1.368 | 1.166 | 0.761 | 0.593 | 1.648 | 1.816 | 2.302
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.094 | 0.095 | 0.077 | 0.318 | 0.127 | 0.251 | 0.097 | 0.095 | 0.080 | 0.102 | 0.096 | 0.103
Avg-Cov | 0.847 | 0.834 | 0.877 | 0.940 | 0.927 | 0.931 | 0.917 | 0.895 | 0.915 | 0.866 | 0.911 | 0.956
Avg-Len | 0.341 | 0.342 | 0.300 | 0.765 | 0.569 | 0.879 | 0.415 | 0.387 | 0.334 | 0.389 | 0.429 | 0.543
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.059 | 0.059 | 0.050 | NA | NA | NA | 0.050 | 0.051 | 0.043 | 0.060 | 0.064 | 0.075
Avg-Cov | 0.988 | 0.993 | 0.991 | NA | NA | NA | 0.942 | 0.930 | 0.936 | 0.916 | 0.929 | 0.937
Avg-Len | 0.353 | 0.387 | 0.319 | NA | NA | NA | 0.237 | 0.237 | 0.202 | 0.255 | 0.286 | 0.347
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.244 | 0.247 | 0.135 | NA | NA | NA | 0.377 | 0.247 | 0.179 | 0.390 | 0.341 | 0.328
Avg-Cov | 0.978 | 0.987 | 0.999 | NA | NA | NA | 0.683 | 0.688 | 0.757 | 0.866 | 0.941 | 0.981
Avg-Len | 1.365 | 1.455 | 1.074 | NA | NA | NA | 0.999 | 0.652 | 0.519 | 1.423 | 1.599 | 1.939
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.083 | 0.084 | 0.068 | NA | NA | NA | 0.088 | 0.085 | 0.071 | 0.090 | 0.086 | 0.092
Avg-Cov | 0.991 | 0.992 | 0.994 | NA | NA | NA | 0.909 | 0.894 | 0.899 | 0.857 | 0.918 | 0.949
Avg-Len | 0.522 | 0.557 | 0.456 | NA | NA | NA | 0.373 | 0.348 | 0.301 | 0.336 | 0.377 | 0.458
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.056 | 0.056 | 0.053 | 0.700 | 1.163 | 0.562 | 0.00 | 0.000 | 0.00 | 0.074 | 0.085 | 0.097
Avg-Cov | 0.991 | 0.992 | 0.980 | 0.952 | 0.948 | 0.948 | 1.00 | 0.999 | 1.00 | 0.953 | 0.951 | 0.953
Avg-Len | 0.348 | 0.350 | 0.312 | 3.483 | 15.279 | 1.778 | 0.00 | 0.000 | 0.00 | 0.373 | 0.427 | 0.479
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.174 | 0.173 | 0.127 | 0.175 | 0.333 | 0.218 | 0.250 | 0.215 | 0.198 | 0.378 | 0.352 | 0.371
Avg-Cov | 0.974 | 0.979 | 0.992 | 0.993 | 0.893 | 0.975 | 0.237 | 0.352 | 0.371 | 0.954 | 0.986 | 0.997
Avg-Len | 1.054 | 1.071 | 0.858 | 1.287 | 0.803 | 1.125 | 0.384 | 0.370 | 0.329 | 2.081 | 2.386 | 2.676
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.068 | 0.068 | 0.062 | 0.129 | 0.212 | 0.345 | 0.056 | 0.055 | 0.053 | 0.095 | 0.098 | 0.108
Avg-Cov | 0.983 | 0.984 | 0.988 | 0.984 | 0.966 | 0.959 | 0.302 | 0.330 | 0.363 | 0.954 | 0.970 | 0.977
Avg-Len | 0.431 | 0.433 | 0.381 | 0.793 | 1.870 | 1.138 | 0.134 | 0.142 | 0.135 | 0.491 | 0.563 | 0.631
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.050 | 0.050 | 0.041 | NA | NA | NA | 0.045 | 0.045 | 0.038 | 0.046 | 0.048 | 0.049
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.934 | 0.929 | 0.948 | 0.891 | 0.922 | 0.932
Avg-Len | 0.717 | 0.750 | 0.571 | NA | NA | NA | 0.214 | 0.213 | 0.182 | 0.179 | 0.207 | 0.221
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.177 | 0.179 | 0.097 | NA | NA | NA | 0.340 | 0.214 | 0.159 | 0.403 | 0.296 | 0.257
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.632 | 0.575 | 0.614 | 0.647 | 0.872 | 0.943
Avg-Len | 3.579 | 3.678 | 2.536 | NA | NA | NA | 0.731 | 0.462 | 0.359 | 1.002 | 1.158 | 1.317
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.067 | 0.066 | 0.055 | NA | NA | NA | 0.064 | 0.061 | 0.052 | 0.074 | 0.063 | 0.060
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.873 | 0.867 | 0.883 | 0.799 | 0.904 | 0.938
Avg-Len | 1.293 | 1.326 | 1.021 | NA | NA | NA | 0.251 | 0.234 | 0.203 | 0.236 | 0.273 | 0.292

Table 5: Simulation results under the Toeplitz (or auto-regressive) covariance
matrix $\Sigma^{\mathrm{ar}}$,
$\mathrm{Laplace}\left(0,\frac{1}{\sqrt{2}}\right)$ distributed noise, and the
MAR setting (22). Nearly all the standard errors for the above average
quantities are less than 0.01 except for those associated with “R-Proj” and
thus omitted.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.212 | 0.212 | 0.180 | 0.497 | 0.569 | 0.939 | 0.228 | 0.227 | 0.195 | 0.233 | 0.235 | 0.267
Avg-Cov | 0.877 | 0.873 | 0.902 | 0.944 | 0.939 | 0.956 | 0.939 | 0.933 | 0.949 | 0.897 | 0.911 | 0.937
Avg-Len | 0.820 | 0.821 | 0.750 | 2.284 | 2.156 | 3.056 | 1.068 | 1.060 | 0.918 | 0.926 | 1.020 | 1.294
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.375 | 0.374 | 0.261 | 0.409 | 0.369 | 0.391 | 0.556 | 0.399 | 0.328 | 0.557 | 0.488 | 0.485
Avg-Cov | 0.810 | 0.816 | 0.899 | 0.950 | 0.887 | 0.961 | 0.779 | 0.815 | 0.843 | 0.861 | 0.944 | 0.988
Avg-Len | 1.237 | 1.243 | 1.051 | 2.024 | 1.589 | 2.093 | 1.519 | 1.277 | 1.103 | 2.110 | 2.324 | 2.948
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.229 | 0.230 | 0.191 | 0.326 | 0.428 | 0.395 | 0.260 | 0.256 | 0.216 | 0.262 | 0.258 | 0.286
Avg-Cov | 0.877 | 0.879 | 0.916 | 0.950 | 0.942 | 0.956 | 0.929 | 0.925 | 0.938 | 0.882 | 0.921 | 0.947
Avg-Len | 0.873 | 0.873 | 0.794 | 1.648 | 1.723 | 2.003 | 1.129 | 1.097 | 0.955 | 1.014 | 1.116 | 1.416
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.163 | 0.165 | 0.145 | NA | NA | NA | 0.162 | 0.159 | 0.133 | 0.187 | 0.199 | 0.227
Avg-Cov | 0.990 | 0.996 | 0.989 | NA | NA | NA | 0.942 | 0.939 | 0.953 | 0.919 | 0.941 | 0.953
Avg-Len | 1.036 | 1.130 | 0.964 | NA | NA | NA | 0.723 | 0.722 | 0.621 | 0.799 | 0.901 | 1.093
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.313 | 0.314 | 0.219 | NA | NA | NA | 0.474 | 0.366 | 0.294 | 0.458 | 0.409 | 0.407
Avg-Cov | 0.985 | 0.990 | 0.996 | NA | NA | NA | 0.729 | 0.742 | 0.790 | 0.888 | 0.958 | 0.988
Avg-Len | 1.896 | 2.015 | 1.565 | NA | NA | NA | 1.345 | 1.104 | 0.964 | 1.821 | 2.050 | 2.487
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.194 | 0.196 | 0.166 | NA | NA | NA | 0.337 | 0.219 | 0.187 | 0.226 | 0.226 | 0.245
Avg-Cov | 0.984 | 0.991 | 0.989 | NA | NA | NA | 0.925 | 0.915 | 0.936 | 0.897 | 0.936 | 0.960
Avg-Len | 1.188 | 1.282 | 1.084 | NA | NA | NA | 0.970 | 0.948 | 0.836 | 0.874 | 0.986 | 1.196
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.165 | 0.165 | 0.163 | 0.818 | 0.814 | 0.862 | 0.00 | 0.001 | 0.00 | 0.247 | 0.282 | 0.318
Avg-Cov | 0.985 | 0.985 | 0.978 | 0.957 | 0.953 | 0.955 | 1.00 | 0.997 | 1.00 | 0.942 | 0.945 | 0.940
Avg-Len | 1.022 | 1.023 | 0.944 | 3.865 | 3.571 | 3.892 | 0.00 | 0.001 | 0.00 | 1.170 | 1.344 | 1.506
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.251 | 0.250 | 0.226 | 0.413 | 0.560 | 0.513 | 0.268 | 0.253 | 0.240 | 0.483 | 0.465 | 0.507
Avg-Cov | 0.976 | 0.973 | 0.977 | 0.989 | 0.949 | 0.960 | 0.239 | 0.325 | 0.327 | 0.968 | 0.992 | 0.996
Avg-Len | 1.565 | 1.571 | 1.333 | 2.625 | 2.187 | 2.728 | 0.481 | 0.512 | 0.454 | 2.665 | 3.058 | 3.428
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.172 | 0.172 | 0.172 | 0.406 | 0.495 | 0.521 | 0.071 | 0.070 | 0.070 | 0.259 | 0.285 | 0.319
Avg-Cov | 0.980 | 0.984 | 0.971 | 0.965 | 0.959 | 0.948 | 0.139 | 0.181 | 0.171 | 0.95 | 0.957 | 0.960
Avg-Len | 1.094 | 1.094 | 1.003 | 2.628 | 2.263 | 2.614 | 0.140 | 0.170 | 0.144 | 1.280 | 1.470 | 1.647
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.143 | 0.143 | 0.124 | NA | NA | NA | 0.143 | 0.138 | 0.119 | 0.141 | 0.144 | 0.145
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.945 | 0.946 | 0.950 | 0.894 | 0.933 | 0.942
Avg-Len | 2.104 | 2.189 | 1.713 | NA | NA | NA | 0.650 | 0.650 | 0.560 | 0.563 | 0.650 | 0.695
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.220 | 0.220 | 0.163 | NA | NA | NA | 0.420 | 0.301 | 0.235 | 0.448 | 0.339 | 0.302
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.689 | 0.716 | 0.763 | 0.733 | 0.904 | 0.967
Avg-Len | 4.693 | 4.788 | 3.424 | NA | NA | NA | 0.985 | 0.797 | 0.684 | 1.283 | 1.483 | 1.582
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.197 | 0.194 | 0.165 | NA | NA | NA | 0.166 | 0.159 | 0.133 | 0.168 | 0.160 | 0.157
Avg-Cov | 1.00 | 1.00 | 1.00 | NA | NA | NA | 0.920 | 0.913 | 0.925 | 0.863 | 0.927 | 0.946
Avg-Len | 2.629 | 2.717 | 2.115 | NA | NA | NA | 0.686 | 0.670 | 0.583 | 0.616 | 0.711 | 0.760

Table 6: Simulation results under the Toeplitz (or auto-regressive) covariance
matrix $\Sigma^{\mathrm{ar}}$, $t_{2}$ distributed noise, and the MAR setting
as in Section 4.1. Nearly all the standard errors for the above average
quantities are less than 0.01 except for those associated with “R-Proj” and
thus omitted.

|  | Oracle
---
(1SE)
| Oracle
---
(min-feas)
| LR
---
(1SE)
| LR
---
(min-feas)
| NB
---
(1SE)
| NB
---
(min-feas)
| RF
---
(1SE)
| RF
---
(min-feas)
| SVM
---
(1SE)
| SVM
---
(min-feas)
| NN
---
(1SE)
| NN
---
(min-feas)
Avg-MAE | 0 | 0 | 0.334 | 0.334 | 0.121 | 0.121 | 0.158 | 0.158 | 0.321 | 0.321 | 0.106 | 0.106
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.034 | 0.033 | 0.048 | 0.047 | 0.035 | 0.034 | 0.037 | 0.036 | 0.045 | 0.044 | 0.034 | 0.033
Avg-Cov | 0.902 | 0.928 | 0.815 | 0.824 | 0.899 | 0.924 | 0.885 | 0.918 | 0.828 | 0.846 | 0.901 | 0.931
Avg-Len | 0.140 | 0.150 | 0.157 | 0.160 | 0.139 | 0.151 | 0.145 | 0.153 | 0.154 | 0.158 | 0.134 | 0.139
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.172 | 0.144 | 0.234 | 0.226 | 0.174 | 0.142 | 0.181 | 0.157 | 0.221 | 0.211 | 0.173 | 0.138
Avg-Cov | 0.904 | 0.974 | 0.871 | 0.887 | 0.899 | 0.977 | 0.904 | 0.961 | 0.875 | 0.899 | 0.902 | 0.978
Avg-Len | 0.747 | 0.800 | 0.842 | 0.858 | 0.746 | 0.807 | 0.776 | 0.820 | 0.826 | 0.843 | 0.744 | 0.810
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.050 | 0.045 | 0.068 | 0.066 | 0.051 | 0.045 | 0.053 | 0.049 | 0.064 | 0.062 | 0.050 | 0.044
Avg-Cov | 0.913 | 0.966 | 0.861 | 0.879 | 0.909 | 0.968 | 0.911 | 0.959 | 0.873 | 0.889 | 0.912 | 0.973
Avg-Len | 0.218 | 0.233 | 0.245 | 0.250 | 0.217 | 0.235 | 0.226 | 0.239 | 0.241 | 0.246 | 0.217 | 0.236
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.035 | 0.037 | 0.050 | 0.053 | 0.035 | 0.038 | 0.036 | 0.038 | 0.047 | 0.050 | 0.036 | 0.038
Avg-Cov | 0.924 | 0.945 | 0.845 | 0.850 | 0.924 | 0.940 | 0.937 | 0.945 | 0.864 | 0.867 | 0.923 | 0.940
Avg-Len | 0.159 | 0.179 | 0.176 | 0.191 | 0.160 | 0.182 | 0.164 | 0.182 | 0.172 | 0.187 | 0.160 | 0.181
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.184 | 0.150 | 0.305 | 0.303 | 0.180 | 0.147 | 0.188 | 0.160 | 0.276 | 0.271 | 0.181 | 0.146
Avg-Cov | 0.940 | 0.991 | 0.778 | 0.853 | 0.951 | 0.995 | 0.938 | 0.987 | 0.819 | 0.889 | 0.950 | 0.992
Avg-Len | 0.853 | 0.960 | 0.939 | 1.019 | 0.857 | 0.974 | 0.877 | 0.973 | 0.923 | 1.002 | 0.855 | 0.969
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.056 | 0.051 | 0.085 | 0.088 | 0.054 | 0.051 | 0.055 | 0.052 | 0.078 | 0.079 | 0.055 | 0.051
Avg-Cov | 0.922 | 0.972 | 0.813 | 0.837 | 0.923 | 0.976 | 0.931 | 0.967 | 0.836 | 0.866 | 0.920 | 0.968
Avg-Len | 0.249 | 0.280 | 0.274 | 0.297 | 0.250 | 0.284 | 0.256 | 0.284 | 0.269 | 0.292 | 0.249 | 0.282
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.0307 | 0.033 | 0.033 | 0.034 | 0.031 | 0.033 | 0.031 | 0.032 | 0.032 | 0.033 | 0.031 | 0.034
Avg-Cov | 0.946 | 0.939 | 0.939 | 0.936 | 0.948 | 0.938 | 0.945 | 0.944 | 0.941 | 0.938 | 0.946 | 0.938
Avg-Len | 0.1485 | 0.161 | 0.158 | 0.161 | 0.149 | 0.162 | 0.150 | 0.160 | 0.155 | 0.159 | 0.149 | 0.164
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.164 | 0.139 | 0.162 | 0.155 | 0.163 | 0.137 | 0.162 | 0.138 | 0.155 | 0.149 | 0.162 | 0.132
Avg-Cov | 0.945 | 0.991 | 0.967 | 0.976 | 0.947 | 0.991 | 0.947 | 0.989 | 0.968 | 0.978 | 0.950 | 0.993
Avg-Len | 0.794 | 0.859 | 0.843 | 0.862 | 0.795 | 0.867 | 0.804 | 0.858 | 0.829 | 0.849 | 0.796 | 0.875
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.047 | 0.045 | 0.049 | 0.048 | 0.047 | 0.046 | 0.047 | 0.045 | 0.048 | 0.047 | 0.047 | 0.045
Avg-Cov | 0.956 | 0.973 | 0.966 | 0.967 | 0.957 | 0.974 | 0.963 | 0.974 | 0.966 | 0.967 | 0.959 | 0.975
Avg-Len | 0.232 | 0.250 | 0.246 | 0.251 | 0.232 | 0.253 | 0.234 | 0.250 | 0.242 | 0.247 | 0.232 | 0.255
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.030 | 0.031 | 0.041 | 0.041 | 0.030 | 0.031 | 0.031 | 0.032 | 0.038 | 0.038 | 0.030 | 0.031
Avg-Cov | 0.938 | 0.949 | 0.879 | 0.883 | 0.940 | 0.949 | 0.944 | 0.945 | 0.898 | 0.903 | 0.938 | 0.948
Avg-Len | 0.1392 | 0.149 | 0.158 | 0.161 | 0.139 | 0.150 | 0.145 | 0.153 | 0.155 | 0.158 | 0.139 | 0.151
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.166 | 0.145 | 0.193 | 0.192 | 0.165 | 0.139 | 0.157 | 0.141 | 0.180 | 0.177 | 0.166 | 0.139
Avg-Cov | 0.934 | 0.972 | 0.927 | 0.932 | 0.933 | 0.976 | 0.945 | 0.973 | 0.936 | 0.945 | 0.935 | 0.979
Avg-Len | 0.745 | 0.798 | 0.845 | 0.860 | 0.744 | 0.804 | 0.775 | 0.818 | 0.829 | 0.845 | 0.742 | 0.806
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.050 | 0.046 | 0.055 | 0.056 | 0.050 | 0.045 | 0.047 | 0.045 | 0.053 | 0.052 | 0.050 | 0.045
Avg-Cov | 0.915 | 0.960 | 0.931 | 0.932 | 0.918 | 0.970 | 0.943 | 0.971 | 0.932 | 0.937 | 0.917 | 0.967
Avg-Len | 0.217 | 0.232 | 0.246 | 0.251 | 0.217 | 0.234 | 0.226 | 0.238 | 0.242 | 0.246 | 0.216 | 0.235

Table 7: Simulation results for our proposed debiasing method when the
propensity scores are estimated by various machine learning methods under the
circulant symmetric covariance matrix $\Sigma^{\mathrm{cs}}$, Gaussian noises
$\mathcal{N}(0,1)$, and the new MAR setting (23). Note that the “Avg-MAE” for
propensity score estimation is independent of the choices of $\beta_{0}$ and
$x$, so we place it on the top row. We also highlight the best results in
terms of the evaluation metrics for each simulation design, excluding those
with the oracle propensity scores, in boldface.

###  D.2 Simulations Under the MCAR Setting

We follow our comparative simulation designs in Section 4.1 but define the
missingness indicators $R_{i},i=1,...,n$ through a MCAR mechanism as
$\mathrm{P}(R_{i}=1)=0.7$ for $i=1,...,n$. Under the MCAR setting, both the
circulant symmetric matrix $\Sigma^{\mathrm{cs}}$ and the Toeplitz matrix
$\Sigma^{\mathrm{ar}}$ will have their inverses satisfying our sufficient
conditions for the sparse approximation to the population dual solution
$\ell_{0}(x)$ in Lemma 1 as we have illuminated in Example 1 as well. The
simulation results are shown in Table 8 and Table 9 under the Gaussian noise
$\mathcal{N}(0,1)$ setting, and the conclusions are similar to the main
takeaways from the MAR simulation studies in Section 4.2. We omit the
simulation results for other types of the noise distribution, since they share
similar patterns with the Gaussian noise setting and we have demonstrated in
Section 4.3 that our proposed debiasing method is robust to other heavy-tailed
noise distributions.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| DL-vdG
---
(CC)
| DL-vdG
---
(IPW)
| DL-vdG
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.033 | 0.033 | 0.028 | 0.262 | 0.376 | 0.304 | 0.104 | 0.283 | 0.315 | 0.033 | 0.034 | 0.028 | 0.043 | 0.039 | 0.039
Avg-Cov | 0.958 | 0.958 | 0.962 | 0.884 | 0.886 | 0.894 | 0.969 | 0.955 | 0.940 | 0.940 | 0.943 | 0.948 | 0.847 | 0.905 | 0.912
Avg-Len | 0.173 | 0.173 | 0.145 | 0.166 | 0.225 | 0.200 | 0.379 | 0.376 | 0.638 | 0.159 | 0.157 | 0.133 | 0.149 | 0.161 | 0.164
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.145 | 0.145 | 0.108 | 0.183 | 0.157 | 0.405 | 0.096 | 0.202 | 0.207 | 0.175 | 0.174 | 0.139 | 0.229 | 0.186 | 0.180
Avg-Cov | 0.993 | 0.994 | 0.999 | 0.876 | 0.868 | 0.908 | 0.998 | 0.997 | 0.996 | 0.919 | 0.871 | 0.916 | 0.851 | 0.941 | 0.949
Avg-Len | 0.969 | 0.969 | 0.844 | 0.744 | 0.616 | 0.618 | 1.255 | 2.215 | 1.784 | 0.774 | 0.678 | 0.608 | 0.808 | 0.872 | 0.888
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.044 | 0.044 | 0.035 | 0.053 | 0.047 | 0.045 | 0.059 | 0.104 | 0.088 | 0.047 | 0.046 | 0.037 | 0.063 | 0.055 | 0.054
Avg-Cov | 0.985 | 0.985 | 0.996 | 0.872 | 0.876 | 0.898 | 0.989 | 0.985 | 0.985 | 0.936 | 0.932 | 0.939 | 0.855 | 0.937 | 0.942
Avg-Len | 0.280 | 0.281 | 0.237 | 0.219 | 0.194 | 0.179 | 0.182 | 0.347 | 0.527 | 0.217 | 0.200 | 0.171 | 0.237 | 0.256 | 0.260
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.042 | 0.042 | 0.037 | 0.073 | 0.499 | 0.119 | NA | NA | NA | 0.035 | 0.036 | 0.030 | 0.037 | 0.038 | 0.040
Avg-Cov | 0.982 | 0.989 | 0.988 | 0.806 | 0.830 | 0.806 | NA | NA | NA | 0.941 | 0.933 | 0.955 | 0.924 | 0.939 | 0.950
Avg-Len | 0.258 | 0.286 | 0.236 | 0.213 | 0.296 | 0.193 | NA | NA | NA | 0.173 | 0.171 | 0.144 | 0.171 | 0.180 | 0.196
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.172 | 0.172 | 0.132 | 0.332 | 0.219 | 0.247 | NA | NA | NA | 0.239 | 0.240 | 0.179 | 0.223 | 0.214 | 0.193
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.768 | 0.834 | 0.810 | NA | NA | NA | 0.859 | 0.812 | 0.896 | 0.905 | 0.932 | 0.967
Avg-Len | 1.639 | 1.758 | 1.514 | 0.979 | 0.810 | 0.772 | NA | NA | NA | 0.913 | 0.807 | 0.724 | 0.922 | 0.975 | 1.057
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.053 | 0.053 | 0.046 | 0.099 | 0.073 | 0.080 | NA | NA | NA | 0.058 | 0.057 | 0.046 | 0.066 | 0.064 | 0.061
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.766 | 0.854 | 0.788 | NA | NA | NA | 0.920 | 0.920 | 0.917 | 0.901 | 0.926 | 0.962
Avg-Len | 0.461 | 0.498 | 0.414 | 0.288 | 0.256 | 0.233 | NA | NA | NA | 0.260 | 0.240 | 0.205 | 0.270 | 0.286 | 0.310
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.026 | 0.026 | 0.022 | 0.995 | 0.214 | 0.349 | 0.320 | 1.289 | 0.740 | 0.00 | 0.00 | 0.00 | 0.032 | 0.034 | 0.035
Avg-Cov | 0.987 | 0.987 | 0.989 | 0.916 | 0.910 | 0.912 | 0.972 | 0.963 | 0.937 | 1.00 | 0.999 | 1.00 | 0.934 | 0.944 | 0.944
Avg-Len | 0.173 | 0.173 | 0.145 | 6.794 | 2.565 | 0.589 | 2.341 | 5.584 | 1.177 | 0.00 | 0.00 | 0.00 | 0.149 | 0.161 | 0.165
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.102 | 0.102 | 0.090 | 0.226 | 0.238 | 0.202 | 0.122 | 0.196 | 0.084 | 0.187 | 0.195 | 0.192 | 0.200 | 0.175 | 0.170
Avg-Cov | 0.998 | 0.998 | 0.999 | 0.922 | 0.908 | 0.910 | 0.998 | 0.997 | 0.997 | 0.003 | 0.050 | 0.007 | 0.897 | 0.957 | 0.965
Avg-Len | 0.969 | 0.970 | 0.844 | 5.187 | 2.197 | 0.596 | 1.343 | 2.516 | 1.728 | 0.010 | 0.059 | 0.025 | 0.807 | 0.870 | 0.891
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.032 | 0.032 | 0.028 | 0.492 | 0.255 | 0.095 | 0.333 | 0.369 | 0.316 | 0.040 | 0.041 | 0.040 | 0.054 | 0.052 | 0.051
Avg-Cov | 0.999 | 0.999 | 0.998 | 0.940 | 0.934 | 0.934 | 0.991 | 0.990 | 0.987 | 0.001 | 0.007 | 0.00 | 0.923 | 0.947 | 0.952
Avg-Len | 0.281 | 0.281 | 0.237 | 2.609 | 3.050 | 1.688 | 2.153 | 1.071 | 28.765 | 0.001 | 0.006 | 0.002 | 0.237 | 0.255 | 0.261
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.034 | 0.034 | 0.029 | 0.283 | 0.408 | 0.317 | NA | NA | NA | 0.032 | 0.033 | 0.025 | 0.034 | 0.034 | 0.035
Avg-Cov | 0.973 | 0.984 | 0.981 | 0.852 | 0.870 | 0.852 | NA | NA | NA | 0.959 | 0.949 | 0.963 | 0.920 | 0.938 | 0.941
Avg-Len | 0.195 | 0.217 | 0.180 | 0.178 | 0.242 | 0.212 | NA | NA | NA | 0.159 | 0.157 | 0.133 | 0.150 | 0.162 | 0.165
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.143 | 0.143 | 0.109 | 0.228 | 0.166 | 0.195 | NA | NA | NA | 0.184 | 0.194 | 0.147 | 0.196 | 0.174 | 0.170
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.834 | 0.880 | 0.876 | NA | NA | NA | 0.915 | 0.852 | 0.893 | 0.910 | 0.957 | 0.963
Avg-Len | 1.203 | 1.299 | 1.122 | 0.800 | 0.661 | 0.634 | NA | NA | NA | 0.775 | 0.679 | 0.608 | 0.810 | 0.877 | 0.891
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.043 | 0.043 | 0.036 | 0.069 | 0.054 | 0.058 | NA | NA | NA | 0.049 | 0.048 | 0.037 | 0.060 | 0.054 | 0.053
Avg-Cov | 0.999 | 1.00 | 0.999 | 0.836 | 0.872 | 0.844 | NA | NA | NA | 0.930 | 0.904 | 0.934 | 0.887 | 0.942 | 0.948
Avg-Len | 0.340 | 0.370 | 0.309 | 0.235 | 0.209 | 0.191 | NA | NA | NA | 0.217 | 0.200 | 0.171 | 0.238 | 0.257 | 0.261

Table 8: Simulation results under the circulant symmetric covariance matrix
$\Sigma^{\mathrm{cs}}$, Gaussian noise $\mathcal{N}(0,1)$, and the MCAR
setting. Nearly all the standard errors for the above average quantities are
less than 0.01 except for those associated with “DL-vdG” and “R-Proj” and thus
omitted.

|  | DL-Jav
---
(CC)
| DL-Jav
---
(IPW)
| DL-Jav
---
(Oracle)
| DL-vdG
---
(CC)
| DL-vdG
---
(IPW)
| DL-vdG
---
(Oracle)
| R-Proj
---
(CC)
| R-Proj
---
(IPW)
| R-Proj
---
(Oracle)
| Refit
---
(CC)
| Refit
---
(IPW)
| Refit
---
(Oracle)
| Debias
---
(min-CV)
| Debias
---
(1SE)
| Debias
---
(min-feas)
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{sp}=\beta_{0,1}^{(a)}$
Avg-Bias | 0.072 | 0.072 | 0.062 | 0.319 | 0.465 | 0.361 | 0.275 | 0.444 | 0.608 | 0.073 | 0.074 | 0.060 | 0.074 | 0.076 | 0.089
Avg-Cov | 0.870 | 0.870 | 0.884 | 0.900 | 0.890 | 0.920 | 0.963 | 0.955 | 0.950 | 0.948 | 0.944 | 0.956 | 0.883 | 0.914 | 0.929
Avg-Len | 0.282 | 0.282 | 0.248 | 0.450 | 0.719 | 0.643 | 0.885 | 1.413 | 1.917 | 0.360 | 0.354 | 0.301 | 0.301 | 0.332 | 0.420
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{de}=\beta_{0,1}^{(b)}$
Avg-Bias | 0.298 | 0.297 | 0.157 | 0.306 | 0.240 | 0.952 | 0.204 | 0.200 | 0.372 | 0.454 | 0.264 | 0.178 | 0.489 | 0.430 | 0.404
Avg-Cov | 0.748 | 0.756 | 0.913 | 0.540 | 0.394 | 0.538 | 0.941 | 0.759 | 0.945 | 0.772 | 0.751 | 0.824 | 0.850 | 0.932 | 0.983
Avg-Len | 0.846 | 0.862 | 0.673 | 0.609 | 0.338 | 0.689 | 1.004 | 0.588 | 0.833 | 1.241 | 0.775 | 0.593 | 1.760 | 1.940 | 2.459
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(1)}$ so that
$m(x^{(1)})=x^{(1)T}\beta_{0}^{pd}=\beta_{0,1}^{(c)}$
Avg-Bias | 0.095 | 0.095 | 0.074 | 0.108 | 0.104 | 0.179 | 0.137 | 0.174 | 0.265 | 0.097 | 0.095 | 0.075 | 0.105 | 0.098 | 0.106
Avg-Cov | 0.847 | 0.847 | 0.894 | 0.848 | 0.806 | 0.854 | 0.954 | 0.920 | 0.925 | 0.911 | 0.900 | 0.921 | 0.867 | 0.929 | 0.964
Avg-Len | 0.345 | 0.346 | 0.301 | 0.418 | 0.307 | 0.611 | 0.659 | 0.647 | 1.490 | 0.426 | 0.395 | 0.335 | 0.400 | 0.440 | 0.558
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{sp}$
Avg-Bias | 0.059 | 0.059 | 0.052 | 0.132 | 0.446 | 0.168 | NA | NA | NA | 0.050 | 0.051 | 0.041 | 0.058 | 0.062 | 0.074
Avg-Cov | 0.982 | 0.990 | 0.993 | 0.882 | 0.870 | 0.878 | NA | NA | NA | 0.935 | 0.937 | 0.952 | 0.925 | 0.933 | 0.942
Avg-Len | 0.357 | 0.391 | 0.319 | 0.318 | 0.682 | 0.300 | NA | NA | NA | 0.241 | 0.241 | 0.202 | 0.260 | 0.292 | 0.354
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{de}$
Avg-Bias | 0.257 | 0.258 | 0.135 | 0.275 | 0.222 | 0.325 | NA | NA | NA | 0.396 | 0.256 | 0.177 | 0.407 | 0.361 | 0.350
Avg-Cov | 0.967 | 0.979 | 1.00 | 0.590 | 0.432 | 0.514 | NA | NA | NA | 0.709 | 0.694 | 0.750 | 0.862 | 0.946 | 0.977
Avg-Len | 1.397 | 1.494 | 1.457 | 0.573 | 0.318 | 0.624 | NA | NA | NA | 1.067 | 0.662 | 0.519 | 1.519 | 1.708 | 2.071
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(2)}$ so that
$m(x^{(2)})=x^{(2)T}\beta_{0}^{pd}$
Avg-Bias | 0.084 | 0.084 | 0.069 | 0.106 | 0.100 | 0.312 | NA | NA | NA | 0.087 | 0.084 | 0.067 | 0.091 | 0.087 | 0.091
Avg-Cov | 0.991 | 0.996 | 0.996 | 0.854 | 0.830 | 0.846 | NA | NA | NA | 0.915 | 0.908 | 0.921 | 0.867 | 0.914 | 0.951
Avg-Len | 0.530 | 0.565 | 0.456 | 0.393 | 0.289 | 0.564 | NA | NA | NA | 0.382 | 0.354 | 0.301 | 0.345 | 0.388 | 0.470
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{sp}=\beta_{0,100}^{(a)}$
Avg-Bias | 0.054 | 0.054 | 0.051 | 1.545 | 1.148 | 1.228 | 0.421 | 1.217 | 0.733 | 0.00 | 0.00 | 0.00 | 0.077 | 0.088 | 0.097
Avg-Cov | 0.990 | 0.990 | 0.981 | 0.956 | 0.956 | 0.950 | 0.972 | 0.967 | 0.958 | 1.00 | 1.00 | 1.00 | 0.953 | 0.956 | 0.955
Avg-Len | 0.352 | 0.352 | 0.312 | 17.123 | 4.612 | 355.74 | 3.970 | 8.027 | 8.075 | 0.00 | 0.00 | 0.00 | 0.380 | 0.436 | 0.489
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{de}=\beta_{0,100}^{(b)}$
Avg-Bias | 0.173 | 0.173 | 0.128 | 0.245 | 0.905 | 0.818 | 0.178 | 0.221 | 0.279 | 0.255 | 0.217 | 0.200 | 0.399 | 0.371 | 0.392
Avg-Cov | 0.978 | 0.980 | 0.994 | 0.762 | 0.494 | 0.584 | 0.988 | 0.889 | 0.954 | 0.229 | 0.337 | 0.391 | 0.969 | 0.993 | 0.995
Avg-Len | 1.078 | 1.097 | 0.858 | 0.720 | 12.286 | 0.833 | 1.288 | 1.680 | 1.097 | 0.388 | 0.353 | 0.331 | 2.223 | 2.550 | 2.858
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(3)}$ so that
$m(x^{(3)})=x^{(3)T}\beta_{0}^{pd}=\beta_{0,100}^{(c)}$
Avg-Bias | 0.065 | 0.065 | 0.062 | 0.336 | 0.253 | 0.387 | 0.302 | 0.209 | 0.394 | 0.059 | 0.057 | 0.056 | 0.103 | 0.107 | 0.117
Avg-Cov | 0.983 | 0.983 | 0.986 | 0.950 | 0.932 | 0.926 | 0.984 | 0.970 | 0.963 | 0.303 | 0.346 | 0.377 | 0.948 | 0.972 | 0.980
Avg-Len | 0.437 | 0.438 | 0.382 | 17.694 | 1.626 | 2.489 | 2.462 | 3.289 | 1.953 | 0.143 | 0.149 | 0.143 | 0.505 | 0.579 | 0.649
|  $d=1000$, $n=900$, sparse $\beta_{0}^{sp}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{sp}$
Avg-Bias | 0.050 | 0.050 | 0.042 | 0.139 | 0.199 | 0.159 | NA | NA | NA | 0.045 | 0.045 | 0.037 | 0.045 | 0.046 | 0.048
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.830 | 0.860 | 0.836 | NA | NA | NA | 0.944 | 0.938 | 0.960 | 0.890 | 0.924 | 0.931
Avg-Len | 0.736 | 0.767 | 0.570 | 0.267 | 0.307 | 0.245 | NA | NA | NA | 0.218 | 0.217 | 0.182 | 0.183 | 0.212 | 0.226
|  $d=1000$, $n=900$, dense $\beta_{0}^{de}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{de}$
Avg-Bias | 0.181 | 0.182 | 0.095 | 0.215 | 0.175 | 0.201 | NA | NA | NA | 0.376 | 0.228 | 0.159 | 0.436 | 0.323 | 0.285
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.668 | 0.556 | 0.676 | NA | NA | NA | 0.616 | 0.560 | 0.613 | 0.650 | 0.867 | 0.944
Avg-Len | 3.694 | 3.792 | 2.537 | 0.479 | 0.268 | 0.302 | NA | NA | NA | 0.782 | 0.470 | 0.359 | 1.070 | 1.238 | 1.320
|  $d=1000$, $n=900$, pseudo-dense $\beta_{0}^{pd}$, and $x^{(4)}$ so that
$m(x^{(4)})=x^{(4)T}\beta_{0}^{pd}$
Avg-Bias | 0.065 | 0.065 | 0.054 | 0.152 | 0.122 | 0.173 | NA | NA | NA | 0.067 | 0.063 | 0.050 | 0.078 | 0.065 | 0.062
Avg-Cov | 1.00 | 1.00 | 1.00 | 0.494 | 0.622 | 0.508 | NA | NA | NA | 0.880 | 0.871 | 0.886 | 0.794 | 0.918 | 0.946
Avg-Len | 1.325 | 1.356 | 1.021 | 0.331 | 0.244 | 0.244 | NA | NA | NA | 0.258 | 0.239 | 0.203 | 0.243 | 0.281 | 0.300

Table 9: Simulation results under the Toeplitz (or auto-regressive) covariance
matrix $\Sigma^{\mathrm{ar}}$, Gaussian noise $\mathcal{N}(0,1)$, and the MCAR
setting. Nearly all the standard errors for the above average quantities are
less than 0.01 except for those associated with “DL-vdG” and “R-Proj” and thus
omitted.

##  Appendix E Proofs

This section consists of the proofs for theorems and propositions in the main
paper as well as other auxiliary results.

###  E.1 Additional Notations

We denote by $\\{X_{i}\\}_{i=1}^{\infty}$ a sequence of independent
$\mathcal{X}$-valued random elements with common probability measure
$\mathrm{P}$. In most cases of this paper, $\mathcal{X}$ is the Euclidean
space $\mathbb{R}^{d}$. For any measure $Q$ and any real-valued $Q$-integrable
function $f$ on the measurable space $(\mathcal{X},\mathcal{A})$, we write
$Qf=\int fdQ$ and denote by $L_{p}(\mathcal{X},\mathcal{A},Q)$ with
$p\in[1,\infty)$ as the space of all real-valued measurable functions $f$ on
$(\mathcal{X},\mathcal{A})$ with finite $L_{p}(Q)$-norm, _i.e._ ,
$\left|\left|f\right|\right|_{Q,p}:=\left(\int|f|^{p}dQ\right)^{\frac{1}{p}}<\infty$.
The empirical measure $\mathbb{P}_{n}$ associated with observations
$\\{X_{1},...,X_{n}\\}$ is defined as the random measure on
$(\mathcal{X},\mathcal{A})$ given by
$\mathbb{P}_{n}=\frac{1}{n}\sum_{i=1}^{n}\delta_{X_{i}}$, where $\delta_{x}$
is the Dirac measure at point $x\in\mathcal{X}$. We denote the empirical
process indexed by a class $\mathcal{F}$ by
$\mathbb{G}_{n}(f)=\sqrt{n}\left(\mathbb{P}_{n}-\mathrm{P}\right)f=\frac{1}{\sqrt{n}}\sum_{i=1}^{n}\left(f(X_{i})-\mathrm{P}f\right),f\in\mathcal{F}$.
For any measure $Q$ on $(\mathcal{X},\mathcal{A})$, we set
$\left|\left|Q\right|\right|_{\mathcal{F}}=\sup_{f\in\mathcal{F}}|Qf|$.

Analogous to the $\psi_{\alpha}$-Orlicz norm (14) for random variables, we
define the $\psi_{\alpha}$-Orlicz norm for a real-valued $Q$-integrable
function $f$ on $(\mathcal{X},\mathcal{A})$ as:

$\left|\left|f\right|\right|_{Q,\psi_{\alpha}}=\inf\left\\{t>0:\int\psi_{\alpha}\left(\frac{|f|}{t}\right)dQ\leq
1\right\\}$ (30)

for $\psi_{\alpha}(u)=\exp(u^{\alpha})-1$.

###  E.2 Proofs of Propositions 1 and 2

######  Proposition 1.

The dual form of the debiasing program (3) is given by

$\min_{\ell(x)\in\mathbb{R}^{d}}\left\\{\frac{1}{4n}\sum_{i=1}^{n}\widehat{\pi}_{i}\left[X_{i}^{T}\ell(x)\right]^{2}+x^{T}\ell(x)+\frac{\gamma}{n}\left|\left|\ell(x)\right|\right|_{1}\right\\}.$

If strong duality holds, then the solution
$\widehat{\bm{w}}(x)\in\mathbb{R}^{n}$ to the primal debiasing program (3) and
the solution $\widehat{\ell}(x)\in\mathbb{R}^{d}$ to its dual formulation (6)
satisfy the following identity as:

$\widehat{w}_{i}(x)=-\frac{1}{2\sqrt{n}}\cdot X_{i}^{T}\widehat{\ell}(x),\quad
i=1,...,n.$

###### Proof of Proposition 1.

The proof is mainly adopted from Lemma 5 of Giessing and Wang (2021). Recall
from (3) that the debiasing program is a convex optimization problem and can
be written as the matrix form:

$\displaystyle\begin{split}&\min_{\bm{w}:=\bm{w}(x)\in\mathbb{R}^{n}}\frac{1}{2}\bm{w}^{T}\widehat{\Psi}\bm{w}\\\
&\text{subject to }\,A\bm{w}\preceq b,\end{split}$ (31)

where
$\widehat{\Psi}=\mathtt{Diag}\left(2\widehat{\pi}_{1},...,2\widehat{\pi}_{n}\right)\in\mathbb{R}^{n\times
n}$, $A=\frac{1}{\sqrt{n}}\begin{pmatrix}-\bm{X}^{T}\widehat{\bm{\Pi}}\\\
\bm{X}^{T}\widehat{\bm{\Pi}}\end{pmatrix}\in\mathbb{R}^{2d\times n}$ with
$\widehat{\bm{\Pi}}=\mathtt{Diag}\left(\widehat{\pi}_{1},...,\widehat{\pi}_{n}\right)\in\mathbb{R}^{n\times
n}$, and $b=\begin{pmatrix}\frac{\gamma}{n}\cdot\mathbbm{1}_{d}-x\\\
\frac{\gamma}{n}\cdot\mathbbm{1}_{d}+x\end{pmatrix}\in\mathbb{R}^{2d}$. This
is because the box constraint
$\left|\left|x-\frac{1}{\sqrt{n}}\sum_{i=1}^{n}w_{i}\cdot\widehat{\pi}_{i}\cdot
X_{i}\right|\right|_{\infty}\leq\frac{\gamma}{n}$ in (3) is equivalent to

$\displaystyle-\frac{\gamma}{n}\cdot\mathbbm{1}_{d}\leq
x-\frac{1}{\sqrt{n}}\sum_{i=1}^{n}w_{i}\cdot\widehat{\pi}_{i}\cdot
X_{i}\leq\frac{\gamma}{n}\cdot\mathbbm{1}_{d}$ $\displaystyle\quad\text{ and
}\quad\frac{1}{\sqrt{n}}\begin{pmatrix}-\bm{X}^{T}\widehat{\bm{\Pi}}\\\
\bm{X}^{T}\widehat{\bm{\Pi}}\end{pmatrix}\bm{w}\leq\begin{pmatrix}\frac{\gamma}{n}\cdot\mathbbm{1}_{d}-x\\\
\frac{\gamma}{n}\cdot\mathbbm{1}_{d}+x\end{pmatrix},$

where $\bm{w}=(w_{1},...,w_{n})^{T}\in\mathbb{R}^{n}$. The Lagrange dual
function is given by

$g(\xi)=\inf_{\bm{w}\in\mathbb{R}^{n}}\left[\frac{1}{2}\bm{w}^{T}\widehat{\Psi}\bm{w}+\xi^{T}(A\bm{w}-b)\right]=-\frac{1}{2}\xi^{T}A\widehat{\Psi}^{-1}A^{T}\xi-\xi^{T}b,$
(32)

where $\xi\in\mathbb{R}^{2d}$ and the infimum is attained at
$\bm{w}=-\widehat{\Psi}^{-1}A^{T}\xi$. It implies the dual of (31) as:

$\displaystyle\max_{\xi:=\xi(x)\in\mathbb{R}^{2d}}\left[-\frac{1}{2}\xi^{T}A\widehat{\Psi}^{-1}A^{T}\xi-\xi^{T}b\right]$
$\displaystyle\text{subject to }\,\xi\succeq\bm{0}.$

If strong duality holds, then the solution
$\widehat{\bm{w}}(x)\in\mathbb{R}^{n}$ to the primal problem can be related to
the solution $\widehat{\xi}(x)\in\mathbb{R}^{2d}$ to the dual problem as:

$\widehat{\bm{w}}(x)=-\widehat{\Psi}^{-1}A^{T}\widehat{\xi}(x).$ (33)

Since $\widehat{\xi}(x)\in\mathbb{R}^{2d}$ is the vector of optimal Lagrangian
multipliers associated with the constrains $A\bm{w}\preceq b$ in the primal
problem (31), it satisfies the complementary slackness conditions

$\left[\xi(x)\right]_{k}\geq 0\quad\text{ and
}\quad\left[A\widehat{\bm{w}}(x)\right]_{k}<b_{k}\quad\implies\quad\left[\xi(x)\right]_{k}=0,$

for any $k=1,...,2d$. We write
$\widehat{\xi}(x)=\left(\bar{\mu},\underline{\mu}\right)$ with
$\bar{\mu},\underline{\mu}\in\mathbb{R}_{+}^{d}$. The above complementary
slackness conditions indicate that

$\bar{\mu}_{k}\cdot\underline{\mu}_{k}=0,\quad k=1,...,d,$ (34)

because by design, at least one of the inequalities
$\left[A\widehat{\bm{w}}(x)\right]_{k}\leq b_{k}$ and
$\left[A\widehat{\bm{w}}(x)\right]_{d+k}\leq b_{d+k}$ should be strict. Take
$\widehat{\ell}(x):=(-\bar{\mu}+\underline{\mu})\in\mathbb{R}^{d}$. Equation
(34) implies that

$|\widehat{\ell}_{k}(x)|=\bar{\mu}_{k}+\underline{\mu}_{k},\quad k=1,...,d,$

and thus,

$-\frac{1}{2}\xi(x)^{T}A\widehat{\Psi}^{-1}A^{T}\xi(x)-\xi(x)^{T}b=-\frac{1}{2n}\widehat{\ell}(x)^{T}\bm{X}^{T}\widehat{\bm{\Pi}}\widehat{\Psi}^{-1}\widehat{\bm{\Pi}}\bm{X}\widehat{\ell}(x)-x^{T}\widehat{\ell}(x)-\frac{\gamma}{n}\left|\left|\widehat{\ell}(x)\right|\right|_{1}.$

Since every $\ell(x)\in\mathbb{R}^{d}$ can be written as
$\ell(x)=\left(-\bar{\mu}+\underline{\mu}\right)$ for some
$\bar{\mu},\underline{\mu}\in\mathbb{R}_{+}^{d}$, the above equation implies
that $\widehat{\ell}(x)$ is the solution to the dual problem as:

$\displaystyle\min_{\ell(x)\in\mathbb{R}^{d}}\left[\frac{1}{2n}\ell(x)^{T}\bm{X}^{T}\widehat{\bm{\Pi}}\widehat{\Psi}^{-1}\widehat{\bm{\Pi}}\bm{X}\ell(x)+x^{T}\ell(x)+\frac{\gamma}{n}\left|\left|\ell(x)\right|\right|_{1}\right]$
$\displaystyle\equiv\min_{\ell(x)\in\mathbb{R}^{d}}\left\\{\frac{1}{4n}\sum_{i=1}^{n}\widehat{\pi}_{i}\left[X_{i}^{T}\ell(x)\right]^{2}+x^{T}\ell(x)+\frac{\gamma}{n}\left|\left|\ell(x)\right|\right|_{1}\right\\}.$

Finally, we observe from (33) that
$\widehat{\bm{w}}(x)=-\frac{1}{\sqrt{n}}\widehat{\Psi}^{-1}\widehat{\bm{\Pi}}\bm{X}\widehat{\ell}(x)$
and therefore,

$\widehat{w}_{i}(x)=-\frac{1}{2\sqrt{n}}\cdot X_{i}^{T}\widehat{\ell}(x),\quad
i=1,...,n.$

The proof is completed. ∎

######  Proposition 2.

Under Assumption 1, the conditional mean squared error of
$\sqrt{n}\,m^{\mathrm{debias}}(x;\bm{w})$ given $\bm{X}=(X_{1},...,X_{n})$ has
the following decomposition as:

$\displaystyle\mathrm{E}\left[\left(\sqrt{n}\,m^{\mathrm{debias}}(x;\bm{w})-\sqrt{n}\,m_{0}(x)\right)^{2}\Big{|}\bm{X}\right]$
$\displaystyle=\underbrace{\sigma_{\epsilon}^{2}\sum_{i=1}^{n}w_{i}^{2}\pi(X_{i})}_{\text{Conditional
variance
I}}+\underbrace{\left(\beta_{0}-\beta\right)^{T}\left[\sum_{i=1}^{n}w_{i}^{2}\pi(X_{i})\left(1-\pi(X_{i})\right)X_{i}X_{i}^{T}\right]\left(\beta_{0}-\beta\right)}_{\text{Conditional
variance II}}$
$\displaystyle\quad+\underbrace{\left[\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}w_{i}\pi(X_{i})X_{i}-x\right)^{T}\sqrt{n}\left(\beta_{0}-\beta\right)\right]^{2}}_{\text{Conditional
bias}}.$

###### Proof of Proposition 2.

Under the MAR condition in Assumption 1, the conditional mean of (9) (scaled
by a factor $\sqrt{n}$) given $\bm{X}=(X_{1},...,X_{n})^{T}$ can be written
as:

$\mathrm{E}\left[\sqrt{n}\,m^{\text{debias}}(x;\bm{w})\big{|}\bm{X}\right]=\sqrt{n}\,x^{T}\beta+\sum_{i=1}^{n}w_{i}\pi(X_{i})X_{i}^{T}\left(\beta_{0}-\beta\right).$

Then, we calculate the conditional mean squared error of (9) under model (1)
with $m_{0}(x)=\mathrm{E}(Y|X=x)=x^{T}\beta_{0}$ as:

$\displaystyle\mathrm{E}\left[\left(\sqrt{n}\,m^{\text{debias}}(x;\bm{w})-\sqrt{n}\,m_{0}(x)\right)^{2}\Big{|}\bm{X}\right]$
$\displaystyle=\mathrm{E}\left\\{\left[\sqrt{n}\,m^{\text{debias}}(x;\bm{w})-\mathrm{E}\left(\sqrt{n}\,m^{\text{debias}}(x;\bm{w})\big{|}\bm{X}\right)\right]^{2}\Big{|}\bm{X}\right\\}$
$\displaystyle\quad+\left\\{\mathrm{E}\left[\sqrt{n}\,m^{\text{debias}}(x;\bm{w})\big{|}\bm{X}\right]-\sqrt{n}\,m_{0}(x)\right\\}^{2}$
$\displaystyle=\mathrm{E}\left\\{\left[\sum_{i=1}^{n}w_{i}(R_{i}-\pi(X_{i}))X_{i}^{T}\left(\beta_{0}-\beta\right)+\sum_{i=1}^{n}w_{i}R_{i}\epsilon_{i}\right]^{2}\Bigg{|}\bm{X}\right\\}$
$\displaystyle\quad+\left[\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}w_{i}\pi(X_{i})X_{i}-x\right)^{T}\sqrt{n}\left(\beta_{0}-\beta\right)\right]^{2}$
$\displaystyle=\sum_{i=1}^{n}w_{i}^{2}\pi(X_{i})\left(1-\pi(X_{i})\right)\left[X_{i}^{T}\left(\beta_{0}-\beta\right)\right]^{2}+\sigma_{\epsilon}^{2}\sum_{i=1}^{n}w_{i}^{2}\pi(X_{i})$
$\displaystyle\quad+\left[\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}w_{i}\pi(X_{i})X_{i}-x\right)^{T}\sqrt{n}\left(\beta_{0}-\beta\right)\right]^{2}$
$\displaystyle=\underbrace{\sigma_{\epsilon}^{2}\sum_{i=1}^{n}w_{i}^{2}\pi(X_{i})}_{\text{Conditional
variance
I}}+\underbrace{\left(\beta_{0}-\beta\right)^{T}\left[\sum_{i=1}^{n}w_{i}^{2}\pi(X_{i})\left(1-\pi(X_{i})\right)X_{i}X_{i}^{T}\right]\left(\beta_{0}-\beta\right)}_{\text{Conditional
variance II}}$
$\displaystyle\quad+\underbrace{\left[\left(\frac{1}{\sqrt{n}}\sum_{i=1}^{n}w_{i}\pi(X_{i})X_{i}-x\right)^{T}\sqrt{n}\left(\beta_{0}-\beta\right)\right]^{2}}_{\text{Conditional
bias}}.$

The result thus follows. ∎

######  Remark 1.

Notice that we can upper bound the “Conditional variance II” term as:

$\displaystyle\sqrt{n}\left(\beta_{0}-\beta\right)^{T}\left[\frac{1}{n}\sum_{i=1}^{n}w_{i}^{2}\pi(X_{i})\left(1-\pi(X_{i})\right)X_{i}X_{i}^{T}\right]\sqrt{n}\left(\beta_{0}-\beta\right)$
$\displaystyle\leq\left(\sqrt{n}\left|\left|\beta_{0}-\beta\right|\right|_{1}\right)^{2}\max_{1\leq
i\leq n}\left|\left|\sqrt{\pi(X_{i})\left(1-\pi(X_{i})\right)}\cdot
w_{i}X_{i}\right|\right|_{\infty}.$

Thus, for some finite-sample case when the “Conditional variance II” term is
not negligible, one may consider optimizing the weights
$\bm{w}=(w_{1},...,w_{n})^{T}\in\mathbb{R}^{n}$ with a more precise debiasing
program as:

$\displaystyle\min_{\bm{w}\in\mathbb{R}^{n}}\sum_{i=1}^{n}\widehat{\pi}_{i}w_{i}^{2}$
$\displaystyle\text{ subject to
}\left|\left|x-\frac{1}{\sqrt{n}}\sum_{i=1}^{n}w_{i}\widehat{\pi}_{i}X_{i}\right|\right|_{\infty}\leq\frac{\gamma_{1}}{n}\quad\text{
and }\quad\max_{1\leq i\leq
n}\left|\left|\sqrt{\widehat{\pi}_{i}\left(1-\widehat{\pi}_{i}\right)}\cdot
w_{i}X_{i}\right|\right|_{\infty}\leq\frac{\gamma_{2}}{n},$

where $\gamma_{1},\gamma_{2}>0$ are two tuning parameters. However, it will be
difficult to derive the dual form of the above debiasing program and tune the
two parameters $\gamma_{1},\gamma_{2}$ simultaneously. Given that there are no
asymptotic efficiency gains for using the above program than our proposed one
(3) by Lemma 4, we still recommend our approach even for finite-sample cases.

###  E.3 Proof of Theorem 1

######  Theorem 3 (Consistency of the Lasso Pilot Estimate with Complete-Case
Data).

Let $\delta\in(0,1)$ and $A_{1},A_{2}>0$ be some absolute constants. Suppose
that Assumptions 1, 2, and 4 (more precisely, Eq.(16)) hold and $n$ is
sufficiently large so that

$\sqrt{\frac{s_{\beta}\log(ed/s_{\beta})}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<\min\left\\{\frac{\kappa_{R}^{2}}{A_{1}\phi_{s_{\beta}}},1\right\\}.$

  1. (a)

If
$\lambda>\frac{4}{n}\left|\left|\sum_{i=1}^{n}R_{i}X_{i}\epsilon_{i}\right|\right|_{\infty}>0$
and $d\geq s_{\beta}+2$, then with probability at least $1-\delta$,

$\left|\left|\widehat{\beta}-\beta_{0}\right|\right|_{2}\lesssim\frac{\sqrt{s_{\beta}}\lambda}{\kappa_{R}^{2}}.$

  2. (b)

If, in addition, Assumption 3 holds and
$\lambda=A_{2}\sigma_{\epsilon}\sqrt{\frac{\phi_{1}\log(d/\delta)}{n}}$, then
with probability at least $1-\delta$,

$\displaystyle\left|\left|\widehat{\beta}-\beta_{0}\right|\right|_{2}$
$\displaystyle\lesssim\frac{\sigma_{\epsilon}}{\kappa_{R}^{2}}\sqrt{\frac{\phi_{1}s_{\beta}\log(d/\delta)}{n}}.$

###### Proof of Theorem 1.

(a) Since $\widehat{\beta}$ minimizes the function
$L(\beta)=\frac{1}{n}\sum_{i=1}^{n}R_{i}\left(Y_{i}-X_{i}^{T}\beta\right)^{2}+\lambda\left|\left|\beta\right|\right|_{1}$
with complete-case data, we know that

$\frac{1}{n}\sum_{i=1}^{n}R_{i}\left(Y_{i}-X_{i}^{T}\widehat{\beta}\right)^{2}+\lambda\left|\left|\widehat{\beta}\right|\right|_{1}\leq\frac{1}{n}\sum_{i=1}^{n}R_{i}\left(Y_{i}-X_{i}^{T}\beta_{0}\right)^{2}+\lambda\left|\left|\beta_{0}\right|\right|_{1}.$

Then, under the linear model (1) in Assumption 1, it implies by direct
calculations or Taylor’s expansion that

$\frac{1}{n}\sum_{i=1}^{n}R_{i}\left(X_{i}^{T}\widehat{v}\right)^{2}\leq\frac{2}{n}\sum_{i=1}^{n}\epsilon_{i}R_{i}X_{i}^{T}\widehat{v}+\lambda\left(\left|\left|\beta_{0}\right|\right|_{1}-\left|\left|\beta_{0}+\widehat{v}\right|\right|_{1}\right)$
(35)

with $\widehat{v}=\widehat{\beta}-\beta_{0}$. Notice that
$\beta_{0,S_{\beta}^{c}}=0$, so we have
$\left|\left|\beta_{0}\right|\right|_{1}=\left|\left|\beta_{0,S_{\beta}}\right|\right|_{1}$
and

$\left|\left|\beta_{0}+\widehat{v}\right|\right|_{1}=\left|\left|\beta_{0,S_{\beta}}+\widehat{v}_{S_{\beta}}\right|\right|_{1}+\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}\geq\left|\left|\beta_{0,S_{\beta}}\right|\right|_{1}-\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}+\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}.$

Plugging this lower bound into (35), we obtain that

$\displaystyle\begin{split}\frac{1}{n}\sum_{i=1}^{n}R_{i}\left(X_{i}^{T}\widehat{v}\right)^{2}&\leq\frac{2}{n}\sum_{i=1}^{n}\epsilon_{i}R_{i}X_{i}^{T}\widehat{v}+\lambda\left(\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}-\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}\right)\\\
&\leq\frac{2\left|\left|\sum_{i=1}^{n}R_{i}X_{i}\epsilon_{i}\right|\right|_{\infty}}{n}\cdot\left|\left|\widehat{v}\right|\right|_{1}+\lambda\left(\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}-\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}\right),\end{split}$
(36)

where we apply Hölder’s inequality to obtain the second inequality. Under the
condition that
$\frac{4}{n}\left|\left|\sum_{i=1}^{n}R_{i}X_{i}\epsilon_{i}\right|\right|_{\infty}<\lambda$,
we can proceed (36) as:

$\displaystyle\begin{split}0\leq\frac{1}{n}\sum_{i=1}^{n}R_{i}\left(X_{i}^{T}\widehat{v}\right)^{2}&\leq\frac{\lambda}{2}\left(\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}+\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}\right)+\lambda\left(\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}-\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}\right)\\\
&=\frac{3\lambda}{2}\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}-\frac{\lambda}{2}\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}.\end{split}$
(37)

Rearranging this inequality shows that
$\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}\leq
3\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}$. Thus, the error
$\widehat{v}=\widehat{\beta}-\beta_{0}$ belongs to the cone
$\mathcal{C}_{1}(S_{\beta},3)$. By Assumption 4 (more precisely, Eq.(16)) and
Lemma 7 (taking $\vartheta=3$), we know that there exists an absolute constant
$A_{0}>0$ such that with probability at least $1-\delta$,

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}R_{i}\left(X_{i}^{T}\widehat{v}\right)^{2}$
$\displaystyle\geq\frac{1}{n}\sum_{i=1}^{n}\mathrm{E}\left[R_{i}\left(X_{i}^{T}\widehat{v}\right)^{2}\right]-A_{0}\phi_{s_{\beta}}\left[\sqrt{\frac{\log(1/\delta)+s_{\beta}\log(ed/s_{\beta})}{n}}+\frac{\log(1/\delta)+s_{\beta}\log(ed/s_{\beta})}{n}\right]\left|\left|\widehat{v}\right|\right|_{2}^{2}$
$\displaystyle\geq\left[\kappa_{R}^{2}-A_{0}\phi_{s_{\beta}}\left(\sqrt{\frac{\log(1/\delta)+s_{\beta}\log(ed/s_{\beta})}{n}}+\frac{\log(1/\delta)+s_{\beta}\log(ed/s_{\beta})}{n}\right)\right]\left|\left|\widehat{v}\right|\right|_{2}^{2}$
$\displaystyle\geq\left[\kappa_{R}^{2}-2A_{0}\phi_{s_{\beta}}\sqrt{\frac{\log(1/\delta)+s_{\beta}\log(ed/s_{\beta})}{n}}\right]\left|\left|\widehat{v}\right|\right|_{2}^{2}$
$\displaystyle\geq\frac{\kappa_{R}^{2}}{2}\left|\left|\widehat{v}\right|\right|_{2}^{2},$

where the last two inequalities hold under our growth condition

$\sqrt{\frac{s_{\beta}\log(ed/s_{\beta})}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<\min\left\\{\frac{\kappa_{R}^{2}}{A_{1}\phi_{s_{\beta}}},1\right\\}$

with $A_{1}=4A_{0}>0$ when $n$ is sufficiently large. Substituting the above
inequality into (37) yields that with probability at least $1-\delta$,

$\displaystyle\frac{\kappa_{R}^{2}}{2}\left|\left|\widehat{v}\right|\right|_{2}^{2}$
$\displaystyle\leq\frac{3\lambda}{2}\left|\left|\widehat{v}_{S_{\beta}}\right|\right|_{1}-\frac{\lambda}{2}\left|\left|\widehat{v}_{S_{\beta}^{c}}\right|\right|_{1}$
$\displaystyle\leq\frac{3}{2}\lambda\sqrt{s_{\beta}}\left|\left|\widehat{v}\right|\right|_{2},$

where the second inequality is due to the Cauchy-Schwarz inequality. The
result now follows by dividing $\left|\left|\widehat{v}\right|\right|_{2}$ on
both sides of the inequality.

(b) It suffices to show that
$\frac{4}{n}\left|\left|\sum_{i=1}^{n}R_{i}X_{i}\epsilon_{i}\right|\right|_{\infty}$
is upper bounded by $\sigma_{\epsilon}\sqrt{\frac{\phi_{1}\log(d/\delta)}{n}}$
up to some absolute constant with probability at least $1-\delta$. Notice that

$\frac{4}{n}\left|\left|\sum_{i=1}^{n}R_{i}X_{i}\epsilon_{i}\right|\right|_{\infty}=\max_{1\leq
k\leq d}\left|\frac{1}{n}\sum_{i=1}^{n}4\epsilon_{i}R_{i}X_{ik}\right|,$

where $X_{ik}$ is the $k$-th element of the covariate $X_{i}\in\mathbb{R}^{d}$
for $i=1,...,n$. By Assumptions 2 and 3 together with Lemma 12 (see also Lemma
2.7.7 in Vershynin 2018), we know that

$\displaystyle\max_{1\leq k\leq
d}\left|\left|4\epsilon_{i}R_{i}X_{ik}\right|\right|_{\psi_{1}}$
$\displaystyle\leq\max_{1\leq k\leq d}\left[4\left(\max_{1\leq i\leq
n}\left|\left|X_{ik}\right|\right|_{\psi_{2}}\right)\left|\left|R_{i}\epsilon_{i}\right|\right|_{\psi_{2}}\right]$
$\displaystyle\lesssim\sqrt{\phi_{1}}\cdot\sigma_{\epsilon}$

for all $i=1,...,n$. Hence, by the union bound and Bernstein’s inequality
(Corollary 2.8.3 in Vershynin 2018), we obtain that for all $t>0$, there
exists an absolute constant $A_{3}>0$ such that

$\displaystyle\mathrm{P}\left(\frac{4}{n}\left|\left|\sum_{i=1}^{n}R_{i}X_{i}\epsilon_{i}\right|\right|_{\infty}>t\right)$
$\displaystyle\leq d\cdot\max_{1\leq k\leq
d}\mathrm{P}\left(\left|\frac{1}{n}\sum_{i=1}^{n}4\epsilon_{i}R_{i}X_{ik}\right|>t\right)$
$\displaystyle\leq
2d\cdot\exp\left(-nA_{3}\min\left\\{\frac{t}{\sigma_{\epsilon}\sqrt{\phi_{1}}},\frac{t^{2}}{\sigma_{\epsilon}^{2}\phi_{1}}\right\\}\right).$

Thus, we conclude that with probability at least $1-\delta$,

$\displaystyle\frac{4}{n}\left|\left|\sum_{i=1}^{n}R_{i}X_{i}\epsilon_{i}\right|\right|_{\infty}$
$\displaystyle\lesssim\sigma_{\epsilon}\sqrt{\phi_{1}}\left[\sqrt{\frac{\log(d/\delta)}{n}}+\frac{\log(d/\delta)}{n}\right]$
$\displaystyle\lesssim\sigma_{\epsilon}\sqrt{\frac{\phi_{1}\log(d/\delta)}{n}},$

where we notice that $\sqrt{\frac{\log(d/\delta)}{n}}$ dominates the bound
under our growth condition
$\sqrt{\frac{s_{\beta}\log(ed/s_{\beta})}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<1$.
Finally, the bound on
$\left|\left|\widehat{\beta}-\beta_{0}\right|\right|_{2}$ follows from
substituting
$\lambda=A_{2}\sigma_{\epsilon}\sqrt{\frac{\phi_{1}\log(d/\delta)}{n}}$ for
some absolute constant into the upper bound of
$\left|\left|\widehat{\beta}-\beta_{0}\right|\right|_{2}$ derived in (a). ∎

###  E.4 Proof of Theorem 2

Before presenting the proof of Theorem 2, we first establish the consistency
of $\widehat{\ell}(x)$ to the sparse $r_{\ell}$-approximation
$\widetilde{\ell}(x)$ of the exact population dual solution (18) under
Assumption 6 and the known/oracle propensity scores
$\pi_{i}=\pi(X_{i}),i=1,...,n$ in the following lemma.

######  Lemma 1 (Consistency of the dual solution $\widehat{\ell}(x)$ when the
propensity score is known).

Let $\delta\in(0,1)$, $x\in\mathbb{R}^{d}$ be a fixed covariate vector, and
$A_{1},A_{2},A_{3}>0$ be some large absolute constants. Suppose that
Assumptions 1, 2, 4, and 6 hold as well as

$\sqrt{\frac{s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<\min\left\\{\frac{\kappa_{R}^{2}}{A_{1}\phi_{s_{\ell}(x)}},1\right\\}\quad\text{
and }\quad d\geq s_{\ell}(x)+2.$

  1. (a)

Assume that
$\gamma\geq\left|\left|\frac{5}{2}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\ell_{0}(x)+5nx\right|\right|_{\infty}$
and
$r_{\ell}<\frac{\kappa_{R}^{2}}{\left|\left|x\right|\right|_{2}}\sqrt{\frac{2\gamma}{13\nu}}$,
where $\nu>0$ is defined in (58) of Lemma 9. Then, with probability at least
$1-\delta$,

$\left|\left|\widehat{\ell}(x)-\widetilde{\ell}(x)\right|\right|_{2}\lesssim\frac{\gamma\sqrt{s_{\ell}(x)}}{n\cdot\kappa_{R}^{2}}+\frac{\phi_{s_{\ell}(x)}}{\kappa_{R}^{2}}\cdot\frac{\left|\left|x\right|\right|_{2}}{\kappa_{R}^{2}}\cdot
r_{\ell}.$

  2. (b)

If, in addition,
$\gamma=A_{2}\frac{\sqrt{\phi_{1}}}{\kappa_{R}}\left|\left|x\right|\right|_{2}\sqrt{n\log(d/\delta)}$
and
$r_{\ell}<\left(\frac{A_{2}^{2}\,\kappa_{R}^{6}\,\phi_{1}}{A_{3}^{2}\left|\left|x\right|\right|_{2}\phi_{s_{\ell}(x)}}\right)^{\frac{1}{4}}\left(\frac{\log(d/\delta)}{n}\right)^{\frac{1}{4}}$,
then with probability at least $1-\delta$,

$\displaystyle\left|\left|\widehat{\ell}(x)-\widetilde{\ell}(x)\right|\right|_{2}$
$\displaystyle\lesssim\frac{\left|\left|x\right|\right|_{2}}{\kappa_{R}^{2}}\left[\sqrt{\frac{\phi_{1}}{\kappa_{R}^{2}}}\sqrt{\frac{s_{\ell}(x)\log(d/\delta)}{n}}+\frac{\phi_{s_{\ell}(x)}}{\kappa_{R}^{2}}\cdot
r_{\ell}\right].$

###### Proof of Lemma 1.

(a) Since $\widehat{\ell}(x)$ is the solution to the dual program (6), we know
that

$\frac{1}{4n}\sum_{i=1}^{n}\pi(X_{i})\left[X_{i}^{T}\widehat{\ell}(x)\right]^{2}+x^{T}\widehat{\ell}(x)+\frac{\gamma}{n}\left|\left|\widehat{\ell}(x)\right|\right|_{1}\leq\frac{1}{4n}\sum_{i=1}^{n}\pi(X_{i})\left[X_{i}^{T}\widetilde{\ell}(x)\right]^{2}+x^{T}\widetilde{\ell}(x)+\frac{\gamma}{n}\left|\left|\widetilde{\ell}(x)\right|\right|_{1}.$

Some rearrangements (or using Taylor’s expansion) give us that

$\displaystyle\begin{split}&\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{4n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]+\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]\left[\widetilde{\ell}(x)-\ell_{0}(x)\right]\\\
&\leq\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\ell_{0}(x)+x\right]^{T}\left[\widetilde{\ell}(x)-\widehat{\ell}(x)\right]+\frac{\gamma}{n}\left[\left|\left|\widetilde{\ell}(x)\right|\right|_{1}-\left|\left|\widehat{\ell}(x)\right|\right|_{1}\right].\end{split}$
(38)

In the sequel, we write
$\widehat{\Delta}(x)=\widehat{\ell}(x)-\widetilde{\ell}(x)$. By Hölder’s
inequality and the (reverse) triangle’s inequality together with the sparsity
of $\widetilde{\ell}(x)$ under Assumption 6, the right-hand side of (38) is
upper bounded by

$\displaystyle\begin{split}&\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\ell_{0}(x)+x\right]^{T}\left[\widetilde{\ell}(x)-\widehat{\ell}(x)\right]+\frac{\gamma}{n}\left[\left|\left|\widetilde{\ell}(x)\right|\right|_{1}-\left|\left|\widehat{\ell}(x)\right|\right|_{1}\right]\\\
&\leq\left|\left|\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\ell_{0}(x)+x\right|\right|_{\infty}\left|\left|\widehat{\Delta}(x)\right|\right|_{1}\\\
&\quad+\frac{\gamma}{n}\left[\left|\left|\left[\widetilde{\ell}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}-\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}+\left[\widetilde{\ell}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}-\left|\left|\left[\widehat{\ell}(x)\right]_{S_{\ell}(x)^{c}}\right|\right|_{1}\right]\\\
&\leq\frac{\gamma}{5n}\left[\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}+\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)^{c}}\right|\right|_{1}\right]+\frac{\gamma}{n}\left[\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}-\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)^{c}}\right|\right|_{1}\right]\\\
&=\frac{6\gamma}{5n}\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}-\frac{4\gamma}{5n}\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)^{c}}\right|\right|_{1}\\\
&\leq\frac{6\gamma}{5n}\sqrt{s_{\ell}(x)}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}\end{split}$
(39)

where
$\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}=\sum_{k\in
S_{\ell}(x)}\left|\widehat{\Delta}_{k}(x)\right|$.

Next, we know from Lemma 9 with $a_{0}=5$ that

$\widehat{\ell}(x)-\widetilde{\ell}(x)\in\mathcal{C}_{1}\left(S_{\ell}(x),3\right)\bigcup
B_{d}^{(1)}\left(0,\,\frac{13r_{\ell}^{2}\left|\left|x\right|\right|_{2}^{2}}{2\kappa_{R}^{4}}\cdot\frac{\nu}{\gamma}\right),$

where
$B_{d}^{(1)}(0,r)=\left\\{v\in\mathbb{R}^{d}:\left|\left|v\right|\right|_{1}\leq
r\right\\}$. By assumption
$r_{\ell}<\frac{\kappa_{R}^{2}}{\left|\left|x\right|\right|_{2}}\sqrt{\frac{2\gamma}{13\nu}}$,
it follows that

$\left[\mathcal{C}_{1}\left(S_{\ell}(x),3\right)\bigcup
B_{d}^{(1)}\left(0,\,\frac{13r_{\ell}^{2}\left|\left|x\right|\right|_{2}^{2}}{2\kappa_{R}^{4}}\cdot\frac{\nu}{\gamma}\right)\right]\bigcap\mathbb{S}^{d-1}=\mathcal{C}_{1}\left(S_{\ell}(x),3\right)\cap\mathbb{S}^{d-1}.$

Hence, by Assumption 4 (more precisely, Remark 2) together with Lemma 8 with
$\vartheta_{\ell}=\frac{r_{\ell}}{\sqrt{1-r_{\ell}^{2}}}<1$, the left-hand
side of (38) is lower bounded by

$\displaystyle\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{4n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]+\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]\left[\widetilde{\ell}(x)-\ell_{0}(x)\right]$
$\displaystyle\geq\frac{\kappa_{R}^{2}}{4}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}-\underbrace{\sup_{v\in\mathcal{C}_{1}(S_{\ell}(x),3)\cap\mathbb{S}^{d-1}}\left|\frac{1}{4n}\sum_{i=1}^{n}\left\\{\pi(X_{i})\left(X_{i}^{T}v\right)^{2}-\mathrm{E}\left[\pi(X_{i})\left(X_{i}^{T}v\right)^{2}\right]\right\\}\right|\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}}_{\textbf{Term
I}}$
$\displaystyle\quad-\underbrace{\sup_{\begin{subarray}{c}u\in\mathcal{C}_{1}(S_{\ell}(x),3)\cap\mathbb{S}^{d-1}\\\
v\in\mathcal{C}_{2}(S_{\ell}(x),1)\cap\mathbb{S}^{d-1}\end{subarray}}\left|\frac{1}{2n}\sum_{i=1}^{n}\left\\{\pi(X_{i})\left(X_{i}^{T}v\right)\left(X_{i}^{T}u\right)-\mathrm{E}\left[\pi(X_{i})\left(X_{i}^{T}v\right)\left(X_{i}^{T}u\right)\right]\right\\}\right|\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}}_{\textbf{Term
II}}$
$\displaystyle\quad-\underbrace{\sup_{\begin{subarray}{c}u\in\mathcal{C}_{1}(S_{\ell}(x),3)\cap\mathbb{S}^{d-1}\\\
v\in\mathcal{C}_{2}(S_{\ell}(x),1)\cap\mathbb{S}^{d-1}\end{subarray}}\left|u^{T}\mathrm{E}\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]v\right|\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}}_{\textbf{Term
III}}.$

We now derive bounds on the remaining three terms. Recall our growth condition
$\sqrt{\frac{s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<1$.
Then, by Lemma 7, we know that with probability at least $1-\delta$,

$\displaystyle\textbf{Term I}+\textbf{Term
II}\lesssim\phi_{s_{\ell}(x)}\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}\left[\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}+r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}\right].$

Moreover, we know from Lemma 4 that there exist discrete sets
$M_{d,1},M_{d,2}\subset B_{d}(0,1)$ such that
$\mathcal{C}_{1}(S_{\ell}(x),3)\cap\mathbb{S}^{d-1}\subset
10\cdot\mathrm{conv}(M_{d,1})$ and
$\mathcal{C}_{2}(S_{\ell}(x),1)\cap\mathbb{S}^{d-1}\subset
6\cdot\mathrm{conv}(M_{d,2})$ with $\left|\left|u\right|\right|_{0}\leq
s_{\ell}(x),\left|\left|v\right|\right|_{0}\leq s_{\ell}(x)$ for all $u\in
M_{d,1},v\in M_{d,2}$. Hence, together with Lemma 5, we obtain that

Term III $\displaystyle\leq\sup_{\begin{subarray}{c}u\in
10\cdot\mathrm{conv}(M_{d,1})\\\ v\in
2(2+\vartheta_{\ell})\cdot\mathrm{conv}(M_{d,2})\end{subarray}}\left|u^{T}\mathrm{E}\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]v\right|\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}$
$\displaystyle=\sup_{\begin{subarray}{c}u\in 10\cdot M_{d,1}\\\ v\in 6\cdot
M_{d,2}\end{subarray}}\left|u^{T}\mathrm{E}\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]v\right|\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}$
$\displaystyle\lesssim\phi_{s_{\ell}(x)}\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}.$

Now, combining (38) and (39) with the above two bounds yields that there
exists an absolute constant $A_{0}>0$ such that with probability at least
$1-\delta$,

$\displaystyle\left[\frac{\kappa_{R}^{2}}{4}-A_{0}\phi_{s_{\ell}(x)}\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}\right]\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}$
$\displaystyle\quad-
A_{0}\left[\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+1\right]\phi_{s_{\ell}(x)}\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}$
$\displaystyle\leq\frac{6\gamma}{5n}\sqrt{s_{\ell}(x)}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}.$

By our growth condition
$\sqrt{\frac{s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<\min\left\\{\frac{\kappa_{R}^{2}}{A_{1}\phi_{s_{\ell}(x)}},1\right\\}$
with $A_{1}=8A_{0}$, it follows that

$\frac{\kappa_{R}^{2}}{4}-A_{0}\phi_{s_{\ell}(x)}\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}\geq\frac{\kappa_{R}^{2}}{8},$

and therefore,

$\displaystyle\frac{\kappa_{R}^{2}}{8}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}$
$\displaystyle\leq\frac{6\gamma}{5n}\sqrt{s_{\ell}(x)}+2A_{0}\left[\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+1\right]\phi_{s_{\ell}(x)}\cdot
r_{\ell}\cdot\frac{\left|\left|x\right|\right|_{2}}{\kappa_{R}^{2}}$
$\displaystyle\lesssim\frac{\gamma}{n}\sqrt{s_{\ell}(x)}+\phi_{s_{\ell}(x)}\cdot
r_{\ell}\cdot\frac{\left|\left|x\right|\right|_{2}}{\kappa_{R}^{2}}$

Finally, dividing $\frac{\kappa_{R}^{2}}{8}$ on both sides of the above
inequality gives us that

$\left|\left|\widehat{\ell}(x)-\widetilde{\ell}(x)\right|\right|_{2}\lesssim\frac{\gamma\sqrt{s_{\ell}(x)}}{n\cdot\kappa_{R}^{2}}+\frac{\phi_{s_{\ell}(x)}}{\kappa_{R}^{2}}\cdot\frac{\left|\left|x\right|\right|_{2}}{\kappa_{R}^{2}}\cdot
r_{\ell}.$

(b) By Lemma 11, we know that there exist absolute constants
$A_{2},A_{2}^{\prime}>0$ such that with probability at least $1-\delta$,

$\displaystyle\left|\left|\frac{1}{n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\ell_{0}(x)+2x\right|\right|_{\infty}$
$\displaystyle\leq
A_{2}^{\prime}\frac{\sqrt{\phi_{1}}}{\kappa_{R}}\left|\left|x\right|\right|_{2}\left[\sqrt{\frac{\log(d/\delta)}{n}}+\frac{\log(d/\delta)}{n}\right]$
$\displaystyle\leq
A_{2}\frac{\sqrt{\phi_{1}}}{\kappa_{R}}\left|\left|x\right|\right|_{2}\sqrt{\frac{\log(d/\delta)}{n}}\equiv\frac{\gamma}{n},$

where we note that $\sqrt{\frac{\log(d/\delta)}{n}}$ asymptotically dominates
the bound under our growth condition
$\sqrt{\frac{s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<1$.
Similarly, we obtain from Lemma 10 that there exists an absolute constant
$A_{3}>0$ such that with probability at least $1-\delta$,

$\displaystyle\sup_{v\in\mathcal{C}_{2}\left(S_{\ell}(x),\vartheta_{\ell}\right)\cap
B_{d}(0,1)}\left|\frac{1}{n}\sum_{i=1}^{n}\pi_{i}\left[X_{i}^{T}v\right]^{2}\right|$
$\displaystyle\leq\frac{2A_{3}}{13}\cdot\phi_{s_{\ell}(x)}\left[1+\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log\left(d/s_{\ell}(x)\right)}{n}}\right]$
$\displaystyle\leq\frac{2A_{3}}{13}\cdot\phi_{s_{\ell}(x)}\equiv\frac{\nu}{n}.$

Hence, with probability at least $1-\delta$,

$\frac{\kappa_{R}^{2}}{\left|\left|x\right|\right|_{2}}\sqrt{\frac{2\gamma}{13\nu}}\geq\left(\frac{A_{2}^{2}\,\kappa_{R}^{6}\,\phi_{1}}{A_{3}^{2}\left|\left|x\right|\right|_{2}\phi_{s_{\ell}(x)}}\right)^{\frac{1}{4}}\left(\frac{\log(d/\delta)}{n}\right)^{\frac{1}{4}}>r_{\ell},$

where the last inequality follows from our condition. Therefore, the
sufficient conditions in (a) hold with probability at least $1-\delta$.
Finally, the bound on
$\left|\left|\widehat{\ell}(x)-\widetilde{\ell}(x)\right|\right|_{2}$ follows
from substituting $\frac{\gamma}{n}\equiv
A_{2}\frac{\sqrt{\phi_{1}}}{\kappa_{R}}\left|\left|x\right|\right|_{2}\sqrt{\frac{\log(d/\delta)}{n}}$
into the upper bound of
$\left|\left|\widehat{\ell}(x)-\widetilde{\ell}(x)\right|\right|_{2}$ in (a).
∎

With the above lemma, we now formally prove the consistency of the dual
solution $\widehat{\ell}(x)$ (Theorem 2). Notice that the major difference
between the proof of Lemma 1 and Theorem 2 is that the propensity scores now
are estimated in Theorem 2.

######  Theorem 4 (Consistency of the dual solution $\widehat{\ell}(x)$).

Let $\delta\in(0,1)$, $x\in\mathbb{R}^{d}$ be a fixed covariate vector, and
$A_{1},A_{2},A_{3}>0$ be some large absolute constants. Suppose that
Assumptions 1, 2, 4, 5, and 6 hold as well as

$\sqrt{\frac{s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}+r_{\pi}<\min\left\\{\frac{\kappa_{R}^{2}}{A_{1}\phi_{s_{\ell}(x)}},1\right\\}\quad\text{
and }\quad d\geq s_{\ell}(x)+2.$

  1. (a)

Suppose that
$\gamma\geq\left|\left|\frac{5}{2}\sum_{i=1}^{n}\widehat{\pi}_{i}X_{i}X_{i}^{T}\ell_{0}(x)+5nx\right|\right|_{\infty}$
and
$r_{\ell}<\min\left\\{\frac{\kappa_{R}^{2}}{\left|\left|x\right|\right|_{2}}\sqrt{\frac{2\gamma}{13\nu}},\,\frac{1}{2}\right\\}$,
where $\nu>0$ is defined in (58) of Lemma 9. Then, with probability at least
$1-\delta$,

$\left|\left|\widehat{\ell}(x)-\widetilde{\ell}(x)\right|\right|_{2}\lesssim\frac{\gamma\sqrt{s_{\ell}(x)}}{n\cdot\kappa_{R}^{2}}+\frac{\phi_{s_{\ell}(x)}\left|\left|x\right|\right|_{2}}{\kappa_{R}^{4}}\cdot
r_{\ell}.$

  2. (b)

We denote $r_{\gamma}\equiv
r_{\gamma}(n,\delta)=\frac{\sqrt{\phi_{1}}}{\kappa_{R}}\left|\left|x\right|\right|_{2}\sqrt{\frac{\log(d/\delta)}{n}}+\frac{\sqrt{\phi_{1}\phi_{s_{\ell}(x)}}\left|\left|x\right|\right|_{2}}{\kappa_{R}^{2}}\cdot
r_{\pi}(n,\delta)$. If, in addition, $\frac{\gamma}{n}=A_{2}\cdot r_{\gamma}$
and

$r_{\ell}\leq\min\left\\{\frac{1}{2},\,\left[\left(\frac{A_{2}^{2}\,\kappa_{R}^{6}\,\phi_{1}}{A_{3}^{2}\left|\left|x\right|\right|_{2}\phi_{s_{\ell}(x)}}\right)^{\frac{1}{4}}\left(\frac{\log(d/\delta)}{n}\right)^{\frac{1}{4}}+\left(\frac{A_{2}^{2}\phi_{1}\kappa_{R}^{4}}{A_{3}^{2}\phi_{s_{\ell}(x)}\left|\left|x\right|\right|_{2}^{2}}\right)^{\frac{1}{4}}\sqrt{r_{\pi}}\right]\right\\},$

then with probability at least $1-\delta$,

$\displaystyle\left|\left|\widehat{\ell}(x)-\widetilde{\ell}(x)\right|\right|_{2}$
$\displaystyle\lesssim\frac{\left|\left|x\right|\right|_{2}}{\kappa_{R}^{3}}\left[\sqrt{\frac{\phi_{1}s_{\ell}(x)\log(d/\delta)}{n}}+\frac{\phi_{s_{\ell}(x)}}{\kappa_{R}}\cdot
r_{\ell}+\frac{\sqrt{\phi_{1}\phi_{s_{\ell}(x)}s_{\ell}(x)}}{\kappa_{R}}\cdot
r_{\pi}\right].$

###### Proof of Theorem 2.

(a) Since $\widehat{\ell}(x)$ minimizes the objective function of the dual
problem (6), we know that

$\frac{1}{4n}\sum_{i=1}^{n}\widehat{\pi}_{i}\left[X_{i}^{T}\widehat{\ell}(x)\right]^{2}+x^{T}\widehat{\ell}(x)+\frac{\gamma}{n}\left|\left|\widehat{\ell}(x)\right|\right|_{1}\leq\frac{1}{4n}\sum_{i=1}^{n}\widehat{\pi}_{i}\left[X_{i}^{T}\widetilde{\ell}(x)\right]^{2}+x^{T}\widetilde{\ell}(x)+\frac{\gamma}{n}\left|\left|\widetilde{\ell}(x)\right|\right|_{1}.$

Some rearrangements (or using Taylor’s expansion) give us that

$\displaystyle\begin{split}&\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{4n}\sum_{i=1}^{n}\widehat{\pi}_{i}X_{i}X_{i}^{T}\right]\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]+\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{2n}\sum_{i=1}^{n}\widehat{\pi}_{i}X_{i}X_{i}^{T}\right]\left[\widetilde{\ell}(x)-\ell_{0}(x)\right]\\\
&\leq\left[\frac{1}{2n}\sum_{i=1}^{n}\widehat{\pi}_{i}X_{i}X_{i}^{T}\ell_{0}(x)+x\right]^{T}\left[\widetilde{\ell}(x)-\widehat{\ell}(x)\right]+\frac{\gamma}{n}\left[\left|\left|\widetilde{\ell}(x)\right|\right|_{1}-\left|\left|\widehat{\ell}(x)\right|\right|_{1}\right].\end{split}$
(40)

We now reduce the above inequality to (38) in Lemma 1 so that we can apply
parts of the results in Lemma 1. Some algebra show that the above inequality
is equivalent to

$\displaystyle\begin{split}&\underbrace{\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{4n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]+\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{2n}\sum_{i=1}^{n}\pi(X_{i})X_{i}X_{i}^{T}\right]\left[\widetilde{\ell}(x)-\ell_{0}(x)\right]}_{\textbf{Term
I}}\\\
&\quad+\underbrace{\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{4n}\sum_{i=1}^{n}\left(\widehat{\pi}_{i}-\pi_{i}\right)X_{i}X_{i}^{T}\right]\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]}_{\textbf{Term
II}}\\\
&\quad+\underbrace{\left[\widehat{\ell}(x)-\widetilde{\ell}(x)\right]^{T}\left[\frac{1}{2n}\sum_{i=1}^{n}\left(\widehat{\pi}_{i}-\pi_{i}\right)X_{i}X_{i}^{T}\right]\left[\widetilde{\ell}(x)-\ell_{0}(x)\right]}_{\textbf{Term
III}}\\\
&\leq\left[\frac{1}{2n}\sum_{i=1}^{n}\widehat{\pi}_{i}X_{i}X_{i}^{T}\ell_{0}(x)+x\right]^{T}\left[\widetilde{\ell}(x)-\widehat{\ell}(x)\right]+\frac{\gamma}{n}\left[\left|\left|\widetilde{\ell}(x)\right|\right|_{1}-\left|\left|\widehat{\ell}(x)\right|\right|_{1}\right].\end{split}$
(41)

We first obtain an upper bound on the right-hand side of the above inequality
via the same arguments as in the proof of Lemma 1. We write
$\widehat{\Delta}(x)=\widehat{\ell}(x)-\widetilde{\ell}(x)$. By Hölder’s
inequality and the (reverse) triangle’s inequality together with the sparsity
of $\widetilde{\ell}(x)$ under Assumption 6, the right-hand side of (40) is
upper bounded by

$\displaystyle\left[\frac{1}{2n}\sum_{i=1}^{n}\widehat{\pi}_{i}X_{i}X_{i}^{T}\ell_{0}(x)+x\right]^{T}\left[\widetilde{\ell}(x)-\widehat{\ell}(x)\right]+\frac{\gamma}{n}\left[\left|\left|\widetilde{\ell}(x)\right|\right|_{1}-\left|\left|\widehat{\ell}(x)\right|\right|_{1}\right]$
$\displaystyle\leq\left|\left|\frac{1}{2n}\sum_{i=1}^{n}\widehat{\pi}_{i}X_{i}X_{i}^{T}\ell_{0}(x)+x\right|\right|_{\infty}\left|\left|\widehat{\Delta}(x)\right|\right|_{1}$
$\displaystyle\quad+\frac{\gamma}{n}\left[\left|\left|\left[\widetilde{\ell}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}-\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}+\left[\widetilde{\ell}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}-\left|\left|\left[\widehat{\ell}(x)\right]_{S_{\ell}(x)^{c}}\right|\right|_{1}\right]$
$\displaystyle\leq\frac{\gamma}{5n}\left[\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}+\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)^{c}}\right|\right|_{1}\right]+\frac{\gamma}{n}\left[\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}-\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)^{c}}\right|\right|_{1}\right]$
$\displaystyle\leq\frac{6\gamma}{5n}\sqrt{s_{\ell}(x)}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}$

where
$\left|\left|\left[\widehat{\Delta}(x)\right]_{S_{\ell}(x)}\right|\right|_{1}=\sum_{k\in
S_{\ell}(x)}\left|\widehat{\Delta}_{k}(x)\right|$.

Next, we derive lower bounds for the terms on the left-hand side of (41). We
know from Lemma 9 by $a_{0}=5$ that

$\widehat{\ell}(x)-\widetilde{\ell}(x)\in\mathcal{C}_{1}\left(S_{\ell}(x),3\right)\bigcup
B_{d}^{(1)}\left(0,\,\frac{13r_{\ell}^{2}\left|\left|x\right|\right|_{2}^{2}}{2\kappa_{R}^{4}}\cdot\frac{\nu}{\gamma}\right),$

where
$B_{d}^{(1)}(0,r)=\left\\{v\in\mathbb{R}^{d}:\left|\left|v\right|\right|_{1}\leq
r\right\\}$. By our condition
$r_{\ell}<\frac{\kappa_{R}^{2}}{\left|\left|x\right|\right|_{2}}\sqrt{\frac{2\gamma}{13\nu}}$,
we have that

$\left[\mathcal{C}_{1}\left(S_{\ell}(x),3\right)\bigcup
B_{d}^{(1)}\left(0,\,\frac{13r_{\ell}^{2}\left|\left|x\right|\right|_{2}^{2}}{2\kappa_{R}^{4}}\cdot\frac{\nu}{\gamma}\right)\right]\bigcap\mathbb{S}^{d-1}=\mathcal{C}_{1}\left(S_{\ell}(x),3\right)\cap\mathbb{S}^{d-1}.$
(42)

$\bullet$ Term I: This term coincides with the left-hand side of (38) in the
proof of Lemma 1. Thus, we conclude that there exists absolute constant
$A,A_{0}>0$ such that with probability at least $1-\delta$,

Term I
$\displaystyle\geq\left[\frac{\kappa_{R}^{2}}{4}-A\phi_{s_{\ell}(x)}\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}\right]\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}$
$\displaystyle\quad-A\left[\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+1\right]\phi_{s_{\ell}(x)}\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}$
$\displaystyle\geq\left[\frac{\kappa_{R}^{2}}{4}-A_{0}\phi_{s_{\ell}(x)}\sqrt{\frac{\log(1/\delta)+s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}\right]\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}-A_{0}\phi_{s_{\ell}(x)}\cdot
r_{\ell}\left|\left|\ell_{0}(x)\right|\right|_{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2},$

where we use our growth condition
$\sqrt{\frac{s_{\ell}(x)\log(ed/s_{\ell}(x))}{n}}+\sqrt{\frac{\log(1/\delta)}{n}}<1$
to obtain the second inequality.

$\bullet$ Term II: By (42) and Lemma 7, we have that with probability at least
$1-\delta$,

$\displaystyle\left|\textbf{Term II}\right|$
$\displaystyle\leq\sup_{v\in\mathcal{C}_{1}(S_{\ell},3)\cap\mathbb{S}^{d-1}}\frac{1}{4n}\sum_{i=1}^{n}\left|\widehat{\pi}_{i}-\pi_{i}\right|\left(X_{i}^{T}v\right)^{2}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}$
$\displaystyle\leq\sup_{v\in\mathcal{C}_{1}(S_{\ell},3)\cap\mathbb{S}^{d-1}}\frac{r_{\pi}(n,\delta)}{4n}\sum_{i=1}^{n}\left\\{\left(X_{i}^{T}v\right)^{2}-\mathrm{E}\left[\left(X_{i}^{T}v\right)^{2}\right]\right\\}\left|\left|\widehat{\Delta}(x)\right|\right|_{2}^{2}$