In this work, we propose a Semi-supervised Triply Robust Inductive transFer LEarning (STRIFLE) approach, which integrates heterogeneous data from label rich source population and label scarce target population to improve the learning accuracy in the target population.
Specifically, we consider a high dimensional covariate shift setting
and employ two nuisance models, a density ratio model and an imputation model, to combine transfer learning and surrogate-assisted semi-supervised learning strategies organically and achieve triple robustness. While the STRIFLE approach requires the target and source populations to share the same conditional distribution of outcome $Y$ given both the surrogate features $\bS$ and predictors $\X$, it allows the true underlying model of $Y \mid \X$ to differ between the two populations due to the potential covariate shift in $\bS$ and $\X$. Different from double robustness, even if both nuisance models are misspecified or the distribution of $Y\mid \bS,\X$ is not the same between the two populations, when the transferred source population and the target population share enough similarities, the triply robust STRIFLE estimator can still partially utilize the source population, and it is guaranteed to be no worse than the target-only surrogate-assisted semi-supervised estimator with negligible errors.
These desirable properties of our estimator are established theoretically and verified in finite-sample via extensive simulation studies. We utilize the STRIFLE estimator to train a Type II diabetes polygenic risk prediction model for the African American target population by transferring knowledge from electronic health records linked genomic data observed in a larger European source population.
Keywords: Covariate shift, surrogate-assisted semi-supervised learning, high dimensional data, inductive transfer learning, model misspecification, triple robustness.

§ INTRODUCTION

§.§ Background

A precision medicine approach is vitally important for clinical care <cit.>.
Reliable and precise personalized prediction of clinical outcome and treatment response is fundamental for successful disease management. Patients with worse progression profiles can be monitored more intensively. More aggressive treatments should be considered for those unlikely to respond to standard therapy while less toxic options may be considered for patients with a good prognosis. Precision medicine approaches are increasingly being incorporated into clinical care.
Advances in genetics, and the growing availability of electronic health records (EHR) and health surveys, present increased  opportunity to develop accurate personalized risks prediction models and
incorporate such models at the bedside to make precise personalized patient care a clinical reality <cit.>.

Personalized risk prediction modeling, however, faces several challenges in practice.
The first challenge is the imbalance and heterogeneity across populations.
Current prevalent predictive modeling schemes are prone to biases when naively applied to noisy and heterogeneous EHR data. Most prediction models are trained to optimize overall accuracy, which may not reflect accuracy within under-represented subgroups. Many factors could contribute to these performance disparities, including different levels of data completeness, data distribution discrepancies, and coding biases.

Many demographic sub-populations
are observed to be underrepresented in large-scale biomedical research, which may lead to less accurate clinical implementation in those sub-populations <cit.>. For example, around
$79\%$ of all GWAS participants are of European descent and clinical treatments of polygenic risk scores is several times more accurate in individuals of European ancestry than other ancestries <cit.>, due to genetic differences across ancestral populations <cit.>.

In general,  prediction models trained to optimize overall accuracy may lead to poor performance in minority populations, in the presence of heterogeneity.
These differences are particularly prevalent in EHR linked studies with additional contributing factors mentioned above.
The presence heterogeneity and the relatively low sample size of the under-represented populations highlights the need for robust transfer learning techniques to effectively improve the prediction accuracy in the underrepresented sub-populations and achieve fairness in precision medicine.

The second challenge arises from the lack of direct observation on the clinical outcomes of interest for training supervised prediction models. Extracting precise clinical outcome information requires resource consuming manual chart review which is not scalable. On the other hand, readily available proxies such as diagnosis codes do not accurately reflect the true outcome information. To efficiently leverage both the limited labeled observations and the imperfect surrogates, it is critical to incorporate semi-supervised (SS) learning strategies to combine information from both labeled and unlabeled observations. The third challenge comes from the high dimensionality of the feature space. Both the number of candidate predictors for the prediction model, such as genetic markers, and the number of candidate surrogates can be large.

Much progress has been made in transfer learning and SS learning in recent years, both in low and high dimensional settings. However, there is a paucity of literature on robust transfer learning under SS settings to  leverage large sized unlabeled source and target data while the label sizes are small. We aim to address this gap by developing a Semi-supervised Triply Robust Inductive transFer LEarning (STRIFLE) method that effectively leverages such data structure to improve the estimation and  prediction accuracy of risk modeling in the underrepresented population. Specifically, we consider a covariate shift setting, which allows the predictors, $\X$, and the surrogates, $\S$, to have different distributions in the target population and the source population; but assume that the two populations share the same conditional distribution of the outcome $Y$ given $\S$ and $\X$. In both populations, we have a small number of labeled observations and a large number of unlabeled observations, respectively. Further, the number of the candidate predictors and the surrogates can be much larger than the number of labeled observations.

§.§ Related Literature and Our Contributions

Our work is closely related to several tracks of recent literature in statistical learning. In this section, we shall introduce them separately and summarize our methodological novelty and contribution in view of these existing literature.

Surrogate-assisted semi-supervised learning. In recent years, extensive research has been conducted on SS learning that makes use of the large amount of unlabeled data to improve the estimation and prediction accuracy with the small amount of labeled data <cit.>. Moreover, in biomedical and EHR studies, highly predictive surrogates of the disease status, such as the diagnostic codes, medication prescriptions, and laboratory tests are readily available for the unlabeled observations. To improve the learning accuracy utilizing the surrogate EHR features as “silver labels", <cit.> proposed a robust surrogate-assisted semi-supervised (SAS) method that imputes the unobserved outcome (e.g. the disease status) by utilizing the candidate predictors (e.g. genetic variants) as well as the surrogates via a sparse working imputation model. Interestingly, their framework allows the underlying risk prediction model to be dense. <cit.> proposed an SS method that adaptively incorporates prior knowledge from EHR surrogates by shrinking the high dimensional sparse estimator towards a direction derived from the unlabelled data with surrogates. However, none of these existing methods can handle covariate shift between the source population and the target population, which could lead to biased or inefficient estimation of the target parameters.

Covariate shift correction. Distributional shift of the covariates between the training (source) and testing (target) data has been frequently studied in recent years. While importance weighting <cit.>, the most common and natural way to adjust for covariate shift, is not robust to misspecified models or excessive estimation errors of the density ratio model. In recent years, methods has been developed for doubly robust covariate shift correction <cit.>. Motivated by semiparametric theory, they introduce an imputation model for $Y$ to augment the importance weighted estimating equations and derive an estimator that is valid when either the density ratio or the imputation model is correct. The main limitation of this strategy is that it purely relies on labels from the source population and fails to give a valid estimation when both models are misspecified, or even worse, when the conditional model of $Y$ is different between the source and target populations. Under the scenario considered in our paper, there is a space to overcome this limitation due to the availability of a small amount of labeled data from the target population.

Knowledge transferring regression.

We shall review some related recent advancement on knowledge transferring with labels for both target and source populations. Recently, <cit.> and <cit.> proposed trans-Lasso, a transfer learning method for high dimensional linear models and generalized linear models (GLM) with Lasso penalty <cit.> to improve the learning accuracy on target population by adaptively borrowing information from potentially transferable source data.
Their core idea is to shrink source estimators learned using source labeled data towards target parameters through Lasso penalties.
Based on this trans-Lasso framework, <cit.> proposed a federated transfer learning approach that integrates heterogeneous data from diverse populations and multiple healthcare institutions. Different from us, existing methods on trans-Lasso neither adjust for covariate shift between the source and target data nor incorporate unlabeled target observations with surrogates. As will be verified in our asymptotic and numerical studies, this could cause a large efficiency loss in our setting.

Our contribution. Our proposed STRIFLE method has two distinctive features, i.e., it is both surrogate-assisted and model-assisted. Assisted by predictive surrogates, we employ a working and nuisance imputation model under certain moment constraints to impute the outcomes for the unlabeled target observations. To integrate the label-rich source population, we introduce a working and nuisance density ratio model embedded in a penalized transfer learning procedure when estimating the imputation model.
The novel embedded density ratio model, estimated using unlabeled observations from the two populations, can potentially increase the transferability of the source population.

Naive combination of transfer learning and SS learning is vulnerable to model misspecification and may lead to erroneous results practically.

When both nuisance models are misspecified, the estimation consistency is no longer guaranteed.

Inspired by trans-Lasso <cit.>,
we employ the two nuisance models as bridges to combine the transfer learning and the SAS techniques organically and achieve triple robustness with a bias correction step.
That is, under the covariate shift setting,  when at least one of the two nuisance models are correctly specified, our proposed STRIFLE method will be able to fully utilize the labeled observations from the  source population; even if both of the nuisance models are misspecified or the conditional distribution of  $Y$ given $\S$ and $\X$ differs across populations, when the transferred source population based on the working density ratio model shares enough similarities with the target population, STRIFLE can still partially utilize source labeled data to boost statistical efficiency. Moreover, STRIFLE is an adaptive transferring approach in the sense that it is guaranteed to be no worse than the target-only SAS estimator with negligible errors under mild conditions.
In addition, our framework is not restricted to Lasso penalty, and allows the use of non-convex penalties such as SCAD <cit.> and MCP<cit.>.
To the best of our knowledge, this is the first work that utilizes useful information from the source population and leverages unlabeled observations from the target population simultaneously and achieves triple robustness.

§.§ Outline of the Paper

The rest of the paper is organized as follows. We introduce the problem setup and an overview of the STRIFLE algorithm and present detailed methodology in Section <ref>, establish theoretical properties of our STRIFLE estimator in Section <ref>, examine its finite-sample performance through simulation studies in Section <ref>, and apply the STRIFLE algorithm to develop a genetic risk prediction model for Type II Diabetes (T2D) in African American (AA) population based on single nucleotide polymorphisms (SNPs) and EHR features with the European population as the source in Section <ref>. The paper is concluded with some discussions in Section  <ref>.

§  METHODOLOGY

§.§ Problem Setup

For the $i$-th observation, $Y_i \in \mR$ denotes the outcome variable,  $\X_i\in \mR^p$ denotes the $p$-dimensional candidate predictors including the intercept term as its first element, and $\S_i \in \mR^q$ denotes the $q$-dimensional surrogates for $Y_i$.

We let $\mT$ and $\mS$ denote the target and source populations; use $L$ and $U$ to index the labeled and unlabeled subsets; and $\mI_{\iota,\jmath}$ denote the index set of observations from target/source labeled/unlabeled populations with $\iota \in \{\mT,\mS\}$ and $\jmath \in \{L,U\}$. We assume that $Y$ is only observed in the labeled subsets while $\Z = (\X\trans, \S\trans)\trans$ is observed in both labeled and unlabeled data. We let
$\Dscr\submT = \Dscr\submTL \cup \Dscr\submTU$ and $\Dscr\submS = \Dscr\submSL \cup \Dscr\submSU$ denote the observed data from the target and the source populations with

\begin{align*}
\Dscr\subiotaL &= \{(Y_i,\Z_i\trans)\trans, i \in \mI\subiotaL\}, \quad \Dscr\subiotaU = \{\Z_i, i \in \mI\subiotaU\} , \quad \iota = \mT, \mS.
\end{align*}

Let $|\mathcal{I}\submTL| = n\submT$, $|\mathcal{I}\submTU| = N\submT$, $|\mathcal{I}_{\mS, L}| = n\submS$,  $|\mathcal{I}_{\mS, U}| = N\submS$.
Let $d = p+q$, $n = n\submS +n\submT$, and $N = N\submS + N\submT$.
We consider the high dimensional setting where $p$ and $q$ grow with $n$ and can be much larger than $n\submT$ and $n$.
In terms of sample sizes, we consider a general setting with only one assumption, $n\submT \leq N\submT$.

Let $p\supiota_\Z$, $p\supiota_{Y\mid \Z}$ , $p\supiota_{\S\mid \X} $, and $p\supiota_{Y\mid \X}$  denote the probability density of $\Z_i$, the conditional density of $Y_i$ given $\Z_i$,  the conditional density of $\S_i$ given $\X_i$, and the conditional density of $Y_i$ given $\X_i$, respectively, in the population $\iota$, for $\iota \in \{\mT,\mS\}$.
We consider covariate shift structure in this work, that is, we assume
$$p\supmS_{Y\mid \Z}(y, \z) = p\supmT_{Y\mid \Z}(y, \z)$$

while allowing  $p\supmS_\Z(\z)$ to be different from $p\supmT_\Z(\z)$.
we allow $p\supmS_{\S\mid \X} \neq p\supmT_{\S\mid \X}$ and consequently, $p\supmS_{Y\mid \X}$ can be different from  $p\supmT_{Y\mid \X}$.
We let  $E\subiota(\cdot)$ denote the expectation taken over the distribution in the population $\iota \in \{\mS,\mT\}$.

We are interested in developing a prediction model for $Y_i \mid \X_i$ in $\mT$ under a working model $\ET(Y \mid \X)=g(\X\trans\bb)$, where $g(\cdot)$ is a known link function. Specifically, our goal is to estimate $\bb_0 \in \mR^p$, the solution to

[_i {Y_i - g(_i)}]=.
We do not require the outcome model $\ET(Y \mid \X)=g(\X\trans\bb)$ to hold and
allow $\bb_0$  to be dense.
Although $\S_i$ can be strongly informative of $Y_i$,  the outcome prediction model in (<ref>) does not include $\S_i$, because they represent noisy proxies of $Y_i$ and cannot be used as predictors.
In our motivating example of developing a polygenic risk prediction  model for T2D, $Y$ is the T2D status, $\S$ includes mentions of laboratory tests and medication prescriptions related to T2d in clinical notes,  while $\X$ includes demographic variables and genetic markers. While $\S$ can be used to impute the missing $Y$ to improve the precision in estimating $\bb$, these variables cannot be used as predictors.

§.§ Overview of the STRIFLE Algorithm

We first give an overview of the proposed STRIFLE procedure. To leverage the unlabeled data and the surrogates, STRIFLE employs a series of robust imputation strategies under a working imputation model for $Y \mid \Z$

m(\Z) = \ET(Y \mid \Z) = \ES(Y \mid\Z) = g(\btheta\trans\Z) .

For any fitted imputation model $\mhat(\Z) = g(\Z\trans \bthetahat)$ with limiting $\bthetabar$ for $\bthetahat$ and $\mbar(\Z) = g(\Z\trans \bar\btheta)$, we may construct a  SS estimator for $\bb_0$ as
() = min_ ∈Ω_ _N+n^(; ) + p_λ_().

_N+n^(;) = -1/N+ n [∑_i∈ℐ Q{g(_i), g(_i)} + ∑_i∈ℐ Q{g(_i), y_i}],
$Q(\cdot, \cdot)$ is the  log-(quasi-)likelihood function (after dropping any dispersion parameter), i.e.,

\[
Q\{g(\x_i\trans\bb),\widehat m(\z_i)\} = \widehat m(\z_i) \x_i\trans \bb - G(\x_i\trans \bb)   \ \ \text{and} \ \ Q\{g(\x_i\trans\bb), y_i\} = y_i \x_i\trans \bb - G(\x_i\trans \bb),
\]

$G(u) = \int_0^u g(u^\prime) du^\prime$, and  $p_{\lambda_{\bb}}(\bb)$ is a penalty function on $\bb$ with a tuning parameter $\lambda_\bb$.

It is not difficult to show that $\wh\bb$ is consistent for $\bb_0$ provided that

E[_i { Y_i - (_i)}] =  .
A simple approach to ensuring consistency of $\wh\bb$ is to directly estimate $\btheta$ using target labeled data as in <cit.>. However, such a target-only approach does not leverage the label rich source data. On the other hand, since $p_{Y\mid \Z}^{(\mS)} = p_{Y\mid \Z}\supmT$ yet $p_{Y\mid \X}^{(\mS)}$ can be different from $p_{Y\mid \X}\supmT$, the learning accuracy w.r.t $\bb_0$ can be improved if we can borrow useful information from $\mS$ when estimating $m(\z)$. Thus the proposed STRIFLE algorithm centers its strategy on robust and efficient estimation of $\btheta$. The STRIFLE estimation of $\btheta$ consists of three key components:

* fitting a density ratio model to estimate $w(\Z) =p\supmT_\Z/ p\supmS_\Z$ as $\what(\Z)$;

* constructing an initial target only  estimation of $\btheta$ as $\bthetahat\supmT$ as well as a
meta-learning of the imputation model using $\Dscr\submT\cup \Dscr\submS$ to obtain $\bthetahat\supmeta$;

* constructing the STRIFLE estimator for $\btheta$, $\bthetahat\supSTRIFLE$, as a convex combination of $\bthetahat\supmT$ and $\bthetahat\supmeta$ to overcome negative transfer under possible model mis-specifications.

The final STRIFLE estimator for $\bb$ is then obtained as

\wh\bb\supSTRIFLE = \wh\bb(\bthetahat\supSTRIFLE)

To further mitigate excessive bias under the high dimensional setting, we also employ cross-fitting strategies when performing the convex combination above. We detail each of the components below.

§.§ Estimation of the Density Ratio Model

To potentially increase the transferability of the source population and efficiently leverage the source data, we fit a working density ratio model utilizing the large number of unlabeled observations from both populations for
$p\supmT_\Z(\z)/ p\supmS_\Z(\z)$:

\begin{equation}
w(\z) = \exp\{ \f(\z)\trans\bz\} , \label{model-DR}
\end{equation}

where $\f(\cdot)$ is a vector of pre-specified basis functions to allow for non-linear effects. If $w(\z)$ is correctly specified, we have $\ES\{w(\Z_i) \f(\Z_i)\} = \ET\{\f(\Z_i)\}$. We define the population parameter $\bz_0$ as
_0 = min_ E[exp{(_i)}]- E_{(_i)}.
Since $\bz_0$ can be estimated with unlabeled data which is of large size, many standard estimators can be used to derive an estimator with desirable convergence rate. In our theoretical derivations, we do not restrict to any specific estimator provided that its convergence rate satisfies Assumption <ref> given in Section 3. One such estimator for $\bz_0$ can be constructed as

\begin{align}
\label{eq:est_w}
\wh \bz = \arg \min_{\bz \in \Omega_{\bz}} \mL_{N}^w(\bz)  +  p_{ \lambda_\bz}(\bz),
\end{align}

\[
\mL_{N}^w(\bz) =\frac{1}{N_{\mS}} \sum_{i\in  \mathcal{I}_{\mS, U}}\exp\{\f(\z_i)\trans\bz\} - \frac{1}{N\submT} \sum_{i\in \mathcal{I}\submTU} \f(\z_i)\trans\bz,
\]

and $p_{ \lambda_\bz}(\bz)$ is a penalty function with a tuning parameter $\lambda_\bz$.
One may choose $p_{ \lambda_\bz}(\bz)$ as commonly used penalty functions including the ridge, Lasso, SCAD, etc.
Here and after, with a slight abuse of notation, we use $p_\lambda(\cdot)$ to denote different types of penalty functions with a tuning parameter $\lambda$.
We show in Appendix <ref> that $\wh \bz$ given in (<ref>) with a LASSO penalty satisfies Assumption <ref>.

§.§ Target-only Estimation and Meta-learning of the Imputation Model

We next construct two estimators for $\btheta$ in $m(\z)$, one using $\Dscr\submTL$ only and one via meta-learning combining $\Dscr\submT$ and $\Dscr\submS$. These estimators are derived to achieve both robustness and efficiency of the estimation of the target parameter $\bb_0$ in the presence of potential model misspecifications. To this end, we first provide two key observations pertaining to estimation of  $m(\z)$.
First, provided that the limiting imputation model $\bar m(\Z)$ satisfies the moment condition (<ref>), we have
$E\submT [\X_i \{\mbar(\Z)  - g(\X_i\trans \bb_0)\}] = \0$,
and can utilize unlabeled observations from target to consistently estimate $\bb_0$. Thus, with target data only, we propose to estimate $\btheta$ as

= min_∈Ω_
Q{ g(_i), y_i}  + p_λ()],
where $p_{\lambda}(\btheta)$ is a penalty function on $\btheta$ with a tuning parameter $\lambda$ such that $p_{\lambda}(\btheta)\to 0$ as $n\submT \to \infty$. Under standard regularity conditions in high dimensional settings,  $\bthetahat$ converges to

:= max_ E[	Q{g(_i), Y_i } ],
and the resulting $\mbar(\z)=g(\z\trans\bthetabar)$ satisfies  the moment condition (<ref>), even if the imputation model fails to hold. Using $\bthetahat\supmT$ in (<ref>) leads to SAS estimator for $\bb_0$.

The second key observation is that, under our covariate shift setting and the moment condition (<ref>),  when the imputation model or the density ratio model is correctly specified,  we have

\[
E\submS[w_0(\Z_i) \Z_i  \{ Y_i -\bar m(\Z_i) \}] = \0.
\]

We further define

\begin{align}
\label{eq:theta*}
\btheta^* =\arg\max_{\btheta}  \left\{\frac{n\submS}{n}E\submS \left[w_0(\Z_i) Q\left\{ g(\Z_i\trans\btheta), Y_i\right\} \right] + \frac{n\submT}{n}E\submT \left[ Q\left\{ g(\Z_i\trans\btheta), Y_i\right\}\right]\right\},
\end{align}

and $m^*(\z) = g(\z\trans\btheta^*)$.
When either $m(\z)$ or $w(\z)$ is correctly specified, $\btheta^* = \bthetabar$ and one may estimate $\bthetabar$ by maximizing an empirical version of (<ref>) utilizing the source labeled observations to derive a doubly robust estimator for $\bthetabar$.
However, when both models are  mis-specified, $\btheta^* \neq \bthetabar$ and such an approach could lead to a biased estimator for $\bthetabar$.

Inspired by the idea of oracle Trans-Lasso <cit.>,
we propose a two-step density-ratio-embedded meta-learning algorithm to estimate $\bthetabar$ by correcting the potential bias $\bdel_0 = \bthetabar - \btheta^*$, as summarized in Algorithm <ref>.
Different from oracle Trans-Lasso, we deal with a more complex loss function with an estimated working density ratio embedded in and allow non-convex penalties.
With large $N\submS$ and $N\submT$, $w(\z)$ can be estimated precisely, which in consequence can increase the transferability of source population.
Our proof framework allows broader class of non-convex loss functions and details are given in Section <ref>.

Robust meta-learning of $\bthetabar$

1.  Using $\Dscr\submSL\cup \Dscr\submTL$ along with $\wh w(\z)$, we estimate $\btheta^*$ as

\[
\wt \btheta  = \arg \min_{\btheta \in \Omega_\btheta}\mL_n(\btheta; \wh \bz) + p_{\lambda_{\btheta}}(\btheta),
\]

\begin{equation}
\label{eq:losstheta*}
\mL_n(\btheta; \bz)
=- \frac{1}{n}\sum_{i\in \mathcal{I}\submSL\cup \mathcal{I}\submTL}w(\z_i; \bz)^{I(i\in \mathcal{I}\submSL)}Q\left\{g(\btheta\trans\z_i), y_i \right\},
\end{equation}

$I(i\in \mathcal{I}_{\mS, L})$ is an indicator function, and $p_{\lambda_{\btheta}}(\btheta)$
is a penalty function on $\btheta$ with a tuning parameter $\lambda_\btheta$.

2. Estimate $\bdel_0$ using  $\Dscr\submTL$ with $\wt \btheta$ as

\[
\wh \bdel = \arg\min_{\bdel \in \Omega_{\bdel}}
\mL_{n\submT}\supmT(\wt \btheta + \bdel)  + p_{\lambda_{\bdel}} (\bdel),
\]

to construct the meta-learning estimator  is $\bthetahat\supmeta = \wt \btheta + \wh \bdel$, where

\begin{equation}
\mL_{n\submT}\supmT(\wt \btheta + \bdel) :=  -\frac{1}{n\submT} \sum_{i\in \mathcal{I}\submTL} Q[g\{ \z_i\trans (\wt \btheta+  \bdel)\},y_i],
\end{equation}

and $p_{\lambda_{\bdel}}(\bdel)$ is a penalty function on $\bdel$ with a tuning parameter $\lambda_\bdel$.

*	With a slight abuse of notation, $p_{\lambda_\btheta}$ and $p_{\lambda_{\bdel}}$ can be different penalty functions with tuning parameter $\lambda_\btheta$ and $\lambda_{\bdel}$, respectively.

In the first step, we estimate $\btheta^*$ by minimizing the empirical version of negative of (<ref>) with a penalty function. From  the empirical gradient

\begin{equation}
\label{eq:gradienttheta*}
\nabla\mL_n(\btheta;\bz) = \frac{1}{n\submS + n\submT}\sum_{i\in \mathcal{I}\submSL\cup \mathcal{I}\submTL}w(\z_i; \bz)^{I(i\in \mathcal{I}\submSL)}\{g(\btheta\trans\z_i) - y_i \}\z_i,
\end{equation}

we can see that when at least one of the two working models is correctly specified, $\bdel_0 = \0$ and $\wt \btheta$ is a consistent estimator of   $\bthetabar$ under standard high dimensional conditions. In this case, labeled observations from the source population can be fully utilized.

When both of the working models are mis-specified yet $\btheta^*$ is not too different from $\bthetabar$,  the labeled observations from $\mS$ can still be partially utilized to leverage the source data and hence the meta estimator $\bthetahat\supmeta$ can still outperforms the target-only estimator $ \bthetahat\supmT$.

We do not restrict to the Lasso penalty and allow  $p_{\lambda_\btheta}$ and $p_{\lambda_\bdel}$ to be different non-convex penalties satisfying certain conditions.

However,when $\|\bdel_0\|_1$ is greater than a certain rate, the second step is unable to fully correct the bias in $\wt \btheta$ since $\bdel_0$ can be too dense to be recovered with limited labeled target data. This negative transfer phenomenon is verified in both our theoretical analyses given in Section <ref> and numerical studies detailed in Section <ref>. We overcome this in
our final step of the STRIFLE estimation as
detailed below.

§.§ STRIFLE Estimator of the Imputation Model

To overcome negative transfer, we define our STRIFLE estimator for $\bthetabar$ as a convex combination of  $\bthetahat\supmT$ and $\bthetahat\supmeta$, and estimate the combination weight via cross-fitting. Specifically, we randomly split observations in $\mathcal{I}\submTL$ into two halves, $\mathcal{I}\submTL\supone$  and $\mathcal{I}\submTL\suptwo$.
We use observations in $\mathcal{I}\submTL\supk$ to construct the target-only estimator $\bthetahat\supmTk$ based on (<ref>), and use observations  in $\mathcal{I}\submSL$ and $\mathcal{I}\submTL\supk$ to construct the meta learning estimator $\bthetahat\supmetak$ based on Algorithm <ref>. The remaining  observations  in $\mathcal{I}\submTL/ \mathcal{I}\submTL\supk$ are used to estimate the coefficient in the convex combination. Inspired by <cit.>,
we consider a binary coefficient $\wh\rho \in\{0,1\}$ inferring whether $\mS$ is transferable:

\begin{equation}
\label{eq:LM_ineq}
\wh \rho = I\left\{\frac{1}{2}\sum_{k=1}^2 \mL^{\rho\supnegk}_{n\submT/2} (\bthetahat\supmetak)
\leq \frac{1}{2}\sum_{k=1}^2\mL^{\rho\supnegk}_{n\submT/2} (\bthetahat\supmTk) - \epsilon_0 \right\},
\end{equation}

\[
\mL^{\rho\supnegk}_{n\submT/2} ( \btheta )  = \frac{2}{n\submT} \sum_{i \in \mathcal{I}\submTL/ \mathcal{I}\submTL\supk} G(\z_i\trans\btheta) -y_i \z_i\trans \btheta,
\]

and $\epsilon_0$ is a non-negative constant chosen to achieve theoretical guarantee.
See Remark <ref> in Appendix <ref> for more details of the choice of $\epsilon_0$.

The final STRIFLE estimator is

\bthetahat\supSTRIFLE = \wh\rho \bthetahat\supmeta + (1 - \wh\rho)\bthetahat\supmT.

Our estimator $\bthetahat\supSTRIFLE$ is guaranteed to be no worse than the target-only estimator $\bthetahat\supmT$ with a small error even if both of the working models are misspecified. Consequently, $\wh\bb\supSTRIFLE$  is triply robust and guaranteed to be no worse than the target-only SAS estimator with a small error which is negligible under mild conditions. When $m(\z)$ or $w(\z)$ are correctly specified, $\wh\bb\supSTRIFLE$ could be substantially more efficient than that of SAS, especially when $n\submS$ is larger than $n\submT$.

§ THEORETICAL RESULTS

In this section, we detail the theoretical properties of the proposed STRIFLE estimator. We start by introducing some notations and
preliminaries. For a vector $\v = (v_1, \ldots, v_p)\trans \in \mathbb{R}^p$, we
define $\|\v\|_0 = |\rm{supp}(\v)|$, where $\rm{supp}(\v) = \{j: v_j
\neq 0\}$
and $|A|$ is the cardinality of a set $A$.
Denote $\|\v\|_{\infty} = \max_{1\leq j \leq p}|v_j|$.
For $S \subseteq \{1, \ldots, p\}$, let $\v_S = \{v_j: j \in S\}$ and $S^c$ be the complement of $S$ .
We use $\lambda_{\min}(\M)$ and
$\lambda_{\max}(\M)$ to denote the minimal and maximal eigenvalue of matrix $\M$, respectively.
For two positive sequences $a_n$ and $b_n$, we use $a_n \precsim b_n$
to denote $a_n \leq Cb_n$ for some constant $C>0$,
and use $a_n \asymp b_n$ to denote $C\leq a_n/b_n \leq C^{\prime}$ for
some constants $C, C^{\prime}>0$.

The sub-exponential norm of a random variable $X$ is defined as $\|X\|_{\psi_1} =\sup_{t\geq1} t^{-1}\{E(|X|^t)\}^{1/t}$.
Note that $\|X\|_{\psi_1} < C_1$ for some constant $C_1$, if $X$ is sub-exponential.
The sub-Gaussian norm of $X$ is defined as $\|X\|_{\psi_2} =
\sup_{t\geq1} t^{-1/2}\{E(|X|^t)\}^{1/t}$.
Note that $\|X\|_{\psi_2} < C_2$ for some constant $C_2$, if $X$ is sub-Gaussian.
More properties regarding sub-exponential and sub-Gaussian random
variables are given in Appendix <ref>.

We first describe the following two sets of assumptions on the design matrices, models, dimensionality, and penalty functions that are required to ensure theoretical derivations of our estimators.

(i) We consider sub-Gaussian designs, where

\[\sup_{i\in \mathcal{I}_{\mS}, j=1, \dots, d}\|\Z_{ij}\|_{\psi_2} \leq C_{\z, \mS}, \text{ and } \sup_{i\in \mathcal{I}_{\mT}, j=1, \dots, d}\|\Z_{ij}\|_{\psi_2} \leq C_{\z, \mT}.\]

Let $C_\z = \max(C_{\z, \mS}, C_{\z, \mT})$.
Further, we assume
0<κ≤λ_min{E(_i_i)} ≤λ_max{E(_i_i)} ≤1/κ < ∞,
0<κ≤λ_min{E(_i_i)} ≤λ_max{E(_i_i)} ≤1/κ < ∞.

(ii) We assume that $g(\cdot)$ is nondecreasing, and
there exists a constant $C_g > 0$ such that $g^\prime(t) \leq C_g$, for all $t \in \mR$.

In both target and source populations,  $\sup_{j=1, \dots, d} |f_j(\z)| \leq C_{\f(\z)}$ almost surely for a constant $C_{\f(\z)}$,    $w_0(\z) = \exp\{ \f(\z)\trans \bz_0\} \geq C_{w, l}$ almost surely for a constant $C_{w, l}$, and $w_0(\z) = \exp\{ \f(\z)\trans \bz_0\} \leq C_w$ almost surely for a constant $C_w$.
Further,  we assume

\[
0 <  \kappa_\f \leq  \lambda_{\min}[E\{\f(\Z_i)\f(\Z_i)\trans\}] \leq \lambda_{\max}[E\{\f(\Z_i)\f(\Z_i)\trans\}] \leq \frac{1}{\kappa_\f} < \infty.
\]

(iv) In  both populations,  $Y_i - g(\Z_i\trans \btheta^*)$ is uniformly sub-Gaussian given $\Z_i$ with sub-Gaussian norm bounded by a positive constant $D_0$.

We further assume that
E({Y_i-g(_i^*)}^2 |_i ) ≤D_1,  and
E({Y_i-g(_i^*)}^2 |_i ) ≤D_1,
for some positive constant $ D_1$.
In the target population, $Y_i - g( \Z_i\trans\bthetabar)$ is uniformly sub-Gaussian given $\Z_i$ with sub-Gaussian norm bounded by $D_2$, and $g(\X_i\trans \bb_0)- g( \Z_i\trans\bthetabar)$ and $g(\X_i\trans \bb_0)-Y_i $ are uniformly sub-Gaussian given $\Z_i$ with sub-Gaussian norm bounded by $D_3$.

Assumptions <ref>(i), <ref>(iii), and <ref>(iv) are standard in high dimensional settings <cit.>. <cit.> imposed <ref>(ii) when studying the high dimensional GLM.

It is not hard to show that <ref>(iv) holds under commonly studied models such as Gaussian linear model and logistic model.

Similar to  Assumption 1 of <cit.>, we introduce Assumption <ref> below to regularize the penalty function $p_{\lambda}(\cdot)$. Different from <cit.>, <cit.>  and <cit.>, we establish the theoretical properties under the framework of <cit.> and <cit.> that allows for nonconvex penalties like SCAD and MCP.

With a slight abuse of notation, we write

\[ p_{\lambda_{\btheta}}(\btheta) = \sum_{j=1}^{d} p_{\lambda_{\btheta}}(\theta_j), \ \ p_{\lambda_{\bdel}}(\bdel) = \sum_{j=1}^{d} p_{\lambda_{\bdel}}(\delta_j)\ \ \text{ and } \ \
p_{\lambda_{\bb}}(\bb) = \sum_{j=1}^{p} p_{\lambda_{\bb}}(\beta_j).\]

(i) The functions $p_{\lambda_\btheta}$, $p_{\lambda_\bdel}$ and $p_{\lambda_\bb}$ satisfy $p_{\lambda_\btheta}(0) = 0$, $p_{\lambda_\bdel}(0) = 0$ and $p_{\lambda_\bb}(0) = 0$, and are symmetric around zero.

(ii) On the nonnegative real line, the functions $p_{\lambda_\btheta}$, $p_{\lambda_\bdel}$ and $p_{\lambda_\bb}$ are nondecreasing.

(iii) For $t>0$, the functions $p_{\lambda_\btheta}(t)/t$, $p_{\lambda_\bdel}(t)/t$ and $p_{\lambda_\bb}(t)/t$
are nonincreasing in $t$.

(iv) The functions $p_{\lambda_\btheta}$,   $p_{\lambda_\bdel}$ and $p_{\lambda_\bb}$ are differentiable for all $t\neq 0$ and subdifferentiable at $t=0$, with $\lim_{t\to 0^+} p^\prime_{\lambda_\btheta}(t) = \lambda_\btheta L_\btheta$, $\lim_{t\to 0^+} p^\prime_{\lambda_\bdel}(t) = \lambda_\bdel L_\bdel$, and  $\lim_{t\to 0^+} p^\prime_{\lambda_\bb}(t) = \lambda_\bb L_\bb$.

(v) There exists $\mu_\btheta\geq 0$, $\mu_\bdel \geq 0$, and $\mu_\bb\geq 0$ such that $p_{\lambda_\btheta, \mu}(t) = p_{\lambda_\btheta}(t) + \mu_\btheta\|t\|_2^2/2$, $p_{\lambda_\bdel, \mu_\bdel}(t) = p_{\lambda_\bdel}(t) + \mu_{\bdel}\|t\|_2^2/2$ and $p_{\lambda_\bb, \mu_\bb}(t) = p_{\lambda_\bb}(t) + \mu_{\bb}\|t\|_2^2/2$ are convex.

It is not hard to verify that commonly used sparse penalties including Lasso, SCAD and MCP all satisfy Assumption <ref>, and the existence and value of $\mu$ for different penalties are discussed in <cit.> in details.

We then introduce an assumption on convergence rates of  density ratio estimation.
Any density ratio estimators satisfying the following assumption can be used and would not destroy
the theoretical properties of our STRIFLE estimator.
The specific construction of the density ratio estimator given in Section 2.3 satisfies this assumption although alternative estimators can also be used.

Denote by $\Omega_0$ the event that

\begin{align*}
\|\wh \bz - \bz_0\|_1 \leq C_{\bz}, \text{ and } \|\wh \bz - \bz_0\|_2^2 = o(1),
\end{align*}

where $C_{\bz}$ is a positive constant, and $\pr(\Omega_0)$ goes to 1 when the sample size goes to infinity.

In addition to Assumptions <ref>, <ref>, and <ref>, we further introduce Assumptions  <ref>, <ref> and <ref> pertaining to estimation of $\bthetabar$ and $\bb_0$ with constants in Appendix <ref>. In short, we assume that $\btheta^*$, $\bthetabar$, $\bdel_0$ and $\bb_0$ are feasible <cit.>, i.e.,
$\|\btheta^*\|_1 \leq R $,
$\|\bthetabar\|_1 \leq R_\bdel$, $\|\bdel_0\|_1\leq R_\bdel$ and  $\|\bb_0\|_1\leq R_\bb$, where $R \precsim \sqrt{n/\log~d }$, $R_\bdel \precsim \sqrt{n\submT/\log~d}$, and $R_\bb\precsim \sqrt{(N\submT + n\submT)/ \log~p}$.
We also assume tuning parameters $\lambda_\btheta \asymp \sqrt{n^{-1}\log ~d}$, $\lambda_\bdel \asymp\sqrt{n\submT^{-1}\log ~d}$, and
$\lambda_\bb = \asymp\sqrt{(N\submT +n\submT)^{-1}\log~p}$.
We define the true support of $\bthetabar$ and $\bb_0$ as $S_\btheta$ and $S_\bb$ with cardinality $s_\btheta$ and $s_\bb$, respectively,
and assume $s_\btheta \precsim  n\submT/\log~d$ and $s_\bb \precsim (N\submT + n\submT)/\log~ p$.

We next establish the theoretical properties for  $\bthetahat\supSTRIFLE$ under Assumptions <ref>-<ref> in subsection <ref>, then establish the theoretical properties for $\wh\bb\supSTRIFLE$ under Assumptions <ref>-<ref>
and compare its theoretical performance with other two estimators in subsection <ref>.

§.§  Theoretical Properties of  $\bthetahat\supSTRIFLE$

The main results on the upper error bounds for $\bthetahat\supSTRIFLE$ is given in Theorem <ref>.
To establish it, we first prove the upper error bounds for $\wt \btheta$, the estimator of $\btheta^*$ in the first step of Algorithm <ref>.
We only impose exact sparsity on $\bthetabar$ but not on $\btheta^*$.  Since we focus on the $l_1$ norm of $\bdel_0$, but not $\|\bdel_0\|_0$ considered in <cit.>, essentially we allow $\btheta^*$ to be $l_1$ sparse.

Further, different from  <cit.>,
we need to take the estimation error  of $w(\z)$ into consideration.

The $l_1$ and $l_2$ error bounds of $\wt \btheta$ are given in the following Proposition. For simplicity, we use the $O_p$ notation here and after to represent rate convergence in probability. Detailed expressions of upper bounds with constants and the proof are given in Appendix <ref>.

Suppose that Assumptions <ref>, <ref>, <ref>, and  <ref> hold.
We have
- ^*_1 = O_p {
s_√(log d/n)   + (_0)_S_^c _1 +  n^2/n^2 √(n/log d)(
- _0_2^2 + n^ ϵ-1/2)},
- ^*_2^2  = O_p {
s_log d/n  + √(log d/n) (_0)_S_^c_1 + n^2/n^2 (
- _0_2^2 + n^ ϵ-1/2)},
where $\epsilon$ is an arbitrarily small positive constant.

In both error bounds, the first term is standard in high dimensional settings <cit.> when $\bdel_0 = \0$, the second term demonstrates the effect of transfer learning, and the last term represents the out-of-sample estimation error of the working density ratio model.
One interesting observation is that  $\|(\bdel_0)_{S_\btheta}\|_1$ does not show up in the upper error bounds. It implies that large difference between $\bthetabar$ and $\btheta^*$ in $\bthetabar$'s support $S_\btheta$ would not destroy the first step in our Algorithm <ref>.

We then establish the $l_1$ and $l_2$ error bounds of $\bthetahat\supmeta$ in the following Theorem. Detailed expressions of the error bounds with constants and the proof are given in Appendix <ref>.

Suppose that Assumptions <ref>, <ref>,  <ref>,  <ref>, and <ref> hold.
We have

\begin{align*}
\|\bthetahat\supmeta - \bthetabar\|_1 = O_p\left(
\|\bdel_0\|_1 +  \|\bDel\|_1 + \sqrt{\frac{\log~d}{n\submT}}\|\bDel\|_1^2 +
\sqrt{\frac{n\submT}{\log~ d}} \|\bDel\|_2^2\right),
\end{align*}

\begin{align*}
\|\bthetahat\supmeta - \bthetabar\|_2^2  = O_p\left(
\sqrt{\frac{\log~d}{n\submT}}\|\bdel_0\|_1 + \frac{\log~ d}{n\submT} \|\bDel\|_1^2
+ \|\bDel\|_2^2\right),
\end{align*}

where $\bDel = \wt \btheta - \btheta^*$.

In typical semi-supervise settings, we have $N_\mS \gg n_\mS$ and $N_\mT \gg n_\mT$, resulting higher order estimation error for $\wh\bz$.
For example,
when $ n\submS^2s_\bz(ns_\btheta)^{-1} =o( N\submS)$,
using the estimator given in Appendix <ref>, $\|\wh\bz - \bz_0\|_2 \precsim s_\bz N\submS^{-1}\log~d $ is
negligible in the upper bounds of
$\|\wt\btheta - \btheta^*\|_1$ and $\|\wt\btheta - \btheta^*\|_2$.

In this case, the results in Theorem <ref> can be simplified, which is given in the following Corollary.

\begin{equation}
\label{eq:simplified}
\|\wh \bz - \bz_0\|_2^2 + n\submS^{ \epsilon-1/2} \precsim \frac{n\log~d}{n\submS^2} s_\btheta,
\end{equation}

under the assumptions of Proposition <ref> and Theorem <ref>, we have

\[
\|\bthetahat\supmeta - \bthetabar\|_1 = O_p\left(  \sqrt{\frac{\log~d}{n}} s_\btheta  + \|\bdel_0\|_1 \right),
\]

\[
\|\bthetahat\supmeta -\bthetabar\|_2^2 =O_p \left( \frac{\log~d}{n} s_\btheta + \sqrt{\frac{\log~d}{ n\submT}}\|\bdel_0\|_1\right).
\]

By Corollary <ref>, when  $w(\z)$ or $m(\z)$ is correctly specified,  then $\bdel_0=\0$,

\[
\|\bthetahat\supmeta -\bthetabar\|_1 = O_p\left(  \sqrt{\frac{\log~d}{n}} s_\btheta  \right), \text{  and   }	\|\bthetahat\supmeta -\bthetabar \|_2^2 =O_p \left( \frac{\log~d}{n} s_\btheta \right),
\]

which are the standard high dimensional  error rates with a sample size of $n=n_\mS+n_\mT$.  In other words, the labeled observations in $\mS$ have been fully utilized to improve the estimation accuracy w.r.t $\bthetabar$.

When $\|\bdel_0\|_1 =o\left(s_\btheta\sqrt{n\submT^{-1} \log~d}\right)$, even if both of the working models are misspecified, we can still gain estimation accuracy by partially utilizing the source labeled observations.
When $ s_\btheta\sqrt{n\submT^{-1} \log~d} = o(\|\bdel_0\|_1)$,
the  error rates are worse than those of the target-only estimator and we begin to have negative transfer.

We present in Theorem <ref> the upper error bound of $\bthetahat\supSTRIFLE$, which overcomes negative transfer when $\|\bdel_0\|_1$ is large. The proof is given in Appendix <ref>.

Suppose that Assumptions <ref>, <ref>, <ref>, <ref>, <ref>, and (<ref>) hold.
We have

\[
\|\bthetahat\supSTRIFLE - \bthetabar \|_2^2  = O_p \left\{ \left(\frac{\log~d}{n} s_\btheta+ \sqrt{\frac{\log~d}{n\submT}} \|\bdel_0\|_1\right) \wedge
\left(  \frac{\log~d}{n\submT}s_\btheta+ n\submT^{-1/2} \right)\right\}.
\]

We can see that $\|\bthetahat\supSTRIFLE - \bthetabar \|_2^2 $ is guaranteed to be no worse than the target-only estimator $\bthetahat\supmT$, with an error of order $n\submT^{-1/2}$, which is the estimation error of $\wh \rho$.
When $n\submT = o(n\submS)$ and $\|\bdel_0\|_1 = o\left(s_\btheta\sqrt{n\submT^{-1} \log~d}\right)$, we can gain estimation accuracy in terms of rate by utilizing $\mS$.

§.§ Theoretical Properties of $\wh \bb\supSTRIFLE$

With triply robust estimator $\bthetahat\supSTRIFLE$, we can leverage unlabeled target observations effectively to improve the learning accuracy of $\bb_0$.
Similar to the SAS estimator, we allow $\bb_0$ to be much denser than $\bthetabar$ when $n\submT \precsim N\submT$.
The upper error bound of $\wh\bb\supSTRIFLE$ is established in the following Theorem with the proof given in Appendix <ref>.

Suppose that Assumptions <ref>, <ref>, <ref>, <ref>, <ref>,  <ref>, and (<ref>) hold.

We have

-_0_2^2  =O_p  { log  p /N+ n s_+ ( log d/ns_+ √(log d/n) _0_1) ∧( log  d/ns_+ n^-1/2)}.

Comparison of  squared $l_2$ error bound of estimation of $\bb_0$ among three methods, where the common term $\bDel_\bb = s_\bb (N\submT + n\submT)^{-1}\log~ p$.
When $\|\bdel_0\|_1 \precsim s_\btheta\sqrt{n\submT^{-1}\log~d}$, Meta and STRIFLE outperform SAS; when $s_\btheta\sqrt{n\submT^{-1}\log~d } = o(\|\bdel_0\|_1)$,  Meta is beaten by SAS but STRIFLE is no worse than SAS with an error of order $n\submT^{-1/2}$.

1cMethod     SAS     Meta     STRIFLE
$0\leq\|\bdel_0\|_1 \precsim s_\btheta  n^{-1} \sqrt{n\submT\log~d}$
$\bDel_\bb  +  n\submT^{-1}\log ~d s_\btheta$
$\bDel_\bb  +  n^{-1} \log ~d s_\btheta$
$\bDel_\bb  +  n^{-1} \log ~d s_\btheta$

$s_\btheta  n^{-1} \sqrt{n\submT\log~d} \ll \|\bdel_0\|_1 \precsim s_\btheta\sqrt{n\submT ^{-1}\log~d }$
$\bDel_\bb  +   n\submT^{-1} \log ~d s_\btheta$
$\bDel_\bb + \sqrt{n\submT^{-1}\log ~d} \|\bdel_0\|_1$
$\bDel_\bb  +\sqrt{n\submT^{-1}\log ~d}  \|\bdel_0\|_1$

$(\log~d)^{-1/2}\gg  \|\bdel_0\|_1 \gg s_\btheta\sqrt{n\submT^{-1}\log~d }$
$\bDel_\bb  +   n\submT^{-1} \log ~d s_\btheta$
$\bDel_\bb + \sqrt{n\submT^{-1}\log ~d} \|\bdel_0\|_1$
$\bDel_\bb  +\sqrt{n\submT^{-1}\log ~d}  \|\bdel_0\|_1$

$\|\bdel_0\|_1 \gg s_\btheta\sqrt{n\submT^{-1}\log~d } \vee (\log~d)^{-1/2}$
$\bDel_\bb   +  n\submT^{-1}\log ~d s_\btheta$
$\bDel_\bb  + \sqrt{n\submT^{-1}\log ~d }  \|\bdel_0\|_1$
$\bDel_\bb  + n\submT^{-1}\log ~d s_\btheta + n\submT^{-1/2}$

To better understand the efficiency of $\bbhat\supSTRIFLE$ gained from the source and unlabeled data, we show in Table <ref> the squared $l_2$ error bound of $\bbhat\supSTRIFLE$ compared to those of $\bbhat\supmeta = \bbhat(\bthetahat\supmeta)$ and the SAS estimator under different scenarios.
We can see that when $w(\z)$ or $m(\z)$ is correctly specified, i.e., $\|\bdel_0\|_1 = 0$, or $\|\bdel_0\|_1 \precsim s_\btheta n^{-1} \sqrt{n\submT \log~d } $,  $\bbhat\supmeta$ and $\bbhat\supSTRIFLE$ both fully utilize the labeled observations in $\mS$  with error bounds of the order

\[
\frac{\log ~ p }{N\submT + n\submT} s_\bb +  \frac{\log ~d}{n}s_\btheta.
\]

When  both $w(\z)$ and $m(\z)$ are misspecified and $ \|\bdel_0\|_1 \precsim s_\btheta\sqrt{n\submT^{-1} \log~d}$,  $\bbhat\supmeta$ and $\bbhat\supSTRIFLE$ both still partially leverage source labeled data with error bounds of order

\[
\frac{\log ~ p }{N\submT + n\submT} s_\bb + \sqrt{ \frac{\log ~d}{n\submT}}\|\bdel_0\|_1 .
\]

In the worst scenario where  $s_\btheta\sqrt{n\submT^{-1} \log~d} = o(\|\bdel_0\|_1)$, the error bound of $\bbhat\supmeta$ is worse than that of SAS estimator
due to negative transfer. However, the difference of error bound of $\bbhat\supSTRIFLE$ and that of SAS estimator is bounded by $n\submT^{-1/2}$, which is the price of estimating $\rho$, i.e., the price of detecting whether source data are transferable.
When the imputation model is relatively dense, i.e.,  $s_\btheta \gg n\submT^{1/2}/\log~d$, our STRIFLE estimator $\wh\bb\supSTRIFLE$ is guaranteed to be no worse than SAS estimator with a negligible error.
When $s_\btheta \ll n\submT^{1/2}/\log~d$ and $s_\bb(N\submT + n\submT)^{-1}\log~p \ll s_\btheta n^{-1}\log~d$, the error of transferability detection dominates and our STRIFLE estimator could be worse than SAS estimator.

§ SIMULATION STUDIES

In this section, we evaluate the finite-sample  performance of the proposed STRIFLE estimator with respect to both estimation and prediction accuracy. We compare the STRIFLE estimator with four benchmark methods: (1) the supervised estimator (SUP) obtained by estimating $\bb_0$ directly using the observations in $\mathcal{I}\submTL$; (2) the SAS estimator obtained by first estimating $m(\z)$ using the observations  in $\mathcal{I}\submTL$ then leveraging the observations  in $\mathcal{I}\submTU$ by minimizing (<ref>); (3) the covariate shift estimator (CS) obtained by first estimating $w(\z)$ using the observations in $\mathcal{I}\submTU$ and $\mathcal{I}\submSU$ then estimating $\bb_0$ by minimizing

\[
-\frac{1}{n\submS+n\submT}\sum_{i\in \mathcal{I}_{\mS, L}\cup\mathcal{I}\submTL}   \wh w(\z_i)^{I(i\in \mathcal{I}_{\mS, L}) }
Q\left\{ g(\x_i\trans \bb), y_i\right\}  +\lambda_\bb \|\bb\|_1;
\]

and (4) the supervised transfer learning estimator (SUPTrans) proposed in <cit.>, which utilizes the source labeled observations through a three-step transferring procedure with a stronger sparsity assumption on $\bb_0$ but does not leverage the unlabeled observations from both populations.

Throughout, we consider binary outcomes with $g(\cdot)$ being anti-logit and use the Lasso penalty for all models for computational ease. All tuning parameters of the Lasso penalties are selected via 5-fold cross-validation.  We fix $n\submT = 150$, $N\submT = N\submS = 10000$, $p=150$, $q=30$, and let $n\submS = 150$ or $1200$. Since $N\submS$ and $N\submT$ are much larger than $d$,
we estimate $\bz\in  \mR^d$ by first minimizing $\mL_{N}^w(\bz) +\lambda_\bz\|\bz\|_2^2$ with $\lambda_\bz=10^{-5}$ to stabilize the optimization, and then applying a hard thresholding with a cutoff $10\sqrt{\log~d/(N\submS +N\submT)}$.

Instead of generating data separately for $\mT$ and $\mS$, we generate $\Z_i$ and $Y_i\mid \Z_i$ for $\Dscr\submS\cup\Dscr\submT$ and then randomly generate a membership variable $R_i = I(i \in \mI\submSL\cup\mI\submSU)$ to assign the $i$th observations to the source population when $R_i = 1$. Specifically, we generate $\Z_i$ mimicking the zero-inflated discrete distribution of EHR features. We first generate latent variables  $\W_i \sim N(\0, \bSig)$, where $\bSig = [\iota^{|j-k|}]_{p\times p}$, for $j, k = 1, \dots, d$.
Then we generate  $X_{ij}  = a(W_{ij})$, for $j=1, \dots, p$,  $S_{i1} =a( \sum_{j=1}^5X_{ij} + W_{i,p+1}) $, $S_{i2} = a( \sum_{j=3}^{7}X_{ij} + W_{i,p+2})$, $S_{i3} = a( \sum_{j=6}^{10}X_{ij} + W_{i,p+3})$, and $S_{ij} = a(W_{i,p+j})$, for $j=4, \dots, q$, where the transformation $a(t) = [\log(1 + e^t)]$. $\X$  and $\S$ are standardized to have mean zero and variance one. We consider $\iota = 0$ resulting $\bSig=\I_p$ and $0.1$ which allows for weak correlations among $\X$.

We consider two models for $Y_i \mid \Z_i$:

(M)           {(Y_i=1|_i)} = 0.2  - X_i1 + X_i2 -X_i3 + 2S_i1 - 2S_i2 +2S_i3;

(M)           	{(Y_i=1|_i)} = -X_i1 + X_i2  + X_i3^2  - S_i2 + 2S_i3 + 2S_i1/1 + exp(- X_i1S_i1^2).

The imputation model $m(\z)=g(\z\trans \btheta)$ is correctly specified under (M$\subcor$) but misspecified under (M$\submis$). For the misspecified scenario under (M$\submis$),  $\bthetabar$ remains exact sparse  when  $\bSig = \I_p$ but only approximately sparse when $\bSig = [0.1^{|j-k|}]$.

We consider two models for generating $R_i$:

(W)            {(R_i=1 |_i, _i)} =X_i1 -X_i2  - X_i3 + S_i1,

(W)           {(R_i=1|_i, _i)} =  1.8S_i2 - 2S_i3 + S_i1 [1 +  X_i1 + X_i2 /1 + exp{- (∑_j=3^10 X_ij)}].

We set $\f(\z)$ to $\z$ when fitting the density ratio model (<ref>), leading to correct specification of (<ref>) with W$\subcor$ and misspecification with W$\submis$.
We have four different sets of configurations by combining M$\subcor$, M$\submis$, W$\subcor$ and W$\submis$ under two correlation structures for $\Z$ and two values for $n\submS$.  Under the scenario with M$\submis$ and $\bSig = [0.1^{|j-k|}]$, the  exact sparsity assumption on $\bthetabar$, required by SAS and STRIFLE that employ an imputation model,  does not hold.

We consider this scenario to examine the robustness of different estimators to the departure of the required assumptions.

To further examine the potential negative transfer consequence from different estimators and demonstrate the triple robustness of our STRIFLE estimator, we additionally consider simulation settings when both  $m(\z)$ and $w(\z)$ are severely misspecified. The theoretical analysis suggests that negative transfer can be more pronounced for the CS estimator and the intermediate Meta estimator  when $\|\bdel_0\|_1 \gg s_\btheta\sqrt{n\submT\log~d}$. To this end, we let  $\bSig = \I_p$ and generate $Y_i$ from

\begin{align*}
{\rm(M\submis^\prime)} \ \  \logit\{\Pr(Y_i=1\mid \Z_i)\} = -1-2X_{i1} + X_{i2}^2  + 3S_{i1} - 3S_{i32} + \frac{1}{1 + \exp(- X_{i3}S_{i3}^2)},
\end{align*}

and $R_i$ from

\[
{\rm(W\submis^\prime)} \ \  \logit\{\Pr(R_i=1\mid \S_i, \X_i)\} =  -2X_{i1} + 2X_{i2} + 3S_{i1} - 3S_{i2} + \frac{S_{i3}(X_{i1} + X_{i2})}{1 + \exp \{- (\sum_{j=3}^{10} X_{ij})\}}.
\]

The rest of the settings remain the same as above.

For each setting, 500 simulation replications are generated to summarize the average performance measures.
For a given estimator $\bbhat$, we report the average absolute bias across $p$ components of $\bbhat$ ($|$Bias$|$), $l_2$ error ($l_2$ Err) of $\bbhat$, and the area under the ROC curve (AUC) of $\bbhat\trans\X$ for classifying $Y$.
The AUC is calculated using the 10000 target unlabeled observations.

We first summarize the results in Table <ref> for settings where at least one of $m(\z)$ and $w(\z)$ is correctly specified, i.e., $\|\bdel_0\|_1$ = 0. Across all settings considered, STRIFLE generally outperforms SUP, SAS, CS and SUPTrans with respect to estimation and prediction accuracy by successfully leveraging target unlabeled observations and fully utilizing source labeled observations.
STRIFLE substantially outperforms SAS, which is also robust against the misspecification of $m(\z)$ and has nothing to do with $w(\z)$,  when $n\submS$ is much greater than $n\submT$, because more useful information can be borrowed from the source population with $\|\bdel_0\|_1$ = 0.
While STRIFLE substantially outperforms SUPTrans, which is adaptive to potential heterogeneity between source (without transformation based on $w(\z)$) and target populations, when the heterogeneity is large, or $N\submT$ is much greater than $n\submS$ and $s_\bb \log~ p$ is much greater than $s_\btheta \log~ d$.

In (C2) with $n\submS = 1200$ and relatively large $s_\btheta$, the AUCs of SUPTrans are slightly better than those of STRIFLE, but STRIFLE has smaller estimation errors.  Further, we can see that the performance of STRIFLE with $\iota=0.1$ is very similar to that with $\iota=0$, which implies that
exact sparsity assumption on $\bthetabar$ is a technical assumption introduced mainly to facilitate the proofs.

Average absolute bias ($|$Bias$|\times 10^{-2}$), average of $l_2$ error ($l_2$ Err), area under the ROC curve (AUC)
of five different methods
under  three different modeling configurations: (C1) both correct (M$\subcor$+W$\subcor$);  (C2) $m(\z)$ misspecified (M$\submis$+W$\subcor$); (C3) $w(\z)$ misspecified (M$\subcor$+W$\submis$) with $\iota = 0$ and $0.1$ and $n\submS =150$ and $1200$.

$|$Bias$|$             $l_2$ Err               AUC             $|$Bias$|$               $l_2$ Err              AUC        $|$Bias$|$             $l_2$ Err               AUC
$\iota =0$ 		    3c (C1) Both correct         3c(C2) $m(\z)$ misspecified
3c(C3) $w(\z)$  misspecified

SUP     1.70    1.02    0.75
2.19       1.20      0.68
1.93    1.13    0.67

SAS    1.29    0.88    0.77
1.70        1.05      0.72
1.32    0.93     0.73
CS$_{n\submS = 150}$    1.50    0.97     0.75
1.98       1.13     0.68
1.79     1.23    0.66

SUPTrans$_{n\submS = 150}$    2.02    1.10     0.75
2.43    1.24     0.70
2.04    1.05     0.70

STRIFLE$_{n\submS = 150}$     1.14    0.77    0.78     1.51   0.96   0.73
1.16    0.93     0.73

CS$_{n\submS = 1200}$    1.20    0.73     0.78
1.34     0.78    0.74
1.68    1.19     0.66

SUPTrans$_{n\submS = 1200}$    1.53    0.81     0.80
1.71     0.86     0.77
1.75    0.91     0.72

STRIFLE$_{n\submS = 1200}$    0.97    0.58     0.80
1.06   0.65    0.76
0.71    0.76     0.75

$\iota = 0.1$		    3c(C1) Both correct     3c(C2) $m(\z)$  misspecified
3c(C3) $w(\z)$  misspecified

SUP      1.70      1.05    0.74
2.18     1.21    0.69
1.96    1.17    0.67

SAS     1.38    0.93     0.76
1.77    1.07    0.73
1.44    0.98     0.73
CS$_{n\submS = 150}$    1.56    0.99    0.74
1.96    1.13    0.69
1.79    1.25      0.66

SUPTrans$_{n\submS = 150}$     2.08    1.16     0.74
2.47    1.26      0.71
2.11    1.08    0.70

STRIFLE$_{n\submS = 150}$     1.19    0.82    0.77        1.55    0.97     0.74
1.27   0.97   0.73

CS$_{n\submS = 1200}$     1.17    0.73    0.78
1.30    0.76    0.75
1.58     1.16    0.67

SUPTrans$_{n\submS = 1200}$          1.56    0.82    0.79
1.71    0.86    0.78
1.76     0.90     0.73

STRIFLE$_{n\submS = 1200}$     1.00   0.60   0.80     1.06    0.66    0.77
0.73     0.78    0.75

Table <ref> summarizes results for configurations (C4) M$\submis$+W$\submis$ and (C5) M$\submis'$+W$\submis'$ with $\iota = 0$, where both models are misspecified and the degree of misspecification is more severe in configuration (C5). For illustration, we now also include results from $\bthetahat\supmeta$ which is expected to suffer from negative transfer under model misspecifications.
When the transferred source and target populations are substantially different with large magnitude of $\|\bdel_0\|_1$ under configuration (C5), as expected, the performances of both Meta and CS estimators are worse than that of the semi-supervised benchmark SAS due to negative transfer.
The SUP estimator, while valid, also suffers from worse precision due to limited sample size.
However, STRIFLE attains comparable performance as that of SAS, because STRIFLE is adaptive to the degree of heterogeneity between the transferred source and target imputation models and robust to negative transfer.
On the other hand, under configuration (C4) where the two populations are reasonably similar with smaller magnitude of $\|\bdel_0\|_1$, STRIFLE can still partially utilize source data with improved performance compared to SAS.
For SUPTrans, which is also adaptive to the degree of heterogeneity between source and target outcome models, i.e.,
the difference between the limit of $\bb$ in source and $\bb_0$,
its performance improves when $n\submS$ increases from 150 to 1200, but still worse than that of STRIFLE under both configurations due to failure to make use of predictive surrogates and leverage the large number of target unlabeled observations.

Average absolute bias ($|$Bias$|\times 10^{-2}$), average of $l_2$ error ($l_2$ Err), area under the ROC curve (AUC)
of six different methods
under two configurations, where both nuisance models are misspecified. Negative transfer does not show up in (C4) but does exist in (C5).

$|$Bias$|$             $l_2$ Err               AUC                $|$Bias$|$             $l_2$ Err               AUC
3c(C4) $M\submis + W\submis$,   $\iota = 0$
3c(C5) $M\submis^\prime + W\submis^\prime$, $\iota = 0$

SUP     2.24     1.25     0.69
1.41    0.94     0.64

SAS     1.52     0.98     0.76
1.02    0.79     0.70
CS$_{n\submS = 150}$     2.03     1.32     0.68
1.54     1.14    0.60

SUPTrans$_{n\submS = 150}$     2.27     1.16     0.72
1.40    0.89     0.65

Meta $_{n\submS = 150}$     1.25    0.95     0.76
1.08    0.84     0.68

STRIFLE$_{n\submS = 150}$    1.35     0.94     0.76
1.02    0.80     0.69

CS$_{n\submS = 1200}$    1.66     1.20     0.70
2.38    1.56     0.55

SUPTrans$_{n\submS = 1200}$     1.84     1.00     0.74
1.46    0.84     0.69

Meta $_{n\submS = 1200}$    0.82     0.74     0.78
1.30    0.85     0.68

STRIFLE$_{n\submS = 1200}$     0.88     0.76     0.78
1.07    0.80     0.69

§ T2D GENETIC RISK PREDICTION FOR AFRICAN AMERICANS

In this section, we apply the proposed STRIFLE method to derive a T2D genetic risk prediction model for AA population using SNPs data along EHR data of participants from Mass General Brigham's (MGB) Biobank.
In this study, more than $87\%$ of the participants are European, but only around $5\%$ are African American (AA). We choose AA as our target population, and European as our source population.
Only a small amount of the binary response $Y$, the manual chart review label for T2D, are available from both populations.
We aim to develop a polygenic risk prediction model for $Y$ based on SNPs, age, and gender for the target AA population with the help of observations from a larger European population as well as observations on $\S$, consisting of counts of T2D related EHR codes selected in <cit.> via a knowledge graph. Since there may exist systematic coding biases in International Classification of Diseases (ICD) billing codes across different demographic populations, we removed ICD codes and only used 76 laboratory tests and medication prescriptions as surrogates. Examples of the surrogates include the count of hemoglobin a1c tests and the number of prescriptions for pioglitazone, a commonly used blood sugar controlling treatment for T2D patients. All count variables are transformed by $x \to \log(1+x)$ due to their skewness. We identified $248$ SNPs from those previously reported as predictive of T2D, low–density lipoprotein, or high–density lipoprotein cholesterol gene <cit.> with additional removal of SNPs with minor allele frequency less than 0.08 and correlations among the SNPs greater than 0.5. Together with age and gender, we have $250$ candidate predictors to form $\X$.  To improve the transferability of the source data, we normalize $\X$ and $\S$ to have zero mean and unit variance within each population.

There are $n\submT = 95$ labeled observations and $N\submT = 2900$ unlabeled observations from AA population; $n\submS = 1137$ labeled observations and $N\submS = 50242$ unlabeled observations from European population.

Similarly to the simulation studies, we derive T2D polygenic risk prediction models using STRIFLE, SUP, CS, SUPTrans and SAS methods. We employ 10-fold cross-validation for tuning parameter selection. To compare the performance of different methods, we use 10-fold cross-validation to estimate the out-of-sample AUC and mean squared error (MSE), which is defined as the mean of squared differences between the predicted probability of T2D  and the true binary T2D status.
We repeat the process 20 times and report the average of the out-of-sample AUC and MSE for each method.
We also use source labeled observations to estimate $\bb$ then validate on target labeled observations to illustrate potential heterogeneity across populations, denoted by SUPSource.
As summarized in Table <ref>,
the STRIFLE estimator yields best estimation and prediction accuracy by successfully utilizing the source population and leveraging the target unlabeled observations.

Prediction (AUC) and Estimation (MSE) accuracy  of estimated $\bb_0$ from SUP, CS, SUPTrans, SAS and STRIFLE methods, compared to supervised source estimator (SUPSource) evaluated on target labeled data.

Source Estimate      5cTarget Estimate
SUPSource     SUP         CS         SUPTrans     SAS     STRIFLE

AUC      0.77
0.66    0.77     0.76        0.79     0.85

MSE     0.21
0.21    0.21    0.20     0.19    0.17

The estimates of $\bb_0$ from all methods are given in Appendix <ref>. We also show in Figure <ref> the coefficient estimates for top signals whose absolute magnitude is above 0.1 in each method.
As expected, age is the most important predictor with similar estimated effect sizes across all methods; while gender appears to be of higher magnitude in source population but not selected as an informative predictor in SUP, SAS and STRIFLE.
The SNP rs7903146 are selected by all of the methods in accordance with  <cit.>.

In general, SNPs selected by SUPSource are very different from those selected by other methods, which further illustrates the heterogeneity across populations and the difference between $E\submS(Y\mid \X)$ and $E\submT(Y\mid \X)$.
The most significant SNP related to T2D among European people is rs264011, whose reference allele is $C=0.965895$ for Europeans and $C=0.71198$ for AAs, but with little literature on its relationship with diabetes.
While for AA people the two most significant SNPs related to T2D are rs174546, which is closely related to fatty acid desaturase <cit.>, and rs7903146, which is suggested as the causal diabetes susceptibility variant in AAs in <cit.>, based on our STRIFLE method.
In consequence, different risk prediction models for T2D should be employed for European people and AA people to achieve algorithm fairness.

The estimates of $\bb_0$ for top signals with absolute magnitude above 0.1 from SUP, CS, SUPTrans, SAS, and STIFLE with SUPSource as a reference.

§ CONCLUSION AND DISCUSSION

In this paper, we have proposed a novel and adaptive transfer learning method in the SS learning framework.
Different from other existing methods, our method is both  model-assisted  and surrogate-assisted.
Compared with existing non-model-assisted transfer learning methods,
when a large number of unlabeled observations from both populations are available, which is common in large scale biomedical studies, the density ratio model can be estimated relatively well, which consequently can improve the transferability of the source population.
By employing the imputation model with predictive surrogates, the target unlabeled observations can be effectively leveraged and the sparsity assumption on the final outcome model can be relaxed, which can be important in practice. For example, in polygenic risk prediction models, usually the signals of SNPs are relatively weak and dense.
Instead of combining these two strategies scrappily, we construct our method, specifically the estimating equations, carefully so that when $p_{Y\mid \Z}\supmS = p_{Y\mid \Z}\supmT$ and one of the two nuisance models is correctly specified, we will be able to fully transfer the source labeled observations, which cannot be achieved using the existing transfer learning methods when the two populations are not exactly the same. Further, even if both of the nuisance models are misspecified or $p_{Y\mid \Z}\supmS = p_{Y\mid \Z}\supmT$ does not hold,
when the transferred source and target imputation models share enough similarities, STRIFLE can still partially utilize source labeled data, and
is guaranteed to be no worse than the target-only SAS method with small errors.

The proposed transferring procedure is only triply robust when applied to the estimation for the imputation model, i.e., the estimation of $\btheta$
under our covariate shift setting.
In fact, the source unlabeled observations can only be transferred to improve the estimation rate when the density ratio model is correctly specified, since $p_{Y\mid \X}\supmT$ can be different from $p_{Y\mid \X}^{(\mS)}$.
How to robustly and efficiently conduct transfer learning directly on the outcome model, i.e., on $\bb$, in the SS learning framework is worth of further research and beyond the scope of this paper.
Moreover, the theoretical results in Theorem <ref> and Corollary <ref> shed light on
transfer learning of other high dimensional nonconvex problems under the restricted strong convexity condition.

Ashley, E. A.  2016, `Towards precision
medicine', Nature Reviews Genetics 17(9), 507–522.

Billings  Florez2010billings2010genetics
Billings, L. K.  Florez, J. C.  2010, `The genetics of type 2 diabetes: what have we
learned from gwas?', Annals of the New York Academy of Sciences 1212(1), 59–77.

Bühlmann  Van
De Geer2011buhlmann2011statistics
Bühlmann, P.  Van De Geer, S.  2011, Statistics for high-dimensional data: methods,
theory and applications, Springer Science & Business Media.

Chakrabortty  Cai2018chakrabortty2018efficient
Chakrabortty, A.  Cai, T.  2018,
`Efficient and adaptive linear regression in semi-supervised settings', The Annals of Statistics 46(4), 1541–1572.

Fan  Li2001fan2001variable
Fan, J.  Li, R.  2001, `Variable
selection via nonconcave penalized likelihood and its oracle properties',
Journal of the American statistical Association 96(456), 1348–1360.

Ferryman  Pitcan2018ferryman2018fairness
Ferryman, K.  Pitcan, M.  2018,
`Fairness in precision medicine', Data & Society 1.

[Grant et al.]Grant, Thorleifsson, Reynisdottir, Benediktsson,
Manolescu, Sainz, Helgason, Stefansson, Emilsson, Helgadottir
et al.2006grant2006variant
Grant, S. F., Thorleifsson, G., Reynisdottir, I., Benediktsson, R., Manolescu,
A., Sainz, J., Helgason, A., Stefansson, H., Emilsson, V., Helgadottir, A.
et al.  2006, `Variant of transcription
factor 7-like 2 (tcf7l2) gene confers risk of type 2 diabetes', Nature
genetics 38(3), 320–323.

[Gronsbell et al.]Gronsbell, Liu, Tian   Cai2022gronsbell2022efficient
Gronsbell, J., Liu, M., Tian, L.  Cai, T.  2022, `Efficient evaluation of prediction rules in
semi-supervised settings under stratified sampling', JOURNAL OF THE
ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY .

Hodson, R.  2016, `Precision medicine', Nature 537(7619), S49–S49.

[Hong et al.]Hong, Rush, Liu, Zhou, Sun, Sonabend, Castro,
Schubert, Panickan, Cai et al.2021hong2021clinical
Hong, C., Rush, E., Liu, M., Zhou, D., Sun, J., Sonabend, A., Castro, V. M.,
Schubert, P., Panickan, V. A., Cai, T. et al.  2021, `Clinical knowledge extraction via sparse embedding
regression (keser) with multi-center large scale electronic health record
data', NPJ digital medicine 4(1), 1–11.

[Hou et al.]Hou, Guo  Cai2021hou2021surrogate
Hou, J., Guo, Z.  Cai, T.  2021,
`Surrogate assisted semi-supervised inference for high dimensional risk
prediction', arXiv preprint arXiv:2105.01264 .

[Huang et al.]Huang, Gretton, Borgwardt, Schölkopf
Huang, J., Gretton, A., Borgwardt, K., Schölkopf, B.  Smola,
A. J.  2007, Correcting sample selection
bias by unlabeled data, in `Advances in neural information processing
systems', pp. 601–608.

Kozak  Anunciado-Koza2009kozak2009ucp1
Kozak, L.  Anunciado-Koza, R.  2009, `Ucp1: its involvement and utility in obesity', International journal of obesity 32(S7), S32.

[Li et al.]Li, Cai  Duan2021li2021targeting
Li, S., Cai, T.  Duan, R.  2021,
`Targeting underrepresented populations in precision medicine: A federated
transfer learning approach', arXiv preprint arXiv:2108.12112 .

[Li et al.]Li, Cai  Li2020li2020transfer
Li, S., Cai, T. T.  Li, H.  2020, `Transfer learning for high-dimensional linear regression: Prediction,
estimation, and minimax optimality', arXiv preprint arXiv:2006.10593 .

[Liu et al.]Liu, Zhang, Liao   Cai2020liu2020doubly
Liu, M., Zhang, Y., Liao, P. K.  Cai, T.  2020, `Augmented transfer regression learning with
semi-non-parametric nuisance models', arXiv preprint arXiv:2010.02521 .

Loh  Wainwright2015loh2015regularized
Loh, P.-L.  Wainwright, M. J.  2015, `Regularized m-estimators with nonconvexity:
Statistical and algorithmic theory for local optima', The Journal of
Machine Learning Research 16(1), 559–616.

[Martin et al.]Martin, Kanai, Kamatani, Okada, Neale   Daly2019martin2019clinical
Martin, A. R., Kanai, M., Kamatani, Y., Okada, Y., Neale, B. M.   Daly, M. J.  2019, `Clinical use of
current polygenic risk scores may exacerbate health disparities', Nature
genetics 51(4), 584–591.

[Negahban et al.]Negahban, Ravikumar, Wainwright, Yu
et al.2012negahban2012unified
Negahban, S. N., Ravikumar, P., Wainwright, M. J., Yu, B. et al.
2012, `A unified framework for
high-dimensional analysis of $ m $-estimators with decomposable
regularizers', Statistical science 27(4), 538–557.

Ning  Liu2017ning2017general
Ning, Y.  Liu, H.  2017, `A
general theory of hypothesis tests and confidence regions for sparse high
dimensional models', The Annals of Statistics 45(1), 158–195.

[Palmer et al.]Palmer, Hester, An, Adeyemo, Rotimi, Langefeld,
Freedman, Ng  Bowden2011palmer2011resequencing
Palmer, N. D., Hester, J. M., An, S. S., Adeyemo, A., Rotimi, C., Langefeld,
C. D., Freedman, B. I., Ng, M. C.  Bowden, D. W.
2011, `Resequencing and analysis of
variation in the tcf7l2 gene in african americans suggests that snp rs7903146
is the causal diabetes susceptibility variant', Diabetes 60(2), 662–668.

[Rodrigues et al.]Rodrigues, Sobrino, Genvigir, Willrich, Arazi,
Dorea, Bernik, Bertolami, Faludi, Brion et al.2013rodrigues2013genetic
Rodrigues, A. C., Sobrino, B., Genvigir, F. D. V., Willrich, M. A. V., Arazi,
S. S., Dorea, E. L., Bernik, M. M. S., Bertolami, M., Faludi, A. A., Brion,
M. et al.  2013, `Genetic variants in
genes related to lipid metabolism and atherosclerosis, dyslipidemia and
atorvastatin response', Clinica Chimica Acta 417, 8–11.

[Sergeant et al.]Sergeant, Hugenschmidt, Rudock, Ziegler, Ivester,
Ainsworth, Vaidya, Case, Langefeld, Freedman
et al.2012sergeant2012differences
Sergeant, S., Hugenschmidt, C. E., Rudock, M. E., Ziegler, J. T., Ivester, P.,
Ainsworth, H. C., Vaidya, D., Case, L. D., Langefeld, C. D., Freedman, B. I.
et al.  2012, `Differences in arachidonic
acid levels and fatty acid desaturase (fads) gene variants in african
americans and european americans with diabetes or the metabolic syndrome',
British Journal of Nutrition 107(4), 547–555.

Tan et al.2020tan2020model
Tan, Z. et al.  2020, `Model-assisted
inference for treatment effects using regularized calibrated estimation with
high-dimensional data', Annals of Statistics 48(2), 811–837.

Tian  Feng2021tian2021transfer
Tian, Y.  Feng, Y.  2021,
`Transfer learning under high-dimensional generalized linear models', arXiv preprint arXiv:2105.14328 .

Tibshirani, R.  1996, `Regression shrinkage
and selection via the lasso', Journal of the Royal Statistical Society:
Series B (Methodological) 58(1), 267–288.

[West et al.]West, Blacksher   Burke2017west2017genomics
West, K. M., Blacksher, E.  Burke, W.  2017, `Genomics, health disparities, and missed
opportunities for the nation’s research agenda', Jama 317(18), 1831–1832.

Zhang, C.-H.  2010, `Nearly unbiased
variable selection under minimax concave penalty', The Annals of
statistics 38(2), 894–942.

[Zhang et al.]Zhang, Chakrabortty   Bradic2021zhang2021double
Zhang, Y., Chakrabortty, A.  Bradic, J.  2021, `Double robust semi-supervised inference for the
mean: Selection bias under mar labeling with decaying overlap', arXiv
preprint arXiv:2104.06667 .

[Zhang et al.]Zhang, Liu, Neykov   Cai2022zhang2022prior
Zhang, Y., Liu, M., Neykov, M.  Cai, T.  2022, `Prior adaptive semi-supervised learning with
application to ehr phenotyping', Journal of Machine Learning Research
23(83), 1–25.

[Zhou et al.]Zhou, Liu, Li  Cai2022zhou2022doubly
Zhou, D., Liu, M., Li, M.  Cai, T.  2022, `Doubly robust augmented model accuracy transfer
inference with high dimensional features', arXiv preprint
arXiv:2208.05134 .

Zhu, X. J.  2005, `Semi-supervised learning
literature survey'.

§ APPENDIX

§.§ Verification of the Density Ratio Estimator for Assumption <ref>

When $d$ is greater than $N\submS$ and $N\submT$,
one feasible method is to minimize (<ref>) with the Lasso penalty. That is,

\begin{align*}
\wh \bz = \arg\min_\bz \frac{1}{N_{\mS}} \sum_{i\in  \mathcal{I}_{\mS, U}}\exp\{ \f(\Z_i)\trans\bz\} - \frac{1}{N\submT} \sum_{i\in \mathcal{I}\submTU} \f(\Z_i)\trans\bz + \lambda_\bz\|\bz\|_1,
\end{align*}

which can be rewritten as

\[
\wh \bz = \arg\min_\bz \left[\frac{1}{N} \sum_{i\in  \mathcal{I}_{\mS, U}\cup \mathcal{I}\submTU} \frac{N}{N\submS}R_i\exp\{ \f(\Z_i)\trans\bz\} +  (R_i -1) \frac{N}{N\submT} \f(\Z_i)\trans\bz\right]+ \lambda_\bz\|\bz\|_1,
\]

where  $R_i = I(i\in  \mathcal{I}_{\mS, U})$, and
$\lambda_\bz \asymp \sqrt{\log~d/N_\mS}$.
Let ${\rm supp}(\bz_0) = S_\bz$, $|S_\bz| = s_\bz$.
Under (iii) in Assumption <ref>,  and $s_\bz\lambda_\bz \leq C_{\bz, 1}$ for some positive constant $C_{\bz, 1}$, by Theorem 1 in <cit.>, we have

\[
\|\wh \bz - \bz_0\|_1 = O_p(s_\bz\lambda_\bz)
\ \ \text{ and } \ \
\|\wh \bz - \bz_0\|_2^2 = O_p(s_\bz\lambda_\bz^2).
\]

Hence,  Assumption <ref> holds.

§.§ Additional Assumptions

In this subsection, we list additional assumptions required for our theoretical results.

We assume that

$\btheta^*$ is feasible <cit.>, i.e., $\|\btheta^*\|_1 \leq R \precsim \sqrt{(n\submS + n\submT)/\log~d }$.

(ii) The tuning parameter $\lambda_\btheta$ satisfies

4/L_ max(∇_n(^*;  _0)_∞,
α_2 e^-η_0/6
√(log d/n+n)) ≤λ_≤e^-η_0 α_2 /10 L_R,
where $\eta_0 = C_{\f(\z)} C_\bz$, and $\alpha_2$ is the RSC constant given in (<ref>).

$n\submS + n\submT \geq\alpha_2^{-2} {12^2} R^2 \max(\tau_1^2, \tau_2^2)\log ~d$,
where $\tau_1$, and $\tau_2$ are  the RSC constants given by (<ref>).

$e^{-\eta_0}\alpha_1 > 5\mu_\btheta/4$,
where $\alpha_1$ is the RSC constant given by (<ref>).

In fact, (i) in Assumption <ref> can be implied by (i) in Assumption <ref>. Hence,  essentially we do not impose any assumptions on   $\btheta^*$.
(ii) and (iii) are similar to the assumptions in Theorem 1 in <cit.>. The term $e^{-\eta_0}$ appears due to the estimation of the working density ratio model.
We have proved that $\|\nabla \mL_n(\btheta^*;  \bz_0)\|_\infty \leq \lambda_\btheta L_\btheta/4$ with high probability in Lemma <ref>. (iv) is needed in the derivations. Take the Lasso penalty as an example, where $\mu_\btheta = 0$, then (iv) always holds.

We assume that

$\bthetabar$ and $\bdel_0$ are feasible, that is,  $\|\bthetabar\|_1 \leq R_\bdel$ and
$\|\bdel_0\|_1\leq R_\bdel$, where $R_\bdel \precsim \sqrt{n\submT/\log~d}$. 	 We also  assume $n\submT \geq cs_\btheta\log~d$, for a sufficiently large constant $c>0$.

(ii) The tuning parameter $\lambda_\bdel$ satisfies

4/L_ max{ ∇_n()_∞, α_4√(log d/n)} ≤λ_≤α_4/12R_L_,
where $\alpha_4$ is the RSC constant given in  (<ref>).

$n\submT \geq \alpha_4^{-2}64 R_\bdel^2 \max\{\tau_3^2, (\tau_4 + \alpha_4/2)^2\}\log ~d$,
where  $\tau_3$ and $\tau_4$ are the RSC constants given in (<ref>).

(iv)Assume that $\alpha_3 > \mu_\bdel$, where
$\alpha_3$ is the RSC constant given in (<ref>).

Feasibility assumption $\|\bthetabar\|_1\leq R_\bdel$ is standard in dealing with nonconvex objective function in high dimensional settings <cit.>.
Assumptions similar to or stronger than  $\|\bdel_0\|_1\leq R_\bdel$ are also assumed in <cit.> and <cit.>.
(ii) and (iii) are similar to the assumptions in Theorem 1 in <cit.>.
We have proved that $\|\nabla \mL_{n\submT}\supmT(\bthetabar)\|_\infty \leq \lambda_\bdel L_\bdel/4$ with high probability in Lemma <ref>.
Take the Lasso penalty as an example, where $\mu_\btheta = 0$, then (iv) always holds.

We assume that

(i) $\|\bb_0\|_1\leq R_\bb$, where $R_\bb\precsim \sqrt{(N\submT + n\submT)/ \log~p}$.

(ii) the tuning parameter $\lambda_\bb$ satisfies
4/L_ max(∇_N+ n^(_0;  )_∞,
√(log p/N+ n)) ≤λ_≤α_7 /12 L_R_,
where $\alpha_7$ is the RSC constant  given in (<ref>).

$N\submT + n\submT
\geq \alpha_7^{-2}64 R_\bb^2 \max\{\tau_6^2, (\tau_7 + \alpha_7/2)^2\}\log ~p$,
where $\tau_6$ and $\tau_7$ are the RSC constant given in (<ref>).

$ \alpha_6 > \mu_\bb$,
where $\alpha_5$ is the RSC constant given in (<ref>).

(ii) and (iii) are similar to the assumptions in Theorem 1 in <cit.>.
We have proved that $\|\nabla \mL_{N\submT + n\submT}^{\bb}(\bb_0;\bthetabar)\|_\infty \leq \lambda_\bb L_\bb/4$ with high probability in  Lemma <ref>.
Take the Lasso penalty as an example, where $\mu_\bb= 0$, then (iv) always holds.

§.§ Choice of $\epsilon_0$

$\epsilon_0$ in (<ref>) is introduced for technical reasons. Based on the proof of Theorem <ref>,
we essentially need
$n\submT^{-1}\log~d\|\bthetahat\supmeta - \bthetabar\|_1^2 \leq	\epsilon_0$
with high probability.
By Theorem <ref>, when $\|\bdel_0\|_1=o(\sqrt{n\submT /\log~ d})$, we have
$n\submT^{-1}\log~d\|\bthetahat\supmeta - \bthetabar\|_1^2 =o_p(1)$.
Hence, practically, we can choose a small positive $\epsilon_0$, or even set $\epsilon_0$ to zero when $n\submT$ is not too small.

§.§ Proof of Proposition <ref>

Denote by $\Omega_1$ the event that

\begin{align*}
\|\nabla \mL_n(\btheta^*; \bz_0)\|_\infty \leq C_{\btheta, \infty}\sqrt{\frac{\log ~d}{n\submS +n\submT}}.
\end{align*}

Under (i) and (iv) in Assumption <ref>,
if $ C_{\btheta, \infty} > K_1/\sqrt{C}$, then
$\pr(\Omega_1) \geq 1 - d^{-\epsilon_1}$, where $\epsilon_1 = CC_{\btheta, \infty}^2/K_1^2- 1$, $C$ is a universal constant, and $K_1 = 2C_w D_0C_\z$.

Recall that

\[
\nabla\mL_n(\btheta^*;\bz_0) = \frac{1}{n\submS+n\submT}\sum_{i\in \mathcal{I}_{\mS, L}\cup \mathcal{I}\submTL}w(\Z_i; \bz_0)^{I(i\in \mathcal{I}_{\mS,L})}\{g(\Z_i\trans \btheta^*) - Y_i \}\Z_i.
\]

Let $V_{ij} = w(\Z_i; \bz_0)^{I(i\in \mathcal{I}_{\mS,L})}\{g(\Z_i\trans \btheta^*) - Y_i \}Z_{ij}$, $i\in \mathcal{I}\submSL\cup \mathcal{I}\submTL$, $j = 1, \dots, d$.
Under (i) and (iv) in Assumption <ref>, by Lemma <ref>, we know $V_{ij}$
is sub-exponential with sub-exponential  norm uniformly bounded by $K_1 = 2C_w D_0C_\z$.
Further, by the definition of $\btheta^*$, $V_{ij}$ is mean-zero.
By Lemma <ref> and the union bound inequality, we have that for any $t > 0$
(∇_n(^*;_0) _∞ ≥t ) ≤dexp{ -Cmin(t^2/K_1^2 ,t/K_1)(n+ n)},
where $C$ is a universal constant.
Let $t = C_{\btheta, \infty}\sqrt{(n\submS + n\submT)^{-1}\log~d} \leq K_1$, then

\begin{align*}
&\pr \left\{\|\nabla\mL_n(\btheta^*;\bz_0) \|_{\infty} \geq C_{\btheta, \infty}\sqrt{(n\submS + n\submT)^{-1}\log~d} \right\}\\
\leq &
d\exp\left\{ -C\min \left(\frac{ C_{\btheta, \infty}^2 (n\submS + n\submT)^{-1}\log~d}{K_1^2} ,\frac{C_{\btheta, \infty}\sqrt{(n\submS + n\submT)^{-1}\log~d} }{K_1}\right)(n\submS + n\submT)\right\} \\
\leq & d\exp\left\{ -\frac{ CC_{\btheta, \infty}^2 }{K_1^2} \log~d \right\} \\
\leq & d^{-\epsilon_1},
\end{align*}

where $\epsilon_1 = CC_{\btheta, \infty}^2/K_1^2- 1$.

Here we choose $C_{\btheta, \infty} \leq\lambda_\btheta L_\btheta \sqrt{(n\submS + n\submT)/\log~ d}/4$.

For any $0<\epsilon<1/2$ and a constant $C_{\epsilon}>0$, denote by $\Omega_2$ the event that

\begin{align*}
(\wh \bz - \bz_0)\trans \T_{1}(\wh \bz - \bz_0)  \leq  \frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon} n\submS^{ \epsilon-1/2}
\end{align*}

given the data in $\mathcal{I}\submSU$ and $\mathcal{I}\submTU$,

\[
\T_{1} = \frac{1}{n\submS}\sum_{i \in \mathcal{I}\submSL}
w(\Z_i; \bz_0) \{ Y_i - g(\Z_i\trans \btheta^*)\}^2 \f(\Z_i)\f(\Z_i)\trans.
\]

Under (iii) and (iv) in Assumption <ref>,
$\pr(\Omega_2\mid \Omega_0) \geq 1 -  \exp\{-
C C_{\epsilon}^2n\submS^{2\epsilon}/K_2^2\}$, where $C$ is a universal constant, and $K_2 = 4C_w \eta_0^2 D_0^2$.

Note that given data in $\mathcal{I}\submSU$ and $\mathcal{I}\submTU$,  in event  $\Omega_0$,

\[w(\Z_i; \bz_0) \{ Y_i - g(\Z_i\trans \btheta^*)\}^2 \{\f(\Z_i)\trans(\wh \bz - \bz_0)\}^2\]
is i.i.d. and  sub-exponential.
By Lemma <ref> and Lemma <ref>, we have that for any $t > 0$,

\[
\pr\left\{ |(\wh \bz - \bz_0)\trans \T_{1}(\wh \bz - \bz_0) - (\wh \bz - \bz_0)\trans E\submS(\T_{1})(\wh \bz - \bz_0)| \geq t \mid \wh \bz \right\}  \leq \exp\left\{ -C\min \left(\frac{t^2}{K_2^2} ,\frac{t}{K_2}\right)n\submS\right\},
\]

where $K_2 = 4C_w \eta_0^2 D_0^2$.
Let $t = C_{\epsilon} n\submS^{ \epsilon-1/2} \leq K_2$, then

\begin{align*}
\pr\left\{ |(\wh \bz - \bz_0)\trans \T_{1}(\wh \bz - \bz_0) - (\wh \bz - \bz_0)\trans E\submS(\T_{1})(\wh \bz - \bz_0)| \geq  C_{\epsilon}n\submS^{ \epsilon-1/2} \mid \wh \bz \right\}  \leq \exp\left\{-
\frac{C C_{\epsilon}^2}{K_2^2} n\submS^{2\epsilon}
\right\}.
\end{align*}

Hence, by (iii) and (iv) in Assumption <ref>, in the event $\Omega_0$,  we have

\begin{align*}
(\wh \bz - \bz_0)\trans \T_{1}(\wh \bz - \bz_0) &\leq (\wh \bz - \bz_0)\trans E\submS(\T_{1})(\wh \bz - \bz_0) +  C_{\epsilon} n\submS^{ \epsilon-1/2} \\
&\leq  \frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon} n\submS^{ \epsilon-1/2}
\end{align*}

holds with probability at least $1 -  \exp\{-
C C_{\epsilon}^2n\submS^{2\epsilon}/K_2^2\}$.

Suppose that (iii) in Assumption <ref> holds. Then
we have

\begin{align*}
\langle  \nabla \mL_n(\breve \btheta;  \bz_0) - \nabla \mL_n( \btheta;  \bz_0) ,  \breve \btheta - \btheta \rangle \leq  \exp(C_{\f(\z)}\|\wh \bz - \bz_0\|_1)
\langle  \nabla \mL_n(\breve \btheta; \wh \bz) - \nabla \mL_n( \btheta; \wh \bz) ,  \breve\btheta - \btheta \rangle,
\end{align*}

for any $\wt \btheta$ and $\btheta \in \mR^d$.

Under (iii) in Assumption <ref>, since we assume that $g(\cdot)$ is non-decreasing, we have

\begin{align*}
&\langle  \nabla \mL_n(\breve \btheta; \bz_0) - \nabla \mL_n( \btheta; \bz_0) ,  \breve \btheta - \btheta \rangle \\
=& \frac{1}{n\submS+n\submT}\sum_{i\in \mathcal{I}\submSL, \mathcal{I}\submTL}w(\Z_i; \bz_0)^{I(i\in \mathcal{I}\submSL)}\{g(\breve \btheta\trans\Z_i) - g(\btheta\trans\Z_i) \}
(\breve \btheta -\btheta)\trans\Z_i \\
=& \frac{1}{n\submS+n\submT}\sum_{i\in \mathcal{I}\submSL, \mathcal{I}\submTL}[w(\Z_i; \wh\bz)\exp\{ \f(\Z_i)\trans (\bz_0 - \wh \bz)\}] ^{I(i\in \mathcal{I}\submSL)}\{g(\breve\btheta\trans\Z_i) - g(\btheta\trans\Z_i) \}
(\breve \btheta -\btheta)\trans\Z_i  \\
\leq& \exp(C_{\f(\z)}\|\wh \bz - \bz_0\|_1) \frac{1}{n\submS+n\submT}\sum_{i\in \mathcal{I}\submSL, \mathcal{I}\submTL}w(\Z_i; \wh\bz)^{I(i\in \mathcal{I}\submSL)}\{g(\breve \btheta\trans\Z_i) - g(\btheta\trans\Z_i) \}
(\breve \btheta -\btheta)\trans\Z_i.
\end{align*}

In the event $\Omega_0\cap \Omega_2$,  under (iii)  in Assumption <ref>, we have

\begin{align*}
& \langle  \nabla \mL_n( \btheta^*;  \bz_0)- \nabla \mL_n( \btheta^*;  \wh \bz) ,  \bDel \rangle  \\
\leq & e^{\eta_0}\frac{n\submS}{n\submS +n\submT}  \left(\frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon} n\submS^{ \epsilon-1/2}\right)^{1/2}
\left\{\frac{1}{n\submS}\sum_{i \in \mathcal{I}\submSL}
w(\Z_i; \bz_0)(\Z_i\trans \bDel)^2 \right\}^{1/2}.
\end{align*}

By Taylor expansion and Cauchy-Schwartz inequality, for
$u \in (0,1]$, we have

\begin{align*}
& \langle - \nabla \mL_n( \btheta^*;  \wh \bz) ,  \bDel \rangle -
\langle - \nabla \mL_n( \btheta^*;  \bz_0) ,  \bDel\rangle \\
=  & \frac{1}{n\submS+n\submT}\sum_{i\in \mathcal{I}\submSL}\{w(\Z_i; \wh \bz) - w(\Z_i; \bz_0) \}\{ Y_i  - g(\Z_i\trans \btheta^*) \}\Z_i\trans\bDel\\
=&  \frac{1}{n\submS+n\submT}\sum_{i \in \mathcal{I}\submSL}
w(\Z_i; \bz_0) \exp\{u \f(\Z_i)\trans(\wh \bz - \bz_0)\} \f(\Z_i)\trans(\wh \bz - \bz_0) \{ Y_i -  g(\Z_i\trans \btheta^*)\}\Z_i\trans\bDel\\
%  \leq &   \exp\{C_{\f_\z}\|\wh \bz - \bz_0\|_1\}
%  \left[ \frac{1}{n\submS+n\submT}\sum_{i \in \mathcal{I}_{\mS, L}}
%  w(\Z_i; \bz_0)\{\Z_i\trans(\wt \btheta - \btheta)\}^2\right]^{1/2} \\
%  &\times \left[\frac{1}{n\submS+n\submT}\sum_{i \in \mathcal{I}_{\mS, L}}
%  w(\Z_i; \bz_0)\{ Y_i - g(\btheta\trans \z_i)\}^2 \{\f(\Z_i)\trans(\wh \bz - \bz_0)\}^2 \right]^{1/2}\\
\leq & \exp(C_{\f(\z)} \|\wh \bz - \bz_0\|_1)   \frac{n\submS}{n\submS+n\submT}
\left\{(\wh \bz - \bz_0)\trans\T_{1}(\wh \bz - \bz_0)\right\}^{1/2}
\left\{\frac{1}{n\submS}\sum_{i \in \mathcal{I}\submSL}
w(\Z_i; \bz_0)(\Z_i\trans\bDel)^2\right\}^{1/2} \\
\leq & e^{\eta_0} \frac{n\submS}{n\submS+n\submT}
\left\{(\wh \bz - \bz_0)\trans\T_{1}(\wh \bz - \bz_0)\right\}^{1/2}
\left\{\frac{1}{n\submS}\sum_{i \in \mathcal{I}\submSL}
w(\Z_i; \bz_0)(\Z_i\trans\bDel)^2\right\}^{1/2},
\end{align*}

\[
\T_{1} = \frac{1}{n\submS}\sum_{i \in \mathcal{I}\submSL}
w(\Z_i; \bz_0) \{ Y_i - g(\Z_i\trans \btheta^*)\}^2 \f(\Z_i)\f(\Z_i)\trans.
\]

By Lemma <ref>,  in the event  $\Omega_0\cap \Omega_2$,

\begin{align*}
(\wh \bz - \bz_0)\trans \T_{1}(\wh \bz - \bz_0) \leq  \frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon}n\submS^{ \epsilon-1/2}.
\end{align*}

Proof of Proposition <ref>:

Let $\bDel = \wt \btheta - \btheta^*$.
Denote by $\Omega_3$ the event that

\begin{align}
\label{eq:RSC1}
\langle \nabla\mL_n(\wt\btheta;\bz_0) -  \nabla\mL_n(\btheta^*; \bz_0), \bDel \rangle  \geq
\begin{cases}
\alpha_1 \|\bDel\|_2^2  - \tau_1\sqrt{ \frac{\log~d}{n\submS+ n\submT}}\|\bDel\|_1^2, & \forall \|\bDel\|_2 \leq  1  \\
\alpha_2 \|\bDel\|_2  - \tau_2\sqrt{ \frac{\log~d}{n\submS + n\submT}}\|\bDel\|_1, & \forall \|\bDel\|_2 > 1
\end{cases}
\end{align}

where $\alpha_1$ and $\alpha_2$ are positive constants, and $\tau_1$ and $\tau_2$ are nonnegative constants. They depend only on $w_0(\cdot)$, $g(\cdot)$, constants $C_{\z}$, $\kappa\submS$, and $\kappa\submT$.
Under (i) (ii) and (iii) in Assumption <ref>,
By Corollary 2 in <cit.> and Proposition 2 in <cit.>, we have $\pr(\Omega_3) \geq 1 - c_1\exp\{- c_2 (n\submS +n\submT)\}$, where $c_1$ and $c_2$ are two constants.

First of all, we show that by choosing an appropriate $\lambda_\btheta$,
we have $\|\bDel\|_2\leq 1$.
In $\Omega_3$, follow the proof of Theorem 1 in <cit.>,
if $\|\bDel\|_2 > 1$,
we have
⟨∇_n(;_0) -  ∇_n(^*; _0), ⟩≥α_2 _2 - τ_2 √(log  d/n+ n)_1.
By Lemma <ref>, in the event $\Omega_0$, we  have

\begin{align*}
\langle \nabla\mL_n(\wt\btheta;\bz_0) -  \nabla\mL_n(\btheta^*; \bz_0), \bDel \rangle
\leq e^{\eta_0}\langle \nabla\mL_n(\wt\btheta;\wh \bz) -  \nabla\mL_n(\btheta^*; \wh \bz), \bDel \rangle.
\end{align*}

Since $\nabla \mL_n(\wt \btheta; \wh \bz) + \nabla p_{\lambda_\btheta}(\wt \btheta) = \0$, then

\[
\langle -\nabla p_{\lambda_\btheta}(\wt \btheta) -  \nabla\mL_n(\btheta^*; \wh \bz), \bDel \rangle \geq e^{-\eta_0} \alpha_2 \|\bDel\|_2 - e^{-\eta_0} \tau_2 \sqrt{\frac{\log~d}{n\submS + n\submT}}\|\bDel\|_1.
\]

Under Assumption <ref>, by Lemma 4 in <cit.>, we have

\[
\|\nabla p_{\lambda_\btheta}(\wt \btheta)\|_\infty \leq \lambda_\btheta L_\btheta.
\]

In the event $\Omega_0 \cap \Omega_1 \cap \Omega_2$, by Lemma <ref> and  Lemma <ref>, we obtain

\begin{align*}
\langle -\nabla p_{\lambda_\btheta}(\wt \btheta) -  \nabla\mL_n(\btheta^*; \wh \bz), \bDel \rangle \leq \left(\lambda_\btheta L_\btheta +   \frac{\lambda_\btheta L_\btheta }{4}\right) \|\bDel\|_1 +T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2},
\end{align*}

T_2,0=e^η_0 n/n+n  (D_1 C_w/κ_  - _0_2^2 + C_ϵ n^ ϵ-1/2)^1/2= o(1),

\[
\T_{2} = \frac{1}{n\submS}\sum_{i \in \mathcal{I}_{\mS, L}}
w(\Z_i; \bz_0)\Z_i\Z_i\trans.
\]

We denote by $\Omega_4$ the event  that
_2≤C_w/κ (
3/2_2^2 + log d/n _1^2
Under (i) and  (iii) in Assumption <ref>,
by Proposition 1 in <cit.>, we have $\pr(\Omega_4) >1- c_3\exp(-c_4n\submS)$.
Further, we have

\begin{align}
\label{eq:lemma13Loh2012}
T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2} \leq T_{2,1}
\|\bDel\|_2 +T_{2,2} \sqrt{\frac{\log~d}{n\submS + n\submT}} \|\bDel\|_1,
\end{align}

T_2,1 = T_2,0√(3C_w/2κ),   and  T_2,2 = T_2,0√((n+ n)C_w/nκ).
Recall that  $\|\bDel\|_1 \leq \|\wt \btheta\|_1 + \|\btheta^*\|_1  \leq 2R$.
Hence, in the event  $\Omega_0 \cap \Omega_1 \cap \Omega_2\cap \Omega_3\cap \Omega_4$, we have

\begin{align*}
& \left(e^{-\eta_0} \alpha_2 - T_{2,1}\right)\|\bDel\|_2\\ \leq &  \left\{\left( e^{-\eta_0} \tau_2+ T_{2,2}  \right)\sqrt{\frac{\log~d}{n\submS + n\submT}} +  \frac{5}{4}\lambda_\btheta L_\btheta \right\} \|\bDel\|_1  \\
\leq &   \left\{\left( e^{-\eta_0} \tau_2+ T_{2,2}  \right)\sqrt{\frac{\log~d}{n\submS + n\submT}} +  \frac{5}{4}\lambda_\btheta L_\btheta \right\} 2R.
\end{align*}

Note that $T_{2,0} = o(1)$, then for large enough $n\submS$, $N_{\mS}$ and $N\submT$,
$T_{2,1} \leq e^{-\eta_0}\alpha_2/2$  and $T_{2,2} \leq e^{-\eta_0}\tau_2/2$.
By our choice of $\lambda_\btheta$ and $R$, which satisfy

\[
\lambda_\btheta \leq \frac{e^{-\eta_0} \alpha_2 }{10 L_\btheta R},
\]

\[
R \leq \frac{\alpha_2}{12\max(\tau_1,\tau_2)} \sqrt{\frac{n\submS +n\submT}{\log~ d }},
\]

we obtain

\[
\|\bDel\|_2 \leq 1.
\]

Therefore, we have

⟨∇_n(;_0) -  ∇_n(^*; _0), ⟩≥α_1 _2^2  - τ_1 log d /n+ n_1^2.

By Lemma <ref>, in $\Omega_0$, since $\nabla \mL_n(\wt \btheta; \wh \bz) + \nabla p_\lambda(\wt \btheta) = \0$, we have

\begin{align}
\label{eq:1}
\langle -\nabla p_{\lambda_\btheta}(\wt \btheta) -  \nabla\mL_n(\btheta^*; \wh \bz), \bDel \rangle \geq e^{-\eta_0}  \alpha_1 \|\bDel\|_2^2  - e^{-\eta_0}\tau_1 \frac{\log~d }{n\submS + n\submT}\|\bDel\|_1^2.
\end{align}

For the penalty function, since $p_{\lambda_\btheta,\mu_\btheta}(\btheta) := p_{\lambda_\btheta}(\btheta) + \mu_\btheta\|\btheta\|_2^2/2$ is convex by assumption, we have
p_λ_,μ_(^*) - p_λ_,μ_() ≥⟨∇p_λ_,μ_(),  ^* -⟩,
implying that

⟨∇p_λ_(), ^* -⟩≤p_λ_(^*) - p_λ_() + μ_/2-^* _2^2.
By (<ref>) and (<ref>), we have

\begin{align*}
&\left(e^{-\eta_0}\alpha_1 - \frac{\mu_\btheta}{2}\right)\|\bDel\|_2^2 \\ \leq& p_{\lambda_\btheta}(\btheta^*) - p_{\lambda_\btheta}(\wt \btheta) -
\langle \nabla \mL_n(\btheta^*;\bz_0), \bDel\rangle
+e^{-\eta_0}\tau_1 \frac{\log~d}{n\submS + n\submT}\|\bDel\|_1^2 + T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2}\\
\leq& p_{\lambda_\btheta}(\btheta^*) -p_{\lambda_\btheta}(\wt \btheta)
+ \left\{\|\nabla \mL_n(\btheta^*;\bz_0)\|_\infty +e^{-\eta_0} \tau_1\frac{\log~d}{n\submS +n\submT}\|\bDel\|_1 \right\} \|\bDel\|_1 + T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2}  \\
\leq& p_{\lambda_\btheta}(\btheta^*) - p_{\lambda_\btheta}(\wt \btheta)
+ \left\{\|\nabla \mL_n(\btheta^*;\bz_0)\|_\infty + 2 R e^{-\eta_0}\tau_1\frac{\log~d}{n\submS +n\submT} \right\} \|\bDel\|_1 + T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2}.
\end{align*}

In $\Omega_1$, by (ii) in Assumption <ref>, we have
∇_n(^*;_0)_∞+ 2 R e^-η_0τ_1log d/n+n ≤λ_L_/4 + α_2/6e^-η_0√(log d/n+ n) ≤λ_L_/4 +λ_L_/4  = λ_L_/2,
and then

(e^-η_0α_1 - μ_/2)_2^2 ≤p_λ_(^*) - p_λ_()  + λ_L_/2_1 + T_2,0(_2)^1/2.

We then derive the upper bound of $\|\bDel\|_1$.
By Assumption <ref>, <cit.> shows that
$p_{\lambda_\btheta}(t)$ is $\lambda_\btheta L_\btheta$-Lipschitz as a function of $t$. In particular, all sub-gradients and derivatives
of $p_{\lambda_\btheta}$ are bounded in magnitude by $\lambda_\btheta L_\btheta$.

λ_L__1 ≤p_λ_() + μ_/2 _2^2,    ∀∈^d.
See the proof of Lemme 4 in <cit.> for details.
Since $p_{\lambda_\btheta}$ is sub-additive and symmetric around zero, then

p_λ_(^*) ≤p_λ_() + p_λ_(- ) =  p_λ_() + p_λ_().
Coming back to (<ref>), by (<ref>)  and  (<ref>), we have

\begin{align*}
\left(e^{-\eta_0}\alpha_1 - \frac{\mu_\btheta}{2}\right)\|\bDel\|_2^2 \leq&
p_{\lambda_\btheta}(\btheta^*) - p_{\lambda_\btheta}(\wt \btheta)
+ \frac{\lambda_\btheta L_\btheta}{2}  \left( \frac{p_{\lambda_\btheta}(\bDel)}{\lambda_\btheta L_\btheta}  +
\frac{\mu_\btheta}{2\lambda_\btheta L_\btheta} \|\bDel\|_2^2\right) + T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2} \\
\leq&  p_{\lambda_\btheta}(\btheta^*) - p_{\lambda_\btheta}(\wt \btheta)
+ \frac{1}{2} p_{\lambda_\btheta}(\bDel) + \frac{\mu_\btheta}{4}\|\bDel\|_2^2 + T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2}\\
\leq& \frac{3}{2}  p_{\lambda_\btheta}(\bDel_{S_\btheta})
+ p_{\lambda_\btheta}\left(\btheta^*_{S_\btheta^c}\right)
- p_{\lambda_\btheta}(\wt \btheta_{S_\btheta^c})
+  \frac{1}{2}p_{\lambda_\btheta}(\bDel_{S_\btheta^c})+ \frac{\mu_\btheta}{4}\|\bDel\|_2^2 + T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2}.
\end{align*}

Note that
p_λ_(_S_^c) - p_λ_(_S_^c) ≤p_λ_( ^*_S_^c).
By the assumption that $e^{-\eta_0}\alpha_1 > 5\mu_\btheta/4 $, we have
0≤(e^-η_0α_1 - 3μ_/4)_2^2 ≤3/2  p_λ_(_S_)
+ 2p_λ_(^*_S^c)
- 1/2p_λ_(_S_^c) + T_2,0(_2)^1/2.

Recall that $\bdel_0 =  \bthetabar-\btheta^*$, and $ \left(\bthetabar\right)_{S^c} = \0$.
0≤(e^-η_0α_1 - 3μ_/4)_2^2 ≤3/2  p_λ_(_S_)
+ 2p_λ_{(_0)_S_^c}
- 1/2p_λ_(_S_^c) + T_2,0(_2)^1/2.

We then need to compare three terms: $3 p_{\lambda_\btheta}(\bDel_{S_\btheta})$, $4p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\}$, and $2T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2}$.

(i) If $3 p_{\lambda_\btheta}(\bDel_{S_\btheta}) \geq \max[4p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\},  2T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2}] $,

(e^-η_0α_1 - 3μ_/4)_2^2 ≤9/2p_λ_(_S_) -  1/2p_λ_(_S_^c) .
Let $A$ denote the index set of the $s_\btheta$ largest elements of $\bDel$ in magnitude.
We have

\begin{align*}
\frac{9}{2}p_{\lambda_\btheta}(\bDel_{S_\btheta}) -  \frac{1}{2}p_{\lambda_\btheta}(\bDel_{S_\btheta^c})
\leq &\frac{9}{2}p_{\lambda_\btheta}(\bDel_{A}) -  \frac{1}{2}p_{\lambda_\btheta}(\bDel_{A^c}) \\
\leq & \frac{9}{2}\lambda_\btheta L_\btheta \|\bDel_A\|_1 - \frac{1}{2}\lambda_\btheta L_\btheta \|\bDel_{A^c}\|_1.
\end{align*}

The last inequality holds by Lemma 5 in <cit.>.

Note that
_A_1 ≤√(s_) _A_2 ≤√(s_) _2,
then we have
(e^-η_0α_1 - 3μ_/4)_2^2 ≤9/2λ_L__A_1 ≤9/2λ_L_√(s_) _2.
_2 ≤18 λ_L_√(s_)/4e^-η_0α_1 - 3μ_.
The $l_1$-bound is given as
_1 ≤ _A_1 + _A^c_1 ≤10 _A_1 ≤10 √(s_)_2 ≤ 180 λ_L_s_/4e^-η_0α_1 - 3μ_.

(ii) If  $4p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\} \geq \max[
3 p_{\lambda_\btheta}(\bDel_{S_\btheta}),

(e^-η_0α_1 - 3μ_/4)_2^2 ≤6 p_λ_{(_0)_S_^c} -  1/2p_λ_(_S_^c).
We have

_2^2 ≤24/4e^-η_0α_1 - 3μ_  p_λ_{(_0)_S_^c}.

\begin{align}
\label{eq:scn2_2}
p_{\lambda_\btheta}(\bDel) =& p_{\lambda_\btheta}(\bDel_{S_\btheta}) + p_{\lambda_\btheta}(\bDel_{S_\btheta^c}) \n\\
\leq & \frac{4}{3}p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\}+ 12 p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\} \n\\
=& \frac{40}{3} p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\}.
\end{align}

By (<ref>), (<ref>) and (<ref>), we have

_1     ≤    1/λ_L_ [40/3 p_λ_{(_0)_S_^c} +μ_/2_2^2]

40/3 + 12 μ_/4e^-η_0α_1 - 3μ_ ).
Under Assumption <ref>, we know

\[p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\} \leq \lambda_\btheta L_\btheta\|(\bdel_0)_{S_\btheta^c}\|_1.\]

Therefore, based on (<ref>) and (<ref>),
we have
_2^2 ≤24 λ_L_/4e^-η_0α_1 - 3μ_  (_0)_S_^c_1,
_1  ≤(
40/3 + 12 μ_/4e^-η_0α_1 - 3μ_ )(_0)_S_^c _1.

(iii) If  $2T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2} \geq \max[4p_{\lambda_\btheta}\left\{(\bdel_0)_{S_\btheta^c}\right\},
3 p_{\lambda_\btheta}(\bDel_{S_\btheta})]$,

\begin{align*}
\left(e^{-\eta_0}\alpha_1 - \frac{3\mu_\btheta}{4}\right)\|\bDel\|_2^2
\leq & 3T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2} -  \frac{1}{2}p_{\lambda_\btheta}(\bDel_{S_\btheta^c})\\
= & 3T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2} -  \frac{1}{2}p_{\lambda_\btheta}(\bDel) + \frac{1}{2}p_{\lambda_\btheta}(\bDel_{S_\btheta})  \\
\leq& \frac{10}{3} T_{2,0}(\bDel\trans\T_{2}\bDel)^{1/2} -  \frac{1}{2}p_{\lambda_\btheta}(\bDel).
\end{align*}

By (<ref>) and (<ref>), we have

\begin{align*}
\left(e^{-\eta_0}\alpha_1 - \frac{3\mu_\btheta}{4}\right)\|\bDel\|_2^2
\leq & \frac{10}{3} T_{2,1}
\|\bDel\|_2 +\frac{10}{3}T_{2,2} \sqrt{\frac{\log~d}{n\submS + n\submT}} \|\bDel\|_1 -  \frac{1}{2}p_{\lambda_\btheta}(\bDel) \\
\leq& \frac{10}{3} T_{2,1}
\|\bDel\|_2 +\frac{10}{3}T_{2,2} \sqrt{\frac{\log~d}{n\submS + n\submT}} \|\bDel\|_1 -  \frac{1}{2}\lambda_\btheta L_\btheta\|\bDel\|_1 + \frac{\mu_\btheta}{2}\|\bDel\|_2^2.
\end{align*}

By rearrangement, we obtain

\begin{align*}
\left(e^{-\eta_0}\alpha_1 - \frac{5\mu_\btheta}{4}\right)\|\bDel\|_2^2 \leq \frac{10}{3} T_{2,1}
\|\bDel\|_2 + \left(\frac{10}{3}T_{2,2} \sqrt{\frac{\log~d}{n\submS + n\submT}} -  \frac{1}{2}\lambda_\btheta L_\btheta \right)\|\bDel\|_1.
\end{align*}

Since $T_{2,2} = o(1)$ and $\lambda_\btheta \asymp \sqrt{(n\submS +n\submT)^{-1} \log~d}$, when $n\submS$, $N\submS$, and $N\submT$ are large enough,

\[
\frac{10}{3}T_{2,2} \sqrt{\frac{\log~d}{n\submS + n\submT}} \leq  \frac{1}{4}\lambda_\btheta L_\btheta.
\]

Hence, under the assumption $e^{-\eta_0}\alpha_1 > 5\mu_\btheta/4$, by (<ref>) and (<ref>),
we have
_2 ≤η_1 (D_1 C_w/κ_  - _0_2^2 + C_ϵ n^ ϵ-1/2)^1/2,

\[
\eta_1 = \frac{40}{12 e^{-\eta_0} \alpha_1 - 15 \mu_\btheta}\sqrt{\frac{3C_w}{2\kappa\submS}}e^{\eta_0} \frac{n\submS}{n\submS +n\submT}.
\]

Further, since $\left(e^{-\eta_0}\alpha_1 - 5\mu_\btheta/4\right)\|\bDel\|_2^2 >0$,

\begin{align*}
\|\bDel\|_1 \leq &
\frac{40}{3\lambda_\btheta L_\btheta} T_{2,1}  \|\bDel\|_2 \\
\leq & \frac{\eta_2}{\lambda_\btheta L_\btheta}\left(\frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon} n\submS^{ \epsilon-1/2}\right),
\end{align*}

\[
\eta_2 = \frac{800C_we^{2\eta_0} n\submS^2}{ 3 \kappa\submS (4 e^{-\eta_0}\alpha_1 - 5 \mu_\btheta)(n\submS +n\submT)^2}.
\]

In conclusion,
in the event $\Omega_0 \cap \Omega_1 \cap \Omega_2\cap \Omega_3\cap \Omega_4$,
we have
_2^2 ≤18^2 λ_^2 L_^2 s_/(4e^-η_0α_1 - 3μ_)^2
+ 24 λ_L_/4e^-η_0α_1 - 3μ_  (_0)_S_^c_1 +  η_1^2 (D_1 C_w/κ_  - _0_2^2 + C_ϵ n^ ϵ-1/2),
_1 ≤ 180 λ_L_s_/4e^-η_0α_1 - 3μ_ +   (
40/3 + 12 μ_/4e^-η_0α_1 - 3μ_ )(_0)_S_^c _1+η_2/λ_L_(D_1 C_w/κ_  - _0_2^2 + C_ϵ n^ ϵ-1/2).
In particular, for the Lasso penalty, where $L_\btheta=1$ and $\mu_\btheta = 0$, we have

\begin{align*}
\|\bDel\|_2^2 \leq
\frac{81\lambda_\btheta^2 s_\btheta}{4e^{-2\eta_0}\alpha_1^2}+
\frac{6 \lambda_\btheta }{e^{-\eta_0}\alpha_1 } \|(\bdel_0)_{S_\btheta^c}\|_1
+ \frac{50 C_w n\submS^2 }{3\alpha_1^2\kappa\submS(n\submS +n\submT)^2}\left(\frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon} n\submS^{ \epsilon-1/2}\right),
\end{align*}

\begin{align*}
\|\bDel\|_1 \leq \frac{45 \lambda_\btheta s_\btheta}{e^{-\eta_0}\alpha_1} + \frac{40}{3}\|(\bdel_0)_{S_\btheta^c} \|_1 + \frac{200 e^{\eta_0}C_wn\submS^2}{3\alpha_1\kappa\submS(n\submS +n\submT)^2\lambda_\btheta }\left(\frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon} n\submS^{ \epsilon-1/2}\right).
\end{align*}

§.§ Proofs of Theorem <ref>

Denote by $\Omega_5$ the event that

\[
\|\nabla \mL_{n\submT}\supmT( \bthetabar)\|_\infty \leq C_{\bdel, \infty}\sqrt{\frac{\log~ d}{n\submT}}.
\]

Under (i) and (iv) in Assumption <ref>,
if $ C_{\bdel, \infty} > K_2/\sqrt{C}$, then
$\pr(\Omega_5) \geq 1 - d^{-\epsilon_2}$, where $\epsilon_2 = CC_{\bdel, \infty}^2/K_2^2- 1$, $C$ is a universal constant, and $K_2 = 2 D_2C_\z$.

Recall that

\[
\nabla\mL_{n\submT}\supmT(\bthetabar) = \frac{1}{n\submT}\sum_{i\in  \mathcal{I}\submTL}\{g(\Z_i\trans \bthetabar) - Y_i \}\Z_i.
\]

Let $V_{ij} = \{g(\Z_i\trans \bthetabar) - Y_i \}Z_{ij}$, $i\in \mathcal{I}\submTL$, $j = 1, \dots, d$.
Under (i) and (iv) in Assumption <ref>, by Lemma <ref>, we know $V_{ij}$
is sub-exponential with sub-exponential  norm uniformly bounded by $K_2 = 2D_2C_\z$.
Further, by the definition of $\bthetabar$, $V_{ij}$ is mean-zero.
By Lemma <ref> and the union bound inequality, we have that for any $t > 0$
(∇_n() _∞ ≥t ) ≤dexp{ -Cmin(t^2/K_2^2 ,t/K_2) n},
where $C$ is a universal constant.
Let $t = C_{\bdel, \infty}\sqrt{ n\submT^{-1}\log~d} \leq K_2$, then

\begin{align*}
&\pr \left\{\|\nabla\mL_{n\submT}\supmT(\bthetabar) \|_{\infty} \geq C_{\bdel, \infty}\sqrt{n\submT^{-1}\log~d} \right\}\\
\leq &
d\exp\left\{ -C\min \left(\frac{ C_{\bdel, \infty}^2 n\submT^{-1}\log~d}{K_2^2} ,\frac{C_{\bdel, \infty}\sqrt{n\submT^{-1}\log~d} }{K_2}\right)n\submT\right\} \\
\leq & d\exp\left\{ -\frac{ CC_{\bdel, \infty}^2 }{K_2^2} \log~d \right\} \\
\leq & d^{-\epsilon_2},
\end{align*}

where $\epsilon_2 = CC_{\bdel, \infty}^2/K_2^2- 1$.

Here we choose $C_{\bdel, \infty} \leq\lambda_\bdel L_\bdel\sqrt{n\submT/\log~ d}/4$.

Proof of Theorem <ref>:

Using the same idea of the proof of Theorem 3 in <cit.>, we assume that $\|\bthetabar\|_2 \leq 1$ in order to simplify our exposition, and similar derivations  can be obtained for general $\|\bthetabar\|_2$.
In $\Omega_0 \cap \Omega_1 \cap \Omega_2$, we have shown that $\|\bDel\|_2 \leq 1$ .
Then $\|\wt \btheta + \bdel_0\|_2 = \|\bthetabar+ \bDel\|_2 \leq 2$.
Note that $n\submT \precsim n\submS$, then
we also have $\| \wt \btheta + \bdel_0\|_1 \precsim \sqrt{n\submT/\log~d}$  by (i) in Assumption <ref> and Proposition <ref>.
Denote by $\Omega_6$ the event that

\begin{align}
\label{eq:RSC2}
\langle  \nabla \mL_{n\submT}\supmT(\wt \btheta + \wh \bdel )  - \nabla \mL_{n\submT}\supmT(\wt \btheta + \bdel_0 ), \bomega  \rangle \geq
\begin{cases}
\alpha_3 \|\bomega\|_2^2  - \tau_3\sqrt{ \frac{\log~d}{n\submT}}\|\bomega\|_1^2, & \forall \|\bomega\|_2 < 3 \\
\alpha_4 \|\bomega\|_2  - \tau_4\sqrt{ \frac{\log~d}{n\submT}}\|\bomega\|_1, & \forall \|\bomega\|_2 \geq 3
\end{cases}
\end{align}

where $\alpha_3$ and $\alpha_4$ are positive constants, and $\tau_3$ and $\tau_4$ are nonnegative constants. They depend only on the GLM, the sub-Gaussian parameter $C_{\z, \mT}$, and the covariance matrix $E\submT(\Z_i\Z_i\trans)$.
Proposition 1 in <cit.> shows that $\pr(\Omega_6) \geq 1 - c_5\exp(- c_6 n\submT)$.
The following derivations are in  $\Omega_0\cap\Omega_1\cap\Omega_2\cap\Omega_3\cap \Omega_4\cap \Omega_5\cap\Omega_6$.
If $\|\bomega\|_2\geq 3$, then
⟨∇_n(+ )  - ∇_n(+ _0 ), ⟩≥α_4 _2  - τ_4√( log d/n)_1.
Since $\nabla \mL_{n\submT}\supmT(\wt \btheta + \wh \bdel ) +\nabla p_{\lambda_\bdel}(\wh \bdel) = \0$, then

⟨-∇p_λ_()  - ∇_n(+ _0 ), ⟩≥α_4 _2  - τ_4√( log d/n)_1.
By Hölder's inequality and triangle inequality, we have

\begin{align*}
&\langle  -\nabla p_{\lambda_\bdel}(\wh \bdel)  - \nabla \mL_{n\submT}\supmT(\wt \btheta + \bdel_0 ), \bomega  \rangle\\
\leq& \left\{ \|\nabla p_{\lambda_\bdel}(\wh \bdel)\|_\infty + \|\nabla \mL_{n\submT}\supmT( \bthetabar)\|_\infty \right\}\|\bomega\|_1 + |\langle \nabla \mL_{n\submT}\supmT(\wt \btheta + \bdel_0 ) -
\nabla \mL_{n\submT}\supmT(\bthetabar), \bomega \rangle |.
\end{align*}

By Lemma 4 in <cit.>, we know $ \|\nabla p_{\lambda_\bdel}(\wh \bdel)\|_\infty \leq \lambda_\bdel L_\bdel$.
By Lemma <ref>, we know that
∇_n( )_∞≤λ_L_/2,
with high probability.
For the upper bound of $|\langle \nabla \mL_{n\submT}\supmT(\wt \btheta + \bdel_0 ) -
\nabla \mL_{n\submT}\supmT(\bthetabar), \bomega \rangle|$,
under (ii) in Assumption <ref>, by Cauchy–Schwarz inequality, and
Proposition 1 in <cit.>,

we have

\begin{align}
\label{eq:effect_delta}
&|\langle \nabla \mL_{n\submT}\supmT(\wt \btheta + \bdel_0 ) -
\nabla \mL_{n\submT}\supmT(\bthetabar), \bomega \rangle| \n\\
=& |\frac{1}{n\submT} \sum_{i\in \mathcal{I}_{\mT,L}}  \left[g\left\{(\bthetabar+ \bDel )\trans \Z_i \right\} -g(\bthetabar\trans \Z_i)\right] \Z_i\trans \bomega| \n\\
\leq & C_g \left( \frac{1}{n\submT} \sum_{i\in \mathcal{I}_{\mT,L}}  \bDel\trans \Z_i\Z_i\trans\bDel\right)^{1/2}\left( \frac{1}{n\submT} \sum_{i\in \mathcal{I}_{\mT,L}}  \bomega\trans \Z_i\Z_i\trans\bomega \right)^{1/2} \n\\
\leq& \frac{C_g }{\sqrt{\kappa\submT}} \left( \frac{1}{n\submT} \sum_{i\in \mathcal{I}_{\mT,L}}  \bDel\trans \Z_i\Z_i\trans\bDel\right)^{1/2}
\left( \frac{3}{2}\|\bomega\|_2^2  + \frac{\log~d}{n\submT}\|\bomega\|_1^2\right)^{1/2}\n
\\
\leq&
\frac{C_g }{\sqrt{\kappa\submT}}
\left( \frac{1}{n\submT} \sum_{i\in \mathcal{I}_{\mT,L}}  \bDel\trans \Z_i\Z_i\trans\bDel\right)^{1/2}   \left( \sqrt{\frac{3}{2}}\|\bomega\|_2 + \sqrt{\frac{\log~d}{n\submT}}\|\bomega\|_1  \right).
\end{align}

By Proposition 1 in <cit.>, we also have

1/n ∑_i∈ℐ_,L  _i_i≤3/2κ_2^2 + 1/κlog d/n _1^2.

Note that $\|\bDel\|_2^2\precsim 1$ and $\lambda_\btheta \|\bDel\|_1 \asymp \|\bDel\|_2^2 $. When $c$ in (i) in Assumption <ref> is large enough, we have
C_g√(3/2κ) (3/2κ_2^2 + 1/κlog d/n _1^2)^1/2 ≤α_4/2,
C_g√(3/2κ)( 1/n ∑_i∈ℐ_,L  _i_i)^1/2  ≤α_4/2.
Note that  $\|\bdel_0\|_1\leq R_\bdel$ and $\|\wh \bdel\|_1\leq R_\bdel$ by assumption, then $\|\bomega\|_1 \leq 2R_\bdel$.
Come back to (<ref>), under (ii) and (iii) in Assumption <ref>, we have
_2 ≤4R_/α_4{
3/2λ_L_+ (τ_4 + α_4/2) √(log d/n)  } ≤1/2 + 1/2  = 1.
Hence, we have

⟨∇_n(+ )  - ∇_n(+ _0 ), ⟩≥α_3 _2^2  - τ_3 log d/n_1^2.
For the penalty function, since $p_{\lambda_\bdel,\mu_\bdel}(\bdel) := p_{\lambda_\bdel}(\bdel) + \mu_\bdel\|\bdel\|_2^2/2$ is convex by assumption, we have
p_λ_,μ_(_0 ) - p_λ_,μ_() ≥⟨∇p_λ_, μ_(),  -⟩,
implying that

⟨∇p_λ_(),  -⟩≤p_λ_(_0) - p_λ_() + μ_/2_2^2.
Since  $\nabla \mL_{n\submT}\supmT(\wt\btheta + \wh \bdel) + p_{\lambda_\bdel}(\wh \bdel) = \0$,
and $\|\bomega \|_1 \leq 2R_\bdel$.
By (<ref>) and (<ref>), we have

(α_3 - μ_/2)_2^2

≤    p_λ_(_0 ) - p_λ_() -
⟨∇_n(+ _0 ), ⟩+ τ_3 log d/n_1^2

=    p_λ_(_0 ) - p_λ_() -
⟨∇_n(), ⟩-  ⟨∇_n(+ )-∇_n(), ⟩+ τ_3 log d/n_1^2.
Using  (<ref>) and the fact that $|ab| \leq ca^2/2 + b^2/(2c)$ for every $c>0$,
we further obtain
(3α_3/4 -μ_/2 )_2^2

p_λ_(_0 ) - p_λ_()
+ 3C_g^2/2κα_3 ( 1/n ∑_i∈ℐ_,L  _i_i)

+   { ∇_n()_∞+ C_g/√(κ)√(log d/n)( 1/n ∑_i∈ℐ_,L  _i_i)^1/2
+ τ_3 log d/n_1}_1.
By Lemma <ref>, we have

Since $\|\bomega\|_1 \leq 2R_\bdel$, by (ii) and (iii) in Assumption <ref>,  we also have

τ_3 log d/n_1 ≤2 R_τ_3log d/n

≤α_4/2√(log d/n) ≤λ_L_/8.

Note that $\|\bDel\|_2^2\precsim 1$ and $\lambda_\btheta \|\bDel\|_1 \asymp \|\bDel\|_2^2 $. When $c$ in (i) in Assumption <ref> is large enough, we have

C_g/√(κ)√(log d/n)
(3/2κ_2^2+ 1/κlog d/n _1^2 )^1/2 ≤λ_L_/8,
C_g/√(κ)√(log d/n)( 1/n ∑_i∈ℐ_,L  _i_i)^1/2≤λ_L_/8.
Then we obtain
(3α_3/4 -μ_/2 )_2^2
≤p_λ_(_0 ) - p_λ_()
+ 3C_g^2/2κα_3 ( 1/n ∑_i∈ℐ_,L  _i_i)
+ 1/2λ_L__1.
Similar to (<ref>), we know
λ_L__1 ≤p_λ_() + μ_/2 _2^2.
(3α_3/4 - 3μ_/4)_2^2 ≤p_λ_(_0) - p_λ_()
+ 1/2p_λ_() + 3C_g^2/2κα_3 ( 1/n ∑_i∈ℐ_,L  _i_i).
p_λ_() -  p_λ_()  ≤p_λ_(_0),
(3α_3/4 - 3μ_/4)_2^2 ≤2p_λ_(_0)  - 1/2p_λ_() + 3C_g^2/2κα_3 ( 1/n ∑_i∈ℐ_,L  _i_i).
If $2p_{\lambda_\bdel}(\bdel_0) \geq3C_g^2\left( n\submT^{-1} \sum_{i\in \mathcal{I}_{\mT,L}}  \bDel\trans \Z_i\Z_i\trans\bDel\right)/(2\kappa\submT\alpha_3) $, we have

(3α_3/4 - 3μ_/4)_2^2 ≤4p_λ_(_0) - 1/2p_λ_().
Under Assumption <ref>, we know

\[p_{\lambda_\bdel}(\bdel_0) \leq \lambda_\bdel L_\bdel\|\bdel_0\|_1,\]

_2^2 ≤16λ_L_/3(α_3- μ_)_0_1.
Since $\alpha_3 > \mu_\bdel$, then by (<ref>) we also have
p_λ_() ≤8p_λ_(_0).
Under Assumption <ref>, we further have

λ_L__1 ≤p_λ_() + μ_/2 _2^2.

\begin{align*}
\|\bomega\|_1 &\leq \frac{8}{\lambda_\bdel L_\bdel} p_{\lambda_\bdel}(\bdel_0) + \frac{\mu_\bdel}{2\lambda_\bdel L_\bdel} \frac{16\lambda_\bdel L_\bdel}{3(\alpha_3- \mu_\bdel )}\|\bdel_0\|_1 \\
& \leq \left\{ 8 +  \frac{8\mu_\bdel}{3(\alpha_3 - \mu_\bdel)}\right\}\|\bdel_0\|_1.
\end{align*}

If $2p_{\lambda_\bdel}(\bdel_0) \leq3C_g^2\left( n\submT^{-1} \sum_{i\in \mathcal{I}_{\mT,L}}  \bDel\trans \Z_i\Z_i\trans\bDel\right)/(2\kappa\submT\alpha_3) $, we have
_2^2 ≤4C_g^2/κα_3(α_3 - μ_)
( 1/n ∑_i∈ℐ_,L  _i_i),
p_λ_() ≤3C_g^2/κα_3 ( 1/n ∑_i∈ℐ_,L  _i_i).
Similarly,  by (<ref>), we have

\[
\|\bomega\|_1 \leq \left\{ \left(3 + \frac{2\mu_\bdel}{\alpha_3-\mu_\bdel} \right) \frac{C_g^2}{\kappa\submT \alpha_3 \lambda_\bdel L_\bdel}\right\}\left( \frac{1}{n\submT} \sum_{i\in \mathcal{I}_{\mT,L}}  \bDel\trans \Z_i\Z_i\trans\bDel\right).
\]

In conclusion, by (<ref>),
we have

\begin{align*}
\|\bomega\|_1 \leq  \left\{ 8 +  \frac{8\mu_\bdel}{3(\alpha_3 - \mu_\bdel)}\right\}\|\bdel_0\|_1 + \left\{ \left(3 + \frac{2\mu_\bdel}{\alpha_3-\mu_\bdel} \right) \frac{C_g^2}{\kappa\submT^2 \alpha_3 \lambda_\bdel L_\bdel}\right\}\left( \frac{3}{2}\|\bDel\|_2^2 + \frac{\log~d}{n\submT} \|\bDel\|_1^2 \right),
\end{align*}

\begin{align*}
\|\bomega\|_2^2 \leq \frac{16\lambda_\bdel L_\bdel}{3(\alpha_3- \mu_\bdel )}\|\bdel_0\|_1 + \frac{4C_g^2}{\kappa\submT^2\alpha_3(\alpha_3 - \mu_\bdel)}
\left( \frac{3}{2}\|\bDel\|_2^2 + \frac{\log~d}{n\submT} \|\bDel\|_1^2 \right),
\end{align*}

and hence

\begin{align*}
\|\bthetahat\supmeta- \bthetabar\|_1 &\leq \left\{ 8 +  \frac{8\mu_\bdel}{3(\alpha_3 - \mu_\bdel)}\right\}\|\bdel_0\|_1 + \left\{ \left(3 + \frac{2\mu_\bdel}{\alpha_3-\mu_\bdel} \right) \frac{C_g^2}{\kappa\submT^2 \alpha_3 \lambda_\bdel L_\bdel}\right\}\left( \frac{3}{2}\|\bDel\|_2^2+ \frac{\log~d}{n\submT} \|\bDel\|_1^2 \right)\\
&+ \|\bDel\|_1,
\end{align*}

\begin{align*}
\|\bthetahat\supmeta- \bthetabar\|_2 &\leq \sqrt{\frac{16\lambda_\bdel L_\bdel}{3(\alpha_3- \mu_\bdel )}\|\bdel_0\|_1 + \frac{4C_g^2}{\kappa\submT^2\alpha_3(\alpha_3 - \mu_\bdel)}
\left( \frac{3}{2}\|\bDel\|_2^2 + \frac{\log~d}{n\submT} \|\bDel\|_1^2 \right)} + \|\bDel\|_2
\end{align*}

with probability at least $1 - \sum_{k=0}^6\pr(\Omega_k)$.

§.§ Proofs of Theorem <ref>

\begin{align*}
&\frac{2}{n\submT} \sum_{i \in \mathcal{I}\submTL^{(2)}}
\left\{G(\z_i\trans\bthetahat\supmetaone) - G(\z_i\trans\bthetabar)  - g(\z_i\trans\bthetabar)\z_i\trans (\bthetahat\supmetaone- \bthetabar ) \right\} \\
\leq &\frac{2}{n\submT} \sum_{i \in \mathcal{I}\submTL^{(2)}}
\left\{G(\z_i\trans\bthetahat\supmTone) - G(\z_i\trans\bthetabar)  - g(\z_i\trans\bthetabar)\z_i\trans ( \bthetahat\supmTone - \bthetabar ) \right\} \\
+&  \frac{2}{n\submT} \sum_{i \in \mathcal{I}\submTL^{(2)}}
\{ y_i - g(\z_i\trans \bthetabar)\}\z_i\trans(\bthetahat\supmetaone- \bthetahat\supmTone)
- \epsilon_{0,1}.
\end{align*}

Under the assumption that $\|\bdel_0\|_1  \precsim \sqrt{n\submT/ \log~d}$, we know $\|\bthetahat\supmetaone- \wh \btheta\supmTone\|_2 \leq \|\bthetahat\supmetaone-  \bthetabar\|_2 + \|\wh \btheta\supmTone - \bthetabar\|_2$ is bounded with high probability.
Since the data used to calculate $ \bthetahat\supmetaone$ and $\wh \btheta\supmTone$ are independent to the data in $\mathcal{I}\submTL^{(2)}$,  $\{Y_i - g(\Z_i\trans \bthetabar)\}\Z_i\trans (\bthetahat\supmetaone- \wh \btheta\supmTone)$, $i \in \mathcal{I}\submTL^{(2)}$ is independent to each other and sub-exponential with finite variance given data in the complement of  ${\mathcal{I}\submTL^{(2)}}$.
Based on the definition of $\bthetabar$, we have

\[
E [\{Y_i - g(\Z_i\trans \bthetabar)\}\Z_i \trans (\bthetahat\supmetaone- \wh \btheta\supmTone)] = E(E\submT[\{Y_i - g(\Z_i\trans \bthetabar)\} \Z_i] \trans (\bthetahat\supmetaone- \wh \btheta\supmTone))= 0.
\]

By Central Limit Theorem, we obtain

\[
\frac{2}{n\submT} \sum_{i \in \mathcal{I}\submTL^{(2)}}
\{ y_i - g(\z_i\trans \bthetabar)\}\z_i\trans(\bthetahat\supmetaone- \wh \btheta\supmTone) = O_p(n\submT^{-1/2}).
\]

By Proposition 1 in <cit.>,  we have

\begin{align*}
&\frac{2}{n\submT} \sum_{i \in \mathcal{I}\submTL^{(2)}}
\left\{G(\z_i\trans \bthetahat\supmetaone) - G(\z_i\trans\bthetabar)  - g(\z_i\trans\bthetabar)\z_i\trans (\bthetahat\supmetaone- \bthetabar ) \right\} +\epsilon_{0,1}\\
\precsim& \|\wh \btheta\supmTone - \bthetabar\|_2^2 + \frac{\log~d}{n\submT} \|\wh \btheta\supmTone - \bthetabar\|_1^2 + n\submT^{-1/2} \\
\precsim& \frac{\log~d}{n\submT}s_\btheta+ n\submT^{-1/2}.
\end{align*}

For the $l_2$ error bounds, note that we assume $\|\bthetabar\|_1\precsim \sqrt{n\submT/\log~d}$, and
by similar derivations in Appendix <ref> and Appendix <ref>, we have  $\|\bthetahat\supmetaone-  \bthetabar\|_2 \leq  3\|\bthetabar\|_2$.
By Proposition 1 in <cit.>, we have

\begin{align*}
&\frac{2}{n\submT} \sum_{i \in \mathcal{I}\submTL^{(2)}}
\left\{G(\z_i\trans \bthetahat\supmetaone) - G(\z_i\trans\bthetabar)  - g(\z_i\trans\bthetabar)\z_i\trans (\bthetahat\supmetaone- \bthetabar ) \right\} + \epsilon_{0,1} \\ &\geq \alpha_5 \|\bthetahat\supmetaone- \bthetabar \|_2^2 - \tau_5 \frac{\log~d}{n\submT} \|\bthetahat\supmetaone- \bthetabar \|_1^2 + \epsilon_{0,1} \end{align*}

with high probability,
where positive  $\alpha_5$  and nonnegative  $\tau_5$ are  constants depending on $g(\cdot)$, $\kappa\submT$ and $C_{\z, \mT}$.
We choose $\epsilon_{0,1}$ such that

\[
\alpha_5- \tau_5 \frac{\log~d}{n\submT} \frac{\|\bthetahat\supmetaone- \bthetabar \|_1^2}{ \|\bthetahat\supmetaone- \bthetabar \|_2^2 }  + \frac{\epsilon_{0,1}}{\|\bthetahat\supmetaone- \bthetabar \|_2^2 } \geq \kappa_{\rm RE}> 0,
\]

where $\kappa_{\rm RE}$ is a positive constant.
Then  we have

\[
\|\bthetahat\supmetaone- \bthetabar \|_2^2 \precsim \frac{\log~d}{n\submT}s_\btheta+ n\submT^{-1/2}.
\]

Hence, we obtain

\[
\|\bthetahat\supSTRIFLE - \bthetabar \|_2^2 \precsim \left(\frac{\log~d}{n\submS +n\submT} s_\btheta+ \sqrt{\frac{\log~d}{n\submT}} \|\bdel_0\|_1\right) \wedge
\left(  \frac{\log~d}{n\submT}s_\btheta+ n\submT^{-1/2} \right).
\]

§.§ Proofs of Theorem <ref>

Denote by $\Omega_8$ the event that

\[
\|\nabla \mL_{N\submT +n\submT}^{\bb}( \bb_0; \bthetabar)\|_\infty \leq C_{\bb, \infty}\sqrt{\frac{\log~p}{N\submT + n\submT}}.
\]

Under (i) and (iv) in Assumption <ref>, 	if $ C_{\bb, \infty} > K_3/\sqrt{C}$, then
$\pr(\Omega_8) \geq 1 - p^{-\epsilon_3}$, where $\epsilon_3 = CC_{\bb, \infty}^2/K_3^2- 1$, $C$ is a universal constant, and $K_3 = 2 D_3C_\z$.

By (<ref>), we have
∇_N+n^(_0;) = 1/N+ n ∑_i∈ℐ _i { g(_0_i) -g(_i) } +
1/N+ n ∑_i∈ℐ_i { g(_0_i) -Y_i}.
Under (i) and (iv) in Assumption <ref>, by Lemma <ref>, we know $ \X_i \{g(\bb_0\trans \X_i) -g(\bthetabar\trans\Z_i) \}$ and $\X_i \{g(\bb_0\trans \X_i) -Y_i \}$
are mean-zero and sub-exponential with sub-exponential  norm uniformly bounded by $K_3 = 2D_3C_\z$.
By Lemma <ref> and the union bound inequality, we have that for any $t > 0$
(	∇_N+n^( _0; )_∞≥t ) ≤p exp{ -Cmin(t^2/K_3^2 ,t/K_3)(N+ n)},
where $C$ is a universal constant.
Let $t = C_{\bb, \infty}\sqrt{(N\submT + n\submT)^{-1}\log~p} \leq K_3$, then

\begin{align*}
&\pr \left\{	\|\nabla \mL_{N\submT +n\submT}^{\bb}( \bb_0; \bthetabar)\|_\infty \geq C_{\bb, \infty}\sqrt{(N\submT + n\submT)^{-1}\log~p} \right\}\\
\leq &
p\exp\left\{ -C\min \left(\frac{ C_{\bb, \infty}^2 (N\submT + n\submT)^{-1}\log~p}{K_3^2} ,\frac{C_{\bb, \infty}\sqrt{(N\submT + n\submT)^{-1}\log~p} }{K_3}\right)(N\submT + n\submT)\right\} \\
\leq & p\exp\left\{ -\frac{ CC_{\bb, \infty}^2 }{K_3^2} \log~p \right\} \\
\leq & p^{-\epsilon_3},
\end{align*}

where $\epsilon_3 = CC_{\bb, \infty}^2/K_3^2- 1$.

In the following derivations,  we choose $C_{\bb, \infty} \leq \lambda_\bb L_\bb \sqrt{(N\submT + n\submT) / \log~p}/4$.

Proof of Theorem <ref>:
By (<ref>), we have
∇_N+n^(; ) = 1/N+ n ∑_i∈ℐ _i {g(_i)- g(_i) } +
1/N+ n ∑_i∈ℐ_i { g(_i) - Y_i }.
Let $\bnu = \wh \bb\supSTRIFLE - \bb_0$.
Denote by $\Omega_7$ the event that

\begin{align*}
\langle \nabla\mL_{N\submT + n\submT}^{\bb}(\wh\bb\supSTRIFLE;\bthetabar) -  \nabla\mL_{N\submT + n\submT}^\bb(\bb_0;\bthetabar), \bnu \rangle  \geq
\begin{cases}
\alpha_6 \|\bnu\|_2^2  - \tau_6\sqrt{ \frac{\log~p}{N\submT+ n\submT}}\|\bnu\|_1^2, & \forall \|\bnu\|_2 \leq  1,  \\
\alpha_7 \|\bnu\|_2  - \tau_7\sqrt{ \frac{\log~p}{N\submT+n\submT}}\|\bnu\|_1, & \forall \|\bnu\|_2 > 1,
\end{cases}
\end{align*}

where $\alpha_6$ and $\alpha_7$ are positive constants, and $\tau_6$ and $\tau_7$ are nonnegative constants. They depend only on $\bthetabar$, $g(\cdot)$, constants $C_{\z}$, and $\kappa\submT$.
Under (i) (ii) (iii) and (iv) in Assumption <ref>,
By Corollary 2 in <cit.> and Proposition 2 in <cit.>, we have $\pr(\Omega_7) \geq 1 - c_7\exp\{- c_8 N\submT\}$, where $c_7$ and $c_8$ are two constants.

Note that

\begin{align*}
&\nabla\mL_{N\submT + n\submT}^{\bb}(\wh \bb\supSTRIFLE;\bthetahat\supSTRIFLE) - \nabla\mL_{N\submT + n\submT}^{\bb}(\bb_0;\bthetahat\supSTRIFLE)\\
=& \nabla\mL_{N\submT + n\submT}^{\bb}(\wh \bb\supSTRIFLE;\bthetabar) - \nabla\mL_{N\submT + n\submT}^{\bb}(\bb_0;\bthetabar)\\
=&  \frac{1}{N\submT+n\submT} \sum_{i\in \mathcal{I}\submTU \cup  \mathcal{I}\submTL} \X_i \{g(\X_i\trans \wh \bb\supSTRIFLE) -g(\bb_0\trans \X_i)\}.
\end{align*}

Then we have

\begin{align}
\label{eq:RSC3}
\langle \nabla\mL_{N\submT + n\submT}^{\bb}(\wh\bb\supSTRIFLE;\bthetahat\supSTRIFLE) -  \nabla\mL_{N\submT + n\submT}^\bb(\bb_0;\bthetahat\supSTRIFLE), \bnu \rangle  \geq
\begin{cases}
\alpha_6 \|\bnu\|_2  - \tau_6\sqrt{ \frac{\log~p}{N\submT+n\submT}}\|\bnu\|_1, & \forall \|\bnu\|_2 \leq  1 \\
\alpha_7 \|\bnu\|_2  - \tau_7 \sqrt{ \frac{\log~p}{N\submT+n\submT}}\|\bnu\|_1, & \forall \|\bnu\|_2 > 1
\end{cases}
\end{align}

holds with high probability.

First of all, we show that by choosing an appropriate $\lambda_\bb$,
we have $\|\bnu\|_2\leq 1$.
In $\Omega_7$, follow the proof of Theorem 1 in <cit.>,
if $\|\bnu\|_2 > 1$,
we have
⟨∇_N+ n^(;) -  ∇_N+ n^(_0;), ⟩≥α_7 _2 - τ_7 √(log  p/N+ n)_1.
Since $\nabla \mL_{N\submT +n\submT}^\bb(\wh\bb\supSTRIFLE;\bthetahat\supSTRIFLE) + \nabla p_{\lambda_\bb}(\wh\bb\supSTRIFLE) = \0$, then

⟨-∇p_λ_() -  ∇_N+ n^(_0; ), ⟩≥α_7 _2 -  τ_7 √(log p/ N+ n)_1.
By Hölder's inequality and triangle inequality, we have

\begin{align*}
&\langle  -\nabla p_{\lambda_\bb}(\wh \bb\supSTRIFLE)  - \nabla\mL_{N\submT + n\submT}^\bb(\bb_0;\bthetahat\supSTRIFLE),\bnu  \rangle\\
\leq& \left\{ \|\nabla p_{\lambda_\bb}(\wh \bb\supSTRIFLE)\|_\infty + \|\nabla \mL_{N\submT+n\submT}^{\bb}( \bb_0;\bthetabar)\|_\infty \right\}\|\bnu\|_1 + |\langle \nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetahat\supSTRIFLE) -
\nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetabar), \bnu \rangle |.
\end{align*}

By Lemma 4 in <cit.>, we know $ \|\nabla p_{\lambda_\bb}(\wh \bb\supSTRIFLE)\|_\infty \leq \lambda_\bb L_\bb$.
By Lemma <ref>, we know that
∇_N^( _0;)_∞≤λ_L_/2,
with high probability.
For the upper bound of $|\langle \nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetahat\supSTRIFLE) -
\nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetabar), \bnu \rangle |$,
under (ii) in Assumption <ref>, by Cauchy-Schwarz inequality, and
Proposition 1 in <cit.>,

we have

\begin{align}
\label{eq:diff_m}
&|\langle \nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetahat\supSTRIFLE) -
\nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetabar), \bnu \rangle | \n\\
=& |\frac{1}{N\submT} \sum_{i\in \mathcal{I}\submTU}  \left[g(\bthetabar\trans \Z_i) -g\{ \Z_i\trans \bthetahat\supSTRIFLE\} \right] \X_i\trans \bnu| \n\\
\leq & C_g \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}\submTU} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]^{1/2}\left( \frac{1}{N\submT} \sum_{i\in \mathcal{I}\submTU}  \bnu\trans \X_i\X_i\trans\bnu \right)^{1/2} \n\\
\leq& \frac{C_g }{\sqrt{\kappa\submT}} \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}\submTU} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]^{1/2}
\left( \frac{3}{2}\|\bnu\|_2^2  +
\frac{N\submT+ n\submT}{N\submT}
\frac{ \log~p}{N\submT + n\submT}\|\bnu\|_1^2\right)^{1/2}\n
\\
\leq&
\frac{C_g }{\sqrt{\kappa\submT}}
\left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]^{1/2}\left( \sqrt{\frac{3}{2}}\|\bnu\|_2 + \sqrt{\frac{2\log~p}{N\submT + n\submT}}\|\bnu\|_1  \right).
\end{align}

The last inequality is due to $N\submT \geq n\submT$.
Since $N\submT^{-1} \log~ p \precsim n\submT^{-1} \log~ d$, then $\|\bthetahat\supSTRIFLE- \bthetabar\|_2^2 \precsim 1$ and $N\submT^{-1}\log~p\|\bthetahat\supSTRIFLE - \bthetabar\|_1^2 \precsim 1$.
When $c$ in (i) in Assumption <ref> is large enough, we have
C_g√(3/2κ) {3/2κ- _2^2 + 1/κlog p/N - _1^2}^1/2 ≤α_7/2,
C_g√(2/κ) [1/N ∑_i∈ℐ_,U {- }_i_i{-  }]^1/2 ≤α_7/2.
Note that  $\|\bb_0\|_1\leq R_\bb$ and $\|\wh \bb\supSTRIFLE\|_1\leq R_\bb$ by assumption, then $\|\bnu\|_1 \leq 2R_\bb$.
Come back to (<ref>), under (ii) and (iii) in Assumption <ref>, we have
_2 ≤4R_/α_7{
3/2λ_L_+ (τ_7 + α_7/2) √(log p/N+ n)  } ≤1/2 + 1/2  = 1.
Hence, we have

⟨∇_N+ n^(;
) -  ∇_N+ n^(_0;), ⟩≥α_6 _2^2 - τ_6log  p/N+ n_1^2.
Similar to (<ref>), by Assumption <ref>, we have
⟨∇p_λ_(),  -⟩≤p_λ_(_0) - p_λ_() + μ_/2_2^2.
By (<ref>), we have

\begin{align*}
\label{eq:bb3}
\left(\alpha_6 - \frac{\mu_\bb}{2}\right)\|\bnu\|_2^2  \leq & p_{\lambda_\bb}(\bb_0) - p_{\lambda_\bb}(\wh\bb\supSTRIFLE)-
\langle \nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetabar), \bnu\rangle\\
& -  \langle \nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetahat\supSTRIFLE) -\nabla \mL_{N\submT +n\submT}^{\bb}(\bb_0;\bthetabar), \bnu \rangle
+ \tau_6 \frac{\log~p}{N\submT + n\submT}\|\bnu\|_1^2.
\end{align*}

Using  (<ref>) and the fact that $|ab| \leq ca^2/2 + b^2/(2c)$ for every $c>0$,
we further obtain

\begin{align*}
&\left(\frac{3\alpha_6}{4} -\frac{\mu_\bb}{2} \right)\|\bnu \|_2^2\\
\leq&
p_{\lambda_\bb}(\bb_0) - p_{\lambda_\bb}(\wh\bb\supSTRIFLE)
+ \frac{3C_g^2}{2\kappa\submT\alpha_6}  \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE - \bthetabar\}\right]
+\left(\| \nabla \mL_N^{\bb}(\bb_0;\bthetabar)\|_\infty \right. \\
& \left.+ \frac{C_g}{\sqrt{\kappa\submT}}\sqrt{\frac{2\log~p}{N\submT + n\submT}} \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]^{1/2}
+ \tau_6 \frac{\log~p}{N\submT + n\submT}\|\bnu \|_1\right)\|\bnu\|_1.
\end{align*}

By Lemma <ref>, we have
Since $\|\bnu\|_1 \leq 2R_\bb$, by (ii) and (iii) in Assumption <ref>, we also have

τ_6 log p/N+ n_1 ≤2 R_τ_6 log p/N+ n

≤α_7/4√(log p/N+ n) ≤λ_L_/8.
Since $N\submT^{-1} \log~ p \precsim n\submT^{-1} \log~ d$, then $\|\bthetahat\supSTRIFLE - \bthetabar\|_2^2 \precsim 1$ and $N\submT^{-1}\log~p\|\bthetahat\supSTRIFLE - \bthetabar\|_1^2 \precsim 1$.
When $c$ in (i) in Assumption <ref> is large enough, we have
C_g/√(κ)√(2log p/N+ n){3/2κ- _2^2 + 1/κlog p/N - _1^2}^1/2 ≤λ_L_/8,
C_g/√(κ)√(2log p/N+ n) [1/N ∑_i∈ℐ_,U {- }_i_i{- }]^1/2≤λ_L_/8.
Then we obtain

\begin{align*}
\left(\frac{3\alpha_6}{4} -\frac{\mu_\bb}{2} \right)\|\bnu \|_2^2
\leq &
p_{\lambda_\bb}(\bb_0) - p_{\lambda_\bb}(\wh\bb\supSTRIFLE)\\
&+ \frac{3C_g^2}{2\kappa\submT\alpha_6} \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]
+ \frac{1}{2}\lambda_\bb L_\bb \|\bnu\|_1.
\end{align*}

Similar to (<ref>), we know

λ_L__1 ≤p_λ_() + μ_/2 _2^2.
We then have

\begin{align*}
\left(\frac{3\alpha_6}{4} -\frac{3\mu_\bb}{4} \right)\|\bnu \|_2^2
p_{\lambda_\bb}(\bb_0) - p_{\lambda_\bb}(\wh\bb) + \frac{1}{2}p_{\lambda_\bb}(\bnu)
+ \frac{3C_g^2}{2\kappa\submT\alpha_6} \\
&\times \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right].
\end{align*}

If $2p_{\lambda_\bb}(\bb_0) \leq3C_g^2\left[N\submT^{-1} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE- \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE - \bthetabar\}\right]/(\kappa\submT\alpha_6) $,
since $p_{\lambda_\bb}(\bnu )- p_{\lambda_\bb}(\wh \bb\supSTRIFLE)  \leq  p_{\lambda_\bb}(\bb_0) $,
we have

\begin{align}
0\leq& \left(\frac{3\alpha_6}{4} -\frac{3\mu_\bb}{4} \right)\|\bnu \|_2^2\n \\
\leq &
2p_{\lambda_\bb}(\bb_0) - \frac{1}{2}p_{\lambda_\bb}(\bnu)
+ \frac{3C_g^2}{2\kappa\submT\alpha_6}  \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]\n \\
\leq&
\frac{9C_g^2}{2\kappa\submT\alpha_6}  \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE- \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right] -\frac{1}{2}p_{\lambda_\bb}(\bnu).
\end{align}

Hence, we have
_2^2≤6C_g^2/κα_6 (α_6 - μ_) [1/N ∑_i∈ℐ_,U {- }_i_i{- }],
p_λ_() ≤9C_g^2/κα_6  [1/N ∑_i∈ℐ_,U {- }_i_i{- }].
By (<ref>), we further have

\begin{align*}
\|\bnu\|_1 &\leq \frac{9C_g^2}{\kappa\submT\alpha_6\lambda_\bb L_\bb}  \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE - \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]+ \frac{\mu_\bb}{2\lambda_\bb L_\bb}\|\bnu\|_2^2\\
& \leq \left( 3 + \frac{\mu_\bb}{\alpha_6 - \mu_\bb}\right)\frac{3C_g^2}{\kappa\submT\alpha_6 \lambda_\bb L_\bb} \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE- \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right].
\end{align*}

If $2p_{\lambda_\bb}(\bb_0) \geq3C_g^2\left[N\submT^{-1} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE- \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE - \bthetabar\}\right]/(\kappa\submT\alpha_6) $, since $p_{\lambda_\bb}(\bnu )  \leq  p_{\lambda_\bb}(\bb_0) +  p_{\lambda_\bb}(\wh \bb\supSTRIFLE)$, then

\begin{align}
\label{eq:bb4}
\left(\frac{3\alpha_6}{4} -\frac{3\mu_\bb}{4} \right)\|\bnu \|_2^2
\frac{3}{2}p_{\lambda_\bb}(\bb_0) -\frac{1}{2}p_{\lambda_\bb}(\wh \bb\supSTRIFLE)
+ \frac{3C_g^2}{2\kappa\submT\alpha_6}  \left[\frac{1}{N\submT} \sum_{i\in \mathcal{I}_{\mT,U}} \{\bthetahat\supSTRIFLE- \bthetabar\}\trans \Z_i\Z_i\trans\{\bthetahat\supSTRIFLE- \bthetabar\}\right]\n\\
& \leq \frac{5}{2}p_{\lambda_\bb}(\bb_0) -\frac{1}{2}p_{\lambda_\bb}(\wh \bb\supSTRIFLE).
\end{align}

By Lemma 5 in <cit.>, we have

5/2p_λ_(_0) -1/2p_λ_() ≤5/2λ_L_ν__1 -1/2 λ_L_ ν_^c_1,
where $\mA$ denotes the index set of the $s_\bb$ largest elements of $\wh \bb\supSTRIFLE - \bb_0$ in magnitude.
Then we obtain the cone condition

\[
5\|\nu_\mA\|_1 \geq \| \nu_{\mA^c}\|_1.
\]

By (<ref>), (<ref>)  and $\|\bnu_\mA\|_1 \leq \sqrt{s_\bb} \|\bnu_\mA\|_2$, we have
_2≤10λ_L_√(s_)/3(α_6 - μ_).
Further, we have
_1 ≤__1  + _^c_1  ≤6__1 ≤6 s__2 ≤20 λ_L_√(s_)/α_6 - μ_.
In conclusion, we have
_1  ≤20 λ_L_√(s_)/α_6 - μ_ +( 3 + μ_/α_6 - μ_)3C_g^2/κα_6 λ_L_ [1/N ∑_i∈ℐ_,U {- }_i_i{- }],
_2≤10λ_L_√(s_)/3(α_6 - μ_) +
√(6C_g^2/κα_6 (α_6 - μ_) [1/N ∑_i∈ℐ_,U {- }_i_i{- }]).
By Proposition 1 in <cit.>, we  have

1/N ∑_i∈ℐ_,U {- }_i_i{- }≤3/2κ- _2^2 + 1/κlog p/N  - _1^2,
with high probability.
Hence, we have
_1  ≤20 λ_L_√(s_)/α_6 - μ_ +( 3 + μ_/α_6 - μ_)3C_g^2/κ^2α_6 λ_L_ {3/2- _2^2 + log p/N - _1^2},
_2≤10λ_L_√(s_)/3(α_6 - μ_)  +
√(6C_g^2/κ^2 α_6 (α_6 - μ_)
{3/2- _2^2 + log p/N - _1^2})
with high probability.

Assume that

\[\frac{D_1 C_w}{\kappa_\f}  \|\wh \bz - \bz_0\|_2^2 + C_{\epsilon} n\submS^{ \epsilon-1/2} \precsim \lambda_\btheta^2 s_\btheta.
\]

By Theorem <ref>,

\[
\|\bdel_0\|_1 \precsim \sqrt{\frac{n\submT}{n\submT + n\submS}}\sqrt{\frac{\log~ d}{n\submS +n\submT}}s_\btheta,
\]

we have

\[
\|\bthetahat\supmeta- \bthetabar \|_2^2 \precsim  \frac{\log~ d}{n\submS +n\submT}s_\btheta,
\]

\[
\frac{\log~p }{N\submT}\|\bthetahat\supmeta- \bthetabar\|_1^2 \precsim \frac{\log~p }{N\submT} \frac{\log~ d}{n\submS +n\submT}s_\btheta ^2,
\]

\[
\frac{\log~p }{N\submT} \frac{\log~ d}{n\submS +n\submT}s_\btheta ^2 \precsim  \frac{\log~ d}{n\submS +n\submT}s_\btheta
\]

under the assumption that $s_\btheta \precsim n\submT/\log ~d$;

\[
\sqrt{\frac{n\submT}{n\submT + n\submS}}\sqrt{\frac{\log~ d}{n\submS +n\submT}}s_\btheta  \precsim  \|\bdel_0\|_1 \precsim \sqrt{\frac{\log~ d}{n\submS +n\submT}}s_\btheta,
\]

we have

\[
\|\bthetahat\supmeta- \bthetabar \|_2^2 \precsim   \sqrt{\frac{\log~ d}{n\submT}}\|\bdel_0\|_1,
\]

\[