$\displaystyle\frac{\lambda_{n}^{2}}{n^{2}}\gg\frac{\lambda_{n}}{n\sqrt{n}}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
\frac{\lambda_{n}^{2}}{n^{2}}\gg\frac{\lambda_{n}\eta_{n}}{n^{2}},$

that is,

(223) $\displaystyle\frac{\lambda_{n}}{\sqrt{n}}\to\infty,\leavevmode\nobreak\
\frac{\lambda_{n}}{n}\to 0,\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
\frac{\lambda_{n}}{\eta_{n}}\to\infty.$

Let
$V_{n}(u):=(n^{2}/\lambda_{n}^{2})(Z_{n}^{\\#}(\beta)-Z_{n}^{\\#}(\beta^{*}))$,
then we have

(224) $\displaystyle V_{n}(u)=$ $\displaystyle
u^{\top}\left(\frac{1}{n}X^{\top}X\right)u-\frac{2\sqrt{n}}{\lambda_{n}}u^{\top}\left(\frac{1}{\sqrt{n}}X^{\top}\varepsilon\right)$
$\displaystyle+$
$\displaystyle\sum_{j}\left(\frac{\left|u_{j}+\frac{n}{\lambda_{n}}\beta^{*}_{j}\right|-\left|\frac{n}{\lambda_{n}}\beta^{*}_{j}\right|}{\left|\beta^{*}_{j}+\frac{z_{j}}{\sqrt{m}}\right|^{\gamma_{1}}}I(\beta^{*}_{j}\neq
0)+\frac{\sqrt{m^{\gamma_{1}}}|u_{j}|}{\left|z_{j}\right|^{\gamma_{1}}}I(\beta^{*}_{j}=0)\right)$
$\displaystyle+$
$\displaystyle\frac{\eta_{n}}{\lambda_{n}}\sum_{j}\left(\left|\beta^{*}_{j}+\frac{z_{j}}{\sqrt{m}}\right|^{\gamma_{2}}I(\beta^{*}_{j}\neq
0)+\frac{\left|z_{j}\right|^{\gamma_{2}}}{\sqrt{m^{\gamma_{2}}}}I(\beta^{*}_{j}=0)\right)\left(\left|u_{j}-\frac{n}{\sqrt{m}\lambda_{n}}z_{j}\right|-\left|\frac{n}{\sqrt{m}\lambda_{n}}z_{j}\right|\right).$

Let $V(u):=\lim_{n\to\infty}V_{n}(u)$, then we have

(225) $\displaystyle
V(u)=\begin{dcases}u^{\top}Cu+\sum_{j}\frac{u_{j}\operatorname{sgn}(\beta^{*}_{j})}{|\beta^{*}_{j}|^{\gamma_{1}}}I(\beta^{*}_{j}\neq
0)&\text{for}\leavevmode\nobreak\ u_{S^{c}}=0,\\\
\infty&\text{otherwise}.\end{dcases}$

Therefore, we obtain

(226)
$\displaystyle\frac{n}{\lambda_{n}}({\hat{\beta}}_{n}^{\\#}-\beta^{*})=\operatorname*{argmin}_{u}V_{n}(u)\overset{d}{\to}\operatorname*{argmin}_{u}V(u)=\operatorname*{argmin}_{u\in\mathcal{U}}\left\\{u^{\top}Cu+\sum_{j}\frac{\operatorname{sgn}(\beta^{*}_{j})}{|\beta^{*}_{j}|^{\gamma_{1}}}u_{j}\right\\},$
(227) $\displaystyle\mathcal{U}:=\left\\{u\leavevmode\nobreak\
\middle|\leavevmode\nobreak\ u_{S^{c}}=0\right\\}.$

####  B.3.2 Proof of Theorem 4.3

Suppose that ${\tilde{\beta}}$ is a $\sqrt{m}$-consistent estimator. Suppose
that ${\hat{\beta}}$ is $l$-consistent and let
$\hat{u}:=l({\hat{\beta}}-\beta^{*})$ where $l=l(n,m,\lambda_{n})$ is a
certain function as defined later. Then, we have

(228) $\displaystyle\forall j\in S,\leavevmode\nobreak\
P(j\in\operatorname{supp}({\hat{\beta}}))\to 1.$

Consider the event $\beta^{*}_{j}\in S^{c}$ and ${\hat{\beta}}_{j}\neq 0$. By
the KKT conditions, for $\beta^{*}_{j}=0$ and
${\hat{\beta}}_{j}\neq{\tilde{\beta}}_{j}$,

(229) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\frac{\sqrt{n}}{l}\hat{u}-\frac{2}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\sqrt{m^{\gamma_{1}}}\lambda_{n}}{\sqrt{n}}\frac{\operatorname{sgn}({\hat{\beta}}_{j})}{|\sqrt{m}{\tilde{\beta}}_{j}|^{\gamma_{1}}}+\frac{\eta_{n}}{\sqrt{nm^{\gamma_{2}}}}\left|\sqrt{m}{\tilde{\beta}}_{j}\right|^{\gamma_{2}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0,$

and for $\beta^{*}_{j}=0$ and ${\hat{\beta}}_{j}={\tilde{\beta}}_{j}$,

(230)
$\displaystyle\left|2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\frac{\sqrt{n}}{l}\hat{u}-\frac{2}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\sqrt{m^{\gamma_{1}}}\lambda_{n}}{\sqrt{n}}\frac{\operatorname{sgn}({\hat{\beta}}_{j})}{|\sqrt{m}{\tilde{\beta}}_{j}|^{\gamma_{1}}}\right|\leq\frac{\eta_{n}}{\sqrt{nm^{\gamma_{2}}}}\left|\sqrt{m}{\tilde{\beta}}_{j}\right|^{\gamma_{2}}.$

Suppose that ${\hat{\beta}}$ is $l$-consistent and that
$\sqrt{m^{\gamma_{1}}}\lambda_{n}/\sqrt{n}\gg 1$,
$\sqrt{m^{\gamma_{1}}}\lambda_{n}/\sqrt{n}\gg\sqrt{n}/l$, and
$\sqrt{m^{\gamma_{1}}}\lambda_{n}/\sqrt{n}\gg\eta_{n}/\sqrt{nm^{\gamma_{2}}}$.
For $l=\sqrt{m},\sqrt{n},n/\lambda_{n}$, these conditions reduce to the
conditions where ${\hat{\beta}}$ is $l$-consistent,
$\sqrt{m^{\gamma_{1}}}\lambda_{n}/\sqrt{n}\to\infty$, and
$\sqrt{m^{\gamma_{1}+\gamma_{2}}}\lambda_{n}/\eta_{n}\to\infty$. Then, in both
${\hat{\beta}}_{j}\neq{\tilde{\beta}}_{j}$ and
${\hat{\beta}}_{j}={\tilde{\beta}}_{j}$ cases, the term including
$\sqrt{m^{\gamma_{1}}/n}\lambda_{n}$ diverges to infinity faster compared to
the rest terms. Therefore, we have

(231) $\displaystyle\forall j\in S^{c},\leavevmode\nobreak\
\lim_{n\to\infty}P(j\in\operatorname{supp}({\hat{\beta}}))=0.$

This concludes

(232) $\displaystyle\lim_{n\to\infty}P(\hat{S}_{n}=S)=1.$

####  B.3.3 Proof of Theorem 4.4

Suppose that ${\tilde{\beta}}$ is a $\sqrt{m}$-consistent estimator. Consider
the event where $\beta^{*}_{j}\neq 0$ and
${\hat{\beta}}_{j}\neq{\tilde{\beta}}_{j}$. Suppose that ${\hat{\beta}}$ is
$l$-consistent and let $\hat{u}:=l({\hat{\beta}}-\beta^{*})$ where
$l=l(n,m,\lambda_{n})$ is a certain function as defined later. By the KKT
conditions, for $\beta^{*}_{j}\neq 0$ and ${\hat{\beta}}_{j}\neq 0$,

(233) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\frac{\sqrt{n}}{l}\hat{u}-\frac{2}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}}{\sqrt{n}}\frac{\operatorname{sgn}({\hat{\beta}}_{j})}{|{\tilde{\beta}}_{j}|^{\gamma_{1}}}+\frac{\eta_{n}}{\sqrt{n}}\left|{\tilde{\beta}}_{j}\right|^{\gamma_{2}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0,$

and for $\beta^{*}_{j}\neq 0$ and ${\hat{\beta}}_{j}=0$,

(234)
$\displaystyle\left|2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\frac{\sqrt{n}}{l}\hat{u}-\frac{2}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\eta_{n}}{\sqrt{n}}\left|{\tilde{\beta}}_{j}\right|^{\gamma_{2}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})\right|\leq\frac{\lambda_{n}}{\sqrt{n}}\frac{1}{|{\tilde{\beta}}_{j}|^{\gamma_{1}}}.$

Suppose that ${\hat{\beta}}$ is $l$-consistent and that $\eta_{n}/\sqrt{n}\gg
1$, $\eta_{n}/\sqrt{n}\gg\sqrt{n}/l$, and
$\eta_{n}/\sqrt{n}\gg\lambda_{n}/\sqrt{n}$. For $l=\sqrt{m},\sqrt{n}$, these
conditions reduce to the conditions where ${\hat{\beta}}$ is $l$-consistent,
$\eta_{n}/\sqrt{n}\to\infty$, and $\eta_{n}/\lambda_{n}\to\infty$. Then, in
both ${\hat{\beta}}_{j}\neq 0$ and ${\hat{\beta}}_{j}=0$ cases, the term
including $\eta_{n}/\sqrt{n}$ diverges to infinity faster compared to the rest
terms. Therefore, we have

(235) $\displaystyle\forall j\in S,\leavevmode\nobreak\
\lim_{n\to\infty}P({\hat{\beta}}_{j}\neq{\tilde{\beta}}_{j})=0.$

This concludes

(236) $\displaystyle\forall j\in S,\leavevmode\nobreak\
\lim_{n\to\infty}P({\hat{\beta}}_{j}={\tilde{\beta}}_{j})=1.$

###  B.4 Proofs for Transfer Lasso with Deterministic Source Parameters

####  B.4.1 Proof of Theorem A.3

For ${\tilde{\beta}}_{j}=0$, we consider the event ${\hat{\beta}}_{j}\neq 0$.
Then, we have by KKT conditions

(237) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}+\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})=0$

The first and second terms converge to some truncated Gaussian-mixture
distribution, and the third term diverges to infinity as
$(\lambda_{n}+\eta_{n})/\sqrt{n}\rightarrow\infty$, which is an contradiction.
Hence, we have for ${\tilde{\beta}}_{j}=0$

(238)
$\displaystyle\lim_{n\rightarrow\infty}P\left({\hat{\beta}}_{j}=0\right)=1$

For ${\tilde{\beta}}_{j}\neq 0$, we consider the event ${\hat{\beta}}_{j}\neq
0$ and ${\hat{\beta}}_{j}\neq{\tilde{\beta}}_{j}$. Then, we have by KKT
conditions,

(239) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})+\frac{\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0$

The third and fourth terms diverge to infinity as
$(\lambda_{n}+\eta_{n})/\sqrt{n}\rightarrow\infty$ if
$\operatorname{sgn}({\hat{\beta}}_{j})=\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})$,
which induces an contradiction. Hence, we have for ${\tilde{\beta}}_{j}\neq 0$

(240)
$\displaystyle\lim_{n\rightarrow\infty}P\left(\min\left\\{0,{\tilde{\beta}}_{j}\right\\}\leq{\hat{\beta}}_{j}\leq\max\left\\{0,{\tilde{\beta}}_{j}\right\\}\right)=1$

####  B.4.2 Proof of Theorem A.4

Asymptotic distribution: Let $u:=\sqrt{n}(\beta-\beta^{*})$. We have

(241) $\displaystyle
Z_{n}^{\mathcal{T}}(\beta;{\tilde{\beta}},\lambda_{n},\eta_{n}):$
$\displaystyle=\frac{1}{n}\|y-X\beta\|_{2}^{2}+\frac{\lambda_{n}}{n}\sum_{j}|\beta_{j}|+\frac{\eta_{n}}{n}|\beta_{j}-{\tilde{\beta}}_{j}|$
(242)
$\displaystyle=\frac{1}{n}\left\|\varepsilon-\frac{1}{\sqrt{n}}Xu\right\|_{2}^{2}+\frac{\lambda_{n}}{n}\sum_{j}\left|\beta^{*}_{j}+\frac{u_{j}}{\sqrt{n}}\right|+\frac{\eta_{n}}{n}\sum_{j}\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}+\frac{u_{j}}{\sqrt{n}}\right|$

and

(243) $\displaystyle
Z_{n}^{\mathcal{T}}(\beta;{\tilde{\beta}},\lambda_{n},\eta_{n})-Z_{n}^{\mathcal{T}}(\beta^{*};{\tilde{\beta}},\lambda_{n},\eta_{n})$
(244) $\displaystyle=$
$\displaystyle\left\|\varepsilon-\frac{1}{\sqrt{n}}Xu\right\|_{2}^{2}-\|\varepsilon\|_{2}^{2}$
(245)
$\displaystyle+\lambda_{n}\sum_{j}\left(\left|\beta^{*}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}\right|\right)+\eta_{n}\sum_{j}\left(\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}\right|\right)$
(246) $\displaystyle=$ $\displaystyle
u^{\top}\left(\frac{1}{n}X^{\top}X\right)u-2u^{\top}\left(\frac{1}{\sqrt{n}}X^{\top}\varepsilon\right)$
(247)
$\displaystyle+\lambda_{n}\sum_{j}\left(\left|\beta^{*}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}\right|\right)+\eta_{n}\sum_{j}\left(\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}\right|\right)$

The first and second terms are the same as (101) and (102). We consider the
third and fourth terms.

(248)
$\displaystyle\lambda_{n}\left(\left|\beta^{*}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}\right|\right)=\begin{cases}\frac{\lambda_{n}}{\sqrt{n}}|u_{j}|&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}=0\\\
\frac{\lambda_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j})&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}\neq 0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
\beta^{*}_{j}(\beta^{*}_{j}+\frac{u_{j}}{\sqrt{n}})\geq 0\\\
-\frac{\lambda_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j})-2\lambda_{n}|\beta^{*}_{j}|&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}\neq 0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
\beta^{*}_{j}(\beta^{*}_{j}+\frac{u_{j}}{\sqrt{n}})<0\end{cases}$ (249)
$\displaystyle\eta_{n}\left(\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}\right|\right)$
(250) $\displaystyle=$
$\displaystyle\begin{cases}\frac{\eta_{n}}{\sqrt{n}}|u_{j}|&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}={\tilde{\beta}}_{j}\\\
\frac{\eta_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}\neq{\tilde{\beta}}_{j}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
(\beta^{*}_{j}-{\tilde{\beta}}_{j})(\beta^{*}_{j}-{\tilde{\beta}}_{j}+\frac{u_{j}}{\sqrt{n}})\geq
0\\\
-\frac{\eta_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})-2\eta_{n}|\beta^{*}_{j}-{\tilde{\beta}}_{j}|&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}\neq{\tilde{\beta}}_{j}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
(\beta^{*}_{j}-{\tilde{\beta}}_{j})(\beta^{*}_{j}-{\tilde{\beta}}_{j}+\frac{u_{j}}{\sqrt{n}})<0\end{cases}$

For large $n$ such that ($n>u_{j}^{2}/\beta_{j}^{*2}$ for $\forall j\in S$)
and ($n>u_{j}^{2}/(\beta^{*}_{j}-{\tilde{\beta}}_{j})^{2}$ for $\forall j$
such that $\beta^{*}_{j}\neq{\tilde{\beta}}_{j}$), we have

(251) $\displaystyle\mathcal{R}_{j}:=$
$\displaystyle\lambda_{n}\left(\left|\beta^{*}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}\right|\right)+\eta_{n}\left(\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}+\frac{u_{j}}{\sqrt{n}}\right|-\left|\beta^{*}_{j}-{\tilde{\beta}}_{j}\right|\right)$
(252) $\displaystyle=$
$\displaystyle\begin{cases}\frac{\lambda_{n}+\eta_{n}}{\sqrt{n}}|u_{j}|&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}=0,{\tilde{\beta}}_{j}=0\\\
\frac{\lambda_{n}}{\sqrt{n}}|u_{j}|-\frac{\eta_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}({\tilde{\beta}}_{j})&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}=0,{\tilde{\beta}}_{j}\neq 0\\\
\frac{\lambda_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j})+\frac{\eta_{n}}{\sqrt{n}}|u_{j}|&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}\neq 0,{\tilde{\beta}}_{j}=\beta^{*}_{j}\\\
\frac{\lambda_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j})+\frac{\eta_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})&\text{if}\leavevmode\nobreak\
\beta^{*}_{j}\neq 0,{\tilde{\beta}}_{j}\neq\beta^{*}_{j}\end{cases}$

Assume $(\lambda_{n}+\eta_{n})/\sqrt{n}\rightarrow\delta_{1}$ and
$(\lambda_{n}-\eta_{n})/\sqrt{n}\rightarrow\delta_{2}$. For $\beta^{*}_{j}=0$
and ${\tilde{\beta}}_{j}=0$,

(253)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
u_{j}=0\\\ \delta_{1}|u_{j}|&\text{if}\leavevmode\nobreak\ u_{j}\neq
0\end{cases}$

For $\beta^{*}_{j}=0,{\tilde{\beta}}_{j}\neq 0$,

(254)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
u_{j}=0\\\ \delta_{2}&\text{if}\leavevmode\nobreak\
u_{j}{\tilde{\beta}}_{j}>0\\\ \delta_{1}|u_{j}|&\text{if}\leavevmode\nobreak\
u_{j}{\tilde{\beta}}_{j}<0\end{cases}$

For $\beta^{*}_{j}\neq 0,{\tilde{\beta}}_{j}=\beta^{*}_{j}$,

(255)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
u_{j}=0\\\ \delta_{1}&\text{if}\leavevmode\nobreak\ u_{j}\beta^{*}_{j}>0\\\
-\delta_{2}|u_{j}|&\text{if}\leavevmode\nobreak\
u_{j}\beta^{*}_{j}<0\end{cases}$

For $\beta^{*}_{j}\neq 0,{\tilde{\beta}}_{j}\neq\beta^{*}_{j}$,

(256)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
u_{j}=0\\\ \delta_{2}|u_{j}|&\text{if}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})\neq\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\operatorname{sgn}(\beta^{*}_{j})>0\\\
-\delta_{2}|u_{j}|&\text{if}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})\neq\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\operatorname{sgn}(\beta^{*}_{j})<0\\\
\delta_{1}|u_{j}|&\text{if}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})=\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\operatorname{sgn}(\beta^{*}_{j})>0\\\
-\delta_{1}|u_{j}|&\text{if}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})=\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
u_{j}\operatorname{sgn}(\beta^{*}_{j})<0\end{cases}$

Furthermore, we assume $\delta_{1}=\infty$ and $\delta_{2}=0$. Then, for
$\beta^{*}_{j}=0$ and ${\tilde{\beta}}_{j}=0$,

(257)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
u_{j}=0\\\ \infty&\text{if}\leavevmode\nobreak\ u_{j}\neq 0\end{cases}$

For $\beta^{*}_{j}=0,{\tilde{\beta}}_{j}\neq 0$,

(258)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
u_{j}{\tilde{\beta}}_{j}\geq 0\\\ \infty&\text{if}\leavevmode\nobreak\
u_{j}{\tilde{\beta}}_{j}<0\end{cases}$

For $\beta^{*}_{j}\neq 0,{\tilde{\beta}}_{j}=\beta^{*}_{j}$,

(259)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
u_{j}\beta^{*}_{j}\leq 0\\\ \infty&\text{if}\leavevmode\nobreak\
u_{j}\beta^{*}_{j}>0\end{cases}$

For $\beta^{*}_{j}\neq 0,{\tilde{\beta}}_{j}\neq\beta^{*}_{j}$,

(260)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})\neq\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{or}\leavevmode\nobreak\ u_{j}=0\\\ \infty&\text{if}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})=\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\operatorname{sgn}(\beta^{*}_{j})>0\\\
-\infty&\text{if}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})=\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
u_{j}\operatorname{sgn}(\beta^{*}_{j})<0\end{cases}$

Overall, we have

(261)
$\displaystyle\mathcal{R}_{j}\rightarrow\begin{cases}0&\text{if}\leavevmode\nobreak\
(\beta^{*}_{j}=0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}=0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
u_{j}=0)\\\ &\text{or}\leavevmode\nobreak\
(\beta^{*}_{j}=0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}\neq 0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
u_{j}{\tilde{\beta}}_{j}\geq 0)\\\ &\text{or}\leavevmode\nobreak\
(\beta^{*}_{j}\neq 0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}=\beta^{*}_{j}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\beta^{*}_{j}\leq 0)\\\
&\text{or}\leavevmode\nobreak\ (\beta^{*}_{j}\neq 0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}\neq\beta^{*}_{j}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
(\operatorname{sgn}(\beta^{*}_{j})\neq\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{or}\leavevmode\nobreak\ u_{j}=0))\\\
\infty&\text{if}\leavevmode\nobreak\ (\beta^{*}_{j}=0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ {\tilde{\beta}}_{j}=0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\neq 0)\\\ &\text{or}\leavevmode\nobreak\
(\beta^{*}_{j}=0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}\neq 0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
u_{j}{\tilde{\beta}}_{j}<0)\\\ &\text{or}\leavevmode\nobreak\
(\beta^{*}_{j}\neq 0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}=\beta^{*}_{j}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\beta^{*}_{j}>0)\\\
&\text{or}\leavevmode\nobreak\ (\beta^{*}_{j}\neq 0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}\neq\beta^{*}_{j}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})=\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}\operatorname{sgn}(\beta^{*}_{j})>0)\\\
-\infty&\text{if}\leavevmode\nobreak\ (\beta^{*}_{j}\neq 0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}\neq\beta^{*}_{j}\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
\operatorname{sgn}(\beta^{*}_{j})=\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\
u_{j}\operatorname{sgn}(\beta^{*}_{j})<0)\end{cases}$

If we assume
$\operatorname{sgn}(\beta^{*}_{j})\neq\operatorname{sgn}(\beta^{*}_{j}-{\tilde{\beta}}_{j})$
for $\forall j$ such that $\beta^{*}_{j}\neq 0$ and
${\tilde{\beta}}_{j}\neq\beta^{*}_{j}$, we exclude the case of
$\mathcal{R}\rightarrow-\infty$, and

(262) $\displaystyle V_{n}(u)\rightarrow
V(u):=\begin{cases}-2u^{\top}W+u^{\top}Cu&\text{if}\leavevmode\nobreak\
(u_{j}=0\leavevmode\nobreak\ \text{for}\leavevmode\nobreak\ \forall
j\leavevmode\nobreak\ \text{s.t.}\leavevmode\nobreak\
\beta^{*}_{j}=0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}=0)\\\ &\text{and}\leavevmode\nobreak\
(u_{j}{\tilde{\beta}}_{j}\geq 0\leavevmode\nobreak\
\text{for}\leavevmode\nobreak\ \forall j\leavevmode\nobreak\
\text{s.t.}\leavevmode\nobreak\ \beta^{*}_{j}=0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ {\tilde{\beta}}_{j}\neq 0)\\\
&\text{and}\leavevmode\nobreak\ (u_{j}\beta^{*}_{j}\leq 0\leavevmode\nobreak\
\text{for}\leavevmode\nobreak\ \forall j\leavevmode\nobreak\
\text{s.t.}\leavevmode\nobreak\ \beta^{*}_{j}\neq 0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ {\tilde{\beta}}_{j}=\beta^{*}_{j})\\\
\infty&\text{otherwise}.\end{cases}$

Since $V_{n}(u)$ is convex and $V(u)$ has a unique minimum, in the same manner
of the proof of Adaptive Lasso, we have

(263)
$\displaystyle\sqrt{n}({\hat{\beta}}-\beta^{*})\overset{d}{\rightarrow}\operatorname*{argmin}_{u\in\mathcal{U}}$
$\displaystyle-2u^{\top}W+u^{\top}Cu$ (267) $\displaystyle\mathcal{U}:=$
$\displaystyle\left\\{u\in\mathbb{R}^{p}\left|\begin{array}[]{l}u_{j}=0\leavevmode\nobreak\
\text{for}\leavevmode\nobreak\ \forall j\leavevmode\nobreak\
\text{s.t.}\leavevmode\nobreak\ \beta^{*}_{j}=0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ {\tilde{\beta}}_{j}=0\\\
u_{j}{\tilde{\beta}}_{j}\geq 0\leavevmode\nobreak\
\text{for}\leavevmode\nobreak\ \forall j\leavevmode\nobreak\
\text{s.t.}\leavevmode\nobreak\ \beta^{*}_{j}=0\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ {\tilde{\beta}}_{j}\neq 0\\\
u_{j}\beta^{*}_{j}\leq 0\leavevmode\nobreak\ \text{for}\leavevmode\nobreak\
\forall j\leavevmode\nobreak\ \text{s.t.}\leavevmode\nobreak\
\beta^{*}_{j}\neq 0\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
{\tilde{\beta}}_{j}=\beta^{*}_{j}\end{array}\right.\right\\}$

Next, we show active/varying variable consistency. We consider the cases of
(i) $j\in S\cap T$, (ii) $j\in S\cap T^{c}$, (iii) $j\in S^{c}\cap T$, and
(iv) $j\in S^{c}\cap T^{c}$.

(i) For $j\in S\cap T$, we have ${\tilde{\beta}}_{j}\neq 0$ because of
$\operatorname{sgn}({\tilde{\beta}}_{j}-\beta^{*}_{j})=\operatorname{sgn}(\beta^{*}_{j})$.
By KKT condition, we have

(268) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})+\frac{\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0$

and

(269)
$\displaystyle\lim_{n\rightarrow\infty}P\left(\min\left\\{0,{\tilde{\beta}}_{j}\right\\}\leq{\hat{\beta}}_{j}\leq\max\left\\{0,{\tilde{\beta}}_{j}\right\\}\right)=1.$

On the other hand, $\sqrt{n}({\hat{\beta}}_{j}-\beta^{*}_{j})$ converges to
some Gaussian-mixture distribution for $j\in S\cap T$. Hence, we have
${\hat{\beta}}_{j}\rightarrow\beta^{*}_{j}$ and

(270)
$\displaystyle\lim_{n\rightarrow\infty}P\left({\hat{\beta}}_{j}=0\leavevmode\nobreak\
\text{or}\leavevmode\nobreak\ {\hat{\beta}}_{j}={\tilde{\beta}}_{j}\right)=0$

This concludes

(271)
$\displaystyle\lim_{n\rightarrow\infty}P\left(0<{\hat{\beta}}_{j}<{\tilde{\beta}}_{j}\leavevmode\nobreak\
\text{or}\leavevmode\nobreak\
{\tilde{\beta}}_{j}<{\hat{\beta}}_{j}<0\right)=1\leavevmode\nobreak\
\text{for}\leavevmode\nobreak\ \forall j\in(S\cap T)$

(ii) For $j\in S\cap T^{c}$, we have $\beta^{*}_{j}\neq 0$ and
$\beta^{*}_{j}={\tilde{\beta}}_{j}$. By KKT condition, we have

(272) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})+\frac{\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j}-\beta^{*}_{j})=0$

and

(273)
$\displaystyle\lim_{n\rightarrow\infty}P\left(\min\left\\{0,\beta^{*}_{j}\right\\}\leq{\hat{\beta}}_{j}\leq\max\left\\{0,\beta^{*}_{j}\right\\}\right)=1$

On the other hand, $\sqrt{n}({\hat{\beta}}_{j}-\beta^{*}_{j})$ converges to
some mixture distribution of truncated Gaussian distribution truncated at zero
and delta distribution at zero for $j\in S\cap T^{c}$. Hence, we have
${\hat{\beta}}_{j}\rightarrow\beta^{*}_{j}\neq 0$ and

(274)
$\displaystyle\lim_{n\rightarrow\infty}P\left({\hat{\beta}}_{j}=0\right)=0$

This concludes

(275)
$\displaystyle\lim_{n\rightarrow\infty}P\left(0<{\hat{\beta}}_{j}\leq{\tilde{\beta}}_{j}\leavevmode\nobreak\
\text{or}\leavevmode\nobreak\
{\tilde{\beta}}_{j}\leq{\hat{\beta}}_{j}<0\right)=1\leavevmode\nobreak\ $
$\displaystyle\text{for}\leavevmode\nobreak\ \forall j\in(S\cap T^{c})$

(iii) For $j\in S^{c}\cap T$, we have $\beta^{*}_{j}=0$,
$\beta^{*}_{j}\neq{\tilde{\beta}}_{j}$, and ${\tilde{\beta}}_{j}\neq 0$. By
KKT condition, we have

(276) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})+\frac{\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0$

and

(277)
$\displaystyle\lim_{n\rightarrow\infty}P\left(\min\left\\{0,\beta^{*}_{j}\right\\}\leq{\hat{\beta}}_{j}\leq\max\left\\{0,\beta^{*}_{j}\right\\}\right)=1$

On the other hand, $\sqrt{n}{\hat{\beta}}_{j}$ converges to some mixture
distribution of truncated Gaussian distribution truncated at zero and delta
distribution at zero for $j\in S\cap T^{c}$. Hence, we have
${\hat{\beta}}_{j}\rightarrow 0\neq{\tilde{\beta}}_{j}$ and

(278)
$\displaystyle\lim_{n\rightarrow\infty}P\left({\hat{\beta}}_{j}={\tilde{\beta}}_{j}\right)=0$

This concludes

(279)
$\displaystyle\lim_{n\rightarrow\infty}P\left(0\leq{\hat{\beta}}_{j}<{\tilde{\beta}}_{j}\leavevmode\nobreak\
\text{or}\leavevmode\nobreak\ {\tilde{\beta}}_{j}<{\hat{\beta}}_{j}\leq
0\right)=1\leavevmode\nobreak\ $ $\displaystyle\text{for}\leavevmode\nobreak\
\forall j\in(S^{c}\cap T)$

(iv) For $j\in S^{c}\cap T^{c}$, we have ${\tilde{\beta}}_{j}=\beta^{*}_{j}=0$
and

(280) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}+\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})=0$

This implies that

(281)
$\displaystyle\lim_{n\rightarrow\infty}P\left({\hat{\beta}}_{j}=0\right)=1$

####  B.4.3 Proof of Theorem A.5

We assume that ${\tilde{\beta}}_{j}=0$ for $\forall j$ such that
$\beta^{*}_{j}=0$, and ${\tilde{\beta}}_{j}\neq\beta^{*}_{j}$ for $\forall j$
such that $\beta^{*}_{j}\neq 0$. Then, $\mathcal{U}=\\{u\leavevmode\nobreak\
|\leavevmode\nobreak\ u_{S^{c}}=0\\}$, and we have asymptotic normality, i.e.,

(282)
$\displaystyle\sqrt{n}({\hat{\beta}}_{S}-\beta^{*}_{S})\overset{d}{\rightarrow}\mathcal{N}(0,\sigma^{2}C_{SS}^{-1})^{\top},\leavevmode\nobreak\
\sqrt{n}{\hat{\beta}}_{S^{c}}\overset{d}{\rightarrow}0.$

On the other hand, this indicates that

(283) $\displaystyle\forall j\in S,\leavevmode\nobreak\
P(j\in\operatorname{supp}({\hat{\beta}}))\rightarrow 1.$

Now, we consider the event $j\in S^{c}$ and
$j\in\operatorname{supp}({\hat{\beta}})$. By the KKT conditions,

(284) $\displaystyle
2\mathbf{x}_{j}^{\top}(y-X{\hat{\beta}})+\lambda_{n}\operatorname{sgn}({\hat{\beta}}_{j})+\eta_{n}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0$
(285) $\displaystyle\Rightarrow$
$\displaystyle\frac{2}{\sqrt{n}}\mathbf{x}_{j}^{\top}(y-X{\hat{\beta}})+\frac{1}{\sqrt{n}}\lambda_{n}\operatorname{sgn}({\hat{\beta}}_{j})+\frac{\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0$
(286) $\displaystyle\Rightarrow$ $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})+\frac{\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j}-{\tilde{\beta}}_{j})=0$

Since we assume ${\tilde{\beta}}_{j}=0$ for $j\in S^{c}$, we have

(287) $\displaystyle
2\left(\frac{1}{n}\mathbf{x}_{j}^{\top}X\right)\sqrt{n}(\beta^{*}-{\hat{\beta}})+2\frac{1}{\sqrt{n}}\mathbf{x}_{j}^{\top}\varepsilon+\frac{\lambda_{n}+\eta_{n}}{\sqrt{n}}\operatorname{sgn}({\hat{\beta}}_{j})=0$

The first and second terms on the left-hand side converge to some normal
distribution, but the the last term on the left-hand side diverges to infinity
if $(\lambda_{n}+\eta_{n})/\sqrt{n}\rightarrow\infty$. Hence, we have for
$\forall j\in S^{c}$,

(288)
$\displaystyle\lim_{n\rightarrow\infty}P\left(j\in\operatorname{supp}({\hat{\beta}})\right)=0.$

This concludes

(289) $\displaystyle\lim_{n\rightarrow\infty}P(\hat{S}_{n}=S)=1$

On the other hand, (4) is symmetric in terms of $0$ and ${\tilde{\beta}}$. If
we reparameterize parameters as

(290)
$\displaystyle{\beta^{*}}^{\prime}:=\beta^{*}-{\tilde{\beta}},\leavevmode\nobreak\
{\tilde{\beta}}^{\prime}:=-{\tilde{\beta}},\leavevmode\nobreak\
{\hat{\beta}}_{n}^{\prime}:={\hat{\beta}}_{n}-{\tilde{\beta}},\leavevmode\nobreak\
y^{\prime}:=y-X{\tilde{\beta}},$ (291)
$\displaystyle\hat{S}_{n}^{\prime}:=\\{j:{\hat{\beta}}_{j}^{\prime}\neq
0\\},\leavevmode\nobreak\ S^{\prime}:=\\{j:{\beta^{*}_{j}}^{\prime}\neq 0\\},$
(292) $\displaystyle\lambda_{n}^{\prime}:=\eta_{n},\leavevmode\nobreak\
\eta_{n}^{\prime}:=\lambda_{n},$

we have

(293) $\displaystyle{\hat{\beta}}_{n}^{\prime}$
$\displaystyle={\hat{\beta}}_{n}-{\tilde{\beta}}$ (294)
$\displaystyle=\operatorname*{argmin}_{\beta^{\prime}}\frac{1}{n}\left\|y-X(\beta^{\prime}+{\tilde{\beta}})\right\|_{2}^{2}+\frac{\lambda_{n}}{n}\left\|(\beta^{\prime}+{\tilde{\beta}})\right\|_{1}+\frac{\eta_{n}}{n}\left\|(\beta^{\prime}+{\tilde{\beta}})-{\tilde{\beta}}^{\prime}\right\|_{1}$
(295)
$\displaystyle=\operatorname*{argmin}_{\beta^{\prime}}\frac{1}{n}\left\|y^{\prime}-X\beta^{\prime}\right\|_{2}^{2}+\frac{\lambda_{n}^{\prime}}{n}\left\|\beta^{\prime}\right\|_{1}+\frac{\eta_{n}^{\prime}}{n}\left\|\beta^{\prime}-{\tilde{\beta}}^{\prime}\right\|_{1}$

Hence, the property for
$\\{{\hat{\beta}}^{\prime}_{n},{\beta^{*}}^{\prime},\hat{S}_{n}^{\prime},S^{\prime}\\}$
is the same as that for $\\{{\hat{\beta}}_{n},\beta^{*},\hat{S}_{n},S\\}$ in
Theorem A.5. In addition, we have

(296) $\displaystyle\hat{S}_{n}^{\prime}=\\{j:{\hat{\beta}}_{j}^{\prime}\neq
0\\}=\\{j:{\hat{\beta}}_{j}\neq{\tilde{\beta}}_{j}\\}=\hat{T}_{n},\leavevmode\nobreak\
S^{\prime}=\\{j:{\beta^{*}_{j}}^{\prime}\neq
0\\}=\\{j:\beta^{*}_{j}\neq{\tilde{\beta}}_{j}\\}=T.$

This concludes

(297)
$\displaystyle\lim_{n\rightarrow\infty}P(\hat{T}_{n}=T)=\lim_{n\rightarrow\infty}P(\hat{S}_{n}^{\prime}=S^{\prime})=1$

###  B.5 Proofs of Transfer Lasso with Initial Estimator in Boundary Region

####  B.5.1 Proofs of Theorem A.6

Let $u:=\sqrt{n}(\beta-\beta^{*})$,
$V_{n}(u):=n(Z_{n}^{\mathcal{T}}(\beta)-Z_{n}^{\mathcal{T}}(\beta^{*}))$, and
$V(u):=\lim_{n\to\infty}V_{n}(u)$. Consider the case where
$\lambda_{n}/\sqrt{n}\to\infty$, $\lambda_{n}/\eta_{n}\to 1$, and
$(\lambda_{n}-\eta_{n})/\sqrt{n}\to\delta_{0}$. Suppose that $n/m\to 0$. In
the same way as in Case II of Proof B.2.2 (Theorem 3.5), we obtain $V_{n}(u)$
as (132). Because

(298)
$\displaystyle\left(\left|u_{j}+\sqrt{m}\beta^{*}_{j}\right|-\left|\sqrt{m}\beta^{*}_{j}\right|\right)\to
u_{j}\operatorname{sgn}(\beta^{*}_{j})I(\beta^{*}_{j}\neq
0)+|u_{j}|I(\beta^{*}_{j}=0),$ (299)
$\displaystyle\left(\left|u_{j}-\sqrt{\frac{n}{m}}z_{j}\right|-\left|\sqrt{\frac{n}{m}}z_{j}\right|\right)\to|u_{j}|,$

and

(300)
$\displaystyle\frac{\lambda_{n}}{\sqrt{n}}\left(u_{j}\operatorname{sgn}(\beta^{*}_{j})I(\beta^{*}_{j}\neq
0)+|u_{j}|I(\beta^{*}_{j}=0)\right)+\frac{\eta_{n}}{\sqrt{n}}|u_{j}|$ (301)
$\displaystyle=$
$\displaystyle\left(\frac{\lambda_{n}}{\sqrt{n}}u_{j}\operatorname{sgn}(\beta^{*}_{j})+\frac{\eta_{n}}{\sqrt{n}}|u_{j}|\right)I(\beta^{*}_{j}\neq
0)+\frac{\lambda_{n}+\eta_{n}}{\sqrt{n}}|u_{j}|I(\beta^{*}_{j}=0),$

we have

(302) $\displaystyle
V(u)=\begin{dcases}u^{\top}Cu-2u^{\top}W-\delta_{0}\sum_{j}|u_{j}|I(\beta^{*}_{j}\neq
0),\\\ \leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \leavevmode\nobreak\ \leavevmode\nobreak\
\leavevmode\nobreak\ \text{if}\leavevmode\nobreak\ \beta^{*}_{j}u_{j}\leq
0\leavevmode\nobreak\ \text{for}\leavevmode\nobreak\ \forall j\in
S\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\
u_{j}=0\leavevmode\nobreak\ \text{for}\leavevmode\nobreak\ \forall j\in
S^{c},\\\ \infty,\leavevmode\nobreak\ \text{otherwise}.\end{dcases}$

Since $V_{n}(u)$ is convex and $V(u)$ has a unique minimum, we obtain

(303)
$\displaystyle\sqrt{n}\left({\hat{\beta}}_{n}-\beta^{*}\right)\to\operatorname*{argmin}_{u\in\mathcal{U}}\left\\{u^{\top}Cu-2u^{\top}W-\delta_{0}\sum_{j}|u_{j}|\right\\},$
(306)
$\displaystyle\mathcal{U}:=\left\\{u\in\mathbb{R}^{p}\left|\begin{array}[]{l}\beta^{*}_{j}u_{j}\leq
0\leavevmode\nobreak\ \text{for}\leavevmode\nobreak\ \forall j\in S,\\\
u_{j}=0\leavevmode\nobreak\ \text{for}\leavevmode\nobreak\ \forall j\in
S^{c}\end{array}\right.\right\\}.$

On the other hand, the Lagrangian function of
$\operatorname*{argmin}_{u\in\mathcal{U}}V(u)$ is

(307) $\displaystyle
u_{S}^{\top}C_{SS}u_{S}-2u_{S}^{\top}W_{S}-\delta_{0}\sum_{j\in
S}|u_{j}|+\sum_{j\in S}\mu_{j}\beta^{*}_{j}u_{j}+\sum_{j\in
S^{c}}\mu_{j}u_{j},$

where $\mu\in\mathbb{R}^{p}$ is the Lagrangian multiplier. Suppose that
$u^{*}_{S^{c}}=0$. Let $S_{1}:=\\{j:j\in S\leavevmode\nobreak\
\text{and}\leavevmode\nobreak\ u_{j}^{*}\neq 0\\}$ and $S_{2}:=\\{j:j\in
S\leavevmode\nobreak\ \text{and}\leavevmode\nobreak\ u_{j}^{*}=0\\}$. By the
KKT conditions of $\operatorname*{argmin}_{u\in\mathcal{U}}V(u)$, we have

(308)
$\displaystyle\begin{dcases}2C_{S_{1}S_{1}}u_{S_{1}}^{*}-2W_{S_{1}}-\delta_{0}\operatorname{sgn}(u_{S_{1}}^{*})+\mu_{S_{1}}\beta^{*}_{S_{1}}=0\\\
\left|2C_{S_{2}S_{1}}u_{S_{1}}^{*}-2W_{S_{2}}+\mu_{S_{2}}\beta^{*}_{S_{2}}\right|\leq\delta_{0}\\\
\beta^{*}_{S}u_{S}^{*}\leq 0\\\ \mu_{S}\geq 0\\\
\mu_{S}\beta^{*}_{S}u_{S}^{*}=0\\\ \mu_{S^{c}}=u_{S^{c}}^{*}=0\end{dcases}$

If $S_{1}\neq\emptyset$, then we have

(309) $\displaystyle
u_{S_{1}}^{*}=C_{S_{1}S_{1}}^{-1}\left(W_{S_{1}}+\delta_{0}\operatorname{sgn}(u_{S_{1}}^{*})\right).$

The probability holding the third inequality in (308) is less than $1$. This
indicates inconsistent variable selection. If $S_{1}=\emptyset$, then we have

(310)
$\displaystyle\left|-2W_{S_{2}}+\mu_{S_{2}}\beta^{*}_{S_{2}}\right|\leq\delta_{0}.$

The probability holding this inequality is less than $1$. This indicates
inconsistent variable selection.

##  Appendix C Additional Empirical Results

We describe additional empirical results.

###  C.1 Inconsistent Initial Estimator

Figure 10: $\ell_{2}$ estimation errors for inconsistent source data (Case A).

Figure 11: Variable selection F1-scores for inconsistent source data (Case A).

Figure 12: $\ell_{2}$ estimation errors for inconsistent source data (Case B).

Figure 13: Variable selection F1-scores for inconsistent source data (Case B).

In this simulation, we simulated inconsistent initial estimators. We generated
target data by $\beta^{*}_{target}=[3,1.5,0,0,2,0,0,\dots,0]^{\top}$. Then, we
generated the source data with parameters different from those of the target
data. We considered the following two cases:

Case A

$\beta^{*}_{source}=[3,1.5,0,0,2,2,0,\dots,0]^{\top}$ (non-zero $\rightarrow$
zero change for $j=6$)

Case B

$\beta^{*}_{source}=[3,1.5,0,0,0,0,0,\dots,0]^{\top}$ (zero $\rightarrow$ non-
zero change for $j=5$)

Other simulation settings were the same as those in Simulation I.

The results are shown in Figures 11 and 11 for Case A, and Figures 13 and 13
for Case B. In Case A, the Transfer Lasso and the Adaptive Transfer Lasso were
superior in estimation error, and the Adaptive Lasso was slightly inferior to
the Transfer Lasso and the Adaptive Transfer Lasso. Moreover, the Adaptive
Lasso and the Adaptive Transfer Lasso were still superior in variable
selection. In Case B, the Adaptive Lasso and the Adaptive Transfer Lasso
performed worse in terms of estimation error. This may be because the initial
estimator was incorrectly estimated close to zero, and regularization was
strongly applied to it. The performance degradation is particularly
significant for the Adaptive Lasso, but not so significant for Adaptive
Transfer Lasso. The superiority of the Adaptive Lasso and the Adaptive
Transfer Lasso also diminished in variable selection. Overall, the Adaptive
Transfer Lasso performed comparatively well in estimation error and variable
selection, but the inconsistent initial estimators reduced its performance.

###  C.2 Other Initial Estimators

We compared other initial estimators for simulations of a large amount of
target data. The initial estimators included

  * •

Ridge (Figure 15, 15)

  * •

Ridgeless [2, 8]: minimum $\ell_{2}$-norm solution of least squares (Figure
17, 17)

  * •

Lassoless [14, 12]: minimum $\ell_{1}$-norm solution of least squares (Figure
19, 19)

The results of Ridge initial estimators were similar to those of Lasso initial
estimators. The results of Ridgeless and Lassoless initial estimators showed
double descent phenomena, but they did not perform as well as Lasso and Ridge.

Figure 14: $\ell_{2}$ estimation errors for a large amount of source data and
Ridge initial estimators.

Figure 15: Variable selection F1-score for a large amount of source data and
Ridge initial estimators.

Figure 16: $\ell_{2}$ estimation errors for a large amount of source data and
Ridgeless initial estimators.

Figure 17: Variable selection F1-score for a large amount of source data and
Ridgeless initial estimators.

Figure 18: $\ell_{2}$ estimation errors for a large amount of source data and
Lassoless initial estimators.

Figure 19: Variable selection F1-score for a large amount of source data and
Lassoless initial estimators.

###  C.3 Other Metrics

We evaluated other metrics for Simulation I (Section 5.2). The metrics
included

  * •

RMSE for prediction evaluation (Figure 20)

  * •

sensitivity (= (# of correct selected variables) / (# of true active
variables)) for feature selection evaluation (Figure 21)

  * •

specificity (= (# of correctly not selected variables) / (# of true inactive
variables)) for feature selection (Figure 22)

  * •

positive predictive value (= (# of correctly selected variables) / (# of
selected variables)) for feature selection evaluation (Figure 23)

  * •

number of active variables for feature selection evaluation (Figure 24)

The results of RMSE were similar to those of $\ell_{2}$ estimation errors, but
the difference among methods got smaller. The results of 4 metrics for feature
selection showed that Adaptive Lasso and Adaptive Transfer Lasso selected
small number of variables and achieved superior performance especially on
specificity and positive predictive value.

Figure 20: RMSE for a large amount of source data. Figure 21: Sensitivity for
a large amount of source data. Figure 22: Specificity for a large amount of
source data. Figure 23: Positive predictive value for a large amount of source
data. Figure 24: Number of active variables for a large amount of source data.