with $\lambda_{n}\propto\log p_{n}$ the second equality holds.
To prove the Equation (22), first we show that for any
$\alpha\in(0,\frac{1}{2})$, the following equation,

$\lim_{n\rightarrow\infty}\frac{\lambda_{n}^{\frac{1}{\alpha}}}{2\Gamma(1+\frac{1}{\alpha})}\int_{-a_{n}}^{a_{n}}\exp\left(-\lambda_{n}|\beta|^{\alpha}\right)d\beta\geq\lim_{n\rightarrow\infty}\frac{\lambda_{n}^{2}}{2\Gamma(3)}\int_{-a_{n}}^{a_{n}}\exp\left(-\lambda_{n}|\beta|^{\frac{1}{2}}\right)d\beta$
(24)

hold. We then prove Equation (22) holds for the special case
$\alpha=\frac{1}{2}$. Then the more general case $0<\alpha<\frac{1}{2}$ will
trivially hold by using Equation (24).
To prove Equation (24), it is enough to show

$\int_{0}^{a_{n}}\left\\{\exp\left[-\lambda_{n}|\beta|^{\alpha}+(\frac{1}{\alpha}-2)\log\lambda_{n}+b\right]-\exp\left[-\lambda_{n}|\beta|^{\frac{1}{2}}\right]\right\\}d\beta\geq
0$

where $b=\log\frac{\Gamma(3)}{\Gamma(1+\frac{1}{\alpha})}$. Consequently, it
is sufficient to show

$(\frac{1}{\alpha}-2)\log\lambda_{n}+b\geq\lambda_{n}|\beta|^{\alpha}(1-|\beta|^{\frac{1}{2}-\alpha})$

for any $|\beta|\in{(0,a_{n})}$. Since $\lim_{n\rightarrow\infty}a_{n}=0$,
then for $n$ sufficient large we just need,

$(\frac{1}{\alpha}-2)\log\lambda_{n}+b\geq\lambda_{n}|a_{n}|^{\alpha}.$

We see that the above inequality hold since
$\lambda_{n}|a_{n}|^{\alpha}\propto\frac{\log
p_{n}}{p_{n}^{\alpha}}\epsilon_{n}^{\alpha}\rightarrow 0$ as
$n\rightarrow\infty$

Since the exponential power prior with $\alpha=\frac{1}{2}$ has the Laplace-
Gamma mixture representation with $\beta|v\sim\mathrm{Laplace}(0,v)$ and
$v\sim\mathrm{Gamma}(\frac{3}{2},\frac{\lambda^{2}}{4})$, we write

$\displaystyle P(|\beta|>a_{n})$
$\displaystyle=\int_{0}^{\infty}P(|\beta|>a_{n}|v)f_{\lambda}(v)dv$
$\displaystyle=\int_{0}^{\infty}e^{-\frac{a_{n}}{v}}f_{\lambda}(v)dv$
$\displaystyle=\int_{0}^{\frac{a_{n}}{C}}e^{-\frac{a_{n}}{v}}f_{\lambda}(v)dv+\int_{\frac{a_{n}}{C}}^{\infty}e^{-\frac{a_{n}}{v}}f_{\lambda}(v)dv$
$\displaystyle\leq
e^{-C}\int_{0}^{\frac{a_{n}}{C}}f_{\lambda}(v)dv+\int_{\frac{a_{n}}{C}}^{\infty}e^{-\frac{a_{n}}{v}}f_{\lambda}(v)dv$
$\displaystyle=\frac{e^{-C}M^{\frac{3}{2}}}{\Gamma(\frac{3}{2})}\left(\frac{\lambda^{2}a_{n}}{4C}\right)^{\frac{3}{2}}\exp\left(-\frac{\lambda^{2}a_{n}}{4C}M\right)+\int_{\frac{a_{n}}{C}}^{\infty}e^{-\frac{a_{n}}{v}}f_{\lambda}(v)dv$
$\displaystyle\leq\frac{e^{-C}M^{\frac{3}{2}}}{\Gamma(\frac{3}{2})}\left(\frac{\lambda^{2}a_{n}}{4C}\right)^{\frac{3}{2}}\exp\left(-\frac{\lambda^{2}a_{n}}{4C}M\right)+\left(\frac{2}{3}\right)^{\frac{3}{2}}\exp\left(\frac{3}{2}\right)\left(\frac{\lambda^{2}a_{n}}{4C}\right)^{\frac{3}{2}}\exp\left(-\frac{\lambda^{2}a_{n}}{4C}\right)$

where $f_{\lambda}(v)$ is the density of
$\mathrm{Gamma}(\frac{3}{2},\frac{\lambda^{2}}{4})$. In the fifth line, the
equality is obtained by applying the mean value theorem to the first term with
$0<M<1$. In the last line, the inequality is obtained by applying the Chernoff
bound to the tail of Gamma distribution. The Chernoff bound for Gamma
distribution is
$\left(\frac{2}{3}\right)^{\frac{3}{2}}\exp\left(\frac{3}{2}\right)\left(\frac{\lambda^{2}a_{n}}{4C}\right)^{\frac{3}{2}}\exp\left(-\frac{\lambda^{2}a_{n}}{4C}\right)$
if $\frac{a_{n}}{C}\geq\frac{6}{\lambda^{2}}$, otherwise it is 1. Hence, the
value of $C$ we choose should satisfy the constraint
$\frac{a_{n}}{C}\geq\frac{6}{\lambda^{2}}$.
Now, we set $C=a_{n}$, then we have

$\displaystyle P(|\beta|>a_{n})$
$\displaystyle\leq\frac{M^{\frac{3}{2}}}{\Gamma(\frac{3}{2})}\left(\frac{\lambda^{2}}{4}\right)^{\frac{3}{2}}\exp\left(-\frac{\lambda^{2}}{4}M\right)+\left(\frac{2}{3}\right)^{\frac{3}{2}}\exp\left(\frac{3}{2}\right)\left(\frac{\lambda^{2}}{4}\right)^{\frac{3}{2}}\exp\left(-\frac{\lambda^{2}}{4}\right)$
$\displaystyle\leq\exp\left(-\frac{\lambda^{2}}{4}M^{\prime}\right)$
$\displaystyle=p_{n}^{-M^{\prime\prime}\log p_{n}},$

for some $M^{\prime}>0$ and $M^{\prime\prime}>0$.

∎

Lemma 7.1 shows that the exponential power prior with
$0<\alpha\leq{\frac{1}{2}}$ can balance the requirement of putting enough mass
around zero and tail thickness. It is easy to verify that the Bayesian LASSO
fails to satisfy the properties in Lemma 7.1 unless assumption A5 changes to
$||\beta_{0}||_{\infty}=O(\sqrt{s_{0}\log p_{n}/n}/p_{n})$ and we set
$\lambda_{n}\propto p_{n}\log p_{n}/\sqrt{\frac{s\log p_{n}}{n}}$ (Song and
Liang,, 2017). This assumption is unrealistic as it requires the magnitude of
$\beta$ to asymptotically decrease to zero. Next, we provide an upper bound
for the covering number in Euclidean space.

######  Lemma 7.2.

For $\|x\|_{p}\leq M$, $\epsilon<1$ and $p\geq 1$, for any $M$, we have

$N(\epsilon,\left\\{x\in\mathbb{R}^{m}:\|x\|_{p}\leq
M\right\\},\|\cdot\|_{p})\leq\left(\frac{3M}{\epsilon}\right)^{m}$

The proof for Lemma 7.2 can be found in Appendix C of Ghosal and Van der
Vaart, (2017).
Now, we derive the analytical expressions based on the Renyi distance for two
Gaussian distributions.

######  Lemma 7.3.

Consider the root of average Renyi divergence of order $\frac{1}{2}$,

$d(P_{\theta_{1}}^{n},P_{\theta_{2}}^{n})=[-n^{-1}\log(\int\sqrt{f_{1}^{n}f_{2}^{n}}dY)]^{1/2}$

where $f_{1}^{n}$ and $f_{2}^{n}$ are their density function. Let
$P_{\theta_{1}}^{n}$ and $P_{\theta_{2}}^{n}$ be the law of the random
variable with Multivariate Gaussian distribution
$N(g_{1}(X),\sigma_{1}^{2}I_{n})$ and $N(g_{2}(X),\sigma_{2}^{2}I_{n})$
respectively. Then

$d(P_{\theta_{1}}^{n},P_{\theta_{2}}^{n})=\left[\frac{1}{2}\log\frac{\sigma_{1}^{2}+\sigma_{2}^{2}}{2\sigma_{1}\sigma_{2}}+\frac{\|g_{1}(X)-g_{2}(X)\|^{2}}{4n(\sigma_{1}^{2}+\sigma_{2}^{2})}\right]^{1/2}.$

###### Proof.

To simplify notation and apply change of the variable in the later part of the
proof, we set $Y=g(X)+\sigma Z$.

$\displaystyle\int\sqrt{f_{1}f_{2}}dY$ $\displaystyle=$
$\displaystyle\frac{1}{(2\pi)^{\frac{n}{2}}}\frac{1}{(\sigma_{1}^{2}\sigma_{2}^{2})^{\frac{n}{4}}}\int\exp\left[-\frac{\|Y-g_{2}(X)\|^{2}}{4\sigma_{2}^{2}}-\frac{\|Y-g_{1}(X)\|^{2}}{4\sigma_{1}^{2}}\right]dY$
$\displaystyle=$
$\displaystyle\frac{1}{(2\pi)^{\frac{n}{2}}}\frac{1}{(\sigma_{1}^{2}\sigma_{2}^{2})^{\frac{n}{4}}}\int\exp\left[-\frac{\|Y-g_{1}(X)+g_{1}(X)-g_{2}(X)\|^{2}}{4\sigma_{2}^{2}}-\frac{\|Y-g_{1}(X)\|^{2}}{4\sigma_{1}^{2}}\right]dY$
$\displaystyle=$
$\displaystyle\frac{1}{(2\pi)^{\frac{n}{2}}}\frac{1}{(\sigma_{1}^{2}\sigma_{2}^{2})^{\frac{n}{4}}}\exp\left[-\frac{\|g_{1}(X)-g_{2}(X)\|^{2}}{4\sigma_{2}^{2}}\right]$
$\displaystyle\cdot\int\exp\left[-\frac{\|Y-g_{1}(X)\|^{2}}{4\sigma_{2}^{2}}-\frac{\|Y-g_{1}(X)\|^{2}}{4\sigma_{1}^{2}}-\frac{2(Y-g_{1}(X))(g_{1}(X)-g_{2}(X))}{4\sigma_{2}^{2}}\right]dY$
$\displaystyle=$
$\displaystyle\left(\frac{\sigma_{1}^{2}}{\sigma_{2}^{2}}\right)^{\frac{n}{4}}\exp\left[-\frac{\|g_{1}(X)-g_{2}(X)\|^{2}}{4\sigma_{2}^{2}}\right]\int\frac{1}{(2\pi)^{\frac{n}{2}}}\exp\left[-\frac{\|Z\|^{2}(\sigma_{1}^{2}+\sigma_{2}^{2})}{4\sigma_{2}^{2}}-\frac{2\sigma_{1}Z^{T}(g_{1}(X)-g_{2}(X))}{4\sigma_{2}^{2}}\right]dZ$
$\displaystyle=$
$\displaystyle\left(\frac{\sigma_{1}^{2}}{\sigma_{2}^{2}}\right)^{\frac{n}{4}}\exp\left[-\frac{\|g_{1}(X)-g_{2}(X)\|^{2}}{4\sigma_{2}^{2}}\right]\int\frac{1}{(2\pi)^{\frac{n}{2}}}\exp\left[-\frac{\|Z\|^{2}(\sigma_{1}^{2}+\sigma_{2}^{2})}{4\sigma_{2}^{2}}-\frac{2\sigma_{1}Z^{T}(g_{1}(X)-g_{2}(X))}{4\sigma_{2}^{2}}\right]dZ$
$\displaystyle=$
$\displaystyle\left(\frac{\sigma_{1}^{2}}{\sigma_{2}^{2}}\right)^{\frac{n}{4}}\exp\left[-\frac{\|g_{1}(X)-g_{2}(X)\|^{2}}{4\sigma_{2}^{2}}\right]\exp\left[\frac{\sigma_{1}^{2}\|g_{1}(X)-g_{2}(X)\|^{2}}{4\sigma_{2}^{2}(\sigma_{1}^{2}+\sigma_{2}^{2})}\right]\int\frac{1}{(2\pi)^{\frac{n}{2}}}\exp\left[-\frac{\left\|Z\sqrt{\sigma_{1}^{2}+\sigma_{2}^{2}}+\frac{\sigma_{1}}{\sqrt{\sigma_{1}^{2}+\sigma_{2}^{2}}}\right\|^{2}}{4\sigma_{2}^{2}}\right]dZ$
$\displaystyle=$
$\displaystyle\left(\frac{\sigma_{1}^{2}}{\sigma_{2}^{2}}\right)^{\frac{n}{4}}\exp\left[\frac{\|g_{1}(X)-g_{2}(X)\|^{2}}{4(\sigma_{1}^{2}+\sigma_{2}^{2})}\right]\left(\frac{2\sigma_{2}^{2}}{\sigma_{1}^{2}+\sigma_{2}^{2}}\right)^{\frac{n}{2}}$

Thus,

$d(P_{\theta_{1}}^{n},P_{\theta_{2}}^{n})=[-n^{-1}\log(\int\sqrt{f_{1}^{n}f_{2}^{n}}dY)]^{1/2}=\left[\frac{1}{2}\log\frac{\sigma_{1}^{2}+\sigma_{2}^{2}}{2\sigma_{1}\sigma_{2}}+\frac{\|g_{1}(X)-g_{2}(X)\|^{2}}{4n(\sigma_{1}^{2}+\sigma_{2}^{2})}\right]^{1/2}.$

We can see that both the first and second term are non-negative and
$d(P_{\theta_{1}}^{n}P_{\theta_{2}}^{n})=0$ if and only if $g_{1}=g_{2}$ and
$\sigma_{1}=\sigma_{2}$. The symmetric property
$d(P_{\theta_{1}}^{n},P_{\theta_{2}}^{n})=d(P_{\theta_{2}}^{n},P_{\theta_{1}}^{n})$
is also easy to see. ∎

Finally, we restate a simple version of Lemma 8.21 from Ghosal and Van der
Vaart, (2017), which will help us to prove Theorem 6.2.

######  Lemma 7.4.

For any $D>0$ and $\epsilon_{n}\geq\frac{1}{\sqrt{n}}$,

$P_{\theta_{0}}^{n}\left(\int\frac{P_{\theta}^{n}}{P_{\theta_{0}}^{n}}d\Pi(\theta)\leq\Pi\left(P_{\theta}^{n}:K(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2},V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2}\right)e^{-(1+D)n\epsilon_{n}^{2}}\right)\leq\frac{1}{D^{2}n\epsilon_{n}^{2}}$

### Proof of the contraction theorem

The key to our proof will be drawing upon the framework of posterior
concentration theory for independent but not identical observations from the
Theorem 1 of the seminal paper by Ghosal et al., (2007). Theorem 1 provides a
very general framework for showing posterior concentration in infinite-
dimensional models. The contraction rate is described in terms of the average
squared Hellinger distance by default. However, as mentioned by Ning et al.,
(2020), closeness in terms of the average squared Hellinger distance does not
imply that the posterior means of the parameters in the two densities are also
close on average in terms of the Euclidean distance. To alleviate the problem,
we use the root of average Renyi divergence of order $\frac{1}{2}$ as
distance. We will construct a likelihood ratio test and then show that such a
test works well for this distance. Finally, we show that the posterior
consistency under this new distance implies the the posterior consistency
under the Euclidean distance.

Restating Theorem 1 of Ghosal et al., (2007) here, suppose that for a sequence
$\epsilon_{n}$ with $\epsilon_{n}\rightarrow{0}$ and
$n\epsilon_{n}^{2}\rightarrow{\infty}$, there exists constants $C_{1}>0$,
$C_{2}>0$ and a sequence of sieves $F_{n}\subset F$, where $F$ is the sample
space (model space) of the prior distribution, such that if the following
three conditions hold:
1\. Prior concentration condition:

$\Pi_{n}\left(P_{\theta}^{n}:K(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2},V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2}\right)\geq\exp\left(-n\varepsilon_{n}^{2}C_{1}\right).$
(25)

where
$K(P_{\theta_{0}}^{n},P_{\theta}^{n})=-P_{\theta_{0}}^{n}\log(P_{\theta}^{n}/P_{\theta_{0}}^{n})$
and
$V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})=P_{\theta_{0}}^{n}|\log(P_{\theta_{0}}^{n}/P_{\theta}^{n})-K(P_{\theta_{0}}^{n},P_{\theta}^{n})|^{2}$

2\. Sieve sequence condition:

$\Pi_{n}\left(F_{n}^{c}\right)\leq\exp\left(-n\varepsilon_{n}^{2}(C_{1}+2)\right)$
(26)

for some $\xi\in(0,1)$ and $j\in\mathbb{N}$
3\. Testing conditions:
There exists a sequence of test function $\phi_{n}$ such that

$\displaystyle
P_{\theta_{0}}^{n}\phi_{n}\leq\exp\left(-C_{2}n\varepsilon_{n}^{2}\right)$
(27) $\displaystyle\sup_{\theta\in
F_{n}:d(P_{\theta_{0}}^{n},P_{\theta}^{n})\geq M\epsilon_{n}}$ $\displaystyle
P_{\theta}^{n}\left(1-\phi_{n}\right)\leq\exp\left(-C_{2}n\varepsilon_{n}^{2}\right)$

Then for sufficiently large M, $\Pi_{n}\left(\theta\subset
F:d\left(P^{n}_{\theta},P^{n}_{\theta_{0}}\right)\geq M\varepsilon_{n}\mid
X_{1},\ldots,X_{n}\right)\rightarrow{0}$ in $P^{n}_{\theta_{0}}-$ probability.

Before begin the proof, we defined the sieve $F_{n}$ by using the concept of
generalized dimension (Bhattacharya et al.,, 2015; Song and Liang,, 2017;
Ročková and George,, 2018), which approximates the model size. We define the
generalized dimension as

$\gamma_{j}(\beta_{j})=I(|\beta_{j}|>a_{n})\,\,\,\text{and}\,\,\,|\gamma(\beta)|=\sum_{j=1}^{p_{n}}\gamma_{j}(\beta_{j})$

where $a_{n}=\frac{\sigma_{0}}{\sqrt{2k}}\sqrt{s_{0}\log p_{n}/n}/p_{n}$ and
$\lim_{n\rightarrow{\infty}}a_{n}=0$. Then we can define the sieve $F_{n}$ as

$F_{n}=\left\\{S_{\beta}:|\gamma(\beta)|\leq{C_{3}s_{0}}\right\\}\cap\left\\{\beta:\sum_{j=1}^{p}|\beta_{j}|^{\alpha}\leq
C_{3}Es_{0}\right\\}\cap\left\\{\sigma^{2}:\frac{1}{n}\leq\sigma^{2}\leq
p_{n}^{C_{3}s_{0}/\delta}\right\\}$

Proof of theorem 3.1

###### Proof.

Part 1. Prior concentration condition:

Let $P_{\theta}^{n}\sim N_{n}(X\beta,\sigma^{2}I_{n})$ and
$P_{\theta_{0}}^{n}\sim N_{n}(X\beta_{0},\sigma_{0}^{2}I_{n})$. Then we have:

$\displaystyle\log\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}$
$\displaystyle=\frac{n}{2}\log\frac{\sigma_{0}^{2}}{\sigma^{2}}+\frac{||Y-X\beta_{0}||^{2}}{2\sigma_{0}^{2}}-\frac{||Y-X\beta||^{2}}{2\sigma^{2}}$
$\displaystyle=\frac{n}{2}\left(\log\frac{\sigma_{0}^{2}}{\sigma^{2}}+\frac{||Y-X\beta_{0}||^{2}}{n\sigma_{0}^{2}}-\frac{||Y-X\beta_{0}+X\beta_{0}-X\beta||^{2}}{n\sigma^{2}}\right)$
$\displaystyle=\frac{n}{2}\left(\log\frac{\sigma_{0}^{2}}{\sigma^{2}}+\frac{||Y-X\beta_{0}||^{2}}{n\sigma_{0}^{2}}-\frac{||Y-X\beta_{0}||^{2}+||X\beta_{0}-X\beta||^{2}+2(Y-X\beta_{0})^{T}(X\beta_{0}-X\beta)}{n\sigma^{2}}\right)$

Thus,

$-P_{\theta_{0}}^{n}\log\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}=\frac{n}{2}\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1-\log\frac{\sigma_{0}^{2}}{\sigma^{2}}+\frac{||X(\beta-\beta_{0})||_{2}^{2}}{\sigma^{2}n}\right)$

Since

$\log\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}=\frac{n}{2}\left(\log\frac{\sigma_{0}^{2}}{\sigma^{2}}+\frac{||Y-X\beta_{0}||^{2}}{n\sigma_{0}^{2}}-\frac{||Y-X\beta_{0}||^{2}}{n\sigma^{2}}-\frac{||X\beta_{0}-X\beta||^{2}}{n\sigma^{2}}-\frac{2(Y-X\beta_{0})^{T}(X\beta_{0}-X\beta)}{n\sigma^{2}}\right)$

we set
$A=\log\frac{\sigma_{0}^{2}}{\sigma^{2}}$,$B=\frac{||Y-X\beta_{0}||^{2}}{n\sigma_{0}^{2}}$,
$C=-\frac{||Y-X\beta_{0}||^{2}}{n\sigma^{2}}$,
$D=-\frac{||X\beta_{0}-X\beta||^{2}}{n\sigma^{2}}$,
$E=-\frac{2(Y-X\beta_{0})^{T}(X\beta_{0}-X\beta)}{n\sigma^{2}}$

$\displaystyle
P_{\theta_{0}}^{n}\left(\log\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}\right)^{2}=$
$\displaystyle\frac{n^{2}}{4}P_{0}(A^{2}+B^{2}+C^{2}+D^{2}+E^{2}$
$\displaystyle+2AB+2AC+2AD+2AE+2BC+2BD+2BE+2CD+2CE+2DE)$

with

$\displaystyle
P_{\theta_{0}}^{n}A^{2}=\left(\log\frac{\sigma_{0}^{2}}{\sigma^{2}}\right)^{2},$
$\displaystyle\quad P_{\theta_{0}}^{n}B^{2}=\frac{n+2}{n},$ $\displaystyle
P_{\theta_{0}}^{n}C^{2}=\frac{n+2}{n}\frac{\sigma_{0}^{4}}{\sigma^{4}},$
$\displaystyle\quad
P_{\theta_{0}}^{n}D^{2}=\frac{||X\beta_{0}-X\beta||^{4}}{n^{2}\sigma^{4}},$
$\displaystyle
P_{\theta_{0}}^{n}E^{2}=\frac{4\sigma_{0}^{2}||X\beta_{0}-X\beta||^{2}}{n^{2}\sigma^{4}},$
$\displaystyle\quad
2P_{\theta_{0}}^{n}A(B+C+D+E)=2\log\frac{\sigma_{0}^{2}}{\sigma^{2}}(1-\frac{\sigma_{0}^{2}}{\sigma^{2}}-\frac{||X\beta_{0}-X\beta||^{2}}{n\sigma^{2}}),$
$\displaystyle
2P_{\theta_{0}}^{n}BC=-\frac{2\sigma_{0}^{2}}{\sigma^{2}}\frac{n+2}{n},$
$\displaystyle\quad
2P_{\theta_{0}}^{n}BD=-\frac{2||X\beta_{0}-X\beta||^{2}}{n\sigma^{2}},$
$\displaystyle 2P_{\theta_{0}}^{n}BE=0,$ $\displaystyle\quad
2P_{\theta_{0}}^{n}CD=\frac{2\sigma_{0}^{2}||X\beta_{0}-X\beta||^{2}}{n\sigma^{4}},$
$\displaystyle 2P_{\theta_{0}}^{n}CE=0,$ $\displaystyle\quad
2P_{\theta_{0}}^{n}DE=0.$

By arranging the terms, we have:

$P_{\theta_{0}}^{n}\left(\log\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}\right)^{2}=\frac{n^{2}}{4}\left[\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1-\log\frac{\sigma_{0}^{2}}{\sigma^{2}}+\frac{||X\beta_{0}-X\beta||_{2}^{2}}{n\sigma^{2}}\right)^{2}+\frac{4\sigma_{0}^{2}||X\beta_{0}-X\beta||_{2}^{2}}{n^{2}\sigma^{4}}+\frac{2}{n}\left(1-\frac{\sigma_{0}^{2}}{\sigma^{2}}\right)^{2}\right]$

Since
$V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})=P_{\theta_{0}}^{n}\left(\log\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}\right)^{2}-K(P_{\theta_{0}}^{n},P_{\theta}^{n})^{2}$,

$V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})=\frac{\sigma_{0}^{2}||X\beta_{0}-X\beta||_{2}^{2}}{\sigma^{4}}+\frac{n}{2}\left(1-\frac{\sigma_{0}^{2}}{\sigma^{2}}\right)^{2}$

For $0<\frac{\sigma_{0}^{2}}{\sigma^{2}}\leq 2$, considering the Taylor series
centered at 1, we have

$\log\frac{\sigma_{0}^{2}}{\sigma^{2}}=\sum_{n=1}^{\infty}\frac{(-1)^{n-1}(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1)^{n}}{n}=\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1\right)-\frac{1}{2}\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1\right)^{2}+\frac{1}{3}\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1\right)^{3}-\frac{1}{4}\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1\right)^{4}+...$

We see that

$\frac{\sigma_{0}^{2}}{\sigma^{2}}-1-\log\frac{\sigma_{0}^{2}}{\sigma^{2}}\leq\frac{1}{2}\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1\right)^{2}$

Define the two events $\mathcal{A}_{1}$ and $\mathcal{A}_{2}$ as follows:

$\displaystyle\mathcal{A}_{1}$
$\displaystyle=\left\\{\sigma^{2}:\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1\right)^{2}\leq{\epsilon_{n}^{2}}\right\\}$
$\displaystyle\mathcal{A}_{2}$
$\displaystyle=\left\\{(\beta,\sigma^{2}):\frac{||X(\beta-\beta_{0})||_{2}^{2}}{\sigma^{2}}\leq\frac{n^{\alpha}\epsilon_{n}^{2}}{p_{n}}\right\\}$

Condition on $\mathcal{A}_{1}\cap\mathcal{A}_{2}$, we see that

$\displaystyle K(P_{\theta_{0}}^{n},P_{\theta}^{n})$
$\displaystyle=\frac{n}{2}\left(\frac{\sigma_{0}^{2}}{\sigma^{2}}-1-\log\frac{\sigma_{0}^{2}}{\sigma^{2}}+\frac{||X(\beta-\beta_{0})||_{2}^{2}}{\sigma^{2}n}\right)\leq\frac{n\epsilon_{n}^{2}}{4}+\frac{n\epsilon_{n}^{2}}{6}\leq
n\epsilon_{n}^{2}$ $\displaystyle V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})$
$\displaystyle=\frac{\sigma_{0}^{2}||X\beta_{0}-X\beta||_{2}^{2}}{\sigma^{4}}+\frac{n}{2}\left(1-\frac{\sigma_{0}^{2}}{\sigma^{2}}\right)^{2}$
$\displaystyle\leq\frac{n^{\alpha}\epsilon_{n}^{2}(1+\epsilon_{n})}{p_{n}}+\frac{n\epsilon_{n}^{2}}{2}$
$\displaystyle\leq n\epsilon_{n}^{2}$

The last inequality is by assumption A1. Then we have:

$\Pi\left(P_{\theta}^{n}:K(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2},V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2}\right)\geq{\Pi(\mathcal{A}_{2}|\mathcal{A}_{1})\Pi(\mathcal{A}_{1})}.$

First, we give a lower bound to $\Pi(\mathcal{A}_{1})$. Since
$\sigma^{2}\sim\mathrm{InvGamma}(\delta,\delta)$, we have

$\displaystyle\Pi(\mathcal{A}_{1})$
$\displaystyle\geq\frac{\delta^{\delta}}{\Gamma(\delta)}\int_{\frac{\sigma_{0}^{2}}{1+\epsilon_{n}}}^{\sigma_{0}^{2}}e^{-\delta/\sigma^{2}}(\sigma^{2})^{-\delta-1}d\sigma^{2}$
$\displaystyle\geq\frac{\delta^{\delta}}{\Gamma(\delta)}\frac{\epsilon_{n}\sigma_{0}^{2}}{1+\epsilon_{n}}\exp\left[-\frac{(1+\epsilon_{n})\delta}{\sigma_{0}^{2}}\right](\sigma_{0}^{2})^{-\delta-1}$
$\displaystyle\geq\exp\left(-n\epsilon_{n}^{2}\right)$

Next, we derive the lower bound for $\Pi(\mathcal{A}_{1}|\mathcal{A}_{1})$.
Condition on $\mathcal{A}_{1}$ and by assumption A3, we have

$\frac{||X(\beta-\beta_{0})||_{2}^{2}}{\sigma^{2}}\leq\frac{\lambda_{\max}(X^{T}X)}{\sigma^{2}}\|\beta-\beta_{0}\|^{2}\leq\frac{2kn^{\alpha}}{\sigma_{0}^{2}}\|\beta-\beta_{0}\|^{2}.$

Define
$\mathcal{A}_{2}^{*}=\left\\{\beta:\|\beta-\beta_{0}\|^{2}\leq\frac{\sigma_{0}^{2}\epsilon_{n}^{2}}{2kp_{n}}\right\\}$,
we have
$\Pi(\mathcal{A}_{2}|\mathcal{A}_{1})\geq\Pi(\mathcal{A}_{2}^{*}|\mathcal{A}_{1})$.
Thus,

$\displaystyle\Pi(\mathcal{A}_{2}|\mathcal{A}_{1})$
$\displaystyle\geq\Pi(\left\|(\beta-\beta_{0})\right\|_{2}^{2}\leq\frac{\sigma_{0}^{2}\epsilon_{n}^{2}}{2kp_{n}})$
$\displaystyle\geq\Pi_{S_{0}}\left(||\beta_{S_{0}}-\beta_{0S_{0}}||_{2}^{2}\leq\frac{\sigma_{0}^{2}\epsilon_{n}^{2}}{2kp_{n}}\frac{s_{0}}{p_{n}}\right)\Pi_{S_{0}^{c}}\left(||\beta_{s_{0}^{c}}||_{2}^{2}\leq\frac{\sigma_{0}^{2}\epsilon_{n}^{2}}{2kp_{n}}\frac{p_{n}-s_{0}}{p_{n}}\right)$
$\displaystyle\geq\prod_{j\in
S_{0}}\Pi\left(|\beta_{0j}-\beta_{j}|\leq\frac{\sigma_{0}\epsilon_{n}}{p_{n}\sqrt{2k}}\right)\prod_{j\in
S_{0}^{C}}\Pi\left(|\beta_{j}|\leq\frac{\sigma_{0}\epsilon_{n}}{p_{n}\sqrt{2k}}\right)$
$\displaystyle\geq\left(\frac{2\sigma_{0}\epsilon_{n}}{p_{n}\sqrt{2k}}\inf_{\beta\in\left[-E,E\right]}\Pi_{\lambda}(\beta)\right)^{s_{0}}\left(1-\int_{|\beta|\geq{a_{n}}}\Pi_{\lambda}(\beta)d\beta\right)^{p_{n}-s_{0}}$
$\displaystyle\geq(2\epsilon_{n})^{s_{0}}\exp\left(-2n\epsilon_{n}^{2}\right)(1-p_{n}^{-(1+u)})^{p_{n}-s_{0}}$
$\displaystyle\geq\exp\left(-Mn\epsilon_{n}^{2}\right),$

for some $M>0$. The fourth and fifth inequality are from Lemma 7.1. The final
inequality is by using the fact that the second term
$\exp\left(-n\epsilon_{n}^{2}\right)$ dominates the first term and the third
term is asymptotically 1.

Combining all the result above, we have

$\Pi\left(P_{\theta}^{n}:K(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2},V_{2,0}(P_{\theta_{0}}^{n},P_{\theta}^{n})\leq
n\varepsilon_{n}^{2}\right)\geq{\Pi(\mathcal{A}_{2}|\mathcal{A}_{1})\Pi(\mathcal{A}_{1})}\geq\exp\left(-C_{1}n\epsilon_{n}^{2}\right)$

where $C_{1}=M_{1}+M_{2}$.

Part 2. Sieve sequence condition:

First,we show the existence of sieve set $F_{n}$ such that

$\Pi_{n}\left(F_{n}^{c}\right)\leq\exp\left(-n\varepsilon_{n}^{2}(C_{1}+2)\right),$

we choose $C_{3}=M(C_{1}+3)$ with constant $M>0$ and define $F_{n}$ as:

$F_{n}=\left\\{S_{\beta}:|\gamma(\beta)|\leq{C_{3}s_{0}}\right\\}\cap\left\\{\beta:\sum_{j=1}^{p}|\beta_{j}|^{\alpha}\leq
C_{3}Es_{0}\right\\}\cap\left\\{\sigma^{2}:\frac{1}{n}\leq\sigma^{2}\leq
p_{n}^{C_{3}s_{0}/\delta}\right\\}$

It is easy to see that:

$\displaystyle\Pi_{n}\left(F_{n}^{c}\right)$
$\displaystyle=\Pi(|\gamma(\beta)|>C_{3}s_{0})+\Pi\left(\left\\{S_{\beta}:|\gamma(\beta)|\leq{C_{3}s_{0}}\right\\}\cap\left\\{\beta:\sum_{j=1}^{p}|\beta_{j}|^{\alpha}>C_{3}Es_{0}\right\\}\right)$
$\displaystyle+\Pi(\sigma^{2}<\frac{1}{n})+\Pi(\sigma^{2}>p_{n}^{C_{3}s_{0}/\delta})$
$\displaystyle\leq\Pi(|\gamma(\beta)|>C_{3}s_{0})+\Pi\left(\left\\{\beta:\sum_{j=1}^{p}|\beta_{j}|^{\alpha}>C_{3}Es_{0}\right\\}\Big{|}\left\\{S_{\beta}:|\gamma(\beta)|\leq{C_{3}s_{0}}\right\\}\right)$
$\displaystyle+\Pi(\sigma^{2}<\frac{1}{n})+\Pi(\sigma^{2}>p_{n}^{C_{3}s_{0}/\delta})$
$\displaystyle=(a)+(b)+(c)+(d)$

We now show
$(a)+(b)+(c)+(d)\leq{4\exp\left(-n\varepsilon_{n}^{2}(C_{1}+3)\right)}$. For
term $(a)$, we see that

$\Pi(|\gamma(\beta)|>C_{3}s_{0})=P(\mathrm{Binomial}(p_{n},v_{n})\geq
C_{3}s_{0})$

where $v_{n}=\int_{|\beta|\geq{a_{n}}}\Pi_{\lambda}(\beta)d\beta$. By Lemma
A.3 in Song and Liang, (2017), we have

$\Pi(|\gamma(\beta)|>C_{3}s_{0})\leq
1-\Phi(\sqrt{2p_{n}H\left[v_{n},(C_{3}s_{0}-1)/p_{n}\right]})\leq\frac{\exp\left\\{-p_{n}H\left[v_{n},(C_{3}s_{0}-1)/p_{n}\right]\right\\}}{\sqrt{2\pi}\sqrt{2p_{n}H\left[v_{n},(C_{3}s_{0}-1)/p_{n}\right]}}$

where
$p_{n}H[v_{n},(C_{3}s_{0}-1)/p_{n}]=(C_{3}s_{0}-1)\log\frac{C_{3}s_{0}-1}{p_{n}v_{n}}+(p_{n}-C_{3}s_{0}+1)\log\frac{p_{n}-C_{3}s_{0}+1}{p_{n}-p_{n}v_{n}}$.
Therefore, to show
$\Pi\left(|\gamma(\beta)|>C_{3}s_{0}\right)\leq\exp\left(-(C_{1}+3)n\epsilon_{n}^{2}\right)$,
it sufficient to show that

$p_{n}H[v_{n},(C_{3}s_{0}-1)/p_{n}]\geq(C_{1}+3)n\epsilon_{n}^{2}$

For the first term of $p_{n}H[v_{n},(C_{3}s_{0}-1)/p_{n}]$ , by Lemma 7.1,
$v_{n}\leq p_{n}^{-(1+u)}$, we have:

$\displaystyle(C_{3}s_{0}-1)\log\frac{C_{3}s_{0}-1}{p_{n}v_{n}}$
$\displaystyle\geq(C_{3}s_{0}-1)\log(C_{3}s_{0}-1)p_{n}^{u}$
$\displaystyle\geq uC_{3}s_{0}\log p_{n}-u\log
p_{n}+(C_{3}s_{0}-1)\log(C_{3}s_{0}-1)$ $\displaystyle\geq
uC_{3}n\epsilon_{n}^{2}-u\log p_{n}$
$\displaystyle=uM(C_{1}+3)n\epsilon_{n}^{2}-u\log p_{n}$ $\displaystyle\geq
0.5uM(C_{1}+3)n\epsilon_{n}^{2}.$

The last inequality will hold for $n$ sufficient large because the term
$n\epsilon_{n}^{2}$ always dominates $\log p_{n}$. We set $M=\frac{2}{u}$,
then we have

$(C_{3}s_{0}-1)\log\frac{C_{3}s_{0}-1}{p_{n}v_{n}}\geq(C_{1}+3)n\epsilon_{n}^{2}.$

For the second term of $p_{n}H[v_{n},(C_{3}s_{0}-1)/p_{n}]$, we have

$0<(p_{n}-C_{3}s_{0}+1)\log\frac{p_{n}-C_{3}s_{0}+1}{p_{n}-p_{n}v_{n}}\approx
C_{3}s_{0}-\frac{(C_{3}s_{0})^{2}}{p_{n}}<n\epsilon_{n}^{2}.$

Consequently,

$p_{n}H[v_{n},(C_{3}s_{0}-1)/p_{n}]\geq(C_{1}+3)n\epsilon_{n}^{2}.$

For term $(b)$, when $n$ is large, it is sufficient to show that

$\Pi\left(\left\\{\beta:\sum_{j:|\beta_{j}|>a_{n}}|\beta_{j}|^{\alpha}>\frac{3}{4}C_{3}Es_{0}\right\\}\Big{|}\left\\{S_{\beta}:|\gamma(\beta)|\leq{C_{3}s_{0}}\right\\}\right)\leq\exp\left(-n\varepsilon_{n}^{2}(C_{1}+3)\right)$

Since $\sum_{j=1}^{C_{3}s_{0}}|\beta_{j}|^{\alpha}$ is Gamma distributed with
shape parameter $\frac{C_{3}s_{0}}{\alpha}$ and scale parameter
$\lambda_{n}$,applying the Chernoff bound to Gamma distribution and choosing
$E$ as any positive constant such that $E>4$, we have

$\pi(\sum_{j=1}^{C_{3}s_{0}}|\beta_{j}|^{\alpha}>\frac{3}{4}C_{3}Es_{0})\leq
e^{-3C_{3}s_{0}\log p_{n}}.$

Therefore,

$\displaystyle\Pi\left(\left\\{\beta:\sum_{j:|\beta_{j}|>a_{n}}|\beta_{j}|^{\alpha}>\frac{3}{4}C_{3}Es_{0}\right\\}\Big{|}\left\\{S_{\beta}:|\gamma(\beta)|\leq{C_{3}s_{0}}\right\\}\right)$
$\displaystyle\leq\sum_{s=1}^{C_{3}s_{0}}\binom{p_{n}}{s}\pi(\sum_{j=1}^{s}|\beta_{j}|^{\alpha}>\frac{3}{4}C_{3}Es_{0})$
$\displaystyle\leq
C_{3}s_{0}\binom{p_{n}}{C_{3}s_{0}}\pi(\sum_{j=1}^{C_{3}s_{0}}|\beta_{j}|^{\alpha}>\frac{3}{4}C_{3}Es_{0})$
$\displaystyle\leq e^{C_{3}s_{0}(\log p_{n}+1)}e^{-3C_{3}s_{0}\log p_{n}}$
$\displaystyle\leq e^{-C_{3}s_{0}\log p_{n}}$

where the second is obtained by the upper bound
$\binom{n}{k}\leq\left(\frac{ne}{k}\right)^{k}$ for $\forall 1\leq k\leq n$.

For term $(c)$

$\Pi(\sigma^{2}<\frac{1}{n})=\Pi(\frac{1}{\sigma^{2}}>n)\leq e^{-n\delta}\leq
e^{-C_{3}s_{0}\log p_{n}\delta}$

where the first inequality is followed by using the Chernoff bound to Gamma
distribution with shape parameter $\delta$ and rate parameter $\delta$ and the
second inequality is followed by assumption A2.

For term $(d)$:

$\begin{aligned}
\Pi(\sigma^{2}>p_{n}^{C_{3}s_{0}/\delta})&=\int_{p_{n}^{C_{3}s_{0}/\delta}}^{\infty}\frac{\delta^{\delta}}{\Gamma\left(\delta\right)}\left(\sigma^{2}\right)^{-\delta-1}e^{-\delta/\sigma^{2}}d\sigma^{2}\\\
&\leq\int_{p_{n}^{C_{3}s_{0}/\delta}}^{\infty}\frac{\delta^{\delta}}{\Gamma\left(\delta\right)}\left(\sigma^{2}\right)^{-\delta-1}d\sigma^{2}\\\
&=\frac{\delta^{\delta+1}}{\Gamma\left(\delta\right)}p_{n}^{-C_{3}s_{0}}\\\
&=\frac{\delta^{\delta+1}}{\Gamma\left(\delta\right)}e^{-C_{3}s_{0}\log
p_{n}}\end{aligned}.$

Part 3. Testing conditions:

When the variance is unknown, techniques from Ghosal et al., (2007); Ghosal
and Van der Vaart, (2017) can not be applied directly. Instead, we follow Ning
et al., (2020)’s approach by constructing the likelihood ratio test with the
sieve $F_{n}$ broken up into small pieces. More precisely, we perform tests in
small covering pieces such that

$\|\beta_{1}-\beta\|_{2}\leq\frac{\sqrt{c}\epsilon_{n}}{\sqrt{v_{1}n}},\quad|\sigma_{1}^{2}-\sigma^{2}|\leq\frac{1}{n^{2}}$
(28)

where $(\beta,\sigma^{2})$ and $(\beta_{1},\sigma_{1}^{2})$ are parameters
which belong to the probability density $f_{\theta}^{n}$ and
$f_{\theta_{1}}^{n}$ respectively.

In each piece, we consider testing $H_{0}:f_{\theta}^{n}=f_{\theta_{0}}^{n}$
against $H_{1}:f_{\theta}^{n}=f_{\theta_{1}}^{n}$ with $\theta_{1}\in F_{n}$
and $d(P_{\theta_{1}}^{n},P_{\theta_{0}}^{n})\geq M\epsilon_{n}$. To do that,
we use the likelihood ratio test
$\phi_{n}=1\left\\{\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}\geq 1\right\\}$
which is the most powerful Neyman-Pearson test.

Since
$d(P_{\theta_{1}}^{n},P_{\theta_{0}}^{n})=[-n^{-1}\log(\int\sqrt{f_{\theta_{1}}^{n}f_{\theta_{0}}^{n}}dY)]^{1/2}\geq
M\epsilon_{n}$, $\int\sqrt{f_{\theta_{1}}^{n}f_{\theta_{0}}^{n}}dY\leq
e^{-nM^{2}\epsilon_{n}^{2}}$ then by Markov inequality

$\displaystyle\mathbb{E}_{f_{\theta_{0}}^{n}}\phi_{n}$
$\displaystyle=\mathbb{E}_{f_{\theta_{0}}^{n}}\left(\sqrt{f_{\theta_{1}}^{n}/f_{\theta_{0}}^{n}}\geq
1\right)\leq\int\sqrt{f_{\theta_{1}}^{n}f_{\theta_{0}}^{n}}dY\leq
e^{-nM^{2}\epsilon_{n}^{2}}$
$\displaystyle\mathbb{E}_{f_{\theta_{1}}^{n}}\left(1-\phi_{n}\right)$
$\displaystyle=\mathbb{E}_{f_{\theta_{1}}^{n}}\left(\sqrt{f_{\theta_{0}}^{n}/f_{\theta_{1}}^{n}}\geq
1\right)\leq\int\sqrt{f_{1}^{n}f_{2}^{n}}dY\leq e^{-nM^{2}\epsilon_{n}^{2}}$

For the type II error, by Cauchy-Schwartz inequality

$\mathbb{E}_{f_{\theta}^{n}}\left(1-\phi_{n}\right)\leq\left\\{\mathbb{E}_{f_{\theta_{1}}^{n}}\left(1-\phi_{n}\right)\right\\}^{1/2}\left\\{\mathbb{E}_{f_{\theta_{1}}^{n}}\left(f_{\theta}^{n}/f_{\theta_{1}}^{n}\right)^{2}\right\\}^{1/2}$

Thus, if we can show that
$\mathbb{E}_{f_{\theta_{1}}^{n}}\left(f_{\theta}^{n}/f_{\theta_{1}}^{n}\right)^{2}\leq
e^{cn\epsilon_{n}^{2}}$ for very small c, then our work finish.

To see this, observe that

$\displaystyle\mathbb{E}_{f_{\theta_{1}}^{n}}\left(f_{\theta}^{n}/f_{\theta_{1}}^{n}\right)^{2}$
$\displaystyle=\left(\frac{\sigma^{2}}{2\sigma^{2}-\sigma_{1}^{2}}\right)^{\frac{n}{2}}\left(\frac{\sigma^{2}}{\sigma_{1}^{2}}\right)^{\frac{n}{2}}\exp\left(\frac{\|g(X)-g_{1}(X)\|^{2}}{2\sigma^{2}-\sigma_{1}^{2}}\right)$
(29)
$\displaystyle=\left(\frac{1}{2-\frac{\sigma_{1}^{2}}{\sigma^{2}}}\right)^{\frac{n}{2}}\left(\frac{\sigma^{2}}{\sigma_{1}^{2}}\right)^{\frac{n}{2}}\exp\left(\frac{\|X\beta-X\beta_{1}\|^{2}}{\sigma^{2}+\sigma_{1}^{2}}\frac{1+\frac{\sigma_{1}^{2}}{\sigma^{2}}}{2-\frac{\sigma_{1}^{2}}{\sigma^{2}}}\right)$

Because $|\sigma_{1}^{2}-\sigma^{2}|\leq\frac{1}{n^{2}}$ and
$\frac{1}{\sigma^{2}}\leq n$,we obtained the upper bounded

$\displaystyle\mathbb{E}_{f_{\theta_{1}}^{n}}\left(f_{\theta}^{n}/f_{\theta_{1}}^{n}\right)^{2}$
$\displaystyle\leq\left(\frac{1}{1-1/n}\right)^{n}\exp\left(\frac{\|X\beta-X\beta_{1}\|^{2}}{\sigma^{2}+\sigma_{1}^{2}}\frac{1+\frac{\sigma_{1}^{2}}{\sigma^{2}}}{2-\frac{\sigma_{1}^{2}}{\sigma^{2}}}\right)$
$\displaystyle\leq\left(\frac{1}{1-1/n}\right)^{n}\exp\left(\frac{2\|X\beta-X\beta_{1}\|^{2}}{\sigma^{2}+\sigma_{1}^{2}}\right)$
$\displaystyle\leq\left(\frac{1}{1-1/n}\right)^{n}\exp\left(\frac{2nv_{1}\|\beta-\beta_{1}\|^{2}}{\sigma^{2}+\sigma_{1}^{2}}\right)$
$\displaystyle\leq\left(\frac{1}{1-1/n}\right)^{n}\exp\left(n^{2}v_{1}\|\beta-\beta_{1}\|^{2}\right)$
$\displaystyle\leq\exp\left(cn\epsilon_{n}^{2}\right)$

where the third inequality is by assumption A3, the fourth inequality is
obtained when $\sigma_{1}^{2}$ and $\sigma^{2}$ are sufficient close($n$
sufficient large). To complete the construction of the test, we need to show
that $\log N^{*}\lesssim n\epsilon_{n}^{2}$, where $N^{*}$ is the number of
covering pieces for sieve $F_{n}$ satisfying $(\ref{eq:pieces})$. We see that

$\displaystyle\log N^{*}$ $\displaystyle\leq\log
N\left(\frac{\sqrt{c}\epsilon_{n}}{\sqrt{v_{1}n}},\left\\{\beta\in
F_{n}\right\\},\|\cdot\|_{2}\right)+\log
N\left(\frac{1}{n^{2}},\left\\{\sigma^{2}:\sigma^{2}\in
F_{n}\right\\},|\cdot|\right)$ $\displaystyle\leq\log
N\left(\frac{\sqrt{c}\epsilon_{n}}{\sqrt{v_{1}n}},\left\\{\beta:|\gamma(\beta)|\leq{C_{3}s_{0}},\|\beta\|_{2}\leq(C_{3}Es_{0})^{\frac{1}{\alpha}}\right\\},\|\cdot\|_{2}\right)$
$\displaystyle+\log N\left(\frac{1}{n^{2}},\left\\{\sigma^{2}:\sigma^{2}\leq
p_{n}^{C_{3}s_{0}/\delta}\right\\},|\cdot|\right)$ $\displaystyle\leq\log
C_{3}s_{0}+\log\binom{p_{n}}{C_{3}s_{0}}+C_{3}s_{0}\log
3\sqrt{v_{1}n}(C_{3}Es_{0})^{\frac{1}{\alpha}}+\log 3+2\log
n+\frac{C_{3}s_{0}\log p_{n}}{\delta}$ $\displaystyle\lesssim
n\epsilon_{n}^{2}$

where in the first term of the second inequality, we are using the fact that
$\|\beta\|_{2}\leq\|\beta\|_{1}\leq\|\beta\|_{\alpha}\leq(C_{3}Es_{0})^{\frac{1}{\alpha}}$
with $0<\alpha<1$. We already show that
$\Pi_{n}\left(d(P_{\theta}^{n},P_{\theta_{0}}^{n})\geq M\varepsilon_{n}\mid
X_{1},\ldots,X_{n}\right)\rightarrow{0}$ in $P^{n}_{\theta_{0}}-$ probability.
Now we want to argue this implies the posterior consistency in terms of
Euclidean distance.
By lemma 3, we see that $d(P_{\theta}^{n},P_{\theta_{0}}^{n})\leq
M\varepsilon_{n}$ implies

$\frac{1}{2}\log\frac{\sigma^{2}+\sigma_{0}^{2}}{2\sigma\sigma_{0}}\leq
M^{2}\epsilon_{n}^{2},\quad\frac{\|X\beta-X\beta_{0}\|}{4n(\sigma^{2}+\sigma_{0}^{2})}\leq
M^{2}\epsilon_{n}^{2}.$

The first inequality above implies that

$2\leq\frac{\sigma}{\sigma_{0}}+\frac{\sigma_{0}}{\sigma}\leq
2e^{2M^{2}\epsilon_{n}^{2}}.$

For $n$ sufficient large, there exists $c$ such that
$\frac{\sigma}{\sigma_{0}}=1+c<e^{2M^{2}\epsilon_{n}^{2}}$, we have
$c<2M^{2}\epsilon_{n}^{2}$. This implies that $|\sigma-\sigma_{0}|\leq
2\sigma_{0}M^{2}\epsilon_{n}^{2}$. By assumption A4 and Theorem 3.2 for $n$
sufficient large, the second inequality implies $\|\beta-\beta_{0}\|_{2}\leq
5\sigma_{0}^{2}M^{2}\epsilon_{n}^{2}/v_{1}.$ Consequently, we have

$\displaystyle\Pi_{n}\left(\|\beta-\beta_{0}\|_{2}\gtrsim\varepsilon_{n}\mid
X_{1},\ldots,X_{n}\right)\rightarrow{0}$
$\displaystyle\Pi_{n}\left(|\sigma-\sigma_{0}|\gtrsim\sigma_{0}\varepsilon_{n}\mid
X_{1},\ldots,X_{n}\right)\rightarrow{0}$

in $P^{n}_{\theta_{0}}-$ probability. ∎

Proof of theorem 3.2

###### Proof.

We define the set of event
$Z_{n}=\left\\{\int\int\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}d\Pi(\beta)\Pi(\sigma^{2})\geq
e^{-(1+D+C_{1})n\epsilon_{n}^{2}}\right\\}$ for some positive constant $D$ and
$T=\left\\{\beta:|\gamma(\beta)|\leq{C_{3}s_{0}}\right\\}$. Then

$\Pi(T^{c}|Y)=\frac{\int\int_{T^{c}}\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}d\Pi(\beta)\Pi(\sigma^{2})}{\int\int\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}d\Pi(\beta)\Pi(\sigma^{2})}.$

We have that:

$P_{\theta_{0}}^{n}\Pi(T^{c}|Y)\leq
P_{\theta_{0}}^{n}\Pi(T^{c}|Y)1_{Z_{n}}+P_{\theta_{0}}^{n}(Z_{n}^{c})$

For the first term of right hand side, we have:

$P_{0}^{n}\Pi(T^{c}|Y)1_{Z_{n}}\leq
e^{(1+D+C_{1})n\epsilon_{n}^{2}}\int\int\int_{T^{c}}\frac{f_{\theta}^{n}}{f_{\theta_{0}}^{n}}d\Pi(\beta)\Pi(\sigma^{2})dP_{0}^{n}\leq{\int_{T^{c}}}\Pi(\beta)=\Pi(|\gamma(\beta)|\leq{C_{3}s_{0}})$

In part 2 of the proof of Theorem 3.1, we see that

$\Pi(|\gamma(\beta)|\leq{C_{3}s_{0}})\leq{\exp\left(-n\varepsilon_{n}^{2}(C_{1}+2)\right)}$

For the second term of the right hand side, by applying Lemma 7.4, we have

$P_{\theta_{0}}^{n}(Z_{n}^{c})\leq{\frac{1}{D^{2}n\epsilon_{n}^{2}}}.$

Combining the two results, we have:

$P_{\theta_{0}}^{n}\Pi(T^{c}|Y)\leq\exp\left(-n\varepsilon_{n}^{2}(C_{1}+2)\right)+\frac{1}{D^{2}n\epsilon_{n}^{2}}$

consequently, as $n\rightarrow\infty$,
$P_{\theta_{0}}^{n}\Pi(T^{c}|Y)\rightarrow 0$. ∎

## 7\. Convergence analysis

Theorem 5.3 will follow directly by the Lemmas below.

######  Lemma 7.5.

The sequence $\left\\{\beta^{i}\right\\}_{i\geq 1}$, returned by the CD
algorithm is a bounded sequence.

###### Proof.

Given the initial sequence $\beta^{1}$, then the entire sequence
$\left\\{\beta^{i}\right\\}_{i\geq 1}$ belongs to the sub-level set
$\left\\{\beta:L(\beta)\leq L(\beta^{1})\right\\}$. This is easy to see as the
convergence of loss function $L(\beta)$ is guaranteed by Theorem 5.1, which
shows that we can achieve the steepest descent in each coordinate. Since for
any $j\in\left\\{1,2,..,p\right\\}$,
$\log(|\beta_{j}|^{\frac{1}{2^{\gamma}}})\leq L(\beta)\leq L(\beta^{1})$, this
implies that $\beta_{j}^{(i)}$ is bounded for any
$j\in\left\\{1,2,..,p\right\\}$. This completes the proof. ∎

######  Lemma 7.6.

Define the descent function as:

$\triangle(\beta_{j}^{(i)},\beta_{j}^{(i+1)})=L_{-j}(\beta_{j}^{(i)})-L_{-j}(\beta_{j}^{(i+1)})$

where $L_{-j}(\cdot)$ is the loss function for $\beta_{j}$ with all other
parameters held fixed. Then $\triangle(\beta_{j}^{(i)},\beta_{j}^{(i+1)})=0$
if and only if $\beta_{j}^{(i)}=\beta_{j}^{(i+1)}$.

###### Proof.

We show that if $\triangle(\beta_{j}^{(i)},\beta_{j}^{(i+1)})=0$, then
$\beta_{j}^{(i)}=\beta_{j}^{(i+1)}$. For the reverse direction, the result is
trivial.
Case 1. $(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}|z_{j}^{(i)}|\geq
h(\tilde{\beta}_{j}^{(i)})\quad\text{and}\quad\delta_{-j}(\beta_{j}^{(i)^{\prime\prime}})<0$.
In this case, $\beta_{j}^{(i+1)}$ is the unique univariate global minimizer of
$L_{-j}(\cdot)$. Thus, $\triangle(\beta_{j}^{(i)},\beta_{j}^{(i+1)})=0$
implies $\beta_{j}^{(i)}=\beta_{j}^{(i+1)}$.(The uniqueness of global minimum
is shown in the step 3 and step 4 of the proof of theorem 5.1).
Case 2. $(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}|z_{j}^{(i)}|\geq
h(\tilde{\beta}_{j}^{(i)})\quad\text{and}\quad\delta_{-j}(\beta_{j}^{(i)^{\prime\prime}})=0$.
In this case, the unique univariate global minimum attains at both
$\operatorname{sign}(z_{j}^{(i)})\beta_{j}^{(i)^{\prime\prime}}$ and $0$. By
the rule of $T(\cdot)$ map (see Theorem 5.3), we have
$\beta_{j}^{(i+1)}=\operatorname{sign}(z_{j}^{(i)})\tilde{\beta}_{j}^{(i)}I(\beta_{j}^{(i)}\neq
0)$. If $\beta_{j}^{(i)}=0$, then $\beta_{j}^{(i+1)}=\beta_{j}^{(i)}=0$. If
$\beta_{j}^{(i)}\neq 0$, $\triangle(\beta_{j}^{(i)},\beta_{j}^{(i+1)})=0$
implies
$\beta_{j}^{(i)}=\beta_{j}^{(i+1)}=\operatorname{sign}(z_{j}^{(i)})\tilde{\beta}_{j}^{(i)}$.
Case 3.$(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}|z_{j}^{(i)}|\geq
h(\tilde{\beta}_{j}^{(i)})\quad\text{and}\quad\delta_{-j}(\beta_{j}^{(i)^{\prime\prime}})>0$.
In this case, $0$ is the unique univariate global minimizer of
$L_{-j}(\cdot)$. Thus, $\triangle(\beta_{j}^{(i)},\beta_{j}^{(i+1)})=0$
implies $\beta_{j}^{(i)}=\beta_{j}^{(i+1)}=0$.
Case 4
$(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}|z_{j}^{(i)}|<h(\tilde{\beta}_{j}^{(i)})$.
In this case, $0$ is the unique univariate global minimizer of
$L_{-j}(\cdot)$. Thus, $\triangle(\beta_{j}^{(i)},\beta_{j}^{(i+1)})=0$
implies $\beta_{j}^{(i)}=\beta_{j}^{(i+1)}=0$. ∎

######  Lemma 7.7.

Suppose there exists two convergent subsequences
$\left\\{\boldsymbol{\beta}^{n_{i}}\right\\}_{i}\subset\left\\{\boldsymbol{\beta}^{i}\right\\}_{i}$
and
$\left\\{\boldsymbol{\beta}^{n_{i}+1}\right\\}_{i}\subset\left\\{\boldsymbol{\beta}^{i}\right\\}_{i}$
such that $\boldsymbol{\beta}^{n_{i}}\rightarrow{\beta^{*}}$ and
$\boldsymbol{\beta}^{n_{i}+1}\rightarrow{\beta^{**}}$. Then their differences
converge to zero:
$\boldsymbol{\beta}^{n_{i}+1}-\boldsymbol{\beta}^{n_{i}}\rightarrow\beta_{j}^{**}-\beta_{j}^{*}=0.$

###### Proof.

First, we want to check the continuity of some functions. We write
$z_{j}=z(\beta_{-j})=X_{j}^{T}(Y-X\beta_{-j})$ and
$h(\tilde{\beta}_{j})=h(\tilde{\beta}_{j}(C_{2}(\beta_{-j})))$. Then the
continuity of $z(\beta_{-j})$ is clear.
We see that both
$C_{2}(\beta_{-j})=\|\beta_{-j}\|_{\frac{1}{2^{\gamma}}}^{{\frac{1}{2^{\gamma}}}}+1/b$
and
$h(\tilde{\beta}_{j})=\tilde{\beta}_{j}+(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}\frac{C_{1}}{\tilde{\beta}_{j}+C_{2}\tilde{\beta}_{j}^{(1-\frac{1}{2^{\gamma}})}}$
are continuous maps. From step 1 of the proof of theorem 5.1,
$\tilde{\beta}_{j}$ is the unique solution of the equation

$f_{1}(C_{2},\beta_{j})=1-(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}C_{1}\frac{1+(1-\frac{1}{2^{\gamma}})C_{2}\beta_{j}^{-\frac{1}{2^{\gamma}}}}{\beta_{j}^{2}\left(1+C_{2}\beta_{j}^{-\frac{1}{2^{\gamma}}}\right)^{2}}=0.$

Since $\frac{\partial
f_{1}}{\partial\beta_{j}}=\psi^{{}^{\prime\prime}}(\beta_{j})>0$, then by
implicit function theorem, $\tilde{\beta}_{j}(C_{2})$ is continuous.
Therefore, the the composition function
$h(\tilde{\beta}_{j}(C_{2}(\beta_{-j})))$ is continuous. Consequently, if
$\boldsymbol{\beta}^{n_{i}}\rightarrow{\beta^{*}}$, we also have
$z_{j}^{(n_{i})}\rightarrow{z_{j}^{(*)}}$ and
$h(\tilde{\beta}_{j}^{(n_{i})})\rightarrow{h(\tilde{\beta}_{j}^{(*)})}$.
Next, we want to show that $\beta_{j}^{**}=T(\beta_{-j}^{*},\beta_{j}^{*})$.
We focus on the $j$th coordinates for arbitrary
$j\in\left\\{1,2,..,p\right\\}$. Without loss of generality, we assume
$z_{j}^{*}\geq 0$. The result from the negative side can be obtained by
symmetry. We consider the problem in three cases:
Case 1.
$(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}^{(*)}<h(\tilde{\beta}_{j}^{(*)})$.
By convergence of the subsequence $\boldsymbol{\beta}^{n_{i}}$, there exists
sufficient large $n^{\prime}$ such that for any $i>n^{\prime}$, we have
$(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}^{(n_{i})}<h(\tilde{\beta}_{j}^{(n_{i})})$.
In this case, $\beta_{j}^{(n_{i}+1)}\equiv 0$. We have
$\beta_{j}^{**}=T(\beta_{-j}^{*},\beta_{j}^{*})=\beta_{j}^{*}$.

Case 2. $(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}^{(*)}\geq
h(\tilde{\beta}_{j}^{(*)})$. By convergence of the subsequence
$\boldsymbol{\beta}^{n_{i}}$, there exists sufficiently large $n^{\prime}$
such that for any $i>n^{\prime}$, we have
$(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}^{(n_{i})}\geq
h(\tilde{\beta}_{j}^{(n_{i})})$. By step 2 and step 3 of the proof of theorem
5.1, there exists local minimizer
${\beta_{j}^{\prime\prime}}\in(\tilde{\beta}_{j},(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}]$
and ${\beta_{j}^{\prime\prime}}$ is the unique solution of the equation

$f_{2}(\beta_{-j},\beta_{j})=\beta_{j}-(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}+(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}\frac{C_{1}}{\beta_{j}+C_{2}\beta_{j}^{1-\frac{1}{2^{\gamma}}}}=0.$

From the step 1 of the proof of theorem 5.1, we also see that $\frac{\partial
f_{2}}{\partial\beta_{j}}=\psi^{{}^{\prime}}(\beta_{j})>0$ for any
$\beta_{j}\in(\tilde{\beta}_{j},(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}]$,
by implicit function theorem, there exists continuous and differential
function $g(\beta_{-j})=\beta_{j}^{{}^{\prime\prime}}$ for
$z_{j}=X_{j}^{T}(Y-X\beta_{-j})\in((\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})h(\tilde{\beta}_{j}^{{}^{n_{i}}}),+\infty)$.
In addition, $g(\cdot)$ is one-to-one map. Hence, if we have
$\boldsymbol{\beta}_{-j}^{n_{i}}\rightarrow{\beta_{-j}^{*}}$ and
$(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}z_{j}^{(*)}\geq
h(\tilde{\beta}_{j}^{(*)})$, this implies
$g(\boldsymbol{\beta}_{-j}^{n_{i}})=\beta_{j}^{(n_{i})^{\prime\prime}}\rightarrow{g(\beta_{-j}^{*})}=\beta_{j}^{*^{\prime\prime}}$.
By continuity of $\delta_{-j}(\cdot)$, we also have

$\delta_{-j}(g(\boldsymbol{\beta}_{-j}^{n_{i}}))=\delta(\beta_{j}^{(n_{i})^{\prime\prime}})\rightarrow\delta_{-j}(g(\beta_{-j}^{*}))=\delta(\beta_{j}^{*^{\prime\prime}}).$

Now, we can further consider three sub cases:
a. $\delta_{-j}(\beta_{j}^{*^{\prime\prime}})<0$. There exists a sufficiently
large $n^{\prime\prime}$ such that for any $i>n^{\prime\prime}>n^{\prime}$, we
have $\delta_{-j}(\beta_{j}^{(n_{i})^{\prime\prime}})<0$. Then
$\beta_{j}^{(n_{i}+1)}=T(\beta_{-j}^{n_{i}},\beta_{j}^{n_{i}})=\operatorname{sign}(z_{j}^{(n_{i})})\beta_{j}^{(n_{i})^{\prime\prime}}$.
Taking the limit on both sides,
$\beta_{j}^{**}=\operatorname{sign}(z_{j}^{*})\beta_{j}^{*^{\prime\prime}}=T(\beta_{-j}^{*},\beta_{j}^{*})$.
b. $\delta_{-j}(\beta_{j}^{*^{\prime\prime}})>0$. There exists a sufficiently
large $n^{\prime\prime}$ such that for any $i>n^{\prime\prime}>n^{\prime}$, we
have $\delta_{-j}(\beta_{j}^{(n_{i})^{\prime\prime}})>0$. Then
$\beta_{j}^{(n_{i}+1)}=T(\beta_{-j}^{n_{i}},\beta_{j}^{n_{i}})=\beta_{j}^{n_{i}}=0$.
We have $\beta_{j}^{**}=T(\beta_{-j}^{*},\beta_{j}^{*})=0$.
c. $\delta_{-j}(\beta_{j}^{*^{\prime\prime}})=0$. There exists a sufficiently
large $n^{\prime\prime}$ such that for any $i>n^{\prime\prime}>n^{\prime}$,
$\delta_{-j}(\beta_{j}^{(n_{i})^{\prime\prime}})$ will approach zero from only
one side. Suppose this is not true, then
$\beta_{j}^{(n_{i}+1)}\rightarrow\left\\{0,\operatorname{sign}(z_{j}^{*})\beta_{j}^{*^{\prime\prime}}\right\\}$.
In other words, $\beta_{j}^{(n_{i}+1)}$ will converge to a limit set rather
than a limit point which contradicts the assumption in the lemma.
If $\delta_{-j}(\beta_{j}^{(n_{i})^{\prime\prime}})$ approach zero from the
right side. Then,

$\beta_{j}^{(n_{i}+1)}=T(\beta_{-j}^{n_{i}},\beta_{j}^{n_{i}})=\operatorname{sign}(z_{j}^{(n_{i})})\tilde{\beta}_{j}^{(n_{i})}I(\beta_{j}^{(n_{i})}\neq
0)=0=\beta_{j}^{n_{i}}.$

Therefore we have, $\beta_{j}^{**}=\beta_{j}^{*}=0$, which also implies
$\beta_{j}^{**}=T(\beta_{-j}^{*},\beta_{j}^{*})=0$.
If $\delta_{-j}(\beta_{j}^{(n_{i})^{\prime\prime}})$ approach zero from the
left side. Then,
$\beta_{j}^{(n_{i}+1)}=T(\beta_{-j}^{n_{i}},\beta_{j}^{n_{i}})=\operatorname{sign}(z_{j}^{(n_{i})})\beta_{j}^{(n_{i})^{\prime\prime}}\neq
0$ and
$\beta_{j}^{(n_{i})}=T(\beta_{-j}^{n_{i}-1},\beta_{j}^{n_{i}-1})=\operatorname{sign}(z_{j}^{(n_{i}-1)})\beta_{j}^{(n_{i}-1)^{\prime\prime}}\neq
0$. This implies that $\beta_{j}^{*}\neq 0$ and $\beta_{j}^{**}\neq 0$. By the
rule of $T(\cdot)$,

$T(\beta_{-j}^{*},\beta_{j}^{*})=\operatorname{sign}(z_{j}^{*})\beta_{j}^{*^{\prime\prime}}I(\beta_{j}^{*}\neq
0)=\operatorname{sign}(z_{j}^{*})\beta_{j}^{*^{\prime\prime}}.$

Taking the limit of
$\beta_{j}^{(n_{i}+1)}=\operatorname{sign}(z_{j}^{(n_{i})})\beta_{j}^{(n_{i})^{\prime\prime}}$,
we have
$\beta_{j}^{**}=\operatorname{sign}(z_{j}^{*})\beta_{j}^{*^{\prime\prime}}$.Therefore
$\beta_{j}^{**}=T(\beta_{-j}^{*},\beta_{j}^{*})$. Finally, we use Lemma 6 to
finish the proof. By continuity of descent function, we have

$\triangle(\beta_{j}^{(n_{i})},\beta_{j}^{(n_{i}+1)})\rightarrow\triangle(\beta_{j}^{*},\beta_{j}^{**})$

since Theorem 5.1 guarantees the non-increasing of the loss function,
$\triangle(\beta_{j}^{(n_{i})},\beta_{j}^{(n_{i}+1)})$ is decreasing and
bounded below by zero. Therefore,

$\triangle(\beta_{j}^{(n_{i})},\beta_{j}^{(n_{i}+1)})\rightarrow\triangle(\beta_{j}^{*},\beta_{j}^{**})=0.$

We have already shown that $\beta_{j}^{**}=T(z_{j}^{*},\beta_{j}^{*})$, then
by Lemma 6, $\triangle(\beta_{j}^{*},\beta_{j}^{**})=0$ if and only if
$\beta_{j}^{*}=\beta_{j}^{**}$. ∎

Proof of Theorem 5.4

###### Proof.

$\displaystyle L(\beta^{\infty}+\alpha e)-L(\beta^{\infty})$
$\displaystyle=\frac{1}{2}\alpha^{2}\|Xe\|_{2}^{2}-\alpha
e^{T}X^{T}(Y-X\beta^{\infty})+$
$\displaystyle(2^{\gamma}P+0.5)\left[\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}+\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+1/b\right)-\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+1/b\right)\right].$

We decompose the perturbation $e$ into two disjoint set:
$e=e_{\hat{S}}+e_{\hat{S}^{c}}$ where $e_{\hat{S}}=eI(j\in\hat{S})$ and
$e_{\hat{S}^{c}}=eI(j\in\hat{S}^{c})$. Then, we have

$\displaystyle\|Xe\|_{2}^{2}$
$\displaystyle=\|Xe_{\hat{S}}+Xe_{\hat{S}^{c}}\|_{2}^{2}$
$\displaystyle\geq\|Xe_{\hat{S}}\|_{2}^{2}-2\|X^{T}X\|\|e_{\hat{S}}\|_{2}\|e_{\hat{S}^{c}}\|_{2}+\|Xe_{\hat{S}^{c}}\|_{2}^{2}$
$\displaystyle\geq\|Xe_{\hat{S}}\|_{2}^{2}-2\|X^{T}X\|\|e_{\hat{S}^{c}}\|_{2}+\|Xe_{\hat{S}^{c}}\|_{2}^{2}$
$\displaystyle\geq\|Xe_{\hat{S}}\|_{2}^{2}-2\|X^{T}X\|\|e_{\hat{S}^{c}}\|_{1}+\|Xe_{\hat{S}^{c}}\|_{2}^{2}$
$\displaystyle\geq\|Xe_{\hat{S}}\|_{2}^{2}-2\|X^{T}X\|\|e_{\hat{S}^{c}}\|_{1}$
$\displaystyle\geq
nv_{1}\|e_{\hat{S}}\|_{2}^{2}-2\|X^{T}X\|\|e_{\hat{S}^{c}}\|_{1}$

where the last inequality is followed by assumption A4. By mean value theorem,
there exists $t\in(0,1)$ such that

$\log(C+x)=\log(C)+\frac{x}{C+tx}.$

Plug in $C=\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}+\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+1/b$ and $x=\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}$, we have

$\displaystyle\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}+\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+1/b\right)-\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+1/b\right)$
$\displaystyle=$
$\displaystyle\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}+\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+1/b\right)-\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+1/b\right)+\frac{\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}}{\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}+\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+t\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+1/b}$ $\displaystyle\geq$
$\displaystyle\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}+\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+1/b\right)-\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+1/b\right)+\frac{\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}}{2\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+2/b}$

where the last inequality holds for $\alpha$ sufficiently small. Combining all
the upper bounds we obtained above, we have

$\frac{L(\beta^{\infty}+\alpha
e)-L(\beta^{\infty})}{\alpha}\geq\psi_{1}(\alpha)+\psi_{2}(\alpha)$

where

$\displaystyle\psi_{1}(\alpha)=$
$\displaystyle-\alpha\|X^{T}X\|\|e_{\hat{S}^{c}}\|_{1}-e_{\hat{S}^{c}}^{T}X_{\hat{S}^{c}}^{T}(Y-X\beta^{\infty})+\frac{1}{\alpha}\frac{(2^{\gamma}P+0.5)\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}}{2\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+2/b}$
$\displaystyle\psi_{2}(\alpha)=$
$\displaystyle\frac{nv_{1}\alpha}{2}\|e_{\hat{S}}\|_{2}^{2}-e_{\hat{S}}^{T}X_{\hat{S}}^{T}(Y-X\beta^{\infty})+$
$\displaystyle\frac{2^{\gamma}P+0.5}{\alpha}\left[\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}+\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}+1/b\right)-\log\left(\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+1/b\right)\right].$

Now we treat each term separately,

$\displaystyle\psi_{1}(\alpha)$
$\displaystyle\geq-\alpha\|X^{T}X\|\sum_{j\in\hat{S}^{c}}|e_{j}|-\sum_{j\in\hat{S}^{c}}|e_{j}||z_{j}^{\infty}|+\frac{1}{\alpha}\frac{(2^{\gamma}P+0.5)\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}}{2\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+2/b}$
$\displaystyle\geq-\sum_{j\in\hat{S}^{c}}(\|X^{T}X\|+h_{j}^{\infty})|e_{j}|+\frac{1}{\alpha}\frac{(2^{\gamma}P+0.5)\sum_{j\in{\hat{S}}}|\alpha
e_{j}|^{\frac{1}{2^{\gamma}}}}{2\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+2/b}$
$\displaystyle>0\quad\text{if and only if
}\quad\alpha<\min_{j\in\hat{S}^{c}}\left\\{\frac{1}{|e_{j}|}\left(\frac{\lambda}{\|X^{T}X\|+h_{j}^{\infty}}\right)^{\frac{1}{1-\frac{1}{2^{\gamma}}}}\right\\}$

where $z_{j}^{\infty}=X_{j}^{T}(Y-X\beta^{\infty})$,
$\lambda=\frac{(2^{\gamma}P+0.5)}{2\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+2/b}$
and the second inequality follows by Theorem 5.1. the KKT condition. Again, by
KKT condition in Theorem 5.1, we have

$\lim_{\alpha\rightarrow
0}\psi_{2}(\alpha)=-e_{\hat{S}}^{T}X_{\hat{S}}^{T}(Y-X\beta^{\infty})+\frac{P+\frac{1}{2^{\gamma+1}}}{\sum_{j\in{\hat{S}}}|\beta_{j}^{\infty}|^{\frac{1}{2^{\gamma}}}+1/b}\sum_{j\in{\hat{S}}}\frac{e_{j}}{|\beta_{j}^{\infty}|^{1-\frac{1}{2^{\gamma}}}}=0.$

∎

## 8\. Oracle properties

Proof of Theorem 5.5
We only show the case (2):
$\lim_{n\rightarrow{\infty}}\frac{p_{n}}{ns_{0}}=\infty$, $b=O\left(\frac{\log
p_{n}}{p_{n}}\right)$ and assumption A9 holds. The argument for case (1) is
exactly the same, so we will omit the details.

###### Proof.

Let
$\hat{\beta}^{T}=\left([(X_{s_{0}}^{T}X_{s_{0}})^{-1}X_{s_{0}}^{T}y]^{T},0^{T}\right)$
be the oracle estimator. We want to show that, with probability tending to 1,
the oracle estimator $\hat{\beta}$ is a strictly local minimizer of
$L(\cdot)$. By theorem 5.1, it suffices to show that, with probability tending
to 1, $\hat{\beta}$ satisfies the KKT conditions:

$\displaystyle(X_{j}^{T}X_{j})^{-1}|z_{j}|<h(\tilde{\beta}_{j})\quad\text{for}\quad
j\in{S_{0}^{c}}$
$\displaystyle(X_{j}^{T}X_{j})^{-1}|z_{j}|>h(\tilde{\beta}_{j})\quad\text{for}\quad
j\in{S_{0}}$
$\displaystyle\beta_{j}=\left[(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}|z_{j}|-(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}\frac{C_{1}}{|\beta_{j}|+C_{2}|\beta_{j}|^{1-\frac{1}{2^{\gamma}}}}\right]\mathrm{sign}(z_{j})\quad\text{for}\quad
j\in{S_{0}}$ $\displaystyle L_{-j}(\beta_{j})<L_{-j}(0)\quad\text{for}\quad
j\in{S_{0}}$

plug in
$\hat{\beta}^{T}=\left([(X_{s_{0}}^{T}X_{s_{0}})^{-1}X_{s_{0}}^{T}y]^{T},0^{T}\right)$
into the KKT condition, we see that

$(X_{j}^{T}X_{j})^{-1}|z_{j}|=\left\\{\begin{array}[]{ll}\left|\beta_{0,j}+\sigma_{0}[(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon]_{j}\right|&\text{
if }j\in{S_{0}}\\\
\left|\frac{\sigma_{0}X_{j}^{T}\epsilon}{n}-\sigma_{0}\frac{X_{j}^{T}X_{S_{0}}}{n}(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon\right|&\text{
if }j\in{S_{0}^{c}}\end{array}\right.$ (30)

by mean value theorem, we have

$\displaystyle\sum_{j\in S_{0}}|\hat{\beta}_{j}|^{\frac{1}{2^{\gamma}}}=$
$\displaystyle\sum_{j\in
S_{0}}|\beta_{0,j}+\sigma_{0}[(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon]_{j}|^{\frac{1}{2^{\gamma}}}$
(31) $\displaystyle=$ $\displaystyle\sum_{j\in
S_{0}}|\beta_{0,j}|^{\frac{1}{2^{\gamma}}}+\frac{\sigma_{0}}{2^{\gamma}}\sum_{j\in
S_{0}}\frac{[(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon]_{j}}{|\beta_{0,j}+\sigma_{0}t_{j}[(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon]_{j}|^{1-\frac{1}{2^{\gamma}}}}$

from some $t=(t_{1},...,t_{p})$ such that $t_{j}\in(0,1)$. Consider the events

$\displaystyle\mathcal{A}_{1}$
$\displaystyle=\left\\{\left\|\frac{X_{s_{0}}^{T}\epsilon}{n}\right\|_{\infty}\leq
2\sqrt{\frac{2\log s_{0}}{n}}\right\\}$ $\displaystyle\mathcal{A}_{2}$
$\displaystyle=\left\\{\left\|\frac{X_{s_{0}^{c}}^{T}\epsilon}{n}\right\|_{\infty}\leq
2\sqrt{\frac{2\log p_{n}}{n}}\right\\}.$

By standard Gaussian tail bounds (Wainwright,, 2019), we have

$\begin{aligned}
&\lim_{n\rightarrow\infty}P\left(\left\|\frac{X_{s_{0}}^{T}\epsilon}{n}\right\|_{\infty}\geq
2\sqrt{\frac{2\log s_{n}}{n}}\right)\leq 2\lim_{n\rightarrow\infty}e^{-\log
s_{n}}=0\\\
&\lim_{n\rightarrow\infty}P\left(\left\|\frac{X_{s_{0}^{c}}^{T}\epsilon}{n}\right\|_{\infty}\geq
2\sqrt{\frac{2\log(p_{n}-s_{n})}{n}}\right)\leq
2\lim_{n\rightarrow\infty}e^{-\log(p_{n}-s_{n})}=0\end{aligned}.$

By assumption A4, $X_{S_{0}}^{T}X_{S_{0}}$ is invertible, then under event
$\mathcal{A}_{1}$, we have

$\displaystyle\|(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon\|_{\infty}$
$\displaystyle\leq\|(X_{S_{0}}^{T}X_{S_{0}})^{-1}\|_{\infty}\|X_{S_{0}}^{T}\epsilon\|_{\infty}$
$\displaystyle\leq\sqrt{s_{0}}\|(X_{s_{0}}^{T}X_{S_{0}})^{-1}\|_{2}\|X_{S_{0}}^{T}\epsilon\|_{\infty}$
$\displaystyle\leq\sqrt{\frac{s_{0}}{v_{1}}}\left\|\frac{X_{s_{0}}^{T}\epsilon}{n}\right\|_{\infty}$
$\displaystyle\leq 2\sqrt{\frac{2s_{0}\log s_{0}}{nv_{1}}}$ $\displaystyle\leq
2\sqrt{\frac{2\log s_{0}}{v_{1}\log p_{n}}}$

where the last inequality is followed by assumption A2. In addition, for $n$
sufficient large, under event $\mathcal{A}_{1}$ and assumption A5, we have
$\tilde{\beta}_{j}<\hat{|\beta_{j}|}=(X_{j}^{T}X_{j})^{-1}|z_{j}|<2E$, which
implies that $\tilde{\beta}_{j}^{\frac{1}{2^{\gamma}}}+C_{2}=O(\sum_{j\in
S_{0}}|\hat{\beta}_{j}|^{\frac{1}{2^{\gamma}}}+b^{-1})$. By Lemma 5.2, we have

$h(\tilde{\beta}_{j})=O\left(\left(\frac{p+\frac{1}{2^{\gamma+1}}}{n\tilde{\beta}_{j}^{\frac{1}{2^{\gamma}}}+nC_{2}}\right)^{\frac{1}{2-\frac{1}{2^{\gamma}}}}\right)=O\left(\left(\frac{p}{ns_{0}+np_{n}/\log
p_{n}}\right)^{\frac{1}{2-\frac{1}{2^{\gamma}}}}\right)=O\left(\left(\frac{\log
p_{n}}{n}\right)^{\frac{1}{2-\frac{1}{2^{\gamma}}}}\right)$

where $\sum_{j\in S_{0}}|\hat{\beta}_{j}|^{\frac{1}{2^{\gamma}}}=O(\sum_{j\in
S_{0}}|\beta_{0,j}|^{\frac{1}{2^{\gamma}}})=O(s_{0})$ by Equation 31 and
assumption A6. Define the event

$\mathcal{A}=\mathcal{A}_{1}\cap\mathcal{A}_{2}\cap\left\\{\left\|\frac{X_{S_{0}^{c}}^{T}X_{S_{0}}}{n}\right\|_{2,\infty}\leq
c\right\\}$

then by assumption A7, $\lim_{n\rightarrow\infty}P(\mathcal{A})=1$. Therefore,
it suffices to show that for sufficiently large $n$ under the event
$\mathcal{A}$ the KKT condition will statisfy. For $j\in S_{0}^{c}$ and $n$
sufficient large, under event $\mathcal{A}$, we have

$\displaystyle\quad\max_{j\in
S_{0}^{c}}\left|\frac{\sigma_{0}X_{j}^{T}\epsilon}{n}-\sigma_{0}\frac{X_{j}^{T}X_{S_{0}}}{n}(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon\right|$
$\displaystyle\leq\sigma_{0}\left\|\frac{X_{s_{0}^{c}}^{T}\epsilon}{n}\right\|_{\infty}+\sigma_{0}\left\|\frac{X_{S_{0}^{c}}^{T}X_{S_{0}}}{n}\right\|_{2,\infty}\left\|(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon\right\|_{2}$
$\displaystyle\leq 2\sigma_{0}\sqrt{\frac{2\log
p_{n}}{n}}+\sigma_{0}c\sqrt{\mathrm{Trace}((X_{S_{0}}^{T}X_{S_{0}})^{-1})}$
$\displaystyle\leq 2\sigma_{0}\sqrt{\frac{2\log
p_{n}}{n}}+\sigma_{0}c\sqrt{\frac{s_{0}}{nv_{1}}}$ $\displaystyle\leq
M\sigma_{0}\sqrt{\frac{\min(\log p_{n},s_{0})}{n}}$ $\displaystyle\leq
M\sigma_{0}\sqrt{\frac{\log p_{n}}{n}}$ $\displaystyle\leq
M\sigma_{0}\left(\frac{\log
p_{n}}{n}\right)^{\frac{1}{2-\frac{1}{2^{\gamma}}}}$
$\displaystyle=O(h(\tilde{\beta}_{j})).$

For $j\in S_{0}$ and $n$ sufficient large, under event $\mathcal{A}$, we have

$\displaystyle\quad\min_{j\in
S_{0}}\left|\beta_{0,j}+\sigma_{0}[(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon]_{j}\right|$
$\displaystyle\geq\min_{j\in
S_{0}}|\beta_{0,j}|-\sigma_{0}\|(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon\|_{\infty}$
$\displaystyle\geq\min_{j\in
S_{0}}|\beta_{0,j}|-\sigma_{0}\|(X_{S_{0}}^{T}X_{S_{0}})^{-1}\|_{\infty}\|X_{S_{0}}^{T}\epsilon\|_{\infty}$
$\displaystyle\geq\min_{j\in
S_{0}}|\beta_{0,j}|-\sigma_{0}\sqrt{s_{0}}\|(X_{s_{0}}^{T}X_{S_{0}})^{-1}\|_{2}\|X_{S_{0}}^{T}\epsilon\|_{\infty}$
$\displaystyle\geq\min_{j\in
S_{0}}|\beta_{0,j}|-\sigma_{0}\sqrt{\frac{s_{0}}{v_{1}}}\left\|\frac{X_{s_{0}}^{T}\epsilon}{n}\right\|_{\infty}$
$\displaystyle\geq\min_{j\in
S_{0}}|\beta_{0,j}|-2\sigma_{0}\sqrt{\frac{2s_{0}\log s_{0}}{nv_{1}}}$
$\displaystyle\geq\min_{j\in S_{0}}|\beta_{0,j}|-2\sigma_{0}\sqrt{\frac{2\log
s_{0}}{v_{1}\log p_{n}}}$ $\displaystyle>\sigma_{0}\left(\frac{\log
p_{n}}{n}\right)^{\frac{1}{2-\frac{1}{2^{\gamma}}}}$
$\displaystyle=O(h(\tilde{\beta}_{j}))$

where the last inequality is followed by assumption A8. Next, we want to check
that, for $j\in S_{0}$ and $n$ sufficient large, under event $\mathcal{A}$, we
have

$\hat{\beta_{j}}=\left[(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}|z_{j}|-(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}\frac{C_{1}}{|\hat{\beta}_{j}|+\hat{C_{2}}|\hat{\beta}_{j}|^{1-\frac{1}{2^{\gamma}}}}\right]\mathrm{sign}(z_{j})$
(32)

where $\hat{C}_{2}=\sum_{j\in
S_{0}}|\hat{\beta}_{j}|^{\frac{1}{2^{\gamma}}}+1/b$. Since
$(\boldsymbol{X}_{j}^{T}\boldsymbol{X}_{j})^{-1}|z_{j}|=\left|\beta_{0,j}+\sigma_{0}[(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T}\epsilon]_{j}\right|=|\hat{\beta}_{j}|$,
it suffices to show

$\lim_{n\rightarrow{\infty}}\frac{C_{1}}{n|\hat{\beta}_{j}|+\hat{C_{2}}n|\hat{\beta}_{j}|^{1-\frac{1}{2^{\gamma}}}}=0.$

We see that under event $\mathcal{A}_{1}$,
$\lim_{n\rightarrow\infty}\hat{\beta}_{j}=\beta_{0,j}$, thus it suffices to
show

$\lim_{n\rightarrow{\infty}}\frac{p_{n}}{n|\beta_{0,j}|+C_{0,2}n|\beta_{0,j}|^{1-\frac{1}{2^{\gamma}}}}=0$

where $C_{0,2}=\sum_{j\in S_{0}}|\beta_{0,j}|^{\frac{1}{2^{\gamma}}}+1/b$. By
assumption A6, $\lim_{n\rightarrow{\infty}}\frac{p_{n}}{ns_{0}}=\infty$ and
$b=O\left(\frac{\log p_{n}}{p_{n}}\right)$, we have:

$\frac{p_{n}}{n|\beta_{0,j}|+C_{0,2}n|\beta_{0,j}|^{1-\frac{1}{2^{\gamma}}}}\leq\frac{p_{n}}{n(\min_{j\in
S_{0}}|\beta_{0,j}|s_{0}+cp_{n}/\log p_{n})}=O\left(\frac{\log
p_{n}}{n}\right)\rightarrow{0}\quad\text{if}\quad n\rightarrow\infty$

where $c$ is some positive constant. Finally, we want to check that, for $j\in
S_{0}$ and $n$ sufficient large, under event $\mathcal{A}$, we have
$L_{-j}(\beta_{j})<L_{-j}(0)$.

$\delta_{-j}(|\hat{\beta}_{j}|)=L_{-j}(\hat{\beta_{j}})-L_{-j}(0)=\frac{(X_{j}^{T}X_{j})}{2}\hat{\beta_{j}}^{2}-|z_{j}||\hat{\beta_{j}}|+(2^{\gamma}p+0.5)\log\left(1+\frac{|\hat{\beta_{j}}|^{\frac{1}{2^{\gamma}}}}{C_{2}}\right).$
(33)

Since $(X_{j}^{T}X_{j})^{-1}|z_{j}|>h(\tilde{\beta}_{j})$ for $j\in{S_{0}}$,
plug in (8.3) into (8.4), then we have

$\delta_{-j}(|\hat{\beta}_{j}|)=-\frac{(X_{j}^{T}X_{j})}{2}{\hat{\beta}_{j}}^{2}-\frac{C_{1}}{1+C_{2}\hat{\beta}_{j}^{-\frac{1}{2^{\gamma}}}}+(2^{\gamma}p+0.5)\log\left(1+\frac{\hat{\beta}_{j}^{\frac{1}{2^{\gamma}}}}{C_{2}}\right).$

We see that to show $\delta_{-j}(|\hat{\beta}_{j}|)<0$ , for $n$ sufficient
large and under event $\mathcal{A}$, that is equivalent to show

$\frac{(2^{\gamma}p+0.5)}{n}\log\left(1+\frac{\hat{\beta}_{j}^{\frac{1}{2^{\gamma}}}}{C_{2}}\right)<\frac{1}{2}{\hat{\beta}_{j}}^{2}$

By assumptions A5,A6 and $b\propto\frac{\log p_{n}}{p_{n}}$, taking
$n\rightarrow{\infty}$, for the left hand side, we have

$\frac{(2^{\gamma}p+0.5)}{n}\log\left(1+\frac{\hat{\beta}_{j}^{\frac{1}{2^{\gamma}}}}{C_{2}}\right)=O\left(\frac{\log
p_{n}}{n}\right)\rightarrow{0},$

while for the right hand side, $\frac{1}{2}{\hat{\beta}_{j}}^{2}=O(1)$. The
result follows immediately. Now we show the asymptotic normality of
$\hat{\sigma}^{2}-\sigma_{0}^{2}$. Under the event $\mathcal{A}$, as
$n\rightarrow\infty$

$\hat{\sigma}^{2}-\sigma_{0}^{2}\rightarrow\frac{\sigma_{0}^{2}}{n-s_{0}}(I_{n}-X_{S_{0}}(X_{S_{0}}^{T}X_{S_{0}})^{-1}X_{S_{0}}^{T})-\sigma_{0}^{2}.$

Hence, we have

$\frac{\hat{\sigma}^{2}-\sigma_{0}^{2}}{\sqrt{n}}\stackrel{{\scriptstyle\mathcal{D}}}{{\rightarrow}}N(0,\sigma_{0}^{4}E[\epsilon^{4}]-\sigma_{0}^{4}).$

∎

## References

  * Andrews and Mallows, (1974) Andrews, D. F. and Mallows, C. L. (1974).  Scale mixtures of normal distributions.  Journal of the Royal Statistical Society: Series B (Methodological), 36(1):99–102.
  * Armagan, (2009) Armagan, A. (2009).  Variational bridge regression.  In Artificial Intelligence and Statistics, pages 17–24.
  * Bhattacharya et al., (2016) Bhattacharya, A., Chakraborty, A., and Mallick, B. K. (2016).  Fast sampling with gaussian scale mixture priors in high-dimensional regression.  Biometrika, page asw042.
  * Bhattacharya et al., (2015) Bhattacharya, A., Pati, D., Pillai, N. S., and Dunson, D. B. (2015).  Dirichlet–laplace priors for optimal shrinkage.  Journal of the American Statistical Association, 110(512):1479–1490.
  * Bühlmann and Van De Geer, (2011) Bühlmann, P. and Van De Geer, S. (2011).  Statistics for high-dimensional data: methods, theory and applications.  Springer Science & Business Media.
  * Carvalho et al., (2010) Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010).  The horseshoe estimator for sparse signals.  Biometrika, 97(2):465–480.
  * Casella, (2001) Casella, G. (2001).  Empirical bayes gibbs sampling.  Biostatistics, 2(4):485–500.
  * Castillo et al., (2015) Castillo, I., Schmidt-Hieber, J., Van der Vaart, A., et al. (2015).  Bayesian linear regression with sparse priors.  Annals of Statistics, 43(5):1986–2018.
  * Datta et al., (2013) Datta, J., Ghosh, J. K., et al. (2013).  Asymptotic properties of bayes risk for the horseshoe prior.  Bayesian Analysis, 8(1):111–132.
  * Devroye, (2009) Devroye, L. (2009).  Random variate generation for exponentially and polynomially tilted stable distributions.  ACM Transactions on Modeling and Computer Simulation (TOMACS), 19(4):1–20.
  * Dicker, (2014) Dicker, L. H. (2014).  Variance estimation in high-dimensional linear models.  Biometrika, 101(2):269–284.
  * Fan et al., (2012) Fan, J., Guo, S., and Hao, N. (2012).  Variance estimation using refitted cross-validation in ultrahigh dimensional regression.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(1):37–65.
  * Fan and Lv, (2008) Fan, J. and Lv, J. (2008).  Sure independence screening for ultrahigh dimensional feature space.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(5):849–911.
  * Fan and Lv, (2011) Fan, J. and Lv, J. (2011).  Nonconcave penalized likelihood with np-dimensionality.  IEEE Transactions on Information Theory, 57(8):5467–5484.
  * Frank and Friedman, (1993) Frank, L. E. and Friedman, J. H. (1993).  A statistical view of some chemometrics regression tools.  Technometrics, 35(2):109–135.
  * Friedman et al., (2010) Friedman, J., Hastie, T., and Tibshirani, R. (2010).  Regularization paths for generalized linear models via coordinate descent.  Journal of statistical software, 33(1):1.
  * Friedman, (2012) Friedman, J. H. (2012).  Fast sparse regression and classification.  International Journal of Forecasting, 28(3):722–738.
  * Gelman et al., (2013) Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., and Rubin, D. B. (2013).  Bayesian data analysis.  CRC press.
  * Ghosal and Van der Vaart, (2017) Ghosal, S. and Van der Vaart, A. (2017).  Fundamentals of nonparametric Bayesian inference, volume 44.  Cambridge University Press.
  * Ghosal et al., (2007) Ghosal, S., Van Der Vaart, A., et al. (2007).  Convergence rates of posterior distributions for noniid observations.  The Annals of Statistics, 35(1):192–223.
  * Gilks and Wild, (1992) Gilks, W. R. and Wild, P. (1992).  Adaptive rejection sampling for gibbs sampling.  Journal of the Royal Statistical Society. Series C (Applied Statistics), 41(2):337–348.
  * Huang et al., (2008) Huang, J., Horowitz, J. L., Ma, S., et al. (2008).  Asymptotic properties of bridge estimators in sparse high-dimensional regression models.  The Annals of Statistics, 36(2):587–613.
  * Knight et al., (2000) Knight, K., Fu, W., et al. (2000).  Asymptotics for lasso-type estimators.  The Annals of statistics, 28(5):1356–1378.
  * Lee et al., (2014) Lee, J. D., Sun, Y., and Saunders, M. A. (2014).  Proximal newton-type methods for minimizing composite functions.  SIAM Journal on Optimization, 24(3):1420–1443.
  * Makalic and Schmidt, (2015) Makalic, E. and Schmidt, D. F. (2015).  A simple sampler for the horseshoe estimator.  IEEE Signal Processing Letters, 23(1):179–182.
  * Mallick and Yi, (2018) Mallick, H. and Yi, N. (2018).  Bayesian bridge regression.  Journal of applied statistics, 45(6):988–1008.
  * Marjanovic and Solo, (2012) Marjanovic, G. and Solo, V. (2012).  On $l\\_q$ optimization and matrix completion.  IEEE Transactions on signal processing, 60(11):5714–5724.
  * Marjanovic and Solo, (2013) Marjanovic, G. and Solo, V. (2013).  On exact l q denoising.  In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 6068–6072. IEEE.
  * Marjanovic and Solo, (2014) Marjanovic, G. and Solo, V. (2014).  $l\\_${$q$} sparsity penalized linear regression with cyclic descent.  IEEE Transactions on Signal Processing, 62(6):1464–1475.
  * Mazumder et al., (2011) Mazumder, R., Friedman, J. H., and Hastie, T. (2011).  Sparsenet: Coordinate descent with nonconvex penalties.  Journal of the American Statistical Association, 106(495):1125–1138.
  * Moran et al., (2018) Moran, G. E., Ročková, V., George, E. I., et al. (2018).  Variance prior forms for high-dimensional bayesian variable selection.  Bayesian Analysis, pages 1091–1119.
  * Ning et al., (2020) Ning, B., Jeong, S., Ghosal, S., et al. (2020).  Bayesian linear regression for multivariate responses under group sparsity.  Bernoulli, 26(3):2353–2382.
  * Park and Casella, (2008) Park, T. and Casella, G. (2008).  The bayesian lasso.  Journal of the American Statistical Association, 103(482):681–686.
  * Park and Van Dyk, (2009) Park, T. and Van Dyk, D. A. (2009).  Partially collapsed gibbs samplers: Illustrations and applications.  Journal of Computational and Graphical Statistics, 18(2):283–305.
  * Polson and Scott, (2012) Polson, N. G. and Scott, J. G. (2012).  Local shrinkage rules, lévy processes and regularized regression.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(2):287–311.
  * Polson et al., (2013) Polson, N. G., Scott, J. G., and Windle, J. (2013).  Bayesian inference for logistic models using pólya–gamma latent variables.  Journal of the American statistical Association, 108(504):1339–1349.
  * Polson et al., (2014) Polson, N. G., Scott, J. G., and Windle, J. (2014).  The bayesian bridge.  Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76(4):713–733.
  * Reid et al., (2016) Reid, S., Tibshirani, R., and Friedman, J. (2016).  A study of error variance estimation in lasso regression.  Statistica Sinica, pages 35–67.
  * Ročková and George, (2016) Ročková, V. and George, E. I. (2016).  Bayesian penalty mixing: the case of a non-separable penalty.  In Statistical Analysis for High-Dimensional Data, pages 233–254. Springer.
  * Ročková and George, (2018) Ročková, V. and George, E. I. (2018).  The spike-and-slab lasso.  Journal of the American Statistical Association, 113(521):431–444.
  * Scott and Berger, (2010) Scott, J. G. and Berger, J. O. (2010).  Bayes and empirical-bayes multiplicity adjustment in the variable-selection problem.  The Annals of Statistics, pages 2587–2619.
  * Song and Liang, (2017) Song, Q. and Liang, F. (2017).  Nearly optimal bayesian shrinkage for high dimensional regression.  arXiv preprint arXiv:1712.08964.
  * Sun and Zhang, (2012) Sun, T. and Zhang, C.-H. (2012).  Scaled sparse linear regression.  Biometrika, 99(4):879–898.
  * Van Dyk and Jiao, (2015) Van Dyk, D. A. and Jiao, X. (2015).  Metropolis-hastings within partially collapsed gibbs samplers.  Journal of Computational and Graphical Statistics, 24(2):301–327.
  * Van Dyk and Park, (2008) Van Dyk, D. A. and Park, T. (2008).  Partially collapsed gibbs samplers: Theory and methods.  Journal of the American Statistical Association, 103(482):790–796.
  * Wainwright, (2019) Wainwright, M. J. (2019).  High-dimensional statistics: A non-asymptotic viewpoint, volume 48.  Cambridge University Press.
  * Wang, (2009) Wang, H. (2009).  Forward regression for ultra-high dimensional variable screening.  Journal of the American Statistical Association, 104(488):1512–1524.
  * West, (1987) West, M. (1987).  On scale mixtures of normal distributions.  Biometrika, 74(3):646–648.
  * Yu and Moyeed, (2001) Yu, K. and Moyeed, R. A. (2001).  Bayesian quantiles regression.  Statistics and Probability Letters, 54:437–447.
  * Zhang et al., (2010) Zhang, C.-H. et al. (2010).  Nearly unbiased variable selection under minimax concave penalty.  The Annals of statistics, 38(2):894–942.
  * Zou and Li, (2008) Zou, H. and Li, R. (2008).  One-step sparse estimates in nonconcave penalized likelihood models.  Annals of statistics, 36(4):1509.