$\tilde{B}^{L}_{\gamma{\tilde{\gamma}}^{*}}=n^{(p_{{\tilde{\gamma}}^{*}}-p_{\gamma})/2}O_{p}(1),$

as we wished to prove.

### Part (ii), ${\bm{\tilde{}}{B}^{L}_{\gamma,{\tilde{\gamma}}^{*}}}$

Suppose now that ${\tilde{\gamma}}^{*}\not\subseteq\gamma$. Then by Lemma S1

$\displaystyle\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}}\pm\tilde{\beta}_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}=b_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}^{T}X_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}^{T}X_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}b_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}-b_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}b_{\gamma\setminus\gamma^{\prime}},$

where we set $\gamma^{\prime}={\tilde{\gamma}}^{*}\cup\gamma$ to be the model
such that $Z_{\gamma^{\prime}}$ includes all columns from
$Z_{{\tilde{\gamma}}^{*}}$ and $Z_{\gamma}$. Note that Lemma S1 applies since
both ${\tilde{\gamma}}^{*}\subseteq\gamma^{\prime}$ and
$\gamma\subseteq\gamma^{\prime}$. Now,
${\tilde{\gamma}}^{*}\subseteq\gamma^{\prime}$ implies that
$\tilde{\beta}^{*}_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}=0$, applying
the same argument as in (S.13) gives that
$b_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}^{T}X_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}^{T}X_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}b_{\gamma^{\prime}\setminus{\tilde{\gamma}}^{*}}=O_{p}(1)$.
Finally consider the term
$b_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}b_{\gamma\setminus\gamma^{\prime}}$.
Analogously to (S.13) let
$W=b_{\gamma\setminus\gamma^{\prime}}^{T}S_{\gamma\setminus\gamma^{\prime}}^{-1}b_{\gamma\setminus\gamma^{\prime}}$
and note that

$\displaystyle\lambda_{1}W\leq
b_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}S_{\gamma\setminus\gamma^{\prime}}S_{\gamma\setminus\gamma^{\prime}}^{-1}b_{\gamma\setminus\gamma^{\prime}}\leq\lambda_{2}W$

where now $(\lambda_{1},\lambda_{2})$ denote the smallest and largest
eigenvalues of
$X_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}S_{\gamma\setminus\gamma^{\prime}}$.
Now from (S.12) we have that

$\sqrt{n}(b_{\gamma\setminus\gamma^{\prime}}-\tilde{\beta}^{*}_{\gamma\setminus\gamma^{\prime}})\stackrel{{\scriptstyle
D}}{{\longrightarrow}}\mathcal{N}(0,S_{\gamma\setminus\gamma^{\prime}}),$

where $\tilde{\beta}^{*}_{\gamma\setminus\gamma^{\prime}}\neq 0$. Therefore by
Slutky’s theorem $W\stackrel{{\scriptstyle
D}}{{\longrightarrow}}\chi^{2}_{p_{\gamma^{\prime}}-p_{\gamma}}(l(\gamma^{\prime},\gamma))$
with non-zero non-centrality parameter
$l(\gamma^{\prime},\gamma)=n(\tilde{\beta}^{*}_{\gamma^{\prime}\setminus\gamma})^{T}S_{\gamma^{\prime}\setminus\gamma}\tilde{\beta}^{*}_{\gamma^{\prime}\setminus\gamma}\neq
0$. Note that $l(\gamma^{\prime},\gamma)$ can be interpreted as the increase
in expected sum of squares in $\tilde{y}$ explained by
$\gamma^{\prime}={\tilde{\gamma}}^{*}\cup\gamma$, relative to that explained
by $\gamma$ alone. Therefore, from (S.10) we obtain

$\displaystyle\frac{1}{n}\log\tilde{B}^{L}_{\gamma{\tilde{\gamma}}^{*}}$
$\displaystyle=\frac{b^{\prime\prime}(0)}{n\phi}[O_{p}(1)-b_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}^{T}X_{\gamma\setminus\gamma^{\prime}}b_{\gamma\setminus\gamma^{\prime}}]<\frac{b^{\prime\prime}(0)}{n\phi}[O_{p}(1)-\lambda_{1}W]$
$\displaystyle\stackrel{{\scriptstyle
P}}{{\longrightarrow}}\phi^{-1}b^{\prime\prime}(0)\lambda_{1}(\tilde{\beta}^{*}_{\gamma^{\prime}\setminus\gamma})^{T}S_{\gamma^{\prime}\setminus\gamma}\tilde{\beta}^{*}_{\gamma^{\prime}\setminus\gamma}>0,$

as we wished to prove.

### Part (i), ${\bm{\tilde{}}{B}^{N}_{\gamma,{\tilde{\gamma}}^{*}}}$

First note that $\mbox{tr}(V_{j}S_{\gamma j})/p_{j}=O_{p}(1/n)$, since $S_{j}$
is a sub-matrix of $(Z_{\gamma}^{T}D_{\gamma}Z_{\gamma})^{-1}$ and we showed
earlier that $Z_{\gamma}^{T}D_{\gamma}Z_{\gamma}/n\stackrel{{\scriptstyle
P}}{{\longrightarrow}}\Sigma_{\gamma z}$, where $\Sigma_{\gamma z}$ is a
strictly positive-definite matrix, and similarly
$V_{j}=Z_{j}^{T}Z_{j}(p_{j}+2)/(np_{j}g_{N})$ converges to a finite positive-
definite matrix. The same reasoning implies that
$\mbox{tr}(V_{j}S_{{\tilde{\gamma}}^{*}j})/p_{j}=O_{p}(1/n)$.

Second, we showed that
$\tilde{\beta}_{{\tilde{\gamma}}^{*}j}\stackrel{{\scriptstyle
P}}{{\longrightarrow}}\tilde{\beta}_{j}^{*}\neq 0$, $\tilde{\beta}_{\gamma
j}\stackrel{{\scriptstyle P}}{{\longrightarrow}}\tilde{\beta}_{j}^{*}\neq 0$
for truly active $j\in{\tilde{\gamma}}^{*}$, and $\tilde{\beta}_{\gamma
j}=O_{p}(1/n)$ for $j\in\gamma\setminus{\tilde{\gamma}}^{*}$. This immediately
implies that

$\displaystyle\frac{\prod_{\gamma_{j}=1}\mbox{tr}(V_{j}S_{\gamma
j})/p_{j}+\tilde{\beta}_{\gamma j}^{T}V_{j}\tilde{\beta}_{\gamma j}/(\phi
p_{j})}{\prod_{{\tilde{\gamma}}^{*}_{j}=1}\mbox{tr}(V_{j}S_{{\tilde{\gamma}}^{*}j})/p_{j}+\tilde{\beta}_{\gamma
j}^{T}V_{j}\tilde{\beta}_{\gamma j}/(\phi p_{j})}=$
$\displaystyle\frac{\prod_{j\in\gamma\setminus{\tilde{\gamma}}^{*}}O_{p}(1/n)+O_{p}(1/n)\prod_{{\tilde{\gamma}}^{*}_{j}=1}O_{p}(1/n)+[(\tilde{\beta}_{j}^{*})^{T}V_{j}\tilde{\beta}_{j}^{*}+o_{p}(1)]/(\phi
p_{j})}{\prod_{{\tilde{\gamma}}^{*}_{j}=1}O_{p}(1/n)+[(\tilde{\beta}_{j}^{*})^{T}V_{j}\tilde{\beta}_{j}^{*}+o_{p}(1)]/(\phi
p_{j})}=O_{p}(n^{-(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}),$

since $(\tilde{\beta}_{j}^{*})^{T}V_{j}\tilde{\beta}_{j}^{*}$ is a non-zero
constant, as we wished to prove.

### Part (ii), ${\bm{\tilde{}}{B}^{N}_{\gamma,{\tilde{\gamma}}^{*}}}$

Arguing as above,

$\displaystyle\frac{\prod_{\gamma_{j}=1}\mbox{tr}(V_{j}S_{\gamma
j})/p_{j}+\tilde{\beta}_{\gamma j}^{T}V_{j}\tilde{\beta}_{\gamma j}/(\phi
p_{j})}{\prod_{{\tilde{\gamma}}^{*}_{j}=1}\mbox{tr}(V_{j}S_{{\tilde{\gamma}}^{*}j})/p_{j}+\tilde{\beta}_{\gamma
j}^{T}V_{j}\tilde{\beta}_{\gamma j}/(\phi
p_{j})}=\frac{\prod_{\gamma_{j}=1}O_{p}(1/n)+\tilde{\beta}_{\gamma
j}^{T}V_{j}\tilde{\beta}_{\gamma j}/(\phi
p_{j})}{\prod_{{\tilde{\gamma}}^{*}_{j}=1}[(\tilde{\beta}_{j}^{*})^{T}V_{j}\tilde{\beta}_{j}^{*}+o_{p}(1)]/(\phi
p_{j})}=O_{p}(n^{-|\tilde{\beta}^{*}_{\gamma}|_{0}})$

where
$|\tilde{\beta}^{*}_{\gamma}|_{0}=\sum_{\gamma_{j}=1}\mbox{I}(\tilde{\beta}_{\gamma
j}^{*})$ is the number of parameters with optimal value equal to 0. Hence

$\displaystyle\frac{1}{n}\log\tilde{B}^{N}_{\gamma,{\tilde{\gamma}}^{*}}=\frac{1}{n}\log\tilde{B}^{L}_{\gamma,{\tilde{\gamma}}^{*}}+o_{p}(1),$

proving the desired result.

##  S8. Proof of Corollary 1

The proof strategy is as follows. First, note that
$\tilde{\phi}_{\gamma}=\phi_{0}-H_{0}^{-1}g_{0}$ converges in probability to a
finite non-zero constant, since $\phi_{0}\stackrel{{\scriptstyle
P}}{{\longrightarrow}}\phi_{0}^{*}$ by Lemma S2,
$g_{0}/n\stackrel{{\scriptstyle P}}{{\longrightarrow}}g_{0}^{*}$ and
$H_{0}/n\stackrel{{\scriptstyle P}}{{\longrightarrow}}H_{0}^{*}$ for some
fixed non-zero $g_{0}^{*}$, $H_{0}^{*}$. Arguing as in the proof of Theorem 2
shows that

$\displaystyle\tilde{B}^{N}_{\gamma{\tilde{\gamma}}^{*}}=\tilde{B}^{L}_{\gamma{\tilde{\gamma}}^{*}}\frac{\prod_{\gamma_{j}=1}\mbox{tr}(V_{j}S_{\gamma
j})/p_{j}+\frac{\tilde{\beta}_{\gamma j}^{T}V_{j}\tilde{\beta}_{\gamma
j}}{\tilde{\phi}_{\gamma}p_{j}}}{\prod_{{\tilde{\gamma}}^{*}_{j}=1}\mbox{tr}(V_{j}S_{{\tilde{\gamma}}^{*}j})/p_{j}+\frac{\tilde{\beta}_{{\tilde{\gamma}}^{*}j}^{T}V_{j}\tilde{\beta}_{{\tilde{\gamma}}^{*}j}}{\tilde{\phi}_{\tilde{\gamma}}^{*}p_{j}}}$

is equal to
$\tilde{B}^{L}_{\gamma{\tilde{\gamma}}^{*}}O_{p}(n^{-(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})})$
when ${\tilde{\gamma}}^{*}\in\gamma$, and equal to
$\tilde{B}^{L}_{\gamma{\tilde{\gamma}}^{*}}O_{p}(n^{-|\tilde{\beta}_{\tilde{\gamma}}^{*}|_{0}})$
when ${\tilde{\gamma}}^{*}\in\gamma$. These expressions for
$\tilde{B}^{N}_{\gamma{\tilde{\gamma}}^{*}}$ are the same as those in Theorem
2 where $\phi$ was assumed known, hence it suffices to show that
$\tilde{B}^{L}_{\gamma{\tilde{\gamma}}^{*}}$ also attains the same rates as in
Theorem 2. To prove this, we use algebraic manipulation and auxiliary Lemma S2
to show that the leading term in the Bayes factor exponent for unknown $\phi$
converges to a constant multiple of that in the known $\phi$ case, which
immediately gives the desired result.

Specifically, recall that

$\displaystyle\tilde{B}^{L}_{\gamma,\gamma^{\prime}}=e^{\frac{b^{\prime\prime}(0)}{2\phi_{0}}[t_{\gamma}\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-t_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}]}(2\pi)^{\frac{p_{\gamma}-p_{\gamma^{\prime}}}{2}}\frac{|H_{\gamma^{\prime}0}|^{\frac{1}{2}}p^{L}(\tilde{\beta}_{\gamma},\tilde{\phi}_{\gamma}\mid\gamma)}{|H_{\gamma
0}|^{\frac{1}{2}}p^{L}(\tilde{\beta}_{\gamma^{\prime}},\tilde{\phi}_{\gamma^{\prime}}\mid\gamma)}$

where

$\displaystyle H_{\gamma 0}$
$\displaystyle=\frac{b^{\prime\prime}(0)}{\phi_{0}}\begin{pmatrix}Z_{\gamma}^{T}Z_{\gamma}&-Z_{\gamma}^{T}\tilde{y}/\phi_{0}\\\
-\tilde{y}^{T}Z_{\gamma}/\phi_{0}&s(\phi_{0})\end{pmatrix}$ $\displaystyle
t_{\gamma}$
$\displaystyle=1+\frac{\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}}{\phi_{0}^{2}(s(\phi_{0})-\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma})}$
$\displaystyle s(\phi_{0})$
$\displaystyle=[2nb(0)/\phi_{0}^{2}+\phi_{0}\sum_{i=1}^{n}\nabla_{\phi\phi}^{2}c(y_{i},\phi_{0})]/b^{\prime\prime}(0).$

Arguing as in the proof of Theorem 2 gives that

$\displaystyle\frac{|H_{\gamma^{\prime}0}|^{\frac{1}{2}}}{|H_{\gamma
0}|^{\frac{1}{2}}}=n^{\frac{(p_{\gamma^{\prime}}-p_{\gamma})}{2}}\frac{|n^{-1}H_{\gamma^{\prime}0}|^{\frac{1}{2}}}{|n^{-1}H_{\gamma
0}|^{\frac{1}{2}}}=n^{\frac{(p_{\gamma^{\prime}}-p_{\gamma})}{2}}\left(\frac{\phi_{0}}{b^{\prime\prime}(0)}\right)^{\frac{p_{\gamma}-p_{\gamma^{\prime}}}{2}}\left[\frac{|\Sigma_{z,\gamma^{\prime}}|^{\frac{1}{2}}}{|\Sigma_{z,\gamma}|^{\frac{1}{2}}}+o_{p}(1)\right],$

where $\Sigma_{z,\gamma^{\prime}}$ and $\Sigma_{z,\gamma}$ are fixed and
strictly positive-definite matrices. This gives an expression analogous to
(S.10), specifically

(S.14)
$\displaystyle\frac{\tilde{B}^{L}_{\gamma\gamma^{\prime}}}{n^{(p_{\gamma^{\prime}}-p_{\gamma})/2}}\left(\frac{b^{\prime\prime}(0)}{2\pi\phi_{0}}\right)^{\frac{p_{\gamma}-p_{\gamma^{\prime}}}{2}}\frac{|\Sigma_{z,\gamma}|^{\frac{1}{2}}p(\tilde{\beta}_{\gamma^{\prime}}^{*},\tilde{\phi}_{\gamma^{\prime}}^{*}\mid\gamma^{\prime})}{|\Sigma_{z,\gamma^{\prime}}|^{\frac{1}{2}}p(\tilde{\beta}_{\tilde{\gamma}}^{*},\tilde{\phi}_{\tilde{\gamma}}^{*}\mid\gamma)}e^{-\frac{b^{\prime\prime}(0)}{2\phi_{0}}(t_{\gamma}\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-t_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}})}\stackrel{{\scriptstyle
P}}{{\longrightarrow}}1,$

where

$\left(\frac{b^{\prime\prime}(0)}{2\pi\phi_{0}}\right)^{(p_{\gamma}-p_{\gamma^{\prime}})/2}\frac{|\Sigma_{z,\gamma}|^{\frac{1}{2}}p(\tilde{\beta}_{\gamma^{\prime}}^{*},\tilde{\phi}_{\gamma^{\prime}}\mid\gamma^{\prime})}{|\Sigma_{z,\gamma^{\prime}}|^{\frac{1}{2}}p(\tilde{\beta}_{\tilde{\gamma}}^{*},\tilde{\phi}_{\gamma}\mid\gamma)}$

converges in probability to a finite non-zero constant that does not depend on
$n$, since $\phi_{0}\stackrel{{\scriptstyle
P}}{{\longrightarrow}}\phi_{0}^{*}$ by Lemma S2 and
$p(\beta_{\gamma},\phi_{\gamma}\mid\gamma)>0$ is continuous by Condition (C3).

Now, set $\gamma^{\prime}={\tilde{\gamma}}^{*}$. The term in the exponent of
(S.14) can be written

$\displaystyle
t_{\gamma}\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-t_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}=t_{\gamma}(\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}})+(t_{\gamma}-t_{{\tilde{\gamma}}^{*}})\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}}$

To complete the proof we show that the second term in the right-hand side is
$O_{p}(1)$, then proceed as in the proof of Theorem 2 to conclude the proof.
Specifically, the asymptotic normality of
$\tilde{\beta}_{{\tilde{\gamma}}^{*}}$ implies that
$\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}}=O_{p}(1)$
(see proof of Theorem 2). Further, in

$\displaystyle
t_{\gamma}=1+\frac{\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}/n}{\phi_{0}^{2}(s(\phi_{0})/n-\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}/n)}$

the term
$\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}/n\stackrel{{\scriptstyle
P}}{{\longrightarrow}}(\tilde{\beta}_{\tilde{\gamma}}^{*})^{T}\Sigma_{\gamma,z}\tilde{\beta}_{\tilde{\gamma}}^{*}>0$
converges in probability to a finite constant by the strong law of large
numbers if $\tilde{\beta}_{\tilde{\gamma}}^{*}\neq 0$, and
$\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}/n=O_{p}(1/n)$
if $\tilde{\beta}_{\tilde{\gamma}}^{*}=0$. Further,

$\displaystyle\frac{s(\phi_{0})}{n}=\frac{1}{b^{\prime\prime}(0)}\left[\frac{2b(0)}{\phi_{0}^{2}}+\frac{\phi_{0}}{n}\sum_{i=1}^{n}\nabla_{\phi\phi}^{2}c(y_{i},\phi_{0})\right]\stackrel{{\scriptstyle
P}}{{\longrightarrow}}\frac{1}{b^{\prime\prime}(0)}\left[\frac{2b(0)}{\phi_{0}^{2}}+\phi_{0}E_{F^{*}}[\nabla_{\phi\phi}^{2}c(y_{i},\phi_{0})]\right]>0.$

Hence by the continuous mapping theorem $t_{\gamma}$ converges in probability
to a finite constant, so does $t_{\gamma}-t_{{\tilde{\gamma}}^{*}}$, implying
that
$(t_{\gamma}-t_{{\tilde{\gamma}}^{*}})\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}}=O_{p}(1)$
and

$\displaystyle
e^{-\frac{b^{\prime\prime}(0)}{2\phi_{0}}(t_{\gamma}\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-t_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}})}=e^{-\frac{b^{\prime\prime}(0)t_{\gamma}}{2\phi_{0}}(\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}})}\times
O_{p}(1)$

The proof is completed by noting that $b^{\prime\prime}(0)t_{\gamma}/\phi_{0}$
converges in probability to a constant and proceeding as in the proof of
Theorem 2, after (S.10), to characterize the exponential term
$\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}}$
for the cases ${\tilde{\gamma}}^{*}\subset\gamma$ and
${\tilde{\gamma}}^{*}\not\subseteq\gamma$.

##  S9. Auxiliary results on sub-Gaussian random vectors

This section contains results on sub-Gaussian random vectors. Although their
main interest for the current work is as auxiliary results used in the proof
of Theorem 3, they have some independent interest in bounding integrals
related to Bayes factors that can bound posterior model probabilities in more
general settings beyond the ALA framework considered in Theorem 3.

We first recall the definition of a sub-Gaussian random vector and the basic
property that linear combinations of sub-Gaussian vectors are sub-Gaussian
(Lemma S3). Lemma S4 proves a useful property that certain quadratic forms of
$n$-dimensional sub-Gaussian vectors can be re-expressed as a quadratic form
of $d$-dimensional sub-Gaussian vectors.

A second set of results in Lemma S5 and Lemma S6 bound right tail and left
tail probabilities (respectively) involving quadratic forms of sub-Gaussian
random vectors. Lemma S5 is an adaptation of Theorem 1 in Hsu et al. (2012).
The interpretation is that, the probability that $s^{T}s$ exceeds a threshold
$\sigma^{2}dq$ that is larger than its expectation $E(s^{T}s)\leq\sigma^{2}d$,
decreases exponentially in $dq/2(1+k_{0})$, where $k_{0}$ is of order
$1/\sqrt{q}$ as $q$ grows, and hence $k_{0}$ becomes arbitrarily close to 0
for large $q$. The obtained bound is analogous to those for quadratic forms of
Gaussian random variables. Regarding Lemma S6, the interpretation is that the
probability that $s^{T}s$ is less than a threshold $a$ that is smaller than
the non-centrality parameter $\mu^{T}\mu$ decreases exponentially in
$-\mu^{T}\mu/\sigma^{2}$.

The last set of results are Proposition 1 and Proposition 2. Both are novel
results. Proposition 1 provides a finite-sample bound for an integral
involving tail probabilities as in Lemma S5. Finally, Proposition 2 provides a
bound for a similar integral involving differences between quadratic forms of
central and non-central sub-Gaussian vectors. In both propositions one should
think of $c$ as being a constant and $h$ being a large number, e.g. in ALA
Bayes factors $h$ typically grows logarithmically in $n$.

If Bayes factors can be expressed (or bounded by) functions involving sub-
Gaussian quadratic forms, as is the case for ALA to Bayes factors, then these
propositions allow bounding the expectation of the posterior probability
assigned to any given model (see Section S10). Such bounds on posterior
probabilities in turn bound their $L_{1}$ convergence rate to 0, which in turn
implies strong frequentist properties for the posterior probabilities, see
Rossell (2018).

We first state the results, and subsequently provide the proofs.

######  Definition 1.

A $d$-dimensional random vector $s=(s_{1},\ldots,s_{d})$ follows a sub-
Gaussian distribution with parameters $\mu\in\mathbb{R}^{d}$ and
$\sigma^{2}>0$, which we denote by $s\sim\mbox{SG}(\mu,\sigma^{2})$ if and
only if

$E\left[\exp\\{\alpha^{T}(s-\mu)\\}\right]\leq\exp\\{\alpha^{T}\alpha\sigma^{2}/2\\}$

for all $\alpha\in\mathbb{R}^{d}$.

######  Lemma S3.

Let $s\sim\mbox{SG}(\mu,\sigma^{2})$ be a sub-Gaussian $d$-dimensional random
vector, and $A$ be a $q\times d$ matrix. Then
$As\sim\mbox{SG}(A\mu,\lambda\sigma^{2})$, where $\lambda$ is the largest
eigenvalue of $A^{T}A$, or equivalently the largest eigenvalue of $AA^{T}$.

######  Lemma S4.

Let $y\sim\mbox{SG}(\mu,\sigma^{2})$ be an $n$-dimensional sub-Gaussian random
vector. Let $W=(y-a)^{T}X(X^{T}X)^{-1}X^{T}(y-a)$, where $X$ is an $n\times d$
matrix such that $X^{T}X$ is invertible. Then $W=s^{T}s$, where
$s\sim\mbox{SG}((X^{T}X)^{-1/2}X^{T}(\mu-a),\sigma^{2})$ is $d$-dimensional.

######  Lemma S5.

Central sub-Gaussian quadratic forms. Right-tail probabilities Let
$s=(s_{1},\ldots,s_{d})\sim\mbox{SG}(0,\sigma^{2})$. Then

  1. (i)

For any $t>0$,

$\displaystyle
P\left(\frac{s^{T}s}{\sigma^{2}}>dt[1+\sqrt{2/t}+1/t]\right)\leq\exp\left\\{-\frac{dt}{2}\right\\}.$

  2. (ii)

For any $q>0$ and any $k_{0}$ such that
$k_{0}\geq\frac{(1+k_{0})}{q}+\sqrt{\frac{2(1+k_{0})}{q}}$,

$\displaystyle
P\left(\frac{s^{T}s}{\sigma^{2}}>dq\right)\leq\exp\left\\{-\frac{dq}{2(1+k_{0})}\right\\}$

  3. (iii)

For any $q\geq 2(1+\sqrt{2})^{2}$,

$\displaystyle
P\left(\frac{s^{T}s}{\sigma^{2}}>dq\right)\leq\exp\left\\{-\frac{dq}{2(1+k_{0})}\right\\}$

where $k_{0}\geq\sqrt{2}(1+\sqrt{2})/\sqrt{q}$.

######  Lemma S6.

Non-central sub-Gaussian quadratic forms. Left-tail probabilities Let
$s=(s_{1},\ldots,s_{d})\sim\mbox{SG}(\mu,\sigma^{2})$. Then

$\displaystyle
P(s^{T}s<a)\leq\exp\left\\{-\frac{\mu^{T}\mu}{8\sigma^{2}}\left(1-\frac{a}{\mu^{T}\mu}\right)^{2}\right\\}.$

######  Proposition 1.

Let $s=(s_{1},\ldots,s_{d})\sim\mbox{SG}(0,\sigma^{2})$ and

$\displaystyle
U(d,h,c)=\int_{0}^{1}P\left(\frac{s^{T}s}{c}>d\log\left(\frac{h}{(1/u-1)^{2/d}}\right)\right)du.$

  1. (i)

Denote $h_{0}=e^{2(1+\sqrt{2})^{2}}$. Suppose that $(c,h)$ satisfy
$\log(h)\geq h_{0}$ and $c\geq\sigma^{2}(1+\sqrt{\sigma^{2}/[c\log h]})$. Then

$\displaystyle U(d,h,c)\leq\frac{2\max\left\\{[\log
h]^{d/2},\log(h^{d/2})\right\\}}{h^{d/2}}$

In particular, the result holds for $c\geq\sigma^{2}(1+\sqrt{1/\log
h})\Leftrightarrow\log h\geq(c/\sigma^{2}-1)^{-2}$.

  2. (ii)

Denote
$h_{0}=\max\\{[\sigma^{2}(1+\epsilon)/c]e^{2(1+\sqrt{2})^{2}},e^{2(1+\sqrt{2})^{2}\sigma^{2}/c}\\}$.
Suppose that $(c,h)$ satisfy $\log(h)\geq h_{0}$ and
$c<\sigma^{2}(1+\sqrt{\sigma^{2}/[c\log h]})$. Then

$\displaystyle
U(d,h,c)\leq\frac{2.5\max\left\\{\left[\log(h^{\frac{c}{\sigma^{2}(1+\epsilon)}})\right]^{d/2},\log\left(h^{\frac{dc}{2\sigma^{2}(1+\epsilon)}}\right)\right\\}}{h^{\frac{dc}{2\sigma^{2}(1+\epsilon\sqrt{2}(1+\sqrt{2}))}}}$

where $\epsilon=\sqrt{\sigma^{2}/[c\log h]}$.

######  Proposition 2.

Let $s_{1}\sim\mbox{SG}(0,\sigma^{2})$ be a $d_{1}$-dimensional and
$s_{2}\sim\mbox{SG}(\mu,\sigma^{2})$ be a $d_{2}$-dimensional sub-Gaussian
random vector where $\sigma^{2}>0$ is finite and $\mu\in\mathbb{R}^{d_{2}}$.
Define

$\displaystyle
U(d_{1},d_{2},c,d,h)=\int_{0}^{1}P\left(\frac{s_{1}^{T}s_{1}-s_{2}^{T}s_{2}}{c}>d\log\left(\frac{h}{(1/u-1)^{2/d}}\right)\right)du$

where $c>0$, $d\in\mathbb{R}$ and $h\geq 1$. Assume that
$\mu^{T}\mu/\log(\mu^{T}\mu)\geq-cd\log(h)+cd_{1}\log(h_{0})$, where $h_{0}$
is the positive constant in Proposition 1.

  1. (i)

Suppose that $c>\sigma^{2}$. Then, there exists a finite $t_{0}$ such that

$\displaystyle
U(d_{1},d_{2},c,d,h)\leq\frac{3\max\left\\{[\frac{2}{d_{1}}\log(l(\mu))]^{d_{1}/2},\log(l(\mu))\right\\}}{l(\mu)}$

for all $\mu^{T}\mu\geq t_{0}$, where
$l(\mu)=h^{d/2}e^{\frac{\mu^{T}\mu}{2c\log(\mu^{T}\mu)}}$. Hence, for any
fixed $a<1$ it holds that
$\lim_{\mu^{T}\mu\rightarrow\infty}U(d_{1},d_{2},c,d,h)/l(\mu)^{a}=0$.

  2. (ii)

Suppose that $c\leq\sigma^{2}$. Then, there exists a finite $t_{0}$ such that

$\displaystyle
U(d_{1},d_{2},c,d,h)\leq\frac{3.5\max\left\\{[\frac{2}{d_{1}}\log(l^{\prime}(\mu))^{1/(1+\epsilon)}]^{d_{1}/2},\log(l^{\prime}(\mu)^{1/(1+\epsilon)})\right\\}}{l^{\prime}(\mu)^{1/(1+\epsilon)}},$

for all $\mu^{T}\mu\geq t_{0}$ and
$\epsilon\geq\sqrt{2}(1+\sqrt{2})\sigma^{2}d_{1}/[c\mu^{T}\mu/\log(\mu^{T}\mu)+d\log(h)]$,
where
$l^{\prime}(\mu)=h^{d/2}e^{\frac{\mu^{T}\mu}{2\sigma^{2}\log(\mu^{T}\mu)}}$.

Hence, for any fixed $a<1$ it holds that
$\lim_{\mu^{T}\mu\rightarrow\infty}U(d_{1},d_{2},c,d,h)/l^{\prime}(\mu)^{a}=0$.

### Proof of Proposition 2

Denote by $w=d\log(h/[1/u-1]^{2/d})$ and let $w^{\prime}>0$ be an arbitrary
number, the union bound gives

$\displaystyle
P\left(\frac{s_{1}^{T}s_{1}-s_{2}^{T}s_{2}}{c}>w\right)=P\left(\frac{s_{1}^{T}s_{1}-s_{2}^{T}s_{2}}{c}>\frac{w}{2}+w^{\prime}-(w^{\prime}-\frac{w}{2})\right)$
$\displaystyle\leq
P\left(\frac{s_{1}^{T}s_{1}}{c}>\frac{w}{2}+w^{\prime}\right)+P\left(\frac{s_{2}^{T}s_{2}}{c}<w^{\prime}-\frac{w}{2}\right).$

We shall take
$w^{\prime}=0.5\log(h^{d}/[1/u-1]^{2}e^{2\mu^{T}\mu/[c\log(\mu^{T}\mu)]})$, so
that $w^{\prime}-\frac{w}{2}=\mu^{T}\mu/[c\log(\mu^{T}\mu)$ and

$w^{\prime}+\frac{w}{2}=d\log\left(\frac{h}{(1/u-1)^{2/d}}\right)+\frac{\mu^{T}\mu}{c\log(\mu^{T}\mu)}.$

Applying Lemma S6 immediately gives that

(S.15)
$\displaystyle\int_{0}^{1}P\left(\frac{s_{2}^{T}s_{2}}{c}<w^{\prime}-\frac{w}{2}\right)du\leq\exp\left\\{-\frac{\mu^{T}\mu}{8\sigma^{2}}\left(1-\frac{1}{\log(\mu^{T}\mu)}\right)^{2}\right\\}.$

Next consider

(S.16)
$\displaystyle\int_{0}^{1}P\left(\frac{s_{1}^{T}s_{1}}{c}>\frac{w}{2}+w^{\prime}\right)du=\int_{0}^{1}P\left(\frac{s_{1}^{T}s_{1}}{c}>d_{1}\log\left(\frac{h^{\frac{d}{d_{1}}}e^{\mu^{T}\mu/[cd_{1}\log(\mu^{T}\mu)]}}{(1/u-1)^{2/d_{1}}}\right)\right)du.$

This integral is in the form required by Proposition 1. That proposition
requires the condition that

$\log(h^{\frac{d}{d_{1}}}e^{\mu^{T}\mu/[cd_{1}\log(\mu^{T}\mu)]})\geq
h_{0}\Leftrightarrow\frac{\mu^{T}\mu}{\log(\mu^{T}\mu)}\geq-
cd\log(h)+cd_{1}\log(h_{0})$

where $h_{0}$ is the finite constant given in Proposition 1 defined separately
for the cases $c>\sigma^{2}$ and $c\leq\sigma^{2}$. Since this condition holds
by assumption, we may apply Proposition 1.

Consider first the case $c>\sigma^{2}$. Proposition 1 Part (i) gives that
(S.16) is

(S.17)
$\displaystyle\leq\frac{2\max\left\\{[\frac{2}{d_{1}}\log(l(\mu))]^{d_{1}/2},\log(l(\mu))\right\\}}{l(\mu)},$

where $l(\mu)=h^{d/2}e^{\mu^{T}\mu/[2c\log(\mu^{T}\mu)]}$. Combining (S.15)
and (S.17) gives

$\displaystyle U(d_{1},d_{2},c,d,h)\leq
e^{-\frac{\mu^{T}\mu}{8\sigma^{2}}\left(1-\frac{1}{\log(\mu^{T}\mu)}\right)^{2}}+\frac{2\max\left\\{[\frac{2}{d_{1}}\log(l(\mu))]^{d_{1}/2},\log(l(\mu))\right\\}}{l(\mu)}.$

Noting that the second term is larger than the first term as $\mu^{T}\mu$
grows gives that there exists $t_{0}$ such that, for all $\mu^{T}\mu>t_{0}$,

$\displaystyle
U(d_{1},d_{2},c,d,h)\leq\frac{3\max\left\\{[\frac{2}{d_{1}}\log(l(\mu))]^{d_{1}/2},\log(l(\mu))\right\\}}{l(\mu)},$

as we wished to prove.

Consider now the case $c\leq\sigma^{2}$. Proposition 1 Part (ii) and simple
algebra gives that (S.16) is

(S.18)
$\displaystyle\leq\frac{2.5\max\left\\{[\frac{2}{d_{1}}\log(l^{\prime}(\mu))^{1/(1+\epsilon)}]^{d_{1}/2},\log(l^{\prime}(\mu)^{1/(1+\epsilon)})\right\\}}{l^{\prime}(\mu)^{1/(1+\epsilon\sqrt{2}(1+\sqrt{2}))}}$

where
$l^{\prime}(\mu)=h^{dc/2\sigma^{2}}e^{\mu^{T}\mu/[2\sigma^{2}\log(\mu^{T}\mu)]}$,
and $\epsilon=\sigma^{2}d_{1}/[c\mu^{T}\mu/\log(\mu^{T}\mu)+d\log(h)]$.
Combining (S.15) and (S.18) gives

$\displaystyle U(d_{1},d_{2},c,d,h)\leq
e^{-\frac{\mu^{T}\mu}{8\sigma^{2}}\left(1-\frac{1}{\log(\mu^{T}\mu)}\right)^{2}}+\frac{2.5\max\left\\{[\frac{2}{d_{1}}\log(l^{\prime}(\mu))^{1/(1+\epsilon)}]^{d_{1}/2},\log(l^{\prime}(\mu)^{1/(1+\epsilon)})\right\\}}{l^{\prime}(\mu)^{1/(1+\epsilon\sqrt{2}(1+\sqrt{2}))}}.$

Noting that the first term is smaller than the second term as $\mu^{T}\mu$
grows gives that there exists $t_{0}$ such that, for all $\mu^{T}\mu>t_{0}$,

$\displaystyle
U(d_{1},d_{2},c,d,h)\leq\frac{3.5\max\left\\{[\frac{2}{d_{1}}\log(l^{\prime}(\mu))^{1/(1+\epsilon^{\prime})}]^{d_{1}/2},\log(l^{\prime}(\mu)^{1/(1+\epsilon^{\prime})})\right\\}}{l^{\prime}(\mu)^{1/(1+\epsilon^{\prime})}},$

where
$\epsilon^{\prime}\geq\sqrt{2}(1+\sqrt{2})\sigma^{2}d_{1}/[c\mu^{T}\mu/\log(\mu^{T}\mu)+d\log(h)]$.

### Proof of Lemma S3

Consider an arbitrary $\alpha\in\mathbb{R}^{d}$, and let
$\tilde{\alpha}=A^{T}\alpha$. Then

$\displaystyle
E\left(e^{\alpha^{T}(As-A\mu)}\right)=E\left(e^{\tilde{\alpha}^{T}(s-\mu)}\right)\leq
e^{\frac{\tilde{\alpha}^{T}\tilde{\alpha}\sigma^{2}}{2}}=e^{\frac{\alpha^{T}AA^{T}\alpha\sigma^{2}}{2}}\leq
e^{\frac{\alpha^{T}\alpha\lambda\sigma^{2}}{2}},$

where $\lambda$ is the largest eigenvalue of $A^{T}A$, as we wished to prove.

### Proof of Lemma S4

Let $s=(X^{T}X)^{-1/2}X^{T}(y-a)\in\mathbb{R}^{d}$, then
$s\sim\mbox{SG}((X^{T}X)^{-1/2}X^{T}(\mu-a),\lambda\sigma^{2})$ by Lemma S3,
where $\lambda$ is the largest eigenvalue of

$\displaystyle(X^{T}X)^{-1/2}X^{T}X(X^{T}X)^{-1/2},$

which is the identity matrix, hence $\lambda=1$, as we wished to prove.

### Proof of Lemma S5

Part (i). Theorem 1 in Hsu et al. (2012) for the $\mu=0$ case gives that

$\displaystyle e^{-w}\geq
P\left(\frac{s^{T}s}{\sigma^{2}}>(d+2\sqrt{dw}+2w)\right)=P\left(\frac{s^{T}s}{\sigma^{2}}>d\left[1+2\sqrt{\frac{w}{d}}+\frac{2w}{d}\right]\right)$

for any $w>0$. Equivalently, letting $t=2w/d$,

$\displaystyle
P\left(\frac{s^{T}s}{\sigma^{2}}>dt\left[1+\sqrt{2/t}+1/t\right]\right)\leq
e^{-dt/2}.$

Part (ii). Let $t_{0}>0$ be fixed and define $k_{0}=1/t_{0}+\sqrt{2c/t_{0}}$.
For all $t\geq t_{0}$, it holds that $k_{0}\geq 1/t+\sqrt{2/t}$. Hence, by
Part (i),

$\displaystyle P\left(\frac{s^{T}s}{\sigma^{2}}>dtk_{0}\right)\leq e^{-dt/2}.$

Equivalently, letting $q=t(1+k_{0})$,

$\displaystyle P\left(\frac{s^{T}s}{\sigma^{2}}>dq\right)\leq
e^{-\frac{dq}{2(1+k_{0})}},$

for any $k_{0}$ such that $k_{0}\geq
1/t+\sqrt{2/t}=(1+k_{0})/q+\sqrt{2(1+k_{0})/q}$.

Part (iii). The result follows by showing that $q\geq 2(1+\sqrt{2})^{2}$ and
$k_{0}=\sqrt{2}(1+\sqrt{2})/\sqrt{q}$ satisfy the condition

$(1+k_{0})/q+\sqrt{2(1+k_{0})/q}\leq k_{0}$

required by Part (ii).

First, for the specified values it holds that $k_{0}\leq 1$ and $k_{0}\leq
q-1$, since $q\geq 2$. Hence

$\displaystyle(1+k_{0})/q+\sqrt{2}\sqrt{(1+k_{0})/q}\leq(1+\sqrt{2})\sqrt{(1+k_{0})/q}$

and it suffices to prove that

$\displaystyle(1+\sqrt{2})\sqrt{(1+k_{0})/q}\leq
k_{0}\Leftrightarrow\sqrt{\frac{1+k_{0}}{k_{0}^{2}}}\leq\frac{\sqrt{q}}{1+\sqrt{2}}.$

Since $k_{0}\leq 1$, we have
$\sqrt{(1+k_{0})/k_{0}^{2}}\leq\sqrt{2/k_{0}^{2}}$, hence it suffices to show

$\displaystyle\frac{\sqrt{2}}{k_{0}}\leq\frac{\sqrt{q}}{1+\sqrt{2}},$

which holds for $k_{0}=\sqrt{2}(1+\sqrt{2})/\sqrt{q}$, as we wished to prove.

### Proof of Lemma S5

Let $t>0$ be an arbitrary real number. Clearly

$\displaystyle P(s^{T}s<a)=P\left(e^{-ts^{T}s}>e^{-ta}\right)\leq
e^{ta}E(e^{-ts^{T}s}),$

the right-hand side following from Markov’s inequality. Since
$e^{-t(s-\mu)^{T}(s-\mu)}<1$,

$\displaystyle
e^{ta}E(e^{-ts^{T}s})=e^{ta}E(e^{-t[(s-\mu)^{T}(s-\mu)+\mu^{T}\mu^{T}+2\mu^{T}(s-\mu)]})\leq
e^{t(a-\mu^{T}\mu)}E(e^{-2t\mu^{T}(s-\mu)})\leq
e^{t(a-\mu^{T}\mu)+2t^{2}\mu^{T}\mu\sigma^{2}},$

where the right-hand side follows from the definition of sub-Gaussianity. The
bound holds for any $t>0$, we obtain the tightest bound by minimizing the
exponent with respect to $t$. Setting its first derivative to 0 gives that the
minimum is attained at $t_{0}=(\mu^{T}\mu-a)/(4\mu^{T}\mu\sigma^{2})$, giving
that

$\displaystyle P(s^{T}s<a)\leq
e^{t_{0}(a-\mu^{T}\mu)+2t_{0}^{2}\mu^{T}\mu\sigma^{2}}=e^{-\frac{(a-\mu^{T}\mu)^{2}}{8\mu^{T}\mu\sigma^{2}}}=e^{-\frac{\mu^{T}\mu}{8\sigma^{2}}(1-a/\mu^{T}\mu)^{2}}.$

### Proof of Proposition 1

Part (i). Let $\underline{u}=[\log(h)/h]^{d/2}\in(0,1)$. Note that

$\displaystyle
U(d,h,c)\leq\underline{u}+\int_{\underline{u}}^{1}P\left(\frac{s^{T}s}{\sigma^{2}}>\frac{cd}{\sigma^{2}}\log\left(\frac{h}{(1/u-1)^{2/d}}\right)\right)du,$

Let $q=(c/\sigma^{2})\log(h/[1/u-1]^{2/d})$. We shall show that $q\geq
2(1+\sqrt{2})^{2}$, satisfying the condition in Lemma S5, Part (iii). To ease
notation let $a=2(1+\sqrt{2})^{2}$. Then $q\geq a$ requires

$\displaystyle\frac{c}{\sigma^{2}}\log\left(\frac{h}{[1/u-1]^{2/d}}\right)\geq
a\Leftrightarrow u\geq\frac{1}{1+(h/e^{a\sigma^{2}/c})^{d/2}},$

which holds since

$\displaystyle\underline{u}=[\log(h)/h]^{d/2}\geq(e^{a\sigma^{2}/c}/h)^{d/2}>\frac{1}{1+(h/e^{a\sigma^{2}/c})^{d/2}}$

given that $\log h\geq e^{a}>e^{a\sigma^{2}/c}$ by assumption, since
$\sigma^{2}<c$. We remark that a possible alternative strategy is to use
Part(ii) from Lemma S5 to set a milder condition on $h$ than the required
$\log h>e^{a}$, and still apply the remainder of the proof’s strategy, however
the algebra becomes somewhat more complicated and for simplicity we use Part
(iii).

Using Lemma S5, Part (iii), gives

$\displaystyle
U(d,h,c)\leq\underline{u}+\int_{\underline{u}}^{1}\left[\frac{(1/u-1)^{2/d}}{h}\right]^{\frac{dc}{2\sigma^{2}[1+k_{u}]}}du$

for $k_{u}=\sqrt{a/q}$. To bound the integrand let
$\underline{q}=(c/\sigma^{2})\log(h/[1/\underline{u}-1]^{2/d})$ and
$\underline{k}=\sqrt{a/\underline{q}}$. Since $q$ is increasing in $u$, it
follows that $k_{u}\leq\underline{k}$. Further, note that the integrand is
$\leq 1$. To see this, for any $u\geq\underline{u}$ it holds that
$(1/u-1)^{2/d}\leq(1/\underline{u}-1)^{2/d}<h/\log(h)$, where $\log(h)\geq 1$
by assumption. Hence the integrand is upper-bounded by taking a smaller power,
specifically

$\displaystyle
U(d,h,c)\leq\underline{u}+\int_{\underline{u}}^{1}\left[\frac{(1/u-1)^{2/d}}{h}\right]^{\frac{dc}{2\sigma^{2}[1+\underline{k}]}}du$

Simple algebra shows that the assumption
$c\geq\sigma^{2}(1+\sqrt{\sigma^{2}/[c\log h]})$ implies that
$c\geq\sigma^{2}(1+\underline{k})$. Hence the integral on the right-hand side
is

$\displaystyle\leq\frac{1}{h^{d/2}}\int_{\underline{u}}^{1}(1/u-1)du=\frac{1}{h^{d/2}}[\log(1)-\log(\underline{u})-(1-\underline{u})]<\frac{1}{h^{d/2}}\log(1/\underline{u}).$

Plugging in $\underline{u}=[\log(h)/h]^{d/2}$ gives

$\displaystyle U(d,h,c)<\left[\frac{\log
h}{h}\right]^{d/2}+\frac{1}{h^{d/2}}\log\left(\frac{h^{d/2}}{[\log(h)]^{d/2}}\right)<\left[\frac{\log
h}{h}\right]^{d/2}+\frac{\log\left(h^{d/2}\right)}{h^{d/2}}$

since $\log(h^{d/2})\geq 1$.

Part (ii). Define $\tilde{c}=\sigma^{2}(1+\epsilon)$, where
$\epsilon=\sqrt{\sigma^{2}/[c\log h]}$, so that $c/\tilde{c}<1$. We shall
split $U(d,h,c)$ by integrating over $u\leq 0.5$ and $u>0.5$, then apply Part
(i) to the first integral and Lemma S5 to the second integral. Specifically,

$\displaystyle\int_{0}^{0.5}P\left(\frac{s^{T}s}{c}>d\log\left(\frac{h}{(1/u-1)^{2/d}}\right)\right)du=\int_{0}^{0.5}P\left(\frac{s^{T}s}{\tilde{c}}>d\log\left(h^{\frac{c}{\tilde{c}}}\left[\frac{u}{1-u}\right]^{\frac{2c}{d\tilde{c}}}\right)\right)du$
$\displaystyle<\int_{0}^{0.5}P\left(\frac{s^{T}s}{\tilde{c}}>d\log\left(h^{\frac{c}{\tilde{c}}}\left[\frac{u}{1-u}\right]^{\frac{2}{d}}\right)\right)du.$

the right-hand side following from $[u/(1-u)]^{c/\tilde{c}}>u/(1-u)$, since
$u/(1-u)\leq 1$ and $c/\tilde{c}<1$. The resulting integral can be bounded by
Part (i), since $\tilde{c}$ satisfies
$\tilde{c}\geq\sigma^{2}(1+\sqrt{\sigma^{2}/[\tilde{c}\log h]})$ and
$h^{c/\tilde{c}}$ satisfies $\log(h^{c/\tilde{c}})\geq e^{2(1+\sqrt{2})^{2}}$,
since

$\log
h\geq\frac{\tilde{c}e^{2(1+\sqrt{2})^{2}}}{c}=\frac{\sigma^{2}(1+\epsilon)e^{2(1+\sqrt{2})^{2}}}{c}$

by assumption. Therefore, applying Part (i) of Proposition 1,

(S.19)
$\displaystyle\int_{0}^{0.5}P\left(\frac{s^{T}s}{\tilde{c}}>d\log\left(h^{\frac{c}{\tilde{c}}}\left[\frac{u}{1-u}\right]^{\frac{2}{d}}\right)\right)du<\frac{2\max\left\\{[\log(h^{c/\tilde{c}})]^{d/2},\log(h^{\frac{dc}{2\tilde{c}}})\right\\}}{h^{\frac{dc}{2\tilde{c}}}}.$

Next consider

$\displaystyle\int_{0.5}^{1}P\left(\frac{s^{T}s}{\tilde{c}}>d\log\left(h^{\frac{c}{\tilde{c}}}\left[\frac{u}{1-u}\right]^{\frac{2c}{d\tilde{c}}}\right)\right)du\leq\int_{0.5}^{1}P\left(\frac{s^{T}s}{\tilde{c}}>d\log\left(h^{\frac{c}{\tilde{c}}}\right)\right)du$
$\displaystyle=0.5P\left(\frac{s^{T}s}{\sigma^{2}}>d\log\left(h^{\frac{c}{\sigma^{2}}}\right)\right),$

since $u/(1-u)\geq 1$ for $u\geq 0.5$. To bound the right-hand side we use
Lemma S5, Part (iii). The lemma requires that
$\log(h^{\frac{c}{\sigma^{2}}})\geq 2(1+\sqrt{2})^{2}$, which holds since

$\log h\geq\frac{\sigma^{2}}{c}2(1+\sqrt{2})\Leftrightarrow h\geq
e^{\frac{\sigma^{2}}{c}2(1+\sqrt{2})}$

by assumption. Thus

(S.20) $\displaystyle
0.5P\left(\frac{s^{T}s}{\sigma^{2}}>d\log\left(h^{\frac{c}{\sigma^{2}}}\right)\right)\leq\frac{1}{2h^{\frac{dc}{2\sigma^{2}(1+k_{0})}}},$

where $k_{0}=\sqrt{2}(1+\sqrt{2})\sqrt{\sigma^{2}/[c\log
h]}=\sqrt{2}(1+\sqrt{2})\epsilon$.

To conclude the proof, plug in $\tilde{c}=\sigma^{2}(1+\epsilon)$ into (S.19)
and combine with (S.20) to obtain

$\displaystyle
U(d,h,c)<\frac{2\max\left\\{[\log(h^{c/[\sigma^{2}(1+\epsilon)]})]^{d/2},\log(h^{\frac{dc}{2\sigma^{2}(1+\epsilon)}})\right\\}}{h^{\frac{dc}{2\sigma^{2}(1+\epsilon)}}}+\frac{1}{2h^{\frac{dc}{2\sigma^{2}\epsilon(1+\sqrt{2}(1+\sqrt{2}))}}}$
$\displaystyle<\frac{2\max\left\\{[\log(h^{c/[\sigma^{2}(1+\epsilon)]})]^{d/2},\log(h^{\frac{dc}{2\sigma^{2}(1+\epsilon)}})\right\\}}{h^{\frac{dc}{2\sigma^{2}(1+\epsilon\sqrt{2}(1+\sqrt{2}))}}}+\frac{1}{2h^{\frac{dc}{2\sigma^{2}(1+\epsilon\sqrt{2}(1+\sqrt{2}))}}},$

giving the desired result.

##  S10. Proof of Theorem 3

Fix any $\gamma\neq{\tilde{\gamma}}^{*}\in\Gamma$. The goal is to show that
the random variable $\tilde{p}(\gamma\mid y)$ converges to 0 in the $L_{1}$
sense, that is $\lim_{n\rightarrow\infty}E_{F^{*}}[\tilde{p}(\gamma\mid
y)]=0$. We also seek to bound its convergence rate, that is find a sequence
$a_{n}$ such that $\lim_{n\rightarrow\infty}E_{F^{*}}[\tilde{p}(\gamma\mid
y)]/a_{n}=0$, where the bound $a_{n}$ is as tight as possible. Since $L_{1}$
convergence implies convergence in probability, this also proves the weaker
result that $\tilde{p}({\tilde{\gamma}}^{*}\mid y)=O_{p}(a_{n})$.

Following the construction in Rossell (2018), note that

$\displaystyle E_{F^{*}}\left(\tilde{p}(\gamma\mid
y)\right)=E_{F^{*}}\left(\left[1+\sum_{\gamma^{\prime}\neq\gamma}\tilde{B}_{\gamma^{\prime}\gamma}\frac{p(\gamma^{\prime})}{p(\gamma)}\right]^{-1}\right)<E_{F^{*}}\left(\left[1+\tilde{B}_{{\tilde{\gamma}}^{*}\gamma}\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}\right]^{-1}\right),$

and that the right-hand side is the expectation of a positive random variable.
Hence its expectation can be found by integrating the survival (or right-tail
probability) function

(S.21)
$\displaystyle\int_{0}^{1}P_{F^{*}}\left(\left[1+\tilde{B}_{{\tilde{\gamma}}^{*}\gamma}\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}\right]^{-1}>u\right)du=\int_{0}^{1}P_{F^{*}}\left(\log\tilde{B}_{\gamma{\tilde{\gamma}}^{*}}>\log\left(\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)(1/u-1)}\right)\right)du.$

Taking the expression for $\tilde{B}^{L}_{\gamma,{\tilde{\gamma}}^{*}}$ in
(S.8) under an arbitrary Normal prior
$p(\beta_{\gamma}\mid\phi,\gamma)=N(\beta_{\gamma};0,\phi
g_{L}V_{\gamma}^{-1}/b^{\prime\prime}(0))$, where $V_{\gamma}$ is a positive-
definite matrix, gives

$\displaystyle\log\tilde{B}^{L}_{\gamma{\tilde{\gamma}}^{*}}=\log\left(\left[\frac{1}{ng_{L}}\right]^{\frac{p_{\gamma}-p_{{\tilde{\gamma}}^{*}}}{2}}\frac{|Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}/n|^{\frac{1}{2}}|V_{\gamma}|^{\frac{1}{2}}}{|Z_{\gamma}^{T}Z_{\gamma}/n|^{\frac{1}{2}}|V_{{\tilde{\gamma}}^{*}}|^{\frac{1}{2}}}\right)+\frac{b^{\prime\prime}(0)}{2\phi}\tilde{W}_{\gamma{\tilde{\gamma}}^{*}}$

where
$\tilde{W}_{\gamma{\tilde{\gamma}}^{*}}=\tilde{\beta}_{\gamma}^{T}(Z_{\gamma}^{T}Z_{\gamma}-V_{\gamma}/g_{L})\tilde{\beta}_{\gamma}-\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}(Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}-V_{{\tilde{\gamma}}^{*}}/g_{L})\tilde{\beta}_{{\tilde{\gamma}}^{*}}$.

We focus on the particular case of Zellner’s prior where
$V_{\gamma}=Z_{\gamma}^{T}Z_{\gamma}/n$. Rearranging terms in (S.21) gives

(S.22)
$\displaystyle\int_{0}^{1}P_{F^{*}}\left(\left[1-\frac{1}{ng_{L}}\right]\frac{W_{\gamma{\tilde{\gamma}}^{*}}}{\phi/b^{\prime\prime}(0)}>(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})\log\left(\frac{h}{(1/u-1)^{\frac{2}{p_{\gamma}-p_{{\tilde{\gamma}}*}}}}\right)\right)du,$

where
$W_{\gamma{\tilde{\gamma}}^{*}}=\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}}$
and

$\displaystyle
h=ng_{L}\left[\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}\right]^{\frac{2}{p_{\gamma}-p_{{\tilde{\gamma}}*}}}.$

The proof’s strategy is based on noting that $W_{\gamma{\tilde{\gamma}}^{*}}$
can be expressed in terms of quadratic forms of sub-Gaussian random variables,
which allows bounding the probability in (S.22) and its integral with respect
to $u$. Given that $\lim_{n\rightarrow\infty}ng_{L}=\infty$ by assumption, to
ease derivations we drop the factor $1-1/(ng_{L})$ from subsequent arguments
(the results follow by noting that $1-1/(ng_{L})$ is arbitrarily close to 1 as
$n$ grows). We consider separately the case
${\tilde{\gamma}}^{*}\subset\gamma$, where $W_{\gamma{\tilde{\gamma}}^{*}}$ is
a quadratic form of a zero-mean sub-Gaussian random vector, and the case
${\tilde{\gamma}}^{*}\not\subseteq\gamma$ where
$W_{\gamma{\tilde{\gamma}}^{*}}$ is the difference between two sub-Gaussian
quadratic forms.

Before proceeding, we remark that it is possible to extend our results to
Normal priors with general $V_{\gamma}$. Briefly, one then obtains an
analogous expression to (S.22) that involves
$\tilde{W}_{\gamma{\tilde{\gamma}}^{*}}$ and determinants
$|V_{\gamma}^{-1}Z_{\gamma}^{T}Z_{\gamma}/n|$. It is still possible to express
$\tilde{W}_{\gamma{\tilde{\gamma}}^{*}}$ in terms of quadratic forms of sub-
Gaussian random variables, which then provides a bound for (S.22) that depends
on eigenvalues of $V_{\gamma}^{-1}Z_{\gamma}^{T}Z_{\gamma}/n$. By placing
suitable conditions on said eigenvalues, one obtains rates analogous to those
for Zellner’s prior. We also remark that, for more general non-Normal priors,
one may replace $p(\beta_{\gamma}\mid\phi,\gamma)$ by a Taylor expansion at 0,
which is equivalent to approximating $p(\beta_{\gamma}\mid\phi,\gamma)$ by a
Normal prior, and then carrying out the proof as outlined above.

###  S10.1. Part (i)

Suppose that ${\tilde{\gamma}}^{*}\subset\gamma$. Let
$X_{\gamma\setminus{\tilde{\gamma}}^{*}}=Z_{\gamma\setminus{\tilde{\gamma}}^{*}}-Z_{{\tilde{\gamma}}^{*}}(Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}})^{-1}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{\gamma\setminus{\tilde{\gamma}}^{*}}$,
from Lemma S1 we can write

$\displaystyle W_{\gamma{\tilde{\gamma}}^{*}}=(\tilde{y}\pm
Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*})^{T}X_{\gamma\setminus{\tilde{\gamma}}^{*}}(X_{\gamma\setminus{\tilde{\gamma}}^{*}}^{T}X_{\gamma\setminus{\tilde{\gamma}}^{*}})^{-1}X_{\gamma\setminus{\tilde{\gamma}}^{*}}^{T}(\tilde{y}\pm
Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*})$
$\displaystyle=(\tilde{y}-Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*})^{T}X_{\gamma\setminus{\tilde{\gamma}}^{*}}(X_{\gamma\setminus{\tilde{\gamma}}^{*}}^{T}X_{\gamma\setminus{\tilde{\gamma}}^{*}})^{-1}X_{\gamma\setminus{\tilde{\gamma}}^{*}}^{T}(\tilde{y}-Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}),$

since
$Z_{{\tilde{\gamma}}^{*}}^{T}X_{\gamma\setminus{\tilde{\gamma}}^{*}}=Z_{{\tilde{\gamma}}^{*}}^{T}Z_{\gamma\setminus{\tilde{\gamma}}^{*}}-Z_{{\tilde{\gamma}}^{*}}^{T}Z_{\gamma\setminus{\tilde{\gamma}}^{*}}=0$.
Since
$\tilde{y}\sim\mbox{SG}(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*},\sigma^{2})$,
Lemma S4 gives that $W_{\gamma{\tilde{\gamma}}^{*}}=s^{T}s$ where
$s\sim\mbox{SG}(0,\sigma^{2})$ is a $p_{\gamma}-p_{{\tilde{\gamma}}^{*}}$
dimensional sub-Gaussian random vector. Therefore the integral in (S.22) is of
the form required by Proposition 1, letting $c=\phi/b^{\prime\prime}(0)$,
$d=p_{\gamma}-p_{\gamma^{\prime}}$ and $h$ as defined in (S.22). We apply
Proposition 1 Part (i) or Part (ii) depending on whether
$\phi/b^{\prime\prime}(0)>\sigma^{2}$ or
$\phi/b^{\prime\prime}(0)<\sigma^{2}$.

Consider first the case $c\geq\sigma^{2}(1+\sqrt{1/\log h})$. Applying
Proposition 1 Part (i) requires the condition that $\log(h)\geq
e^{2(1+\sqrt{2})^{2}}$. Note that, as $n$ grows,

$\displaystyle\log(h)=\log(ng_{L})+\frac{2}{p_{\gamma}-p_{{\tilde{\gamma}}*}}\log\left(\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}\right)\gg
1$

by assumption. Therefore the condition holds for large enough $n$ and
Proposition 1 Part (i) gives

$\displaystyle
E_{F^{*}}\left(\left[1+\tilde{B}_{{\tilde{\gamma}}^{*}\gamma}\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}\right]^{-1}\right)\leq\frac{2\max\left\\{[\log
h]^{d/2},\log(h^{d/2})\right\\}}{h^{d/2}}$

for all $n\geq n_{0}$, and some fixed $n_{0}$, as we wished to prove.

Consider next the case $c<\sigma^{2}(1+\sqrt{\sigma^{2}/[c\log h]})$. Applying
Proposition 1 Part (ii) requires the condition that
$\log(h)\geq\max\\{[\sigma^{2}(1+\epsilon)/c]e^{2(1+\sqrt{2})^{2}},e^{2(1+\sqrt{2})^{2}\sigma^{2}/c}\\}$,
which again holds for large enough $n$, since $(c,\sigma^{2})$ are constant
and $\epsilon$ is arbitrarily close to 0 as $h$ grows. Hence, Proposition 1
Part (ii) gives

$\displaystyle
E_{F^{*}}\left(\left[1+\tilde{B}_{{\tilde{\gamma}}^{*}\gamma}\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}\right]^{-1}\right)\leq\frac{2.5\max\left\\{\left[\log(h^{\frac{c}{\sigma^{2}(1+\epsilon)}})\right]^{d/2},\log\left(h^{\frac{dc}{2\sigma^{2}(1+\epsilon)}}\right)\right\\}}{h^{\frac{dc}{2\sigma^{2}(1+\epsilon\sqrt{2}(1+\sqrt{2}))}}}$

where $\epsilon=\sqrt{\sigma^{2}/[c\log h]}$, for all $n\geq n_{0}$ and some
fixed $n_{0}$, as we wished to prove.

###  S10.2. Part (ii)

Suppose that ${\tilde{\gamma}}^{*}\not\subseteq\gamma$. Denote by
$\gamma^{\prime}={\tilde{\gamma}}^{*}\cup\gamma$ the model such that
$Z_{\gamma^{\prime}}$ includes all columns from $Z_{{\tilde{\gamma}}^{*}}$ and
$Z_{\gamma}$. Note that

$W_{\gamma{\tilde{\gamma}}^{*}}=\tilde{\beta}_{\gamma}^{T}Z_{\gamma}^{T}Z_{\gamma}\tilde{\beta}_{\gamma}-\tilde{\beta}_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}+\tilde{\beta}_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}^{T}Z_{\gamma^{\prime}}\tilde{\beta}_{\gamma^{\prime}}-\tilde{\beta}_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}^{T}Z_{{\tilde{\gamma}}^{*}}\tilde{\beta}_{{\tilde{\gamma}}^{*}}=W_{\gamma^{\prime}{\tilde{\gamma}}^{*}}-W_{\gamma^{\prime}\gamma}.$

Since ${\tilde{\gamma}}^{*}\subset\gamma^{\prime}$, by Lemma S4 it holds that
$W_{\gamma^{\prime}{\tilde{\gamma}}^{*}}=s_{1}^{T}s_{1}$ where
$s_{1}\sim\mbox{SG}(0,\sigma^{2})$ is a
$(p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}})$-dimensional sub-Gaussian
vector. Further, since $\gamma\subset\gamma^{\prime}$, by Lemma S1,

$\displaystyle W_{\gamma^{\prime}\gamma}$ $\displaystyle=(\tilde{y}\pm
Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*})^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}(\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma})^{-1}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}(\tilde{y}\pm
Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*})$
$\displaystyle=(\tilde{y}-Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*})^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}(\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma})^{-1}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}(\tilde{y}-Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*}),$

where
$\tilde{Z}_{\gamma^{\prime}\setminus\gamma}=(I-Z_{\gamma}(Z_{\gamma}^{T}Z_{\gamma})^{-1}Z_{\gamma}^{T})Z_{\gamma^{\prime}\setminus\gamma}$,
and the right-hand side follows from
$Z_{\gamma}^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}=0$ and direct
algebra.

Since
$\tilde{y}-Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*}\sim\mbox{SG}(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}-Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*},\sigma^{2})$
by assumption, by Lemma S4 we may write
$W_{\gamma^{\prime}\gamma}=s_{2}^{T}s_{2}$, where
$s\sim\mbox{SG}(\mu,\sigma^{2})$ is a
$(p_{\gamma^{\prime}}-p_{\gamma})$-dimensional sub-Gaussian random vector, and
$\mu=(\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma})^{-1/2}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}-Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*})\neq
0$. Therefore (S.22) is equal to

(S.23)
$\displaystyle\int_{0}^{1}P_{F^{*}}\left(\frac{s_{1}^{T}s_{1}-s_{2}^{T}s_{2}}{\phi/b^{\prime\prime}(0)}>(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})\log\left(\frac{h}{(1/u-1)^{2/(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}}\right)\right)du.$

The latter expression has the form required by Proposition 2, setting
$c=\phi/b^{\prime\prime}(0)$,
$d_{1}=p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}}$,
$d_{2}=p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}}$ and
$d=p_{\gamma}-p_{{\tilde{\gamma}}^{*}}$. The non-centrality parameter, denoted
$\lambda_{\gamma}$, is

$\displaystyle\lambda_{\gamma}=\mu^{T}\mu=(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}-Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*})^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}(\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}\tilde{Z}_{\gamma^{\prime}\setminus\gamma})^{-1}\tilde{Z}_{\gamma^{\prime}\setminus\gamma}^{T}(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}-Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*})$
$\displaystyle=(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*})^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}\setminus\gamma}[Z_{{\tilde{\gamma}}^{*}\setminus\gamma}^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}\setminus\gamma}]^{-1}Z_{{\tilde{\gamma}}^{*}\setminus\gamma}^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*},$

where $H_{\gamma}=Z_{\gamma}(Z_{\gamma}^{T}Z_{\gamma})^{-1}Z_{\gamma}^{T}$ is
the projection matrix onto the column span of $Z_{\gamma}$, and to obtain the
right-hand side we used that $I-H_{\gamma}$ is idempotent, that the optimal
linear projection
$Z_{\gamma}{\tilde{\beta}}_{\gamma}^{*}=H_{\gamma}Z_{\gamma^{*}}{\tilde{\beta}}_{\gamma^{*}}^{*}$,
and noting that
$Z_{\gamma^{\prime}\setminus\gamma}=Z_{\gamma^{*}\setminus\gamma}$.

The expression for $\lambda_{\gamma}$ can be simplified. First, note that
$Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}=Z_{{\tilde{\gamma}}^{*}\setminus\gamma}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}\setminus\gamma}^{*}+Z_{\gamma\cap{\tilde{\gamma}}^{*}}b_{\gamma\cap{\tilde{\gamma}}^{*}}$,
where $b_{\gamma\cap{\tilde{\gamma}}^{*}}$ is the subset of
${\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}$ corresponding to the columns of
$Z_{\gamma\cap{\tilde{\gamma}}^{*}}$. Hence

$\displaystyle(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*})^{T}(I-H_{\gamma})=({\tilde{\beta}}_{{\tilde{\gamma}}^{*}\setminus\gamma}^{*})^{T}Z_{{\tilde{\gamma}}^{*}\setminus\gamma}^{T}(I-H_{\gamma})+b_{\gamma\cap{\tilde{\gamma}}^{*}}^{T}Z_{\gamma\cap{\tilde{\gamma}}^{*}}^{T}(I-H_{\gamma})=({\tilde{\beta}}_{{\tilde{\gamma}}^{*}\setminus\gamma}^{*})^{T}Z_{{\tilde{\gamma}}^{*}\setminus\gamma}^{T}(I-H_{\gamma})$

since $Z_{\gamma\cap{\tilde{\gamma}}^{*}}^{T}(I-H_{\gamma})=0$, as
$Z_{\gamma\cap{\tilde{\gamma}}^{*}}$ is in the linear span of $Z_{\gamma}$.
Therefore

$\displaystyle\lambda_{\gamma}$
$\displaystyle=({\tilde{\beta}}_{{\tilde{\gamma}}^{*}\setminus\gamma}^{*})^{T}Z_{{\tilde{\gamma}}^{*}\setminus\gamma}^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}\setminus\gamma}[Z_{{\tilde{\gamma}}^{*}\setminus\gamma}^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}\setminus\gamma}]^{-1}Z_{{\tilde{\gamma}}^{*}\setminus\gamma}^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}$
$\displaystyle=(Z_{{\tilde{\gamma}}^{*}\setminus\gamma}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}\setminus\gamma}^{*})^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}=(Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*})^{T}(I-H_{\gamma})Z_{{\tilde{\gamma}}^{*}}{\tilde{\beta}}_{{\tilde{\gamma}}^{*}}^{*}.$

Applying Proposition 2 requires the following condition to hold:

$\displaystyle\frac{\mu^{T}\mu}{c\log(\mu^{T}\mu)}\geq-d\log(h)+d_{1}\log(h_{0})$
(S.24)
$\displaystyle=(p_{{\tilde{\gamma}}^{*}}-p_{\gamma})\log\left(ng_{L}\right)+2\log\left(\frac{p(\gamma)}{p({\tilde{\gamma}}^{*})}\right)+(p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}})\log(h_{0})$

where $\log(h_{0})>0$ and $h_{0}$ is the positive finite constant in
Proposition 1.

The fact that (S.24) holds follows directly from Assumption (D3). If
$p_{\gamma}\geq p_{{\tilde{\gamma}}^{*}}$, then
$p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}}\leq p_{\gamma}$ and

(S.25)
$\displaystyle\eqref{eq:condition_ncp_proof}\leq(p_{{\tilde{\gamma}}^{*}}-p_{\gamma})\log\left(ng_{L}\right)+2\log\left(\frac{p(\gamma)}{p({\tilde{\gamma}}^{*})}\right)+p_{\gamma}\log(h_{0})\ll\frac{\mu^{T}\mu}{\log(\mu^{T}\mu)},$

where the right-hand side holds by assumption. If
$p_{\gamma}<p_{{\tilde{\gamma}}^{*}}$, then
$p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}}\leq p_{{\tilde{\gamma}}^{*}}$
and

(S.26)
$\displaystyle\eqref{eq:condition_ncp_proof}\leq(p_{{\tilde{\gamma}}^{*}}-p_{\gamma})\log\left(ng_{L}\right)+2\log\left(\frac{p(\gamma)}{p({\tilde{\gamma}}^{*})}\right)+p_{{\tilde{\gamma}}^{*}}\log(h_{0})\ll\frac{\mu^{T}\mu}{\log(\mu^{T}\mu)},$

which again holds by assumption.

To complete the proof we apply Proposition 1. Consider first the case
$c=\phi/b^{\prime\prime}(0)>\sigma^{2}$. By Proposition 2 Part (i), there
exists a finite $t_{0}$ such that (S.23) is

$\displaystyle\leq\frac{3\max\left\\{[\frac{2}{p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}}}\log(l(\mu))]^{(p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}})/2},\log(l(\mu))\right\\}}{l(\mu)}$

for all $\mu^{T}\mu\geq t_{0}$, where
$l(\mu)=h^{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})/2}e^{\frac{\mu^{T}\mu
b^{\prime\prime}(0)}{2\phi\log(\mu^{T}\mu)}}$. Given that
$\lim_{n\rightarrow\infty}\mu^{T}\mu=\infty$ by assumption and

$\frac{\mu^{T}\mu
b^{\prime\prime}(0)}{2\phi\log(\mu^{T}\mu)}\gg(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})/2\log(h)$

from (S.24), for any $a<1$ there exists a finite $n_{0}$ such that, for all
$n\geq n_{0}$,

(S.27) $\displaystyle\eqref{eq:integral_survfun_nonnested}\leq
e^{-a\left[\frac{\mu^{T}\mu
b^{\prime\prime}(0)}{2\phi\log(\mu^{T}\mu)}+\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})\log(h)}{2}\right]}.$

Consider now the case $c=\phi/b^{\prime\prime}(0)\leq\sigma^{2}$. By
Proposition 2 Part (ii), there exists a finite $t_{0}$ such that

$\displaystyle\eqref{eq:integral_survfun_nonnested}\leq\frac{3.5\max\left\\{[\frac{2}{p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}}}\log(l^{\prime}(\mu))^{1/(1+\epsilon)}]^{(p_{\gamma^{\prime}}-p_{{\tilde{\gamma}}^{*}})/2},\log(l^{\prime}(\mu)^{1/(1+\epsilon)})\right\\}}{l^{\prime}(\mu)^{1/(1+\epsilon)}},$

where $\lim_{\mu^{T}\mu\rightarrow\infty}\epsilon=0$,
$l^{\prime}(\mu)=h^{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})/2}e^{\frac{\mu^{T}\mu}{2\sigma^{2}\log(\mu^{T}\mu)}}$.
Hence, for any $a<1$ there exists a finite $n_{0}$ such that, for all $n\geq
n_{0}$,

(S.28) $\displaystyle\eqref{eq:integral_survfun_nonnested}\leq
e^{-a\left[\frac{\mu^{T}\mu}{2\sigma^{2}\log(\mu^{T}\mu)}+\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})\log(h)}{2}\right]}.$

The desired result is obtained by noting that

$\displaystyle\exp\left\\{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})\log(h)}{2}\right\\}=(ng_{L})^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}$

and combining (S.27) and (S.28) to obtain

(S.29) $\displaystyle\eqref{eq:integral_survfun_nonnested}\leq
e^{-a\left[\frac{\mu^{T}\mu}{2\max\\{\sigma^{2},\phi/b^{\prime\prime}(0)\\}\log(\mu^{T}\mu)}\right]}\left(\frac{p(\gamma)/p({\tilde{\gamma}}^{*})}{(ng_{L})^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}}\right)^{a}.$

##  S11. Proof of Theorem 4

###  S11.1. Part (i)

Let $S_{l}=\\{\gamma:|\gamma|=l,{\tilde{\gamma}}^{*}\subset\gamma\\}$ be the
set of size $l$ models, and $\Gamma_{l}=\\{\gamma:|\gamma|=l,p(\gamma)>0\\}$
be those receiving positive prior probability (e.g. satisfying any specified
hierarchical/group constraints). Then

$\displaystyle E_{F_{0}}(p(S\mid
y))=E_{F_{0}}\left(\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\sum_{\gamma\in
S_{l}\cap\Gamma_{l}}\tilde{p}(\gamma\mid y)\right)\leq
E_{F_{0}}\left(\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\sum_{\gamma\in
S_{l}}\tilde{p}(\gamma\mid y)\right).$

From Theorem 3 Part (i), for each $\gamma$ there is a finite $n_{0\gamma}$
such that

(S.30) $\displaystyle E_{F_{0}}\left(\tilde{p}(\gamma\mid
y)\right)\leq\frac{2\max\left\\{[(2/[p_{\gamma}-p_{{\tilde{\gamma}}^{*}}])\log
r_{\gamma}^{a}]^{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})/2},\log(r_{\gamma}^{a})\right\\}}{r_{\gamma}^{a}}$

for all $n\geq n_{0\gamma}$, a constant $a$ such that $a=1$ if
$\phi/b^{\prime\prime}(0)>\sigma^{2}$ and
$a<\phi/(b^{\prime\prime}(0)\sigma^{2})$ if
$\phi/b^{\prime\prime}(0)\leq\sigma^{2}$, and

$\displaystyle
r_{\gamma}=\left(ng\right)^{\frac{p_{\gamma}-p_{{\tilde{\gamma}}^{*}}}{2}}\frac{p({\tilde{\gamma}}^{*})}{p(\gamma)}.$

To alleviate algebra, by observing that
$\log(r_{\gamma}^{a})\succeq[(2/[p_{\gamma}-p_{{\tilde{\gamma}}^{*}}])\log
r_{\gamma}^{a}]^{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})/2}$ for
$p_{\gamma}-p_{{\tilde{\gamma}}^{*}}\geq 2$ and that
$\log(r_{\gamma}^{a})\ll[(2/[p_{\gamma}-p_{{\tilde{\gamma}}^{*}}])\log
r_{\gamma}^{a}]^{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})/2}$ for
$p_{\gamma}-p_{{\tilde{\gamma}}^{*}}=1$, it is simple to show that the bound
$E_{F_{0}}(\tilde{p}(\gamma\mid y))\leq\log(r_{\gamma}^{a})/r_{\gamma}^{a}$
gives the same asymptotic rate as (S.30). Further, inspecting the proof of
Theorem 3 shows that $n_{0\gamma}$ grows with the number of model parameters
$p_{\gamma}$, and that if $p_{\gamma}\ll n$ then $n_{0}=\max_{\gamma\in
S,|\gamma|=\bar{J}}n_{0\gamma}$ can be taken to be fixed.

We consider separately the cases $\phi/b^{\prime\prime}(0)>\sigma^{2}$ and
$\phi/b^{\prime\prime}(0)\leq\sigma^{2}$.

#### Case $\phi/b^{\prime\prime}(0)>\sigma^{2}$

Then $a=1$, and for any $n\geq n_{0}$,

(S.31) $\displaystyle E_{F_{0}}(p(S\mid
y))\leq\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\sum_{\gamma\in
S_{l}}\frac{\log\left((ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}p({\tilde{\gamma}}^{*})/p(\gamma)\right)}{(ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}}\frac{p(\gamma)}{p({\tilde{\gamma}}^{*})}.$

To prove the desired result we plug in the expression of
$p(\gamma)/p({\tilde{\gamma}}^{*})$, then use algebraic manipulation and the
Binomial coefficient ordinary generating function to carry out the sum in
(S.31). First note that

(S.32)
$\displaystyle\frac{p(\gamma)}{p({\tilde{\gamma}}^{*})}=\frac{p(|\gamma|)}{p(|{\tilde{\gamma}}^{*}|)}\frac{{J\choose|{\tilde{\gamma}}^{*}|}}{{J\choose|\gamma|}}=p^{-c(|\gamma|-|{\tilde{\gamma}}^{*}|)}{|\gamma|\choose|{\tilde{\gamma}}^{*}|}{J-|{\tilde{\gamma}}^{*}|\choose|\gamma|-|{\tilde{\gamma}}^{*}|}^{-1}.$

Denote by $\omega(|\gamma|)=p({\tilde{\gamma}}^{*})/p(\gamma)$. The right-hand
side in (S.31) is

$\displaystyle=\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}p^{-c(l-|{\tilde{\gamma}}^{*}|)}{l\choose|{\tilde{\gamma}}^{*}|}m_{l}$

where

$\displaystyle m_{l}={J-|{\tilde{\gamma}}^{*}|\choose
l-|{\tilde{\gamma}}^{*}|}^{-1}\sum_{\gamma\in
S_{l}}\frac{\log\left((ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}\omega(l)\right)}{(ng)^{\frac{p_{\gamma}-p_{{\tilde{\gamma}}^{*}}}{2}}}$

is an average penalization term across the ${J-|{\tilde{\gamma}}^{*}|\choose
l-|{\tilde{\gamma}}^{*}|}$ models included in $S_{l}$. Since
$p_{\gamma}-p_{{\tilde{\gamma}}^{*}}>q(|\gamma|-|{\tilde{\gamma}}^{*}|)$,
where $q$ is the smallest group size,

$\displaystyle\eqref{eq:bound_pp_spurious2}$
$\displaystyle\leq\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}{l\choose|{\tilde{\gamma}}^{*}|}\frac{\log\left((ng)^{\frac{q(l-|{\tilde{\gamma}}^{*}|)}{2}}\omega(l)\right)}{p^{c(l-|{\tilde{\gamma}}^{*}|)}(ng)^{\frac{q(l-|{\tilde{\gamma}}^{*}|)}{2}}}$
(S.33)
$\displaystyle\leq\log\left((ng)^{\frac{q(\bar{J}-|{\tilde{\gamma}}^{*}|)}{2}}\max_{l\geq|{\tilde{\gamma}}^{*}|+1}\omega(l)\right)(p^{c}(ng)^{q/2})^{|{\tilde{\gamma}}^{*}|}\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}{l\choose|{\tilde{\gamma}}^{*}|}\frac{1}{\left(p^{c}(ng)^{\frac{q}{2}}\right)^{l}}.$

To complete the proof, recall that the Binomial coefficient ordinary
generating function gives that $\sum_{l=k+1}^{\infty}{l\choose
k}b^{l}=b^{k}[(1-b)^{-(k+1)}-1]$ for any $b$. Plugging in
$b=p^{-c}(ng)^{-q/2}$ and $k=|{\tilde{\gamma}}^{*}|$,

$\displaystyle(p^{c}(ng)^{q/2})^{|{\tilde{\gamma}}^{*}|}\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}{l\choose|{\tilde{\gamma}}^{*}|}\frac{1}{\left(p^{c}(ng)^{\frac{q}{2}}\right)^{l}}\leq(1-p^{-c}(ng)^{-q/2})^{-(|{\tilde{\gamma}}^{*}|+1)}-1.$

Using that $ng\rightarrow\infty$ by assumption and the definition of the
exponential function

$\displaystyle\asymp\exp\left\\{\frac{|{\tilde{\gamma}}^{*}|+1}{p^{c}(ng)^{q/2}}\right\\}-1\asymp\frac{|{\tilde{\gamma}}^{*}|+1}{p^{c}(ng)^{q/2}}$

since
$\lim_{n\rightarrow\infty}[|{\tilde{\gamma}}^{*}|+1]/[p^{c}(ng)^{q/2}]=0$ by
assumption and $\lim_{z\rightarrow\infty}(e^{z}-1)/z=1$. Combining this
expression with (S.33)

$\displaystyle\log\left((ng)^{\frac{q(\bar{J}-|{\tilde{\gamma}}^{*}|)}{2}}\max_{l\geq|{\tilde{\gamma}}^{*}|+1}\omega(l)\right)\frac{|{\tilde{\gamma}}^{*}|+1}{p^{c}(ng)^{q/2}}.$

Using Stirling’s bounds $\sqrt{2\pi}i^{i+1/2}e^{-i}<i!<ei^{i+1/2}e^{-i}$ gives

$\displaystyle\omega(l)\leq\frac{e^{2}}{2\pi}p^{c(l-|{\tilde{\gamma}}^{*}|)}(J-\bar{J})^{\bar{J}-|{\tilde{\gamma}}^{*}|}\left(1+\frac{\bar{J}-|{\tilde{\gamma}}^{*}|}{J-\bar{J}}\right)^{J-|{\tilde{\gamma}}^{*}|+\frac{1}{2}}.$
$\displaystyle\log\omega(l)\leq\log\left(\frac{e^{2}}{2\pi}\right)+(\bar{J}-|{\tilde{\gamma}}^{*}|)[c\log(p)+\log(J)]+(J-|{\tilde{\gamma}}^{*}|+\frac{1}{2})\log\left(1+\frac{\bar{J}-|{\tilde{\gamma}}^{*}|}{J-\bar{J}}\right).$

Using that $l\leq\bar{J}$, $J\leq p$, and that
$\lim_{n\rightarrow\infty}\bar{J}/J=0$ by assumption, for large enough $n$

(S.34)
$\displaystyle\log\max_{l\geq|{\tilde{\gamma}}^{*}|}\omega(l)\leq(\bar{J}-|{\tilde{\gamma}}^{*}|)[c\log(p)+\log(J)+\epsilon]\leq(\bar{J}-|{\tilde{\gamma}}^{*}|)[(c+1)\log(p)+\epsilon]$

for any fixed $\epsilon>0$, as we wished to prove.

#### Case $\phi/b^{\prime\prime}(0)\leq\sigma^{2}$

Then we take $a$ to be a constant smaller than but arbitrarily close to
$\phi/(b^{\prime\prime}(0)\sigma^{2})\leq 1$. Then the right-hand side in
(S.31) is

$\displaystyle\leq
a\log\left((ng)^{\frac{q(\bar{J}-|{\tilde{\gamma}}^{*}|)}{2}}\max_{l\geq|{\tilde{\gamma}}^{*}|+1}\omega(l)\right)\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\left(\frac{1}{p^{ac}(ng)^{aq/2}}\right)^{l-|{\tilde{\gamma}}^{*}|}{l\choose|{\tilde{\gamma}}^{*}|}^{a}{J-|{\tilde{\gamma}}^{*}|\choose
l-|{\tilde{\gamma}}^{*}|}^{1-a}$ (S.35)
$\displaystyle<a\log\left((ng)^{\frac{q(\bar{J}-|{\tilde{\gamma}}^{*}|)}{2}}\max_{l\geq|{\tilde{\gamma}}^{*}|+1}\omega(l)\right)\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\left(\frac{(J-|{\tilde{\gamma}}^{*}|)^{1-a}}{p^{ac}(ng)^{aq/2}}\right)^{l-|{\tilde{\gamma}}^{*}|}{l\choose|{\tilde{\gamma}}^{*}|}$

since $a<1$ implies that
${l\choose|{\tilde{\gamma}}^{*}|}^{a}<{l\choose|{\tilde{\gamma}}^{*}|}$ and
${J-|{\tilde{\gamma}}^{*}|\choose
l-|{\tilde{\gamma}}^{*}|}<(J-|{\tilde{\gamma}}^{*}|)^{l-|{\tilde{\gamma}}^{*}|}$.
Using the Binomial coefficient ordinary generating function gives that the sum
in (S.35) is

(S.36)
$\displaystyle<\left[\left(1-\frac{(J-|{\tilde{\gamma}}^{*}|)^{1-a}}{p^{ac}(ng)^{aq/2}}\right)^{-(|{\tilde{\gamma}}^{*}|+1)}-1\right],$

It is easy to show that the assumption that
$\lim_{n\rightarrow\infty}(|{\tilde{\gamma}}^{*}|+1)(ng)^{aq/2}p^{(c+1)a-1}=\infty$
implies that $p^{ac}(ng)^{aq/2}\gg(J-|{\tilde{\gamma}}^{*}|)^{1-a}$, hence the
definition of the exponential function implies that (S.36) converges to

$\displaystyle\exp\left\\{\frac{(|{\tilde{\gamma}}^{*}|+1)(J-|{\tilde{\gamma}}^{*}|)^{1-a}}{p^{ac}(ng)^{aq/2}}\right\\}-1\asymp\frac{(|{\tilde{\gamma}}^{*}|+1)(J-|{\tilde{\gamma}}^{*}|)^{1-a}}{p^{ac}(ng)^{aq/2}},$

since $\lim_{z\rightarrow\infty}(e^{z}-1)/z=1$. Combining this expression with
(S.35) and the bound
$\log\max_{l\geq|{\tilde{\gamma}}^{*}|}\omega(l)\leq(\bar{J}-|{\tilde{\gamma}}^{*}|)[(c+1)\log(p)+\epsilon]$
given in (S.34), we obtain

$\displaystyle(\bar{J}-|{\tilde{\gamma}}^{*}|)[\log\left((ng)^{q/2}\right)+(c+1)\log(p)+\epsilon]\frac{(|{\tilde{\gamma}}^{*}|+1)(J-|{\tilde{\gamma}}^{*}|)^{1-a}}{p^{ac}(ng)^{aq/2}}.$

Since $(J-|{\tilde{\gamma}}^{*}|)<p$, we have
$(J-|{\tilde{\gamma}}^{*}|)^{1-a}/p^{ac}<1/p^{a(c+1)-1}$, concluding the
proof.

###  S11.2. Part (ii)

Let
$\lambda_{\gamma}=\frac{\mu^{T}\mu}{2\max\\{\sigma^{2},\phi/b^{\prime\prime}(0)\\}\log(\mu^{T}\mu)}$.
The proof strategy is to split $\tilde{p}(S^{c}\mid y)$ into two terms
corresponding to models with smaller and larger size than
$|{\tilde{\gamma}}^{*}|$.

$\displaystyle E_{F_{0}}\left(\tilde{p}(S^{c}\mid
y)\right)=E_{F_{0}}\left(\sum_{l=0}^{|{\tilde{\gamma}}^{*}|}\sum_{|\gamma|=l,\gamma\not\supset{\tilde{\gamma}}^{*}}\tilde{p}(\gamma\mid
y)\right)+E_{F_{0}}\left(\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\sum_{|\gamma|=l,\gamma\not\supset{\tilde{\gamma}}^{*}}\tilde{p}(\gamma\mid
y)\right).$

Consider the second term. Using expression (S.29) proven in Theorem 3 Part
(ii), there exists a finite
$n_{0}=\max_{|\gamma|>|{\tilde{\gamma}}^{*}|,\gamma\not\supset{\tilde{\gamma}}^{*}}n_{0\gamma}$
such that

(S.37) $\displaystyle
E_{F_{0}}\left(\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\sum_{|\gamma|=l,\gamma\not\supset{\tilde{\gamma}}^{*}}\tilde{p}(\gamma\mid
y)\right)\leq\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\sum_{|\gamma|=l,\gamma\not\supset{\tilde{\gamma}}^{*}}\left(\frac{e^{-\lambda_{\gamma}}}{(ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}}\frac{p(\gamma)}{p({\tilde{\gamma}}^{*})}\right)^{b}$

for all $n\geq n_{0}$ and any fixed $b<1$, which we take to be arbitrarily
close to 1. The existence of such a finite $n_{0}$ uniformly bounding all
$n_{0\gamma}$ follows from Condition (D4), specifically using that
$e^{\lambda_{\gamma}}(ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}p({\tilde{\gamma}}^{*})/p(\gamma)$
is uniformly bounded by
$e^{\bar{\lambda}}(ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}p({\tilde{\gamma}}^{*})/p(\gamma)$,
which under Condition (D4) converges to infinity, and arguing as in (S.25).

Let $q=\min\\{p_{1},\ldots,p_{J}\\}$ be the size of the smallest group and
$\bar{\lambda}=\min_{|\gamma|>|{\tilde{\gamma}}^{*}|,\gamma\not\subset{\tilde{\gamma}}^{*}}\lambda_{\gamma}$.
Using the expression for $p(\gamma)/p({\tilde{\gamma}}^{*})$ and noting that
there are ${J\choose l}-{J-|{\tilde{\gamma}}^{*}|\choose
l-|{\tilde{\gamma}}^{*}|}$ models of size $l$ such that
$\gamma\not\supset{\tilde{\gamma}}^{*}$, gives that (S.37) is bounded above by

$\displaystyle\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\frac{e^{-b\bar{\lambda}}}{\left(p^{bc}(ng)^{bq/2}\right)^{l-|{\tilde{\gamma}}^{*}|}}{l\choose|{\tilde{\gamma}}^{*}|}^{b}{J-|{\tilde{\gamma}}^{*}|\choose
l-|{\tilde{\gamma}}^{*}|}^{-b}\left[{J\choose
l}-{J-|{\tilde{\gamma}}^{*}|\choose l-|{\tilde{\gamma}}^{*}|}\right]$
$\displaystyle\leq\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\frac{e^{-b\bar{\lambda}}}{\left(p^{bc}(ng)^{bq/2}\right)^{l-|{\tilde{\gamma}}^{*}|}}{l\choose|{\tilde{\gamma}}^{*}|}\left(\frac{l-|{\tilde{\gamma}}^{*}|}{J-|{\tilde{\gamma}}^{*}|}\right)^{b(l-|{\tilde{\gamma}}^{*}|)}J^{l}$
$\displaystyle<e^{-b\bar{\lambda}}J^{|{\tilde{\gamma}}^{*}|}\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\frac{J^{l-|{\tilde{\gamma}}^{*}|}}{\left(p^{bc}(ng)^{bq/2}\right)^{l-|{\tilde{\gamma}}^{*}|}}\left(\frac{\bar{J}-|{\tilde{\gamma}}^{*}|}{J-|{\tilde{\gamma}}^{*}|}\right)^{b(l-|{\tilde{\gamma}}^{*}|)}{l\choose|{\tilde{\gamma}}^{*}|}$
$\displaystyle=e^{-b\bar{\lambda}}J^{|{\tilde{\gamma}}^{*}|}\sum_{l=|{\tilde{\gamma}}^{*}|+1}^{\bar{J}}\left(\frac{J^{1-b}(\bar{J}-|{\tilde{\gamma}}^{*}|)^{b}}{(1-|{\tilde{\gamma}}^{*}|/J)^{b}p^{bc}(ng)^{bq/2}}\right)^{l-|{\tilde{\gamma}}^{*}|}{l\choose|{\tilde{\gamma}}^{*}|}$

since $b<1$ implies that
${l\choose|{\tilde{\gamma}}^{*}|}^{b}<{l\choose|{\tilde{\gamma}}^{*}|}$. Note
that this expression is analogous to (S.35). Therefore, we may use the
Binomial coefficient ordinary generating function to carry out the sum and the
exponential function to obtain a simpler asymptotically equivalent expression,
to show that

$\displaystyle\eqref{eq:bound_pp_bignonspur}\leq
e^{-b\bar{\lambda}}J^{|{\tilde{\gamma}}^{*}|}\frac{(|{\tilde{\gamma}}^{*}|+1)J^{1-b}(\bar{J}-|{\tilde{\gamma}}^{*}|)^{b}}{(1-|{\tilde{\gamma}}^{*}|/J)^{b}p^{bc}(ng)^{bq/2}}$
(S.38) $\displaystyle\leq e^{(|{\tilde{\gamma}}^{*}|+1)\log
J}\frac{(|{\tilde{\gamma}}^{*}|+1)(\bar{J}-|{\tilde{\gamma}}^{*}|)^{b}}{[e^{\bar{\lambda}}p^{c}(ng)^{q/2}]^{b}}=\frac{(|{\tilde{\gamma}}^{*}|+1)e^{(|{\tilde{\gamma}}^{*}|+1)\log
J+\log{\bar{J}}}}{[e^{\bar{\lambda}}p^{c}(ng)^{q/2}]^{b}}$

for large enough $n$.

Consider now the models of size $|\gamma|\leq|{\tilde{\gamma}}^{*}|$. Let
$\underline{\lambda}=\min_{|\gamma|\leq|{\tilde{\gamma}}^{*}|}\lambda_{\gamma}/\max\\{|{\tilde{\gamma}}^{*}|-|\gamma|,1\\}$,
$q^{\prime}=\max\\{p_{1},\ldots,p_{J}\\}$ the largest group size and note that

$\frac{e^{-\lambda_{\gamma}}}{(ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}}\leq\frac{e^{-\underline{\lambda}\max\\{|{\tilde{\gamma}}^{*}|-|\gamma|,1\\}}}{(ng)^{(|\gamma|-|{\tilde{\gamma}}^{*}|)/2}}.$

Using again Theorem 3 Part (ii), there exists a finite
$n_{0}^{\prime}=\max_{|\gamma|\leq|{\tilde{\gamma}}^{*}|}n_{0\gamma}$ such
that

$\displaystyle
E_{F_{0}}\left(\sum_{l=0}^{|{\tilde{\gamma}}^{*}|}\sum_{|\gamma|=l}\tilde{p}(\gamma\mid
y)\right)\leq\sum_{l=0}^{|{\tilde{\gamma}}^{*}|}\sum_{|\gamma|=l}\left(\frac{e^{-\lambda_{\gamma}}}{(ng)^{\frac{(p_{\gamma}-p_{{\tilde{\gamma}}^{*}})}{2}}}\frac{p(\gamma)}{p({\tilde{\gamma}}^{*})}\right)^{b}$
(S.39) $\displaystyle\leq
e^{-\underline{\lambda}}{J\choose|{\tilde{\gamma}}^{*}|}+\sum_{l=0}^{|{\tilde{\gamma}}^{*}|-1}\left(\frac{e^{\underline{\lambda}}}{p^{bc}(ng)^{q^{\prime}/2}}\right)^{l-|{\tilde{\gamma}}^{*}|}\left(\frac{{J\choose|{\tilde{\gamma}}^{*}|}}{{J\choose
l}}\right)^{b}{J\choose l}$

for all $n>n_{0}^{\prime}$ and any fixed $b<1$, the right-hand side following
from noting that there are ${J\choose l}$ models of size
$|\gamma|=l<|{\tilde{\gamma}}^{*}|$ and that
$(ng)^{p_{{\tilde{\gamma}}^{*}}-p_{\gamma}}\leq(ng)^{q^{\prime}}$. The
existence of a finite $n_{0}^{\prime}$ uniformly bounding all $n_{0\gamma}$
follows from Condition (D5), specifically bounding $\lambda_{\gamma}$ with
$\underline{\lambda}$ and arguing as in (S.26).

Using that ${J\choose l}^{1-b}<J^{l(1-b)}$ and
${J\choose|{\tilde{\gamma}}^{*}|}\leq J^{|{\tilde{\gamma}}^{*}|}$, the
geometric series, and simple algebra we obtain

$\displaystyle\eqref{eq:bound_pp_smallnonspur}$ $\displaystyle\leq
e^{-\underline{\lambda}+|{\tilde{\gamma}}^{*}|\log
J}+e^{|{\tilde{\gamma}}^{*}|\log
J}\left[e^{-\underline{\lambda}}p^{bc}(ng)^{q^{\prime}/2}\right]^{|{\tilde{\gamma}}^{*}|}\sum_{l=0}^{|{\tilde{\gamma}}^{*}|-1}\left(\frac{e^{\underline{\lambda}}J^{(1-b)}}{\left(p^{bc}(ng)^{q^{\prime}/2}\right)}\right)^{l}$
$\displaystyle<e^{-\underline{\lambda}+|{\tilde{\gamma}}^{*}|\log
J}+e^{|{\tilde{\gamma}}^{*}|\log
J}[e^{-\underline{\lambda}}p^{bc}(ng)^{q^{\prime}/2}]^{|{\tilde{\gamma}}^{*}|+1}$
$\displaystyle<\frac{e^{|{\tilde{\gamma}}^{*}|\log
J}}{e^{\underline{\lambda}}}+\frac{e^{|{\tilde{\gamma}}^{*}|\log
J}}{e^{(|{\tilde{\gamma}}^{*}|+1)[\underline{\lambda}-c\log(p)-\log(ng^{q^{\prime}/2})]}}.$

The proof concludes by combining this expression with (S.38), and noting that
$\bar{J}\leq J$.

##  S12. Supplementary results

###  S12.1. Computation times and posterior inference in Poisson regression

$p=10$, $n\in\\{100,500,1000,5000\\}$ |  $n=500$, $p\in\\{5,10,25,50\\}$
---|---
|
|
Figure S1. Poisson simulation. The top panels show the average run time
(seconds) in a single-core i7 processor with Ubuntu 20.04 operating system,
and the bottom panels the average posterior inclusion probabilities for
variables that are truly active ($\beta_{j}^{*}\neq 0$) and truly inactive
($\beta_{j}^{*}=0$). The left panels correspond to fixed $p=10$ and various
$n$, and the right panels to fixed $n=500$ and various $p$

We illustrate the computation times and quality of inference in a setting
where the data are truly Poisson with $\beta^{*}=(0,\ldots,0,0.5,1)$ and
$z_{i}\sim N(0,\Sigma)$ with $\Sigma_{ii}=1$ and $\Sigma_{ij}=0.5$ for all
$i\neq j$. Figure S1 summarizes the results. The top panels show that ALA run
times are significantly more scalable as either $n$ or $p$ grow. The bottom
panels show that ALA tends to assign lower posterior inclusion probabilities
$P(\beta_{j}\neq 0\mid y)$ than LA, particularly for small $n$ or large $p$.
For comparison, the figure also displays results for ALA where no over-
dispersion adjustment is applied. This unadjusted version assigns relatively
high posterior inclusion probabilities to truly inactive variables, even for
large $n$.

###  S12.2. Combining ALA with importance sampling and variable screening

| ALA | ALA - 1 iter | ALA - 2 iter | LA | ALA screening + LA
---|---|---|---|---|---
$z_{1}$ | 0.0343 | 1.0000 | 1.0000 | 1.0000 | 1.000
$z_{2}$ | 0.0548 | 0.1000 | 0.1109 | 0.1147 | -
$z_{3}$ | 0.0619 | 0.0904 | 0.0981 | 0.1014 | -
$z_{4}$ | 0.0436 | 0.0683 | 0.0735 | 0.0760 | -
$z_{5}$ | 0.0450 | 0.0719 | 0.0773 | 0.0796 | -
$z_{6}$ | 0.0434 | 0.0722 | 0.0781 | 0.0806 | -
$z_{7}$ | 0.0528 | 0.0771 | 0.0827 | 0.0854 | -
$z_{8}$ | 0.0464 | 0.0739 | 0.0802 | 0.0829 | -
$z_{9}$ | 0.9743 | 0.9978 | 0.9986 | 0.9987 | 0.9999
$z_{10}$ | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000
Table S1.  Marginal posterior inclusion probabilities in logistic simulation with $p=10$, $n=1000$, $\beta_{1}^{*}=2$, $\beta_{9}^{*}=0.5$, $\beta_{10}^{*}=1$, $\beta_{2}^{*}=\ldots,\beta_{8}^{*}=0$ | ALA | ALA - 1 iter | ALA - 2 iter | LA | ALA screening + LA
---|---|---|---|---|---
$z_{1}$ | 0.0061 | 0.0173 | 0.0179 | 0.0188 | -
$z_{2}$ | 0.0066 | 0.0171 | 0.0174 | 0.0179 | -
$z_{3}$ | 0.0066 | 0.0132 | 0.0138 | 0.0143 | -
$z_{4}$ | 0.9602 | 1.0000 | 1.0000 | 1.0000 | 1.000
$z_{5}$ | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.000
$z_{1}^{2}$ | 0.0049 | 0.0000 | 0.0020 | 0.0113 | -
$z_{2}^{2}$ | 0.0054 | 0.0000 | 0.0031 | 0.0099 | -
$z_{3}^{2}$ | 0.0054 | 0.0000 | 0.0078 | 0.0118 | -
$z_{4}^{2}$ | 0.9996 | 0.0000 | 0.0000 | 0.0269 | 0.1043
$z_{5}^{2}$ | 1.0000 | 0.0000 | 0.0000 | 0.0273 | 0.1059
Table S2.  Marginal posterior inclusion probabilities in Poisson simulation with $p=10$, $n=1000$, $\beta_{4}^{*}=0.5$, $\beta_{5}^{*}=1$ and $\beta_{j}^{*}=0$ for $j\not\in\\{4,5\\}$ |
---|---
Figure S2.  Distribution of importance sampling weights. Left: logistic
simulation with $p=10$, $n=1000$, $\beta_{1}^{*}=2$, $\beta_{9}^{*}=0.5$,
$\beta_{10}^{*}=1$, $\beta_{2}^{*}=\ldots,\beta_{8}^{*}=0$. Right: Poisson
simulation with $p=10$, $n=1000$, $\beta_{4}^{*}=0.5$, $\beta_{5}^{*}=1$ and
$\beta_{j}^{*}=0$ for $j\not\in\\{4,5\\}$

This section discusses the use of ALA as a tool to pre-screen high posterior
probability models. Specifically, we consider a strategy suggested by a
referee based on using importance sampling to re-weight the models sampled
from the ALA posterior. We illustrate that such strategy can lead to
degenerate weights. We also illustrate a referee suggestion on using ALA to
screen out inactive variables, and show that such strategy is effective in our
examples.

Let $\tilde{p}(\gamma\mid y)$ be the posterior probability assigned by ALA and
$\hat{p}(\gamma\mid y)$ that by the Laplace approximation (LA). Assuming that
one has $B$ samples $\gamma^{(1)},\ldots,\gamma^{(B)}$ from
$\tilde{p}(\gamma\mid y)$, importance sampling draws samples from
$\hat{p}(\gamma\mid y)$ by assigning weights
$w(\gamma^{(b)})=\hat{p}(\gamma\mid y)/\tilde{p}(\gamma\mid y)$ for
$b=1,\ldots,B$. For this importance sampling strategy to be effective, it is
necessary that the weights are bounded away from infinity. From a theoretical
point of view, this is in general not true. As shown by our theory, as the
sample size $n\rightarrow\infty$ the ALA and LA posteriors concentrate on two
different models $\tilde{\gamma}^{*}$ and $\gamma^{*}$ respectively, hence the
weight $w(\gamma^{*})\stackrel{{\scriptstyle P}}{{\longrightarrow}}\infty$.

This asymptotic argument does not rule out that in practice for finite $n$
there can be situations where importance sampling is actually effective. To
illustrate this issue we consider two examples. The first example retakes the
logistic regression simulation from Section S12.1 and illustrates a situation
where importance sampling is effective. The second example is a Poisson
regression where one obtains degenerate weights. In both cases we have
$n=1000$, $p=10$.

Consider first the logistic regression example, where recall that
$\beta_{1}^{*}=2$ for the intercept term, $\beta_{9}^{*}=0.5$,
$\beta_{10}^{*}=1$, and $\beta_{2}^{*}=\ldots,\beta_{8}^{*}=0$. Figure S2
(left) shows the distribution of the importance sampling weights across all
$2^{10}$ models for the default ALA expanding at $\beta_{0}=0$, and ALA with
refined $\beta_{0}$ obtained by taking 1 and 2 Newton–Raphson iterations. The
Newton–Raphson iterations led to a significantly improved behavior of the
weights. For the default ALA no infinite weights were observed, the maximum
being $\max_{\gamma}w(\gamma)=64.6$. Table S1 shows the posterior marginal
inclusion probabilities. Interestingly, all versions of ALA and LA provided
strong evidence for including the truly active covariates 9-10 and discarding
the truly inactive covariates 2-8, the main discrepancy being that the default
ALA excluded the intercept.

Consider next a Poisson regression setting where the design matrix has 10
covariates (plus the intercept, whose inclusion is forced into the model).
Covariates 1-5 were drawn from a multivariate Gaussian with zero mean, unit
variances and all pairwise correlations set to 0.5. Covariates 6-10 were the
quadratic effect corresponding to covariates 1-5, e.g. $z_{ij+5}=z_{ij}^{2}$
for $j=1,\ldots,5$. We simulated $n=1000$ observations from a Poisson data-
generating model where only Covariates 4-5 had a linear effect, and no
quadratic effects were present. Specifically, the Poisson log-expectation was
$z_{i}^{T}\beta^{*}$, where $\beta_{4}^{*}=0.5$, $\beta_{5}^{*}=1$ and all
remaining $\beta_{j}^{*}=0$. Figure S2 (right) shows the distribution of the
importance sampling weights across all $2^{10}$ models for the three ALA
versions described above. In all cases there is a significant number of models
with practically infinite weight (up to numerical precision). Interestingly,
the marginal posterior inclusion probabilities in Table S2 reveal that,
despite the weight degeneracy, ALA was effective at detecting both truly
active covariates 4-5, and it also discarded several truly spurious
covariates.

These findings suggest that one may use ALA as a tool to screen out
unpromising covariates, and then using LA to obtain posterior probabilities on
the remaining ones. We tested this strategy in the logistic and Poisson
regression examples. In both cases we screened out parameters with ALA
marginal posterior inclusion probability $\tilde{p}(\gamma_{j}=1\mid y)<0.5$.
The last columns in Tables S1-S2 that the use of screening followed by LA
calculations selected the correct model that only includes truly active
covariates, although in the Poisson case the inclusion probabilities for two
truly inactive parameters were somewhat inflated.

Altogether, these results suggest that there is promise in using ALA as a tool
to pre-screen active variables. In fact, Theorem 1 gives conditions where such
screening has theoretical guarantees of working asymptotically. Regarding the
use of importance sampling, while also potentially interesting, our examples
show that designing effective strategies require a more careful study that is
left as future work.

###  S12.3. Group constraints in linear regression with categorical predictors

Categorical $x_{i1}$ ($\beta_{1}^{*}=0.3$, $J=50$, $p=51$) |  Categorical $x_{i1}$ ($\beta_{1}^{*}=0.6$, $J=50$, $p=51$)
---|---
|
Categorical $x_{i1}$ ($\beta_{1}^{*}=0.3$, $J=5$, $p=6$) |  Categorical $x_{i1}$ ($\beta_{1}^{*}=0.6$, $J=5$, $p=6$)
|
Figure S3. Proportion of correct model selections in Gaussian simulations with
groups defined by a categorical $x_{i1}\in\\{1,2,3\\}$ and true effects
$(-\beta_{1}^{*},\beta_{1}^{*})$

We present a simulation example where one wishes to incorporate group
constraints. We report averaged results across 150 simulations. Continuous
covariates were generated from a multivariate Normal with zero mean, unit
variances and 0.5 pairwise correlations. The regularization parameter for
grLASSO, grMCP and grSCAD was set via 10-fold cross-validation.

In the example there are $J$ groups. The first group corresponds to a
categorical covariate $x_{i1}\in\\{1,2,3\\}$, coded in the design matrix via
two binary indicators $(z_{i1},z_{i2})\in\\{0,1\\}^{2}$, and the remaining
$J-1$ groups to individual continuous covariates $x_{i2},\ldots,x_{ip}$.
Hence, the design matrix $Z$ is $n\times(J+1)$, where $z_{ij+1}=x_{ij}$ for
$j=2,\ldots,J$. The data generating truth was

$y_{i}=-\beta_{1}^{*}z_{i1}+\beta_{1}^{*}z_{i2}+z_{i3}0.5+\epsilon_{i}$

where $\epsilon=(\epsilon_{1},\ldots,\epsilon_{n})\sim\mathcal{N}(0,I)$. We
considered $n\in[50,1000]$, $J\in\\{5,50\\}$ groups and
$\beta_{1}^{*}\in\\{0.3,0.6\\}$.

The top panels in Figure S3 show the proportion of correct model selections in
the $J=50$ covariates case, and the bottom panels for the $J=5$ case. The
proportion of correct selections for ALA-based inference under gMOM priors was
higher than for exact calculations under the gZellner prior, which was in turn
higher than for the three considered penalized likelihood methods.

$n=50$, $x_{i}\sim\mathcal{N}(0,I)$ |  $n=50$, $x_{i}\sim\mathcal{N}(0,V)$
---|---
|
$n=200$, $x_{i}\sim\mathcal{N}(0,I)$ |  $n=200$, $x_{i}\sim\mathcal{N}(0,V)$
|
$n=500$, $x_{i}\sim\mathcal{N}(0,I)$ |  $n=500$, $x_{i}\sim\mathcal{N}(0,V)$
|
Figure S4. Simulated linear regression, gMOM prior. Mean error
$\log(\tilde{p}^{N}(y\mid\gamma)/p^{N}(y\mid\gamma))$ for
$x_{i}\sim\mathcal{N}(0,I)$ and $x_{i}\sim\mathcal{N}(0,V)$, random non-
diagonal $V$

###  S12.4. Poverty line data

The listing below provides the names of the covariates in the poverty line
example, the name of the corresponding variables in the dataset, the codes
corresponding to each category, and a basic descriptive summary providing the
number of individuals in each category.

Table LABEL:tab:poverty_maineffects_alamodel_mle displays the maximum
likelihood parameter estimates for the model selected by ALA (marginal
posterior inclusion probability $>0.5$) in the main effects analysis. Table S4
displays the estimated marginal posterior inclusion probabilities by LA and
ALA for each group of variables corresponding to main effects and
interactions. Table LABEL:tab:poverty_alamodel_mle provides the maximum
likelihood estimates for variables selected by ALA.

For a discussion of the variables selected by ALA and LA, and also GLASSO in
the main effects analysis, see Section 5.3. Regarding GLASSO in the
interactions analysis, it selected 4 main effects and 16 interaction terms.
For 1 out of the 16 interactions (hispanic vs citizenship) the two
corresponding main effects were also selected. The main effects were hispanic,
race, citizenship and occupation. The interactions were female vs. marital,
race and difficulty, marital status vs age, citizenship and hours worked,
hispanic vs. citizenship and worker class, age vs citizen, class worker vs
age, moving state and hours worked, and hours worked vs education, age,
firmsize and moving state.

    -------------------------------------------------------------------------------
    Female (female)

        0     1
    45798 43957

    -------------------------------------------------------------------------------
    Hispanic origin (hispanic)

        0     1
    72001 17754

    -------------------------------------------------------------------------------
    Marital status (marital)

        Divorced      Married NeverMarried    Separated      Widowed
            9088        54068        23419         1986         1194

    -------------------------------------------------------------------------------
    Education level (edu)

       Adv     CG    HSD No HSD  ProfD     SC
      9245  21021  26054   7438    966  25031

    No HSD: no high-school diploma; HSD: high-school diploma; ProfD: professional
    degree; SC: some college; CG: college graduate; Adv: advanced degree (MSc/PhD)

    -------------------------------------------------------------------------------
    Age (age)

      18   19   20   21   22   23   24   25   26   27   28   29   30   31   32   33
     313  578  863 1009 1253 1510 1737 1949 1907 2083 2178 2304 2252 2260 2217 2259
      34   35   36   37   38   39   40   41   42   43   44   45   46   47   48   49
    2283 2379 2264 2388 2470 2419 2540 2413 2277 2276 2346 2389 2355 2357 2386 2300
      50   51   52   53   54   55   56   57   58   59   60   61   62   63   64   65
    2206 2163 2040 1999 2077 1977 1832 1656 1635 1537 1374 1292 1084 1004  764  601

    -------------------------------------------------------------------------------

    Race (race)

                        Asian                     Black Hawaiian/Pacific Islander
                         6388                     11403                       533
              Native American                     White
                         1179                     70252

    -------------------------------------------------------------------------------
    Citizenship status (citizen)

    Born abroad, US parents                 Born US                 Citizen
                        805                   71581                    7964
                Not citizen
                       9405

    -------------------------------------------------------------------------------
    Nativity status (nativity)

          Foreign born    Foreign parents  One native parent Two native parents
                 18770               4451               2872              63573
               Unknown
                    89

    -------------------------------------------------------------------------------
    Occupation (occ)

     architect/engineer   arts/sports/media business operations      computer/maths
                   1644                1362                2399                3318
           construction           education          extraction             farming
                   5072                5508                 143                 559
                finance                food              health        installation
                   2332                3537                7866                3596
                  legal         maintenance          management              office
                    926                3460                8681               13031
          personal care          production          protective               sales
                   2427                6326                1990                7310
                science      social service          technician      transportation
                   1002                1788                 392                5086

    -------------------------------------------------------------------------------
    Firm size (firmsize)

              0    1 (1-24)   2 (25-99) 3 (100-499) 4 (500-999)   5 (>1000)
           1424       23023        9071       12191        5102       38944

    -------------------------------------------------------------------------------
    Presence of any impairment/difficulty (difficulty)

        0     1
    87380  2375

    -------------------------------------------------------------------------------
    Class of worker (classworker)

    Government employee               Other       Self-employed         Wage/salary
                  15615                1455                4816               67869

    -------------------------------------------------------------------------------
    Moved state (movedstate)

    From out US     From US          No
            238        8820       80697

    -------------------------------------------------------------------------------
    Number of hours worked / week (hoursworked)

       35    36    37    38    39    40
     4505  1814   765  1680   237 80754
    -------------------------------------------------------------------------------

Table S3. Model selected by ALA in poverty line example. Point estimates and 95% confidence intervals from a maximum likelihood fit | MLE | 95% CI
---|---|---
age | -0.32 | (-0.364,-0.281)
female | 0.33 | (0.248,0.41)
difficulty | 0.34 | (0.154,0.533)
hispanic | 0.25 | (0.149,0.344)
hours worked | -0.10 | (-0.125,-0.07)
Marital status |  |
Divorced | 0 |
Married | -0.76 | (-0.871,-0.644)
NeverMarried | -0.31 | (-0.43,-0.184)
Separated | 0.24 | (0.061,0.424)
Widowed | -0.21 | (-0.525,0.096)
Education |  |
No HSD | 0 |
HSD | -0.59 | (-0.688,-0.493)
ProfD | -1.83 | (-2.552,-1.117)
SC | -0.96 | (-1.077,-0.85)
CG | -1.44 | (-1.585,-1.294)
Adv | -1.99 | (-2.251,-1.731)
Race |  |
Asian | 0 |
Black | 0.53 | (0.347,0.707)
Hawaiian/Pacific Islander | -0.01 | (-0.446,0.431)
Native American | 0.71 | (0.436,0.984)
White | -0.03 | (-0.202,0.14)
Citizenship |  |
Born abroad, US parents | 0 |
Born US | 0.61 | (0.093,1.135)
Citizen | 0.77 | (0.24,1.304)
Not citizen | 1.15 | (0.62,1.671)
Occupation |  |
architect/Engineer | 0 |
arts/sports/media | 0.68 | (0.017,1.342)
business operations | -0.15 | (-0.84,0.549)
computer/maths | -0.08 | (-0.753,0.593)
construction | 0.88 | (0.283,1.466)
education | 0.99 | (0.381,1.593)
extraction | 0.74 | (-0.311,1.786)
farming | 1.30 | (0.668,1.941)
finance | -0.14 | (-0.838,0.568)
food | 1.37 | (0.782,1.96)
health | 0.65 | (0.056,1.241)
installation | 0.59 | (-0.016,1.201)
legal | -0.85 | (-1.923,0.219)
maintenance | 1.19 | (0.596,1.781)
management | 0.27 | (-0.329,0.868)
office | 0.38 | (-0.208,0.969)
personal care | 1.02 | (0.418,1.617)
production | 0.74 | (0.153,1.334)
protective | 0.62 | (-0.03,1.275)
sales | 0.87 | (0.283,1.459)
science | 0.54 | (-0.235,1.319)
social service | 0.74 | (0.085,1.395)
technician | 0.39 | (-0.563,1.346)
transportation | 0.86 | (0.271,1.454)
Firm size |  |
0 | 0 |
1-24 | -0.37 | (-1.209,0.46)
25-99 | -0.63 | (-1.474,0.213)
100-499 | -0.85 | (-1.694,-0.006)
500-999 | -0.73 | (-1.581,0.12)
$>$1000 | -0.77 | (-1.614,0.065)
Worker class |  |
Wage/salary | 0 |
Self-employed | 0.73 | (0.591,0.863)
Gov employee | -0.44 | (-0.589,-0.283)
Other | 2.08 | (1.247,2.905)
Moved state |  |
From out US | 0 |
From US | -0.84 | (-1.224,-0.451)
No | -1.35 | (-1.733,-0.973)
|  $P(\gamma_{j}=1\mid y)$ (LA) |  $\tilde{P}(\gamma_{j}=1\mid y)$ (ALA)
---|---|---
female | 1.000 | 1.000
hispanic | 1.000 | 1.000
marital | 1.000 | 1.000
education | 1.000 | 1.000
age | 1.000 | 1.000
race | 1.000 | 1.000
citizen | 1.000 | 1.000
occupation | 1.000 | 1.000
firmsize | 1.000 | 1.000
classworker | 1.000 | 1.000
movedstate | 1.000 | 1.000
hoursworked | 1.000 | 1.000
female:marital | 1.000 | 1.000
female:edu | 0.982 | 0.996
difficulty | 0.979 | 0.936
hispanic:marital | 1.000 | 0.530
female:hispanic | 0.501 | 0.989
maritalMarried:edu | 0.000 | 1.000
hispanic:movedstate | 0.000 | 1.000
maritalMarried:movedstate | 0.000 | 1.000
edu:movedstate | 0.000 | 1.000
edu:firmsize | 0.990 | 0.000
female:race | 0.211 | 0.000
hispanic:firmsize | 0.087 | 0.000
hispanic:race | 0.018 | 0.000
race:movedstate | 0.000 | 0.009
female:movedstate | 0.000 | 0.007
female:firmsize | 0.005 | 0.000
nativity | 0.001 | 0.000
Table S4. Poverty data. Terms with either LA or ALA marginal posterior inclusion probability $>0.001$ Table S5. Model selected by ALA in poverty line example. Point estimates and 95% confidence intervals from a maximum likelihood fit | MLE | 95% CI
---|---|---
female | 0.93 | (0.655,1.202)
hispanic | 0.49 | (-0.29,1.272)
maritalMarried | -0.08 | (-1.534,1.368)
maritalNeverMarried | -0.25 | (-1.698,1.194)
maritalSeparated | -14.19 | (-1166.616,1138.246)
maritalWidowed | -0.14 | (-1.13,0.842)
eduHSD | -0.33 | (-1.32,0.661)
eduProfD | -10.74 | (-200.424,178.944)
eduSC | 0.67 | (-0.524,1.854)
eduCG | -0.05 | (-1.115,1.01)
eduAdv | -0.19 | (-1.645,1.257)
age | -0.36 | (-0.406,-0.324)
raceBlack | 0.32 | (0.142,0.493)
raceHawaiian/Pacific Islander | -0.03 | (-0.452,0.39)
raceNative American | 0.49 | (0.221,0.754)
raceWhite | -0.15 | (-0.322,0.011)
citizenBorn US | 0.50 | (0.005,0.993)
citizenCitizen | 0.62 | (0.112,1.121)
citizenNot citizen | 0.99 | (0.496,1.493)
occarts/sports/media | 1.04 | (0.387,1.703)
occbusiness operations | 0.11 | (-0.578,0.803)
occcomputer/maths | 0.04 | (-0.625,0.709)
occconstruction | 1.29 | (0.705,1.885)
occeducation | 1.12 | (0.519,1.724)
occextraction | 0.92 | (-0.095,1.929)
occfarming | 1.63 | (0.996,2.26)
occfinance | 0.16 | (-0.54,0.855)
occfood | 1.61 | (1.025,2.2)
occhealth | 0.91 | (0.315,1.497)
occinstallation | 0.89 | (0.281,1.493)
occlegal | -0.45 | (-1.507,0.612)
occmaintenance | 1.56 | (0.975,2.155)
occmanagement | 0.66 | (0.064,1.255)
occoffice | 0.65 | (0.062,1.235)
occpersonal care | 1.51 | (0.918,2.11)
occproduction | 1.04 | (0.454,1.631)
occprotective | 0.59 | (-0.053,1.236)
occsales | 1.24 | (0.658,1.829)
occscience | 0.62 | (-0.152,1.391)
occsocial service | 0.94 | (0.289,1.593)
occtechnician | 0.51 | (-0.434,1.461)
occtransportation | 1.19 | (0.597,1.776)
difficulty1 | 0.34 | (0.161,0.528)
movedstateFrom US | -1.03 | (-2.637,0.571)
movedstateNo | -1.62 | (-3.198,-0.036)
hoursworked | -0.11 | (-0.138,-0.084)
female:hispanic | -0.18 | (-0.34,-0.017)
female:maritalMarried | -1.19 | (-1.431,-0.944)
female:maritalNeverMarried | 0.02 | (-0.212,0.26)
female:maritalSeparated | 0.01 | (-0.386,0.402)
female:maritalWidowed | 0.04 | (-0.781,0.857)
hispanic:maritalMarried | 0.46 | (0.18,0.729)
hispanic:maritalNeverMarried | -0.04 | (-0.304,0.229)
hispanic:maritalSeparated | 0.27 | (-0.126,0.665)
hispanic:maritalWidowed | 0.33 | (-0.384,1.034)
female:eduHSD | -0.08 | (-0.272,0.114)
female:eduProfD | 0.24 | (-1.261,1.732)
female:eduSC | -0.15 | (-0.374,0.071)
female:eduCG | -0.66 | (-0.933,-0.38)
female:eduAdv | -0.86 | (-1.389,-0.338)
maritalMarried:eduHSD | -0.30 | (-0.64,0.046)
maritalNeverMarried:eduHSD | -0.19 | (-0.526,0.154)
maritalSeparated:eduHSD | -0.13 | (-0.609,0.346)
maritalWidowed:eduHSD | -0.30 | (-1.114,0.52)
maritalMarried:eduProfD | 10.53 | (-179.127,200.195)
maritalNeverMarried:eduProfD | 8.68 | (-180.987,198.35)
maritalSeparated:eduProfD | -1.23 | (-486.004,483.542)
maritalWidowed:eduProfD | 1.33 | (-746.496,749.153)
maritalMarried:eduSC | -0.78 | (-1.145,-0.41)
maritalNeverMarried:eduSC | -0.52 | (-0.869,-0.162)
maritalSeparated:eduSC | -0.17 | (-0.691,0.345)
maritalWidowed:eduSC | -0.37 | (-1.24,0.508)
maritalMarried:eduCG | -0.64 | (-1.095,-0.182)
maritalNeverMarried:eduCG | -0.47 | (-0.91,-0.027)
maritalSeparated:eduCG | -0.55 | (-1.353,0.262)
maritalWidowed:eduCG | 0.00 | (-1.141,1.147)
maritalMarried:eduAdv | -0.89 | (-1.658,-0.127)
maritalNeverMarried:eduAdv | -0.55 | (-1.306,0.205)
maritalSeparated:eduAdv | -1.39 | (-3.487,0.715)
maritalWidowed:eduAdv | -0.10 | (-2.267,2.057)
hispanic:movedstateFrom US | -0.49 | (-1.259,0.269)
hispanic:movedstateNo | -0.32 | (-1.063,0.423)
maritalMarried:movedstateFrom US | -0.01 | (-1.449,1.427)
maritalNeverMarried:movedstateFrom US | 0.39 | (-1.041,1.814)
maritalSeparated:movedstateFrom US | 14.68 | (-1137.747,1167.115)
maritalWidowed:movedstateFrom US | 0.19 | (-0.768,1.14)
maritalMarried:movedstateNo | 0.21 | (-1.211,1.623)
maritalNeverMarried:movedstateNo | 0.37 | (-1.042,1.782)
maritalSeparated:movedstateNo | 14.53 | (-1137.9,1166.962)
eduHSD:movedstateFrom US | -0.02 | (-0.985,0.947)
eduProfD:movedstateFrom US | -0.41 | (-3.5,2.671)
eduSC:movedstateFrom US | -1.09 | (-2.26,0.08)
eduCG:movedstateFrom US | -0.53 | (-1.549,0.491)
eduAdv:movedstateFrom US | -0.67 | (-2.041,0.702)
eduHSD:movedstateNo | -0.04 | (-0.977,0.908)
eduProfD:movedstateNo | -1.56 | (-4.357,1.238)
eduSC:movedstateNo | -1.09 | (-2.24,0.053)
eduCG:movedstateNo | -0.61 | (-1.594,0.373)
eduAdv:movedstateNo | -0.87 | (-2.154,0.419)

###  S12.5. Colon cancer survival

| Selected variables | Concordance Index
---|---|---
| Original | Original + Fake | Overlap |
gMOM ALA | 3 | 3 | 3 | 0.64
gZellner ALA | 11 | 12 | 1 | 0.66
Cox LASSO | 11 | 14 | 9 | 0.69
AFT LASSO | 6 | 1 | 1 | 0.51
Table S6. Colon cancer survival data. Number of selected variables within
original $p=175$, original + 50 fake variables, overlap between these two
sets, and concordance index (leave-one-out cross-validation)

Calon et al. (2012) showed a strong association between the average expression
of 172 genes related to fibroblasts and that of growth factor TGFB were
associated with lower colon cancer survival (time until recurrence), in human
patients in cancer stages 1-3. We perform a deeper analysis of their data
where we seek to identify which individual genes are associated with survival,
in addition to the clinical variable tumor stage, for a total of $p=175$. The
data records survival times for $n=260$ patients. Out of these only 50 were
observed, and the remaining 210 were censored, posing a challenging inference
problem.

Table 1 reports run times for the ALA- and LA-based analyses, for illustration
both under a uniform prior on the model space, and under the Beta-Binomial
prior. The ALA brought significant speed-ups, e.g. from 55 hours to 3.3-11
minutes under the uniform model prior. Given that the computational exercise
under the uniform prior is particularly considerable, we focus on this
analysis and the ALA-based results, as the cost of the LA is impractical in
this setting.

In empirical data it is hard to evaluate what method performs best in terms of
model selection. To explore this issue, we first assessed the predictive
ability of each method via the leave-one-out cross-validated concordance index
(Harrell Jr. et al., 1996). Briefly, higher values of the index indicate a
higher proportion of pairs of predicted survival times that matched the order
of the observed survival times. Cox-LASSO, gMOM-ALA and gZellner-ALA achieved
similar indexes (Table S6), whereas for AFT-LASSO it was lower.

Next, to acknowledge that our goal is not prediction but model selection, we
added 50 fake genes that do not truly have an effect on survival but are
correlated with the original genes. Specifically, we selected the first 50
genes in the original data, and each of the 50 fake genes was obtained by
adding standard Gaussian noise to the original genes. The average correlation
between each fake versus original gene was 0.70. We then run all methods on
the combined original data plus the 50 fake genes. The results are in Table
S6. Interestingly, gMOM-ALA returned the same highest posterior probability
model in both datasets, which included genes ESM1, GAS1 and PDPN, agreeing
with current theory on non-local priors helping control false discoveries. In
contrast the remaining methods returned a different result, in particular for
gZellner-ALA and AFT-LASSO the overlap with the original analysis was poor.

To assess the biological plausibility of the results, according to
genecards.org (Stelzer et al., 2016) ESM1 is related to endothelium disorders,
growth factor receptor binding and gastric cancer networks, GAS1 plays a role
in growth and tumor suppression, and PDPN mediates effects on cell migration
and adhesion, all of these are processes potentially associated with tumor
growth and metastasis. In fact ESM1 was also selected by gZellner-ALA and Cox-
LASSO, and GAS1 by AFT-LASSO. We remark that these results offer no guarantee
that the gMOM-ALA solution matches better the unknown biological truth than
other methods, but they illustrate that the use of ALA led to significant
computational savings and its combination with non-local priors can help
reduce false positives.

## References

  * Bakin [1999] Sergey Bakin.  _Adaptive regression and model selection in data mining problems_.  PhD thesis, The Australian National University, Canberra, Australia, 5 1999.
  * Barbieri and Berger [2004] M.M. Barbieri and J.O. Berger.  Optimal predictive model selection.  _The Annals of Statistics_ , 32(3):870–897, 2004\.
  * Bien et al. [2013] Jacob Bien, Jonathan Taylor, and Robert Tibshirani.  A LASSO for hierarchical interactions.  _Annals of statistics_ , 41(3):1111–1141, 2013\.
  * Breheny and Huang [2015] Patrick Breheny and Jian Huang.  Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors.  _Statistics and Computing_ , 25:173–187, 2015.
  * Burridge [1981] J. Burridge.  A note on maximum likelihood estimation for regression models using grouped data.  _Journal of the Royal Statistical Society B_ , 43(1):41–45, 1981.
  * Calon et al. [2012] A. Calon, E. Espinet, S. Palomo-Ponce, D.V.F. Tauriello, M. Iglesias, M.V. Céspedes, M. Sevillano, C. Nadal, P. Jung, X.H.-F. Zhang, D. Byrom, A. Riera, D. Rossell, R. Mangues, J. Massague, E. Sancho, and E. Batlle.  Dependency of colorectal cancer on a tgf-beta-driven programme in stromal cells for metastasis initiation.  _Cancer Cell_ , 22(5):571–584, 2012.
  * Carbonetto and Stephens [2012] Peter Carbonetto and Matthew Stephens.  Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies.  _Bayesian analysis_ , 7(1):73–108, 2012.
  * Castillo et al. [2015] I. Castillo, J. Schmidt-Hieber, and A.W. van der Vaart.  Bayesian linear regression with sparse priors.  _The Annals of Statistics_ , 43(5):1986–2018, 2015.
  * Dawid [1999] A.P. Dawid.  The trouble with Bayes factors.  Technical report, University College London, 1999.
  * Fan and Li [2001] J. Fan and R. Li.  Variable selection via nonconcave penalized likelihood and its oracle properties.  _Journal of the American Statistical Association_ , 96:1348–1360, 2001.
  * Flood et al. [2020] Sarah Flood, Miriam King, Renae Rodgers, Steven Ruggles, and J Robert Warren.  Integrated public use microdata series, current population survey: Version 7.0 [dataset].  _Minneapolis, MN: IPUMS_ , 2020.  doi: 10.18128/D030.V7.0.
  * Griffin et al. [2020] J.E. Griffin, Latuszynski, and M. F. J. Steel.  In search of lost (mixing) time: Adaptive Markov chain monte carlo schemes for Bayesian variable selection with very large p.  _Biometrika_ , 108(1):53–69, 2020.
  * Harrell Jr. et al. [1996] F.E. Harrell Jr., K.L. Lee, and D.B. Mark.  Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.  _Statistics in medicine_ , 15(4):361–387, 1996\.
  * Hjort and Pollard [2011] N.L. Hjort and D. Pollard.  Asymptotics for minimisers of convex processes.  _arXiv_ , 1107.3806:1–24, 2011.
  * Hsu et al. [2012] D. Hsu, S. Kakade, and T. Zhang.  A tail inequality for quadratic forms of subgaussian random vectors.  _Electronic Communications in Probability_ , 17(52):1–6, 2012.
  * Huang et al. [2016] Xichen Huang, Jin Wang, and Feng Liang.  A variational algorithm for Bayesian variable selection.  _arXiv_ , 1602.07640:1–33, 2016.
  * Johnson [2013] V.E. Johnson.  Uniformly most powerful bayesian tests.  _Annals of statistics_ , 41(4):1716–1741, 2013\.
  * Johnson and Rossell [2010] V.E. Johnson and D. Rossell.  On the use of non-local prior densities for default Bayesian hypothesis tests.  _Journal of the Royal Statistical Society B_ , 72:143–170, 2010.
  * Johnson and Rossell [2012] V.E. Johnson and D. Rossell.  Bayesian model selection in high-dimensional settings.  _Journal of the American Statistical Association_ , 24(498):649–660, 2012.
  * Jordan et al. [1999] Michael I Jordan, Zoubin Ghahramani, Tommi S Jaakkola, and Lawrence K Saul.  An introduction to variational methods for graphical models.  _Machine learning_ , 37(2):183–233, 1999.
  * Kan [2008] R. Kan.  From moments of sum to moments of product.  _Journal of Multivariate Analalysis_ , 99:542–554, 2008\.
  * Kass et al. [1990] R.E. Kass, L. Tierney, and J.B. Kadane.  The validity of posterior expansions based on Laplace’s method.  _Bayesian and likelihood methods in statistics and econometrics_ , 7:473–488, 1990.
  * Minka [2001] Thomas Peter Minka.  _A family of algorithms for approximate Bayesian inference_.  PhD thesis, Massachusetts Institute of Technology, 4 2001.
  * Muirhead [2009] Robb J Muirhead.  _Aspects of multivariate statistical theory_.  John Wiley & Sons, 2009.
  * Narisetty and He [2014] N.N. Narisetty and X. He.  Bayesian variable selection with shrinking and diffusing priors.  _The Annals of Statistics_ , 42(2):789–817, 2014\.
  * Rahaman-Khan and Shaw [2019] M.H. Rahaman-Khan and J.E.H. Shaw.  Variable selection for accelerated lifetime models with synthesized estimation techniques.  _Statistical Methods in Medical Research_ , 28(3):937–952, 2019.
  * Rossell [2018] D. Rossell.  A framework for posterior consistency in model selection.  _arXiv_ , 1806.04071:1–58, 2018.
  * Rossell and Rubio [2018] D. Rossell and F.J. Rubio.  Tractable bayesian variable selection: beyond normality.  _Journal of the American Statistical Association_ , 113(524):1742–1758, 2018.
  * Rossell and Rubio [2019] D. Rossell and F.J. Rubio.  Additive Bayesian variable selection under censoring and misspecification.  _arXiv_ , 1907.13563:1–57, 2019.
  * Rossell and Telesca [2017] D. Rossell and D. Telesca.  Non-local priors for high-dimensional estimation.  _Journal of the American Statistical Association_ , 112:254–265, 2017.
  * Rue et al. [2009] H. Rue, S. Martino, and N. Chopin.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations.  _Journal of the Royal Statistical Society B_ , 71(2):319–392, 2009.
  * Ruli et al. [2016] E. Ruli, N. Sartori, and L. Ventura.  Improved Laplace approximation for marginal likelihoods.  _Electronic Journal of Statistics_ , 10(2):3986–4009, 2016.
  * Schäfer and Chopin [2013] C. Schäfer and N. Chopin.  Sequential Monte Carlo on large binary sampling spaces.  _Statistics and Computing_ , 23(2):163–184, 2013\.
  * Scheipl et al. [2012] F. Scheipl, L. Fahrmeir, and T. Kneib.  Spike-and-slab priors for function selection in structured additive regression models.  _Journal of the American Statistical Association_ , 107(500):1518–1532, 2012.
  * Schwarz [1978] G. Schwarz.  Estimating the dimension of a model.  _Annals of Statistics_ , 6:461–464, 1978.
  * Scott and Berger [2006] J.G. Scott and J.O. Berger.  An exploration of aspects of Bayesian multiple testing.  _Journal of Statistical Planning and Inference_ , 136(7):2144–2162, July 2006.  doi: 10.1016/j.jspi.2005.08.031.  URL http://dx.doi.org/10.1016/j.jspi.2005.08.031.
  * Shin et al. [2018] M. Shin, A. Bhattacharya, and V.E. Johnson.  Scalable Bayesian variable selection using nonlocal prior densities in ultrahigh-dimensional settings.  _Statistica Sinica_ , 28(2):1053–1078, 2018.
  * Silvapulle and Burridge [1986] M.J. Silvapulle and J. Burridge.  Existence of maximum likelihood estimates in regression models for grouped and ungrouped data.  _Journal of the Royal Statistical Society B_ , 48(1):100–106, 1986.
  * Simon et al. [2011] N. Simon, J. Friedman, T. Hastie, and R. Tibshirani.  Regularization paths for Cox’s proportional hazards model via coordinate descent.  _Journal of Statistical Software_ , 39(5):1, 2011\.
  * Stelzer et al. [2016] G. Stelzer, N. Rosen, I. Plaschkes, S. Zimmerman, M. Twik, S. Fishilevich, T.I. Stein, R. Nudel, I. Lieder, Y. Mazor, S. Kaplan, D. Dahary, D. Warshawsky, Y. Guan-Golan, A. Kohn, N. Rappaport, M. Safran, and D. Lancet.  The Genecards suite: from gene data mining to disease genome sequence analyses.  _Current protocols in Bioinformatics_ , 54(1):1–30, 2016.
  * van der Vaart [1998] A.W. van der Vaart.  _Asymptotic statistics_.  Cambridge University Press, New York, 1998.
  * Yang et al. [2016] Y. Yang, M.J. Wainwright, and M.I. Jordan.  On the computational complexity of high-dimensional Bayesian variable selection.  _The Annals of Statistics_ , 44(6):2497–2532, 2016.
  * Zanella and Roberts [2019] G. Zanella and G. Roberts.  Scalable importance tempering and Bayesian variable selection.  _Journal of the Royal Statistical Society B_ , 81(3):489–517, 2019.
  * Zhang [2010] C.-H. Zhang.  Nearly unbiased variable selection under minimax concave penalty.  _The Annals of Statistics_ , 38(2):894–942, 2010\.