# Strong Screening Rules for Group-based SLOPE Models

Fabio Feser
Department of Mathematics
Imperial College London
London, SW7 2AZ
<EMAIL_ADDRESS>
&Marina Evangelou
Department of Mathematics
Imperial College London
London, SW7 2AZ
<EMAIL_ADDRESS>

###### Abstract

Tuning the regularization parameter in penalized regression models is an
expensive task, requiring multiple models to be fit along a path of
parameters. Strong screening rules drastically reduce computational costs by
lowering the dimensionality of the input prior to fitting. We develop strong
screening rules for group-based Sorted L-One Penalized Estimation (SLOPE)
models: Group SLOPE and Sparse-group SLOPE. The developed rules are applicable
for the wider family of group-based OWL models, including OSCAR. Our
experiments on both synthetic and real data show that the screening rules
significantly accelerate the fitting process. The screening rules make it
accessible for group SLOPE and sparse-group SLOPE to be applied to high-
dimensional datasets, particularly those encountered in genetics.

###  1 Introduction

As the amount of data collected increases, the emergence of high-dimensional
data, where the number of features ($p$) is much larger than the number of
observations ($n$), is becoming increasingly common in fields ranging from
genetics to finance. Performing regression and discovering relevant features
on these datasets is a challenging task, as classical statistical methods tend
to break down. The most popular approach to meeting this challenge is the
Lasso [30], which has given rise to the general penalized regression framework

$\hat{\beta}(\lambda)=\operatorname*{arg\,min}_{\beta\in\mathbb{R}^{p}}\left\\{f(\beta)+\lambda
J(\beta;v)\right\\},$ (1)

where $f$ is the loss function, $J$ is a convex penalty norm, $v\succeq 0$ are
penalty parameters, and $\lambda>0$ is the regularization parameter.

A key aspect of fitting a penalized model is to tune the value of the
regularization parameter, which determines the level of sparsity in the fitted
model. Several approaches exist for tuning this parameter, including cross-
validation [8, 15] and exact solution path algorithms [10], but can often be
computationally expensive. Generally, the model is fit along an $l$-length
path $\lambda_{1}\geq\ldots\geq\lambda_{l}\geq 0$. Finding approaches for
speeding up the model fitting process can lead to large computational savings
when tuning. Screening rules are popular methods that reduce the dimensions of
the input dimensionality by discarding irrelevant features before the fitting
process. Discarding features, especially in high-dimensional settings, has a
tremendous impact on the computational costs.

Denoting the active set of non-zero parameters at $\lambda_{k+1}$ by
$\mathcal{A}_{v}(\lambda_{k+1})=\\{i\in\\{1,\dots,p\\}:\hat{\beta}_{i}(\lambda_{k+1})\neq
0\\}$, the goal of a screening rule is to use the solution at $\lambda_{k}$ to
recover a screened set of features, $\mathcal{S}_{v}(\lambda_{k+1})$, which is
a superset of $\mathcal{A}_{v}(\lambda_{k+1})$. The screened set can then be
used as input for calculating the fitted values. By finding the smallest
possible screened set, that still contains the active set, the number of
irrelevant features included in the optimization process is reduced.

There are two types of screening rules: safe and heuristic. Safe rules, as
suggested by the name, provide guarantees that any variables discarded are in
fact inactive at the solution. The seminal work of El Ghaoui et al. [11]
introduced the Safe Feature Elimination Rule (SAFE) for the lasso, in which
safe regions were constructed to eliminate non-active features. Other examples
of safe rules are given in [2, 25, 34], including sample screening [28].
Heuristic rules, on the other hand, tend to discard more variables, but this
increased screening efficiency can lead to active variables being incorrectly
discarded. Examples of heuristic screening rules are given in [1, 13, 31].
Amongst the proposed heuristic rules, of particular interest is the strong
screening rule framework developed for the lasso [31]. As strong screening
rules can lead to violations, Tibshirani et al. [31] complemented their rules
with a check of the Karush–Kuhn–Tucker (KKT) [16] optimality conditions to
ensure no violations occur. There are also hybrid screening regimes, which
combine safe and heuristic rules [32, 37].

A strong screening rule is formulated through the KKT conditions for Equation
1, given by:

$\mathbf{0}\in\nabla f(\beta)+\lambda\partial J(\beta;v).$ (2)

If the gradient $\nabla f(\hat{\beta}(\lambda_{k+1}))$ was available, the set
$\mathcal{A}_{v}(\lambda_{k+1})$ could be identified by checking the
subdifferential of the norm at zero. That is, identifying for which variables

$-\nabla f(\beta)\in\lambda\partial
J(\mathbf{0};v)=\\{x\in\mathbb{R}^{p}:J^{*}(x;\lambda v)\leq 1\\},$ (3)

where $J^{*}$ refers to the dual norm of $J$. The subdifferential at zero,
$\partial J(\mathbf{0};v)$, can also be seen to be the unit ball of its dual
norm. As the gradient at $\lambda_{k+1}$ is not available in practice, an
approximation of the gradient is used to find a screened subset of the
features, $\mathcal{S}_{v}(\lambda_{k+1})$. The screened variables are
combined with the previously active ones to form the reduced input space,
$\mathcal{E}_{v}(\lambda_{k+1})=\mathcal{S}_{v}(\lambda_{k+1})\cup\mathcal{A}_{v}(\lambda_{k})$.
The fitted values are then calculated by solving Equation 1 using only the
variables in $\mathcal{E}_{v}(\lambda_{k+1})$. Optimality of the fitted values
is checked for using the KKT conditions (Equation 2), with violations added
into $\mathcal{E}_{v}$. This ensures features are not discarded that should be
active.

####  1.1 Contributions

Over the years, several adaptive extensions to the lasso have been proposed,
including the Sorted L-One Penalized Estimation (SLOPE) method [4]. SLOPE
applies the sorted $\ell_{1}$ norm
$J_{\text{slope}}(\beta;v)=\sum_{i=1}^{p}v_{i}|\beta|_{(i)}$, where
$v_{1}\geq\ldots\geq v_{p}\geq 0,|\beta|_{(1)}\geq\ldots\geq|\beta|_{(p)}$.
One key advantage of SLOPE is its ability to control the variable false
discovery rate (FDR) under orthogonal data. Both safe and strong rules have
been proposed for SLOPE [3, 12, 17], as well as exact solution path algorithms
[9, 24]. As SLOPE is a non-separable penalty, safe screening rules require
repeated screening during the fitting process [17]. This can be a
computational bottleneck, as the calculation of the safe regions required for
screening are often expensive.

Strong screening rules, on the other hand, are usually more time efficient as
the extra cost of the screening and KKT checks tend to be outweighed by the
significant savings of fitting on a reduced input space [31]. Motivated by
this, we introduce strong screening rules for two group-based extensions of
SLOPE: Group SLOPE (gSLOPE) [6] and Sparse-group SLOPE (SGS) [14].

Group SLOPE (gSLOPE) is an extension of SLOPE for selecting groups, defined by
the norm

$J_{\text{gslope}}(\beta;w)=\sum_{g=1}^{m}\sqrt{p_{g}}w_{g}\|\beta^{(g)}\|_{2},$
(4)

such that $\beta^{(g)}\in\mathbb{R}^{p_{g}}$ is a vector of the group
coefficients and $p_{g}$ denotes the group sizes [6]. The norm has ordered
penalty weights $w_{1}\geq\ldots\geq w_{g}\geq 0$ (described in Appendix A.1)
which are matched to
$\sqrt{p_{1}}\|\beta^{(1)}\|_{2}\geq\ldots\geq\sqrt{p_{m}}\|\beta^{(m)}\|_{2}$.

Sparse-group SLOPE (SGS) was proposed as an extension to SLOPE and gSLOPE,
applying both variable and group penalization for bi-level selection [14]. For
$\alpha\in[0,1]$ with variable weights $v$ and group weights $w$ (described in
Appendix B.1), the SGS norm is defined as

$J_{\text{sgs}}(\beta;\alpha,v,w)=\alpha
J_{\text{slope}}(\beta;v)+(1-\alpha)J_{\text{gslope}}(\beta;w).$

Both of these approaches were developed for the analysis of features that
naturally belong to groups. This situation is frequently encountered in the
analysis of genetics data, where genes are grouped into pathways for the
completion of a specific biological task. The gSLOPE approach selects groups
of features, whereas SGS performs bi-level selection. Both approaches have
been shown to inherit SLOPE’s ability to control FDR under orthogonal data:
gSLOPE controls the group FDR [6] and SGS controls both variable and group
FDRs [14].

Section 2 introduces a sparse-group screening rule framework. Based on this,
strong screening rules for gSLOPE and SGS are developed and presented in
Sections 3 and 4. The proofs of the theorems and propositions presented in
Sections 2 and 3 are provided in the Appendices A.2 and B.3. Through a series
of simulations and the analysis of real datasets (Sections 5), the proposed
screening rules are shown to significantly reduce computational runtime.
Additionally, due to the reduced input dimensionality offered by screening
rules, issues with convergence in the analysis of large datasets are shown to
be alleviated. These improvements are achieved without affecting the solution
optimality.

The screening rules proposed in this manuscript are further applicable to
group-based Ordered Weighted $\ell_{1}$ (OWL) models. SLOPE is a particular
case of an OWL model through the definition of its weights. As the proposed
screening rules only require that the penalty sequences ($v,w$) are ordered,
they can be used for any group-based OWL models; for example, the Octagonal
Shrinkage and Clustering Algorithm for Regression (OSCAR) model [5]. Results
for group-based OSCAR models are provided in Appendix E.

###### Notation.

The feature space is divided into a set of $m$ non-overlapping groups, where
$\mathcal{G}_{1},\dots,\mathcal{G}_{m}$ denote the set of variable indices for
each group. The set of active groups is denoted by
$\mathcal{A}_{g}=\\{i\in\\{1,\dots,m\\}:\|\hat{\beta}^{(i)}\|_{2}\neq 0\\}$
whereas $\mathcal{Z}=\\{i\in\\{1,\dots,m\\}:\|\hat{\beta}^{(i)}\|_{2}=0\\}$
presents the inactive groups. The set of variable indices of the active and
inactive groups are denoted by $\mathcal{G}_{\mathcal{A}}$ and
$\mathcal{G}_{\mathcal{Z}}$, respectively. The cardinality of a vector $x$ is
denoted by $\operatorname{card}(x)$. The operators $(\cdot)_{\downarrow}$ and
$(\cdot)_{|\downarrow|}$ sort a vector into decreasing and decreasing absolute
form. We use $\preceq$ to denote element-wise less than or equal to. The
operator $\mathcal{O}(\cdot)$ returns the index of a vector sorted in
decreasing absolute form. The cumulative summation operator applied on a
vector is denoted by
$\text{cumsum}(x)=[x_{1},x_{1}+x_{2},\ldots,\sum_{i=1}^{\operatorname{card}(x)}x_{i}]$.

###  2 Screening rule framework

Algorithm 1 Sparse-group screening framework

Input: $\lambda\in\mathbb{R}^{l}$, $\mathbf{X}\in\mathbb{R}^{n\times
p},y\in\mathbb{R}^{n}$

for $k=1$ to $l-1$ do

$\mathcal{S}_{g}(\lambda_{k+1})$ $\leftarrow$ group screening on full input

$\mathcal{S}_{v}(\lambda_{k+1})$ $\leftarrow$ variable screening on
$g\in\mathcal{S}_{g}(\lambda_{k+1})$

$\mathcal{E}_{v}\leftarrow\mathcal{S}_{v}(\lambda_{k+1})\cup\mathcal{A}_{v}(\lambda_{k})$

compute $\hat{\beta}_{\mathcal{E}_{v}}(\lambda_{k+1})$

$\mathcal{K}_{v}\leftarrow$ variable KKT violations on
$\hat{\beta}(\lambda_{k+1})$

while $\operatorname{card}(\mathcal{K}_{v})>0$ do

$\mathcal{E}_{v}\leftarrow\mathcal{E}_{v}\cup\mathcal{K}_{v}$

compute $\hat{\beta}_{\mathcal{E}_{v}}(\lambda_{k+1})$

$\mathcal{K}_{v}\leftarrow$ variable KKT violations on
$\hat{\beta}(\lambda_{k+1})$

end while

end for

Output:
$\hat{\beta}(\lambda_{1}),\ldots,\hat{\beta}(\lambda_{l})\in\mathbb{R}^{p\times
l}$

Sparse-group models apply both variable and group penalization, such that
Equation 1 is extended to include an additional convex penalty function.
Examples of sparse-group models include the Sparse-group Lasso (SGL) [29] and
SGS [14]. Safe screening rules are available for SGL which perform bi-level
screening, i.e. both group and variable screening [21, 33], but there are no
strong rules which perform bi-level screening. The strong screening rule
framework proposed by Tibshirani et al. [31] does not fully extend to sparse-
group or non-separable penalties, whilst the strong screening rule derived for
SGL in Liang et al. [19] applies only group-level screening.

###### Framework.

We introduce a new framework (Algorithm 1) for applying strong screening rules
to sparse-group penalties that allow for bi-level screening, based on the
framework introduced by Tibshirani et al. [31]. In the sparse-group framework,
a screened set of groups is computed, $\mathcal{S}_{g}$, from the full input.
An additional layer of screening is then performed to compute
$\mathcal{S}_{v}$ using $\mathcal{S}_{g}$. This forms the reduced input set
for fitting, $\mathcal{E}_{v}$. The aim of finding $\mathcal{E}_{v}$ is the
same as for the lasso and SLOPE strong rules, but the framework first discards
irrelevant groups. KKT checks are performed on $\mathcal{E}_{v}$ to ensure no
violations have occurred. The framework applied to gSLOPE and SGS is presented
in Appendix D.

Based on the framework described in Algorithm 1, the screening rules for SGS
are derived (Section 4). The screening rule for gSLOPE (Section 3) follows a
similar framework, where $\mathcal{S}_{v}$ is taken as all variables within
the groups of $\mathcal{S}_{g}$. The KKT checks are then performed only on the
groups.

By applying bi-level screening for SGS, a substantially larger proportion of
variables are discarded, than if just group screening were applied (Figure 1).
Using only group screening for SGS, without the additional variable screening
step, resulted in the number of variables in the screened set scaling poorly
with increased dimensionality. This demonstrates that bi-level screening
effectively manages the scaling of SGS under rising dimensions.

(a) Applied to synthetic data (Section 5.1). Data generated under a linear
model for $p=500,5000$. The results are averaged over $100$ repetitions.

(b) Applied to two real genetics datasets (Section 5.2), with groups formed
using pathways. The results are averaged over the nine pathway collections.

Figure 1: The proportion of variables in $\mathcal{S}_{v}$ relative to the
full input for SGS, shown for group and bi-level screening, plotted as a
function of the regularization path, with $95\%$ confidence intervals.

###  3 Group SLOPE

The strong screening rule for gSLOPE presented in this section is formulated
by computing the zero condition of its subdifferential (as per Equation 3)
derived in Theorem 3.1. To derive the subdifferential, we define the operator

$[b]_{\mathcal{G},q}:=(p_{1}^{q}\|b^{(1)}\|_{2},\dots,p_{m}^{q}\|b^{(m)}\|_{2})^{\top}.$

In particular, $[b]_{\mathcal{G}_{\mathcal{Z}},-0.5}$ is the operator applied
only to the inactive groups using the quotient $q=-0.5$.

######  Theorem 3.1 (gSLOPE subdifferential).

The subdifferential for gSLOPE is given by

$\partial
J_{\text{gslope}}(\beta;w)=\begin{dcases}\\{x\in\mathbb{R}^{\operatorname{card}{\mathcal{G}_{\mathcal{Z}}}}:{[x]}_{\mathcal{G}_{\mathcal{Z}},-0.5}\in\partial
J_{\text{slope}}(0;w_{\mathcal{Z}})\\},&\text{at 0.}\\\
w_{g}\sqrt{p_{g}}\frac{\beta^{(g)}}{\|\beta^{(g)}\|_{2}},&\text{otherwise.}\end{dcases}$

The choice of $q=-0.5$ leads to
$J_{\text{gslope}}^{*}(x;w)=J_{\text{slope}}^{*}([x]_{\mathcal{G},-0.5})$,
which allows the gSLOPE subdifferential to be written in terms of the SLOPE
one [6]. Combining the KKT conditions (Equation 2) with the gSLOPE
subdifferential (Theorem 3.1) reveals that a group is zero if
$h(\lambda):=([\nabla
f(\hat{\beta}(\lambda))]_{\mathcal{G},-0.5})_{\downarrow}\in\partial
J_{\text{slope}}(\mathbf{0};\lambda w_{\mathcal{Z}})$. Using the
subdifferential of SLOPE [17], this is given by

$\text{cumsum}(h(\lambda)-\lambda w_{\mathcal{Z}})\preceq\mathbf{0}.$ (5)

This condition is checked efficiently using the algorithm proposed for the
SLOPE strong rule (Algorithm A1) leading to the strong rule for gSLOPE
(Proposition 3.2). The algorithm assumes that the indices for the inactive
predictors will be ordered last in the input $c$ and the features
$|\hat{\beta}|_{\downarrow}$ [17].

######  Proposition 3.2 (Strong screening rule for gSLOPE).

Taking $c=h(\lambda_{k+1})$ and $\phi=\lambda_{k+1}w$ as inputs for Algorithm
A1 returns a superset $\mathcal{S}_{g}(\lambda_{k+1})$ of the active set
$\mathcal{A}_{g}(\lambda_{k+1})$.

The gradient at path value $k+1$ is not available for the computation of
$h(\lambda_{k+1})$, so an approximation is required that does not lead to any
violations in Algorithm A1. By the cumsum condition in this algorithm, an
approximation for a group $g$ is sought such that $h_{g}(\lambda_{k+1})\leq
h_{g}(\lambda_{k})+R_{g}$, where $R_{g}\geq 0$ needs to be determined. An
approximation is found by assuming that $h_{g}(\lambda_{k+1})$ is a Lipschitz
function of $\lambda_{k+1}$ with respect to the $\ell_{1}$ norm, that is,

$\left|h_{g}(\lambda_{k+1})-h_{g}(\lambda_{k})\right|\leq
w_{g}|\lambda_{k+1}-\lambda_{k}|.$

By again noting that
$J_{\text{gslope}}^{*}(x)=J_{\text{slope}}^{*}([x]_{\mathcal{G},-0.5})$, it
can be seen that the assumption is equivalent to those used to derive strong
rules for the lasso and SLOPE [17, 31]. By the reverse triangle inequality,

$|h_{g}(\lambda_{k+1})|\leq|h_{g}(\lambda_{k})|+\lambda_{k}w_{g}-\lambda_{k+1}w_{g},$

leading to the choice $R_{g}=\lambda_{k}w_{g}-\lambda_{k+1}w_{g}$ and the
gradient approximation strong screening rule.

######  Proposition 3.3 (Gradient approximation strong screening rule for
gSLOPE).

Taking $c=h(\lambda_{k})+\lambda_{k}w-\lambda_{k+1}w$ and
$\phi=\lambda_{k+1}w$ as inputs for Algorithm A1, and assuming that for any
$k\in\\{1,\dots,l-1\\}$,

$\left|h_{g}(\lambda_{k+1})-h_{g}(\lambda_{k})\right|\leq
w_{g}|\lambda_{k+1}-\lambda_{k}|,\;\forall g=1,\dots,m,$

and $\mathcal{O}(h(\lambda_{k+1}))=\mathcal{O}(h(\lambda_{k}))$, then the
algorithm returns a superset $\mathcal{S}_{g}(\lambda_{k+1})$ of the active
set $\mathcal{A}_{g}(\lambda_{k+1})$.

###  4 Sparse-group SLOPE

This section presents the group and variable screening rules for SGS. They are
derived using the SGS KKT conditions, formulated in terms of SLOPE and gSLOPE
(by the sum rule of subdifferentials):

$-\nabla f(\beta)\in\lambda\alpha\partial
J_{\text{slope}}(\beta;v)+\lambda(1-\alpha)\partial
J_{\text{gslope}}(\beta;w).$ (6)

####  4.1 Group screening

For inactive groups, the KKT conditions (Equation 6) for SGS are

$\displaystyle(\nabla f(\beta)+\lambda\alpha\partial
J_{\text{slope}}(\mathbf{0};v))_{\mathcal{G}_{\mathcal{Z}}}\in\lambda(1-\alpha)\partial
J_{\text{gslope}}(\mathbf{0};w_{\mathcal{Z}}),$
$\displaystyle\underset{\text{by Equation
\ref{eqn:gslope_sub_condition}}}{\implies}$
$\displaystyle\text{cumsum}\Bigl{(}\bigl{(}[\nabla
f(\beta)+\lambda\alpha\partial
J_{\text{slope}}(\mathbf{0};v)]_{\mathcal{G}_{\mathcal{Z}},-0.5}\bigr{)}_{\downarrow}-\lambda(1-\alpha)w_{\mathcal{Z}}\Bigr{)}\preceq\mathbf{0}.$
(7)

The problem reduces to that of the gSLOPE screening rule (Section 3), with
input given by $c=([\nabla f(\beta)+\lambda\alpha\partial
J_{\text{slope}}(\mathbf{0};v)]_{\mathcal{G},-0.5})_{\downarrow}$ and
$\phi=\lambda(1-\alpha)w$. To determine the form of the quantity $\partial
J_{\text{slope}}(\mathbf{0};v)$, the term inside the
$[\cdot]_{\mathcal{G}_{\mathcal{Z}},-0.5}$ operator needs to be as small as
possible for Equation 7 to be satisfied. This term is found to be the soft
thresholding operator, $S(\nabla f(\beta),\lambda\alpha):=\text{sign}(\nabla
f(\beta))(|\nabla f(\beta)|-\lambda\alpha)_{+}$ (with the derivation presented
in Appendix B.2).

By using the soft-thresholding operator, a valuable connection between SGS and
SGL is made, as this operator is used in the gradient update step for SGL
[29]. Such a connection has the potential to lead to new and more efficient
optimization approaches for SGS that are more closely related to those used to
solve SGL, similar to the recently developed coordinate descent algorithm for
SLOPE [18].

Using this function, the (non-approximated) strong group screening rule for
SGS is given in Proposition B.1. Using a similar Lipschitz assumption as for
the gSLOPE rule gives the gradient approximation strong group screening rule
for SGS (Proposition 4.1).

######  Proposition 4.1 (Gradient approximation strong group screening rule
for SGS).

Let $\tilde{h}(\lambda):=([S(\nabla
f(\hat{\beta}(\lambda)),\lambda\alpha)]_{\mathcal{G},-0.5})_{\downarrow}$.
Taking
$c=\tilde{h}(\lambda_{k})+\lambda_{k}(1-\alpha)w-\lambda_{k+1}(1-\alpha)w$ and
$\phi=\lambda_{k+1}(1-\alpha)w$ as inputs for Algorithm A1, and assuming that
for any $k\in\\{1,\dots,l-1\\}$,

$\left|\tilde{h}_{g}(\lambda_{k+1})-\tilde{h}_{g}(\lambda_{k})\right|\leq(1-\alpha)w_{g}|\lambda_{k+1}-\lambda_{k}|,\forall
g=1,\dots,m,$

and
$\mathcal{O}(\tilde{h}(\lambda_{k+1}))=\mathcal{O}(\tilde{h}(\lambda_{k}))$,
then the algorithm returns a superset $\mathcal{S}_{g}(\lambda_{k+1})$ of
$\mathcal{A}_{g}(\lambda_{k+1})$.

####  4.2 Variable screening

Whilst the group screening for SGS returns a superset, $\mathcal{S}_{g}$, of
the active group set, exploiting the sparse-group penalization of SGS allows
for the screened set to be further reduced by also applying variable
screening. The KKT conditions (Equation 6) for a zero variable,
$j\in\mathcal{G}_{g}$, in an active group, $g\in\mathcal{A}_{g}$, are

$-\nabla_{j}f(\beta)\in\lambda\alpha\partial
J_{\text{slope}}(\mathbf{0};v_{j}).$ (8)

The gSLOPE subdifferential term vanishes as the numerator is zero in Theorem
3.1. The problem reduces to that of SLOPE variable screening, applied only to
the variables that belong to groups in $\mathcal{A}_{g}$ and scaled by
$\alpha$. The gradient approximated rule is formalized in Proposition 4.2 (the
non-approximated version is presented in Proposition B.2).

######  Proposition 4.2 (Gradient approximation strong variable screening rule
for SGS).

Let $\bar{h}(\lambda)=(\nabla f(\hat{\beta}(\lambda)))_{|\downarrow|}$. Taking
$c=|\bar{h}(\lambda_{k+1})|+\lambda_{k}\alpha v-\lambda_{k+1}\alpha v$ and
$\phi=\lambda_{k+1}\alpha v$ for only the variables in the groups in
$\mathcal{A}_{g}(\lambda_{k+1})$ as inputs for Algorithm A1 , and assuming
that for any $k\in\\{1,\dots,l-1\\}$,

$\left|\bar{h}_{j}(\lambda_{k+1})-\bar{h}_{j}(\lambda_{k})\right|\leq\alpha
v_{j}|\lambda_{k+1}-\lambda_{k}|,\forall
j\in\mathcal{G}_{\mathcal{A}_{g}(\lambda_{k+1})},$

and $\mathcal{O}(\bar{h}(\lambda_{k+1}))=\mathcal{O}(\bar{h}(\lambda_{k}))$,
then the algorithm returns a superset $\mathcal{S}_{v}(\lambda_{k+1})$ of
$\mathcal{A}_{v}(\lambda_{k+1})$.

In practice $\mathcal{A}_{g}(\lambda_{k+1})$ is not available, as this is
exactly what we are trying to superset with any screening rule. However,
Proposition 4.1 states that it is contained within
$\mathcal{S}_{g}(\lambda_{k+1})$, so that the screened set can be used instead
and complemented with KKT checks to ensure there are no violations, which
would be performed anyway. For the checks, the group KKT conditions (Equation
6) are checked. For any active groups or groups violating the check
(indicating that they should be active), the variable KKT conditions (Equation
8) are checked on the variables within those groups, to identify those that
need to be added to $\mathcal{E}_{v}$.

###  5 Results

In this section, the effectiveness of the screening rules for gSLOPE and SGS
is illustrated through the analysis of both synthetic (Section 5.1) and real
data (Section 5.2). Any mentions of $\mathcal{E},\mathcal{A}$ in reference to
group and variable metrics refers to $\mathcal{E}_{g},\mathcal{A}_{g}$ and
$\mathcal{E}_{v},\mathcal{A}_{v}$ respectively. For SGS, $\mathcal{E}_{g}$ is
the groups with representation in $\mathcal{E}_{v}$ (the groups fitted).
Computational information is presented in Appendix F.1.

####  5.1 Synthetic data analysis

###### Set up.

For the synthetic data, a multivariate Gaussian design matrix,
$\mathbf{X}\sim\mathcal{N}(\mathbf{0},\boldsymbol{\Sigma})\in\mathbb{R}^{400\times
p}$, was used and the within-group correlation set to $\Sigma_{i,j}=\rho$,
where $i$ and $j$ belong to the same group. The correlation and number of
features were varied between $\rho\in\\{0,0.3,0.6,0.9\\}$ and
$p\in\\{500,1625,2750,3875,5000\\}$, producing $20$ simulation cases. Each
simulation case was repeated $100$ times. Two models were considered: linear
and logistic. For the linear model, the output was generated as
$y=\mathbf{X}\beta+\mathcal{N}(0,1)$ and for the logistic model the class
probabilities were calculated using
$\sigma(\mathbf{X}\beta+\mathcal{N}(0,1))$, where $\sigma$ is the sigmoid
function. Groups of sizes between $3$ and $25$ were considered, of which
$15\%$ were set to active. Within each group, $30\%$ of the variables were set
to active with the signal sampled from $\beta\sim\mathcal{N}(0,5)$.

gSLOPE and SGS were each fit along a log-linear path of $l=50$ regularization
parameters using warm starts. The first path value $\lambda_{1}$ was set to
the exact value at which the first group enters the model (given in Appendix
A.3 for gSLOPE and Appendix B.4 for SGS). The final regularization value was
set to $\lambda_{50}=0.05\lambda_{1}$. The data was $\ell_{2}$ standardized
and for the linear model an intercept was fitted. Both models had FDR-control
parameters set to $0.05$, and $\alpha=0.95$ for SGS. Even though the models
were fit using the adaptive three operator splitting algorithm (ATOS) [26], it
should be noted that the screening rules are independent of the choice of
fitting algorithm.

Primarily, the results for the linear model are presented, and the
corresponding plots for the logistic model are presented in Appendix F.3.2.

###### Screening efficiency.

By comparing the sizes of the fitting set $(\mathcal{E})$ to the active set
($\mathcal{A})$, we can understand the difference between the minimum
dimensionality needed against the dimensionality provided by the screening
rules. As expected, these sizes increase as $\lambda$ decreases (Figure 2). In
particular, the gap between sets widens as more features enter the model. The
size of $\mathcal{E}$ remains far below the size of the full input space even
at the lowest value of $\lambda$, showing that the screening rules have a
benefit along the whole path. This finding is found to be independent of
$\rho$, $p$, and model fitted (see Appendix F.3).

Figure 2: The number of groups/variables in $\mathcal{E},\mathcal{A}$ for both
gSLOPE and SGS as a function of the regularization path for the linear model
with $p=2750,\rho=0.6,m=197$. The results are averaged over $100$ repetitions,
with the shaded regions corresponding to $95\%$ confidence intervals.

Figure 3: The proportion of groups/variables in $\mathcal{E},\mathcal{A}$,
relative to the full input, shown for gSLOPE and SGS. This is shown as a
function of the correlation ($\rho$), averaged over all cases of the input
dimension ($p$), with $100$ repetitions for each $p$, for both linear and
logistic models, with standard errors shown.

The performance of the screening rules are generally similar across linear and
logistic models (Figure 3). As the within-group correlation increases, the
size of both the screened and active sets decreases, causing the variables to
move in the same direction within the same group. Due to the shape of the
convex penalty of SLOPE, highly correlated features are clustered together
[36]. This leads to smaller active sets, and in turn, also smaller screened
sets. Additionally, it can be seen that the size difference between the
screened and active sets decreases with correlation. Across the different
correlation values, the linear model had smaller set sizes than the logistic
one.

###### Runtime performance.

A key metric of performance for a screening rule is the time taken to fit a
path of models. Increasing the value of $p$ demonstrates the computational
cost reduction of applying a screening rule (Figure 4). For the smaller values
of $p$, the runtime tends to be similar whether screening is applied or not.
However, once the input dimension increases, the benefit of screening can be
seen clearly. Additionally, increasing the within-group correlation tends to
lead to smaller active and screened sets, but the benefit of screening appears
to be robust under different correlations.

Figure 4: Runtime (in seconds) for fitting $50$ models along a path, shown for
screening against no screening as a function of $p$, broken down into
different correlation cases, for the linear model. The results are averaged
over $100$ repetitions, with standard errors shown.

Figure 5: Runtime (in seconds) for fitting $100$ models along a path for
screening against no screening for gSLOPE and SGS, averaged across the nine
collections of pathways for each dataset, with standard errors shown.

A clear improvement with regards to runtime can be seen aggregating the
results across all simulation cases (Table 1). In particular, for both models
considered, screening halves the runtime for fitting $50$ models for SGS.

####  5.2 Real data experiments

Table 1: Runtime (in seconds) for fitting $50$ models along a path, shown for
screening against no screening, for the linear and logistic models. The
results are averaged across all cases of the correlation ($\rho$) and
dimensionality ($p$), with standard errors shown.

Method | Type | Screen (s) | No screen (s)
---|---|---|---
gSLOPE | Linear | $1016\pm 21$ | $1623\pm 27$
gSLOPE | Logistic | $814\pm 8$ | $1409\pm 11$
SGS | Linear | $735\pm 15$ | $1830\pm 34$
SGS | Logistic | $407\pm 2$ | $859\pm 6$

Table 2: The sizes of the sets $\mathcal{A}$ and $\mathcal{E}$ (with standard
errors), averaged across $100$ path values and pathway collections for each of
the two datasets. SGS var/grp shows the corresponding metrics for
$\mathcal{A}_{v},\mathcal{E}_{v}$ and $\mathcal{A}_{g},\mathcal{E}_{g}$ for
the bi-level screening procedure.

Method | Dataset | $\operatorname{card}(\mathcal{A})$ | $\operatorname{card}(\mathcal{E})$
---|---|---|---
gSLOPE | Cancer | $66\pm 1$ | $120\pm 2$
gSLOPE | Colitis | $32\pm 1$ | $56\pm 2$
SGS grp | Cancer | $50\pm 1$ | $115\pm 2$
SGS grp | Colitis | $18\pm 1$ | $87\pm 3$
SGS var | Cancer | $71\pm 1$ | $212\pm 4$
SGS var | Colitis | $25\pm 1$ | $209\pm 6$

###### Datasets.

The screening rules were applied to two gene expression datasets with binary
responses. The datasets were previously analyzed to measure the predictive
performance of sparse-group regression models on large datasets [14, 29]. The
cancer dataset contains gene data for $60$ breast cancer patients who had been
treated with tamoxifen for 5 years, classified into binary labels depending on
whether the cancer recurred [20]. The colitis dataset uses transcriptional
profiles in peripheral blood mononuclear cells of $127$ patients to determine
which patients have an inflammatory bowel disease (ulcerative colitis or
Crohn’s). Of the $127$ samples, $42$ of them are controls [7].

The genes of both datasets were assigned to pathways (groups) that were
obtained from the Molecular Signature Database.111https://www.gsea-
msigdb.org/gsea/msigdb/human/collections.jsp Nine collections of pathways were
downloaded and as the pathways included different genes, nine unique design
matrices for each dataset were created, each with a unique grouping structure.
This gave $18$ design matrices in total to which the screening rules were
applied.

Both gSLOPE and SGS were fitted with their FDR-control parameters set to
$0.01$ and for SGS $\alpha=0.99$. Each model was fitted along a path of $100$
regularization parameters, with $\lambda_{1}$ set as described in Appendices
A.3 and B.4, and $\lambda_{100}=0.01\lambda_{1}$. The data was $\ell_{2}$
standardized and no intercept was used.

###### Results.

For both gSLOPE and SGS, the screening rules lead to considerably faster
runtimes (Figure 5). In particular, the use of the screening rules
significantly benefits the gSLOPE approach. With the reduced input feature
space, any issues with convergence of the fitting algorithm are substantially
decreased. For the cancer dataset, $186$ model fits (out of the $900$) did not
reach convergence with no screening, compared to only $16$ with screening. A
similar trend occurred for the colitis dataset. As gSLOPE applies no variable
penalization, it is forced to fit all variables within a group. For datasets
with large groups, such as those considered here, this leads to a problematic
fitting process which can include many noisy variables.

The analysis of the real data further illustrates the benefits of the bi-level
screening to the runtime and performance of SGS. Figure 1(b) illustrates that
for these large genetics datasets, the bi-level screening allows the input
dimensionality for SGS to be reduced to a much greater extent than by just
group screening.

Both screening rules can drastically reduce the input dimensionality as seen
in Table 2. For the colitis dataset, the SGS screening rule reduces the input
dimensionality to just $2\%$ of the full space. This comes without the cost of
solution consistency. The estimated model coefficients with and without
screening were very close to each other, with their $\ell_{2}$ distances of
order $10^{-8}$ (Appendices F.2 and F.4).

####  5.3 KKT violations

The screening rules rely on assumptions that can fail. In these cases, KKT
checks are implemented to ensure that no active variables are excluded from
$\mathcal{E}_{v}$. A KKT violation occurs when a variable is not included in
the screening set, $\mathcal{S}_{v}$, but should be according to the KKT
optimality conditions (Equation 2), in which case it is added into the fitting
set, $\mathcal{E}_{v}$. KKT violations are very rare for gSLOPE (Figure 6),
occurring on the simulated data infrequently toward the start of the path. On
the real data, no KKT violations occurred for gSLOPE.

(a) Group violations for gSLOPE and variable violations for SGS, fitted to
linear models, averaged over all cases of $p$ and the correlation ($\rho$).

(b) Variable violations for SGS applied to the cancer and colitis datasets,
averaged over the nine pathway collections.

Figure 6: The proportion of KKT violations relative to the full input space,
as a function of the regularization path.

For SGS, KKT violations are more common, as the additional layer of screening
carries with it additional assumptions. Additionally, the subdifferential term
in Equation 7 was chosen to be as small as possible, leading to tighter
screened sets. The increased violations are reflective of this. On both the
simulated and real datasets, SGS recorded an increasing number of KKT
violations as the fitted models became denser (Figure 6). The shape of the
increasing number of KKT violations mirrors the log-linear shape of the
regularization path. The trend of increasing KKT violations with denser models
is also observed for the SLOPE strong screening rule [17].

Despite the increased number of KKT violations for SGS, which amounts to the
additional computational cost of performing the KKT checks and repeated sub-
fitting in Algorithm 1, the overall screening procedure still results in
substantial runtime improvements, as evidenced in Tables 1 and 2. This shows
that screening rules can provide improvements to computational cost, even with
assumptions that are often violated.

###  6 Discussion

In this manuscript, we have developed strong screening rules for group-based
SLOPE models: Group SLOPE and Sparse-group SLOPE, neither of which have any
previous screening rules. The screening rule for gSLOPE screens out irrelevant
groups before fitting. The screening rules for SGS perform bi-level screening,
based on our proposed sparse-group screening framework.

The two proposed screening rules differ from the existing SLOPE screening
rules both in construction and in final outcome. gSLOPE only performs
screening through the groups, forcing it to keep all variables within the
screened groups, even noisy ones, in the optimization process. SGS, similarly
to SLOPE and in contrast to gSLOPE, performs variable screening after its
group screening step. As additional assumptions are required for SGS to
perform bi-level selection, its computational efficiency is not as effective
as the one observed with the SLOPE screening rules [17]. The superiority of
SGS in variable selection and predictive performance when group information is
available overcomes the additional computational burden [14].

Through comprehensive analysis of synthetic and real data, we illustrate that
the screening rules lead to dramatic improvements in the runtime of gSLOPE and
SGS models, as well as for group-based OSCAR models (Appendix E). This is
achieved without affecting model accuracy. This is particularly important in
datasets where $p\gg n$, such as genetics ones, which is the main motivation
behind SLOPE [4]. The screening rules presented in this manuscript allow
group-based SLOPE, and by extension, group-based OWL models, to achieve
computational fitting times more in line with lasso-based models, making them
more widely accessible.

###### Limitations.

One of the limitations of our proposed screening rules for SGS is the number
of assumptions made. A future direction includes the exploration of
alternative strong screening rules that require fewer assumptions.
Alternatively, the development of safe rules, which guarantee that only
inactive variables are discarded, could be an avenue for future exploration,
potentially incorporating safe and strong rules together in a hybrid rule [32,
37]. A comparison between safe and the proposed strong rules developed in this
manuscript could provide further insight into which type of screening is most
beneficial for non-separable penalties.

### References

  * Ahmed and Bajwa [2019] Talal Ahmed and Waheed U. Bajwa.  Exsis: Extended sure independence screening for ultrahigh-dimensional linear models.  _Signal Processing_ , 159:33–48, 2019.  ISSN 0165-1684.  doi: https://doi.org/10.1016/j.sigpro.2019.01.018.
  * Atamturk and Gomez [2020] Alper Atamturk and Andres Gomez.  Safe screening rules for $\ell_{0}$-regression from perspective relaxations.  In _Proceedings of the 37th International Conference on Machine Learning_ , pages 421–430. PMLR, 2020.
  * Bao et al. [2020] Runxue Bao, Bin Gu, and Heng Huang.  Fast OSCAR and OWL Regression via Safe Screening Rules.  In _Proceedings of the 37th International Conference on Machine Learning_ , pages 653–663. PMLR, 2020.
  * Bogdan et al. [2015] Małgorzata Bogdan, Ewout van den Berg, Chiara Sabatti, Weijie Su, and Emmanuel J Candès.  SLOPE—Adaptive variable selection via convex optimization.  _The Annals of Applied Statistics_ , 9(3), 2015.  ISSN 1932-6157.  doi: 10.1214/15-AOAS842.
  * Bondell and Reich [2008] Howard D. Bondell and Brian J. Reich.  Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR.  _Biometrics_ , 64(1):115–123, 2008.  ISSN 0006-341X.  doi: 10.1111/j.1541-0420.2007.00843.x.
  * Brzyski et al. [2019] Damian Brzyski, Alexej Gossmann, Weijie Su, and Małgorzata Bogdan.  Group SLOPE – Adaptive Selection of Groups of Predictors.  _Journal of the American Statistical Association_ , 114(525):419–433, 2019.  doi: 10.1080/01621459.2017.1411269.
  * Burczynski et al. [2006] Michael E Burczynski, Ron L Peterson, Natalie C Twine, Krystyna A Zuberek, Brendan J Brodeur, Lori Casciotti, Vasu Maganti, Padma S Reddy, Andrew Strahs, Fred Immermann, Walter Spinelli, Ulrich Schwertschlag, Anna M Slager, Monette M Cotreau, and Andrew J Dorner.  Molecular Classification of Crohn’s Disease and Ulcerative Colitis Patients Using Transcriptional Profiles in Peripheral Blood Mononuclear Cells.  _The Journal of Molecular Diagnostics_ , 8(1):51–61, 2006.  ISSN 15251578.  doi: 10.2353/jmoldx.2006.050079.
  * Chetverikov et al. [2021] Denis Chetverikov, Zhipeng Liao, and Victor Chernozhukov.  On cross-validated Lasso in high dimensions.  _The Annals of Statistics_ , 49(3):1300 – 1317, 2021.  doi: 10.1214/20-AOS2000.
  * Dupuis and Tardivel [2023] Xavier Dupuis and Patrick J C Tardivel.  The Solution Path of SLOPE.  Working paper, 2023.  URL https://hal.science/hal-04100441.
  * Efron et al. [2004] Bradley Efron, Trevor Hastie, Iain Johnstone, and Robert Tibshirani.  Least angle regression.  _The Annals of Statistics_ , 32(2):407–499, 2004.  doi: 10.1214/009053604000000067.
  * El Ghaoui et al. [2010] Laurent El Ghaoui, Vivian Viallon, and Tarek Rabbani.  Safe feature elimination in sparse supervised learning.  Technical Report UCB/EECS-2010-126, EECS Department, University of California, Berkeley, 2010.
  * Elvira and Herzet [2021] Clément Elvira and Cédric Herzet.  Safe rules for the identification of zeros in the solutions of the SLOPE problem.  _SIAM Journal on Mathematics of Data Science_ , 5(1):147–173, 2021.  doi: 10.1137/21m1457631.
  * Fan and Lv [2008] Jianqing Fan and Jinchi Lv.  Sure independence screening for ultrahigh dimensional feature space.  _Journal of the Royal Statistical Society: Series B (Statistical Methodology)_ , 70(5):849–911, 2008.  doi: https://doi.org/10.1111/j.1467-9868.2008.00674.x.
  * Feser and Evangelou [2023] Fabio Feser and Marina Evangelou.  Sparse-group SLOPE: adaptive bi-level selection with FDR-control.  _arXiv preprint arXiv:2305.09467_ , 2023.
  * Homrighausen and McDonald [2018] Darren Homrighausen and Daniel J. McDonald.  A study on tuning parameter selection for the high-dimensional lasso.  _Journal of Statistical Computation and Simulation_ , 88(15):2865–2892, 2018.  doi: 10.1080/00949655.2018.1491575.
  * Kuhn and Tucker [1950] H W Kuhn and A W Tucker.  Nonlinear programming.  In _Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability_ , pages 481–492, Berkeley, Los Angeles, USA, 1950. University of California Press.
  * Larsson et al. [2020] Johan Larsson, Małgorzata Bogdan, and Jonas Wallin.  The strong screening rule for SLOPE.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 14592–14603. Curran Associates, Inc., 2020.
  * Larsson et al. [2022] Johan Larsson, Quentin Klopfenstein, Mathurin Massias, and Jonas Wallin.  Coordinate Descent for SLOPE.  _Proceedings of Machine Learning Research_ , 206:4802–4821, 2022.  ISSN 26403498.
  * Liang et al. [2022] Xiaoxuan Liang, Aaron Cohen, Anibal Solón Heinsfeld, Franco Pestilli, and Daniel J. McDonald.  sparsegl: An R Package for Estimating Sparse Group Lasso.  _arXiv preprint arXiv:2208.02942_ , 2022.
  * Ma et al. [2004] Xiao-Jun Ma, Zuncai Wang, Paula D Ryan, Steven J Isakoff, Anne Barmettler, Andrew Fuller, Beth Muir, Gayatry Mohapatra, Ranelle Salunga, J.Todd Tuggle, Yen Tran, Diem Tran, Ana Tassin, Paul Amon, Wilson Wang, Wei Wang, Edward Enright, Kimberly Stecker, Eden Estepa-Sabal, Barbara Smith, Jerry Younger, Ulysses Balis, James Michaelson, Atul Bhan, Karleen Habin, Thomas M Baer, Joan Brugge, Daniel A Haber, Mark G Erlander, and Dennis C Sgroi.  A two-gene expression ratio predicts clinical outcome in breast cancer patients treated with tamoxifen.  _Cancer Cell_ , 5(6):607–616, 2004.  ISSN 15356108.  doi: 10.1016/j.ccr.2004.05.015.
  * Ndiaye et al. [2016a] Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, and Joseph Salmon.  GAP Safe Screening Rules for Sparse-Group Lasso.  In _Advances in Neural Information Processing Systems_ , volume 29. Curran Associates, Inc., 2016a.
  * Ndiaye et al. [2016b] Eugene Ndiaye, Olivier Fercoq, Alexandre Gramfort, and Joseph Salmon.  Gap Safe screening rules for sparsity enforcing penalties.  _Journal of Machine Learning Research_ , 18, 2016b.  ISSN 15337928.
  * Negrinho and Martins [2014] Renato Negrinho and André F T Martins.  Orbit regularization.  In _Proceedings of the 27th International Conference on Neural Information Processing Systems_ , volume 2, pages 3221–3229. MIT Press, 2014.
  * Nomura [2020] Shunichi Nomura.  An Exact Solution Path Algorithm for SLOPE and Quasi-Spherical OSCAR.  _arXiv preprint arXiv:2010.15511_ , 2020.
  * Ogawa et al. [2013] Kohei Ogawa, Yoshiki Suzuki, and Ichiro Takeuchi.  Safe screening of non-support vectors in pathwise svm computation.  In _Proceedings of the 30th International Conference on Machine Learning_ , pages 1382–1390. PMLR, 2013.
  * Pedregosa and Gidel [2018] Fabian Pedregosa and Gauthier Gidel.  Adaptive three operator splitting.  In _Proceedings of the 35th International Conference on Machine Learning_ , volume 80 of _Proceedings of Machine Learning Research_ , pages 4085–4094. PMLR, 2018.
  * Schneider and Tardivel [2022] Ulrike Schneider and Patrick Tardivel.  The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation.  _Journal of Machine Learning Research_ , 23:1–36, 2022.
  * Shibagaki et al. [2016] Atsushi Shibagaki, Masayuki Karasuyama, Kohei Hatano, and Ichiro Takeuchi.  Simultaneous safe screening of features and samples in doubly sparse modeling.  In _Proceedings of The 33rd International Conference on Machine Learning_ , volume 48 of _Proceedings of Machine Learning Research_ , pages 1577–1586. PMLR, 2016.
  * Simon et al. [2013] Noah Simon, Jerome Friedman, Trevor Hastie, and Robert Tibshirani.  A Sparse-Group Lasso.  _Journal of Computational and Graphical Statistics_ , 22(2):231–245, 2013.  ISSN 1061-8600.  doi: 10.1080/10618600.2012.681250.
  * Tibshirani [1996] Robert Tibshirani.  Regression Shrinkage and Selection Via the Lasso.  _Journal of the Royal Statistical Society: Series B (Methodological)_ , 58(1):267–288, 1996.  ISSN 00359246.  doi: 10.1111/j.2517-6161.1996.tb02080.x.
  * Tibshirani et al. [2010] Robert Tibshirani, Jacob Bien, Jerome Friedman, Trevor Hastie, Noah Simon, Jonathan Taylor, and Ryan J. Tibshirani.  Strong rules for discarding predictors in lasso-type problems.  _Journal of the Royal Statistical Society. Series B: Statistical Methodology_ , 74(2):245–266, 2010.  ISSN 13697412.  doi: 10.1111/j.1467-9868.2011.01004.x.
  * Wang and Breheny [2022] Chuyi Wang and Patrick Breheny.  Adaptive hybrid screening for efficient lasso optimization.  _Journal of Statistical Computation and Simulation_ , 92(11):2233–2256, 2022.  doi: 10.1080/00949655.2021.2025376.
  * Wang and Ye [2014] Jie Wang and Jieping Ye.  Two-Layer Feature Reduction for Sparse-Group Lasso via Decomposition of Convex Sets.  _Advances in Neural Information Processing Systems_ , 3:2132–2140, 2014.  ISSN 10495258.
  * Wang et al. [2013] Jie Wang, Jiayu Zhou, Peter Wonka, and Jieping Ye.  Lasso screening rules via dual Polytope Projection.  In _Proceedings of the 26th International Conference on Neural Information Processing Systems_ , volume 1, pages 1070–1078. Curran Associates Inc., 2013.
  * Zeng and Figueiredo [2014a] Xiangrong Zeng and Mário A. T. Figueiredo.  Decreasing weighted sorted $\ell_{1}$ regularization.  _IEEE Signal Processing Letters_ , 21:1240–1244, 2014a.
  * Zeng and Figueiredo [2014b] Xiangrong Zeng and Mário A T Figueiredo.  The atomic norm formulation of OSCAR regularization with application to the Frank-Wolfe algorithm.  In _2014 22nd European Signal Processing Conference (EUSIPCO)_ , pages 780–784, 2014b.
  * Zeng et al. [2021] Yaohui Zeng, Tianbao Yang, and Patrick Breheny.  Hybrid safe–strong rules for efficient optimization in lasso-type problems.  _Computational Statistics & Data Analysis_, 153:107063, 2021.  ISSN 0167-9473.  doi: https://doi.org/10.1016/j.csda.2020.107063.

## Appendix

###  Appendix A Group SLOPE

####  A.1 Penalty weights

The penalty weights for gSLOPE were derived to provide group FDR-control under
orthogonal designs [6]. For the FDR-control parameter $q_{g}\in(0,1)$, they
are given by (where the indexing corresponds to the sorted groups)

$w_{i}^{\text{max}}=\max_{j=1,\dots,m}\left\\{\frac{1}{\sqrt{p_{j}}}F^{-1}_{\chi_{p_{j}}}(1-q_{g}i/m)\right\\},\;\text{for}\;i=1,\dots,m,$

where $F_{\chi_{p_{j}}}$ is the cumulative distribution function of a $\chi$
distribution with $p_{j}$ degrees of freedom. A relaxation to this sequence is
applied in [6], to give

$w_{i}^{\text{mean}}=\overline{F}^{-1}_{\chi_{p_{j}}}(1-q_{g}i/m),\;\text{where}\;\overline{F}_{\chi_{p_{j}}}(x):=\frac{1}{m}\sum_{j=1}^{m}F_{\chi_{p_{j}}}(\sqrt{p_{j}}x).$
(9)

The mean sequence weights defined in Equation 9 are used for all gSLOPE
numerical simulations in this manuscript (shown in Figure A1).

Figure A1: The gSLOPE weights, $w$, shown for Figure 4 for
$p=500,m=100,q_{g}=0.05$.

####  A.2 Theory

###### Proof of Theorem 3.1.

The proof is similar to that of Theorem 2.7 in [6], where the subdifferential
of gSLOPE is derived under equal groups. It is derived here under more general
terms. The subdifferential needs to be derived under two cases:

  1. 1.

Inactive groups, $\mathcal{G}_{\mathcal{Z}}$.

  2. 2.

Active groups, $\mathcal{G}_{\mathcal{A}}$.

Case 1: For inactive groups, we consider the subdifferential at zero. The
subdifferential of a norm at zero is given by the dual norm of the unit ball
[27],

$\partial
J_{\text{gslope}}(\mathbf{0};w)=\mathbf{B}_{J_{\text{gslope}}^{*}(\mathbf{0};w)}[0,1]=\\{x:J_{\text{gslope}}^{*}(x;w)\leq
1\\}.$

The dual norm for gSLOPE is given by [6]

$J_{\text{gslope}}^{*}(x;w)=J_{\text{slope}}^{*}([x]_{\mathcal{G},-0.5}).$

Hence, the dual norm unit ball is

$\mathbf{B}_{J_{\text{gslope}}^{*}(\mathbf{0};w)}[0,1]=\\{x:[x]_{\mathcal{G},-0.5}\in\mathbf{B}_{J_{\text{slope}}^{*}(\mathbf{0};w)}[0,1]\\},$

where
$\mathbf{B}_{J_{\text{slope}}^{*}(\mathbf{0};w)}[0,1]=\\{x\in\mathbb{R}^{m}:\text{cumsum}(|x|_{\downarrow}-w)\preceq\mathbf{0}\\}$
is the unit ball of the dual norm to $J_{\text{slope}}$ [4]. Using this, the
subdifferential at zero for the inactive groups, $\mathcal{Z}$, is given by

$\partial
J_{\text{gslope}}(\mathbf{0};w_{\mathcal{Z}})=\\{x\in\mathbb{R}^{\operatorname{card}(\mathcal{G}_{\mathcal{Z}})}:[x]_{\mathcal{G}_{\mathcal{Z}},-0.5}\in\partial
J_{\text{slope}}(\mathbf{0};w_{\mathcal{Z}})\\}.$

Case 2: Without loss of generality, denote the group index $s$ such that
$\|\beta^{(g)}\|_{2}=0$ for $g>s$ (inactive groups) and
$\|\beta^{(g)}\|_{2}\neq 0$ for $g\leq s$ (active groups). In other words,
$g\in\mathcal{G}_{\mathcal{A}}$ if $g\leq s$. Define a set
$D=\\{d\in\mathbb{R}^{p}:\|\beta^{(1)}+d^{(1)}\|_{2}>\ldots>\|\beta^{(s)}+d^{(s)}\|_{2},\|\beta^{(s)}+d^{(s)}\|_{2}>\|d^{(g)}\|_{2},g>s\\}$.
By definition of a subdifferential, if $x\in\partial
J_{\text{gslope}}(\beta;w)$, then for all $d\in D$

$\sum_{g=1}^{m}\sqrt{p_{g}}w_{g}\|\beta^{(g)}+d^{(g)}\|_{2}\geq\sum_{g=1}^{m}\sqrt{p_{g}}w_{g}\|\beta^{(g)}\|_{2}+x^{\top}d.$

Splitting this up into whether the groups are active (whether $g\leq s$):

$\displaystyle\sum_{g=1}^{s}\sqrt{p_{g}}w_{g}\|\beta^{(g)}+d^{(g)}\|_{2}+\sum_{g=s+1}^{m}\sqrt{p_{g}}w_{g}\|d^{(g)}\|_{2}\geq$
$\displaystyle\sum_{g=1}^{s}\sqrt{p_{g}}w_{g}\|\beta^{(g)}\|_{2}$ (10)
$\displaystyle+\sum_{g=1}^{s}x^{(g)T}d^{(g)}+\sum_{g=s+1}^{m}x^{(g)T}d^{(g)}.$

Now, for $g\in\mathcal{G}_{\mathcal{A}}$, define a new set $D_{g}=\\{d\in
D:d^{(j)}\equiv\mathbf{0},j\neq g\\}$. Taking $d\in D_{g}$, Equation 10
becomes

$\sqrt{p_{g}}w_{g}\|\beta^{(g)}+d^{(g)}\|_{2}\geq\sqrt{p_{g}}w_{g}\|\beta^{(g)}\|_{2}+x^{(g)T}d^{(g)}.$

Since the set $\\{d^{(g)}:d\in D_{g}\\}$ is open in $\mathbb{R}^{p_{g}}$ and
contains zero, by Corollary G.1 in [6], it follows that $x^{(g)}\in\partial
f_{g}(b^{(g)})$ for
$f_{g}:\mathbb{R}^{p_{g}}\rightarrow\mathbb{R},f_{g}(x)=w_{g}\sqrt{p_{g}}\|x\|_{2}$.
Now, for $g\leq s$, $f_{g}$ is differentiable in $\beta^{(g)}$, giving

$x^{(g)}=w_{g}\sqrt{p_{g}}\frac{\beta^{(g)}}{\|\beta^{(g)}\|_{2}},$

proving the result. ∎

###### Proof of Proposition 3.2.

Suppose we have $\mathcal{B}\neq\emptyset$ after running the algorithm. Then,
plugging in $h(\lambda_{k+1})=([\nabla
f(\hat{\beta}(\lambda_{k+1}))]_{\mathcal{G},-0.5})_{\downarrow}$ gives

$\text{cumsum}\Bigl{(}\bigl{(}([\nabla
f(\hat{\beta}(\lambda_{k+1}))]_{\mathcal{G},-0.5})_{\downarrow}\bigr{)}_{\mathcal{B}}-\lambda_{k+1}w_{\mathcal{B}}\Bigr{)}\prec\mathbf{0},$

so that by the gSLOPE subdifferential (Theorem 3.1) all groups in
$\mathcal{B}$ are inactive. This is valid by the KKT conditions (Equation 2),
as we know that $-\nabla f(\hat{\beta}(\lambda_{k+1}))\in\partial
J_{\text{gslope}}(\mathbf{0};w)$. Hence, $\mathcal{S}_{g}(\lambda_{k+1})$ will
contain the active set $\mathcal{A}_{g}(\lambda_{k+1})$. ∎

###### Proof of Proposition 3.3.

Since $\text{cumsum}(y)\succeq\text{cumsum}(x)\iff y\succeq x$ [17], we only
need to show for a group $g$,

$|h_{g}(\lambda_{k+1})|\leq|h_{g}(\lambda_{k})|+\lambda_{k}w_{g}-\lambda_{k+1}w_{g}.$

Applying the reverse triangle inequality to the Lipschitz assumption gives

$\displaystyle|h_{g}(\lambda_{k+1})|-|h_{g}(\lambda_{k})|\leq\left|h_{g}(\lambda_{k+1})-h_{g}(\lambda_{k})\right|\leq\lambda_{k}w_{g}-\lambda_{k+1}w_{g}$
$\displaystyle\implies$
$\displaystyle|h_{g}(\lambda_{k+1})|\leq|h_{g}(\lambda_{k})|+\lambda_{k}w_{g}-\lambda_{k+1}w_{g},$

proving the result. ∎

####  A.3 Path start derivation

The aim is to find the value of $\lambda$ at which the first group enters the
model. When all features are zero, the gSLOPE KKT conditions (Equation 2) are

$\mathbf{0}\in\nabla f(\mathbf{0})+\lambda\partial
J_{\text{gslope}}(\mathbf{0};w).$

This is satisfied when

$[\nabla f(\mathbf{0})]_{\mathcal{G},-0.5}\in\partial
J_{\text{slope}}(\mathbf{0};\lambda w)\implies\text{cumsum}\bigl{(}([\nabla
f(\mathbf{0})]_{\mathcal{G},-0.5})_{\downarrow}-\lambda
w\bigr{)}\preceq\mathbf{0}.$

Rearranging this gives

$\lambda\succeq\text{cumsum}\bigl{(}([\nabla
f(\mathbf{0})]_{\mathcal{G},-0.5})_{\downarrow}\bigr{)}\oslash\text{cumsum}(w),$

where $\oslash$ denotes Hadamard division. Picking the maximum possible
$\lambda$ such that this holds yields

$\lambda_{1}=\max\left\\{\text{cumsum}\bigl{(}([\nabla
f(\mathbf{0})]_{\mathcal{G},-0.5})_{\downarrow}\bigr{)}\oslash\text{cumsum}(w)\right\\}.$

This can be verified by noting that $\lambda_{1}=J^{*}_{\text{gslope}}(\nabla
f(\mathbf{0});w)$ [22]. Now,
$J^{*}_{\text{gslope}}(x;w)=J^{*}_{\text{slope}}([x]_{\mathcal{G},-0.5};w)$
[6]. The dual norm of SLOPE is given by [23]

$J^{*}_{\text{slope}}(x;w)=\max\left\\{\text{cumsum}(|x|_{\downarrow})\oslash\text{cumsum}(w)\right\\}.$

Therefore, $\lambda_{1}$ is as before.

###  Appendix B Sparse-group SLOPE

####  B.1 Penalty weights

The penalty weights for SGS provide variable and group FDR-control
simultaneously, under orthogonal designs [14]. They are given by (where the
indexing corresponds to the sorted variables/groups)

$\displaystyle
v_{i}^{\text{max}}=\max_{j=1,\dots,m}\left\\{\frac{1}{\alpha}F_{\mathcal{N}}^{-1}\left(1-\frac{q_{v}i}{2p}\right)-\frac{1}{3\alpha}(1-\alpha)a_{j}w_{j}\right\\},\;i=1,\dots,p,$
(11) $\displaystyle
w_{i}^{\text{max}}=\max_{j=1,\dots,m}\left\\{\frac{F_{\text{FN}}^{-1}(1-\frac{q_{g}i}{m})-\alpha\sum_{k\in\mathcal{G}_{j}}v_{k}}{(1-\alpha)p_{j}}\right\\},\;i=1,\dots,m,$
(12)

where $F_{\chi_{p_{j}}}$ is the cumulative distribution function of a $\chi$
distribution with $p_{j}$ degrees of freedom, $F_{\mathcal{N}}$ is the
cumulative distribution function of a folded Gaussian distribution, and
$a_{j}$ is a quantity that requires estimation. The estimator
$\hat{a}_{j}=\lfloor\alpha p_{j}\rfloor$ is proposed in [14]. As with gSLOPE
(Appendix A.1), a relaxation is possible, giving the weights

$\displaystyle
v_{i}^{\text{mean}}=\overline{F}_{\mathcal{N}}^{-1}\left(1-\frac{q_{v}i}{2p}\right),\;\text{where}\;\overline{F}_{\mathcal{N}}(x):=\frac{1}{m}\sum_{j=1}^{m}F_{\mathcal{N}}\left(\alpha
x+\frac{1}{3}(1-\alpha)a_{j}w_{j}\right),$ (13) $\displaystyle
w_{i}^{\text{mean}}=\overline{F}_{\text{FN}}^{-1}\left(1-\frac{q_{g}i}{p}\right),\;\text{where}\;\overline{F}_{\text{FN}}(x):=\frac{1}{m}\sum_{j=1}^{m}F_{\text{FN}}\left((1-\alpha)p_{j}x+\alpha\sum_{k\in\mathcal{G}_{j}}v_{k}\right).$
(14)

The mean weights defined in Equations 13 and 14 are used for all SGS numerical
simulations in this manuscript (shown in Figure A2).

Figure A2: The SGS weights, $(v,w)$, shown for Figure 4 for
$p=500,m=100,q_{v}=0.05,q_{g}=0.05,\alpha=0.95$.

####  B.2 Derivation of soft thresholding operator

To determine the form of the quantity $\partial
J_{\text{slope}}(\mathbf{0};v)$, consider that for Equation 7 to be satisfied,
the term inside the $[\cdot]$ operator needs to be as small as possible. Now,

$\partial
J_{\text{slope}}(\mathbf{0};v)=\\{y:\text{cumsum}(|y|)\preceq\text{cumsum}(v)\\}.$

Note that $\text{cumsum}(y)\preceq\text{cumsum}(x)\iff y\preceq x$. We
consider the cases:

  1. 1.

$\nabla_{i}f(\beta)>\lambda\alpha v_{i}$: choose $y_{i}=-v_{i}$.

  2. 2.

$\nabla_{i}f(\beta)<-\lambda\alpha v_{i}$: choose $y_{i}=v_{i}$.

  3. 3.

$\nabla_{i}f(\beta)\in[-\lambda\alpha v_{i},\lambda\alpha v_{i}]$: choose
$y_{i}=\nabla_{i}f(\beta)/\alpha\lambda$.

Hence, the term becomes

$S(\nabla f(\beta),\lambda\alpha):=\text{sign}(\nabla f(\beta))(|\nabla
f(\beta)|-\lambda\alpha)_{+},$

which is the soft thresholding operator.

####  B.3 Theory

######  Proposition B.1 (Strong group screening rule for SGS).

Let $\tilde{h}(\lambda):=([S(\nabla
f(\beta),\lambda\alpha)]_{\mathcal{G},-0.5})_{\downarrow}$. Then taking
$c=\tilde{h}(\lambda_{k+1})$ and $\phi=(1-\alpha)\lambda_{k+1}w$ as inputs for
Algorithm A1 returns a superset $\mathcal{S}_{g}(\lambda_{k+1})$ of the active
set $\mathcal{A}_{g}(\lambda_{k+1})$.

###### Proof of Proposition B.1.

The proof is similar to that of Proposition 3.2. Suppose we have
$\mathcal{B}\neq\emptyset$ after running the algorithm. Then,

$\displaystyle\text{cumsum}(\tilde{h}_{\mathcal{B}}(\lambda_{k+1})-\lambda_{k+1}(1-\alpha)w_{\mathcal{B}})$
$\displaystyle\prec\mathbf{0}$
$\displaystyle\implies\text{cumsum}\Bigl{(}\bigl{(}[S(\nabla
f(\beta),\lambda_{k+1}\alpha)]_{\mathcal{G},-0.5})_{\downarrow}\bigr{)}_{\mathcal{B}}-\lambda_{k+1}(1-\alpha)w_{\mathcal{B}}\Bigr{)}$
$\displaystyle\prec,\mathbf{0}$

so that by the SGS subdifferential (Equation 7), all groups in $\mathcal{B}$
are inactive. Hence, $\mathcal{S}_{g}(\lambda_{k+1})$ will contain the active
set $\mathcal{A}_{g}(\lambda_{k+1})$. ∎

###### Proof of Proposition 4.1.

The proof is identical to that of Proposition 3.3, replacing $h_{g}(\cdot)$
with $\tilde{h}_{g}(\cdot)$ and $\lambda_{k+1}w$ by
$\lambda_{k+1}(1-\alpha)w$. ∎

######  Proposition B.2 (Strong variable screening rule for SGS).

Let $\bar{h}(\lambda)=|(\nabla f(\hat{\beta}(\lambda)))|_{\downarrow}$. Then
taking $c=\bar{h}(\lambda_{k+1})$ and $\phi=\lambda_{k+1}\alpha v$ for only
the variables contained in the groups in $\mathcal{A}_{g}(\lambda_{k+1})$ in
Algorithm A1 returns a superset $\mathcal{S}_{v}(\lambda_{k+1})$ of the active
set $\mathcal{A}_{v}(\lambda_{k+1})$.

###### Proof.

Suppose we have $\mathcal{B}\neq\emptyset$ after running the algorithm. Then,
we have

$\text{cumsum}(\bar{h}_{\mathcal{B}}(\lambda_{k+1})-\lambda_{k+1}\alpha
v_{\mathcal{B}})\prec\mathbf{0}\implies\text{cumsum}\Bigl{(}\bigl{(}|\nabla
f(\hat{\beta}(\lambda_{k+1}))|_{\downarrow}\bigr{)}_{\mathcal{B}}-\lambda_{k+1}\alpha
v_{\mathcal{B}}\Bigr{)}\prec\mathbf{0},$ (15)

so that by the SGS subdifferential for non-zero groups (Equation 8), all
variables in $\mathcal{B}$ are inactive. Hence,
$\mathcal{S}_{v}(\lambda_{k+1})$ will contain the active set
$\mathcal{A}_{v}(\lambda_{k+1})$. ∎

###### Proof for Proposition 4.2.

The proof is identical to that of Proposition 3.3, replacing $h_{g}(\cdot)$
with $\bar{h}_{g}(\cdot)$, $\lambda_{k+1}v$ with $\lambda_{k+1}\alpha v$, and
considering only variables in the groups contained in
$\mathcal{A}_{g}(\lambda_{k+1})$. ∎

####  B.4 Path start derivation

The aim is to find the value of $\lambda$ at which the first variable enters
the model. When all features are zero, the SGS KKT conditions (Equation 6) are

$\displaystyle-\nabla f(\mathbf{0})\in\lambda(1-\alpha)\partial
J_{\text{gslope}}(\mathbf{0};w)+\lambda\alpha\partial
J_{\text{slope}}(\mathbf{0};v)$ $\displaystyle\implies$
$\displaystyle-\frac{1}{\lambda}\nabla f(\mathbf{0})-(1-\alpha)\partial
J_{\text{gslope}}(\mathbf{0};w)\in\alpha\partial
J_{\text{slope}}(\mathbf{0};v)$ $\displaystyle\implies$
$\displaystyle\text{cumsum}\left(\left|-\frac{1}{\lambda}\nabla
f(\mathbf{0})-(1-\alpha)\partial
J_{\text{gslope}}(\mathbf{0};w)\right|_{\downarrow}-\alpha
v\right)\preceq\mathbf{0}.$

By the reverse triangle inequality and ordering of the group weights

$\displaystyle\frac{1}{\lambda}\text{cumsum}\left(|\nabla
f(\mathbf{0})|_{\downarrow}\right)\preceq\text{cumsum}((1-\alpha)|\partial
J_{\text{gslope}}(\mathbf{0};w)|-\alpha v)$ $\displaystyle\implies$
$\displaystyle\lambda\succeq\text{cumsum}(|\nabla
f(\mathbf{0})|_{\downarrow})\oslash\text{cumsum}((1-\alpha)|\partial
J_{\text{gslope}}(\mathbf{0};w)|-\alpha v).$

Now, note that for $x\in J_{\text{gslope}}(\mathbf{0};w)$, it holds

$\text{cumsum}([x]_{\mathcal{G},-0.5}-w)\preceq\mathbf{0}\implies[x]_{\mathcal{G},-0.5}\preceq
w\implies\|x^{(g)}\|_{2}\leq\sqrt{p_{g}}w_{g},\forall g\in\mathcal{G}.$

This is satisfied at the upper limit at $x=\tau\omega$, where $\tau$ and
$\omega$ are expanded vectors of the group sizes ($\sqrt{p_{g}})$ and penalty
weights ($w_{g}$) to $p$ dimensions, so that each variable within the same
group is assigned the same value. Hence,

$\lambda_{1}=\max\left\\{\text{cumsum}(|\nabla
f(\mathbf{0})|_{\downarrow})\oslash\text{cumsum}((1-\alpha)\tau\omega-\alpha
v)\right\\}.$

###  Appendix C SLOPE Algorithm

Algorithm A1 Cumsum algorithm from [17]

Input: $c\in\mathbb{R}^{p},\phi\in\mathbb{R}^{p}$, where
$\phi_{1}\geq\cdots\geq$ $\phi_{p}\geq 0$

$\mathcal{S},\mathcal{B}\leftarrow\varnothing$

for $i=1$ to $p$ do

$\mathcal{B}\leftarrow\mathcal{B}\cup\\{i\\}$

if $\text{cumsum}(c_{\mathcal{B}}-\phi_{\mathcal{B}})\geq 0$ then

$\mathcal{S}\leftarrow\mathcal{S}\cup\mathcal{B}$

$\mathcal{B}\leftarrow\varnothing$

end if

end for

Output: $\mathcal{S}$

###  Appendix D Screening rule framework

####  D.1 Group SLOPE algorithm

For the following is performed for $k=1,\ldots,l-1$:

  1. 1.

Set
$\mathcal{E}_{g}=\mathcal{S}_{g}(\lambda_{k+1})\cup\mathcal{A}_{g}(\lambda_{k})$,
where $\mathcal{S}_{g}(\lambda_{k+1})$ is obtained using Proposition 3.3.

  2. 2.

Compute $\hat{\beta}(\lambda_{k+1})$ by Equation 1 with the gSLOPE norm using
only the groups in $\mathcal{E}_{g}.$ For any groups not in $\mathcal{E}_{g}$,
$\hat{\beta}(\lambda_{k+1})$ is set to zero.

  3. 3.

Check the KKT conditions (Equation 2) for all groups at this solution.

  4. 4.

If there are no violations, we are done and keep $\hat{\beta}(\lambda_{k+1})$.
Otherwise, add the violating groups into $\mathcal{E}$ and return to Step 2.

####  D.2 SGS algorithm

For the following is performed for $k=1,\ldots,l-1$:

  1. 1.

Group screen step: Calculate $\mathcal{S}_{g}(\lambda_{k+1})$ using
Proposition 4.1.

  2. 2.

Variable screen step: Set
$\mathcal{E}_{v}=\mathcal{S}_{v}(\lambda_{k+1})\cup\mathcal{A}_{v}(\lambda_{k})$,
where $\mathcal{S}_{v}(\lambda_{k+1})$ is obtained using Proposition 4.2 with
only the groups in $\mathcal{S}_{g}(\lambda_{k+1})$.

  3. 3.

Compute $\hat{\beta}(\lambda_{k+1})$ by Equation 1 with the SGS norm using
only the features in $\mathcal{E}_{v}$. For features not in $\mathcal{E}_{v}$,
$\hat{\beta}(\lambda_{k+1})$ is set to zero.

  4. 4.

Check the KKT conditions (Equation 6) for all features at this solution.

  5. 5.

If there are no violations, we are done and keep $\hat{\beta}(\lambda_{k+1})$,
otherwise add in the violating variables into $\mathcal{E}_{g}$ and return to
Step 3.

###  Appendix E Group-based OSCAR

This section presents an extension of the proposed screening rules for group-
based OSCAR models. Firstly the OSCAR models are presented, and subsequently,
the performance of the screening rules on synthetic data is presented.

####  E.1 Model definition

The Ordered Weighted $\ell_{1}$ (OWL) framework is defined as [35]

$\hat{\beta}=\operatorname*{arg\,min}_{\beta}\left\\{\nabla f(\beta)+\lambda
J_{\text{owl}}(\beta;v)\right\\},$

where $J_{\text{owl}}(\beta;v)=\sum_{i=1}^{p}v_{i}|\beta|_{(i)}$,
$|\beta|_{(1)}\geq\ldots\geq|\beta|_{(p)}$, and $v$ are non-negative non-
increasing weights. SLOPE is a special case of OWL where the weights are taken
to be the Benjamini-Hochberg critical values [4]. Octagonal Shrinkage and
Clustering Algorithm for Regression (OSCAR) [5] is a further special case of
OWL (often referred to as OWL with linear decay) where for a variable $i$, the
weights are taken to be $v_{i}=\sigma_{1}+\sigma_{2}(p-i)$, and
$\sigma_{1},\sigma_{2}$ are to be set. In Bao et al. [3] they are set to
$\sigma_{1}=d_{i}\|\mathbf{X}^{\top}y\|_{\infty},\sigma_{2}=\sigma_{1}/p$,
where $d_{i}=i\times e^{-2}$.

Group OSCAR (gOSCAR) and Sparse-group OSCAR (SGO) are defined using the
frameworks provided by gSLOPE [6] and SGS [14], respectively, but instead
using the weights (for a variable $i$ and a group $g$)

$v_{i}=\sigma_{1}+\sigma_{2}(p-i),\;w_{g}=\sigma_{1}+\sigma_{3}(m-g),\;\sigma_{3}=\sigma_{1}/m.$
(16)

with the weights visualised in Figure A3.

Figure A3: The SGO weights, $(v,w)$, shown for Figure 4 for
$p=500,m=100,q_{v}=0.05,q_{g}=0.05,\alpha=0.95$.

####  E.2 Results

Similar observations and conclusions made for the screening rules of gSLOPE
and SGS are made for gOSCAR and SGO (Figures A4 \- A8).

Figure A4 illustrates the effectiveness of the bi-level screening of SGO,
similar to the effectiveness observed for SGS. Figures A5 \- A6 showcase the
efficiency of the screening rules on the proportion of the selected
groups/variables. The screening rules are found to be effective across
different data characteristics, with the running time of the models
significantly decreasing (Figure A7). Similarly to SGS, KKT violations are
more common compared to the ones observed for gOSCAR (Figure A8). This is due
to the additional assumptions made at the additional screening layer of SGO
and SGS. Similarly to Figure 6(a), the shape of the increasing number of KKT
violations mirrors the log-linear shape of the regularization path.

Figure A4: The proportion of variables in $\mathcal{S}_{v}$ relative to the
full input for SGO, shown for group and bi-level screening plotted as a
function of the regularization path, applied to the synthetic data (Section
5.1). The data are generated under a linear model for $p=500,5000$. The
results are averaged over 100 repetitions and 95% confidence intervals are
shown (the SGO equivalent of Figure 1 (a)).

Figure A5: The number of groups/variables in $\mathcal{E},\mathcal{A}$ for
both gOSCAR and SGO as a function of the regularization path for the linear
model with $p=2750,\rho=0.6,m=197$. The results are averaged over $100$
repetitions, with the shaded regions corresponding to $95\%$ confidence
intervals (the gOSCAR/SGO equivalent of Figure 2).

Figure A6: The proportion of groups/variables in $\mathcal{E},\mathcal{A}$,
relative to the full input, shown for gOSCAR and SGO. This is shown as a
function of the correlation ($\rho$), averaged over all cases of the input
dimension ($p$), with $100$ repetitions for each $p$, for both linear and
logistic models, with standard errors shown (the gOSCAR/SGO equivalent of
Figure 3).

Figure A7: Runtime (in seconds) for fitting $50$ models along a path, shown
for screening against no screening as a function of $p$, broken down into
different correlation cases, for the linear model. The results are averaged
over $100$ repetitions, with standard errors shown (the OSCAR equivalent of
Figure 4).

Figure A8: The proportion of KKT violations relative to the full input space,
as a function of the regularization path. Group violations for gOSCAR and
variable violations for SGS, fitted to linear models, averaged over all cases
of $p$ and $\rho$ (the gOSCAR/SGO equivalent of Figure 6(a)).

###  Appendix F Results

####  F.1 Computational information

The simulated experiments were executed on a high-performance computing
cluster (x86-64 Linux GNU) and the real data analysis was conducted on a Apple
Macbook Air (M1, 8GB).

####  F.2 Model Comparison

This section presents the accuracy of the models with and without screening,
by comparing the $\ell_{2}$ distances observed between the screened and non-
screened fitted values.

###### Synthetic data.

For the linear model, the maximum $\ell_{2}$ distances observed between the
screened and non-screened fitted values were of order $10^{-5}$ for gSLOPE and
$10^{-8}$ for SGS (Table F.3). Across the different cases, 98000 models were
fit in total for each approach (excluding the models for $\lambda_{1}$, where
no screening is applied). Of these model fits, there were no instances for
gSLOPE where $\mathcal{E}$ was not a superset of $\mathcal{A}$. There was only
one instance (out of the 98000) that this occurred for SGS, where
$\mathcal{E}$ was missing a single variable contained in $\mathcal{A}$ (which
had a non-screen fitted value of $\hat{\beta}=-0.004$).

For the logistic model, the maximum $\ell_{2}$ distances observed between the
screened and non-screened fitted values were of order $10^{-8}$ for both
gSLOPE and SGS (Table F.3.2). Across the different cases, 98000 models were
fit in total for each approach (excluding the models for $\lambda_{1}$, where
no screening is applied). Of these model fits, there were no instances for
gSLOPE or SGS where $\mathcal{E}$ was not a superset of $\mathcal{A}$.

###### Real data.

In the real data analysis, the estimated coefficients with and without
screening were very close to each other (with $\ell_{2}$ to be of order
$10^{-8}$) for both SGS and gSLOPE (Table F.4). No instances occurred where
$\mathcal{E}$ was not a superset of $\mathcal{A}$.

####  F.3 Additional results from the simulation study

Table A1: Variable screening metrics for SGS using linear and logistic models
for the simulation study presented in Section 5.1. The number of variables in
$\mathcal{A}_{v},\mathcal{S}_{v},\mathcal{E}_{v}$, and $\mathcal{K}_{v}$ are
shown, averaged across all $20$ cases of the correlation ($\rho$) and $p$.
Standard errors are shown.
Method | Type | $\operatorname{card}(\mathcal{A}_{v})$ | $\operatorname{card}(\mathcal{S}_{v})$ | $\operatorname{card}(\mathcal{E}_{v})$ | $\operatorname{card}(\mathcal{K}_{v})$
---|---|---|---|---|---
SGS | Linear | $179\pm 3$ | $313\pm 5$ | $363\pm 6$ | $51\pm 1$
SGS | Logistic | $230\pm 3$ | $405\pm 5$ | $472\pm 6$ | $66\pm 1$

Table A2: General and group screening metrics for SGS and gSLOPE using linear
and logistic models for the simulation study presented in Section 5.1. General
metrics: the runtime (in seconds) for screening against no screening, the
number of fitting iterations for screening against no screening, and the
$\ell_{2}$ distance between the fitted values obtained with screening and no
screening. Group screening metrics: the number of groups in
$\mathcal{A}_{g},\mathcal{S}_{g},\mathcal{E}_{g}$, and $\mathcal{K}_{g}$. The
results are averaged across all $20$ cases of the correlation ($\rho$) and
$p$. Standard errors are shown.

Method | Type |  Runtime (s) |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---
|  Runtime (s) | screen |  |  |  |  |  |  |  |
| $\operatorname{card}(\mathcal{A}_{g})$ | $\operatorname{card}(\mathcal{S}_{g})$ | $\operatorname{card}(\mathcal{E}_{g})$ | $\operatorname{card}(\mathcal{K}_{g})$ |  Num it |  |  |  |  |
|  Num it |  |  |  | screen |  |  |  |  |
|  $\ell_{2}$ dist |  |  |  |  |  |  |  |  |
| to no screen |  |  |  |  |  |  |  |  |
gSLOPE | Linear | $1016\pm 21$ | $1623\pm 27$ | $55\pm 1$ | $76\pm 1$ | $76\pm 1$ | $0.006\pm 0.004$ | $333\pm 6$ | $351\pm 6$ | $2\times 10^{-6}\pm 1\times 10^{-6}$
gSLOPE | Logistic | $814\pm 8$ | $1409\pm 11$ | $71\pm 1$ | $97\pm 1$ | $97\pm 1$ | $0.014\pm 0.014$ | $78\pm 1$ | $83\pm 1$ | $1\times 10^{-8}\pm 1\times 10^{-8}$
SGS | Linear | $735\pm 15$ | $1830\pm 34$ | $61\pm 1$ | $84\pm 1$ | $91\pm 1$ | $21\pm 1$ | $91\pm 3$ | $708\pm 12$ | $2\times 10^{-9}\pm 3\times 10^{-9}$
SGS | Logistic | $407\pm 2$ | $859\pm 6$ | $84\pm 1$ | $107\pm 1$ | $118\pm 1$ | $22\pm 0.3$ | $7\pm 0.2$ | $51\pm 0.8$ | $4\times 10^{-9}\pm 3\times 10^{-10}$

#####  F.3.1 Additional results for the linear model

(a) $p=500,\rho=0$

(b) $p=500,\rho=0.9$

(c) $p=5000,\rho=0$

(d) $p=5000,\rho=0.9$

Figure A9: The number of groups/variables in $\mathcal{E},\mathcal{A}$ as a
function of the regularization path for the linear model with SGS and gSLOPE,
shown for different values of the correlation ($\rho$) and $p$. The results
are averaged over $100$ repetitions, with $95\%$ confidence intervals shown.

Table A3: Variable screening metrics for SGS using a linear model for the
simulation study presented in Section 5.1. The number of variables in
$\mathcal{A}_{v},\mathcal{S}_{v},\mathcal{E}_{v}$, and $\mathcal{K}_{v}$ are
shown. The results are shown for different values of $p$, averaged across
$\rho\in\\{0,0.3,0.6,0.9\\}$. Standard errors are shown.
Method | $p$ | $\operatorname{card}(\mathcal{A}_{v})$ | $\operatorname{card}(\mathcal{S}_{v})$ | $\operatorname{card}(\mathcal{E}_{v})$ | $\operatorname{card}(\mathcal{K}_{v})$
---|---|---|---|---|---
SGS | $500$ | $19\pm 1$ | $24\pm 1$ | $28\pm 1$ | $4\pm 0.2$
SGS | $1625$ | $83\pm 3$ | $138\pm 5$ | $161\pm 6$ | $22\pm 1$
SGS | $2750$ | $188\pm 7$ | $316\pm 10$ | $370\pm 12$ | $54\pm 2$
SGS | $3875$ | $268\pm 9$ | $470\pm 14$ | $548\pm 16$ | $78\pm 3$
SGS | $5000$ | $334\pm 10$ | $618\pm 17$ | $712\pm 20$ | $95\pm 3$

Table A4: General and group screening metrics for SGS and gSLOPE using linear
models for the simulation study presented in Section 5.1. General metrics: the
runtime (in seconds) for screening against no screening, the number of fitting
iterations for screening against no screening, and the $\ell_{2}$ distance
between the fitted values obtained with screening and no screening. Group
screening metrics: the number of groups in
$\mathcal{A}_{g},\mathcal{S}_{g},\mathcal{E}_{g}$, and $\mathcal{K}_{g}$. The
results are shown for different values of $p$, averaged across
$\rho\in\\{0,0.3,0.6,0.9\\}$. Standard errors are shown.

Method | $p$ |  Runtime (s) |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---
|  Runtime (s) | screen |  |  |  |  |  |  |  |
| $\operatorname{card}(\mathcal{A}_{g})$ | $\operatorname{card}(\mathcal{S}_{g})$ | $\operatorname{card}(\mathcal{E}_{g})$ | $\operatorname{card}(\mathcal{K}_{g})$ |  Num it |  |  |  |  |
|  Num it |  |  |  | screen |  |  |  |  |
|  $\ell_{2}$ dist |  |  |  |  |  |  |  |  |
| to no screen |  |  |  |  |  |  |  |  |
gSLOPE | $500$ | $89\pm 1$ | $144\pm 1$ | $9\pm 0.2$ | $10\pm 0.3$ | $10\pm 0.3$ | $0.005\pm 0.005$ | $47\pm 1$ | $55\pm 1$ | $6\times 10^{-6}\pm 7\times 10^{-6}$
gSLOPE | $1625$ | $231\pm 5$ | $453\pm 5$ | $26\pm 1$ | $36\pm 1$ | $36\pm 1$ | $0.005\pm 0.006$ | $203\pm 6$ | $222\pm 6$ | $1\times 10^{-6}\pm 1\times 10^{-6}$
gSLOPE | $2750$ | $1061\pm 15$ | $2296\pm 25$ | $56\pm 2$ | $75\pm 2$ | $75\pm 2$ | $0.004\pm 0.005$ | $270\pm 9$ | $350\pm 9$ | $5\times 10^{-7}\pm 7\times 10^{-7}$
gSLOPE | $3875$ | $1765\pm 83$ | $2800\pm 113$ | $82\pm 3$ | $114\pm 3$ | $114\pm 3$ | $0.006\pm 0.008$ | $549\pm 19$ | $546\pm 18$ | $3\times 10^{-7}\pm 5\times 10^{-7}$
gSLOPE | $5000$ | $1937\pm 63$ | $2422\pm 66$ | $102\pm 3$ | $147\pm 4$ | $147\pm 4$ | $0.007\pm 0.012$ | $594\pm 21$ | $581\pm 19$ | $2\times 10^{-7}\pm 3\times 10^{-7}$
SGS | $500$ | $94\pm 1$ | $133\pm 2$ | $9\pm 0.2$ | $11\pm 0.3$ | $12\pm 0.3$ | $3\pm 0.2$ | $28\pm 1$ | $74\pm 3$ | $5\times 10^{-10}\pm 4\times 10^{-10}$
SGS | $1625$ | $416\pm 8$ | $1129\pm 19$ | $25\pm 1$ | $37\pm 1$ | $41\pm 1$ | $12\pm 0.3$ | $62\pm 7$ | $511\pm 21$ | $2\times 10^{-9}\pm 2\times 10^{-9}$
SGS | $2750$ | $639\pm 14$ | $2137\pm 47$ | $62\pm 2$ | $82\pm 2$ | $89\pm 3$ | $20\pm 1$ | $80\pm 6$ | $791\pm 28$ | $2\times 10^{-9}\pm 6\times 10^{-9}$
SGS | $3875$ | $939\pm 31$ | $2862\pm 96$ | $93\pm 3$ | $124\pm 3$ | $136\pm 4$ | $30\pm 1$ | $112\pm 8$ | $1049\pm 34$ | $5\times 10^{-9}\pm 1\times 10^{-8}$
SGS | $5000$ | $1586\pm 66$ | $2891\pm 128$ | $119\pm 4$ | $164\pm 3$ | $180\pm 4$ | $39\pm 1$ | $171\pm 11$ | $1118\pm 37$ | $2\times 10^{-9}\pm 4\times 10^{-9}$

#####  F.3.2 Additional results for the logistic model

This section presents additional results for the logistic model. Similar
trends to the ones observed for the linear model are seen.

Figure A10: The number of groups/variables in $\mathcal{E},\mathcal{A}$ as a
function of the regularization path for the logistic model with
$p=2750,\rho=0.6,m=197$, shown for gSLOPE and SGS. The results are averaged
over $100$ repetitions, with $95\%$ confidence intervals shown. This figure is
the equivalent of Figure 2 for the logistic model.

(a) $p=500,\rho=0$

(b) $p=500,\rho=0.9$

(c) $p=5000,\rho=0$

(d) $p=5000,\rho=0.9$

Figure A11: The number of groups/variables in $\mathcal{E},\mathcal{A}$ as a
function of the regularization path for the logistic model with SGS and
gSLOPE, shown for different values of the correlation ($\rho$) and $p$. The
results are averaged over $100$ repetitions, with $95\%$ confidence intervals
shown.

Figure A12: Runtime (in seconds) for screening against no screening as a
function of $p$, broken down into different correlation cases, for the
logistic model. The results are averaged over $100$ repetitions, with standard
errors shown.

Table A5: Variable screening metrics for SGS using a logistic model for the
simulation study presented in Section 5.1. The number of variables in
$\mathcal{A}_{v},\mathcal{S}_{v},\mathcal{E}_{v}$, and $\mathcal{K}_{v}$ are
shown. The results are shown for different values of $p$, averaged across
$\rho\in\\{0,0.3,0.6,0.9\\}$. Standard errors are shown.
Method | $p$ | $\operatorname{card}(\mathcal{A}_{v})$ | $\operatorname{card}(\mathcal{S}_{v})$ | $\operatorname{card}(\mathcal{E}_{v})$ | $\operatorname{card}(\mathcal{K}_{v})$
---|---|---|---|---|---
SGS | $500$ | $53\pm 2$ | $71\pm 3$ | $89\pm 4$ | $19\pm 1$
SGS | $1625$ | $157\pm 5$ | $248\pm 8$ | $291\pm 9$ | $44\pm 1$
SGS | $2750$ | $247\pm 7$ | $420\pm 11$ | $491\pm 13$ | $71\pm 2$
SGS | $3875$ | $316\pm 9$ | $571\pm 14$ | $663\pm 16$ | $92\pm 3$
SGS | $5000$ | $375\pm 10$ | $717\pm 16$ | $824\pm 19$ | $107\pm 3$

Table A6: General and group screening metrics for SGS and gSLOPE using
logistic models for the simulation study presented in Section 5.1. General
metrics: the runtime (in seconds) for screening against no screening, the
number of fitting iterations for screening against no screening, and the
$\ell_{2}$ distance between the fitted values obtained with screening and no
screening. Group screening metrics: the number of groups in
$\mathcal{A}_{g},\mathcal{S}_{g},\mathcal{E}_{g}$, and $\mathcal{K}_{g}$. The
results are shown for different values of $p$, averaged across
$\rho\in\\{0,0.3,0.6,0.9\\}$. Standard errors are shown.

Method | $p$ |  Runtime (s) |  |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---|---
|  Runtime (s) | screen |  |  |  |  |  |  |  |
| $\operatorname{card}(\mathcal{A}_{g})$ | $\operatorname{card}(\mathcal{S}_{g})$ | $\operatorname{card}(\mathcal{E}_{g})$ | $\operatorname{card}(\mathcal{K}_{g})$ |  Num it |  |  |  |  |
|  Num it |  |  |  | screen |  |  |  |  |
|  $\ell_{2}$ dist |  |  |  |  |  |  |  |  |
| to no screen |  |  |  |  |  |  |  |  |
gSLOPE | $500$ | $49\pm 1$ | $76\pm 1$ | $26\pm 1$ | $31\pm 1$ | $31\pm 1$ | $0.003\pm 0.004$ | $31\pm 1$ | $40\pm 1$ | $4\times 10^{-8}\pm 5\times 10^{-8}$
gSLOPE | $1625$ | $138\pm 3$ | $203\pm 3$ | $43\pm 1$ | $54\pm 2$ | $54\pm 2$ | $0.004\pm 0.006$ | $78\pm 2$ | $79\pm 1$ | $1\times 10^{-8}\pm 5\times 10^{-9}$
gSLOPE | $2750$ | $987\pm 11$ | $1641\pm 16$ | $72\pm 2$ | $95\pm 3$ | $95\pm 3$ | $0.008\pm 0.013$ | $87\pm 2$ | $89\pm 1$ | $1\times 10^{-8}\pm 5\times 10^{-9}$
gSLOPE | $3875$ | $1441\pm 26$ | $2398\pm 31$ | $98\pm 3$ | $135\pm 3$ | $135\pm 3$ | $0.031\pm 0.054$ | $95\pm 2$ | $98\pm 1$ | $7\times 10^{-9}\pm 3\times 10^{-9}$
gSLOPE | $5000$ | $1454\pm 29$ | $2727\pm 40$ | $118\pm 3$ | $168\pm 4$ | $168\pm 4$ | $0.022\pm 0.041$ | $101\pm 2$ | $109\pm 1$ | $4\times 10^{-9}\pm 1\times 10^{-9}$
SGS | $500$ | $118\pm 1$ | $113\pm 1$ | $28\pm 1$ | $33\pm 1$ | $38\pm 1$ | $8\pm 0.2$ | $6\pm 0.3$ | $29\pm 1$ | $8\times 10^{-9}\pm 7\times 10^{-10}$
SGS | $1625$ | $248\pm 2$ | $538\pm 9$ | $50\pm 2$ | $59\pm 2$ | $64\pm 2$ | $11\pm 0.4$ | $7\pm 1$ | $63\pm 3$ | $5\times 10^{-9}\pm 1\times 10^{-9}$
SGS | $2750$ | $374\pm 2$ | $868\pm 12$ | $85\pm 3$ | $104\pm 2$ | $115\pm 3$ | $21\pm 1$ | $7\pm 0.4$ | $57\pm 2$ | $3\times 10^{-9}\pm 5\times 10^{-10}$
SGS | $3875$ | $558\pm 4$ | $1280\pm 19$ | $116\pm 3$ | $148\pm 3$ | $164\pm 3$ | $30\pm 1$ | $8\pm 0.4$ | $54\pm 1$ | $2\times 10^{-9}\pm 2\times 10^{-10}$
SGS | $5000$ | $737\pm 5$ | $1498\pm 19$ | $141\pm 4$ | $188\pm 3$ | $209\pm 4$ | $40\pm 1$ | $8\pm 0.4$ | $54\pm 1$ | $1\times 10^{-9}\pm 2\times 10^{-10}$

####  F.4 Additional results from the real data analysis

Table A7: Variable screening metrics for SGS applied to real data in 5.2. The
number of variables in $\mathcal{A}_{v},\mathcal{S}_{v},\mathcal{E}_{v}$, and
$\mathcal{K}_{v}$ are shown. The results are averaged across the nine pathway
collections, with standard errors shown.
Dataset | Method |  Input |  |  |  |
---|---|---|---|---|---|---
| $\operatorname{card}(\mathcal{A}_{v})$ | $\operatorname{card}(\mathcal{S}_{v})$ | $\operatorname{card}(\mathcal{E}_{v})$ | $\operatorname{card}(\mathcal{K}_{v})$ |  |
SGS | Cancer | $5651$ | $71\pm 1$ | $211\pm 4$ | $212\pm 4$ | $1\pm 0.04$
SGS | Colitis | $10259$ | $25\pm 1$ | $208\pm 6$ | $209\pm 6$ | $1\pm 0.07$
Table A8: General and group screening metrics for SGS and gSLOPE applied to
real data in 5.2. General metrics: the runtime (in seconds) for screening
against no screening, the number of fitting iterations for screening against
no screening (with the number of occasions of failed convergence given in
brackets), and the $\ell_{2}$ distance between the fitted values obtained with
screening and no screening. Group screening metrics: the number of groups in
$\mathcal{A}_{g},\mathcal{S}_{g},\mathcal{E}_{g}$, and $\mathcal{K}_{g}$. The
results are averaged across the nine pathway collections, with standard errors
shown.
Method | Dataset |  Input |  |  |  |  |  |  |
---|---|---|---|---|---|---|---|---|---
| $\operatorname{card}(\mathcal{A}_{g})$ | $\operatorname{card}(\mathcal{S}_{g})$ | $\operatorname{card}(\mathcal{E}_{g})$ | $\operatorname{card}(\mathcal{K}_{g})$ |  Num it |  |  |  |
|  Num it |  |  |  | screen (num failed) |  |  |  |
|  $\ell_{2}$ dist |  |  |  |  |  |  |  |
| to no screen |  |  |  |  |  |  |  |
gSLOPE | Cancer | $592$ | $66\pm 1$ | $120\pm 2$ | $120\pm 2$ | $0\pm 0$ | $1719\pm 63(16)$ | $4453\pm 103(186)$ | $7\times 10^{-8}\pm 7\times 10^{-9}$
gSLOPE | Colitis | $655$ | $32\pm 1$ | $56\pm 2$ | $56\pm 2$ | $0\pm 0$ | $1215\pm 73(28)$ | $3904\pm 127(140)$ | $3\times 10^{-7}\pm 5\times 10^{-8}$
SGS | Cancer | $592$ | $50\pm 1$ | $167\pm 2$ | $115\pm 2$ | $67\pm 0.8$ | $72\pm 3(0)$ | $127\pm 4(0)$ | $1\times 10^{-8}\pm 1\times 8^{-10}$
SGS | Colitis | $655$ | $18\pm 0.6$ | $136\pm 4$ | $87\pm 3$ | $61\pm 2$ | $156\pm 9(0)$ | $1348\pm 62(0)$ | $4\times 10^{-8}\pm 6\times 10^{-9}$