Mod Min | 18.13 (0.63) | 110.67 (1.3) | 9.14 (0.32) | 51.99 (0.66)
Mod 1SE | 34.64 (0.96) | 132.97 (1.54) | 18.08 (0.54) | 65.4 (0.89)
AR2 Min | 18.35 (0.63) | 110.69 (1.29) | 9.14 (0.32) | 52.05 (0.66)
AR2 1SE | 35.3 (0.99) | 133.79 (1.56) | 19.99 (0.54) | 68.21 (0.91)
R2 Cut | 77.07 (2.09) | 187.98 (2.28) | 50.31 (1.82) | 112.37 (1.51)
AIC | 24.12 (0.64) | 112.62 (1.23) | 12.98 (0.35) | 57.37 (0.65)
BIC | 29.12 (0.83) | 136.75 (1.81) | 15.13 (0.42) | 70.12 (0.87)
ERIC | 32.23 (0.91) | 164.4 (2.56) | 16.47 (0.43) | 81.64 (1.11)
GCV | 24.46 (0.66) | 112.66 (1.24) | 13.16 (0.36) | 57.28 (0.64)
Relaxed Min | 13.75 (0.61) | 102.47 (1.77) | 7.96 (0.34) | 43.42 (0.75)
Relaxed 1SE | 34.52 (0.96) | 130.12 (1.81) | 17.97 (0.47) | 59.71 (0.84)
Table 28: Average false negatives for 100 replications of simulated data with $n=1000$. Standard errors given in parentheses. | SNR = 1.25 | SNR = 5
---|---|---
| $p^{*}=5$ | $p^{*}=33$ | $p^{*}=5$ | $p^{*}=33$
CV Min | 0 (0) | 0.09 (0.03) | 0 (0) | 0.01 (0.01)
CV 1SE | 0.01 (0.01) | 0.45 (0.06) | 0 (0) | 0.02 (0.01)
Mod Min | 0.01 (0.01) | 0.29 (0.05) | 0 (0) | 0.02 (0.01)
Mod 1SE | 0.06 (0.02) | 1.05 (0.1) | 0 (0) | 0.07 (0.03)
AR2 Min | 0.01 (0.01) | 0.24 (0.05) | 0 (0) | 0.02 (0.01)
AR2 1SE | 0.08 (0.03) | 1.1 (0.11) | 0 (0) | 0.1 (0.03)
R2 Cut | 0.07 (0.03) | 1.2 (0.12) | 0 (0) | 0.1 (0.04)
AIC | 0 (0) | 0.09 (0.03) | 0 (0) | 0.01 (0.01)
BIC | 0.01 (0.01) | 0.47 (0.07) | 0 (0) | 0.02 (0.01)
ERIC | 0.01 (0.01) | 1.01 (0.11) | 0 (0) | 0.03 (0.02)
GCV | 0 (0) | 0.07 (0.03) | 0 (0) | 0.01 (0.01)
Relaxed Min | 0.01 (0.01) | 1.17 (0.12) | 0 (0) | 0.18 (0.05)
Relaxed 1SE | 0.05 (0.02) | 1.71 (0.16) | 0 (0) | 0.27 (0.05)

### B.2 Simulation Study for Correlated Predictors

Additional results from the simulation study described in the main document
for correlated predictors can be found in Tables 37 through 48. These results
include average model size, average FDR, average false negatives, and average
prediction bias for traditional CV, $\alpha$-modified and Average $R^{2}$ CV,
and the Relaxed Lasso using a minimum APE rule.

Table 29: Average Hamming Distance for 100 replications of CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  5.1 (0.4) |  6.3 (0.4) |  8.8 (0.8) |  11.4 (1.3) |  11.5 (1.1) |  |  3.8 (0.3) |  5.8 (0.4) |  9.8 (1.1) |  13.4 (1.4) |  14.9 (1.2)
500 |  |  1.9 (0.2) |  3.2 (0.4) |  4.4 (0.4) |  5.1 (0.6) |  7.6 (0.6) |  |  1.8 (0.2) |  2.9 (0.3) |  4.7 (0.4) |  4.9 (0.4) |  7.2 (0.7)
1000 |  |  1.5 (0.2) |  1.9 (0.2) |  3 (0.4) |  4.1 (0.4) |  4.3 (0.5) |  |  1.2 (0.2) |  1.4 (0.2) |  2.9 (0.4) |  4.4 (0.5) |  4.3 (0.5)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  12.4 (0.3) |  16.2 (0.6) |  18.5 (0.7) |  22 (1.3) |  22.7 (1.5) |  |  11 (0.4) |  15.5 (0.6) |  21.6 (1.1) |  24.7 (1.5) |  30.2 (1.7)
500 |  |  7.1 (0.4) |  10.1 (0.4) |  12.6 (0.6) |  15 (0.6) |  18.9 (1) |  |  5.1 (0.3) |  7.4 (0.4) |  12.4 (0.6) |  16.2 (0.7) |  20.2 (0.9)
1000 |  |  4.6 (0.3) |  6.8 (0.4) |  9.4 (0.6) |  12.7 (0.7) |  14.7 (0.8) |  |  3.5 (0.3) |  5.1 (0.3) |  9.3 (0.5) |  11.5 (0.7) |  14.2 (0.9)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  39.3 (0.8) |  46.7 (0.4) |  50.8 (0.4) |  53.7 (0.8) |  57.5 (1.2) |  |  20.8 (1.4) |  39.3 (0.6) |  48.6 (0.6) |  55.8 (1.1) |  59.2 (1.2)
500 |  |  13.5 (1.3) |  28.2 (1) |  40.2 (0.9) |  48.4 (0.9) |  55.3 (1.5) |  |  3.1 (0.6) |  19.5 (0.7) |  35.8 (0.8) |  56.5 (1.7) |  74.3 (2.2)
1000 |  |  8.4 (1.1) |  21.4 (1) |  33.1 (0.9) |  42.3 (1.1) |  54.3 (1.3) |  |  1.4 (0.4) |  15.3 (0.5) |  29.2 (0.8) |  43.2 (1.1) |  60.3 (1.6)
Table 30: Average Hamming Distance for 100 replications of AR2 CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  4.3 (0.3) |  4.7 (0.4) |  5.8 (0.4) |  7.1 (0.8) |  6.8 (0.4) |  |  2.3 (0.2) |  3.1 (0.3) |  5 (0.5) |  5.5 (0.6) |  7.1 (0.6)
500 |  |  1.3 (0.1) |  1.7 (0.2) |  2.4 (0.3) |  2.3 (0.3) |  4 (0.5) |  |  0.7 (0.1) |  1 (0.2) |  1.7 (0.2) |  1.8 (0.3) |  2 (0.3)
1000 |  |  0.8 (0.1) |  1 (0.2) |  1.3 (0.2) |  1.5 (0.2) |  1.5 (0.3) |  |  0.3 (0.1) |  0.6 (0.1) |  1 (0.2) |  1.2 (0.2) |  1.8 (0.4)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  12.7 (0.3) |  15.5 (0.5) |  17.4 (0.8) |  17.6 (0.8) |  18 (0.6) |  |  9.6 (0.4) |  13.4 (0.5) |  18.3 (0.9) |  19 (1.1) |  22.2 (1.3)
500 |  |  6.6 (0.4) |  8.3 (0.5) |  9.2 (0.5) |  11.1 (0.5) |  13.7 (0.7) |  |  3.7 (0.3) |  4.5 (0.4) |  6.6 (0.5) |  8.8 (0.6) |  11.4 (0.9)
1000 |  |  3.4 (0.3) |  4.9 (0.4) |  6.4 (0.5) |  8.1 (0.6) |  9.9 (0.7) |  |  1.6 (0.2) |  2.5 (0.3) |  4.4 (0.5) |  5.9 (0.6) |  7.4 (0.7)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  40.4 (1.1) |  46.5 (0.5) |  50.8 (0.5) |  53.1 (0.8) |  54.6 (1) |  |  23.4 (1.7) |  39.8 (0.7) |  48.5 (0.5) |  53 (0.8) |  54.3 (0.8)
500 |  |  14.4 (1.5) |  28.6 (1.2) |  38.7 (0.9) |  47.1 (1) |  52.4 (1.3) |  |  4.2 (0.8) |  17.3 (0.8) |  29.5 (1) |  43.6 (1.1) |  61.1 (1.9)
1000 |  |  9.5 (1.2) |  20.5 (1.1) |  29.6 (1.1) |  36.4 (1.1) |  47.2 (1.5) |  |  1.7 (0.5) |  11.6 (0.7) |  19.9 (1) |  29.5 (1.3) |  40.6 (1.6)
Table 31: Average Hamming Distance for 100 replications of $\alpha$-modified CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  4.3 (0.3) |  4.7 (0.3) |  6.7 (0.8) |  8.7 (1.3) |  9.7 (1.4) |  |  2.7 (0.2) |  3.6 (0.3) |  7.3 (1.1) |  7.6 (0.9) |  10.3 (1)
500 |  |  1.1 (0.1) |  1.7 (0.2) |  2.2 (0.2) |  2.5 (0.3) |  3.5 (0.4) |  |  1.1 (0.1) |  1.1 (0.2) |  2.4 (0.3) |  2.3 (0.3) |  2.6 (0.3)
1000 |  |  0.7 (0.1) |  1.1 (0.2) |  1.3 (0.2) |  1.8 (0.3) |  1.6 (0.2) |  |  0.6 (0.1) |  0.8 (0.1) |  1.3 (0.2) |  1.6 (0.3) |  2.3 (0.4)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  12.6 (0.3) |  15.4 (0.5) |  16.4 (0.6) |  20.4 (1.5) |  20.9 (1.7) |  |  10.1 (0.4) |  14.2 (0.5) |  19.8 (1.1) |  21.7 (1.5) |  28.6 (1.9)
500 |  |  6.4 (0.4) |  8.1 (0.4) |  9.2 (0.5) |  11.1 (0.5) |  12.3 (0.6) |  |  4 (0.3) |  5.2 (0.4) |  7.8 (0.6) |  10.6 (0.7) |  13 (1)
1000 |  |  3.4 (0.3) |  4.7 (0.4) |  6.5 (0.5) |  8.1 (0.6) |  10 (0.7) |  |  2.2 (0.2) |  3.1 (0.3) |  5.8 (0.5) |  7.3 (0.7) |  8.6 (0.7)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  42.9 (1) |  48 (0.3) |  50.5 (0.4) |  53.6 (1) |  56.5 (1.6) |  |  22.1 (1.6) |  40.4 (0.7) |  49 (0.6) |  56.4 (1.4) |  58.7 (1.6)
500 |  |  14.7 (1.5) |  28.9 (1.2) |  37.7 (1) |  44.5 (0.9) |  49 (1) |  |  3.3 (0.6) |  17.7 (0.8) |  30.6 (0.9) |  47.2 (1.2) |  64.3 (1.9)
1000 |  |  9.3 (1.3) |  20.4 (1.2) |  29.1 (1.1) |  34.4 (1.2) |  41.6 (0.9) |  |  1.4 (0.4) |  12.2 (0.7) |  21.4 (1) |  32.1 (1.3) |  45.2 (1.8)
Table 32: Average Hamming Distance for 100 replications of Relaxed Lasso with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  6.3 (0.5) |  9 (0.9) |  14.2 (2) |  19.9 (3.3) |  35.9 (5.2) |  |  3.6 (0.4) |  6.1 (0.8) |  15.1 (2.6) |  21.8 (3.7) |  31.8 (4.9)
500 |  |  2.8 (0.4) |  3.9 (0.6) |  4.8 (0.6) |  5.6 (0.9) |  8.8 (1.7) |  |  1.7 (0.3) |  1.6 (0.4) |  2.7 (0.5) |  3.3 (0.6) |  4 (1.2)
1000 |  |  1.9 (0.4) |  2.2 (0.4) |  3.7 (0.8) |  4 (0.7) |  6.3 (1.8) |  |  1.1 (0.3) |  1.4 (0.4) |  2 (0.5) |  2.4 (0.7) |  1.5 (0.4)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  13.9 (0.5) |  21.5 (0.9) |  27.6 (2.4) |  37.6 (3.7) |  59.2 (6) |  |  12.9 (0.6) |  19.1 (1.1) |  30.1 (2.3) |  49 (4.4) |  66.5 (5.6)
500 |  |  8.9 (0.6) |  11.6 (1) |  13.5 (1.1) |  14.9 (1.4) |  21 (2) |  |  4 (0.5) |  5 (0.7) |  7.3 (1.1) |  8.3 (1.4) |  9.7 (1.2)
1000 |  |  4.7 (0.5) |  7.7 (0.9) |  11.6 (1.5) |  12.3 (1.5) |  13.7 (1.6) |  |  1.5 (0.3) |  3 (0.7) |  4.6 (0.8) |  5.4 (1) |  5.7 (1.1)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  33.6 (1.2) |  44.6 (0.5) |  56.6 (1.4) |  67.3 (2.9) |  78.3 (4.1) |  |  14.8 (1.4) |  38.8 (0.6) |  63.6 (2.6) |  83.5 (4.1) |  97.5 (5.3)
500 |  |  5.7 (0.9) |  30.1 (0.9) |  48.3 (1.4) |  62.5 (2.2) |  74.8 (3.1) |  |  0.9 (0.3) |  18.1 (1.2) |  30.2 (1.9) |  51.1 (3.1) |  68.8 (4.2)
1000 |  |  2.1 (0.5) |  23.2 (1.2) |  38.8 (1.9) |  46.1 (2.9) |  63.9 (3.8) |  |  0.3 (0.2) |  12.2 (1.2) |  15.6 (1.6) |  22.4 (2.7) |  31.7 (3.8)
Table 33: Average number of selected variables 100 replications of CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  6.5 (0.4) |  7.4 (0.4) |  9.1 (0.9) |  11.2 (1.4) |  10.7 (1.2) |  |  8 (0.3) |  10 (0.4) |  13.7 (1.1) |  16.8 (1.4) |  18.2 (1.2)
500 |  |  6.3 (0.2) |  7.4 (0.3) |  8.4 (0.4) |  8.9 (0.5) |  11.3 (0.6) |  |  6.7 (0.2) |  7.8 (0.3) |  9.5 (0.4) |  9.7 (0.4) |  11.9 (0.7)
1000 |  |  6.1 (0.2) |  6.5 (0.2) |  7.4 (0.3) |  8.4 (0.4) |  8.6 (0.5) |  |  6.2 (0.2) |  6.3 (0.2) |  7.8 (0.4) |  9.4 (0.5) |  9.2 (0.5)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  10.5 (0.6) |  11.4 (1) |  9.2 (1) |  11.9 (1.6) |  11.2 (1.7) |  |  19.2 (0.7) |  22.1 (0.9) |  25.3 (1.5) |  24 (1.9) |  28.3 (2.1)
500 |  |  15.6 (0.4) |  19.2 (0.6) |  20.2 (0.7) |  22.1 (0.8) |  24.3 (1.1) |  |  18.8 (0.3) |  21.1 (0.4) |  25.4 (0.6) |  29.4 (0.8) |  33.2 (1)
1000 |  |  17.2 (0.3) |  18.7 (0.4) |  20.7 (0.5) |  23 (0.7) |  24.6 (0.7) |  |  18.3 (0.3) |  19.6 (0.3) |  23.8 (0.5) |  25.7 (0.6) |  28.4 (0.8)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  10.7 (0.8) |  9 (0.9) |  8.7 (1) |  8.6 (1.1) |  11.2 (1.7) |  |  29.2 (1.4) |  27.6 (1.9) |  21.4 (1.8) |  20 (2.1) |  18.4 (1.8)
500 |  |  36.5 (1.3) |  40.9 (1.8) |  45.1 (2.4) |  41.6 (2.7) |  42.1 (3.1) |  |  46.9 (0.6) |  59.1 (1.3) |  70.6 (2.1) |  88.4 (3.2) |  103.4 (4.1)
1000 |  |  41.6 (1.1) |  49.3 (1.7) |  56.3 (2.3) |  60.7 (2.5) |  66.5 (3.2) |  |  48.6 (0.4) |  61.6 (0.7) |  74.7 (1.2) |  87.3 (1.6) |  102.5 (2.3)
Table 34: Average number of selected variables 100 replications of AR2 CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  4.6 (0.4) |  4.5 (0.5) |  5 (0.5) |  6.1 (0.9) |  5.3 (0.5) |  |  5.7 (0.3) |  6.5 (0.4) |  7.9 (0.5) |  7.8 (0.6) |  9.3 (0.7)
500 |  |  4.9 (0.2) |  5.2 (0.2) |  5.5 (0.4) |  5.1 (0.3) |  6.9 (0.5) |  |  5.3 (0.1) |  5.6 (0.2) |  6.3 (0.3) |  6.4 (0.3) |  6.4 (0.3)
1000 |  |  5 (0.1) |  5.3 (0.2) |  5.2 (0.2) |  5.4 (0.3) |  5.3 (0.3) |  |  5.2 (0.1) |  5.5 (0.1) |  5.8 (0.2) |  6 (0.2) |  6.7 (0.4)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  9 (0.8) |  9.1 (1) |  7.6 (1.2) |  6.3 (1.1) |  6 (0.8) |  |  15.7 (0.8) |  17.6 (1) |  19.7 (1.5) |  15.1 (1.6) |  17.5 (1.9)
500 |  |  13.1 (0.5) |  15.4 (0.7) |  14.6 (0.7) |  16.6 (0.8) |  17.2 (1.2) |  |  16.3 (0.4) |  17.6 (0.4) |  18.6 (0.6) |  21.4 (0.7) |  23.8 (0.9)
1000 |  |  14.9 (0.3) |  15.4 (0.5) |  16.3 (0.6) |  16.7 (0.7) |  18.2 (0.9) |  |  16.2 (0.2) |  16.8 (0.3) |  18.3 (0.5) |  19.6 (0.6) |  21.1 (0.7)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  9.6 (1.1) |  9.2 (1.3) |  9.5 (1.4) |  7.6 (1.3) |  7.6 (1.5) |  |  26.6 (1.7) |  25.4 (2) |  18.3 (2) |  12.3 (1.9) |  10.2 (1.4)
500 |  |  35.6 (1.5) |  38.2 (2.1) |  39.7 (2.6) |  37.4 (3) |  36.2 (3.3) |  |  45.8 (0.8) |  54.6 (1.3) |  61.6 (2.1) |  72.8 (2.7) |  86.7 (4)
1000 |  |  40.5 (1.2) |  45.8 (1.8) |  49 (2.3) |  51.3 (2.4) |  56.1 (3.3) |  |  48.3 (0.5) |  56.6 (0.8) |  64.3 (1.1) |  72.3 (1.6) |  81.2 (1.9)
Table 35: Average number of selected variables 100 replications of $\alpha$-modified CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  4.6 (0.4) |  4.5 (0.4) |  5.7 (0.9) |  7.6 (1.4) |  8 (1.5) |  |  6.4 (0.3) |  7.2 (0.3) |  10.7 (1.1) |  10.4 (0.9) |  12.8 (1.1)
500 |  |  4.9 (0.2) |  5.4 (0.2) |  5.2 (0.3) |  5.3 (0.3) |  6.2 (0.5) |  |  5.9 (0.2) |  5.9 (0.2) |  7.2 (0.3) |  6.9 (0.4) |  7.1 (0.3)
1000 |  |  5 (0.1) |  5.5 (0.2) |  5.2 (0.2) |  5.6 (0.3) |  5.4 (0.3) |  |  5.5 (0.1) |  5.7 (0.1) |  6.2 (0.2) |  6.6 (0.3) |  7.1 (0.4)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  7.6 (0.7) |  7.8 (1) |  4.8 (0.9) |  9.3 (2) |  8.5 (1.9) |  |  16.9 (0.8) |  19.7 (1) |  22 (1.7) |  18.9 (2) |  25.2 (2.5)
500 |  |  13.4 (0.6) |  15.1 (0.7) |  14.7 (0.8) |  16.4 (0.8) |  15.4 (0.9) |  |  17.4 (0.3) |  18.6 (0.5) |  20 (0.7) |  23.5 (0.7) |  25.4 (1)
1000 |  |  15.5 (0.3) |  15.4 (0.5) |  16.3 (0.6) |  16.6 (0.8) |  18.4 (0.8) |  |  16.9 (0.3) |  17.6 (0.3) |  20 (0.5) |  21.3 (0.7) |  22.5 (0.8)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  7.1 (1) |  5 (0.8) |  6 (1.2) |  7.4 (1.7) |  9.4 (2.2) |  |  27.9 (1.6) |  24.7 (2.1) |  18.6 (2.2) |  18.6 (2.7) |  16.1 (2.4)
500 |  |  35.3 (1.5) |  36.2 (2) |  35.4 (2.4) |  28.1 (2.6) |  25.2 (2.9) |  |  46.7 (0.6) |  56.3 (1.3) |  64.2 (2) |  77.4 (2.8) |  90.5 (4.1)
1000 |  |  40.7 (1.3) |  45.2 (1.9) |  48.1 (2.3) |  47.5 (2.3) |  47.4 (2.8) |  |  48.5 (0.4) |  58.2 (0.7) |  66.7 (1) |  75.7 (1.5) |  86.6 (2.1)
Table 36: Average number of selected variables 100 replications of Relaxed Lasso with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  8.3 (0.6) |  10.6 (0.9) |  14.8 (2.1) |  20.3 (3.4) |  36.1 (5.3) |  |  7.9 (0.5) |  10.4 (0.8) |  19.1 (2.6) |  25.3 (3.7) |  35.3 (4.9)
500 |  |  7.6 (0.4) |  8.4 (0.6) |  9.1 (0.6) |  9.7 (0.9) |  12.7 (1.7) |  |  6.7 (0.3) |  6.6 (0.4) |  7.6 (0.5) |  8.2 (0.6) |  8.7 (1.2)
1000 |  |  6.8 (0.4) |  7.1 (0.4) |  8.4 (0.8) |  8.6 (0.7) |  10.8 (1.8) |  |  6.1 (0.3) |  6.4 (0.4) |  6.9 (0.5) |  7.4 (0.7) |  6.4 (0.4)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  15.9 (1) |  19.8 (1.5) |  23.1 (2.9) |  31.1 (4.2) |  51.8 (6.5) |  |  22.6 (0.9) |  27.3 (1.3) |  35.3 (2.7) |  51.3 (4.9) |  67.6 (6.2)
500 |  |  19.4 (0.7) |  21.9 (1) |  22 (1.2) |  22.5 (1.5) |  27.5 (1.9) |  |  18 (0.5) |  18.9 (0.7) |  20.6 (1) |  21.5 (1.3) |  22.2 (1.1)
1000 |  |  17.9 (0.5) |  20.9 (0.9) |  23.7 (1.6) |  23.4 (1.4) |  24 (1.6) |  |  16.4 (0.3) |  17.7 (0.6) |  19.2 (0.8) |  19.9 (0.9) |  19.9 (1)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  16.4 (1.2) |  17.4 (1.5) |  24.8 (2.9) |  30.5 (4.2) |  37.4 (5.1) |  |  35.2 (1.4) |  40.9 (2.1) |  56.2 (5) |  63 (6) |  68 (6.7)
500 |  |  44.3 (0.9) |  54.3 (2) |  62.5 (3) |  65.2 (4.1) |  70.7 (5) |  |  49.1 (0.3) |  60.1 (1.3) |  67.4 (2) |  84.5 (3.6) |  98.8 (4.9)
1000 |  |  47.9 (0.5) |  60.1 (1.5) |  70 (2.5) |  69.8 (3.1) |  81.7 (4.5) |  |  49.7 (0.2) |  59.4 (1.2) |  60.2 (1.5) |  64.5 (2.6) |  71.9 (3.6)
Table 37: Average false discovery rate from 100 replications of CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  0.4 (0) |  0.5 (0) |  0.6 (0) |  0.6 (0) |  0.7 (0) |  |  0.4 (0) |  0.5 (0) |  0.6 (0) |  0.6 (0) |  0.7 (0)
500 |  |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0) |  0.5 (0) |  |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0) |  0.5 (0)
1000 |  |  0.2 (0) |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.3 (0) |  |  0.1 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.3 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  0.3 (0) |  0.5 (0) |  0.6 (0) |  0.7 (0) |  0.6 (0) |  |  0.4 (0) |  0.5 (0) |  0.6 (0) |  0.6 (0) |  0.7 (0)
500 |  |  0.2 (0) |  0.4 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0) |  |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.6 (0)
1000 |  |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  |  0.2 (0) |  0.2 (0) |  0.4 (0) |  0.4 (0) |  0.4 (0)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  0 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.6 (0) |  |  0 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.6 (0)
500 |  |  0 (0) |  0.2 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0) |  |  0 (0) |  0.2 (0) |  0.4 (0) |  0.5 (0) |  0.6 (0)
1000 |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  |  0 (0) |  0.2 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0)
Table 38: Average false discovery rate for 100 replications of AR2 CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  0.3 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0) |  |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0) |  0.5 (0)
500 |  |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.1 (0) |  0.3 (0) |  |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.1 (0) |  0.2 (0)
1000 |  |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0) |  |  0 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.6 (0) |  0.6 (0) |  |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0) |  0.6 (0)
500 |  |  0.2 (0) |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.4 (0) |  |  0.1 (0) |  0.2 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0)
1000 |  |  0.1 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.3 (0) |  |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.2 (0) |  0.3 (0)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  0 (0) |  0.3 (0) |  0.5 (0) |  0.6 (0) |  0.7 (0) |  |  0 (0) |  0.3 (0) |  0.3 (0) |  0.4 (0) |  0.6 (0)
500 |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0)
1000 |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0) |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.4 (0)
Table 39: Average false discovery rate of 100 replications for $\alpha$-modified CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  0.3 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0) |  |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0)
500 |  |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0)
1000 |  |  0 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0) |  |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.2 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  0.3 (0) |  0.4 (0) |  0.4 (0) |  0.5 (0) |  0.6 (0) |  |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0) |  0.6 (0)
500 |  |  0.2 (0) |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.4 (0) |  |  0.2 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0)
1000 |  |  0.1 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.3 (0) |  |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.3 (0) |  0.3 (0)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  0 (0) |  0.2 (0) |  0.4 (0) |  0.5 (0) |  0.6 (0) |  |  0 (0) |  0.3 (0) |  0.3 (0) |  0.4 (0) |  0.6 (0)
500 |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0) |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.5 (0) |  0.5 (0)
1000 |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.4 (0) |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.4 (0) |  0.5 (0)
Table 40: Average false discovery rate for 100 replications of Relaxed Lasso with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  0.5 (0) |  0.5 (0) |  0.6 (0) |  0.6 (0) |  0.7 (0) |  |  0.3 (0) |  0.4 (0) |  0.5 (0) |  0.5 (0) |  0.6 (0)
500 |  |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.3 (0) |  0.3 (0) |  |  0.2 (0) |  0.1 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0)
1000 |  |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  0.4 (0) |  0.6 (0) |  0.6 (0) |  0.8 (0) |  0.8 (0) |  |  0.4 (0) |  0.5 (0) |  0.6 (0) |  0.7 (0) |  0.8 (0)
500 |  |  0.3 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0) |  0.5 (0) |  |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.3 (0)
1000 |  |  0.2 (0) |  0.2 (0) |  0.3 (0) |  0.3 (0) |  0.3 (0) |  |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.2 (0)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  0 (0) |  0.3 (0) |  0.5 (0) |  0.7 (0) |  0.7 (0) |  |  0 (0) |  0.3 (0) |  0.5 (0) |  0.6 (0) |  0.8 (0)
500 |  |  0 (0) |  0.3 (0) |  0.5 (0) |  0.6 (0) |  0.6 (0) |  |  0 (0) |  0.2 (0) |  0.3 (0) |  0.5 (0) |  0.5 (0)
1000 |  |  0 (0) |  0.3 (0) |  0.4 (0) |  0.4 (0) |  0.5 (0) |  |  0 (0) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.3 (0)
Table 41: Average number of false negatives from 100 replications of CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  1.8 (0.1) |  2 (0.1) |  2.4 (0.1) |  2.6 (0.2) |  2.9 (0.1) |  |  0.4 (0.1) |  0.4 (0.1) |  0.5 (0.1) |  0.8 (0.1) |  0.9 (0.1)
500 |  |  0.3 (0.1) |  0.4 (0.1) |  0.5 (0.1) |  0.6 (0.1) |  0.6 (0.1) |  |  0.1 (0) |  0 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0)
1000 |  |  0.2 (0) |  0.2 (0) |  0.3 (0.1) |  0.3 (0.1) |  0.3 (0.1) |  |  0 (0) |  0 (0) |  0 (0) |  0 (0) |  0 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  8.4 (0.4) |  9.9 (0.4) |  12.2 (0.3) |  12.5 (0.3) |  13.2 (0.2) |  |  3.4 (0.4) |  4.2 (0.3) |  5.6 (0.4) |  7.8 (0.4) |  8.5 (0.4)
500 |  |  3.2 (0.4) |  3 (0.3) |  3.7 (0.4) |  4 (0.4) |  4.8 (0.4) |  |  0.6 (0.1) |  0.7 (0.1) |  1 (0.2) |  0.9 (0.2) |  1 (0.2)
1000 |  |  1.2 (0.2) |  1.6 (0.2) |  1.9 (0.3) |  2.4 (0.3) |  2.5 (0.3) |  |  0.1 (0) |  0.2 (0.1) |  0.3 (0.1) |  0.4 (0.1) |  0.4 (0.1)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  39.3 (0.8) |  43.8 (0.6) |  46.1 (0.5) |  47.6 (0.2) |  48.2 (0.3) |  |  20.8 (1.4) |  30.9 (1.2) |  38.6 (0.8) |  42.9 (0.6) |  45.4 (0.4)
500 |  |  13.5 (1.3) |  18.7 (1.4) |  22.5 (1.4) |  28.4 (1.3) |  31.6 (1.2) |  |  3.1 (0.6) |  5.2 (0.9) |  7.6 (1) |  9.1 (1.1) |  10.4 (1.2)
1000 |  |  8.4 (1.1) |  11.1 (1.3) |  13.4 (1.3) |  15.8 (1.3) |  18.9 (1.4) |  |  1.4 (0.4) |  1.8 (0.4) |  2.2 (0.5) |  3 (0.5) |  3.9 (0.7)
Table 42: Average number of false negatives for 100 replications of AR2 CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  2.4 (0.2) |  2.6 (0.2) |  2.9 (0.2) |  3 (0.2) |  3.3 (0.2) |  |  0.8 (0.1) |  0.8 (0.1) |  1.1 (0.1) |  1.4 (0.1) |  1.4 (0.2)
500 |  |  0.7 (0.1) |  0.7 (0.1) |  0.9 (0.1) |  1.1 (0.1) |  1 (0.1) |  |  0.2 (0.1) |  0.2 (0) |  0.2 (0) |  0.2 (0) |  0.3 (0.1)
1000 |  |  0.4 (0.1) |  0.3 (0.1) |  0.6 (0.1) |  0.6 (0.1) |  0.6 (0.1) |  |  0.1 (0) |  0 (0) |  0.1 (0) |  0.1 (0) |  0.1 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  9.3 (0.4) |  10.7 (0.4) |  12.4 (0.3) |  13.1 (0.2) |  13.5 (0.2) |  |  4.4 (0.5) |  5.4 (0.4) |  6.8 (0.5) |  9.5 (0.4) |  9.9 (0.4)
500 |  |  4.2 (0.4) |  3.9 (0.4) |  4.8 (0.5) |  4.8 (0.5) |  5.8 (0.5) |  |  1.2 (0.3) |  0.9 (0.2) |  1.5 (0.3) |  1.2 (0.2) |  1.3 (0.2)
1000 |  |  1.7 (0.3) |  2.2 (0.3) |  2.5 (0.4) |  3.2 (0.4) |  3.4 (0.4) |  |  0.2 (0.1) |  0.3 (0.1) |  0.5 (0.2) |  0.7 (0.2) |  0.6 (0.2)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  40.4 (1.1) |  43.7 (0.8) |  45.7 (0.6) |  47.8 (0.3) |  48.5 (0.3) |  |  23.4 (1.7) |  32.2 (1.3) |  40.1 (1) |  45.3 (0.6) |  47.1 (0.4)
500 |  |  14.4 (1.5) |  20.2 (1.6) |  24.5 (1.6) |  29.8 (1.4) |  33.1 (1.3) |  |  4.2 (0.8) |  6.3 (1) |  9 (1.2) |  10.4 (1.3) |  12.2 (1.5)
1000 |  |  9.5 (1.2) |  12.3 (1.4) |  15.3 (1.5) |  17.5 (1.5) |  20.5 (1.6) |  |  1.7 (0.5) |  2.5 (0.6) |  2.8 (0.6) |  3.6 (0.7) |  4.7 (0.8)
Table 43: Average number of false negatives of 100 replications for $\alpha$-modified CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  2.3 (0.2) |  2.6 (0.2) |  3 (0.1) |  3.1 (0.2) |  3.3 (0.1) |  |  0.7 (0.1) |  0.7 (0.1) |  0.8 (0.1) |  1.1 (0.1) |  1.3 (0.1)
500 |  |  0.6 (0.1) |  0.7 (0.1) |  1 (0.1) |  1.1 (0.1) |  1.1 (0.1) |  |  0.1 (0) |  0.1 (0) |  0.1 (0) |  0.2 (0.1) |  0.3 (0.1)
1000 |  |  0.4 (0.1) |  0.3 (0.1) |  0.5 (0.1) |  0.6 (0.1) |  0.6 (0.1) |  |  0 (0) |  0 (0) |  0.1 (0) |  0 (0) |  0.1 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  10 (0.4) |  11.3 (0.4) |  13.3 (0.2) |  13.1 (0.3) |  13.7 (0.2) |  |  4.1 (0.4) |  4.8 (0.4) |  6.4 (0.5) |  8.9 (0.5) |  9.2 (0.4)
500 |  |  4 (0.4) |  4 (0.4) |  4.8 (0.5) |  4.9 (0.5) |  6 (0.5) |  |  0.8 (0.2) |  0.8 (0.2) |  1.4 (0.3) |  1.1 (0.2) |  1.3 (0.2)
1000 |  |  1.4 (0.2) |  2.1 (0.3) |  2.6 (0.4) |  3.3 (0.4) |  3.3 (0.4) |  |  0.1 (0.1) |  0.2 (0.1) |  0.4 (0.1) |  0.5 (0.1) |  0.6 (0.2)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  42.9 (1) |  46.5 (0.5) |  47.3 (0.5) |  48.1 (0.4) |  48.6 (0.3) |  |  22.1 (1.6) |  32.8 (1.3) |  40.2 (0.9) |  43.9 (0.7) |  46.3 (0.5)
500 |  |  14.7 (1.5) |  21.3 (1.6) |  26.1 (1.6) |  33.2 (1.4) |  36.9 (1.3) |  |  3.3 (0.6) |  5.7 (0.9) |  8.2 (1.1) |  9.9 (1.3) |  11.9 (1.4)
1000 |  |  9.3 (1.3) |  12.6 (1.5) |  15.5 (1.5) |  18.4 (1.5) |  22.1 (1.6) |  |  1.4 (0.4) |  2 (0.4) |  2.4 (0.5) |  3.2 (0.6) |  4.3 (0.7)
Table 44: Average number of false negatives for 100 replications of Relaxed Lasso with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  1.5 (0.1) |  1.7 (0.1) |  2.2 (0.1) |  2.3 (0.2) |  2.4 (0.2) |  |  0.4 (0.1) |  0.4 (0.1) |  0.5 (0.1) |  0.8 (0.1) |  0.7 (0.1)
500 |  |  0.1 (0) |  0.2 (0.1) |  0.3 (0.1) |  0.4 (0.1) |  0.6 (0.1) |  |  0 (0) |  0 (0) |  0 (0) |  0 (0) |  0.1 (0)
1000 |  |  0 (0) |  0.1 (0) |  0.1 (0) |  0.2 (0) |  0.2 (0.1) |  |  0 (0) |  0 (0) |  0 (0) |  0 (0) |  0 (0)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  6.5 (0.4) |  8.3 (0.4) |  9.7 (0.4) |  10.7 (0.4) |  11.2 (0.3) |  |  2.7 (0.3) |  3.4 (0.3) |  4.9 (0.4) |  6.4 (0.4) |  7 (0.4)
500 |  |  2.2 (0.3) |  2.4 (0.3) |  3.3 (0.3) |  3.7 (0.3) |  4.3 (0.3) |  |  0.5 (0.1) |  0.6 (0.1) |  0.8 (0.1) |  0.9 (0.2) |  1.2 (0.2)
1000 |  |  0.9 (0.1) |  0.9 (0.2) |  1.5 (0.2) |  1.9 (0.2) |  2.4 (0.2) |  |  0 (0) |  0.1 (0) |  0.2 (0) |  0.3 (0.1) |  0.4 (0.1)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  33.6 (1.2) |  38.6 (0.9) |  40.9 (0.9) |  43.4 (0.7) |  45.5 (0.5) |  |  14.8 (1.4) |  24 (1.2) |  28.7 (1.4) |  35.2 (1.1) |  39.8 (0.8)
500 |  |  5.7 (0.9) |  12.9 (1.2) |  17.9 (1.3) |  23.6 (1.3) |  27 (1.3) |  |  0.9 (0.3) |  4 (0.7) |  6.4 (0.7) |  8.3 (1) |  10.1 (1)
1000 |  |  2.1 (0.5) |  6.5 (0.8) |  9.4 (0.9) |  13.1 (0.9) |  16.1 (1.2) |  |  0.3 (0.2) |  1.4 (0.2) |  2.7 (0.4) |  4 (0.4) |  4.9 (0.5)
Table 45: Average prediction bias from 100 replications of CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  22.8 (0.9) |  24.2 (0.9) |  27.7 (1) |  29.9 (1.4) |  29.2 (1.1) |  |  12 (0.5) |  12.8 (0.6) |  15.6 (0.7) |  16.6 (0.8) |  15.9 (0.7)
500 |  |  29.2 (1.1) |  30.9 (1.4) |  35.2 (1.5) |  35 (1.6) |  38.7 (1.8) |  |  15.1 (0.7) |  15.8 (0.7) |  17.8 (0.8) |  17.4 (0.8) |  19.8 (1)
1000 |  |  33.7 (1.5) |  33.2 (1.4) |  37.5 (1.7) |  39.7 (2) |  38.8 (1.8) |  |  17.6 (0.8) |  16.6 (0.7) |  19.1 (0.9) |  19.2 (0.9) |  19.1 (1)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  47.5 (2) |  50 (1.6) |  54.8 (1.7) |  56.1 (1.6) |  56.7 (1.5) |  |  28 (1.4) |  29.6 (0.9) |  33.7 (1.3) |  38.2 (1.2) |  37.3 (1.1)
500 |  |  67.6 (3) |  67.7 (3) |  73 (3.1) |  72.9 (3) |  79.6 (2.8) |  |  35.3 (1.7) |  35.7 (1.7) |  39.3 (2) |  38.5 (1.8) |  42.7 (1.9)
1000 |  |  65.8 (3.2) |  71.8 (3.4) |  78.6 (3.6) |  88.7 (4.6) |  90 (3.3) |  |  34.1 (1.8) |  37 (1.7) |  39.6 (1.9) |  45.3 (2.3) |  46.8 (2)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  96.1 (2.9) |  105.4 (3.1) |  102.2 (2) |  105.6 (2.3) |  106.1 (2.5) |  |  58.3 (2.3) |  70.8 (2.3) |  75.6 (1.8) |  84.2 (2.1) |  85.1 (1.7)
500 |  |  128.9 (5.8) |  146.7 (5.5) |  156.8 (5.1) |  172.6 (5.2) |  180 (4.7) |  |  69.8 (3.5) |  80.2 (4) |  90.9 (3.9) |  99 (4.4) |  103.2 (4.1)
1000 |  |  141.9 (6.8) |  164.4 (7.3) |  178.9 (7.4) |  187.1 (6.4) |  207.8 (7) |  |  73.1 (3.9) |  86.1 (4) |  94.6 (4.2) |  99.3 (4.3) |  112 (4.7)
Table 46: Average prediction bias for 100 replications of AR2 CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  21.9 (0.8) |  24 (1) |  26.2 (1.1) |  27.9 (1.3) |  26.9 (1.1) |  |  12.9 (0.6) |  13.9 (0.7) |  15.4 (0.7) |  16.8 (0.8) |  16.7 (0.8)
500 |  |  28.4 (1.3) |  29.7 (1.4) |  34.7 (1.7) |  32.3 (1.5) |  36.1 (1.8) |  |  16.3 (0.8) |  16.6 (0.9) |  18 (0.8) |  17 (0.8) |  19.2 (0.9)
1000 |  |  32.6 (1.5) |  33.1 (1.5) |  35.1 (1.6) |  37.7 (2.1) |  35.8 (1.7) |  |  19 (0.9) |  17.9 (0.8) |  19.4 (1) |  19.5 (1) |  19.4 (0.9)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  47.9 (2) |  48.8 (1.5) |  53.5 (1.6) |  52.7 (1.4) |  52.3 (1.1) |  |  29.8 (1.6) |  31.7 (1.2) |  35.4 (1.4) |  38.9 (1.3) |  38.4 (1.1)
500 |  |  66.1 (2.7) |  67.3 (3.1) |  71.1 (3) |  71.7 (3.1) |  77.8 (3.3) |  |  37.3 (2) |  37.1 (1.9) |  39.7 (2.2) |  38.4 (2) |  42.4 (2.1)
1000 |  |  66.7 (3.3) |  69.5 (3.4) |  75.7 (3.4) |  85.3 (4.4) |  88.5 (3.5) |  |  35.2 (1.8) |  38.2 (1.9) |  41.9 (2.1) |  45.8 (2.6) |  46.2 (2.3)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  95.2 (2.7) |  101 (3) |  95.6 (1.7) |  100.9 (2.1) |  103 (2.6) |  |  61.2 (2.5) |  71.5 (2.6) |  75 (1.9) |  85.5 (1.9) |  85.9 (1.9)
500 |  |  128.9 (5.4) |  148.6 (5.5) |  157.7 (5) |  172.8 (5.2) |  178.2 (4.3) |  |  73.1 (3.7) |  82.9 (4.1) |  94.1 (4.2) |  101.8 (4.8) |  105.2 (4.5)
1000 |  |  142.9 (6.5) |  164.4 (7) |  181 (7.4) |  185.8 (6.6) |  204.6 (6.7) |  |  76.8 (4) |  89.5 (4.6) |  97 (4.5) |  99.9 (4.7) |  112.2 (5.2)
Table 47: Average number of prediction bias of 100 replications for $\alpha$-modified CV 1SE with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  21.6 (0.8) |  23.7 (1) |  26.9 (1.1) |  28.1 (1.3) |  28.4 (1.2) |  |  11.8 (0.5) |  12.9 (0.7) |  14.8 (0.7) |  15.7 (0.8) |  15.6 (0.7)
500 |  |  28 (1.2) |  29.5 (1.4) |  34.2 (1.6) |  32.4 (1.5) |  37 (2.1) |  |  14.9 (0.7) |  15.2 (0.8) |  16.6 (0.8) |  16 (0.8) |  18 (0.9)
1000 |  |  32.1 (1.4) |  31.7 (1.4) |  34.8 (1.6) |  37.4 (2.1) |  35.8 (1.7) |  |  17.1 (0.8) |  16.2 (0.7) |  17.9 (1) |  17.9 (0.9) |  18.3 (0.9)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  48.3 (1.9) |  50.6 (1.6) |  54.7 (1.5) |  54.2 (1.5) |  54.7 (1.3) |  |  28.4 (1.4) |  30.2 (1.1) |  34.1 (1.4) |  37.6 (1.2) |  36.7 (1)
500 |  |  65.2 (2.6) |  67.1 (3) |  71.1 (3) |  72.1 (3.2) |  79.2 (3.2) |  |  34.8 (1.8) |  34.9 (1.8) |  38 (2.1) |  36.8 (1.9) |  41.2 (2.1)
1000 |  |  65 (3.2) |  69.2 (3.2) |  75.8 (3.3) |  85.7 (4.5) |  88.7 (3.5) |  |  33 (1.7) |  36 (1.8) |  39.1 (2) |  43.2 (2.4) |  44.3 (2.2)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  97 (2.4) |  103.6 (2.8) |  98 (1.6) |  104.4 (2.1) |  105.4 (2.4) |  |  59.5 (2.3) |  71.1 (2.2) |  75.1 (1.7) |  82.5 (1.9) |  84.5 (2.2)
500 |  |  128.7 (5.3) |  150.9 (5.4) |  161.4 (4.8) |  179 (5) |  185.8 (4.1) |  |  69.5 (3.5) |  80.5 (4) |  91.8 (4.1) |  99.4 (4.7) |  103.6 (4.4)
1000 |  |  141.3 (6.6) |  166 (7.2) |  182.3 (7.4) |  190.1 (6.7) |  210.6 (6.7) |  |  73.1 (3.8) |  85.7 (4.2) |  94.3 (4.3) |  97.3 (4.6) |  109.3 (5)
Table 48: Average prediction bias for 100 replications of Relaxed Lasso with correlated predictors. Standard errors given in parentheses. | p |  50 |  100 |  200 |  400 |  800 |  |  50 |  100 |  200 |  400 |  800
---|---|---|---|---|---|---|---|---|---|---|---|---
n |  | $p^{*}=5;SNR=1.25;\bar{\sigma}=4.71$ |  | $p^{*}=5;SNR=5;\bar{\sigma}=2.35$
100 |  |  18.3 (0.7) |  19.7 (0.9) |  25.2 (1.2) |  25.3 (1.3) |  29.8 (1.8) |  |  9.1 (0.5) |  10.2 (0.6) |  13.1 (0.8) |  14.7 (0.9) |  15.5 (1)
500 |  |  15.3 (1) |  19.3 (1.4) |  21.3 (1.3) |  20.9 (1.3) |  24.9 (1.7) |  |  8 (0.4) |  8.4 (0.6) |  9.8 (0.7) |  9.3 (0.7) |  10.6 (0.8)
1000 |  |  14.6 (1) |  16.1 (1.2) |  18 (1.5) |  20 (1.6) |  19.5 (1.4) |  |  7.7 (0.5) |  7.6 (0.5) |  8.4 (0.6) |  8 (0.6) |  7.8 (0.5)
n |  | $p^{*}=15;SNR=1.25;\bar{\sigma}=8.3$ |  | $p^{*}=15;SNR=5;\bar{\sigma}=4.15$
100 |  |  40.4 (1.5) |  44.3 (1.6) |  47.9 (1.6) |  50.6 (1.7) |  53.4 (1.9) |  |  24.8 (1.1) |  27.3 (1.1) |  30.6 (1.2) |  33.9 (1.3) |  34.7 (1.1)
500 |  |  49.2 (2.3) |  52.8 (2.4) |  56.5 (2.5) |  57.3 (2.7) |  63.5 (2.6) |  |  24.2 (1.5) |  24.9 (1.6) |  27 (1.8) |  27.4 (1.8) |  31.1 (2)
1000 |  |  44 (2.7) |  47.4 (2.6) |  53.8 (3) |  61.4 (3.7) |  64.9 (3.2) |  |  19.3 (1.2) |  21 (1.3) |  23.1 (1.6) |  27.1 (2.1) |  26.3 (1.9)
n |  | $p^{*}=50;SNR=1.25;\bar{\sigma}=15.07$ |  | $p^{*}=50;SNR=5;\bar{\sigma}=7.53$
100 |  |  81.6 (2) |  89.2 (2.6) |  90.1 (2) |  98 (3.2) |  104.4 (4.1) |  |  52.4 (1.9) |  60.2 (2) |  61.7 (1.6) |  70.1 (2.2) |  72.2 (2.3)
500 |  |  102.7 (4) |  125 (4.4) |  136.1 (4) |  148.7 (4) |  154.6 (3.7) |  |  57.2 (2.8) |  69.7 (3.4) |  79.4 (3.4) |  88.5 (3.8) |  93.2 (3.5)
1000 |  |  105.5 (4.6) |  134.2 (5.6) |  151.1 (5.5) |  160.2 (5.4) |  178 (5.6) |  |  55.2 (2.8) |  69.5 (3.4) |  76.4 (4.1) |  81.1 (4.1) |  90.5 (4.8)

Figures 11, 12, and 13 give the average false positives, false negatives, and
prediction bias for the simulated data provided in the main document’s Figure
(6). Similar to independent predictors, larger sample sizes maintain low false
negatives, but for $n=100$ and $n=500$ with the smaller SNR, average false
negatives increase with $p^{*}.$

Figure 11: Average false positives from 100 replications of simulated data
with $n=100,n=500,n=1000$ and $p=100$ with independent predictors for both SNR
$=1.25$ and SNR $=5$. Thin dotted lines represent the mean $\pm$ one standard
error. Figure 12: Average false negatives from 100 replications of simulated
data with $n=100,n=500,n=1000$ and $p=100$ with independent predictors for
both SNR $=1.25$ and SNR $=5$. Thin dotted lines represent the mean $\pm$ one
standard error. Figure 13: Average prediction bias from 100 replications of
simulated data with $n=100,n=500,n=1000$ and $p=100$ with independent
predictors for both SNR $=1.25$ and SNR $=5$. Thin dotted lines represent the
mean $\pm$ one standard error.

Finally, we evaluate the methods for limited data settings, letting $n=100$,
$p=\\{100,800\\}$, SNR $=\\{1.25,5\\}$ , and varying $p^{*}$. Once again, X
contained correlated columns and more methods are evaluated in Figure 14 and
Tables 49, 50, and 51. Figure 15 and Tables 52 through 54 give results from
the case where $n=1000$. Average false negatives are higher for all methods
when predictors are correlated. Particularly for large $p^{*}$ all methods
struggle to capture the true model, though the Relaxed Lasso seemingly
performs the best. For the $n=1000$ case, Mod 1SE, AR2 1SE, and Relaxed 1SE
all have the smallest average FDRs so it seems that the new methods are most
competitive with the Relaxed Lasso when sample sizes are large.

Figure 14: Average false positives and prediction error plots from 100 replications of simulated data with $n=100$ and $p=800$ for both SNR $=0.5$ and SNR $=2$ with correlated predictors. Thin dotted lines represent the mean $\pm$ one standard error. Table 49: Average false discovery rate for 100 replications of simulated data with correlated predictors and $n=100$. Standard errors given in parentheses. | $p=100$ | $p=800$
---|---|---
| SNR = 1.25 | SNR = 5 | SNR = 1.25 | SNR = 5
|  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$
CV Min |  0.75 (0.01) |  0.48 (0.01) |  0.74 (0.01) |  0.5 (0.01) |  0.86 (0.01) |  0.84 (0.01) |  0.87 (0.01) |  0.76 (0.02)
CV 1SE |  0.53 (0.03) |  0.34 (0.02) |  0.52 (0.02) |  0.4 (0.01) |  0.65 (0.03) |  0.64 (0.04) |  0.7 (0.02) |  0.61 (0.03)
Mod Min |  0.63 (0.03) |  0.4 (0.02) |  0.67 (0.02) |  0.48 (0.01) |  0.7 (0.03) |  0.77 (0.03) |  0.78 (0.02) |  0.7 (0.03)
Mod 1SE |  0.35 (0.03) |  0.28 (0.03) |  0.34 (0.02) |  0.35 (0.02) |  0.45 (0.04) |  0.64 (0.04) |  0.53 (0.03) |  0.49 (0.04)
AR2 Min |  0.67 (0.02) |  0.5 (0.02) |  0.64 (0.02) |  0.49 (0.01) |  0.71 (0.03) |  0.81 (0.02) |  0.73 (0.02) |  0.7 (0.02)
AR2 1SE |  0.38 (0.03) |  0.36 (0.03) |  0.28 (0.02) |  0.34 (0.02) |  0.45 (0.04) |  0.76 (0.03) |  0.44 (0.03) |  0.55 (0.03)
R2 Cut |  0.94 (0) |  0.66 (0) |  0.91 (0) |  0.57 (0.01) |  0.95 (0) |  0.9 (0) |  0.91 (0) |  0.84 (0)
AIC |  0.68 (0.02) |  0.43 (0.02) |  0.7 (0.01) |  0.48 (0.01) |  0.79 (0.02) |  0.79 (0.02) |  0.81 (0.01) |  0.7 (0.02)
BIC |  0.5 (0.03) |  0.33 (0.02) |  0.5 (0.02) |  0.39 (0.01) |  0.66 (0.03) |  0.67 (0.04) |  0.67 (0.02) |  0.59 (0.03)
ERIC |  0.37 (0.03) |  0.18 (0.02) |  0.4 (0.02) |  0.21 (0.02) |  0.98 (0) |  0.94 (0) |  0.98 (0) |  0.92 (0)
GCV |  0.77 (0.01) |  0.5 (0.01) |  0.74 (0.01) |  0.52 (0.01) |  0.98 (0) |  0.94 (0) |  0.98 (0) |  0.92 (0)
Relaxed Min |  0.55 (0.03) |  0.44 (0.02) |  0.39 (0.03) |  0.45 (0.01) |  0.7 (0.03) |  0.82 (0.02) |  0.55 (0.03) |  0.75 (0.02)
Relaxed 1SE |  0.4 (0.03) |  0.32 (0.03) |  0.25 (0.03) |  0.36 (0.02) |  0.5 (0.04) |  0.69 (0.04) |  0.44 (0.03) |  0.61 (0.03)
Table 50: Average prediction bias for 100 replications of simulated data with correlated predictors and $n=100$. Standard errors given in parentheses. | $p=100$ | $p=800$
---|---|---
| SNR = 1.25 | SNR = 5 | SNR = 1.25 | SNR = 5
|  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$
CV Min |  20.35 (0.79) |  63.73 (1.34) |  11.82 (0.45) |  43.31 (1.38) |  25.17 (1.06) |  73.02 (2.39) |  15.93 (0.79) |  56.79 (1.43)
CV 1SE |  25.74 (1.1) |  78.01 (1.83) |  14.04 (0.63) |  50.31 (1.69) |  28.69 (0.95) |  88.89 (2.69) |  15.65 (0.69) |  71.1 (1.47)
Mod Min |  21.14 (0.82) |  73.03 (1.59) |  11.81 (0.5) |  45.3 (1.42) |  27.07 (1.19) |  80.28 (2.8) |  15.1 (0.85) |  60.23 (1.62)
Mod 1SE |  23.74 (0.89) |  80.66 (1.81) |  13.73 (0.64) |  51.53 (1.74) |  26.92 (0.86) |  83.82 (2.13) |  15.29 (0.74) |  68.23 (1.4)
AR2 Min |  21.65 (0.89) |  73.11 (1.75) |  11.92 (0.52) |  45.59 (1.43) |  26.4 (1.06) |  78.8 (2.22) |  14.32 (0.76) |  57.97 (1.27)
AR2 1SE |  23.96 (0.91) |  78 (2.07) |  14.99 (0.73) |  52.87 (1.83) |  27.22 (0.87) |  81.7 (2) |  16.78 (0.86) |  68.97 (1.36)
R2 Cut |  41.31 (1.83) |  105.19 (4.59) |  17.04 (0.74) |  45.12 (1.77) |  34.2 (1.67) |  91.36 (4.27) |  15.15 (0.76) |  49.18 (1.82)
AIC |  20.17 (0.71) |  65.73 (1.22) |  11.87 (0.47) |  44.71 (1.42) |  23.98 (0.78) |  73.47 (1.98) |  14.07 (0.61) |  59.81 (1.15)
BIC |  22.83 (0.84) |  72.56 (1.07) |  12.79 (0.51) |  49.32 (1.49) |  26.43 (0.75) |  81.36 (2.02) |  15.19 (0.65) |  66.64 (1.08)
ERIC |  29.44 (1.56) |  98.14 (3.53) |  15.06 (0.7) |  76.35 (1.76) |  47.01 (2.32) |  122.18 (5.97) |  23.78 (1.18) |  62.67 (3.08)
GCV |  22.53 (0.99) |  67.48 (1.81) |  12.43 (0.49) |  44.43 (1.61) |  47.01 (2.32) |  122.18 (5.97) |  23.78 (1.18) |  62.67 (3.08)
Relaxed Min |  20.9 (0.88) |  66.67 (1.62) |  11.37 (0.57) |  44.59 (1.49) |  29.14 (1.76) |  80.59 (3.22) |  14.85 (1.04) |  58.67 (2.05)
Relaxed 1SE |  25.38 (1.14) |  77.77 (1.83) |  13.27 (0.69) |  50.73 (1.76) |  29.07 (1.21) |  89.21 (3.14) |  14.46 (0.84) |  67.88 (1.8)
Table 51: Average false negatives for 100 replications of simulated data with correlated predictors and $n=100$. Standard errors given in parentheses. | $p=100$ | $p=800$
---|---|---
| SNR = 1.25 | SNR = 5 | SNR = 1.25 | SNR = 5
|  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$ |  $p^{*}=5$ |  $p^{*}=33$
CV Min |  1.33 (0.14) |  21.88 (0.65) |  0.25 (0.05) |  10.86 (0.77) |  2 (0.14) |  28.78 (0.44) |  0.44 (0.08) |  25.19 (0.51)
CV 1SE |  2.1 (0.14) |  27.72 (0.45) |  0.55 (0.07) |  16.5 (0.8) |  2.85 (0.14) |  31.65 (0.18) |  0.69 (0.1) |  29.09 (0.31)
Mod Min |  1.57 (0.15) |  25.05 (0.63) |  0.38 (0.07) |  11.82 (0.83) |  2.39 (0.15) |  29.13 (0.52) |  0.54 (0.09) |  25.76 (0.62)
Mod 1SE |  2.68 (0.15) |  29.94 (0.44) |  0.87 (0.12) |  18.17 (0.94) |  3.27 (0.15) |  31.69 (0.28) |  0.95 (0.13) |  30.01 (0.34)
AR2 Min |  1.5 (0.16) |  20.75 (0.89) |  0.43 (0.08) |  11.31 (0.81) |  2.27 (0.15) |  29.26 (0.4) |  0.71 (0.11) |  26.9 (0.41)
AR2 1SE |  2.64 (0.15) |  27.74 (0.6) |  1.04 (0.13) |  18.9 (0.98) |  3.3 (0.15) |  31.92 (0.17) |  1.17 (0.14) |  30.49 (0.26)
R2 Cut |  0.35 (0.06) |  5.21 (0.27) |  0.07 (0.03) |  6.2 (0.35) |  1.56 (0.13) |  25.88 (0.27) |  0.39 (0.07) |  22.5 (0.34)
AIC |  1.49 (0.14) |  24.55 (0.56) |  0.29 (0.06) |  12.39 (0.79) |  2.27 (0.14) |  30.6 (0.21) |  0.59 (0.09) |  27.3 (0.36)
BIC |  2.17 (0.13) |  28.63 (0.33) |  0.55 (0.07) |  17.23 (0.86) |  3.02 (0.13) |  31.83 (0.15) |  0.72 (0.1) |  29.23 (0.3)
ERIC |  2.95 (0.12) |  31.54 (0.17) |  0.78 (0.09) |  29.26 (0.3) |  1.19 (0.11) |  19.79 (0.29) |  0.26 (0.06) |  16.1 (0.33)
GCV |  1.19 (0.13) |  20.42 (0.72) |  0.24 (0.05) |  10.45 (0.7) |  1.19 (0.11) |  19.79 (0.29) |  0.26 (0.06) |  16.1 (0.33)
Relaxed Min |  1.7 (0.14) |  24.03 (0.62) |  0.44 (0.08) |  13.4 (0.9) |  2.38 (0.14) |  28.73 (0.48) |  0.63 (0.09) |  24.62 (0.61)
Relaxed 1SE |  2.42 (0.14) |  28.39 (0.48) |  0.85 (0.1) |  18.19 (0.88) |  3.09 (0.14) |  31.57 (0.25) |  0.99 (0.13) |  28.83 (0.42)
Figure 15: Average false positives and prediction error plots from 100 replications of simulated data with $n=1000$ and $p=100$ for both SNR $=0.5$ and SNR $=2$ with correlated predictors. Thin dotted lines represent the mean $\pm$ one standard error. Table 52: Average false discovery rate for 100 replications of simulated data with correlated predictors and $n=1000$. Standard errors given in parentheses. | SNR = 1.25 | SNR = 5
---|---|---
| $p^{*}=5$ | $p^{*}=33$ | $p^{*}=5$ | $p^{*}=33$
CV Min | 0.7 (0.01) | 0.48 (0.01) | 0.7 (0.01) | 0.5 (0.01)
CV 1SE | 0.23 (0.02) | 0.24 (0.01) | 0.21 (0.02) | 0.26 (0.01)
Mod Min | 0.58 (0.02) | 0.45 (0.01) | 0.56 (0.02) | 0.45 (0.01)
Mod 1SE | 0.08 (0.02) | 0.18 (0.01) | 0.12 (0.02) | 0.19 (0.01)
AR2 Min | 0.59 (0.02) | 0.47 (0.01) | 0.56 (0.02) | 0.45 (0.01)
AR2 1SE | 0.08 (0.02) | 0.17 (0.01) | 0.09 (0.02) | 0.15 (0.01)
R2 Cut | 0.1 (0.02) | 0.19 (0.01) | 0.14 (0.02) | 0.2 (0.01)
AIC | 0.7 (0.01) | 0.47 (0.01) | 0.69 (0.01) | 0.49 (0.01)
BIC | 0.35 (0.02) | 0.18 (0.01) | 0.35 (0.02) | 0.23 (0.01)
ERIC | 0.26 (0.02) | 0.08 (0.01) | 0.25 (0.02) | 0.13 (0.01)
GCV | 0.71 (0.01) | 0.48 (0.01) | 0.71 (0.01) | 0.5 (0.01)
Relaxed Min | 0.2 (0.03) | 0.24 (0.02) | 0.09 (0.02) | 0.11 (0.01)
Relaxed 1SE | 0.05 (0.01) | 0.11 (0.01) | 0.04 (0.01) | 0.06 (0.01)
Table 53: Average prediction bias for 100 replications of simulated data with correlated predictors and $n=1000$. Standard errors given in parentheses. | SNR = 1.25 | SNR = 5
---|---|---
| $p^{*}=5$ | $p^{*}=33$ | $p^{*}=5$ | $p^{*}=33$
CV Min | 22.47 (1.11) | 91.73 (3.45) | 11.27 (0.52) | 48.34 (1.83)
CV 1SE | 36.11 (1.77) | 113.94 (4.37) | 17.87 (0.87) | 59.71 (2.39)
Mod Min | 21.09 (1.17) | 93.33 (3.54) | 10.24 (0.55) | 47.5 (1.92)
Mod 1SE | 34.13 (1.7) | 115.84 (4.52) | 16.99 (0.91) | 58.69 (2.43)
AR2 Min | 21.11 (1.17) | 93.62 (3.62) | 10.27 (0.56) | 47.53 (1.92)
AR2 1SE | 34.79 (1.74) | 115.86 (4.65) | 18.7 (0.96) | 62.24 (2.68)
R2 Cut | 67.19 (6.77) | 143.47 (11.71) | 29.92 (4.11) | 64.7 (2.06)
AIC | 22.7 (1.11) | 92.76 (3.49) | 11.28 (0.51) | 48.61 (1.84)
BIC | 26.92 (1.25) | 120.22 (4.63) | 13.21 (0.58) | 59.85 (2.4)
ERIC | 30.27 (1.51) | 155.85 (5.53) | 14.84 (0.73) | 74.46 (3.78)
GCV | 23.18 (1.26) | 92.49 (3.44) | 11.37 (0.51) | 48.7 (1.85)
Relaxed Min | 17.52 (1.34) | 90.79 (3.71) | 7.61 (0.49) | 42.96 (2.1)
Relaxed 1SE | 34.35 (1.97) | 115.03 (4.66) | 16.61 (0.84) | 55.85 (2.53)
Table 54: Average false negatives for 100 replications of simulated data with correlated predictors and $n=1000$. Standard errors given in parentheses. | SNR = 1.25 | SNR = 5
---|---|---
| $p^{*}=5$ | $p^{*}=33$ | $p^{*}=5$ | $p^{*}=33$
CV Min | 0.04 (0.02) | 1.38 (0.27) | 0 (0) | 0.12 (0.05)
CV 1SE | 0.24 (0.05) | 4.56 (0.6) | 0.02 (0.01) | 0.43 (0.14)
Mod Min | 0.06 (0.03) | 1.54 (0.29) | 0 (0) | 0.12 (0.05)
Mod 1SE | 0.39 (0.07) | 5.56 (0.71) | 0.04 (0.02) | 0.48 (0.14)
AR2 Min | 0.06 (0.03) | 1.4 (0.27) | 0 (0) | 0.12 (0.05)
AR2 1SE | 0.4 (0.07) | 5.58 (0.73) | 0.06 (0.03) | 0.72 (0.2)
R2 Cut | 0.38 (0.07) | 5.27 (0.71) | 0.04 (0.02) | 0.46 (0.14)
AIC | 0.03 (0.02) | 1.68 (0.3) | 0 (0) | 0.11 (0.04)
BIC | 0.16 (0.04) | 6.52 (0.72) | 0 (0) | 0.52 (0.13)
ERIC | 0.22 (0.05) | 12.45 (0.95) | 0 (0) | 1.83 (0.44)
GCV | 0.03 (0.02) | 1.58 (0.29) | 0 (0) | 0.11 (0.04)
Relaxed Min | 0.09 (0.03) | 3.31 (0.39) | 0 (0) | 0.66 (0.11)
Relaxed 1SE | 0.37 (0.06) | 7.12 (0.66) | 0.03 (0.02) | 1.25 (0.2)

### B.3. Simulation Study for Non-Convex Penalties

This set of simulations examines the effect of the modification on the non-
convex penalties, SCAD [15] and MC+ [16]. Both penalties are most commonly
expressed in terms of its derivative which, for SCAD at a single point,
$\beta_{j}$, is:

$\displaystyle
P^{\prime}_{\lambda}(\beta_{j})=\lambda_{1}\Big{[}I(\beta_{j}\leq\lambda_{1})+\frac{(\lambda_{2}\beta_{j}-\beta_{j})_{+}}{(\lambda_{2}-1)\beta_{j}}I(\beta_{j}>\lambda_{1})\Big{]}\
.\ $ (75)

The derivative of the MC+ penalty for a single coefficient, $\beta_{j}$, is:

$\displaystyle
P_{\lambda}^{\prime}(\beta_{j})=\begin{cases}\text{sign}(\beta_{j})\Big{(}\lambda_{1}-\frac{|\beta_{j}|}{\lambda_{2}}\Big{)}&\text{if}\hskip
5.0pt\lvert\beta_{j}\rvert\leq\lambda_{1}\lambda_{2}\\\
0&\text{otherwise}\end{cases}\ .\ $ (76)

We investigated SCAD with $\lambda_{2}=3.7$ and MC+ with $\lambda_{2}=3$, both
with and without the $\alpha$-Modification using 10-fold CV with a 1SE rule.
We generated data from the following model:

$\textbf{y}_{n\times 1}=\beta^{*}_{0}+\textbf{X}_{n\times
p}\boldsymbol{\beta}_{p\times 1}^{*}+\boldsymbol{\epsilon}_{n\times 1}$ (77)

where X was drawn from a Standard Normal distribution, $\beta^{*}_{0}$ and
$\boldsymbol{\beta}^{*}$ were drawn from the same distribution as the Lasso
simulations above, and SNR = {1.25, 5}. We include an intercept here because
the R package ncvreg does not have include a no-intercept option. The
optimization to find $\hat{\alpha}_{\lambda}$ is slightly more complicated
with an intercept term. We now optimize the likelihood function over both
$\alpha$ and $\beta_{0}$, the intercept. Figures 16 and 17 include the average
False Positives, False Negatives, and Prediction Bias across increasing values
of $p^{*}$ when $n=100$ and $p=100$ over 500 replications. Note the similarity
between the two approaches.

Figure 16: Average false positives, prediction error, and false negatives from
500 replications of simulated data with $n=100$ and $p=100$ for SNR = 5. Thin
dotted lines represent the mean $\pm$ one standard error.

Although average false positives for the non-convex penalties are lower than
or equal to that of the $\alpha$-Modified Lasso, their average false negatives
tend to be higher, particularly as $p^{*}$ increases. Prediction bias is
slightly worse for $\alpha$-Modified SCAD and MC+ than unmodified, but similar
to the $\alpha$-Modified Lasso for small $p^{*}$ and similar to unmodified for
larger $p^{*}$. This is a fairly limited simulation study, capturing only one
$n$ and $p$. Furthermore, for non-convex penalties, we no longer have a
guarantee that $\hat{\alpha}_{\lambda}$ will always be greater than 1, so the
bias-reduction aim of the modification is in question. Allowing the
$\alpha$-Modification to compete with and enhance non-convex penalties may be
the subject of future research.

Figure 17: Average false positives, prediction error, and false negatives from
500 replications of simulated data with $n=100$ and $p=100$ for SNR = 5. Thin
dotted lines represent the mean $\pm$ one standard error.

### B.4. Simulation Study for Non-Gaussian Errors

A final simulation study replicates the independent predictor setting found in
the main document but elements of $\boldsymbol{\epsilon}$ are now
independently drawn from a Laplace distribution with mean zero and scale
parameter selected to maintain a consistent signal-to-noise ratio for each new
data set. Tables 18 through 21 give the average Hamming Distance, False
Positives, False Negatives, and Prediction Bias from these simulations.

Figure 18: Average Hamming Distance from 100 replications of simulated data
with $p=100$ for both SNR $=1.25$ and SNR $=5$ independent Laplacian errors.
Thin dotted lines represent the mean $\pm$ one standard error. Figure 19:
Average false positives from 100 replications of simulated data with $p=100$
for both SNR $=1.25$ and SNR $=5$ independent Laplacian errors. Thin dotted
lines represent the mean $\pm$ one standard error. Figure 20: Average false
negatives from 100 replications of simulated data with $p=100$ for both SNR
$=1.25$ and SNR $=5$ independent Laplacian errors. Thin dotted lines represent
the mean $\pm$ one standard error. Figure 21: Average prediction bias from 100
replications of simulated data with $p=100$ for both SNR $=1.25$ and SNR $=5$
independent Laplacian errors. Thin dotted lines represent the mean $\pm$ one
standard error.

## C. EMG Application Approximations and Results

The model given in the main document does not involve raw data, so some
manipulations are required in order to perform estimation. The response,
velocity ($y_{i}$), must be estimated from hand position, $z_{i}$ and
covariate functions in the model must be created from discrete observations.
These tasks are performed exactly as described in [1].

Furthermore, the model also requires approximations for $\gamma_{j}$. We
represent all $\gamma_{j}$ using:
$\gamma_{j}(t,z)\approx\sum_{l=1}^{L}\sum_{m=1}^{M}\omega_{l}(t)\tau_{m}(z)\beta_{jlm}$
where $\omega_{l}(t)$ and $\tau_{m}(z)$ are the same univariate basis
functions for all $\gamma_{j}$, and the $\beta_{jlm}$’s are basis coefficients
that must be estimated. In matrix form, we have
$\gamma_{j}(t,z)\approx\boldsymbol{\omega}^{T}(t)\textbf{B}_{j}\boldsymbol{\tau}(z)$
where
$\boldsymbol{\omega}^{T}(t)=\big{(}\omega_{1}(t),...,\omega_{L}(t)\big{)}$,
$\boldsymbol{\tau}^{T}(z)=\big{(}\tau_{1}(z),...,\tau_{M}(z)\big{)}$, and
$\textbf{B}_{j}=\big{(}\beta_{jlm}\big{)}$ is the $L\times M$ coefficient
matrix.

The integral in the model is approximated:

$\int_{-\delta}^{0}X_{ij}(t)\gamma_{j}(t,z_{i})dt\approx\bigg{(}\sum_{k=-\delta}^{0}x_{(i+k)j}\boldsymbol{\omega(k)^{T}}\bigg{)}\textbf{B}_{j}\boldsymbol{\tau}(z_{i})=\textbf{X}_{ij\omega}^{T}\textbf{B}_{j}\boldsymbol{\tau}(z_{i}).$
(78)

As noted in the main document, we use the penalized least squares approach to
functional estimation proposed in [41] and adapted in [1].

Tables 55 through 58 show the results from multiple stages of adaptive
weighting, the SAFE procedure. After only one round, the differences in tuning
parameter selection approaches disappear, giving the same results for APE,
AR2, and Mod CV. The differences reappear again after three stages, but only
for AR2 CV, which still overselects by a single coefficient for FR2, even
after a fourth stage.

Table 55: Variable selection results for EMG finger movements after one stage of adaptive weighting. FP indicates the total number of false positives in the model and Size is the total number of EMG signals contained in the model. |  | FC1 | FC2 | FC3 | FR1 | FR2 | FR3 | Mean
---|---|---|---|---|---|---|---|---
APE | FP | 0 | 0 | 0 | 0 | 1 | 1 | 0.33
Size | 2 | 2 | 2 | 2 | 4 | 3 | 2.5
AR2 | FP | 0 | 0 | 1 | 0 | 1 | 0 | 0.33
Size | 2 | 2 | 3 | 2 | 4 | 2 | 2.5
Mod | FP | 0 | 0 | 0 | 0 | 1 | 1 | 0.33
Size | 2 | 2 | 2 | 2 | 4 | 3 | 2.5
Table 56: Variable selection results for EMG finger movements after two stages of adaptive weighting. FP indicates the total number of false positives in the model and Size is the total number of EMG signals contained in the model.. |  | FC1 | FC2 | FC3 | FR1 | FR2 | FR3 | Mean
---|---|---|---|---|---|---|---|---
APE | FP | 0 | 0 | 0 | 0 | 1 | 0 | 0.17
Size | 2 | 2 | 2 | 2 | 4 | 2 | 2.33
AR2 | FP | 0 | 0 | 0 | 0 | 1 | 0 | 0.17
Size | 2 | 2 | 2 | 2 | 4 | 2 | 2.33
Mod | FP | 0 | 0 | 0 | 0 | 1 | 0 | 0.17
Size | 2 | 2 | 2 | 2 | 4 | 2 | 2.33
Table 57: Variable selection results for EMG finger movements after three stages of adaptive weighting. FP indicates the total number of false positives in the model and Size is the total number of EMG signals contained in the model. |  | FC1 | FC2 | FC3 | FR1 | FR2 | FR3 | Mean
---|---|---|---|---|---|---|---|---
APE | FP | 0 | 0 | 0 | 0 | 0 | 0 | 0
Size | 2 | 2 | 2 | 2 | 3 | 2 | 2.17
AR2 | FP | 0 | 0 | 0 | 0 | 1 | 0 | 0.17
Size | 2 | 2 | 2 | 2 | 4 | 2 | 2.33
Mod | FP | 0 | 0 | 0 | 0 | 0 | 0 | 0
Size | 2 | 2 | 2 | 2 | 3 | 2 | 2.17
Table 58: Variable selection results for EMG finger movements after the four stages of adaptive weighting recommended by [1]. FP indicates the total number of false positives in the model and Size is the total number of EMG signals contained in the model. |  | FC1 | FC2 | FC3 | FR1 | FR2 | FR3 | Mean
---|---|---|---|---|---|---|---|---
APE | FP | 0 | 0 | 0 | 0 | 0 | 0 | 0
Size | 2 | 2 | 2 | 2 | 3 | 2 | 2.17
AR2 | FP | 0 | 0 | 0 | 0 | 1 | 0 | 0.17
Size | 2 | 2 | 2 | 2 | 4 | 2 | 2.33
Mod | FP | 0 | 0 | 0 | 0 | 0 | 0 | 0
Size | 2 | 2 | 2 | 2 | 3 | 2 | 2.17

## References

  * [1] J. Stallrich, M. N. Islam, A.-M. Staicu, D. Crouch, L. Pan, H. Huang, Optimal EMG placement for a robotic prosthesis controller with sequential, adaptive functional estimation (SAFE), The Annals of Applied Statistics 14 (3) (2020) 1164 – 1181.  doi:10.1214/20-AOAS1324.
  * [2] R. Tibshirani, Regression shrinkage and selection via the lasso, Journal of the Royal Statistical Society. Series B (Methodological) 58 (1) (1996) 267–288.
  * [3] H. Akaike, A new look at the statistical model identification, IEEE Transactions on Automatic Control 19 (6) (1974) 716–723.
  * [4] G. Schwarz, Estimating the dimension of a model, The Annals of Statistics 6 (2) (1978) 461–464.
  * [5] Y. Feng, Y. Yu, Consistent cross-validation for tuning parameter selection in high-dimensional variable selection, arXiv preprint arXiv:1308.5390 1 (2.1) (2013).
  * [6] T. Hastie, J. Friedman, R. Tisbshirani, The Elements of statistical learning: data mining, inference, and prediction, Springer, 2017.
  * [7] R. Lockhart, J. Taylor, R. J. Tibshirani, R. Tibshirani, A significance test for the lasso, The Annals of Statistics 42 (2) (2014) 413–468.  doi:10.1214/13-AOS1175.
  * [8] C.-H. Zhang, S. S. Zhang, Confidence intervals for low dimensional parameters in high dimensional linear models, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 76 (1) (2014) 217–242.
  * [9] H. Zou, The adaptive lasso and its oracle properties, Journal of the American Statistical Association 101 (476) (2006) 1418–1429.
  * [10] N. Meinshausen, Relaxed lasso, Computational Statistics & Data Analysis 52 (1) (2007) 374 – 393.
  * [11] L. Breiman, Better subset regression using the nonnegative garrote, Technometrics 37 (4) (1995) 373–384.
  * [12] I. E. Frank, J. H. Friedman, A statistical view of some chemometrics regression tools, Technometrics 35 (2) (1993) 109–135.
  * [13] K. Knight, W. Fu, Asymptotics for lasso-type estimators, The Annals of Statistics 28 (5) (2000) 1356–1378.
  * [14] J. Huang, J. L. Horowitz, S. Ma, Asymptotic properties of bridge estimators in sparse high-dimensional regression models, The Annals of Statistics 36 (2) (2008) 587 – 613.  doi:10.1214/009053607000000875.
  * [15] J. Fan, R. Li, Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (456) (2001) 1348–1360.  arXiv:https://doi.org/10.1198/016214501753382273, doi:10.1198/016214501753382273.
  * [16] C.-H. Zhang, Nearly unbiased variable selection under minimax concave penalty, The Annals of Statistics 38 (2) (2010) 894 – 942.  doi:10.1214/09-AOS729.
  * [17] Y. Wu, Can’t ridge regression perform variable selection?, Technometrics 63 (2) (2021) 263–271.  arXiv:https://doi.org/10.1080/00401706.2020.1791254, doi:10.1080/00401706.2020.1791254.
  * [18] M. Yuan, Y. Lin, Model selection and estimation in regression with grouped variables, Journal of the Royal Statistical Society Series B 68 (2006) 49–67.
  * [19] T. Hastie, R. Tibshirani, Generalized Additive Models, Statistical Science 1 (3) (1986) 297 – 310.  doi:10.1214/ss/1177013604.
  * [20] J. O. Ramsay, B. W. Silverman, Functional Data Analysis, Springer, 2005.
  * [21] Y. Nardi, A. Rinaldo, On the asymptotic properties of the group lasso estimator for linear models, Electronic Journal of Statistics 2 (none) (2008) 605 – 633.  doi:10.1214/08-EJS200.
  * [22] C. M. Hurvich, C.-L. Tsai, Regression and time series model selection in small samples, Biometrika 76 (2) (1989) 297–307.  arXiv:https://academic.oup.com/biomet/article-pdf/76/2/297/737009/76-2-297.pdf, doi:10.1093/biomet/76.2.297.
  * [23] R. Nishii, Asymptotic Properties of Criteria for Selection of Variables in Multiple Regression, The Annals of Statistics 12 (2) (1984) 758 – 765.  doi:10.1214/aos/1176346522.
  * [24] P. Buhlmann, S. van de Geer, Statistics for high-dimensional data, Springer Series in Statistics, Springer, Heidelberg, 2011, methods, theory and applications.
  * [25] F. K. C. Hui, D. I. Warton, S. D. Foster, Tuning parameter selection for the adaptive lasso using ERIC, Journal of the American Statistical Association 110 (509) (2015) 262–269.
  * [26] S. Geisser, The predictive sample reuse method with applications, Journal of the American Statistical Association 70 (350) (1975) 320–328.
  * [27] D. M. Allen, The relationship between variable selection and data agumentation and a method for prediction, Technometrics 16 (1) (1974) 125–127.
  * [28] M. Stone, Cross-validatory choice and assessment of statistical predictions, Journal of the Royal Statistical Society. Series B (Methodological) 36 (2) (1974) 111–147.
  * [29] D. Krstajic, L. Buturovic, D. Leahy, S. Thomas, Cross-validation pitfalls when selecting and assessing regression and classification models, Journal of cheminformatics 6 (2014) 10.
  * [30] P. Craven, G. Wahba, Smoothing noisy data with spline functions, Numerische Mathematik 31 (4) (1978) 377–403.  doi:10.1007/BF01404567.
  * [31] G. H. Golub, M. Heath, G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics 21 (2) (1979) 215–223.
  * [32] D. Homrighausen, D. J. McDonald, A study on tuning parameter selection for the high-dimensional lasso, Journal of Statistical Computation and Simulation 88 (15) (2018) 2865–2892.
  * [33] S. van de Geer, P. Bühlmann, Y. Ritov, R. Dezeure, On asymptotically optimal confidence regions and tests for high-dimensional models, The Annals of Statistics 42 (3) (Jun 2014).  doi:10.1214/14-aos1221.
  * [34] A. Javanmard, A. Montanari, Confidence intervals and hypothesis testing for high-dimensional statistical models, in: C. Burges, L. Bottou, M. Welling, Z. Ghahramani, K. Weinberger (Eds.), Advances in Neural Information Processing Systems, Vol. 26, Curran Associates, Inc., 2013.
  * [35] J. Taylor, R. J. Tibshirani, Statistical learning and selective inference, Proceedings of the National Academy of Sciences 112 (25) (2015) 7629–7634.  arXiv:https://www.pnas.org/content/112/25/7629.full.pdf, doi:10.1073/pnas.1507583112.
  * [36] J. D. Lee, D. L. Sun, Y. Sun, J. E. Taylor, Exact post-selection inference, with application to the lasso, The Annals of Statistics 44 (3) (2016) 907 – 927\.  doi:10.1214/15-AOS1371.
  * [37] X. Shi, B. Liang, Q. Zhang, Post-selection inference of generalized linear models based on the lasso and the elastic net, Communications in Statistics - Theory and Methods 0 (0) (2020) 1–18.  arXiv:https://doi.org/10.1080/03610926.2020.1821892, doi:10.1080/03610926.2020.1821892.
  * [38] B. Efron, T. Hastie, I. Johnstone, R. Tibshirani, Least angle regression, The Annals of Statistics 32 (2) (2004) 407–499.
  * [39] R. J. Tibshirani, J. E. Taylor, R. A. Lockhart, R. Tibshirani, Exact post-selection inference for sequential regression procedures, Journal of the American Statistical Association 111 (2014) 600 – 620.
  * [40] H. Zou, T. Hastie, Regularization and variable selection via the elastic net, Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67 (2) (2005) 301–320.
  * [41] J. Gertheiss, A. Maity, A.-M. Staicu, Variable selection in generalized functional linear models, Stat 2 (1) (2013) 86–101.
  * [42] D. Chetverikov, Z. Liao, V. Chernozhukov, On cross-validated Lasso in high dimensions, The Annals of Statistics 49 (3) (2021) 1300 – 1317.  doi:10.1214/20-AOS2000.
  * [43] K. Knight, W. Fu, Asymptotics for lasso-type estimators, The Annals of Statistics 28 (5) (2000) 1356–1378.