# A consistent estimator of the proportion of nonzero Normal means under
certain strong covariance dependence

Xiongzhi Chen111Corresponding author. Center for Statistics and Machine
Learning and Lewis-Sigler Institute, Princeton University, Princeton, NJ
08544, USA; E-mail<EMAIL_ADDRESS>and Rebecca W. Doerge222
Department of Statistics and Department of Agronomy, Purdue University, West
Lafayette, IN 47907, USA; E-mail<EMAIL_ADDRESS>
###### Abstract

Motivated by multiple testing related to marginal regression analysis for
dependent high-dimensional data, we consider multiple testing which components
of a Normal random vector have zero means, where the proportion of components
that have nonzero means plays an important role. We construct a consistent
estimator of the proportion when components of the Normal random vector can be
strongly dependent. Our estimator is perhaps the first among estimators of
this proportion that has theoretically ensured consistency under strong
dependence. It is accurate and stable as demonstrated by extensive simulation
studies. Its utility is illustrated by an application to multiple testing
related to an association study based on brain imaging data.
Keywords: Concave partially penalized least squares; Fourier transform;
multiple hypothesis testing; proportion of nonzero Normal means; proportion of
true null hypotheses.
MSC 2010 subject classifications: Primary 62H15, 62F12; Secondary 62H25.

###### Abstract

We provide the proofs of Theorem 1, Theorem 2, and Theorem 3, and additional
simulation results for the new estimator under different minimal signal
strengths of the nonzero Normal means.

##  1 Introduction

Estimating the proportion $\pi_{0}$ of true null hypotheses is of considerable
importance due to the routine implementation of false discovery rate (FDR)
control in various scientific endeavors, such as genomics, neuroscience,
economics and finance. The proportion $\pi_{0}$ is the key ingredient of many
adaptive FDR procedures (Storey et al., 2004; Benjamini et al., 2006;
Blanchard and Roquain, 2009), such that a less conservative estimator of
$\pi_{0}$ tends to make the corresponding FDR procedures more powerful
(Benjamini and Yekutieli, 2001). It is also the key component of the two-
groups models and local FDR (Efron et al., 2001; Ploner et al., 2006) and the
positive FDR and q-value (Storey, 2003), such that an accurate estimate of
$\pi_{0}$ is needed to determine the two-groups models and that a more
accurate estimate of $\pi_{0}$ leads to more accurate estimates of the local
FDR, FDR or q-value of the corresponding FDR procedures.

Equally important is the dual quantity, $\pi=1-\pi_{0}$, the proportion of
false nulls, and an accurate estimate of $\pi$ induces an accurate estimate of
$\pi_{0}$. Most existing estimators of $\pi_{0}$ or $\pi$, including those in
Storey (2002); Langaas et al. (2005); Benjamini et al. (2006); Jin (2008),
were developed under the assumption that the p-values or test statistics are
weakly dependent. However, p-values and test statistics derived from high-
dimensional data can be strongly dependent (see examples in Klebanov and
Yakovlev (2007); Leek and Storey (2008)), and none of these estimators have
been shown to be consistent under strong dependence.

Motivated by multiple testing for marginal regression analysis for dependent
high-dimensional data, we consider estimating the proportion of components of
a Normal random vector that have nonzero means. More specifically, suppose the
vector $\mathbf{Z}$ is Normally distributed with mean vector
$\boldsymbol{\mu}=\left(\mu_{1},...,\mu_{m}\right)^{T}$ and a known
correlation matrix $\mathbf{\Sigma}$ representing certain types of strong
dependence. Consider simultaneously testing the null hypothesis
$H_{i}:\mu_{i}=0$ for $1\leq i\leq m$. We will construct a consistent
estimator of the proportion of nonzero Normal means,

$\pi=m^{-1}\left|\left\\{1\leq i\leq m:\mu_{i}\neq 0\right\\}\right|$ (1)

when the correlation matrix $\mathbf{\Sigma}$ encodes strong covariance
dependence and has a few dominant eigenvalues.

The new estimator provides a partial but positive answer to the quest
suggested by Fan, Han and Gu (2012) on consistently estimating the proportion
of nonzero Normal means under arbitrary covariance dependence. It is perhaps
the first among estimators of this proportion that has theoretically ensured
consistency under strong dependence. Two by-products of developing the new
estimator are an extension to more general settings of the Fourier transform
method to estimate $\pi$ in Jin (2008) and an alternative method to estimate
the (principal) factors in order to adjust for dependence in multiple testing;
see details in Section 4.3 and Section 4.2.

The rest of the article is organized as follows. The motivation for the setup
of the problem is given in Section 2, and an application of the new estimator
to multiple testing in an association study based on brain imaging data is
provided in Section 3. We develop the consistent estimator of $\pi$ in Section
4 and present simulation studies on it in Section 5. We end the article with a
discussion in Section 6. This article has supplementary material.

##  2 Motivation of study

The setup for multiple testing which components of a Normal random vector have
zero means when their correlation matrix is known is motivated by multiple
testing for marginal regression analysis for dependent high-dimensional data;
see Owen (2005), Fan, Han and Gu (2012) and Azriel and Schwartzman (2015) for
examples of such analyses in gene expression studies, genome wide association
studies and brain imaging analysis, respectively. In these studies, the aim is
to identify covariates that are associated with the response variable when
there are many candidate covariates while maintaining the FDR at a target
level. To this end, the response variable is regressed on each covariate to
produce an estimated regression coefficient. Then multiple testing is
conducted to assess which regression coefficients are nonzero statistically,
and conclusions are drawn on which covariates are associated with the response
variable.

To formally state the above procedure, let $y$ be the response variable and
$\left\\{x_{i}\right\\}_{i=1}^{m}$ be the covariates. The $i$th null
hypothesis $\tilde{H}_{i}$ is “no association between $y$ and $x_{i}$”, and
the multiple testing problem is to simultaneously assess the null hypotheses
$\\{\tilde{H}_{i}\\}_{i=1}^{m}$. Let $\left\\{y_{j}\right\\}_{j=1}^{n}$ be a
sample of $y$ and $\left\\{x_{ij}\right\\}_{j=1}^{n}$ be a sample of $x_{i}$
for each $i$, where $m\gg n$ usually holds. For each $i$, the marginal
regression model

$y_{j}=\alpha_{i0}+\alpha_{i1}x_{ij}+\varepsilon_{ij},\text{\quad}j=1,\ldots,n,$
(2)

is used, where $\varepsilon_{ij}$’s are i.i.d. with mean $0$ and standard
deviation $\tau$. This converts testing $\tilde{H}_{i}$ to testing if
$\alpha_{i1}=0$.

Let $\hat{\alpha}_{i1}$ be the least squares estimate of $\alpha_{i1}$,
$s_{k}$ be the sample standard deviation of $x_{k}$, and

$Z_{i}=\frac{\hat{\alpha}_{i1}}{\tau/\left(\sqrt{n}s_{i}\right)}.$ (3)

Then, when $n$ is not very small, each $Z_{i}$ will be approximately Normally
distributed with mean $\mu_{i}=\sqrt{n}\alpha_{i1}s_{i}/\tau$ and standard
deviation $1$. When $\varepsilon_{ij}$’s are Normally distributed,
$\mathbf{Z}=\left(Z_{1},\ldots,Z_{m}\right)^{T}$, conditioning on all
$x_{ij}$’s, is Normally distributed with mean vector

$\boldsymbol{\mu}=\left(\mu_{1},\ldots,\mu_{m}\right)^{T}\text{\quad with
\quad}\mu_{i}=\sqrt{n}\alpha_{i1}s_{i}/\tau$ (4)

and covariance matrix

$\boldsymbol{\Sigma}=\left(r_{kl}\right)\text{, where }r_{kl}\text{ is the
sample correlation between }x_{k}\text{ and }x_{l}$ (5)

Note that $\boldsymbol{\Sigma}$ is a correlation matrix since $Z_{i}$’s have
standard deviation $1$.

Now testing if $\alpha_{i1}=0$ is equivalent to testing $H_{i}:\mu_{i}=0$
using $\bf{Z}$, i.e., assessing if an $x_{i}$ is associated with $y$ is
converted to testing $H_{i}:\mu_{i}=0$. In other words, we are testing which
components of the Normal random vector $\bf{Z}$ have zero means. To control
the FDR or estimate the local FDR or q-value when simultaneously testing
$H_{i}:\mu_{i}=0$ for $1\leq i\leq m$, the proportion of nonzero Normal means
$\pi$ defined in (1), i.e., $\pi=m^{-1}\left|\left\\{1\leq i\leq m:\mu_{i}\neq
0\right\\}\right|$ needs to be estimated accurately.

We would like to emphasize that the population covariance matrix of
$\mathbf{Z}$ is the correlation matrix $\boldsymbol{\Sigma}$ and that
$\boldsymbol{\Sigma}$ is known based on the sample $x_{ij}$ for $i=1,\ldots,m$
and $j=1,\ldots,n$ when $\tau$ is known. In practice, $\tau^{2}$ can be
estimated by

$\hat{\tau}^{2}=\frac{1}{m\left(n-2\right)}\sum_{i=1}^{m}\sum_{j=1}^{n}\left(\tilde{y}_{j}-\hat{\alpha}_{i1}\tilde{x}_{ij}\right)^{2}$

when $n>2$, where $\tilde{x}_{ij}=x_{ij}-n^{-1}\sum_{j=1}^{n}x_{ij}$ and
$\tilde{y}_{j}=y_{j}-n^{-1}\sum_{j=1}^{n}y_{j}$. Since $m\left(n-2\right)$ is
usually large, the variability of $\hat{\tau}^{2}$ can be ignored so that
$\hat{\tau}^{2}$ can be taken as $\tau^{2}$. If we set
$\tilde{\mathbf{Z}}=\left(\tilde{Z}_{1},\ldots,\tilde{Z}_{m}\right)^{T}$ with
$\tilde{Z}_{i}=\frac{\hat{\alpha}_{i1}}{\hat{\tau}/\left(\sqrt{n}s_{i}\right)}$,
then $\tilde{\mathbf{Z}}$ can be regarded as having the same distribution as
$\bf{Z}$. Correspondingly, we are back to testing which components of the
Normal random vector $\tilde{\mathbf{Z}}$ have nonzero means with a known
correlation matrix $\boldsymbol{\Sigma}$.

##  3 An application: multiple testing in association study

As an application, we apply the new estimator to estimate the proportion of
nonzero Normal means after marginal regression analysis has been applied to a
high-dimensional data set. The data set was obtained from brain imaging in a
study of the cortical thickness of adults who had a diagnosis of attention
deficit/hyperactivity disorder (ADHD) as children (Proal et al., 2011). The
data set contains cortical thickness measurements for around $80,000$ cortical
voxels and behavioral measurements for each of the $n=139$ individuals. Here
we would like to identify voxels that may affect the overall behavior of an
individual by assessing the association between the cortical thickness of each
voxel to a global assessment of behavior. Namely, the response variable is the
global assessment of behavior and the covariates are the voxel-wise cortical
thicknesses.

Figure 1: Estimating the proportion of Normal test statistics with nonzero
means when assessing the association between the global assessment of behavior
and cortical thickness of each voxel. Panel (a) is the empirical density of
the test statistics, approximated by the blue curve obtained using the method
in Azriel and Schwartzman (2015). Panel (b) is the heat map of the correlation
matrix for the test statistics. It shows that the dependence among the test
statistics is strong but approximately structured. Panel (c) displays the
histogram of the correlations between the test statistics. It shows that a
large proportion of the test statistics are strongly correlated. Panel (d)
shows the largest $20$ eigenvalues of the correlation matrix, for which the
largest eigenvalue dominates.

The marginal regression analysis described in Section 2 is applied to the data
set. To ensure the Normality assumption in each marginal regression model is
not seriously violated, we apply the Shapiro test to assess the Normality of
the residuals for each marginal regression and select $Z_{i}$’s for which the
Shapiro test has p-value $>0.05$. Among these $Z_{i}$’s, we further select
those whose absolute values are $\geq 0.45$ and whose correlation matrix
$\boldsymbol{\Sigma}$ is approximately structured; see panels (b), (c) and (d)
of Figure 1. This gives $m=1000$ test statistics $Z_{i}$, whose histogram is
shown in panel (a) of Figure 1. Coincidentally, the minimal absolute value of
these $Z_{i}$’s is $1.548647$. However, the correlations between a large
proportion of $Z_{i}$’s are fairly strong; see panel (b) of Figure 1.

Our new estimator $\tilde{\pi}$ of the proportion $\pi$ of nonzero Normal
means is $0.0189$ for these $Z_{i}$’s, i.e., the proportion $\pi_{0}$ of zero
Normal means is $1-0.0189=0.9810$. Thus, only a few voxels are correlated with
the global assessment of behavior by their cortical thicknesses. This is
consistent with the finding in Azriel and Schwartzman (2015) that in this data
set correlations may cause the impression of many true associations. However,
the estimators in Benjamini et al. (2006), Wang et al. (2011) and Langaas et
al. (2005) estimate $\pi_{0}$ to be $0.5101$, $0$ and $0.00002$ respectively,
which are all misleading.

On the other hand, $\pi$ being estimated to be $0.0189$ by our estimator
suggests that almost all $Z_{i}$’s under consideration have mean $0$.
Consequently, the empirical density of these $Z_{i}$’s may be approximated by
the random function developed in Azriel and Schwartzman (2015). We applied
their random function with $13$ latent variables, which seems to give the best
approximation among choices of different numbers of latent variables; see the
blue curve in panel (a) of Figure 1. It can be seen that the approximation is
good. This is consistent with our estimate of $\pi$ being $0.0189$. However,
the use of $13$ latent variables by the random function in the approximation
reflects the fact that there are a few voxels that are associated with the
global assessment of behavior and that the correlations among the test
statistics $Z_{i}$ cannot be fully accounted for by few latent variables or
principal factors; see panel (b) of Figure 1.

##  4 Consistently estimating the proportion

Our strategy to develop the consistent estimator of $\pi$ is explained as
follows. First, we utilize the _principal factor approximation_ (PFA)
developed in Fan, Han and Gu (2012) to decompose $\mathbf{Z}$ into two
independent Normal random vectors, such that the major vector contributes the
major part of the covariance dependence between the components of $\mathbf{Z}$
and the minor vector consists of “weakly dependent” (to be specified in
Section 4.1) random variables. We then estimate the major vector. To do this,
we establish two properties of a type of _concave partially penalized least
squares_ (CPPLS) estimate when the involved Normal random errors have
heterogenous variances and are weakly dependent and when there are incidental
parameters. Finally, we extend the _Fourier transform method_ (FTM) to
estimate $\pi$ in Jin (2008) to the case of weakly dependent Normal random
errors with heterogeneous variances, and apply the extended FTM to components
of the mean-shifted, estimated minor vector to estimate $\pi$ consistently.

Before presenting the theory, we introduce some conventions and notations. We
take the convention that a vector is always a column vector, and use
$\mathbf{p}\sim\mathsf{N}_{m}\left(\mathbf{u},\mathbf{S}\right)$ to denote
that $\mathbf{p}$ is an $m$-dimensional Normally distributed random vector
with mean vector $\mathbf{u}$ and covariance matrix $\mathbf{S}$. Further, for
two sequences $\left\\{b_{m}\right\\}_{m\geq 1}$ and
$\left\\{b^{\ast}_{m}\right\\}_{m\geq 1}$, the “big O” notation
$b_{m}=O\left(b^{\ast}_{m}\right)$ means that $|b_{m}|\leq C|b^{\ast}_{m}|$
for some constant $C>0$ for sufficiently large $m$; for two positive sequences
$\left\\{b_{m}\right\\}_{m\geq 1}$ and $\left\\{b^{\ast}_{m}\right\\}_{m\geq
1}$, the “small o” notation $b_{m}=o\left(b^{\ast}_{m}\right)$ means that
$\lim_{m\rightarrow\infty}b_{m}\left(b^{\ast}_{m}\right)^{-1}=0$. Finally,
$\mathbb{P}$ denotes the probability measure and $\mathbb{E}$ the expectation
with respect to $\mathbb{P}$.

###  4.1 Decomposing the Normal vector via PFA

Recall that $\mathbf{Z}$ is an $m$-dimensional Normal vector with mean
$\boldsymbol{\mu}=\left(\mu_{1},...,\mu_{m}\right)^{T}$ and a known
correlation matrix $\mathbf{\Sigma}$. Let the spectral decomposition of
$\mathbf{\Sigma}$ have eigenvalues
$\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{m}\geq 0$ and corresponding
orthonormal eigenvectors $\boldsymbol{\rho}_{i}$, $i=1,...,m$. It is easy to
verify that for a fixed $\delta>0$ there exists $k$ between $0$ and $m-1$ ($k$
is dependent on $m$) such that

$m^{-1}\sqrt{\lambda_{k+1}^{2}+...+\lambda_{m}^{2}}=O\left(m^{-\delta}\right).$
(6)

Assume $m$ is large and pick $k$ such that (6) holds. Let
$\mathbf{\tilde{w}}=\left(\tilde{w}_{1},...,\tilde{w}_{m}\right)^{T}\sim\mathsf{N}_{m}\left(\mathbf{0},\mathbf{I}\right)$,
$\mathbf{w}^{\ast}_{k}=\left(\tilde{w}_{1},...,\tilde{w}_{k}\right)^{T}$,
$\mathbf{G}_{k}=\left(\sqrt{\lambda_{1}}\boldsymbol{\rho}_{1},\cdots,\sqrt{\lambda_{k}}\boldsymbol{\rho}_{k}\right)$
and
$\mathbf{v}=\sum_{j=k+1}^{m}{\sqrt{\lambda_{j}}\boldsymbol{\rho}_{j}\tilde{w}_{j}}=\left(v_{1},...,v_{m}\right)^{T}$,
where $\bf{I}$ is the identity matrix. Then we have

$\mathbf{Z}=\boldsymbol{\mu}+\mathbf{G}_{k}\mathbf{w}^{\ast}_{k}+\mathbf{v},$
(7)

or equivalently $\mathbf{Z}=\boldsymbol{\mu}+\boldsymbol{\eta}+\mathbf{v}$
upon setting $\boldsymbol{\eta}=\mathbf{G}_{k}\mathbf{w}_{k}^{\ast}$. Clearly,
$\boldsymbol{\eta}$ is independent of $\mathbf{v}$, and both are
$m$-dimensional Normal random vectors. Components of $\mathbf{w}^{\ast}_{k}$
are called the “principal factors” in Fan, Han and Gu (2012).

Let $\mathbf{C}=\left(c_{ij}\right)_{m\times m}$ be the covariance matrix of
$\mathbf{v}$. Since $k$ has been chosen such that (6) holds, we see

$m^{-2}\sum_{1\leq i,j\leq m}\left|c_{ij}\right|\leq
m^{-1}\left\|\mathbf{C}\right\|_{F}=O\left(m^{-\delta}\right)\text{,}$ (8)

where $\left\|\cdot\right\|_{F}$ denotes the Frobenius norm of a matrix. In
this case, components of $\mathbf{v}$ are called “weakly dependent” by Fan,
Han and Gu (2012). In view of this, we call $\boldsymbol{\eta}$ and
$\mathbf{v}$ the “major” and “minor” vectors, respectively. To summarize,
using the PFA we have decomposed $\mathbf{Z}$ into the sum of the major and
minor vectors, such that the minor vector contains weakly dependent entries
with heterogeneous variances.

Let $\sigma_{i}^{2}$ be the variance of the $i$th entry $\eta_{i}$ of the
major vector $\boldsymbol{\eta}$ and

$a_{i}=\left(1-\sigma_{i}^{2}\right)^{-1/2}\text{ for }1\leq i\leq m$ (9)

(with $0^{-1/2}=+\infty$). Then each $a_{i}$ is the reciprocal of the standard
deviation of the $i$th entry $v_{i}$ of the minor vector $\mathbf{v}$. The
important quantity $a_{\left(1\right)}=\min\nolimits_{1\leq i\leq m}a_{i}$
measures the speed of PFA, in that larger $a_{\left(1\right)}$ indicates
faster PFA. Note that $a_{\left(1\right)}$ tends to be large when
$\boldsymbol{\Sigma}$ has a few dominant eigenvalues.

###  4.2 Estimating the major vector by CPPLS

Now we turn to estimating the major vector $\boldsymbol{\eta}$. Since the mean
vector $\boldsymbol{\mu}$ contains a proportion $\pi_{0}$ of zero entries and
is usually sparse, we estimate $\boldsymbol{\eta}$ by taking
$\boldsymbol{\mu}$ as the incidental parameter. Specifically, we consider the
following estimate

$\boldsymbol{\hat{\theta}}=\left(\boldsymbol{\hat{\mu}}^{T},\boldsymbol{\hat{\eta}}^{T}\right)^{T}\in\operatorname*{arg\,min}_{\boldsymbol{\beta}\in\mathbb{R}^{m},\boldsymbol{\xi}\in\mathbb{R}^{m}}L\left(\boldsymbol{\beta},\boldsymbol{\xi};\lambda\right)\text{,}$
(10)

of
$\boldsymbol{\theta}=\left(\boldsymbol{\mu}^{T},\boldsymbol{\eta}^{T}\right)^{T}$,
where

$L\left(\boldsymbol{\beta},\boldsymbol{\xi};\lambda\right)=\left\|\mathbf{Z}-\boldsymbol{\beta}-\boldsymbol{\xi}\right\|^{2}+\sum_{i=1}^{m}\rho_{\lambda}\left(\left|\beta_{i}\right|\right)\text{,}$
(11)

and $\operatorname*{arg\,min}$ denotes the set of local minimizers. In (11),
the univariate function $\rho_{\lambda}\left(\cdot\right)$ is the _minimax
concave penalty_ (MCP) in Zhang (2010) with tuning parameter $\lambda>0$.
Specifically, $\rho_{\lambda}$ is defined by its derivative as

$\rho_{\lambda}^{\prime}\left(t\right)=a^{-1}\left(a\lambda-t\right)_{+}1_{\left\\{t\geq
0\right\\}}$ (12)

with a second tuning parameter $a>0$, where
$\left(x\right)_{+}=\max\left\\{0,x\right\\}$.

Since the components of $\mathbf{v}$ in model (7) are _weakly dependent and
have heterogeneous variances_ (WDH), $\rho_{\lambda}\left(\cdot\right)$ is
concave, and in (10) only $\boldsymbol{\beta}$ but not $\boldsymbol{\xi}$ is
penalized, we call (11) a CPPLS with Normal errors that are WDH. This is
different than the settings of most penalized linear regression methods where
the random errors have identical variances and there are no incidental
parameters. We point out that penalizing $\boldsymbol{\beta}$ in (11) is to
exploit the sparsity of $\boldsymbol{\mu}$ so that a more accurate estimate of
$\boldsymbol{\eta}$ can be obtained and that a local minimizer
$\boldsymbol{\hat{\theta}}$ in (10) is sufficient to construct the consistent
estimator of $\pi$; see Theorem 1 and Theorem 3. When we implement our
consistent estimator of $\pi$, we will directly obtain an estimate of the
principal factors $\mathbf{w}_{k}^{\ast}$ by an alternating optimization
technique from the equivalent formulation of (10) and (11); see Section 4.5
for details.

Let

$\gamma_{m}=a_{\left(1\right)}^{-1}\sqrt{2\log m}\text{\quad with
\quad}a_{\left(1\right)}=\min\limits_{1\leq i\leq m}a_{i},$ (13)

where we recall $a_{\left(1\right)}^{-1}$ is the maximum of the standard
deviations of the $v_{i}$’s. The quantity
$\gamma_{m}=a_{\left(1\right)}^{-1}\sqrt{2\log m}$ is roughly the maximum of
the $m$ entries $v_{i}$ of the minor vector $\bf{v}$. We now show the
existence of a solution to (10) such that $\boldsymbol{\hat{\eta}}$ has the
needed property (but not having to be consistent) for the extended FTM to
estimate $\pi$ later in Section 4.3 and Section 4.4.

######  Theorem 1.

Choose $k$ such that (6) holds. If $\mu^{\ast}=\sup\nolimits_{m\geq
1}\max\nolimits_{1\leq i\leq m}\left|\mu_{i}\right|$ is finite,
$\left\|\boldsymbol{\mu}\right\|=O\left(m^{\delta_{1}}\right)$ and
$\gamma_{m}=O\left(m^{2\left(\delta_{1}-1\right)}\right)$ for some
$0<\delta_{1}<1$, then, with probability approaching to $1$ as
$m\rightarrow\infty$, there exists an estimate $\boldsymbol{\hat{\eta}}$ of
$\boldsymbol{\eta}$ from (10) such that
$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|=O\left(m^{\delta_{1}}\right)$.

Since $0<\textrm{rank}\left(\mathbf{G}_{k}\right)=k\leq m-1$, Theorem 1 allows
us to construct an estimate $\mathbf{\hat{w}}\in\mathbb{R}^{k}$ of
$\mathbf{w}^{\ast}_{k}$ such that
$\boldsymbol{\hat{\eta}}=\mathbf{G}_{k}\hat{\mathbf{w}}$ and
$\left\|\mathbf{G}_{k}\hat{\mathbf{w}}-\mathbf{G}_{k}\mathbf{w}_{k}^{\ast}\right\|=O\left(m^{\delta_{1}}\right)$.
However, Theorem 1 does not say that $\hat{\boldsymbol{\eta}}$ is consistent
since only
$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|=O\left(m^{\delta_{1}}\right)$
for some $0<\delta_{1}<1$ is needed. We briefly comment on the conditions
involved in Theorem 1. The requirement that $\mu^{\ast}$ is finite and
$\left\|\boldsymbol{\mu}\right\|=O\left(m^{\delta_{1}}\right)$ for some
$0<\delta_{1}<1$ means that the magnitude of the largest nonzero Normal mean
does not have to large but it should not decrease to zero very rapidly as
$m\rightarrow\infty$. This requirement is easily satisfied. The condition
$\gamma_{m}=O\left(m^{2\left(\delta_{1}-1\right)}\right)$ for some
$0<\delta_{1}<1$ means that the maximal noise level induced by entries $v_{i}$
of the minor vector $\mathbf{v}$ should decrease to zero at appropriate speed
as $m\rightarrow\infty$. This can be achieved when $\boldsymbol{\Sigma}$ has a
few large eigenvalues of magnitude $O(m)$ or a few dominant eigenvalues; see
the types of covariance dependence and $\boldsymbol{\Sigma}$ in Section 5.2.

###  4.3 Extending the Fourier transform method

Once we are able to control the difference between the estimate
$\boldsymbol{\hat{\eta}}$ of the major vector $\boldsymbol{\eta}$ (see Theorem
1), it is possible to estimate $\pi$ from the mean-shifted, estimated minor
vector

$\mathbf{\hat{v}}^{\ast}=\mathbf{Z}-\boldsymbol{\hat{\eta}}=\left(\hat{v}_{1}^{\ast},\ldots,\hat{v}_{m}^{\ast}\right)^{T}\text{,}$
(14)

where
$\boldsymbol{\hat{\eta}}=\mathbf{Z}-\mathbf{G}_{k}\hat{\mathbf{w}}=\left(\hat{\eta}_{1},\ldots,\hat{\eta}_{m}\right)^{T}$
is given in Theorem 1. The superior performance of the Fourier transform (FT)
based estimator of $\pi$ in Jin (2008) relies crucially on the assumption that
the Normal random vector has independent or strongly mixing components and
that components with zero means have the same variance. However, noticing that
$v_{i}$, $i=1,...,m$ in model (7) are weakly dependent (which is different
than being strongly mixing) and have heterogeneous null distributions (i.e.,
potentially with different variances $a_{i}^{-2}$ even if their means are
zero), we need to extend the FT-based estimator so that it can applied to
$\mathbf{\hat{v}}^{\ast}$ in (14).

Let
$\phi_{\mu,\sigma}\left(x\right)=\left(\sqrt{2\pi}\sigma\right)^{-1}\exp\left(-2^{-1}\sigma^{-2}\left(x-\mu\right)^{2}\right)$
for $\sigma>0$. Define

$\kappa_{\sigma}\left(t;x\right)=\int_{\left[-1,1\right]}\omega\left(\zeta\right)\exp\left(2^{-1}\left(t\zeta\sigma\right)^{2}\right)\cos\left(t\zeta
x\right)d\zeta\text{,}$ (15)

and

$\psi\left(t;\mu\right)=\int_{\left[-1,1\right]}\exp\left(\sqrt{-1}t\mu\zeta\right)\omega\left(\zeta\right)d\zeta$
(16)

for some $\omega\left(\cdot\right)$ that is bounded on $\left[-1,1\right]$,
symmetric around $0$ and Lebesgue integrates to $1$. We have
$\mathbb{E}\left[\kappa_{\sigma}\left(t,X\right)\right]=\psi\left(t;\mu\right)=\hat{\omega}\left(t\mu\right)$
when $X\sim\mathsf{N}_{1}\left(\mu,\sigma^{2}\right)$, where $\hat{f}$ denotes
the Fourier transform of a Lebesgue integrable function $f$ on $\mathbb{R}$.
Obviously, $\psi\left(t;0\right)=1$ and

$\mathbb{E}\left[\kappa_{1}\left(t;X\right)\right]=\left(\kappa_{1}\left(t;\cdot\right)\ast\phi_{0,1}\right)\left(\mu\right)=\psi\left(t;\mu\right)=\mathbb{E}\left[\kappa_{\sigma}\left(t;X\right)\right]=\left(\kappa_{\sigma}\left(t;\cdot\right)\ast\phi_{0,\sigma}\right)\left(\mu\right)\text{,}$

where $\ast$ denotes the convolution. Further, Lemma 7.1 in Jin (2006) holds
for the pair of functions in (15) and (16). Let $v_{j}^{\ast}=\mu_{j}+v_{j}$
and $\mathbf{v}^{\ast}=\left(v_{1}^{\ast},...,v_{m}^{\ast}\right)^{T}$. Then
$v_{j}^{\ast}\sim\mathsf{N}_{1}\left(\mu_{j},a_{i}^{-2}\right)$. Following Jin
(2008), we define the underlying phase function

$\varphi\left(t\right)=\varphi\left(t;\boldsymbol{\mu},m\right)=\dfrac{1}{m}\sum_{j=1}^{m}\left[1-\psi\left(t;\mu_{j}\right)\right]$
(17)

and empirical phase function

$\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)=\dfrac{1}{m}\sum_{j=1}^{m}\left[1-\kappa_{a_{j}^{-1}}\left(t;v_{j}^{\ast}\right)\right]\text{.}$
(18)

We remark that $\kappa_{\sigma}\left(\cdot;\cdot\right)$ and
$\varphi_{m}\left(\cdot;\mathbf{\cdot}\right)$ generalize corresponding
functions in Jin (2008) by allowing for heterogeneous variance $a_{j}^{-2}$
for each $v_{j}^{\ast}$ since there $\kappa_{1}\left(\cdot;\cdot\right)$ has
been used.

The performance of the FT-based estimator hinges upon how accurately
$\varphi_{m}$ approximates $\varphi$, the oracle that knows the true value
$\pi$. Specifically, the smaller
$\delta_{m}\left(t\right)=\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right|$
is, the more accurately $\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)$
estimates $\pi$. So, we first need to bound $\delta_{m}\left(t\right)$. On the
other hand, $\mathbf{v}^{\ast}$ is unknown and estimated by
$\mathbf{\hat{v}}^{\ast}$ in (14). Therefore, we also need to bound
$\hat{\delta}_{m}\left(t\right)=\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi_{m}\left(t;\mathbf{\hat{v}}^{\ast}\right)\right|$.
Through bounds on both $\delta_{m}\left(t\right)$ and
$\hat{\delta}_{m}\left(t\right)$, we will be able develop an FT-based
estimator under the settings of our model (7). In fact, we have the following:

######  Theorem 2.

Choose $k$ such that (6) holds. Then, for any $\tilde{\epsilon}>0$, with
probability at least
$1-\tilde{\varepsilon}^{-2}p_{m}\left(t,a_{\left(1\right)}\right)$,

$\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right|\leq
M\tilde{\epsilon}\int_{0}^{1}\exp\left(2^{-1}t^{2}\zeta^{2}a_{\left(1\right)}^{-2}\right)d\zeta,$
(19)

where

$p_{m}\left(t,a_{\left(1\right)}\right)=t^{2}a_{\left(1\right)}^{-2}m^{-1}+Mt^{2}m^{-\delta}$
(20)

for some finite constant $M>0$. In particular, when $\sqrt{2\log
m}=o\left(a_{\left(1\right)}\right)$,

$\lim\limits_{m\rightarrow\infty}\sup\limits_{0<t\leq\sqrt{2\gamma\log
m}}\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right|=0$
(21)

almost surely for any $\gamma\in\left(0,1\right)$.

Theorem 2 bounds $\delta_{m}\left(t\right)$ when the random errors, i.e.,
$v_{j}$’s, have heterogeneous variances and are weakly dependent. Therefore,
it generalizes the bound on $\delta_{m}\left(t\right)$ in Jin and Cai (2007)
and Jin (2008), and we have extended the FT-based estimator to weakly
dependent Normal random errors with heterogeneous variances. The proof of
Theorem 2 uses a different technique than that in Jin (2008).

With regard to the conditions used in Theorem 2, we have the following
remarks: inequality (6) is always achievable by choosing a suitable $k$ as we
have pointed out earlier right after Theorem 1; the requirement $\sqrt{2\log
m}=o\left(a_{\left(1\right)}\right)$ means that the maximal standard deviation
$a_{\left(1\right)}^{-1}$ of the $v_{i}$’s should decrease to zero faster than
$\left(2\log m\right)^{-1/2}$, which is achievable when $\boldsymbol{\Sigma}$
has a few dominant eigenvalues; see the types of covariance dependence and
$\boldsymbol{\Sigma}$ in Section 5.2.

###  4.4 A consistent estimator of the proportion

We are ready to introduce the new, consistent estimator of the proportion
$\pi$ of nonzero Normal means. When the difference
$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|$ (see Theorem 1),
the overall speed $a_{\left(1\right)}$ of PFA, and the property of the nonzero
$\mu_{i}$’s are compatible with each other, the Lipschitz property of
$\kappa_{\sigma}$ (defined by (15)) will imply that

$\varphi_{m}\left(t;\mathbf{\hat{v}}^{\ast}\right)=\dfrac{1}{m}\sum_{j=1}^{m}\left[1-\kappa_{a_{j}^{-1}}\left(t;\hat{v}_{j}^{\ast}\right)\right]$
(22)

converges to $\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)$ in probability. In
other words, the plug-in estimator
$\tilde{\pi}=\varphi_{m}\left(t;\mathbf{\hat{v}}^{\ast}\right)$ will
consistently estimate $\pi$.

######  Theorem 3.

Let $\mu_{\ast}=\min\left\\{\left|\mu_{i}\right|:\left|\mu_{i}\right|\neq
0\right\\}$. Choose $k$ such that (6) holds and let $\pi\in\left(0,1\right)$
be independent of $m$. If the assumptions of Theorem 1 hold with
$0<\delta_{1}<2^{-1}$, and
$\lim\limits_{m\rightarrow\infty}\mu_{\ast}\sqrt{2\log m}=\infty$, then
$\tilde{\pi}=\varphi_{m}\left(\sqrt{2\gamma\log
m};\mathbf{\hat{v}}^{\ast}\right)$ with $\mathbf{\hat{v}}^{\ast}$ in (14) and
any $\gamma\in\left(0,1\right)$ consistently estimates $\pi$ as
$m\rightarrow\infty$.

Theorem 3 provides a consistent estimator $\tilde{\pi}$ of $\pi$ when the
correlation matrix $\boldsymbol{\Sigma}$ encodes strong dependence but has a
few dominant eigenvalues. It induces a consistent estimator of $\pi_{0}=1-\pi$
as $\tilde{\pi}_{0}=1-\tilde{\pi}$. Theorem 1 and Theorem 3 show that the
consistency of the new estimator $\tilde{\pi}$ does not require that of
estimating the major vector $\boldsymbol{\eta}$, the principal factors
$\mathbf{w}_{k}^{\ast}$, or the mean vector $\boldsymbol{\mu}$. They also show
that, due to the needed speed for the PFA to control the dependence among the
entries of the minor vector, the new estimator $\tilde{\pi}$ may not be
consistent for $\pi$ when $\mathbf{Z}$ has an arbitrary pair of mean vector
$\boldsymbol{\mu}$ and correlation matrix $\boldsymbol{\Sigma}$.

We briefly discuss the conditions involved in Theorem 3, which contain those
used in Theorem 1. The conditions in Theorem 1 are used to obtain an estimate
$\hat{\boldsymbol{\eta}}$ of $\boldsymbol{\eta}$ such that
$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|=O\left(m^{\delta_{1}}\right)$
for some $0<\delta_{1}<2^{-1}$, so that we can apply the extended FTM to the
entries of $\mathbf{\hat{v}}^{\ast}=\mathbf{Z}-\boldsymbol{\hat{\eta}}$ to
estimate $\pi$. They are satisfied when $\boldsymbol{\Sigma}$ has a few
eigenvalues of magnitude $O(m)$ or a few dominant eigenvalues as discussed
right after Theorem 1; see the types of covariance dependence and
$\boldsymbol{\Sigma}$ in Section 5.2. The condition
$\lim\limits_{m\rightarrow\infty}\mu_{\ast}\sqrt{2\log m}=\infty$ simply
requires the minimal magnitude of the nonzero Normal means not to be extremely
small. This condition is easily satisfied and was also used in Jin and Cai
(2007) and Jin (2008), and it is compatible with the conditions in Theorem 1.
Finally, the scale of the frequency domain is set as $\sqrt{2\gamma\log m}$
for the empirical phase function
$\varphi_{m}\left(t;\mathbf{\hat{v}}^{\ast}\right)$ in (22) as done also in
Jin (2008), so that the convergence rate of
$\varphi_{m}\left(t;\mathbf{\hat{v}}^{\ast}\right)$ compensates the
oscillations induced by the noise variables $v_{i}$ and
$\tilde{\pi}=\varphi_{m}\left(\sqrt{2\gamma\log
m};\mathbf{\hat{v}}^{\ast}\right)$ can consistently estimate $\pi$ for any
$\gamma\in\left(0,1\right)$. This property of $\tilde{\pi}$ allows to pick any
scale from the family $\left\\{\sqrt{2\gamma\log
m}:\gamma\in\left(0,1\right)\right\\}$ for the frequency domain in order to
consistently estimate $\pi$.

###  4.5 Implementation of the new estimator

In this section, we present the implementation of
$\tilde{\pi}=\varphi_{m}\left(t;\mathbf{\hat{v}}^{\ast}\right)$. PFA is easily
implemented by the spectral decomposition of the correlation matrix
$\boldsymbol{\Sigma}$ and then choosing

$k_{\delta}=\min\left\\{k:m^{-1}\sqrt{\lambda_{k+1}^{2}+...+\lambda_{m}^{2}}\leq
m^{-\delta}\right\\}$

with some $\delta\in\left(0,1\right)$. We set $\delta=0.5$ for the simulations
in Section 5 and the application in Section 3. After setting up the PFA, we
then implement the CPPLS using the MCP in (12) with $a=3.7$ (see Zhang (2010)
for the rationale on choosing $a=3.7$). Specifically, we solve the equivalent
problem

$\boldsymbol{\tilde{\theta}}=\left(\boldsymbol{\hat{\mu}}^{T},\mathbf{\hat{w}}^{T}\right)^{T}\in\operatorname*{arg\,min}_{\boldsymbol{\beta}\in\mathbb{R}^{m},\mathbf{w}\in\mathbb{R}^{k}}L\left(\boldsymbol{\beta},\mathbf{w};\lambda\right)$

with

$L\left(\boldsymbol{\beta},\mathbf{w};\lambda\right)=\left\|\mathbf{Z}-\boldsymbol{\beta}-\mathbf{G}_{k}\mathbf{w}\right\|^{2}+\sum\limits_{i=1}^{m}\rho_{\lambda}\left(\left|\beta_{i}\right|\right)$

via alternating optimization (e.g., Bezdek and Hathaway, 2002) through the
following steps:

  1. 1.

Set $j=0$ and initialize
$\mathbf{\tilde{w}}_{\ast}^{\left(j\right)}=\mathbf{0}\in\mathbb{R}^{k}$.

  2. 2.

Use the R package sparsenet of Mazumder et al. (2011) to compute the solution
path of
$\boldsymbol{\hat{\beta}}_{\lambda}^{\left(j\right)}\in\operatorname*{arg\,min}\limits_{\boldsymbol{\beta}\in\mathbb{R}^{m}}L\left(\boldsymbol{\beta},\mathbf{\tilde{w}}_{\ast}^{\left(j\right)};\lambda\right)$
for $\lambda\in
S_{\lambda}=\left\\{\tilde{\lambda}_{k^{\ast}},...,\tilde{\lambda}_{0}\right\\}$,
a grid of descendingly ordered $\lambda$ values automatically set by
sparsenet, where $\boldsymbol{\hat{\beta}}_{\lambda}^{\left(j\right)}$ has
been indexed by $\lambda\in$ $S_{\lambda}$. Pick
$\boldsymbol{\hat{\beta}}_{\ast}^{\left(j\right)}$,
$\lambda_{\ast}^{\left(j\right)}$ for which

$\left(\boldsymbol{\hat{\beta}}_{\ast}^{\left(j\right)T},\lambda_{\ast}^{\left(j\right)}\right)^{T}=\operatorname*{arg\,min}\limits_{\boldsymbol{\beta}\in\mathbb{R}^{m},\lambda\in
S_{\lambda}}L\left(\boldsymbol{\beta},\mathbf{\tilde{w}}_{\ast}^{\left(j\right)};\lambda\right).$

  3. 3.

Compute
$\mathbf{\tilde{w}}_{\ast}^{\left(j+1\right)}=\mathbf{D}^{-1}\mathbf{T}^{T}\left(\mathbf{z}-\boldsymbol{\hat{\beta}}_{\ast}^{\left(j\right)}\right)$,
where
$\mathbf{D}=\mathrm{diag}\left\\{\sqrt{\lambda_{1}},\cdots,\sqrt{\lambda_{k}}\right\\}$,
$\mathbf{T}=\left(\boldsymbol{\rho_{1}},\cdots,\boldsymbol{\rho_{k}}\right)$,
and $\mathbf{z}$ is the observation from $\mathbf{Z}$. Note that the
$\lambda_{i}$’s are the eigenvalues of $\boldsymbol{\Sigma}$ and
$\boldsymbol{\rho}_{i}$’s are their corresponding eigenvectors.

  4. 4.

Set $j$ to be $j+1$. Repeat steps 2. and 3. until
$\left|l^{\left(j+1\right)}-l^{\left(j\right)}\right|<\tau_{\ast}$ with
$\tau_{\ast}=10^{-5}$, where
$l^{(j)}=L\left(\boldsymbol{\beta}_{\ast}^{\left(j\right)},\mathbf{w}_{\ast}^{\left(j\right)};\lambda_{\ast}^{\left(j\right)}\right)$
.

Suppose a local minimizer
$\boldsymbol{\tilde{\theta}}=\left(\boldsymbol{\hat{\beta}}^{T},\mathbf{\hat{w}}^{T}\right)$
has been found, we compute
$\mathbf{\hat{v}}^{\ast}=\mathbf{z}-\mathbf{G}_{k}\hat{\mathbf{w}}$ as in
(14). To implement the last step, i.e., the Fourier transform in (22), for
$\tilde{\pi}$, we use the triangular density
$\omega\left(x\right)=\left(1-\left|x\right|\right)1_{\left\\{\left|x\right|\leq
1\right\\}}$ and modify the R codes of Jin (2008) to accommodate (17) and (18)
since the standard deviations $a_{j}^{-1}$ of the $v_{j}$’s are not equal to
each other and $v_{j}$’s are weakly dependent. The modified codes are then
applied to the components of $\mathbf{\hat{v}}_{\ast}$ to estimate $\pi$.

##  5 Simulation studies

We first describe the estimators of $\pi$ to be compared and then present the
simulation studies. By the duality between $\pi$ and $\pi_{0}=1-\pi$, we will
transform an estimator $\hat{\pi}_{0}$ of $\pi_{0}$ into
$\hat{\pi}=1-\hat{\pi}_{0}$, and compare the new estimator $\tilde{\pi}$ with
$\hat{\pi}^{B}=1-\hat{\pi}_{0}^{B}$ with the median based estimator

$\hat{\pi}_{0}^{B}=\left(m-\left[m/2\right]+1\right)\left[m\left(1-p_{\left(\left[m/2\right]\right)}\right)\right]^{-1}$

in Benjamini et al. (2006), $\hat{\pi}^{L}=1-\hat{\pi}_{0}^{L}$ with
$\hat{\pi}_{0}^{L}$ in Langaas et al. (2005), and
$\hat{\pi}^{W}=1-\hat{\pi}_{0}^{W}$ with $\hat{\pi}_{0}^{W}$ (also called
“SLIM”) in Wang et al. (2011). It should be noted that $\hat{\pi}^{W}$ was
claimed by its developers to be robust to dependence. Since the estimators
$\hat{\pi}^{B}$, $\hat{\pi}^{L}$ and $\hat{\pi}^{W}$ are based on p-values, we
will convert each observed Normal random variable $z_{i}$ into
$p_{i}=1-\Phi\left(|z_{i}|\right)$ and apply each of these estimators to the
$m$ p-values to estimate $\pi$, where $\Phi$ is the cumulative distribution
function of the standard Normal random variable
$\mathsf{N}_{1}\left(0,1\right)$.

###  5.1 Simulation setup

We will choose $5$ values of $\pi$, and for each value of $\pi$ simulate $5$
types of covariance dependence and $5$ levels of minimal signal strength. Set
$m=2000$. For each $\pi\in\left\\{0.1,0.2,0.3,0.4,0.5\right\\}$ and
$\mu_{\ast}\in\left\\{3,2,1.5,1,0.45\right\\}$, execute the following:

Step 1.

Set $m_{0}=m\left(1-\pi\right)$ and $m_{1}=m\pi$. For the mean vector
$\boldsymbol{\mu}\in\mathbb{R}^{m}$, set $\mu_{j}=0$ for $1\leq j\leq m_{0}$
but generate $|\mu_{j}|$ for $m_{0}+1\leq j\leq m$ independently from the
uniform distribution on $\left[\mu_{\ast},\mu_{\ast}+1\right]$. The signs of
$\mu_{j}$ for $m_{0}+1\leq j\leq m$ are generated from $m_{1}$ independent
Bernoulli random variables each taking values $\pm 1$ with equal probability.

Step 2.

Generate
$\boldsymbol{\varepsilon}=\left(\varepsilon_{1},...,\varepsilon_{m+3}\right)^{T}$
whose entries are $m+3$ i.i.d. realizations of the standard Normal random
variable.

Step 3.

Pick one of the $5$ types of covariance dependence:

  * •

Block: set $Z_{i}^{\ast}=\varepsilon_{i}$ for $1\leq i\leq 0.95m$ but
$Z_{i}^{\ast}=\sum\limits_{j=1}^{c_{0}}\left(-1\right)^{j+1}0.2\varepsilon_{j}+\sqrt{1-0.2^{2}c_{0}}\varepsilon_{i}$
with $c_{0}=0.01m$ for $0.95m+1\leq i\leq m$. The corresponding correlation
matrix $\boldsymbol{\Sigma}$ is not block diagonal but has three blocks of
size $100$ of dependent $Z_{i}^{\ast}$’s, one in the upper right, the other
the lower left, and the third the lower right, where some correlations are
strong.

  * •

Equal correlation (“Equal Corr.”): set
$\boldsymbol{\Sigma}=\rho\mathbf{I}+\left(1-\rho\right)\mathbf{11}^{T}$ with
$\rho=0.5$ and generate
$\mathbf{Z}^{\ast}\sim\mathsf{N}_{m}\left(\mathbf{0},\mathbf{\Sigma}\right)$,
where $\mathbf{1}$ is a column vector of $1$’s.

  * •

Three factors (“3 Factors”):
$Z_{i}^{\ast}=-0.25\varepsilon_{1}+0.2\varepsilon_{2}-0.125\varepsilon_{3}+\varepsilon_{i+3}$
for $1\leq i\leq m$. The correlation between $Z_{i}^{\ast}$ and $Z_{j}^{\ast}$
for $i\neq j$ is $4^{-2}+5^{-2}+8^{-2}\approx 0.118$, i.e., the
$Z_{i}^{\ast}$’s are equally correlated but the equal correlations are
generated by $3$ factors $\varepsilon_{1}$, $\varepsilon_{2}$ and
$\varepsilon_{3}$.

  * •

Two components (“Two Comp.”): set $Z_{1}^{\ast}=\varepsilon_{1}$,
$Z_{i}^{\ast}=2^{-1/2}\left(-\varepsilon_{1}+\varepsilon_{i}\right)$ for
$2\leq i\leq m$. The correlation between $Z_{1}^{\ast}$ and $Z_{i}^{\ast}$ for
$i\neq 1$ is $2^{-1/2}$, and that between $Z_{i}^{\ast}$ and $Z_{j}^{\ast}$
for $i,j>1$ and $i\neq j$ is $0.5$. The corresponding correlation matrix
$\boldsymbol{\Sigma}$ has two components, one for equal correlation $2^{-1/2}$
and the other for equal correlation $0.5$.

  * •

Unstructured: generate a $m\times 4$ matrix $\mathbf{Q}$ of i.i.d.
observations from the standard Normal random variable, compute the sample
correlation matrix $\mathbf{H}$ of $\mathbf{Q}$, and set
$\boldsymbol{\Sigma}=0.4\mathbf{H}+0.6\mathbf{I}$. There is no structure for
the correlations among the $\mathbf{Z}^{\ast}$’s when they have correlation
matrix $\boldsymbol{\Sigma}$.

Step 4.

Set $\mathbf{Z}^{\ast}=\left(Z_{1}^{\ast},...,Z_{m}^{\ast}\right)^{T}$ and
$\mathbf{Z}=\boldsymbol{\mu}+\mathbf{Z}^{\ast}$, for which $\mathbf{Z}$ has
correlation matrix $\boldsymbol{\Sigma}$; generate an observation $\mathbf{z}$
from $\mathbf{Z}$ and apply each of the four estimators $\tilde{\pi}$,
$\hat{\pi}^{B}$, $\hat{\pi}^{L}$ and $\hat{\pi}^{W}$ to estimate $\pi$ based
on $\mathbf{z}$.

Step 5.

Repeat Steps 1. to 4. $100$ times.

We refer to “Block”, “Equal correlation”, “Three factors” and “Two components”
as “structured covariance dependence”. Each of the three types of dependence,
“Equal correlation”, “Three factors” and “Two components”, represents long
range dependence which is neither weak dependence nor strong mixing. So, these
are types of strong but structured covariance dependence for components of a
Normal random vector. The “Unstructured” dependence can induce long range
dependence too. For each of the structured covariance dependence, the
corresponding $\boldsymbol{\Sigma}$ has one dominant eigenvalue; see Figure 2
for the $20$ largest eigenvalues for each type of $\boldsymbol{\Sigma}$.

Figure 2: The $20$ largest eigenvalues of the correlation matrix
$\boldsymbol{\Sigma}$ for each type of dependence. For each structured
covariance dependence, the corresponding correlation matrix
$\boldsymbol{\Sigma}$ has one dominant eigenvalue, whereas for “Unstructured”
covariance dependence $\boldsymbol{\Sigma}$ has $4$ dominant eigenvalues.

###  5.2 Summary for simulation results

An estimator $\hat{\pi}$ of the proportion $\pi$ of nonzero Normal means is
said to be better if it is no larger than $\pi$ (in order to maintain
conservative FDR control or estimation) and has smaller bias and standard
deviation (i.e., is more accurate and stable). Recall that $\mu_{\ast}$ is the
minimum of the absolute values of the nonzero means, i.e., the minimal signal
strength. For $\mu_{\ast}=3$, Figure 3 visualizes the performances of the
estimators of $\pi$ and Table 1 records the biases and _sample standard
deviations_ (std dev) of these estimators. They show that the new estimator
$\tilde{\pi}$ is the best for structured covariance dependence. Results for
$\mu_{\ast}\in\left\\{2,1.5,1,0.45\right\\}$ are given in the supplementary
material. We omit reporting the performance of $\hat{\pi}^{L}$ since it is
always close to $1$.

For structured covariance dependence, the following are observed: (i) the new
estimator $\tilde{\pi}$ is the best when the minimal signal strength is
relatively large; (ii) it is also the best when $\pi<0.4$ regardless of the
minimal signal strength; (iii) it always has the smallest variance. For
unstructured covariance dependence, the following are observed: (i) the new
estimator tends to be more accurate as the minimal signal strength becomes
smaller when $\pi$ is not large; (ii) it is the best when the minimal signal
strength is small and $\pi\leq 0.3$; (iii) it is the best when all
combinations of values of $\pi$ and the minimal signal strength are
considered; see Figure 3 and the figures in the supplementary material. These
observations suggest that the new estimator can be safely applied to estimate
$\pi$ in applications where our problem setup is appropriate (see the examples
in Section 2 and Section 3), the proportion of nonzero Normal means $\pi\leq
0.3$, and the correlation matrix of the test statistics is approximately
structured with a few dominating eigenvalues.

Finally, the SLIM estimator $\hat{\pi}^{W}$ and the median based estimator
$\hat{\pi}^{B}$ usually have relatively large variance when the covariance
dependence is “Equal correlation” or “Two components”. For structured
covariance dependence, when $\pi\geq 0.4$ but $\mu_{\ast}\leq 1$, i.e., when
the signals are dense and relatively weak, the median based estimator
$\hat{\pi}^{B}$ is more accurate than the new estimator. This is likely
because in this case more information on the distribution of nonzero Normal
means is available to $\hat{\pi}^{B}$. For unstructured covariance dependence,
when $\mu_{\ast}\geq 2$ but $\pi\leq 0.3$, the SLIM estimator $\hat{\pi}^{W}$
is slightly more accurate than the new estimator; see Table 1 and the tables
in the supplementary material. This is likely because in this case the SLIM
estimator better utilizes the cumulative distribution function (CDF) of the
p-values since it is based on the quantiles of local estimates of $\pi$
obtained from different parts of such a CDF.

Figure 3: The excess $d\left(\hat{\pi}\right)=\hat{\pi}-\pi$ of an estimator $\hat{\pi}$ of the proportion $\pi$ of nonzero Normal means when the minimum of the absolute values of the nonzero Normal means $\mu_{\ast}=3$, where the dashed line marks $\hat{\pi}=\pi$. A better estimator $\hat{\pi}$ of $\pi$ should have non-positive excess and smaller variance and be less downwardly biased. The new estimator “New” is overall the best.  | Bias | Std Dev
---|---|---
| Benjamini | New | SLIM | Benjamini | New | SLIM
Block | $\pi=0.1$ | 0.25596 | -0.0030166 | 0.061457 | 0.004794 | 0.011244 | 0.02679
$\pi=0.2$ | 0.18357 | -0.0039275 | 0.040868 | 0.004941 | 0.010976 | 0.02324
$\pi=0.3$ | 0.11486 | -0.0054009 | 0.029470 | 0.003895 | 0.010209 | 0.01943
$\pi=0.4$ | 0.05051 | -0.0057885 | 0.012503 | 0.003202 | 0.009552 | 0.01619
$\pi=0.5$ | -0.01558 | -0.0070828 | 0.006517 | 0.001575 | 0.009948 | 0.01503
Equal Corr. | $\pi=0.1$ | 0.24873 | -0.0007765 | 0.051670 | 0.055512 | 0.004958 | 0.07836
$\pi=0.2$ | 0.17267 | -0.0016526 | 0.033969 | 0.049102 | 0.004933 | 0.06953
$\pi=0.3$ | 0.10052 | -0.0030615 | 0.022090 | 0.040617 | 0.004321 | 0.05733
$\pi=0.4$ | 0.03526 | -0.0040539 | 0.014974 | 0.027338 | 0.004237 | 0.04356
$\pi=0.5$ | -0.01813 | -0.0051752 | 0.008714 | 0.002569 | 0.003711 | 0.03332
3 Factors | $\pi=0.1$ | 0.25553 | -0.0002324 | 0.058420 | 0.013090 | 0.008719 | 0.03000
$\pi=0.2$ | 0.18261 | -0.0005969 | 0.040527 | 0.012133 | 0.008399 | 0.02878
$\pi=0.3$ | 0.11361 | -0.0019539 | 0.029154 | 0.009903 | 0.007921 | 0.02341
$\pi=0.4$ | 0.04898 | -0.0033888 | 0.019038 | 0.006551 | 0.007873 | 0.01969
$\pi=0.5$ | -0.01617 | -0.0052064 | 0.008945 | 0.001535 | 0.007080 | 0.01509
Two Comp. | $\pi=0.1$ | 0.24870 | -0.0005933 | 0.053347 | 0.055771 | 0.004695 | 0.07483
$\pi=0.2$ | 0.17245 | -0.0019353 | 0.034035 | 0.049453 | 0.004394 | 0.06772
$\pi=0.3$ | 0.10041 | -0.0024518 | 0.022508 | 0.040502 | 0.004414 | 0.05448
$\pi=0.4$ | 0.03490 | -0.0039782 | 0.010536 | 0.027430 | 0.003828 | 0.04582
$\pi=0.5$ | -0.01810 | -0.0052380 | 0.003586 | 0.002730 | 0.003735 | 0.03563
Unstructured | $\pi=0.1$ | 0.25444 | 0.0897267 | 0.049023 | 0.023004 | 0.050224 | 0.04321
$\pi=0.2$ | 0.18125 | 0.0757672 | 0.038145 | 0.020739 | 0.042348 | 0.03765
$\pi=0.3$ | 0.11149 | 0.0639421 | 0.025340 | 0.018255 | 0.036052 | 0.03251
$\pi=0.4$ | 0.04690 | 0.0507392 | 0.017732 | 0.012268 | 0.029720 | 0.02373
$\pi=0.5$ | -0.01668 | 0.0359593 | 0.004582 | 0.001856 | 0.021465 | 0.02174
Table 1: Bias and sample standard deviation (std dev) of estimators of the
proportion $\pi$ of nonzero Normal means when the minimum of the absolute
values of the nonzero Normal means $\mu_{\ast}=3$. A better estimator
$\hat{\pi}$ of $\pi$ should have non-positive bias and smaller variance and be
less downwardly biased. The new estimator “New” is overall the best.

##  6 Discussion

For multiple testing which components of a Normal random vector have zero
means, we have developed a consistent estimator of the proportion of nonzero
Normal means when the components can be strongly dependent and their
correlation matrix has a few dominating eigenvalues. The new estimator is
accurate and stable. It can be applied to multiple testing related to marginal
regression analysis for high-dimensional data where the random errors are
Normally distributed.

In order to develop this estimator, we have established the existence and
property of a concave partially penalized least squares (CPPLS) estimate with
weakly dependent, heterogenous Normal random errors in the presence of
incidental parameters. We have also extended the Fourier transform method to
estimate this proportion in Jin (2008) to the case of weakly dependent Normal
random errors that have heterogeneous variances. Our results on the CPPLS
estimate of the principal factors can be strengthened to provide an
alternative to several strategies for adjusting for dependence in multiple
testing. These include the method in Friguet et al. (2009) to estimate the
factors by the expectation-maximization (EM) algorithm (Dempster et al.,
1977), that in Fan, Han and Gu (2012) to estimate the principal factors by
$L_{1}$-regression under the assumption $\pi\approx 0$, and that in Desai and
Storey (2012) to estimate the shared variation by ridge-type shrinkage. Our
extension of the FTM of Jin (2008) is nontrivial since the Normal random
errors have heterogenous variances and are weakly dependent. Accordingly, we
have used a general technique to derive the bounds on the difference between
the phase function and its estimate.

We point out that it may still be possible to consistently estimate the
proportion of nonzero Normal means when the correlation matrix of the Normal
random vector is unknown but not accurately estimated. Note that $\pi$ is
invariant under any permutation of the indices of the means that keeps $\pi$
unchanged and that it is invariant under nonzero scaling of the means. Since a
permutation corresponds to an orthogonal matrix and a nonzero scaling to a
nonsingular diagonal matrix, the two invariant properties of $\pi$ imply that
the estimation of $\pi$ is quite robust to the inaccuracy in estimating the
factors (or major vector), the means and the correlation/covariance matrix. We
leave the exploration along this line to future research.

##  7 Supplementary material

We provide the proofs of Theorem 1, Theorem 2, and Theorem 3, and additional
simulation results for the new estimator under different minimal signal
strengths of the nonzero Normal means.

## Acknowledgements

This research was funded in part by a National Science Foundation Plant Genome
Research Program grant (No. IOS-1025976) to R.W. Doerge, and it produced the
technical report Chen and Doerge (2012). The authors would like to thank
Philip T. Reiss and Armin Schwartzman for providing the brain imaging data set
and R codes used for the analysis in Azriel and Schwartzman (2015), Jiashun
Jin for providing the technical report Jin (2006), Jianqing Fan for helpful
pointers, and Runlong Tang for sharing the manuscript Fan, Tang and Shi
(2012).

## References

  * (1)
  * Azriel and Schwartzman (2015) Azriel, D. and Schwartzman, A. (2015).  The empirical distribution of a large number of correlated normal variables, J. Amer. Statist. Assoc. 110(511): 1217–1228.
  * Benjamini et al. (2006) Benjamini, Y., Krieger, A. M. and Yekutieli, D. (2006).  Adaptive linear step-up procedures that control the false discovery rate, Biometrika 93(3): 491–507.
  * Benjamini and Yekutieli (2001) Benjamini, Y. and Yekutieli, D. (2001).  The control of the false discovery rate in mutliple testing under dependency, Ann. Statist. 29(4): 1165–1188.
  * Bezdek and Hathaway (2002) Bezdek, J. C. and Hathaway, R. J. (2002).  Some notes on alternating optimization, Lecture Notes in Computer Science pp. 187–195.
  * Blanchard and Roquain (2009) Blanchard, G. and Roquain, E. (2009).  Adaptive false discovery rate control under independence and dependence, J. Mach. Learn. Res. 10: 2837–2871.
  * Chen and Doerge (2012) Chen, X. and Doerge, R. (2012).  Estimating the proportion of nonzero normal means under certain strong covariance dependence, Technical report. Department of statistics, Purdue University, West Lafayette .
  * Dempster et al. (1977) Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977).  Maximum likelihood from incomplete data via the em algorithm, J. R. Statist. Soc. Ser. B 39(1): 1–38.
  * Desai and Storey (2012) Desai, K. H. and Storey, J. D. (2012).  Cross-dimensional inference of dependent high-dimensional data, J. Amer. Statist. Assoc. 107(497): 135–151.
  * Efron et al. (2001) Efron, B., Tibshirani, R., Storey, J. D. and Tusher, V. (2001).  Empirical bayes analysis of a microarray experiment, J. Amer. Statist. Assoc. 96(456): 1151–1160.
  * Fan, Han and Gu (2012) Fan, J., Han, X. and Gu, W. (2012).  Estimating false discovery proportion under arbitrary covariance dependence, J. Amer. Statist. Assoc. 107(499): 1019–1035.
  * Fan, Tang and Shi (2012) Fan, J., Tang, R. and Shi, X. (2012).  Partial consistency with sparse incidental parameters, arxiv.org/pdf/1210.6950 .
  * Friguet et al. (2009) Friguet, C., Kloareg, M. and Causeur, D. (2009).  A factor model approach to multiple testing under dependence, J. Amer. Statist. Assoc. 104(488): 1406–1415.
  * Jin (2006) Jin, J. (2006).  Propotions of nonzero normal means: universal oracle equivalence and uniformly consistent estimations, Technical report. Department of statistics, Purdue University, West Lafayette .
  * Jin (2008) Jin, J. (2008).  Proportion of non-zero normal means: universal oracle equivalences and uniformly consistent estimators, J. R. Statist. Soc. Ser. B 70(3): 461–493.
  * Jin and Cai (2007) Jin, J. and Cai, T. T. (2007).  Estimating the null and the proportion of nonnull effects in large-scale multiple comparisons, J. Amer. Statist. Assoc. 102(478): 495–506.
  * Klebanov and Yakovlev (2007) Klebanov, L. and Yakovlev, A. (2007).  Diverse correlation structures in gene expression data and their utility in improving statistical inference, Ann. Statist. 1(2): 538–559.
  * Langaas et al. (2005) Langaas, M., Henery, L. B. and Ferkingstad, E. (2005).  Estimating the proportion of true null hypotheses, with application to dna microarray data, J. R. Statist. Soc. Ser. B 67(4): 555–572.
  * Leek and Storey (2008) Leek, J. T. and Storey, J. D. (2008).  A general framework for multiple testing dependence, Proc. Natl. Acad. Sci. U.S.A. 105(48): 18718–18723.
  * Lyons (1988) Lyons, R. (1988).  Strong laws of large numbers for weakly correlated random variables, The Michigan Mathematical Journal 35: 353–359.
  * Mazumder et al. (2011) Mazumder, R., Friedman, J. H. and Hastie, T. (2011).  parsenet: Coordinate descent with nonconvex penalties, J. Amer. Statist. Assoc. 106(495): 1125–1138.
  * Owen (2005) Owen, A. B. (2005).  Variance of the number of false discoveries, 67(3): 411–426.
  * Ploner et al. (2006) Ploner, A., Calza, S., Gusnanto, A. and Pawitan, Y. (2006).  Multidimensional local false discovery rate for microarray studies, Bioinformatics 22(5): 556–565.
  * Proal et al. (2011) Proal, E., Reiss, P. T., Klein, R. G. and et al (2011).  Brain gray matter deficits at 33-year follow-up in adults with attention-deficit/hyperactivity disorder established in childhood, Arch. Gen. Psychiatry 68(11): 1122–1134.
  * Storey (2003) Storey, J. (2003).  The positive false discovery rate: a Bayesian intepretation and the q-value, Ann. Statist. 3(6): 2013–2035.
  * Storey (2002) Storey, J. D. (2002).  A direct approach to false discovery rates, J. R. Statist. Soc. Ser. B 64(3): 479–498.
  * Storey et al. (2004) Storey, J. D., Taylor, J. E. and Siegmund, D. (2004).  Strong control, conservative point estimation in simultaneous conservative consistency of false discover rates: a unified approach, J. R. Statist. Soc. Ser. B 66(1): 187–205.
  * Wang et al. (2011) Wang, H.-Q., Tuominen, L. K. and Tsai, C.-J. (2011).  SLIM: A sliding linear model for estimating the proportion of true null hypotheses in datasets with dependence structures, Bioinformatics 27: 225–231.
  * Zhang (2010) Zhang, C.-H. (2010).  Nearly unbiased variable selection under minimax concave penalty, Ann. Statist. 38(2): 894–942.

Supplement for “A consistent estimator of the proportion of
nonzero Normal means under certain strong covariance dependence”

Xiongzhi Chen‡‡‡Corresponding author. Center for Statistics and Machine
Learning and Lewis-Sigler Institute, Princeton University, Princeton, NJ
08544, USA; E-mail<EMAIL_ADDRESS>and Rebecca W. Doerge§§§
Department of Statistics and Department of Agronomy, Purdue University, West
Lafayette, IN 47907, USA; E-mail<EMAIL_ADDRESS>

##  Appendix A Proofs

In the proofs, $\mathbb{P}$ denotes the probability measure, $\mathbb{E}$ the
expectation with respect to $\mathbb{P}$, “$\mathbb{P}$-a.s.” means “almost
surely in $\mathbb{P}$”, and cov and var are the covariance and variance
operator with respect to $\mathbb{P}$, respectively. Further, $O_{\mathbb{P}}$
and $o_{\mathbb{P}}$ are the probabilistic versions of the “big $O$” and
“small $o$” notations, respectively. Finally, $M$ denotes a generic, positive
constant whose values can vary at different occurrences. We will use the same
notations and settings in the main paper, and all asymptotics are for when
$m\rightarrow\infty$.

###  A.1 Proof of Theorem 1

Let $\operatorname*{sgn}$ be the sign function. It is easy to see that
$\boldsymbol{\hat{\mu}}=\left(\hat{\mu}_{1},...,\hat{\mu}_{m}\right)^{T}$ and
$\boldsymbol{\hat{\eta}}=\left(\hat{\eta}_{1},...,\hat{\eta}_{m}\right)^{T}$
necessarily satisfy

$\left\\{\begin{array}[]{l}\hat{\eta}_{i}=Z_{i}-\hat{\mu}_{i}\text{ for }1\leq
i\leq m\text{,}\\\
2\left(\hat{\mu}_{i}+\hat{\eta}_{i}-Z_{i}\right)+\rho_{\lambda}^{\prime}\left(\left|\hat{\mu}_{i}\right|\right)\operatorname*{sgn}\left(\hat{\mu}_{i}\right)=0\text{
if }\hat{\mu}_{i}\neq 0\text{,}\\\
-2^{-1}\lambda\leq\hat{\mu}_{i}+\hat{\eta}_{i}-Z_{i}\leq 2^{-1}\lambda\text{
if }\hat{\mu}_{i}=0\text{.}\end{array}\right.$

Let $\hat{I}_{0}=\left\\{1\leq i\leq m:\hat{\mu}_{i}=0\right\\}$ and
$\hat{I}_{1}=\left\\{1,...,m\right\\}\setminus\hat{I}_{0}$. Then
$\hat{\eta}_{i}=Z_{i}$ and $\hat{\eta}_{i}-\eta_{i}=\mu_{i}+v_{i}$ for
$i\in\hat{I}_{0}$ whereas $\hat{\eta}_{i}=Z_{i}-\hat{\mu}_{i}$ and
$\left|\hat{\mu}_{i}\right|\geq a\lambda$ for $i\in\hat{I}_{1}$. Therefore,

$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|^{2}=\sum_{i\in\hat{I}_{0}}\left(\mu_{i}+v_{i}\right)^{2}+\sum_{i\in\hat{I}_{1}}\left(\mu_{i}-\hat{\mu}_{i}+v_{i}\right)^{2}\text{.}$
(A.1)

It is left to pick $\boldsymbol{\hat{\mu}}$ and $a\lambda$ such that
$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|=O\left(m^{\delta_{1}}\right)$.

Let $E_{1}=\left\\{\max_{1\leq i\leq
m}\left|v_{i}\right|\leq\gamma_{m}\right\\}$, where we recall that $v_{i}$’s
are components of $\mathbf{v}$. Note
$\lim\limits_{m\rightarrow\infty}a_{\left(1\right)}\gamma_{m}=\lim\limits_{m\rightarrow\infty}\sqrt{2\log
m}=\infty$. By Mill’s ratio,

$\mathbb{P}\left(E_{1}^{C}\right)\leq\sum\limits_{i=1}^{m}\mathbb{P}\left(\left\\{a_{i}\left|v_{i}\right|>a_{\left(1\right)}\gamma_{m}\right\\}\right)\leq\left(2\log
m\right)^{-1/2}+o\left(1\right)\text{.}$

So, $\mathbb{P}\left(E_{1}\right)=1+o\left(1\right)$. Hereunder, we condition
our arguments on the event $E_{1}$ and do not mention this explicitly when a
proposition holds _with probability approaching to $1$_ (abbreviated as
“wpa-1”).

Set $\left|\hat{\mu}_{i}\right|=a\lambda$ for $i\in\hat{I}_{1}$. Then

$\displaystyle\sum_{i\in\hat{I}_{1}}\left(\mu_{i}-\hat{\mu}_{i}+v_{i}\right)^{2}$
$\displaystyle\leq$
$\displaystyle\sum_{i\in\hat{I}_{1}}\left(\mu_{i}-\hat{\mu}_{i}\right)^{2}+\left|\hat{I}_{1}\right|\gamma_{m}^{2}+2\sum_{i,j\in\hat{I}_{1};i\neq
j}\left|\mu_{i}-\hat{\mu}_{i}\right|\gamma_{m}$ $\displaystyle\leq$
$\displaystyle\sum_{i\in\hat{I}_{1}}\mu_{i}^{2}+2a\lambda\sum_{i,j\in\hat{I}_{1};i\neq
j}\left|\mu_{i}\right|+\left|\hat{I}_{1}\right|a^{2}\lambda^{2}+\left|\hat{I}_{1}\right|^{2}\left(a\lambda+\mu^{\ast}\right)\gamma_{m}+\left|\hat{I}_{1}\right|\gamma_{m}^{2}$
$\displaystyle\leq$
$\displaystyle\sum_{i\in\hat{I}_{1}}\mu_{i}^{2}+a\lambda\left|\hat{I}_{1}\right|^{2}\mu^{\ast}+\left|\hat{I}_{1}\right|a^{2}\lambda^{2}+\left|\hat{I}_{1}\right|^{2}\left(a\lambda+\mu^{\ast}\right)\gamma_{m}+\left|\hat{I}_{1}\right|\gamma_{m}^{2}\text{.}$

On the other hand,

$\displaystyle\sum_{i\in\hat{I}_{0}}\left(\mu_{i}+v_{i}\right)^{2}$
$\displaystyle\leq$
$\displaystyle\sum_{i\in\hat{I}_{0}}\mu_{i}^{2}+2\gamma_{m}\sum_{i,j\in\hat{I}_{1};i\neq
j}\left|\mu_{i}\right|+\left|\hat{I}_{1}\right|\gamma_{m}^{2}$
$\displaystyle\leq$
$\displaystyle\sum_{i\in\hat{I}_{0}}\mu_{i}^{2}+\gamma_{m}\left|\hat{I}_{0}\right|^{2}\mu^{\ast}+\left|\hat{I}_{0}\right|\gamma_{m}^{2}\text{.}$

Definitely, we can set $\hat{I}_{1}$ and $a\lambda$ such that
$\left|\hat{I}_{1}\right|\geq 1$, $\lim\limits_{m\rightarrow\infty}a\lambda=0$
but $a\lambda\left|\hat{I}_{1}\right|^{2}=O\left(m^{2\delta_{1}}\right)$.
This, with the finiteness of $\mu^{\ast}$, implies

$\sum_{i\in\hat{I}_{1}}\left(\mu_{i}-\hat{\mu}_{i}+v_{i}\right)^{2}=\sum_{i\in\hat{I}_{1}}\mu_{i}^{2}+O\left(m^{2\delta_{1}}\right)+\left|\hat{I}_{1}\right|\gamma_{m}^{2}+O\left(\left|\hat{I}_{1}\right|^{2}\gamma_{m}\right)\text{.}$

Similarly, we can set $\hat{I}_{0}$ such that $\left|\hat{I}_{0}\right|\geq 1$
and

$\sum_{i\in\hat{I}_{0}}\left(\mu_{i}+v_{i}\right)^{2}\leq\sum_{i\in\hat{I}_{0}}\mu_{i}^{2}+\left|\hat{I}_{0}\right|\gamma_{m}^{2}+O\left(\left|\hat{I}_{0}\right|^{2}\gamma_{m}\right)\text{.}$

With the above choice of $\hat{I}_{0}$, $\hat{I}_{1}$ and $a\lambda$, we
obtain

$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|^{2}=\left\|\boldsymbol{\mu}\right\|^{2}+O\left(m^{2\delta_{1}}\right)+O\left(m\gamma_{m}^{2}\right)+O\left(m^{2}\gamma_{m}\right)\text{.}$

By the assumptions on $\boldsymbol{\mu}$ and $\gamma_{m}$, we have
$\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|=O\left(m^{\delta_{1}}\right)$.
The proof is completed.

###  A.2 Proof of Theorem 2

Pick a $\zeta\in\left[-1,1\right]$. Let
$\tilde{s}_{m}\left(t\right)=m^{-1}\sum\limits_{j=1}^{m}\cos\left(t\zeta
v_{j}^{\ast}\right)$,
$\tilde{s}\left(t\right)=\mathbb{E}\left[\tilde{s}_{m}\left(t\right)\right]$
and $d_{E,m}=\tilde{s}_{m}\left(t\right)-\tilde{s}\left(t\right)$. Then,
$\textrm{var}\left[d_{E,m}\right]=I_{1}+I_{2}$ with
$I_{1}=m^{-2}\sum\limits_{i=1}^{m}\textrm{var}\left[\cos\left(t\zeta
v_{j}^{\ast}\right)\right]$ and $I_{2}=2m^{-2}\sum\limits_{1\leq i<j\leq
m}\tilde{c}_{ij}$, where

$\tilde{c}_{ij}=\textrm{cov}\left[\cos\left(t\zeta
v_{i}^{\ast}\right),\cos\left(t\zeta v_{j}^{\ast}\right)\right].$

Define $A_{m}\left(t\right)=\left\\{t:0<t\leq\sqrt{2\gamma\log m}\right\\}$.
Given $\tilde{\epsilon}>0$, our strategy has two steps: (a) derive a general
bound on $\textrm{var}\left[d_{E,m}\right]$ to deduce (19), i.e.

$\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right|\leq
M\tilde{\epsilon}\int_{0}^{1}\exp\left(2^{-1}t^{2}\zeta^{2}a_{\left(1\right)}^{-2}\right)d\zeta;$

(b) with

$t\in A_{m}\left(t\right)\text{ and }\sqrt{2\log
m}=o\left(a_{\left(1\right)}\right)\text{,}$ (A.2)

show $\textrm{var}\left[d_{E,m}\right]=O\left(m^{-\hat{\delta}}\right)$ for
some $0<\hat{\delta}<\delta$, which invokes Theorem 6 of Lyons (1988), a
strong law of large numbers (SLLN) for weakly dependent random variables, to
imply

$\lim\limits_{m\rightarrow\infty}d_{E,m}=0\text{, }\mathbb{P}\text{-a.s.}$
(A.3)

Let $\chi_{i}=\exp\left(\sqrt{-1}t\zeta v_{i}^{\ast}\right)$ for $1\leq i\leq
m.$ Since $v_{i}+v_{j}$ is Normally distributed as
$\mathsf{N}_{1}\left(0,\tilde{a}_{ij}\right)$ with mean $0$ and variance
$\tilde{a}_{ij}=a_{i}^{-2}+a_{j}^{-2}+2c_{ij}$ for $i\neq j$, we have

$\mathbb{E}\left[\chi_{i}\chi_{j}\right]=\exp\left(\sqrt{-1}t\zeta\left(u_{i}+u_{j}\right)\right)\exp\left(-2^{-1}t^{2}\zeta^{2}\tilde{a}_{ij}\right)\text{,}$

where we recall $c_{ij}=\textrm{cov}\left(v_{i},v_{j}\right)$ when $i\neq j$.
Therefore,

$\textrm{cov}\left[\chi_{i},\chi_{j}\right]=\exp\left(\sqrt{-1}t\zeta\left(u_{i}+u_{j}\right)\right)g_{ij}$

with

$g_{ij}=\exp\left(-2^{-1}t^{2}\zeta^{2}\left(a_{i}^{-2}+a_{j}^{-2}\right)\right)\left(\exp\left(-t^{2}\zeta^{2}c_{ij}\right)-1\right)\text{.}$

Similarly, since $v_{i}-v_{j}$ is Normally distributed as
$\mathsf{N}_{1}\left(0,\hat{a}_{ij}\right)$ with
$\hat{a}_{ij}=a_{i}^{-2}+a_{j}^{-2}-2c_{ij}$ for $i\neq j$, we have

$\textrm{cov}\left[\chi_{i},\overline{\chi_{j}}\right]=\exp\left(\sqrt{-1}t\zeta\left(u_{i}-u_{j}\right)\right)\hat{g}_{ij}$

with

$\hat{g}_{ij}=\exp\left(-2^{-1}t^{2}\zeta^{2}\left(a_{i}^{-2}+a_{j}^{-2}\right)\right)\left(\exp\left(t^{2}\zeta^{2}c_{ij}\right)-1\right)\text{,}$

where $\overline{\chi_{j}}$ is the complex conjugate of $\chi_{j}$. Therefore,

$\displaystyle\tilde{c}_{ij}$ $\displaystyle=$ $\displaystyle
2^{-1}\left[\Re\left(\textrm{cov}\left[\chi_{i},\chi_{j}\right]\right)+\Re\left(\textrm{cov}\left[\chi_{i},\overline{\chi_{j}}\right]\right)\right]$
$\displaystyle=$ $\displaystyle
2^{-1}\left\\{\cos\left[t\zeta\left(u_{i}+u_{j}\right)\right]g_{ij}+\cos\left[t\zeta\left(u_{i}-u_{j}\right)\right]\hat{g}_{ij}\right\\}\text{,}$

where $\Re\left(\cdot\right)$ denotes the real part.

From

$\left|\exp\left(\pm t^{2}\zeta^{2}c_{ij}\right)-1\right|\leq
t^{2}\zeta^{2}\left|c_{ij}\right|\exp\left(t^{2}\zeta^{2}\hat{c}_{ij}\right)\leq
t^{2}\zeta^{2}\left|c_{ij}\right|\exp\left(t^{2}\zeta^{2}a_{\left(1\right)}^{-2}\right)$

for some $\hat{c}_{ij}$ that is strictly between $0$ and $\pm c_{ij}$, we see
that, for each fixed $t$ and $\zeta$,

$\left|\tilde{c}_{ij}\right|=\exp\left(-t^{2}\zeta^{2}a_{\left(1\right)}^{-2}\right)\left|\exp\left(-t^{2}\zeta^{2}c_{ij}\right)-1\right|\leq
t^{2}\zeta^{2}\left|c_{ij}\right|\text{.}$

Therefore,

$\left|I_{2}\right|\leq Mt^{2}\zeta^{2}m^{-2}\sum_{1\leq i<j\leq
m}\left|c_{ij}\right|\leq Mt^{2}m^{-\delta}\text{,}$

for some constant $M>0$, where we have used (6) and (8) (i.e., the components
of the minor vector are weakly dependent). On the other hand,
$\cos\left(t\zeta v_{i}^{\ast}\right)=\cos\left(t\zeta\mu_{i}\right)-t\zeta
v_{i}\sin\left(t\zeta\tilde{v}_{i}^{\ast}\right)$ for some
$\tilde{v}_{i}^{\ast}$ strictly between $\mu_{i}$ and $v_{i}^{\ast}$, and

$\textrm{var}\left[\cos\left(t\zeta
v_{i}^{\ast}\right)\right]=t^{2}\zeta^{2}\textrm{var}\left[v_{i}\sin\left(t\zeta\tilde{v}_{i}^{\ast}\right)\right]\leq
t^{2}\zeta^{2}\mathbb{E}\left[v_{i}^{2}\right]=t^{2}\zeta^{2}a_{i}^{-2}\text{.}$

So,

$I_{1}\leq m^{-1}\max\limits_{1\leq i\leq m}\textrm{var}\left[\cos\left(t\zeta
v_{i}^{\ast}\right)\right]=t^{2}a_{\left(1\right)}^{-2}m^{-1}\text{.}$

Hence, $\textrm{var}\left[d_{E,m}\right]\leq
p_{m}\left(t,a_{\left(1\right)}\right)$ and

$\mathbb{P}\left(\left\\{\left|d_{E,n}\right|>\tilde{\varepsilon}\right\\}\right)\leq\dfrac{\textrm{var}\left[d_{E,m}\right]}{\tilde{\varepsilon}^{2}}\leq\dfrac{p_{m}\left(t,a_{\left(1\right)}\right)}{\tilde{\varepsilon}^{2}}\text{,}$

where $p_{m}\left(t,a_{\left(1\right)}\right)$ is defined by (20), i.e.,

$p_{m}\left(t,a_{\left(1\right)}\right)=t^{2}a_{\left(1\right)}^{-2}m^{-1}+Mt^{2}m^{-\delta}.$

From

$\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right|\leq
2\int_{0}^{1}\exp\left(2^{-1}t^{2}\zeta^{2}a_{\left(1\right)}^{-2}\right)\left|\omega\left(\zeta\right)\right|\left|\tilde{s}_{m}\left(t\right)-\tilde{s}\left(t\right)\right|d\zeta\text{,}$

we see that (19), i.e.,

$\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right|\leq
M\tilde{\epsilon}\int_{0}^{1}\exp\left(2^{-1}t^{2}\zeta^{2}a_{\left(1\right)}^{-2}\right)d\zeta;$

holds with probability at least
$1-\tilde{\varepsilon}^{-2}p_{m}\left(t,a_{\left(1\right)}\right)$. Finally,
when (A.2) holds, we have

$\sup\limits_{\left(t,\zeta\right)\in\left[0,\sqrt{2\log{m}}\right]\times\left[-1,1\right]}p_{m}\left(t,a_{\left(1\right)}\right)=O\left(m^{-\hat{\delta}}\right)$

for any $0<\hat{\delta}<\delta$, which implies (A.3). Further, (A.2) implies

$\lim\limits_{m\rightarrow\infty}\sup_{\left(\gamma,\zeta\right)\in\left[0,1\right]\times\left[-1,1\right]}\gamma\zeta^{2}a_{\left(1\right)}^{-2}\log
m=0$

and

$\lim\limits_{m\rightarrow\infty}\int_{0}^{1}\exp\left(\gamma\zeta^{2}a_{\left(1\right)}^{-2}\log
m\right)d\zeta=1\text{,}$

where we have used the dominated convergence theorem (DCT). Thus, (21), i.e.,

$\lim\limits_{m\rightarrow\infty}\sup\limits_{0<t\leq\sqrt{2\gamma\log
m}}\left|\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right|=0$

holds almost surely for any $\gamma\in\left(0,1\right)$. This completes the
proof.

###  A.3 Proof of Theorem 3

Let $I_{1}^{\ast}=\left\\{j:|\mu_{j}|>0\right\\}$. Let $t^{\ast}=$
$\sqrt{2\gamma\log m}$. By the decomposition
$\varphi_{m}\left(t^{\ast};\mathbf{\hat{v}}^{\ast}\right)-\pi=r_{1}+r_{2}+r_{3}$,
where
$r_{1}=\varphi_{m}\left(t^{\ast};\mathbf{\hat{v}}^{\ast}\right)-\varphi_{m}\left(t^{\ast};\mathbf{v}^{\ast}\right),$
$r_{2}=\varphi_{m}\left(t^{\ast};\mathbf{v}^{\ast}\right)-\varphi\left(t^{\ast}\right)$
and $r_{3}=\varphi\left(t^{\ast}\right)-\pi$, we see that, to show
$\varphi_{m}\left(t^{\ast};\mathbf{\hat{v}}^{\ast}\right)-\pi$ converges to
zero in probability, it suffices to show $r_{i}=o_{\mathbb{P}}\left(1\right)$
for $i=1,2,3$.

First,
$\left|t^{\ast}\mu_{j}\right|\geq\left|t^{\ast}\mu_{\ast}\right|\rightarrow\infty$
for $j\in I_{1}^{\ast}$ and
$\lim\limits_{t^{\ast}\rightarrow\infty}\varphi\left(t^{\ast}\right)=\pi$,
which implies $r_{3}=o\left(1\right)$. Secondly, since
$m^{2}\gamma_{m}=O\left(m^{2\delta_{1}}\right)$ for $0<\delta_{1}<2^{-1}$
implies $\sqrt{2\log m}=o\left(a_{\left(1\right)}\right)$, Theorem 2 implies

$|r_{2}|\leq\sup\limits_{0\leq t\leq
t^{\ast}}\left|\pi^{-1}\left[\varphi_{m}\left(t;\mathbf{v}^{\ast}\right)-\varphi\left(t\right)\right]\right|=o_{\mathbb{P}}\left(1\right).$

Therefore, it is left to show $r_{1}=o_{\mathbb{P}}\left(1\right)$.

Clearly,

$|r_{1}|\leq
m^{-1}\sum_{j=1}^{m}\int_{-1}^{1}\left|\omega\left(\zeta\right)\right|\exp\left(2^{-1}{\left(t^{\ast}\zeta
a_{\left(1\right)}^{-1}\right)}^{2}\right)t^{\ast}\left|\zeta\left(v_{j}^{\ast}-\hat{v}_{j}^{\ast}\right)\sin\left(t^{\ast}\zeta\tilde{v}_{j}^{\ast}\right)\right|d\zeta$

for some $\tilde{v}_{j}^{\ast}$ strictly between $v_{j}^{\ast}$ and
$\hat{v}_{j}^{\ast}$. Therefore, for some constant $M>0$,

$|r_{1}|\leq
Mt^{\ast}m^{-1}\sum_{j=1}^{m}\left|v_{j}^{\ast}-\hat{v}_{j}^{\ast}\right|\leq
Mt^{\ast}m^{-1/2}\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|.$

Since the assumptions of Theorem 1 hold with $0<\delta_{1}<2^{-1}$, we have

$t^{\ast}m^{-1/2}\left\|\boldsymbol{\hat{\eta}}-\boldsymbol{\eta}\right\|\leq
Mm^{-1/2}m^{\delta_{1}}\sqrt{2\log m}=o_{\mathbb{P}}\left(1\right).$

Thus, $r_{1}=o_{\mathbb{P}}\left(1\right)$ and the new estimator is
consistent. This completes the proof.

##  Appendix B Simulation results when minimal signal strength is $2$

Figure B.1: The excess $d\left(\hat{\pi}\right)=\hat{\pi}-\pi$ of an estimator $\hat{\pi}$ of the proportion $\pi$ of nonzero Normal means when the minimum of the absolute values of the nonzero Normal means $\mu_{\ast}=2$, where the dashed line marks $\hat{\pi}=\pi$. A better estimator $\hat{\pi}$ of $\pi$ should have non-positive excess and smaller variance and be less downwardly biased. The new estimator “New” is overall the best. | Bias | Std Dev
---|---|---
| Benjamini | New | SLIM | Benjamini | New | SLIM
Block | $\pi=0.1$ | 0.25419 | -0.014123 | 0.0473018 | 0.004791 | 0.011922 | 0.02906
$\pi=0.2$ | 0.17846 | -0.023529 | 0.0104083 | 0.004862 | 0.012706 | 0.02618
$\pi=0.3$ | 0.10534 | -0.031114 | -0.0253431 | 0.004397 | 0.012091 | 0.02369
$\pi=0.4$ | 0.03299 | -0.040947 | -0.0540270 | 0.003688 | 0.011331 | 0.02334
$\pi=0.5$ | -0.04141 | -0.051469 | -0.0796037 | 0.003476 | 0.013769 | 0.01770
Equal Corr. | $\pi=0.1$ | 0.24728 | -0.007783 | 0.0414034 | 0.054264 | 0.005273 | 0.07092
$\pi=0.2$ | 0.16896 | -0.017141 | -0.0024242 | 0.046204 | 0.005294 | 0.06306
$\pi=0.3$ | 0.09378 | -0.026396 | -0.0298384 | 0.036331 | 0.005103 | 0.04441
$\pi=0.4$ | 0.02339 | -0.034508 | -0.0560876 | 0.023223 | 0.005540 | 0.02979
$\pi=0.5$ | -0.04453 | -0.042546 | -0.0805302 | 0.006821 | 0.005016 | 0.01912
Three Factors | $\pi=0.1$ | 0.25377 | -0.007472 | 0.0449588 | 0.012572 | 0.008664 | 0.02873
$\pi=0.2$ | 0.17780 | -0.015670 | 0.0073633 | 0.010953 | 0.008519 | 0.02546
$\pi=0.3$ | 0.10435 | -0.024700 | -0.0247023 | 0.008860 | 0.008035 | 0.02460
$\pi=0.4$ | 0.03239 | -0.033361 | -0.0536669 | 0.005573 | 0.008583 | 0.02071
$\pi=0.5$ | -0.04252 | -0.045129 | -0.0808634 | 0.003026 | 0.008066 | 0.01689
Two Comp. | $\pi=0.1$ | 0.24726 | -0.007399 | 0.0426443 | 0.054458 | 0.004820 | 0.06928
$\pi=0.2$ | 0.16894 | -0.016524 | 0.0006644 | 0.046675 | 0.005052 | 0.05951
$\pi=0.3$ | 0.09401 | -0.025075 | -0.0303219 | 0.036513 | 0.005015 | 0.04602
$\pi=0.4$ | 0.02321 | -0.034048 | -0.0562381 | 0.023041 | 0.004737 | 0.03104
$\pi=0.5$ | -0.04563 | -0.043493 | -0.0845221 | 0.007394 | 0.004882 | 0.01881
Unstructured | $\pi=0.1$ | 0.25261 | 0.078054 | 0.0402548 | 0.022265 | 0.048054 | 0.03827
$\pi=0.2$ | 0.17663 | 0.054929 | 0.0032062 | 0.019681 | 0.039897 | 0.03298
$\pi=0.3$ | 0.10342 | 0.030467 | -0.0279321 | 0.015767 | 0.031933 | 0.03103
$\pi=0.4$ | 0.03094 | 0.006424 | -0.0547324 | 0.010271 | 0.024221 | 0.02262
$\pi=0.5$ | -0.04300 | -0.020007 | -0.0798758 | 0.004244 | 0.014929 | 0.01842
Table B.1: Bias and sample standard deviation (std dev) of estimators of the
proportion $\pi$ of nonzero Normal means when the minimum of the absolute
values of the nonzero Normal means $\mu_{\ast}=2$. A better estimator
$\hat{\pi}$ of $\pi$ should have non-positive bias and smaller variance and be
less downwardly biased. The new estimator “New” is overall the best.

##  Appendix C Simulation results when minimal signal strength is $1.5$

Figure C.1: The excess $d\left(\hat{\pi}\right)=\hat{\pi}-\pi$ of an estimator $\hat{\pi}$ of the proportion $\pi$ of nonzero Normal means when the minimum of the absolute values of the nonzero Normal means $\mu_{\ast}=1.5$, where the dashed line marks $\hat{\pi}=\pi$. A better estimator $\hat{\pi}$ of $\pi$ should have non-positive excess and smaller variance and be less downwardly biased. The new estimator “New” is overall the best.  | Bias | Std Dev
---|---|---
| Benjamini | New | SLIM | Benjamini | New | SLIM
Block | $\pi=0.1$ | 0.251239 | -0.02859 | 0.03061 | 0.005244 | 0.012226 | 0.02780
$\pi=0.2$ | 0.171458 | -0.05259 | -0.02855 | 0.005536 | 0.011961 | 0.02461
$\pi=0.3$ | 0.093704 | -0.07496 | -0.07159 | 0.004831 | 0.012025 | 0.02345
$\pi=0.4$ | 0.015120 | -0.10247 | -0.12107 | 0.004615 | 0.011275 | 0.02177
$\pi=0.5$ | -0.062935 | -0.12464 | -0.16016 | 0.004301 | 0.012714 | 0.01912
Equal Corr. | $\pi=0.1$ | 0.245220 | -0.02272 | 0.03848 | 0.053462 | 0.005501 | 0.06705
$\pi=0.2$ | 0.163943 | -0.04790 | -0.03300 | 0.044930 | 0.005671 | 0.06405
$\pi=0.3$ | 0.084956 | -0.07289 | -0.07837 | 0.035441 | 0.005697 | 0.05238
$\pi=0.4$ | 0.008581 | -0.09584 | -0.12486 | 0.024455 | 0.006716 | 0.04157
$\pi=0.5$ | -0.066525 | -0.11862 | -0.16469 | 0.011959 | 0.006021 | 0.03133
3 Factors | $\pi=0.1$ | 0.250940 | -0.02256 | 0.03108 | 0.012354 | 0.008811 | 0.03082
$\pi=0.2$ | 0.171241 | -0.04626 | -0.02198 | 0.010703 | 0.009120 | 0.02397
$\pi=0.3$ | 0.092923 | -0.07104 | -0.07174 | 0.008347 | 0.008424 | 0.02640
$\pi=0.4$ | 0.015106 | -0.09444 | -0.11993 | 0.006036 | 0.009406 | 0.02097
$\pi=0.5$ | -0.064713 | -0.12332 | -0.16786 | 0.004150 | 0.009297 | 0.02094
Two Comp. | $\pi=0.1$ | 0.245060 | -0.02238 | 0.03654 | 0.053556 | 0.005032 | 0.06732
$\pi=0.2$ | 0.163905 | -0.04662 | -0.03104 | 0.045450 | 0.005361 | 0.06488
$\pi=0.3$ | 0.085247 | -0.07087 | -0.07673 | 0.035645 | 0.005492 | 0.05235
$\pi=0.4$ | 0.008764 | -0.09532 | -0.12412 | 0.023999 | 0.005743 | 0.04048
$\pi=0.5$ | -0.067362 | -0.12043 | -0.16524 | 0.012664 | 0.005727 | 0.02882
Unstructured | $\pi=0.1$ | 0.249922 | 0.06448 | 0.02616 | 0.021811 | 0.048529 | 0.03801
$\pi=0.2$ | 0.170336 | 0.02643 | -0.02451 | 0.019033 | 0.040653 | 0.03948
$\pi=0.3$ | 0.091818 | -0.01258 | -0.07581 | 0.015448 | 0.034280 | 0.03266
$\pi=0.4$ | 0.013838 | -0.05185 | -0.12380 | 0.010985 | 0.027010 | 0.02879
$\pi=0.5$ | -0.065132 | -0.09237 | -0.16962 | 0.005999 | 0.018722 | 0.02375
Table C.1: Bias and sample standard deviation (std dev) of estimators of the
proportion $\pi$ of nonzero Normal means when the minimum of the absolute
values of the nonzero Normal means $\mu_{\ast}=1.5$. A better estimator
$\hat{\pi}$ of $\pi$ should have non-positive bias and smaller variance and be
less downwardly biased. The new estimator “New” is overall the best.

##  Appendix D Simulation results when minimal signal strength is $1$

Figure D.1: The excess $d\left(\hat{\pi}\right)=\hat{\pi}-\pi$ of an estimator $\hat{\pi}$ of the proportion $\pi$ of nonzero Normal means when the minimum of the absolute values of the nonzero Normal means $\mu_{\ast}=1$, where the dashed line marks $\hat{\pi}=\pi$. A better estimator $\hat{\pi}$ of $\pi$ should have non-positive excess and smaller variance and be less downwardly biased. The new estimator “New” is overall the best when $\pi<0.4$. | Bias | Std Dev
---|---|---
| Benjamini | New | SLIM | Benjamini | New | SLIM
Block | $\pi=0.1$ | 0.245957 | -0.04825 | 0.01839 | 0.005497 | 0.011937 | 0.02837
$\pi=0.2$ | 0.161015 | -0.09324 | -0.05349 | 0.005513 | 0.012646 | 0.02498
$\pi=0.3$ | 0.075451 | -0.14118 | -0.13249 | 0.004933 | 0.011847 | 0.02240
$\pi=0.4$ | -0.008615 | -0.18822 | -0.19711 | 0.004885 | 0.011040 | 0.02106
$\pi=0.5$ | -0.093345 | -0.23572 | -0.26009 | 0.004702 | 0.012653 | 0.02117
Equal Corr. | $\pi=0.1$ | 0.241180 | -0.04626 | 0.14697 | 0.053735 | 0.005582 | 0.26926
$\pi=0.2$ | 0.155209 | -0.09529 | -0.05549 | 0.046329 | 0.006044 | 0.06793
$\pi=0.3$ | 0.071163 | -0.13860 | -0.12985 | 0.038140 | 0.006346 | 0.07068
$\pi=0.4$ | -0.012609 | -0.18545 | -0.19737 | 0.030060 | 0.007522 | 0.06367
$\pi=0.5$ | -0.096940 | -0.23512 | -0.26692 | 0.021829 | 0.007367 | 0.06070
Three Factors | $\pi=0.1$ | 0.246505 | -0.04425 | 0.01510 | 0.012261 | 0.009987 | 0.02951
$\pi=0.2$ | 0.160753 | -0.09302 | -0.05686 | 0.010697 | 0.009969 | 0.02676
$\pi=0.3$ | 0.075914 | -0.13882 | -0.12332 | 0.008874 | 0.009748 | 0.02715
$\pi=0.4$ | -0.008183 | -0.18409 | -0.19474 | 0.007228 | 0.010166 | 0.02765
$\pi=0.5$ | -0.093632 | -0.23435 | -0.26177 | 0.005357 | 0.010227 | 0.02490
Two Comp. | $\pi=0.1$ | 0.240983 | -0.04578 | 0.13871 | 0.053916 | 0.005523 | 0.27153
$\pi=0.2$ | 0.155600 | -0.09186 | -0.05598 | 0.046797 | 0.006392 | 0.07209
$\pi=0.3$ | 0.070082 | -0.14165 | -0.13441 | 0.039092 | 0.006409 | 0.07113
$\pi=0.4$ | -0.012546 | -0.18511 | -0.20071 | 0.029788 | 0.006578 | 0.06477
$\pi=0.5$ | -0.095580 | -0.23234 | -0.26579 | 0.021121 | 0.006424 | 0.06042
Unstructured | $\pi=0.1$ | 0.245087 | 0.04575 | 0.02019 | 0.021652 | 0.051253 | 0.03633
$\pi=0.2$ | 0.159096 | -0.01542 | -0.05924 | 0.018245 | 0.044311 | 0.03805
$\pi=0.3$ | 0.075125 | -0.07051 | -0.12931 | 0.016123 | 0.038958 | 0.03538
$\pi=0.4$ | -0.009509 | -0.12753 | -0.19460 | 0.012958 | 0.033182 | 0.03458
$\pi=0.5$ | -0.094581 | -0.18777 | -0.26273 | 0.009914 | 0.027927 | 0.03491
Table D.1: Bias and sample standard deviation (std dev) of estimators of the
proportion $\pi$ of nonzero Normal means when the minimum of the absolute
values of the nonzero Normal means $\mu_{\ast}=1$. A better estimator
$\hat{\pi}$ of $\pi$ should have non-positive bias and smaller variance and be
less downwardly biased. The new estimator “New” is overall the best when
$\pi<0.4$.

##  Appendix E Simulation results when minimal signal strength is $0.45$

Figure E.1: The excess $d\left(\hat{\pi}\right)=\hat{\pi}-\pi$ of an estimator $\hat{\pi}$ of the proportion $\pi$ of nonzero Normal means when the minimum of the absolute values of the nonzero Normal means $\mu_{\ast}=0.45$, where the dashed line marks $\hat{\pi}=\pi$. A better estimator $\hat{\pi}$ of $\pi$ should have non-positive excess and smaller variance and be less downwardly biased. The new estimator “New” is overall the best when $\pi<0.4$. | Bias | Std Dev
---|---|---
| Benjamini | New | SLIM | Benjamini | New | SLIM
Block | $\pi=0.1$ | 0.23945 | -0.07003 | 0.13999 | 0.005094 | 0.012275 | 0.30106
$\pi=0.2$ | 0.14752 | -0.14233 | -0.06759 | 0.005283 | 0.012476 | 0.12139
$\pi=0.3$ | 0.05458 | -0.21564 | -0.17694 | 0.005034 | 0.012507 | 0.02591
$\pi=0.4$ | -0.03794 | -0.29012 | -0.26412 | 0.005127 | 0.013514 | 0.02565
$\pi=0.5$ | -0.12977 | -0.36424 | -0.35232 | 0.005528 | 0.013013 | 0.02554
Equal Corr. | $\pi=0.1$ | 0.23544 | -0.07268 | 0.53766 | 0.055891 | 0.005364 | 0.38548
$\pi=0.2$ | 0.14254 | -0.14847 | 0.33706 | 0.051612 | 0.005972 | 0.39554
$\pi=0.3$ | 0.05141 | -0.21878 | 0.14399 | 0.046523 | 0.006069 | 0.38112
$\pi=0.4$ | -0.04019 | -0.29192 | -0.02156 | 0.042268 | 0.007171 | 0.34644
$\pi=0.5$ | -0.13267 | -0.36826 | -0.16599 | 0.037492 | 0.007023 | 0.33717
3 Factors | $\pi=0.1$ | 0.23971 | -0.07115 | 0.08490 | 0.012650 | 0.010405 | 0.24650
$\pi=0.2$ | 0.14685 | -0.14689 | -0.06656 | 0.011602 | 0.010402 | 0.12239
$\pi=0.3$ | 0.05498 | -0.21895 | -0.16891 | 0.010737 | 0.010130 | 0.03146
$\pi=0.4$ | -0.03675 | -0.29099 | -0.25970 | 0.010095 | 0.010581 | 0.02957
$\pi=0.5$ | -0.13000 | -0.36777 | -0.35140 | 0.008433 | 0.010582 | 0.02901
Two Comp. | $\pi=0.1$ | 0.23528 | -0.07252 | 0.52016 | 0.056066 | 0.005478 | 0.38460
$\pi=0.2$ | 0.14364 | -0.14537 | 0.32067 | 0.051706 | 0.006208 | 0.39345
$\pi=0.3$ | 0.05079 | -0.22168 | 0.12641 | 0.047270 | 0.006179 | 0.38413
$\pi=0.4$ | -0.04020 | -0.29189 | -0.06215 | 0.041698 | 0.006677 | 0.33270
$\pi=0.5$ | -0.13161 | -0.36584 | -0.17868 | 0.037143 | 0.006294 | 0.31426
Unstructured | $\pi=0.1$ | 0.23877 | 0.02449 | 0.26522 | 0.022070 | 0.054233 | 0.36265
$\pi=0.2$ | 0.14544 | -0.05801 | 0.05228 | 0.020112 | 0.049378 | 0.29151
$\pi=0.3$ | 0.05424 | -0.13505 | -0.14387 | 0.019006 | 0.047700 | 0.14959
$\pi=0.4$ | -0.03867 | -0.21312 | -0.25001 | 0.017243 | 0.044206 | 0.09389
$\pi=0.5$ | -0.13083 | -0.29451 | -0.33146 | 0.015898 | 0.042353 | 0.11841
Table E.1: Bias and sample standard deviation (std dev) of estimators of the
proportion $\pi$ of nonzero Normal means when the minimum of the absolute
values of the nonzero Normal means $\mu_{\ast}=0.45$. A better estimator
$\hat{\pi}$ of $\pi$ should have non-positive bias and smaller variance and be
less downwardly biased. The new estimator “New” is overall the best when
$\pi<0.4$.