# Asymptotic Independence of the Sum and Maximum of Dependent Random Variables
with Applications to High-Dimensional Tests

Long Feng
School of Statistics and Data Science, LPMC & KLMDASR, Nankai University
and
Tiefeng Jiang
School of Statistics, University of Minnesota
and
Xiaoyun Li
Cognitive Computing Lab, Baidu Research
and
Binghui Liu
School of Mathematics and Statistics & KLAS, Northeast Normal University
Corresponding author.

###### Abstract

For a set of dependent random variables, without stationary or the strong
mixing assumptions, we derive the asymptotic independence between their sums
and maxima. Then we apply this result to high-dimensional testing problems,
where we combine the sum-type and max-type tests and propose a novel test
procedure for the one-sample mean test, the two-sample mean test and the
regression coefficient test in high-dimensional setting. Based on the
asymptotic independence between sums and maxima, the asymptotic distributions
of test statistics are established. Simulation studies show that our proposed
tests have good performance regardless of data being sparse or not. Examples
on real data are also presented to demonstrate the advantages of our proposed
methods.

Keywords: Asymptotic normality; Asymptotic independence; Extreme-value
distribution; High-dimensional tests; Large $p$ and small $n$

##  1 Introduction

Statistical independence is a very simple structure and is convenient in
statistical inference and applications. In this paper, we study the asymptotic
independence between two common statistics: the extreme-value statistic
$M_{p}=\max_{1\leq i\leq p}X_{i}$ and the sum $S_{p}=\sum_{i=1}^{p}X_{i}$,
where $\\{X_{i}\\}_{i=1}^{p}$ is a sequence of dependent random variables.
This theoretical results will be applied to three high-dimensional testing
problems with numerical examples.

###  1.1 Independence Between Sum and Maximum

In the past few decades, great efforts have been devoted in understanding the
asymptotic joint distribution of $M_{p}$ and $S_{p}$. In an early research,
Chow and Teugels (1978) established the asymptotic independence between
$M_{p}$ and $S_{p}$ for independent and identically distributed random
variables. To overcome the limitation of the required assumptions, Anderson
and Turkman (1991), Anderson and Turkman (1993), Anderson and Turkman (1995)
and Hsing (1995) generalized the asymptotic result to the case that
$\\{X_{i}\\}_{i=1}^{p}$ is strong mixing; for the concept “strong mixing” and
its properties, see, for example, the survey paper Bradley (2005) and the
literature therein. In particular, Hsing (1995) showed that for a stationary
sequence, strong mixing property and asymptotic normality of $S_{p}$ are
basically enough to guarantee the asymptotic independence of the sum and
maximum. However, it is shown in Davis and Hsing (1995) that in the case of
infinite variance, $M_{p}$ and $S_{p}$ are not asymptotically independent
because the asymptotic behavior of $S_{p}$ is dominated by that of the extreme
order statistic. In addition, Ho and Hsing (1996), Ho and McCormick (1999),
McCormick and Qi (2000) and Peng and Nadarajah (2003) considered the joint
limit distribution of the maximum and sum of stationary Gaussian sequence
$\\{X_{i}\\}_{i=1}^{p}$ in which $E(X_{i})=0$, $\mbox{Var}(X_{i})=1$ and
$r(p)=E(X_{i}X_{i+p})$. Under different conditions on $r(p)$, the joint
limiting distributions of maxima and sums are different. Specifically, Ho and
Hsing (1996) showed that $M_{p}$ and $S_{p}$ are asymptotically independent as
long as $\lim_{p\to\infty}r(p)\log p=0$; the two statistics are not
independent provided $\lim_{p\to\infty}r(p)\log p=\rho\in(0,\infty)$. For the
rest situations, by assuming $\lim_{p\to\infty}\frac{\log
p}{p}\sum_{i=1}^{p}|r(i)-r(p)|=0$, Ho and McCormick (1999) and McCormick and
Qi (2000) obtained the asymptotic independence of $M_{p}-(S_{p}/n)$ and
$S_{p}$.

All these results are based on the stationary assumption that the covariance
structure among $\\{X_{i}\\}_{i=1}^{p}$ has the property that
$E(X_{i}X_{i+h})=E(X_{1}X_{1+h})$ for each integer $h$ and $i=1,\cdots,p-h$.
This is a common assumption in research, however, it is not easy to be
checked. Even though it can be verified by hypothesis testing, the stationary
property still may not hold up to certain statistical errors. In fact, in many
scenarios this assumption is not true. For example, for stock data of US S&P
500 index in which stock returns are considered as variables, if stocks are
ordered alphabetically by names, the two stocks, such as AAPL and MSFT, may
have both far distance and strong correlation, which does not satisfy the
stationary assumption.

In this work, we study the asymptotic independence between
$\tilde{S}_{p}=\sum_{i=1}^{p}Z_{i}^{2}$ and $\tilde{M}_{p}=\max_{1\leq i\leq
p}Z_{i}^{2}$ without stationary assumption. In our case each $Z_{i}$ is
marginally $N(0,1)$ and the covariance matrix of $Z_{i}$’s, denoted by
$\mathbb{\Sigma}_{p}=(\sigma_{ij})_{1\leq i,j\leq p}$, satisfies certain
conditions. Specifically, we first establish the asymptotically normality of
$\tilde{S}_{p}$ if
$[\mbox{tr}(\mathbb{\Sigma}_{p}^{2+\delta})]^{2}\cdot[\mbox{tr}(\mathbb{\Sigma}_{p}^{2})]^{-2-\delta}\to
0$ for some $\delta>0$. Then, we show that the limit distribution of the
maximum $\tilde{M}_{p}-2\log p+\log\log p$ is a Gumbel distribution under
conditions on the covariance matrix $\mathbb{\Sigma}_{p}$. Finally, we prove
the asymptotic independence between $\tilde{S}_{p}$ and $\tilde{M}_{p}$ under
the conditions $\max_{1\leq i<j\leq p}|\sigma_{ij}|\leq\varrho$ and
$\max_{1\leq i\leq p}\sum_{j=1}^{p}\sigma_{ij}^{2}\leq(\log p)^{C}$, together
with two additional conditions on the maximum and minimum eigenvalues of
${\boldsymbol{\Sigma}}_{p}$. These theoretical results are novel and
essentially different from the existing ones in which the stationary property
is required. Since these results are universal, they may provide many useful
implications. In this paper, we will apply the above asymptotic independence
results to three high-dimensional hypothesis testing problems: one-sample mean
test, two-sample mean test and the regression coefficient test.

###  1.2 High-Dimensional Hypothesis Testing

High-dimensional hypothesis testing is an important research area in modern
statistics. It has been frequently used in many application fields, such as
genomics, medical imaging, risk management and web search. The motivation of
studying high-dimensional test is that traditional tests, such as the
Hotelling $T$-squared test, do not work in general when the data dimension is
larger than the sample size due to the singularity of sample covariance
matrix. A nature way to amend this problem is replacing the sample covariance
matrix appearing in the Hotelling $T$-squared test statistic with a
nonsingular matrix, such as the identity matrix and the diagonal matrix of
sample covariance matrix. In this way, for example, Srivastava (2009), Park
and Ayyala (2013), Wang et al. (2015), Feng et al. (2016), Feng et al. (2015)
and Feng et al. (2017) developed tests for one-sample mean problem, while Bai
and Saranadasa (1996), Srivastava and Du (2008), Chen and Qin (2010), and
Gregory et al. (2015) developed tests for two-sample mean problem. In
addition, Goeman et al. (2006) and Lan et al. (2014), for instance, considered
testing regression coefficients in high-dimensional linear models. All these
tests are sum-type tests, based on the summation of parameter estimators. It
is well known that the sum-type tests generally perform well as data are
dense, i.e. most of the parameters are nonzero under the local alternative.
However, it may be inefficient when data are sparse, where only a few
parameters are nonzero under local alternative. To establish high-dimensional
tests for sparse data, Cai et al. (2014), Zhong et al. (2013) and Chen et al.
(2019) proposed some max-type tests, which typically perform well on sparse
data, but worse when the data become dense.

In practice, it is often difficult to determine whether data are sparse or
not. Thus, many efforts have been devoted to develop tests with good and
robust performance under both data conditions. For example, Fan et al. (2015)
proposed a power enhancement procedure by a screening technique for high-
dimensional tests. They combined the power enhancement component with an
asymptotically pivotal statistic to strengthen powers under sparse
alternatives. Xu et al. (2016) initiated an adaptive test for high-dimensional
two-sample mean test. It combines information across a class of sum-of-powers
tests, including tests based on the sum-of-squares of the mean differences and
the supremum mean difference. Wu et al. (2019) extended the adaptive test to
generalized linear models. In He et al. (2021), the authors constructed
$U$-statistics of different orders that are asymptotically independent of the
max-type test statistics in high-dimensional tests, upon which an adaptive
testing procedure is proposed. However, these results are based on Hsing
(1995), which require data to be sampled from stationary and $\alpha$-mixing
random variables. In fact, the $\alpha$-mixing property is hardly checked in
practice, which greatly limits the application of these methods. In this
paper, by using the novel asymptotic independence analysis between the sum and
maximum aforementioned, we solve the problem without the stationary assumption
or the $\alpha$-mixing property and propose a series of high-dimensional tests
including one-sample mean test, two-sample mean test and the regression
coefficient test. Numerical results demonstrate strong robustness of the
proposed tests regardless data being sparse or not.

The main contributions of this paper are listed as follows. (1) We establish
the asymptotic distribution of the maximum of dependent Gaussian random
variables under a general assumption. (2) We prove the asymptotic independence
between the sum and maximum of dependent Gaussian random variables without the
stationary or the $\alpha$-mixing property. (3) We propose three high-
dimensional combo-type tests based on the above asymptotic properties. They
are one-sample mean test, two-sample mean test and the regression coefficient
test. Numerical examples on simulated and real-world data demonstrate strong
robustness of our tests, on both sparse and dense datasets.

The rest of the paper is organized as follows. In Section 2, we state our
theoretical results, including the asymptotic distributions of the sum and
maximum statistics, and the asymptotic independence between them. In Section
3, we propose a series of tests for high-dimensional data based on these
theoretical results. Then, we demonstrate the simulation results of the
proposed tests in comparison with some existing ones in Section 4, followed by
two applications in Section 5. Finally, we present some concluding remarks in
Section 6, while providing some extended results and technical proofs in the
supplementary material.

##  2 Asymptotic Independence of Sum and Maximum of Dependent Random Variables

First, in this section, we study the asymptotic normality of the sum of
dependent random variables. For each $p\geq 2$, let $Z_{p1},\cdots,Z_{pp}$ be
$N(0,1)$-distributed random variables with $p\times p$ covariance matrix
$\mathbb{\Sigma}_{p}$. If there is no danger of confusion, we simply write
“$Z_{1},\cdots,Z_{p}$” for “$Z_{p1},\cdots,Z_{pp}$” and “$\mathbb{\Sigma}$”
for “$\mathbb{\Sigma}_{p}$”. The following assumption is needed:

$\displaystyle\lim_{p\to\infty}\frac{[\mbox{tr}(\mathbb{\Sigma}^{2+\delta})]^{2}}{[\mbox{tr}(\mathbb{\Sigma}^{2})]^{2+\delta}}=0\
\ \mbox{for some}\ \delta>0.$ (1)

Assumption (1) with $\delta=2$ is the same as condition (3.7) in Chen and Qin
(2010), and here we make it more general. Although in applications the true
covariance matrix $\boldsymbol{\Sigma}$ is usually unknown, this condition
assures the practitioners that our results would be applicable to a wide range
of problems. For instance, if all eigenvalues of ${\boldsymbol{\Sigma}}$ are
bounded above and are bounded below from zero, it is trivial to see that (1)
holds.

######  THEOREM 1

Under Assumption (1),
$\frac{Z_{1}^{2}+\cdots+Z_{p}^{2}-p}{\sqrt{2\mbox{tr}(\mathbb{\Sigma}^{2})}}\to
N(0,1)$ in distribution as $p\to\infty.$

Theorem 1 shows that the sum of squares of the dependent Gaussian random
variables has the asymptotic normality if the covariance matrix satisfies
Assumption (1).

Next, for the same Gaussian random variables, we consider the asymptotic
distribution of $\max_{1\leq i\leq p}Z_{i}^{2}$. The following assumption will
be imposed:

$\displaystyle\mbox{Let }{\boldsymbol{\Sigma}}=(\sigma_{ij})_{1\leq i,j\leq
p}.\mbox{ For some }\varrho\in(0,1),\mbox{ assume
}|\sigma_{ij}|\leq\varrho\mbox{ for all }1\leq i<j\leq p\mbox{ and }$
$\displaystyle p\geq 2.\mbox{ Suppose }\\{\delta_{p};\,p\geq 1\\}\mbox{ and
}\\{\kappa_{p};\,p\geq 1\\}\mbox{ are positive constants with
}\delta_{p}=o(1/\log p)$ $\displaystyle\mbox{and }\kappa=\kappa_{p}\to 0\mbox{
as }p\to\infty.\mbox{ For }1\leq i\leq p,\mbox{ define }B_{p,i}=\big{\\{}1\leq
j\leq p;\,|\sigma_{ij}|\geq\delta_{p}\big{\\}}$ $\displaystyle\mbox{and}\
C_{p}=\big{\\{}1\leq i\leq p;\,|B_{p,i}|\geq p^{\kappa}\big{\\}}.\mbox{ We
assume that }|C_{p}|/p\to 0\ \mbox{as}\ p\to\infty.\ \ \ \ \ \ \ $ (2)

######  THEOREM 2

Suppose Assumption (2) holds. Then $\max_{1\leq i\leq p}Z_{i}^{2}-2\log
p+\log\log p$ converges to a Gumbel distribution with cdf
$F(x)=\exp\\{-\frac{1}{\sqrt{\pi}}e^{-x/2}\\}$ as $p\to\infty$.

######  REMARK 1

Cai et al. (2014) obtained the above limiting distribution of $\max_{1\leq
i\leq p}Z_{i}^{2}$ under the assumption that $\max_{1\leq i\leq
p}\sum_{j=1}^{p}\sigma_{ij}^{2}\leq C_{0}$ for each $p\geq 1$, where $C_{0}$
is a constant free of $p$. In the following we will see that their result is a
special case of Theorem 2. In fact, let $\delta_{p}=(\log p)^{-2}$ for $p\geq
e^{e}$, then for each $1\leq i\leq p$,
$\delta_{p}^{2}\cdot|B_{p,i}|\leq\sum_{j=1}^{p}\sigma_{ij}^{2}\leq C_{0}$.
Hence, $|B_{p,i}|\leq C_{0}\cdot(\log p)^{2}<p^{\kappa}$ where
$\kappa=\kappa_{p}:=5(\log\log p)/\log p$ for large $p$. As a result,
$|C_{p}|=0$, which implies the results of Theorem 2.

A closely related but not exactly the same result by Fan and Jiang (2019)
shows that $\delta_{p}=o(1/\log p)$ in Assumption (2) can not be relaxed.
Their statistic is $\max_{1\leq i\leq p}Z_{i}$ in contrast to $\max_{1\leq
i\leq p}|Z_{i}|$ here. We expect that $\delta_{p}=o(1/\log p)$ is also the
critical threshold for $\max_{1\leq i\leq p}|Z_{i}|$.

Theorem 2 is proved by using the spirit of the proof of Lemma 6 from Cai et
al. (2014). There are two purposes to derive the result. First, the conditions
imposed in our theorem is weaker than those required in Lemma 6 from Cai et
al. (2014), which has been discussed in Remark 1. This allows us to apply this
type of results to a more general covariance matrix $\mathbb{\Sigma}$.
Secondly, part of the steps in the proof of Theorem 2 will also be used in the
proof of Theorem 3 stated next.

To proceed, we need more notations and an additional assumption. For two
sequences of numbers $\\{a_{p}\geq 0;\,p\geq 1\\}$ and $\\{b_{p}>0;\,p\geq
1\\}$, we write $a_{p}\ll b_{p}$ if $\lim_{p\to\infty}\frac{a_{p}}{b_{p}}=0.$
The following assumption will be used:

$\displaystyle\mbox{There exist }C>0\mbox{ and }\varrho\in(0,1)\mbox{ so that
}\max_{1\leq i<j\leq p}|\sigma_{ij}|\leq\varrho\mbox{ and }\max_{1\leq i\leq
p}\sum_{j=1}^{p}\sigma_{ij}^{2}\leq(\log p)^{C}$ $\displaystyle\mbox{for all}\
p\geq 3;\ p^{-1/2}(\log
p)^{C}\ll\lambda_{min}(\mathbb{\Sigma})\leq\lambda_{max}(\mathbb{\Sigma})\ll\sqrt{p}(\log
p)^{-1}\ \mbox{and}\ $
$\displaystyle\lambda_{max}(\mathbb{\Sigma})/\lambda_{min}(\mathbb{\Sigma})=O(p^{\tau})\
\mbox{for some}\ \tau\in(0,1/4).$ (3)

Assumption (2) is actually stronger than both (1) and (2). To see this, assume
(2) holds now. To derive (1), observe that
$\mbox{tr}(\mathbb{\Sigma}^{2+\delta})\leq
p\cdot\lambda_{max}(\mathbb{\Sigma})^{2+\delta}$ and
$\mbox{tr}(\mathbb{\Sigma}^{2})\geq p\cdot\lambda_{min}(\mathbb{\Sigma})^{2}$.
Then
$\frac{[\mbox{tr}(\mathbb{\Sigma}^{2+\delta})]^{2}}{[\mbox{tr}(\mathbb{\Sigma}^{2})]^{2+\delta}}\leq\frac{1}{p^{\delta}}\cdot\Big{(}\frac{\lambda_{max}(\mathbb{\Sigma})}{\lambda_{min}(\mathbb{\Sigma})}\Big{)}^{4+2\delta}=O\Big{(}\frac{1}{p^{\delta-(4+2\delta)\tau}}\Big{)}\to
0$ by choosing $\delta=2$ and using the assumption $\tau\in(0,1/4)$ stated in
(2). We then get (1) with $\delta=2$. To deduce (2), we replace “$C_{0}$” in
Remark 1 with “$(\log p)^{C}$”. By the same argument as that in Remark 1 and
choosing $\delta_{p}=(\log p)^{-2}$, we see $|B_{p,i}|\leq C_{0}\cdot(\log
p)^{C+2}<p^{\kappa},$ where $\kappa=\kappa_{p}:=(C+3)(\log\log p)/\log p$ for
$p\geq e^{e}$. Hence, $|C_{p}|=0$ and Assumption (2) holds.

######  THEOREM 3

Under Assumption (2), the following holds:
$\frac{Z_{1}^{2}+\cdots+Z_{p}^{2}-p}{\sqrt{2\mbox{tr}(\mathbb{\Sigma}^{2})}}$
and $\max_{1\leq i\leq p}Z_{i}^{2}-2\log p+\log\log p$ are asymptotically
independent as $p\to\infty.$

Importantly, notice that the above asymptotic independence result holds
without the stationary assumption or the $\alpha$-mixing condition. Regarding
the assumption on the spectrum, in high-dimensional statistics literature, it
is common to assume
$[\lambda_{min}(\mathbb{\Sigma}),\lambda_{max}(\mathbb{\Sigma})]\subset[a,b],$
with $0<a<b<\infty$. Note that this is stronger than our assumption on the
eigenvalues of $\mathbb{\Sigma}$ in (2). In fact, Assumption (2) allows that
the largest eigenvalue goes to infinity and the smallest eigenvalue goes to
zero. Thus, Theorem 3 provides more general result and more freedom and
practicality in application.

##  3 Application: High-Dimensional Testing Problems

In this section, we will apply the theoretical results derived in Section 2 to
three high-dimensional testing problems: one-sample mean test, two-sample mean
test and the regression coefficient test. The first and third tests will be
presented in the following two subsections, while two-sample mean test will be
presented in the supplementary material.

###  3.1 One-Sample Mean Test

Assume ${\boldsymbol{X}}_{1},\cdots,{\boldsymbol{X}}_{n}$ are independent and
identically distributed $p$-dimensional random vectors from
$N({\boldsymbol{\mu}},{\boldsymbol{\Sigma}})$. The classical one-sample mean
testing problem considers

$\displaystyle
H_{0}:{\boldsymbol{\mu}}=\boldsymbol{0}~{}~{}\textrm{versus}~{}~{}H_{1}:{\boldsymbol{\mu}}\not=\boldsymbol{0}.$
(4)

In the traditional setting where $p$ is fixed, this topic is covered in
classic textbooks on multivariate analysis such as in Anderson (2003), Eaton
(1983) and Muirhead (1982). Starting from this century, a tremendous effort
has been made for the test towards the high-dimensional setting, where both
$n$ and $p$ go to infinity. In the following we will highlight part of these
work en route to a problem we are interested in: the test (4) under the
situation $n\leq p$. This is a typical problem of interest in high-dimensional
statistics with small $n$ and large $p$.

Let $\bar{{\boldsymbol{X}}}=\frac{1}{n}\sum_{i=1}^{n}\mathbb{X}_{i}$ and
$\hat{{\bf
S}}=\frac{1}{n}\sum_{i=1}^{n}({\boldsymbol{X}}_{i}-\bar{{\boldsymbol{X}}})({\boldsymbol{X}}_{i}-\bar{{\boldsymbol{X}}})^{T}$
be the sample mean and the sample covariance matrix of
${\boldsymbol{X}}_{1},\cdots,{\boldsymbol{X}}_{n}$, respectively. The
Hotelling $T^{2}$-statistic is defined by
$n\bar{{\boldsymbol{X}}}^{T}\hat{{\bf S}}^{-1}\bar{{\boldsymbol{X}}}$; see
Hotelling (1931). For the case with $n>p$, Bai and Saranadasa (1996) studied
the Hotelling statistic. When $n\leq p$, however, the matrix $\hat{{\bf S}}$
is no longer invertible, which motivates the design of new statistics. By
replacing $\hat{{\bf S}}$ with its diagonal matrix in the Hotelling
$T^{2}$-statistic, Srivastava and Du (2008) and Srivastava (2009) proposed a
scale-invariant test for (4), defined by

$\displaystyle T^{(1)}_{sum}=\frac{n\bar{{\boldsymbol{X}}}^{T}\hat{{\bf
D}}^{-1}\bar{{\boldsymbol{X}}}-(n-1)p/(n-3)}{\sqrt{2[\mathrm{tr}(\hat{{\bf
R}}^{2})-p^{2}/(n-1)]}},$ (5)

where $\hat{{\bf D}}$ is the diagonal matrix of the sample covariance matrix
$\hat{{\bf S}}$, and $\hat{{\bf R}}=\hat{{\bf D}}^{-1/2}\hat{{\bf S}}\hat{{\bf
D}}^{-1/2}$ is the sample correlation matrix. The major ingredient of
$T^{(1)}_{sum}$ can be written as a sum of random variables, so we sometimes
call it a “sum-type” statistic. In general, the performance of sum-type
statistics are not ideal in sparse cases when only a few entries in
$\boldsymbol{\mu}$ in the sum are non-zero; see Cai et al. (2014) for more
detailed discussion. Zhong et al. (2013) proposed two alternative tests by
first thresholding two statistics based on the sample means and then
maximizing over a range of thresholding levels. Denote
$\bar{{\boldsymbol{X}}}=(\bar{{\boldsymbol{X}}}_{1},\cdots,\bar{{\boldsymbol{X}}}_{p})^{T}$.
The $L_{2}$-version of the thresholding statistic is

$\displaystyle
T_{HC2}=\max_{s\in\mathcal{S}}\frac{T_{2n}(s)-\hat{\mu}(s)}{\hat{\sigma}(s)},$
(6)

where $\mathcal{S}$ is a subset of the interval $(0,1)$,

$\displaystyle T_{2n}(s)$
$\displaystyle=\sum_{j=1}^{p}n\left(\bar{{\boldsymbol{X}}}_{j}/\sigma_{j}\right)^{2}I\left(|\bar{{\boldsymbol{X}}}_{j}|\geq\sigma_{j}\sqrt{\lambda_{s}/n}\right),$
$\displaystyle\hat{\mu}(s)$
$\displaystyle=p\left\\{2\lambda_{p}^{1/2}(s)\phi(\lambda_{p}^{1/2}(s))+2\bar{\Phi}(\lambda_{p}^{1/2}(s))\right\\},$
$\displaystyle\hat{\sigma}^{2}(s)$
$\displaystyle=p\left\\{2\left[\lambda_{p}^{3/2}(s)+3\lambda_{p}^{1/2}(s)\right]\phi(\lambda_{p}^{1/2}(s))+6\bar{\Phi}(\lambda_{p}^{1/2}(s))\right\\}.$

Here $\lambda_{s}(p)=2s\log p$, and $\phi(\cdot)$, $\bar{\Phi}(\cdot)$ are the
density and survival functions of the standard normal distribution,
respectively. Fan et al. (2015) proposed a novel procedure by adding a power
enhancement component which is asymptotically zero under the null and diverges
under some specific regions of alternatives. Their test statistic is

$\displaystyle J=J_{0}+J_{1},$ (7)

where the power enhancement component $J_{0}$ is
$J_{0}=\sqrt{p}\sum_{j=1}^{p}\bar{{\boldsymbol{X}}}_{j}^{2}\hat{\sigma}_{j}^{-2}I(|\bar{{\boldsymbol{X}}}_{j}|>\hat{\sigma}_{j}\delta_{p,n}),$
and $J_{1}$ is the standard Wald statistic
$J_{1}=\frac{\bar{{\boldsymbol{X}}}^{T}\widehat{\mathrm{var}}^{-1}(\hat{{\boldsymbol{X}}})\bar{{\boldsymbol{X}}}-p}{2\sqrt{p}}.$
Here $\hat{\sigma}_{j}^{2}$ is the sample variance of the $j$th coordinate of
the population vector, $\delta_{p,n}$ is a thresholding parameter and
$\widehat{\mathrm{var}}^{-1}(\hat{{\boldsymbol{X}}})$ is a consistent
estimator of the asymptotic inverse covariance matrix of
$\bar{{\boldsymbol{X}}}$. However, the power enhancement component would be
negligible if the signal is not very strong. As we mentioned before, Cai et
al. (2014) showed that extreme-value statistics are particularly powerful
against sparse alternatives and possess certain optimal properties. Hence, we
propose a statistic by compromising the sum-type statistic from (5) and an
extreme-value statistic, based on our results in Section 2, which will be
compared with aforementioned baselines numerically in Section 4.1. As will be
confirmed later, our method performs very well regardless of the sparsity of
the alternative hypothesis.

We now formally introduce our approach. Define

$\displaystyle T^{(1)}_{max}=n\cdot\max_{1\leq i\leq
p}\frac{\bar{{\boldsymbol{X}}}_{i}^{2}}{\hat{\sigma}_{ii}^{2}},$ (8)

where $\bar{{\boldsymbol{X}}}_{i}$ is the $i$th coordinate of
$\bar{{\boldsymbol{X}}}=\frac{1}{n}(\mathbb{X}_{1}+\cdots\mathbb{X}_{n})\in\mathbb{R}^{p}$
and $\hat{\sigma}_{ii}^{2}$ is the sample variance of the $i$th coordinate of
the population vector, that is, if we write
$\mathbb{X}_{j}=(x_{1j},\cdots,x_{pj})^{T}$ for each $1\leq j\leq n$, then
$\hat{\sigma}_{ii}^{2}$ is the sample variance of the i.i.d. random variables
$x_{i1},x_{i2},\cdots,x_{in}.$ Firstly, the asymptotic distribution of
$T^{(1)}_{max}$ will be presented which needs more notations. Let ${\bf
R}={\bf D}^{-1/2}{\boldsymbol{\Sigma}}{\bf D}^{-1/2}=(\rho_{ij})_{1\leq
i,j\leq p}$ denote the population correlation matrix, where ${\bf D}$ is the
diagonal matrix of ${\boldsymbol{\Sigma}}$. The following assumption will be
imposed:

$\displaystyle\mbox{There exists}\ \epsilon\in\Big{(}\frac{1}{2},1\Big{]}\
\mbox{and}\ K>1\ \mbox{such that}\ K^{-1}p^{\epsilon}\leq n\leq Kp^{\epsilon}\
\mbox{and}$ $\displaystyle\sup_{p\geq 2}\frac{1}{p}\mathrm{tr}({\bf
R}^{i})<\infty\,\mbox{ for }i=2,3,4.$ (9)

Note that (3.1) is the same as assumptions (3.1) and (3.2) from Srivastava
(2009). If the eigenvalues of the correlation matrix ${\bf R}$ are bounded,
the second condition of (3.1) will hold automatically. For rigor of
mathematics, we assume $n$ depends on $p$ and sometimes write $n_{p}$ when
there is a possible confusion.

######  THEOREM 4

Under the null hypothesis in (4), the following holds as $p\to\infty$:

  * (i)

If (3.1) holds, then $T^{(1)}_{sum}\to N(0,1)$ in distribution;

  * (ii)

If (2) holds with “$\mathbb{\Sigma}$” replaced by “$\mathbb{R}$” and $\log
p=o(n^{1/3})$, then $T^{(1)}_{max}-2\log p+\log\log p$ converges weakly to a
Gumbel distribution with cdf $F(x)=\exp\\{-\frac{1}{\sqrt{\pi}}\exp(-x/2)\\}$;

  * (iii)

Assume (3.1) is true. If (2) holds with “$\mathbb{\Sigma}$” replaced by
“$\mathbb{R}$”, then $T^{(1)}_{sum}$ and $T^{(1)}_{max}-2\log p+\log\log p$
are asymptotically independent.

Part (i) of the above theorem is from Srivastava (2009), which is also a
corollary of the recent work by Jiang and Li (2021). For the sum-type test, a
level-$\alpha$ test will be performed through rejecting $H_{0}$ when
$T^{(1)}_{sum}$ is larger than the $(1-\alpha)$-quantile
$z_{\alpha}=\Phi^{-1}(1-\alpha)$ where $\Phi(y)$ is the cdf of $N(0,1)$. For
the max-type test, a level-$\alpha$ test will then be performed through
rejecting $H_{0}$ when $T^{(1)}_{max}-2\log p+\log\log p$ is larger than the
$(1-\alpha)$-quantile $q_{\alpha}=-\log\pi-2\log\log(1-\alpha)^{-1}$ of the
Gumbel distribution $F(x)$.

Based on Theorem 4, we propose a combo-type test statistic by combining the
max-type and the sum-type tests. It is defined by

$\displaystyle T_{com}^{(1)}=\min\\{P^{(1)}_{S},P^{(1)}_{M}\\},$ (10)

where $P^{(1)}_{S}=1-\Phi\left\\{T^{(1)}_{sum}\right\\}$ and
$P^{(1)}_{M}=1-F(T^{(1)}_{max}-2\log p+\log\log p).$ Note that $P^{(1)}_{S}$
and $P^{(1)}_{M}$ are the $p$-values for the tests by using statistics
$T^{(1)}_{sum}$ and $T^{(1)}_{max}$, separately, and $T_{com}^{(1)}$ is
defined by the smaller one, whose asymptotic distribution can be characterized
by the minimum of two standard uniform random variables.

######  COROLLARY 1

Assume the conditions in Theorem 4(iii) hold. Then $T_{com}^{(1)}$ from (10)
converges weakly to a distribution with density $G(w)=2(1-w)I(0\leq w\leq 1)$
as $p\to\infty.$

According to Corollary 1, the proposed combo-type test allows us to perform a
level-$\alpha$ test by rejecting the null hypothesis when
$T_{com}^{(1)}<1-\sqrt{1-\alpha}\approx\frac{\alpha}{2}$ as $\alpha$ is small.
We now discuss the power functions. First, the power function of our combo-
type test is

$\displaystyle\beta^{(1)}_{C}({\boldsymbol{\mu}},\alpha)$ $\displaystyle=$
$\displaystyle
P\left(T_{com}^{(1)}<1-\sqrt{1-\alpha}\right)=P\left(P^{(1)}_{M}<1-\sqrt{1-\alpha}~{}or~{}P^{(1)}_{S}<1-\sqrt{1-\alpha}\right)$
(11) $\displaystyle\geq$
$\displaystyle\max\left\\{P\left(P^{(1)}_{S}<1-\sqrt{1-\alpha}\right),P\left(P^{(1)}_{M}<1-\sqrt{1-\alpha}\right)\right\\}$
$\displaystyle\approx$
$\displaystyle\max\left\\{\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha/2),\beta^{(1)}_{M}({\boldsymbol{\mu}},\alpha/2)\right\\}$

when $\alpha$ is small, where $\beta^{(1)}_{M}({\boldsymbol{\mu}},\alpha)$ and
$\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha)$ are the power functions of
$T_{max}^{(1)}$ and $T_{sum}^{(1)}$ with significant level $\alpha$,
respectively. From Srivastava (2009), the power function of $T_{sum}^{(1)}$ is

$\displaystyle\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha)=\lim_{p\to\infty}\Phi\left(-z_{\alpha}+\frac{n{\boldsymbol{\mu}}^{T}{\bf
D}^{-1}{\boldsymbol{\mu}}}{\sqrt{2\mathrm{tr}({\bf R}^{2})}}\right),$ (12)

where $z_{\alpha}=\Phi^{-1}(1-\alpha)$ is the $(1-\alpha)$-quantile of
$N(0,1)$. Due to (11), we have
$\beta^{(1)}_{C}({\boldsymbol{\mu}},\alpha)\geq\lim_{p\to\infty}\Phi\left(-z_{\alpha/2}+\frac{n{\boldsymbol{\mu}}^{T}{\bf
D}^{-1}{\boldsymbol{\mu}}}{\sqrt{2\mathrm{tr}({\bf R}^{2})}}\right)$. Denote
${\bf D}=\mathrm{diag}(\sigma_{11}^{2},\cdots,\sigma_{pp}^{2})$. By the same
argument at that from Theorem 2 in Cai et al. (2014), the asymptotic power of
$T_{max}^{(1)}$ converges to one if $\max_{1\leq i\leq
p}|\mu_{i}/{\sigma_{ii}}|\geq c\sqrt{\log p/n}$ for a certain constant $c$,
and also the nonzero $\mu_{i}$ are randomly uniformly sampled with sparsity
level $\gamma<1/4$, i.e., the number of nonzero $\mu_{i}$ is less than
$p^{\gamma},\gamma<1/4$. Thus, according to (11), the power function of our
proposed test $T_{com}^{(1)}$ also converges to one in this case. Similarly,
according to Theorem 3 in Cai et al. (2014), the condition $\max_{1\leq i\leq
p}|\mu_{i}/{\sigma_{ii}}|\geq c\sqrt{\log p/n}$ is minimax rate optimal for
testing against sparse alternatives. If $c$ is sufficiently small, then any
$\alpha$-level test is unable to reject the null hypothesis with probability
tending to one. It is shown in Cai et al. (2014) that $T_{max}^{(1)}$ enjoys a
certain optimality against sparse alternatives. By (11), our test
$T_{com}^{(1)}$ also has this optimality.

In order to get a rough picture of the asymptotic power comparison between
$T_{sum}^{(1)},T_{max}^{(1)}$ and $T_{com}^{(1)}$, now we simply assume that
${\boldsymbol{\Sigma}}={\bf I}_{p}$. There are $m$ nonzeros $\mu_{i}$ and they
are all equal to $\delta\neq 0$. Equation (12) gives
$\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha)=\lim_{p\to\infty}\Phi\left(-z_{\alpha}+\frac{nm\delta^{2}}{\sqrt{2p}}\right)$.

We consider two special cases:

  * (1)

Dense case: $\delta=O(n^{-\xi})$ and $m=O(p^{1/2}n^{2\xi-1})$ with
$\xi\in(1/2,5/6]$. We also assume $\log p=o(n^{\xi-\frac{1}{2}})$, hence $\log
p=o(n^{1/3})$. As a consequence, the requirement on $p$ vs $n$ imposed in
Theorem 4(ii) is fulfilled. Obviously, the number of nonzero $\mu_{i}$ goes to
infinity. The power function for $T_{max}^{(1)}$ is given by
$\beta^{(1)}_{M}({\boldsymbol{\mu}},\alpha)=P\left(T_{max}^{(1)}-2\log
p+2\log\log p>q_{\alpha}\right)$. In this case, we will show in Section S2.4
of the supplementary material that
$\beta^{(1)}_{M}({\boldsymbol{\mu}},\alpha)\approx\alpha$, which means that
$T_{max}^{(1)}$ is not effective or useful. Consequently, we have
$\beta^{(1)}_{C}({\boldsymbol{\mu}},\alpha)\approx\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha/2)$.
When the significant level $\alpha$ is small, the difference between
$\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha)$ and
$\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha/2)$ is negligible. So our proposed
test $T_{com}^{(1)}$ has similar performance as $T_{sum}^{(1)}$ in this dense
case.

  * (2)

Sparse case: $\delta=c\sqrt{\log p/n}$ for sufficient large constant $c$ and
$m=o((\log p)^{-1}p^{1/2})$. Here the value of $m$ is much smaller than that
in (1) and hence confirms the notion of “sparse”. In this case,
$\frac{nm\delta^{2}}{\sqrt{2p}}\to 0$, so
$\beta^{(1)}_{S}({\boldsymbol{\mu}},\alpha)\approx\alpha$ and $T_{sum}^{(1)}$
is not effective or useful. Yet,
$\beta^{(1)}_{M}({\boldsymbol{\mu}},\alpha)\to 1$ by an argument similar to
Theorem 2 from Cai et al. (2014) as discussed above, which also leads to
$\beta^{(1)}_{C}({\boldsymbol{\mu}},\alpha)\to 1$ in this sparse case.
Additionally, in Fan et al. (2015) the quantity $\delta_{p,n}$ is chosen to be
$\log\log n\sqrt{\log p/n}$, which implies that the screening set
$\\{i:\sqrt{n}|\bar{{\boldsymbol{X}}}_{i}|>\log\log n\sqrt{\log p}\\}$ would
be empty as probability tending to one. Thus, the power enhancement component
of Fan et al. (2015) would be negligible in this case, which makes the
standardized Wald test statistic the same as $T_{sum}^{(1)}$ since
${\boldsymbol{\Sigma}}={\bf I}_{p}$. That is, their test is also ineffective
in this sparse case.

The above theoretical results and analysis, together with the simulation in
the next section, indicate that our proposed test $T_{com}^{(1)}$ performs
very well regardless of the sparsity of the alternative hypothesis, which is
more convenient to use in various practical scenarios.

Due to space limitation, we will present the combo-type two-sample mean test
as well as its simulation results in the supplementary material.

###  3.2 Test for Regression Coefficients

Then, we apply our theory to the testing problem in high-dimensional
regression. Let
${\boldsymbol{X}}=({\boldsymbol{X}}_{1},\cdots,{\boldsymbol{X}}_{n})^{T}$ be
independent and identically distributed $p$-dimensional covariates, and
${\boldsymbol{Y}}=(Y_{1},\cdots,Y_{n})^{T}$ be the corresponding independent
responses. For simplicity, we assume $E{\boldsymbol{X}}_{i}=0$, $\forall i$.
We introduce a decomposition of sample point ${\boldsymbol{X}}_{i}$ as
${\boldsymbol{X}}_{i}=({\boldsymbol{X}}_{ia}^{T},{\boldsymbol{X}}_{ib}^{T})^{T}$
with ${\boldsymbol{X}}_{ia}=(X_{i1},\cdots,X_{iq})^{T}\in\mathbb{R}^{q}$ and
${\boldsymbol{X}}_{ib}=(X_{i(q+1)},\cdots,X_{ip})^{T}\in\mathbb{R}^{p-q}$,
where $q$ is smaller than the sample size $n$, and $n$ is much smaller than
$p$. We consider the following standard linear regression model:

$\displaystyle
Y_{i}={\boldsymbol{X}}_{i}^{T}\boldsymbol{\beta}+\varepsilon_{i}={\boldsymbol{X}}_{ia}^{T}\boldsymbol{\beta}_{a}+{\boldsymbol{X}}_{ib}^{T}\boldsymbol{\beta}_{b}+\varepsilon_{i},$
(13)

where
$\boldsymbol{\beta}=(\boldsymbol{\beta}_{a}^{T},\boldsymbol{\beta}_{b}^{T})^{T}\in\mathbb{R}^{p}$,
$\boldsymbol{\beta}_{a}\in\mathbb{R}^{q}$ and
$\boldsymbol{\beta}_{b}\in\mathbb{R}^{p-q}$ are the regression coefficient
vectors. The random noises $\\{\varepsilon_{i};\,1\leq i\leq n\\}$ are
independent with $E\varepsilon_{i}=0$ and
$\mbox{Var}(\varepsilon_{i})=\sigma^{2}$ for each $i$, and are also
independent of the data ${\boldsymbol{X}}$. In this section, we consider the
following testing problem:

$\displaystyle
H_{0}:\boldsymbol{\beta}_{b}=0\mbox{~{}~{}vs.~{}~{}}H_{1}:\boldsymbol{\beta}_{b}\not=0$
(14)

under the situation that $p$ is much larger than $q$ and the sample size $n$
is small. When
$({\boldsymbol{X}}_{1a}^{T},\cdots,{\boldsymbol{X}}_{na}^{T})^{T}$ is a null
vector, (14) is equivalent to test $H_{0}:\boldsymbol{\beta}=0$ vs.
$H_{1}:\boldsymbol{\beta}\not=0$.

For this problem, Goeman et al. (2006) and Goeman et al. (2011) proposed an
empirical Bayes test. It is formulated via a score test on the hyper parameter
of a prior distribution on the regression coefficients. By excluding the
inverse term in the classical $F$-statistic, Zhong and Chen (2011) proposed a
$U$-statistic to extend their results in factorial designs.

Denote $\boldsymbol{\beta}=(\beta_{1},\cdots,\beta_{p})^{T}$. To motivate our
test procedure, let us consider a set of related but not exactly the same test
as follows:

$\displaystyle H_{0j}:\beta_{j}=0\mbox{~{}~{}vs.~{}~{}}H_{1j}:\beta_{j}\not=0$
(15)

for each $j=q+1,\cdots,p$. Some notations are needed before we proceed. For
the set of features included in the index set of “$a$”, let
${\boldsymbol{X}}_{a}=({\boldsymbol{X}}_{1a},\cdots,{\boldsymbol{X}}_{na})^{T},\
\ {\bf H}_{a}={\bf X}_{a}({\bf X}_{a}^{T}{\bf X}_{a})^{-1}{\bf X}_{a}^{T}$ and
$\tilde{\bf X}_{j}=(\mathbb{I}_{n}-{\bf H}_{a})(X_{1j},\cdots,X_{nj})^{T}$,
for each $j$. Notice ${\boldsymbol{X}}_{a}$ is $n\times q$, ${\bf H}_{a}$ is
$n\times n$ and $\tilde{\bf X}_{j}\in\mathbb{R}^{n}$. For the “$b$” part, we
define

$\displaystyle{\boldsymbol{X}}_{b}=({\boldsymbol{X}}_{1b},\cdots,{\boldsymbol{X}}_{nb})^{T}\
\ \ \mbox{and}\ \ \ \tilde{\bf X}_{b}=(\tilde{\bf X}_{q+1},\cdots,\tilde{\bf
X}_{p})=(\mathbb{I}_{n}-{\bf
H}_{a}){\boldsymbol{X}}_{b};~{}\hat{{\boldsymbol{\Sigma}}}_{b|a}=n^{-1}\tilde{\bf
X}_{b}^{T}\tilde{\bf X}_{b},$ (16)

where both ${\boldsymbol{X}}_{b}$ and $\tilde{\bf X}_{b}$ are $n\times(p-q)$
and $\hat{{\boldsymbol{\Sigma}}}_{b|a}$ is $(p-q)\times(p-q)$. Regarding the
response vector, the residual vector and the sample variance, we denote
$\hat{{\boldsymbol{\varepsilon}}}=({\bf I}_{n}-{\bf H}_{a}){\boldsymbol{Y}}$
and
$\hat{\sigma}^{2}=(n-q)^{-1}\hat{{\boldsymbol{\varepsilon}}}^{T}\hat{{\boldsymbol{\varepsilon}}}$.
For each test in (15), the classical partial F-test is given by
${F}_{j}=\frac{\boldsymbol{Y}^{T}{\tilde{\bf X}_{j}}(\tilde{\bf
X}_{j}^{T}\tilde{\bf X}_{j})^{-1}\tilde{\bf
X}_{j}^{T}\boldsymbol{Y}}{\boldsymbol{Y}^{T}[{\bf I}_{n}-{\bf X}({\bf
X}^{T}{\bf X})^{-1}{\bf X}^{T}]\boldsymbol{Y}/(n-p)}$. However, as $p>n$, the
statistic $F_{j}$ becomes problematic since ${\bf X}^{T}{\bf X}$ is not
invertible. To overcome this issue, we replace the denominator of $F_{j}$ by
$\boldsymbol{Y}^{T}({\bf I}_{n}-{\bf H}_{a})\boldsymbol{Y}/(n-q)$. Under the
null hypothesis, $\hat{\sigma}^{2}$ is an unbiased estimator of $\sigma^{2}$;
see (S.154) in the supplementary material. Thus, the test statistic for (15)
becomes $\tilde{F}_{j}=\frac{\boldsymbol{Y}^{T}\tilde{\bf X}_{j}(\tilde{\bf
X}_{j}^{T}\tilde{\bf X}_{j})^{-1}\tilde{\bf
X}_{j}^{T}\boldsymbol{Y}}{\boldsymbol{Y}^{T}({\bf I}_{n}-{\bf
H}_{a})\boldsymbol{Y}/(n-q)}$. Back to the testing problem (14) of interest,
we will handle it by combining $\tilde{F}_{j}$ together. There are two
classical ways to synthesize them. The first one is the sum-type test
statistic: $S_{F}=\sum_{j=q+1}^{p}\tilde{F}_{j}$. Obviously, we have
$S_{F}=T_{1}/\hat{\sigma}^{2}$ with
$T_{1}=\sum_{j=q+1}^{p}\boldsymbol{Y}^{T}\tilde{\bf X}_{j}(\tilde{\bf
X}_{j}^{T}\tilde{\bf X}_{j})^{-1}\tilde{\bf X}_{j}^{T}\boldsymbol{Y}$. By
normalization, without loss of generality, assume $\tilde{\bf
X}_{j}^{T}\tilde{\bf X}_{j}=n$ for each $j=q+1,\cdots,p.$ Then
$T_{1}=\frac{1}{n}\sum_{j=q+1}^{p}\boldsymbol{Y}^{T}\tilde{\bf
X}_{j}\tilde{\bf
X}_{j}^{T}\boldsymbol{Y}=\frac{1}{n}\hat{\boldsymbol{\varepsilon}}^{T}{\boldsymbol{X}}_{b}{\boldsymbol{X}}_{b}^{T}\hat{{\boldsymbol{\varepsilon}}}$.
By standardizing $T_{1}$ with estimators of the mean and standard deviation of
$T_{1}$, we propose the sum-type statistic

$\displaystyle
T^{(3)}_{sum}=\frac{n^{-1}\hat{\boldsymbol{\varepsilon}}^{T}{\boldsymbol{X}}_{b}{\boldsymbol{X}}_{b}^{T}\hat{{\boldsymbol{\varepsilon}}}-n^{-1}(n-q)(p-q)\hat{\sigma}^{2}}{\sqrt{2\hat{\sigma}^{4}\,\widehat{\mbox{tr}\big{(}{\boldsymbol{\Sigma}}_{b|a}^{2}\big{)}}}},$
(17)

where the denominator is
$\widehat{\mathrm{tr}({\boldsymbol{\Sigma}}^{2}_{b|a})}=\frac{n^{2}}{(n+1-q)(n-q)}\Big{\\{}\mathrm{tr}\big{(}\hat{{\boldsymbol{\Sigma}}}_{b|a}^{2}\big{)}-\frac{1}{n-q}\mathrm{tr}^{2}(\hat{{\boldsymbol{\Sigma}}}_{b|a})\Big{\\}}$.
The second statistic we are interested in is of the max-type, defined by

$\displaystyle T^{(3)}_{max}=\max_{q+1\leq j\leq p}\tilde{F}_{j}.$ (18)

Then, we will study the asymptotic distributions of $T^{(3)}_{sum}$ and
$T^{(3)}_{max}$, as well as their asymptotic independence, based on which a
combo-type test will be proposed. To begin with, we introduce some additional
assumptions that will be used.

Define
${\boldsymbol{\Sigma}}_{b|a}=E[\mbox{Cov}({\boldsymbol{X}}_{ib}|{\boldsymbol{X}}_{ia})]=(\sigma_{jk}^{*})$
as a $(p-q)\times(p-q)$ matrix. Without loss of generality, we assume that
${\boldsymbol{\Sigma}}_{b|a}$ is normalized such that its diagonal entries are
equal to one, i.e., $\sigma_{jj}^{*}=1$ for each $j$. Next, we introduce
moment conditions on a conditional predictor. For each $i=1,\cdots,n$, by
regressing ${\boldsymbol{X}}_{ib}$ on ${\boldsymbol{X}}_{ia}$, the residual
vector is given by ${\boldsymbol{X}}_{ib}^{*}={\boldsymbol{X}}_{ib}-{\bf
B}{\boldsymbol{X}}_{ia}\in\mathbb{R}^{p-q}$, where ${\bf
B}:=\mbox{Cov}({\boldsymbol{X}}_{ib},{\boldsymbol{X}}_{ia})\cdot[\mbox{Cov}({\boldsymbol{X}}_{ia})]^{-1}$
is a $(p-q)\times q$ matrix. By the previous assumption
$E{\boldsymbol{X}}_{i}=0$, we immediately have that
$E({\boldsymbol{X}}_{ib}^{*})=0$ and
$\mbox{Cov}({\boldsymbol{X}}_{ib}^{*})={\boldsymbol{\Sigma}}_{b|a}=(\sigma_{ij}^{*})$.
Define ${\bf
X}_{b}^{*}=({\boldsymbol{X}}_{1b}^{*},\cdots,{\boldsymbol{X}}_{nb}^{*})^{T}$,
which is a $n\times(p-q)$ matrix. The conditions we will use later on are
stated below.

$\displaystyle{\boldsymbol{X}}_{ib}^{*}\sim
N(\mathbb{0},{\boldsymbol{\Sigma}}_{b|a})\ \mbox{and the diagonal entries of
}{\boldsymbol{\Sigma}}_{b|a}\ \mbox{are all}\ \mbox{equal to}\ 1;$ (19)
$\displaystyle\mbox{There exists a constant}\ \tau>1\ \mbox{such that}\
\tau^{-1}<\lambda_{min}({\boldsymbol{\Sigma}}_{b|a})\leq\lambda_{max}({\boldsymbol{\Sigma}}_{b|a})<\tau.$
(20)

Assumption (20) is the same as condition (C1) from Lan et al. (2014), which is
also a common assumption in literature for research on high-dimensional data;
see, for example, Fan et al. (2008), Rothman et al. (2008), Zhang and Huang
(2008) and Wang (2009). We assume that both $n$ and $q$ depend on $p$ and
limits will be taken as $p\to\infty.$ The random errors
$\varepsilon_{1},\cdots,\varepsilon_{n}$ are assumed to be i.i.d. with
$E\varepsilon_{i}=0$ and $\mbox{Var}(\varepsilon_{i})=\sigma^{2}$ for each
$i$, and no Gaussian assumption is needed for $\varepsilon_{i}$. Our main
result for high-dimensional regression coefficient test is as follows.

######  THEOREM 5

Assume (19) and (20) hold and (2) also holds with “$\mathbb{\Sigma}$” replaced
by “${\boldsymbol{\Sigma}}_{b|a}$”. Suppose $p=o(n^{3})$, $q=o(p)$, $q\leq
n^{\delta}$ for some $\delta\in(0,1)$ and $E(|\epsilon_{1}|^{\ell})<\infty$
with $\ell=14(1-\delta)^{-1}$. Under $H_{0}$ from (14), as $p\to\infty$ we
have:

  * (i)

$T^{(3)}_{sum}\to N(0,1)$ in distribution;

  * (ii)

$T^{(3)}_{max}-2\log(p-q)+\log\log(p-q)$ converges weakly to a distribution
with cdf $F(x)=\exp\\{-\frac{1}{\sqrt{\pi}}\exp(-\frac{x}{2})\\}$,
$x\in\mathbb{R}$;

  * (iii)

$T^{(3)}_{sum}$ and $T^{(3)}_{max}-2\log(p-q)+\log\log(p-q)$ are
asymptotically independent.

It is known from Lan et al. (2014) that
$\frac{n^{-1}\hat{\boldsymbol{\varepsilon}}^{T}{\boldsymbol{X}}_{b}{\boldsymbol{X}}_{b}^{T}\hat{{\boldsymbol{\varepsilon}}}-\hat{\sigma}^{2}n^{-1}(p-q)\mathrm{tr}({\bf
M}({\bf I}_{n}-{\bf
H}_{a}))}{\sqrt{2\sigma^{4}\,\mbox{tr}\big{(}{\boldsymbol{\Sigma}}_{b|a}^{2}\big{)}}}$
converges to $N(0,1)$ in distribution, with ${\bf
M}:=(p-q)^{-1}\sum_{j=q+1}^{p}\tilde{\bf X}_{j}\tilde{\bf X}_{j}^{T}$.
Although this is not a statistic (since unknown parameters appear in the
denominator), it describes the asymptotic behavior of
$\hat{\boldsymbol{\varepsilon}}^{T}{\boldsymbol{X}}_{b}{\boldsymbol{X}}_{b}^{T}\hat{{\boldsymbol{\varepsilon}}}$.
The numerator of our statistic $T^{(3)}_{sum}$ in (17) is simpler, because no
computation of $\mathrm{tr}({\bf M}({\bf I}_{n}-{\bf H}_{a}))$ is needed.

We now study the implications of Theorem 5 and discuss the rejection rules.
For the sum-type test, a level-$\alpha$ test will be performed through
rejecting $H_{0}$ when $T^{(3)}_{sum}$ is larger than the
$(1-\alpha)$-quantile $z_{\alpha}=\Phi^{-1}(1-\alpha)$ of $N(0,1)$. For the
max-type test, a level-$\alpha$ test will then be performed through rejecting
$H_{0}$ when $T^{(3)}_{max}-2\log(p-q)+\log\log(p-q)$ is larger than the
$(1-\alpha)$-quantile $q_{\alpha}=-\log\pi-2\log\log(1-\alpha)^{-1}$ of
$F(x)$.

Analogously, a combined test is defined through

$\displaystyle T_{com}^{(3)}=\min\big{\\{}P^{(3)}_{S},P^{(3)}_{M}\big{\\}},$
(21)

where $P^{(3)}_{S}=1-\Phi(T_{sum}^{(3)})$ and
$P^{(3)}_{M}=1-F(T_{max}^{(3)}-2\log(p-q)+\log\log(p-q))$ with
$F(x)=\exp\\{-\frac{1}{\sqrt{\pi}}\exp\left(-\frac{x}{2}\right)\\}.$ Similar
to Corollary 1, the proposed combo-type test allows us to perform a
level-$\alpha$ test by rejecting the null hypothesis when
$T_{com}^{(3)}<1-\sqrt{1-\alpha}\approx\frac{\alpha}{2}$.

Again, using the same argument as (11), the power function of our combo-type
test $\beta_{C}^{(3)}({\boldsymbol{\mu}},\alpha)$ is larger than
$\max\\{\beta_{M}^{(3)}({\boldsymbol{\mu}},\alpha),\beta_{S}^{(3)}({\boldsymbol{\mu}},\alpha)\\}$,
where $\beta^{(3)}_{M}({\boldsymbol{\mu}},\alpha)$ and
$\beta^{(3)}_{S}({\boldsymbol{\mu}},\alpha)$ are the power functions of
$T_{max}^{(3)}$ and $T_{sum}^{(3)}$ at significant level $\alpha$,
respectively. To demonstrate the power of the tests, assume the simple case
where ${\boldsymbol{X}}_{a}$ is null vector and
$\mbox{Cov}({\boldsymbol{X}}_{b})={\bf I}_{p}$. From Zhong and Chen (2011),
the power function of $T_{sum}^{(3)}$ is
$\beta_{S}^{(3)}(\boldsymbol{\beta},\alpha)=\lim_{p\to\infty}\Phi\left(-z_{\alpha}+\frac{n\boldsymbol{\beta}^{T}\boldsymbol{\beta}}{\sqrt{2p}\sigma^{2}}\right)$.
In addition, assuming $\boldsymbol{\beta}$ only contains $m$ non-zeros all
equal to $\delta\not=0$, leading to
$\beta_{S}^{(3)}(\boldsymbol{\beta},\alpha)=\lim_{p\to\infty}\Phi\left(-z_{\alpha}+\frac{nm\delta^{2}}{\sqrt{2p}\sigma^{2}}\right)$.
For the non-sparse case with $\delta=O(n^{-\xi})$ and
$m=O(p^{1/2}n^{2\xi-1})$, we have
$\beta_{M}^{(3)}({\boldsymbol{\mu}},\alpha)\approx\alpha$ and
$\beta_{C}^{(3)}({\boldsymbol{\mu}},\alpha)\approx\beta_{S}^{(3)}({\boldsymbol{\mu}},\alpha/2)$.
For the sparse case where $\delta=c\sqrt{\log p/n}$ with sufficient large $c$
and $m=o((\log p)^{-1}p^{1/2})$, we have
$\beta_{S}^{(3)}({\boldsymbol{\mu}},\alpha)\approx\alpha$ and
$\beta_{C}^{(3)}({\boldsymbol{\mu}},\alpha)\approx\beta_{M}^{(3)}({\boldsymbol{\mu}},\alpha/2)\to
1$. Again, in this testing problem for high-dimensional regression, the
combined test statistics also exhibits good performance under both sparse and
dense alternative hypotheses.

##  4 Simulation Results

In this section, we carry out a series of simulation study on the testing
problems studied in the previous section, to compare different test statistics
and validate the advantage of the proposed combo-type tests.

###  4.1 One-Sample Test Problem

Firstly, we conduct numerical examples on the one-sample test problem. We
compare our combo-type test $T_{com}$ in (10) (abbreviated as COM) with the
sum-type test $T^{(1)}_{sum}$ in (5) by Srivastava (2009) (abbreviated as
SUM), the max-type test $T^{(1)}_{max}$ in (8) (abbreviated as MAX), the
Higher Criticism test $T_{HC2}$ from (6) by Zhong et al. (2013) (abbreviated
as HC2) and the power enhancement test $J$ from (7) by Fan et al. (2015)
(abbreviated as FLY). The dataset is simulated as follows.

EXAMPLE 1. We consider
${\boldsymbol{X}}_{i}={\boldsymbol{\mu}}+{\boldsymbol{\Sigma}}^{1/2}{\boldsymbol{z}}_{i}$
for $i=1\cdots,n$, and each component of ${\boldsymbol{z}}_{i}$ is
independently generated from three distributions: (1) the normal distribution
$N(0,1)$; (2) the t distribution $t(3)/\sqrt{3}$; (3) the mixture normal
random variable $V/\sqrt{1.8}$, where $V\ \mbox{has density function}\
0.1f_{1}(x)+0.9f_{2}(x)$ with $f_{1}(x)$ and $f_{2}(x)$ being the densities of
$N(0,9)$ and $N(0,1)$, respectively. We will work on two different sample
sizes with $n=100,200$ and three different dimensions with $p=200,400,600$.
Under the null hypothesis, we set ${\boldsymbol{\mu}}=\boldsymbol{0}$ and the
significance level $\alpha=0.05$. The following three scenarios of covariance
matrices will be considered.

  * (I)

AR(1) model: ${\boldsymbol{\Sigma}}=(0.5^{|i-j|})_{1\leq i,j\leq p}$.

  * (II)

${\boldsymbol{\Sigma}}={\bf D}^{1/2}{\bf R}{\bf D}^{1/2}$ with ${\bf
D}=\mathrm{diag}(\sigma_{1}^{2},\cdots,\sigma_{p}^{2})$ and ${\bf R}={\bf
I}_{p}+\boldsymbol{b}\boldsymbol{b}^{T}-\check{{\bf B}}$, where
$\sigma_{i}^{2}$ are generated independently from $Uniform(1,2)$,
$\boldsymbol{b}=(b_{1},\cdots,b_{p})^{T}$ and $\check{{\bf
B}}=\mathrm{diag}(b_{1}^{2},\cdots,b_{p}^{2})$. The first $[p^{0.3}]$ entries
of ${\boldsymbol{b}}$ are independently sampled from $Uniform(0.7,0.9)$, and
the remaining entries are set to be zero, where $[\cdot]$ denotes taking
integer part.

  * (III)

${\boldsymbol{\Sigma}}=\boldsymbol{\gamma}\boldsymbol{\gamma}^{T}+({\bf
I}_{p}-\rho_{\epsilon}{\boldsymbol{W}})^{-1}({\bf
I}_{p}-\rho_{\epsilon}{\boldsymbol{W}}^{T})^{-1},$ where
$\boldsymbol{\gamma}=(\gamma_{1},\cdots,\gamma_{[p^{\delta_{\gamma}}]},0,0,\cdots,0)^{T}$.
Here $\gamma_{i}$ with $i=1,\cdots,[p^{\delta_{\gamma}}]$ are generated
independently from $Uniform(0.7,0.9)$. Let $\rho_{\epsilon}=0.5$ and
$\delta_{\gamma}=0.3$. Let ${\boldsymbol{W}}=(w_{i_{1}i_{2}})_{1\leq
i_{1},i_{2}\leq p}$ have a so-called rook form, i.e., all elements of
${\boldsymbol{W}}$ are zero except that
$w_{i_{1}+1,i_{1}}=w_{i_{2}-1,i_{2}}=0.5$ for $i_{1}=1,\cdots,p-2$ and
$i_{2}=3,\cdots,p$, and $w_{1,2}=w_{p,p-1}=1$.

Table 1:  Sizes of tests for Example 1 with Scenario (I), $\alpha=0.05$. | Distribution | (1) | (2) | (3)
---|---|---|---|---
| $p$ | 200 | 400 | 600 | 200 | 400 | 600 | 200 | 400 | 600
$n=100$ | MAX | 0.053 | 0.062 | 0.082 | 0.026 | 0.052 | 0.045 | 0.044 | 0.039 | 0.061
| SUM | 0.064 | 0.064 | 0.060 | 0.052 | 0.050 | 0.059 | 0.063 | 0.058 | 0.064
| COM | 0.063 | 0.069 | 0.059 | 0.040 | 0.059 | 0.055 | 0.056 | 0.047 | 0.061
| HC2 | 0.028 | 0.044 | 0.034 | 0.033 | 0.029 | 0.032 | 0.038 | 0.025 | 0.044
| FLY | 0.014 | 0.009 | 0.004 | 0.003 | 0.003 | 0.002 | 0.025 | 0.018 | 0.014
$n=200$ | MAX | 0.046 | 0.060 | 0.049 | 0.045 | 0.041 | 0.045 | 0.042 | 0.045 | 0.032
| SUM | 0.065 | 0.068 | 0.058 | 0.053 | 0.057 | 0.062 | 0.056 | 0.054 | 0.056
| COM | 0.056 | 0.068 | 0.048 | 0.042 | 0.047 | 0.052 | 0.043 | 0.050 | 0.039
| HC2 | 0.019 | 0.027 | 0.030 | 0.031 | 0.024 | 0.023 | 0.029 | 0.020 | 0.029
| FLY | 0.005 | 0.000 | 0.000 | 0.003 | 0.000 | 0.000 | 0.017 | 0.012 | 0.005
Table 2:  Sizes of tests for Example 1 with Scenario (II), $\alpha=0.05$. | Distribution | (1) | (2) | (3)
---|---|---|---|---
| $p$ | 200 | 400 | 600 | 200 | 400 | 600 | 200 | 400 | 600
$n=100$ | MAX | 0.058 | 0.070 | 0.065 | 0.044 | 0.037 | 0.039 | 0.048 | 0.042 | 0.047
| SUM | 0.053 | 0.067 | 0.056 | 0.054 | 0.052 | 0.048 | 0.054 | 0.055 | 0.045
| COM | 0.055 | 0.057 | 0.061 | 0.054 | 0.044 | 0.040 | 0.043 | 0.047 | 0.047
| HC2 | 0.022 | 0.011 | 0.013 | 0.005 | 0.015 | 0.005 | 0.011 | 0.011 | 0.006
| FLY | 0.022 | 0.011 | 0.011 | 0.013 | 0.010 | 0.006 | 0.024 | 0.015 | 0.007
$n=200$ | MAX | 0.053 | 0.054 | 0.076 | 0.025 | 0.042 | 0.025 | 0.044 | 0.040 | 0.041
| SUM | 0.053 | 0.057 | 0.060 | 0.053 | 0.051 | 0.052 | 0.055 | 0.065 | 0.060
| COM | 0.058 | 0.061 | 0.066 | 0.037 | 0.045 | 0.044 | 0.043 | 0.053 | 0.055
| HC2 | 0.003 | 0.011 | 0.006 | 0.010 | 0.006 | 0.003 | 0.004 | 0.005 | 0.008
| FLY | 0.037 | 0.033 | 0.025 | 0.030 | 0.022 | 0.011 | 0.032 | 0.026 | 0.015
Table 3:  Sizes of tests for Example 1 with Scenario (III), $\alpha=0.05$. | Distribution | (1) | (2) | (3)
---|---|---|---|---
| $p$ | 200 | 400 | 600 | 200 | 400 | 600 | 200 | 400 | 600
$n=100$ | MAX | 0.054 | 0.066 | 0.059 | 0.053 | 0.040 | 0.033 | 0.049 | 0.039 | 0.043
| SUM | 0.052 | 0.055 | 0.059 | 0.053 | 0.048 | 0.060 | 0.059 | 0.064 | 0.061
| COM | 0.053 | 0.066 | 0.059 | 0.053 | 0.050 | 0.040 | 0.062 | 0.046 | 0.051
| HC2 | 0.034 | 0.038 | 0.035 | 0.032 | 0.030 | 0.025 | 0.036 | 0.030 | 0.030
| FLY | 0.013 | 0.003 | 0.005 | 0.013 | 0.001 | 0.000 | 0.020 | 0.013 | 0.010
$n=200$ | MAX | 0.053 | 0.058 | 0.063 | 0.034 | 0.027 | 0.038 | 0.049 | 0.039 | 0.050
| SUM | 0.061 | 0.065 | 0.062 | 0.044 | 0.058 | 0.068 | 0.063 | 0.058 | 0.057
| COM | 0.065 | 0.075 | 0.069 | 0.033 | 0.048 | 0.047 | 0.059 | 0.051 | 0.053
| HC2 | 0.035 | 0.032 | 0.032 | 0.019 | 0.029 | 0.019 | 0.029 | 0.023 | 0.024
| FLY | 0.001 | 0.001 | 0.000 | 0.004 | 0.001 | 0.000 | 0.016 | 0.011 | 0.002

Tables 1, 2, 3 report the empirical sizes of the five tests. SUM, MAX and COM
can control the empirical sizes very well in most cases. However, the
empirical sizes of HC2 and FLY can be much smaller than the nominal level in
some cases.

Next, we examine the power of each test. Our simulation shows that the power
comparisons are similar for any combination of $(n,p)$ with $n=100,200$ and
$p=200,400,600$. Hence, we present the case $n=100$ and $p=200$ for
conciseness. Define ${\boldsymbol{\mu}}=(\mu_{1},\cdots,\mu_{p})^{T}$. For
different number of nonzero-mean variables $m=1,\cdots,20$, we consider
$\mu_{j}=\delta$ for $0<j\leq m$ and $\mu_{j}=0$ for $j>m$. The parameter
$\delta$ is chosen as $||{\boldsymbol{\mu}}||^{2}=m\delta^{2}=0.5$. Figure 1
reports the power of the five tests. The power of MAX decreases as the number
of nonzero-mean variables increases, which is as expected because, generally
speaking, the max-type test is more powerful in sparse case and less powerful
in non-sparse case. The power of SUM slightly increases with $m$ and is higher
than the power of HC2 and FLY in all cases. The proposed COM is as powerful as
MAX when the number of variables with nonzero means is small (sparse case),
and almost has the same power as SUM when the number of variables with nonzero
means grows. In general, COM possesses the advantages of both MAX (in sparse
case) and SUM (in non-sparse case), and outperforms HC2 and FLY in all
scenarios. Observe that all the tests, except for COM, favor either the sparse
or non-sparse case. Since in practice it is hard to justify whether the true
underlying model is sparse or not, our proposed COM test, with its strong
robustness, should be a more favorable choice over the competing approaches.

Figure 1: Power vs. number of variables with nonzero means for Example 4.1.
The $x$-lab $m$ denotes the number of variables with non-zero means; the
$y$-lab is the empirical power.

###  4.2 Regression Coefficient Test Problem

Then, we present our simulation results on the regression coefficient testing
problem. We will compare our combo-type test $T_{com}^{(3)}$ (abbreviated as
COM) from (21) against the test $T^{(3)}_{sum}$ (abbreviated as SUM) from
(17), the test $T^{(3)}_{max}$ (abbreviated as MAX) from (18) and the
empirical Bayes test proposed by Goeman et al. (2006) (abbreviated as EB).

EXAMPLE 2. We generate data from (13), where the regression coefficients
$\beta_{j}$ for $j\in\\{1,2,\cdots,q\\}$ are simulated from a standard normal
distribution, and then we set $\beta_{j}=0$ for $j>q$. In addition, the
predictor vector is given by
${\boldsymbol{X}}_{i}={\boldsymbol{\Sigma}}^{1/2}{\boldsymbol{z}}_{i}$ for
$i=1\cdots,n$, and each component of ${\boldsymbol{z}}_{i}$ is independently
generated from three distributions: (1) the normal distribution $N(0,1)$; (2)
the exponential distribution $\exp(1)-1$; (3) the mixture normal distribution
$V/\sqrt{1.8}$, where $V$ is as in Example 1.

Moreover, the random error $\varepsilon_{i}$ is independently generated from a
standard normal distribution. We report the Scenarios (I) result where
$\mbox{Cov}({\boldsymbol{X}}_{i})={\boldsymbol{\Sigma}}=(\sigma_{jk})\in\mathbb{R}^{p\times
p}$ with $\sigma_{jk}=0.5^{|j-k|}$, while the results in the other two cases
are similar. We consider two different sample sizes $n=100,200$, three
different dimensions $p=200,400,600$ and two dimension of predictors in the
reduced model $q=0$ or $5$.

Table 4:  Sizes of tests for Example 2, $\alpha=0.05$. | Distribution | (1) | (2) | (3)
---|---|---|---|---
| $p$ | 200 | 400 | 600 | 200 | 400 | 600 | 200 | 400 | 600
$q=0$
$n=100$ | MAX | 0.032 | 0.024 | 0.027 | 0.026 | 0.032 | 0.026 | 0.027 | 0.036 | 0.036
| EB | 0.047 | 0.047 | 0.047 | 0.049 | 0.052 | 0.050 | 0.059 | 0.044 | 0.044
| SUM | 0.061 | 0.064 | 0.065 | 0.060 | 0.072 | 0.062 | 0.079 | 0.058 | 0.060
| COM | 0.044 | 0.043 | 0.047 | 0.046 | 0.05 | 0.043 | 0.055 | 0.047 | 0.044
$n=200$ | MAX | 0.032 | 0.042 | 0.038 | 0.030 | 0.032 | 0.033 | 0.045 | 0.035 | 0.041
| EB | 0.063 | 0.052 | 0.049 | 0.038 | 0.042 | 0.036 | 0.053 | 0.041 | 0.055
| SUM | 0.069 | 0.06 | 0.057 | 0.054 | 0.052 | 0.045 | 0.063 | 0.049 | 0.064
| COM | 0.053 | 0.051 | 0.048 | 0.036 | 0.039 | 0.044 | 0.058 | 0.042 | 0.050
$q=5$
$n=100$ | MAX | 0.032 | 0.029 | 0.024 | 0.024 | 0.030 | 0.020 | 0.031 | 0.037 | 0.029
| EB | 0.048 | 0.050 | 0.055 | 0.056 | 0.058 | 0.061 | 0.061 | 0.054 | 0.045
| SUM | 0.046 | 0.049 | 0.048 | 0.063 | 0.059 | 0.064 | 0.063 | 0.052 | 0.045
| COM | 0.038 | 0.037 | 0.031 | 0.043 | 0.047 | 0.041 | 0.048 | 0.048 | 0.026
$n=200$ | MAX | 0.030 | 0.031 | 0.030 | 0.037 | 0.033 | 0.033 | 0.034 | 0.032 | 0.030
| EB | 0.068 | 0.051 | 0.047 | 0.045 | 0.052 | 0.046 | 0.058 | 0.066 | 0.057
| SUM | 0.067 | 0.051 | 0.049 | 0.049 | 0.054 | 0.045 | 0.070 | 0.068 | 0.061
| COM | 0.048 | 0.045 | 0.036 | 0.040 | 0.040 | 0.037 | 0.051 | 0.049 | 0.049

We report the empirical sizes of the tests in Table 4. We observe that the
empirical size of MAX tends to be smaller than the nominal level. EB and SUM,
as well as the proposed COM, control the empirical sizes well for most of the
times.

We compare the power of the tests with $n=100$, $p=200$. Each entry of
${\boldsymbol{z}}_{i}\in\mathbb{R}^{p}$ is generated from the standard normal
distribution (i.e. case (1) in Example 2). Define
$\boldsymbol{\beta}_{b}=\kappa\cdot(\beta_{q+1},\cdots,\beta_{p})^{T}$. Let
$m$ denote the number of nonzero coefficients. For $m=1,\cdots,50$, we
consider $\beta_{j}\sim N(0,1),q<j\leq q+m$ and $\beta_{j}=0$, $j>q+m$. The
parameter $\kappa$ is chosen so that $||\boldsymbol{\beta}_{b}||^{2}=0.5$. As
we see from the plots, EB performs similarly to SUM. When the number of
nonzero coefficients is small, MAX is more powerful than EB and SUM. In
contrast, when the number of nonzero coefficients is large, EB and SUM
outperform MAX. Once again, the proposed COM has same power as MAX in the
sparse case, and has similar performance to EB and SUM in the non-sparse case.
As we mentioned earlier, the results show the benefits of COM, as the true
model is usually unknown in practical applications. The proposed COM provides
good testing power in all cases.

Figure 2: Power vs. the number of nonzero coefficients for Example 2. The
$x$-lab $m$ denotes the number of non-zero coefficients; the $y$-lab is the
empirical power.

Figure 3: Histogram of sample means of stock monthly return rates in S&P 500.

##  5 Real Data Application

In this section, we further apply the results and test statistics obtained in
Section 3 to two real data: a US stock data (dense model) and a search engine
data (sparse model). As will be seen, the proposed combo-type test, COM,
performs well on both datasets. Thus, it could serve as a “universal” test in
practice no matter the true model is sparse or not.

###  5.1 US Stock Data

We apply the methods developed for one-sample mean test in Section 3.1 to a
pricing problem in finance. Specifically, we investigate how financial returns
of assets are related to their risk-free returns. Let
$X_{ij}=R_{ij}-\textrm{rf}_{i}$ denote the excess return of the $j$th asset at
time $i$ for $i=1,\cdots,n$ and $j=1,\cdots,p$, where $R_{ij}$ is the return
on asset $j$ during period $i$ and $\textrm{rf}_{i}$ is the risk-free return
rate of all assets during period $i$. We study the following pricing model

$\displaystyle X_{ij}=\mu_{j}+\xi_{ij},$ (22)

for $i=1,\cdots,n$ and $j=1,\cdots,p$, or, in vector form,
${\boldsymbol{X}}_{i}={\boldsymbol{\mu}}+\boldsymbol{\xi}_{i}$, where
${\boldsymbol{X}}_{i}=(X_{i1},\dots,X_{ip})^{T}$,
${\boldsymbol{\mu}}=(\mu_{1},\dots,\mu_{p})^{T}$, and
$\boldsymbol{\xi}_{i}=(\xi_{i1},\dots,\xi_{ip})^{T}$ is the zero-mean error
vector. The pricing model in (22) is the zero-factor model within the well
known Arbitrage Pricing Theory (Ross, 1976), where “zero-factor” means that no
additional factor is used to model the price. A common null hypothesis to be
considered under the pricing model (22) is
$H_{0}:{\boldsymbol{\mu}}=\boldsymbol{0}$, which means that the excess return
of any asset is zero on average, i.e. the return rate of any asset $R_{ij}$ is
equal to the risk-free return rate $\textrm{rf}_{i}$ on average.

We consider the monthly return rates of the stocks that constitute the S&P 500
index over the period from January 2005 to November 2018. Since the stocks
that made up the index changed over time and some stocks were created during
this period, we only consider a total of 374 stocks that were included in the
index during the entire time range. Figure 3 shows the sample mean of each
stock in this period. We observe that most average returns are positive. In
fact, as we enlarge the time range (increasing sample size $n$), the p-values
of MAX, SUM and COM are eventually smaller than 0.05. These results suggest
rejection of the null that the asset return does not only comes from the risk-
free rates (on average), which is consistent with the views of many economists
(Fama and French, 1993, 2015).

Table 5:  Rejecting rates of each test in US stock Data. | MAX | SUM | COM
---|---|---|---
$n=30$ | 0.35 | 0.39 | 0.40
$n=50$ | 0.40 | 0.51 | 0.51
$n=70$ | 0.44 | 0.67 | 0.62
$n=100$ | 0.52 | 0.86 | 0.83

We further evaluate the tests by a random sampling procedure. Specifically, we
randomly choose $n$ samples from the whole dataset and apply MAX, SUM and COM
on this new sample. For each $n$, we repeat this experiment for 1000 times.
Table 5 reports the rejecting rates for each method with different $n$. From
Table 5, we observe that SUM outperforms MAX in all cases by providing higher
rejection rates. This is not surprising because for this data, the number of
variables with nonzero means (assets with non-zero expected excess return)
might be large, which is the case where sum-type tests could typically perform
better than max-type tests. On the other hand, the combo-type test COM
performs similarly as SUM overall. Therefore, COM does not lose efficiency in
this problem.

###  5.2 Search Engine Data

We now use the data from Lan et al. (2014) to make a case study on the
regression coefficient test. The dataset is obtained from an online mobile
phone retailer. It contains a total of $n=98$ daily records. The response $Y$
is the revenue from the retailer’s online sales. The explanatory variable $V$
stands for the advertising spending on each of $p=164$ different keywords that
were bid for Baidu, the leading search engine in China. We sort these
explanatory variables by the correlation with the response, from high to low,
and denote $V_{1},V_{2},\cdots$, etc. Since the sales vary with each day of
the week, we introduce a 6-dimension indictor variables $W$ to represent
Sunday to Friday. So there are 170 explanatory variables $X=(W,V)$ in our
model. We will analyze it via the theory established in Section 3.2.

For different values of $k$, set
${\boldsymbol{X}}_{a}^{(k)}=(W,V_{1},\cdots,V_{k})^{T}$ and the rest variables
${\boldsymbol{X}}_{b}^{(k)}=(V_{k+1},\cdots,V_{p})^{T}$. We consider the
linear model
${\boldsymbol{Y}}=({\boldsymbol{X}}_{a}^{(k)})^{T}\boldsymbol{\beta}_{a}^{(k)}+({\boldsymbol{X}}_{b}^{(k)})^{T}\boldsymbol{\beta}_{b}^{(k)}+\boldsymbol{\varepsilon}$,
to test whether advertising spending on the rest of keywords $X_{b}^{(k)}$
could provide a significant contribution to online sales, conditional on the
effect of $X_{ak}$, i.e. we test
$H_{0}:\boldsymbol{\beta}_{b}^{(k)}=\boldsymbol{0}$. We adopt the tests
introduced in Example 2 in Section 4.2, i.e. MAX, SUM, EB and COM.

Table 6:  $p$-values of each test in Search Engine Data. | MAX | SUM | EB | COM
---|---|---|---|---
$k=0$ | 0.0056 | 0.1432 | 0.1100 | 0.0112
$k=1$ | 0.0025 | 0.2295 | 0.1776 | 0.0050
$k=2$ | 0.2083 | 0.1319 | 0.2464 | 0.2465
$k=3$ | 0.1508 | 0.1387 | 0.2877 | 0.2583

Table 6 reports the $p$-values of each tests with different $k$, which
controls the sparsity of the true model. The significant level is set to be
$\alpha=0.05$. From the results, we see that there are two keywords that are
significant to the response (revenue) because both MAX and COM reject the null
hypothesis as long as $k<2$ (note that these are two powerful methods for
sparse model). When $k\geq 2$, all these four tests do not reject the null
hypothesis, suggesting that the rest of keywords are not significant to the
response. Notice that when $k<2$, SUM and EB fail to reject the null
hypothesis at the significant level $0.05$, which shows their poor performance
with sparse model, consistent with our theoretical claims and simulation
results. On the contrary, COM succeeds in identifying the significant keywords
in this problem, illustrating its edge over SUM and EB.

##  6 Concluding Remarks

In this paper, we prove the asymptotic independence between the sum and
maximum of dependent random variables without stationary assumptions or strong
mixing conditions. Then we apply our results to high-dimensional testing
problems. Our proposed combo-type tests perform well regardless data being
sparse or not. Now we make some comments.

1\. The normal assumption is essential in the proof of asymptotic
independence. Hence, we also assume the Gaussian assumption in the high-
dimensional test problems. In literature, we may not need the Gaussian
assumption to analyze the asymptotic distribution of the sum-type and max-type
test statistics, e.g., Cai et al. (2014); Chen and Qin (2010). To prove the
asymptotic independence between the sum and maximum of non-normal dependent
random variables deserves further investigation.

2\. To obtain the asymptotic distribution of the sum and maximum of dependent
random variables, we assume the correlations between the random variables are
not very strong. Recently, there has been much literature that consider high-
dimensional testing problems without the weak correlation assumption, such as
Wang and Xu (2021); Zhang et al. (2020). The analogue of our asymptotic
independence result between the sum and maximum of dependent random variables
with arbitrary covariance structures is also a very interesting and
challenging problem.

3\. The asymptotic independence results in Theorem 3 is universal. We believe
it can be generalized and applied to many other applications, such as change
point detection and statistical process controls.

Acknowledgement. We thank Professor Wei Xiong for very helpful discussions.
The authors in this paper are listed in alphabetical order.

## References

  * Anderson and Turkman (1991) Anderson, C. and K. Turkman (1991).  The joint limiting distribution of sums and maxima of stationary sequences.  Journal of Applied Probability 28(1), 33–44.
  * Anderson and Turkman (1993) Anderson, C. and K. Turkman (1993).  Limiting joint distributions of sums and maxima in a statistical context.  Theory of Probability & Its Applications 37(2), 314–316.
  * Anderson and Turkman (1995) Anderson, C. and K. Turkman (1995).  Sums and maxima of stationary sequences with heavy tailed distributions.  Sankhyā: The Indian Journal of Statistics, Series A, 1–10.
  * Anderson (2003) Anderson, T. W. (2003).  An Introduction to Multivariate Statistical Analysis.  John Wiley & Sons.
  * Bai and Saranadasa (1996) Bai, Z. and H. Saranadasa (1996).  Effect of high dimension: by an example of a two sample problem.  Statistica Sinica, 311–329.
  * Bai and Silverstein (2010) Bai, Z. and J. W. Silverstein (2010).  Spectral analysis of large dimensional random matrices, Volume 20.  Springer.
  * Bradley (2005) Bradley, R. C. (2005).  Basic properties of strong mixing conditions. a survey and some open questions.  Probability surveys 2, 107–144.
  * Cai et al. (2014) Cai, T. T., W. Liu, and Y. Xia (2014).  Two-sample test of high dimensional means under dependence.  Journal of the Royal Statistical Society: Series B: Statistical Methodology, 349–372.
  * Chen et al. (2019) Chen, S. X., J. Li, and P.-S. Zhong (2019).  Two-sample and anova tests for high dimensional means.  The Annals of Statistics 47(3), 1443–1474.
  * Chen and Qin (2010) Chen, S. X. and Y.-L. Qin (2010).  A two-sample test for high-dimensional data with applications to gene-set testing.  The Annals of Statistics 38(2), 808–835.
  * Chow and Teicher (1997) Chow, Y. S. and H. Teicher (1997).  Probability theory: Independence, interchangeability, martingales.
  * Chow and Teugels (1978) Chow, T. L. and J. L. Teugels (1978).  The sum and the maximum of iid random variables.  In Proceedings of the 2nd Prague Symposium on Asymptotic Statistics, Volume 45, pp. 394–403.
  * Davis and Hsing (1995) Davis, R. A. and T. Hsing (1995).  Point process and partial sum convergence for weakly dependent random variables with infinite variance.  The Annals of Probability, 879–917.
  * Dembo and Zeitouni (1998) Dembo, A. and O. Zeitouni (1998).  Large Deviations Techniques and Applications.  Springer, 2nd ed.
  * Eaton (1983) Eaton, M. L. (1983).  Multivariate statistics: a vector space approach.  JOHN WILEY & SONS, INC., 605 THIRD AVE., NEW YORK, NY 10158, USA, 1983, 512.
  * Fama and French (1993) Fama, E. F. and K. R. French (1993).  Common risk factors in the returns on stocks and bonds.  Journal of Financial Economics 33, 3–56.
  * Fama and French (2015) Fama, E. F. and K. R. French (2015).  A five-factor asset pricing model.  Journal of financial economics 116(1), 1–22.
  * Fan et al. (2008) Fan, J., Y. Fan, and J. Lv (2008).  High dimensional covariance matrix estimation using a factor model.  Journal of Econometrics 147(1), 186–197.
  * Fan and Jiang (2019) Fan, J. and T. Jiang (2019).  Largest entries of sample correlation matrices from equi-correlated normal populations.  The Annals of Probability 47(5), 3321–3374.
  * Fan et al. (2015) Fan, J., Y. Liao, and J. Yao (2015).  Power enhancement in high-dimensional cross-sectional tests.  Econometrica 83(4), 1497–1541.
  * Feng et al. (2016) Feng, L., C. Zou, and Z. Wang (2016).  Multivariate-sign-based high-dimensional tests for the two-sample location problem.  Journal of the American Statistical Association 111(514), 721–735.
  * Feng et al. (2015) Feng, L., C. Zou, Z. Wang, and L. Zhu (2015).  Two-sample behrens-fisher problem for high-dimensional data.  Statistica Sinica, 1297–1312.
  * Feng et al. (2017) Feng, L., C. Zou, Z. Wang, and L. Zhu (2017).  Composite t 2 test for high-dimensional data.  Statistica Sinica, 1419–1436.
  * Goeman et al. (2006) Goeman, J. J., S. A. Van De Geer, and H. C. Van Houwelingen (2006).  Testing against a high dimensional alternative.  Journal of the Royal Statistical Society: Series B (Statistical Methodology) 68(3), 477–493.
  * Goeman et al. (2011) Goeman, J. J., H. C. Van Houwelingen, and L. Finos (2011).  Testing against a high-dimensional alternative in the generalized linear model: asymptotic type i error control.  Biometrika, 381–390.
  * Gregory et al. (2015) Gregory, K. B., R. J. Carroll, V. Baladandayuthapani, and S. N. Lahiri (2015).  A two-sample test for equality of means in high dimension.  Journal of the American Statistical Association 110(510), 837–849.
  * He et al. (2021) He, Y., G. Xu, C. Wu, and W. Pan (2021).  Asymptotically independent u-statistics in high-dimensional testing.  The Annals of Statistics 49(1), 154–181.
  * Ho and Hsing (1996) Ho, H.-C. and T. Hsing (1996).  On the asymptotic joint distribution of the sum and maximum of stationary normal random variables.  Journal of Applied Probability 33(1), 138–145.
  * Ho and McCormick (1999) Ho, H.-C. and W. P. McCormick (1999).  Asymptotic distribution of sum and maximum for gaussian processes.  Journal of Applied Probability 36(4), 1031–1044.
  * Horn and Johnson (2012) Horn, R. A. and C. R. Johnson (2012).  Matrix analysis.  Cambridge university press.
  * Hotelling (1931) Hotelling, H. (1931).  The generalization of student’s ratio.  The Annals of Mathematical Statistics 2(3), 360–378.
  * Hsing (1995) Hsing, T. (1995).  A note on the asymptotic independence of the sum and maximum of strongly mixing stationary random variables.  The Annals of Probability, 938–947.
  * Jiang and Li (2021) Jiang, T. and P. Li (2021).  Mean test with fewer observation than dimension and ratio unbiased estimator for correlation matrix.  arXiv preprint arXiv:2108.06892.
  * Lan et al. (2014) Lan, W., H. Wang, and C.-L. Tsai (2014).  Testing covariates in high-dimensional regression.  Annals of the Institute of Statistical Mathematics 66(2), 279–301.
  * McCormick and Qi (2000) McCormick, W. and Y. Qi (2000).  Asymptotic distribution for the sum and maximum of gaussian processes.  Journal of Applied Probability 37(4), 958–971.
  * Muirhead (1982) Muirhead, R. J. (1982).  Aspects of multivariate statistical theory.  John Wiley & Sons.
  * Park and Ayyala (2013) Park, J. and D. N. Ayyala (2013).  A test for the mean vector in large dimension and small samples.  Journal of Statistical Planning and Inference 143(5), 929–943.
  * Peng and Nadarajah (2003) Peng, Z. and S. Nadarajah (2003).  On the joint limiting distribution of sums and maxima of stationary normal sequence.  Theory of Probability & Its Applications 47(4), 706–709.
  * Ross (1976) Ross, S. A. (1976).  The arbitrage theory of capital asset pricing.  Journal of Economic Theory 13(3), 341–360.
  * Rothman et al. (2008) Rothman, A. J., P. J. Bickel, E. Levina, and J. Zhu (2008).  Sparse permutation invariant covariance estimation.  Electronic Journal of Statistics 2, 494–515.
  * Srivastava (2009) Srivastava, M. S. (2009).  A test for the mean vector with fewer observations than the dimension under non-normality.  Journal of Multivariate Analysis 100(3), 518–532.
  * Srivastava and Du (2008) Srivastava, M. S. and M. Du (2008).  A test for the mean vector with fewer observations than the dimension.  Journal of Multivariate Analysis 99(3), 386–402.
  * Wang (2009) Wang, H. (2009).  Forward regression for ultra-high dimensional variable screening.  Journal of the American Statistical Association 104(488), 1512–1524.
  * Wang et al. (2015) Wang, L., B. Peng, and R. Li (2015).  A high-dimensional nonparametric multivariate test for mean vector.  Journal of the American Statistical Association 110(512), 1658–1669.
  * Wang and Xu (2021) Wang, R. and W. Xu (2021).  An approximate randomization test for high-dimensional two-sample behrens-fisher problem under arbitrary covariances.  arXiv preprint arXiv:2108.01860.
  * Wu et al. (2019) Wu, C., G. Xu, and W. Pan (2019).  An adaptive test on high-dimensional parameters in generalized linear models.  Statistica Sinica 29(4), 2163–2186.
  * Xu et al. (2016) Xu, G., L. Lin, P. Wei, and W. Pan (2016).  An adaptive two-sample test for high-dimensional means.  Biometrika 103(3), 609–624.
  * Zhang and Huang (2008) Zhang, C.-H. and J. Huang (2008).  The sparsity and bias of the lasso selection in high-dimensional linear regression.  The Annals of Statistics 36(4), 1567–1594.
  * Zhang et al. (2020) Zhang, J.-T., J. Guo, B. Zhou, and M.-Y. Cheng (2020).  A simple two-sample test in high dimensions based on l 2-norm.  Journal of the American Statistical Association 115(530), 1011–1027.
  * Zhong and Chen (2011) Zhong, P.-S. and S. X. Chen (2011).  Tests for high-dimensional regression coefficients with factorial designs.  Journal of the American Statistical Association 106(493), 260–274.
  * Zhong et al. (2013) Zhong, P.-S., S. X. Chen, and M. Xu (2013).  Tests alternative to higher criticism for high-dimensional means under sparsity and column-wise dependence.  The Annals of Statistics 41(6), 2820–2851.

SUPPLEMENTARY MATERIAL of “Asymptotic Independence of the Sum and Maximum of
Dependent Random Variables with Applications to High-Dimensional Tests”

Two-sample mean test

We propose the combo-type two-sample mean test and present its simulation
results in comparison with some of its competitors.

Technical proofs

We provide the technical proofs of the theoretical results in Sections 2, 3 of
the text and Section S1 of the supplementary material.

##  S1 Two-sample Mean Test

###  S1.1 Testing Procedure

Here, we consider the two-sample mean testing problem in the high-dimensional
setting. Assume that
$\\{{\boldsymbol{X}}_{i1},\cdots,{\boldsymbol{X}}_{in_{i}}\\}$ for $i=1,2$ are
two independent random samples with sizes $n_{1}$ and $n_{2}$, and from
$p$-variate normal distributions
$N({\boldsymbol{\mu}}_{1},{\boldsymbol{\Sigma}})$ and
$N({\boldsymbol{\mu}}_{2},{\boldsymbol{\Sigma}})$, respectively. Consider

$\displaystyle H_{0}:{\boldsymbol{\mu}}_{1}={\boldsymbol{\mu}}_{2}\ \
\mbox{versus}\ \ H_{1}:{\boldsymbol{\mu}}_{1}\neq{\boldsymbol{\mu}}_{2}.$
(S.1)

For the case where dimension $p$ is fixed, the classic Hotelling’s $T^{2}$
test statistic is

$\displaystyle\frac{n_{1}n_{2}}{n_{1}+n_{2}}(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2})^{T}\hat{{\bf
S}}^{-1}(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2}),$ (S.2)

where $\bar{{\boldsymbol{X}}}_{i}$ is the sample mean vector of the $i$th
sample and $\hat{{\bf S}}$ is the pooled sample covariance matrix defined by

$\displaystyle\hat{{\bf
S}}=\frac{1}{n_{1}+n_{2}}\Big{[}\sum_{j=1}^{n_{1}}(\mathbb{X}_{1j}-\bar{\mathbb{X}}_{1})(\mathbb{X}_{1j}-\bar{\mathbb{X}}_{1})^{T}+\sum_{j=1}^{n_{2}}(\mathbb{X}_{2j}-\bar{\mathbb{X}}_{2})(\mathbb{X}_{2j}-\bar{\mathbb{X}}_{2})^{T}\Big{]}.$
(S.3)

Let $n=n_{1}+n_{2}$. In the high-dimensional case with $p>n$, $\hat{{\bf S}}$
is not guaranteed to be invertible. Under the assumption that $p/n\to
c\in(0,\infty)$, Bai and Saranadasa (1996) proposed a test statistic
$(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2})^{T}(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2})$
by replacing $\hat{{\bf S}}$ in (S.2) with the identity matrix. Without the
restriction on $n$ and $p$, Chen and Qin (2010) constructed a different test
statistic by excluding the term
$\sum_{j=1}^{n_{i}}{\boldsymbol{X}}_{ij}^{T}{\boldsymbol{X}}_{ij}$ for $i=1$
and $2$ from
$(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2})^{T}(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2})$.
However, the above two tests are not scale-invariant. A statistic
$T({\boldsymbol{X}}_{11},\cdots,{\boldsymbol{X}}_{1n_{1}},{\boldsymbol{X}}_{21},\cdots,{\boldsymbol{X}}_{2n_{2}})$
is said to be location-scale invariant if the corresponding value of $T$ is
not changed provided “${\boldsymbol{X}}_{ij}$” is replaced by
“$a{\boldsymbol{X}}_{ij}+b$” for all $i,j$, where $a$ and $b$ are arbitrary
constants free of $i$ and $j$. We say $T$ is scale-invariant if the above
holds with $b=0$. For this reason, many efforts have been devoted to construct
location-scale invariant test procedures, including Srivastava and Du (2008);
Gregory et al. (2015); Feng et al. (2015), to name a few. In particular,
Srivastava and Du (2008) considered the following sum-type test statistic

$\displaystyle
T^{(2)}_{sum}=\frac{\frac{n_{1}n_{2}}{n_{1}+n_{2}}(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2})^{T}\hat{{\bf
D}}^{-1}(\bar{{\boldsymbol{X}}}_{1}-\bar{{\boldsymbol{X}}}_{2})-\frac{(n_{1}+n_{2}-2)p}{(n_{1}+n_{2}-4)}}{\sqrt{2\big{[}\mathrm{tr}(\hat{{\bf
R}}^{2})-\frac{p^{2}}{(n_{1}+n_{2}-2)}\big{]}c_{p,n}}},$ (S.4)

where $\hat{{\bf D}}$ is the diagonal matrix of $\hat{{\bf S}}$ in (S.3) and
$\hat{{\bf R}}=\hat{{\bf D}}^{-1/2}\hat{{\bf S}}\hat{{\bf D}}^{-1/2}$ is the
pooled sample correlation matrix, and $c_{p,n}=1+\frac{\mathrm{tr}(\hat{{\bf
R}}^{2})}{p^{3/2}}$. The statistic $T^{(2)}_{sum}$ is location-scale
invariant. Similar to the discussion in the paragraph above (8), the above
sum-type tests usually do not perform well for sparse data. For the sparse
alternative, Chen et al. (2019) extended the work of Zhong et al. (2013) by
studying the statistic

$\displaystyle
M_{L_{n}}=\max_{s\in\mathcal{S}_{n}}\frac{L_{n}(s)-\hat{\mu}_{L_{n}(s),0}}{\hat{\sigma}_{L_{n}(s),0}}.$
(S.5)

Note that this formula follows (4.3) from Chen et al. (2019), where the
notations “$L_{n}(s)$, $\mathcal{S}_{n}$, $\hat{\mu}_{L_{n}(s),0}$,
$\hat{\sigma}_{L_{n}(s),0}$” are quite involved; interested readers are
referred to their paper for more details. Later on, we will compare our
proposed test with $M_{L_{n}}$ in (S.5). Again, for sparse data, Cai et al.
(2014) proposed the following max-type test statistic

$\displaystyle T^{(2)}_{max}=\frac{n_{1}n_{2}}{n_{1}+n_{2}}\max_{1\leq i\leq
p}\frac{(\bar{{\boldsymbol{X}}}_{1i}-\bar{{\boldsymbol{X}}}_{2i})^{2}}{\hat{\sigma}_{ii}^{2}},$

where $\bar{{\boldsymbol{X}}}_{ji}$ is the $i$th coordinate of
$\bar{{\boldsymbol{X}}}_{j}\in\mathbb{R}^{p}$ for $j=1,2$ and $1\leq i\leq p$
and $\hat{\sigma}_{ii}^{2}$ is the $i$th diagonal element of $\hat{{\bf S}}$
in (S.3). Similar to the one-sample test case, this test statistic is
particularly powerful against sparse alternatives with certain optimality. We
will study the asymptotic behavior of the sum-type and max-type tests, and
design a new test that takes advantage of both worlds.

Recall that $\mathbb{\Sigma}$ is the covariance matrix shared by two
populations, and let $\mathbb{D}$ be the diagonal matrix of $\mathbb{\Sigma}$
such that $\mathbb{R}:=\mathbb{D}^{-1/2}\mathbb{\Sigma}\mathbb{D}^{-1/2}$ is
the population correlation matrix. For soundness, assume that the sample sizes
$n_{1}$ and $n_{2}$ both depend on $p$. Now we apply the theoretical results
in Section 2 to the two-sample mean test as follows.

######  THEOREM 6

Assume the null hypothesis in (S.1) holds and
$\lim_{p\to\infty}n_{1}/n_{2}\to\kappa\in(0,\infty)$. The following are true
as $p\to\infty$:

  * (i)

If (3.1) holds, then $T^{(2)}_{sum}\to N(0,1)$ in distribution;

  * (ii)

If (2) holds with “$\mathbb{\Sigma}$” being replaced by “$\mathbb{R}$” and
$\log p=o(n^{1/3})$, then $T^{(2)}_{max}-2\log p+\log\log p$ converges weakly
to a Gumbel distribution with cdf
$F(x)=\exp\\{-\frac{1}{\sqrt{\pi}}\exp(-x/2)\\}$;

  * (iii)

Assume (3.1) holds. If (2) is true with “$\mathbb{\Sigma}$” replaced by
“$\mathbb{R}$”, then $T^{(2)}_{sum}$ and $T^{(2)}_{max}-2\log p+\log\log p$
are asymptotically independent.

Part (i) of Theorem S6 is from Srivastava and Du (2008). Recently Jiang and Li
(2021) obtained a general theory, which also leads to the same conclusion.
Same as in the one-sample test, for test $T^{(2)}_{sum}$, a level-$\alpha$
test rejects $H_{0}$ when $T^{(2)}_{sum}>z_{\alpha}=\Phi^{-1}(1-\alpha)$ of
$N(0,1)$. For the max-type test, a level-$\alpha$ test will then be carried
out through rejecting the null hypothesis when $T^{(2)}_{max}-2\log p+\log\log
p>q_{\alpha}=-\log\pi-2\log\log(1-\alpha)^{-1}$ of the distribution function
in Theorem S6(ii).

Relying on Theorem S6, we propose the following test statistic which utilizes
the max-type and sum-type tests. Define

$\displaystyle T_{com}^{(2)}=\min\\{P^{(2)}_{M},P^{(2)}_{S}\\},$ (S.6)

where $P^{(2)}_{M}=1-F(T_{max}^{(2)}-2\log p+\log\log p)$ with
$F(y)=e^{-\pi^{-1/2}e^{-y/2}}$ and $P^{(2)}_{S}=1-\Phi(T_{sum}^{(2)}).$ are
the $p$-values of the two tests, respectively. Similar to Corollary 1, we
immediately obtain the following result by the asymptotic independence.

######  COROLLARY 2

Assume the condition in Theorem S6(iii) holds. Then $T_{com}^{(2)}$ in (S.6)
converges weakly to a distribution with density $G(w)=2(1-w)I(0\leq w\leq 1)$
as $p\to\infty.$

According to Corollary S2, the proposed combo-type test leads us to perform a
level-$\alpha$ test by rejecting the null hypothesis when
$T_{com}^{(2)}<1-\sqrt{1-\alpha}\approx\frac{\alpha}{2}$ when $\alpha$ is
small. Now we analyze the power of the test $T_{com}^{(2)}$.

Write
${\boldsymbol{\mu}}=({\boldsymbol{\mu}}_{1},{\boldsymbol{\mu}}_{2})^{T}$.
Similar to (11), the power function of our combo-type test
$\beta_{C}^{(2)}({\boldsymbol{\mu}},\alpha)$ is larger than
$\max\\{\beta_{M}^{(2)}({\boldsymbol{\mu}},\alpha),\beta_{S}^{(2)}({\boldsymbol{\mu}},\alpha)\\}$,
where $\beta^{(2)}_{M}({\boldsymbol{\mu}},\alpha)$ and
$\beta^{(2)}_{S}({\boldsymbol{\mu}},\alpha)$ are the power functions of
$T_{max}^{(2)}$ and $T_{sum}^{(2)}$ with significant level $\alpha$,
respectively. Following Srivastava and Du (2008), the power function of
$T_{sum}^{(2)}$ is given by

$\displaystyle\beta^{(2)}_{S}({\boldsymbol{\mu}},\alpha)=\lim_{p\to\infty}\Phi\left(-z_{\alpha}+\frac{\frac{n_{1}n_{2}}{n_{1}+n_{2}}({\boldsymbol{\mu}}_{1}-{\boldsymbol{\mu}}_{2})^{T}{\bf
D}^{-1}({\boldsymbol{\mu}}_{1}-{\boldsymbol{\mu}}_{2})}{\sqrt{2\mathrm{tr}({\bf
R}^{2})}}\right).$

Thus, we have

$\displaystyle\beta^{(2)}_{C}({\boldsymbol{\mu}},\alpha)\geq\lim_{p\to\infty}\Phi\left(-z_{\alpha/2}+\frac{\frac{n_{1}n_{2}}{n_{1}+n_{2}}({\boldsymbol{\mu}}_{1}-{\boldsymbol{\mu}}_{2})^{T}{\bf
D}^{-1}({\boldsymbol{\mu}}_{1}-{\boldsymbol{\mu}}_{2})}{\sqrt{2\mathrm{tr}({\bf
R}^{2})}}\right).$

Further write ${\boldsymbol{\mu}}_{1}=(\mu_{11},\cdots,\mu_{1p})^{T}$ and
${\boldsymbol{\mu}}_{2}=(\mu_{21},\cdots,\mu_{2p})^{T}$, and define
$\delta_{i}=\mu_{1i}-\mu_{2i}$ for $i=1,\cdots,p.$ We have an analogous set of
analysis and claims as for the one-sample test. Firstly, by Theorem 2 from Cai
et al. (2014), the asymptotic power of $T_{max}^{(2)}$ converges to one if
$\max_{1\leq i\leq p}|\delta_{i}/{\sigma_{ii}}|\geq c\sqrt{\log p/n}$ for a
certain constant $c$ and if the sparsity level $\gamma<1/4$ and the locations
of the non-zero variables are randomly and uniformly selected from
$\\{1,\cdots,p\\}$, meaning that the power function of our proposed test
$T_{com}^{(2)}$ also converges to one under this situation. Secondly,
according to Theorem 3 from Cai et al. (2014), the condition $\max_{1\leq
i\leq p}|\delta_{i}/{\sigma_{ii}}|\geq c\sqrt{\log p/n}$ is minimax rate-
optimal for testing against sparse alternatives, and such optimality also
holds for our test $T_{max}^{(2)}$.

Similar to the one-sample test problem, we consider a special case with
${\boldsymbol{\Sigma}}={\bf I}_{p}$. There are $m$ nonzeros $\delta_{i}$ and
they are all equal to $\delta\not=0$. Thus,

$\displaystyle\beta_{S}^{(2)}=\lim_{p\to\infty}\Phi\left(-z_{\alpha}+\frac{n_{1}n_{2}m\delta^{2}}{n\sqrt{2p}}\right).$

Take $\xi>0$ such that $p^{1/2}n^{2\xi-1}\to\infty.$ For the non-sparse case:
$\delta=O(n^{-\xi})$ and $m=o(p^{1/2}n^{2\xi-1})$, we have
$\beta_{M}^{(2)}({\boldsymbol{\mu}},\alpha)\approx\alpha$ and
$\beta_{C}^{(2)}({\boldsymbol{\mu}},\alpha)\approx\beta_{S}^{(2)}({\boldsymbol{\mu}},\alpha/2)$.
For the sparse case: $\delta=c\sqrt{\log p/n}$ with sufficient large $c$ and
$m=o((\log p)^{-1}p^{1/2})$, we have
$\beta_{S}^{(2)}({\boldsymbol{\mu}},\alpha)\approx\alpha$ and
$\beta_{C}^{(2)}({\boldsymbol{\mu}},\alpha)\approx\beta_{M}^{(2)}({\boldsymbol{\mu}},\alpha/2)\to
1$.

###  S1.2 Simulation Results

Then, we present the simulation results for the two-sample test problem, where
our test $T_{com}^{(2)}$ (abbreviated as COM) from (S.6) will be compared with
the sum-type test $T^{(2)}_{sum}$ from (S.4) proposed by Srivastava and Du
(2008) (abbreviated as SUM), the max-type test $T^{(2)}_{max}$ proposed by Cai
et al. (2014) (abbreviated as MAX) and the Higher Criticism test proposed by
Chen et al. (2019) (abbreviated as HC2).

Recall the three scenarios of covariance matrices appeared in (I), (II) and
(III) after Example 1. Since the conclusions from all three scenarios are
similar, here we only present the results when the covariance matrix follows
Scenario (I), i.e., ${\boldsymbol{\Sigma}}=(0.5^{|i-j|})_{1\leq i,j\leq p}$.

Example 3. We consider
${\boldsymbol{X}}_{ki}={\boldsymbol{\mu}}_{k}+{\boldsymbol{\Sigma}}^{1/2}{\boldsymbol{z}}_{ki}$
for $k=1,2$ and $i=1\cdots,n$, and each component of ${\boldsymbol{z}}_{ki}$
is independently generated from three distributions: (1) $N(0,1)$; (2)
$t$-distribution, $t(5)/\sqrt{5/3}$; (3) the mixture normal random variable
$V/\sqrt{1.8}$, where $V$ is as in Example 1.

We consider two different sample sizes $n=100,200$ and three different
dimensions $p=200,400,600$. Under the null hypothesis, we set
${\boldsymbol{\mu}}_{1}={\boldsymbol{\mu}}_{2}=\boldsymbol{0}$. The
significance level is chosen such that $\alpha=0.05$. Again, for the
alternative hypothesis, we only present on $n=100,p=200$ and
${\boldsymbol{\mu}}_{2}=\boldsymbol{0}$ since the observations from different
combinations of $n$ and $p$ are similar. Define
${\boldsymbol{\mu}}_{1}=(\mu_{11},\cdots,\mu_{1p})^{T}$. For different number
of nonzero-mean variables $m=1,\cdots,20$, we consider $\mu_{j1}=\delta$ for
$0<j\leq m$ and $\mu_{1j}=0$ for $j>m$. The parameter $\delta$ is chosen such
that $||{\boldsymbol{\mu}}_{1}||^{2}=m\delta^{2}=1$.

Table S7:  Sizes of tests for Example 3, $\alpha=0.05$. | Distribution | (1) | (2) | (3)
---|---|---|---|---
| $p$ | 200 | 400 | 600 | 200 | 400 | 600 | 200 | 400 | 600
$n=100$ | MAX | 0.057 | 0.063 | 0.058 | 0.062 | 0.061 | 0.060 | 0.060 | 0.066 | 0.064
| SUM | 0.055 | 0.058 | 0.057 | 0.060 | 0.064 | 0.065 | 0.055 | 0.061 | 0.061
| COM | 0.057 | 0.074 | 0.060 | 0.064 | 0.065 | 0.068 | 0.061 | 0.061 | 0.062
| HC2 | 0.043 | 0.035 | 0.044 | 0.047 | 0.037 | 0.052 | 0.042 | 0.043 | 0.053
$n=200$ | MAX | 0.053 | 0.053 | 0.051 | 0.049 | 0.062 | 0.061 | 0.044 | 0.065 | 0.055
| SUM | 0.052 | 0.065 | 0.059 | 0.065 | 0.061 | 0.064 | 0.052 | 0.056 | 0.053
| COM | 0.044 | 0.049 | 0.056 | 0.061 | 0.063 | 0.058 | 0.046 | 0.061 | 0.053
| HC2 | 0.030 | 0.025 | 0.025 | 0.030 | 0.035 | 0.036 | 0.025 | 0.020 | 0.033
Figure S4: Power vs. number of variables with non-zero means for Example 3.
The $x$-lab $m$ denotes the number of variables with non-zero means; the
$y$-lab is the empirical power.

Table S7 reports the empirical sizes of the compared tests. We see that all
the tests control the empirical sizes in most cases except that the sizes of
HC2 are a little smaller than the nominal level when $n=200$.

Figure S4 shows the power of each test, where we observe a similar pattern as
in Example 1. The power of MAX declines as the number of variables with
nonzero means is increasing. The power of SUM and COM are always larger than
that of HC2 in all cases. The proposed COM matches the power of MAX when the
number of variables with nonzero means is small, and almost has same power as
SUM when $m$ is large. This justifies the superiority of the proposed combo-
type test in the two-sample testing problem, regardless of the sparsity of the
data.

##  S2 Technical Proofs

###  S2.1 Proof of Theorem 1

Proof of Theorem 1. Let $\xi_{1},\xi_{2},\cdots$ be i.i.d.
$N(0,1)$-distributed random variables. Let $\mathbb{\Sigma}^{1/2}$ be a non-
negative definite matrix such that
$\mathbb{\Sigma}^{1/2}\cdot\mathbb{\Sigma}^{1/2}=\mathbb{\Sigma}.$ Then
$(Z_{1},\cdots,Z_{p})^{T}$ and
$\mathbb{\Sigma}^{1/2}(\xi_{1},\cdots,\xi_{p})^{T}$ have the same
distribution. As a consequence, $Z_{1}^{2}+\cdots+Z_{p}^{2}$ has the same
distribution as that of

$\displaystyle(\xi_{1},\cdots,\xi_{p})\mathbb{\Sigma}^{1/2}\cdot\mathbb{\Sigma}^{1/2}(\xi_{1},\cdots,\xi_{p})^{T}=(\xi_{1},\cdots,\xi_{p})\mathbb{\Sigma}(\xi_{1},\cdots,\xi_{p})^{T}.$
(S.7)

Let $\lambda_{p,1},\lambda_{p,2},\cdots,\lambda_{p,p}$ be the eigenvalues of
$\mathbb{\Sigma}=\mathbb{\Sigma}_{p}$ and $\mathbb{O}$ be a $p\times p$
orthogonal matrix such that
$\mathbb{\Sigma}_{p}=\mathbb{O}^{T}\,\mbox{diag}(\lambda_{p1},\cdots,\lambda_{pp})\mathbb{O}$.
In particular, since all of the diagonal entries of $\mathbb{\Sigma}$ are $1$,
we have $\lambda_{p,1}+\cdots+\lambda_{p,p}=p$. By the orthogonal invariance
of normal distributions, $\mathbb{O}(\xi_{1},\cdots,\xi_{p})^{T}$ and
$(\xi_{1},\cdots,\xi_{p})^{T}$ have the same distribution. By (S.7),
$Z_{1}^{2}+\cdots+Z_{p}^{2}$ is equal to

$\displaystyle\big{[}\mathbb{O}(\xi_{1},\cdots,\xi_{p})^{T}\big{]}^{T}\,\mbox{diag}(\lambda_{p,1},\cdots,\lambda_{p,p})\,\big{[}\mathbb{O}(\xi_{1},\cdots,\xi_{p})^{T}\big{]},$

and hence has the same distribution as that of
$\lambda_{p,1}\xi_{1}^{2}+\cdots+\lambda_{p,p}\xi_{p}^{2}$. It is easy to see
$E(Z_{1}^{2}+\cdots+Z_{p}^{2})=p$ and

$\displaystyle\mbox{Var}(Z_{1}^{2}+\cdots+Z_{p}^{2})$ $\displaystyle=$
$\displaystyle\lambda_{p,1}^{2}\mbox{Var}(\xi_{1}^{2})+\cdots+\lambda_{p,p}^{2}\mbox{Var}(\xi_{p}^{2})$
$\displaystyle=$ $\displaystyle 2\lambda_{p,1}^{2}+\cdots+2\lambda_{p,p}^{2}$
$\displaystyle=$ $\displaystyle 2\cdot\mbox{tr}(\mathbb{\Sigma}^{2}).$

Easily, $m_{k}:=E(|\xi_{1}^{2}-1|^{k})<\infty$ for any $k\geq 1$. Note that
$\lambda_{p,1}\xi_{1}^{2}+\cdots+\lambda_{p,p}\xi_{p}^{2}$ is a sum of
independent random variables. Then,

$\displaystyle\frac{1}{[2\cdot\mbox{tr}(\mathbb{\Sigma}^{2})]^{(2+\delta)/2}}\sum_{i=1}^{p}E\big{|}\lambda_{p,i}\xi_{i}^{2}-E(\lambda_{p,i}\xi_{i}^{2})\big{|}^{2+\delta}$
$\displaystyle\leq$
$\displaystyle\frac{m_{2+\delta}}{[\mbox{tr}(\mathbb{\Sigma}^{2})]^{(2+\delta)/2}}\cdot\sum_{i=1}^{p}\lambda_{p,i}^{2+\delta}$
$\displaystyle=$ $\displaystyle
m_{2+\delta}\cdot\frac{\mbox{tr}(\mathbb{\Sigma}^{2+\delta})}{[\mbox{tr}(\mathbb{\Sigma}^{2})]^{(2+\delta)/2}},$

which goes to zero by Assumption (1). Therefore, by the Lyapunov central limit
theorem,
$(\lambda_{p,1}\xi_{1}^{2}+\cdots+\lambda_{p,p}\xi_{p}^{2}-p)/\sqrt{2\mbox{tr}(\mathbb{\Sigma}^{2})}$
converges weakly to $N(0,1)$ as $p\to\infty.$ This implies that
$(Z_{1}^{2}+\cdots+Z_{p}^{2}-p)/\sqrt{2\mbox{tr}(\mathbb{\Sigma}^{2})}$
converges weakly to $N(0,1)$ as $p\to\infty.$ $\square$

###  S2.2 Proof of Theorem 2

For a graph $G$, we say vertices $i$ and $j$ are neighbors if there is an edge
between them. For a set $A$, we write $|A|$ for its cardinality. We first
prove some lemmas.

######  LEMMA S.1

Let $G=(V,E)$ be an undirected graph with $n=|V|\geq 4$ vertices. Write
$V=\\{v_{1},\cdots,v_{n}\\}$. Assume each vertex in $V$ has at most $q$
neighbors. Let $G_{t}$ be the set of subgraphs of $G$ such that each subgraph
has $t$ vertices and at least one edge. The following are true.

(i) $|G_{t}|\leq qn^{t-1}$ for any $2\leq t\leq n$.

(ii) Fix integer $t$ with $2\leq t\leq n$. Let $G_{t}^{\prime}\subset G_{t}$
such that each member of $G_{t}^{\prime}$ is a clique, that is, any two
vertices are neighbors. Then $|G_{t}^{\prime}|\leq nq^{t-1}.$

The following conclusions are true for integer $t$ with $3\leq t\leq n$.

(iii) For $j=2,\cdots,t-1$, let $H_{j}$ be the subset of
$(i_{1},\cdots,i_{t})$ from $G_{t}$ satisfying the following: there exists a
subgraph $S$ of $\\{i_{1},\cdots,i_{t}\\}$ with $|S|=j$ and without any edge
such that any vertex from $\\{i_{1},\cdots,i_{t}\\}\backslash S$ has at least
two neighbors in $S$. Then $|H_{j}|\leq(qt)^{t-j+1}n^{j-1}$.

(iv) For $j=2,\cdots,t-1$, let $H_{j}^{\prime}$ be the subset of
$(i_{1},\cdots,i_{t})$ from $G_{t}$ satisfying the following: for any subgraph
$S$ of $\\{i_{1},\cdots,i_{t}\\}$ with $|S|=j$ and without any edge, we know
any vertex from $\\{i_{1},\cdots,i_{t}\\}\backslash S$ has at least one
neighbor in $S$. Then $|H_{j}^{\prime}|\leq(qt)^{t-j}n^{j}.$

Proof of Lemma S.1. (i). Choose one vertex from $V$ and choose one of its
neighbors. The total number of ways to do this is $nq$. The total number of
ways to fill the rest of $t-2$ vertices arbitrarily is no more than $n^{t-2}$.
Hence, $|G_{t}|\leq nq\cdot n^{t-2}=qn^{t-1}.$

(ii). To form a clique from $G_{t}$, we first choose a vertex with $n$ ways.
The next vertex has to be one of its $q$ neighbors, the third vertex has to be
one of the neighbors of the first two vertices at the same time. Thus the
number of choices for the third vertex is no more than $q$. This has to be
true for the picks of the remaining vertices to form a clique. So
$|G_{t}^{\prime}|\leq nq^{t-1}$.

(iii). Now we figure out the ways to get $(i_{1},\cdots,i_{t})\in H_{t}$. The
number of ways to get $i_{1}$ is at most $n$. Once $i_{1}$ is chosen, another
vertex $b_{1}$ from its neighbors has to be picked to be an element in
$\\{i_{1},\cdots,i_{t}\\}\backslash S$. Once $b_{1}$ is taken, since at leat
two members from $(i_{1},\cdots,i_{t})$ are neighbors of $b_{1}$, a third
vertex $i_{2}$ has to be in $S$. Keep in mind that $i_{2}$ has to be a
neighbor of $b_{1}$. Thus, the total number of ways to pick these three
vertices is at most $n\cdot q\cdot q.$ The rest of $j-2$ vertices in $S$ have
at most $n^{j-2}$ choices to satisfy the requirement; the rest $t-j-1$
vertices from $\\{i_{1},\cdots,i_{t}\\}\backslash S$ have to be the neighbors
of the vertices in $S$, which amounts to at most $(qt)^{t-j-1}$ ways to fill
the $t-j$ vertices. So $|H_{t}|\leq nq^{2}\cdot
n^{j-2}\cdot(qt)^{t-j-1}=(qt)^{t-j+1}n^{j-1}.$

(iv). The choices of $S$ with $|S|=j$ is no more than $n^{j}$. Any of the rest
$t-j$ vertices from $(i_{1},\cdots,i_{t})$ must be the neighbor of a vertex
from the chosen $j$ vertices. The total number of neighbors is at most $qt$.
This amounts to no more than $(qt)^{t-j}$ ways to achieve this. Hence
$|H_{j}^{\prime}|\leq(qt)^{t-j}n^{j}$. $\square$

######  LEMMA S.2

For $p\geq 1$, let $R=R_{p}$ be positive integers with
$\lim_{p\to\infty}R_{p}/p=1$. Let $Z_{p1},\cdots,Z_{pR}$ be
$N(0,1)$-distributed random variables with covariance matrix
$\mathbb{\Sigma}=\mathbb{\Sigma}_{p}=(\sigma_{ij})_{p\times p}.$ Assume
$|\sigma_{ij}|\leq\delta_{p}$ for all $1\leq i<j\leq R$ and $p\geq 1$, where
$\\{\delta_{p};\,p\geq 1\\}$ are constants satisfying $0<\delta_{p}=o(1/\log
p).$ Given $x\in\mathbb{R}$, set $z=\big{(}2\log p-\log\log p+x\big{)}^{1/2}.$
Then, for any fixed $m\geq 1$, we have

$\displaystyle\Big{(}\frac{\sqrt{\pi}p}{e^{-x/2}}\Big{)}^{m}\cdot
P(|Z_{pi_{1}}|>z,\cdots,|Z_{pi_{m}}|>z)\to 1$

as $p\to\infty$ uniformly for all $1\leq i_{1}<\cdots<i_{m}\leq R.$

Proof of Lemma S.2. To ease notation, we write “$Z_{1},\cdots,Z_{R}$” for
“$Z_{p1},\cdots,Z_{pR}$” if there is no danger of confusion. First, $z$ is
well-defined as $p$ is large. Note that $(Z_{1},\cdots,Z_{m})^{T}\sim
N(\mathbb{0},\mathbb{\Sigma}_{m})$, where
$\mathbb{\Sigma}_{m}=(\sigma_{ij})_{1\leq i,j\leq m}$. Recall the density
function of $N(\mathbb{0},\mathbb{\Sigma}_{m})$ is given by
$\frac{1}{(2\pi)^{m/2}\mbox{det}(\mathbb{\Sigma}_{m})^{1/2}}\exp\big{(}-\frac{1}{2}x^{T}\mathbb{\Sigma}_{m}^{-1}x\big{)}$
for all $x:=(x_{1},\cdots,x_{m})^{T}\in\mathbb{R}^{m}$. It follows that

$\displaystyle P(Z_{1}>z,\cdots,Z_{m}>z)$ $\displaystyle=$
$\displaystyle\frac{1}{(2\pi)^{m/2}\mbox{det}(\mathbb{\Sigma}_{m})^{1/2}}\int_{z}^{\infty}\cdots\int_{z}^{\infty}\exp\Big{(}-\frac{1}{2}x^{T}\mathbb{\Sigma}_{m}^{-1}x\Big{)}\,dx_{1}\cdots
dx_{m}.$ (S.8)

For two non-negative definite matrices $\mathbb{A}$ and $\mathbb{B}$, we write
$\mathbb{A}\leq\mathbb{B}$ if $\mathbb{B}-\mathbb{A}$ is also non-negative
definite. We need to understand $\mathbb{\Sigma}_{m}^{-1}$ and
$\mbox{det}(\mathbb{\Sigma}_{m})$ on the right hand side of (S2.2). First we
claim that

$\displaystyle(1-2m\delta_{p})\mathbb{I}_{m}\leq\mathbb{\Sigma}_{m}^{-1}\leq(1+2m\delta_{p})\mathbb{I}_{m}$
(S.9)

and that

$\displaystyle 1-m!\delta_{p}\leq\mbox{det}(\mathbb{\Sigma}_{m})^{-1/2}\leq
1+m!\delta_{p}$ (S.10)

as $p$ is sufficiently large.

In fact, by the Gershgorin disc theorem (see, e.g., Horn and Johnson (2012)),
all eigenvalues of $\mathbb{\Sigma}_{m}$ have to be in the set

$\displaystyle\bigcup_{1\leq i\leq m}\Big{(}\sigma_{ii}-\sum_{j\neq
i}\sigma_{ij},\sigma_{ii}+\sum_{j\neq i}\sigma_{ij}\Big{)}.$

By assumption, $\sigma_{ii}=1$ and $|\sigma_{ij}|\leq\delta_{p}$ for all
$1\leq i<j\leq R$. Thus, all of the eigenvalues of $\mathbb{\Sigma}_{m}$ are
between $1-m\delta_{p}$ and $1+m\delta_{p}$. The two bounds are positive as
$p$ is sufficiently large. On the other hand, $(1-m\delta_{p})^{-1}\leq
1+2m\delta_{p}$ and $(1+m\delta_{p})^{-1}\geq 1-2m\delta_{p}$ as $p$ is
sufficiently large. The assertion (S.9) is obtained.

Second, $\mbox{det}(\mathbb{\Sigma}_{m})$ is the sum of $m!$ terms. The term
as the product of the diagonal entries of $\mathbb{\Sigma}_{m}$ is $1$; each
of the remaining $m!-1$ terms is the product of $m$ entries from which at
least one is an off-diagonal entry. Therefore,
$|\mbox{det}(\mathbb{\Sigma}_{m})-1|\leq m!\delta_{p}.$ This implies
$1-m!\delta_{p}\leq\mbox{det}(\mathbb{\Sigma}_{m})\leq 1+m!\delta_{p}$.
Trivially, $(1+u)^{-1/2}=1-\frac{u}{2}(1+O(u))$ as $u\to 0$. The statement
(S.10) is confirmed.

The claim (S.9) implies that $(1-2m\delta_{p})|x|^{2}\leq
x^{T}\mathbb{\Sigma}_{m}^{-1}x\leq(1+2m\delta_{p})|x|^{2}$ for each
$x\in\mathbb{R}^{m}.$ The tail bound of Gaussian variable gives $P(N(0,1)\geq
t)\sim\frac{1}{\sqrt{2\pi}\,t}e^{-t^{2}/2}$ as $t\to\infty.$ Since
$P(N(0,\sigma^{2})\geq t)=P(N(0,1)\geq t/\sigma)$ for any $\sigma>0$, for any
$\epsilon\in(0,1/2)$, there exists $t_{0}>0$ such that

$\displaystyle(1-\epsilon)\frac{\sigma}{\sqrt{2\pi}\,t}e^{-t^{2}/(2\sigma^{2})}\leq
P(N(0,\sigma^{2})\geq
t)\leq(1+\epsilon)\frac{\sigma}{\sqrt{2\pi}\,t}e^{-t^{2}/(2\sigma^{2})}$
(S.11)

for all $t\geq\sigma t_{0}.$ Thus, from (S2.2)-(S.11), we upper bound the
probability of interest by

$\displaystyle P(Z_{1}>z,\cdots,Z_{m}>z)$ $\displaystyle\leq$
$\displaystyle\frac{1+m!\delta_{p}}{(2\pi)^{m/2}}\int_{z}^{\infty}\cdots\int_{z}^{\infty}\exp\Big{[}-\frac{(1-2m\delta_{p})}{2}(x_{1}^{2}+\cdots+x_{m}^{2})\Big{]}\,dx_{1}\cdots
dx_{m}$ $\displaystyle=$ $\displaystyle(1+m!\delta_{p})\sigma^{m}\cdot
P\big{(}N(0,\sigma^{2})\geq z\big{)}^{m}$ $\displaystyle\leq$
$\displaystyle(1+m!\delta_{p})\sigma^{2m}(1+\epsilon)^{m}\cdot\Big{[}\frac{1}{\sqrt{2\pi}\,z}e^{-(1-2m\delta_{p})z^{2}/2}\Big{]}^{m}$

as $z\geq\sigma t_{0}$, where $\sigma:=(1-2m\delta_{p})^{-1/2}.$ Now

$\displaystyle\frac{1}{z}e^{-(1-2m\delta_{p})z^{2}/2}$ $\displaystyle=$
$\displaystyle e^{m\delta_{p}z^{2}}\cdot\frac{1}{z}e^{-z^{2}/2}$
$\displaystyle=$ $\displaystyle e^{o(1)}\cdot\frac{1+o(1)}{\sqrt{2\log
p}}\cdot\exp\Big{[}-\frac{1}{2}(2\log p-\log\log p+x)\Big{]}$ $\displaystyle=$
$\displaystyle[1+o(1)]\cdot\frac{e^{-x/2}}{\sqrt{2}p}$

as $p\to\infty$ since $\delta_{p}=o(1/\log p)$, where the last $o(1)$ depends
on $m$ and $p$. Consequently,

$\displaystyle
P(Z_{1}>z,\cdots,Z_{m}>z)\leq\Big{(}\frac{e^{-x/2}}{2\sqrt{\pi}p}\Big{)}^{m}C_{m}$

where

$\displaystyle
C_{m}:=(1+m!\delta_{p})\sigma^{2m}(1+\epsilon)^{m}[1+o(1)]^{m}\leq(1+2\epsilon)^{m}$

as $p$ is sufficiently large because $\delta_{p}\to 0$. In summary, for fixed
$m\geq 1$,

$\displaystyle
P(Z_{1}>z,\cdots,Z_{m}>z)\leq(1+2\epsilon)^{m}\Big{(}\frac{e^{-x/2}}{2\sqrt{\pi}p}\Big{)}^{m}$
(S.12)

as $p$ is sufficiently large. Similarly, from (S2.2)-(S.11), a lower bound can
be established as

$\displaystyle P(Z_{1}>z,\cdots,Z_{m}>z)$ $\displaystyle\geq$
$\displaystyle\frac{1-m!\delta_{p}}{(2\pi)^{m/2}}\int_{z}^{\infty}\cdots\int_{z}^{\infty}\exp\Big{(}-\frac{(1+2m\delta_{p})}{2}(x_{1}^{2}+\cdots+x_{m}^{2})\Big{)}\,dx_{1}\cdots
dx_{m}$ $\displaystyle=$ $\displaystyle(1-m!\delta_{p})\sigma_{1}^{m}\cdot
P\big{(}N(0,\sigma_{1}^{2})\geq z\big{)}^{m}$ $\displaystyle\geq$
$\displaystyle(1-m!\delta_{p})\sigma_{1}^{2m}(1-\epsilon)^{m}\cdot\Big{[}\frac{1}{\sqrt{2\pi}\,z}e^{-(1+2m\delta_{p})z^{2}/2}\Big{]}^{m}$

where $\sigma_{1}:=(1+2m\delta_{p})^{-1/2}.$ By taking care of each term above
as in the previous arguments, we can get a reverse inequality of (S.12) with
“$1+2\epsilon$” replaced by “$1-2\epsilon$”. Consequently, we know that

$\displaystyle\Big{(}\frac{e^{-x/2}}{2\sqrt{\pi}p}\Big{)}^{m}\cdot(1-2\epsilon)^{m}\leq
P(Z_{1}>z,\cdots,Z_{m}>z)\leq\Big{(}\frac{e^{-x/2}}{2\sqrt{\pi}p}\Big{)}^{m}\cdot(1+2\epsilon)^{m}$
(S.13)

as $p\geq p(m,\epsilon)$, where $p(m,\epsilon)\geq 1$ is a constant that
depends on $m$ and $\epsilon$ only. Now we consider the decomposition

$\displaystyle P(|Z_{1}|>z,\cdots,|Z_{m}|>z)=\sum
P(\eta_{1}Z_{1}>z,\cdots,\eta_{m}Z_{m}>z),$ (S.14)

where the summation is over all the $2^{m}$ possible cases with $\eta_{1}=\pm
1,\cdots,\eta_{m}=\pm 1.$ Notice that
$|\mbox{Cov}(\eta_{i}Z_{i},\eta_{j}Z_{j})|=|\mbox{Cov}(Z_{i},Z_{j})|\leq\delta_{p}$,
and the derivation of (S.13) depends on $\delta_{p}$ rather than the exact
values of $\sigma_{ij}$’s. By (S.14), we have

$\displaystyle\Big{(}\frac{e^{-x/2}}{\sqrt{\pi}p}\Big{)}^{m}\cdot(1-2\epsilon)^{m}\leq
P(|Z_{1}|>z,\cdots,|Z_{m}|>z)\leq\Big{(}\frac{e^{-x/2}}{\sqrt{\pi}p}\Big{)}^{m}\cdot(1+2\epsilon)^{m}$

as $p\geq p(m,\epsilon)$. Based on the same reasoning, for any
$\epsilon\in(0,1/2)$, we have

$\displaystyle\Big{(}\frac{e^{-x/2}}{\sqrt{\pi}p}\Big{)}^{m}\cdot(1-2\epsilon)^{m}\leq
P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{m}}|>z)\leq\Big{(}\frac{e^{-x/2}}{\sqrt{\pi}p}\Big{)}^{m}\cdot(1+2\epsilon)^{m}$

as $p\geq p(m,\epsilon)$ uniformly for all $1\leq i_{1}<\cdots<i_{m}\leq R.$
Because the probability above does is independent of $\epsilon$, by letting
$p\to\infty$ and $\epsilon\downarrow 0$, we have that

$\displaystyle\Big{(}\frac{\sqrt{\pi}p}{e^{-x/2}}\Big{)}^{m}\cdot
P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{m}}|>z)\to 1$

as $p\to\infty$ uniformly for all $1\leq i_{1}<\cdots<i_{m}\leq R$, which
completes the proof. $\square$

For any $m\times m$ symmetric matrix $\mathbb{M}$, we use the notation
$\|\mathbb{M}\|$ to denote its spectral norm, that is,
$\|\mathbb{M}\|=\max\\{\lambda_{max}(\mathbb{M}),-\lambda_{min}(\mathbb{M})\\}$.
Evidently, $\|\mathbb{M}\|^{2}\leq\mbox{ tr}(\mathbb{M}^{T}\mathbb{M})$. Also,
$\|\mathbb{M}\|=\lambda_{max}(\mathbb{M})$ if $\mathbb{M}$ is non-negative
definite. Obviously, for any symmetric matrix $\mathbb{M}$, if
$\|\mathbb{M}\|\leq a$ then
$x^{T}\mathbb{M}x\leq\lambda_{max}(\mathbb{M})x^{T}x\leq ax^{T}x$ for any
$x\in\mathbb{R}^{m}$. The same argument applies to $-\mathbb{M}$. Thus, by
symmetry we know that

$\displaystyle-ax^{T}x\leq x^{T}\mathbb{M}x\leq ax^{T}x$ (S.15)

for any $x\in\mathbb{R}^{m}.$ The following lemma would be further needed.

######  LEMMA S.3

Let $m\geq 2$ and $Z_{1},\cdots,Z_{m}$ be $N(0,1)$-distributed random
variables with positive definite covariance matrix
$\mathbb{\Sigma}=(\sigma_{ij})_{m\times m}.$ For some $1>\varrho>\delta>0$,
assume $|\sigma_{12}|\leq\varrho$, and $|\sigma_{ij}|\leq\delta$ for all
$1\leq i<j\leq m$ but $(i,j)\neq(1,2)$. Then, if
$\delta\leq\frac{1}{8m^{2}}(1-\varrho)^{3}$, we have

$\displaystyle P(|Z_{1}|>z,\cdots,|Z_{m}|>z)\leq\frac{2^{m}}{z}\cdot
e^{-\alpha z^{2}/2}$ (S.16)

for all $z>0$, where $\alpha=m-\frac{1}{4}(\varrho+3).$

Proof of Lemma S.3. Let $\mathbb{1}=(1,\cdots,1)^{T}\in\mathbb{R}^{m}$ and

$\displaystyle\mathbb{a}=(a_{1},\cdots,a_{m})^{T}=\frac{\mathbb{\Sigma}^{-1}\mathbb{1}}{\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}}.$
(S.17)

Then $\mathbb{1}^{T}\mathbb{a}=1$. We claim that (which will be prove later)

$\displaystyle a_{i}\geq 0$ (S.18)

for all $i=1,2,\cdots,m.$ Assuming this is true, then obviously,

$\displaystyle P(Z_{1}>z,\cdots,Z_{m}>z)\leq
P((Z_{1},\cdots,Z_{m})\mathbb{a}\geq z).$

Let $Y=(Z_{1},\cdots,Z_{m})\mathbb{a}$. Then $Y\sim
N(0,\mathbb{a}^{T}\mathbb{\Sigma}\mathbb{a})$. By (S.17),
$\mathbb{a}^{T}\mathbb{\Sigma}\mathbb{a}=(\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1})^{-1}.$
Therefore,

$\displaystyle P(Z_{1}>z,\cdots,Z_{m}>z)\leq$ $\displaystyle
P\big{(}N(0,1)\geq(\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1})^{1/2}z\big{)}$
$\displaystyle\leq$
$\displaystyle\frac{1}{\sqrt{2\pi}}\cdot\frac{1}{(\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1})^{1/2}z}\cdot
e^{-(\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1})z^{2}/2}$ (S.19)

for any $z>0$, where in the last step we use a well-known inequality of the
Gaussian tail: $P(N(0,1)\geq y)\leq\frac{1}{\sqrt{2\pi}\,y}e^{-y^{2}/2}$ for
any $y>0$. Firstly, if $m=2$, then

$\displaystyle\mathbb{\Sigma}^{-1}=\frac{1}{1-\sigma_{12}^{2}}\begin{pmatrix}1&-\sigma_{12}\\\
-\sigma_{12}&1\end{pmatrix}.$

It is easy to check that
$\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}=\frac{2}{1+\sigma_{12}}\geq\frac{2}{1+\varrho}\geq
2-\frac{1}{4}(\varrho+3)=\alpha.$ Notice $\alpha>1$. We have from (S2.2) that

$\displaystyle P(Z_{1}>z,Z_{2}>z)\leq\frac{1}{z}\cdot e^{-\alpha z^{2}/2}.$
(S.20)

So the conclusion holds for $m=2$. From now on, we assume $m\geq 3$.

Step 1: the proof of (S.18). Define

$\displaystyle\mathbb{\Sigma}_{2}=\begin{pmatrix}1&\sigma_{12}\\\
\sigma_{12}&1\end{pmatrix}\ \ \mbox{and}\ \ \
\mathbb{A}=\begin{pmatrix}\mathbb{\Sigma}_{2}&\mathbb{0}\\\
\mathbb{0}^{T}&\mathbb{I}_{m-2}\end{pmatrix}$

where $\mathbb{0}$ is a $2\times(m-2)$ matrix whose entries are all equal to
zero. Trivially, the eigenvalues of $\mathbb{\Sigma}_{2}$ are $1+\sigma_{12}$
and $1-\sigma_{12}$, respectively. Basic algebra gives

$\displaystyle\mathbb{\Sigma}_{2}^{-1}=\frac{1}{1-\sigma_{12}^{2}}\begin{pmatrix}1&-\sigma_{12}\\\
-\sigma_{12}&1\end{pmatrix}\ \ \mbox{and}\ \
\mathbb{A}^{-1}=\begin{pmatrix}\mathbb{\Sigma}_{2}^{-1}&\mathbb{0}\\\
\mathbb{0}&\mathbb{I}_{m-2}\end{pmatrix},$ (S.21)

and the eigenvalues of $\mathbb{A}^{-1}$ are $1$ with $m-2$ folds,
$\frac{1}{1+\sigma_{12}}$ and $\frac{1}{1-\sigma_{12}}$, respectively, which
by assumption bounds the spectral norm as
$\|\mathbb{A}^{-1}\|\leq\frac{1}{1-\varrho}.$ Also,
$\|\mathbb{A}-\mathbb{\Sigma}\|^{2}\leq\mbox{tr}[(\mathbb{A}-\mathbb{\Sigma})^{T}(\mathbb{A}-\mathbb{\Sigma})]\leq
m^{2}\delta^{2}$ since $|\sigma_{ij}|\leq\delta$ for all $1\leq i<j\leq m$ but
$(i,j)\neq(1,2)$. By the fact that
$\mathbb{\Sigma}^{-1}-\mathbb{A}^{-1}=\mathbb{A}^{-1}(\mathbb{A}-\mathbb{\Sigma})\mathbb{\Sigma}^{-1}$,
we obtain

$\displaystyle\|\mathbb{\Sigma}^{-1}-\mathbb{A}^{-1}\|\leq$
$\displaystyle\|\mathbb{A}^{-1}\|\cdot\|\mathbb{A}-\mathbb{\Sigma}\|\cdot\|\mathbb{\Sigma}^{-1}\|\leq\frac{m\delta}{1-\varrho}\cdot\|\mathbb{\Sigma}^{-1}\|.$
(S.22)

In particular, this implies from the triangle inequality that

$\displaystyle\|\mathbb{\Sigma}^{-1}\|\leq\frac{1}{1-\varrho}+\frac{m\delta}{1-\varrho}\cdot\|\mathbb{\Sigma}^{-1}\|.$

By assumption $\delta\leq\frac{1}{8m^{2}}(1-\varrho)^{3}$, we know
$m\delta+\varrho<(1-\varrho)+\varrho=1$. By solving the inequality we obtain
$\|\mathbb{\Sigma}^{-1}\|\leq(1-\varrho-m\delta)^{-1}\leq\frac{2}{1-\varrho}$.
Substituting this to (S.22) we get

$\displaystyle\|\mathbb{\Sigma}^{-1}-\mathbb{A}^{-1}\|\leq\frac{2m\delta}{(1-\varrho)^{2}}.$
(S.23)

From (S.17), we know
$a_{i}=\mathbb{e}_{i}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}/\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}$
for $1\leq i\leq m$, where $\mathbb{e}_{i}=(0,\cdots,1,\cdots,0)^{T}$ and the
only “$1$” appears in the $i$th position. As $\mathbb{\Sigma}$ is positive
definite, $\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}>0.$ To show (S.18), it
is enough to show
$\tilde{a}_{i}:=\mathbb{e}_{i}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}>0$ for each
$i$. Define $h_{i}=\mathbb{e}_{i}^{T}\mathbb{A}^{-1}\mathbb{1}$, which equals
the $i$th row sum of $\mathbb{A}^{-1}$. We have

$\displaystyle|\tilde{a}_{i}-h_{i}|=|\mathbb{e}_{i}^{T}(\mathbb{\Sigma}^{-1}-\mathbb{A}^{-1})\mathbb{1}|\leq$
$\displaystyle\|\mathbb{e}_{i}\|\cdot\|(\mathbb{\Sigma}^{-1}-\mathbb{A}^{-1})\mathbb{1}\|$
$\displaystyle\leq$
$\displaystyle\|\mathbb{e}_{i}\|\cdot\|\mathbb{\Sigma}^{-1}-\mathbb{A}^{-1}\|\cdot\|\mathbb{1}\|$
$\displaystyle\leq$ $\displaystyle\frac{2m^{2}\delta}{(1-\varrho)^{2}}$ (S.24)

induced by (S.23). Therefore, $\tilde{a}_{i}\geq
h_{i}-\frac{2m^{2}\delta}{(1-\varrho)^{2}}$. Now observe from (S.21) that

$\displaystyle
h_{1}=h_{2}=\frac{1-\sigma_{12}}{1-\sigma_{12}^{2}}=\frac{1}{1+\sigma_{12}}\geq\frac{1}{1+\varrho},$

and $h_{i}=1$ for $i=3,\cdots,m.$ Thus, (S2.2) and condition
$\delta\leq\frac{1}{8m^{2}}(1-\varrho)^{3}\leq\frac{1}{2m^{2}}\frac{(1-\varrho)^{3}}{1+\varrho}$
conclude that $\tilde{a}_{i}>0$ for each $i$, which implies (S.18).

Step 2: the proof of (S.16). From (S.15) and (S.23),

$\displaystyle\mathbb{1}^{T}(\mathbb{\Sigma}^{-1}-\mathbb{A}^{-1})\mathbb{1}\geq-\frac{(2m\delta)\mathbb{1}^{T}\mathbb{1}}{(1-\varrho)^{2}}=-\frac{2m^{2}\delta}{(1-\varrho)^{2}}.$

As a result, we have

$\displaystyle\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}$
$\displaystyle\geq$
$\displaystyle\mathbb{1}^{T}\mathbb{A}^{-1}\mathbb{1}-\frac{2m^{2}\delta}{(1-\varrho^{2})^{2}}$
$\displaystyle=$
$\displaystyle\frac{2-2\sigma_{12}}{1-\sigma_{12}^{2}}+m-2-\frac{2m^{2}\delta}{(1-\varrho)^{2}}$
$\displaystyle\geq$ $\displaystyle
m-1+\frac{1-\varrho}{1+\varrho}-\frac{2m^{2}\delta}{(1-\varrho)^{2}}$

by using the assumption $|\sigma_{12}|\leq\varrho$. By assumption
$\delta\leq\frac{1}{8m^{2}}(1-\varrho)^{3}$, we further obtain

$\displaystyle\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}\geq
m-1+\frac{1-\varrho}{2(1+\varrho)}.$

In particular, $\mathbb{1}^{T}\mathbb{\Sigma}^{-1}\mathbb{1}\geq
m-1\geq\frac{1}{4}m$ for $m\geq 3.$ Then, we can establish from (S2.2) that

$\displaystyle
P(Z_{1}>z,\cdots,Z_{m}>z)\leq\frac{1}{z\sqrt{m}}\exp\Big{\\{}-\frac{z^{2}}{2}\Big{(}m-1+\frac{1-\varrho}{2(1+\varrho)}\Big{)}\Big{\\}},$
(S.25)

under the assumption $\delta\leq\frac{1}{8m^{2}}(1-\varrho)^{3}$. In the above
derivation, the true values of $\sigma_{ij}$’s are not used, instead their
bounds $\varrho$ and $\delta$ are relevant. Therefore, (S.20) and (S.25) still
hold if each “$Z_{i}$” is replaced by “$\eta_{i}Z_{i}$” with $\eta_{i}=\pm 1$.
Trivially, $-1+\frac{1-\varrho}{2(1+\varrho)}\geq-\frac{1}{4}(\varrho+3)$ for
each $\varrho\in(0,1)$. This combining with (S.25) and (S.14) yields (S.16).
$\square$

To prove Theorem 2, we need a notation. Let $p\geq 2$ and
$(\sigma_{ij})_{p\times p}$ be a non-negative definite matrix. For $\delta>0$
and a set $A\subset\\{1,2,\cdots,m\\}$ with $2\leq m\leq p$, define

$\displaystyle\wp(A)=\max\Big{\\{}|S|;\,S\subset A\ \mbox{and}\ \max_{i\in
S,j\in S,i\neq j}|\sigma_{ij}|\leq\delta\Big{\\}}.$ (S.26)

Specifically, $\wp(A)$ takes possible values $0,2\cdots,|A|$, where we regard
$|\varnothing|=0$. If $\wp(A)=0$, then $|\sigma_{ij}|>\delta$ for all $i\in A$
and $j\in A.$

Proof of Theorem 2. For any $x\in\mathbb{R}$, write

$\displaystyle z=\big{(}2\log p-\log\log p+x\big{)}^{1/2},$ (S.27)

which is well defined as $p$ is sufficiently large. We will not mention this
matter again since the conclusion is valid as $p\to\infty.$ It suffices to
show

$\displaystyle\lim_{p\to\infty}P\Big{(}\max_{1\leq i\leq
p}|Z_{i}|>z\Big{)}=1-\exp\Big{(}-\frac{1}{\sqrt{\pi}}e^{-x/2}\Big{)}$ (S.28)

as $p\to\infty.$ The proof will be divided into a few steps.

Step 1: reducing “$\\{1\leq i\leq p\\}$” in (S.28) to a set of friendly
indices. First,

$\displaystyle P\big{(}|N(0,1)|\geq z\big{)}\sim$
$\displaystyle\frac{2}{\sqrt{2\pi}z}e^{-z^{2}/2}$ $\displaystyle\sim$
$\displaystyle\frac{1}{\sqrt{\pi}}\cdot\frac{1}{\sqrt{\log
p}}\exp\Big{\\{}-\frac{1}{2}\big{(}2\log p-\log\log p+x\big{)}\Big{\\}}$
$\displaystyle\sim$ $\displaystyle\frac{1}{\sqrt{\pi}}\cdot\frac{e^{-x/2}}{p}$
(S.29)

as $p\to\infty$, where for two sequences of real numbers $A_{p}$ and $H_{p}$,
the notion $A_{p}\sim H_{p}$ means that $A_{p}/H_{p}\to 1$ as $p\to\infty.$
Immediately, a union bound implies

$\displaystyle P\Big{(}\max_{i\in C_{p}}|Z_{i}|>z\Big{)}\leq|C_{p}|\cdot
P\big{(}|N(0,1)|\geq z\big{)}\to 0,$

as $p\to\infty$, where we recall the definition $C_{p}:=\\{1\leq i\leq
p;\,|B_{p,i}|\geq p^{\kappa}\\}$ with $B_{p,i}=\big{\\{}1\leq j\leq
p;\,|\sigma_{ij}|\geq\delta_{p}\big{\\}}$. Further denote $D_{p}:=\\{1\leq
i\leq p;\,|B_{p,i}|<p^{\kappa}\\}$. By assumption, $|D_{p}|/p\to 1$ as
$p\to\infty$. It follows that

$\displaystyle P\Big{(}\max_{i\in D_{p}}|Z_{i}|>z\Big{)}$ $\displaystyle\leq$
$\displaystyle P\Big{(}\max_{1\leq i\leq p}|Z_{i}|>z\Big{)}$
$\displaystyle\leq$ $\displaystyle P\Big{(}\max_{i\in
D_{p}}|Z_{i}|>z\Big{)}+P\Big{(}\max_{i\in C_{p}}|Z_{i}|>z\Big{)}.$

Therefore, to prove (S.28), it is enough to show

$\displaystyle\lim_{p\to\infty}P\Big{(}\max_{i\in
D_{p}}|Z_{i}|>z\Big{)}=1-\exp\Big{(}-\frac{1}{\sqrt{\pi}}e^{-x/2}\Big{)},$
(S.30)

as $p\to\infty$ asymptotically.

Step 2: estimation of $P(\max_{i\in D_{p}}|Z_{i}|>z)$ via the inclusion-
exclusion formula. Set

$\displaystyle\alpha_{t}=\sum P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)$ (S.31)

for $1\leq t\leq p$, where the sum runs over all $i_{1}\in
D_{p},\cdots,i_{t}\in D_{p}$ such that $i_{1}<\cdots<i_{t}$. Then,

$\displaystyle\sum_{t=1}^{2k}(-1)^{t-1}\alpha_{t}\leq P\Big{(}\max_{i\in
D_{p}}|Z_{i}|>z\Big{)}\leq\sum_{t=1}^{2k+1}(-1)^{t-1}\alpha_{t}$ (S.32)

for any $k\geq 1.$ We will prove next that

$\displaystyle\lim_{p\to\infty}\alpha_{t}=\frac{1}{t!}\pi^{-t/2}e^{-tx/2}$
(S.33)

for each $t\geq 1.$ Assuming this is true, let $p\to\infty$ in (S.32), we have

$\displaystyle\sum_{t=1}^{2k}(-1)^{t-1}\frac{1}{t!}\Big{(}\frac{1}{\sqrt{\pi}}e^{-x/2}\Big{)}^{t}\leq\liminf_{p\to\infty}P\Big{(}\max_{i\in
D_{p}}|Z_{i}|>z\Big{)}$
$\displaystyle\leq\limsup_{p\to\infty}P\Big{(}\max_{i\in
D_{p}}|Z_{i}|>z\Big{)}$ $\displaystyle\leq$
$\displaystyle\sum_{t=1}^{2k+1}(-1)^{t-1}\frac{1}{t!}\Big{(}\frac{1}{\sqrt{\pi}}e^{-x/2}\Big{)}^{t}$

for each $k\geq 1$. By letting $k\to\infty$ and using the Taylor expansion of
the function $f(x)=1-e^{-x}$, we obtain (S.30). It remains to verify (S.33).
Evidently, by (S2.2) and the assumption $|D_{p}|/p\to 1$, we immediately see
(S.33) holds as $t=1.$ Now we prove (S.33) for any $t\geq 2$.

Recalling $D_{p}:=\\{1\leq i\leq p;\,|B_{p,i}|<p^{\kappa}\\}$, we write

$\displaystyle\big{\\{}(i_{1},\cdots,i_{t})\in(D_{p})^{t};\,i_{1}<\cdots<i_{t}\big{\\}}=F_{t}\cup
G_{t},$

where

$\displaystyle F_{t}:=$
$\displaystyle\big{\\{}(i_{1},\cdots,i_{t})\in(D_{p})^{t};\,i_{1}<\cdots<i_{t}\
\mbox{and}\ |\sigma_{i_{r}i_{s}}|\leq\delta_{p}\ \mbox{for all}\ 1\leq r<s\leq
t\\};$ $\displaystyle G_{t}:=$
$\displaystyle\big{\\{}(i_{1},\cdots,i_{t})\in(D_{p})^{t};\,i_{1}<\cdots<i_{t}\
\mbox{and}\ |\sigma_{i_{r}i_{s}}|>\delta_{p}\ \mbox{ for a pair}\ (r,s)\
\mbox{with}\ 1\leq r<s\leq t\big{\\}}.$ (S.34)

Now, think $D_{p}$ as graph with $|D_{p}|$ vertices, with $|D_{p}|\leq p$ and
$|D_{p}|/p\to 1$ by assumption. Any two different vertices from $D_{p}$, say,
$i$ and $j$ are connected if $|\sigma_{ij}|>\delta_{p}.$ In this case we also
say there is an edge between them. By the definition $D_{p}$, each vertex in
the graph has at most $p^{\kappa}$ neighbors. Replacing “$n$”, “$q$” and “$t$”
in Lemma S.1(i) with “$|D_{p}|$”, “$p^{\kappa}$” and “$t$”, respectively, we
have that $|G_{t}|\leq p^{t+\kappa-1}$ for each $2\leq t\leq p$. Therefore,
$\binom{|D_{p}|}{t}\geq|F_{t}|\geq\binom{|D_{p}|}{t}-p^{t+\kappa-1}$. Since
$D_{p}/p\to 1$ and $\kappa=\kappa_{p}\to 0$ as $p\to\infty$, we know

$\displaystyle\lim_{p\to\infty}\frac{|F_{t}|}{p^{t}}=\frac{1}{t!}.$ (S.35)

Decomposing (S.31), we see that

$\displaystyle\alpha_{t}=\sum_{(i_{1},\cdots,i_{t})\in
F_{t}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)+\sum_{(i_{1},\cdots,i_{t})\in
G_{t}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z).$

From Lemma S.2 and (S.35) we have

$\displaystyle\sum_{(i_{1},\cdots,i_{t})\in
F_{t}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)\to\frac{1}{t!}\Big{(}\frac{e^{-x/2}}{\sqrt{\pi}}\Big{)}^{t}=\frac{1}{t!}\pi^{-t/2}e^{-tx/2},$

as $p\to\infty.$ As a consequence, to derive (S.33), we only need to show that

$\displaystyle\sum_{(i_{1},\cdots,i_{t})\in
G_{t}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)\to 0$ (S.36)

as $p\to\infty$ asymptotically for each $t\geq 2.$

Step 3: the proof of (S.36). If $t=2$, the sum of probabilities in (S.36) is
bounded by $|G_{2}|\cdot\max_{1\leq i<j\leq p}P(|Z_{i}|>z,|Z_{j}|>z)$. By
Lemma S.1(i), $|G_{2}|\leq p^{\kappa+1}$. Since $|\sigma_{ij}|\leq\varrho$, by
Lemma S.3,

$\displaystyle
P(|Z_{i}|>z,|Z_{j}|>z)\leq\exp\big{[}-(5-\varrho)z^{2}/8\big{]}\leq\frac{(\log
p)^{C}}{p^{(5-\varrho)/4}}$ (S.37)

uniformly for all $1\leq i<j\leq p$ as $p$ is sufficiently large, where $C>0$
is a constant not depending on $p.$ We then know (S.36) holds. The remaining
job is to prove (S.36) for $t\geq 3.$

Take $\delta=\delta_{p}$ in (S.26) for the definition of $\wp(A)$ and compare
it with $G_{t}$ from (S.34). To proceed, we further classify $G_{t}$ into the
following subsets

$\displaystyle G_{t,j}=\big{\\{}(i_{1},\cdots,i_{t})\in
G_{t};\,\wp(\\{i_{1},\cdots,i_{t}\\})=j\big{\\}},$

for $j=0,2,\cdots,t-1.$ By the definition of $G_{t}$, we know that
$G_{t}=\cup_{j=0,2,\cdots,t-1}G_{t,j}$. Since $t\geq 3$ is fixed, to show
(S.36), it suffices to prove

$\displaystyle\sum_{(i_{1},\cdots,i_{t})\in
G_{t,j}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)\to 0$ (S.38)

for all $j\in\\{0,2,\cdots,t-1\\}$.

Assume $(i_{1},\cdots,i_{t})\in G_{t,0}$, which implies
$|\sigma_{i_{r}i_{s}}|>\delta_{p}$ for all $1\leq r<s\leq t$ by (S.26). Hence,
the subgraph $\\{i_{1},\cdots,i_{t}\\}\subset G_{t}$ is a clique. Taking
$n=|D_{p}|\leq p$, $t=t$ and $q=p^{\kappa}$ into Lemma (S.1)(ii), we get
$|G_{t,0}|\leq p^{1+\kappa(t-1)}\leq p^{1+t\kappa}$. Thus, we obtain

$\displaystyle\eqref{what_tie}\leq p^{1+t\kappa}\cdot\max_{1\leq i<j\leq
p}P(|Z_{i}|>z,|Z_{j}|>z)\leq p^{1+t\kappa}\cdot\frac{(\log
p)^{C}}{p^{(5-\varrho)/4}}\to 0$ (S.39)

as $p\to\infty$ by using (S.37). So, (S.38) holds with $t=0.$

Now we assume $(i_{1},\cdots,i_{t})\in G_{t,j}$ with $j\in\\{2,\cdots,t-1\\}$.
By definition, there exits $S\subset\\{i_{1},\cdots,i_{t}\\}$ such that
$\max_{i\in S,j\in S,i\neq j}|\sigma_{ij}|\leq\delta_{p}$ and for each
$k\in\\{i_{1},\cdots,i_{t}\\}\backslash S$, there exists $i\in S$ satisfying
$|\sigma_{ik}|>\delta_{p}$. Looking at the last statement we see two
possibilities: (i) for each $k\in\\{i_{1},\cdots,i_{t}\\}\backslash S$, there
exist at leat two indices, say, $i\in S$, $j\in S$ with $i\neq j$ satisfying
$|\sigma_{ik}|>\delta_{p}$ and $|\sigma_{jk}|>\delta_{p}$; (ii) there exists
$k\in\\{i_{1},\cdots,i_{t}\\}\backslash S$ for which
$|\sigma_{ik}|>\delta_{p}$ for an unique $i\in S$. However, for
$(i_{1},\cdots,i_{t})\in G_{t,j}$, (i) and (ii) could happen at the same time
for different $S$, say, (i) holds for $S_{1}$ and (ii) holds for $S_{2}$
simultaneously. Thus, to differentiate the two cases, we consider the
following two types of sets. Denote

$\displaystyle H_{t,j}=$ $\displaystyle\big{\\{}(i_{1},\cdots,i_{t})\in
G_{t,j};\,\ \mbox{there exist}\ S\subset\\{i_{1},\cdots,i_{t}\\}\ \mbox{with}\
|S|=j\ \mbox{and}$ $\displaystyle\max_{i\in S,j\in S,i\neq
j}|\sigma_{ij}|\leq\delta_{p}\ \mbox{such that}\ \mbox{for any}\
k\in\\{i_{1},\cdots,i_{t}\\}\backslash S\ \mbox{there exist}\ r\in S,$
$\displaystyle s\in S,r\neq s\ \mbox{satisfying}\
\min\\{|\sigma_{kr}|,|\sigma_{ks}|\\}>\delta_{p}\big{\\}}.$ (S.40)

Replacing “$n$”, “$q$” and “$t$” in Lemma S.1(iii) with “$|D_{p}|$”,
“$p^{\kappa}$” and “$t$”, respectively, we have that $|H_{t,j}|\leq t^{t}\cdot
p^{j-1+(t-j+1)\kappa}$ for each $t\geq 3$. Analogously, set

$\displaystyle H_{t,j}^{\prime}=$
$\displaystyle\big{\\{}(i_{1},\cdots,i_{t})\in G_{t,j};\,\ \mbox{for any}\
S\subset\\{i_{1},\cdots,i_{t}\\}\ \mbox{with}\ |S|=j\ \mbox{and}$
$\displaystyle\max_{i\in S,j\in S,i\neq j}|\sigma_{ij}|\leq\delta_{p}\
\mbox{there exists}\ k\in\\{i_{1},\cdots,i_{t}\\}\backslash S\ \mbox{such
that}\ |\sigma_{kr}|>\delta_{p}$ $\displaystyle\mbox{for a unique}\ r\in
S\\}.$ (S.41)

From Lemma S.1(iv) we see $|H_{t,j}^{\prime}|\leq t^{t}\cdot
p^{j+(t-j)\kappa}.$ It is easy to see $G_{t,j}=H_{t,j}\cup H_{t,j}^{\prime}$.
Therefore, to show (S.38), we only need to prove both

$\displaystyle\sum_{(i_{1},\cdots,i_{t})\in
H_{t,j}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)\to 0$ (S.42)

and

$\displaystyle\sum_{(i_{1},\cdots,i_{t})\in
H_{t,j}^{\prime}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)\to 0$ (S.43)

as $p\to\infty$ for $j=2,\cdots,t-1$. In fact, let $S$ be as in (S.40), then
using Lemma S.2, the probability in (S.42) is bounded by $P(\bigcap_{l\in
S}\\{|Z_{l}|>z\\})\leq C\cdot p^{-j}$ uniformly for all $S$ as $p$ is
sufficiently large, where $C$ is a constant independent of $p$. This leads to

$\displaystyle\sum_{(i_{1},\cdots,i_{t})\in
H_{t,j}}P(|Z_{i_{1}}|>z,\cdots,|Z_{i_{t}}|>z)\leq$ $\displaystyle t^{t}\cdot
p^{j-1+(t-j+1)\kappa}\cdot\big{(}C\cdot p^{-j}\big{)}\leq(Ct^{t})\cdot
p^{-1+t\kappa},$

as $p$ is sufficiently large. By the assumption that $\kappa=\kappa_{p}\to 0$,
we arrive at (S.42).

Finally we validate (S.43). Recall the definition of $H_{t,j}^{\prime}$. For
$(i_{1},\cdots,i_{t})\in H_{t,j}^{\prime}$, pick
$S\subset\\{i_{1},\cdots,i_{t}\\}$ with $|S|=j$, $\max_{i\in S,j\in S,i\neq
j}|\sigma_{ij}|\leq\delta_{p}$ and $k\in\\{i_{1},\cdots,i_{t}\\}\backslash S$
such that $\delta_{p}<|\sigma_{kr}|\leq\varrho$ for a unique $r\in S.$ Then
each probability in (S.43) is bounded by

$\displaystyle P\Big{(}|Z_{k}|>z,\bigcap_{l\in S}\\{|Z_{l}|>z\\}\Big{)},$

for $2\leq j\leq t-1$. Taking $m=j+1$ and applying Lemma S.3, the probability
above is dominated by

$\displaystyle\frac{2^{j+1}}{z}\cdot\exp\Big{\\{}-\frac{z^{2}}{2}\Big{(}j+\frac{1-\varrho}{4}\Big{)}\Big{\\}}=O\Big{(}\frac{(\log
p)^{c}}{p^{j+(1-\varrho)/4}}\Big{)},$

for some constant $c$. As stated earlier, $|H_{t,j}^{\prime}|\leq t^{t}\cdot
p^{j+(t-j)\kappa}.$ by union bound, since $\kappa=\kappa_{p}\to 0$, we see the
sum from (S.43) is of order $O(p^{-(1-\varrho)/8})$. Hence, (S.43) holds. We
have proved (S.38) for any $j\in\\{0,2,\cdots,t-1\\}$, which concludes the
proof. $\square$

###  S2.3 Proof of Theorem 3

The proof of Theorem 3 is involved. A preparation with a few of lemmas is
given below.

######  LEMMA S.4

Let $\mathbb{A}$ and $\mathbb{B}$ be nonnegative definite matrices. Then

$\displaystyle\mbox{tr}\,(\mathbb{A}\mathbb{B})\leq\lambda_{max}(\mathbb{A})\,\mbox{tr}(\mathbb{B}).$

Proof of Lemma S.4. Assume $\mathbb{A}$ and $\mathbb{B}$ are $n\times n$
matrices. There is an orthogonal matrix $\mathbb{O}$ such that
$\mathbb{A}=\mathbb{O}^{T}\mathbb{\Lambda}\mathbb{O}$, where
$\mathbb{\Lambda}=\mbox{diag}(\lambda_{1},\cdots,\lambda_{n})$ and
$\lambda_{1}\geq\cdots\geq\lambda_{n}\geq 0.$ Observe
$\mbox{tr}\,(\mathbb{A}\mathbb{B})=\mbox{tr}\,[\mathbb{\Lambda}(\mathbb{O}\mathbb{B}\mathbb{O}^{T})]$,
$\mbox{tr}\,(\mathbb{O}\mathbb{B}\mathbb{O}^{T})=\mbox{tr}(\mathbb{B})$ and
$\lambda_{max}(\mathbb{A})=\lambda_{max}(\mathbb{\Lambda})$. Thus, without
loss of generality, we assume $\mathbb{A}=\mathbb{\Lambda}$. Write
$\mathbb{B}=(b_{ij})$. Then, $b_{ii}\geq 0$ for each $i$ and

$\displaystyle\mbox{tr}\,(\mathbb{A}\mathbb{B})=\sum_{i=1}^{n}\lambda_{i}b_{ii}\leq\lambda_{1}\cdot\sum_{i=1}^{n}b_{ii}=\lambda_{max}(\mathbb{A})\,\mbox{tr}(\mathbb{B}).$

The proof is completed. $\square$.

The following is a well-known formula for the conditional distributions of
multivariate normal distributions; see, for example, p. 12 from Muirhead
(1982).

######  LEMMA S.5

Let $\mathbb{X}\sim N(\boldsymbol{\mu},\mathbb{\Sigma})$ with
$\mathbb{\Sigma}$ being invertible. Partition $\mathbb{X},\boldsymbol{\mu}$
and $\mathbb{\Sigma}$ as

$\displaystyle\mathbb{X}=\begin{pmatrix}\mathbb{X}_{1}\\\
\mathbb{X}_{2}\end{pmatrix},\ \ \ \
\mathbb{{\boldsymbol{\mu}}}=\begin{pmatrix}\mathbb{{\boldsymbol{\mu}}}_{1}\\\
\mathbb{{\boldsymbol{\mu}}}_{2}\end{pmatrix},\ \ \ \
\mathbb{\Sigma}=\begin{pmatrix}\mathbb{\Sigma}_{11}&\mathbb{\Sigma}_{12}\\\
\mathbb{\Sigma}_{21}&\mathbb{\Sigma}_{22}\end{pmatrix}$ (S.44)

where $\mathbb{X}_{2}\sim
N(\mathbb{{\boldsymbol{\mu}}}_{2},\mathbb{\Sigma}_{22})$. Set
$\mathbb{\Sigma}_{22\cdot
1}=\mathbb{\Sigma}_{22}-\mathbb{\Sigma}_{21}\mathbb{\Sigma}_{11}^{-1}\mathbb{\Sigma}_{12}$.
Then
$\mathbb{X}_{2}-\mathbb{\Sigma}_{21}\mathbb{\Sigma}_{11}^{-1}\mathbb{X}_{1}\sim
N(\mathbb{{\boldsymbol{\mu}}}_{2}-\mathbb{\Sigma}_{21}\mathbb{\Sigma}_{11}^{-1}\mathbb{{\boldsymbol{\mu}}}_{1},\mathbb{\Sigma}_{22\cdot
1})$ and is independent of $\mathbb{X}_{1}$.

######  LEMMA S.6

Let $(Z_{1},\cdots,Z_{p})^{T}\sim N(\mathbb{0},\mathbb{\Sigma})$. Under the
notion of Lemma S.5, for $1\leq d<p$, write
${\boldsymbol{X}}_{1}=(Z_{1},\cdots,Z_{d})^{T}$ and
${\boldsymbol{X}}_{2}=(Z_{d+1},\cdots,Z_{p})^{T}$. Define
$\mathbb{U}_{p}={\boldsymbol{X}}_{2}-{\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{X}}_{1}$
and
$\mathbb{V}_{p}={\boldsymbol{\Sigma}}_{21}{\boldsymbol{\Sigma}}_{11}^{-1}{\boldsymbol{X}}_{1}$.
Assume $\mathbb{\Sigma}=(\sigma_{ij})_{p\times p}$ with
$\sum_{j=1}^{p}\sigma_{ij}^{2}\leq K_{p}$ for each $1\leq i\leq p$, where
$K_{p}$ is a constant depending on $p$ only. Then there exists a constant