# Distance-based regression analysis for measuring associations∗

SHI Yuke $\cdot$ ZHANG Wei $\cdot$ LIU Aiyi $\cdot$ LI Qizhai SHI Yuke
_LSC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences,
Beijing 100190, China; University of Chinese Academy of Sciences, Beijing
100049, China._<EMAIL_ADDRESS>
ZHANG Wei
_LSC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences,
Beijing 100190, China._<EMAIL_ADDRESS>
LIU Aiyi
_Biostatistics and Bioinformatics Branch, Eunice Kennedy Shriver National
Institute of Child Health; Human Development, National Institutes of Health,
Bethesda, MD 20847, USA._<EMAIL_ADDRESS>
LI Qizhai (Corresponding author)
_LSC, Academy of Mathematics and Systems Science, Chinese Academy of Sciences,
Beijing 100190, China; University of Chinese Academy of Sciences, Beijing
100049, China._<EMAIL_ADDRESS>
∗This work was partially supported by Beijing Natural Science Foundation
(Z180006).

DOI:

Received: x x 20xx / Revised: x x 20xx

©The Editorial Office of JSSC & Springer-Verlag GmbH Germany 2018

Distance-based regression model, as a nonparametric multivariate method, has
been widely used to detect the association between variations in a distance or
dissimilarity matrix for outcomes and predictor variables of interest in
genetic association studies, genomic analyses, and many other research areas.
Based on it, a pseudo-$F$ statistic which partitions the variation in distance
matrices is often constructed to achieve the aim. To the best of our
knowledge, the statistical properties of the pseudo-$F$ statistic has not yet
been well established in the literature. To fill this gap, we study the
asymptotic null distribution of the pseudo-$F$ statistic and show that it is
asymptotically equivalent to a mixture of chi-squared random variables. Given
that the pseudo-$F$ test statistic has unsatisfactory power when the
correlations of the response variables are large, we propose a square-root
$F$-type test statistic which replaces the similarity matrix with its square
root. The asymptotic null distribution of the new test statistic and power of
both tests are also investigated. Simulation studies are conducted to validate
the asymptotic distributions of the tests and demonstrate that the proposed
test has more robust power than the pseudo-$F$ test. Both test statistics are
exemplified with a gene expression dataset for a prostate cancer pathway.

Asymptotic distribution, Chi-squared-type mixture, Nonparametric test,
Pseudo-$F$ test, Similarity matrix.

##  1 Introduction

Distance-based regression model, as a multivariate method testing the
relationship between variation in a distance or dissimilarity matrix of
outcomes and predictor variables of interest, has been widely used in a
variety of applications including genetics [8], genomics [26, 15], microbiome
[24, 16, 12, 20], and other research area such as geoscience
[14],environmental science [22], and oceanography [1, 4]. Unlike conventional
multivariate methods, the starting point of distance-based regression model is
pairwise distance or dissimilarity of subjects instead of individuals’
observations. This model was originally introduced by McArdle and Anderson
[13] to analyze simultaneous responses of multiple species to several factors
in ecological experimental designs. Subsequently, with the development of
high-throughput technologies, it has been received considerable attentions in
high-dimensional data analysis. For example, Wessel and Schork [21] applied
distance-based regression model to relate variations measured by a genomic
distance for multiple phenotypes with multiple variants; Zapala and Schork
[26] proposed a multivariate regression approach based on a distance matrix to
detect associations between expression patterns on groups of genes and related
predictor variables; Chen et al. [3] developed a distance-based statistical
test based on generalized UniFrac distances to detect the association of
microbiome composition and environment covariates. Gambi et al. [5] applied it
to detect the impaction of historical sulfide mine tailings discharge on
meiofaunal assemblages.

By partitioning the variation in a distance or dissimilarity matrix, a
pseudo-$F$ test statistic, analogous to the conventional univariate $F$
statistic in ANOVA or general linear models, is constructed to detect the
association between multiple response and predictor variables [18]. However,
to the best of our knowledge, the distribution or asymptotic distribution of
the pseudo-$F$ test statistic is unknown with only some results under normal
assumption and Euclidean or Mahalanobis distance measures [9]. For the general
cases, it has not been studied yet in the literature. Resampling procedures
are often used to calculate the statistical significance of the pseudo-$F$
test statistic [13, 21, 26, 11, 8, 3]. However, they are generally
computationally intensive, especially for high-dimensional data.

To address this problem, we study the asymptotic distribution of the
pseudo-$F$ statistic. We show that under the null hypothesis, the pseudo-$F$
statistic is asymptotically equivalent to a mixture of Chi-squared random
variables, with the weights proportional to the eigenvalues of the similarity
matrix. Such Chi-squared-type mixture variables could have a large critical
value when the correlation coefficients of the response variables are large,
since in this case the eigenvalues of the similarity matrix vary widely. Thus
the pseudo-$F$ statistic may suffer from a substantial power loss. So we
propose a square-root $F$-type test statistic, which reduces the difference in
the eigenvalues of the similarity matrix by replacing the similarity matrix
with its square root. The asymptotic null distribution is also established for
the new test statistic.

The rest of the article is organized as follows. We introduce the pseudo-$F$
test statistic and the proposed test statistic in Section 2. Main theoretical
results for the test statistics are given in Section 3. Simulation studies are
conducted to validate the asymptotic distribution of the pseudo-$F$ and
square-root $F$-type test statistics in Section 4. In Section 5, we illustrate
the two tests with gene expression data from a prostate cancer pathway study.
Some conclusions are drawn in Section 6. Technical details are relegated to
the Appendix.

##  2 Test statistics

We introduce the pseudo-$F$ test statistic by taking a similarity matrix as
the starting point since the distance matrix can be easily transferred to a
similarity matrix. Let $S=(s_{ij})_{n\times n}$ be a similarity matrix for $k$
response variables among $n$ subjects, where $s_{ij}$ is the pairwise
similarity between the subjects $i$ and $j$ constructed based on a positive
definite kernel $\psi(y_{i},y_{j})$, and $y_{i}$ and $y_{j}$ are two high-
dimensional independent observations of the responses variable $\mathbb{Y}$,
$i,j=1,\cdots,n$. Let
$\mathbb{X}=(\mathbb{X}_{1},\cdots,\mathbb{X}_{m})^{\top}$ be $m$ predictor
variables of interest. Denote the observation of $\mathbb{X}$ for the $i$th
subject by $x_{i}=(x_{i1},\cdots,x_{im})^{\top}$, $i=1,\cdots,n$. Write
$X=(x_{1},\cdots,x_{n})^{\top}$. We refer to the regression model relating $S$
and $X$ as a distance-based regression model, denoted by $S\sim X$. Our
interest is in testing the null hypothesis $H_{0}$: there is no association
between $\mathbb{Y}$ and $\mathbb{X}$. A $F$-type test statistic proposed by
McArdle and Anderson [13, page. 3] can be used to do it, which is given by

$T_{pseudo}=\frac{\text{tr}\big{(}H_{X}{{\bf H}S{\bf
H}}/m\big{)}}{\text{tr}\big{(}({\bf I}_{n}-{H_{X}}){\bf H}S{\bf
H}/(n-m)\big{)}},$

where $H_{X}=X(X^{\top}X)^{-1}X^{\top}$ is the traditional hat” matrix, ${\bf
H}={\bf I}_{n}-{\bf 1}_{n}{\bf 1}_{n}^{\top}/n$, ${\bf I}_{n}$ is the $n\times
n$ identity matrix, ${\bf 1}_{n}$ is an $n$-dimensional column vector with all
elements being 1, and $\text{tr}(\cdot)$ represents the trace of a matrix.
Since the distribution or asymptotic distribution of $T_{pseudo}$ is unknown,
its statistical significance is conventionally calculated using resampling
procedures, which is usually computationally intensive, especially when $n$ is
large. One of the goals of this article is to establish the asymptotic
distribution for the pseudo-$F$ test statistic $T_{pseudo}$.

The numerical results in Section 4 below show that $T_{pseudo}$ is sensitive
to the correlation matrix of the response variables. To boost the power when
the response variables are moderately or highly correlated, we propose a
square-root $F$-type test statistic, which uses the square root of the
similarity matrix. The square-root $F$-type statistic is defined as

$T_{sqrt}=\frac{\text{tr}\big{(}H_{X}({\bf H}S{\bf
H})^{1/2}/m\big{)}}{\text{tr}\big{(}({\bf I}_{n}-H_{X})({\bf H}S{\bf
H})^{1/2}/(n-m)\big{)}},$

where the square root $B$ of a matrix $A$ means $A=B\times B$. Consider the
eigenvalues $\\{\lambda_{i}\\}_{i=1}^{n}$ and eigenfunctions
$\\{v_{i}\\}_{i=1}^{n}$ of $\mathaccent 869{S}={\bf H}S{\bf H}$ and
$\mathaccent
869{S}^{1/2}=(v_{1},\cdots,v_{n})diag(\lambda_{1}^{1/2},\cdots,\lambda_{n}^{1/2})(v_{1},\cdots,v_{n})^{\top}$.
The numerator of $T_{pseudo}$ and $T_{sqrt}$ now can be intuitively expressed
as $\sum_{i=1}^{n}\lambda_{i}v_{i}^{\top}\mathaccent 869{v}_{i}/m$ and
$\sum_{i=1}^{n}\lambda_{i}^{1/2}v_{i}^{\top}\mathaccent 869{v}_{i}/m$ where
the eigenvalues of $H_{x}$ are all $1$ and the eigenfunctions of $H_{x}$ are
$\\{\mathaccent 869{v}_{i}\\}_{i=1}^{n}$. Superficially, when the response
variables are moderately correlated, our square-root method may increase the
weight of valuable factors to boost the power.

##  3 Theoretic properties and implementation

In this section, we derive the asymptotic distributions of the pseudo-$F$ and
the proposed test statistics. Assume that $E(\mathbb{X})=\mu$ and
$\text{cov}(\mathbb{X})=\Delta=(\delta_{ij})_{m\times m}$. It thus follows
that $E(\mathbb{X}\mathbb{X}^{\top})=\Delta+\mu\mu^{\top}\triangleq\mathaccent
869{\Delta}=(\mathaccent 869{\delta}_{ij})_{m\times m}$. Denote the
eigenvalues of $\mathaccent 869{S}={\bf H}S{\bf H}$ by
$\lambda_{1}\geq\lambda_{2}\geq\cdots\geq\lambda_{n}\geq 0$. Besides, let
$\lambda^{*}_{1}\geq\cdots\geq\lambda^{*}_{n}\geq 0$ be the solutions to the
equation
$\int\psi_{0}(y_{1},y_{2})u_{i}(y_{1})p(y_{1})dy_{1}=\lambda^{*}_{i}u_{i}(y_{2}),$
where
$\psi_{0}(y_{1},y_{2})=\psi(y_{1},y_{2})-E_{y_{1}}(\psi(y_{1},y_{2}))-E_{y_{2}}(\psi(y_{1},y_{2}))+E_{y_{1},y_{2}}(\psi(y_{1},y_{2}))$
and $u_{i}(y_{1})$ is the eigenfunction of the kernel $\psi_{0}(y_{1},y_{2})$
corresponding to $\lambda^{*}_{i}$ for $i=1,\cdots,n$. Throughout, we assume
that there exists positive values $c_{0}$ and $c_{1}$, such that
$E(\mathbb{X}_{j}^{4})\leq c_{0}$ and $E(s_{ii}^{2})\leq c_{1}$ for
$j=1,\cdots,m$ and $i=1,\cdots,n$. The symbol $``\stackrel{{\scriptstyle
d}}{{\longrightarrow}}"$ represents “converges in distribution to” and
$``\stackrel{{\scriptstyle p}}{{\longrightarrow}}"$ means “converges in
probability to”.

###  3.1 Asymptotical null distributions

In the following two lemmas, we give the asymptotic null distribution of the
numerator and denominator of the pseudo-$F$ test statistic, respectively.

######  Lemma 3.1.

Under the null hypothesis $H_{0}$, the numerator of $T_{pseudo}$,
$\text{tr}\big{(}H_{X}{\bf H}S{\bf H}\big{)}$, has the same asymptotic
distribution as $\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}\xi_{i}$ when
$n\rightarrow\infty$, where
$\xi_{i}=\varpi_{i}+\varrho_{i}/(1+\mu^{\top}\Delta^{-1}\mu)$,
$\varpi_{1},\ldots,\varpi_{n}$ are independently identical distributed (iid)
chi-squared random variables with $m-1$ degrees of freedom,
$\varrho_{1},\cdots,\varrho_{n}$ are iid chi-squared random variables with 1
degrees of freedom, i.e.,
$\varpi_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{m-1}$ and
$\varrho_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{1}$, and
$\varpi_{i}$ and $\varrho_{i}$ are independent, $i=1,\cdots,n$. In particular,
when $\mu=\bm{0}_{m}$ an $m$-dimensional column vector with all 0 units,
$\xi_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{m}$.

######  Lemma 3.2.

As $n\rightarrow\infty$, the denominator of $T_{pseudo}$,

$\displaystyle\text{tr}\big{(}({\bf I}_{n}-H_{X}){\bf H}S{\bf
H}/(n-m)\big{)}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}E(s_{11})-E(s_{12}),$

where $E(s_{11})=E(\psi(y_{1},y_{1}))$, $E(s_{12})=E(\psi(y_{1},y_{2}))$, and
$y_{1}$ and $y_{2}$ are two independent observations of $\mathbb{Y}$.

All those proofs are presented in Appendix. Both lemmas lead to the following
theorem concerning the asymptotic distribution of the pseudo-$F$ test
statistic.

######  Theorem 3.3.

Under the null hypothesis $H_{0}$, $T_{pseudo}$ has the same asymptotic
distribution with $m^{-1}\sum_{i=1}^{n}w_{i}\xi_{i}$ when
$n\rightarrow\infty$, where
$\xi_{i}=\varpi_{i}+\varrho_{i}/(1+\mu^{\top}\Delta^{-1}\mu)$,
$\varpi_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{m-1}$ and
$\varrho_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{1}$ are
mutually independent, $w_{i}=\lambda_{i}/\sum_{j=1}^{n}\lambda_{j}$, and
$\\{\lambda_{1},\cdots,\lambda_{n}\\}$ are eigenvalues of ${\bf H}S{\bf H}$ in
descending order, $i=1,\cdots,n$. In particular, when $\mu={\bm{0}}_{m}$,
$\xi_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{m}$,
$i=1,\cdots,n$.

This theorem shows that the pseudo-$F$ test statistic has the same asymptotic
distribution as the random variable of Chi-squared-type mixtures. From the
proofs presented in the Appendix, we can see that the asymptotic results do
not depend on the dimension of response variables and are valid for high-
dimensional data. To derive the asymptotic distribution of the proposed test
statistic $T_{sqrt}$, we need the following assumption.

$\sum_{i=1}^{\infty}(\lambda^{*}_{i})^{1/2}<\infty$ and
$\sum_{i=1}^{\infty}\big{|}(\lambda_{i}/n)^{1/2}-(\lambda^{*}_{i})^{1/2}\big{|}\stackrel{{\scriptstyle
P}}{{\longrightarrow}}0$.

Note that Gretton et al. [7] also assumed that
$\sum_{i=1}^{\infty}(\lambda^{*}_{i})^{1/2}<\infty$ and Zhang [27] shows that
$(\lambda_{i}/n)^{1/2}\stackrel{{\scriptstyle
P}}{{\longrightarrow}}(\lambda^{*}_{i})^{1/2}$ for $i=1,2,\cdots$. It implies
that the assumption can be easily satisfied when the number of eigenvalues of
kernel $\psi(\cdot,\cdot)$ are finite, such as linear kernel. By the Theorem 1
of Gretton et al. [7] and Assumption 3.1, we have the following proposition.

######  Proposition 3.4.

Assume that $\\{z_{1},z_{2},\cdots\\}$ is an infinite sequence of i.i.d.
standard Gaussian variables and Assumption 3.1 holds. Then
$\sum_{i=1}^{\infty}\big{(}(\lambda_{i}/n)^{1/2}-(\lambda^{*}_{i})^{1/2}\big{)}z_{i}^{2}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}0$ as $n\rightarrow\infty$.

######  Lemma 3.5.

For any symmetric semidefinite matrices $\bf D_{1}$ and $\bf D_{2}$ of n
dimension, the eigenvalues of ${\bf D}={\bf D}_{1}{\bf D}_{2}$ are nonnegative
such that $\text{tr}({\bf D}^{\top}{\bf D})\leq\text{tr}({\bf D})^{2}$.

When $\bf D$ is a symmetric semidefinite matrix, the result in 3.5 naturally
holds, since the sum of squares of the nonnegative eigenvalues should be
smaller than the squares of the sum of them. Lemma 3.5 is vital to show that a
matrix formed by the product of two symmetric semidefinite matrices still have
the same property. Based on it and proposition 3.4, the asymptotic null
distribution of the proposed test statistic thus follows.

######  Theorem 3.6.

When Assumptions $1$ holds, $T_{sqrt}$ has the same asymptotic distribution
with $m^{-1}\sum_{i=1}^{n}\eta_{i}\xi_{i}$ when $n\rightarrow\infty$, where
$\xi_{i}=\varpi_{i}+\varrho_{i}/(1+\mu^{\top}\Delta^{-1}\mu)$,
$\varpi_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{m-1}$ and
$\varrho_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{1}$ are
mutually independent, and
$\eta_{i}=\lambda_{i}^{1/2}/\sum_{j=1}^{n}\lambda_{j}^{1/2}$, and
$\\{\lambda_{1},\cdots,\lambda_{n}\\}$ are eigenvalues of ${\bf H}S{\bf H}$ in
descending order, $i=1,\cdots,n$. In particular, when $\mu=\bm{0}_{m}$,
$\xi_{i}\stackrel{{\scriptstyle\text{iid}}}{{\thicksim}}\chi^{2}_{m}$,
$i=1,\cdots,n$.

###  3.2 Computation issue

$T_{pseudo}$ and $T_{sqrt}$ have asymptotical null distribution of the same
form with two mixtures of Chi-squared random variables, whose density
functions involve multiple integrations. To ease computation and increase
efficiency, we employ a parameter bootstrap procedure to approximate them in
finite-sample cases as the explicit formulas of the distributions of
$T_{pseudo}$, $T_{sqrt}$ and the two mixtures are all intricate.

A parameter bootstrap procedure
Step 1. Set a large $B$, for example $B=1000$.
Step 2. Randomly generate $n$ pairs of observations from the Chi-squared
distributions with $m-1$ and $1$ degrees of freedom, denoted by
$\\{(\varpi_{i},\varrho_{i})|i=1,\cdots,n\\}$. Let
$T_{1}=m^{-1}\sum_{i=1}^{n}w_{i}[\varpi_{i}+\varrho_{i}/(1+\mu^{\top}\Delta^{-1}\mu)]$
and
$T_{2}=m^{-1}\sum_{i=1}^{n}\eta_{i}[\varpi_{i}+\varrho_{i}/(1+\mu^{\top}\Delta^{-1}\mu)]$.
Step 3. Repeat Step 2 $B$ times and denote the obtained statistics $T_{1}$ and
$T_{2}$ by $T_{11},\cdots,T_{1B}$ and $T_{21},\cdots,T_{2B}$.
Step 4. By the law of large numbers, the p-values of $T_{pseudo}$ and
$T_{sqrt}$ can be empirically estimated by

$p_{pseudo}=\frac{1}{B}\sum\limits_{i=1}^{B}I\\{T_{1i}\geq t_{pseudo}\\}$

and

$p_{sqrt}=\frac{1}{B}\sum\limits_{i=1}^{B}I\\{T_{2i}\geq t_{sqrt}\\},$

where $t_{pseudo}$ and $t_{sqrt}$ is the observed value of $T_{pseudo}$ and
$T_{sqrt}$ respectively and $I\\{\cdot\\}$ is an indicator function.

It is worth pointing out that the proposed parameter bootstrap procedure is
much faster than the original bootstrap procedure. For a sample of size $n$,
the bootstrap procedure needs to generate $n$ individual observations, perform
$n(n-1)/2$ calculations to obtain the similarity matrix and conduct two matrix
multiplications, while the proposed procedure just needs to generate $2n$
random samples and calculate some summations, which runs ver fast. For
example, it takes 0.57 seconds to implement a parameter bootstrap for $B=1000$
and $n=500$ using Inter Core (TM) i9-9900 CPU, and takes 246.85 seconds to
implement the original bootstrap procedure under the same setting.

In addition to the parameter bootstrap procedure, one can use the Box scaled
$\chi^{2}$-approximation [2] or the generalized gamma distribution
approximation [10] by matching several cumulants of them. Based on Theorems 1
and 2, the cumulants of $T_{pseudo}$ and $T_{sqrt}$ can be estimated easily.
In particular, the $l$th cumulant of $T_{pseudo}$ and $T_{sqrt}$ are estimated
as $b_{l}=2^{l-1}(l-1)!m_{0}\sum_{i=1}^{n}(w_{i}/m)^{l}$ and
$h_{l}=2^{l-1}(l-1)!m_{0}\sum_{i=1}^{n}(\eta_{i}/m)^{l}$ respectively,
$l=1,2,3,4$, where $m_{0}=m-1+1/(1+\mu^{\top}\Delta^{-1}\mu)$,
$w_{i}=\lambda_{i}/\sum_{j=1}^{n}\lambda_{j}$,
$\eta_{i}=\lambda_{i}^{1/2}/\sum_{j=1}^{n}\lambda_{j}^{1/2}$, and
$\\{\lambda_{1},\cdots,\lambda_{n}\\}$ are eigenvalues of ${\bf H}S{\bf H}$ in
a descending order, $i=1,\cdots,n$. Let a random variable $X_{g}$ follow a
generalized gamma distribution with the density function being

$f_{g}(x)=\frac{v_{g}x^{v_{g}w_{g}-1}}{\sigma_{g}^{v_{g}w_{g}}\Gamma(w_{g})}\exp\\{-(\frac{x}{\sigma_{g}})^{v}\\},x>0,$

where $v_{g}$, $w_{g}$ and $\sigma_{g}$ are the parameters. We recommend to
use the distribution of $X_{g}+\theta_{g}$ to approximate those of
$T_{pseudo}$ and $T_{sqrt}$, where $\theta_{g}$ is a location parameter. The
unknown parameters $v_{g}$, $w_{g}$, $\sigma_{g}$, and $\theta_{g}$ can be
obtained by solving the equations

$\left\\{\begin{array}[]{ccl}b_{1}~{}(\hbox{or}~{}h_{1})&=&m_{1}+\theta_{g}\\\
b_{2}~{}(\hbox{or}~{}h_{2})&=&m_{2}-m_{1}^{2}\\\
b_{3}~{}(\hbox{or}~{}h_{3})&=&m_{3}-3m_{2}m_{1}+2m_{1}^{3}\\\
b_{4}~{}(\hbox{or}~{}h_{4})&=&m_{4}-4m_{3}m_{1}-3m_{2}^{2}+12m_{2}m_{1}^{2}-6m_{1}^{4},\end{array}\right.$

where
$m_{l}=E(X_{g}^{l})=\sigma_{g}^{l}\Gamma(v_{g}^{-1}l+w_{g})/\Gamma(w_{g})$ is
the $l$th moment of $X_{g}$, $l=1,2,3,4$, and $\Gamma(\cdot)$ is a Gamma
function.

###  3.3 Power analysis

To study the asymptotic power of the pseudo-$F$ test, we consider the
multivariate linear model $Y=X\bm{\beta}+\varepsilon$, where
$Y=(y_{1},\ldots,y_{n})^{\top}$ with $y_{i}=(y_{i1},\cdots,y_{iq})^{\top}$,
${\bm{\beta}}=(\beta_{ij})_{m\times q}$ is the effect of the predictor
variables $\mathbb{X}$ on the responses variables $\mathbb{Y}$ and
$\varepsilon=(\varepsilon_{1},\cdots,\varepsilon_{n})^{\top}$ are random
errors with $E(\varepsilon_{i})={\bf 0}_{n\times q}$ and
$\text{cov}(\varepsilon_{i})=\Delta_{\varepsilon}$ for $i=1,\cdots,n$. Then
for a linear kernel $\psi$, i.e., $s_{ij}=\psi(\bm{y}_{i},\bm{y}_{j})$, we
have the following theorem.

######  Theorem 3.7.

Assume that for $0<\iota<0.5$, $\max_{1\leq i\leq m,1\leq j\leq
q}n^{0.5-\iota}|\beta_{ij}|=\mathaccent 869{c}\neq 0$ as $n\rightarrow\infty$,
then for the linear kernel $\psi(\cdot,\cdot)$ and $\mathaccent
869{\tau}_{0}>0$, we have

$\lim_{n\rightarrow\infty}P\Big{(}\Big{|}\frac{T_{pseudo}}{n^{2\iota}}-\frac{1}{m}\frac{\text{tr}(\mathaccent
869{\Delta}^{-1}\Delta\mathaccent 869{\bm{\beta}}\mathaccent
869{\bm{\beta}}^{\top}\Delta^{\top})}{\text{tr}(\Delta_{\varepsilon})}\Big{|}>\mathaccent
869{\tau}_{0}\Big{)}=0.$

It thus follows that
$P\big{(}T_{pseudo}>F_{1}^{-1}(1-\alpha)\big{)}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}1$ as $n\rightarrow\infty$, where $F_{1}$ is the
asymptotic distribution function of $T_{pseudo}$, $F_{1}^{-1}(1-\alpha)$ is
the $(1-\alpha)$-quantile of $F_{1}$ and $\alpha$ is the nominal significance
level.

Theorem 3.7 shows that when the effect $\bm{\beta}$ is relatively small, even
as small as $n^{\iota-0.5}$ with tiny $\iota$, the power of the pseudo-$F$
test still converges to one in probability as $n\rightarrow\infty$. Actually,
when $\bm{\beta}$ is fixed, the asymptotic power of the square-root $F$-type
test statistic has the same performance.

######  Theorem 3.8.

For the linear kernel $\psi(\cdot,\cdot)$,
$P\big{(}T_{sqrt}>F_{2}^{-1}(1-\alpha)\big{)}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}1$ as $n\rightarrow\infty$, where $F_{2}$ is the
asymptotic distribution function of $T_{sqrt}$, $F_{2}^{-1}(1-\alpha)$ is the
$(1-\alpha)$-quantile of $F_{2}$, and $\alpha$ is the nominal significance
level.

##  4 Simulation studies

###  4.1 Accuracy of p-value calculation

We evaluate through simulation studies the p-value calculation accuracy of the
asymptotic distributions of the pseudo-$F$ ($T_{pseudo}$) and square-root
$F$-type ($T_{sqrt}$) test statistics for various correlation matrices. The
similarity matrix for the response variables is constructed based on the
measure of inner product, that is, $S=YY^{\top}$. The observations for the
predictor variables $\mathbb{X}$ are generated from the $m$-dimensional normal
distribution $N({\bf 1}_{m},{\it\bm{\Theta}}_{x})$ with mean vector ${\bf
1}_{m}=(1,\cdots,1)^{\top}$ and covariance matrix
${\it\bm{\Theta}}_{x}=\big{(}\theta_{ij}^{(x)}\big{)}_{m\times m}$, where
$\theta_{ij}^{(x)}=\rho_{x}^{|i-j|}$ with $\rho_{x}=0.5$. The observations for
$\mathbb{Y}$ are generated from $N_{k}({\bf 0}_{k},{\it\bm{\Theta}}_{y})$. We
consider the following two correlation models for
${\it\bm{\Theta}}_{y}=\big{(}\theta_{ij}^{(y)}\big{)}_{k\times k}$.

  * •

Model 1 (AR(1) correlation): Let $\theta_{ij}^{(y)}=\rho_{y}^{|i-j|}$ for
$1\leq i,j\leq k$, where $\rho_{y}=0.3,0.8$.

  * •

Model 2 (Equal correlation): Let $\theta_{ij}^{(y)}=\rho_{y}$ for $1\leq i\neq
j\leq k$ and $\theta_{ii}^{(y)}=1$ for $1\leq i\leq k$, where
$\rho_{y}=0.3,0.8$.

Thus there are a total of four correlation matrices for the outcome variables
$\mathbb{Y}$. We set the sample size $n$ to be 500. Let $m=5$ and $k=10$. For
each correlation setting, 10000 simulation replicates are performed to
evaluate the empirical size of the tests at significance levels ranging from 0
to 1. In each simulation, the p-values of the $T_{pseudo}$ and $T_{sqrt}$ are
calculated based on the asymptotic distribution and Monte Carlo method with
$B=2000$ replicates.

Figures 1 and 2 below display the empirical sizes of $T_{pseudo}$ and
$T_{sqrt}$ based on the asymptotic distribution against significance levels
under various correlation matrix settings. The figures show that the empirical
sizes of $T_{pseudo}$ and $T_{sqrt}$ are always very close to the
corresponding significance levels, even in the case of small significance
levels. It indicates good accuracy of the asymptotic distributions derived in
Theorems 3.3 and 3.6 and such accuracy is not sensitive to the correlation
structure and magnitude.

Figure 1: Empirical sizes ($-\log_{10}$) of the pseudo-$F$ test statistic
($T_{pseudo}$) based on the asymptotic distribution against the significance
levels ($-\log_{10}$) for two correlation models. Figure 2: Empirical sizes
($-\log_{10}$) of the proposed square-root $F$-type test statistic
($T_{sqrt}$) based on the asymptotic distribution against the significance
levels ($-\log_{10}$) for two correlation models.

###  4.2 Power comparison

Next we compare the type I error rates and powers of $T_{pseudo}$ and
$T_{sqrt}$. The multivariate response data are generated based on the linear
model $Y=X{\it\bm{\beta}}+\varepsilon$, where $\varepsilon\sim N_{k}({\bf
0}_{k},{\it\bm{\Theta}}_{\varepsilon})$. The correlation matrix
${\it\bm{\Theta}}_{\varepsilon}=\big{(}\theta_{ij}^{(\varepsilon)}\big{)}_{k\times
k}$ is set to have the following two correlation structures.

  * •

Model 1 (AR(1) correlation): Let
$\theta_{ij}^{(\varepsilon)}=\rho_{\varepsilon}^{|i-j|}$ for $1\leq i,j\leq
k$, where $\rho_{\varepsilon}=0.3,0.8$.

  * •

Model 2 (Equal correlation): Let
$\theta_{ij}^{(\varepsilon)}=\rho_{\varepsilon}$ for $1\leq i\neq j\leq k$ and
$\sigma_{ii}^{(\varepsilon)}=1$ for $1\leq i\leq k$, where
$\rho_{\varepsilon}=0.3,0.8$.

The observations $X$ for the predictor variables are generated from the
multivariate normal distribution $N({\bf 1}_{m},{\it\bm{\Theta}}_{x})$ with
${\it\bm{\Theta}}_{x}=\big{(}\theta_{ij}^{(x)}\big{)}_{m\times m}$ and
$\theta_{ij}^{(x)}=0.5^{|i-j|}$. For the two tests, we define the similarity
matrix as $S=YY^{\top}$. To investigate the performance of the two tests under
different sparsity of the “signals”, we choose the percentage $\tau$ of
nonzero elements of ${\it\bm{\beta}}$ from
$\\{0\%,20\%,40\%,60\%,80\%,100\%\\}$. The null hypothesis corresponds to the
case $\tau=0\%$. The signals (i.e., nonzero elements of ${\it\bm{\beta}}$) are
set to be $\\{\log(k)/(25\tau km)\\}^{1/2}+(1/k)\times N(0,0.01)$, to make the
powers comparable in different settings, where $N(0,0.01)$ is the normal
distribution with mean 0 and variance 0.01 and $\tau km$ is the total number
of signals. We set $k=10,m=5$ and $n=500$. The empirical type I error rates
and powers of the tests are calculated based on 1000 simulation replicates; in
each simulation, 2000 Monte Carlo samples are drawn to calculate the p-values
of the tests based on the asymptotic distributions. The nominal significance
level is 0.05.

Table 1: Type I error rates and powers of the pseudo-$F$ ($T_{pseudo}$) and square-root F-type ($T_{sqrt}$) test statistics under two correlation models. Correlation | Precentage |  | $\rho=0.3$ |  | $\rho=0.8$
---|---|---|---|---|---
Model | of nonzeros |  | $T_{pseudo}$ | $T_{sqrt}$ |  | $T_{pseudo}$ | $T_{sqrt}$
1 | 0% |  | 0.057 | 0.054 |  | 0.051 | 0.048
| 20% |  | 0.909 | 0.886 |  | 0.507 | 0.940
| 40% |  | 0.825 | 0.741 |  | 0.395 | 0.663
| 60% |  | 0.874 | 0.794 |  | 0.463 | 0.603
| 80% |  | 0.884 | 0.778 |  | 0.478 | 0.597
| 100% |  | 0.881 | 0.803 |  | 0.469 | 0.493
2 | 0% |  | 0.051 | 0.053 |  | 0.051 | 0.049
| 20% |  | 0.787 | 0.957 |  | 0.306 | 1.000
| 40% |  | 0.669 | 0.810 |  | 0.249 | 0.980
| 60% |  | 0.706 | 0.778 |  | 0.280 | 0.930
| 80% |  | 0.683 | 0.652 |  | 0.305 | 0.718
| 100% |  | 0.710 | 0.571 |  | 0.305 | 0.390

Table 1 presents the empirical type I error rates and powers of the two tests
for various ${\bm{\Sigma}}_{x}$ and $\tau$. It can be seen from the table that
both tests can control the type I error rates adequately and the proposed test
$T_{sqrt}$ is generally more powerful than $T_{pseudo}$, especially when the
correlation coefficients of the response variables are large. For example,
under the AR(1) correlation model with $\rho=0.8$, the powers of $T_{pseudo}$
and $T_{sqrt}$ for $\tau=20\%$ are 0.507 and 0.940. The superiority of
$T_{sqrt}$ diminishes as the sparsity level of signals increases, but can
still outperform $T_{pseudo}$ when the correlation coefficient is large. This
implies that the proposed test tends to gain more power than $T_{pseudo}$ when
the signals are sparse. $T_{pseudo}$ is slightly more powerful than $T_{sqrt}$
when the response variables are weakly dependent with decaying correlation
(e.g., AR(1) correlation model with $\rho=0.3$). For example, under the AR(1)
correlation model with $\rho=0.3$, the powers of $T_{pseudo}$ and $T_{sqrt}$
for $\tau=60\%$ are 0.874 and 0.794. In summary, the proposed test has more
robust power than the pseudo-$F$ statistic with respect to various sparsity
levels of signals and correlation magnitudes; it performs consistently well
across all settings.

##  5 Applications

We exemplify the tests using the gene expression data from a prostate cancer
study [19], whose aim is to investigate whether gene expression differences
underlie common clinical and pathological features of prostate cancer. This
can be achieved by comparing gene expression differences between tumor and
normal prostate samples. The data of expression profiles of approximately
12,600 genes from 52 tumor and 50 normal prostate specimens were collected. We
confine our analysis to the genes of 10 pathways: Alanine, aspartate and
glutamate metabolism (map00250); Pathogenic Escherichia coli infection
(map05130); Viral myocarditis (map05416); PPAR signaling pathway (map03320);
Rheumatoid arthritis (map05323); Tight junction (map04530); Regulation of
actin cytoskeleton (map04810); Hypertrophic cardiomyopathy (map05410); Cardiac
muscle contraction (map04260); TGF-beta signaling pathway (map04350). Detailed
descriptions of these pathways are available in [23], [28], [17], and among
others.

Here for illustration, we consider using the gene expression patterns as the
response variables and the status of tumor as the predictor variable.
Likewise, the measure of inner product is used to construct the similarity
matrix, that is, $S=YY^{\top}$. We are interested in whether the expression
patterns of the genes in a pathway are associated with prostate tumor. Table 2
presents the p values of the pseudo $F$ and square-root $F$-type test
statistics for testing the association between gene expression patterns and
prostate tumor. The p value results are calculated based on 10000 Monte Carlo
replicates. This table shows that, the p values of the pseudo $F$ test are
always larger than those of the square-root $F$-type test for the 10 pathways,
indicating that the proposed test is more powerful than the pseudo F test.
Moreover, for some pathways such as map05130 and map04810, the pseudo $F$ test
fails to detect any expression pattern difference between the normal and tumor
samples under the significance level of 0.05, while the proposed test can
detect a difference.

Table 2: P values of the pseudo F ($T_{pseudo}$) and square-root F-type ($T_{sqrt}$) test statistics for the association between gene expression patterns and prostate tumor. Pathway | Numbers of genes | $T_{pseudo}$ | $T_{sqrt}$
---|---|---|---
map00250 | 33 | 0.0396 | 0.0130
map05130 | 80 | 0.1797 | 0.0134
map05416 | 98 | 0.0738 | 0.0155
map03320 | 66 | 0.0055 | 0.0025
map05323 | 118 | 0.0097 | 0.0010
map04530 | 160 | 0.0564 | 0.0072
map04810 | 279 | 0.2297 | 0.0121
map05410 | 118 | 0.2385 | 0.0185
map04260 | 72 | 0.0007 | 0.0002
map04350 | 128 | 0.0478 | 0.0034

##  6 Conclusion

Distance-based regression model is very effective to detect the relationship
between high-dimensional response variables and predictor variables of
interest. It has a wide applications in many research fields related to
statistics. One drawback is the intensive computation for the calculation of
statistical significance, due to the lack of asymptotic null distribution. In
this work, we establish the asymptotic distribution for the pseudo-$F$ test
statistic based on the distance-based regression model and propose a new test
which is more powerful than the original pseudo-$F$ test when the correlation
coefficient of the outcomes for measuring the similarity is relatively large.
The proposed theory is anticipated to further broaden the application of
distance-based regression model.

The asymptotic property of the pseudo-$F$ test only requires that the kernel
for the similarity matrix is a positive definite and does not impose any
restrictions on the dimension of outcomes. So it is free to handle high-
dimensional data and is expected to have a wide application in high-
dimensional association studies.

## Acknowledgement

We would like to thank the editor and two anonymous reviewers for their
insighful comments.

## References

  * Bertocci et al. [2012] Bertocci I, Araújo R, Incera M, Arenas F, Pereira R, Abreu H, Larsen K and Sousa-Pinto I, Benthic assemblages of rock pools in northern portugal: seasonal and between-pool variability, Scientia Marina, 2012, 76(4): 781-789.
  * Box [1954] Box GEP, Some theorems on quadratic forms applied in the study of analysis of variance problems, I. Effect of inequality of variance in the one-way classification, The Annals of Mathematical Statistics, 1954, 25: 290-302.
  * Chen et al. [2012] Chen J, Bittinger K, Charlson E S, Hoffmann C, Lewis J, Wu G D, Collman R G, Bushman F D and Li H, Associating microbiome cmposition with environmental covariates using generalized UniFrac distances, Bioinformatics, 2012, 28(16): 2106-2113.
  * Consoli et al. [2013] Consoli P, Romeo T, Ferraro M, Sarà G and Andaloro F, Factors affecting fish assemblages associated with gas platforms in the Mediterranean Sea, Journal of Sea Research, 2013, 77: 45-52.
  * Gambi et al. [2020] Gambi C, Canals M, Corinaldesi C, Dell’Anno A, Manea E, Pusceddu A, Sanchez-Vidal A and Danovaro R, Impact of historical sulfide mine tailings discharge on meiofaunal assemblages (Portmán Bay, Mediterranean Sea), Science of The Total Environment, 2020, 736: 139641.
  * Gower [1966] Gower JC, Some distance properties of latent root and vector methods used in multivariate analysis, Biometrika, 1966,53, 325-338.
  * Gretton et al. [2009] Gretton A, Fukumizu K, Harchaoui Z and Sriperumbudur B, A Fast, Consistent Kernel Two-Sample Test, Advances in Neural Information Processing Systems, 2009,23: 673-681.
  * Han and Pan [2010] Han F and Pan W, Powerful multi-marker association tests: unifying genomic distance-based regression and logistic regression, Genetic Epidemiology, 2010, 34(7): 680-688.
  * Li et al. [2019] Li J, Zhang W, Zhang S and Li Q, A theoretic study of a distance-based regression model, Science in China Series A: Mathematics, 2019, 62(5): 979-998.
  * Li et al. [2014] Li Q, Hu J, Ding J and Zheng G, Fisher’s method of combining dependent statistics using generalizations of the gamma distribution: with applications to genetic pleiotropic associations, Biostatistics, 2014, 15: 284-295.
  * Li et al. [2009] Li Q, Wacholder S, Hunter D J, Hoover R N, Chanock S, Thomas G and Yu K , Genetic background comparison using distance-based regression, with applications in population stratification evaluation and adjustment, Genetic Epidemiology, 2009, 33(5): 432-441.
  * Liang, Bushman and FitzGerald [2015] Liang X, Bushman F D and FitzGerald G A , Rhythmicity of the intestinal microbiota is regulated by gender and the host circadian clock, Proceedings of the National Academy of Sciences, 2015, 112(33): 10479-10484.
  * McArdle and Anderson [2001] McArdle B and Anderson M, Fitting multivariate models to community data: a comment on distance-based redundancy analysis, Ecology, 2001, 82: 290-297.
  * Molari et al. [2018] Molari M, Guilini K, Lott C, Weber M, de Beer D, Meyer S, Ramette A, Wegener G, Wenzhöfer F, Martin D et al., CO2 leakage alters biogeochemical and ecological functions of submarine sands, Science Advances, 2018, 4(2): eaao2040.
  * Nievergelt, Libiger and Schork [2007] Nievergelt C M, Libiger O and Schork N J, Generalized analysis of molecular variance, PLoS Genet, 2007, 3(4): e51.
  * Norman et al. [2015] Norman J M, Handley S A , Baldridge M T, Droit L, Liu C Y, Keller B C, Kambal A, Monaco C L, Zhao G, Fleshner P et al., Disease-specific alterations in the enteric virome in inflammatory bowel disease, Cell, 2015, 160(3): 447-460.
  * Pinaud et al. [2018] Pinaud L, Sansonetti P J and Phalipon A, Host cell targeting by enteropathogenic bacteria T3SS effectors, Trends in Microbiology, 2018, 26(4): 266-283.
  * Reiss et al. [2010] Reiss P T, Stevens M H H, Shehzad Z, Petkova E and Milham M P, On distance-based permutation tests for between-group comparisons, Biometrics, 2010, 66(2): 636-643.
  * Singh et al. [2002] Singh D, Febbo P G, Ross K, Jackson D G, Manola J, Ladd C, Tamayo P, Renshaw A A, D’Amico A V, Richie J P et al., Gene expression correlates of clinical prostate cancer behavior, Cancer Cell, 2002, 1(2): 203-209.
  * Wang, Yang and Zhao [2019] Wang T, Yang C and Zhao H, Prediction analysis for microbiome sequencing data, Biometrics, 2019, 75(3): 875-884.
  * Wessel and Schork [2006] Wessel J and Schork N J, Generalized genomic distance-based regression methodology for multilocus association analysis, The American Journal of Human Genetics, 2006, 79(5): 792-806.
  * White et al. [2020] White L, O’Connor N, Yang Q, Emmerson M and Donohue I, Individual species provide multifaceted contributions to the stability of ecosystems, Nature Ecology $\&$ Evolution, 2020, 12(4): 1594-1601.
  * Wu [1998] Wu G, Intestinal mucosal amino acid catabolism, Journal of Nutrition, 1998, 128(8): 1249-1252.
  * Wu et al. [2011] Wu G D, Chen J, Hoffmann C, Bittinger K, Chen Y, Keilbaugh S A, Bewtra M, Knights D, Walters W A, Knight R et al., Linking long-term dietary patterns with gut microbial enterotypes, Science, 2011, 334(6052): 105-108.
  * Xu et al. [2016] Xu G, Lin L, Wei P and Pan W , An adaptive two-sample test for high-dimensional means, Biometrika, 2016, 103(3): 609-624.
  * Zapala and Schork [2006] Zapala M A and Schork N J Multivariate regression analysis of distance matrices for testing associations between gene expression patterns and related variables, Proceedings of the National Academy of Sciences, 2006, 103(51): 19430-19435.
  * Zhang [2012] Zhang K, Peters J, Janzing D and Schölkopf B, Kernel-based conditional independence test and application in causal discovery,” In Proceedings of the Conference on Uncertainty in Artificial Intelligence (UAI), 2012, 804-813.
  * Zihni et al. [2016] Zihni C, Mills C, Matter K and Balda M S, Tight junctions: from simple barriers to multifunctional molecular gates, Nature Reviews Molecular Cell Biology, 2016, 17(9): 564-580.

## Appendix

### Proof of Lemma 3.1.

Denote ${\it\Omega}=X\mathaccent 869{\Delta}^{-1}X^{\top}$. Note that
${\it\Omega}$ can be regarded as a random variable constructed based on a
weighted inner product, which is a positive definite kernel. Then the
numerator of $T_{pseudo}$ can be written as

$\text{tr}\big{(}H_{X}\mathaccent
869{S}\big{)}=\frac{1}{n}\text{tr}\big{(}{\it\Omega}\mathaccent
869{S}\big{)}+\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent
869{S}\big{)}.$

We first show that $\frac{1}{n}\text{tr}\big{(}{{\it\Omega}{\bf H}S{\bf
H}}\big{)}$ has the same asymptotical distribution with
$\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}\xi_{i}$ under $H_{0}$. To this end, we
denote the eigenvalues of $X\mathaccent 869{\Delta}^{-1}X^{\top}$ by
$\mathaccent 869{\lambda}_{1}\geq\cdots\geq\mathaccent 869{\lambda}_{m}\geq 0$
and $\mathaccent 869{\lambda}_{m+1}=\cdots=\mathaccent 869{\lambda}_{n}=0$.
Let $\mathaccent 869{\lambda}^{*}_{1}\geq\cdots\geq\mathaccent
869{\lambda}^{*}_{n}\geq 0$ be the solutions to the equation

$\int\mathaccent 869{\psi}(x_{1},x_{2})\mathaccent
869{u}_{i}(x_{1})p(x_{1})dx_{1}=\mathaccent 869{\lambda}^{*}_{i}\mathaccent
869{u}_{i}(x_{2})$ (1)

with $\mathaccent 869{\psi}(x_{1},x_{2})=x_{1}^{\top}\mathaccent
869{\Delta}^{-1}x_{2}-E_{x_{1}}(x_{1}^{\top}\mathaccent
869{\Delta}^{-1}x_{2})-E_{x_{2}}(x_{1}^{\top}\mathaccent
869{\Delta}^{-1}x_{2})+E_{x_{1},x_{2}}(x_{1}^{\top}\mathaccent
869{\Delta}^{-1}x_{2})=(x_{1}-\mu)^{\top}\mathaccent
869{\Delta}^{-1}(x_{2}-\mu),$ where $\mathaccent 869{u}_{i}(x_{1})$ is the
eigenfunction of the kernel $\mathaccent 869{\psi}(x_{1},x_{2})$ corresponding
to $\mathaccent 869{\lambda}^{*}_{i}$. Note that eigenfunctions are
orthonormal with respect to the probability measure $p(x_{1})$, i.e.,

$\int\mathaccent 869{u}_{i}(x_{1})\mathaccent
869{u}_{j}(x_{1})p(x_{1})dx_{1}=\mathbb{I}_{i=j},$ (2)

where $\mathbb{I}_{\bf E}$ denoting the indicator function of ${\bf E}$. Zhang
[27] shows that $\frac{1}{n}\mathaccent
869{\lambda}_{i}\stackrel{{\scriptstyle p}}{{\longrightarrow}}\mathaccent
869{\lambda}^{*}_{i}$,for $i=1,\cdots,n$. Then $\mathaccent
869{\lambda}^{*}_{m+1}=\cdots=\mathaccent 869{\lambda}^{*}_{n}=0$.

When $\mu=\bm{0}$, it then follows that
$E_{x_{1}}(e_{i}^{\top}\Delta^{-1/2}{x_{1}x_{1}}^{\top}\mathaccent
869{\Delta}^{-1}x_{2})=e_{i}^{\top}\Delta^{-1/2}x_{2}$ and
$E_{x_{1}}(e_{i}^{\top}\Delta^{-1/2}{x_{1}x_{1}}^{\top}\Delta^{-1/2}e_{j})=\mathbb{I}_{i=j}$,
where $\Delta^{-1/2}\Delta^{-1/2}=\Delta$. This implies that $\mathaccent
869{\lambda}^{*}_{i}=1$ and $\mathaccent
869{u}_{i}(x_{1})=e_{i}^{\top}\Delta^{-1/2}x_{1}$ for $i=1,\cdots,m$. When
$\mu\neq\bm{0}$, for $i,j=1,\cdots,m-1,$ it can be proved that $\mathaccent
869{\lambda}^{*}_{i}=1$ and $\mathaccent
869{\lambda}^{*}_{m}=1/(1+\mu^{\top}\Delta^{-1}\mu)$ corresponding to
$\mathaccent 869{u}_{i}(x_{1})=v_{i}^{\top}(x_{1}-\mu)$ and $\mathaccent
869{u}_{m}(x_{1})=v_{m}^{\top}(x_{1}-\mu)$ satisfy the equations (1) and (2),
where $v_{m}=\Delta^{-1}\mu/\mu^{\top}\Delta^{-1}\mu$ and
$v_{1},\cdots,v_{m-1}$ are the solutions to the equations $v_{i}^{\top}\mu=0$
and $v_{i}^{\top}\Delta v_{j}=\mathbb{I}_{i=j}$.

We now show that $\mathaccent 869{\lambda}^{*}_{1}=\cdots=\mathaccent
869{\lambda}^{*}_{m-1}=1$, $\mathaccent
869{\lambda}^{*}_{m}=1/(1+\mu^{\top}\Delta^{-1}\mu)$ and $\mathaccent
869{\lambda}^{*}_{m+1}=\cdots=\mathaccent 869{\lambda}^{*}_{n}=0$. By Theorem
3 in Zhang [27], under $H_{0}$, $\frac{1}{n}\text{tr}\big{(}{{\it\Omega}{\bf
H}S{\bf H}}\big{)}$ has the same asymptotic distribution as
$\sum_{i,j=1}^{n}\lambda^{*}_{i}\mathaccent
869{\lambda}^{*}_{j}z^{2}_{ij}=\sum_{i=1}^{n}\lambda^{*}_{i}\xi_{i}$, where
$z_{ij}$ are iid standard Gaussian variables and
$\xi_{i}=\sum_{j=1}^{m}\mathaccent
869{\lambda}^{*}_{j}z^{2}_{ij}=\varpi_{i}+\varrho_{i}/(1+\mu^{\top}\Delta^{-1}\mu)$
. In addition, by Theorem 1 in Gretton et al. [7], we have
$\sum_{i=1}^{\infty}(\frac{1}{n}\lambda_{i}-\lambda^{*}_{i})\xi_{i}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}0$. That is,
$\sum_{i=1}^{n}\frac{1}{n}\lambda_{i}\xi_{i}$ has the same asymptotic
distribution as $\sum_{i=1}^{n}\lambda^{*}_{i}\xi_{i}$. It thus follows that
$\frac{1}{n}\text{tr}\big{(}{{\it\Omega}{\bf H}S{\bf H}}\big{)}$ has the same
asymptotic distribution as $\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}\xi_{i}$.

Next we show that $\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent
869{S}\big{)}$ converges to 0 in probability. Write

$\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent
869{S}\big{)}=\text{tr}(AB)=\sum_{i=1}^{m}\sum_{j=1}^{m}a_{ij}b_{ij},$

where $A=(a_{ij})_{m\times m}=(\frac{1}{n}X^{\top}X)^{-1}-\mathaccent
869{\Delta}^{-1}$ and $B=(b_{ij})_{m\times m}=\frac{1}{n}X^{\top}\mathaccent
869{S}X$. By the law of large numbers, it can be obtained that the $(i,j)$th
entry of $\frac{1}{n}X^{\top}X=\frac{1}{n}\sum_{i=1}^{n}x_{i}x_{i}^{\top}$
converges in probability to $\mathaccent 869{\delta}_{ij}$, that is
$a_{ij}\stackrel{{\scriptstyle p}}{{\longrightarrow}}0$, $i,j=1,\cdots,m$.
Note that $(e_{i}-e_{j})^{\top}S(e_{i}-e_{j})\geq 0$ and
$(e_{i}+e_{j})^{\top}S(e_{i}+e_{j})\geq 0$ for a positive definite kernel
matrix $S$. Then we have $2|s_{ij}|\leq s_{ii}+s_{jj}$ and $|E(s_{ij})|\leq
E(s_{ii})$, $i,j=1,\cdots,m$. It follows that

$\displaystyle
E(s_{ij}^{2})\leq\frac{1}{4}E((s_{ii}+s_{jj})^{2})=\frac{1}{2}E(s_{ii}^{2})+\frac{1}{2}E(s_{ii}s_{jj})\leq
E(s_{ii}^{2}),$
$\displaystyle|E(s_{ij}s_{lk})|\leq\frac{1}{2}E(s_{ij}^{2})+\frac{1}{2}E(s_{lm}^{2})\leq
E(s_{ii}^{2})=c_{1},$

for $i,j,l,k=1,\cdots,m$. Similarly, we can obtain that
$|E(x_{j_{1}i_{1}}x_{j_{2}i_{1}}x_{j_{3}i_{2}}x_{j_{4}i_{2}})|\leq\frac{1}{4}E\big{(}x_{j_{1}i_{1}}^{4}+x_{j_{2}i_{1}}^{4}+x_{j_{3}i_{2}}^{4}+x_{j_{4}i_{2}}^{4}\big{)}\leq
c_{0}$ for $i_{1},i_{2}=1,\cdots,m$ and $j_{1},j_{2},j_{3},j_{4}=1,\cdots,n$.

Define $\overline{x}=\frac{1}{n}\sum_{j=1}^{n}x_{j}$ and
$\overline{x}=(\overline{x}_{1},\cdots,\overline{x}_{m})$. Then we can write

$\displaystyle
B=\frac{1}{n}\sum_{i=1}^{n}\sum_{j=1}^{n}(x_{i}-\overline{x})s_{ij}(x_{j}-\overline{x})^{\top}.$

Owing to $x_{i}-\overline{x}=(x_{i}-\mu)-(\overline{x}-\mu)$, we can assume
that $E(\mathbb{X})=\bm{0}_{m}$ to estimate the expectation and variance of
$b_{ij}$ under $H_{0}$,

$\displaystyle|E(B)|=$
$\displaystyle\big{|}\frac{1}{n}\sum_{i=1}^{n}\sum_{j=1}^{n}E(s_{ij})E\big{(}(x_{i}-\overline{x})(x_{j}-\overline{x})^{\top}\big{)}\big{|}$
$\displaystyle\leq$ $\displaystyle
E(s_{ii})\frac{1}{n}\sum_{i=1}^{n}\big{|}E\big{(}(x_{i}-\overline{x})(x_{i}-\overline{x})^{\top}\big{)}\big{|}+E(s_{ii})\frac{1}{n}\sum_{i\neq
j=1}^{n}\big{|}E\big{(}(x_{i}-\overline{x})(x_{j}-\overline{x})^{\top}\big{)}\big{|}$
$\displaystyle\leq$ $\displaystyle
E(s_{ii})|\Delta|+E(s_{ii})|\Delta|=2E(s_{ii})|\Delta|,$

implying that $E(b_{i_{1}i_{2}})$ is finite, $i_{1},i_{2}=1,\cdots,n$. Through
some algebraic manipulations, it can be obtained that

$\displaystyle E(b_{i_{1}i_{2}}^{2})=$
$\displaystyle\displaystyle\frac{1}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}c_{s}E\big{(}({x_{j_{1}i_{1}}}-\overline{x}_{i_{1}})({x_{j_{2}i_{1}}}-\overline{x}_{i_{1}})({x_{j_{3}i_{2}}}-\overline{x}_{i_{2}}))({x_{j_{4}i_{2}}}-\overline{x}_{i_{2}})\big{)}$
$\displaystyle=$
$\displaystyle\displaystyle\frac{1}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}c_{s}E\big{(}{x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{x_{j_{4}i_{2}}}\big{)}-\displaystyle\frac{4}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}c_{s}E\big{(}{x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{\overline{x}_{i_{2}}}\big{)}$
$\displaystyle+\displaystyle\frac{2}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}c_{s}E\big{(}{x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{\overline{x}_{i_{2}}}{\overline{x}_{i_{2}}}\big{)}+\displaystyle\frac{4}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}c_{s}E\big{(}{x_{j_{1}i_{1}}}{\overline{x}_{i_{1}}}{x_{j_{3}i_{2}}}{\overline{x}_{i_{2}}}\big{)}$
$\displaystyle-\displaystyle\frac{4}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}c_{s}E\big{(}{x_{j_{1}i_{1}}}{\overline{x}_{i_{1}}}{\overline{x}_{i_{2}}}{\overline{x}_{i_{2}}}\big{)}+\displaystyle\frac{1}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}c_{s}E\big{(}{\overline{x}_{i_{1}}}{\overline{x}_{i_{1}}}{\overline{x}_{i_{2}}}{\overline{x}_{i_{2}}}\big{)}$
$\displaystyle\triangleq$
$\displaystyle\kappa_{1}+\kappa_{2}+\kappa_{3}+\kappa_{4}+\kappa_{5}+\kappa_{6},$

where $c_{s}=E(s_{j_{1}j_{2}}s_{j_{3}j_{4}})$. Then using the assumptions
$E(\mathbb{X}_{1}^{4})\leq c_{0}$ and $E(s_{11}^{2})\leq c_{1}$, we have

$\displaystyle|\kappa_{1}|\leq$
$\displaystyle\displaystyle\frac{c_{1}}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}\big{|}E\big{(}x_{j_{1}i_{1}}x_{j_{2}i_{1}}x_{j_{3}i_{2}}x_{j_{4}i_{2}}\big{)}\big{|}$
$\displaystyle\leq$
$\displaystyle\displaystyle\frac{c_{1}}{n}\max_{i_{1},i_{2}}E({x^{2}_{1i_{1}}}{x^{2}_{1i_{2}}})+c_{1}\max_{i_{1},i_{2}}E({x^{2}_{1i_{1}}}{x^{2}_{2i_{2}}})+2c_{1}\max_{i_{1},i_{2}}|E({x_{1i_{1}}}{x_{2i_{1}}}{x_{1i_{2}}}{x_{2i_{2}}})|$
$\displaystyle\leq$
$\displaystyle(\displaystyle\frac{1}{n}c_{0}c_{1}+c_{0}c_{1}+2c_{0}c_{1})\leq
4c_{0}c_{1}.$

Similarly,

$\displaystyle|\kappa_{2}|\leq$
$\displaystyle\displaystyle\frac{4}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|c_{s}||E({x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{\overline{x}_{i_{2}}})|$
$\displaystyle\leq$
$\displaystyle\displaystyle\frac{4c_{1}}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|E({x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{x_{j_{4}i_{2}}})|\leq
16c_{0}c_{1},$ $\displaystyle|\kappa_{3}|\leq$
$\displaystyle\displaystyle\frac{2}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|c_{s}||E({x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{\overline{x}_{i_{2}}}{\overline{x}_{i_{2}}})|$
$\displaystyle\leq$
$\displaystyle\displaystyle\frac{2c_{1}}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|E({x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{x_{j_{4}i_{2}}})|\leq
8c_{0}c_{1},$ $\displaystyle|\kappa_{4}|\leq$
$\displaystyle\displaystyle\frac{4}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|c_{s}||E({x_{j_{1}i_{1}}}{\overline{x}_{i_{1}}}{x_{j_{3}i_{2}}}{\overline{x}_{i_{2}}})|$
$\displaystyle\leq$
$\displaystyle\displaystyle\frac{4c_{1}}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|E({x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{x_{j_{4}i_{2}}})|\leq
16c_{0}c_{1},$ $\displaystyle|\kappa_{5}|\leq$
$\displaystyle\displaystyle\frac{4}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|c_{s}||E({x_{j_{1}i_{1}}}{\overline{x}_{i_{1}}}{\overline{x}_{i_{2}}}{\overline{x}_{i_{2}}})|$
$\displaystyle\leq$
$\displaystyle\displaystyle\frac{4c_{1}}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|E({x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{x_{j_{4}i_{2}}})|\leq
16c_{0}c_{1},$ $\displaystyle|\kappa_{6}|\leq$
$\displaystyle\displaystyle\frac{1}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|c_{s}||E({\overline{x}_{i_{1}}}{\overline{x}_{i_{1}}}{\overline{x}_{i_{2}}}{\overline{x}_{i_{2}}})|$
$\displaystyle\leq$
$\displaystyle\displaystyle\frac{c_{1}}{n^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}=1}^{n}|E({x_{j_{1}i_{1}}}{x_{j_{2}i_{1}}}{x_{j_{3}i_{2}}}{x_{j_{4}i_{2}}})|\leq
4c_{0}c_{1}.$

It thus follows that $E(b_{i_{1}i_{2}}^{2})\leq
4c_{0}c_{2}(1+4+6+4+1)=64c_{0}c_{1}$ for $i_{1},i_{2}=1,\cdots,m$ and
$j_{1},j_{2},j_{3},j_{4}=1,\cdots,n$. Then $E(b_{i_{1}i_{2}})$ and
$\text{var}(b_{i_{1}i_{2}})$ are finite. Further by the Chebyshev’s
inequality, $b_{i_{1}i_{2}}$ is bounded in probability. Therefore,
$\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent 869{S}\big{)}$
converges to 0 in probability. This completes the proof.

### Proof of Lemma 3.2.

Let $\mathbb{H}$ be a reproducing kernel Hilbert space on $\mathbb{X}$, with a
continuous feature mapping $\phi(x)$. With the positive definite kernel
function
$s_{ij}=\psi(y_{i},y_{j})=\langle\phi({y_{i}}),\phi({y_{j}})\rangle_{\mathbb{H}}$,
it can be obtained that

$\displaystyle\frac{1}{n^{2}}{\bf 1}^{\top}_{n}S{\bf
1}_{n}=\langle\frac{1}{n}\sum_{i=1}^{n}\phi({y_{i}}),\frac{1}{n}\sum_{j=1}^{n}\phi({y_{j}})\rangle_{\mathbb{H}}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}E(\langle\phi({y_{i}}),\phi({y_{j}})\rangle_{\mathbb{H}})=E(s_{ij}).$

By the law of large numbers, we have

$\frac{1}{n}\sum_{i=1}^{n}\lambda_{i}=\frac{1}{n}\text{tr}(\mathaccent
869{S})=\frac{1}{n}\text{tr}(S)-\frac{1}{n^{2}}{\bf 1}^{\top}_{n}S{\bf
1}_{n}\stackrel{{\scriptstyle p}}{{\longrightarrow}}E(s_{ii})-E(s_{ij}).$

By Lemma 3.1, $\text{tr}\big{(}H_{X}\mathaccent
869{S}\big{)}/\sum_{i=1}^{n}\frac{\lambda_{i}}{n}$ has the same asymptotic
distribution as $\sum_{i=1}^{n}w_{i}\xi_{i}$, where
$w_{i}=\lambda_{i}/\sum_{j=1}^{n}\lambda_{j}$. Since $\sum_{i=1}^{n}w_{i}=1$
and $0\leq w_{i}\leq 1$, $i=1,\cdots,n$, it then follows that

$E\big{(}\sum_{i=1}^{n}w_{i}\xi_{i}\big{)}=m_{0}\sum_{i=1}^{n}w_{i}=m_{0}~{}~{}\text{and}~{}~{}\text{var}\big{(}\sum_{i=1}^{n}w_{i}\xi_{i}\big{)}=2m_{0}\sum_{i=1}^{n}w_{i}^{2}\leq
2m_{0}\sum_{i=1}^{n}w_{i}=2m_{0},$

where $m_{0}=1/(1+\mu^{\top}\Delta^{-1}\mu)+m-1$. Hence, by Chebyshev’s
inequality, for any $\tau_{0}>0$,

$\displaystyle
P\Big{(}\Big{|}\frac{1}{n}\sum_{i=1}^{n}w_{i}\xi_{i}\Big{|}\geq\tau_{0}+\frac{m_{0}}{n}\Big{)}$
$\displaystyle\leq
P\Big{(}\Big{|}\frac{1}{n}\sum_{i=1}^{n}w_{i}\xi_{i}-\frac{m_{0}}{n}\Big{|}\geq\tau_{0}\Big{)}$
$\displaystyle\leq\frac{2m_{0}}{n^{2}\tau_{0}^{2}}.$

Then for any $\tau_{1}=2\tau_{0}>0$,

$\lim\limits_{n\rightarrow\infty}P\Big{(}\Big{|}\frac{1}{n}\sum_{i=1}^{n}w_{i}\xi_{i}\Big{|}\geq\tau_{1}\Big{)}=0.$

It follows that $\text{tr}\big{(}H_{X}\mathaccent
869{S}\big{)}/\sum_{i=1}^{n}\lambda_{i}$ converges in distribution to zero,
that is, $\text{tr}\big{(}H_{X}\mathaccent
869{S}\big{)}/\sum_{i=1}^{n}\lambda_{i}$ converges in probability to zero.
Then we conclude that

$\displaystyle\displaystyle\frac{1}{n-m}\text{tr}\big{(}({\bf
I}_{n}-H_{X})\mathaccent 869{S}\big{)}$
$\displaystyle=\frac{\text{tr}\big{(}({\bf I}_{n}-H_{X})\mathaccent
869{S}\big{)}}{\sum_{i=1}^{n}\lambda_{i}}\displaystyle\frac{1}{n-m}\sum_{i=1}^{n}\lambda_{i}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}E(s_{ii})-E(s_{ij}).$

### Proof of Theorem 3.3.

Theorem 3.3 is a direct consequence of Lemma 3.1 and Lemma 3.2 .

### Proof of Proposition 3.4.

When Assumption 3.1 holds, by Theorem 1 of Gretton et al. [7] and the
Chebyshev’s inequality, we have
$\sum_{i=1}^{\infty}(\frac{\lambda_{i}}{n})^{1/2}z_{i}^{4}$ and
$\sum_{i=1}^{\infty}(\lambda_{i}^{*})^{1/2}z_{i}^{4}$ are bounded in
probability, as $n\rightarrow\infty$. Combining this and the Cauchy-Schwarz
inequality leads to

$\displaystyle\Big{|}\sum_{i=1}^{\infty}\big{(}(\frac{\lambda_{i}}{n})^{1/2}-(\lambda^{*}_{i})^{1/2}\big{)}z_{i}^{2}\Big{|}\leq$
$\displaystyle\Big{\\{}\sum_{i=1}^{\infty}(\frac{\lambda_{i}}{n})^{1/2}z_{i}^{4}\Big{\\}}^{1/2}\Big{\\{}\sum_{i=1}^{\infty}\Big{|}(\frac{\lambda_{i}}{n})^{1/4}-(\lambda^{*}_{i})^{1/4}\Big{|}^{2}\Big{\\}}^{1/2}$
$\displaystyle+\Big{\\{}\sum_{i=1}^{\infty}(\frac{\lambda_{i}}{n})^{1/2}z_{i}^{4}\Big{\\}}^{1/2}\Big{\\{}\sum_{i=1}^{\infty}\Big{|}(\frac{\lambda_{i}}{n})^{1/4}-(\lambda^{*}_{i})^{1/4}\Big{|}^{2}\Big{\\}}^{1/2}$
$\displaystyle\leq$
$\displaystyle\Big{\\{}\sum_{i=1}^{\infty}(\frac{\lambda_{i}}{n})^{1/2}z_{i}^{4}\Big{\\}}^{1/2}\Big{\\{}\sum_{i=1}^{\infty}\Big{|}(\frac{\lambda_{i}}{n})^{1/2}-(\lambda^{*}_{i})^{1/2}\Big{|}\Big{\\}}^{1/2}$
$\displaystyle+\Big{\\{}\sum_{i=1}^{\infty}(\lambda_{i}^{*})^{1/2}z_{i}^{4}\Big{\\}}^{1/2}\Big{\\{}\sum_{i=1}^{\infty}\Big{|}{(\frac{\lambda_{i}}{n})}^{1/2}-{\lambda^{*}_{i}}^{1/2}\Big{|}\Big{\\}}^{1/2}\stackrel{{\scriptstyle
P}}{{\longrightarrow}}0.$

### Proof of Lemma 3.5.

For any symmetric semidefinite matrices ${\bf D}_{1}$, there exits a symmetric
matrices $\bf P$, so that ${\bf D}_{1}={\bf P}{\bf P}^{\top}$. For any n
dimension vector $\bf x$, we have ${\bf x}^{\top}{\bf P}^{\top}{\bf D}_{2}{\bf
P}{\bf x}\geq 0$. It follows that ${\bf P}^{\top}{\bf D}_{2}{\bf P}$ is a
symmetric semidefinite matrice. Thus ${\bf D}={\bf D}_{1}{\bf D}_{2}={\bf
PP}^{\top}{\bf D}_{2}$ has the same eigenvalues as ${\bf P}^{\top}{\bf
D}_{2}{\bf P}$ whose eigenvalues are nonnegative. Consequently,
$\text{tr}({\bf D}^{\top}{\bf
D})=\sum_{i=1}^{n}\lambda_{i,D}^{2}\leq\big{(}\sum_{i=1}^{n}\lambda_{i,D}\big{)}^{2}=\text{tr}({\bf
D})^{2}$, where $\lambda_{i,D}$ is eigenvalue of $\bf D$, $i=1,\cdots,n$.

### Proof of Theorem 3.6.

For ${\it\Omega}=X\mathaccent 869{\Delta}^{-1}X^{\top}$, it can be obtained
that

$n^{1/2}\text{tr}\big{(}H_{X}\mathaccent
869{S}^{1/2}\big{)}=\text{tr}\big{(}{\it\Omega}(\mathaccent
869{S}/n)^{1/2}\big{)}+n^{1/2}\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent
869{S}^{1/2}\big{)}.$

We first show that $\text{tr}\big{(}{{\it\Omega}(\mathaccent
869{S}/n)^{1/2}}\big{)}$ has the same asymptotical distribution with
$\sum_{i=1}^{n}(\lambda_{i}/n)^{1/2}\xi_{i}$ under $H_{0}$. By extending the
proof of Theorem 3 in Zhang [27], under $H_{0}$,
$\text{tr}\big{(}{{\it\Omega}(\mathaccent 869{S}/n)^{1/2}}\big{)}$ has the
same asymptotic distribution as
$\sum_{i,j=1}^{n}(\lambda^{*}_{i})^{1/2}\mathaccent
869{\lambda}^{*}_{j}z^{2}_{ij}=\sum_{i=1}^{n}(\lambda^{*}_{i})^{1/2}\xi_{i}$.
In addition, by proposition 3.4, we have
$\sum_{i=1}^{n}\big{(}(\lambda_{i}/n)^{1/2}-(\lambda^{*}_{i})^{1/2}\big{)}\xi_{i}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}0$. It thus follows that
$\text{tr}\big{(}{{\it\Omega}(\mathaccent 869{S}/n)^{1/2}}\big{)}$ has the
same asymptotic distribution as $\sum_{i=1}^{n}(\lambda_{i}/n)^{1/2}\xi_{i}$.

Next we show that
$n^{1/2}\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent
869{S}^{1/2}\big{)}$ converges to 0 in probability. Write

$n^{1/2}\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent
869{S}^{1/2}\big{)}=\text{tr}(A\mathaccent
869{B})=\sum_{i=1}^{m}\sum_{j=1}^{m}a_{ij}\mathaccent 869{b}_{ij},$

where $A=(a_{ij})_{m\times m}=(\frac{1}{n}X^{\top}X)^{-1}-\mathaccent
869{\Delta}^{-1}\stackrel{{\scriptstyle p}}{{\longrightarrow}}0$ and
$\mathaccent 869{B}=(\mathaccent 869{b}_{ij})_{m\times m}=X^{\top}(\mathaccent
869{S}/n)^{1/2}X$. Denote $B_{0}=\mathaccent 869{\Delta}^{-1/2}\mathaccent
869{B}\mathaccent 869{\Delta}^{-1/2}$ and by Lemma 3.5

$\displaystyle\text{tr}(B_{0}^{\top}B_{0})=\text{tr}\big{(}\mathaccent
869{\Delta}^{-1}\mathaccent 869{B}\mathaccent 869{\Delta}^{-1}\mathaccent
869{B}\big{)}=\text{tr}\big{(}{\it\Omega}(\mathaccent
869{S}/n)^{1/2}{\it\Omega}(\mathaccent
869{S}/n)^{1/2}\big{)}\leq\Big{(}\text{tr}\big{(}{\it\Omega}(\mathaccent
869{S}/n)^{1/2}\big{)}\Big{)}^{1/2},$

where $\text{tr}\big{(}{{\it\Omega}(\mathaccent 869{S}/n)^{1/2}}\big{)}$ tends
to $\sum_{i=1}^{\infty}(\lambda^{*}_{i})^{1/2}\xi_{i}$ who is bounded in
probability. It implies that $\text{tr}(B_{0}^{\top}B_{0})$, i.e., the
$(i,j)$th entry of $B_{0}$ is bounded in probability such that the $(i,j)$th
entry of $\mathaccent 869{B}=\mathaccent 869{\Delta}^{1/2}B_{0}\mathaccent
869{\Delta}^{1/2}$ is as well bounded, $i,j=1,\cdots,m$. One can derive that
$n^{1/2}\text{tr}\big{(}({H_{X}-\frac{1}{n}{\it\Omega}})\mathaccent
869{S}^{1/2}\big{)}$ converges to 0 in probability. It follows that
$\text{tr}\big{(}H_{X}(n\mathaccent 869{S})^{1/2}\big{)}$ has the same
asymptotic distribution as $\sum_{i=1}^{n}(\lambda_{i}/n)^{1/2}\xi_{i}$ as
$n\rightarrow\infty$.

By the extension of Lemma 3.2, we have $\text{tr}\big{(}({\bf
I}_{n}-H_{X})\mathaccent
869{S}^{1/2}\big{)}/\sum_{i=1}^{n}\lambda_{i}^{1/2}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}1$ as $n\rightarrow\infty$. Consequently, $T_{sqrt}$ has
the same asymptotic distribution with $m^{-1}\sum_{i=1}^{n}\eta_{i}\xi_{i}$.

### Proof of Theorem 3.7.

For the linear kernel $\psi(\cdot,\cdot)$, it follows that

$\displaystyle\frac{1}{n}X^{\top}\mathaccent
869{S}X=\frac{1}{n}\sum_{i=1}^{n}\sum_{j=1}^{n}(x_{i}-\overline{x})s_{ij}(x_{j}-\overline{x})^{\top}$
$\displaystyle=$
$\displaystyle\Big{[}(n)^{-1}\sum_{i=1}^{n}(x_{i}-\overline{x})(x_{i}^{\top}\bm{\beta}+\varepsilon_{i}^{\top})\Big{]}\Big{[}\sum_{j=1}^{n}(\bm{\beta}^{\top}x_{j}+\varepsilon_{j})(x_{j}-\overline{x})^{\top}\Big{]}.$

Let
$G=(n)^{-1/2}\sum_{i=1}^{n}(x_{i}-\overline{x})(x_{i}^{\top}\bm{\beta}+\varepsilon_{i}^{\top})=\frac{1}{n^{1-\iota}}\sum_{i=1}^{n}(x_{i}-\overline{x})x_{i}^{\top}\mathaccent
869{\bm{\beta}}_{n}+(n)^{-1/2}\sum_{i=1}^{n}(x_{i}-\overline{x})\varepsilon_{i}^{\top}\triangleq
G_{1}+G_{2}$, where $G_{2}$ is bounded in probability by the Chebyshev’s
inequality and then $G_{2}/n^{\iota}$ are converge in probability to zero. By
the law of large numbers, it can be obtained that

$Q_{1}=\frac{1}{n}\sum_{i=1}^{n}s_{ij}=\frac{1}{n}\sum_{i=1}^{n}(x_{i}^{\top}\bm{\beta}+\varepsilon_{i}^{\top})(\bm{\beta}^{\top}x_{i}+\varepsilon_{i})\stackrel{{\scriptstyle
p}}{{\longrightarrow}}\Delta_{\varepsilon}~{}~{}\text{and}~{}~{}\
Q_{2}=\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{j=1}^{n}s_{ij}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}0$

and the $(i,j)$th entry of $G_{1}/n^{\iota}$ converges in probability to the
$(i,j)$th entry of $\Delta\mathaccent 869{\bm{\beta}}$ for $i=1,\cdots,m$ and
$j=1,\cdots,q$. Then we have the numerator of $T$,

$\frac{1}{n^{2\iota}}\text{tr}\big{(}H_{X}\mathaccent
869{S}\big{)}=\frac{1}{n^{2\iota}}\text{tr}\big{(}(\frac{1}{n}X^{\top}X)^{-1}GG^{\top}\big{)}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}\text{tr}(\mathaccent 869{\Delta}^{-1}\Delta\mathaccent
869{\bm{\beta}}\mathaccent 869{\bm{\beta}}^{\top}\Delta^{\top})>0.$

and the denominator of $T_{pseudo}$, $\text{tr}\big{(}\mathaccent
869{S}/n\big{)}=\text{tr}(Q_{1}-Q_{2})\stackrel{{\scriptstyle
p}}{{\longrightarrow}}\text{tr}(\Delta_{\varepsilon})>0.$ It follows that

$\frac{T_{pseudo}}{n^{2\iota}}=\frac{n-m}{mn}\frac{\text{tr}\big{(}H_{X}\mathaccent
869{S}/n^{2\iota}\big{)}}{\text{tr}\big{(}({\bf I}_{n}-H_{X})\mathaccent
869{S}/n\big{)}}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}\frac{1}{m}\frac{\text{tr}(\mathaccent
869{\Delta}^{-1}\Delta\mathaccent 869{\bm{\beta}}\mathaccent
869{\bm{\beta}}^{\top}\Delta^{\top})}{\text{tr}(\Delta_{\varepsilon})}>0$

and then for any $\mathaccent 869{\tau}_{0}>0$,

$\lim_{n\rightarrow\infty}P\Big{(}\Big{|}\frac{T_{pseudo}}{n^{2\iota}}-\frac{1}{m}\frac{\text{tr}(\mathaccent
869{\Delta}^{-1}\Delta\mathaccent 869{\bm{\beta}}\mathaccent
869{\bm{\beta}}^{\top}\Delta^{\top})}{\text{tr}(\Delta_{\varepsilon})}\Big{|}>\mathaccent
869{\tau}_{0}\Big{)}=0$

Now we show that
$P\big{(}T_{pseudo}>F_{1}^{-1}(1-\alpha)\big{)}=P\big{(}T_{pseudo}/n^{2\iota}>F_{1}^{-1}(1-\alpha)/n^{2\iota}\big{)}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}1$ as $n\rightarrow\infty$.

### Proof of Theorem 3.8.

Denote $A_{0}=H_{X}(\mathaccent 869{S}/n)^{1/2}$ whose eigenvalues are
nonnegative by Lemma 3.5 and then $\text{tr}\big{(}H_{X}(\mathaccent
869{S}/n)^{1/2})=\text{tr}(A_{0})\geq\text{tr}(A_{0}^{\top}A_{0})^{1/2}=\text{tr}\big{(}H_{X}\mathaccent
869{S}/n)$ where $\text{tr}\big{(}H_{X}\mathaccent 869{S}/n)$ converges to a
positive constant in probability by Theorem 3.7. It follows that

$\frac{T_{sqrt}}{n}=\frac{\text{tr}\big{(}H_{X}\mathaccent
869{S}^{1/2}/mn\big{)}}{\text{tr}\big{(}({\bf I}_{n}-H_{X})\mathaccent
869{S}^{1/2}/(n-m)\big{)}}\geq\frac{n-m}{n}\frac{\text{tr}(A_{0}^{\top}A_{0})^{1/2}}{\sum_{i=1}^{n}(\lambda_{i}/n)^{1/2}}>0.$

Consequently, we have
$P\big{(}T_{sqrt}>F_{2}^{-1}(1-\alpha)\big{)}=P\big{(}T_{sqrt}/n>F_{2}^{-1}(1-\alpha)/n\big{)}\stackrel{{\scriptstyle
p}}{{\longrightarrow}}1$ as $n\rightarrow\infty$.