# The limiting spectral distribution of the sample canonical correlation
matrix

Xiaozhuo<EMAIL_ADDRESS>[ Key Laboratory of Applied
Statistics of MOE School of Mathematics and Statistics Northeast Normal
University Changchun, Jilin 130024, Chinapresep=, ]e1

###### Abstract

In this paper, we investigate the spectral properties of the sample canonical
correlation (SCC) matrix under the alternative hypothesis to provide a more
comprehensive description of the association between two sets of variables.
Our research involves establishing the relationship between the eigenvalues of
the SCC matrix and the block correlation matrix, as well as proving the
universality of the Stieltjes transform of the limiting spectral distribution
(LSD) of the block correlation matrix. By combining the results from the
normal case, we establish the limiting spectral distribution (LSD) of the SCC
matrix with a general underlying distribution under the arbitrary rank
alternative hypothesis. Finally, we present several simulated examples and
find that they fit well with our theoretical results.

60B10,

62H10,

Canonical correlation analysis,

Block correlation matrix,

Limiting spectral distribution,

###### keywords:

[class=MSC]

###### keywords:

##  1 Model and Relationship

When we encounter multivariables or high dimensional data, principal component
analysis as a traditional research method for data reduction and
interpretation, fought against data explosion by focusing on the significant
information reflected in original data. However, when confronted with more
complex problems in practical applications such as statistical learning,
wireless communications, financial economics and population genetics,
Canonical Correlation Analysis (CCA) was proposed to identify and quantify the
associations between two sets of variables.

Traditional multivariate analysis textbooks [9, 10, 14, 1] use matrix language
to deconstruct the CCA problem and draw a heuristic conclusion: the key
quantity that reflects the correlation between two groups of variables is the
characteristic space of the canonical correlation matrix. Specifically, for
$p+q$ dimensional general population $(\tilde{{\bf x}}^{\prime},\tilde{{\bf
y}}^{\prime})^{\prime}$ with mean $\bf 0$, the covariance between $\tilde{{\bf
x}}$ and $\tilde{{\bf y}}$ has the following expression

$\displaystyle\mathbb{V}ar\begin{pmatrix}\tilde{{\bf x}}\\\ \tilde{{\bf
y}}\end{pmatrix}=\begin{pmatrix}\tilde{\boldsymbol{\Sigma}}_{xx},\tilde{\boldsymbol{\Sigma}}_{xy}\\\
\tilde{\boldsymbol{\Sigma}}_{yx},\tilde{\boldsymbol{\Sigma}}_{yy}\end{pmatrix}.$

The aimed canonical correlation coefficients can be characterized as the
square roots of the eigenvalues of the population canonical correlation (PCC)
matrix
$\mathcal{C}_{xy}=\tilde{\mathbf{\Sigma}}_{xx}^{-1/2}\tilde{\mathbf{\Sigma}}_{xy}\tilde{\mathbf{\Sigma}}_{yy}^{-1}\tilde{\mathbf{\Sigma}}_{yx}\tilde{\mathbf{\Sigma}}_{xx}^{-1/2}$.
According to scale invariance, we can simply assume that the covariance matrix
of the population $({\bf x}^{\prime},{\bf y}^{\prime})^{\prime}$ with mean
$\bf 0$ has the following structure:

$\displaystyle\mathbb{V}ar\begin{pmatrix}{\bf x}\\\ {\bf
y}\end{pmatrix}=\begin{pmatrix}{\bf I}_{p},&\boldsymbol{\Lambda}\\\
\boldsymbol{\Lambda},&{\bf I}_{q}\end{pmatrix}.$

For sample matrices ${\bf X}_{p\times n}$ and ${\bf Y}_{q\times n}$ from
population ${\bf x}$ and ${\bf y}$ respectively, write ${\bf
S}_{xx}=\frac{1}{n}{\bf X}{\bf X}^{\prime}$, ${\bf S}_{yy}=\frac{1}{n}{\bf
Y}{\bf Y}^{\prime}$, and ${\bf S}_{xy}=\frac{1}{n}{\bf X}{\bf Y}^{\prime}$.
Then, we define the sample canonical correlation (SCC) matrix as
$\mathcal{S}_{xy}={\bf S}_{xx}^{-1/2}{\bf S}_{xy}{\bf S}_{yy}^{-1}{\bf
S}_{yx}{\bf S}_{xx}^{-1/2}$ to spy on the information of population canonical
correlations.

We pause to review some important studies on the spectrum of the SCC matrix.
Under the null hypothesis, i.e., ${\bf x}$ and ${\bf y}$ are independent, Yang
and Pan established the LSD of SCC matrix [16] and the central limiting
theorem(CLT) for linear spectral statistics (LSS) [17]. Under finite rank case
of alternative hypothesis, i.e., the fixed rank of $\boldsymbol{\Lambda}$, Bao
et al. [6] gave the asymptotic distribution of spiked eigenvalues and extreme
eigenvalues with Gaussian entries, then Yang and Ma showed the universality of
the above results in [15, 13]. For the general alternative hypothesis, i.e.,
the arbitrary rank of $\boldsymbol{\Lambda}$, Bai et al. [2] proposed a new
way to transform the eigenvalues of the SCC matrix to the function of that of
the noncentral Fisher matrix under Gaussian assumption. Then under the same
condition, Hou et al. [11] gave the CLT for spiked eigenvalues of the SCC
matrix via the corresponding results of noncentral Fisher matrix. Now we face
the problem of asymptotic results for the SCC matrix under alternative
hypothesis and general underlying distribution.

Noticed it is hard to extend the results to the data matrices with generally
distributed entries using the idea in [2], because it depended crucially on
the equivalence of uncorrelation and independence of normal distribution.
Essentially, the goal of CCA is to seek the relationship between the block
${\bf x}$ and block ${\bf y}$ in random vector $({\bf x}^{\prime},{\bf
y}^{\prime})^{\prime}$, which is the same as that of the block correlation
matrix. Block correlation matrix dated back to Bao et al. [5], which was
developed by sample correlation matrix. Recently the results [7] about this
concept are expanded to general cases, for example, no matter whether $k$ is
fixed or divergent, more mild conditions for dimensional size ratio.

Our approach adapts the same strategy as in [16]. The key innovation is to
find the relationship between the SCC matrix and block correlation matrix and
to get the exact expression for Stieltjes transform of the LSD of the SCC
matrix. Now we focus on case $k=2$ to calculate the relationship between the
SCC matrix and block correlation matrix. We construct the block correlation
matrix ${\bf B}$ for sample matrices ${\bf X}_{p\times n}$ and ${\bf
Y}_{q\times n}$, where

$\displaystyle{\bf B}=$ $\displaystyle\begin{pmatrix}({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}},&\boldsymbol{0}\\\ \boldsymbol{0},&({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}\end{pmatrix}\begin{pmatrix}{\bf X}{\bf
X}^{\prime},&{\bf X}{\bf Y}^{\prime}\\\ {\bf Y}{\bf X}^{\prime},&{\bf Y}{\bf
Y}^{\prime}\end{pmatrix}\begin{pmatrix}({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}},&\boldsymbol{0}\\\ \boldsymbol{0},&({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}\end{pmatrix}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}{\bf I}_{p},&({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}}{\bf X}{\bf Y}^{\prime}({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}\\\ ({\bf Y}{\bf Y}^{\prime})^{-\frac{1}{2}}{\bf
Y}{\bf X}^{\prime}({\bf X}{\bf X}^{\prime})^{-\frac{1}{2}},&{\bf
I}_{q}\end{pmatrix}.$

The block correlation matrix presented above is a natural extension of the
sample correlation matrix in block form. Considering the Green function
$G_{b}(z)=({\bf B}-z)^{-1}$ of ${\bf B}$, we have

$\displaystyle G_{b}(z)=\left[\begin{pmatrix}{\bf I}_{p},&({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}}{\bf X}{\bf Y}^{\prime}({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}\\\ ({\bf Y}{\bf Y}^{\prime})^{-\frac{1}{2}}{\bf
Y}{\bf X}^{\prime}({\bf X}{\bf X}^{\prime})^{-\frac{1}{2}},&{\bf
I}_{q}\end{pmatrix}-\begin{pmatrix}z{\bf I}_{p},&\boldsymbol{0}\\\
\boldsymbol{0},&z{\bf I}_{q}\end{pmatrix}\right]^{-1}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}(1-z){\bf I}_{p},&({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}}{\bf X}{\bf Y}^{\prime}({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}\\\ ({\bf Y}{\bf Y}^{\prime})^{-\frac{1}{2}}{\bf
Y}{\bf X}^{\prime}({\bf X}{\bf X}^{\prime})^{-\frac{1}{2}},&(1-z){\bf
I}_{q}\end{pmatrix}^{-1}$ $\displaystyle=$
$\displaystyle\begin{pmatrix}[(1\\!-\\!z){\bf
I}_{p}\\!-\\!\frac{1}{1\\!-\\!z}({\bf X}{\bf X}^{\prime})^{-\frac{1}{2}}{\bf
X}{\bf P}_{y}{\bf X}^{\prime}({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}}]^{-1},&\ast\\\ \ast,&[(1\\!-\\!z){\bf
I}_{q}\\!-\\!\frac{1}{1\\!-\\!z}({\bf Y}{\bf Y}^{\prime})^{-\frac{1}{2}}{\bf
Y}{\bf P}_{x}{\bf Y}^{\prime}({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}]^{-1}\end{pmatrix}$ $\displaystyle=$
$\displaystyle(1-z)\begin{pmatrix}[(1\\!-\\!z)^{2}{\bf I}_{p}\\!-\\!({\bf
X}{\bf X}^{\prime})^{-\frac{1}{2}}{\bf X}{\bf P}_{y}{\bf X}^{\prime}({\bf
X}{\bf X}^{\prime})^{-\frac{1}{2}}]^{-1},&\ast\ast\\\
\ast\ast,&[(1\\!-\\!z)^{2}{\bf I}_{q}\\!-\\!({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}{\bf Y}{\bf P}_{x}{\bf Y}^{\prime}({\bf Y}{\bf
Y}^{\prime})^{-\frac{1}{2}}]^{-1}\end{pmatrix}.$

Taking trace for the matrix on both sides of the equation, that is

$\displaystyle{\rm Tr}G_{b}=(z-1)[{\rm Tr}G_{xy}((1-z)^{2})+{\rm
Tr}G_{yx}((1-z)^{2})],$

where the function $\frac{1}{p}{\rm Tr}G_{xy}(z):=\frac{1}{p}{\rm
Tr}(\mathcal{S}_{xy}-z)^{-1}$ is the Stieltjes transform of the SCC matrix,
and $\frac{1}{q}{\rm Tr}G_{yx}(z):=\frac{1}{q}{\rm
Tr}(\mathcal{S}_{yx}-z)^{-1}$ is similar. Notice that $\mathcal{S}_{xy}$ and
$\mathcal{S}_{yx}$ share the same non-zero eigenvalues, so the limit of
$\frac{1}{q}{\rm Tr}G_{xy}$ under the alternative hypothesis can be converted
to the limit of $\frac{1}{p+q}{\rm Tr}G_{b}$. This implies the spectral
distributions of ${\bf B}$ and the SCC matrix have some relationship at a
global level.

Considering the correlation coefficients matrix $\boldsymbol{\Lambda}$, we can
assume the entries of ${\bf W}_{p\times n}$ and ${\bf Y}_{q\times n}$ are
i.i.d., and

$\displaystyle{\bf X}_{p\times n}=\boldsymbol{\Lambda}_{p\times q}{\bf
Y}_{q\times n}+\boldsymbol{\Gamma}{\bf W}_{p\times n},\ {\bf Y}_{q\times
n}={\bf Y}_{q\times n}$ (1)

where $\boldsymbol{\Gamma}\boldsymbol{\Gamma}^{\prime}={\bf
I}_{p}-\boldsymbol{\Lambda}\boldsymbol{\Lambda}^{\prime}$. Because of the
scale invariance in form (1), we replace ${\bf X},{\bf Y},{\bf W}$ in (1) with
the $\frac{1}{\sqrt{n}}{\bf X},\frac{1}{\sqrt{n}}{\bf
Y},\frac{1}{\sqrt{n}}{\bf W}$, but still expressed as ${\bf X},{\bf Y},{\bf
W}$, respectively.

The paper is organized as follows. In Section 2, we provide the main results
of the paper. Sections 3-5 show the important stages in the proof of Theorem
2. The lemmas appearing in previous sections and their proof are stated in
Section 6.

##  2 Main Result

######  Theorem 1.

Denote the ordered non-zero eigenvalues of $\mathcal{S}_{xy}$ by
$l_{1}^{2}\geq l_{2}^{2}\geq\dots\geq l_{\min\\{p,q\\}}^{2}$, then the $p+q$
eigenvalues of ${\bf B}$ will consist of

$\displaystyle\\{\underbrace{1+l_{1},\dots,1+l_{\min\\{p,q\\}}}_{\min\\{p,q\\}},\underbrace{1,\dots,1}_{\max\\{p,q\\}-\min\\{p,q\\}},\underbrace{1-l_{\min\\{p,q\\}},\dots,1-l_{1}}_{\min\\{p,q\\}}\\}.$

Moreover, at the global level, we have

$\displaystyle{\rm Tr}G_{b}=(z-1)[{\rm Tr}G_{xy}((1-z)^{2})+{\rm
Tr}G_{yx}((1-z)^{2})].$

######  Remark 1.

We establish a general result in Lemma 1. The proof of Theorem 1 is a specific
instance where we select the square root matrix of $\mathcal{S}_{xy}$ as ${\bf
A}$. By combining this with the argument for ${\rm Tr}G_{b}$ in Section 1, we
successfully complete the proof.

######  Assumption 1 (Assumption on the dimensions).

Assume that $p/n\to c_{1}$ and $q/n\to c_{2}$ as $n\to\infty$, such that
$c_{1}+c_{2}<1$. Meanwhile, we always work with the additional conditions:
$p/q\to c_{3}<1$.

######  Assumption 2 (Assumption on matrix entries).

Assume that real matrices ${\bf W}_{p\times n}$ and ${\bf Y}_{q\times n}$ in
(1) have i.i.d. entries. Furthermore, for all $a\in[\\![p]\\!]$ and
$b\in[\\![n]\\!]$, the distribution of $W_{ab}$ belong distribution set
$\mathcal{F}$, and for all $a\in[\\![q]\\!]$ and $b\in[\\![n]\\!]$, the
distribution of $Y_{ab}$ also belong distribution set $\mathcal{F}$, where

$\displaystyle\mathcal{F}=\\{F_{y}:\mathbb{E}[z]=0,\mathbb{E}[|z|^{2}]=1,\
\mbox{and}\ \mathbb{E}[|z|^{4}]<\infty\\}.$

######  Assumption 3 (Assumption on $\boldsymbol{\Lambda}$).

We assume the operator norm of $\boldsymbol{\Lambda}$ is bounded and keep away
from $1$. So we conclude the operator norm of $\boldsymbol{\Gamma}$ is bounded
and away from $0$. The empirical distribution the singular values of
$\boldsymbol{\Lambda}$ converges weakly to a non random distribution $H$.

######  Theorem 2.

Under Assumptions 1-3, we have the LSDs of the SCC matrix and its
corresponding block correlation matrix are universal. Then Stieltjes transform
of the LSD of the SCC matrix is the unique solution of the following equation
set:

$\displaystyle G_{1}(z,m)$
$\displaystyle\frac{c_{2}(1-z)[(1-z)m-1]}{1-c_{2}+c_{1}[(1-z)m-1]}=\int\left[\frac{n}{q}\frac{x^{2}}{1-x^{2}}-\frac{G_{2}(z,m)}{G_{1}(z,m)}\right]^{-1}dH(x)$
(2) $\displaystyle G_{1}(z,m)=$ $\displaystyle
1-c_{1}-c_{1}[(1-z)m-1]+\frac{c_{1}(1-c_{2})(1-z)[(1-z)m-1]}{1-c_{2}+c_{1}z[(1-z)m-1]}$
$\displaystyle G_{2}(z,m)=$
$\displaystyle\frac{1-c_{1}-c_{2}+c_{1}(1-z)m}{c_{2}(1-z)}\left[1-\frac{(1-z)(1-c_{1})}{1-c_{2}-c_{1}z+c_{1}zm(1-z)}\right].$

###### Proof.

Here we roughly introduce the proof strategy. Under normal case, we can
transform the SCC matrix to noncentral Fisher matrix, and through the study of
the noncentral Fisher matrix, we have the corresponding results for the SCC
matrix. Intuitively, the result of the LSD should be universality, now the
restriction is that the relationship does not work under non-Gaussian case. We
choose another transformation which holds under the general case, that is, the
block correlation matrix under some appropriate alternative hypothesis. Given
the result of the normality in [2], we only need the universality of the LSD
of the SCC matrix, that is, it is enough to consider that of the corresponding
block correlation matrix ${\bf B}$. Notice the non-zero eigenvalues of ${\bf
B}$ equal that of ${\bf H}$, where

$\displaystyle{\bf H}={\bf X}^{\prime}({\bf X}{\bf X}^{\prime})^{-1}{\bf
X}+{\bf Y}^{\prime}({\bf Y}{\bf Y}^{\prime})^{-1}{\bf Y}:={\bf P}_{x}+{\bf
P}_{y}.$

So in subsequent sections, we mainly prove that the LSD of ${\bf H}$ is
universal. ∎

##  3 Expectation convergence

In this section, we will show the difference of ${\rm tr}({\bf H}-z)^{-1}$
(${\rm tr}$ means normalized trace as $\frac{1}{n}{\rm Tr}$) between Gaussian
and general case in the sense of expectation. There are two random parts
contained in matrix ${\bf H}$, then we substitute them step by step. Here the
substitute method is not Lindeberg’s method but another continuous method
widely used in [17, 12]. Write ${\bf W}(s)=s^{1/2}{\bf W}+(1-s)^{1/2}\hat{{\bf
W}}$, where the matrix $\hat{{\bf W}}$ with standard normal entries is
independent with ${\bf W}$. Let ${\bf H}_{N}$ and ${\bf H}(s)$ denote the
matrix in which ${\bf W}$ is replaced by $\hat{{\bf W}}$ and ${\bf W}(s)$
respectively, but ${\bf Y}$ is still under general distribution. Then we have

$\displaystyle\mathbb{E}{\rm tr}({\bf H}-z)^{-1}-\mathbb{E}{\rm tr}({\bf
H}_{N}-z)^{-1}=\int_{0}^{1}\frac{\partial}{\partial s}\mathbb{E}{\rm tr}({\bf
H}(s)-z)^{-1}ds$ $\displaystyle=$ $\displaystyle-\int_{0}^{1}\mathbb{E}{\rm
tr}({\bf H}(s)-z)^{-2}\frac{\partial{\bf P}_{x(s)}}{\partial s}ds.$

Similar to the derivative of a matrix by a scalar, we have

$\displaystyle\frac{\partial{\bf P}_{x(s)}}{\partial s}=\frac{\partial[{\bf
X}(s)]^{\prime}\left({\bf X}(s)[{\bf X}(s)]^{\prime}\right)^{-1}{\bf
X}(s)}{\partial s}$ $\displaystyle=$ $\displaystyle\frac{1}{2}({\bf I}-{\bf
P}_{x(s)})\left(\frac{{\bf W}}{\sqrt{s}}\\!-\\!\frac{\hat{{\bf
W}}}{\sqrt{1\\!-\\!s}}\right)^{\prime}\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)+\frac{1}{2}{\bf Q}(s)^{\prime}\boldsymbol{\Gamma}\left(\frac{{\bf
W}}{\sqrt{s}}\\!-\\!\frac{\hat{{\bf W}}}{\sqrt{1\\!-\\!s}}\right)({\bf I}-{\bf
P}_{x(s)})$

where by definition ${\bf Q}(s)=({\bf X}(s){\bf X}(s)^{\prime})^{-1}{\bf
X}(s)$. Then we have

$\displaystyle\mathbb{E}{\rm tr}({\bf H}(s)-z)^{-2}\frac{\partial{\bf
P}_{x(s)}}{\partial s}$ $\displaystyle=$ $\displaystyle\mathbb{E}{\rm
tr}\left(\frac{{\bf W}}{\sqrt{s}}\\!-\\!\frac{\hat{{\bf
W}}}{\sqrt{1\\!-\\!s}}\right)^{\prime}\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)({\bf H}(s)-z)^{-2}({\bf I}-{\bf P}_{x(s)})$ $\displaystyle=$
$\displaystyle\frac{1}{n}\mathbb{E}\left[\frac{1}{\sqrt{s}}\sum_{ij}W_{ji}\Phi_{ji}-\frac{1}{\sqrt{1-s}}\sum_{ij}\hat{W}_{ji}\Phi_{ji}\right]$

where

$\displaystyle\Phi_{ji}:=[\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)({\bf
H}(s)-z)^{-2}({\bf I}-{\bf P}_{x(s)})]_{ji}.$ (3)

So

$\displaystyle\mathbb{E}{\rm tr}({\bf H}\\!-\\!z)^{-1}\\!-\\!\mathbb{E}{\rm
tr}({\bf
H}_{N}\\!-\\!z)^{-1}=\\!-\\!\int_{0}^{1}\frac{1}{n}\mathbb{E}\left[\frac{1}{\sqrt{s}}\sum_{ij}W_{ji}\Phi_{ji}\\!-\\!\frac{1}{\sqrt{1\\!-\\!s}}\sum_{ij}\hat{W}_{ji}\Phi_{ji}\right]ds.$

Then we need to calculate
$\mathbb{E}\frac{1}{\sqrt{s}}\sum_{ij}W_{ji}\Phi_{ji}$ and
$\mathbb{E}\frac{1}{\sqrt{1-s}}\sum_{ij}\hat{W}_{ji}\Phi_{ji}$ respectively.
Here $\hat{W}_{ji}$ is from standard normal distribution, so

$\displaystyle\mathbb{E}\sum_{ij}\hat{W}_{ji}\Phi_{ji}=\frac{1}{n}\sum_{ij}\mathbb{E}\frac{\partial\Phi_{ji}}{\partial\hat{W}_{ji}}$

exactly holds. By performing basic matrix calculus, we conclude the term with
$\kappa_{2}$ in $\mathbb{E}\frac{1}{\sqrt{s}}\sum_{ij}W_{ji}\Phi_{ji}$ and
$\mathbb{E}\frac{1}{\sqrt{1-s}}\sum_{ij}\hat{W}_{ji}\Phi_{ji}$ are exactly
same, see more details for Lemma 2. So the terms with third and higher order
cumulants of $\mathbb{E}\frac{1}{\sqrt{s}}\sum_{ij}W_{ji}\Phi_{ji}$ are
remained in their difference, next we bound it via the following formulas:

$\displaystyle\frac{1}{n}\mathbb{E}\left[\frac{1}{\sqrt{s}}\sum_{ij}W_{ji}\Phi_{ji}-\frac{1}{\sqrt{1-s}}\sum_{ij}\hat{W}_{ji}\Phi_{ji}\right]=\frac{1}{n}\frac{1}{\sqrt{s}}\sum_{ij}\mathbb{E}e_{1}$

where the term $e_{1}$ results from cumulant expansion and

$\displaystyle
e_{1}=O(1)\mathbb{E}[|W_{ji}|^{3}I_{\\{|W_{ji}|>t\\}}]\sup_{W_{ji}\in\mathbb{R}}\left|\frac{\partial^{2}\Phi_{ji}}{\partial
W^{2}_{ji}}\right|+O(1)\mathbb{E}|W_{ji}|^{3}\sup_{|W_{ji}|<t}\left|\frac{\partial^{2}\Phi_{ji}}{\partial
W^{2}_{ji}}\right|.$

Note that all terms in $\frac{\partial^{2}\Phi_{ji}}{\partial W^{2}_{ji}}$ can
be expressed in terms of the following factors

$\displaystyle[\boldsymbol{\Gamma}^{\prime}({\bf X}(s){\bf
X}(s)^{\prime})^{-1}\boldsymbol{\Gamma}]_{jj},[\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)G^{b}{\bf Q}^{\prime}(s)\boldsymbol{\Gamma}]_{jj},[G^{b}]_{ii},[{\bf
P}_{x(s)}G^{b}]_{ii},$ $\displaystyle[{\bf P}_{x(s)}G^{b}{\bf
P}_{x(s)}]_{ii},[\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)G^{b}]_{ji},[\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)G^{b}{\bf
P}_{x(s)}]_{ji}$

where $b=0,1,2$ and $G=({\bf H}(s)-z)^{-1}$ (we omit $s$ without aumbiguity).

According to the Assumption 2, we have $\|\boldsymbol{\Gamma}\|$ is bounded.
For any fixed $z\in\mathbb{C}^{+}$, $\|G(z)\|$ has the trivial bound
$\frac{1}{v}$. Here we need Lemma 3 to bound $\|{\bf Q}\|$ with high
probability. Meanwhile from the moment assumption on matrix entries, we have

$\displaystyle\mathbb{E}[|W_{ji}|^{3}I_{\\{|W_{ji}|>t\\}}]=o\left(\frac{1}{n^{3/2}}\right)\mbox{
and }\mathbb{E}|W_{ji}|^{3}=O\left(\frac{1}{n^{3/2}}\right).$

Considering there is at least one off-diagonal element in term
$\frac{\partial^{2}\Phi_{ji}}{\partial W^{2}_{ji}}$, and choosing one of off-
diagonal element denoted $A_{ij}$, we have

$\displaystyle\frac{1}{n}\frac{1}{\sqrt{s}}\sum_{ij}\mathbb{E}[|W_{ji}|^{3}I_{\\{|W_{ji}|>t\\}}]\sup_{W_{ji}\in\mathbb{R}}\left|\frac{\partial^{2}\Phi_{ji}}{\partial
W^{2}_{ji}}\right|=\mathbb{E}\frac{1}{n}\sum_{ij}|A_{ij}|o(\frac{1}{n^{3/2}})$
$\displaystyle\leq$
$\displaystyle\frac{1}{n}\sqrt{\mathbb{E}\sum_{ij}|A_{ij}|^{2}}\sqrt{np}\times
o(\frac{1}{n^{3/2}})=o(\frac{1}{n}),$ and
$\displaystyle\frac{1}{n}\frac{1}{\sqrt{s}}\sum_{ij}\mathbb{E}|W_{ji}|^{3}\sup_{|W_{ji}|<t}\left|\frac{\partial^{2}\Phi_{ji}}{\partial
W^{2}_{ji}}\right|=\mathbb{E}\frac{1}{n}\sum_{ij}|A_{ij}|O(\frac{1}{n^{3/2}})$
$\displaystyle\leq$
$\displaystyle\frac{1}{n}\sqrt{\mathbb{E}\sum_{ij}|A_{ij}|^{2}}\sqrt{np}\times
O(\frac{1}{n^{3/2}})=O(\frac{1}{n}).$

This implies $\mathbb{E}{\rm tr}({\bf H}-z)^{-1}-\mathbb{E}{\rm tr}({\bf
H}_{N}-z)^{-1}=O(\frac{1}{n})$. Now we need to confirm that after replacing
${\bf Y}$ with $\hat{{\bf Y}}$, whose elements are with standard normal
distribution, the limit of the quality $\mathbb{E}{\rm tr}({\bf H}-z)^{-1}$
does not change, i.e.,

$\displaystyle\mathbb{E}{\rm tr}({\bf H}-z)^{-1}-\mathbb{E}{\rm tr}({\bf
H}_{N}-z)^{-1}=-\int_{0}^{1}O(\frac{1}{n})ds=O(\frac{1}{n}),$

where the notion ${\bf H}$ is the matrix in which ${\bf W}$ with Gaussian case
and ${\bf Y}$ with general distribution. Similar to the previous interpolation
step, we have

$\displaystyle\mathbb{E}{\rm tr}({\bf H}-z)^{-1}-\mathbb{E}{\rm tr}({\bf
H}_{N}-z)^{-1}=\int_{0}^{1}\frac{\partial}{\partial t}\mathbb{E}{\rm tr}({\bf
H}(t)-z)^{-1}dt$ $\displaystyle=$ $\displaystyle-\int_{0}^{1}\mathbb{E}{\rm
tr}({\bf H}(t)-z)^{-2}\left[\frac{\partial{\bf P}_{x(t)}}{\partial
t}+\frac{\partial{\bf P}_{y(t)}}{\partial t}\right]dt.$

Set

$\displaystyle{\bf Y}(t)=\sqrt{t}{\bf Y}+\sqrt{1-t}\hat{{\bf Y}},\ {\bf
X}(t)=\boldsymbol{\Lambda}{\bf Y}(t)+\boldsymbol{\Gamma}\hat{{\bf W}},\ {\bf
U}(t)=({\bf Y}(t){\bf Y}^{\prime}(t))^{-1}{\bf Y}(t),$

by the derivative calculation of a matrix by a scalar, we have

$\displaystyle\frac{\partial{\bf P}_{x(t)}}{\partial t}=\frac{\partial{\bf
X}^{\prime}(t)({\bf X}(t){\bf X}^{\prime}(t))^{-1}{\bf X}(t)}{\partial t}$
$\displaystyle=$ $\displaystyle({\bf I}\\!-\\!{\bf
P}_{x(t)})\frac{1}{2}\left(\frac{1}{\sqrt{t}}\boldsymbol{\Lambda}{\bf
Y}\\!-\\!\frac{1}{\sqrt{1\\!-\\!t}}\boldsymbol{\Lambda}\hat{{\bf
Y}}\right)^{\prime}{\bf Q}(t)\\!+\\!\frac{1}{2}{\bf
Q}(t)^{\prime}\left(\frac{1}{\sqrt{t}}\boldsymbol{\Lambda}{\bf
Y}\\!-\\!\frac{1}{\sqrt{1\\!-\\!t}}\boldsymbol{\Lambda}\hat{{\bf
Y}}\right)({\bf I}\\!-\\!{\bf P}_{x(t)})$

and

$\displaystyle\frac{\partial{\bf P}_{y(t)}}{\partial t}=\frac{\partial{\bf
Y}^{\prime}(t)({\bf Y}(t){\bf Y}^{\prime}(t))^{-1}{\bf Y}(t)}{\partial t}$
$\displaystyle=$ $\displaystyle\frac{1}{2}({\bf I}-{\bf
P}_{y(t)})\left(\frac{{\bf Y}}{\sqrt{t}}\\!-\\!\frac{\hat{{\bf
Y}}}{\sqrt{1\\!-\\!t}}\right)^{\prime}{\bf U}(t)+\frac{1}{2}{\bf
U}(t)^{\prime}\left(\frac{{\bf Y}}{\sqrt{t}}\\!-\\!\frac{\hat{{\bf
Y}}}{\sqrt{1\\!-\\!t}}\right)({\bf I}-{\bf P}_{y(t)}).$

Then

$\displaystyle\mathbb{E}{\rm tr}({\bf H}(t)-zI)^{-2}\left[\frac{\partial{\bf
P}_{x(t)}}{\partial t}+\frac{\partial{\bf P}_{y(t)}}{\partial
t}\right]=\frac{1}{n}\sum_{l=1}^{2}\mathbb{E}\left[\frac{1}{\sqrt{t}}\sum_{ij}Y_{ji}\Psi_{ji}^{l}\\!-\\!\frac{1}{\sqrt{1-t}}\sum_{ij}\hat{Y}_{ji}\Psi_{ji}^{l}\right]$

holds, where for $l=1,2$, $\Psi_{ji}^{1}=[\boldsymbol{\Lambda}^{\prime}{\bf
Q}(t)G^{2}({\bf I}-{\bf P}_{x(t)})]_{ji},\ \Psi_{ji}^{2}=[{\bf U}(t)G^{2}({\bf
I}-{\bf P}_{y(t)})]_{ji}.$ Similarly, we need to calculate
$\mathbb{E}\frac{1}{\sqrt{t}}\sum_{ij}Y_{ji}\Psi^{l}_{ji}$ and
$\mathbb{E}\frac{1}{\sqrt{1-t}}\sum_{ij}\hat{Y}_{ji}\Psi^{l}_{ji}$
respectively. Here $\hat{Y}_{ji}$ is standard normal distribution, so we have
exactly

$\displaystyle\mathbb{E}\sum_{ij}\hat{Y}_{ji}\Psi^{l}_{ji}=\frac{1}{n}\sum_{ij}\mathbb{E}\frac{\partial\Psi^{l}_{ji}}{\partial\hat{Y}_{ji}}.$

From the arguments in Lemma 4, we conclude the term with $\kappa_{2}$ in
$\mathbb{E}\frac{1}{\sqrt{t}}\sum_{ij}Y_{ji}\Psi^{l}_{ji}$ and
$\mathbb{E}\frac{1}{\sqrt{1-t}}\sum_{ij}\hat{Y}_{ji}\Psi^{l}_{ji}$ are exactly
same for $l=1,2$. So we need to calculate and bound the terms with third and
later cumulants of $\mathbb{E}\frac{1}{\sqrt{t}}\sum_{ij}Y_{ji}\Psi^{l}_{ji}$:

$\displaystyle\sum_{l=1}^{2}\frac{1}{n}\mathbb{E}\left[\frac{1}{\sqrt{t}}\sum_{ij}Y_{ji}\Psi^{l}_{ji}-\frac{1}{\sqrt{1\\!-\\!t}}\sum_{ij}\hat{Y}_{ji}\Psi^{l}_{ji}\right]=\frac{1}{n}\frac{1}{\sqrt{t}}\sum_{l=1}^{2}\sum_{ij}\mathbb{E}e_{2}^{l},$
$\displaystyle
e_{2}^{l}=O(1)\mathbb{E}[|Y_{ji}|^{3}I_{\\{|Y_{ji}|>t\\}}]\sup_{Y_{ji}\in\mathbb{R}}\left|\frac{\partial^{2}\Psi^{l}_{ji}}{\partial
Y^{2}_{ji}}\right|+O(1)\mathbb{E}|Y_{ji}|^{3}\sup_{|Y_{ji}|<t}\left|\frac{\partial^{2}\Psi^{l}_{ji}}{\partial
Y^{2}_{ji}}\right|,$

where the term $e_{2}^{l}$ results from cumulant expansion.

We notice that the bound of $\left|\frac{\partial^{2}\Psi^{l}_{ji}}{\partial
Y^{2}_{ji}}\right|$ is only related with its property (at least one off-
diagonal element factor), not with the specific form. Similarly,

$\frac{1}{n}\frac{1}{\sqrt{t}}\sum_{l=1}^{2}\sum_{ij}\mathbb{E}e_{2}^{l}=O(\frac{1}{n}).$

##  4 Martingale difference decomposition

Define $\mathbb{E}_{k}=\mathbb{E}(\cdot|w_{1},y_{1},\dots,w_{k},y_{k})$, then
we have

$\displaystyle{\rm tr}({\bf H}-z)^{-1}-\mathbb{E}{\rm tr}({\bf
H}-z)^{-1}=\sum_{k=1}^{n}(\mathbb{E}_{k}-\mathbb{E}_{k-1}){\rm tr}({\bf
H}-z)^{-1}$ $\displaystyle=$
$\displaystyle\sum_{k=1}^{n}(\mathbb{E}_{k}-\mathbb{E}_{k-1})[{\rm tr}({\bf
H}-z)^{-1}-{\rm tr}({\bf H}^{(k)}-z)^{-1}],$

where

$\displaystyle{\bf H}^{(k)}={\bf P}_{x}^{(k)}+{\bf P}_{y}^{(k)}=({\bf
X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf X}^{(k)})^{\prime}]^{-1}{\bf
X}^{(k)}+({\bf Y}^{(k)})^{\prime}[{\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime}]^{-1}{\bf Y}^{(k)},$

and

$\displaystyle{\bf X}={\bf X}^{(k)}+x_{k}e_{k}^{\prime},\ {\bf Y}={\bf
Y}^{(k)}+y_{k}e_{k}^{\prime},\ {\bf X}^{(k)}=\boldsymbol{\Lambda}{\bf
Y}^{(k)}+\boldsymbol{\Gamma}{\bf W}^{(k)}.$

By the calculation in Lemma 5, we obtain that

$\displaystyle{\bf H}-{\bf H}^{(k)}={\bf P}_{x}-{\bf P}_{x}^{(k)}+{\bf
P}_{y}-{\bf P}_{y}^{(k)}=\mbox{the sum of 8 rank 1
matrices}:=\boldsymbol{\alpha}_{k}\boldsymbol{\beta}_{k}.$

Let $(\boldsymbol{\alpha}_{k})_{n\times 8}$,
$(\boldsymbol{\beta}_{k})_{8\times n}$ denote the sets of column, row vector
respectively, see Lemma 5 for more details. And by the formula (0.7.4.1) of
[8], we have

$\displaystyle({\bf H}\\!-\\!z)^{-1}\\!-\\!({\bf
H}^{(k)}\\!-\\!z)^{-1}=\\!-\\!({\bf
H}^{(k)}\\!-\\!z)^{-1}\boldsymbol{\alpha}_{k}({\bf
I}_{8}\\!+\\!\boldsymbol{\beta}_{k}({\bf
H}^{(k)}\\!-\\!z)^{-1}\boldsymbol{\alpha}_{k})^{-1}\boldsymbol{\beta}_{k}({\bf
H}^{(k)}\\!-\\!z)^{-1}.$

There exists a deterministic matrix ${\bf M}$ w.r.t. random vectors
$y_{k},w_{k}$, which approximates ${\bf I}_{8}+\boldsymbol{\beta}_{k}({\bf
H}^{(k)}-z)^{-1}\boldsymbol{\alpha}_{k}$ in the expectation sense by Lemma 6.
For this matrix with fixed dimension, we have

$\displaystyle{\bf I}_{8}\\!+\\!\boldsymbol{\beta}_{k}({\bf
H}^{(k)}\\!-\\!z)^{-1}\boldsymbol{\alpha}_{k}\\!=\\!{\bf M}\\!+\\!\tilde{{\bf
M}},\quad\mbox{and}\quad({\bf I}_{8}\\!+\\!\boldsymbol{\beta}_{k}({\bf
H}^{(k)}\\!-\\!z)^{-1}\boldsymbol{\alpha}_{k})^{-1}={\bf
M}^{-1}\\!+\\!\widetilde{{\bf M}^{-1}}$

where $\mathbb{E}|\tilde{{\bf M}}_{ij}|^{8}=O(\frac{1}{n})$ and
$\mathbb{E}|(\widetilde{{\bf M}^{-1}})_{ij}|^{4}=O(\frac{1}{n})$. Then
martingale decomposition term becomes

$\displaystyle{\rm tr}({\bf H}\\!-\\!z)^{-1}\\!-\\!{\rm tr}({\bf
H}^{(k)}\\!-\\!z)^{-1}$ $\displaystyle=$
$\displaystyle\\!-\\!\frac{1}{n}\sum_{i,j=1}^{8}{\bf
M}_{ij}^{-1}\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}\\!-\\!z)^{-2}\boldsymbol{\alpha}_{k,i}\\!+\\!\frac{1}{n}\sum_{i,j=1}^{8}\widetilde{{\bf
M}^{-1}}_{ij}\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}\\!-\\!z)^{-2}\boldsymbol{\alpha}_{k,i}.$

First we consider the remainder term

$\displaystyle\sum_{n=1}^{\infty}\mathbb{P}\left(\left|\sum_{k=1}^{n}(\mathbb{E}_{k}-\mathbb{E}_{k-1})\frac{1}{n}\sum_{i,j=1}^{8}\widetilde{{\bf
M}^{-1}}_{ij}\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}\right|>\varepsilon\right)$
$\displaystyle\leq$
$\displaystyle\sum_{n=1}^{\infty}\frac{1}{\varepsilon^{2}}\mathbb{E}\left|\sum_{k=1}^{n}(\mathbb{E}_{k}-\mathbb{E}_{k-1})\frac{1}{n}\sum_{i,j=1}^{8}\widetilde{{\bf
M}^{-1}}_{ij}\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}\right|^{2}$ $\displaystyle\leq$
$\displaystyle\sum_{n=1}^{\infty}\frac{1}{\varepsilon^{2}}\mathbb{E}\sum_{k=1}^{n}\left|(\mathbb{E}_{k}-\mathbb{E}_{k-1})\frac{1}{n}\sum_{i,j=1}^{8}\widetilde{{\bf
M}^{-1}}_{ij}\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}\right|^{2}$ $\displaystyle\leq$
$\displaystyle\sum_{n=1}^{\infty}\frac{1}{\varepsilon^{2}}\mathbb{E}\sum_{k=1}^{n}\frac{1}{n^{2}}\left|\sum_{i,j=1}^{8}\widetilde{{\bf
M}^{-1}}_{ij}\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}\right|^{2}$ $\displaystyle\leq$
$\displaystyle\sum_{n=1}^{\infty}\frac{1}{\varepsilon^{2}}\mathbb{E}\sum_{k=1}^{n}\frac{1}{n^{2}}\sum_{i,j,i^{\prime},j^{\prime}=1}^{8}\left|\widetilde{{\bf
M}^{-1}}_{ij}\widetilde{{\bf
M}^{-1}}_{i^{\prime}j^{\prime}}\right|^{2}n^{2\delta}+\sum_{n=1}^{\infty}\frac{1}{n}\mathbb{P}(\mathcal{A}^{c})$
$\displaystyle\leq$
$\displaystyle\sum_{n=1}^{\infty}\frac{1}{\varepsilon^{2}}\sum_{k=1}^{n}\frac{n^{2\delta}}{n^{2}}\sum_{i,j,i^{\prime},j^{\prime}=1}^{8}\sqrt{\mathbb{E}|\widetilde{{\bf
M}^{-1}}_{ij}|^{4}}\sqrt{\mathbb{E}|\widetilde{{\bf
M}^{-1}}_{i^{\prime}j^{\prime}}|^{4}}+\sum_{n=1}^{\infty}\frac{1}{n}\mathbb{P}(\mathcal{A}^{c})$
$\displaystyle=$
$\displaystyle\sum_{n=1}^{\infty}O\left(\frac{1}{n^{2(1-\delta)}\varepsilon^{2}}\right)+\sum_{n=1}^{\infty}O\left(\frac{1}{n^{1+2\delta}}\right)<\infty,$

where the first two inequalities are due to Chebyshev’s inequality and
Burkholder inequality, respectively, and we define the event

$\displaystyle\mathcal{A}\\!=\\!\\{\mbox{for any
}i,j,i^{\prime},j^{\prime},|\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}\\!-\\!zI)^{-2}\boldsymbol{\alpha}_{k,i}|<n^{\delta}\ \mbox{and
}|\boldsymbol{\beta}_{k,j^{\prime}}({\bf
H}^{(k)}\\!-\\!z)^{-2}\boldsymbol{\alpha}_{k,i^{\prime}}|<n^{\delta}\\}.$

The specific forms of $\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}$ include

$\displaystyle\chi^{\prime}_{k}G^{2}\chi_{k},\
\omega^{\prime}_{k}G^{2}\omega_{k},\ \chi^{\prime}_{k}G^{2}\omega_{k},\
e^{\prime}_{k}G^{2}\chi_{k},\ e^{\prime}_{k}G^{2}\omega_{k},\
e^{\prime}_{k}G^{2}e_{k}.$

By Cauchy-Schwarz inequality and the formula (2.3) of [3], we have

$\displaystyle\mathbb{P}(\mathcal{A}^{c})\leq\frac{8^{4}}{n^{2\delta}}\mathbb{E}\left|\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}\boldsymbol{\beta}_{k,j^{\prime}}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i^{\prime}}\right|^{2}=O\left(\frac{1}{n^{2\delta}}\right).$

The remainder term can be ignorable in the sense of almost sure. Now we
consider the main term

$\displaystyle\delta_{k}=\frac{1}{n}(\mathbb{E}_{k}-\mathbb{E}_{k-1})\left[\sum_{i,j=1}^{8}{\bf
M}_{ij}^{-1}\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}\right]$ $\displaystyle=$
$\displaystyle\frac{1}{n}(\mathbb{E}_{k}-\mathbb{E}_{k-1})\left[\sum_{i,j=1}^{8}{\bf
M}_{ij}^{-1}\left(\boldsymbol{\beta}_{k,j}({\bf
H}^{(k)}-z)^{-2}\boldsymbol{\alpha}_{k,i}-{\rm tr}({\bf
H}^{(k)}-z)^{-2}\mathbb{E}\boldsymbol{\alpha}_{k,i}\boldsymbol{\beta}_{k,j}\right)\right]$
$\displaystyle\sum_{n=1}^{\infty}\mathbb{P}\left(\left|\sum_{k=1}^{n}\delta_{k}\right|>\varepsilon\right)\leq\sum_{n=1}^{\infty}\frac{1}{\varepsilon^{2}}\mathbb{E}\left|\sum_{k=1}^{n}\delta_{k}\right|^{2}\leq\sum_{n=1}^{\infty}\frac{1}{\varepsilon^{2}}\sum_{k=1}^{n}\mathbb{E}\left|\delta_{k}\right|^{2}=\sum_{n=1}^{\infty}O\left(\frac{1}{n^{2}\varepsilon^{2}}\right)<\infty.$

The first two inequality are due to Markov inequality and Burkholder
inequality, respectively, and the next equality is due to the fact
$\mathbb{E}|\delta_{k}|^{2}=O(\frac{1}{n^{3}})$. By Borel-Cantelli Lemma, we
have ${\rm tr}({\bf H}-z)^{-1}-\mathbb{E}{\rm tr}({\bf
H}-z)^{-1}\overset{a.s.}{\to}0.$

##  5 The equality for Stieltjes transform of LSD

From the arguments in Section 3 and 4, we have

$\displaystyle{\rm tr}({\bf H}-zI)^{-1}-{\rm tr}({\bf H}_{N}-zI)^{-1}$
$\displaystyle=$ $\displaystyle{\rm tr}({\bf H}-zI)^{-1}-\mathbb{E}{\rm
tr}({\bf H}-zI)^{-1}+\mathbb{E}{\rm tr}({\bf H}-zI)^{-1}-\mathbb{E}{\rm
tr}({\bf H}_{N}-zI)^{-1}$ $\displaystyle+\mathbb{E}{\rm tr}({\bf
H}_{N}-zI)^{-1}-{\rm tr}({\bf H}_{N}-zI)^{-1}\overset{a.s.}{\to}0.$

From Theorem 1, we have for the random variables with general underlying
distribution,

$\displaystyle\frac{1}{p+q}{\rm Tr}G_{b}(z)=$
$\displaystyle(z-1)(\frac{1}{p+q}{\rm Tr}G_{xy}((1-z)^{2})+\frac{1}{p+q}{\rm
Tr}G_{yx}((1-z)^{2}))$ $\displaystyle=$
$\displaystyle(z-1)\left(\frac{q-p}{p+q}\frac{1}{-(1-z)^{2}}+\frac{2p}{p+q}\frac{1}{p}{\rm
Tr}G_{xy}((1-z)^{2})\right),$

where we use the fact that the spectrum of $({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}}{\bf X}{\bf P}_{y}{\bf X}^{\prime}({\bf X}{\bf
X}^{\prime})^{-\frac{1}{2}}$ and $({\bf Y}{\bf Y}^{\prime})^{-\frac{1}{2}}{\bf
Y}{\bf P}_{x}{\bf Y}^{\prime}({\bf Y}{\bf Y}^{\prime})^{-\frac{1}{2}}$ differs
by $|p-q|$ zero eigenvalues. That is $f(\frac{1}{p}{\rm
Tr}G_{xy}((1-z)^{2}))-f(\frac{1}{p}{\rm
Tr}G_{xy}^{N}((1-z)^{2}))\overset{a.s.}{\to}0$, then we have $\frac{1}{p}{\rm
Tr}G_{xy}((1-z)^{2})-\frac{1}{p}{\rm
Tr}G_{xy}^{N}((1-z)^{2})\overset{a.s.}{\to}0$.

From the relationship in [2], under Gaussian assumption the Stieltjes
transform of the SCC matrix and the corresponding noncentral Fisher matrix
satisfies

$\displaystyle m(z)$
$\displaystyle=\int\frac{1}{t-z}dF^{SCC}(t)=\int\frac{1}{t-z}dF^{F}\left(\frac{(n-q)t}{q(1-t)}\right)$
(4) $\displaystyle=\int\frac{1}{\frac{q}{n-q}x/(1+\frac{q}{n-q}x)-z}dF^{F}(x)$
$\displaystyle=\int\frac{1+\frac{q}{n-q}x}{\frac{q}{n-q}x-z(1+\frac{q}{n-q}x)}dF^{F}(x)=\frac{n-q}{q(1-z)}\int\frac{1+\frac{q}{n-q}x}{x-\frac{z(n-q)}{q(1-z)}}dF^{F}(x)$
$\displaystyle=\frac{n-q}{q(1-z)}\int\frac{\frac{q}{n-q}[x-\frac{z(n-q)}{q(1-z)}]+\frac{z}{1-z}+1}{x-\frac{z(n-q)}{q(1-z)}}dF^{F}(x)$
$\displaystyle=\frac{1}{1\\!-\\!z}+\frac{n\\!-\\!q}{q(1\\!-\\!z)^{2}}\int\frac{1}{x\\!-\\!\frac{z(n\\!-\\!q)}{q(1\\!-\\!z)}}dF^{F}(x)=\frac{1}{1\\!-\\!z}+\frac{n\\!-\\!q}{q(1\\!-\\!z)^{2}}m_{F}\left(\frac{z(n\\!-\\!q)}{q(1\\!-\\!z)}\right).$

The second equality in the first line is due to the relationship between
$F^{SCC}(\cdot)$ and $F^{F}(\cdot)$,

$\displaystyle
F^{SCC}(t)\\!=\\!\frac{1}{p}\sum_{i=1}^{p}I\\{\lambda_{i}^{SCC}\leq
t\\}\\!=\\!\frac{1}{p}\sum_{i=1}^{p}I\left\\{\frac{(n\\!-\\!q)\lambda_{i}^{SCC}}{q(1\\!-\\!\lambda_{i}^{SCC})}\leq\frac{(n\\!-\\!q)t}{q(1\\!-\\!t)}\right\\}\\!=\\!F^{F}\left(\frac{(n\\!-\\!q)t}{q(1\\!-\\!t)}\right).$

The equality in the second line is due to the variable substitution

$\displaystyle x=\frac{(n-q)t}{q(1-t)}\Leftrightarrow
t=\frac{qx}{n-q}/(1+\frac{q}{n-q}x).$

Notice $F^{F}(\cdot)$ is the LSD of noncentral Fisher matrix with noncentral
parameter matrix $\boldsymbol{\Xi}=\frac{n}{q}{\bf T}(\frac{1}{n}{\bf Q}{\bf
Q})^{-1}{\bf T}^{\prime}$, where the eigenvalues of ${\bf T}{\bf T}^{\prime}$
is
$\frac{(\lambda_{i}^{\boldsymbol{\Lambda}})^{2}}{1-(\lambda_{i}^{\boldsymbol{\Lambda}})^{2}}$,
and the dimension of ${\bf Q}$ is $p\times n$. By the results of sample
covariance matrix, we have the Stieltjes transform of the LSD of
$\boldsymbol{\Xi}$ satisfies

$\displaystyle
m_{\boldsymbol{\Xi}}(z)=\int\frac{1}{t(1-\frac{p}{n}-\frac{p}{n}zm_{\boldsymbol{\Xi}}(z))-z}dH\left(\sqrt{\frac{\frac{q}{n}t}{1+\frac{q}{n}t}}\right)$

where we use the fact

$\displaystyle
H(x)\\!=\\!\lim_{p\to\infty}\frac{1}{p}\sum_{i=1}^{p}I\\{\lambda_{i}^{\boldsymbol{\Lambda}}\leq
x\\}\\!=\\!\lim_{p\to\infty}\frac{1}{p}\sum_{i=1}^{p}I\left\\{\frac{n/q(\lambda_{i}^{\boldsymbol{\Lambda}})^{2}}{1\\!-\\!(\lambda_{i}^{\boldsymbol{\Lambda}})^{2}}\leq\frac{n/qx^{2}}{1\\!-\\!x^{2}}\right\\}\\!=\\!H^{\frac{n}{q}TT^{\prime}}\left(\frac{n/qx^{2}}{1\\!-\\!x^{2}}\right),$

and by variable substitution, we have

$\displaystyle
H^{\frac{n}{q}TT^{\prime}}(t)=H\left(\sqrt{\frac{\frac{q}{n}t}{1+\frac{q}{n}t}}\right).$

According to the main result in [18], we have the Stieltjes transform of
$F^{F}(\cdot)$ with noncentral parameter matrix $\boldsymbol{\Xi}$ satisfies

$\displaystyle m_{F}(z)\\!=\\!$
$\displaystyle\int\left[\frac{t}{1\\!+\\!(\frac{p}{q}\\!+\\!\frac{p}{n\\!-\\!q}z)m_{F}}\\!+\\!\frac{1\\!-\\!\frac{p}{q}}{1\\!+\\!\frac{p}{n\\!-\\!q}zm_{F}}\\!-\\!z\frac{1\\!+\\!(\frac{p}{q}\\!+\\!\frac{p}{n\\!-\\!q}z)m_{F}}{1\\!+\\!\frac{p}{n\\!-\\!q}zm_{F}}\right]^{-1}dH^{\boldsymbol{\Xi}}(t)$
$\displaystyle\frac{m_{F}(z)}{1\\!+\\!(\frac{p}{q}\\!+\\!\frac{p}{n-q}z)m_{F}}\\!=\\!$
$\displaystyle\int\left[t\\!+\\!\frac{(1\\!-\\!\frac{p}{q})[1\\!+\\!(\frac{p}{q}\\!+\\!\frac{p}{n\\!-\\!q}z)m_{F}]}{1\\!+\\!\frac{p}{n\\!-\\!q}zm_{F}}\\!-\\!z\frac{[1\\!+\\!(\frac{p}{q}\\!+\\!\frac{p}{n\\!-\\!q}z)m_{F}]^{2}}{1\\!+\\!\frac{p}{n\\!-\\!q}zm_{F}}\right]^{-1}dH^{\boldsymbol{\Xi}}(t)$
$\displaystyle=$ $\displaystyle
m_{\boldsymbol{\Xi}}\left(z\frac{[1+(\frac{p}{q}+\frac{p}{n-q}z)m_{F}]^{2}}{1+\frac{p}{n-q}zm_{F}}-\frac{(1-\frac{p}{q})[1+(\frac{p}{q}+\frac{p}{n-q}z)m_{F}]}{1+\frac{p}{n-q}zm_{F}}\right).$

That means the pair

$\displaystyle\left(z\frac{[1+(\frac{p}{q}+\frac{p}{n-q}z)m_{F}]^{2}}{1+\frac{p}{n-q}zm_{F}}-\frac{(1-\frac{p}{q})[1+(\frac{p}{q}+\frac{p}{n-q}z)m_{F}]}{1+\frac{p}{n-q}zm_{F}},\quad\frac{m_{F}(z)}{1+(\frac{p}{q}+\frac{p}{n-q}z)m_{F}}\right)$

satisfies the equation of the pair $(a,b)$

$\displaystyle
b=\int\frac{1}{t(1-\frac{p}{n}-\frac{p}{n}ab)-a}dH\left(\sqrt{\frac{\frac{q}{n}t}{1+\frac{q}{n}t}}\right).$
(5)

According to the relationship (4) between $m$ and $m_{F}$, we have the pair

$\displaystyle\frac{1\\!+\\!\frac{p}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]}{1\\!+\\!\frac{pz}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]}\left\\{\frac{z(n\\!-\\!q)}{q(1\\!-\\!z)}\left[1\\!+\\!\frac{p[(1\\!-\\!z)m\\!-\\!1]}{n\\!-\\!q}\right]\\!-\\!\left(1\\!-\\!\frac{p}{q}\right)\right\\}$

and

$\displaystyle\frac{\frac{q(1\\!-\\!z)}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]}{1\\!+\\!\frac{p}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]}$

satisfies the same equation of the pair $(a,b)$.

Then substituting the above pair in formula (5), we have

$\displaystyle\frac{\frac{q(1-z)}{n-q}[(1-z)m-1]}{1+\frac{p}{n-q}[(1-z)m-1]}=\int\frac{1}{tG_{1}(z,m)-G_{2}(z,m)}dH\left(\sqrt{\frac{\frac{q}{n}t}{1\\!+\\!\frac{q}{n}t}}\right)$
$\displaystyle\Leftrightarrow
G_{1}(z,m)\frac{\frac{q(1-z)}{n-q}[(1-z)m-1]}{1+\frac{p}{n-q}[(1-z)m-1]}=\int_{0}^{1}\left[\frac{n}{q}\frac{x^{2}}{1-x^{2}}-\frac{G_{2}(z,m)}{G_{1}(z,m)}\right]^{-1}dH(x)$
(6)

where

$\displaystyle G_{1}(z,m)\\!=$ $\displaystyle
1\\!-\\!\frac{p}{n}\\!-\\!\frac{p}{n}\frac{\frac{q(1\\!-\\!z)}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]}{1\\!+\\!\frac{pz}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]}\left\\{\frac{z(n\\!-\\!q)}{q(1\\!-\\!z)}\left[1\\!+\\!\frac{p[(1\\!-\\!z)m\\!-\\!1]}{n\\!-\\!q}\right]-\left(1\\!-\\!\frac{p}{q}\right)\right\\}$
$\displaystyle G_{2}(z,m)\\!=$
$\displaystyle\frac{1\\!+\\!\frac{p}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]}{1\\!+\\!\frac{pz}{n\\!-\\!q}[(1\\!-\\!\\!z)m\\!-\\!1]}\left\\{\frac{z(n\\!-\\!q)}{q(1\\!-\\!z)}\left[1\\!+\\!\frac{p}{n\\!-\\!q}[(1\\!-\\!z)m\\!-\\!1]\right]\\!-\\!\left(1\\!-\\!\frac{p}{q}\right)\right\\}.$

The Stieltjes transform of the LSD of the SCC matrix satisfies the equation
(6).

According to the relationship between ${\rm Tr}G_{b}$ and ${\rm Tr}G_{xy}$ in
Theorem 1, we have the pair

$\displaystyle\left((1-z)^{2},\frac{(c_{1}+c_{2})m_{b}+\frac{c_{1}-c_{2}}{1-z}}{2c_{1}(z-1)}\right)$

satisfies the equation (6) as $(z,m)$, where $m_{b}$ means the limit of
$\frac{1}{p+q}{\rm Tr}G_{b}$, the Stieltjes transform of the LSD of the block
correlation matrix ${\bf B}$.

Moreover, we choose Gamma distribution samples with dimension setting:
$(p,q,n)=(1000,3000,8000)$, to give a histogram for comparison between the ESD
and the LSD of $\mathcal{S}_{xy}$. In first subfigure of Figure 1, we consider
the fixed rank alternative case, that is the same with null case except some
outliers. In second subfigure of Figure 1, we consider the case of rank $p/2$,
and there is an interesting phenomenon because of the spectrum gap of $H$,
where more details and further results need more exploration of the properties
of LSD. In last subfigure of Figure 1, we consider the full rank alternative
case, and they all fit well with our theoretical results.

$\boldsymbol{\Lambda}\boldsymbol{\Lambda}^{\prime}=\frac{1}{2}{\bf
I}_{5}\oplus 0{\bf I}_{p-5}$

$\boldsymbol{\Lambda}\boldsymbol{\Lambda}^{\prime}=\frac{1}{2}{\bf
I}_{p/2}\oplus 0{\bf I}_{p/2}$

$\boldsymbol{\Lambda}\boldsymbol{\Lambda}^{\prime}=\frac{1}{2}{\bf I}_{p}$

Figure 1: LSD of SCC matrix with underlying distribution G(4,2).

##  6 Some Lemmas

######  Lemma 1.

Consider the real matrix ${\bf B}$ and ${\bf A}$ have the following structure:

$\displaystyle{\bf B}=\begin{pmatrix}{\bf I}_{p},&{\bf A}\\\ {\bf
A}^{\prime},&{\bf I}_{q}\end{pmatrix},$

and denote the $\min\\{p+q\\}$ largest singular values of ${\bf A}$ by
$l_{1}\geq l_{2}\geq\dots\geq l_{\min\\{p,q\\}}$, then the $p+q$ eigenvalues
of ${\bf B}$ will consist of

$\displaystyle\\{\underbrace{1+l_{1},\dots,1+l_{\min\\{p,q\\}}}_{\min\\{p,q\\}},\underbrace{1,\dots,1}_{\max\\{p,q\\}-\min\\{p,q\\}},\underbrace{1-l_{\min\\{p,q\\}},\dots,1-l_{1}}_{\min\\{p,q\\}}\\}.$

###### Proof.

Without loss of generality, we assume $p<q$. Set singular value decomposition
(SVD) for ${\bf A}_{p\times q}={\bf U}{\bf L}{\bf V}^{\prime}$, where ${\bf
U}_{p\times p}=(u_{1},\dots,u_{p})$, ${\bf V}_{q\times
q}=(v_{1},\dots,v_{q})$, ${\bf U}^{\prime}{\bf U}={\bf I}_{p}$, ${\bf
V}^{\prime}{\bf V}={\bf I}_{q}$. Denote the eigenvalue of ${\bf B}$ by
$\lambda$.

First, we will show $\\{1+l_{1},\dots,1+l_{i},\dots,1+l_{p}\\}$ are the
eigenvalues of ${\bf B}$ with eigenvectors
$\\{(u_{1}^{\prime},v_{1}^{\prime})^{\prime},\dots,(u_{i}^{\prime},v_{i}^{\prime})^{\prime},\dots,(u_{p}^{\prime},v_{p}^{\prime})^{\prime}\\}$.
According to SVD for ${\bf A}$, we have ${\bf
A}v_{i}=\sum_{j=1}^{p}l_{j}u_{j}v_{j}^{\prime}v_{i}=l_{i}u_{i}$ and ${\bf
A}^{\prime}u_{i}=\sum_{j=1}^{p}l_{j}v_{j}u_{j}^{\prime}u_{i}=l_{i}v_{i}$, and
then

$\displaystyle\begin{pmatrix}{\bf I}_{p},&{\bf A}\\\ {\bf A}^{\prime},&{\bf
I}_{q}\end{pmatrix}\begin{pmatrix}u_{i}\\\
v_{i}\end{pmatrix}=\begin{pmatrix}{\bf I}_{p}u_{i}+{\bf A}v_{i}\\\ {\bf
A}^{\prime}u_{i}+{\bf
I}_{q}v_{i}\end{pmatrix}=\begin{pmatrix}(1+l_{i})u_{i}\\\
(1+l_{i})v_{i}\end{pmatrix}=(1+l_{i})\begin{pmatrix}u_{i}\\\
v_{i}\end{pmatrix}.$

Second, we will show $\\{1-l_{1},\dots,1-l_{i},\dots,1-l_{p}\\}$ are the
eigenvalues of ${\bf B}$ with eigenvectors
$\\{(\tilde{u}_{p+1}^{\prime},\tilde{v}_{p+1}^{\prime})^{\prime},\dots,(\tilde{u}_{p+i}^{\prime},\tilde{v}_{p+i}^{\prime})^{\prime},\dots,(\tilde{u}_{2p}^{\prime},\tilde{v}_{2p}^{\prime})^{\prime}\\}$.
According to the definition of eigenvalue and eigenvector, we have

$\displaystyle\begin{pmatrix}{\bf I}_{p},&{\bf A}\\\ {\bf A}^{\prime},&{\bf
I}_{q}\end{pmatrix}\begin{pmatrix}u\\\ v\end{pmatrix}=\begin{pmatrix}u+{\bf
A}v\\\ {\bf A}^{\prime}u+v\end{pmatrix}=\lambda\begin{pmatrix}u\\\
v\end{pmatrix}.$

Because $(u^{\prime},v^{\prime})^{\prime}$ and
$(u_{i}^{\prime},v_{i}^{\prime})^{\prime}$ are the eigenvectors of ${\bf B}$
with different eigenvalues $\lambda$ and $1+l_{i}$, they are orthogonal, i.e.,
$u_{i}^{\prime}u+v_{i}^{\prime}v=0$ for any $i=1,\dots,p$. The above equals to

$\begin{pmatrix}(1-\lambda)u+{\bf A}v\\\ {\bf
A}^{\prime}u+(1-\lambda)v\end{pmatrix}=0\
\Rightarrow\left\\{\begin{array}[]{lr}(1-\lambda)u_{i}^{\prime}u+u_{i}^{\prime}{\bf
A}v=0\\\ v_{i}^{\prime}{\bf
A}^{\prime}u+(1-\lambda)v_{i}^{\prime}v=0\end{array}.\right.$ (7)

We have

$\displaystyle u_{i}^{\prime}{\bf A}=({\bf
A}^{\prime}u_{i})^{\prime}=(l_{i}v_{i})^{\prime}=l_{i}v_{i}^{\prime},\ \
v_{i}^{\prime}{\bf A}^{\prime}=({\bf
A}v_{i})^{\prime}=(l_{i}u_{i})^{\prime}=l_{i}u_{i}^{\prime}.$

Then the equation set (7) becames

$\left\\{{\begin{array}[]{l}{(1-\lambda)[u_{i}^{\prime}u+v_{i}^{\prime}v]+[l_{i}-(1-\lambda)]v_{i}^{\prime}v=0}\\\
{(1-\lambda)[u_{i}^{\prime}u+v_{i}^{\prime}v]+[l_{i}-(1-\lambda)]u_{i}^{\prime}u=0}\end{array}}\right.\Rightarrow\left\\{{\begin{array}[]{l}{[l_{i}-(1-\lambda)]v_{i}^{\prime}v=0}\\\
{[l_{i}-(1-\lambda)]u_{i}^{\prime}u=0}\end{array}}\right..$

We confirm $\lambda=1-l_{i}$. If not, the dimension of the space spanned by
$(u_{1},u_{2},\dots,u_{p})$ equals to $p$, there is no $u$ that satisfies
$u_{i}^{\prime}u=0$ for all $i$. When the index $i$ varies, the corresponding
eigenvector $(u,v)$ changes accordingly, and will fill the orthogonal subspace
of the subspace spanned by
$\\{(u_{1}^{\prime},v_{1}^{\prime})^{\prime},\dots,(u_{i}^{\prime},v_{i}^{\prime})^{\prime},$
$\dots,(u_{p}^{\prime},v_{p}^{\prime})^{\prime}\\}$.

Last, we will show the remaining eigenvalues of ${\bf B}$ are all $1$. There
are $q-p$ $v$ such that each is orthogonal to $\\{v_{1},\dots,v_{p}\\}$,
because there are $q$ orthogonal columns in ${\bf V}$. Extend the base
$\\{v_{p+1},v_{p+2},\dots,v_{q}\\}$ in $q$-dimensional linear space to the
base
$\\{(0^{\prime},v_{p+1}^{\prime})^{\prime},(0^{\prime},v_{p+2}^{\prime})^{\prime},\dots,(0^{\prime},v_{q}^{\prime})^{\prime}\\}$
in $p+q$-dimensional linear space. Now, ${\bf
A}v_{p+i}=\sum_{j=1}^{p}l_{j}u_{j}v_{j}^{\prime}v_{p+i}=0$ and ${\bf
A}^{\prime}u_{p+i}=0$. That is,

$\displaystyle\begin{pmatrix}{\bf I}_{p},&{\bf A}\\\ {\bf A}^{\prime},&{\bf
I}_{q}\end{pmatrix}\begin{pmatrix}0\\\
v_{p+i}\end{pmatrix}=\begin{pmatrix}{\bf A}v_{p+i}\\\
v_{p+i}\end{pmatrix}=1\begin{pmatrix}0\\\ v_{p+i}\end{pmatrix}.$

∎

######  Lemma 2.

Under the assumptions and notations of Theorem 2, we have

$\displaystyle\frac{1}{\sqrt{1-s}}\frac{\partial\Phi_{ji}}{\partial\hat{W}_{ji}}=\frac{1}{\sqrt{s}}\frac{\partial\Phi_{ji}}{\partial
W_{ji}}.$

###### Proof.

According to the notation (3) and chain rule, we have

$\displaystyle\frac{\partial\Phi_{ji}}{\partial\hat{W}_{ji}}=\frac{\partial[\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)({\bf H}(s)-zI)^{-2}({\bf I}-{\bf
P}_{x(s)})]_{ji}}{\partial\hat{W}_{ji}},$ $\displaystyle=$
$\displaystyle\left[\boldsymbol{\Gamma}^{\prime}\frac{\partial{\bf
Q}(s)}{\partial\hat{W}_{ji}}G^{2}(s)({\bf I}-{\bf
P}_{x(s)})-\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)G(s)\frac{\partial{\bf
P}_{x(s)}}{\partial\hat{W}_{ji}}G^{2}(s)({\bf I}-{\bf P}_{x(s)})\right.$
$\displaystyle-\left.\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)G^{2}(s)\frac{\partial{\bf P}_{x(s)}}{\partial\hat{W}_{ji}}G(s)({\bf
I}-{\bf P}_{x(s)})-\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)G^{2}(s)\frac{\partial{\bf P}_{x(s)}}{\partial\hat{W}_{ji}}\right]_{ji}.$

To calculate the above derivatives, we need some matrix derivative results

$\displaystyle\frac{\partial{\bf X}(s)}{\partial\hat{W}_{ji}}$
$\displaystyle=\sqrt{1\\!-\\!s}\boldsymbol{\Gamma}1_{ji},\ \frac{\partial{\bf
X}(s)^{\prime}}{\partial\hat{W}_{ji}}=\sqrt{1\\!-\\!s}1_{ij}\boldsymbol{\Gamma}^{\prime},\
\frac{\partial{\bf
X}(s)}{\partial{W}_{ji}}=\sqrt{s}\boldsymbol{\Gamma}1_{ji},\
\frac{\partial{\bf
X}(s)^{\prime}}{\partial{W}_{ji}}=\sqrt{s}1_{ij}\boldsymbol{\Gamma}^{\prime},$
$\displaystyle\frac{\partial{\bf Q}(s)}{\partial\hat{W}_{ji}}$
$\displaystyle=\\!-\\!({\bf X}(s){\bf
X}(s)^{\prime})^{-1}\left(\frac{\partial{\bf X}(s)}{\partial\hat{W}_{ji}}{\bf
X}(s)^{\prime}\\!+\\!{\bf X}(s)\frac{\partial{\bf
X}(s)^{\prime}}{\partial\hat{W}_{ji}}\right){\bf Q}(s)\\!+\\!({\bf X}(s){\bf
X}(s)^{\prime})^{-1}\frac{\partial{\bf X}(s)}{\partial\hat{W}_{ji}}$
$\displaystyle=\sqrt{1-s}[({\bf X}(s){\bf
X}(s)^{\prime})^{-1}\boldsymbol{\Gamma}1_{ji}({\bf I}-{\bf P}_{x(s)})-{\bf
Q}(s)1_{ij}\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)],$
$\displaystyle\frac{\partial{\bf Q}(s)}{\partial{W}_{ji}}$
$\displaystyle=\sqrt{s}[({\bf X}(s){\bf
X}(s)^{\prime})^{-1}\boldsymbol{\Gamma}1_{ji}({\bf I}-{\bf P}_{x(s)})-{\bf
Q}(s)1_{ij}\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)],$
$\displaystyle\frac{\partial{\bf P}_{x(s)}}{\partial\hat{W}_{ji}}=$
$\displaystyle\frac{\partial{\bf X}(s)^{\prime}}{\partial\hat{W}_{ji}}{\bf
Q}(s)+{\bf X}(s)^{\prime}\frac{\partial{\bf Q}(s)}{\partial\hat{W}_{ji}}$
$\displaystyle=$
$\displaystyle\sqrt{1-s}1_{ij}\boldsymbol{\Gamma}^{\prime}{\bf
Q}(s)+\sqrt{1-s}[{\bf Q}^{\prime}(s)\boldsymbol{\Gamma}1_{ji}({\bf I}-{\bf
P}_{x(s)})-{\bf P}_{x(s)}1_{ij}\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)]$
$\displaystyle=$ $\displaystyle\sqrt{1-s}({\bf I}-{\bf
P}_{x(s)})1_{ij}\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)+\sqrt{1-s}{\bf
Q}^{\prime}(s)\boldsymbol{\Gamma}1_{ji}({\bf I}-{\bf P}_{x(s)}),$
$\displaystyle\frac{\partial{\bf P}_{x(s)}}{\partial{W}_{ji}}=$
$\displaystyle\sqrt{s}({\bf I}-{\bf
P}_{x(s)})1_{ij}\boldsymbol{\Gamma}^{\prime}{\bf Q}(s)+\sqrt{s}{\bf
Q}^{\prime}(s)\boldsymbol{\Gamma}1_{ji}({\bf I}-{\bf P}_{x(s)}).$

After substituting the above results into
$\frac{\partial\Phi_{ji}}{\partial\hat{W}_{ji}}$ and
$\frac{\partial\Phi_{ji}}{\partial W_{ji}}$ respectively, the proof is
completed. ∎

######  Lemma 3.

Under the assumptions of Theorem 2, for matrix ${\bf Q}(s)$, we have the
operator norm of ${\bf X}(s)$ and $({\bf X}(s){\bf X}(s)^{\prime})^{-1}$ is
order $1$ with high probability, respectively, that is, there exist $b^{+}$
and $b^{-}$ such that

$\displaystyle\mathbb{P}(s_{1}({\bf X}(s))>2b^{+})=o(n^{-l}),\
\mathbb{P}\left(\mbox{for all large n, }\|({\bf X}(s){\bf
X}^{\prime}(s))^{-1}\|\leq\frac{1}{kb^{-}}\right)=1,$

where $k$ is related to the operator norm of $\boldsymbol{\Lambda}$.

###### Proof.

According to the result of the largest eigenvalues of sample covariance
matrix, we have the estimate

$\displaystyle\mathbb{P}(s_{1}({\bf W}(s))>2)=o(n^{-l})$

holds for any $l$. And it holds when replacing ${\bf W}(s)$ with ${\bf Y}$.
Even we do not need to truncate their entries.

Obviously $\|\boldsymbol{\Gamma}\|<1$, define ${\bf
C}=\boldsymbol{\Gamma}^{-1}\boldsymbol{\Lambda}$ and choose constant $b^{+}$
such that $s_{1}({\bf C})=b^{+}-2$. According to Theorem A.8 and A.9 in [4],
we have

$\displaystyle s_{1}({\bf X}(s))\leq\|\boldsymbol{\Gamma}\|s_{1}({\bf C}{\bf
Y}+{\bf W}(s)))\leq s_{1}({\bf C}{\bf Y})+s_{1}({\bf W}(s)).$

We can easily get

$\displaystyle\mathbb{P}(s_{1}({\bf X}(s))>2b^{+})\leq\mathbb{P}(s_{1}({\bf
C})s_{1}({\bf Y})+s_{1}({\bf W}(s))>2b^{+})$ $\displaystyle\leq$
$\displaystyle\mathbb{P}(\\{s_{1}({\bf Y})>2\\}\cup\\{s_{1}({\bf
W}(s))>2\\}\cup\\{s_{1}({\bf C})>b^{+}-1\\})$ $\displaystyle\leq$
$\displaystyle\mathbb{P}(s_{1}({\bf Y})>2)+\mathbb{P}(s_{1}({\bf
W}(s))>2)+\mathbb{P}(s_{1}({\bf C})>b^{+}-1)=o(n^{-l}).$

Next we consider the operator norm of $({\bf X}(s){\bf X}(s)^{\prime})^{-1}$,
which equals to that of $[({\bf C}{\bf Y}+{\bf W}(s))({\bf C}{\bf Y}+{\bf
W}(s))^{\prime}]^{-1}$. First we know when $c_{1}<1$, LSD of the later one has
no mass at $0$. Next, we need to prove $0$ is outside the support, which is
due to the sufficient condition for determining the support. Let $b$ satisfies
the following equation set

$\left\\{\begin{aligned}
\frac{m}{1+c_{1}m}&=\int\frac{dH^{y}(t)}{t+(1-c_{1})(1+c_{1}m)}\\\
b&=1+c_{1}m\end{aligned}\right.$

then we have

$\displaystyle\frac{1}{c_{1}}\left(1-\frac{1}{b}\right)=m_{H^{y}}(-(1-c_{1})(1+c_{1}m))=m_{H^{y}}\left(-(1-c_{1})b\right).$
(8)

When $b\to 0^{+}$, the left hand side
$\frac{1}{c_{1}}\left(1-\frac{1}{b}\right)\to-\infty$ and the right hand side
$m_{H^{y}}\left(-(1-c_{1})b\right)\to\int_{0}^{+\infty}\frac{1}{t}dH^{y}(t)>0$;
when $b\to+\infty$, the left hand side
$\frac{1}{c_{1}}\left(1-\frac{1}{b}\right)\to\frac{1}{c_{1}}>0$, and the right
hand side $m_{H^{y}}\left(-(1-c_{1})b\right)\to 0^{+}$. Noting the left hand
side is monotone continuous function, according to Existence Theorem of Zero
Points, we have the above equation about $b\in\mathbb{R}$ has at least one
real solution in positive axis. Then we analysis the positive solution $b$. By
formula (8) and basic calculation

$\displaystyle
m_{H^{y}}^{-1}\left(\frac{1}{c_{1}}\left(1-\frac{1}{b}\right)\right)=-(1-c_{1})(1+c_{1}m)=-(1-c_{1})b,$

we have the derivatives of discriminant function

$\displaystyle
x^{\prime}(b)=\frac{-2}{b^{2}}m_{H^{y}}^{-1}\left(\frac{1}{c_{1}}\left(1\\!-\\!\frac{1}{b}\right)\right)\\!+\\!\frac{1}{b^{2}}\frac{\partial
m_{H^{y}}^{-1}(\frac{1}{c_{!}}(1\\!-\\!\frac{1}{b}))}{\partial\frac{1}{c_{1}}(1\\!-\\!\frac{1}{b})}\frac{\partial\frac{1}{c_{1}}(1\\!-\\!\frac{1}{b})}{\partial
b}\\!+\\!\frac{-1}{b^{2}}(1\\!-\\!c_{1}),$

where

$\displaystyle m_{H^{y}}(z)=\int\frac{dH^{y}(t)}{t-z},\ \
m^{\prime}_{H^{y}}(z)=\int\frac{1}{(t-z)^{2}}dH^{y}(t),\ \
(m_{H^{y}}^{-1})^{\prime}(z)=\frac{1}{m^{\prime}_{H^{y}}(z)}.$

Moreover, we have

$\displaystyle
x^{\prime}(b)=\frac{1-c_{1}}{b^{2}}+\frac{1}{b^{4}c_{1}}\left[\int\frac{dH^{y}(t)}{[t-\frac{1}{c_{1}}(1-\frac{1}{b})]^{2}}\right]^{-1}>0,$

which implies

$\displaystyle
x(b)=\frac{1}{b^{2}}m_{H^{y}}^{-1}\left(\frac{1}{c_{1}}\left(1-\frac{1}{b}\right)\right)+\frac{1}{b}(1-c_{1})=0\in
S_{F}^{c}.$

Considering $S_{F}^{c}$ is open set, then $0$ belongs to it, there exists
$b^{-}>0$ such that $[0,b^{-})\subset S_{F}^{c}$. Combining the spectral
property of $\boldsymbol{\Lambda}$ and the exact separation result for the
information plus noise matrix, we have for some existing $b^{-}>0$

$\displaystyle\mathbb{P}(\mbox{for all large n, }\lambda_{min}[({\bf C}{\bf
Y}+{\bf W}(s))({\bf C}{\bf Y}+{\bf W}(s))^{\prime}]\geq b^{-})$
$\displaystyle\leq$ $\displaystyle\mathbb{P}\left(\mbox{for all large n,
}\|({\bf X}(s){\bf X}^{\prime}(s))^{-1}\|\leq\frac{1}{kb^{-}}\right)=1.$

∎

######  Lemma 4.

Similar with Lemma 2, under the assumptions and notations of Theorem 2, we
have for $l=1,2$,

$\displaystyle\frac{1}{\sqrt{1-t}}\frac{\partial\Psi_{ji}^{l}}{\partial\hat{Y}_{ji}}=\frac{1}{\sqrt{t}}\frac{\partial\Psi_{ji}^{l}}{\partial
Y_{ji}}.$ (9)

###### Proof.

For $l=1$, we can easily get

$\displaystyle\frac{\partial\Psi^{1}_{ji}}{\partial\hat{Y}_{ji}}=\frac{\partial[\boldsymbol{\Lambda}^{\prime}{\bf
Q}(t)({\bf H}(t)-zI)^{-2}({\bf I}-{\bf
P}_{x(t)})]_{ji}}{\partial\hat{Y}_{ji}}$ $\displaystyle=$
$\displaystyle\left[\boldsymbol{\Lambda}^{\prime}\frac{\partial{\bf
Q}(t)}{\partial\hat{Y}_{ji}}G^{2}(t)({\bf I}-{\bf
P}_{x(t)})\right]_{ji}\\!-\\!\left[\boldsymbol{\Lambda}^{\prime}{\bf
Q}(t)G(t)\left(\frac{\partial{\bf
P}_{x(t)}}{\partial\hat{Y}_{ji}}\\!+\\!\frac{\partial{\bf
P}_{y(t)}}{\partial\hat{Y}_{ji}}\right)G^{2}(t)({\bf I}-{\bf
P}_{x(t)})\right]_{ji}$ $\displaystyle\\!-\\!$
$\displaystyle\left[\boldsymbol{\Lambda}^{\prime}{\bf
Q}(t)G^{2}(t)\left(\frac{\partial{\bf
P}_{x(t)}}{\partial\hat{Y}_{ji}}\\!+\\!\frac{\partial{\bf
P}_{y(t)}}{\partial\hat{Y}_{ji}}\right)G(t)({\bf I}-{\bf
P}_{x(t)})\right]_{ji}\\!-\\!\left[\boldsymbol{\Lambda}^{\prime}{\bf
Q}(t)G^{2}(t)\frac{\partial{\bf P}_{x(t)}}{\partial\hat{Y}_{ji}}\right]_{ji}.$

We need some matrix derivative results

$\displaystyle\frac{\partial{\bf X}(t)}{\partial\hat{Y}_{ji}}$
$\displaystyle=\sqrt{1-t}\boldsymbol{\Lambda}1_{ji},\ \frac{\partial{\bf
X}^{\prime}(t)}{\partial\hat{Y}_{ji}}=\sqrt{1-t}1_{ij}\boldsymbol{\Lambda}^{\prime},\
\frac{\partial{\bf
X}(t)}{\partial{Y}_{ji}}=\sqrt{t}\boldsymbol{\Lambda}1_{ji},\
\frac{\partial{\bf
X}^{\prime}(t)}{\partial{Y}_{ji}}=\sqrt{t}1_{ij}\boldsymbol{\Lambda}^{\prime},$
$\displaystyle\frac{\partial{\bf Q}(t)}{\partial\hat{Y}_{ji}}$
$\displaystyle=-({\bf X}(t){\bf X}^{\prime}(t))^{-1}(\frac{\partial{\bf
X}(t)}{\partial\hat{Y}_{ji}}{\bf X}^{\prime}(t)+{\bf X}(t)\frac{\partial{\bf
X}^{\prime}(t)}{\partial\hat{Y}_{ji}}){\bf Q}(t)+({\bf X}(t){\bf
X}^{\prime}(t))^{-1}\frac{\partial{\bf X}(t)}{\partial\hat{Y}_{ji}}$
$\displaystyle=\\!-\\!\sqrt{1\\!-\\!t}({\bf X}(t){\bf
X}^{\prime}(t))^{-1}(\boldsymbol{\Lambda}1_{ji}{\bf X}^{\prime}(t)\\!+\\!{\bf
X}(t)1_{ij}\boldsymbol{\Lambda}^{\prime}){\bf
Q}(t)\\!+\\!\sqrt{1\\!-\\!t}({\bf X}(t){\bf
X}^{\prime}(t))^{-1}\boldsymbol{\Lambda}1_{ji}$
$\displaystyle=\sqrt{1-t}[({\bf X}(t){\bf
X}^{\prime}(t))^{-1}\boldsymbol{\Lambda}1_{ji}({\bf I}-{\bf P}_{x(t)})-{\bf
Q}(t)1_{ij}\boldsymbol{\Lambda}^{\prime}{\bf Q}(t)],$
$\displaystyle\frac{\partial{\bf Q}(t)}{\partial{Y}_{ji}}$
$\displaystyle=\sqrt{t}[({\bf X}(t){\bf
X}^{\prime}(t))^{-1}\boldsymbol{\Lambda}1_{ji}({\bf I}-{\bf P}_{x(t)})-{\bf
Q}(t)1_{ij}\boldsymbol{\Lambda}^{\prime}{\bf Q}(t)],$

and

$\displaystyle\frac{\partial{\bf P}_{x(t)}}{\partial\hat{Y}_{ji}}=$
$\displaystyle\frac{\partial{\bf X}(t)}{\partial\hat{Y}_{ji}}{\bf Q}(t)+{\bf
X}(t)\frac{\partial{\bf Q}(t)}{\partial\hat{Y}_{ji}}$ $\displaystyle=$
$\displaystyle\sqrt{1-t}1_{ij}\boldsymbol{\Lambda}^{\prime}{\bf
Q}(t)+\sqrt{1-t}[{\bf Q}^{\prime}(t)\boldsymbol{\Lambda}1_{ji}({\bf I}-{\bf
P}_{x(t)})-{\bf P}_{x(t)}1_{ij}\boldsymbol{\Lambda}^{\prime}{\bf Q}(t)]$
$\displaystyle=$ $\displaystyle\sqrt{1-t}({\bf I}-{\bf
P}_{x(t)})1_{ij}\boldsymbol{\Lambda}^{\prime}{\bf Q}(t)+\sqrt{1-t}{\bf
Q}^{\prime}(t)\boldsymbol{\Lambda}1_{ji}({\bf I}-{\bf P}_{x(t)}),$
$\displaystyle\frac{\partial{\bf P}_{x(t)}}{\partial{Y}_{ji}}=$
$\displaystyle\sqrt{t}({\bf I}-{\bf
P}_{x(t)})1_{ij}\boldsymbol{\Lambda}^{\prime}{\bf Q}(t)+\sqrt{t}{\bf
Q}^{\prime}(t)\boldsymbol{\Lambda}1_{ji}({\bf I}-{\bf P}_{x(t)}).$

Moreover, for $l=2$, we need the following results:

$\displaystyle\frac{\partial{\bf U}(t)}{\partial{\hat{Y}}_{ji}}$
$\displaystyle=\sqrt{1-t}[({\bf Y}(t){\bf Y}^{\prime}(t))^{-1}1_{ji}({\bf
I}-{\bf P}_{y(t)})-{\bf U}(t)1_{ij}{\bf U}(t)],$
$\displaystyle\frac{\partial{\bf U}(t)}{\partial{Y}_{ji}}$
$\displaystyle=\sqrt{t}[({\bf Y}(t){\bf Y}^{\prime}(t))^{-1}1_{ji}({\bf
I}-{\bf P}_{y(t)})-{\bf U}(t)1_{ij}{\bf U}(t)],$
$\displaystyle\frac{\partial{\bf P}_{y(t)}}{\partial{\hat{Y}}_{ji}}$
$\displaystyle=\sqrt{1-t}({\bf I}-{\bf P}_{y(t)})1_{ij}{\bf
U}(t)+\sqrt{1-t}{\bf U}^{\prime}(t)1_{ji}({\bf I}-{\bf P}_{y(t)}),$
$\displaystyle\frac{\partial{\bf P}_{y(t)}}{\partial{Y}_{ji}}$
$\displaystyle=\sqrt{t}({\bf I}-{\bf P}_{y(t)})1_{ij}{\bf U}(t)+\sqrt{t}{\bf
U}^{\prime}(t)1_{ji}({\bf I}-{\bf P}_{y(t)}).$

Observed the difference between the derivative terms $\frac{\partial{\bf
Q}(t)}{\partial{Y}_{ji}}$, $\frac{\partial{\bf P}_{x(t)}}{\partial{Y}_{ji}}$,
$\frac{\partial{\bf U}}{\partial{Y}_{ji}}$, $\frac{\partial{\bf
P}_{y(t)}}{\partial{Y}_{ji}}$ and $\frac{\partial{\bf
Q}(t)}{\partial\hat{Y}_{ji}}$, $\frac{\partial{\bf
P}_{x(t)}}{\partial\hat{Y}_{ji}}$, $\frac{\partial{\bf
U}}{\partial\hat{Y}_{ji}}$, $\frac{\partial{\bf
P}_{y(t)}}{\partial\hat{Y}_{ji}}$ lies solely in their coefficients, which can
be canceled out by the coefficients in formula (9). ∎

######  Lemma 5.

Under the assumptions and notations of Theorem 2, we calculate ${\bf
P}_{x}-{\bf P}_{x}^{(k)}+{\bf P}_{y}-{\bf
P}_{y}^{(k)}=\boldsymbol{\alpha}_{k}\boldsymbol{\beta}_{k}$, and obtain

$\displaystyle\boldsymbol{\alpha}_{k}=$
$\displaystyle(\frac{e_{k}}{1+\gamma_{k}},\frac{\chi_{k}}{1+\gamma_{k}},\frac{e_{k}}{1+\gamma_{k}},-\frac{\chi_{k}}{1+\gamma_{k}},\frac{e_{k}}{1+\eta_{k}},\frac{\omega_{k}}{1+\eta_{k}},\frac{e_{k}}{1+\eta_{k}},-\frac{\omega_{k}}{1+\eta_{k}})$
$\displaystyle\boldsymbol{\beta}_{k}=$
$\displaystyle(\chi_{k},e_{k},\gamma_{k}e_{k},\chi_{k},\omega_{k},e_{k},\eta_{k}e_{k},\omega_{k})^{\prime}$

where

$\displaystyle\gamma_{k}=$
$\displaystyle(\boldsymbol{\Lambda}y_{k}+\boldsymbol{\Gamma}w_{k})^{\prime}[{\bf
X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}+\boldsymbol{\Gamma}w_{k}),\quad\eta_{k}=y_{k}^{\prime}[{\bf
Y}^{(k)}({\bf Y}^{(k)})^{\prime}]^{-1}y_{k},$ $\displaystyle\chi_{k}=$
$\displaystyle({\bf X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}+\boldsymbol{\Gamma}w_{k}),\quad\quad\omega_{k}=({\bf
Y}^{(k)})^{\prime}[{\bf Y}^{(k)}({\bf Y}^{(k)})^{\prime}]^{-1}y_{k}.$

###### Proof.

By the formula (0.7.4.1) in [8], we have

$\displaystyle{\bf P}_{x}-{\bf P}_{x}^{(k)}={\bf X}^{\prime}({\bf X}{\bf
X}^{\prime})^{-1}{\bf X}-({\bf X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}{\bf X}^{(k)}$ $\displaystyle=$
$\displaystyle(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}({\bf
X}{\bf X}^{\prime})^{-1}{\bf X}+({\bf X}^{(k)})^{\prime}[({\bf X}{\bf
X}^{\prime})^{-1}-({\bf X}^{(k)}({\bf X}^{(k)})^{\prime})^{-1}]{\bf X}$
$\displaystyle+({\bf X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle=$
$\displaystyle(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}({\bf
X}{\bf X}^{\prime})^{-1}{\bf
X}^{(k)}+(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}({\bf
X}{\bf
X}^{\prime})^{-1}(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle+({\bf X}^{(k)})^{\prime}[({\bf X}{\bf
X}^{\prime})^{-1}\\!-\\!({\bf X}^{(k)}({\bf X}^{(k)})^{\prime})^{-1}]{\bf
X}^{(k)}+({\bf X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle+({\bf X}^{(k)})^{\prime}[({\bf X}{\bf
X}^{\prime})^{-1}\\!-\\!({\bf X}^{(k)}({\bf
X}^{(k)})^{\prime})^{-1}](\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}\\!+\\!\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle=$
$\displaystyle(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}\\!+\\!\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}[{\bf
X}^{(k)}({\bf X}^{(k)})^{\prime}]^{-1}{\bf
X}^{(k)}\\!+\\!(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}\\!+\\!\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}\left(({\bf
X}{\bf X}^{\prime})^{-1}\\!-\\![{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}\right){\bf X}^{(k)}$
$\displaystyle+(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}[{\bf
X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle+(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}\left([{\bf
X}{\bf X}^{\prime}]^{-1}-[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}\right)(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle+\\!({\bf X}^{(k)})^{\prime}\left(({\bf X}{\bf
X}^{\prime})^{-1}\\!-\\![{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}\right){\bf X}^{(k)}\\!+\\!({\bf
X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}\\!+\\!\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle+({\bf X}^{(k)})^{\prime}\left(({\bf X}{\bf
X}^{\prime})^{-1}-[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}\right)(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime}).$

Here we need the following results:

$\displaystyle{\bf X}{\bf X}^{\prime}-{\bf X}^{(k)}({\bf X}^{(k)})^{\prime}=$
$\displaystyle
x_{k}x_{k}^{\prime}=(\boldsymbol{\Lambda}y_{k}+\boldsymbol{\Gamma}w_{k})(\boldsymbol{\Lambda}y_{k}+\boldsymbol{\Gamma}w_{k})^{\prime},$
$\displaystyle({\bf X}{\bf X}^{\prime})^{-1}\\!-\\![{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}=$ $\displaystyle\\!-\\!\frac{[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}\\!+\\!\boldsymbol{\Gamma}w_{k})(\boldsymbol{\Lambda}y_{k}\\!+\\!\boldsymbol{\Gamma}w_{k})^{\prime}[{\bf
X}^{(k)}({\bf X}^{(k)})^{\prime}]^{-1}}{1\\!+\\!\gamma_{k}}.$

The we have

$\displaystyle{\bf P}_{x}-{\bf P}_{x}^{(k)}=$
$\displaystyle(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}[{\bf
X}^{(k)}({\bf X}^{(k)})^{\prime}]^{-1}{\bf
X}^{(k)}+\gamma_{k}e_{k}e_{k}^{\prime}$
$\displaystyle-\frac{\gamma_{k}}{1+\gamma_{k}}(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})^{\prime}[{\bf
X}^{(k)}({\bf X}^{(k)})^{\prime}]^{-1}{\bf X}^{(k)}$
$\displaystyle-\frac{\gamma_{k}^{2}}{1+\gamma_{k}}e_{k}e_{k}^{\prime}+({\bf
X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}e_{k}^{\prime}+\boldsymbol{\Gamma}w_{k}e_{k}^{\prime})$
$\displaystyle-\\!\frac{1}{1\\!+\\!\gamma_{k}}({\bf X}^{(k)})^{\prime}({\bf
X}^{(k)}({\bf
X}^{(k)})^{\prime})^{-1}(\boldsymbol{\Lambda}y_{k}\\!+\\!\boldsymbol{\Gamma}w_{k})(\boldsymbol{\Lambda}y_{k}\\!+\\!\boldsymbol{\Gamma}w_{k})^{\prime}({\bf
X}^{(k)}({\bf X}^{(k)})^{\prime})^{-1}{\bf X}^{(k)}$
$\displaystyle-\frac{\gamma_{k}}{1+\gamma_{k}}({\bf X}^{(k)})^{\prime}({\bf
X}^{(k)}({\bf
X}^{(k)})^{\prime})^{-1}(\boldsymbol{\Lambda}y_{k}+\boldsymbol{\Gamma}w_{k})e_{k}^{\prime}$
$\displaystyle=$
$\displaystyle\frac{1}{1+\gamma_{k}}(e_{k}\chi_{k}^{\prime}+\chi_{k}e_{k}^{\prime})+\frac{\gamma_{k}}{1+\gamma_{k}}e_{k}e_{k}^{\prime}-\frac{1}{1+\gamma_{k}}\chi_{k}\chi_{k}^{\prime},$

abbreviated $({\bf X}^{(k)})^{\prime}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}(\boldsymbol{\Lambda}y_{k}+\boldsymbol{\Gamma}w_{k})$
as $\chi_{k}$. Similarly,

$\displaystyle{\bf P}_{y}-{\bf P}_{y}^{(k)}=$ $\displaystyle
e_{k}y^{\prime}_{k}[{\bf Y}^{(k)}({\bf Y}^{(k)})^{\prime}]^{-1}{\bf Y}^{(k)}$
$\displaystyle-e_{k}y^{\prime}_{k}\frac{({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}}{1+y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}}{\bf Y}^{(k)}$
$\displaystyle+e_{k}y^{\prime}_{k}[{\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime}]^{-1}y_{k}e_{k}^{\prime}$ $\displaystyle-
e_{k}y^{\prime}_{k}\frac{({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}}{1+y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}}y_{k}e_{k}^{\prime}$ $\displaystyle-({\bf
Y}^{(k)})^{\prime}\frac{({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}}{1+y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}}{\bf Y}^{(k)}$ $\displaystyle-({\bf
Y}^{(k)})^{\prime}\frac{({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}}{1+y^{\prime}_{k}({\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime})^{-1}y_{k}}y_{k}e_{k}^{\prime}$ $\displaystyle+({\bf
Y}^{(k)})^{\prime}[{\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime}]^{-1}y_{k}e_{k}^{\prime}$ $\displaystyle=$
$\displaystyle\frac{1}{1+\eta_{k}}(e_{k}\omega_{k}^{\prime}+\omega_{k}e_{k}^{\prime})+\frac{\eta_{k}}{1+\eta_{k}}e_{k}e_{k}^{\prime}-\frac{1}{1+\eta_{k}}\omega_{k}\omega_{k}^{\prime},$

abbreviated $({\bf Y}^{(k)})^{\prime}[{\bf Y}^{(k)}({\bf
Y}^{(k)})^{\prime}]^{-1}y_{k}$ as $\omega_{k}$. ∎

######  Lemma 6.

Under the assumptions and notations of Theorem 2, for the matrix ${\bf M}$
with fixed dimension, we have

$\displaystyle{\bf I}_{8}+\boldsymbol{\beta}_{k}({\bf
H}^{(k)}-z)^{-1}\boldsymbol{\alpha}_{k}={\bf M}+\tilde{{\bf M}}$
$\displaystyle({\bf I}_{8}+\boldsymbol{\beta}_{k}({\bf
H}^{(k)}-z)^{-1}\boldsymbol{\alpha}_{k})^{-1}={\bf M}^{-1}+\widetilde{{\bf
M}^{-1}},$

where for $l=1,2,4,$ $\mathbb{E}|\tilde{{\bf M}}_{ij}|^{2l}=O(\frac{1}{n})$
and for $r=1,2,$ $\mathbb{E}|(\widetilde{{\bf
M}^{-1}})_{ij}|^{2r}=O(\frac{1}{n})$.

###### Proof.

In order to calculate higher order moments, we truncate the entries of
$\mathbf{Y}$ and $\mathbf{W}$ following the approach described in [3], that
is, $y_{ij}^{t}=y_{ij}I_{\\{|y_{ij}|<\varepsilon_{n}\sqrt{n}\\}}$. And

$\displaystyle\|F^{{\bf H}}-F^{{\bf H}^{t}}\|\leq$
$\displaystyle\frac{1}{n}rank({\bf P}_{x}-{\bf P}_{x^{t}}+{\bf P}_{y}-{\bf
P}_{y^{t}})$ $\displaystyle\leq$ $\displaystyle\frac{c}{n}rank({\bf Y}-{\bf
Y}^{t})+\frac{c}{n}rank({\bf W}-{\bf W}^{t})$ $\displaystyle\leq$
$\displaystyle\frac{c}{n}\sum_{i=1}^{q}\sum_{j=1}^{n}I_{\\{|y_{ij}|\geq\varepsilon_{n}\sqrt{n}\\}}+\frac{c}{n}\sum_{i=1}^{p}\sum_{j=1}^{n}I_{\\{|w_{ij}|\geq\varepsilon_{n}\sqrt{n}\\}},$

then we have

$\displaystyle\mathbb{P}(\|F^{{\bf H}}-F^{{\bf
H}^{t}}\|\\!\geq\\!\delta)\\!\leq\\!\mathbb{P}(\frac{c}{n}\sum_{i=1}^{q}\sum_{j=1}^{n}I_{\\{|y_{ij}|\geq\varepsilon_{n}\sqrt{n}\\}}\\!\geq\\!\frac{\delta}{2})\\!+\\!\mathbb{P}(\frac{c}{n}\sum_{i=1}^{p}\sum_{j=1}^{n}I_{\\{|w_{ij}|\geq\varepsilon_{n}\sqrt{n}\\}}\\!\geq\\!\frac{\delta}{2}).$

According to concentration inequality and step 2 in [16], we have $\|F^{{\bf
H}}-F^{{\bf H}^{t}}\|\overset{a.s.}{\to}0$. Define
$y_{ij}^{c}=y_{ij}^{t}-\mathbb{E}y_{ij}^{t}$, we consider

$\displaystyle\|F^{{\bf H}^{t}}-F^{{\bf H}^{c}}\|\leq$
$\displaystyle\frac{1}{n}rank({\bf P}_{x^{t}}-{\bf P}_{x^{c}}+{\bf
P}_{y^{t}}-{\bf P}_{y^{c}})$ $\displaystyle\leq$
$\displaystyle\frac{c}{n}rank({\bf Y}^{t}-{\bf Y}^{c})+\frac{c}{n}rank({\bf
W}^{t}-{\bf W}^{c})\leq\frac{c}{n}$

where for any $i,j$, $\\{\mathbb{E}y_{ij}^{t}\\}$ and
$\\{\mathbb{E}w_{ij}^{t}\\}$ are the same, respectively. That is
$\mathbb{E}{\bf Y}^{t}$ and $\mathbb{E}{\bf W}^{t}$ are both rank one. Next
write $\sigma_{y}=\mathbb{E}|y_{ij}^{c}|^{2}$,
$y_{ij}^{s}=y_{ij}^{c}/\sigma_{y}$, we consider

$\displaystyle L^{3}(F^{{\bf H}^{s}},F^{{\bf H}^{c}})\leq{\rm tr}({\bf
P}_{x^{s}}-{\bf P}_{x^{c}}+{\bf P}_{y^{s}}-{\bf P}_{y^{c}})({\bf
P}_{x^{s}}-{\bf P}_{x^{c}}+{\bf P}_{y^{s}}-{\bf P}_{y^{c}})$

noticing ${\bf P}_{y^{c}}$ is scale invariant, that is ${\bf P}_{y^{s}}={\bf
P}_{y^{c}}$. Then we have

$\displaystyle L^{3}(F^{{\bf H}^{s}},F^{{\bf H}^{c}})\leq{\rm tr}({\bf
P}_{x^{s}}-{\bf P}_{x^{c}})({\bf P}_{x^{s}}-{\bf P}_{x^{c}})\leq
c\left(\frac{\sigma_{w}}{\sigma_{y}}-1\right)$

where

$\displaystyle{\bf P}_{x^{s}}\\!=\\!\left(\boldsymbol{\Lambda}\frac{{\bf
Y}^{c}}{\sigma_{y}}\\!+\\!\boldsymbol{\Gamma}\frac{{\bf
W}^{c}}{\sigma_{w}}\right)^{\prime}\left[\left(\boldsymbol{\Lambda}\frac{{\bf
Y}^{c}}{\sigma_{y}}\\!+\\!\boldsymbol{\Gamma}\frac{{\bf
W}^{c}}{\sigma_{w}}\right)\left(\boldsymbol{\Lambda}\frac{{\bf
Y}^{c}}{\sigma_{y}}\\!+\\!\boldsymbol{\Gamma}\frac{{\bf
W}^{c}}{\sigma_{w}}\right)^{\prime}\right]^{-1}\left(\boldsymbol{\Lambda}\frac{{\bf
Y}^{c}}{\sigma_{y}}\\!+\\!\boldsymbol{\Gamma}\frac{{\bf
W}^{c}}{\sigma_{w}}\right)$ $\displaystyle\\!=\\!$
$\displaystyle\left(\frac{\sigma_{w}}{\sigma_{y}}\boldsymbol{\Lambda}{\bf
Y}^{c}\\!+\\!\boldsymbol{\Gamma}{\bf
W}^{c}\right)^{\prime}\left[\left(\frac{\sigma_{w}}{\sigma_{y}}\boldsymbol{\Lambda}{\bf
Y}^{c}\\!+\\!\boldsymbol{\Gamma}{\bf
W}^{c}\right)\left(\frac{\sigma_{w}}{\sigma_{y}}\boldsymbol{\Lambda}{\bf
Y}^{c}\\!+\\!\boldsymbol{\Gamma}{\bf
W}^{c}\right)^{\prime}\right]^{-1}\left(\frac{\sigma_{w}}{\sigma_{y}}\boldsymbol{\Lambda}{\bf
Y}^{c}\\!+\\!\boldsymbol{\Gamma}{\bf W}^{c}\right)$

which equals ${\bf X}^{s}-{\bf
X}^{c}=(\frac{\sigma_{w}}{\sigma_{y}}-1)\boldsymbol{\Lambda}{\bf Y}^{c}$.
Because the truncation is the same with that of [3], obviously, we have
$\sigma_{y}\to 1$ and $\sigma_{w}\to 1$, which implies
$\sigma_{w}/\sigma_{y}-1\to 0$. In summary, truncation, centering and
normalization do not change the limit of target LSD, so we assume the entries
of ${\bf W}$ and ${\bf Y}$ are bounded by $\varepsilon_{n}\sqrt{n}$ from now
on.

We first calculate the expectation for term $\boldsymbol{\beta}_{k}({\bf
H}^{(k)}-z)^{-1}\boldsymbol{\alpha}_{k}$ to approximate it. Here we mainly
apply the following equalities:

$\displaystyle\mathbb{E}\chi_{k}^{\prime}({\bf H}^{(k)}-z)^{-1}\chi_{k}={\rm
tr}({\bf H}^{(k)}-z)^{-1}({\bf Q}^{(k)})^{\prime}{\bf
Q}^{(k)}:=\gamma_{k}^{\Lambda,H},$
$\displaystyle\mathbb{E}\chi_{k}^{\prime}({\bf H}^{(k)}-z)^{-1}\omega_{k}={\rm
tr}\boldsymbol{\Lambda}^{\prime}{\bf Q}^{(k)}({\bf H}^{(k)}-z)^{-1}({\bf
U}^{(k)})^{\prime}:=\theta_{k}^{\Lambda},$
$\displaystyle\mathbb{E}\omega_{k}^{\prime}({\bf
H}^{(k)}-z)^{-1}\omega_{k}={\rm tr}{\bf U}^{(k)}({\bf H}^{(k)}-z)^{-1}({\bf
U}^{(k)})^{\prime}:=\eta_{k}^{H},$ $\displaystyle\mathbb{E}e_{k}^{\prime}({\bf
H}^{(k)}-z)^{-1}e_{k}=({\bf H}^{(k)}-z)^{-1}_{kk}:=\theta_{k},$
$\displaystyle\mathbb{E}\gamma_{k}={\rm tr}[{\bf X}^{(k)}({\bf
X}^{(k)})^{\prime}]^{-1}:=\gamma_{k}^{0},\quad\mathbb{E}\eta_{k}={\rm tr}[{\bf
Y}^{(k)}({\bf Y}^{(k)})^{\prime}]^{-1}:=\eta_{k}^{0},$

then we have

$\displaystyle{\bf
M}\\!=\\!\begin{pmatrix}1,&\frac{\gamma_{k}^{\Lambda,H}}{1+\gamma_{k}^{0}},&0,&-\frac{\gamma_{k}^{\Lambda,H}}{1+\gamma_{k}^{0}},&0,&\frac{\theta_{k}^{\Lambda}}{1+\eta_{k}^{0}},&0,&-\frac{\theta_{k}^{\Lambda}}{1+\eta_{k}^{0}}\\\
\frac{\theta_{k}}{1+\gamma_{k}^{0}},&1,&\frac{\theta_{k}}{1+\gamma_{k}^{0}},&0,&\frac{\theta_{k}}{1+\eta_{k}^{0}},&0,&\frac{\theta_{k}}{1+\eta_{k}^{0}},&0\\\
\frac{\gamma_{k}^{0}\theta_{k}}{1+\gamma_{k}^{0}},&0,&1+\frac{\gamma_{k}^{0}\theta_{k}}{1+\gamma_{k}^{0}},&0,&\frac{\gamma_{k}^{0}\theta_{k}}{1+\eta_{k}^{0}},&0,&\frac{\gamma_{k}^{0}\theta_{k}}{1+\eta_{k}^{0}},&0\\\
0,&\frac{\gamma_{k}^{\Lambda,H}}{1+\gamma_{k}^{0}},&0,&1-\frac{\gamma_{k}^{\Lambda,H}}{1+\gamma_{k}^{0}},&0,&\frac{\theta_{k}^{\Lambda}}{1+\eta_{k}^{0}},&0,&-\frac{\theta_{k}^{\Lambda}}{1+\eta_{k}^{0}}\\\
0,&\frac{\theta_{k}^{\Lambda}}{1+\gamma_{k}^{0}},&0,&-\frac{\theta_{k}^{\Lambda}}{1+\gamma_{k}^{0}},&1,&\frac{\eta_{k}^{H}}{1+\eta_{k}^{0}},&0,&-\frac{\eta_{k}^{H}}{1+\eta_{k}^{0}}\\\
\frac{\theta_{k}}{1+\gamma_{k}^{0}},&0,&\frac{\theta_{k}}{1+\gamma_{k}^{0}},&0,&\frac{\theta_{k}}{1+\eta_{k}^{0}},&1,&\frac{\theta_{k}}{1+\eta_{k}^{0}},&0\\\
\frac{\eta_{k}^{0}\theta_{k}}{1+\gamma_{k}^{0}},&0,&\frac{\eta_{k}^{0}\theta_{k}}{1+\gamma_{k}^{0}},&0,&\frac{\eta_{k}^{0}\theta_{k}}{1+\eta_{k}^{0}},&0,&1+\frac{\eta_{k}^{0}\theta_{k}}{1+\eta_{k}^{0}},&0\\\
0,&\frac{\theta_{k}^{\Lambda}}{1+\gamma_{k}^{0}},&0,&-\frac{\theta_{k}^{\Lambda}}{1+\gamma_{k}^{0}},&0,&\frac{\eta_{k}^{H}}{1+\eta_{k}^{0}},&0,&1-\frac{\eta_{k}^{H}}{1+\eta_{k}^{0}}\\\
\end{pmatrix}.$

Obviously, we have for $l=1,2,4,$

$\displaystyle\mathbb{E}|1\\!+\\![\boldsymbol{\beta}_{k}({\bf
H}^{(k)}\\!-\\!z)^{-1}\boldsymbol{\alpha}_{k}]_{ij}\\!-\\!{\bf
M}_{ij}|^{2l}=O\left(\frac{1}{n}\right).$

Combining the formula
$A^{-1}\\!-\\!B^{-1}=\\!-\\!B^{-1}(A\\!-\\!B)A^{-1}=\\!-\\!B^{-1}(A\\!-\\!B)B^{-1}\\!+\\!B^{-1}(A\\!-\\!B)B^{-1}(A\\!-\\!B)A^{-1}$,
we have

$\displaystyle\mathbb{E}|\widetilde{{\bf
M}^{-1}}_{ij}|^{2}=C\mathbb{E}|\tilde{{\bf
M}}_{ij}|^{2}+C\sqrt{\mathbb{E}|\tilde{{\bf
M}}_{ij}|^{4}}\sqrt{\mathbb{E}|\tilde{{\bf
M}}_{i^{\prime}j^{\prime}}|^{4}}=O\left(\frac{1}{n}\right),$
$\displaystyle\mathbb{E}|\widetilde{{\bf
M}^{-1}}_{ij}|^{4}=C\mathbb{E}|\tilde{{\bf
M}}_{ij}|^{4}+C\sqrt{\mathbb{E}|\tilde{{\bf
M}}_{ij}|^{8}}\sqrt{\mathbb{E}|\tilde{{\bf
M}}_{i^{\prime}j^{\prime}}|^{8}}=O\left(\frac{1}{n}\right),$

where we need the entrices in $[{\bf I}_{8}+\boldsymbol{\beta}_{k}({\bf
H}-z)^{-1}\boldsymbol{\alpha}_{k}]^{-1}$ and ${\bf M}^{-1}$ are bounded with
probability 1.

Consider the matrix ${\bf M}-{\bf I}$ has three linearly independent rows, its
rank is $3$. We calculate the imaginary part of non-zero eigenvalues $\lambda$
via the fact that two determinants of $4\times 4$ minor of $\Im{\bf
M}-\Im\lambda{\bf I}$ are zero, which are

$\displaystyle\det\begin{pmatrix}-\Im\lambda,&\frac{\Im\theta_{k}}{1+\gamma_{k}^{0}},&\frac{\Im\theta_{k}}{1+\eta_{k}^{0}},&0\\\
\gamma_{k}^{0}\Im\lambda,&-\Im\lambda,&0,&0\\\
\eta_{k}^{0}\Im\lambda,&0,&-\Im\lambda,&0\\\
\Im\lambda,&0,&0,&-\Im\lambda\end{pmatrix}=0=\det\begin{pmatrix}0,&0,&-\frac{\Im\theta_{k}^{\Lambda}}{1+\gamma_{k}^{0}},&-\Im\lambda-\frac{\Im\eta_{k}^{H}}{1+\eta_{k}^{0}}\\\
0,&-\Im\lambda,&-\frac{\Im\gamma_{k}^{\Lambda,H}}{1+\gamma_{k}^{0}},&-\frac{\Im\theta_{k}^{\Lambda}}{1+\eta_{k}^{0}}\\\
0,&\Im\lambda,&-\Im\lambda,&0\\\ -\Im\lambda,&0,&0,&\Im\lambda\end{pmatrix}.$

The we have $\Im\lambda$ satisfies the following equations

$\displaystyle(\Im\lambda)^{3}\left[\Im\lambda-\left(\frac{\gamma_{k}^{0}}{1+\gamma_{k}^{0}}-\frac{\eta_{k}^{0}}{1+\eta_{k}^{0}}\right)\Im\theta_{k}\right]=0$
$\displaystyle(\Im\lambda)^{2}\left[(\Im\lambda)^{2}+\left(\frac{\Im\gamma_{k}^{\Lambda,H}}{1+\gamma_{k}^{0}}+\frac{\Im\eta_{k}^{H}}{1+\eta_{k}^{0}}\right)\Im\lambda-\frac{(\Im\theta_{k}^{\Lambda})^{2}-\Im\gamma_{k}^{\Lambda,H}\Im\eta_{k}^{H}}{(1+\gamma_{k}^{0})(1+\eta_{k}^{0})}\right]=0,$

where $\gamma_{k}^{0}$ and $\eta_{k}^{0}$ are positive. Because when $\Im
z>0$, $\Im\theta_{k}>0$, we have $\Im\lambda_{1}\neq 0$. Notice that the
formula $|{\rm tr}(AB^{\prime})|^{2}\leq{\rm tr}(AA^{\prime}){\rm
tr}(BB^{\prime})$, and when $A=cB$, the equality holds. Combining the
following facts

$\displaystyle\Im\theta_{k}^{\Lambda}=$ $\displaystyle(\Im z){\rm
tr}\boldsymbol{\Lambda}^{\prime}{\bf Q}^{(k)}({\bf H}^{(k)}-z)^{-1}({\bf
H}^{(k)}-\bar{z})^{-1}({\bf U}^{(k)})^{\prime}$
$\displaystyle\Im\gamma_{k}^{\Lambda,H}=$ $\displaystyle(\Im z){\rm tr}{\bf
Q}^{(k)}({\bf H}^{(k)}-z)^{-1}({\bf H}^{(k)}-\bar{z})^{-1}({\bf
Q}^{(k)})^{\prime}$ $\displaystyle\Im\eta_{k}^{H}=$ $\displaystyle(\Im z){\rm
tr}{\bf U}^{(k)}({\bf H}^{(k)}-z)^{-1}({\bf H}^{(k)}-\bar{z})^{-1}({\bf
U}^{(k)})^{\prime}$

we have
$(\Im\theta_{k}^{\Lambda})^{2}-\Im\gamma_{k}^{\Lambda,H}\Im\eta_{k}^{H}<0$
strictly holds, then $\Im\lambda_{2}\neq 0$ and $\Im\lambda_{3}\neq 0$. This
implies the non-zero eigenvalues of ${\bf M}-{\bf I}$ are complex, away from
$-1$. ∎

## Acknowledgments

The author is very grateful to Zhidong Bai for many stimulate conversations
concerning the proof of Lemmas. The author also thanks Jiang Hu for inspiring
to explore the relationship (Theorem 1).

## References

  * [1] [author] Anderson, Theodore W.T. W. (2003). An Introduction to Multivariate Statistical Analysis, Third ed. Wiley.
  * [2] [author] Bai, ZhidongZ., Hou, ZhiqiangZ., Hu, JiangJ., Jiang, DandanD. and Zhang, XiaozhuoX. (2021). Limiting canonical distribution of two large dimensional random vectors. In Methodology and Applications of Statistics. 213–238. Springer.
  * [3] [author] Bai, ZingdongZ. and Silverstein, J. W.J. W. (2004). CLT for linear spectral statistics of large-dimensional sample covariance matrices. The Annals of Probability 32 553–605.
  * [4] [author] Bai, ZhidongZ. and Silverstein, Jack W.J. W. (2010). Spectral Analysis of Large Dimensional Random Matrices, second ed. Springer Series in Statistics. Springer, New York. 10.1007/978-1-4419-0661-8 2567175
  * [5] [author] Bao, ZhigangZ., Hu, JiangJ., Pan, GuangmingG. and Zhou, WangW. (2017). Test of independence for high-dimensional random vectors based on freeness in block correlation matrices. Electronic Journal of Statistics 11 1527–1548.
  * [6] [author] Bao, ZhigangZ., Hu, JiangJ., Pan, GuangmingG. and Zhou, WangW. (2019). Canonical correlation coefficients of high-dimensional Gaussian vectors: finite rank case. The Annals of Statistics 47 612–640. 10.1214/18-AOS1704 3909944
  * [7] [author] Bao, ZhigangZ., Hu, JiangJ., Xu, XiaocongX. and Zhang, XiaozhuoX. (2022). Spectral statistics of sample block correlation matrices. arXiv preprint arXiv:2207.06107.
  * [8] [author] Horn, Roger A.R. A. and Johnson, Charles R.C. R. (2012). Matrix Analysis. Cambridge University Press.
  * [9] [author] Hotelling, HaroldH. (1935). The most predictable criterion. Journal of Educational Psychology 26 139–142.
  * [10] [author] Hotelling, HaroldH. (1936). Relations between two sets of variables. Biometrika 28 321–377.
  * [11] [author] Hou, ZhiqiangZ., Zhang, XiaozhuoX., Bai, ZhidongZ. and Hu, JiangJ. (2023). Spiked eigenvalues of noncentral Fisher matrix with applications. Bernoulli 29 3171–3197.
  * [12] [author] Knowles, AnttiA. and Yin, JunJ. (2017). Anisotropic local laws for random matrices. Probability Theory and Related Fields 169 257–352.
  * [13] [author] Ma, ZongmingZ. and Yang, FanF. (2023). Sample canonical correlation coefficients of high-dimensional random vectors with finite rank correlations. Bernoulli 29 1905–1932.
  * [14] [author] Muirhead, Robb J.R. J. (1982). Aspects of Multivariate Statistical Theory. Wiley.
  * [15] [author] Yang, FanF. (2022). Limiting distribution of the sample canonical correlation coefficients of high-dimensional random vectors. Electronic Journal of Probability 27 1–71.
  * [16] [author] Yang, Y.Y. and Pan, G.G. (2012). The convergence of the empirical distribution of canonical correlation coefficients. Electronic Journal of Probability 17 1–13.
  * [17] [author] Yang, Y.Y. and Pan, G.G. (2015). Independence test for high dimensional data based on regularized canonical correlation coefficients. The Annals of Statistics 43 467–500.
  * [18] [author] Zhang, XiaozhuoX., Bai, ZhidongZ. and Jiang, HuH. (2023). Limiting spectral distribution of high-dimensional noncentral Fisher matrices and its analysis. SCIENCE CHINA Mathematics 66 393–408. https://doi.org/10.1007/s11425-020-1958-1