U_{\perp}^{\top}TV\right\rVert^{2}_{{\rm F}}-\left\lVert
U_{\perp}^{\top}T\widehat{V}\right\rVert^{2}_{{\rm
F}}\right\rvert+\left\lvert\left\lVert\widehat{U}_{\perp}^{\top}TV\right\rVert^{2}_{{\rm
F}}-\left\lVert\widehat{U}_{\perp}^{\top}T\widehat{V}\right\rVert^{2}_{{\rm
F}}\right\rvert.$

It follows that

$\displaystyle\left\lvert\left\lVert U_{\perp}^{\top}TV\right\rVert^{2}_{{\rm
F}}-\left\lVert U_{\perp}^{\top}T\widehat{V}\right\rVert^{2}_{{\rm
F}}\right\rvert\leq\left\lVert
U^{\top}_{\perp}T(VV^{\top}-\widehat{V}\widehat{V}^{\top})\right\rVert_{{\rm
F}}^{2}+2\left\lvert\langle
U^{\top}_{\perp}T(VV^{\top}-\widehat{V}\widehat{V}^{\top}),U^{\top}_{\perp}TV\rangle\right\rvert$
$\displaystyle\leq
C_{1}\mu^{2}\tau\frac{\|T\|_{\ell_{1}}^{2}}{d_{2}}\cdot\left(\frac{\sigma_{\xi}^{2}}{\lambda_{\min}^{2}}\right)\frac{rd_{1}^{2}d_{2}\log
d_{1}}{n}+C_{2}\|U^{\top}_{\perp}T{V}\|_{\mathrm{F}}\|T\|_{\ell_{1}}\mu\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{r\tau
d_{1}^{2}\log d_{1}}{n}},$

and

$\displaystyle\left\lvert\left\lVert
U_{\perp}^{\top}T\widehat{V}\right\rVert^{2}_{{\rm
F}}-\left\lVert\widehat{U}_{\perp}^{\top}T\widehat{V}\right\rVert^{2}_{{\rm
F}}\right\rvert\leq\left\lVert(\widehat{U}\widehat{U}^{\top}-UU^{\top})T\widehat{V}\widehat{V}^{\top}\right\rVert_{{\rm
F}}^{2}+2\left\lvert\langle
U^{\top}_{\perp}T(VV^{\top}-\widehat{V}\widehat{V}^{\top}),U^{\top}_{\perp}T\widehat{V}\rangle\right\rvert$
$\displaystyle\leq
C_{1}\mu^{2}\tau\frac{\|T\|_{\ell_{1}}^{2}}{d_{1}}\cdot\left(\frac{\sigma_{\xi}^{2}}{\lambda_{\min}^{2}}\right)\frac{rd_{1}^{2}d_{2}\log
d_{1}}{n}+C_{2}(\|U^{\top}_{\perp}TV\|_{\mathrm{F}}$ $\displaystyle\
+{\|U^{\top}_{\perp}T(\widehat{V}\widehat{V}^{\top}-VV^{\top})\|_{\mathrm{F}}})\|T\|_{\ell_{1}}\mu\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{r\tau
d_{1}^{2}\log d_{1}}{n}}$ $\displaystyle\leq
C_{1}\mu^{2}\tau\frac{\|T\|_{\ell_{1}}^{2}}{d_{1}}\cdot\left(\frac{\sigma_{\xi}^{2}}{\lambda_{\min}^{2}}\right)\frac{rd_{1}^{2}d_{2}\log
d_{1}}{n}+C_{2}\|U^{\top}_{\perp}T{V}\|_{\mathrm{F}}\|T\|_{\ell_{1}}\mu\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{r\tau
d_{1}^{2}\log d_{1}}{n}}$
$\displaystyle+C_{3}\mu^{2}\tau\frac{\|T\|_{\ell_{1}}^{2}}{d_{2}}\cdot\left(\frac{\sigma_{\xi}^{2}}{\lambda_{\min}^{2}}\right)\frac{rd_{1}^{2}d_{2}\log
d_{1}}{n}.$

We then can show that

$\displaystyle\left|1-\frac{\widehat{s}_{T}}{\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}}\right|\leq\left\lvert
1-\frac{\widehat{s}_{T}^{2}}{\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}^{2}}\right\rvert\leq
C_{2}\mu\frac{\|T\|_{\ell_{1}}}{\left\lVert\mathcal{P}_{M}(T)\right\rVert_{{\rm
F}}}\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{\tau
r\alpha_{d}d_{1}d_{2}\log d_{1}}{n}},$ (33)

which indicates the fact that, with probability at least $1-C\log
d_{1}d_{1}^{-\tau}$,

$\displaystyle\left\lvert\frac{\widehat{\sigma}_{\xi}\widehat{s}_{T}}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}}-1\right\rvert$
$\displaystyle\leq\left\lvert\left(\frac{\widehat{\sigma}_{\xi}}{\sigma_{\xi}}-1\right)\left(\frac{\widehat{s}_{T}}{\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}}-1\right)\right\rvert+\left\lvert\frac{\widehat{\sigma}_{\xi}}{\sigma_{\xi}}-1\right\rvert+\left\lvert\frac{\widehat{s}_{T}}{\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}}-1\right\rvert$ $\displaystyle\leq\frac{C_{1}\tau\log
d_{1}}{\sqrt{n}}+C_{2}\gamma_{n}^{2}+C_{3}\mu\frac{\|T\|_{\ell_{1}}}{\left\lVert\mathcal{P}_{M}(T)\right\rVert_{{\rm
F}}}\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{\tau
r\alpha_{d}d_{1}d_{2}\log d_{1}}{n}}.$

Applying again the Lipschitz property of $\Phi(t)$ to the decomposition (32),
we can obtain the second statement.

∎

###  B.2 Proof of Theorem 3

###### Proof.

Since in Lemma 1, 2, and 3, the inequalities uniformly hold for all $T$, we
can write the couple $(W_{T_{1}},W_{T_{2}})$ as

$(W_{T_{1}},W_{T_{2}})=\left(\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{1})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{1})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}+\Delta_{T_{1}},\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{2})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{2})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}+\Delta_{T_{2}}\right),$

where with probability at least $1-C\log d_{1}d_{1}^{-\tau}$,

$\displaystyle\max\left\\{\left\lvert\Delta_{T_{1}}\right\rvert,\left\lvert\Delta_{T_{2}}\right\rvert\right\\}\leq$
$\displaystyle
C_{4}\tau\kappa_{0}\mu^{2}\frac{\sigma_{\xi}}{\lambda_{\min}}\left(\frac{\|T_{1}\|_{\ell_{1}}}{\|\mathcal{P}_{M}(T_{1})\|_{\rm
F}}+\frac{\|T_{2}\|_{\ell_{1}}}{\|\mathcal{P}_{M}(T_{2})\|_{\rm
F}}\right)\sqrt{\frac{\alpha_{d}rd_{1}d_{2}\log^{2}d_{1}}{n}}$ (34)
$\displaystyle+C_{5}\tau\gamma_{n}\sqrt{\log d_{1}},$

by the same argument in the proof of Theorem 1. Notice that
$\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{i})\rangle$, $i=1,2$ are the sum of
i.i.d. random variables, with covariance:

$\displaystyle\mathbb{E}\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{1})\rangle\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{2})\rangle$
$\displaystyle=\frac{d_{1}^{2}d_{2}^{2}}{n^{2}}\sum_{i\in
I_{2}}\xi_{i}^{2}\left\langle
X_{i},\mathcal{P}_{M}(T_{1})\right\rangle\left\langle
X_{i},\mathcal{P}_{M}(T_{2})\right\rangle$
$\displaystyle=\frac{d_{1}d_{2}}{n}\sigma_{\xi}^{2}\sum_{i\in[d_{1}]}\sum_{j\in[d_{2}]}e_{i}^{\top}\mathcal{P}_{M}(T_{1})e_{j}e_{i}^{\top}\mathcal{P}_{M}(T_{2})e_{j}$
$\displaystyle=\frac{d_{1}d_{2}}{n}\sigma_{\xi}^{2}\left\langle\mathcal{P}_{M}(T_{1}),\mathcal{P}_{M}(T_{2})\right\rangle.$

Then we have

$\operatorname{cov}\left(\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{1})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{1})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}},\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{2})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{2})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}\right)=:\rho_{T_{1},T_{2}}.$

We jointly control the c.d.f. of them by multivariate Berry-Essen theorem
(Stein,, 1972; Raič,, 2019)

$\displaystyle\sup_{t_{1},t_{2}\in\mathbb{R}}\left\lvert{\mathbb{P}}\left(\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{1})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{1})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}\leq
t_{1},\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{2})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{2})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}\leq
t_{2}\right)-\Phi_{\rho_{T_{1},T_{2}}}(t_{1},t_{2})\right\rvert\leq
C\mu\sqrt{\frac{rd_{1}}{n}}.$ (35)

The gradient bound $\left\lVert\nabla\Phi_{\rho}(t_{1},t_{2})\right\rVert\leq
C$ indicates the Lipschitz property of $\Phi_{\rho}(t_{1},t_{2})$, which
suggests the desired probability bound. ∎

###  B.3 Proof of Theorem 4

We proceed to prove Theorem 4 in the sequel by three steps: we first show that
$\mathbb{I}(W_{T}^{(i)}>t)$ follows weak dependency and asymptotic
symmetricity for $T\in\mathcal{H}_{0}$ when $t$ is in a certain region
$[0,L_{n}]$; then we show that with high probability, the data-driven
threshold is in the region $[0,L_{n}]$; finally we control the power when
strong signals dominate signals in the non-null set. Since $n=O(d_{1}d_{2})$
in general, we take $h_{n}=\Omega({\frac{\sqrt{\log
d_{2}}}{d_{2}}}\vee(rd_{1}/n)^{1/4})$ in the proof for simplicity. Here
$h_{n}$ can be smaller as long as $n$ is large. Denote
$E_{0}\in\sigma(\mathcal{D}_{0})$ as the event that we initialize our
algorithm with the required accuracy using $\mathcal{D}_{0}$. We first start
with the asymptotic property of ${W}^{(i)}_{T}$.

####  B.3.1 Weak dependence and symmetricity

From the proof of Theorem 1 and definition of $h_{n}$, we have the following
claim of the asymptotic normality of ${W}^{(1)}_{T}$.

######  Proposition 4.

Conditional on $E_{0}$, there exits a constant $C_{2}$ such that $W^{(1)}_{T}$
follows the asymptotic normality rate:

$\displaystyle\sup_{t\in\mathbb{R}}\left\lvert{\mathbb{P}}(W^{(1)}_{T}>t|E_{0})-\Phi(-t)\right\rvert\leq
C_{2}h_{n},$ (36)

for any $T\in\mathcal{H}_{0}$.

This proposition implies the asymptotic symmetricity of ${W}^{(1)}_{T}$,
${W}^{(2)}_{T}$ conditioned on $E_{0}$, which is crucial for the following
analysis. Since ${W}^{(1)}_{T}$, ${W}^{(2)}_{T}$ are asymptotically normal,
the c.d.f. of their product $W_{T}^{\mathsf{Rank}}$ will converge to an
asymptotically conditional symmetric random variable. We will show that,
conditional on the splits $\mathcal{D}_{0}$ and $\mathcal{D}_{1}$,
$W_{T}^{\mathsf{Rank}}$ is asymptotically symmetric for $T\in\mathcal{H}_{0}$.
Define

$G(t)=\frac{\sum_{T\in\mathcal{H}_{0}}{\mathbb{P}}({W}^{(1)}_{T}Z>t|\mathcal{D}_{0},\mathcal{D}_{1})}{q_{0}},$

where $Z$ is a standard Gaussian variable. Here since
${W}^{(1)}_{T}\in\sigma(\mathcal{D}_{0},\mathcal{D}_{1})$, ${W}^{(1)}_{T}$ is
fixed conditional on $\mathcal{D}_{0},\mathcal{D}_{1}$.

Denote
$L_{n}=G^{-1}(\frac{\epsilon_{n}\eta_{n}}{q_{0}})=\inf\left\\{t:G(t)\leq\frac{\epsilon_{n}\eta_{n}}{q_{0}}\right\\}$,
where $\epsilon_{n}$ is a rate to be specified later. We can exploit the
following asymptotic symmetric property of ${W}^{(1)}_{T}$ to investigate the
population version of the following $Ratio$:

$Ratio=\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L)}{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L)}.$

Here, we introduce a weaker characterization of strong signals, that is

$\mathcal{S}=\left\\{T\in\mathcal{H}:\frac{\sqrt{n}\left\lvert
M_{T}-\theta_{T}\right\rvert}{{\sigma_{\xi}\sqrt{d_{1}d_{2}}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{{\rm
F}}}\sqrt{(\log q+\log d)}}\geq C_{\mathsf{gap}}\right\\},$ (37)

with $\eta_{n}=\left\lvert\mathcal{S}\right\rvert$ for some large constant
$C_{\mathsf{gap}}$. In the following proof, we will actually focus on this
definition of strong signals. This condition is actually weaker than in our
main text, because $\left\lVert\mathcal{P}_{M}(T)\right\rVert_{{\rm
F}}\leq\left\lVert
T\right\rVert_{\ell_{1}}\max_{i,j}\left\lVert\mathcal{P}_{M}(e_{i}e_{j}^{\top})\right\rVert_{{\rm
F}}\leq 3{\mu}\left\lVert T\right\rVert_{\ell_{1}}\sqrt{\frac{r}{d_{2}}}$.
Thus, all the signals that satisfy condition (15) can also satisfy condition
(37), meaning that the $\eta_{n}$ defined here is always larger than that
defined in (15).

######  Lemma 4.

Conditional on $E_{0}$ and $\mathcal{D}_{1}$, we have

$\sup\limits_{0\leq t\leq
L_{n}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t)}{q_{0}G(t)}-1\right\rvert\leq
C_{3}\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}.$

###### Proof.

We only focus on small $h_{n}$. For each $T\in\mathcal{H}_{0}$, conditional on
$E_{0}$ and $\mathcal{D}_{1}$, Proposition 4 implies that

$\displaystyle\left\lvert{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t)-{\mathbb{P}}({W}^{(1)}_{T}Z>t|E_{0},\mathcal{D}_{1})\right\rvert\leq
C_{2}h_{n}.$

The definition of $L_{n}$ also implies
$G(t)\geq\frac{\epsilon_{n}\eta_{n}}{q_{0}}$. Then, we can derive the
following uniform bound of convergence:

$\sup\limits_{0\leq t\leq
L_{n}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t)}{q_{0}G(t)}-1\right\rvert\leq\sup\limits_{0\leq
t\leq
L_{n}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t)-G(t)}{q_{0}G(L_{n})}\right\rvert\leq
C_{3}\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}.$

∎

Then, we explore the weak dependency of linear forms under signals and
correlations assumptions. We will show that, with high probability, the
$Ratio$ can be very close to its population version described in Lemma 4.
Although we already have the intuition of dependency between different
$W_{T}^{(1)}$ by Theorem 3, the rate provided is not enough for FDR control
based on $W_{T}^{\mathsf{Rank}}$. Here, we study the correlation of
$W_{T}^{\mathsf{Rank}}$ between different $T$ with a more delicate analysis.
Let $T_{1}$, and $T_{2}$ be two different indexing matrices in
$\mathcal{H}_{0}$. To this end, we introduce the following Lemma:

######  Lemma 5 (Weak dependency of null statistics).

Conditional on $E_{0}$, $\mathcal{D}_{1}$,

$\displaystyle\sup_{0\leq t\leq
L_{n}}\frac{\sum_{(T_{i},T_{j})\in\mathcal{H}_{0,\text{weak}}^{2}}\left\lvert\operatorname{cov}(\mathbb{I}(W_{T_{i}}^{\mathsf{Rank}}>t),\mathbb{I}(W_{T_{j}}^{\mathsf{Rank}}>t))\right\rvert}{q_{0}^{2}G^{2}(t)}\leq
C_{1}\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+C_{2}\frac{1}{\left(\epsilon_{n}\eta_{n}q_{0}\right)^{\nu/2}},$
(38)

where $\nu$ is the weak correlation parameter defined in (13).

###### Proof.

Suppose we have a pair $(T_{1},T_{2})\in\mathcal{H}_{0,\text{weak}}^{2}$. Here
we adopt the notation in the proof of Theorem 2: denote

$\displaystyle(W_{T_{1}}^{(i)},W_{T_{2}}^{(i)})$
$\displaystyle=\left(\frac{\left\langle\widehat{Z}_{1}^{(i)},\mathcal{P}_{M}(T_{1})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{1})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}+\Delta_{T_{1}}^{(i)},\frac{\left\langle\widehat{Z}_{1}^{(i)},\mathcal{P}_{M}(T_{2})\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T_{2})\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}+\Delta_{T_{2}}^{(i)}\right)$
$\displaystyle:=\left(\widetilde{W}^{(i)}_{T_{1}}+\Delta_{T_{1}}^{(i)},\widetilde{W}^{(i)}_{T_{2}}+\Delta_{T_{2}}^{(i)}\right),\
i=1,2,$

where $\widehat{Z}_{1}^{(i)}$, $\Delta^{(i)}_{T}$ are defined analogously as
in the proof of Theorem 2. We have
$\mathbb{E}(\widetilde{W}^{(1)}_{T})^{2}=\mathbb{E}(\widetilde{W}^{(2)}_{T})^{2}=1$.
Here $\widetilde{W}^{(1)}_{T}$ and $\widetilde{W}^{(2)}_{T}$ are standardized
averages of $n$ i.i.d. samples and can be regarded as the cores which lead to
asymptotic normality of $W^{(1)}_{T}$, $W^{(2)}_{T}$. By the proof of Theorem
1, the remainder term $\Delta^{(i)}_{T}$ is controlled by:

${\mathbb{P}}\left(\left\lvert\Delta^{(i)}_{T}\right\rvert>c_{1}h_{n}|E_{0}\right)\leq
C_{2}\frac{\log d_{1}}{d_{1}^{2}}.$ (39)

For $i=1$, as is shown in the proof of Theorem 2, by multivariate Berry–Esseen
theorem, $(\widetilde{W}^{(2)}_{T_{1}},\widetilde{W}^{(2)}_{T_{2}})$ converges
to normal variable $\omega_{1}\sim\mathcal{N}(0,R)$ conditional on $E_{0}$
where $R_{11}=R_{22}=1$ and
$R_{12}=R_{21}=\operatorname{cov}(\widetilde{W}^{1}_{T_{1}},\widetilde{W}^{1}_{T_{2}})=\rho_{T_{1},T_{2}}$
with the error bound:

$\displaystyle\left\lvert{\mathbb{P}}\left((\widetilde{W}^{(2)}_{T_{1}},\widetilde{W}^{(2)}_{T_{2}})\in{A}\middle|E_{0}\right)-{\mathbb{P}}(\omega_{1}\in{A})\right\rvert\leq
C\mu\sqrt{\frac{rd_{1}}{n}},$ (40)

for any convex set ${A}\subseteq\mathbb{R}^{2}$. Here the
$\rho:=\rho_{T_{1},T_{2}}$ is the correlation between $W^{(1)}_{T}$,
$W^{(2)}_{T}$ defined in (12). By the following calculation of the covariance
between $\mathbb{I}(W_{T_{1}}^{\mathsf{Rank}}>t)$ and
$\mathbb{I}(W_{T_{2}}^{\mathsf{Rank}}>t)$ conditional on $E_{0}$ and
$\mathcal{D}_{1}$, we have:

$\displaystyle\left\lvert\operatorname{cov}(\mathbb{I}(W_{T_{1}}^{\mathsf{Rank}}>t),\mathbb{I}(W_{T_{2}}^{\mathsf{Rank}}>t))\right\rvert$
$\displaystyle=\left\lvert{\mathbb{P}}(W^{(1)}_{T_{1}}W^{(2)}_{T_{1}}>t,W^{(1)}_{T_{2}}W^{(2)}_{T_{2}}>t)-{\mathbb{P}}(W^{(1)}_{T_{1}}W^{(2)}_{T_{1}}>t){\mathbb{P}}(W^{(1)}_{T_{2}}W^{(2)}_{T_{2}}>t)\right\rvert$
$\displaystyle\leq\left\lvert{\mathbb{P}}(W^{(1)}_{T_{1}}W^{(2)}_{T_{1}}>t,W^{(1)}_{T_{2}}W^{(2)}_{T_{2}}>t)-{\mathbb{P}}({W}^{(1)}_{T_{1}}w_{11}>t,{W}^{(1)}_{T_{2}}w_{12}>t)\right\rvert\text{
(1)}$
$\displaystyle+\left\lvert{\mathbb{P}}(W^{(1)}_{T_{1}}W^{(2)}_{T_{1}}>t){\mathbb{P}}(W^{(1)}_{T_{2}}W^{(2)}_{T_{2}}>t)-{\mathbb{P}}({W}^{(1)}_{T_{1}}w_{11}>t){\mathbb{P}}({W}^{(1)}_{T_{2}}w_{12}>t)\right\rvert\text{
(2)}$
$\displaystyle+\left\lvert{\mathbb{P}}({W}^{(1)}_{T_{1}}w_{11}>t,{W}^{(1)}_{T_{2}}w_{12}>t)-{\mathbb{P}}({W}^{(1)}_{T_{1}}w_{11}>t){\mathbb{P}}({W}^{(1)}_{T_{2}}w_{12}>t)\right\rvert\text{
(3)}.$

Term (1), (2), (3) can be controlled separately. For (1), conditional on
$E_{0}$ and $\mathcal{D}_{1}$, we invoke multivariate Berry–Esseen theorem
(40) to bound the joint c.d.f. of $(W^{(2)}_{T_{1}},W^{(2)}_{T_{2}})$ by

$\displaystyle{\mathbb{P}}(W^{(2)}_{T_{1}}>t_{1},W^{(2)}_{T_{2}}>t_{2})$
$\displaystyle\leq{\mathbb{P}}(W^{(2)}_{T_{1}}>t_{1},W^{(2)}_{T_{2}}>t_{2},\left\lvert\Delta^{(2)}_{T_{1}}\right\rvert\leq
c_{1}h_{n},\left\lvert\Delta^{(2)}_{T_{2}}\right\rvert\leq
c_{1}h_{n})+\frac{2c_{2}\log d_{1}}{d_{1}^{2}}$
$\displaystyle\leq{\mathbb{P}}(\widetilde{W}^{(2)}_{T_{1}}>t_{1}-c_{1}h_{n},\widetilde{W}^{(2)}_{T_{2}}>t_{2}-c_{1}h_{n})+\frac{2c_{2}\log
d_{1}}{d_{1}^{2}}$
$\displaystyle\leq{\mathbb{P}}(\omega_{11}>t_{1},\omega_{12}>t_{2})+c_{1}\left[\phi(t_{1}){\mathbb{P}}(\omega_{12}>t_{2}|\omega_{11}=t_{1})+\phi(t_{2}){\mathbb{P}}(\omega_{11}>t_{1}|\omega_{12}=t_{2})\right]h_{n}+C_{2}h_{n}^{2},$

where we apply Taylor expansion to the c.d.f. of normal distribution
$\omega_{1}\sim\mathcal{N}(0,R)$ and apply the upper bound $\log d_{1}\cdot
d_{1}^{-2}\leq h_{n}^{2}$. Analogously, it also holds that

$\displaystyle{\mathbb{P}}(W^{(2)}_{T_{1}}>t_{1},W^{(2)}_{T_{2}}>t_{2})\geq{\mathbb{P}}(\widetilde{W}^{(2)}_{T_{1}}>t_{1}+c_{1}h_{n},\widetilde{W}^{(2)}_{T_{2}}>t_{2}+c_{1}h_{n})-\frac{2c_{2}\log
d_{1}}{d_{1}^{2}}$
$\displaystyle\geq{\mathbb{P}}(\omega_{11}>t_{1},\omega_{12}>t_{2})-c_{1}\left[\phi(t_{1}){\mathbb{P}}(\omega_{12}>t_{2}|\omega_{11}=t_{1})+\phi(t_{2}){\mathbb{P}}(\omega_{11}>t_{1}|\omega_{12}=t_{2})\right]h_{n}-C_{2}h_{n}^{2}.$

We conclude that, conditional on $E_{0}$ and $\mathcal{D}_{1}$

$\displaystyle\left\lvert{\mathbb{P}}(W^{(2)}_{T_{1}}>t_{1},W^{(2)}_{T_{2}}>t_{2})-{\mathbb{P}}(\omega_{11}>t_{1},\omega_{12}>t_{2})\right\rvert$
$\displaystyle\leq
c_{1}\left[\phi(t_{1}){\mathbb{P}}(\omega_{12}>t_{2}|\omega_{11}=t_{1})+\phi(t_{2}){\mathbb{P}}(\omega_{11}>t_{1}|\omega_{12}=t_{2})\right]h_{n}+C_{2}h_{n}^{2}.$

Using the Lipschitz property of $\Phi(t)$, we have

$\displaystyle\left\lvert{\mathbb{P}}(W^{(1)}_{T_{1}}W^{(2)}_{T_{1}}>t,W^{(1)}_{T_{2}}W^{(2)}_{T_{2}}>t)-{\mathbb{P}}(W^{(1)}_{T_{1}}\omega_{11}>t,W^{(1)}_{T_{2}}\omega_{12}>t)\right\rvert$
(41) $\displaystyle\leq
2c_{1}h_{n}\left({\mathbb{P}}(W^{(1)}_{T_{1}}\omega_{11}>t)+{\mathbb{P}}(W^{(1)}_{T_{2}}\omega_{12}>t)\right)+Ch_{n}^{2}.$

For (2), the proof of Lemma 4 also implies the following bound

$\displaystyle\left\lvert{\mathbb{P}}(W^{(1)}_{T_{1}}W^{(2)}_{T_{1}}>t){\mathbb{P}}(W^{(1)}_{T_{2}}W^{(2)}_{T_{2}}>t)-{\mathbb{P}}({W}^{(1)}_{T_{1}}w_{11}>t){\mathbb{P}}({W}^{(1)}_{T_{2}}w_{12}>t)\right\rvert$
(42) $\displaystyle\leq
2c_{1}h_{n}\left({\mathbb{P}}(W^{(1)}_{T_{1}}\omega_{11}>t)+{\mathbb{P}}(W^{(1)}_{T_{2}}\omega_{12}>t)\right)+Ch_{n}^{2},$

by the same argument conditional on $E_{0}$ and $\mathcal{D}_{1}$. Our next
step is to compare the c.d.f. of $(\omega_{1},\omega_{2})$ with standard
Gaussian $(Z_{1},Z_{2})$ to control the term (3), the difference between
${\mathbb{P}}(\omega_{11}>t_{1},\omega_{12}>t_{2})$ and
$\Phi(-t_{1})\Phi(-t_{2})$. Since
$(T_{1},T_{2})\in\mathcal{H}_{0,\text{weak}}^{2}$, the covariance between
$w_{11},w_{12}$ is thus bounded by: $\left\lvert\rho\right\rvert\leq
cq_{0}^{-\nu}$.

We invoke the property of bivariate Gaussian copula (Meyer,, 2013):

$\displaystyle\left\lvert{\mathbb{P}}(\omega_{11}>t_{1},\omega_{12}>t_{2})-\Phi(-t_{1})\Phi(-t_{2})\right\rvert=\left\lvert\int_{0}^{\rho}\phi_{2}(-t_{1},-t_{2},z)dz\right\rvert,$

where $\phi_{2}(x,y,z)$ is the p.d.f of bivariate normal distribution with
correlation coefficient $z$. Without loss of generality, assume
$t_{1},t_{2}>0$ are away from $0$. Thus, it is clear that

$\displaystyle\left\lvert{\mathbb{P}}(\omega_{11}>t_{1},\omega_{12}>t_{2})-\Phi(-t_{1})\Phi(-t_{2})\right\rvert$
$\displaystyle\leq\int_{0}^{\rho}\phi_{2}(-t_{1},-t_{2},z)dz,$
$\displaystyle\leq\frac{\rho}{2\pi\sqrt{1-\rho^{2}}}\exp\left(-\frac{t_{1}^{2}+t_{2}^{2}}{2}+\frac{\rho
t_{1}t_{2}}{\left(1-\rho^{2}\right)}\right)$
$\displaystyle\leq\frac{2\rho}{2\pi}\exp\left(-\frac{t_{1}^{2}+t_{2}^{2}}{2}(1-c\rho)\right)$
$\displaystyle=2\rho\left[\phi(-t_{1})\phi(-t_{2})\right]^{1-c\rho}.$

For any $\nu>0$, there exist $C_{\nu}>0$ such that $\Phi(-t)^{\nu}\leq
C_{\nu}/t$ for all $t>0$. Because by Mill’s ratio, we have:

$\Phi(-t)^{\nu}\leq\frac{\phi(-t)^{\nu}}{t^{\nu}}\leq
C_{\nu}\frac{1}{t^{1-\nu}}\frac{1}{t^{\nu}}=C_{\nu}\frac{1}{t},$

where we use the fact that $\phi(-t)^{\nu}\leq C_{\nu}t^{-(1-\nu)}$. Now
combine this with the upper bound of $\phi(-t)$: $\phi(-t)\leq C(t+1)\Phi(-t)$
, we have:

$\displaystyle\left\lvert{\mathbb{P}}(\omega_{11}>t_{1},\omega_{12}>t_{2})-\Phi(-t_{1})\Phi(-t_{2})\right\rvert$
$\displaystyle\leq 2\rho\left[\phi(-t_{1})\phi(-t_{2})\right]^{1-c\rho}$ (43)
$\displaystyle\leq
2\rho\left[C(\Phi(-t_{1})^{-\nu}+1)\Phi(-t_{1})(\Phi(-t_{2})^{-\nu}+1)\Phi(-t_{2})\right]^{1-c\rho}$
$\displaystyle\leq
C\rho\left[\Phi(-t_{1})\Phi(-t_{2})\right]^{(1-\nu)(1-c\rho)},$

for the term (3). Together with (41), (42), we can show that

$\displaystyle\sup_{0\leq t\leq
L_{n}}\sum_{(T_{i},T_{j})\in\mathcal{H}_{0,\text{weak}}^{2}}\frac{\left\lvert\operatorname{cov}(\mathbb{I}(W_{T_{i}}^{\mathsf{Rank}}>t),\mathbb{I}(W_{T_{j}}^{\mathsf{Rank}}>t))\right\rvert}{q_{0}^{2}G^{2}(t)}\leq\frac{8c_{1}h_{n}q_{0}G(t)}{q_{0}^{2}G(t)^{2}}$
$\displaystyle+\sup_{0\leq t\leq
L_{n}}\frac{\sum_{(T_{i},T_{j})\in\mathcal{H}_{0,\text{weak}}^{2}}C\rho\left[{\mathbb{P}}({W}^{(1)}_{T_{i}}Z>t){\mathbb{P}}({W}^{(1)}_{T_{j}}Z>t)\right]^{(1-\nu)(1-c\rho)}}{q_{0}^{2}G(t)^{2}}$
$\displaystyle\leq C\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+\sup_{0\leq t\leq
L_{n}}\frac{C\rho\left(\sum_{T\in\mathcal{H}_{0}}{\mathbb{P}}({W}^{(1)}_{T}Z>t)^{(1-\nu)(1-c\rho)}\right)^{2}}{q_{0}^{2}G(t)^{2}}$
$\displaystyle\leq C\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+\sup_{0\leq t\leq
L_{n}}C\rho\frac{\left(G(t)\right)^{2(1-\nu)(1-c\rho)}}{G(t)^{2}}$
$\displaystyle\leq
C\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+C{\rho}(\frac{q_{0}}{\epsilon_{n}\eta_{n}})^{3\nu}.$

The argument above is valid for any $\nu$, thus, we choose $3\nu$ to be the
$\frac{\nu}{2}$, where $\nu$ is defined in (13). It thus finishes the proof. ∎

We now apply the weak dependency yielded in Lemma 5 to derive a uniform bound
between $R$ and its population version:

######  Lemma 6.

For any $\varepsilon>0$, conditional on $E_{0}$ and $\mathcal{D}_{1}$, it
holds that

${\mathbb{P}}\left(\sup\limits_{0\leq t\leq
L_{n}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t)}{q_{0}G(t)}-1\right\rvert\geq\varepsilon\right)\leq\frac{C}{\varepsilon^{2}}\log(\frac{q_{0}}{\epsilon_{n}\eta_{n}})\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}}\right)^{\frac{1}{2}}+\left(\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+\frac{1}{\left(\epsilon_{n}\eta_{n}q_{0}\right)^{v/2}}\right)^{\frac{1}{2}}\right).$

###### Proof.

To prove the uniform convergence in probability, we define a grid on
$[0,L_{n}]$:

$\left\\{t_{k}=G^{-1}\left(\frac{1}{2}(2G(L_{n}))^{\frac{k}{K}}\right)\right\\}_{k=0}^{K},$

which equates each $G(t_{k})$ with $\frac{1}{2}(2G(L_{n}))^{\frac{k}{K}}$.
Then for each $t\in[t_{k-1},t_{k})$, the ratio can be bounded by:

$\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t_{k})}{q_{0}G(t_{k-1})}\leq\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t)}{q_{0}G(t)}\leq\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t_{k-1})}{q_{0}G(t_{k})}.$

Define $(2G(L_{n}))^{\frac{1}{K}}=r_{K}$, we have $G(t_{k})/G(t_{k-1})=r_{K}$,
and

$\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t)}{q_{0}G(t)}-1\right\rvert\leq\sup_{i=k-1,k}\frac{1}{r_{K}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t_{i})}{q_{0}G(t_{i})}-1\right\rvert+\left\lvert
r_{K}-1\right\rvert\vee\left\lvert\frac{1}{r_{K}}-1\right\rvert,$

for each $t\in[t_{k-1},t_{k})$. Then for any $t\in[0,L_{n}]$, it suffices to
control the quantities

$\sup_{k=0,\dots,K}\frac{1}{r_{K}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t_{k})}{q_{0}G(t_{k})}-1\right\rvert\leq\sup_{k=0,\dots,K}\frac{1}{r_{K}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t_{k})-{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t_{k})}{q_{0}G(t_{k})}\right\rvert+C_{3}\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}$

and $\left\lvert
r_{K}-1\right\rvert\vee\left\lvert\frac{1}{r_{K}}-1\right\rvert$ by
Proposition 4. Denote

$D_{n}=\sup\limits_{k=0,\dots,K}\frac{1}{r_{K}}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t_{k})-{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t_{k})}{q_{0}G(t_{k})}\right\rvert.$

It follows that

$\displaystyle\mathbb{E}D_{n}^{2}\leq\frac{K}{r_{K}^{2}}\mathbb{E}\left\lvert\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>t_{k})-{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t_{k})}{q_{0}G(t_{k})}\right\rvert^{2}$
(44)
$\displaystyle\leq\frac{K}{r_{K}^{2}}\frac{\sum\limits_{(T_{1},T_{2})\in\mathcal{H}^{2}_{0,\text{weak}}}\left\lvert\operatorname{cov}(\mathbb{I}(W_{T_{1}}^{\mathsf{Rank}}>t),\mathbb{I}(W_{T_{2}}^{\mathsf{Rank}}>t))\right\rvert+\sum\limits_{T_{1},T_{2}\in\mathcal{H}^{2}_{0,\text{strong}}}\left\lvert\operatorname{cov}(\mathbb{I}(W_{T_{1}}^{\mathsf{Rank}}>t),\mathbb{I}(W_{T_{2}}^{\mathsf{Rank}}>t))\right\rvert}{q_{0}^{2}G^{2}(t)},$

for any $t\in\\{t_{k}\\}$. Since the number of strong dependency pairs
$\left\lvert\mathcal{H}^{2}_{0,\text{strong}}\right\rvert\leq\beta_{\mathsf{s}}q_{0}^{2}$,
we have

$\frac{\sum\limits_{T_{1},T_{2}\in\mathcal{H}^{2}_{0,\text{strong}}}\left\lvert\operatorname{cov}(\mathbb{I}(W_{T_{1}}^{\mathsf{Rank}}>t),\mathbb{I}(W_{T_{2}}^{\mathsf{Rank}}>t))\right\rvert}{q_{0}^{2}G^{2}(t)}\leq\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}},$

for any $t\in[0,L_{n}]$. For the weak dependency pair,

$\frac{\sum\limits_{T_{1},T_{2}\in\mathcal{H}^{2}_{0,\text{weak}}}\left\lvert\operatorname{cov}(\mathbb{I}(W_{T_{1}}^{\mathsf{Rank}}>t),\mathbb{I}(W_{T_{2}}^{\mathsf{Rank}}>t))\right\rvert}{q_{0}^{2}G^{2}(t)}\leq
C_{1}\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+C_{2}\frac{1}{\left(\eta_{n}q_{0}\right)^{v/2}},$

where we apply our previous results in Lemma 5. What remains for us is to
specify the density of grid $K$. Choose a constant $\varsigma$ and we set

$K=\log(\frac{q_{0}}{\epsilon_{n}\eta_{n}})\min\left\\{\left(\frac{q_{0}^{2}\beta_{\mathsf{s}}}{\eta_{n}^{2}\epsilon_{n}}\right)^{-\varsigma},\left(\frac{q_{0}h_{n}}{\eta_{n}\epsilon_{n}}+\frac{1}{\left(\epsilon_{n}\eta_{n}q_{0}\right)^{v/2}}\right)^{-\varsigma}\right\\},$

then it is clear that
$1\leq\frac{1}{r_{k}}\leq\left[\frac{q_{0}}{\epsilon_{n}\eta_{n}}\right]^{1/K}\to
1$, and
$K(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}}+\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}})\to
0$. Therefore

$\displaystyle\left\lvert
r_{K}-1\right\rvert\vee\left\lvert\frac{1}{r_{K}}-1\right\rvert\leq
C\frac{1}{K}\log(\frac{q_{0}}{\epsilon_{n}\eta_{n}})\leq\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}}\right)^{\varsigma}+\left(\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+\frac{1}{\left(\epsilon_{n}\eta_{n}q_{0}\right)^{v/2}}\right)^{\varsigma}\right)$
$\displaystyle\mathbb{E}D^{2}_{n}\leq
CK\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\eta_{n}^{2}}+\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}\right)\leq
C\log(\frac{q_{0}}{\epsilon_{n}\eta_{n}})\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}}+\frac{1}{\left(\epsilon_{n}\eta_{n}q_{0}\right)^{v/2}}\right)^{1-\varsigma}+\left(\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}\right)^{1-\varsigma}\right).$

We can finish the proof of uniform convergence by using Markov’s inequality
with $\varsigma=\frac{1}{2}$. ∎

Recall the main theorem. For the ratio
$\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L)}{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L)}$,
we have

$Ratio=\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L)}{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L)}=\frac{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L)}{q_{0}G(t)}\cdot\frac{q_{0}G(t)}{\sum_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L)}.$

Then, it’s clear that, under the event that $L\leq L_{n}$, if Lemma 6 holds
for a $\varepsilon$, then we have

$Ratio\leq\frac{1+\varepsilon}{1-\varepsilon}\leq
1+\frac{2\varepsilon}{1-\varepsilon}\leq 1+3\varepsilon,$

with probability at least
$1-\frac{C}{\varepsilon^{2}}\log(\frac{q_{0}}{\epsilon_{n}\eta_{n}})\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}}\right)^{\frac{1}{2}}+\left(\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+(\epsilon_{n}\eta_{n}q_{0})^{-\nu/2}\right)^{\frac{1}{2}}\right)$.
By Lemma 6, we now successfully reduce our problem to proving our data-driven
threshold $L\leq L_{n}$ with high probability.

####  B.3.2  Threshold control

The gist of asymptotic threshold control is that when we choose $L_{n}$ as the
threshold and $d_{1},d_{2},n$ go large, entries with strong signals in
$\mathcal{S}$ can always pass the test, and other entries with weak signals or
no signal can pass the test will little possibility. We first focus on the
entries with strong signals. Denote the standardized signal
$\delta_{T}=(M_{T}-\theta_{T})/({\sigma}_{\xi}\|{\mathcal{P}}_{M}(T)\|_{\rm
F}\sqrt{d_{1}d_{2}/n})$, and $\widehat{W}^{1}_{T}=W^{(1)}_{T}-\delta_{T}$,
$\widehat{W}^{2}_{T}=W^{(2)}_{T}-\delta_{T}$. Given any $T\in\mathcal{H}_{1}$,
following the argument that is similar to the proof in Lemma 4, conditional on
$E_{0}$, we have

$\sup_{t\in\mathbb{R}}\left\lvert{\mathbb{P}}(W_{T}^{\mathsf{Rank}}>t)-{\mathbb{P}}\left((Z_{1}+\delta_{T})(Z_{2}+\delta_{T})>t\right)\right\rvert\leq
Ch_{n}.$

Here $(Z_{1},Z_{2})$ is standard Gaussian. Without loss of generality, assume
$M_{T}-\theta_{T}>0$. Then,

$\displaystyle{\mathbb{P}}(W_{T}^{\mathsf{Rank}}<L_{n})\leq$
$\displaystyle{\mathbb{P}}\left((Z_{1}+\delta_{T})(Z_{2}+\delta_{T})<L_{n}\right)+Ch_{n}$
$\displaystyle\leq$ $\displaystyle
1-{\mathbb{P}}\left((Z_{1}+\delta_{T})(Z_{2}+\delta_{T})\geq
L_{n}\right)+Ch_{n}$ $\displaystyle\leq$ $\displaystyle
1-{\mathbb{P}}\left(Z_{1}\geq-\delta_{T}+\sqrt{L_{n}}\right)^{2}+Ch_{n}.$

Here, we use the fact that

$\left\\{Z_{1}\geq-\delta_{T}+\sqrt{L_{n}}\right\\}\cap\left\\{Z_{2}\geq-\delta_{T}+\sqrt{L_{n}}\right\\}\subseteq\left\\{(Z_{1}+\delta_{T})(Z_{2}+\delta_{T})\geq
L_{n}\right\\}.$

An upper bound of $G(t)$ is given by

$G(t)=\frac{\sum_{T\in\mathcal{H}_{0}}{\mathbb{P}}({W}^{(1)}_{T}Z>t|\mathcal{D}_{0},\mathcal{D}_{1})}{q_{0}}\leq\frac{\sqrt{2}}{\sqrt{\pi}}\exp{\left(-\frac{t^{2}}{2\max_{T\in\mathcal{H}_{0}}\left\lvert{W}^{(1)}_{T}\right\rvert^{2}}\right)}.$

From Theorem 1, an uniform upper bound of
$\left\lvert{W}^{(1)}_{T}\right\rvert$ is given by:

${\mathbb{P}}\left(\max_{T\in\mathcal{H}_{0}}\left\lvert{W}^{(1)}_{T}\right\rvert\geq
C(h_{n}+\sqrt{\log d_{1}+\log
q})\middle|\mathcal{D}_{0}\right)\leq\frac{1}{d_{1}^{2}}.$

If $T\in\mathcal{S}$, then $\delta_{T}\geq C_{\mathsf{gap}}\sqrt{\log
d_{1}+\log q}$ by the definition of $\mathcal{S}$. The definition of $L_{n}$
implies that $L_{n}\leq
C\sqrt{\log(\frac{q_{0}}{\epsilon_{n}\eta_{n}})}\cdot\sqrt{\log d_{1}+\log
q}\ll\log(\frac{1}{h_{n}})\vee(\log d_{1}+\log q)$. Generally, we have
$d^{-10}\leq h_{n}$, thus the term $\log(\frac{1}{h_{n}})$ can be omitted.
Assume $C_{\mathsf{gap}}$ is large. It is clear that

${\mathbb{P}}\left(Z_{1}\geq-\delta_{T}+\sqrt{L_{n}}\right)^{2}\geq{\mathbb{P}}\left(Z_{1}\geq-C\sqrt{(\log
d_{1}+\log q)}\right)^{2}\geq(1-ch_{n})^{2},$

i.e., ${\mathbb{P}}(W_{T}^{\mathsf{Rank}}<L_{n})\leq Ch_{n}$. For any
$\varepsilon>0$, we compute the probability that
$\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})=\eta_{n}$
by finding its complement:

$\displaystyle{\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})\leq(1-\varepsilon)\eta_{n})$
$\displaystyle={\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<L_{n})>\varepsilon\eta_{n})\leq\frac{\sum\limits_{T\in\mathcal{S}}{\mathbb{P}}(W_{T}^{\mathsf{Rank}}<L_{n})}{\varepsilon\eta_{n}}\leq
Ch_{n}/\varepsilon,$

i.e.,
${\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})\leq(1-\varepsilon)\eta_{n})\to
0$,
${\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})\geq\eta_{n})\to
1$. This indicates that, all the signals in $\mathcal{S}$ can pass our test.
For our data-driven threshold (11), we have

$\displaystyle\sum\limits_{T\in\mathcal{H}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})\geq\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})\geq\frac{3}{4}\eta_{n},$
(45)

with probability at least $1-Ch_{n}$

Consider the probability
${\mathbb{P}}(\sum\limits_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L_{n})\geq\frac{\alpha}{4}\eta_{n})$
for the no-signal linear forms $T\in\mathcal{H}_{0}$. As we have shown in the
proof of Lemma 6, we have

$\displaystyle{\mathbb{P}}(\sum\limits_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L_{n})\geq
2\epsilon_{n}\eta_{n})\leq\log(\frac{q_{0}}{\epsilon_{n}\eta_{n}})\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}}\right)^{\frac{1}{2}}+\left(\frac{h_{n}q_{0}}{\epsilon_{n}\eta_{n}}+(\epsilon_{n}\eta_{n}q_{0})^{-\nu/2}\right)^{\frac{1}{2}}\right)\to
0,$

and consequently, by taking $\epsilon_{n}=\alpha/8$,

$\displaystyle{\mathbb{P}}(\sum\limits_{T\in\mathcal{H}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L_{n})\geq\frac{3}{4}\alpha\eta_{n})$
$\displaystyle\leq{\mathbb{P}}(2\sum\limits_{T\in\mathcal{H}_{0}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L_{n})\geq\frac{\alpha}{2}\eta_{n})+{\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}<-L_{n})\geq\frac{\alpha}{4}\eta_{n})$
(46)
$\displaystyle\leq\log(\frac{q_{0}}{\alpha\eta_{n}})\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\alpha^{2}\eta_{n}^{2}}\right)^{\frac{1}{2}}+\left(\frac{h_{n}q_{0}}{\alpha\eta_{n}}+(\epsilon_{n}\eta_{n}q_{0})^{-\nu/2}\right)^{\frac{1}{2}}\right)+Ch_{n}.$

Combining (45) and (46), it is sufficient to conclude that

${\mathbb{P}}\left(\frac{\sum\limits_{T\in\mathcal{H}}\mathbb{I}\left(T:W_{T}^{\mathsf{Rank}}<-L_{n}\right)}{\left(\sum\limits_{T\in\mathcal{H}}\mathbb{I}\left(T:W_{T}^{\mathsf{Rank}}>L_{n}\right)\right)\vee
1}\geq\alpha\right)\leq\log(\frac{q_{0}}{\alpha\eta_{n}})\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\alpha^{2}\eta_{n}^{2}}\right)^{\frac{1}{2}}+\left(\frac{h_{n}q_{0}}{\alpha\eta_{n}}+(\alpha\eta_{n}q_{0})^{-\nu/2}\right)^{\frac{1}{2}}\right)+Ch_{n},$

i.e., ${\mathbb{P}}(L\leq L_{n})\to 1$.

####  B.3.3 Power control

From the discussion on the threshold control, it is clear that for any
$\varepsilon$,

${\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})\leq(1-\varepsilon)\eta_{n})\leq\frac{\sum\limits_{T\in\mathcal{S}}{\mathbb{P}}(W_{T}^{\mathsf{Rank}}<L_{n})}{\varepsilon\eta_{n}}\leq
Ch_{n}/\varepsilon.$

Under the event that $L\leq L_{n}$, this also implies that with probability at
least $1-Ch_{n}/\varepsilon$,

$(1-\varepsilon)\eta_{n}\leq\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L_{n})\leq\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L).$

The probability of $\\{L\leq L_{n}\\}$ is lower bounded in Section B.3.2. We
can, therefore, get the power:

$\text{POWER}=\frac{\sum\limits_{T\in\mathcal{H}_{1}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L)}{q_{1}}\geq\frac{\sum\limits_{T\in\mathcal{S}}\mathbb{I}(W_{T}^{\mathsf{Rank}}>L)}{\eta_{n}}\cdot\frac{\eta_{n}}{q_{1}}\geq(1-\varepsilon)\frac{\eta_{n}}{q_{1}},$

with probability at least:

$1-C\log(\frac{q_{0}}{\alpha\eta_{n}})\left(\left(\frac{\beta_{\mathsf{s}}q_{0}^{2}}{\alpha^{2}\eta_{n}^{2}}\right)^{\frac{1}{2}}+\left(\frac{h_{n}q_{0}}{\alpha\eta_{n}}+(\alpha\eta_{n}q_{0})^{-\nu/2}\right)^{\frac{1}{2}}\right)-C\varepsilon^{-1}h_{n}.$

###  B.4 Proof of Proposition 1

###### Proof.

By definition, we can equally use the covariance matrix
$\mathbf{Q}^{*}=(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*})^{-1}=\left(\Sigma_{{\mathcal{A}}}^{-\frac{1}{2}\top}\Sigma_{{\mathcal{A}}}^{-\frac{1}{2}}\right)^{-1}$
to derive the correlation coefficient matrix. Here in the proof, we use bold
symbols like $\mathbf{Q}$ to distinguish our analysis from the $Q$ in the
Algorithm 2 of the main text, although they lead to the same correlation
structure. We will show that, if two linear forms indexed by $T_{i}$, $T_{j}$
are weakly correlated in $\mathbf{Q}^{*}$, i.e.,

$\left\lvert\frac{\mathbf{Q}^{*}_{ij}}{\sqrt{\mathbf{Q}^{*}_{ii}\mathbf{Q}^{*}_{jj}}}\right\rvert=\frac{\left\lvert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}e_{j}\right\rvert}{\sqrt{\mathbf{Q}^{*}_{ii}\mathbf{Q}^{*}_{jj}}}\leq
C_{1}q_{n}^{-\nu},$

then, in the data-driven covariance matrix $\mathbf{Q}$, they are also weakly
correlated:

$\left\lvert\frac{\mathbf{Q}_{ij}}{\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}}\right\rvert\leq
C_{2}q_{n}^{-\nu},$

with probability at least $1-Cd_{1}^{-2}\log d_{1}$. By definition, the
covariance matrix of $\widehat{\mathsf{w}}^{(2)}$ without normalization is

$\displaystyle\mathbf{Q}=$
$\displaystyle\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}\Sigma\mathbf{X}^{\top}\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}$
$\displaystyle=$
$\displaystyle\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}+\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\Delta\Sigma\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1},$

where we define
$\Delta\Sigma=\mathbf{X}\Sigma\mathbf{X}^{\top}-I=\widehat{\Sigma}^{-\frac{1}{2}}(\Sigma-\widehat{\Sigma})\widehat{\Sigma}^{-\frac{1}{2}}$.
The following Lemma characterizes the precision of our covariance estimation:

######  Lemma 7.

Suppose that we use $\widehat{U}$, $\widehat{V}$ obtained from
$\mathcal{D}_{0}$, $\mathcal{D}_{1}$ to estimate $\Sigma$:

$\widehat{\Sigma}=T_{{\mathcal{H}}}(I_{d_{1}d_{2}}-\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top})T_{{\mathcal{H}}}^{\top}.$

Then with probability at least $1-Cd_{1}^{-2}\log d_{1}$, we have

$\left\lVert\Sigma^{-\frac{1}{2}}(\Sigma-\widehat{\Sigma})\Sigma^{-\frac{1}{2}}\right\rVert\leq
C\frac{\kappa_{T}\sigma_{\xi}}{\lambda_{\min}}\left(\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\wedge
1\right)\sqrt{\frac{\kappa_{1}d_{1}^{2}d_{2}\log d_{1}}{n}}.$ (47)

For simplicity, we denote
$\kappa_{T}^{\prime}=\kappa_{T}\left(\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\wedge
1\right)$. Lemma 7 implies the bound of eigenvalue :
$\left\lvert\lambda_{i}(\Sigma^{-\frac{1}{2}}\widehat{\Sigma}\Sigma^{-\frac{1}{2}})-1\right\rvert=o_{p}(1)$
for all eigenvalues. Thus, the eigenvalues of its inverse can also be bounded
by the rate in (47), i.e.,

$\left\lVert\Delta\Sigma\right\rVert\leq
C\frac{\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{\kappa_{1}d_{1}^{2}d_{2}\log
d_{1}}{n}}.$

We then have

$\left\lvert\mathbf{Q}_{ij}-\mathbf{Q}_{ij}^{*}\right\rvert\leq\left\lvert
e_{i}^{\top}\left(\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}-\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}\right)e_{j}\right\rvert+\left\lvert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\Delta\Sigma\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{j}\right\rvert.$
(48)

Denote
$\mathbf{Q}^{\prime}=\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}$.
The first term in (48) can be controlled by:

$\displaystyle\left\lvert
e_{i}^{\top}\left(\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}-\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}\right)e_{j}\right\rvert$
(49) $\displaystyle=\left\lvert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}-\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}e_{j}\right\rvert$
$\displaystyle\leq
C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}\sqrt{\mathbf{Q}_{ii}^{\prime}\mathbf{Q}_{jj}^{*}},$

where we use the fact that

$\displaystyle\left\lVert\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}-\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right\rVert$
$\displaystyle\leq\left\lVert\widehat{\Sigma}^{-1}-\Sigma^{-1}\right\rVert\leq\left\lVert\Sigma^{-1}(\widehat{\Sigma}-\Sigma)\Sigma^{-1}\right\rVert+O\left(\left\lVert\widehat{\Sigma}-\Sigma\right\rVert^{2}\right)$
$\displaystyle\leq\frac{1}{\lambda_{\min}(\Sigma)}\left\lVert\Sigma^{-\frac{1}{2}}(\Sigma-\widehat{\Sigma})\Sigma^{-\frac{1}{2}}\right\rVert+o\left(\left\lVert\widehat{\Sigma}-\Sigma\right\rVert\right)$
$\displaystyle\leq
C\frac{\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}(\Sigma)\lambda_{\min}}\sqrt{\frac{\kappa_{1}d_{1}^{2}d_{2}\log
d_{1}}{n}},$

by Fréchet derivative (Higham,, 2008; Al-Mohy and Higham,, 2009) and Lemma 7,
and also we have

$\displaystyle\left\lVert\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}e_{j}\right\rVert^{2}$
$\displaystyle\leq\frac{1}{\lambda_{\min}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)}\left\lVert
e^{\top}_{j}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}e_{j}\right\rVert$
$\displaystyle\leq\lambda_{\max}(\Sigma)\mathbf{Q}_{jj}^{*},$

$\displaystyle\left\lVert\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}\right\rVert^{2}$
$\displaystyle\leq\frac{1}{\lambda_{\min}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)}\big{|}e^{\top}_{i}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}\big{|}$
$\displaystyle+\frac{1}{\lambda_{\min}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)}\big{|}e^{\top}_{i}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}-\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}\big{|}$
$\displaystyle\leq\lambda_{\max}(\Sigma)\mathbf{Q}_{ii}^{\prime}+C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}\left\lVert\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}\right\rVert^{2},$

which is equivalent to

$\displaystyle\left\lVert\left(\mathbf{X}_{\mathcal{A}}^{*\top}\mathbf{X}_{\mathcal{A}}^{*}\right)^{-1}e_{j}\right\rVert$
$\displaystyle\leq\sqrt{\lambda_{\max}(\Sigma)\mathbf{Q}_{jj}^{*}}$
$\displaystyle\left\lVert\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}\right\rVert$
$\displaystyle\leq(1+c)\sqrt{\lambda_{\max}(\Sigma)\mathbf{Q}_{ii}^{\prime}}.$

The second term in (48) can be bounded given that

$\displaystyle\left\lvert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\Delta\Sigma\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{j}\right\rvert$
$\displaystyle\leq\left\lVert\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{j}\right\rVert\left\lVert\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}\right\rVert\left\lVert\Delta\Sigma\right\rVert$
$\displaystyle=\sqrt{\mathbf{Q}_{ii}^{\prime}\mathbf{Q}_{jj}^{\prime}}\left\lVert\Delta\Sigma\right\rVert$
$\displaystyle\leq
C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}\sqrt{\mathbf{Q}_{ii}^{\prime}\mathbf{Q}_{jj}^{\prime}}.$

However, notice that,

$\left\lvert\frac{\mathbf{Q}_{ii}-\mathbf{Q}_{ii}^{\prime}}{\mathbf{Q}_{ii}^{\prime}}\right\rvert\leq
C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}.$

We can conclude that

$\left\lvert\mathbf{Q}_{ij}-\mathbf{Q}_{ij}^{*}\right\rvert\leq
C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}\left(\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}^{*}}+\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}\right).$

Setting $i=j$, we also have

$\frac{\left\lvert\mathbf{Q}_{jj}-\mathbf{Q}_{ij}^{*}\right\rvert}{\mathbf{Q}_{jj}}\leq
C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}\left(\sqrt{1+{\frac{\left\lvert\mathbf{Q}_{jj}-\mathbf{Q}_{jj}^{*}\right\rvert}{\mathbf{Q}_{jj}}}}+1\right),$

i.e.,

$\frac{\left\lvert\mathbf{Q}_{jj}-\mathbf{Q}_{jj}^{*}\right\rvert}{\mathbf{Q}_{jj}}\leq
C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}.$

We now compare the difference of correlation coefficients:

$\displaystyle\left\lvert\frac{\mathbf{Q}_{ij}}{\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}}-{\frac{\mathbf{Q}_{ij}^{*}}{\sqrt{\mathbf{Q}_{ii}^{*}\mathbf{Q}_{jj}^{*}}}}\right\rvert$
$\displaystyle\leq\frac{\left\lvert\mathbf{Q}_{ij}-\mathbf{Q}_{ij}^{*}\right\rvert}{\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}}+\left\lvert\mathbf{Q}_{ij}^{*}\right\rvert\frac{\left\lvert\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}-\sqrt{\mathbf{Q}_{ii}^{*}\mathbf{Q}_{jj}^{*}}\right\rvert}{\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}\sqrt{\mathbf{Q}_{ii}^{*}\mathbf{Q}_{jj}^{*}}}$
$\displaystyle+\left\lvert\mathbf{Q}_{ij}-\mathbf{Q}_{ij}^{*}\right\rvert\frac{\left\lvert\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}-\sqrt{\mathbf{Q}_{ii}^{*}\mathbf{Q}_{jj}^{*}}\right\rvert}{\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}\sqrt{\mathbf{Q}_{ii}^{*}\mathbf{Q}_{jj}^{*}}}$
$\displaystyle\leq
C\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}.$

If the assumption on the signal strength, i.e.,

$\frac{\kappa_{1}^{1.5}\kappa_{T}^{\prime}\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}\lesssim\frac{1}{q^{\nu}},$

is satisfied, we also have
${\left\lvert\mathbf{Q}_{ij}\right\rvert}/{\sqrt{\mathbf{Q}_{ii}\mathbf{Q}_{jj}}}\lesssim
q^{-\nu}$, which indicates that these two linear forms are also weakly
correlated in data-driven covariance matrix $\mathbf{Q}$. ∎

###  B.5 Proof of Proposition 2

###### Proof.

We start with the decomposition of LASSO response
$\mathbf{y}_{1}=\mathbf{X}\mathbf{W}^{(1)}$:

$\mathbf{y}_{1}=\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}\widehat{\mathsf{w}}+\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}\widetilde{\mathbf{W}},$

where
$\widehat{\mathsf{w}}_{i}=\frac{M_{T_{i}}-\theta_{T_{i}}}{\widehat{\sigma}_{\xi}^{(1)}\sqrt{d_{1}d_{2}}\widehat{s}_{T_{i}}^{(1)}}\sqrt{n}$
is the standardized signals with variance estimation with respect to $T_{i}$,
$\widetilde{\mathbf{W}}_{i}=\mathbf{W}_{i}^{(1)}/\widehat{s}_{T_{i}}^{(1)}-\mathsf{w}_{i}$
is the asymptotic normal noise. Here recall that $M_{T_{i}}:=\langle
M,T_{i}\rangle$ and
$\widehat{s}_{T_{i}}^{(1)}=\big{\|}{\mathcal{P}}_{\widehat{M}^{(1)}}(T_{i})\big{\|}_{\rm
F}$.

Our loading matrix is $\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}$, with

$\lambda_{\min}(\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D})=\frac{1}{\left\lVert\widehat{\Sigma}^{\frac{1}{2}}\widehat{D}^{-1}\right\rVert}=\frac{1}{\sqrt{\left\lVert\widehat{D}^{-1}\widehat{\Sigma}\widehat{D}^{-1}\right\rVert}}\geq\frac{1}{\sqrt{\left\lVert\widehat{D}^{-1}{\Sigma}\widehat{D}^{-1}\right\rVert+\left\lVert\widehat{D}^{-1}\left({\Sigma}-\widehat{\Sigma}\right)\widehat{D}^{-1}\right\rVert}}$

By (33) in the proof of Theorem 1, we have $\left\lvert
1-\widehat{s}_{T}^{(1)}/s_{T}\right\rvert\leq
C_{2}\frac{\mu\beta_{T}}{\beta_{0}}\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{\alpha_{d}d_{1}^{2}d_{2}\log
d_{1}}{n}}$ with probability at least $1-Cd_{1}^{-2}\log d_{1}$. Here $D:={\rm
diag}(s_{T_{1}},\cdots,s_{T_{q}})$. Thus, the absolute value of the diagonal
matrix can be controlled by:

$\left\lvert D^{-1}-\widehat{D}^{-1}\right\rvert\preceq
C_{2}\frac{\mu\beta_{T}}{\beta_{0}}\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{\alpha_{d}d_{1}^{2}d_{2}\log
d_{1}}{n}}D^{-1}.$ (50)

This indicates that

$\left\lVert\widehat{D}^{-1}{\Sigma}\widehat{D}^{-1}\right\rVert\leq(1+c)\left\lVert
D^{-1}\Sigma D^{-1}\right\rVert\leq\frac{3}{2}\kappa_{1},$

for a small $c$ as long as
$\frac{\mu\beta_{T}}{\beta_{0}}\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{\alpha_{d}d_{1}^{2}d_{2}\log
d_{1}}{n}}\to 0$; and also

$\left\lVert\widehat{D}^{-1}\left({\Sigma}-\widehat{\Sigma}\right)\widehat{D}^{-1}\right\rVert\leq(1+c)\left\lVert{D}^{-1}\left({\Sigma}-\widehat{\Sigma}\right){D}^{-1}\right\rVert\leq
C\frac{\beta_{T}\mu\sigma_{\xi}}{\beta_{0}\lambda_{\min}}\sqrt{\frac{\alpha_{d}\kappa_{1}qd_{1}^{2}d_{2}\log
d_{1}}{n}},$

which can be derived following the same steps as Lemma 7. It thus gives the
well-conditioning of our loading matrix in LASSO:

$\lambda_{\min}\left(\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}\right)\geq\frac{1}{\sqrt{2\kappa_{1}}}.$

Following a classic argument on the LASSO precision analysis (van de Geer and
Bühlmann,, 2009; Bühlmann and Van De Geer,, 2011), we have

$\displaystyle\left\lVert\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}\left(\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right)\right\rVert^{2}\leq$
$\displaystyle
2\left\langle\widehat{D}\widehat{\Sigma}^{-1}\widehat{D}\widetilde{\mathbf{W}},\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rangle+2\lambda\left(\left\lVert\widehat{\mathsf{w}}\right\rVert_{\ell_{1}}-\left\lVert\widehat{\mathsf{w}}^{(1)}\right\rVert_{\ell_{1}}\right)$
$\displaystyle\leq$
$\displaystyle\mathcal{\lambda}\left\lVert\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rVert_{\ell_{1}}+2\lambda\left(\left\lVert\widehat{\mathsf{w}}\right\rVert_{\ell_{1}}-\left\lVert\widehat{\mathsf{w}}^{(1)}\right\rVert_{\ell_{1}}\right),$

where we define $\lambda$ as the value that
${\mathbb{P}}\left(2\left\lVert\widehat{D}\widehat{\Sigma}^{-1}\widehat{D}\widetilde{\mathbf{W}}\right\rVert_{\infty}\geq\lambda\right)\leq
d_{1}^{-2}$. It is thus clear that

$\displaystyle\left\lVert\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}\left(\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right)\right\rVert^{2}\leq$
$\displaystyle
3\lambda\left\lVert\widehat{\mathsf{w}}^{(1)}_{s}-\widehat{\mathsf{w}}_{s}\right\rVert_{\ell_{1}}\leq
3\lambda\sqrt{q_{1}}\left\lVert\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rVert.$

Here, we use the subscript $s$ to denote the support set of $\mathsf{w}$.
Combined with the well-conditioning property of
$\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}$, we have

$\frac{1}{2\kappa_{1}}\left\lVert\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rVert^{2}\leq\left\lVert\widehat{\Sigma}^{-\frac{1}{2}}\widehat{D}\left(\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right)\right\rVert^{2}\leq
3\lambda\sqrt{q_{1}}\left\lVert\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rVert,$

i.e.,
$\left\lVert\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rVert\leq
6\lambda\kappa_{1}\sqrt{q_{1}}$. Then, it amounts to determining the
regularization level $\lambda$. Notice that
$\widehat{D}\widetilde{\mathbf{W}}={D}\widehat{\mathbf{W}}$, where
$\widehat{\mathbf{W}}_{i}=\mathbf{W}_{i}^{(1)}/s_{T_{i}}-\frac{M_{T_{i}}-\theta_{T_{i}}}{\widehat{\sigma}_{\xi}s_{T_{i}}\sqrt{d_{1}d_{2}}}\sqrt{n}$.
Here $\widehat{\mathbf{W}}_{i}$ and $\widetilde{\mathbf{W}}_{i}$ only differ
in the sampling variance $s_{T_{i}}$. We adopt the notation in the proof of
Theorem 1: we define an average of i.i.d. matrix as
$\widehat{Z}_{1}=\frac{d_{1}d_{2}}{n}\sum_{i\in I_{2}}\xi_{i}X_{i}$, and split
the noise
$\widehat{\mathbf{W}}=\widehat{\mathbf{W}}_{1}+\widehat{\mathbf{W}}_{2}$,
where

$\widehat{\mathbf{W}}_{1i}=\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T_{i})\right\rangle}{\sigma_{\xi}s_{T_{i}}\sqrt{d_{1}d_{2}/n}},\text{\leavevmode\nobreak\
for\leavevmode\nobreak\ each\leavevmode\nobreak\ }i\in[q].$ (51)

By Theorem 1, we have
$\left\lVert\widehat{\mathbf{W}}_{2}\right\rVert_{\infty}\leq Ch_{n}$, with
probability at least $1-Cd_{1}^{2}$. Therefore,

$\displaystyle
2\left\lVert\widehat{D}\widehat{\Sigma}^{-1}\widehat{D}\widetilde{\mathbf{W}}\right\rVert_{\infty}$
$\displaystyle=2\left\lVert\widehat{D}\widehat{\Sigma}^{-1}{D}\widehat{\mathbf{W}}\right\rVert_{\infty}\leq
2(1+ch_{n})\left\lVert{D}\widehat{\Sigma}^{-1}{D}\widehat{\mathbf{W}}\right\rVert_{\infty}$
$\displaystyle\leq
3\left(\left\lVert{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}\right\rVert_{\infty}+\left\lVert{D}(\widehat{\Sigma}^{-1}-{\Sigma}^{-1}){D}\widehat{\mathbf{W}}\right\rVert_{\infty}\right)$
$\displaystyle\leq
3\left(\left\lVert{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}_{1}\right\rVert_{\infty}+\left\lVert{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}_{2}\right\rVert_{\infty}+\left\lVert{D}(\widehat{\Sigma}^{-1}-{\Sigma}^{-1}){D}\widehat{\mathbf{W}}\right\rVert_{\infty}\right).$

For any $i$, it is clear that

$\displaystyle
e_{i}^{\top}{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}_{1}=\frac{\left\langle\operatorname{Vec}(\widehat{Z}_{1})^{\top},e_{i}^{\top}{D}{\Sigma}^{-1}T_{{\mathcal{H}}}(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top})\right\rangle}{\sigma_{\xi}\sqrt{d_{1}d_{2}/n}},$

with

$\displaystyle\mathbb{E}\left(e_{i}^{\top}{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}_{1}\right)^{2}$
$\displaystyle=e_{i}^{\top}D\Sigma^{-1}De_{i}\leq\kappa_{1}.$

According to Bernstein inequality, we have

$\frac{1}{\sqrt{n}}\left\lvert\frac{e_{i}^{\top}{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}_{1}}{\left(e_{i}^{\top}D\Sigma^{-1}De_{i}\right)^{\frac{1}{2}}}\right\rvert\leq
C_{1}\sqrt{\frac{(\log d_{1}+\log q)}{n}}+C_{2}\frac{\sqrt{rd_{1}}(\log
d_{1}+\log q)}{n},$

with probability at least $1-q^{-1}d_{1}^{-2}$. This indicates that

${\mathbb{P}}\left(\left\lVert{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}_{1}\right\rVert_{\infty}\geq
C\sqrt{\kappa_{1}(\log d_{1}+\log q)}\right)\leq d_{1}^{-2}.$

If we use $\widehat{U}$, $\widehat{V}$ to estimate $\Sigma$, then a
corresponding accuracy in $\|\cdot\|_{\infty}$-norm is given by:

######  Lemma 8.

If we use $\widehat{\Sigma}$ to approximate $\Sigma$, then

$\left\lVert{D}(\widehat{\Sigma}^{-1}-{\Sigma}^{-1}){D}\right\rVert_{\infty}\leq
C\left(\kappa_{\infty}\sqrt{\kappa_{1}}+\kappa_{1}^{1.5}\kappa_{T}\left(\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\wedge
1\right)\right)\frac{\beta_{T}\mu\sigma_{\xi}}{\beta_{0}\lambda_{\min}}\sqrt{\frac{\alpha_{d}qd_{1}^{2}d_{2}\log
d_{1}}{n}}.$

Here $\|M\|_{\infty}:=\max_{i}\|e_{i}^{\top}M\|_{\ell_{1}}$ and
$\kappa_{\infty}:=\|R^{-1}\|_{\infty}$ where $R=D^{-1}\Sigma D^{-1}$.

Notice that, since $\widehat{\mathbf{W}}_{1i}$ is standardized, Bernstein
inequality also gives the bound:

$\left\lVert\widehat{\mathbf{W}}_{1}\right\rVert_{\infty}\leq C\sqrt{\log
d_{1}+\log q},$

with probability at least $1-d_{1}^{-2}$. This indicates that, with
probability at least $1-Cd^{-2}_{1}$, we have

$2\left\lVert\widehat{D}\widehat{\Sigma}^{-1}\widehat{D}\widetilde{\mathbf{W}}\right\rVert_{\infty}\leq
C\left(\sqrt{\kappa_{1}(\log d_{1}+\log q)}+\kappa_{\infty}h_{n}\right)\leq
C\sqrt{\kappa_{1}(\log d_{1}+\log q)},$

as long as
$\left(\kappa_{\infty}h_{n}\right)\vee\left(\left(\kappa_{\infty}\sqrt{\kappa_{1}}+\kappa_{1}^{1.5}\kappa_{T}\left(\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\wedge
1\right)\right)\frac{\beta_{T}\mu\sigma_{\xi}}{\beta_{0}\lambda_{\min}}\sqrt{\frac{\alpha_{d}qd_{1}^{2}d_{2}\log
d_{1}}{n}}\right)\leq c\sqrt{\kappa_{1}}$ for some small constant $c$. Here we
use the fact
$\left\lVert{D}{\Sigma}^{-1}{D}\widehat{\mathbf{W}}_{2}\right\rVert_{\infty}\leq
C\kappa_{\infty}h_{n}$.This leads to the error bound of
$\widehat{\mathsf{w}}^{(1)}$:

$\left\lVert\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rVert_{\infty}\leq\left\lVert\widehat{\mathsf{w}}^{(1)}-\widehat{\mathsf{w}}\right\rVert\leq
6\lambda\kappa_{1}\sqrt{q_{1}}\leq C\kappa_{1}^{1.5}\sqrt{q_{1}(\log
d_{1}+\log q)}.$

Since for each $i$,

$\left\lvert\widehat{\mathsf{w}}_{i}-\mathsf{w}_{i}\right\rvert\leq\left(\frac{C_{1}\tau\log
d_{1}}{\sqrt{n}}+C_{2}\gamma_{n}^{2}+C_{3}\mu\frac{\|T\|_{\ell_{1}}}{\|T\|_{\rm
F}\beta_{0}}\cdot\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{\tau\alpha_{d}d_{1}^{2}d_{2}\log
d_{1}}{n}}\right)\left\lvert\mathsf{w}_{i}\right\rvert\leq
Ch_{n}\left\lvert\mathsf{w}_{i}\right\rvert,$

we finish the proof. ∎

###  B.6 Proof of Proposition 3

###### Proof.

We proceed to discuss the asymptotic normality of each
$e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{y}_{2}$:
since $\mathbf{y}_{2}=\mathbf{X}\mathbf{W}^{(2)}$, with

$\mathbf{y}_{2}=\widehat{\Sigma}^{-\frac{1}{2}}{D}\widehat{\mathsf{w}}+\widehat{\Sigma}^{-\frac{1}{2}}{D}\widehat{\mathbf{W}},$

where, with a slight abuse of notation, we define
$\widehat{\mathsf{w}}_{i}=\frac{M_{T}-\theta_{T}}{\widehat{\sigma}_{\xi}\sqrt{d_{1}d_{2}}{s}_{T}}\sqrt{n}$
is the standardized signals with variance estimation,
$\widehat{\mathbf{W}}_{i}=\mathbf{W}_{i}/s_{T_{i}}-\widehat{\mathsf{w}}_{i}$
is the asymptotic normal noise. From the proof of Theorem 1, it is clear that
$\widehat{\mathsf{w}}_{i}$ is close enough to ${\mathsf{w}}_{i}$. Notice that,
here, we do not assume $\mathcal{H}_{1}\subseteq\mathcal{A}$. For any
$i\in\mathcal{A}$ , we have

$\displaystyle
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{y}_{2}=e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\left[\mathbf{X}_{\mathcal{A}},\mathbf{X}_{\mathcal{A}^{c}}\right]D(\widehat{\mathsf{w}}+\widehat{\mathbf{W}})$
$\displaystyle=s_{T_{i}}\widehat{\mathsf{w}}_{i}+e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\widehat{\mathbf{W}}+e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}^{c}}D_{\mathcal{A}^{c}}\widehat{\mathsf{w}}_{\mathcal{A}^{c}}\
$
$\displaystyle=s_{T_{i}}\widehat{\mathsf{w}}_{i}+e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\left(\widehat{\mathbf{W}}_{1}+\widehat{\mathbf{W}}_{2}\right)+e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}^{c}}D_{\mathcal{A}^{c}}\widehat{\mathsf{w}}_{\mathcal{A}^{c}},$

where the noise decomposition
$\widehat{\mathbf{W}}=\widehat{\mathbf{W}}_{1}+\widehat{\mathbf{W}}_{2}$ is
the same as (51). If $T_{i}\in\mathcal{A}\cap\mathcal{H}_{0}$, we have
$\mathsf{w}_{i}=0$, thus
$e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{y}_{2}=e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\left(\widehat{\mathbf{W}}_{1}+\widehat{\mathbf{W}}_{2}\right)$.
We investigate the following terms: (i) the asymptotic normality of
$e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\widehat{\mathbf{W}}_{1}$,
(ii) the vanishing of
$e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\widehat{\mathbf{W}}_{2}$,
and (iii) the bias introduced by inconsistent screening
$e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}^{c}}D_{\mathcal{A}^{c}}\widehat{\mathsf{w}}_{\mathcal{A}^{c}}$.

(i) the asymptotic normality of
$\widehat{\beta}_{i}:=e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\widehat{\mathbf{W}}_{1}$.
Conditional on $\mathcal{D}_{0}$ and $\mathcal{D}_{1}$, $\widehat{\beta}_{i}$
can be viewed as sum of i.i.d. independent random variables:

$\widehat{\beta}_{i}=\frac{\left\langle\operatorname{Vec}(\widehat{Z}_{1}),e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}T_{{\mathcal{H}}}\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rangle}{\sigma_{\xi}\sqrt{d_{1}d_{2}/n}}.$
(52)

The variance of $\widehat{\beta}_{i}$ is given by

$\displaystyle\mathbb{E}\widehat{\beta}_{i}^{2}$ $\displaystyle=\left\lVert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}T_{{\mathcal{H}}}\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert^{2}$
$\displaystyle=e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}\Sigma\mathbf{X}^{\top}\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}=\mathbf{Q}_{ii}.$

The third-order moment of each component is also derived by

$\displaystyle\mathbb{E}$
$\displaystyle\left\lvert\sqrt{d_{1}d_{2}/n}\frac{\left\langle\operatorname{Vec}(\xi_{i}X_{i}),e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}T_{{\mathcal{H}}}\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rangle}{\sigma_{\xi}\mathbf{Q}_{ii}^{\frac{1}{2}}}\right\rvert^{3}$
$\displaystyle\leq
C\frac{\sqrt{d_{1}d_{2}}}{n^{1.5}}\frac{\left\lvert\left\langle\operatorname{Vec}(X_{i}),e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}T_{{\mathcal{H}}}\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rangle\right\rvert}{\mathbf{Q}_{ii}^{\frac{1}{2}}}$
$\displaystyle=C\frac{\sqrt{d_{1}d_{2}}}{n^{1.5}}\frac{\left\lvert\left\langle\operatorname{Vec}(X_{i})\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right),e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}T_{{\mathcal{H}}}\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rangle\right\rvert}{\mathbf{Q}_{ii}^{\frac{1}{2}}}$
$\displaystyle\leq
C\frac{\sqrt{d_{1}d_{2}}}{n^{1.5}}\left\lVert\operatorname{Vec}(X_{i})\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert_{\rm F}$ $\displaystyle\leq
C\frac{\mu\sqrt{rd_{1}}}{n^{1.5}},$

where we use the incoherence condition in the last inequality. It is thus
suggested that:

$\left\lvert{\mathbb{P}}\left(\frac{\widehat{\beta}_{i}}{\sqrt{\mathbf{Q}_{ii}}}\leq
t\middle|\mathcal{D}_{0},\mathcal{D}_{1}\right)-\Phi(t)\right\rvert\leq
C\mu\sqrt{\frac{rd_{1}}{n}}.$ (53)

(ii) the vanishing of
$\Delta{\beta}_{i}:=e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\widehat{\mathbf{W}}_{2}$.
By the proof of Theorem 1, we have
$\left\lVert\widehat{\mathbf{W}}_{2}\right\rVert_{\infty}\leq Ch_{n}$, with
probability at least $1-Cd_{1}^{-2}\log d_{1}$. Thus, by writing
$\mathbf{X}=[\mathbf{X}_{\mathcal{A}},\mathbf{X}_{\mathcal{\mathcal{A}}^{c}}]$,
we have

$\frac{\left\lvert\Delta{\beta}_{i}\right\rvert}{\sqrt{\mathbf{Q}_{ii}}}=\frac{\left\lvert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}D\widehat{\mathbf{W}}_{2}\right\rvert}{\sqrt{\mathbf{Q}_{ii}}}\leq\frac{\left\lvert
s_{T_{i}}\widehat{\mathbf{W}}_{2i}\right\rvert+\left\lvert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}^{c}}D_{\mathcal{A}^{c}}\widehat{\mathbf{W}}_{2,\mathcal{A}^{c}}\right\rvert}{\sqrt{\mathbf{Q}_{ii}}}.\\\
$

Using the definition of $C_{\infty}$, it follows that

$\frac{\left\lvert\Delta{\beta}_{i}\right\rvert}{\sqrt{\mathbf{Q}_{ii}}}\leq
CC_{\infty}h_{n},$

uniformly for all $i$ with probability at least $1-C\log d_{1}d_{1}^{-2}$.

(iii) the bias
$e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}^{c}}D_{\mathcal{A}^{c}}\widehat{\mathsf{w}}_{\mathcal{A}^{c}}$
can be surely controlled by

$\frac{\left\lvert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}^{c}}D_{\mathcal{A}^{c}}\widehat{\mathsf{w}}_{\mathcal{A}^{c}}\right\rvert}{\sqrt{\mathbf{Q}_{ii}}}\leq
C\cdot
C_{\infty}(h_{n}+\left\lVert\mathsf{w}_{\mathcal{A}^{c}}\right\rVert_{\infty}).$

Then, combing (i), (ii), and (iii) by the Lipschiz property of $\Phi(t)$, we
have

$\left\lvert{\mathbb{P}}\left(\frac{\widehat{\mathsf{w}}^{(2)}_{i}}{\sqrt{\mathbf{Q}_{ii}}}\leq
t\middle|\mathcal{D}_{0},\mathcal{D}_{1}\right)-\Phi(t)\right\rvert\leq C\cdot
C_{\infty}(h_{n}+\left\lVert\mathsf{w}_{\mathcal{A}^{c}}\right\rVert_{\infty})+C\mu\sqrt{\frac{rd_{1}}{n}}\leq
C\cdot
C_{\infty}(h_{n}+\left\lVert\mathsf{w}_{\mathcal{A}^{c}}\right\rVert_{\infty}).$

∎

###  B.7 Proof of Theorem 5

###### Proof.

In the following proof, we write
$h_{n}+\left\lVert\mathsf{w}_{\mathcal{A}^{c}}\right\rVert_{\infty}$ as
$h_{n}$ for notational simplicity. The proof essentially follows the proof of
Theorem 4. Define the expected false rejection:

$\widetilde{G}(t)=\frac{\sum_{{T_{i}}\in\mathcal{H}_{0}\cap\mathcal{A}}{\mathbb{P}}({\widehat{\mathsf{w}}_{i}^{(1)}\frac{\sqrt{\mathbf{Q}_{ii}}}{\widehat{\sigma}_{wi}}}Z>t|\mathcal{D}_{0},\mathcal{D}_{1})}{q_{0n}},$

where
$\widehat{\sigma}_{wi}^{2}=e_{i}^{\top}\big{(}\mathbf{X}_{{\mathcal{A}}}^{\top}\mathbf{X}_{{\mathcal{A}}}\big{)}^{-1}e_{i}$
is defined in Algorithm 3. Denote

$L_{n}^{\prime}=\widetilde{G}^{-1}\left(\frac{\epsilon_{n}\eta_{n}^{\prime}}{q_{0n}}\right)=\inf\left\\{t:\widetilde{G}(t)\leq\frac{\epsilon_{n}\eta_{n}^{\prime}}{q_{0n}}\right\\},$

where $\epsilon_{n}$ is a rate to be specified later, and
$q_{0n}=\left\lvert\mathcal{A}\cap\mathcal{H}_{0}\right\rvert$. We can rewrite
Lemma 4, 5, and 6 as:

######  Lemma 9.

Conditional on $E_{0}$ and $\mathcal{D}_{1}$, we have

$\sup\limits_{0\leq t\leq
L_{n}}\left\lvert\frac{\sum_{{T_{i}}\in\mathcal{H}_{0}\cap\mathcal{A}}{\mathbb{P}}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}>t)}{q_{0n}\widetilde{G}(t)}-1\right\rvert\leq
C_{3}\frac{C_{\infty}h_{n}q_{0n}}{\epsilon_{n}\eta_{n}^{\prime}}.$

Here we use $\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}$ to indicate the
combined statistics $\widehat{\mathsf{w}}_{T_{i}}^{\mathsf{Rank}}$

######  Lemma 10 (Weak dependency of null features).

Conditional on $E_{0}$, $\mathcal{D}_{1}$,

$\displaystyle\sup_{0\leq t\leq
L_{n}^{\prime}}\frac{\sum_{(T_{i},T_{j})\in\mathcal{H}_{0\mathcal{A},\text{weak}}^{2}}\left\lvert\operatorname{cov}(\mathbb{I}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}>t),\mathbb{I}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}>t))\right\rvert}{q_{0n}^{2}\widetilde{G}^{2}(t)}\leq
C_{1}\frac{C_{\infty}h_{n}q_{0n}}{\epsilon_{n}\eta_{n}^{\prime}}+C_{2}\frac{1}{\left(\epsilon_{n}\eta_{n}^{\prime}q_{0n}\right)^{v/2}}.$
(54)

######  Lemma 11.

For any $\varepsilon>0$, conditional on $E_{0}$ and $\mathcal{D}_{1}$, it
holds that

$\displaystyle{\mathbb{P}}\left(\sup\limits_{0\leq t\leq
L_{n}^{\prime}}\left\lvert\frac{\sum_{T_{i}\in\mathcal{H}_{0}\cap\mathcal{A}}\mathbb{I}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}>t))}{q_{0n}\widetilde{G}(t)}-1\right\rvert\geq\varepsilon\right)$
$\displaystyle\leq\frac{C}{\varepsilon^{2}}\log(\frac{q_{0n}}{\epsilon_{n}\eta_{n}^{\prime}})\left(\left(\frac{\beta_{\mathsf{s}}^{\prime}q_{0n}^{2}}{\epsilon_{n}^{2}\eta_{n}^{{}^{\prime}2}}\right)^{\frac{1}{2}}+\left(\frac{C_{\infty}h_{n}q_{0n}}{\epsilon_{n}\eta_{n}^{\prime}}+\frac{1}{\left(\epsilon_{n}\eta_{n}^{\prime}q_{0n}\right)^{v/2}}\right)^{\frac{1}{2}}\right).$

The proof of Lemma 9, 10, and 11 is same as that in Lemma 4, 5, and 6, and
thus omitted. These lemmas imply that, if $L\leq L_{n}^{\prime}$, then we have
$Ratio\leq 1+3\varepsilon$ with probability at least

$1-\frac{C}{\varepsilon^{2}}\log(\frac{q_{0n}}{\epsilon_{n}\eta_{n}^{\prime}})\left(\left(\frac{\beta_{\mathsf{s}}^{\prime}q_{0n}^{2}}{\epsilon_{n}^{2}\eta_{n}^{2}}\right)^{\frac{1}{2}}+\left(\frac{C_{\infty}h_{n}q_{0n}}{\epsilon_{n}\eta_{n}^{\prime}}+\frac{1}{\left(\epsilon_{n}\eta_{n}^{\prime}q_{0n}\right)^{v/2}}\right)^{\frac{1}{2}}\right).$

We then prove the probability of ${\mathbb{P}}(L\leq L_{n}^{\prime})$ can be
very large. A matching upper bound of $\widetilde{G}(t)$ is given by,
similarly as in the proof Theorem 4,

$\widetilde{G}(t)=\frac{\sum_{{T_{i}}\in\mathcal{H}_{0}\cap\mathcal{A}}{\mathbb{P}}({\widehat{\mathsf{w}}_{i}^{(1)}\frac{\sqrt{\mathbf{Q}_{ii}}}{\widehat{\sigma}_{wi}}}Z>t|\mathcal{D}_{0},\mathcal{D}_{1})}{q_{0n}}\leq\frac{\sqrt{2}}{\sqrt{\pi}}\exp{\left(-\frac{t^{2}}{2\max_{T\in\mathcal{H}_{0}\cap\mathcal{A}}\left\lvert\widehat{\mathsf{w}}_{i}^{(1)}\frac{\sqrt{\mathbf{Q}_{ii}}}{\widehat{\sigma}_{wi}}\right\rvert^{2}}\right)}.$

The LASSO results presented in Proposition 2 show that, the
$\left\lvert\widehat{\mathsf{w}}_{i}^{(1)}\right\rvert$ can be uniformly
bounded by:

$\displaystyle\max_{T\in\mathcal{H}_{0}\cap\mathcal{A}}\left\lvert\widehat{\mathsf{w}}_{i}^{(1)}\frac{\sqrt{\mathbf{Q}_{ii}}}{\widehat{\sigma}_{wi}}\right\rvert$
$\displaystyle\leq\max_{T\in\mathcal{H}_{0}\cap\mathcal{A}}\left\lvert\widehat{\mathsf{w}}_{i}^{(1)}\right\rvert\frac{\sqrt{e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}\Sigma\mathbf{X}^{\top}\mathbf{X}_{\mathcal{A}}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}}}{\sqrt{e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}}}$
$\displaystyle\leq\max_{T\in\mathcal{H}_{0}\cap\mathcal{A}}\left\lvert\widehat{\mathsf{w}}_{i}^{(1)}\right\rvert\left(1+\frac{\left\lVert
e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}\mathbf{X}_{\mathcal{A}}^{\top}\right\rVert\sqrt{\left\lVert\mathbf{X}\Sigma\mathbf{X}^{\top}-I\right\rVert}}{\sqrt{e_{i}^{\top}\left(\mathbf{X}_{\mathcal{A}}^{\top}\mathbf{X}_{\mathcal{A}}\right)^{-1}e_{i}}}\right)$
$\displaystyle\leq(1+c)\max_{T\in\mathcal{H}_{0}\cap\mathcal{A}}\left\lvert\widehat{\mathsf{w}}_{i}^{(1)}\right\rvert$
$\displaystyle\leq C\kappa_{1}^{1.5}\sqrt{q_{1}(\log d_{1}+\log q)},$

with probability at least $1-Cd_{1}^{-2}$. Here we use the fact that
$\left\lVert\mathbf{X}\Sigma\mathbf{X}^{\top}-I\right\rVert\leq\frac{1}{1-c}$
if we have its inverse
$\left\lVert\Sigma^{-\frac{1}{2}}\widehat{\Sigma}\Sigma^{-\frac{1}{2}}-I\right\rVert\leq
c$. The definition of $L_{n}^{\prime}$ implies that

$L_{n}^{\prime}\leq
C\sqrt{\log(\frac{q_{0n}}{\epsilon_{n}\eta_{n}^{\prime}})}\cdot
C\kappa_{1}^{1.5}\sqrt{q_{1}(\log d_{1}+\log
q)}\ll\sqrt{\log(\frac{1}{h_{n}})}\cdot\kappa_{1}^{1.5}\sqrt{q_{1}(\log
d_{1}+\log q)}.$

If $T_{i}\in\mathcal{S}$, then $\left\lvert\delta_{T_{i}}\right\rvert\geq
C_{\mathsf{gap}}\sqrt{\log\frac{1}{h_{n}}}\vee\kappa_{1}^{1.5}\sqrt{q_{1}(\log
d_{1}+\log q)}$ by the definition of $\mathcal{S}$, and also the LASSO
estimation:

$\left\lvert\widehat{\mathsf{w}}_{i}^{(1)}\right\rvert\geq
C\kappa_{1}^{1.5}\sqrt{q_{1}(\log d_{1}+\log q)},$

by our assumption. Assume $C_{\mathsf{gap}}$ is large enough, and
$\delta_{T_{i}}>0$. Then we have

$\displaystyle{\mathbb{P}}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}<L_{n}^{\prime})\leq$
$\displaystyle{\mathbb{P}}\left(\widehat{\mathsf{w}}_{i}^{(1)}(Z_{2}+\delta_{T_{i}}\frac{s_{T_{i}}}{\sqrt{\mathbf{Q}_{ii}}})<L_{n}^{\prime}\right)+C_{\infty}h_{n}$
$\displaystyle\leq$ $\displaystyle
1-{\mathbb{P}}\left((Z_{2}+\delta_{T_{i}}\frac{s_{T_{i}}}{\sqrt{\mathbf{Q}_{ii}}})\geq
L_{n}^{\prime}/\widehat{\mathsf{w}}_{i}^{(1)}\right)+C_{\infty}h_{n}$
$\displaystyle\leq$ $\displaystyle
1-{\mathbb{P}}\left(Z_{2}\geq-\delta_{T_{i}}+\sqrt{\log\frac{1}{h_{n}}}\right)+C_{\infty}h_{n}$
$\displaystyle\leq$
$\displaystyle{\mathbb{P}}\left(Z_{2}\leq-2\sqrt{\log\frac{1}{h_{n}}}\right)+C_{\infty}h_{n}$
$\displaystyle\leq$ $\displaystyle 2C_{\infty}h_{n}.$

We compute the probability:

$\displaystyle{\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}>L_{n}^{\prime})\leq(1-\varepsilon)\eta_{n}^{\prime})$
$\displaystyle={\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}<L_{n})>\varepsilon\eta_{n}^{\prime})$
$\displaystyle\leq\frac{\sum\limits_{T\in\mathcal{S}}{\mathbb{P}}(\widehat{\mathsf{w}}_{i}^{\mathsf{Rank}}<L_{n}^{\prime})}{\varepsilon\eta_{n}^{\prime}}\leq
CC_{\infty}h_{n}/\varepsilon,$

i.e.,
${\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(\widehat{\mathsf{w}}_{T}^{\mathsf{Rank}}>L_{n}^{\prime})\leq(1-\varepsilon)\eta_{n}^{\prime})\to
0$,
${\mathbb{P}}(\sum\limits_{T\in\mathcal{S}}\mathbb{I}(\widehat{\mathsf{w}}_{T}^{\mathsf{Rank}}>L_{n}^{\prime})\geq\eta_{n}^{\prime})\to
1$. By taking $\epsilon_{n}=\alpha/8$, other steps essentially follow the
proof of Theorem 4.

∎

##  Appendix C Proofs of Auxiliary Results

###  C.1 Verification of (10)

It has been shown in the proof of Theorem 1 that the test statistics $W_{T}$
can be decomposed as

$W_{T}=\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T)\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}+\Delta_{T},$

where $\Delta_{T}$ is a vanishing term with the rate of convergence described
in (34). Suppose also the distribution of $\xi$ is symmetric. Denote $I_{1}$
the index set of observations in sample ${\mathcal{D}}_{1}$. Therefore, for
any integer $k\geq 2$, we have

$\displaystyle\mathbb{E}\left\lvert
W_{T}\right\rvert^{2k}\gtrsim\mathbb{E}\left\lvert\frac{\left\langle\widehat{Z}_{1},\mathcal{P}_{M}(T)\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}\sqrt{d_{1}d_{2}/n}}\right\rvert^{2k}=\mathbb{E}\left\lvert\frac{\sqrt{d_{1}d_{2}/n}\sum_{i\in
I_{1}}\xi_{i}\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}}\right\rvert^{2k}$
$\displaystyle\geq\left(\mathbb{E}\left\lvert\frac{\sqrt{d_{1}d_{2}/n}\sum_{i\in
I_{1}}\xi_{i}\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle}{\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}}\right\rvert^{4}\right)^{k/2}$
$\displaystyle\geq\left(\frac{d_{1}^{2}d_{2}^{2}\left(\sum_{i\in
I_{1}}\mathbb{E}\xi_{i}^{4}\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle^{4}+\sum_{i,j\in I_{1},i\neq
j}\mathbb{E}\xi_{i}^{2}\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle^{2}\xi_{j}^{2}\left\langle
X_{j},\mathcal{P}_{M}(T)\right\rangle^{2}\right)}{n^{2}\sigma_{\xi}^{4}\left\lVert\mathcal{P}_{M}(T)\right\rVert^{4}_{\rm
F}}\right)^{k/2}$
$\displaystyle\gtrsim\left(\frac{d_{1}^{2}d_{2}^{2}\mathbb{E}\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle^{4}}{n\left\lVert\mathcal{P}_{M}(T)\right\rVert^{4}_{\rm
F}}+1\right)^{k/2}$
$\displaystyle=\left(\frac{d_{1}d_{2}\sum_{i\in[d_{1}],j\in[d_{2}]}\mathcal{P}_{M}(T)_{i,j}^{4}}{n\left\lVert\mathcal{P}_{M}(T)\right\rVert^{4}_{\rm
F}}+1\right)^{k/2}.$

If the energy of $\mathcal{P}_{M}(T)$ is concentrated in a few entries, e.g.,
there exists an index set $J$ such that the entries in $J$ can dominate other
entries, i.e.,

$\sum_{(i,j)\in J}\mathcal{P}_{M}(T)_{i,j}^{2}\geq\sum_{(i,j)\notin
J}\mathcal{P}_{M}(T)_{i,j}^{2},$

with $s_{0}:=\left\lvert J\right\rvert=O(1)$, then we have

$\displaystyle\frac{\sum_{i\in[d_{1}],j\in[d_{2}]}\mathcal{P}_{M}(T)_{i,j}^{4}}{\left\lVert\mathcal{P}_{M}(T)\right\rVert^{4}_{\rm
F}}\geq\frac{\sum_{(i,j)\in
J}\mathcal{P}_{M}(T)_{i,j}^{4}}{4\left(\sum_{(i,j)\in
J}\mathcal{P}_{M}(T)_{i,j}^{2}\right)^{2}}\geq\frac{1}{4s_{0}}\geq\Omega(1),$

and thus, we have

$\displaystyle\sqrt[2k]{\mathbb{E}\left\lvert
W_{T}\right\rvert^{2k}}\gtrsim\left(\frac{d_{1}d_{2}}{n}\right)^{1/4}.$

###  C.2 Proof of Theorem 8

###### Proof.

Define the c.d.f. of the product of two standard normal random variables as
$\Psi(t)$, also $\tilde{\Psi}(t):=1-\Psi(t)$. The c.d.f. of standard normal
distribution is denoted by $\Phi(t)$ by convention, with
$\tilde{\Phi}(t):=1-\Phi(t)$. For $j=1$, we have

$\displaystyle{\mathbb{P}}(Y_{1}>t|H_{0})$
$\displaystyle=\Psi(-t)=\tilde{\Psi}(t)$ $\displaystyle
F_{1}(z,t):={\mathbb{P}}(Y_{1}>t|z,H_{1})$
$\displaystyle={\mathbb{P}}\left(\frac{(\xi_{1}+\xi_{2}+2\delta)^{2}}{4}-\frac{(\xi_{1}-\xi_{2})^{2}}{4}>t\right)={\mathbb{P}}\left(\frac{(Z_{1}+\sqrt{2}\delta)^{2}}{2}-\frac{Z_{2}^{2}}{2}>t\right)$
$\displaystyle=\int_{\mathbb{R}}\left[\tilde{\Phi}(\sqrt{2t+y_{2}^{2}}-\sqrt{\frac{2}{p}}z)+\tilde{\Phi}(\sqrt{2t+y_{2}^{2}}+\sqrt{\frac{2}{p}}z)\right]dy_{2}.$

Here $\xi_{1}$, $\xi_{2}$, and $Z_{1}$, $Z_{2}$ are all standard normal random
variables. Thus $L_{p1}=\tilde{\Psi}^{-1}(p)$. Calculate the first order and
second order derivative of $F_{1}(z,t)$ with respect to $z$ when $t=L_{p1}$ :

$\displaystyle\partial_{z}F_{1}(0,L_{p1})$ $\displaystyle=0$
$\displaystyle\partial_{z}^{2}F_{1}(0,L_{p1})$
$\displaystyle=\frac{8}{q}\int_{0}^{+\infty}-f^{\prime}(\sqrt{2L_{p1}+y_{2}^{2}})dy_{2}=\frac{8}{p}f(-\sqrt{2L_{p1}}).$

Since $\tilde{\Psi}(t)<\sqrt{2}\tilde{\Phi}(\sqrt{2t})$, we have
$L_{p1}<\frac{1}{2}\tilde{\Phi}^{-1}(p/\sqrt{2})^{2}$. When $x\to 0$, we have

$\sqrt{2(\log(\frac{1-r_{1}}{x})-\frac{1}{2}\log\log(\frac{1-r_{1}}{x}))}\leq\tilde{\Phi}^{-1}(x)\leq\sqrt{2(\log(\frac{1}{x})-\frac{1}{2+r_{2}}\log\log(\frac{1}{x}))}.$

for any small $r_{1},r_{2}>0$. Thus we have
$L_{p1}<\frac{1}{2}\tilde{\Phi}^{-1}(p/\sqrt{2})^{2}\leq\log(\frac{\sqrt{2}}{p})-\frac{1}{2+r_{2}}\log\log(\frac{\sqrt{2}}{p})$,
and the second order derivative

$\displaystyle\partial_{z}^{2}F_{1}(0,L_{p1})$
$\displaystyle=\frac{8}{p}f(-\sqrt{2L_{p1}})\geq
c(\log(\frac{\sqrt{2}}{p}))^{1/(2+r_{2})},$

is non-vanishing.

For $j=2$, we have

$\displaystyle{\mathbb{P}}(Y_{2}>t|H_{0})$
$\displaystyle={\mathbb{P}}(\xi_{1}>t,\xi_{2}>t)+{\mathbb{P}}(\xi_{1}<-t,\xi_{2}<-t)=2\tilde{\Phi}^{2}(t)$
$\displaystyle F_{2}(z,t):={\mathbb{P}}(Y_{2}>t|z,H_{1})$
$\displaystyle={\mathbb{P}}(\xi_{1}+\mu>t,\xi_{2}+\mu>t)+{\mathbb{P}}(\xi_{1}+\mu<-t,\xi_{2}+\mu<-t)$
$\displaystyle=\tilde{\Phi}^{2}(t+\sqrt{\frac{1}{p}}z)+\tilde{\Phi}^{2}(t-\sqrt{\frac{1}{p}}z).$

In this case, the threshold $L_{p2}=\tilde{\Phi}^{-1}(\sqrt{\frac{p}{2}})$.
Compute the derivatives of $F_{2}$:

$\displaystyle\partial_{z}F_{2}(0,L_{p2})$ $\displaystyle=0$
$\displaystyle\partial_{z}^{2}F_{2}(0,L_{p2})$
$\displaystyle=\frac{4}{p}(f^{2}(-L_{p2})+\tilde{\Phi}(L_{p2})f^{\prime}(-L_{p2}))\geq
c((\log(\sqrt{\frac{2}{p}}))^{1/(2+r_{2})}+1),$

which also has a non-vanishing second-order derivative.

For $j=3$, we have

$\displaystyle{\mathbb{P}}(Y_{3}>t|H_{0})$
$\displaystyle={\mathbb{P}}(\xi_{1}+\xi_{2}>t,\xi_{1}>0,\xi_{2}>0)+{\mathbb{P}}(\xi_{1}+\xi_{2}<-t,\xi_{1}<0,\xi_{2}<0)$
$\displaystyle=2{\mathbb{P}}(Y_{1}>\frac{t}{\sqrt{2}},-Y_{1}<Y_{2}<Y_{1})=2\tilde{\Phi}(\frac{t}{\sqrt{2}})(1-\tilde{\Phi}(\frac{t}{\sqrt{2}}))$
$\displaystyle{\mathbb{P}}(Y_{3}>t|z,H_{1})$
$\displaystyle={\mathbb{P}}(Z_{1}>\frac{t-2\mu}{\sqrt{2}},-Z_{1}-\sqrt{2}\mu<Z_{2}<Z_{1}+\sqrt{2}\mu)$
$\displaystyle+{\mathbb{P}}(Z_{1}<\frac{-t-2\mu}{\sqrt{2}},Z_{1}+\sqrt{2}\mu<Z_{2}<-Z_{1}-\sqrt{2}\mu)$
$\displaystyle\leq\tilde{\Phi}(\frac{t}{\sqrt{2}})\left(\phi(\frac{t}{\sqrt{2}}+\sqrt{2}\mu)+\phi(\frac{t}{\sqrt{2}}-\sqrt{2}\mu)\right)$
$\displaystyle F_{3}(z,t):=$
$\displaystyle\tilde{\Phi}(\frac{t}{\sqrt{2}})\left(\phi(\frac{t}{\sqrt{2}}+\sqrt{\frac{2}{p}}z)+\phi(\frac{t}{\sqrt{2}}-\sqrt{\frac{2}{p}}z)\right).$

Compute the derivatives of $F_{3}$:

$\displaystyle\partial_{z}F_{3}(0,L_{p3})$ $\displaystyle=0$
$\displaystyle\partial_{z}^{2}F_{3}(0,L_{p3})$
$\displaystyle=\frac{4}{p}\tilde{\Phi}(\frac{L_{p3}}{\sqrt{2}})f^{\prime}(\frac{L_{p3}}{\sqrt{2}})\leq
0.$

If $\delta_{0}=o(\sqrt{\frac{1}{\pi}})$, we have $z=\sqrt{p}\delta\to 0$. By
Taylor’s theorem, we have

$\operatorname{Power}_{W_{j}}(L_{pj})=p+\mathbb{E}_{\mathbf{\Theta}}\partial_{z}F_{j}(0,L_{pj})z+\mathbb{E}_{\mathbf{\Theta}}\frac{1}{2}\partial_{z}^{2}F_{j}(0,L_{pj})z^{2}+o(\mathbb{E}_{\mathbf{\Theta}}z^{2}),$

(or $\leq$ for $j=3$ ). Plugging in the derivatives of $j=1,2,3$, clearly we
have
$\operatorname{Power}_{W_{1}}(L_{p1})\geq\operatorname{Power}_{W_{3}}(L_{p3})$,
and
$\operatorname{Power}_{W_{2}}(L_{p2})\geq\operatorname{Power}_{W_{3}}(L_{p3})$;
for the second order derivative of $F_{1}$ and $F_{2}$, we also have

$\displaystyle\partial_{z}^{2}F_{1}(0,L_{p1})-\partial_{z}^{2}F_{2}(0,L_{p2})$
$\displaystyle=\frac{4}{p}(2f(-\sqrt{2L_{p1}})-f^{2}(-L_{p2})-\tilde{\Phi}(L_{p2})f^{\prime}(-L_{p2}))$
$\displaystyle\geq
c\frac{1}{p}\exp\left(-\frac{1}{2}\tilde{\Phi}^{-1}(p/\sqrt{2})^{2}\right)\left(1-\exp\left(\frac{1}{2}\tilde{\Phi}^{-1}(p/\sqrt{2})^{2}-\tilde{\Phi}^{-1}(\sqrt{\frac{p}{2}})^{2}\right)\right).$

Since

$\displaystyle\frac{1}{2}\tilde{\Phi}^{-1}(p/\sqrt{2})^{2}-\tilde{\Phi}^{-1}(\sqrt{\frac{p}{2}})^{2}=(\frac{1}{\sqrt{2}}\tilde{\Phi}^{-1}(p/\sqrt{2})+\tilde{\Phi}^{-1}(\sqrt{\frac{p}{2}}))(\frac{1}{\sqrt{2}}\tilde{\Phi}^{-1}(p/\sqrt{2})-\tilde{\Phi}^{-1}(\sqrt{\frac{p}{2}}))$
$\displaystyle\leq(\frac{1}{\sqrt{2}}\tilde{\Phi}^{-1}(p/\sqrt{2})+\tilde{\Phi}^{-1}(\sqrt{\frac{p}{2}}))$
$\displaystyle\cdot(\sqrt{\log(\frac{\sqrt{2}}{p})-\frac{1}{1+r_{2}}\log\log(\frac{\sqrt{2}}{p})}-\sqrt{\log(\frac{2(1-r_{1})^{2}}{p})-\log\log((1-r_{1})\sqrt{\frac{2}{p}})})$
$\displaystyle\to-\infty,$

we have $\partial_{z}^{2}F_{1}(0,L_{p1})-\partial_{z}^{2}F_{2}(0,L_{p2})\geq
0$, thus
$\operatorname{Power}_{W_{1}}(L_{p1})\geq\operatorname{Power}_{W_{2}}(L_{p2})$.
Translating the $\operatorname{Power}_{W_{j}}(L_{pj})$ to
$\operatorname{Power}_{W_{j}}(L_{\alpha j})$, we finish our proof. ∎

###  C.3 Proof of Lemma 1

###### Proof.

To show this, we first state the perturbation of singular subspaces with
respect to different norms:

######  Lemma 12 (Xia and Yuan, (2021), Lemma 2).

Under good initialization and signal requirements, there exists an absolute
constant $C>0$ such that conditional on $E_{0}$, if $n\geq Cd_{1}\log d_{1}$,
then with probability at least $1-d_{1}^{-\tau}$,

$\left\lVert\widehat{Z}\right\rVert\leq
C_{2}\sqrt{\tau}(1+\gamma_{n})\sigma_{\xi}\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}},$

and

$\max\left\\{\left\lVert
UU^{\top}-\widehat{U}\widehat{U}^{\top}\right\rVert,\left\lVert
VV^{\top}-\widehat{V}\widehat{V}^{\top}\right\rVert\right\\}\leq
C_{2}\frac{\sqrt{\tau}\left(1+\gamma_{n}\right)\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}}.$

######  Lemma 13 (Xia and Yuan, (2021), Theorem 4).

Under good initialization and signal requirements, there exists an absolute
constant $C>0$ such that conditional on $E_{0}$, if $n\geq C\mu^{2}rd_{1}\log
d_{1}$, then with probability at least $1-5\log d_{1}\cdot d_{1}^{-\tau}$,

$\left\lVert UU^{\top}-\widehat{U}\widehat{U}^{\top}\right\rVert_{2,\max}\leq
C_{2}\mu\frac{\sqrt{\tau}\left(1+\gamma_{n}\right)\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{rd_{1}d_{2}\log
d_{1}}{n}},$

and

$\left\lVert VV^{\top}-\widehat{V}\widehat{V}^{\top}\right\rVert_{2,\max}\leq
C_{2}\mu\frac{\sqrt{\tau}\left(1+\gamma_{n}\right)\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{rd_{1}^{2}\log
d_{1}}{n}}.$

Then under the event when Lemma 12 and Lemma 13 both hold, it follows that

$\displaystyle\left\lvert\left\langle\widehat{U}\widehat{U}^{\top}\widehat{Z}\widehat{V}\widehat{V}^{\top}-UU^{\top}\widehat{Z}VV^{\top},T\right\rangle\right\rvert\leq\left\lVert
T\right\rVert_{\ell_{1}}\left\lVert\widehat{U}\widehat{U}^{\top}\widehat{Z}\widehat{V}\widehat{V}^{\top}-UU^{\top}\widehat{Z}VV^{\top}\right\rVert_{\max},$

and by triangular inequality,

$\displaystyle\left\lVert\widehat{U}\widehat{U}^{\top}\widehat{Z}\widehat{V}\widehat{V}^{\top}-UU^{\top}\widehat{Z}VV^{\top}\right\rVert_{\max}\leq$
$\displaystyle\left\lVert(\widehat{U}\widehat{U}^{\top}-UU^{\top})\widehat{Z}VV^{\top}\right\rVert_{\max}+\left\lVert
UU^{\top}\widehat{Z}(\widehat{V}\widehat{V}^{\top}-VV^{\top})\right\rVert_{\max}$
$\displaystyle+\left\lVert(\widehat{U}\widehat{U}^{\top}-UU^{\top})\widehat{Z}(\widehat{V}\widehat{V}^{\top}-VV^{\top})\right\rVert_{\max}$
$\displaystyle\leq$
$\displaystyle\left\|\widehat{Z}\right\|\left(\left\|\widehat{U}\widehat{U}^{\top}-UU^{\top}\right\|_{2,{\max}}\|V\|_{2,{\max}}+\left\lVert\widehat{V}\widehat{V}^{\top}-VV^{\top}\right\rVert_{2,\max}\left\lVert
U\right\rVert_{2,{\max}}\right)$
$\displaystyle+\left\|\widehat{Z}\right\|\left\|\widehat{U}\widehat{U}^{\top}-UU^{\top}\right\|_{2,{\max}}\left\lVert\widehat{V}\widehat{V}^{\top}-VV^{\top}\right\rVert_{2,\max}$
$\displaystyle\leq$ $\displaystyle
C_{2}\tau\mu^{2}\frac{\sigma_{\xi}}{\lambda_{\min}}\sqrt{\frac{rd_{1}^{2}d_{2}\log
d_{1}}{n}}\cdot\sigma_{\xi}\sqrt{\frac{rd_{1}\log d_{1}}{n}}.$

Thus, we conclude that under the event with probability larger than
$1-d^{-\tau}-5\log d_{1}\cdot d_{1}^{-\tau}\geq 1-6d_{1}^{-\tau}\log d_{1}$,
the desired bound holds. ∎

###  C.4 Proof of Lemma 2

###### Proof.

Denote $I_{1}$ the index set of observations in the sample
${\mathcal{D}}_{1}$. Since

$\displaystyle\left\langle
UU^{\top}\widehat{Z}_{2}V_{\perp}V_{\perp}^{\top}+U_{\perp}U^{\top}_{\perp}\widehat{Z}_{2}VV^{\top},T\right\rangle+\left\langle
UU^{\top}\widehat{Z}_{2}VV^{\top},T\right\rangle=\left\langle\widehat{Z}_{2},\mathcal{P}_{M}(T)\right\rangle$
$\displaystyle=\frac{d_{1}d_{2}}{n}\sum_{i\in
I_{1}}\left\langle\widehat{\Delta},X_{i}\right\rangle\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle-\left\langle\widehat{\Delta},\mathcal{P}_{M}(T)\right\rangle,$

and

$\displaystyle\left\lvert\frac{d_{1}d_{2}}{n}\left\langle\widehat{\Delta},X_{i}\right\rangle\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle-\frac{1}{n}\left\langle\widehat{\Delta},\mathcal{P}_{M}(T)\right\rangle\right\rvert$
$\displaystyle\leq\sqrt{3}\frac{\left\lVert\widehat{\Delta}\right\rVert_{\max}\mu\sqrt{rd_{1}^{2}d_{2}}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}+\sqrt{d_{1}d_{2}}\left\lVert\widehat{\Delta}\right\rVert_{\max}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}}{n},$
$\displaystyle\mathbb{E}\left\lvert\frac{d_{1}d_{2}}{n}\left\langle\widehat{\Delta},X_{i}\right\rangle\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle-\frac{1}{n}\left\langle\widehat{\Delta},\mathcal{P}_{M}(T)\right\rangle\right\rvert^{2}\leq\frac{d_{1}d_{2}\left\lVert\widehat{\Delta}\right\rVert_{\max}^{2}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}^{2}}{n^{2}},$

combined with initialization assumption and, by Bernstein inequality, we have

$\displaystyle\left\lvert\frac{d_{1}d_{2}}{n}\sum_{i\in
I_{1}}\left\langle\widehat{\Delta},X_{i}\right\rangle\left\langle
X_{i},\mathcal{P}_{M}(T)\right\rangle-\left\langle\widehat{\Delta},\mathcal{P}_{M}(T)\right\rangle\right\rvert\leq
C\gamma_{n}\sigma_{\xi}\left\lVert\mathcal{P}_{M}(T)\right\rVert_{\rm
F}\sqrt{\frac{\tau d_{1}d_{2}\log d_{1}}{n}},$

with probability at least $1-2d^{-\tau}$. Here $n\geq C\mu^{2}rd_{1}\log
d_{1}$. ∎

###  C.5 Proof of Lemma 7

###### Proof.

Notice that both
$\left(I_{d_{1}d_{2}}-\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}\right)$
and $\left(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)$ are projection matrices with $P=P^{2}$. We
thus have

$\displaystyle\widehat{\Sigma}-\Sigma$
$\displaystyle=T_{{\mathcal{H}}}\left((I_{d_{1}d_{2}}-\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top})-(I_{d_{1}d_{2}}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top})\right)T_{{\mathcal{H}}}^{\top}$ (55)
$\displaystyle=T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(I-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}$
$\displaystyle+T_{{\mathcal{H}}}\left(I-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}$
$\displaystyle+T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}.$

We apply (55) to the error
$\Sigma^{-\frac{1}{2}}(\widehat{\Sigma}-\Sigma)\Sigma^{-\frac{1}{2}}$:

$\displaystyle\left\lVert\Sigma^{-\frac{1}{2}}(\widehat{\Sigma}-\Sigma)\Sigma^{-\frac{1}{2}}\right\rVert\leq
2\left\lVert\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(I-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}\Sigma^{-\frac{1}{2}}\right\rVert$
$\displaystyle+\left\lVert\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}\Sigma^{-\frac{1}{2}}\right\rVert.$

Notice that $\left\lVert\left(I-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}\Sigma^{-\frac{1}{2}}\right\rVert\leq
1$. We only need to focus on the term

$\displaystyle\left\lVert\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$
$\displaystyle\leq\left\lVert\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\right\rVert\left\lVert\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$
$\displaystyle\leq\sqrt{\kappa_{1}}\kappa_{T}\left(\left\lVert\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\right)\otimes
V_{\perp}V_{\perp}^{\top}\right\rVert+\left\lVert
U_{\perp}U_{\perp}^{\top}\otimes\left(\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-V_{\perp}V_{\perp}^{\top}\right)\right\rVert\right.$
$\displaystyle+\left.\left\lVert\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\right)\otimes\left(\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-V_{\perp}V_{\perp}^{\top}\right)\right\rVert\right)$
$\displaystyle\leq
C_{2}\sqrt{\kappa_{1}}\kappa_{T}\frac{\sqrt{\tau}\left(1+\gamma_{n}\right)\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}},$

where we use the definition of $\kappa_{T}$ and the perturbation of singular
subspaces in Lemma 12. Moreover, when $T_{{\mathcal{H}}}$ is sparse, we use
$e_{T,k}\in\mathbb{R}^{d_{1}\times d_{2}}$,
$k\in[\operatorname{supp}(T_{{\mathcal{H}}})]$ to indicate the collective
supports of all the $\operatorname{vec}(T_{i})$. We then have

$\displaystyle\left\lVert\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$ (56)
$\displaystyle=\left\lVert\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\sum_{k=1}^{\operatorname{supp}(T_{{\mathcal{H}}})}e_{T,k}e_{T,k}^{\top}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$
$\displaystyle\leq\sqrt{\kappa_{1}}\kappa_{T}\operatorname{supp}(T_{{\mathcal{H}}})\max_{k}\left\lVert
e_{T,k}^{\top}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$
$\displaystyle\leq\sqrt{\kappa_{1}}\kappa_{T}\operatorname{supp}(T_{{\mathcal{H}}})\max_{k}\left(\left\lVert
e_{T,k}^{\top}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\right)\otimes
V_{\perp}V_{\perp}^{\top}\right\rVert\right.$ $\displaystyle+\left\lVert
e_{T,k}^{\top}U_{\perp}U_{\perp}^{\top}\otimes\left(\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-V_{\perp}V_{\perp}^{\top}\right)\right\rVert+\left.\left\lVert
e_{T,k}^{\top}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\right)\otimes\left(\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-V_{\perp}V_{\perp}^{\top}\right)\right\rVert\right).$

Since each $e_{T,k}$ can also be represented as $e_{T,k}=e_{T,k}^{1}\otimes
e_{T,k}^{2}$, where $e_{T,k}^{1}\in\mathbb{R}^{d_{1}}$ and
$e_{T,k}^{2}\in\mathbb{R}^{d_{2}}$ are also canonical bases, we then have

$\displaystyle\left\lVert\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$
$\displaystyle\lesssim\sqrt{\kappa_{1}}\kappa_{T}\operatorname{supp}(T_{{\mathcal{H}}})\left(\left\lVert
UU^{\top}-\widehat{U}\widehat{U}^{\top}\right\rVert_{2,\max}+\left\lVert
VV^{\top}-\widehat{V}\widehat{V}^{\top}\right\rVert_{2,\max}\right)$
$\displaystyle\leq
C\sqrt{\kappa_{1}}\kappa_{T}\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\frac{\sqrt{\tau}\left(1+\gamma_{n}\right)\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}},$

because the higher-order error can be dominated. The rate $\gamma_{n}$
converges to 0, which means that the whole error can be controlled by:

$\left\lVert\Sigma^{-\frac{1}{2}}(\widehat{\Sigma}-\Sigma)\Sigma^{-\frac{1}{2}}\right\rVert\leq
C\frac{\kappa_{T}\sigma_{\xi}}{\lambda_{\min}}\cdot\left(\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\wedge
1\right)\sqrt{\frac{\kappa_{1}d_{1}^{2}d_{2}\log d_{1}}{n}}.$

∎

###  C.6 Proof of Lemma 8

###### Proof.

Denote $E=\Sigma-\widehat{\Sigma}$. By Fréchet derivative, as long as
$\left\lVert E\right\rVert=\left\lVert\Sigma-\widehat{\Sigma}\right\rVert$ is
small for any operator norm, $\widehat{\Sigma}^{-1}-{\Sigma}^{-1}$ can be
dominated by its Fréchet derivative ${\Sigma}^{-1}E{\Sigma}^{-1}$. Therefore,
We have

$\left\lVert{D}(\widehat{\Sigma}^{-1}-{\Sigma}^{-1}){D}\right\rVert_{\infty}\leq\left\lVert{D}{\Sigma}^{-1}E{\Sigma}^{-1}{D}\right\rVert_{\infty}+o(\left\lVert
E\right\rVert_{\infty}).$

We only need to study the convergence rate of
$\left\lVert{D}{\Sigma}^{-1}E{\Sigma}^{-1}{D}\right\rVert_{\infty}$ as $E$ is
small. This term, however, can be decomposed following (55), i.e.,

$\displaystyle\left\lVert{D}{\Sigma}^{-1}E{\Sigma}^{-1}{D}\right\rVert_{\infty}$
$\displaystyle\leq\left\lVert{D}{\Sigma}^{-1}D\right\rVert_{\infty}\left\lVert
D^{-1}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(I-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}{\Sigma}^{-1}{D}\right\rVert_{\infty}$
$\displaystyle+\left\lVert{D}{\Sigma}^{-1}T_{{\mathcal{H}}}\left(I-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}{\Sigma}^{-1}{D}\right\rVert_{\infty}$
$\displaystyle+\left\lVert{D}{\Sigma}^{-1}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}{\Sigma}^{-1}{D}\right\rVert_{\infty}$
$\displaystyle\leq\kappa_{\infty}\sqrt{q}\left\lVert
D^{-1}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert_{2,\max}\sqrt{\kappa_{1}}$
$\displaystyle+\sqrt{\kappa_{1}}\sqrt{q}\left\lVert\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}{\Sigma}^{-1}{D}\right\rVert$
$\displaystyle+q\kappa_{1}^{2}\left\lVert
D^{-1}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert_{2,\max}^{2}$ $\displaystyle\leq
C\left(\kappa_{\infty}\sqrt{\kappa_{1}}+\left\lVert
T_{{\mathcal{H}}}\right\rVert_{2}\kappa_{1}/\sqrt{\lambda_{\min}(\Sigma)}\right)\frac{\beta_{T}\mu\sigma_{\xi}}{\beta_{0}\lambda_{\min}}\sqrt{\frac{\alpha_{d}qd_{1}^{2}d_{2}\log
d_{1}}{n}}$ $\displaystyle\leq
C\left(\kappa_{\infty}\sqrt{\kappa_{1}}+\kappa_{1}^{1.5}\kappa_{T}\right)\frac{\beta_{T}\mu\sigma_{\xi}}{\beta_{0}\lambda_{\min}}\sqrt{\frac{\alpha_{d}qd_{1}^{2}d_{2}\log
d_{1}}{n}},$

where the 2-max norm here can be bounded by:

$\displaystyle\left\lVert
D^{-1}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert_{2,\max}$ (57)
$\displaystyle\leq\max_{T_{i}\in\mathcal{H}}\frac{\sum_{j\in[d_{1}]}\sum_{k\in[d_{2}]}\left\lvert
T_{i}(j,k)\left[\left(UU^{\top}-\widehat{U}\widehat{U}^{\top}\right)\otimes
V_{\perp}V_{\perp}^{\top}+U_{\perp}U_{\perp}^{\top}\otimes\left(\widehat{V}\widehat{V}^{\top}-VV^{\top}\right)\right]\cdot
e_{j}\otimes e_{k}\right\rvert}{s_{T_{i}}}$
$\displaystyle\quad+\max_{T_{i}\in\mathcal{H}}\frac{\sum_{j\in[d_{1}]}\sum_{k\in[d_{2}]}\left\lvert
T_{i}(j,k)\left(UU^{\top}-\widehat{U}\widehat{U}^{\top}\right)\otimes\left(\widehat{V}\widehat{V}^{\top}-VV^{\top}\right)\cdot
e_{j}\otimes e_{k}\right\rvert}{s_{T_{i}}}$ $\displaystyle\leq
C\max_{T_{i}\in\mathcal{H}}\frac{\left\lVert
T\right\rVert_{\ell_{1}}}{\left\lVert T\right\rVert_{\rm
F}\beta_{0}\sqrt{r/d_{1}}}\frac{\mu\left(1+\gamma_{n}\right)\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{rd_{1}^{2}\log
d_{1}}{n}}$ $\displaystyle\leq
C\frac{\beta_{T}\mu\sigma_{\xi}}{\beta_{0}\lambda_{\min}}\sqrt{\frac{\alpha_{d}d_{1}^{2}d_{2}\log
d_{1}}{n}}.$

Here, we use the 2-max norm bound in Lemma 13, the alignment assumption, and
the definition of $\kappa_{T}$. Moreover, the norm
$\left\lVert\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)T_{{\mathcal{H}}}^{\top}{\Sigma}^{-1}{D}\right\rVert$
can also bounded by

$\displaystyle\left\lVert
D\Sigma^{-1}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\left(I-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$ $\displaystyle\leq\left\lVert
D\Sigma^{-\frac{1}{2}}\cdot\Sigma^{-\frac{1}{2}}T_{{\mathcal{H}}}\left(\widehat{U}_{\perp}\widehat{U}_{\perp}^{\top}\otimes\widehat{V}_{\perp}\widehat{V}_{\perp}^{\top}-U_{\perp}U_{\perp}^{\top}\otimes
V_{\perp}V_{\perp}^{\top}\right)\right\rVert$
$\displaystyle\lesssim{\kappa_{1}}\kappa_{T}\operatorname{supp}(T_{{\mathcal{H}}})\left(\left\lVert
UU^{\top}-\widehat{U}\widehat{U}^{\top}\right\rVert_{2,\max}+\left\lVert
VV^{\top}-\widehat{V}\widehat{V}^{\top}\right\rVert_{2,\max}\right)$
$\displaystyle\leq
C\kappa_{1}\kappa_{T}\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\frac{\sqrt{\tau}\left(1+\gamma_{n}\right)\sigma_{\xi}}{\lambda_{\min}}\cdot\sqrt{\frac{d_{1}^{2}d_{2}\log
d_{1}}{n}},$

where we use the sparsity of $T_{{\mathcal{H}}}$ following (56). This gives
the desired bound

$\left\lVert{D}{\Sigma}^{-1}E{\Sigma}^{-1}{D}\right\rVert_{\infty}\leq
C\left(\kappa_{\infty}\sqrt{\kappa_{1}}+\kappa_{1}^{1.5}\kappa_{T}\left(\frac{\operatorname{supp}(T_{{\mathcal{H}}})}{\sqrt{d_{2}}}\wedge
1\right)\right)\frac{\beta_{T}\mu\sigma_{\xi}}{\beta_{0}\lambda_{\min}}\sqrt{\frac{\alpha_{d}qd_{1}^{2}d_{2}\log
d_{1}}{n}}.$

∎