$\displaystyle\leq(1-c_{0})\cdot 2\sum_{k}\sum_{\begin{subarray}{c}i\in
S_{k},m\in S_{k},\\\ i\neq m\end{subarray}}\sum_{1\leq j\leq
p}\Bigl{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\Bigr{)}^{2}N_{i}N_{m}\Omega_{ij}\Omega_{mj}$
$\displaystyle\leq(1-c_{0})\cdot\Theta_{n4}.$ (A.103)

Combining (A.100), (A.101), (A.102), and (A.103) finishes the proof.

∎

###  A.18 Proof of Proposition A.2

By Lemmas A.6 and A.12,

$\displaystyle\mathrm{Var}(T)\gtrsim\Theta_{n2}+\Theta_{n3}+\Theta_{n4}\gtrsim\sum_{k}\|\mu_{k}\|^{2}.$
(A.104)

By Lemma A.11,

$\displaystyle\mathrm{Var}(V)$
$\displaystyle\lesssim\sum_{k}\frac{\|\mu_{k}\|^{2}}{n_{k}^{2}\bar{N}_{k}^{2}}\vee\sum_{k}\frac{\|\mu_{k}\|_{3}^{3}}{n_{k}\bar{N}_{k}}$
(A.105)

Using a similar argument based on Chebyshev’s inequality as in the proof of
Proposition A.1 and applying (A.104) and (A.105), we have

$\displaystyle\frac{|V-\mathbb{E}V|}{\mathrm{Var}(T)}\gtrsim\frac{|V-\mathbb{E}V|}{\sum_{k}\|\mu_{k}\|^{2}}=o_{\mathbb{P}}(1).$
(A.106)

Next, by Lemma A.10 and (A.104),

$\displaystyle\mathbb{E}V=\Theta_{n2}+\Theta_{n3}+\Theta_{n4}\lesssim\mathrm{Var}(T).$
(A.107)

Combining (A.106) and (A.107) finishes the proof. ∎

###  A.19 Proof of Proposition A.5

From the proof of Lemma A.10, we have

$\displaystyle V^{*}=V_{1}=\Theta_{n2}+A_{11}+A_{2},$

and the terms on the right-hand-side are mutually uncorrelated. From (A.93),
we have

$\displaystyle\mathrm{Var}(A_{11})\lesssim\sum_{i}\frac{\|\Omega_{i}\|_{3}^{3}}{N_{i}}$
$\displaystyle\mathrm{Var}(A_{2})\lesssim\sum_{i}\frac{\|\Omega_{i}\|^{2}}{N_{i}^{2}}.$

Hence

$\displaystyle\mathbb{E}V^{*}$ $\displaystyle=\Theta_{n2}$
$\displaystyle\mathrm{Var}(V^{*})$
$\displaystyle\lesssim\sum_{i}\frac{\|\Omega_{i}\|_{3}^{3}}{N_{i}}\vee\sum_{i}\frac{\|\Omega_{i}\|^{2}}{N_{i}^{2}}.$
(A.108)

Since $K=n$ and the null hypothesis holds, we have
$\Theta_{n1}\equiv\Theta_{n4}\equiv 0$. Moreover, by (A.71), we have

$\displaystyle\Theta_{n3}\lesssim\|\mu\|^{2}\ll\Theta_{n2}\asymp
n\|\mu\|^{2}.$

It follows that

$\displaystyle\mathrm{Var}(T)=[1+o(1)]\Theta_{n2}\asymp n\|\mu\|^{2}.$ (A.109)

Thus by (A.108) and Chebyshev’s inequality, we have

$\displaystyle\frac{V^{*}}{\mathrm{Var}(T)}=\frac{V^{*}-\mathbb{E}V^{*}}{\mathrm{Var}(T)}+\frac{\mathbb{E}V^{*}}{\mathrm{Var}(T)}=o_{\mathbb{P}}(1)+1+o(1),$

as desired.

∎

###  A.20 Proof of Proposition A.6

By Lemmas A.6 and A.12,

$\displaystyle\mathrm{Var}(T)\gtrsim\Theta_{n2}+\Theta_{n3}\gtrsim\sum_{i}\|\Omega_{i}\|^{2}.$
(A.110)

By (A.108),

$\displaystyle\mathrm{Var}(V^{*})$
$\displaystyle\lesssim\sum_{i}\frac{\|\Omega_{i}\|^{2}}{N_{i}^{2}}\vee\sum_{i}\frac{\|\Omega_{i}\|_{3}^{3}}{N_{i}}$
(A.111)

Using a similar argument based on Chebyshev’s inequality as in the proof of
Proposition A.1 and applying (A.110) and (A.111), we have

$\displaystyle\frac{|V^{*}-\mathbb{E}V^{*}|}{\mathrm{Var}(T)}\gtrsim\frac{|V^{*}-\mathbb{E}V^{*}|}{\sum_{i}\|\Omega_{i}\|^{2}}=o_{\mathbb{P}}(1).$
(A.112)

Next, by Lemma A.10 and (A.110),

$\displaystyle\mathbb{E}V^{*}=\Theta_{n2}\lesssim\mathrm{Var}(T).$ (A.113)

Combining (A.106) and (A.113) finishes the proof. ∎

##  Appendix B Proofs of asymptotic normality results

The goal of this section is to prove Theorems 3.1 and 3.2. The argument relies
on the martingale central limit theorem and the lemmas stated below. As a
preliminary, we describe a martingale decomposition of $T$ under the null.

Define

$\displaystyle U=\mathbf{1}_{p}^{\prime}(U_{3}+U_{4}),\qquad\text{and
}\,\,\,S=\mathbf{1}_{p}^{\prime}U_{2}.$

By Lemma A.1, we have $T=U+S$ under the null hypothesis. It holds that

$\displaystyle
U=\sum_{i<i^{\prime}}\sigma_{i,i^{\prime}}\sum_{r=1}^{N_{i}}\sum_{s=1}^{N_{i^{\prime}}}\big{(}\sum_{j}Z_{ijr}Z_{i^{\prime}js}\big{)}.$
(B.1)

where we define

$\sigma_{i,i^{\prime}}=\begin{cases}2\big{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\big{)}&\quad\text{
if }i,i^{\prime}\in S_{k}\text{ for some }k\\\ -\frac{2}{n\bar{N}}&\quad\text{
else}.\end{cases}$

Define a sequence of random variables

$\displaystyle
D_{\ell,s}=\sum_{i\in[\ell-1]}\sigma_{i,\ell}\sum_{r=1}^{N_{i}}\sum_{j}Z_{ijr}Z_{\ell
js}$ (B.2)

indexed by $(\ell,s)\in\\{(i,r)\\}_{1\leq i\leq n,1\leq r\leq N_{i}}$, where
these tuples are placed in lexicographical order. Precisely, we define

$(\ell_{1},s_{1})\prec(\ell_{2},s_{2})$

if either

  * •

$\ell_{1}<\ell_{2}$, or

  * •

$\ell_{1}=\ell_{2}$ and $s_{1}<s_{2}$.

Observe that

$\sum_{\ell,s}D_{\ell,s}=U.$

Next define $\mathcal{F}_{\prec(\ell,s)}$ to be the $\sigma$-field generated
by $\\{Z_{i:r}\\}_{(i,r)\prec(\ell,s)}$. Observe that

$\mathbb{E}[D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)}]=0,$

and hence $\\{D_{\ell,s}\\}$ is a martingale difference sequence. Turning to
$S$, we have

$\displaystyle S=\sum_{i=1}^{n}\sigma_{i}\sum_{r<s}\sum_{j}Z_{ijr}Z_{ijs}.$
(B.3)

where we define

$\sigma_{i}=2\Bigl{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\Bigr{)}\frac{N_{i}}{N_{i}-1}$

if $i\in S_{k}$. Define

$\displaystyle E_{\ell,s}=\sigma_{\ell}\sum_{r\in[s-1]}\sum_{j}Z_{\ell
jr}Z_{\ell js}.$ (B.4)

Note that $E_{\ell,1}=0$. Order $(\ell,s)$ lexicographically as above, and
recall that $\mathcal{F}_{\prec(\ell,s)}$ is the $\sigma$-field generated by
$\\{Z_{i:r}\\}_{(i,r)\prec(\ell,s)}$. Observe that

$\mathbb{E}[E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)}]=0,$

and hence $\\{E_{\ell,s}\\}$ is a martingale difference sequence. We have

$\displaystyle\sum_{(\ell,s)}\sigma_{\ell}\sum_{r\in[s-1]}\sum_{j}Z_{\ell
jr}Z_{\ell js}$
$\displaystyle=\sum_{\ell=1}^{n}\sum_{s=1}^{N_{\ell}}\sigma_{\ell}\sum_{r\in[s-1]}\sum_{j}Z_{\ell
jr}Z_{\ell js}=S.$

Define

$\displaystyle\mathcal{M}_{\ell,s}=D_{\ell,s}+E_{\ell,s},\quad\widetilde{\mathcal{M}}_{\ell,s}=\frac{\mathcal{M}_{\ell,s}}{\sqrt{\mathrm{Var}(T)}}.$
(B.5)

Thus we obtain the martingale decomposition:

$\displaystyle
T=U+S=\sum_{(\ell,s)}[D_{\ell,s}+E_{\ell,s}]=\sum_{(\ell,s)}\mathcal{M}_{\ell,s}.$
(B.6)

The technical results below are crucial to the proof of Theorem 3.1 given in
Section B.1. Theorem 3.2 then follows easily from Theorem 3.1 and Theorem A.1.

######  Lemma B.1.

Let $\widetilde{\mathcal{M}}_{\ell,s}$ be defined as in (B.5). It holds that

$\displaystyle\mathbb{E}\bigg{[}\sum_{(\ell,s)}\mathrm{Var}\big{(}\widetilde{\mathcal{M}}_{\ell,s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{)}\bigg{]}=1.$

######  Lemma B.2.

Suppose that $\min N_{i}\geq 2$ and $\max\|\Omega_{i}\|_{\infty}\leq 1-c_{0}$.
Under the null hypothesis, it holds that

$\displaystyle\mathrm{Var}\bigg{(}\sum_{(\ell,s)}\mathrm{Var}(D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})\bigg{)}\lesssim\big{(}\sum_{k}\frac{1}{n_{k}\bar{N}_{k}}\big{)}\|\mu\|_{3}^{3}+K\|\mu\|_{4}^{4}.$

######  Lemma B.3.

Suppose that $\min N_{i}\geq 2$ and $\max\|\Omega_{i}\|_{\infty}\leq 1-c_{0}$.
Under the null hypothesis, it holds that

$\displaystyle\sum_{(\ell,s)}\mathbb{E}D_{\ell,s}^{4}\lesssim\big{(}\sum_{k}\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\big{)}\|\mu\|^{2}+\big{(}\sum_{k}\frac{1}{n_{k}\bar{N}_{k}}\big{)}\|\mu\|_{3}^{3}\,,$

######  Lemma B.4.

Suppose that $\min N_{i}\geq 2$ and and $\max\|\Omega_{i}\|_{\infty}\leq
1-c_{0}$. Then we have

$\displaystyle\mathrm{Var}\bigg{(}\sum_{(\ell,s)}\mathrm{Var}(\tilde{E}_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})\bigg{)}\lesssim\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{3}\|\Omega_{i}\|_{3}^{3}}{n_{k}^{4}\bar{N}_{k}^{4}}\vee\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{4}\|\Omega_{i}\|_{4}^{4}}{n_{k}^{4}\bar{N}_{k}^{4}}$ (B.7)

######  Lemma B.5.

Suppose that $\min N_{i}\geq 2$ and and $\max\|\Omega_{i}\|_{\infty}\leq
1-c_{0}$. Then we have

$\displaystyle\sum_{(\ell,s)}\mathbb{E}\,E_{\ell,s}^{4}\lesssim\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{2}\|\Omega_{i}\|^{2}}{n_{k}^{4}\bar{N}_{k}^{4}}\vee\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{3}\|\Omega_{i}\|_{3}^{3}}{n_{k}^{4}\bar{N}_{k}^{4}}$

######  Lemma B.6.

Under either the null or alternative, it holds that

$\displaystyle\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{2}\|\Omega_{i}\|^{2}}{n_{k}^{4}\bar{N}_{k}^{4}}$
$\displaystyle\leq\sum_{k}\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\|\mu_{k}\|^{2}$
$\displaystyle\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{3}\|\Omega_{i}\|_{3}^{3}}{n_{k}^{4}\bar{N}_{k}^{4}}$
$\displaystyle\leq\sum_{k}\frac{1}{n_{k}\bar{N}_{k}}\|\mu_{k}\|_{3}^{3}$
$\displaystyle\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{4}\|\Omega_{i}\|_{4}^{4}}{n_{k}^{4}\bar{N}_{k}^{4}}$
$\displaystyle\leq\sum_{k}\|\mu_{k}\|_{4}^{4}$

###  B.1 Proof of Theorem 3.1

By the martingale central limit theorem (see e.g. Hall and Heyde (2014)), we
have that $T/\sqrt{\mathrm{Var}(T)}\Rightarrow N(0,1)$ if the following
conditions are satisfied:

$\displaystyle\sum_{(\ell,s)}\mathrm{Var}\big{(}\widetilde{\mathcal{M}}_{\ell,s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{)}$
$\displaystyle\stackrel{{\scriptstyle\mathbb{P}}}{{\to}}1$ (B.8)
$\displaystyle\sum_{(\ell,s)}\mathbb{E}\big{[}\widetilde{\mathcal{M}}_{\ell,s}^{2}{\bf
1}_{|\widetilde{\mathcal{M}}_{\ell,s}|>\varepsilon}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{]}$
$\displaystyle\stackrel{{\scriptstyle\mathbb{P}}}{{\to}}0,\quad\text{ for any
}\varepsilon>0.$ (B.9)

It is known that (B.9), which is a Lindeberg-type condition, is implied by the
Lyapunov-type condition

$\displaystyle\sum_{(\ell,s)}\mathbb{E}\widetilde{\mathcal{M}}_{\ell,s}^{4}=o(1).$
(B.10)

See e.g. Jin et al. (2018).

Since (3.1) holds,

$\displaystyle\mathrm{Var}(T)\gtrsim\Theta=\Theta_{n2}+\Theta_{n3}+\Theta_{n4}\gtrsim
K\|\mu\|^{2}.$ (B.11)

Recall that

$\widetilde{\mathcal{M}}_{\ell,s}=\frac{\mathcal{M}_{\ell,s}}{\mathrm{Var}(T)}=\frac{D_{\ell,s}+E_{\ell,s}}{\mathrm{Var}(T)},$

Note that (B.8) holds if

$\displaystyle\mathbb{E}\bigg{[}\mathrm{Var}\big{(}\widetilde{\mathcal{M}}_{\ell,s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{)}\bigg{]}$
$\displaystyle\to 1,\,\,\text{ and}$ (B.12)
$\displaystyle\mathrm{Var}\bigg{(}\mathrm{Var}\big{(}\widetilde{\mathcal{M}}_{\ell,s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{)}\bigg{)}$
$\displaystyle\to 0.$ (B.13)

Recall that (B.12) holds by Lemma B.1.

Next note that

$\displaystyle\mathbb{E}(D_{\ell,s}E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})=0,$

by inspection of the expressions for $D_{\ell,s}$ and $E_{\ell,s}$ in (B.2)
and (B.4). Therefore

$\displaystyle\mathrm{Var}(\mathcal{M}_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})=\mathrm{Var}(D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})+\mathrm{Var}(E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)}).$

Hence by (B.11); Lemmas B.2, B.4 , and B.6; and the assumption (3.4), under
the null hypothesis, we have

$\displaystyle\mathrm{Var}\bigg{(}\mathrm{Var}\big{(}\widetilde{\mathcal{M}}_{\ell,s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{)}\bigg{)}$
$\displaystyle\leq\frac{1}{\mathrm{Var}(T)^{2}}\bigg{[}\mathrm{Var}\bigg{(}\mathrm{Var}\big{(}D_{\ell,s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{)}\bigg{)}+\mathrm{Var}\bigg{(}\mathrm{Var}\big{(}E_{\ell,s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{)}\bigg{)}\bigg{]}$
$\displaystyle\lesssim\frac{1}{K^{2}\|\mu\|^{4}}\bigg{[}\big{(}\sum_{k}\frac{1}{n_{k}\bar{N}_{k}}\big{)}\|\mu\|_{3}^{3}+K\|\mu\|_{4}^{4}\big{)}\|\mu\|^{2}\bigg{]}=o(1).$

This proves (B.13). Thus, (B.12) and (B.13) are established, which proves
(B.8).

Similarly, (B.10) (and thus (B.9)) holds by (B.11); Lemmas (B.3), (B.5), and
(B.6), and the assumption (3.4). Combining (B.8) and (B.9) verifies the
conditions of the martingale central limit theorem, so we conclude that
$T/\sqrt{\mathrm{Var}(T)}\Rightarrow N(0,1)$. Since
$\mathrm{Var}(T)=[1+o(1)]\Theta_{n}$ by (3.4) and Lemma A.7, the proof is
complete.

∎

We record a useful proposition that records the weaker conditions under which
$T/\sqrt{\mathrm{Var}(T)}$ is asymptotically normal.

######  Proposition B.1.

Recall that $\alpha_{n}$ is defined as

$\alpha_{n}:=\max\left\\{\sum_{k=1}^{K}\frac{\|\mu_{k}\|_{3}^{3}}{n_{k}\bar{N}_{k}},\quad\sum_{k=1}^{K}\frac{\|\mu_{k}\|^{2}}{n_{k}^{2}\bar{N}_{k}^{2}}\right\\}\bigg{/}\bigg{(}\sum_{k=1}^{K}\|\mu_{k}\|^{2}\bigg{)}^{2}$
(B.14)

in (3.2). If under the null hypothesis,

$\displaystyle\alpha_{n}=\max\left\\{\sum_{k=1}^{K}\frac{\|\mu_{k}\|_{3}^{3}}{n_{k}\bar{N}_{k}},\quad\sum_{k=1}^{K}\frac{\|\mu_{k}\|^{2}}{n_{k}^{2}\bar{N}_{k}^{2}}\right\\}\bigg{/}\bigg{(}K\|\mu\|^{2}\bigg{)}^{2}\to
0,\qquad\text{ and }\,\,\frac{\|\mu\|_{4}^{4}}{K\|\mu\|^{4}}\to 0,$ (B.15)

then $T/\sqrt{\mathrm{Var}(T)}\Rightarrow N(0,1)$.

####  B.1.1 Proof of Theorem 3.2

By our assumptions, Proposition A.1 holds and $V/\mathrm{Var}(T)\to 1$. Thus
the variance estimate $V$ is consistent under the null. Theorem 3.2 follows
immediately from Slutsky’s theorem and Theorem 3.1. ∎

###  B.2 Proof of Lemma B.1

By Lemma A.1, $S$ and $U$ are uncorrelated, and it holds that

$\displaystyle\mathrm{Var}(T)=\mathrm{Var}(S)+\mathrm{Var}(U).$ (B.16)

Next note that

$\displaystyle\mathbb{E}(D_{\ell,s}E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})=0,$

by inspection of the expressions for $D_{\ell,s}$ and $E_{\ell,s}$ in (B.2)
and (B.4). Therefore

$\displaystyle\mathrm{Var}(\mathcal{M}_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})=\mathrm{Var}(D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})+\mathrm{Var}(E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)}).$

Observe that

$\displaystyle\mathbb{E}\bigg{[}\sum_{(\ell,s)}\mathrm{Var}(E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})\bigg{]}$
$\displaystyle=\sum_{(\ell,s)}\mathbb{E}E_{\ell,s}^{2}=\sum_{(\ell,s)}\sigma_{\ell}^{2}\sum_{r,r^{\prime}\in[s-1]}\sum_{j,j^{\prime}}\mathbb{E}[Z_{\ell
jr}Z_{\ell js}Z_{\ell j^{\prime}r^{\prime}}Z_{\ell j^{\prime}s}]$
$\displaystyle=\sum_{(\ell,s)}\sigma_{\ell}^{2}\sum_{r\in[s-1]}\sum_{j,j^{\prime}}\mathbb{E}[Z_{\ell
jr}Z_{\ell j^{\prime}r}Z_{\ell js}Z_{\ell j^{\prime}s}]$
$\displaystyle=\sum_{\ell=1}^{n}\sigma_{\ell}^{2}\sum_{s\in[N_{\ell}]}\sum_{r\in[s-1]}\mathbb{E}\big{(}\sum_{j}Z_{\ell
jr}Z_{\ell js}\big{)}^{2}$ $\displaystyle=\mathrm{Var}(S).$ (B.17)

The last line is obtained noting that $S$ as defined in (B.3) is a sum of
uncorrelated terms over $(i,r,s)$.

Similarly, we have

$\displaystyle\mathbb{E}\bigg{[}\sum_{(\ell,s)}\mathrm{Var}(D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})\bigg{]}$
$\displaystyle=\mathbb{E}\bigg{[}\sum_{(\ell,s)}\mathbb{E}\big{[}D_{\ell,s}^{2}|\mathcal{F}_{\prec(\ell,s)}\big{]}\bigg{]}=\sum_{(\ell,s)}\mathbb{E}\big{[}D_{\ell,s}^{2}\big{]}$
$\displaystyle=\sum_{(\ell,s)}\sum_{i\in[\ell-1]}\sigma_{i,\ell}^{2}\mathrm{Var}\big{(}\sum_{r=1}^{N_{i}}\sum_{j}Z_{ijr}Z_{\ell
js}\big{)}$
$\displaystyle=\sum_{\ell}\sum_{i\in[\ell-1]}\sigma_{i,\ell}^{2}\mathrm{Var}\big{(}\sum_{r=1}^{N_{i}}\sum_{s=1}^{N_{\ell}}Z_{ijr}Z_{\ell
js}\big{)}$ $\displaystyle=\mathrm{Var}(U).$ (B.18)

The lemma follows by combining (B.16)–(B.18). ∎

###  B.3 Proof of Lemma B.2

Let $M_{k}=n_{k}\bar{N}_{k}$ and $M=n\bar{N}$. Define

$\displaystyle\Sigma=\frac{1}{M}\sum_{k}M_{k}\Sigma_{k}=\frac{1}{M}\sum_{\ell\in[n]}N_{\ell}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}.$ (B.19)

Our main goal is to control the conditional variance process. Define

$\displaystyle\delta_{jj^{\prime}\ell}=\mathbb{E}Z_{\ell jr}Z_{\ell
j^{\prime}r}=\begin{cases}\Omega_{\ell j}(1-\Omega_{\ell j})&\quad\text{ if
}j=j^{\prime}\\\ -\Omega_{\ell j}\Omega_{\ell j^{\prime}}&\quad\text{
else}.\end{cases}$ (B.20)

Observe that

$\displaystyle\mathrm{Var}(D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})$
$\displaystyle=\mathbb{E}[\sum_{i,i^{\prime}\in[\ell-1]}\sum_{r,r^{\prime}}\sum_{j_{1},j_{2}}\sigma_{i\ell}\sigma_{i^{\prime}\ell}Z_{ij_{1}r}Z_{\ell
j_{1}s}Z_{i^{\prime}j_{2}r^{\prime}}Z_{\ell
j_{2}s}|\mathcal{F}_{\prec(\ell,s)}]$
$\displaystyle=\sum_{i,i^{\prime}\in[\ell-1]}\sum_{r,r^{\prime}}\sum_{j_{1},j_{2}}\sigma_{i\ell}\sigma_{i^{\prime}\ell}Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}\mathbb{E}[Z_{\ell
j_{1}s}Z_{\ell j_{2}s}]$
$\displaystyle=\sum_{i,i^{\prime}\in[\ell-1]}\sum_{r,r^{\prime}}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\sum_{j_{1},j_{2}}\delta_{j_{1}j_{2}\ell}Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}$

Define

$\displaystyle\alpha_{ii^{\prime}j_{1}j_{2}}=\sum_{\ell>i^{\prime}}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\delta_{j_{1}j_{2}\ell}.$
(B.21)

Thus

$\displaystyle\sum_{(\ell,s)}\mathrm{Var}(D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})$
$\displaystyle=\sum_{\ell,s}\sum_{i,i^{\prime}\in[\ell-1]}\sum_{r=1}^{N_{i}}\sum_{r^{\prime}=1}^{N_{i^{\prime}}}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\sum_{j_{1},j_{2}}\delta_{j_{1}j_{2}\ell}\,Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}$
$\displaystyle=\sum_{i}\sum_{r=1}^{N_{i}}\sum_{r^{\prime}=1}^{N_{i}}\,\sum_{j_{1},j_{2}}\bigg{(}\sum_{\ell>i}N_{\ell}\sigma_{i\ell}^{2}\delta_{j_{1}j_{2}\ell}\,\bigg{)}Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}$
$\displaystyle\quad+2\sum_{i<i^{\prime}}\sum_{r=1}^{N_{i}}\sum_{r^{\prime}=1}^{N_{i^{\prime}}}\,\sum_{j_{1},j_{2}}\bigg{(}\sum_{\ell>i^{\prime}}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\delta_{j_{1}j_{2}\ell}\,\bigg{)}Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}$
$\displaystyle=\sum_{i}\sum_{r=1}^{N_{i}}\sum_{r^{\prime}=1}^{N_{i}}\,\sum_{j_{1},j_{2}}\alpha_{iij_{1}j_{2}}Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}$
$\displaystyle\quad+2\sum_{i<i^{\prime}}\sum_{r=1}^{N_{i}}\sum_{r^{\prime}=1}^{N_{i^{\prime}}}\,\sum_{j_{1},j_{2}}\alpha_{ii^{\prime}j_{1}j_{2}}Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}.$

Define

$\displaystyle\zeta_{iri^{\prime}r^{\prime}}=\sum_{j_{1},j_{2}}\alpha_{ii^{\prime}j_{1}j_{2}}Z_{ij_{1}r}Z_{i^{\prime}j_{2}r^{\prime}}.$
(B.22)

Then

$\displaystyle\sum_{(\ell,s)}\mathrm{Var}(D_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})$
$\displaystyle=\sum_{i}\sum_{r\in[N_{i}]}\zeta_{irir}+\bigg{(}2\sum_{i}\sum_{r<r^{\prime}\in[N_{i}]}\zeta_{irir^{\prime}}+2\sum_{i<i^{\prime}}\sum_{r=1}^{N_{i}}\sum_{r^{\prime}=1}^{N_{i^{\prime}}}\zeta_{iri^{\prime}r^{\prime}}\bigg{)}$
$\displaystyle=:V_{1}+V_{2}$

With this decomposition, Lemma B.2 follows directly from Lemmas B.7 and B.8
stated below and proved in the next remainder of this subsection.

######  Lemma B.7.

It holds that

$\displaystyle\mathrm{Var}(V_{1})\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}.$

######  Lemma B.8.

It holds that

$\displaystyle\mathrm{Var}(V_{2})$ $\displaystyle\lesssim K\|\mu\|_{4}^{4}$

∎

####  B.3.1 Statement and proof of Lemma B.9

The proofs of Lemmas B.7 and B.8 heavily rely on the following intermediate
result that bounds the coefficients $\alpha_{ii^{\prime}j_{1}j_{2}}$ in all
cases.

######  Lemma B.9.

It holds that

$\displaystyle\alpha_{ii^{\prime}j_{1}j_{2}}$
$\displaystyle\lesssim\begin{cases}\frac{1}{M_{k}}\mu_{j_{1}}&\quad\text{ if
}i,i^{\prime}\in S_{k},j_{1}=j_{2}\\\
\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}&\quad\text{
if }i,i^{\prime}\in S_{k},j_{1}\neq j_{2}\\\
\frac{1}{M}\mu_{j_{1}}&\quad\text{ if }i\in S_{k_{1}},i^{\prime}\in
S_{k_{2}},k_{1}\neq k_{2},j_{1}=j_{2}\\\
\frac{1}{M}\sum_{a=1}^{2}\Sigma_{k_{a}j_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}&\quad\text{
if }i\in S_{k_{1}},i^{\prime}\in S_{k_{2}},k_{1}\neq k_{2},j_{1}\neq
j_{2}\end{cases}$

###### Proof.

If $j_{1}=j_{2}$ and $i,i^{\prime}\in S_{k}$, we have

$\displaystyle|\alpha_{ii^{\prime}j_{1}j_{1}}|$
$\displaystyle=|\sum_{\ell>i^{\prime}}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\delta_{j_{1}j_{1}\ell}|\leq\sum_{k^{\prime}=1}^{K}\sum_{\ell\in
S_{k^{\prime}}}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\delta_{j_{1}j_{1}\ell}$
$\displaystyle\lesssim\frac{1}{M_{k}}\cdot\frac{1}{M_{k}}\sum_{\ell\in
S_{k}}N_{\ell}\Omega_{\ell
j_{1}}+\frac{1}{M}\cdot\frac{1}{M}\sum_{\ell\in[n]}N_{\ell}\Omega_{\ell
j_{1}}\lesssim\frac{1}{M_{k}}\mu_{j_{1}}+\frac{1}{M}\mu_{j_{1}}\lesssim\frac{1}{M_{k}}\mu_{j_{1}}.$

If $j_{1}\neq j_{2}$ and $i,i^{\prime}\in S_{k}$, we have

$\displaystyle|\alpha_{ii^{\prime}j_{1}j_{2}}|$
$\displaystyle=|\sum_{\ell>i^{\prime}}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\delta_{j_{1}j_{2}\ell}|\leq\sum_{\ell\in[n]}N_{\ell}|\sigma_{i\ell}\sigma_{i^{\prime}\ell}|\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}$
$\displaystyle\lesssim\frac{1}{M_{k}}\cdot\frac{1}{M_{k}}\sum_{\ell\in
S_{k}}N_{\ell}\Omega_{\ell j_{1}}\Omega_{\ell
j_{2}}+\frac{1}{M}\cdot\frac{1}{M}\sum_{\ell\in[n]}N_{\ell}\Omega_{\ell
j_{1}}\Omega_{\ell
j_{2}}\lesssim\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}.$

If $i\neq i^{\prime}$, $j_{1}=j_{2}$, and $i\in S_{k_{1}},i^{\prime}\in
S_{k_{2}}$ where $k_{1}\neq k_{2}$, we have

$\displaystyle|\alpha_{ii^{\prime}j_{1}j_{1}}|$
$\displaystyle=|\sum_{\ell>i^{\prime}}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\delta_{j_{1}j_{1}\ell}|\leq\sum_{\ell}N_{\ell}|\sigma_{i\ell}\sigma_{i^{\prime}\ell}|\Omega_{\ell
j_{1}}$
$\displaystyle\lesssim\frac{1}{M}\cdot\sum_{a=1}^{2}\frac{1}{M_{k_{a}}}\sum_{\ell\in
S_{k_{a}}}N_{\ell}\Omega_{\ell
j_{1}}+\frac{1}{M}\cdot\frac{1}{M}\sum_{\ell\in[n]}N_{\ell}\Omega_{\ell
j_{1}}=\frac{3}{M}\mu_{j_{1}}.$

If $i\neq i^{\prime}$, $j_{1}\neq j_{2}$, and $i\in S_{k_{1}},i^{\prime}\in
S_{k_{2}}$ where $k_{1}\neq k_{2}$, we have

$\displaystyle|\alpha_{ii^{\prime}j_{1}j_{2}}|$
$\displaystyle=|\sum_{\ell>i^{\prime}}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\delta_{j_{1}j_{2}\ell}|\lesssim\sum_{\ell}N_{\ell}\sigma_{i\ell}\sigma_{i^{\prime}\ell}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}$
$\displaystyle\lesssim\frac{1}{M}\cdot\sum_{a=1}^{2}\frac{1}{M_{k_{a}}}\sum_{\ell\in
S_{k_{a}}}N_{\ell}\Omega_{\ell j_{1}}\Omega_{\ell
j_{2}}+\frac{1}{M}\cdot\frac{1}{M}\sum_{\ell\in[n]}N_{\ell}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}$
$\displaystyle\leq\frac{1}{M}\sum_{a=1}^{2}\Sigma_{k_{a}j_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}.$

∎

####  B.3.2 Proof of Lemma B.7

We have

$\displaystyle\mathrm{Var}(V_{1})$
$\displaystyle=\sum_{i,r}\mathbb{E}\zeta_{irir}^{2}.$

Next by symmetry,

$\displaystyle\mathbb{E}\zeta_{irir}^{2}$
$\displaystyle=\sum_{j_{1},j_{2},j_{3},j_{4}}\alpha_{iij_{1}j_{2}}\alpha_{iij_{3}j_{4}}\,\mathbb{E}Z_{ij_{1}r}Z_{ij_{3}r}Z_{ij_{2}r}Z_{ij_{4}r}$
$\displaystyle\lesssim\sum_{j_{1}}\alpha_{iij_{1}j_{1}}^{2}\Omega_{ij_{1}}+\sum_{j_{1}\neq
j_{4}}\alpha_{iij_{1}j_{1}}\alpha_{iij_{1}j_{4}}\,\Omega_{ij_{1}}\Omega_{ij_{4}}$
$\displaystyle\quad+\sum_{j_{1}\neq
j_{3}}\alpha_{iij_{1}j_{1}}\alpha_{iij_{3}j_{3}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}+\sum_{j_{1}\neq
j_{2}}\alpha_{iij_{1}j_{2}}^{2}\,\Omega_{ij_{1}}\Omega_{ij_{2}}$
$\displaystyle\quad+\sum_{j_{1},j_{3},j_{4}(dist.)}\alpha_{iij_{1}j_{1}}\alpha_{iij_{3}j_{4}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{ij_{4}}+\sum_{j_{1},j_{2},j_{4}(dist.)}\alpha_{iij_{1}j_{2}}\alpha_{iij_{1}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{4}}$
$\displaystyle\quad+\sum_{j_{1},j_{2},j_{3},j_{4}(dist.)}\alpha_{iij_{1}j_{2}}\alpha_{iij_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{3}}\Omega_{ij_{4}}=:\sum_{a=1}^{7}B_{a,i,r}$

Thus

$\displaystyle\mathrm{Var}(V_{1})$
$\displaystyle\lesssim\sum_{a}\bigg{(}\underbrace{\sum_{i,r}B_{a,i,r}}_{=:B_{a}}\bigg{)}.$

We analyze $B_{1}$– $B_{7}$ separately, bounding the
$\alpha_{ii^{\prime}j_{r}j_{s}}$ coefficients using Lemma B.9.

For $B_{1}$,

$\displaystyle B_{1}$
$\displaystyle\lesssim\sum_{i,r}\sum_{j_{1}}\alpha_{iij_{1}j_{2}}^{2}\Omega_{ij_{1}}\lesssim\sum_{k=1}^{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1}}(\frac{1}{M_{k}}\mu_{j_{1}})^{2}\Omega_{ij_{1}}$
$\displaystyle\lesssim\sum_{k}\sum_{j_{1}}(\frac{1}{M_{k}}\mu_{j_{1}})^{2}M_{k}\mu_{j_{1}}\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}.$
(B.23)

For $B_{2}$,

$\displaystyle B_{2}$ $\displaystyle\lesssim\sum_{i,r}\sum_{j_{1}\neq
j_{4}}\alpha_{iij_{1}j_{1}}\alpha_{iij_{1}j_{4}}\,\Omega_{ij_{1}}\Omega_{ij_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1}\neq
j_{4}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{4}}+\frac{1}{M}\Sigma_{j_{1}j_{4}}\big{)}\cdot\Omega_{ij_{1}}\Omega_{ij_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{j_{1}\neq
j_{4}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{4}}+\frac{1}{M}\Sigma_{j_{1}j_{4}}\big{)}\cdot
M_{k}\Sigma_{kj_{1}j_{4}}$
$\displaystyle\lesssim\sum_{k}\frac{1}{M_{k}}\sum_{j_{1}\neq
j_{4}}\Sigma_{kj_{1}j_{4}}^{2}\mu_{j_{1}}+\sum_{k}\frac{1}{M}\sum_{j_{1}\neq
j_{4}}\Sigma_{kj_{1}j_{4}}\Sigma_{j_{1}j_{4}}\mu_{j_{1}}$
$\displaystyle\lesssim\sum_{k}\frac{\mathbf{1}^{\prime}\Sigma_{k}^{\circ
2}\mu}{M_{k}}+\sum_{k}\frac{\mathbf{1}^{\prime}(\Sigma_{k}\circ\Sigma)\mu}{M}=\sum_{k}\frac{\mathbf{1}^{\prime}\Sigma_{k}^{\circ
2}\mu}{M_{k}}$

Next,

$\displaystyle\sum_{j_{1}\neq j_{4}}\Sigma_{kj_{1}j_{4}}^{2}\mu_{j_{1}}$
$\displaystyle=\sum_{j_{1}\neq j_{4}}\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{ij_{4}}\Omega_{i^{\prime}j_{4}}\cdot\mu_{j_{1}}$
$\displaystyle\leq\sum_{j_{1}}\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\mu_{j_{1}}\cdot\big{(}\sum_{j_{4}}\Omega_{ij_{4}}\Omega_{i^{\prime}j_{4}}\big{)}$
$\displaystyle\leq\sum_{j_{1}}\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\cdot\mu_{j_{1}}$
$\displaystyle\leq\sum_{j_{1}}\mu_{j_{1}}^{3}=\|\mu\|_{3}^{3},$ (B.24)

and similarly

$\displaystyle\sum_{j_{1}\neq
j_{4}}\Sigma_{kj_{1}j_{4}}\Sigma_{j_{1}j_{4}}\mu_{j_{1}}$
$\displaystyle=\sum_{j_{1}\neq j_{4}}\frac{1}{M_{k}M}\sum_{i\in
S_{k},i^{\prime}\in[n]}N_{i}N_{i^{\prime}}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{ij_{4}}\Omega_{i^{\prime}j_{4}}\cdot\mu_{j_{1}}$
$\displaystyle\leq\sum_{j_{1}}\frac{1}{M_{k}M}\sum_{i\in
S_{k},i^{\prime}\in[n]}N_{i}N_{i^{\prime}}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\mu_{j_{1}}$
$\displaystyle=\sum_{j_{1}}\mu_{j_{1}}^{3}=\|\mu\|_{3}^{3}.$

Thus

$\displaystyle
B_{2}\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}.$ (B.25)

For $B_{3}$,

$\displaystyle B_{3}$ $\displaystyle\lesssim\sum_{i,r}\sum_{j_{1}\neq
j_{3}}\alpha_{iij_{1}j_{1}}\alpha_{iij_{3}j_{3}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}$
$\displaystyle\lesssim\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1}\neq
j_{3}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\frac{1}{M_{k}}\mu_{j_{3}}\cdot\Omega_{ij_{1}}\Omega_{ij_{3}}$
$\displaystyle\lesssim\sum_{k}\sum_{j_{1}\neq
j_{3}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\frac{1}{M_{k}}\mu_{j_{3}}\cdot
M_{k}\Sigma_{kj_{1}j_{3}}\lesssim\sum_{k}\frac{\mu^{\prime}\Sigma_{k}\mu}{M_{k}}.$

We have by Cauchy-Schwarz,

$\displaystyle\mu^{\prime}\Sigma_{k}\mu$
$\displaystyle=\frac{1}{M_{k}}\sum_{i\in
S_{k}}N_{i}\mu^{\prime}\Omega_{i}\Omega_{i^{\prime}}^{\prime}\mu$
$\displaystyle=\frac{1}{M_{k}}\sum_{i\in
S_{k}}N_{i}\big{(}\sum_{j}\mu_{j}\Omega_{ij}\big{)}^{2}$
$\displaystyle\leq\frac{1}{M_{k}}\sum_{i\in
S_{k}}N_{i}\big{(}\sum_{j}\Omega_{ij}\big{)}\big{(}\sum_{j}\mu_{j}^{2}\Omega_{ij}\big{)}$
$\displaystyle=\sum_{j}\mu_{j}^{3}=\|\mu\|_{3}^{3}.$ (B.26)

Thus

$\displaystyle
B_{3}\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}$ (B.27)

For $B_{4}$,

$\displaystyle B_{4}$ $\displaystyle\lesssim\sum_{i,r}\sum_{j_{1}\neq
j_{2}}\alpha_{iij_{1}j_{2}}^{2}\,\Omega_{ij_{1}}\Omega_{ij_{2}}\lesssim\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1}\neq
j_{2}}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}^{2}\,\Omega_{ij_{1}}\Omega_{ij_{2}}$
$\displaystyle\lesssim\sum_{k}\sum_{j_{1}\neq
j_{2}}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}^{2}\,\cdot
M_{k}\Sigma_{kj_{1}j_{2}}\lesssim\sum_{k}\frac{{\bf{1}}^{\prime}(\Sigma_{k}^{\circ
3}){\bf{1}}}{M_{k}}+\sum_{k}\frac{M_{k}}{M^{2}}{\bf{1}}^{\prime}(\Sigma_{k}\circ\Sigma^{\circ
2}){\bf{1}}$
$\displaystyle\lesssim\big{(}\sum_{k}\frac{{\bf{1}}^{\prime}(\Sigma_{k}^{\circ
3}){\bf{1}}}{M_{k}}\big{)}+\frac{1}{M}\mathbf{1}^{\prime}(\Sigma^{\circ
3})\mathbf{1}.$

First,

$\displaystyle{\bf{1}}^{\prime}(\Sigma_{k}^{\circ 3}){\bf{1}}$
$\displaystyle=\frac{1}{M_{k}^{3}}\sum_{i_{1},i_{2},i_{3}\in
S_{k}}N_{i_{1}}N_{i_{2}}N_{i_{3}}\big{(}\sum_{j}\Omega_{i_{1}j}\Omega_{i_{2}j}\Omega_{i_{3}j}\big{)}^{2}$
$\displaystyle\leq\frac{1}{M_{k}^{3}}\sum_{i_{1},i_{2},i_{3}\in
S_{k}}N_{i_{1}}N_{i_{2}}N_{i_{3}}\cdot\sum_{j}\Omega_{i_{1}j}\Omega_{i_{2}j}\Omega_{i_{3}j}=\sum_{j}\mu_{j}^{3}=\|\mu\|_{3}^{3},$

and similarly,

$\displaystyle{\bf{1}}^{\prime}(\Sigma^{\circ
3}){\bf{1}}=\frac{1}{M^{3}}\sum_{i_{1},i_{2},i_{3}\in[n]}N_{i_{1}}N_{i_{2}}N_{i_{3}}\big{(}\sum_{j}\Omega_{i_{1}j}\Omega_{i_{2}j}\Omega_{i_{3}j}\big{)}^{2}\leq\|\mu\|_{3}^{3}.$

Thus

$\displaystyle
B_{4}\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}$ (B.28)

For $B_{5}$,

$\displaystyle B_{5}$
$\displaystyle\lesssim\sum_{i,r}\sum_{j_{1},j_{3},j_{4}(dist.)}\alpha_{iij_{1}j_{1}}\alpha_{iij_{3}j_{4}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{ij_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i\in
S_{k}}N_{i}\sum_{j_{1},j_{3},j_{4}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot(\frac{1}{M_{k}}\Sigma_{kj_{3}j_{4}}+\frac{1}{M}\Sigma_{j_{3}j_{4}})\cdot\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{ij_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i\in
S_{k}}\sum_{j_{1},j_{3},j_{4}}\frac{N_{i}\mu_{j_{1}}\Sigma_{kj_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{ij_{4}}}{M_{k}^{2}}+\sum_{k}\sum_{i\in
S_{k}}\sum_{j_{1},j_{3},j_{4}}\frac{N_{i}\mu_{j_{1}}\Sigma_{j_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{ij_{4}}}{M_{k}M}$
$\displaystyle=:B_{51}+B_{52}.$

We have

$\displaystyle B_{51}$
$\displaystyle=\sum_{k}\frac{1}{M_{k}^{3}}\sum_{i_{1},i_{2}\in
S_{k}}\sum_{j_{1},j_{3},j_{4}}N_{i_{1}}N_{i_{2}}\mu_{j_{1}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{2}j_{3}}\Omega_{i_{1}j_{4}}\Omega_{i_{2}j_{4}}$
$\displaystyle=\sum_{k}\frac{1}{M_{k}^{3}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}(\Omega_{i_{1}}^{\prime}\mu)\cdot(\Omega_{i_{1}}^{\prime}\Omega_{i_{2}})^{2}$
$\displaystyle\leq\sum_{k}\frac{1}{M_{k}^{3}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\cdot\Omega_{i_{1}}^{\prime}\mu\cdot\Omega_{i_{1}}^{\prime}\Omega_{i_{2}}$
$\displaystyle=\sum_{k}\frac{1}{M_{k}^{2}}\sum_{i_{1}}N_{i_{1}}\mu^{\prime}\Omega_{i_{1}}\Omega_{i_{1}}^{\prime}\mu=\frac{1}{M_{k}}\mu^{\prime}\Sigma_{k}\mu\leq\sum_{k}\frac{1}{M_{k}}\|\mu\|_{3}^{3}.$
(B.29)

In the last line we apply (B.26). Similarly,

$\displaystyle B_{52}$
$\displaystyle=\sum_{k}\frac{1}{M_{k}M^{2}}\sum_{i_{1}\in
S_{k},i_{2}\in[n]}\sum_{j_{1},j_{3},j_{4}}N_{i_{1}}N_{i_{2}}\mu_{j_{1}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{2}j_{3}}\Omega_{i_{1}j_{4}}\Omega_{i_{2}j_{4}}$
$\displaystyle\leq\sum_{k}\frac{1}{M_{k}M^{2}}\sum_{i_{1}\in
S_{k},i_{2}\in[n]}N_{i_{1}}N_{i_{2}}\cdot\Omega_{i_{1}}^{\prime}\mu\cdot\Omega_{i_{1}}^{\prime}\Omega_{i_{2}}$
$\displaystyle\leq\sum_{k}\frac{1}{M_{k}M}\sum_{i_{1}\in
S_{k}}N_{i_{1}}\mu^{\prime}\Omega_{i_{1}}\Omega_{i_{1}}^{\prime}\mu\leq\sum_{k}\frac{1}{M}\|\mu\|_{3}^{3}.$
(B.30)

Thus

$\displaystyle
B_{5}\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}.$ (B.31)

For $B_{6}$,

$\displaystyle B_{6}$ $\displaystyle\lesssim\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{4}(dist.)}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{4}}+\frac{1}{M}\Sigma_{j_{1}j_{4}}\big{)}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{4}}\frac{\Sigma_{kj_{1}j_{2}}^{2}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{4}}}{M_{k}^{2}}+2\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{4}}\frac{\Sigma_{kj_{1}j_{2}}\Sigma_{j_{1}j_{2}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{4}}}{M_{k}M}$
$\displaystyle\quad+\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{4}}\frac{\Sigma_{j_{1}j_{2}}^{2}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{4}}}{M^{2}}=:B_{61}+B_{62}+B_{63}.$

First,

$\displaystyle B_{61}$ $\displaystyle\leq\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{4}}\frac{\Sigma_{kj_{1}j_{2}}^{2}\Omega_{ij_{1}}}{M_{k}^{2}}=\sum_{k}\frac{1}{M_{k}}\mathbf{1}^{\prime}\Sigma_{k}^{\circ
2}\mu\leq\sum_{k}\frac{1}{M_{k}}\|\mu\|_{3}^{3},$

where we applied (B.24). Similarly,

$\displaystyle B_{62}$
$\displaystyle\lesssim\sum_{k}\frac{1}{M_{k}}\|\mu\|_{3}^{3},\text{ and}$
$\displaystyle B_{63}$
$\displaystyle\lesssim\sum_{k}\frac{1}{M_{k}}\|\mu\|_{3}^{3}.$

Thus

$\displaystyle
B_{6}\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}.$ (B.32)

For $B_{7}$, we have

$\displaystyle B_{7}$
$\displaystyle\lesssim\sum_{j_{1},j_{2},j_{3},j_{4}(dist.)}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{3}j_{4}}+\frac{1}{M}\Sigma_{j_{3}j_{4}}\big{)}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{3}}\Omega_{ij_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{3},j_{4}}\frac{\Sigma_{kj_{1}j_{2}}\Sigma_{kj_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{3}}\Omega_{ij_{4}}}{M_{k}^{2}}$
$\displaystyle\quad+2\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{3},j_{4}}\frac{\Sigma_{kj_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{3}}\Omega_{ij_{4}}}{M_{k}M}$
$\displaystyle\quad+\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{3},j_{4}}\frac{\Sigma_{j_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{3}}\Omega_{ij_{4}}}{M^{2}}=:B_{71}+B_{72}+B_{73}.$

Note that

$\displaystyle\Sigma_{kj_{1}j_{2}}$ $\displaystyle=\frac{1}{M_{k}}\sum_{i\in
S_{k}}N_{i}\Omega_{ij_{1}}\Omega_{ij_{2}}\leq\frac{1}{M_{k}}\sum_{i\in
S_{k}}N_{i}\Omega_{ij_{1}}=\mu_{j_{1}},\text{ and}$
$\displaystyle\Sigma_{j_{1}j_{2}}$
$\displaystyle=\frac{1}{M}\sum_{i\in[n]}N_{i}\Omega_{ij_{1}}\Omega_{ij_{2}}\leq\frac{1}{M}\sum_{i\in[n]}N_{i}\Omega_{ij_{1}}=\mu_{j_{1}}.$
(B.33)

Thus

$\displaystyle B_{71}$ $\displaystyle\leq\sum_{k}\sum_{i\in
S_{k}}\sum_{r\in[N_{i}]}\sum_{j_{1},j_{2},j_{3},j_{4}}\frac{\mu_{j_{1}}\Sigma_{kj_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{ij_{3}}\Omega_{ij_{4}}}{M_{k}^{2}}$
$\displaystyle\leq\sum_{k}\sum_{i\in
S_{k}}\sum_{j_{1},j_{3},j_{4}}\frac{N_{i}\mu_{j_{1}}\Sigma_{kj_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{ij_{4}}}{M_{k}^{2}}\leq\sum_{k}\frac{1}{M_{k}}\|\mu\|_{3}^{3}$

where we applied (B.29). Similarly,

$\displaystyle B_{72}$
$\displaystyle\lesssim\sum_{k}\frac{1}{M_{k}}\|\mu\|_{3}^{3},\text{ and}$
$\displaystyle B_{73}$
$\displaystyle\lesssim\sum_{k}\frac{1}{M_{k}}\|\mu\|_{3}^{3}.$

Thus

$\displaystyle
B_{7}\lesssim\big{(}\sum_{k}\frac{1}{M_{k}}\big{)}\|\mu\|_{3}^{3}.$ (B.34)

Combining the results for $B_{1}$–$B_{7}$ concludes the proof.

∎

####  B.3.3 Proof of Lemma B.8

We have

$\displaystyle\mathrm{Var}(V_{2})\lesssim
4\sum_{(i,r)\neq(i^{\prime},r^{\prime})}\mathbb{E}\zeta_{irir^{\prime}}^{2},$

where $r\in[N_{i}]$ and $r\in[N_{i^{\prime}}]$ in the summation above.

By symmetry, if $(i,r)\neq(i^{\prime},r^{\prime})$,

$\displaystyle\mathbb{E}$
$\displaystyle\zeta_{iri^{\prime}r^{\prime}}^{2}=\sum_{j_{1},j_{2},j_{3},j_{4}}\alpha_{ii^{\prime}j_{1}j_{2}}\alpha_{ii^{\prime}j_{3}j_{4}}\,\mathbb{E}Z_{ij_{1}r}Z_{ij_{3}r}\,\mathbb{E}Z_{i^{\prime}j_{2}r^{\prime}}Z_{i^{\prime}j_{4}r^{\prime}}$
$\displaystyle\lesssim\sum_{j_{1}}\alpha_{ii^{\prime}j_{1}j_{1}}^{2}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}+\sum_{j_{1}\neq
j_{4}}\alpha_{ii^{\prime}j_{1}j_{1}}\alpha_{ii^{\prime}j_{1}j_{4}}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+\sum_{j_{1}\neq
j_{3}}\alpha_{ii^{\prime}j_{1}j_{1}}\alpha_{ii^{\prime}j_{3}j_{3}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{3}}+\sum_{j_{1}\neq
j_{2}}\alpha_{ii^{\prime}j_{1}j_{2}}^{2}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}$
$\displaystyle\quad+\sum_{j_{1},j_{3},j_{4}(dist.)}\alpha_{ii^{\prime}j_{1}j_{1}}\alpha_{ii^{\prime}j_{3}j_{4}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}+\sum_{j_{1},j_{2},j_{4}(dist.)}\alpha_{ii^{\prime}j_{1}j_{2}}\alpha_{ii^{\prime}j_{1}j_{4}}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+\sum_{j_{1},j_{2},j_{3},j_{4}(dist.)}\alpha_{ii^{\prime}j_{1}j_{2}}\alpha_{ii^{\prime}j_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}=:\sum_{a}^{7}C_{a,i,r}.$
(B.35)

Thus

$\displaystyle\mathrm{Var}(V_{2})\lesssim\sum_{a=1}^{7}\sum_{(i,r)\neq(i^{\prime},r^{\prime})}C_{a,i,r}\lesssim\sum_{a=1}^{7}\underbrace{\sum_{i,i^{\prime}}N_{i}N_{i^{\prime}}C_{a,i,r}}_{=:C_{a}}.$

Next we analyze $C_{1},\ldots,C_{7}$, bounding the
$\alpha_{ii^{\prime}j_{r}j_{s}}$ coefficients using Lemma B.9.

For $C_{1}$,

$\displaystyle C_{1}$ $\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}\sum_{j_{1}}N_{i}N_{i^{\prime}}\alpha_{ii^{\prime}j_{1}j_{1}}^{2}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}+\sum_{k\neq
k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}\sum_{j_{1}}N_{i}N_{i^{\prime}}\alpha_{ii^{\prime}j_{1}j_{1}}^{2}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}\sum_{j_{1}}N_{i}N_{i^{\prime}}(\frac{1}{M_{k}}\mu_{j_{1}})^{2}\Omega_{ij_{1}}\Omega_{ij_{1}}+\sum_{k\neq
k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}\sum_{j_{1}}(\frac{1}{M}\mu_{j_{1}})^{2}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}$
$\displaystyle\lesssim\sum_{k}\sum_{j_{1}}\mu_{j_{1}}^{4}+\sum_{k\neq
k^{\prime}}\sum_{j_{1}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\mu_{j_{1}}^{4}\lesssim
K\|\mu\|_{4}^{4}.$ (B.36)

For $C_{2}$,

$\displaystyle C_{2}$ $\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{4}}\alpha_{ii^{\prime}j_{1}j_{1}}\alpha_{ii^{\prime}j_{1}j_{4}}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+\sum_{k\neq k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{4}}\alpha_{ii^{\prime}j_{1}j_{1}}\alpha_{ii^{\prime}j_{1}j_{4}}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{4}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{4}}+\frac{1}{M}\Sigma_{j_{1}j_{4}}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+\sum_{k\neq k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{4}}\frac{1}{M}\mu_{j_{1}}\cdot\big{(}\frac{1}{M}\sum_{a\in\\{k,k^{\prime}\\}}\Sigma_{aj_{1}j_{4}}+\frac{1}{M}\Sigma_{j_{1}j_{4}}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{4}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\big{(}\frac{1}{M_{k}}\mu_{j_{1}}+\frac{1}{M}\mu_{j_{1}}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+\sum_{k\neq k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{4}}\frac{1}{M}\mu_{j_{1}}\cdot\big{(}\frac{2}{M}\mu_{j_{1}}+\frac{1}{M}\mu_{j_{1}}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{j_{1}}\big{(}\mu_{j_{1}}^{4}+\frac{M_{k}}{M}\mu_{j_{1}}^{4}\big{)}+\sum_{k\neq
k^{\prime}}\sum_{j_{1}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\mu_{j_{1}}^{4}\lesssim
K\|\mu\|_{4}^{4}.$ (B.37)

where we applied (B.33).

For $C_{3}$,

$\displaystyle C_{3}$
$\displaystyle\lesssim\bigg{(}\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}+\sum_{k\neq k^{\prime}}\sum_{i\in
S_{k},i^{\prime}\in S_{k^{\prime}}}N_{i}N_{i^{\prime}}\bigg{)}\sum_{j_{1}\neq
j_{3}}\alpha_{ii^{\prime}j_{1}j_{1}}\alpha_{ii^{\prime}j_{3}j_{3}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{3}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{3}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\frac{1}{M_{k}}\mu_{j_{3}}\cdot\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{3}}$
$\displaystyle\quad+\sum_{k\neq k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{3}}\frac{1}{M}\mu_{j_{1}}\cdot\frac{1}{M}\mu_{j_{3}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{3}}$
$\displaystyle=\sum_{k}\sum_{j_{1}\neq
j_{3}}\mu_{j_{1}}\mu_{j_{3}}\Sigma_{kj_{1}j_{3}}^{2}+\sum_{k\neq
k^{\prime}}\sum_{j_{1}\neq
j_{3}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\mu_{j_{1}}\mu_{j_{3}}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{1}j_{3}}$
$\displaystyle\leq\big{(}\sum_{k}\mu^{\prime}\Sigma_{k}^{\circ
2}\mu\big{)}+\mu^{\prime}\Sigma^{\circ 2}\mu.$

First, by Cauchy–Schwarz,

$\displaystyle\mu^{\prime}\Sigma_{k}^{\circ 2}\mu$
$\displaystyle=\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\big{(}\sum_{j}\mu_{j}\Omega_{ij}\Omega_{i^{\prime}j}\big{)}^{2}$
$\displaystyle=\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\big{(}\sum_{j}\Omega_{ij}\Omega_{i^{\prime}j}\big{)}\sum_{j}\mu_{j}^{2}\Omega_{ij}\Omega_{i^{\prime}j}$
$\displaystyle\leq\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j}\mu_{j}^{2}\Omega_{ij}\Omega_{i^{\prime}j}=\sum_{j}\mu_{j}^{4}=\|\mu\|_{4}^{4}.$
(B.38)

Similarly

$\displaystyle\mu^{\prime}\Sigma^{\circ 2}\mu\lesssim\|\mu\|_{4}^{4}.$ (B.39)

Hence

$\displaystyle C_{3}\lesssim K\|\mu\|_{4}^{4}.$ (B.40)

For $C_{4}$,

$\displaystyle C_{4}$
$\displaystyle\lesssim\bigg{(}\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}+\sum_{k\neq k^{\prime}}\sum_{i\in
S_{k},i^{\prime}\in S_{k^{\prime}}}N_{i}N_{i^{\prime}}\bigg{)}\sum_{j_{1}\neq
j_{2}}\alpha_{ii^{\prime}j_{1}j_{2}}^{2}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{2}}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}^{2}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}$
$\displaystyle\quad+\sum_{k\neq k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{2}}\big{(}\frac{1}{M}\sum_{a\in\\{k,k^{\prime}\\}}^{2}\Sigma_{aj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}^{2}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{2}}\big{(}\frac{1}{M_{k}^{2}}\Sigma_{kj_{1}j_{2}}^{2}+\frac{1}{M^{2}}\Sigma_{j_{1}j_{2}}^{2}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}$
$\displaystyle\quad+\sum_{k\neq k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{2}}\big{(}\frac{1}{M^{2}}\sum_{a\in\\{k,k^{\prime}\\}}^{2}\Sigma_{aj_{1}j_{2}}^{2}+\frac{1}{M^{2}}\Sigma_{j_{1}j_{2}}^{2}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}=:C_{41}+C_{42}$

First,

$\displaystyle C_{41}$ $\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{2}}\frac{1}{M_{k}^{2}}\Sigma_{kj_{1}j_{2}}^{2}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}+\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1}\neq
j_{2}}\frac{1}{M^{2}}\Sigma_{j_{1}j_{2}}^{2}\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}$
$\displaystyle\lesssim\sum_{k}\sum_{j_{1}\neq
j_{2}}\Sigma_{kj_{1}j_{2}}^{2}\mu_{j_{1}}\mu_{j_{2}}+\sum_{k}\sum_{j_{1}\neq
j_{2}}\frac{M_{k}^{2}}{M^{2}}\Sigma_{j_{1}j_{2}}^{2}\mu_{j_{1}}\mu_{j_{2}}\leq\sum_{k}\mu^{\prime}\Sigma_{k}^{\circ
2}\mu+\sum_{k}\frac{M_{k}^{2}}{M^{2}}\mu^{\prime}\Sigma^{\circ 2}\mu.$

Similarly,

$\displaystyle C_{42}$ $\displaystyle\lesssim\sum_{k\neq
k^{\prime}}\sum_{j_{1}\neq
j_{2}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\Sigma_{kj_{1}j_{2}}^{2}\mu_{j_{1}}\mu_{j_{2}}+\sum_{k\neq
k^{\prime}}\sum_{j_{1}\neq
j_{2}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\Sigma_{j_{1}j_{2}}^{2}\mu_{j_{1}}\mu_{j_{2}}$
$\displaystyle\lesssim\sum_{k\neq
k^{\prime}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\big{(}\mu^{\prime}\Sigma_{k}^{\circ
2}\mu+\mu^{\prime}\Sigma^{\circ 2}\mu\big{)}$

Combining the previous two displays and applying (B.38) and (B.39), we have

$\displaystyle C_{4}$ $\displaystyle\lesssim K\|\mu\|_{4}^{4}.$ (B.41)

For $C_{5}$,

$\displaystyle C_{5}$
$\displaystyle\lesssim\bigg{(}\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}+\sum_{k\neq k^{\prime}}\sum_{i\in
S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\bigg{)}\sum_{j_{1},j_{3},j_{4}(dist.)}\alpha_{ii^{\prime}j_{1}j_{1}}\alpha_{ii^{\prime}j_{3}j_{4}}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1},j_{3},j_{4}}\frac{1}{M_{k}}\mu_{j_{1}}\cdot\big{(}\frac{1}{M_{k}}\Sigma_{kj_{3}j_{4}}+\frac{1}{M}\Sigma_{j_{3}j_{4}}\big{)}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+\sum_{k\neq k^{\prime}}\sum_{i\in S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\sum_{j_{1},j_{3},j_{4}}\frac{1}{M}\mu_{j_{1}}\big{(}\frac{1}{M}\sum_{a\in\\{k,k^{\prime}\\}}^{2}\Sigma_{aj_{3}j_{4}}+\frac{1}{M}\Sigma_{j_{3}j_{4}}\big{)}\,\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle=\sum_{k}\sum_{j_{1},j_{3},j_{4}}\mu_{j_{1}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{kj_{1}j_{4}}+\sum_{k}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}}{M}\mu_{j_{1}}\Sigma_{j_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{kj_{1}j_{4}}$
$\displaystyle\quad+2\sum_{k\neq
k^{\prime}}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\mu_{j_{1}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{1}j_{4}}+\sum_{k\neq
k^{\prime}}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\mu_{j_{1}}\Sigma_{j_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{1}j_{4}}$
$\displaystyle=C_{51}+C_{52}+2C_{53}+C_{54}$

For $C_{51}$, we have

$\displaystyle C_{51}$
$\displaystyle=\sum_{k}\frac{1}{M_{k}^{3}}\sum_{i_{1},i_{2},i_{3}\in
S_{k}}N_{i_{1}}N_{i_{2}}N_{i_{3}}\langle\mu\circ\Omega_{i_{1}},\Omega_{i_{2}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{3}}\rangle$
$\displaystyle=\sum_{k}\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\langle\mu\circ\Omega_{i_{1}},\Omega_{i_{2}}\rangle\cdot\langle\Omega_{i_{1}},\Sigma_{k}\Omega_{i_{2}}\rangle$
$\displaystyle\leq\sum_{k}\bigg{(}\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\langle\mu\circ\Omega_{i_{1}},\Omega_{i_{2}}\rangle^{2}\bigg{)}^{1/2}\bigg{(}\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\langle\Omega_{i_{1}},\Sigma_{k}\Omega_{i_{2}}\rangle^{2}\bigg{)}^{1/2}$
$\displaystyle=:\sum_{k}C_{511k}^{1/2}\cdot C_{512k}^{1/2}.$ (B.42)

We have by Cauchy–Schwarz that

$\displaystyle C_{511k}$
$\displaystyle=\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\big{(}\sum_{j}\mu_{j}\Omega_{i_{1}j}\Omega_{i_{2}j}\big{)}^{2}$
$\displaystyle\leq\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\big{(}\sum_{j}\mu_{j}^{2}\Omega_{i_{1}j}\Omega_{i_{2}j}\big{)}\big{(}\sum_{j}\Omega_{i_{1}j}\Omega_{i_{2}j}\big{)}\leq\|\mu\|_{4}^{4},$

and similarly

$\displaystyle C_{512k}$
$\displaystyle=\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\big{(}\sum_{j_{1},j_{2}}\Omega_{i_{1}j_{1}}\Sigma_{kj_{1}j_{2}}\Omega_{i_{2}j_{2}}\big{)}^{2}$
$\displaystyle=\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}}N_{i_{1}}N_{i_{2}}\big{(}\sum_{j_{1},j_{2}}\Omega_{i_{1}j_{1}}\Sigma_{kj_{1}j_{2}}^{2}\Omega_{i_{2}j_{2}}\big{)}\big{(}\sum_{j_{1},j_{2}}\Omega_{i_{1}j_{1}}\Omega_{i_{2}j_{2}}\big{)}$
$\displaystyle\leq\frac{1}{M_{k}^{2}}\sum_{i_{1},i_{2}}N_{i_{1}}N_{i_{2}}\big{(}\sum_{j_{1},j_{2}}\Omega_{i_{1}j_{1}}\Sigma_{kj_{1}j_{2}}^{2}\Omega_{i_{2}j_{2}}\big{)}=\mu^{\prime}\,\Sigma_{k}^{\circ
2}\,\mu$ (B.43)

Since by Cauchy–Schwarz,

$\displaystyle\mu^{\prime}\,\Sigma_{k}^{\circ 2}\,\mu$
$\displaystyle=\sum_{j_{1},j_{2}}\mu_{j_{1}}\mu_{j_{2}}\big{(}\frac{1}{M_{k}}\sum_{i\in
S_{k}}N_{i}\Omega_{ij_{1}}\Omega_{ij_{2}}\big{)}^{2}=\frac{1}{M_{k}^{2}}\sum_{j_{1},j_{2}}\mu_{j_{1}}\mu_{j_{2}}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\Omega_{ij_{1}}\Omega_{ij_{2}}\Omega_{i^{\prime}j_{1}}\Omega_{i^{\prime}j_{2}}$
$\displaystyle=\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}\big{(}\sum_{j}\mu_{j}\Omega_{ij}\Omega_{i^{\prime}j}\big{)}^{2}\leq\frac{1}{M_{k}^{2}}\sum_{i,i^{\prime}\in
S_{k}}\sum_{j}\mu_{j}^{2}\Omega_{ij}\Omega_{i^{\prime}j}\leq\|\mu\|_{4}^{4}$
(B.44)

we have in total $C_{512k}\lesssim K\|\mu\|_{4}^{4}$. Combining the result
with the bound for $C_{511k}$ implies that

$\displaystyle C_{51}\lesssim K\|\mu\|_{4}^{4}.$

Next we study $C_{52}$ using a similar argument.

$\displaystyle C_{52}$
$\displaystyle=\sum_{k}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}}{M}\mu_{j_{1}}\Sigma_{j_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{kj_{1}j_{4}}$
$\displaystyle=\sum_{k}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}}{M}\mu_{j_{1}}\big{(}\frac{1}{M}\sum_{i_{1}\in[n]}N_{i_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}\big{)}\big{(}\frac{1}{M_{k}}\sum_{i_{2}\in
S_{k}}N_{i_{2}}\Omega_{i_{2}j_{1}}\Omega_{i_{2}j_{3}}\big{)}\big{(}\frac{1}{M_{k}}\sum_{i_{3}\in
S_{k}}N_{i_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}\big{)}$
$\displaystyle=\sum_{k}\frac{1}{M^{2}M_{k}}\sum_{j_{1},j_{2},j_{3}}\sum_{\begin{subarray}{c}i_{1}\in[n]\\\
i_{2},i_{3}\in
S_{k}\end{subarray}}N_{i_{1}}N_{i_{2}}N_{i_{3}}\langle\mu\circ\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{2}}\rangle$
$\displaystyle=\sum_{k}\frac{1}{M^{2}}\sum_{i_{2},i_{3}\in[S_{k}]}N_{i_{2}}N_{i_{3}}\langle\mu\circ\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{3}},\Sigma\Omega_{i_{2}}\rangle$
$\displaystyle\leq\sum_{k}\bigg{(}\frac{1}{M^{2}}\sum_{i_{2},i_{3}\in[S_{k}]}N_{i_{2}}N_{i_{3}}\langle\mu\circ\Omega_{i_{2}},\Omega_{i_{3}}\rangle^{2}\bigg{)}^{1/2}\bigg{(}\frac{1}{M^{2}}\sum_{i_{2},i_{3}\in[S_{k}]}N_{i_{2}}N_{i_{3}}\langle\Omega_{i_{3}},\Sigma\Omega_{i_{2}}\rangle\bigg{)}^{1/2}$
$\displaystyle=:\sum_{k}C_{521k}^{1/2}C_{522k}^{1/2}.$ (B.45)

Observe that $C_{521k}=C_{511k}$, and thus $C_{521}\lesssim\|\mu\|^{4}$ by
(B.43). With a similar argument as in (B.44) we obtain
$C_{522k}\lesssim\|\mu\|_{4}^{4}$. Hence we obtain

$\displaystyle C_{52}\leq\sum_{k}C_{521k}^{1/2}C_{522k}^{1/2}\lesssim
K\|\mu\|_{4}^{4}.$

For $C_{53}$, we have

$\displaystyle C_{53}=\sum_{k\neq
k^{\prime}}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\mu_{j_{1}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{1}j_{4}}$
$\displaystyle\leq\sum_{k}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}}{M}\mu_{j_{1}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{j_{1}j_{4}}$
$\displaystyle=\sum_{k}\sum_{j_{1},j_{3},j_{4}}\frac{M_{k}}{M}\mu_{j_{1}}\big{(}\frac{1}{M_{k}}\sum_{i_{1}\in
S_{k}}N_{i_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}\big{)}\big{(}\frac{1}{M_{k}}\sum_{i_{2}\in
S_{k}}N_{i_{2}}\Omega_{i_{2}j_{1}}\Omega_{i_{2}j_{3}}\big{)}\big{(}\frac{1}{M}\sum_{i_{3}\in[n]}N_{i_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}\big{)}$
$\displaystyle=\sum_{k}\frac{1}{M^{2}M_{k}}\sum_{\begin{subarray}{c}i_{1},i_{2}\in
S_{k}\\\
i_{3}\in[n]\end{subarray}}N_{i_{1}}N_{i_{2}}N_{i_{3}}\langle\mu\circ\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{2}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle$
$\displaystyle=\sum_{k}\frac{1}{M^{2}}\sum_{i_{2}\in
S_{k},i_{3}\in[n]}N_{i_{2}}N_{i_{3}}\langle\mu\circ\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Sigma_{k}\Omega_{i_{3}}\rangle.$
(B.46)

We then upper bound the last line using a similar strategy as in that we used
for $C_{51}$ and $C_{52}$, respectively. We omit the details and state the
final bound:

$\displaystyle C_{53}\lesssim K\|\mu\|_{4}^{4}$ (B.47)

Finally for $C_{54}$, summing over $k,k^{\prime}$ we obtain

$\displaystyle C_{54}$
$\displaystyle\leq\sum_{j_{1},j_{3},j_{4}}\mu_{j_{1}}\Sigma_{j_{3}j_{4}}\Sigma_{j_{1}j_{3}}\Sigma_{j_{1}j_{4}}=\frac{1}{M^{3}}\sum_{i_{1},i_{2},i_{3}\in[n]}N_{i_{1}}N_{i_{2}}N_{i_{3}}\langle\mu\circ\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{2}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle.$
(B.48)

We then proceed as in (B.46) to control the right-hand side. We omit the
details and state the final bound:

$\displaystyle C_{54}\lesssim K\|\mu\|_{4}^{4}.$ (B.49)

Combining the results for $C_{51},\ldots,C_{54}$, we see that

$\displaystyle C_{5}\lesssim K\|\mu\|^{4}.$

For $C_{6}$, we have

$\displaystyle C_{6}\leq\bigg{(}\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}+\sum_{k\neq k^{\prime}}\sum_{i\in
S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\bigg{)}\sum_{j_{1},j_{2},j_{4}}\alpha_{ii^{\prime}j_{1}j_{2}}\alpha_{ii^{\prime}j_{1}j_{4}}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{4}}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{4}}+\frac{1}{M}\Sigma_{j_{1}j_{4}}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle+\sum_{\begin{subarray}{c}k\neq k^{\prime}\\\ i\in
S_{k},i^{\prime}\in S_{k^{\prime}}\\\
j_{1},j_{2},j_{4}\end{subarray}}N_{i}N_{i^{\prime}}\big{(}\frac{1}{M}\sum_{a\in\\{k,k^{\prime}\\}}^{2}\Sigma_{aj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}\big{(}\frac{1}{M}\sum_{a\in\\{k,k^{\prime}\\}}^{2}\Sigma_{aj_{1}j_{4}}+\frac{1}{M}\Sigma_{j_{1}j_{4}}\big{)}\,\Omega_{ij_{1}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle=:C_{61}+C_{62}.$

For $C_{61}$, we have

$\displaystyle C_{61}$ $\displaystyle=\sum_{k}\sum_{i^{\prime}\in
S_{k}}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{4}}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}\Sigma_{kj_{1}j_{4}}\mu_{j_{1}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+2\sum_{k}\sum_{i^{\prime}\in
S_{k}}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{4}}\frac{1}{M}\Sigma_{kj_{1}j_{2}}\Sigma_{j_{1}j_{4}}\mu_{j_{1}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle+\quad\sum_{k}\sum_{i^{\prime}\in
S_{k}}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{4}}\frac{M_{k}}{M^{2}}\Sigma_{j_{1}j_{2}}\Sigma_{j_{1}j_{4}}\mu_{j_{1}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}=:C_{611}+2C_{612}+C_{613}.$

Relabeling indices, we see that

$\displaystyle C_{611}$
$\displaystyle=\sum_{k}\sum_{j_{1},j_{2},j_{4}}\mu_{j_{1}}\Sigma_{kj_{1}j_{2}}\Sigma_{kj_{1}j_{4}}\Sigma_{kj_{2}j_{4}}=C_{51}$

Hence, $C_{611}\lesssim K\|\mu\|_{4}^{4}$. Next,

$\displaystyle
C_{612}\leq\sum_{k}\frac{M_{k}}{M}\sum_{j_{1},j_{2},j_{4}}\mu_{j_{1}}\Sigma_{kj_{1}j_{2}}\Sigma_{j_{1}j_{4}}\Sigma_{kj_{2}j_{4}}\lesssim
K\|\mu\|^{4},$

where we applied (B.46). Similarly,

$\displaystyle C_{613}$
$\displaystyle=\sum_{k}\frac{M_{k}^{2}}{M^{2}}\sum_{j_{1},j_{2},j_{4}}\mu_{j_{1}}\Sigma_{j_{1}j_{2}}\Sigma_{j_{1}j_{4}}\Sigma_{kj_{2}j_{4}}\leq\sum_{j_{1},j_{2},j_{4}}\mu_{j_{1}}\Sigma_{j_{1}j_{2}}\Sigma_{j_{1}j_{4}}\Sigma_{j_{2}j_{4}}\lesssim
K\|\mu\|^{4},$

where in the final bound we apply (B.48) and (B.49). Combining the results
above for $C_{611},C_{612},C_{613}$, we obtain

$\displaystyle C_{61}\lesssim K\|\mu\|_{4}^{4}$ (B.50)

The argument for $C_{62}$ is very similar, so we omit proof and state the
final bound. We have

$\displaystyle C_{62}\lesssim K\|\mu\|_{4}.$

Thus

$\displaystyle C_{6}\lesssim K\|\mu\|_{4}^{4}$

For $C_{7}$, we have

$\displaystyle C_{7}\lesssim\bigg{(}\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}+\sum_{k\neq k^{\prime}}\sum_{i\in
S_{k},i^{\prime}\in
S_{k^{\prime}}}N_{i}N_{i^{\prime}}\bigg{)}\sum_{j_{1},j_{2},j_{3},j_{4}}\alpha_{ii^{\prime}j_{1}j_{2}}\alpha_{ii^{\prime}j_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\lesssim\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{3},j_{4}}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}\big{(}\frac{1}{M_{k}}\Sigma_{kj_{3}j_{4}}+\frac{1}{M}\Sigma_{j_{3}j_{4}}\big{)}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle+\sum_{k\neq
k^{\prime}}\sum_{\begin{subarray}{c}j_{1},j_{2},j_{3},j_{4}\\\ i\in
S_{k},i^{\prime}\in
S_{k^{\prime}}\end{subarray}}N_{i}N_{i^{\prime}}\big{(}\frac{1}{M}\sum_{a\in\\{k,k^{\prime}\\}}^{2}\Sigma_{aj_{1}j_{2}}+\frac{1}{M}\Sigma_{j_{1}j_{2}}\big{)}\big{(}\frac{1}{M}\sum_{a\in\\{k,k^{\prime}\\}}^{2}\Sigma_{aj_{3}j_{4}}+\frac{1}{M}\Sigma_{j_{3}j_{4}}\big{)}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle=:C_{71}+C_{72}$

Write

$\displaystyle C_{71}$ $\displaystyle=\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{3},j_{4}}\frac{1}{M_{k}^{2}}\Sigma_{kj_{1}j_{2}}\Sigma_{kj_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+2\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{3},j_{4}}\frac{1}{M_{k}M}\Sigma_{j_{1}j_{2}}\Sigma_{kj_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}$
$\displaystyle\quad+\sum_{k}\sum_{i,i^{\prime}\in
S_{k}}N_{i}N_{i^{\prime}}\sum_{j_{1},j_{2},j_{3},j_{4}}\frac{1}{M^{2}}\Sigma_{j_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Omega_{ij_{1}}\Omega_{ij_{3}}\Omega_{i^{\prime}j_{2}}\Omega_{i^{\prime}j_{4}}=:C_{711}+2C_{712}+C_{713}.$

For $C_{711}$, we have

$\displaystyle C_{711}$
$\displaystyle=\sum_{k}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{kj_{1}j_{2}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{kj_{2}j_{4}}$
$\displaystyle=\sum_{k}\frac{1}{M_{k}^{4}}\sum_{i_{1},i_{2},i_{3},i_{4}\in
S_{k}}N_{i_{1}}N_{i_{2}}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{4}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{4}}\rangle$
$\displaystyle=\frac{1}{M_{k}^{2}}\sum_{k}\sum_{i_{3},i_{4}}N_{i_{3}}N_{i_{4}}\big{(}\Omega_{i_{3}}^{\prime}\Sigma_{k}\Omega_{i_{4}}\big{)}^{2}=\sum_{k}\frac{1}{M_{k}^{2}}\sum_{i_{3},i_{4}}N_{i_{3}}N_{i_{4}}\big{(}\sum_{j,j^{\prime}}\Omega_{i_{3}j}^{\prime}\Sigma_{kjj^{\prime}}\Omega_{i_{4}j^{\prime}}\big{)}^{2}$
$\displaystyle\leq\sum_{k}\frac{1}{M_{k}^{2}}\sum_{i_{3},i_{4}}N_{i_{3}}N_{i_{4}}\sum_{j,j^{\prime}}\Omega_{i_{3}j}^{\prime}\Sigma_{kjj^{\prime}}^{2}\Omega_{i_{4}j^{\prime}}\leq\sum_{k}\sum_{j,j^{\prime}}\mu_{j}\Sigma_{kjj^{\prime}}^{2}\mu_{j^{\prime}}\lesssim
K\|\mu\|_{4}^{4}.$ (B.51)

In the last line we applied Cauchy–Schwarz and (B.44). For $C_{712}$, we have
similarly

$\displaystyle C_{712}$
$\displaystyle=\sum_{k}\frac{M_{k}}{M}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{j_{1}j_{2}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{kj_{2}j_{4}}$
$\displaystyle=\sum_{k}\frac{1}{M^{2}M_{k}}\sum_{\begin{subarray}{c}i_{1}\in[n]\\\
i_{2},i_{3},i_{4}\in
S_{k}\end{subarray}}N_{i_{1}}N_{i_{2}}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{4}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{4}}\rangle$
$\displaystyle=\sum_{k}\frac{M_{k}}{M^{2}}\sum_{i_{1}\in[n],i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\langle\Omega_{i_{1}},\Sigma_{k}\Omega_{i_{2}}\rangle^{2}\leq\sum_{k}\frac{M_{k}}{M^{2}}\sum_{i_{1}\in[n],i_{2}\in
S_{k}}N_{i_{1}}N_{i_{2}}\sum_{j,j^{\prime}}\Omega_{i_{1}j}\Sigma_{kjj^{\prime}}^{2}\Omega_{i_{2}j^{\prime}}$
$\displaystyle\leq\sum_{k}\frac{M_{k}^{2}}{M^{2}}\sum_{j,j^{\prime}}\mu_{j}\Sigma_{kjj^{\prime}}^{2}\mu_{j^{\prime}}\lesssim
K\|\mu\|_{4}^{4}.$ (B.52)

Next,

$\displaystyle C_{713}$
$\displaystyle=\sum_{k}\frac{M_{k}^{2}}{M^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{j_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{kj_{2}j_{4}}$
$\displaystyle=\sum_{k}\frac{1}{M^{4}}\sum_{\begin{subarray}{c}i_{1},i_{2}\in[n]\\\
i_{3},i_{4}\in
S_{k}\end{subarray}}N_{i_{1}}N_{i_{2}}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{4}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{4}}\rangle,$

and applying a similar strategy as in (B.51), (B.52) leads to the bound
$C_{713}\lesssim K\|\mu\|_{4}^{4}$. Thus

$\displaystyle C_{71}\lesssim K\|\mu\|_{4}^{4}.$

Next , by symmetry and summing over $i\in S_{k},i^{\prime}\in S_{k^{\prime}}$,
we have

$\displaystyle C_{72}$ $\displaystyle=\sum_{k\neq
k^{\prime}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}}\bigg{[}2\Sigma_{kj_{1}j_{2}}\Sigma_{kj_{3}j_{4}}+2\Sigma_{k^{\prime}j_{1}j_{2}}\Sigma_{kj_{3}j_{4}}+4\Sigma_{kj_{1}j_{2}}\Sigma_{j_{3}j_{4}}+\Sigma_{j_{1}j_{2}}\Sigma_{j_{3}j_{4}}\bigg{]}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{2}j_{4}}$
$\displaystyle=:2C_{721}+2C_{722}+4C_{723}+C_{724}$

First,

$\displaystyle C_{721}$
$\displaystyle\leq\sum_{k}\frac{M_{k}}{M}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{kj_{1}j_{2}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{j_{2}j_{4}}=C_{712}\lesssim
K\|\mu\|_{4}^{4}$

by (B.52). Next,

$\displaystyle C_{722}$ $\displaystyle=\sum_{k\neq
k^{\prime}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{k^{\prime}j_{1}j_{2}}\Sigma_{kj_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{2}j_{4}}$
$\displaystyle\leq\sum_{k,k^{\prime}}\frac{1}{M^{2}M_{k}M_{k^{\prime}}}\sum_{\begin{subarray}{c}i_{1},i_{2}\in
S_{k}\\\ i_{3},i_{4}\in
S_{k^{\prime}}\end{subarray}}N_{i_{1}}N_{i_{2}}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{4}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{4}}\rangle$
$\displaystyle=\sum_{k,k^{\prime}}\frac{M_{k}}{M^{2}M_{k^{\prime}}}\sum_{i_{3},i_{4}\in
S_{k^{\prime}}}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{3}},\Sigma_{k}\Omega_{i_{4}}\rangle^{2}\leq\sum_{k,k^{\prime}}\frac{M_{k}}{M^{2}M_{k^{\prime}}}\sum_{i_{3},i_{4}\in
S_{k^{\prime}}}N_{i_{3}}N_{i_{4}}\sum_{j,j^{\prime}}\Omega_{i_{3}j}\Sigma_{kjj^{\prime}}^{2}\Omega_{i_{4}j^{\prime}}$
$\displaystyle\leq\sum_{k,k^{\prime}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\mu^{\prime}\Sigma_{k}^{\circ
2}\mu\leq\|\mu\|_{4}^{4},$ (B.53)

where we applied Cauchy-Schwarz in the penultimate line and (B.44) in the last
line.

For $C_{723}$, we have

$\displaystyle C_{723}$ $\displaystyle=\sum_{k\neq
k^{\prime}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{kj_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{2}j_{4}}\leq\sum_{k}\frac{M_{k}}{M}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{kj_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{j_{2}j_{4}}$
$\displaystyle=\sum_{k}\frac{1}{M^{3}M_{k}}\sum_{\begin{subarray}{c}i_{1},i_{3}\in
S_{k}\\\
i_{2},i_{4}\in[n]\end{subarray}}N_{i_{1}}N_{i_{2}}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{4}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{4}}\rangle$
$\displaystyle=\sum_{k}\frac{1}{M^{2}}\sum_{i_{3}\in
S_{k},i_{4}\in[n]}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{3}},\Sigma_{k}\Omega_{i_{4}}\rangle\langle\Omega_{i_{3}},\Sigma\Omega_{i_{4}}\rangle$
$\displaystyle\leq\frac{1}{2}\sum_{k}\frac{1}{M^{2}}\sum_{i_{3}\in
S_{k},i_{4}\in[n]}N_{i_{3}}N_{i_{4}}\big{(}\langle\Omega_{i_{3}},\Sigma_{k}\Omega_{i_{4}}\rangle^{2}+\langle\Omega_{i_{3}},\Sigma\Omega_{i_{4}}\rangle^{2}\big{)}$

Using a similar technique as in (B.51)–(B.53) and applying (B.38), (B.39) we
obtain

$\displaystyle C_{723}\lesssim\|\mu\|_{4}^{4}.$

Finally, for $C_{724}$ we have

$\displaystyle C_{724}$ $\displaystyle=\sum_{k\neq
k^{\prime}}\frac{M_{k}M_{k^{\prime}}}{M^{2}}\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{j_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Sigma_{kj_{1}j_{3}}\Sigma_{k^{\prime}j_{2}j_{4}}\leq\sum_{j_{1},j_{2},j_{3},j_{4}}\Sigma_{j_{1}j_{2}}\Sigma_{j_{3}j_{4}}\Sigma_{j_{1}j_{3}}\Sigma_{j_{2}j_{4}}$
$\displaystyle=\frac{1}{M^{4}}\sum_{i_{1},i_{2},i_{3},i_{4}\in[n]}N_{i_{1}}N_{i_{2}}N_{i_{3}}N_{i_{4}}\langle\Omega_{i_{1}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{1}},\Omega_{i_{4}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{3}}\rangle\langle\Omega_{i_{2}},\Omega_{i_{4}}\rangle$

The details are very similar to (B.51)–(B.53), so we omit them and simply
state the final bound:

$\displaystyle C_{724}$ $\displaystyle\lesssim\|\mu\|_{4}^{4}$

Combining the bounds for $C_{721},C_{722},C_{723}$, and $C_{724}$ yields

$\displaystyle C_{7}\lesssim K\|\mu\|_{4}^{4}.$

Combining the bounds for $C_{1}$–$C_{7}$ proves the result.

∎

###  B.4 Proof of Lemma B.3

We have

$\displaystyle\mathbb{E}D_{\ell,s}^{4}=\mathbb{E}\bigg{[}\big{(}\sum_{i\in[\ell-1]}\sigma_{i,\ell}\sum_{r=1}^{N_{i}}\sum_{j}Z_{ijr}Z_{\ell
js}\big{)}^{4}\bigg{]}$
$\displaystyle=\sum_{i_{1},i_{2},i_{3},i_{4}\in[\ell-1]}\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\sum_{\begin{subarray}{c}r_{1},r_{2},r_{3},r_{4}\\\
j_{1},j_{2},j_{3},j_{4}\end{subarray}}\mathbb{E}\big{[}Z_{i_{1}j_{1}r_{1}}Z_{\ell
j_{1}s}Z_{i_{2}j_{2}r_{2}}Z_{\ell j_{2}s}Z_{i_{3}j_{3}r_{3}}Z_{\ell
j_{3}s}Z_{i_{4}j_{4}r_{4}}Z_{\ell j_{4}s}\big{]}$
$\displaystyle=\sum_{i_{1},i_{2},i_{3},i_{4}\in[\ell-1]}\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\sum_{\begin{subarray}{c}r_{1},r_{2},r_{3},r_{4}\\\
j_{1},j_{2},j_{3},j_{4}\end{subarray}}\mathbb{E}\big{[}Z_{i_{1}j_{1}r_{1}}Z_{i_{2}j_{2}r_{2}}Z_{i_{3}j_{3}r_{3}}Z_{i_{4}j_{4}r_{4}}\big{]}\mathbb{E}\big{[}Z_{\ell
j_{1}s}Z_{\ell j_{2}s}Z_{\ell j_{3}s}Z_{\ell j_{4}s}\big{]}$
$\displaystyle=\sum_{j_{1},j_{2},j_{3},j_{4}}\mathbb{E}[Z_{\ell j_{1}s}Z_{\ell
j_{2}s}Z_{\ell j_{3}s}Z_{\ell
j_{4}s}]\sum_{\begin{subarray}{c}i_{1},i_{2},i_{3},i_{4}\in[\ell-1]\\\
r_{1},r_{2},r_{3},r_{4}\end{subarray}}\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{i_{1}j_{1}r_{1}}Z_{i_{2}j_{2}r_{2}}Z_{i_{3}j_{3}r_{3}}Z_{i_{4}j_{4}r_{4}}]$
$\displaystyle=:\sum_{j_{1},j_{2},j_{3},j_{4}}\mathbb{E}[Z_{\ell
j_{1}s}Z_{\ell j_{2}s}Z_{\ell j_{3}s}Z_{\ell
j_{4}s}]A_{j_{1},j_{2},j_{3},j_{4}}$ (B.54)

In the summations above, $r_{t}$ ranges over $[N_{i_{t}}]$.

Observe that

$\displaystyle|\mathbb{E}[Z_{\ell j_{1}s}Z_{\ell j_{2}s}Z_{\ell j_{3}s}Z_{\ell
j_{4}s}]|\lesssim\begin{cases}\Omega_{\ell j_{1}}&\quad\text{ if
}j_{1}=j_{2}=j_{3}=j_{4}\\\ \Omega_{\ell j_{1}}\Omega_{\ell j_{4}}&\quad\text{
if }j_{1}=j_{2}=j_{3},j_{4}\neq j_{1}\\\ \Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}&\quad\text{ if }j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq j_{3}\\\ \Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}&\quad\text{ if
}j_{1}=j_{2},j_{1},j_{3},j_{4}\,\,dist.\\\ \Omega_{\ell j_{1}}\Omega_{\ell
j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}&\quad\text{ if
}j_{1},j_{2},j_{3},j_{4}\,\,dist.\\\ \end{cases}$ (B.55)

Up to permutation of the indices $j_{1},\ldots,j_{4}$, this accounts for all
possible cases.

To proceed we also bound $A_{j_{1},j_{2},j_{3},j_{4}}$ by casework on the
number of distinct $j$ indices. For brevity we define
$\omega_{t}=(i_{t},r_{t})$ and slightly abuse notation, letting
$Z_{\omega_{t},j}=Z_{i_{t}jr_{t}}$. Further let
$\mathcal{I}_{\ell}=\\{\omega=(i,r):i\in[\ell],1\leq r\leq N_{i}\\}$. Our goal
is to control

$\displaystyle
A_{j_{1},j_{2},j_{3},j_{4}}=\sum_{\omega_{1},\omega_{2},\omega_{3},\omega_{4}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j_{1}}Z_{\omega_{2}j_{2}}Z_{\omega_{3}j_{3}}Z_{\omega_{4}j_{4}}].$
(B.56)

To do this, we study (B.56) in five cases that cover all possibilities (up to
permutation of the indices $j_{1},\ldots,j_{4}$).

Case 1: $j_{1}=j_{2}=j_{3}=j_{4}$. Define $j=j_{1}$. It holds that

$\displaystyle\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}$
$\displaystyle\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j}Z_{\omega_{2}j}Z_{\omega_{3}j}Z_{\omega_{4}j}]$
$\displaystyle=\begin{cases}\sigma_{i_{1}\ell}^{4}\,\mathbb{E}Z_{\omega_{1}j}^{4}\lesssim\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j}&\quad\text{
if }\omega_{1}=\omega_{2}=\omega_{3}=\omega_{4}\\\
\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\mathbb{E}Z_{\omega_{1}j}^{2}\mathbb{E}Z_{\omega_{3}j}^{2}\lesssim\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\Omega_{i_{1}j}\Omega_{i_{3}j}&\quad\text{
if
}\omega_{1}=\omega_{2},\omega_{3}=\omega_{4},\omega_{1}\neq\omega_{3}\end{cases}$
(B.57)

Up to permutation of the indices $\omega_{1},\ldots,\omega_{4}$, this accounts
for all cases such that (B.57) is nonvanishing. To be precise, by symmetry, it
also holds that for all permutations $\pi:[4]\to[4]$ that if
$\omega_{\pi(1)}=\omega_{\pi(2)},\omega_{\pi(3)}=\omega_{\pi(4)},\omega_{\pi(1)}\neq\omega_{\pi(3)}$,
then

$\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j}Z_{\omega_{2}j}Z_{\omega_{3}j}Z_{\omega_{4}j}]\lesssim\sigma_{i_{\pi(1)}\ell}^{2}\sigma_{i_{\pi(3)}\ell}^{2}\Omega_{i_{\pi(1)}j}\Omega_{i_{\pi(3)}j}.$

In all other cases besides those considered above, we have

$\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j}Z_{\omega_{2}j}Z_{\omega_{3}j}Z_{\omega_{4}j}]=0$

by independence.

Therefore,

$\displaystyle A_{jjjj}$
$\displaystyle\lesssim\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\Omega_{ij}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\Omega_{i_{1}j}\Omega_{i_{3}j}$
(B.58)

In the remaining Cases 2–6, we follow the same strategy of writing out bounds
for

$\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j_{1}}Z_{\omega_{2}j_{2}}Z_{\omega_{3}j_{3}}Z_{\omega_{4}j_{4}}]$

that cover all nonzero cases, up to permutation of the indices
$\omega_{1},\ldots,\omega_{4}$.

Case 2: $j_{1}=j_{2}=j_{3},j_{1}\neq j_{4}$. It holds that

$\displaystyle\sigma_{i_{1}\ell}$
$\displaystyle\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j_{1}}Z_{\omega_{2}j_{1}}Z_{\omega_{3}j_{1}}Z_{\omega_{4}j_{4}}]$
$\displaystyle\quad=\begin{cases}\sigma_{i_{1}\ell}^{4}\mathbb{E}[Z_{\omega_{1}j_{1}}^{3}Z_{\omega_{1}j_{4}}]\lesssim\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{4}}&\quad\text{
if }\omega_{1}=\omega_{2}=\omega_{3}=\omega_{4}\\\
\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\mathbb{E}Z_{\omega_{1}j_{1}}^{2}\mathbb{E}Z_{\omega_{3}j_{1}}Z_{\omega_{3}j_{4}}\lesssim\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}&\quad\text{
if }\omega_{1}=\omega_{2},\omega_{3}=\omega_{4},\omega_{1}\neq\omega_{3}\\\
\end{cases}$ (B.59)

Up to permutation of the indices $\omega_{1},\ldots,\omega_{4}$, this accounts
for all cases such that (B.59) is nonvanishing. Thus

$\displaystyle
A_{j_{1},j_{1},j_{1},j_{4}}\lesssim\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{4}}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}$
(B.60)

Case 3: $j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq j_{3}$. It holds that

$\displaystyle\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j_{1}}Z_{\omega_{2}j_{1}}Z_{\omega_{3}j_{3}}Z_{\omega_{4}j_{3}}]$
$\displaystyle=\begin{cases}\sigma_{i_{1}\ell}^{4}\mathbb{E}Z_{\omega_{1}j_{1}}^{2}Z_{\omega_{1}j_{3}}^{2}\lesssim\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}&\text{
if }\omega_{1}=\omega_{2}=\omega_{3}=\omega_{4}\\\
\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\mathbb{E}Z_{\omega_{1}j_{1}}^{2}\mathbb{E}Z_{\omega_{3}j_{3}}^{2}\lesssim\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}&\text{
if }\omega_{1}=\omega_{2},\omega_{3}=\omega_{4},\omega_{1}\neq\omega_{3}\\\
\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\mathbb{E}Z_{\omega_{1}j_{1}}Z_{\omega_{1}j_{3}}\mathbb{E}Z_{\omega_{2}j_{1}}Z_{\omega_{2}j_{3}}\lesssim\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{2}j_{1}}\Omega_{i_{2}j_{3}}&\text{
if
}\omega_{1}=\omega_{3},\omega_{2}=\omega_{4},\omega_{1}\neq\omega_{2}.\end{cases}$
(B.61)

Up to permutation of the indices $\omega_{1},\ldots,\omega_{4}$, this accounts
for all cases such that (B.61) is nonvanishing. Thus by symmetry,

$\displaystyle A_{j_{1},j_{1},j_{3},j_{3}}$
$\displaystyle\lesssim\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}$
(B.62)
$\displaystyle\quad+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{3}}$

Case 4: $j_{1}=j_{2}$ and $j_{1},j_{3},j_{4}$ distinct. We have

$\displaystyle\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j_{1}}Z_{\omega_{2}j_{1}}Z_{\omega_{3}j_{3}}Z_{\omega_{4}j_{4}}]$
$\displaystyle=\begin{cases}\sigma_{i_{1}\ell}^{4}\mathbb{E}Z_{\omega_{1}j_{1}}^{2}Z_{\omega_{1}j_{3}}Z_{\omega_{1}j_{4}}\lesssim\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}&\text{
if }\omega_{1}=\omega_{2}=\omega_{3}=\omega_{4}\\\
\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\mathbb{E}Z_{\omega_{1}j_{1}}^{2}\mathbb{E}Z_{\omega_{3}j_{3}}Z_{\omega_{3}j_{4}}\lesssim\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}&\text{
if }\omega_{1}=\omega_{2},\omega_{3}=\omega_{4},\omega_{1}\neq\omega_{3}\\\
\sigma_{i_{1}\ell}^{2}\sigma_{i_{2}\ell}^{2}\mathbb{E}Z_{\omega_{1}j_{1}}Z_{\omega_{1}j_{3}}\mathbb{E}Z_{\omega_{2}j_{1}}Z_{\omega_{2}j_{4}}\lesssim\sigma_{i_{1}\ell}^{2}\sigma_{i_{2}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{2}j_{1}}\Omega_{i_{2}j_{4}}&\text{
if
}\omega_{1}=\omega_{3},\omega_{2}=\omega_{4},\omega_{1}\neq\omega_{2}\end{cases}$
(B.63)

Up to permutation of the indices $\omega_{1},\ldots,\omega_{4}$, this accounts
for all cases such that (B.63) is nonvanishing. Thus

$\displaystyle A_{j_{1},j_{1},j_{3},j_{4}}$
$\displaystyle\lesssim\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}$
(B.64)
$\displaystyle\quad\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{2}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}.$

Case 5: $j_{1},j_{2},j_{3},j_{4}$ distinct. For this final case, it holds that

$\displaystyle\sigma_{i_{1}\ell}\sigma_{i_{2}\ell}\sigma_{i_{3}\ell}\sigma_{i_{4}\ell}\mathbb{E}[Z_{\omega_{1}j_{1}}Z_{\omega_{2}j_{2}}Z_{\omega_{3}j_{3}}Z_{\omega_{4}j_{4}}]$
$\displaystyle=\begin{cases}\sigma_{i_{1}\ell}^{4}\mathbb{E}Z_{\omega_{1}j_{1}}Z_{\omega_{1}j_{2}}Z_{\omega_{1}j_{3}}Z_{\omega_{1}j_{4}}\lesssim\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}&\text{
if }\omega_{1}=\omega_{2}=\omega_{3}=\omega_{4}\\\
\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\mathbb{E}Z_{\omega_{1}j_{1}}Z_{\omega_{1}j_{2}}\mathbb{E}Z_{\omega_{3}j_{3}}Z_{\omega_{3}j_{4}}\lesssim\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}&\text{
if }\omega_{1}=\omega_{2},\omega_{3}=\omega_{4},\omega_{1}\neq\omega_{3}\\\
\end{cases}$

The above accounts for all nonzero cases, up to permutation of
$\omega_{1},\omega_{2},\omega_{3},\omega_{4}$. Hence

$\displaystyle
A_{j_{1},j_{2},j_{3},j_{4}}\lesssim\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}.$
(B.65)

Finally we control the fourth moment using the casework above. By (B.54) and
symmetry,

$\displaystyle\mathbb{E}D_{\ell,s}^{4}$
$\displaystyle\lesssim\sum_{j}\mathbb{E}[Z_{\ell js}Z_{\ell js}Z_{\ell
js}Z_{\ell js}]A_{j,j,j,j}+\sum_{j_{1}\neq j_{4}}\mathbb{E}[Z_{\ell
j_{1}s}Z_{\ell j_{1}s}Z_{\ell j_{1}s}Z_{\ell
j_{4}s}]A_{j_{1},j_{1},j_{1},j_{4}}$ $\displaystyle\quad+\sum_{j_{1}\neq
j_{3}}\mathbb{E}[Z_{\ell j_{1}s}Z_{\ell j_{1}s}Z_{\ell j_{3}s}Z_{\ell
j_{3}s}]A_{j_{1},j_{1},j_{3},j_{3}}+\sum_{j_{1},j_{3},j_{4}\,dist.}\mathbb{E}[Z_{\ell
j_{1}s}Z_{\ell j_{1}s}Z_{\ell j_{3}s}Z_{\ell
j_{4}s}]A_{j_{1},j_{1},j_{3},j_{4}}$
$\displaystyle\quad+\sum_{j_{1},j_{2},j_{3},j_{4}\,dist.}\mathbb{E}[Z_{\ell
j_{1}s}Z_{\ell j_{2}s}Z_{\ell j_{3}s}Z_{\ell
j_{4}s}]A_{j_{1},j_{2},j_{3},j_{4}}$ $\displaystyle=:F_{1\ell s}+F_{2\ell
s}+F_{3\ell s}+F_{4\ell s}+F_{5\ell s}$ (B.66)

By (B.55), (B.58), (B.60) ,(B.62), (B.64), and (B.65),

$\displaystyle F_{1\ell s}$ $\displaystyle\lesssim\sum_{j}\Omega_{\ell
j}\bigg{(}\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\Omega_{ij}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\Omega_{i_{1}j}\Omega_{i_{3}j}\bigg{)}$
$\displaystyle F_{2\ell s}$ $\displaystyle\lesssim\sum_{j_{1}\neq
j_{4}}\Omega_{\ell j_{1}}\Omega_{\ell
j_{4}}\bigg{(}\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{4}}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}\bigg{)}$
$\displaystyle F_{3\ell s}$ $\displaystyle\lesssim\sum_{j_{1}\neq
j_{3}}\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\bigg{(}\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}$
$\displaystyle\qquad+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{3}}\bigg{)}$
$\displaystyle F_{4\ell s}$
$\displaystyle\lesssim\sum_{j_{1},j_{3},j_{4}\,dist.}\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\bigg{(}\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}$
$\displaystyle\qquad+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}.\bigg{)}$
$\displaystyle F_{5\ell s}$
$\displaystyle\lesssim\sum_{j_{1},j_{2},j_{3},j_{4}\,dist.}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\bigg{(}\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}$
$\displaystyle\qquad+\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}\bigg{)}.$

Define

$\displaystyle F_{11\ell s}$
$\displaystyle=\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\sum_{j}\Omega_{\ell
j}\Omega_{ij}$ $\displaystyle F_{21\ell s}$
$\displaystyle=\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\sum_{j_{1}\neq
j_{4}}\Omega_{\ell j_{1}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{4}}$ $\displaystyle F_{31\ell s}$
$\displaystyle=\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\sum_{j_{1}\neq
j_{3}}\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}$ $\displaystyle F_{41\ell s}$
$\displaystyle=\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\sum_{j_{1},j_{3},j_{4}\,dist.}\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}$
$\displaystyle F_{51\ell s}$
$\displaystyle=\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{4}\,\sum_{j_{1},j_{2},j_{3},j_{4}\,dist.}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{1}j_{3}}\Omega_{i_{1}j_{4}}$

and

$\displaystyle F_{12\ell s}$
$\displaystyle=\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\sum_{j}\Omega_{\ell
j}\Omega_{i_{1}j}\Omega_{i_{3}j}$ $\displaystyle F_{22\ell s}$
$\displaystyle=\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\sum_{j_{1}\neq
j_{4}}\Omega_{\ell j_{1}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}$
$\displaystyle F_{32\ell s}$
$\displaystyle=\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\sum_{j_{1}\neq
j_{3}}\big{[}\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}+\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{3}}\big{]}$
$\displaystyle F_{42\ell s}$
$\displaystyle=\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\sum_{j_{1},j_{3},j_{4}\,dist.}\big{[}\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad+\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}\big{]}$
$\displaystyle F_{52\ell s}$
$\displaystyle=\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\,\sum_{j_{1},j_{2},j_{3},j_{4}\,dist.}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}$

Note that $\sum_{x=1}^{2}F_{tx\ell s}=F_{t\ell s}$ for all $t\in[5]$. Using
the fact that $\sum_{j}\Omega_{ij}=1$, we have

$\displaystyle\sum_{t}F_{t1\ell s}$ $\displaystyle\lesssim F_{11\ell
s}=\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\sum_{j}\Omega_{\ell
j}\Omega_{ij}=\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\langle\Omega_{\ell},\Omega_{i}\rangle.$
(B.67)

To control $\sum_{t}F_{t2\ell s}$ , observe that, since $\Omega_{ij}\leq 1$
for all $i,j$,

$\displaystyle\sum_{j}\Omega_{\ell
j}\Omega_{i_{1}j}=\langle\Omega_{\ell},\Omega_{i_{1}}\circ\Omega_{i_{3}}\rangle$
$\displaystyle\sum_{j_{1}\neq j_{4}}\Omega_{\ell j_{1}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}\leq\langle\Omega_{\ell},\Omega_{i_{1}}\circ\Omega_{i_{3}}\rangle\cdot\langle\Omega_{\ell},\Omega_{i_{3}}\rangle$
$\displaystyle\sum_{j_{1}\neq j_{3}}\big{[}\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}+\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{3}}\big{]}\leq
2\langle\Omega_{\ell},\Omega_{i_{1}}\rangle\cdot\langle\Omega_{\ell},\Omega_{i_{3}}\rangle$
$\displaystyle\sum_{j_{1},j_{3},j_{4}\,dist.}\big{[}\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}+\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{3}}\Omega_{i_{3}j_{1}}\Omega_{i_{3}j_{4}}\big{]}\leq
2\langle\Omega_{\ell},\Omega_{i_{1}}\rangle\langle\Omega_{\ell},\Omega_{i_{3}}\rangle^{2}$
$\displaystyle\sum_{j_{1},j_{2},j_{3},j_{4}\,dist.}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}\Omega_{i_{1}j_{1}}\Omega_{i_{1}j_{2}}\Omega_{i_{3}j_{3}}\Omega_{i_{3}j_{4}}\leq\langle\Omega_{\ell},\Omega_{i_{1}}\rangle^{2}\langle\Omega_{\ell},\Omega_{i_{3}}\rangle^{2}.$

These bounds are relatively sharp, and it is clear that the first and third
lines dominate. Furthermore $as$. Hence,

$\displaystyle\sum_{t}F_{t2\ell s}\lesssim F_{12\ell s}+F_{32\ell
s}\lesssim\sum_{\omega_{1}\neq\omega_{3}\in\mathcal{I}_{\ell-1}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\big{[}\langle\Omega_{\ell},\Omega_{i_{1}}\circ\Omega_{i_{3}}\rangle+\langle\Omega_{\ell},\Omega_{i_{1}}\rangle\cdot\langle\Omega_{\ell},\Omega_{i_{3}}\rangle\big{]}.$
(B.68)

Observe that if $\ell\in S_{k}$, then

$\displaystyle\sum_{\omega}\sigma_{i\ell}^{4}\Omega_{ij}$
$\displaystyle\leq\sum_{i\in
S_{k}}\frac{1}{n_{k}^{4}\bar{N}_{k}^{4}}N_{i}\Omega_{ij}+\sum_{k^{\prime}=1}^{K}\sum_{i\in
S_{k^{\prime}}}\frac{1}{n^{4}\bar{N}^{4}}N_{i}\Omega_{ij}$ (B.69)
$\displaystyle\leq\frac{1}{n_{k}^{3}\bar{N}_{k}^{3}}\mu_{kj}+\frac{1}{n^{3}\bar{N}^{3}}\mu_{j},$
(B.70)

and

$\displaystyle\sum_{\omega}\sigma_{i\ell}^{2}\Omega_{ij}$
$\displaystyle\leq\sum_{i\in
S_{k}}\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}N_{i}\Omega_{ij}+\sum_{k^{\prime}=1}^{K}\sum_{i\in
S_{k^{\prime}}}\frac{1}{n\bar{N}}N_{i}\Omega_{ij}$
$\displaystyle\leq\frac{1}{n_{k}\bar{N}_{k}}\mu_{kj}+\frac{1}{n\bar{N}}\mu_{j}.$

Next,

$\displaystyle\sum_{(\ell,s)}\sum_{t}F_{t1\ell s}$
$\displaystyle\lesssim\sum_{(\ell,s)}\sum_{\omega\in\mathcal{I}_{\ell-1}}\sigma_{i\ell}^{4}\langle\Omega_{\ell},\Omega_{i}\rangle.$
$\displaystyle\lesssim\sum_{(\ell,s)}\sum_{j}\Omega_{\ell
j}\big{(}\frac{1}{n_{k}^{3}\bar{N}_{k}^{3}}\mu_{kj}+\frac{1}{n^{3}\bar{N}^{3}}\mu_{j}\big{)}$
$\displaystyle\lesssim\sum_{j}\sum_{k}\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\mu_{kj}^{2}+\sum_{j}\sum_{k}\frac{1}{n^{2}\bar{N}^{2}}\mu_{j}^{2}\lesssim\sum_{k}\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\|\mu_{k}\|^{2},$
(B.71)

where we applied that $\|\mu\|^{2}\lesssim\sum_{k}\|\mu_{k}\|^{2}$ (see
(A.72)). Furthermore,

$\displaystyle\sum_{(\ell,s)}\sum_{t}F_{t2\ell
s}\leq\sum_{k=1}^{K}\sum_{\ell\in
S_{k}}N_{\ell}\sum_{\omega_{1},\omega_{3}}\sigma_{i_{1}\ell}^{2}\sigma_{i_{3}\ell}^{2}\big{[}\langle\Omega_{\ell},\Omega_{i_{1}}\circ\Omega_{i_{3}}\rangle+\langle\Omega_{\ell},\Omega_{i_{1}}\rangle\cdot\langle\Omega_{\ell},\Omega_{i_{3}}\rangle\big{]}$
$\displaystyle\lesssim\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}\bigg{[}\sum_{j}\Omega_{\ell_{j}}\big{(}\frac{1}{n_{k}\bar{N}_{k}}\mu_{kj}+\frac{1}{n\bar{N}}\mu_{j}\big{)}^{2}+\bigg{(}\sum_{j}\Omega_{\ell
j}\cdot\big{(}\frac{1}{n_{k}\bar{N}_{k}}\mu_{kj}+\frac{1}{n\bar{N}}\mu_{j}\big{)}\bigg{)}^{2}\bigg{]}$
$\displaystyle\lesssim\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}\sum_{j}\Omega_{\ell_{j}}\big{(}\frac{1}{n_{k}\bar{N}_{k}}\mu_{kj}+\frac{1}{n\bar{N}}\mu_{j}\big{)}^{2}$

In the last line we apply Cauchy–Schwarz. Continuing, we have

$\displaystyle\sum_{(\ell,s)}\sum_{t}F_{t2\ell s}$
$\displaystyle\lesssim\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}\sum_{j}\Omega_{\ell_{j}}\big{(}\frac{1}{n_{k}\bar{N}_{k}}\mu_{kj}+\frac{1}{n\bar{N}}\mu_{j}\big{)}^{2}$
$\displaystyle\lesssim\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}\sum_{j}\Omega_{\ell_{j}}\big{(}\frac{1}{n_{k}\bar{N}_{k}}\mu_{kj}\big{)}^{2}+\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}\sum_{j}\Omega_{\ell_{j}}\big{(}\frac{1}{n\bar{N}}\mu_{j}\big{)}^{2}$
$\displaystyle\lesssim\sum_{k}\frac{\|\mu_{k}\|_{3}^{3}}{n_{k}\bar{N}_{k}}+\sum_{k}\frac{\|\mu\|_{3}^{3}}{n\bar{N}}\lesssim\sum_{k}\frac{\|\mu_{k}\|_{3}^{3}}{n_{k}\bar{N}_{k}},$
(B.72)

where we applied (A.93). Combining (B.66), (B.71) and (B.72), we have

$\displaystyle\sum_{(\ell,s)}\mathbb{E}D_{\ell,s}^{4}$
$\displaystyle\lesssim\sum_{(\ell,s)}\sum_{x=1}^{2}\sum_{t=1}^{5}F_{tx\ell
s}\lesssim\sum_{k}\frac{\|\mu_{k}\|^{2}}{n_{k}^{2}\bar{N}_{k}^{2}}+\sum_{k}\frac{\|\mu_{k}\|_{3}^{3}}{n_{k}\bar{N}_{k}}\,,$

as desired.

###  B.5 Proof of Lemma B.4

$\displaystyle\mathrm{Var}\bigg{[}\sum_{(\ell,s)}\mathrm{Var}(\tilde{E}_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})\bigg{]}$
$\displaystyle\to 0$ (B.73)

Next we study (B.73). We have

$\displaystyle\mathrm{Var}(E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})$
$\displaystyle=\mathbb{E}[E_{\ell,s}^{2}|\mathcal{F}_{\prec(\ell,s)}]=\sigma_{\ell}^{2}\sum_{r,r^{\prime}\in[s-1]}\sum_{j,j^{\prime}}\mathbb{E}\big{[}Z_{\ell
jr}Z_{\ell js}Z_{\ell j^{\prime}r^{\prime}}Z_{\ell
j^{\prime}s}\big{|}\mathcal{F}_{\prec(\ell,s)}\big{]}$
$\displaystyle=\sigma_{\ell}^{2}\sum_{r,r^{\prime}\in[s-1]}\sum_{j,j^{\prime}}Z_{\ell
jr}Z_{\ell j^{\prime}r^{\prime}}\mathbb{E}[Z_{\ell js}Z_{\ell j^{\prime}s}]$
$\displaystyle=\sigma_{\ell}^{2}\sum_{r,r^{\prime}\in[s-1]}\sum_{j,j^{\prime}}\delta_{jj^{\prime}\ell}Z_{\ell
jr}Z_{\ell j^{\prime}r^{\prime}},$ (B.74)

where we let

$\displaystyle\delta_{jj^{\prime}\ell}=\mathbb{E}Z_{\ell js}Z_{\ell
j^{\prime}s}=\begin{cases}\Omega_{\ell j}(1-\Omega_{\ell j})&\quad\text{ if
}j=j^{\prime}\\\ -\Omega_{\ell j}\Omega_{\ell j^{\prime}}&\quad\text{
else}.\end{cases}$ (B.75)

Define

$\displaystyle\varphi_{\ell r\ell
r^{\prime}}=\sum_{j,j^{\prime}}\delta_{jj^{\prime}\ell}Z_{\ell jr}Z_{\ell
j^{\prime}r^{\prime}}.$ (B.76)

By (B.74) we have

$\displaystyle\sum_{(\ell,s)}\mathrm{Var}(E_{\ell,s}|\mathcal{F}_{\prec(\ell,s)})$
$\displaystyle=\sum_{\ell=1}^{n}\sum_{s=1}^{N_{\ell}}\sum_{r,r^{\prime}\in[s-1]}\sigma_{\ell}^{2}\,\varphi_{\ell
r\ell r^{\prime}}$
$\displaystyle=\sum_{\ell=1}^{n}\sum_{s=1}^{N_{\ell}}\big{[}\sum_{r\in[s-1]}\sigma_{\ell}^{2}\,\varphi_{\ell
r\ell r}+2\sum_{r<r^{\prime}\in[s-1]}\sigma_{\ell}^{2}\,\varphi_{\ell r\ell
r^{\prime}}\big{]}$
$\displaystyle=\sum_{\ell=1}^{n}\sum_{r=1}^{N_{\ell}}\sum_{s\in[N_{\ell}]:s>r}\sigma_{\ell}^{2}\,\varphi_{\ell
r\ell
r}+2\sum_{\ell=1}^{s}\sum_{r<r^{\prime}\in[N_{\ell}]}\sum_{s\in[N_{\ell}]:s>r^{\prime}}\sigma_{\ell}^{2}\,\varphi_{\ell
r\ell r^{\prime}}$
$\displaystyle=\sum_{\ell=1}^{n}\sum_{r=1}^{N_{\ell}}(N_{\ell}-r)\sigma_{\ell}^{2}\,\varphi_{\ell
r\ell
r}+2\sum_{\ell=1}^{s}\sum_{r<r^{\prime}\in[N_{\ell}]}(N_{\ell}-r^{\prime})\sigma_{\ell}^{2}\,\varphi_{\ell
r\ell r^{\prime}}$ $\displaystyle\equiv S_{1}+S_{2}.$

Observe that $S_{1}$ and $S_{2}$ are uncorrelated. In addition, the terms in
the summation defining $S_{1}$ are uncorrelated; the same holds for $S_{2}$
also.

First we study $S_{2}$. Next,

$\displaystyle\mathbb{E}\varphi_{\ell r\ell r^{\prime}}^{2}$
$\displaystyle=\sum_{j_{1},j_{2},j_{3},j_{4}}\delta_{j_{1}j_{2},\ell}\delta_{j_{3}j_{4},\ell}\,\mathbb{E}Z_{\ell
j_{1}r}Z_{\ell j_{2}r^{\prime}}Z_{\ell j_{3}r}Z_{\ell j_{4}r^{\prime}}$
$\displaystyle=\sum_{j_{1},j_{2},j_{3},j_{4}}\delta_{j_{1}j_{2}\ell}\delta_{j_{3}j_{4}\ell}\mathbb{E}Z_{\ell
j_{1}r}Z_{\ell j_{3}r}\mathbb{E}Z_{\ell j_{2}r^{\prime}}Z_{\ell
j_{4}r^{\prime}}.$ (B.77)

First we study $V_{2}$. By casework,

$\displaystyle|\delta_{j_{1}j_{2}\ell}\delta_{j_{3}j_{4}\ell}\mathbb{E}Z_{\ell
j_{1}r}Z_{\ell j_{3}r}\mathbb{E}Z_{\ell j_{2}r^{\prime}}Z_{\ell
j_{4}r^{\prime}}|$ (B.78)
$\displaystyle=\begin{cases}\delta_{jj\ell}^{2}\mathbb{E}Z_{\ell
jr}^{2}\mathbb{E}Z_{\ell jr^{\prime}}^{2}\lesssim\Omega_{\ell
j}^{4}\quad&\text{ if }j_{1}=\cdots=j_{4}\\\
\delta_{j_{1}j_{1}\ell}\delta_{j_{1}j_{4}\ell}|\mathbb{E}Z_{\ell
j_{1}r}^{2}\mathbb{E}Z_{\ell j_{1}r^{\prime}}Z_{\ell
j_{4}r^{\prime}}|\lesssim\Omega_{\ell j_{1}}^{4}\Omega_{\ell j_{4}}^{2}&\text{
if }j_{1}=j_{2}=j_{3},j_{1}\neq j_{4}\\\
\delta_{j_{1}j_{1}\ell}\delta_{j_{3}j_{3}\ell}\mathbb{E}Z_{\ell j_{1}r}Z_{\ell
j_{3}r}\mathbb{E}Z_{\ell j_{1}r^{\prime}}Z_{\ell
j_{3}r^{\prime}}\lesssim\Omega_{\ell j_{1}}^{3}\Omega_{\ell j_{3}}^{3}&\text{
if }j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq j_{3}\\\
\delta_{j_{1}j_{2}\ell}^{2}\mathbb{E}Z_{\ell j_{1}r}^{2}\mathbb{E}Z_{\ell
j_{2}r^{\prime}}^{2}\lesssim\Omega_{\ell j_{1}}^{3}\Omega_{\ell
j_{2}}^{3}&\text{ if }j_{1}=j_{3},j_{2}=j_{4},j_{1}\neq j_{2}\\\
\delta_{j_{1}j_{1}\ell}\delta_{j_{3}j_{4}\ell}\mathbb{E}Z_{\ell j_{1}r}Z_{\ell
j_{3}r}\mathbb{E}Z_{\ell j_{1}r^{\prime}}Z_{\ell
j_{4}r^{\prime}}\lesssim\Omega_{\ell j_{1}}^{3}\Omega_{\ell
j_{3}}^{2}\Omega_{\ell j_{4}}^{2}&\text{ if
}j_{1}=j_{2},j_{1},j_{3},j_{4}\,\,dist.\\\
\delta_{j_{1}j_{2}\ell}\delta_{j_{1}j_{4}\ell}\mathbb{E}Z_{\ell
j_{1}r}^{2}\mathbb{E}Z_{\ell j_{2}r^{\prime}}Z_{\ell
j_{4}r^{\prime}}\lesssim\Omega_{\ell j_{1}}^{3}\Omega_{\ell
j_{2}}^{2}\Omega_{\ell j_{4}}^{2}&\text{ if
}j_{1}=j_{3},j_{1},j_{2},j_{4}\,\,dist.\\\
\delta_{j_{1}j_{2}\ell}\delta_{j_{3}j_{4}\ell}\mathbb{E}Z_{\ell j_{1}r}Z_{\ell
j_{3}r}\mathbb{E}Z_{\ell j_{2}r^{\prime}}Z_{\ell
j_{4}r^{\prime}}\lesssim\Omega_{\ell j_{1}}^{2}\Omega_{\ell
j_{2}}^{2}\Omega_{\ell j_{3}}^{2}\Omega_{\ell j_{4}}^{2}&\text{ if
}j_{1},j_{2},j_{3},j_{4}\,\,dist.\end{cases}$

Up to permutation of the indices $j_{1},\ldots,j_{4}$, all nonzero terms of
(B.77) take one of the forms above. By (B.78) and Cauchy–Schwarz, we have

$\displaystyle\mathbb{E}\varphi_{\ell r\ell r^{\prime}}^{2}$
$\displaystyle\lesssim\|\Omega_{\ell}\|_{4}^{4}+\|\Omega_{\ell}\|_{4}^{4}\|\Omega_{\ell}\|^{2}+2\|\Omega_{\ell}\|_{3}^{6}+2\|\Omega_{\ell}\|_{3}^{3}\|\Omega_{\ell}\|^{4}+\|\Omega_{\ell}\|^{8}\lesssim\|\Omega_{\ell}\|_{4}^{4}.$
(B.79)

Recalling that $\\{\varphi_{\ell r\ell
r^{\prime}}\\}_{\ell,r<r^{\prime}\in[N_{\ell}]}$ are mutually uncorrelated, it
follows that

$\displaystyle\mathrm{Var}(S_{2})$
$\displaystyle\lesssim\sum_{\ell}\sum_{r<r^{\prime}\in[N_{\ell}]}(N_{\ell}-r^{\prime})^{2}\sigma_{\ell}^{2}\mathbb{E}\varphi^{2}_{\ell
r\ell r^{\prime}}$
$\displaystyle\lesssim\sum_{\ell}\sum_{r<r^{\prime}\in[N_{\ell}]}(N_{\ell}-r^{\prime})^{2}\sigma_{\ell}^{4}\|\Omega_{\ell}\|_{4}^{4}$
$\displaystyle\lesssim\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}^{4}\cdot\frac{1}{n_{k}^{4}\bar{N}_{k}^{4}}\|\Omega_{\ell}\|_{4}^{4}.$
(B.80)

Next we study $S_{1}$. We have

$\displaystyle\mathbb{E}\varphi_{\ell r\ell r}^{2}$
$\displaystyle=\sum_{j_{1},j_{2},j_{3},j_{4}}\delta_{j_{1}j_{2}\ell}\delta_{j_{3}j_{4}\ell}\mathbb{E}Z_{\ell
j_{1}r}Z_{\ell j_{2}r}Z_{\ell j_{3}r}Z_{\ell j_{4}r}.$

We have the following bounds by casework.

$\displaystyle|\delta_{j_{1}j_{2}\ell}\delta_{j_{3}j_{4}\ell}\mathbb{E}Z_{\ell
j_{1}r}Z_{\ell j_{2}r}Z_{\ell j_{3}r}Z_{\ell j_{4}r}|$ (B.81)
$\displaystyle=\begin{cases}\delta_{jj\ell}^{2}\mathbb{E}Z_{\ell
jr}^{4}\lesssim\Omega_{\ell j}^{3}\quad&\text{ if }j_{1}=\cdots=j_{4}\\\
\delta_{j_{1}j_{1}\ell}\delta_{j_{1}j_{4}\ell}|\mathbb{E}Z_{\ell
j_{1}r}^{3}Z_{\ell j_{4}r}|\lesssim\Omega_{\ell j_{1}}^{3}\Omega_{\ell
j_{4}}^{2}&\text{ if }j_{1}=j_{2}=j_{3},j_{1}\neq j_{4}\\\
\delta_{j_{1}j_{1}\ell}\delta_{j_{3}j_{3}\ell}\mathbb{E}Z_{\ell
j_{1}r}^{2}Z_{\ell j_{3}r}^{2}\lesssim\Omega_{\ell j_{1}}^{2}\Omega_{\ell
j_{3}}^{2}&\text{ if }j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq j_{3}\\\
\delta_{j_{1}j_{2}\ell}^{2}\mathbb{E}Z_{\ell j_{1}r}^{2}Z_{\ell
j_{2}r}^{2}\lesssim\Omega_{\ell j_{1}}^{3}\Omega_{\ell j_{2}}^{3}&\text{ if
}j_{1}=j_{3},j_{2}=j_{4},j_{1}\neq j_{3}\\\
\delta_{j_{1}j_{1}\ell}\delta_{j_{3}j_{4}\ell}|\mathbb{E}Z_{\ell
j_{1}r}^{2}Z_{\ell j_{3}r}Z_{\ell j_{4}r}|\lesssim\Omega_{\ell
j_{1}}^{2}\Omega_{\ell j_{3}}^{2}\Omega_{\ell j_{4}}^{2}&\text{ if
}j_{1}=j_{2},j_{1},j_{3},j_{4}\,\,dist.\\\
\delta_{j_{1}j_{2}\ell}\delta_{j_{1}j_{4}\ell}|\mathbb{E}Z_{\ell
j_{1}r}^{2}Z_{\ell j_{2}r}Z_{\ell j_{4}}|\lesssim\Omega_{\ell
j_{1}}^{3}\Omega_{\ell j_{2}}^{2}\Omega_{\ell j_{4}}^{2}&\text{ if
}j_{1}=j_{3},j_{1},j_{2},j_{4}\,\,dist.\\\
\delta_{j_{1}j_{2}\ell}\delta_{j_{3}j_{4}\ell}|\mathbb{E}Z_{\ell
j_{1}r}Z_{\ell j_{2}r}Z_{\ell j_{3}r}Z_{\ell j_{4}r}|\lesssim\Omega_{\ell
j_{1}}^{2}\Omega_{\ell j_{2}}^{2}\Omega_{\ell j_{3}}^{2}\Omega_{\ell
j_{4}}^{2}&\text{ if }j_{1},j_{2},j_{3},j_{4}\,\,dist.\end{cases}$

Up to symmetry, this accounts for all possible (nonzero) cases. Hence by
Cauchy–Schwarz,

$\displaystyle\mathbb{E}\varphi_{\ell r\ell r}^{2}$
$\displaystyle\lesssim\|\Omega_{\ell}\|_{3}^{3}+\|\Omega_{\ell}\|_{3}^{3}\|\Omega_{\ell}\|^{2}+\|\Omega_{\ell}\|^{4}+\|\Omega_{\ell}\|_{3}^{6}+\|\Omega_{\ell}\|^{6}+\|\Omega_{\ell}\|_{3}^{3}\|\Omega_{\ell}\|^{4}+\|\Omega_{\ell}\|^{8}\lesssim\|\Omega_{\ell}\|_{3}^{3}.$
(B.82)

Recalling that $\\{\varphi_{\ell r\ell r}\\}_{\ell,r\in[N_{\ell}]}$ is an
uncorrelated collection of random variables, we have

$\displaystyle\mathrm{Var}(S_{1})$
$\displaystyle\lesssim\sum_{\ell}\sum_{r\in[N_{\ell}]}(N_{\ell}-r)^{2}\sigma_{\ell}^{4}\mathbb{E}\varphi_{\ell
r\ell r}^{2}$
$\displaystyle\lesssim\sum_{\ell}\sum_{r\in[N_{\ell}]}(N_{\ell}-r)^{2}\sigma_{\ell}^{4}\|\Omega_{\ell}\|_{3}^{3}$
$\displaystyle\lesssim\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}^{3}\cdot\frac{1}{n_{k}^{4}\bar{N}_{k}^{4}}\|\Omega_{\ell}\|_{3}^{3}.$
(B.83)

Combining (B.83) and (B.80) proves the result. ∎

###  B.6 Proof of Lemma B.5

We have

$\displaystyle\mathbb{E}E_{\ell,s}^{4}$
$\displaystyle=\sum_{r_{1},r_{2},r_{3},r_{4}\in[s-1]}\,\,\sigma_{\ell}^{4}\sum_{j_{1},j_{2},j_{3},j_{4}}\mathbb{E}Z_{\ell
j_{1}r_{1}}Z_{\ell j_{1}s}Z_{\ell j_{2}r_{2}}Z_{\ell j_{2}s}Z_{\ell
j_{3}r_{3}}Z_{\ell j_{3}s}Z_{\ell j_{4}r_{4}}Z_{\ell j_{4}s}$
$\displaystyle=\sigma_{\ell}^{4}\sum_{j_{1},j_{2},j_{3},j_{4}}\bigg{[}\,\,\mathbb{E}[Z_{\ell
j_{1}s}Z_{\ell j_{2}s}Z_{\ell j_{3}s}Z_{\ell
j_{4}s}]\cdot\underbrace{\sum_{r_{1},r_{2},r_{3},r_{4}\in[s-1]}\mathbb{E}[Z_{\ell
j_{1}r_{1}}Z_{\ell j_{2}r_{2}}Z_{\ell j_{3}r_{3}}Z_{\ell
j_{4}r_{4}}]}_{=:B_{\ell,s;j_{1},j_{2},j_{3},j_{4}}}\,\,\bigg{]}$ (B.84)

We have by exhaustive casework that

$\displaystyle|\mathbb{E}[Z_{\ell j_{1}r_{1}}Z_{\ell j_{2}r_{2}}Z_{\ell
j_{3}r_{3}}Z_{\ell j_{4}r_{4}}]|$ (B.85)
$\displaystyle=\begin{cases}\mathbb{E}Z_{\ell
j_{1}r_{1}}^{4}\lesssim\Omega_{\ell j_{1}}&\text{ if
}\begin{subarray}{c}j_{1}=j_{2}=j_{3}=j_{4};\\\
r_{1}=r_{2}=r_{3}=r_{4}\end{subarray}\\\ \mathbb{E}Z_{\ell
j_{1}r_{1}}^{2}\mathbb{E}Z_{\ell j_{1}r_{3}}^{2}\lesssim\Omega_{\ell
j_{1}}^{2}&\text{ if }\begin{subarray}{c}j_{1}=j_{2}=j_{3}=j_{4};\\\
r_{1}=r_{2},r_{3}=r_{4},r_{1}\neq r_{3}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}^{3}Z_{\ell j_{4}r_{1}}]|\lesssim\Omega_{\ell j_{1}}\Omega_{\ell
j_{4}}&\text{ if }\begin{subarray}{c}j_{1}=j_{2}=j_{3},j_{1}\neq j_{4};\\\
r_{1}=r_{2}=r_{3}=r_{4}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}^{2}\mathbb{E}Z_{\ell j_{1}r_{3}}Z_{\ell
j_{4}r_{3}}]|\lesssim\Omega_{\ell j_{1}}^{2}\Omega_{\ell j_{4}}&\text{ if
}\begin{subarray}{c}j_{1}=j_{2}=j_{3},j_{1}\neq j_{4};\\\
r_{1}=r_{2},r_{3}=r_{4},r_{1}\neq r_{3}\end{subarray}\\\ |\mathbb{E}Z_{\ell
j_{1}r_{1}}^{2}Z_{\ell j_{3}r_{1}}^{2}|\lesssim\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}&\text{ if }\begin{subarray}{c}j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq
j_{3};\\\ r_{1}=r_{2}=r_{3}=r_{4}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}^{2}Z_{\ell j_{3}r_{3}}^{2}]|\lesssim\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}&\text{ if
}\begin{subarray}{c}j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq j_{3};\\\
r_{1}=r_{2},r_{3}=r_{4},r_{1}\neq r_{3}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}Z_{\ell j_{3}r_{1}}\mathbb{E}Z_{\ell j_{1}r_{2}}Z_{\ell
j_{3}r_{2}}]|\lesssim\Omega_{\ell j_{1}}^{2}\Omega_{\ell j_{3}}^{2}&\text{ if
}\begin{subarray}{c}j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq j_{3};\\\
r_{1}=r_{3},r_{2}=r_{4},r_{1}\neq r_{2}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}^{2}Z_{\ell j_{3}r_{1}}Z_{\ell j_{4}r_{1}}]|\lesssim\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}&\text{ if
}\begin{subarray}{c}j_{1}=j_{2},j_{1},j_{3},j_{4}\,\,dist.;\\\
r_{1}=r_{2}=r_{3}=r_{4}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}^{2}\mathbb{E}Z_{\ell j_{3}r_{3}}Z_{\ell
j_{4}r_{3}}]|\lesssim\Omega_{\ell j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}&\text{ if
}\begin{subarray}{c}j_{1}=j_{2},j_{1},j_{3},j_{4}\,\,dist.;\\\
r_{1}=r_{2},r_{3}=r_{4},r_{1}\neq r_{3}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}Z_{\ell j_{3}r_{1}}\mathbb{E}Z_{\ell j_{1}r_{2}}Z_{\ell
j_{4}r_{2}}]|\lesssim\Omega_{\ell j_{1}}^{2}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}&\text{ if
}\begin{subarray}{c}j_{1}=j_{2},j_{1},j_{3},j_{4}\,\,dist.;\\\
r_{1}=r_{3},r_{2}=r_{4},r_{1}\neq r_{2}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}Z_{\ell j_{2}r_{1}}Z_{\ell j_{3}r_{1}}Z_{\ell
j_{4}r_{1}}]|\lesssim\Omega_{\ell j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell
j_{3}}\Omega_{\ell j_{4}}&\text{ if
}\begin{subarray}{c}j_{1},j_{2},j_{3},j_{4}\,\,dist;\\\
r_{1}=r_{2}=r_{3}=r_{4}\end{subarray}\\\ |\mathbb{E}[Z_{\ell
j_{1}r_{1}}Z_{\ell j_{2}r_{1}}\mathbb{E}Z_{\ell j_{3}r_{3}}Z_{\ell
j_{4}r_{3}}]|\lesssim\Omega_{\ell j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell
j_{3}}\Omega_{\ell j_{4}}&\text{ if
}\begin{subarray}{c}j_{1},j_{2},j_{3},j_{4}\,\,dist;\\\
r_{1}=r_{2},r_{3}=r_{4},r_{1}\neq r_{3}\end{subarray}\end{cases}$

Up to permutation of the indices $j_{1},j_{2},j_{3},j_{4}$ and
$r_{1},r_{2},r_{3},r_{4}$, this accounts for all possible cases such that
(B.85) is nonzero. Therefore,

$\displaystyle B_{\ell,s;j_{1},j_{2},j_{3},j_{4}}$
$\displaystyle\lesssim\begin{cases}s\Omega_{\ell j_{1}}+s^{2}\Omega_{\ell
j_{1}}^{2}&\quad\text{ if }j_{1}=j_{2}=j_{3}=j_{4}\\\ s\Omega_{\ell
j_{1}}\Omega_{\ell j_{4}}+s^{2}\Omega_{\ell j_{1}}^{2}\Omega_{\ell
j_{4}}&\quad\text{ if }j_{1}=j_{2}=j_{3},j_{1}\neq j_{4}\\\ s\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}+s^{2}\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}&\quad\text{ if }j_{1}=j_{2},j_{3}=j_{4},j_{1}\neq j_{3}\\\
s\Omega_{\ell j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}+s^{2}\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}&\quad\text{ if
}j_{1}=j_{2},j_{1},j_{3},j_{4}\,\,dist.\\\ s\Omega_{\ell j_{1}}\Omega_{\ell
j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}+s^{2}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}&\quad\text{ if
}j_{1},j_{2},j_{3},j_{4}\,\,dist.\\\ \end{cases}$

Up to permutation of $j_{1},j_{2},j_{3},j_{4}$, this accounts for all possible
cases. Returning to (B.84), we have by applying (B.55) and the previous
display that

$\displaystyle\mathbb{E}E_{\ell,s}^{4}$
$\displaystyle\lesssim\sigma_{\ell}^{4}\bigg{(}\sum_{j}\Omega_{\ell
j}(s\Omega_{\ell j}+s^{2}\Omega_{\ell j}^{2})+\sum_{j_{1}\neq
j_{4}}\Omega_{\ell j_{1}}\Omega_{\ell j_{4}}(s\Omega_{\ell j_{1}}\Omega_{\ell
j_{4}}+s^{2}\Omega_{\ell j_{1}}^{2}\Omega_{\ell j_{4}})$
$\displaystyle\quad+\sum_{j_{1}\neq j_{3}}\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}(s\Omega_{\ell j_{1}}\Omega_{\ell j_{3}}+s^{2}\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}})$
$\displaystyle\quad+\sum_{j_{1},j_{3},j_{4}(dist.)}\Omega_{\ell
j_{1}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}(s\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\Omega_{\ell j_{4}}+s^{2}\Omega_{\ell j_{1}}\Omega_{\ell
j_{3}}\Omega_{\ell j_{4}})$
$\displaystyle\quad+\sum_{j_{1},j_{2},j_{3},j_{4}\,dist.}\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell j_{4}}(s\Omega_{\ell
j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell j_{3}}\Omega_{\ell
j_{4}}+s^{2}\Omega_{\ell j_{1}}\Omega_{\ell j_{2}}\Omega_{\ell
j_{3}}\Omega_{\ell j_{4}})\bigg{)}$ $\displaystyle\lesssim
s\sigma_{\ell}^{4}\|\Omega_{\ell}\|^{2}+s^{2}\sigma_{\ell}^{4}\|\Omega_{\ell}\|_{3}^{3}.$

In the third line we group the coefficients of $s$ and $s^{2}$ and use the
fact that $\|\Omega_{\ell}\|^{4}\leq\|\Omega_{\ell}\|_{3}^{3}$ by
Cauchy–Schwarz. Therefore

$\displaystyle\sum_{(\ell,s)}\mathbb{E}E_{\ell,s}^{4}$
$\displaystyle\lesssim\sum_{(\ell,s)}s\sigma_{\ell}^{4}\|\Omega_{\ell}\|^{2}+\sum_{(\ell,s)}s^{2}\sigma_{\ell}^{4}\|\Omega_{\ell}\|_{3}^{3}$
$\displaystyle=\sum_{k}\sum_{\ell\in
S_{k}}\sum_{s\in[N_{\ell}]}s\sigma_{\ell}^{4}\|\Omega_{\ell}\|^{2}+\sum_{k}\sum_{\ell\in
S_{k}}\sum_{s\in[N_{\ell}]}s^{2}\sigma_{\ell}^{4}\|\Omega_{\ell}\|_{3}^{3}$
$\displaystyle\lesssim\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}^{2}\cdot\frac{1}{n_{k}^{4}\bar{N}_{k}^{4}}\|\Omega_{\ell}\|^{2}+\sum_{k}\sum_{\ell\in
S_{k}}N_{\ell}^{3}\cdot\frac{1}{n_{k}^{4}\bar{N}_{k}^{4}}\|\Omega_{\ell}\|_{3}^{3},$

as desired. ∎

###  B.7 Proof of Lemma B.6

We have

$\displaystyle\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{2}\|\Omega_{i}\|^{2}}{n_{k}^{4}\bar{N}_{k}^{4}}$
$\displaystyle\leq\sum_{k}\frac{1}{n_{k}^{4}\bar{N}_{k}^{4}}\sum_{i,m\in
S_{k}}N_{i}N_{m}\langle\Omega_{i},\Omega_{m}\rangle$
$\displaystyle=\sum_{k}\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\|\mu_{k}\|^{2},$

which establishes the first claim.

Similarly,

$\displaystyle\sum_{k}\sum_{i\in
S_{k}}\frac{N_{i}^{3}\|\Omega_{i}\|_{3}^{3}}{n_{k}^{4}\bar{N}_{k}^{4}}$
$\displaystyle\leq\sum_{k}\frac{1}{n_{k}^{4}\bar{N}_{k}^{4}}\sum_{i,m,m^{\prime}\in
S_{k}}N_{i}N_{m}N_{m^{\prime}}\sum_{j}\Omega_{ij}\Omega_{mj}\Omega_{m^{\prime}j}$
$\displaystyle\leq\sum_{k}\frac{1}{n_{k}\bar{N}_{k}}\|\mu_{k}\|_{3}^{3},$

which proves the second claim.

The third claim follows similarly and we omit the proof.

∎

##  Appendix C Proofs of other main lemmas and theorems

###  C.1 Proof of Lemma 2.1

We start from computing $\mathbb{E}[(\hat{\mu}_{kj}-\hat{\mu}_{j})^{2}]$.
Write $X_{ij}=N_{i}(\Omega_{ij}+Y_{ij})$. It follows by elementary calculation
that

$\hat{\mu}_{kj}-\hat{\mu}_{j}=\mu_{kj}-\mu_{j}+\Bigl{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\Bigr{)}\sum_{i\in
S_{k}}N_{i}Y_{ij}-\sum_{1\leq\ell\leq K:\ell\neq
k}\frac{1}{n_{\ell}\bar{N}_{\ell}}\sum_{i\in S_{\ell}}N_{i}Y_{ij}.$

For different $k$, the variables $\sum_{i\in S_{k}}N_{i}Y_{ij}$ are
independent of each other. It follows that

$\displaystyle\mathbb{E}[(\hat{\mu}_{kj}$
$\displaystyle-\hat{\mu}_{j})^{2}]=(\mu_{kj}-\mu_{j})^{2}+\Bigl{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\Bigr{)}^{2}\mathbb{E}\Bigl{[}\Bigl{(}\sum_{i\in
S_{k}}N_{i}Y_{ij}\Bigr{)}^{2}\Bigr{]}+\sum_{\ell:\ell\neq
k}\frac{1}{n^{2}\bar{N}^{2}}\mathbb{E}\Bigl{[}\Bigl{(}\sum_{i\in
S_{\ell}}N_{i}Y_{ij}\Bigr{)}^{2}\Bigr{]}$ (C.1)
$\displaystyle=(\mu_{kj}-\mu_{j})^{2}+\Bigl{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\Bigr{)}^{2}\sum_{i\in
S_{k}}N_{i}\Omega_{ij}(1-\Omega_{ij})+\sum_{\ell:\ell\neq
k}\frac{1}{n^{2}\bar{N}^{2}}\sum_{i\in
S_{\ell}}N_{i}\Omega_{ij}(1-\Omega_{ij})$ (C.2)
$\displaystyle=(\mu_{kj}-\mu_{j})^{2}+\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\Bigl{(}1-\frac{n_{k}\bar{N}_{k}}{n\bar{N}}\Bigr{)}\sum_{i\in
S_{k}}N_{i}\Omega_{ij}(1-\Omega_{ij})$ (C.3)
$\displaystyle\qquad+\frac{1}{n^{2}\bar{N}^{2}}\biggl{[}\Bigl{(}1-\frac{n\bar{N}}{n_{k}\bar{N}_{k}}\Bigr{)}\sum_{i\in
S_{k}}N_{i}\Omega_{ij}(1-\Omega_{ij})+\sum_{\ell:\ell\neq k}\sum_{i\in
S_{\ell}}N_{i}\Omega_{ij}(1-\Omega_{ij})\biggr{]}$ (C.4)
$\displaystyle=(\mu_{kj}-\mu_{j})^{2}+\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\Bigl{(}1-\frac{n_{k}\bar{N}_{k}}{n\bar{N}}\Bigr{)}\sum_{i\in
S_{k}}N_{i}\Omega_{ij}(1-\Omega_{ij})$ (C.5)
$\displaystyle\qquad-\frac{1}{n\bar{N}n_{k}\bar{N}_{k}}\underbrace{\biggl{[}\sum_{i\in
S_{k}}N_{i}\Omega_{ij}(1-\Omega_{ij})-\frac{n_{k}\bar{N}_{k}}{n\bar{N}}\sum_{\ell=1}^{K}\sum_{i\in
S_{\ell}}N_{i}\Omega_{ij}(1-\Omega_{ij})\biggr{]}}_{\delta_{kj}}.$ (C.6)

Since $X_{ij}$ follows a binomial distribution, it is easy to see that
$\mathbb{E}[X_{ij}]=N_{i}\Omega_{ij}$ and
$\mathbb{E}[X_{ij}^{2}]=(\mathbb{E}[X_{ij}])^{2}+\mathrm{Var}(X_{ij})=N_{i}^{2}\Omega^{2}_{ij}+N_{i}\Omega_{ij}(1-\Omega_{ij})$.
Combining them gives

$\mathbb{E}[X_{ij}(N_{i}-X_{ij})]=N_{i}(N_{i}-1)\Omega_{ij}(1-\Omega_{ij}).$
(C.7)

Define

$\hat{\zeta}_{kj}=(\hat{\mu}_{kj}-\hat{\mu}_{j})^{2}-\frac{1}{n_{k}^{2}\bar{N}_{k}^{2}}\Bigl{(}1-\frac{n_{k}\bar{N}_{k}}{n\bar{N}}\Bigr{)}\sum_{i\in
S_{k}}\frac{X_{ij}(N_{i}-X_{ij})}{N_{i}-1},$

It follows from (C.1)-(C.7) that

$\mathbb{E}[\hat{\zeta}_{kj}]=(\mu_{kj}-\mu_{j})^{2}-\frac{1}{n\bar{N}n_{k}\bar{N}_{k}}\delta_{kj}.$
(C.8)

We are ready to compute $\mathbb{E}[T]$. By definition,
$T=\sum_{j=1}^{p}\sum_{k=1}^{K}n_{k}\bar{N}_{k}\hat{\zeta}_{kj}$ and
$\rho^{2}=\sum_{j,k}(\mu_{kj}-\mu_{j})^{2}$. Consequently,

$\mathbb{E}[T]=\sum_{j=1}^{p}\sum_{k=1}^{K}n_{k}\bar{N}_{k}\Bigl{[}(\mu_{kj}-\mu_{j})^{2}-\frac{1}{n\bar{N}n_{k}\bar{N}_{k}}\delta_{kj}\Bigr{]}=\rho^{2}-\frac{1}{n\bar{N}}\sum_{j=1}^{p}\sum_{k=1}^{K}\delta_{kj}.$
(C.9)

We use the definition of $\delta_{kj}$ in (C.1). It is seen that for each
$1\leq j\leq p$,

$\sum_{k=1}^{K}\delta_{kj}=\sum_{k=1}^{K}\sum_{i\in
S_{k}}N_{i}\Omega_{ij}(1-\Omega_{ij})-\Bigl{(}\sum_{k=1}^{K}\frac{n_{k}\bar{N}_{k}}{n\bar{N}}\Bigr{)}\sum_{\ell=1}^{K}\sum_{i\in
S_{\ell}}N_{i}\Omega_{ij}(1-\Omega_{ij})=0.$ (C.10)

Combining (C.9)-(C.10) gives $\mathbb{E}[T]=\rho^{2}$. This proves the claim.
∎

###  C.2 Proof of Theorem 3.3

First we show that

$\displaystyle\mathrm{Var}(T)\lesssim\Theta_{n}$ (C.11)

Recall

$\displaystyle\Theta_{n1}$
$\displaystyle=4\sum_{k=1}^{K}\sum_{j=1}^{p}n_{k}\bar{N}_{k}(\mu_{kj}-\mu_{j})^{2}\mu_{kj}$
$\displaystyle\Theta_{n2}$ $\displaystyle=2\sum_{k=1}^{K}\sum_{i\in
S_{k}}\sum_{j=1}^{p}\Bigl{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\Bigr{)}^{2}\frac{N_{i}^{3}}{N_{i}-1}\Omega_{ij}^{2}$
$\displaystyle\Theta_{n3}$
$\displaystyle=\frac{2}{n^{2}\bar{N}^{2}}\sum_{1\leq k\neq\ell\leq
K}\sum_{i\in S_{k}}\sum_{m\in
S_{\ell}}\sum_{j=1}^{p}N_{i}N_{m}\Omega_{ij}\Omega_{mj}$
$\displaystyle\Theta_{n4}$
$\displaystyle=2\sum_{k=1}^{K}\sum_{\begin{subarray}{c}i\in S_{k},m\in
S_{k},\\\ i\neq
m\end{subarray}}\sum_{j=1}^{p}\Bigl{(}\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}}\Bigr{)}^{2}N_{i}N_{m}\Omega_{ij}\Omega_{mj}.$

and that $\sum_{a=1}^{4}\Theta_{na}=\Theta_{n}$.

By Lemma A.2, we immediately have

$\displaystyle\mathrm{Var}({\bf 1}_{p}^{\prime}U_{1})\leq\Theta_{n1}.$ (C.12)

For $U_{2}$, it is shown in the Proof of Lemma A.3 that

$\mathrm{Var}({\bf 1}_{p}^{\prime}U_{2})=4\sum_{k=1}^{K}\sum_{i\in
S_{k}}\sum_{1\leq r<s\leq
N_{i}}\frac{\theta_{i}}{N_{i}(N_{i}-1)}\bigl{[}\|\Omega_{i}\|^{2}+O(\|\Omega_{i}\|_{3}^{3})].$

Thus

$\displaystyle\mathrm{Var}({\bf 1}_{p}^{\prime}U_{2})$ $\displaystyle\lesssim
4\sum_{k=1}^{K}\sum_{i\in S_{k}}\sum_{1\leq r<s\leq
N_{i}}\frac{\theta_{i}}{N_{i}(N_{i}-1)}\|\Omega_{i}\|^{2}$
$\displaystyle=2\sum_{k=1}^{K}\sum_{i\in
S_{k}}\theta_{i}\|\Omega_{i}\|^{2}=\Theta_{n2}$ (C.13)

Next we study $U_{3}$. Using that $\Omega_{mj^{\prime}}\leq 1$ and
$\|\Omega_{i}\|_{1}=1$, we have

$\displaystyle\sum_{k\neq\ell}\frac{n_{k}n_{\ell}\bar{N}_{k}\bar{N}_{\ell}}{n^{2}\bar{N}^{2}}{\bf
1}_{p}^{\prime}(\Sigma_{k}\circ\Sigma_{\ell}){\bf 1}_{p}$
$\displaystyle=\frac{2}{n^{2}\bar{N}^{2}}\sum_{k\neq\ell}\sum_{i\in
S_{k}}\sum_{m\in
S_{\ell}}\sum_{j,j^{\prime}}N_{i}N_{m}\Omega_{ij}\Omega_{ij^{\prime}}\Omega_{mj}\Omega_{mj^{\prime}}$
$\displaystyle\leq\frac{2}{n^{2}\bar{N}^{2}}\sum_{k\neq\ell}\sum_{i\in
S_{k}}\sum_{m\in
S_{\ell}}\sum_{j}N_{i}N_{m}\Omega_{ij}\Omega_{mj}\sum_{j^{\prime}}\Omega_{ij^{\prime}}$
$\displaystyle=\frac{2}{n^{2}\bar{N}^{2}}\sum_{k\neq\ell}\sum_{i\in
S_{k}}\sum_{m\in S_{\ell}}\sum_{j}N_{i}N_{m}\Omega_{ij}\Omega_{mj}.$

Therefore by Lemma A.4,

$\displaystyle\mathrm{Var}(\mathbf{1}_{p}^{\prime}U_{3})\lesssim\frac{2}{n^{2}\bar{N}^{2}}\sum_{1\leq
k\neq\ell\leq K}\sum_{i\in S_{k}}\sum_{m\in
S_{\ell}}\sum_{j=1}^{p}N_{i}N_{m}\Omega_{ij}\Omega_{mj}=\Theta_{n3}.$ (C.14)

Similarly for $U_{4}$, we have by the Proof of Lemma A.5 that

$\displaystyle\mathrm{Var}({\bf 1}_{p}^{\prime}U_{4})$
$\displaystyle=4\sum_{k=1}^{K}\sum_{\begin{subarray}{c}i\in S_{k},m\in
S_{k}\\\
i<m\end{subarray}}\kappa_{im}\Bigl{(}\sum_{j}\Omega_{ij}\Omega_{mj}+\delta_{im}\Bigr{)}$
(C.15) $\displaystyle\lesssim\sum_{k=1}^{K}\sum_{\begin{subarray}{c}i\in
S_{k},m\in S_{k}\\\
i<m\end{subarray}}\kappa_{im}\sum_{j}\Omega_{ij}\Omega_{mj}=\Theta_{n4}.$
(C.16)

Above we use that $|\delta_{im}|\leq\sum_{j}\Omega_{ij}\Omega_{mj}$ and recall
that
$\kappa_{im}=(\frac{1}{n_{k}\bar{N}_{k}}-\frac{1}{n\bar{N}})^{2}N_{i}N_{m}$.

Observe that by Lemma 2.1,

$\displaystyle\Theta_{n1}=4\sum_{k=1}^{K}\sum_{j=1}^{p}n_{k}\bar{N}_{k}(\mu_{kj}-\mu_{j})^{2}\mu_{kj}\lesssim\max_{k}\|\mu_{k}\|_{\infty}\cdot\rho^{2}=\max_{k}\|\mu_{k}\|_{\infty}\cdot\mathbb{E}\,T.$
(C.17)

Since (3.1) holds, Lemma A.6 applies and

$\displaystyle\Theta_{n_{2}}+\Theta_{n3}+\Theta_{n4}\asymp\sum_{k}\|\mu_{k}\|^{2}.$
(C.18)

Combining (C.11), (C.17), and (C.18) proves the theorem. ∎

###  C.3 Proof of Theorem 3.4

To prove Theorem 3.4, we must prove the following claims:

  1. (a)

Under the alternative hypothesis, $\psi\to\infty$ in probability.

  2. (b)

For any fixed $\alpha\in(0,1)$, the level-$\alpha$ DELVE test has an
asymptotic level of $\alpha$ and an asymptotic power of $1$.

  3. (c)

If we choose $\alpha=\alpha_{n}$ such that $\alpha_{n}\to 0$ and
$1-\Phi(\mathrm{SNR}_{n})=o(\alpha_{n})$, where $\Phi$ is the CDF of $N(0,1)$,
then the sum of type I and type II errors of the DELVE test converges to $0$.

We show the first claim, that $\psi\to\infty$, under the alternative
hypothesis and the conditions of Theorem 3.4. In particular, recall we assume
that

$\displaystyle\frac{\rho^{2}}{\sqrt{\sum_{k=1}^{K}\|\mu_{k}\|^{2}}}=\frac{n\bar{N}\|\mu\|^{2}\omega_{n}^{2}}{\sqrt{\sum_{k=1}^{K}\|\mu_{k}\|^{2}}}\to\infty.$
(C.19)

Our first goal is to show that

$\displaystyle
T/\sqrt{\mathrm{Var}(T)}\stackrel{{\scriptstyle\mathbb{P}}}{{\to}}\infty$
(C.20)

under the alternative. By Chebyshev’s inequality, it suffices to show that

$\displaystyle\mathbb{E}\,T\gg\sqrt{\mathrm{Var}(T)}.$ (C.21)

By Theorem 3.3,

$\displaystyle\mathrm{Var}(T)\lesssim\sum_{k}\|\mu_{k}\|^{2}+\max_{k}\|\mu_{k}\|_{\infty}\cdot\mathbb{E}T=\sum_{k}\|\mu_{k}\|^{2}+\max_{k}\|\mu_{k}\|_{\infty}\cdot\rho^{2}$
(C.22)

By (C.19),

$\mathbb{E}T=\rho^{2}\gg\sqrt{\sum_{k=1}^{K}\|\mu_{k}\|^{2}}\geq\max_{1\leq
k\leq K}\|\mu_{k}\|_{\infty}.$

Therefore,

$\displaystyle\sqrt{\max_{1\leq k\leq
K}\|\mu_{k}\|_{\infty}}\cdot\rho\ll\rho^{2}=\mathbb{E}T.$ (C.23)

Moreover, by (C.19),

$\displaystyle\sum_{k}\|\mu_{k}\|^{2}\ll\rho^{4}=(\mathbb{E}T)^{2}.$ (C.24)

Combining (C.22), (C.23), and (C.24) implies (C.20).

Next we show that $V>0$ with high probability (i.e., with probability tending
to $1$ as $n\bar{N}\to\infty$). Recall that by Lemmas A.6, A.10, and A.11,

$\displaystyle\mathbb{E}V$
$\displaystyle=\Theta_{n2}+\Theta_{n3}+\Theta_{n4}\gtrsim\sum_{k}\|\mu_{k}\|^{2}>0,\,\text{
and}$ (C.25) $\displaystyle\mathrm{Var}(V)$
$\displaystyle\lesssim\sum_{k}\frac{\|\mu_{k}\|^{2}}{n_{k}^{2}\bar{N}_{k}^{2}}\vee\sum_{k}\frac{\|\mu_{k}\|_{3}^{3}}{n_{k}\bar{N}_{k}}.$
(C.26)

Using this, the Markov inequality, and (3.4), we have

$\displaystyle\mathbb{P}\bigl{(}V<\mathbb{E}[V]/2\bigr{)}\leq\mathbb{P}\bigl{(}|V-\mathbb{E}[V]|\geq\mathbb{E}[V]/2\bigr{)}\leq\frac{4\mathrm{Var}(V)}{(\mathbb{E}[V])^{2}}=o(1),$
(C.27)

which implies that $V>0$ with high probability.

To finish the proof of the first claim, note that the assumptions of
Proposition A.2 are satisfied and we have
$V/\mathrm{Var}(T)=O_{\mathbb{P}}(1)$. By this, (C.20), and (C.27), we have

$\displaystyle\psi=\frac{T{\bf
1}_{V>0}}{\sqrt{V}}=\frac{\sqrt{\mathrm{Var}(T)}}{\sqrt{V}}\cdot\frac{T}{\sqrt{\mathrm{Var}(T)}}\cdot{\bf
1}_{V>0}\gtrsim\frac{T}{\sqrt{\mathrm{Var}(T)}}\to\infty$

in probability.

The second claim follows directly from the first claim and Theorem 3.2.

To prove the third claim, by Chebyshev’s inequality and
$T/\sqrt{\mathrm{Var}(T)}\to\infty$, it follows that
$T>(1/2)\mathbb{E}T=(1/2)\rho^{2}$ with high probability as
$n\bar{N}\to\infty$. By a similar Chebyshev argument as above, it also holds
that $V<(3/2)\mathbb{E}V$ with high probability as $n\bar{N}\to\infty$. Recall
that
$\mathbb{E}V=\Theta_{n2}+\Theta_{n3}+\Theta_{n4}\lesssim\sum_{k}\|\mu_{k}\|^{2}$
by Lemmas A.6 and A.10. Thus, with high probability as $n\bar{N}\to\infty$, we
have

$\psi=T{\bf
1}_{V>0}/\sqrt{V}\gtrsim\rho^{2}/\sqrt{\mathbb{E}V}\gtrsim\frac{n\bar{N}\|\mu\|^{2}\omega_{n}^{2}}{\sqrt{\sum_{k}\|\mu_{k}\|^{2}}}=\mathrm{SNR}_{n}.$

Choosing $\alpha_{n}$ as specified yield the third claim. The proof is
complete since all three claims are established.

∎

###  C.4 Proof of Theorem 3.5

Without loss of generality, we assume $p$ is even and write $m=p/2$. Let
$\mu\in\mathbb{R}^{m}$ be a nonnegative vector with $\|\mu\|_{1}=1/2$ . Let
$\tilde{\mu}=(\mu^{\prime},\mu^{\prime})^{\prime}\in\mathbb{R}^{p}$. We
consider the null hypothesis:

$H_{0}:\qquad\Omega_{i}=\tilde{\mu},\qquad 1\leq i\leq n.$ (C.28)

We pair it with a random alternative hypothesis. Let
$b_{1},b_{2},\ldots,b_{m}$ be a collection of i.i.d. Rademacher variables. Let
$z_{1},z_{2},\ldots,z_{K}$ denote an independent collection of i.i.d.
Rademacher random variables conditioned on the event $|\sum_{k}z_{k}|\leq
100\sqrt{K}.$ For a properly small sequence $\omega_{n}>0$ of positive
numbers, let

$H_{1}:\qquad\Omega_{ij}=\begin{cases}\mu_{j}\bigl{(}1+\omega_{n}(n_{k}\bar{N}_{k})^{-1}\big{(}\frac{1}{K}\sum_{k\in
K}n_{k}\bar{N}_{k}\big{)}z_{k}b_{j}\bigr{)},&\mbox{if }1\leq j\leq m,i\in
S_{k}\cr\tilde{\mu}_{j}\bigl{(}1-\omega_{n}(n_{k}\bar{N}_{k})^{-1}\big{(}\frac{1}{K}\sum_{k\in
K}n_{k}\bar{N}_{k}\big{)}z_{k}b_{j-m}\bigr{)},&\mbox{if }m+1\leq j\leq 2m,i\in
S_{k}\end{cases}$ (C.29)

In this section we slightly abuse notation, using $\omega_{n}$ to refer to the
(deterministic) sequence above and reserving $\omega(\Omega)$ for the random
quantity

$\omega(\Omega)=\sqrt{\frac{1}{n\bar{N}\|\mu\|^{2}}\sum_{k=1}^{K}n_{k}\bar{N}_{k}\|\mu_{k}-\mu\|^{2}}.$
(C.30)

As long as

$\omega_{n}\leq\frac{\min_{k}n_{k}\bar{N}_{k}}{\frac{1}{K}\sum_{k\in[K]}n_{k}\bar{N}_{k}}=\frac{\min_{k}n_{k}\bar{N}_{k}}{n\bar{N}/K},$

then $\Omega_{ij}\geq 0$ for all $i\in[n],j\in[p]$. Furthermore, for each
$1\leq i\leq n$, we have $\|\Omega_{i}\|_{1}=2\|\mu\|_{1}=1$. We suppose there
exists a constant $c\in(0,1)$ such that

$\displaystyle cK^{-1}n\bar{N}\leq n_{k}\bar{N}_{k}$ $\displaystyle\leq
c^{-1}K^{-1}n\bar{N}\quad\text{for all }k\in[K]$ (C.31)

With (C.31) in hand, we may assume without loss of generality that

$\displaystyle\omega_{n}\leq c/2$ (C.32)

This assumption implies that (C.29) is well-defined and moreover
$\Omega_{ij}\asymp\mu_{j}$.

Next we characterize the random quantity $\omega(\Omega)$ in terms of
$\omega_{n}$.

######  Lemma C.1.

Let $\omega^{2}(\Omega)$ be as in (C.30). When $\Omega$ follows Model (C.29),
there exists a constant $c_{1}\in(0,1)$ such that
$c_{1}\omega_{n}^{2}\leq\omega^{2}(\Omega)\leq c_{1}^{-1}\omega_{n}^{2}$ with
probability $1$.

The proof of Lemma C.1 is given in Section C.4.1. By Lemma C.1, under the
model (C.29) it holds with probability $1$ that

$\displaystyle\frac{n\bar{N}\|\mu\|^{2}\omega^{2}(\Omega)}{\sqrt{\sum_{k=1}^{K}\|\mu_{k}\|^{2}}}\asymp
K^{-1/2}n\bar{N}\|\mu\|\omega_{n}^{2}.$ (C.33)

Above we use that $\Omega_{ij}\asymp\mu_{j}$ , since we assume (C.32)

We also require Proposition C.1 below, whose proof is given in Section C.4.2.

######  Proposition C.1.

Suppose that (C.31) and (C.32) hold. Consider the pair of hypotheses in
(C.28)-(C.29) and let $\mathbb{P}_{0}$, and $\mathbb{P}_{1}$ be the respective
probability measures. If

$\frac{n\bar{N}\|\mu\|^{2}\omega^{2}(\Omega)}{\sqrt{\sum_{k=1}^{K}\|\mu_{k}\|^{2}}}\asymp
K^{-1/2}n\bar{N}\|\mu\|\omega_{n}^{2}\to 0,$

then the chi-square distance between $\mathbb{P}_{0}$ and $\mathbb{P}_{1}$
converges to $0$.

Now we prove Theorem 3.5. Let $\delta_{n}$ denote an arbitrary sequence
tending to $0$. Without loss of generality, we may assume that $\delta_{n}\leq
c^{*}$ for a small absolute constant $c^{*}\in(0,1)$. Note that
$K^{-1/2}n\bar{N}\geq 1$ since $K\leq n$. Thus for appropriate choice of
sequences of $\mu=\mu_{n}$ and $\omega_{n}\leq c/2$ in models (C.28), (C.29)
and applying (C.33), we obtain

$\displaystyle
2\delta_{n}\geq\frac{n\bar{N}\|\mu\|^{2}\omega^{2}(\Omega)}{\sqrt{\sum_{k=1}^{K}\|\mu_{k}\|^{2}}}\geq\delta_{n}.$
(C.34)

Recall the definitions of $\mathcal{Q}^{*}_{0n}$ and $\mathcal{Q}^{*}_{1n}$ in
(3.8). Let $\Pi$ denote the distribution on
$\xi=\\{(N_{i},\Omega_{i},\ell_{i})\\}\in\mathcal{Q}^{*}_{1n}$ induced by
(C.29). Let $\xi_{0}$ denote the parameter associated to the simple null
hypothesis in (C.28) associated to our choice of $\mu$ and $\omega_{n}$
satisfying (C.34). We have by standard manipulations,

$\displaystyle\mathcal{R}(\mathcal{Q}^{*}_{0n},\mathcal{Q}^{*}_{1n})$
$\displaystyle:=\inf_{\Psi\in\\{0,1\\}}\Bigl{\\{}\sup_{\xi\in{\cal
Q}_{0n}^{*}(c_{0},\epsilon_{n})}\mathbb{P}_{\xi}(\Psi=1)+\sup_{\xi\in{\cal
Q}_{1n}^{*}(\delta_{n};c_{0},\epsilon_{n})}\mathbb{P}_{\xi}(\Psi=0)\Bigr{\\}}$
$\displaystyle=\inf_{\Psi\in\\{0,1\\}}\Bigl{\\{}\sup_{\xi\in{\cal
Q}_{0n}^{*}(c_{0},\epsilon_{n}),\xi^{\prime}\in{\cal
Q}_{1n}^{*}(\delta_{n};c_{0},\epsilon_{n})}\big{[}\mathbb{P}_{\xi}(\Psi=1)+\mathbb{P}_{\xi}(\Psi=0)\big{]}$
$\displaystyle\geq\inf_{\Psi\in\\{0,1\\}}\Bigl{\\{}\sup_{\xi\in{\cal
Q}_{0n}^{*}(c_{0},\epsilon_{n})}\mathbb{E}_{\xi^{\prime}\sim\Pi}\bigg{[}\mathbb{P}_{\xi}(\Psi=1)+\mathbb{P}_{\xi^{\prime}}(\Psi=0)\bigg{]}\bigg{\\}}$
$\displaystyle\geq\inf_{\Psi\in\\{0,1\\}}\Bigl{\\{}\mathbb{E}_{\xi^{\prime}\sim\Pi}\bigg{[}\mathbb{P}_{\xi_{0}}(\Psi=1)+\mathbb{P}_{\xi^{\prime}}(\Psi=0)\bigg{]}\bigg{\\}}$
$\displaystyle=\inf_{\Psi\in\\{0,1\\}}\Bigl{\\{}\mathbb{P}_{0}(\Psi=1)+\mathbb{P}_{1}(\Psi=0)\bigg{\\}}.$

In the last line we recall the definition of $\mathbb{P}_{0}$ and
$\mathbb{P}_{1}$ in (C.28) and (C.29), noting that for all events $E$,

$\mathbb{P}_{1}(E)=\mathbb{E}_{\xi^{\prime}\sim\pi}\,\mathbb{P}_{\xi^{\prime}}(E).$

Next, by the Neyman–Pearson lemma and the standard inequality
$\text{TV}(P,Q)\leq\sqrt{\chi^{2}(P,Q)}$ (see e.g. Chapter 2 of Tsybakov
(2008)),

$\displaystyle\mathcal{R}(\mathcal{Q}^{*}_{0n},\mathcal{Q}^{*}_{1n})$
$\displaystyle\geq\inf_{\Psi\in\\{0,1\\}}\Bigl{\\{}\mathbb{P}_{0}(\Psi=1)+\mathbb{P}_{1}(\Psi=0)\bigg{\\}}$
$\displaystyle=1-\text{TV}\big{(}\mathbb{P}_{0},\mathbb{P}_{1}\big{)}\geq
1-\sqrt{\chi^{2}(\mathbb{P}_{0},\mathbb{P}_{1})}.$

By Proposition C.1, as $\delta_{n}\to 0$ we have
$\chi^{2}(\mathbb{P}_{0},\mathbb{P}_{1})\to 0$ and thus
$\mathcal{R}(\mathcal{Q}^{*}_{0n},\mathcal{Q}^{*}_{1n})\to 1$, as desired.

∎

####  C.4.1 Proof of Proposition C.1

Next, we perform a change of parameters that preserves the signal strength and
chi-squared distance. The testing problem (C.28) and (C.29) has parameters
$\Omega_{ij},N_{i},\bar{N}_{k},n_{k},n,$ and $K$. Let $\mathbb{P}_{0}$ and
$\mathbb{P}_{1}$ denote the distributions corresponding to the null and
alternative hypotheses, respectively. For each $k\in[K]$, we combine all
documents in sample $k$ to obtain new null and alternative distributions
$\tilde{\mathbb{P}}_{0}$ and $\tilde{\mathbb{P}}_{1}$ with parameters
$\tilde{\Omega}_{ij},\tilde{N}_{i},\bar{\tilde{N}}_{i},\tilde{n}_{i},\tilde{n},$
and $\tilde{K}$ such that

$\displaystyle\tilde{K}$ $\displaystyle=K=\tilde{n}$
$\displaystyle\tilde{N}_{i}$ $\displaystyle=n_{i}\bar{N}_{i}$ for
$i\in[\tilde{K}]$ $\displaystyle\bar{\tilde{N}}_{i}$
$\displaystyle\equiv\tilde{N}_{i}$ for $i\in[\tilde{K}]$
$\displaystyle\tilde{n}_{i}$ $\displaystyle=1$ $\displaystyle\quad\text{ for
$i\in[\tilde{K}]$ }.$ (C.35)

For notational ease, we define
$\tilde{N}:=\bar{\tilde{N}}=\frac{1}{K}\sum_{k\in[K]}n_{k}\bar{N}_{k}$.
Furthermore, we have $\tilde{\Omega}_{i}=\mu$ for all $i\in[\tilde{n}]$ under
the null $\tilde{\Omega}_{i}=\mu_{i}$ for all $i\in[\tilde{n}]$ under the
alternative. Explicitly, in the reparameterized model, we have the null
hypothesis

$H_{0}:\qquad\Omega_{i}=\tilde{\mu},\qquad 1\leq i\leq n.$ (C.36)

and alternative hypothesis

$H_{1}:\qquad\Omega_{ij}=\begin{cases}\mu_{j}\bigl{(}1+\omega_{n}\tilde{N}_{i}^{-1}\tilde{N}z_{i}b_{j}\bigr{)},&\mbox{if
}1\leq j\leq
m,\cr\tilde{\mu}_{j}\bigl{(}1-\omega_{n}\tilde{N}_{i}^{-1}\tilde{N}z_{i}b_{j-m}\bigr{)},&\mbox{if
}m+1\leq j\leq 2m.\end{cases}$ (C.37)

for all $i\in[\tilde{K}]=[K]=[\tilde{n}]$. Observe that the likelihood ratio
is preserved:
$\frac{d\mathbb{P}_{0}}{d\mathbb{P}_{1}}=\frac{\tilde{d}\mathbb{P}_{0}}{d\tilde{\mathbb{P}}_{1}}$
and also $\omega(\Omega)=\omega(\widetilde{\Omega})$. For simplicity we work
with this reparameterized model in this proof.

If $z_{1},\ldots,z_{\tilde{n}}$ are independent Rademacher random variables
then with probability at least $1/2$ it holds that

$\displaystyle|\sum_{i}z_{i}|\leq 100\sqrt{\tilde{n}}$ (C.38)

by Hoeffding’s inequality. Recall that our random model is defined in (C.29)
where (i) $z_{1},\ldots,z_{\tilde{n}}$ are independent Rademacher random
variables conditioned on the event $|\sum_{i}z_{i}|\leq 100\sqrt{\tilde{n}}$,
and (ii) $b_{1},\ldots,b_{m}$ are independent Rademacher random variables.

Now we study $\omega^{2}(\widetilde{\Omega})$. For each $1\leq j\leq m$, we
have
$\widetilde{\Omega}_{ij}=\mu_{j}(1+\omega_{n}\tilde{N}_{i}^{-1}\tilde{N}z_{i}b_{j})$.
Define
$\eta_{j}=(\tilde{n}\tilde{N})^{-1}\sum_{i=1}^{\tilde{n}}\tilde{N}_{i}\widetilde{\Omega}_{ij}=\mu_{j}(1+\omega_{n}\bar{z}b_{j})$
for $1\leq j\leq m$ and
$\eta_{j}=(\tilde{n}\tilde{N})^{-1}\sum_{i=1}^{\tilde{n}}\tilde{N}_{i}\widetilde{\Omega}_{ij}=\tilde{\mu}_{j}(1-\omega_{n}\bar{z}b_{j})$
for $m<j\leq 2m$. We have

$\displaystyle\sum_{i=1}^{\tilde{n}}\sum_{j=1}^{p}\tilde{N}_{i}(\widetilde{\Omega}_{ij}-\eta_{j})^{2}$
$\displaystyle=2\sum_{i=1}^{\tilde{n}}\sum_{j=1}^{m}\tilde{N}_{i}\cdot\mu_{j}^{2}\omega_{n}^{2}\frac{\tilde{N}^{2}}{\tilde{N}_{i}^{2}}(z_{i}-\bar{z})^{2}b_{j}^{2}$
$\displaystyle=2\omega_{n}^{2}\tilde{N}^{2}\|\mu\|^{2}\sum_{i=1}^{\tilde{n}}\tilde{N}_{i}^{-1}(z_{i}-\bar{z})^{2}.$

By (C.38), $|\bar{z}|\leq 100\sqrt{\tilde{n}}$. Thus $|z_{i}-\bar{z}|\asymp
1$. Write
$\tilde{N}_{*}=(\tilde{n}^{-1}\sum_{i=1}^{\tilde{n}}\tilde{N}_{i}^{-1})$. It
follows that

$\sum_{i=1}^{\tilde{n}}\sum_{j=1}^{p}\tilde{N}_{i}(\widetilde{\Omega}_{ij}-\eta_{j})^{2}\asymp\omega_{n}^{2}\tilde{N}^{2}\|\mu\|^{2}\cdot\tilde{n}\tilde{N}_{*}^{-1}.$

Note that $\tilde{N}\geq\tilde{N}_{*}$. Additionally, by assumption (C.31),
$\tilde{N}_{i}\asymp\tilde{N}\leq c^{-1}\tilde{N}_{*}$. It follows that

$\sum_{i=1}^{\tilde{n}}\sum_{j=1}^{p}\tilde{N}_{i}(\widetilde{\Omega}_{ij}-\eta_{j})^{2}\asymp\tilde{n}\tilde{N}\|\mu\|^{2}\omega_{n}^{2}.$
(C.39)

Moreover,
$\|\eta\|^{2}=\sum_{j=1}^{p}\mu_{j}^{2}(1+\omega_{n}\bar{z}b_{j})^{2}$. By our
conditioning on the event in (C.38),

$|\omega_{n}\bar{z}b_{j}|\lesssim\omega_{n}{\tilde{n}}^{-1/2}.$

Since $\omega_{n}\leq 1$ and $\sum_{j}b_{j}=0$, we have

$\displaystyle\|\eta\|^{2}=\|\mu\|^{2}+\sum_{j=1}^{p}\mu_{j}^{2}\omega_{n}^{2}\bar{z}^{2}=\|\mu\|^{2}[1+O(\tilde{n}^{-1})]\asymp\|\mu\|^{2}.$
(C.40)

Hence

$\omega^{2}(\widetilde{\Omega})=\omega^{2}(\Omega)\asymp\omega_{n}^{2},\qquad\mbox{where
recall}\quad\omega(\widetilde{\Omega})=\frac{\sum_{i=1}^{\tilde{n}}\sum_{j=1}^{p}\tilde{N}_{i}(\widetilde{\Omega}_{ij}-\eta_{j})^{2}}{{\tilde{n}}\tilde{N}\|\eta\|^{2}}.$
(C.41)

This finishes the proof. ∎

####  C.4.2 Proof of Proposition C.1

In this proof, we continue to employ the reparametrization in (C.35). As
discussed there, this reparametrization preserves the likelihood ratio and
thus the chi-square distance.

By definition,
$\chi^{2}(\mathbb{P}_{0},\mathbb{P}_{1})=\int(\frac{d\mathbb{P}_{1}}{d\mathbb{P}_{0}})^{2}d\mathbb{P}_{0}-1$.
It suffices to show that

$\int\Bigl{(}\frac{d\mathbb{P}_{1}}{d\mathbb{P}_{0}}\Bigr{)}^{2}d\mathbb{P}_{0}=1+o(1).$
(C.42)

From the density of of multinomial distribution,
$d\mathbb{P}_{0}=\prod_{i,j}\tilde{\mu}_{j}^{X_{ij}}$, and
$d\mathbb{P}_{1}=\mathbb{E}_{b,z}[\prod_{i,j}\widetilde{\Omega}_{ij}^{X_{ij}}]$.
It follows that

$\frac{d\mathbb{P}_{1}}{d\mathbb{P}_{0}}=\mathbb{E}_{b,z}\biggl{[}\prod_{i=1}^{\tilde{n}}\prod_{j=1}^{p}\Bigl{(}\frac{\widetilde{\Omega}_{ij}}{\tilde{\mu}_{j}}\Bigr{)}^{X_{ij}}\biggr{]}.$

Let $b^{(0)}=(b^{(0)}_{1},\ldots,b^{(0)}_{m})^{\prime}$ and
$z^{(0)}=(z^{(0)}_{1},\cdots,z^{(0)}_{\tilde{n}})^{\prime}$ be independent
copies of $b$ and $z$. We construct $\widetilde{\Omega}^{(0)}_{ij}$ similarly
as in (C.37). It is seen that

$\displaystyle\int\Bigl{(}\frac{d\mathbb{P}_{1}}{d\mathbb{P}_{0}}\Bigr{)}^{2}d\mathbb{P}_{0}$
$\displaystyle=\mathbb{E}_{X}\mathbb{E}_{b,z,b^{(0)},z^{(0)}}\biggl{[}\prod_{i=1}^{\tilde{n}}\prod_{j=1}^{p}\Bigl{(}\frac{\widetilde{\Omega}_{ij}\widetilde{\Omega}^{(0)}_{ij}}{\tilde{\mu}^{2}_{j}}\Bigr{)}^{X_{ij}}\biggr{]}$
(C.43)
$\displaystyle=\mathbb{E}_{b,z,b^{(0)},z^{(0)}}\biggl{\\{}\prod_{i=1}^{\tilde{n}}\mathbb{E}_{X_{i}}\biggl{[}\prod_{j=1}^{p}\Bigl{(}\frac{\widetilde{\Omega}_{ij}\widetilde{\Omega}^{(0)}_{ij}}{\tilde{\mu}^{2}_{j}}\Bigr{)}^{X_{ij}}\biggr{]}\biggr{\\}}$
(C.44)
$\displaystyle=\mathbb{E}_{b,z,b^{(0)},z^{(0)}}\biggl{\\{}\prod_{i=1}^{\tilde{n}}\biggl{(}\sum_{j=1}^{p}\tilde{\mu}_{j}\cdot\frac{\widetilde{\Omega}_{ij}\widetilde{\Omega}^{(0)}_{ij}}{\tilde{\mu}_{j}^{2}}\Bigr{)}^{\tilde{N}_{i}}\biggr{]}\biggr{\\}}$
(C.45) $\displaystyle=\mathbb{E}[\exp(M)],\quad\mbox{with}\quad
M:=\sum_{i=1}^{\tilde{n}}\tilde{N}_{i}\log\Bigl{(}\sum_{j=1}^{p}\tilde{\mu}_{j}^{-1}\widetilde{\Omega}_{ij}\widetilde{\Omega}^{(0)}_{ij}\Bigr{)}.$
(C.46)

Here, the third line follows from the moment generating function of a
multinomial distribution. We plug in the expression of
$\widetilde{\Omega}_{ij}$ in (C.29). By direct calculations,

$\displaystyle\sum_{j=1}^{p}\tilde{\mu}_{j}^{-1}\widetilde{\Omega}_{ij}\widetilde{\Omega}^{(0)}_{ij}$
$\displaystyle=\sum_{j=1}^{m}\mu_{j}\bigl{(}1+\omega_{n}\tilde{N}_{i}^{-1}\tilde{N}z_{i}b_{j}\bigr{)}\bigl{(}1+\omega_{n}\tilde{N}_{i}^{-1}\tilde{N}z^{(0)}_{i}b^{(0)}_{j}\bigr{)}$