$\displaystyle\rho_{r,\phi}^{\delta}(X_{[1,n]})-\rho_{r,\phi}^{\delta}(X_{[1,n)})-\rho_{r,\phi}^{\delta}(Y_{[1,n]})+\rho_{r,\phi}^{\delta}(Y_{[1,n)})$
$\displaystyle=\mathfrak{R}_{X}^{(3)}-\mathfrak{R}_{Y}^{(3)}+\sum_{j=2}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2)}-\mathfrak{R}_{Y_{j}}^{(3,2)}\right],$

where
$\mathfrak{R}_{W_{j}}^{(3,2)}=\frac{1}{2}\mathfrak{R}_{W_{j}}^{(3,2,1)}+\mathfrak{R}_{W_{j}}^{(3,2,2)}+\mathfrak{R}_{W_{j}}^{(3,2,3)}$,
and

$\displaystyle\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X_{[1,n]})]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y_{[1,n]})]\right\rvert$
$\displaystyle\leq\left\lvert\rho_{r,\phi}^{\delta}(X_{[1,n)})-\rho_{r,\phi}^{\delta}(Y_{[1,n)})\right\rvert+\left\lvert\rho_{r,\phi}^{\delta}(X_{[1,n]})-\rho_{r,\phi}^{\delta}(X_{[1,n)})-\rho_{r,\phi}^{\delta}(Y_{[1,n]})+\rho_{r,\phi}^{\delta}(Y_{[1,n)})\right\rvert$
$\displaystyle\leq\left\lvert\mathfrak{R}_{X}^{(3)}-\mathfrak{R}_{Y}^{(3)}\right\rvert+\sum_{j=1}^{n-1}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,1)}-\mathfrak{R}_{Y_{j}}^{(3,1)}\right]\right\rvert+\sum_{j=2}^{n-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2)}-\mathfrak{R}_{Y_{j}}^{(3,2)}\right]\right\rvert.$

Remainder lemma. Similar to Section 4.1, the remainder terms
$\mathfrak{R}_{W}^{(3)}$, $\mathfrak{R}_{W_{j}}^{(3,1)}$ and
$\mathfrak{R}_{W_{j}}^{(3,2)}$ are upper bounded by conditional anti-
concentration probability bounds. For $q>0$, let

$\widetilde{L}_{q,j}\equiv\sum_{j^{\prime}=j-3}^{j+3}L_{q,j^{\prime}}\enskip\text{and}\enskip\widetilde{L}_{q,[k]_{j-2}}\equiv\sum_{k^{\prime}=k-3}^{k+3}L_{q,[k^{\prime}]_{j-2}},$

where $[k^{\prime}]_{j-2}$ is $k^{\prime}$ modulo $j-2$, and
$\widetilde{\nu}_{q,j}$ and $\widetilde{\nu}_{q,[k]_{j-2}}$ are similarly
defined.

######  Lemma A.3.

Suppose that Assumption (MIN-EV) holds. For $W$ representing either $X$ or $Y$
and $j\in\mathbb{Z}_{n}$,

$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W}^{(3)}\right]\right\rvert$
$\displaystyle\leq
C\left[\widetilde{L}_{3,n}+\phi^{q-3}\widetilde{\nu}_{q,n}\right]\frac{(\log(ep))^{3/2}}{\delta^{3}}\min\\{\kappa_{(2,n-1)}(\delta_{0}^{o})+\kappa^{o}_{n},1\\},$
$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W_{j}}^{(3,1)}\right]\right\rvert$
$\displaystyle\leq
C\left[\widetilde{L}_{3,j}+\phi^{q-3}\widetilde{\nu}_{q,j}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\min\\{\kappa_{[1,j-3)}(\delta_{n-j}^{o})+\kappa^{o}_{j},1\\},$
$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W_{j}}^{(3,2)}\right]\right\rvert$
$\displaystyle\leq C\left[\begin{aligned}
\widetilde{L}_{3,j}+\widetilde{L}_{3,n}+\phi^{q-3}\left(\widetilde{\nu}_{q,j}+\nu_{q,n}\right)\end{aligned}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}$
$\displaystyle\quad\times\min\\{\kappa_{(1,j-2)}(\delta_{n-j}^{o})+\kappa^{o}_{j},1\\},$

where
$\delta_{n-j}^{2}\equiv\delta^{2}+\underline{\sigma}^{2}\max\\{n-j,0\\}$,
$\delta^{o}_{n-j}\equiv 12\delta_{n-j}\sqrt{\log(pn)}$ and
$\kappa^{o}_{j}\equiv\frac{\delta_{n-j}\log(ep)}{\sigma_{\min}\sqrt{\max\\{j,1\\}}}$,
as long as $\delta\geq\sigma_{\min}$ and $\phi\delta\geq\frac{1}{\log(ep)}$.

Permutation argument. We apply the permutation arguement to Eq. 13 as in Eq.
30.

$\displaystyle\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X_{[1,n]})]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y_{[1,n]})]\right\rvert$
$\displaystyle\leq\frac{1}{n}\sum_{j_{o}=1}^{n}\sum_{j=1}^{n-1}\left|\mathbb{E}\left[\rho_{r,\phi}^{\delta}(X_{(j_{o},j_{o}+j)}+X_{j_{o}+j}+Y_{(j_{o}+j,j_{o}+n]})\right.\right.$
$\displaystyle\hskip
144.54pt\left.\left.-\rho_{r,\phi}^{\delta}(X_{(j_{o},j_{o}+j)}+Y_{j_{o}+j}+Y_{(j_{o}+j,j_{o}+n]})\right]\right|,$

where the indices of $X$ and $Y$ are defined in $\mathbb{Z}_{n}$. Together
with the results in Lemma A.3,

$\displaystyle\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X_{[1,n]})]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y_{[1,n]})]\right\rvert$
$\displaystyle\leq\frac{C}{n}\sum_{j_{o}=1}^{n}\left[\widetilde{L}_{3,j_{o}}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}}\right]\frac{(\log(ep))^{3/2}}{\delta^{3}}\min\\{1,\kappa_{j_{o}+(2,n-1)}(\delta_{0}^{o})+\kappa^{o}_{n}\\}$
$\displaystyle\quad+\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j=2}^{n-2}\left[\widetilde{L}_{3,j_{o}}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\min\\{1,\kappa_{j_{o}+(1,j-2)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\quad+\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j=1}^{n-1}\left[\widetilde{L}_{3,j_{o}+j}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}+j}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\min\\{1,\kappa_{j_{o}+(1,j-3)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\},$

where $j_{o}+(2,n-1)$ is the shifted interval of $(2,n-1)$ by $j_{o}$ in
$\mathbb{Z}_{n}$, namely, $\\{[j_{o}+3]_{n},\dots,[j_{o}+n-2]_{n}\\}$.

Partitioning the sum. We partition the summation over $j$ at
$J_{n}=n(1-\frac{\sigma_{\min}^{2}}{\underline{\sigma}^{2}\log^{2}(4ep)})$. A
notable distinction from Section 4.1 is that we should also take averages over
$j_{o}$ alongside the summations over $j$. For $j<J_{n}$,

$\displaystyle\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j<J_{n}}\left[\widetilde{L}_{3,j_{o}+j}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}+j}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\min\\{1,\kappa_{j_{o}+(1,j-3)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\leq\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j<J_{n}}\left[\widetilde{L}_{3,j_{o}+j}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}+j}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}$
$\displaystyle=C\sum_{j<J_{n}}\left[\overline{L}_{3}+\phi^{q-3}\overline{\nu}_{q}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}$
$\displaystyle\leq\frac{C}{\sqrt{n}}\left[\overline{L}_{3}+\phi^{q-3}\overline{\nu}_{q}\right]\frac{(\log(ep))^{5/2}}{\underline{\sigma}^{2}\sigma_{\min}},$

because of Eq. 16. For $j\geq J_{n}$,

$\displaystyle\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j\geq
J_{n}}\left[\widetilde{L}_{3,j_{o}+j}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}+j}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\min\\{1,\kappa_{j_{o}+(0,+j)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\leq\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j\geq
J_{n}}\left[\widetilde{L}_{3,j_{o}+j}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}+j}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\kappa_{(j_{o},j_{o}+j)}(\delta_{n-j}^{o})$
$\displaystyle\quad+C\sum_{j\geq
J_{n}}\left[\overline{L}_{3}+\phi^{q-3}\overline{\nu}_{q}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\kappa^{o}_{j-1}.$

The last term is upper bounded by

$\displaystyle C\sum_{j\geq
J_{n}}\left[\overline{L}_{3}+\phi^{q-3}\overline{\nu}_{q}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\kappa^{o}_{j-1}$
$\displaystyle\leq\frac{C}{\sqrt{n}}\left[\overline{L}_{3}+\phi^{q-3}\overline{\nu}_{q}\right]\frac{(\log(ep))^{5/2}}{\underline{\sigma}^{2}\sigma_{\min}}\log\left(1+\frac{\sqrt{n}\underline{\sigma}}{\delta}\right)$

because of Eq. 17. In sum, we obtain the following induction from
$\kappa_{I}(\delta)$ for $I\subset\mathbb{Z}_{n}$ to

$\mu_{[1,n]}=\mu(X_{[1,n]},Y_{[1,n]}).$

######  Lemma A.4.

If Assumptions (MIN-VAR), (MIN-EV) and (VAR-EV) hold, then for any
$\delta\geq\sigma_{\min}$,

$\displaystyle\mu_{[1,n]}$
$\displaystyle\leq\frac{C}{\sqrt{n}}\frac{\delta\log(ep)+\sqrt{\log(ep)}/\phi}{\sigma_{\min}}+\frac{C}{\sqrt{n}}\left[\overline{L}_{3}+\phi^{q-3}\overline{\nu}_{q}\right]\frac{(\log(ep))^{5/2}}{\underline{\sigma}^{2}\sigma_{\min}}\log(en)$
$\displaystyle\quad+\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\left[\widetilde{L}_{3,j_{o}}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\kappa_{j_{o}+(3,j-2)}(\delta_{n-j}^{o})$
$\displaystyle\quad+\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n-1}\left[\widetilde{L}_{3,j_{o}+j}+\phi^{q-3}\widetilde{\nu}_{q,j_{o}+j}\right]\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\kappa_{j_{o}+(1,j-4)}(\delta_{n-j}^{o}),$

for some absolute constant $C>0$.

Anti-concentration inequality. We note the following monotonic property of
$\kappa$:

######  Lemma A.5.

Suppose that $I_{0}$ and $I$ are intervals in $\mathbb{Z}_{n}$ satisfying
$I^{\prime}\subset I$ and that $\delta^{\prime}\geq\delta>0$. Then,
$\kappa_{I}(\delta)\leq\kappa_{I^{\prime}}(\delta^{\prime})$.

###### Proof.

$\displaystyle\kappa_{[1,i]}(\delta)$
$\displaystyle=\sup_{r\in\mathbb{R}^{p}}\mathbb{P}[{X}_{I}\in
A_{r,\delta}|\mathscr{X}_{\mathbb{Z}_{n}\setminus I}]$
$\displaystyle=\sup_{r\in\mathbb{R}^{p}}\mathbb{E}\left[\mathbb{P}[{X}_{I^{\prime}}\in
A_{r-X_{I\setminus I^{\prime}},\delta}|\mathscr{X}_{\mathbb{Z}_{n}\setminus
I^{\prime}}]\big{|}\mathscr{X}_{\mathbb{Z}_{n}\setminus I}\right]$
$\displaystyle\leq\sup_{r\in\mathbb{R}^{p}}\mathbb{E}\left[\mathbb{P}[{X}_{[1,i_{o}]}\in
A_{r-X_{I\setminus
I^{\prime}},\delta^{\prime}}|\mathscr{X}_{\mathbb{Z}_{n}\setminus
I^{\prime}}]\big{|}\mathscr{X}_{\mathbb{Z}_{n}\setminus I}\right]$
$\displaystyle\leq\sup_{r\in\mathbb{R}^{p}}\mathbb{E}\left[\kappa_{I^{\prime}}(\delta^{\prime})\big{|}\mathscr{X}_{\mathbb{Z}_{n}\setminus
I}\right]=\kappa_{I^{\prime}}(\delta^{\prime}).$

∎

Thus, for any $\delta>0$,
$\kappa_{[1,i]}(\delta)\leq\sum_{j=1}^{i_{o}-1}\kappa_{[j,j+i_{o}+1]}$ where
$i_{o}=\lfloor\frac{i}{2}\rfloor$. Thus, we obtain the following averaged
version of Lemma 4.5.

######  Lemma A.6.

Suppose that Assumptions (MIN-VAR) and (MIN-EV) hold. For any $i\in[6,n)$ and
$\delta>0$,

$\displaystyle\kappa_{[1,i]}(\delta)$
$\displaystyle\leq\frac{C}{i_{o}-1}\sum_{j=1}^{i_{o}-1}\left(\frac{\sqrt{\log(ep)}}{\varepsilon}~{}(\nu_{1,j+1}+\nu_{1,j+i_{o}})~{}\kappa_{j+(1,i_{o})}(\varepsilon^{o})+\mu_{j+(1,i_{o})}\right)$
$\displaystyle\quad+\min\left\\{1,C\frac{\delta+2\varepsilon^{o}}{\sigma_{\min}}\sqrt{\frac{\log(ep)}{i_{o}-2}}\right\\}+\frac{C}{\sigma_{\min}}\overline{\nu}_{1,(1,i)}\sqrt{\frac{\log(ep)}{i_{o}-2}},$

where $i_{o}\equiv\lfloor\frac{i}{2}\rfloor$ and $\varepsilon^{o}\equiv
20\varepsilon\sqrt{\log(p(n_{o}-2))}$, as long as
$\varepsilon\geq\sigma_{\min}$

Dual Induction. Let our induction hypothesis on $n$ be

$\sqrt{n}\mu_{[1,n]}\leq\widetilde{\mu}_{1,n}\overline{L}_{3}+\widetilde{\mu}_{2,n}\overline{\nu}_{q}^{1/(q-2)},$
(HYP-BE-3)

where

$\displaystyle\widetilde{\mu}_{1,n}$
$\displaystyle=\mathfrak{C}_{1}\frac{(\log(ep))^{3/2}\sqrt{\log(pn)}}{\underline{\sigma}^{2}\sigma_{\min}}\log\left(en\right),$
$\displaystyle\widetilde{\mu}_{2,n}$
$\displaystyle=\mathfrak{C}_{2}\frac{\log(ep)\sqrt{\log(pn)}}{\underline{\sigma}^{2/(q-2)}\sigma_{\min}}\log\left(en\right)$

for universal constants $\mathfrak{C}_{1}$ and $\mathfrak{C}_{2}$ whose values
do not change in this subsection. If $\mathfrak{C}_{1},\mathfrak{C}_{2},\geq
1$, then (HYP-BE-3), requiring $\mu_{[1,n]}\leq 2$ only, trivially holds for
$n\leq 16$.

Now we consider the case of $n>16$. Suppose that the induction hypotheses hold
for all intervals with lengths smaller than $n$. We first derive an anti-
concentration inequality for any such intervals. Without loss of generality,
we only consider the intervals $[1,i]$ with $i<n$. We claim that

$\sqrt{i}\kappa_{[1,i]}(\delta)\leq\widetilde{\kappa}_{1,i}\overline{L}_{3,[1,i]}+\widetilde{\kappa}_{2,i}\overline{\nu}_{q,[1,i]}^{1/(q-2)}+\widetilde{\kappa}_{3,i}\overline{\nu}_{2,[1,i]}^{1/2}+\widetilde{\kappa}_{4}\delta,$
(HYP-AC-3)

where $\widetilde{\kappa}_{1,i}=\mathfrak{C}_{1,\kappa}\widetilde{\mu}_{1,i}$,
$\widetilde{\kappa}_{2,i}=\mathfrak{C}_{2,\kappa}\widetilde{\mu}_{2,i}$,
$\widetilde{\kappa}_{3,i}=\mathfrak{C}_{3,\kappa}\frac{\log(ep)\sqrt{\log(pi)}}{\sigma_{\min}}$
and
$\widetilde{\kappa}_{4}=\mathfrak{C}_{4,\kappa}\frac{\sqrt{\log(ep)}}{\sigma_{\min}}$
for some universal constants $\mathfrak{C}_{1,\kappa}$,
$\mathfrak{C}_{2,\kappa}$, $\mathfrak{C}_{3,\kappa}$, and
$\mathfrak{C}_{4,\kappa}$ whose values do not change over lines. If
$\mathfrak{C}_{1,\kappa},\mathfrak{C}_{2,\kappa},\mathfrak{C}_{3,\kappa},\mathfrak{C}_{4,\kappa},\geq
1$, then (HYP-BE-3), requiring $\kappa_{[1,n]}(\delta)\leq 1$ for all
$\delta>0$ only, trivially holds for $i\leq 16$. For $i>16$, assume that (HYP-
AC-3) holds for all smaller $i$’s. By Lemma A.6, for any
$\varepsilon\geq\sigma_{\min}$ and $\delta>0$,

$\displaystyle\kappa_{[1,i]}(\delta)$
$\displaystyle\leq\frac{C}{i_{o}-1}\sum_{j=1}^{i_{o}-1}\frac{\sqrt{\log(ep)}}{\varepsilon}(\nu_{1,j+1}+\nu_{1,j+i_{o}})\min\\{1,\kappa_{j+(1,i_{o})}(\varepsilon^{o})\\}$
$\displaystyle\quad+\frac{C}{i_{o}-1}\sum_{j=1}^{i_{o}-1}\mu_{j+(1,i_{o})}+C\frac{\delta+2\varepsilon^{o}}{\sigma_{\min}}\sqrt{\frac{\log(ep)}{i_{o}-2}}+C\frac{\overline{\nu}_{1,(1,i)}}{\sigma_{\min}}\frac{\log(ep)}{\sqrt{i_{o}-2}},$

where $i_{o}=\lfloor\frac{i}{2}\rfloor$,
$\varepsilon^{o}=20\varepsilon\sqrt{\log(p(i_{o}-2))}$ and $C>0$ is an
absolute constant. Due to (HYP-AC-3) and (HYP-BE-3) on intervals
$j+(1,i_{o})\subsetneq[1,i]$,

$\displaystyle\kappa_{[1,i]}(\delta)$
$\displaystyle\leq\frac{C}{(i_{o}-1)^{3/2}}\sum_{j=1}^{i_{o}-1}\frac{\sqrt{\log(ep)}}{\varepsilon}(\nu_{1,j+1}+\nu_{1,j+i_{o}})$
$\displaystyle\quad\times\left[\widetilde{\kappa}_{1,i_{o}-2}\overline{L}_{3,j+(1,i_{o})}+\widetilde{\kappa}_{2,i_{o}-2}\overline{\nu}_{q,j+(1,i_{o})}^{1/(q-2)}+\widetilde{\kappa}_{3,i_{o}-2}\overline{\nu}_{2,j+(1,i_{o})}^{1/2}+\widetilde{\kappa}_{4}{\delta}\right]$
$\displaystyle\quad+\frac{C}{(i_{o}-1)^{3/2}}\sum_{j=1}^{i_{o}-1}\widetilde{\mu}_{1,i_{o}-2}{\overline{L}_{3,j+(1,i_{o})}}+\frac{C}{(i_{o}-1)^{3/2}}\sum_{j=1}^{i_{o}-1}\widetilde{\mu}_{2,i_{o}-2}{\overline{L}_{4,j+(1,i_{o})}^{1/2}}$
$\displaystyle\quad+\frac{C}{(i_{o}-1)^{3/2}}\sum_{j=1}^{i_{o}-1}\widetilde{\mu}_{3,i_{o}-2}{\overline{\nu}_{q,j+(1,i_{o})}^{1/(q-2)}}+C\frac{\delta+2\varepsilon^{o}}{\sigma_{\min}}\sqrt{\frac{\log(ep)}{i_{o}-2}}+C\frac{\overline{\nu}_{1,(1,i)}}{\sigma_{\min}}\frac{\log(ep)}{\sqrt{i_{o}-2}}.$

To provide an upper bound in terms of $\overline{L}_{3}$ and
$\overline{\nu}_{q}$, we use the following lemma based on Jensen’s inequality.

######  Lemma A.7.

Suppose that $j,k\geq\frac{n}{2}$. For any $q_{1},q_{2},q_{3}>0$ and
$\alpha\leq 1$,

$\displaystyle\frac{1}{n}\sum_{j_{o}=1}^{n}L_{q_{1},j_{o}+j}\overline{L}_{q_{2},(j_{o},j_{o}+j)}^{\alpha}\leq
C\overline{L}_{q_{1}}\overline{L}_{q_{2}}^{\alpha}$

and

$\displaystyle\frac{1}{n}\sum_{j_{o}=1}^{n}\frac{L_{q_{1},j_{o}+j}}{j-1}\sum_{k_{o}=1}^{j-1}L_{q_{2},j_{o}+[k_{o}+k]_{j-1}}\overline{L}_{q_{3},j_{o}+(l,k+l)_{j-1}}^{\alpha}\leq
C\overline{L}_{q_{1}}\overline{L}_{q_{2}}\overline{L}_{q_{3}}^{\alpha},$

where $j_{o}+(k_{o},k_{o}+k)_{j-1}$ is the shifted interval of
$(k_{o},k_{o}+k)_{j-1}$ by $j_{o}$ in $\mathbb{Z}_{n}$, namely,
$\\{j_{o}+[k_{o}+1]_{j-1},\dots,j_{o}+[k_{o}+k-1]_{j-1}\\}$. The same
inequality holds when some $L$ is replaced with $\nu$.

Based on the lemma,

$\displaystyle\frac{1}{i_{o}-1}\sum_{j=1}^{i_{o}-1}\overline{L}_{3,j+(1,i_{o})}$
$\displaystyle\leq C\overline{L}_{3,(2,j-1)},$
$\displaystyle\frac{1}{i_{o}-1}\sum_{j=1}^{i_{o}-1}(\nu_{1,j+1}+\nu_{1,j+i_{o}})\overline{L}_{3,j+(1,i_{o})}$
$\displaystyle\leq C\overline{\nu}_{1,(1,i)}\overline{L}_{3,(2,i-1)},$
$\displaystyle\frac{1}{i_{o}-1}\sum_{j=1}^{i_{o}-1}(\nu_{1,j+1}+\nu_{1,j+i_{o}})\overline{\nu}_{2,j+(1,i_{o})}^{1/2}$
$\displaystyle\leq C\overline{\nu}_{1,(1,i)}\overline{\nu}_{2,(2,i-1)}^{1/2}.$

Similar inequalities hold with $\overline{\nu}_{q}^{1/(q-2)}$ in place of
$\overline{L}_{3}$. As a result, we obtain a similar recursive inequality on
$\widetilde{\kappa}$’s with Eq. 22 except that $L_{3,\max}$, $\nu_{q,\max}$,
$\nu_{1,\max}$ and $\nu_{2,\max}$ are replaced with $\overline{L}_{q,[1,i]}$,
$\overline{\nu}_{q,[1,i]}$, $\overline{\nu}_{1,[1,i]}$ and
$\overline{\nu}_{2,[1,i]}$, respectively. Solving the recursive inequality, we
prove (HYP-AC-3). The proof of (HYP-BE-3) also proceeds similarly using Lemma
A.7. In the end, we obtain a similar recursive inequality for
$\widetilde{\mu}$’s with Eq. 24 except that $L_{3,\max}$ and $\nu_{q,\max}$
are replaced with $\overline{L}_{3}$ and $\overline{\nu}_{q}$, respectively.
Solving the recursive inequality, we prove the desired (HYP-BE-3).

###  A.4 Proof of Theorem 2.2

We recall from Section A.3 that

$\displaystyle\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X_{[1,n]})]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y_{[1,n]})]\right\rvert$
$\displaystyle\leq\left\lvert\rho_{r,\phi}^{\delta}(X_{[1,n)})-\rho_{r,\phi}^{\delta}(Y_{[1,n)})\right\rvert+\left\lvert\rho_{r,\phi}^{\delta}(X_{[1,n]})-\rho_{r,\phi}^{\delta}(X_{[1,n)})-\rho_{r,\phi}^{\delta}(Y_{[1,n]})+\rho_{r,\phi}^{\delta}(Y_{[1,n)})\right\rvert$
$\displaystyle\leq\left\lvert\mathfrak{R}_{X}^{(3)}-\mathfrak{R}_{Y}^{(3)}\right\rvert+\sum_{j=1}^{n-1}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,1)}-\mathfrak{R}_{Y_{j}}^{(3,1)}\right]\right\rvert+\sum_{j=2}^{n-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2)}-\mathfrak{R}_{Y_{j}}^{(3,2)}\right]\right\rvert.$

In Section 4.4, we improved the rate by decomposing the third order remainder
$\mathfrak{R}^{(3,1)}_{W_{j}}$ when the fourth moments were finite. Namely,

$\displaystyle\sum_{j=1}^{n-1}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,1)}-\mathfrak{R}_{Y_{j}}^{(3,1)}\right]\right\rvert$
$\displaystyle\leq\frac{C}{\sqrt{n}}\overline{L}_{3}\frac{(\log(ep))^{2}}{\underline{\sigma}^{2}\sigma_{\min}}+\sum_{j=1}^{n-1}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,1)}-\mathfrak{R}_{Y_{j}}^{(4,1)}\right]\right\rvert+\sum_{j=3}^{n-1}\sum_{k=1}^{j-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{k}}^{(6)}-\mathfrak{R}_{X_{j},Y_{k}}^{(6)}\right]\right\rvert.$

In $1$-ring dependence cases, there exists additional third-order remainder
terms $\mathfrak{R}^{(3)}_{W}$ and $\mathfrak{R}^{(3,2)}_{X_{j},W_{k}}$, which
came up while breaking the $1$-ring dependence in Section A.3. Based on Tyalor
expansions up to order $4$ and the second moment matching between $X_{j}$ and
$Y_{j}$, we decompose the additional remainder terms. First,

$\displaystyle\mathfrak{R}_{X}^{(3)}$ (36)
$\displaystyle=\frac{1}{6}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}^{\otimes
3}\right\rangle+\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-3)}),X_{n-2}\otimes
X_{n-1}\otimes X_{n}\right\rangle$
$\displaystyle\quad+\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-2)}),X_{n-1}\otimes
X_{n}\otimes
X_{1}\right\rangle+\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(3,n-1)}),X_{n}\otimes
X_{1}\otimes X_{2}\right\rangle$
$\displaystyle\quad+\frac{1}{2}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}),X_{n-1}\otimes
X_{n}\otimes(X_{n-1}+X_{n})\right\rangle$
$\displaystyle\quad+\frac{1}{2}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}),X_{n}\otimes
X_{1}\otimes(X_{1}+X_{n})\right\rangle$
$\displaystyle\quad+\mathfrak{R}_{X}^{(4)},$

where $\mathfrak{R}_{X}^{(4)}$ is specified in Appendix C. This is the same
for $\rho_{r,\phi}^{\delta}(Y_{[1,n]})-\rho_{r,\phi}^{\delta}(Y_{[1,n)})$ but
with $Y$ in place of $X$. To bound the third-order moment terms, we re-apply
the Lindeberg swapping and the Taylor expansion up to the sixth order as we
did to
$\left\langle\mathbb{E}[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-1)}+Y_{(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$ in Section 4.4. As a result,

$\displaystyle\left\lvert\mathfrak{R}_{X}^{(3)}-\mathfrak{R}_{Y}^{(3)}\right\rvert\leq\frac{C}{\sqrt{n}}\overline{L}_{3}\frac{(\log(ep))^{2}}{\underline{\sigma}^{2}\sigma_{\min}}+\left\lvert\mathfrak{R}_{X}^{(4)}-\mathfrak{R}_{Y}^{(4)}\right\rvert+\sum_{k=2}^{n-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6)}-\mathfrak{R}_{X,Y_{k}}^{(6)}\right]\right\rvert,$

where $\mathfrak{R}_{X,W_{k}}^{(6)}$ is the sixth-order remainder term
specified in Appendix C.

For
$\sum_{j=2}^{n-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2)}-\mathfrak{R}_{Y_{j}}^{(3,2)}\right]\right\rvert$,
by the Taylor expansion up to the fourth order,

$\sum_{j=2}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2)}-\mathfrak{R}_{Y_{j}}^{(3,2)}\right]=\sum_{j=2}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,2)}-\mathfrak{R}_{Y_{j}}^{(4,2)}\right],$

where $\mathfrak{R}_{X_{j}}^{(4,2)}$ is the fourth-order remainder, specified
in Section C.3. Putting all the above results together, we get

$\displaystyle\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X_{[1,n]})]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y_{[1,n]})]\right\rvert$
$\displaystyle\leq\frac{C}{\sqrt{n}}\overline{L}_{3}\frac{(\log(ep))^{2}}{\underline{\sigma}^{2}\sigma_{\min}}+\left\lvert\mathfrak{R}_{X}^{(4)}-\mathfrak{R}_{Y}^{(4)}\right\rvert+\sum_{j=1}^{n-1}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,1)}-\mathfrak{R}_{Y_{j}}^{(4,1)}\right]\right\rvert+\sum_{j=2}^{n-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,2)}-\mathfrak{R}_{Y_{j}}^{(4,2)}\right]\right\rvert$
$\displaystyle\quad+\sum_{k=2}^{n-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6)}-\mathfrak{R}_{X,Y_{k}}^{(6)}\right]\right\rvert+\sum_{j=3}^{n-1}\sum_{k=1}^{j-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{k}}^{(6)}-\mathfrak{R}_{X_{j},Y_{k}}^{(6)}\right]\right\rvert.$

Remainder lemma. Similar to Section 4.1, the remainder terms are upper bounded
by conditional anti-concentration probability bounds. For $q>0$, let

$\widetilde{L}_{q,j}\equiv\sum_{j^{\prime}=j-4}^{j+4}L_{q,j^{\prime}}\enskip\text{and}\enskip\widetilde{L}_{q,[k]_{j-2}}\equiv\sum_{k^{\prime}=k-4}^{k+4}L_{q,[k^{\prime}]_{j-2}},$

where $[k^{\prime}]_{j-2}$ is $k^{\prime}$ modulo $j-2$, and
$\widetilde{\nu}_{q,j}$ and $\widetilde{\nu}_{q,[k]_{j-2}}$ are similarly
defined.

######  Lemma A.8.

Suppose that Assumption (MIN-EV) holds. For $W$ representing either $X$ or $Y$
and $j,k\in\mathbb{Z}_{n}$ such that $k\leq j-2$,

$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W}^{(4)}\right]\right\rvert$
$\displaystyle\leq
C\phi\left[\widetilde{L}_{4,n}\frac{(\log(ep))^{3/2}}{\delta^{3}}+\widetilde{\nu}_{q,n}\frac{{(\log(ep))}^{(q-1)/2}}{\delta^{q-1}}\right]$
$\displaystyle\quad\times\min\\{\kappa_{(3,n-2)}(\delta_{0}^{o})+\kappa^{o}_{n},1\\},$
$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W_{j}}^{(4,1)}\right]\right\rvert$
$\displaystyle\leq
C\phi\left[\widetilde{L}_{4,j}\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}+\widetilde{\nu}_{q,j}\frac{{(\log(ep))}^{(q-1)/2}}{\delta_{n-j}^{q-1}}\right]$
$\displaystyle\quad\times\min\\{\kappa_{[1,j-4)}(\delta_{n-j}^{o})+\kappa^{o}_{j},1\\},$
$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W_{j}}^{(4,2)}\right]\right\rvert$
$\displaystyle\leq C\phi\left[\begin{aligned}
\left(\widetilde{L}_{4,j}+\widetilde{L}_{4,n}\right)\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}+\left(\widetilde{\nu}_{q,j}+\widetilde{\nu}_{q,n}\right)\frac{{(\log(ep))}^{(q-1)/2}}{\delta_{n-j}^{q-1}}\end{aligned}\right]$
$\displaystyle\quad\times\min\\{\kappa_{(1,j-2)}(\delta_{n-j}^{o})+\kappa^{o}_{j},1\\},$
$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{X,W_{k}}^{(6)}\right]\right\rvert$
$\displaystyle\leq C\phi\widetilde{L}_{3,n}\left[\begin{aligned}
\widetilde{L}_{3,[k]_{n-2}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\widetilde{\nu}_{q,[k]_{n-2}}\frac{{(\log(ep))}^{(q+2)/2}}{\delta_{n-k}^{q+2}}\end{aligned}\right]$
$\displaystyle\quad\times\min\\{\kappa_{[1,k-3)}(\delta_{n-k}^{o})+\kappa^{o}_{k},1\\}.$
$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j},W_{k}}^{(6)}\right]\right\rvert$
$\displaystyle\leq C\phi\widetilde{L}_{3,j}\left[\begin{aligned}
\widetilde{L}_{3,[k]_{j-2}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\widetilde{\nu}_{q,[k]_{j-2}}\frac{{(\log(ep))}^{(q+2)/2}}{\delta_{n-k}^{q+2}}\end{aligned}\right]$
$\displaystyle\quad\times\min\\{\kappa_{[1,k-3)}(\delta_{n-k}^{o})+\kappa^{o}_{k},1\\}.$

where
$\delta_{n-j}^{2}\equiv\delta^{2}+\underline{\sigma}^{2}\max\\{n-j,0\\}$,
$\delta^{o}_{n-j}\equiv 12\delta_{n-j}\sqrt{\log(pn)}$ and
$\kappa^{o}_{j}\equiv\frac{\delta_{n-j}\log(ep)}{\sigma_{\min}\sqrt{\max\\{j,1\\}}}$,
as long as $\delta\geq\sigma_{\min}$ and $\phi\delta\geq\frac{1}{\log(ep)}$.

Permutation argument. Back to Eq. 31, for $3\leq j\leq n-1$,

$\displaystyle\left\langle\mathbb{E}[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,j-1)}+Y_{(j+1,n]})-\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{[1,j-1)}+Y_{(j+1,n]})],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$
$\displaystyle\leq\frac{1}{j-2}\sum_{k_{o}=1}^{j-2}\sum_{k=1}^{j-2}\Big{\langle}\mathbb{E}[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[k_{o},k_{o}+k)_{j-2}}+X_{[k_{o}+k]_{j-2}}+Y_{(k_{o}+k,k_{o}+j-1)_{j-2}\cup(j,n]})$
$\displaystyle\hskip
36.135pt-\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[k_{o},k_{o}+k)_{j-2}}+Y_{[k_{o}+k]_{j-2}}+Y_{(k_{o}+k,k_{o}+j-1)_{j-2}\cup(j,n]})],\mathbb{E}[X_{j}^{\otimes
3}]\Big{\rangle}$ $\displaystyle\leq
C\phi\frac{\widetilde{L}_{3,j}}{j-2}\sum_{k_{o}=1}^{j-2}\sum_{k=1}^{j-2}\left[\widetilde{L}_{3,[k_{o}+k^{\prime}]_{j-2}}\frac{(\log(ep))^{3/2}}{\delta_{n-k}^{3}}+\widetilde{\nu}_{q,[k_{o}+k^{\prime}]_{j-2}}\frac{{(\log(ep))}^{(q-1)/2}}{(\delta_{n-k}/2)^{q-1}}\right]$
$\displaystyle\quad\hskip
79.49744pt\times\min\\{\kappa_{(k_{o},k_{o}+k-3)_{j-2}}(\delta_{n-k}^{o})+\kappa^{o}_{k},1\\}.$

The permutation arguement also applies to the first Lindeberg swapping in Eq.
30. Together with the results in Lemma A.8,

$\displaystyle\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X_{[1,n]})]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y_{[1,n]})]\right\rvert$
$\displaystyle\leq\frac{1}{n}\sum_{j_{o}=1}^{n}\sum_{j=1}^{n-1}\mathbb{E}\left[\rho_{r,\phi}^{\delta}(X_{(j_{o},j_{o}+j)}+X_{j_{o}+j}+Y_{(j_{o}+j,j_{o}+n]})-\rho_{r,\phi}^{\delta}(X_{(j_{o},j_{o}+j)}+Y_{j_{o}+j}+Y_{(j_{o}+j,j_{o}+n]})\right]$
$\displaystyle\leq\frac{C}{\sqrt{n}}\overline{L}_{3}\frac{(\log(ep))^{2}}{\underline{\sigma}^{2}\sigma_{\min}}$
$\displaystyle\quad+\frac{C\phi}{n}\sum_{j_{o}=1}^{n}\left[\widetilde{L}_{4,j_{o}}\frac{(\log(ep))^{3/2}}{\delta^{3}}+\widetilde{\nu}_{q,j_{o}}\frac{{(\log(ep))}^{(q-1)/2}}{(\delta/2)^{q-1}}\right]\min\\{1,\kappa_{j_{o}+(3,n-2)}(\delta_{0}^{o})+\kappa^{o}_{n}\\}$
$\displaystyle\quad+\frac{C\phi}{n}\sum_{j_{o}=1}^{n}\sum_{j=2}^{n-2}\left[\widetilde{L}_{4,j_{o}}\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}+\widetilde{\nu}_{q,j_{o}}\frac{{(\log(ep))}^{(q-1)/2}}{(\delta_{n-j}/2)^{q-1}}\right]\min\\{1,\kappa_{j_{o}+(1,j-2)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\quad+\frac{C\phi}{n}\sum_{j_{o}=1}^{n}\sum_{j=1}^{n-1}\left[\widetilde{L}_{4,j_{o}+j}\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}+\widetilde{\nu}_{q,j_{o}+j}\frac{{(\log(ep))}^{(q-1)/2}}{(\delta_{n-j}/2)^{q-1}}\right]\min\\{1,\kappa_{j_{o}+(1,j-4)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\quad+\frac{C\phi}{n}\sum_{j_{o}=1}^{n}\sum_{j=3}^{n}\frac{\widetilde{L}_{3,j_{o}+j}}{j-2}\sum_{k_{o}=1}^{j-2}\sum_{k=1}^{j-2}\left[\widetilde{L}_{3,j_{o}+[k_{o}+k]_{j-2}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\widetilde{\nu}_{q,j_{o}+[k_{o}+k]_{j-2}}\frac{{(\log(ep))}^{(q+2)/2}}{(\delta_{n-k}/2)^{q+2}}\right]$
$\displaystyle\quad\hskip
144.54pt\times\min\\{1,\kappa_{j_{o}+(k_{o},k_{o}+k-3)_{j-2}}(\delta_{n-k}^{o})+\kappa^{o}_{k}\\},$

where $j_{o}+(k_{o},k_{o}+k)_{j-1}$ is the shifted interval of
$(k_{o},k_{o}+k)_{j-1}$ by $j_{o}$ in $\mathbb{Z}_{n}$, namely,
$\\{j_{o}+[k_{o}+1]_{j-1},\dots,j_{o}+[k_{o}+k-1]_{j-1}\\}$.

Partitioning the sum. Again, we partition the summations at
$J_{n}=n(1-\frac{\sigma_{\min}^{2}}{\underline{\sigma}^{2}\log^{2}(4ep)})$.
The calculations of the first three summations are similar to those in Section
A.3. Here we only take a look at the last summation where the summation
iterates. For $k<J_{n}$,

$\displaystyle\frac{1}{n}\sum_{j_{o}=1}^{n}\sum_{j=3}^{n}\frac{\widetilde{L}_{3,j_{o}+j}}{j-2}\sum_{k_{o}=1}^{j-2}\sum_{k=1}^{(j-2)\wedge\lfloor
J_{n}\rfloor}\left[\widetilde{L}_{3,j_{o}+[k_{o}+k]_{j-2}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\widetilde{\nu}_{q,j_{o}+[k_{o}+k]_{j-2}}\frac{{(\log(ep))}^{(q+2)/2}}{\delta_{n-k}^{q+2}}\right]$
$\displaystyle\leq\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j=3}^{n}\widetilde{L}_{3,j_{o}+j}\sum_{k=1}^{(j-2)\wedge\lfloor
J_{n}\rfloor}\left[\overline{L}_{3}\frac{n(\log(ep))^{5/2}}{(j-2)\delta_{n-k}^{5}}+\overline{\nu}_{q}\frac{n{(\log(ep))}^{(q+2)/2}}{(j-2)\delta_{n-k}^{q+2}}\right]$
$\displaystyle\leq\frac{C}{\sqrt{n}}\sum_{j=3}^{n}\overline{L}_{3}\left[\overline{L}_{3}\frac{n(\log(ep))^{7/2}}{(j-2)\underline{\sigma}^{2}\sigma_{\min}\delta_{n-\lfloor
J_{n}\rfloor}^{2}}+\overline{\nu}_{q}\frac{n{(\log(ep))}^{(q+4)/2}}{(j-2)\underline{\sigma}^{2}\sigma_{\min}\delta_{n-\lfloor
J_{n}\rfloor}^{q-1}}\right]$
$\displaystyle\leq\frac{C}{\sqrt{n}}\overline{L}_{3}\left[\overline{L}_{3}\frac{(\log(ep))^{7/2}}{\underline{\sigma}^{4}\sigma_{\min}}+\overline{\nu}_{q}\frac{{(\log(ep))}^{(q+4)/2}}{\underline{\sigma}^{4}\sigma_{\min}\delta^{q-3}}\right]\log(en),$

where the third inequality comes from Eq. 16 and the last inequality comes
from the fact that $\sum_{j=3}^{n}\frac{n}{(j-2)\delta^{2}_{n-\lfloor
J_{n}\rfloor}}\leq\sum_{j=3}^{n}\frac{n}{(j-2)(n-\lfloor
J_{n}\rfloor)\underline{\sigma}^{2}}\leq\frac{C}{\underline{\sigma}^{2}}\log(en)$.
For $k>J_{n}$,

$\displaystyle\frac{1}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\frac{\widetilde{L}_{3,j_{o}+j}}{j-2}\sum_{k_{o}=1}^{j-2}\sum_{k=\lceil
J_{n}\rceil}^{j-2}\left[\widetilde{L}_{3,j_{o}+[k_{o}+k]_{j-2}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\widetilde{\nu}_{q,j_{o}+[k_{o}+k]_{j-2}}\frac{{(\log(ep))}^{(q+2)/2}}{\delta_{n-k}^{q+2}}\right]$
$\displaystyle\hskip
151.76744pt\times\min\\{1,\kappa_{j_{o}+(k_{o},k_{o}+k)_{j-2}}(\delta_{n-k}^{o})+\kappa^{o}_{k}\\}$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\frac{\widetilde{L}_{3,j_{o}+j}}{j-2}\sum_{k_{o}=1}^{j-2}\sum_{k=\lceil
J_{n}\rceil}^{j-2}\left[\widetilde{L}_{3,j_{o}+[k_{o}+k]_{j-2}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\widetilde{\nu}_{q,j_{o}+[k_{o}+k]_{j-2}}\frac{{(\log(ep))}^{(q+2)/2}}{\delta_{n-k}^{q+2}}\right]$
$\displaystyle\quad\hskip
151.76744pt\times\kappa_{j_{o}+(k_{o},k_{o}+k)_{j-2}}(\delta_{n-k}^{o})$
$\displaystyle\quad+\frac{C}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\widetilde{L}_{3,j_{o}+j}\sum_{k=\lceil
J_{n}\rceil}^{j-2}\left[\overline{L}_{3,j_{o}+(0,j)}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\overline{\nu}_{q,j_{o}+(0,j)}\frac{{(\log(ep))}^{(q+2)/2}}{\delta_{n-k}^{q+2}}\right]\kappa^{o}_{k}.$

Based on Lemma A.7,

$\displaystyle\frac{1}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\widetilde{L}_{3,j_{o}+j}\sum_{k=\lceil
J_{n}\rceil}^{j-1}\left[\overline{L}_{3,(j_{o},j_{o}+j)}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\overline{\nu}_{q,(j_{o},j_{o}+j)}\frac{{(\log(ep))}^{(q+2)/2}}{\delta_{n-k}^{q+2}}\right]\kappa^{o}_{k}$
$\displaystyle\leq\frac{C}{n^{3/2}}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\widetilde{L}_{3,j_{o}+j}\left[\overline{L}_{3,(j_{o},j_{o}+j)}\frac{(\log(ep))^{7/2}}{\underline{\sigma}^{2}\sigma_{\min}\delta_{n-j}^{2}}+\overline{\nu}_{q,(j_{o},j_{o}+j)}\frac{{(\log(ep))}^{(q+4)/2}}{\underline{\sigma}^{2}\sigma_{\min}\delta_{n-j}^{q-1}}\right]$
$\displaystyle\leq\frac{C}{\sqrt{n}}\sum_{j=\lceil
J_{n}\rceil}^{n}\overline{L}_{3}\left[\overline{L}_{3}\frac{(\log(ep))^{7/2}}{\underline{\sigma}^{2}\sigma_{\min}\delta_{n-j}^{2}}+\overline{\nu}_{q}\frac{{(\log(ep))}^{(q+4)/2}}{\underline{\sigma}^{2}\sigma_{\min}\delta_{n-j}^{q-1}}\right]$
$\displaystyle\leq\frac{C}{\sqrt{n}}\overline{L}_{3}\left[\overline{L}_{3}\frac{(\log(ep))^{7/2}}{\underline{\sigma}^{4}\sigma_{\min}}+\overline{\nu}_{q}\frac{{(\log(ep))}^{(q+4)/2}}{\underline{\sigma}^{4}\sigma_{\min}\delta^{q-3}}\right]\log\left(1+\frac{\sqrt{n}\underline{\sigma}}{\delta}\right),$

where the first and last inequliaties follow Eqs. 16 and 23, respectively. In
sum, we obtain the following induction lemma.

######  Lemma A.9.

If Assumptions (MIN-VAR), (MIN-EV) and (VAR-EV) hold, then for any
$\delta\geq\sigma_{\min}$,

$\displaystyle\mu_{[1,n]}$
$\displaystyle\leq\frac{C}{\sqrt{n}}\frac{\delta\log(ep)}{\sigma_{\min}}+\frac{C}{\sqrt{n}}\frac{\sqrt{\log(ep)}}{\phi\sigma_{\min}}+\frac{C}{\sqrt{n}}\overline{L}_{3}\frac{(\log(ep))^{2}}{\underline{\sigma}^{2}\sigma_{\min}}$
$\displaystyle\quad+\frac{C\phi}{\sqrt{n}}\left[\overline{L}_{4}\frac{(\log(ep))^{5/2}}{\underline{\sigma}^{2}\sigma_{\min}}+\overline{\nu}_{q}\frac{(\log(ep))^{(q+1)/2}}{\underline{\sigma}^{2}\sigma_{\min}(\delta/2)^{q-4}}\right]\log(en)$
$\displaystyle\quad+\frac{C\phi}{\sqrt{n}}\overline{L}_{3}\left[\overline{L}_{3}\frac{(\log(ep))^{7/2}}{\underline{\sigma}^{4}\sigma_{\min}}+\overline{\nu}_{q}\frac{{(\log(ep))}^{(q+4)/2}}{\underline{\sigma}^{4}\sigma_{\min}(\delta/2)^{q-3}}\right]\log(en)$
$\displaystyle\quad+\frac{C\phi}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\left[\widetilde{L}_{4,j_{o}}\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}+\widetilde{\nu}_{q,j_{o}}\frac{{(\log(ep))}^{(q-1)/2}}{(\delta_{n-j}/2)^{q-1}}\right]\kappa_{j_{o}+(3,j-2)}(\delta_{n-j}^{o})$
$\displaystyle\quad+\frac{C\phi}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n-1}\left[\widetilde{L}_{4,j_{o}+j}\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}+\widetilde{\nu}_{q,j_{o}+j}\frac{{(\log(ep))}^{(q-1)/2}}{(\delta_{n-j}/2)^{q-1}}\right]\kappa_{j_{o}+(1,j-4)}(\delta_{n-j}^{o})$
$\displaystyle\quad+\frac{C\phi}{n}\sum_{j_{o}=1}^{n}\sum_{j=\lceil
J_{n}\rceil}^{n}\frac{\widetilde{L}_{3,j_{o}+j}}{j-2}$
$\displaystyle\quad\quad\quad\times\sum_{k_{o}=1}^{j-2}\sum_{k=\lceil
J_{n}\rceil}^{j-2}\left[\widetilde{L}_{3,j_{o}+[k_{o}k]_{j-2}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\widetilde{\nu}_{q,j_{o}+[k_{o}+k]_{j-2}}\frac{(\log(ep))^{(q+2)/2}}{(\delta_{n-k}/2)^{q+2}}\right]$
$\displaystyle\quad\hskip
86.72377pt\times\kappa_{j_{o}+(k_{o},k_{o}+k-3)_{j-2}}(\delta_{n-k}^{o}),$

for some absolute constant $C>0$.

Dual Induction. Based on Lemmas A.9 and A.6, we proceed the dual induction in
the same steps as Section A.1, but using Lemma A.7 as in Section A.3. As a
result, we prove for any $n$,

$\sqrt{n}\mu_{[1,n]}\leq\widetilde{\mu}_{1,n}\overline{L}_{3}+\widetilde{\mu}_{2,n}\overline{L}_{4}^{1/2}+\widetilde{\mu}_{3,n}\overline{\nu}_{q}^{1/(q-2)},$
(HYP-BE-4)

where

$\displaystyle\widetilde{\mu}_{1,n}$
$\displaystyle=\mathfrak{C}_{1}\frac{(\log(ep))^{3/2}\sqrt{\log(pn)}}{\underline{\sigma}^{2}\sigma_{\min}}\log\left(en\right),$
$\displaystyle\widetilde{\mu}_{2,n}$
$\displaystyle=\mathfrak{C}_{2}\frac{\log(ep)\sqrt{\log(pn)}}{\underline{\sigma}\sigma_{\min}}\log\left(en\right)$
$\displaystyle\widetilde{\mu}_{3,n}$
$\displaystyle=\mathfrak{C}_{3}\frac{\log(ep)\sqrt{\log(pn)}}{\underline{\sigma}^{2/(q-2)}\sigma_{\min}}\log\left(en\right)$

for universal constants $\mathfrak{C}_{1}$, $\mathfrak{C}_{2}$ and
$\mathfrak{C}_{3}$ whose values do not change over lines, and for any $i<n$,

$\sqrt{i}\kappa_{[1,i]}(\delta)\leq\widetilde{\kappa}_{1,i}\overline{L}_{3,[1,i]}+\widetilde{\kappa}_{2,i}\overline{L}_{4,[1,i]}^{1/2}+\widetilde{\kappa}_{3,i}\overline{\nu}_{q,[1,i]}^{1/(q-2)}+\widetilde{\kappa}_{4,i}\overline{\nu}_{2,[1,i]}^{1/2}+\widetilde{\kappa}_{5}\delta,$
(HYP-AC-4)

where $\widetilde{\kappa}_{1,i}=\mathfrak{C}_{1,\kappa}\widetilde{\mu}_{1,i}$,
$\widetilde{\kappa}_{2,i}=\mathfrak{C}_{2,\kappa}\widetilde{\mu}_{2,i}$,
$\widetilde{\kappa}_{3,i}=\mathfrak{C}_{3,\kappa}\widetilde{\mu}_{3,i}$,
$\widetilde{\kappa}_{4,i}=\mathfrak{C}_{4,\kappa}\frac{\log(ep)\sqrt{\log(pi)}}{\sigma_{\min}}$
and
$\widetilde{\kappa}_{5}=\mathfrak{C}_{5,\kappa}\frac{\sqrt{\log(ep)}}{\sigma_{\min}}$
for some universal constants $\mathfrak{C}_{1,\kappa}$,
$\mathfrak{C}_{2,\kappa}$, $\mathfrak{C}_{3,\kappa}$,
$\mathfrak{C}_{4,\kappa}$ and $\mathfrak{C}_{5,\kappa}$ whose values do not
change over lines. This proves the desired Berry–Esseen bound.

##  Appendix B Proofs of Lemmas

###  B.1 Proof of Lemma 4.2

The smoothing lemma is the result of the serial application of the following
two lemmas. Lemma B.2 is a corollary of Theorem 2.1 in Chernozhukov,
Chetverikov and Koike (2023). We provide a standalone proof in Section B.2.

######  Lemma B.1 (Lemma 1, Kuchibhotla and Rinaldo, 2020).

Suppose that $X$ is a $p$-dimensional random vector, and $Y\sim N(0,\Sigma)$
is a $p$-dimensional Gaussian random vector. Then, for any $\delta>0$ and a
standard Gaussian random vector $Z$,

$\mu(X,Y)\leq C\mu(X+\delta Z,Y+\delta
Z)+C\frac{\delta\log(ep)}{\sqrt{\min_{i=1,\dots,p}\Sigma_{ii}}}.$

######  Lemma B.2.

Suppose that $X$ is a $p$-dimensional random vector, and $Y\sim N(0,\Sigma)$
is a $p$-dimensional Gaussian random vector. Then, for any $\phi>0$,

$\displaystyle\mu\left(X,Y\right)\leq\sup_{r\in\mathbb{R}^{d}}\left\lvert\mathbb{E}[f_{r,\phi}(X)]-\mathbb{E}[f_{r,\phi}(Y)]\right\rvert+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}.$

For any $\delta>0$,

$\mu\left(X,Y\right)\leq\mu\left(X+\delta Z,Y+\delta
Z\right)+C\frac{\delta\log(ep)}{\sqrt{\min_{i=1,\dots,p}\Sigma_{ii}}}.$ (37)

Then, for any $\delta>0$ and $\phi>0$,

$\displaystyle\mu\left(X+\delta Z,Y+\delta Z\right)$ $\displaystyle\leq
C\sup_{r\in\mathbb{R}^{d}}\left\lvert\mathbb{E}[f_{r,\phi}(X+\delta
Z)]-\mathbb{E}[f_{r,\phi}(Y+\delta
Z)]\right\rvert+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}$
$\displaystyle\leq
C\sup_{r\in\mathbb{R}^{d}}\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X)]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y)]\right\rvert+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}.$

In sum,

$\mu(X,Y)\leq
C\sup_{r\in\mathbb{R}^{d}}\left\lvert\mathbb{E}[\rho_{r,\phi}^{\delta}(X)]-\mathbb{E}[\rho_{r,\phi}^{\delta}(Y)]\right\rvert+C\frac{\delta\log(ep)+\sqrt{\log(ep)}/\phi}{\sqrt{\min_{i=1,\dots,p}\Sigma_{ii}}}.$

###  B.2 Proof of Lemma B.2

By Lemma B.1,

$\displaystyle\mathbb{P}[X\in A_{r}]$
$\displaystyle\leq\mathbb{E}[f_{r,\phi}(X)]=\mathbb{E}[f_{r,\phi}(Y)]+\mathbb{E}[f_{r,\phi}(X)]-\mathbb{E}[f_{r,\phi}(Y)]$
$\displaystyle\leq\mathbb{P}[Y\in
A_{r+\frac{1}{\phi}\mathbf{1}}]+\mathbb{E}[f_{r,\phi}(X)]-\mathbb{E}[f_{r,\phi}(Y)]$
$\displaystyle\leq\mathbb{P}[Y\in
A_{r}]+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}},$

and similarly,

$\displaystyle\mathbb{P}[Y\in A_{r}]$ $\displaystyle\leq\mathbb{P}[Y\in
A_{r-\frac{1}{\phi}\mathbf{1}}]+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}$
$\displaystyle\leq\mathbb{E}[f_{r-\frac{1}{\phi}\mathbf{1},\phi}(Y)]+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}$
$\displaystyle\leq\mathbb{E}[f_{r-\frac{1}{\phi}\mathbf{1},\phi}(X)]+\mathbb{E}[f_{r-\frac{1}{\phi}\mathbf{1},\phi}(Y)]-\mathbb{E}[f_{r-\frac{1}{\phi}\mathbf{1},\phi}(X)]+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}$
$\displaystyle\leq\mathbb{P}[X\in
A_{r}]+\mathbb{E}[f_{r-\frac{1}{\phi}\mathbf{1},\phi}(Y)]-\mathbb{E}[f_{r-\frac{1}{\phi}\mathbf{1},\phi}(X)]+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}.$

Hence,

$\sup_{r\in\mathbb{R}^{p}}\left\lvert\mathbb{P}[X\in A_{r}]-\mathbb{P}[Y\in
A_{r}]\right\rvert\leq\sup_{r\in\mathbb{R}^{p}}\left\lvert\mathbb{E}[f_{r,\phi}(X)]-\mathbb{E}[f_{r,\phi}(Y)]\right\rvert+\frac{C}{\phi}\sqrt{\frac{\log(ep)}{\min_{i=1,\dots,p}\Sigma_{ii}}}.$

###  B.3 Proof of Remainder Lemmas

We observe that all remainder terms are in forms of

$\displaystyle\frac{1}{(\beta-1)!}\int_{0}^{1}(1-t)^{\beta-1}\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta}(X_{J_{1}}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{J_{2}}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\right]dt,$

where $1\leq\beta\leq\alpha$ and $J_{1},J_{2}$ and $J$ are subsets of
$\mathbb{Z}_{n}$ satisfying $\mathscr{X}_{J_{1}}$, $\mathscr{Y}_{J_{2}}$ and
$\mathscr{W}_{\\{j_{1},\dots,j_{\alpha-\beta}\\}\cup J}$ are mutually
independent. We also note that
$\mathscr{X}_{J_{1}}\cup\mathscr{Y}_{J_{2}}\cup\\{W^{\not\\!\perp\\!\\!\\!\\!\perp}\\}$
is independent from $\mathscr{W}_{\\{j_{1},\dots,j_{\alpha-\beta}\\}}$. Here
we prove a succinct form of the remainder lemmas:

######  Lemma B.3.

Suppose that
$\operatorname{Var}[Y_{J_{2}}|\mathscr{Y}_{J_{2}^{\mathsf{C}}}]\succeq\underline{\sigma}_{J_{2}}^{2}I_{p}$
for some $\underline{\sigma}^{2}_{J_{2}}>0$ and $p$-dimensional identity
matrix $I_{p}$. Then, for any $\gamma_{1},\gamma_{2}\in[0,1]$ and $\eta>0$,
remainder terms in the above form satisfy

$\displaystyle\left\lvert\int_{0}^{1}(1-t)^{\beta-1}\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta}(X_{J_{1}}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{J_{2}}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\right]dt\right\rvert$ $\displaystyle\leq
C\left[\frac{(\log(ep))^{\alpha/2}}{{\delta^{\prime}}^{\alpha}}\min\left\\{1,\kappa_{J_{1}}\left(\delta^{o}\right)+\kappa^{o}\right\\}\right]$
$\displaystyle\quad\times\left[\begin{aligned}
&\frac{\phi^{\gamma_{1}}{\delta^{\prime}}^{\gamma_{1}}}{(\log(ep))^{\gamma_{1}/2}}\left\lVert\mathbb{E}\left[\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert\right]\right\rVert_{\infty}\\\
&+\frac{\phi^{\gamma_{2}}{\delta^{\prime}}^{\gamma_{2}-\eta}}{(\log(ep))^{(\gamma_{2}-\eta)/2}}\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert W_{J}\rVert_{\infty}^{\beta}(\lVert
W_{J}\rVert_{\infty}^{\eta}+\lVert
W^{\not\\!\perp\\!\\!\\!\\!\perp}\rVert_{\infty}^{\eta})\right]\end{aligned}\right],$

where $\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert$ is the element-wise absolute operation,
$\delta^{\prime}\equiv\sqrt{\delta^{2}+\underline{\sigma}_{J_{2}}^{2}}$,
$\delta^{o}\equiv 12\delta^{\prime}\sqrt{\log(p\lvert J_{1}\rvert)}$, and
$\kappa^{o}\equiv\frac{\delta^{\prime}\log(ep)}{\sigma_{\min}\sqrt{\lvert
J_{1}\rvert}}$ as long as $\delta\geq\sigma_{\min}$ and
$\phi\delta\geq\frac{1}{\log(ep)}$.

We prove the above generalized remainder lemma here. Because
$\operatorname{Var}[Y_{J_{2}}|\mathscr{Y}_{J_{2}^{\mathsf{C}}}]\succeq\underline{\sigma}_{J_{2}}^{2}I_{p}$,

$Y_{J_{2}}|\mathscr{Y}_{J_{2}^{\mathsf{C}}}\overset{d}{=}Y^{o}_{J_{2}}+\underline{\sigma}_{J_{2}}\cdot
Z,~{}~{}\text{almost surely},$

where $Y^{o}_{J_{2}}$ is the Gaussian random variable with mean
$\mathbb{E}[Y_{J_{2}}|\mathscr{Y}_{J_{2}^{\mathsf{C}}}]$ and variance
$\operatorname{Var}[Y_{J_{2}}|\mathscr{Y}_{J_{2}^{\mathsf{C}}}]-\underline{\sigma}_{J_{2}}^{2}I_{p}$.
For brevity, let $W^{o}\equiv X_{J_{1}}+Y^{o}_{J_{2}}$. Because
$\delta^{\prime}=\sqrt{\delta^{2}+\underline{\sigma}_{J_{2}}^{2}}$, the
remainder term is decomposed into

$\displaystyle\int_{0}^{1}(1-t)^{\beta-1}\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta}(X_{J_{1}}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{J_{2}}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\right]dt$
$\displaystyle=\int_{0}^{1}(1-t)^{\beta-1}\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\right]dt$
$\displaystyle=\int_{0}^{1}(1-t)^{\beta-1}\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\mathbb{I}_{W_{J},1}\right]dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)^{\beta-1}\mathbb{E}\left[\left\langle\nabla^{\alpha-1}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\mathbb{I}_{W_{J},2}\right]dt$
$\displaystyle=:\int_{0}^{1}(1-t)^{\beta-1}T_{W_{J},1}(t)dt+\int_{0}^{1}(1-t)^{\beta-1}T_{W_{J},2}(t)dt,$

where

$\displaystyle\mathbb{I}_{W_{J},1}$ $\displaystyle=\mathbb{I}_{\\{\lVert
W_{J}\rVert_{\infty}\text{ and }\lVert
W^{{\not\\!\perp\\!\\!\\!\\!\perp}}\rVert_{\infty}<\frac{\delta^{\prime}}{\sqrt{\log(ep)}}\\}},$
$\displaystyle\mathbb{I}_{W_{J},2}$ $\displaystyle=\mathbb{I}_{\\{\lVert
W_{J}\rVert_{\infty}\text{ or }\lVert
W^{{\not\\!\perp\\!\\!\\!\\!\perp}}\rVert_{\infty}\geq\frac{\delta^{\prime}}{\sqrt{\log(ep)}}\\}}.$

We upper-bound the terms $T_{W_{J},1}(t)$ and $T_{W_{J},2}(t)$ separately.
First,

$\displaystyle\left\lvert T_{W_{J},1}(t)\right\rvert$ (38)
$\displaystyle=\left\lvert\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\mathbb{I}_{W_{J},1}\right]\right\rvert$
$\displaystyle=\left\lvert\mathbb{E}\left[\sum_{i_{1},\dots,i_{\alpha}}\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+tW_{J})\cdot\prod_{k\leq\alpha-\beta}W_{j_{k}}^{(i_{k})}\prod_{k>\alpha-\beta}W_{J}^{(i_{k})}\cdot\mathbb{I}_{W_{J},1}\right]\right\rvert$
$\displaystyle\leq\mathbb{E}\left[\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+z)\right\rvert\cdot\left\lvert\prod_{k\leq\alpha-\beta}W_{j_{k}}^{(i_{k})}\prod_{k>\alpha-\beta}W_{J}^{(i_{k})}\right\rvert\cdot\mathbb{I}_{W_{J},1}\right]$
$\displaystyle\leq\sum_{i_{1},\dots,i_{\alpha}}\mathbb{E}\left[\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+z)\right\rvert\cdot\left\lvert\prod_{k\leq\alpha-\beta}W_{j_{k}}^{(i_{k})}\prod_{k>\alpha-\beta}W_{J}^{(i_{k})}\right\rvert\cdot\mathbb{I}_{W_{J},1}\right]$
$\displaystyle=\sum_{i_{1},\dots,i_{\alpha}}\mathbb{E}\left[\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+z)\right\rvert\right]\mathbb{E}\left[\left\lvert\prod_{k\leq\alpha-\beta}W_{j_{k}}^{(i_{k})}\prod_{k>\alpha-\beta}W_{J}^{(i_{k})}\right\rvert\cdot\mathbb{I}_{W_{J},1}\right]$
$\displaystyle\leq\mathbb{E}\left[\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}_{j}+z)\right\rvert\right]\left\lVert\mathbb{E}\left[\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert\right]\right\rVert_{\infty},$

where $\mathcal{B}=\\{z:\lVert
z\rVert_{\infty}\leq\frac{2\delta^{\prime}}{\sqrt{\log(ep)}}\\}$, and
$\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert$ is the element-wise absolute operation. The
fifth equality follows the independence between $W^{o}$ and
$\mathscr{W}_{\\{j_{1},\dots,j_{\alpha-\beta}\\}\cup J}$. We decompose the
expectation term on the last line by

$\displaystyle\mathbb{E}\left[\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}_{j}+z)\right\rvert\right]$
(39) $\displaystyle\leq\mathbb{E}\left[\begin{aligned}
\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}_{j}+z)\right\rvert\cdot\mathbb{I}_{W^{o},1}\end{aligned}\right]$
$\displaystyle\quad+\mathbb{E}\left[\begin{aligned}
\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}_{j}+z)\right\rvert\cdot\mathbb{I}_{W^{o},2}\end{aligned}\right],$

where the value of $h>0$ will be determined later, and

$\displaystyle\mathbb{I}_{W^{o},1}$ $\displaystyle=\mathbb{I}_{\left\\{\lVert
W^{o}-\partial A_{r}\rVert_{\infty}\leq
12{\delta^{\prime}}\sqrt{\log(ph)}\right\\}},$
$\displaystyle\mathbb{I}_{W^{o},2}$ $\displaystyle=\mathbb{I}_{\left\\{\lVert
W^{o}-\partial
A_{r}\rVert_{\infty}>12{\delta^{\prime}}\sqrt{\log(ph)}\right\\}}.$

For the first term, based on Lemmas 6.1 and 6.2 of Chernozhukov, Chetverikov
and Koike (2023),

$\displaystyle\mathbb{E}\left[\begin{aligned}
\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+z)\right\rvert\cdot\mathbb{I}_{W^{o},1}\end{aligned}\right]$
$\displaystyle\leq\sup_{w\in\mathbb{R}^{p}}\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(w+z)\right\rvert\cdot\mathbb{E}\left[\mathbb{I}_{W^{o},1}\right]$
$\displaystyle\leq
C\frac{\phi^{\gamma_{1}}(\log(ep))^{(\alpha-\gamma_{1})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{1}}}\mathbb{P}\left[\lVert
X_{J_{1}}-\partial A_{r-Y_{J_{2}}^{o}}\rVert_{\infty}\leq
12{\delta^{\prime}}\sqrt{\log(ph)}\right]$ $\displaystyle\leq
C\frac{\phi^{\gamma_{1}}(\log(ep))^{(\alpha-\gamma_{1})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{1}}}\min\left\\{1,\kappa_{J_{1}}\left(12{\delta^{\prime}}\sqrt{\log(ph)}\right)\right\\},$

for any $\gamma_{1}\in[0,1]$. For the last term, based on Lemma 10.5 of Lopes
(2022),

$\displaystyle\mathbb{E}\left[\begin{aligned}
\sum_{i_{1},\dots,i_{\alpha}}\sup_{z\in\mathcal{B}}\left\lvert\nabla^{(i_{1},\dots,i_{\alpha})}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+z)\right\rvert\cdot\mathbb{I}_{W^{o},2}\end{aligned}\right]\leq
C\frac{1}{{\delta^{\prime}}^{\alpha}h}.$

Plugging the last two results in Eqs. 38 and 39, the resulting upper bound for
$T_{W_{J},1}(t)$ is

$\displaystyle\left\lvert T_{W_{J},1}(t)\right\rvert$ $\displaystyle\leq
C\left[\frac{\phi^{\gamma_{1}}(\log(ep))^{(\alpha-\gamma_{1})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{1}}}\min\left\\{1,\kappa_{J_{1}}\left(12{\delta^{\prime}}\sqrt{\log(ph)}\right)\right\\}+\frac{1}{{\delta^{\prime}}^{\alpha}h}\right]$
$\displaystyle\quad\times\left\lVert\mathbb{E}\left[\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert\right]\right\rVert_{\infty},$

for any $t\in[0,1]$, $\gamma_{1}\in[0,1]$ and $h>0$. Replacing the minimum
with $1$ and minimizing over $h>0$, we get

$\displaystyle\left\lvert T_{W_{J},1}(t)\right\rvert\leq
C\frac{\phi^{\gamma_{1}}(\log(ep))^{(\alpha-\gamma_{1})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{1}}}\left\lVert\mathbb{E}\left[\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert\right]\right\rVert_{\infty}.$

Moreover, plugging-in

$h=\frac{\sigma_{\min}}{\phi^{\gamma_{1}}{\delta^{\prime}}^{1+\gamma_{1}}}\sqrt{\frac{\lvert
J_{1}\rvert}{(\log(ep))^{\alpha-\gamma_{1}+2}}},$

we obtain

$\displaystyle\left\lvert T_{W_{J},1}(t)\right\rvert\leq
C\frac{\phi^{\gamma_{1}}(\log(ep))^{(\alpha-\gamma_{1})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{1}}}\left\lVert\mathbb{E}\left[\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert\right]\right\rVert_{\infty}(\kappa_{J_{1}}(\delta^{o})+\kappa^{o}),$

where $\delta^{o}=12\delta^{\prime}\sqrt{\log(p\lvert J_{1}\rvert)}$ and
$\kappa^{o}=\frac{\delta^{\prime}\log(ep)}{\sigma_{\min}\sqrt{\lvert
J_{1}\rvert}}$. In sum,

$\displaystyle\left\lvert T_{W_{J},1}(t)\right\rvert$ $\displaystyle\leq
C\frac{\phi^{\gamma_{1}}(\log(ep))^{(\alpha-\gamma_{1})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{1}}}\left\lVert\mathbb{E}\left[\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert\right]\right\rVert_{\infty}\min\\{1,\kappa_{J_{1}}(\delta^{o})+\kappa^{o}\\},$

for any $t\in[0,1]$, $\gamma_{1}\in[0,1]$ as long as $\delta\geq\sigma_{\min}$
and $\phi\delta\geq\frac{1}{\log(ep)}$. Now we bound

$\displaystyle\left\lvert
T_{W_{J},2}(t)\right\rvert=\left\lvert\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}(W^{o}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\mathbb{I}_{W_{J},2}\right]\right\rvert.$

Conditional on $W^{\not\\!\perp\\!\\!\\!\\!\perp}$ and $W_{J}$,

$\displaystyle\mathbb{E}\left[\left.\left\lVert\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}\left(W^{o}+W^{{\not\\!\perp\\!\\!\\!\\!\perp}}+tW_{J}\right)\right\rVert_{1}\right|W^{{\not\\!\perp\\!\\!\\!\\!\perp}},W_{J}\right]$
$\displaystyle\leq\mathbb{E}\left[\left.\left\lVert\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}\left(W^{o}+W^{{\not\\!\perp\\!\\!\\!\\!\perp}}+tW_{J}\right)\right\rVert_{1}\mathbb{I}_{\\{\lVert
W^{o}-\partial A_{r^{\prime}}\rVert_{\infty}\leq
10\delta^{\prime}\sqrt{\log(ph)}\\}}\right|W^{{\not\\!\perp\\!\\!\\!\\!\perp}},W_{J}\right]$
$\displaystyle\quad+\mathbb{E}\left[\left.\left\lVert\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}\left(W^{o}+W^{{\not\\!\perp\\!\\!\\!\\!\perp}}+tW_{J}\right)\right\rVert_{1}\mathbb{I}_{\\{\lVert
W^{o}-\partial
A_{r^{\prime}}\rVert_{\infty}>10\delta^{\prime}\sqrt{\log(ph)}\\}}\right|W^{{\not\\!\perp\\!\\!\\!\\!\perp}},W_{J}\right],$

where $r^{\prime}=r-W^{\not\\!\perp\\!\\!\\!\\!\perp}-tW_{J}$ is deterministic
given $W^{\not\\!\perp\\!\\!\\!\\!\perp}$ and $W_{J}$. Applying Lemmas 6.1,
6.2 of Chernozhukov, Chetverikov and Koike (2023) and Lemma 10.5 of Lopes
(2022) to the two terms, respectively,

$\displaystyle\mathbb{E}\left[\left.\left\lVert\nabla^{\alpha}\rho_{r,\phi}^{\delta^{\prime}}\left(W^{o}+W^{{\not\\!\perp\\!\\!\\!\\!\perp}}+tW_{J}\right)\right\rVert_{1}\right|W^{{\not\\!\perp\\!\\!\\!\\!\perp}},W_{J}\right]$
$\displaystyle\leq
C\frac{\phi^{\gamma_{2}}(\log(ep))^{(\alpha-\gamma_{2})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{2}}}\mathbb{P}\left[\left.\lVert
W^{o}-\partial A_{r^{\prime}}\rVert_{\infty}\leq
10\delta^{\prime}\sqrt{\log(ph)}\right|W^{\not\\!\perp\\!\\!\\!\\!\perp},W_{J}\right]+C\frac{1}{\delta^{\prime\alpha}h}$
$\displaystyle\leq
C\frac{\phi^{\gamma_{2}}(\log(ep))^{(\alpha-\gamma_{2})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{2}}}\min\left\\{1,\kappa_{J_{1}}\left(10\delta^{\prime}\sqrt{\log(ph)}\right)\right\\}+C\frac{1}{\delta^{\prime\alpha}h},$

almost surely, for any $\gamma_{2}\in[0,1]$ and $h>0$. Putting the last two
results together,

$\displaystyle\left\lvert T_{W_{J},2}(t)\right\rvert$ $\displaystyle\leq
C\left(\frac{\phi^{\gamma_{2}}(\log(ep))^{(\alpha-\gamma_{2})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{2}}}\min\left\\{1,\kappa_{J_{1}}\left(10\delta^{\prime}\sqrt{\log(ph)}\right)\right\\}+\frac{1}{\delta^{\prime\alpha}h}\right)$
$\displaystyle\quad\times\mathbb{E}\left[\begin{aligned}
\prod_{k=1}^{\alpha-\beta}\lVert W_{j_{k}}\rVert_{\infty}\lVert
W_{J}\rVert_{\infty}^{\beta}\mathbb{I}_{W_{J},2}\end{aligned}\right],$

for any $\gamma_{2}\in[0,1]$ and $h>0$. Because
$\mathbb{I}_{W_{J},2}=\mathbb{I}_{\\{\lVert W_{J}\rVert_{\infty}\text{ or
}\lVert
W^{{\not\\!\perp\\!\\!\\!\\!\perp}}\rVert_{\infty}\geq\frac{\delta^{\prime}}{\sqrt{\log(ep)}}\\}}\geq\mathbb{I}_{\\{\lVert
W_{J}\rVert_{\infty}>\frac{\delta^{\prime}}{\sqrt{\log(ep)}}\\}}+\mathbb{I}_{\\{\lVert
W^{\not\\!\perp\\!\\!\\!\\!\perp}\rVert_{\infty}>\frac{\delta^{\prime}}{\sqrt{\log(ep)}}\\}}$,

$\displaystyle\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert
W_{J}\rVert_{\infty}^{\beta}\mathbb{I}_{W_{J},2}\right]$
$\displaystyle\leq\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert
W_{J}\rVert_{\infty}^{\beta}\mathbb{I}_{\\{\lVert
W_{J}\rVert_{\infty}>\frac{\delta^{\prime}}{\sqrt{\log(ep)}}\\}}\right]+\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert
W_{J}\rVert_{\infty}^{\beta}\mathbb{I}_{\\{\lVert
W^{\not\\!\perp\\!\\!\\!\\!\perp}\rVert_{\infty}>\frac{\delta^{\prime}}{\sqrt{\log(ep)}}\\}}\right]$
$\displaystyle\leq\frac{(\log(ep))^{\eta/2}}{\delta^{\prime\eta}}\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert W_{J}\rVert_{\infty}^{\beta}(\lVert
W_{J}\rVert_{\infty}^{\eta}+\lVert
W^{\not\\!\perp\\!\\!\\!\\!\perp}\rVert_{\infty}^{\eta})\right],$

for any $\eta>0$. The resulting upper bound for $T_{W_{J},2}(t)$ is

$\displaystyle\left\lvert T_{W_{J},2}(t)\right\rvert$ $\displaystyle\leq
C\left[\frac{\phi^{\gamma_{2}}(\log(ep))^{(\alpha-\gamma_{2})/2}}{{\delta^{\prime}}^{\alpha-\gamma_{2}}}\min\left\\{1,\kappa_{J_{1}}\left(10\delta^{\prime}\sqrt{\log(ph)}\right)\right\\}+\frac{1}{\delta^{\prime\alpha}h}\right]$
$\displaystyle\quad\times\frac{(\log(ep))^{\eta/2}}{\delta^{\prime\eta}}\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert W_{J}\rVert_{\infty}^{\beta}(\lVert
W_{J}\rVert_{\infty}^{\eta}+\lVert
W^{\not\\!\perp\\!\\!\\!\\!\perp}\rVert_{\infty}^{\eta})\right],$

for any $\gamma_{2}\in[0,1]$, $\eta>0$ and $h>0$. By similar choices of $h$
with for $T_{W_{J},1}(t)$, we obtain

$\displaystyle\left\lvert T_{W_{J},2}(t)\right\rvert$ $\displaystyle\leq
C\left[\frac{\phi^{\gamma_{2}}(\log(ep))^{(\alpha-\gamma_{2}+\eta)/2}}{{\delta^{\prime}}^{\alpha-\gamma_{2}+\eta}}\min\left\\{1,\kappa_{J_{1}}\left(\delta^{o}\right)+\kappa^{o}\right\\}\right]$
$\displaystyle\quad\times\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert W_{J}\rVert_{\infty}^{\beta}(\lVert
W_{J}\rVert_{\infty}^{\eta}+\lVert
W^{\not\\!\perp\\!\\!\\!\\!\perp}\rVert_{\infty}^{\eta})\right],$

for any $t\in[0,1]$ and $\eta>0$ as long as $\delta\geq\sigma_{\min}$ and
$\phi\delta\geq\frac{1}{\log(ep)}$. Putting the upperbounds for
$T_{W_{J},1}(t)$ and $T_{W_{J},2}(t)$ together, we obtain the following upper
bound for the entire remainder term:

$\displaystyle\left\lvert\int_{0}^{1}(1-t)^{\beta-1}\mathbb{E}\left[\left\langle\nabla^{\alpha}\rho_{r,\phi}^{\delta}(X_{J_{1}}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{J_{2}}+tW_{J}),(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rangle\right]dt\right\rvert$ $\displaystyle\leq
C\left[\frac{(\log(ep))^{\alpha/2}}{{\delta^{\prime}}^{\alpha}}\min\left\\{1,\kappa_{J_{1}}\left(\delta^{o}\right)+\kappa^{o}\right\\}\right]$
$\displaystyle\quad\times\left[\begin{aligned}
&\frac{\phi^{\gamma_{1}}{\delta^{\prime}}^{\gamma_{1}}}{(\log(ep))^{\gamma_{1}/2}}\left\lVert\mathbb{E}\left[\left\lvert(\otimes_{k=1}^{\alpha-\beta}W_{j_{k}})\otimes
W_{J}^{\otimes\beta}\right\rvert\right]\right\rVert_{\infty}\\\
&+\frac{\phi^{\gamma_{2}}{\delta^{\prime}}^{\gamma_{2}-\eta}}{(\log(ep))^{(\gamma_{2}-\eta)/2}}\mathbb{E}\left[\prod_{k=1}^{\alpha-\beta}\lVert
W_{j_{k}}\rVert_{\infty}\lVert W_{J}\rVert_{\infty}^{\beta}(\lVert
W_{J}\rVert_{\infty}^{\eta}+\lVert
W^{\not\\!\perp\\!\\!\\!\\!\perp}\rVert_{\infty}^{\eta})\right]\end{aligned}\right],$

for any $\gamma_{1},\gamma_{2}\in[0,1]$ and $\eta>0$ as long as
$\delta\geq\sigma_{\min}$ and $\phi\delta\geq\frac{1}{\log(ep)}$. This proves
Lemma B.3.

Lemma B.3 implies all the remainder theorems in the main text and appendices
of this paper. For example, in Lemma 4.3, one of the third-order remainder
terms in $\mathfrak{R}^{(3,1)}_{W_{j}}$ is

$\displaystyle\mathbb{E}\left[\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W^{\mathsf{C}}_{[j,j]}+tW_{j}\right),X_{j}^{\otimes
3}\right\rangle~{}dt\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(X_{[1,j-1)}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{(j+1,n]}+tW_{j}\right),X_{j}^{\otimes
3}\right\rangle~{}dt\right],$

where $W_{[i,j]}^{\mathsf{C}}\equiv X_{[1,i)}+Y_{(j,n]}$ and
$W^{\not\\!\perp\\!\\!\\!\\!\perp}\equiv X_{j-1}+Y_{j+1}$. Because of
Assumption (MIN-EV),
$\operatorname{Var}[Y_{(j+1,n]}|\mathscr{Y}_{(j+1,n]}]\succeq\underline{\sigma}^{2}\max\\{n-j-3,0\\}\cdot
I_{p}$. Let
$\underline{\sigma}_{j}^{2}\equiv\underline{\sigma}^{2}\max\\{j,0\\}$ and
$\delta_{j}^{2}\equiv\sqrt{\delta^{2}+\underline{\sigma}_{j}^{2}}$. Applying
Lemma B.3 to this term with $\gamma_{1}=0$ and $\gamma_{2}=\eta=q-3$, we
obtain for $q\geq 3$,

$\displaystyle\left\lvert\mathbb{E}\left[\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(X_{[1,j-1)}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{(j+1,n]}+tW_{j}\right),W_{j}^{\otimes
3}\right\rangle~{}dt\right]\right\rvert$ $\displaystyle\leq
C\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\min\\{1,\kappa_{[1,j-1)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\quad\times\left[\left\lVert\mathbb{E}\left[\left\lvert
W_{j}^{\otimes
3}\right\rvert\right]\right\rVert_{\infty}+\phi^{q-3}\mathbb{E}\left[\lVert
W_{j}\rVert_{\infty}^{3}(\lVert W_{j}\rVert_{\infty}^{q-3}+\lVert
X_{j-1}+Y_{j+1}\rVert_{\infty}^{q-3})\right]\right]$ $\displaystyle\leq
C\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\left[L_{3,j}+(2\phi)^{q-3}(\nu_{q,j-1}+\nu_{q,j}+\nu_{q,j+1})\right]\min\\{1,\kappa_{[1,j-1)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\},$

where $\delta^{o}_{n-j}\equiv 12\delta_{n-j}\sqrt{\log(pn)}$ and
$\kappa^{o}_{j}\equiv\frac{\delta_{n-j}\log(ep)}{\sigma_{\min}\sqrt{\max\\{j,1\\}}}$
as long as $\delta\geq\sigma_{\min}$ and $\phi\delta\geq\frac{1}{\log(ep)}$.
Based on similar applications of Lemma B.3 to the other terms of
$\mathfrak{R}^{(3,1)}_{X_{j}}$, for $q\geq 3$,

$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W_{j}}^{(3,1)}\right]\right\rvert$
$\displaystyle\leq
C\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\left(\sum_{j^{\prime}=j-3}^{j+3}L_{3,j^{\prime}}+(2\phi)^{q-3}\sum_{j^{\prime}=j-3}^{j+3}\nu_{q,j^{\prime}}\right)$
$\displaystyle\quad\times\min\\{\kappa_{[1,j-3)}(\delta_{n-j}^{o})+\kappa^{o}_{j},1\\},$

as long as $\delta\geq\sigma_{\min}$ and $\phi\delta\geq\frac{1}{\log(ep)}$.
Similar arguments and bounds apply to $\mathfrak{R}_{W}^{(3)}$ and
$\mathfrak{R}_{W_{j}}^{(3,2)}$. For the fourth-order remainder terms, we
consider the first term of $\mathfrak{R}_{W_{j}}^{(4,1)}$:

$\displaystyle\mathbb{E}\left[\int_{0}^{1}(1-t)^{3}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(W^{\mathsf{C}}_{[j,j]}+tX_{j}),X_{j}^{\otimes
4}\right\rangle~{}dt\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}(1-t)^{3}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{[1,j-1)}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{(j+1,n]}+tX_{j}),X_{j}^{\otimes
4}\right\rangle~{}dt\right].$

Applying Lemma B.3 to this term with $\gamma_{1}=\gamma_{2}=1$ and $\eta=q-4$,
we obtain for $q\geq 4$,

$\displaystyle\left\lvert\mathbb{E}\left[\int_{0}^{1}(1-t)^{3}\left\langle\nabla^{4}\varphi_{r}^{\varepsilon}\left(X_{[1,j-1)}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{(j+1,n]}+tW_{j}\right),W_{j}^{\otimes
4}\right\rangle~{}dt\right]\right\rvert$ $\displaystyle\leq
C\phi\min\\{1,\kappa_{[1,j-1)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\quad\times\left[\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\left\lVert\mathbb{E}\left[\left\lvert
W_{j}^{\otimes
4}\right\rvert\right]\right\rVert_{\infty}+\frac{(\log(ep))^{(q-1)/2}}{(\delta_{n-j}/2)^{q-1}}\mathbb{E}\left[\lVert
W_{j}\rVert_{\infty}^{4}(\lVert W_{j}\rVert_{\infty}^{q-4}+\lVert
X_{j-1}+Y_{j+1}\rVert_{\infty}^{q-4})\right]\right]$ $\displaystyle\leq
C\phi\left[\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}L_{4,j}+\frac{(\log(ep))^{(q-1)/2}}{(\delta_{n-j}/2)^{q-1}}(\nu_{q,j-1}+\nu_{q,j}+\nu_{q,j+1})\right]\min\\{1,\kappa_{[1,j-1)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\},$

as long as $\delta\geq\sigma_{\min}$ and $\phi\delta\geq\frac{1}{\log(ep)}$.
Based on similar applications of Lemma B.3 to the other terms of
$\mathfrak{R}^{(4,1)}_{X_{j}}$, for $q\geq 4$,

$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{W_{j}}^{(4,1)}\right]\right\rvert$
$\displaystyle\leq
C\phi\left(\frac{(\log(ep))^{3/2}}{\delta_{n-j}^{3}}\sum_{j^{\prime}=j-4}^{j+4}L_{4,j^{\prime}}+\frac{(\log(ep))^{(q-1)/2}}{(\delta_{n-j}/2)^{q-1}}\sum_{j^{\prime}=j-4}^{j+4}\nu_{q,j^{\prime}}\right)$
$\displaystyle\quad\times\min\\{\kappa_{[1,j-4)}(\delta_{n-j}^{o})+\kappa^{o}_{j},1\\},$

as long as $\delta\geq\sigma_{\min}$ and $\phi\delta\geq\frac{1}{\log(ep)}$.
Similar arguments and bounds apply to $\mathfrak{R}_{W}^{(4)}$ and
$\mathfrak{R}_{W_{j}}^{(4,2)}$. Finally, for the sixth-order remainder terms,
we consider the first term of $\mathfrak{R}_{X_{j},W_{k}}^{(6,1)}$:

$\displaystyle\mathbb{E}\left[\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+tX_{k}+Y_{(k,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes W_{k}^{\otimes 3}\right\rangle~{}dt\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+W^{\not\\!\perp\\!\\!\\!\\!\perp}+Y_{(k+1,j-1)\cup(j+1,n)})+tX_{k},\mathbb{E}[X_{j}^{\otimes
3}]\otimes W_{k}^{\otimes 3}\right\rangle~{}dt\right],$

where $W^{\not\\!\perp\\!\\!\\!\\!\perp}\equiv X_{k-1}+Y_{k+1}$. Applying
Lemma B.3 to this term with $\gamma_{1}=\gamma_{2}=1$ and $\eta=q-3$, we
obtain for $q\geq 4$,

$\displaystyle\left\lvert\mathbb{E}\left[\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+tX_{k}+Y_{(k,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes W_{k}^{\otimes 3}\right\rangle~{}dt\right]\right\rvert$
$\displaystyle\leq
C\phi\min\\{1,\kappa_{[1,j-1)}(\delta_{n-j}^{o})+\kappa^{o}_{j}\\}$
$\displaystyle\quad\times\left[\begin{aligned}
&\frac{(\log(ep))^{3/2}}{\delta_{n-k}^{3}}\left\lVert\mathbb{E}\left[\left\lvert
X_{j}^{\otimes 3}\right\rvert\right]\mathbb{E}\left[\left\lvert W_{k}^{\otimes
3}\right\rvert\right]\right\rVert_{\infty}\\\
&+\frac{(\log(ep))^{(q-1)/2}}{(\delta_{n-k}/2)^{q-1}}\left\lVert\mathbb{E}\left[\left\lvert
X_{j}^{\otimes
3}\right\rvert\right]\right\rVert_{\infty}\mathbb{E}\left[\lVert
W_{k}\rVert_{\infty}^{3}(\lVert W_{k}\rVert_{\infty}^{q-3}+\lVert
X_{k-1}+Y_{k+1}\rVert_{\infty}^{q-3})\right]\end{aligned}\right]$
$\displaystyle\leq C\phi
L_{3,j}\left(L_{3,k}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+(\nu_{q,k-1}+\nu_{q,k}+\nu_{q,k+1})\frac{{(\log(ep))}^{(q+2)/2}}{(\delta_{n-k}/2)^{q+2}}\right)$
$\displaystyle\quad\times\min\\{\kappa_{[1,k-1)}(\delta_{n-k-5}^{o})+\kappa^{o}_{k},1\\}.$

Similarly, we get

$\displaystyle\left\lvert\mathbb{E}\left[\mathfrak{R}_{X_{j},W_{k}}^{(6,1)}\right]\right\rvert$
$\displaystyle\leq C\phi L_{3,j}\left(\begin{aligned}
\sum_{k^{\prime}=k-2}^{k+2}L_{3,k^{\prime}}\frac{(\log(ep))^{5/2}}{\delta_{n-k}^{5}}+\sum_{k^{\prime}=k-3}^{k+3}\nu_{q,k^{\prime}}\frac{{(\log(ep))}^{(q+2)/2}}{(\delta_{n-k}/2)^{q+2}}\end{aligned}\right)$
$\displaystyle\quad\times\min\\{\kappa_{[1,k-3)}(\delta_{n-k}^{o})+\kappa^{o}_{k},1\\}.$

Similar arguments and bounds apply to the other sixth-order remainder terms.

###  B.4 Proof of Anti-concentration Lemma

We use a similar approach with Section B.3 to upper bound
$\mathbb{E}\left[\mathfrak{R}_{X_{\\{2,i-1\\}},1}|\mathscr{X}_{(i,n]}\right]$.
First, we decompose
$\mathbb{E}\left[\mathfrak{R}_{X_{\\{2,i-1\\}},1}|\mathscr{X}_{(i,n]}\right]$
by

$\displaystyle\mathbb{E}\left[\mathfrak{R}_{X_{\\{2,i-1\\}},1}|\mathscr{X}_{(i,n]}\right]$
$\displaystyle=\mathbb{E}\left[\left.\int_{0}^{1}\left\langle\nabla\varphi_{r,\delta+\varepsilon^{o}}^{\varepsilon}(X_{[1,i]}-tX_{\\{2,i-1\\}}),X_{\\{2,i-1\\}}\right\rangle
dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle=\mathbb{E}\left[\left.\int_{0}^{1}\left\langle\nabla\varphi_{r,\delta+\varepsilon^{o}}^{\varepsilon}(X_{[1,i]}-tX_{\\{2,i-1\\}}),\mathscr{X}_{\\{2,i-1\\}}\right\rangle\mathbb{I}_{\leq}(t)dt\right|X_{(i,n]}\right]$
$\displaystyle\quad+\mathbb{E}\left[\left.\int_{0}^{1}\left\langle\nabla\varphi_{r,\delta+\varepsilon^{o}}^{\varepsilon}(X_{[1,i]}-tX_{\\{2,i-1\\}}),X_{\\{2,i-1\\}}\right\rangle\mathbb{I}_{>}(t)dt\right|\mathscr{X}_{(i,n]}\right],$

where $\mathbb{I}_{\leq}(t)=\mathbb{I}\\{\lVert
X_{[1,i]}-tX_{\\{2,i-1\\}}-\partial
A_{r,\delta+\varepsilon^{o}}\rVert_{\infty}\leq\varepsilon^{o}\\}$,
$\mathbb{I}_{>}(t)=\mathbb{I}\\{\lVert X_{[1,i]}-tX_{\\{2,i-1\\}}-\partial
A_{r,\delta+\varepsilon^{o}}\rVert_{\infty}>\varepsilon^{o}\\}$. Based on
Lemma 2.3 of Fang and Koike (2020b),

$\displaystyle\mathbb{E}\left[\left.\int_{0}^{1}\left\langle\nabla\varphi_{r,\delta+\varepsilon^{o}}^{\varepsilon}(X_{[1,i]}-tX_{\\{2,i-1\\}}),X_{\\{2,i-1\\}}\right\rangle\mathbb{I}_{\leq}(t)dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle\leq\mathbb{E}\left[\left.\int_{0}^{1}\lVert\nabla\varphi_{r,\delta+\varepsilon^{o}}^{\varepsilon}(X_{[1,i]}-tX_{\\{2,i-1\\}})\rVert_{1}\mathbb{I}_{\leq}(t)\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle\leq
C\frac{\sqrt{\log(ep)}}{\varepsilon}\mathbb{E}\left[\left.\int_{0}^{1}\mathbb{I}_{\leq}(t)\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}dt\right|\mathscr{X}_{(i,n]}\right]$

We use Tonelli’s theorem to switch the order between the integration and the
expectation conditional on $\mathscr{X}_{(i-2,3)}$:

$\displaystyle\mathbb{E}\left[\left.\int_{0}^{1}\mathbb{I}_{\leq}(t)\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle\leq\mathbb{E}\left[\left.\int_{0}^{1}\mathbb{E}\left[\left.\mathbb{I}_{\leq}(t)\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}\right||\mathscr{X}_{(i-2,n+3)}\right]dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle=\mathbb{E}\left[\left.\int_{0}^{1}\left(\begin{aligned}
&\mathbb{P}[X_{[3,i-2]}\in
A_{r_{1},\varepsilon^{o}}|\mathscr{X}_{(i-2,n+3)}]\\\
&+\mathbb{P}[X_{[3,i-2]}\in
A_{r_{2},\varepsilon^{o}}|\mathscr{X}_{(i-2,n+3)}]\end{aligned}\right)\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle\leq\kappa_{[3,i-2]}(\varepsilon^{o})\mathbb{E}[\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}|\mathscr{X}_{(i,n]}]=\kappa_{[3,i-2]}(\varepsilon^{o})(\nu_{1}(X_{2})+\nu_{1}(X_{i-1})),$

where
$r_{1}=r-(1-t)X_{\\{2,i-1\\}}-X_{1}-X_{i}-(\delta+\varepsilon^{o})\mathbf{1}$
and
$r_{2}=r-(1-t)X_{\\{2,i-1\\}}-X_{1}-X_{i}+(\delta+\varepsilon^{o})\mathbf{1}$
are Borel measurable functions with respect to $X_{(i-2,n+3)}$. On the other
hand, based on Lemma 10.5 of Lopes (2022),

$\displaystyle\mathbb{E}\left[\left.\int_{0}^{1}\left\langle\nabla\varphi_{r,\delta+\varepsilon^{o}}^{\varepsilon}(X_{[1,i]}-tX_{\\{2,i-1\\}}),X_{\\{2,i-1\\}}\right\rangle\mathbb{I}_{>}(t)dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle\leq\mathbb{E}\left[\left.\int_{0}^{1}\lVert\nabla\varphi_{r,\delta+\varepsilon^{o}}^{\varepsilon}(X_{[1,i]}-tX_{\\{2,i-1\\}})\rVert_{1}\mathbb{I}_{>}(t)\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle=\frac{1}{\varepsilon
h^{4}}\mathbb{E}\left[\left.\int_{0}^{1}\mathbb{E}\left[\left.\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}\right|\mathscr{X}_{(i-2,n+3)}\right]dt\right|\mathscr{X}_{(i,n]}\right]$
$\displaystyle=\frac{1}{\varepsilon h^{4}}\mathbb{E}[\lVert
X_{\\{2,i-1\\}}\rVert_{\infty}|\mathscr{X}_{(i,n]}]=\frac{1}{\varepsilon
h^{4}}(\nu_{1,2}+\nu_{1,i-1}).$

Putting the above result and Eq. 21 back to Eq. 20, we get for any $h>0$,

$\displaystyle\mathbb{P}[X_{[1,i]}\in A_{r,\delta}|\mathscr{X}_{(i,n]}]$
$\displaystyle\leq(\nu_{1,2}+\nu_{1,i-1})\left(C\frac{\sqrt{\log(ep)}}{\varepsilon}\max\left\\{1,\kappa_{[3,i-2]}(\varepsilon^{o})\right\\}+\frac{1}{\varepsilon
h^{4}}\right)$
$\displaystyle\quad+C\frac{\delta+2\varepsilon^{o}}{\sigma_{\min}}\sqrt{\frac{\log(ep)}{i-2}}+2\mu_{[3,i-2]}.$

Plugging in
$h=\left(\frac{\sigma_{\min}}{\varepsilon}\sqrt{\frac{i-2}{(\log(ep))^{2}}}\right)^{1/4}$,

$\displaystyle\mathbb{P}[X_{[1,i]}\in A_{r,\delta}|\mathscr{X}_{(i,n]}]$
$\displaystyle\leq
C(\nu_{1,2}+\nu_{1,i-1})\frac{\sqrt{\log(ep)}}{\varepsilon}\max\left\\{1,\kappa_{[3,i-2]}(\varepsilon^{o})+\kappa^{o}_{i-2}\right\\}$
$\displaystyle\quad+C\frac{\delta+2\varepsilon^{o}}{\sigma_{\min}}\sqrt{\frac{\log(ep)}{i-2}}+2\mu_{[3,i-2]},$

where $\varepsilon^{o}=10\varepsilon\sqrt{\log(p\max\\{i_{0}-2,1\\})}$ and
$\kappa^{o}_{j}=\frac{\varepsilon}{\sigma_{\min}}\sqrt{\frac{\log(ep)}{\max\\{j,1\\}}}$,
as long as $\varepsilon\geq\sigma_{\min}$. Because the righthand side is not
dependent on $r$, for any $\varepsilon\geq\sigma_{\min},\delta>0$,

$\displaystyle\kappa_{[1,i]}(\delta)$ $\displaystyle\leq
C\left((\nu_{1,2}+\nu_{1,i-1})\frac{\sqrt{\log(ep)}}{\varepsilon}\kappa_{[3,i-2]}(\varepsilon^{o})+\mu_{[3,i-2]}\right)$
$\displaystyle\quad+C\frac{\delta+2\varepsilon^{o}}{\sigma_{\min}}\sqrt{\frac{\log(ep)}{i-2}}+C\frac{\overline{\nu}_{1,(1,i)}}{\sigma_{\min}}\frac{\log(ep)}{\sqrt{i-2}}.$

###  B.5 Proof of Lemma A.7

Suppose that $j,k\geq\frac{n}{2}$ and $\alpha\leq 1$. Because $x\mapsto
x^{\alpha}$ is concave, by Jensen’s inequality, for any $q_{1},q_{2}>0$,

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}L_{q_{1},i+j}\overline{L}_{q_{2},(i,i+j)}^{\alpha}$
$\displaystyle\leq\overline{L}_{q_{1}}\left(\frac{\frac{1}{n}\sum_{i=1}^{n}L_{q_{1},i+j}\overline{L}_{q_{2},(i,i+j)}}{\overline{L}_{q_{1}}}\right)^{\alpha}$
$\displaystyle\leq\overline{L}_{q_{1}}\left(\frac{\frac{1}{n(j-1)}\sum_{i=1}^{n}L_{q_{1},i}\sum_{i=1}^{n}L_{q_{2},i}}{\overline{L}_{q_{1}}}\right)^{\alpha}$
$\displaystyle\leq\overline{L}_{q_{1}}\left(\frac{n}{j-1}\overline{L}_{q_{2}}\right)^{\alpha}\leq
2^{\alpha}\overline{L}_{q_{1}}\overline{L}_{q_{2}}.$

Similarly, for any $q_{1},q_{2},q_{3}>0$,

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}\frac{L_{q_{1},i+j}}{j-1}\sum_{l=1}^{j-1}L_{q_{2},i+[k+l]_{j-1}}\overline{L}_{q_{3},i+(l,k+l)_{j-1}}^{\alpha}$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}\frac{L_{q_{1},i+j}}{j-1}\sum_{l=1}^{j-1}L_{q_{2},i+[k+l]_{j-1}}\left(\frac{\frac{1}{n}\sum_{i=1}^{n}\frac{L_{q_{1},i+j}}{j-1}\sum_{l=1}^{j-1}L_{q_{2},i+[k+l]_{j-1}}\overline{L}_{q_{3},i+(l,k+l)_{j-1}}}{\frac{1}{n}\sum_{i=1}^{n}\frac{L_{q_{1},i+j}}{j-1}\sum_{l=1}^{j-1}L_{q_{2},i+[k+l]_{j-1}}}\right)^{\alpha}$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}L_{q_{1},i+j}\overline{L}_{q_{2},(i,i+j)}\left(\frac{\frac{1}{j-1}\sum_{i=1}^{n}\frac{L_{q_{1},i+j}}{j-1}\sum_{l=1}^{j-1}L_{q_{2},i+[k+l]_{j-1}}\overline{L}_{q_{3}}}{\frac{1}{n}\sum_{i=1}^{n}\frac{L_{q_{1},i+j}}{j-1}\sum_{l=1}^{j-1}L_{q_{2},i+[k+l]_{j-1}}}\right)^{\alpha}$
$\displaystyle\leq\frac{1}{n}\sum_{i=1}^{n}L_{q_{1},i+j}\overline{L}_{q_{2},(i,i+j)}\left(\frac{n}{j-1}\overline{L}_{q_{3}}\right)^{\alpha}$
$\displaystyle\leq\frac{2^{\alpha}}{n}\sum_{i=1}^{n}L_{q_{1},i+j}\overline{L}_{q_{2},(i,i+j)}\overline{L}_{q_{3}}^{\alpha}$
$\displaystyle\leq
2^{\alpha+1}\overline{L}_{q_{1}}\overline{L}_{q_{2}}\sum_{l=1}^{j-1}L_{q_{2},i+[k+l]_{j-1}}\overline{L}_{q_{3}}^{\alpha}.$

##  Appendix C Details of the Taylor expansions for $m=1$

###  C.1 Breaking the ring

We apply the Taylor expansion to $\rho_{r,\phi}^{\delta}(X_{[1,n]})$ centered
at $\rho_{r,\phi}^{\delta}(X_{[1,n)})$ as follows:

$\displaystyle\rho_{r,\phi}^{\delta}(X_{[1,n]})-\rho_{r,\phi}^{\delta}(X_{[1,n)})$
$\displaystyle=\left\langle\nabla\rho_{r,\phi}^{\delta}(X_{[1,n)}),X_{n}\right\rangle+\frac{1}{2}\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{[1,n)}),X_{n}^{\otimes
2}\right\rangle$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,n)}+tX_{n}),X_{n}^{\otimes
3}\right\rangle~{}dt.$

Because $X_{[1,n)}$ and $X_{n}$ are dependent via $X_{1}$ and $X_{n-1}$ due to
$1$-dependency, we re-apply the Taylor expansion to
$\nabla\rho_{r,\phi}^{\delta}(X_{[1,n)})$ and
$\nabla^{2}\rho_{r,\phi}^{\delta}(X_{[1,n)})$ centered at $X_{(1,n-1)}$:

$\displaystyle\left\langle\nabla\rho_{r,\phi}^{\delta}(X_{[1,n)}),X_{n}\right\rangle$
$\displaystyle=\left\langle\nabla\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}\right\rangle+\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}\otimes(X_{1}+X_{n-1})\right\rangle$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}\otimes(X_{1}+X_{n-1})^{\otimes
2}\right\rangle~{}dt,\enskip\text{and}\enskip$
$\displaystyle\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{[1,n)}),X_{n}^{\otimes
2}\right\rangle$
$\displaystyle=\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}^{\otimes
2}\right\rangle$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}^{\otimes
2}\otimes(X_{1}+X_{n-1})\right\rangle~{}dt.$

Similarly,

$\displaystyle\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}\otimes
X_{1}\right\rangle$
$\displaystyle=\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}),X_{n}\otimes
X_{1}\right\rangle$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}+tX_{2}),X_{n}\otimes
X_{1}\otimes X_{2}\right\rangle~{}dt,\enskip\text{and}\enskip$
$\displaystyle\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}\otimes
X_{n-1}\right\rangle$
$\displaystyle=\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}),X_{n}\otimes
X_{n-1}\right\rangle$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}+tX_{2}),X_{n}\otimes
X_{n-1}\otimes X_{n-2}\right\rangle~{}dt.$

In sum, because $\mathbb{E}[X_{j}]=0$,

$\displaystyle\mathbb{E}\left[\rho_{r,\phi}^{\delta}(X_{[1,n]})-\rho_{r,\phi}^{\delta}(X_{[1,n)})\right]$
$\displaystyle=\frac{1}{2}\left\langle\mathbb{E}[\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-1)})],\mathbb{E}[X_{n}^{\otimes
2}]\right\rangle+\left\langle\mathbb{E}[\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(2,n-1)})],\mathbb{E}[X_{n}\otimes
X_{1}]\right\rangle$
$\displaystyle\quad+\left\langle\mathbb{E}[\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-2)})],\mathbb{E}[X_{n-1}\otimes
X_{n}]\right\rangle+\mathbb{E}\left[\mathfrak{R}_{X}^{(3)}\right],$

where $\mathfrak{R}_{X}^{(3)}$ is the summation of all the above third-order
remainder terms. That is,

$\displaystyle\mathfrak{R}_{X}^{(3)}$
$\displaystyle=\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,n)}+tX_{n}),X_{n}^{\otimes
3}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}\otimes(X_{1}+X_{n-1})^{\otimes
2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}^{\otimes
2}\otimes(X_{1}+X_{n-1})\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}+tX_{2}),X_{n}\otimes
X_{1}\otimes X_{2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}+tX_{2}),X_{n}\otimes
X_{n-1}\otimes X_{n-2}\right\rangle~{}dt$

To further decompose $\mathfrak{R}_{X}^{(3)}$, we apply the Taylor expansion
up to order $4$. For example,

$\displaystyle\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,n)}+tX_{n}),X_{n}^{\otimes
3}\right\rangle~{}dt$
$\displaystyle=\frac{1}{6}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,n)}),X_{n}^{\otimes
3}\right\rangle+\frac{1}{6}\int_{0}^{1}(1-t)^{3}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{[1,n)}+tX_{n}),X_{n}^{\otimes
4}\right\rangle~{}dt.$

Again, to break the dependency between
$\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,n)})$ and $X_{n}^{\otimes 3}$, we re-
apply the Taylor expansion centered at $X_{1}+X_{n-1}$.

$\displaystyle\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,n)}),X_{n}^{\otimes
3}\right\rangle$
$\displaystyle=\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}^{\otimes
3}\right\rangle$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}^{\otimes
3}\otimes(X_{1}+X_{n-1})\right\rangle~{}dt.$

Repeating this to the other third-order remainder terms, we get

$\displaystyle\mathfrak{R}_{X}^{(3)}$
$\displaystyle=\frac{1}{6}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}),X_{n}^{\otimes
3}\right\rangle+\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-3)}),X_{n-2}\otimes
X_{n-1}\otimes X_{n}\right\rangle$
$\displaystyle\quad+\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-2)}),X_{n-1}\otimes
X_{n}\otimes
X_{1}\right\rangle+\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(3,n-1)}),X_{n}\otimes
X_{1}\otimes X_{2}\right\rangle$
$\displaystyle\quad+\frac{1}{2}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}),X_{n-1}\otimes
X_{n}\otimes(X_{n-1}+X_{n})\right\rangle$
$\displaystyle\quad+\frac{1}{2}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}),X_{n}\otimes
X_{1}\otimes(X_{1}+X_{n})\right\rangle$
$\displaystyle\quad+\mathfrak{R}_{X}^{(4)},$

where

$\displaystyle\mathfrak{R}_{X}^{(4)}$
$\displaystyle=\frac{1}{6}\int_{0}^{1}(1-t)^{3}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{[1,n)}+tX_{n}),X_{n}^{\otimes
4}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}\otimes(X_{1}+X_{n-1})^{\otimes
3}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}^{\otimes
2}\otimes(X_{1}+X_{n-1})^{\otimes 2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{6}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-1)}+t(X_{1}+X_{n-1})),X_{n}^{\otimes
3}\otimes(X_{1}+X_{n-1})\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}+tX_{2}),X_{n}\otimes
X_{1}\otimes X_{2}^{\otimes 2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}+tX_{n-2}),X_{n}\otimes
X_{n-1}\otimes X_{n-2}^{\otimes 2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}+tX_{2}),X_{n}\otimes
X_{1}^{\otimes 2}\otimes X_{2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}+tX_{n-2}),X_{n}\otimes
X_{n-1}^{\otimes 2}\otimes X_{n-2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(2,n-1)}+tX_{2}),X_{n}^{\otimes
2}\otimes X_{1}\otimes X_{2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-2)}+tX_{n-2}),X_{n}^{\otimes
2}\otimes X_{n-1}\otimes X_{n-2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(3,n-1)}+tX_{3}),X_{n}\otimes
X_{1}\otimes X_{2}\otimes X_{3}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(2,n-2)}+t(X_{2}+X_{n-2})),X_{n}\otimes
X_{n-1}\otimes X_{1}\otimes(X_{2}+X_{n-2})\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(X_{(1,n-3)}+tX_{n-3}),X_{n}\otimes
X_{n-1}\otimes X_{n-2}\otimes X_{n-3}\right\rangle~{}dt.$

The third-order moments in Eq. 36 are further decomposed as in Section C.4. As
a result,

$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-1)})\right],\mathbb{E}[X_{n}^{\otimes
3}]\right\rangle$
$\displaystyle=\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{(1,n-1)})\right],\mathbb{E}[X_{n}^{\otimes
3}]\right\rangle+\sum_{k=2}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6,1)}-\mathfrak{R}_{X,Y_{k}}^{(6,1)}\right],$
$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-3)})\right],\mathbb{E}[X_{n-2}\otimes
X_{n-1}\otimes X_{n}]\right\rangle$
$\displaystyle=\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{(1,n-3)})\right],\mathbb{E}[X_{n-2}\otimes
X_{n-1}\otimes
X_{n}]\right\rangle+\sum_{k=2}^{n-4}\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6,2)}-\mathfrak{R}_{X,Y_{k}}^{(6,2)}\right],$
$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-2)})\right],\mathbb{E}[X_{n-1}\otimes
X_{n}\otimes X_{1}]\right\rangle$
$\displaystyle=\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{(2,n-2)})\right],\mathbb{E}[X_{n-1}\otimes
X_{n}\otimes
X_{1}]\right\rangle+\sum_{k=3}^{n-3}\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6,3)}-\mathfrak{R}_{X,Y_{k}}^{(6,3)}\right],$
$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(3,n-1)})\right],\mathbb{E}[X_{n}\otimes
X_{1}\otimes X_{2}]\right\rangle$
$\displaystyle=\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{(3,n-1)})\right],\mathbb{E}[X_{n}\otimes
X_{1}\otimes
X_{2}]\right\rangle+\sum_{k=4}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6,4)}-\mathfrak{R}_{X,Y_{k}}^{(6,4)}\right],$
$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(1,n-2)})\right],\mathbb{E}[X_{n-1}\otimes
X_{n}\otimes(X_{n-1}+X_{n})]\right\rangle$
$\displaystyle=\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{(1,n-2)})\right],\mathbb{E}[X_{n-1}\otimes
X_{n}\otimes(X_{n-1}+X_{n})]\right\rangle+\sum_{k=2}^{n-3}\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6,5)}-\mathfrak{R}_{X,Y_{k}}^{(6,5)}\right],\enskip\text{and}\enskip$
$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(2,n-1)})\right],\mathbb{E}[X_{n}\otimes
X_{1}\otimes(X_{n-1}+X_{n})]\right\rangle$
$\displaystyle=\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{(2,n-1)})\right],\mathbb{E}[X_{n}\otimes
X_{1}\otimes(X_{1}+X_{n})]\right\rangle+\sum_{k=3}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6,6)}-\mathfrak{R}_{X,Y_{k}}^{(6,6)}\right].$

Because $Y$ is Gaussian, by Lemma 6.2 in Chernozhukov, Chetverikov and Koike
(2023) and Assumption (VAR-EV),

$\displaystyle\left\lvert\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(Y_{(1,n-1)})\right],\mathbb{E}[X_{n}^{\otimes
3}]\right\rangle\right\rvert$
$\displaystyle\leq\frac{C}{n^{3/2}}L_{3,n}\frac{(\log(ep))^{3/2}}{\underline{\sigma}^{3}}$
$\displaystyle\leq\frac{C}{n^{3/2}}L_{3,n}\frac{(\log(ep))^{2}}{\underline{\sigma}^{2}\sigma_{\min}}.$

Putting all the terms together,

$\displaystyle\left\lvert\mathfrak{R}_{X}^{(3)}-\mathfrak{R}_{Y}^{(3)}\right\rvert\leq\frac{C}{\sqrt{n}}\overline{L}_{3}\frac{(\log(ep))^{2}}{\underline{\sigma}^{2}\sigma_{\min}}+\left\lvert\mathfrak{R}_{X}^{(4)}-\mathfrak{R}_{Y}^{(4)}\right\rvert+\sum_{k=2}^{n-2}\left\lvert\mathbb{E}\left[\mathfrak{R}_{X,X_{k}}^{(6)}-\mathfrak{R}_{X,Y_{k}}^{(6)}\right]\right\rvert,$

where
$\mathfrak{R}_{X,W_{k}}^{(6)}=\frac{1}{6}\mathfrak{R}_{X,W_{k}}^{(6,1)}+\mathfrak{R}_{X,W_{k}}^{(6,2)}+\mathfrak{R}_{X,W_{k}}^{(6,3)}+\mathfrak{R}_{X,W_{k}}^{(6,4)}+\frac{1}{2}\mathfrak{R}_{X,W_{k}}^{(6,5)}+\frac{1}{2}\mathfrak{R}_{X,W_{k}}^{(6,6)}$.

###  C.2 First Lindeberg swapping under $1$-dependence

We apply Taylor’s expansion to each term with $\varphi_{r}^{\varepsilon}$ of
the Lindeberg swapping in Eq. 15. We only show the expansions for
$j=3,\dots,n-2$ here, but the calculations for $j=1,2,n-1$ and $n$ are
similar. We recall the notations

$W_{[i,j]}^{\mathsf{C}}\equiv X_{[1,i)}+Y_{(j,n]}\enskip\text{and}\enskip
W_{[i,j]}^{{\perp\\!\\!\\!\\!\perp}}\equiv W_{[i-1,j+1]}^{\mathsf{C}}.$

First,

$\displaystyle\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]}+X_{j})$
$\displaystyle=\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]})+\left\langle\nabla\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]}),X_{j}\right\rangle+\frac{1}{2}\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]}),X_{j}^{\otimes
2}\right\rangle$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]}+tX_{j}),X_{j}^{\otimes
3}\right\rangle~{}dt.$

We further apply Taylor’s expansion to the second and third terms:

$\displaystyle\left\langle\nabla\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]}),X_{j}\right\rangle$
$\displaystyle=\left\langle\nabla\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\right\rangle+\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes(X_{j-1}+Y_{j+1})\right\rangle$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-1}+Y_{j+1})\right),X_{j}\otimes(X_{j-1}+Y_{j+1})^{\otimes
2}\right\rangle~{}dt,$
$\displaystyle\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]}),X_{j}^{\otimes
2}\right\rangle$
$\displaystyle=\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}^{\otimes
2}\right\rangle$
$\displaystyle+\int_{0}^{1}(1-t)\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-1}+Y_{j+1})\right),X_{j}^{\otimes
2}\otimes(X_{j-1}+Y_{j+1})\right\rangle~{}dt.$

Last,

$\displaystyle\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes(X_{j-1}+Y_{j+1})\right\rangle$
$\displaystyle=\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes
X_{j-1}\right\rangle+\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes
Y_{j+1}\right\rangle$
$\displaystyle=\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes
X_{j-1}\right\rangle+\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j+1]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes
Y_{j+1}\right\rangle$
$\displaystyle+\int_{0}^{1}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}}+tX_{j-2}\right),X_{j}\otimes
X_{j-1}\otimes X_{j-2}\right\rangle~{}dt$
$\displaystyle+\int_{0}^{1}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j,j+1]}^{{\perp\\!\\!\\!\\!\perp}}+tY_{j+2}\right),X_{j}\otimes
Y_{j+1}\otimes Y_{j+2})\right\rangle~{}dt.$

In sum,

$\displaystyle\mathbb{E}\left[\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]}+X_{j})\right]$
$\displaystyle=\mathbb{E}\left[\begin{aligned}
&\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]})+\left\langle\nabla\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\right\rangle+\frac{1}{2}\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}^{\otimes
2}\right\rangle\\\
&+\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes
X_{j-1}\right\rangle+\left\langle\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j+1]}^{{\perp\\!\\!\\!\\!\perp}}),X_{j}\otimes
Y_{j+1}\right\rangle+\mathfrak{R}_{X_{j}}^{(3,1)}\end{aligned}\right]$
$\displaystyle=\mathbb{E}\left[\varphi_{r}^{\varepsilon}(W^{\mathsf{C}}_{[j,j]})\right]+\left\langle\mathbb{E}\left[\nabla\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}})\right],\mathbb{E}\left[X_{j}\right]\right\rangle+\frac{1}{2}\left\langle\mathbb{E}\left[\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}})\right],\mathbb{E}\left[X_{j}^{\otimes
2}\right]\right\rangle$
$\displaystyle\quad+\left\langle\mathbb{E}\left[\nabla^{2}\varphi_{r}^{\varepsilon}(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}})\right],\mathbb{E}\left[X_{j}\otimes
X_{j-1}\right]\right\rangle+\mathbb{E}[\mathfrak{R}_{X_{j}}^{(3,1)}],$

where

$\displaystyle\mathfrak{R}_{X_{j}}^{(3,1)}$
$\displaystyle=\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W^{\mathsf{C}}_{[j,j]}+tX_{j}\right),X_{j}^{\otimes
3}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-1}+Y_{j+1})\right),X_{j}\otimes(X_{j-1}+Y_{j+1})^{\otimes
2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-1}+Y_{j+1})\right),X_{j}^{\otimes
2}\otimes(X_{j-1}+Y_{j+1})\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}}+tX_{j-2}\right),X_{j}\otimes
X_{j-1}\otimes X_{j-2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W_{[j,j+1]}^{{\perp\\!\\!\\!\\!\perp}}+tY_{j+2}\right),X_{j}\otimes
Y_{j+1}\otimes Y_{j+2})\right\rangle~{}dt.$

$\mathfrak{R}_{Y_{j}}^{(3,1)}$ is similarly derived. For $j=1,2,n-1$ and $n$,
$\mathfrak{R}_{W_{j}}^{(3,1)}$ is the same as Eq. 40 but with zero in place of
non-existing terms. Summing over $j=1,\dots,n$,

$\displaystyle\sum_{j=1}^{n}\mathbb{E}\left[\rho_{r,\phi}^{\delta}(W^{\mathsf{C}}_{[j,j]}+X_{j})-\rho_{r,\phi}^{\delta}(W^{\mathsf{C}}_{[j,j]}+Y_{j})\right]=\sum_{j=1}^{n}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,1)}\right]$

To further decompose $\mathfrak{R}_{X_{j}}^{(3,1)}$, we apply the Taylor
expansion up to order $4$. For example, where $j=4,\dots,n-3$,

$\displaystyle\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{3}\varphi_{r}^{\varepsilon}\left(W^{\mathsf{C}}_{[j,j]}+tX_{j}\right),X_{j}^{\otimes
3}\right\rangle~{}dt$
$\displaystyle=\frac{1}{6}\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(W^{\mathsf{C}}_{[j,j]}),X_{j}^{\otimes
3}\right\rangle+\frac{1}{6}\int_{0}^{1}(1-t)^{3}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(W^{\mathsf{C}}_{[j,j]}+tX_{j}),X_{j}^{\otimes
4}\right\rangle~{}dt.$

Again, to break the dependency between
$\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,n)})$ and $X_{n}^{\otimes 3}$, we re-
apply the Taylor expansion centered at $X_{1}+X_{n-1}$.

$\displaystyle\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(W^{\mathsf{C}}_{[j,j]}),X_{j}^{\otimes
3}\right\rangle$
$\displaystyle=\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}(W^{\perp\\!\\!\\!\\!\perp}_{[j,j]}),X_{j}^{\otimes
3}\right\rangle$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(W^{\perp\\!\\!\\!\\!\perp}_{[j,j]}+t(X_{j-1}+Y_{j+1})),X_{n}^{\otimes
3}\otimes(X_{j-1}+Y_{j+1})\right\rangle~{}dt.$

Repeating this to the other third-order remainder terms, we get

$\displaystyle\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,1)}\right]$
$\displaystyle=\frac{1}{6}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,j-1)}+Y_{(j+1,n]})\right],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$
$\displaystyle\quad+\frac{1}{2}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,j-2)}+Y_{(j+1,n]})\right],\mathbb{E}[X_{j-1}^{\otimes
2}\otimes X_{j}]+\mathbb{E}[X_{j-1}\otimes X_{j}^{\otimes 2}]\right\rangle$
$\displaystyle\quad+\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,j-3)}+Y_{(j+1,n]})\right],\mathbb{E}\left[X_{j-2}\otimes
X_{j-1}\otimes X_{j}\right]\right\rangle$
$\displaystyle\quad+\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,1)}\right],$

where

$\displaystyle\mathfrak{R}_{X_{j}}^{(4,1)}$
$\displaystyle=\frac{1}{6}\int_{0}^{1}(1-t)^{3}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}(W^{\mathsf{C}}_{[j,j]}+tX_{j}),X_{j}^{\otimes
4}\right\rangle~{}dt$ (40)
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-1}+Y_{j+1})\right),X_{j}\otimes(X_{j-1}+Y_{j+1})^{\otimes
3}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-1}+Y_{j+1})\right),X_{j}^{\otimes
2}\otimes(X_{j-1}+Y_{j+1})^{\otimes 2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{6}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j,j]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-1}+Y_{j+1})\right),X_{j}^{\otimes
3}\otimes(X_{j-1}+Y_{j+1})\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}}+tX_{j-2}\right),X_{j}\otimes
X_{j-1}\otimes X_{j-2}^{\otimes 2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j,j+1]}^{{\perp\\!\\!\\!\\!\perp}}+tY_{j+2}\right),X_{j}\otimes
Y_{j+1}\otimes Y_{j+2}^{\otimes 2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}}+tX_{j-2}\right),X_{j}\otimes
X_{j-1}^{\otimes 2}\otimes X_{j-2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j,j+1]}^{{\perp\\!\\!\\!\\!\perp}}+tY_{j+2}\right),X_{j}\otimes
Y_{j+1}^{\otimes 2}\otimes Y_{j+2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j-1,j+1]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-2}+Y_{j+2})\right),X_{j-1}\otimes
X_{j}\otimes Y_{j+1}\otimes X_{j-2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j-1,j+1]}^{{\perp\\!\\!\\!\\!\perp}}+t(X_{j-2}+Y_{j+2})\right),X_{j-1}\otimes
X_{j}\otimes Y_{j+1}\otimes Y_{j+2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j-1,j]}^{{\perp\\!\\!\\!\\!\perp}}+tX_{j-2}\right),X_{j}^{\otimes
2}\otimes X_{j-1}\otimes X_{j-2}\right\rangle~{}dt$
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j,j+1]}^{{\perp\\!\\!\\!\\!\perp}}+tY_{j+2}\right),X_{j}^{\otimes
2}\otimes Y_{j+1}\otimes Y_{j+2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j-2,j]}^{{\perp\\!\\!\\!\\!\perp}}+tX_{j-3}\right),X_{j}\otimes
X_{j-1}\otimes X_{j-2}\otimes X_{j-3}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(W_{[j,j+2]}^{{\perp\\!\\!\\!\\!\perp}}+tY_{j+3}\right),X_{j}\otimes
Y_{j+1}\otimes Y_{j+2}\otimes Y_{j+3}\right\rangle~{}dt.$

$\mathfrak{R}_{Y_{j}}^{(4,1)}$ is similarly derived. For $j=1,2,3,n-2,n-1$ and
$n$, $\mathfrak{R}_{W_{j}}^{(4,1)}$ is the same as Eq. 40 but with zero in
place of proper terms.

###  C.3 Second Lindeberg swapping

For each $j=2,\dots,n-2$,

$\displaystyle\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}\left(X_{(1,j)}+X_{j}+Y_{(j,n-1)}\right),X_{n}^{\otimes
2}\right\rangle$
$\displaystyle=\left\langle\nabla^{2}\rho_{r,\phi}^{\delta}\left(X_{(1,j)}+Y_{(j,n-1)}\right),X_{n}^{\otimes
2}\right\rangle+\mathfrak{R}_{X_{j}}^{(3,2,1)}$

where
$\mathfrak{R}_{X_{j}}^{(3,2,1)}=\int_{0}^{1}\langle\nabla^{3}\rho_{r,\phi}^{\delta}\left(X_{(1,j)}+tX_{j}+Y_{(j,n-1)}\right),X_{n}^{\otimes
2}\otimes X_{j}\rangle~{}dt$. Furthermore,

$\displaystyle\mathfrak{R}_{X_{j}}^{(3,2,1)}$
$\displaystyle=\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}\left(X_{(1,j)}+Y_{(j,n-1)}\right),X_{n}^{\otimes
2}\otimes X_{j}\right\rangle$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(X_{(1,j)}+tX_{j}+Y_{(j,n-1)}\right),X_{n}^{\otimes
2}\otimes X_{j}^{\otimes 2}\right\rangle~{}dt$
$\displaystyle=\left\langle\nabla^{3}\rho_{r,\phi}^{\delta}\left(X_{(1,j-1)}+Y_{(j+1,n-1)}\right),X_{n}^{\otimes
2}\otimes X_{j}\right\rangle$
$\displaystyle\quad+\int_{0}^{1}(1-t)\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(X_{(1,j)}+tX_{j}+Y_{(j,n-1)}\right),X_{n}^{\otimes
2}\otimes X_{j}^{\otimes 2}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}\left\langle\nabla^{4}\rho_{r,\phi}^{\delta}\left(X_{(1,j-1)}+t(X_{j-1}+Y_{j+1})+Y_{(j+1,n-1)}\right),X_{n}^{\otimes
2}\otimes X_{j}\otimes(X_{j-1}+Y_{j+1})\right\rangle~{}dt.$

Because $\mathbb{E}[X_{n}^{\otimes 2}\otimes X_{j}]=\mathbb{E}[X_{n}^{\otimes
2}\otimes Y_{j}]=0$,

$\displaystyle\left\langle\mathbb{E}\left[\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-1)})\right],\mathbb{E}\left[X_{n}^{\otimes
2}\right]\right\rangle-\left\langle\mathbb{E}\left[\nabla^{2}\rho_{r,\phi}^{\delta}(Y_{(1,n-1)})\right],\mathbb{E}\left[X_{n}^{\otimes
2}\right]\right\rangle$
$\displaystyle=\sum_{j=2}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2,1)}-\mathfrak{R}_{Y_{j}}^{(3,2,1)}\right]=\sum_{j=2}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,2,1)}-\mathfrak{R}_{Y_{j}}^{(4,2,1)}\right],$

where $\mathfrak{R}_{X_{j}}^{(4,2,1)}$ is the fourth-order remainder term
above. Similarly,

$\displaystyle\left\langle\mathbb{E}\left[\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(2,n-1)})\right],\mathbb{E}\left[X_{n}\otimes
X_{1}\right]\right\rangle-\left\langle\mathbb{E}\left[\nabla^{2}\rho_{r,\phi}^{\delta}(Y_{(2,n-1)})\right],\mathbb{E}\left[X_{n}\otimes
X_{1}\right]\right\rangle$
$\displaystyle=\sum_{j=3}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2,2)}-\mathfrak{R}_{Y_{j}}^{(3,2,2)}\right]=\sum_{j=3}^{n-2}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,2,2)}-\mathfrak{R}_{Y_{j}}^{(4,2,2)}\right],\enskip\text{and}\enskip$
$\displaystyle\left\langle\mathbb{E}\left[\nabla^{2}\rho_{r,\phi}^{\delta}(X_{(1,n-2)})\right],\mathbb{E}\left[X_{n}\otimes
X_{n-1}\right]\right\rangle-\left\langle\mathbb{E}\left[\nabla^{2}\rho_{r,\phi}^{\delta}(Y_{(1,n-2)})\right],\mathbb{E}\left[X_{n}\otimes
X_{n-1}\right]\right\rangle$
$\displaystyle=\sum_{j=2}^{n-3}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(3,2,3)}-\mathfrak{R}_{Y_{j}}^{(3,2,3)}\right]=\sum_{j=2}^{n-3}\mathbb{E}\left[\mathfrak{R}_{X_{j}}^{(4,2,3)}-\mathfrak{R}_{Y_{j}}^{(4,2,3)}\right].$

We define
$\mathfrak{R}_{W_{j}}^{(3,2)}=\frac{1}{2}\mathfrak{R}_{W_{j}}^{(3,2,1)}+\mathfrak{R}_{W_{j}}^{(3,2,2)}+\mathfrak{R}_{W_{j}}^{(3,2,3)}$
and
$\mathfrak{R}_{W_{j}}^{(4,2)}=\frac{1}{2}\mathfrak{R}_{W_{j}}^{(4,2,1)}+\mathfrak{R}_{W_{j}}^{(4,2,2)}+\mathfrak{R}_{W_{j}}^{(4,2,3)}$.

###  C.4 Third Lindeberg swapping

For $j=3,\dots,n-1$,

$\displaystyle\Big{\langle}\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-1)}+Y_{(j+1,n)})\right]-\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(Y_{[1,j-1)}+Y_{(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\Big{\rangle}.$
$\displaystyle=\sum_{k=1}^{j-2}\Big{\langle}\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+X_{k}+Y_{(k,j-1)\cup(j+1,n)})\right]$
$\displaystyle{$- \mathbb{E}\left[ \nabla^3
\rho_{r,\phi}^\varepsilon(X_{[1,k)} + Y_k + Y_{(k,j-1) \cup(j+1,n)}) \right],
\mathbb{E}[ X_j^{\otimes 3} ] \Big{\rangle}.$}$

We first consider the case with $6\leq j\leq n-1$. For $3\leq k\leq j-4$, the
Taylor expansion centered at $X_{[1,k)}+Y_{(k,j-1)\cup(j+1,n)}$ implies

$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+X_{k}+Y_{(k,j-1)\cup(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$
$\displaystyle=\left\langle\mathbb{E}[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+Y_{(k,j-1)\cup(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$
$\displaystyle\quad+\mathbb{E}\left[\left\langle\nabla^{4}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+Y_{(k,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}\right\rangle\right]$
$\displaystyle\quad+\frac{1}{2}\mathbb{E}\left[\left\langle\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+Y_{(k,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}^{\otimes 2}\right\rangle\right]$
$\displaystyle\quad+\frac{1}{2}\mathbb{E}\left[\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+tX_{k}+Y_{(k,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}^{\otimes 3}\right\rangle~{}dt\right].$

For the inner product terms with dependent factors, we repeat the Taylor
expansion:

  1. 1.

$\displaystyle\mathbb{E}\left[\left\langle\nabla^{4}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+Y_{(k,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}\right\rangle\right]$
$\displaystyle=\mathbb{E}\left[\left\langle\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+Y_{(k+1,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}\otimes(X_{k-1}+Y_{k+1})\right\rangle\right]$
$\displaystyle\quad+\mathbb{E}\Big{[}\int_{0}^{1}(1-t)\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+t(X_{k-1}+Y_{k+1})+Y_{(k+1,j-1)\cup(j+1,n)}),$
$\displaystyle{$\mathbb{E}[X_j^{\otimes 3}] \otimes X_k \otimes(X_{k-1} +
Y_{k+1})^{\otimes 2} \Big{\rangle}dt \Big{]}$},$

  2. 2.

$\displaystyle\mathbb{E}\left[\left\langle\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+Y_{(k,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}^{\otimes 2}\right\rangle\right]$
$\displaystyle=\left\langle\mathbb{E}\left[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+Y_{(k+1,j-1)\cup(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}\left[X_{k}^{\otimes 2}\right]\right\rangle$
$\displaystyle\quad+\mathbb{E}\Big{[}\int_{0}^{1}\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+t(X_{k-1}+Y_{k+1})+Y_{(k+1,j-1)\cup(j+1,n)}),$
E[X_j^⊗3] ⊗X_k^⊗2 ⊗(X_k-1 + Y_k+1) ⟩ dt ],

  3. 3.

$\displaystyle\mathbb{E}\left[\left\langle\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+Y_{(k+1,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}\otimes(X_{k-1}+Y_{k-1})\right\rangle\right]$
$\displaystyle=\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-2)}+Y_{(k+1,j-1)\cup(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[X_{k}\otimes X_{k-1}]\right\rangle$
$\displaystyle\quad+\mathbb{E}\Big{[}\int_{0}^{1}\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-2)}+tX_{k-2}+Y_{(k+1,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}\otimes X_{k-1}\otimes X_{k-2}\Big{\rangle}~{}dt\Big{]}$
$\displaystyle\quad+\mathbb{E}\Big{[}\int_{0}^{1}\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+tY_{k+2}+Y_{(k+2,j-1)\cup(j+1,n)}),\mathbb{E}[X_{j}^{\otimes
3}]\otimes X_{k}\otimes Y_{k+1}\otimes Y_{k+2}\Big{\rangle}~{}dt\Big{]}.$

For
$\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+Y_{k}+Y_{(k,j-1)\cup(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$, the calculation is the same but with $Y_{k}$ in place of
$X_{k}$. By the second moment matching,

$\displaystyle\Big{\langle}\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+X_{k}+Y_{(k,j-1)\cup(j+1,n)})\right]$
$\displaystyle{$- \mathbb{E}\left[ \nabla^3
\rho_{r,\phi}^\varepsilon(X_{[1,k)} + Y_k + Y_{(k,j-1) \cup(j+1,n)}) \right],
\mathbb{E}[ X_j^{\otimes 3} ] \Big{\rangle}$}$
$\displaystyle=\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-2)}+Y_{(k+1,j-1)\cup(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[X_{k}\otimes X_{k-1}]\right\rangle$
$\displaystyle\quad-\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+Y_{(k+2,j-1)\cup(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[Y_{k}\otimes Y_{k+1}]\right\rangle$
$\displaystyle\quad+\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{k}}^{(6,1)}-\mathfrak{R}_{X_{j},Y_{k}}^{(6,1)}\right],$

where for $W_{k}=X_{k}$ or $Y_{k}$,

$\displaystyle\mathfrak{R}_{X_{j},W_{k}}^{(6,1)}$ (41)
$\displaystyle=\frac{1}{2}\int_{0}^{1}(1-t)^{2}\left\langle\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k)}+tW_{k}+Y_{(k,j-1)\cup(j+1,n)}),X_{j}^{\otimes
3}\otimes W_{k}^{\otimes 3}\right\rangle~{}dt$
$\displaystyle\quad+\int_{0}^{1}(1-t)\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+t(X_{k-1}+Y_{k+1})+Y_{(k+1,j-1)\cup(j+1,n)}),$
X_j^⊗3 ⊗W_k ⊗(X_k-1 + Y_k+1)^⊗2 ⟩dt
$\displaystyle\quad+\frac{1}{2}\int_{0}^{1}\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+t(X_{k-1}+Y_{k+1})+Y_{(k+1,j-1)\cup(j+1,n)}),$
X_j^⊗3 ⊗W_k^⊗2 ⊗(X_k-1 + Y_k+1) ⟩ dt
$\displaystyle\quad+\int_{0}^{1}\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-2)}+tX_{k-2}+Y_{(k+1,j-1)\cup(j+1,n)}),X_{j}^{\otimes
3}\otimes W_{k}\otimes X_{k-1}\otimes X_{k-2}\Big{\rangle}~{}dt$
$\displaystyle\quad+\int_{0}^{1}\Big{\langle}\nabla^{6}\rho_{r,\phi}^{\varepsilon}(X_{[1,k-1)}+tY_{k+2}+Y_{(k+2,j-1)\cup(j+1,n)}),X_{j}^{\otimes
3}\otimes W_{k}\otimes Y_{k+1}\otimes Y_{k+2}\Big{\rangle}~{}dt.$

For $k=1$,

$\displaystyle\Big{\langle}\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{1}+Y_{(1,j-1)\cup(j+1,n)})\right]-\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(Y_{1}+Y_{(1,j-1)\cup(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\Big{\rangle}$
$\displaystyle=-\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(Y_{(3,j-1)\cup(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[Y_{1}\otimes Y_{2}]\right\rangle$
$\displaystyle\quad+\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{1}}^{(6,1)}-\mathfrak{R}_{X_{j},Y_{1}}^{(6,1)}\right],$

where $\mathfrak{R}_{X_{j},W_{1}}^{(6,1)}$ is the same as Eq. 41 but with
$Y_{n}$ and $Y_{n-1}$ in place of $X_{k-1}$ and $X_{k-2}$, respectively. For
$k=2$,

$\displaystyle\Big{\langle}\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{1}+X_{2}+Y_{(2,j-1)\cup(j+1,n)})\right]-\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{1}+Y_{2}+Y_{(2,j-1)\cup(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\Big{\rangle}$
$\displaystyle=\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(Y_{(3,j-1)\cup(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[X_{2}\otimes X_{1}]\right\rangle$
$\displaystyle\quad-\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(Y_{(4,j-1)\cup(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[Y_{2}\otimes Y_{3}]\right\rangle$
$\displaystyle\quad+\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{2}}^{(6,1)}-\mathfrak{R}_{X_{j},Y_{2}}^{(6,1)}\right],$

where $\mathfrak{R}_{X_{j},W_{2}}^{(6,1)}$ is the same as Eq. 41 but with
$Y_{n}$ in place of $X_{k-2}$. For $k=j-3$,

$\displaystyle\Big{\langle}\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-3)}+X_{j-3}+Y_{\\{j-2\\}\cup(j+1,n)})\right]$
$\displaystyle{$- \mathbb{E}\left[ \nabla^3
\rho_{r,\phi}^\varepsilon(X_{[1,j-3)} + Y_{j-3} + Y_{\\{j-2\\} \cup(j+1,n)})
\right], \mathbb{E}[ X_j^{\otimes 3} ] \Big{\rangle}$}$
$\displaystyle=\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-5)}+Y_{(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[X_{j-3}\otimes X_{j-4}]\right\rangle$
$\displaystyle\quad-\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-4)}+Y_{(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[Y_{j-3}\otimes Y_{j-2}]\right\rangle$
$\displaystyle\quad+\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{j-3}}^{(6,1)}-\mathfrak{R}_{X_{j},Y_{j-3}}^{(6,1)}\right],$

where $\mathfrak{R}_{X_{j},W_{2}}^{(6,1)}$ is the same as Eq. 41 but with $0$
in place of $Y_{k+2}$. For $k=j-2$,

$\displaystyle\Big{\langle}\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-2)}+X_{j-2}+Y_{(j+1,n)})\right]-\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-2)}+Y_{j-2}+Y_{(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\Big{\rangle}$
$\displaystyle=\left\langle\mathbb{E}[\nabla^{5}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-4)}+Y_{(j+1,n)})],\mathbb{E}[X_{j}^{\otimes
3}]\otimes\mathbb{E}[X_{j-2}\otimes X_{j-3}]\right\rangle$
$\displaystyle\quad+\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{j-2}}^{(6,1)}-\mathfrak{R}_{X_{j},Y_{j-2}}^{(6,1)}\right],$

where $\mathfrak{R}_{X_{j},W_{2}}^{(6,1)}$ is the same as Eq. 41 but with $0$
in place of $Y_{k+1}$ and $Y_{k+2}$. By the second moment matching,

$\displaystyle\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\varepsilon}(X_{[1,j-1)}+Y_{(j+1,n)})-\nabla^{3}\rho_{r,\phi}^{\varepsilon}(Y_{[1,j-1)}+Y_{(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$
$\displaystyle=\sum_{k=1}^{j-2}\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{k}}^{(6,1)}-\mathfrak{R}_{X_{j},Y_{k}}^{(6,1)}\right].$

This is also the same for $j\in[1,5]$, where some terms in
$\mathfrak{R}_{X_{j},W_{k}}^{(6,1)}$ are zero when appropriate. In sum,

$\displaystyle\sum_{j=1}^{n}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{[1,j-1)}+Y_{(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle$
$\displaystyle=\sum_{j=1}^{n}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{[1,j-1)}+Y_{(j+1,n)})\right],\mathbb{E}[X_{j}^{\otimes
3}]\right\rangle+\sum_{j=3}^{n}\sum_{k=1}^{j-2}\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{k}}^{(6,1)}-\mathfrak{R}_{X_{j},Y_{k}}^{(6,1)}\right].$

Similarly,

$\displaystyle\sum_{j=1}^{n}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(0\vee(j+2-n),j-2)}+Y_{(j+2,n)})\right],\mathbb{E}[X_{j}^{\otimes
2}\otimes X_{j+1}]+\mathbb{E}[X_{j}\otimes X_{j+1}^{\otimes 2}]\right\rangle$
$\displaystyle=\sum_{j=1}^{n}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(0\vee(j+2-n),j-2)}+Y_{(j+2,n)})\right],\mathbb{E}[X_{j}^{\otimes
2}\otimes X_{j+1}]+\mathbb{E}[X_{j}\otimes X_{j+1}^{\otimes 2}]\right\rangle$
$\displaystyle\quad+\sum_{j=4}^{n}\sum_{k=1}^{j-3}\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{k}}^{(6,2)}-\mathfrak{R}_{X_{j},Y_{k}}^{(6,2)}\right],\enskip\text{and}\enskip$
$\displaystyle\sum_{j=1}^{n}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(X_{(0\vee(j+3-n),j-3)}+Y_{(j+3,n)})\right],\mathbb{E}\left[X_{j}\otimes
X_{j+1}\otimes X_{j+2}\right]\right\rangle$
$\displaystyle=\sum_{j=1}^{n}\left\langle\mathbb{E}\left[\nabla^{3}\rho_{r,\phi}^{\delta}(Y_{(0\vee(j+3-n),j-3)}+Y_{(j+3,n)})\right],\mathbb{E}\left[X_{j}\otimes
X_{j+1}\otimes X_{j+2}\right]\right\rangle$
$\displaystyle\quad+\sum_{j=5}^{n}\sum_{k=1}^{j-4}\mathbb{E}\left[\mathfrak{R}_{X_{j},X_{k}}^{(6,3)}-\mathfrak{R}_{X_{j},Y_{k}}^{(6,3)}\right],$

where $\mathfrak{R}_{X_{j},W_{k}}^{(6,2)}$ and
$\mathfrak{R}_{X_{j},W_{k}}^{(6,3)}$ are similarly derived as in Eq. 41.