that conditional on the intersection of events (A.33) and (A.34), we have

$\frac{\\#(S^{*}\cap\widehat{S})}{\\#S^{*}}\geq\frac{\\#(S^{*}\cap\mathcal{S})}{\\#S^{*}}\geq
1-(q+1)(1+\phi)c_{0}k_{1n}^{-1}.$ (A.37)

Since it holds that

$\displaystyle\begin{split}\mathbb{E}\Big{[}\frac{\\#(S^{*}\cap\widehat{S})}{\\#S^{*}}\Big{]}&\geq\Big{(}1-(q+1)(1+\phi)c_{0}k_{1n}^{-1}\Big{)}\\\
&\quad\times\mathbb{P}\Big{(}\frac{\\#(S^{*}\cap\widehat{S})}{\\#S^{*}}\geq
1-(q+1)(1+\phi)c_{0}k_{1n}^{-1}\Big{)}\\\
&\geq\Big{(}1-(q+1)(1+\phi)c_{0}k_{1n}^{-1}\Big{)}\mathbb{P}\Big{(}\text{event
in \eqref{power.new.23}}\cap\text{event in
\eqref{power.new.24}}\Big{)},\end{split}$ (A.38)

to establish (19) we need only to prove (A.35) and construct the probability
lower bounds for events in (A.33) and (A.34).

To show (A.35), note that by the definition of $T^{k}$’s and the assumption
that $T^{k}<\infty$, we have that conditional on event given in (A.33),

$\begin{split}1+\\#\\{j:W_{j}^{k}\leq-T^{k}\\}&\leq\tau_{1}(\\#\\{j:W_{j}^{k}\geq
T^{k}\\})\\\ &\leq\tau_{1}K(\\#S^{*})\end{split}$ (A.39)

for each $k\in\\{1,\cdots,q+1\\}$. In view of (A.39), it holds that for each
nonzero $e_{j}^{k}$,

$\begin{split}e_{j}^{k}&=\frac{p}{\\#\\{s:W_{s}^{k}\leq-T^{k}\\}+1}\\\
&\geq\frac{p}{\tau_{1}K(\\#S^{*})}\\\
&\geq\frac{p}{\tau^{*}\times(1-(1+q)(1+\phi)c_{0}k_{1n}^{-1})\times(\\#S^{*})},\end{split}$
(A.40)

where we have used the definitions of $e_{j}^{k}$’s, $\phi$, and $\tau_{1}$.
Then from (A.40) and the definition
$e_{j}=(q+1)^{-1}\sum_{k=1}^{q+1}e_{j}^{k}$, we can deduce that conditional on
event (A.33),

$\min_{j\in\mathcal{S}}e_{j}\geq\frac{p}{\tau^{*}\times(1-(1+q)(1+\phi)c_{0}k_{1n}^{-1})\times(\\#S^{*})},$

which establishes (A.35).

It remains to provide the probability upper bounds for the complementary
events of (A.33) and (A.34), which are given in (A.43) and (A.41),
respectively, below. By the result of the first assertion of Theorem 3 and the
assumptions (Conditions 6–7 and that Condition 5 is satisfied for the Lasso
estimates applied to each subsample in $H_{k}$ in Algorithm 1), it holds that

$\begin{split}\mathbb{P}(\textnormal{complementary event of
}\eqref{power.new.24})\leq(q+1)\times(k_{2n}+k_{3n})\end{split}$ (A.41)

for all large $n$.

On the other hand, it follows from the assumption of $\\#S^{*}>0$ that
conditional on event $\\{\\#\\{j:W_{j}^{k}\geq T^{k}\\}>K(\\#S^{*})\\}$,

$\begin{split}\frac{\\#(\\{j:W_{j}^{k}\geq
T^{k}\\}\cap(S^{*})^{c})}{\\#\\{j:W_{j}^{k}\geq T^{k}\\}\vee
1}&\geq\frac{\\#(\\{j:W_{j}^{k}\geq T^{k}\\}-\\#S^{*}}{\\#\\{j:W_{j}^{k}\geq
T^{k}\\}\vee 1}\\\ &>\frac{\\#(\\{j:W_{j}^{k}\geq
T^{k}\\}-\\#(\\{j:W_{j}^{k}\geq T^{k}\\}\times K^{-1}}{\\#\\{j:W_{j}^{k}\geq
T^{k}\\}\vee 1}\\\ &\geq\frac{K-1}{K}.\end{split}$ (A.42)

Therefore, from (20) and (A.42) some simple calculations give that

$\begin{split}&\mathbb{P}(\\{\\#\\{j:W_{j}^{k}\geq
T^{k}\\}>K(\\#S^{*})\\})\times\frac{K-1}{K}+\mathbb{P}(\\{\\#\\{j:W_{j}^{k}\geq
T^{k}\\}\leq K(\\#S^{*})\\})\times 0\\\
&\leq\mathbb{E}\left(\frac{\\#(\\{j:W_{j}^{k}\geq
T^{k}\\}\cap(S^{*})^{c})}{\\#\\{j:W_{j}^{k}\geq T^{k}\\}\vee 1}\right)\\\
&\leq\tau_{1}+\theta_{\varepsilon},\end{split}$

which yields that

$\begin{split}\mathbb{P}(\textnormal{complementary event of
}\eqref{power.new.23})\leq(q+1)\times\frac{(\tau_{1}+\theta_{\varepsilon})K}{K-1}.\end{split}$
(A.43)

This establishes the desired conclusion in (19) of Theorem 3.

Proof of (18). Finally, we show the second assertion (18) of Theorem 3. Let us
observe that by the construction of Algorithm 1 and the definition of
$\mathcal{S}$ in (A.36), it holds that

$\mathbb{P}(\\{\widehat{S}=\emptyset\\}\cup\\{\mathcal{S}\subset\widehat{S}\\})=1.$

Then by (A.34)–(A.37), (A.41), and the fact that $\phi\leq 3$ (recall that
$\phi$ is defined at the beginning of this proof), we can obtain the desired
result in (18). It is worth mentioning that we do not require
$\tau_{1}\leq\tau^{*}$ here. This completes the proof of Theorem 3.

###  A.4 Proof of Corollary 1

The conclusion of Corollary 1 follows from the proof of (15) in Theorem 2
given in Section A.2.

###  A.5 Proof of Corollary 2

Under Condition 2 and the assumptions that $\\{\boldsymbol{x}_{i}\\}_{i\geq
1}$ satisfies Condition 4 with $h$-step and constants $C_{0}>0$ and
$0\leq\rho<1$ and $Y_{i}$ is $\boldsymbol{x}_{i+1}$-measurable, we will prove
in Section B.1 that for each $1\leq k\leq q+1$,

$\sup_{\mathcal{D}\in\mathcal{R}^{\\#H_{k}\times(2p+1)}}|\mathbb{P}(\mathcal{X}_{k}\in\mathcal{D})-\mathbb{P}(\mathcal{X}_{k}^{\pi}\in\mathcal{D})|\\\
\leq\\#H_{k}\times\rho^{q}\times C_{0},$ (A.44)

where we recall that
$\mathcal{X}_{k}=\\{\boldsymbol{x}_{i},\widetilde{\boldsymbol{x}}_{i},Y_{i}\\}_{i\in
H_{k}}$ and
$\mathcal{X}_{k}^{\pi}=\\{\boldsymbol{x}_{i}^{\pi},\widetilde{\boldsymbol{x}}_{i}^{\pi},Y_{i}^{\pi}\\}_{i\in
H_{k}}$ for each $k\in\\{1,\cdots,q+1\\}$. Combining (A.44) and
$\sum_{k=1}^{q+1}\\#H_{k}\leq n$ leads to the desired result in (6).

Next, we deal with the second assertion of Corollary 2. When Condition 2 holds
and $\\{Y_{i},\boldsymbol{x}_{i}\\}_{i=1}^{n}$ is also an i.i.d. sample,
$\\{Y_{i},\boldsymbol{x}_{i},\widetilde{\boldsymbol{x}}_{i}\\}_{i=1}^{n}$ is
an i.i.d. sample. Therefore, it follows from the fact that
$(Y_{i}^{\pi},\boldsymbol{x}_{i}^{\pi},\widetilde{\boldsymbol{x}}_{i}^{\pi})$’s
are i.i.d. with
$(Y_{i}^{\pi},\boldsymbol{x}_{i}^{\pi},\widetilde{\boldsymbol{x}}_{i}^{\pi})$
having the same distribution as
$(Y_{1},\boldsymbol{x}_{1},\widetilde{\boldsymbol{x}}_{1})$ that (A.44) holds
with $\rho=0$, which concludes the proof of Corollary 2.

###  A.6 Proof of Proposition 1

For the reader’s convenience, we provide some basic knowledge about time-
homogeneous Markov chains here. Two sufficient conditions for a process
$\\{\boldsymbol{Q}_{t}\\}$ to admit a transition kernel are 1) for each Borel
set $A$ and each $t$,

$\mathbb{P}(\boldsymbol{Q}_{t}\in
A|\boldsymbol{Q}_{t-1})=\mathbb{P}(\boldsymbol{Q}_{t}\in
A|\boldsymbol{Q}_{t-j},j<1),$

the so-called the “Markov property,” and 2) the conditional distribution of
$\boldsymbol{Q}_{t}$ given $\boldsymbol{Q}_{t-1}$ are the same for each $t$.
Processes satisfying these two conditions are known as time-homogeneous Markov
chains. A useful sufficient condition for verifying that a process is a time-
homogeneous Markov chain is to check whether the process can be written as
$\boldsymbol{Q}_{t}=F(\boldsymbol{Q}_{t-1},\boldsymbol{\varepsilon}_{t})$ for
some measurable $F(\cdot,\cdot)$ and identically distributed innovative random
vectors $\\{\boldsymbol{\varepsilon}_{t}\\}$ such that
$\boldsymbol{\varepsilon}_{t}$ is independent of $\boldsymbol{Q}_{t-j}$ with
$j\geq 1$. It can be shown that $\\{\boldsymbol{x}_{t}\\}$ in Example 3 is a
time-homogeneous Markov chain, and we omit the details on proving such claim
for simplicity.

Next, let us consider Example 6 below, which is more general than Example 3.
In particular, $\\{\boldsymbol{x}_{t}\\}$ in Example 3 is a special case of
$\\{\boldsymbol{z}_{t}\\}$ in Example 6.

######  Example 6 (Gaussian linear processes).

Let $\\{\boldsymbol{z}_{t}\coloneqq(Y_{t1},\cdots,Y_{tp})^{{}^{T}}\\}$ be such
that for $l=1,\cdots,p$,
$Y_{tl}=\sum_{i=0}^{\infty}(\vec{w}_{i}(l))^{{}^{T}}\boldsymbol{\delta}_{t-i}$,
where $\vec{w}_{i}(l)$ is an $\iota$-dimensional coefficient vector such that
for each $h\geq 0$,

$\max_{1\leq l\leq p}\sum_{i\geq
h}\left\lVert\vec{w}_{i}(l)\right\rVert_{1}\leq C_{1}e^{-s_{1}h}$ (A.45)

with some positive $C_{1}$ and $s_{1}$, and $\boldsymbol{\delta}_{t}$’s are
i.i.d. $\iota$-dimensional Gaussian random vectors with zero mean and
covariance matrix $\Sigma$. In addition, assume that
$\lambda_{\max}(\Sigma)<L_{3}$ and
$\lambda_{\min}(\mathbb{E}(\boldsymbol{z}_{1}\boldsymbol{z}_{1}^{{}^{T}}))>l_{1}$
for some positive $L_{3}$ and $l_{1}$, where $\lambda_{\max}(\cdot)$ and
$\lambda_{\min}(\cdot)$ denote the largest and smallest eigenvalues of a given
matrix, respectively.

We use Propositions 3.1.1–3.1.2 of Brockwell and Davis [10] to obtain the
stationarity of Example 6; the details on this are omitted. Thus,
$\\{\boldsymbol{x}_{t}\\}$ in Example 3 is a stationary time-homogeneous
Markov chain; equivalently, the stationary distribution and transition kernel
of $\\{\boldsymbol{x}_{t}\\}$ exist.

######  Remark 2.

Notice that time-homogeneous Markov chains are not always stationary;
particularly, a random walk process can be a time-homogeneous Markov chain.
Also, note that Example 6 may not be a time-homogeneous Markov chain. With
regularity conditions assumed, for a stationary
$Q_{t}=\sum_{j=0}^{\infty}\beta_{j}\varepsilon_{t-j}$ to be a time-homogeneous
Markov chain, it is usually required that $Q_{t}$ can be written as
$Q_{t}=\sum_{j=1}^{k}\gamma_{j}Q_{t-j}+\varepsilon_{t}$ for some positive
integer $k$. Without further assumptions, the linear processes in Example 6
may not admit such representation.

Let $\\{\boldsymbol{z}_{t}^{(h)}\\}$ in Example 6 with dimensionality $p_{h}$
and stationary distribution $\pi_{h}(\cdot)$ be given. Note that we do not
assume a transition kernel for Example 6 and that all parameters (except for
constant $C_{1}$, $s_{1}$, $l_{1}$, and $L_{3}$) in Example 6 may change for
each $h$, but we drop the superscript or subscript $h$ for simplicity of
presentation whenever there is no confusion. Since Example 3 admits a
transition kernel and it is a special case of Example 6, to prove Proposition
1 it suffices to show that for all large $h$, there exist some constants
$0\leq\rho<1$, $0<C_{0}<\infty$, and measurable functions
$V_{h}:\mathbb{R}^{p_{h}}\longrightarrow[0,\infty)$ such that for each integer
$t$,

$\begin{split}&\sup_{\mathcal{D}\in\mathcal{R}^{p_{h}}}\left|\mathbb{P}(\boldsymbol{z}_{t+h}^{(h)}\in\mathcal{D})-\mathbb{P}(\boldsymbol{z}_{t+h}^{(h)}\in\mathcal{D}\
|\ \boldsymbol{z}_{t}^{(h)})\right|\leq
V_{h}(\boldsymbol{z}_{t}^{(h)})\rho^{h}C_{3}\end{split}$ (A.46)

almost surely for some constant $C_{3}>0$, and

$C_{0}\geq\sup_{h>0}\int_{\mathbb{R}^{p_{h}}}V_{h}(x)\pi_{h}(dx).$

To ease the technical presentation, we first introduce some necessary
notation. For each $h$, denote by

$\begin{split}U_{1t}^{(h)}&\coloneqq\left(\sum_{i=h}^{\infty}(\vec{w}_{i}(1))^{{}^{T}}\boldsymbol{\delta}_{t-i},\cdots,\sum_{i=h}^{\infty}(\vec{w}_{i}(p_{h}))^{{}^{T}}\boldsymbol{\delta}_{t-i}\right)^{{}^{T}},\\\
U_{2t}^{(h)}&\coloneqq\left(\sum_{i=0}^{h-1}(\vec{w}_{i}(1))^{{}^{T}}\boldsymbol{\delta}_{t-i},\cdots,\sum_{i=0}^{h-1}(\vec{w}_{i}(p_{h}))^{{}^{T}}\boldsymbol{\delta}_{t-i}\right)^{{}^{T}},\end{split}$
(A.47)

and let $V_{1t}^{(h)}$ and $V_{2t}^{(h)}$ be independent copies of
$U_{1t}^{(h)}$ and $U_{2t}^{(h)}$, respectively, where the superscript or
subscript $h$ represents the truncation length. Observe that
$U_{1t}^{(h)}+U_{2t}^{(h)}$ is an instance of $\boldsymbol{z}_{t}$ in Example
6. Due to the Gaussian innovations, the stationary distribution $\pi_{h}$ is
the distribution of $V_{11}^{(0)}$, which is the same as that of
$V_{1t}^{(h)}+V_{2t}^{(h)}$ for each $t$ and $h$.

Let us repeat the needed statement (A.46) with the newly defined notation. For
all large $h$, there exist some constants $0\leq\rho<1$, $0<C_{0}<\infty$, and
measurable functions $V_{h}:\mathbb{R}^{p_{h}}\longrightarrow[0,\infty)$ such
that for each integer $t$,

$\begin{split}&\sup_{\mathcal{D}\in\mathcal{R}^{p_{h}}}\left|\mathbb{P}(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D})-\mathbb{P}(U_{1(t+h)}^{(h)}+U_{2(t+h)}^{(h)}\in\mathcal{D}\
|\ U_{1t}^{(h)}+U_{2t}^{(h)})\right|\\\ &\quad\leq
V_{h}(U_{1t}^{(h)}+U_{2t}^{(h)})\rho^{h}C_{3}\end{split}$ (A.48)

almost surely for some constant $C_{3}>0$, and

$C_{0}\geq\sup_{h>0}\int_{\mathbb{R}^{p_{h}}}V_{h}(x)\pi_{h}(dx).$ (A.49)

If (A.48) holds for some $t$, it holds for each integer $t$ because the
process is stationary. Notice that the technical analysis here does not depend
on the Markov property. For the remaining proof of Proposition 1, we tend to
omit the term almost surely when the equality or inequality holds clearly
almost surely.

Let us begin with establishing (A.48). In view of assumption (8), let
$s_{3}>0$ and $0<\delta_{0}<1$ be given such that $0<s_{3}<s_{1}$ and
$s_{2}<\delta_{0}s_{3}$. For each positive integer $h$, we have

$p_{h}\exp{(-\delta_{0}s_{3}h)}\leq C_{2}\exp{((s_{2}-\delta_{0}s_{3})h)}.$
(A.50)

We claim that for all large $h$ and each $t$, it holds that for each
$\mathcal{D}\in\mathcal{R}^{p_{h}}$,

$\begin{split}&\left|\mathbb{P}\left(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\right)-\mathbb{P}\left(U_{1(t+h)}^{(h)}+U_{2(t+h)}^{(h)}\in\mathcal{D}\
\Big{|}\ U_{1t}^{(h)}+U_{2t}^{(h)}\right)\right|\\\
&\leq\mathbb{P}\left(\left\lVert V_{11}^{(h)}\right\rVert_{\infty}\geq
e^{-s_{3}h}\right)+\mathbb{P}(\left\lVert
U_{1(t+h)}^{(h)}\right\rVert_{\infty}\geq e^{(-s_{3}h)}\ |\
U_{1t}^{(h)}+U_{2t}^{(h)})\\\ &\quad+2\mathbb{P}\left(\left\lVert
V_{21}^{(h)}\right\rVert_{\infty}\geq
e^{(1-\delta_{0})s_{3}h}-2e^{-s_{3}h}\right)+\frac{8p_{h}}{\underline{c}}e^{-\delta_{0}s_{3}h},\end{split}$
(A.51)

where $\underline{c}>0$ is some constant and
$\left\lVert\vec{z}\right\rVert_{\infty}\coloneqq\max_{1\leq i\leq k}|z_{i}|$
for $\vec{z}=(z_{1},\cdots,z_{k})^{{}^{T}}\in\mathbb{R}^{k}$. The proof of
claim (LABEL:glp7) above is presented in Section B.2.

We next construct some upper bounds for the first and third terms on the RHS
of (LABEL:glp7). It follows from
$\left\lVert\cdot\right\rVert_{1}^{2}\geq\left\lVert\cdot\right\rVert_{2}^{2}$
and (A.45) that for each $q$ and $t$,

$\begin{split}\textnormal{Var}(\sum_{i\geq
h}\vec{w}_{i}^{{}^{T}}(q)\boldsymbol{\delta}_{t-i})&\leq\sum_{i\geq
h}\left\lVert\vec{w}_{i}(q)\right\rVert_{2}^{2}\lambda_{\max}(\Sigma_{h})\\\
&\leq\left(\sum_{i\geq
h}\left\lVert\vec{w}_{i}(q)\right\rVert_{1}\right)^{2}\lambda_{\max}(\Sigma_{h})\\\
&\leq\left(C_{1}\exp{(-s_{1}h)}\right)^{2}\lambda_{\max}(\Sigma_{h}),\end{split}$

where $\Sigma_{h}$ denotes the covariance matrix of the underlying Gaussian
random vectors $\boldsymbol{\delta}_{t}$’s (the superscript $h$ is dropped)
associated with $\\{\boldsymbol{z}_{t}^{(h)}\\}$ in Example 6. Combining this,
the fact that $\boldsymbol{\delta}_{t}$’s are Gaussian random vectors, and
Markov’s inequality, it holds that for each $h\geq 0$,

$\begin{split}\mathbb{P}&\left(\left\lVert
V_{11}^{(h)}\right\rVert_{\infty}\geq\exp{(-s_{3}h)}\right)\\\ &\quad\leq
p_{h}\mathbb{P}\left(C_{1}\exp{(-s_{1}h)}\sqrt{\lambda_{\max}(\Sigma_{h})}|Z|\geq\exp{(-s_{3}h)}\right)\\\
&\quad\leq
p_{h}\mathbb{E}(e^{|Z|})\exp{\left[-\left(C_{1}\sqrt{\lambda_{\max}(\Sigma_{h})}\right)^{-1}\exp{\left((s_{1}-s_{3})h\right)}\right]},\end{split}$
(A.52)

where $Z$ denotes a Gaussian random variable with zero mean and unit variance.
Similarly, we can show that for all large $h$, it holds that

$\begin{split}&\mathbb{P}\left(\left\lVert
V_{21}^{(h)}\right\rVert_{\infty}\geq\exp{((1-\delta_{0})s_{3}h)}-2\exp{(-s_{3}h)}\right)\\\
&\quad\leq
p_{h}\mathbb{P}\left(C_{1}\sqrt{\lambda_{\max}(\Sigma_{h})}|Z|\geq\exp{((1-\delta_{0})s_{3}h)}-1\right)\\\
&\quad\leq
p_{h}\mathbb{E}(e^{|Z|})\exp{\left[-\left(C_{1}\sqrt{\lambda_{\max}(\Sigma_{h})}\right)^{-1}(\exp{((1-\delta_{0})s_{3}h)}-1)\right]}.\end{split}$
(A.53)

We are now ready to construct the $V_{h}$ function. Let $g$ be a measurable
function such that $g(U_{1t}^{(h)}+U_{2t}^{(h)})$ is a version of
$\mathbb{P}(\left\lVert U_{1(t+h)}^{(h)}\right\rVert_{\infty}\geq
e^{(-s_{3}h)}\ |\ U_{1t}^{(h)}+U_{2t}^{(h)})$. It follows from the assumption
that $\lambda_{\max}(\Sigma_{h})$ is bounded by a constant,
$\mathbb{E}(e^{|Z|})<\infty$, (A.50), (A.52), and (LABEL:glp5) that there
exist some constants $C_{3}>0$ and $0\leq\rho<1$ such that for each positive
$h$, $C_{3}\rho^{h}$ is larger than the summation of the first, third, and
fourth terms on the RHS of (LABEL:glp7). For each $h$, let us define function
$V_{h}$ as

$V_{h}(x)\coloneqq\begin{cases}2\quad\textnormal{ if }g(x)\leq
C_{3}\rho^{h},\\\ 2\rho^{-h}C_{3}^{-1}\quad\textnormal{
otherwise}.\end{cases}$

Then combining (LABEL:glp7) and the definitions of $\rho$, $C_{3}$, and
$V_{h}$ leads to (A.48).

Finally, we deal with (A.49). By Markov’s inequality, the definition of $g$,
and $\mathbb{P}(\left\lVert
V_{11}^{(h)}\right\rVert_{\infty}\geq\exp{(-s_{3}h)})$
$=\mathbb{P}(\left\lVert
U_{1(t+h)}^{(h)}\right\rVert_{\infty}\geq\exp{(-s_{3}h)})$, we can deduce that

$\begin{split}\int
V_{h}(x)d\pi_{h}(x)&=\mathbb{E}(V_{h}(U_{1t}^{(h)}+U_{2t}^{(h)}))\\\ &\leq
2+2(C_{3}\rho^{h})^{-1}\mathbb{P}(g(U_{1t}^{(h)}+U_{2t}^{(h)})>C_{3}\rho^{h})\\\
&\leq 2+2(C_{3}\rho^{h})^{-2}\mathbb{E}(g(U_{1t}^{(h)}+U_{2t}^{(h)}))\\\
&=2+2(C_{3}\rho^{h})^{-2}\mathbb{P}\left(\left\lVert
V_{11}^{(h)}\right\rVert_{\infty}\geq\exp{(-s_{3}h)}\right).\end{split}$
(A.54)

For the first equality, recall that $U_{1t}^{(h)}+U_{2t}^{(h)}$ has the
stationary distribution. Therefore, by (A.54), (A.52), and (A.50), for all
large $h$, it holds that $\int V_{h}(x)d\pi_{h}(x)$ is bounded by a constant,
which leads to (A.49). This completes the proof of Proposition 1.

##  Appendix B Some key lemmas and additional technical details

We will provide in this section the additional technical details and some key
lemmas. In particular, we rely on measure theory for valid arguments for the
manipulation of integration when the conditional distributions are involved.

###  B.1 Proof of Claim (A.44)

Let us first make a simple observation. For any $q_{1}$-dimensional random
vectors $X_{1},Y_{1}$ and $q_{2}$-dimensional random vectors $X_{2},Y_{2}$
such that $X_{2}=F(X_{1})$ and $Y_{2}=F(Y_{1})$ for some measurable
$F:\mathbb{R}^{q_{1}}\longmapsto\mathbb{R}^{q_{2}}$, it holds that

$\begin{split}\sup_{\mathcal{D}\in\mathcal{R}^{q_{2}}}|\mu_{X_{2}}(\mathcal{D})-\mu_{Y_{2}}(\mathcal{D})|&=\sup_{\mathcal{D}\in\mathcal{R}^{q_{2}}}|\mu_{X_{1}}(F^{-1}(\mathcal{D}))-\mu_{Y_{1}}(F^{-1}(\mathcal{D}))|\\\
&\leq\sup_{\mathcal{A}\in\mathcal{R}^{q_{1}}}|\mu_{X_{1}}(\mathcal{A})-\mu_{Y_{1}}(\mathcal{A})|,\end{split}$
(A.55)

where $F^{-1}(\cdot)$ denotes the inverse mapping of $F(\cdot)$. With the aid
of (A.55), we now deal with the case of $k=1$ below. Let $\boldsymbol{M}$ be
given as in (LABEL:M0) with $l=2$, $h=q-1$, and
$\boldsymbol{z}_{i}=\boldsymbol{x}_{i}$. Similarly, let $\boldsymbol{M}^{\pi}$
be given as in (LABEL:M0) with $l=2$, $h=q-1$, and i.i.d. random vectors
$(\boldsymbol{z}_{1}^{\pi_{i}},\boldsymbol{z}_{2}^{\pi_{i}})$’s such that
$(\boldsymbol{z}_{1}^{\pi_{1}},\boldsymbol{z}_{2}^{\pi_{1}})$ and
$(\boldsymbol{x}_{1},\boldsymbol{x}_{2})$ have the same distribution. Then it
follows from Lemma 1 in Section B.3 that

$\begin{split}&\sup_{\mathcal{D}\in\mathcal{R}^{\\#H_{1}\times(2p)}}|\mathbb{P}((\boldsymbol{x}_{i+1},\boldsymbol{x}_{i},i\in
H_{1})\in\mathcal{D})-\mathbb{P}((\boldsymbol{z}_{2}^{\pi_{i}},\boldsymbol{z}_{1}^{\pi_{i}},i\in
H_{1})\in\mathcal{D})|\\\ &\quad\leq\\#H_{1}\times\rho^{q}\times
C_{0}.\end{split}$ (A.56)

By the assumption that $Y_{i}$ is $\boldsymbol{x}_{i+1}$-measurable, we have
that $(Y_{i},\boldsymbol{x}_{i})=F(\boldsymbol{x}_{i+1},\boldsymbol{x}_{i})$
for some measurable $F:\mathbb{R}^{2p}\longmapsto\mathbb{R}^{1+p}$. Then it
follows from the assumption that each $(Y_{i}^{\pi},\boldsymbol{x}_{i}^{\pi})$
and $(Y_{1},\boldsymbol{x}_{1})$ have the same distribution and the assumption
that each $(\boldsymbol{z}_{1}^{\pi_{i}},\boldsymbol{z}_{2}^{\pi_{i}})$ and
$(\boldsymbol{x}_{1},\boldsymbol{x}_{2})$ have the same distribution that

$\\{F(\boldsymbol{z}_{2}^{\pi_{i}},\boldsymbol{z}_{1}^{\pi_{i}})\\}_{i=1}^{n}\
\textnormal{ and }\ \\{(Y_{i}^{\pi},\boldsymbol{x}_{i}^{\pi})\\}_{i=1}^{n}$
(A.57)

have the same distribution. Hence, from (A.55)–(A.57), we can deduce that

$\begin{split}&\sup_{\mathcal{D}\in\mathcal{R}^{\\#H_{1}\times(1+p)}}|\mathbb{P}((Y_{i},\boldsymbol{x}_{i},i\in
H_{1})\in\mathcal{D})-\mathbb{P}((Y_{i}^{\pi},\boldsymbol{x}_{i}^{\pi},i\in
H_{1})\in\mathcal{D})|\\\
&=\sup_{\mathcal{D}\in\mathcal{R}^{\\#H_{1}\times(1+p)}}|\mathbb{P}((\boldsymbol{x}_{i+1},\boldsymbol{x}_{i},i\in
H_{1})\in
F^{-1}(\mathcal{D}))-\mathbb{P}((\boldsymbol{z}_{2}^{\pi_{i}},\boldsymbol{z}_{1}^{\pi_{i}},i\in
H_{1})\in F^{-1}(\mathcal{D}))|\\\ &\leq\\#H_{1}\times\rho^{q}\times
C_{0}.\end{split}$ (A.58)

Finally, we can apply Lemma 7 in Section B.9 to control the distributional
variation results from the inclusion of knockoffs. Using Definition 2 and
Conditions 1–2, we can show that 1)
$(Y_{i},\boldsymbol{x}_{i},\widetilde{\boldsymbol{x}}_{i})$’s are identically
distributed; 2) $(\widetilde{\boldsymbol{x}}_{i},i\in H_{1})$ are independent
conditional on $(\boldsymbol{x}_{i},i\in H_{1})$, and that
$\widetilde{\boldsymbol{x}}_{i}$ is independent of $(\boldsymbol{x}_{q},q\in
H_{1}\backslash\\{i\\})$ conditional on $\boldsymbol{x}_{i}$ for each $i\in
H_{1}$; 3) $(Y_{i},i\in H_{1})$ is independent of
$(\widetilde{\boldsymbol{x}}_{i},i\in H_{1})$ conditional on
$(\boldsymbol{x}_{i},i\in H_{1})$; and 4) the above results also hold for
$(Y_{i}^{\pi},\boldsymbol{x}_{i}^{\pi},\widetilde{\boldsymbol{x}}_{i}^{\pi})$’s.
By these results, an application of the first assertion of Lemma 7 concludes
the proof of (A.44) for the case with $k=1$. The other cases with $2\leq k\leq
q+1$ can be dealt with similarly. This concludes the proof of Claim (A.44).

###  B.2 Proof of Claim (LABEL:glp7)

We denote by $f_{h}(x)$ the density function of $V_{21}^{(h)}$, that is,

$f_{h}(x)\coloneqq\frac{1}{\sqrt{(2\pi)^{p_{h}}|\Sigma_{h}^{V}|}}\exp{\left(-\frac{1}{2}x^{{}^{T}}(\Sigma_{h}^{V})^{-1}x\right)},$

where $\Sigma_{h}^{V}$ is the corresponding covariance matrix and $|A|$ stands
for the determinant of a given matrix $A$. By the assumptions of Gaussian
linear processes (this is where we need
$\lambda_{\min}(\mathbb{E}(\boldsymbol{z}_{1}^{(h)}(\boldsymbol{z}_{1}^{(h)})^{{}^{T}}))>l_{1}$),
there exist some constant $\underline{c}>0$ and positive integer $\bar{h}$
such that

$\min_{k\geq\bar{h}}\lambda_{\min}(\mathbb{E}(V_{21}^{(k)}V_{21}^{(k)^{T}}))>\underline{c}>0.$
(A.59)

In view of (A.59), for all $h\geq\bar{h}$ we have
$\lambda_{\min}(\Sigma_{h}^{V})>\underline{c}$.

To facilitate the technical analysis, we will make use of the following facts.

  1. 1)

For all $x<1.79$, it holds that

$\exp{(x)}\leq 1+x+x^{2}.$ (A.60)

To get the specific value of $1.79$, we use the first and second order
derivatives of $\exp{(x)}$ and $1+x+x^{2}$ to conclude that there exists some
positive number $x_{0}$ such that when $x\leq x_{0}$, (A.60) holds, and when
$x>x_{0}$, it holds that $\exp{(x)}>1+x+x^{2}$. Then a direct calculation
shows that $\exp{(1.79)}<5.994<1+1.79+1.79^{2}$, which gives $x_{0}\geq 1.79$.

  2. 2)

By (A.59), for all large $h$, we have that for each
$\Delta,x\in\mathbb{R}^{p_{h}}$ with
$\left\lVert\Delta\right\rVert_{2}\leq\left\lVert x\right\rVert_{2}$,

$|x^{{}^{T}}(\Sigma_{h}^{V})^{-1}\Delta+\frac{1}{2}\Delta^{{}^{T}}(\Sigma_{h}^{V})^{-1}\Delta|\leq
2\left\lVert
x\right\rVert_{2}\left\lVert\Delta\right\rVert_{2}\underline{c}^{-1}.$ (A.61)

If furthermore $2\left\lVert
x\right\rVert_{2}\left\lVert\Delta\right\rVert_{2}\underline{c}^{-1}<1.79$,
then it follows from (A.60)–(A.61) that

$\begin{split}|f_{h}(x+\Delta)-f_{h}(x)|&\leq
f_{h}(x)\Big{|}\exp{\left(-x^{{}^{T}}(\Sigma_{h}^{V})^{-1}\Delta-\frac{1}{2}\Delta^{{}^{T}}(\Sigma_{h}^{V})^{-1}\Delta\right)}-1\Big{|}\\\
&\leq f_{h}(x)\big{(}2\left\lVert
x\right\rVert_{2}\left\lVert\Delta\right\rVert_{2}\underline{c}^{-1}+(2\left\lVert
x\right\rVert_{2}\left\lVert\Delta\right\rVert_{2}\underline{c}^{-1})^{2}\big{)}.\end{split}$
(A.62)

  3. 3)

We show that for each $\mathcal{D}\in\mathcal{R}^{p_{h}}$,
$\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)})$ is a version of
$\mathbb{P}(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\ |\ V_{1t}^{(h)})$ and in
particular, for each $t$, $h>0$, and $\mathcal{D}\in\mathcal{R}^{p_{h}}$,

$\mathbb{P}(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\ |\
V_{1t}^{(h)})=\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)}).$

To this end, let us define a measurable function
$g(x)\coloneqq\int_{\mathbb{R}^{p_{h}}}\mu_{V_{21}^{(h)}}(dx_{2})\boldsymbol{1}_{x_{2}\in\mathcal{D}-x}$
with $\mathcal{D}-x\coloneqq\\{z-x:z\in\mathcal{D}\\}$ and write
$\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)})=g(V_{1t}^{(h)})$ to see that
$\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)})$ is
$\sigma(V_{1t}^{(h)})$-measurable. Observe that if we can show that for each
$\mathcal{A}\in\mathcal{R}^{p_{h}}$,

$\int_{\mathcal{A}}\mu_{V_{1t}^{(h)}}(dx_{1})\mu_{V_{21}^{(h)}}(\mathcal{D}-x_{1})=\int_{\\{V_{1t}^{(h)}\in\mathcal{A}\\}}\mathbb{P}(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\
|\ V_{1t}^{(h)})d\mathbb{P},$ (A.63)

then we can apply the change of variables formula to the left-hand side of
(A.63) and use the definition of conditional expectation to obtain the desired
result. It remains to prove (A.63).

Since $V_{1t}^{(h)}$ and $V_{2t}^{(h)}$ are independent for each $t$ and
$h>0$, it holds that for each $\mathcal{D},\mathcal{A}\in\mathcal{R}^{p_{h}}$,

$\begin{split}&\mathbb{P}(\\{V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\\}\cap\\{V_{1t}^{(h)}\in\mathcal{A}\\})\\\
&\quad=\int_{\mathbb{R}^{2p_{h}}}\mu_{V_{1t}^{(h)},V_{2t}^{(h)}}(dx_{1}\times
dx_{2})\boldsymbol{1}_{x_{1}+x_{2}\in\mathcal{D}}\boldsymbol{1}_{x_{1}\in\mathcal{A}}\\\
&\quad=\int_{\mathbb{R}^{2p_{h}}}\mu_{V_{1t}^{(h)}}(dx_{1})\mu_{V_{2t}^{(h)}}(dx_{2})\boldsymbol{1}_{x_{1}+x_{2}\in\mathcal{D}}\boldsymbol{1}_{x_{1}\in\mathcal{A}}\\\
&\quad=\int_{\mathbb{R}^{p_{h}}}\boldsymbol{1}_{(x_{1}\in\mathcal{A})}\mu_{V_{1t}^{(h)}}(dx_{1})\int_{\mathbb{R}^{p_{h}}}\mu_{V_{2t}^{(h)}}(dx_{2})\boldsymbol{1}_{x_{2}\in\mathcal{D}-x_{1}}\\\
&\quad=\int_{\mathbb{R}^{p_{h}}}\boldsymbol{1}_{(x_{1}\in\mathcal{A})}\mu_{V_{1t}^{(h)}}(dx_{1})\mu_{V_{2t}^{(h)}}(\mathcal{D}-x_{1})\\\
&\quad=\int_{\mathbb{R}^{p_{h}}}\boldsymbol{1}_{(x_{1}\in\mathcal{A})}\mu_{V_{1t}^{(h)}}(dx_{1})\mu_{V_{21}^{(h)}}(\mathcal{D}-x_{1})\\\
&\quad=\int_{\mathcal{A}}\mu_{V_{1t}^{(h)}}(dx_{1})\mu_{V_{21}^{(h)}}(\mathcal{D}-x_{1}),\end{split}$
(A.64)

where the second equality above is due to the independence. Moreover, since
$\mathbb{P}(\\{V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\\}\cap\\{V_{1t}^{(h)}\in\mathcal{A}\\}\
|\
V_{1t}^{(h)})=\boldsymbol{1}_{V_{1t}^{(h)}\in\mathcal{A}}\mathbb{P}(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\
|\ V_{1t}^{(h)})$, by the law of total expectation we have that

$\begin{split}&\mathbb{P}(\\{V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\\}\cap\\{V_{1t}^{(h)}\in\mathcal{A}\\})\\\
&=\mathbb{E}(\mathbb{P}(\\{V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\\}\cap\\{V_{1t}^{(h)}\in\mathcal{A}\\}\
|\ V_{1t}^{(h)}))\\\
&=\int_{\\{V_{1t}^{(h)}\in\mathcal{A}\\}}\mathbb{P}(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\
|\ V_{1t}^{(h)})d\mathbb{P}.\end{split}$ (A.65)

Hence, combining (A.64)–(A.65) leads to (A.63).

  4. 4)

Observe that $U_{2(t+h)}^{(h)}$ is independent of $U_{1t}^{(h)}+U_{2t}^{(h)}$
and $U_{1(t+h)}^{(h)}$. Thus, for each $\mathcal{D}\in\mathcal{R}^{p_{h}}$, we
have that

$\begin{split}&\mathbb{P}(U_{1(t+h)}^{(h)}+U_{2(t+h)}^{(h)}\in\mathcal{D}\ |\
U_{1t}^{(h)}+U_{2t}^{(h)},U_{1(t+h)}^{(h)})\\\
&=\mathbb{P}(U_{1(t+h)}^{(h)}+U_{2(t+h)}^{(h)}\in\mathcal{D}\ |\
U_{1(t+h)}^{(h)}).\end{split}$

Using such representation, similar arguments as in 3) above, and the fact that
$\mu_{U_{2(t+h)}^{(h)}}$ is identical to $\mu_{V_{21}^{(h)}}$, we can show
that

$\mathbb{P}(U_{1(t+h)}^{(h)}+U_{2(t+h)}^{(h)}\in\mathcal{D}\ |\
U_{1t}^{(h)}+U_{2t}^{(h)},U_{1(t+h)}^{(h)})=\mu_{V_{21}^{(h)}}(\mathcal{D}-U_{1(t+h)}^{(h)}).$

  5. 5)

Denote by $Q\coloneqq\left\\{\left\lVert V_{1t}^{(h)}\right\rVert_{\infty}\geq
e^{(-s_{3}h)}\right\\}$ and $G\coloneqq\left\\{\left\lVert
U_{1(t+h)}^{(h)}\right\rVert_{\infty}\geq e^{(-s_{3}h)}\right\\}$. Then it
follows from the definition of $\mu_{V_{21}^{(h)}}$ that for each
$\mathcal{D}\in\mathcal{R}^{p_{h}}$,

$\begin{split}&\boldsymbol{1}_{Q^{c}\cap
G^{c}}\left(\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)})-\mu_{V_{21}^{(h)}}(\mathcal{D}-U_{1(t+h)}^{(h)})\right)\\\
&\leq\sup_{\left\lVert\Delta_{i}\right\rVert_{\infty}<e^{-s_{3}h}}\Big{|}\int_{\mathcal{D}-\Delta_{2}}\big{(}f_{h}(x+\Delta_{2}-\Delta_{1})-f_{h}(x)\big{)}dx\Big{|}.\end{split}$
(A.66)

We are now ready to establish the desired upper bound. For each integer $h>0$,
$t$, and $\mathcal{D}\in\mathcal{R}^{p_{h}}$, it holds that

$\begin{split}&\left|\mathbb{P}\left(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\right)-\mathbb{P}\left(U_{1(t+h)}^{(h)}+U_{2(t+h)}^{(h)}\in\mathcal{D}\
\Big{|}\ U_{1t}^{(h)}+U_{2t}^{(h)}\right)\right|\\\
&=\Bigg{|}\mathbb{E}\left[\mathbb{P}(V_{1t}^{(h)}+V_{2t}^{(h)}\in\mathcal{D}\
|\ V_{1t}^{(h)})\right]\\\
&\quad-\mathbb{E}\Bigg{[}\mathbb{P}(U_{1(t+h)}^{(h)}+U_{2(t+h)}^{(h)}\in\mathcal{D}\
|\ U_{1t}^{(h)}+U_{2t}^{(h)},U_{1(t+h)}^{(h)})\ \Bigg{|}\
U_{1t}^{(h)}+U_{2t}^{(h)}\Bigg{]}\Bigg{|}.\end{split}$ (A.67)

By 3) and 4) above, for each $\mathcal{D}\in\mathcal{R}^{p_{h}}$, we have

$\begin{split}&\textnormal{RHS of
}(\ref{glp11})=\Big{|}\mathbb{E}\left[\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)})\right]-\mathbb{E}\left[\mu_{V_{21}^{(h)}}(\mathcal{D}-U_{1(t+h)}^{(h)})\
\Big{|}\ U_{1t}^{(h)}+U_{2t}^{(h)}\right]\Big{|}.\end{split}$ (A.68)

Since $V_{1t}^{(h)}$ is an independent copy, it follows that

$\begin{split}\textnormal{RHS of
(\ref{glp12})}&=\Big{|}\mathbb{E}\left[\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)})-\mu_{V_{21}^{(h)}}(\mathcal{D}-U_{1(t+h)}^{(h)})\
\Big{|}\ U_{1t}^{(h)}+U_{2t}^{(h)}\right]\Big{|}.\end{split}$ (A.69)

Next, we separate the expectation according to events $Q$ and $G$ as

$\begin{split}&\textnormal{RHS of (\ref{glp16})}\\\
&=\Big{|}\mathbb{E}\left[\left(\boldsymbol{1}_{Q\cup
G}+\boldsymbol{1}_{Q^{c}\cap
G^{c}}\right)\left(\mu_{V_{21}^{(h)}}(\mathcal{D}-V_{1t}^{(h)})-\mu_{V_{21}^{(h)}}(\mathcal{D}-U_{1(t+h)}^{(h)})\right)\
\Big{|}\ U_{1t}^{(h)}+U_{2t}^{(h)}\right]\Big{|}.\end{split}$ (A.70)

Then by (A.66) and some simple calculations, we can show that

$\begin{split}&\textnormal{RHS of (\ref{glp17})}\\\ &\leq\mathbb{P}(G\cup Q\
|\
U_{1t}^{(h)}+U_{2t}^{(h)})+\sup_{\left\lVert\Delta_{i}\right\rVert_{\infty}<e^{-s_{3}h}}\Big{|}\int_{\mathcal{D}-\Delta_{2}}\big{(}f_{h}(x+\Delta_{2}-\Delta_{1})-f_{h}(x)\big{)}dx\Big{|}.\end{split}$
(A.71)

For the first term on the RHS of (LABEL:glp18), it follows from the
definitions of $Q$ and $G$ and the stationarity of $V_{1t}^{(h)}$ that

$\begin{split}&\mathbb{P}(G\cup Q\ |\
U_{1t}^{(h)}+U_{2t}^{(h)})\leq\mathbb{P}\left(\left\lVert
V_{11}^{(h)}\right\rVert_{\infty}\geq e^{(-s_{3}h)}\right)\\\
&\quad+\mathbb{P}\Big{(}\left\lVert U_{1(t+h)}^{(h)}\right\rVert_{\infty}\geq
e^{(-s_{3}h)}\ |\ U_{1t}^{(h)}+U_{2t}^{(h)}\Big{)}.\end{split}$ (A.72)

For the second term on the RHS of (LABEL:glp18), it holds that

$\begin{split}&\sup_{\left\lVert\Delta_{i}\right\rVert_{\infty}<e^{-s_{3}h}}\Big{|}\int_{\mathcal{D}-\Delta_{2}}\big{(}f_{h}(x+\Delta_{2}-\Delta_{1})-f_{h}(x)\big{)}dx\Big{|}\\\
&\leq 2\mathbb{P}\left(\left\lVert
V_{21}^{(h)}\right\rVert_{\infty}\geq\exp{((1-\delta_{0})s_{3}h)}-2\exp{(-s_{3}h)}\right)\\\
&\quad+\sup_{\left\lVert\Delta_{i}\right\rVert_{\infty}<e^{-s_{3}h}}\Big{|}\int_{x\in\mathcal{D}-\Delta_{2},\,\left\lVert
x\right\rVert_{\infty}<e^{(1-\delta_{0})s_{3}h}}\big{(}f_{h}(x+\Delta_{2}-\Delta_{1})-f_{h}(x)\big{)}dx\Big{|}.\end{split}$
(A.73)

We proceed with dealing with the last term on the RHS of (LABEL:glp215). Let
$x_{1},x_{2}$ denote two vectors in $\mathbb{R}^{p_{h}}$. Then, it follows
from (A.50) that

$\lim\sup_{h\rightarrow\infty}\sup_{\begin{subarray}{c}\left\lVert
x_{2}\right\rVert_{2}\leq 2\sqrt{p_{h}}e^{-s_{3}h}\\\ \left\lVert
x_{1}\right\rVert_{2}\leq\sqrt{p_{h}}e^{(1-\delta_{0})s_{3}h}\end{subarray}}\left\lVert
x_{1}\right\rVert_{2}\left\lVert x_{2}\right\rVert_{2}\underline{c}^{-1}=0.$
(A.74)

Let us define $\Delta\coloneqq\Delta_{2}-\Delta_{1}$. In light of the fact
that $\left\lVert z\right\rVert_{2}\leq\sqrt{p_{h}}\left\lVert
z\right\rVert_{\infty}$ for all $z\in\mathbb{R}^{p_{h}}$, (A.62), (A.74), and
the fact that $\int_{x\in\mathbb{R}^{p_{h}}}f_{h}(x)dx=1$, it holds that for
all large $h$,

$\begin{split}&\sup_{\left\lVert\Delta_{i}\right\rVert_{\infty}<e^{-s_{3}h}}\Big{|}\int_{\begin{subarray}{c}x\in\mathcal{D}-\Delta_{2},\,\left\lVert
x\right\rVert_{\infty}<e^{(1-\delta_{0})s_{3}h}\end{subarray}}\big{(}f_{h}(x+\Delta_{2}-\Delta_{1})-f_{h}(x)\big{)}dx\Big{|}\\\
&\quad\leq\sup_{\left\lVert\Delta\right\rVert_{\infty}<2e^{-s_{3}h},\left\lVert\Delta_{2}\right\rVert_{\infty}<e^{-s_{3}h}}\Big{|}\int_{\begin{subarray}{c}x\in\mathcal{D}-\Delta_{2},\,\left\lVert
x\right\rVert_{\infty}<e^{(1-\delta_{0})s_{3}h}\end{subarray}}\big{(}f_{h}(x+\Delta)-f_{h}(x)\big{)}dx\Big{|}\\\
&\quad\leq\sup_{\begin{subarray}{c}\left\lVert\Delta\right\rVert_{2}\leq
2\sqrt{p_{h}}e^{-s_{3}h}\\\ \left\lVert
x\right\rVert_{2}\leq\sqrt{p_{h}}e^{(1-\delta_{0})s_{3}h}\end{subarray}}(2\left\lVert
x\right\rVert_{2}\left\lVert\Delta\right\rVert_{2}\underline{c}^{-1}+(2\left\lVert
x\right\rVert_{2}\left\lVert\Delta\right\rVert_{2}\underline{c}^{-1})^{2})\\\
&\quad\leq\sup_{\begin{subarray}{c}\left\lVert\Delta\right\rVert_{2}\leq
2\sqrt{p_{h}}e^{-s_{3}h}\\\ \left\lVert
x\right\rVert_{2}\leq\sqrt{p_{h}}e^{(1-\delta_{0})s_{3}h}\end{subarray}}4\left\lVert
x\right\rVert_{2}\left\lVert\Delta\right\rVert_{2}\underline{c}^{-1}\\\
&\quad\leq 8\underline{c}^{-1}p_{h}\exp{(-\delta_{0}s_{3}h)}.\end{split}$
(A.75)

Therefore, combining (LABEL:glp11)–(LABEL:glp215), (A.75), and the
stationarity of the process yields the desired conclusion. This completes the
proof of Claim (LABEL:glp7).

###  B.3 Lemma 1 and its proof

The theoretical foundation of our subsampling method is provided by Lemma 1,
which concerns the asymptotical independence of the $\beta$-mixing random
vectors in each subsample. Since (LABEL:M0) below for Lemma 1 involves
stacking up stationary elements columnwise (there are $l$ elements in a row of
matrices in (LABEL:M0) below), it is unclear whether we can directly apply
Lemma 4.1 of [40] to our setting. Thus, we provide our self-contained proof
for Lemma 1. Our technical analysis of Lemma 1 seems to be the first formal
proof for results on the asymptotically independent blocks due to the
$\beta$-mixing and subsampling.

Consider a $p$-dimensional vector-valued stationary process
$\\{\boldsymbol{z}_{t}\\}$. Let $n$, $\bar{n}$, $h$, and $l$ be positive
integers such that

$\bar{n}=\sup\\{s\in\mathbb{N}:s(l+h)-h\ \leq n\\}>0.$ (A.76)

We construct two $\bar{n}\times(lp)$ design matrices as

$\begin{split}&\boldsymbol{M}\coloneqq\left(\begin{array}[]{c}\boldsymbol{z}_{(1-1)\times(l+h)+1}^{{}^{T}},\cdots,\boldsymbol{z}_{(2-1)\times(l+h)-h}^{{}^{T}}\\\
\cdots\\\
\boldsymbol{z}_{(\bar{n}-1)\times(l+h)+1}^{{}^{T}},\cdots,\boldsymbol{z}_{\bar{n}\times(l+h)-h}^{{}^{T}}\end{array}\right)\\\
&\text{and }\
\boldsymbol{M}^{\pi}\coloneqq\left(\begin{array}[]{c}(\boldsymbol{z}_{1}^{\pi_{1}})^{{}^{T}},\cdots,(\boldsymbol{z}_{l}^{\pi_{1}})^{{}^{T}}\\\
\cdots\\\
(\boldsymbol{z}_{1}^{\pi_{\bar{n}}})^{{}^{T}},\cdots,(\boldsymbol{z}_{l}^{\pi_{\bar{n}}})^{{}^{T}}\end{array}\right),\end{split}$
(A.77)

where
$\\{(\boldsymbol{z}_{1}^{\pi_{t}},\cdots,\boldsymbol{z}_{l}^{\pi_{t}})\\}_{t}$
is an i.i.d. sequence such that
$(\boldsymbol{z}_{1}^{\pi_{1}},\cdots,\boldsymbol{z}_{l}^{\pi_{1}})$ has the
same distribution as $(\boldsymbol{z}_{1},\cdots,\boldsymbol{z}_{l})$. Here,
matrix $\boldsymbol{M}$ is obtained by removing $h$ random vectors in the
process after each consecutive $l$ random vectors, and then stacking up the
remaining random vectors. Lemma 1 below characterizes the distributional
distance between $\boldsymbol{M}$ with dependent rows and
$\boldsymbol{M}^{\pi}$ with i.i.d. rows.

######  Lemma 1.

Assume that the $p$-dimensional process $\\{\boldsymbol{z}_{t}\\}$ satisfies
Condition 4 with $(h+1)$-step and constants $0\leq\rho<1$ and $C_{0}>0$. Then
it holds that

$\sup_{\mathcal{D}\in\mathcal{R}^{lp\bar{n}}}\Big{|}\mathbb{P}(\boldsymbol{M}\in\mathcal{D})-\mathbb{P}(\boldsymbol{M}^{\pi}\in\mathcal{D})\Big{|}\leq
n\rho^{h+1}C_{0},$ (A.78)

where random matrices $\boldsymbol{M}$ and $\boldsymbol{M}^{\pi}$ are defined
in (LABEL:M0).

Proof. If we view the rows of $\boldsymbol{M}$ as random mappings, a simple
version of this problem is to establish an upper bound of the total variation
distance between the distributions of $U_{1},\cdots,U_{\bar{n}}$ and their
i.i.d. counterparts denoted as $U_{1}^{\pi},\cdots,U_{\bar{n}}^{\pi}$. The
main technique used here is to separate the total variation distance into
$\textnormal{TV}_{1},\cdots,\textnormal{TV}_{\bar{n}}$ introduced below and
control them separately as

$\begin{split}U_{1},U_{2},U_{3},\cdots,U_{\bar{n}}&\quad\underbrace{\longleftrightarrow}_{\textnormal{TV}_{1}}\quad
U_{1}^{\pi},U_{2},U_{3},\cdots,U_{\bar{n}}\quad\underbrace{\longleftrightarrow}_{\textnormal{TV}_{2}}\quad
U_{1}^{\pi},U_{2}^{\pi},U_{3},\cdots,U_{\bar{n}}\\\
&\quad\underbrace{\longleftrightarrow}_{\textnormal{TV}_{3}}\quad\cdots\quad\underbrace{\longleftrightarrow}_{\textnormal{TV}_{\bar{n}}}\quad
U_{1}^{\pi},U_{2}^{\pi},\cdots,U_{\bar{n}}^{\pi}.\end{split}$ (A.79)

By the technique in (A.79) above, for each step we can focus on the total
variation distance between two processes with only one distinct part. For
example, for the $j$th and $(j+1)$th processes, the distinct part is $U_{j}$
and $U_{j}^{\pi}$. We will present the formal proof next.

To ease the technical presentation, let us first introduce some notation.
Denote the distributions as

$\begin{split}&\mu_{1}\coloneqq\mu_{(\boldsymbol{z}_{(1-1)(l+h)+1\ :\
1(l+h)-h},\cdots,\boldsymbol{z}_{(i-1)(l+h)+1\ :\
i(l+h)-h},\cdots,\boldsymbol{z}_{(\bar{n}-1)(l+h)+1\ :\ \bar{n}(l+h)-h})},\\\
&\mu_{\bar{n}+1}\coloneqq\mu_{(\boldsymbol{z}^{\pi_{1}},\cdots,\boldsymbol{z}^{\pi_{\bar{n}}})},\\\
&\mu_{j}\coloneqq\mu_{(\boldsymbol{z}^{\pi_{1}},\cdots,\boldsymbol{z}_{(j-1)(l+h)+1\
:\ j(l+h)-h},\cdots,\boldsymbol{z}_{(\bar{n}-1)(l+h)+1\ :\
\bar{n}(l+h)-h})}\end{split}$ (A.80)

for $j=2,\cdots,\bar{n}$. Observe that we have $\mu_{\bar{n}}=\mu_{\bar{n}+1}$
since the process is stationary. With the notation introduced above, the
desired conclusion is an upper bound for
$\frac{1}{2}\left\lVert\mu_{1}-\mu_{\bar{n}+1}\right\rVert_{TV}$.

For completeness, we state some important properties of the transition kernel
$p:\mathbb{R}^{p}\times\mathcal{R}^{p}\longrightarrow[0,1]$ of a stationary
Markov chain with stationary distribution $\pi$. 1) For each integer $t$ and
$\mathcal{D}\in\mathcal{R}^{p}$, $p(\boldsymbol{z}_{t},\mathcal{D})$ is a
version of
$\mathbb{P}(\boldsymbol{z}_{t+1}\in\mathcal{D}|\boldsymbol{z}_{t})$. 2) For
each measurable function $f$ and $\mathcal{D}\in\mathcal{R}^{p}$,
$\int_{\mathcal{D}}p(\vec{x},d\vec{y})f(\vec{y})$ is a measurable function of
$\vec{x}$, and hence for each $\mathcal{D}_{k}\in\mathcal{R}^{p}$,

$\int_{\mathcal{D}_{1}}\pi(d\vec{x}_{1})\int_{\mathcal{D}_{2}}p(\vec{x}_{1},d\vec{x}_{2})\cdots\int_{\mathcal{D}_{k}}p(\vec{x}_{k-1},d\vec{x}_{k})f(\vec{x}_{k})$

is a well-defined integral. 3) For each measurable function $f$ and
$\mathcal{D}\in\mathcal{R}^{p}$,

$\int_{\mathbb{R}^{p}}\pi(d\vec{x})\int_{\mathcal{D}}p(d\vec{x},d\vec{y})f(\vec{y})=\int_{\mathcal{D}}\pi(d\vec{x})f(\vec{x}).$
(A.81)

4) We have an expression of $\mu_{j}$ as given in (LABEL:airv1) below, where
we indicate each part of the distribution of $\mu_{j}$ according to

$\underbrace{\boldsymbol{z}^{\pi_{1}}}_{\textnormal{1st
part}},\cdots,\underbrace{\boldsymbol{z}_{(j-1)(l+h)+1\ :\
j(l+h)-h}}_{\textnormal{$j$th
part}},\cdots,\underbrace{\boldsymbol{z}_{(\bar{n}-1)(l+h)+1\ :\
\bar{n}(l+h)-h}}_{\textnormal{$\bar{n}$th part}}.$

Such representation follows from the first two properties, and the details on
deriving it can be found in Section 5.2 of [16]. For each
$\mathcal{D}\in\mathcal{R}^{lp\bar{n}}$, $\mu_{j}(\mathcal{D})$ admits the
following representation

$\begin{split}&\mu_{j}(\mathcal{D})=\int_{\mathcal{D}}\underbrace{\pi(d\vec{x}_{1})\times\cdots\times
p(\vec{x}_{l-1},d\vec{x}_{l})}_{\textnormal{$1$st part}}\times\cdots\\\
&\hskip
10.00002pt\times\underbrace{\pi(d\vec{x}_{(j-1)(l+h)+1})\times\cdots\times
p(\vec{x}_{j(l+h)-h-1},d\vec{x}_{j(l+h)-h})}_{\textnormal{$j$th
part}}\times\cdots\\\ &\hskip
10.00002pt\times\underbrace{p^{h+1}(\vec{x}_{j(l+h)-h},d\vec{x}_{j(l+h)+1})\times\cdots\times
p(\vec{x}_{(j+1)(l+h)-h-1},d\vec{x}_{(j+1)(l+h)-h})}_{\textnormal{$(j+1)$th
part}}\times\cdots\\\ &\hskip
10.00002pt\times\underbrace{p^{h+1}(\vec{x}_{(\bar{n}-1)(l+h)-h},d\vec{x}_{(\bar{n}-1)(l+h)+1})\times\cdots\times
p(\vec{x}_{\bar{n}(l+h)-h-1},d\vec{x}_{\bar{n}(l+h)-h})}_{\bar{n}\textnormal{th
part}},\end{split}$ (A.82)

where $\underbrace{\vec{x}_{1},\cdots,\vec{x}_{l}}_{\textnormal{$1$st
part}},\cdots,\underbrace{\vec{x}_{(\bar{n}-1)(l+h)+1},\cdots,\vec{x}_{\bar{n}(l+h)-h}}_{\bar{n}\textnormal{th
part}}$ stand for the corresponding running variables with
$\vec{x}_{k}\in\mathbb{R}^{p}$ for each $k$.

Let us make use of a critical observation that

$\frac{1}{2}\left\lVert\mu_{1}-\mu_{\bar{n}+1}\right\rVert_{TV}\leq\sum_{i=1}^{\bar{n}}\sup_{\mathcal{D}\in\mathcal{R}^{lp\bar{n}}}|\mu_{j}(\mathcal{D})-\mu_{j+1}(\mathcal{D})|.$
(A.83)

We will bound each term in the above summation separately. Let us fix $1\leq
j\leq\bar{n}$. We notice that $\mu_{j}$ and $\mu_{j+1}$ are almost identical
except for the $(j+1)$th part in (LABEL:airv1). By a careful comparison, we
see that for $\mu_{j}$, the $(j+1)$th part starts with
$p^{h+1}(\vec{x}_{j(l+h)-h},d\vec{x}_{j(l+h)+1})$, whereas the $(j+1)$th part
of $\mu_{j+1}$ starts with $\pi(d\vec{x}_{j(l+h)+1})$. To see the difficulty
for bounding each term on the right-hand side (RHS) of (A.83) using such
observation, note that $\int_{\mathcal{D}}|\mu_{1}(dx)-\mu_{2}(dx)|$ is not a
well-defined integral for a Borel set $\mathcal{D}$ and two measures $\mu_{1}$
and $\mu_{2}$ since there are two $dx$’s inside the integration. To have a
valid argument for this bound, we use the Radon–Nikodym theorem [16] to
replace the underlying measures with measurable functions (the Radon–Nikodym
derivatives). The arguments follow mainly those for the proof of Lemma 5 in
Section B.7.

By Condition 4, for each $\vec{x}\in\mathbb{R}^{p}$ it holds that
$p(\vec{x},\cdot)$ is dominated by the Lebesgue measure. Since $p$ is the
transition kernel of the stationary Markov chain, this entails that 1)
$p^{h+1}(\vec{x},\cdot)$ is dominated by the Lebesgue measure for each
$\vec{x}\in\mathbb{R}^{p}$ and 2) $\pi(\cdot)$ is also dominated by the
Lebesgue measure. By 1) and the Radon–Nikodym Theorem, there exists a
nonnegative measurable function on $\mathbb{R}^{2p}$, which is denoted as $r$,
such that for each $\vec{x}\in\mathbb{R}^{p}$ and
$\mathcal{D}\in\mathcal{R}^{p}$,

$p^{h+1}(\vec{x},\mathcal{D})=\int_{\mathcal{D}}r(\vec{x},\vec{y})\ d\vec{y}.$

This measurable function is simply the Radon–Nikodym derivative [16], and
$r(\vec{x},\vec{y})$ is also called the probability density functions of
$\boldsymbol{z}_{t+h+1}$ conditional on $\boldsymbol{z}_{t}$. In particular,
for each $\mathcal{D}_{1},\mathcal{D}_{2}\in\mathcal{R}^{p}$, we have that

$\mathbb{P}((\boldsymbol{z}_{t},\boldsymbol{z}_{t+h+1})\in\mathcal{D}_{1}\times\mathcal{D}_{2})=\int_{\vec{x}\in\mathcal{D}_{1}}\pi(d\vec{x})\int_{\vec{y}\in\mathcal{D}_{2}}r(\vec{x},\vec{y})d\vec{y}.$

For more details on the conditional probability density functions, see Example
4.1.6 of [16]. Furthermore, by 2) we denote by $r_{\pi}(\vec{x})$ the
Radon–Nikodym derivative such that for each $\mathcal{D}\in\mathcal{R}^{p}$,

$\pi(\mathcal{D})=\int_{\mathcal{D}}r_{\pi}(\vec{x})d\vec{x}.$

Thus, we can obtain that

$\begin{split}&\mu_{j}(\mathcal{D})=\int_{\mathcal{D}}\cdots\\\
&\quad\times\underbrace{r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})d\vec{x}_{j(l+h)+1}\times\cdots\times
p(\vec{x}_{(j+1)(l+h)-h-1},d\vec{x}_{(j+1)(l+h)-h})}_{\textnormal{$(j+1)$th
part}}\\\
&\quad\times\cdots\times\underbrace{p^{h+1}(\vec{x}_{(\bar{n}-1)(l+h)-h},d\vec{x}_{(\bar{n}-1)(l+h)+1})\times\cdots\times
p(\vec{x}_{\bar{n}(l+h)-h-1},d\vec{x}_{\bar{n}(l+h)-h})}_{\bar{n}\textnormal{th
part}}.\end{split}$ (A.84)

A similar expression also holds for $\mu_{j+1}$ with $r_{\pi}$. We will bound
$|\mu_{j}(\mathcal{D})-\mu_{j+1}(\mathcal{D})|$ next.

It follows from (A.84) and the fact of $p\leq 1$ that for each
$\mathcal{D}\in\mathcal{R}^{lp\bar{n}}$,

$\begin{split}|&\mu_{j}(\mathcal{D})-\mu_{j+1}(\mathcal{D})|\\\
&=\left|\int_{\mathcal{D}}\underbrace{\cdots}_{1,\cdots,j}\times\left(r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})-r_{\pi}(\vec{x}_{j(l+h)+1})\right)d\vec{x}_{j(l+h)+1}\times\cdots\right|\\\
&\leq\int_{\mathcal{D}}\underbrace{\cdots}_{1,\cdots,j}\times\left|r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})-r_{\pi}(\vec{x}_{j(l+h)+1})\right|d\vec{x}_{j(l+h)+1}\times\cdots\\\
&\leq\int_{\mathbb{R}^{lpj+p}}\underbrace{\cdots}_{1\textnormal{st},\cdots,j\textnormal{th
parts}}\times\left|r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})-r_{\pi}(\vec{x}_{j(l+h)+1})\right|d\vec{x}_{j(l+h)+1}.\end{split}$
(A.85)

To bound the RHS of (A.85), we separate the modulus into positive and negative
parts and get rid of the modulus operation. Let $\mathcal{D}_{+}$ and
$\mathcal{D}_{-}$ be two disjoint Borel sets such that

$\begin{split}&\int_{\mathbb{R}^{lpj+p}}\underbrace{\cdots}_{1,\cdots,j}\times\left|r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})-r_{\pi}(\vec{x}_{j(l+h)+1})\right|d\vec{x}_{j(l+h)+1}\\\
&=\int_{\mathcal{D}_{+}}\underbrace{\cdots}_{1,\cdots,j}\times\left(r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})-r_{\pi}(\vec{x}_{j(l+h)+1})\right)d\vec{x}_{j(l+h)+1}\\\
&\quad+\int_{\mathcal{D}_{-}}\underbrace{\cdots}_{1\textnormal{st},\cdots,j\textnormal{th
parts}}\times\left(r_{\pi}(\vec{x}_{j(l+h)+1})-r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})\right)d\vec{x}_{j(l+h)+1}.\end{split}$
(A.86)

To proceed, we exploit arguments involving “cross-sections.” For any
$\mathcal{D}\in\mathbb{R}^{k_{1}+k_{2}}$ and $\vec{x}\in\mathbb{R}^{k_{1}}$
with $k_{1}$ and $k_{2}$ some positive integers, let us define the cross-
section at $\vec{x}$ as
$\mathcal{D}_{\vec{x}}\coloneqq\\{\vec{y}:(\vec{x},\vec{y})\in\mathcal{D}\\}$.
See Section B.4 for more detail on cross-sections. The it holds that

$\begin{split}&\int_{\mathcal{D}_{+}}\underbrace{\cdots}_{1,\cdots,j}\times\left(r(\vec{x}_{j(l+h)-h},\vec{x}_{j(l+h)+1})-r_{\pi}(\vec{x}_{j(l+h)+1})\right)d\vec{x}_{j(l+h)+1}\\\
&=\int_{\mathcal{D}_{+}}\underbrace{\cdots}_{1,\cdots,j}\times
p^{h+1}(\vec{x}_{j(l+h)-h},d\vec{x}_{j(l+h)+1})-\int_{\mathcal{D}_{+}}\underbrace{\cdots}_{1,\cdots,j}\times\pi(d\vec{x}_{j(l+h)+1})\\\
&=\int_{\mathbb{R}^{lpj}}\underbrace{\cdots}_{1,\cdots,j}\times
p^{h+1}(\vec{x}_{j(l+h)-h},(\mathcal{D}_{+})_{z})-\int_{\mathbb{R}^{lpj}}\underbrace{\cdots}_{1,\cdots,j}\times\pi((\mathcal{D}_{+})_{z})\\\
&=\int_{\mathbb{R}^{lpj}}\underbrace{\cdots}_{1\textnormal{st},\cdots,j\textnormal{th
parts}}\times\left(p^{h+1}(\vec{x}_{j(l+h)-h},(\mathcal{D}_{+})_{z})-\pi((\mathcal{D}_{+})_{z})\right),\end{split}$
(A.87)

where $z$ represents
$(\vec{x}_{1}^{{}^{T}},\cdots,\vec{x}_{j(l+h)-h}^{{}^{T}})^{{}^{T}}$ in the
integration. Here, we have used the definition of the Radon–Nikodym derivative
to get the first equality in (LABEL:airv6). The second equality in
(LABEL:airv6) is justified by the fact that $\pi$ is a distribution and hence
a transition kernel, and an application of Lemma 3 in Section B.5. To apply
Lemma 3 in (LABEL:airv6), we can regard $\boldsymbol{z}_{j(l+h)+1}$ as
$\boldsymbol{X}_{3}$, $\boldsymbol{z}_{j(l+h)-h}$ as $\boldsymbol{X}_{2}$, and
the remaining variables as $\boldsymbol{X}_{1}$ in Lemma 3, and notice that
(A.90) is satisfied due to the Markov property. Similar arguments can be
applied to $\mathcal{D}_{-}$ too.

In view of (LABEL:airv5) and (LABEL:airv6), it follows from the fact of
$\mathcal{D}_{+}\cap\mathcal{D}_{-}=\emptyset$, (A.81), and Condition 4 that

$\begin{split}\textnormal{RHS of
}(\ref{airv4})&=\int_{\mathbb{R}^{lpj}}\underbrace{\cdots}_{1,\cdots,j}\times\Big{[}\left(p^{h+1}(\vec{x}_{j(l+h)-h},(\mathcal{D}_{+})_{z})-\pi((\mathcal{D}_{+})_{z})\right)\\\
&\quad-\left(p^{h+1}(\vec{x}_{j(l+h)-h},(\mathcal{D}_{-})_{z})-\pi((\mathcal{D}_{-})_{z})\right)\Big{]}\\\
&\leq\int_{\mathbb{R}^{lpj}}\underbrace{\cdots}_{1\textnormal{st},\cdots,j\textnormal{th
parts}}\times\left\lVert
p^{h+1}(\vec{x}_{j(l+h)-h},\cdot)-\pi(\cdot)\right\rVert_{TV}\\\
&=\int_{\mathbb{R}^{p}}\pi(d\vec{x}_{j(l+h)-h})\left\lVert
p^{h+1}(\vec{x}_{j(l+h)-h},\cdot)-\pi(\cdot)\right\rVert_{TV}\\\
&\leq\int_{\mathbb{R}^{p}}V(\vec{x})\pi(d\vec{x})\rho^{h+1}C,\end{split}$
(A.88)

where $C>0$ is given in Condition 4. By Condition 4, we can further show that

$\textnormal{RHS of }(\ref{airv7})\leq C_{0}\rho^{h+1},$

where $C_{0}>0$ is a constant such that
$\int_{\mathbb{R}^{p}}V(\vec{x})\pi(d\vec{x})C\leq C_{0}$. Therefore,
combining (A.83), (A.88), and the fact of $\bar{n}\leq n$, we can see that the
upper bound is given by $n\rho^{h+1}C_{0}$, which concludes the proof of Lemma
1.

###  B.4 Lemma 2 and its proof

To ease the technical presentation, let us first introduce some notation. For
any $\mathcal{D}\subset\mathbb{R}^{k_{1}+k_{2}}$ and $x\in\mathbb{R}^{k_{1}}$
with $k_{1}$ and $k_{2}$ some positive integers, we define the cross-section
at $x$ as $\mathcal{D}_{x}\coloneqq\\{y:(x,y)\in\mathcal{D}\\}$. A standard
operation on $\mathcal{D}_{x}$ is as follows. If
$\mathcal{D}_{1}\subset\mathcal{D}_{2}$, then we have that for each $x$,
$(\mathcal{D}_{1})_{x}\subset(\mathcal{D}_{2})_{x}$ and
$(\mathcal{D}_{2}\backslash\mathcal{D}_{1})_{x}=(\mathcal{D}_{2})_{x}\backslash(\mathcal{D}_{1})_{x}$.
In addition, for any set $\mathcal{D}\subset\mathbb{R}^{k}$ and
$x\in\mathbb{R}^{k}$, we denote by $\mathcal{D}-x$ the set
$\\{y-x:y\in\mathcal{D}\\}$. The expectation $\int f(x)\mu(dx)$ for a
measurable function $f$ with respect to some random vector $\boldsymbol{X}$
with distribution $\mu$ is written as $\int_{\mathbb{R}^{k}}fd\mu$ whenever
the running variables are obvious. We also use the product notation in the
integration to specify the running variables such as $\int
f(x_{2})\mu(dx_{1}\times dx_{2})$.

######  Lemma 2.

Let $\boldsymbol{X}$ and $\boldsymbol{Y}$ be $k_{1}$-dimensional and
$k_{2}$-dimensional random vectors, respectively. Assume that
$h:\mathbb{R}^{k_{1}}\times\mathcal{R}^{k_{2}}\longrightarrow[0,1]$ is a
transition kernel such that for each $\mathcal{D}\in\mathcal{R}^{k_{2}}$,
$h(\boldsymbol{X},\mathcal{D})$ is a version of
$\mathbb{P}(\boldsymbol{Y}\in\mathcal{D}\ |\ \boldsymbol{X})$. Then it holds
that

  1. 1)

For each $\mathcal{D}\in\mathcal{R}^{k_{1}+k_{2}}$ and
$x\in\mathbb{R}^{k_{1}}$, $\mathcal{D}_{x}\in\mathcal{R}^{k_{2}}$.

  2. 2)

For each $\mathcal{D}\in\mathcal{R}^{k_{1}+k_{2}}$,
$h(\cdot,\mathcal{D}_{\cdot})$ is $\mathcal{R}^{k_{1}}$-measurable.

  3. 3)

For each $\mathcal{D}\in\mathcal{R}^{k_{1}+k_{2}}$,
$\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\mathcal{D})=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)h(x,\mathcal{D}_{x})$.

Proof. We begin with showing part 1). Let $L$ be the collection of sets in
$\mathcal{R}^{k_{1}+k_{2}}$ satisfying the required conditions; that is, for
each $\mathcal{D}\in L$, it holds that for each $x\in\mathbb{R}^{k_{1}}$,
$\mathcal{D}_{x}\in\mathcal{R}^{k_{2}}$. Then it is easy to verify that $L$
contains all the rectangles of form $\mathcal{A}\times\mathcal{B}$, where
$\mathcal{A}\in\mathcal{R}^{k_{1}}$ and $\mathcal{B}\in\mathcal{R}^{k_{2}}$.
By the basic set operations, it holds that for each $x\in\mathbb{R}^{k_{1}}$
and $E,E_{i}\in\mathcal{R}^{k_{1}+k_{2}}$,

  1. a)

$(E_{x})^{c}=(\\{y:(x,y)\in E\\})^{c}=\\{y:(x,y)\in E^{c}\\}=(E^{c})_{x}$;

  2. b)

$\cup_{i}(E_{i})_{x}=\cap_{i}((E_{i})_{x})^{c}=\cap_{i}(E_{i}^{c})_{x}=(\cap_{i}E_{i}^{c})_{x}=(\cup_{i}E_{i})_{x}$.

Thus, for $E,E_{i}\in L$, we have that $E^{c},\cup_{i}E_{i}\in L$. This shows
that $L$ is a $\sigma$-algebra. Since $L$ contains all the rectangles, we
obtain the conclusion in part 1) of Lemma 2.

We next proceed to establish part 2). Since $\mathcal{D}_{x}$ is a measurable
set, $h(x,\mathcal{D}_{x})$ is a well-defined function of $x$. Let $L$ be the
collection of sets such that for each $\mathcal{D}\in L$,
$h(\cdot,\mathcal{D}_{\cdot})$ is $\mathcal{R}^{k_{1}}$-measurable. Since for
each $\mathcal{A}\in\mathcal{R}^{k_{1}}$ and
$\mathcal{B}\in\mathcal{R}^{k_{2}}$, it holds that

$h(x,(\mathcal{A}\times\mathcal{B})_{x})=h(x,\mathcal{B})\boldsymbol{1}_{\\{x\in\mathcal{A}\\}},$

which is a measurable function of $x$, we can see that $L$ contains all such
rectangles. Moreover, if $\mathcal{D}_{1},\mathcal{D}_{2}\in L$ with
$\mathcal{D}_{1}\subset\mathcal{D}_{2}$, then it follows that for each $x$,
$(\mathcal{D}_{1})_{x}\subset(\mathcal{D}_{2})_{x}$ and
$(\mathcal{D}_{2}\backslash\mathcal{D}_{1})_{x}=(\mathcal{D}_{2})_{x}\backslash(\mathcal{D}_{1})_{x}$,
and hence

$h(x,(\mathcal{D}_{2}\backslash\mathcal{D}_{1})_{x})=h(x,(\mathcal{D}_{2})_{x}\backslash(\mathcal{D}_{1})_{x})=h(x,(\mathcal{D}_{2})_{x})-h(x,(\mathcal{D}_{1})_{x}).$

Observe that the RHS of the equality above is measurable since the subtraction
of measurable functions is still measurable. Next, if $\mathcal{D}_{i}\in L$
and $\mathcal{D}_{i}\subset\mathcal{D}_{i+1}$, by the continuity of measure,
we have that for each $x\in\mathbb{R}^{k_{1}}$,
$\lim_{n\rightarrow\infty}h(x,(\cup_{i=1}^{n}\mathcal{D}_{i})_{x})=h(x,(\cup_{i=1}^{\infty}\mathcal{D}_{i})_{x})$.
Thus, $h(x,(\cup_{i=1}^{\infty}\mathcal{D}_{i})_{x})$ is a measurable function
of $x$, and we have $\cup_{i=1}^{\infty}\mathcal{D}_{i}\in L$. This shows that
$L$ is a $\lambda$-system containing the set of all the rectangles. Hence, by
Lemma 8 in Section B.12, we see that $L$ contains the $\sigma$-algebra
generated by the set, which concludes the proof for part 2) of Lemma 2.

Finally, let us show part 3). Note that the RHS of the assertion is well-
defined due to part 2) of Lemma 2. By the definition of the conditional
expectation, the change of variables formula, and the fact that for each
$x\in\mathbb{R}^{k_{1}}$, $\mathcal{A}\in\mathcal{R}^{k_{1}}$,
$\mathcal{B}\in\mathcal{R}^{k_{2}}$,

$\boldsymbol{1}_{\\{x\in\mathcal{A}\\}}h(x,\mathcal{B})=h(x,(\mathcal{A}\times\mathcal{B})_{x}),$

we can deduce that

$\begin{split}\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\mathcal{A}\times\mathcal{B})&=\int_{\Omega}\boldsymbol{1}_{\\{\boldsymbol{X}\in\mathcal{A}\\}}\
\mathbb{P}(\boldsymbol{Y}\in\mathcal{B}\ |\ \boldsymbol{X})d\mathbb{P}\\\
&=\int_{\Omega}\boldsymbol{1}_{\\{\boldsymbol{X}\in\mathcal{A}\\}}\
h(\boldsymbol{X},\mathcal{B})d\mathbb{P}\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
\boldsymbol{1}_{\\{x\in\mathcal{A}\\}}\ h(x,\mathcal{B})\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
h(x,(\mathcal{A}\times\mathcal{B})_{x}),\end{split}$ (A.89)

where $\Omega$ represents the underlying probability space.

Denote by $L$ the collection of sets in $\mathcal{R}^{k_{1}+k_{2}}$ satisfying
the required condition; that is, for each $\mathcal{D}\in L$, it holds that
$\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\mathcal{D})=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
h(x,(\mathcal{D})_{x})$. In view of (A.89), $L$ contains all the rectangles in
$\mathbb{R}^{k_{1}+k_{2}}$. In addition, we will make use of the following two
facts.

  1. a)

If $\mathcal{D}_{1},\mathcal{D}_{2}\in L$ and
$\mathcal{D}_{1}\subset\mathcal{D}_{2}$, then an application of similar
arguments to those in the proof for part 2) of Lemma 2 leads to

$\begin{split}\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\mathcal{D}_{2}\backslash\mathcal{D}_{1})&=\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\mathcal{D}_{2})-\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\mathcal{D}_{1})\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
\Big{(}h(x,(\mathcal{D}_{2})_{x})-h(x,(\mathcal{D}_{1})_{x})\Big{)}\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
h(x,(\mathcal{D}_{2})_{x}\backslash(\mathcal{D}_{1})_{x})\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
h(x,(\mathcal{D}_{2}\backslash\mathcal{D}_{1})_{x}),\end{split}$

which shows that $\mathcal{D}_{2}\backslash\mathcal{D}_{1}\in L$.

  2. b)

Assume that $\mathcal{D}_{i}\in L$ for each $n$ and
$\mathcal{D}_{i}\subset\mathcal{D}_{i+1}$. Then it follows from the continuity
of measure, the definition of $L$, and the monotone convergence theorem that

$\begin{split}\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\cup_{i=1}^{\infty}\mathcal{D}_{i})&=\lim_{n\rightarrow\infty}\mathbb{P}((\boldsymbol{X}^{{}^{T}},\boldsymbol{Y}^{{}^{T}})\in\mathcal{D}_{n})\\\
&=\lim_{n\rightarrow\infty}\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
h(x,(\mathcal{D}_{n})_{x})\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
\lim_{n\rightarrow\infty}h(x,(\mathcal{D}_{n})_{x})\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{X}}(dx)\
h(x,(\cup_{i=1}^{\infty}\mathcal{D}_{i})_{x}),\end{split}$

which shows that $\cup_{i=1}^{\infty}D_{i}\in L$.

Therefore, using the aforementioned facts, an application of Lemma 8 leads to
the conclusion in part 3) of Lemma 2. This completes the proof of of Lemma 2.

###  B.5 Lemma 3 and its proof

######  Lemma 3.

Let $\boldsymbol{X}_{1}$, $\boldsymbol{X}_{2}$, and $\boldsymbol{X}_{3}$ be
$k_{1}$-dimensional, $k_{2}$-dimensional, and $k_{3}$-dimensional random
vectors, respectively, such that for each $\mathcal{D}\in\mathcal{R}^{k_{3}}$,

$\mathbb{P}(\boldsymbol{X}_{3}\in\mathcal{D}\ |\
\boldsymbol{X}_{2})=\mathbb{P}(\boldsymbol{X}_{3}\in\mathcal{D}\ |\
\boldsymbol{X}_{2},\boldsymbol{X}_{1}).$ (A.90)

We define a transition kernel
$h:\mathbb{R}^{k_{2}}\times\mathcal{R}^{k_{3}}\longrightarrow[0,1]$ such that
for each $\mathcal{D}\in\mathcal{R}^{k_{3}}$,
$h(\boldsymbol{X}_{2},\mathcal{D})$ is a version of
$\mathbb{P}(\boldsymbol{X}_{3}\in\mathcal{D}\ |\ \boldsymbol{X}_{2})$. Then it
holds that

  1. 1)

For each $\mathcal{D}\in\mathcal{R}^{k_{3}}$,
$h(\boldsymbol{X}_{2},\mathcal{D})$ is a version of
$\mathbb{P}(\boldsymbol{X}_{3}\in\mathcal{D}\ |\
\boldsymbol{X}_{2},\boldsymbol{X}_{1})$.

  2. 2)

For each $\mathcal{D}\in\mathcal{R}^{k_{1}+k_{2}+k_{3}}$,
$h(\boldsymbol{X}_{2},\mathcal{D}_{(\boldsymbol{X}_{1},\boldsymbol{X}_{2})})\textnormal{
is a version of
}\mathbb{P}((\boldsymbol{X}_{1},\boldsymbol{X}_{2},\boldsymbol{X}_{3})\in\mathcal{D}\
|\ \boldsymbol{X}_{2},\\\ \boldsymbol{X}_{1})$.

Proof. We first show part 1). Since $h(\boldsymbol{X}_{2},\mathcal{D})$ is a
version of $\mathbb{P}(\boldsymbol{X}_{3}\in\mathcal{D}\ |\
\boldsymbol{X}_{2})$, $h(\boldsymbol{X}_{2},\mathcal{D})$ is
$\sigma(\boldsymbol{X}_{2})$-measurable, and hence
$\sigma(\boldsymbol{X}_{1},\boldsymbol{X}_{2})$-measurable. In conjunction
with (A.90) and the definition of conditional expectation, we see that
$h(\boldsymbol{X}_{2},\mathcal{D})$ a version of
$\mathbb{P}(\boldsymbol{X}_{3}\in\mathcal{D}\ |\
\boldsymbol{X}_{2},\boldsymbol{X}_{1})$. This yields the conclusion in part 1)
of Lemma 3. We then establish part 2). Let us first verify that
$h(\boldsymbol{X}_{2},\mathcal{D}_{\boldsymbol{X}_{1},\boldsymbol{X}_{2}})$ is
$\sigma(\boldsymbol{X}_{1},\boldsymbol{X}_{2})$-measurable for each
$\mathcal{D}\in\mathcal{R}^{k_{1}+k_{2}+k_{3}}$. We start with an observation
that for each $\mathcal{D}_{1}\in\mathcal{R}^{k_{1}}$,
$\mathcal{D}_{2}\in\mathcal{R}^{k_{2}}$,
$\mathcal{D}_{3}\in\mathcal{R}^{k_{3}}$, it holds that

$h(\boldsymbol{X}_{2},(\mathcal{D}_{1}\times\mathcal{D}_{2}\times\mathcal{D}_{3})_{\boldsymbol{X}_{1},\boldsymbol{X}_{2}})=h(\boldsymbol{X}_{2},\mathcal{D}_{3})\boldsymbol{1}_{\\{\boldsymbol{X}_{1}\in\mathcal{D}_{1}\\}}\boldsymbol{1}_{\\{\boldsymbol{X}_{2}\in\mathcal{D}_{2}\\}},$

which is $\sigma(\boldsymbol{X}_{1},\boldsymbol{X}_{2})$-measurable. This
shows that for each Borel rectangle $\mathcal{D}$,
$h(\boldsymbol{X}_{2},\mathcal{D}_{\boldsymbol{X}_{1},\boldsymbol{X}_{2}})$ is
$\sigma(\boldsymbol{X}_{1},\boldsymbol{X}_{2})$-measurable. In conjunction
with similar arguments to those in the proof of Lemma 2 in Section B.4, the
desired result follows.

Let $\mathcal{D}\in\mathcal{R}^{k_{1}+k_{2}+k_{3}}$ be given. Then we will
show that for each $\mathcal{B}\in\mathcal{R}^{k_{1}+k_{2}}$,

$\begin{split}&\int
h(\boldsymbol{X}_{2},\mathcal{D}_{\boldsymbol{X}_{1},\boldsymbol{X}_{2}})\
\boldsymbol{1}_{\\{(\boldsymbol{X}_{1},\boldsymbol{X}_{2})\in\mathcal{B}\\}}\
d\mathbb{P}\\\
&=\int\mathbb{P}((\boldsymbol{X}_{1},\boldsymbol{X}_{2},\boldsymbol{X}_{3})\in\mathcal{D}\
|\ \boldsymbol{X}_{2},\boldsymbol{X}_{1})\
\boldsymbol{1}_{\\{(\boldsymbol{X}_{1},\boldsymbol{X}_{2})\in\mathcal{B}\\}}\
d\mathbb{P}.\end{split}$ (A.91)

The left-hand side of (LABEL:le42) is well-defined since
$h(\boldsymbol{X}_{2},\mathcal{D}_{\boldsymbol{X}_{1},\boldsymbol{X}_{2}})$ is
measurable. To show (LABEL:le42), we again apply similar arguments to those in
the proof of Lemma 2. Specifically, let $L$ be the collection of sets in
$\mathcal{R}^{k_{1}+k_{2}+k_{3}}$ such that (LABEL:le42) holds. Since for each
$\mathcal{D}_{1}\in\mathcal{R}^{k_{1}}$,
$\mathcal{D}_{2}\in\mathcal{R}^{k_{2}}$,
$\mathcal{D}_{3}\in\mathcal{R}^{k_{3}}$, we have

$\begin{split}&\mathbb{P}((\boldsymbol{X}_{1},\boldsymbol{X}_{2},\boldsymbol{X}_{3})\in(\mathcal{D}_{1}\times\mathcal{D}_{2}\times\mathcal{D}_{3})\
|\ \boldsymbol{X}_{2},\boldsymbol{X}_{1})\\\
&=\mathbb{P}(\boldsymbol{X}_{3}\in\mathcal{D}_{3}\ |\
\boldsymbol{X}_{2},\boldsymbol{X}_{1})\boldsymbol{1}_{\\{\boldsymbol{X}_{1}\in\mathcal{D}_{1}\\}}\boldsymbol{1}_{\\{\boldsymbol{X}_{2}\in\mathcal{D}_{2}\\}}\\\
&=h(\boldsymbol{X}_{2},\mathcal{D}_{3})\boldsymbol{1}_{\\{\boldsymbol{X}_{1}\in\mathcal{D}_{1}\\}}\boldsymbol{1}_{\\{\boldsymbol{X}_{2}\in\mathcal{D}_{2}\\}}\\\
&=h(\boldsymbol{X}_{2},(\mathcal{D}_{1}\times\mathcal{D}_{2}\times\mathcal{D}_{3})_{\boldsymbol{X}_{1},\boldsymbol{X}_{2}}),\end{split}$

it holds that $L$ contains all Borel rectangles
$\mathcal{D}\in\mathcal{R}^{k_{1}+k_{2}+k_{3}}$. The remaining arguments
follow those in the proof of Lemma 2. Finally, since
$h(\boldsymbol{X}_{2},\mathcal{D}_{\boldsymbol{X}_{1},\boldsymbol{X}_{2}})$ is
$\sigma(\boldsymbol{X}_{1},\boldsymbol{X}_{2})$-measurable, by (LABEL:le42)
and the definition of conditional expectation, we can obtain the conclusion in
part 2) of Lemma 3. This concludes the proof of Lemma 3.

###  B.6 Lemma 4 and its proof

######  Lemma 4.

Let $\\{\boldsymbol{U}_{i}\\}$ and $\\{\boldsymbol{V}_{i}\\}$ be sequences of
$k_{1}$-dimensional and $k_{2}$-dimensional random vectors, respectively.
Assume that $(\boldsymbol{U}_{i},\boldsymbol{V}_{i})$’s are identically
distributed. Then there exists a transition kernel
$g:\mathbb{R}^{k_{1}}\times\mathcal{R}^{k_{2}}\longrightarrow[0,1]$ such that
for each $i$ and $\mathcal{D}\in\mathcal{R}^{k_{2}}$,
$g(\boldsymbol{U}_{i},\mathcal{D})$ is a version of
$\mathbb{P}(\boldsymbol{V}_{i}\in\mathcal{D}\ |\ \boldsymbol{U}_{i})$.

Proof. For each $(\boldsymbol{U}_{i},\boldsymbol{V}_{i})$, there exists a
transition kernel
$g_{i}:\mathbb{R}^{k_{1}}\times\mathcal{R}^{k_{2}}\longrightarrow[0,1]$ such
that for each $\mathcal{D}\in\mathcal{R}^{k_{2}}$,
$g_{i}(\boldsymbol{U}_{i},\mathcal{D})$ is a version of
$\mathbb{P}(\boldsymbol{V}_{i}\in\mathcal{D}\ |\ \boldsymbol{U}_{i})$; see,
for example, Theorem 4.1.18 of [16]. By this and the fact that
$\mu_{\boldsymbol{U}_{i}}=\mu_{\boldsymbol{U}_{1}}$ for each $i$, we have that
for each $\mathcal{A}\in\mathcal{R}^{k_{1}}$ and
$\mathcal{B}\in\mathcal{R}^{k_{2}}$,

$\begin{split}\mathbb{P}((\boldsymbol{U}_{i},\boldsymbol{V}_{i})\in\mathcal{A}\times\mathcal{B})&=\int_{\Omega}\boldsymbol{1}_{\\{\boldsymbol{U}_{i}\in\mathcal{A}\\}}\mathbb{P}(\boldsymbol{V}_{i}\in\mathcal{B}\
|\ \boldsymbol{U}_{i})d\mathbb{P}\\\
&=\int_{\Omega}\boldsymbol{1}_{\\{\boldsymbol{U}_{i}\in\mathcal{A}\\}}\
g_{i}(\boldsymbol{U}_{i},\mathcal{B})d\mathbb{P}\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{U}_{i}}(dx)\
\boldsymbol{1}_{\\{x\in\mathcal{A}\\}}\ g_{i}(x,\mathcal{B})\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{U}_{1}}(dx)\
\boldsymbol{1}_{\\{x\in\mathcal{A}\\}}\ g_{1}(x,\mathcal{B})\\\
&=\int_{\mathbb{R}^{k_{1}}}\mu_{\boldsymbol{U}_{i}}(dx)\
\boldsymbol{1}_{\\{x\in\mathcal{A}\\}}\ g_{1}(x,\mathcal{B})\\\
&=\int_{\Omega}\ \boldsymbol{1}_{\\{\boldsymbol{U}_{i}\in\mathcal{A}\\}}\
g_{1}(\boldsymbol{U}_{i},\mathcal{B})d\mathbb{P},\end{split}$ (A.92)

where $\Omega$ represents the underlying probability space, the third and last
equalities above follow from the change of variables formula, and the fourth
and fifth equalities above are due to the assumption of identical
distributions. Therefore, it follows from (A.92) and the definition of
conditional expectation that $g_{1}(\boldsymbol{U}_{i},\mathcal{D})$ is a
version of $\mathbb{P}(\boldsymbol{V}_{i}\in\mathcal{D}\ |\
\boldsymbol{U}_{i})$ for each $i$ and $\mathcal{D}\in\mathcal{R}^{k_{2}}$.
This completes the proof of Lemma 4.

###  B.7 Lemma 5 and its proof

Intuitively, Lemma 5 below extracts the total variation distance from a
difference of two integrals. The results are natural but the arguments are
somewhat delicate. We note that if the density functions $f_{X}$ and $f_{Y}$
exist, then it holds that

$\begin{split}\sup_{\mathcal{D}\in\mathcal{R}}|\int_{\mathcal{D}}f_{X}(z)dz-\int_{\mathcal{D}}f_{Y}(z)dz|&=\sup_{\mathcal{D}\in\mathcal{R}}\int_{\mathcal{D}}|f_{X}(z)-f_{Y}(z)|dz\\\
&=\frac{1}{2}\left\lVert\mu_{X}-\mu_{Y}\right\rVert_{TV}.\end{split}$

However, the same calculation does not apply directly to distributions because
the integral $\int|\mu_{X}(dz)-\mu_{Y}(dz)|$ is not well-defined due to the
two $dz$’s inside the integration. Lemma 5 provides valid arguments in such
situations.

######  Lemma 5.

1) Let $\mu$ and $\nu$ be two probability measures and
$f:\mathbb{R}^{K}\longrightarrow\mathbb{R}$ a measurable function with $0\leq
f\leq 1$. Then it holds that

$\sup_{\mathcal{D}\in\mathcal{R}^{K}}\Big{|}\int_{\mathcal{D}}f(x)\mu(dx)-\int_{\mathcal{D}}f(x)\nu(dx)\Big{|}\leq\frac{1}{2}\left\lVert\mu-\nu\right\rVert_{TV}.$

2) Let
$p(\cdot,\cdot):\mathbb{R}^{K_{1}}\times\mathcal{R}^{K_{2}}\longmapsto[0,1]$
be a transition kernel and $\mu$ a probability measure on
$\mathcal{R}^{K_{2}}$ such that for each $x\in\mathbb{R}^{K_{1}}$,
$p(x,\cdot)$ is dominated by $\mu$. Further let $\nu$ be a probability measure
on $\mathcal{R}^{K_{1}}$ and $0\leq f\leq 1$ a measurable function on
$\mathbb{R}^{K_{2}}$. Then it holds that

$\begin{split}&\sup_{\mathcal{D}\in\mathcal{R}^{K_{1}+K_{2}}}\Big{|}\int_{\mathcal{D}}\nu(dx_{1})p(x_{1},dx_{2})f(x_{2})-\int_{\mathcal{D}}\nu(dx_{1})\mu(dx_{2})f(x_{2})\Big{|}\\\
&\leq\frac{1}{2}\int_{\mathbb{R}^{K_{1}}}\nu(dx_{1})\left\lVert
p(x_{1},\cdot)-\mu(\cdot)\right\rVert_{TV}.\end{split}$ (A.93)

Proof. We start with proving the first assertion. By the definition of the
total variation distance, we can show that there exists some set
$\mathcal{A}\in\mathcal{R}^{K}$ such that

$\int_{\mathcal{A}}\mu(dx)-\int_{\mathcal{A}}\nu(dx)=\frac{1}{2}\left\lVert\mu-\nu\right\rVert_{TV}.$

Further it holds that for each $\mathcal{B}\in\mathcal{R}^{K}$ with
$\mathcal{B}\subset\mathcal{A}$,

$\int_{\mathcal{B}}\mu(dx)-\int_{\mathcal{B}}\nu(dx)\geq 0.$

Let us define $\mathcal{D}_{j}\coloneqq\mathcal{A}\cap\\{x:\frac{j-1}{M}\leq
f(x)<\frac{j}{M}\\}$ for $j=1,\cdots,M+1$ with $M$ some positive integer.
Denote by $\bar{f}$ a step function such that on $\mathcal{D}_{j}$, it holds
that $\bar{f}=\frac{j}{M}$. Then we can deduce that

$\begin{split}&\left|\int_{\cup_{j}\mathcal{D}_{j}}f(x)\mu(dx)-\int_{\cup_{j}\mathcal{D}_{j}}f(x)\nu(dx)-\left(\int_{\cup_{j}\mathcal{D}_{j}}\bar{f}(x)\mu(dx)-\int_{\cup_{j}\mathcal{D}_{j}}\bar{f}(x)\nu(dx)\right)\right|\\\
&\leq\int_{\cup_{j}\mathcal{D}_{j}}|f(x)-\bar{f}(x)|\mu(dx)+\int_{\cup_{j}\mathcal{D}_{j}}|f(x)-\bar{f}(x)|\nu(dx)\\\
&\leq\frac{1}{M}\left(\int_{\cup_{j}\mathcal{D}_{j}}\mu(dx)+\int_{\cup_{j}\mathcal{D}_{j}}\nu(dx)\right)\\\
&\leq\frac{2}{M}.\end{split}$ (A.94)

In light of the construction of $\mathcal{D}_{j}$’s and $\bar{f}$ above, we
have that

$\begin{split}\int_{\cup_{j}\mathcal{D}_{j}}&\bar{f}(x)\mu(dx)-\int_{\cup_{j}\mathcal{D}_{j}}\bar{f}(x)\nu(dx)=\sum_{j}\int_{\mathcal{D}_{j}}\bar{f}(x)\mu(dx)-\int_{\mathcal{D}_{j}}\bar{f}(x)\nu(dx)\\\
&=\sum_{j}\frac{j}{M}\left(\int_{\mathcal{D}_{j}}\mu(dx)-\int_{\mathcal{D}_{j}}\nu(dx)\right)\\\
&\leq\sum_{j}\left(\int_{\mathcal{D}_{j}}\mu(dx)-\int_{\mathcal{D}_{j}}\nu(dx)\right)\\\
&=\frac{1}{2}\left\lVert\mu-\nu\right\rVert_{TV}.\end{split}$ (A.95)

Then it follows from $\mathcal{A}=\cup_{j}\mathcal{D}_{j}$, (LABEL:sup5),
(A.95), and the fact that the positive integer $M$ can be arbitrarily large
that

$\left|\int_{\mathcal{A}}f(x)\mu(dx)-\int_{\mathcal{A}}f(x)\nu(dx)\right|\leq\frac{1}{2}\left\lVert\mu-\nu\right\rVert_{TV}.$
(A.96)

Using similar arguments as above, we can show that

$\sup_{\mathcal{D}\in\mathcal{R}^{K}}\left|\int_{\mathcal{D}}f(x)\mu(dx)-\int_{\mathcal{D}}f(x)\nu(dx)\right|=\left|\int_{\mathcal{A}}f(x)\mu(dx)-\int_{\mathcal{A}}f(x)\nu(dx)\right|.$
(A.97)

Therefore, combining (A.96) and (A.97) results in the desired conclusion in
part 1) of Lemma 5.

We now proceed with showing the second assertion. Since $\left\lVert
p(x_{1},\cdot)-\mu(\cdot)\right\rVert_{TV}$ is a measurable function in
$x_{1}$, the RHS of (LABEL:sup12) is well defined. Such claim can be
established using similar arguments to those in Theorem 5.2.2 of Durrett [16];
for simplicity, we omit the details. By the assumptions, let
$f_{1}(x_{1},x_{2})$ be the Radon–Nikodym derivative such that for each
$x_{1}\in\mathbb{R}^{K_{1}}$ and $\mathcal{D}\in\mathcal{R}^{K_{2}}$,

$\int_{\mathcal{D}}p(x_{1},dx_{2})=\int_{\mathcal{D}}f_{1}(x_{1},x_{2})\mu(dx_{2}).$

If $\mu$ is the Lebesgue measure and for each
$\mathcal{D}\in\mathcal{R}^{K_{2}}$, $p(\boldsymbol{X},\mathcal{D})$ is a
version of $\mathbb{P}(\boldsymbol{Y}\in\mathcal{D}|\boldsymbol{X})$ for some
random mappings $\boldsymbol{Y}$ and $\boldsymbol{X}$ dominated by the
Lebesgue measure, such a Radon–Nikodym derivative is usually referred to as
the conditional (on the density function of $\boldsymbol{X}$) probability
density function; for this, see also Example 4.1.6 in [16]. Thus, for each
$\mathcal{D}\in\mathcal{R}^{K_{1}+K_{2}}$, we have that

$\begin{split}&\Big{|}\int_{\mathcal{D}}\nu(dx_{1})p(x_{1},dx_{2})f(x_{2})-\int_{\mathcal{D}}\nu(dx_{1})\mu(dx_{2})f(x_{2})\Big{|}\\\
&=\Big{|}\int_{\mathcal{D}}\nu(dx_{1})(f_{1}(x_{1},x_{2})-1)\mu(dx_{2})f(x_{2})\Big{|}.\end{split}$
(A.98)

Next let us define $\mathcal{D}^{*}$ as

$\mathcal{D}^{*}\coloneqq\arg\sup_{\mathcal{D}\in\mathcal{R}^{K_{1}+K_{2}}}\int_{\mathcal{D}}\nu(dx_{1})(f_{1}(x_{1},x_{2})-1)\mu(dx_{2})f(x_{2})$
(A.99)

such that for each $\mathcal{A}\subset\mathcal{D}^{*}$, the integration of
(A.99) over $\mathcal{A}$ is nonnegative. Then by (LABEL:sup14), (A.99), and
the assumption of $0\leq f\leq 1$, it holds that

$\begin{split}&\sup_{\mathcal{D}\in\mathcal{R}^{K_{1}+K_{2}}}\Big{|}\int_{\mathcal{D}}\nu(dx_{1})p(x_{1},dx_{2})f(x_{2})-\int_{\mathcal{D}}\nu(dx_{1})\mu(dx_{2})f(x_{2})\Big{|}\\\
&\leq\int_{\mathcal{D}^{*}}\nu(dx_{1})(f_{1}(x_{1},x_{2})-1)\mu(dx_{2}).\end{split}$
(A.100)

Thus, it follows from the definition of $f_{1}$, Lemma 2, and the definition
of the total variation norm that

$\begin{split}&\int_{\mathcal{D}^{*}}\nu(dx_{1})(f_{1}(x_{1},x_{2})-1)\mu(dx_{2})\\\
&=\int_{\mathbb{R}^{K_{1}}}\nu(dx_{1})(p(x_{1},(D^{*})_{x_{1}})-\mu((D^{*})_{x_{1}}))\\\
&\leq\int_{\mathbb{R}^{K_{1}}}\nu(dx_{1})\frac{1}{2}\left\lVert
p(x_{1},\cdot)-\mu(\cdot)\right\rVert_{TV}.\end{split}$ (A.101)

Here Lemma 2 is applicable to the integral with $\mu$ since $\mu$ can be seen
as a transition kernel. Therefore, combining (LABEL:sup15) and (LABEL:sup16)
yields the conclusion in part 2) of Lemma 5. This concludes the proof of Lemma
5.

###  B.8 Lemma 6 and its proof

######  Lemma 6.

Let $\\{(\boldsymbol{Y}_{i},\boldsymbol{X}_{i})\\}$ and
$\\{(\boldsymbol{V}_{i},\boldsymbol{U}_{i})\\}$ be two sequences of
identically distributed random vectors with $\boldsymbol{Y}_{i}$ and
$\boldsymbol{X}_{i}$ $k_{1}$-dimensional and $k_{2}$-dimensional,
respectively. Assume that there exists some positive integer $K$ such that for
each $\mathcal{D}_{i}\in\mathcal{R}^{k_{2}}$ with $i=1,\cdots,K$,

$\begin{split}\mathbb{P}(\cap_{i=1}^{K}\\{\boldsymbol{X}_{i}\in\mathcal{D}_{i}\\}\
|\
\boldsymbol{Y}_{j},j=1,\cdots,K)&=\Pi_{i=1}^{K}\mathbb{P}(\boldsymbol{X}_{i}\in\mathcal{D}_{i}\
|\ \boldsymbol{Y}_{i}),\\\
\mathbb{P}(\cap_{i=1}^{K}\\{\boldsymbol{U}_{i}\in\mathcal{D}_{i}\\}\ |\
\boldsymbol{V}_{j},j=1,\cdots,K)&=\Pi_{i=1}^{K}\mathbb{P}(\boldsymbol{U}_{i}\in\mathcal{D}_{i}\
|\ \boldsymbol{V}_{i}).\end{split}$

Let us define
$\boldsymbol{Y}\coloneqq(\boldsymbol{Y}_{1}^{{}^{T}},\cdots,\boldsymbol{Y}_{K}^{{}^{T}})^{{}^{T}}$
and
$\boldsymbol{V}\coloneqq(\boldsymbol{V}_{1}^{{}^{T}},\cdots,\boldsymbol{V}_{K}^{{}^{T}})^{{}^{T}}$,
and $\boldsymbol{X}$ and $\boldsymbol{U}$ are defined similarly. Then it holds
that

  1. 1)

There exists a transition kernel
$h:\mathbb{R}^{Kk_{1}}\times\mathcal{R}^{Kk_{2}}\longrightarrow[0,1]$ such
that for each $\mathcal{B}\in\mathcal{R}^{Kk_{2}}$,
$h(\boldsymbol{Y},\mathcal{B})$ and $h(\boldsymbol{V},\mathcal{B})$ are
versions of $\mathbb{P}(\boldsymbol{X}\in\mathcal{B}\ |\ \boldsymbol{Y})$ and
$\mathbb{P}(\boldsymbol{U}\in\mathcal{B}\ |\ \boldsymbol{V})$, respectively.

  2. 2)

We have

$\begin{split}&\sup_{\mathcal{D}\in\mathcal{R}^{K(k_{1}+k_{2})}}\Big{|}\mathbb{P}\Big{(}(\boldsymbol{Y}^{{}^{T}},\boldsymbol{X}^{{}^{T}})\in\mathcal{D}\Big{)}-\mathbb{P}\Big{(}(\boldsymbol{V}^{{}^{T}},\boldsymbol{U}^{{}^{T}})\in\mathcal{D}\Big{)}\Big{|}\leq\frac{1}{2}\left\lVert\mu_{\boldsymbol{Y}}-\mu_{\boldsymbol{V}}\right\rVert_{TV},\end{split}$

where $\mu_{\boldsymbol{Y}}$ and $\mu_{\boldsymbol{V}}$ denote the
distributions of $\boldsymbol{Y}$ and $\boldsymbol{V}$, respectively.

Proof. By assumption, it holds that for each
$\mathcal{A}\in\mathcal{R}^{Kk_{1}}$ and $B_{i}\in\mathcal{R}^{k_{2}}$ with
$i=1,\cdots,K$,

$\begin{split}\mathbb{P}((\boldsymbol{Y}^{{}^{T}},\boldsymbol{X}^{{}^{T}})\in\mathcal{A}\times(\bigtimes_{i=1}^{K}B_{i}))&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{Y}\in\mathcal{A}\\}}\
\mathbb{P}\Big{(}\cap_{i=1}^{K}\\{\boldsymbol{X}_{i}\in B_{i}\\}\ |\
\boldsymbol{Y}\Big{)}\Big{]}\\\
&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{Y}\in\mathcal{A}\\}}\
\Pi_{i=1}^{K}\mathbb{P}\Big{(}\boldsymbol{X}_{i}\in B_{i}\ |\
\boldsymbol{Y}_{i}\Big{)}\Big{]},\end{split}$ (A.102)

where $\bigtimes_{i=1}^{K}B_{i}\coloneqq(B_{1},\cdots,B_{K})$. Similarly, we
can show that

$\begin{split}\mathbb{P}((\boldsymbol{V}^{{}^{T}},\boldsymbol{U}^{{}^{T}})\in\mathcal{A}\times(\bigtimes_{i=1}^{K}B_{i}))=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{V}\in\mathcal{A}\\}}\
\Pi_{i=1}^{K}\mathbb{P}\Big{(}\boldsymbol{U}_{i}\in B_{i}\ |\
\boldsymbol{V}_{i}\Big{)}\Big{]}.\end{split}$ (A.103)

Since $(\boldsymbol{Y}_{i}^{{}^{T}},\boldsymbol{X}_{i}^{{}^{T}})$ and
$(\boldsymbol{V}_{i}^{{}^{T}},\boldsymbol{U}_{i}^{{}^{T}})$ with $i\geq 1$ are
identically distributed, by Lemma 4 in Section B.6, there exists a transition
kernel $h_{1}$ such that for each $i$ and $\mathcal{D}\in\mathcal{R}^{k_{2}}$,
$h_{1}(\boldsymbol{Y}_{i},\mathcal{D})$ and
$h_{1}(\boldsymbol{V}_{i},\mathcal{D})$ are versions of
$\mathbb{P}(\boldsymbol{X}_{i}\in\mathcal{D}\ |\ \boldsymbol{Y}_{i})$ and
$\mathbb{P}(\boldsymbol{U}_{i}\in\mathcal{D}\ |\ \boldsymbol{V}_{i})$,
respectively.

Let us define
$h_{2}:\mathbb{R}^{Kk_{1}}\times\mathcal{R}^{Kk_{2}}\longrightarrow[0,1]$ such
that for each
$x=(x_{1}^{{}^{T}},\cdots,x_{K}^{{}^{T}})^{{}^{T}}\in\mathbb{R}^{Kk_{1}}$,
$h_{2}(x,\cdot)$ is a probability measure such that for each
$\mathcal{D}_{i}\in\mathcal{R}^{k_{2}}$ with $i=1,\cdots,K$,

$h_{2}(x,\bigtimes_{i=1}^{K}\mathcal{D}_{i})=\Pi_{i=1}^{K}h_{1}(x,\mathcal{D}_{i}).$
(A.104)

We make a useful claim below.

######  Claim 1.

$h_{2}$ is a transition kernel satisfying (A.104).

The proof of Claim 1 is provided in Section B.10. Then it follows from
(A.102), (A.103), and Claim 1 above that for each
$\mathcal{B}_{i}\in\mathcal{R}^{k_{2}}$ with $i=1,\cdots,K$,

$\begin{split}\mathbb{P}((\boldsymbol{Y}^{{}^{T}},\boldsymbol{X}^{{}^{T}})\in\mathcal{A}\times(\bigtimes_{i=1}^{K}B_{i}))&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{Y}\in\mathcal{A}\\}}\
\Pi_{i=1}^{K}\mathbb{P}\Big{(}\boldsymbol{X}_{i}\in B_{i}\ |\
\boldsymbol{Y}_{i}\Big{)}\Big{]}\\\
&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{Y}\in\mathcal{A}\\}}\
\Pi_{i=1}^{K}h_{1}\Big{(}\boldsymbol{Y}_{i},B_{i}\Big{)}\Big{]}\\\
&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{Y}\in\mathcal{A}\\}}\
h_{2}\Big{(}\boldsymbol{Y},\bigtimes_{i=1}^{K}B_{i}\Big{)}\Big{]},\end{split}$
(A.105)

and similarly,

$\begin{split}\mathbb{P}((\boldsymbol{V}^{{}^{T}},\boldsymbol{U}^{{}^{T}})\in\mathcal{A}\times(\bigtimes_{i=1}^{K}B_{i}))=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{V}\in\mathcal{A}\\}}\
h_{2}\Big{(}\boldsymbol{V},\bigtimes_{i=1}^{K}B_{i}\Big{)}\Big{]}.\end{split}$
(A.106)

By the construction of $h_{2}$, (A.105), (A.106), and Lemma 8 in Section B.12,
it holds that for each $\mathcal{A}\in\mathcal{R}^{Kk_{1}}$ and
$\mathcal{B}\in\mathcal{R}^{Kk_{2}}$,

$\begin{split}\mathbb{P}((\boldsymbol{Y}^{{}^{T}},\boldsymbol{X}^{{}^{T}})\in\mathcal{A}\times\mathcal{B}))&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{Y}\in\mathcal{A}\\}}\
h_{2}\Big{(}\boldsymbol{Y},\mathcal{B}\Big{)}\Big{]},\\\
\mathbb{P}((\boldsymbol{V}^{{}^{T}},\boldsymbol{U}^{{}^{T}})\in\mathcal{A}\times\mathcal{B}))&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{\boldsymbol{V}\in\mathcal{A}\\}}\
h_{2}\Big{(}\boldsymbol{V},\mathcal{B}\Big{)}\Big{]}.\end{split}$ (A.107)

Since $h_{2}$ is a transition kernel, we see that for each
$\mathcal{B}\in\mathcal{R}^{Kk_{2}}$, $h_{2}(\boldsymbol{Y},\mathcal{B})$ is
$\sigma(\boldsymbol{Y})$-measurable. Thus, in view of (A.107) we have that for
each $\mathcal{B}\in\mathcal{R}^{Kk_{2}}$, $h_{2}(\boldsymbol{Y},\mathcal{B})$
is a version of $\mathbb{P}(\boldsymbol{X}\in\mathcal{B}\ |\ \boldsymbol{Y})$.
A similar result for $\boldsymbol{V}$ and $\boldsymbol{U}$ can also be
obtained, which leads to the first assertion.

Finally, by Lemma 2 and Lemma 5 in Sections B.4 and B.7, respectively, we can
deduce that for each $\mathcal{D}\in\mathcal{R}^{K(k_{1}+k_{2})}$,

$\begin{split}&\Big{|}\mathbb{P}((\boldsymbol{Y}^{{}^{T}},\boldsymbol{X}^{{}^{T}})\in\mathcal{D})-\mathbb{P}((\boldsymbol{V}^{{}^{T}},\boldsymbol{U}^{{}^{T}})\in\mathcal{D})\Big{|}\\\
&=\Big{|}\int_{\mathbb{R}^{Kk_{1}}}h_{2}(x,\mathcal{D})\mu_{\boldsymbol{Y}}(dx)-\int_{\mathbb{R}^{Kk_{1}}}h_{2}(x,\mathcal{D})\mu_{\boldsymbol{V}}(dx)\Big{|}\\\
&\leq\frac{1}{2}\left\lVert\mu_{\boldsymbol{Y}}-\mu_{\boldsymbol{U}}\right\rVert_{TV},\end{split}$
(A.108)

which yields the second assertion. This completes the proof of Lemma 6.

###  B.9 Lemma 7 and its proof

Let $\widetilde{\boldsymbol{U}}$ and $\widetilde{\boldsymbol{V}}$ be the
knockoffs counterparts of $r\times c$ design matrices $\boldsymbol{U}$ and
$\boldsymbol{V}$, respectively. The corresponding response vectors are denoted
as $\boldsymbol{u}$ and $\boldsymbol{v}$, respectively. Lemma 7 below ensures
that the knockoffs matrix construction does not cause additional variation in
distribution in terms of the total variation distance.

######  Lemma 7.

Assume that 1) the rows of $(\boldsymbol{U},\widetilde{\boldsymbol{U}})$ and
$(\boldsymbol{V},\widetilde{\boldsymbol{V}})$ are identically distributed, 2)
the row vectors of $\widetilde{\boldsymbol{U}}$ are independent random vectors
conditional on $\boldsymbol{U}$, 3) the $i$th row of
$\widetilde{\boldsymbol{U}}$ is independent of the other rows of
$\boldsymbol{U}$ conditional on the $i$th row of $\boldsymbol{U}$, and 4)
$\boldsymbol{u}$ is independent of $\widetilde{\boldsymbol{U}}$ conditional on
$\boldsymbol{U}$. In addition, we assume the same for
$(\boldsymbol{v},\boldsymbol{V},\widetilde{\boldsymbol{V}})$. Then it holds
that

$\sup_{\mathcal{D}\in\mathcal{R}^{r(1+2c)}}\left|\mathbb{P}\Big{(}(\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D}\Big{)}-\mathbb{P}\Big{(}(\boldsymbol{v},\boldsymbol{V},\widetilde{\boldsymbol{V}})\in\mathcal{D}\Big{)}\right|\leq\frac{1}{2}\left\lVert\mu_{\boldsymbol{u,\boldsymbol{U}}}-\mu_{\boldsymbol{v},\boldsymbol{V}}\right\rVert_{TV}$

and

$\sup_{\mathcal{D}\in\mathcal{R}^{2rc}}\left|\mathbb{P}\Big{(}(\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D}\Big{)}-\mathbb{P}\Big{(}(\boldsymbol{V},\widetilde{\boldsymbol{V}})\in\mathcal{D}\Big{)}\right|\leq\frac{1}{2}\left\lVert\mu_{\boldsymbol{U}}-\mu_{\boldsymbol{V}}\right\rVert_{TV}.$

Proof. We start with showing the first assertion. Denote the $i$th rows of
$\widetilde{\boldsymbol{U}}$ and $\boldsymbol{U}$ by
$\widetilde{\boldsymbol{U}}_{i\bullet}$ and $\boldsymbol{U}_{i\bullet}$,
respectively. Then for each $\mathcal{D}_{i}\in\mathcal{R}^{r}$, we have that

$\begin{split}\mathbb{P}(\cap_{i=1}^{c}\\{\widetilde{\boldsymbol{U}}_{i\bullet}\in\mathcal{D}_{i}\\}|\boldsymbol{U})&=\Pi_{i=1}^{c}\mathbb{P}(\widetilde{\boldsymbol{U}}_{i\bullet}\in\mathcal{D}_{i}|\boldsymbol{U})\\\
&=\Pi_{i=1}^{c}\mathbb{P}(\widetilde{\boldsymbol{U}}_{i\bullet}\in\mathcal{D}_{i}|\boldsymbol{U}_{i\bullet}),\end{split}$

where the first equality follows from assumption 2) and the second one is due
to assumption 3). By this, an application of Lemma 6 shows that there exists a
transition kernel $h:\mathbb{R}^{rc}\times\mathcal{R}^{rc}\longmapsto[0,1]$
such that for each $\mathcal{D}\in\mathcal{R}^{rc}$,
$h(\boldsymbol{U},\mathcal{D})$ and $h(\boldsymbol{V},\mathcal{D})$ are
versions of $\mathbb{P}(\widetilde{\boldsymbol{U}}\in\mathcal{D}\ |\
\boldsymbol{U})$ and $\mathbb{P}(\widetilde{\boldsymbol{V}}\in\mathcal{D}\ |\
\boldsymbol{V})$, respectively.

We will make use of the claim below.

######  Claim 2.

For each $\mathcal{D}\in\mathcal{R}^{r(1+2c)}$, it holds that

$\begin{split}\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D})&=\int_{\mathbb{R}^{r(1+c)}}h(x_{2},\mathcal{D}_{x_{1},x_{2}})\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2}),\\\
\mathbb{P}((\boldsymbol{v},\boldsymbol{V},\widetilde{\boldsymbol{V}})\in\mathcal{D})&=\int_{\mathbb{R}^{r(1+c)}}h(x_{2},\mathcal{D}_{x_{1},x_{2}})\,\mu_{\boldsymbol{v},\boldsymbol{V}}(dx_{1}\times
dx_{2}),\end{split}$

where $x_{1}$ and $x_{2}$ denote $r$-dimensional and $(rc)$-dimensional
vectors, respectively.

The proof of Claim 2 is presented in Section B.11. Then it follows from Claim
2 above, Lemma 5, and the fact of $0\leq h\leq 1$ that for each
$\mathcal{D}\in\mathcal{R}^{r(1+2c)}$,

$\begin{split}&\Big{|}\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D})-\mathbb{P}((\boldsymbol{v},\boldsymbol{V},\widetilde{\boldsymbol{V}})\in\mathcal{D})\Big{|}\\\
&=\Big{|}\int_{\mathbb{R}^{r(1+c)}}h(x_{2},\mathcal{D}_{x_{1},x_{2}})\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2})-\int_{\mathbb{R}^{r(1+c)}}h(x_{2},\mathcal{D}_{x_{1},x_{2}})\,\mu_{\boldsymbol{v},\boldsymbol{V}}(dx_{1}\times
dx_{2})\Big{|}\\\
&\leq\frac{1}{2}\left\lVert\mu_{\boldsymbol{u},\boldsymbol{U}}-\mu_{\boldsymbol{v},\boldsymbol{V}}\right\rVert_{TV},\end{split}$

which leads to the conclusion in the first assertion. Using similar arguments
as above, we can establish the second assertion. This concludes the proof of
Lemma 7.

###  B.10 Proof of Claim 1

Note that given $x$, the existence of the probability measure $h_{2}(x,\cdot)$
is guaranteed by an application of Theorem 1.7.1 in [16]. Let $L$ be the
collection of sets such that if $\mathcal{D}\in L$, $h_{2}(x,\mathcal{D})$ is
a measurable function of $x$. We will make three observations due to the
definition of $h_{2}$. (i) $L$ contains all Borel rectangles since the product
of measurable functions is still a measurable function. (ii) For
$\mathcal{D}_{1},\mathcal{D}_{2}\in L$ with
$\mathcal{D}_{1}\subset\mathcal{D}_{2}$, it holds that
$h_{2}(\cdot,\mathcal{D}_{2}\backslash\mathcal{D}_{1})$ is measurable and
hence $\mathcal{D}_{2}\backslash\mathcal{D}_{1}\in L$. (iii) For
$\mathcal{D}_{i}\subset\mathcal{D}_{i+1}$, $\mathcal{D}_{i}\in L$, and
$\mathcal{D}\coloneqq\cup_{i=1}^{\infty}\mathcal{D}_{i}$, we have

$h_{2}(\cdot,\mathcal{D})=\sup_{i}h_{2}(\cdot,\mathcal{D}_{i})$

which is measurable, and thus $\mathcal{D}\in L$. Therefore, it follows from
these facts and Lemma 8 in Section B.12 that for each
$\mathcal{D}\in\mathcal{R}^{Kk_{2}}$, $h_{2}(\cdot,\mathcal{D})$ is
measurable. This completes the proof of Claim 1.

###  B.11 Proof of Claim 2

Let us first show the first assertion. For each Borel rectangle
$\mathcal{A}_{1}\times\mathcal{A}_{2}\times\mathcal{A}_{3}\in\mathcal{R}^{r}\times\mathcal{R}^{rc}\times\mathcal{R}^{rc}$,
it holds that

$\begin{split}&\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{A}_{1}\times\mathcal{A}_{2}\times\mathcal{A}_{3})=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{(\boldsymbol{u},\boldsymbol{U})\in\mathcal{A}_{1}\times\mathcal{A}_{2}\\}}\
\mathbb{P}(\widetilde{\boldsymbol{U}}\in\mathcal{A}_{3}\ |\
\boldsymbol{u},\boldsymbol{U})\Big{]}\\\
&=\mathbb{E}\Big{[}\boldsymbol{1}_{\\{(\boldsymbol{u},\boldsymbol{U})\in\mathcal{A}_{1}\times\mathcal{A}_{2}\\}}\
\mathbb{P}(\widetilde{\boldsymbol{U}}\in\mathcal{A}_{3}\ |\
\boldsymbol{U})\Big{]}\\\
&=\int_{\mathbb{R}^{r(1+c)}}h(x_{2},(\mathcal{A}_{1}\times\mathcal{A}_{2}\times\mathcal{A}_{3})_{x_{1},x_{2}})\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2}),\end{split}$

where the second equality is due to the assumption of Lemma 7 and the last
equality is because of the definition of $h$. Let $L$ be a collection of sets
in $\mathcal{R}^{r(1+2c)}$ such that if $\mathcal{D}\in L$,

$\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D})=\int_{\mathbb{R}^{r(1+c)}}h(x_{2},\mathcal{D}_{x_{1},x_{2}})\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2}).$

Then we can see that $L$ contains all Borel rectangles which collection is a
$\pi$-system.

Let us make a few observations below.

  1. 1)

The set $L$ contains $\mathbb{R}^{r(1+2c)}$.

  2. 2)

If $\mathcal{D}_{1},\mathcal{D}_{2}\in L$ and
$\mathcal{D}_{1}\subset\mathcal{D}_{2}$, then some basic measure and
integration operations as well as the operation of $\mathcal{D}_{x}$ give that

$\begin{split}&\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D}_{2}\backslash\mathcal{D}_{1})=\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D}_{2})-\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D}_{1})\\\
&=\int_{\mathbb{R}^{r(1+c)}}\Big{(}h(x_{2},(\mathcal{D}_{2})_{x_{1},x_{2}})-h(x_{2},(\mathcal{D}_{1})_{x_{1},x_{2}})\Big{)}\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2})\\\
&=\int_{\mathbb{R}^{r(1+c)}}h(x_{2},(\mathcal{D}_{2}\backslash\mathcal{D}_{1})_{x_{1},x_{2}})\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2}),\end{split}$

which leads to $\mathcal{D}_{2}\backslash\mathcal{D}_{1}\in L$.

  3. 3)

If $\mathcal{D}_{n}\in L$ and $\mathcal{D}_{n}\subset\mathcal{D}_{n+1}$, then
it follows from the continuity of measure and the monotone convergence theorem
that

$\begin{split}&\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\cup_{n}\mathcal{D}_{n})=\lim_{n}\mathbb{P}((\boldsymbol{u},\boldsymbol{U},\widetilde{\boldsymbol{U}})\in\mathcal{D}_{n})\\\
&=\lim_{n}\int_{\mathbb{R}^{r(1+c)}}h(x_{2},(\mathcal{D}_{n})_{x_{1},x_{2}})\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2})\\\
&=\int_{\mathbb{R}^{r(1+c)}}h(x_{2},(\cup_{n}\mathcal{D}_{n})_{x_{1},x_{2}})\,\mu_{\boldsymbol{u},\boldsymbol{U}}(dx_{1}\times
dx_{2}),\end{split}$

which results in $\cup_{n}\mathcal{D}_{n}\in L$.

Therefore, using the aforementioned facts, an application of Lemma 8 in
Section B.12 yields the conclusion in the first assertion. The conclusion in
the second assertion can be shown in a similar fashion, which concludes the
proof of Claim 2.

###  B.12 Lemma 8

######  Definition ($\pi$-system and $\lambda$-system).

A collection of sets $P$ is said to be a $\pi$-system if for any $A,B\in P$,
$A\cap B\in P$. A collection $L$ of sets in $\Omega$ is said to be a
$\lambda$-system if

  1. 1)

$\Omega\in L$;

  2. 2)

If $A\subset B$ and $A,B\in L$, then $B\backslash A\in L$;

  3. 3)

If $A_{n}\in L$ and $A_{n}\subset A_{n+1}$, then $\cup_{n}A_{n}\in L$.

######  Lemma 8 ($\pi-\lambda$ Theorem in [16]).

If $P$ is a $\pi$-system and $L$ is a $\lambda$-system that contains $P$, then
the smallest $\sigma$-algebra containing $P$ is also contained in $L$.