$I\\!\\!E_{\theta}\left[\left(\overline{U}_{t}\right)^{2}\right]=I\\!\\!E_{\theta}\left[\left(\overline{Q^{N}M_{t}}\right)^{2}\right]+H^{N}_{\theta,t},$

where

$H^{N}_{\theta,t}=2I\\!\\!E_{\theta}\left[\overline{Q^{N}M_{t}}\
\overline{(Q^{N}-I_{N})(X_{0}-X_{t})}\right]+I\\!\\!E_{\theta}\left[\left(\overline{(Q^{N}-I_{N})(X_{0}-X_{t})}\right)^{2}\right].$

Hence, to conclude the proof, it remains to show that we can write
$H^{N}_{\theta,t}=S^{N}_{\theta,t}+r^{N}_{\theta,t}$ where
$|r^{N}_{\theta,t}|\leq K(1-\lambda)^{t}N^{-1}$ and $S^{N}_{\theta,t}$ is such
that $|S^{N}_{\theta,2t}-S^{N}_{\theta,t}|\leq K(1-\lambda)^{t}N^{-1}$, for
all $t\geq 1$ and $N\geq 1$. To see that, first of all, observe that

$\displaystyle\overline{(Q^{N}-I_{N})(X_{0}-X_{t})}$
$\displaystyle=N^{-1}\sum_{i=1}^{N}\sum_{j=1}^{N}(Q^{N}(i,j)-I_{N}(i,j))(X_{j,0}-X_{j,t})$
$\displaystyle=N^{-1}\sum_{j=1}^{N}(X_{j,0}-X_{j,t})(c^{N}_{j}-1),$

where $c^{N}=(Q^{N})^{\intercal}1_{N}$, so that

$\displaystyle\left(\overline{(Q^{N}-I_{N})(X_{0}-X_{t})}\right)^{2}$
$\displaystyle=N^{-2}\sum_{i=1}^{N}\sum_{j=1}^{N}(c^{N}_{i}-1)(c^{N}_{j}-1)(X_{i,0}-X_{i,t})(X_{j,0}-X_{j,t})$
$\displaystyle=N^{-2}\sum_{i=1}^{N}\sum_{j=1}^{N}(c^{N}_{i}-1)(c^{N}_{j}-1)(Y_{i,0}-Y_{i,t})(Y_{j,0}-Y_{j,t}),$

where the second equality follows from the definition of
$Y_{i,t}=X_{i,t}-I\\!\\!E_{\theta}[X_{i,t}]$ and the stationarity of the
process $(X_{t})_{t\in\mathbb{Z}}$. Using the stationarity once more and the
above equation, we then deduce that

$I\\!\\!E_{\theta}\left[\left(\overline{(Q^{N}-I_{N})(X_{0}-X_{t})}\right)^{2}\right]=2N^{-2}\sum_{i=1}^{N}\sum_{j=1}^{N}(c^{N}_{i}-1)(c^{N}_{j}-1)\left[\operatorname{\mathop{Cov}}_{\theta}(Y_{i,0},Y_{j,0})\right.\\\
\left.-\operatorname{\mathop{Cov}}_{\theta}(Y_{i,0},Y_{j,t})\right],$

so that by applying Lemma 3.8 and using Inequality (42), we obtain that

$I\\!\\!E_{\theta}\left[\left(\overline{(Q^{N}-I_{N})(X_{0}-X_{t})}\right)^{2}\right]=\frac{2}{N^{2}}\sum_{i=1}^{N}\sum_{j=1}^{N}(c^{N}_{i}-1)(c^{N}_{j}-1)\operatorname{\mathop{Cov}}_{\theta}(Y_{i,0},Y_{j,0})+r^{N}_{\theta,t},$

where $|r^{N}_{\theta,t}|\leq K(1-\lambda)^{t}N^{-1}$.

Now, proceeding similarly as above, one can check that

$\displaystyle I\\!\\!E_{\theta}\left[\overline{Q^{N}M_{t}}\
\overline{(Q^{N}-I_{N})(X_{0}-X_{t})}\right]=N^{-2}\sum_{i=1}^{N}\sum_{j=1}^{N}c^{N}_{i}(c^{N}_{j}-1)I\\!\\!E_{\theta}\left[M_{i,t}(X_{j,0}-X_{j,t})\right].$

For each $1\leq i,j\leq N$ and $t\geq 1$, denote
$S^{N}_{\theta,t}(i,j)=I\\!\\!E_{\theta}\left[M_{i,t}(X_{j,0}-X_{j,t})\right].$
We claim that there exists a constant $K>0$ such that for all $1\leq i,j\leq
N$ and $t\geq 1$, the following inequality holds: $|S^{N}_{\theta,t}(i,j)|\leq
K(\delta_{ij}+N^{-1})$ and

$|S^{N}_{\theta,2t}(i,j)-S^{N}_{\theta,t}(i,j)|\leq K(1-\lambda)^{t}N^{-1}.$

Once this claim is proved, using Inequality (42) and Lemma 3.8 it is immediate
to check that $H^{N}_{\theta,t}=S^{N}_{\theta,t}+r^{N}_{\theta,t}$, where

$S^{N}_{\theta,t}=N^{-2}\sum_{i=1}^{N}\sum_{j=1}^{N}\left[c^{N}_{i}(c^{N}_{j}-1)S^{N}_{\theta,t}(i,j)+2(c^{N}_{i}-1)(c^{N}_{j}-1)\text{Cov}_{\theta}(X_{i,0},Y_{j,0})\right]$

satisfies $|S^{N}_{\theta,2t}-S^{N}_{\theta,t}|\leq K(1-\lambda)^{t}N^{-1}$
and $|S^{N}_{\theta,t}|\leq KN^{-1}$, implying the result.

To prove the claim above, we first write

$S^{N}_{\theta,t}(i,j)=I\\!\\!E_{\theta}\left[M_{i,t}(X_{j,0}-X_{j,t})\right]=\sum_{s=1}^{t}I\\!\\!E_{\theta}\left[(X_{i,s}-p_{\theta,i}(X_{s-1}))(X_{j,0}-X_{j,t})\right]$

and then use the fact that

$I\\!\\!E_{\theta}\left[(X_{i,s}-p_{\theta,i}(X_{s-1}))X_{j,0}\right]=I\\!\\!E_{\theta}\left[X_{j,0}I\\!\\!E_{\theta}\left[(X_{i,s}-p_{\theta,i}(X_{s-1}))|\mathcal{F}_{s-1}\right]\right]=0$

to deduce that

$I\\!\\!E_{\theta}\left[M_{i,t}(X_{j,0}-X_{j,t})\right]=\sum_{s=1}^{t}\left(I\\!\\!E_{\theta}\left[p_{\theta,i}(X_{s-1})X_{j,t}\right]-I\\!\\!E_{\theta}\left[X_{i,s}X_{j,t}\right]\right).$

Next, we use (1) and the stationarity to obtain that

$\sum_{s=1}^{t}I\\!\\!E_{\theta}\left[p_{\theta,i}(X_{s-1})X_{j,t}\right]=tm^{N}_{j}(\mu-(1-\lambda)L^{N,\bullet-}_{i})\\\
+(1-\lambda)\sum_{k=1}^{N}A^{N}(i,k)\sum_{s=1}^{t}I\\!\\!E_{\theta}\left[X_{k,s-1}X_{j,t}\right].$

Hence, using that
$I\\!\\!E_{\theta}\left[X_{k,s-1}X_{j,t}\right]=\operatorname{\mathop{Cov}}_{\theta}(X_{k,s-1},X_{j,t})+m_{k}^{N}m_{j}^{N}$
and the stationarity once more, one can check that

$\sum_{s=1}^{t}I\\!\\!E_{\theta}\left[p_{\theta,i}(X_{s-1})X_{j,t}\right]=tm^{N}_{j}(\mu-(1-\lambda)L^{N,\bullet-}_{i}+(1-\lambda)(A^{N}m^{N})_{i})\\\
+(1-\lambda)\sum_{k=1}^{N}A^{N}(i,k)\sum_{s=1}^{t}\operatorname{\mathop{Cov}}_{\theta}\left(X_{k,0},X_{j,s}\right),$

and similarly that

$\sum_{s=1}^{t}I\\!\\!E_{\theta}\left[X_{i,s}X_{j,t}\right]=tm_{i}^{N}m^{N}_{j}-\sum_{s=0}^{t-1}\operatorname{\mathop{Cov}}_{\theta}\left[X_{i,0}X_{j,s}\right].$

Therefore, combining the last two equations and using (12), we obtain that

$S^{N}_{\theta,t}(i,j)=(1-\lambda)\sum_{k=1}^{N}A^{N}(i,k)\sum_{s=1}^{t}\operatorname{\mathop{Cov}}_{\theta}\left(X_{k,0},X_{j,s}\right)+\sum_{s=0}^{t-1}\operatorname{\mathop{Cov}}_{\theta}\left(X_{i,0},X_{j,s}\right),$

so that Lemma 3.8 allows us to conclude that

$\left|S^{N}_{\theta,2t}(i,j)-S^{N}_{\theta,t}(i,j)\right|\leq\\\
\leq(1-\lambda)\sum_{k=1}^{N}|A^{N}(i,k)|\sum_{s=t+1}^{2t}|\operatorname{\mathop{Cov}}_{\theta}\left(X_{k,0},X_{j,s}\right)|+\sum_{s=t}^{2t-1}|\operatorname{\mathop{Cov}}_{\theta}\left(X_{i,0},X_{j,s}\right)|\\\
\leq\frac{K(1-\lambda)^{t}}{N}\left(\sum_{s=0}^{t-1}(1-\lambda)^{s}\right)\left[(1-\lambda)^{2}N|||A^{N}|||_{\infty}+1\right]\leq\frac{K(1-\lambda)^{t}}{N}.$

Finally, using once more Lemma 3.8, one can easily check that
$|S^{N}_{\theta,t}(i,j)|\leq K(\delta_{ij}+N^{-1})$ proving the claim. ∎

The next result will be important to find the leading term of
$I\\!\\!E_{\theta}\left[\left(\overline{Q^{N}M_{t}}\right)^{2}\right]$.

######  Lemma 6.3.

For all $t\geq 1,$ the following equality holds.

$I\\!\\!E_{\theta}\left[(X_{t}-p_{\theta}(X_{t-1}))^{2}\right]=m^{N}-(m^{N})^{2}+r^{N}_{\theta},$

where $\|r^{N}_{\theta}\|_{\infty}\leq KN^{-1}$ for some constant $K>0$
depending only on $\lambda$.

######  Remark 6.4.

Note that $X_{t}$, $p_{\theta}(X_{t-1})$ and $m^{N}$ are vectors so that the
square terms in Lemma 6.3 have to be interpreted in terms of the Hadamard
product of vectors. Also, the remainder term $r^{N}_{\theta}$ does not depend
on time $t$ because the process $(X_{t})_{t\in\mathbb{Z}}$ is assumed to be
stationary.

###### Proof.

First, observe that

$\displaystyle
I\\!\\!E_{\theta}\left[(X_{i,t}-p_{i,\theta}(X_{t-1}))^{2}\right]$
$\displaystyle=$ $\displaystyle
I\\!\\!E_{\theta}\left[X_{i,t}\right]-I\\!\\!E_{\theta}\left[p^{2}_{i,\theta}(X_{t-1})\right]$
$\displaystyle=$ $\displaystyle
m^{N}_{i}-I\\!\\!E_{\theta}\left[p^{2}_{i,\theta}(X_{0})\right],$

where the second equality follows by the stationarity. Now, from (1), we have
that

$p_{i,\theta}(X_{0})=\left(\mu-(1-\lambda)L^{N,\bullet-}_{i}\right)+(1-\lambda)(A^{N}X_{0})_{i},$

so that

$I\\!\\!E_{\theta}\left[p^{2}_{i,\theta}(X_{0})\right]=(\mu-(1-\lambda)L^{N,\bullet-}_{i})^{2}+2(\mu-(1-\lambda)L^{N,\bullet-}_{i})(1-\lambda)(A^{N}I\\!\\!E_{\theta}\left[X_{0}\right])_{i}\\\
+(1-\lambda)^{2}I\\!\\!E_{\theta}\left[((A^{N}X_{0})_{i})^{2}\right].$ (79)

Next, notice that

$I\\!\\!E_{\theta}\left[((A^{N}_{\theta}X_{0})_{i})^{2}\right]=\sum_{j=1}^{N}\sum_{k=1}^{N}A^{N}(i,j)A^{N}(i,k)I\\!\\!E_{\theta}\left[X_{k,0}X_{j,0}\right],$

so that using that
$I\\!\\!E_{\theta}\left[X_{k,0}X_{j,0}\right]=m^{N}_{j}m^{N}_{k}+\operatorname{\mathop{Cov}}_{\theta}(X_{k,0},X_{j,0})$,
we deduce that

$I\\!\\!E_{\theta}\left[((A^{N}X_{0})_{i})^{2}\right]=\left(\sum_{j=1}^{N}A^{N}(i,j)m^{N}_{j}\right)^{2}+\widetilde{r}^{N}_{\theta,i}$
(80)

where

$\widetilde{r}^{N}_{\theta,i}=\sum_{j=1}^{N}\sum_{k=1}^{N}A^{N}(i,j)A^{N}(i,k)\operatorname{\mathop{Cov}}_{\theta}(X_{k,0},X_{j,0}).$

Using the fact that $|A^{N}(i,j)|\leq 1/N$ and Lemma 3.8, we have

$|\widetilde{r}^{N}_{\theta,i}|\leq
N^{-2}\sum_{j=1}^{N}\sum_{k=1}^{N}|\operatorname{\mathop{Cov}}_{\theta}[X_{k,0},X_{j,0}]|\leq
N^{-2}(N+N^{2}K(1-\lambda)N^{-1})\leq KN^{-1}.$

Finally, recalling that $m^{N}=I\\!\\!E_{\theta}\left[X_{0}\right]$, it
follows from (79) and (80) that

$I\\!\\!E_{\theta}\left[p^{2}_{i,\theta}(X_{0})\right]=\left[(\mu-(1-\lambda)L^{N,\bullet-}_{i})+(1-\lambda)(A^{N}m^{N})_{i}\right]^{2}+(1-\lambda)^{2}\widetilde{r}^{N}_{\theta,i}.$

The result then follows from equation (13). ∎

As a consequence of Lemma 6.3, we will now find the leading term of
$I\\!\\!E_{\theta}\left[\left(\overline{Q^{N}M_{t}}\right)^{2}\right]$.

######  Lemma 6.5.

For any $t\geq 1$ and $N\geq 1$,

$I\\!\\!E_{\theta}\left[\Big{|}\overline{Q^{N}M_{t}}\Big{|}^{2}\right]=\frac{t}{N}w^{N}_{\infty}+r^{N}_{\theta,t},$

where $w^{N}_{\infty}$ is defined in (73) and $r^{N}_{\theta,t}$ is such that
$|r^{N}_{\theta,t}|\leq KtN^{-2}$ for some $K$ depending only on $\lambda$.

###### Proof.

We start observing that

$\overline{Q^{N}M_{t}}=N^{-1}\sum_{i=1}^{N}\sum_{j=1}^{N}Q^{N}(i,j)M_{j,t}=N^{-1}\sum_{j=1}^{N}c^{N}_{j}M_{j,t},$

and then we use that for all $s\neq h$ and $1\leq j,k\leq N$,

$I\\!\\!E_{\theta}\left[(X_{i,s}-p_{i,\theta}(X_{s-1}))(X_{k,h}-p_{k,\theta}(X_{h-1}))\right]=0,$

to deduce that

$\displaystyle
I\\!\\!E_{\theta}\left[\Big{|}\overline{Q^{N}M_{t}}\Big{|}^{2}\right]$
$\displaystyle=$ $\displaystyle
N^{-2}\sum_{j=1}^{N}\sum_{k=1}^{N}c^{N}_{j}c^{N}_{k}I\\!\\!E_{\theta}\left[M_{j,t}M_{k,t}\right]$
$\displaystyle=$ $\displaystyle
N^{-2}\sum_{j=1}^{N}(c^{N}_{j})^{2}\sum_{s=1}^{t}I\\!\\!E_{\theta}\left[\left(X_{j,s}-p_{j,\theta}(X^{N}_{s-1})\right)^{2}\right].$

Hence, it follows from Lemma 6.3 that

$\displaystyle
I\\!\\!E_{\theta}\left[\Big{|}\overline{Q^{N}M_{t}}\Big{|}^{2}\right]$
$\displaystyle=$ $\displaystyle
N^{-2}\sum_{j=1}^{N}(c^{N}_{j})^{2}\sum_{s=1}^{t}I\\!\\!E_{\theta}\left[\left(X_{j,s}-p_{j,\theta}(X_{s-1})\right)^{2}\right]$
$\displaystyle=$ $\displaystyle
tN^{-2}\sum_{j=1}^{N}(c^{N}_{j})^{2}m^{N}_{j}(1-m^{N}_{j})+tN^{-2}\langle(c^{N})^{2},r^{N}_{\theta}\rangle$
$\displaystyle=$ $\displaystyle
tN^{-1}w^{N}_{\infty}+tN^{-2}\langle(c^{N})^{2},r^{N}_{\theta}\rangle.$

Finally, by applying Hölder inequality and then using that $\|v\|_{1}\leq
N\|v\|_{\infty}$ for $v\in\mathbb{R}^{N}$, we obtain

$\langle(c^{N})^{2},r^{N}_{\theta}\rangle\leq\|(c^{N})^{2}\|_{\infty}\|r^{N}_{\theta}\|_{1}\leq\|(c^{N})^{2}\|_{\infty}N\|r^{N}_{\theta}\|_{\infty},$

and the result follows from inequality (42) and Lemma 6.3. ∎

For later use, let us mention the following immediate corollary of Lemmas 6.2
and 6.5.

######  Lemma 6.6.

There exists a constant $K>0$ depending only on $\lambda$ such that for all
$t\geq 1$ and $N\geq 1$,

$I\\!\\!E_{\theta}\left[\left(\overline{U}_{t}\right)^{2}\right]\leq
KtN^{-1}.$

###### Proof.

First, observe that $w^{N}_{\infty}$ defined in (73) satisfies
$|w^{N}_{\infty}|\leq\lambda^{-2}/4$ by Inequality (42), and then combine this
fact with Lemmas 6.2 and 6.5. ∎

We are now in position to deal with the term $D^{N,3}_{T}$ defined in (77).

######  Proposition 6.7.

There exists a constant $K>0$ depending only on $\lambda$ such that for all
$N\geq 1$, $T\geq 2$ and $1\leq\Delta\leq\lfloor T/2\rfloor$,

$I\\!\\!E_{\theta}\left[D^{N,3}_{T}\right]\leq
K\left(\frac{1}{N}+\frac{(1-\lambda)^{\Delta}}{\Delta}+\frac{\Delta}{T}\right).$

###### Proof.

First, observe that by stationarity,

$D^{N,3}_{T}=\Big{|}\frac{2N}{T}\lfloor T/{2\Delta}\rfloor
I\\!\\!E_{\theta}\left[(\overline{U}_{2\Delta})^{2}\right]-\frac{N}{T}\lfloor
T/{\Delta}\rfloor
I\\!\\!E_{\theta}\left[(\overline{U}_{\Delta})^{2}\right]-w^{N}_{\infty}\Big{|}.$

Then, by using the triangle inequality, one can check that

$\Big{|}\frac{2N}{T}\lfloor T/{2\Delta}\rfloor
I\\!\\!E_{\theta}\left[(\overline{U}_{2\Delta})^{2}\right]-\frac{N}{T}\lfloor
T/{\Delta}\rfloor
I\\!\\!E_{\theta}\left[(\overline{U}_{\Delta})^{2}\right]-w^{N}_{\infty}\Big{|}\\\
\leq\Big{|}\frac{N}{\Delta}\left(I\\!\\!E_{\theta}\left[(\overline{U}_{2\Delta})^{2}\right]-I\\!\\!E_{\theta}\left[(\overline{U}_{\Delta})^{2}\right]\right)-w^{N}_{\infty}\Big{|}+\frac{2N}{T}\left(I\\!\\!E_{\theta}\left[(\overline{U}_{2\Delta})^{2}\right]+I\\!\\!E_{\theta}\left[(\overline{U}_{\Delta})^{2}\right]\right).$

Then Lemmas 6.2, 6.5 and 6.6 imply the result. ∎

It remains to deal with $D^{N,2}_{\Delta,T}$ and $D^{N,2}_{2\Delta,T}$. To
that end, we shall use two additional lemmas. The first one is the following.

######  Lemma 6.8.

There exists a constant $K>0$ depending only on $\lambda$ such that for all
$N\geq 1$, $\Delta\geq 1$ and $t\geq 2,$

$\operatorname{\mathop{Cov}}_{\theta}\left[\left(\overline{U}_{\Delta}\right)^{2},\left(\overline{U}_{t\Delta}-\overline{U}_{(t-1)\Delta}\right)^{2}\right]\leq
KN^{-2}(1-\lambda)^{(t-2)\Delta}.$

###### Proof.

First, use that $\overline{U}_{t}=N^{-1}\sum_{i=1}^{N}\sum_{s=1}^{t}Y_{{i,s}}$
to deduce that

$\operatorname{\mathop{Cov}}_{\theta}\left[\left(\overline{U}_{\Delta}\right)^{2},\left(\overline{U}_{t\Delta}-\overline{U}_{(t-1)\Delta}\right)^{2}\right]=N^{-4}\sum_{z_{1}\in
F_{1}}\sum_{z_{2}\in F_{1}}\sum_{z_{3}\in F_{t}}\sum_{z_{4}\in
F_{t}}\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}},Y_{z_{3}}Y_{z_{4}}\right],$

where $z_{k}=(i_{k},t_{k})$ for $k\in\\{1,\ldots,4\\}$, and
$F_{s}:=\\{1,\ldots,N\\}\times\\{(s-1)\Delta+1,\ldots,s\Delta\\}$ for $s\geq
1$. In the rest of the proof, we shall denote ${\bf
z}=(z_{1},z_{2},z_{3},z_{4})$ and $B({\bf
z})=\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}},Y_{z_{3}}Y_{z_{4}}\right].$
Notice that the covariance function $B({\bf z})$ is symmetric with respect to
$z_{1}$ and $z_{2}$, as well as with respect to $z_{3}$ and $z_{4}$. For each
$p\in\\{1,\ldots,6\\}$ and $s\geq 1$, we denote $\mathcal{C}_{p,s}$ the set of
all vectors ${\bf z}\in F_{1}\times F_{1}\times F_{s}\times F_{s}$ satisfying
the conditions of item $p$ of Lemma 3.9 and such that $t_{1}\leq t_{2}$ and
$t_{3}\leq t_{4}$. With this notation, one can check using the symmetry of the
covariance function that the following inequality holds:

$\operatorname{\mathop{Cov}}_{\theta}\left[\left(\overline{U}_{\Delta}\right)^{2},\left(\overline{U}_{t\Delta}-\overline{U}_{(t-1)\Delta}\right)^{2}\right]\leq
4N^{-4}\sum_{p=1}^{6}\sum_{{\bf z}\in\mathcal{C}_{p,t}}|B({\bf z})|.$ (81)

Notice that $\mathcal{C}_{1,t}=\mathcal{C}_{2,t}=\mathcal{C}_{5,t}=\emptyset$
for $t\geq 2$, so that we need only to control the sum of covariances for the
cases $p\in\\{3,4,6\\}$ . This is done in the 3 steps below.

Step 1. Here, we show that there exists a constant $K>0$ such that for all
$N\geq 1$, $\Delta\geq 1$ and $t\geq 2$,

$\sum_{{\bf z}\in\mathcal{C}_{3,t}}|B({\bf z})|\leq
KN(1-\lambda)^{(t-2)\Delta+1}.$

First, note that item 3 of Lemma 3.9 implies that

$\displaystyle\sum_{{\bf z}\in\mathcal{C}_{3,t}}|B({\bf z})|$
$\displaystyle\leq
KN^{2}\sum_{t_{1}=1}^{\Delta}\sum_{t_{3}=(t-1)\Delta+1}^{t\Delta}(1-\lambda)^{(t_{3}-t_{1})}N^{-1}$
$\displaystyle=KN(1-\lambda)^{(t-2)\Delta+1}\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\sum_{t_{1}=1}^{\Delta}(1-\lambda)^{\Delta-
t_{1}}.$

Step 1 follows by noticing that $\sum_{t_{1}=1}^{\Delta}(1-\lambda)^{\Delta-
t_{1}}=\sum_{k=0}^{\Delta-1}(1-\lambda)^{k},$ and then by using that
$\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\leq\sum_{k=0}^{\infty}(1-\lambda)^{k}\leq\lambda^{-1}$.

Step 2. Here, we prove that there exists a constant $K>0$ such that for all
$N\geq 1$, $\Delta\geq 1$ and $t\geq 2$,

$\sum_{{\bf z}\in\mathcal{C}_{4,t}}|B({\bf z})|\leq
KN(1-\lambda)^{(t-2)\Delta+1}.$

First, by Remark 3.10, we can write

$\sum_{{\bf z}\in\mathcal{C}_{4,t}}|B({\bf z})|=\sum_{{\bf
z}\in\mathcal{C}_{4,t}}{{\mathbf{1}}}_{\\{z_{1}=z_{2},z_{3}\neq
z_{4}\\}}|B({\bf z})|+\sum_{{\bf
z}\in\mathcal{C}_{4,t}}{{\mathbf{1}}}_{\\{z_{1}\neq
z_{2},z_{3}=z_{4}\\}}|B({\bf z})|.$

To conclude, we apply item 4 of Lemma 3.9 to obtain an upper bound for each
one of the terms on the right hand side of the above identity. Since both
terms are treated very similarly, we explain how we handle the first term
only. By applying item 4 of Lemma 3.9 to the first one, we obtain that

$\sum_{{\bf z}\in\mathcal{C}_{4,t}}{{\mathbf{1}}}_{\\{z_{1}=z_{2},z_{3}\neq
z_{4}\\}}|B({\bf z})|\leq
KN^{3}\sum_{t_{1}=1}^{\Delta}\sum_{t_{3}=(t-1)\Delta+1}^{t\Delta}\sum_{t_{4}=t_{3}}^{t\Delta}(1-\lambda)^{(t_{4}-t_{3})+(t_{3}-t_{1})}N^{-2}.$

Now, since for any $(t-1)\Delta+1\leq t_{3}\leq t\Delta$,

$\sum_{t_{4}=t_{3}}^{t\Delta}(1-\lambda)^{(t_{4}-t_{3})}=\sum_{k=0}^{t\Delta-
t_{3}}(1-\lambda)^{k}\leq\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\leq\lambda^{-1},$
(82)

we can then proceed as in Step 1 to conclude that

$\sum_{{\bf z}\in\mathcal{C}_{4,t}}{{\mathbf{1}}}_{\\{z_{1}=z_{2},z_{3}\neq
z_{4}\\}}|B({\bf z})|\leq KN(1-\lambda)^{(t-2)\Delta+1}.$

Step 3. Here, we prove that there exists a constant $K>0$ such that for all
$N\geq 1$, $\Delta\geq 1$ and $t\geq 2$,

$\sum_{{\bf z}\in\mathcal{C}_{6,t}}|B({\bf z})|\leq
KN^{2}(1-\lambda)^{(t-2)\Delta+1}.$

By applying item 6 of Lemma 3.9 and inequality (82), one can check that

$\sum_{{\bf z}\in\mathcal{C}_{6,t}}|B({\bf z})|\leq
KN^{4}\sum_{t_{1}=1}^{\Delta}\sum_{t_{2}=t_{1}}^{\Delta}\sum_{t_{3}=(t-1)\Delta+1}^{t\Delta}N^{-2}(1-\lambda)^{t_{3}-t_{1}}.$

Some algebraic computations imply that

$\sum_{t_{1}=1}^{\Delta}\sum_{t_{2}=t_{1}}^{\Delta}\sum_{t_{3}=(t-1)\Delta+1}^{t\Delta}(1-\lambda)^{t_{3}-t_{1}}\leq(1-\lambda)^{(t-2)\Delta+1}\left(\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\right)\\\
\times\left(\sum_{k=0}^{\Delta-1}(k+1)(1-\lambda)^{k}\right).$

Putting together these estimates and using the fact that
$\sum_{k=0}^{\infty}(k+1)(1-\lambda)^{k}<\infty$, we conclude the proof of
Step 3.

Finally, combining Steps 1 through 3 and (81), the result follows. ∎

######  Lemma 6.9.

There exists a constant $K$ depending only on $\lambda,\mu$ and $p,$ such that
for all $N\geq 1$ and $\Delta\geq 1$,

$\operatorname{\mathop{Var}}_{\theta}[(\overline{U}_{\Delta})^{2}]\leq
K\Delta^{2}N^{-2}.$

###### Proof.

We use the notation of the proof of Lemma 6.8, except that we now denote
${\mathcal{C}_{1,6}}$ the set of all vectors $\bf z\in F_{1}\times F_{1}\times
F_{1}\times F_{1}$ satisfying the conditions of item 7. of Lemma 3.9, that is,
without imposing the constraint that $t_{1}>t_{4}$ or $t_{3}>t_{2}$. By taking
$t=1$ in inequality (81), it follows that

$\operatorname{\mathop{Var}}_{\theta}\left[\left(\overline{U}_{\Delta}\right)^{2}\right]\leq
4N^{-4}\left(\sum_{p=1}^{6}\sum_{{\bf z}\in\mathcal{C}_{1,p}}|B({\bf
z})|\right).$ (83)

So we need to show that the sum of covariances is at most $K\Delta^{2}N^{2}$.
This is done in the 5 steps below.

Step 1. First, we show that there exists a constant $K>0$ such that for all
$N\geq 1$ and $\Delta\geq 1$,

$\sum_{p=1}^{2}\sum_{{\bf z}\in\mathcal{C}_{1,p}}|B({\bf z})|\leq K(\Delta
N)^{2}.$

By item 1 and item 2 of Lemma 3.9, we have that

$\sum_{p=1}^{2}\sum_{{\bf z}\in\mathcal{C}_{1,p}}|B({\bf
z})|\leq|\mathcal{C}_{1,1}|+|\mathcal{C}_{1,2}|\leq\Delta N+2\Delta N(\Delta
N-1)\leq 3(\Delta N)^{2},$

so that the result follows with $K=3$.

Step 2. Next, we show that there exists a constant $K>0$ such that for all
$N\geq 1$ and $\Delta\geq 1$,

$\sum_{{\bf z}\in\mathcal{C}_{1,3}}|B({\bf z})|\leq KN\Delta^{2}.$

Item 3 of Lemma 3.9, ensures that

$\sum_{{\bf z}\in\mathcal{C}_{1,3}}|B({\bf z})|\leq
KN^{2}\sum_{t_{1}=1}^{\Delta}\sum_{t_{3}=1}^{\Delta}(1-\lambda)^{|t_{3}-t_{1}|}N^{-1}.$

Since
$\sum_{t_{1}=1}^{\Delta}\sum_{t_{3}=1}^{\Delta}(1-\lambda)^{|t_{3}-t_{1}|}\leq\Delta^{2}$,
the result follows from the previous inequality.

Step 3. Here, we show that there exists a constant $K>0$ such that for all
$N\geq 1$ and $\Delta\geq 1$,

$\sum_{{\bf z}\in\mathcal{C}_{1,4}}|B({\bf z})|\leq KN\Delta^{2}.$

We proceed very similarly as in the proof of Step 2 of Lemma 6.8. First,
observe that we can write (see Remark 3.10),

$\sum_{{\bf z}\in\mathcal{C}_{1,4}}|B({\bf z})|=\sum_{{\bf
z}\in\mathcal{C}_{1,4}}{{\mathbf{1}}}_{\\{z_{1}=z_{2},z_{3}\neq
z_{4}\\}}|B({\bf z})|+\sum_{{\bf
z}\in\mathcal{C}_{1,4}}{{\mathbf{1}}}_{\\{z_{1}\neq
z_{2},z_{3}=z_{4}\\}}|B({\bf z})|.$

To obtain the result, we apply item 4 of Lemma 3.9 to obtain an upper bound
for each term on the right hand side of the above identity. Since both terms
are treated very similarly, we explain how we deal with the first term only.
By applying Point 4 of Lemma 3.9 to the first one, we obtain that

$\sum_{{\bf z}\in\mathcal{C}_{1,4}}{{\mathbf{1}}}_{\\{z_{1}=z_{2},z_{3}\neq
z_{4}\\}}|B({\bf z})|\leq
KN^{3}\sum_{t_{1}=1}^{\Delta}\sum_{t_{3}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{3}\leq
t_{4}\\}}(1-\lambda)^{|s_{3}-s_{2}|+|s_{2}-s_{1}|}N^{-2},$

where $s_{1}\leq s_{2}\leq s_{3}$ denotes the ordering of the triple
$(t_{1},t_{3},t_{4})$. Next, we use that $|s_{3}-s_{2}|+|s_{2}-s_{1}|\geq
t_{4}-t_{3}$ to deduce that

$\displaystyle\sum_{t_{1}=1}^{\Delta}\sum_{t_{3}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{3}\leq
t_{4}\\}}(1-\lambda)^{|s_{3}-s_{2}|+|s_{2}-s_{1}|}$
$\displaystyle\leq\Delta\sum_{t_{3}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{3}\leq
t_{4}\\}}(1-\lambda)^{t_{4}-t_{3}}$
$\displaystyle\leq\Delta^{2}\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\leq\lambda^{-1}\Delta^{2}.$

As a consequence, we obtain that

$\sum_{{\bf z}\in\mathcal{C}_{1,4}}{{\mathbf{1}}}_{\\{z_{1}=z_{2},z_{3}\neq
z_{4}\\}}|B({\bf z})|\leq KN\lambda^{-1}\Delta^{2},$

and the result follows.

Step 4. In this step, we show that there exists a constant $K>0$ such that for
all $N\geq 1$ and $\Delta\geq 1$,

$\sum_{{\bf z}\in\mathcal{C}_{1,5}}|B({\bf z})|\leq KN^{2}\Delta^{2}.$

We start observing that

$\sum_{{\bf z}\in\mathcal{C}_{1,5}}|B({\bf z})|=\sum_{{\bf
z}\in\mathcal{C}_{1,5}}{{\mathbf{1}}}_{\\{z_{1}=z_{3},z_{2}\neq
z_{4}\\}}|B({\bf z})|+\sum_{{\bf
z}\in\mathcal{C}_{1,5}}{{\mathbf{1}}}_{\\{z_{1}\neq
z_{3},z_{2}=z_{4}\\}}|B({\bf z})|.$

Hence, it suffices to provide an upper bound for each term on the right hand
side of the above inequality. We will explain how we deal with the first one
only. The second one can be treated similarly. By item 5 of Lemma 3.9, we have
that

$\sum_{{\bf z}\in\mathcal{C}_{1,5}}{{\mathbf{1}}}_{\\{z_{1}=z_{3},z_{2}\neq
z_{4}\\}}|B({\bf z})|\leq
KN^{3}\sum_{t_{1}=1}^{\Delta}\sum_{t_{2}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{2}\leq
t_{4}\\}}(1-\lambda)^{(t_{4}-t_{2})}N^{-1}.$

Hence, by observing that

$\sum_{t_{1}=1}^{\Delta}\sum_{t_{2}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{2}\leq
t_{4}\\}}(1-\lambda)^{(t_{4}-t_{2})}\leq\Delta^{2}\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\leq\Delta^{2}\lambda^{-1},$

the result follows.

Step 5. Finally, we show that there exists a constant $K>0$ such that for all
$N\geq 1$ and $\Delta\geq 1$,

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}|B({\bf z})|\leq KN^{2}\Delta^{2}.$

We start from

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}|B({\bf z})|\leq\sum_{{\bf
z}\in{\mathcal{C}_{1,6}}}\left[{{\mathbf{1}}}_{\\{t_{3}\leq t_{4}\leq
t_{1}\leq t_{2}\\}}+{{\mathbf{1}}}_{\\{t_{1}\leq t_{2}\leq t_{3}\leq
t_{4}\\}}\right]|B({\bf z})|+\sum_{{\bf
z}\in{\mathcal{C}_{1,6}}}\left[{{\mathbf{1}}}_{\\{t_{1}\leq t_{3}\leq
t_{2}\leq t_{4}\\}}\right.\\\ \left.+{{\mathbf{1}}}_{\\{t_{3}\leq t_{1}\leq
t_{4}\leq t_{2}\\}}+{{\mathbf{1}}}_{\\{t_{1}\leq t_{3}\leq t_{4}\leq
t_{2}\\}}+{{\mathbf{1}}}_{\\{t_{3}\leq t_{1}\leq t_{2}\leq
t_{4}\\}}\right]|B({\bf z})|.$

In the sequel, we handle the two terms on the right-hand side of the above
inequality separately.

Step 5.1 Here, we show that there exists a constant $K>0$ such that for all
$N\geq 1$ and $\Delta\geq 1$,

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}\left[{{\mathbf{1}}}_{\\{t_{3}\leq
t_{4}\leq t_{1}\leq t_{2}\\}}+{{\mathbf{1}}}_{\\{t_{1}\leq t_{2}\leq t_{3}\leq
t_{4}\\}}\right]|B({\bf z})|\leq KN^{2}\Delta.$

By item 6 of Lemma 3.9,

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}{{\mathbf{1}}}_{\\{t_{3}\leq t_{4}\leq
t_{1}\leq t_{2}\\}}|B({\bf z})|\leq
KN^{4}\sum_{t_{1}=1}^{\Delta}\sum_{t_{2}=1}^{\Delta}\sum_{t_{3}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{3}\leq
t_{4}\leq t_{1}\leq t_{2}\\}}\times\\\ (1-\lambda)^{t_{2}-t_{3}}N^{-2}.$

We upper bound

$\sum_{t_{1}=1}^{\Delta}\sum_{t_{2}=1}^{\Delta}\sum_{t_{3}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{3}\leq
t_{4}\leq t_{1}\leq
t_{2}\\}}(1-\lambda)^{t_{2}-t_{3}}\leq\Delta\left(\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\right)^{3},$

such that

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}{{\mathbf{1}}}_{\\{t_{3}\leq t_{4}\leq
t_{1}\leq t_{2}\\}}|B({\bf z})|\leq KN^{2}\Delta.$

Proceeding similarly, we can also check that $\sum_{{\bf
z}\in{\mathcal{C}_{1,6}}}{{\mathbf{1}}}_{\\{t_{3}\leq t_{4}\leq t_{1}\leq
t_{2}\\}}|B({\bf z})|\leq KN^{2}\Delta$. This concludes the proof of Step 5.1

Step 5.2 Here we show that there exists a constant $K>0$ such that for all
$N\geq 1$ and $\Delta\geq 1$,

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}\left[{{\mathbf{1}}}_{\\{t_{1}\leq
t_{3}\leq t_{2}\leq t_{4}\\}}+{{\mathbf{1}}}_{\\{t_{3}\leq t_{1}\leq t_{4}\leq
t_{2}\\}}+{{\mathbf{1}}}_{\\{t_{1}\leq t_{3}\leq t_{4}\leq
t_{2}\\}}+{{\mathbf{1}}}_{\\{t_{3}\leq t_{1}\leq t_{2}\leq
t_{4}\\}}\right]|B({\bf z})|\leq KN^{2}\Delta^{2}.$

From item 7 of Lemma 3.9, we deduce that

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}{{\mathbf{1}}}_{\\{t_{1}\leq t_{3}\leq
t_{2}\leq t_{4}\\}}|B({\bf z})|\leq K\sum_{{\bf
z}\in{\mathcal{C}_{1,6}}}{{\mathbf{1}}}_{\\{t_{1}\leq t_{3}\leq t_{2}\leq
t_{4}\\}}(1-\lambda)^{(t_{4}-t_{2})+(t_{3}-t_{1})}N^{-2}.$

Since

$\sum_{t_{1}=1}^{\Delta}\sum_{t_{2}=1}^{\Delta}\sum_{t_{3}=1}^{\Delta}\sum_{t_{4}=1}^{\Delta}{{\mathbf{1}}}_{\\{t_{1}\leq
t_{3}\leq t_{2}\leq
t_{4}\\}}(1-\lambda)^{(t_{4}-t_{2})+(t_{3}-t_{1})}\leq\Delta^{2}\left(\sum_{k=0}^{\Delta-1}(1-\lambda)^{k}\right)^{2},$

it then follows that

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}{{\mathbf{1}}}_{\\{t_{1}\leq t_{3}\leq
t_{2}\leq t_{4}\\}}(1-\lambda)^{(t_{1}-t_{3})+(t_{2}-t_{4})}N^{-2}\leq
N^{2}\Delta^{2}\lambda^{-2}$

such that

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}{{\mathbf{1}}}_{\\{t_{1}\leq t_{3}\leq
t_{2}\leq t_{4}\\}}|B({\bf z})|\leq KN^{2}\Delta^{2}.$

Proceeding similarly, we can also check the three other cases, concluding the
proof of Step 5.2.

Combining Steps 5.1, 5.2 and 5.3, we obtain

$\sum_{{\bf z}\in{\mathcal{C}_{1,6}}}|B({\bf z})|\leq KN^{2}\Delta^{2},$

concluding the proof of Step 5.

By summing all the upper bounds provided by the Steps 1 through 5, we conclude
the proof of Lemma (6.9).

∎

We are now in position to handle the terms $D^{N,2}_{\Delta,T}$ and
$D^{N,2}_{2\Delta,T}$.

######  Proposition 6.10.

There exists a constant $K$ depending only on $\lambda$ such that for all
$N\geq 1$, $T\geq 2$ and $1\leq\Delta\leq\lfloor T/2\rfloor$, we have that

$I\\!\\!E_{\theta}\left[\left(D^{N,2}_{\Delta,T}\right)^{2}+\left(D^{N,2}_{2\Delta,T}\right)^{2}\right]\leq
K\frac{\Delta}{T}.$

###### Proof.

Clearly, the result will follow if we can show that
$I\\!\\!E_{\theta}\left[(D^{N,2}_{\Delta,T})^{2}\right]\leq K\Delta T^{-1}$.
To establish this inequality, first observe that

$\displaystyle
I\\!\\!E_{\theta}\left[\left(D^{N,2}_{\Delta,T}\right)^{2}\right]$
$\displaystyle=$
$\displaystyle\frac{N^{2}}{T^{2}}\operatorname{\mathop{Var}}_{\theta}\left[\sum_{t=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{U}_{t\Delta}-\overline{U}_{(t-1)\Delta}\right)^{2}\right]$
$\displaystyle=$ $\displaystyle\frac{N^{2}}{T^{2}}\sum_{t=1}^{\lfloor
T/\Delta\rfloor}\sum_{s=1}^{\lfloor
T/\Delta\rfloor}\operatorname{\mathop{Cov}}_{\theta}\left[\left(\overline{U}_{t\Delta}-\overline{U}_{(t-1)\Delta}\right)^{2},\left(\overline{U}_{s\Delta}-\overline{U}_{(s-1)\Delta}\right)^{2}\right].$

Next, we use the stationarity of the system to conclude that

$I\\!\\!E_{\theta}\left[\left(D^{N,2}_{\Delta,T}\right)^{2}\right]=\frac{N^{2}}{T^{2}}\lfloor
T/\Delta\rfloor\operatorname{\mathop{Var}}_{\theta}[(\overline{U}_{\Delta})^{2}]\\\
+2\frac{N^{2}}{T^{2}}\sum_{t=2}^{\lfloor T/\Delta\rfloor}\left(\lfloor
T/\Delta\rfloor-t+1\right)\operatorname{\mathop{Cov}}_{\theta}\left[\left(\overline{U}_{\Delta}\right)^{2},\left(\overline{U}_{t\Delta}-\overline{U}_{(t-1)\Delta}\right)^{2}\right].$

As a consequence of Lemma 6.8 and of the fact that
$\sum_{k=0}^{\infty}(1-\lambda)^{k\Delta}<\infty$, it then follows that

$I\\!\\!E_{\theta}\left[\left(D^{N,2}_{\Delta,T}\right)^{2}\right]\leq\frac{N^{2}}{T\Delta}\operatorname{\mathop{Var}}_{\theta}[(\overline{U}_{\Delta})^{2}]+K(\Delta
T)^{-1}.$

Hence, using Lemma 6.9, we establish the desired inequality and conclude the
proof. ∎

We now are in position to prove the convergence of the estimator $\hat{w}$.

######  Theorem 6.11.

There exists a constant $K$ depending only on $\lambda$ such that for all
$\varepsilon>0$, $N\geq 1$, $T\geq 2$ and $1\leq\Delta\leq\lfloor T/2\rfloor$,

$I\\!\\!P\left(\Big{|}\widehat{w}-w\Big{|}\geq\varepsilon\right)\leq\frac{K}{\varepsilon}\left(\sqrt{\frac{\Delta}{T}}+\frac{1}{N}+\frac{(1-\lambda)^{\Delta}}{\Delta}\right).$
(84)

###### Proof.

By the triangle inequality, we have that

$I\\!\\!E_{\theta}\left[|\widehat{w}-w|\right]\leq
I\\!\\!E_{\theta}\left[|\widehat{w}-w^{N}_{\infty}|\right]+I\\!\\!E_{\theta}\left[|w^{N}_{\infty}-w|\right].$

Combining Propositions 6.1, 6.7 and 6.10 with inequalities (64) and (74), and
then applying the Markov inequality together with the fact that
$\Delta/T<\sqrt{\Delta/T}$ (for $\Delta/T<1$) , the result follows. ∎

##  7 Convergence of the spatial variance

The aim of this section is to prove the convergence of the estimator
$\hat{v}$. The main result is Theorem 7.5. Let us denote

$v^{N}_{\infty}:=\sum_{i=1}^{N}(m^{N}_{i}-\overline{m^{N}})^{2}.$ (85)

As it appears in the analysis below, $v^{N}_{\infty}$ is the limit of
$\hat{v}$ when $T\to\infty$ while $\theta$ is fixed. Recall that

$\hat{v}=\frac{1}{T^{2}}\sum_{i=1}^{N}(Z_{i,T}-\overline{Z}_{T})^{2}-\frac{N}{T^{2}}\overline{Z}_{T}+\frac{1}{T^{3}}\sum_{i=1}^{N}(Z_{i,T})^{2},$

and that $\hat{m}=\overline{Z}_{T}/T$. Therefore,

$|\hat{v}-v^{N}_{\infty}|\leq\Upsilon_{T}^{N,1}+\Upsilon_{T}^{N,2}+\Upsilon_{T}^{N,3},$

where

$\Upsilon_{T}^{N,1}=\left|\sum_{i=1}^{N}\left[\left(\frac{Z_{i,T}}{T}-\frac{\bar{Z}_{T}}{T}\right)^{2}-(m_{i}^{N}-\overline{m^{N}})^{2}\right]-\frac{N}{T}\overline{m^{N}}+\frac{1}{T}\sum_{i=1}^{N}(m_{i}^{N})^{2}\right|,$
$\Upsilon_{T}^{N,2}=\frac{N}{T}\left|\hat{m}-\overline{m^{N}}\right|,\mbox{
and
}\Upsilon_{T}^{N,3}=\frac{1}{T}\left|\sum_{i=1}^{N}\left(\frac{Z_{i,T}}{T}\right)^{2}-(m_{i}^{N})^{2}\right|.$

######  Proposition 7.1.

There exists a constant $K$ depending only on $\lambda$ such that for all
$N\geq 1$ and $T\geq 1$,

$I\\!\\!E_{\theta}\left[\Upsilon_{T}^{N,2}+\Upsilon_{T}^{N,3}\right]\leq
K\left(\frac{N}{T^{2}}+\frac{{N}^{1/2}}{T^{3/2}}\left(1+\sqrt{\sum_{i=1}^{N}\left|m^{N}_{i}-\overline{m^{N}}\right|^{2}}\right)\right).$

###### Proof.

First, remind that $I\\!\\!E_{\theta}[Z_{i,T}/T]=m_{i}^{N}$ and so
$I\\!\\!E_{\theta}[\hat{m}]=\overline{m^{N}}$. On the one hand, by Cauchy-
Schwarz, $I\\!\\!E_{\theta}[\Upsilon_{T}^{N,2}]\leq
NT^{-1}\left(\operatorname{\mathop{Var}}_{\theta}(\hat{m})\right)^{-1/2}\leq
KN^{1/2}T^{-3/2}$ by Proposition 5.1. On the other hand, by first rewriting
$\Upsilon_{T}^{N,3}$ as (recall that ${U}_{i,T}=Z_{i,T}-Tm^{N}_{i}$),

$\Upsilon_{T}^{N,3}=\frac{1}{T}\left|\sum_{i=1}^{N}\left(\frac{U_{i,T}}{T}\right)^{2}+2\sum_{i=1}^{N}(m^{N}_{i}-\overline{m^{N}})\frac{U_{i,T}}{T}+2\frac{N}{T}\overline{m^{N}}\
\overline{U}_{T}\right|$

and then using the triangle inequality, we obtain that

$I\\!\\!E_{\theta}\left[\Upsilon_{T}^{N,3}\right]\leq\frac{1}{T^{3}}\sum_{i=1}^{N}I\\!\\!E_{\theta}\left[\left(U_{i,T}\right)^{2}\right]+\frac{2}{T^{2}}\sum_{i=1}^{N}|m^{N}_{i}-\overline{m^{N}}|I\\!\\!E_{\theta}\left[\left|U_{i,T}\right|\right]+2\frac{N}{T^{2}}\overline{m^{N}}I\\!\\!E_{\theta}\left[\left|\overline{U}_{T}\right|\right].$

Now, by stationarity and Lemma 3.8,

$\operatorname{\mathop{Var}}_{\theta}\left(Z_{i,T}\right)=T\operatorname{\mathop{Var}}_{\theta}\left(X_{i,0}\right)+2\sum_{t=1}^{T}(T-t)\operatorname{\mathop{Cov}}_{\theta}(X_{i,0},X_{i,t})\\\
\leq T\operatorname{\mathop{Var}}_{\theta}\left(X_{i,0}\right)+KT/N\leq KT,$
(86)

so that

$\frac{1}{T^{3}}\sum_{i=1}^{N}I\\!\\!E_{\theta}\left[\left(U_{i,T}\right)^{2}\right]=\frac{1}{T^{3}}\sum_{i=1}^{N}\text{Var}_{\theta}\left(Z_{i,T}\right)\leq
KNT^{-2}.$

Moreover, by using Cauchy-Schwarz inequality and then Jensen inequality, we
deduce that

$\sum_{i=1}^{N}|m^{N}_{i}-\overline{m^{N}}|I\\!\\!E_{\theta}\left[\left|U_{i,T}\right|\right]\leq\sqrt{\sum_{i=1}^{N}\left|m^{N}_{i}-\overline{m^{N}}\right|^{2}}\sqrt{\sum_{i=1}^{N}I\\!\\!E_{\theta}\left[\left(U_{i,T}\right)^{2}\right]}.$

Hence, using once more Inequality (86), we obtain that

$\frac{2}{T^{2}}\sum_{i=1}^{N}|m^{N}_{i}-\overline{m^{N}}|I\\!\\!E_{\theta}\left[\left|U_{i,T}\right|\right]\leq
K\frac{N^{1/2}}{T^{3/2}}\sqrt{\sum_{i=1}^{N}\left|m^{N}_{i}-\overline{m^{N}}\right|^{2}}.$

Finally, by combining Jensen inequality and Lemma 6.6, it follows that
$I\\!\\!E_{\theta}\left[\left|\overline{U}_{T}\right|\right]\leq\sqrt{I\\!\\!E_{\theta}\left[\left|\overline{U}_{T}\right|^{2}\right]}\leq
KT^{1/2}N^{-1/2}$ so that

$2\frac{N}{T^{2}}\overline{m^{N}}I\\!\\!E_{\theta}\left[\left|\overline{U}_{T}\right|\right]\leq
KN^{1/2}T^{-3/2},$

where we have also used that that $|\overline{m^{N}}|\leq 1$. Putting together
the above estimates, it then follows that

$I\\!\\!E_{\theta}\left[\Upsilon_{T}^{N,3}\right]\leq
K\left(\frac{N}{T^{2}}+\frac{{N}^{1/2}}{T^{3/2}}\sqrt{\sum_{i=1}^{N}\left|m^{N}_{i}-\overline{m^{N}}\right|^{2}}\right),$

and the result follows since we have already proved that
$I\\!\\!E_{\theta}[\Upsilon_{T}^{N,2}]\leq KN^{1/2}T^{-3/2}$.

∎

We now turn to the study of $\Upsilon_{T}^{N,1}.$ Using that

$(a-b)^{2}-(\bar{a}-\bar{b})^{2}=(a-b)^{2}-(a-\bar{b})^{2}+(a-\bar{a})^{2}+2(a-\bar{a})(\bar{a}-\bar{b}),$

we obtain that
$\Upsilon_{T}^{N,1}\leq\Upsilon_{T}^{N,1,1}+\Upsilon_{T}^{N,1,2}+\Upsilon_{T}^{N,1,3},$
where

$\Upsilon_{T}^{N,1,1}=\left|\sum_{i=1}^{N}\left(\frac{Z_{i,T}}{T}-\frac{\bar{Z}_{T}}{T}\right)^{2}-\left(\frac{Z_{i,T}}{T}-\overline{m^{N}}\right)^{2}\right|,$
$\Upsilon_{T}^{N,1,2}=\left|\sum_{i=1}^{N}\left(\frac{Z_{i,T}}{T}-m_{i}^{N}\right)^{2}-\frac{N}{T}\overline{m^{N}}+\frac{1}{T}\sum_{i=1}^{N}(m_{i}^{N})^{2}\right|$

and

$\Upsilon_{T}^{N,1,3}=2\left|\sum_{i=1}^{N}(\frac{Z_{i,T}}{T}-m_{i}^{N})(m_{i}^{N}-\overline{m^{N}})\right|.$

Furthermore, remind that ${U}_{i,t}=Z_{i,t}-tm^{N}_{i}$ and so we write
$\Upsilon_{T}^{N,1,2}=\Upsilon_{T}^{N,1,2,1}+\Upsilon_{T}^{N,1,2,2},$ with

$\Upsilon_{T}^{N,1,2,1}=\left|\sum_{i=1}^{N}\left(\frac{U_{i,T}}{T}\right)^{2}-I\\!\\!E_{\theta}\left(\left(\frac{U_{i,T}}{T}\right)^{2}\right)\right|$

and

$\Upsilon_{T}^{N,1,2,2}=\left|\sum_{i=1}^{N}I\\!\\!E_{\theta}\left(\left(\frac{U_{i,T}}{T}\right)^{2}\right)-\frac{N}{T}\overline{m^{N}}+\frac{1}{T}\sum_{i=1}^{N}(m_{i}^{N})^{2}\right|.$

######  Proposition 7.2.

For all $N\geq 1$ and $T\geq 1$, there exists a constant $K$ depending only on
$\lambda$ such that

$I\\!\\!E_{\theta}\left[\Upsilon_{T}^{N,1,1}+\Upsilon_{T}^{N,1,2,2}\right]\leq
K\frac{1}{T}.$

###### Proof.

Adapting the argument used in the beginning of Proposition 6.1, we have

$\Upsilon_{T}^{N,1,1}=N\left(\frac{\bar{Z}_{T}}{T}-\bar{m}_{\theta}^{N}\right)^{2}.$

Hence, Proposition 5.1 implies that $I\\!\\!E_{\theta}\Upsilon_{T}^{N,1,1}\leq
K/T$.

Second, remark that $\Upsilon_{T}^{N,1,2,2}$ is not random and rewrites as

$\Upsilon_{T}^{N,1,2,2}=\left|\sum_{i=1}^{N}\operatorname{\mathop{Var}}_{\theta}\left(\frac{Z_{i,T}}{T}\right)-\frac{N}{T}\overline{m^{N}}+\frac{1}{T}\sum_{i=1}^{N}(m_{i}^{N})^{2}\right|.$

Using Equation (86) and the fact that
$\operatorname{\mathop{Var}}_{\theta}(X_{i,0})=m_{i}^{N}-(m_{i}^{N})^{2}$ we
conclude that $I\\!\\!E_{\theta}\Upsilon_{T}^{N,1,2,2}\leq K/T.$ ∎

######  Proposition 7.3.

For all $N\geq 1$ and $T\geq 1$, there exists a constant $K$ depending only on
$\lambda$ such that

$I\\!\\!E_{\theta}|\Upsilon_{T}^{N,1,3}|^{2}\leq\frac{K}{T}\sum_{i=1}^{N}|m_{i}^{N}-\overline{m^{N}}|^{2}.$

###### Proof.

Clearly,

$I\\!\\!E_{\theta}|\Upsilon_{T}^{N,1,3}|^{2}\leq\frac{4}{T^{2}}\sum_{i,j=1}^{N}|\operatorname{\mathop{Cov}}_{\theta}(U_{i,T},U_{j,T})||m_{i}^{N}-\overline{m^{N}}||m_{j}^{N}-\overline{m^{N}}|.$

But, by stationarity and using Lemma 3.8,

$\left|\operatorname{\mathop{Cov}}_{\theta}(U_{i,T},U_{j,T})\right|\leq\sum_{s,t=1}^{T}\left|\operatorname{\mathop{Cov}}_{\theta}(X_{i,s},X_{j,t})\right|\leq
T\left|\operatorname{\mathop{Cov}}_{\theta}(X_{i,0},X_{j,0})\right|+KT/N.$

Moreover,

$\left|\operatorname{\mathop{Cov}}_{\theta}(X_{i,0},X_{j,0})\right|\leq(m^{N}_{i}-(m_{i}^{N})^{2}){{\mathbf{1}}}_{\\{i=j\\}}+K/N{{\mathbf{1}}}_{\\{i\neq
j\\}},$

such that all in all

$I\\!\\!E_{\theta}|\Upsilon_{T}^{N,1,3}|^{2}\\\
\leq\frac{K}{T}\left(\sum_{i=1}^{N}\left[\frac{1}{N}+(m^{N}_{i}-(m_{i}^{N})^{2})\right](m_{i}^{N}-\overline{m^{N}})^{2}+\frac{1}{N}\sum_{i\neq
j}|m_{i}^{N}-\overline{m^{N}}||m_{j}^{N}-\overline{m^{N}}|\right).$

Since $m^{N}_{i}-(m_{i}^{N})^{2}\leq 1$ and
$|m_{i}^{N}-\overline{m^{N}}||m_{j}^{N}-\overline{m^{N}}|\leq(m_{i}^{N}-\overline{m^{N}})^{2}+(m_{j}^{N}-\overline{m^{N}})^{2},$
the conclusion follows. ∎

######  Proposition 7.4.

For all $N\geq 1$ and $T\geq 1$, there exists a constant $K$ depending only on
$\lambda$ such that

$I\\!\\!E_{\theta}|\Upsilon_{T}^{N,1,2,1}|^{2}\leq KN/T^{2}.$

###### Proof.

We have that

$I\\!\\!E_{\theta}|\Upsilon_{T}^{N,1,2,1}|^{2}=\frac{1}{T^{4}}\sum_{i,j=1}^{N}\operatorname{\mathop{Cov}}_{\theta}((U_{i,T})^{2},(U_{j,T})^{2})\\\
=\frac{1}{T^{4}}\sum_{i,j=1}^{N}\sum_{s=1}^{T}\sum_{t=1}^{T}\operatorname{\mathop{Cov}}_{\theta}(Y_{i,s}Y_{i,s},Y_{j,t}Y_{j,t})+\frac{1}{T^{4}}\sum_{i,j=1}^{N}\sum_{s\neq
s^{\prime}}\sum_{t\neq
t^{\prime}}\operatorname{\mathop{Cov}}_{\theta}(Y_{i,s}Y_{i,s^{\prime}},Y_{j,t}Y_{j,t^{\prime}})\\\
=:S_{1}+S_{2}.$ (87)

By Lemma 3.9, item 1 and item 3, $S_{1}\leq KN/T^{3}.$ To deal with $S_{2},$
we adapt the arguments of the proof of Lemma 6.9. As before, write ${\bf
z}=(z_{1},z_{2},z_{3},z_{4}),$ and $B({\bf
z})=\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}},Y_{z_{3}}Y_{z_{4}}\right].$
This expression is symmetric with respect to $t,t^{\prime}$ and also with
respect to $s,s^{\prime}.$ Now, introduce the set
$F^{(2)}=\\{(z_{1},z_{2}):z_{1}=(i,s),z_{2}=(i,s^{\prime}),i\in\\{1,\dots,N\\},s,s^{\prime}\in\\{1,\dots,T\\},s\neq
s^{\prime}\\}$ and define, for $p\in\\{2,5,7\\}$, $\mathcal{P}_{p,T}$ as the
set of all vectors $\mathbf{z}\in F^{(2)}\times F^{(2)}$ satisfying the
conditions of item $p$ of Lemma 3.9. Moreover, notice that
$\mathbf{z}\in\mathcal{P}_{5,T}\cup\mathcal{P}_{2,T}$ implies that the first
coordinates (denoted $i$ and $j$ above) of the four couples in $\mathbf{z}$
are equal.

Then

$S_{2}=\frac{1}{T^{4}}[\sum_{{\bf z}\in\mathcal{P}_{7,T}}B({\bf z})+\sum_{{\bf
z}\in\mathcal{P}_{5,T}}B({\bf z})+\sum_{{\bf z}\in\mathcal{P}_{2,T}}B({\bf
z})].$

Following Step 5 of the proof of Lemma 6.9, it is easy so see that

$\sum_{{\bf z}\in\mathcal{P}_{7,T}}B({\bf z})\leq KT^{2}.$

Moreover, Lemma 3.9, item 5, implies that

$\sum_{{\bf z}\in\mathcal{P}_{5,T}}B({\bf z})\leq KT^{2}.$

Finally,

$\sum_{{\bf z}\in\mathcal{P}_{2,T}}B({\bf z})\leq
K|\mathcal{P}_{2,T}|=KNT^{2}.$

All in all we therefore obtain that $S_{2}\leq KN/T^{2},$ implying the
assertion. ∎

We are now in position to prove the main result of this section.

######  Theorem 7.5.

For all $N\geq 1$ and $T\geq 1$, there exists a constant $K$ depending only on
$\lambda$ such that

$I\\!\\!E|\hat{v}-v|\leq
K\left(\frac{N}{T^{2}}+\frac{1}{\sqrt{T}}+\frac{\sqrt{N}}{T}+\frac{1}{\sqrt{N}}\right).$

###### Proof.

Combining Propositions 7.1 to 7.4, we have

$I\\!\\!E_{\theta}|\hat{v}-v^{N}_{\infty}|\leq
K\left(1+\left(\sum_{i=1}^{N}(m_{i}^{N}-\overline{m^{N}})^{2}\right)^{1/2}\right)\left(\frac{N}{T^{2}}+\frac{1}{\sqrt{T}}+\frac{\sqrt{N}}{T}\right).$
(88)

Recall that Proposition 4.7 gives the upper bound

$I\\!\\!E\left[\left|\left\|m^{N}-\overline{m^{N}}1_{N}\right\|_{2}^{2}-v\right|\right]\leq\frac{K}{\sqrt{N}}.$

Thus, by the sub-additivity of $x\mapsto\sqrt{x},$

$I\\!\\!E\left[\left(\sum_{i=1}^{N}(m_{i}^{N}-\overline{m^{N}})^{2}\right)^{1/2}\right]=I\\!\\!E\left\|m^{N}-\overline{m^{N}}1_{N}\right\|_{2}\leq\left(I\\!\\!E\left\|m^{N}-\overline{m^{N}}1_{N}\right\|^{2}_{2}\right)^{1/2}\leq\\\
\left(I\\!\\!E\left[\left|\left\|m^{N}-\overline{m^{N}}1_{N}\right\|_{2}^{2}-v\right|\right]\right)^{1/2}+\sqrt{v}\leq
K.$ (89)

Integrating (88) against $I\\!\\!P$ and using (89), we obtain

$I\\!\\!E|\hat{v}-v^{N}_{\infty}|\leq
K\left(\frac{N}{T^{2}}+\frac{1}{\sqrt{T}}+\frac{\sqrt{N}}{T}\right).$

The assertion then follows using Proposition 4.7, observing that
$v^{N}_{\infty}=\left\|m^{N}-\overline{m^{N}}1_{N}\right\|^{2}_{2}.$ ∎

##  8 Conclusion: proof of Theorem 2.1

In this section we prove our main result, Theorem 2.1.

###### Proof of Theorem 2.1.

Inequalities (3) and (5) have been proved in Theorems 5.2 and 6.11,
respectively. As for Inequality (4), we argue as follows. First, observe that
(4) holds automatically when $N^{1/2}/T>1$.

Suppose now that $N^{1/2}/T^{2}\leq 1$. By combining Theorem 7.5 and Markov
inequality and then using that $N/T^{2}\leq N^{1/2}/T$ and
$T^{-1/2}=N^{1/4}T^{-1/2}N^{-1/4}\leq N^{1/2}T^{-1}+N^{-1/2}$, we obtain that
for any $\epsilon\in(0,1)$,

$I\\!\\!P(|\hat{v}-v|\geq\varepsilon)\leq\frac{K}{\varepsilon}\left(\frac{N^{1/2}}{T}+N^{-1/2}\right),$

proving (4), and the result follows. ∎

##  9 Simulation study

Before showing some numerical results, the practical implementation of the
method (especially the inversion of the function $\Psi$ defined by (8)) and
the choice of $\Delta$ are thoroughly discussed.

###  9.1 Practical implementation

The computation of the three estimators $\hat{m},\hat{v},\hat{w}$ is obvious.
However, the choice and computation of the two inverse functions $\Phi^{(+)}$
and $\Phi^{(-)}$, defined in Appendix E, need further explanations.

First, one has to compute $d^{(a)}$ defined in Equation (106). To avoid
numerical errors, we replaced the condition $\kappa(m,w)=4*r_{+}*r_{-}$ by the
condition $|\kappa-4*r_{+}*r_{-}|<10^{-4}$. Moreover, we replaced
$(4r_{+}r_{-})^{2}-4r_{+}r_{-}+\kappa(m,w)$ by
$\max\\{0,(4r_{+}r_{-})^{2}-4r_{+}r_{-}+\kappa(m,w)\\}$ to ensure that the
square-root stays in $\mathbb{R}_{+}$. Remark that in those two cases, we have
$d^{(+)}=d^{(-)}$. In particular, there is no issue regarding the choice of
$\Phi^{(+)}$ or $\Phi^{(-)}$. Out of those two cases, we use $\Phi^{(-)}$
whenever

$r_{+}\geq 1/2\quad\text{or}\quad\kappa(m,w)\geq
4r_{+}r_{-}\quad\text{or}\quad d^{(+)}(m,w)>2r_{-}.$

The first two conditions are chosen according to Proposition E.3. The last one
is supported by the fact that, if $r_{+}<1/2$ and $\kappa(m,w)<4r_{+}r_{-}$
then $D(\lambda,p)<2r_{-}$ (see Proposition E.1) and
$d^{(-)}(m,w)<d^{(+)}(m,w)$. Finally, if all the conditions above are not
satisfied then the choice between $a=+$ and $a=-$ is arbitrarily made.

Second, one has to compute $\phi_{1}^{(a)}$. On the one hand, to avoid
numerical errors and the choice between $a=+$ and $a=-$, we replaced the
condition $r_{+}=1/2$ by $|r_{+}-r_{-}|<10^{-3}$. On the other hand, when
$\phi_{1}^{(a)}$ is applied to the estimators $\hat{m}$ and $\hat{w}$, one may
end up with a negative value. Yet, $\phi_{1}^{(a)}$ should be non negative.
Hence, we replaced $\phi_{1}^{(a)}(m,w)$ by $|\phi_{1}^{(a)}(m,w)|$ (for
instance in Equation (110)).

Finally, when $\Phi^{(+)}$ or $\Phi^{(-)}$ are applied to the estimators
$(\hat{m},\hat{v},\hat{w})$, one may end up with estimators
$(\hat{\mu},\hat{\lambda},\hat{p})$ which do not belong to the admissible set
$\Lambda$ defined above Equation (7). In that case we chose to clip the
values. For instance, clip the value of $\lambda$ to $0$ if negative, or to
$1$ if strictly larger than $1$.

The numerics experiments were made using Julia programming langage and a
package should be available for the next revision of the paper.

###  9.2 Choice of the tuning parameter

From the bound obtained in Equation (10), the optimal choice of $\Delta$ is
$\Delta=-\log(T)/(2\log(1-\lambda))$. In that case,

$\frac{(1-\lambda)^{\Delta}}{\Delta}+\sqrt{\frac{\Delta}{T}}=\frac{-2\log(1-\lambda)}{T^{1/2}\log(T)}+\frac{\sqrt{\log(T)}}{T^{1/2}}.$

Since the value of $\lambda$ is not known, the choice $\Delta=\log(T)$ is made
by default. However, the choice $\Delta=1$ seems to give the best results
among several tests and across several choices of parameters (see Figure 1).
This is the reason why most of the plots of the next section are made with
$\Delta=1$.

###  9.3 Numerical results

Let us first give the framework of our simulations. If not specified
otherwise, the following values are used:

$N=500,\quad r_{+}=.5,\quad\beta=.5,\quad\lambda=.5,\quad
p=.5,\quad\Delta=1,\quad N_{\rm simu}=1000.$

Furthermore, the fractions of excitatory and inhibitory components are chosen
as $r_{+}^{N}=\lceil r_{+}N\rceil$ and $r_{-}^{N}=N-|\mathcal{P}_{+}|$.

In all the plots, the performance of one (or several) of our estimators is
displayed. Let us denote $\hat{\vartheta}$ one of these estimators and
$\vartheta$ its corresponding true parameter value. The solid lines correspond
to the median of $|\hat{\vartheta}-\vartheta|$ computed over $N_{\rm simu}$
simulations and plotted as a function of the time horizon $T$. Furthermore, we
add marks (circles or horizontal bars) on the right end of the plot. These
marks correspond to the median of $|\hat{\vartheta}_{\infty}-\vartheta|$
computed over $N_{\rm simu}$ simulations, where $\hat{\vartheta}_{\infty}$ is
the theoretical limit of $\hat{\vartheta}$ as $T\to\infty$ while the
environment $\theta$ is fixed. More precisely, according to Sections 5 to 7,
we have

$\hat{m}_{\infty}=\overline{m^{N}},\quad\hat{v}_{\infty}=\sum_{i=1}^{N}(m^{N}_{i}-\overline{m^{N}})^{2},\quad\hat{w}_{\infty}=\frac{1}{N}\sum_{k=1}^{N}(c^{N}_{k})^{2}\left(m^{N}_{k}-(m^{N}_{k})^{2}\right).$

Of course, these quantities are unknown in practice but we are able to compute
them here since the parameters used for the simulation are known. Finally, the
corresponding limit estimators of $\mu$, $\lambda$ and $p$ are defined by:

$\left(\hat{\mu}_{\infty},\hat{\lambda}_{\infty},\hat{p}_{\infty}\right)=\Phi^{(a)}\left(\hat{m}_{\infty},\hat{v}_{\infty},\hat{w}_{\infty}\right),$

where the choice of $a\in\\{+,-\\}$ is made according to Section 9.1.

Figure 1:  Absolute estimation error for the six estimators and their
theoretical limits. The $y$-axis is in log-scale. Each line or mark correspond
to a median computed over $N_{\rm simu}=100$ simulations. The panels
correspond to the choices $\Delta=\log(T)$ and $\Delta=1$ from left to right.

Figure 2:  Absolute estimation error of $\hat{p}$ and its theoretical limit.
Each line or mark correspond to a median computed over $N_{\rm simu}=1000$
simulations. The panels correspond to different choices of varying parameter
(the non-varying parameters are chosen according to the default values given
in Section 9.3). The values of the varying parameter are given by the color
legends.

Figure 1 compares the performance of all the estimators. The two panels
correspond to two choices of $\Delta$ that will be discussed below. On the two
panels, it is clear that the convergence of $\hat{m}$ is faster than all the
others (which is expected from our analysis). Furthermore, for large $T$,
$\hat{m}$, $\hat{v}$ and $\hat{\lambda}$ are really close to their theoretical
limits $\hat{m}_{\infty}$, $\hat{v}_{\infty}$ and $\hat{\lambda}_{\infty}$
respectively. Note that this is not the case for $\hat{w}$, $\hat{\mu}$ and
$\hat{p}$. In particular, it seems that $\hat{p}$ has the slowest convergence
rate.

Figure 1 gives also a comparison of the performance of our estimators with
respect to the tuning parameter $\Delta$. Either $\Delta$ is chosen as a
function of $T$ by $\Delta=\log(T)$ or $\Delta$ is fixed to the value $1$.
Obviously, the estimators $\hat{m},\hat{v}$ for finite $T$ and all the limit
estimators $\hat{\vartheta}_{\infty}$ do not depend on $\Delta$ so the
differences between the two plots are due to randomness only. However, it
seems that the convergence of $\hat{w}$ is significantly faster when
$\Delta=1$. In turn, it implies the same improvement for the triplet
$(\hat{\mu},\hat{\lambda},\hat{p})$.

Figure 2 gives an overview of the performance of the estimator $\hat{p}$ as
one of the parameters ($N$, $r_{+}$, $\beta$, $\lambda$ or $p$) varies. The
choice to vary $\beta$ instead of $\mu=\lambda\beta$ is made because the set
of admissible values of $\beta$ is independent of $\lambda$ (which is not the
case for $\mu$). Remark that when $p=0.5$, which is the case for most of the
panels in Figure 2, the value $0.25$ correspond to the median of the absolute
error of the most naive estimator: estimate $p$ by a random uniform value in
$[0,1]$. Overall, the estimation error of the limit $\hat{p}_{\infty}$ is
usually of the order of $0.01$ and that of the estimator $\hat{p}$ computed
for $T=1000$ is approximately 20 times larger. As $N$ increases, the
performance of $\hat{p}$ for $T$ fixed deteriorates. This phenomenon is
encoded in the factor $\sqrt{N}/T$ appearing in Theorem 2.1 for example. Also,
the marks corresponding to the estimation errors of the theoretical limit
$\hat{p}_{\infty}$ for different values of $N$ are ordered as expected: the
estimation error goes to $0$ as $N$ goes to infinity. As $r_{+}$ varies, the
performance of $\hat{p}$ seems not to vary too much. However, it is important
to note that the case $r_{+}=0.4$ corresponds to a case where the choice of
$a\in\\{+,-\\}$ is not obvious and so it is arbitrary (see Section 9.1). Half
of the time, the wrong $a$ is chosen. In particular, it is the reason why the
mark corresponding to the limit estimator is around $0.06$ instead of $0.01$
for all the other cases. As $\beta$ varies, the performances of $\hat{p}$ and
$\hat{p}_{\infty}$ seem not to vary too much. This is quite expected. As
$\lambda$ increases towards $1$, the performances of $\hat{p}$ and
$\hat{p}_{\infty}$ decreases. This is expected since $\lambda\to 1$ implies
that the strength of interactions in the system goes to $0$. In particular, it
is more and more difficult to estimate $p$ which is closely related to the
interactions. Nevertheless, contrarily to our upper-bounds which goes to
infinity as $\lambda\to 0$ (see the remark below Theorem 2.1), the performance
is good for $\lambda=0.1$. Finally, the performance of $\hat{p}$ as $p$ varies
is more complex to analyze. For instance, for $p=0.9$, our method gives
$\hat{p}=1$ most of the time whence the eventually constant violet curve at
$y=0.1$. In the case $p=0.1$, the performance of $\hat{p}$ seems to be poor
despite the fact that its limit $\hat{p}_{\infty}$ (blue mark) achieves good
results. This phenomenon is also expected because as $p\to 0$ the interactions
in the system vanishes.

##  Appendix A Construction of the process

In this section we prove Theorem 3.3.

###### Proof of Theorem 3.3.

The stationarity of the process $X$ follows from the fact that its
construction is invariant under time shift. Hence, to conclude the proof, it
remains to show that, conditionally on each realization of the random
environment $\theta$, $X$ is Markovian and has the transition probabilities
given by (2) and (1). This can be done as follows. First, observe that (27)
implies that for each $i\in\\{1,\ldots,N\\}$, there exists a deterministic
function
$f_{\theta,i}:\\{0,1\\}^{N}\times\\{0,1\\}\times\\{0,1,\ldots,N\\}\to\\{0,1\\}$
such that for all $t\in\mathbb{Z}$,

$X_{i,t}=f_{\theta,i}(X_{t-1},\xi_{i,t},J_{i,t}).$

Then defining the function

$f_{\theta}(x,\xi,J)=(f_{\theta,1}(x,\xi_{1},J_{1}),\ldots,f_{\theta,N}(x,\xi_{N},J_{N})),$

where $x\in\\{0,1\\}^{N}$, $\xi=(\xi_{1},\ldots,\xi_{N})\in\\{0,1\\}^{N}$ and
$J=(J_{1},\ldots,J_{N})\in\\{0,1,\ldots,N\\}^{N}$, we can write for all
$t\in\mathbb{Z}$,

$X_{t}=f_{\theta}(X_{t-1},\xi_{t},J_{t}),$

where $\xi_{t}=(\xi_{1,t},\ldots,\xi_{N,t})$ and
$J_{t}=(J_{1,t},\ldots,J_{N,t})$. This representation together with the fact
the sequence $(\xi_{t},J_{t})_{t\in\mathbb{Z}}$ is i.i.d ensures that
$(X_{t})_{t\in\mathbb{Z}}$ is a Markov Chain. Moreover, since the pairs
$(\xi_{1,t},J_{1,t}),\ldots,(\xi_{N,t},J_{N,t})$ are independent, the random
variables $X_{i,t}=f_{\theta,i}(X_{t-1},\xi_{i,t},J_{i,t})$,
$i\in\\{1,\ldots,N\\}$, are clearly conditionally independent given $X_{t-1}$.
Therefore, to conclude the proof, it remains to show that
$I\\!\\!P_{\theta}(X_{i,t}=1|X_{t-1}=x)=p_{\theta,i}(x)$ for all
$x\in\\{0,1\\}^{N}$, where $p_{\theta,i}(x)$ is given by (1). To see that,
write $z=(i,t)$ and observe that (27) allows us to write,

$I\\!\\!P_{\theta}(X_{z}=1|X_{t-1}=x)=I\\!\\!P_{\theta}(J_{z}=0,\xi_{z}=1)+\sum_{j\in\mathcal{P}_{+}}\theta_{ij}x_{j}I\\!\\!P_{\theta}(J_{z}=j|X_{t-1}=x)\\\
+\sum_{j\in\mathcal{P}_{-}}\theta_{ij}(1-x_{j})I\\!\\!P_{\theta}(J_{z}=j|X_{t-1}=x).$

Using that $X_{t-1}$ is measurable with respect to
$\sigma(\xi_{k,s},J_{k,s},s\leq t-1,k\in\\{1,\ldots,N\\})$ and the fact that
both $J_{z}$ and $\xi_{z}$ are independent of all random variables $J_{k,s}$,
$\xi_{k,s}$ for $s\leq t-1$ and $k\in\\{1,\ldots,N\\}$, we have that

$I\\!\\!P_{\theta}(J_{z}=0,\xi_{z}=1|X_{t-1}=x)=I\\!\\!P_{\theta}(J_{z}=0,\xi_{z}=1),$

and also that for each $j\in\\{1,\ldots,N\\},$

$I\\!\\!P_{\theta}(J_{z}=j|X_{t-1}=x)=I\\!\\!P_{\theta}(J_{z}=j)=\frac{(1-\lambda)}{N}.$

As a consequence, it follows that

$I\\!\\!P_{\theta}(X=1|X_{t-1}=x)=I\\!\\!P_{\theta}(J_{z}=0,\xi_{z}=1)+\frac{(1-\lambda)}{N}\sum_{j\in\mathcal{P}_{+}}\theta_{ij}x_{j}+\sum_{j\in\mathcal{P}_{-}}\theta_{ij}(1-x_{j}).$

Since the random variables $J_{z}$ and $\xi_{z}$ are also independent, and
$I\\!\\!P_{\theta}(J_{z}=0)=\lambda,$ $I\\!\\!P_{\theta}(\xi_{z}=1)=\beta$ and
$\mu=\lambda\beta$, it follows that
$I\\!\\!P_{\theta}(X_{i,t}=1|X_{t-1}=x)=p_{\theta,i}(x)$, concluding the
proof. ∎

##  Appendix B Coupling results

In this section we prove the coupling result stated in Lemmas 3.4 and 3.5.

Let us denote $(J,\xi)$ the sequence $(J_{z},\xi_{z})_{z}$ defined in the
backward regeneration representation of the process $X$. Since we are working
for a fixed matrix $\theta$ in this section, we omit its dependence in what
follows. Due to the backward regeneration representation there exist
measurable functions $f$, $g$, $r$ and $\tau$ (which depend on $\theta$) such
that, for all $z\in\mathcal{Z}$,

$I^{z}=(I^{z}_{s})_{s\in\mathbb{Z}}=f(z,J)\text{ and
}X_{z}=g(I^{z},\xi_{r(I^{z})}),$

where $r(I^{z})=(I^{z}_{\tau(I^{z})},\tau(I^{z}))$ is the regenerating site
associated with $z$ and $\tau(I^{z})=\inf\\{s\in\mathbb{Z}:I^{z}_{s}>0\\}$ is
the last time that the random walk $I^{z}$ is not in the state $0$. In what
follows, we denote $I=(I^{z})_{z\in\cal Z}$.

Now, consider $5$ independent sequences

$(J^{(1)},\xi^{(1)}),\ldots,(J^{(5)},\xi^{(5)}),$

all distributed as the sequence $(J,\xi)$, and which are furthermore
independent of $(J,\xi)$. By convenience of notation, let us denote
$(J^{(0)},\xi^{(0)}):=(J,\xi)$. In other words, the layer (0) corresponds to
the original process, and we add five i.i.d. layers on top of that. We then
define the backward random walks $I^{(i)}=(I^{(i),z})_{z\in\mathcal{Z}}$ by
$I^{(i),z}=f(z,J^{(i)})$, for each $0\leq i\leq 5$. Clearly, the collections
of random walks $I^{(0)},\ldots,I^{(5)}$ are independent. We will need the
layers $(0)-(4)$ to construct the tilde versions, and the layers $(0)$ and
$(5)$ to construct the hat versions.

#### Construction of the tilde versions

For $i\in\\{1,\ldots,4\\}$, let us denote

${\tau}^{c}_{i}=\sup\left\\{\tau^{c}_{z_{i},z_{j}}:j\in\\{1,\ldots,4\\}\setminus\\{i\\}\right\\},$

the first time at which $I^{z_{i}}$ coalesces with (at least) one of the $3$
other random walks (observe that ${\tau}^{c}_{i}=-\infty$ if $I^{z_{i}}$ does
not coalesce with any of the other $3$ random walks), and denote
$z^{c}_{i}=(I^{z_{i}}_{{\tau}^{c}_{i}},{\tau}^{c}_{i})$ the site of
coalescence in case ${\tau}^{c}_{i}>-\infty$. Then, for all $1\leq i\leq 4$
and $s\in\mathbb{Z}$, we define

$\tilde{I}^{z_{i}}_{s}=I^{(0),z_{i}}_{s}\mathbf{1}_{s>\tau^{c}_{i}}+I^{(i),z^{c}_{i}}_{s}\mathbf{1}_{s\leq\tau^{c}_{i}}.$
(90)

In words, the random walk $\tilde{I}^{z_{i}}$ follows the random walk
$I^{z_{i}}$ until it coalesces with one of the random walks $I^{z_{j}}$,
$j\in\\{1,\ldots,4\\}\setminus\\{i\\}$. After this time, the random walk
$\tilde{I}^{z_{i}}$ follows the independent random walk $I^{(i),z_{i}^{c}}$
associated with the site of coalescence $z_{i}^{c}$. With the notation
$\tilde{L}^{z_{i}}_{s}=i\cdot 1_{s\leq{\tau}^{c}_{i}}\in\\{0,\dots,4\\}$,
Equation (90) rewrites as

$\tilde{I}^{z_{i}}_{s}=I^{(0),z_{i}}_{s}\mathbf{1}_{\tilde{L}^{z_{i}}_{s}=0}+I^{(i),z^{c}_{i}}_{s}\mathbf{1}_{\tilde{L}^{z_{i}}_{s}=i},$

and some computations give the following easy recursion formula:

$\tilde{I}^{z_{i}}_{s-1}=J_{(\tilde{I}^{z_{i}}_{s},s)}^{(\tilde{L}^{z_{i}}_{s})},\text{
for all $s\leq t_{i}$},$ (91)

which can be compared with Equation (26). Moreover, remark that the layer
processes $\tilde{L}$ are measurable functions of the state processes
$\tilde{I}$ since

$\tau^{c}_{z_{i},z_{j}}=\sup\left\\{s\in\mathbb{Z}:I^{z_{i}}_{s}=I^{z_{j}}_{s}\notin\\{0,\infty\\}\right\\}=\sup\left\\{s\in\mathbb{Z}:\tilde{I}^{z_{i}}_{s}=\tilde{I}^{z_{j}}_{s}\notin\\{0,\infty\\}\right\\}.$

#### Construction of the hat versions

Let us denote, for $i=1,2$,

$\hat{\tau}_{i}=\sup\left\\{\tau^{c}_{z_{i},z_{j}}:j\in\\{3,4\\}\right\\},$

and denote $\hat{z}^{c}_{i}=(I^{z_{i}}_{\hat{\tau}_{i}},\hat{\tau}_{i})$ the
corresponding site of coalescence in case $\hat{\tau}_{i}>-\infty$. For
instance, $\hat{\tau}_{1}$ corresponds to the first time at which $I^{z_{1}}$
coalesces with either $I^{z_{3}}$ or $I^{z_{4}}$. Then, for $i=1,2$, we define
the auxiliary process, for all $s\in\mathbb{Z}$,

$\hat{I}^{(5),z_{i}}_{s}=I^{z_{i}}_{s}\mathbf{1}_{s>\hat{\tau}_{i}}+I^{(5),\hat{z}^{c}_{i}}_{s}\mathbf{1}_{s\leq\hat{\tau}_{i}}.$

The construction is not yet done since these two random walks may be missing
some coalescence in order to mimic the joint distribution of
$(X_{z_{1}},X_{z_{2}}).$ To circumvent this problem, we define

$\hat{\tau}_{1,2}=\sup\left\\{s\in\mathbb{Z}:\hat{I}^{(5),z_{1}}_{s}=\hat{I}^{(5),z_{2}}_{s}\notin\\{0,\infty\\}\right\\},$

and

$\hat{I}^{z_{i}}=\hat{I}^{(5),z_{i}},\mbox{ if
}\hat{\tau}_{1,2}\leq\hat{\tau}_{i},$

while we put, for all $s\in\mathbb{Z}$,

$\hat{I}^{z_{i}}_{s}=I^{z_{i}}_{s}\mathbf{1}_{s>\hat{\tau}_{1,2}}+\hat{I}^{(5),z_{3-i}}_{s}\mathbf{1}_{s\leq\hat{\tau}_{1,2}},\mbox{
if }\hat{\tau}_{1,2}>\hat{\tau}_{i}.$ (92)

For instance, if $I^{z_{1}}$ first coalesces, say, with $I^{z_{3}},$ then it
switches to its hat-version $\hat{I}^{z_{1}}$ and remains sticked to this
version forever. However, if $I^{z_{2}}$ hits $\hat{I}^{z_{1}}$ before hitting
$I^{z_{3}}$ or $I^{z_{4}},$ then it coalesces with $\hat{I}^{z_{1}}$ and
remains sticked to it forever. In words, we do not modify the random walk with
the first (the largest in time) coalescence time $\hat{\tau}_{i}$, but we
possibly modify the other one in between the times $\hat{\tau}_{1}$ and
$\hat{\tau}_{2}$. Like Equation (91), we have the following recursion:

$\hat{I}^{z_{i}}_{s-1}=J_{(\hat{I}^{z_{i}}_{s},s)}^{(\hat{L}^{z_{i}}_{s})},\text{
for all $s\leq t_{i}$},$ (93)

where

$\hat{L}^{z_{1}}_{s}=5\cdot 1_{s\leq\hat{\tau}_{1}}+5\cdot
1_{\hat{\tau}_{1}<s\leq\hat{\tau}_{1,2}}1_{\hat{\tau}_{1,2}\leq\hat{\tau}_{2}}$
(94)

with a similar definition for $\hat{L}^{z_{2}}_{s}.$ In other words, in any
case, process $1$ is in layer $5$ starting from time $\hat{\tau}_{1}$, that is
$\hat{L}^{z_{1}}_{s}=5$ for $s\leq\hat{\tau}_{1}$. But, if process $1$
coalesces with process $2$ (at time $\hat{\tau}_{1,2}$) and if process $2$ is
already evolving on layer $5$ at that time, then process $1$ switches to layer
$5$ at that coalescence time $\hat{\tau}_{1,2}$.

Once again, the layer processes $\hat{L}^{z_{1}}$ and $\hat{L}^{z_{2}}$ are
measurable functions of the state processes
$(\hat{I}^{z_{1}},\hat{I}^{z_{2}},I^{z_{3}},I^{z_{4}})$.

Now, let us formalize the fact that the processes $I$ do not depend on the
whole sequences $J^{(0)},\dots,J^{(5)}$ but merely on a small subset of those.
Let $\gamma\in\\{0,1,\dots,N,\infty\\}^{\mathbb{Z}}$ and
$\ell\in\\{0,\dots,5\\}^{\mathbb{Z}}$ be some generic trajectories of the
processes $I$ and $L$. Let us then define
$J_{\gamma}^{\ell}=(J_{(\gamma_{s},s)}^{(\ell_{s})})_{s\in\mathbb{Z}}\in\\{0,1,\dots,N\\}^{\mathbb{Z}}$
and, for all $z=(i,t)\in\mathcal{Z}$,
$f^{|}_{z}:\\{0,1,\dots,N\\}^{\mathbb{Z}}\to\\{0,1,\dots,N,\infty\\}^{\mathbb{Z}}$
given by

$(f^{|}_{z}(J_{\gamma}^{\ell}))_{s}=\begin{cases}\infty&\text{if $s>t$}\\\
i&\text{if $s=t$}\\\
J_{(\gamma_{s+1},s+1)}^{(\ell_{s+1})}&\text{else}.\end{cases}$

Finally, let us remark that:

  1. 1.

the definition of $I^{z_{i}}$ implies that
$\\{I^{z_{i}}=\gamma\\}=\\{f^{|}_{z_{i}}(J_{\gamma}^{\ell^{\emptyset}})=\gamma\\}$
where $\ell^{\emptyset}_{s}=0$ for all $s\in\mathbb{Z}$;

  2. 2.

Equation (91) and the measurability of the layer processes imply that for all
$\tilde{\gamma}^{1},\dots,\tilde{\gamma}^{4}\in\\{0,1,\dots,N,\infty\\}^{\mathbb{Z}}$,
there exist
$\tilde{\ell}^{1},\dots,\tilde{\ell}^{4}\in\\{0,\dots,4\\}^{\mathbb{Z}}$ such
that

$\cap_{i=1}^{4}\\{\tilde{I}^{z_{i}}=\tilde{\gamma}^{i}\\}=\cap_{i=1}^{4}\\{\tilde{I}^{z_{i}}=\tilde{\gamma}^{i},\tilde{L}^{z_{i}}=\tilde{\ell}^{i}\\}=\cap_{i=1}^{4}\\{f^{|}_{z_{i}}(J_{\tilde{\gamma}^{i}}^{\tilde{\ell}^{i}})=\tilde{\gamma}^{i}\\};$

  3. 3.

Equation (93) and the measurability of the layer processes imply that for all
choices
$\hat{\gamma}^{1},\hat{\gamma}^{2},\gamma^{3},\gamma^{4},\tilde{\gamma}^{3},\tilde{\gamma}^{4}\in\\{0,1,\dots,N,\infty\\}^{\mathbb{Z}}$,
there exist $\hat{\ell}^{1},\hat{\ell}^{2}\in\\{0,5\\}^{\mathbb{Z}}$ and
$\tilde{\ell}^{3},\tilde{\ell}^{4}\in\\{0,3,4\\}^{\mathbb{Z}}$ such that

$\displaystyle A$ $\displaystyle:=$
$\displaystyle\left(\cap_{i=1}^{2}\\{\hat{I}^{z_{i}}=\hat{\gamma}^{i}\\}\right)\cap\left(\cap_{j=3}^{4}\\{I^{z_{j}}=\gamma^{j},\tilde{I}^{z_{j}}=\tilde{\gamma}^{j}\\}\right)$
$\displaystyle=$
$\displaystyle\left(\cap_{i=1}^{2}\\{\hat{I}^{z_{i}}=\hat{\gamma}^{i},\hat{L}^{z_{i}}=\hat{\ell}^{i}\\}\right)\cap\left(\cap_{j=3}^{4}\\{I^{z_{j}}=\gamma^{j},\tilde{I}^{z_{j}}=\tilde{\gamma}^{j},\tilde{L}^{z_{j}}=\tilde{\ell}^{j}\\}\right)$
$\displaystyle=$
$\displaystyle\left(\cap_{i=1}^{2}\\{f^{|}_{z_{i}}(J_{\hat{\gamma}^{i}}^{\hat{\ell}^{i}})=\hat{\gamma}^{i}\\}\right)\cap\left(\cap_{j=3}^{4}\\{f^{|}_{z_{j}}(J_{\gamma^{j}}^{\ell^{\emptyset}})=\gamma^{j},f^{|}_{z_{j}}(J_{\tilde{\gamma}^{j}}^{\tilde{\ell}^{j}})=\tilde{\gamma}^{j}\\}\right).$

First, let us summarize the coupling properties of the processes $I$.

######  Proposition B.1.

The processes $I$, $\tilde{I}$ and $\hat{I}$ satisfy the following coupling
properties:

  1. (i)

for all $i=1,\dots,4$, $\\{\tilde{I}^{z_{i}}\neq
I^{z_{i}}\\}=\cup_{j\in\\{1,\ldots,4\\}\setminus\\{i\\}}\\{z_{i}\leftrightsquigarrow
z_{j}\\}$,

  2. (ii)

we have that

$\\{\hat{I}^{z_{1}}\neq
I^{z_{1}}\\}\subset\bigcup_{k=3}^{4}\\{z_{1}\leftrightsquigarrow
z_{k}\\}\cup\left(\bigcup_{k=3}^{4}\\{z_{2}\leftrightsquigarrow
z_{k}\\}\cap\\{\tau^{R}_{z_{1}}<t_{3}\vee t_{4}\\}\right)$

and

$\\{\hat{I}^{z_{2}}\neq
I^{z_{2}}\\}\subset\bigcup_{k=3}^{4}\\{z_{2}\leftrightsquigarrow
z_{k}\\}\cup\left(\bigcup_{k=3}^{4}\\{z_{1}\leftrightsquigarrow
z_{k}\\}\cap\\{\tau^{R}_{z_{2}}<t_{3}\vee t_{4}\\}\right).$

###### Proof.

This follows directly from the construction of the processes and the remark
that

$\left\\{\hat{\tau}_{i}<\hat{\tau}_{1,2}\leq\hat{\tau}_{3-i}\right\\}\subset\bigcup_{k=3}^{4}\\{z_{3-i}\leftrightsquigarrow
z_{k}\\}\cap\\{\tau^{R}_{z_{i}}<t_{3}\vee t_{4}\\}.$

∎

We are now in position to prove the following independence properties of the
processes $I$.

######  Proposition B.2.

The processes $I$, $\tilde{I}$ and $\hat{I}$ satisfy the following
independence properties:

  1. (i)

for all $i$, $\tilde{I}^{z_{i}}$ is independent of
$(I^{z_{j}},\tilde{I}^{z_{j}})_{j\neq i}$;

  2. (ii)

$(\hat{I}^{z_{1}},\hat{I}^{z_{2}})$ is independent of
$(I^{z_{j}},\tilde{I}^{z_{j}})_{j=3,4}$.

  3. (iii)

$\hat{I}^{z_{1}}$ is independent of $\tilde{I}^{z_{2}},$ and $\hat{I}^{z_{2}}$
is independent of $\tilde{I}^{z_{1}}.$

###### Proof.

#### Proof of (i)

Assume without loss of generality that $i=1$. Let
$\tilde{\gamma}^{1},\dots,\tilde{\gamma}^{4},\gamma^{2},\dots,\gamma^{4}\in\\{0,1,\dots,2N,\infty\\}^{\mathbb{Z}},$
and let $\tilde{\ell}_{1},\ldots,\tilde{\ell}_{4}$ be the associated layer
processes, which are uniquely defined being measurable functions of
$\tilde{\gamma}^{1},\dots,\tilde{\gamma}^{4},\gamma^{2},\dots,\gamma^{4}.$ By
the point 2 above, we have

$\displaystyle A$ $\displaystyle=$
$\displaystyle\left(\cap_{i=1,\dots,4}\\{\tilde{I}^{z_{1}}=\tilde{\gamma}^{1}\\}\right)\cap\left(\cap_{j=2,\dots,4}\\{I^{z_{j}}=\gamma^{j}\\}\right)$
$\displaystyle=$
$\displaystyle\\{f^{|}_{z_{1}}(J_{\tilde{\gamma}^{1}}^{\tilde{\ell}^{1}})=\tilde{\gamma}^{1}\\}\cap\left(\cap_{j=2}^{4}\\{f^{|}_{z_{j}}(J_{\tilde{\gamma}^{j}}^{\tilde{\ell}^{j}})=\tilde{\gamma}^{j},f^{|}_{z_{j}}(J_{\gamma^{j}}^{\ell^{\emptyset}})=\gamma^{j}\\}\right)$

By construction, if $I\\!\\!P(A)$ is non null, then the set
$\\{(\tilde{\gamma}^{1}_{s},\tilde{\ell}^{1}_{s},s),s\in\mathbb{Z},\tilde{\gamma}^{1}_{s}\notin\\{0,\infty\\}\\}$
is disjoint from the sets
$\\{(\tilde{\gamma}^{j}_{s},\tilde{\ell}^{j}_{s},s),s\in\mathbb{Z},\tilde{\gamma}^{j}_{s}\notin\\{0,\infty\\}\\}$
and
$\\{(\gamma^{j}_{s},0,s),s\in\mathbb{Z},\gamma^{j}_{s}\notin\\{0,\infty\\}\\}$
for $j>1$ so that the two events in the final expression of $A$ are
independent, and item (i) follows.

#### Proof of (ii)

Let
$\hat{\gamma}^{1},\hat{\gamma}^{2},\gamma^{3},\gamma^{4},\tilde{\gamma}^{3},\tilde{\gamma}^{4}\in\\{0,1,\dots,N,\infty\\}^{\mathbb{Z}},$
and let $\hat{l}^{1},\hat{l}^{2},\tilde{l}^{3},\tilde{l}^{4}$ be the
associated layer processes. By the point 3 above, we have

$\displaystyle A$ $\displaystyle=$
$\displaystyle\left(\cap_{i=1}^{2}\\{\hat{I}^{z_{i}}=\hat{\gamma}^{i}\\}\right)\cap\left(\cap_{j=3}^{4}\\{I^{z_{j}}=\gamma^{j},\tilde{I}^{z_{j}}=\tilde{\gamma}^{j}\\}\right)$
$\displaystyle=$
$\displaystyle\left(\cap_{i=1}^{2}\\{f^{|}_{z_{i}}(J_{\hat{\gamma}^{i}}^{\hat{\ell}^{i}})=\hat{\gamma}^{i}\\}\right)\cap\left(\cap_{j=3}^{4}\\{f^{|}_{z_{j}}(J_{\gamma^{j}}^{\ell^{\emptyset}})=\gamma^{j},f^{|}_{z_{j}}(J_{\tilde{\gamma}^{j}}^{\tilde{\ell}^{j}})=\tilde{\gamma}^{j}\\}\right).$

By construction, if $I\\!\\!P(A)$ is non null, then the sets
$\\{(\hat{\gamma}^{1}_{s},\hat{\ell}^{1}_{s},s),s\in\mathbb{Z},\hat{\gamma}^{1}_{s}\notin\\{0,\infty\\}\\}$
and
$\\{(\hat{\gamma}^{2}_{s},\hat{\ell}^{2}_{s},s),s\in\mathbb{Z},\hat{\gamma}^{2}_{s}\notin\\{0,\infty\\}\\}$
are disjoint from the sets
$\\{(\tilde{\gamma}^{j}_{s},\tilde{\ell}^{j}_{s},s),s\in\mathbb{Z},\tilde{\gamma}^{j}_{s}\notin\\{0,\infty\\}\\}$
and
$\\{(\gamma^{j}_{s},0,s),s\in\mathbb{Z},\gamma^{j}_{s}\notin\\{0,\infty\\}\\}$
for $j=3,4$ so that the two events in the final expression of $A$ are
independent, and item (ii) follows.

#### Proof of (iii)

Let
$\hat{\gamma}^{1},\tilde{\gamma}^{2}\in\\{0,1,\dots,N,\infty\\}^{\mathbb{Z}},$
and let $\hat{\ell}^{1}\in\\{0,5\\}^{\mathbb{Z}}$ and
$\tilde{\ell}^{2}\in\\{0,2\\}^{\mathbb{Z}}$ be two fixed layer processes that
are compatible with the event
$\\{\hat{I}^{z_{1}}=\hat{\gamma}^{1}\\}\cap\\{\tilde{I}^{z_{2}}=\tilde{\gamma}^{2}\\}$.
By compatible, we mean for example that supposing that $\hat{\gamma}^{1}$ and
$\tilde{\gamma}^{2}$ meet at some time $s\leq t_{1}\vee t_{2},$ it is not
possible to have $\hat{l}^{1}_{u}=\tilde{l}^{2}_{u}$ for $u\leq s.$

Then, analogously to the proof of point (ii) above, the sets
$\\{(\hat{\gamma}^{1}_{s},\hat{\ell}^{1}_{s},s),s\in\mathbb{Z},\hat{\gamma}_{s}^{1}\notin\\{0,\infty\\}\\}$
and
$\\{(\tilde{\gamma}^{2}_{s},\tilde{\ell}^{2}_{s},s),s\in\mathbb{Z},\tilde{\gamma}_{s}^{2}\notin\\{0,\infty\\}\\}$
are disjoint sets implying the independence. ∎

######  Proposition B.3.

For all $i$, $\tilde{I}^{z_{i}}$ has the same distribution as $I^{z_{i}}$ and
$(\hat{I}^{z_{1}},\hat{I}^{z_{2}})$ has the same distribution as
$(I^{z_{1}},I^{z_{2}})$.

###### Proof.

Remind that the layers $J^{(0)},\dots,J^{(5)}$ are i.i.d. so that it is easy
to prove that $\tilde{I}^{z_{i}}$ is a backward random walk with the same
transitions as $I^{z_{i}}$ (compare Equation (91) with Equation (26)) which in
turn implies that they share the same distribution.

We now turn to the second part of the proof. For the marginals, the same
argument as above applies (thanks to Equation (93)). Hence, $\hat{I}^{z_{i}}$
has the same distribution as $I^{z_{i}}$. Then, we show that the process
$(\hat{I}^{z_{1}}_{s},\hat{I}^{z_{2}}_{s})_{s}$ is a two-dimensional Markov
chain (backwards in time) which has the same transitions as
$(I^{z_{1}}_{s},I^{z_{2}}_{s})_{s}.$ Since both chains start from the same
initial conditions, this implies the desired result.

Since transitions between $t_{1}\vee t_{2}$ and $t_{1}\wedge t_{2}$ only
concern one of the two processes, evolving according to the right marginals,
we do only need to consider transitions $s\to s-1$ for $s\leq t_{1}\wedge
t_{2}.$ Fix $i_{1},i_{2},j_{1},j_{2}\in\\{0,\ldots,N\\}.$ Let us first discuss
the case $i_{1}\neq i_{2}.$

Then for any $l_{1},l_{2}\in\\{0,5\\},$ writing

$A:=\\{\hat{I}^{z_{1}}_{s}=i_{1},\hat{I}^{z_{2}}_{s}=i_{2},\hat{L}^{z_{1}}_{s}=l_{1},\hat{L}^{z_{2}}_{s}=l_{2}\\},$

we have that

$I\\!\\!P(\hat{I}^{z_{1}}_{s-1}=j_{1},\hat{I}^{z_{2}}_{s-1}=j_{2}|A)=I\\!\\!P(J^{(l_{1})}_{(i_{1},s)}=j_{1},J^{(l_{2})}_{(i_{2},s)}=j_{2}|A).$

By construction, $J^{(l_{1})}_{(i_{1},s)}$ and $J^{(l_{2})}_{(i_{2},s)}$ are
independent (since $i_{1}\neq i_{2}),$ they have the same joint distribution
as $(J^{(0)}_{(i_{1},s)},J^{(0)}_{(i_{2},s)}),$ and they are independent of
$A$ (since $A$ depends only on decisions strictly after time $s$). Thus

$I\\!\\!P(\hat{I}^{z_{1}}_{s-1}=j_{1},\hat{I}^{z_{2}}_{s-1}=j_{2}|A)=I\\!\\!P(J^{(0)}_{(i_{1},s)}=j_{1},J^{(0)}_{(i_{2},s)}=j_{2}).$

Summing over all possible choices of $l_{1}$ and $l_{2},$ this implies that

$I\\!\\!P(\hat{I}^{z_{1}}_{s-1}=j_{1},\hat{I}^{z_{2}}_{s-1}=j_{2}|\hat{I}^{z_{1}}_{s}=i_{1},\hat{I}^{z_{2}}_{s}=i_{2})=I\\!\\!P(J^{(0)}_{(i_{1},s)}=j_{1},J^{(0)}_{(i_{2},s)}=j_{2}).$
(95)

We now discuss the case $i_{1}=i_{2}.$ Then necessarily $l_{1}=l_{2}$ and
thus, by our coalescence construction, $j_{1}=j_{2}.$ With the same notation
for the set $A$ as above and using the same independence argument, it still
holds that

$I\\!\\!P(\hat{I}^{z_{1}}_{s-1}=j_{1}=\hat{I}^{z_{2}}_{s-1}|A)=I\\!\\!P(J^{(l_{1})}_{(i_{1},s)}=j_{1}|A)=I\\!\\!P(J^{(l_{1})}_{(i_{1},s)}=j_{1})=I\\!\\!P(J^{(0)}_{(i_{1},s)}=j_{1}).$

Summing over all possible values of $l_{1}$ implies that

$I\\!\\!P(\hat{I}^{z_{1}}_{s-1}=j_{1}=\hat{I}^{z_{2}}_{s-1}|\hat{I}^{z_{1}}_{s}=i_{1}=\hat{I}^{z_{2}}_{s})=I\\!\\!P(J^{(0)}_{(i_{1},s)}=j_{1}).$
(96)

The transition probabilities given by Equations (95) and (96) correspond
exactly to the transition probabilities of $(I^{z_{1}}_{s},I^{z_{2}}_{s})$ \-
see Equation (26). ∎

We can now proceed to the proofs of Lemmas 3.4 and 3.5

###### Proof of Lemma 3.4.

Four tilde processes are constructed in this section so we write the proof for
$k=4$. Nevertheless, the proof can easily be generalized to any $k\geq 2$.

Let $z_{1},\dots,z_{4}$ be four different sites in $\mathcal{Z}$. For all
$i=1,\dots,4$, let $\tilde{I}^{z_{i}}$ be the backward random walks defined by
(90), and $\tilde{L}^{z_{i}}$ be the associated layer process. We denote
$\tilde{L}^{z_{i}}_{-\infty}=\lim_{s\to-\infty}\tilde{L}^{z_{i}}_{s}\in\\{0,i\\}$
the terminal layer. Then, remind the functions $g$ and $r$ defined in the
beginning of Appendix B and define, for all $i=1,\dots,4$,

$\tilde{X}_{z_{i}}=g\left(\tilde{I}^{z_{i}},\xi^{(0)}_{r(\tilde{I}^{z_{i}})}\right)\mathbf{1}_{\tilde{L}^{z_{i}}_{-\infty}=0}+g\left(\tilde{I}^{z_{i}},\xi^{(i)}_{r(\tilde{I}^{z_{i}})}\right)\mathbf{1}_{\tilde{L}^{z_{i}}_{-\infty}=i}.$

In comparison, remind that $X_{z_{i}}=g(I^{z_{i}},\xi^{(0)}_{r(I^{z_{i}})})$.
Using the fact that the $\xi^{(j)}$’s are i.i.d. with Propositions B.2 and
B.3, one deduces items (i) and (ii). Finally, item (iii) follows from the fact
that $\\{\tilde{X}^{z_{i}}\neq X^{z_{i}}\\}\subset\\{\tilde{I}^{z_{i}}\neq
I^{z_{i}}\\}$ and Proposition B.1. ∎

###### Proof of Lemma 3.5.

Let $z_{1},\dots,z_{4}$ be four different sites in $\mathcal{Z}$. For all
$i=1,\dots,4$, let us define $\tilde{X}_{z_{i}}$ as in the proof of Lemma 3.4.
In particular, items (iv) to (vi) follow from Lemma 3.4.

Then, for $i=1,2$, let $\hat{I}^{z_{i}}$ be the backward random walks defined
by (92), and $\hat{L}^{z_{i}}$ be the associated layer process. We denote
$\hat{L}^{z_{i}}_{-\infty}=\lim_{s\to-\infty}\tilde{L}^{z_{i}}_{s}\in\\{0,5\\}$
the terminal layer. Then, remind the functions $g$ and $r$ defined in the
beginning of Appendix B and define, for all $i=1,\dots,4$,

$\hat{X}_{z_{i}}=g\left(\hat{I}^{z_{i}},\xi^{(0)}_{r(\hat{I}^{z_{i}})}\right)\mathbf{1}_{\hat{L}^{z_{i}}_{-\infty}=0}+g\left(\hat{I}^{z_{i}},\xi^{(5)}_{r(\hat{I}^{z_{i}})}\right)\mathbf{1}_{\hat{L}^{z_{i}}_{-\infty}=5}.$

Using the fact that the $\xi^{(j)}$’s are i.i.d., item (i) follows from
Proposition B.3, items (ii) and (v) follow respectively from items (ii) and
(iii) of Proposition B.2. Finally, items (ii) and (iii) follow from item (ii)
of Proposition B.1. ∎

##  Appendix C Coalescence of two or more backward random walks

In this section we prove Proposition 3.7. Throughout the proof we will
consider partitions of the sets of cardinal 2, 3 and 4. For a finite set $E$,
we denote by $\mathcal{P}(E)$ the set of its partitions. For ease of notation,
we consider a notation which we exemplify in the case of a set $\\{a,b,c\\}$
with three different elements. In this case, the set
$\mathcal{P}(\\{a,b,c\\})$ has 5 elements which are :

  * •

$\\{\\{a\\},\\{b\\},\\{c\\}\\}$ written as $a\\!\dagger\\!b\\!\dagger\\!c$,

  * •

$\\{\\{a,b\\},\\{c\\}\\}$, written as $ab\\!\dagger\\!c$,

  * •

$\\{\\{a,c\\},\\{b\\}\\}$, written as $ac\\!\dagger\\!b$,

  * •

$\\{\\{a\\},\\{b,c\\}\\}$, written as $a\\!\dagger\\!bc$,

  * •

$\\{\\{a,b,c\\}\\}$, written as $abc$.

###### Proof of Proposition 3.7.

Without loss of generality, we assume in this proof that $t_{1}\geq t_{2}\geq
t_{3}\geq t_{4}$.

#### Proof of item (i)

Consider the backward random walks $I^{z_{1}}$ and $I^{z_{2}}$ from time
$t=t_{1}$ to time $t=-\infty$. Let us introduce the following notation: in
what follows, the letter ‘$S$’ stands for the state of a random walk;
$S^{1}_{t}$ will be the state at time $t$ of the random walk associated to
$z_{1},$ and $S^{2}_{t}$ the state at time $t$ of the random walk associated
to $z_{2}$. We define $S^{1}_{t}=1_{\infty}$ if $t>t_{1},$ $S^{1}_{t}=1_{0}$
if $I^{z_{1}}_{t}=0$, $S^{1}_{t}=1$ else, and by analogy,
$S^{2}_{t}=2_{\infty}$ if $t>t_{2},$ $S^{2}_{t}=2_{0}$ if $I^{z_{2}}_{t}=0,$
$S^{2}_{t}=2$ else.

In what follows, we extend the notation introduced above for partitions of
sets. Let us denote $P_{t}=S^{1}_{t}\\!\dagger\\!S^{2}_{t}$ if
$t>\tau^{c}_{z_{1},z_{2}}$ and $P_{t}=S^{1}_{t}S^{2}_{t}$ else. Then, the
backward process $(P_{t})_{-\infty<t\leq t_{1}}$ is a time in-homogeneous
backward Markov chain on the state space
$\mathcal{P}_{2}=\cup_{a\in\\{1_{\infty},1,1_{0}\\}}\cup_{b\in\\{2_{\infty},2,2_{0}\\}}\\{ab,a\\!\dagger\\!b\\}$.
At time $t=t_{1}$, the backward process starts from the initial condition
$P_{t_{1}}=1\\!\dagger\\!2_{\infty}$ if $t_{1}>t_{2},$ and
$P_{t_{1}}=1\\!\dagger\\!2$ if $t_{1}=t_{2}$ (note that in that case
$z_{1}\neq z_{2}$ implies that $i_{1}\neq i_{2}$ so that $P_{t_{1}}=12$ is not
possible). By Proposition 3.1, the random variable
$\min\\{\tau^{R}_{z_{1}},\tau^{R}_{z_{2}}\\}>-\infty$ almost surely, implying
that $P_{-\infty}:=\lim_{t\to-\infty}P_{t}$ exists almost surely and
$P_{-\infty}\in\\{1_{0}2_{0},1_{0}\\!\dagger\\!2_{0}\\}$, that is the set of
absorbing states. One can check that

$I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\\})=I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}2_{0}\,|\,P_{t_{1}}=1\\!\dagger\\!2_{\infty}\right),$
(97)

in case $t_{1}>t_{2},$ and

$I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\\})=I\\!\\!P_{\theta}(P_{-\infty}=1_{0}2_{0}\,|\,P_{t_{1}}=1\\!\dagger\\!2),$
(98)

in case $t_{1}=t_{2}.$

Now, for $t\leq t_{1}$, let us denote $Q_{t}(x,y)$ the transition
probabilities of the time in-homogeneous backward Markov chain
$(P_{t})_{-\infty<t\leq t_{1}}$:

$Q_{t}(x,y)=I\\!\\!P_{\theta}\left(P_{t-1}=y|P_{t}=x\right),\;x,y\in\mathcal{P}_{2}.$

Assume for now that $t_{1}>t_{2}$. Since we are only interested in computing
the coalescence probability (97), we only need to give the transition
probabilities encountered on the path from state $1\\!\dagger\\!2_{\infty}$ to
state $1_{0}2_{0}$. For $t>t_{2}+1$, the only relevant transition probability
is $Q_{t}(1\\!\dagger\\!2_{\infty},1\\!\dagger\\!2_{\infty})=1-\lambda$. At
time $t=t_{2}+1$ there are two relevant transitions:
$Q_{t_{2}+1}(1\\!\dagger\\!2_{\infty}$,$1\\!\dagger\\!2)=(1-\lambda)(1-1/N)$
and $Q_{t_{2}+1}(1\\!\dagger\\!2_{\infty},12)=(1-\lambda)/N$.

For $t\leq t_{2}$, there are four relevant transition probabilities:

  * •

$Q_{t}(1\\!\dagger\\!2,1\\!\dagger\\!2)=(1-\lambda)^{2}(1-1/2)$

  * •

$Q_{t}(1\\!\dagger\\!2,12)=(1-\lambda)^{2}/N$,

  * •

$Q_{t}(12,1_{0}2_{0})=1-Q_{t}(12,12)=\lambda$.

Figure 3 gives a graphical representation of the relevant transitions just
described in case $t_{2}<t_{1}$.

$1\\!\dagger\\!2_{\infty}$$1\\!\dagger\\!2$$12$$1_{0}2_{0}$$1-\lambda$$(1-\lambda)(1-1/N)$$\frac{1-\lambda}{N}$$(1-\lambda)^{2}(1-1/N)$$\frac{(1-\lambda)^{2}}{N}$$1-\lambda$$\lambda$$1$
Figure 3: Graph of the relevant transitions of the backward process $P$ used
to compute a bound for $I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\\})$ when $t_{1}\geq t_{2}$. The starting node is
$1\\!\dagger\\!2_{\infty}$ if $t_{1}>t_{2}$ and $1\\!\dagger\\!2$ else. On
each edge, the corresponding transition probability is given. The gray
vertical line separates two temporal zones: the right one corresponds to times
$t>t_{2}+1$ and the left one corresponds to times $t\leq t_{2}+1$.

Starting from Equation (97), one can check that

$\mathbb{P}_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\\})=\mathbb{P}_{\theta}(P_{t_{2}}=12|P_{t_{1}}=1\\!\dagger\\!2_{\infty})\mathbb{P}_{\theta}(P_{-\infty}=1_{0}2_{0}|P_{t_{2}}=12)\\\
+\mathbb{P}_{\theta}(P_{t_{2}}=1\\!\dagger\\!2|P_{t_{1}}=1\\!\dagger\\!2_{\infty})\mathbb{P}_{\theta}(P_{-\infty}=1_{0}2_{0}|P_{t_{2}}=1\\!\dagger\\!2).$

Now, by the dynamics of the Markov chain $(P_{t})_{t\leq t_{1}}$ we have that
$\mathbb{P}_{\theta}(P_{t_{2}}=1\\!\dagger\\!2|P_{t_{1}}=1\\!\dagger\\!2_{\infty})=(1-\lambda)^{t_{1}-t_{2}}$
and
$\mathbb{P}_{\theta}(P_{t_{2}}=12|P_{t_{1}}=1\\!\dagger\\!2_{\infty})=(1-\lambda)^{t_{1}-t_{2}}/N$.
Moreover, $\mathbb{P}_{\theta}(P_{-\infty}=1_{0}2_{0}|P_{t_{2}}=12)=1$ by
Proposition 3.1. By observing that

$\displaystyle\mathbb{P}_{\theta}(P_{-\infty}=1_{0}2_{0}|P_{t_{2}}=1\\!\dagger\\!2)$
$\displaystyle=$
$\displaystyle\frac{(1-\lambda)^{2}}{N}\sum_{k=0}^{\infty}(1-\lambda)^{2k}(1-1/N)^{k}$
$\displaystyle\leq$
$\displaystyle\frac{(1-\lambda)^{2}}{N}\frac{1}{1-(1-\lambda)^{2}},$

and by putting all pieces together, we then obtain that

$\displaystyle I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\\})\leq\frac{(1-\lambda)^{|t_{1}-t_{2}|}}{N}\left[1+\frac{(1-\lambda)^{2}}{{1-(1-\lambda)^{2}}}\right]=\frac{(1-\lambda)^{|t_{1}-t_{2}|}}{N(1-(1-\lambda)^{2})}.$
(99)

In the case $t_{2}=t_{1}$, we have

$\displaystyle I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\\})=\mathbb{P}_{\theta}(P_{-\infty}=1_{0}2_{0}|P_{t_{2}}=1\\!\dagger\\!2)\\\
\leq\frac{(1-\lambda)^{2}}{N}\frac{1}{1-(1-\lambda)^{2}}\leq\frac{(1-\lambda)}{N(1-(1-\lambda)^{2})},$
(100)

so that item (i) follows from inequalities (99) and (100).

#### Proof of item (ii)

By analogy to the case with two sites, we define $S^{3}_{t}=3_{\infty}$ if
$t>t_{3},$ $S^{3}_{t}=3_{0}$ if $I^{z_{3}}_{t}=0$, $S^{3}_{t}=3$ else. Let us
describe the backward process $(P_{t})_{-\infty<t\leq t_{1}}$ in that case.
First note that for all $t\leq t_{1}$, $P_{t}$ is a partition of
$\\{S^{1}_{t},S^{2}_{t},S^{3}_{t}\\}$ and in particular
$P_{s}\in\mathcal{P}_{3}=\cup_{a\in\\{1_{\infty},1,1_{0}\\}}\cup_{b\in\\{2_{\infty},2,2_{0}\\}}\cup_{c\in\\{3_{\infty},3,3_{0}\\}}\mathcal{P}(\\{a,b,c\\})$.
The choice of the partition is induced by the equivalence relation defined by

$S^{k}_{t}\leftrightarrow_{t}S^{\ell}_{t}\mbox{ if and only if
}t\leq\tau_{c}^{z_{k},z_{\ell}}.$

This equivalence relation naturally induces a partition of
$\\{S^{1}_{t},S^{2}_{t},S^{3}_{t}\\}$ and we define $P_{t}$ as this partition.
Similarly to the proof of item (i), we have

$I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow z_{2}\leftrightsquigarrow
z_{3}\\})=I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}2_{0}3_{0}\,|\,P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\right),$
(101)

in case $t_{1}>t_{2}\geq t_{3}$ for instance, where
$P_{-\infty}=\lim_{t\to-\infty}P_{t}$ exists almost surely and
$P_{-\infty}\in{\cal{P}}(\\{1_{0},2_{0},3_{0}\\}).$ Assume for now that
$t_{1}>t_{2}>t_{3}$ which is the most general case. The other cases can be
treated similarly.

For $t\leq t_{1}$, let us denote the transition probabilities of the chain

$Q_{t}(x,y)=I\\!\\!P_{\theta}\left(P_{t-1}=y|P_{t}=x\right),\;x,y\in\mathcal{P}_{3}.$

Since we are only interested in computing the coalescence probability (101),
we only need to give the transition probabilities encountered on the path from
state $1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}$ to state
$1_{0}2_{0}3_{0}$. For $t>t_{2}+1$, the only relevant transition probability
is
$Q_{t}(1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty},1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty})=1-\lambda$.
At time $t=t_{2}+1$, there are two relevant transition probabilities:
$Q_{t_{2}+1}(1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty},1\\!\dagger\\!2\\!\dagger\\!3_{\infty})=(1-\lambda)(1-1/N)$
and
$Q_{t_{2}+1}(1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty},12\\!\dagger\\!3_{\infty})=(1-\lambda)/(N)$.

For $t_{3}+1<t\leq t_{2}$ the three relevant transition probabilities are :

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3_{\infty},1\\!\dagger\\!2\\!\dagger\\!3_{\infty})=(1-\lambda)^{2}(1-1/N)\leq(1-\lambda)^{2}$,

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3_{\infty},12\\!\dagger\\!3_{\infty})=(1-\lambda)^{2}/N\leq(1-\lambda)N^{-1}$,

  * •

$Q_{t}(12\\!\dagger\\!3_{\infty},12\\!\dagger\\!3_{\infty})=(1-\lambda)$.

At time $t=t_{3}+1$, the seven relevant transition probabilities are :

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3_{\infty},1\\!\dagger\\!2\\!\dagger\\!3)=(1-\lambda)^{2}(1-1/N)(1-2/N)\leq(1-\lambda)^{2}$,

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3_{\infty},1\\!\dagger\\!23)=Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3_{\infty},13\\!\dagger\\!2)=Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3_{\infty},12\\!\dagger\\!3)\leq(1-\lambda)^{2}N^{-1}$,

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3_{\infty},123)=(1-\lambda)^{2}/N^{2}$.

  * •

$Q_{t}(12\\!\dagger\\!3_{\infty},12\\!\dagger\\!3)=(1-\lambda)(1-/N)\leq(1-\lambda),$

  * •

$Q_{t}(12\\!\dagger\\!3_{\infty},123)=(1-\lambda)/N$.

Finally, for $t\leq t_{3}$, the thirteen relevant transition probabilities are
:

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3,1\\!\dagger\\!2\\!\dagger\\!3)\leq(1-\lambda)^{3}$,

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3,12\\!\dagger\\!3)=Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3,1\\!\dagger\\!23)=Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3,2\\!\dagger\\!13)\leq(1-\lambda)^{3}N^{-1}$,

  * •

$Q_{t}(1\\!\dagger\\!2\\!\dagger\\!3,123)\leq(1-\lambda)^{3}N^{-2}$,

  * •

$Q_{t}(12\\!\dagger\\!3,12\\!\dagger\\!3)=Q_{t}(1\\!\dagger\\!23,1\\!\dagger\\!23)=Q_{t}(2\\!\dagger\\!13,2\\!\dagger\\!13)\leq(1-\lambda)^{2}$,

  * •

$Q_{t}(12\\!\dagger\\!3,123)=Q_{t}(1\\!\dagger\\!23,123)=Q_{t}(2\\!\dagger\\!13,123)\leq(1-\lambda)^{2}N^{-1}$,

  * •

$Q_{t}(123,1_{0}2_{0}3_{0})=1-Q_{t}(123,123)=\lambda$.

$1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}$$1\\!\dagger\\!2\\!\dagger\\!3_{\infty}$$12\\!\dagger\\!3_{\infty}$$1-\lambda$$1-\lambda$$\frac{1-\lambda}{N}$$(1-\lambda)^{2}$$\frac{(1-\lambda)^{2}}{N}$$1-\lambda$IIIII
Figure 4: Graph of the relevant transitions of the backward process $P$ (for
times $t>t_{3}+1$) used to compute a bound for
$I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow z_{2}\leftrightsquigarrow
z_{3}\\})$ when $t_{1}\geq t_{2}\geq t_{3}$. The starting node is
$1\\!\dagger\\!2_{\infty}\\!\dagger\\!2_{\infty}$ if $t_{1}>t_{2}>3$,
$1\\!\dagger\\!2\\!\dagger\\!3_{\infty}$ if $t_{1}=t_{2}>t_{3}$ and
$1\\!\dagger\\!2\\!\dagger\\!3$ (which appears in Figure 5) else. On each
edge, an upper bound of the corresponding transition probability is given. The
gray vertical line separates two temporal zones: zone II corresponds to times
$t_{3}+1<t\leq t_{2}+1$ and zone III corresponds to times $t>t_{2}+1$.
$1\\!\dagger\\!2\\!\dagger\\!3_{\infty}$$12\\!\dagger\\!3_{\infty}$$13\\!\dagger\\!2$$1\\!\dagger\\!23$$12\\!\dagger\\!3$$1\\!\dagger\\!2\\!\dagger\\!3$$123$$1_{0}2_{0}3_{0}$$(1-\lambda)^{2}$$\frac{(1-\lambda)^{2}}{N}$$1-\lambda$$(1-\lambda)^{2}$$\frac{(1-\lambda)^{2}}{N}$$\frac{(1-\lambda)^{2}}{N^{2}}$$1-\lambda$$\frac{1-\lambda}{N}$$(1-\lambda)^{2}$$\frac{(1-\lambda)^{2}}{N}$$(1-\lambda)^{3}$$\frac{(1-\lambda)^{3}}{N}$$\frac{(1-\lambda)^{3}}{N^{2}}$$1-\lambda$$\lambda$$1$III
Figure 5: Graph of the relevant transitions of the backward process $P$ (for
times $t\leq t_{3}+1$) used to compute a bound for
$I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow z_{2}\leftrightsquigarrow
z_{3}\\})$ when $t_{1}\geq t_{2}\geq t_{3}$. The starting node is
$1\\!\dagger\\!2_{\infty}\\!\dagger\\!2_{\infty}$ (which appears in Figure 4)
if $t_{1}>t_{2}>3$, $1\\!\dagger\\!2\\!\dagger\\!3_{\infty}$ if
$t_{1}=t_{2}>t_{3}$ and $1\\!\dagger\\!2\\!\dagger\\!3$ else. On some of the
edges, an upper bound of the corresponding transition probability is given
(the nodes $1\\!\dagger\\!23$, $13\\!\dagger\\!2$ and $12\\!\dagger\\!3$ play
almost the same role that is why we omit some the transition probabilities).
The gray vertical line separates two temporal zones: zone I corresponds to
times $t\leq t_{3}+1$ and zone II corresponds to times $t_{3}+1<t\leq
t_{2}+1$.

Looking at the Figures 4 and 5, the Markov property of $(P_{t})_{t\leq t_{1}}$
permits to compute the probabilities of ending in state $1_{0}2_{0}3_{0}$ from
any of the other states of the graph. These two figures highlight three time
zones denoted I, II and III, which are depicted from left to right
corresponding to the forward evolution of time (hence the arrows go from right
to left since they correspond to a backward dynamics).

In the following, we decompose the computation of the probability in Equation
(101) with respect to the three zones. To simplify the notation, let us denote
for all $t\in\mathbb{Z}$ and $\Theta\in\mathcal{P}_{3}$,
$p(t,\Theta)=I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}2_{0}3_{0}\,|\,P_{t}=\Theta\right)$.

For zone I, we have to consider initial conditions starting at time $t_{3}$
(see Figure 5). That is,

$p(t_{3},1\\!\dagger\\!23)=p(t_{3},12\\!\dagger\\!3)=p(t_{3},13\\!\dagger\\!2)\leq
K(1-\lambda)^{2}N^{-1},$

and

$p(t_{3},1\\!\dagger\\!2\\!\dagger\\!3)\leq
K(1-\lambda)^{3}\left[3p(t_{3},1\\!\dagger\\!23)N^{-1}+N^{-2}\right]\leq
K(1-\lambda)^{3}N^{-2}.$

For zone II, we have for all $t_{3}<t\leq t_{2}$,

$p(t,12\\!\dagger\\!3_{\infty})=(1-\lambda)^{t-t_{3}}[p(t_{3},12\\!\dagger\\!3)+N^{-1}p(t_{3},123)]\leq
K(1-\lambda)^{t-t_{3}}N^{-1},$

since we have to wait in the middle part during $t-t_{3}-1$ steps and then
take the edge from $12\\!\dagger\\!3_{\infty}$ to $12\\!\dagger\\!3$ or the
edge from $12\\!\dagger\\!3_{\infty}$ to $123$. Moreover, we have

$\displaystyle p(t_{2},1\\!\dagger\\!2\\!\dagger\\!3_{\infty})$
$\displaystyle\leq$ $\displaystyle
K\left\\{\sum_{k=1}^{t_{2}-t_{3}-1}(1-\lambda)^{2k}p(t_{2}-k,12\\!\dagger\\!3_{\infty})N^{-1}\right.$
$\displaystyle\left.+(1-\lambda)^{2(t_{2}-t_{3})}\left[3p(t_{3},1\\!\dagger\\!23)N^{-1}+p(t_{3},1\\!\dagger\\!2\\!\dagger\\!3)\right]\right\\}$
$\displaystyle\leq$ $\displaystyle
K\left\\{(1-\lambda)^{t_{2}-t_{3}}\sum_{k=1}^{t_{2}-t_{3}-1}(1-\lambda)^{k}N^{-2}+4(1-\lambda)^{2(t_{2}-t_{3})}(1-\lambda)^{2}N^{-2}\right\\}$
$\displaystyle\leq$ $\displaystyle K(1-\lambda)^{t_{2}-t_{3}}N^{-2}.$

Finally, using zone III, we have

$\displaystyle I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\leftrightsquigarrow z_{3}\\})$ $\displaystyle=$ $\displaystyle
p(t_{1},1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty})$ $\displaystyle=$
$\displaystyle(1-\lambda)^{t_{1}-t_{2}}[p(t_{2},1\\!\dagger\\!2\\!\dagger\\!3_{\infty})(1-1/2N)+p(t_{2},12\\!\dagger\\!3_{\infty})(2N)^{-1}],$

which ends the proof.

#### Proof of item (iii)

The backward process $(P_{t})_{-\infty<t\leq t_{1}}$ is naturally extended to
the case of four sites and we use the decomposition

$\\{z_{1}\leftrightsquigarrow z_{2}\\}\cap\\{z_{3}\leftrightsquigarrow
z_{4}\\}=\\{z_{1}\leftrightsquigarrow z_{2}\leftrightsquigarrow
z_{3}\leftrightsquigarrow z_{4}\\}\cup\\{z_{1}\leftrightsquigarrow
z_{2}\not\leftrightsquigarrow z_{3}\leftrightsquigarrow z_{4}\\}.$

Assume for now that $t_{1}>t_{2}\geq t_{3}\geq t_{4}$ so that
$P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}.$
It suffices to control

$I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}2_{0}3_{0}4_{0}\,|\,P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\right)\\\
+I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}2_{0}\\!\dagger\\!3_{0}4_{0}\,|\,P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\right).$

The list of the relevant transitions and the corresponding graphs are not
written here due to their huge size. However the computations can be made in
the same manner. For instance, one can prove that

$I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}2_{0}3_{0}4_{0}\,|\,P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\right)\leq
K(1-\lambda)^{t_{1}-t_{4}}N^{-3},$

and

$I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}2_{0}\\!\dagger\\!3_{0}4_{0}\,|\,P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\right)\leq
K(1-\lambda)^{(t_{1}-t_{2})+(t_{3}-t_{4})}N^{-2},$

which in turn give the desired result. Finally, the same upper bound holds if
$t_{1}>t_{2}$ is not satisfied by replacing the initial condition (for
instance $t_{1}=t_{2}>t_{3}\geq t_{4}$ corresponds to the initial condition
$P_{t_{1}}=1\\!\dagger\\!2\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}$ if
$z_{1}\neq z_{2}$).

#### Proof of item (iv)

We will only prove the case $t_{1}\wedge t_{2}>t_{3}\vee t_{4}$. The case in
which $t_{1}\wedge t_{2}=t_{3}\vee t_{4}$ follows along the same lines. In the
sequel, as before, we assume without loss of generality that $t_{1}\geq
t_{2}>t_{3}\geq t_{4}$ (the case where $t_{1}<t_{2}$ can be treated in the
same way). In the following, we use the backward process
$(P_{t})_{-\infty<t\leq t_{1}}$ with four sites like in the proof of item
(iii). Then, since $t_{3}+1\leq t_{2},$

$\\{z_{1}\leftrightsquigarrow z_{3}\leftrightsquigarrow
z_{4}\not\leftrightsquigarrow
z_{2}\\}\cap\\{\tau^{R}_{z_{2}}<t_{3}\\}\subset\\{z_{1}\leftrightsquigarrow
z_{3}\leftrightsquigarrow z_{4}\not\leftrightsquigarrow
z_{2}\\}\cap\\{\tau^{R}_{z_{2}}<t_{3}+1\\}\\\
=\left\\{P_{-\infty}=1_{0}3_{0}4_{0}\\!\dagger\\!2_{0}\right\\}\cap\\{P_{t_{3}+1}=1\\!\dagger\\!2\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\\}.$

Assume for now that $t_{1}>t_{2}$. It follows from the same arguments as above
that

$I\\!\\!P_{\theta}\left(P_{-\infty}=1_{0}3_{0}4_{0}\\!\dagger\\!2_{0}\,|\,P_{t_{3}+1}=1\\!\dagger\\!2\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\right)\leq
K(1-\lambda)^{t_{3}+1-t_{4}}N^{-2},$

and

$I\\!\\!P_{\theta}\left(P_{t_{3}+1}=1\\!\dagger\\!2\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\,|\,P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}\right)\leq(1-\lambda)^{t_{1}-(t_{3}+1)+t_{2}-(t_{3}+1)}.$

which in turn give the desired result.

Finally, if $t_{1}=t_{2}$, the same upperbound holds with the initial
condition
$P_{t_{1}}=1\\!\dagger\\!2_{\infty}\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}$
replaced by
$P_{t_{1}}=1\\!\dagger\\!2\\!\dagger\\!3_{\infty}\\!\dagger\\!4_{\infty}$. ∎

##  Appendix D Auxiliary results

The goal of this section is to prove Lemma 4.2. Before doing so, we recall
some classical results.

######  Lemma D.1.

Consider i.i.d. random variables $(B_{i})_{1\leq i\leq N}$ distributed as
$\text{Ber}(p)$ with $p\in[0,1]$ and let $\bar{B}=N^{-1}\sum_{i=1}^{N}B_{i}$.
Then, for each $\alpha\geq 1$ there exists a constant $C_{\alpha}>0$ depending
only on $\alpha$ such that

$I\\!\\!E\left[|\bar{B}-p|^{\alpha}\right]\leq C_{\alpha}N^{-\alpha/2}.$

###### Proof.

Hoeffding’s inequality implies that for any $x>0$,

$I\\!\\!P\left(|\bar{B}-p|\geq x\right)\leq 2e^{-2x^{2}N}.$

Now, using the tail sum formula for the expectation one can show that

$I\\!\\!E\left[|\bar{B}-p|^{\alpha}\right]\leq
2\alpha\int_{0}^{\infty}x^{\alpha-1}e^{-2x^{2}N}dx=\alpha\sqrt{\frac{2\pi}{N}}I\\!\\!E\left[Z^{\alpha-1}1_{Z>0}\right],$

where $Z\sim\mathcal{N}(0,(4N)^{-1})$. Finally, by observing that
$Y=(4N)^{1/2}Z\sim\mathcal{N}(0,1)$, it follows that

$\alpha\sqrt{\frac{2\pi}{N}}I\\!\\!E\left[Z^{\alpha-1}1_{Z>0}\right]=\alpha\frac{\sqrt{2\pi}}{2^{\alpha-1}}N^{-\alpha/2}I\\!\\!E\left[Y^{\alpha-1}1_{Y>0}\right],$

and the result follows. ∎

######  Lemma D.2.

Let $(B_{ij})_{1\leq i,j\leq N}$ be i.i.d. random variables distributed as
$\text{Ber}(p)$ with $p\in[0,1]$. For $U,V\subset[N]$ and
$\epsilon\in\\{-1,1\\}$, define $W_{i}=\epsilon N^{-1}\sum_{j\in V}B_{ij}$ for
$i\in U$, and $W_{i}=0$, for $i\in U^{c}$. Denote $W=(W_{1},\ldots,W_{N})$.
Then, for any $\alpha\geq 1$, there exists a constant $K_{\alpha}>0$ depending
only on $\alpha$ such that

$I\\!\\!E\left(\|W-p\epsilon|V|N^{-1}1_{U}\|^{\alpha}_{2}\right)\leq
K_{\alpha}\left(\frac{|U|}{|V|}\right)^{\alpha/2}.$ (102)

###### Proof.

Jensen inequality combined with Lemma D.1 and the fact that $|V|/N\leq 1$
implies that for any $\alpha\geq 1$,

$I\\!\\!E\left[\|W-p\epsilon|V|N^{-1}1_{U}\|^{2\alpha}_{2}\right]\leq|U|^{\alpha}\left(|V|N^{-1}\right)^{2\alpha}(C_{2\alpha}|V|^{-(2\alpha)/2})\leq
C_{2\alpha}\left(\frac{|U|}{|V|}\right)^{\alpha},$

where the constant $C_{2\alpha}$ is the one of Lemma D.1. By using Jensen
inequality once more, we deduce that

$I\\!\\!E\left[\|W-p\epsilon|V|N^{-1}1_{U}\|^{\alpha}_{2}\right]\leq\left(I\\!\\!E\left[\|W-p\epsilon|V|N^{-1}1_{U}\|^{2\alpha}_{2}\right]\right)^{1/2},$

and the result follows, putting $K_{\alpha}:=C_{2\alpha}^{1/2}.$ ∎

######  Lemma D.3.

Consider a vector $v=(v_{1},\ldots,v_{N})\in\mathbb{R}^{N}$ supported on
$S\subseteq\\{1,\ldots,N\\}$. For any $\xi\in\mathbb{R}$, the following
inequality holds:

$\left\|v-\frac{N}{|S|}\bar{v}1_{S}\right\|_{2}\leq\left\|v-\xi
1_{S}\right\|_{2},$ (103)

where $\bar{v}=N^{-1}\sum_{i=1}^{N}v_{i}=N^{-1}\sum_{i\in S}v_{i}$.

###### Proof.

Consider $U\sim\text{Unif}(\\{v_{i}:i\in S\\})$, and observe that
$I\\!\\!E[U]=\bar{v}N|S|^{-1}$ and
$\operatorname{\mathop{Var}}(U)=|S|^{-1}\|v-N|S|^{-1}\bar{v}1_{S}\|_{2}$.
Since
$\operatorname{\mathop{Var}}(U)=\inf_{\xi\in\mathbb{R}}I\\!\\!E\left[(U-\xi)^{2}\right]$,
it follows that for any $\xi\in\mathbb{R}$,

$|S|^{-1}\left\|v-\frac{N}{|S|}\bar{v}1_{S}\right\|_{2}\leq
I\\!\\!E\left[(U-\xi)^{2}\right]=|S|^{-1}\left\|v-\xi 1_{S}\right\|_{2},$

implying the result. ∎

We are now in position of proving Lemma 4.2.

###### Proof of Lemma 4.2.

Inequalities (47) and (48) follow from Lemmas D.3 and D.2 respectively. Next,
we prove Inequality (46). First, note that
$(r^{N}_{a}r^{N}_{b})^{-1}\overline{L^{N,ab}}-(bp)=b(|\mathcal{P}_{a}||\mathcal{P}_{b}|)^{-1}\sum_{i\in\mathcal{P}_{a}}\sum_{j\in\mathcal{P}_{b}}(\theta_{ij}-p),$
so that

$\displaystyle
I\\!\\!E\left[\left|(r^{N}_{a}r^{N}_{b})^{-1}\overline{L^{N,ab}}-(bp)\right|^{2}\right]$
$\displaystyle=\operatorname{\mathop{Var}}\left((|\mathcal{P}_{a}||\mathcal{P}_{b}|)^{-1}\sum_{i\in\mathcal{P}_{a}}\sum_{j\in\mathcal{P}_{b}}\theta_{ij}\right)$
$\displaystyle=p(1-p)(|\mathcal{P}_{a}||\mathcal{P}_{b}|)^{-1}=N^{-2}p(1-p)(r^{N}_{a}r^{N}_{b})^{-1}.$

Since $p(1-p)\leq 1/4$, we conclude that

$I\\!\\!E\left[\left|(r^{N}_{a}r^{N}_{b})^{-1}\overline{L^{N,ab}}-(bp)\right|^{2}\right]\leq(N^{-2}/4)\max_{c,d}\\{(r^{N}_{c}r^{N}_{d})^{-1}\\}.$

Similar arguments can be applied to show that

$I\\!\\!E\left[\left|(r^{N}_{a}r^{N}_{b})^{-1}\overline{C^{N,ab}}-(ap)\right|^{2}\right]\leq(N^{-2}/4)\max_{c,d}\\{(r^{N}_{c}r^{N}_{d})^{-1}\\}.$

Inequality (46) follows then from the above inequalities and from the fact
that $x\vee y\leq x+y$ for any $x,y\geq 0$.

We now establish Inequality (45). To that end, first observe that

$I\\!\\!E\left[\|A^{N}\widetilde{L}^{N,ab}\|_{2}^{2}\right]=NI\\!\\!E\left[\left(\sum_{j\in\mathcal{P}_{a}}A^{N}(1,j)\widetilde{L}^{N,ab}_{j}\right)^{2}\right].$

Next, observe that if we denote $Z_{j}=L^{N,ab}_{j}-(pb)r^{N}_{b}$ for
$j\in\mathcal{P}_{a}$, then we can write (recall that
$A^{N}(i,j)=bN^{-1}\theta_{ij}$ for $1\leq i\leq N$ and
$j\in\mathcal{P}_{b}$),

$\left[\sum_{j\in\mathcal{P}_{a}}A^{N}(1,j)\widetilde{L}^{N,ab}_{j}\right]^{2}=N^{-2}\left[\sum_{j\in\mathcal{P}_{a}}\theta_{1j}Z_{j}-\left(\sum_{j\in\mathcal{P}_{a}}\theta_{1j}\right)\left(|\mathcal{P}_{a}|^{-1}\sum_{k\in\mathcal{P}_{a}}Z_{k}\right)\right]^{2},$

so that by applying Jensen’s inequality we obtain that

$I\\!\\!E\left[\|A^{N}\widetilde{L}^{N,ab}\|_{2}^{2}\right]\leq\\\
2N^{-1}\left[I\\!\\!E\left[\left(\sum_{j\in\mathcal{P}_{a}}\theta_{1j}Z_{j}\right)^{2}\right]+I\\!\\!E\left[\left(\sum_{j\in\mathcal{P}_{a}}\theta_{1j}\right)^{2}\left(|\mathcal{P}_{a}|^{-1}\sum_{k\in\mathcal{P}_{a}}Z_{k}\right)^{2}\right]\right].$

By observing that $\sum_{j\in\mathcal{P}_{a}}\theta_{1j}\leq|\mathcal{P}_{a}|$
and using that $(Z_{k})_{k\in\mathcal{P}_{b}}$ are i.i.d. centered random
variables, one can check that

$\displaystyle
I\\!\\!E\left[\left(\sum_{j\in\mathcal{P}_{a}}\theta_{1j}\right)^{2}\left(|\mathcal{P}_{a}|^{-1}\sum_{k\in\mathcal{P}_{a}}Z_{k}\right)^{2}\right]$
$\displaystyle\leq|\mathcal{P}_{a}|^{2}\operatorname{\mathop{Var}}\left(|\mathcal{P}_{a}|^{-1}\sum_{k\in\mathcal{P}_{a}}Z_{k}\right)$
$\displaystyle=|\mathcal{P}_{a}|\operatorname{\mathop{Var}}\left(Z_{k}\right)=r^{N}_{a}r^{N}_{b}p(1-p)\leq
1/4$

where in the last inequality we have used that
$\operatorname{\mathop{Var}}\left(Z_{k}\right)=\operatorname{\mathop{Var}}\left(L^{N,ab}_{k}\right)=|\mathcal{P}_{b}|p(1-p)/N^{2}$.

Now, note that if $j\neq 1,$ then $\theta_{1j}$ is independent of $Z_{j}$ so
that $I\\!\\!E[\theta_{1j}Z_{j}]=I\\!\\!E[\theta_{1j}]I\\!\\!E[Z_{j}]=0$,
because $Z_{j}$ is centered. Moreover, one can check that
$(\theta_{1j}Z_{j})_{j\in\mathcal{P}_{a}:j\neq 1}$ are independent. Hence, by
combining these facts with Jensen’s inequality, we can deduce that

$\displaystyle
I\\!\\!E\left[\left(\sum_{j\in\mathcal{P}_{a}}\theta_{1j}Z_{j}\right)^{2}\right]$
$\displaystyle\leq 2\left[I\\!\\!E\left[\left(\sum_{j\in\mathcal{P}_{a}:j\neq
1}\theta_{1j}Z_{j}\right)^{2}\right]+I\\!\\!E\left[\left(\theta_{11}Z_{1}\right)^{2}\right]\right]$
$\displaystyle\leq
2\left[\operatorname{\mathop{Var}}\left(\sum_{j\in\mathcal{P}_{a}:j\neq
1}\theta_{1j}Z_{j}\right)+\operatorname{\mathop{Var}}\left(Z_{1}\right)\right]$
$\displaystyle\leq
2\left[(|\mathcal{P}_{a}|-1)p\operatorname{\mathop{Var}}(Z_{j})+\operatorname{\mathop{Var}}\left(Z_{1}\right)\right]\leq
2r^{N}_{a}r^{N}_{b}p(1-p)\leq 1/2.$

Combining the last three inequalities we show that Inequality (45) holds.

It remains to show Inequality (44). In the case $a_{1}\neq a$, the two vectors
$\widetilde{L}^{N,ab}$ and $\widetilde{L}^{N,ab_{1}}$ have disjoint supports
so that $\left<\widetilde{L}^{N,ab},\widetilde{L}^{N,a_{1}b_{1}}\right>=0$
a.s. and (45) trivially holds.

From now on, consider that $a_{1}=a$. Let us denote
$Y^{N}_{a,bb_{1}}=\left<\widetilde{L}^{N,ab},\widetilde{L}^{N,ab_{1}}\right>$
and prove that there exists a universal constant $K$ such that

$I\\!\\!E\left[\left|Y^{N}_{a,bb_{1}}-\delta_{bb_{1}}r_{a}r_{b}p(1-p)\right|^{2}\right]\leq
K\left[I\\!\\!E\left[\left|Y^{N}_{a,bb_{1}}-I\\!\\!E\left[Y^{N}_{a,bb_{1}}\right]\right|^{2}\right]+N^{-2}\right].$
(104)

First, consider the case $b_{1}\neq b$. Remind that $L^{N,ab}$ only depends on
$\\{\theta_{ij},i\in\mathcal{P}_{a},j\in\mathcal{P}_{b}\\}$, and remark that
$\mathbb{E}\left[\widetilde{L}^{N,ab}_{i}\right]=0$ for all $i=1,\dots,N$. In
particular, the vectors $L^{N,ab}$ and $L^{N,ab_{1}}$ are independent and

$I\\!\\!E\left[Y^{N}_{a,bb_{1}}\right]=\left<I\\!\\!E\left[\widetilde{L}^{N,ab}\right],I\\!\\!E\left[\widetilde{L}^{N,ab_{1}}\right]\right>=0,$

so that

$I\\!\\!E\left[\left|Y^{N}_{a,bb_{1}}-\delta_{bb_{1}}r_{a}r_{b}p(1-p)\right|^{2}\right]=I\\!\\!E\left[\left|Y^{N}_{a,bb_{1}}-I\\!\\!E\left[Y^{N}_{a,bb_{1}}\right]\right|^{2}\right].$

Now, consider the case $b_{1}=b$. In that case,
$Y^{N}_{a,bb_{1}}=\|\widetilde{L}^{N,ab}\|_{2}^{2}$, and observe that

$I\\!\\!E\left[\|\widetilde{L}^{N,ab}\|_{2}^{2}\right]=|\mathcal{P}_{a}|I\\!\\!E\left[\left(L_{i}^{N,ab}-\frac{1}{|\mathcal{P}_{a}|}\sum_{j\in\mathcal{P}_{a}}L_{j}^{N,ab}\right)^{2}\right],$

for any fixed $i\in\mathcal{P}_{a}$. Next, observe that if we denote
$Z_{j}=L^{N,ab}_{j}-(pb)r^{N}_{b}$ for $j\in\mathcal{P}_{a}$, then we can
write

$I\\!\\!E\left[\|\widetilde{L}^{N,ab}\|_{2}^{2}\right]=|\mathcal{P}_{a}|I\\!\\!E\left[\left(Z_{i}(1-|\mathcal{P}_{a}|^{-1})-\frac{1}{|\mathcal{P}_{a}|}\sum_{j\in\mathcal{P}_{a}:j\neq
i}Z_{j}\right)^{2}\right].$

Since $(Z_{j})_{j\in\mathcal{P}_{a}}$ are i.i.d. centered random variables,
one can check that

$I\\!\\!E\left[\left(Z_{i}(1-|\mathcal{P}_{a}|^{-1})-\frac{1}{|\mathcal{P}_{a}|}\sum_{j\in\mathcal{P}_{a}:j\neq
i}Z_{j}\right)^{2}\right]=(1-|\mathcal{P}_{a}|^{-1})\operatorname{\mathop{Var}}(Z_{i}),$

which, in turn, implies that

$\displaystyle I\\!\\!E\left[\|\widetilde{L}^{N,ab}\|_{2}^{2}\right]$
$\displaystyle=|\mathcal{P}_{a}|(1-|\mathcal{P}_{a}|^{-1})\operatorname{\mathop{Var}}(Z_{i})$
$\displaystyle=|\mathcal{P}_{a}|(1-|\mathcal{P}_{a}|^{-1})\operatorname{\mathop{Var}}(L^{N,ab}_{i})=r^{N}_{a}r^{N}_{b}p(1-p)-N^{-1}r^{N}_{b}p(1-p).$

Combining the above identity with Jensen’s inequality (and the assumption we
made on the sequence of fractions $r_{a}^{N}$), we deduce that Equation (104)
is satisfied.

Next, notice that $(|\mathcal{P}_{a}|-1)^{-1}Y^{N}_{a,bb_{1}}$ can be seen as
the empirical covariance of the random variables $L^{N,ab}_{i}$ and
$L^{N,ab_{1}}_{i}$, $i\in\mathcal{P}_{a}$. In this perspective,
$I\\!\\!E\left[\left|Y^{N}_{a,bb_{1}}-I\\!\\!E\left[Y^{N}_{a,bb_{1}}\right]\right|^{2}\right]$
corresponds to $(|\mathcal{P}_{a}|-1)^{2}$ times the variance of the empirical
covariance of the random variables $L^{N,ab}_{i}$ and $L^{N,ab_{1}}_{i}$,
$i\in\mathcal{P}_{a}$. It is well-known (e.g., see page 363, exercise 7.45b,
Casella and Berger for the case of the empirical variance) that the latter is
equal to

$|\mathcal{P}_{a}|^{-1}\left(\mu_{2,2}+\frac{\mu_{2,0}\mu_{0,2}-(|\mathcal{P}_{a}|-2)\mu_{1,1}^{2}}{|\mathcal{P}_{a}|-1}\right),$

where
$\mu_{r,s}=I\\!\\!E\left[\left(L^{N,ab}_{i}-I\\!\\!E\left[L^{N,ab}_{i}\right]\right)^{r}\left(L^{N,ab_{1}}_{i}-I\\!\\!E\left[L^{N,ab_{1}}_{i}\right]\right)^{s}\right]$.
We have
$\mu_{2,2}=I\\!\\!E[(L^{N,ab}_{i}-bpr^{N}_{b})^{2}(L^{N,ab_{1}}_{i}-b_{1}pr^{N}_{b_{1}})^{2}]\leq(I\\!\\!E[(L^{N,ab}_{i}-bpr^{N}_{b})^{4}]I\\!\\!E[(L^{N,ab_{1}}_{i}-b_{1}pr^{N}_{b_{1}})^{4}])^{1/2}$
by Cauchy Schwarz inequality,
$\mu_{2,0}=I\\!\\!E[(L^{N,ab}_{i}-bpr^{N}_{b})^{2}]$,
$\mu_{0,2}=I\\!\\!E[(L^{N,ab_{1}}_{i}-b_{1}pr^{N}_{b_{1}})^{2}]$ and
$\mu_{1,1}=\delta_{bb_{1}}I\\!\\!E[(L^{N,ab}_{i}-bpr^{N}_{b})^{2}]$ by
independence. By applying Lemma D.1, we know that, for some universal constant
$K$, $\mu_{2,2}\leq KN^{-2}$ and $\mu_{2,0}+\mu_{0,2}+\mu_{1,1}\leq KN^{-1}$.
Finally,

$I\\!\\!E\left[\left|Y^{N}_{a,bb_{1}}-I\\!\\!E\left[Y^{N}_{a,bb_{1}}\right]\right|^{2}\right]\leq
K\frac{(|\mathcal{P}_{a}|-1)^{2}}{|\mathcal{P}_{a}|}\left(N^{-2}+\frac{N^{-2}-(|\mathcal{P}_{a}|-2)N^{-2}}{|\mathcal{P}_{a}|-1}\right)\leq
KN^{-1},$

which implies the result. ∎

######  Lemma D.4.

Let $(V^{N})_{N},(v^{N})_{N}$ be two sequences of random vectors such that for
all $N>0$, $V^{N},v^{N}\in\mathbb{R}^{N}$. Assume that there exists a constant
$K$ such that, for all $N>0$, $v^{N}$ can be written as
$v^{N}=v_{1}^{N}+v_{2}^{N}+v_{3}^{N}$, where
$v_{1}^{N},v_{2}^{N},v_{3}^{N}\in\mathbb{R}^{N}$, and

  1. (i)

$\mathbb{E}\left[||V^{N}||_{2}^{4}\right]+||V^{N}||_{\infty}^{2}\leq K$ almost
surely,

  2. (ii)

$\mathbb{E}\left[||v_{1}^{N}||_{1}^{2}+||v_{2}^{N}||_{2}^{4}\right]^{2}\leq
K$,

  3. (iii)

$\mathbb{E}\left[\langle V^{N},v_{3}^{N}\rangle^{2}\right]\leq K$.

Then, for all $N>0$,

$\mathbb{E}\left[\langle V^{N},v^{N}\rangle^{2}\right]\leq 3(K^{2}+2K).$

###### Proof.

First, by Holder inequality,

$\mathbb{E}\left[\langle
V^{N},v^{N}_{1}\rangle^{2}\right]\leq\mathbb{E}\left[\|V^{N}\|_{\infty}^{2}\|v_{1}^{N}\|_{1}^{2}\right]\leq
K\mathbb{E}\left[\|v_{1}^{N}\|_{1}^{2}\right]\leq K^{2}.$

Secondly, using Cauchy Schwarz inequality twice, we have

$\mathbb{E}\left[\langle
V^{N},v^{N}_{2}\rangle^{2}\right]\leq\mathbb{E}\left[\|V^{N}\|_{2}^{2}\|v_{2}^{N}\|_{2}^{2}\right]\leq\left(\mathbb{E}\left[\|V^{N}\|_{2}^{4}\right]\mathbb{E}\left[\|v_{2}^{N}\|_{2}^{4}\right]\right)^{1/2}\leq
K.$

And, $\mathbb{E}\left[\langle V^{N},v_{3}^{N}\rangle^{2}\right]\leq K$ by
assumption. Finally, we conclude by combining those three inequalities thanks
to the convexity of the square function. ∎

######  Lemma D.5.

Let $a\in\\{-,+\\}$. Let $(u^{N})_{N},(v^{N})_{N}$ be two sequences of random
vectors such that for all $N>0$, $u^{N},v^{N}\in\mathbb{R}^{N}$ are supported
in $\mathcal{P}_{a}$. Assume that there exist two random variables
$u_{3}^{\infty},v_{3}^{\infty}\in\mathbb{R}$ and a constant $K$ such that, for
all $N>0$, $u^{N}=u_{1}^{N}+u_{2}^{N}+u_{3}^{N}1_{\mathcal{P}_{a}}$ and
$v^{N}=v_{1}^{N}+v_{2}^{N}+v_{3}^{N}1_{\mathcal{P}_{a}}$, where
$u_{1}^{N},u_{2}^{N},v_{1}^{N},v_{2}^{N}\in\mathbb{R}^{N}$ are random vectors
supported in $\mathcal{P}_{a}$ and $u_{3}^{N},v_{3}^{N}\in\mathbb{R}$ are
random variables, and

  * •

$\mathbb{E}\left[||u_{1}^{N}||_{1}^{2}+||u_{2}^{N}||_{2}^{4}+N^{2}(\overline{u_{2}^{N}}^{2})\right]+||u_{1}^{N}+u_{2}^{N}||_{\infty}+|u_{3}^{N}|+|u_{3}^{\infty}|\leq
K$ almost surely,

  * •

$\mathbb{E}\left[||v_{1}^{N}||_{1}^{2}+||v_{2}^{N}||_{2}^{4}+N^{2}(\overline{v_{2}^{N}})^{2}\right]+||v_{1}^{N}+v_{2}^{N}||_{\infty}+|v_{3}^{N}|+|v_{3}^{\infty}|\leq
K$ almost surely,

  * •

$\mathbb{E}\left[|u_{3}^{N}-u_{3}^{\infty}|^{2}+|v_{3}^{N}-v_{3}^{\infty}|^{2}\right]\leq
KN^{-2}$.

Then, there exists an other constant $K$ (independent of $N$) such that, for
all $N>0$,

$\mathbb{E}\left[\left|\frac{1}{N}\langle u^{N},v^{N}\rangle-
r_{a}v_{3}^{\infty}u_{3}^{\infty}\right|^{2}\right]\leq KN^{-2}.$

###### Proof.

Since $u^{N}$ and $v^{N}$ are decomposed into three vectors each, there are
naturally nine contributions to the scalar product $\langle
u^{N},v^{N}\rangle$.

Let us first prove that the limit term $r_{a}v_{3}^{\infty}u_{3}^{\infty}$
comes from the contribution of the constant parts, i.e. $N^{-1}\langle
u^{N}_{3}1_{\mathcal{P}_{a}},v^{N}_{3}1_{\mathcal{P}_{a}}\rangle=r_{a}^{N}u_{3}^{N}v_{3}^{N}$.
Indeed,

$\displaystyle\left|r_{a}^{N}u_{3}^{N}v_{3}^{N}-r_{a}v_{3}^{\infty}u_{3}^{\infty}\right|$
$\displaystyle\leq$
$\displaystyle\left|r_{a}^{N}u_{3}^{N}\right|\left|v_{3}^{N}-v_{3}^{\infty}\right|+\left|r_{a}^{N}v_{3}^{\infty}\right|\left|u_{3}^{N}-u_{3}^{\infty}\right|+\left|u_{3}^{\infty}v_{3}^{\infty}\right|\left|r_{a}^{N}-r_{a}\right|$
$\displaystyle\leq$ $\displaystyle
K\left(\left|u_{3}^{N}-u_{3}^{\infty}\right|+\left|v_{3}^{N}-v_{3}^{\infty}\right|\right)+K^{2}\left|r_{a}^{N}-r_{a}\right|,$

so that $\mathbb{E}\left[\left|N^{-1}\langle
u^{N}_{3}1_{\mathcal{P}_{a}},v^{N}_{3}1_{\mathcal{P}_{a}}\rangle-
r_{a}v_{3}^{\infty}u_{3}^{\infty}\right|^{2}\right]\leq KN^{-2}$ thanks to the
assumptions on $u_{3}^{N}$, $v_{3}^{N}$ and the sequence of fractions
$r_{a}^{N}$.

Then, let us prove that the other contributions to the scalar product are
negligible. We have,

$\mathbb{E}\left[\left<u_{1}^{N},v^{N}\right>^{2}\right]\leq\mathbb{E}\left[\|u_{1}^{N}\|_{1}^{2}\|v^{N}\|_{\infty}^{2}\right]\leq
K\mathbb{E}\left[\|u_{1}^{N}\|_{1}^{2}\right]\leq K,$

and similarly $\mathbb{E}\left[\left<u^{N},v_{1}^{N}\right>^{2}\right]\leq K$.
Applying Cauchy-Schwarz inequality twice, we have

$\mathbb{E}\left[\left<u_{2}^{N},v_{2}^{N}\right>^{2}\right]\leq\mathbb{E}\left[\|u_{2}^{N}\|_{2}^{2}\|v_{2}^{N}\|_{2}^{2}\right]\leq\left(\mathbb{E}\left[\|u_{2}^{N}\|_{2}^{4}\right]\mathbb{E}\left[\|v_{2}^{N}\|_{2}^{4}\right]\right)^{1/2}\leq
K.$

Finally,

$\mathbb{E}\left[\left<u_{2}^{N},v_{3}^{N}1_{\mathcal{P}_{a}}\right>^{2}\right]\leq|v_{3}^{N}|^{2}\mathbb{E}\left[\left<u_{2}^{N},1_{\mathcal{P}_{a}}\right>^{2}\right]=|v_{3}^{N}|^{2}\mathbb{E}\left[N^{2}(\overline{u_{2}^{N}})^{2}\right]\leq
K,$

and similarly
$\mathbb{E}\left[\left<u_{3}^{N}1_{\mathcal{P}_{a}},v_{2}^{N}\right>^{2}\right]\leq
K$. Finally, we conclude by combining all those inequalities thanks to the
convexity of the square function. ∎

##  Appendix E Inversion of $\Psi$

The main objective of this section is to prove Proposition 2.2 and in
particular provide the expressions of $\Phi^{(+)}$ and $\Phi^{(-)}$. Here is a
collection of obvious preliminary results on the functions $D$ and $\Psi$.

######  Proposition E.1.

For all $(\mu,\lambda,p)\in\Lambda$,

  1. 1.

if $r_{+}<1/2$, then $1<D(\lambda,p)<2r_{-}$;

  2. 2.

if $r_{+}=1/2$, then $D(\lambda,p)=1$;

  3. 3.

if $r_{+}>1/2$, then $2r_{-}<D(\lambda,p)<1$.

Whatever the value of $r_{+}$, the image $\Psi(\Lambda)$ is included in
$(0,1)\times(0,\infty)^{2}$.

Finally, if $r_{+}=1/2$, then
$\Psi_{3}(\mu,\lambda,p)>\Psi_{1}(\mu,\lambda,p)[1-\Psi_{1}(\mu,\lambda,p)]$,
for all $(\mu,\lambda,p)\in\Lambda$.

###### Proof.

The statements regarding the function $D$ are obvious.

Let $r_{+}\in[0,1]$ and $(\mu,\lambda,p)\in\Lambda$. For all $k=1,2,3$, the
fact that $\Psi_{k}(\mu,\lambda,p)\in(0,\infty)$ is obvious. It only remains
to prove that $\Psi_{1}(\mu,\lambda,p)<1$. This follows from

$D(\lambda,p)-(\mu+(1-\lambda)pr_{-})=1-\mu-(1-\lambda)pr_{+}>1-\lambda-(1-\lambda)r_{+}=(1-\lambda)pr_{-}\geq
0.$

Finally, if $r_{+}=1/2$, we have $D(\lambda,p)=1$ and the statement follows
from the fact that $1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}>1$. ∎

From now on, the objective is to invert the function $\Psi$ on the set of
admissible parameters $\Lambda$. In view of Proposition E.1, it suffices to
find this inverse function on the set

$\mathcal{M}=\begin{cases}(0,1)\times(0,\infty)^{2},&\text{ if }r_{+}\neq
1/2;\\\ \left\\{(m,v,w)\in(0,1)\times(0,\infty)^{2}:w>m(1-m)\right\\},&\text{
if }r_{+}=1/2.\\\ \end{cases}$ (105)

Hence, from now on, $(m,v,w)$ will denote an arbitrary vector in $\mathcal{M}$
(in particular, it is not related to $(\mu,\lambda,p)$). Remind the function
$\kappa$ defined in Equation 9 and let us define
$d^{(+)},d^{(-)}:(0,1)\times(0,\infty)\to\mathbb{R}$ by, for $a\in\\{-,+\\}$,

$d^{(a)}(m,w)=\begin{cases}\frac{4r_{+}r_{-}+a\sqrt{(4r_{+}r_{-})^{2}-4r_{+}r_{-}+\kappa(m,w)}}{4r_{+}r_{-}-\kappa(m,w)},&\
\text{if}\ \kappa(m,w)\neq 4r_{+}r_{-};\\\ (8r_{+}r_{-})^{-1},&\ \text{if}\
\kappa(m,w)=4r_{+}r_{-}.\\\ \end{cases}$ (106)

If $(\mu,\lambda,p)$ and $(m,v,w)$ are related through (6) then the value
$d^{(a)}(m,w)$ is a candidate to be equal to $D(\lambda,p)$. The values
$d^{(+)}(m,w)$ and $d^{(-)}(m,w)$ are the roots of the quadratic equation

$[4r_{+}r_{-}-\kappa(m,w)]u^{2}-(8r_{+}r_{-})u+1=0.$ (107)

On the one hand, $d^{(-)}$ is positive and one can check that it is $C^{1}$
(even when $\kappa(m,w)\to 4r_{+}r_{-}$). On the other hand, $d^{(+)}$ may be
negative and goes to infinity when $\kappa(m,w)\to 4r_{+}r_{-}$. In view of
this remark and Proposition E.1, $d^{(-)}(m,w)$ is expected to be the good
candidate to be equal to $D(\lambda,p)$. This is true most of the time but not
for the whole range of parameters (see Proposition E.3 below).

From now on, let $a\in\\{-,+\\}$. Let us define
$\phi_{1}^{(a)}:(0,1)\times(0,\infty)\to(0,\infty)$ by

$\phi_{1}^{(a)}(m,w)=\begin{cases}w[m(1-m)]^{-1}-1,&\ \text{if}\ r_{+}=1/2;\\\
[1-d^{(a)}(m,w)]^{2}(r_{+}-r_{-})^{-2},&\ \text{else},\\\ \end{cases}$ (108)

and $\phi_{2}^{(a)}:\mathcal{M}\to(1,\infty)$ by

$\phi_{2}^{(a)}(m,v,w)=1+\frac{v}{[(m-r_{-})^{2}+r_{+}r_{-}]\phi_{1}^{(a)}(m,w)}.$
(109)

If $(\mu,\lambda,p)$ and $(m,v,w)$ are related through (6) then
$\phi_{1}^{(a)}(m,w)$ (_respectively_ $\phi_{2}^{(a)}(m,v,w)$) are two
candidates to be equal to $(1-\lambda)^{2}p^{2}$ (_resp._ $p^{-1}$). Then, for
$k=1,2,3$, let $\Phi_{k}^{(a)}:\mathcal{M}\to\mathbb{R}$ be defined by,

$\begin{cases}\Phi_{1}^{(a)}(m,v,w)=m\left(1-(r_{+}-r_{-})\sqrt{\phi_{1}^{(a)}(m,w)}\right)-r_{-}\sqrt{\phi_{1}^{(a)}(m,w)},\\\
\Phi_{2}^{(a)}(m,v,w)=1-\phi_{2}^{(a)}(m,v,w)\sqrt{\phi_{1}^{(a)}(m,w)},\\\
\Phi_{3}^{(a)}(m,v,w)=(\phi_{2}^{(a)}(m,v,w))^{-1}.\end{cases}$ (110)

Finally, let $\Phi^{(a)}:\mathcal{M}\to\mathbb{R}^{3}$ be defined by the three
coordinate functions above. Let us remark that all the functions involved in
the definition of $\Phi^{(a)}$ are obviously $C^{\infty}$ except $d^{(a)}$.
Nevertheless, it is easy to check that $d^{(-)}$ is (at least) $C^{1}$ when
$\kappa(m,w)\to 4r_{+}r_{-}$. In turn, $\Phi^{(-)}$ is regular.

######  Proposition E.2.

The function $\Phi^{(-)}$ is $C^{1}$.

Finally, the functions $\Phi^{(+)}$ and $\Phi^{(-)}$ are related with $\Psi$
in the following sense.

######  Proposition E.3.

Whatever the value of $r_{+}$, the following results hold.

  1. 1.

For all $(\mu,\lambda,p)\in\Lambda$,
$(\mu,\lambda,p)\in\\{\Phi^{(+)}\circ\Psi(\mu,\lambda,p),\Phi^{(-)}\circ\Psi(\mu,\lambda,p)\\}$,

  2. 2.

moreover, if $r_{+}\geq 1/2$ or

$\kappa(\Psi_{1}(\mu,\lambda,p),\Psi_{3}(\mu,\lambda,p))\geq 4r_{+}r_{-},$

then $(\mu,\lambda,p)=\Phi^{(-)}\circ\Psi(\mu,\lambda,p)$.

  3. 3.

Let $(m,v,w)\in\mathcal{M}$ and $a\in\\{-,+\\}$. If
$\Phi^{(a)}(m,v,w)\in\Lambda$ and

$r_{+}=1/2\text{ or
}\operatorname{sgn}(1-d^{(a)}(m,w))=\operatorname{sgn}(r_{+}-r_{-}),$

then $\Psi\circ\Phi^{(a)}(m,v,w)=(m,v,w)$.

######  Remark E.4.

Remind that $d^{(a)}(m,w)$ is a candidate for $D(\lambda,p)$. In that regard,
the sign condition appearing in item 2 above is consistent with Proposition
E.1.

###### Proof.

#### Proof of 1

Let $(\mu,\lambda,p)\in\Lambda$. In the following, we shorten the notation
$\Psi_{k}=\Psi_{k}(\mu,\lambda,p)$. Finally, we denote
$\Psi=(\Psi_{1},\Psi_{2},\Psi_{3})\in\mathbb{R}^{3}$.

Let us first consider the case $r_{+}=1/2$. In that case, the $\phi$ and
$\Phi$ functions do not depend on the superscript $a\in\\{-,+\\}$ and we omit
it in the following. Furthermore, $D(\lambda,p)=1$ and $\Psi$ reduces to

$\begin{cases}\Psi_{1}=\mu+(1-\lambda)p/2\\\
\Psi_{2}=(1-\lambda)^{2}p(1-p)[(\Psi_{1}-1/2)^{2}+1/4]\\\
\Psi_{3}=\Psi_{1}[1-\Psi_{1}][1+(1-\lambda)^{2}p^{2}].\end{cases}$

Hence, it is easy to check that
$\phi_{1}(\Psi_{1},\Psi_{3})=(1-\lambda)^{2}p^{2}$ and

$\phi_{2}(\Psi)=1+\frac{(1-\lambda)^{2}p(1-p)[(\Psi_{1}-1/2)^{2}+1/4]}{[(\Psi_{1}-1/2)^{2}+1/4](1-\lambda)^{2}p^{2}}=1+\frac{1-p}{p}=\frac{1}{p}.$

Then, we conclude by

$\begin{cases}\Phi_{1}^{(a)}(\Psi)=\Psi_{1}-(1-\lambda)p/2=\mu,\\\
\Phi_{2}^{(a)}(\Psi)=1-\frac{1}{p}(1-\lambda)p=\lambda,\\\
\Phi_{3}^{(a)}(\Psi)=(1/p)^{-1}=p.\end{cases}$

Let us then consider the case $r_{+}\neq 1/2$. First, we remark that

$\kappa(\Psi_{1},\Psi_{3})=(r_{+}-r_{-})^{2}\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{D(\lambda,p)^{2}}.$

Then, substituting $(1-\lambda)p=(1-D(\lambda,p))/(r_{+}-r_{-})$ into the
equation aboves gives that

$\kappa(\Psi_{1},\Psi_{3})=\frac{(r_{+}-r_{-})^{2}+4r_{+}r_{-}(1-D(\lambda,p))^{2}}{D(\lambda,p)^{2}}.$

Then, using the fact that
$(r_{+}-r_{-})^{2}=(r_{+}+r_{-})^{2}-4r_{+}r_{-}=1-4r_{+}r_{-}$, it follows
that

$\kappa(\Psi_{1},\Psi_{3})D(\lambda,p)^{2}=1+4r_{+}r_{-}[D(\lambda,p)^{2}-2D(\lambda,p)],$

which means that $D(\lambda,p)$ is a solution of the quadratic equation

$[4r_{+}r_{-}-\kappa(\Psi_{1},\Psi_{3})]X^{2}-(8r_{+}r_{-})X+1=0.$

By definition of $d^{(+)}$ and $d^{(-)}$, we necessarily have
$D(\lambda,p)\in\\{d^{(+)}(\Psi_{1},\Psi_{3}),d^{(-)}(\Psi_{1},\Psi_{3})\\}$.
In turn, since $(1-D(\lambda,p))^{2}(r_{+}-r_{-})^{-2}=(1-\lambda)^{2}p^{2}$,
we have

$(1-\lambda)^{2}p^{2}\in\\{\phi_{1}^{(+)}(\Psi_{1},\Psi_{2}),\phi_{1}^{(-)}(\Psi_{1},\Psi_{2})\\},$

and the proof is concluded in the same manner as the case $r_{+}=1/2$.

#### Proof of 2

Let us first remark that, if $r_{+}=1/2$ or
$\kappa(\Psi_{1},\Psi_{3})=4r_{+}r_{-}$, then $\Phi^{(+)}=\Phi^{(-)}$ and the
result is trivial.

On the one hand, if $\kappa(\Psi_{1},\Psi_{3})>4r_{+}r_{-}$ then the root
$d^{(+)}(\Psi_{1},\Psi_{3})$ is negative which implies that
$D(\lambda,p)=d^{(-)}(\Psi_{1},\Psi_{3})$. On the other hand, if $r_{+}>1/2$
and $\kappa(\Psi_{1},\Psi_{3})<4r_{+}r_{-}$ then

$d^{(+)}(\Psi_{1},\Psi_{3})\geq\frac{4r_{+}r_{-}}{4r_{+}r_{-}-\kappa(\Psi_{1},\Psi_{3})}>1$

Yet, we know that $D(\lambda,p)<1$ by Proposition E.1 which implies that
$D(\lambda,p)=d^{(-)}(\Psi_{1},\Psi_{3})$.

As a summary, we have $D(\lambda,p)=d^{(-)}(\Psi_{1},\Psi_{3})$ in any case.
In turn, it implies that
$(1-\lambda)^{2}p^{2}=\phi_{1}^{(-)}(\Psi_{1},\Psi_{3})$ and the proof is
concluded as above.

#### Proof of 3

Let $(m,v,w)\in\mathcal{M}$, and $a\in\\{-,+\\}$ such that
$\Phi^{(a)}(m,v,w)\in\Lambda$. In the following, we shorten the notation
$d^{(a)}=d^{(a)}(m,w)$, $\phi_{1}^{(a)}=\phi_{1}^{(a)}(m,w)$,
$\phi_{2}^{(a)}=\phi_{2}^{(a)}(m,v,w)$ and
$\Phi_{k}^{(a)}=\Phi_{k}^{(a)}(m,v,w)$ for $k=1,2,3$. Finally, we denote
$\Phi^{(a)}=(\Phi_{1}^{(a)},\Phi_{2}^{(a)},\Phi_{3}^{(a)})\in\Lambda$.

Without any condition, it is easy to check that
$D(\Phi_{2}^{(a)},\Phi_{3}^{(a)})=1-(r_{+}-r_{-})\sqrt{\phi_{1}^{(a)}}$. In
particular, it gives

$\displaystyle\Psi_{1}(\Phi^{(a)})$ $\displaystyle=$
$\displaystyle\frac{\Phi_{1}^{(a)}+(1-\Phi_{2}^{(a)})\Phi_{3}^{(a)}r_{-}}{1-(r_{+}-r_{-})\sqrt{\phi_{1}^{(a)}}}$
$\displaystyle=$
$\displaystyle\frac{m\left[1-(r_{+}-r_{-})\sqrt{\phi_{1}^{(a)}}\right]-r_{-}\sqrt{\phi_{1}^{(a)}}+r_{-}\sqrt{\phi_{1}^{(a)}}}{1-(r_{+}-r_{-})\sqrt{\phi_{1}^{(a)}}}=m,$

and

$\displaystyle\Psi_{2}(\Phi^{(a)})$ $\displaystyle=$
$\displaystyle(1-\Phi_{2}^{(a)})^{2}\Phi_{3}^{(a)}(1-\Phi_{3}^{(a)})[(m-r_{-})^{2}+r_{+}r_{-}]$
$\displaystyle=$
$\displaystyle\phi_{1}^{(a)}[\phi_{2}^{(a)}-1][(m-r_{-})^{2}+r_{+}r_{-}]=v.$

For the last coordinate, we use the condition stated in the Proposition.

Let us first consider the case $r_{+}=1/2$. In that case, the $\phi$ and
$\Phi$ functions do not depend on the superscript $a\in\\{-,+\\}$ and we omit
it in the following. First, remark that
$D(\Phi_{2}^{(a)},\Phi_{3}^{(a)})=1-(r_{+}-r_{-})\sqrt{\phi_{1}^{(a)}}=1$ in
that case, so that

$\displaystyle\Psi_{3}(\Phi^{(a)})$ $\displaystyle=$ $\displaystyle
m(1-m)[1+(1-\Phi_{2}^{(a)})^{2}(\Phi_{3}^{(a)})^{2}]$ $\displaystyle=$
$\displaystyle m(1-m)\left[1+\frac{w}{m(1-m)}-1\right]=w.$

Let us then consider the case $r_{+}\neq 1/2$ and assume the sign condition:
$\operatorname{sgn}(1-d^{(a)})=\operatorname{sgn}(r_{+}-r_{-})$. This
condition implies that
$D(\Phi_{2}^{(a)},\Phi_{3}^{(a)})=1-(r_{+}-r_{-})\sqrt{\phi_{1}^{(a)}}=d^{(a)}$.
Hence,

$\displaystyle\Psi_{3}(\Phi^{(a)})$ $\displaystyle=$ $\displaystyle
m(1-m)[1+4r_{+}r_{-}(1-\Phi_{2}^{(a)})^{2}(\Phi_{3}^{(a)})^{2}]/(d^{(a)})^{2}$
$\displaystyle=$ $\displaystyle
m(1-m)\left[1+4r_{+}r_{-}\frac{(1-d^{(a)})^{2}}{(r_{+}-r_{-})^{2}}\right]/(d^{(a)})^{2}.$

Yet,

$4r_{+}r_{-}(1-d^{(a)})^{2}=4r_{+}r_{-}-1+(1-8r_{+}r_{-}d^{(a)}+4r_{+}r_{-}(d^{(a)})^{2})=4r_{+}r_{-}-1+\kappa(m,w)(d^{(a)})^{2}$

because $d^{(a)}$ solves the quadratic equation (107). Finally, using the
definition of $\kappa$ and the fact that $1-4r_{+}r_{-}=(r_{+}-r_{-})^{2}$ we
get that $\Psi_{3}(\Phi^{(a)})=w$ which concludes the proof. ∎

[Acknowledgments] This research has been conducted while J.C. was in Statify
team at Centre Inria de l’Université Grenoble Alpes.

G.O. was supported by the Serrapilheira Institute (grant number Serra –
2211-42049), FAPERJ (grant E-26/201.397/2021) and CNPq (grants 303166/2022-3).
J.C. and E.L. were supported by ANR-19-CE40-0024 (CHAllenges in MAthematical
NEuroscience).

## References

  * Basu and Michailidis (2015) [author] Basu, SumantaS. and Michailidis, GeorgeG. (2015). Regularized estimation in sparse high-dimensional time series models. The Annals of Statistics 43 1535 – 1567. 10.1214/15-AOS1315
  * Bresler (2015) [author] Bresler, GuyG. (2015). Efficiently Learning Ising Models on Arbitrary Graphs. In Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing. STOC ’15 771–782. Association for Computing Machinery, New York, NY, USA. 10.1145/2746539.2746631
  * Byol Kim and Kolar (2021) [author] Byol Kim, Song LiuS. L. and Kolar, MladenM. (2021). Two-Sample Inference for High-Dimensional Markov Networks. Journal of the Royal Statistical Society Series B: Statistical Methodology 83 939–962.
  * De Santis and Piccioni (2015) [author] De Santis, EmilioE. and Piccioni, MauroM. (2015). One-Dimensional Infinite Memory Imitation Models with Noise. Journal of Statistical Physics 161 346–364. 10.1007/s10955-015-1335-5
  * Delattre and Fournier (2016) [author] Delattre, SylvainS. and Fournier, NicolasN. (2016). Statistical inference versus mean field limit for Hawkes processes. Electronic Journal of Statistics 10 1223 – 1295.
  * Duarte et al. (2019) [author] Duarte, AlineA., Galves, AntonioA., Löcherbach, EvaE. and Ost, GuilhermeG. (2019). Estimating the interaction graph of stochastic neural dynamics. Bernoulli 25 771-792.
  * Eichler (2012) [author] Eichler, MichaelM. (2012). Graphical modelling of multivariate time series. Probability Theory and Related Fields 153 233–268.
  * Lauritzen (1996) [author] Lauritzen, Steffen L.S. L. (1996). Graphical Models. Oxford University Press.
  * Lerasle and Takahashi (2016) [author] Lerasle, M.M. and Takahashi, D. Y.D. Y. (2016). Sharp oracle inequalities and slope heuristic for specification probabilities estimation in discrete random fields. Bernoulli 22 325-344.
  * Liu (2020) [author] Liu, ChenguangC. (2020). Statistical Inference for a partially observed interacting System of Hawkes Processes. Stochastic Processes and Their Applications 130 5636-5694.
  * Montanari and Pereira (2009) [author] Montanari, AndreaA. and Pereira, JoseJ. (2009). Which graphical models are difficult to learn? In Advances in Neural Information Processing Systems (Y.Y. Bengio, D.D. Schuurmans, J.J. Lafferty, C.C. Williams and A.A. Culotta, eds.) 22\. Curran Associates, Inc.
  * Ravikumar, Wainwright and Lafferty (2010) [author] Ravikumar, PradeepP., Wainwright, Martin J.M. J. and Lafferty, John D.J. D. (2010). High-dimensional Ising model selection using $\ell_{1}$-regularized logistic regression. Ann. Stat. 38 1287–1319. 10.1214/09-AOS691
  * Reynaud-Bouret, Rivoirard and Tuleau-Malot (2013) [author] Reynaud-Bouret, PatriciaP., Rivoirard, VincentV. and Tuleau-Malot, ChristineC. (2013). Inference of functional connectivity in Neurosciences via Hawkes processes. In 2013 IEEE Global Conference on Signal and Information Processing 317-320.
  * Strogatz (2001) [author] Strogatz, Steven H.S. H. (2001). Exploring complex networks. Nature 410 268–276.