# Inferring the dependence graph density of binary graphical models in high
dimension

Julien Chevallierlabel=e1]julien.chevallier1@univ-grenoble-
alpes.fr0000-0002-0736-8487 [ Eva Löcherbachlabel=e2]Eva.Locherbach@univ-
paris1.fr0000-0003-4436-2532 [ Guilherme
<EMAIL_ADDRESS>[ Univ. Grenoble Alpes,
CNRS, Inria, Grenoble INP111Institute of Engineering Univ. Grenoble Alpes,
LJK, 38000 Grenoble, France,  Université Paris 1 Panthéon Sorbonne,  Institute
of Mathematics, Federal University of Rio de Janeiro,

###### Abstract

We consider a system of binary interacting chains describing the dynamics of a
group of $N$ components that, at each time unit, either send some signal to
the others or remain silent otherwise. The interactions among the chains are
encoded by a directed Erdös-Rényi random graph with unknown parameter
$p\in(0,1).$ Moreover, the system is structured within two populations
(excitatory chains versus inhibitory ones) which are coupled via a mean field
interaction on the underlying Erdös-Rényi graph. In this paper, we address the
question of inferring the connectivity parameter $p$ based only on the
observation of the interacting chains over $T$ time units. In our main result,
we show that the connectivity parameter $p$ can be estimated with rate
$N^{-1/2}+N^{1/2}/T+(\log(T)/T)^{1/2}$ through an easy-to-compute estimator.
Our analysis relies on a precise study of the spatio-temporal decay of
correlations of the interacting chains. This is done through the study of
coalescing random walks defining a backward regeneration representation of the
system. Interestingly, we also show that this backward regeneration
representation allows us to perfectly sample the system of interacting chains
(conditionally on each realization of the underlying Erdös-Rényi graph) from
its stationary distribution. These probabilistic results have an interest in
its own.

62M05,

60J10,

60K35,

62F12,

Dependence graph inference,

Markov chain,

mean field limit,

perfect simulation,

###### keywords:

[class=MSC]

###### keywords:

, and

###### Contents

  1. 1 Introduction
  2. 2 Model definition, notation and main results
    1. 2.1 Heuristics for the spatio-temporal mean
    2. 2.2 Heuristics for the spatial variance
    3. 2.3 Heuristics for the temporal variance
  3. 3 Backward regeneration scheme
    1. 3.1 Backward regeneration representation
    2. 3.2 Coalescence of the backward random walks
  4. 4 Study of a random matrix
  5. 5 Convergence of the spatio-temporal mean
  6. 6 Convergence of the temporal variance
  7. 7 Convergence of the spatial variance
  8. 8 Conclusion: proof of Theorem 2.1
  9. 9 Simulation study
    1. 9.1 Practical implementation
    2. 9.2 Choice of the tuning parameter
    3. 9.3 Numerical results
  10. A Construction of the process
  11. B Coupling results
  12. C Coalescence of two or more backward random walks
  13. D Auxiliary results
  14. E Inversion of $\Psi$

##  1 Introduction

Understanding how to infer and interpret interactions between measured
components of large scale systems is a central question in many scientific
fields, including Neuroscience, Statistical Physics, and Social Networks
Strogatz (2001). This has been done traditionally in the framework of
probabilistic graphical models Lauritzen (1996), in which the interaction
among the components of the system are encoded through a graph, sometimes
referred to as the graph of conditional dependencies. For such models, a
fundamental question is that of estimating the underlying graph of conditional
dependencies or some function of it from data. Over the past three decades,
this question has been extensively investigated not only under the assumption
that the data is a set of independent and identically distributed observations
and the underlying graphical model has pairwise interactions Bresler (2015);
Montanari and Pereira (2009); Ravikumar, Wainwright and Lafferty (2010), but
also for time dependent data Eichler (2012); Duarte et al. (2019); Reynaud-
Bouret, Rivoirard and Tuleau-Malot (2013) or graphical models with high order
interactions Basu and Michailidis (2015); Lerasle and Takahashi (2016). In the
recent years, understating the theoretical guarantees of the proposed methods
(e.g, consistency, computational complexity) in the high-dimension setting
became a major issue, as now the simultaneous activity of many components can
be routinely recorded. In high-dimension, most of the works assume that the
underlying graph of dependencies is sparse in a suitable sense, showing that
under this assumption the proposed method performs reasonably well. The cases
in which the graphs of dependencies are dense, on the other hand, are much
less studied. In this setting we are only aware of the works of Byol Kim and
Kolar (2021); Delattre and Fournier (2016) and Liu (2020). In these works the
focus is not on the estimation of the graph of dependencies of the
corresponding graphical model, as we detail in what follows.

In Byol Kim and Kolar (2021), particularly motivated by applications in
Genetics and Neuroscience, the authors propose a density ratio approach to
estimate the “difference network”, the difference between two graphs of
dependencies. The key assumption in their approach is that this difference
network is sparse and, in particular, their method can also be applied when
each individual graph is dense as long as their difference is sparse. Delattre
and Fournier (2016) consider a graphical model with pairwise interactions for
which the graph of dependencies is a realization of a weighted Erdős-Rényi
random graph with edge probability $p$. Specifically, Delattre and Fournier
(2016) work with $N$ Hawkes point processes on $[0,T]$ \- a class of point
processes in which the intensity at any given time is a linear function of the
past events - where each pair of these point processes are independently
coupled with probability $p$. Moreover, the existing interactions are
excitatory and of mean-field type (i.e., the occurrence of an event increases
the chance of a new event to occur and the increment scales as $1/N$). The
main goal of the paper is not to estimate the random graph of dependencies but
rather the density of connections of this graph (the parameter $p$) based only
on the information present in the $N$ point processes on $[0,T]$. Under some
few extra assumptions, the authors show that this can be done with a precision
of the order $N^{-1/2}+N^{1/2}/m_{T}$ (up to some correction factor) where
$m_{T}$ denotes the mean number of events per point process. In Liu (2020),
the author complements the analysis started in Delattre and Fournier (2016) by
investigating the problem of estimating the density of connections $p$ in the
same setting with the difference being that now one has access only to the
information present in $K$ Hawkes point processes for some large $K\leq N$.
Under this constraint, Liu (2020) proposes an estimator of the parameter $p$
with rate of convergence $K^{-1/2}+N/(K^{1/2}m_{T})+N/(Km^{1/2}_{T}).$

In the present paper, we consider a graphical model which is defined in terms
of a system of $N$ interacting $\\{0,1\\}$-valued chains
$X=\\{X_{i,t},t\in\mathbb{Z},1\leq i\leq N\\}$ and a random matrix
$\theta=(\theta_{ij})_{1\leq i,j\leq N}$ of i.i.d entries distributed as
$\text{Ber}(p)$. The event $X_{i,t}=1$ indicates that the $i$-th chain sends
some signal at time $t$, and $X_{i,t}=0$ otherwise. Let
$X_{t}=(X_{1,t},\ldots,X_{N,t})$ be the configuration of the system $X$ at
time $t$. The model is defined as follows. Conditionally on $\theta$, the
system $X$ evolves as a stationary Markov chain on the state space
$\\{0,1\\}^{N}$ in which the conditional distribution of $X_{t}$ given that
$X_{t-1}=x$ is that of $N$ independent Bernoulli random variables with
parameter $p_{\theta,i}(x)$, $i=1,\ldots,N$, where

$p_{\theta,i}(x)=\mu+(1-\lambda)\left(\frac{1}{N}\sum_{j\in\mathcal{P}_{+}}\theta_{ij}x_{j}+\frac{1}{N}\sum_{j\in\mathcal{P}_{-}}\theta_{ij}(1-x_{j})\right),\
x=(x_{1},\ldots,x_{N}).$ (1)

Here, $0<\lambda<1$ and $0\leq\mu\leq\lambda$ are parameters, and
$\mathcal{P}_{+}$, $\mathcal{P}_{-}$ are subsets of $[N]=\\{1,\ldots,N\\}$
forming a partition. Neither the parameters $\mu$ and $\lambda$ nor the
subsets $\mathcal{P}_{+}$ and $\mathcal{P}_{-}$ depend on the random matrix
$\theta$.

The function $p_{\theta,i}(x)$ appearing in (1) models the probability of
observing a new signal for the $i$-th chain given that the configuration of
the system at the precedent time is $x$. The parameter $\mu$ models the
baseline activity of each interacting chain. Note that the probability
$p_{\theta,i}(x)$ depends on the past $x$ only through the values $x_{j}$ for
which $\theta_{ij}=1$, so that the random matrix $\theta$ encodes the
interaction among the chains of the system. In light of this, $\theta$ can be
considered as the proxy for the graph of conditional dependencies of the
model. Note also that each chain is either excitatory or inhibitory in the
model; the set $\cal{P}_{+}$ denotes the set of excitatory chains and
$\cal{P}_{-}$ the set of inhibitory ones. An excitatory chain increases the
probability of future signals to occur whenever it sends any signal, whereas
the signals sent by any inhibitory chain reduce the probability of occurrence
of new signals. The increment of this changing is $(1-\lambda)/N$, implying
that the interactions are of mean-field type and that $1-\lambda$ parametrizes
the strength of the interactions.

Throughout the paper, we denote $r^{N}_{+}=|\mathcal{P}_{+}|/N$ and
$r^{N}_{-}=|\mathcal{P}_{-}|/N$ the fraction of excitatory and inhibitory
components of the model, respectively, and suppose that there are values
$0<r_{+},r_{-}<1$ satisfying $r_{+}+r_{-}=1$ such that
$|r^{N}_{+}-r_{+}|\vee|r^{N}_{-}-r_{-}|\leq KN^{-1}$ for some universal
constant $K$. One can easily check that a choice satisfying this assumption
with $K=1$ is $|\mathcal{P}_{+}|=\lceil r_{+}N\rceil$ and
$|\mathcal{P}_{-}|=N-|\mathcal{P}_{+}|$.

#### Problem formulation

The goal of this paper is to address the following statistical question. By
observing a sample $X_{1},\ldots,X_{T}$ of the system $X$, can we estimate the
asymptotic density of connections $p$ of the random matrix $\theta$, knowing
only the size of the system $N$ and the asymptotic fraction of excitatory and
inhibitory components $r_{+}$ and $r_{-}$? This means that we do not assume
any prior knowledge on $\mu,\lambda,p,\theta,\mathcal{P}_{+},\mathcal{P}_{-}$,
i.e., they are all unknown.

Although the estimation problem investigated here is similar to the ones
considered in Delattre and Fournier (2016); Liu (2020), our work is different
from theirs in at least two substantial ways. First, in their model the
interaction between each pair of components can only be excitatory, whereas it
can be either excitatory or inhibitory in ours, making the analysis of the
estimation problem (especially the study of the random environment) more
challenging. Second, a crucial step in the analysis is to study the decay of
correlation between $X_{i,t}$ and $X_{j,s}$ (and of products of these), for
each fixed realization of the random matrix $\theta$. In our case, this is
done through the study of coalescing random walks defining a backward
regeneration representation of the system $X$. Interestingly, we show that
this backward regeneration representation allows us to perform perfect
simulation of the system $X$. That is, one can exactly sample the system $X$
(conditionally on $\theta$) from its unique stationary distribution. These
probabilistic results have an interest in its own. In Delattre and Fournier
(2016) and Liu (2020), on the other hand, the analysis of the model for each
realization of the graph of conditional dependence follows a rather different
approach, fundamentally based on martingale arguments. Moreover, the question
of how to perfectly sample the underlying process is not discussed.

As will become apparent in our results, we can not only estimate the parameter
$p$ but also the parameters $\mu$ and $\lambda$. In our main result, we show
that the parameters $(\mu,\lambda,p)$ can be simultaneously estimated with
rate $N^{-1/2}+N^{1/2}/T+(\log(T)/T)^{1/2}$, under some extra conditions on
the asymptotic faction $r_{+}$ of excitatory components of the model. Let us
mention that this convergence rate is consistent with the one found in
Delattre and Fournier (2016) (up to a $\log(T)$ factor), because the quantity
$m_{T}$ (adapted to our setting) increases linearly in $T$.

We are particularly motivated by the statistical analysis of neuronal
networks. Neurons communicate to each other by firing short electrical pulses,
often called spikes. The spiking times of a network of neurons depend on its
graph of interaction, a combinatorial structure (typically unknown) encoding
the type of interaction (excitatory or inhibitory) between each pair of
neurons in the network. An important question in Neuroscience is to understand
which features of this graph can be inferred from the spiking times of the
recorded neurons. Our results suggests that one such feature is the density of
connections of the graph, as long as we know the true fraction $r_{+}$ of
excitatory neurons of the network. In many situation, this turns out to be the
case. For example, in humans and many other mammalian species the value
$r_{+}$ is known and ranges from $0.7$ to $0.8$.

In the next section we define our model rigorously, introduce some general
notation used throughout the paper, state our main results and give some
heuristics behind them. At the end of that section, we also provide the
organization of the rest of the paper.

##  2 Model definition, notation and main results

#### Model definition

We consider a system of $N$ interacting chains
$X=\\{X_{i,t},t\in\mathbb{Z},1\leq i\leq N\\}$ taking values in $\\{0,1\\}$
denoting the presence or the absence of a signal at a given time. This system
evolves in a random environment which is given by the realization of a
directed Erdös-Rényi random graph via the selection of $N^{2}$ i.i.d.
Bernoulli random variables $\theta_{ij}\sim\text{Ber}(p),1\leq i,j\leq N$ with
$0\leq p\leq 1$. Conditionally on the realization of
$\theta=(\theta_{ij})_{1\leq i,j\leq N}$, the evolution of the system $X$ is
that of a stationary Markov chain on the state space $\\{0,1\\}^{N}$ with
transition probabilities (which depend on $\theta$) given as follows. Writing
$X_{t}=(X_{1,t},\ldots,X_{N,t})$ and $x,y\in\\{0,1\\}^{N}$, we have, for all
$t\in\mathbb{Z}$,

$I\\!\\!P_{\theta}(X_{t}=y|X_{t-1}=x)=\prod_{i=1}^{N}(p_{\theta,i}(x))^{y_{i}}(1-p_{\theta,i}(x))^{(1-y_{i})},$
(2)

where $p_{\theta,i}(x)=I\\!\\!P_{\theta}(X_{i,t}=1|X_{t-1}=x)$ is defined in
(1). The existence and uniqueness of a stationary version of the Markov chain
having transition probabilities as defined in (2) and (1) is granted by
Theorem 3.3 presented in Section 3.

#### Notation

Throughout the paper, the letters $t,s$ denote some time values whereas the
letters $i,j$ denote some spatial values, that is the index of the
corresponding component. For a vector $v$ in $\mathbb{R}^{N}$, the notation
$\overline{v}$ denotes the spatial mean
$\overline{v}=N^{-1}\sum_{i=1}^{N}v_{i}$. In agreement with the left-hand side
of (2), we write $I\\!\\!P_{\theta}$ to denote the probability measure under
which the environment $\theta$ is kept fixed and the process $X$ is
distributed as the unique stationary version of the Markov chain having
transition probabilities as defined in (2) and (1). The expectation taken with
respect to $I\\!\\!P_{\theta}$ is denoted by $I\\!\\!E_{\theta}$. The variance
and covariance computed from $I\\!\\!E_{\theta}$ are denoted
$\text{Var}_{\theta}$ and $\text{Cov}_{\theta}$ respectively. Moreover, we
write $I\\!\\!P$ to denote the probability measure under which the random
environment $\theta=(\theta_{ij})_{1\leq i,j\leq N}$ is distributed as a
collection of i.i.d. random variables with distribution $\text{Ber}(p)$ and
the conditional distribution of the process $X$ given $\theta$ is that of the
process $X$ under $I\\!\\!P_{\theta}$, i.e., the following identity holds
$I\\!\\!P(X\in\cdot|\theta)=I\\!\\!P_{\theta}(X\in\cdot)$. Finally, we denote
$I\\!\\!E$ the expectation taken with respect to the probability measure
$I\\!\\!P$, and Var and Cov the variance and covariance, respectively,
computed from $I\\!\\!E$.

#### Main results

We are given a sample $X_{1},\ldots,X_{T}$ of a stationary Markov chain with
transition probabilities given by (2) and (1), associated to some realization
of the random environment $\theta$ which is not observed. The goal is to
estimate the parameter $p$, the asymptotic density of connections of the
underlying random environment. We consider an estimator of $p$ which is a
function of three other estimators defined below. In what follows, let
$\overline{X}_{t}=N^{-1}\sum_{i=1}^{N}X_{i,t}$ denote the average number of
signals emitted by the system at time $t$; $Z_{i,t}=\sum_{s=1}^{t}X_{i,s}$
denote the number of signals emitted by the $i$-th component of the system in
the discrete interval $\\{1,\ldots,t\\}$ and
$\overline{Z}_{t}=N^{-1}\sum_{i=1}^{N}Z_{i,t}$ denote the average number of
signals emitted by the system in the interval $\\{1,\ldots,t\\}$. For
notational convenience, we set $\overline{Z}_{0}:=0$. With this notation, our
three main estimators are defined as follows:

$\hat{m}=\frac{\overline{Z}_{T}}{T},\ \ \
\hat{v}=\frac{(T+1)N}{T^{3}}\left[\frac{1}{N}\sum_{i=1}^{N}\left(Z_{i,T}\right)^{2}-\frac{T}{(T+1)}\left(\overline{Z}_{T}+\left(\overline{Z}_{T}\right)^{2}\right)\right]\
\text{and}\ $ $\hat{w}=2\mathcal{W}_{2\Delta}-\mathcal{W}_{\Delta},\
\text{with}\ \mathcal{W}_{\Delta}=\frac{N}{T}\sum_{k=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{k\Delta}-\overline{Z}_{(k-1)\Delta}-\Delta\hat{m}\right)^{2},$

where $\Delta\in\\{1,\ldots,\lfloor T/2\rfloor\\}$ is a tuning parameter. The
estimator $\hat{m}$ is the _spatio-temporal mean_ of signals emitted by the
system in the observed interval $\\{1,\ldots,T\\}$. As we explain in Section
2.2, the estimator $\hat{v}$ is related to the empirical variance of the
random variables $Z_{1,T},\ldots,Z_{N,T}$, i.e., a variance over the spatial
components of the system. The estimator $\hat{w}$ is computed from the
empirical variance of the random variables
$\overline{Z}_{\ell\Delta},\overline{Z}_{2\ell\Delta}-\overline{Z}_{\ell\Delta},\ldots,\overline{Z}_{T}-\overline{Z}_{T-\ell\Delta}$,
i.e., the empirical variance of the mean number of signals emitted by the
system over different time intervals. For these reasons, the estimators
$\hat{v}$ and $\hat{w}$ are called _spatial variance_ and _temporal variance_
respectively.

Obviously, the three estimators $\hat{m},\hat{v},\hat{w}$ depend on $N$ and
$T$ (as well as the tuning parameter $\Delta$ for $\hat{w}$). We chose not to
specify this dependence in the notation to keep notation as light as possible.

######  Theorem 2.1.

There exists a constant $K>0$ depending only on $\lambda$ such that for all
$\varepsilon\in(0,1)$, $N\geq 1$, $T\geq 2$ and $1\leq\Delta\leq\lfloor
T/2\rfloor$,

$\displaystyle
I\\!\\!P(|\hat{m}-m|\geq\varepsilon)\leq\frac{K}{\varepsilon}\left(\frac{1}{\sqrt{TN}}+\frac{1}{N}\right),$
(3) $\displaystyle
I\\!\\!P(|\hat{v}-v|\geq\varepsilon)\leq\frac{K}{\varepsilon}\left(\frac{\sqrt{N}}{T}+\frac{1}{\sqrt{N}}\right),$
(4) $\displaystyle
I\\!\\!P(|\hat{w}-w|\geq\varepsilon)\leq\frac{K}{\varepsilon}\left(\frac{1}{N}+\frac{(1-\lambda)^{\Delta}}{\Delta}+\sqrt{\frac{\Delta}{T}}\right),$
(5)

where the vector $(m,v,w)$ is given by

$\begin{cases}m=\frac{\mu+(1-\lambda)pr_{-}}{1-p(1-\lambda)(r_{+}-r_{-})},\\\
v=(1-\lambda)^{2}p(1-p)((m-r_{-})^{2}+r_{+}r_{-}),\\\
w=m(1-m)\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{(1-p(1-\lambda)(r_{+}-r_{-}))^{2}}.\end{cases}$
(6)

The proof of Theorem 2.1 is given in Section 8. The constant $K$ depends only
on the parameter $\lambda$. Let us mention that this constant diverges when
$\lambda\to 0$ (because of the inversion of the linear system, see for
instance the bound of Lemma 4.1) and $\lambda\to 1$ (because the dependence
between the components of the model becomes too weak, see for instance the
bounds of Proposition 3.7).

Once the parameters $(m,v,w)$ are estimated from the sample, one wants to
deduce some estimators of the parameters $(\mu,\lambda,p)$. Here is how one
can proceed.

Let $\Lambda=\\{(\mu,\lambda,p)\in(0,1)^{3}:0<\mu<\lambda\\}$ be the open set
of admissible parameters. For all $(\mu,\lambda,p)\in\Lambda$, let us define

$D(\lambda,p)=1-(1-\lambda)p(r_{+}-r_{-})>0,$ (7)

the denominator that appears in the expressions of $m$ and $w$. Then, for
$k=1,2,3$, let $\Psi_{k}:\Lambda\to\mathbb{R}$ be defined by

$\begin{cases}\Psi_{1}(\mu,\lambda,p)=(\mu+(1-\lambda)pr_{-})/D(\lambda,p)\\\
\Psi_{2}(\mu,\lambda,p)=(1-\lambda)^{2}p(1-p)[(\Psi_{1}(\mu,\lambda,p)-r_{-})^{2}+r_{+}r_{-}]\\\
\Psi_{3}(\mu,\lambda,p)=\Psi_{1}(\mu,\lambda,p)[1-\Psi_{1}(\mu,\lambda,p)][1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}]/D(\lambda,p)^{2}.\end{cases}$
(8)

Finally, let $\Psi:\Lambda\to\mathbb{R}^{3}$ be defined by the three
coordinate functions above so that Equation (6) rewrites as
$(m,v,w)=\Psi(\mu,\lambda,p)$. Finally, let us remark that, whatever the value
of $r_{+}$, the image $\Psi(\Lambda)$ is included in
$(0,1)\times(0,\infty)^{2}$ (see Proposition E.1 stated and proved in the
Appendix E).

The following proposition gives some information about the inversion of the
function $\Psi$, by means of an auxiliary function
$\kappa:(0,1)\times(0,\infty)\to(0,\infty)$ which is defined by

$\kappa(m,w)=(r_{+}-r_{-})^{2}\frac{w}{m(1-m)}.$ (9)

######  Proposition 2.2.

Whatever the value of $r_{+}$, there exist two explicit functions $\Phi^{(+)}$
and $\Phi^{(-)}$ such that the following results hold.

  1. 1.

For all $(\mu,\lambda,p)\in\Lambda$,
$(\mu,\lambda,p)\in\\{\Phi^{(+)}\circ\Psi(\mu,\lambda,p),\Phi^{(-)}\circ\Psi(\mu,\lambda,p)\\}$,

  2. 2.

Moreover, if $r_{+}\geq 1/2$ or

$\kappa(\Psi_{1}(\mu,\lambda,p),\Psi_{3}(\mu,\lambda,p))\geq 4r_{+}r_{-},$

then $(\mu,\lambda,p)=\Phi^{(-)}\circ\Psi(\mu,\lambda,p)$.

Proposition 2.2 is deduced as a corollary of Proposition E.3 stated and proved
in the Appendix E. Furthermore, the expressions of the functions $\Phi^{(+)}$
and $\Phi^{(-)}$ are also given in Appendix E.

We are now in position to deduce the following corollary of Theorem 2.1.

######  Corollary 2.3.

Assume that the condition 2 of Proposition 2.2 is satisfied. Then there exists
a constant $K>0$ depending only on $\lambda$ such that for all
$\varepsilon\in(0,1)$, $N\geq 1$, $T\geq 2$ and $1\leq\Delta\leq\lfloor
T/2\rfloor$,

$I\\!\\!P(||\Phi^{(-)}(\hat{m},\hat{v},\hat{w})-(\mu,\lambda,p)||_{\infty}\geq\varepsilon)\leq\frac{K}{\varepsilon}\left(\frac{\sqrt{N}}{T}+\frac{1}{\sqrt{N}}+\frac{(1-\lambda)^{\Delta}}{\Delta}+\sqrt{\frac{\Delta}{T}}\right).$
(10)

###### Proof.

By definition of $\Psi$, remark that Theorem 2.1 gives a control of
$||(\hat{m},\hat{v},\hat{w})-\Psi(\mu,\lambda,p)||_{\infty}$. Hence, the
result follows from the facts that $\Phi^{(-)}$ is locally Lipschitz (see
Proposition E.2) and $\Phi^{(-)}\circ\Psi(\mu,\lambda,p)=(\mu,\lambda,p)$. ∎

In practice, a value must be chosen for the tuning parameter $\Delta$. The
question of how to choose $\Delta$ with respect to $T$ is discussed in Section
9. In what follows we present some heuristic arguments leading to the
convergence stated in Theorem 2.1.

###  2.1 Heuristics for the spatio-temporal mean

Denote
$m^{N}:=(m^{N}_{1},\ldots,m^{N}_{N})=I\\!\\!E_{\theta}\left[X_{0}\right]$.
Notice that $m^{N}$ depends on the realization of $\theta$, but we omit to
specify this dependence to keep concise notation. Throughout this section, the
notation $a\approx b$ is used to express the fact that $a$ and $b$ are
expected to be close to each other as either $N$ or $T$ are large enough. By
ergodicity, we must have as $T\to\infty$,

$\frac{1}{T}\sum_{t=1}^{T}X_{t}\to m^{N},\ \ I\\!\\!P_{\theta}-a.s.,$

so that we expect for $T$ large enough that

$\hat{m}=\frac{1}{N}\sum_{i=1}^{N}\frac{1}{T}\sum_{t=1}^{T}X_{i,t}\approx\frac{1}{N}\sum_{i=1}^{N}m^{N}_{i}:=\overline{m^{N}}.$
(11)

Hence, to find the limit of $\hat{m}$ one needs to understand the asymptotic
behavior of $\overline{m^{N}}.$ To that end, observe that by taking first the
expectation in both sides of (1) and then using the stationarity, one can
check that, for all $1\leq i\leq N$,

$m^{N}_{i}=\mu+(1-\lambda)\left(\frac{1}{N}\sum_{j\in\mathcal{P}_{+}}\theta_{ij}m^{N}_{j}+\frac{1}{N}\sum_{j\in\mathcal{P}_{-}}\theta_{ij}(1-m^{N}_{j})\right).$
(12)

Denoting $A^{N}(i,j)=N^{-1}\theta_{ij}$ for all $1\leq i\leq N$ and
$j\in\mathcal{P}_{+}$ and $A^{N}(i,j)=-N^{-1}\theta_{ij}$ for all $1\leq i\leq
N$ and $j\in\mathcal{P}_{-}$, we can then write the above system of equations
in matrix form as follows:

$m^{N}=\mu 1_{N}+(1-\lambda)\left(A^{N}m^{N}-L^{N,\bullet-}\right),$

where $1_{N}$ denotes the $N$-dimensional vector with all entries equal to 1
and $L^{N,\bullet-}=(L^{N,\bullet-}_{1},\ldots,L^{N,\bullet-}_{N})$ is given
by $L^{N,\bullet-}_{i}=\sum_{j\in\mathcal{P}_{-}}A^{N}(i,j).$ Therefore, we
deduce that

$\left(I_{N}-(1-\lambda)A^{N}\right)m^{N}=\mu
1_{N}-(1-\lambda)L^{N,\bullet-},$

where $I_{N}$ denotes the identify matrix of size $N$. Let us denote
$Q^{N}=\left(I_{N}-(1-\lambda)A^{N}\right)^{-1}$. The random matrix $Q^{N}$ is
well-defined whenever $\lambda>0$ (see Section 4 below). Granted that $Q^{N}$
is well-defined, we then have that

$m^{N}=\mu Q^{N}1_{N}-(1-\lambda)Q^{N}L^{N,\bullet-}.$ (13)

Then, remark that $L^{N,\bullet-}=A^{N}1_{\mathcal{P}_{-}}$, where
$1_{\mathcal{P}_{-}}$ is the $N$-dimensional vector with value 1 for the
coordinates in $\mathcal{P}_{-}$ and value 0 otherwise. Using the series
expansion $Q^{N}=\sum_{k=0}^{\infty}(1-\lambda)^{k}(A^{N})^{k}$, one can
deduce that $Q^{N}A^{N}=(Q^{N}-I_{N})/(1-\lambda)$. Hence,
$(1-\lambda)Q^{N}L^{N,\bullet-}=\ell^{N,\bullet-}-1_{\mathcal{P}_{-}}$ with
$\ell^{N,\bullet-}=Q^{N}1_{\mathcal{P}_{-}}$. Similarly, we denote
$\ell^{N}=Q^{N}1_{N}$, so that Equation (13) rewrites as

$m^{N}=\mu\ell^{N}+1_{\mathcal{P}_{-}}-\ell^{N,\bullet-}.$

Now, let us give some heuristics on the asymptotic behavior of $\ell^{N}$ and
$\ell^{N,\bullet-}$. To that end, denote $L^{N}=A^{N}1_{N}$ and observe that
by the definition of $\ell^{N}$, using once more the series expansion of
$Q^{N}$,

$\ell^{N}=1_{N}+(1-\lambda)L^{N}+\sum_{k=2}^{\infty}(1-\lambda)^{k}(A^{N})^{k}1_{N}.$

Now, whenever $N$ is large enough, we expect that
$L^{N}_{i}=\sum_{j=1}^{N}A^{N}(i,j)\approx p(r_{+}-r_{-})$ for each $1\leq
i\leq N$, and by induction, we expect that

$\left((A^{N})^{k}1_{N}\right)_{i}\approx\left[p(r_{+}-r_{-})\right]^{k-1}L^{N}_{i},$

for each $1\leq i\leq N$. Hence, for $N$ sufficiently large

$\ell^{N}\approx 1_{N}+\frac{1}{D(\lambda,p)}(1-\lambda)L^{N},$

where $D(\lambda,p)$ is defined in (7). Similarly, by definition of
$\ell^{N,\bullet-}$, we have

$\ell^{N,\bullet-}=1_{\mathcal{P}_{-}}+(1-\lambda)L^{N,\bullet-}+\sum_{k=2}^{\infty}(1-\lambda)^{k}(A^{N})^{k}1_{\mathcal{P}_{-}}.$

Now, whenever $N$ is large enough, we expect that $L^{N,\bullet-}\approx-
pr_{-}1_{N}$, and by the previous induction, we expect that

$(A^{N})^{k}1_{\mathcal{P}_{-}}\approx-
pr_{-}\left[p(r_{+}-r_{-})\right]^{k-2}L^{N}.$

Hence,

$\ell^{N,\bullet-}\approx
1_{\mathcal{P}_{-}}+(1-\lambda)L^{N,\bullet-}-\frac{(1-\lambda)^{2}pr_{-}}{D(\lambda,p)}L^{N}.$

All in all, we expect that

$m^{N}\approx\mu\left(1_{N}+\frac{1-\lambda}{D(\lambda,p)}L^{N}\right)-(1-\lambda)L^{N,\bullet-}+\frac{(1-\lambda)^{2}pr_{-}}{D(\lambda,p)}L^{N}.$
(14)

Now, remind that $L^{N}\approx p(r_{+}-r_{-})1_{N}$ and
$L^{N,\bullet-}\approx-pr_{-}1_{N}$, so that

$\overline{m^{N}}\approx\mu\left(1+\frac{(1-\lambda)p(r_{+}-r_{-})}{D(\lambda,p)}\right)+(1-\lambda)pr_{-}+\frac{(1-\lambda)^{2}p^{2}(r_{+}-r_{-})r_{-}}{D(\lambda,p)}=m,$
(15)

where $m$ is defined in (6). Combining (11) and (15), we expect that for $T$
and $N$ large enough,

$\hat{m}\approx\frac{\mu+(1-\lambda)pr_{-}}{D(\lambda,p)}=m.$

This heuristics is made precise in Theorem 5.2.

###  2.2 Heuristics for the spatial variance

First note that we can always write that

$\operatorname{\mathop{Var}}(Z_{i,T})-I\\!\\!E\left[\operatorname{\mathop{Var}}_{\theta}(Z_{i,T})\right]=\operatorname{\mathop{Var}}\left(I\\!\\!E_{\theta}\left[Z_{i,T}\right]\right).$

The three terms above are considered from left to right.

First, the term $\operatorname{\mathop{Var}}(Z_{i,T})$ can be estimated from
the data via an empirical variance, that is,

$\operatorname{\mathop{Var}}(Z_{i,T})\approx\frac{1}{N}\sum_{i=1}^{N}(Z_{i,T}-\overline{Z}_{T})^{2}.$
(16)

Second, the term
$I\\!\\!E\left[\operatorname{\mathop{Var}}_{\theta}(Z_{i,T})\right]$ can also
be estimated from the data. Indeed, we expect that the temporal covariance of
the model vanishes sufficiently fast (see Lemma 3.8 for a precise statement)
so that, for $T$ large enough,

$I\\!\\!E\left[\operatorname{\mathop{Var}}_{\theta}(Z_{i,T})\right]\approx
TI\\!\\!E\left[\operatorname{\mathop{Var}}_{\theta}(X_{i,0})\right].$ (17)

Given the random matrix $\theta$, we know that $X_{i,0}$ is a Bernoulli
variable with parameter $m_{i}^{N}$ by stationarity of the process, so that

$\operatorname{\mathop{Var}}_{\theta}(X_{i,0})=m_{i}^{N}-(m_{i}^{N})^{2}.$

Yet, we expect that $T^{-1}Z_{i,T}\approx m_{i}^{N}$ which implies that we can
estimate $I\\!\\!E\left[\operatorname{\mathop{Var}}_{\theta}(X_{i,0})\right]$
and get

$\frac{1}{N}\sum_{i=1}^{N}\frac{Z_{i,T}}{T}-\left(\frac{Z_{i,T}}{T}\right)^{2}=\frac{\overline{Z}_{T}}{T}-\frac{1}{N}\sum_{i=1}^{N}\left(\frac{Z_{i,T}}{T}\right)^{2}\approx
I\\!\\!E\left[\operatorname{\mathop{Var}}_{\theta}(X_{i,0})\right].$ (18)

Finally, we may express the limit of the last term. Since
$I\\!\\!E_{\theta}\left[Z_{i,T}\right]=Tm^{N}_{i}$, we can start from the
heuristics (14) and look at its variance. It is easy to check that
$\operatorname{\mathop{Var}}(L^{N}_{i})=N^{-1}p(1-p)$ and
$\operatorname{\mathop{Var}}(L^{N,\bullet-}_{i})=\operatorname{\mathop{Cov}}(L^{N}_{i},L^{N,\bullet-}_{i})=N^{-1}p(1-p)r_{-}$.
Hence, we expect that

$\displaystyle\operatorname{\mathop{Var}}(m^{N}_{i})$ $\displaystyle\approx$
$\displaystyle
N^{-1}p(1-p)\left[\left(\frac{\mu(1-\lambda)+(1-\lambda)^{2}pr_{-}}{D(\lambda,p)}\right)^{2}\right.$
$\displaystyle\left.\quad-2\left(\frac{\mu(1-\lambda)+(1-\lambda)^{2}pr_{-}}{D(\lambda,p)}\right)(1-\lambda)r_{-}+(1-\lambda)^{2}r_{-}\right]$
$\displaystyle=$ $\displaystyle
N^{-1}(1-\lambda)^{2}p(1-p)(m^{2}-2mr_{-}+r_{-})=N^{-1}v,$

where $m$ and $v$ are defined in (6) and $D(\lambda,p)$ is defined in (7).
Therefore,

$\frac{N}{T^{2}}\operatorname{\mathop{Var}}\left(I\\!\\!E_{\theta}\left[Z_{i,T}\right]\right)\approx
v.$ (19)

Combining Equations (16)-(19), we have

$\hat{v}=\frac{1}{T^{2}}\sum_{i=1}^{N}(Z_{i,T}-\overline{Z}_{T})^{2}-\frac{N}{T^{2}}\overline{Z}_{T}+\frac{1}{T^{3}}\sum_{i=1}^{N}(Z_{i,T})^{2}\approx
v.$

This heuristics is made precise in Theorem 7.5.

###  2.3 Heuristics for the temporal variance

Let us remark that

$\frac{\Delta}{N}{\cal
W}_{\Delta}=\frac{\Delta}{T}\sum_{k=1}^{T/\Delta}\left[(\overline{Z}_{k\Delta}-\overline{Z}_{(k-1)\Delta})-\Delta
T^{-1}\overline{Z}_{T}\right]^{2}.$

Because we expect that the temporal covariance of the model vanishes
sufficiently fast, we should have (supposing $\Delta\ll T$) that for $T$
large,

$\frac{\Delta}{N}{\cal
W}_{\Delta}\approx\operatorname{\mathop{Var}}_{\theta}\left[\overline{Z}_{\Delta}\right]=\operatorname{\mathop{Var}}_{\theta}\left[\overline{U}_{\Delta}\right],$
(20)

where ${U}_{t}=({U}_{1,t},\ldots,{U}_{N,t})$ for each $t\geq 1$, with

${U}_{i,t}=Z_{i,t}-I\\!\\!E_{\theta}[Z_{i,t}]\ \text{and}\
\overline{U}_{t}=N^{-1}\sum_{i=1}^{N}{U}_{i,t}.$ (21)

Next, (1) implies that for all $1\leq i\leq N$,

$U_{i,t}=M_{i,t}+(1-\lambda)\sum_{j=1}^{N}A^{N}(i,j)\sum_{s=1}^{t}\left(X_{j,s-1}-I\\!\\!E_{\theta}\left[X_{j,s-1}\right]\right),$

where $M_{i,t}=\sum_{s=1}^{t}(X_{i,s}-p_{\theta,i}(X_{s-1}))$ is a martingale.
From the above identity, one can deduce that

$U_{t}=M_{t}+(1-\lambda)A^{N}U_{t}+R^{N}_{t},$ (22)

where $M_{t}=(M_{1,t},\ldots,M_{N,t})$ and
$R^{N}_{t}=(1-\lambda)A^{N}(X_{0}-X_{t})$. From our analysis, we have that the
term $R^{N}_{t}$ is negligible (see Lemma 6.2), so that it follows from (22)
that $U_{t}\approx M_{t}+(1-\lambda)A^{N}U_{t},$ or equivalently,

$U_{t}\approx Q^{N}M_{t}.$

Hence, introducing the vector
$(Q^{N})^{\intercal}1_{N}=c^{N}=(c^{N}_{1},\ldots,c^{N}_{N})$ where
$(Q^{N})^{\intercal}$ denotes the transpose of $Q^{N}$, we deduce that

$\overline{U}_{t}\approx\overline{Q^{N}M_{t}}=\frac{1}{N}\sum_{i=1}^{N}\sum_{j=1}^{N}Q^{N}(i,j)M_{j,t}=\frac{1}{N}\sum_{j=1}^{N}c^{N}_{j}M_{j,t},$

which, in turn, implies that

$\operatorname{\mathop{Var}}_{\theta}\left[\overline{U}_{t}\right]\approx\operatorname{\mathop{Var}}_{\theta}\left[\overline{Q^{N}M_{t}}\right]=\frac{1}{N^{2}}\sum_{i=1}^{N}\sum_{j=1}^{N}c_{i}^{N}c_{j}^{N}I\\!\\!E_{\theta}\left[M_{i,t}M_{j,t}\right].$

Since the martingales $(M_{i,t})_{t}$ and $(M_{j,t})_{t}$ are orthogonal for
any $i\neq j$, i.e. $I\\!\\!E_{\theta}\left[M_{i,t}M_{j,t}\right]=0$, one
obtains that

$\operatorname{\mathop{Var}}_{\theta}\left[\overline{U}_{t}\right]\approx\frac{1}{N^{2}}\sum_{i=1}^{N}(c^{N}_{i})^{2}I\\!\\!E_{\theta}\left[\left(M_{i,t}\right)^{2}\right].$
(23)

Then, by means of the quadratic variation and the stationarity of the process,
we have

$I\\!\\!E_{\theta}\left[\left(M_{i,t}\right)^{2}\right]=I\\!\\!E_{\theta}\left[\sum_{s=1}^{t}\left(X_{i,s}-p_{\theta,i}(X_{s-1})\right)^{2}\right]=tI\\!\\!E_{\theta}\left[\left(X_{i,1}-p_{\theta,i}(X_{0})\right)^{2}\right].$
(24)

Yet, $p_{\theta,i}(X_{0})\approx m_{i}^{N}$ and, conditionally on $\theta$,
$X_{i,1}$ is distributed as a Bernoulli random variable with parameter
$m_{i}^{N}$ so that (see Lemma 6.3)

$I\\!\\!E_{\theta}\left[\left(X_{i,1}-p_{\theta,i}(X_{0})\right)^{2}\right]\approx
m_{i}^{N}-(m_{i}^{N})^{2}.$ (25)

Combining (20) and (23)-(25), we expect that

${\cal
W}_{\Delta}\approx\frac{1}{N}\sum_{i=1}^{N}(c^{N}_{i})^{2}(m_{i}^{N}-(m_{i}^{N})^{2}).$

Now, as $N$ is large, we have $m_{i}^{N}-(m_{i}^{N})^{2}\approx m(1-m)$ for
all $1\leq i\leq N$. Finally, to get the asymptotics of
$N^{-1}\sum_{i=1}^{N}(c^{N}_{i})^{2}$, one can follow the same lines as in
Subsection 2.1 to check that

$c^{N}\approx 1_{N}+\frac{(1-\lambda)}{D(\lambda,p)}C^{N},$

with $C^{N}=(A^{N})^{\intercal}1_{N}$ and $D(\lambda,p)$ as defined in (7).
Then, by observing that $C^{N}_{k}\approx p$ for $k\in\mathcal{P}_{+}$ and
$C^{N}_{k}\approx-p$ for $k\in\mathcal{P}_{-}$, one can check that
$(c^{N}_{k})^{2}\approx(1+a)^{2}$ for $k\in\mathcal{P}_{+}$ and
$(c^{N}_{k})^{2}\approx(1-a)^{2}$ for $k\in\mathcal{P}_{-}$, with

$a=\frac{(1-\lambda)p}{D(\lambda,p)}.$

Hence,

$\displaystyle\frac{1}{N}\sum_{k=1}^{N}(c^{N}_{k})^{2}$ $\displaystyle\approx$
$\displaystyle 1+a^{2}+2a(r_{+}-r_{-})$ $\displaystyle=$
$\displaystyle\frac{1+(1-\lambda)^{2}p^{2}(1-(r_{+}-r_{-})^{2})}{(D(\lambda,p))^{2}}=\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{(D(\lambda,p))^{2}}.$

All in all, we find that

${\cal W}_{\Delta}\approx
m(1-m)\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{(D(\lambda,p))^{2}}=w.$

This heuristics is made precise in Theorem 6.11.

By relying on the above heuristics now replacing $2\Delta$ by $\Delta$, we
also deduce that ${\cal W}_{2\Delta}$ should converge to $w$ so that

$\hat{w}=2{\cal W}_{2\Delta}-{\cal W}_{\Delta}\approx w.$

The reason of estimating $w$ using $\hat{w}$ instead of ${\cal W}_{\Delta}$
(or ${\cal W}_{2\Delta}$) alone is to improve the convergence rate (see the
beginning of Section 6 and Lemma 6.2 ).

#### Organization of the rest of the paper

The rest of this paper is devoted to the proof of Theorem 2.1. Its proof is an
immediate consequence of Theorems 5.2, 6.11 and 7.5 and is presented in
Section 8. The proofs of Theorems 5.2, 6.11 and 7.5 are given in Sections 5, 6
and 7, respectively. Their proofs are based on two main ingredients. First,
conditionally on a realization of the environment $\theta,$ a precise study of
the decay of correlation of $X_{i,t}$ and $X_{j,s}$ (and of products of
these). This is done by means of a backward regeneration scheme presented in
Section 3. Second, a study of the random matrices $A^{N}$ and $Q^{N}$ and
their associated row and columns sums, which is carried out in Section 4
below. In Section 9, we illustrate the performance of our estimators through
simulations. Finally, many auxiliary results used throughout the paper are
proved in Appendices A, B, C, D and E.

##  3 Backward regeneration scheme

Recall formula (1). First of all, since $\mu\leq\lambda,$ let us introduce
$\beta:=\mu/\lambda$ such that $0\leq\beta\leq 1.$ Then we may interpret
formula (1) in the following way. At any given time $t$, the $i-$th component
first decides to update independently of anything else with probability
$\lambda.$ If it does so, then it decides to send a signal ($X_{i,t}=1$) with
probability $\beta,$ else it does not send a signal. Moreover, if the
component does not update independently of anything else, then it chooses one
of its $N$ neighbors (including $i$ itself) randomly according to the uniform
distribution. Suppose it has chosen $j$ as its neighbor. Then there are three
possibilities:

  * •

if $\theta_{ij}=0$, then $X_{i,t}=0$,

  * •

if $\theta_{ij}=1$ and $j\in\cal{P}_{+}$, then $X_{i,t}$ copies the value of
$X_{j,t-1}$,

  * •

if $\theta_{ij}=1$ and $j\in\cal{P}_{-}$, then $X_{i,t}$ copies $1-X_{j,t-1}$.

This interpretation is formalized in Equation (27) below, where the process is
constructed via a backward regeneration scheme.

Models like this are called imitation models and have been studied in the one-
dimensional frame in De Santis and Piccioni (2015). Since $\lambda>0,$ at each
time step the probability of making an update which is independent of anything
else is strictly positive, and this implies the existence of a unique
invariant measure of the system as well as the possibility of perfectly
sampling from it, which is formalized in the following section.

###  3.1 Backward regeneration representation

#### Notation

In what follows, we denote by $z=(i,t)$ some space-time coordinate in
$\mathcal{Z}=\\{1,\dots,N\\}\times\mathbb{Z}$ and we write $X_{z}$ instead of
$X_{i,t}$ for all $z=(i,t)\in\mathcal{Z}$.

To construct the Markov chain $X=\\{X_{i,t},t\in\mathbb{Z},1\leq i\leq N\\}$
via a backward regeneration representation, we introduce the following random
variables. To each space-time coordinate $z\in\mathcal{Z}$, we associate a
couple of independent random variables $(J_{z},\xi_{z})$ taking values in
$\\{0,1,\ldots,N\\}\times\\{0,1\\}$ such that
$I\\!\\!P(J_{z}=0)=\lambda,I\\!\\!P(J_{z}=k)=(1-\lambda)\frac{1}{N}$ for
$1\leq k\leq N,$ and $I\\!\\!P(\xi_{z}=1)=1-I\\!\\!P(\xi_{z}=0)=\beta$.
Moreover, we suppose that the couples $((J_{z},\xi_{z}),z\in{\cal Z})$ are
independent. By convention, let us define $J_{(0,t)}=0$ for all
$t\in\mathbb{Z}$.

Then, for any $z=(i,t)\in\mathcal{Z}$, let us define a backward random walk
$I^{z}=(I^{z}_{s})_{s\in\mathbb{Z}}$ taking values in the state space
$\\{0,1,\dots,N,\infty\\}$, where $I^{z}_{s}=\infty$ for all $s>t$,
$I^{z}_{t}=i$ and, as $s$ decreases, $I^{z}$ follows the space coordinates
given by the $J$ variables, that is

$I^{z}_{s-1}=J_{(I^{z}_{s},s)},\text{ for all $s\leq t$.}$ (26)

Because of the convention $J_{(0,t)}=0$, the state $0$ is a cemetery for the
process $I^{z}$. Let us denote $\tau^{R}_{z}$ the last time that the random
walk $I^{z}$ is not in the cemetery state $0$:

$\tau^{R}_{z}=\inf\\{s\in\mathbb{Z}:I_{s}^{z}>0\\}.$

Reaching the cemetery happens after a finite time, almost surely for all the
sites.

######  Proposition 3.1.

For any $\theta$, it holds that $I\\!\\!P_{\theta}(\forall
z\in\mathcal{Z},\tau^{R}_{z}>-\infty)=1$.

###### Proof.

Since $\mathcal{Z}$ is countable, it suffices to check that
$I\\!\\!P_{\theta}(\tau^{R}_{z}>-\infty)=1$ for all $z\in\mathcal{Z}$.

For each $k\geq 1$,

$I\\!\\!P(\tau^{R}_{z}\leq t-k)=I\\!\\!P(I^{z}_{t-k}\neq
0,\ldots,I^{z}_{t-1}\neq 0)\leq(1-\lambda)^{k},$

where we used the independence between the random variables $J_{\tilde{z}}$
and the fact that $I\\!\\!P(J_{\tilde{z}}\neq 0)=1-\lambda.$ It follows that
$I\\!\\!P(\tau^{R}_{z}=-\infty)=\lim_{k\to\infty}I\\!\\!P(\tau^{R}_{z}\leq
t-k)=0$. ∎

The time $\tau^{R}_{z}$ is called a regeneration time for state $z$. Let us
then define the set of regenerating sites by

${\mathcal{R}}=\\{z\in{\cal Z}:J_{z}=0\\}.$

######  Remark 3.2.

The sites $z=(i,t)$ such that $\theta_{iJ_{z}}=0$ and $J_{z}\geq 1$ could also
be considered as regenerating sites. Since we are interested in results valid
for any realization of $\theta$, we did not include those sites in the
definition of $\mathcal{R}$.

Observe that since $I^{z}_{\tau^{R}_{z}-1}=J_{(I^{z}_{\\\
tau^{R}_{z}},\tau^{R}_{z})}=0$, we have that
$(I^{z}_{\tau^{R}_{z}},\tau^{R}_{z})\in\mathcal{R}$ is a regenerating site.

In summary, the random walk $I^{z}$ equals $+\infty$ for all times $u>t,$ that
is, before it “starts to live”. Then, the random walk lives in
$\\{1,\dots,N\\}$ until it reaches $0$ and thus regenerates. It then remains
in state $0$ forever.

For all $z=(i,t)\in\mathcal{Z}$, we define $X_{z}$ as follows:

$X_{z}=\xi_{z}{{\mathbf{1}}}_{z\in\mathcal{R}}+\theta_{iJ_{z}}\left(X_{J_{z},t-1}\mathbf{1}_{J_{z}\in\cal{P}_{+}}+(1-X_{J_{z},t-1})\mathbf{1}_{J_{z}\in\cal{P}_{-}}\right){{\mathbf{1}}}_{z\notin\mathcal{R}}.$
(27)

Note that, if $z\in\mathcal{R}$, then $X_{z}$ is sampled according to the
Bernoulli random variable $\xi_{z}$. When $z\notin\mathcal{R}$, the random
variables $X_{z}$ are defined recursively. On $\Omega_{0}=\left\\{\forall
z\in\mathcal{Z},\tau^{R}_{z}>-\infty\right\\}$, this recursion ends in finite
time for every site and so the process $X$ constructed via (27) is well-
defined. Proposition 3.1 ensures that $I\\!\\!P_{\theta}(\Omega_{0})=1$,
implying that the process $X$ is well-defined almost surely. In the theorem
below, we prove that the process $X$ is a stationary Markov chain with
transition probability given by (2) and (1).

######  Theorem 3.3.

Let $X=(X_{z})_{z\in\mathcal{Z}}$ be the process defined through (27). Then,
conditionally on $\theta$, $X$ is a stationary Markov chain with transition
probability given by (2) and (1).

The proof of Theorem 3.3 is given in Appendix A. The regeneration
representation presented above induces a natural (random) partitioning of the
state space $\mathcal{Z}$ via the equivalence relation of coalescence defined
by

$z_{1}\leftrightsquigarrow z_{2}\mbox{ if and only if }\exists
s\in\mathbb{Z},I^{z_{1}}_{s}=I^{z_{2}}_{s}\notin\\{0,\infty\\}.$

The next section is devoted to a study of this equivalence relation.

###  3.2 Coalescence of the backward random walks

For any $z_{1},z_{2}\in\mathcal{Z}$, let us define the random time

$\tau^{c}_{z_{1},z_{2}}=\sup\left\\{s\in\mathbb{Z}:I^{z_{1}}_{s}=I^{z_{2}}_{s}\notin\\{0,\infty\\}\right\\},$
(28)

namely the time at which $z_{1}$ and $z_{2}$ coalesce (with the convention
that $\tau_{c}^{z_{1},z_{2}}=-\infty$ if they do not coalesce, denoted by
$z_{1}\not\leftrightsquigarrow z_{2}$). The coalescence property is related to
the dependence between the coordinates of the process $X$ as stated in the two
lemmas below. In words, if $z_{1}$ does not coalesce with $z_{2}$ then
$X_{z_{1}}$ behaves as a copy of itself which is independent of $X_{z_{2}}$.
Their proofs are given in Appendix B.

######  Lemma 3.4 (All independent coupling).

Let $k\geq 2$ and consider $k$ different points $z_{1},\dots,z_{k}$ in
$\mathcal{Z}$. There exist random variables
$\tilde{X}_{z_{1}},\dots,\tilde{X}_{z_{k}}$ satisfying the following
properties: for each $i\in\\{1,\ldots,k\\}$,

  1. (i)

$\tilde{X}_{z_{i}}\,{\buildrel d\over{=}}\,X_{z_{i}}$ (i.e.,
$\tilde{X}_{z_{i}}$ and $X_{z_{i}}$ have the same distribution);

  2. (ii)

$\tilde{X}_{z_{i}}$ is independent of
$(X_{z_{j}},\tilde{X}_{z_{j}})_{j\in\\{1,\ldots,k\\}\setminus\\{i\\}};$

  3. (iii)

$\\{\tilde{X}_{z_{i}}\neq
X_{z_{i}}\\}\subset\cup_{j\in\\{1,\ldots,k\\}\setminus\\{i\\}}\\{z_{i}\leftrightsquigarrow
z_{j}\\}.$

######  Lemma 3.5 (Blockwise independence).

Let $z_{1},\dots,z_{4}$ be four different points in $\mathcal{Z}$. There exist
random variables $\tilde{X}_{z_{1}},\dots,\tilde{X}_{z_{4}}$ and
$\hat{X}_{z_{1}},\hat{X}_{z_{2}}$ satisfying the following properties:

  1. (i)

$(\hat{X}_{z_{1}},\hat{X}_{z_{2}})\,{\buildrel
d\over{=}}\,(X_{z_{1}},X_{z_{2}})$;

  2. (ii)

$(\hat{X}_{z_{1}},\hat{X}_{z_{2}})$ is independent of
$(X_{z_{3}},X_{z_{4}},\tilde{X}_{z_{3}},\tilde{X}_{z_{4}})$;

  3. (iii)

$\\{\hat{X}_{z_{1}}\neq
X_{z_{1}}\\}\subset\cup_{k=3}^{4}\\{z_{1}\leftrightsquigarrow
z_{k}\\}\cup\left(\\{\tau^{R}_{z_{1}}<t_{3}\vee
t_{4}\\}\cap(\cup_{k=3}^{4}\\{z_{2}\leftrightsquigarrow z_{k}\\})\right)$,

  4. (iv)

$\\{\hat{X}_{z_{2}}\neq
X_{z_{2}}\\}\subset\cup_{k=3}^{4}\\{z_{2}\leftrightsquigarrow
z_{k}\\}\cup\left(\\{\tau^{R}_{z_{2}}<t_{3}\vee
t_{4}\\}\cap(\cup_{k=3}^{4}\\{z_{1}\leftrightsquigarrow z_{k}\\})\right)$.

  5. (v)

$\hat{X}_{z_{1}}$ is independent of $\tilde{X}_{z_{2}}$ and $\hat{X}_{z_{2}}$
is independent of $\tilde{X}_{z_{1}}$.

Moreover, for each $i\in\\{1,\ldots,4\\}$,

  1. (iv)

$\tilde{X}_{z_{i}}\,{\buildrel d\over{=}}\,X_{z_{i}}$;

  2. (v)

$\tilde{X}_{z_{i}}$ is independent of
$(\tilde{X}_{z_{j}},X_{z_{j}},I^{z_{j}})_{j\in\\{1,\ldots,4\\}\setminus\\{i\\}}$;

  3. (vi)

$\\{\tilde{X}_{z_{i}}\neq
X_{z_{i}}\\}\subset\cup_{j\in\\{1,\ldots,4\\}\setminus\\{i\\}}\\{z_{i}\leftrightsquigarrow
z_{j}\\}.$

######  Remark 3.6.

Let us describe with some words the item (iii) above: if the hat version
$\hat{X}_{z_{1}}$ is different from the standard version $X_{z_{1}}$, then we
know that

  * •

$z_{1}$ coalesces with $z_{3}$ or $z_{4}$, or

  * •

$z_{2}$ coalesces with $z_{3}$ or $z_{4}$ and the (backward) regeneration time
of $z_{1}$ is less than the starting times of $z_{3}$ or $z_{4}$.

In the next result, we provide upper bounds for the probability of events
involving the coalescence of two or more backward random walks.

######  Proposition 3.7.

Let $z_{k}=(i_{k},t_{k})\in\mathcal{Z}$, for $k=1,\dots,4$, be four different
sites. There exists a constant $K$ only depending on $\lambda$ such that the
following inequalities hold.

  1. (i)

$\displaystyle I\\!\\!P_{\theta}\left(\\{z_{1}\leftrightsquigarrow
z_{2}\\}\right)\leq\frac{(1-\lambda)^{|t_{1}-t_{2}|\vee
1}}{1-(1-\lambda)^{2}}N^{-1}.$

  2. (ii)

$\displaystyle I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\leftrightsquigarrow z_{3}\\})\leq
K(1-\lambda)^{(\overline{t}-\underline{t})}N^{-2},$ where
$\overline{t}=t_{1}\vee t_{2}\vee t_{3}$ and $\underline{t}=t_{1}\wedge
t_{2}\wedge t_{3}$.

  3. (iii)

$\displaystyle I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\\}\cap\\{z_{3}\leftrightsquigarrow z_{4}\\})\leq
K\left((1-\lambda)^{|t_{1}-t_{2}|+|t_{3}-t_{4}|}N^{-2}+(1-\lambda)^{(\overline{t}-\underline{t})}N^{-3}\right),$
where $\overline{t}=t_{1}\vee t_{2}\vee t_{3}\vee t_{4}$ and
$\underline{t}=t_{1}\wedge t_{2}\wedge t_{3}\wedge t_{4}.$

  4. (iv)

If $t_{1}\wedge t_{2}\geq t_{3}\vee t_{4}$,

$I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow z_{3}\leftrightsquigarrow
z_{4}\not\leftrightsquigarrow z_{2}\\}\cap\\{\tau^{R}_{z_{2}}<t_{3}\vee
t_{4}\\})\leq K(1-\lambda)^{t_{1}+t_{2}-t_{3}-t_{4}}N^{-2}.$

The proof of Proposition 3.7 is given in Appendix C. For all
$z\in\mathcal{Z}$, let us denote $Y_{z}=X_{z}-I\\!\\!E_{\theta}(X_{z})$ the
centered version of $X_{z}$ and observe that $|Y_{z}|\leq 1$ almost surely. As
a consequence of Lemma 3.5 and Proposition 3.7, we obtain several upper bounds
on the covariance of : the variables $Y_{z}$’s (Lemma 3.8), products of the
variables $Y_{z}$’s (Lemma 3.9). This last result is the main result of this
section.

######  Lemma 3.8.

Let $z_{1}=(i_{1},t_{1})$ and $z_{2}=(i_{2},t_{2})$ two points in
$\mathcal{Z}$. There exists a constant $K$ only depending on $\lambda$ such
that: if $z_{1}\neq z_{2}$,

$\left|\operatorname{\mathop{Cov}}_{\theta}\left[X_{z_{1}},X_{z_{2}}\right]\right|=\left|\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}},Y_{z_{2}}\right]\right|\leq
K(1-\lambda)^{|t_{1}-t_{2}|\vee 1}N^{-1}.$

Otherwise, the quantity above is obviously bounded by $1$.

###### Proof.

The proof is the same as the one for item 3 of Lemma 3.9 below. ∎

######  Lemma 3.9.

Let $z_{k}=(i_{k},t_{k})\in\mathcal{Z},$ $k=1,\dots,4$, and denote
$B=\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}}\,,\,Y_{z_{3}}Y_{z_{4}}\right]$
and $E=\\{z_{k}:k=1,\dots,4\\}$. There exists a constant $K$ only depending on
$\lambda$ such that:

  1. 1.

If $\\#E=1,$ then $|B|\leq 1.$

  2. 2.

If $\\#E=2$, $z_{1}\neq z_{2}$ and $z_{3}\neq z_{4}$, then $|B|\leq 1.$

  3. 3.

If $\\#E=2$, $z_{1}=z_{2}$ and $z_{3}=z_{4}$, then

$|B|\leq K(1-\lambda)^{|t_{1}-t_{3}|}N^{-1}.$

  4. 4.

If $\\#E=3$ and $z_{1}=z_{2}$ or $z_{3}=z_{4}$, then

$|B|\leq K(1-\lambda)^{|s_{3}-s_{2}|+|s_{2}-s_{1}|}N^{-2},$

where $s_{1}\leq s_{2}\leq s_{3}$ is an ordering of
$\\{t_{1},t_{2},t_{3},t_{4}\\}$.

  5. 5.

If $\\#E=3$, $z_{1}\neq z_{2}$ and $z_{3}\neq z_{4}$, for instance assume that
$z_{1}=z_{3}$ (and so $z_{2}\neq z_{4}$), then

$|B|\leq K(1-\lambda)^{|t_{2}-t_{4}|}N^{-1}.$

  6. 6.

If $\\#E=4$ and $t_{1}\wedge t_{2}\geq t_{3}\vee t_{4}$, then

$|B|\leq K(1-\lambda)^{t_{1}\vee t_{2}-t_{3}\wedge t_{4}}N^{-2}.$

  7. 7.

If $\\#E=4$, then

$|B|\leq K(1-\lambda)^{|s_{2}-s_{1}|+|s_{3}-s_{4}|}N^{-2},$

where $s_{1}\leq\dots\leq s_{4}$ is an ordering of $\\{t_{1},\dots,t_{4}\\}$.

######  Remark 3.10.

Observe that we cannot have $z_{1}=z_{2}$ and $z_{3}=z_{4}$ simultaneously in
item 4 of Lemma 3.9. In particular, in the case $z_{1}=z_{2}$ (resp.
$z_{3}=z_{4}$), the set of time indices $\\{t_{1},t_{2},t_{3},t_{4}\\}$
reduces to the set $\\{t_{1},t_{3},t_{4}\\}$ (resp.
$\\{t_{1},t_{2},t_{3}\\}$).

###### Proof.

The statements in both items 1 and 2 are trivial, so that we only need to
prove items 3 through 7.

#### Proof of item 3

In this case,
$B=\operatorname{\mathop{Cov}}_{\theta}[Y^{2}_{z_{1}},Y^{2}_{z_{3}}]$. Let
$\tilde{X}_{z_{1}}$ be a random variable distributed as $X_{z_{1}}$,
independent of $X_{z_{3}}$ and such that
$\\{X_{z_{1}}\neq\tilde{X}_{z_{1}}\\}\subset\\{z_{1}\leftrightsquigarrow
z_{3}\\}$. The existence of such random variables is ensured by Lemma 3.4 with
$k=2$. In particular, we have that
$I\\!\\!E_{\theta}[\tilde{X}_{z_{1}}]=I\\!\\!E_{\theta}[X_{z_{1}}]$. Denote
$\tilde{Y}_{z_{1}}=\tilde{X}_{z_{1}}-I\\!\\!E_{\theta}[\tilde{X_{z_{1}}}]$ and
observe that $\\{\tilde{Y}_{z_{1}}\neq Y_{z_{1}}\\}=\\{\tilde{X}_{z_{1}}\neq
X_{z_{1}}\\}$. By using the properties satisfied by random variables
$\tilde{X}_{z_{1}},$ one can check that $\tilde{Y}_{z_{1}}$ is independent of
$Y_{z_{3}}$, so that

$B=\operatorname{\mathop{Cov}}_{\theta}[Y^{2}_{z_{1}},Y^{2}_{z_{3}}]=I\\!\\!E_{\theta}[(Y^{2}_{z_{1}}-\tilde{Y}^{2}_{z_{1}})Y^{2}_{z_{3}}].$

Since $\\{\tilde{Y}_{z_{1}}\neq Y_{z_{1}}\\}=\\{\tilde{X}_{z_{1}}\neq
X_{z_{1}}\\}\subset\\{z_{1}\leftrightsquigarrow z_{3}\\}$ and
$\max\\{|Y_{z_{1}}|,|\tilde{Y}_{z_{1}}|,|Y_{z_{3}}|\\}\leq 1$ almost surely,
it follows that

$|B|\leq I\\!\\!P_{\theta}\left(z_{1}\leftrightsquigarrow z_{3}\right),$

so that the result follows item (i) of Proposition 3.7.

#### Proof of item 4

Suppose that $z_{1}=z_{2}.$ Then
$B=\operatorname{\mathop{Cov}}_{\theta}[Y^{2}_{z_{1}},Y_{z_{3}}Y_{z_{4}}]$. By
Lemma 3.4 with $k=3$, there exist random variables
$\tilde{Y}_{z_{1}},\tilde{Y}_{z_{3}}$ and $\tilde{Y}_{z_{4}}$ defined in such
a way that the following properties hold for each $i\in\\{1,3,4\\}$: 1)
$\tilde{Y}_{z_{i}}$ has the same law as $Y_{z_{i}}$; 2) $\tilde{Y}_{z_{i}}$ is
independent of $\tilde{Y}_{z_{j}}$ and $Y_{z_{j}}$ for all
$j\in\\{1,3,4\\}\setminus\\{i\\}$; 3) $\\{\tilde{Y}_{z_{i}}\neq
Y_{z_{i}}\\}\subset\cup_{j\in\\{1,3,4\\}\setminus\\{i\\}}\\{z_{i}\leftrightsquigarrow
z_{j}\\}.$ Using first that $Y^{2}_{z_{1}}$ has same law as
$\tilde{Y}^{2}_{z_{1}}$ and then that
$I\\!\\!E_{\theta}[\tilde{Y}_{z_{i}}]=I\\!\\!E_{\theta}[Y_{z_{i}}]=0$, we can
deduce from properties 1 and 2 above that

$\displaystyle B$ $\displaystyle=$ $\displaystyle
I\\!\\!E_{\theta}[(Y^{2}_{z_{1}}-\tilde{Y}^{2}_{z_{1}})Y_{z_{3}}Y_{z_{4}}]$
$\displaystyle=$ $\displaystyle
I\\!\\!E_{\theta}[(Y^{2}_{z_{1}}-\tilde{Y}^{2}_{z_{1}})(Y_{z_{3}}-\tilde{Y}_{z_{3}})(Y_{z_{4}}-\tilde{Y}_{z_{4}})].$

By combining property 3, the above identity and the fact that
$\leftrightsquigarrow$ is an equivalence relation, we can deduce that

$|B|\leq I\\!\\!E_{\theta}\left[1_{\\{z_{1}\leftrightsquigarrow
z_{3}\leftrightsquigarrow
z_{4}\\}}\right]=I\\!\\!P_{\theta}(\\{z_{1}\leftrightsquigarrow
z_{2}\leftrightsquigarrow z_{3}\\}),$

so that the result follows from item (ii) of Proposition 3.7.

#### Proof of item 5

Let us consider the case in which $z_{1}=z_{3}$. Let $z_{1},z_{2},z_{4}$ be
three different sites. The goal is to bound the covariance

$B=\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}},Y_{z_{1}}Y_{z_{4}}\right]=I\\!\\!E_{\theta}\left[Y^{2}_{z_{1}}Y_{z_{2}}Y_{z_{4}}\right]-\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}}\right]\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{4}}\right].$
(29)

Arguing exactly like in the proof of item 3, one can show that
$\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}}\right]\leq
K(1-\lambda)^{|t_{1}-t_{2}|}N^{-1}$ and
$\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{4}}\right]\leq
K(1-\lambda)^{|t_{1}-t_{4}|}N^{-1}$, so that

$\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{2}}\right]\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{4}}\right]\leq
K(1-\lambda)^{|t_{1}-t_{2}|+|t_{1}-t_{4}|}N^{-2}.$ (30)

We will now deal with the term
$I\\!\\!E_{\theta}\left[Y^{2}_{z_{1}}Y_{z_{2}}Y_{z_{4}}\right].$ For that end,
let $\tilde{X}_{z_{1}}$, $\tilde{X}_{z_{2}}$ and $\tilde{X}_{z_{4}}$ be the
random variables defined in Lemma 3.4 with $k=3$, and denote
$\tilde{Y}_{z_{i}}=\tilde{X}_{z_{i}}-I\\!\\!E_{\theta}(X_{z_{i}})$,
$i\in\\{1,2,4\\}$ the corresponding centered versions. By using the
independence properties of the random variables $\tilde{Y}_{z_{i}}$ (and the
fact that they are all centered), one can check that

$I\\!\\!E_{\theta}\left[Y^{2}_{z_{1}}Y_{z_{2}}Y_{z_{4}}\right]=I\\!\\!E_{\theta}\left[(Y^{2}_{z_{1}}-\tilde{Y}^{2}_{z_{1}})(Y_{z_{2}}-\tilde{Y}_{z_{2}})(Y_{z_{4}}-\tilde{Y}_{z_{4}})\right]+I\\!\\!E_{\theta}[\tilde{Y}^{2}_{z_{1}}]\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{2}}Y_{z_{4}}\right].$

Arguing exactly like in the proof of item 4, one can deduce that

$I\\!\\!E_{\theta}\left[(Y^{2}_{z_{1}}-\tilde{Y}^{2}_{z_{1}})(Y_{z_{2}}-\tilde{Y}_{z_{2}})(Y_{z_{4}}-\tilde{Y}_{z_{4}})\right]\leq
K(1-\lambda)^{|s_{3}-s_{2}|+|s_{2}-s_{1}|}N^{-2},$ (31)

where $s_{1}\leq s_{2}\leq s_{3}$ is an ordering of $\\{t_{1},t_{2},t_{4}\\}$.
Finally, since
$\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{2}}Y_{z_{4}}\right]\leq
K(1-\lambda)^{|t_{2}-t_{4}|}N^{-1}$, the random variables $\tilde{Y}_{z_{k}}$
are bounded by 1 almost surely and
$|t_{2}-t_{4}|\leq\min\\{|t_{2}-t_{1}|+|t_{1}-t_{4}|,|s_{3}-s_{2}|+|s_{2}-s_{1}|\\}$,
it follows from (29), (30) and (31) that

$|B|\leq K(1-\lambda)^{|t_{2}-t_{4}|}N^{-1},$

concluding the proof.

#### Proof of item 6

Let $z_{1},\dots,z_{4}$ be four different points in $\cal Z$ and denote by
$\hat{X}_{z_{1}},\hat{X}_{z_{2}}$,
$\tilde{X}_{z_{1}},\ldots,\tilde{X}_{z_{4}}$ the random variables defined in
Lemma 3.5. Moreover, denote by
$\tilde{Y}_{z_{k}}=\tilde{X}_{z_{k}}-I\\!\\!E_{\theta}(\tilde{X}_{z_{k}})$,
$1\leq k\leq 4$ and
$\hat{Y}_{z_{k}}=\hat{X}_{z_{k}}-I\\!\\!E_{\theta}(\hat{X}_{z_{k}})$, $1\leq
k\leq 2$, their centered versions. First of all, observe that by using items
(i), (ii) and (iii) of Lemma 3.5, we can rewrite B as

$B=I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{\\{z_{1}\leftrightsquigarrow
z_{3}\\}\cup\\{z_{1}\leftrightsquigarrow
z_{4}\\}\cup\\{z_{2}\leftrightsquigarrow
z_{3}\\}\cup\\{z_{2}\leftrightsquigarrow z_{4}\\}}\right].$

In the remaining of the proof, we adopt the following notation. For
$i,j,k,\ell\in\\{1,\ldots,4\\}$, we write $\\{ijk\ell\\}$, $\\{ijk|\ell\\}$,
$\\{ij|k\ell\\}$ and $\\{ij|k|\ell\\}$ to denote, respectively, the events
$\\{z_{i}\leftrightsquigarrow z_{j}\leftrightsquigarrow
z_{k}\leftrightsquigarrow z_{\ell}\\},$ $\\{z_{i}\leftrightsquigarrow
z_{j}\leftrightsquigarrow z_{k}\not\leftrightsquigarrow z_{\ell}\\}$,
$\\{z_{i}\leftrightsquigarrow z_{j}\not\leftrightsquigarrow
z_{k}\leftrightsquigarrow z_{\ell}\\}$ and $\\{z_{k}\not\leftrightsquigarrow
z_{i}\leftrightsquigarrow z_{j}\not\leftrightsquigarrow z_{\ell}\\}.$ With
this notation, observe that we can write

$1_{\\{z_{1}\leftrightsquigarrow z_{3}\\}\cup\\{z_{1}\leftrightsquigarrow
z_{4}\\}\cup\\{z_{2}\leftrightsquigarrow
z_{3}\\}\cup\\{z_{2}\leftrightsquigarrow
z_{4}\\}}=1_{\\{1234\\}}+1_{\\{123|4\\}}+1_{\\{124|3\\}}+1_{\\{134|2\\}}+1_{\\{234|1\\}}\\\
+1_{\\{13|24\\}}+1_{\\{14|23\\}}+1_{\\{13|2|4\\}}+1_{\\{14|2|3\\}}+1_{\\{23|1|4\\}}+1_{\\{24|1|4\\}}.$

Let us denote

$M=\max\\{I\\!\\!P_{\theta}(1234),I\\!\\!P_{\theta}(13|24),I\\!\\!P_{\theta}(14|23),\\\
I\\!\\!P_{\theta}(\\{134|2\\}\cap\\{\tau^{R}_{z_{2}}<t_{3}\vee
t_{4}\\}),I\\!\\!P_{\theta}(\\{234|1\\}\cap\\{\tau^{R}_{z_{1}}<t_{3}\vee
t_{4}\\})\\}.$

#### Step 1

Since the random variables $Y_{z_{i}}$’s and $\hat{Y}_{z_{i}}$’s are bounded
by $1$ almost surely, we clearly have

$\left|I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}\left(1_{\\{1234\\}}+1_{\\{13|24\\}}+1_{\\{14|23\\}}\right)\right]\right|\leq
6M.$

#### Step 2

Let $E$ denote one of the following events $\\{123|4\\}$ or $\\{124|3\\}$.
Then, we prove the following inequality

$\left|I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{E}\right]\right|\leq
2M.$ (32)

To do so, let us assume that $E=\\{123|4\\}$. The other case is treated
similarly. By item (vi) of Lemma 3.5, we have that
$Y_{z_{4}}=\tilde{Y}_{z_{4}}$ on $E$, so that

$I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{E}\right]=I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{E}\right].$

By using that $1_{\\{123|4\\}}+1_{\\{1234\\}}=1_{\\{123\\}}$ is
$\sigma(I^{z_{1}},I^{z_{2}},I^{z_{3}})$-measurable and the fact that
$\tilde{Y}_{z_{4}}$ is a centered random variable which is independent of
$\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}1_{\\{123\\}}$
(thanks to items (ii) and (v) of Lemma 3.5), it follows that

$I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{E}\right]=-I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{\\{1234\\}}\right].$

Finally, we conclude like in Step 1.

#### Step 3

Let $E$ denote one of the following events $\\{134|2\\}$ or $\\{234|1\\}$.
Then, we prove the following inequality

$\left|I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{E}\right]\right|\leq
5M.$ (33)

To do so, let us assume that $E=\\{134|2\\}$. The other case is treated
similarly. In the following, let us denote $E_{1}=E\cap\\{\tau^{R}_{z_{2}}\geq
t_{3}\vee t_{4}\\}$ and $E_{2}=E\cap\\{\tau^{R}_{z_{2}}<t_{3}\vee t_{4}\\}$.
On the one hand, we use the fact that both

$I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{E_{2}}\right]\text{
and
}I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}-\hat{Y}_{z_{1}}\right)\tilde{Y}_{z_{2}}Y_{z_{3}}Y_{z_{4}}1_{E_{2}}\right]$

are upper bounded by $2I\\!\\!P_{\theta}\left(E_{2}\right)$. On the other
hand, we know by items (iii) and (vi) of Lemma 3.5, that
$\hat{Y}_{z_{2}}=Y_{z_{2}}=\tilde{Y}_{z_{2}}$ on $E_{1}$. Combining those two
properties, we can get

$\left|I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{E}\right]\right|\leq\left|I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}-\hat{Y}_{z_{1}}\right)\tilde{Y}_{z_{2}}Y_{z_{3}}Y_{z_{4}}1_{E}\right]\right|+4I\\!\\!P_{\theta}\left(E_{2}\right).$

By using that $1_{\\{134|2\\}}+1_{\\{1234\\}}=1_{\\{134\\}}$ is
$\sigma(I^{z_{1}},I^{z_{3}},I^{z_{4}})$-measurable and the fact that
$\tilde{Y}_{2}$ is a centered random variable which is independent of
$\left(Y_{z_{1}}-\hat{Y}_{z_{1}}\right)Y_{z_{3}}Y_{z_{4}}1_{\\{134\\}}$
(thanks to item (v) of Lemma 3.5), it follows that

$I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}-\hat{Y}_{z_{1}}\right)\tilde{Y}_{z_{2}}Y_{z_{3}}Y_{z_{4}}1_{\\{134|2\\}}\right]=-I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}-\hat{Y}_{z_{1}}\right)\tilde{Y}_{z_{2}}Y_{z_{3}}Y_{z_{4}}1_{\\{1234\\}}\right].$

Finally, we conclude like in Step 1.

#### Step 4

Let $E$ denote one of the following events: $\\{13|2|4\\}$, $\\{14|2|3\\}$,
$\\{23|1|4\\}$ or $\\{24|1|3\\}$. Then, we prove the following inequality

$\left|I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{E}\right]\right|\leq
7M.$

To do so, let us assume that $E=\\{13|2|4\\}$. The other cases are treated
similarly. First, we combine the fact that $Y_{z_{4}}=\tilde{Y}_{z_{4}}$ on
$E$ (this holds by item (vi) of Lemma 3.5) together with the decomposition
$1_{\\{13|2|4\\}}=1_{13|2}-1_{13|24}-1_{134|2}$ to deduce that

$I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{\\{13|2|4\\}}\right]=I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{\\{13|2\\}}\right]\\\
-I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{\\{13|24\\}}\right]-I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{\\{134|2\\}}\right].$

Now, since $\tilde{Y}_{z_{4}}$ is centered and independent of
$Y_{z_{1}},\hat{Y}_{z_{1}},Y_{z_{2}},\hat{Y}_{z_{2}},Y_{z_{3}}$ and $1_{13|2}$
(because this random variable is
$\sigma(I^{z_{1}},I^{z_{2}},I^{z_{3}})$-measurable), we have that

$I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{\\{13|2\\}}\right]=0,$

which implies that

$I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}Y_{z_{4}}1_{\\{13|2|4\\}}\right]=-I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{\\{13|24\\}}\right]\\\
-I\\!\\!E_{\theta}\left[\left(Y_{z_{1}}Y_{z_{2}}-\hat{Y}_{z_{1}}\hat{Y}_{z_{2}}\right)Y_{z_{3}}\tilde{Y}_{z_{4}}1_{\\{134|2\\}}\right].$

The first term on the right-hand side of the above equality is in absolute
value at most $2M$. The second one can be dealt with proceeding similarly as
Step 3.

Combining steps 1 to 4, we get $|B|\leq 48M$, and items (iii) and (iv) of
Proposition 3.7 give the upperbound $M\leq K(1-\lambda)^{t_{1}\vee
t_{2}-t_{3}\wedge t_{4}}N^{-2},$ which allows to conclude.

#### Proof of item 7

By symmetry, we only need to consider the case $t_{1}\leq t_{2}$ and
$t_{3}\leq t_{4}.$ The case $t_{1}\geq t_{3}$ or $t_{2}\leq t_{3}$ has already
been treated in item 6. So suppose that either $t_{1}\leq t_{3}\leq t_{2}\leq
t_{4}$ or that $t_{3}\leq t_{1}\leq t_{4}\leq t_{2}.$ Since
$I\\!\\!E_{\theta}[Y_{z_{i}}]=0$, we can write

$B=\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{3}},Y_{z_{2}}Y_{z_{4}}\right]+\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}},Y_{z_{3}}\right]\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{2}},Y_{z_{4}}\right]\\\
-\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}},Y_{z_{2}}\right]\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{3}},Y_{z_{4}}\right].$
(34)

By Lemma 3.8, it follows that

$|\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}},Y_{z_{3}}\right]\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{2}},Y_{z_{4}}\right]|+|\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}},Y_{z_{2}}\right]\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{3}},Y_{z_{4}}\right]|\\\
\leq
KN^{-2}\left[(1-\lambda)^{|t_{1}-t_{3}|+|t_{2}-t_{4}|}+(1-\lambda)^{|t_{1}-t_{2}|+|t_{3}-t_{4}|}\right].$
(35)

Also, from item 6 above, we have that

$\operatorname{\mathop{Cov}}_{\theta}\left[Y_{z_{1}}Y_{z_{3}},Y_{z_{2}}Y_{z_{4}}\right]\leq
K(1-\lambda)^{\max t_{i}-\min t_{i}}N^{-2}.$ (36)

Combing (34), (35) and (36), we obtain the assertion. ∎

##  4 Study of a random matrix

The aim of this section is to obtain the convergence of three quantities
related to the random environment $\theta$. The main result of this section is
Proposition 4.7.

#### Notation

Hereafter, for any subset $S$ of $[N]$, we write $1_{S}$ to indicate the
$N$-dimensional vector having value $1$ in each coordinate belonging to $S$
and value $0$ in the remaining coordinates. To alleviate the notation, we will
simply write $1_{N}$ and $0_{N}$ instead of $1_{[N]}$ and $1_{\emptyset}$,
respectively. For any vector $v\in\mathbb{R}^{N}$,
$\overline{v}=N^{-1}\sum_{i=1}^{N}v_{i}$ denotes the arithmetic mean of the
coordinates of $v$, $\|v\|_{r}=\left(\sum_{i=1}^{N}|v_{i}|^{r}\right)^{1/r}$
where $r\in[1,\infty)$, denotes the $r$-norm of $v$ and
$\|v\|_{\infty}=\max_{1\leq i\leq N}|v_{i}|$ its $\infty$-norm. For vectors
$v,u\in\mathbb{R}^{N}$, we write $v\odot u$ to denote the vector whose $i-$th
coordinate is $v_{i}u_{i},$ for $1\leq i\leq N$. In other words, $v\odot u$ is
the Hadamard product between the vectors $v$ and $u$. To shorten the notation,
we will simply note $v^{2}$ instead of $v\odot v$. Observe that, with this
notation, the orthogonal projection of a vector $v\in\mathbb{R}^{N}$ on the
coordinates in $S\subseteq[N]$ can be written as $1_{S}\odot v$. In
particular, all coordinates in $S^{c}$ of the vector $1_{S}\odot v$ are null.
One can always write $v=1_{S}\odot v+1_{S^{c}}\odot v.$

For any $N$-by-$N$ matrix $B$ with real entries, we denote $B^{\intercal}$ its
transpose. For all $r\in[1,\infty]$, we denote $|||B|||_{r}$ the operator norm
of $B$ associated to the $r$-norm $\|\cdot\|_{r}:$

$|||B|||_{r}=\sup_{v\in\mathbb{R}^{N}:v\neq
0_{N}}\frac{\|Bv\|_{r}}{\|v\|_{r}}.$

It is well-known that $|||B|||_{1}$ and $|||B|||_{\infty}$ may be defined
alternatively as

$|||B|||_{1}=\max_{1\leq j\leq N}\sum_{i=1}^{N}|B(i,j)|\ \text{and}\
|||B|||_{\infty}=\max_{1\leq i\leq N}\sum_{j=1}^{N}|B(i,j)|.$

The following fact will also be used in the sequel: for each $r\in(1,\infty)$,
it holds that

$|||B|||_{r}\leq|||B|||^{1/r}_{1}|||B|||^{1-1/r}_{\infty}.$ (37)

Recall that $A^{N}=(A^{N}(i,j))_{1\leq i,j\leq N}$ is a random matrix such
that its $i$-th row, for each $1\leq i\leq N$, is given by

$A^{N}(i,j)=\begin{cases}N^{-1}\theta_{ij},\ \text{if}\ j\in\cal{P}_{+},\\\
-N^{-1}\theta_{ij},\ \text{if}\ j\in\cal{P}_{-}.\end{cases}$

One can check that $\max\\{|||A^{N}|||_{1},|||A^{N}|||_{\infty}\\}\leq 1$, so
that (37) implies that $|||A^{N}|||_{r}\leq 1$ for all $r\in(1,\infty)$ as
well. Recall that $\lambda>0$. Under this condition, the random matrix
$Q^{N}:=\left(I-(1-\lambda)A^{N}\right)^{-1}=\sum_{n=0}^{\infty}(1-\lambda)^{n}(A^{N})^{n}$
is well-defined and satisfies $|||Q^{N}|||_{r}\leq\lambda^{-1}$ for all
$r\in[1,\infty].$ For later use, let us observe that the easy-to-check
properties $|||(Q^{N})^{\intercal}|||_{1}=|||Q^{N}|||_{\infty}$ and
$|||(Q^{N})^{\intercal}|||_{\infty}=|||Q^{N}|||_{1}$ combined with inequality
(37) imply that $|||(Q^{N})^{\intercal}|||_{r}\leq\lambda^{-1}$ for any
$r\in[1,\infty].$

As suggest the heuristics presented in Section 2, a crucial step in our
analysis is to study both the sum of rows and columns of the matrix $Q^{N}$.
By the definition of the matrix $Q^{N}$ itself, these quantities are related
to the corresponding counterparts computed from the matrix $A^{N}$.

In what follows, for $a,b\in\\{-,+\\}$, we denote
$L^{N,ab}=1_{\mathcal{P}_{a}}\odot A^{N}1_{\mathcal{P}_{b}}$ and
$C^{N,ab}=1_{\mathcal{P}_{a}}\odot(A^{N})^{\intercal}1_{\mathcal{P}_{b}}$,
where $(A^{N})^{\intercal}$ denotes the transpose of the matrix $A^{N}$. On
the one hand, both vectors have null coordinates outside of the set
$\mathcal{P}_{a}$. On the other hand, each coordinate $i\in\mathcal{P}_{a}$ of
the random vector $L^{N,ab}$ (resp. $C^{N,ab}$) is obtained by summing the
entries in ${\cal{P}}_{b}$ of the $i$-th row (resp. column) of the matrix
$A^{N}$. Alternatively, the random vectors $L^{N,ab}$ and $C^{N,ab}$ can be
defined as follows:

$L^{N,ab}_{i}=\begin{cases}\sum_{j\in{\cal{P}}_{b}}A^{N}(i,j),\ \text{if}\
i\in{\cal P}_{a},\\\ 0,\ \text{otherwise,}\end{cases}$ (38)

and

$C^{N,ab}_{i}=\begin{cases}\sum_{j\in{\cal{P}}_{b}}A^{N}(j,i),\ \text{if}\
i\in{\cal P}_{a},\\\ 0,\ \text{otherwise. }\end{cases}$ (39)

For $a\in\\{-,+\\}$, we denote $L^{N,a\bullet}=1_{\mathcal{P}_{a}}\odot
A^{N}1_{N}$ and
$C^{N,a\bullet}=1_{\mathcal{P}_{a}}\odot(A^{N})^{\intercal}1_{N}$. Note that
the coordinates not belonging to $\mathcal{P}_{a}$ of these two vectors are
also $0$. Besides, each coordinate $i\in\mathcal{P}_{a}$ of the random vector
$L^{N,a\bullet}$ (resp. $C^{N,a\bullet}$) is given by the sum over all entries
of the $i$-th row (resp. column) of the matrix $A^{N}$. One can easily check
that

$L^{N,a\bullet}=L^{N,a+}+L^{N,a-}\ \text{and}\
C^{N,a\bullet}=C^{N,a+}+C^{N,a-},$ (40)

so that the vectors $L^{N,a\bullet}$ and $C^{N,a\bullet}$ could be defined
alternatively through these identities.

For $b\in\\{-,+\\}$, we denote $L^{N,\bullet b}=A^{N}1_{\mathcal{P}_{b}}$ and
$C^{N,\bullet b}=(A^{N})^{\intercal}1_{\mathcal{P}_{b}}$. Observe that each
coordinate $1\leq i\leq N$ of the random vector $L^{N,\bullet b}$ (resp.
$C^{N,\bullet b}$) is given by the sum over the entries in $\mathcal{P}_{b}$
of the $i$-th row (resp. column) of the matrix $A^{N}$. One can also verify
that

$L^{N,\bullet b}=L^{N,+b}+L^{N,-b}\ \text{and}\ C^{N,\bullet
b}=C^{N,+b}+C^{N,-b}.$ (41)

Similarly as above, for $a,b\in\\{-,+\\}$, let us denote $\ell^{N,ab}$ (resp.
$c^{N,ab}$) the random vector obtained by summing, for each row (resp. column)
in ${\cal{P}}_{a}$ of the random matrix $Q^{N}$, the entries in
${\cal{P}}_{b}$. Also, we denote $\ell^{N,a\bullet}=\ell^{N,a+}+\ell^{N,a-}$
and $\ell^{N,\bullet b}=\ell^{N,+b}+\ell^{N,-b}$ for $a,b\in\\{-,+\\}$, and
define $\ell^{N}=\ell^{N,+\bullet}+\ell^{N,-\bullet}$. Note that
$\ell^{N}=Q^{N}1_{N}$, i.e., $\ell^{N}$ corresponds to the random vector
obtained by summing the rows of $Q^{N}$. The vectors $c^{N,\bullet a}$,
$c^{N,\bullet a}$ and $c^{N}$ are defined in a similar way. In particular,
note that $c^{N}=(Q^{N})^{\intercal}1_{N}$ is given as the sum of the columns
of $Q^{N}$. Here $(Q^{N})^{\intercal}$ denotes the transpose of the matrix
$Q^{N}$. For later use, let us also observe that
$\overline{\ell^{N}}=\overline{c^{N}}$.

######  Lemma 4.1.

Assume that $1\geq\lambda>0$. Then,

$\displaystyle\max\left\\{\max_{a,b\in\\{-,+\\}}\left\\{\|\ell^{N,ab}\|_{\infty},\|c^{N,ab}\|_{\infty}\right\\},\|\ell^{N}\|_{\infty},\|c^{N}\|_{\infty}\right\\}\leq\lambda^{-1}.\
$ (42)

###### Proof.

To show (42), first observe that

$\max_{1\leq i\leq
N}\max\left\\{\max_{a,b\in\\{-,+\\}}\left\\{|\ell^{N,ab}_{i}|,|c^{N,ab}_{i}|\right\\},|\ell^{N}_{i}|,|c^{N}_{i}|\right\\}\leq\max\\{|||Q^{N}|||_{1},|||Q^{N}|||_{\infty}\\},$

and then use the fact that
$\max\\{|||Q^{N}|||_{1},|||Q^{N}|||_{\infty}\\}\leq\lambda^{-1}$. ∎

In the sequel, for each $a,b\in\\{-,+\\}$, let

$\widetilde{L}^{N,ab}=L^{N,ab}-(r^{N}_{a})^{-1}\overline{L^{N,ab}}1_{\mathcal{P}_{a}}\
\text{and}\
\widetilde{C}^{N,ab}=C^{N,ab}-(r^{N}_{a})^{-1}\overline{C^{N,ab}}1_{\mathcal{P}_{a}}.$
(43)

The vectors $\widetilde{L}^{N,ab}$ and $\widetilde{C}^{N,ab}$ can be thought
as the population-wise centered versions of the vectors $L^{N,ab}$ and
$C^{N,ab}$ respectively, in the sense that
$\overline{\widetilde{L}^{N,ab}}=\overline{\widetilde{C}^{N,ab}}=0.$ Observe
that these vectors are well-defined for all $N\geq 1$ such that $r^{N}_{a}>0$.
In the next result, we collect some bounds which will be used throughout the
section.

In what follows, for $a,b\in\\{-,+\\}$, we write $\delta_{ab}$ to denote the
Kronecker delta between $a$ and $b$.

######  Lemma 4.2.

Let $a,b,a_{1},b_{1}\in\\{-,+\\}$. The following inequalities hold for all
$N\geq 1$ such that $r^{N}_{-}\wedge r^{N}_{+}>0,$

$\displaystyle\mathbb{E}\left[\left|\left<\widetilde{L}^{N,ab},\widetilde{L}^{N,a_{1}b_{1}}\right>-\delta_{aa_{1}}\delta_{bb_{1}}r_{a}r_{b}p(1-p)\right|^{2}\right]\leq
KN^{-1},$ (44)
$\displaystyle\mathbb{E}\left[||A\widetilde{L}^{ab}||_{2}^{2}\right]\leq
KN^{-1},$ (45)
$I\\!\\!E\left[\max_{a,b\in\\{-,+\\}}\left\\{\left|(r^{N}_{a}r^{N}_{b})^{-1}\overline{L^{N,ab}}-(bp)\right|^{2}\vee\left|(r^{N}_{a}r^{N}_{b})^{-1}\overline{C^{N,ab}}-(ap)\right|^{2}\right\\}\right]\\\
\leq K\max_{a,b}\left\\{(r^{N}_{a}r^{N}_{b})^{-1}\right\\}N^{-2},\ \text{and}$
(46)
$\|\widetilde{L}^{N,ab}\|_{2}\leq||L^{N,ab}-(bpr^{N}_{b})1_{{\cal{P}}_{a}}||_{2}\
\text{and}\
\|\widetilde{C}^{N,ab}\|_{2}\leq||C^{N,ab}-(apr^{N}_{b})1_{{\cal{P}}_{a}}||_{2},$
(47)

where $K$ is some universal constant. Moreover, for each $\alpha\geq 1$, there
exists a constant $K_{\alpha}$ depending only on $\alpha$ such that, for all
$N\geq 1$ satisfying $r^{N}_{-}\wedge r^{N}_{+}>0$,

$I\\!\\!E\left[\max_{a,b\in\\{-,+\\}}\left\\{||L^{N,ab}-(bpr^{N}_{b})1_{{\cal{P}}_{a}}||^{\alpha}_{2}\vee||C^{N,ab}-(apr^{N}_{b})1_{{\cal{P}}_{a}}||^{\alpha}_{2}\right\\}\right]\\\
\leq
K_{\alpha}\max_{a\in\\{-,+\\}}\left\\{\left(r^{N}_{a}/r^{N}_{b}\right)^{\alpha/2}\right\\}.$
(48)

The proof of Lemma 4.2 is postponed to Appendix D.

######  Remark 4.3.

Let $a,b\in\\{-,+\\}$. Because of the relations (40) and (41), some statements
of Lemma 4.2 admit immediate corollaries. For instance,

  * •

$C^{N,\bullet b}=C^{N,+b}+C^{N,-b}$ and $\widetilde{C}^{N,\bullet
b}=\widetilde{C}^{N,+b}+\widetilde{C}^{N,-b}$ are orthogonal sums, and in
particular one can deduce from Equations (47) and (48) that

$I\\!\\!E\left[||\widetilde{C}^{N,\bullet b}||^{\alpha}_{2}\right]\leq
I\\!\\!E\left[||C^{N,\bullet
b}-pr^{N}_{b}(1_{{\cal{P}}_{+}}-1_{{\cal{P}}_{-}})||^{\alpha}_{2}\right]\leq
2K_{\alpha}\max_{a\in\\{-,+\\}}\left\\{\left(r^{N}_{a}/r^{N}_{b}\right)^{\alpha/2}\right\\},$

  * •

or, $L^{N,a\bullet}=L^{N,a+}+L^{N,a-}$ and
$\widetilde{L}^{N,a\bullet}=\widetilde{L}^{N,a+}+\widetilde{L}^{N,a-},$ and in
particular one can deduce from Equations (47) and (48) that

$\|\widetilde{L}^{N,a\bullet}\|_{2}\leq||L^{N,a+}-pr^{N}_{+}1_{{\cal{P}}_{a}}||_{2}+||L^{N,a-}+pr^{N}_{-}1_{{\cal{P}}_{a}}||_{2},$

and

$I\\!\\!E\left[||L^{N,a\bullet}-p(r^{N}_{+}-r^{N}_{-})1_{{\cal{P}}_{a}}||^{\alpha}_{2}\right]\leq
K^{\prime}_{\alpha}\max_{a\in\\{-,+\\}}\left\\{\left(r^{N}_{a}/r^{N}_{b}\right)^{\alpha/2}\right\\},$

for some $K^{\prime}_{\alpha}$ which may be different from $K_{\alpha}$.

Obviously, this remark also applies to the next lemma which states the same
kind of results as Lemma 4.2 but for the $\ell$ and $c$ vectors instead of the
$L$ and $C$ vectors.

Like we did for $L$ and $C$, we define, for each $a,b\in\\{-,+\\}$

$\widetilde{\ell}^{N,ab}=\ell^{N,ab}-(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}1_{\mathcal{P}_{a}}\
\text{and}\
\widetilde{c}^{N,ab}=c^{N,ab}-(r^{N}_{a})^{-1}\overline{c^{N,ab}}1_{\mathcal{P}_{a}}.$
(49)

Moreover, we denote
$\widetilde{\ell}^{N,a\bullet}=\widetilde{\ell}^{N,a+}+\widetilde{\ell}^{N,a-}$,
$\widetilde{\ell}^{N,\bullet
b}=\widetilde{\ell}^{N,+b}+\widetilde{\ell}^{N,-b}$ for $a,b\in\\{-,+\\}$, and
$\widetilde{\ell}^{N}=\widetilde{\ell}^{N,+\bullet}+\widetilde{\ell}^{N,-\bullet}$.

######  Lemma 4.4.

Assume that $1\geq\lambda>0$. There exists a constant $K>0$ which depends on
$\lambda$ such that for all $N\geq 1$ and $a,b\in\\{-,+\\}$, it holds that

$\displaystyle
I\\!\\!E\left[\Big{|}\overline{\ell^{N,ab}}-\frac{r_{a}\delta_{ab}+b(1-\lambda)pr_{+}r_{-}}{1-(1-\lambda)p(r_{+}-r_{-})}\Big{|}^{2}\right]\leq\frac{K}{N^{2}},$
(50) $\displaystyle
I\\!\\!E\left[\Big{|}\overline{c^{N,a\bullet}}-r_{a}\frac{\left[1+(2a)(1-\lambda)p(1-r_{a})\right]}{1-(1-\lambda)p(r_{+}-r_{-})}\Big{|}^{2}\right]\leq\frac{K}{N^{2}},$
(51) $\displaystyle I\\!\\!E\left[||\widetilde{\ell}^{N,\bullet
b}||_{2}^{4}\right]\leq K\text{ and }I\\!\\!E\left[||\widetilde{c}^{N,\bullet
b}||_{2}^{4}\right]\leq K,$ (52) $\displaystyle
I\\!\\!E\left[\left|||\widetilde{\ell}^{N,\bullet
b}||_{2}^{2}-\frac{(1-\lambda)^{2}p(1-p)\left[r_{b}+(1-\lambda)pr_{+}r_{-}(2b+(1-\lambda)p)\right]}{(1-(1-\lambda)p(r_{+}-r_{-}))^{2}}\right|\right]\leq\frac{K}{\sqrt{N}},$
(53) $\displaystyle
I\\!\\!E\left[\left|||\widetilde{\ell}^{N}||_{2}^{2}-\frac{(1-\lambda)^{2}p(1-p)}{(1-(1-\lambda)p(r_{+}-r_{-}))^{2}}\right|\right]\leq\frac{K}{\sqrt{N}}.$
(54)

######  Remark 4.5.

For later use, let us observe that an immediate consequence of Equation (50)
is that there exists a constant $K>0$ such that $N\geq 1$,

$I\\!\\!E\left[\left|\overline{\ell^{N}}-\frac{1}{1-(1-\lambda)p(r_{+}-r_{-})}\right|^{2}\right]\leq\frac{K}{N^{2}}.$
(55)

###### Proof of Lemma 4.4.

It suffices to show that these inequalities hold for all $N$ sufficiently
large. Recall that $r^{N}_{a}=|{\cal{P}}_{a}|/N$, for $a\in\\{-,+\\}$.
Throughout the proof, we assume that $N$ is large enough ($N\geq N_{0}$)
ensuring that $r^{N}_{+}\wedge r^{N}_{-}\geq r_{\rm min}>0$ for some $r_{\rm
min}$ sufficiently small depending only on the choice of $r_{+}$ and $r_{-}$.
In what follows, we shall denote $K$ a constant which may depend on $\lambda$
and which may change from one line to another. The proof is divided in several
steps.

Step 1. Consider the event

$\mathcal{A}_{N}=\bigcap_{a\in\\{-,+\\}}\bigcap_{b\in\\{-,+\\}}\left\\{||L^{N,ab}-(bpr^{N}_{b})1_{{\cal{P}}_{a}}||_{2}\vee||C^{N,ab}-(apr^{N}_{b})1_{{\cal{P}}_{a}}||_{2}\leq
N^{1/4}\right\\}.$

For later use, let us observe that Points (ii) and (iii) of Lemma 4.2 imply
that $\mathcal{A}_{N}$ is included in the event

$\mathcal{G}_{N}=\bigcap_{b\in\\{-,+\\}}\left\\{\|L^{N,\bullet
b}-(bpr^{N}_{b})1_{N}\|_{2}\vee\|L^{N,b\bullet}-p(r^{N}_{+}-r^{N}_{-})1_{\mathcal{P}_{b}}\|_{2}\right.\\\
\left.\vee||C^{N,\bullet
b}-pr^{N}_{b}(1_{{\cal{P}}_{+}}-1_{{\cal{P}}_{-}})||_{2}\vee||C^{N,b\bullet}-(bp)1_{{\cal{P}}_{b}}||_{2}\leq
2N^{1/4}\right\\}.$

Combining Point (iv) of Lemma 4.2 with Markov’s inequality, one can show that
for any $\alpha\geq 1$, there exits a constant $K_{\alpha}>0$ such that

$\mathbb{P}\left(\mathcal{G}_{N}\right)\geq\mathbb{P}\left(\mathcal{A}_{N}\right)\geq
1-K_{\alpha}N^{-\alpha/4}.$

Hence, $\mathcal{A}_{N}$ is a large probability event and we will first
consider the expectations appearing in Lemma 4.4 under the event
$\mathcal{A}_{N}$ only. The last step of the proof explains how to conclude.

Step 2. Let $a,b\in\\{-,+\\}$. Here, we prove that

$\text{(i)}\ \
\widetilde{\ell}^{N,ab}=(1-\lambda)\left[1_{\mathcal{P}_{a}}\odot
A^{N}\widetilde{\ell}^{N,\bullet
b}+\sum_{e\in\\{-,+\\}}(r^{N}_{e})^{-1}\overline{\ell^{N,eb}}\widetilde{L}^{N,ae}+\epsilon^{N,ab}1_{\mathcal{P}_{a}}\right],$

where

$\text{(ii)}\ \ |\epsilon^{N,ab}|\leq(r^{N}_{a}N)^{-1}||C^{N,\bullet
a}-pr_{a}(1_{\mathcal{P}_{+}}-1_{\mathcal{P}_{-}})||_{2}||\widetilde{\ell}^{N,\bullet
b}||_{2}.$

Starting from $\ell^{N,\bullet
b}=Q^{N}1_{\mathcal{P}_{b}}=(I_{N}-(1-\lambda)A^{N})^{-1}1_{\mathcal{P}_{b}}$,
we get $\ell^{N,\bullet b}=1_{\mathcal{P}_{b}}+(1-\lambda)A^{N}\ell^{N,\bullet
b}$. Hence, it implies that
$\ell^{N,ab}=1_{\mathcal{P}_{a}}\odot\left(1_{\mathcal{P}_{b}}+(1-\lambda)A^{N}\ell^{N,\bullet
b}\right)$ and
$(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}=\delta_{ab}+(1-\lambda)(r^{N}_{a}N)^{-1}\left<A^{N}\ell^{\bullet
b},1_{\mathcal{P}_{a}}\right>$, so that

$\widetilde{\ell}^{N,ab}=(1-\lambda)\left[1_{\mathcal{P}_{a}}\odot
A^{N}\ell^{N,\bullet b}-(r^{N}_{a}N)^{-1}\left<A^{N}\ell^{N,\bullet
b},1_{\mathcal{P}_{a}}\right>1_{\mathcal{P}_{a}}\right].$

Then, using the substitution $\ell^{N,\bullet b}=\widetilde{\ell}^{N,\bullet
b}+\sum_{e\in\\{-,+\\}}(r^{N}_{e})^{-1}\overline{\ell^{N,eb}}1_{\mathcal{P}_{e}}$
in the two terms above and the fact that $1_{\mathcal{P}_{a}}\odot
A^{N}1_{\mathcal{P}_{e}}=L^{N,ae}$, we get point (i) with

$\epsilon^{N,ab}=-(r^{N}_{a}N)^{-1}\left<A^{N}\widetilde{\ell}^{N,\bullet
b},1_{\mathcal{P}_{a}}\right>=-(r^{N}_{a}N)^{-1}\left<\widetilde{\ell}^{N,\bullet
b},(A^{N})^{\intercal}1_{\mathcal{P}_{a}}\right>.$

Yet, $(A^{N})^{\intercal}1_{\mathcal{P}_{a}}=C^{N,\bullet a}$ by definition
and $\left<\tilde{\ell}^{N,\bullet
b},1_{\mathcal{P}_{+}}\right>=\left<\tilde{\ell}^{N,\bullet
b},1_{\mathcal{P}_{-}}\right>=0$ by construction so that

$\epsilon^{N,ab}=-(r^{N}_{a}N)^{-1}\left<\widetilde{\ell}^{N,\bullet
b},C^{N,\bullet a}-pr_{a}(1_{\mathcal{P}_{+}}-1_{\mathcal{P}_{-}})\right>,$

and point (ii) follows from Cauchy-Schwarz inequality.

Step 3. Proceeding as in Step 2, one can show that

$\text{(i)}\ \ \widetilde{c}^{N,ab}=(1-\lambda)\left[1_{\mathcal{P}_{a}}\odot
A^{N}\widetilde{c}^{N,\bullet
b}+\sum_{e\in\\{-,+\\}}(r^{N}_{e})^{-1}\overline{c^{N,eb}}\widetilde{C}^{N,ae}+\gamma^{N,ab}1_{\mathcal{P}_{a}}\right],$

where

$\text{(ii)}\ \ |\gamma^{N,ab}|\leq(r^{N}_{a}N)^{-1}||L^{N,\bullet
a}-(apr_{a})1_{N}||_{2}||\widetilde{c}^{N,\bullet b}||_{2}.$

Step 4. Here, we prove Equation (52).

From step 2, one can sum for $a\in\\{-,+\\}$ to obtain

$\widetilde{\ell}^{N,\bullet
b}=(1-\lambda)\left[A^{N}\widetilde{\ell}^{N,\bullet
b}+\sum_{a\in\\{-,+\\}}\sum_{e\in\\{-,+\\}}(r^{N}_{e})^{-1}\overline{\ell^{N,eb}}\widetilde{L}^{N,ae}+\sum_{a\in\\{-,+\\}}\epsilon^{N,ab}1_{\mathcal{P}_{a}}\right].$
(56)

Then, using the fact that
$||1_{\mathcal{P}_{a}}||_{2}=N^{1/2}(r^{N}_{a})^{1/2}\leq N^{1/2}$, we get

$(1-\lambda)^{-1}||\widetilde{\ell}^{N,\bullet
b}||_{2}\leq|||A^{N}|||_{2}||\widetilde{\ell}^{N,\bullet b}||_{2}\\\
+4\max_{e\in\\{-,+\\}}\left\\{(r^{N}_{e})^{-1}|\overline{\ell^{N,eb}}|\right\\}\max_{a,e\in\\{-,+\\}}\left\\{\|\widetilde{L}^{N,ae}\|_{2}\right\\}+N^{1/2}\sum_{a\in\\{-,+\\}}|\epsilon^{N,ab}|.$

Recall that $|||A^{N}|||_{2}\leq 1$ and $|\overline{\ell^{N,eb}}|\leq
r^{N}_{e}\lambda^{-1}$ (see inequality (42)). On the event
$\mathcal{A}_{N}\supset\mathcal{G}_{N}$, we have $||C^{N,\bullet
a}-pr^{N}_{a}(1_{{\cal{P}}_{+}}-1_{{\cal{P}}_{-}})||_{2}\leq 2N^{1/4}$, and
using Point (ii) of Step 2, we have

${{\mathbf{1}}}_{\mathcal{A}_{N}}(1-\lambda)^{-1}||\widetilde{\ell}^{N,\bullet
b}||_{2}\leq\|\widetilde{\ell}^{N,\bullet
b}\|_{2}\left[1+\frac{4}{r^{N}_{+}\wedge
r^{N}_{-}}N^{-1/4}\right]+4\lambda^{-1}\max_{a,e\in\\{-,+\\}}\left\\{\|\widetilde{L}^{N,ae}\|_{2}\right\\}.$

Now, since $r^{N}_{+}\wedge r^{N}_{-}\geq r_{\rm min}$, it follows that
$1+\frac{4}{r^{N}_{+}\wedge r^{N}_{-}}N^{-1/4}\leq 1+4r_{\rm
min}^{-1}N^{-1/4}<(1-\lambda)^{-1}$ for $N\geq N_{1}$ for some $N_{1}$
sufficiently large. As a consequence, there exists a constant $K$ such that

${{\mathbf{1}}}_{\mathcal{A}_{N}}||\widetilde{\ell}^{N,\bullet b}||_{2}\leq
K\max_{a,e\in\\{-,+\\}}\left\\{\|\widetilde{L}^{N,ae}\|_{2}\right\\},$ (57)

and Equations (47) and (48) of Lemma 4.2 implies that

$I\\!\\!E\left[{{\mathbf{1}}}_{\mathcal{A}_{N}}||\widetilde{\ell}^{N,\bullet
b}||^{4}_{2}\right]\leq
KI\\!\\!E\left[\max_{a,b\in\\{-,+\\}}\left\\{\|\widetilde{L}^{N,ab}\|^{4}_{2}\right\\}\right]\leq
K,$

for all $N\geq N_{1}$. Taking the maximum value of
$I\\!\\!E\left[{{\mathbf{1}}}_{\mathcal{A}_{N}}||\tilde{\ell}^{N,\bullet
b}||^{4}_{2}\right]$ over all $N\in[N_{0},N_{1}]$, we get

$I\\!\\!E\left[{{\mathbf{1}}}_{\mathcal{A}_{N}}||\widetilde{\ell}^{N,\bullet
b}||^{4}_{2}\right]\leq K,$

for all $N\geq N_{0}$. Yet, inequality 42 implies that
$|\widetilde{\ell}^{N,ab}_{i}|\leq 2\lambda^{-1}$ which in turn implies that
$||\widetilde{\ell}^{N,\bullet b}||^{4}_{2}\leq 4\lambda^{-2}N^{2}$. Hence, to
get rid of the term ${{\mathbf{1}}}_{\mathcal{A}_{N}}$ it suffices to write

$I\\!\\!E\left[||\widetilde{\ell}^{N,\bullet b}||^{4}_{2}\right]\leq
4\lambda^{-2}N^{2}\mathbb{P}\left((\mathcal{A}_{N})^{c}\right)+I\\!\\!E\left[{{\mathbf{1}}}_{\mathcal{A}_{N}}||\widetilde{\ell}^{N,\bullet
b}||^{4}_{2}\right]\leq K,$

where we used the last inequality of Step 1 with $\alpha=8$.

Finally, the proof of the inequality $I\\!\\!E\left[||\widetilde{c}^{N,\bullet
b}||_{2}^{4}\right]\leq K$ follows the same line and is therefore omitted.

Step 5. Here, we prove Equation (50).

We have already used in Step 2 that
$\ell^{N,ab}=1_{\mathcal{P}_{a}}\odot\left(1_{\mathcal{P}_{b}}+(1-\lambda)A^{N}\ell^{N,\bullet
b}\right)$ Hence, we deduce that

$\displaystyle\overline{\ell^{N,ab}}$ $\displaystyle=$ $\displaystyle
r^{N}_{a}\delta_{ab}+(1-\lambda)N^{-1}\sum_{i\in\mathcal{P}_{a}}\sum_{j=1}^{N}A^{N}(i,j)\ell^{N,\bullet
b}_{j}$ $\displaystyle=$ $\displaystyle
r^{N}_{a}\delta_{ab}+(1-\lambda)N^{-1}\sum_{j=1}^{N}C_{j}^{N,\bullet
a}\ell_{j}^{N,\bullet b}$ $\displaystyle=$ $\displaystyle
r^{N}_{a}\delta_{ab}+(1-\lambda)pr^{N}_{a}\left(\overline{\ell^{N,+b}}-\overline{\ell^{N,-b}}\right)+\eta^{N,ab},$

where $\eta^{N,ab}=(1-\lambda)N^{-1}\left<C^{N,\bullet
a}-pr^{N}_{a}(1_{\mathcal{P}_{+}}-1_{\mathcal{P}_{-}}),\ell^{N,\bullet
b}\right>$. In particular, we have

$\begin{cases}\overline{\ell^{N,+b}}-\overline{\ell^{N,-b}}=\frac{(br^{N}_{b})}{1-(1-\lambda)p(r^{N}_{+}-r^{N}_{-})}+\eta^{N,+b}-\eta^{N,-b},\\\
\overline{\ell^{N,+b}}+\overline{\ell^{N,-b}}=\frac{r^{N}_{b}+2b(1-\lambda)pr^{N}_{+}r^{N}_{-}}{1-(1-\lambda)p(r^{N}_{+}-r^{N}_{-})}+(1-\lambda)p(\eta^{N,+b}-\eta^{N,-b})+\eta^{N,+b}+\eta^{N,-b},\end{cases}$

and so

$\overline{\ell^{N,ab}}=\frac{r^{N}_{a}\delta_{ab}+b(1-\lambda)pr^{N}_{+}r^{N}_{-}}{1-(1-\lambda)p(r^{N}_{+}-r^{N}_{-})}+\frac{(1-\lambda)p}{2}(\eta^{N,+b}-\eta^{N,-b})+\eta^{N,ab}.\\\
$

Since by our assumption, $|r^{N}_{+}-r_{+}|+|r^{N}_{-}-r_{-}|\leq KN^{-1}$, we
have

$\left|\frac{r^{N}_{a}\delta_{ab}+b(1-\lambda)pr^{N}_{+}r^{N}_{-}}{1-(1-\lambda)p(r^{N}_{+}-r^{N}_{-})}-\frac{r_{a}\delta_{ab}+b(1-\lambda)pr_{+}r_{-}}{1-(1-\lambda)p(r_{+}-r_{-})}\right|\leq
K(|r^{N}_{+}-r_{+}|+|r^{N}_{-}-r_{-}|)\leq KN^{-1},$

for some constant $K$ that may depend on $\lambda$, and the result will follow
once we check that $I\\!\\!E[(\eta^{N,ab})^{2}]\leq KN^{-2}$.

To this end, we write
$\eta^{N,ab}=(1-\lambda)N^{-1}(\eta^{N,ab}_{+}+\eta^{N,ab}_{-})$ where, for
$e\in\\{-,+\\}$,
$\eta^{N,ab}_{e}=\left<C^{N,ea}-(ep)r^{N}_{a}1_{\mathcal{P}_{e}},\ell^{N,eb}\right>$.
Remind that
$\ell^{N,eb}=\widetilde{\ell}^{N,eb}+(r_{e}^{N})^{-1}\overline{\ell^{N,eb}}1_{\mathcal{P}_{e}}$.
Then, Lemma D.4 can be applied with
$V^{N}=C^{N,ea}-(ep)r^{N}_{a}1_{\mathcal{P}_{e}}$, $v^{N}_{1}=0$,
$v_{2}^{N}=\widetilde{\ell}^{N,eb}$ and
$v_{3}^{N}=(r_{e}^{N})^{-1}\overline{\ell^{N,eb}}1_{\mathcal{P}_{e}}$:
assumption (i) is satisfied thanks to Equation (48) and the fact that
$\|C^{N,ea}\|_{\infty}\leq 1$, assumption (ii) is satisfied thanks to Equation
(52), assumption (iii) is satisfied thanks to Equations (42) and (46) because
$\left<V^{N},v_{3}^{N}\right>=(r_{e}^{N})^{-1}\overline{\ell^{N,eb}}\,\overline{V^{N}}$.

Hence, $I\\!\\!E[(\eta^{N,ab}_{e})^{2}]\leq K$ which in turn implies that
$I\\!\\!E[(\eta^{N,ab})^{2}]\leq KN^{-2}$.

Step 6. Here we prove Equation (51).

Starting from

$c^{N}=(Q^{N})^{\intercal}1_{N}=\left(I_{N}-(1-\lambda)\left(A^{N}\right)^{\intercal}\right)^{-1}1_{N},$

we deduce that
$c^{N,+\bullet}+c^{N,-\bullet}=c^{N}=1_{N}+(1-\lambda)\left(A^{N}\right)^{\intercal}c^{N}$,
so that

$\displaystyle\overline{c^{N,a\bullet}}$ $\displaystyle=$ $\displaystyle
r^{N}_{a}+(1-\lambda)N^{-1}\sum_{i\in\mathcal{P}_{a}}\sum_{j=1}^{N}A^{N}(j,i)c_{j}^{N}$
$\displaystyle=$ $\displaystyle
r^{N}_{a}+(1-\lambda)N^{-1}\sum_{j=1}^{N}L_{j}^{N,\bullet a}c_{j}^{N}$
$\displaystyle=$ $\displaystyle
r^{N}_{a}+(1-\lambda)(apr^{N}_{a})\overline{c^{N}}+\xi^{N,a},$

where $\xi^{N,a}=(1-\lambda)N^{-1}\left<L^{N,\bullet
a}-(apr^{N}_{a})1_{N},c^{N}\right>$. Since
$\overline{c^{N}}=\overline{\ell^{N}}$, to conclude the proof of this step it
suffices to use Equation (50) and to show that $I\\!\\!E[(\xi^{N,a})^{2}]\leq
KN^{-2}$.

Proceeding as before, we can write
$\xi^{N,a}=(1-\lambda)N^{-1}(\xi^{N,a}_{+}+\xi^{N,a}_{-})$ where, for
$e\in\\{-,+\\}$,
$\xi^{N,a}_{e}=\left<L^{N,ea}-(ap)r^{N}_{a}1_{\mathcal{P}_{e}},c^{N,e\bullet}\right>$.
Remind that
$c^{N,eb}=\widetilde{c}^{N,eb}+(r_{e}^{N})^{-1}\overline{c^{N,eb}}1_{\mathcal{P}_{e}}$.
Then, Lemma D.4 can be applied with
$V^{N}=L^{N,ea}-(ap)r^{N}_{a}1_{\mathcal{P}_{e}}$, $v^{N}_{1}=0$ ,
$v_{2}^{N}=\widetilde{c}^{N,e\bullet}$ and
$v_{3}^{N}=(r_{e}^{N})^{-1}\overline{c^{N,e\bullet}}1_{\mathcal{P}_{e}}$:
assumption (i) is satisfied thanks to Equation (48) and the fact that
$\|L^{N,ea}\|_{\infty}\leq 1$, assumption (ii) is satisfied thanks to Equation
(52), assumption (iii) is satisfied thanks to Equations (42) and (46) because
$\left<V^{N},v_{3}^{N}\right>=(r_{e}^{N})^{-1}\overline{c^{N,e\bullet}}\,\overline{V^{N}}$.

Hence, $I\\!\\!E[(\xi^{N,a}_{e})^{2}]\leq K$ which in turn implies that
$I\\!\\!E[(\xi^{N,a})^{2}]\leq KN^{-2}$.

Step 7. Here, we prove that $I\\!\\!E\left[B^{N,\bullet b}\right]\leq
KN^{-1/2}$ where

$B^{N,\bullet b}=\left|||\widetilde{\ell}^{N,\bullet
b}||_{2}^{2}-||(1-\lambda)\sum_{a\in\\{-,+\\}}(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}\widetilde{L}^{N,\bullet
a}||_{2}^{2}\right|.$

Denoting $y^{N,\bullet b}=\widetilde{\ell}^{N,\bullet
b}-(1-\lambda)\sum_{a\in\\{-,+\\}}(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}\widetilde{L}^{N,\bullet
a}$, one can use Equation (56) to get

$y^{N,\bullet b}=(1-\lambda)\left[A^{N}y^{N,\bullet
b}+(1-\lambda)\sum_{a\in\\{-,+\\}}(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}(A^{N}\widetilde{L}^{N,\bullet
a})+\sum_{a\in\\{-,+\\}}\epsilon^{N,ab}1_{\mathcal{P}_{a}}\right].$

Using the same kind of arguments as in the beginning of Step 4 (except the use
of the event $\mathcal{A}_{N}$), we have

$((1-\lambda)^{-1}-1)\|y^{N,\bullet b}\|_{2}\leq
4(1-\lambda)\lambda^{-1}\max_{a,e\in\\{-,+\\}}\left\\{\|A^{N}\widetilde{L}^{N,ae}\|_{2}\right\\}\\\
+\frac{2}{r^{N}_{+}\wedge r^{N}_{-}}N^{-1/2}||\widetilde{\ell}^{N,\bullet
b}||_{2}\max_{a\in\\{-,+\\}}||C^{N,\bullet
a}-pr_{a}(1_{\mathcal{P}_{+}}-1_{\mathcal{P}_{-}})||_{2}.$

Furthermore, using Cauchy-Schwarz inequality, Step 4 of the current Lemma and
Equation (48) of Lemma 4.2, we have

$I\\!\\!E\left[||\widetilde{\ell}^{N,\bullet
b}||_{2}^{2}\max_{a\in\\{-,+\\}}||C^{N,\bullet
a}-pr_{a}(1_{\mathcal{P}_{+}}-1_{\mathcal{P}_{-}})||_{2}^{2}\right]\leq K.$

Combining the two equations above by convexity of the square function and then
Equation (45) of Lemma 4.2, we prove that $y^{N,\bullet b}$ is negligible in
the sense that

$I\\!\\!E\left[\|y^{N,\bullet b}\|_{2}^{2}\right]\leq
K\left\\{I\\!\\!E\left[\max_{a,e\in\\{-,+\\}}\left\\{\|A^{N}\widetilde{L}^{N,ae}\|_{2}^{2}\right\\}\right]+N^{-1}\right\\}\leq\frac{K}{N}.$

In turn, we can prove that $B^{N,\bullet b}$ is negligible. More precisely, we
factorize

$\displaystyle B^{N,\bullet b}$ $\displaystyle=$
$\displaystyle\left|||\widetilde{\ell}^{N,\bullet
b}||_{2}-||(1-\lambda)\sum_{a\in\\{-,+\\}}(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}\widetilde{L}^{N,\bullet
a}||_{2}\right|$ $\displaystyle\times\left(||\widetilde{\ell}^{N,\bullet
b}||_{2}+||(1-\lambda)\sum_{a\in\\{-,+\\}}(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}\widetilde{L}^{N,\bullet
a}||_{2}\right)$ $\displaystyle\leq$ $\displaystyle||y^{N,\bullet
b}||_{2}\left(||\widetilde{\ell}^{N,\bullet
b}||_{2}+K\max_{a\in\\{-,+\\}}||\widetilde{L}^{N,\bullet a}||_{2}\right),$

so that, by Cauchy-Schwarz inequality, and then using Step 4 of the current
Lemma and Equations (47) and (48) of Lemma 4.2, we get

$I\\!\\!E\left[B^{N,\bullet b}\right]\leq K\sqrt{I\\!\\!E\left[\|y^{N,\bullet
b}\|_{2}^{2}\right]}\sqrt{I\\!\\!E\left[||\widetilde{\ell}^{N,\bullet
b}||_{2}^{2}+\max_{a\in\\{-,+\\}}||\widetilde{L}^{N,\bullet
a}||_{2}^{2}\right]}\leq\frac{K}{\sqrt{N}}.$

Step 8. Using the same arguments as Step 7, one can prove that

$I\\!\\!E\left[\left|||\widetilde{\ell}^{N}||_{2}^{2}-||(1-\lambda)\sum_{a\in\\{-,+\\}}(r^{N}_{a})^{-1}\overline{\ell^{N,a\bullet}}\widetilde{L}^{N,\bullet
a}||_{2}^{2}\right|\right]\leq\frac{K}{\sqrt{N}}.$

Step 9. Here we prove, for all $a_{1},a_{2},b\in\\{-,+\\}$, that

$I\\!\\!E\left[\left|\frac{\overline{\ell^{N,a_{1}b}}}{r^{N}_{a_{1}}}\frac{\overline{\ell^{N,a_{2}b}}}{r^{N}_{a_{2}}}\left<\widetilde{L}^{N,\bullet
a_{1}},\widetilde{L}^{N,\bullet
a_{2}}\right>-\delta_{a_{1}a_{2}}\frac{\overline{\ell^{\infty,a_{1}b}}}{r_{a_{1}}}\frac{\overline{\ell^{\infty,a_{2}b}}}{r_{a_{2}}}r_{a_{1}}p(1-p)\right|\right]\leq\frac{K}{\sqrt{N}},$

where
$\overline{\ell^{\infty,ab}}:=\frac{r_{a}\delta_{ab}+b(1-\lambda)pr_{+}r_{-}}{1-(1-\lambda)p(r_{+}-r_{-})}$
is the limit appearing in Equation (50).

Since the coordinates of $\ell^{N,ab}$ are bounded by inequality (42), we know
that there exists a constant $K$ such that

$\left|\frac{\overline{\ell^{N,a_{1}b}}}{r^{N}_{a_{1}}}\frac{\overline{\ell^{N,a_{2}b}}}{r^{N}_{a_{2}}}-\frac{\overline{\ell^{\infty,a_{1}b}}}{r_{a_{1}}}\frac{\overline{\ell^{\infty,a_{2}b}}}{r_{a_{2}}}\right|\leq
K\left(\left|\frac{\overline{\ell^{N,a_{1}b}}}{r^{N}_{a_{1}}}-\frac{\overline{\ell^{\infty,a_{1}b}}}{r_{a_{1}}}\right|+\left|\frac{\overline{\ell^{N,a_{2}b}}}{r^{N}_{a_{2}}}-\frac{\overline{\ell^{\infty,a_{2}b}}}{r_{a_{2}}}\right|\right).$

Hence, using Equation (50), Cauchy-Schwarz inequality twice and finally
Equations (47) and (48) of Lemma 4.2, we have

$I\\!\\!E\left[\left|\frac{\overline{\ell^{N,a_{1}b}}}{r^{N}_{a_{1}}}\frac{\overline{\ell^{N,a_{2}b}}}{r^{N}_{a_{2}}}-\frac{\overline{\ell^{\infty,a_{1}b}}}{r_{a_{1}}}\frac{\overline{\ell^{\infty,a_{2}b}}}{r_{a_{2}}}\right|\left|\left<\widetilde{L}^{N,\bullet
a_{1}},\widetilde{L}^{N,\bullet a_{2}}\right>\right|\right]\\\
\leq\frac{K}{N}I\\!\\!E\left[\left<\widetilde{L}^{N,\bullet
a_{1}},\widetilde{L}^{N,\bullet
a_{2}}\right>^{2}\right]^{1/2}\leq\frac{K}{N}\max_{a\in\\{-,+\\}}I\\!\\!E\left[||\widetilde{L}^{N,\bullet
a}||_{2}^{2}\right]^{1/2}\leq\frac{K}{N}.$

Finally, we conclude the step by combining the above equation with Equation
(44) of Lemma 4.2.

Step 10. Using the same arguments as in Step 9, one can prove that, for all
$a_{1},a_{2}\in\\{-,+\\}$,

$I\\!\\!E\left[\left|\frac{\overline{\ell^{N,a_{1}\bullet}}}{r^{N}_{a_{1}}}\frac{\overline{\ell^{N,a_{2}\bullet}}}{r^{N}_{a_{2}}}\left<\widetilde{L}^{N,\bullet
a_{1}},\widetilde{L}^{N,\bullet
a_{2}}\right>-\delta_{a_{1}a_{2}}\frac{\overline{\ell^{\infty,a_{1}\bullet}}}{r_{a_{1}}}\frac{\overline{\ell^{\infty,a_{2}\bullet}}}{r_{a_{2}}}r_{a_{1}}p(1-p)\right|\right]\leq\frac{K}{\sqrt{N}},$

where
$\overline{\ell^{\infty,a\bullet}}:=\overline{\ell^{\infty,a+}}+\overline{\ell^{\infty,a-}}=\frac{r_{a}}{1-(1-\lambda)p(r_{+}-r_{-})}$.

Step 11. Here we prove Equation (53). According to Step 7, the limit of
$||\widetilde{\ell}^{N,\bullet b}||_{2}^{2}$ is related to the limit of
$||(1-\lambda)\sum_{a\in\\{-,+\\}}(r^{N}_{a})^{-1}\overline{\ell^{N,ab}}\widetilde{L}^{N,\bullet
a}||_{2}^{2}$ which can be expanded as

$(1-\lambda)^{2}\sum_{a_{1},a_{2}\in\\{-,+\\}}\frac{\overline{\ell^{N,a_{1}b}}}{r^{N}_{a_{1}}}\frac{\overline{\ell^{N,a_{2}b}}}{r^{N}_{a_{2}}}\left<\widetilde{L}^{N,\bullet
a_{1}},\widetilde{L}^{N,\bullet a_{2}}\right>.$

Hence, combining Steps 7 and 9, we have

$I\\!\\!E\left[\left|||\widetilde{\ell}^{N,\bullet
b}||_{2}^{2}-(1-\lambda)^{2}p(1-p)\sum_{a\in\\{-,+\\}}\left(\frac{\overline{\ell^{\infty,ab}}}{r_{a}}\right)^{2}\,r_{a}\right|\right]\leq\frac{K}{\sqrt{N}}.$

In order to conclude this step, it suffices to simplify

$\displaystyle\sum_{a\in\\{-,+\\}}\left(\frac{\overline{\ell^{\infty,ab}}}{r_{a}}\right)^{2}\,r_{a}$
$\displaystyle=$
$\displaystyle\sum_{a\in\\{-,+\\}}\left(\frac{\delta_{ab}+b(1-\lambda)p(1-r_{a})}{1-(1-\lambda)p(r_{+}-r_{-})}\right)^{2}\,r_{a}$
$\displaystyle=$
$\displaystyle\frac{\left(1+b(1-\lambda)p(1-r_{b})\right)^{2}\,r_{b}+\left(b(1-\lambda)pr_{b}\right)^{2}\,(1-r_{b})}{(1-(1-\lambda)p(r_{+}-r_{-}))^{2}}$
$\displaystyle=$
$\displaystyle\frac{r_{b}+2b(1-\lambda)pr_{b}(1-r_{b})+(1-\lambda)^{2}p^{2}r_{b}(1-r_{b})}{(1-(1-\lambda)p(r_{+}-r_{-}))^{2}}$
$\displaystyle=$
$\displaystyle\frac{r_{b}+(1-\lambda)pr_{+}r_{-}(2b+(1-\lambda)p)}{(1-(1-\lambda)p(r_{+}-r_{-}))^{2}}.$

Step 12. Following the lines of Step 11 (and using Steps 8 and 10), one can
prove Equation (54). Let us mention that the final simplification here is

$\displaystyle\sum_{a\in\\{-,+\\}}\left(\frac{\overline{\ell^{\infty,a\bullet}}}{r_{a}}\right)^{2}\,r_{a}$
$\displaystyle=$
$\displaystyle\sum_{a\in\\{-,+\\}}\left(\frac{1}{1-(1-\lambda)p(r_{+}-r_{-})}\right)^{2}\,r_{a}$
$\displaystyle=$ $\displaystyle(1-(1-\lambda)p(r_{+}-r_{-}))^{-2}.$

∎

The following lemma gives a way to control the $l^{2}$ norms of the fully
centered vectors by the population wise centered vectors.

######  Lemma 4.6.

Assume that $1\geq\lambda>0$. There exists a constant $K>0$ which depends on
$\lambda$ such that for all $N\geq 1$ such that $r^{N}_{+}\wedge r^{N}_{-}>0$,
it holds that

$\displaystyle
I\\!\\!E\left[\left|\left\|\ell^{N}-\overline{\ell^{N}}1_{N}\right\|_{2}^{2}-\left\|\widetilde{\ell}^{N}\right\|_{2}^{2}\right|\right]\leq\frac{K}{N},$
(58) $\displaystyle
I\\!\\!E\left[\left|\langle\ell^{N}-\overline{\ell^{N}}1_{N},(1-\lambda)^{2}(Q^{N}L^{N,\bullet-}-\overline{Q^{N}L^{N,\bullet-}})\rangle-\langle\widetilde{\ell}^{N},\widetilde{\ell}^{N,\bullet-}\rangle\right|\right]\leq
KN^{-1},$ (59) $\displaystyle
I\\!\\!E\left[\left|(1-\lambda)\|QL^{N,\bullet}-\overline{Q^{N}L^{N,\bullet-}}\|^{2}_{2}-\|\widetilde{\ell}^{N,\bullet-}\|^{2}_{2}\right|\right]\leq
KN^{-1}.$ (60)

###### Proof.

It suffices to show that these inequalities hold for all $N$ sufficiently
large. Recall that $r^{N}_{a}=|{\cal{P}}_{a}|/N$, for $a\in\\{-,+\\}$. As in
the previous proofs, in what follows we assume that $N$ is large enough
($N\geq N_{0}$) ensuring that $r^{N}_{+}\wedge r^{N}_{-}\geq r_{\rm min}>0$
for some $r_{\rm min}$ sufficiently small depending only on the choice of
$r_{+}$ and $r_{-}$. Also, we shall denote $K$ a constant which may depend on
$\lambda$ and which may change from one line to another. The proof is divided
in several steps.

We will first prove (58). To see that, we start by observing that

$\displaystyle\|\ell^{N}-\overline{\ell^{N}}1_{N}\|^{2}_{2}-\|\widetilde{\ell}^{N}\|^{2}_{2}$
$\displaystyle=\sum_{a\in\\{-,+\\}}\sum_{i\in\mathcal{P}_{a}}(\ell^{N,a\bullet}_{i}-\overline{\ell^{N}})^{2}-(\ell^{N,a\bullet}_{i}-(r^{N}_{a})^{-1}\overline{\ell^{N,a\bullet}})^{2}$
$\displaystyle=\sum_{a\in\\{-,+\\}}((r^{N}_{a})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}})\sum_{i\in\mathcal{P}_{a}}(\ell^{N,a\bullet}_{i}-\overline{\ell^{N}}),$
$\displaystyle=N\sum_{a\in\\{-,+\\}}r^{N}_{a}\left((r^{N}_{a})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}}\right)^{2},$

where in the second equality we have used that $x^{2}-y^{2}=(x-y)(x+y)$ for
all $x,y\in\mathbb{R}$ and
$\sum_{i\in\mathcal{P}_{a}}(\ell^{N,a\bullet}_{i}-(r^{N}_{a})^{-1}\overline{\ell^{N,a\bullet}})=0$.
Then, we combine (50), (55) and (42) together with Jensen inequality to obtain
that

$NI\\!\\!E\left[\left((r^{N}_{a})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}}\right)^{2}\right]\leq
KN\left(I\\!\\!E\left[\left((r_{a})^{-1}\overline{\ell^{N,a\bullet}}-\frac{1}{1-(1-\lambda)p(r_{+}-r_{-})}\right)^{2}\right]\right.\\\
\left.+I\\!\\!E\left[|r^{N}_{a}-r_{a}|^{2}\right]+I\\!\\!E\left[\left(\overline{\ell^{N}}-\frac{1}{1-(1-\lambda)p(r_{+}-r_{-})}\right)^{2}\right]\right)\leq
KN^{-1},$

so that

$I\\!\\!E\left[\left|\|\ell^{N}-\overline{\ell^{N}}1_{N}\|^{2}_{2}-\|\widetilde{\ell}^{N}\|^{2}_{2}\right|\right]\leq
KN^{-1}\sum_{a\in\\{-,+\\}}r^{N}_{a}=KN^{-1},$

which proves (58).

We will now prove (59). To that end, first note that

$(1-\lambda)(Q^{N}L^{N,\bullet-}-\overline{Q^{N}L^{N,\bullet-}})=\ell^{N,\bullet-}-\overline{\ell^{N,\bullet-}}1_{N}+r^{N}_{-}1_{\mathcal{P}_{+}}-r^{N}_{+}1_{\mathcal{P}_{-}}.$
(61)

Then, note that

$\displaystyle\langle\ell^{N}-\overline{\ell^{N}}1_{N},r^{N}_{-}1_{\mathcal{P}_{+}}-r^{N}_{+}1_{\mathcal{P}_{-}}\rangle$
$\displaystyle=r^{N}_{-}\langle\ell^{N,+\bullet}-\overline{\ell^{N}}1_{\mathcal{P}_{+}},1_{\mathcal{P}_{+}}\rangle-r^{N}_{+}\langle\ell^{N,-\bullet}-\overline{\ell^{N}}1_{\mathcal{P}_{-}},1_{\mathcal{P}_{-}}\rangle$
$\displaystyle=Nr^{N}_{-}\left[\overline{\ell^{N,+\bullet}}-\overline{\ell^{N}}r^{N}_{+}\right]-Nr^{N}_{+}\left[\overline{\ell^{N,-\bullet}}-\overline{\ell^{N}}r^{N}_{-}\right].$

Next, by using that
$\ell^{N}-\overline{\ell^{N}}1_{N}=\widetilde{\ell}^{N}+\sum_{a\in\\{-,+\\}}1_{\mathcal{P}_{a}}((r^{N}_{a})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}})$
and that
$\ell^{N,\bullet-}-\overline{\ell^{N,\bullet-}}1_{N}=\widetilde{\ell}^{N,\bullet-}+\sum_{a\in\\{-,+\\}}1_{\mathcal{P}_{a}}((r^{N}_{a})^{-1}\overline{\ell^{N,a-}}-\overline{\ell^{N,\bullet-}})$,
one can check that

$\langle\ell^{N}-\overline{\ell^{N}}1_{N},\ell^{N,\bullet-}-\overline{\ell^{N,\bullet-}}1_{N}\rangle=\langle\widetilde{\ell}^{N},\widetilde{\ell}^{N,\bullet-}\rangle\\\
+N\sum_{a\in\\{-,+\\}}r^{N}_{a}\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}}\right]\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a-}}-\overline{\ell^{N,\bullet-}}\right].$

Now, observe that using the fact that

$\frac{\delta_{a-}-(1-\lambda)pr_{+}r_{-}(r_{a})^{-1}}{1-(1-\lambda)p(r_{+}-r_{-})}+ar_{-a}=\frac{r_{-}-2(1-\lambda)pr_{-}r_{+}}{1-(1-\lambda)p(r_{+}-r_{-})},$

we can write

$\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}}\right]\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a-}}-\overline{\ell^{N,\bullet-}}\right]=-ar_{-a}\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}}\right]+\xi^{N}_{a},$

where

$\xi^{N}_{a}=\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}}\right]\left[\left((r^{N}_{a})^{-1}\overline{\ell^{N,a-}}-\frac{(\delta_{a-}-(1-\lambda)pr_{+}r_{-}r^{-1}_{-a})}{1-(1-\lambda)p(r_{+}-r_{-})}\right)\right.\\\
\left.+\left(\frac{(r_{-}-2(1-\lambda)pr_{+}r_{-})}{1-(1-\lambda)p(r_{+}-r_{-})}\right)\right].$

Combining (50) and (55) with Jensen inequality, one can show that
$I\\!\\!E\left[|\xi^{N}_{a}|\right]\leq KN^{-2}$, so that

$N\sum_{a\in\\{-,+\\}}r^{N}_{a}\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}}-\overline{\ell^{N}}\right]\left[(r_{a}^{N})^{-1}\overline{\ell^{N,a-}}-\overline{\ell^{N,\bullet-}}\right]=-Nr_{-}\left[\overline{\ell^{N,+\bullet}}-\overline{\ell^{N}}r_{+}^{N}\right]\\\
+Nr_{+}\left[\overline{\ell^{N,-\bullet}}-\overline{\ell^{N}}r_{-}^{N}\right]+\xi^{N}$

where $\xi^{N}=N\sum_{a\in\\{-,+\\}}r^{N}_{a}\xi^{N}_{a}$ satisfies
$I\\!\\!E\left[|\xi^{N}|\right]\leq KN^{-1}.$ Therefore, putting together all
previous identities, we deduce that

$\langle\ell^{N}-\overline{\ell^{N}}1_{N},(1-\lambda)(Q^{N}L^{N,\bullet-}-\overline{Q^{N}L^{N,\bullet-}})\rangle-\langle\widetilde{\ell}^{N},\widetilde{\ell}^{N,\bullet-}\rangle=N(r^{N}_{-}-r_{-})\left[\overline{\ell^{N,+\bullet}}-\overline{\ell^{N}}r^{N}_{+}\right]\\\
N(r_{+}-r^{N}_{+})\left[\overline{\ell^{N,-\bullet}}-\overline{\ell^{N}}r^{N}_{-}\right]+\xi^{N},$

and the result follows from (50), (55) and the assumption that
$|r^{N}_{a}-r_{a}|\leq KN^{-1}$.

Hence, it remains to prove only (60). Starting from (61), one can check that
(recall the definition of $\widetilde{\ell}^{N,ab}$ given in (49))

$(1-\lambda)(Q^{N}L^{N,\bullet-}-\overline{Q^{N}L^{N,\bullet-}})=\widetilde{\ell}^{N,\bullet-}+\sum_{a\in\\{-,+\\}}1_{\mathcal{P}_{a}}\left[(r^{N}_{a})^{-1}\overline{\ell^{N,a-}}-\overline{\ell^{N,\bullet-}}+ar^{N}_{-a}\right],$

which together with the fact that $\langle
1_{\mathcal{P}_{+}},1_{\mathcal{P}_{-}}\rangle=\langle
1_{\mathcal{P}_{a}},\widetilde{\ell}^{N,\bullet-}\rangle=0$ implies that

$\displaystyle(1-\lambda)^{2}\|(Q^{N}L^{N,\bullet-}-\overline{Q^{N}L^{N,\bullet-}})\|^{2}_{2}-\widetilde{\ell}^{N,\bullet-}=\sum_{a\in\\{-,+\\}}|\mathcal{P}_{a}|\left[(r^{N}_{a})^{-1}\overline{\ell^{N,a-}}-\overline{\ell^{N,\bullet-}}+ar^{N}_{-a}\right]^{2}.$

Proceeding similarly as in the proof of (59), one can show that

$\sum_{a\in\\{-,+\\}}|\mathcal{P}_{a}|I\\!\\!E\left[\left[(r^{N}_{a})^{-1}\overline{\ell^{N,a-}}-\overline{\ell^{N,\bullet-}}+ar^{N}_{-a}\right]^{2}\right]\leq
KN^{-1},$

concluding the proof of the lemma.

∎

We are now able to present the main result of this section.

######  Proposition 4.7.

Assume that $1\geq\lambda>0$. There exists a constant $K>0$ which depends on
$\lambda$ such that for all $N\geq 1$, it holds that

$\displaystyle
I\\!\\!E\left[\left|\overline{m^{N}}-m\right|^{2}\right]\leq\frac{K}{N^{2}},$
(62) $\displaystyle
I\\!\\!E\left[\left|\left\|m^{N}-\overline{m^{N}}1_{N}\right\|_{2}^{2}-v\right|\right]\leq\frac{K}{\sqrt{N}},$
(63) $\displaystyle
I\\!\\!E\left[\left|\frac{1}{N}\sum_{i=1}^{N}\left(c^{N}_{i}\right)^{2}\left(m^{N}_{i}-\left(m^{N}_{i}\right)^{2}\right)-w\right|^{2}\right]\leq\frac{K}{N^{2}},$
(64)

where $m^{N}$ is defined in (13) and $m,v,w$ are defined in (6).

###### Proof.

First, remind that
$\overline{m^{N}}=\mu\overline{\ell^{N}}-(1-\lambda)\overline{Q^{N}L^{N,\bullet-}}$.

Step 1. Here we prove Equation (62). Thanks to Remark 4.5, it suffices to
prove that

$I\\!\\!E\left[\left|\overline{Q^{N}L^{N,\bullet-}}+\frac{pr_{-}}{1-(1-\lambda)p(r_{+}-r_{-})}\right|^{2}\right]\leq\frac{K}{N^{2}}.$

First, note that

$Q^{N}L^{N,\bullet-}=-pr^{N}_{-}\ell^{N}+Q^{N}(L^{N,\bullet-}+pr^{N}_{-}1_{N}).$

Hence, it follows that

$\displaystyle\overline{Q^{N}L^{N,\bullet-}}$ $\displaystyle=$ $\displaystyle-
pr^{N}_{-}\overline{\ell^{N}}+N^{-1}\sum_{i=1}^{N}\sum_{j=1}^{N}Q^{N}(i,j)(L_{j}^{N,\bullet-}+pr^{N}_{-})$
$\displaystyle=$ $\displaystyle-
pr^{N}_{-}\overline{\ell^{N}}+N^{-1}\sum_{j=1}^{N}(L_{j}^{N,\bullet-}+pr^{N}_{-})c^{N}_{j}$
$\displaystyle=$ $\displaystyle-
pr^{N}_{-}\overline{\ell^{N}}+(1-\lambda)^{-1}\xi^{N,-},$

where
$\xi^{N,-}=(1-\lambda)N^{-1}\left<L^{N,\bullet-}+pr^{N}_{-}1_{N},c^{N}\right>$
is already defined in Step 6 of the proof of Lemma 4.4. From there, we know
that $I\\!\\!E\left[|\xi^{N,-}|^{2}\right]\leq KN^{-2}$. As a consequence, it
follows that

$I\\!\\!E\left[\Big{|}\overline{Q^{N}L^{N,\bullet-}}+\frac{pr_{-}}{1-(1-\lambda)p(r_{+}-r_{-})}\Big{|}^{2}\right]\leq\\\
K\left(I\\!\\!E\left[\Big{|}\overline{\ell^{N}}-\frac{1}{1-(1-\lambda)p(r_{+}-r_{-})}\Big{|}^{2}\right]+N^{-2}\right)\leq
KN^{-2},$

where in the last inequality we used Equation (55).

Step 2. Here we prove Equation (63). Expanding the scalar product
$\left\|m^{N}-\overline{m^{N}}1_{N}\right\|_{2}^{2}$ and using Lemma 4.6, we
have

$I\\!\\!E\left[\left|\left\|m^{N}-\overline{m^{N}}1_{N}\right\|_{2}^{2}-\left(\mu^{2}\|\widetilde{\ell}^{N}\|_{2}^{2}-2\mu\left<\widetilde{\ell}^{N},\widetilde{\ell}^{N,\bullet-}\right>+\|\widetilde{\ell}^{N,\bullet-}\|_{2}^{2}\right)\right|\right]\leq\frac{K}{N}.$

Yet, by the polarization identity,
$-2\left<\widetilde{\ell}^{N},\widetilde{\ell}^{N,\bullet-}\right>=\left(\|\widetilde{\ell}^{N,\bullet+}\|_{2}^{2}-\|\widetilde{\ell}^{N}\|_{2}^{2}-\|\widetilde{\ell}^{N,\bullet-}\|_{2}^{2}\right)$,
so that

$I\\!\\!E\left[\left|\left\|m^{N}-\overline{m^{N}}1_{N}\right\|_{2}^{2}-\left((\mu^{2}-\mu)\|\widetilde{\ell}^{N}\|_{2}^{2}+\mu\|\widetilde{\ell}^{N,\bullet+}\|_{2}^{2}+(1-\mu)\|\widetilde{\ell}^{N,\bullet-}\|_{2}^{2}\right)\right|\right]\leq\frac{K}{N},$

and we conclude this step thanks to Equations (53) and (54) as soon as we
check the simplification:

$\displaystyle I$ $\displaystyle=$
$\displaystyle(\mu^{2}-\mu)+\mu\left[r_{+}+(1-\lambda)pr_{+}r_{-}(2+(1-\lambda)p)\right]$
$\displaystyle+(1-\mu)\left[r_{-}+(1-\lambda)pr_{+}r_{-}(-2+(1-\lambda)p)\right]$
$\displaystyle=$ $\displaystyle\mu^{2}+\mu(-1+r_{+}-r_{-})+r_{-}$
$\displaystyle+(1-\lambda)pr_{+}r_{-}[\mu(2+(1-\lambda)p+2-(1-\lambda)p)-2+(1-\lambda)p]$
$\displaystyle=$ $\displaystyle\mu^{2}-2\mu
r_{-}+r_{-}+(1-\lambda)pr_{+}r_{-}(4\mu-2+(1-\lambda)p)$ $\displaystyle=$
$\displaystyle(\mu+(1-\lambda)pr_{-})^{2}+r_{-}(1-(1-\lambda)p(r_{+}-r_{-}))(1-2\mu-(1-\lambda)p)$

and the identity

$\frac{1-2\mu-(1-\lambda)p}{1-(1-\lambda)p(r_{+}-r_{-})}=1-2m.$

Step 3. Here we prove that

$I\\!\\!E\left[\left|\frac{1}{N}\sum_{i=1}^{N}\ell^{N}_{i}(c^{N}_{i})^{2}-\frac{1+4(1-\lambda)^{2}p^{2}r_{-}r_{+}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{3}}\right|^{2}\right]\leq\frac{K}{N^{2}}.$
(65)

Notice that
$\sum_{i=1}^{N}\ell_{i}^{N}(c_{i}^{N})^{2}=\sum_{a\in\\{-,+\\}}\left<\ell^{N,a\bullet},(c^{N,a\bullet})^{2}\right>$
and
$(c^{N,a\bullet})^{2}=(\widetilde{c}^{N,a\bullet})^{2}+2(r^{N}_{a})^{-1}\overline{c^{N,a\bullet}}\widetilde{c}^{N,a\bullet}+((r^{N}_{a})^{-1}\overline{c^{N,a\bullet}})^{2}1_{\mathcal{P}_{a}}$
for each $a\in\\{-,+\\}$. We argue that Lemma D.5 can be applied to

$\begin{cases}u_{1}^{N}=0,\,u_{2}^{N}=\widetilde{\ell}^{N,a\bullet},\,u_{3}^{N}=(r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}},\\\
u_{3}^{\infty}=(1-(1-\lambda)p(r_{+}-r_{-}))^{-1},\\\
v_{1}^{N}=(\widetilde{c}^{N,a\bullet})^{2},\,v_{2}^{N}=(r_{a}^{N})^{-1}\overline{c^{N,a\bullet}}\widetilde{c}^{N,a\bullet},\,v_{3}^{N}=((r_{a}^{N})^{-1}\overline{c^{N,a\bullet}})^{2},\\\
v_{3}^{\infty}=\left(\frac{\left[1+(2a)(1-\lambda)p(1-r_{a})\right]}{1-(1-\lambda)p(r_{+}-r_{-})}\right)^{2}.\end{cases}$

Indeed, the assumptions are satisfied thanks to Equations (42), (50), (52),
the fact that $\widetilde{\ell}^{N,a\bullet}$ and $\widetilde{c}^{N,a\bullet}$
are centered (for instance $\overline{\widetilde{\ell}^{N,a\bullet}}=0$) and
the following argument: using the fact that $\overline{c^{N,a\bullet}}\leq 1$
and the previous step, we have

$I\\!\\!E\left[\left|v_{3}^{N}-v_{3}^{\infty}\right|^{2}\right]\leq
KI\\!\\!E\left[\left|\overline{c^{N,a\bullet}}-r_{a}\frac{\left[1+(2a)(1-\lambda)p(1-r_{a})\right]}{1-(1-\lambda)p(r_{+}-r_{-})}\right|^{2}\right]\leq\frac{K}{N^{2}}.$

The conclusion of Lemma D.5 is

$I\\!\\!E\left[\left|\frac{1}{N}\left<\ell^{N,a\bullet},(c^{N,a\bullet})^{2}\right>-r_{a}\frac{\left[1+(2a)(1-\lambda)p(1-r_{a})\right]^{2}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{3}}\right|^{2}\right]\leq\frac{K}{N^{2}}.$

Finally, the conclusion of this step follows from the fact that

$\sum_{a\in\\{-,+\\}}r_{a}\left[1+(2a)(1-\lambda)p(1-r_{a})\right]^{2}=1+4(1-\lambda)^{2}p^{2}r_{-}r_{+}.$
(66)

Step 4. Here we prove that

$I\\!\\!E\left[\left|\frac{1}{N}\sum_{i=1}^{N}(\ell^{N}_{i}c^{N}_{i})^{2}-\frac{1+4(1-\lambda)^{2}p^{2}r_{-}r_{+}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{4}}\right|^{2}\right]\leq\frac{K}{N^{2}}.$
(67)

Notice that
$\sum_{i=1}^{N}(\ell_{i}^{N}c_{i}^{N})^{2}=\sum_{a\in\\{-,+\\}}\left<(\ell^{N,a\bullet})^{2},(c^{N,a\bullet})^{2}\right>$.
We argue that Lemma D.5 can be applied to

$\begin{cases}u_{1}^{N}=(\widetilde{\ell}^{N,a\bullet})^{2},\,u_{2}^{N}=(r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}}\widetilde{\ell}^{N,a\bullet},\,u_{3}^{N}=((r_{a}^{N})^{-1}\overline{\ell^{N,a\bullet}})^{2},\\\
u_{3}^{\infty}=(1-(1-\lambda)p(r_{+}-r_{-}))^{-2},\\\
v_{1}^{N}=(\widetilde{c}^{N,a\bullet})^{2},\,v_{2}^{N}=(r_{a}^{N})^{-1}\overline{c^{N,a\bullet}}\widetilde{c}^{N,a\bullet},\,v_{3}^{N}=((r_{a}^{N})^{-1}\overline{c^{N,a\bullet}})^{2},\\\
v_{3}^{\infty}=\left(\frac{\left[1+(2a)(1-\lambda)p(1-r_{a})\right]}{1-(1-\lambda)p(r_{+}-r_{-})}\right)^{2}.\end{cases}$

Indeed, the assumptions are satisfied thanks to Equations (42), (50), (52),
the fact that $\widetilde{\ell}^{N,a\bullet}$ and $\widetilde{c}^{N,a\bullet}$
are centered (for instance $\overline{\widetilde{\ell}^{N,a\bullet}}=0$) and
the fact that
$I\\!\\!E\left[\left|u_{3}^{N}-u_{3}^{\infty}\right|^{2}\right]\leq KN^{-2}$
can be proved like we did in the previous step for $v_{3}^{N}$

The conclusion of Lemma D.5 is

$I\\!\\!E\left[\left|\frac{1}{N}\left<(\ell^{N,a\bullet})^{2},(c^{N,a\bullet})^{2}\right>-r_{a}\frac{\left[1+(2a)(1-\lambda)p(1-r_{a})\right]^{2}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{4}}\right|^{2}\right]\leq\frac{K}{N^{2}}.$

Finally, the conclusion of this step follows one again from Equation (66).

Step 5. In this step, we show the following result:

$N^{-1}\langle(c^{N})^{2},m^{N}\rangle=(\mu+(1-\lambda)pr^{N}_{-})N^{-1}\langle(c^{N})^{2},{\ell}^{N}\rangle+\xi^{N,(2),-},$
(68)

where $I\\!\\!E\left[(\xi^{N,(2),-})^{2}\right]\leq KN^{-2}$ for some constant
$K>0$.

On the one hand, recall that
$m^{N}=\mu\ell^{N}-(1-\lambda)Q^{N}L^{N,\bullet-}=(\mu+(1-\lambda)pr^{N}_{-})\ell^{N}-(1-\lambda)Q^{N}\left(L^{N,\bullet-}+pr^{N}_{-}1_{N}\right)$.
On the other hand, remark that
$(c^{N})^{2}=(c^{N,+\bullet})^{2}+(c^{N,-\bullet})^{2}$. Hence, (68) is
satisfied with
$\xi^{N,(2),-}=-(1-\lambda)N^{-1}(\xi^{N,(2),-}_{+}+\xi^{N,(2),-}_{-})$ where
for each $e\in\\{-,+\\}$,

$\xi^{N,(2),-}_{e}=\langle(Q^{N})[L^{N,\bullet-}+pr^{N}_{-}1_{N}],(c^{N,e\bullet})^{2}\rangle=\langle
L^{N,\bullet-}+pr^{N}_{-}1_{N},(Q^{N})^{\intercal}(c^{N,e\bullet})^{2}\rangle.$

Therefore, it remains to prove that
$I\\!\\!E\left[(\xi^{N,(2),-}_{e})^{2}\right]\leq K$ for some constant $K>0$.
Remind that
$(c^{N,e\bullet})^{2}=(\widetilde{c}^{N,e\bullet})^{2}+2(r^{N}_{e})^{-1}\overline{c^{N,e\bullet}}\widetilde{c}^{N,e\bullet}+((r^{N}_{e})^{-1}\overline{c^{N,e\bullet}})^{2}1_{\mathcal{P}_{e}}$
for each $e\in\\{-,+\\}$. Then, Lemma D.4 can be applied with
$V^{N}=L^{N,\bullet-}+pr^{N}_{-}1_{N}$,
$v^{N}_{1}=(Q^{N})^{\intercal}(\widetilde{c}^{N,e\bullet})^{2}$,
$v_{2}^{N}=2(r_{e}^{N})^{-1}\overline{c^{N,e\bullet}}(Q^{N})^{\intercal}\widetilde{c}^{N,e\bullet}$
and
$v_{3}^{N}=((r_{e}^{N})^{-1}\overline{c^{N,e\bullet}})^{2}(Q^{N})^{\intercal}1_{\mathcal{P}_{e}}=((r_{e}^{N})^{-1}\overline{c^{N,e\bullet}})^{2}c^{N,e\bullet}$.
Assumption (i) is satisfied thanks to Equation (48) and the fact that
$\|L^{N,\bullet-}\|_{\infty}\leq 1$, assumption (ii) is satisfied thanks to
Equations (52), the facts that
$\|(\widetilde{c}^{N,e\bullet})^{2}\|_{1}=\|\widetilde{c}^{N,e\bullet}\|_{2}^{2}$
and
$\max\\{|||(Q^{N})^{\intercal}|||_{2},|||(Q^{N})^{\intercal}|||_{1}\\}\leq\lambda^{-1}$,
assumption (iii) is satisfied because
$\left<V^{N},v_{3}^{N}\right>=((r_{e}^{N})^{-1}\overline{c^{N,e\bullet}})^{2}\left<L^{N,\bullet-}+pr^{N}_{-}1_{N},c^{N,e\bullet}\right>=((r_{e}^{N})^{-1}\overline{c^{N,e\bullet}})^{2}\xi^{N,-}_{e}$
which is defined and controlled in Step 7.

Hence, $I\\!\\!E[(\xi^{N,(2),-}_{e})^{2}]\leq K$ which in turn implies that
$I\\!\\!E\left[(\xi^{N,(2),-})^{2}\right]\leq KN^{-2}$.

Step 6. In this step, we show the following result:

$N^{-1}\langle(c^{N})^{2},(m^{N})^{2}\rangle=(\mu+(1-\lambda)pr^{N}_{-})^{2}N^{-1}\langle(c^{N})^{2},({\ell}^{N})^{2}\rangle+\xi^{N,(3),-}+\xi^{N,(2,2),-},$
(69)

where $\xi^{N,(3),-}$ and $\xi^{N,(2,2),-}$ satisfy
$I\\!\\!E\left[(\xi^{N,(3),-})^{2}+(\xi^{N,(2,2),-})^{2}\right]\leq KN^{-2}$
for some constant $K>0$.

On the one hand, from the identity
$m^{N}=(\mu+(1-\lambda)pr^{N}_{-})\ell^{N}-(1-\lambda)Q^{N}(L^{N,\bullet-}+pr^{N}_{-}1_{N})$,
one can check that

$(m^{N})^{2}=(\mu+(1-\lambda)pr^{N}_{-})^{2}(\ell^{N})^{2}\\\
-2(\mu+(1-\lambda)pr^{N}_{-})(1-\lambda)\ell^{N}\odot
Q^{N}(L^{N,\bullet-}+pr^{N}_{-}1_{N})+(1-\lambda)^{2}(Q^{N}(L^{N,\bullet-}+pr^{N}_{-}1_{N}))^{2}.$

On the other hand, remind that
$(c^{N})^{2}=(c^{N,+\bullet})^{2}+(c^{N,-\bullet})^{2}$ and
$\ell^{N}=\ell^{N,+\bullet}+\ell^{N,-\bullet}$. Hence, Equation (69) is
satisfied with
$\xi^{N,(3),-}=-2(\mu+(1-\lambda)pr^{N}_{-})(1-\lambda)N^{-1}(\xi^{N,(3),-}_{+}+\xi^{N,(3),-}_{-})$
where, for each $e\in\\{-,+\\}$,

$\xi^{N,(3),-}_{e}=\left<(Q^{N})^{\intercal}[(c^{N,e\bullet})^{2}\odot\ell^{N,e\bullet}],L^{N,\bullet-}+pr^{N}_{-}1_{N}\right>,$

and
$\xi^{N,(2,2),-}=(1-\lambda)^{2}N^{-1}(\xi^{N,(2,2),-}_{+}+\xi^{N,(2,2),-}_{-})$
where

$\xi^{N,(2,2),-}_{e}=\left<(Q^{N})^{\intercal}[(c^{N,e\bullet})^{2}\odot(Q^{N}(L^{N,\bullet-}+pr^{N}_{-}1_{N}))],L^{N,\bullet-}+pr^{N}_{-}1_{N}\right>.$

Let us first check that $I\\!\\!E\left[(\xi^{N,(3),-}_{e})^{2}\right]\leq K$.

Remind that
$\ell^{N,e\bullet}=\widetilde{\ell}^{N,e\bullet}+(r^{N}_{e})^{-1}\overline{\ell^{N,e\bullet}}1_{\mathcal{P}_{e}}$
for each $e\in\\{-,+\\}$. Then, Lemma D.4 can be applied with
$V^{N}=L^{N,\bullet-}+pr^{N}_{-}1_{N}$, $v^{N}_{1}=0$,
$v_{2}^{N}=(Q^{N})^{\intercal}[(c^{N,e\bullet})^{2}\odot\widetilde{\ell}^{N,e\bullet}]$
and
$v_{3}^{N}=(r_{e}^{N})^{-1}\overline{\ell^{N,e\bullet}}(Q^{N})^{\intercal}(c^{N,e\bullet})^{2}$.
Assumption (i) is satisfied thanks to Equation (48) and the fact that
$\|L^{N,\bullet-}\|_{\infty}\leq 1$, assumption (ii) is satisfied thanks to
Equations (42), (52), the facts that
$|||(Q^{N})^{\intercal}|||_{2}\leq\lambda^{-1}$ and
$\|(c^{N,e\bullet})^{2}\odot\widetilde{\ell}^{N,e\bullet}\|_{2}\leq\|c^{N,e\bullet}\|_{\infty}^{2}\|\widetilde{\ell}^{N,e\bullet}\|_{2}$,
assumption (iii) is satisfied because
$\left<V^{N},v_{3}^{N}\right>=(r_{e}^{N})^{-1}\overline{\ell^{N,e\bullet}}\left<L^{N,\bullet-}+pr^{N}_{-}1_{N},(Q^{N})^{\intercal}(c^{N,e\bullet})^{2}\right>=(r_{e}^{N})^{-1}\overline{\ell^{N,e\bullet}}\xi^{N,(2),-}_{e}$
which is defined and controlled in Step 11. Hence,
$I\\!\\!E[(\xi^{N,(3),-}_{e})^{2}]\leq K$.

Now, let us check that $I\\!\\!E\left[(\xi^{N,(2,2),-}_{e})^{2}\right]\leq K$.

Remind that
$(c^{N,e\bullet})^{2}=(\widetilde{c}^{N,e\bullet})^{2}+2(r^{N}_{e})^{-1}\overline{c^{N,e\bullet}}\widetilde{c}^{N,e\bullet}+((r^{N}_{e})^{-1}\overline{c^{N,e\bullet}})^{2}1_{\mathcal{P}_{e}}$
for each $e\in\\{-,+\\}$. Then, Lemma D.4 can be applied with
$V^{N}=L^{N,\bullet-}+pr^{N}_{-}1_{N}$,

$v^{N}_{1}=(Q^{N})^{\intercal}[(\widetilde{c}^{N,e\bullet})^{2}\odot(Q^{N}V^{N})],$
$v_{2}^{N}=2(r^{N}_{e})^{-1}\overline{c^{N,e\bullet}}(Q^{N})^{\intercal}[\widetilde{c}^{N,e\bullet}\odot(Q^{N}V^{N})],$

and

$v_{3}^{N}=((r^{N}_{e})^{-1}\overline{c^{N,e\bullet}})^{2}(Q^{N})^{\intercal}[1_{\mathcal{P}_{e}}\odot(Q^{N}V^{N})].$

Assumption (i) is satisfied thanks to Equation (48) and the fact that
$\|L^{N,\bullet-}\|_{\infty}\leq 1$, assumption (ii) is satisfied thanks to
Equations (42), (52), the facts that
$\|(\widetilde{c}^{N,e\bullet})^{2}\|_{1}=\|\widetilde{c}^{N,e\bullet}\|_{2}^{2}$,
$\max\\{|||(Q^{N})^{\intercal}|||_{2},|||(Q^{N})^{\intercal}|||_{1},|||Q^{N}|||_{\infty}\\}\leq\lambda^{-1}$
and $\|V^{N}\|_{\infty}\leq 2$, assumption (iii) is satisfied thanks to
Equation (48) because

$\displaystyle\left|\left<V^{N},v_{3}^{N}\right>\right|$ $\displaystyle=$
$\displaystyle((r^{N}_{e})^{-1}\overline{c^{N,e\bullet}})^{2}\left|\left<V^{N},(Q^{N})^{\intercal}[1_{\mathcal{P}_{e}}\odot(Q^{N}V^{N})]\right>\right|$
$\displaystyle\leq$
$\displaystyle((r^{N}_{e})^{-1}\overline{c^{N,e\bullet}})^{2}\left\|V^{N}\right\|_{2}|||(Q^{N})^{\intercal}|||_{2}\left\|1_{\mathcal{P}_{e}}\odot(Q^{N}V^{N})\right\|_{2}$
$\displaystyle\leq$
$\displaystyle((r^{N}_{e})^{-1}\overline{c^{N,e\bullet}})^{2}\left\|V^{N}\right\|_{2}|||(Q^{N})^{\intercal}|||_{2}\left\|Q^{N}V^{N}\right\|_{2}$
$\displaystyle\leq$
$\displaystyle((r^{N}_{e})^{-1}\overline{c^{N,e\bullet}})^{2}|||(Q^{N})^{\intercal}|||_{2}\,|||Q^{N}|||_{2}\left\|V^{N}\right\|_{2}^{2}.$

Hence, $I\\!\\!E\left[(\xi^{N,(2,2),-}_{e})^{2}\right]\leq K$ which ends the
step.

Step 7. Here we prove Equation (64).

First remark that
$N^{-1}\sum_{i=1}^{N}\left(c^{N}_{i}\right)^{2}\left(m^{N}_{i}-\left(m^{N}_{i}\right)^{2}\right)=N^{-1}\left<(c^{N})^{2},m^{N}-(m^{N})^{2}\right>$.
Then, combining Equations (68) and (65), we know that
$N^{-1}\left<(c^{N})^{2},m^{N}\right>$ converges to (remind the definition of
$m$ in (6))

$(\mu+(1-\lambda)pr_{-})\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{3}}=m\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{2}}.$

Similarly, combining Equations (69) and (67), we know that
$N^{-1}\left<(c^{N})^{2},(m^{N})^{2}\right>$ converges to

$(\mu+(1-\lambda)pr_{-})^{2}\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{4}}=m^{2}\frac{1+4(1-\lambda)^{2}p^{2}r_{+}r_{-}}{(1-(1-\lambda)p(r_{+}-r_{-}))^{2}}.$

Finally, one ends up with the definition of $w$ by summing these two limits,
which ends this step. ∎

##  5 Convergence of the spatio-temporal mean

The aim of this section is to prove the convergence of the estimator
$\hat{m}$. The main result is Theorem 5.2. Let us denote

$m_{\infty}^{N}=\overline{m^{N}}=\mu\overline{\ell^{N}}-(1-\lambda)\overline{Q^{N}L^{N,\bullet-}},$
(70)

where we use Equation (13) for the last equality. The next result shows that
$m_{\infty}^{N}$ is the limit of $\hat{m}$ when $T\to\infty$ while $\theta$ is
fixed.

######  Proposition 5.1.

There exists a constant $K$ depending only on $\lambda$ such that for all
$T\geq 1$, $N\geq 1$,

$\operatorname{\mathop{Var}}_{\theta}\left(\widehat{m}\right)\leq K(TN)^{-1},$
(71)

and

$I\\!\\!E_{\theta}[\widehat{m}]=m_{\infty}^{N}.$

###### Proof.

Since we start from the stationary distribution, we have

$I\\!\\!E_{\theta}[\widehat{m}]=T^{-1}N^{-1}\sum_{t=1}^{T}\sum_{i=1}^{N}I\\!\\!E_{\theta}[X_{i,t}]=N^{-1}\sum_{i=1}^{N}m^{N}_{i}=\overline{m^{N}}.$

Then, note that $\widehat{m}=(TN)^{-1}\sum_{t=1}^{T}\sum_{i=1}^{N}X_{i,t}$ so
that

$\displaystyle\operatorname{\mathop{Var}}_{\theta}(\widehat{m})$
$\displaystyle\leq(TN)^{-2}\sum_{t_{1},t_{2}=1}^{T}\sum_{i_{1},i_{2}=1}^{N}|\operatorname{\mathop{Cov}}_{\theta}(X_{i_{1},t_{1}},X_{i_{2},t_{2}})|$
$\displaystyle\leq(TN)^{-1}\left[\frac{2K}{\lambda}+K(1-\lambda)+1\right],$

where the last inequality comes from Lemma 3.8.

∎

We now are in condition to prove the convergence of the spatio-temporal mean.

######  Theorem 5.2.

There exists a constant $K$ depending only on $\lambda$ such that for all
$T\geq 1$, $N\geq 1$ and $\varepsilon>0$,

$I\\!\\!P\left(\Big{|}\widehat{m}-\frac{\mu+(1-\lambda)pr_{-}}{1-p(1-\lambda)(r_{+}-r_{-})}\Big{|}\geq\varepsilon\right)\leq\frac{K}{\varepsilon}\left(\frac{1}{\sqrt{TN}}+\frac{1}{N}\right).$
(72)

###### Proof.

By the triangle inequality we have that

$I\\!\\!E\left[\Big{|}\widehat{m}-\frac{\mu+(1-\lambda)pr_{-}}{1-p(1-\lambda)(r_{+}-r_{-})}\Big{|}\right]\leq
I\\!\\!E\left[\Big{|}\widehat{m}-(\mu\overline{\ell^{N}_{\theta}}+(1-\lambda)\overline{Q^{N}L^{N,\bullet-}})\Big{|}\right]\\\
+\mu
I\\!\\!E\left[\Big{|}\overline{\ell^{N}}-\frac{1}{1-p(1-\lambda)(r_{+}-r_{-})}\Big{|}\right]\\\
+(1-\lambda)I\\!\\!E\left[\Big{|}\overline{Q^{N}L^{N,\bullet-}}-\frac{pr_{-}}{1-p(1-\lambda)(r_{+}-r_{-})}\Big{|}\right].$

Combining Jensen’s inequality with Propositions 4.7 and 5.1, the result
follows. ∎

##  6 Convergence of the temporal variance

The aim of this section is to prove the convergence of the estimator
$\hat{w}$. The main result is Theorem 6.11.

Let us denote

$w^{N}_{\infty}=\frac{1}{N}\sum_{k=1}^{N}(c^{N}_{k})^{2}\left(m^{N}_{k}-(m^{N}_{k})^{2}\right),$
(73)

and recall that

$\widehat{w}=2\mathcal{W}_{2\Delta}-\mathcal{W}_{\Delta},\ \text{with}\
\mathcal{W}_{\Delta}=\frac{N}{T}\sum_{k=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{k\Delta}-\overline{Z}_{(k-1)\Delta}-\Delta\hat{m}\right)^{2}.$

As it appears in the analysis below, $w^{N}_{\infty}$ is the limit of
$\mathcal{W}_{\Delta}$ when $T,\Delta\to\infty$. Hence, the choice
$\widehat{w}=\mathcal{W}_{\Delta}$ would give a consistent estimator. However,
the two estimators $\mathcal{W}_{2\Delta}$ and $\mathcal{W}_{\Delta}$ share a
common bias which can be eliminated by considering
$\widehat{w}=2\mathcal{W}_{2\Delta}-\mathcal{W}_{\Delta}$, which in turn
drastically improves the rate of convergence. This common bias is related with
the quantity $S^{N}_{\theta,t}$ defined in Lemma 6.2.

Note that by the triangle inequality, the following inequality holds:

$\Big{|}\widehat{w}-w^{N}_{\infty}\Big{|}\leq
2D^{N,1}_{2\Delta,T}+D^{N,1}_{\Delta,T}+2D^{N,2}_{2\Delta,T}+D^{N,2}_{\Delta,T}+D^{N,3}_{T},$
(74)

where

$D^{N,1}_{\Delta,T}=\frac{N}{T}\left|\sum_{s=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}-\Delta\widehat{m}\right)^{2}\right.\\\
\left.-\sum_{s=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}-I\\!\\!E_{\theta}\left[\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}\right]\right)^{2}\right|,$
(75) $D^{N,2}_{T}=\frac{N}{T}\left|\sum_{s=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}-I\\!\\!E_{\theta}\left[\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}\right]\right)^{2}\right.\\\
\left.-I\\!\\!E_{\theta}\left[\sum_{s=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}-I\\!\\!E_{\theta}\left[\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}\right]\right)^{2}\right]\right|,$
(76)

and

$D^{N,3}_{T}=\left|\frac{2N}{T}I\\!\\!E_{\theta}\left[\sum_{s=1}^{\lfloor
T/2\Delta\rfloor}\left(\overline{Z}_{2s\Delta}-\overline{Z}_{2(s-1)\Delta}-I\\!\\!E_{\theta}\left[\overline{Z}_{2s\Delta}-\overline{Z}_{2(s-1)\Delta}\right]\right)^{2}\right]\right.\\\
\left.-\frac{N}{T}I\\!\\!E_{\theta}\left[\sum_{s=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}-I\\!\\!E_{\theta}\left[\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}\right]\right)^{2}\right]-w^{N}_{\infty}\right|.$
(77)

######  Proposition 6.1.

There exists a constant $K$ depending only on $\lambda$ such that for all
$N\geq 1$, $T\geq 2$ and $1\leq\Delta\leq\lfloor T/2\rfloor$, we have that

$I\\!\\!E_{\theta}\left[D^{N,1}_{2\Delta,T}+D^{N,1}_{\Delta,T}\right]\leq
K\frac{\Delta}{T}.$

###### Proof.

Using that $\sum_{s=1}^{\lfloor
T/\Delta\rfloor}\left(\overline{Z}_{s\Delta}-\overline{Z}_{(s-1)\Delta}\right)=\lfloor
T/\Delta\rfloor\Delta\widehat{m}$ and
$I\\!\\!E_{\theta}[\overline{Z}_{\Delta}]=\Delta
I\\!\\!E_{\theta}\left[\widehat{m}\right]$, one can check that

$D^{N,1}_{\Delta,T}=\frac{N}{T}\Delta^{2}\left\lfloor\frac{T}{\Delta}\right\rfloor\Big{|}\widehat{m}-I\\!\\!E_{\theta}\left[\widehat{m}\right]\Big{|}^{2}\leq
N\Delta\Big{|}\widehat{m}-I\\!\\!E_{\theta}\left[\widehat{m}\right]\Big{|}^{2},$

which implies that $I\\!\\!E_{\theta}\left[D^{N,1}_{\Delta,T}\right]\leq
N\Delta\operatorname{\mathop{Var}}_{\theta}(\widehat{m})$. Replacing $\Delta$
by $2\Delta$ in the previous identity, we obtain
$I\\!\\!E_{\theta}\left[D^{N,1}_{2\Delta,T}\right]\leq
2N\Delta\operatorname{\mathop{Var}}_{\theta}(\widehat{m})$. Hence, the result
follows from Proposition 5.1. ∎

To deal with the other two terms, we need to obtain a fine estimate on
$\operatorname{\mathop{Var}}_{\theta}\left(\overline{Z}_{\Delta}\right)=I\\!\\!E_{\theta}\left[\left(\overline{U}_{\Delta}\right)^{2}\right]$
(recall (21)). This can be done as follows. Recall that for all $t\geq 1$ and
$1\leq i\leq N$,

$M_{i,t}=\sum_{s=1}^{t}(X_{i,s}-p_{\theta,i}(X_{s-1})),\text{ and
}M_{t}=(M_{1,t},\ldots,M_{N,t}).$

Let us denote $\mathcal{F}_{t}=\sigma(X_{0},\ldots,X_{t})$. From the fact that
the process $(X_{t})_{t\in\mathbb{Z}}$ is Markovian, it follows that
$(M_{t})_{t\geq 1}$ is a martingale with respect to the filtration
$(\mathcal{F}_{t})_{t\geq 1}$.

Recall that $Q^{N}=(I_{N}-(1-\lambda)A^{N})^{-1}.$ Then (22) implies

$U_{t}=Q^{N}M_{t}+(Q^{N}-I_{N})(X_{0}-X_{t}),\ t\geq 1.$ (78)

######  Lemma 6.2.

There exists a constant $K$ depending only on $\lambda$ such that for all
$t\geq 1$ and $N\geq 1$,

$I\\!\\!E_{\theta}\left[\left(\overline{U}_{t}\right)^{2}\right]=I\\!\\!E_{\theta}\left[\left(\overline{Q^{N}M_{t}}\right)^{2}\right]+S^{N}_{\theta,t}+r^{N}_{\theta,t},$

where $|r^{N}_{\theta,t}|\leq K(1-\lambda)^{t}N^{-1}$ and $S^{N}_{\theta,t}$
has the following properties: $|S^{N}_{\theta,t}|\leq KN^{-1}$ and
$|S^{N}_{\theta,2t}-S^{N}_{\theta,t}|\leq K(1-\lambda)^{t}N^{-1}$.

###### Proof.

Starting from Equation (78), one can check that for any $t\geq 1$,