# Mean-field limits for non-linear Hawkes processes with excitation and
inhibition

by P. Pfaffelhuber, S. Rotter and J. Stiefel
_Albert-Ludwigs University Freiburg_

###### Abstract

We study a multivariate, non-linear Hawkes process $Z^{N}$ on the complete
graph with $N$ nodes. Each vertex is either excitatory (probability $p$) or
inhibitory (probability $1-p$). We take the mean-field limit of $Z^{N}$,
leading to a multivariate point process $\bar{Z}$. If $p\neq\tfrac{1}{2}$, we
rescale the interaction intensity by $N$ and find that the limit intensity
process solves a deterministic convolution equation and all components of
$\bar{Z}$ are independent. In the critical case, $p=\tfrac{1}{2}$, we rescale
by $N^{1/2}$ and obtain a limit intensity, which solves a stochastic
convolution equation and all components of $\bar{Z}$ are conditionally
independent.

†† _AMS 2000 subject classification._ 60G55 (Primary) , 60F05 (Secondary).††
_Keywords and phrases._ Multivariate Hawkes process; Volterra equation; spike
train

##  1 Introduction

In [19, 21], Hawkes processes were introduced as self-excitatory point
processes. Today, they are used in various fields of applications including
seismology [33, 17], interactions in social networks [41, 31], finance [14,
20] and neuroscience [18, 27]. In the classical univariate, linear Hawkes
process, the firing rate at time $t$ is a linear function of
$I_{t}:=\sum_{i}\varphi(t-T_{i})$, where the $T_{i}$’s are previous jump
times. In this case, since rates cannot become negative, $\varphi\geq 0$ is
required, leading to a self-excitatory process.

Our main motivation to study Hawkes processes comes from the neurosciences. In
a graph, vertices model neurons, whereas the (directed) edges are synapses
linking the neurons. A point process indexed by the vertices models action
potentials or spike trains of electrical impulses. Communication via synapses
leads to correlated point processes such that each spike in one neuron
influences the rate by which a neighboring vertex fires. In this field, it is
known that neurons cannot only excite others, but inhibition is another
important factor (see e.g. [30]). The main goal of this paper is to obtain
limit results for the multivariante, non-linear Hawkes process, where the
firing rate at time $t$ is $h(I_{t})$ and $\varphi\leq 0$ can occur as well,
which we interpret as inhibition.

Nonlinear Hawkes processes have been studied to some extent in the past
decades. [4, 32] focus on ergodic properties using Lipschitz conditions of the
transfer function $h$. In [39, 40], central limit theorems and large deviation
results for a univariate nonlinear Hawkes process are given. The case of
inhibition in a multivariate setting was studied in [6] using a thinning
process representation, while [11] uses renewal techniques to establish limit
theorems in the univariate setting.

Mean-field models have frequently been applied in the life sciences; see e.g.
[36] where particular applications in neuroscience are discussed. Mean-field
limits of nonlinear Hawkes processes were first studied by [12]. This has been
extended to age-dependent nonlinear Hawkes processes by [8, 7], and to
specific models including multiple classes [13]. In addition to the law of
large numbers in the mean-field model, a central limit theorem is proved in
[7, 23]. Another branch of research studies a mean-field limit of a spatially
extended (geometric) Hawkes process, showing a law of large numbers [9] and
central limit theorem [10]. Here, a first proof of the neural field equation
could be obtained.

In the present paper, our approach is a combination and extension of  [8],[23]
and [15], with a special focus on models including excitatory and inhibitory
neurons/vertices. We assume that each vertex/neuron is either excitatory or
inhibitory, i.e. excites or inhibits all of its neighbors. We will denote by
$p$ the fraction of excitatory vertices/neurons and distinguish the critical
case $p=1/2$ from the non-critical one. For the latter, we obtain in Theorem 1
a classical mean-field result, i.e. by rescaling the interaction intensity by
$N$, a deterministic limit of the intensity and independent point processes
driven by this intensity arises. We also provide a central limit result for
the intensity. In Theorem 2, we are dealing with the critical case. Here, we
rescale the interaction intensity by $N^{1/2}$, leading to a limiting
intensity which is itself stochastic, and drives conditionally independent
point processes.  We will discuss connections of our findings to previous work
in Remarks 3.4 and 3.6.

The mean-field model we discuss here is unrealistic for applications in
neuroscience for various reasons. Above all, not all neurons are connected.
Sparse graphs are one possibility for a more realistic model [5], while
multiple classes of differently interconnected neurons are another option
[13]. For a detailed discussion on implications of our findings for
neuroscience, see Section 4.

##  2 Model and main results

We use the following general model for a non-linear Hawkes process:

######  Definition 2.1 (Multi-variate, non-linear Hawkes process).

Let $\mathbb{G}=(\mathbb{V},\mathbb{E})$ be some finite, directed graph, and
write $ji\in\mathbb{E}$ if $j\to i$ is an edge in $\mathbb{G}$. Consider a
family of measurable, real-valued functions $(\varphi_{ji})_{ji\in\mathbb{E}}$
and a family of real-valued, non-negative functions
$(h_{i})_{i\in\mathbb{V}}$. Then, a point process $Z=(Z^{i})_{i\in\mathbb{V}}$
(with state space $\mathbb{N}_{0}^{\mathbb{V}}$) is a multi-variate non-linear
Hawkes process with interaction kernels $(\varphi_{ji})_{ji\in\mathbb{E}}$ and
transfer functions $(h_{i})_{i\in\mathbb{V}}$, if $Z^{i},Z^{j}$ do not jump
simultaneously for $i\neq j$, almost surely, and the compensator of $Z^{i}$
has the form $(\int_{0}^{t}\lambda_{s}^{i}ds)_{t\geq 0}$ with

$\lambda_{t}^{i}:=\lambda_{t}^{i}(Z_{s<t}):=h_{i}\Big{(}\sum_{j:ji\in\mathbb{E}}\int_{0}^{t-}\varphi_{ji}(t-s)dZ_{s}^{j}\Big{)},\qquad
i\in\mathbb{V}.$

We need some mimimal conditions such that the multivariate, non-linear Hawkes
process is well-defined (i.e. exists). As mentioned in [12], Remark 5, the law
of the non-linear Hawkes process is well-defined, provided that the following
assumption holds.

######  Assumption 2.2.

All interaction kernels $\varphi_{ji},ji\in\mathbb{E}$ are locally integrable,
and all transfer functions $(h_{i})_{i\in\mathbb{V}}$ are Lipschitz
continuous.

######  Remark 2.3 (Interpretation and initial condition).

  1. 1.

If $dZ_{s}^{j}=1$, we call $\varphi_{ji}(t-s)dZ_{s}^{j}$ the influence of the
point at time $s$ in vertex $j$ on vertex $i$.

  2. 2.

Consider the case of monotonically increasing transfer functions. If
$\varphi_{ji}\leq 0$, we then say that vertex $j$ inhibits $i$, since any
point in $Z^{j}$ decreases the jump rate of $Z^{i}$. Otherwise, if
$\varphi_{ji}\geq 0$, we say that $j$ excites $i$.

  3. 3.

In our formulation, we have $Z_{0}^{i}=0,i\in\mathbb{V}$, with the consequence
that the $dZ_{u}^{j}$-integral in (5.5) could also be extended to $-\infty$
without any change. We note that it would also be possible to use some initial
condition, i.e. some (fixed) $(Z_{t}^{i})_{i\in\mathbb{V},t\leq 0}$, and
extend the integral to the negative reals.

Let us now come to the mean-field model, where we now fix some basic
assumptions. Note that we will show convergence for large graphs, i.e. all
processes come with a scaling parameter $N$, which determines the size of the
graph.

######  Assumption 2.4 (Mean-field setting).

Let

  1. 1.

$\mathbb{G}_{N}=(\mathbb{V}_{N},\mathbb{E}_{N})$ be the complete graph on $N$
vertices, i.e. $\mathbb{V}_{N}=\\{1,...,N\\}$ and $ji,ij\in\mathbb{E}$ for all
$i,j\in\mathbb{V}$;

  2. 2.

$h_{i}=h$ for all $i\in\mathbb{V}_{N}$ where $h\geq 0$ is bounded, $h$ and
$\sqrt{h}$ are Lipschitz with constant $h_{Lip}$;

  3. 3.

$\varphi_{ji}=\theta_{N}U_{j}\varphi$ for all $j,i\in\mathbb{V}_{N}$, where
$U_{1},U_{2},...$ are iid with $\mathbb{P}(U_{1}=1)=1-\mathbb{P}(U_{1}=-1)=p$,
$\varphi\in\mathcal{C}_{b}^{1}([0,\infty))$, the set of bounded continuously
differentiable functions with bounded derivative, and
$\theta_{N}\in\mathbb{R}$.

The form of $\varphi_{ji}$ implies that node $j$ is exciting all other nodes
with probability $p$, and inhibiting all other nodes with probability $1-p$.
By the law of large numbers, we have that

$\displaystyle\frac{1}{N}\sum_{j=1}^{N}U_{j}\xrightarrow{N\to\infty}2p-1$
(2.1) almost surely, and by the central limit theorem,
$\displaystyle\frac{1}{\sqrt{N}}\sum_{j=1}^{N}(U_{j}+1-2p)=:W_{N}\xRightarrow{N\to\infty}W\sim
N(0,4p(1-p)),$ (2.2)

where $\Rightarrow$ denotes weak convergence. For convergence, this suggests
to use $\theta_{N}=\tfrac{1}{N}$ for $p\neq\tfrac{1}{2}$ and
$\theta_{N}=\tfrac{1}{\sqrt{N}}$ for $p=\tfrac{1}{2}$.

##  3 Results on the mean-field model

Our main goal is to give a limit result on the family $Z^{N}$, the
multivariate, non-linear Hawkes process on the graph $\mathbb{G}_{N}$ with
interaction kernels and transfer functions as given in Assumption 2.4. We
distinguish the cases $p\neq\tfrac{1}{2}$ (Theorem 1) and $p=\tfrac{1}{2}$
(Theorem 2). In both cases, the limit compensator is given by
$(\int_{0}^{t}h(I_{s})ds)_{t\geq 0}$, where $I$ is the weak limit of $I^{N}$
(see (3.1) and (3.9)), and we find the following limit results: For
$p\neq\tfrac{1}{2}$, we have that $(I_{t})_{t\geq 0}$ follows a linear,
deterministic convolution equation, and all components of the limit of $Z^{N}$
are independent. For $p=\tfrac{1}{2}$, the critical case, the process
$(I_{t})_{t\geq 0}$ is the unique solution of a stochastic convolution
equation and all components of $Z^{N}$ are conditionally independent given
$I$. Below, we denote by $\Rightarrow$ weak convergence in
$\mathcal{D}_{\mathbb{R}^{n}}([0,\infty))$, the space of cadlag paths, which
is equipped with the Skorohod topology; see e.g. Chapter 3 in [16]. The proof
of the following result can be found in Section 5.4.

######  Theorem 1 (Mean-field limit of multi-variate non-linear Hawkes
processes, $p\neq\tfrac{1}{2}$).

Let Assumption 2.4 hold with $p\neq\tfrac{1}{2}$ and
$\theta_{N}=\tfrac{1}{N}$. Let $Z^{N}=(Z^{N,1},...,Z^{N,N})$ be the
multivariate, non-linear Hawkes process from Definition 2.1, and

$\displaystyle
I^{N}(t):=\frac{1}{N}\sum_{j=1}^{N}\int_{0}^{t-}U_{j}\varphi(t-s)dZ_{s}^{j,N}.$
(3.1)

  1. 1.

Then, $I^{N}\xrightarrow{N\to\infty}I$, uniformly on compact time intervals in
$L^{2}$, where $I=(I_{t})_{t\geq 0}$ is the unique solution of the integral
equation

$\displaystyle I_{t}=(2p-1)\int_{0}^{t}\varphi(t-s)h(I_{s})ds.$ (3.2)

  2. 2.

For all $n=1,2,...$,
$(Z^{N,1},...,Z^{N,n})\xRightarrow{N\to\infty}(\bar{Z}^{1},...,\bar{Z}^{n}),$
where $\bar{Z}^{1},...,\bar{Z}^{n}$ are independent and $\bar{Z}^{i}$ is a
simple point process with intensity at time $t$ given by $h(I_{t})$,
$i=1,...,n$. It is possible to build $\bar{Z}^{1},...,\bar{Z}^{n}$, such that
the convergence is almost surely (in Skorohod-distance).

  3. 3.

Assume that $h\in\mathcal{C}^{1}(\mathbb{R})$ and that $h^{\prime}$ is bounded
and Lipschitz. Then $\sqrt{N}(I^{N}-I)\xRightarrow{N\to\infty}K$, where $K$ is
the unique (strong) solution of

$\displaystyle K_{t}$ $\displaystyle=\int_{0}^{t}\varphi(t-s)dG_{s},$ (3.3)
$\displaystyle G_{t}$
$\displaystyle=\int_{0}^{t}(Wh(I_{s})+(2p-1)h^{\prime}(I_{s})K_{s})ds+\int_{0}^{t}\sqrt{h(I_{s})}dB_{s},$

where $W$ is given by (2.2) and $B$ is a Brownian motion independent from $W$.

######  Remark 3.1.

The form of (3.2) tells us that $I$ follows a linear Volterra convolution
equation [3]. Turning into a differential equation, we write, using Fubini,

$\displaystyle\frac{dI}{dt}$
$\displaystyle=\frac{d}{dt}(2p-1)\Big{(}\int_{0}^{t}\int_{s}^{t}\varphi^{\prime}(r-s)h(I_{s})drds+\int_{0}^{t}\varphi(0)h(I_{s})ds\Big{)}$
$\displaystyle=(2p-1)\Big{(}\int_{0}^{t}\varphi^{\prime}(t-s)h(I_{s})ds+\varphi(0)h(I_{t})\Big{)}.$

In particular, the special choice of $\varphi(s)=e^{-\lambda s}$ gives

$\displaystyle\frac{dI}{dt}$
$\displaystyle={\color[rgb]{0,0,0}-\lambda(2p-1)\int_{0}^{t}\varphi(t-s)h(I_{s})ds+(2p-1)h(I_{t})=-\lambda
I_{t}+(2p-1)h(I_{t}),}$

i.e. $I$ follows some ordinary differential equation in this case.

While Theorem 1 is concerned with convergence of the limit intensity of the
multivariate, non-linear Hawkes process, we are also in the situation to study
convergence of the average intensity of $Z^{N}$. The proof of the next
corollary is found in Section 5.5.

######  Corollary 3.2.

Let $Z^{N}=(Z^{N,1},...,Z^{N,N})$ be as in Theorem 1, and
$\bar{Z}=(\bar{Z}^{1},\bar{Z}^{2},...)$ be as in Theorem 1.2. Then,

$\displaystyle\frac{1}{N}\sum_{j=1}^{N}\Big{(}Z^{N,j}-\int_{0}^{.}h(I^{N}_{s})ds\Big{)}\xrightarrow{N\to\infty}0$
(3.4) and
$\displaystyle\frac{1}{N}\sum_{j=1}^{N}\Big{(}Z^{N,j}-\bar{Z}^{j}\Big{)}\xrightarrow{N\to\infty}0$
(3.5)

in probability, uniformly on compact intervals. Moreover,

$\displaystyle\frac{1}{\sqrt{N}}\sum_{j=1}^{N}\Big{(}Z^{N,j}-\int_{0}^{.}h(I_{s}^{N})ds\Big{)}\xRightarrow{N\to\infty}\int_{0}^{.}\sqrt{h(I_{s})}d\widetilde{B}_{s}$
(3.6)

for some Brownian motion $\widetilde{B}$, and (writing
$h(I):=(h(I_{t}))_{t\geq 0}$)

$\displaystyle\sqrt{N}\big{(}h(I^{N})-h(I)\big{)}\xRightarrow{N\to\infty}h^{\prime}(I)K.$
(3.7)

Let $B,\widetilde{B}$ be correlated Brownian motions with
$\mathbf{E}[B_{t}\widetilde{B}_{t}]=(2p-1)t,t\geq 0$. Use $B$ in the
definition of $K$ in (3.3). Then,

$\displaystyle\frac{1}{\sqrt{N}}\sum_{j=1}^{N}\Big{(}Z^{N,j}-\int_{0}^{.}h(I_{s})ds\Big{)}\xRightarrow{N\to\infty}\int_{0}^{.}h^{\prime}(I_{s})K_{s}ds+\int_{0}^{.}\sqrt{h(I_{s})}d\widetilde{B}_{s}.$
(3.8)

######  Remark 3.3 (Correlation between $B$ and $\widetilde{B}$).

Let us briefly discuss the correlated Brownian motions appearing in (3.8).
Clearly, the left hand sides of (3.6) and (3.7) sum to the left hand side of
(3.8). The limits $\int_{0}^{.}\sqrt{h(I_{s})}d\widetilde{B}_{s}$ and $K$
appearing on the right hand sides of (3.6) and (3.7) are weak limits of sums
of compensated point processes. While in (3.6), we sum over all point
processes in the system, $I^{N}$ in (3.7) distinguishes between nodes with
different signs $U_{j}$. Hence, the correlation is positive for the proportion
$p$ of point processes with positive sign, and negative for the proportion
$1-p$ of point processes with negative sign, summing to $p-(1-p)=2p-1$. For
more details, see the proof in Section 5.5.

######  Remark 3.4 (Connections of Theorem 1 to previous work).

Connections of Theorem 1.1 and Theorem 1.2 to [8, 23]. First note that 1. and
2. of Theorem 1 are a particular case of [8], Theorem 4.1. In their notation,
we have choosen $H_{ij}=U_{j}h$ as interaction functions, which satisfy their
condition (9), as stated in their remark 2.1 on the synaptic weights. We
reformulate the result to introduce 3. of of Theorem 1, which does not appear
in [8], and to allow for a comparison to the result and proof of our Theorem
2.

Note further that Theorem 1.1 implies that the compensator
$(\Lambda_{t})_{t\geq 0}$ of each of the independent limiting processes
$(\bar{Z}^{i}_{t})_{t\geq 0}$ is

$\Lambda_{t}=\int_{0}^{t}h(I_{s})ds=\int_{0}^{t}h\Big{(}(2p-1)\int_{0}^{s}\varphi(s-r)h(I_{r})dr\Big{)}ds=\int_{0}^{t}h\Big{(}(2p-1)\int_{0}^{s}\varphi(s-r)d\Lambda_{r}\Big{)}ds.$

See also (8) of[12] and (5), (7), (8) of [23] for the connection to (3.2),
both for the special case $p=1$ and $W=0$. In contrast to our setting, recall
that [23] require $\varphi(0)=0$.
Connections of Theorem 1.3 to [7], [23]. Equation (5.10) from Theorem 5.6 in
[7] gives an expression for the fluctuation of age dependent Hawkes processes.
In the case where the transfer function becomes independent of the age,
$\Psi(s,x)\equiv\Psi(x)$ in their notation, we discover our equation (3.3) in
the special case $p=1$ and $W=0$. As Corollary 2.2 of [23] is a special case
of this Theorem 5.6, (33) in that corollary is precisely (3.3), and the
convergence of that Corollary for the special case $p=1$ (as well as $W=0$ and
$\varphi(0)=0$) coincides with the statement of Theorem 1.3.
Connections of Corollary 3.2 to [23]. The convergence in (3.7) generalizes
Theorem 2.4 of [23] to the case $p\neq 1$, $W\neq 0$ and $\varphi(0)\neq 0$.
Moreover, the convergence in (3.8) generalizes Theorem 2.1 of [23]: the right
hand side of (3.8) for $p=1$, $W=0$ has $B=\widetilde{B}$ and hence equals $G$
from (3.3). Using integration by parts and $\varphi(0)=0$ we obtain

$\displaystyle 0$
$\displaystyle=\varphi(0)G_{t}-\varphi(t)G_{0}=\int_{0}^{t}\varphi(t-s)dG_{s}-\int_{0}^{t}G_{s}\varphi^{\prime}(t-s)ds.$

Therefore we can reshape $G$ from (3.3),

$\displaystyle G_{t}$
$\displaystyle=\int_{0}^{t}h^{\prime}(I_{u})\int_{0}^{u}\varphi(u-s)dG_{s}du+\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}$
$\displaystyle=\int_{0}^{t}h^{\prime}(I_{u})\int_{0}^{u}\varphi^{\prime}(u-s)G_{s}dsdu+\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}$
$\displaystyle=\int_{0}^{t}G_{s}\int_{s}^{t}\varphi^{\prime}(u-s)h^{\prime}(I_{u})duds+\int_{0}^{t}\sqrt{h(I_{s})}dB_{s},$

to obtain their (24).

We now turn to the case $p=\tfrac{1}{2}$, where we can rescale the intensity
by $\sqrt{N}$ rather than $N$. This is more involved since the limit of
$I^{N}$ turns out to be a stochastic process. For the proof of the following
Theorem, see Section 5.3.

######  Theorem 2 (Mean-field limit of multi-variate non-linear Hawkes
processes, $p=\tfrac{1}{2}$).

Let Assumption 2.4 hold with $p=\tfrac{1}{2}$ and
$\theta_{N}=\tfrac{1}{\sqrt{N}}$. Let $Z^{N}=(Z^{N,1},...,Z^{N,N})$ be the
multivariate, non-linear Hawkes process fom Definition 2.1, and

$\displaystyle
I^{N}(t):=\frac{1}{\sqrt{N}}\sum_{j=1}^{N}\int_{0}^{t-}U_{j}\varphi(t-s)dZ_{s}^{j,N}.$
(3.9)

  1. 1.

For all $w\in\mathbb{R}$, and a Brownian motion $B$, the stochastic integral
equation

$\displaystyle
I_{t}=I_{0}+w\int_{0}^{t}\varphi(t-s)h(I_{s})ds+\int_{0}^{t}\varphi(t-s)\sqrt{h(I_{s})}dB_{s}$
(3.10)

has a unique strong solution.

  2. 2.

Let $W\sim N(0,1)$, $B$ be an independent Brownian motion and $I$ be the
solution of (3.10) with $w$ replaced by $W$. Then, for all $n=1,2,...$,
$(I^{N},Z^{N,1},...,Z^{N,n})\xRightarrow{N\to\infty}(I,\bar{Z}^{1},...,\bar{Z}^{n})$,
where $\bar{Z}^{1},...,\bar{Z}^{n}$ are conditionally independent given $I$
and $\bar{Z}^{i}$ is a simple point process with intensity at time $t$ given
by $h(I_{t})$, $i=1,...,n$.

Again, we discuss convergence of the average intensity of $Z^{N}$. The proof
is in Section 5.5.

######  Corollary 3.5.

Let $Z^{N}=(Z^{N,1},...,Z^{N,N})$ be as in Theorem 2, and
$(I,\bar{Z}^{1},\bar{Z}^{2},...)$ be as in Theorem 2.2. Then, (3.4), (3.5),
(3.6) hold.

We stress that the results corresponding to (3.7) and (3.8) do not hold here
since $I^{N}\Rightarrow I$, so no further upscaling is possible.

######  Remark 3.6 (Some connections to [15]).

Our model and the statements of Theorem 2.2 should be compared to the findings
in [15]. In their model, the size of interactions is given by a centered
probability measure, so at each spike of a neuron, the potential given to
other neurons in the system may change. In contrast, in our model, the
potential given to other neurons is fixed for each neuron, but varies from
neuron to neuron. The latter case seems more relevant in neuroscience; see the
discussion in Section 4. Still, results from Theorem 2 can be compare to
Theorem 1.4(ii) and Theorem 1.7. of [15], where the same type of convergence
is shown, but the interaction kernel $\varphi$ is an exponential function. As
we see in Remark 3.7.2 below, this special choice makes the intensity $I$ a
Markov process. However, as the intensity of both models is still comparable,
we find (3) from [15] in our Remark 3.7.2 below by inserting $w=0$.

######  Remark 3.7 (Some properties of $I$).

  1. 1.

If $\varphi^{\prime}$ exists, it is not hard to see that $I_{t}$ is a
semimartingale, since

$\displaystyle\int_{0}^{t}\varphi(t-u)\sqrt{h(I_{u})}dB_{u}$
$\displaystyle=\int_{0}^{t}\int_{0}^{s}\varphi^{\prime}(t-s)\sqrt{h(I_{u})}dB_{u}ds+\int_{0}^{t}\varphi(0)\sqrt{h(I_{u})}dB_{u}$

by the stochastic Fubini Theorem [3, Theorem 4.A], where the first term on the
right hand side has finite variation and the second is a (local) martingale.
This calculation also shows that $I$ has finite variation provided that
$\varphi(0)=0$.

  2. 2.

Consider the special case $\varphi(s)=e^{-\lambda s}$. Then, a solution of
(3.10) also solves

$dI=(wh(I)-\lambda I)dt+\sqrt{h(I)}dB$

and in particular is a Markov process. In order to see this, we write

$\displaystyle I_{t}-I_{0}=e^{-\lambda t}\Big{(}w\int_{0}^{t}e^{\lambda
s}h(I_{s})ds+\int_{0}^{t}e^{\lambda s}\sqrt{h(I_{s})}dB_{s}\Big{)}.$

Using integration by parts (on the processes $A_{t}:=e^{-\lambda t}$, which
satisfies $dA=-\lambda Adt$, and the term $C_{t}$ in brackets), we find that

$\displaystyle I_{t}-I_{0}$
$\displaystyle=A_{t}C_{t}=\int_{0}^{t}A_{s}dC_{s}+\int_{0}^{t}C_{s}dA_{s}$
$\displaystyle=w\int_{0}^{t}h(I_{s})ds+\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}-\lambda\int_{0}^{t}A_{s}C_{s}ds$
$\displaystyle=\int_{0}^{t}(wh(I_{s})-\lambda
I_{s})ds+\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}.$

##  4 Implications for neuroscience

The human brain is very large. It comprises 86 billion neurons, along with
other types of cells [24]. Each individual neuron makes contact with hundreds
or thousands of other neurons, forming a very complex network. It should be
thought of as a highly integrated circuit, made of biological parts, which are
capable of processing electro-chemical signals and transmitting them to
distant neurons with high fidelity. Neurons use electrical impulses (“action
potentials” or “spikes”) to communicate with other neurons in the same
network. Sequences of these impulses evolve in time (“spike trains”),
involving the collective activity of many interacting neurons. These
multivariate signals are believed to reflect the dynamic “computations”
performed by these networks. The biological “purpose” of these processes
covers a wide range from immediate control of bodily functions, like
activating a muscle, to abstract information processing, like making a complex
decision [30].

Neurons communicate with other neurons in a network by activating specialized
functional contacts between cells (“synapses”). In the most common type of
synapses, diffusing chemicals are used for signaling (“neurotransmitters”). In
a nutshell, each spike of the sender neuron triggers a transient change in
excitability of the receiver neuron. This can either increase or decrease the
tendency of the receiver neuron to fire an action potential. We speak of
“excitation” and “inhibition”, respectively. A hallmark of normal brain
function is a robust balance of excitation and inhibition. Brain diseases, in
contrast, are often linked with a marked imbalance of these two forces [38].

A whole zoo of mathematical models exist to mimic the dynamical processes of
synaptic interaction in networks of spiking neurons. Most of them are
difficult to analyze, due to the discrete nature of pulse trains and an
essential nonlinearity in the spike generation mechanism. From a mathematical
point of view, the Hawkes process seems like a very reasonable choice as it
can directly reflect several important biological features. First, as a
stochastic point process it is by construction based on pulse-coded signaling.
Second, the use of interaction kernels automatically reflects the causal
nature of communication in a physical system with recurrent loops, allowing
only spikes in the past to influence the future evolution of the network.
Third, as a multivariate process with coupled components it is able to capture
and consistently describe the collective activity dynamics.

The linear Hawkes process [22] has been employed as a mathematical model of
networks of spiking neurons, as it can directly reflect excitatory synaptic
interactions between spiking neurons using positive interaction kernels [28].
An important shortcoming of the linear model is that inhibitory synaptic
interaction cannot be included in a consistent way. Negative interaction
kernels are not allowed, as they might yield a negative point process
intensity. To enable the study of networks with excitation-inhibition balance,
however, negative interaction kernels must be admitted [35]. A nonlinear link
function can fix this deficit, and effective descriptions of nonlinear
biological neurons combining linear synaptic interaction with a nonlinear link
function are actually well-known in computational neuroscience. They have been
called cascade models [25], and from the viewpoint of statistics they lead to
generalized linear models [34]. For that reason, the nonlinear Hawkes process
is a natural extension of Hawkes’ classical model, greatly extending its range
of applications.

The specific model described in this paper can be further generalized,
allowing for a different interaction kernel for each pair of nodes in the
network. This includes networks with arbitrary graph topology, setting kernels
to zero where a synapse is absent [35]. In brain networks, the amplitude of
individual interaction kernels (“synaptic weight”) is thought to change as a
result of learning. The promise for the biologist is to be able to investigate
networks, the structure of which is either designed for a specific purpose, or
reflects previous experience and memory. The challenge for the mathematician
is to manage the lack of symmetry in these networks, and a genuinely
multivariate description becomes necessary.

There is an ongoing debate about the significance of scaling synaptic weights
for networks of infinite size [37, 2]. As the number of neurons in brain
networks is typically quite high, such limit may represent a meaningful
approximation to large brains. The difficulty is that large brains also have
to accommodate constraints with regard to the number of possible connections,
imposed by the distance of neurons in space and by the volume occupied by
fibers. So it might be more fruitful to rephrase the question a bit [1]: Which
aspect of the input to a neuron from within the network is more important: Is
it the mean drift, or is it the amplitude of the fluctuations? If there is a
perfect balance between excitation and inhibition, the drift is zero, and the
fluctuations take over and drive the neuron in a stochastic fashion. For the
slightest imbalance of the inputs, however, keeping the drift bounded in very
large networks forces the fluctuations to be very small, and the dynamics
becomes more deterministic. This is where biophysical intuition is in line
with the result of mathematical analysis presented in this paper.

##  5 Proofs

We start off in Subsection 5.1 with some preliminary results on convergence of
Poisson processes and convolution equations. We proceed in 5.2 with a
reformulation of the multivariate linear Hawkes process using a time-change
equation. Then, we prove Theorem 2 in Sebsection 5.3 and Theorem 1 in
Subsection 5.4. We give the proof of Theorem 2 first because it is more
involved than the proof of Theorem 1, mainly since the limit process $I$ is
stochastic.

###  5.1 Preliminairies

We need the following (rather standard) convergence results for Poisson
processes.

######  Lemma 5.1 (Convergence of Poisson processes).

Let $Y$ be a unit rate Poisson process, defined on some probability space
$(\Omega,\mathcal{F},\mathbb{P})$. Then, for all $t>0$,

$\displaystyle\sup_{0\leq s\leq
t}\frac{1}{N}(Y(Ns)-Ns)\xrightarrow{N\to\infty}0$ (5.1)

almost surely (and in $L^{2}$). In addition,

$\displaystyle\Big{(}\frac{1}{\sqrt{N}}(Y(Nt)-Nt)\Big{)}_{t\geq
0}\xRightarrow{N\to\infty}B$ (5.2)

for some Brownian motion $B$. We can extend $(\Omega,\mathcal{F},\mathbb{P})$
by a Brownian motion $B$ such that, for all $t\geq 0$,

$\displaystyle\sup_{0\leq s\leq
t}\Big{|}\frac{1}{\sqrt{N}}(Y(Ns)-Ns)-B_{s}\Big{|}\xrightarrow{N\to\infty}0$
(5.3)

almost surely and in $L^{2}$.

###### Proof.

For (5.1), we use Doob’s martingale inequality for the martingale
$(Y(s)-s)_{s\geq 0}$ and the fact that the fourth centered moment of a Poisson
random variable with parameter $\lambda$ is $3\lambda^{2}+\lambda$ in order to
see

$\displaystyle\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\Big{(}\frac{1}{N}(Y(Ns)-Ns)\Big{)}^{4}\Big{]}\leq\Big{(}\frac{4}{3}\Big{)}^{4}\mathbb{E}\Big{[}\Big{(}\frac{1}{N}(Y(Nt)-Nt)\Big{)}^{4}\Big{]}={\color[rgb]{0,0,0}\Big{(}\frac{4}{3N}\Big{)}^{4}}\big{(}3(Nt)^{2}+Nt\big{)},$

which is summable. Using Borel-Cantelli, almost-sure convergence follows.
Next, (5.2) follows by an application of Donsker’s theorem ([26] Ch VII, Cor.
3.11). By Skorohod’s Theorem, we can extend our probability space such that
this convergence is almost surely with respect to Skorohod distance, and by
continuity of $B$ it is equivalent to uniform convergence, i.e. (5.3) holds
almost surely for all $t\geq 0$. For the $L^{2}$-convergence in (5.3), it
suffices to show the uniform integrability of $\sup_{0\leq s\leq
t}\Big{(}\frac{1}{\sqrt{N}}(Y(Ns)-Ns)-B_{s}\Big{)}^{2}$. Using Doob’s and
Minkovski’s inequalities, we compute

$\displaystyle\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\Big{(}\frac{1}{\sqrt{N}}(Y(Ns)-Ns)-B_{s}\Big{)}^{4}\Big{]}^{1/4}$
$\displaystyle\leq\frac{4}{3}\mathbb{E}\Big{[}\Big{(}\frac{1}{\sqrt{N}}(Y(Nt)-Nt)-B_{t}\Big{)}^{4}\Big{]}^{1/4}$
$\displaystyle\leq\frac{4}{3}\Big{(}\mathbb{E}\Big{[}\Big{(}\frac{1}{\sqrt{N}}(Y(Nt)-Nt)\Big{)}^{4}\Big{]}^{1/4}+\mathbb{[}(B_{t})^{4}]^{1/4}\Big{)}$
$\displaystyle=\frac{4}{3}((3t^{2}+t/N)^{1/4}+(3t^{2})^{1/4})$

showing the desired uniform integrability and $L^{2}$-convergence follows. ∎

In order to bound the value of a convolution equation by its integrator we
need the following

######  Lemma 5.2.

Let $J$ be the sum of an Itô process with bounded coefficients and a càdlàg
pure-jump process, and let $\varphi\in\mathcal{C}_{b}^{1}([0,\infty))$. Then

$\displaystyle\sup_{0\leq s\leq
t}\Big{(}\int_{0}^{s}\varphi(s-r)dJ_{r}\Big{)}^{2}\leq
C(\varphi,t)\cdot\sup_{0\leq s\leq t}J_{s}^{2}.$ (5.4)

###### Proof.

Wlog, we have $J_{0}=0$. By [3, Theorem 4.A] and Fubini’s theorem for
Lebesgue-integrals we can apply the Stochastic Fubini Theorem to $J$, hence

$\displaystyle\sup_{0\leq s\leq
t}\Big{(}\int_{0}^{s}\varphi(s-r)dJ_{r}\Big{)}^{2}$ $\displaystyle=\sup_{0\leq
s\leq
t}\Big{(}\int_{0}^{s}\Big{(}\varphi(0)+\int_{r}^{s}\varphi^{\prime}(s-u)du\Big{)}dJ_{r}\Big{)}^{2}$
$\displaystyle\leq 2\,||\varphi||^{2}\cdot\sup_{0\leq s\leq
t}J_{\color[rgb]{0,0,0}s}^{2}+2\sup_{0\leq s\leq
t}\Big{(}\int_{0}^{s}\int_{0}^{u}\varphi^{\prime}(s-u)dJ_{r}du\Big{)}^{2}$
$\displaystyle=2\,||\varphi||^{2}\cdot\sup_{0\leq s\leq
t}J_{s}^{2}+2\sup_{0\leq s\leq
t}\Big{(}\int_{0}^{s}\varphi^{\prime}(s-u)J_{u}du\Big{)}^{2}$
$\displaystyle\leq
2\,(||\varphi||^{2}+t||\varphi^{\prime}||^{2})\cdot\sup_{0\leq s\leq
t}J_{s}^{2}.$

∎

###  5.2 Reformulation of Hawkes processes

Alternative descriptions of non-linear Hawkes processes have been given in the
literature. Above all, the construction using a Poisson random measures is
widely used; see e.g. Proposition 3 in [12]. Here, we rely on the following
construction using time-change equations (see e.g. Chapter 6 of [16]), which
we give here without proof.

######  Lemma 5.3.

Let $\mathbb{G}=(\mathbb{V},\mathbb{E})$, $(\varphi_{ji})_{ji\in\mathbb{E}}$
and $(h_{i})_{i\in\mathbb{V}}$ be as in Definition 2.1, and let Assumption 2.2
hold. A point process $Z=(Z^{i})_{i\in\mathbb{V}}$ is a multivariate, non-
linear Hawkes process with interaction kernels
$(\varphi_{ji})_{ji\in\mathbb{E}}$ and transfer functions
$(h_{i})_{i\in\mathbb{V}}$, if and only if it is the weak solution of the
time-change equations

$\displaystyle
Z_{t}^{i}=Y_{i}\Big{(}\int_{0}^{t}\lambda_{s}^{i}ds\Big{)}=Y_{i}\Big{(}\int_{0}^{t}h_{i}\Big{(}\sum_{j:ji\in\mathbb{E}}\int_{0}^{s-}\varphi_{ji}(s-u)dZ_{u}^{j}\Big{)}ds\Big{)},$
(5.5)

where $(Y_{i})_{i\in\mathbb{V}}$ is a family of independent unit rate Poisson
processes.

Under Assumption 2.4, the time-change equations from this lemma read, with
independent unit rate Poisson processes $Y_{1},...,Y_{N}$,

$\displaystyle Z_{t}^{N,i}$
$\displaystyle=Y_{i}\Big{(}\int_{0}^{t}h\Big{(}\sum_{j=1}^{N}\int_{0}^{s-}\theta_{N}U_{j}\varphi(s-u)dZ_{u}^{N,j}\Big{)}ds\Big{)}.$
(5.6) We rewrite this as $\displaystyle Z_{t}^{N,i}$
$\displaystyle=Y_{i}\Big{(}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}$ (5.7) with
$\displaystyle I_{s}^{N}$
$\displaystyle=\int_{0}^{s-}\varphi(s-u)dJ_{u}^{N}\quad\text{ and }\quad
J_{u}^{N}=\theta_{N}\sum_{j=1}^{N}U_{j}Z_{u}^{N,j}.$ (5.8)

Now, we rewrite $J^{N}$ further.  Observe that $\sum_{j:U_{j}=1}Y_{j}$ is a
Poisson process with rate
$\tfrac{1}{2}\sum_{j=1}^{N}(U_{j}+1)=pN+\tfrac{1}{2}\sqrt{N}W_{N}$ (recall
$W_{N}$ from (2.2)) and $\sum_{j:U_{j}=-1}Y_{j}$ is a Poisson process with
rate $(1-p)N-\tfrac{1}{2}\sqrt{N}W_{N}$. Hence we have that, for two
independent unit rate Poisson processes $Y^{+}$ and $Y^{-}$,

$\displaystyle J_{t}^{N}$
$\displaystyle=\theta_{N}\sum_{j=1}^{N}U_{j}Y_{j}\Big{(}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}$
(5.9)
$\displaystyle=\theta_{N}Y^{+}\Big{(}\big{(}pN+\tfrac{1}{2}\sqrt{N}W_{N}\big{)}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-\theta_{N}Y^{-}\Big{(}\big{(}(1-p)N-\tfrac{1}{2}\sqrt{N}W_{N}\big{)}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}.$

We extend our probability space such that the rescaled Poisson processes
$Y^{+},Y^{-}$ converge to Brownian motions $B^{+},B^{-}$ almost surely (as in
(5.3)). Consider the Brownian motion $\hat{B}=(\hat{B}(t))_{t\geq 0}$ with
$\hat{B}_{t}=B^{+}_{pt}-B^{-}_{(1-p)t}$. Since
$\Big{(}\hat{B}\Big{(}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}\Big{)}_{t\geq 0}$ is a
continuous martingale, we can extend the probability space by another Brownian
motion $B^{N}$, such that

$\displaystyle\hat{B}\Big{(}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}=\int_{0}^{t}\sqrt{h(I_{s}^{N})}dB_{s}^{N}\quad\text{for
}t\geq 0,$ (5.10)

by [29, Theorem 16.10].

###  5.3 Proof of Theorem 2

First, strong existence and uniqueness for (3.10) follows from [3, Theorem
3.A]. For the convergence result, we work on a probability space where the
convergence $W_{N}\to W$ from (2.2) holds almost surely. We will show in Steps
1 and 2 convergence of $J^{N}$ to $J$ with

$\displaystyle
J_{t}=W\int_{0}^{t}h(I_{s})ds+\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}$ (5.11)

and from $I^{N}$ to $I$ from (3.10) with $w$ replaced by $W$. Step 3 then
gives the desired convergence of $(I^{N},Z^{N,1},...,Z^{N,n})$. We use
processes $\widetilde{J}^{N}$ and $\widetilde{I}^{N}$ which are unique strong
solutions of

$\displaystyle\widetilde{J}^{N}_{t}$
$\displaystyle=W_{N}\int_{0}^{t}h\big{(}\widetilde{I}_{s}^{N}\big{)}ds+\int_{0}^{t}\sqrt{h(\widetilde{I}_{s}^{N})}dB_{s}^{N},$
$\displaystyle\widetilde{I}_{s}^{N}$
$\displaystyle=\int_{0}^{s}\varphi(s-r)d\widetilde{J}^{N}_{r},$

where $B^{N}$ is the Brownian motion defined by (5.10). Again, strong
existence and uniqueness follows from [3, Theorem 3.A]. In Step 1, we will
show that $J^{N}-\widetilde{J}^{N}\to 0$ and $I^{N}-\widetilde{I}^{N}\to 0$
uniformly on compact sets in $L^{2}$, conditional on
$(U_{i})_{i\in\mathbb{N}}$. In Step 2, we show the remaining conditional
convergence $\widetilde{I}^{N}-I\to 0$ and conclude $I^{N}-I\to 0$ in
probability.

Step 1: Convergence $J^{N}-\widetilde{J}^{N}\to 0$ and
$I^{N}-\widetilde{I}^{N}\to 0$.
Recall $Y^{+}$ and $Y^{-}$ from Section 5.2. For
$\theta_{N}=\tfrac{1}{\sqrt{N}}$ and $p=\tfrac{1}{2}$ in (5.9) we get that,

$J_{t}^{N}=\frac{1}{\sqrt{N}}Y^{+}\Big{(}\tfrac{N+\sqrt{N}W_{N}}{2}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-\frac{1}{\sqrt{N}}Y^{-}\Big{(}\tfrac{N-\sqrt{N}W_{N}}{2}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}.$

We write

$\displaystyle J_{t}^{N}-\widetilde{J}_{t}^{N}$
$\displaystyle=A_{t}^{N}+W_{N}C_{t}^{N}+D_{t}^{N}\quad\text{with}$ (5.12)
$\displaystyle A_{t}^{N}$
$\displaystyle=\frac{1}{\sqrt{N}}Y^{+}\Big{(}\frac{N+\sqrt{N}W_{N}}{2}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-\frac{1}{\sqrt{N}}Y^{-}\Big{(}\frac{N-\sqrt{N}W_{N}}{2}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad-
W_{N}\int_{0}^{t}h(I_{s}^{N})ds-\hat{B}\Big{(}\int_{0}^{t}h(I_{s}^{N})ds\Big{)},$
$\displaystyle C_{t}^{N}$
$\displaystyle=\int_{0}^{t}h(I_{s}^{N})ds-\int_{0}^{t}h(\widetilde{I}_{s}^{N})ds,$
$\displaystyle D_{t}^{N}$
$\displaystyle=\int_{0}^{t}\sqrt{h(I_{s}^{N})}dB^{N}_{s}-\int_{0}^{t}\sqrt{h(\widetilde{I}_{s}^{N})}dB^{N}_{s}.$

We make the disclaimer, that we will use $C>0$ for a variable, which only
depends on $t,h,\varphi$, and which might change from line to line.  We write
$\mathbb{E}_{U}$ for the conditional expectation with respect to
$U=(U_{i})_{i\in\mathbb{N}}$. The function $h$ is bounded, therefore, for some
unit rate Poisson process $Y$,

$\displaystyle\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq t}(A_{s}^{N})^{2}]$
$\displaystyle\leq\mathbb{E}{{}_{U}}\Big{[}\sup_{0\leq s\leq
t||h||}\Big{(}\frac{1}{\sqrt{N}}Y^{+}\Big{(}\frac{N+\sqrt{N}W_{N}}{2}s\Big{)}-\frac{1}{\sqrt{N}}Y^{-}\Big{(}\frac{N-\sqrt{N}W_{N}}{2}s\Big{)}$
(5.13)
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad-
W_{N}s-\hat{B}(s)\Big{)}^{2}\Big{]}$ $\displaystyle\leq
C\cdot\Big{(}\mathbb{E}{{}_{U}}\Big{[}\sup_{0\leq s\leq
t||h||}\Big{(}\frac{1}{\sqrt{N}}(Y^{+}(Ns/2)-Ns/2)-B^{+}(s/2)\Big{)}^{2}\Big{]}$
$\displaystyle\qquad+\mathbb{E}{{}_{U}}\Big{[}\sup_{0\leq s\leq
t||h||}\Big{(}\frac{1}{\sqrt{N}}(Y^{-}(Ns/2)-Ns/2)-B^{-}(s/2)\Big{)}^{2}\Big{]}$
$\displaystyle\qquad+\mathbb{E}{{}_{U}}\Big{[}\sup_{0\leq
s\leq|W_{N}|t||h||}\Big{(}\frac{1}{\sqrt{N}}(Y(\sqrt{N}s))-s\Big{)}^{2}\Big{]}\Big{)}$
$\displaystyle{\color[rgb]{0,0,0}\leq
o(1)+C\cdot\mathbb{E}_{U}\Big{[}\Big{(}\frac{1}{\sqrt{N}}(Y(\sqrt{N}|W_{N}|t||h||s))-|W_{N}|t||h||s\Big{)}^{2}\Big{]}}$
$\displaystyle{\color[rgb]{0,0,0}\leq
o(1)+C\frac{1}{\sqrt{N}}W_{N}}\xrightarrow{N\to\infty}0$

almost surely, by Lemma 5.1 and Doob’s inequality. Then, using Jensen,

$\displaystyle\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq t}(W_{N}C_{s}^{N})^{2}]$
$\displaystyle{\color[rgb]{0,0,0}\leq\sup_{N}W_{N}^{2}}\mathbb{E}{{}_{U}}\Big{[}\sup_{0\leq
s\leq
t}\Big{(}\int_{0}^{s}h(I_{r}^{N})-h(\widetilde{I}_{r}^{N})dr\Big{)}^{2}\Big{]}$
$\displaystyle\leq{\color[rgb]{0,0,0}\sup_{N}W_{N}^{2}}th_{Lip}^{2}\int_{0}^{t}\mathbb{E}{{}_{U}}[(I_{s}^{N}-\widetilde{I}_{s}^{N})^{2}]ds$

Note that $\sup_{N}W_{N}^{2}<\infty$, as $W_{N}$ converges to $W$, which is
finite almost surely. By Doob’s inequality and Ito’s isometry,

$\displaystyle\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq t}(D_{s}^{N})^{2}]$
$\displaystyle\leq
C\cdot\mathbb{E}{{}_{U}}[(D_{t}^{N})^{2}]=C\cdot\mathbb{E}{{}_{U}}\Big{[}\Big{(}\int_{0}^{t}\sqrt{h(I_{s}^{N})}dB^{N}_{s}-\int_{0}^{t}\sqrt{h(\widetilde{I}_{s}^{N})}dB^{N}_{s}\Big{)}^{2}\Big{]}$
$\displaystyle=C\cdot\mathbb{E}{{}_{U}}\Big{[}\int_{0}^{t}\big{(}\sqrt{h(I_{s}^{N})}-\sqrt{h(\widetilde{I}_{s}^{N})}\big{)}^{2}ds\Big{]}$
$\displaystyle\leq
C\cdot\int_{0}^{t}\mathbb{E}{{}_{U}}[(I_{s}^{N}-\widetilde{I}_{s}^{N})^{2}]ds.$

In order to bound $\sup_{0\leq s\leq t}(W_{N}C_{s}^{N})^{2}$ and $\sup_{0\leq
s\leq t}(D_{s}^{N})^{2}$ by $\sup_{0\leq s\leq
t}(J_{s}^{N}-\widetilde{J}_{s}^{N})^{2}$, apply Lemma 5.2 to
$J_{s}^{N}-\widetilde{J}_{s}^{N}$. We obtain

$\displaystyle\mathbb{E}{{}_{U}}\big{[}\sup_{s\leq
t}(J_{s}^{N}-\widetilde{J}_{s}^{N})^{2}\big{]}$ $\displaystyle\leq
C\cdot\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq
t}(A_{s}^{N})^{2}]+C\cdot\int_{0}^{t}\mathbb{E}{{}_{U}}[(I_{s}^{N}-\widetilde{I}_{s}^{N})^{2}]ds$
$\displaystyle\leq C\cdot\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq
t}(A_{s}^{N})^{2}]+C\cdot\int_{0}^{t}\mathbb{E}{{}_{U}}[\sup_{r\leq
s}(J_{r}^{N}-\widetilde{J}_{r}^{N})^{2}]ds.$

By the Gronwall inequality and (5.13), we conclude that, for all $t\geq 0$,

$\displaystyle\mathbb{E}{{}_{U}}\big{[}\sup_{s\leq
t}(J_{s}^{N}-\widetilde{J}_{s}^{N})^{2}\big{]}$
$\displaystyle\xrightarrow{N\to\infty}0$ and with Lemma 5.2 also
$\displaystyle\mathbb{E}{{}_{U}}\big{[}\sup_{s\leq
t}(I_{s}^{N}-\widetilde{I}_{s}^{N})^{2}\big{]}$
$\displaystyle\xrightarrow{N\to\infty}0,$

almost surely.

Step 2: Convergence $\widetilde{I}^{N}-I\to 0$
Since $B^{N}$ is a Brownian motion, we have that, in distribution,

$\displaystyle\widetilde{I}_{t}^{N}$
$\displaystyle=W_{N}\int_{0}^{t}\varphi(t-s)h(\widetilde{I}_{s}^{N})ds+\int_{0}^{t}\varphi(t-s)\sqrt{h(\widetilde{I}_{s}^{N})}dB_{s}^{N},$
$\displaystyle I_{t}$
$\displaystyle=W\int_{0}^{t}\varphi(t-s)h(I_{s})ds+\int_{0}^{t}\varphi(t-s)\sqrt{h(\widetilde{I}_{s})}dB_{s}^{N}.$

Therefore, on this probability space,

$\displaystyle\widetilde{I}_{t}^{N}-I_{t}$
$\displaystyle=E_{t}^{N}+F_{t}^{N}+G_{t}^{n}\text{ with}$ $\displaystyle
E_{t}^{n}$ $\displaystyle:=(W_{N}-W)\int_{0}^{t}\varphi(t-s)h(I_{s})ds,$
$\displaystyle F_{t}^{N}$
$\displaystyle:=W_{N}\int_{0}^{t}\varphi(t-s)(h(\widetilde{I}_{s}^{N})-h(I_{s}))ds,$
$\displaystyle G_{t}^{N}$
$\displaystyle:=\int_{0}^{t}\varphi(t-s)\Big{(}\sqrt{h(\widetilde{I}_{s}^{N})}-\sqrt{h(I_{s})}\Big{)}dB_{s}^{N}.$

Since, again using some constant $C<\infty$, which can change from line to
line,

$\displaystyle\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq t}(E_{s}^{N})^{2}]$
$\displaystyle\leq
C\cdot{\color[rgb]{0,0,0}(W_{N}-W)^{2}}\xrightarrow{N\to\infty}0,$
$\displaystyle\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq t}(F_{s}^{N})^{2}]$
$\displaystyle\leq
C\cdot{\color[rgb]{0,0,0}\sup_{N}W_{N}^{2}}\int_{0}^{t}\mathbb{E}{{}_{U}}[\sup_{0\leq
r\leq s}(\widetilde{I}_{r}^{N}-I_{r})^{2}]ds,$
$\displaystyle\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq t}(G_{s}^{N})^{2}]$
$\displaystyle\leq C\cdot\int_{0}^{t}\mathbb{E}{{}_{U}}[\sup_{0\leq r\leq
s}(\widetilde{I}_{r}^{N}-I_{r})^{2}]ds,$ we find that
$\displaystyle\mathbb{E}{{}_{U}}[\sup_{0\leq s\leq
t}(\widetilde{I}_{s}^{N}-I_{s})^{2}]$ $\displaystyle\leq
C\cdot{\color[rgb]{0,0,0}(W_{N}-W)^{2}}+C\cdot\int_{0}^{t}\mathbb{E}{{}_{U}}[\sup_{0\leq
r\leq s}(\widetilde{I}_{r}^{N}-I_{r})^{2}]ds,$

and the Gronwall inequality again implies that $\mathbb{E}{{}_{U}}[\sup_{0\leq
s\leq t}(\widetilde{I}_{s}^{N}-I_{s})^{2}]\xrightarrow{N\to\infty}0$, almost
surely. Combining Step 1 and Step 2 we obtain $\mathbb{E}_{U}[\sup_{0\leq
s\leq t}(I_{s}^{N}-I_{s})^{2}]\xrightarrow{N\to\infty}0$. We deduce
$\mathbb{E}_{U}[1\wedge\sup_{0\leq s\leq
t}(I_{s}^{N}-I_{s})^{2}]\xrightarrow{N\to\infty}0$ and, by dominated
convergence, $\mathbb{E}[1\wedge\sup_{0\leq s\leq
t}(I_{s}^{N}-I_{s})^{2}]\xrightarrow{N\to\infty}0$. Hence we have shown that
$\sup_{0\leq s\leq t}(I_{s}^{N}-I_{s})^{2}\to 0$ in probability.

Step 3: Convergence of $(I^{N},Z^{N,1},...,Z^{N,n})$
The predictable quadratic variation of the compensated process
$Z^{N,i}_{t}-\int_{0}^{t}h(I_{s}^{N})ds$ is given by $\langle
Z^{N,i}-\int_{0}^{\cdot}h(I_{s}^{N})ds\rangle_{t}=\int_{0}^{t}h(I_{s}^{N})ds$.
By [26][Chapter VI, Corollary 3.33, Theorem 4.13], tightness of
$(Z^{N,1},...,Z^{N,n})$ follows from convergence of $\sum_{i\leq n}\langle
Z^{N,i}-\int_{0}^{\cdot}h(I_{s}^{N})ds\rangle_{t}=n\int_{0}^{t}h(I_{s}^{N})ds$.
We already know that $I^{N}\Rightarrow I$, hence $(I^{N},Z^{N,1},...,Z^{N,n})$
is tight. To identify the limit, let us try to extend the state space such
that the Hawkes process becomes Markovian. Recall that each $Z^{N,i}$ jumps at
time $t$ at rate $h(I^{N}(t))$ with
$I^{N}(t)=\int_{0}^{t}\varphi(t-s)dJ^{N}_{s}$ and $J_{s}^{N}$ as given in
(5.8). The process $J^{N}$ jumps at time $t$ from $x$ to $x+\theta_{N}$ at
rate $h\Big{(}\int_{0}^{t}\varphi(t-s)dJ^{N}_{s}\Big{)}\cdot u^{+}$ with
$u^{+}:=\sum_{j}1_{U_{j}=1}$ and to $x-\theta_{N}$ at rate
$h\Big{(}\int_{0}^{t}\varphi(t-s)dJ^{N}_{s}\Big{)}\cdot u^{-}$ with
$u^{-}=\sum_{j}1_{U_{j}=-1}=N-u^{+}$. Note that the process $J^{N}$ satisfies,
for smooth $f$,

$\displaystyle\frac{\mathbb{E}\Big{[}f(J^{N}_{t+\varepsilon})-f(J^{N}_{t})|\mathcal{F}_{t}\Big{]}}{\varepsilon}$
$\displaystyle=h(I_{t}^{N})u^{+}\Big{[}f(J^{N}_{t}+\theta_{N})-f(J^{N}_{t})\Big{]}+h(I_{t}^{N})u^{-}\Big{[}f(J^{N}_{t}-\theta_{N})-f(J^{N}_{t})\Big{]}+o(\varepsilon)$
$\displaystyle=f^{\prime}(J^{N}_{t})\cdot\Big{(}h(I_{t}^{N})\underbrace{(u^{+}_{N}-u^{-}_{N})\theta_{N}}_{=W_{N}}\Big{)}+\tfrac{1}{2}f^{\prime\prime}(J^{N}_{t})\cdot
h(I_{t}^{N})\underbrace{(u^{+}_{N}+u^{-}_{N})\theta_{N}^{2}}_{=1}+o(\varepsilon)$
$\displaystyle=f^{\prime}(J^{N}_{t})\cdot
h(I_{t}^{N})W_{N}+\tfrac{1}{2}f^{\prime\prime}(J^{N}_{t})\cdot
h(I_{t}^{N})+o(\varepsilon).$

We obtain a similar expression if we ignore the jumps of $Z^{N,i},i=1,...,n$,
i.e. if we replace $J^{N}$ by $J^{N(n)}=\theta_{N}\sum_{j>n}U_{j}Z^{N,j}$. In
this case we have to replace $W_{N}$ by $W_{N(n)}=\theta_{N}\sum_{j>n}U_{j}$
and $(u^{+}_{N}+u^{-}_{N})\theta_{N}^{2}$ by $(N-n)/N$. In the limit
$N\to\infty$ we obtain the same expression, as $W_{N(n)}\to W$ and $(N-n)/N\to
1$. From now on write $J_{[0,t]}$ for the path of a process $J$ on the
interval $[0,t]$ and

$\displaystyle G_{J}^{N(n)}f(J_{[0,t]}^{\color[rgb]{0,0,0}N})$
$\displaystyle=f^{\prime}(J_{t}^{\color[rgb]{0,0,0}N})\cdot
h(I_{t}^{\color[rgb]{0,0,0}N})W_{N(n)}+\tfrac{N-n}{2N}f^{\prime\prime}(J_{t}^{\color[rgb]{0,0,0}N})\cdot
h(I_{t}^{\color[rgb]{0,0,0}N})$ $\displaystyle G_{J}f(J_{[0,t]})$
$\displaystyle=f^{\prime}(J_{t})\cdot
h(I_{t})W+\tfrac{1}{2}f^{\prime\prime}(J_{t})\cdot h(I_{t})$

The generator $G^{N}$ of the Markov process
$\big{(}J^{N}_{[0,t]},Z^{N,1}_{t},...,Z^{N,n}_{t}\big{)}$ with domain
$\mathcal{D}$ consisting of functions of the form
$f_{0,...,n}(J_{[0,t]},k_{1},...,k_{n})=f_{0}(J_{t})f_{1}(k_{1})...f_{n}(k_{n})$,
where we suppose the functions $f_{1},...,f_{n}$ to be bounded functions on
$\mathbb{N}$, and $f_{0}$ to be a smooth function on $\mathbb{R}$ with bounded
second derivative, is given by

$\displaystyle G^{N}f_{0,..,n}(J_{[0,t]}^{\color[rgb]{0,0,0}N},Z^{N,1}_{t}$
$\displaystyle,...,Z^{N,n}_{t})=G^{N(n)}_{J}f_{0}(J_{[0,t]}^{\color[rgb]{0,0,0}N})\prod_{i=1}^{n}f_{i}(Z^{N,i}_{t})$
$\displaystyle+h(I_{t})\sum_{j=1}^{n}\big{(}f_{0}(J_{t}+\theta_{N}U_{j})f_{j}(Z^{N,j}_{t}+1)-f_{0}(J_{t})f_{j}(Z^{N,j}_{t})\big{)}\prod_{i\neq
j}f_{i}(Z^{N,i}_{t}).$

By [16][Chapter 4, Proposition 1.7],

$\displaystyle
f_{0}(J^{N}_{t})\prod_{i=1}^{n}f_{i}(Z^{N,i}_{t})-\int_{0}^{t}G^{N}f_{0}...f_{n}(J_{[0,s]}^{\color[rgb]{0,0,0}N},Z^{N,1}_{s},...,Z^{N,n}_{s})ds$

is a martingale with respect to the filtration generated by $J,Z^{N,i}$. It is
equivalent that

$\displaystyle\mathbb{E}\Big{[}\Big{(}f_{0}(J^{N}_{t})\prod_{i=1}^{n}f_{i}(Z^{N,i}_{t})-f_{0}(J^{N}_{s})\prod_{i=1}^{n}f_{i}(Z^{N,i}_{s})-\int_{s}^{t}G^{N}f_{0}...f_{n}(J_{[0,r]}^{\color[rgb]{0,0,0}N},Z^{N,1}_{r},...,Z^{N,n}_{r})ds\Big{)}$
$\displaystyle\prod_{j=1}^{m}g_{j}(J_{[0,t_{j}]}^{\color[rgb]{0,0,0}N},Z^{N,1}_{t_{j}},...,Z^{N,n}_{t_{j}})\Big{]}=0,$
(5.14)

for all $m\in\mathbb{N},0\leq t_{1}\leq...\leq t_{m}\leq s<t$ and continuous
functions $g_{j}$. To identify the limit, assume
$\big{(}J^{N},Z^{N,1},...,Z^{N,n}\big{)}\Rightarrow\big{(}\bar{J},\bar{Z}^{1},...,\bar{Z}^{n}\big{)}$
in distribution with respect to Skorohod distance. By continuity of
projection, $J^{N}\Rightarrow\bar{J}$, hence $\bar{J}=J$. In Step 1 and 2 we
have already shown the stronger convergence $J^{N}\to J$ and $I^{N}\to I$ with
respect to uniform topology. As $G^{N}f$ converges to some $Gf$,
$\theta_{N}\to 0$ and the evaluation at timepoints $t$ is continuous with
respect to uniform topology, we can pass to the limit in (5.14) and obtain
that

$\displaystyle
f_{0}(J_{t})\prod_{i=1}^{n}f_{i}(\bar{Z}^{i}_{t})-\int_{0}^{t}Gf_{0,...,n}(J_{s},\bar{Z}^{1}_{s},...,\bar{Z}^{n}_{s})ds$
(5.15)

is a martingale for any $f_{0,...,n}\in\mathcal{D}$, where

$\displaystyle Gf_{0,..,n}(J_{[0,t]},Z^{1}_{t},...,Z^{n}_{t})=$ $\displaystyle
G_{J}f_{0}(J_{[0,t]})\prod_{i=1}^{n}f_{i}(Z^{i}_{t})$
$\displaystyle+h(I_{t})f_{0}(J_{t})\sum_{j=1}^{n}\big{(}f_{j}(Z^{j}_{t}+1)-f_{j}(Z^{j}_{t})\big{)}\prod_{i\neq
j}f_{i}(Z^{i}_{t}).$

It remains to read off the distribution of $(J,\bar{Z}^{1},...,\bar{Z}^{n})$
from (5.15). Choose $f_{0}=1$ and $f_{i}(k)=\exp(-u_{i}k)$ for $u_{j}\geq 0$.
Then

$\displaystyle 0=$
$\displaystyle\mathbb{E}\Big{[}f_{0}(J_{t})\prod_{i=1}^{n}f_{i}(\bar{Z}^{i}_{t})-\int_{0}^{t}Gf_{0}...f_{n}(J_{s},\bar{Z}^{1}_{s},...,\bar{Z}^{n}_{s})ds\Big{]}$
$\displaystyle=$
$\displaystyle\mathbb{E}\big{[}\prod_{i=1}^{n}e^{-u_{i}\bar{Z}^{i}_{t}}-\int_{0}^{t}h(I_{s})\sum_{j=1}^{n}\big{(}e^{-u_{j}}-1\big{)}\prod_{i=1}^{n}e^{-u_{i}\bar{Z}^{i}_{s}}ds\big{]}$
$\displaystyle=$
$\displaystyle\mathbb{E}\Big{[}\mathbb{E}\big{[}\prod_{i=1}^{n}e^{-u_{i}\bar{Z}^{i}_{t}}|I\big{]}-\int_{0}^{t}h(I_{s})\sum_{j=1}^{n}\big{(}e^{-u_{j}}-1\big{)}\mathbb{E}\big{[}{\color[rgb]{0,0,0}\prod_{i=1}^{n}}e^{-u_{i}\bar{Z}^{i}_{s}}|I\big{]}ds\Big{]},$

where, in the last equality, we have interchanged the $ds$-integral and
conditional expectation with respect to $I$, and used the $I$-measurability of
$h(I)$. The function in brackets is nonnegative, hence

$\displaystyle\mathbb{E}\Big{[}\prod_{i=1}^{n}e^{-u_{i}\bar{Z}^{i}_{t}}|I\Big{]}=\prod_{j=1}^{n}\exp\Big{(}(e^{-u_{j}}-1)\int_{0}^{t}h(I_{s})ds\Big{)}.$

As claimed, given $I$ the processes $\bar{Z}^{i}$ are independent poisson
processes with intensity at time $t$ given by $h(I_{t})$.

###  5.4 Proof of Theorem 1

Proof of 1.:
Define

$\displaystyle J_{t}=(2p-1)\int_{0}^{t}h(I_{s})ds,\text{ such that
}I_{t}=\int_{0}^{t}\varphi(t-s)dJ_{s}.$ (5.16)

For $\theta_{N}=\tfrac{1}{N}$ in (5.9) we get that,

$\displaystyle J_{t}^{N}-J_{t}$
$\displaystyle=\frac{1}{N}\Big{(}Y^{+}\Big{(}\big{(}pN+\tfrac{1}{2}\sqrt{N}W_{N}\big{)}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-Y^{+}\Big{(}pN\int_{0}^{t}h(I_{s}^{N})ds\Big{)}\Big{)}$
$\displaystyle\quad-\frac{1}{N}\Big{(}Y^{-}\Big{(}\big{(}(1-p)N-\tfrac{1}{2}\sqrt{N}W_{N}\big{)}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-Y^{-}\Big{(}(1-p)N\int_{0}^{t}h(I_{s}^{N})ds\Big{)}\Big{)}$
$\displaystyle\qquad\qquad+\frac{1}{N}Y^{+}\Big{(}pN\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-p\int_{0}^{t}h(I_{s}^{N})ds$
$\displaystyle\qquad\qquad-\frac{1}{N}Y^{-}\Big{(}(1-p)N\int_{0}^{t}h(I_{s}^{N})ds\Big{)}+(1-p)\int_{0}^{t}h(I_{s}^{N})ds$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\qquad+(2p-1)\int_{0}^{t}h(I_{s}^{N})-h(I_{s})ds,$

so

$\displaystyle\sup_{0\leq s\leq t}(J_{s}^{N}-J_{s})^{2}$
$\displaystyle\leq\varepsilon_{N}+C\cdot\Big{(}\sup_{0\leq s\leq
t||h||}\Big{(}\frac{1}{N}Y^{+}\Big{(}pNs\Big{)}-ps\Big{)}^{2}+\Big{(}\frac{1}{N}Y^{-}\Big{(}(1-p)Ns\Big{)}-(1-p)s\Big{)}^{2}\Big{)}$
$\displaystyle\qquad\qquad\qquad+(2p-1)^{2}h_{Lip}\cdot\sup_{0\leq s\leq
t}\int_{0}^{s}(I_{r}^{N}-I_{r})^{2})ds$
$\displaystyle\qquad\qquad\qquad\qquad\qquad\qquad\leq\varepsilon_{N}+C\cdot\int_{0}^{t}\sup_{0\leq
r\leq s}(I_{r}^{N}-I_{r})^{2}ds$ (5.17)

with $\varepsilon_{N}\xrightarrow{N\to\infty}0$ (by Lemma 5.1). By the
Gronwall inequality and Lemma 5.2 we can conclude

$\displaystyle\sup_{0\leq s\leq
t}\Big{(}J_{s}^{N}-J_{s}\Big{)}^{2}\xrightarrow{N\to\infty}0\qquad\text{and}\qquad\sup_{0\leq
s\leq t}\Big{(}I_{s}^{N}-I_{s}\Big{)}^{2}\xrightarrow{N\to\infty}0.$ (5.18)

Proof of 2.:
Define $\bar{Z}^{i}_{t}=Y_{i}\big{(}\int_{0}^{t}h(I_{s})ds\big{)}$, where
$Y_{1},...,Y_{N}$ are independent Poisson process as in (5.7). Fix
$\omega\in\Omega$, such that $\sup_{0\leq s\leq
t}|I_{s}^{N}(\omega)-I_{s}|\rightarrow 0$. As $h$ is Lipschitz,
$\int_{0}^{t}h(I_{s}^{N}(\omega))ds\rightarrow\int_{0}^{t}h(I_{s})ds$, hence
for any point of continuity of $t\mapsto\bar{Z}^{i}_{t}(\omega)$ we can
conclude $Z^{i}_{t}(\omega)\rightarrow\bar{Z}^{i}_{t}(\omega)$. As the points
of continuity are dense in $[0,\infty)$, convergence in Skorohod-distance
follows from [26][Theorem 2.15 c)(ii)].

Proof of 3.:
First, strong existence and uniqueness for (3.3) follows from [3]. The proof
proceeds similarly to the proof of Theorem 2, Step 1 and 2. Wlog, we assume
that $W_{N}\to W$ almost surely and in $L^{2}$. Recall the pair $(G,K)$ from
(3.3) (for some Brownian motion $B$). Further, we define the auxiliary
processes $\widetilde{G}^{N},\widetilde{K}^{N}$ by (recall from (5.10) the
Definition of the Brownian motion $B^{N}$)

$\displaystyle\widetilde{K}^{N}_{t}$
$\displaystyle=\int_{0}^{t}\varphi(t-s)d\widetilde{G}^{N}_{s}$ (5.19)
$\displaystyle\widetilde{G}^{N}_{t}$
$\displaystyle=\int_{0}^{t}Wh(I_{s})+(2p-1)h^{\prime}(I_{s})\widetilde{K}^{N}(s)ds+\int_{0}^{t}\sqrt{h(I_{s})}dB^{N}_{s}.$

In our proof, we will first show convergence of
$\sqrt{N}(J^{N}-J)-\widetilde{G}^{N}\rightarrow 0$ and
$\sqrt{N}(I^{N}-I)-\widetilde{K}^{N}\rightarrow 0$. We then replace $B^{N}$ by
$B$ and find $\widetilde{K}^{N}\Rightarrow K$, since
$(\widetilde{K}^{N},\widetilde{G}^{N})\sim(K,G)$. The process
$\sqrt{N}(J^{N}-J)-\widetilde{G}^{N}$ is described by the following equation:

$\displaystyle\sqrt{N}(J^{N}_{t}-J_{t})-\widetilde{G}^{N}_{t}=A^{N}_{t}+C^{N}_{t}+D^{N}_{t}+(2p-1)E^{N}_{t}$
with $\displaystyle A_{t}^{N}$
$\displaystyle=\frac{1}{\sqrt{N}}Y^{+}\Big{(}\Big{(}pN+\frac{\sqrt{N}W_{N}}{2}\Big{)}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-\frac{1}{\sqrt{N}}Y^{-}\Big{(}\Big{(}(1-p)N-\frac{\sqrt{N}W_{N}}{2}\Big{)}\int_{0}^{t}h(I_{s}^{N})ds\Big{)}$
$\displaystyle\qquad\qquad\qquad\qquad-\sqrt{N}(2p-1)\int_{0}^{t}h(I_{s}^{N})ds-
W_{N}\int_{0}^{t}h(I_{s}^{N})ds-\hat{B}\Big{(}\int_{0}^{t}h(I_{s}^{N})ds\Big{)},$
$\displaystyle C_{t}^{N}$
$\displaystyle=W_{N}\int_{0}^{t}h(I_{s}^{N})ds-W\int_{0}^{t}h({I}_{s})ds,$
$\displaystyle D_{t}^{N}$
$\displaystyle=\int_{0}^{t}\sqrt{h(I_{s}^{N})}dB^{N}_{s}-\int_{0}^{t}\sqrt{h(I_{s})}dB^{N}_{s},$
$\displaystyle E_{t}^{N}$
$\displaystyle=\int_{0}^{t}\sqrt{N}\big{(}h(I^{N}_{s})-h(I_{s})\big{)}-h^{\prime}(I_{s})\widetilde{K}^{N}_{s}ds.$

As in the proof of Theorem 2, Step 1, we can show that $C^{N},D^{N}\to 0$ (as
we already know that $I^{N}\to I$ and $W_{N}\rightarrow W$) uniformly on
compact time intervals in $L^{2}$. The uniform $L^{2}$-convergence $A^{N}\to
0$ follows from Lemma 5.1. For $E^{N}$ we obtain

$\displaystyle
E^{N}_{t}=\int_{0}^{t}\big{(}\sqrt{N}(I^{N}_{s}-I_{s})-\widetilde{K}^{N}_{s}\big{)}h^{\prime}(I^{N}_{s})+\sqrt{N}R_{1}h(I^{N}_{s},I_{s})ds,$

where $R_{1}h(I^{N}_{s},I_{s})$ is the first order remainder in Taylor’s
formula. Using the Peano form of the remainder we obtain for some
$\xi^{N}_{s}$ between $I^{N}_{s}$ and $I_{s}$

$\displaystyle\Big{(}\sqrt{N}R_{1}h(I^{N}_{s},I_{s})\Big{)}^{2}$
$\displaystyle=N\Big{(}(h^{\prime}(\xi^{N}_{s})-h^{\prime}(I_{s}))^{2}(I^{N}_{s}-I_{s})^{2}\Big{)}$
$\displaystyle\leq(h^{\prime}_{Lip})^{2}N(I^{N}_{s}-I_{s})^{4}.$

Therefore

$\displaystyle\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\big{(}E^{N}_{s}\big{)}^{2}\Big{]}$ $\displaystyle\leq
2\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\Big{(}\int_{0}^{s}\big{(}\sqrt{N}(I^{N}_{u}-I_{u})-\widetilde{K}^{N}_{u}\big{)}h^{\prime}(I^{N}_{u})du\Big{)}^{2}\Big{]}$
$\displaystyle\qquad\qquad+2\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\Big{(}\int_{0}^{t}\big{(}\sqrt{N}R_{1}h(I^{N}_{u},I_{u})\big{)}du\Big{)}^{2}\Big{]}$
$\displaystyle\leq
2t||h^{\prime}||^{2}\int_{0}^{t}\mathbb{E}\Big{[}\big{(}\sqrt{N}(I^{N}_{s}-I_{s})-\widetilde{K}^{N}_{s}\big{)}^{2}\Big{]}ds$
$\displaystyle\qquad\qquad+2t^{2}(h^{\prime}_{Lip})^{2}\mathbb{E}\Big{[}\sup_{0\leq
s\leq t}N(I^{N}_{s}-I_{s})^{4}\Big{]}.$

We can show convergence of the second term similarly to the proof of 1., as

$\displaystyle N\cdot\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\Big{(}\frac{1}{N}(Y^{\pm}(Ns)-Ns)\Big{)}^{4}\Big{]}$
$\displaystyle\leq\Big{(}\frac{4}{3}\Big{)}^{4}\frac{1}{N^{3}}\mathbb{E}\Big{[}\Big{(}(Y^{\pm}(Nt)-Nt)\Big{)}^{4}\Big{]}$
$\displaystyle=\Big{(}\frac{4}{3}\Big{)}^{4}\Big{(}\frac{3t^{2}}{N}+\frac{t}{N^{2}}\Big{)}\xrightarrow{N\to\infty}0.$

For the first term we apply Lemma 5.2 in order to obtain

$\displaystyle\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\Big{(}\big{(}\sqrt{N}(I^{N}_{s}-I_{s})-\widetilde{K}^{N}_{s}\big{)}\Big{)}^{2}\Big{]}\leq
C\cdot\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\Big{(}\big{(}\sqrt{N}(J^{N}_{s}-J_{s})-\widetilde{G}^{N}_{s}\big{)}\Big{)}^{2}\Big{]}.$

Summing up the results on $A^{N},C^{N},D^{N}$ and $E^{N}$ we conclude

$\displaystyle\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\big{(}\sqrt{N}(J^{N}_{s}-J_{s})-\widetilde{G}^{N}_{s}\big{)}^{2}\Big{]}\leq\varepsilon_{N}+C\int_{0}^{t}\mathbb{E}\Big{[}\sup_{0\leq
u\leq
s}\big{(}\sqrt{N}(J^{N}_{u}-J_{u})-\widetilde{G}^{N}_{u}\big{)}^{2}\Big{]}ds$

for $\varepsilon_{N}\to 0$ and by the Gronwall inequality

$\displaystyle\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\big{(}\sqrt{N}(J^{N}_{s}-J_{s})-\widetilde{G}^{N}_{s}\big{)}^{2}\Big{]}\xrightarrow{N\to\infty}0.$

Applying Lemma 5.2 it follows that

$\displaystyle\mathbb{E}\Big{[}\sup_{0\leq s\leq
t}\big{(}\sqrt{N}(I^{N}_{s}-I_{s})-\widetilde{K}^{N}_{s}\big{)}^{2}\Big{]}\xrightarrow{N\to\infty}0.$

###  5.5 Proof of Corollaries 3.2 and 3.5

###### Proof of Corollary 3.2.

Recall $Y^{+}$ and $Y^{-}$ from Section 5.2, and which are used frequently in
the proof of Theorem 1. For (3.4), using (5.7) and Lemma 5.1, for the Poisson
process $Y=(Y(t))_{t\geq 0}$, given by $Y(t)=Y^{+}(pt)+Y^{-}((1-p)t)$,

$\displaystyle\sup_{0\leq t\leq T}$
$\displaystyle\Big{|}\frac{1}{N}\sum_{j=1}^{N}\Big{(}Z_{t}^{N,j}-\int_{0}^{t}h(I_{s}^{N})ds\Big{)}\Big{|}=\sup_{0\leq
t\leq
T}\Big{|}\frac{1}{N}Y\Big{(}N\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-\int_{0}^{t}h(I_{s}^{N})ds\Big{|}$
(5.20) $\displaystyle\leq\sup_{0\leq t\leq
T||h||}\Big{|}\frac{1}{N}(Y(Nt)-Nt)\Big{|}=0.$

For (3.5), by the law of large numbers,
$\frac{1}{N}\sum_{j=1}^{N}\bar{Z}^{j}_{T}\xrightarrow{N\to\infty}\int_{0}^{T}h(I_{t})dt$,
as well as $h(I^{N})\xrightarrow{N\to\infty}h(I)$ in $L^{2}$, uniformly on
compact time intervals by Theorem 1.1, therefore

$\displaystyle\frac{1}{N}\sum_{j=1}^{N}\Big{(}Z^{N,j}-\bar{Z}^{j}\Big{)}$
$\displaystyle=\frac{1}{N}\sum_{j=1}^{N}\Big{(}Z^{N,j}-\int_{0}^{.}h(I_{s}^{N})ds\Big{)}$
$\displaystyle\qquad\qquad-\frac{1}{N}\sum_{j=1}^{N}\Big{(}\bar{Z}^{j}-\int_{0}^{.}h(I_{s})ds\Big{)}+\int_{0}^{.}(h(I_{s}^{N})-h(I_{s}))ds\xrightarrow{N\to\infty}0.$

For (3.6), we write, with the same Poisson process as in (5.20), and with the
Brownian motions $B^{\prime}$ arising as limit of the compensated Poisson
process $Y$, and some $\widetilde{B}$,

$\displaystyle\frac{1}{\sqrt{N}}$
$\displaystyle\sum_{j=1}^{N}\Big{(}Z^{N,j}_{t}-\int_{0}^{t}h(I_{s}^{N})ds\Big{)}=\frac{1}{\sqrt{N}}\Big{(}Y\Big{(}N\int_{0}^{t}h(I_{s}^{N})ds\Big{)}-N\int_{0}^{t}h(I_{s}^{N})ds\Big{)}$
(5.21)
$\displaystyle=B^{\prime}\Big{(}\int_{0}^{t}h(I_{s})ds\Big{)}+o(1)=\int_{0}^{t}\sqrt{h(I_{s})}d\widetilde{B}_{s}+o(1).$

Next, for (3.7), with $K$ as in Theorem 1.3,

$\displaystyle\sqrt{N}(h(I_{t}^{N})-h(I_{t}))\Big{)}=\sqrt{N}h^{\prime}(I_{t})(I_{t}^{N}-I_{t})+o(1)=h^{\prime}(I_{t})K_{t}+o(1).$

Last, we note that the left hand side of (3.8) is a sum of the left hand sides
of (3.6) and (3.7). We obtain the result by summing the limits of these
equations, once we determine the correlation structure of the martingales
$\int_{0}^{.}\sqrt{h(I_{s})}dB_{s}$ in (3.3) and
$\int_{0}^{.}\sqrt{h(I_{s})}d\widetilde{B}_{s}$ in (3.6) on a joint
probability space. We will show that joint weak convergence of (3.6) and (3.7)
holds if

$\displaystyle\mathbb{E}[\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}\int_{0}^{t}\sqrt{h(I_{s})}d\widetilde{B}_{s}]=(2p-1)\int_{0}^{t}h(I_{s})ds,$
(5.22)

implying the result. Recall from Section 5.2 the limits $B^{+}$ and $B^{-}$ of
the rescaled Poisson processes $Y^{+}$ and $Y^{-}$. By the construction of $Y$
above (5.20), we find that $B^{\prime}_{t}=B^{+}_{pt}+B^{-}_{(1-p)t}$ is the
limit of the rescaled Poisson process $Y$. In addition,
$\hat{B}_{t}=B^{+}_{pt}-B^{-}_{(1-p)t}$ arises above (5.10). Clearly, by
independence of $B^{+}$ and $B^{-}$, we have
$\mathbb{E}[\hat{B}_{t}B_{t}^{\prime}]=\mathbb{E}[(B_{pt}^{+}+B_{(1-p)t}^{-})(B_{pt}^{+}-B_{(1-p)t}^{-})]=(2p-1)t$.

From the proof of Theorem 1.3, we know that the convergence in this result is
even in probability if we exchange $K$ by $\widetilde{K}^{N}$ from (5.19).
Hence, using $I^{N}\to I$, we find for the integral appearing in (3.3), on a
joint probability space, and taking (5.10) into account,

$\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}=\lim_{N\to\infty}\int_{0}^{t}\sqrt{h(I^{N}_{s})}dB^{N}_{s}=\lim_{N\to\infty}\hat{B}\Big{(}\int_{0}^{t}h(I^{N}_{s})ds\Big{)}=\hat{B}\Big{(}\int_{0}^{t}h(I_{s})ds\Big{)}.$

From this and (5.21), we can thus write

$\displaystyle\mathbb{E}\Big{[}\int_{0}^{t}\sqrt{h(I_{s})}dB_{s}\int_{0}^{t}\sqrt{h(I_{s})}d\widetilde{B}_{s}\Big{]}$
$\displaystyle=\mathbb{E}\Big{[}\hat{B}\Big{(}\int_{0}^{t}h(I_{s})ds\Big{)}\cdot
B^{\prime}\Big{(}\int_{0}^{t}h(I_{s})ds\Big{)}\Big{]}$
$\displaystyle=(2p-1)\int_{0}^{t}h(I_{s})ds$

and we are done. ∎

###### Proof of Corollary 3.5.

Actually, the proof of (3.4), (3.5) and (3.6) is literally the same as in the
proof above. ∎

## References

  * Ahmadian and Miller [2021] Ahmadian, Y. and K. D. Miller (2021, Aug).  What is the dynamical regime of cerebral cortex?  Neuron.
  * Barral and Reyes [2016] Barral, J. and A. D. Reyes (2016).  Synaptic scaling rule preserves excitatory–inhibitory balance and salient neuronal network dynamics.  Nature Neuroscience 19(12), 1690–1696.
  * Berger and Mizel [1980] Berger, M. A. and V. J. Mizel (1980).  Volterra equations with Itô integrals—I.  The Journal of Integral Equations 2(3), 187–245.
  * Brémaud and Massoulié [1996] Brémaud, P. and L. Massoulié (1996, 02).  Stability of Nonlinear Hawkes Processes.  Ann. Appl. Probab. 24(3), 1563–1588.
  * Brunel [2000] Brunel, N. (2000).  Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons.  J Comput Neurosci 8(3), 183–208.
  * Chen et al. [2019] Chen, S., A. Shojaie, E. Shea-Brown, and D. Witten (2019).  The Multivariate Hawkes Process in High Dimensions: Beyond Mutual Excitation.  https://arxiv.org/abs/1707.04928.
  * Chevallier [2017a] Chevallier, J. (2017a).  Fluctuations for mean-field interacting age-dependent Hawkes processes.  Elec. J. Probab. 22(42), 1–49.
  * Chevallier [2017b] Chevallier, J. (2017b).  Mean-field limit of generalized Hawkes processes.  Stoch. Proc. Appl. 127, 3870–3912.
  * Chevallier et al. [2019] Chevallier, J., A. Duarte, E. Löcherbach, and G. Ost (2019).  Mean field limits for nonlinear spatially extended Hawkes processes with exponential memory kernels.  Stoch. Proc. Appl. 129, 1–27.
  * Chevallier and Ost [2020] Chevallier, J. and G. Ost (2020).  Fluctuations for spatially extended Hawkes processes.  Stoch. Proc. Appl. 130, 5510–5542.
  * Costa et al. [2020] Costa, M., C. Graham, L. Marsalle, and V. C. Tran (2020).  Renewal in Hawkes processes with self-excitation and inhibition.  Adv. Appl. Probab. 52, 879–915.
  * Delattre et al. [2016] Delattre, S., N. Fournier, and M. Hoffmann (2016).  Hawkes processes on large networks.  Ann. Appl. Probab. 26(1), 216–261.
  * Ditlevsen and Löcherbach [2017] Ditlevsen, S. and E. Löcherbach (2017).  Multi-class oscillating systems of interacting neurons.  Stoch. Proc. Appl. 127, 1840–1869.
  * E. Bacry and Muzy [2015] E. Bacry, I. M. and J.-F. Muzy (2015).  Hawkes processes in finance.  https://arxiv.org/pdf/1502.04592;Hawkes.
  * Erny et al. [2020] Erny, X., E. Löcherbach, and D. Loukianova (2020).  Mean field limits for interacting hawkes processes in a diffusive regime.  https://arxiv.org/abs/1904.06985.
  * Ethier and Kurtz [1986] Ethier, S. and T. Kurtz (1986).  Markov Processes. Characterization and Convergence.  John Wiley, New York.
  * Fox et al. [2016] Fox, E., F. Schoenberg, and J. Gordon (2016).  Spatially inhomogeneous background rate estimators and uncertainty quantification for nonparametric Hawkes point process models of earthquake occurrences.  Ann. Appl. Stat. 10, 1725–1756.
  * Gerhard et al. [2017] Gerhard, F., M. Deger, and W. Truccolo (2017, 02).  On the stability and dynamics of stochastic spiking neuron models: Nonlinear Hawkes process and point process GLMs.  PLoS Comput Biol 13(2), e1005390.
  * Hawkes [1971a] Hawkes, A. (1971a).  Spectra of some self-exciting and mutually exciting point processes.  Biometrika 58, 83–90.
  * Hawkes [2018] Hawkes, A. (2018).  Hawkes processes and their applications to finance: a review.  Quantitative Finance 18(2), 193–198.
  * Hawkes and Oakes [1974] Hawkes, A. and D. Oakes (1974).  A Cluster Process Representation of a Self-Exciting Process.  J. Appl. Probab. 11(3), 493–503.
  * Hawkes [1971b] Hawkes, A. G. (1971b).  Spectra of some self-exciting and mutually exciting point processes.  Biometrika 58(1), 83–90.
  * Heesen and Stannat [2021] Heesen, S. and W. Stannat (2021).  Fluctuation limits for mean-field interacting nonlinear Hawkes processes.  Stoch. Proc. Appl. 139, 280–297.
  * Herculano-Houzel [2009] Herculano-Houzel, S. (2009).  The human brain in numbers: a linearly scaled-up primate brain.  Frontiers in Human Neuroscience 3, 31.  https://doi.org/10.3389/neuro.09.031.2009.
  * Herz et al. [2006] Herz, A. V. M., T. Gollisch, C. K. Machens, and D. Jaeger (2006).  Modeling single-neuron dynamics and computations: a balance of detail and abstraction.  Science 314(5796), 80–85.
  * Jacod and Shiryaev [2003] Jacod, J. and A. Shiryaev (2003).  Limit theorems for stochastic processes, Volume 288.  Springer Science & Business Media.
  * Jovanovic et al. [2015] Jovanovic, S., J. Hertz, and S. Rotter (2015, Apr).  Cumulants of Hawkes point processes.  Phys Rev E Stat Nonlin Soft Matter Phys 91(4), 042802\.
  * Jovanović et al. [2015] Jovanović, S., J. Hertz, and S. Rotter (2015).  Cumulants of Hawkes point processes.  Physical Review E 91(4), 042802.
  * Kallenberg [2002] Kallenberg, O. (2002).  Foundations of Modern Probability. 2nd ed. Probability and Its Applications. New York, NY: Springer.
  * Kandel et al. [2012] Kandel, E. R., J. H. Schwartz, T. M. Jessell, S. A. Siegelbaum, and A. J. Hudspeth (2012).  Principles of Neural Science (5th ed.).  New York: McGraw-Hill.
  * Lukasik et al. [2016] Lukasik, M., P. K. Srijith, D. Vu, K. Bontcheva, A. Zubiaga, and T. Cohn (2016, August).  Hawkes processes for continuous time sequence classification: an application to rumour stance classification in Twitter.  In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Berlin, Germany, pp. 393–398. Association for Computational Linguistics.
  * Massoulié [1998] Massoulié, L. (1998).  Stability results for a general class of interacting point processes dynamics, and applications.  Stoch. Proc. Appl. 75, 1–30.
  * Ogata [1988] Ogata, Y. (1988).  Statistical models for earthquake occurrences and residual analysis for point processes.  J. Amer. Statist. Assoc. 83, 9–27.
  * Paninski et al. [2004] Paninski, L., J. W. Pillow, and E. P. Simoncelli (2004).  Maximum likelihood estimation of a stochastic integrate-and-fire neural encoding model.  Neural computation 16(12), 2533–2561.
  * Pernice et al. [2011] Pernice, V., B. Staude, S. Cardanobile, and S. Rotter (2011).  How structure determines correlations in neuronal networks.  PLoS Comput Biol 7(5), e1002059.
  * Pra [2017] Pra, P. D. (2017).  Stochastic mean-field dynamics and applications to life sciences.  In Giambattista Giacomin, Stefano Olla, Ellen Saada, Herbert Spohn, and Gabriel Stoltz, editors, Stochastic Dynamics Out of Equilibrium, pp. 3–27. Springer.
  * Vreeswijk and Sompolinsky [1998] Vreeswijk, C. v. and H. Sompolinsky (1998).  Chaotic balanced state in a model of cortical circuits.  Neural Computation 10(6), 1321–1371.
  * Yizhar et al. [2011] Yizhar, O., L. E. Fenno, M. Prigge, F. Schneider, T. J. Davidson, D. J. O’shea, V. S. Sohal, I. Goshen, J. Finkelstein, J. T. Paz, et al. (2011).  Neocortical excitation/inhibition balance in information processing and social dysfunction.  Nature 477(7363), 171–178.
  * Zhu [2013] Zhu, L. (2013).  Central Limit Theorem for Nonlinear Hawkes Processes.  J. Appl. Probab. 50, 760–771.
  * Zhu [2015] Zhu, L. (2015).  Large deviations for Markovian nonlinear Hawkes processes.  Ann. Appl. Probab. 25, 548–581.
  * Ziplin et al. [2016] Ziplin, J., F. Schoenberg, K. Coronges, and A. Bertozzi (2016).  Point-process models of social network interactions: Parameter estimation and missing data recovery.  Euro. Jnl of Applied Mathematics 27, 502–529.