# Stochastic reaction networks within interacting compartments

David F. Anderson Department of Mathematics, University of Wisconsin-Madison,
<EMAIL_ADDRESS>Aidan S. Howells Department of Mathematics, University
of Wisconsin-Madison<EMAIL_ADDRESS>Corresponding author.

###### Abstract

Stochastic reaction networks, which are usually modeled as continuous-time
Markov chains on $\mathbb{Z}^{d}_{\geq 0}$, and simulated via a version of the
“Gillespie algorithm,” have proven to be a useful tool for the understanding
of processes, chemical and otherwise, in homogeneous environments. There are
multiple avenues for generalizing away from the assumption that the
environment is homogeneous, with the proper modeling choice dependent upon the
context of the problem being considered. One such generalization was recently
introduced in [11], where the proposed model includes a varying number of
interacting compartments, or cells, each of which contains an evolving copy of
the stochastic reaction system. The novelty of the model is that these
compartments also interact via the merging of two compartments (including
their contents), the splitting of one compartment into two, and the appearance
and destruction of compartments. In this paper we begin a systematic
exploration of the mathematical properties of this model. We (i) obtain
basic/foundational results pertaining to explosivity, transience, recurrence,
and positive recurrence of the model, (ii) explore a number of examples
demonstrating some possible non-intuitive behaviors of the model, and (iii)
identify the limiting distribution of the model in a special case that
generalizes three formulas from an example in [11].

##  1 Introduction

Stochastic reaction networks are now commonly utilized to model various types
of systems in the biological sciences. These mathematical models are often
continuous-time Markov chains and are used when the counts of at least some of
the underlying “species,” which are most commonly different molecule types,
are low. In this low copy-number case, the state of the model is a vector
giving the integer counts of the different species and transitions are
governed by the different possible “reactions” that can take place. These
models are typically simulated via the Gillespie algorithm [15, 16] or the
next reaction method [14, 2]. See [5], and references therein, for more on
this type of model.

One potential drawback to the standard model is that it assumes a homogeneous
environment. There are multiple ways to generalize, however. One common
generalization is to split the state space itself into differed fixed pieces
(often called “voxels”) and then allow for transitions between adjacent voxels
[17, 24]. Thinking of the size of the voxels going to zero leads naturally to
a model with continuous space in which the state of the system is given by the
type, position, velocity, etc. of each particle in the system. A reaction can
then only take place when the necessary constituent molecules are near each
other (with the precise mechanism for defining when they are “near enough”
left to the modeler). One of the first examples of such a continuous space
model was introduced by Doi [10]. More generally, there are a whole class of
continuous space models known as reaction-diffusion models. For a brief
overview of such models, see [12]. For a comparison of two specific such
models, with an approachable introduction, see [1]; for a more general
approach, see the introduction of [9].

A different approach to generalize from the homogeneous case is to imagine
some fixed collection of compartments and model the dynamics within each
compartment in the usual way (as a continuous-time Markov chain as described
in the first paragraph above) while also allowing for interactions between
adjacent compartments. This is the approach taken in [21] in an ecological
context (their “patches” are our “compartments”). However, ideally one might
like to also account for situations like in biological tissue, where reactions
take place in cells that are not static but, for example, can appear, divide,
possibly merge, or even be destroyed. That is the approach presented in a
recent paper by Duso and Zechner, where they developed a Markov model for
stochastic reaction networks within interacting compartments [11]. In
particular, their model consists of two basic components:

  1. 1.

a stochastic model of a chemical reaction network;

  2. 2.

a dynamic model of compartments, or cells, which themselves undergo basic
transitions such as (i) arrivals, (ii) departures, (iii) mergers, and (iv)
divisions. In the context of [11], these four transition types are referred to
as inflows, exits, coagulations, and fragmentations, respectively.

Each compartment, or cell, contains a copy of the (evolving) chemical reaction
network. When two cells merge, their contents are combined. When a cell
divides, its contents are randomly split among the two new daughter cells.
Beyond the framework itself, their paper focuses on the framework’s practical
use, using moment closure methods to derive estimates for various population
statistics which are then validated by simulation. They also derive stationary
distributions for some special cases.

In the present paper, we attempt to lay the groundwork for exploration of
mathematical questions about the Markov chain model developed in [11]. We
focus on the special case where the compartments can only enter, leave, merge,
and divide, all according to mass action kinetics and unaffected by their
contents. Questions pertaining to recurrence, transience, and explosivity are
all considered. We show that in most, but not all, parameter regimes the
overall qualitative behavior of the model (i.e., recurrence, transience, or
explosivity) is the same as that of the associated stochastic reaction
network. We also analyze myriad examples that, taken together, demonstrate
some of the non-intuitive (and interesting) possible behaviors of the model.
Moreover, we derive the stationary distribution for the model in the case
where the chemistry inside the compartments is well understood in the sense
that a formula for the distribution is known for all time (e.g., the DR models
of [6]) and the compartments themselves are not allowed to interact (but are
not totally static, being allowed to enter and leave the system). Two special
cases of this stationary distribution are provided as illustration, both of
which generalize formulas from an example in [11].

Before moving on, we warn the reader that in the field of epidemiology, the
term “compartment model” has a different meaning. There the compartments are
what we would call species. For example, they would speak of an SIR model as
dividing individuals into a susceptible compartment, an infected compartment,
and a recovered compartment. See e.g. [8].

A standard knowledge of continuous-time Markov chains is assumed. See for
example Norris [23] for a detailed introduction to the topic. For notational
convenience, we will use the following shorthand notations: for any two
vectors $v,w\in\mathbb{R}^{d}_{\geq 0}$ and any vector
$x,y\in\mathbb{Z}^{d}_{\geq 0}$ we denote

$\displaystyle v^{w}=\prod_{i=1}^{d}(v_{i})^{w_{i}}\quad\text{and}\quad
x!=\prod_{i=1}^{d}(x_{i})!\quad\text{and}\quad\binom{x}{y}=\prod_{i=1}^{d}\binom{x_{i}}{y_{i}},$

with the conventions that $0^{0}=1$ and that $\binom{x}{y}=0$ for $y<0$ or
$y>x$. Moreover, we will always use $d$ to represent the number of species in
the model. Finally, for $x\in\mathbb{Z}^{d}_{\geq 0}$ we define
$e_{x}:\mathbb{Z}^{d}_{\geq 0}\to\mathbb{Z}$ to be the function taking the
value of one at $x$ and zero otherwise.

The remainder of the paper is outlined as follows. In section 2, we fully
specify the model. Further, we give two different mathematical representations
that are both useful and prove some first basic properties. In the brief
section 3, we prove that the full model is explosive if and only if the
associated reaction network is. In section 4, we give conditions for when the
full model is recurrent, positive recurrent, or transient. Finally, in section
5, we provide the stationary distribution for a special class of models.

##  2 The reaction network within interacting compartments (RNIC) model

As discussed in the introduction, the full model we consider here consists of
two sub-models: (i) a stochastic reaction network and (ii) a dynamic model of
compartments, or cells, each of which contains an evolving copy of the
stochastic reaction network. We first describe these sub-models individually
and then specify how they are combined to make the full model.

###  2.1 Stochastic reaction networks

Suppose we have a finite set $\mathcal{S}$, whose elements we shall call
_species_ , and a directed graph whose vertices are unique linear combinations
of species with non-negative integer coefficients. The edges of the graph are
called _reactions_ ; let $\mathcal{R}$ denote the set of reactions. The linear
combinations which appear as vertices in the graph are called _complexes_ ;
the set of complexes will be denoted $\mathcal{C}$. A _chemical reaction
network_ (or just _reaction network_ ; _CRN_ for short) is the tuple
$\mathcal{I}=(\mathcal{S},\mathcal{C},\mathcal{R})$, where $\mathcal{S}$,
$\mathcal{C}$ and $\mathcal{R}$ are as above. See Figure 1 for an example
reaction network.

${A+B}$${0}$${B}$${A+2B}$${A}$${2B}$ Figure 1:  The CRN with species $A$ and
$B$ and reactions $A+B\to 0$, $0\to B$, $B\to 0$, $2B\to 0$, and $A+2B\to A$.
Note that $0$ here denotes the linear combination $0A+0B$.

When talking about specific reaction networks, the species will usually be
represented by capital Latin letters. When talking generally, there will be
$d$ species $S_{1},\dots,S_{d}$. In this case we will identify
$\mathbb{Z}^{d}$ with the space of linear combinations of species with integer
coefficients. That is, we naturally identify $\nu\in\mathcal{C}$ with the
vector in $\mathbb{Z}^{d}$ whose $i$th element is the coefficient of $S_{i}$
in $\nu$. We will speak of reactions $\nu\to\nu^{\prime}\in\mathcal{R}$, or
sometimes, when we wish to enumerate the reactions as
$\\{\nu_{r}\to\nu_{r}^{\prime}\\}$, we will simply write $r\in\mathcal{R}$.

There are multiple ways to associate a mathematical model to a given reaction
network, including the use of a deterministic ODE [26], a diffusion process
[20, 4], and a continuous-time Markov chain [5]. The only one of concern to us
here is the continuous-time Markov chain model with stochastic mass-action
kinetics, in which the state of the system is a vector giving the number of
each species present and transitions are determined by the reactions. To fully
specify the model, positive (or sometimes, merely non-negative) numbers,
called rate constants, are assigned to each reaction. If the reaction
$\nu\to\nu^{\prime}$ has rate constant $\kappa$, then in state $x$ that
particular reaction occurs with rate $\kappa\binom{x}{\nu}$ and when it occurs
the chain transitions to state $x+\nu^{\prime}-\nu$. So the reactions will
happen with rate proportional to the number of ways the chemicals can combine
to allow them to happen, and $\kappa$ is the constant of proportionality. If
$\mathcal{K}$ is a set of rate constants, one for each reaction, we denote by
$\mathcal{I}_{\mathcal{K}}=(\mathcal{S},\mathcal{C},\mathcal{R},\mathcal{K})$
the corresponding stochastic mass-action system. If we let
$\kappa_{\nu\to\nu^{\prime}}$ be the rate constant for the reaction
$\nu\to\nu^{\prime}$, then the Markov chain transitions from state
$x\in\mathbb{Z}^{d}_{\geq 0}$ to state $y\in\mathbb{Z}^{d}_{\geq 0}$ with rate

$\displaystyle
q(x,y)=\mathop{\sum_{\nu\to\nu^{\prime}\in\mathcal{R}}}_{\nu^{\prime}-\nu=y-x}\kappa_{\nu\to\nu^{\prime}}\binom{x}{\nu}=\mathop{\sum_{\nu\to\nu^{\prime}\in\mathcal{R}}}_{\nu^{\prime}-\nu=y-x}\kappa_{\nu\to\nu^{\prime}}\prod_{j=1}^{d}\binom{x_{j}}{\nu_{j}}$
(1)

where the sum is over those reactions for which $\nu^{\prime}-\nu=y-x$. For
$r=\nu_{r}\to\nu^{\prime}_{r}\in\mathcal{R}$, we denote the rate of the
reaction $r$ in state $x\in\mathbb{Z}_{\geq 0}^{d}$ by $\lambda_{r}(x)$:

$\displaystyle\lambda_{r}(x)=\kappa_{r}\binom{x}{\nu_{r}}$ (2)

Note that $\lambda_{\nu\to\nu^{\prime}}(x)=0$ if $x_{i}<\nu_{i}$ for some $i$,
since $\binom{m}{k}=0$ for $k>m$. Note also that not all authors take the same
conventions as we do here. In fact, the convention we use here pertaining to
our rate constants is more in line with the biology literature [28]. In the
mathematical literature it is more common to use a falling factorial
$\lambda_{\nu\to\nu^{\prime}}(x)=\kappa_{\nu\to\nu^{\prime}}\prod_{j}(x_{j})(x_{j}-1)\cdots(x_{j}-\nu_{j}+1)=\kappa_{\nu\to\nu^{\prime}}\frac{x!}{(x-\nu)!}$,
at the cost that their rate constant $\kappa$ is no longer the constant of
proportionality when the reaction takes multiple inputs [5]. This choice plays
no fundamental role in our results, but makes certain expressions cleaner in
the present context.

We note here that many of the results found in this paper can be generalized
to systems with kinetics, i.e., rate functions $\lambda_{r}$, other than mass-
action. See Remark 4.7.

Put more succinctly, we have a Markov process on $\mathbb{Z}^{d}_{\geq 0}$
with infinitesimal generator

$\mathcal{L}f(x)=\sum_{r\in\mathcal{R}}\lambda_{r}(x)(f(x+\nu_{r}^{\prime}-\nu_{r})-f(x)),$

where $\lambda_{r}$ is determined via (2), and the above is valid for all
functions $f$ that are compactly supported [13]. The Kolmogorov forward
equation, often called the chemical master equation in the context of reaction
networks, is then

$\displaystyle\frac{d}{dt}P_{\mu}(x,t)=\sum_{r\in\mathcal{R}}\lambda_{r}(x-(\nu_{r}^{\prime}-\nu_{r}))P_{\mu}(x-(\nu_{r}^{\prime}-\nu_{r}),t)-\sum_{r\in\mathcal{R}}\lambda_{r}(x)P_{\mu}(x,t),$

where $P_{\mu}(x,t)=P_{\mu}(X(t)=x)$ is the probability the process $X$ is in
state $x\in\mathbb{Z}^{d}_{\geq 0}$ at time $t$, given an initial distribution
of $\mu$. We take the convention that $P_{\mu}(x,t)=0$ for
$x\notin\mathbb{Z}^{d}_{\geq 0}$.

One way to represent the solution to the stochastic model described above is
via a representation developed and popularized by Thomas Kurtz. Let
$\\{Y_{r}\\}_{r\in\mathcal{R}}$ be a collection of independent, unit-rate
Poisson processes, one for each possible reaction, and let $X(t),t\geq 0$, be
the solution to

$\displaystyle
X(t)=X(0)+\sum_{r\in\mathcal{R}}Y_{r}\left(\int_{0}^{t}\lambda_{r}(X(s))ds\right)(\nu_{r}^{\prime}-\nu_{r}),$
(3)

then $X$ is a continuous-time Markov chain that satisfies the conditions of
the model specified above [5, 13, 19].

######  Example 2.1:

Suppose we assign rate constants to the example CRN in Figure 1 as follows:

${A+B}$${0}$${B}$${A+2B}$${A}$${2B}$$\scriptstyle{10}$$\scriptstyle{2}$$\scriptstyle{\kappa}$$\scriptstyle{8}$$\scriptstyle{6}$
(4)

Let $x=(a,b)\in\mathbb{Z}_{\geq 0}^{2}$ denote an arbitrary state of the
system. For the particular choice of rate constants given above the positive
transition rates $q((a,b),\cdot)$, for $a,b\in\mathbb{Z}_{\geq 0}$, are

Reaction(s) |  |  | Transition |  |  | Rate
---|---|---|---|---|---|---
$A+B\to 0$ |  |  | $(a,b)\mapsto(a-1,b-1)$ |  |  | $10ab$
$0\to B$ |  |  | $(a,b)\mapsto(a,b+1)$ |  |  | $2$
$2B\to 0$ and $A+2B\to A$ |  |  | $(a,b)\mapsto(a,b-2)$ |  |  | $\displaystyle 6\frac{b(b-1)}{2}+8a\frac{b(b-1)}{2}$
$B\to 0$ |  |  | $(a,b)\mapsto(a,b-1)$ |  |  | $\kappa b$

We chose to write $6\frac{b(b-1)}{2}+8a\frac{b(b-1)}{2}$ instead of
$3b(b-1)+4ab(b-1)$ to emphasize our choice of intensity functions. Note that
all other rates, such as $q((a,b),(a+1,b))$ or $q((a,b),(a+12,b-3))$, are
zero. $\triangle$

###  2.2 Compartment model

Having fully specified our CRN,
$\mathcal{I}_{\mathcal{K}}=(\mathcal{S},\mathcal{C},\mathcal{R},\mathcal{K})$,
we turn to our next sub-model: the compartment model. As mentioned in the
introduction, we will assume that compartments, or cells, can arrive, depart,
merge, and divide. We can use the notation of chemical reaction networks to
describe the four possibilities visually via a reaction network,

${0}$${C}$${2C}$

with $0\to C$ representing arrivals, $C\to 0$ representing departures, $C\to
2C$ representing division, or fragmentation, and $2C\to C$ representing
mergers, or coagulations. Moreover, we assume that the stochastic model
tracking the number of compartments behaves as a standard stochastic reaction
network as already described in the previous section (however, see Remark 4.7
for an allowable generalization to the choice of kinetics). We will term this
reaction network the compartment network, and denote it by
$\mathcal{H}=(\mathcal{S}_{\text{comp}},\mathcal{C}_{\text{comp}},\mathcal{R}_{\text{comp}})$.
Note that $\mathcal{S}_{\text{comp}}=\\{C\\}$ and $\mathcal{C}_{\text{comp}}$
is a subset of $\\{0,C,2C\\}$ (depending on which rate constants are non-
zero). If rate constants are added as follows,

${0}$${C}$${2C}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$$\scriptstyle{\kappa_{F}}$$\scriptstyle{\kappa_{C}}$

where each $\kappa_{E},\kappa_{I},\kappa_{C},\kappa_{F}\geq 0$, then we will
denote the corresponding stochastic mass-action system by
$\mathcal{H}_{\mathcal{K}}=(\mathcal{S}_{\text{comp}},\mathcal{C}_{\text{comp}},\mathcal{R}_{\text{comp}},\mathcal{K}_{\text{comp}})$.
According to (3), if we denote by $M_{C}(t)$ the number of compartments at
time $t$, then one way to represent this model is as the solution to

$\displaystyle M_{C}(t)$
$\displaystyle=M_{C}(0)+Y_{I}\left(\kappa_{I}t\right)-Y_{E}\left(\int_{0}^{t}\kappa_{E}M_{C}(s)ds\right)+Y_{F}\left(\int_{0}^{t}\kappa_{F}M_{C}(s)ds\right)$
$\displaystyle\hskip 36.135pt-
Y_{C}\left(\int_{0}^{t}\kappa_{C}\frac{M_{C}(s)(M_{C}(s)-1)}{2}\,ds\right),$

where $Y_{I},Y_{E},Y_{F}$, and $Y_{C}$ are independent unit-rate Poisson
processes.

###  2.3 Specifying the full, combined model

Our full model, which we will term a reaction network within interacting
compartments (RNIC), begins with two networks, one representing the dynamics
of the compartments themselves and one representing the chemistry taking place
inside the compartments.

  * •

A CRN $\mathcal{H}_{\mathcal{K}}$ of the form $0\leftrightarrows
C\leftrightarrows 2C$, called the compartment network. The state of this CRN
(in $\mathbb{Z}_{\geq 0}$) will be the number of compartments.

  * •

An CRN $\mathcal{I}_{\mathcal{K}}$, called the chemistry (or _I nternal
network_), with $d$ species.

The behavior of the model between transitions of the compartment model is
straightforward: the CRN within each compartment evolves independently as a
Markov chain with transition rates specified by (1). All that remains is to
specify what happens to the full model at the transition times of the
compartment model. Hence, there are four cases to consider.

  * •

An arrival: $0\to C$. We assume the existence of a probability measure $\mu$
on $\mathbb{Z}^{d}_{\geq 0}$. Each time an arrival event occurs, we add a new
compartment whose initial state is chosen according to $\mu$, independent of
the past. (Note that $\mu$ is not necessary when $\kappa_{I}=0$.)

  * •

A departure: $C\to 0$. When a departure event occurs, we choose one of the
compartments, uniformly at random, for deletion.

  * •

A merger: $2C\to C$. When a merger event occurs, we select two compartments,
uniformly at random. We replace the chosen compartments with a single
compartment. The state of the new compartment is the sum of the states of the
two it replaced.

  * •

A division: $C\to 2C$. When a division event occurs, we select a compartment,
uniformly at random. We replace the chosen compartment with two new
compartments, whose initial states are determined by having each molecule from
the chosen compartment select one of the two new compartments uniformly. For
example, if there are $n_{A}$ type $A$ species in the chosen compartment, then
one of the new compartments will get a number of $A$ molecules given by a
binomial distribution with parameters $n_{A}$ and $p=\tfrac{1}{2}$, and the
other compartment will get $n_{A}$ minus that value.

This whole system will be denoted
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$.

######  Remark 2.2:

Above, we assume that when divisions, i.e., compartment transitions of the
form $C\to 2C$, happen, each molecule picks a new compartment uniformly at
random. This assumption makes the constructions in this paper easier. However,
our proofs only require that the total number of each species across
compartments is preserved when each division happens.

Similar to our network representations for reaction networks, we can specify
the above model through a picture of the following form:

${\mathcal{I}_{\mathcal{K}}}$${0}$${C}$${2C}$${\mu}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$$\scriptstyle{\kappa_{F}}$$\scriptstyle{\kappa_{C}}$
(5)

where “$\mathcal{I}_{\mathcal{K}}$” is a stand-in for a standard CRN diagram,
such as the one in (4).

######  Example 2.3:

If $\mathcal{I}_{\mathcal{K}}$ is exactly the network diagrammed in Example
2.1 and $\mu$ is the point mass with $3$ molecules of $A$ and $4$ molecules of
$B$, we would write

$\leavevmode\hbox to220.26pt{\vbox
to47.52pt{\pgfpicture\makeatletter\hbox{\hskip 110.13pt\lower-23.56941pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{\offinterlineskip{}{}{{{}}{{}}{{}}{{}}{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-110.13pt}{-23.56941pt}\pgfsys@invoke{
}\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip
18.21002pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-13.90448pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${A+B}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip
18.21002pt\hfil&\hfil\hskip 30.80551pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${0}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip
6.80554pt\hfil&\hfil\hskip 32.3489pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-4.0434pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${B}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip
8.34894pt\hfil&\hfil\hskip 44.70999pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-16.40448pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${A+2B}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip
20.71002pt\hfil&\hfil\hskip 32.05551pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.75pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${A}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip
8.05554pt\hfil\cr\vskip 18.00005pt\cr\hfil\hskip
10.84895pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-6.54341pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${2B}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip
10.84895pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}{{{{}}}{{}}{{}}{{}}{{}}{{}}{{}}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {}{ {}{}{}}{}{ {}{}{}}
{{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-73.50996pt}{11.91669pt}\pgfsys@lineto{-50.30995pt}{11.91669pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-50.10997pt}{11.91669pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-65.20998pt}{14.26945pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{10}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{}{}{ {}{}{}}{}{
{}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{{}}{}{}{{{}{}}}{{}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}}
{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-35.89891pt}{14.93057pt}\pgfsys@lineto{-12.6989pt}{14.93057pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-12.49892pt}{14.93057pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-25.84892pt}{17.28334pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{2}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{}{}{ {}{}{}}{}{
{}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{{}}{}{}{{{}{}}}{{}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}}
{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-12.29893pt}{8.9028pt}\pgfsys@lineto{-35.49895pt}{8.9028pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{-1.0}{0.0}{0.0}{-1.0}{-35.69893pt}{8.9028pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-26.11548pt}{3.53615pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{\kappa}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}
{{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{70.21895pt}{11.91669pt}\pgfsys@lineto{93.41896pt}{11.91669pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{93.61894pt}{11.91669pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{80.26894pt}{14.26945pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{3}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}{ {}{}{}}{}{ {}{}{}}
{{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-80.87103pt}{-13.63394pt}\pgfsys@lineto{-50.2418pt}{6.97906pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{0.82962}{0.55832}{-0.55832}{0.82962}{-50.0759pt}{7.09071pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ }}{ } {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-63.03775pt}{-10.07967pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{$\scriptstyle{5}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope{}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\hskip
21.68121pt\leavevmode\hbox to99.56pt{\vbox
to23.8pt{\pgfpicture\makeatletter\hbox{\hskip 49.77908pt\lower-16.47635pt\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\pgfsys@setlinewidth{0.4pt}\pgfsys@invoke{ }\nullfont\hbox
to0.0pt{\pgfsys@beginscope\pgfsys@invoke{
}{}{}{}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{\offinterlineskip{}{}{{{}}{{}}{{}}}{{{}}}{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-49.77908pt}{-7.07637pt}\pgfsys@invoke{
}\hbox{\vbox{\halign{\pgf@matrix@init@row\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding&&\pgf@matrix@step@column{\pgf@matrix@startcell#\pgf@matrix@endcell}&#\pgf@matrix@padding\cr\hfil\hskip
6.80554pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-2.5pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${0}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip
6.80554pt\hfil&\hfil\hskip 32.23676pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-3.93124pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${C}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{}}}&\hskip
8.23679pt\hfil&\hfil\hskip 34.73676pt\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{}{{ {}{}}}{ {}{}}
{{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-6.43124pt}{0.0pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{ }\hbox{${2C}$}
}}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}&\hskip
10.73679pt\hfil\cr}}}\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}{{{{}}}{{}}{{}}{{}}}} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}} {{}}{}{}{}{ {}{}{}}{}{ {}{}{}}{}{ {}{}{}} {{{{{}}{
{}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{{}}{}{}{{{}{}}}{{}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}}
{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-35.968pt}{-1.56248pt}\pgfsys@lineto{-12.76799pt}{-1.56248pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-12.56801pt}{-1.56248pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-27.22067pt}{2.15695pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\hbox{$\scriptstyle{\kappa_{I}}$} }}\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{}{}{ {}{}{}}{}{
{}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{{}}{}{}{{{}{}}}{{}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}}
{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{-12.36803pt}{-7.59026pt}\pgfsys@lineto{-35.56804pt}{-7.59026pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{-1.0}{0.0}{0.0}{-1.0}{-35.76802pt}{-7.59026pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-27.77623pt}{-12.95691pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\hbox{$\scriptstyle{\kappa_{E}}$} }}\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{}{}{ {}{}{}}{}{
{}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{{}}{}{}{{{}{}}}{{}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}}
{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{4.50554pt}{-1.56248pt}\pgfsys@lineto{27.70555pt}{-1.56248pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{27.90553pt}{-1.56248pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{12.72508pt}{2.15695pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\hbox{$\scriptstyle{\kappa_{F}}$} }}\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{}}{}{}{}{ {}{}{}}{}{
{}{}{}}{}{ {}{}{}} {{{{{}}{ {}{}}{}{}{{}{}}}}}{}{{{{{}}{
{}{}}{}{}{{}{}}}}}{{}}{}{}{}{}{}{{{}{}}}{{}}{}{}{{{}{}}}{{}}\hbox{\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{{}}{}{}} \pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}}\hbox{{\pgfsys@beginscope\pgfsys@invoke{ }{{}{{}}{}{}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} {{}}{}{{}}{}{{}}
{}{}{}{}{{{}{}}}{}{{}}{}{}{}{{{}{}}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@setlinewidth{0.39998pt}\pgfsys@invoke{
}{}{}{}{}{{}}{}{}{{}}\pgfsys@moveto{28.10551pt}{-7.59026pt}\pgfsys@lineto{4.9055pt}{-7.59026pt}\pgfsys@stroke\pgfsys@invoke{
}{{}{{}}{}{}{{}}{{{}}}}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{-1.0}{0.0}{0.0}{-1.0}{4.70552pt}{-7.59026pt}\pgfsys@invoke{
}\pgfsys@invoke{ \lxSVG@closescope }\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}{{}}}}\hbox{\hbox{{\pgfsys@beginscope\pgfsys@invoke{
}{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} {
}{{{{}}\pgfsys@beginscope\pgfsys@invoke{
}\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{12.71648pt}{-12.95691pt}\pgfsys@invoke{
}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@rgb@stroke{0}{0}{0}\pgfsys@invoke{
}\pgfsys@color@rgb@fill{0}{0}{0}\pgfsys@invoke{
}\hbox{$\scriptstyle{\kappa_{C}}$} }}\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope
\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope{{
{}{}{}}}{}{}\hss}\pgfsys@discardpath\pgfsys@invoke{\lxSVG@closescope
}\pgfsys@endscope\hss}}\lxSVG@closescope\endpgfpicture}}\hskip
36.135pt\delta_{(3,4)}(a,b)$

$\triangle$

See also Example 2.8 for another specific example.

There are multiple avenues for generalizations. For example, when a merger
occurs it could be that not all the molecules make it into the new
compartment, or when a division occurs it could be that some molecules are
lost, or there is a non-uniform mechanism for distributing the molecules.
Moreover, it could be that the rate of compartment fragmentation or exit
depends on the internal state of the compartment. These models all fall under
the more general framework given in [11] and could be studied mathematically
in the future if there is a desire, but for the initial development of the
mathematics we choose to keep things simpler.

####  2.3.1 Simulation representation

There are multiple ways to describe a Markov model satisfying the information
given in the ingredients
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$. The
first we give is what we term a “simulation” representation in which we
enumerate the compartments and track the counts of the species in each
compartment.

The simulation representation will be a Markov chain $F^{\mathrm{sim}}$ whose
state is a finite vector of elements of $\mathbb{Z}^{d}_{\geq 0}$, where $d$,
as always, is the number of species. We first describe the model via an
example. Afterwards we will provide the mathematical details.

######  Example 2.4:

Consider again the model from Example 2.3. Suppose that at time $T$ there are
4 compartments, where the first has two $A$ and two $B$, the second has no $A$
and one $B$, the third again has two of each, and the last has one $A$ and
twelve $B$. Then the state of the model $F^{\mathrm{sim}}$ would be the vector

$\displaystyle\left(\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}0\\\
1\end{array}\right],\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}1\\\ 12\end{array}\right]\right).$

We now suppose that at time $T$ a transition occurs. We first consider four
possibilities if the transition is due to a reaction of the compartment model.

  * •

Suppose first that the compartment transition is an inflow event. We will make
the convention that the new compartment due to an inflow reaction will always
be placed at the end of the vector of states. Hence, because the initial
distribution for arriving compartments is a point mass at $(3,4)$ the new
state of the full system is

$\displaystyle\left(\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}0\\\
1\end{array}\right],\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}1\\\
12\end{array}\right],\left[\begin{array}[]{c}3\\\ 4\end{array}\right]\right).$

  * •

Next suppose that the compartment transition is an exit event. In this case we
must choose a compartment at random, delete it from the vector, and re-index
the other components. Thus, we start by choosing from $\\{1,2,3,4\\}$, each
with probability $1/4$. Suppose that the value $3$ is chosen so that the third
compartment will be deleted. In this case, the new state of the full system is

$\displaystyle\left(\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}0\\\
1\end{array}\right],\left[\begin{array}[]{c}1\\\ 12\end{array}\right]\right).$

  * •

Now suppose that the compartment transition is a merger, or coagulation. Now
we must select two compartments at random and combine their contents. We will
always choose that the combined contents of the compartments will be placed
within the compartment with the lower index and will delete the compartment
with the higher index. Thus, assuming we choose the compartments indexed 1 and
2, we then merge the first and second compartments and place their contents
into compartment 1 (since it has the smaller index of the two chosen) and then
delete the second compartment. The resulting state is

$\displaystyle\left(\left[\begin{array}[]{c}2\\\
3\end{array}\right],\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}1\\\ 12\end{array}\right]\right).$

  * •

Finally, we suppose that the compartment transition is a fragmentation. The
procedure will be as follows. We will first choose the index of the
compartment that fragments, we then create two new compartments and will then
split the contents between these new compartments (with each particular
molecule choosing between the new compartments with equal probability). The
originally chosen compartment will be deleted and the two new compartments
will be placed at the end of the vector of states.

For example, suppose we choose compartment 3 for fragmentation (which occurs
with probability $1/4$). We then split the contents of the original third
compartment (four molecules total, $2$ of $A$ and $2$ of $B$) uniformly at
random between the two new compartments. Suppose for concreteness that we
split as $\left[\begin{array}[]{c}1\\\ 2\end{array}\right]$ and
$\left[\begin{array}[]{c}1\\\ 0\end{array}\right]$. Then, after deleting the
3rd compartment and adding these two onto the end we have a new state for the
full model of

$\displaystyle\left(\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}0\\\
1\end{array}\right],\left[\begin{array}[]{c}1\\\
12\end{array}\right],\left[\begin{array}[]{c}1\\\
2\end{array}\right],\left[\begin{array}[]{c}1\\\ 0\end{array}\right]\right).$

It is also possible that the transition at time $T$ was due to a reaction
taking place within one of the compartments. For example, if the reaction
$A+2B\to A$ happens inside the fourth compartment, then the state of the whole
system, $F^{\mathrm{sim}}$, will become

$\displaystyle\left(\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}0\\\
1\end{array}\right],\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}1\\\ 10\end{array}\right]\right).$

$\triangle$

Now we give the formal mathematical description of $F^{\mathrm{sim}}$. First,
let $\\{M_{C}(t)\\}_{t\geq 0}$ be the Markov chain associated to the
compartment network $\mathcal{H}_{\mathcal{K}}$. Then $M_{C}(t)$ will be the
number of compartments at time $t$. Let $\\{T_{i}\\}_{i=0}^{\infty}$ be the
jump times for this Markov chain, where $T_{0}=0$. For any $i\geq 0$ and any
$j=1,\dots,M_{C}(T_{i})$, let $\\{X^{i}_{j}(t)\\}_{t\in[T_{i},T_{i+1}]}$ be
realizations of the Markov chain associated to $\mathcal{I}_{\mathcal{K}}$
with initial distributions (at time $T_{i}$) specified below. Suppose that for
any $i_{1},i_{2}$ and $j_{1},j_{2}$ with either $i_{1}\neq i_{2}$ or
$j_{1}\neq j_{2}$, the chains $X^{i_{1}}_{j_{1}}$ and $X^{i_{2}}_{j_{2}}$ are
independent conditional on their initial conditions, and suppose that the
initial distributions are chosen in the following manner (which are just
formal characterizations of the details provided in the example above):

  * •

If the compartment transition at time $T_{i+1}$ was an inflow event ($0\to
C$), then let $X^{i+1}_{j}(T_{i+1})=X^{i}_{j}(T_{i+1})$ for
$j=1,\dots,M_{C}(T_{i})$, and for $j=M_{C}(T_{i+1})=M_{C}(T_{i})+1$ let
$X^{i+1}_{j}(T_{i+1})$ be distributed according to $\mu$, independently of
everything in the past.

  * •

If the compartment transition at time $T_{i+1}$ was an exit event $(C\to 0$),
then let $J_{i}$ be chosen uniformly at random from
$\\{1,\cdots,M_{C}(T_{i})\\}$, independently of everything in the past. Let
$X^{i+1}_{j}(T_{i+1})=X^{i}_{j}(T_{i+1})$ for $j<J_{i}$, and let
$X^{i+1}_{j}(T_{i+1})=X^{i}_{j+1}(T_{i+1})$ for $j\geq J_{i}$.

  * •

If the compartment transition at time $T_{i+1}$ was a merger, or coagulation,
event ($2C\to C$), then let $J^{1}_{i}$ and $J^{2}_{i}$ be chosen uniformly at
random from $\\{1,\cdots,M_{C}(T_{i})\\}$ and
$\\{1,\cdots,M_{C}(T_{i})\\}\setminus\\{J^{1}_{i}\\}$, respectively,
independent of everything in the past. Let
$X^{i+1}_{j}(T_{i+1})=X^{i}_{j}(T_{i+1})$ for
$j<\max\\{J^{1}_{i},J^{2}_{i}\\}$ with $j\neq\min\\{J^{1}_{i},J^{2}_{i}\\}$,
let $X^{i+1}_{j}(T_{i+1})=X^{i}_{j+1}(T_{i+1})$ for
$j\geq\max\\{J^{1}_{i},J^{2}_{i}\\}$, and let
$X^{i+1}_{j}(T_{i+1})=X^{i}_{J^{1}_{i}}(T_{i+1})+X^{i}_{J^{2}_{i}}(T_{i+1})$
for $j=\min\\{J^{1}_{i},J^{2}_{i}\\}$.

  * •

If the compartment transition at time $T_{i+1}$ was a fragmentation event
($C\to 2C$), then let $J_{i}$ be chosen uniformly at random from
$\\{1,\cdots,M_{C}(T_{i})\\}$, independently of everything in the past. Let
$\\{Z^{i}_{k}(x):x\in\mathbb{Z}^{d},k=1,\dots,d\\}$ be a collection of random
variables, independent of each other and everything else, with
$Z^{i}_{k}(x)\sim\mathrm{Binom}\left(0.5,x_{k}\right)$. Let $Z^{i}(x)$ denote
the vector $\big{(}Z^{i}_{1}(x),\cdots,Z^{i}_{d}(x)\big{)}$. Let
$X^{i+1}_{j}(T_{i+1})=X^{i}_{j}(T_{i+1})$ for $j<J_{i}$, let
$X^{i+1}_{j}(T_{i+1})=X^{i}_{j+1}(T_{i+1})$ for
$j=J_{i},\dots,M_{C}(T_{i})-1$, and for $j=M_{C}(T_{i})$ let
$X^{i+1}_{j}(T_{i+1})=Z^{i}(X^{i}_{J_{i}}(T_{i+1}))$ and
$X^{i+1}_{j+1}(T_{i+1})=X^{i}_{J_{i}}(T_{i+1})-X^{i+1}_{j}(T_{i+1})$.

Let $F^{\mathrm{sim}}(t)$ be the vector
$\left(X^{i}_{1}(t),X^{i}_{2}(t),\cdots,X^{i}_{M_{C}(t)}(t)\right)$, where $i$
is such that $T_{i}\leq t<T_{i+1}$.

######  Lemma 2.5:

The process $\\{F^{\mathrm{sim}}(t)\\}_{t\geq 0}$ is a continuous time Markov
chain with state space $\bigcup_{m\geq 0}\left(\mathbb{Z}_{\geq
0}^{d}\right)^{m}$, the space of finite tuples of elements of
$\mathbb{Z}_{\geq 0}^{d}$.

###### Proof.

To show that this is a Markov process we have to show that the holding times
are exponential and the updates are independent of the holding times. To see
that the holding times are exponential, notice that since $M_{C}$ is a Markov
chain it has exponential holding times, and similarly for each $X^{i}_{j}$.
But the holding times for these processes are independent, and the minimum of
independent exponential random variables is itself exponential.

Furthermore, the minimum of a (finite) collection of independent exponential
random variables is independent of the index at which the minimum occurs, so
the updates are indeed independent of the holding times.

The fact that $F^{\mathrm{sim}}$ takes values in the space of finite tuples is
equivalent to $M_{C}$ being finite for all time, which in turn is equivalent
to the fact that $M_{C}$ is not explosive, regardless of the choice of rate
constants in $\mathcal{H}_{\mathcal{K}}$. This is a standard result in the
theory of $1$-$d$ mass action stochastic reaction networks; see for instance
[29]. ∎

####  2.3.2 An explicit construction of the simulation representation

We discuss one way of constructing the model described in Section 2.3.1, in
the spirit of the Kurtz representation (3). Here, by “construction” we mean an
explicit detailing of the random processes and random variables needed to
generate a single realization of the process. The construction is of interest
since it is amenable to analysis, coupling methods, simulation methods, etc.
The construction will be used later in this paper to verify some behaviors of
Example 4.23.

Let $\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ be
as above. Suppose that $M_{C}(0)$ is the initial number of compartments in the
system and further suppose that $M_{C}$ is given as the solution to

$\displaystyle\begin{split}M_{C}(t)&=M_{C}(0)+Y_{I}\left(\kappa_{I}t\right)-Y_{E}\left(\int_{0}^{t}\kappa_{E}M_{C}(s)ds\right)+Y_{F}\left(\int_{0}^{t}\kappa_{F}M_{C}(s)ds\right)\\\
&\hskip 36.135pt-
Y_{C}\left(\int_{0}^{t}\kappa_{C}\frac{M_{C}(s)(M_{C}(s)-1)}{2}\,ds\right),\end{split}$
(6)

where $Y_{I},Y_{E},Y_{F}$, and $Y_{C}$ are independent unit-rate Poisson
processes. Then $M_{C}$ is the Markov chain on $\mathbb{Z}_{\geq 0}$
associated to $\mathcal{H}_{\mathcal{K}}$, so that $M_{C}(t)$ gives the number
of compartments at any time $t\geq 0$.

The jump times of the counting processes
$R_{I}(t)=Y_{I}\left(\kappa_{I}t\right)$,
$R_{E}(t)=Y_{E}\left(\int_{0}^{t}\kappa_{E}M_{C}(s)ds\right)$,
$R_{F}(t)=Y_{F}\left(\int_{0}^{t}\kappa_{F}M_{C}(s)ds\right)$, and
$R_{C}(t)=Y_{C}\left(\int_{0}^{t}\kappa_{C}\frac{M_{C}(s)(M_{C}(s)-1)}{2}\,ds\right)$
determine when the RNIC model transitions due to changes in the count of the
compartments. To each such transition we will also require a collection of
random variables needed to carry out the updates in the RNIC model. We detail
these random variables below. Before proceeding, we remind the reader that a
uniform random variable can be used to generate a sample from a given
distribution in a number of ways. For some standard transformation methods
see, for example, [7, Section 5.2].

In the description below all random variables are independent of each other
and of the Poisson processes $Y_{I},Y_{E},Y_{F},Y_{C}$. We require:

  * •

A collection of independent uniform random variables $\\{u_{i}^{I}\\}$,
$i=1,2,\dots$. When $R_{I}(T)-R_{I}(T-)=1$, the random variable
$u_{R_{I}(T)}^{I}$ is used to generate a sample from $\mu$.

  * •

A collection of independent uniform random variables $\\{u_{i}^{E}\\}$,
$i=1,2,\dots$. When $R_{E}(T)-R_{E}(T-)=1$, the random variable
$u_{R_{E}(T)}^{E}$ is used to determine which compartment exits at that time.

  * •

Two collections of independent uniform random variables: (i)
$\\{u_{i}^{F}\\}$, $i=1,2,\dots$, and (ii) an array $\\{\hat{u}_{i,j}^{F}\\}$,
$i,j\in\\{1,2,\dots\\}$. When $R_{F}(T)-R_{F}(T-)=1$, the random variable
$u_{R_{F}(T)}^{F}$ is used to determine which compartment fragments. We then
utilize the finite collection $\\{\hat{u}^{F}_{R_{F}(T),j}\\}$, $j=1,\dots,M$,
where $M$ is the total number of molecules in the chosen compartment, to
divide the different molecules between the two new cells.

  * •

A collection of independent uniform random variables $\\{u_{i}^{C}\\}$,
$i=1,2,\dots$. When $R_{C}(T)-R_{C}(T-)=1$, the random variable
$u_{R_{C}(T)}^{C}$ is used to determine which two compartments are chosen to
merge.

Note that the collections detailed above are chosen before a realization is
generated. Said differently, the realization of the RNIC model is a function
of these independent random variables.

All that remains is to give the timing of the different chemical reactions.
One method is the following. Let $\\{Y_{r}\\}_{r\in\mathcal{R}}$ be a
collection of independent (of each other, and all other random objects so
far), unit-rate Poisson processes, one for each possible reaction in
$\mathcal{I}_{\mathcal{K}}$. Moreover, for each $r\in\mathcal{R}$, let
$\\{u^{r}_{i}\\}$, $i=1,2,\dots$ be a collection of independent uniform random
variables. Then, for $r\in\mathcal{R}$, we let

$R_{r}(t)=Y_{r}\left(\mathop{\sum_{i\geq 0}}_{T_{i}\leq
t}\int_{T_{i}}^{T_{i+1}\wedge
t}\sum_{j=1}^{M_{C}(T_{i})}\lambda_{r}(X^{i}_{j}(s))ds\right),$

where the $T_{i}$ are the jump times of the process $M_{C}$, $X_{j}^{i}(s)$ is
the state of the process in compartment $j$ at time $s$, and $\lambda_{r}$ is
given as in (2). Then $R_{r}$ is the counting process that jumps by $+1$ when
the $r$th reaction takes place in _some_ compartment. When
$R_{r}(T)-R_{r}(T-)=1$, meaning a reaction has taken place somewhere, we use
$u^{r}_{R_{r}(T)}$ to determine the compartment within which the reaction took
place. In particular, the probability that it took place in compartment $k$ is
simply

$\frac{\lambda_{r}(X_{k}^{i}(T-))}{\sum_{j=1}^{M_{C}(T_{i})}\lambda_{r}(X_{j}^{i}(T-))}.$

####  2.3.3 A coarse-grained representation

While the description (and construction) above is often convenient for the
sake of analysis and simulation, it is sometimes not the most natural way to
think about these models. For example, suppose we have a model with a single
species, denoted $S$, and for which there are two compartments at time $t$, so
that $M_{C}(t)=2$. It is reasonable to think that we would not care to
distinguish the situation in which there are $6$ molecules of species $S$ in
the first compartment and $2$ in the second, which is the state $(6,2)$,
versus the situation of $2$ molecules of $S$ in the first and $6$ in the
second, which is the state $(2,6)$. In this situation, we would simply care
that we have one compartment with two $S$ molecules, another with six, and
there are no other compartments.

To handle this, we consider a function $n:\mathbb{Z}_{\geq
0}\to\mathbb{Z}_{\geq 0}$ in which $n_{x}:=n(x)$ gives the number of
compartments present with precisely $x$ molecules of $S$ (hence the notation
that “$n$” gives the number of compartments with different counts). In this
case, the state of the example system described above would simply be the
function with

$n_{x}=\begin{cases}1,&\text{ if }x=2\\\ 1,&\text{ if }x=6\\\
0,&\text{else}.\end{cases}$

Note that in this one-dimensional case we can also think of $n$ as an
“infinite vector.” For example, in our example above we would have

$n=(0,0,1,0,0,0,1,0,0,\dots),$

with only zeros continuing on.

For another example, we could consider the case discussed in Example 2.4,
where there are two species $A$ and $B$ and the state for the simulation
representation was

$\displaystyle\left(\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}0\\\
1\end{array}\right],\left[\begin{array}[]{c}2\\\
2\end{array}\right],\left[\begin{array}[]{c}1\\\ 12\end{array}\right]\right).$

In this case, the state could naturally be described by the function

$n_{x}=\begin{cases}2,&\text{ if }x=\left[\begin{array}[]{c}2\\\
2\end{array}\right]\vspace{.1in}\\\ 1,&\text{ if
}x=\left[\begin{array}[]{c}0\\\ 1\end{array}\right]\vspace{.1in}\\\ 1,&\text{
if }x=\left[\begin{array}[]{c}1\\\ 12\end{array}\right]\\\
0,&\text{else}.\end{cases}$

Note that in this example, it is not natural to view $n$ as an “infinite
vector.” Instead, it would be natural to view it as an “infinite array” with a
two in the $(2,2)$ component, ones in the $(0,1)$ and $(1,12)$ components, and
zeros elsewhere.

Thus, we may take the following approach, as done in [11]. The state space of
the coarse-grained model will be

$\displaystyle\begin{split}\mathcal{N}:={}&\\{\text{functions
}n:\mathbb{Z}^{d}_{\geq 0}\to\mathbb{Z}_{\geq 0}\text{ with compact
support}\\}\\\ ={}&\\{\text{functions }n:\mathbb{Z}^{d}_{\geq
0}\to\mathbb{Z}_{\geq 0}\text{ with finite support}\\}\\\
={}&\\{\text{functions }n:\mathbb{Z}^{d}_{\geq 0}\to\mathbb{Z}_{\geq 0}\text{
with finite $\ell^{1}$ norm}\\},\end{split}$ (7)

where we observe that all three sets are the same. Given $n:\mathbb{Z}_{\geq
0}^{d}\to\mathbb{Z}_{\geq 0}$, we write $n=(n_{x})_{x\in\mathbb{Z}^{d}_{\geq
0}}$. For each possible state $x\in\mathbb{Z}^{d}_{\geq 0}$ of the chemistry,
$n_{x}\in\mathbb{Z}_{\geq 0}$ represents the number of compartments whose
chemistry has that particular state. Given Markov chains $M_{C}$ and
$X^{i}_{j}$ as defined in Section 2.3.1, let $N$ be the process where
$N_{x}(t)$ is the number of compartments in state $x\in\mathbb{Z}^{d}_{\geq
0}$ at time $t\geq 0$:

$N_{x}(t)=\sum_{i=0}^{\infty}\mathbb{I}\\{t\in[T_{i},T_{i+1})\\}\sum_{j=1}^{M_{C}(T_{i})}\mathbb{I}\\{X^{i}_{j}(t)=x\\}.$

Note that the total number of compartments at time $t\geq 0$ can be recovered
from $N(t)$ via

$M_{C}(t)=\left\lVert
N(t)\right\rVert_{\ell^{1}}:=\sum_{x\in\mathbb{Z}^{d}_{\geq 0}}N_{x}(t).$

Note also that the process $N$ transitions iff $F^{\mathrm{sim}}$ does. This
fact is important enough that we state it as a lemma:

######  Lemma 2.6:

Let $F^{\mathrm{sim}}$ and $N$ be as above. Then $N$ undergoes a transition at
time $t$ iff $F^{\mathrm{sim}}$ does.

###### Proof.

On the one hand, $N$ is defined as a function of $F^{\mathrm{sim}}$ and so $N$
cannot transition if $F^{\mathrm{sim}}$ does not. On the other hand, all
possible transitions of $F^{\mathrm{sim}}$ cause a change in $N$: If
$F^{\mathrm{sim}}$ transitions because $M_{C}$ does, then $\left\lVert
N\right\rVert_{\ell^{1}}=M_{C}$ changes, whereas if $F^{\mathrm{sim}}$ changes
otherwise then the contents of some single compartment updated, which changes
$N$. ∎

For the lemma below, we recall that for $x\in\mathbb{Z}^{d}_{\geq 0}$ we
define $e_{x}$ to be the function taking the value of one at $x$ and zero
otherwise.

######  Lemma 2.7:

Let $N(t)$ be as defined above. Then $\\{N(t)\\}_{t\geq 0}$ is a Markov chain
taking values in $\mathcal{N}$, defined in (7). Moreover, for
$n\in\mathcal{N}$, the transitions rates are as follows:

Transition type |  |  |  | Rate
---|---|---|---|---
Compartment inflow |  | $n\mapsto n+e_{x}$ |  | $\kappa_{I}\mu(x)$
Compartment exit |  | $n\mapsto n-e_{x}$ |  | $\kappa_{E}n_{x}$
Compartment coagulation, $x\neq y$ |  | $n\mapsto n+e_{x+y}-e_{x}-e_{y}$ |  | $\kappa_{C}n_{x}n_{y}$
Compartment coagulation |  | $n\mapsto n+e_{2x}-2e_{x}$ |  | $\kappa_{C}\displaystyle\binom{n_{x}}{2}$
Compartment fragmentation |  | $n\mapsto n-e_{x+y}+e_{x}+e_{y}$ |  | $\kappa_{F}n_{x+y}\varphi(x+y,x)$
($x=y$ allowed here) |  |  |  |
Internal reaction $r\in\mathcal{R}$ |  | $n\mapsto n-e_{x}+e_{x+\nu_{r}^{\prime}-\nu_{r}}$ |  | $n_{x}\kappa_{r}\displaystyle\binom{x}{\nu_{r}}$

where

$\varphi(z,x):=\prod_{k=1}^{d}\binom{z_{k}}{x_{k}}2^{-z_{k}}$

so that the distribution of the resulting compartments after a fragmentation
is independently binomial in each species. Note that each row mentioning $x$
or $y$ corresponds to an infinite family of transitions and in the last row
$r\in\mathcal{R}$ also ranges over all reactions of the reaction network
$\mathcal{I}$.

###### Proof.

The fact that $N$ has finite support follows from the fact that
$F^{\mathrm{sim}}$ is always a finite tuple, proved in Lemma 2.5.

The fact that $N$ is Markovian with the rates given follows from consideration
of the infinitesimal behavior of $F^{\mathrm{sim}}$. For example, for $x\neq
y\in\mathbb{Z}^{d}_{\geq 0}$,

$\displaystyle\mathbb{P}(N(t+h)=n+e_{x+y}-e_{x}-e_{y}|N(t)=n)$
$\displaystyle=\kappa_{C}n_{x}n_{y}h+o(h),\text{ as $h\to 0$},$

since, to leading order, the probability that some compartment in state $x$
merges with a compartment in state $y$ in the time interval $[t,t+h)$ is
$\kappa_{C}n_{x}n_{y}h$. The other rows of the table follow similarly. ∎

######  Example 2.8:

Consider the following possible compartment model:

${0}$${S}$${0}$${C}$${2C}$${\frac{1}{2}\delta_{5}+\frac{1}{2}\delta_{17}}$$\scriptstyle{\kappa_{b}}$$\scriptstyle{\kappa_{d}}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$$\scriptstyle{\kappa_{C}}$

Here we are keeping track of some chemical $S$ which forms with rate
$\kappa_{b}$ and degrades with rate $\kappa_{d}$. Compartments are allowed to
enter with rate $\kappa_{I}$, and new compartments that enter this way have
either $5$ or $17$ molecules of $S$, each with probability $1/2$. Compartments
can also exit with rate constant $\kappa_{E}$, and merge (or coagulate) with
rate constant $\kappa_{C}$. Since there is only one species, the state space
for the chemistry is $\mathbb{Z}_{\geq 0}^{1}=\mathbb{Z}_{\geq 0}$. As we
detail below, we will be assuming mass-action kinetics; in this case that
means when the model is in state $n\in\mathcal{N}$ the transition rates are
given by

Transition type |  |  |  | Rate
---|---|---|---|---
Compartment inflow |  | $n\mapsto n+e_{5}$ |  | $\kappa_{I}/2$
Compartment inflow |  | $n\mapsto n+e_{17}$ |  | $\kappa_{I}/2$
Compartment exit |  | $n\mapsto n-e_{x}$ |  | $\kappa_{E}n_{x}$
Compartment coagulation ($x\neq y$) |  | $n\mapsto n+e_{x+y}-e_{x}-e_{y}$ |  | $\kappa_{C}n_{x}n_{y}$
Compartment coagulation |  | $n\mapsto n+e_{2x}-2e_{x}$ |  | $\kappa_{C}\displaystyle\binom{n_{x}}{2}$
$S$ birth |  | $n\mapsto n-e_{x}+e_{x+1}$ |  | $\kappa_{b}n_{x}$
$S$ death |  | $n\mapsto n-e_{x}+e_{x-1}$ |  | $\kappa_{d}n_{x}x$

As before, each row mentioning $x$ or $y$ corresponds to an infinite family of
transitions, one for each $x\neq y\in\mathbb{Z}^{d}_{\geq 0}$, and as always
$e_{x}$ is the unit vector in direction $x$. $\triangle$

##  3 Non-Explosivity

A Markov Chain is explosive if it can undergo infinitely many transitions in
finite time. The formal definition is below [23].

######  Definition 3.1 (Explosivity):

Let $\\{X(t)\\}_{t\geq 0}$ be a continuous-time Markov chain with countable
state space $\mathbb{S}$. For each $m\in\mathbb{Z}_{\geq 0}$, let $\tau_{m}$
be the time of the $m$-th transition of $X$ (formally, $\tau_{0}=0$ and
$\tau_{m}=\inf\\{t>\tau_{m-1}:X(t)\neq X(\tau_{m-1})\\}$), and let
$\tau_{\infty}=\lim_{m\to\infty}\tau_{m}$. We say that $X$ explodes if
$\tau_{\infty}<\infty$. If there is some state $x\in\mathbb{S}$ such that with
positive probability $X$ explodes when started in state $x$, we say that $X$
is explosive.

We will show that explosivity for the RNIC model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ is
determined by explosivity for the internal reaction network
$\mathcal{I}_{\mathcal{K}}$. But to even talk about explosivity for
$\mathcal{F}$ instead of just the Markov chains $F^{\mathrm{sim}}$ or $N$, we
need the following simple proposition.

######  Proposition 3.2:

Suppose we have a RNIC
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$. Let
$F^{\mathrm{sim}}$ and $N$ be the corresponding simulation and coarse-grained
representations. Then $F^{\mathrm{sim}}$ is explosive iff $N$ is explosive.

###### Proof.

This is immediate from lemma 2.6, which says that $F^{\mathrm{sim}}$ and $N$
transition at the same times. ∎

In light of the proposition, we will speak merely of
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ being
explosive, and check the explosivity of either $F^{\mathrm{sim}}$ or $N$
depending on convenience. As it turns out, it will be most convenient to check
explosivity for $F^{\mathrm{sim}}$. (Indeed, the fact that explositivity is
more easily checked for $F^{\mathrm{sim}}$ is one of the major reasons for
introducing $F^{\mathrm{sim}}$ in the first place.)

######  Theorem 3.3:

Suppose we have a RNIC
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$. Then
$\mathcal{F}$ is explosive iff $\mathcal{I}_{\mathcal{K}}$ is explosive.

###### Proof.

First, suppose that $\mathcal{I}_{\mathcal{K}}$ is explosive. As discussed
above, we intend to show that $F^{\mathrm{sim}}$ is explosive. By assumption,
there is some $x\in\mathbb{Z}^{d}$ such that when the Markov chain
corresponding to $\mathcal{I}_{\mathcal{K}}$ is started in state $x$ it
explodes with positive probability. In particular, there is some finite
(nonrandom) time $t$ so that the chemistry undergoes infinitely many
transitions before time $t$ with positive probability. Start
$F^{\mathrm{sim}}$ in the state with one compartment whose state is $x$. With
positive probability, no compartment transitions happen before time $t$. But
the compartment transition times are independent of what is happening inside
them by construction, and the compartment evolves according to
$\mathcal{I}_{\mathcal{K}}$, so on the event that no compartment transition
happens before time $t$ the compartment undergoes infinitely many transitions
before time $t$ with positive probability. It follows that $F^{\mathrm{sim}}$
is explosive.

Conversely, suppose that $\mathcal{I}_{\mathcal{K}}$ is not explosive. Note
that $\mathcal{H}$, the compartment network, is not explosive for any choice
of rate constants (see e.g. [29]). So with probability one $F^{\mathrm{sim}}$
undergoes only finitely many compartment transitions in finite time. But
between each pair of consecutive compartment transitions there are finitely
many compartments each evolving according to $\mathcal{I}_{\mathcal{K}}$, and
by assumption each of these undergoes only finitely many reactions in finite
time a.s.. It follows that $F^{\mathrm{sim}}$ undergoes only finitely many
transitions total in finite time, and hence is not explosive. ∎

##  4 Transience, recurrence, and positive recurrence

The following definitions are standard. For example, see [23].

######  Definition 4.1:

Let $M$ be a Markov chain with countable state space $\mathbb{S}$, and for
$x\in\mathbb{S}$ let $T_{x}=\inf\\{t>0:M_{t}=x\text{ but }\exists
s\in[0,t],M_{s}\neq x\\}$ be the first time the process returns to $x$ (or
just arrives at $x$, if the process does not start from $x$). If
$\mathbb{P}_{x}(T_{x}<\infty)=1$, we say that the state $x$ is _recurrent_ ,
and if $\mathbb{E}_{x}(T_{x})<\infty$ we say that the state $x$ is _positive
recurrent_. A state which is not recurrent is called _transient_ , and a
recurrent state which is not positive recurrent is _null recurrent_. If
$\mathbb{P}_{x}(T_{y}<\infty)>0$ we say that $y$ is _reachable_ from $x$. If
every state $x\in\mathbb{S}$ is positive recurrent, null recurrent, or
transient, we say $M$ is _positive recurrent_ , _null recurrent_ , or
_transient_ , respectively.

A standard fact about (positive) recurrence is that it is a class property:

######  Proposition 4.2 (Theorems 3.4.1(iv) and 3.5.3(i)$\iff$(ii) in [23]):

Suppose that $y$ is reachable from $x$ and $x$ is recurrent (resp. positive
recurrent). Then $y$ is recurrent (resp. positive recurrent).

In other words, if you can get between $x$ and $y$ with positive probability
(in both directions), then $x$ and $y$ are either both transient, both null
recurrent, or both positive recurrent. So for irreducible chains (ones where
you can pass between any two points of the state space with positive
probability), the chain $M$ is always positive recurrent, null recurrent, or
transient.

Before proceeding with the theory, we summarize the results of this section
with a table. The way to read Table 1 is as follows:

  * •

Suppose we have a RNIC
$(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$, and $N$ is the
associated coarse-grained model.

  * •

The top row indicates possible dynamics (transient, null recurrent, or
positive recurrent) for $\mathcal{I}_{\mathcal{K}}$, the chemical model, and
the left column indicates possible dynamics for $\mathcal{H}_{\mathcal{K}}$,
the compartment model. Since the possible dynamics for $N$ will turn out to
depend crucially on whether the compartments can exit ($\kappa_{E}>0$) or not
($\kappa_{E}=0$), the left column is further subdivided along these lines.

  * •

Several cells are marked “Impossible”, because $\mathcal{H}_{\mathcal{K}}$
cannot be null recurrent if $\kappa_{E}=0$.

  * •

The numbers inside each cell refer to the relevant theorems, lemmas, or
examples that demonstrates the result.

|  | Chemistry ($\mathcal{I}_{\mathcal{K}}$) |
---|---|---|---
|  | Transient (Trans.) | Null Recurrent (NR) | Positive Recurrent (PR)
| Transient |  $N$ must be Transient
| Remark 4.4
Compartments ($\mathcal{H}_{\mathcal{K}}$) | NR | $\kappa_{E}=0$ | Impossible
Lemma 4.3
$\kappa_{E}>0$ |  $N$ must be Null Recurrent
Theorem 4.5
PR | $\kappa_{E}=0$ |  $N$ can be Trans. |  $N$ can be Trans. |  $N$ can be Trans.
Ex 4.11 | Ex 4.21 | Ex 4.17, 4.19, 4.23
$N$ can be PR |  $N$ can be PR |  $N$ can be PR
Ex 4.13 | Ex 4.15 | Ex 4.19
| $\kappa_{E}>0$ |  $N$ must be Positive Recurrent
| Theorem 4.5
Table 1: The possibly dynamics for $N$, classified in terms of the dynamics
for $\mathcal{H}_{\mathcal{K}}$ and $\mathcal{I}_{\mathcal{K}}$. In the above
“NR” and “PR” stand for “null recurrent” and “positive recurrent”,
respectively, whereas “Trans.” stands for “transient.”

Note that in all cases where we give an example of a recurrent $N$, the
example is actually positive recurrent. We suspect that null recurrent
examples will also exist, but we felt it more interesting to cover the
behavioral extremes.

Moving to our theory, we begin by considering the dynamics of the compartment
model of section 2.2, which takes the form of a relatively simple reaction
network, namely,

${0}$${C}$${2C}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$$\scriptstyle{\kappa_{F}}$$\scriptstyle{\kappa_{C}}$
(8)

The (positive) recurrence of this model is already completely classified [29].
We state this classification now as a lemma.

######  Lemma 4.3:

Consider the CRN in (8).

  * •

Suppose $\kappa_{I}=0$. Then $0$ is an absorbing state. If some other rate
constant is non-zero then all other states are transient, whereas if all four
rate constants are zero then all states are absorbing.

  * •

Suppose $\kappa_{I}>0$ and $\kappa_{E}>0$. The irreducible state space is
$\\{0,1,2,\dots\\}$ and:

    * –

If $\kappa_{C}>0$, then the chain is positive recurrent.

    * –

If $\kappa_{C}=0$ but $\kappa_{F}<\kappa_{E}$, then the chain is positive
recurrent.

    * –

If $\kappa_{C}=0$ and $\kappa_{F}>\kappa_{E}$, then the chain is transient.

    * –

If $\kappa_{C}=0$ and $\kappa_{F}=\kappa_{E}$, then either
$\kappa_{I}\leq\kappa_{E}$ and the chain is null recurrent, or
$\kappa_{I}>\kappa_{E}$ and the chain is transient.

  * •

Suppose $\kappa_{I}>0$ and $\kappa_{E}=0$. Then all statements remain the same
as in the case $\kappa_{I}>0$ and $\kappa_{E}>0$ except the irreducible state
space is now $\\{1,2,\dots\\}$ (and the state 0 is transient).

Now we begin with our positive results. The first fact is simple enough to be
stated as a remark:

######  Remark 4.4:

Notice that if $N$ is the course-grained representation for
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ and
$n$ is a (positive) recurrent state for $N$, then the number of compartments
in $n$, $\left\lVert n\right\rVert_{\ell^{1}}$, is a (positive) recurrent
state for $\mathcal{H}_{\mathcal{K}}$, since the return time to $\left\lVert
n\right\rVert_{\ell^{1}}$ is bounded by the return time to $n$.

Said succinctly, if $n$ is a positive recurrent state of the full model, then
so is $\left\lVert n\right\rVert_{\ell^{1}}$ for the compartment model. One
might hope that the converse would be true, and it turns out under relatively
mild assumptions it is:

######  Theorem 4.5:

Consider a non-explosive model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ where
$\kappa_{E}>0$, and let $N$ be its course-grained representation. Then a state
$n$ is (positive) recurrent for $N$ iff $n$ is reachable from the empty state
$\vec{0}$ for $N$ and the state $\left\lVert n\right\rVert_{\ell^{1}}$ is
(positive) recurrent for $\mathcal{H}_{\mathcal{K}}$.

###### Proof.

If $\kappa_{I}=0$ the conclusions of the theorem are clear, since by Lemma 4.3
the state with no compartments is absorbing for both $N$ and
$\mathcal{H}_{\mathcal{K}}$ and all other states are transient. From here on
we assume $\kappa_{I}>0$.

Let $M_{C}=\left\lVert N\right\rVert_{\ell^{1}}$ be the number of
compartments; recall that $M_{C}$ is a Markov chain which evolves according to
$\mathcal{H}_{\mathcal{K}}$. Suppose first that $n$ is recurrent for $N$. By
Remark 4.4, $\left\lVert n\right\rVert_{\ell^{1}}$ is recurrent for
$\mathcal{H}_{\mathcal{K}}$. Since $\kappa_{E}>0$ and $\kappa_{I}>0$, by Lemma
4.3 $\mathcal{H}_{\mathcal{K}}$ is irreducible, so $\mathcal{H}_{\mathcal{K}}$
eventually hits zero with probability one when started from $\left\lVert
n\right\rVert_{\ell^{1}}$. But when $M_{C}$ hits zero, $N=\vec{0}$. Since $n$
is recurrent for $N$, it must be that $N$ eventually returns to state $n$
after hitting state $\vec{0}$. This proves that $n$ is reachable from
$\vec{0}$ for $N$.

Now suppose that $n$ is reachable from $\vec{0}$ and the state $\left\lVert
n\right\rVert_{\ell^{1}}$ is positive recurrent (resp. recurrent) for
$\mathcal{H}_{\mathcal{K}}$. Since $\mathcal{H}_{\mathcal{K}}$ is irreducible
as in the previous paragraph, it follows that zero is positive recurrent
(resp. recurrent) for $\mathcal{H}_{\mathcal{K}}$. But $N=\vec{0}$ exactly
when $M_{C}$ is $0$, so $\vec{0}$ is positive recurrent (resp. recurrent) for
$N$. But positive recurrence (resp. recurrence) is a class property and by
assumption $n$ is reachable from $\vec{0}$, so we conclude that $n$ is
positive recurrent (resp. recurrent) for $N$, as desired. ∎

The same theorem holds, mutatis mutandis, for $F^{\mathrm{sim}}$. The proof is
the same so we omit it.

######  Theorem 4.6:

Consider a non-explosive model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ where
$\kappa_{E}>0$, and let $F^{\mathrm{sim}}$ be its simulation representation.
Then a state $(x_{1},\dots,x_{k})$ is (positive) recurrent for
$F^{\mathrm{sim}}$ iff $(x_{1},\cdots,x_{k})$ is reachable from the empty
vector $()$ for $F^{\mathrm{sim}}$ and the state $k$ is (positive) recurrent
for $\mathcal{H}_{\mathcal{K}}$.

######  Remark 4.7:

Theorems 4.5 and 4.6 hold under more general assumptions. Note that the key
idea of both is that $0$ is (positive) recurrent for
$\mathcal{H}_{\mathcal{K}}$. Hence, one can generalize to the situation in
which $\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$
has non-mass action kinetics for either $\mathcal{I}_{\mathcal{K}}$ or
$\mathcal{H}_{\mathcal{K}}$, so long as the system is non-explosive and $0$ is
(positive) recurrent for $\mathcal{H}_{\mathcal{K}}$.

###  4.1 Lyapunov Functions

In what follows we will need to make use of the theory of Lyapunov functions
for Markov chains. This short section is devoted to introducing the extent of
the theory we will use.

The following theorem is well-known. In full generality, it is due to Meyn and
Tweedie [22]. The version below is a specialization to the countable state
space case. For a proof of the version given below, see the later paper [3].

######  Theorem 4.8:

Let $X$ be a continuous-time Markov chain on a countable state space
$\mathbb{S}$ with generator $\mathcal{L}$. Suppose there exists a finite set
$K\subset\mathbb{S}$ and a positive function $V$ on $\mathbb{S}$ such that

$\mathcal{L}V(x)\leq-1$

for all $x\in\mathbb{S}\setminus K$. Suppose further that $V$ is “norm-like,”
in the sense that $\\{x\in\mathbb{S}:V(x)<B\\}$ is finite for every $B>0$.
Then each state in a closed, irreducible component of $\mathbb{S}$ is positive
recurrent. Moreover, if $\tau_{x_{0}}$ is the time for the process to enter
the union of the closed irreducible components given an initial condition
$x_{0}$, then $\mathbb{E}_{x_{0}}[\tau_{x_{0}}]<\infty$.

We will also need the following, which provides a method to check for
transience.

######  Theorem 4.9:

Let $X$ be a non-explosive continuous-time Markov chain on a countable
discrete state space $\mathbb{S}$ with generator $\mathcal{L}$. Let
$B\subset\mathbb{S}$, and let $\tau_{B}$ be the time for the process to enter
$B$. Suppose there is some bounded function $V$ such that for all $x\in
B^{c}$,

$\mathcal{L}V(x)\geq 0.$

Then $\mathbb{P}_{x_{0}}(\tau_{B}<\infty)<1$ for any $x_{0}$ such that

$\sup_{x\in B}V(x)<V(x_{0}).$

For a version of the theorem above that applies in much greater generality,
see Theorem 3.3(i) in [27]. Our theorem is not an immediate corollary of
theirs (they define restricted versions of the chain $X$ and state their
theorem in terms of the generators of the restricted processes), so we will
provide a proof of Theorem 4.9 in the appendix.

###  4.2 Instructive examples

We now consider some examples. The first is an application of Theorem 4.5, and
the rest show the various ways the conclusion of the theorem can fail if the
hypothesis $\kappa_{E}>0$ is not satisfied. These examples also serve to
illustrate various techniques that are useful for analysing recurrence and
transience of RNIC models. In Example 4.13, positive recurrence for the RNIC
is shown via a Lyapunov function, applying Theorem 4.8. In Example 4.21,
transience for the RNIC is shown via a Lyapunov function, applying Theorem
4.9. And in Example 4.23, transience for the RNIC is shown with the help of
the construction of $F^{\mathrm{sim}}$ given in section 2.3.2.

In the following, any rate constants not specified are assumed to be positive.

######  Example 4.10:

Consider the following RNIC.

${0}$${2S}$${0}$${C}$${2C}$${\delta_{0}}$$\scriptstyle{\kappa_{b}}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$$\scriptstyle{\kappa_{C}}$

where $\delta_{0}$ is the point mass at zero (so each compartment enters
empty). Even though $\mathcal{I}_{\mathcal{K}}$ is transient, by Theorem 4.5
the empty state is positive recurrent for $N$. Any state where every
compartment has an even number of $S$ molecules is reachable from the empty
state, hence positive recurrent. Any state where any compartment has an odd
number of $S$ molecules is not reachable from the empty state, hence
transient. $\triangle$

In all of the remaining examples in this section, we have $\kappa_{E}=0$ and
hence the state $0$ will be transient for $\mathcal{H}_{\mathcal{K}}$. Hence,
when discussing the properties of the model we restrict ourselves to the state
space $\mathcal{N}\setminus\\{0\\}$ that does not include the state with zero
compartments.

The case where $\kappa_{E}=0$ is more complicated than the $\kappa_{E}\neq 0$
case. For one thing, it is no longer enough just to look at
$\mathcal{H}_{\mathcal{K}}$ to decide if all states are transient. Indeed, if
Example 4.10 is modified so that $\kappa_{E}=0$ then every state becomes
transient, despite the fact that all states are positive recurrent for the
compartment network $\mathcal{H}_{\mathcal{K}}$:

######  Example 4.11:

Consider the model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$
described by

${0}$${2S}$${0}$${C}$${2C}$${\delta_{0}}$$\scriptstyle{\kappa_{b}}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{C}}$
(9)

where $\delta_{0}$ is again the point mass at zero.

We reiterate that this is exactly the same as the previous example but with
$\kappa_{E}$ set to zero. However, that is enough to make every state
transient for $\mathcal{F}$:

######  Proposition 4.12:

In the RNIC model (9), $\mathcal{I}_{\mathcal{K}}$ is transient,
$\mathcal{H}_{\mathcal{K}}$ is positive recurrent on the irreducible state
space $\\{1,2,\dots\\}$, and $N$ (the coarse-grained model corresponding to
$\mathcal{F}$) is transient.

###### Proof.

Except for the zero-compartment state (which cannot be returned to), all
states are positive recurrent for $\mathcal{H}_{\mathcal{K}}$ by Lemma 4.3.
However, the total number of $S$ molecules across all compartments can never
shrink, and grows with some positive rate (at least $\kappa_{b}$, and larger
if there are more compartments), so all states are transient for $N$. ∎

Thus we see that, in this example, the long-term behavior of
$\mathcal{H}_{\mathcal{K}}$ and the course-grained model $N$ are different.
$\triangle$

The above example shows that when $\kappa_{E}=0$ and
$\mathcal{I}_{\mathcal{K}}$ is transient, $\mathcal{F}$ may be transient even
if $\mathcal{H}_{\mathcal{K}}$ is not. However, this need not always be the
case. Below we have an example that demonstrates that, when $\kappa_{E}=0$ and
$\mathcal{I}_{\mathcal{K}}$ is transient, it is still possible for
$\mathcal{F}$ to be positive recurrent.

######  Example 4.13:

Consider the model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$
described by

${2A+B}$${B}$${0}$${A}$${0}$${C}$${2C}$${\delta_{(0,1)}(a,b)}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{6}$
(10)

where $\delta_{(0,1)}$ is a point mass with zero $A$ molecules and one $B$
molecule. We will show that the chemical model $\mathcal{I}_{\mathcal{K}}$ is
transient but that the course-grained model, $N$, is positive recurrent.
Intuitively, this can be understood in the following manner: $B$ should be
thought of as an enzyme that degrades the substrate $A$. Without the
compartment model, the enzyme would simply disappear over time, and then the
substrate would grow without bound (from the reaction $0\to A$). However, each
compartment brings in a new enzyme allowing for the further degradation of
$A$.

######  Proposition 4.14:

In the RNIC model (10), $\mathcal{I}_{\mathcal{K}}$ is transient,
$\mathcal{H}_{\mathcal{K}}$ is positive recurrent on the irreducible state
space $\\{1,2,\dots\\}$, and $N$ (the coarse-grained model corresponding to
$\mathcal{F}$) is positive recurrent.

###### Proof.

$\mathcal{H}_{\mathcal{K}}$ is positive recurrent by Lemma 4.3.
$\mathcal{I}_{\mathcal{K}}$ is transient by the discussion above.

It just remains to check positive recurrence of $N$. For $n\in\mathcal{N}$,
let $C(n)=\left\lVert
n\right\rVert_{\ell^{1}}=\sum_{a=0}^{\infty}\sum_{b=0}^{\infty}n_{(a,b)}$
denote the number of compartments, and let
$A(n)=\sum_{a=0}^{\infty}\sum_{b=0}^{\infty}an_{(a,b)}$ and
$B(n)=\sum_{a=0}^{\infty}\sum_{b=0}^{\infty}bn_{(a,b)}$ be the total number of
$A$ and $B$ molecules, respectively, across all compartments. Define
$V:\mathcal{N}\to[0,\infty)$ via

$\displaystyle V(n)=\begin{cases}A(n)+B(n)+5C(n)-1&B(n)\neq 0\\\
A(n)+B(n)+5C(n)+7&B(n)=0.\end{cases}$

We claim that this is a Lyapunov function for $N$. An upper bound for
$\mathcal{L}V(n)$, the generator applied to $V$ at $n$, is given by

$\displaystyle\mathcal{L}V(n)\leq\begin{cases}-B(n)+7-15C(n)(C(n)-1)&B(n)\geq
2\text{ and }C(n)\geq 2\\\ 14-15C(n)(C(n)-1)&B(n)=1\text{ and }C(n)\geq 2\\\
-1-15C(n)(C(n)-1)&B(n)=0\\\ -A(n)(A(n)-1)B(n)-B(n)+7&B(n)\geq 2\text{ and
}C(n)=1\\\ -A(n)(A(n)-1)+14&B(n)=1\text{ and }C(n)=1\end{cases}$

Note that the first two rows are upper bounds and the last three rows are
exact. Specifically, in the first two rows we neglected the contribution of
the $2A+B\to B$ reaction — unlike everything else it crucially depends on how
the $A$ and $B$ molecules are distributed across the compartments.

We see that $\mathcal{L}V(n)\leq-1$ for all $n$ outside a finite set of
states—for instance, you could take the states where there is exactly one
compartment and it has at most $7$ $B$ and at most $4$ $A$. So $V$ is indeed a
Lyapunov function for $N$, and hence $N$ is positive recurrent by Theorem 4.8.
∎

$\triangle$

In the previous example we saw that even when $\kappa_{E}=0$, positive
recurrent compartments $\mathcal{H}_{\mathcal{K}}$ can still tame transient
chemistry $\mathcal{I}_{\mathcal{K}}$. It should not be surprising, then, that
positive recurrent compartments can tame null recurrent chemistry in the same
manner. For the sake of filling in Table 1 completely, we present a
modification of Example 4.13 where $\mathcal{I}_{\mathcal{K}}$ is null
recurrent instead of transient.

######  Example 4.15:

Consider the model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$
described by

${2A+B}$${B}$${0}$${A}$${2A}$${0}$${C}$${2C}$${\delta_{(0,1)}(a,b)}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{6}$
(11)

where $\delta_{(0,1)}$ is a point mass with zero $A$ molecules and one $B$
molecule.

The verification of this example is similar enough to that of Example 4.13
that we provide only a sketch.

######  Proposition 4.16:

In the RNIC model (11), $\mathcal{I}_{\mathcal{K}}$ is null recurrent on the
irreducible state space $\\{0,1,2,\dots\\}\times\\{0\\}$,
$\mathcal{H}_{\mathcal{K}}$ is positive recurrent on the irreducible state
space $\\{1,2,\dots\\}$, and $N$ (the coarse-grained model corresponding to
$\mathcal{F}$) is positive recurrent.

###### Proof Sketch.

Similarly to Example 4.13, $\mathcal{H}_{\mathcal{K}}$ is positive recurrent
and $\mathcal{I}_{\mathcal{K}}$ is eventually reduces (after all the $B$
molecules degrade) to the network

${0}$${A}$${2A.}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$

This model is null recurrent by Lemma 4.3.

As for $N$, let $V$ be the very same Lyapunov function used to prove positive
recurrence in Example 4.13. The only difference between this example and that
one is the addition of the reactions $A\to 0$ and $A\to 2A$. But notice that
the contribution of $A\to 0$ in $\mathcal{L}V(n)$ is $-A(n)$, and the
contribution of $A\to 2A$ is $A(n)$. These are equal and opposite, so
$\mathcal{L}V(n)$ is exactly the same in this example and Example 4.13. Thus
the remainder of the proof is identical. ∎

$\triangle$

Examples 4.11 and 4.13 showed that
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ can be
either positive recurrent or transient when $\kappa_{E}=0$ and
$\mathcal{I}_{\mathcal{K}}$ is transient. The next few examples are dedicated
to showing the same when $\mathcal{I}_{\mathcal{K}}$ is recurrent. First, if
new compartments enter with a huge number of molecules, it can overwhelm
otherwise positive recurrent chemistry:

######  Example 4.17:

Consider the RNIC model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$
described by

${0}$${S}$${0}$${C}$${2C}$${\mu,}$$\scriptstyle{\kappa_{b}}$$\scriptstyle{\kappa_{d}}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{F}}$$\scriptstyle{\kappa_{C}}$
(12)

where $\mu$ is not yet specified.

######  Proposition 4.18:

Let $N$ be the coarse-grained model associated with the RNIC model (12). For
any choice of non-negative rate constants such that $\kappa_{I}>0$, there is a
distribution $\mu$ on the non-negative integers such that $N$ is transient.

###### Proof.

We will show that in the case $\kappa_{b}=0$, $\mu$ can be chosen so that the
total number of $S$ molecules is itself a transient Markov chain. The case of
$\kappa_{b}>0$ then immediately follows by a coupling argument. That portion
of the proof is straightforward and is omitted.

Let $M(t)$ denote the number of $S$ molecules across all compartments at time
$t$. Under the assumption that $\kappa_{b}=0$, $M$ is a Markov chain which
transitions from state $m\in\mathbb{N}$ to state $m-1$ with rate $\kappa_{d}m$
and to state $m+j$ with rate $\kappa_{I}\mu(j)$.

Our plan is the following: we will recursively define an increasing sequence
of integers $m_{k}$ for $k=1,2,3,\dots$, and define $\mu(m_{k})=2^{-k}$ and
$\mu(j)=0$ otherwise. For $k=2,3,4,\dots$, we will let $A_{k}$ denote the
event that the process $M$ reaches $m_{k-1}$ before it reaches (or exceeds)
$m_{k+1}$. It then suffices to show that
$\sup_{k}\mathbb{P}_{m_{k}}(A_{k})<1/2$ to prove transience of $M$.

Continuing, we begin by letting $m_{1}=0$. Now suppose $m_{1},\dots,m_{k-1}$
have been defined. We will show that for any $\varepsilon>0$ it is possible to
pick $m_{k}$ so that $\mathbb{P}_{m_{k}}(A_{k})<\varepsilon$ regardless of the
values chosen for $m_{k+1},m_{k+2},\dots$. To show this, we make the following
observations.

  1. 1.

Since $M$ can only go down by one at a time, to get from $m_{k}$ to $m_{k-1}$
before hitting a state equal to or larger than $m_{k+1}$, the process must
visit every state $m_{k},m_{k}-1,\cdots,m_{k-1}+1$ at least once.

  2. 2.

On the event $A_{k}$, during each visit to each of the states
$m_{k-1}+1,\dots,m_{k}$ there was no transition of size $+m_{k+1}$ (for in
that case the state of $M$ would would necessarily reach or exceed $m_{k+1}$).

The probability of the process $M$ transitioning up by $m_{k+1}$ while in
state $m$ is $\frac{2^{-(k+1)}\kappa_{I}}{\kappa_{I}+\kappa_{d}m}$ because the
total rate out of state $m$ is $\kappa_{I}+\kappa_{d}m$, and the rate of
inflows of size $m_{k+1}$ in state $m$ is
$\mu(m_{k+1})\kappa_{I}=2^{-k-1}\kappa_{I}$. Hence, combining the above
observations we see

$\displaystyle\mathbb{P}_{m_{k}}(A_{k})$
$\displaystyle\leq\prod_{m=m_{k-1}+1}^{m_{k}}\left(1-\frac{2^{-(k+1)}\kappa_{I}}{\kappa_{I}+\kappa_{d}m}\right)$
$\displaystyle\leq\prod_{m=m_{k-1}+1}^{m_{k}}\exp\left(-\frac{2^{-(k+1)}\kappa_{I}}{\kappa_{I}+\kappa_{d}m}\right)=\exp\left(-2^{-(k+1)}\kappa_{I}\sum_{m=m_{k-1}+1}^{m_{k}}\frac{1}{\kappa_{I}+\kappa_{d}m}\right),$

where above we use the bound $1-x\leq e^{-x}$.

If $m_{k-1}$ is fixed and we send $m_{k}\to\infty$ in the sum above, we get
$\infty$ (it’s a tail of a harmonic series). Therefore,
$\mathbb{P}_{m_{k}}(A_{k})$ can be made as small as we like by choosing
$m_{k}$ big enough. We conclude that for appropriate choice of $m_{k}$, the
process $M$ is transient, and hence so is $N$. ∎

Hence, so long as $\kappa_{E}=0$, a distribution $\mu$ that is “bad enough”
can cause the whole model to be transient even if the chemical model
$\mathcal{I}_{\mathcal{K}}$ is positive recurrent. $\triangle$

In the previous example, the distribution $\mu$ of incoming compartments was
unbounded. As it turns out,
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ can be
transient even when $\mathcal{I}_{\mathcal{K}}$ and
$\mathcal{H}_{\mathcal{K}}$ are positive recurrent and $\mu$ is bounded. The
simplest, though not only, reason this can occur is the existence of some
conservation law, as the next example demonstrates. Put simply, the total
amount of species $A$ and $B$ is preserved by the chemistry, so any inflow of
those species, no matter how small, will overwhelm it.

######  Example 4.19:

Consider the RNIC model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$
described by

${A}$${B}$${0}$${C}$${2C}$${\mu}$$\scriptstyle{\kappa_{a}}$$\scriptstyle{\kappa_{b}}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{F}}$$\scriptstyle{\kappa_{C}}$
(13)

where $\mu$ is not yet specified.

######  Proposition 4.20:

Let $N$ be the coarse-grained model associated with the system $\mathcal{F}$
from (13). If $\mu$ is any measure on $\mathbb{Z}_{\geq 0}^{2}$ other than the
trivial measure $\delta_{(0,0)}$, then $N$ is transient even though all states
are positive recurrent for $\mathcal{I}_{\mathcal{K}}$. On the other hand, if
$\mu=\delta_{(0,0)}$ then $N$ is positive recurrent.

###### Proof.

$\mathcal{I}_{\mathcal{K}}$ is not irreducible, but when it is partitioned
into closed irreducible communicating classing, all are finite, and hence all
states are positive recurrent. As always when $\kappa_{E}=0$ but
$\kappa_{C}>0$, the empty state is transient for $\mathcal{H}_{\mathcal{K}}$
but all other states are positive recurrent.

For $n\in\mathcal{N}$, let
$S(n)=\sum_{a=0}^{\infty}\sum_{b=0}^{\infty}(a+b)n_{(a,b)}$ denote the sum of
the number of $A$ and $B$ molecules, combined across all compartments in $n$.

First suppose that $\mu\neq\delta_{(0,0)}$. Then $S(N(t))$ cannot shrink, and
grows with positive probability every time a compartment enters. So $N$ is
transient in this case.

Now suppose $\mu=\delta_{(0,0)}$. For $n\in\mathcal{N}$, let $C(n)=\left\lVert
n\right\rVert_{\ell^{1}}$ be the number of compartments in state $n$, and let
$V(n)=2C(n)$. Then

$\displaystyle\mathcal{L}V(n)=2\kappa_{I}+2\kappa_{F}C(n)-\kappa_{C}C(n)(C(n)-1),$

where $\mathcal{L}$ is the generator of $N$. This is less than $-1$ outside a
finite set because it is quadratic in $C(n)$ with negative leading term,
provided we restrict the state space to $\\{n\in\mathcal{N}:S(n)=S(N(0))\\}$.
So Theorem 4.8 applies and $N$ is positive recurrent, as claimed. ∎

$\triangle$

A natural question at this point is whether, if the behaviors in the last two
examples are ruled out, $N$ can still be transient when
$\mathcal{I}_{\mathcal{K}}$ and $\mathcal{H}_{\mathcal{K}}$ are both
separately recurrent. Specifically, if $\mathcal{I}_{\mathcal{K}}$ and
$\mathcal{H}_{\mathcal{K}}$ are both recurrent, there are no conservation
laws, and the number of molecules that an incoming compartment can have is
bounded, can $N$ be transient? The answer is yes, as the next example
demonstrates.

######  Example 4.21:

Consider the RNIC model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$
described by

${0}$${S}$${2S}$${0}$${C}$${2C}$${\delta_{1},}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{1}$
(14)

where $\delta_{1}$ is the point mass at one $S$.

######  Proposition 4.22:

Let $N$ be the coarse-grained model associated to the network
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ from
(14). Then $\mathcal{I}_{\mathcal{K}}$ is recurrent with no conservation laws
and the number of molecules in new compartments is bounded, however every
state is transient for $N$.

###### Proof.

$\mathcal{I}_{\mathcal{K}}$ is (null) recurrent, and
$\mathcal{H}_{\mathcal{K}}$ is positive recurrent on the irreducible state
space $\\{1,2,\dots\\}$, by Lemma 4.3.

It remains to show that every state is transient for $N$. As in all examples
with $\kappa_{E}=0$, the state with zero compartments can never be returned to
and we restrict the state space of the chain to $\mathcal{N}\setminus\\{0\\}$.
With this assumption the state space is a closed irreducible set, so it
suffices to pick one state and show that it is transient. We will show $e_{0}$
(the state with one empty compartment) is transient. Denoting a state of $N$
by $n$, let $C(n)=\sum_{x=0}^{\infty}n_{x}$ and
$S(n)=\sum_{x=0}^{\infty}x\cdot n_{x}$ denote the total number of compartments
and $S$ molecules, respectively. Define $V:\mathcal{N}\to[0,1]$ by

$V(n)=\frac{S(n)}{1+S(n)}.$

If $\mathcal{L}$ denotes the generator of $N$, notice that

$\displaystyle\mathcal{L}V(n)$
$\displaystyle=(C(n)+S(n)+1)\left(\frac{S(n)+1}{S(n)+2}-\frac{S(n)}{S(n)+1}\right)+S(n)\left(\frac{S(n)-1}{S(n)}-\frac{S(n)}{S(n)+1}\right)$
$\displaystyle=\frac{C(n)+S(n)+1}{(S(n)+2)(S(n)+1)}-\frac{1}{S(n)+1}$
$\displaystyle=\frac{C(n)-1}{(S(n)+2)(S(n)+1)}$ $\displaystyle\geq 0$

for all $n\in\mathcal{N}\setminus\\{0\\}$. In particular, if $B=\\{e_{0}\\}$,
we can apply Theorem 4.9 to conclude that when $N$ is started from
$e_{0}+e_{1}$ (the state with two compartments, one empty and the other with
one $S$), then the probability of reaching $B$ is less than $1$. But when $N$
is started from $e_{0}$, it reaches $e_{0}+e_{1}$ with positive probability
(the transition from $e_{0}$ to $e_{0}+e_{1}$ corresponds to an inflow event).
Putting these together, when $N$ is started from $e_{0}$ it fails to return
with positive probability, and hence $e_{0}$ is transient. As discussed, this
is enough to conclude that all states are transient for $N$. ∎

$\triangle$

In the previous example $\mathcal{I}_{\mathcal{K}}$ was _null_ recurrent. One
may still be tempted to think that perhaps if it were _positive_ recurrent
then the whole process must be. The next example demonstrates that even this
is not guaranteed.

######  Example 4.23:

Consider the compartment model described by

${A+B}$${0}$${B}$${0}$${C}$${2C}$${\delta_{(m,0)}(a,b)}$${2B}$${A}$$\scriptstyle{10}$$\scriptstyle{2}$$\scriptstyle{1}$$\scriptstyle{1}$$\scriptstyle{2}$$\scriptstyle{10}$
(15)

where $m$ is some non-negative integer and $\delta_{(m,0)}$ is the point mass
at $m$ molecules of $A$ and zero of $B$. Let $\gamma>0$ denote the expected
number of compartments in stationarity.

######  Proposition 4.24:

Let $\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ be
the compartment model from (15), and let $N$ be the associated coarse-grained
model. Then $\mathcal{I}_{\mathcal{K}}$ is positive recurrent, but $N$ is
transient when $m>\gamma$.

###### Proof.

That $\mathcal{I}_{\mathcal{K}}$ is positive recurrent is witnessed by the
Lyapunov function

$\displaystyle V(a,b)=\begin{cases}3a+3&b=0\\\ 3a+3b-2&b\geq 1\end{cases}$

Indeed, if $\mathcal{A}$ denotes the generator of $\mathcal{I}_{\mathcal{K}}$,
then

$\displaystyle\mathcal{A}V(a,b)$ $\displaystyle=\begin{cases}3(1)-2(2)&b=0\\\
3(1)+3(2)-1(10a)&b=1\\\ 3(1)+3(2)-6(20a)-1(10)&b=2\\\
3(1)+3(2)-6(10ab)-6(5b(b-1))&b\geq 3\end{cases}$
$\displaystyle=\begin{cases}-1&b=0\\\ 9-10a&b=1\\\ -1-120a&b=2\\\
9-60ab-30b(b-1)&b\geq 3\end{cases}$

This is at most $-1$ away from $(0,1)$, so by Theorem 4.8
$\mathcal{I}_{\mathcal{K}}$ is positive recurrent.

Now regarding transience of $N$, let $F^{\mathrm{sim}}$ be the simulation
representation of $\mathcal{F}$, so that $N$ is a fuction of
$F^{\mathrm{sim}}$. Let $X_{A}$ and $X_{B}$ denote the total number of $A$ and
$B$ molecules, respectively, across all compartments in $N$ (equivalently,
across all compartments in $F^{\mathrm{sim}}$). To show that $N$ is transient,
we will show that $X_{A}(t)\to\infty$ a.s., as $t\to\infty$. To do this, we
will make use of the construction of $F^{\mathrm{sim}}$ from section 2.3.2.
Let $Y_{I}$ and $Y_{C}$ be as in that section, so that the process $M_{C}$ for
the number of compartments is given by

$\displaystyle M_{C}(t)$
$\displaystyle=M_{C}(0)+Y_{I}\left(t\right)-Y_{C}\left(\int_{0}^{t}\frac{M_{C}(s)(M_{C}(s)-1)}{2}ds\right).$

Similarly, for $r\in\\{A+B\to 0,0\to B,2B\to 0,0\to A\\}$ let $Y_{r}$ be as in
section 2.3.2, and let $R_{r}$ be the associated counting process for the
number of times reaction $r$ has occurred across all compartments, so that

$R_{r}(t)=Y_{r}\left(\mathop{\sum_{i\geq 0}}_{T_{i}\leq
t}\int_{T_{i}}^{T_{i+1}\wedge
t}\sum_{j=1}^{M_{C}(T_{i})}\lambda_{r}(X^{i}_{j}(s))ds\right),$

where the $T_{i}$ are the jump times of the process $M_{C}$, $X_{j}^{i}(s)$ is
the state of the process in compartment $j$ at time $s$, and $\lambda_{r}$ is
given as in (2). Then

$\displaystyle X_{A}(t)$ $\displaystyle=X_{A}(0)+R_{0\to
A}(t)+mY_{I}\left(t\right)-R_{A+B\to 0}(t)$ $\displaystyle=X_{A}(0)+Y_{0\to
A}\left(\int_{0}^{t}M_{C}(s)ds\right)+mY_{I}\left(t\right)-R_{A+B\to 0}(t).$

Notice that in the last line above we were able to simplify the expression for
$R_{0\to A}$ in terms of $Y_{0\to A}$ from the expression given above for
$R_{r}$ in general. This was done by making use of the fact that the total
rate of this reaction across all compartments, $\sum_{j}\lambda_{0\to
A}(X_{j}^{i}(s))$, is exactly the total number of compartments $M_{C}(s)$. We
cannot hope to do the same for $R_{A+B\to 0}$ because the rate of that
reaction depends on how the molecules are distributed across the compartments.
However, notice that the total number of times the reaction $A+B\to 0$ fires
is at most the total number of $B$ molecules ever present in the system:

$\displaystyle R_{A+B\to 0}(t)$ $\displaystyle\leq X_{B}(0)+R_{0\to B}(t)$
$\displaystyle=X_{B}(0)+Y_{0\to B}\left(2\int_{0}^{t}M_{C}(s)ds\right).$

Therefore,

$\displaystyle X_{A}(t)$ $\displaystyle\geq X_{A}(0)-X_{B}(0)+Y_{0\to
A}\left(\int_{0}^{t}M_{C}(s)ds\right)+mY_{I}\left(t\right)-Y_{0\to
B}\left(2\int_{0}^{t}M_{C}(s)ds\right).$

Recall that $\gamma$ denotes the expected number of $C$ in the CRN
$\mathcal{H}_{\mathcal{K}}$ at stationarity. By the CTMC ergodic theorem (see
Theorem 45 in Chapter 4 of [25]), $\frac{1}{t}\int_{0}^{t}M_{C}(s)ds\to\gamma$
almost surely as $t\to\infty$. This will matter in its own right; it also
follows that $\int_{0}^{t}M_{C}(s)ds\to\infty$ a.s. as $t\to\infty$. It is a
standard fact about unit Poisson processes $Y$ that $Y(t)/t\to 1$ a.s. as
$t\to\infty$. Composing this Poisson limit with the limit from the previous
sentence, we get that

$\displaystyle\frac{Y_{0\to
B}\left(2\int_{0}^{t}M_{C}(s)ds\right)}{2\int_{0}^{t}M_{C}(s)ds}\to 1$

a.s. as $t\to\infty$, and similarly for $Y_{0\to A}$. Putting this all
together we have

$\displaystyle\lim_{t\to\infty}\frac{X_{A}(t)}{t}$
$\displaystyle\geq\lim_{t\to\infty}\Bigg{[}\frac{Y_{0\to
A}\left(\int_{0}^{t}M_{C}(s)ds\right)}{\int_{0}^{t}M_{C}(s)ds}\cdot\frac{1}{t}\int_{0}^{t}M_{C}(s)ds+m\frac{Y_{I}(t)}{t}$
$\displaystyle\hskip 72.26999pt-\frac{Y_{0\to
B}\left(2\int_{0}^{t}M_{C}(s)ds\right)}{2\int_{0}^{t}M_{C}(s)ds}\cdot\frac{2}{t}\int_{0}^{t}M_{C}(s)ds\Bigg{]}$
$\displaystyle=\gamma+m-2\gamma.$

almost surely. Therefore, as long as the integer $m$ is (strictly) larger than
$\gamma$, $X_{A}(t)/t$ is converging almost surely to a positive number. In
this case $X_{A}(t)\to\infty$ a.s. as $t\to\infty$, and hence $N$ is
transient. ∎

Note that the above example shows the potential usefulness of the RNIC
representation provided in section 2.3.2. $\triangle$

##  5 Stationary Distribution in a Special Case

In light of Theorem 4.5, whenever $\mathcal{H}_{\mathcal{K}}$ is positive
recurrent and $\kappa_{E}>0$, then $N$, the coarse-grained model associated to
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$, is
positive recurrent for at least some states. In this case, the standard theory
of Markov chains tells us that there is a stationary distribution supported on
those states. Ideally, it would be possible to write down a formula for this
stationary distribution in terms of information about the CRNs
$\mathcal{I}_{\mathcal{K}}$ and $\mathcal{H}_{\mathcal{K}}$. Under the further
assumption that $\kappa_{C}=0=\kappa_{F}$ (so that compartments are not
interacting), we are able to do so.

######  Theorem 5.1:

Consider a non-explosive model
$\mathcal{F}=(\mathcal{I}_{\mathcal{K}},\mathcal{H}_{\mathcal{K}},\mu)$ with
$\kappa_{F}=\kappa_{C}=0$, and $\kappa_{E}>0$:

${\mathcal{I}_{\mathcal{K}}}$${0}$${C}$${\mu}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$

Let $N$ be the coarse-grained model associated to $\mathcal{F}$. For
$x\in\mathbb{Z}_{\geq 0}^{d}$ and $t\in[0,\infty)$, let $P_{\mu}(x,t)$ denote
the probability that $\mathcal{I}_{\mathcal{K}}$ is in state $x$ at time $t$
when started from time zero with initial distribution $\mu$. For
$x\in\mathbb{Z}_{\geq 0}^{d}$ define $\alpha(x)$ via

$\displaystyle\alpha(x)=\int_{0}^{\infty}P_{\mu}(x,t)\kappa_{E}e^{-\kappa_{E}t}\mathrm{d}t,$

and define a distribution $\pi$ on $\mathcal{N}$ via

$\displaystyle\pi(n)=\left(\prod_{x\in\mathbb{Z}_{\geq
0}^{d}}\frac{\alpha(x)^{n_{x}}}{n_{x}!}\right)\cdot\left[e^{-\kappa_{I}/\kappa_{E}}\cdot\left(\frac{\kappa_{I}}{\kappa_{E}}\right)^{\left\lVert
n\right\rVert_{\ell^{1}}}\right]$

Then $\pi$ is the unique stationary distribution for $N$.

######  Remark 5.2:

To apply Theorem 5.1, one needs to know, not just the stationary distribution
for the chemistry, but the distribution for all time. This restriction may
seem daunting, and indeed for many models this distribution is not known. One
class of models where it is know are the DR models of [6]. A second class of
models are monomolecular reaction networks with arbitrary initial conditions —
see [18]. Note that [6] allows for more general networks (all monomolecular
networks satisfy the DR condition), but [18] allows for more general initial
conditions (the DR paper requires Poisson initial conditions).

###### Proof of Theorem 5.1..

Note that by Theorem 4.5, any state which is reachable from the zero state is
positive recurrent, and all other states are transient. Furthermore, notice
that $N$ is irreducible if restricted to the set of states which are reachable
from the zero state, since zero is reachable from any state. Thus there is a
unique stationary distribution. To prove that the $\pi$ given above is indeed
this unique stationary distribution, it suffices to show that $\pi$ is a
distribution and $\pi Q=0$, where $Q$ is the transition rate matrix for $N$.
That $\pi$ is a distribution follows from the fact that $\alpha$ is a
distribution, which we will check later in the proof. So fix
$n\in\mathcal{N}$; we wish to show that
$\sum_{n^{\prime}\in\mathcal{N}}\pi(n^{\prime})q(n^{\prime},n)=0$.

Note that there are only three possible types of transitions: inflow of
compartment, outflow of compartment, and transition of reaction network.
Expanding the sum above into three terms, one for each of these types of
transitions, the desired equality can be written

$\displaystyle\sum_{x\in\mathbb{Z}_{\geq 0}^{d}}\Bigg{[}\pi(n-e_{x})$
$\displaystyle q(n-e_{x},n)+\pi(n+e_{x})q(n+e_{x},n)$
$\displaystyle+\sum_{j}\pi(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}})q(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}},n)\Bigg{]}$
$\displaystyle=\pi(n)\sum_{x\in\mathbb{Z}_{\geq
0}^{d}}\left(q(n,n+e_{x})+q(n,n-e_{x})+\sum_{j}q(n,n-e_{x}+e_{x+\nu^{\prime}_{j}-\nu_{j}})\right)$

or

$\displaystyle\sum_{x\in\mathbb{Z}_{\geq 0}^{d}}\Bigg{[}\pi(n-e_{x})$
$\displaystyle\kappa_{I}\mu(x)+\pi(n+e_{x})\kappa_{E}(n_{x}+1)$
$\displaystyle+\sum_{j}\pi(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}})(n_{x-\nu^{\prime}_{j}+\nu_{j}}+1)\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}\Bigg{]}$
(16) $\displaystyle=\pi(n)\sum_{x\in\mathbb{Z}_{\geq
0}^{d}}\left(\kappa_{I}\mu(x)+\kappa_{E}n_{x}+\sum_{j}n_{x}\kappa_{j}\binom{x}{\nu_{j}}\right)$

To prove this equality, we will consider two cases. Suppose first that $n$ is
such that $n_{y}>0$ for some $y\in\mathbb{Z}_{\geq 0}^{d}$ with $\alpha(y)=0$,
and fix such a $y$. Then $\alpha(y)$ participates in the product defining
$\pi(n)$, and hence $\pi(n)=0$. Thus the right-hand side of (16) is zero; we
claim that the left-hand side is also zero. Specifically, we will argue for
each $x$ and each $j$, each of the three terms in the sum is zero. So fix $x$
and $j$:

  * •

$\pi(n-e_{x})\kappa_{I}\mu(x)$: Notice that if $x\neq y$ then $\pi(n-e_{x})=0$
for the same reason that $\pi(n)=0$. If $x=y$ then $\mu(x)=0$, since if
$\mu(y)>0$ it would be the case that $P_{\mu}(y,t)>0$ for all small enough
$t$, and hence the integral defining $\alpha(y)$ would be positive.

  * •

$\pi(n+e_{x})\kappa_{E}(n_{x}+1)$: Regardless of $x$, $\pi(n+e_{x})=0$ for the
same reason that $\pi(n)=0$.

  * •

$\pi(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}})(n_{x-\nu^{\prime}_{j}+\nu_{j}}+1)\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}$:
As before, if $x\neq y$ then $\pi(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}})=0$.
Suppose towards a contradiction that
$\pi(n-e_{y}+e_{y-\nu^{\prime}_{j}+\nu_{j}})\neq 0$ and that
$\kappa_{j}\binom{y-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}\neq 0$. Then
$\pi(n-e_{y}+e_{y-\nu^{\prime}_{j}+\nu_{j}})\neq 0$ implies that
$\alpha(y-\nu^{\prime}_{j}+\nu_{j})\neq 0$, and hence
$P_{\mu}(y-\nu^{\prime}_{j}+\nu_{j},t)\neq 0$ for some $t$. But this means
that the state $y-\nu^{\prime}_{j}+\nu_{j}$ is reachable for $\mathcal{I}$
when started with initial distribution $\mu$. But
$\kappa_{j}\binom{y-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}\neq 0$ implies that $y$
is reachable from $y-\nu^{\prime}_{j}+\nu_{j}$ for $\mathcal{I}$ via the
$j$-th reaction, so we conclude that $y$ is reachable from $\mu$. But this
implies that $P_{\mu}(y,t)\neq 0$ for $t>0$, which in turn means that
$\alpha(y)>0$. This contradicts our choice of $y$, so it must be that our
assumption was wrong: either $\pi(n-e_{y}+e_{y-\nu^{\prime}_{j}+\nu_{j}})=0$
or $\kappa_{j}\binom{y-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}=0$. But either of
those imply the desired equality
$\pi(n-e_{y}+e_{y-\nu^{\prime}_{j}+\nu_{j}})(n_{y-\nu^{\prime}_{j}+\nu_{j}}+1)\kappa_{j}\binom{y-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}=0$.

This proves that (16) reduces to $0=0$ in this case. The reminder of the proof
will be devoted to the second case; namely, the case where $n$ is such that
$n_{y}=0$ for all $y\in\mathbb{Z}_{\geq 0}^{d}$ with $\alpha(y)=0$.

Let $\mathbb{X}=\\{x\in\mathbb{Z}_{\geq 0}^{d}:\alpha(x)\neq 0\\}$. We claim
that for every $x\notin\mathbb{X}$ and every $j$, every summand in (16) is
zero. So fix $x\notin\mathbb{X}$ and $j$:

  * •

$\pi(n-e_{x})\kappa_{I}\mu(x)$: Since $\alpha(x)=0$, by choice of $n$ we have
$n_{x}=0$. But this means that $n-e_{x}$ is negative at $x$ and hence
$n-e_{x}\notin\mathcal{N}$, so $\pi(n-e_{x})=0$.

  * •

$\pi(n+e_{x})\kappa_{E}(n_{x}+1)$: Notice that $\alpha(x)=0$ participates in
the product defining $\pi(n+e_{x})$, and hence $\pi(n+e_{x})=0$.

  * •

$\pi(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}})(n_{x-\nu^{\prime}_{j}+\nu_{j}}+1)\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}$:
As before, $n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}}\notin\mathcal{N}$ and hence
$\pi(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}})=0$.

  * •

$\kappa_{I}\mu(x)$: Since $\alpha(x)=0$, it must be the case that $\mu(x)=0$,
as otherwise $P_{\mu}(x,t)$ would be positive for sufficiently small $t$.

  * •

$\kappa_{E}n_{x}$: Since $\alpha(x)=0$, by choice of $n$ we have $n_{x}=0$.

  * •

$n_{x}\kappa_{j}\binom{x}{\nu_{j}}$: Once again, $n_{x}=0$.

Thus we have shown that terms with $x\notin\mathbb{X}$ do not contribute to
(16). So to complete the proof, we have only to show that

$\displaystyle\sum_{x\in\mathbb{X}}\Bigg{[}$
$\displaystyle\pi(n-e_{x})\kappa_{I}\mu(x)+\pi(n+e_{x})\kappa_{E}(n_{x}+1)$
$\displaystyle+\sum_{j}\pi(n-e_{x}+e_{x-\nu^{\prime}_{j}+\nu_{j}})(n_{x-\nu^{\prime}_{j}+\nu_{j}}+1)\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}\Bigg{]}$
(17) $\displaystyle\hskip
108.405pt=\pi(n)\sum_{x\in\mathbb{X}}\left(\kappa_{I}\mu(x)+\kappa_{E}n_{x}+\sum_{j}n_{x}\kappa_{j}\binom{x}{\nu_{j}}\right).$

Let $x\in\mathbb{X}$ be arbitrary. Integration by parts gives

$\displaystyle\int_{0}^{\infty}\left(\frac{\mathrm{d}}{\mathrm{d}t}P_{\mu}(x,t)\right)\kappa_{E}e^{-\kappa_{E}t}\mathrm{d}t$
$\displaystyle=\kappa_{E}e^{-\kappa_{E}t}P_{\mu}(x,t)\Big{|}_{t=0}^{t=\infty}-\int_{0}^{\infty}P_{\mu}(x,t)(-\kappa_{E}^{2}e^{-\kappa_{E}t})\mathrm{d}t$
$\displaystyle=-\kappa_{E}\mu(x)+\kappa_{E}\alpha(x).$

Because $P_{\mu}$ is the distribution for $\mathcal{I}_{\mathcal{K}}$, the
Kolmogorov forward equations for $\mathcal{I}_{\mathcal{K}}$ tell us that

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}P_{\mu}(x,t)$
$\displaystyle=\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}P_{\mu}(x-\nu_{j}^{\prime}+\nu_{j},t)-\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x}{\nu_{j}}P_{\mu}(x,t)$

for each $t$. Plugging this in above and rearranging yields

$\displaystyle\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}\alpha(x-\nu_{j}^{\prime}+\nu_{j})-\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x}{\nu_{j}}\alpha(x)$
$\displaystyle=-\kappa_{E}\mu(x)+\kappa_{E}\alpha(x)$
$\displaystyle\kappa_{E}\frac{\mu(x)}{\alpha(x)}+\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}\frac{\alpha(x-\nu_{j}^{\prime}+\nu_{j})}{\alpha(x)}$
$\displaystyle=\kappa_{E}+\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x}{\nu_{j}}.$

Note that we did not divide by zero in the second line because $\alpha(x)\neq
0$ by definition of $\mathbb{X}$. Since $x\in\mathbb{X}$ was arbitrary, we can
multiply through by $n_{x}$ and sum over $x$, which yields

$\displaystyle\sum_{x\in\mathbb{X}}\bigg{(}n_{x}\kappa_{E}\frac{\mu(x)}{\alpha(x)}+n_{x}\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}$
$\displaystyle\frac{\alpha(x-\nu_{j}^{\prime}+\nu_{j})}{\alpha(x)}\bigg{)}$
$\displaystyle=\sum_{x\in\mathbb{X}}\bigg{(}n_{x}\kappa_{E}+n_{x}\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x}{\nu_{j}}\bigg{)}.$
(18)

Now we claim that $\mu$ and $\alpha$ are both probability measures supported
on $\mathbb{X}$. We know that $\mu$ is a probability measure by assumption; it
is supported on $\mathbb{X}$ because if $\mu(x)>0$ then $P_{\mu}(x,t)>0$ for
small enough $t$ and hence $\alpha(x)>0$. We know that $\alpha$ is supported
on $\mathbb{X}$ by definition of $\mathbb{X}$; to see that it is a probability
measure, use the fact that the integrand in the definition of $\alpha$ is non-
negative to interchange a sum over $x$ with the integral and then apply the
fact that $P_{\mu}(x,t)$ is a probability measure for each $t$. Therefore
$\mu$ and $\alpha$ are both probability measures supported on $\mathbb{X}$, as
claimed; it follows that
$\sum_{x\in\mathbb{X}}\kappa_{I}\mu(x)=\kappa_{I}=\sum_{x\in\mathbb{X}}\kappa_{I}\alpha(x)$.
So adding $\kappa_{I}$ to both sides of (18) gives

$\displaystyle\sum_{x\in\mathbb{X}}\bigg{(}n_{x}\kappa_{E}\frac{\mu(x)}{\alpha(x)}+\kappa_{I}\alpha(x)+n_{x}\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}$
$\displaystyle\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}\frac{\alpha(x-\nu_{j}^{\prime}+\nu_{j})}{\alpha(x)}\bigg{)}$
$\displaystyle=\sum_{x\in\mathbb{X}}\bigg{(}\kappa_{I}\mu(x)+n_{x}\kappa_{E}+n_{x}\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x}{\nu_{j}}\bigg{)}.$
(19)

Now notice that, directly from the definition of $\pi$, we have

$\displaystyle\frac{\pi(n-e_{x})}{\pi(n)}$
$\displaystyle=\frac{n_{x}}{\alpha(x)}\frac{\kappa_{E}}{\kappa_{I}}$
$\displaystyle\frac{\pi(n+e_{x})}{\pi(n)}$
$\displaystyle=\frac{\alpha(x)}{n_{x}+1}\frac{\kappa_{I}}{\kappa_{E}}$
$\displaystyle\frac{\pi(n-e_{x}+e_{x-\nu_{j}^{\prime}+\nu_{j}})}{\pi(n)}$
$\displaystyle=\frac{\alpha(x-\nu_{j}^{\prime}+\nu_{j})}{\alpha(x)}\frac{n_{x}}{n_{x-\nu_{j}^{\prime}+\nu_{j}}+1},$

where the last equality holds for each reaction $\nu_{j}\to\nu_{j}^{\prime}$.
Applying these three in order on the left-hand side of (19), we get

$\displaystyle\sum_{x\in\mathbb{X}}$
$\displaystyle\bigg{(}\kappa_{I}\mu(x)\frac{\pi(n-e_{x})}{\pi(n)}+\kappa_{E}(n_{x}+1)\frac{\pi(n+e_{x})}{\pi(n)}$
$\displaystyle+\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}(n_{x-\nu_{j}^{\prime}+\nu_{j}}+1)\frac{\pi(n-e_{x}+e_{x-\nu_{j}^{\prime}+\nu_{j}})}{\pi(n)}\bigg{)}$
$\displaystyle=\sum_{x\in\mathbb{X}}\bigg{(}\kappa_{I}\mu(x)+n_{x}\kappa_{E}+n_{x}\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x}{\nu_{j}}\bigg{)}$
$\displaystyle\sum_{x\in\mathbb{X}}$
$\displaystyle\bigg{(}\kappa_{I}\mu(x)\pi(n-e_{x})+\kappa_{E}(n_{x}+1)\pi(n+e_{x})$
$\displaystyle+\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x-\nu^{\prime}_{j}+\nu_{j}}{\nu_{j}}(n_{x-\nu_{j}^{\prime}+\nu_{j}}+1)\pi(n-e_{x}+e_{x-\nu_{j}^{\prime}+\nu_{j}})\bigg{)}$
$\displaystyle=\pi(n)\sum_{x\in\mathbb{X}}\bigg{(}\kappa_{I}\mu(x)+n_{x}\kappa_{E}+n_{x}\sum_{\nu_{j}\to\nu_{j}^{\prime}}\kappa_{j}\binom{x}{\nu_{j}}\bigg{)},$

which is exactly the desired equality, (17). ∎

Let us now consider some examples of applying this result.

######  Example 5.3:

Let $\lambda\geq 0$, and consider the compartment system

${0}$${S}$${0}$${C}$${\operatorname{Poisson}(\lambda)}$$\scriptstyle{\kappa_{b}}$$\scriptstyle{\kappa_{d}}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$

Then the stationary distribution of the system is given by

$\displaystyle\pi(n)=\left(\prod_{x=0}^{\infty}\frac{\alpha(x)^{n_{x}}}{n_{x}!}\right)\cdot\left[e^{-\kappa_{I}/\kappa_{E}}\cdot\left(\frac{\kappa_{I}}{\kappa_{E}}\right)^{\left\lVert
n\right\rVert_{\ell^{1}}}\right],$

where

$\displaystyle\alpha(x)=\int_{0}^{\infty}\exp\left\\{-(\lambda-\kappa_{b}/\kappa_{d})e^{-\kappa_{d}t}-\kappa_{b}/\kappa_{d}\right\\}\frac{((\lambda-\kappa_{b}/\kappa_{d})e^{-\kappa_{d}t}+\kappa_{b}/\kappa_{d})^{x}}{x!}\kappa_{E}e^{-\kappa_{E}t}\mathrm{d}t.$

###### Proof.

Check that the distribution

$\displaystyle
P_{\lambda}(x,t):=\exp\left\\{-(\lambda-\kappa_{b}/\kappa_{d})e^{-\kappa_{d}t}-\kappa_{b}/\kappa_{d}\right\\}\frac{((\lambda-\kappa_{b}/\kappa_{d})e^{-\kappa_{d}t}+\kappa_{b}/\kappa_{d})^{x}}{x!}$

is $\operatorname{Poisson}(\lambda)$ at time $t=0$ and satisfies

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}P_{\lambda}(x,t)$
$\displaystyle=\kappa_{b}P_{\lambda}(x-1,t)+\kappa_{d}(x+1)P_{\lambda}(x+1,t)-\kappa_{b}P_{\lambda}(x,t)-\kappa_{d}xP_{\lambda}(x,t)$

for each $x$ and $t$, and apply Theorem 5.1. ∎

$\triangle$

In the previous example, notice that the expected value of $\alpha$ is

$\displaystyle\sum_{x=0}^{\infty}x\alpha(x)$
$\displaystyle=\int_{0}^{\infty}\sum_{x=0}^{\infty}\exp\left\\{-(\lambda-\kappa_{b}/\kappa_{d})e^{-\kappa_{d}t}-\kappa_{b}/\kappa_{d}\right\\}\frac{((\lambda-\kappa_{b}/\kappa_{d})e^{-\kappa_{d}t}+\kappa_{b}/\kappa_{d})^{x+1}}{x!}\kappa_{E}e^{-\kappa_{E}t}\mathrm{d}t$
$\displaystyle=\int_{0}^{\infty}(\lambda-\kappa_{b}/\kappa_{d})\kappa_{E}e^{-(\kappa_{d}+\kappa_{E})t}+\frac{\kappa_{b}\kappa_{E}}{\kappa_{d}}e^{-\kappa_{E}t}\mathrm{d}t$
$\displaystyle=\frac{(\lambda-\kappa_{b}/\kappa_{d})\kappa_{E}}{\kappa_{d}+\kappa_{E}}+\frac{\kappa_{b}}{\kappa_{d}}$
$\displaystyle=\frac{\lambda\kappa_{E}+\kappa_{b}}{\kappa_{d}+\kappa_{E}}.$

This matches [11], where the same example is consider in section 2.A (see
specifically their equation [20] and the following discussion). Note that in
[11], though the expected value of $\alpha$ is calculated in general, an
explicit formula for $\alpha(x)$ (in their notation, $P_{\infty}(x)$) is given
in only two cases. The first is the case where
$\lambda=\kappa_{b}/\kappa_{d}$, where (in section S7.4 of their SI Appendix)
they remark that $\alpha$ is Poission with mean $\lambda$. This matches the
formula we give above in Example 5.3. The second case they cover is the one
where $\kappa_{d}=0$. In that case they obtain

$\displaystyle\alpha(x)=(1-\xi)\xi^{x}e^{\lambda(1/\xi-1)}\frac{\Gamma(1+x,\lambda/\xi)}{x!},$

where $\xi=\kappa_{b}/(\kappa_{b}+\kappa_{E})$ and $\Gamma$ is the upper
incomplete Gamma function. One can check that this agrees with our next
example, Example 5.4, in the case where $\mu$ is taken to be Poission with
parameter $\lambda$ by applying the binomial theorem in our formula and then
making a change of variable in the integral.

The following example is interesting for a few reasons. First, the chemistry
is not converging to any sort of stationary distribution, and yet the whole
compartment model is. Second, notice that when $\mu$ is not a Poisson
distribution, $P_{\mu}(x,t)$ is not a Poisson distribution in $x$ for all $t$
unlike the previous example or more generally the DR models from Remark 5.2.
Third, as discussed above, it generalizes an example from [11].

######  Example 5.4:

Let $\mu$ be a probability distribution on $\mathbb{Z}_{\geq 0}$, and consider
the compartment system

${0}$${S}$${0}$${C}$${\mu,}$$\scriptstyle{\kappa_{b}}$$\scriptstyle{\kappa_{I}}$$\scriptstyle{\kappa_{E}}$

Then the stationary distribution of the system is given by

$\displaystyle\pi(n)=\left(\prod_{x=0}^{\infty}\frac{\alpha(x)^{n_{x}}}{n_{x}!}\right)\cdot\left[e^{-\kappa_{I}/\kappa_{E}}\cdot\left(\frac{\kappa_{I}}{\kappa_{E}}\right)^{\left\lVert
n\right\rVert_{\ell^{1}}}\right],$

where

$\displaystyle\alpha(x)=\int_{0}^{\infty}e^{-\kappa_{b}t}\left(\sum_{m=0}^{x}\frac{\kappa_{b}^{m}t^{m}}{m!}\mu(x-m)\right)\kappa_{E}e^{-\kappa_{E}t}\mathrm{d}t.$

###### Proof.

Check that the distribution

$\displaystyle
P_{\mu}(x,t):=e^{-\kappa_{b}t}\left(\sum_{m=0}^{x}\frac{\kappa_{b}^{m}t^{m}}{m!}\mu(x-m)\right)$

satisfies

$\displaystyle\frac{\mathrm{d}}{\mathrm{d}t}P_{\mu}(x,t)$
$\displaystyle=\kappa_{b}P_{\mu}(x-1,t)-\kappa_{b}P_{\mu}(x,t),$

with initial condition $P_{\mu}(x,0)=\mu(x)$, and apply Theorem 5.1. ∎

$\triangle$

##  6 Appendix

The purpose of this section is to prove Theorem 4.9, which we recall says the
following:

######  Theorem 6.1:

Let $X$ be a non-explosive continuous-time Markov chain on a countable
discrete state space $\mathbb{S}$ with generator $\mathcal{L}$. Let
$B\subset\mathbb{S}$, and let $\tau_{B}$ be the time for the process to enter
$B$. Suppose there is some bounded function $V$ such that for all $x\in
B^{c}$,

$\mathcal{L}V(x)\geq 0.$

Then $\mathbb{P}_{x_{0}}(\tau_{B}<\infty)<1$ for any $x_{0}$ such that

$\sup_{x\in B}V(x)<V(x_{0}).$

Just like the theorem itself, the proof draws heavy inspiration from [27].
Before providing the proof, we state the following well-known result:

######  Lemma 6.2 (Dynkin’s Formula):

Suppose $X$ is a Markov chain with finite state space $\mathbb{S}$, and let
$\mathcal{L}$ be the generator of $X$. Then for any a.s. bounded stopping time
$\tau$ and any $x\in\mathbb{S}$, we have

$\mathbb{E}_{x}[f(X_{\tau})]=f(x)+\mathbb{E}_{x}\left[\int_{0}^{\tau}\mathcal{L}f(X_{s})\mathrm{d}s\right]$

In fact Dynkin’s Formula is well-known in much greater generality than what is
stated above, but as stated it is not hard to prove and is enough for our
purposes.

###### Proof of Theorem 4.9.

Define $W$ on $\mathbb{S}$ via $W=V-\sup_{x\in B}V(x)$. Notice that $W(x_{0})$
is strictly positive, $W$ is nonpositive on $B$, and
$\mathcal{L}W=\mathcal{L}V$. Fix some enumeration of $\mathbb{S}$ in which
$x_{0}$ is the first element, and for $m\in\mathbb{N}$ let $\mathbb{S}_{m}$
denote the first $m$ elements of $\mathbb{S}$. Let $\tau_{m}$ be the first
time $X$ is not in $\mathbb{S}_{m}$. Let $\Delta$ be a new state not in
$\mathbb{S}$, and for $m\in\mathbb{N}$ define a new Markov chain $X^{m}$ via

$X^{m}_{t}=\begin{cases}X_{t}&t<\tau_{m}\\\ \Delta&t\geq\tau_{m}\end{cases}$

Notice that $X^{m}$ has finite state space $\mathbb{S}_{m}\cup\\{\Delta\\}$.
Notice that $W$ is bounded since $V$ is, let $C=\sup_{x\in\mathbb{S}}W(x)$,
and extend $W$ to a function on $\mathbb{S}\cup\\{\Delta\\}$ by setting
$W(\Delta)=C$. Let $\mathcal{L}_{m}$ denote the generator of the process
$X^{m}$; we claim that $\mathcal{L}W(x)\leq\mathcal{L}_{m}W(x)$ whenever
$x\in\mathbb{S}_{m}$. Indeed, notice that

$\displaystyle\mathbb{E}_{x}[W(X_{t})]$
$\displaystyle=\sum_{y\in\mathbb{S}}W(y)\mathbb{P}_{x}(X_{t}=y)$
$\displaystyle=\sum_{y\in\mathbb{S}}W(y)\mathbb{P}_{x}(X_{t}=y,t<\tau_{m})+\sum_{y\in\mathbb{S}}W(y)\mathbb{P}_{x}(X_{t}=y,t\geq\tau_{m})$
$\displaystyle\leq\sum_{y\in\mathbb{S}}W(y)\mathbb{P}_{x}(X_{t}=y,t<\tau_{m})+\sum_{y\in\mathbb{S}}C\mathbb{P}_{x}(X_{t}=y,t\geq\tau_{m})$
$\displaystyle=\sum_{y\in\mathbb{S}_{m}}W(y)\mathbb{P}_{x}(X_{t}^{m}=y)+W(\Delta)\mathbb{P}_{x}(X_{t}^{m}=\Delta)$
$\displaystyle=\mathbb{E}_{x}[W(X^{m}_{t})],$

and hence

$\mathcal{L}W(x)=\lim_{t\searrow
0}\frac{\mathbb{E}_{x}[W(X_{t})]-W(x)}{t}\leq\lim_{t\searrow
0}\frac{\mathbb{E}_{x}[W(X^{m}_{t})]-W(x)}{t}=\mathcal{L}_{m}W(x),$

as claimed. Now for any $m$, applying Dynkin’s Formula to the chain $X^{m}$
with finite stopping time $\tau_{B}\wedge\tau_{m}\wedge m$ yields

$\mathbb{E}_{x_{0}}[W(X^{m}_{\tau_{B}\wedge\tau_{m}\wedge
m})]=W(x_{0})+\mathbb{E}_{x_{0}}\left[\int_{0}^{\tau_{B}\wedge\tau_{m}\wedge
m}\mathcal{L}_{m}W(X^{m}_{s})\mathrm{d}s\right].$

But for $s<\tau_{B}\wedge\tau_{m}$ we have $X^{m}_{s}=X_{s}\in
B^{c}\cap\mathbb{S}_{m}$ and hence

$\displaystyle\mathcal{L}_{m}W(X^{m}_{s})=\mathcal{L}_{m}W(X_{s})\geq\mathcal{L}W(X_{s})=\mathcal{L}V(X_{s})\geq
0.$

So the integrand in Dynkin’s Formula is non-negative, and

$\displaystyle W(x_{0})$
$\displaystyle\leq\mathbb{E}_{x_{0}}[W(X^{m}_{\tau_{B}\wedge\tau_{m}\wedge
m})]$
$\displaystyle=\mathbb{E}_{x_{0}}[W(X^{m}_{\tau_{B}})\mathbb{I}_{\tau_{B}<\tau_{m}\wedge
m}]+\mathbb{E}_{x_{0}}[W(X^{m}_{\tau_{m}\wedge
m})\mathbb{I}_{\tau_{B}\geq\tau_{m}\wedge m}]$
$\displaystyle\leq\mathbb{E}_{x_{0}}[W(X^{m}_{\tau_{B}})\mathbb{I}_{\tau_{B}<\tau_{m}\wedge
m}]+C\mathbb{P}_{x_{0}}(\tau_{B}\geq\tau_{m}\wedge m).$

Note that $X_{\tau_{B}}^{m}\in B$ on the event $\tau_{B}<\tau_{m}\wedge m$.
Hence $W(X_{\tau_{B}}^{m})\mathbb{I}_{\tau_{B}<\tau_{m}\wedge m}\leq 0$, and

$W(x_{0})\leq C\mathbb{P}_{x_{0}}(\tau_{B}\geq\tau_{m}\wedge m)$

Since $X$ is assumed to be non-explosive, $\tau_{m}\to\infty$ as $m\to\infty$,
so taking $m\to\infty$ above gives

$W(x_{0})\leq C\mathbb{P}_{x_{0}}(\tau_{B}=\infty).$

But $W(x_{0})$ is strictly positive and $0<W(x_{0})\leq C<\infty$, so
$\mathbb{P}_{x_{0}}(\tau_{B}=\infty)\neq 0$. That is,
$\mathbb{P}_{x_{0}}(\tau_{B}<\infty)<1$, as desired. ∎

######  Remark 6.3:

Note that the proof above gives us a lower bound for the probability that the
process never returns to the set $B$: