#  Stochastic Extinction
An Average Lyapunov Function Approach

Juraj Földes and Declan Stacy  Juraj Földes 111Juraj Földes was partly
supported under the grant NSF DMS-1816408
Department of Mathematics, University of Virginia
Kerchof Hall, Charlottesville, VA 22904-4137<EMAIL_ADDRESS>Declan
Stacy
Department of Mathematics, University of Virginia (Undergraduate student)
Kerchof Hall, Charlottesville, VA 22904-4137<EMAIL_ADDRESS>
###### Abstract.

We study the stability of $\mathcal{M}_{0}$, an invariant subset of a Markov
process $(X_{t})_{t\geq 0}$ on a metric space $\mathcal{M}$. By building the
theory of average Lyapunov functions, we formulate general criteria based on
the signs of Lyapunov exponents that guarantee extinction
($X_{t}\to\mathcal{M}_{0}$ as $t\to\infty$). Additionally, we provide
applications to a stochastic SIS epidemic model on a network with regime-
switching, a stochastic differential equation version of the Lorenz system, a
general class of discrete-time ecological models, and stochastic Kolmogorov
systems. In many examples we improve existing results by removing unnecessary
assumptions or providing sharper criteria for the extinction.

Keywords: Stability, general Markov processes, invariant subsets, extinction,
Lyapunov function, Lyapunov exponent, Martingale, average Lyapunov function,
invariant measures

2020 MSC: 37H20, 37H30, 60H10, 60J05, 60J25, 60J70 60F15, 92D25

##  1\. Introduction

The stability or instability of an invariant subset of a state space is a
central problem in the analysis of stochastic and deterministic dynamical
systems. Specifically, suppose a Markov process $(X_{t})_{t\geq 0}$ takes
values in a metric space $(\mathcal{M},d)$ and there is a closed subset
$\mathcal{M}_{0}\subset\mathcal{M}$, called the extinction set, which is
invariant: $X_{t}\in\mathcal{M}_{0}$ for all $t\geq 0$ if
$X_{0}\in\mathcal{M}_{0}$. We investigate criteria ensuring that
$\mathcal{M}_{0}$ is stable, meaning (roughly) that
$d(X_{0},\mathcal{M}_{0})\ll 1$ implies $d(X_{t},\mathcal{M}_{0})\to 0$ as
$t\to\infty$ (extinction). In addition, we assume that $X_{t}$ can only
approach $\mathcal{M}_{0}$ “asymptotically,” meaning $X_{t}$ cannot enter
$\mathcal{M}_{0}$ in finite time. For the theory of extinction in finite time
we refer the reader to [MV12] and for a detailed discussion on the finite
versus asymptotic extinction see [Ben23, Sch17]. The stability and closely
related questions, such as the rate of convergence of
$d(X_{t},\mathcal{M}_{0})\to 0$ and the dependence on certain parameters, have
been extensively investigated in many contexts.

In this manuscript we consider stochastic processes $(X_{t})_{t\geq 0}$, and
therefore the stability and the rate of convergence can in general depend on
the realization of the process. Also, we allow $(X_{t})_{t\geq 0}$ to have
jump discontinuities, and in particular our results also apply to discrete-
time Markov chains. Some processes which can be analyzed using our methods
include:

Ecological models, including stochastic replicator dynamics in evolutionary
game theory and more traditional predator-prey type systems (see [BS19a, HN18,
HNS22, Rud03, KS21, SBA11, Che94, CE89, CW81, Tur77, BHS08, HS19, HNC21,
BL16]), where points in $\mathcal{M}$ represent the population densities of
some interacting species and $\mathcal{M}_{0}$ is the subset where certain
species are extinct.

Epidemiological models (see [GGH+11, HDAD15, KABY18, ZAE24, LLC17, BO23,
SL24]), where points in $\mathcal{M}$ represent the prevalence of some disease
and $\mathcal{M}_{0}$ (usually a single point) is the subset where the disease
is eradicated.

Chemostat models (see [NNY20, GJH+20]), where points in $\mathcal{M}$
represent the concentration levels of bacteria and substrate and
$\mathcal{M}_{0}$ is the subset where the concentration of bacteria is $0$.

Piecewise Deterministic Markov Processes (PDMP) (see [BS19b, Str21, BH12]),
where $\mathcal{M}$ is the product of some phase space and a finite set of
“switches,” which can be thought of as different environments or regimes. For
a fixed regime, the process induces a deterministic flow on the phase space
with a fixed point $0$. Then $\mathcal{M}_{0}$ consists of points which have
$0$ in the phase space component.

Models for turbulence of fluid flow ([CZH21, BKP22a, BKP22b, FGHRW24]), where
$\mathcal{M}_{0}$ consists of non-turbulent states.

In this paper we introduce a general framework that provides stability
criteria, rates of convergence, and dependence on the parameters with wide
applicability, in particular, to the settings mentioned above. We leverage the
powerful average Lyapunov function technique, substantially extending the
ideas from [Hof81, Hut84, Sch00, GH03], where average Lyapunov functions were
used to analyze deterministic ecological models.

The present work is partly motivated by [Ben23], where the question of
instability of $\mathcal{M}_{0}$, or so called persistence, was investigated.
We remark that although our general setting and technical assumptions are
quite similar and our results are in some sense dual to those in [Ben23], the
methods involved in the proofs and applications of our results are quite
different, as discussed in detail below.

###  1.1. Average Lyapunov Function.

Lyapunov functions are a powerful tool for analyzing the stability of
dynamical systems. Lyapunov functions are functions $f$ which have certain
properties, such as $f\geq 0$ is nonnegative or $f(x)\to\infty$ as
$x\to\mathcal{M}_{0}$. Also, a bound on the evolution of $f(X_{t})$ is
assumed, for example $f(X_{t+1})-f(X_{t})\leq 0$, so that $f(X_{t})$ is
decreasing along positive integer time steps. For continuous-time processes,
it is more common to postulate bounds on $\mathcal{L}f$, where $\mathcal{L}f$
encodes the average rate of change of $f(X_{t})$. For example, if $X_{t}$
solves a deterministic differential equation $\frac{d}{dt}X_{t}=F(X_{t})$,
then $\mathcal{L}f=\nabla f\cdot F$ since $\frac{d}{dt}f(X_{t})=\nabla
f(X_{t})\cdot\frac{d}{dt}X_{t}=(\nabla f\cdot F)(X_{t})$. For random $X_{t}$,
$\mathcal{L}f$ is something like $\frac{d}{dt}\mathbb{E}[f(X_{t})]$. For
precise definitions and properties $\mathcal{L}f$ see the main body of the
manuscript below. After finding a Lyapunov function $f$ with suitable
properties and bounds on $\mathcal{L}f$, one can derive various conclusions
about the long-term behavior of $X_{t}$, such as convergence to some point or
staying inside some set forever. However, even for deterministic $X_{t}$,
Lyapunov functions are often extremely difficult to construct, usually because
$\mathcal{L}f$ needs to satisfy bounds on the entirety of $\mathcal{M}$ or
$\mathcal{M}\backslash\mathcal{M}_{0}$.

Unlike a traditional Lyapunov function, an average Lyapunov function $V$ does
not require bounds on $\mathcal{L}V$ everywhere. Instead, one considers the
average value of $\mathcal{L}V$ with respect to certain measures $\mu$,
meaning one only needs to estimate

$\mu\mathcal{L}V:=\int\mathcal{L}V(x)d\mu(x)\,,$

which we refer to as an (average) Lyapunov exponent. For example, in our paper
we consider $\mu\mathcal{L}V$ for all invariant measures $\mu$ on
$\mathcal{M}_{0}$. Due to their weaker assumptions, average Lyanpunov
functions are a lot easier to construct than traditional Lyapunov functions,
but using them to derive properties of $X_{t}$ is significantly harder.
Further complications arise in the stochastic setting because $\mathcal{L}V$
only gives information about the expectation of the derivative of $V(X_{t})$,
and so additional requirements on $V$ are needed to show that, with a high
probability, the expectation accurately captures the behavior of $X_{t}$. For
example, often sufficient additional information is an estimate on the growth
of $Var(V(X_{t}))$, quantified by $\Gamma V:=\mathcal{L}V^{2}-2V\mathcal{L}V$.
To summarize, traditional Lyapunov functions are difficult to construct and
are problem-specific, whereas average Lyapunov functions are often obvious to
construct and instead the challenge lies in computing the Lyapunov exponents
(which is problem-specific) and in using their properties to determine the
long term behavior of $X_{t}$, which should not be problem-specific. We remark
that both approaches are usually more involved in the stochastic setting.

Most of the previous attempts to use an average Lyapunov function $V$ to
analyze the long-term behavior of $X_{t}$ involve constructing a more
traditional Lyapunov function which is some sort of combination of $e^{cV}$
for some $c\in\mathbb{R}$ and an additional Lyapunov function $W$ which
prevents $X_{t}\to\infty$ in finite time. The combination of $e^{cV}$ and $W$
is delicate, and moreover it is quite nontrivial to deduce bounds on
$\mathcal{L}e^{cV}$ or $\mathbb{E}[e^{cV(X_{t+T})}-e^{cV(X_{t})}]$ for some
$T>0$ large enough assuming only some bounds on $\mu\mathcal{L}V$ for certain
measures $\mu$. As a consequence, these approaches are unsuitable for
generalization and require additional unnecessary assumptions.

There are two notable arguments we found in the literature that avoid this
sort of construction. The first one in [Kha12] only applies to linear
stochastic differential equations, and therefore it cannot be easily
generalized. However, its use of a change of coordinates is crucial, and we
often leverage this technique when applying our theory to examples. The second
is the proof by contradiction method presented in [Ben23] for solving the
problem of persistence (coexistence), which cannot be applied to our problem
of extinction. However, [Ben23] and the present work share the use of several
tools such as certain martingales and empirical occupation measures. Both of
these arguments are discussed in greater detail below in the introduction.

###  1.2. Our Method

In this paper we use an entirely new approach in order to prove our general
results, which, in particular, apply to all of the examples listed above, see
below for more details. Although the proof of our main result 3.2 is somewhat
involved, the result itself is a universal tool that can be easily applied. We
show that the long-term behavior of many processes is completely determined by
Lyapunov exponents depending only on the behavior on an arbitrarily small
neighborhood of the extinction set $\mathcal{M}_{0}$. In practice, this allows
one to linearize the system about $\mathcal{M}_{0}$ and study that instead of
the original system. Since our results apply to many different types of
processes, each of which has a different notion of linearization, we do not
phrase our results in terms of linearization, but we discuss this connection
to linearization later in the introduction.

We formulate sufficient conditions that guarantee $X_{t}\to\mathcal{M}_{0}$ as
$t\to\infty$ (extinction) in terms of an average Lyapunov function $V$ defined
on some subset of $\mathcal{M}\backslash\mathcal{M}_{0}$ and a corresponding
Lyapunov exponent $\alpha$ which depends only on the behavior of $X_{t}$ near
$\mathcal{M}_{0}$. The central idea of the proof is to construct a clever
sequence of stopping times $\tau_{n}$ so that the discrete-time process
$V(Y_{n})\coloneqq V(X_{\tau_{n}})$ behaves similarly to a Brownian motion
with drift (the mean and variance of $V(Y_{n})$ grow linearly in $n$). This
allows us to conclude that $X_{t}$ stays close to $\mathcal{M}_{0}$ for all
times (stability in probability), and consequently ergodic theory-type
arguments imply that $X_{t}$ must approach $\mathcal{M}_{0}$ as $t\to\infty$
(asymptotic stability). If $\mathcal{M}_{0}$ satisfies an accessibility
criterion, then a renewal-type argument yields that the extinction occurs with
probability $1$, regardless of the initial condition (global asymptotic
stability). Additionally, the speed of the convergence is given explicitly as
a function of the Lyapunov exponent, for example, if
$V(x)=-\log{d(x,\mathcal{M}_{0})}$, then
$d(X_{t}^{x},\mathcal{M}_{0})=O(e^{-\alpha t})$.

The main difficulties and novelties of the proof occur in the construction of
the stopping times $\tau_{n}$, which can be split into two phases. In the
first phase we run the process until $V(X_{t})$ or its approximation
$\int_{0}^{t}\mathcal{L}V(X_{s})ds$ increases enough, and otherwise we stop
the process before they decrease too much. Obtaining the appropriate bounds is
quite nontrivial because our average Lyapunov function $V$ is not defined on
$\mathcal{M}_{0}$, the set on which we have the most information about the
long-term behavior of $X_{t}$. However, we assume that $\mathcal{L}V$ extends
continuously to $\mathcal{M}_{0}$, and so for initial conditions
$X_{0}\in\mathcal{M}_{0}$ we can use ergodic arguments to control
$\int_{0}^{t}\mathcal{L}V(X_{s})ds$, a proxy for $V(X_{t})-V(X_{0})$. To
extend this control to initial conditions $X_{0}$ outside of $\mathcal{M}_{0}$
we require a strong continuous dependence of $X_{t}$ on $X_{0}$, namely
continuity of the law of $\\{X_{t}\\}_{t\geq 0}$ viewed as a random element of
the Skorokhod space $\mathbb{D}_{[0,\infty)}(\mathcal{M})$. It is well-known
that such property holds when $X_{t}$ is Feller, but, motivated by
applications, we need to extend it to “$C_{b}$-Feller” processes, which
requires new proofs and additional stability assumptions. In the second phase,
we let the process (quickly) return to a fixed compact set. The compactness is
essential since the success of the first phase relies on having uniform bounds
on $\mathbb{E}[V(X_{\tau_{1}})-V(X_{0})]$ and
$\mathbb{E}[(V(X_{\tau_{1}})-V(X_{0}))^{2}]$, but for each point in
$\mathcal{M}_{0}$ we can only obtain bounds which hold in a small
neighborhood. Additionally, we must use clever manipulations of certain
martingales to show that the second phase does not diminish the increase of
$V(X_{t})$ from the first phase.

###  1.3. A Brief Summary of Persistence/Coexistence Theory

Before further discussing the details of our manuscript, we summarize the
persistence result proved in [Ben23] in order to introduce important notation
and underlying ideas for our manuscript in a technically easier setting. Since
the assumptions of the persistence result are almost complementary to ours,
together [Ben23] and our paper completely characterize the long-term behavior
of many Markov Processes. However, we again stress that our paper solves a
different problem than [Ben23] does, and thus the methods and proofs are quite
different.

In some sense the opposite to extinction, $d(X_{t},\mathcal{M}_{0})\to 0$ as
$t\to\infty$, is persistence, which heuristically means that $X_{t}$ spends an
arbitrarily large fraction of time in compact subsets of
$\mathcal{M}\setminus\mathcal{M}_{0}$. To formalize this definition, we
introduce the empirical occupation measures on $\mathcal{M}$ as

(1.1) $\mu_{t}=\frac{1}{t}\int_{0}^{t}\delta_{X_{s}}ds,$

which are random measures tracking the time-averaged behavior of $X_{t}$. Then
under mild stability assumptions it is known (see 4.6 and 4.7) that
$(\mu_{t})_{t\geq 0}$ is tight and all limiting measures as $t\to\infty$ lie
in the set $P_{inv}(\mathcal{M})$ of invariant measures on $\mathcal{M}$,
which represent the possible stationary behaviors of $X_{t}$. In other words,
the invariant measures completely characterize the possible long-term time-
averaged behaviors of $X_{t}$. While extinction forces all limiting measures
of $(\mu_{t})_{t\geq 0}$ as $t\to\infty$ to be supported on $\mathcal{M}_{0}$,
persistence means that they are all supported on
$\mathcal{M}\setminus\mathcal{M}_{0}$. Thus, in the context of empirical
occupation measures, the notions of persistence and extinction are in some
sense dual.

The crucial assumption that guarantees persistence in [Ben23] is the existence
of an average Lyapunov function
$V:\mathcal{M}\setminus\mathcal{M}_{0}\to\mathbb{R}$ such that
$\mathcal{L}V:\mathcal{M}\setminus\mathcal{M}_{0}\to\mathbb{R}$ extends to a
continuous function $H$ on $\mathcal{M}$ and the “$H$-exponent” or “average
Lyapunov exponent”

(1.2) $\Lambda\coloneqq\sup_{\mu\in P_{inv}(\mathcal{M}_{0})}\mu
H\coloneqq\sup_{\mu\in P_{inv}(\mathcal{M}_{0})}\int H(x)d\mu(x)$

is strictly negative. In our paper we instead assume that a similar quantity

(1.3) $\alpha\coloneqq\inf_{\mu\in P_{inv}(\mathcal{M}_{0})}\mu H$

is strictly positive. Heuristically, $\Lambda<0$ (resp. $\alpha>0$) implies an
average decrease (resp. increase) of $t\mapsto V(X_{t})$ when $X_{t}$ is close
to $\mathcal{M}_{0}$. In the case of $\Lambda<0$, it is further assumed that
$V\geq 0$, and so we obtain that $X_{t}$ cannot spend much time close to
$\mathcal{M}_{0}$, while for $\alpha>0$ we further assume that $V(x)\to\infty$
as $x\to\mathcal{M}_{0}$, and so $X_{t}$ must approach $\mathcal{M}_{0}$.

To provide more justifications, we summarize the argument given in [Ben23] for
the easier case of $\Lambda<0$ (persistence). Along with the empirical
occupation measures (1.1), a key tool in the proof is

(1.4) $M^{V}_{t}\coloneqq V(X_{t})-V(X_{0})-\int_{0}^{t}H(X_{s})ds\,,$

a martingale which allows us to relate the behavior of $V(X_{t})$, which is
not even defined if $X_{0}\in\mathcal{M}_{0}$, to that of
$\int_{0}^{t}H(X_{s})ds$, which can be easily analyzed using ergodic theory.

Specifically, for any initial condition
$X_{0}\in\mathcal{M}\setminus\mathcal{M}_{0}$ and all large times $t$ it holds
that

(1.5)
$\frac{V(X_{t})}{t}\approx\frac{1}{t}\int_{0}^{t}H(X_{s})ds=\mu_{t}H\approx\int
H(x)d\mu(x)\,,$

where $\mu\in P_{inv}(\mathcal{M})$ is a limit point of $\mu_{t}$. Hence,
$V\geq 0$ implies $\int H(x)d\mu(x)\geq 0$. Suppose for contradiction that
$\mu(\mathcal{M}_{0})>0$. Since it is assumed that $\mathcal{M}_{0}$ and
$\mathcal{M}\setminus\mathcal{M}_{0}$ are both invariant sets, we can
decompose $\mu$ as

$\mu=\mu(\mathcal{M}_{0})\mu_{1}+(1-\mu(\mathcal{M}_{0}))\mu_{2}\,,$

where $\mu_{1}\in P_{inv}(\mathcal{M}_{0}),\mu_{2}\in
P_{inv}(\mathcal{M}\setminus\mathcal{M}_{0})$. If $\mu_{2}$ is ergodic and
$X_{0}$ is distributed as $\mu_{2}$, then

$\mu_{2}H=\lim_{t\to\infty}\mu_{t}H=\lim_{t\to\infty}\frac{V(X_{t})}{t}=0\,,$

where in the last equality we used Birkhoff’s ergodic theorem to conclude that
$X_{t}$ enters a compact subset of $\mathcal{M}\setminus\mathcal{M}_{0}$
infinitely often ($V$ is bounded on such a compact subset). Then by ergodic
decomposition $\mu_{2}H=0$ for any $\mu_{2}\in
P_{inv}(\mathcal{M}\setminus\mathcal{M}_{0})$. However, we assumed
$\mu_{1}H\leq\Lambda<0$, and consequently we obtain

$0\leq\mu
H=\mu(\mathcal{M}_{0})\mu_{1}H+(1-\mu(\mathcal{M}_{0}))\mu_{2}H\leq\mu(\mathcal{M}_{0})\Lambda<0\,,$

a contradiction.

Rigorously justifying (1.5) requires verifying two technical details. The
first is $\frac{V(X_{t})}{t}\approx\frac{1}{t}\int_{0}^{t}H(X_{s})ds$, which
is equivalent to the strong law of large numbers for (1.4):
$\lim_{t\to\infty}\frac{M^{V}_{t}}{t}=0$, which is assumed to hold in [Ben23].
The second is $\mu_{t}H\approx\int H(x)d\mu(x)$, which by the ergodic theorems
is valid for $\mu$-a.e. initial condition. However, the proof requires the
validity for any initial condition in $\mathcal{M}\setminus\mathcal{M}_{0}$,
which is deduced via the existence of a traditional Lyapunov function $W$ with
certain properties (3). In practice, $W$ can be almost always constructed. The
existence of $W$ also yields the correct stability criteria that ensure the
tightness and the invariance of limit points of the empirical occupation
measures, which are also key to the proof.

The assumptions (besides $\alpha>0$ versus $\Lambda<0$) in our work and in
[Ben23] are similar and let us highlight, mostly technical, differences. For
example, instead of assuming that martingales satisfy the strong law of large
numbers, in our manuscript we assume the stronger condition given in our 5,
which involves another traditional Lyapunov function $U$. Although 5 is
stronger, in all examples we were able to locate in the literature, the strong
law of large numbers is verified via 5, and so for practical purposes there is
no difference in the assumptions. In addition, for the special cases of pure
jump processes and SDEs (see 9.6 and 9.3), we show that $U$ can be frequently
constructed as a function of $W$, so 5 is automatically satisfied.
Additionally, 5 is satisfied if $\mathcal{M}$ is compact (in fact almost all
of the technical assumptions are trivial in this case, see 3.13).

Other differences in our assumptions and [Ben23] are intrinsic to the problem
of extinction. For extinction we need to assume that $V(X_{t})\to\infty$
implies that $d(X_{t},\mathcal{M}_{0})\to 0$ since by our techniques we cannot
directly estimate $d(X_{t},\mathcal{M}_{0})$, but we can obtain a lot of
information about $V(X_{t})$. On the other hand, compared to [Ben23] we do not
need to assume that $V\geq 0$ or that $V$ is defined on all of
$\mathcal{M}\setminus\mathcal{M}_{0}$ (we only require $V$ to be defined on an
open dense subset), which is essential for certain applications.

The similarities of our assumptions and the ones in [Ben23] have significant
practical consequences; for a large class of Markov processes, it suffices to
investigate the sign of the $H$-exponents $\Lambda,\alpha$ to determine
persistence or extinction. This is an improvement over the existing
literature, where two completely separate arguments were needed, one for
persistence and one for extinction. In the examples below we focus on
extinction, but with little additional extra work it is possible to formulate
assumptions guaranteeing persistence. Similarly, the examples in [Ben23] have
extinction analogues based on our theorems.

###  1.4. Our Results as Linearization

Before applying our techniques to complicated systems, we discuss a more
classical example (linear SDEs) which illustrates the connection between the
$H$-exponent $\alpha$ and the long-term behavior of $X_{t}$. We highlight
[Kha12]’s use of a change of coordinates, a tool which is also applicable to
more complicated examples considered in this manuscript. Finally, we remark
that in the context of many (nonlinear) SDEs with a fixed point $0$, our 3.1
yields that $0$ is stable for the nonlinear SDE if $0$ is stable for its
corresponding linearized system, and a similar principle also applies to SDEs
with more complicated extinction sets.

In [Kha12] the authors discuss the long term behavior of the linear stochastic
system

(1.6) $dx_{t}=Ax_{t}dt+\Sigma x_{t}dW_{t}\,,$

where $A,\Sigma\in\mathbb{R}^{m\times m}$ are fixed $m\times m$ matrices,
$x_{t}\in\mathcal{M}:=\mathbb{R}^{m}$, and $W_{t}$ is a Brownian motion. In
this case $\mathcal{M}_{0}\coloneqq\\{0\\}$ contains only the origin, and we
would like to find conditions that guarantee $x_{t}\to 0$ as $t\to\infty$ and
the rate of convergence.

Since the system is linear, we expect an exponential convergence (if it
happens). Thus we set the average Lyapunov function to be $V(x)=-\log|x|$,
where $|x|\coloneqq\Big{(}\sum_{i=1}^{m}x_{i}^{2}\Big{)}^{1/2}$ denotes the
Euclidean norm, defined for all $x\notin\mathcal{M}_{0}$. Since this function
only depends on $|x|$, it is natural to transform the problem to polar
coordinates $(v,r)\in S^{m-1}\times[0,\infty)\eqqcolon\mathcal{N}$, where
$S^{m-1}\coloneqq\\{v\in\mathbb{R}^{m}\mid\|v\|=1\\}$ is a sphere.

Using Itô formula one can easily show that $\mathcal{L}$ is a differential
operator and compute $\mathcal{L}V$ (see Section 9 below for details). Then
one notices that $\lim_{x\to 0}\mathcal{L}V(x)$ depends on the direction that
$x$ approaches $0$, that is, $\lim_{t\to 0}\mathcal{L}V(tv)$ with $v\in
S^{m-1}$ depends on $v$, and in particular $\mathcal{L}V$ cannot be
continuously extended to $\mathcal{M}$. However, in polar coordinates the
origin can be interpreted as $\mathcal{N}_{0}\coloneqq S^{m-1}\times\\{0\\}$,
that is, $\mathcal{M}_{0}$ becomes a sphere, and then $\mathcal{L}V$ indeed
extends to a continuous function on $\mathcal{N}$, as detailed in the argument
below. Such approach is common in our examples, where one establishes a
“boundary” of the possible trajectories of $X_{t}$ by expanding
$\mathcal{M}_{0}$ to account for all the directions along which $X_{t}$ could
approach $\mathcal{M}_{0}$. We formalize this change of variables in 3.9
below.

To provide more details, we set $x_{t}=r_{t}v_{t}$ and then Itô’s formula
yields that $(v_{t},r_{t})$ satisfies an SDE

$\displaystyle dv_{t}$ $\displaystyle=f(v_{t})dt+\sigma(v_{t})dW_{t}$
$\displaystyle dr_{t}$ $\displaystyle=r_{t}g(v_{t})dt+r_{t}\eta(v_{t})dW_{t}$

for continuous functions $f,\sigma:S^{m-1}\to\mathbb{R}^{m}$,
$g,\eta:S^{m-1}\to\mathbb{R}$, and $V(v,r)\coloneqq-\log{r}$ satisfies

$dV_{t}=\Big{[}-g(v_{t})+\frac{\eta(v_{t})^{2}}{2}\Big{]}dt-\eta(v_{t})dW_{t}\,.$

For $\mathcal{L}V(v,r)\coloneqq-g(v)+\frac{\eta(v)^{2}}{2}$ we have that

$M^{V}_{t}\coloneqq
V(v_{t},r_{t})-V(v_{0},r_{0})-\int_{0}^{t}\mathcal{L}V(v_{s},r_{s})ds=-\int_{0}^{t}\eta(v_{s})dW_{s}$

is a square-integrable martingale whose quadratic variation
$\int_{0}^{t}\eta^{2}(v_{s})ds$ is bounded by $Ct$ for some constant $C>0$.
Applying the strong law of large numbers for martingales we have that
$\lim_{t\to\infty}\frac{M_{t}^{V}}{t}=0$ a.s. Thus, if $P_{inv}(S^{m-1})$
denotes the set of invariant measures of $v_{t}$ on $S^{m-1}$ and

$\alpha\coloneqq\inf_{\mu\in P_{inv}(S^{m-1})}\int\mathcal{L}V(v,0)d\mu(v)$

is the Lyapunov exponent, then Birkhoff’s ergodic theorem implies that for
most initial conditions

(1.7) $\liminf_{t\to\infty}\frac{-\log r_{t}}{t}\geq\alpha\,.$

In particular, if $\alpha>0$ then $x_{t}\to 0$ exponentially fast with rate
$\alpha$. Of course, determining the positivity of $\alpha$ may be involved
and possible only in special cases. For example in the deterministic case of
$\Sigma=0$, we may explicitly solve (1.6) to see that $-\alpha$ is simply the
largest real part of eigenvalues of $A$.

Observe that this argument, in particular the conclusion (1.7), crucially
depends on the fact that both $\mathcal{L}V$ and the right hand side of
$dv_{t}$ are only functions of $v_{t}$ as opposed to $(v_{t},r_{t})$. This
does not necessarily hold for the more general system

(1.8) $dx_{t}=\hat{A}(x_{t})dt+\hat{\Sigma}(x_{t})dW_{t}\,,$

where $\hat{A},\hat{\Sigma}:\mathbb{R}^{m}\to\mathbb{R}^{m}$ are continuously
differentiable functions with $\hat{A}(0)=\hat{\Sigma}(0)=0$. A natural
question is whether (1.8) has the same long term behavior as the “linearized”
system (1.6) with $A=\nabla\hat{A}(0)$ and $\Sigma=\nabla\hat{\Sigma}(0)$.
After the polar change of coordinates as above, we obtain that (1.6) and (1.8)
have the same behavior when the initial distribution is supported on the set
$\\{r=0\\}$, and also that $\mathcal{L}V(v,0)$ and $\alpha$ are the same for
both systems. Since in this paper we show (see, for example, 3.1) that the
stability of $\mathcal{M}_{0}$ is determined by $\alpha$ (determined by
behavior on $\mathcal{M}_{0}$ only), we conclude that if $0$ is attractive for
the linearized system (1.6), meaning $\alpha>0$, then the nonlinear system
(1.8) is stable as well (compare to [Kha12, Theorem 7.1]). Additionally, our
3.11 immediately implies that $\alpha$ depends continuously on $A,\Sigma$, and
thus the stability is preserved if the coefficients of the equation are
perturbed slightly.

The reasoning presented above is a general principle that extends beyond the
case where $\mathcal{M}_{0}$ is a singleton. Although it is not formally
stated in our manuscript, our main results imply that to determine whether
$\mathcal{M}_{0}$ is stable (meaning extinction occurs), it suffices to
linearize the system about $\mathcal{M}_{0}$ and determine whether
$\mathcal{M}_{0}$ (or a suitable “blown up” version $\mathcal{N}_{0}$ after a
change of coordinates) is stable for the linearized system. Indeed, we show
that the stability of $\mathcal{N}_{0}$ for the nonlinear and linearized
systems occurs when the corresponding $H$-exponent $\alpha$ is positive, and
since $\alpha$ only depends on the behavior of $X_{t}$ near the extinction
set, both values of $\alpha$ are exactly the same for the linearized and
nonlinear systems. This principle of linearization is one of the main
advantages of our technique; we only need to analyze $X_{t}$ and
$\mathcal{L}V$ near $\mathcal{M}_{0}$, as opposed to on the entirety of
$\mathcal{M}$, greatly simplifying our analysis of the important examples
discussed in further detail below.

###  1.5. Discussion of our Examples

Since we consider general Markov processes including jump processes and SDEs
with Markovian switching, our results have a wide range of applications to
both discrete-time and continuous-time Markov chains. Due to the multitude of
applications and also because our focus is on the development of universal
tools, we decided to present representative examples that motivated the
development of our general theory. Notably, we omit a discussion of chemostat
models, but we remark that our results can be used to simplify the proof of
the general result [NNY20, Theorem 2.2] in the case of $\lambda<0$.

####  1.5.1. SIS Model

The first example is the SIS (susceptible-infected-susceptible) epidemic model
on a network with Markovian switching analyzed in [BO23]. Each node $i$ of the
network can be infected at time $t$ with probability $x_{i}(t)$ based on the
interactions given by the network. The network, the rate of transmission, and
the rate of recovery are changing in time according to a Markov process on a
finite state space, where each state represents a different environment or
regime. In [BO23], conditions ensuring that $x(t)\to 0$ exponentially fast are
established by examining the Lyapunov function
$V(x)=-\frac{1}{2}\log{\|x\|^{2}}$. Specifically, using their assumptions and
Itô’s formula [BO23] proves that $\mathcal{L}V$ is positive and then the rate
of change of the quadratic variation of the martingale

(1.9) $V(x(t))-V(x(0))-\int_{0}^{t}\mathcal{L}V(x(s))ds$

is bounded and thus (1.9) satisfies the strong law of large numbers. Unlike
our approach, the authors of [BO23] needed to bound $\mathcal{L}V$ for all
possible inputs $x\neq 0$, not just for $x$ approaching $0$. Consequently, the
assumptions posited in [BO23] to guarantee extinction include an extra term
(which they denote by $K$) that is related to the strength of the noise on the
entire state space.

The theory developed in the present paper uses simpler analysis, only
requiring us to compute $\mathcal{L}V$ as $x$ approaches $0$. This allows us
to remove the unnecessary $K$ term in our improved conditions that guarantee
extinction. Using the corresponding persistence theory [Ben23], one can show
that our extinction conditions are optimal if the network topology remains
constant and only the rates of transmission and recovery are switching.

####  1.5.2. Lorenz System

In the second example, we consider the Lorenz system, which is a well-studied
simplified (deterministic) model of fluid dynamics and a prototypical model
for chaos in three variables $(X,Y,Z)\in\mathbb{R}^{3}$. The stochastic Lorenz
system with additive white noise of strength $\hat{\alpha}$ in the $Z$
component was analyzed in [CZH21]. Here we provide a simplified analysis of
the stochastic Lorenz system, focusing on the range of parameters for which
the $Z$-axis is a global attractor for the deterministic system. In this case,
it is shown in [CZH21] that for small $\hat{\alpha}$, the solutions converge
to the $Z$-axis almost surely, while for large $\hat{\alpha}$, persistence
occurs.

The proof in [CZH21] is divided into multiple steps which connect the
persistence or extinction to the sign of a Lyapunov-type exponent
$\lambda_{\hat{\alpha}}$ ([CZH21, Theorem 4.1]), and then the sign of
$\lambda_{\hat{\alpha}}$ is analyzed ([CZH21, Theorem 5.2]). To connect the
persistence or extinction to the sign of $\lambda_{\hat{\alpha}}$, first the
known Lyapunov function $V_{1}$ for the deterministic system is used to
control the process at spatial infinity. Next, after a cylindrical-like change
of coordinates from $(X,Y,Z)$ to $(r,\theta,z)$, where $r=-\infty$ (a
cylinder) corresponds to $X=Y=0$ (the $Z$-axis), the authors use the average
Lyapunov function $V(r,\theta,z)=-r$ to construct a more traditional Lyapunov
function $V_{0}:=e^{-\kappa r}(1-\kappa g_{\hat{\alpha}}+V_{2})$, where
$V_{2}$ is similar to $V_{1}$, the sign of $\kappa$ is the same as the sign of
$\lambda_{\hat{\alpha}}$, and $g_{\hat{\alpha}}$ (not given explicitly) is the
solution to a PDE related to the linearization of the system about the
$Z$-axis. Then they show that $V:=V_{0}+V_{1}$ satisfies $\mathcal{L}V\leq
K-cV$ for some constants $K,c>0$ and $V$ blows-up at infinity. If
$\lambda_{\hat{\alpha}}>0$, $V$ also blows up near the $Z$-axis, which
provides persistence. If $\lambda_{\hat{\alpha}}<0$, then $V\approx 0$ near
the $Z$-axis and satisfies $\mathcal{L}V\lesssim-\Gamma{V}$ in a small
neighborhood around the $Z$-axis, where $\Gamma{V}$ is the average rate of
change of the quadratic variation. By a renewal-type argument and a comparison
to Brownian motion with drift, the authors obtain extinction, that is,
convergence to the $Z$-axis.

On the other hand, our theory immediately implies [CZH21, Theorem 4.1] using
only the standard Lyapunov function $V_{1}$ and the average Lyapunov function
$V(r,\theta,z)=-r$, bypassing all of the difficult steps outlined in the
previous paragraph. We are also able to greatly simplify the arguments made in
[CZH21, Section 5], which analyses the behavior of the Lyapunov exponent
$\lambda_{\hat{\alpha}}$ for small and large $\hat{\alpha}$. Specifically,
3.11 below implies that if we perturb the coefficients of an SDE, then the
average Lyapunov exponent $\alpha$ from (1.3) does not change a lot (recall
that $P_{inv}(\mathcal{M}_{0})$ is the set of invariant measures supported on
$\mathcal{M}_{0}$). If $\hat{\alpha}=0$, then $P_{inv}(\mathcal{M}_{0})$ may
have multiple ergodic measures $\mu_{0}$ which are each supported on a point
or a circle, but they can all be easily characterized and thus $\alpha$ can be
computed and shown to be positive. However, when $\hat{\alpha}>0$ then
$P_{inv}(\mathcal{M}_{0})$ has exactly one measure $\mu_{\hat{\alpha}}$, which
is supported on a cylinder, and both $\mu_{\hat{\alpha}}$ and
$\mu_{\hat{\alpha}}H=-\lambda_{\hat{\alpha}}$ are difficult to analyze or
estimate. Yet, the continuity from 3.11 implies that $\mu_{\hat{\alpha}}H>0$
for small enough $\hat{\alpha}$, and thus extinction occurs. Since our paper
is focused on extinction rather than persistence, we do not discuss the other
half of [CZH21, Theorem 5.2] (large $\hat{\alpha}$ implies persistence), but
it would be interesting to see if that analysis can be simplified as well.

####  1.5.3. Ecological Models

Historically, the average Lyapunov function technique was primarily developed
for deterministic ecological models by mathematical biologists and ecologists
[Hof81, Hut84, Sch00, GH03] who recognized the importance of the so-called
“invasion” rates $r_{i}(\mu)$ associated to each species $i$ and invariant
measure $\mu$. Intuitively, $r_{i}(\mu)$ measures how quickly a small
population of the $i$ species will grow (invade the environment) when the
populations of all the species are distributed according to $\mu$.

Conditions ensuring whether or not a collection of species can coexist in
terms of $(r_{i}(\mu))_{i,\mu}$ were known mainly for special cases until the
recent works of [HN18] (for the case of SDEs), [BS19a] (for discrete time
systems with a globally attractive compact set), [HNC21] (for general discrete
time systems and also SDEs), and [NNY21a, NNY21b] (for stochastic equations
with delay). Since the invasion rates are essentially average Lyapunov
exponents, average Lyapunov functions are fundamental to their proofs.

However, instead of working with the average Lyapunov function $V$, the
arguments in cited papers typically use tools like the log-laplace transform
to instead analyze a more traditional Lyapunov function $\hat{V}$ which is
constructed using $V$. The analysis of $\hat{V}$ is quite involved since it
came from an average Lyapunov function and thus a sufficient time needs to
pass before $\hat{V}$ starts behaving like a traditional Lyapunov function. In
other words, instead of investigating $\mathcal{L}\hat{V}$, one has to analyze
$\mathbb{E}[\hat{V}(X(t+T))-\hat{V}(X(t))]$ for all $T$ in some interval,
which is typically harder and often involves rather strong conditions. The
theory developed in this manuscript allows us to easily recover many of these
results while also weakening their assumptions.

For example, under our weaker assumptions we show that [HNC21, Theorem 2.4] is
a consequence of our 3.5 and [HNC21, Theorem 2.1] follows from the
corresponding theory developed in [Ben23]. We are also able to recover [HNC21,
Theorem 2.5]. Since the continuous- and discrete-time ecological models share
the same main ideas, we decided to provide details only for the latter one.
This is partly because an application to SDEs is already illustrated via the
noisy Lorenz system above, and partly because the calculations involved in the
analysis of the discrete-time models cannot be done explicitly and thus
require more work. In Section 10.4 we formulate the proper (weaker)
assumptions so that the continuous-time analogues of the results mentioned
above (studied in [HN18]) hold, and briefly discuss how they can be proven
using our techniques.

We emphasize that while almost all of the applications discussed above are
direct consequences of the theory developed in this manuscript and require
very little additional work to analyze, the proof of our improved version of
[HNC21, Theorem 2.5] is quite nontrivial and illustrates the power of using
our robustness result 3.11 in conjunction with the change of variables
technique from 3.9. We construct an average Lyapunov function $V$, but
$\mathcal{L}V$ does not extend continuously to $\mathcal{M}$ without some
change of variables, the choice of which is not obvious. Since
$\mathcal{M}_{0}$ has a “corner”, we consider changes of variables $\pi_{p}$
for $1>p>0$ which approximate $\mathcal{M}_{0}$ by a smooth surface which
resembles the unit ball in $L^{p}$. Each $\pi_{p}$ maps a simplex-like set
$\mathcal{N}_{0}$ to $\mathcal{M}_{0}$ differently, and thus each $p$ induces
a different Markov process $Y^{p}$ (which can be thought of as
$\pi_{p}^{-1}(X)$) on $\mathcal{N}_{0}$, a different set of invariant measures
$P_{inv}^{p}(\mathcal{N}_{0})$ for $Y^{p}$, and also a different $H_{p}$,
which is defined as the continuous extension of $\mathcal{L}V\circ\pi_{p}$ to
$\mathcal{N}_{0}$. Each of these objects is quite hard to analyze for any
individual $p>0$. However, we show that, as $p\to 0$, $H_{p}$ converges to a
(simpler) function $H_{0}$ and also the dynamics governing $Y^{p}$ converge in
some sense to those of a simple Markov process $Y^{0}$, making $\inf_{\mu\in
P_{inv}^{0}(\mathcal{N}_{0})}\mu H_{0}>0$ easy to calculate. Using the
continuity result 3.11, we conclude that if $p$ is small enough, then
$\inf_{\mu\in P_{inv}^{p}(\mathcal{N}_{0})}\mu H_{p}>0$, so that $\pi_{p}$
gives a change of variables for which 3.9 can be applied to, allowing us to
conclude our improved version of [HNC21, Theorem 2.5].

###  1.6. Organization of Paper

In Section 2 we introduce our notation and key assumptions. In Section 3 we
give precise formulations of our main results and techniques. Section 4 is
devoted to proving fundamental estimates on important martingales related to
our Lyapunov functions, some ergodic-type facts about empirical occupation
measures, and a result concerning the long-term behavior of a class of
discrete-time semimartingales. These results are the backbone of the proof of
the crucial 3.2. In Section 5 we generalize some results known for
$C_{0}$-Feller processes to the more general class of “$C_{b}$-Feller”
processes. Here we also discuss the Skorokhod topology and the continuity of
relevant functions on these Skorokhod spaces. The main theoretical novelties
occur in Section 6, where we prove 3.2. Another main 3.5 is proved in Section
7. In Section 8 we include technical results relating the extension of the
generator $\mathcal{L}$ from the domains
$\mathcal{D}(\mathcal{M}),\mathcal{D}_{2}(\mathcal{M})$ (2.7, 8.6) to the
extended domains
$\mathcal{D}^{ext}_{+}(\mathcal{M}),\mathcal{D}^{ext}_{2}(\mathcal{M})$ (2.9,
2.10). In Section 9 we provide sufficient conditions that verify the bulk of
our technical assumptions in special cases: if the Markov process is a
switching diffusion or an SDE driven by Brownian motion. Also, we show how our
theory can be applied to discrete-time Markov chains. Finally, the
applications of our theory to the SIS model, Lorenz system, and ecological
models are given in Section 10.

For the reader primarily interested in applications, we recommend first
reading Section 2 and Section 3 (skipping the proofs), and then for each
example in Section 10 first reading the corresponding section of Section 9
(skipping the proofs). For example, one should be familiar with the statements
in Section 9.3 before reading Section 10.3.

##  2\. Notation and Assumptions

Let $(\mathcal{M},d)$ be a locally compact Polish (complete and separable)
metric space, $\mathcal{M}_{0}\subset\mathcal{M}$ be a closed set, and
$\mathcal{M}_{+}\subset\mathcal{M}_{0}^{c}$ be open and dense in $\mathcal{M}$
(where $\mathcal{M}_{0}^{c}$ denotes the complement of $\mathcal{M}_{0}$). The
set $\mathcal{M}_{0}$ can be viewed as an “extinction” set and
$\mathcal{M}_{+}$ as the set of possible initial conditions. Without loss of
generality we assume $d\leq 1$, otherwise we replace $d$ by $\min(d,1)$. We
endow $\mathcal{M}$ with the Borel $\sigma$\- algebra. For each
$x\in\mathcal{M}$, let $X_{t}^{x}$ be a homogeneous Markov process defined on
$\mathcal{M}$ which has cadlag sample paths. By this we mean that there is a
filtered probability space $(\Omega,\mathcal{F},\\{\mathcal{F}_{t}\\}_{t\geq
0},\mathbb{P})$ (which we assume is complete and right-continuous) and a
family of $\mathcal{M}-$valued random variables
$\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq 0}$ such that:

  * •

$X_{0}^{x}=x$ and $t\mapsto X_{t}^{x}$ is cadlag (right continuous with left
limits) a.s.

  * •

$X^{x}_{\cdot}$ is adapted to $\\{\mathcal{F}_{t}\\}_{t\geq 0}$, meaning
$X_{t}^{x}$ is $\mathcal{F}_{t}$ measurable for each $t\geq 0$.

  * •

For all bounded measurable functions $f:\mathcal{M}\to\mathbb{R}$, the map

$[0,\infty)\times\mathcal{M}\ni(t,x)\mapsto\mathcal{P}_{t}f(x)\coloneqq\mathbb{E}[f(X_{t}^{x})]$

is measurable and for any $s,t\geq 0$ we assume (homogeneity) that

$\mathcal{P}_{s}f(X_{t}^{x})=\mathbb{E}[f(X_{t+s}^{x})|\mathcal{F}_{t}]\,.$

It is standard to prove that $(\mathcal{P}_{t})_{t\geq 0}$ defines a
semigroup: for any $s,t\geq 0$ it holds that
$\mathcal{P}_{s+t}f=\mathcal{P}_{s}\mathcal{P}_{t}f$. Also, the definition of
$\mathcal{P}_{t}f$ makes sense as long as $f:\mathcal{M}\to\mathbb{R}$ is
measurable and bounded from below (or above), possibly taking on the value
$\infty$ (or $-\infty$).

Our setup is summarized in the following definition.

######  Definition 2.1.

For a Polish space $\mathcal{M}$, $\mathcal{M}_{0}\subset\mathcal{M}$ closed,
$\mathcal{M}_{+}\subset\mathcal{M}_{0}^{c}$ open and dense, and
$\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq 0}$ a Markov process on $\mathcal{M}$
with cadlag sample paths as defined above, we call
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$ a Markov quadruple.

For the rest of the section, fix a Markov quadruple
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$.

###  2.1. Invariant Sets and the Feller Property

Next we list two basic assumptions on our process $X_{t}^{x}$ and the sets
$\mathcal{M}_{0},\mathcal{M}_{+}$. First, we suppose that if an initial
condition is in $\mathcal{M}_{0}$ (respectively $\mathcal{M}_{+}$) then
$X_{t}^{x}$ belongs to $\mathcal{M}_{0}$ (respectively $\mathcal{M}_{+}$) for
all $t\geq 0$. Second, we assume a standard continuity on the law of
$X_{t}^{x}$ as a function of the initial condition $x$ and time $t$. Such a
condition is commonly known as the “Feller” property.

######  Definition 2.2.

A measurable set $A\subset\mathcal{M}$ is invariant if $x\in A$ implies that
almost surely $X_{t}^{x}\in A$ for all $t\geq 0$.

######  Remark 2.3.

Some authors use the expression “forward invariant set” instead of our
invariant set, since we do not require that $X_{t}^{x}\in A$ for some $t>0$
implies $x\in A$. However, there should not be any confusion, since we do not
use backward evolution in the present manuscript.

######  Assumption 1.

$\mathcal{M}_{0}$ and $\mathcal{M}_{+}$ are invariant.

Let $C_{b}(\mathcal{M})$ denote the space of bounded continuous functions on
$\mathcal{M}$ endowed with the supremum norm
$\|f\|=\sup_{x\in\mathcal{M}}|f(x)|$. The following continuity assumption is
vital for our analysis:

######  Assumption 2.

Assume that $X_{t}^{x}$ is Feller in the sense that for any $f\in
C_{b}(\mathcal{M})$ and $t\geq 0$, $\mathcal{P}_{t}f\in C_{b}(\mathcal{M})$,
and also $\mathcal{P}_{t}f\to f$ pointwise as $t\downarrow 0$.

######  Remark 2.4.

Note that we use “$C_{b}$” Feller continuity as opposed to the usual “$C_{0}$”
Feller continuity (just replace $C_{b}$ by $C_{0}$ in 2, where $C_{0}$ is the
space of functions vanishing at infinity) because in many examples
(particularly ecological ones) the semigroup is not $C_{0}$ Feller. See
[Ben23, Remark 1 and Example 1] for more details. We also note that the
requirement of right-continuity in time ($\mathcal{P}_{t}f\to f$ pointwise as
$t\downarrow 0$) is automatically satisfied since our process is assumed to
have cadlag paths.

######  Remark 2.5.

Under an additional stability assumption which we define below in 3, our
definition of Feller is equivalent to the seemingly stronger requirements of
[Ben23, Hypothesis 2]. This is a consequence of the quasi-left continuity
showed in 8.2 and (a slight modification of the proof of) 5.1, which is
similar to the proof of strong continuity for $C_{0}$ Feller semigroups.

######  Definition 2.6.

If
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$ is a Markov quadruple such that 1 and 2 are satisfied, then
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$ is called a Feller quadruple.

###  2.2. Two Important Martingales

In this section we introduce important definitions that are used in Section
2.3, as well as two martingales that are central to our analysis.

A central object in the study of Feller processes is the generator
$\mathcal{L}$ of the Markov semigroup $\mathcal{P}_{t}$ defined on its domain
$\mathcal{D}(\mathcal{M})\subset C_{b}(\mathcal{M})$ as follows:

######  Definition 2.7.

Suppose that $\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq 0}$ is a Feller process
(see 2) with the Markov semigroup $\mathcal{P}_{s}$ (see • ‣ Section 2). Then
the domain $\mathcal{D}(\mathcal{M})\subset C_{b}(\mathcal{M})$ of the
generator $\mathcal{L}$ is the set of all $f\in C_{b}(\mathcal{M})$ such that:

  1. (i)

for all $x\in\mathcal{M}$, $\lim_{s\downarrow
0}\frac{\mathcal{P}_{s}f(x)-f(x)}{s}$ exists and we set it equal to
$\mathcal{L}f(x)$.

  2. (ii)

$\mathcal{L}f\in C_{b}(\mathcal{M})$.

  3. (iii)

$\sup_{s>0}\Big{\|}\frac{\mathcal{P}_{s}f-f}{s}\Big{\|}<\infty$.

Heuristically, since $\mathcal{L}f(x)$ is the average rate of change of
$\mathbb{E}[f(X_{t}^{x})]$ at $t=0$, then

(2.1) $M_{t}^{f}(x)\coloneqq
f(X_{t}^{x})-f(x)-\int_{0}^{t}\mathcal{L}f(X_{s}^{x})ds$

should be a martingale, and this is proven in 8.3 below for
$f\in\mathcal{D}(\mathcal{M})$.

The variance of (2.1) is related to the Carre du Champ operator $\Gamma$
defined on $\mathcal{D}_{2}(\mathcal{M})=\\{f\in\mathcal{D}(\mathcal{M})\mid
f^{2}\in\mathcal{D}(\mathcal{M})\\}$ as $\Gamma
f=\mathcal{L}f^{2}-2f\mathcal{L}f$. It is a folklore result that $\Gamma$
gives the (predictable) quadratic variation of $M_{t}^{f}(x)$ as $\langle
M^{f}(x)\rangle_{t}=\int_{0}^{t}\Gamma f(X_{s}^{x})ds$, or equivalently that

(2.2) $(M^{f}_{t}(x))^{2}-\int_{0}^{t}\Gamma f(X_{s}^{x})ds$

is a martingale. We make this precise in 8.4 and 8.8 below.

######  Remark 2.8.

Informally, in the context of SDEs, $\mathcal{L}f$ (resp. $\Gamma f$) is the
coefficient of the “$dt$” part of “$df$” (resp. “$(df)^{2}$”). Then the
formula for $\Gamma$ is given by Itô’s formula: $df^{2}=2fdf+(df)^{2}$ so that
$(df)^{2}=df^{2}-2fdf$.

We wish to apply the operators $\mathcal{L}$ and $\Gamma$ to Lyapunov
functions introduced below that are unbounded and possibly defined only on a
subset $A$ of $\mathcal{M}$. This motivates the definitions of
$\mathcal{D}^{ext}_{+}(A)$ and $\mathcal{D}^{ext}_{2}(A)$ below, which are
general classes of functions for which (2.1) and possibly (2.2) are (local)
martingales for suitable functions $\mathcal{L}f,\Gamma f$. In Section 8 and
Section 9 we show that $\mathcal{D}^{ext}_{+}(A)$ and
$\mathcal{D}^{ext}_{2}(A)$ are in some sense the closures of
$\\{f\in\mathcal{D}(\mathcal{M})\mid f\geq 0\\}$ and
$\mathcal{D}_{2}(\mathcal{M})$, respectively, and we give easily verifiable
general conditions for ensuring that functions are in
$\mathcal{D}^{ext}_{+}(A)$ or $\mathcal{D}^{ext}_{2}(A)$.

######  Definition 2.9.

Let $A\subset\mathcal{M}$ be an open invariant set (see 2.2). We let
$\mathcal{D}^{ext}_{+}(A)$ be the set of all continuous $f:A\to[0,\infty)$
such that there exists a continuous function $\mathcal{L}f:A\to\mathbb{R}$
such that $M^{f}_{\cdot}(x)$ in (2.1) is a cadlag local martingale for all
$x\in A$.

######  Definition 2.10.

Let $A\subset\mathcal{M}$ be an open invariant set (see 2.2). We let
$\mathcal{D}^{ext}_{2}(A)$ be set of all continuous $f:A\to\mathbb{R}$ such
that there exist continuous functions $\mathcal{L}f:A\to\mathbb{R}$ and
$\Gamma f:A\to[0,\infty)$ such that $M^{f}_{\cdot}(x)$ in (2.1) is a cadlag
square integrable martingale and the stochastic process in (2.2) is a
martingale for all $x\in A$.

Next, we formulate a sufficient condition which ensures that (2.1) satisfies
the strong law for martingales:

######  Definition 2.11.

Let $A\subset\mathcal{M}$ be an open invariant set and
$f\in\mathcal{D}^{ext}_{2}(A)$. Then we say $f$ has linearly bounded quadratic
variation if for all $x\in A$ it holds that

$\sup_{t\geq 1}\frac{1}{t}\int_{0}^{t}\mathcal{P}_{s}\Gamma f(x)ds\leq C(x)$

for some continuous function $C:\overline{A}\to\mathbb{R}$, where
$\overline{A}$ denotes the closure of $A$.

In the following lemma we formulate a law of large numbers for the martingales
$(M_{t}^{f}(x))_{t\geq 0}$.

######  Lemma 2.12.

Assume that $A\subset\mathcal{M}$ is an open invariant set and
$f\in\mathcal{D}^{ext}_{2}(A)$ has linearly bounded quadratic variation (see
2.10, 2.11). Then for all $x\in A$,
$\lim_{t\to\infty}\frac{M_{t}^{f}(x)}{t}=0$ a.s. (recall that $M_{t}^{f}(x)$
was defined in (2.1)).

###### Proof.

The proof closely follows the argument in [Ben23, page 76] and we repeat it
here for completeness.

Since $f$ has linearly bounded quadratic variation, we have
$\mathbb{E}[(M_{t}^{f}(x))^{2}]\leq tC(x)$ for $t\geq 1$. For any integer $n$
and any $\epsilon>0$, Doob’s inequality for right continuous martingales
implies that

$\displaystyle\mathbb{P}\Big{(}\sup_{2^{n}\leq t\leq
2^{n+1}}\frac{|M_{t}^{f}(x)|}{t}\geq\epsilon\Big{)}$
$\displaystyle\leq\mathbb{P}\Big{(}\sup_{t\leq 2^{n+1}}|M_{t}^{f}(x)|\geq
2^{n}\epsilon\Big{)}\leq\frac{1}{2^{2n}\epsilon^{2}}2^{n+1}C(x)$
$\displaystyle=\frac{2C(x)}{2^{n}\epsilon^{2}}$

and the assertion follows from Borel-Cantelli lemma. ∎

###  2.3. Lyapunov Functions

In this section we provide the assumptions necessary for our main results. In
3 we suppose that there is a Lyapunov function which is large near spatial
infinity and we use it to prove tightness of appropriate measures and to
estimate return times to compact sets. 4 is crucial as it gives the existence
of an “average Lyapunov function” that forces $X_{t}^{x}$ to (on average) move
towards $\mathcal{M}_{0}$ if $X_{t}^{x}$ is close to $\mathcal{M}_{0}$. 5
guarantees that the variances of the mentioned Lyapunov functions evaluated at
$X_{t}^{x}$ have a controlled growth in time.

######  Definition 2.13.

For $f:\mathcal{M}\to[0,\infty)$, we call $f$ proper if $f$ is continuous and
has compact sublevel sets, that is for each $m\in[0,\infty)$ the set $\\{f\leq
m\\}\coloneqq\\{x:f(x)\leq m\\}$ is compact.

######  Assumption 3.

There are proper maps $W,W^{\prime}:\mathcal{M}\to[1,\infty)$ and a constant
$K>0$ such that:

  1. (i)

$W\in\mathcal{D}^{ext}_{2}(\mathcal{M})$ (see 2.10).

  2. (ii)

$\mathcal{L}W\leq K-W^{\prime}$.

######  Remark 2.14.

Since $\mathcal{M}$ is a locally compact Polish space, if
$A\subset\mathcal{M}$ is open, then there are compact sets
$K_{n}\subset\mathcal{M}$ such that $K_{n}\subset K_{n+1}^{\circ}$ (where
$K^{\circ}$ denotes the interior of $K$) and $\cup_{n}K_{n}=A$. When
$A=\mathcal{M}$, the fact that $W$ is proper means that we may take
$K_{n}=\\{W\leq n\\}$ (the sublevel sets).

######  Definition 2.15.

Let $P(\mathcal{M})$ denote the set of all Borel probability measures on
$\mathcal{M}$. For $\mu\in P(\mathcal{M})$, $S\subset\mathcal{M}$ measurable
and $f:S\to\mathbb{R}$ measurable, we use the shorthand

$\mu f:=\int_{S}fd\mu\,.$

We endow $P(\mathcal{M})$ with topology of weak convergence, that is,
$\mu_{n}\to\mu$ if for all $f\in C_{b}(\mathcal{M})$, $\mu_{n}f\to\mu f$.

######  Definition 2.16.

For $F\subset\mathcal{M}$ a closed invariant set (see 2.2), let
$P_{inv}(F)\subset P(\mathcal{M})$ denote the set of invariant measures on
$F$, that is $\mu\in P_{inv}(F)$ if $\mu(F)=1$ and $\mu\mathcal{P}_{t}f=\mu f$
for all $f\in C_{b}(\mathcal{M})$ and $t\geq 0$.

######  Remark 2.17.

By the Tietze extension theorem, $\mu\in P_{inv}(F)$ is equivalent to $\mu\in
P(F)$ and $\mu\mathcal{P}_{t}f=\mu f$ for all $f\in C_{b}(F)$ and $t\geq 0$.
Thus, if we view $\\{X_{t}^{x}\\}_{x\in F,t\geq 0}$ as a Feller Markov process
on the Polish space $F$ there is no ambiguity in writing $P_{inv}(F)$.

######  Definition 2.18.

We say a function $f:A\to\mathbb{R}$ (where $A\subset\mathcal{M}$) vanishes
over a function $g:\mathcal{M}\to(0,\infty)$ if there is a sequence of compact
sets $(V_{n})_{n\geq 1}$ such that $\cup_{n}V_{n}=\mathcal{M}$ and

$\lim_{n\to\infty}\sup_{x\in A\setminus V_{n}}\frac{|f(x)|}{g(x)}\leq 0\,.$

Note that we have $\leq 0$ instead of $=0$ only because of the convention
$\sup\emptyset=-\infty$. If $A\setminus V_{n}\neq\emptyset$ then the limit is
equal to $0$.

######  Assumption 4.

For $W^{\prime}$ as in 3, there is
$V\in\mathcal{D}^{ext}_{2}(\mathcal{M}_{+})$ such that

  1. (i)

For $x_{n}\in\mathcal{M}_{+}$, $V(x_{n})\to\infty$ implies
$x_{n}\to\mathcal{M}_{0}$ (meaning $d(x_{n},\mathcal{M}_{0})\to 0$).

  2. (ii)

$\mathcal{L}V$ vanishes over $W^{\prime}$.

  3. (iii)

$\mathcal{L}V$ extends to a continuous function $H:\mathcal{M}\to\mathbb{R}$
and there is a constant $\alpha>0$ such that $\mu H\geq\alpha$ for all $\mu\in
P_{inv}(\mathcal{M}_{0})$.

######  Remark 2.19.

In view of 2.18, the density of $\mathcal{M}_{+}$ implies that $H$, the
continuous extension of $\mathcal{L}V$, also vanishes over $W^{\prime}$.

######  Remark 2.20.

It is shown below in 2.22 that under our assumptions $H$ is $\mu$-integrable
for all $\mu\in P_{inv}(\mathcal{M})$, so the condition $\mu H\geq\alpha$ in 4
(iii) makes sense.

The function $V$ from 4 could be called an “average Lyapunov function.”
Heuristically, under our continuity assumptions, to assess if
$\mathcal{M}_{0}$ is an “attractor” in some sense, it should be enough to
examine the dynamics of $X_{t}^{x}$ on $\mathcal{M}_{0}$. 4 (iii) asserts that
on $\mathcal{M}_{0}$, the time-averaged value of $H$ should be at least
$\alpha$, so if $X_{t}^{x}$ stays near $\mathcal{M}_{0}$ for a long time then
$V(X_{t}^{x})$ should grow at least as $t\mapsto\alpha t$, and consequently
$V(X_{t}^{x})\to\infty$ as $t\to\infty$. Then, by 4 (i), $X_{t}^{x}$
approaches $\mathcal{M}_{0}$. To show that $X_{t}^{x}$ stays near
$\mathcal{M}_{0}$ most of the time is not straightforward and it is one of the
major challenges of the present paper.

The next assumption is to ensure that $V$ and $W$ have linearly bounded
quadratic variation (see 2.11):

######  Assumption 5.

Let $W$ and $V$ be respectively as in 3 and 4. Assume that there are
continuous maps $U,U^{\prime}:\mathcal{M}\to[0,\infty)$ and a constant $K>0$
such that:

  1. (i)

$U\in\mathcal{D}^{ext}_{+}(\mathcal{M})$ (see 2.9).

  2. (ii)

$\mathcal{L}U\leq K-U^{\prime}$.

  3. (iii)

$\Gamma W\leq KU^{\prime}$.

  4. (iv)

$\Gamma V\leq KU^{\prime}$.

######  Remark 2.21.

Note that we may without loss of generality assume that the constants $K$ in 3
and 5 are the same, since increasing $K$ keeps the relevant inequalities
valid.

Finally, we list some consequences of our assumptions that will be useful for
examples and also needed for the proof of 3.11.

######  Lemma 2.22.

Suppose 3 and 5 (i)-(iii) hold true and let $W^{\prime},K$ be as in 3. Then
for any $\mu\in P_{inv}(\mathcal{M})$ we have $\mu W^{\prime}\leq K$, and thus
$P_{inv}(\mathcal{M})$ is compact. Consequently, if
$H:\mathcal{M}\to\mathbb{R}$ is a continuous function which vanishes over
$W^{\prime}$, then $H$ is $\mu$-integrable for all $\mu\in
P_{inv}(\mathcal{M})$. In addition, $\mu\mapsto\mu H$ is a continuous function
on $P_{inv}(\mathcal{M})$ and thus $\inf_{\mu\in P_{inv}(\mathcal{M}_{0})}\mu
H>0$ is equivalent to $\mu H>0$ for all ergodic $\mu\in
P_{inv}(\mathcal{M}_{0})$.

###### Proof.

For the proof of $\mu W^{\prime}\leq K$ we refer an interested reader to
[Ben23, Theorem 2.2ii], and we remark that the argument follows from 4.6, 4.8,
Birkhoff’s ergodic theorem, and ergodic decomposition theorem.

The compactness of $P_{inv}(\mathcal{M})$ is a consequence of the tightness of
$P_{inv}(\mathcal{M})$ which follows from Chebyshev inequality since $\mu
W^{\prime}\leq K$ and $W^{\prime}$ is proper (see 2.13). By [BH22, Proposition
4.15] (detailed in 4.7), $\mu\mapsto\mu H$ is a continuous function of $\mu\in
P_{inv}(\mathcal{M})$, and so $\inf_{\mu\in P_{inv}(\mathcal{M}_{0})}\mu H>0$
is equivalent to $\mu H>0$ for all $\mu\in P_{inv}(\mathcal{M}_{0})$. It
suffices to consider only ergodic $\mu$ since $P_{inv}(\mathcal{M}_{0})$ is
convex with extreme points being ergodic measures. ∎

##  3\. Main Results

In this section we introduce the main theorems that are proved in the
remainder of the paper. In particular, we claim that under 1 – 5, on the
Markov process $X_{t}^{x}$, the set $\mathcal{M}_{0}$ is an “attractor” in the
following sense. If an initial condition $y$ is close to $\mathcal{M}_{0}$,
then with high probability $X_{t}^{y}$ approaches $\mathcal{M}_{0}$ as
$t\to\infty$ with rate determined by $V$ and $\alpha$. For precise statements
see 3.1 and 3.2. Since 3.1 is a direct consequence of 3.2 we only prove the
latter in Section 6 after proving preliminary lemmas in Section 4 and Section
5.

The next main result, 3.5, asserts that if $\mathcal{M}_{0}$ satisfies some
accessbility conditions, then almost surely $X_{t}^{y}$ approaches
$\mathcal{M}_{0}$ as $t\to\infty$ for any $y\in\mathcal{M}_{+}$. The proof
that 3.2 is a consequence of 3.5 is given in Section 7.

In many cases, we need to enlarge $\mathcal{M}_{0}$ in order to fully capture
the dynamics of $X_{t}^{x}$ as $x\to\mathcal{M}_{0}$. For example, if
$\mathcal{M}=\mathbb{R}^{n}$ and $\mathcal{M}_{0}$ is a single point, then it
is natural to blow up $\mathcal{M}_{0}$ into a sphere $S^{n-1}$ and
$\mathcal{M}$ into $S^{n-1}\times[0,\infty)$, essentially transforming the
problem to polar coordinates. Such transformation is formalized in 3.7. The
“blow up” procedure is used if $\mathcal{L}V$ from 4 cannot be extended
continuously to all of $\mathcal{M}$ (in particular to $\mathcal{M}_{0}$), but
it can be extended continuously to a larger “blown up” space. Then 3.9 applies
and asserts that the conclusions of 3.1, 3.2, and 3.5 still hold.

The most crucial of all of our assumptions is 4 (iii), which states that
$\inf_{\mu\in P_{inv}(\mathcal{M}_{0})}\mu H>0$. Then, informally, 3.11 claims
that $(H,X_{t}^{x})\mapsto\inf_{\mu\in P_{inv}(\mathcal{M}_{0})}\mu H$ is
(lower semi-)continuous.

Finally, in 3.13 we state that several assumptions of the main results are
immediately satisfied if $\mathcal{M}$ is compact. Let us proceed with the
rigorous statements of the main results.

######  Theorem 3.1.

Suppose that 1 – 5 are valid for the Markov quadruple
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$. If $x\in\mathcal{M}_{0}$ is such that $\lim_{y\to x}V(y)=\infty$, then

$\lim_{y\to
x,y\in\mathcal{M}_{+}}\mathbb{P}\Big{(}\liminf_{t\to\infty}\frac{V(X_{t}^{y})}{t}\geq\alpha\Big{)}=1.$

######  Theorem 3.2.

Suppose that 1 – 5 are valid for the Markov quadruple
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$. Then for every $M,\delta>0$ there is $D>0$ such that for any
$y\in\mathcal{M}_{+}\cap\\{V\geq D\\}\cap\\{W\leq M\\}$ we have

$\mathbb{P}\Big{(}\liminf_{t\to\infty}\frac{V(X_{t}^{y})}{t}\geq\alpha\Big{)}\geq
1-\delta.$

######  Remark 3.3.

Note that 3.2 implies 3.1 by setting $M=W(x)+1$. Also, by 4 (i),
$V(X_{t}^{y})\to\infty$ implies $X_{t}^{y}\to\mathcal{M}_{0}$ as $t\to\infty$,
and therefore $\liminf_{t\to\infty}\frac{V(X_{t}^{y})}{t}\geq\alpha$ could be
replaced with $X_{t}^{y}\to\mathcal{M}_{0}$ in 3.1 and 3.2. Alternatively,
using results in Section 4.2 one could replace
$\liminf_{t\to\infty}\frac{V(X_{t}^{y})}{t}\geq\alpha$ by the statement that
all limit points of the empirical occupation measures $\mu_{t}^{y}$ (defined
in 4.4) as $t\to\infty$ lie in $P_{inv}(\mathcal{M}_{0})$.

For our next result, we need to review the definition of accessibility. The
equivalence of the listed conditions in 3.4 is proved in 7.1.

######  Definition 3.4.

For $x\in\mathcal{M}$ and open $\mathcal{U}\subset\mathcal{M}$ we say
$\mathcal{U}$ is accessible from $x$ if any of the following equivalent
conditions hold:

  1. (1)

$\int_{0}^{\infty}e^{-t}\mathbb{P}(X_{t}^{x}\in\mathcal{U})dt>0$.

  2. (2)

There exists $t\geq 0$ such that $\mathbb{P}(X_{t}^{x}\in\mathcal{U})>0$.

  3. (3)

$\mathbb{P}(\exists t\geq 0\text{ such that }X_{t}^{x}\in\mathcal{U})>0$.

######  Theorem 3.5.

Suppose that 1 – 5 are valid for the Markov quadruple
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$. Assume additionally that
$\mathcal{M}=\mathcal{M}_{+}\cup\mathcal{M}_{0}$ and that every point
$x\in\mathcal{M}_{+}$ satisfies the following accessibility condition:

(3.1) $\exists M>0\text{ such that }\forall D>0,\\{V>D\\}\cap\\{W<M\\}\text{
is accessible from }x\,.$

Then

$\mathbb{P}\Big{(}\liminf_{t\to\infty}\frac{V(X_{t}^{x})}{t}\geq\alpha\Big{)}=1$

for all $x\in\mathcal{M}_{+}$.

######  Remark 3.6.

A sufficient condition for (3.1) to hold for $x\in\mathcal{M}_{+}$ is that
there is $y\in\mathcal{M}_{0}$ such that $\lim_{z\to
y,z\in\mathcal{M}_{+}}V(z)=\infty$ and $y$ is accessible from $x$ in the sense
that for all open sets $\mathcal{U}$ containing $y$, $\mathcal{U}$ is
accessible from $x$. Indeed, set $M=W(y)+1$ and for any $D>0$ choose
$\epsilon>0$ small enough that
$\mathcal{U}\coloneqq\\{d(y,\cdot)<\epsilon\\}$, the ball of radius $\epsilon$
around $y$, satisfies
$\mathcal{U}\cap\mathcal{M}_{+}\subset\\{V>D\\}\cap\\{W<M\\}$. We have by
assumption that $\mathcal{U}$ is accessible from $x$. Since
$x\in\mathcal{M}_{+}$ and $\mathcal{M}_{+}$ is invariant by 1, we have
$\mathbb{P}(X_{t}^{x}\in\mathcal{U})\leq\mathbb{P}(X_{t}^{x}\in\\{V>D\\}\cap\\{W<M\\})$
for any $t>0$, and thus $\\{V>D\\}\cap\\{W<M\\}$ is also accessible from $x$.

######  Definition 3.7.

Let
$(\mathcal{N},\mathcal{N}_{0},\mathcal{N}_{+},\\{Y_{t}^{y}\\}_{y\in\mathcal{N},t\geq
0})$ and
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$ be Markov quadruples (see 2.1). Then $\pi:\mathcal{N}\to\mathcal{M}$ is
called a quadruple map if $\pi$ is a continuous surjection such that:

  1. (1)

$\mathcal{N}_{0}=\pi^{-1}(\mathcal{M}_{0})$ and
$\mathcal{N}_{+}=\pi^{-1}(\mathcal{M}_{+})$.

  2. (2)

$\pi^{-1}(\mathcal{K})$ is compact for all $\mathcal{K}\subset\mathcal{M}$
compact.

  3. (3)

For $y_{n}\in\mathcal{N}_{+}$, $\pi(y_{n})\to\mathcal{M}_{0}$ implies
$y_{n}\to\mathcal{N}_{0}$.

  4. (4)

For all $y\in\mathcal{N}$, $\pi(Y_{\cdot}^{y})=X_{\cdot}^{\pi(y)}$ almost
surely.

######  Remark 3.8.

It follows from 3.7 that if
$(\mathcal{N},\mathcal{N}_{0},\mathcal{N}_{+},\\{Y_{t}^{y}\\}_{y\in\mathcal{N},t\geq
0})$ is a Feller quadruple (2.6) then so is
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$, but we do not use this fact. In this case, if $\mu\in
P_{inv}(\mathcal{N})$ (respectively $P_{inv}(\mathcal{N}_{0})$) then it
standard to prove from 3.7 that the pushforward measure $\pi^{*}\mu$ (given by
$\pi^{*}\mu f=\mu f\circ\pi$ for $f\in C_{b}(\mathcal{M})$) is in
$P_{inv}(\mathcal{M})$ (respectively in $P_{inv}(\mathcal{M}_{0})$).

######  Theorem 3.9.

Let
$(\mathcal{N},\mathcal{N}_{0},\mathcal{N}_{+},\\{Y_{t}^{y}\\}_{y\in\mathcal{N},t\geq
0})$ and
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$ be Feller quadruples (see 2.6) and $\pi:\mathcal{N}\to\mathcal{M}$ is a
quadruple map. Then 3.2 (and thus its corollaries 3.1 and 3.5) remain valid
with 4 (iii) replaced with the assumption that:

(3.2) $\mathcal{L}V\circ\pi$ extends to a continuous function
$H:\mathcal{N}\to\mathbb{R}$ and there is a constant $\alpha>0$ such that $\mu
H\geq\alpha$ for all $\mu\in P_{inv}(\mathcal{N}_{0})$.

######  Remark 3.10.

The assumptions of 3.9 are somewhat redundant, since by 2.6, 1 – 2 are
automatically satisfied.

###### Proof.

As an consequence of 2.9, 2.10, and 3.7 we immediately obtain that for any
$f:\mathcal{M}\to\mathbb{R}$ it holds that

  1. (1)

If $f\in\mathcal{D}^{ext}_{+}(\mathcal{M})$, then
$f\circ\pi\in\mathcal{D}^{ext}_{+}(\mathcal{N})$ and we may take
$\mathcal{L}(f\circ\pi)=\mathcal{L}f\circ\pi$.

  2. (2)

If $f\in\mathcal{D}^{ext}_{2}(\mathcal{M})$, then
$f\circ\pi\in\mathcal{D}^{ext}_{2}(\mathcal{N})$ and we may take
$\mathcal{L}(f\circ\pi)=\mathcal{L}f\circ\pi$, $\Gamma(f\circ\pi)=\Gamma
f\circ\pi$.

Also, (1) and (2) hold with $\mathcal{M},\mathcal{N}$ replaced with
$\mathcal{M}_{+},\mathcal{N}_{+}$.

Suppose 1 – 5 are valid for
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$, where we replace 4 (iii) with (3.2). Then it follows easily from 2.6 and
3.7 that 1 – 5 are satisfied for
$(\mathcal{N},\mathcal{N}_{0},\mathcal{N}_{+},\\{Y_{t}^{y}\\}_{y\in\mathcal{N},t\geq
0})$ with $W,W^{\prime},U,U^{\prime}$ from 3 and 5 being replaced respectively
by $W\circ\pi,W^{\prime}\circ\pi,U\circ\pi,U^{\prime}\circ\pi,V\circ\pi$.

Thus, by 3.2 for every $M>0,\delta>0$ there is a $D>0$ such that for any
$y\in\mathcal{N}_{+}\cap\\{V\circ\pi\geq D\\}\cap\\{W\circ\pi\leq M\\}$ we
have

$\mathbb{P}\Big{(}\liminf_{t\to\infty}\frac{V(\pi(Y_{t}^{y}))}{t}\geq\alpha\Big{)}\geq
1-\delta.$

If $x\in\mathcal{M}_{+}\cap\\{V\geq D\\}\cap\\{W\leq M\\}$ then by
surjectivity of $\pi$ and $\mathcal{N}_{+}=\pi^{-1}(\mathcal{M}_{+})$ there is
some $y\in\mathcal{N}_{+}\cap\\{V\circ\pi\geq D\\}\cap\\{W\circ\pi\leq M\\}$
such that $\pi(y)=x$ and so the claim follows by noting that
$\pi(Y_{\cdot}^{y})=X_{\cdot}^{x}$. ∎

######  Theorem 3.11.

Let $\Theta$ be a compact metric space and let
$\\{\\{X_{\theta,t}^{x}\\}_{x\in\mathcal{M},t\geq 0}\\}_{\theta\in\Theta}$ be
a collection of Markov processes. For each $\theta\in\Theta$ denote
$P_{inv}^{\theta}(\mathcal{M})$ the set of invariant probability measures (see
2.16) for $X_{\theta,t}^{x}$. Define the Markov process $Y_{t}^{(\theta,x)}$
on $\Theta\times\mathcal{M}$ by
$Y_{t}^{(\theta,x)}=(\theta,X_{\theta,t}^{x})$.

Suppose that $Y_{t}^{(\theta,x)}$ satisfies 2 and that each
$\\{X_{\theta,t}^{x}\\}_{x\in\mathcal{M},t\geq 0}$ satisfies 3 and 5 (i)-(iii)
with the same $W,W^{\prime},K,U,U^{\prime}$ (independent of $\theta$). Then if
$\theta_{n},\theta_{\infty}\in\Theta$ and $\mu_{n}\in
P^{\theta_{n}}_{inv}(\mathcal{M})$ are such that
$\theta_{n}\to\theta_{\infty}$, then there is a subsequence of $\mu_{n}$
converging to some $\mu\in P_{inv}^{\theta_{\infty}}(\mathcal{M})$.

Thus, if $H:\Theta\times\mathcal{M}\to\mathbb{R}$ is a continuous function
which vanishes over $(\theta,x)\mapsto W^{\prime}(x)$ (see 2.18), then for all
$\theta_{\infty}\in\Theta$ we have

(3.3) $\liminf_{\theta\to\theta_{\infty}}\inf_{\mu\in
P^{\theta}_{inv}(\mathcal{M})}\mu H_{\theta}\geq\inf_{\mu\in
P^{\theta_{\infty}}_{inv}(\mathcal{M})}\mu H_{\theta_{\infty}}\,,$

where $H_{\theta}(x)\coloneqq H(\theta,x)$. In other words, the function
$\theta\mapsto\inf_{\mu\in P^{\theta}_{inv}(\mathcal{M})}\mu H_{\theta}$ is
lower semicontinuous.

######  Remark 3.12.

The condition that
$\\{Y_{t}^{(\theta,x)}\\}_{(\theta,x)\in\Theta\times\mathcal{M}}$ satisfies 2
is equivalent to the following convergence of the semigroups
$\mathcal{P}_{t}^{\theta}$ of $\\{X_{\theta,t}^{x}\\}_{x\in\mathcal{M},t\geq
0}$: if $f\in C_{b}(\mathcal{M})$ and $\Theta\ni\theta_{n}\to\theta\in\Theta$,
then $\mathcal{P}_{t}^{\theta_{n}}f\to\mathcal{P}_{t}^{\theta_{\infty}}f$
uniformly on compact subsets of $\mathcal{M}$.

###### Proof.

We use $\tilde{W}$, $\tilde{W^{\prime}}$, $\tilde{U}$, $\tilde{U^{\prime}}$ to
denote the maps defined by precomposing $W$, $W^{\prime}$, $U$, $U^{\prime}$
with $(\theta,x)\mapsto x$ so that by the definition
$Y_{t}^{(\theta,x)}=(\theta,X_{\theta,t}^{x})$ it follows that 3 and 5
(i)-(iii) are satisfied for $Y_{t}^{(\theta,x)}$ with $\tilde{W}$,
$\tilde{W^{\prime}}$, $\tilde{U}$, $\tilde{U^{\prime}}$ in place of $W$,
$W^{\prime}$, $U$, $U^{\prime}$. Let $\theta_{n},\theta_{\infty}\in\Theta$ and
$\mu_{n}\in P^{\theta_{n}}_{inv}(\mathcal{M})$ be such that
$\theta_{n}\to\theta_{\infty}$. Let $P_{inv}(\Theta\times\mathcal{M})$ denote
the set of invariant measures for
$\\{Y_{t}^{(\theta,x)}\\}_{(\theta,x)\in\Theta\times\mathcal{M}}$. It readily
follows from 2.16 and $Y_{t}^{(\theta,x)}=(\theta,X_{\theta,t}^{x})$ that
$\nu_{n}\coloneqq\delta_{\theta_{n}}\otimes\mu_{n}\in
P_{inv}(\Theta\times\mathcal{M})$, where $\delta_{\theta_{n}}$ is the dirac
delta measure at $\theta_{n}$ and $\otimes$ denotes the usual product of
measures. By 2.22 we have that there is a subsequence of $\nu_{n}$ converging
to some $\nu\in P(\Theta\times\mathcal{M})$. Without loss of generality we
assume $\nu=\lim_{n\to\infty}\nu_{n}$ (otherwise pass to a sub-sequence). By
Portmanteau theorem, $\nu=\delta_{\theta_{\infty}}\otimes\mu$ for some $\mu\in
P(\mathcal{M})$ such that $\mu=\lim_{n\to\infty}\mu_{n}$. We conclude the
proof by showing that $\mu\in P_{inv}^{\theta_{\infty}}(\mathcal{M})$, or
equivalently that $\nu\in P_{inv}(\Theta\times\mathcal{M})$. Indeed, if $f\in
C_{b}(\Theta\times\mathcal{M})$ and $t\geq 0$ then

$\nu
f=\lim_{n\to\infty}\nu_{n}f=\lim_{n\to\infty}\nu_{n}\mathcal{P}^{Y}_{t}f=\nu\mathcal{P}^{Y}_{t}f\,,$

where $\mathcal{P}^{Y}_{t}$ denotes the semigroup of
$\\{Y_{t}^{(\theta,x)}\\}_{(\theta,x)\in\Theta\times\mathcal{M}}$.

The claim (3.3) is a consequence of the first claim and 2.22. ∎

######  Lemma 3.13.

For compact $\mathcal{M}$, 3, 5, and 4 (ii) are satisfied if
$\sup_{x\in\mathcal{M}_{+}}\Gamma V(x)<\infty$.

###### Proof.

If $\mathcal{M}$ is compact then constant functions $g(x)=c>0$ are proper (as
defined in 2.13) and all functions vanish over $g$ (take $V_{n}=\mathcal{M}$
in 2.18). Thus, we may simply take $W,U,W^{\prime},U^{\prime}\equiv 1$ and
$K=1+\sup_{x\in\mathcal{M}_{+}}\Gamma V(x)$. ∎

##  4\. General Facts

This section contains statements and proofs of some technical results related
to the martingales $M_{t}^{W},M_{t}^{V},M_{t}^{U}$ (see (2.1)), the empirical
occupation measures (see 4.4), and discrete semimartingales. Recall that
functions $W,V$, and $U$ were defined in 3–5. In Section 4.1 we prove
estimates on stopping times that are frequently used in Section 5 and Section
6. In Section 4.2 we show that almost surely the empirical occupation measures
are tight and that all of their limit points (with time approaching to
infinity) are invariant measures. We also obtain a sufficient conditions that
imply 3.2 and these conditions are verified below in 6.1. Finally, in Section
4.3 we recall and prove some basic facts about discrete-time
(semi-)martingales which serve as the motivation for the proof of 6.1. In the
entire section we fix a Markov quadruple
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$ satisfying 1–5.

###  4.1. Optional Stopping Inequalities

######  Lemma 4.1.

Let $W,W^{\prime}$ and $U,U^{\prime}$ be as in 3 and 5 respectively. For any
$x\in\mathcal{M}$ and a stopping time $\tau$ such that
$\mathbb{E}[\tau]<\infty$, we have

$\mathbb{E}\Big{[}W(X_{\tau}^{x})+\int_{0}^{\tau}W^{\prime}(X_{s}^{x})ds\Big{]}\leq
W(x)+K\mathbb{E}[\tau].$

In particular, $\mathcal{P}_{t}W+\int_{0}^{t}\mathcal{P}_{s}W^{\prime}ds\leq
W+Kt$. The same inequalities hold with $W,W^{\prime}$ replaced by
$U,U^{\prime}$ respectively.

###### Proof.

Let $M_{t}\coloneqq M_{t}^{W}(x)$ be the local martingale defined in (2.1).
Let $(\tau_{n})_{n\in\mathbb{N}}$ being a localizing sequence, meaning
$\tau_{n}$ is an increasing sequence of bounded stopping times such that
$\tau_{n}\uparrow\infty$ almost surely and $M_{t\wedge\tau_{n}}$ is a
martingale for all $n$. Then, by the optional stopping theorem and the
definition (2.1), for each $n\geq 1$ we have

$0=\mathbb{E}[M_{\tau_{n}\wedge\tau}]=\mathbb{E}\Big{[}W(X_{\tau_{n}\wedge\tau}^{x})-W(x)-\int_{0}^{\tau_{n}\wedge\tau}\mathcal{L}W(X_{s}^{x})ds\Big{]}.$

By 3 we have $W\geq 1$ a d $\mathcal{L}W\leq K-W^{\prime}\leq K$, and
therefore

$W(X_{\tau_{n}\wedge\tau}^{x})-W(x)-\int_{0}^{\tau_{n}\wedge\tau}\mathcal{L}W(X_{s}^{x})ds\geq-W(x)-K(\tau_{n}\wedge\tau)\geq-W(x)-K\tau\,.$

Since $\mathbb{E}[W(x)+K\tau]<\infty$, Fatou’s lemma and
$X^{x}_{\tau_{n}\wedge\tau}\to X^{x}_{\tau}$ almost surely as $n\to\infty$
give

$0\geq\mathbb{E}\Big{[}W(X_{\tau}^{x})-W(x)-\int_{0}^{\tau}\mathcal{L}W(X_{s}^{x})ds\Big{]}\,.$

Using $\mathcal{L}W\leq K-W^{\prime}$ again, we have

$0\geq\mathbb{E}\Big{[}W(X_{\tau}^{x})-W(x)-K\tau+\int_{0}^{\tau}W^{\prime}(X_{s}^{x})ds\Big{]}\,.$

Adding $W(x)+K\mathbb{E}[\tau]<\infty$ to both sides proves the first claim.

Setting $\tau=t$ gives

$\mathbb{E}\Big{[}W(X_{t}^{x})+\int_{0}^{t}W^{\prime}(X_{s}^{x})ds\Big{]}\leq
W(x)+Kt.$

By definition of $\mathcal{P}_{t}$ we have
$\mathbb{E}[W(X_{t}^{x})]=\mathcal{P}_{t}W(x)$, and consequently

$\mathbb{E}\Big{[}\int_{0}^{t}W^{\prime}(X_{s}^{x})ds\Big{]}=\int_{0}^{t}\mathbb{E}[W^{\prime}(X_{s}^{x})]ds=\int_{0}^{t}\mathcal{P}_{s}W^{\prime}(x)ds$

by Tonelli’s theorem and the definition of $\mathcal{P}_{s}$, proving the
second claim. The proof for $U,U^{\prime}$ is analogous with 3 replaced by 5.
∎

######  Corollary 4.2.

For any $x\in\mathcal{M}$ and any stopping time $\tau$ with
$\mathbb{E}[\tau]<\infty$ we have

(4.1) $\mathbb{E}[M^{W}_{\tau}(x)^{2}]=\mathbb{E}\Big{[}\int_{0}^{\tau}\Gamma
W(X_{s}^{x})ds\Big{]}\leq K(U(x)+K\mathbb{E}[\tau])\,,$

where the martingale $M_{t}^{f}(x)$ was defined in (2.1). Similarly, for any
$x\in\mathcal{M}_{+}$ and any stopping time $\tau$ with
$\mathbb{E}[\tau]<\infty$ we have

$\mathbb{E}[M^{V}_{\tau}(x)^{2}]=\mathbb{E}\Big{[}\int_{0}^{\tau}\Gamma
V(X_{s}^{x})ds\Big{]}\leq K(U(x)+K\mathbb{E}[\tau]).$

Specifically, $V$ and $W$ have linearly bounded quadratic variation (see
2.11).

###### Proof.

We only prove the claim for $W$ as the proof is analogous for $V$.

Since $W\in\mathcal{D}^{ext}_{2}(\mathcal{M})$ (see 3),
$M_{t}^{W}(x)^{2}-\int_{0}^{t}\Gamma W(X_{s}^{x})ds$ is a martingale by 2.10.
By the Optional stopping theorem

$\mathbb{E}\Big{[}(M_{t\wedge\tau}^{W}(x))^{2}-\int_{0}^{t\wedge\tau}\Gamma
W(X_{s}^{x})ds\Big{]}=0.$

By 5, $\Gamma W\leq KU^{\prime}$ and so 4.1 gives

(4.2) $\displaystyle\mathbb{E}[(M_{t\wedge\tau}^{W}(x))^{2}]$
$\displaystyle=\mathbb{E}\Big{[}\int_{0}^{t\wedge\tau}\Gamma
W(X_{s}^{x})ds\Big{]}$ $\displaystyle\leq
K\mathbb{E}\Big{[}\int_{0}^{t\wedge\tau}U^{\prime}(X_{s}^{x})ds\Big{]}\leq
K(U(x)+K\mathbb{E}[t\wedge\tau]).$

Thus,

$\sup_{t\geq 0}\mathbb{E}\Big{[}(M_{t\wedge\tau}^{W}(x))^{2}\Big{]}\leq
K(U(x)+K\mathbb{E}[\tau])<\infty$

and by the Martingale convergence theorem, $M_{t\wedge\tau}^{W}(x)\to
M_{\tau}^{W}(x)$ in $L^{2}$. Passing $t\to\infty$ in (4.2) implies that

$\mathbb{E}[M^{W}_{\tau}(x)^{2}]=\mathbb{E}\Big{[}\int_{0}^{\tau}\Gamma
W(X_{s}^{x})ds\Big{]}\leq K(U(x)+K\mathbb{E}[\tau])$

and (4.1) follows. To show that $W$ has linearly bounded quadratic variation
(see 2.11), we note that for $x\in\mathcal{M}$, by (4.2) with $\tau=t$ and
Tonelli’s theorem

$\sup_{t\geq 1}\frac{1}{t}\int_{0}^{t}\mathcal{P}_{s}\Gamma
W(x)ds\leq\sup_{t\geq 1}\frac{K(U(x)+Kt)}{t}\leq KU(x)+K^{2}\,,$

as desired. ∎

######  Corollary 4.3.

If $x\in\mathcal{M}_{+}$ and $\tau$ is a stopping time such that
$\mathbb{E}[\tau]<\infty$, then

$\mathbb{E}\Big{[}\int_{0}^{\tau}|H(X_{s}^{x})|ds\Big{]}<\infty\qquad\textrm{and}\quad\mathbb{E}[V(X_{\tau}^{x})-V(x)]=\mathbb{E}\Big{[}\int_{0}^{\tau}H(X_{s}^{x})ds\Big{]}.$

###### Proof.

Recall from 4 that $V\in\mathcal{D}^{ext}_{2}(\mathcal{M}_{+})$ and
$\mathcal{L}V$ is the restriction of $H$ to $\mathcal{M}_{+}$. Hence, by 2.10
and 4.2, for each $t\geq 0$ we have that

$M_{t}\coloneqq M_{\tau\wedge t}^{V}(x)=V(X_{\tau\wedge
t}^{x})-V(x)-\int_{0}^{\tau\wedge t}H(X_{s}^{x})ds$

is a square-integrable martingale with

$\sup_{t\geq 0}\mathbb{E}[M_{t}^{2}]\leq K(U(x)+K\mathbb{E}[\tau]).$

By Martingale convergence theorem, $M_{t}\to
V(X_{\tau}^{x})-V(x)-\int_{0}^{\tau}H(X_{s}^{x})ds$ in $L^{2}$ as $t\to\infty$
and thus

(4.3)
$0=\mathbb{E}\Big{[}V(X_{\tau}^{x})-V(x)-\int_{0}^{\tau}H(X_{s}^{x})ds\Big{]}\,.$

Since $H$ vanishes over $W^{\prime}$ (see 2.19, 2.18), there are constants
$A>0,b>0$ such that $|H|\leq A+bW^{\prime}$. Consequently, 4.1 gives

(4.4) $\displaystyle\mathbb{E}\Big{[}\int_{0}^{\tau}|H(X_{s}^{x})|ds\Big{]}$
$\displaystyle\leq
A\mathbb{E}[\tau]+b\mathbb{E}\Big{[}\int_{0}^{\tau}W^{\prime}(X_{s}^{x})ds\Big{]}$
$\displaystyle\leq bW(x)+(A+bK)\mathbb{E}[\tau]<\infty,$

which proves the claim after adding
$\mathbb{E}[\int_{0}^{\tau}H(X_{s}^{x})ds]$ to both sides of (4.3). ∎

###  4.2. Empirical Occupation Measures

The empirical occupation measures as defined below in 4.4 capture the behavior
of $X_{t}^{x}$ averaged over time, and are central to the proof of 3.2.

######  Definition 4.4.

For any $x\in\mathcal{M}$ and $t>0$ let $\mu_{t}^{x}$ denote the empirical
occupation measure

(4.5)
$\mu_{t}^{x}(\omega)\coloneqq\frac{1}{t}\int_{0}^{t}\delta_{X_{s}^{x}(\omega)}ds,$

where $\delta_{y}$ is the Dirac measure concentrated at $y$, that is,
$\delta_{y}f=f(y)$. Note that $\omega\in\Omega$, and therefore $\mu_{t}^{x}$
is a random measure which depends on $\omega$, but often this dependence is
suppressed.

######  Remark 4.5.

Since the paths $t\mapsto X_{t}^{x}$ are cadlag a.s., almost surely all
continuous functions on $\mathcal{M}$ (in particular $W^{\prime}$ and $H$) are
$\mu_{t}^{x}$-integrable for all $t>0$.

For what follows, recall 2.15 and 2.16.

######  Lemma 4.6.

If $x\in\mathcal{M}$ (respectively $x\in\mathcal{M}_{0}$), then almost surely
the following holds. For every sequence $t_{n}\to\infty$ such that
$\mu_{t_{n}}^{x}(\omega)$ converges to some $\mu\in P(\mathcal{M})$, then
$\mu\in P_{inv}(\mathcal{M})$ (respectively $\mu\in
P_{inv}(\mathcal{M}_{0})$).

###### Proof.

The claim is proven for $\mathcal{M}$ in [Ben23, Theorem 2.2ii].

For any $x\in\mathcal{M}_{0}$, additionally note that Portmanteau theorem and
1 imply that if $\mu_{t_{n}}^{x}\to\mu$, then
$\mu(\mathcal{M}_{0})\geq\limsup_{n\to\infty}\mu_{t_{n}}^{x}(\mathcal{M}_{0})=1$
a.s. ∎

######  Lemma 4.7.

[BH22, Proposition 4.15] If $f:\mathcal{M}\to\mathbb{R}_{+}$ is proper and
$\mu_{n}\in P(\mathcal{M})$ is such that
$\limsup_{n\to\infty}\mu_{n}f<\infty$, then $(\mu_{n})_{n\in\mathbb{N}}$ is
tight. If furthermore $g:\mathcal{M}\to\mathbb{R}$ is a continuous function
which vanishes over $f$, then $\mu_{n}\to\mu$ implies $\mu_{n}g\to\mu g$.

######  Corollary 4.8.

For $\mu_{t}^{x}$ as in 4.4 we have
$\limsup_{t\to\infty}\mu_{t}^{x}(W^{\prime})\leq K$ a.s. Thus, almost surely
$(\mu_{t_{n}}^{x})_{n\in\mathbb{N}}$ is tight for all $t_{n}\to\infty$.

###### Proof.

A similar proof can be found in [Ben23, Theorem 2.2ii] but we include the
details here for completeness.

Since $W$ has linearly bounded quadratic variation (4.2), $M_{t}^{W}(x)$
satisfies the strong law for martingales (2.12), and thus
$\frac{-W(X_{t}^{x})}{t}+\mu_{t}^{x}(\mathcal{L}W)\to 0$ a.s. as $t\to\infty$.
Moreover, from $W\geq 0$ and $\mathcal{L}W\leq K-W^{\prime}$ (see 3) it
follows that

$\displaystyle 0$
$\displaystyle\leq\liminf_{t\to\infty}\frac{W(X_{t}^{x})}{t}=\liminf_{t\to\infty}\mu_{t}^{x}(\mathcal{L}W)\leq\liminf_{t\to\infty}K-\mu_{t}^{x}W^{\prime}\,,$

and therefore

$\limsup_{t\to\infty}\mu_{t}^{x}W^{\prime}\leq K\,.$

Hence, since $W^{\prime}$ is proper, by 4.7 almost surely
$(\mu_{t_{n}}^{x}(\omega))_{n\in\mathbb{N}}$ is tight for all
$t_{n}\to\infty$. ∎

######  Lemma 4.9.

For all $x\in\mathcal{M}_{+}\cup\mathcal{M}_{0}$ and $\mu_{t}^{x}$ defined in
(4.5), $\mathbb{P}$-a.s the following are equivalent:

  1. (i)

$\liminf_{t\to\infty}\mu_{t}^{x}H\geq\alpha$.

  2. (ii)

$\liminf_{t\to\infty}\frac{V(X_{t}^{x})}{t}\geq\alpha$ or
$x\in\mathcal{M}_{0}$.

  3. (iii)

If $t_{n}\to\infty$ with $\mu_{t_{n}}^{x}\to\mu$ for some $\mu\in
P(\mathcal{M})$, then $\mu(\mathcal{M}_{0})=1$.

###### Proof.

To show (i) implies (ii) we assume $x\in\mathcal{M}_{+}$ and show that a.s.

$\liminf_{t\to\infty}\mu_{t}^{x}H\geq\alpha\quad\textrm{ implies
}\quad\liminf_{t\to\infty}\frac{V(X_{t}^{x})}{t}\geq\alpha\,.$

By the definition of $M^{V}_{t}$ in (2.1) and $\mathcal{L}V$ being the
restriction of $H$ to $\mathcal{M}_{+}$, we have

$\frac{M^{V}_{t}(x)}{t}=\frac{V(X_{t}^{x})}{t}-\frac{V(x)}{t}-\frac{1}{t}\int_{0}^{t}H(X_{s}^{x})ds=\frac{V(X_{t}^{x})}{t}-\frac{V(x)}{t}-\mu_{t}^{x}H\,.$

By 4.2, $V$ has linearly bounded quadratic variation, so by 2.12
$\frac{M^{V}_{t}(x)}{t}\to 0$ as $t\to\infty$, proving the claim.

To show (ii) implies (iii), we first note that if $x\in\mathcal{M}_{+}$ and
$\liminf_{t\to\infty}\frac{V(X_{t}^{x})}{t}\geq\alpha$ a.s., then by 4 (i) it
follows that $d(X_{t}^{x},\mathcal{M}_{0})\to 0$ a.s. as $t\to\infty$. Also,
if $x\in\mathcal{M}_{0}$, then $d(X_{t}^{x},\mathcal{M}_{0})=0$ for each
$t\geq 0$ by 1 (see 2.2). In either case, the Portmanteau Theorem implies

$\mu(\mathcal{M}_{0})=\lim_{\epsilon\downarrow
0}\mu(\\{d(\cdot,\mathcal{M}_{0})\leq\epsilon\\})\geq\lim_{\epsilon\downarrow
0}\limsup_{n\to\infty}\mu_{t_{n}}^{x}(\\{d(\cdot,\mathcal{M}_{0})\leq\epsilon\\})=1\,.$

To show (iii) implies (i), first note that by 4.8, almost surely for all
$t_{n}\to\infty$, the sequence $(\mu_{t_{n}}^{x})_{n\in\mathbb{N}}$ is tight
and thus has a further subsequence $\mu_{t_{n_{k}}}^{x}$ which converges to
some $\mu$. Applying 4.6 and (iii), $\mu\in P_{inv}(\mathcal{M}_{0})$, so 4
(iii) implies $\mu H\geq\alpha$. Since $H$ vanishes over $W^{\prime}$ (2.19)
and $\limsup_{t\to\infty}\mu_{t}^{x}(W^{\prime})\leq K$ (4.8), then
$\mu_{t_{n_{k}}}^{x}H\to\mu H\geq\alpha$ by 4.7. Thus,
$\liminf_{t\to\infty}\mu_{t}^{x}H\geq\alpha$. ∎

######  Corollary 4.10.

For $y\in\mathcal{M}_{+}$, we have a.s.

$\lim_{M\to\infty}\limsup_{t\to\infty}d(X_{t}^{y},\mathcal{M}_{0})\mathbbm{1}_{W(X_{t}^{y})<M}=0\quad\textrm{implies}\quad\liminf_{t\to\infty}\frac{V(X_{t}^{y})}{t}\geq\alpha.$

###### Proof.

We claim that a.s.

(4.6)
$\lim_{M\to\infty}\limsup_{t\to\infty}d(X_{t}^{y},\mathcal{M}_{0})\mathbbm{1}_{W(X_{t}^{y})<M}=0$

implies

(4.7) $\lim_{\epsilon\downarrow
0}\limsup_{t\to\infty}\mu_{t}^{y}(\\{d(\cdot,\mathcal{M}_{0})>\epsilon\\})=0\,.$

Indeed, suppose (4.6) holds and fix any $\epsilon>0$. By Chebyshev inequality
and 4.8, there is a $K_{\epsilon}=\frac{2K}{\epsilon}$ such that a.s.

(4.8)
$\limsup_{t\to\infty}\mu_{t}^{y}(\\{W^{\prime}>K_{\epsilon}\\})<\epsilon\,.$

Since $W^{\prime}$ is proper (3), the set $\\{W^{\prime}\leq K_{\epsilon}\\}$
is compact, so by continuity of $W$ there is $M>0$ such that $\\{W\geq
M\\}\subset\\{W^{\prime}>K_{\epsilon}\\}$. Combining this with (4.8) gives

(4.9) $\limsup_{t\to\infty}\mu_{t}^{y}(\\{W\geq
M\\})\leq\limsup_{t\to\infty}\mu_{t}^{y}(\\{W^{\prime}>K_{\epsilon}\\})<\epsilon\,.$

Using (4.6), by making $M$ large enough we may also assume

$\limsup_{t\to\infty}d(X_{t}^{y},\mathcal{M}_{0})\mathbbm{1}_{W(X_{t}^{y})<M}<\epsilon\,.$

In particular, there is a (random) $T>0$ such that
$d(X_{t}^{y},\mathcal{M}_{0})\mathbbm{1}_{W(X_{t}^{y})<M}<\epsilon$ for all
$t\geq T$. Thus,

(4.10)
$\limsup_{t\to\infty}\mu_{t}^{y}(\\{W<M\\}\cap\\{d(\cdot,\mathcal{M}_{0})>\epsilon\\})=0\,.$

Since

$\\{d(\cdot,\mathcal{M}_{0})>\epsilon\\}\subset\\{W\geq
M\\}\cup(\\{W<M\\}\cap\\{d(\cdot,\mathcal{M}_{0})>\epsilon\\}),$

then (4.9) and (4.10) show that

$\limsup_{t\to\infty}\mu_{t}^{y}(\\{d(\cdot,\mathcal{M}_{0})>\epsilon\\})<\epsilon$

and since $\epsilon>0$ is arbitrary, we have proven the claim that almost
surely (4.6) implies (4.7).

To finish the proof, we assume (4.7) holds and show that
$\liminf_{t\to\infty}\frac{V(X_{t}^{y})}{t}\geq\alpha$. By 4.9, it suffices to
show that for all $t_{n}\to\infty$, every limit point $\mu$ of
$(\mu_{t_{n}}^{y})_{n\in\mathbb{N}}$ satisfies $\mu(\mathcal{M}_{0})=1$. Due
to (4.7) and the Portmanteau theorem,

$\mu(\mathcal{M}_{0}^{c})=\lim_{\epsilon\downarrow
0}\mu(\\{d(\cdot,\mathcal{M}_{0})>\epsilon\\})\leq\lim_{\epsilon\downarrow
0}\liminf_{n\to\infty}\mu^{y}_{t_{n}}(\\{d(\cdot,\mathcal{M}_{0})>\epsilon\\})=0\,,$

as desired. ∎

###  4.3. Discrete-Time Semimartingales

######  Lemma 4.11.

Let $\\{M_{n}\\}_{n\in\mathbb{N}}$ be a (discrete-time) martingale with
respect to some filtration $\\{\mathcal{G}_{n}\\}_{n\in\mathbb{N}}$ with
$M_{0}=0$ and $\mathbb{E}[(M_{n}-M_{n-1})^{2}\mid\mathcal{G}_{n-1}]\leq B$ for
some constant $B>0$. Then for every $b>0$, $\delta>0$ there is some $C>0$ such
that

$\mathbb{P}(|M_{n}|\leq bn+C\text{ for all }n\geq 0)>1-\delta,$

and $C$ depends only on $\delta$, $B$, and $b$.

###### Proof.

The proof is similar to the proof of 2.12. First note that since
$\mathbb{E}[M_{n}\mid\mathcal{G}_{n-1}]=M_{n-1}$,

$\mathbb{E}[(M_{n}-M_{n-1})^{2}\mid\mathcal{G}_{n-1}]=\mathbb{E}[M_{n}^{2}\mid\mathcal{G}_{n-1}]-M_{n-1}^{2}.$

Thus $\mathbb{E}[M_{n}^{2}\mid\mathcal{G}_{n-1}]\leq B+M_{n-1}^{2}$, and by
induction $\mathbb{E}[M_{n}^{2}]\leq Bn$. By Doob’s inequality, for any
integer $m\geq 0$

$\mathbb{P}\Big{(}\max_{2^{m}\leq k\leq
2^{m+1}}\frac{|M_{k}|}{k}\geq\frac{b}{2}\Big{)}\leq\mathbb{P}\Big{(}\max_{k\leq
2^{m+1}}|M_{k}|\geq
2^{m}\frac{b}{2}\Big{)}\leq\frac{4}{2^{2m}b^{2}}2^{m+1}B=\frac{8B}{2^{m}b^{2}}\,,$

and so, for any integer $n\geq 0$,

$\mathbb{P}\Big{(}\sup_{2^{n}\leq k}\frac{|M_{k}|}{k}\geq
b\Big{)}\leq\sum_{m=n}^{\infty}\mathbb{P}\Big{(}\max_{2^{m}\leq k\leq
2^{m+1}}\frac{|M_{k}|}{k}\geq\frac{b}{2}\Big{)}\leq\frac{64B}{2^{n}b^{2}}.$

Choose $n$ large enough so that $\frac{64B}{2^{n}b^{2}}<\frac{\delta}{2}$
(note that this $n$ only depends on $\delta,B$ and $b$). On the other hand,
for any $C>0$, applying Doob’s inequality again gives

$\mathbb{P}(\max_{0\leq k\leq 2^{n}}|M_{k}|>C)\leq\frac{B2^{n}}{C^{2}}.$

For $C$ large enough, only depending on $\delta,B,$ and $n$, we have
$\frac{B2^{n}}{C^{2}}<\frac{\delta}{2}$, which proves the claim. ∎

######  Corollary 4.12.

Let $(Z_{n})_{n\in\mathbb{N}}$ be a sequence of random variables adapted to
some filtration $\\{\mathcal{G}_{n}\\}_{n\in\mathbb{N}}$ such that

$Z_{0}=0,\qquad\mathbb{E}[(Z_{n}-Z_{n-1})^{2}\mid\mathcal{G}_{n-1}]\leq
B,\qquad\mathbb{E}[Z_{n}-Z_{n-1}\mid\mathcal{G}_{n-1}]\geq b$

for some constants $B,b>0$. Then for every $\delta>0$ there is $C>0$ depending
only on $\delta$, $B$, and $b$ such that

$\mathbb{P}\Big{(}Z_{n}\geq\frac{b}{2}n-C\text{ for all }n\geq
0\Big{)}>1-\delta\,.$

###### Proof.

It is standard to show that

$M_{n}\coloneqq\sum_{i=1}^{n}Z_{i}-Z_{i-1}-\mathbb{E}[(Z_{i}-Z_{i-1})\mid\mathcal{G}_{i-1}]$

is a martingale with respect to $\\{\mathcal{G}_{n}\\}_{n\geq 0}$. Then,

$\mathbb{E}[(M_{n}-M_{n-1})^{2}\mid\mathcal{G}_{n-1}]=\mathbb{E}[(Z_{n}-Z_{n-1}-\mathbb{E}[(Z_{n}-Z_{n-1})\mid\mathcal{G}_{n-1}])^{2}\mid\mathcal{G}_{n-1}]\\\
=\mathbb{E}[(Z_{n}-Z_{n-1})^{2}\mid\mathcal{G}_{n-1}]-\mathbb{E}[(Z_{n}-Z_{n-1})\mid\mathcal{G}_{n-1}]^{2}\leq
B,$

and therefore the assumptions of 4.11 are satisfied. Hence, there is some
$C>0$ such that

$\mathbb{P}\Big{(}|M_{n}|\leq\frac{b}{2}n+C\text{ for all }n\geq
0\Big{)}>1-\delta,$

where $C$ depends only on $\delta$, $B$, and $b$. Since

$Z_{n}=M_{n}+\sum_{i=1}^{n}\mathbb{E}[(Z_{i}-Z_{i-1})\mid\mathcal{G}_{i-1}]\geq
M_{n}+bn,$

then

$\mathbb{P}\Big{(}Z_{n}\geq\frac{b}{2}n-C\text{ for all }n\geq
0\Big{)}\geq\mathbb{P}\Big{(}|M_{n}|\leq\frac{b}{2}n+C\text{ for all }n\geq
0\Big{)}>1-\delta\,,$

as desired. ∎

##  5\. Continuous Dependence on Initial Condition

In this section we extend results known for $C_{0}$-Feller processes to our
weaker “$C_{b}$” definition of Feller (2) combined with 3. Notably, we show
that the strong Markov property holds and the entire path of the process
(viewed as a random element of the Skorkhod space
$\mathbb{D}_{[0,\infty)}(\mathcal{M})$, defined later) depends continuously on
the initial condition. We also recall some facts about the Skorokhod topology
and show that certain functions between Skorokhod spaces are continuous. On a
first read most of this section (especially the proofs) can be skipped, but
the statements of 5.2 and 5.6 should be understood well before reading Section
6. We begin with a technical lemma that is not used outside of this section:

######  Lemma 5.1.

If $\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq 0}$ is Feller (see 2) satisfying 3
then for any $f\in C_{b}(\mathcal{M})$ we have that $\mathcal{P}_{t}f\to f$
uniformly on compact sets as $t\downarrow 0$.

###### Proof.

Let $K_{m}=\\{W\leq m\\}$ and let $A\subset C_{b}(\mathcal{M})$ be the set of
functions $f$ such that $\mathcal{P}_{t}f\to f$ uniformly on each $K_{m}$ as
$t\to 0$.

Let $f\in C_{b}(\mathcal{M})$ and $\lambda>0$ be arbitrary. We claim that $A$
contains the function $R_{\lambda}f$, where
$R_{\lambda}=\int_{0}^{\infty}e^{-\lambda s}\mathcal{P}_{s}ds$ denotes the
resolvent operator. Indeed, by Fubini’s theorem
$\mathcal{P}_{t}R_{\lambda}=\int_{0}^{\infty}e^{-\lambda
s}\mathcal{P}_{s+t}ds$ and since $\mathcal{P}_{t}$ is a contraction, we obtain

$\displaystyle\|\mathcal{P}_{t}R_{\lambda}f-R_{\lambda}f\|$
$\displaystyle=\Big{\|}\int_{0}^{\infty}e^{-\lambda
s}\mathcal{P}_{s+t}fds-\int_{0}^{\infty}e^{-\lambda
s}\mathcal{P}_{s}fds\Big{\|}$ $\displaystyle\leq(e^{\lambda
t}-1)\Big{\|}\int_{0}^{\infty}e^{-\lambda
s}\mathcal{P}_{s}fds\Big{\|}+e^{\lambda t}\Big{\|}\int_{0}^{t}e^{-\lambda
s}\mathcal{P}_{s}fds\Big{\|}$ $\displaystyle\leq 2\|f\|\frac{e^{\lambda
t}-1}{\lambda}\,.$

and the claim follows after passing $t\to 0$.

Since $\lambda R_{\lambda}=\int_{0}^{\infty}e^{-s}\mathcal{P}_{s/\lambda}ds$,
by 2 and Dominated Convergence Theorem, $\lambda R_{\lambda}f\to f$ pointwise
as $\lambda\to\infty$. For any integer $m\geq 1$, Riesz-Markov Representation
Theorem implies that for any linear functional $F$ on $C_{b}(K_{m})$ there is
a finite Borel measure $\mu$ such that $F(f)=\mu f$ for any $f\in
C_{b}(K_{m})$. Since $\|\lambda R_{\lambda}f\|\leq\|f\|$, the Dominated
Convergence Theorem implies $\mu(\lambda R_{\lambda}f)\to\mu f$, and therefore
$\lambda R_{\lambda}f\rightharpoonup f$ in the weak topology on
$C_{b}(K_{m})$. By Mazur’s lemma (see [Bre11, Corollary 3.8]), for every
$m\in\mathbb{N}$ there are convex combinations $f_{n}$ of $\lambda
R_{\lambda}f$ such that $f_{n}\to f$ uniformly on $K_{m}$. By a diagonal
argument, we can assume the convex combinations satisfy $f_{n}\to f$ uniformly
on each $K_{m}$. Since $A$ is closed under finite convex combinations,
$f_{n}\in A$, and since $\|\lambda R_{\lambda}f\|\leq\|f\|$ we also have
$\sup_{n}\|f_{n}\|\leq\|f\|<\infty$.

Next we show that our fixed $f\in C_{b}(\mathcal{M})$ belongs to $A$. Fix
$K_{m}$ and let $\|\cdot\|_{m}$ denote the sup norm over $K_{m}$, that is,
$\|f\|_{m}=\sup_{x\in K_{m}}|f(x)|$. For any $t>0$, $M>0$, and
$x\in\mathcal{M}$ we have

$\mathcal{P}_{t}f_{n}(x)-\mathcal{P}_{t}f(x)=\mathbb{E}[f_{n}(X^{x}_{t})-f(X^{x}_{t})]\\\
=\mathbb{E}[(f_{n}(X^{x}_{t})-f(X^{x}_{t}))\mathbbm{1}_{X^{x}_{t}\in
K_{Mm}}]+\mathbb{E}[(f_{n}(X^{x}_{t})-f(X^{x}_{t}))\mathbbm{1}_{X^{x}_{t}\not\in
K_{Mm}}]\,,$

and therefore

$\|\mathcal{P}_{t}f_{n}-\mathcal{P}_{t}f\|_{m}\leq\|f_{n}-f\|_{Mm}+\|\mathbb{P}(X_{t}^{x}\notin
K_{Mm})\|_{m}(\|f_{n}\|+\|f\|),$

where $\mathbb{P}(X_{t}^{x}\notin K_{Mm})$ is viewed as a function of $x$. By
Markov inequality and 4.1,

$\mathbb{P}(X_{t}^{x}\notin
K_{Mm})=\mathbb{P}(W(X_{t}^{x})>Mm)\leq\frac{\mathcal{P}_{t}W(x)}{Mm}\leq\frac{W(x)+Kt}{Mm},$

and consequently

$\|\mathcal{P}_{t}f_{n}-\mathcal{P}_{t}f\|_{m}\leq\|f_{n}-f\|_{Mm}+\frac{m+Kt}{Mm}(\|f_{n}\|+\|f\|).$

Also,

$\|\mathcal{P}_{t}f-f\|_{m}\leq\|\mathcal{P}_{t}f_{n}-\mathcal{P}_{t}f\|_{m}+\|\mathcal{P}_{t}f_{n}-f_{n}\|_{m}+\|f_{n}-f\|_{m}\,.$

The middle term converges to $0$ as $t\to 0$ because $f_{n}\in A$. The last
term and $\|f_{n}-f\|_{Mm}$ converge to $0$ as $n\to\infty$ since $f_{n}\to f$
uniformly on each $K_{m}$. By letting $t\to 0$ and then $n\to\infty$, we have
for any $M>0$,

$\lim_{t\to
0}\|\mathcal{P}_{t}f-f\|_{m}\leq\frac{\sup_{n}\|f_{n}\|+\|f\|}{M}.$

Since $M$ and $m$ were arbitrary, therefore $\mathcal{P}_{t}f\to f$ uniformly
on each $K_{m}$ as $t\to 0$. Consequently, $f\in A$. Since $f\in
C_{b}(\mathcal{M})$ was arbitrary, by our definition of $A$ the claim is
proven. ∎

One important consequence of 2 is the strong Markov property. In fact, the
proof is identical to the one for $C_{0}$ Feller semigroups. In order to state
the strong Markov property, we introduce the Skorokhod space
$\mathbb{D}_{[0,\infty)}(\mathcal{M})$, which consists of all cadlag functions
$f:[0,\infty)\to\mathcal{M}$ endowed with the Skorokhod ($J_{1}$-)topology.
Specifically, $f_{n}\to f$ if and only if there exists increasing bijections
$\lambda_{n}:[0,\infty)\to[0,\infty)$ such that $f_{n}\circ\lambda_{n}\to f$
and $\lambda_{n}\to\textnormal{Id}$ uniformly on compact subsets of
$[0,\infty)$, where Id denotes the identity map on $[0,\infty)$. It is known
that $\mathbb{D}_{[0,\infty)}(\mathcal{M})$ is a Polish space and that the
Borel $\sigma$\- algebra is generated by the projections $\pi_{t}:f\mapsto
f(t)$ (see [Kal21, Lemma A5.3]). Thus, there is no difference between
processes $\\{X_{t}\\}_{t\in[0,\infty)}$ with cadlag sample paths and random
elements $X$ of $\mathbb{D}_{[0,\infty)}(\mathcal{M})$. In what follows,
$X^{x}$ denotes the random variable given by $\Omega\ni\omega\mapsto
X_{\cdot}^{x}(\omega)\in\mathbb{D}_{[0,\infty)}(\mathcal{M})$.

######  Lemma 5.2.

If $\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq 0}$ is Feller (see 2), then it
satisfies the strong Markov property, meaning that if
$\phi:\mathbb{D}_{[0,\infty)}(\mathcal{M})\to[0,\infty)$ is a measurable
function and $\tau$ is a stopping time, then
$\tilde{\phi}:\mathcal{M}\to[0,\infty)$ given by

$\tilde{\phi}(y)\coloneqq\mathbb{E}[\phi(X^{y})]$

is measurable and for any $x\in\mathcal{M}$ we have

$\mathbb{E}[\mathbbm{1}_{\tau<\infty}\phi(X_{\tau+\cdot}^{x})\mid\mathcal{F}_{\tau}]=\mathbbm{1}_{\tau<\infty}\tilde{\phi}(X_{\tau}^{x}).$

###### Proof.

The proof is the same as [LG16, Theorems 6.16 and 6.17]. We remark that our
notation differs from [LG16]. For example, we use $\mathcal{M}$ instead of
$E$, $X_{t}^{x}$ instead of $Y_{t}$, $\mathcal{P}_{t}$ instead of $Q_{t}$, and
$C_{b}(\mathcal{M})$ instead of $C_{0}(E)$. Note that $C_{b}(\mathcal{M})$
differs from $C_{0}(E)$, but the proof in [LG16] is unchanged if $C_{0}(E)$ is
replaced by $C_{b}(\mathcal{M})$ as it only uses continuity of functions in
$C_{0}(E)$, but not the vanishing at infinity. The book [LG16] also uses a
stronger definition of Feller, but in the proofs of Theorems 6.16 and 6.17, it
only uses that $Q_{t}:C_{0}(E)\to C_{0}(E)$, which in our notation translates
to $\mathcal{P}_{t}:C_{b}(\mathcal{M})\to C_{b}(\mathcal{M})$ (see 2). ∎

######  Remark 5.3.

For example, if $\tau$ is a finite stopping time, $s>0$, and $f\in
C_{b}(\mathcal{M})$, then applying the strong Markov property detailed in 5.2
with $\phi(x(\cdot)):=x(s)$ we have

(5.1)
$\mathcal{P}_{s}f(X_{\tau}^{x})=\mathbb{E}[f(X_{\tau+s}^{x})|\mathcal{F}_{\tau}]\,.$

The Feller property (2) tells us that if $x_{n}\to x$, then $X_{t}^{x_{n}}\to
X_{t}^{x}$ (in distribution). In other words, the law of the process at time
$t$ depends continuously on the initial position. Using 3, we can actually
show that the entire path of the process (viewed as a random element of
$\mathbb{D}_{[0,\infty)}(\mathcal{M})$) depends continuously on the initial
condition.

First, we recall an Arzela-Ascoli-type result for weak convergence on
$\mathbb{D}_{[0,\infty)}(\mathcal{M})$:

######  Lemma 5.4.

If $Y^{n},Y$ are $\mathbb{D}_{[0,\infty)}(\mathcal{M})$-valued random
variables, then $Y^{n}\to Y$ in distribution if both statements hold:

  1. (i)

For any $0\leq t_{1}<\ldots<t_{m}$,
$(Y^{n}(t_{1}),\ldots,Y^{n}(t_{m}))\to(Y(t_{1}),\ldots,Y(t_{m}))$ in
distribution.

  2. (ii)

For any $t>0$, any sequence $(\tau_{n})_{n\geq 1}$ of stopping times with
$\tau_{n}\leq t$, and any sequence $(h_{n})_{n\geq 1}$ of positive constants
such that $h_{n}\to 0$, we have
$\mathbb{E}[d(Y_{\tau_{n}}^{n},Y_{\tau_{n}+h_{n}}^{n})]\to 0$.

######  Remark 5.5.

By convergence in distribution we mean that the laws of the random variables
converge weakly, and in (i) we view each $(Y^{n}(t_{1}),\ldots,Y^{n}(t_{m}))$
as a $\mathcal{M}^{m}$-valued random variable. In (ii), we view each $Y^{n}$
as a collection of $\mathcal{M}$-valued random variables $Y^{n}_{t}\coloneqq
Y^{n}(t)$ and $\tau_{n}$ is a stopping time with respect to the filtration
generated by $\\{Y^{n}_{t}\\}_{t\geq 0}$.

###### Proof.

Can be found in [Kal21, Theorem 23.9i and Theorem 23.11]. ∎

The proof of the following lemma is inspired by [Kal21, Theorem 17.25].

######  Lemma 5.6.

Suppose $\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq 0}$ is Feller (see 2) and
satisfies 3. If $x_{n}\to x$ then $X^{x_{n}}\to X^{x}$ in distribution.
(Recall we use $X^{x}$ to denote the random variable given by
$\Omega\ni\omega\mapsto
X_{\cdot}^{x}(\omega)\in\mathbb{D}_{[0,\infty)}(\mathcal{M})$.)

###### Proof.

It suffices to show conditions (i) and (ii) of 5.4.

Proof of (i). The proof is by induction on $m$. If $m=1$, then for any $f\in
C_{b}(\mathcal{M})$, 2 implies $\mathcal{P}_{t_{1}}f\in C_{b}(\mathcal{M})$,
and therefore

$\mathbb{E}[f(X^{x_{n}}(t_{1}))]=\mathcal{P}_{t_{1}}f(x_{n})\to\mathcal{P}_{t_{1}}f(x)=\mathbb{E}[f(X^{x}(t_{1}))].$

Suppose the claim holds for $m-1$ and fix $0\leq t_{1}<\ldots<t_{m}$ and let
$f\in C_{b}(\mathcal{M}^{m})$ be Lipschitz. Recall that by Portmanteau theorem
it suffices to consider only Lipschitz functions $f$ to show weak convergence
of measures, or equivalently the convergence in distribution of the
corresponding random variables. Let
$\phi:\mathbb{D}_{[0,\infty)}(\mathcal{M})\to\mathbb{R}$ be given by

$\phi(x(\cdot))=f(x(0),x(t_{2}-t_{1}),\dots,x(t_{m}-t_{1}))$

and $\tilde{\phi}:\mathcal{M}\to\mathbb{R}$ be as defined in 5.2:

(5.2)
$\tilde{\phi}(y)\coloneqq\mathbb{E}[\phi(X^{y})]=\mathbb{E}[f(y,X^{y}(t_{2}-t_{1}),\dots,X^{y}(t_{m}-t_{1}))]\,.$

Applying the (strong) Markov property (5.2) with $\tau=t_{1}$, for any
$x\in\mathcal{M}$ we have

(5.3)
$\mathbb{E}[f(X^{x}(t_{1}),X^{x}(t_{2}),\dots,X^{x}(t_{m}))]=\mathbb{E}[\tilde{\phi}(X^{x}(t_{1}))]=\mathcal{P}_{t_{1}}\tilde{\phi}(x)\,.$

Next, we show that the left hand side of (5.3) is continuous in $x$, which by
2 follows once we prove that $\tilde{\phi}\in C_{b}(\mathcal{M})$. By our
inductive hypothesis,

$\mathcal{M}\ni y\mapsto(X^{y}(t_{2}-t_{1}),\dots,X^{y}(t_{m}-t_{1}))\in
P(\mathcal{M}^{m-1})$

is continuous (again we conflate random variables with their laws).

Since $f\in C_{b}(\mathcal{M}^{m})$ is Lipschitz,

$\mathcal{M}\ni y\mapsto f_{y}\in
C_{b}(\mathcal{M}^{m-1})\quad\text{where}\quad
f_{y}(x_{1},\dots,x_{m-1})\coloneqq f(y,x_{1},\dots,x_{m-1})$

is continuous. Since

$P(\mathcal{M}^{m-1})\times C_{b}(\mathcal{M}^{m-1})\ni(\mu,f)\mapsto\mu
f\in\mathbb{R}$

is continuous, by (5.2) $\tilde{\phi}$ is continuous, as desired.

Proof of (ii) Let $t>0$, $(\tau_{n})_{n\geq 1}$, and $(h_{n})_{n\geq 1}$ be as
in 5.4 (ii). Define $\phi_{n}:\mathbb{D}_{[0,\infty)}(\mathcal{M})\to[0,1]$ as
$\phi_{n}(Y)=d(Y(0),Y(h_{n}))$ and
$\tilde{\phi}_{n}(y)=\mathbb{E}[\phi_{n}(X^{y})]$. By the strong Markov
property (5.2),

(5.4)
$\mathbb{E}[d(X_{\tau_{n}}^{x_{n}},X_{\tau_{n}+h_{n}}^{x_{n}})]=\mathbb{E}[\tilde{\phi}_{n}(X_{\tau_{n}}^{x_{n}})]=\nu_{n}\tilde{\phi}_{n},$

where $\nu_{n}$ denotes the law of $X_{\tau_{n}}^{x_{n}}$. Since $\tau_{n}\leq
t$, 4.1 implies

$\nu_{n}W=\mathbb{E}[W(X_{\tau_{n}}^{x_{n}})]\leq W(x_{n})+Kt.$

Since $x_{n}\to x$, $\sup_{n}W(x_{n})<\infty$, and therefore
$\sup_{n}\nu_{n}W<\infty$. This implies

(5.5) $\lim_{m\to\infty}\sup_{n}\nu_{n}(K_{m})=0\,.$

Note that by the Markov property,

(5.6)
$\tilde{\phi}_{n}(x)=\mathbb{E}[d(x,X_{h_{n}}^{x})]=\mathbb{E}[d_{x}(X_{h_{n}}^{x})]=\mathcal{P}_{h_{n}}d_{x}(x)\,,$

where $d_{x}:\mathcal{M}\to[0,1]$ given by $d_{x}(y)=d(x,y)$ is continuous.

Let $K_{m}=\\{W\leq m\\}$ and $\|\cdot\|_{m}$ be the supremum norm over
$K_{m}$ as in the proof of 5.1. Then for any $m$ and any $x,y\in K_{m}$, we
obtain

$\displaystyle|\tilde{\phi}_{n}(x)-\tilde{\phi}_{n}(y)|$
$\displaystyle=|\mathcal{P}_{h_{n}}d_{x}(x)-\mathcal{P}_{h_{n}}d_{y}(y)|$
$\displaystyle\leq\|\mathcal{P}_{h_{n}}d_{x}-\mathcal{P}_{h_{n}}d_{y}\|+|\mathcal{P}_{h_{n}}d_{x}(x)-\mathcal{P}_{h_{n}}d_{x}(y)|$
$\displaystyle\leq
d(x,y)+|\mathcal{P}_{h_{n}}d_{x}(x)-\mathcal{P}_{h_{n}}d_{x}(y)|$

and by the Feller property (see 2) we have that $\mathcal{P}_{h_{n}}d_{x}$,
and thus $\tilde{\phi}_{n}$, is continuous. In addition, by using $d_{x}(x)=0$
it holds that

$\displaystyle|\mathcal{P}_{h_{n}}d_{x}(x)-\mathcal{P}_{h_{n}}d_{x}(y)|$
$\displaystyle=|\mathcal{P}_{h_{n}}d_{x}(x)-d_{x}(x)-d_{x}(y)+d_{x}(y)-\mathcal{P}_{h_{n}}d_{x}(y)|$
$\displaystyle\leq d(x,y)+2\|\mathcal{P}_{h_{n}}d_{x}-d_{x}\|_{m}\,,$

and consequently

(5.7) $\displaystyle|\tilde{\phi}_{n}(x)-\tilde{\phi}_{n}(y)|\leq
2d(x,y)+2\|\mathcal{P}_{h_{n}}d_{x}-d_{x}\|_{m}\,.$

5.1 implies $\|\mathcal{P}_{h_{n}}d_{x}-d_{x}\|_{m}\to 0$ as $n\to\infty$, and
thus by (5.7) and continuity of each $\tilde{\phi}_{n}$ we have that
$\\{\tilde{\phi}_{n}\\}_{n\in\mathbb{N}}$ is equicontinuous on $K_{m}$. By 2
and $h_{n}\to 0$, for each $x\in\mathcal{M}$, $\mathcal{P}_{h_{n}}d_{x}\to
d_{x}$ pointwise, which by (5.6) means that $\tilde{\phi}_{n}\to 0$ pointwise
since $d_{x}(x)=0$. Then Arzela-Ascoli theorem yields that
$\tilde{\phi}_{n}\to 0$ uniformly on each $K_{m}$. Since
$\sup_{n}\|\tilde{\phi}_{n}\|\leq 1$, then by (5.4) and (5.5),
$\mathbb{E}[d(X_{\tau_{n}}^{x_{n}},X_{\tau_{n}+h_{n}}^{x_{n}})]=\nu_{n}\tilde{\phi}_{n}\to
0$, which proves (ii). ∎

The proof of the next lemma is standard for the more familiar topology of
uniform convergence on compact sets, but since we work with the Skorokhod
topology we provide more details.

######  Lemma 5.7.

The following functions are continuous:

  1. (i)

$x(\cdot)\mapsto g(x(\cdot))$ from $\mathbb{D}_{[0,\infty)}(\mathcal{M})$ to
$\mathbb{D}_{[0,\infty)}(\mathbb{R})$, where $g:\mathcal{M}\to\mathbb{R}$ is
continuous.

  2. (ii)

$x(\cdot)\mapsto\int_{0}^{\cdot}x(s)ds$ from
$\mathbb{D}_{[0,\infty)}(\mathbb{R})$ to
$\mathbb{D}_{[0,\infty)}(\mathbb{R})$.

  3. (iii)

$x(\cdot)\mapsto\inf_{0\leq s\leq\cdot}x(s)$ from
$\mathbb{D}_{[0,\infty)}(\mathbb{R})$ to
$\mathbb{D}_{[0,\infty)}(\mathbb{R})$.

  4. (iv)

$x(\cdot)\mapsto x(T)$ from $\\{x\in\mathbb{D}_{[0,\infty)}(\mathbb{R})\mid
x\text{ continuous}\\}\eqqcolon C_{[0,\infty)}(\mathbb{R})\to\mathbb{R}$,
where $T$ is a constant.

###### Proof.

The claim (iv) holds true, since the subspace topology of
$C_{[0,\infty)}(\mathbb{R})\subset\mathbb{D}_{[0,\infty)}(\mathbb{R})$
coincides with the topology of uniform convergence on compact sets (see for
example [Kal21, Theorem 23.9iii]).

Next, we prove (i) – (iii). In all three cases the corresponding map denoted
$F:\mathbb{D}_{[0,\infty)}(A)\to\mathbb{D}_{[0,\infty)}(\mathbb{R})$ (where
$A=\mathcal{M}$ or $\mathbb{R}$) is continuous if we endow
$\mathbb{D}_{[0,\infty)}(A)$ and $\mathbb{D}_{[0,\infty)}(\mathbb{R})$ with
the topology of uniform convergence on compact sets. Thus, by the definition
of the Skorokhod topology it suffices to show that if
$f_{n}\in\mathbb{D}_{[0,\infty)}(A)$ and there exists an increasing bijection
$\lambda_{n}:[0,\infty)\to[0,\infty)$ such that $f_{n}\circ\lambda_{n}\to f$
and $\lambda_{n}\to\textnormal{Id}$ uniformly on compact subsets of
$[0,\infty)$, then

(5.8) $F(f_{n})\circ\lambda_{n}-F(f_{n}\circ\lambda_{n})\to 0$

uniformly on compact subsets of $[0,\infty)$ (we already noted that
$F(f_{n}\circ\lambda_{n})\to F(f)$).

For (i) and (iii), (5.8) is immediate since respectively $(g\circ
f_{n})\circ\lambda_{n}=g\circ(f_{n}\circ\lambda_{n})$, and $\inf_{0\leq
s\leq\lambda_{n}(t)}f_{n}(s)=\inf_{0\leq s\leq t}f_{n}(\lambda_{n}(s))$ for
each $t$.

For (ii) we need to show that for all $T>0$

(5.9)
$\int_{0}^{\lambda_{n}(t)}f_{n}(s)ds-\int_{0}^{t}f_{n}(\lambda_{n}(s))ds\to 0$

uniformly for $t\in[0,T]$. Since $(f_{n})_{n\geq 1}$ converges in
$\mathbb{D}_{[0,\infty)}(\mathbb{R})$, $(f_{n})_{n\geq 1}$ is relatively
compact in $\mathbb{D}_{[0,\infty)}(\mathbb{R})$ and thus by [Kal21, Theorem
A5.4] we have

(5.10) $\lim_{h\downarrow 0}\sup_{n}\tilde{w}_{T+1}(f_{n},h)=0\,,$

where for any $g:[0,T+1)\to\mathcal{M}$ and $h>0$

$\tilde{w}_{T+1}(g,h)=\inf_{(I_{k})}\max_{k}\sup_{r,s\in I_{k}}|g(r)-g(s)|$

and the infimum is taken over all partitions of the interval $[0,T+1)$ into
sub-intervals $I_{k}=[u_{k},v_{k})$ with $v_{k}-u_{k}\geq h$ when $v_{k}<T+1$.

To prove (5.9) we fix any $T>0$ and denote

$C:=1+\sup_{t\in[0,T+1]}|f(t)|$

For any fixed $\epsilon>0$, by (5.10) there is a finite partition of $[0,T+1)$
into $M=M(\epsilon)$ intervals $I_{k}=[u_{k},v_{k})$ such that $\sup_{s,t\in
I_{k}}|f_{n}(s)-f_{n}(t)|<\epsilon$ for any $k\leq M$, $n\geq 1$. Fix $N\geq
1$ such that for any $n\geq N$ we have

$\sup_{t\in[0,T]}|\lambda_{n}(t)-t|<\frac{\epsilon}{CM}\wedge 1\quad\text{ and
}\quad C\geq\sup_{t\in[0,T]}|f_{n}(\lambda_{n}(t))|\,.$

Then for any $n\geq N$ and $t\in[0,T]$

$\displaystyle
J_{1}(t):=\Big{|}\int_{0}^{\lambda_{n}(t)}f_{n}(s)ds-\int_{0}^{t}f_{n}(s)ds\Big{|}\leq
C\sup_{t\in[0,T]}|\lambda_{n}(t)-t|\leq\frac{\epsilon}{M}$

and

$\displaystyle Q_{k}$
$\displaystyle:=\int_{u_{k}}^{v_{k}}|f_{n}(s)-f_{n}(\lambda_{n}(s))|ds\leq\int_{u_{k}+\frac{\epsilon}{CM}}^{v_{k}-\frac{\epsilon}{CM}}|f_{n}(s)-f_{n}(\lambda_{n}(s))|ds$
$\displaystyle+\int_{u_{k}}^{u_{k}+\frac{\epsilon}{CM}}|f_{n}(s)-f_{n}(\lambda_{n}(s))|ds+\int_{v_{k}-\frac{\epsilon}{CM}}^{v_{k}}|f_{n}(s)-f_{n}(\lambda_{n}(s))|ds$
$\displaystyle\leq\epsilon(v_{k}-u_{k})+2C\frac{2\epsilon}{CM}=\epsilon(v_{k}-u_{k})+\frac{4\epsilon}{M}\,,$

where in the last inequality we used that $s,\lambda_{n}(s)\in[u_{k},v_{k})$
if $s\in[u_{k}+\frac{\epsilon}{CM},v_{k}-\frac{\epsilon}{CM})$. The desired
result (5.9) follows once we observe that

$\Big{|}\int_{0}^{\lambda_{n}(t)}f_{n}(s)ds-\int_{0}^{t}f_{n}(\lambda_{n}(s))ds\Big{|}\leq
J_{1}+\sum_{k=1}^{M}Q_{k}\leq\epsilon(T+1)+4\epsilon+\frac{\epsilon}{M}\,.$

∎

##  6\. Proof of Main Theorem

In this entire section, we fix a Markov quadruple
$(\mathcal{M},\mathcal{M}_{0},\mathcal{M}_{+},\\{X_{t}^{x}\\}_{x\in\mathcal{M},t\geq
0})$ (see 2.1) satisfying 1 – 5. We aim to provide a proof of 3.2 which is
split into two parts. In the first step we show that for any compact set
$\mathcal{K}$, if $V(x)$ is big enough for some $x\in\mathcal{M}_{+}$, then,
with high probability, $X_{t}^{x}$ is close to $\mathcal{M}_{0}$ whenever $t$
is such that $X_{t}^{x}\in\mathcal{K}$. Furthermore, we establish that there
is a sequence of times $t_{n}\to\infty$ such that $V(X_{t_{n}}^{x})\to\infty$
as $n\to\infty$ and $X_{t_{n}}^{x}\in\mathcal{K}$ for all $n$. Specifically,
we prove the following theorem.

######  Theorem 6.1.

There is an $N>0$ such that for all $\delta>0,M>N$ there is $D>0$ such that
for all $y\in\mathcal{M}_{+}\cap\\{V>D\\}\cap\\{W\leq M\\}$, we have

$\mathbb{P}\Big{(}\sup_{t\geq
0}d(X_{t}^{y},\mathcal{M}_{0})\mathbbm{1}_{W(X_{t}^{y})<M}\leq\frac{1}{M}\text{
and }\limsup_{t\to\infty,W(X_{t}^{y})\leq M}V(X_{t}^{y})=\infty\Big{)}\geq
1-\delta\,,$

where $\limsup_{t\to\infty,W(X_{t}^{y})\leq M}V(X_{t}^{y})=\infty$ means that
there is some sequence of times $t_{n}\to\infty$ such that
$W(X_{t_{n}}^{y})\leq M$ and $V(X_{t_{n}}^{y})\to\infty$.

In the second step we show that 6.1 implies 3.2, which amounts to showing that
if $V(x)$ is big enough, then almost surely all limit points of the sequence
of measures $\mu_{t}^{x}$ (see 4.4) are supported on $\mathcal{M}_{0}$.

To prove 6.1 we sample $X_{t}^{x}$ at special stopping times that are
introduced and analyzed in Section 6.1, Section 6.2, and Section 6.3. Then, we
obtain a discrete-time Markov chain which, after composition with $V$, induces
a discrete-time semimartingale satisfying the assumptions of 4.12. We make
this argument precise in Section 6.4 to prove 6.1 and then finally prove 3.2
in Section 6.5.

###  6.1. Good Stopping Time

For this section, recall 4. We start by showing that as long as $y$ is close
enough to $\mathcal{M}_{0}$, there is some bounded stopping time $\tau_{y,T}$
(see 6.4 below) for which $V(X_{\tau_{y,T}}^{y})-V(y)$ is (on average) large.
The idea is to estimate $V(X_{t}^{y})-V(y)$ by bounding
$\int_{0}^{t}H(X^{y}_{s})ds$. Since we have information about the behavior of
$\int_{0}^{t}H(X^{y}_{s})ds$ when $y\in\mathcal{M}_{0}$, we can use the
continuity with respect to initial conditions shown in Section 5 to infer
similar behavior when $y$ is close to $\mathcal{M}_{0}$.

######  Lemma 6.2.

For every $x\in\mathcal{M}_{0}$, $\delta>0$, and $S>0$ there is $T\geq S$ and
$\epsilon>0$ such that for all $y\in\mathcal{M}$ with $d(y,x)<\epsilon$,

$P_{y,T}\coloneqq\mathbb{P}\Big{(}\int_{0}^{T}H(X_{s}^{y})ds>\frac{\alpha}{2}T\text{
and }\inf_{0\leq t^{\prime}\leq
T}\int_{0}^{t^{\prime}}H(X_{s}^{y})ds>-\frac{\alpha}{2}T\Big{)}>1-\delta\,.$

###### Proof.

Fix $x\in\mathcal{M}_{0}$. Then 4.9(ii) is satisfied, and therefore 4.9(i)
holds true, that is, $\liminf_{t\to\infty}\mu_{t}^{x}(H)\geq\alpha$ a.s. for
$\mu_{t}^{x}$ as in 4.4. Hence, almost surely there is a finite (random)
$\mathcal{T}$ such that
$\mu_{t}^{x}H=\frac{1}{t}\int_{0}^{t}H(X_{s}^{x})ds>\frac{\alpha}{2}$ for all
$t\geq\mathcal{T}$. By making $\mathcal{T}$ larger if necessary, we may
suppose $\inf_{0\leq t^{\prime}\leq
t}\int_{0}^{t^{\prime}}H(X_{s}^{y})ds>-\frac{\alpha}{2}t$ for all
$t\geq\mathcal{T}$. It follows that $\liminf_{T\to\infty}P_{x,T}=1$, so there
is a (deterministic) $T>S$ such that $P_{x,T}\geq 1-\frac{\delta}{2}$.

Define the map $F_{T}:\mathbb{D}_{[0,\infty)}(\mathcal{M})\to\mathbb{R}$ which
is the composition of the following maps

$\mathbb{D}_{[0,\infty)}(\mathcal{M})\ni x(\cdot)\mapsto
H(x(\cdot))\mapsto\int_{0}^{\cdot}H(x(s))ds\mapsto\inf_{0\leq t\leq
T}\int_{0}^{t}H(x(s))ds\,.$

Then, $F$ is continuous by 5.7, and similarly
$G_{T}:\mathbb{D}_{[0,\infty)}(\mathcal{M})\to\mathbb{R}$ given by

$G_{T}(x(\cdot)):=\int_{0}^{T}H(x(s))ds$

is continuous. Thus,
$Q:=\big{\\{}Y\in\mathbb{D}_{[0,\infty)}(\mathcal{M}):G_{T}(Y)>\frac{\alpha}{2}T\text{
and }F_{T}(Y)>-\frac{\alpha}{2}T\big{\\}}$ is open and

$P_{y,T}=\mathbb{P}(X^{y}_{\cdot}\in Q)\,.$

Then 5.6 and the Portmanteau theorem give us

$\liminf_{y\to x}P_{y,T}\geq P_{x,T}\geq 1-\frac{\delta}{2}\,.$

Thus, by choosing $\epsilon>0$ small enough, $d(y,x)<\epsilon$ implies
$P_{y,T}>1-\delta$. ∎

######  Lemma 6.3.

For every $\delta>0$ and compact $\mathcal{K}\subset\mathcal{M}$ there is
$S>0$ such that for all $x\in\mathcal{K}\cap\mathcal{M}_{+}$ and $T\geq S$,

$\mathbb{P}\Big{(}\sup_{0\leq t\leq
T}\Big{|}V(X_{t}^{x})-V(x)-\int_{0}^{t}H(X_{s}^{x})ds\Big{|}>\frac{\alpha}{2}T\Big{)}\leq\delta\,.$

###### Proof.

Let $M_{t}^{V}(x)$ be as in (2.1). By 4.2 $V$ has linearly bounded quadratic
variation (see 2.11), so there is a continuous function
$C:\overline{\mathcal{M}_{+}}\to\mathbb{R}$ such that for $T\geq 1$ and
$x\in\overline{\mathcal{M}_{+}}$ it holds that

(6.1) $\int_{0}^{T}\mathcal{P}_{s}\Gamma V(x)ds\leq C(x)T\,.$

By Tonelli’s theorem and 4.2

$\mathbb{E}[(M_{T}^{V}(x))^{2}]=\int_{0}^{T}\mathcal{P}_{s}\Gamma V(x)ds\leq
C(x)T\,.$

Then by Doob’s inequality and (6.1),

$\mathbb{P}\Big{(}\sup_{0\leq t\leq
T}|M_{t}^{V}(x)|>\frac{\alpha}{2}T\Big{)}\leq\frac{4}{T^{2}\alpha^{2}}\mathbb{E}[(M_{T}^{V}(x))^{2}]\leq\frac{4}{T\alpha^{2}}C(x).$

Since $C$ is continuous and thus bounded on the compact set
$\mathcal{K}\cap\overline{\mathcal{M}_{+}}$,

$\lim_{T\to\infty}\sup_{x\in\mathcal{K}\cap\mathcal{M}_{+}}\mathbb{P}\Big{(}\sup_{0\leq
t\leq T}|M_{t}^{V}(x)|>\frac{\alpha}{2}T\Big{)}=0,$

which proves the claim. ∎

######  Definition 6.4.

For any $x\in\mathcal{M}_{+}$ and $T>0$ define the bounded stopping time

$\tau_{x,T}\coloneqq\inf\\{t\geq 0:|V(X_{t}^{x})-V(x)|>\alpha
T\\}\wedge\inf\Big{\\{}t\geq
0:\Big{|}\int_{0}^{t}H(X_{s}^{x})ds\Big{|}>\frac{\alpha}{2}T\Big{\\}}\wedge
T.$

######  Lemma 6.5.

For every $x\in\mathcal{M}_{0}$ and $S>0$ there is $\epsilon>0$ and $T\geq S$
such that for all $y\in\mathcal{M}_{+}$ with $d(y,x)<\epsilon$,
$\mathbb{E}[V(X_{\tau_{y,T}}^{y})-V(y)]\geq\frac{\alpha}{4}T$.

###### Proof.

For any $y\in\mathcal{M}_{+}$ and $T>0$ we define the events

$Q\coloneqq\Big{\\{}\int_{0}^{T}H(X_{s}^{y})ds>\frac{\alpha}{2}T\Big{\\}}\cap\Big{\\{}\inf_{0\leq
t^{\prime}\leq
T}\int_{0}^{t^{\prime}}H(X_{s}^{y})ds>-\frac{\alpha}{2}T\Big{\\}}\\\
\cap\Big{\\{}\sup_{0\leq t\leq
T}\Big{|}V(X_{t}^{y})-V(y)-\int_{0}^{t}H(X_{s}^{y})ds\Big{|}\leq\frac{\alpha}{2}T\Big{\\}}=:Q_{1}\cap
Q_{2}\cap Q_{3}\,,$

and

$\Big{\\{}\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\geq\frac{\alpha}{2}T\Big{\\}}=:R\,.$

Fix some compact neighborhood $\mathcal{K}$ of $x$ (recall that $\mathcal{M}$
is locally compact). By 6.3, there is some $S>0$ such that for all
$y\in\mathcal{K}\cap\mathcal{M}_{+}$ and $T\geq S$,

(6.2) $\mathbb{P}(Q_{3}^{c})=\mathbb{P}\Big{(}\sup_{0\leq t\leq
T}\Big{|}V(X_{t}^{y})-V(y)-\int_{0}^{t}H(X_{s}^{y})ds\Big{|}>\frac{\alpha}{2}T\Big{)}\leq\frac{1}{8}.$

By 6.2, there is a $T\geq S$ and $\epsilon>0$ such that for all
$y\in\mathcal{M}$ with $d(y,x)<\epsilon$, it holds that

(6.3) $\mathbb{P}(Q_{1}\cap Q_{2})=P_{y,T}>\frac{7}{8}\,.$

We may decrease $\epsilon>0$ if necessary such that $d(y,x)<\epsilon$ implies
$y\in\mathcal{K}$. We claim that $Q\subset R$. Indeed, for fixed $\omega\in
Q$, if $\tau_{y,T}(\omega)=T$, then $\omega\in Q_{1}$ implies $\omega\in R$.
If $\tau_{y,T}(\omega)<T$, then either
$\Big{|}\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\Big{|}\geq\frac{\alpha}{2}T$, in
which case $\omega\in Q_{2}$ implies $\omega\in R$, or
$|V(X_{\tau_{y,T}}^{y})-V(y)|\geq\alpha T$, in which case $\omega\in Q_{3}$
yields that

$\displaystyle\alpha T$ $\displaystyle\leq|V(X_{\tau_{y,T}}^{y})-V(y)|$
$\displaystyle\leq\Big{|}V(X_{\tau_{y,T}}^{y})-V(y)-\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\Big{|}+\Big{|}\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\Big{|}$
$\displaystyle\leq\frac{\alpha}{2}T+\Big{|}\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\Big{|}\,,$

and again $\omega\in Q_{2}$ implies $\omega\in R$, as desired.

Hence, by (6.2) and (6.3) we have for all $y\in\mathcal{M}_{+}$ with
$d(y,x)<\epsilon$ that

(6.4) $\mathbb{P}(R)\geq\mathbb{P}(Q)=\mathbb{P}((Q_{1}\cap Q_{2})\setminus
Q_{3}^{c})\geq\mathbb{P}(Q_{1}\cap
Q_{2})-\mathbb{P}(Q_{3}^{c})\geq\frac{7}{8}-\frac{1}{8}=\frac{3}{4}\,.$

By the continuity of $t\mapsto\int_{0}^{t}H(X_{s}^{y})ds$ and the definition
of $\tau_{y,T}$ (6.4) we have
$\Big{|}\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\Big{|}\leq\frac{\alpha}{2}T$, and
thus by (6.4) we conclude

$\displaystyle\mathbb{E}\Big{[}\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\Big{]}\geq\mathbb{P}(R)\frac{\alpha}{2}T+(1-\mathbb{P}(R))\frac{-\alpha}{2}T\geq\frac{\alpha}{4}T\,.$

Since $\tau_{y,T}$ is a bounded stopping time, by 4.3

$\mathbb{E}[V(X_{\tau_{y,T}}^{y})-V(y)]=\mathbb{E}\Big{[}\int_{0}^{\tau_{y,T}}H(X_{s}^{y})ds\Big{]},$

which proves the claim. ∎

###  6.2. Return Time

As mentioned at the beginning of this section, we wish to sample the paths of
$X_{t}^{y}$ at a sequence of stopping times $\tau_{n}$ so that
$V(X_{\tau_{n}}^{y})$ satisfies 4.12. This means that we not only need a
uniform lower bound on $\mathbb{E}[V(X^{y}_{\tau_{1}})-V(y)]$, but also a
uniform upper bound on $\mathbb{E}[(V(X^{y}_{\tau_{1}})-V(y))^{2}]$. Given any
$x\in\mathcal{M}_{0}$ and $S>0$, 6.5 gives us the required uniform lower bound
(namely, $\frac{\alpha}{4}S$) using $\tau_{1}=\tau_{y,T_{x}}$ for some
$T_{x}\geq S$. However, this is only a local result holding for $y$ close
enough to $x$. In particular, if $\mathcal{M}_{0}$ is not compact there is no
reason to believe that $\sup_{x\in\mathcal{M}_{0}}T_{x}<\infty$. Since it is
unclear how to obtain an upper bound on
$\mathbb{E}[(V(X^{y}_{\tau_{y,T}})-V(y))^{2}]$ which is independent of $T$, we
need to modify $\tau_{y,T}$ to ensure that $X_{\tau_{n}}^{y}$ stays in some
fixed compact set. In particular, we let the process run until $X^{y}_{t}$
reenters a fixed compact set at time $\sigma_{x,T,m}$ (see 6.6 below). The
remainder of this section is dedicated to proving an upper bound on
$\mathbb{E}[|V(X_{\sigma_{x,T,m}}^{x})-V(X_{\tau_{x,T}}^{x})|]$ which ensures
that $\mathbb{E}[V(X_{\sigma_{x,T,m}}^{x})-V(x)]$ is big.

######  Definition 6.6.

For $x\in\mathcal{M}_{+}$, $T>0$, and $m>0$ we define the stopping time

$\sigma_{x,T,m}\coloneqq\inf\\{t\geq\tau_{x,T}\mid W(X_{t}^{x})<m\\}\,,$

where $\tau_{x,T}$ is as in 6.4.

######  Lemma 6.7.

For $x\in\mathcal{M}_{+}$, $T>0$, and $m>0$,
$X_{\sigma_{x,T,m}}^{x}\in\mathcal{M}_{+}\cap\\{W\leq m\\}$ a.s.

###### Proof.

By 1 and 2.2, $X_{\sigma_{x,T,m}}^{x}\in\mathcal{M}_{+}$ a.s. By the right-
continuity of the sample paths of $X_{t}^{x}$ and the definition of
$\sigma_{x,T,m}$ (6.6), $X_{\sigma_{x,T,m}}^{x}\in\\{W\leq m\\}$ a.s. ∎

For the rest of the section, recall that $K>0$ is as in 3.

######  Lemma 6.8.

There is $N>0$ such that for all $m\geq N$ and $x\in\mathcal{M}$, the stopping
time $\eta_{m}(x)\coloneqq\inf\\{t\geq 0\mid W(X_{t}^{x})<m\\}$ satisfies

$\tilde{\eta}_{m}(x)\coloneqq\mathbb{E}[\eta_{m}(x)]\leq W(x).$

###### Proof.

Since $W^{\prime}$ is proper, there is a compact set
$\mathcal{K}\subset\mathcal{M}$ such that $W^{\prime}(x)\geq K+1$ for
$x\notin\mathcal{K}$. Let $N$ be the maximum value of $W+1$ on $\mathcal{K}$.
Fix any $m\geq N$ and $x\in\mathcal{M}$ and we simplify the notation by
setting $\eta\coloneqq\eta_{m}(x)$. Then by the definition of $\eta$ we have

$\inf_{0\leq t<\eta}W(X_{t}^{x})\geq m\geq N\,.$

Since $X_{t}^{x}\notin\mathcal{K}$ implies $W^{\prime}(X_{t}^{x})\geq K+1$ for
all $0\leq t<\eta$, then $W\geq 0$ and 4.1 applied to $\eta\wedge t$ yield for
any $t\geq 0$

$(K+1)\mathbb{E}[\eta\wedge t]\leq\mathbb{E}\Big{[}W(X_{\eta\wedge
t}^{x})+\int_{0}^{\eta\wedge t}W^{\prime}(X_{s}^{x})ds\Big{]}\leq
W(x)+K\mathbb{E}[\eta\wedge t].$

Thus, $\mathbb{E}[\eta\wedge t]\leq W(x)$ and the claim follows from the
Monotone convergence theorem after passing $t\to\infty$. ∎

######  Corollary 6.9.

There is $N>0$ such that for $x\in\mathcal{M}_{+}$, $T>0$, and $m\geq N$,

$\mathbb{E}[\sigma_{x,T,m}-\tau_{x,T}]\leq W(x)+KT.$

In particular,

$\mathbb{E}[\sigma_{x,T,m}]\leq W(x)+(K+1)T.$

###### Proof.

Let $N>0$ be as in 6.8, fix any $m\geq N$ and let $\eta_{m}(x)$ be as in 6.8.
By 6.8, $\eta_{m}(x)$ is almost surely finite, and then from the strong Markov
property (5.2) it follows that

$\mathbb{E}[\sigma_{x,T,m}-\tau_{x,T}\mid\mathcal{F}_{\tau_{x,T}}]=\tilde{\eta}_{m}(X_{\tau_{x,T}}^{x})\leq
W(X_{\tau_{x,T}}^{x}).$

Since $\tau_{x,T}\leq T$, by 4.1

$\mathbb{E}[W(X_{\tau_{x,T}}^{x})]\leq W(x)+KT,$

which proves the first claim. The second one follows since $\tau_{x,T}\leq T$.
∎

######  Lemma 6.10.

For every $\beta>0$ there is $N>0$ such that for all $T>0$, $m\geq N$, and
$x\in\mathcal{M}_{+}\cap\\{W\leq m\\}$,

$\mathbb{E}[V(X_{\sigma_{x,T,m}}^{x})-V(X_{\tau_{x,T}}^{x})]\geq-\beta(K+1)(m+KT).$

###### Proof.

Since $H$ vanishes over $W^{\prime}$ (2.19) and $W$ is proper (3), there is
$N>0$ such that $W(y)\geq N-1$ implies $|H(y)|\leq\beta W^{\prime}(y)$.
Increase $N$ if necessary such that the assumptions of 6.8 are satisfied and
fix $m\geq N$.

For any $y\in\mathcal{M}_{+}\cap\\{W\geq m\\}$, let $\eta\coloneqq\eta_{m}(y)$
be as in 6.8 and recall that $\mathbb{E}[\eta]\leq W(y)$. Since
$W(X^{y}_{t})\geq m\geq N$ for $t\in(0,\eta)$, we have
$|H(X^{y}_{t})|\leq\beta W^{\prime}(X^{y}_{t})$ for $t\in(0,\eta)$, and so it
follows from 4.3 that

$\mathbb{E}[V(X_{\eta}^{y})-V(y)]=\mathbb{E}\Big{[}\int_{0}^{\eta}H(X_{s}^{y})ds\Big{]}\geq\beta\mathbb{E}\Big{[}\int_{0}^{\eta}-W^{\prime}(X_{s}^{y})ds\Big{]}.$

By 4.1 and $\mathbb{E}[\eta]\leq W(y)$,

$\mathbb{E}\Big{[}\int_{0}^{\eta}W^{\prime}(X_{s}^{y})ds\Big{]}\leq
W(y)+K\mathbb{E}[\eta]\leq(K+1)W(y)\,,$

and therefore

(6.5) $\mathbb{E}[V(X_{\eta}^{y})-V(y)]\geq-\beta(K+1)W(y).$

Fix $T>0$ and $x\in\mathcal{M}_{+}\cap\\{W\leq m\\}$ and note that $x$ belongs
to a different set than $y$ above.

By the definition of $\sigma_{x,T,m}$, the strong Markov property (5.2), and
(6.5) we obtain

$\displaystyle\mathbb{E}[V(X_{\sigma_{x,T,m}}^{x})-V(X_{\tau_{x,T}}^{x})]$
$\displaystyle=\mathbb{E}[(V(X_{\sigma_{x,T,m}}^{x})-V(X_{\tau_{x,T}}^{x}))\mathbbm{1}_{W(X_{\tau_{x,T}}^{x})\geq
m}]$ $\displaystyle\geq-\beta(K+1)\mathbb{E}[W(X_{\tau_{x,T}}^{x})].$

Noting that $\tau_{x,T}\leq T$ (see 6.4), applying 4.1, and recalling that
$W(x)\leq m$ proves the claim. ∎

######  Corollary 6.11.

There is $N>0$ such that for all $m\geq N$ there is $S>0$ such that for all
$T\geq S$

$\inf_{x\in\mathcal{M}_{+}\cap\\{W\leq
m\\}}\mathbb{E}[V(X_{\sigma_{x,T,m}}^{x})-V(X_{\tau_{x,T}}^{x})]\geq-\frac{\alpha
T}{8}\,,$

where $\alpha>0$ is as in 4.

###### Proof.

Let $\beta=\frac{\alpha}{16K(K+1)}$, let $N>0$ be as in 6.10, and fix $m\geq
N$. For any $T>0$, by 6.10

$\inf_{x\in\mathcal{M}_{+}\cap\\{W\leq
m\\}}\mathbb{E}[V(X_{\sigma_{x,T,m}}^{x})-V(X_{\tau_{x,T}}^{x})]\geq-\beta(K+1)(m+KT).$

Let $S=\frac{m}{K}$ so that $T\geq S$ implies $m+KT\leq 2KT$, which gives

$\beta(K+1)(m+KT)\leq\frac{\alpha T}{8},$

proving the claim. ∎

######  Definition 6.12.

For the rest of the paper, we fix $N>0$ large enough to satisfy 6.8, 6.9, and
6.11. In particular, recalling the proof of 6.8 we may assume that $W(x)\geq
N$ implies $W^{\prime}(x)\geq K+1$ and thus $\mathcal{L}W(x)\leq-1$ by 3.

For every $\epsilon>0$ and any closed set $\mathcal{S}\subset\mathcal{M}$ we
define

(6.6) $N_{\epsilon}(\mathcal{S}):=\\{y\in\mathcal{M}\mid
d(y,\mathcal{S})=\inf_{x\in\mathcal{S}}d(y,x)<\epsilon\\}\,.$

######  Lemma 6.13.

Let $N$ be as in 6.12. For every $m\geq N$ there is $\epsilon>0$, $n\geq 1$,
and $T_{1},\dots,T_{n}>0$ such that for all $y\in\mathcal{M}_{+}\cap\\{W\leq
m\\}\cap N_{\epsilon}(\mathcal{M}_{0})$ there is some $1\leq i\leq n$ such
that $\mathbb{E}[V(X_{\sigma_{y,T_{i},m}}^{y})-V(y)]\geq\frac{\alpha
T_{i}}{8}$.

###### Proof.

Fix any $m\geq N$. By 6.11, there is $S>0$ such that for all $T\geq S$

$\inf_{x\in\mathcal{M}_{+}\cap\\{W\leq
m\\}}\mathbb{E}[V(X_{\sigma_{x,T,m}}^{x})-V(X_{\tau_{x,T}}^{x})]\geq-\frac{\alpha
T}{8}.$

By 6.5, for every $x\in\mathcal{M}_{0}$ there is $\epsilon_{x}>0$ and
$T_{x}\geq S$ such that for all $y\in\mathcal{M}_{+}$ with
$d(y,x)<\epsilon_{x}$, it holds that
$\mathbb{E}[V(X_{\tau_{y,T_{x}}}^{y})-V(y)]\geq\frac{\alpha T_{x}}{4}$. Thus,
if $y\in\mathcal{M}_{+}\cap\\{W\leq m\\}$ and $d(y,x)<\epsilon_{x}$,

$\mathbb{E}[V(X_{\sigma_{y,T_{x},m}}^{y})-V(y)]\geq\frac{\alpha T_{x}}{8}.$

Hence, by the compactness of $\mathcal{M}_{0}\cap\\{W\leq m\\}$, there is
$\epsilon^{\prime}>0$ and $T_{1},\dots,T_{n}>0$ such that for all
$y\in\mathcal{M}_{+}\cap\\{W\leq m\\}\cap
N_{\epsilon^{\prime}}(\mathcal{M}_{0}\cap\\{W\leq m\\})$ there is some $1\leq
i\leq n$ such that
$\mathbb{E}[V(X_{\sigma_{y,T_{i},m}}^{y})-V(y)]\geq\frac{\alpha T_{i}}{8}$. To
finish the claim, we show that there is $\epsilon>0$ such that

$\\{W\leq m\\}\cap N_{\epsilon}(\mathcal{M}_{0})\subset\\{W\leq m\\}\cap
N_{\epsilon^{\prime}}(\mathcal{M}_{0}\cap\\{W\leq m\\})\,.$

Since $\\{W\leq m\\}$ is compact, then $\\{W\leq m\\}\setminus
N_{\epsilon^{\prime}}(\mathcal{M}_{0}\cap\\{W\leq m\\})$ is compact and is
covered by the collection of open sets $(\\{W\leq
m\\}\setminus\overline{N_{\epsilon}(\mathcal{M}_{0})})_{\epsilon>0}$, because

$\bigcup_{\epsilon>0}\\{W\leq
m\\}\backslash\overline{N_{\epsilon}(\mathcal{M}_{0})}=\\{W\leq
m\\}\backslash\mathcal{M}_{0}=\\{W\leq
m\\}\backslash(\mathcal{M}_{0}\cap\\{W\leq m\\})$

which clearly contains $\\{W\leq m\\}\backslash
N_{\epsilon^{\prime}}(\mathcal{M}_{0}\cap\\{W\leq m\\})$. Thus there is
$\epsilon>0$ such that

$\\{W\leq m\\}\backslash N_{\epsilon^{\prime}}(\mathcal{M}_{0}\cap\\{W\leq
m\\})\subset\\{W\leq m\\}\backslash\overline{N_{\epsilon}(\mathcal{M}_{0})},$

which means that

$\\{W\leq m\\}\cap N_{\epsilon^{\prime}}(\mathcal{M}_{0}\cap\\{W\leq
m\\})\supset\\{W\leq
m\\}\cap\overline{N_{\epsilon}(\mathcal{M}_{0})}\supset\\{W\leq m\\}\cap
N_{\epsilon}(\mathcal{M}_{0})$

and the proof is finished. ∎

###  6.3. Bounding the Variance

In this section we upper bound
$\mathbb{E}[(V(X_{\sigma_{x,T,m}}^{x})-V(x))^{2}]$, where $\sigma_{x,T,m}$ was
introduced in 6.6. We start by bounding $\mathbb{E}[\sigma_{x,T,m}^{2}]$.

######  Lemma 6.14.

For every $T>0$ and $m>N$ (see 6.12) we have

$\sup_{x\in\mathcal{M}_{+}\cap\\{W\leq
m\\}}\mathbb{E}[\sigma_{x,T,m}^{2}]<\infty\,,$

where $\sigma_{x,T,m}$ was defined in 6.6.

###### Proof.

Fix $x\in\mathcal{M}_{+}\cap\\{W\leq m\\}$, denote $\sigma=\sigma_{x,T,m}$,
and note that $\sigma$ is a.s. finite by 6.9. By 3 (i) (see 2.10) and 5 (i)
(see 2.9),

$M_{t}\coloneqq(M_{t}^{W}(x))^{2}-\int_{0}^{t}\Gamma
W(X_{s}^{x})ds+KM_{t}^{U}(x)$

is a local martingale since it is a sum of a martingale and a local
martingale. By 5 (iii) and (ii), $\Gamma W\leq KU^{\prime}\leq
K(K-\mathcal{L}U)$, and therefore by the definition of $M_{t}^{U}(x)$ (see
(2.1)) we have for each $t\geq 0$ that

(6.7)
$M_{t}\geq(M_{t}^{W}(x))^{2}-K^{2}t+K(U(X_{t}^{x})-U(x))\geq-K^{2}t-KU(x).$

In particular,

$\inf_{t\geq 0}M_{t\wedge\sigma}\geq-K^{2}\sigma-KU(x)$

and by 6.9,

(6.8) $\mathbb{E}[K^{2}\sigma]\leq K^{2}(W(x)+(K+1)T)\,.$

Thus the martingale $M_{t\wedge\sigma}$ is uniformly bounded from below by an
integrable random variable, so by Fatou’s lemma, optional stopping and (6.7),
we obtain

(6.9)
$0\geq\mathbb{E}[M_{\sigma}]\geq\mathbb{E}[(M_{\sigma}^{W}(x))^{2}-K^{2}\sigma-
KU(x)].$

By applying the inequality $(a+b)^{2}\geq\frac{1}{2}a^{2}-b^{2}$ twice,

$\displaystyle(M_{\sigma}^{W}(x))^{2}$