#  On the existence of Markovian randomized equilibria in Dynkin games of war-
of-attrition-type

Sören Christensen Kiel University, Mathematical Department
email<EMAIL_ADDRESS>Boy Schultz Kiel University, Mathematical
Department
email<EMAIL_ADDRESS>
###### Abstract

In optimal stopping problems, a Markov structure guarantees Markovian optimal
stopping times (first exit times). Surprisingly, there is no analogous result
for Markovian stopping games once randomization is required. This paper
addresses this gap by proving the existence of Markov-perfect equilibria in a
specific type of stopping game – a general nonzero-sum Dynkin games of the
war-of-attrition type with underlying linear diffusions. Our main mathematical
contribution lies in the development of appropriate topologies for Markovian
randomized stopping times. This allows us to establish the existence of
equilibria within a tractable and interpretable class of stopping times,
paving the way for further analysis of Markovian stopping games.

2020 MSC: Primary 91A55; secondary 60G40, 91A15
key words: Dynkin games, diffusion, war of attrition, Markovian randomized
stopping times, Markov-perfect equilibria.

##  1 Introduction

The aim of this paper is to develop a method to prove the existence of
Markovian randomized equilibria for stopping games with underlying linear
diffusions.

###  1.1 Stopping problems and stopping games

To clarify the relevance and the basic idea, we first give a brief overview of
the stopping problem, i.e. the game with one player, since in this case the
theoretical results are already well established, and then compare the
situation with that of stopping games. In general optimal stopping problems,
one asks for the maximisation of
$V=\sup_{\tau}\mathbb{E}\left[G_{\tau}\right]$ over all stopping times $\tau$.
For a concrete solution, however, more structure is needed to make the problem
accessible. This structure is typically given by a driving Markov process $X$,
in our case a linear diffusion, and a value function of the form

$\displaystyle
V(x)=\sup_{\tau}\mathbb{E}_{x}\left[e^{-r\tau}F(X_{\tau})\right],\enskip
X_{0}=x\in E.$

In this setting, it is well known that under certain assumptions the first
exit time $\tau^{\ast}=\inf\\{t\geq 0:X_{t}\notin D\\}$ from the continuation
region $D=\\{x\in E:V(x)>F(x)\\}$ is optimal. It is therefore sufficient to
restrict the optimization from the unmanageably large class of all stopping
times to the class of first exit times, which means in particular that the
stopping decision only depends on the current state of the process. This makes
the problem time-consistent (Bellman principle). Moreover, it opens the door
to the solution of the problem, e.g. via an associated free boundary problem
or in the form of concave envelopes [39, 17, 11].

However, many important real-world situations involve two or more decision
makers with possibly conflicting goals, leading to stopping games. Stopping
games arise in many applications, such as preemption games [29], game options
[34], wars of attrition [21], and market entry/exit decisions [27, 29, 30,
49].

Starting with [22], _Dynkin games_ have become a fundamental mathematical
model for stopping games. In these, two players $i=1,2$ choose stopping times
$\tau_{i}$ and are interested in maximizing functionals of the form

$J_{i}(\tau_{i},\tau_{j})=\mathbb{E}\left[F^{i}_{\tau_{i}}\mathbb{I}_{\\{\tau_{i}<\tau_{j}\\}}+G^{i}_{\tau_{j}}\mathbb{I}_{\\{\tau_{j}<\tau_{i}\\}}+H^{i}_{\tau_{i}}\mathbb{I}_{\\{\tau_{i}=\tau_{j}\\}}\right],\quad
i\in\\{1,2\\},j=3-i.$

The question of the existence and representation of equilibria is then
investigated. There are basically two types of results. In Markovian games, a
number of papers study the existence and characterization of equilibria in the
first exit times, reflecting the Markovian structure of the problem. However,
this can only be expected under rather restrictive ordering conditions on the
payoffs, see e.g. [1, 2, 7, 10, 20, 24, 25, 28].

For other classes of stopping games, i.e. if the mentioned ordering conditions
are not fulfilled, there are no equilibria in first exit times anymore.
Instead, it is now necessary to consider randomized stopping times [43] to
obtain the existence of ($\epsilon$-)equilibria [44, 36, 35, 48]. While these
results at least essentially prove the existence of equilibria, a central
challenge remains: in general, the randomized stopping times have a strongly
path dependent nature and form an almost unmanageably large class. Even with
an underlying Markov structure, it remains unclear whether the equilibrium
stopping decision is Markovian in the sense that it depends only on the
current state of the underlying process – as one would hope from the case of
optimal stopping problems. Therefore, it is necessary to search for a more
specific class of randomized stopping times that allows a clearer
characterization and game-theoretic interpretation of the equilibria.

Although Dynkin games are probably the most fundamental class of stopping
games, many other classes have been considered. On the one hand, these are
variants of Dynkin games, e.g. games with more than two players [31, 37],
games with partial information. [18, 19], but also contests [26, 42] and
equilibrium formulations of time-inconsistent problems [5, 4, 13, 32]. We will
not discuss these problems in detail here, but our goal in this paper is to
develop a technique that has the potential to be applicable to these
situations as well.

###  1.2 Markovian randomized stopping times

From the previous discussion, it seems natural to consider Markovian stopping
times as candidates for equilibria in stopping games. In contrast to stopping
problems, however, these must be randomized times in order to have the
potential for proving general existence theorems in the Markovian case.

Examples of such Markovian randomized stopping times have already been
considered in certain stopping games. First, there are stopping time of the
form

$\displaystyle\inf\\{t\geq 0:X_{t}\notin D\mbox{ or }A_{t}\geq
E\\},\;\;A_{t}:=\int_{0}^{t}\psi(X_{t})dt,$ (1)

where $D$ is a set in the state space of $X$, $E\sim\mbox{Exp}(1)$ independent
of all other random sources and $\psi$ some nonnegative function. For Dynkin
games such equilibria have been considered in [45] and for time-inconsistent
problems in [14, 12]. In the time-inconsistent problem discussed in [47], it
turns out that stopping times of the form (1) are not sufficient to obtain
equilibria and in [8] the considered class was extended to

$\displaystyle
A_{t}:=\sum_{i=1}^{n}\psi_{i}l_{t}^{x_{i}}(X)+\int_{0}^{t}\psi(X_{t})dt,$ (2)

where $\left(l_{t}^{x_{i}}(X)\right)_{t\geq 0}$ is the _local time_ of $X$ at
$x_{i}$, $n$ is a fixed number, $x_{i}\in\\{x_{1},\dots,x_{n}\\}\subset D$,
$\psi_{i}\in(0,\infty)$ are fixed constants. So, in the initial states $x_{i}$
the process can now be stopped with probability in $(0,1)$. Such stopping
times with $\psi=0$ were also considered in [23] in the context of zero-sum
Dynkin games with heterogeneous beliefs.

[21] investigates a variant of the stopping game considered later in this
article, a war-of-attrition game for an underlying linear diffusion. First,
the general class of Markovian randomized stopping times for linear diffusions
is introduced and studied, and it is discussed why the term “Markovian” is
justified. It is also emphasized that the general class of such stopping times
can be described by sets and Radon measures. We will return to this in the
following section. Based on this, Markov-perfect equilibria are characterized
via a variational system and are found to be given by special additive
functionals of the form (2).

However, theorems on the existence of equilibria in Markovian randomized
stopping times are not given in any of the previous articles. The only result
in this direction known to the authors was recently given for the discrete-
time and discrete-space case in [16], see also [15] for related results in
discrete time-inconsistent problems.

###  1.3 Main contributions and structure of the paper

The previous discussion shows that one of the central open questions in the
treatment of Markovian stopping games is the existence of Markovian
equilibria. The aim of this paper is to take a step in this direction and to
give such a result for an important class of Dynkin games for underlying
linear diffusions. More specifically, we consider war-of-attrition-type
stopping games, in which it is well known that randomisation is necessary.
However, we have treated the problem in such a way that our approach can also
serve as a blueprint for other stopping games. In Section 2 we will introduce
the setting and formulate the main result, Theorem 2.4. Unsurprisingly, our
approach to existence is an application of a Kakutani-type fixed point theorem
to a best-response mapping. The key question is what topology to use on the
space of Markovian randomized stopping times $\mathcal{Z}$. We address this
question in Section 3, where we introduce two topologies. In particular, we
will show that $\mathcal{Z}$ has suitable compactness properties, see Theorem
3.14. This result is also of interest on its own. As a first step towards the
general proof of existence, in Section 4 we then consider an auxiliary problem
in which the players are only allowed to stop in a finite subset of the state
space. In this case, the space underlying the fixed point problem is finite-
dimensional, so that the topological questions are easier to handle. We then
use the compactness of $\mathcal{Z}$ to select a convergent subsequence from
the equilibria of the auxiliary problems. Its limit in $\mathcal{Z}$ is then
the candidate for an equilibrium, which we will verify in Section 5. This
concludes the proof of Theorem 2.4. The proof method presented here is by no
means limited to the chosen framework, which we discuss in more detail in
Section 6.

##  2 Setup and main result

In this paper we consider a regular linear Itô diffusion
$X=(X_{t})_{t\in[0,\infty)}$ taking values in an interval $I$ with
$I\subset\mathbb{R}$ and defined on a filtered probability space
$(\Omega,\mathcal{F},(\mathcal{F}_{t})_{t\in[0,\infty)},\mathbb{P})$
satisfying the usual hypotheses. Generally we assume that the behavior of $X$
in the interior $I^{\circ}$ of $I$ is governed by the stochastic differential
equation

$\displaystyle dX_{t}=\mu(X_{t})dt+\sigma(X_{t})dW_{t},\quad X_{0}=x\in
I^{\circ}$

with an $(\mathcal{F}_{t})_{t\in[0,\infty)}$-adapted, real valued, standard
Brownian motion $W=(W_{t})_{t\in[0,\infty)}$ and Lipschitz continuous
coefficients $\mu:I\to\mathbb{R}$, $\sigma:I\to(0,\infty)$. A jointly
continuous version of the local time process of $X$ at $y\in I$ will be
denoted by $(l^{y}_{t})_{t\in[0,\infty)}$. The existence of such a version
follows e.g. from [41, Theorem (44.2)]. We set
$\mathcal{F}^{X}_{\infty}:=\sigma(X_{t}:t\geq 0)$ and denote the canonical
shift operator associated to $X$ by $\theta$. Let
$E_{1},E_{2}\sim\operatorname{Exp}(1)$ be two fixed random variables on
$(\Omega,\mathcal{F},\mathbb{P})$ such that $E_{1},E_{2}$ and $X$ are
independent. If $Y$ is any random variable on
$(\Omega,\mathcal{F},(\mathcal{F}_{t})_{t\in[0,\infty)},\mathbb{P})$ with
values in some metric space, we denote its distribution by $\mathbb{P}^{Y}$.
As usual, we write $\mathbb{P}_{x}$ for the conditional distribution of
$\mathbb{P}$ given $X_{0}=x$ and $\mathbb{E}_{x}$ for the corresponding
expectation operator.

For locally compact, second countable metric spaces $\mathcal{X}$ we use the
following notations for the spaces of measures, probability measures, and
Radon measures on $\mathcal{X}$, respectively:

$\displaystyle\mathcal{M}(\mathcal{X})$ $\displaystyle:=\\{\mu:\mu\geq 0\text{
measure on }\mathcal{X}\\},$ $\displaystyle\mathcal{M}^{1}(\mathcal{X})$
$\displaystyle:=\\{\mu\in\mathcal{M}(\mathcal{X}):\mu(\mathcal{X})=1\\},$
$\displaystyle\operatorname{RM}(\mathcal{X})$
$\displaystyle:=\\{\mu\in\mathcal{M}(\mathcal{X}):\mu\text{ is locally
finite}\\}$

Moreover, we regard $\operatorname{RM}(\varnothing)=\\{0\\}$, where $0$
denoted the zero measure. We denote the first exit time from measurable
$D\subset I$ by

$\displaystyle\tau^{D}$ $\displaystyle:=\inf\\{t\geq 0:X_{t}\not\in D\\}$

and the first entry time into $y\in I$ or measurable $B\subset I$ by

$\displaystyle\eta_{y}:=\inf\\{t\geq 0:X_{t}=y\\},\quad\eta^{B}:=\\{t\geq
0:X_{t}\in B\\},$

respectively. Further, we denote the set of all
$(\mathcal{F}_{t})_{t\in[0,\infty)}$-stopping times by $\mathcal{T}$.

We now come to the introduction of the general class of Markovian randomized
stopping times, extending the notation of (1). For this purpose, we consider
stopping times that stop at the first exit from $D$ and in $D$ with a stopping
rate described by the measure $\lambda$. This is specified as follows. Let
$D\subset I$ open (in $I$) and $\lambda\in\operatorname{RM}(D)$. We set the
additive functional $A^{D,\lambda}=(A^{D,\lambda}_{t})_{t\in[0,\infty)}$
generated by $X,D$ and $\lambda$ to be given by

$\displaystyle
A^{D,\lambda}_{t}(\omega):=\int_{D}l^{y}_{t}(\omega)\lambda(dy)+\infty\mathds{1}_{\\{{\tau^{D}(\omega)\leq
t}\\}},\quad t\geq 0.$

Other general notations used in the following are $\overline{D}$ for the
closure of a set $D\subset\mathbb{R}$, $D^{\circ}$ for its interior, $\partial
D$ for its boundary and $D^{c}$ for its complement $I\setminus D$. Given
$\mathcal{X},\mathcal{Y}$ and $\mathcal{X}^{\prime}\subset\mathcal{X}$ and a
function $f\colon\mathcal{X}\to\mathcal{Y}$ we denote the restriction of $f$
to $\mathcal{X}^{\prime}$ by $f|_{\mathcal{X}^{\prime}}$.

######  Definition 2.1.

(Markovian randomized stopping time)
Let $D\subset I$ open (in $I$), $\lambda\in\operatorname{RM}(D)$, and
$E\sim\operatorname{Exp}(1)$ a random variable on
$(\Omega,\mathcal{F},\mathbb{P})$ that is independent of $X$. We define the
Markovian randomized (stopping) time generated by $D$, $\lambda$ and $E$ as

$\displaystyle\tau^{D,\lambda}:=\inf\\{t\geq 0:A_{t}^{D,\lambda}\geq E\\}.$

The space of all Markovian randomized times based on the random variable $E$
is denoted by

$\displaystyle\mathcal{Z}:=\mathcal{Z}(E):=\\{\tau^{D,\lambda}:D\subset
I\text{ open (in }I),\lambda\in\operatorname{RM}(D)\\}$

and is a subset of the set of all randomized stopping times
$\operatorname{\mathcal{R}}:=\operatorname{\mathcal{R}}(E)$. For
$\tau=\tau^{D,\lambda}\in\mathcal{Z}$, we set $A^{\tau}:=A^{D,\lambda}$. For
open or closed $U\subset I$ we set

$\displaystyle\mathcal{Z}^{U}:=\mathcal{Z}^{U}(E):=\\{\tau^{D,\lambda}\in\mathcal{Z}(E):D\subset
U\\},$
$\displaystyle\mathcal{Z}_{U}:=\mathcal{Z}_{U}(E):=\\{\tau^{D,\lambda}\in\mathcal{Z}(E):I\setminus
D\subset U,\,\operatorname{supp}(\lambda)\subset U\\}.$

In the following we specify the generating random variable $E$ only if
necessary.

One way to make the idea that $\tau\in\mathcal{Z}$ is Markovian precise is
that with a natural extension of the shift operator $\theta$ it holds that

$\displaystyle\mathds{1}_{\\{{\tau\geq\sigma}\\}}\tau=\mathds{1}_{\\{{\tau\geq\sigma}\\}}\theta_{\sigma}\circ\tau+\sigma$

in distribution for all $\mathcal{F}^{X}_{\infty}$-measurable $\sigma$, see
[12]. More detailed discussions can be found in [21, Section 3], see also [9,
I.II.4] for the connection to killing times. In the following, player 1 will
use the random device $E=E_{1}$ introduced above for randomization and player
2 will use $E_{2}$.

We consider a Dynkin game, i.e. a two player game of stopping based on the
reward functionals

$\displaystyle
J^{1}:I\times\operatorname{\mathcal{R}}(E_{1})\times\operatorname{\mathcal{R}}(E_{2})\to\mathbb{R},$
$\displaystyle
J^{1}(x,\tau_{1},\tau_{2}):=\mathbb{E}_{x}[e^{-r_{1}\tau_{1}}g_{1}(X_{\tau_{1}})\mathds{1}_{\\{{\tau_{1}\leq\tau_{2}}\\}}+e^{-r_{1}\tau_{2}}f_{1}(X_{\tau_{2}})\mathds{1}_{\\{{\tau_{1}>\tau_{2}}\\}}],$
$\displaystyle
V^{1}(x,\tau_{2}):=\sup_{\tau_{1}^{\prime}\in\operatorname{\mathcal{R}}(E_{1})}J^{1}(x,\tau_{1}^{\prime},\tau_{2})$
(3) $\displaystyle
J^{2}:I\times\operatorname{\mathcal{R}}(E_{1})\times\operatorname{\mathcal{R}}(E_{2})\to\mathbb{R},$
$\displaystyle
J^{2}(x,\tau_{1},\tau_{2}):=\mathbb{E}_{x}[e^{-r_{2}\tau_{2}}g_{2}(X_{\tau_{2}})\mathds{1}_{\\{{\tau_{2}\leq\tau_{1}}\\}}+e^{-r_{2}\tau_{1}}f_{2}(X_{\tau_{1}})\mathds{1}_{\\{{\tau_{2}>\tau_{1}}\\}}],$
$\displaystyle
V^{2}(x,\tau_{1}):=\sup_{\tau_{2}^{\prime}\in\operatorname{\mathcal{R}}(E_{2})}J^{1}(x,\tau_{1},\tau_{2}^{\prime})$

where $g_{1},g_{2},f_{1},f_{2}:I\to[0,\infty)$ are continuous and
$r_{1},r_{2}\geq 0$. Here, player $i\in\\{1,2\\}$ tries to maximize the
function $J^{i}$ by choosing the third argument, i.e.
$\tau_{i}\in\operatorname{\mathcal{R}}(E_{i})$, for given first and second
arguments. If necessary, we use the convention
$g_{1}(X_{\infty}):=g_{2}(X_{\infty}):=0$.

######  Definition 2.2.

(equilibrium)
Let $x\in I$. We call a pair
$(\tau_{1},\tau_{2})\in\operatorname{\mathcal{R}}(E_{1})\times\operatorname{\mathcal{R}}(E_{2})$
equilibrium at $x$ if

$\displaystyle J^{1}(x,\tau_{1},\tau_{2})\geq
J^{1}(x,\tau_{1}^{\prime},\tau_{2})\quad\text{and}$ (4) $\displaystyle
J^{2}(x,\tau_{1},\tau_{2})\geq J^{2}(x,\tau_{1},\tau_{2}^{\prime})$ (5)

for all $\tau_{1}^{\prime}\in\operatorname{\mathcal{R}}(E_{1})$ and all
$\tau_{2}^{\prime}\in\operatorname{\mathcal{R}}(E_{2})$. If
$(\tau_{1},\tau_{2})$ is an equilibrium at $x$ for all $x\in I$, we call
$(\tau_{1},\tau_{2})$ Markov perfect equilibrium or just equilibrium for
brevity.

In the following, we will look for Markov-perfect equilibria in the much
smaller and manageable class $\mathcal{Z}\subseteq\operatorname{\mathcal{R}}$.
However, it should be noted that these must satisfy the equilibrium conditions
(4) and (5) for general challengers
$\tau^{\prime}\in\operatorname{\mathcal{R}}$.

######  Assumptions.

  1. (A)

$g_{1}\leq f_{1}$ and $g_{2}\leq f_{2}$.

  2. (B)

$X$ lives on a compact interval $I\subset\mathbb{R}$ with absorbing boundary
points $y\in\partial I$.

  3. (C)

$f_{1}(y)=g_{1}(y)$ and $f_{2}(y)=g_{2}(y)$ for all $y\in\partial I$.

######  Remark 2.3.

The first assumptions correspond to the general situation of a war of
attrition, where both players prefer to stop second, so that a randomization
strategy seems appropriate. The boundary behavior of $X$ is technical. For the
third condition on the the absorbing points $y\in\partial I$, it is easily
seen that the equality $f_{i}(y)=g_{i}(y)$ for one player $i$ is actually
necessary for the existence of equilibria.

A large part of the following argumentation does not require all assumptions.
We will indicate which ones we actually use at which point. We discuss
possible generalizations in Section 6. Under all three assumptions, however,
the following general existence theorem is valid. We will give the proof of
this in the next three sections.

######  Theorem 2.4.

Under the assumptions (A), (B) and (C) there exists a Markov perfect
equilibrium for (3).

##  3 Topologization of Markovian randomized times

Our approach to proving the existence theorem is to apply a suitable fixed
point theorem of Kakutani type (see, e.g., [38, Lemma 20.1]) to the best
response mapping. The prerequisites for the application of such fixed point
theorems are always variants of the following:

  * •

compactness of the underlying space

  * •

closed graph of the best response mapping

  * •

convexity of the best response sets

It can be seen that the choice of topology is essential and that there is no
canonical topology for the space of Markovian randomized stopping times. We
will actually consider two different topologies for our line of argument. One
is based on the description of $\mathcal{Z}$ by a representing measure, the
other on the distribution of the stopped process. We would like to emphasize
that the following results are not based on the specific stopping game we look
at, but are of general interest. In particular the assumptions (A), (B), (C)
are not needed in this entire section.

###  3.1 $\mathcal{M}$-topology

First, we consider a topology that we will use later to prove fixed point
theorems in the discretized game. It is based on the fact that discrete
measures can be described by their counting densities. More precisely, in the
following definition it is considered which local-time stopping rates
$\lambda^{D}(\\{x\\})\in[0,\infty]$ a stopping time $\tau^{D,\lambda}$ uses on
a discrete set $U$. Now, the $\mathcal{M}$-topology is built so that
$\tau,\tau_{1},\tau_{2},...\in\mathcal{Z}_{U}$ converge to $\tau$ if and only
if the stopping rates converge for all $x\in U$. More precisely:

######  Definition 3.1.

($\mathcal{M}$-topology)
Let $U\subset I$ be finite or countable. Let
$\iota_{1}:\mathcal{Z}_{U}\to\mathcal{M}(U)$ be given by

$\displaystyle\tau^{D,\lambda}\mapsto\lambda^{D},\quad\lambda^{D}(A):=\begin{cases}\lambda(A),&\text{if
}A\subset D,\\\ \infty,&\text{if }A\setminus D\neq\varnothing,\end{cases}\quad
A\in\mathcal{B}(U).$

Further let $\iota_{2}:\mathcal{M}(U)\to[0,1]^{U}$ be given by

$\displaystyle\mu\mapsto\left(\frac{\mu(\\{x\\})}{1+\mu(\\{x\\})}\right)_{x\in
U},$

where we write $\frac{\infty}{1+\infty}:=1$. We equip $\mathcal{M}(U)$ with
the pullback topology under $\iota_{2}$. We define the $\mathcal{M}$-topology
as the pullback topology on $\mathcal{Z}(U)$ under the mapping $\iota_{1}$ or
$\iota:=\iota_{2}\circ\iota_{1}$ respectively. For $\tau\in\mathcal{Z}_{U}$ we
call $\iota_{1}(\tau)$ stopping rate (of $\tau$) and for $x\in U$ we call
$\iota_{1}(\tau)(\\{x\\})$ stopping rate (of $\tau$) at $x$.

######  Remark 3.2.

  1. (i)

For our (generally non-symmetric) Dynkin games, we will encounter the product
space $\mathcal{Z}_{U}(E_{1})\times\mathcal{Z}_{U}(E_{2})$ that encodes the
strategies of both players. The $\mathcal{M}$-topology on the product space
will be defined as the product topology of the $\mathcal{M}$-topologies on
both components. Equivalently, we could define an embedding
$\overline{\iota}:\mathcal{Z}(E_{1})\times\mathcal{Z}(E_{2})\to[0,1]^{U}\times[0,1]^{U}$,
$\overline{\iota}:=(\iota,\iota)$ and define the $\mathcal{M}$-topology on
$\mathcal{Z}_{U}(E_{1})\times\mathcal{Z}_{U}(E_{2})$ as the pullback topology
under $\overline{\iota}$.

  2. (ii)

For finite or countable $U\subset I$, the mappings $\iota_{1}$ and $\iota_{2}$
(and thus the composition $\iota$) are one-to-one and onto, so the spaces
$[0,1]^{U},\mathcal{M}(U)$ and $\mathcal{Z}_{U}$ with the
$\mathcal{M}$-topology are homeomorphic. In the case of finite $U$, we will
see in the next section that this allows to use the compactness and convexity
of the space $[0,1]^{U}\subset\mathbb{R}^{U}$ in order to construct a best
response mapping that allows for the application of Kakutani’s fixed point
theorem [38, Lemma 20.1]. A similar approach was used for discrete processes
in [16, 15].

  3. (iii)

For uncountable $U\subset I$, it is well-known that measures on infinite
spaces are not characterized by the one-point evaluations. In our terminology,
this means that the mapping $\iota_{2}$ is not one-to-one in this case. This
prohibits the same approach for such $U$, especially for $U=I$. This makes it
impossible to use the topology $\mathcal{M}$ fruitfully for the general case.

######  Lemma 3.3.

Let $U\subset I$ finite,
$\tau,\tau_{1},\tau_{2},\tau_{3},...\in\mathcal{Z}_{U}$ with
$\tau_{n}\stackrel{{\scriptstyle n\to\infty}}{{\longrightarrow}}\tau$ in the
$\mathcal{M}$-topology.

  1. (i)

For all $T\in[0,\infty)$ we have

$A^{\tau_{n}}_{t}(\omega)\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}A^{\tau}_{t}(\omega)$

uniformly in $t\in[0,T]$ for all $\omega\in\\{A^{\tau}_{T}<\infty\\}$.

  2. (ii)

For all $x\in I$ there is a $\mathbb{P}_{x}$-nullset $N_{1}$ such that for all
$\varepsilon>0$, $t\in[0,\infty)$ and all
$\omega\in\\{A^{\tau}_{t}=\infty\\}\setminus N_{1}$ we have

$\displaystyle A^{\tau_{n}}_{t+\varepsilon}(\omega)\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}A^{\tau}_{t+\varepsilon}(\omega)=A^{\tau}_{t}(\omega)=\infty.$

  3. (iii)

$\tau_{n}\stackrel{{\scriptstyle n\to\infty}}{{\longrightarrow}}\tau$
$\mathbb{P}_{x}$-a.s. for all $x\in I$.

  4. (iv)

We have $X_{\tau_{n}}\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}X_{\tau}$ $\mathbb{P}_{x}$-a.s. for all $x\in
I$.

###### Proof.

(i) By choosing $N\in\mathbb{N}$ such that
$|\iota(\tau_{n})-\iota(\tau)|<1-\max\\{\iota(\tau)(x):\iota(\tau)(x)<1\\}$
for all $n\geq N$ we get

$\displaystyle\tau_{n}^{\infty}:=\\{x\in
U:\iota_{1}(\tau_{n})(\\{x\\})=\infty\\}\subset\tau^{\infty}:=\\{x\in
U:\iota_{1}(\tau)(\\{x\\})=\infty\\}$

for all $n\geq N$.

Let $T\in[0,\infty),t\in[0,T]$ and $\omega\in\\{A^{\tau}_{T}<\infty\\}$. Now

$\displaystyle\infty>A^{\tau}_{T}(\omega)=\int
l^{y}_{T}(\omega)\iota_{1}(\tau)(dy)+\infty\mathds{1}_{{[\eta^{\tau^{\infty}}(\omega),\infty)}}(T)=\sum_{y\in
U\setminus\tau^{\infty}}l^{y}_{T}(\omega)\iota_{1}(\tau)(\\{z\\})+\infty\mathds{1}_{{[\eta^{\tau^{\infty}}(\omega),\infty)}}(T)$

We read off

$\displaystyle 0\leq l^{y}_{t}(\omega)\leq l^{y}_{T}(\omega)=0\quad\forall
y\in\tau^{\infty},\quad\eta^{\tau^{\infty}}(\omega)>T.$

For $n\geq N$ this yields

$\displaystyle|A^{\tau_{n}}_{t}(\omega)-A^{\tau}_{t}(\omega)|$
$\displaystyle=$
$\displaystyle\left|\int_{U\setminus\tau^{\infty}_{n}}l^{y}_{t}(\omega)\iota_{1}(\tau_{n})(dy)+\infty\mathds{1}_{{[\eta^{\tau^{\infty}}(\omega),\infty)}}(t)-\int_{U\setminus\tau^{\infty}}l^{y}_{t}(\omega)\iota_{1}(\tau)(dy)-\infty\mathds{1}_{{[\eta^{\tau^{\infty}}(\omega),\infty)}}(t)\right|$
$\displaystyle\leq$ $\displaystyle\sum_{z\in
U\setminus\tau^{\infty}}l^{y}_{t}(\omega)|\iota_{1}(\tau_{n})(\\{z\\})-\iota_{1}(\tau)(\\{z\\})|\leq\sum_{z\in
U\setminus\tau^{\infty}}l^{y}_{T}(\omega)|\iota_{1}(\tau_{n})(\\{z\\})-\iota_{1}(\tau)(\\{z\\})|\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}0.$

(ii) Let $t\geq 0$ and $\omega\in\\{A^{\tau}_{t}=\infty\\}$. We have

$\displaystyle
A^{\tau_{n}}_{t+\varepsilon}(\omega)\geq\int_{U}l^{y}_{t+\varepsilon}(\omega)\iota_{1}(\tau_{n})(dy)+\infty\mathds{1}_{{[\eta^{\tau^{\infty}_{n}}(\omega),\infty)}}(t+\varepsilon),$

while

$\displaystyle\infty=A^{\tau}_{t}(\omega)=\int_{U\setminus\tau^{\infty}}l^{y}_{t}(\omega)\iota_{1}(\tau)(dy)+\infty\mathds{1}_{{[\eta^{\tau^{\infty}}(\omega),\infty)}}(t+\varepsilon)$

i.e. $\eta^{\tau^{\infty}}(\omega)\leq t$. Thus
$\eta^{\tau^{\infty}}(\omega)<t+\varepsilon$. Choose a
$\mathbb{P}_{x}$-nullset $N_{1}$ such that $l^{z}_{t+\varepsilon}(\omega)>0$
for all $\omega\in\Omega\setminus N_{1}$ and some $z\in\tau^{\infty}$. Note
that $N_{1}$ can be chosen independently from $t,\varepsilon$ and $z$ since
$\tau^{\infty}$ is finite. The monotonicity of
$l^{y}_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}$
allows to consider only rational $t,\varepsilon$ and then collect all the
corresponding exception nullsets. Applying $\iota_{1}(\tau)(\\{z\\})=\infty$,
this leads to

$\displaystyle
A^{\tau_{n}}_{t+\varepsilon}(\omega)\geq\int_{U}l^{z}_{t+\varepsilon}(\omega)\iota_{1}(\tau_{n})(dy)\geq
l^{z}_{t+\varepsilon}(\omega)\iota_{1}(\tau_{n})(\\{z\\})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}l^{z}_{t+\varepsilon}(\omega)\iota_{1}(\tau)(\\{z\\})=\infty$

for all $\omega\in\\{A^{\tau}_{t}=\infty\\}\setminus N_{1}$.

(iii) Let $\varepsilon>0$. We start by showing
$\liminf_{n\to\infty}\tau_{n}\geq\tau-\varepsilon$. For that let
$t\leq\tau-\varepsilon$. By definition $A^{\tau_{n}}_{t}<E$. By (i)

$\displaystyle A^{\tau_{n}}_{t}\leq
A^{\tau_{n}}_{t-\varepsilon}\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}A^{\tau}_{t-\varepsilon}<E$

and thus $\tau_{n}>t$ for sufficiently large $n\in\mathbb{N}$. Since $t$ was
arbitrary, this finishes the first part. Next, we show the inequality
$\limsup_{n\to\infty}\tau_{n}\leq\tau+\varepsilon$. On
$\\{A^{\tau}_{\tau}=\infty\\}$, by (ii), we get
$A^{\tau_{n}}_{\tau+\varepsilon}\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}\infty\geq E$ $\mathbb{P}_{x}$-a.s. and thus
$\tau_{n}\leq\tau+\varepsilon$ $\mathbb{P}_{x}$-a.s. for sufficiently large
$n$. On $\\{E\leq A^{\tau}_{\tau}<\infty\\}$ we have
$\eta^{\tau^{\infty}}>\tau$. Since $\int_{0}^{\infty}\mathds{1}_{\\{{X_{s}\neq
y}\\}}l^{y}_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}(ds)=0$
$\mathbb{P}_{x}$-a.s., we infer

$\displaystyle\int^{\tau}_{0}\mathds{1}_{\\{{X_{s}\not\in\operatorname{supp}(\iota_{1}(\tau))}\\}}dA^{\tau}_{s}=\sum_{z\in\operatorname{supp}(\iota_{1}(\tau))}\int^{\tau}_{0}\mathds{1}_{\\{{X_{s}\not\in\operatorname{supp}(\iota_{1}(\tau))}\\}}l^{z}_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}(ds)\cdot\iota_{1}(\tau)(\\{z\\})$

$\mathbb{P}_{x}$-a.s. on $\\{E\leq A^{\tau}_{\tau}<\infty\\}$. This, however,
implies $X_{\tau}\in\operatorname{supp}(\iota_{1}(\tau))$
$\mathbb{P}_{x}$-a.s. on $\\{E\leq A^{\tau}_{\tau}<\infty\\}$. Putting these
thing together we get

$\displaystyle A^{\tau_{n}}_{\tau+\varepsilon}=\theta_{\tau}\circ
A^{\tau_{n}}_{\varepsilon}+A^{\tau_{n}}_{\tau}=\int\theta_{\tau}\circ
l^{z}_{\varepsilon}\iota_{1}(\tau_{n})(dz)+A^{\tau_{n}}_{\tau}$
$\displaystyle=$
$\displaystyle\sum_{z\in\operatorname{supp}(\iota_{1}(\tau_{n}))}\theta_{\tau}\circ
l^{z}_{\varepsilon}\iota_{1}(\tau_{n})(dz)+A^{\tau_{n}}_{\tau}\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}\sum_{z\in\operatorname{supp}(\iota_{1}(\tau))}\theta_{\tau}\circ
l^{z}_{\varepsilon}\iota_{1}(\tau)(dz)+A^{\tau}_{\tau}$ (6)

$\mathbb{P}_{x}$-a.s. on $\\{E\leq A^{\tau}_{\tau}<\infty\\}$. Using
$\mathbb{P}_{z}(l^{z}_{\varepsilon}>0)=1$ for all $z\in I$ and
$\mathbb{P}_{x}(X_{\tau}\in\operatorname{supp}(\iota_{1}(\tau))|E\leq
A^{\tau}_{\tau}<\infty)=1$ one sees that

$\displaystyle\mathbb{P}_{x}\left(\sum_{z\in\operatorname{supp}(\iota_{1}(\tau))}\theta_{\tau}\circ
l^{z}_{\varepsilon}\iota_{1}(\tau)(dz)>0\bigg{|}E\leq
A^{\tau}_{\tau}<\infty\right)$ $\displaystyle=$
$\displaystyle\mathbb{E}_{x}\left[\mathbb{E}_{x}\left[\mathds{1}_{\\{{\sum_{z\in\operatorname{supp}(\iota_{1}(\tau))}\theta_{\tau}\circ
l^{z}_{\varepsilon}\iota_{1}(\tau)(dz)>0}\\}}\big{|}\mathcal{F}_{\tau}\right]\bigg{|}E\leq
A^{\tau}_{\tau}<\infty\right]$ $\displaystyle=$
$\displaystyle\mathbb{E}_{x}\left[\mathbb{E}_{X_{\tau}}\left[\mathds{1}_{\\{{\sum_{z\in\operatorname{supp}(\iota_{1}(\tau))}l^{z}_{\varepsilon}\iota_{1}(\tau)(dz)>0}\\}}\right]\bigg{|}E\leq
A^{\tau}_{\tau}<\infty\right]=1.$

Combined with (6) we get

$\displaystyle A^{\tau_{n}}_{\tau+\varepsilon}>A^{\tau}_{\tau}\geq E$

for $\mathbb{P}_{x}$-a.a. $\omega\in\\{E\leq A^{\tau}_{\tau}<\infty\\}$ and
sufficiently large $n$. This implies $\tau_{n}\leq\tau+\varepsilon$
$\mathbb{P}_{x}$-a.s. on $\\{E\leq A^{\tau}_{\tau}<\infty\\}$. Sending
$\varepsilon\searrow 0$ yields the claim.

(iv) is immediate by (iii) and continuity of $X$ paths. ∎

###  3.2 $\mathcal{M}^{1}_{x}$-topology

The second topology is based on the idea that on $\mathcal{Z}$ we should
choose a notion of convergence such that convergence of the stopping times
just corresponds to the convergence of the distributions of the stopped
process generated by it. As a first step, we start with the following
definition.

######  Definition 3.4.

($\mathcal{M}^{1}_{x}$-topology)
Let $x\in I$ and $U\subset I$ open. Further, we equip the space
$\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$ with the Prokhorov
metric. Let
$\kappa_{x}:\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})\to\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$
be given by

$\displaystyle(\tau^{D_{1},\lambda_{1}},\tau^{D_{2},\lambda_{2}})\mapsto\mathbb{P}_{x}^{(\tau^{D_{1},\lambda_{1}},X_{\tau^{D_{1},\lambda_{1}}},\tau^{D_{2},\lambda_{2}},X_{\tau^{D_{2},\lambda_{2}}})}.$

We define the $\mathcal{M}^{1}_{x}$-topology as the pullback topology on
$\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$ under the mapping
$\kappa_{x}$.

The rest of the subsection will essentially be about showing the compactness
properties of this space. The result is the following:

######  Proposition 3.5.

Let $U\subset I$ be open with compact closure $\overline{U}\subset I$ (closure
taken in $\mathbb{R}$) and $x\in U$. Then,
$\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$ equipped with the
$\mathcal{M}^{1}_{x}$-topology is a compact topological space, i.e., the set
$\kappa_{x}(\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2}))\subset\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$
is weakly compact.

Since $([0,\infty]\times\overline{U})^{2}$ is compact,
$\kappa_{x}(\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2}))$ is relatively
weakly compact by Prokhorov’s theorem. As usual the closedness will be much
more challenging. We will divide the proof into several lemmas. We begin by
introducing some notation and explaining the general approach.

$\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$ is metric, so it
suffices to show sequential closedness in order to complete the compactness
proof. For this, let
$((\tau^{D^{1}_{n},\lambda^{1}_{n}},\tau^{D^{2}_{n},\lambda^{2}_{n}}))_{n\in\mathbb{N}}\in(\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2}))^{\mathbb{N}}$
be a sequence of tuples of Markovian randomized times such that
$\kappa_{x}(\tau^{D^{1}_{n},\lambda^{1}_{n}},\tau^{D^{2}_{n},\lambda^{2}_{n}})$
converges to a probability measure
$\mu\in\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$. We denote the
marginals of $\mu$ by $\mu_{1},\mu_{2},\mu_{3},\mu_{4}$. Closedness now means
that the limiting distribution $\mu$ of the sequence is generated by Markovian
randomized times. That is, we have to find
$(\tau^{1},\tau^{2}):=(\tau^{D^{1},\lambda^{1}},\tau^{D^{2},\lambda^{2}})\in\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$
such that $\kappa_{x}(\tau^{1},\tau^{2})=\mu$. To abbreviate we set
$(\tau^{1}_{n},\tau^{2}_{n}):=(\tau^{D^{1}_{n},\lambda^{1}_{n}},\tau^{D^{2}_{n},\lambda^{2}_{n}})$
for each $n\in\mathbb{N}$. All these objects and notations are fixed
throughout this section. In addition, all the notions introduced in the Lemmas
3.6, 3.7, 3.9, and 3.10 are fixed for the entire section as well.

In Lemma 3.6 we construct the candidate for the continuation regions
$D_{1}:=(L^{1},R^{1})$, $D_{2}:=(L^{2},R^{2})$ of $\tau^{1},\tau^{2}$ around
$x$ via the supports of the measures $\mu_{2},\mu_{4}$ and derive the order of
$L^{i},R^{i},i\in\\{1,2\\}$ and the limits of the supports of the measures
$\mathbb{P}_{x}^{X_{\tau^{1}_{n}}},\mathbb{P}_{x}^{X_{\tau^{2}_{n}}}$. In
Lemma 3.7 we show a consistency condition that justifies to construct the
continuation regions of $\tau^{1},\tau^{2}$ as the supports of the measures
$\mu_{2},\mu_{4}$. In Lemma 3.9 we construct $\lambda^{i}$, $i\in\\{1,2\\}$ as
the vague limit of a sequence of measures related to the sequence
$(\lambda^{i}_{n})_{n\in\mathbb{N}}$. The main issue to overcome is that the
measures $\lambda^{i}_{n}\in\operatorname{RM}((L^{i}_{n},R^{i}_{n}))$ may have
different domains. Here we use the ordering (8) to find a common domain. In
Lemma 3.10 we will establish $\tau^{i}_{n}\to\tau^{i}$ $\mathbb{P}_{x}$-a.s.
along suitably chosen subsequences. This is obtained by applying Lemma 3.7 and
identifying local times stopped at certain (first entry) times for fixed
$\omega$ as test functions for vague convergence to obtain suitable
convergence of the additive functionals defining the stopping times
$\tau^{1},\tau^{2}$. Based on this, we show the remaining closedness statement
of the proof of Proposition 3.5.

######  Lemma 3.6.

Set

$\displaystyle L^{1}:=\sup\left\\{y\in
I\cap(-\infty,x]:\mu_{2}(I\cap(-\infty,y))=0\right\\},$ $\displaystyle
R^{1}:=\inf\left\\{y\in I\cap[x,\infty):\mu_{2}(I\cap(y,\infty))=0\right\\},$
$\displaystyle L^{2}:=\sup\left\\{y\in
I\cap(-\infty,x]:\mu_{4}(I\cap(-\infty,y))=0\right\\},$ $\displaystyle
R^{2}:=\inf\left\\{y\in I\cap[x,\infty):\mu_{4}(I\cap(y,\infty))=0\right\\}$

using the conventions $\sup\varnothing:=\inf I,\inf\varnothing:=\sup I$. With
that, we define $D^{1}:=(L^{1},R^{1})$, $D^{2}:=(L^{2},R^{2})$. Note that this
includes the option $D^{i}=(x,x)=\varnothing$ for $i\in\\{1,2\\}$. Let

$\displaystyle L^{1}_{n}:=\sup(I\setminus D^{1}_{n})\cap(-\infty,x],$
$\displaystyle R^{1}_{n}:=\inf(I\setminus D^{1}_{n})\cap[x,\infty),$
$\displaystyle L^{2}_{n}:=\sup(I\setminus D^{2}_{n})\cap(-\infty,x],$
$\displaystyle R^{2}_{n}:=\inf(I\setminus D^{2}_{n})\cap[x,\infty),$

and for $i\in\\{1,2\\}$ set

$\displaystyle\tilde{L}^{i}:=\limsup_{n\to\infty}L^{i}_{n},\quad\tilde{R}^{i}:=\liminf_{n\to\infty}R^{i}_{n}.$
(7)

Then, for $i\in\\{1,2\\}$ we have

$\displaystyle-\infty<\inf U\leq\tilde{L}^{i}\leq L^{i}\leq x\leq
R^{i}\leq\tilde{R}^{i}\leq\sup U<\infty.$ (8)

###### Proof.

Note that $I\setminus D^{i}_{n}\supset\partial U$ for each $n$ and
$(-\infty,x]\cap\partial U\neq\varnothing\neq[x,\infty)\cap\partial U$, so
$\inf U\leq\tilde{L}^{i}$ and $\tilde{R}^{i}\leq\sup U$. $-\infty<\inf U$ and
$\sup U<\infty$ are a consequence of compactness of $\overline{U}$. By choice
of $L^{i}_{n},R^{i}_{n}$, we have

$\displaystyle\mathbb{P}_{x}^{X_{\tau^{i}_{n}}}(I\cap((-\infty,L^{i}_{n})\cup(R^{i}_{n},\infty)))=0$

for all $n\in\mathbb{N}$. Since for all $\varepsilon>0$ and all
$N\in\mathbb{N}$, there is some $n\geq N$ such that
$L^{i}_{n}>\tilde{L}^{i}-\varepsilon$ or
$R^{i}_{n}<\tilde{R}^{i}+\varepsilon$, respectively, this implies

$\displaystyle\liminf_{n\to\infty}\mathbb{P}_{x}^{X_{\tau^{i}_{n}}}(I\cap(-\infty,\tilde{L}^{i}-\varepsilon))=0\quad\text{and}\quad\liminf_{n\to\infty}\mathbb{P}_{x}^{X_{\tau^{i}_{n}}}(I\cap(\tilde{R}^{i}+\varepsilon,\infty))=0.$

By the Portmanteau theorem we get

$\displaystyle\mu_{2i}(I\cap((-\infty,\tilde{L}^{i})\cup(\tilde{R}^{i},\infty)))=\mu_{2i}(I\cap((-\infty,\tilde{L}^{i})))+\mu_{2i}(I\cap((\tilde{R}^{i},\infty)))$
$\displaystyle=$ $\displaystyle\lim_{\varepsilon\searrow
0}\big{(}\mu_{2i}(I\cap((-\infty,\tilde{L}^{i}-\varepsilon)))+\mu_{2i}(I\cap((\tilde{R}^{i}+\varepsilon,\infty)))\big{)}$
$\displaystyle\leq$ $\displaystyle\lim_{\varepsilon\searrow
0}\big{(}\liminf_{n\to\infty}\mathbb{P}_{x}^{X_{\tau_{n}}}(I\cap((-\infty,\tilde{L}^{i}-\varepsilon)))+\liminf_{n\to\infty}\mathbb{P}_{x}^{X_{\tau_{n}}}(I\cap((\tilde{R}^{i}+\varepsilon,\infty)))\big{)}=0.$

With the definition of $L^{i},R^{i}$, we obtain $\tilde{L}^{i}\leq L^{i}$ and
$R^{i}\leq\tilde{R}^{i}$. $L^{i}\leq x\leq R^{i}$ is trivial. ∎

######  Lemma 3.7.

Let $i\in\\{1,2\\}$ and $\tilde{L}^{i},\tilde{R}^{i}$ as in (7).

  1. (i)

If $L^{i}=x=R^{i}$ then
$\liminf_{n\to\infty}\lambda^{i}_{n}((x-\varepsilon,x+\varepsilon)\cap
D^{i}_{n})=\infty$ for all $\varepsilon>0$ or $\tilde{L}^{i}=L^{i}$ or
$\tilde{R}^{i}=R^{i}$.

  2. (ii)

If $L^{i}=x<R^{i}$ then
$\liminf_{n\to\infty}\lambda^{i}_{n}((x-\varepsilon,x+\varepsilon)\cap
D^{i}_{n})=\infty$ for all $\varepsilon>0$ or $\tilde{L}^{i}=L^{i}$.

  3. (iii)

If $L^{i}<x=R^{i}$ then
$\liminf_{n\to\infty}\lambda^{i}_{n}((x-\varepsilon,x+\varepsilon)\cap
D^{i}_{n})=\infty$ for all $\varepsilon>0$ or $\tilde{R}^{i}=R^{i}$.

  4. (iv)

If $L^{i}<x<R^{i}$ then
$\liminf_{n\to\infty}\lambda^{i}_{n}((L^{i}-\varepsilon,L^{i}+\varepsilon)\cap
D^{i}_{n})=\infty$ for all $\varepsilon>0$ or $\tilde{L}^{i}=L^{i}$ as well as
$\liminf_{n\to\infty}\lambda^{i}_{n}((R^{i}-\varepsilon,R^{i}+\varepsilon)\cap
D^{i}_{n})=\infty$ for all $\varepsilon>0$ or $\tilde{R}^{i}=R^{i}$.

######  Remark 3.8.

It will turn out, that the cases the (ii) and (iii) of Lemma 3.7 are in fact
empty, cf. Lemma 3.9 (i).

###### Proof.

If any of (i), (ii), (iii) or (iv) were false, using that $\tilde{L}^{i}\neq
L^{i}$ implies $\tilde{L}^{i}<L^{i}$ and $\tilde{R}^{i}\neq R^{i}$ implies
$\tilde{R}^{i}<R^{i}$ by Lemma 3.6 (8), we could find some $\varepsilon>0$
such that

$\displaystyle
K:=\limsup_{n\to\infty}\lambda^{i}_{n}([L^{i}-\varepsilon,x+\varepsilon])<\infty\quad\text{or}\quad\limsup_{n\to\infty}\lambda^{i}_{n}([x-\varepsilon,R^{i}+\varepsilon])<\infty$

with $x+\varepsilon\in(L^{i},R^{i})$, $\tilde{L}^{i}<L^{i}-\varepsilon$ or
$x-\varepsilon\in(L^{i},R^{i})$, $\tilde{R}^{i}>R^{i}+\varepsilon$
respectively (in case of (iv) we get both). W.l.o.g. we only argue for
$L^{i}$. Note that $\tau^{i}_{n}$ stops immediately upon hitting $L^{i}_{n}$,
so if $X$ reaches $L^{i}-\varepsilon$ without being stopped by $\tau^{i}_{n}$,
and then hits $L^{i}_{n}$ before reentering $L^{i}-\frac{\varepsilon}{2}$, the
process will be stopped in $I\setminus(L^{i}-\frac{\varepsilon}{2},\infty)$.
This can be rephrased as $X_{\tau^{i}_{n}}\in
I\setminus(L^{i}-\frac{\varepsilon}{2},\infty)$ $\mathbb{P}_{x}$-a.s. on
$\\{\eta_{L^{i}-\varepsilon}<\tau^{i}_{n}\\}\cap\\{\theta_{\eta_{L^{i}-\varepsilon}}\circ\eta_{L^{i}_{n}-\varepsilon}<\theta_{\eta_{L^{i}-\varepsilon}}\circ\eta_{L^{i}-\frac{\varepsilon}{2}}\\}$.
Combining this with the fact that
$\mathbb{E}_{z}[\mathds{1}_{\\{{\eta_{a}<\eta_{b}}\\}}]$, $a\leq z\leq b$ is a
continuous function of $a$, [33, p. 119] and
$\tilde{L}^{i}=\limsup_{n\to\infty}L^{i}_{n}$, we obtain the following
estimate, where the right hand side is positive since the exponential term is
positive $\mathbb{P}_{x}$-a.s. and
$\mathbb{E}_{z}[\mathds{1}_{\\{{\eta_{a}<\eta_{b}}\\}}]$ is positive for all
$a<z<b$:

$\displaystyle\limsup_{n\to\infty}\mathbb{P}_{x}(X_{\tau^{i}_{n}}\in
I\setminus(L^{i}-\frac{\varepsilon}{2},\infty))$ $\displaystyle\geq$
$\displaystyle\limsup_{n\to\infty}\mathbb{P}_{x}(\\{\eta_{L^{i}-\varepsilon}<\tau^{i}_{n}\\}\cap\\{\theta_{\eta_{L^{i}}}\circ\eta_{L^{i}_{n}-\varepsilon}<\theta_{\eta_{L^{i}-\varepsilon}}\circ\eta_{L^{i}-\frac{\varepsilon}{2}}\\})$
$\displaystyle=$
$\displaystyle\limsup_{n\to\infty}\mathbb{E}_{x}[\mathds{1}_{\\{{\eta_{L^{i}-\varepsilon}<\tau^{i}_{n}}\\}}\mathbb{E}_{X_{\eta_{L^{i}-\varepsilon}}}[\mathds{1}_{\\{{\eta_{L^{i}_{n}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]]$
$\displaystyle=$
$\displaystyle\limsup_{n\to\infty}\mathbb{E}_{x}[\mathds{1}_{\\{{\eta_{L^{i}-\varepsilon}<\tau^{i}_{n}}\\}}]\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{L^{i}_{n}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]$
$\displaystyle\geq$
$\displaystyle\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{\tilde{L}^{i}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]\limsup_{n\to\infty}\mathbb{E}_{x}[\mathds{1}_{\\{{\eta_{L^{i}-\varepsilon}<\tau^{i}_{n}}\\}}]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{\tilde{L}^{i}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]\limsup_{n\to\infty}\mathbb{P}_{x}(\eta_{L^{i}-\varepsilon}<\tau^{i}_{n})$
$\displaystyle\geq$
$\displaystyle\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{\tilde{L}^{i}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]\limsup_{n\to\infty}\mathbb{P}_{x}(E_{i}<A^{D^{i}_{n},\lambda^{i}_{n}}_{\eta_{L^{i}-\varepsilon}})$
$\displaystyle\geq$
$\displaystyle\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{\tilde{L}^{i}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]\limsup_{n\to\infty}\mathbb{P}_{x}\left(E_{i}<\int
l^{y}_{\eta_{L^{i}-\varepsilon}}\lambda^{i}_{n}(dy)\right)$
$\displaystyle\geq$
$\displaystyle\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{\tilde{L}^{i}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]\limsup_{n\to\infty}\mathbb{E}_{x}\left[e^{-\int
l^{y}_{\eta_{L^{i}-\varepsilon}}d\lambda^{i}_{n}}\right]$ $\displaystyle\geq$
$\displaystyle\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{\tilde{L}^{i}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]\limsup_{n\to\infty}\mathbb{E}_{x}[\mathds{1}_{\\{{l^{y}_{\eta_{L^{i}-\varepsilon}}=0\forall
y\in
I\setminus[L^{i}-\varepsilon,x+\varepsilon]}\\}}e^{-\sup_{y\in[L^{i}-\varepsilon,x+\varepsilon]}l^{y}_{\eta_{L^{i}-\varepsilon}}(K+\varepsilon)}]$
$\displaystyle\geq$
$\displaystyle\mathbb{E}_{L^{i}-\varepsilon}[\mathds{1}_{\\{{\eta_{\tilde{L}^{i}}<\eta_{L^{i}-\frac{\varepsilon}{2}}}\\}}]\mathbb{E}_{x}[\mathds{1}_{\\{{\eta_{L^{i}-\varepsilon}<\eta_{x+\varepsilon}}\\}}e^{-\sup_{y\in[L^{i}-\varepsilon,x+\varepsilon]}l^{y}_{\eta_{L^{i}-\varepsilon}}(K+\varepsilon)}]>0.$

Now by definition of $L^{i}$ and Portmanteau theorem we get a contradiction
via

$\displaystyle
0=\mu_{2i}(I\setminus(L^{i}-\frac{\varepsilon}{2},\infty))\geq\limsup_{n\to\infty}\mathbb{P}_{x}(X_{\tau^{i}_{n}}\in
I\setminus(L^{i}-\frac{\varepsilon}{2},\infty))>0.$

∎

######  Lemma 3.9.

  1. (i)

Let $i\in\\{1,2\\}$. If $L^{i}=x$ or $R^{i}=x$, then there is a subsequence
$(\tau_{n_{k}}^{i})_{k\in\mathbb{N}}$ of $(\tau^{i}_{n})_{n\in\mathbb{N}}$
such that $\tau^{i}_{n_{k}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0$ $\mathbb{P}_{x}$-a.s. and thus $R^{i}=x$ and
$L^{i}=x$.

  2. (ii)

Suppose $L^{i}<x<R^{i}$. Let
$(\delta_{m})_{m\in\mathbb{N}}\in((0,(R^{i}-x)\wedge(x-L^{i})))^{\mathbb{N}}$
with $\delta_{m}\searrow 0$ and

$d\lambda_{n,m}^{i}:=\mathds{1}_{{[L^{i}+\delta_{m},R^{i}-\delta_{m}]}}d\lambda^{i}_{n}\in\operatorname{RM}((L^{i}_{n},R^{i}_{n})).$

    1. (a)

For each fixed $m\in\mathbb{N}$, there is some $N_{m}\in\mathbb{N}$ such that
the measure
$\lambda_{n,m}^{i}|_{[L^{i}+\delta_{m},R^{i}-\delta_{m}]}\in\operatorname{RM}([L^{i}+\delta_{m},R^{i}-\delta_{m}])$
is well defined for all $n\geq N_{m}$.

    2. (b)

For all $m\in\mathbb{N}$ and $n\geq N_{m}$ we extend
$\lambda_{n,m}^{i}|_{[L^{i}+\delta_{m},R^{i}-\delta_{m}]}$ by 0 to a measure
on $(L^{i},R^{i})$ and denote the extension by $\tilde{\lambda}^{i}_{n,m}$.
For each $m\in\mathbb{N}$ we have

$\sup_{n\geq N_{m}}\tilde{\lambda}^{i}_{n,m}((L^{i},R^{i}))<\infty.$

    3. (c)

Let $(n_{k})_{k\in\mathbb{N}}\in\mathbb{N}^{\mathbb{N}}$ be strictly
increasing. There is a subsequence $(n_{k_{m}})_{m\in\mathbb{N}}$ such that
$n_{k_{m}}\geq N_{m}$ for all $m\in\mathbb{N}$ and

$\displaystyle\tilde{\lambda}^{i}_{n_{k_{m}},m}\stackrel{{\scriptstyle
m\to\infty}}{{\longrightarrow}}\tilde{\lambda}^{i}$

vaguely for some $\tilde{\lambda}^{i}\in\operatorname{RM}((L^{i},R^{i}))$.

  3. (iii)

Based on this, we can construct $\lambda^{1},\lambda^{2}$ as follows:

    1. (a)

If $L^{1}=x,L^{2}=x$ we set
$\lambda^{1}:=\lambda^{2}:=0\in\operatorname{RM}(\varnothing)$.

    2. (b)

If $L^{i}=x$ and $L^{j}<x<R^{j}$ for $i\neq j\in\\{1,2\\}$, we set
$\lambda^{i}:=0$ as before and apply (iic) with $(n_{k})_{k\in\mathbb{N}}$
being such that $\tau^{i}_{n_{k}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0$ $\mathbb{P}_{x}$-a.s. to find a subsequence
$(n_{k_{m}})_{m\in\mathbb{N}}$ and
$\tilde{\lambda}^{j}\in\operatorname{RM}((L^{j},R^{j}))$ such that
$\tilde{\lambda}^{j}_{n_{k_{m}},m}\stackrel{{\scriptstyle
m\to\infty}}{{\longrightarrow}}\tilde{\lambda}^{j}$ vaguely. Then we set
$\lambda^{j}:=\tilde{\lambda}^{j}$.

    3. (c)

If $L^{i}<x<R^{i}$ for all $i\in\\{1,2\\}$, we apply (iic) with
$(n)_{n\in\mathbb{N}}$ in place of $(n_{k})_{k\in\mathbb{N}}$ to find a
subsequence $(n_{k})_{k\in\mathbb{N}}$ and
$\tilde{\lambda}^{j}\in\operatorname{RM}((L^{j},R^{j}))$ $j=1$? such that
$\tilde{\lambda}^{1}_{n_{k},k}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}\tilde{\lambda}^{1}$ vaguely. Then we apply
(iic) to the subsequence $(n_{k})_{k\in\mathbb{N}}$ we obtained in the last
step to find a further subsequence $(n_{k_{m}})_{m\in\mathbb{N}}$ and
$\tilde{\lambda}^{2}\in\operatorname{RM}((L^{2},R^{2}))$ such that
$\tilde{\lambda}^{2}_{n_{k_{m}},m}\stackrel{{\scriptstyle
m\to\infty}}{{\longrightarrow}}\tilde{\lambda}^{2}$ vaguely. We now set
$\lambda^{1}:=\tilde{\lambda}^{1}$ and $\lambda^{2}:=\tilde{\lambda}^{2}$.

  4. (iv)

There is a subsequence $(n_{k})_{k\in\mathbb{N}}$ such that for each
$i\in\\{1,2\\}$ either $\tau^{i}_{n_{k}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0$ $\mathbb{P}_{x}$-a.s. or
$\tilde{\lambda}^{i}_{n_{k},k}\stackrel{{\scriptstyle
m\to\infty}}{{\longrightarrow}}\lambda^{i}$ vaguely on
$\operatorname{RM}((L^{i},R^{i}))$ with $L^{i}<x<R^{i}$.

###### Proof.

(i) By Lemma 3.7, $x=L^{i}=\tilde{L}^{i}$, $x=R^{i}=\tilde{R}^{i}$ or
$\liminf_{n\to\infty}\lambda^{i}_{n}((x-\varepsilon,x+\varepsilon)\cap
D^{i}_{n})=\infty$ for all $\varepsilon>0$. We start by treating the case
$x=L^{i}=\tilde{L}^{i}$. We choose $(n_{k})_{k\in\mathbb{N}}$ such that
$L^{i}_{n_{k}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}\tilde{L}^{i}=x$. Then

$\displaystyle
0\leq\tau^{i}_{n_{k}}\leq\eta_{L^{i}_{n_{k}}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0\quad\mathbb{P}_{x}\text{-a.s.}$

The case $x=R^{i}=\tilde{R}^{i}$ case be treated analogously. Next we assume
$\liminf_{n\to\infty}\lambda^{i}_{n}((x-\varepsilon,x+\varepsilon)\cap
D^{i}_{n})=\infty$ for all $\varepsilon>0$. Let $\varepsilon>0$. The
assumption yields

$\displaystyle
A^{D^{i}_{n},\lambda^{i}_{n}}_{\tau^{(x-\varepsilon,x+\varepsilon)}}\geq\int
l^{y}_{\tau^{(x-\varepsilon,x+\varepsilon)}}\lambda^{i}_{n}(dy)\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}\infty$

$\mathbb{P}_{x}$-a.s. since $y\mapsto
l^{y}_{\tau^{(x-\varepsilon,x+\varepsilon)}}$ is continuous with
$\mathbb{P}_{x}(l^{y}_{\tau^{(x-\varepsilon,x+\varepsilon)}}>0)=1$. From this
we infer

$\displaystyle\mathbb{P}_{x}(\tau^{i}_{n}>\tau^{(x-\varepsilon,x+\varepsilon)})=\mathbb{P}_{x}(E_{i}>A^{D^{i}_{n},\lambda^{i}_{n}}_{\tau^{(x-\varepsilon,x+\varepsilon)}})\geq\mathbb{P}_{x}\left(E_{i}>\int
l^{y}_{\tau^{(x-\varepsilon,x+\varepsilon)}}\lambda^{i}_{n}(dy)\right)\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}0.$

Since $\varepsilon>0$ was arbitrary, we can choose a subsequence
$(\tau^{i}_{n_{k}})_{k\in\mathbb{N}}$ of $(\tau^{i}_{n})_{n\in\mathbb{N}}$
such that
$\sum_{k\in\mathbb{N}}\mathbb{P}_{x}(\tau^{i}_{n_{k}}>\tau^{(x-\frac{1}{k},x+\frac{1}{k})})<\infty$.
By Borel Cantelli’s lemma
$\mathbb{P}_{x}(\limsup_{k\to\infty}\\{\tau^{i}_{n_{k}}>\tau^{(x-\frac{1}{k},x+\frac{1}{k})}\\})=0$.
This implies

$\displaystyle
0\leq\limsup_{k\to\infty}\tau^{i}_{n_{k}}\leq\lim_{k\to\infty}\tau^{(x-\frac{1}{k},x+\frac{1}{k})}=0\quad\mathbb{P}_{x}\text{-a.s.}$

Since $X$ has continuous paths, we infer $X_{\tau^{i}_{n_{k}}}\to x$
$\mathbb{P}_{x}$-a.s. and thus
$\mathbb{P}_{x}^{X_{\tau_{n_{k}}^{i}}}\to\delta_{x}$, where $\delta_{x}$
denotes the Dirac measure in $x$. By uniqueness of weak limits, this yields
$\mu_{2i}=\delta_{x}$. With this, $L^{i}=x=R^{i}$ follows from the definition.

(iia) By Lemma 3.6 (8) given
$\delta_{m}\in(0,(R^{i}-x)\wedge(x-L^{i})),m\in\mathbb{N}$ there is some
$N_{m}\in\mathbb{N}$ such that for all $n\geq N_{m}$ we have
$L^{i}_{n}\leq\limsup_{n\to\infty}L^{i}_{n}+\delta_{m}=\tilde{L}^{i}+\delta_{m}\leq
L^{i}+\delta_{m}<R^{i}-\delta_{m}\leq\tilde{R}^{i}-\delta_{m}=\liminf_{n\to\infty}R^{i}_{n}-\delta_{m}\leq
R^{i}_{n}$. I.e.
$(L^{i}_{n},R^{i}_{n})\supset[L^{i}+\delta_{m},R^{i}-\delta_{m}]$ and thus
$\lambda_{n,m}^{i}|_{[L^{i}+\delta_{m},R^{i}-\delta_{m}]}\in\operatorname{RM}([L^{i}+\delta_{m},R^{i}-\delta_{m}])$
is well defined.

(iib) By definition
$\tilde{\lambda}^{i}_{n,m}((L^{i},R^{i}))=\lambda^{i}_{n,m}([L^{i}+\delta_{m},R^{i}-\delta_{m}])$.
Since Radon measures are finite on compacts, we have
$\lambda^{i}_{n,m}([L^{i}+\delta_{m},R^{i}-\delta_{m}])<\infty$ for all fixed
$n\geq N_{m}$. Thus it suffices to show
$\limsup_{n\to\infty}\lambda^{i}_{n,m}([L^{i}+\delta_{m},R^{i}-\delta_{m}])<\infty$.
We argue by contradiction so assume this were false. Since

$\displaystyle\lambda^{i}_{n,m}([L^{i}+\delta_{m},R^{i}-\delta_{m}]).=$
$\displaystyle\lambda_{n}^{i}(\\{x\\})+\lambda_{n}^{i}([L^{i}+\delta_{m},x))+\lambda_{n}^{i}((x,R^{i}-\delta_{m}]),$

this implies one of the following:

  1. 1.

$\lambda_{n_{k}}^{i}(\\{x\\})\to\infty$ for $k\to\infty$ or

  2. 2.

$\lambda_{n_{k}}^{i}([L^{i}+\delta_{m},x))\to\infty$ for $k\to\infty$ or

  3. 3.

$\lambda_{n_{k}}^{i}((x,R^{i}-\delta_{m}])\to\infty$ for $k\to\infty$.

If (1) using
$\mathbb{P}_{x}(l^{x}_{\tau^{(L^{i}+\delta_{m},R^{i}-\delta_{m})}}>0)=1$ we
obtain

$\displaystyle\mathbb{P}_{x}^{X_{\tau^{i}_{n_{k}}}}(I\setminus[L^{i}+\delta_{m},R^{i}-\delta_{m}])\leq\mathbb{P}_{x}(l^{x}_{\tau^{(L^{i}+\delta_{m},R^{i}-\delta_{m})}}\lambda^{i}_{n_{k},m}(\\{x\\})<E_{i})\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0.$

With Portmanteau’s theorem, this implies

$\displaystyle\mu_{2i}(I\setminus[L^{i}+\delta_{m},R^{i}-\delta_{m}])\leq\liminf_{n\to\infty}\mathbb{P}_{x}^{X_{\tau^{i}_{n}}}(I\cap(-\infty,L^{i}+\delta_{m}))$
$\displaystyle\leq\liminf_{n\to\infty}\mathbb{P}_{x}^{X_{\tau^{i}_{n}}}(I\setminus[L^{i}+\delta_{m},R^{i}-\delta_{m}])=0,$

contradicting the infimal choice of $L^{i}$. We finish by treating the case
(2), (3) being analogous. First note that for all $y\in[L^{i}+\delta_{m},x]$
we have $l^{y}_{\eta_{L^{i}-\delta_{m+1}}}>0$ $\mathbb{P}_{x}$-a.s. Using the
joint continuity of the local time, we infer

$\displaystyle\inf_{y\in[L^{i}+\delta_{m},x]}l^{y}_{\eta_{L^{i}-\delta_{m+1}}}>0\quad\mathbb{P}_{x}\text{-a.s.}$

That yields

$\displaystyle\mu_{2i}(I\setminus(-\infty,L^{i}+\delta_{m+1}))\leq\lim_{k\to\infty}\mathbb{P}_{x}^{X_{\tau^{i}_{n_{k}}}}(I\setminus[L^{i}+\delta_{m+1},\infty))$
$\displaystyle\leq$
$\displaystyle\lim_{k\to\infty}\mathbb{P}_{x}\left(\int_{[L^{i}+\delta_{m},x)}l^{y}_{\eta_{L^{i}-\delta_{m+1}}}\lambda^{i}_{n_{k}}(dy)<E_{i}\right)$
$\displaystyle\leq$
$\displaystyle\lim_{k\to\infty}\mathbb{P}_{x}\left(\left(\inf_{y\in[L^{i}+\delta_{m},x]}l^{y}_{\eta_{L^{i}-\delta_{m+1}}}\right)\lambda^{i}_{n_{k}}([L^{i}+\delta_{m},x))\right)=0,$

once again contradicting the choice of $L^{i}$.

(iic) Since the following arguments do not depend on
$(n_{k})_{k\in\mathbb{N}}$, we assume we assume
$(n_{k})_{k\in\mathbb{N}}=(n)_{n\in\mathbb{N}}$ w.l.o.g. By [3, Korollar 31.3]
for an open or closed interval $\chi\subset\mathbb{R}$ and each $c>0$ the set
$\\{\mu\in\operatorname{RM}(\chi):\mu(\chi)\leq c\\}$ is vaguely compact. Thus
by (iib) the sequence $(\tilde{\lambda}^{i}_{n,m})_{n\geq N_{m}}$ has a
subsequence $(\tilde{\lambda}^{i}_{n^{1}_{k},1})_{k\in\mathbb{N}}$ with
$n^{1}_{1}\geq N_{1}$ that converges vaguely to a limit
$\tilde{\lambda}^{i}_{1}\in\operatorname{RM}((L^{i},R^{i}))$. Now in the same
way, for each $m\geq 2$ we iteratively choose subsequences
$(\tilde{\lambda}^{i}_{n^{m}_{k},m})_{k\in\mathbb{N}}$ of
$(\tilde{\lambda}^{i}_{n^{m-1}_{k},m-1})_{k\in\mathbb{N}}$ with $n^{m}_{1}\geq
N_{m}$ that converge vaguely to limit points
$\tilde{\lambda}^{i}_{m}\in\operatorname{RM}((L^{i},R^{i}))$.

Next we show
$\tilde{\lambda}^{i}_{m}|_{[L^{i}+\delta_{j},R^{i}-\delta_{j}]}=\tilde{\lambda}^{i}_{m^{\prime}}|_{[L^{i}+\delta_{j},R^{i}-\delta_{j}]}$
for all $m^{\prime}\geq m>j\in\mathbb{N}$ and $n\geq N$. For that let
$f:[L^{i}+\delta_{j},R^{i}-\delta_{j}]\to\mathbb{R}$ be continuous and
bounded. $f$ can be extended to a continuous bounded function
$\overline{f}:(L^{i},R^{i})\to\mathbb{R}$ such that
$\overline{f}|_{(L^{i},L^{i}+\delta_{j+1})\cup(R^{i}-\delta_{j+1},R^{i})}\equiv
0$. By definition
$\lambda^{i}_{n,m}|_{[L^{i}+\delta_{m},R^{i}-\delta_{m}]}=\lambda^{i}_{n,m^{\prime}}|_{[L^{i}+\delta_{m},R^{i}-\delta_{m}]}$
for all $m\in\mathbb{N}$ and all $n\geq N$. From
$\operatorname{supp}(f)\subset[L^{i}+\delta_{m},R^{i}-\delta_{m}]$ we infer

$\displaystyle\int_{(L^{i},R^{i})}\overline{f}d\tilde{\lambda}^{i}_{m}=\lim_{k\to\infty}\int_{(L^{i},R^{i})}\overline{f}d\lambda^{i}_{n^{m^{\prime}}_{k},m}=\lim_{k\to\infty}\int_{(L^{i},R^{i})}\overline{f}d\lambda^{i}_{n^{m^{\prime}}_{k},m^{\prime}}=\int_{(L^{i},R^{i})}\overline{f}d\tilde{\lambda}^{i}_{m^{\prime}}$

which proves the above claim. With that we are able to consistently define the
measure $\tilde{\lambda}^{i}\in\operatorname{RM}((L^{i},R^{i}))$ via

$\displaystyle\tilde{\lambda}^{i}(K):=\tilde{\lambda}^{i}_{j}(K)$

for any compact $K\subset(L^{i},R^{i})$, where $j\geq 2$ is chosen such that
$K\subset[L^{i}+\delta_{j-1},R^{i}-\delta_{j-1}]$. Now we show
$\lambda^{i}_{n^{m}_{m},m}\to\lambda^{i}$ vaguely for $m\to\infty$. For this,
let $f\in\operatorname{C}_{\operatorname{cpt}}((L^{i},R^{i}))$, i.e., a
continuous function with compact support. We choose $j\in\mathbb{N}$ such that
$\operatorname{supp}(f)\subset[L^{i}+\delta_{j},R^{i}-\delta_{j}]$. Now as
above and by construction of $\lambda^{i}$ we obtain

$\displaystyle\lim_{m\to\infty}\int_{(L^{i},R^{i})}fd\lambda^{i}_{n^{m}_{m},m}=\lim_{m\to\infty}\int_{(L^{i},R^{i})}fd\lambda^{i}_{n^{m}_{m},j+1}=\int_{(L^{i},R^{i})}fd\tilde{\lambda}^{i}_{j+1}=\int_{(L^{i},R^{i})}fd\lambda^{i}$

as claimed.

(iii) For (iiia) and (iiic) there is nothing to prove and for (iiib) we find
the subsequence $(n_{k})_{k\in\mathbb{N}}$ by (i).

(iv) In case of (iiia) by (i) we find $(n_{k})_{k\in\mathbb{N}}$ and
$(n_{k}^{\prime})_{k\in\mathbb{N}}$ such that
$\tau^{1}_{n_{k}}\stackrel{{\scriptstyle k\to\infty}}{{\longrightarrow}}0$ and
$\tau^{2}_{n_{k}^{\prime}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0$ $\mathbb{P}_{x}$-a.s. This implies
$X_{\tau^{1}_{n_{k}}}\stackrel{{\scriptstyle k\to\infty}}{{\longrightarrow}}x$
and $X_{\tau^{2}_{n_{k}^{\prime}}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}x$ $\mathbb{P}_{x}$-a.s. By uniqueness of weak
limits we obtain
$\mu_{1}=\delta_{0},\mu_{2}=\delta_{x},\mu_{3}=\delta_{0},\mu_{4}=\delta_{x}$,
i.e. $\mu=\delta_{0}\otimes\delta_{x}\otimes_{0}\otimes\delta_{x}$. Thus
$(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})\to\mu$ in
distribution already yields
$(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})\to(0,x,0,x)$ in
probability. Thus there is $(n_{k}^{\prime\prime})_{k\in\mathbb{N}}$ such that
$(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})\to(0,x,0,x)$
$\mathbb{P}_{x}$-a.s. In the cases (iiib) and (iiic) it is sufficient to
choose $(n_{k_{m}})_{m\in\mathbb{N}}$. ∎

######  Lemma 3.10.

Let $(n_{k})_{k\in\mathbb{N}}$ be the sequence provided by Lemma 3.9 (iv). We
consider $i\in\\{1,2\\}$ such that
$\tilde{\lambda}^{i}_{n_{k},k}\stackrel{{\scriptstyle
m\to\infty}}{{\longrightarrow}}\lambda^{i}$ vaguely on
$\operatorname{RM}((L^{i},R^{i}))$ with $L^{i}<x<R^{i}$.

  1. (i)

There is a $\mathbb{P}_{x}$-nullset $N_{1}$ such that for all $t\geq 0$ and
all $\omega\in\\{t<\tau^{D^{i}}\\}\setminus N_{1}$

$\displaystyle
A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t}(\omega)\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}A^{D^{i},\lambda^{i}}_{t}(\omega)\leq
A^{D^{i},\lambda^{i}}_{T}(\omega)<\infty.$

  2. (ii)

We have $A^{D^{i},\lambda^{i}}_{\tau^{i}+\varepsilon}>E_{i}$
$\mathbb{P}_{x}$-a.s. for every $\varepsilon>0$.

  3. (iii)

There is a $\mathbb{P}_{x}$-nullset $N_{2}$ such that for all $t\geq 0$ and
all $\omega\in\\{t>\tau^{D^{i}}\\}\setminus N_{2}$

$\displaystyle
A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t}(\omega)\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}\infty.$

  4. (iv)

We have $\tau^{i}_{n_{k}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}\tau^{i}$ and
$X_{\tau^{i}_{n_{k}}}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}X_{\tau^{i}}$ $\mathbb{P}_{x}$-a.s.

###### Proof.

(i) Let $N_{1}^{\prime}$ be a nullset such that
$\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)\stackrel{{\scriptstyle\varepsilon\searrow
0}}{{\longrightarrow}}\tau^{(L^{i},R^{i})}(\omega)=\tau^{D^{i}}(\omega)$. This
yields
$A^{D^{i},\lambda^{i}}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)\stackrel{{\scriptstyle\varepsilon\searrow
0}}{{\longrightarrow}}A^{D^{i},\lambda^{i}}_{t}(\omega)$ and
$A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)\stackrel{{\scriptstyle\varepsilon\searrow
0}}{{\longrightarrow}}A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t}(\omega)$
uniformly in $k\in\mathbb{N}$ for each $\omega\in\Omega\setminus N_{2}$ and
all $t<\tau^{D^{i}}(\omega)$, which justifies the interchanging of limits in
the calculation to follow. In the next line we will use that for $D\subset I$
open and $\lambda\in\operatorname{RM}(D)$ by definition we have
$A^{D,\lambda}_{t}(\omega)=\int l^{y}_{t}(\omega)\lambda(dy)$ for
$t<\tau^{D}(\omega)$. Then we invoke the fact that
$\lambda^{i}_{n_{k}}|_{[L^{i}+\varepsilon,R^{i}-\varepsilon]}=\tilde{\lambda}^{i}_{n_{k},k}|_{[L^{i}+\varepsilon,R^{i}-\varepsilon]}$
for sufficiently large $k$ by definition. To find the limit in $k$, let
$l^{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}}(\omega):(L^{i},R^{i})\to\mathbb{R},y\mapsto
l^{y}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}}(\omega)$ and note
that $l^{y}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}}(\omega)=0$
for all $y\not\in(L^{i}+\varepsilon,R^{i}-\varepsilon)$ and all
$\omega\in\Omega$. Thus there is a nullset $N_{1}^{\prime\prime}$ such that
$l^{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}}(\omega)\in\operatorname{C}_{\operatorname{cpt}}((L^{i},R^{i}))$
for all $\omega\in\Omega\setminus N_{1}^{\prime\prime}$ and all
$t<\tau^{D^{i}}(\omega)$, so we can use vague convergence. Proceeding as
explained yields

$\displaystyle\lim_{k\to\infty}A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t}(\omega)=\lim_{k\to\infty}\lim_{\varepsilon\searrow
0}A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)$
$\displaystyle=$ $\displaystyle\lim_{\varepsilon\searrow
0}\lim_{k\to\infty}A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)=\lim_{\varepsilon\searrow
0}\lim_{k\to\infty}\int_{D^{i}_{n_{k}}}l^{y}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)d\lambda^{i}_{n_{k}}$
$\displaystyle=$ $\displaystyle\lim_{\varepsilon\searrow
0}\lim_{k\to\infty}\int_{D^{i}}l^{y}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)d\tilde{\lambda}^{i}_{n_{k},k}=\lim_{\varepsilon\searrow
0}\int_{D^{i}}l^{y}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)d\lambda^{i}$
$\displaystyle=$ $\displaystyle\lim_{\varepsilon\searrow
0}A^{D^{i},\lambda^{i}}_{t\wedge\tau^{(L^{i}+\varepsilon,R^{i}-\varepsilon)}(\omega)}(\omega)=A^{D^{i},\lambda^{i}}_{t}(\omega)$

for all $\omega\in\Omega\setminus N_{1}$ with $N_{1}:=N_{1}^{\prime}\cup
N_{1}^{\prime\prime}$.

(ii) If $A^{D^{i},\lambda^{i}}_{\tau^{i}}=\infty$ the claim is trivial, so we
are left with the set $\\{A^{D^{i},\lambda^{i}}_{\tau^{i}}<\infty\\}$. First
note that
$\\{A^{D^{i},\lambda^{i}}_{\tau^{i}}<\infty\\}=\\{A^{D^{i},\lambda^{i}}_{\tau^{i}}=E_{i}\\}$
by continuity of $[0,\tau^{D^{i}})\ni t\mapsto A^{D^{i},\lambda^{i}}_{t}$ and
$\tau^{D^{i}}=\inf\\{t\geq 0:A^{D^{i},\lambda^{i}}_{t}=\infty\\}$. Thus
$X_{\tau^{i}}\in D^{i}$. Next we show
$X_{\tau^{i}}\in\operatorname{supp}(\lambda^{i})$. Fix some
$\omega\in\\{X_{\tau^{i}}\not\in\operatorname{supp}(\lambda^{i})\\}$. Now,
since $\operatorname{supp}(\lambda^{i})$ is closed, there is some
$\varepsilon^{\prime}>0$ such that
$X_{\tau^{i}-t}(\omega)\not\in\operatorname{supp}(\lambda^{i})$ for all
$t\in[0,\varepsilon^{\prime}]$. This means
$\theta_{\tau^{i}-\varepsilon^{\prime}}\circ
l^{y}_{\varepsilon^{\prime}}(\omega)=0$ for all
$y\in\operatorname{supp}(\lambda^{i})$. This yields

$\displaystyle
E_{i}(\omega)=A^{D^{i},\lambda^{i}}_{\tau^{i}(\omega)}(\omega)=\int
l^{y}_{\tau^{i}(\omega)}(\omega)\lambda^{i}(dy)=\int
l^{y}_{\tau^{i}(\omega)-\varepsilon^{\prime}}(\omega)+\theta_{\tau^{i}-\varepsilon^{\prime}}\circ
l^{y}_{\varepsilon^{\prime}}(\omega)\lambda^{i}(dy)$ $\displaystyle=$
$\displaystyle\int
l^{y}_{\tau^{i}(\omega)-\varepsilon^{\prime}}\lambda^{i}(dy)+\int\theta_{\tau^{i}-\varepsilon^{\prime}}\circ
l^{y}_{\varepsilon^{\prime}}(\omega)\lambda^{i}(dy)=\int
l^{y}_{\tau^{i}(\omega)-\varepsilon^{\prime}}\lambda^{i}(dy)=A^{D^{i},\lambda^{i}}_{\tau^{i}(\omega)-\varepsilon^{\prime}}(\omega),$

which contradicts $\tau^{i}=\inf\\{t\geq 0:A^{D^{i},\lambda^{i}}_{t}\geq
E_{i}\\}$. Since $y\mapsto\theta_{\tau^{i}}\circ l^{y}_{\varepsilon}$ is
$\mathbb{P}_{x}$-a.s. bounded away from $0$ in every neighborhood of
$X_{\tau^{i}}\in\operatorname{supp}(\lambda^{i})$ and $\lambda^{i}$ puts
positive weight on a neighborhood of very such point, we get

$\displaystyle A^{D^{i},\lambda^{i}}_{\tau^{i}+\varepsilon}\geq\int
l^{y}_{\tau^{i}+\varepsilon}\lambda^{i}(dy)=\int
l^{y}_{\tau^{i}}\lambda^{i}(dy)+\int\inf\theta_{\tau^{i}}\circ
l^{y}_{\varepsilon}d\lambda^{i}(dy)>\int
l^{y}_{\tau^{i}}\lambda^{i}(dy)=A^{D^{i},\lambda^{i}}_{\tau^{i}}=E_{i}$

$\mathbb{P}_{x}$-a.s. as claimed.

(iii) W.l.o.g. we show the claim for $\mathbb{P}_{x}$-a.a.
$\\{t>\tau^{D^{i}}\\}\cap\\{\eta_{L^{i}}<\eta_{R^{i}}\\}$. The set
$\\{t>\tau^{D^{i}}\\}\cap\\{\eta_{L^{i}}<\eta_{R^{i}}\\}$ can be treated
analogously. For every $\varepsilon>0$ we have

$\displaystyle
c_{k}:=\max\left\\{|L^{i}_{n_{k}}-L^{i}|,\frac{1}{\lambda^{i}_{n_{k}}((L^{i}-\varepsilon,L^{i}+\varepsilon)\cap
D_{n_{k}}^{i})}\right\\}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0.$

If this were not the case we could find a subsequence
$(c_{k_{j}})_{j\in\mathbb{N}}$ that is bounded away from 0. Now we could
repeat the arguments from Lemma 3.7 for the subsequence
$(\lambda^{i}_{n_{k_{j}}})_{j\in\mathbb{N}}$ in place of the original sequence
$(\lambda^{i}_{n})_{n\in\mathbb{N}}$ to reach a contradiction. Keep in mind,
that for this argument $L^{i}$ would remain the same as it only depends on
$\mu$ (which is unchanged when passing on to a subsequence), whereas
$\tilde{L}^{i}$ would have to be replaced by
$\limsup_{j\to\infty}L^{i}_{n_{k_{j}}}<L^{i}$.

Now we choose some nullset $N_{2}^{\prime}$ such that for all
$\omega\in(\\{t>\tau^{D^{i}}\\}\cap\\{\eta_{L^{i}}<\eta_{R^{i}}\\})\setminus
N_{2}^{\prime}$ there is some $\varepsilon>0$ with
$\inf_{y\in(L^{i}-\varepsilon,L^{i}+\varepsilon)}\theta_{\tau^{D^{i}}}\circ
l^{y}_{t-\tau^{D^{i}}(\omega)}(\omega)>0$. Additionally, we choose a nullset
$N_{2}^{\prime\prime}$ such that $\eta_{y}(\omega)\to\eta_{L^{i}}(\omega)$ for
all $\omega\in\Omega\setminus N_{2}^{\prime\prime}$ and set
${}_{2}:=N_{2}^{\prime}\cup N_{2}^{\prime\prime}$. Now for all
$\omega\in(\\{t>\tau^{D^{i}}\\}\cap\\{\eta_{L^{i}}<\eta_{R^{i}}\\})\setminus
N_{2}(\supset\\{\tau^{D^{i}}=\eta_{L^{i}}\\})$ there is an $\varepsilon>0$
such that

$\displaystyle\max\left\\{(\tau^{D^{i}_{n_{k}}}(\omega)-\tau^{D^{i}}(\omega))_{+},\frac{1}{\int\theta_{\tau^{D^{i}}}\circ
l^{y}_{t-\tau^{D^{i}}(\omega)}(\omega)d\lambda^{i}_{n_{k}}}\right\\}$
$\displaystyle\leq$
$\displaystyle\max\left\\{|\eta_{L^{i}_{n_{k}}}(\omega)-\eta_{L^{i}}(\omega)|,\frac{1}{\inf_{y\in(L^{i}-\varepsilon,L^{i}+\varepsilon)}\theta_{\tau^{D^{i}}}\circ
l^{y}_{t-\tau^{D^{i}}(\omega)}(\omega)\lambda^{i}_{n_{k}}((L^{i}-\varepsilon,L^{i}+\varepsilon)\cap
D^{i}_{n_{k}})}\right\\}$ $\displaystyle\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}0.$

This means, given any $C>0$, putting these things together there is some
$K\in\mathbb{N}$ such that
$(\tau^{D^{i}_{n_{k}}}(\omega)-\tau^{D^{i}}(\omega))_{+}<t-\tau^{D^{i}}(\omega)$
(i.e. $t\geq\tau^{D^{i}_{n_{k}}}(\omega)$) or $\int\theta_{\tau^{D^{i}}}\circ
l^{y}_{t-\tau^{D^{i}}(\omega)}(\omega)d\lambda^{i}_{n_{k}}\geq C$ for all
$k\geq K$. Thus, for the same $\omega$ and $k\geq K$

$\displaystyle A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{t}(\omega)=\int
l^{y}_{t}(\omega)\lambda^{i}_{n_{k}}(dy)+\infty\mathds{1}_{\\{{t\geq\tau^{D^{i}_{n_{k}}}(\omega)}\\}}$
$\displaystyle=$ $\displaystyle\int\theta_{\tau^{D^{i}}}\circ
l^{y}_{t-\tau^{D^{i}}(\omega)}(\omega)d\lambda^{i}_{n_{k}}+\int
l^{y}_{\tau^{D^{i}(\omega)}}(\omega)\lambda^{i}_{n_{k}}(dy)+\infty\mathds{1}_{\\{{t\geq\tau^{D^{i}_{n_{k}}}(\omega)}\\}}$
$\displaystyle\geq$ $\displaystyle\int
l^{y}_{\tau^{D^{i}(\omega)}}(\omega)\lambda^{i}_{n_{k}}(dy)+\infty\mathds{1}_{\\{{t\geq\tau^{D^{i}_{n_{k}}}(\omega)}\\}}\geq
C.$

(iv) Let $\varepsilon>0$. By definition,
$A^{D^{i},\lambda^{i}}_{(\tau^{i}(\omega)-\varepsilon)\vee
0}(\omega)<E_{i}(\omega)$ for all $\omega\in\Omega\setminus N^{\prime}$ with a
$\mathbb{P}_{x}$-nullset $N^{\prime}$. By (i),
$A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{\tau^{i}(\omega)-\varepsilon}<E_{i}(\omega)$
for all $\omega\in\Omega\setminus(N^{\prime}\cup N_{1})$ and sufficiently
large $k\in\mathbb{N}$, so
$\tau^{i}_{n_{k}}(\omega)>\tau^{i}(\omega)-\varepsilon$ for these $k$ and
$\omega$. By (ii), there is a $\mathbb{P}_{x}$-nullset $N^{\prime\prime}$ such
that
$A^{D^{i},\lambda^{i}}_{\tau^{i}(\omega)+\varepsilon}(\omega)>E_{i}(\omega)$
for all $\omega\in\Omega\setminus N^{\prime\prime}$. By (i)
$A^{D^{i}_{n_{k}},\lambda^{i}_{n_{k}}}_{\tau^{i}(\omega)-\varepsilon}>E_{i}(\omega)$
for all
$\omega\in\\{\tau^{i}+\varepsilon<\tau^{D^{i}}\\}\setminus(N^{\prime\prime}\cup
N_{1})$ and sufficiently large $k\in\mathbb{N}$, so
$\tau^{i}_{n_{k}}(\omega)\leq\tau^{i}(\omega)+\varepsilon$ for these $k$ and
$\omega$. By (iii) we have the same conclusion for all
$\omega\in\\{\tau^{i}+\varepsilon>\tau^{D^{i}}\\}\setminus(N^{\prime\prime}\cup
N_{2})$. If $\omega\in\\{\tau^{i}+\varepsilon=\tau^{D^{i}}\\}$ we replace
$\varepsilon$ by any $\varepsilon^{\prime}<\varepsilon$. Sending
$\varepsilon\searrow 0$ along any fixed sequence gives
$\tau^{i}_{n_{k}}\to\tau^{i}$ outside the union of all the countably many
exception sets associated with the chosen sequence. ∎

###### Proof.

(of Proposition 3.5)
As described above, weak compactness holds by Prokhorov’s theorem. Relative
closeness holds by putting the previous pieces together: By Lemma 3.9 (iv) and
Lemma 3.10 (iv) there is an indexing sequence $(n_{k})_{k\in\mathbb{N}}$ such
that $(\tau^{1}_{n_{k}},\tau^{2}_{n_{k}})\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}(\tau^{1},\tau^{2})$ $\mathbb{P}_{x}$-a.s. By
path continuity of $X$ we obtain
$(\tau^{1}_{n_{k}},X_{\tau^{1}_{n_{k}}},\tau^{2}_{n_{k}},X_{\tau^{2}_{n_{k}}})\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})$
$\mathbb{P}_{x}$-a.s. In particular,
$\mathbb{P}_{x}^{(\tau^{1}_{n_{k}},X_{\tau^{1}_{n_{k}}},\tau^{2}_{n_{k}},X_{\tau^{2}_{n_{k}}})}\stackrel{{\scriptstyle
k\to\infty}}{{\longrightarrow}}\mathbb{P}_{x}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}$.
By uniqueness of weak limits we infer
$\mu=\mathbb{P}_{x}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}$. ∎

###  3.3 $\mathcal{M}^{1}$-product topology

The $\mathcal{M}^{1}_{x}$-topology already has good properties. However, since
we are interested in Markov-perfect equilibria, the consideration for a fixed
$x$ is not sufficient. We must therefore find a suitable notions of
convergence that takes into account all possible initial points at once.

######  Definition 3.11.

($\mathcal{M}^{1}$-product topology)
Let $U\subset I$ open. Let
$\kappa:\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})\to\bigtimes_{x\in
U\cap\mathbb{Q}}\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$ be given
by

$\displaystyle(\tau^{D_{1},\lambda_{1}},\tau^{D_{2},\lambda_{2}})\mapsto(\kappa_{x}(\tau^{D_{1},\lambda_{1}},\tau^{D_{2},\lambda_{2}}))_{x\in
U\cap\mathbb{Q}}.$

We define the $\mathcal{M}^{1}$-product topology as the pullback topology on
$\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$ under the mapping
$\kappa$.

The consideration of rational starting points has the technical background
that we thus obtain a countable product space. The following two results
ensure that this is not a significant limitation.

######  Lemma 3.12.

Let $U\subset I$ open with compact closure $\overline{U}\subset I$ (closure
taken in $\mathbb{R}$),
$((\tau^{1}_{n},\tau^{2}_{n}))_{n\in\mathbb{N}}=((\tau^{D^{1}_{n},\lambda^{1}_{n}},\tau^{D^{2}_{n},\lambda^{2}_{n}}))_{n\in\mathbb{N}}$
a sequence in $\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$,
$(\tau^{1},\tau^{2})=(\tau^{D^{1},\lambda^{1}},\tau^{D^{2},\lambda^{2}})\in\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$
such that $(\tau^{1}_{n},\tau^{2}_{n})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}(\tau^{1},\tau^{2})$ in the
$\mathcal{M}^{1}$-product topology and let $f:[0,\infty]\times
I\times[0,\infty]\times I\to\mathbb{R}$ be bounded and uniformly continuous.
Then, the sequence $\varphi=(\varphi_{n})_{n\in\mathbb{N}}$ of mappings

$\displaystyle\varphi_{n}:\,U\to\mathbb{R},\;\;x\mapsto\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]=\int
fd\mathbb{P}_{x}^{(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})}$

is equicontinuous.

###### Proof.

Let $x\in U$, $\varepsilon>0$ and $\delta>0$ such that
$|f(t_{1},x_{1},t_{2},x_{2})-f(s_{1},y_{1},s_{2},y_{2})|<\varepsilon$ for all
$(t_{1},x_{1},t_{2},x_{2}),(s_{1},y_{1},s_{2},y_{2})\in[0,\infty]\times
I\times[0,\infty]\times I$ with
$|t_{1}-s_{1}|,|t_{2}-s_{2}|,|x_{1}-y_{1}|,|x_{2}-y_{2}|\leq\delta$. Set
$B=(x-\frac{\delta}{2},x+\frac{\delta}{2})$ and $S:=\sup|f|$.

By our standing assumptions on $I$, the point $x$ is non-singular in the sense
of [33, Section 3.4.]. By [33, Section 4.4.]
$\mathbb{P}_{y}(\eta_{x}<\infty)\geq\mathbb{P}_{y}(\eta_{x}<\tau^{B})\stackrel{{\scriptstyle
y\to x}}{{\longrightarrow}}1$. Thus by [33, Section 3.3.(10c)]
$\mathbb{P}_{y}(\eta_{x}<\delta)\stackrel{{\scriptstyle y\to
x}}{{\longrightarrow}}1$. For all $\tau^{D,\lambda}\in\mathcal{Z}^{U}$ with
$x\not\in D$ this yields

$\displaystyle\mathbb{P}_{y}(\tau^{D,\lambda}\leq\tau^{B}\wedge\delta)\geq\mathbb{P}_{y}(\eta_{x}<\tau^{B}\wedge\delta)\stackrel{{\scriptstyle
y\to x}}{{\longrightarrow}}1.$ (9)

In case $x\not\in D^{1}\cup D^{2}$ we infer

$\displaystyle|\mathbb{E}_{x}[f(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})]-\mathbb{E}_{y}[f(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})]|$
$\displaystyle\leq$
$\displaystyle|f(0,x,0,x)-\mathbb{E}_{y}[\mathds{1}_{\\{{\tau^{1}\vee\tau^{2}\leq\tau^{B}\wedge\delta}\\}}f(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})]|+S\mathbb{P}_{y}(\tau^{1}\vee\tau^{2}>\tau^{B}\wedge\delta)$
$\displaystyle\leq$
$\displaystyle|f(0,x,0,x)|(1-\mathbb{P}_{y}(\tau^{1}\vee\tau^{2}>\tau^{B}\wedge\delta))+\varepsilon+S\mathbb{P}_{y}(\tau^{1}\vee\tau^{2}>\tau^{B}\wedge\delta)\stackrel{{\scriptstyle
y\to x}}{{\longrightarrow}}\varepsilon$

independent from $\tau^{1},\tau^{2}$. I.e. provided $x\not\in D^{1}_{n}\cup
D^{2}_{n}$ the same estimate can be repeated for $\tau^{1}_{n},\tau^{2}_{n}$
for every $n$.

Now assume $x\in D^{1}\cap D^{2}$. By Lemma 3.9 (iib) we may assume that there
is neighborhood $B^{\prime}\subset D^{1}\cap D^{2}$ of $x$ such that
$B^{\prime}\subset\bigcap_{n\in\mathbb{N}}(D^{1}_{n}\cap D^{2}_{n})$ and
$\Lambda:=\sup_{i\in\\{1,2\\},n\in\mathbb{N}}\lambda^{i}_{n}(B^{\prime})<\infty$.
W.l.o.g. let $B^{\prime}=B$. Now for $i=1,2$

$\displaystyle
A^{\tau^{i}_{n}}_{\eta_{x}\wedge\tau^{B}\wedge\delta}=A^{D^{i}_{n},\lambda^{i}_{n}}_{\eta_{x}\wedge\tau^{B}\wedge\delta}=\int
l^{y}_{\eta_{x}\wedge\tau^{B}\wedge\delta}d\lambda^{i}_{n}(y)\leq\int\eta_{x}\wedge\tau^{B}\wedge\delta
d\lambda^{i}_{n}(y)\leq\Lambda(\eta_{x}\wedge\tau^{B}\wedge\delta).$

and so

$\displaystyle\mathbb{P}_{y}(\tau^{1}_{n}\wedge\tau^{2}_{n}>\eta_{x}\wedge\tau^{B}\wedge\delta)=\mathbb{P}_{y}(A^{\tau^{1}_{n}}_{\eta_{x}\wedge\tau^{B}\wedge\delta}<E_{1}\,\text{
and }\,A^{\tau^{2}_{n}}_{\eta_{x}\wedge\tau^{B}\wedge\delta}<E_{2})$
$\displaystyle\geq$
$\displaystyle\mathbb{P}_{y}(\Lambda(\eta_{x}\wedge\tau^{B}\wedge\delta)<E_{1}\wedge
E_{2})=\mathbb{E}_{y}(e^{-2\Lambda(\eta_{x}\wedge\tau^{B}\wedge\delta)})\stackrel{{\scriptstyle
y\to x}}{{\to}}1$ (10)

independent from $n$. To abbreviate, let
$\sigma:=\eta_{x}\wedge\tau^{B}\wedge\delta$. Using that $\tau^{1}_{n}$ and
$\tau^{2}_{n}$ are Markovian, we obtain

$\displaystyle|\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]-\mathbb{E}_{y}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]|$
$\displaystyle\leq$
$\displaystyle|\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]-\mathbb{E}_{y}[\mathds{1}_{\\{{\tau^{1}_{n}\wedge\tau^{2}_{n}>\sigma}\\}}f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]|+S\mathbb{E}_{y}(\mathds{1}_{\\{{\tau^{1}_{n}\wedge\tau^{2}_{n}\leq\sigma}\\}})$
$\displaystyle\leq$
$\displaystyle|\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]-\mathbb{E}_{y}[\mathds{1}_{\\{{\tau^{1}_{n}\wedge\tau^{2}_{n}>\sigma}\\}}f(\theta_{\sigma}\circ\tau^{1}_{n}+\sigma,X_{\theta_{\sigma}\circ\tau^{1}_{n}+\sigma},\theta_{\sigma}\circ\tau^{2}_{n}+\sigma,X_{\theta_{\sigma}\circ\tau^{2}_{n}+\sigma})]|$
$\displaystyle+S(1-\mathbb{E}_{y}(e^{-2\Lambda\sigma}))$ $\displaystyle\leq$
$\displaystyle|\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]-\mathbb{E}_{y}[\mathds{1}_{\\{{\tau^{1}_{n}\wedge\tau^{2}_{n}>\sigma}\\}}f(\theta_{\sigma}\circ\tau^{1}_{n},X_{\theta_{\sigma}\circ\tau^{1}_{n}+\sigma},\theta_{\sigma}\circ\tau^{2}_{n},X_{\theta_{\sigma}\circ\tau^{2}_{n}+\sigma})]|+\varepsilon$
$\displaystyle+S(1-\mathbb{E}_{y}(e^{-2\Lambda\sigma}))$ $\displaystyle=$
$\displaystyle|\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]-\mathbb{E}_{y}[\mathds{1}_{\\{{\tau^{1}_{n}\wedge\tau^{2}_{n}>\sigma}\\}}\mathbb{E}_{X_{\sigma}}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]]|+\varepsilon+S(1-\mathbb{E}_{y}(e^{-2\Lambda\sigma}))$
$\displaystyle=$
$\displaystyle|\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]-\mathbb{E}_{y}[\mathds{1}_{\\{{\tau^{1}_{n}\wedge\tau^{2}_{n}>\sigma}\\}}\mathds{1}_{\\{{\eta_{x}=\sigma}\\}}]\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]|+S\mathbb{P}_{y}(\eta_{x}>\sigma)+\varepsilon$
$\displaystyle+S(1-\mathbb{E}_{y}(e^{-2\Lambda\sigma}))$ $\displaystyle\leq$
$\displaystyle|\mathbb{E}_{x}[f(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})]|(1-\mathbb{E}_{y}[\mathds{1}_{\\{{\eta_{x}=\sigma}\\}}e^{-2\Lambda\sigma}])+S\mathbb{P}_{y}(\eta_{x}>\sigma)+\varepsilon+S(1-\mathbb{E}_{y}(e^{-2\Lambda\sigma}))$
$\displaystyle\stackrel{{\scriptstyle y\to x}}{{\to}}\varepsilon$

independent from $n$ by (9) and (10). For the cases $x\in D_{1}\setminus
D^{2}$ and $x\in D^{2}\setminus D^{1}$ we can make essentially the same
estimate combining the arguments from the cases $x\not\in D^{1}\cup D^{2}$ and
$x\in D^{1}\cap D^{2}$. Sending $\varepsilon\searrow 0$ we infer the claim. ∎

######  Proposition 3.13.

Let $U\subset I$ open with compact closure $\overline{U}\subset I$ (closure
taken in $\mathbb{R}$), $((\tau^{1}_{n},\tau^{2}_{n}))_{n\in\mathbb{N}}$ a
sequence in $\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$ and
$(\tau^{1},\tau^{2})\in\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$
such that $(\tau^{1}_{n},\tau^{2}_{n})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}(\tau^{1},\tau^{2})$ in the
$\mathcal{M}^{1}$-product topology. Then,

$\displaystyle\mathbb{P}_{x}^{(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})}\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}\mathbb{P}_{x}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}$

weakly for all $x\in I$.

###### Proof.

For $x\in U\cap\mathbb{Q}$, the claim is immediate by definition. For $x\in
I\setminus U$

$\displaystyle\mathbb{P}_{x}^{(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})}=\delta_{0}\otimes\delta_{x}\otimes\delta_{0}\otimes\delta_{x}=\mathbb{P}_{x}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}$

so the claim is once again trivial. Let $x\in U\setminus\mathbb{Q}$ and
$f:[0,\infty]\times I\times[0,\infty]\times I\to\mathbb{R}$ bounded and
uniformly continuous. For all $y\in U\cap\mathbb{Q}$ we have

$\displaystyle\bigg{|}\int
fd\mathbb{P}_{x}^{(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})}-\int
fd\mathbb{P}_{x}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}\bigg{|}$
$\displaystyle\leq$ $\displaystyle\bigg{|}\int
fd\mathbb{P}_{x}^{(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})}-\int
fd\mathbb{P}_{y}^{(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})}\bigg{|}$
$\displaystyle+\bigg{|}\int
fd\mathbb{P}_{y}^{(\tau^{1}_{n},X_{\tau^{1}_{n}},\tau^{2}_{n},X_{\tau^{2}_{n}})}-\int
fd\mathbb{P}_{y}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}\bigg{|}$
$\displaystyle+\bigg{|}\int
fd\mathbb{P}_{y}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}-\int
fd\mathbb{P}_{x}^{(\tau^{1},X_{\tau^{1}},\tau^{2},X_{\tau^{2}})}\bigg{|}.$

By the above the middle term on the right hand side converges to 0 for
$n\to\infty$. By Lemma 3.12 the first and third term converge to 0 for $y\to
x$ independent from $n$. By [46, Theorem 1.1.1] this proves the claim. ∎

We come to the main result of this section.

######  Theorem 3.14.

Let $U\subset I$ open with compact closure $\overline{U}\subset I$ (closure
taken in $\mathbb{R}$). Then,
$\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$ equipped with the
$\mathcal{M}^{1}$-product topology is a compact topological space.

######  Remark 3.15.

More explicitly, the previous theorem states that the set

$\kappa(\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2}))\subset\bigtimes_{x\in
U\cap\mathbb{Q}}\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$

is compact in the product topology on $\bigtimes_{x\in
U\cap\mathbb{Q}}\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$. Note
that this result is in fact not a direct consequence of Tychonoff’s theorem.
The point is that, for a given $x\in I$, the $\mathcal{M}^{1}_{x}$ topology
identifies all Markovian randomized times
$\tau^{D,\lambda},\tau^{D^{\prime},\lambda^{\prime}}$ with the property that
the connected components $D_{x}$ and $D_{x}^{\prime}$ of $x$ in $D$,
$D^{\prime}$ respectively, coincide, as well as the generating measures
restricted to these connected components, i.e.
$\lambda|_{D_{x}}=\lambda^{\prime}|_{D_{x}^{\prime}}$. Thus, although every
sequence $(\kappa(\tau^{1}_{n},\tau^{2}_{n}))_{n\in\mathbb{N}}$ has a
subsequence that converges to a family of measures $(\mu_{x})_{x\in
U\cap\mathbb{Q}}$, it is not immediately obvious that there is a single pair
of Markovian randomized times $(\tau^{1},\tau^{2})$ such that
$\kappa(\tau^{1},\tau^{2})=(\mu_{x})_{x\in U\cap\mathbb{Q}}$.

###### Proof.

In the proof of Proposition 3.5, we have already shown that
$\mathcal{M}^{1}_{x}(([0,\infty]\times\overline{U})^{2})$, $x\in I,$ is a
compact space. Thus, by Tychonoff’s theorem the product space $\bigtimes_{x\in
U\cap\mathbb{Q}}\mathcal{M}^{1}_{x}(([0,\infty]\times\overline{U})^{2})$ is
compact as well. This yields relative compactness of
$\kappa(\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2}))$.

As a countable product of metric spaces, $\bigtimes_{x\in
U\cap\mathbb{Q}}\mathcal{M}^{1}_{x}(([0,\infty]\times\overline{U})^{2})$ is
metric, so it suffices to show sequential closedness in order to finish the
proof of compactness. To this end, let
$((\tau^{D^{1}_{n},\lambda^{1}_{n}},\tau^{D^{2}_{n},\lambda^{2}_{n}}))_{n\in\mathbb{N}}\in(\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2}))^{\mathbb{N}}$
be a sequence of tuples of Markovian randomized times such that
$\kappa(\tau^{D^{1}_{n},\lambda^{1}_{n}},\tau^{D^{2}_{n},\lambda^{2}_{n}})$
converges in the product topology, i.e. pointwise in $x\in U\cap\mathbb{Q}$,
to a family of measures $(\mu_{x})_{x\in U\cap\mathbb{Q}}\in\bigtimes_{x\in
U\cap\mathbb{Q}}\mathcal{M}^{1}_{x}(([0,\infty]\times\overline{U})^{2})$. We
have to find
$(\tau^{1},\tau^{2}):=(\tau^{D^{1},\lambda^{1}},\tau^{D^{2},\lambda^{2}})\in\mathcal{Z}^{U}(E_{1})\times\mathcal{Z}^{U}(E_{2})$
such that $\kappa(\tau^{1},\tau^{2})=(\mu_{x})_{x\in U\cap\mathbb{Q}}$. To
abbreviate, we set
$(\tau^{1}_{n},\tau^{2}_{n}):=(\tau^{D^{1}_{n},\lambda^{1}_{n}},\tau^{D^{2}_{n},\lambda^{2}_{n}})$
for each $n\in\mathbb{N}$.

Given $x\in I\cap\mathbb{Q}$, we write

$\displaystyle L^{i}_{x}:=L^{i},$ $\displaystyle R^{i}_{x}:=R^{i},$
$\displaystyle\tilde{L}^{i}_{x}:=\tilde{L}^{i},$
$\displaystyle\tilde{R}^{i}:=\tilde{R}^{i},$ $\displaystyle
D^{i}_{x}:=(L^{i}_{x},R^{i}_{x}),$
$\displaystyle\lambda^{i}_{x}:=\lambda^{i}\in\operatorname{RM}(D^{i}_{x}),$

where $L^{i},R^{i},\tilde{L}^{i},\tilde{R}^{i}$ are the objects defined in
Lemma 3.6 corresponding to $x$ and $\lambda^{i}$ being the corresponding
measure constructed in Lemma 3.9.

By definition

$\displaystyle\mathbb{P}_{x}^{(\tau^{D^{1}_{x},\lambda^{1}_{x}},X_{\tau^{D^{1}_{x},\lambda^{1}_{x}}},\tau^{D^{2}_{x},\lambda^{2}_{x}},X_{\tau^{D^{2}_{x},\lambda^{2}_{x}}})}=\mathbb{P}_{x}^{(\tau^{D^{1},\lambda^{1}},X_{\tau^{D^{1},\lambda^{1}}},\tau^{D^{2},\lambda^{2}},X_{\tau^{D^{2},\lambda^{2}}})}$
(11)

for all open $D^{1}_{x}\subset D^{1}$, $D^{2}_{x}\subset D^{2}$,
$\lambda^{1}\in\operatorname{RM}(D^{1}),\lambda^{2}\in\operatorname{RM}(D^{2})$
such that $L^{1}_{x},R^{1}_{x}\not\in D^{1}$, $L^{2},R^{2}\not\in D^{2}$ and
$\lambda^{1}|_{D^{1}_{x}}=\lambda^{1}_{x}$,
$\lambda^{2}|_{D^{2}_{x}}=\lambda^{1}_{x}$.

We now define our candidate for the limit point via

$\displaystyle D^{i}:=\bigcup_{x\in U\cap\mathbb{Q}}D^{i}_{x},$
$\displaystyle\lambda^{i}|_{D^{i}_{x}}:=\lambda^{i}_{x},\quad i\in\\{1,2\\}.$
(12)

Next we will now show that $D^{i}_{x},\lambda^{i}_{x},i\in\\{1,2\\}$ do not
depend on the limiting distribution $\mu_{x}$, but only on the characteristics
of the sequence $((\tau^{1}_{n},\tau^{2}_{n}))_{n\in\mathbb{N}}$. For that we
define

$\displaystyle\hat{L}^{i}_{x}:=\sup\left\\{y\leq
x:\lim_{n\to\infty}\lambda^{i}_{n}((y-\varepsilon,y+\varepsilon))=\infty\,\,\forall\varepsilon>0\right\\},$
$\displaystyle\hat{R}^{i}_{x}:=\sup\left\\{y\geq
x:\lim_{n\to\infty}\lambda^{i}_{n}((y-\varepsilon,y+\varepsilon))=\infty\,\,\forall\varepsilon>0\right\\}$

and show

$\displaystyle L^{i}_{x}=\max\\{\tilde{L}^{i}_{x},\hat{L}^{i}_{x}\\},\quad
R^{i}_{x}=\min\\{\tilde{R}^{i}_{x},\hat{R}^{i}_{x}\\}.$ (13)

W.l.o.g. we only argue for the case of $L^{i}_{x}$, the one for $R^{i}_{x}$
being analogous. $L^{i}_{x}\geq\tilde{L}^{i}_{x}$ has been shown in Lemma 3.6
(8). By Lemma 3.9 (iv) and (iib), using the notation from the lemma, we get

$\displaystyle\lambda^{i}_{x}((y-\varepsilon,y+\varepsilon))\leq\liminf_{k\to\infty}\tilde{\lambda}^{i}_{n_{k},k}((y-\varepsilon,y+\varepsilon))\leq\sup_{n\geq
N_{m}}\tilde{\lambda}^{i}_{n,m}((L^{i}_{x},R^{i}_{x}))<\infty$

whenever $(y-\varepsilon,y+\varepsilon)\subset(L^{i}_{x},R^{i}_{x})$. This
implies $L^{i}_{x}\geq\hat{L}^{i}_{x}$. Finally if
$L^{i}_{x}>\tilde{L}^{i}_{x}$, then
$\lim_{n\to\infty}\lambda^{i}_{n}([L^{i}_{x}-\varepsilon,L^{i}_{x}+\varepsilon])=\infty$
for all $\varepsilon>0$ by Lemma 3.7, so $L^{i}_{x}=\hat{L}^{i}_{x}$. With
that (13) is shown.

By definition the right hand side of (13) only depends on the characteristics
of the sequence in the sense that

$\displaystyle
L^{i}_{x}=\max\\{\tilde{L}^{i}_{x},\hat{L}^{i}_{x}\\}=\sup\left\\{y\leq
x:\liminf_{n\to\infty}\infty\mathds{1}_{\\{{D^{i}_{n}\cap(y-\varepsilon,y+\varepsilon)\neq\varnothing}\\}}+\lambda^{i}_{n}((y-\varepsilon,y+\varepsilon))=\infty\,\,\forall\varepsilon>0\right\\}.$
(14)

Repeating the argument for $R^{i}_{x}$ we infer

$\displaystyle
R^{i}_{x}=\min\\{\tilde{R}^{i}_{x},\hat{R}^{i}_{x}\\}=\inf\left\\{y\geq
x:\liminf_{n\to\infty}\infty\mathds{1}_{\\{{D^{i}_{n}\cap(y-\varepsilon,y+\varepsilon)\neq\varnothing}\\}}+\lambda^{i}_{n}((y-\varepsilon,y+\varepsilon))=\infty\,\,\forall\varepsilon>0\right\\}.$

Together that yields

$\displaystyle y\in(L^{i}_{x},R^{i}_{x})$ $\displaystyle\Longrightarrow
L^{i}_{y}=L^{i}_{x},R^{i}_{y}=R^{i}_{x},$ $\displaystyle y=L^{i}_{x}$
$\displaystyle\Longrightarrow L^{i}_{y}=L^{i}_{x},R^{i}_{y}=L^{i}_{x},$
$\displaystyle y=R^{i}_{x}$ $\displaystyle\Longrightarrow
L^{i}_{y}=R^{i}_{x},R^{i}_{y}=R^{i}_{x}.$

Additionally note that $\lambda^{i}_{x}$ does only depend on
$D^{i}_{x}=(L^{i}_{x},R^{i}_{x})$. This shows that $D^{i},\lambda^{i}$ are
well defined by (12) and that this choice satisfies (11). By Proposition 3.5
and Lemma 3.9

$\displaystyle\kappa_{x}(\tau^{1}_{n},\tau^{2}_{n})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}\mathbb{P}_{x}^{(\tau^{D^{1}_{x},\lambda^{1}_{x}},X_{\tau^{D^{1}_{x},\lambda^{1}_{x}}},\tau^{D^{2}_{x},\lambda^{2}_{x}},X_{\tau^{D^{2}_{x},\lambda^{2}_{x}}})}\left(=\mathbb{P}_{x}^{(\tau^{D^{1},\lambda^{1}},X_{\tau^{D^{1},\lambda^{1}}},\tau^{D^{2},\lambda^{2}},X_{\tau^{D^{2},\lambda^{2}}})}\right)$

weakly for all $x\in U$, i.e.

$\displaystyle\kappa(\tau^{1}_{n},\tau^{2}_{n})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}\mathbb{P}_{x}^{(\tau^{D^{1},\lambda^{1}},X_{\tau^{D^{1},\lambda^{1}}},\tau^{D^{2},\lambda^{2}},X_{\tau^{D^{2},\lambda^{2}}})}$

in $\bigtimes_{x\in
U\cap\mathbb{Q}}\mathcal{M}^{1}(([0,\infty]\times\overline{U})^{2})$. ∎

##  4 Existence of Markovian equilibria in discretized games

In order to prove our Theorem 2.4, we introduce a discretized versions of the
Dynkin game under consideration. We start by establishing existence of
equilibria in the discretized game by applying a fixed point theorem based on
the $\mathcal{M}$-topology.

In order to determine the $\mathcal{M}$-continuity points of $J^{1},J^{2}$ in
Lemma 4.6 we use the structural Assumptions (A), (B). While Assumption (A) is
at the core of this lemma, Assumption (B) is there to clarify the boundary
behavior of $X$, since other behaviors would require different treatment.
Assumption (C) complements the requirements (29), (30) for the best response
mapping, when Lemma 4.6 is applied in the case described in Remark 4.7 to
ensure closedness of the graph of the best response. By working out versions
of Lemma 4.5 and Lemma 4.6 specifically for $J^{1},J^{2}$ at $\partial I$,
Assumption (C) could be generalized.

######  Definition 4.1.

For $U\subset I$, let

$\displaystyle\mathcal{T}_{U}:=\\{\tau\in\mathcal{T}:X_{\tau}\in U\\}.$

We call a pair
$(\tau_{1},\tau_{2})\in(\mathcal{T}_{U}\cup\mathcal{Z}_{U}(E_{1}))\times(\mathcal{T}_{U}\cup\mathcal{Z}_{U}(E_{2}))$
$U$-equilibrium at $x$ if

$\displaystyle J^{1}(x,\tau_{1},\tau_{2})\geq
J^{1}(x,\tau_{1}^{\prime},\tau_{2})\quad\text{and}$ (15) $\displaystyle
J^{2}(x,\tau_{1},\tau_{2})\geq J^{2}(x,\tau_{1},\tau_{2}^{\prime})$ (16)

for all $\tau_{1}^{\prime}\in\mathcal{T}_{U}\cup\mathcal{Z}_{U}(E_{1})$ and
all $\tau_{2}^{\prime}\in\mathcal{T}_{U}\cup\mathcal{Z}_{U}(E_{2})$. If
$(\tau_{1},\tau_{2})$ is an equilibrium at $x$ for all $x\in I$, we call
$(\tau_{1},\tau_{2})$ Markov perfect $U$-equilibrium or just $U$-equilibrium
for brevity.

First, we rewrite the best response functional for the players when the
opponent has chosen a Markov randomized time.

######  Lemma 4.2.

Let $\tau_{2}=\tau^{D,\lambda}\in\mathcal{Z}(E_{2})$, $A:=A^{\tau_{2}}$,
$A_{t-}:=\lim_{s\nearrow t}A_{s}$ and

$F:I\to\mathbb{R},\;x\mapsto\mathbb{E}_{x}\left[\int_{[0,\tau^{D})}e^{-A_{s}-r_{1}s}f_{1}(X_{s})dA_{s}\right].$

Then,

$\displaystyle J^{1}(x,\tau,\tau_{2})=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$

for all $\tau\in\operatorname{\mathcal{R}}(E_{1})$ and all $x\in U$. Here it
is consistent to define $F(X_{\infty})=0$ in order to reproduce the payoff 0
on $\\{\tau=\infty=\tau_{2}\\}$.

###### Proof.

We first calculate the conditional distribution of $\tau_{2}$ under
$\mathbb{P}_{x}$ given $\mathcal{F}^{X}_{\infty},\sigma(E_{1})$. For $t\geq 0$
we have

$\displaystyle\mathbb{P}_{x}(\tau_{2}>t|\mathcal{F}^{X}_{\infty},\sigma(E_{1}))=\mathbb{P}_{x}(A_{t}<E_{2}|\mathcal{F}^{X}_{\infty},\sigma(E_{1}))=e^{-A_{t}}.$

Thus
$d(-e^{-A_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}})$
(i.e. the Markov kernel
$\Omega\times\mathcal{B}([0,\infty])\ni(\omega,(a,b])\mapsto
e^{-A_{a}(\omega)}-e^{-A_{b}(\omega)}$) is a regular version of the
conditional distribution of $\tau_{2}$ given
$\mathcal{F}^{X}_{\infty},\sigma(E_{1})$. Applying a change of variables for
finite variation processes, cf. [40, p. 42], and using that $A$ only jumps in
$\tau^{D}$, we find

$\displaystyle
e^{-A_{a}(\omega)}-e^{-A_{b}(\omega)}=\int_{(a,b\wedge\tau^{D})}e^{A_{s-}}dA_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}(s)+e^{A_{\tau^{D}-}}-e^{A_{\tau^{D}}}.$

This yields

$\displaystyle
J^{1}(x,\tau,\tau_{2})=\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\leq\tau^{2}}\\}}e^{-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{2}}\\}}e^{-r_{1}\tau_{2}}g_{1}(X_{\tau_{2}})]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}[e^{-r_{1}\tau}g_{1}(X_{\tau})\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\leq\tau^{2}}\\}}|\mathcal{F}^{X}_{\infty},\sigma(E_{1})]+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau>\tau^{2}}\\}}e^{-r_{1}\tau_{2}}f_{1}(X_{\tau_{2}})|\mathcal{F}^{X}_{\infty},\sigma(E_{1})]]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}\left[e^{-r_{1}\tau}g_{1}(X_{\tau})\int_{[\tau,\infty)}d(-e^{-A_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}})+\int_{[0,\tau)}e^{-r_{1}s}f_{1}(X_{s})d(-e^{-A_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}})(s)\right]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}\bigg{[}e^{-r_{1}\tau}g_{1}(X_{\tau})e^{-A_{\tau-}}+\int_{[0,\tau\wedge\tau^{D})}e^{-r_{1}s}f_{1}(X_{s})e^{-A_{s-}}dA_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}(s)$
$\displaystyle+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})(e^{-A_{\tau^{D}-}}-e^{-A_{\tau^{D}}})\bigg{]}$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}\left[e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\int_{[0,\tau\wedge\tau^{D})}e^{-A_{s-}-r_{1}s}f_{1}(X_{s})dA_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}(s)+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})\right]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}\left[e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+F(x)-\int_{[\tau\wedge\tau^{D},\tau^{D})}e^{-A_{s}-r_{1}s}f_{1}(X_{s})dA_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}(s)+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})\right]$
$\displaystyle=$ $\displaystyle
F(x)+\mathbb{E}_{x}\bigg{[}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}\bigg{(}g_{1}(X_{\tau})+\mathbb{E}_{x}\bigg{[}\theta_{\tau}\circ\int_{[0,\tau^{D})}e^{-A_{s}-r_{1}s}f_{1}(X_{s})dA_{\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}}}(s)\bigg{|}\mathcal{F}_{\tau}\bigg{]}\bigg{)}\bigg{]}$
$\displaystyle+\mathbb{E}_{x}\big{[}\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))\big{]}$
$\displaystyle=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})].$

∎

The previous lemma guarantees that the best-response problem is a Markovian
stopping problem, so we can apply the standard theory. This only needs to be
tightened slightly in order to actually characterize all maxima in
$\mathcal{Z}$, which is necessary to formally establish the indifference
principle.

######  Lemma 4.3.

Let $U\subset I$ and consider the best response problem

$\displaystyle
V^{1}_{U}(x,\tau_{2})=\sup_{\tau\in\mathcal{T}_{U}\cup\mathcal{Z}_{U}(E_{1})}J^{1}(x,\tau,\tau_{2}),\quad
x\in U.$ (17)

  1. (i)

If $U$ is finite, then there is an optimal stopping time
$\tau^{*}\in\mathcal{Z}_{U}(E_{1})$ for the problem (17), i.e.

$\displaystyle V_{U}^{1}(x)=J^{1}(x,\tau^{*},\tau_{2})\;\;\mbox{for all $x\in
U$.}$

  2. (ii)

If $U_{1}\subseteq U_{2}\subseteq\dots\subseteq I$ are finite and
$\bigcup_{m}U_{m}$ is dense in $I$, then

$V_{I}^{1}(x)=\sup_{m\in\mathbb{N}}\sup_{\tau\in\mathcal{Z}_{U_{m}}(E_{1})}J^{1}(x,\tau,\tau_{2})\;\;\mbox{for
all $x\in U$.}$

  3. (iii)

Let $\tau_{2}=\tau^{D,\lambda}\in\mathcal{Z}(E_{2})$. If $U$ is finite,
$\tau\in\mathcal{Z}_{U}$ with $\iota_{1}(\tau)(\\{x\\})\in(0,\infty)$ for some
$x\in U\cap D$ is optimal in (17) (for the fixed $x$/for all initial states),
then all $\tau^{\prime}\in\mathcal{Z}_{U}$ with
$\iota_{1}(\tau^{\prime})(\\{y\\})=\iota_{1}(\tau)(\\{y\\})$ for all $y\in
U\setminus\\{x\\}$ are optimal for (17) (for the fixed $x$/for all initial
states).

###### Proof.

(i) From the previous lemma it is immediately clear that player 1 never stops
in $D^{c}$ if player 2 has chosen
$\tau_{2}=\tau^{D,\lambda}\in\mathcal{Z}_{U}(E_{2})$. This means that player 1
in $D^{c}$ always receives $f_{1}(x)$. In addition, player 1 should never stop
in states $y\in U\cap D$ with $g_{1}(y)-F(y)<0$. Indeed, let $y\in U\cap D$
with $g_{1}(y)-F(y)<0$ and $\mathbb{P}_{x}(X_{\tau}=y,\tau<\tau^{D})>0$. Now
set $\tilde{\tau}:=\tau+\mathds{1}_{\\{{X_{\tau}=y}\\}}\infty$. By Lemma 4.2
using that $g_{1},f_{1}\geq 0$ we get

$\displaystyle J^{1}(x,\tau,\tau_{2})=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{X_{\tau}\neq
y}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{X_{\tau}=y}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle<$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{X_{\tau}\neq
y}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle\leq$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tilde{\tau}<\tau^{D}}\\}}e^{-A_{\tilde{\tau}}-r_{1}\tilde{\tau}}(g_{1}(X_{\tilde{\tau}})-F(X_{\tilde{\tau}}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tilde{\tau}\geq\tau^{D}}\\}}(e^{-A_{\tilde{\tau}-}-r_{1}\tilde{\tau}}g_{1}(X_{\tilde{\tau}})+\mathds{1}_{\\{{\tilde{\tau}>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle=$ $\displaystyle J^{1}(x,\tilde{\tau},\tau_{2}).$

We see that in order to solve the best response problem for
$\tau_{2}=\tau^{D,\lambda}\in\mathcal{Z}_{U}(E_{2})$, player 1 faces a
standard stopping problem of the form

$\displaystyle\hat{V}_{U}(x)=\sup_{\tau\in\operatorname{\mathcal{R}}(E_{1})}\mathbb{E}_{x}\left(e^{-A_{\tau}-r_{1}\tau}h_{U}(Y_{\tau})\right),\quad
x\in I,$ (18)

$h_{U}$ upper semicontinuous, $Y$ a diffusion process with state space
$J\subseteq I$. Here, $Y$ is $X$ absorbed in $D^{c}$ and

$h_{U}(x)=\begin{cases}(g_{1}(x)-F(x))^{+},&x\in D\cap U,\\\ f_{1}(x),&x\in
D^{c}\cap U,\\\ 0,&x\not\in U.\end{cases}$

A solution to such problems in the greatest generality is given in [6]. In
fact, this provides the existence of a maximizing first exit time $\tau^{*}$
in the class $\mathcal{T}$, since our function $h=h_{U}$ obviously satisfies
that the relevant maxima in [6, Section 3/6] are attained. By the
characterization of randomized stopping times by the mixing of stopping times
in $\mathcal{T}$ ([48, Section 7]), it is immediately clear that $\tau^{*}$ is
actually a maximum in the class of all randomized stopping times.

(ii) Here, too, the findings from [6] directly deliver the desired result,
even in the class of first exit times from sets with boundaries in $U_{m}$:
From the proofs in [6, Section 5] you can immediately see that stopping at
points whose values converge towards the maximum of the functions under
consideration delivers the desired result.

(iii) Recall from (i) that an optimal strategy of player 1 never stops in
states $y\in U\cap D$ with $g_{1}(y)-F(y)<0$. Let $\sigma:=\inf\\{t\geq
0:X_{t}\in U\setminus\\{X_{0}\\}\\}$. By Lemma 4.2

$\displaystyle J^{1}(x,\tau,\tau_{2})=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle=$ $\displaystyle
F(x)+(g_{1}(x)-F(x))\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(e^{-A_{\tau-}-r_{1}\tau}g_{1}(X_{\tau})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}}))]$
$\displaystyle=$ $\displaystyle
F(x)+(g_{1}(x)-F(x))\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})].$
(19)

We proceed by showing $g_{1}(x)-F(x)=0$. For that we assume $g_{1}(x)-F(x)>0$.
Let $\xi\sim\operatorname{Bin}(1,\mathbb{P}_{x}(\tau\geq\sigma))$ be
$\mathcal{F}_{0}$ measurable and independent from $X,E_{1},E_{2}$ and consider
the $(\mathcal{F}_{t})_{t\in[0,\infty)}$-stopping time

$\displaystyle\tau^{*}:=\xi\cdot\theta_{\sigma}\circ\tau+\sigma.$

As mentioned above, since $\tau$ is a Markovian randomized time it has the
property
$\mathds{1}_{\\{{\tau\geq\sigma}\\}}\tau\stackrel{{\scriptstyle\operatorname{d}}}{{=}}\mathds{1}_{\\{{\tau\geq\sigma}\\}}\theta_{\sigma}\circ\tau+\sigma$
and thus
$\mathds{1}_{\\{{\tau\geq\sigma}\\}}\tau\stackrel{{\scriptstyle\operatorname{d}}}{{=}}\mathds{1}_{\\{{\tau^{*}\geq\sigma}\\}}\tau^{*}$
due to the choice of $\xi$. The assumption
$\iota_{1}(\tau)(\\{x\\})\in(0,\infty)$ yields
$\mathbb{P}_{x}(\tau\geq\sigma)\in(0,1)$ and $\mathbb{P}_{x}(A_{\tau}=0)=0$,
so by (19)

$\displaystyle
J^{1}(x,\tau,\tau_{2})=F(x)+(g_{1}(x)-F(x))\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle<$ $\displaystyle
F(x)+(g_{1}(x)-F(x))\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle=F(x)+(g_{1}(x)-F(x))\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{*}<\sigma}\\}}\mathds{1}_{\\{{\tau^{*}<\tau^{D}}\\}}]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{*}\geq\sigma}\\}}\mathds{1}_{\\{{\tau^{*}<\tau^{D}}\\}}e^{-A_{\tau^{*}}-r_{1}\tau}(g_{1}(X_{\tau^{*}})-F(X_{\tau^{*}}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{*}=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau^{*}>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle=$ $\displaystyle J^{1}(x,\tau^{*},\tau_{2}),$

which contradict the assumption that $\tau$ is optimal in (18) for the fixed
$x\in I$.

Next we show

$\displaystyle\mathds{1}_{\\{{\tau<\tau^{D}}\\}}(g_{1}(X_{\tau})-F(X_{\tau}))+\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(g_{1}(X_{\tau^{D}})+f_{1}(X_{\tau^{D}}))=0\quad\mathbb{P}_{x}\text{-a.s.}$

We already established ”$\geq$“, so assume
$\mathds{1}_{\\{{\tau<\tau^{D}}\\}}(g_{1}(X_{\tau})-F(X_{\tau}))+\mathds{1}_{\\{{\tau\geq\tau^{D}}\\}}(g_{1}(X_{\tau^{D}})+f_{1}(X_{\tau^{D}}))>0$
on a set with positive probability. Since $g_{1}(x)-F(x)=0$,
$\mathbb{P}_{x}(\tau<\sigma)>0$ and $\mathbb{P}_{x}(\sigma\geq\tau^{D})=1$ we
obtain

$\displaystyle
J^{1}(x,\tau,\tau_{2})=F(x)+(g_{1}(x)-F(x))\mathbb{E}_{x}[\mathds{1}_{\\{{\tau<\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\sigma}\\}}\mathds{1}_{\\{{\tau<\tau^{D}}\\}}e^{-A_{\tau}-r_{1}\tau}(g_{1}(X_{\tau})-F(X_{\tau}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle<$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau\geq\sigma}\\}}\mathds{1}_{\\{{\theta_{\sigma}\circ\tau+\sigma<\tau^{D}}\\}}e^{-A_{\theta_{\sigma}\circ\tau+\sigma}-r_{1}\tau}(g_{1}(X_{\theta_{\sigma}\circ\tau+\sigma})-F(X_{\theta_{\sigma}\circ\tau+\sigma}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\theta_{\sigma}\circ\tau+\sigma=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\theta_{\sigma}\circ\tau+\sigma>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle=$ $\displaystyle
F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\theta_{\sigma}\circ\tau+\sigma<\tau^{D}}\\}}e^{-A_{\theta_{\sigma}\circ\tau+\sigma}-r_{1}\tau}(g_{1}(X_{\theta_{\sigma}\circ\tau+\sigma})-F(X_{\theta_{\sigma}\circ\tau+\sigma}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\theta_{\sigma}\circ\tau+\sigma=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\theta_{\sigma}\circ\tau+\sigma>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle=$ $\displaystyle
J^{1}(x,\theta_{\sigma}\circ\tau+\sigma,\tau_{2})$

to contradict the optimality of $\tau$. We conclude
$J^{1}(x,\tau,\tau_{2})=F(x)$

For every $\tau^{\prime}$ with
$\iota_{1}(\tau^{\prime})(\\{y\\})=\iota_{1}(\tau)(\\{y\\})$ for all $y\in
U\setminus\\{x\\}$ we have

$\displaystyle 0=$
$\displaystyle\mathbb{E}_{x}[\mathds{1}_{\\{{\tau=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{\prime}=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau^{\prime}>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})].$

Additionally $\operatorname{supp}(\mathbb{P}_{x}^{X_{\tau^{\prime}}})\cap
D\subset\operatorname{supp}(\mathbb{P}_{x}^{X_{\tau}})\cap D)$, so

$\displaystyle\mathds{1}_{\\{{\tau^{\prime}<\tau^{D}}\\}}(g_{1}(X_{\tau^{\prime}})-F(X_{\tau^{\prime}}))=0\quad\mathbb{P}_{x}\text{-a.s.}$

thus by Lemma 4.2

$\displaystyle
J^{1}(x,\tau^{\prime},\tau_{2})=F(x)+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{\prime}<\tau^{D}}\\}}e^{-A_{\tau^{\prime}}-r_{1}\tau^{\prime}}(g_{1}(X_{\tau^{\prime}})-F(X_{\tau^{\prime}}))]$
$\displaystyle+\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{\prime}=\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}g_{1}(X_{\tau^{D}})+\mathds{1}_{\\{{\tau^{\prime}>\tau^{D}}\\}}e^{-A_{\tau^{D}-}-r_{1}\tau^{D}}f_{1}(X_{\tau^{D}})]$
$\displaystyle=$ $\displaystyle F(x)=J^{1}(x,\tau,\tau_{2}),$

i.e. $\tau^{\prime}$ is optimal in (18) for the fixed $x\in I$. The for all
$x\in I$ version of the claim follows immediately as the problem is Markovian.

∎

######  Remark 4.4.

(on solutions of the problem (18))
After we have already used the well-known general stopping theory for linear
diffusions from [6] in the previous proof, we now summarize some direct
consequences from it for optimal stopping times, which will be useful to us
later. More precisely, we construct the largest optimal stopping time for our
situation, while typically in the literature the smallest optimal stopping
time is considered.

Let $U\subset I$ be finite and $x\in U$. We consider the problem (18) with the
notation introduced in the proof of Lemma 4.3, in particular we denote the
state space of $Y$ under $\mathbb{P}_{x}$ by $J:=J_{x}$. Additionally let
$A\subset\partial J$ be the absorbing boundary points. By [6, Section 3] we
have

$\displaystyle
e^{-A_{t}-r_{1}t}h_{U}(Y_{t})=e^{-A_{t}-r_{1}t}(p\psi_{+}(Y_{t})+(1-p)\psi_{-}(Y_{t}))\frac{h_{U}(Y_{t})}{p\psi_{+}(Y_{t})+(1-p)\psi_{-}(Y_{t})}$

for some continuous functions $\psi_{+},\psi_{-}:I\to\mathbb{R}$ (the
increasing and decreasing, respectively, positive $(A_{t})_{t}$-harmonic
functions) and $p\in[0,1]$ such that
$Z_{t}:=Z^{p}_{t}:=e^{-A_{t}-r_{1}t}(p\psi_{+}(Y_{t})+(1-p)\psi_{-}(Y_{t}))$
is a non-negative local martingale. Moreover, for starting values outside of
$A$, the stopping problem (18) is equivalent to maximizing
$\frac{h_{U}(y)}{p\psi_{+}(y)+(1-p)\psi_{-}(y)}$, while starting in $A$ the
problem is trivial. Thus setting

$S:=S_{p}:=\bigg{\\{}y\in
J:\frac{h_{U}(y)}{p\psi_{+}(y)+(1-p)\psi_{-}(y)}=\sup_{z\in
I}\frac{h_{U}(z)}{p\psi_{+}(z)+(1-p)\psi_{-}(z)}\bigg{\\}},$

every stopping time $\tau^{*}$ that is optimal in (18) started at $y\in
J\setminus A$ must satisfy $\mathbb{P}_{y}(Y_{\tau^{*}}\not\in S,\exists t\geq
0:Y_{t}\in S)=0$. Note that by construction $S\subset U$ whenever $h_{U}>0$ on
a set with positive probability.

Assume that $\sigma:=\inf\\{t\geq 0:Y_{t}\in A\\}$ is optimal in (18) for some
starting value $y\in J$ of $Y$. By [6, Section 6] there is a $p\in[0,1]$ such
that $A\subset S$ and $(Z_{t\wedge\sigma})_{t\in[0,\infty)}$ is a true
$\mathbb{P}_{y}$-martingale. Thus, by optional sampling, all stopping times
$\tau\leq\sigma$ with $\mathbb{P}_{y}(Y_{\tau}\not\in S,\exists t\geq
0:Y_{t}\in S)=0$ for all $y\in J$ are optimal in (18) starting at $y$.

Clearly, an optimal stopping time must stop with maximal probability before
$Y$ is absorbed in a state that does not realize the maximal value of the
stopping problem. This is put into more precise terms in the following. Let
$L:=A\setminus S$ denote the set of absorbing boundary points of $Y$ at which
the maximum value of the problem (18) started in $J\setminus A$ is not
realized. If $\tau^{*}$ is optimal in (18) starting at all $y\in J\setminus
A$, then $\mathbb{P}_{z}(\tau^{*}=0)=1$ for all $z\in S$ that minimize
$z^{\prime}\mapsto|z^{\prime}-a|$ among all $z^{\prime}\in S$ for some $a\in
L$; we denote the set of these $z$ by $S^{*}$. To show this, let
$\tilde{\sigma}:=\inf\\{t\geq 0:Y_{t}\in L\\}$. As $h_{U}$ is non negative,
$\mathbb{E}_{z}[e^{-A_{\tau^{*}}-r_{1}\tau^{*}}h_{U}(Y_{\tau^{*}})]\leq\mathbb{E}_{z}[e^{-A_{\tau^{*}\wedge\tilde{\sigma}}-r_{1}\tau^{*}\wedge\tilde{\sigma}}h_{U}(Y_{\tau^{*}\wedge\tilde{\sigma}})]$,
but $\mathbb{P}_{z}(\tau^{*}\wedge\tilde{\sigma}\not\in S,\exists t\geq
0:Y_{t}\in S)\geq\mathbb{P}_{z}(\tau^{*}>\tilde{\sigma})>0$ as $Y$ is a
regular diffusion on $J$.

We conclude that whenever $h_{U}>0$ on a non null set, the stopping time
$\hat{\tau}:=\inf\\{t\geq 0:Y_{t}\in S^{*}\cup\\{y\in A:h_{U}(y)>0\\}\\}$ is
the largest optimal stopping time in (18) for all starting values of $Y$. [6,
Lemma 1] provides that $(Z_{t\wedge\hat{\tau}})_{t\in[0,\infty)}$ is
martingale for all starting values of $Y$. Optional sampling yields that all
stopping times $\tau\leq\hat{\tau}$ with $\mathbb{P}_{y}(Y_{\tau}\not\in
S,\tau<\hat{\tau})=0$ are optimal.

The following lemma captures the fact that two Markovian randomized strategies
with independent randomization will, with probability 1, never stop at the
same time while $X$ is outside the closure of the stopping region of either
strategy. This lemma turns out to be a useful tool for the characterization of
the continuity points of the mappings $J^{1},J^{2}$, see Lemma 4.6.

In accordance with the notations introduced in Lemma 3.6 and the proof of
Theorem 3.14, we start by introducing a notation that will be used for the
remainder of this paper. Given any $x\in I$ and a Markovian randomized time
$\tau=\tau^{D,\lambda}\in\mathcal{Z}$ we set

$\displaystyle L^{\tau}_{x}:=\sup((I\setminus D)\cap(-\infty,x]),\quad
R^{\tau}_{x}:=\inf((I\setminus D)\cap[x,\infty)).$

Here we rely on the conventions
$\inf\varnothing:=\infty,\sup\varnothing:=-\infty$.

######  Lemma 4.5.

(quasi independence of Markovian randomized times)
Let $\tau_{1}:=\tau^{D_{1},\lambda_{1}}\in\mathcal{Z}(E_{1})$,
$\tau_{2}:=\tau^{D_{2},\lambda_{2}}\in\mathcal{Z}(E_{2})$ and $x\in I$.

  1. (i)

If $L^{\tau_{1}}_{x}\neq L^{\tau_{2}}_{x}$ and $R^{\tau_{1}}_{x}\neq
R^{\tau_{2}}_{x}$, then $\mathbb{P}_{x}(\tau_{1}=\tau_{2})=0$.

  2. (ii)

If $L^{\tau_{1}}_{x}\neq L^{\tau_{2}}_{x}$, then
$\mathbb{P}_{x}(\eta^{R^{\tau_{1}}_{x}}>\tau_{1},\tau_{1}=\tau_{2})=0$.

  3. (iii)

If $R^{\tau_{1}}_{x}\neq R^{\tau_{2}}_{x}$, then
$\mathbb{P}_{x}(\eta^{L^{\tau_{1}}_{x}}>\tau_{1},\tau_{1}=\tau_{2})=0$.

###### Proof.

(i) Set $S:=(L^{\tau_{1}}_{x},R^{\tau_{1}}_{x})$,
$T:=(L^{\tau_{2}}_{x},R^{\tau_{2}}_{x})$. We start by gathering some
properties of Markovian randomized times to use later on in this proof.

By definition we have

$\displaystyle\\{\tau_{1}<\tau^{S}\\}\subset\\{A^{\tau_{1}}_{\tau_{1}}<\infty\\}\quad\text{and}$
$\displaystyle\\{\tau_{2}<\tau^{T}\\}\subset\\{A^{\tau_{2}}_{\tau_{2}}<\infty\\}.$
(20)

Moreover,

$\displaystyle\\{\tau_{1}=\tau_{2}\\}\subset\\{A^{\tau_{1}}_{\tau_{2}}\geq
E_{1}\\}\cap\\{A^{\tau_{1}}_{\tau_{2}-\varepsilon}<E_{1}\,\,\forall\varepsilon>0\\}\quad\text{and}$
$\displaystyle\\{\tau_{2}=\tau_{1}\\}\subset\\{A^{\tau_{2}}_{\tau_{1}}\geq
E_{2}\\}\cap\\{A^{\tau_{2}}_{\tau_{1}-\varepsilon}<E_{2}\,\,\forall\varepsilon>0\\}.$
(21)

The mappings

$\displaystyle[0,\tau^{S})\ni t\mapsto
A^{\tau_{1}}_{t}\in[0,\infty)\quad\text{and}\quad[0,\tau^{T})\ni t\mapsto
A^{\tau_{2}}_{t}\in[0,\infty).$

are continuous, while

$\displaystyle\tau^{S}:=\inf\\{t\geq
0:A^{\tau_{1}}_{t}=\infty\\}\quad\tau^{T}:=\inf\\{t\geq
0:A^{\tau_{2}}_{t}=\infty\\}.$

We conclude

$\displaystyle\\{\tau_{1}=\tau_{2}\\}\cap\\{\tau_{1}<\tau^{S}\\}\cap\\{\tau_{2}=\tau^{T}\\}$
$\displaystyle\subset$
$\displaystyle\\{\tau_{1}=\tau^{T}\\}\cap\\{\tau_{1}<\tau^{S}\\}$
$\displaystyle\stackrel{{\scriptstyle\eqref{prop1}}}{{\subset}}$
$\displaystyle\\{\tau_{1}=\tau^{T}\\}\cap\\{A^{\tau_{1}}_{\tau^{T}}=A^{\tau_{1}}_{\tau_{1}}<\infty\\}$
$\displaystyle\stackrel{{\scriptstyle\eqref{prop2}}}{{\subset}}$
$\displaystyle\\{A^{\tau_{1}}_{\tau^{T}}\geq
E_{1}\\}\cap\\{A^{\tau_{1}}_{\tau^{T}-\varepsilon}<E_{1}\,\,\forall\varepsilon>0\\}\cap\\{A^{\tau_{1}}_{\tau^{T}}<\infty\\}$
$\displaystyle\subset$ $\displaystyle\\{A^{\tau_{1}}_{\tau^{T}}=E_{1}\\}$ (22)

and analogously

$\displaystyle\\{\tau_{1}=\tau_{2}\\}\cap\\{\tau_{1}=\tau^{S}\\}\cap\\{\tau_{2}<\tau^{T}\\}\subset\\{A^{\tau_{2}}_{\tau^{S}}=E_{2}\\}.$
(23)

Similarly,

$\displaystyle\\{\tau_{1}=\tau_{2}\\}\cap\\{\tau_{1}<\tau^{S}\\}\cap\\{\tau_{2}<\tau^{T}\\}$
$\displaystyle\stackrel{{\scriptstyle\eqref{prop1}}}{{\subset}}$
$\displaystyle\\{\tau_{1}=\tau_{2}\\}\cap\\{A^{\tau_{1}}_{\tau_{2}}=A^{\tau_{1}}_{\tau_{1}}<\infty\\}$
$\displaystyle\stackrel{{\scriptstyle\eqref{prop2}}}{{\subset}}$
$\displaystyle\\{A^{\tau_{1}}_{\tau_{2}}\geq
E_{1}\\}\cap\\{A^{\tau_{1}}_{\tau_{2}-\varepsilon}<E_{1}\,\,\forall\varepsilon>0\\}\cap\\{A^{\tau_{1}}_{\tau_{2}}<\infty\\}$
$\displaystyle\subset$ $\displaystyle\\{A^{\tau_{1}}_{\tau_{2}}=E_{1}\\}$ (24)

Since $X_{\tau_{1}}\in\\{L^{\tau_{1}}_{x},R^{\tau_{1}}_{x}\\}$ and
$X_{\tau_{2}}\in\\{L^{\tau_{2}}_{x},R^{\tau_{2}}_{x}\\}$, the assumption
yields

$\displaystyle\\{\tau_{1}=\tau_{2}\\}\cap\\{\tau_{1}=\tau^{S}\\}\cap\\{\tau_{2}=\tau^{T}\\}\subset\\{\tau^{S}=\tau^{T}\\}=\varnothing.$
(25)

For the final estimate, we start by applying (22), (23), (24) and (25). Then
we find that the conditional expectations vanish since
$A^{\tau_{1}}_{\tau^{T}}$ and $A^{\tau_{2}}_{\tau^{S}}$ are
$\mathcal{F}^{X}_{\infty}$-measurable and thus independent from
$\sigma(E_{1},E_{2})$ and $A^{\tau_{1}}_{\tau_{2}}$ is
$\sigma(\mathcal{F}^{X}_{\infty}\cup\sigma(E_{2}))$-measurable and thus
independent from $\sigma(E_{1})$ while $E_{1},E_{2}$ put no weight in
singletons.

$\displaystyle\mathbb{P}_{x}(\tau_{1}=\tau_{2})$ $\displaystyle=$
$\displaystyle\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\tau^{S},\tau_{2}<\tau^{T})+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}=\tau^{S},\tau_{2}<\tau^{T})$
$\displaystyle+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\tau^{S},\tau_{2}=\tau^{T})+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}=\tau^{S},\tau_{2}=\tau^{T})$
$\displaystyle\leq$
$\displaystyle\mathbb{E}_{x}[\mathds{1}_{\\{{A^{\tau_{1}}_{\tau^{T}}=E_{1}}\\}}]+\mathbb{E}_{x}[\mathds{1}_{\\{{A^{\tau_{2}}_{\tau^{S}}=E_{2}}\\}}]+\mathbb{E}_{x}[\mathds{1}_{\\{{A^{\tau_{1}}_{\tau^{2}}=E_{1}}\\}}]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}[\mathbb{E}_{x}[\mathds{1}_{\\{{A^{\tau_{1}}_{\tau^{T}}=E_{1}}\\}}|\mathcal{F}^{X}_{\infty}]]+\mathbb{E}_{x}[\mathbb{E}_{x}[\mathds{1}_{\\{{A^{\tau_{2}}_{\tau^{S}}=E_{2}}\\}}|\mathcal{F}^{X}_{\infty}]]+\mathbb{E}_{x}[\mathbb{E}_{x}[\mathds{1}_{\\{{A^{\tau_{1}}_{\tau^{2}}=E_{1}}\\}}|\sigma(\mathcal{F}^{X}_{\infty}\cup\sigma(E_{2}))]]$
$\displaystyle=$ $\displaystyle 0$ (26)

To show (ii) and (iii), we can proceed analogously replacing (25) by

$\displaystyle\\{\tau_{1}=\tau_{2}\\}\cap\\{\tau_{1}<\eta^{R^{\tau_{1}}_{x}}\\}\cap\\{\tau_{1}=\tau^{S}\\}\cap\\{\tau_{2}=\tau^{T}\\}\subset\\{\tau^{S}=\tau^{T}\\}\cap\\{\tau_{1}<\eta^{L^{\tau_{1}}_{x}}\\}=\varnothing\,\,\,\text{or}$
$\displaystyle\\{\tau_{1}=\tau_{2}\\}\cap\\{\tau_{1}<\eta^{L^{\tau_{1}}_{x}}\\}\cap\\{\tau_{1}=\tau^{S}\\}\cap\\{\tau_{2}=\tau^{T}\\}\subset\\{\tau^{S}=\tau^{T}\\}\cap\\{\tau_{1}<\eta^{L^{\tau_{1}}_{x}}\\}=\varnothing$

respectively. Instead of (26) we have

$\displaystyle\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\eta^{L^{\tau_{1}}_{x}})$
$\displaystyle\leq$
$\displaystyle\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\tau^{S},\tau_{2}<\tau^{T})+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}=\tau^{S},\tau_{2}<\tau^{T})$
$\displaystyle+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\tau^{S},\tau_{2}=\tau^{T})+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}=\tau^{S},\tau_{2}=\tau^{T},\tau_{1}<\eta^{L^{\tau_{1}}_{x}})=...=0$

or

$\displaystyle\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\eta^{R^{\tau_{1}}_{x}})$
$\displaystyle\leq$
$\displaystyle\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\tau^{S},\tau_{2}<\tau^{T})+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}=\tau^{S},\tau_{2}<\tau^{T})$
$\displaystyle+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}<\tau^{S},\tau_{2}=\tau^{T})+\mathbb{P}_{x}(\tau_{1}=\tau_{2},\tau_{1}=\tau^{S},\tau_{2}=\tau^{T},\tau_{1}<\eta^{R^{\tau_{1}}_{x}})=...=0$

respectively to finish the proof. ∎

######  Lemma 4.6.

($\mathcal{M}$-continuity of $J$)
Assume (A), (B). Let $U\subset I$ finite, $x\in I$,
$(\tau^{1}_{n},\tau^{2}_{n})_{n\in\mathbb{N}}\in(\mathcal{Z}_{U}(E_{1})\times\mathcal{Z}_{U}(E_{2}))^{\mathbb{N}}$
and $(\tau^{1},\tau^{2})\in\mathcal{Z}_{U}(E_{1})\times\mathcal{Z}_{U}(E_{2})$
with $(\tau^{1}_{n},\tau^{2}_{n})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}(\tau^{1},\tau^{2})$ in the
$\mathcal{M}$-topology.

  1. (i)

If $L^{\tau^{1}}_{x}\neq L^{\tau^{2}}_{x}$ whenever
$f_{1}(L^{\tau^{1}}_{x})>g_{1}(L^{\tau^{1}}_{x})$ and $R^{\tau^{1}}_{x}\neq
R^{\tau^{2}}_{x}$ whenever $f_{1}(R^{\tau^{1}}_{x})>g_{1}(R^{\tau^{1}}_{x})$
then

$J^{1}(x,\tau^{1}_{n},\tau^{2}_{n})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}J^{1}(x,\tau^{1},\tau^{2})$

and
$J^{1}(x,\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}},\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}})$
is continuous in $(\tau^{1},\tau^{2})$.

  2. (ii)

If $L^{\tau^{1}}_{x}\neq L^{\tau^{2}}_{x}$ whenever
$f_{2}(L^{\tau^{2}}_{x})>g_{2}(L^{\tau^{2}}_{x})$ and $R^{\tau^{1}}_{x}\neq
R^{\tau^{2}}_{x}$ whenever $f_{2}(R^{\tau^{2}}_{x})>g_{2}(R^{\tau^{2}}_{x})$
then

$J^{2}(x,\tau^{1}_{n},\tau^{2}_{n})\stackrel{{\scriptstyle
n\to\infty}}{{\longrightarrow}}J^{2}(x,\tau^{1},\tau^{2})$

and
$J^{2}(x,\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}},\mathchoice{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\displaystyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\textstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptstyle\bullet\,$}}}}}{\mathbin{\vbox{\hbox{\scalebox{0.6}{$\,\scriptscriptstyle\bullet\,$}}}}})$
is continuous in $(\tau^{1},\tau^{2})$.

######  Remark 4.7.

By setting

$\displaystyle g^{i}(-\infty):=g^{i}(\inf I)$ $\displaystyle
g^{i}(\infty):=g^{i}(\sup I)$ $\displaystyle f^{i}(-\infty):=f^{i}(\inf I)$
$\displaystyle f^{i}(\infty):=f^{i}(\sup I)$

for $i=1,2$, Lemma 4.6 allows for $L^{\tau^{1}}_{x}=L^{\tau^{2}}_{x}\not\in
I$, $R^{\tau^{1}}_{x}=R^{\tau^{2}}_{x}\not\in I$ or
$\mathbb{P}_{x}(\tau^{1}\wedge\tau^{2}=\infty)>0$.

###### Proof.

(of Lemma 4.6)
We only show (i) as (ii) can be shown in exactly the same way. W.l.o.g. we
only treat the case $L^{\tau^{1}}_{x}=L^{\tau^{2}}_{x}$, $R^{\tau^{1}}_{x}\neq
R^{\tau^{2}}_{x}$. The other cases $L^{\tau^{1}}_{x}\neq L^{\tau^{2}}_{x}$,
$R^{\tau^{1}}_{x}=R^{\tau^{2}}_{x}$ and $L^{\tau^{1}}_{x}=L^{\tau^{2}}_{x}$,
$R^{\tau^{1}}_{x}=R^{\tau^{2}}_{x}$ are analogous using the appropriate parts
of Lemma 4.5. We have

$\displaystyle
J^{1}(x,\tau^{1},\tau^{2})=\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{1}\leq\tau^{2}}\\}}e^{-r_{1}\tau^{1}}g(X_{\tau^{1}})+\mathds{1}_{\\{{\tau^{1}>\tau^{2}}\\}}e^{-r_{1}\tau^{2}}f_{1}(X_{\tau^{2}})]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}[(\mathds{1}_{\\{{\tau^{1}\leq\tau^{2}}\\}}e^{-r_{1}\tau^{1}}g(X_{\tau^{1}})+\mathds{1}_{\\{{\tau^{1}>\tau^{2}}\\}}e^{-r_{1}\tau^{2}}f_{1}(X_{\tau^{2}}))(\mathds{1}_{\\{{\eta^{L^{\tau^{1}}_{x}}\leq\tau^{1}\wedge\tau^{2}}\\}}+\mathds{1}_{\\{{\eta^{L^{\tau^{1}}_{x}}>\tau^{1}\wedge\tau^{2}}\\}})],$
$\displaystyle
J^{1}(x,\tau^{1},\tau^{2})=\mathbb{E}_{x}[\mathds{1}_{\\{{\tau^{1}_{n}\leq\tau^{2}_{n}}\\}}e^{-r_{1}\tau^{1}_{n}}g(X_{\tau^{1}_{n}})+\mathds{1}_{\\{{\tau^{1}_{n}>\tau^{2}_{n}}\\}}e^{-r_{1}\tau^{2}_{n}}f_{1}(X_{\tau^{2}_{n}})]$
$\displaystyle=$
$\displaystyle\mathbb{E}_{x}[(\mathds{1}_{\\{{\tau^{1}_{n}\leq\tau^{2}_{n}}\\}}e^{-r_{1}\tau^{1}_{n}}g(X_{\tau^{1}_{n}})+\mathds{1}_{\\{{\tau^{1}_{n}>\tau^{2}_{n}}\\}}e^{-r_{1}\tau^{2}_{n}}f_{1}(X_{\tau^{2}_{n}}))(\mathds{1}_{\\{{\eta^{L^{\tau^{1}}_{x}}\leq\tau^{1}\wedge\tau^{2}}\\}}+\mathds{1}_{\\{{\eta^{L^{\tau^{1}}_{x}}>\tau^{1}\wedge\tau^{2}}\\}})].$

By Lemma 4.5 (iii), we have
$\mathbb{P}_{x}(\eta^{L^{\tau^{1}}_{x}}>\tau^{1}\wedge\tau^{2},\tau^{1}=\tau^{2})=0$.
Together with Lemma 3.3 (iii), (iv)

$\displaystyle(\mathds{1}_{\\{{\tau^{1}_{n}\leq\tau^{2}_{n}}\\}}e^{-r_{1}\tau^{1}_{n}}g(X_{\tau^{1}_{n}})+\mathds{1}_{\\{{\tau^{1}_{n}>\tau^{2}_{n}}\\}}e^{-r_{1}\tau^{2}_{n}}f_{1}(X_{\tau^{2}_{n}}))\mathds{1}_{\\{{\eta^{L^{\tau^{1}}_{x}}>\tau^{1}\wedge\tau^{2}}\\}}$
$\displaystyle\stackrel{{\scriptstyle n\to\infty}}{{\longrightarrow}}$