# Semimartingle Representation of a class of Semi-Markov Dynamics

Anindya Goswami  IISER Pune, India<EMAIL_ADDRESS>,  Subhamay Saha
IIT Guwahati, India<EMAIL_ADDRESS>and  Ravishankar Kapildev Yadav
IISER Pune, India<EMAIL_ADDRESS>
###### Abstract.

We consider a class of semi-Markov processes (SMP) such that the embedded
discrete time Markov chain may be non-homogeneous. The corresponding augmented
processes are represented as semi-martingales using stochastic integral
equation involving a Poisson random measure. The existence and uniqueness of
the equation are established. Subsequently, we show that the solution is
indeed a SMP with desired transition rate. Finally, we derive the law of the
bivariate process obtained from two solutions of the equation having two
different initial conditions.

This research was supported in part by NBHM 02011/1/2019/NBHM(RP)R& D-II/585,
DST/INTDAAD/P-12/2020, and DST FIST (SR/FST/MSI-105). The second author would
like to acknowledge the support from CSIR SRF. The support and the resources
provided by ‘PARAM Brahma Facility’ under the National Supercomputing Mission,
Government of India at the Indian Institute of Science Education and Research
(IISER) Pune are gratefully acknowledged.

*Corresponding author

Key words Poisson random measure, non-homogeneous Semi-Markov Processes, semi-
Markov system
AMSC 60G55, 60H20, 60K15

##  1\. Introduction

Apparently [17] and [24], which were presented at the International Congress
of Mathematicians held at Amsterdam in 1954, are the first available
literature that discuss the mathematical aspects of semi-Markov processes
(SMP). In Lévy’s work, [17], the definition of SMP was presented as a
generalization of Markov chain. Around the same time, independent to Lévy’s
work, Smith [24] and then Tackas [25] have also introduced SMP. Below we
present a modern version of the definition.

######  Definition 1.1.

$X=\\{X_{t}\\}_{t\geq 0}$ defined on a complete probability space
$(\Omega,\mathcal{B},P)$ is an SMP with state space $\mathcal{X}$ if

  1. (1)

$X$ is piece-wise constant r.c.l.l. process with discontinuities at
$T_{1}<T_{2}<\cdots<T_{n}<\cdots$,

  2. (2)

and for each $n\geq 1,j\in\mathcal{X},$ and $y>0,$

$P[X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq
y\mid(X_{0},T_{0}),(X_{T_{k}},T_{k})\;\forall 1\leq k\leq
n]=P[X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y\mid X_{T_{n}}]$ (1.1)

where $T_{0}\leq 0<T_{1}$.

$X$ is pure if $T_{n}\xrightarrow{a.s.}\infty$ as $n\rightarrow\infty$. If,
the right side of (1.1) is independent on $n$, then the SMP $X$ is called time
homogeneous. Otherwise, $X$ is called non-homogeneous SMP. Here, $T_{n}$ is
called $n$th transition time.

We recall that in Chapter 1 of [9], a book by Boris Harmalov, a stepped SMP is
introduced and in subsequent chapters further generalizations to continuous
state space appears. We confine ourselves to the study of SMPs on a countable
state space. The continuous-time discrete-state homogeneous Markov chains are
included in this class. The class under consideration is such that the
embedded discrete time Markov chain may also be non-homogeneous. However, a
more general non-homogeneity that appears in a continuous time Markov process
having time dependent transition rate is not included.

Unlike Definition 1.1, SMP has been defined by many authors using renewal
processes. For example, in [22], Pyke has introduced the SMPs by specifying
the conditional distribution of next state and holding time given the past
states and past holding times. We notice that the $\sigma$-algebra generated
by the transition times is identical to that generated by the initial time and
past holding times. Thus the conditional distribution in [22] is identical to
the conditional probability in Definition 1.1. In [22], a concept of
regularity has been introduced for assuring that the chain is pure. Besides,
classification of the states has also been studied there. On the other hand in
[23], by considering the finite state SMPs, Pyke has derived expressions for
the distribution functions of first passage times, as well as for the marginal
distribution function. Furthermore, the limiting behavior of a Markov Renewal
process has been discussed, and the stationary probabilities have been
derived. Various aspects and approaches regarding limiting behavior has also
been studied by Orey [20], around the same time. In [6], Bennet has studied
some properties of sub-chain of SMP, which are obtained via regenerating
points, if they exist.

It has been noted by several authors (see Nummelin [19], Athreya et al [1] and
references therein) that an SMP can be augmented with the age process to
obtain a jointly Markov process whose Feller property and infinitesimal
generator can be derived (see Chapter 2, [16]). In a recent work [5] Elliott
has presented a semimartingale dynamics of semi-Markov chain in contrast to
the traditional description of a semi-Markov chain in terms of a renewal
process. This presentation is different from that in [7] and [8] where a
semimartingale representation appears using an integration with respect to a
Poisson random measure(PRM). Such semimartingale representations are useful
for studying several aspects including the stochastic flow of semi-Markov
dynamics.

Apparently, SMP beyond the class of time homogeneity has been first studied in
[10] and [18] Various different aspects and generalisations of non-homogeneous
SMP(NHSMP) has further been explored in [12, 13, 14, 15]. In these references
several applications of NHSMP has also been emphasised.

In this paper we consider a system of SDEs driven by a PRM and having
coefficients depending on a given transition rate function and an additional
gaping parameter. It is worth noting that neither the coefficients are
compactly supported nor the intensity measure is finite. So, we produce a
self-contained proof of the existence and uniqueness of the solution to the
SDE. We further show that the state component of the solution is a pure semi-
Markov process with the given transition rate function. The law of a single
solution of course do not depend on the gaping parameter. However, we show,
the joint distribution of a couple of solutions with different initial
conditions do depend on the additional gaping parameter. The combination of
state processes of two solutions forms a semi-Markov system(SMS) or a
component-wise semi-Markov (CSM) process having dependent components. The SMSs
[26, 27, 28], or CSMs [3, 4] with independent components have been introduced
for modelling some random dynamics. However, a CSM with dependent components
has not been studied in the literature yet. The law of the CSM has been
calculated in terms of the infinitesimal generator of the augmented process.
As per our knowledge, the present paper is the first work on the correlated
semi-Markov system. The immense applicability of semi-Markov processes and
semi-Markov systems is well known. So, the formulation and the results related
to the correlated semi-Markov system, presented in this paper have
significantly expanded the scope of further theoretical and applied studies.
In particular this has opened up the possibility of studying a system
generated by a semi-Markov flow. The questions related to the meeting and
merging events of multiple particles of a semi-Markov flow have far reaching
implications. In view of these, the present paper appears as a stepping stone
for several future studies.

The rest of this paper is arranged in the following manner. We present a class
of SMPs as semi-martingales using an SDE involving PRM in Section 2. Here we
first consider the SDE and then we show that it has almost sure unique
solution. The state component is shown to possess semi-Markov law in Section
3. The transition parameters of the semi-Markov process are also obtained in
that section. In Section 4 the joint distribution of solutions having two
different initial conditions has been derived in terms of the infinitesimal
generator of the combined process. Some concluding remarks and the scope of
future research directions are added in Section 5.

##  2\. SDE using Poisson Random Measure

Let $(\Omega,\mathcal{F},\\{\mathcal{F}_{t}\\}_{t\geq 0},P)$ be the underlying
filtered probability space satisfying the usual hypothesis and
$\mathcal{X}=\\{1,2,\mathellipsis\\}$ the state space. We wish to construct a
semi-Markov process on this state space with a given transition rate function.
To this end we first embed $\mathcal{X}$ in $\mathbb{R}$ and endow with a
total order $\prec_{1}$, which in turn induces a total order $\prec_{2}$ on
$\mathcal{X}_{2}:=\\{(i,j)\in\mathcal{X}^{2}\ \mid\ i\neq j\\}$ by following
lexicographic order. Let $\mathcal{B}(\mathbb{R}^{d})$ denote the Borel
$\sigma$-algebra on $\mathbb{R}^{d}$ and $m_{d}$ denote the Lebesgue measure
on $\mathbb{R}^{d}.$ Also let $\mathbb{N}_{0}$ and $\mathbb{R}_{+}$ denote the
set of non-negative integers and the set of non-negative real numbers
respectively. For each $y\in\mathbb{R}_{+}$, $n\in\mathbb{N}_{0}$,
$\lambda(y,n):=(\lambda_{ij}(y,n))$ denote a matrix in which the
$i\textsuperscript{th}$ diagonal element is negative of
$\lambda_{i}(y,n):=\sum_{j\in\mathcal{X}\setminus\\{i\\}}\lambda_{ij}(y,n)$
and for each $(i,j)\in\mathcal{X}_{2}$,
$\lambda_{ij}\colon\mathbb{R}_{+}\times\mathbb{N}_{0}\to\mathbb{R}_{+}$ is a
bounded measurable function satisfying the following two assumptions.

  1. (A1).

If
$c_{i}:=\underset{j\in\mathcal{X}\setminus\\{i\\}}{\sum}\|\lambda_{ij}(\cdot,\cdot)\|_{L^{\infty}_{(\mathbb{R}_{+}\times\mathbb{N}_{0})}}$,
$c:=\sup_{i}c_{i}<\infty$.

  2. (A2).

For each $n$ in $\mathbb{N}_{0}$, and $i$ in $\mathcal{X}$,
$\lim_{y\rightarrow\infty}\gamma_{i}(y,n)=\infty$, where
$\gamma_{i}(y,n):=\int_{0}^{y}\lambda_{i}(y^{\prime},n)dy^{\prime}$.

For each $(i,j)\in\mathcal{X}_{2},$ we consider another measurable function
$\tilde{\lambda}_{ij}\colon\mathbb{R}_{+}\times\mathbb{N}_{0}\to\mathbb{R}_{+}$
such that for each $y\in\mathbb{R}_{+}$, and $n\in\mathbb{N}_{0}$

$\displaystyle\lambda_{ij}(y,n)\leq\tilde{\lambda}_{ij}(y,n),$ (2.1)

and also for almost every $y\in\mathbb{R}_{+}$ and $n\in\mathbb{N}_{0}$

$\displaystyle\tilde{\lambda}_{ij}(y,n)\leq\|\lambda_{ij}(\cdot,\cdot)\|_{L^{\infty}_{(\mathbb{R}_{+}\times\mathbb{N}_{0})}}.$
(2.2)

Now for each $y\in\mathbb{R}_{+}$, and $n\in\mathbb{N}_{0}$, with the help of
$\lambda(y,n)$ and $\tilde{\lambda}(y,n):=(\tilde{\lambda}_{ij}(y,n))$, we
introduce a disjoint collection of intervals
$\Lambda:=\\{\Lambda_{ij}(y,n)\colon(i,j)\in\mathcal{X}_{2}\\}$, by

$\displaystyle\Lambda_{ij}(y,n)=\left(\sum_{(i^{\prime},j^{\prime})\prec_{2}(i,j)}\tilde{\lambda}_{i^{\prime}j^{\prime}}(y,n)\right)\
+\Big{[}0,\lambda_{ij}(y,n)\Big{)}$ (2.3)

where $a+B=\\{a+b\mid b\in B\\}$ for $a\in\mathbb{R},B\subset\mathbb{R}$.
Clearly, for each $y$ and $n$, the interval $\Lambda_{ij}(y,n)$ is of length
$\lambda_{ij}(y,n)$. If $C_{i}:=c_{i}+\sum_{k\prec_{1}i}c_{k}$, according to
(A1) and (2.2), the set
$\Lambda_{i}(y,n):=\cup_{j\in\mathcal{X}\setminus\\{i\\}}\Lambda_{ij}(y,n)$ is
contained in the finite interval $\left[0,C_{i}\right]$, for almost every
$y\in\mathbb{R}_{+}$ and each $n$. Using the above intervals we define
$h_{\Lambda}\colon\mathcal{X}\times\mathbb{R}_{+}\times\mathbb{N}_{0}\times\mathbb{R}\rightarrow\mathbb{R}$
as

$h_{\Lambda}(i,y,n,v):=\sum_{j\in\mathcal{X}\setminus\\{i\\}}\\!(j-i)\mathds{1}_{\Lambda_{ij}(y,n)}(v)$
(2.4)

and
$g_{\Lambda}\colon\mathcal{X}\times\mathbb{R}_{+}\times\mathbb{N}_{0}\times\mathbb{R}\rightarrow\mathbb{R}$
as

$g_{\Lambda}(i,y,n,v):=\sum_{j\in\mathcal{X}\setminus\\{i\\}}\\!\mathds{1}_{\Lambda_{ij}(y,n)}(v).$
(2.5)

Here $\mathds{1}_{B}$ is the indicator function of the set $B$. These
functions are piece-wise constant in $v$ variable. Using these, we consider
the following system of coupled stochastic integral equations in $X$, $Y$ and
$N$

$\displaystyle X_{t}$
$\displaystyle=X_{0}+\displaystyle\int_{0+}^{t}\int_{\mathbb{R}}h_{\Lambda}(X_{u-},Y_{u-},N_{u-},v)\wp(du,dv)$
(2.6) $\displaystyle Y_{t}$
$\displaystyle=Y_{0}+t-\displaystyle\int_{0+}^{t}(Y_{u-})\int_{\mathbb{R}}g_{\Lambda}(X_{u-},Y_{u-},N_{u-},v)\wp(du,dv)$
(2.7) $\displaystyle N_{t}$
$\displaystyle=\int_{0+}^{t}\int_{\mathbb{R}}g_{\Lambda}(X_{u-},Y_{u-},N_{u-},v)\wp(du,dv)$
(2.8)

for $t\geq 0$, where $\int_{0+}^{t}$ is integration over the interval $(0,t]$,
$\wp(du,\ dv)$ is a PRM on $\mathbb{R}_{+}\times\mathbb{R}$ with intensity
$m_{2}(du,dv)$, and defined on the probability space $(\Omega,\mathcal{F},P).$
We also assume that $\\{\wp((0,t]\times dv)\\}_{t\geq 0}$ is adapted to
$\\{\mathcal{F}_{t}\\}_{t\geq 0}$. For $t\geq 0$, we rewrite the above coupled
integral equation (2.6)-(2.8) as a vector form given by,

$\displaystyle
Z_{t}=Z_{0}+\int_{0}^{t}a(Z_{u-})du+\int_{0+}^{t}\int_{\mathbb{R}}J(Z_{u-},v)\wp(du,dv),$
(2.9)

where for each $t\geq 0$, $Z_{t}=(X_{t},Y_{t},N_{t})$, $N_{0}=0$, $z=(i,y,n)$,
$a(z)=(0,1,0)$, and $J(z,v)$ is the vector
$(h_{\Lambda}(z,v),-yg_{\Lambda}(z,v),g_{\Lambda}(z,v))$.

######  Remark 2.1.

The existing results on the general theory of SDE are not directly applicable
for assuring existence and uniqueness of solution to the system of integral
equations (2.6)-(2.8). We recall that Theorem 3.4 (p-474) of [2] assumes
compact support of the integrands whereas Theorem IV.9.1 (p-231) of [11]
assumes finiteness of the intensity measure on the complement of a
neighbourhood of origin. For the system (2.6)-(2.8) neither the integrands are
compactly supported nor the intensity of Poisson random measure is finite in
the complement of any bounded set. For this reason we produce an original
proof of existence and uniqueness of (2.6)-(2.8).

######  Theorem 2.2.

Under assumption (A1), there exists an increasing sequence of stopping times
$\\{T_{n}\\}_{n=1}^{\infty}$ such that the following hold.

  1. (1)

The coupled system of stochastic integral equations (2.6)-(2.8) has a unique
strong solution $(X,Y,N)={(X_{t},Y_{t},N_{t})}_{t\in[0,\tau)}$ where
$\tau=\lim_{n\to\infty}T_{n}$.

  2. (2)

Almost surely $X$, $Y$, and $N$ have r.c.l.l. paths respectively and they jump
only at $T_{n}$, for each $n$. While $X$ and $N$ are piece-wise constants, $Y$
is piece-wise linear.

  3. (3)

We set $T_{0}:=-Y_{0}$. For each $n\in\mathbb{N}$, (i) $Y_{T_{n}}=0$, (ii)
$Y_{T_{n}-}=T_{n}-T_{n-1}$, and (iii) $N_{T_{n}}=n$. Also (iv)
$Y_{t}=t-T_{N_{t}}$ for all $t\in\mathbb{R}_{+}$.

Proof.  We recall that due to Assumption (A1) and (2.2), for almost every
$y\in\mathbb{R}_{+}$, and each $i,n$ the union of intervals
$\\{\Lambda_{ij}(y,n)\mid{j\in\mathcal{X}\setminus\\{i\\}}\\}$ is contained in
$[0,C_{i}]$. For each $\omega\in\Omega$ and $i\in\mathcal{X}$ we define the
set
$\mathcal{D}_{i}:=\\{s\in(0,\infty)\mid\wp(\omega)(\\{s\\}\times[0,C_{i}])>0\\}$,
the collection of time coordinates of the point masses of a realisation of the
PRM $\wp(\omega)$ in which the second component is not more than $C_{i}$.
Since, the interval $[0,C_{i}]$ has finite Lebesgue measure, by Lemma A.1,
$\mathcal{D}_{i}$ has no limit points in $\mathbb{R}$ with probability 1. Thus
for each $i\in\mathcal{X}$ we can enumerate $\mathcal{D}_{i}$, in increasing
order

$\mathcal{D}_{i}=\\{\sigma^{i}_{l}\\}_{l=1}^{\infty},\textrm{ where
}\sigma^{i}_{1}<\cdots<\sigma^{i}_{l}<\sigma^{i}_{l+1}<\cdots\textrm{ for each
}\omega.$ (2.10)

For each $l\in\mathbb{N},\sigma^{i}_{l}\colon\Omega\to(0,\infty]$ and
$\\{\sigma^{i}_{l}\leq t\\}=\\{\omega\mid\wp((0,t]\times[0,C_{i}])\geq
l\\}\in\mathcal{F}_{t}$, as $\wp((0,t]\times[0,C])$ is $\mathcal{F}_{t}$
measurable. Hence $\sigma^{i}_{l}$ is a stopping time for each
$i\in\mathcal{X}$ and $l\geq 1$. As $\mathcal{D}_{i}$ has no limit points,
$\sigma^{i}_{l}\uparrow\infty$ as $l\to\infty$ almost surely.

Here we plan to construct an increasing sequence
$\\{\bar{\sigma}_{m}\\}_{m=1}^{\infty}$ so that we can first define a solution
to equations (2.6)-(2.8) on the time interval $[0,\ \bar{\sigma}_{1}]$ and
then on the next time interval $(\bar{\sigma}_{1},\ \bar{\sigma}_{2}]$, and so
on. For a fixed $\omega\in\Omega$, $X$, $Y$ at time $t=0$ are
$X_{0}\in\mathcal{X}$, and $Y_{0}\in\mathbb{R}_{+}$ respectively. Using
(2.10), we consider the increasing sequence $\mathcal{D}_{X_{0}}$ and call the
first element of this sequence as $\bar{\sigma}_{1}$. Hence,

$\wp(\omega)\left([0,\ \bar{\sigma}_{1})\times\ [0,\ C_{X_{0}}]\right)=0,$

and thus for $t\in[0,\ \bar{\sigma}_{1})$

$\displaystyle X_{t}(\omega)$
$\displaystyle=X_{0}+\int_{0+}^{t}\int_{0}^{C_{X_{0}}}h_{\Lambda}(X_{u-},\
Y_{u-},N_{u-},\ v)\wp(\omega)(du,\ dv)=X_{0},$ $\displaystyle Y_{t}(\omega)$
$\displaystyle=Y_{0}+t-\int_{0+}^{t}\int_{0}^{C_{X_{0}}}(Y_{u-})g_{\Lambda}(X_{u-},\
Y_{u-},N_{u-},\ v)\wp(\omega)(du,\ dv)=Y_{0}+t,$ $\displaystyle N_{t}(\omega)$
$\displaystyle=\int_{0+}^{t}\int_{0}^{C_{X_{0}}}g_{\Lambda}(X_{u-},\
Y_{u-},N_{u-},\ v)\wp(\omega)(du,\ dv)=0.$

Note that in the above integrations the domain $\mathbb{R}$ has been replaced
by the compact set $[0,C_{X_{0}}]$, as the integrands vanish outside this
interval. Hence at $t=\bar{\sigma}_{1},$

$\displaystyle X_{\bar{\sigma}_{1}}(\omega)$
$\displaystyle=X_{0}+\int_{0}^{C_{X_{0}}}h_{\Lambda}(X_{0},\
Y_{\bar{\sigma}_{1}-},N_{0},\ v)\wp(\omega)(\\{\bar{\sigma}_{1}\\}\times dv),$
$\displaystyle Y_{\bar{\sigma}_{1}}(\omega)$
$\displaystyle=Y_{0}+\bar{\sigma}_{1}-\int_{0}^{C_{X_{0}}}(Y_{\bar{\sigma}_{1}-})g_{\Lambda}(X_{0},\
Y_{\bar{\sigma}_{1}-},N_{0},\ v)\wp(\omega)(\\{\bar{\sigma}_{1}\\}\times dv),$
$\displaystyle N_{\bar{\sigma}_{1}}(\omega)$
$\displaystyle=\int_{0}^{C_{X_{0}}}g_{\Lambda}(X_{0},\
Y_{\bar{\sigma}_{1}-},N_{0},\ v)\wp(\omega)(\\{\bar{\sigma}_{1}\\}\times\
dv).$

Hence the solution is unique in the time interval $[0,\ \bar{\sigma}_{1}]$.
Again by using (2.10), we have an increasing sequence
$\mathcal{D}_{X_{\bar{\sigma_{1}}}}$. As
$\mathcal{D}_{X_{\bar{\sigma}_{1}}}\cap(\bar{\sigma}_{1},\infty)$ is nonempty
almost surely, there is the first element of the set, denoted by
$\bar{\sigma}_{2}$. Thus we have,

$\wp(\omega)\left((\bar{\sigma}_{1},\ \bar{\sigma}_{2})\times\ [0,\
C_{X_{\bar{\sigma}_{1}}}]\right)=0$

and for $t\in(\bar{\sigma}_{1},\ \bar{\sigma}_{2})$, as before

$\displaystyle X_{t}(\omega)$
$\displaystyle=X_{\bar{\sigma}_{1}}+\int_{\bar{\sigma}_{1}+}^{t}\int_{0}^{C_{X_{\bar{\sigma}_{1}}}}h_{\Lambda}(X_{u-},\
Y_{u-},N_{u-},\ v)\wp(\omega)\ (du,\ dv)=X_{\bar{\sigma}_{1}},$ $\displaystyle
Y_{t}(\omega)$
$\displaystyle=Y_{\bar{\sigma}_{1}}+(t-\bar{\sigma}_{1})-\int_{\bar{\sigma}_{1}+}^{t}\int_{0}^{C_{X_{\bar{\sigma}_{1}}}}(Y_{u-})g_{\Lambda}(X_{u-},\
Y_{u-},N_{u-},\ v)\wp(\omega)\ (du,\
dv)=Y_{\bar{\sigma}_{1}}+(t-\bar{\sigma}_{1}),$ $\displaystyle N_{t}(\omega)$
$\displaystyle=N_{\bar{\sigma}_{1}}+\int_{\bar{\sigma}_{1}+}^{t}\int_{0}^{C_{X_{\bar{\sigma}_{1}}}}g_{\Lambda}(X_{u-},\
Y_{u-},N_{u-},\ v)\wp(\omega)(du,\ dv)=N_{\bar{\sigma}_{1}}.$

Hence by using above equalities, at $t=\bar{\sigma}_{2},$

$\displaystyle X_{\bar{\sigma}_{2}}(\omega)$
$\displaystyle=X_{\bar{\sigma}_{1}}+\int_{0}^{C_{X_{\bar{\sigma}_{1}}}}h_{\Lambda}(X_{\bar{\sigma}_{1}},\
Y_{\bar{\sigma}_{2}-},N_{\bar{\sigma}_{1}},\
v)\wp(\omega)(\\{\bar{\sigma}_{2}\\}\times dv),$ $\displaystyle
Y_{\bar{\sigma}_{2}}(\omega)$
$\displaystyle=Y_{\bar{\sigma}_{1}}+(\bar{\sigma}_{2}-\bar{\sigma}_{1})-\int_{0}^{C_{X_{\bar{\sigma}_{1}}}}(Y_{\bar{\sigma}_{1}-})g_{\Lambda}(X_{\bar{\sigma}_{1}},\
Y_{\bar{\sigma}_{2}-},N_{\bar{\sigma}_{1}},\
v)\wp(\omega)(\\{\bar{\sigma}_{2}\\}\times dv),$ $\displaystyle
N_{\bar{\sigma}_{2}}(\omega)$
$\displaystyle=N_{\bar{\sigma}_{1}}+\int_{0}^{C_{X_{\bar{\sigma}_{1}}}}g_{\Lambda}(X_{\bar{\sigma}_{1}},\
Y_{\bar{\sigma}_{2}-},N_{\bar{\sigma}_{1}},\
v)\wp(\omega)(\\{\bar{\sigma}_{2}\\}\times\ dv).$

Continuing in the similar way we can construct a solution in a unique manner
for each consecutive interval $(\bar{\sigma}_{m},\ \bar{\sigma}_{m+1}],$ where
$m\geq 2$. Moreover, for a fixed $\omega$,
$X_{t}(\omega)=X_{\bar{\sigma}_{m}(\omega)}$ for all
$t\in[\bar{\sigma}_{m}(\omega),\bar{\sigma}_{m+1}(\omega))$. Hence $X$ is an
r.c.l.l. and piece-wise constant process almost surely on
$[0,\lim_{m\to\infty}\bar{\sigma}_{m}]$. By this, Part (1) and (2) would have
followed if $\\{\bar{\sigma}_{m}\\}_{m\geq 1}$ were the jump times, which may
not be true. For this reason, now we select an appropriate sub-sequence of
$\bar{\sigma}_{m}$ which are the jump times. To this end we first note that
$\int_{0}^{C_{X_{t-}}}g_{\Lambda}(X_{t-},\
Y_{t-},N_{t-},v)\wp(\omega)(\\{t\\}\times dv)$ is zero for all
$t\in(\bar{\sigma}_{m},\bar{\sigma}_{m+1})$ for every $m\geq 1$. Using this
and $X_{\bar{\sigma}_{m}-}=X_{\bar{\sigma}_{m-1}}$,
$N_{\bar{\sigma}_{m}-}=N_{\bar{\sigma}_{m-1}}$ we introduce

$\displaystyle l_{1}:=\min\\{m\geq
1\colon\int_{0}^{C_{X_{\bar{\sigma}_{m-1}}}}g_{\Lambda}(X_{\bar{\sigma}_{m-1}},\
Y_{{\bar{\sigma}}_{m}-},N_{{\bar{\sigma}}_{m-1}},\
v)\wp(\omega)(\\{{\bar{\sigma}}_{m}\\}\times dv)\neq 0\\}.$ (2.11)

From (2.4) and (2.5) it is evident that $h_{\Lambda}$ and $g_{\Lambda}$ have
identical supports in $v$ variable. Hence the integrals
$\int_{I}h_{\Lambda}(X_{t-},Y_{t-},N_{t-}v)\wp(\\{t\\}\times dv)$ is non-zero
if and only if $\int_{I}g_{\Lambda}(X_{t-},Y_{t-},N_{t-}v)\wp(\\{t\\}\times
dv)$ is non-zero, where $I$ is any interval. Then $t={\bar{\sigma}}_{l_{1}}$
is the first time when both the integrals

$\int_{[0,C_{X_{t-}}]}g_{\Lambda}(X_{t-},\ Y_{t-},N_{t-},\
v)\wp(\omega)(\\{t\\}\times dv)\quad\textrm{ and
}\quad\int_{[0,C_{X_{t-}}]}h_{\Lambda}(X_{t-},\ Y_{t-},N_{t-},\
v)\wp(\omega)(\\{t\\}\times dv)$

are non-zero. Consequently, $Y_{t}=Y_{0}+t$, $X_{t}=X_{0}$, and $N_{t}=0$, for
all $t\in[0,\bar{\sigma}_{l_{1}})$. Hence $X_{{\bar{\sigma}}_{l_{1}}-}=X_{0}$,
$Y_{{\bar{\sigma}}_{l_{1}}-}=Y_{0}+\bar{\sigma}_{l_{1}}$ and
$N_{{\bar{\sigma}}_{l_{1}}-}=0$. Furthermore, at $t=\bar{\sigma}_{l_{1}}$,

$\displaystyle 0$
$\displaystyle\neq\int_{0+}^{\bar{\sigma}_{l_{1}}}\int_{0}^{C_{X_{\bar{\sigma}_{l_{1}-1}}}}g_{\Lambda}(X_{t-},\
Y_{t-},N_{t-},\ v)\wp(\omega)(dt,dv)$
$\displaystyle=\int_{0}^{C_{X_{\bar{\sigma}_{l_{1}-1}}}}g_{\Lambda}(X_{\bar{\sigma}_{l_{1}-1}},\
Y_{\bar{\sigma}_{l_{1}}-},N_{\bar{\sigma}_{l_{1}-1}},\
v)\wp(\omega)(\\{\bar{\sigma}_{l_{1}}\\}\times dv)=1,$

using (2.11) and the facts that
$\wp(\omega)(\\{\bar{\sigma}_{l_{1}}\\}\times[0,C_{X_{\bar{\sigma}_{l_{1}-1}}}])=1$
and $g_{\Lambda}(i,y,n,v)\in\\{0,1\\}$ for every $i$, $y$, $n$, and $v$. Thus
from (2.8) and above expressions

$\displaystyle N_{\bar{\sigma}_{l_{1}}}=$
$\displaystyle\int_{0+}^{\bar{\sigma}_{l_{1}}}\int_{0}^{C_{X_{\bar{\sigma}_{l_{1}-1}}}}g_{\Lambda}(X_{t-},\
Y_{t-},N_{t-},\ v)\wp(\omega)(dt,dv)=1.$

Similarly, using (2.7) we have $Y_{\bar{\sigma}_{l_{1}}}=0$. To see this, we
write
$Y_{\bar{\sigma}_{l_{1}}}=Y_{\bar{\sigma}_{l_{1}-}}+(Y_{\bar{\sigma}_{l_{1}}}-Y_{\bar{\sigma}_{l_{1}}-})$,
i.e.,

$\displaystyle Y_{\bar{\sigma}_{l_{1}}}=$ $\displaystyle
Y_{\bar{\sigma}_{l_{1}-}}-\int_{0}^{C_{X_{\bar{\sigma}_{l_{1}-1}}}}(Y_{\bar{\sigma}_{l_{1}}-})g_{\Lambda}(X_{\bar{\sigma}_{l_{1}}-},Y_{\bar{\sigma}_{l_{1}}-},N_{\bar{\sigma}_{l_{1}}-},\
v)\wp(\omega)(\\{\bar{\sigma}_{l_{1}}\\}\times dv)$ $\displaystyle=$
$\displaystyle Y_{\bar{\sigma}_{l_{1}}-}-Y_{\bar{\sigma}_{l_{1}}-}=0.$

In general, for every $n\geq 1$, we set

$l_{n+1}:=\min\left\\{m>l_{n}\colon\int_{0}^{C_{X_{\bar{\sigma}_{m-1}}}}g_{\Lambda}(X_{\bar{\sigma}_{m-1}},\
Y_{\bar{\sigma}_{m}-},N_{\bar{\sigma}_{m-1}},\
v)\wp(\omega)(\\{\bar{\sigma}_{m}\\}\times dv)\neq 0\right\\}.$ (2.12)

In other words, for every $t\geq 0$,

$\displaystyle\int_{\mathbb{R}}g_{\Lambda}(X_{t-},\ Y_{t-},N_{t-}\
v)\wp(\\{t\\}\times dv)=\left\\{\begin{array}[]{ll}1&,\ \mathrm{if}\
t=\bar{\sigma}_{l_{n}}\textrm{ for some }n\geq 1\\\ 0&,\
\mathrm{otherwise}.\end{array}\right.$ (2.15)

From (2.15) and (2.8) we get

$\displaystyle N_{t}=$ $\displaystyle\sum_{\\{r\geq
1\mid\bar{\sigma}_{l_{r}}\leq
t\\}}1=\sum_{r=1}^{\infty}\mathds{1}_{[\bar{\sigma}_{l_{r}},\infty)}(t).$
(2.16)

Moreover, from (2.15) and (2.7) we get

$\displaystyle Y_{t}=$ $\displaystyle Y_{0}+t-\sum_{\\{r\geq
1\mid\bar{\sigma}_{l_{r}}\leq
t\\}}Y_{\bar{\sigma}_{l_{r}}-}=Y_{0}+t-\sum_{r=1}^{\infty}(Y_{\bar{\sigma}_{l_{r}}-})\mathds{1}_{[\bar{\sigma}_{l_{r}},\infty)}(t).$
(2.17)

Furthermore, as the support of $g_{\Lambda}$ and $h_{\Lambda}$ are identical,
from (2.12) we have that $\int_{\mathbb{R}}h_{\Lambda}(X_{t-},\ Y_{t-},N_{t-}\
v)\wp(\\{t\\}\times dv)$ is nonzero if and only if $t=\bar{\sigma}_{l_{n}}$
for some $n\geq 1$. Therefore, if $n\geq 1$ is such that
$\bar{\sigma}_{l_{n}}\leq t<\bar{\sigma}_{l_{n+1}}$, then
$X_{t}=X_{\bar{\sigma}_{l_{n}}}$. Thus we can write

$\displaystyle X_{t}=$ $\displaystyle X_{0}+\sum_{\\{r\geq
1\mid\bar{\sigma}_{l_{r}}\leq
t\\}}(X_{\bar{\sigma}_{l_{r}}}-X_{\bar{\sigma}_{l_{r-1}}})=X_{0}+\sum_{r=1}^{\infty}(X_{\bar{\sigma}_{l_{r}}}-X_{\bar{\sigma}_{l_{r-1}}})\mathds{1}_{[\bar{\sigma}_{l_{r}},\infty)}(t).$

Hence $X$ and $N$ are r.c.l.l., and piece-wise constant and $Y$ is r.c.l.l.,
and piece-wise linear. We denote $T_{n}:=\bar{\sigma}_{l_{n}}$ for each $n\geq
1$. From above, it is evident that $\\{T_{n}\\}_{n\geq 1}$ is the desired
sequence of stopping times at which the processes $X$, $Y$, and $N$ jump and
the properties in parts (1) and (2) hold.

Next for proving part (3), we first rewrite (2.17)

$\displaystyle Y_{t}=$ $\displaystyle Y_{0}+t-\sum_{\\{r\geq 1\mid T_{r}\leq
t\\}}Y_{T_{r}-}$

and thus for all $n\in\mathbb{N},$

$\displaystyle Y_{T_{n}}=$ $\displaystyle Y_{0}+T_{n}-\sum_{\\{r\geq 1\mid
T_{r}\leq T_{n}\\}}Y_{T_{r}-}$ $\displaystyle=$
$\displaystyle\left(Y_{0}+T_{n}-\sum_{\\{r\geq 1\mid
T_{r}<T_{n}\\}}Y_{T_{r}-}\right)-Y_{T_{n}-}$ $\displaystyle=$ $\displaystyle
Y_{T_{n}-}-Y_{T_{n}-}=0.$

Thus (i) holds. For showing (ii) we first recall that
$Y_{T_{1}-}=Y_{0}+T_{1}$, which is same as $T_{1}-T_{0}$. Now for $n>1$ using
$Y_{T_{n-1}}=0$ we get

$\displaystyle Y_{T_{n}-}=$ $\displaystyle\left(Y_{0}+T_{n}-\sum_{\\{r\geq
1\mid T_{r}<T_{n}\\}}Y_{T_{r}-}\right)$ $\displaystyle=$
$\displaystyle(T_{n}-T_{n-1})+\left(Y_{0}+T_{n-1}-\sum_{\\{r\geq 1\mid
T_{r}\leq T_{n-1}\\}}Y_{T_{r}-}\right)$ $\displaystyle=$
$\displaystyle(T_{n}-T_{n-1})+Y_{T_{n-1}}=(T_{n}-T_{n-1}).$

Hence (ii) is shown. Again, (2.16) implies that $N_{t}=\max\\{r:T_{r}\leq
t\\}$. That is $N_{T_{n}}=\max\\{r:T_{r}\leq T_{n}\\}=n$, which proves (iii).
Finally, (iv) follows using (2.17) and (ii) as below

$\displaystyle Y_{t}=$ $\displaystyle Y_{0}+t-\sum_{\\{1\leq r\leq
N_{t}\\}}(T_{r}-T_{r-1})=Y_{0}+t-(T_{N_{t}}-T_{0})=t-T_{N_{t}}.\qed$

######  Remark 2.3.

The jump times $\\{T_{n}\\}_{n\geq 1}$ of the process $X$ are called the
transition times. We need to prove that this sequence diverges to infinity
almost surely for establishing that the solution is globally determined with
probability 1. This is accomplished in the following Theorem.

######  Theorem 2.4.

There exists a unique strong solution $Z=(X_{t},Y_{t},N_{t})_{t\geq 0}$ of the
coupled SDE (2.6)-(2.8)

Proof.  Let $\\{T_{n}\\}_{n\geq 1}$ be as in Theorem 2.2. Now we will show
that $T_{n}$ diverges. Let $\tau:=\lim_{n\to\infty}T_{n}$. Clearly, for a
fixed $\epsilon>0$, the event $\\{\tau<\infty\\}$ is a subset of
$\cup_{n_{0}\geq 1}\\{T_{n}-T_{n-1}<\epsilon,\;\forall n\geq n_{0}\\}$. For
the sake of brevity, we denote $\\{T_{n}-T_{n-1}<\epsilon\\}$ as $A_{n}$ for
each $n$. Therefore, we will show $P(\cap_{n\geq n_{0}}A_{n})=0$ for each
$n_{0}$ which is enough for proving $P(\tau<\infty)=0$. To this end we first
compute the conditional probability of the event $A_{n}$ given the
observations till $T_{n-1}$, using the properties of Poisson random measure.
Indeed, using part (3) of Theorem 2.2, occurrence of $A_{n}$ is same as having
Poisson random measure of
$\cup_{0<y<\epsilon}\left(\\{T_{n-1}+y\\}\times\Lambda_{X_{T_{n-1}}}(y,n-1)\right)$
positive. Thus using the Lebesgue intensity of $\wp$, we have

$\displaystyle P\left(A_{n}\mid T_{n-1},X_{T_{n-1}},\ldots,T_{0},X_{0}\right)$
$\displaystyle=P\left(\wp\left(\underset{0<y<\epsilon}{\cup}\left(\\{T_{n-1}+y\\}\times\Lambda_{X_{T_{n-1}}}(y,n-1)\right)\right)\neq
0\mid T_{n-1},X_{T_{n-1}},\ldots,T_{0},X_{0}\right)$
$\displaystyle=\left(1-e^{-\int_{0}^{\epsilon}\lambda_{X_{T_{n-1}}}(y,n-1)dy}\right)$
$\displaystyle\leq\left(1-e^{-\epsilon c}\right)$ (2.18)

which is a deterministic constant. We recall that the last inequality is due
to Assumption (A1). Next we note that if

$\displaystyle E\left[\prod_{n_{0}\leq n\leq
l}\mathds{1}_{A_{n}}\right]\leq\left(1-e^{-\epsilon
c}\right)E\left[\prod_{n_{0}\leq n\leq l-1}\mathds{1}_{A_{n}}\right]$ (2.19)

holds for all $l\geq n_{0}$, using that repeatedly, we get

$\displaystyle P\left(\cap_{n_{0}\leq n}A_{n}\right)$ $\displaystyle\leq
P\left(\cap_{n_{0}\leq n\leq l}A_{n}\right)=E\left[\prod_{n_{0}\leq n\leq
l}\mathds{1}_{A_{n}}\right]\leq(1-e^{-\epsilon c})^{l-n_{0}+1}$

for all $l\geq n_{0}$. The right side clearly vanishes as $l\to\infty$, and
thus $P(\cap_{n_{0}\leq n}A_{n})=0$, as desired, provided (2.19) holds.
Finally we show (2.19) below using the property of conditional expectation,
and inequality (2)

$\displaystyle E\left[\prod_{n_{0}\leq n\leq l}\mathds{1}_{A_{n}}\right]$
$\displaystyle=E\left[E\left[\prod_{n_{0}\leq n\leq
l}\mathds{1}_{A_{n}}\Big{|}\;T_{l-1},X_{T_{l-1}},\ldots,T_{0},X_{0}\right]\right]$
$\displaystyle=E\left[\left(\prod_{n_{0}\leq n\leq
l-1}\mathds{1}_{A_{n}}\right)E\left[\mathds{1}_{A_{l}}\Big{|}\;T_{l-1},X_{T_{l-1}},\ldots,T_{0},X_{0}\right]\right]$
$\displaystyle\leq\left(1-e^{-\epsilon c}\right)E\left[\prod_{n_{0}\leq n\leq
l-1}\mathds{1}_{A_{n}}\right]$

for all $l\geq n_{0}$. Hence the proof is complete. ∎ The above theorem
essentially asserts that the jump process $X$ is pure. In the next section we
show that $X$ is a semi-Markov process.

##  3\. Law of the Solution

###  3.1. Semi-Markovity

######  Theorem 3.1.

Let $Z=(X,Y,N)=\\{(X_{t},\ Y_{t},N_{t})\\}_{t\geq 0}$ be the unique strong
solution to (2.6)-(2.8). Then the following hold.

  1. i.

The process $\\{X_{t}\\}_{t\geq 0}$ is a pure SMP.

  2. ii.

The embedded chain $\\{X_{T_{n}}\\}_{n\geq 1}$ is a discrete time Markov
chain.

Proof.  First we will prove that $X$ is SMP. We have already seen in the proof
of Theorem 2.2, that $X$ is an r.c.l.l. process. Next, we need to show (1.1),
i.e., for each $n\in\mathbb{N}$, $y\in\mathbb{R}_{0}$, $j\in\mathcal{X}$

$P[X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y\mid
X_{0},T_{0},X_{T_{1}},T_{1},\ldots,X_{T_{n}},T_{n}]=P[X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq
y\mid X_{T_{n}}].$ (3.1)

We note that the left side is equal to

$\displaystyle P(T_{n+1}-T_{n}\leq
y\mid(X_{0},{T_{0}}),(X_{T_{1}},{T_{1}}),\ldots,(X_{T_{n}},{T_{n}}))$
$\displaystyle\times
P(X_{T_{n+1}}=j\mid(X_{0},{T_{0}}),(X_{T_{1}},{T_{1}}),\ldots,(X_{T_{n}},{T_{n}}),\\{T_{n+1}-T_{n}\leq
y\\}).$ (3.2)

Each of the two conditional probabilities is further simplified below. For
almost every $\omega\in\Omega$, Equation (2.15) and Theorem 2.2 part
(3)(iii)-(iv) imply that for any $n\geq 1$

$\int_{T_{n}+}^{T_{n}+t}(u-T_{n})\int_{\mathbb{R}}g_{\Lambda}(X_{T_{n}},u-T_{n},n,v)\,\wp(du,dv)=\begin{cases}0,&\text{
for }t<T_{n+1}-T_{n}\\\ T_{n+1}-T_{n},&\text{ for
}t=T_{n+1}-T_{n}.\end{cases}$

Hence, by a suitable change of variable, almost surely $T_{n+1}-T_{n}$ is the
first occurrence of a non-zero value of the following map

$t\mapsto\int_{0+}^{t}u\int_{\mathbb{R}}g_{\Lambda}(X_{T_{n}},u,n,v)\,\wp(T_{n}+du,dv)$

and that occurs at $t=T_{n+1}-T_{n}$. Again, since $\wp(T_{n}+du,dv)$ is
independent to $\mathcal{F}_{T_{n}}$ we obtain, $T_{n+1}-T_{n}$ is
conditionally independent to $\mathcal{F}_{T_{n}}$ given $X_{T_{n}}$. Thus for
all $n\geq 1$

$\displaystyle P\left(T_{n+1}-T_{n}\leq
y\mid(X_{0},{T_{0}}),(X_{T_{1}},{T_{1}}),\dots,(X_{T_{n}},{T_{n}})\right)=P(T_{n+1}-T_{n}\leq
y\mid X_{T_{n}}).$ (3.3)

By substituting $t$ equal to $T_{n}$ and $T_{n+1}$ in Equation (2.6), and
using Theorem 2.2 part (3) (ii)-(iii) we get

$\displaystyle
X_{T_{n+1}}=X_{T_{n}}+\int_{\mathbb{R}}h_{\Lambda}(X_{T_{n}},T_{n+1}-T_{n},n,v)\,\wp(\\{T_{n+1}\\}\times
dv),$ (3.4)

as $X_{T_{n+1}-}=X_{T_{n}}$, $Y_{T_{n+1}-}=T_{n+1}-T_{n}$ and
$N_{T_{n+1}-}=n$. Thus using (3.4)

$\displaystyle
P\left(X_{T_{n+1}}=j\mid(X_{0},{T_{0}}),(X_{T_{1}},{T_{1}}),\ldots,(X_{T_{n}},{T_{n}}),\\{T_{n+1}-T_{n}\leq
y\\}\right)$
$\displaystyle=P\bigg{(}\int_{\mathbb{R}}h_{\Lambda}(X_{T_{n}},T_{n+1}-T_{n},n,v)\,\wp(\\{T_{n}+(T_{n+1}-T_{n})\\}\times
dv)=$ $\displaystyle j-X_{T_{n}}\Big{|}$
$\displaystyle(X_{0},{T_{0}}),(X_{T_{1}},{T_{1}}),\ldots,(X_{T_{n}},{T_{n}}),$
$\displaystyle\\{T_{n+1}-T_{n}\leq y\\}\bigg{)}.$

Again, using the independence of $\wp(T_{n}+du,dv)$ to $\mathcal{F}_{T_{n}}$
and conditional independence of $T_{n+1}-T_{n}$ to $\mathcal{F}_{T_{n}}$ given
$X_{T_{n}}$ we conclude, the right side expression is equal to

$\displaystyle
P\left(\int_{\mathbb{R}}h_{\Lambda}(X_{T_{n}},T_{n+1}-T_{n},n,v)\,\wp(\\{T_{n}+(T_{n+1}-T_{n})\\}\times
dv)=j-X_{T_{n}}~{}\Big{|}~{}X_{T_{n}},\\{T_{n+1}-T_{n}\leq y\\}\right)$

which is again equal to $P\left(X_{T_{n+1}}=j\mid
X_{T_{n}},\\{T_{n+1}-T_{n}\leq y\\}\right)$ using (3.4). Thus, using this
simplification and (3.3) in (3.1), we obtain for all $n\geq 1$

$\displaystyle P(X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq
y\mid(X_{0},{T_{0}}),(X_{T_{1}},{T_{1}}),\ldots,(X_{T_{n}},{T_{n}}))$
$\displaystyle=P\left(T_{n+1}-T_{n}\leq y\mid
X_{T_{n}}\right)P\left(X_{T_{n+1}}=j\mid X_{T_{n}},\\{T_{n+1}-T_{n}\leq
y\\}\right)$ $\displaystyle=P\left(X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y\mid
X_{T_{n}}\right).$

Hence, part $(i)$ is proved.
For every $n\geq 1$, taking $y\to\infty$ on both sides in Equation (3.1), we
get

$P[X_{T_{n+1}}=j\mid
X_{T_{0}},T_{0},X_{T_{1}},T_{1},\ldots,X_{T_{n}},T_{n}]=P[X_{T_{n+1}}=j\mid
X_{T_{n}}].$

Hence the part $(ii)$. ∎

###  3.2. Transition Parameters

For each $i\in\mathcal{X},n\in\mathbb{N}_{0}$, we define a function
$F(\cdot\mid i,n)\colon[0,\infty)\rightarrow[0,1]$ as

$F(y\mid i,n):=1-e^{-\gamma_{i}(y,n)}$ (3.5)

where $\gamma_{i}(y,n)$ is as in (A2). Clearly, $F(\cdot\mid i,n)$ is
differentiable almost everywhere. To see this, we note that $\gamma_{i}(y,n)$
is an integral of a bounded Lebesgue measurable function, and thus is
absolutely continuous in $y$. Let $f(y\mid i,n)$ be the almost everywhere
derivative of $F(y\mid i,n)$. We also define for each $y\in\mathbb{R}_{+}$,
$n\in\mathbb{N}_{0}$, a matrix
$p(y,n):=(p_{ij}(y,n))_{\mathcal{X}\times\mathcal{X}}$, such that

$p_{ij}(y,n):=\begin{cases}\dfrac{\lambda_{ij}(y,n)}{\lambda_{i}(y,n)}\mathds{1}_{(0,\infty)}(\lambda_{i}(y,n)),&\textrm{
if }j\neq i\\\ \mathds{1}_{\\{0\\}}(\lambda_{i}(y,n)),&\textrm{ if
}j=i.\end{cases}$ (3.6)

Since, $\lambda_{ij}(y,n)\geq 0$ and
$\sum_{j\in\mathcal{X}\setminus\\{i\\}}\lambda_{ij}(y,n)=\lambda_{i}(y,n)$,
for each $n\in\mathbb{N}_{0}$ and $y\in\mathbb{R}_{+}$, $p(y,n)$ is a
transition probability matrix. The following proposition asserts that $p(y,n)$
gives the conditional probability of selecting a state at the time of $n+1$st
transition given the age $y$ and location $i$ of the previous state.
Furthermore, the map $F(\cdot\mid i,n)$ as in (3.5) is also asserted as the
conditional cumulative distribution function of the holding time at the $n$th
state given that is $i$.

######  Theorem 3.2.

Let $Z=(X,Y,N)$ be the solution to (2.6)-(2.8), $i\in\mathcal{X}$,
$y\in\mathbb{R}_{+}$ and $n\geq 1$ then the following hold.

  1. i.

$F(\cdot\mid i,n)$ is the conditional cumulative distribution function of the
holding time of the process $X$.

  2. ii.

For all $j\neq X_{T_{n}}$,
$p_{X_{T_{n}}j}(T_{n+1}-T_{n},n)=P[X_{T_{n+1}}=j\mid X_{T_{n}},T_{n+1}-T_{n}]$
almost surely.

Proof.  We recall that, the intensity of $\wp$ is Lebesgue measure, and the
Lebesgue measure of $\\{(u,v)\in(T_{n},T_{n}+y)\times\mathbb{R}_{+}\mid
v\in\Lambda_{i}(u-T_{n},n)\\}$ is
$\int_{T_{n}}^{T_{n}+y}\lambda_{i}(u-T_{n},n)du$ which is equal to
$\gamma_{i}(y,n)$ (see (A2)). Using (2.5) and (2.15), the conditional
probability of no transition in the next $y$ unit time, given that the
$n\textsuperscript{th}$ transition happens now to state $i,$ is given by
$e-{}\gamma_{i}(y,n)$ Using the above fact, the conditional cumulative
distribution function at $y$ of the holding time after the $n$th transition,
given the state is $i$, is

$\displaystyle P[T_{n+1}-T_{n}\leq y\mid X_{T_{n}}=i]$
$\displaystyle=1-P[X_{t}=X_{t-},\forall t\in(T_{n},T_{n}+y]\mid X_{T_{n}}=i]$
$\displaystyle=1-e^{-\gamma_{i}(y,n)}$

for all $y\in\mathbb{R}_{+}$ and $i\in\mathcal{X}.$ Thus (i) follows from
(3.5).

We note that, for $j\neq i$, $P[X_{T_{n+1}}=j\mid
X_{T_{n}}=i,T_{n+1}-T_{n}=y]$ is the conditional probability of the event that
the $n+1$st state is $j$, given that $T_{n+1}=T_{n}+y$ and the $n$th state is
$i$. Using (3.4), the above is the conditional probability that a Poisson
point mass appears in $\\{T_{n}+y\\}\times\Lambda_{ij}(y,n)$ given that the
point mass lies somewhere in $\\{T_{n}+y\\}\times\Lambda_{i}(y,n)$ and no
transition of $X$ occurs during $(T_{n},T_{n}+y)$. If these three events are
denoted by $A$, $B$, and $C$ respectively, then the conditional probability
$P(A\mid B\cap C)$ can be simplified as $P(A\mid B)$ because $C$ is
independent to both $A$ and $B$. Thus using the Lebesgue intensity of $\wp$,

$\displaystyle P\left[X_{T_{n+1}}=j\mid X_{T_{n}}=i,T_{n+1}-T_{n}=y\right]=$
$\displaystyle
P\left[\wp(\\{T_{n}+y\\}\times\Lambda_{ij}(y,n))=1\mid\wp(\\{T_{n}+y\\}\times\Lambda_{i}(y,n))=1\right]$
$\displaystyle=$
$\displaystyle\dfrac{|\Lambda_{ij}(y,n)|}{|\Lambda_{i}(y,n)|}$
$\displaystyle=$ $\displaystyle\dfrac{\lambda_{ij}(y,n)}{\lambda_{i}(y,n)}$

for every $y\in\mathbb{R}_{+}$, and $j\neq i$, provided $\lambda_{i}(y,n)\neq
0$. Thus from (3.6), we get $p_{ij}(y,n)$ is equal to
$P\left[X_{T_{n+1}}=j\mid X_{T_{n}}=i,T_{n+1}-T_{n}=y\right]$ when
$\lambda_{i}(y,n)\neq 0$. Next we note that $\lambda_{i}(y,n)=0$ if and only
if
$\frac{d}{dy}F(y\mid i,n)=0$, i.e., the density of $T_{n+1}-T_{n}$ is zero at
$y$. Hence (ii) holds. ∎

######  Remark 3.3.

Using (3.5), we note that under Assumptions (A1) and (A2), $F(y\mid i,n)<1$
for all $y\in\mathbb{R}_{+}$ and ${\lim}_{y\rightarrow\infty}F(y\mid i,n)=1$
using (3.5). Thus, the holding times are unbounded but finite almost surely.
By dropping (A1), one may include a class of SMPs having bounded holding
times. However, we exclude that class from our discussion. It is also
important to note that the SMPs having discontinuous cdf of holding times are
also not considered in the present setting. Nevertheless, the present study
subsumes countable-state continuous time Markov chains and the processes
having age dependent transitions as appear in [8, 21].

######  Proposition 3.4.

We have, for almost every $y\geq 0$ and $n\geq 1$,

$p_{ij}(y,n)\dfrac{f(y\mid i,n)}{1-F(y\mid
i,n)}=\begin{cases}\lambda_{ij}(y,n),&\textrm{ for }i\neq j,\\\ 0,&\textrm{
for }i=j.\end{cases}$

Proof.  By differentiating both sides of (3.5), we obtain $f(y\mid
i,n)=\lambda_{i}(y,n)e^{-\gamma_{i}(y,n)}$ for every $y\in\mathbb{R}_{+}$.
This is equal to $\lambda_{i}(y,n)(1-F(y\mid i,n))$ using (3.5). Hence, for
every $y\in\mathbb{R}_{+}$, $n\geq 1$ and $i\in\mathcal{X}$

$\displaystyle\frac{f(y\mid i,n)}{1-F(y\mid i,n)}$
$\displaystyle=\lambda_{i}(y,n).$ (3.7)

If $i\neq j$, for every $y\in\mathbb{R}_{+}$, using (3.6)

$\displaystyle p_{ij}(y,n)\dfrac{f(y\mid i,n)}{1-F(y\mid i,n)}=$
$\displaystyle\dfrac{\lambda_{ij}(y,n)}{\lambda_{i}(y,n)}\times\lambda_{i}(y,n)\mathds{1}_{(0,\infty)}(\lambda_{i}(y,n))=\lambda_{ij}(y,n)$

as $0\leq\lambda_{ij}(y,n)\leq\lambda_{i}(y,n)$. The case for $i=j$ follows
from (3.6) and (3.7) directly as $p_{ii}(y,n)\dfrac{f(y\mid i,n)}{1-F(y\mid
i,n)}$ is equal to $\lambda_{i}(y,n)\mathds{1}_{\\{0\\}}(\lambda_{i}(y,n))$
which is zero. ∎

######  Theorem 3.5.

Let $X$ be a SMP as in Theorem 3.1. Then, the associated kernel is given by

$P[X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y\mid
X_{T_{n}}=i]=\int_{0}^{y}e^{-\gamma_{i}(s,n)}\lambda_{ij}(s,n)\ ds,$

which is denoted by $Q_{ij}(y,n)$ for every $y>0$, $n\geq 1$ and $i\neq j.$

Proof.  Using Theorem 3.2 (i) and (ii) and Theorem 2.2 part (3) (ii)

$\displaystyle P\left[X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y\mid
X_{T_{n}}=i\right]$
$\displaystyle=\mathbb{E}\left[P\left(X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y\mid
X_{T_{n}}=i,T_{n+1}-T_{n}\right)\mid X_{T_{n}}=i\right]$
$\displaystyle=\int_{0}^{\infty}\mathds{1}_{[0,y]}(s)P\left[X_{T_{n+1}}=j\mid
X_{T_{n}}=i,T_{n+1}-T_{n}=s\right]f(s\mid i,n)\,ds$
$\displaystyle=\int_{0}^{y}p_{ij}(s,n)f(s\mid i,n)\,ds.$

For each $i\neq j$, using Proposition 3.4 and (3.5), the right side of above
can be rewritten as

$\displaystyle\int_{0}^{y}(1-F(s\mid i,n))\lambda_{ij}(s,n)\,ds\
=\int_{0}^{y}e^{-\gamma_{i}(s,n)}\lambda_{ij}(s,n)\,ds=Q_{ij}(y,n).$

∎

######  Proposition 3.6.

Let $X$ be a SMP as in Theorem 3.1. Then, $\lambda(n,y)$ is the instantaneous
transition rate matrix.

Proof. The rate of transition from state $i$ to $j$ at age $y$ is given by

$\displaystyle\lim_{h\to
0}\frac{1}{h}\left[P(X_{T_{n+1}}=j,T_{n+1}-T_{n}\in(y,y+h]\mid
X_{T_{n}}=i,\\{T_{n+1}-T_{n}>y\\})\right]$ $\displaystyle=\lim_{h\to
0}\frac{1}{h}\frac{P(X_{T_{n+1}}=j,T_{n+1}-T_{n}\in(y,y+h]\mid
X_{T_{n}}=i)}{P(T_{n+1}-T_{n}>y\mid X_{T_{n}}=i)}$ $\displaystyle=\lim_{h\to
0}\frac{1}{h}\frac{P(X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y+h\mid
X_{T_{n}}=i)-P(X_{T_{n+1}}=j,T_{n+1}-T_{n}\leq y\mid
X_{T_{n}}=i)}{1-P(T_{n+1}-T_{n}\leq y\mid X_{T_{n}}=i)}.$

Using Theorem 3.5, the above limit is equal to
$\frac{\frac{d}{dy}Q_{ij}(y,n)}{1-F(y\mid i,n)}$ which can further be
simplified as $\lambda_{ij}(y,n)$. ∎

######  Remark 3.7.

We have obtained $Q_{ij}(y,n)=\int_{0}^{y}p_{ij}(s,n)f(s\mid i,n)\,ds$ in the
proof of Theorem 3.5, which expresses $Q_{ij}(\cdot,n)$ in terms of the
$p_{ij}(\cdot,n)$, and $f(\cdot\mid i,n)$. These parameters give the age
dependent transition probabilities and the conditional holding time densities.
In an alternative conditioning, the kernel can also be expressed as
$Q_{ij}(y,n)=P\left[X_{T_{n+1}}=j\mid
X_{T_{n}}=i\right]P\left[T_{n+1}-T_{n}\leq y\mid
X_{T_{n}}=i,X_{T_{n+1}}=j\right]$, which is the product of transition
probabilities of embedded chain and the conditional cdf of holding time given
the current and the next states. Generally an SMP is characterized using the
transition kernel $Q$. Although, instantaneous transition rate matrix
$\lambda$ also characterizes an SMP, $Q$ is considered more fundamental as
$\lambda$ exists only if $Q$ is differentiable. In that case, each of $Q$ and
$\lambda$ can be expressed in terms of another, which is evident from the
above two results.

##  4\. Component-wise Semi-Markov Process

######  Definition 4.1.

A pure jump process $X$ on a countable state space $\mathcal{X}$ is called a
component-wise Semi-Markov Process (CSM) if there is a bijection
$\Gamma:\mathcal{X}\to\prod_{i=1}^{d}\mathcal{X}_{i}$, such that each
component of $\Gamma(X)$ is a semi-Markov process, where $d$ is a positive
integer and for each $i\leq d$, $\mathcal{X}_{i}$ is an at-most countable non-
empty set.

Without loss of generality, we assume that
$\mathcal{X}=\prod_{i=1}^{d}\mathcal{X}_{i}$ and $X^{i}$ is a semi-Markov
process on $\mathcal{X}_{i}$ for each $i\leq d$. Here $\Gamma$ is the identity
map. Next we consider a specific CSM process of dimension $2$.

######  Notation 4.2.

Fix $i,j\in\mathcal{X}$ and $y_{1},y_{2}\geq 0$. Let
$Z^{1}=(X^{1},Y^{1},N^{1})$ and $Z^{2}=(X^{2},Y^{2},N^{2})$ be the strong
solutions of (2.6)-(2.8) with two different initial conditions. At a fixed
time $s(>0)$ we denote

$i=X^{1}_{s},y_{1}=Y^{1}_{s},n_{1}=N^{1}_{s}$ (4.1)

and

$j=X_{s}^{2},y_{2}=Y_{s}^{2},n_{2}=N^{2}_{s}$ (4.2)

respectively. We also denote $(Z^{1},Z^{2})$ as $Z$.

Throughout this part, we will use the notation described above. For the sake
of computing some specific parameters connected to the law of Z, we limit
ourselves to the following choice of $\tilde{\lambda}$.

  1. (A3).

For all $y\in\mathbb{R}_{+}$, $n\in\mathbb{N}_{0}$ and
$(i,j)\in\mathcal{X}_{2}$,
$\tilde{\lambda}_{ij}(y,n)=\max(\|\lambda_{ij}(\cdot,\cdot)\|_{L^{\infty}_{(\mathbb{R}_{+}\times\mathbb{N}_{0})}},\lambda_{ij}(y,n))$.

Since, $Z^{1}$ and $Z^{2}$ as in Notation 4.2 are Markov, $Z=(Z^{1},Z^{2})$ is
also Markov. It has state, component-wise age and component-wise number of
transitions $X=(X^{1},X^{2})$, $Y=(Y^{1},Y^{2})$ and $N=(N^{1},N^{2})$
respectively. While each of $X^{1}$ and $X^{2}$ is semi-Markov, the pure jump
process $X$ is not. Rather, $X$ is a component-wise semi-Markov process (CSM)
and the Markov process $Z$ is called the augmented process of CSM $X$. Since,
for our case, the components of the CSM $X$ are driven by a single Poisson
random measure, they are not independent. In view of this, it is interesting
to derive the law of $X$ by finding the the infinitesimal generator
$\mathcal{A}$ of the augmented process $Z=(Z^{1},Z^{2})$. Under (A3),
$\mathcal{A}$ is obtained below using Itô’s lemma for r.c.l.l. semi-
martingales. Let
$\varphi\colon(\mathcal{X}\times\mathbb{R}_{+}\times\mathbb{N}_{0})^{2}\to\mathbb{R}$
be bounded and continuously differentiable in its continuous variables. Then
using (2.9)

$\displaystyle d\varphi(Z^{1}_{t},Z^{2}_{t})-\left(\frac{\partial}{\partial
y_{1}}+\frac{\partial}{\partial y_{2}}\right)\varphi(Z^{1}_{t},Z^{2}_{t})dt$
$\displaystyle=\varphi(Z^{1}_{t},Z^{2}_{t})-\varphi(Z^{1}_{t-},Z^{2}_{t-})$
$\displaystyle=\varphi\left(Z^{1}_{t-}+\int_{\mathbb{R}_{+}}J(Z^{1}_{t-},v)\wp(dt,dv),Z^{2}_{t-}+\int_{\mathbb{R}_{+}}J(Z^{2}_{t-},v)\wp(dt,dv)\right)-\varphi(Z^{1}_{t-},Z^{2}_{t-})$
$\displaystyle=\int_{\mathbb{R}_{+}}\left[\varphi(Z^{1}_{t-}+J(Z^{1}_{t-},v),Z^{2}_{t-}+J(Z^{2}_{t-},v))-\varphi(Z^{1}_{t-},Z^{2}_{t-})\right]\wp(dt,dv)$
$\displaystyle=\left(\int_{\mathbb{R}_{+}}\left[\varphi(Z^{1}_{t-}+J(Z^{1}_{t-},v),Z^{2}_{t-}+J(Z^{2}_{t-},v))-\varphi(Z^{1}_{t-},Z^{2}_{t-})\right]dv\right)dt+dM_{t}$

where $M$ is the martingale obtained by integration wrt the compensated
Poisson random measure $\wp(dt,dv)-dtdv$. For simplifying the above integral
term, we impose (A3) and divide the derivation in two complementary cases.

Case 1: Assume $X^{1}_{t-}\neq X^{2}_{t-}$. Now under (A3), the intervals
$\Lambda_{X^{1}_{t-}j_{1}}(Y^{1}_{t-},N^{1}_{t-})$ and
$\Lambda_{X^{2}_{t-}j_{2}}(Y^{2}_{t-},N^{2}_{t-})$ are disjoint for every
$j_{1}\in\mathcal{X}\setminus\\{X^{1}_{t-}\\},j_{2}\in\mathcal{X}\setminus\\{X^{2}_{t-}\\}$
and $N^{1}_{t-},N^{2}_{t-}$. Thus by considering these intervals where the
integrand is non-zero constants, we get

$\displaystyle\int_{\mathbb{R}_{+}}\left[\varphi(Z^{1}_{t-}+J(Z^{1}_{t-},v),Z^{2}_{t-}+J(Z^{2}_{t-},v))-\varphi(Z^{1}_{t-},Z^{2}_{t-})\right]dv$
$\displaystyle=\int_{\overset{2}{\underset{k=1}{\bigcup}}\left(\underset{j\in\mathcal{X}\setminus\\{X^{k}_{t-}\\}}{\cup}\Lambda_{X^{k}_{t-}j}(Y^{k}_{t-},N^{k}_{t-})\right)}\left[\varphi(Z^{1}_{t-}+J(Z^{1}_{t-},v),Z^{2}_{t-}+J(Z^{2}_{t-},v))-\varphi(Z^{1}_{t-},Z^{2}_{t-})\right]dv$
$\displaystyle=\underset{j\in\mathcal{X}\setminus\\{X^{1}_{t-}\\}}{\sum}\left[\varphi(j,0,N^{1}_{t-}+1,Z^{2}_{t-})-\varphi(Z^{1}_{t-},Z^{2}_{t-})\right]|\Lambda_{X^{1}_{t-}j}(Y^{1}_{t-},N^{1}_{t-})|$
$\displaystyle\;\;\;\;\;+\underset{j\in\mathcal{X}\setminus\\{X^{2}_{t-}\\}}{\sum}\left[\varphi(Z^{1}_{t-},j,0,N^{2}_{t-}+1)-\varphi(Z^{1}_{t-},Z^{2}_{t-})\right]|\Lambda_{X^{2}_{t-}j}(Y^{2}_{t-},N^{2}_{t-})|$

where $|I|$ is the length of the interval $I$.

Case 2: Assume that $X^{1}_{t-}=X^{2}_{t-}=i$ say. Also recall that under
(A3), the intervals $\Lambda_{ij}(y_{1},n_{1})$ and
$\Lambda_{ij}(y_{2},n_{2})$ are having identical left end points for almost
every $y_{1},y_{2}\geq 0$ and $n_{1},n_{2}\in\mathbb{N}_{0}$. So,
$\Lambda_{X^{1}_{t-}j_{1}}(Y^{1}_{t-},N^{1}_{t-})$ and
$\Lambda_{X^{2}_{t-}j_{2}}(Y^{2}_{t-},N^{2}_{t-})$ are not disjoint when
$j_{1}=j_{2}$. Therefore,

$\displaystyle\int_{\mathbb{R}_{+}}\left[\varphi(Z^{1}_{t-}+J(Z^{1}_{t-},v),Z^{2}_{t-}+J(Z^{2}_{t-},v))-\varphi(Z^{1}_{t-},Z^{2}_{t-})\right]dv$
$\displaystyle=\int_{\underset{\begin{subarray}{c}j\in\mathcal{X}\setminus\\{i\\}\end{subarray}}{\cup}\left(\Lambda_{ij}(Y^{1}_{t-},N^{1}_{t-})\cup\Lambda_{{ij}}(Y^{2}_{t-},N^{2}_{t-})\right)}[\varphi(Z^{1}_{t-}+J(Z^{1}_{t-},v),Z^{2}_{t-}+J(Z^{2}_{t-},v))-\varphi(Z^{1}_{t-},Z^{2}_{t-})]dv$
$\displaystyle=\underset{j\in\mathcal{X}\setminus\\{i\\}}{\sum}[\varphi(j,0,N^{1}_{t-}+1,Z^{2}_{t-})-\varphi(Z^{1}_{t-},Z^{2}_{t-})]|\Lambda_{ij}(Y^{1}_{t-},N^{1}_{t-})\setminus\Lambda_{ij}(Y^{2}_{t-},N^{2}_{t-})|$
$\displaystyle\;\;\;\;+\underset{j\in\mathcal{X}\setminus\\{i\\}}{\sum}[\varphi(Z^{1}_{t-},j,0,N^{2}_{t-}+1)-\varphi(Z^{1}_{t-},Z^{2}_{t-})]|\Lambda_{ij}(Y^{2}_{t-},N^{2}_{t-})\setminus\Lambda_{ij}(Y^{1}_{t-},N^{1}_{t-})|$
$\displaystyle\;\;\;\;+\underset{j\in\mathcal{X}\setminus\\{i\\}}{\sum}[\varphi(j,0,N^{1}_{t-}+1,j,0,N^{2}_{t-}+1,)-\varphi(Z^{1}_{t-},Z^{2}_{t-})]|\Lambda_{ij}(Y^{1}_{t-},N^{1}_{t-})\cap\Lambda_{ij}(Y^{2}_{t-},N^{2}_{t-})|.$

Hence by combining the expressions under both the cases,

$\displaystyle\mathcal{A}\varphi(z_{1},z_{2})$
$\displaystyle:=\left(\frac{\partial}{\partial y_{1}}+\frac{\partial}{\partial
y_{2}}\right)\varphi(z_{1},z_{2})+\int_{\mathbb{R}_{+}}\left[\varphi(z_{1}+J(z_{1},v),z_{2}+J(z_{2},v))-\varphi(z_{1},z_{2})\right]dv$
$\displaystyle=\left(\frac{\partial}{\partial y_{1}}+\frac{\partial}{\partial
y_{2}}\right)\varphi(z_{1},z_{2})+\underset{j\in\mathcal{X}\setminus\\{i_{1}\\}}{\sum}(\lambda_{i_{1}j}(y_{1},n_{1})-\delta_{i_{1}i_{2}}\lambda_{i_{2}j}(y_{2},n_{2}))^{+}[\varphi(j,0,n_{1}+1,z_{2})-\varphi(z_{1},z_{2})]$
$\displaystyle\;\;\;\;+\underset{j\in\mathcal{X}\setminus\\{i_{2}\\}}{\sum}(\lambda_{i_{2}j}(y_{2},n_{2})-\delta_{i_{1},i_{2}}\lambda_{i_{1}j}(y_{1},n_{1}))^{+}[\varphi(z_{1},j,0,n_{2}+1)-\varphi(z_{1},z_{2})]$
$\displaystyle\;\;\;\;+\delta_{i_{1},i_{2}}\underset{j\in\mathcal{X}\setminus\\{i_{1},i_{2}\\}}{\sum}(\lambda_{i_{1}j}(y_{1},n_{1})\wedge\lambda_{i_{2}j}(y_{2},n_{2}))[\varphi(j,0,n_{1}+1,j,0,n_{2}+1)-\varphi(z_{1},z_{2})]$
(4.3)

where $z_{1}=(i_{1},y_{1},n_{1})$, $z_{2}=(i_{2},y_{2},n_{2})$, $\delta_{ij}$
is Kronecker delta, $a^{+}=\max(0,a)$ and $a\wedge b=\min(a,b)$. Thus we have
proved the following theorem.

######  Theorem 4.3.

Under (A3), the infinitesimal generator $\mathcal{A}$ of the augmented process
$Z=(Z^{1},Z^{2})$ is given by (4) where $Z^{1}=(X^{1},Y^{1},N^{1})$ and
$Z^{2}=(X^{2},Y^{2},N^{2})$ are as in Notation 4.2.

##  5\. Conclusion

We study strong solutions of a two-dimensional jump SDE driven by a Poisson
random measure with Lebesgue intensity. The coefficients are chosen depending
on a given transition rate function and an additional gaping parameter. Since,
the coefficients are not compactly supported and also the intensity measure is
not finite, the existence result is not straightforward. We have first proved
the local existence and then established the global existence of a unique
strong solution. We then show that the state component of the solution is a
pure semi-Markov process with the given transition rate function and the other
component is the age process. Although the law of a single solution does not
depend on the additional gaping parameter, the joint distribution of a couple
of solutions with different initial conditions does depend on that. Under a
simplified assumption on the gaping parameter, we derive the law of the
bivariate process by calculating the infinitesimal generator of the augmented
process.

To the best of our knowledge, the SDE under consideration or its any
generalizations have not been studied in the literature. The approach of the
proof of existence is significantly original. The SDE also gives a semi-
Martingale representation of a semi-Markov process which need not be time-
homogeneous. This representation is also not present in the literature. The
detailed proof of the fact that the solution gives a semi-Markov process with
desired transition kernel is valuable for all future study of this
representation. This is a vital contribution of this paper. Finally, the
semimartingale representation has been used to generate a correlated semi-
Markov system with multiple members.

In veiw of the immense applicability of the correlated semi-Markov system, the
formulation and the results presented in this paper are important. It is
evident that SDE (2.6)-(2.8) generates a semi-Markov flow. Using the explicit
construction of the solution of the SDE, the questions related to the meeting
and merging events of multiple particles of a semi-Markov flow can be
addressed in future studies.

## Acknowledgement

The authors are grateful to Tanmay Patankar for some helpful discussions.

## Appendix A

######  Lemma A.1.

For each fixed $\omega\in\Omega,$ consider the set
$\mathcal{D}:=\\{s^{\prime}\in\ (0,\
\infty)\mid\wp(\omega)(\\{s^{\prime}\\}\times E)>0\\},$ where
$E\in\mathcal{B}(\mathbb{R}),$ and $\wp$ is a Poisson random measure with
intensity $m_{2}.$ If $m_{1}(E)<\infty,$ then set $\mathcal{D}$ has no limit
point in $\mathbb{R}$ almost surely.

Proof.  If $m_{1}(E)<\infty$, for any natural number $n$,
$\wp(\omega)([0,n]\times E)$ is a Poisson random variable with mean $n\times
m_{1}(E)$. Hence $P(\wp([0,n]\times E)<\infty)=1$. Thus $\mathcal{D}\cap[0,n]$
is finite with probability 1 for each $n\geq 1$. Hence
$P(\underset{n>1}{\cap}\\{\omega\ \mid\mathcal{D}\cap[0,n]\textrm{ is
finite}\\})=1$. Therefore, $\mathcal{D}$ has no limit point in $\mathbb{R}$
w.p. $1.$

## References

  * [1] K. B. Athreya, D. McDonald, and P. Ney, Limit Theorems for Semi-Markov Processes and Renewal Theory for Markov Chains, The Annals of Probability, 6 (1978), pp. 788 – 797.
  * [2] E. Çınlar, Probability and Stochastics, Graduate Texts in Mathematics, Springer New York, 2011.
  * [3] M. K. Das, A. Goswami, and T. S. Patankar, Pricing derivatives in a regime switching market with time inhomogenous volatility, Stochastic Analysis and Applications, 36 (2018), pp. 700–725.
  * [4] M. K. Das, A. Goswami, and N. Rana, Risk sensitive portfolio optimization in a jump diffusion model with regimes, SIAM Journal on Control and Optimization, 56 (2018), pp. 1550–1576.
  * [5] R. J. Elliott, The semi-martingale dynamics and generator of a continuous time semi-Markov chain, Journal of Stochastic Analysis, 1 (2020), p. 1.
  * [6] B. L. Fox, Semi-Markov Processes: A Primer, RAND Corporation, Santa Monica, CA, 1968.
  * [7] M. K. Ghosh and A. Goswami, Risk minimizing option pricing in a semi-Markov modulated market, SIAM J. Control. Optim., 48 (2009), pp. 1519–1541.
  * [8] M. K. Ghosh and S. Saha, Stochastic Processes with Age-Dependent Transition Rates, Stochastic Analysis and Applications, 29 (2011), pp. 511–522.
  * [9] B. Harlmov, Continuous semi‐Markov processes, John Wiley & Sons, Ltd, 2008.  doi: https://doi.org/10.1002/9780470610923.
  * [10] J. M. Hoem, Inhomogeneous semi-markov processes, select actuarial tables, and duration-dependence in demography, 1972.
  * [11] N. Ikeda and S. Watanabe, Stochastic differential equations and diffusion processes, North-Holland, Amsterdam, 1981\.
  * [12] J. Janssen and R. de Dominicis, Finite non-homogeneous semi-markov processes: Theoretical and computational aspects, Insurance: Mathematics and Economics, 3 (1984), pp. 157–165.
  * [13] J. Janssen and R. Manca, Numerical solution of non-homogeneous semi-markov processes in transient case*, Methodology And Computing In Applied Probability, 3 (2001), pp. 271–293.
  * [14] , Applied semi-Markov processes, Springer Science & Business Media, 2006.
  * [15] , Semi-Markov risk models for finance, insurance and reliability, Springer Science & Business Media, 2007.
  * [16] S. Kotz, I. Gikhman, and A. Skorokhod, The Theory of Stochastic Processes II, Classics in Mathematics, Springer Berlin Heidelberg, 1977.
  * [17] P. Levy, Systemes semi-markoviens ’aau plus une infinite denombrable d’etats possibles, Proc. Int. Congr.Math., Amsterdam, 2 (1954), p. 294.
  * [18] A. I. Manu, Non homogeneous semi-markov processes, Stud. Lere. Mat, (1972), pp. 529–533.
  * [19] E. Nummelin, Limit theorems for $\alpha$-recurrent semi-Markov processes, Advances in Applied Probability, 8 (1976), p. 531–547.
  * [20] S. Orey, Change of Time Scale For Markov Processes, Transactions of the American Mathematical Society, 99 (1961), pp. 384–397.
  * [21] T. Patankar, Assset Pricing in a Semi-Markov Modulated Market with Time-dependent Volatility, Master’s thesis, IISER Pune, 2016.
  * [22] R. Pyke, Markov renewal processes: definitions and preliminary properties, The Annals of Mathematical Statistics, (1961), pp. 1231–1242.
  * [23] , Markov renewal processes with finitely many states, The Annals of Mathematical Statistics, (1961), pp. 1243–1259.
  * [24] W. L. Smith, Regenerative stochastic processes, Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences, 232 (1955), pp. 6–31.
  * [25] L. Takács, Some investigations concerning recurrent stochastic processes of a certain type, Maygyar Tud. Akad. Mad. Kutató int. Kozl, 3 (1954), pp. 115–128.
  * [26] A. Vasileiou and P.-C. G. Vassiliou, An inhomogeneous semi-markov model for the term structure of credit risk spreads, Advances in Applied Probability, 38 (2006), p. 171–198.
  * [27] P.-C. Vassiliou, Exotic properties of non homogeneous markov and semi-markov systems, Communications in Statistics - Theory and Methods, 42 (2013), pp. 2971–2990.
  * [28] P.-C. G. Vassiliou and A. A. Papadopoulou, Non-homogeneous semi-markov systems and maintainability of the state sizes, Journal of Applied Probability, 29 (1992), p. 519–534.