# Approximation of A Class of Non-Zero-Sum Investment and Reinsurance Games
for Regime-Switching Jump-Diffusion Models

Trang Bui Department of Mathematics, Wayne State University, Detroit, MI
48202. Email<EMAIL_ADDRESS>The research was supported in part by the
Air Force Office of Scientific Research under Grant FA9550-18-1-0268. Xiang
Cheng Centre for Actuarial Studies, Department of Economics, The University of
Melbourne, VIC 3010, Australia<EMAIL_ADDRESS>Zhuo Jin Centre for
Actuarial Studies, Department of Economics, The University of Melbourne, VIC
3010, Australia<EMAIL_ADDRESS>George Yin Department of Mathematics,
Wayne State University, Detroit, MI 48202. Email<EMAIL_ADDRESS>The
research was supported in part by the Air Force Office of Scientific Research
under Grant FA9550-18-1-0268.

###### Abstract

This work develops an approximation procedure for a class of non-zero-sum
stochastic differential investment and reinsurance games between two insurance
companies. Both proportional reinsurance and excess-of loss reinsurance
policies are considered. We develop numerical algorithms to obtain the Nash
equilibrium by adopting the Markov chain approximation methodology and
applying the dynamical programming principle for the nonlinear integro-
differential Hamilton-Jacobi-Isaacs (HJI) equations. Furthermore, we establish
the convergence of the approximation sequences and the approximation to the
value functions. Numerical examples are presented to illustrate the
applicability of the algorithms.

Key Words. Stochastic control, non-zero-sum game, investment, reinsurance,
regime switching.

##  1 Introduction

Insurers tend to accumulate relatively large amounts of cash or cash
equivalents through the written insurance portfolio. Investing the surplus in
a financial market in order to pay future claims and to avoid financial ruin
becomes a natural choice. In terms of financial performance the investment
income allows significant pricing flexibility in underwriting to the insurers.
The surplus is allowed to be invested in a financial market in continuous
time.

On the other hand, reinsurance has been considered as an effective risk
management tool for insurance companies to transfer their risk exposure to
another commercial institution. The primary insurer pays the reinsurer a
certain portion of the premiums. In return, the reinsurer is obliged to share
the risk of large claims with the primary insurer. Proportional reinsurance
and excess-of-loss reinsurance are two major types of reinsurance strategies.
With proportional reinsurance, the reinsurance company covers a fixed
percentage of losses. The fraction of risk shared by the reinsurance company
is determined when the reinsurance contract is sold. The other type of
reinsurance policy is nonproportional reinsurance. The most common
nonproportional reinsurance policy is excess-of-loss reinsurance, where the
primary insurance carrier (called cedent) will pay all of the claims up to a
predetermined amount (termed retention level).

The optimal risk controls for an insurance corporation has been studied
extensively since the classical collective risk model was introduced by
Lundberg,  (1903). The insurance companies can reduce or eliminate the risk of
loss by involving in a reinsurance program and reinvesting in the stock
market. Particularly, advanced stochastic control theory and dynamic
programming principle are widely used to design the optimal reinsurance and
investment strategies of an insurance company in previous works. Asmussen et
al.,  (2000) used diffusion approximation to find an optimal policy balancing
the risk and expected profits for a financial corporation with excess-of-loss
reinsurance. Choulli et al.,  (2001) investigated the case of excess-of-loss
reinsurance for an insurance company facing a constant liability payment to
maximize the expected present value of total future dividend pay-outs. Hald
and Schimidli,  (2004) studied the optimal proportional reinsurance policy to
maximize the adjustment coefficient of the ruin probability with the variance
premium principle; see also Browne,  (1995), Chen et al.,  (2010), Zhang and
Siu,  (2012), and Zhang et al.,  (2016).

Recently, the extension of optimal investment and reinsurance problem (Nash
equilibrium) in the context of stochastic differential games including zero-
sum games and non-zero-sum games has been developed rapidly; see the existence
of the Nash equilibrium of non-zero-sum stochastic differential game with $N$
players over infinite time horizon in Bensoussan and Frehse,  (2000). The
existence of the Nash equilibrium of a non-zero-sum stochastic differential
game between two insurance companies in Zeng,  (2010). Liu and Yiu,  (2013)
studied a zero-sum stochastic differential reinsurance and investment game
between two competing insurance companies under VaR constraints for the
purpose of risk management. Bensoussan et al.,  (2014) investigated a class of
non-zero sum stochastic differential game between two insurers by using the
objectives of relative performance and obtained explicit solutions for optimal
reinsurance and investment strategies. Chen and Shen,  (2017) formulated a
stochastic Stackelberg differential reinsurance game between an insurer and a
reinsurer, allowing them to consider the benefits of both parties in the
reinsurance contracts. Under the criteria of maximizing the expected utilities
of the players’ terminal surpluses, the Stackelberg equilibrium strategies and
value functions are obtained by using the backward stochastic differential
equation (BSDE) approach; see also Taksar and Zeng,  (2011), Pun and Wong,
(2016), and Yan et al.,  (2017).

Furthermore, people have recently realized that stochastic hybrid models have
advantages to capture discrete movements (such as random environment, market
trends, interest rates, business cycles, etc.) in the insurance market. For
example, we can consider an insurance market with two modes to represent the
dynamic insurance cycle. Market mode 1 represents a “soft” market, where the
investment return is high and the premium rate is low. While market mode 2
represents a “hard” market, where the investment return is low and the premium
rate is high. In different markets, insurance companies adopt completely
different strategies in investment and policy sales. The insurance company is
more likely to expand its business and write more policies when market is in
mode 1. The potential loss from the difference between premium income and
claims are compensated by the high investment returns. The hybrid systems
enable the consideration of the coexistence of continuous dynamics and
discrete events in the systems. To reflect the hybrid feature, one of the
recent trends is to use a finite state Markov process to describe the
transitions among different regimes. The Markov-modulated switching systems
are known as regime-switching systems. The formulation of regime-switching
models is a more general and versatile framework to describe the complicated
financial markets and their inherent uncertainty and randomness. Because the
control strategies are affected by the asset prices on the stock market and
economic trends change quickly, Markovian regime-switching processes were
introduced widely to capture movements of random environment. In Wei et al.,
(2010), the optimal dividend and proportional reinsurance strategy under
utility criteria are studied for the regime-switching compound Poisson model.
Sotomayor and Cadenillas,  (2011) studies the optimal dividend problem in the
regime-switching model when the dividend rates are bounded, unbounded, and
when there are fixed costs and taxes corresponding to the dividend payments.
Zhu,  (2014) studied the dividend optimization for a regime-switching
diffusion model with restricted dividend rates. A comprehensive study of
switching diffusions with “state-dependent” switching is in Yin and Zhu,
(2010). Bensoussan et al.,  (2014) provided closed-form Nash equilibrium for a
mixed regime-switching Cramér-Lundberg diffusion approximation process; see
also related works Bensoussan et al.,  (2012) and Bensoussan et al.,  (2017)
for regime-switching models for real options and real options with
competition. Jin et al.,  (2013) designed numerical methods for a zero-sum
stochastic differential reinsurance game with regime-switching.

In this work, we are concerned with an insurance market including two
insurance companies. The two competing insurance companies adopt optimal
investment and reinsurance strategies to manage the insurance portfolios. The
surplus process of each insurance company is subject to the randomness of the
market. Following the work of Bensoussan et al.,  (2014), the randomness of
the market is modelled by a continuous-time finite-state Markov chain and an
independent market-index process. Nevertheless, we model the surplus process
as a regime-switching jump-diffusion process, in lieu of a mixed regime-
switching Cramér-Lundberg diffusion approximation process. This allows us to
work with both proportional and excess-of-loss reinsurance policies.
Equilibrium strategies are studied by solving a system of HJI (Hamilton-
Jacobi-Isaacs) equations for the value functions of various players, derived
from the principle of dynamic programming. Owing to the inclusion of the
random switching environment and jump processes, the system of HJI equations
becomes more complicated and closed-form solutions are virtually impossible to
obtain. Thus, we adopt the Markov chain approximation method (MCAM) developed
in Kushner and Dupuis,  (2001) to deal with a system of HJI (HJI) equations
arising from the associated game problems. The convergence of the
approximation sequence to the jump process and the convergence of the value
function will be established. In the actual computation, we will use our
approximation schemes for constant absolute risk aversion (CARA) insurers.

As far as the significance of the contributions is concerned, this paper
reveals clearly the advantage in the following aspects. First, the problem of
finding the optimal investment and reinsurance strategy between two insurance
companies under different insurance market models involves a stochastic hybrid
system. Closed-form solutions can only be obtained in special cases in
Bensoussan et al.,  (2014) in a diffusion approximation model. Comparing with
work in Bensoussan et al.,  (2014), a more versatile jump-diffusion regime-
switching non-zero-sum game model is formulated. Jin et al.,  (2013) only
studies a zero-sum reinsurance game, where the state process is degenerated to
one dimension. While in the non-zero-sum game of the current work, a high-
dimension system of HJI equations is obtained, which adds much difficulty to
construct the approximating Markov chain. Second, the aforementioned
literature mainly considers proportional reinsurance strategies in the
reinsurance games. Pun and Wong,  (2016) derives the Nash equilibrium of the
robust non-zero-sum stochastic differential reinsurance game where the
reinsurance strategies are proportional. Due to the unboundedness of the
excess-of-loss strategies, it is very hard to develop the Nash equilibrium
strategies for an reinsurance game. In addition to the investment, finding the
explicit optimal game strategies are virtually impossible. To the best of our
knowledge, this is the first paper to study the non-zero-sum investment and
reinsurance games in a jump-diffusion regime-switching with nonproportional
reinsurance strategies. Third, the reinsurance premium principle, which is a
generalized nonlinear functional in our model, adds further difficulties to
design the game strategies. Meng et al.,  (2013) and Meng et al.,  (2015)
investigate the nonlinear premium principles in a simplified reinsurance game
formulation. Our models considers the investment and reinsurance game in more
complex formulation and presents the impact of the nonlinearity of the premium
principle on investment and reinsurance strategies. Hence, the constructed
numerical algorithm shows its advantage in finding the game strategies under
our complex stochastic hybrid system.

The rest of the paper is organized as follows. A generalized formulation for
surplus processes and the associated control problem are presented in Section
2. We design the numerical algorithm based on MCAM in Section 3. A directly
constructed approximating Markov chain is constructed and is proved to be
locally consistent with the original processes. Section 4 deals with the
convergence of the approximated process and the value functions. Numerical
examples are reported in Section 5 to illustrate the performance of the
method. Section 6 concludes the paper with a few more remarks.

##  2 Formulation

Let $(\Omega,{\cal F},{\cal F}_{t},P)$ be a complete filtered probability
space, where the filtration $\\{{\cal F}_{t}\\}_{t\geq 0}$ satisfies the usual
condition and ${\cal F}_{t}={\cal F}_{t}^{\alpha}\vee{\cal F}_{t}^{W}$. We
work with a finite horizon $[0,T]$, where $T<\infty$ is a positive real
number. The processes $\\{\alpha(t)\\}_{t\geq 0}$ and $\\{W(t)\\}_{t\geq
0}=\\{W_{Z}(t),W_{S}(t)\\}_{t\geq 0}$ are defined below. We use $K$ as a
generic constant throughout the paper, whose value may change for different
appearances.

###  2.1 Insurance Models

We are considering an insurance market including two competing insurance
companies. Each of the two insurance companies adopts optimal investment and
reinsurance strategies to manage the insurance portfolios. The surplus process
of each insurance company is subject to the random fluctuation of the market.
Following the work of Bensoussan et al.,  (2014), the randomness of the market
is modelled by a continuous-time finite-state Markov chain and an independent
market-index process.

To delineate the random economy environment and other random economic factors,
we use a continuous-time Markov chain $\alpha(t)$ taking values in a finite
space ${\cal M}=\\{1,\ldots,m\\}$. The states of economy are represented by
the Markov chain $\alpha(t)$. Let the continuous-time Markov chain $\alpha(t)$
be generated by $Q=(q_{ij})\in{\mathbb{R}}^{m\times m}$. That is,

${P}\\{\alpha(t+\delta)=j|\alpha(t)=i,\alpha(s),s\leq
t\\}=\left\\{\begin{array}[]{ll}q_{ij}\delta+o(\delta),\qquad\hfill{\rm if}\
j\neq i,\\\ 1+q_{ii}\delta+o(\delta),\qquad\hfill{\rm if}\
j=i,\end{array}\right.$ (2.1)

where $q_{ij}\geq 0$ for $i,j=1,2,\ldots,m$ with $j\neq i$ and
$\sum_{j\in{\cal M}}q_{ij}=0$ for each $i\in{\cal M}$.

Furthermore, we are considering the insurance portfolios in a financial market
with a market index $Z(t)$, whose prices satisfies

$dZ(t)=\mu_{Z}(t,Z(t))dt+\sigma_{Z}(t,Z(t))dW_{Z}(t),$ (2.2)

where $W_{Z}(t)$ is a standard Brownian motion. Denote by $\\{{\cal
F}_{t}^{W_{Z}}\\}$ the filtration generated by Brownian motion
$\\{W_{Z}(t)\\}_{t\geq 0}$. We note that $Z(t)$ captures the dynamics of the
financial market. The cash flows of the insurance companies such as the
premiums of insurance policies, claims, and expenses, are subject to the
performance of financial market. Hence, the key parameters of the surplus
process are defined as functionals of both the finite-state Markov chain
$\alpha(t)$ and market index $Z(t)$.

Following the classical Cramér-Lundberg process, we assume that
$\widehat{X}_{k}(t),k\in\\{1,2\\}$, the surplus of insurance company $k$
without investment and reinsurance satisfies

$d\widehat{X}_{k}(t)=c_{k}(\alpha(t),Z(t))dt-Y_{k}(t),\ \ t\geq 0,$ (2.3)

where $\widehat{X}_{k}(0):=\hat{x}_{k}$ is the initial surplus,
$c_{k}(\alpha(t),Z(t))$ is the rate of premium, and
$Y_{k}(t)=\displaystyle\sum_{i=1}^{N_{k}(t)}A^{k}_{i}$ is a compound Poisson
process with the claim size $A^{k}_{i}$ with $\\{A^{k}_{i}:i>1\\}$ being a
sequence of positive, independent and identically distributed random
variables.

In this paper, we consider a Poisson measure in lieu of the traditionally used
Poisson process. Suppose $\Theta\subset{\mathbb{R}}_{+}$ is a compact set and
the function $q_{k}(\cdot)$ is the magnitude of the claim sizes.

$N_{k}(t,H)=\hbox{ number of claims on }[0,t]\hbox{ with claim size taking
values in }H\in\Theta,$ (2.4)

counts the number of claims up to time $t$, which is a Poisson counting
process. For $k=1,2$, $Y_{k}(t)$ is jump processes representing claims for
each company with arrival rate $\lambda_{k}$. Note that claim frequencies
depends on the economy and financial market states. The function
$q_{k}(\alpha(t),Z(t),\rho_{k})$ is assumed to be the magnitude of the claim
sizes, where $\rho_{k}$ has distribution $\Pi_{k}(\cdot)$, and
$q_{k}(i,\cdot,\rho_{k})$ is continuous for each $\rho_{k}$ and each
$i\in{\cal M}$. At different regimes and financial market states, taking into
consideration of random environment, the values of $q_{k}(i,\cdot,\rho_{k})$
could be much different. Then the Poisson measure $N_{k}(\cdot,\cdot)$ has
intensity $\lambda_{k}dt\times\Pi_{k}(d\rho_{k})$ where
$\Pi_{k}(d\rho_{k})=f(\rho_{k})d\rho_{k}$.

Let $\nu_{k,n}$ denote the time of the $n$-th claim and
$\zeta_{k,n}=\nu_{k,n+1}-\nu_{k,n}$. Let $\\{\zeta_{k,v},\rho_{k},v\geq n\\}$
be independent of
$\\{X_{k}(s),\alpha(s),s\leq\nu_{k,n},\zeta_{k,v},\rho_{k},v\leq n\\}$. Then
the $n$-th claim term is $q_{k}(\alpha(\nu_{k,n}),Z(\nu_{k,n}),\rho_{k})$, and
the claim amount of $Y_{k}$ can be written as $\displaystyle
Y_{k}(t)=\sum_{\nu_{k,n}\leq t}q_{k}(\alpha(\nu_{k,n}),(\nu_{k,n}),\rho_{k})$.

###  2.2 Reinsurance and Investment

Let $a_{k}(t)$ be the ${\cal F}_{t}$-progressively measurable process valued
in $[0,1]$, an exogenous retention level, which is a control chosen by the
insurance company representing the reinsurance policy and $g(a_{k})$ is the
reinsurance premium rate. Denote by ${\cal A}_{k}:=\\{a_{k}(t):0\leq
a_{k}(t)\leq 1,0\leq t\leq T\\}$, the set of reinsurance strategies of insurer
$k$. Recall that $A^{k}_{i}$ is the size of the $i$th claim. Let
$A^{k}_{i}(a_{k})$ be the fraction of each claim paid by the primary insurance
company. Then the aggregation claim amount paid by the primary insurance
company is denoted as $Y_{k}^{a_{k}}(t)$.

######  Remark 2.1.

Note that both the claim frequencies and severities are depending on the
Markov regimes and market index. It is a more general formulation compared
with the work in Bensoussan et al.,  (2014), where only the claim frequencies
depends on the Markov regimes and market index. Therefore, the diffusion
approximation in Bensoussan et al.,  (2014) is simplified due to the state-
independence of claim severity. Further, with the compound Poisson jumps, the
surplus process forms a controlled jump-diffusion regime-switching process. We
aim to find optimal reinsurance strategies under the jump-diffusion regime-
switching process formulation numerically.

The insurance companies invest in both risk-free assets $S_{0}(t)$ and risky
assets $S(t)$ with prices satisfying

$\left\\{\begin{array}[]{ll}&\\!\\!\\!\displaystyle\frac{dS_{0}(t)}{S_{0}(t)}=r(\alpha(t))dt,\\\
&\\!\\!\\!\displaystyle\frac{dS(t)}{S(t)}=\mu_{S}(\alpha(t),Z(t))dt+\sigma_{S}(\alpha(t),Z(t))dW_{S}(t),\end{array}\right.$
(2.5)

where $r(\alpha(t))$ and $\mu(\alpha(t),Z(t))$ are the return rates of the
risk-free and risky assets, respectively; $\sigma_{S}(\alpha(t),Z(t))$ is the
corresponding volatility; $W_{S}(t)$ is a standard Brownian motion independent
of $W_{Z}(t)$. For $k=1,2,$ the investment behavior of the insurer $k$ is
modelled as a portfolio process $b_{k}(t)$, where $b_{k}(t)$ is invested in
the risky asset $S(t)$. Let ${\cal B}_{k}=\\{b_{k}(t):0\leq t\leq T\\}$ denote
the set of investment strategies of insurer $k$.

Combining the reinsurance and investment strategies, the surplus process of
the insurance company $k$, denoted by $\widetilde{X}_{k}(t)$, follows

$\begin{cases}d\widetilde{X}_{k}(t)=\left\\{r(\alpha(t))\widetilde{X}_{k}(t)+b_{k}(t)[\mu_{S}(\alpha(t),Z(t))-r(\alpha(t))]+c_{k}(\alpha(t),Z(t))-g(a_{k}(t))\right\\}dt\\\
\hskip
45.0pt+b_{k}(t)\sigma_{S}(\alpha(t),Z(t))dW_{S}(t)-dY_{k}^{a_{k}}(t),\\\
\widetilde{X}_{k}(0)=\widetilde{x}_{k},\end{cases}$ (2.6)

where $\displaystyle
Y_{k}^{a_{k}}(t)=\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\tilde{q}_{k}(q_{k},a_{k})N_{k}(dt,d\rho_{k})$
and $\tilde{q}_{k}(q_{k},a_{k})$ is the magnitude of the claim sizes with
respect to the surplus process.

In this work, we model the competition of two insurance companies with
investment and reinsurance schemes in finite time horizon using a game
theoretic formulation. The performance of each company is measured by the
relative performance of their surpluses against their competitor’s. Thus, the
competition between the two companies formulates a game with two players, each
of which can adjust its reinsurance strategies based on the competitor’s
scheme. Let the relative surplus performance for insurance company $k$ be
$X_{k}(t):=\widetilde{X}_{k}(t)-\kappa_{k}\widetilde{X}_{l}(t)$. Hence,
$X_{k}(t)$ is governed by the following dynamic system

$\begin{array}[]{ll}&\\!\\!\\!\displaystyle dX_{k}(t)\\\
&\\!\\!\\!\displaystyle=\sum_{i\in{\cal
M}}I_{\\{\alpha(t)=i\\}}\bigg{\\{}r(i)X_{k}(t)+(b_{k}(t)-\kappa_{k}b_{l}(t))[\mu_{S}(i,Z(t))-r(i)]+c_{k}(i,Z(t))-\kappa_{k}c_{l}(i,Z(t))\\\
&\displaystyle\quad-(g(a_{k}(t))-\kappa_{k}g(a_{l}(t)))dt+(b_{k}(t)-\kappa_{k}b_{l}(t))\sigma_{S}(i,Z(t))dW_{S}(t)\bigg{\\}}-dY_{k}^{a_{k}}(t)+\kappa_{k}dY_{l}^{a_{l}}(t),\end{array}$
(2.7)

where

$Y_{k}^{a_{k}}(t)=\int_{0}^{t}\int_{{{\mathbb{R}}_{+}}}\tilde{q}_{k}(q_{k},a_{k})N_{k}(dt,d\rho_{k}).$

###  2.3 Proportional Reinsurance

We allow the insurance companies to continuously reinsure a fraction of its
claim with the retention level $a_{k}\in[0,1]$, $k=1,2$. Note that $a_{k}$ is
the exogenous retention level, and the control chosen by the insurance company
for the reinsurance policy. Then
$\tilde{q}(q_{k},a_{k})=a_{k}(t)q_{k}(\alpha(t),Z(t),\rho_{k})$. We have

$Y_{k}^{a_{k}}(t)=\displaystyle\sum_{i=1}^{N_{k}(t)}A^{k}_{i}(a_{k})=\sum_{i=1}^{N_{k}(t)}a_{k}A^{k}_{i}.$

Considering the proportional reinsurance strategies, for $k=1,2$. The relative
surplus process of the insurance company $k$, under the reinsurance and
investment, follows

$\begin{cases}&dX_{k}(t)=\bigg{\\{}r(\alpha(t))X_{k}(t)+(b_{k}(t)-\kappa_{k}b_{l}(t))[\mu_{S}(\alpha(t),Z(t))-r(\alpha(t))]+c_{k}(\alpha(t),Z(t))\\\
&\quad-\kappa_{k}c_{l}(\alpha(t),Z(t))-[g(a_{k}(t))-\kappa_{k}g(a_{l}(t))]\bigg{\\}}dt+(b_{k}(t)-\kappa_{k}b_{l}(t))\sigma_{S}(\alpha(t),Z(t))dW_{S}(t)\\\
&\quad-
a_{k}(t)\int_{{\mathbb{R}}_{+}}q_{k}(\alpha(t),Z(t),\rho_{k})N_{t}(dt,d\rho_{k})+\kappa_{k}a_{l}(t)\int_{{\mathbb{R}}_{+}}q_{l}(\alpha(t),Z(t),\rho_{l})N_{l}(dt,d\rho_{l})\\\
X_{k}(0)&=\widetilde{x}_{k}-\kappa_{k}\widetilde{x}_{l}.\end{cases}$ (2.8)

###  2.4 Excess-of-loss Reinsurance

We allow the insurance companies to continuously reinsure its claim and pay
all of the claims up to a pre-given level of amount (termed retention level).
We still let $a_{k}$, $k=1,2$ be the retention level chosen by the insurance
company to determine the reinsurance policy. We have that

$Y_{k}^{a_{k}}(t)=\displaystyle\sum_{i=1}^{N_{k}(t)}A^{k}_{i}(a_{k})=\sum_{i=1}^{N_{k}(t)}(A^{k}_{i}\wedge
a_{k}).$

Then $\tilde{q}_{k}(q_{k},a_{k})=q_{k}(\alpha(t),Z(t),\rho_{k})\wedge
a_{k}(t)$.

Considering the excess-of-loss reinsurance strategies, for $k=1,2$, the
relative surplus process of the insurance company $k$, under the reinsurance
control and investment, follows

$\begin{cases}dX_{k}(t)&=\bigg{\\{}r(\alpha(t))X_{k}(t)+(b_{k}(t)-\kappa_{k}b_{l}(t))[\mu_{S}(\alpha(t),Z(t))-r(\alpha(t))]+c_{k}(\alpha(t),Z(t))\\\
&\quad-\kappa_{k}c_{l}(\alpha(t),Z(t))-[g(a_{k}(t))-\kappa_{k}g(a_{l}(t))]\bigg{\\}}dt+(b_{k}(t)-\kappa_{k}b_{l}(t))\sigma_{S}(\alpha(t),Z(t))dW_{S}(t)\\\
&\quad-\int_{{\mathbb{R}}_{+}}(q_{k}(\alpha(t),Z(t),\rho_{k})\wedge
a_{k})N_{k}(dt,d\rho_{k})+\kappa_{k}\int_{{\mathbb{R}}_{+}}(q_{l}(\alpha(t),Z(t),\rho_{l})\wedge
a_{l})N_{l}(dt,d\rho_{l})\\\
X_{k}(0)&=\widetilde{x}_{k}-\kappa_{k}\widetilde{x}_{l}.\end{cases}$ (2.9)

###  2.5 Control Problem

For $k=1,2$, insurer $k$ has a utility function
$U_{k}:{\mathbb{R}}\rightarrow{\mathbb{R}}$, where $U_{k}$ is assumed to be
increasing, strictly concave, and satisfies Inada conditions, i.e.,

$\partial_{x}U_{k}(-\infty)=+\infty,\qquad\partial_{x}U_{k}(+\infty)=0.$

Following the work Espinosa and Touzi,  (2015), the insurer $k$ aims to
maximize the expected utility of his relative performance at the terminal time
$T$ by adopting a pair of investment and reinsurance strategy
$u_{k}=(a_{k},b_{k})\in{\cal A}_{k}\times{\cal B}_{k}$, denote
${\mathcal{U}}_{k}\triangleq{\cal A}_{k}\times{\cal B}_{k}$. For an arbitrary
pair of admissible control
$u=(u_{1},u_{2})\in{\mathcal{U}}\triangleq{\mathcal{U}}_{1}\times{\mathcal{U}}_{2}$,
the objective function is

$\begin{array}[]{ll}J^{k}(t,x_{k},z,i,u)&\\!\\!\\!\displaystyle={\mathbb{E}}\left[U_{k}\left((1-\kappa_{k})\widetilde{X}_{k}(T)+\kappa_{k}(\widetilde{X}_{k}(T)-\widetilde{X}_{l}(T))\right)\right]\\\
&\\!\\!\\!\displaystyle={\mathbb{E}}\left[U_{k}\left(\widetilde{X}_{k}(T)-\kappa_{k}\widetilde{X}_{l}(T)\right)\right],\end{array}$
(2.10)

for $k\neq l\in\\{1,2\\}$. For $k=1,2$, $\kappa_{k}$ measures the sensitivity
of insurer $k$ to the performance of his competitor.

The control $u_{k}=(a_{k},b_{k})$ with $k\in\\{1,2\\}$ is said to be
admissible if $a_{k}$ and $b_{k}$ satisfy

  * (i)

$a_{k}(t),b_{k}(t)$ are nonnegative for any $t\geq 0$,

  * (ii)

Both $a_{k}$, $b_{k}$ are adapted to ${\cal F}_{t}$.

  * (iii)

$J^{k}(t,x_{k},z,i,u)<\infty$ for any admissible pair $u_{k}=(a_{k},b_{k})$.

For $k=1,2,$ let $\mathfrak{B}({\mathcal{U}}_{k}\times[0,\infty))$ be the
$\sigma-$ algebra of Borel subsets of ${\mathcal{U}}_{k}\times[0,\infty)$. We
use a relaxed control formulation; see Kushner and Dupuis,  (2001) for a
definition and more discussions. Recall that an admissible relaxed control
$m_{k}(\cdot)$ is a measure on
$\mathfrak{B}({\mathcal{U}}_{k}\times[0,\infty))$ such that
$m_{k}({\mathcal{U}}_{k}\times[0,t))=t$ for each $t\geq 0$. With the given
probability space, we say that $m_{k}(\cdot)$ is an admissible relaxed
(stochastic) control for ${\cal F}$, if $m_{k}(\cdot,\omega)$ is a
deterministic relaxed control with probability one and if
$m_{k}(A\times[0,t])$ is ${\cal F}_{t}$-adapted for all
$A\in\mathfrak{B}({\mathcal{U}}_{k})$.

Given a relaxed control $m_{k}(\cdot)$ of $u_{k}(\cdot)$, we define the
derivative $m_{t,k}(\cdot)$ such that

$m_{k}(K)=\int_{{\mathcal{U}}_{k}\times[0,\infty)}I_{\\{(u_{k},t)\in
K\\}}m_{t,k}(d\phi_{k})dt$

for all $K\in\mathfrak{B}({\mathcal{U}}_{k}\times[0,\infty))$, and that for
each $t$, $m_{t,k}(\cdot)$ is a measure on $\mathfrak{B}({\mathcal{U}}_{k})$
satisfying $m_{t,k}({\mathcal{U}}_{k})=1$. For example, we can define
$m_{t,k}(\cdot)$ in any convenient way for $t=0$ and as the left-hand
derivative for $t>0$,

$m_{t,k}(A)=\lim_{\varrho\to
0}\dfrac{m_{k}(A\times[t-\varrho,t])}{\varrho},\qquad\forall
A\in\mathfrak{B}({\mathcal{U}}_{k}).$

Note that $m_{k}(d\phi_{k}dt)=m_{t,k}(d\phi_{k})dt$. It is natural to define
the relaxed control representation $m_{k}(\cdot)$ of $u_{k}(\cdot)$ by
$m_{t,k}(A)=I_{\\{u_{k}(t)\in A\\}}$ $\forall
A\in\mathfrak{B}({\mathcal{U}}_{k}).$ Define the relaxed control
$m(\cdot)=(m_{1}(\cdot)\times m_{2}(\cdot))$ with derivative
$m_{t}(\cdot)=m_{t,1}(\cdot)\times m_{t,2}(\cdot)$. Thus $m(\cdot)$ is a
measure on the Borel sets of ${\mathcal{U}}\times[0,\infty)$.

###  2.6 Nash Equilibrium

A Nash equilibrium $u^{*}=(u_{1}^{*},u_{2}^{*})\in{\mathcal{U}}$ is achieved
such that

$\begin{array}[]{ll}&\\!\\!\\!\displaystyle{\mathbb{E}}[U_{1}(\widetilde{X}_{1}^{u_{1}}(T)-\kappa_{1}\widetilde{X}_{2}^{u_{2}^{*}}(T))]\leq{\mathbb{E}}[U_{1}(\widetilde{X}_{1}^{u_{1}^{*}}(T)-\kappa_{1}\widetilde{X}_{2}^{u_{2}^{*}}(T))],\\\
&\\!\\!\\!\displaystyle{\mathbb{E}}[U_{2}(\widetilde{X}_{2}^{u_{2}}(T)-\kappa_{2}\widetilde{X}_{1}^{u_{1}^{*}}(T))]\leq{\mathbb{E}}[U_{2}(\widetilde{X}_{2}^{u_{2}^{*}}(T)-\kappa_{1}\widetilde{X}_{1}^{u_{1}^{*}}(T))].\end{array}$
(2.11)

For $\alpha(t)=i\in{\cal M}$, $Z(t)=z$, and $X_{k}(t)=x_{k}$, where $0\leq
t\leq T$ and $k\neq l\in\\{1,2\\}$, the value function of insurance company
$k$ follows

$V^{k}(t,x_{k},z,i)=\sup_{u_{k}\in{\cal
H}_{k}}{\mathbb{E}}[U_{k}(\widetilde{X}_{k}^{u_{k}}(T)-\kappa_{k}\widetilde{X}_{l}^{u_{l}^{*}}(T))],$
(2.12)

where $V^{k}(\cdot,\cdot,\cdot,\cdot)$ is the value function in
${\mathbb{R}}_{+}\times{\mathbb{R}}\times{\mathbb{R}}\times{\cal M}$.

To obtain the system of Hamilton-Jacobi-Bellman (HJB) equations, we assume the
existence of optimal control. For an arbitrary $u_{k}\in{\mathcal{U}}_{k}$,
$\alpha(t)=i\in{\cal M}$, $k\neq l\in\\{1,2\\},$ and
$V^{k}(\cdot,\cdot,\cdot,i)\in
C^{2}({\mathbb{R}}_{+}\times{\mathbb{R}}\times{\mathbb{R}}\times{\cal M})$,
define an integro-differential operator ${\cal L}^{u_{k},u_{l}}$ by

$\begin{array}[]{ll}{\cal
L}^{u_{k},u_{l}}V^{k}(t,x_{k},z,i)&\\!\\!\\!\displaystyle=V^{k}_{x_{k}}(t,x_{k},z,i)\bigg{\\{}r(i)x_{k}+(b_{k}-\kappa_{k}b_{l})[\mu(i,z)-r(i)]+c_{k}(i,z)-\kappa_{k}c_{l}(i,z)\\\
&\displaystyle\quad-(g(a_{k})-\kappa_{k}g(a_{l}))\bigg{\\}}+V^{k}_{z}(t,x_{k},z,i)m(t,z)\\\
&\displaystyle\quad+\frac{1}{2}(b_{k}-\kappa_{k}b_{l})^{2}\sigma^{2}_{S}(i,z)V_{x_{k}x_{k}}(t,x_{k},z,i)+\frac{1}{2}\sigma^{2}_{Z}(t,z)V_{zz}(t,x_{k},z,i)\\\
&\displaystyle\quad-\lambda_{k}\int_{{\mathbb{R}}_{+}}[V^{k}(t,x_{k}-\tilde{q}(i,z,\rho_{k}),z,i)-V^{k}(t,x_{k},z,i)]f(\rho_{k})d\rho_{k}\\\
&\displaystyle\quad+\lambda_{l}\int_{{\mathbb{R}}_{+}}[V^{k}(t,x_{k}+\kappa_{k}\tilde{q}(i,z,\rho_{l}),z,i)-V^{k}(t,x_{k},z,i)]f(\rho_{l})d\rho_{l}\\\
&\displaystyle\quad+QV(t,x_{k},z,\cdot)(i),\end{array}$ (2.13)

where

$QV(t,x_{k},z,\cdot)(i)=\sum_{j\neq i}q_{ij}(V(t,x_{k},z,j)-V(t,x_{k},z,i)).$

Formally, for $k\neq l\in\\{1,2\\}$, we conclude that $V^{k}$ satisfies the
following system of integro-differential HJI (Hamilton-Jacobi-Isaacs)
equations: for each $i\in{\cal M}$,

$\begin{cases}&V^{k}_{t}(t,x_{k},z,i)+\displaystyle\sup_{u_{k}\in{\cal
H}_{k}}{\cal L}^{u_{k},u_{l}^{*}}V^{k}(t,x_{k},z,i)=0,\\\
&V^{k}(T,x_{k},z,i)=U_{k}(x_{k}).\end{cases}$ (2.14)

##  3 Numerical Algorithm

We begin by construction a discrete-time, finite-state, controlled Markov
chain to approximate the controlled diffusion process with regime-switching in
the absence of jumps with the dynamic system

$\begin{array}[]{ll}dX_{k}(t)&\\!\\!\\!\displaystyle=\sum_{i\in{\cal
M}}I_{\\{\alpha(t)=i\\}}\bigg{\\{}r(i)X_{k}(t)+(b_{k}(t)-\kappa_{k}b_{l}(t))[\mu_{S}(i,Z(t))-r(i)]+c_{k}(i,Z(t))-\kappa_{k}c_{l}(i,Z(t))\\\
&\displaystyle\quad-(g(a_{k})+\kappa_{k}g(a_{l}))dt+(b_{k}(t)-\kappa_{k}b_{l}(t))\sigma_{S}(i,Z(t))dW_{S}(t)\bigg{\\}},\\\
dZ(t)&\\!\\!\\!\displaystyle=\mu_{Z}(t,Z(t))dt+\sigma_{Z}(t,Z(t))dW_{Z}(t)\\\
X_{k}(0)&\\!\\!\\!\displaystyle=\widetilde{x}_{k}-\kappa_{k}\widetilde{x}_{l}\\\
Z(0)&\\!\\!\\!\displaystyle=z_{0}.\end{array}$ (3.1)

Because the value function depends on both the state $x$ and the time variable
$t$, two stepsizes are needed. That is, we need to discretize both the state
and time. We use $h>0$ as the stepsize of the state and $\delta>0$ as the
stepsize for the time. In fact, for any given $T>0$, we use
$N=N(\delta)=\lfloor T/\delta\rfloor$.

Let $e_{i}$ denote the unit vector in the $i$-th coordinate direction and
${\mathbb{R}}^{3}_{h}$ denote the uniform $h$-grid on ${\mathbb{R}}^{3}$; i.e.
${\mathbb{R}}^{3}_{h}=\\{(x_{1},x_{2},z):(x_{1},x_{2},z)=h(k_{1}e_{1}+k_{2}e_{2}+k_{3}e_{3});k_{1},k_{2},k_{3}=0,\pm
1,\pm 2,\dots\\}$. We use $S_{h}={\mathbb{R}}^{3}_{h}$, denote
$x=(x_{1},x_{2},z)$ and $y=(y_{1},y_{2},z^{\ast})$.

We can rewrite the system in the short form as the following:

$\begin{array}[]{ll}dX(t)=\mu(X(t),\alpha(t),u(t))dt+\sigma(X(t),\alpha(t),u(t))dW(t),\end{array}$
(3.2)

where

$\begin{array}[]{rl}&\\!\\!\\!\displaystyle\\!\\!\\!\\!\mu(X(t),\alpha(t),u(t))\\\
&\\!\\!\\!\displaystyle=\begin{pmatrix}r(\alpha(t))X_{1}(t)+(b_{1}(t)-\kappa_{1}b_{2}(t))[\mu_{S}(\alpha(t),Z(t))-r(\alpha(t))]+c_{1}(\alpha(t),Z(t))-\kappa_{1}c_{2}(\alpha(t),Z(t))\\\
-(g(a_{1}(t))+\kappa_{1}g(a_{2}(t)))\\\
r(\alpha(t))X_{2}(t)+(b_{2}(t)-\kappa_{2}b_{1}(t))[\mu_{S}(\alpha(t),Z(t))-r(\alpha(t))]+c_{2}(\alpha(t),Z(t))-\kappa_{2}c_{1}(\alpha(t),Z(t))\\\
-(g(a_{2}(t))+\kappa_{2}g(a_{1}(t)))\\\ \mu_{Z}(t,Z(t))\end{pmatrix}\\\
&\\!\\!\\!\displaystyle\\!\\!\\!\\!\sigma(X(t),\alpha(t),u(t))\\\
&\\!\\!\\!\displaystyle={\rm{diag}}\bigg{(}(b_{1}(t)-\kappa_{1}b_{2}(t))\sigma_{S}(\alpha(t),Z(t)),(b_{2}(t)-\kappa_{2}b_{1}(t))\sigma_{S}(\alpha(t),Z(t)),\sigma_{Z}(t,Z(t))\bigg{)}\\\
&\\!\\!\\!\displaystyle\\!\\!\\!W(t)=\bigg{(}W_{S}(t),W_{S}(t),W_{Z}(t)\bigg{)}^{\prime},\end{array}$

where $A^{\prime}$ is the transpose of $A$. Let
$\\{(\xi^{h,\delta}_{n},\alpha^{h,\delta}_{n},n<\infty\\}$ be a controlled
discrete-time Markov chain on ${\mathbb{R}}^{3}_{h}\times{\cal M}$ and denote
by $p^{h,\delta}_{D}\big{(}(x,i),(y,j),n\delta|\phi\big{)}$ the transition
probability from a state $(x,i)$ to another state $(y,j)$, for
$\phi\in{\mathcal{U}}$. We use $u^{h,\delta}_{n}$ to denote the random
variable that is the control action for the chain at discrete time $n$ and
$p^{h,\delta}_{D}$ is so defined that the constructed Markov chain’s evolution
well approximates the local behavior of the controlled regime-switching
diffusion (2.7).

For each $k=1,2$, we construct the transition probability
$p^{h,\delta}_{k,D}\big{(}(x,i),(y,j),n\delta|\phi\big{)}$ which is associated
with $J^{k}(t,x,i,\phi)={\mathbb{E}}\left[U_{k}\left(X_{k}(T)\right)\right]$
satisfying the followings:

$\begin{cases}&J^{k}_{t}(t,x,i,\phi)+\displaystyle{\cal
L}^{u_{k},u_{l}}J^{k}(t,x,i,\phi)=0,\\\
&J^{k}(T,x,i,\phi)=U_{k}(x_{k}).\end{cases}$ (3.3)

To figure out the form of $p^{h,\delta}_{k,D}\big{(}(x,i),(y,j)|\phi\big{)}$,
we define a finite difference approximation to (3.3) as

$\begin{array}[]{rl}&J^{k}_{t}(t,x,i,\phi)\rightarrow\dfrac{J^{k}(t+\delta,x,i,\phi)-J^{k}(t,x,i,\phi)}{\delta},\\\
&J^{k}_{x_{k}}(t,x,i,\phi)\rightarrow\dfrac{J^{k}(t,x+he_{k},i,\phi)-J^{k}(t,x,i,\phi)}{h}\\\
&\quad\hbox{if}\
r(i)x_{k}+(b_{k}-\kappa_{k}b_{l})[\mu_{S}(i,z)-r(i)]+c_{k}(i,z)-\kappa_{k}c_{l}(i,z)-(g(a_{k})-\kappa_{k}g(a_{l}))>0,\\\
&J^{k}_{x_{k}}(t,x,i,\phi)\rightarrow\dfrac{J^{k}(t,x,i,\phi)-J^{k}(t,x-he_{k},i,\phi)}{h}\\\
&\quad\hbox{if}\
r(i)x_{k}+(b_{k}-\kappa_{k}b_{l})[\mu_{S}(i,z)-r(i)]+c_{k}(i,z)-\kappa_{k}c_{l}(i,z)-(g(a_{k})-\kappa_{k}g(a_{l}))<0,\\\
&J^{k}_{x_{k}x_{k}}(t,x,i,\phi)\rightarrow\dfrac{J^{k}(t,x+he_{k},i,\phi)-2J^{k}(t,x,i,\phi)+J^{k}(t,x-he_{k},i,\phi)}{h^{2}},\\\
&J^{k}_{z}(t,x,i,\phi)\rightarrow\dfrac{J^{k}(t,x+he_{3},i,\phi)-J^{k}(t,x,i,\phi)}{h}\quad\hbox{if}\quad\mu_{Z}(t,z)>0,\\\
&J^{k}_{z}(t,x,i,\phi)\rightarrow\dfrac{J^{k}(t,x,i,\phi)-J^{k}(t,x-he_{3},i,\phi)}{h}\quad\hbox{if}\quad\mu_{Z}(t,z)<0,\\\
&J^{k}_{zz}(t,x,i,\phi)\rightarrow\dfrac{J^{k}(t,x+he_{3},i,\phi)-2J^{k}(t,x,i,\phi)+J^{k}(t,x-he_{3},i,\phi)}{h^{2}}.\end{array}$

To proceed, define

$\begin{array}[]{ll}&p^{h,\delta}_{k,D}\big{(}(x,i),(x\pm
he_{k},i),n\delta|\phi\big{)}=\frac{\delta}{h}\bigg{\\{}r(i)x_{k}+(b_{k}-\kappa_{k}b_{l})[\mu_{S}(i,z)-r(i)]+c_{k}(i,z)-\kappa_{k}c_{l}(i,z)\\\
&\quad-(g(a_{k})-\kappa_{k}g(a_{l}))\bigg{\\}}^{\pm}+\frac{\delta}{2h^{2}}(b_{k}-\kappa_{k}b_{l})^{2}\sigma^{2}_{S}(i,z),\\\
&p^{h,\delta}_{k,D}\big{(}(x,i),(x\pm
he_{3},i),n\delta|\phi\big{)}=\frac{\delta}{h}\mu_{Z}(t,z)^{\pm}+\frac{\delta}{2h^{2}}\sigma_{Z}^{2}(t,z),\\\
&p^{h,\delta}_{k,D}\big{(}(x,i),(x,j),n\delta|\phi\big{)}=q_{ij}\delta,\\\
&p^{h,\delta}_{k,D}\big{(}(x,i),(x,i),n\delta|\phi\big{)}=1+q_{ii}\delta-\frac{\delta}{h}\left|r(i)x_{k}+(b_{k}-\kappa_{k}b_{l})[\mu_{S}(i,z)-r(i)]+c_{k}(i,z)-\kappa_{k}c_{l}(i,z)\right.\\\
&\left.\quad-(g(a_{k})-\kappa_{k}g(a_{l}))\right|-\frac{\delta}{h}\mu_{Z}(t,z)-\frac{\delta}{h^{2}}(b_{k}-\kappa_{k}b_{l})^{2}\sigma^{2}_{S}(i,z)-\frac{\delta}{h^{2}}\sigma_{Z}^{2}(t,z),\\\
&p^{h,\delta}_{k,D}\big{(}\cdot\big{)}=0,\quad\hbox{otherwise},\end{array}$
(3.4)

where $K^{+}=\max\\{K,0\\}$ and $K^{-}=\min\\{-K,0\\}$. By choosing $\delta$
and $h$ appropriately, we can have
$p^{h,\delta}_{k,D}\big{(}(x,i),(x,i),n\delta|\phi\big{)}$ given in (3.4)
nonnegative. Thus, $p^{h,\delta}(\cdot|\phi)$ are well-defined transition
probability.

Next, we need to approximate the Poisson jumps for ensuring the local
properties of claims for (2.7). We can rewrite the system in the matrix form
as follow

$dX(t)=\mu(X(t),\alpha(t),u(t))dt+\sigma(X(t),\alpha(t),u(t))dB(t)+\widetilde{Y}_{1}(t)e_{1}+\widetilde{Y}_{2}(t)e_{2},$
(3.5)

where $\widetilde{Y}_{k}(t)$ is the jump process w.r.t the surplus process
$X_{k}(t)$, for $k=1,2$.

The relative surplus process $X_{k}(t)$ is determined by two jump terms with
the arriving rate $\lambda_{k}$ and $\lambda_{l}$, respectively. Denote by
$R_{k}(t)$ the difference of the two jumps. That is,

$R_{k}(t)=\int_{{\mathbb{R}}_{+}}\widetilde{q}_{k}(q_{k},a_{k})N_{k}(dt,d\rho_{k})-\kappa_{k}\int_{{\mathbb{R}}_{+}}\widetilde{q}_{l}(q_{l},a_{l})N_{l}(dt,d\rho_{l}).$

Since the difference of two Poisson processes is again a Poisson process,
events in the new process $R_{k}(t)$ will occur according to a Poisson process
with the rate $\lambda$. with $\lambda=\lambda_{k}+\lambda_{l}$; and each
event, independently, will be from the first jump process with probability
$\lambda_{k}/(\lambda_{k}+\lambda_{l})$, yielding the generic claim size

$\begin{array}[]{rl}\widetilde{A}^{k}=\begin{cases}A^{k}(a_{k}),&\quad\hbox{with
probability }\frac{\lambda_{k}}{\lambda_{k}+\lambda_{l}},\\\ \\\
-\kappa_{k}A^{l}(a_{l}),&\quad\hbox{with probability
}\frac{\lambda_{l}}{\lambda_{k}+\lambda_{l}}.\end{cases}\end{array}$

Suppose that the current state is $\xi^{h,\delta}_{n}=x$,
$\alpha^{h,\delta}_{n}=i$, and control is $u^{h,\delta}_{n}=\phi$. The next
interpolation interval is determined by (3.4) and
$\widetilde{q}_{k}^{h}(q_{k}(i,z,\rho_{k}),a_{k})$ is the nearest value of
$\widetilde{q}_{k}(q_{k}(i,z,\rho_{k}),a_{k})$ so that
$\xi^{h,\delta}_{n+1}\in S_{h}$. Then
$|\widetilde{q}_{k}^{h}(q_{k}(i,z,\rho_{k}),a_{k})-\widetilde{q}_{k}(q_{k}(i,z,\rho_{k}),a_{k})|\rightarrow
0$ as $h\rightarrow 0$, uniformly in x. To present the claim terms, we
determine the next case $(\xi^{h,\delta}_{n+1},\alpha^{h,\delta}_{n+1})$ by
noting:

  1. 1.

No claims occur in $[n\delta,n\delta+\delta)$ with probability
$1-\lambda\delta+o(\delta)$, we determine
$(\xi^{h,\delta}_{n+1},\alpha^{h,\delta}_{n+1})$ by transition probability
$p^{h,\delta}_{k,D}(\cdot)$ as in (3.4).

  2. 2.

There is a claim of the relative surplus process $X_{k}(t)$ in
$[n\delta,n\delta+\delta)$ with probability $\lambda\delta+o(\delta)$, we
determine $(\xi^{h,\delta}_{n+1},\alpha^{h,\delta}_{n+1})$ by

$\xi^{h,\delta}_{n+1}=\xi^{h,\delta}_{n}-\widetilde{q}_{k}^{h}(q_{k}(i,z,\rho_{k}),a_{k})e_{k},\qquad\alpha^{h,\delta}_{n+1}=\alpha^{h,\delta}_{n}.$

So, we define

$p^{h,\delta}_{k}\big{(}(x,i),(y,j)|\phi\big{)}=(1-\lambda\delta+o(\delta))p^{h,\delta}_{k,D}\big{(}(x,i),(y,j)|\phi\big{)}+(\lambda\delta+o(\delta))\Pi_{k}\\{\rho:\widetilde{q}_{h}(i,z,\rho)e_{k}=x-y\\}.$
(3.6)

######  Definition 3.1.

A controlled Markov chain
$\\{(\xi^{h,\delta}_{n},\alpha^{h,\delta}_{n}),n<\infty\\}$ with the one-step
transition probability $p^{h,\delta}\big{(}(x,i),(y,j)|\phi\big{)}$ is given
by

$p^{h,\delta}\big{(}(x,i),(y,j)|\phi\big{)}=p^{h,\delta}_{1}\big{(}(x,i),(y,j)|\phi\big{)}I_{\\{k=1\\}}+p^{h,\delta}_{2}\big{(}(x,i),(y,j)|\phi\big{)}I_{\\{k=2\\}},$
(3.7)

where $k$ is the index of the cost function.

The piecewise constant interpolations
$\xi^{h,\delta}(\cdot),\alpha^{h,\delta}(\cdot)$, and $u^{h,\delta}(\cdot)$
are defined as

$\xi^{h,\delta}(t)=\xi^{h,\delta}_{n},\quad\alpha^{h,\delta}(t)=\alpha^{h,\delta}_{n},\quad
u^{h,\delta}(t)=u^{h,\delta}_{n}\quad\hbox{for }t\in[n\delta,n\delta+\delta)$
(3.8)

Use $\mathbb{E}^{\phi,h,\delta}_{x,i,n},\text{Var}^{\phi,h,\delta}_{x,i,n}$,
and ${P}^{\phi,h,\delta}_{x,i,n}$ to denote the conditional expectation,
variance, and marginal probability given
$\\{\xi^{h,\delta}_{\iota},\alpha^{\delta}_{\iota},u^{h,\delta}_{\iota},\iota\leq
n,\xi^{h,\delta}_{n}=x,\alpha^{h,\delta}_{n}=i,u^{h,\delta}_{n}=\phi\\}$,
respectively. Define the difference
$\Delta\xi^{h,\delta}_{n}=\xi^{h,\delta}_{n+1}-\xi^{h,\delta}_{n}$.

With the approximation of the Markov chain constructed above, we can obtain an
approximation of the utility function as follows:

$\begin{array}[]{ll}J^{k,\delta}(n\delta,x,i,u^{h,\delta})&=\displaystyle(1-\lambda\delta+o(\delta))\sum_{(y,j)}p^{h,\delta}_{k}\bigg{(}(x,i),(y,j)|u^{h,\delta}\bigg{)}J^{k,\delta}(n\delta+\delta,y,j,u^{h,\delta})\\\
&+\displaystyle(\lambda_{k}\delta+o(\delta))\int_{{\mathbb{R}}_{+}}J^{k,\delta}(n\delta+\delta,x_{k}-\widetilde{q}_{k}^{h}(q_{k}(i,z,\rho_{k}),a^{h,\delta}_{k}),z,i,u^{h,\delta})\Pi_{k}(d\rho_{k})\\\
&+\displaystyle(\lambda_{l}\delta+o(\delta))\int_{{\mathbb{R}}_{+}}J^{k,\delta}(n\delta+\delta,x_{k}+\kappa_{k}\widetilde{q}_{l}^{h}(q_{l}(i,z,\rho_{l}),a^{h,\delta}_{l}),z,i,u^{h,\delta})\Pi_{l}(d\rho_{l}).\end{array}$
(3.9)

Moreover,

$\begin{array}[]{ll}V^{k,h,\delta}(n\delta,x,i)=\displaystyle\sup_{u_{k,n}^{h,\delta}}J^{k,h,\delta}(n\delta,x,i,u^{h,\delta}).\end{array}$
(3.10)

######  Definition 3.2.

The sequence $\\{(\xi^{h,\delta}_{n},\alpha^{h,\delta}_{n})\\}$ is said to be
locally consistent, if it

  1. 1.

There is a transition probability $p^{h,\delta}_{D}$ is locally consistent in
the sense

$\begin{array}[]{ll}&\mathbb{E}^{\phi,h,\delta}_{x,i,n}[\Delta\xi^{h,\delta}_{n}]=\mu^{h,\delta}(x,i,\phi)\delta+o(\delta),\\\
&\text{Var}^{\phi,h,\delta}_{x,i,n}[\Delta\xi^{h,\delta}_{n}]=\sigma^{h,\delta}(x,i,\phi)\delta+o(\delta).\end{array}$
(3.11)

where

$\begin{array}[]{rl}&\mu^{h,\delta}(x,i,\phi)=\begin{pmatrix}r(i)x_{1}+(b_{1}-\kappa_{1}b_{2})[\mu_{S}(i,z)-r(i)]+c_{1}(i,z)-\kappa_{1}c_{2}(i,z)-g(a_{1})+\kappa_{1}g(a_{2})\\\
r(i)x_{2}+(b_{2}-\kappa_{2}b_{1})[\mu_{S}(i,z)-r(i)]+c_{2}(i,z)-\kappa_{2}c_{1}(i,z)-g(a_{2})+\kappa_{2}g(a_{1})\\\
\mu_{Z}(t,z)\end{pmatrix}\\\
&\sigma^{h,\delta}(x,i,\phi)={\rm{diag}}\bigg{(}(b_{1}-\kappa_{1}b_{2})^{2}\sigma_{S}^{2}(i,z),(b_{2}-\kappa_{2}b_{1})^{2}\sigma_{S}^{2}(i,z),\sigma_{Z}^{2}(t,z)\bigg{)}.\end{array}$

  2. 2.

The one-step transition probability $p^{h,\delta}((x,i),(y,j)|\phi)$ for the
chain can be represented in the factored form:

$\begin{array}[]{rl}p^{h,\delta}((x,i),(y,j)|\phi)=(1-\lambda\delta+o(\delta))p^{h,\delta}_{D}((x,i),(y,j)|\phi)+(\lambda\delta+o(\delta))\Pi\\{\rho:q_{h}(i,z,\rho)=x-y\\}\end{array}$

##  4 Convergence of Numerical Approximation

###  4.1 Representations of Approximation Sequences

To proceed, we first show that the constructed Markov chain is locally
consistent. This ensures that our approximation is reasonable in certain
sense.

######  Lemma 4.1.

The Markov chain $\xi^{h,\delta}_{n}$ with transition probabilities
$p^{h,\delta}(\cdot)$ defined in (3.7) is locally consistent with the
stochastic differential equation in (2.7).

Proof. Define
$p^{h,\delta}_{D}(\cdot)=p^{h,\delta}_{1,D}(\cdot)I_{\\{k=1\\}}+p^{h,\delta}_{2,D}(\cdot)I_{\\{k=2\\}}$Using
(3.4) and (3.7). It is easy to see that

$\begin{array}[]{rl}\mathbb{E}^{\phi,h,\delta}_{x,i,n}[\Delta\xi^{h,\delta}_{n}]&\\!\\!\\!\displaystyle=\mathbb{E}^{\phi,h,\delta}_{x,i,n}[\Delta\xi^{h,\delta}_{n}I_{\\{k=1\\}}]+\mathbb{E}^{\phi,h,\delta}_{x,i,n}[\Delta\xi^{h,\delta}I_{\\{k=2\\}}]\\\
\\\
&\\!\\!\\!\displaystyle=\begin{pmatrix}r(i)x_{1}+(b_{1}-\kappa_{1}b_{2})[\mu_{S}(i,z)-r(i)]+c_{1}(i,z)-\kappa_{1}c_{2}(i,z)-g(a_{1})+\kappa_{1}g(a_{2})\\\
r(i)x_{2}+(b_{2}-\kappa_{2}b_{1})[\mu_{S}(i,z)-r(i)]+c_{2}(i,z)-\kappa_{2}c_{1}(i,z)-g(a_{2})+\kappa_{2}g(a_{1})\\\
\mu_{Z}(t,z)\end{pmatrix}\delta.\end{array}$

Likewise, we obtain
$\mathbb{E}^{\phi,h,\delta}_{x,i,n}\big{[}\Delta\xi^{h,\delta}_{n}(\Delta\xi^{h,\delta}_{n})^{\prime}\big{]}$
and $\text{Var}^{\phi,h,\delta}_{x,i,n}[\Delta\xi^{h,\delta}_{n}]$. $\Box$

Let $\xi^{h,\delta}(0)=x,\alpha^{h,\delta}(0)=i$. Define the relaxed control
representation $m^{h,\delta}_{k}(\cdot)$ of $u^{h,\delta}_{k}(\cdot)$ by using
its derivative $m^{h,\delta}_{t,k}(A)=I_{\\{u^{h,\delta}(t)\in A\\}}$. Let
$H^{h,\delta}_{n}$ denote the event that
$\xi^{h,\delta}_{n},\alpha^{h,\delta}_{n}$ is determined by the case of “no
claim occurs” and use $T^{\delta}_{n}$ to denote the event of “one claim
occurs”. Let $I_{H^{h,\delta}_{n}}$ and $I_{T^{h,\delta}_{n}}$ be
corresponding indicator functions, respectively. Then
$I_{H^{h,\delta}_{n}}+I_{T^{h,\delta}_{n}}=1$ and we can write

$\begin{array}[]{ll}\xi^{h,\delta}(t)&\displaystyle=\xi^{h,\delta}(0)+\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}[\Delta\xi^{h,\delta}_{r}I_{H^{h,\delta}_{r}}+\Delta\xi^{h,\delta}_{r}I_{T^{h,\delta}_{r}}]\\\
&\displaystyle=x+\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}\mathbb{E}^{\phi,h,\delta}_{x,i,r}[\Delta\xi^{h,\delta}_{r}I_{H^{h,\delta}_{r}}]+\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}(\Delta\xi^{h,\delta}_{r}-\mathbb{E}^{\phi,h,\delta}_{x,i,r}\Delta\xi^{h,\delta}_{r})I_{H^{h,\delta}_{r}}+\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}\Delta\xi^{h,\delta}_{r}I_{T^{h,\delta}_{r}}.\end{array}$
(4.1)

Define ${\cal F}^{h,\delta}_{n}$ as the smallest $\sigma$-algebra generated by
$\\{\xi^{h,\delta}_{r},\alpha^{h,\delta}_{r},m^{h,\delta}_{r},H^{h,\delta}_{r},r\leq
n\\}$ and ${\cal F}^{h,\delta}_{t}$ as the smallest $\sigma$\- algebra
generated by
$\\{\xi^{h,\delta}(s),\alpha^{h,\delta}(s),m^{h,\delta}(s),H^{h,\delta}(s),s\leq
t\\}$.

For $k,l=1,2$ and $k\neq l$, denote

$\begin{array}[]{ll}M^{h,\delta}(t)&\\!\\!\\!\displaystyle=\displaystyle\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}(\Delta\xi^{h,\delta}_{r}-\mathbb{E}^{\phi,h,\delta}_{x,i,r}\Delta\xi^{h,\delta}_{r})I_{H^{h,\delta}_{r}}\\\
Y^{h,\delta}_{k}(t)&\\!\\!\\!\displaystyle=-\displaystyle\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}[\Delta\xi^{h,\delta}_{r}]^{\prime}e_{k}I_{T^{h,\delta}_{r}}\\\
&\\!\\!\\!\displaystyle=\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\widetilde{q}^{h}_{k}(q_{k}(\alpha^{h,\delta}(s),Z^{h,\delta}(s),\rho_{k}),a^{h,\delta}_{k}(s))N^{h,\delta}_{k}(ds,d\rho_{k})\\\
&\displaystyle\quad-\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\kappa_{k}\widetilde{q}^{h}_{l}(q_{l}(\alpha^{h,\delta}(s),Z^{h,\delta}(s),\rho_{l}),a^{h,\delta}_{l}(s))N^{h,\delta}_{l}(ds,d\rho_{l}).\end{array}$
(4.2)

Then $M^{h,\delta}(t)$ is a martingale with respect to ${\cal
F}^{h,\delta}_{\lfloor t/\delta\rfloor}$. Now, we represent $M^{h,\delta}(t)$
similar to the diffusion term in (2.7). Define $W^{h,\delta}_{k}(\cdot)$ as

$\begin{array}[]{ll}W^{h,\delta}(t)&=\displaystyle\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}[\sigma^{h,\delta}(x,\phi,i)]^{-1}(\Delta\xi^{h,\delta}_{r}-\mathbb{E}^{\phi,h,\delta}_{x,i,r}\Delta\xi^{h,\delta}_{r})I_{H^{h,\delta}_{r}}\\\
&=\displaystyle\int_{0}^{t}[\sigma^{h,\delta}(\xi^{h,\delta}(s),\alpha^{h,\delta}(s),u^{h,\delta}(s))]^{-1}dM^{h,\delta}(s).\end{array}$
(4.3)

The local consistency leads to

$\begin{array}[]{rl}&\displaystyle\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}\mathbb{E}^{\phi,h,\delta}_{x,i,r}[\Delta\xi^{h,\delta}_{r}I_{H^{h,\delta}_{r}}]\\\
&=\displaystyle\sum_{r=0}^{\lfloor
t/\delta\rfloor-1}\begin{pmatrix}r(\alpha^{h,\delta}_{r})\xi^{\delta}_{r}e_{1}+(b^{h,\delta}_{1,r}-\kappa_{1}b^{h,\delta}_{2,r})[\mu(\alpha^{h,\delta}_{r},\xi^{h,\delta}_{r}e_{3})-r(\alpha^{h,\delta}_{r})]\\\
+c_{1}(\alpha^{h,\delta}_{r},\xi^{h,\delta}_{r}e_{3})-\kappa_{1}c_{2}(\alpha^{h,\delta}_{r},\xi^{h,\delta}_{r}e_{3})-g(a^{h,\delta}_{1,r})+\kappa_{1}g(a^{h,\delta}_{2,r})\\\
r(\alpha^{h,\delta}_{r})\xi^{h,\delta}_{r}e_{2}+(b^{h,\delta}_{2,r}-\kappa_{2}b^{h,\delta}_{1,r})[\mu(\alpha^{h,\delta}_{r},\xi^{h,\delta}_{r}e_{3})-r(\alpha^{h,\delta}_{r})]\\\
+c_{2}(\alpha^{h,\delta}_{r},\xi^{h,\delta}_{r}e_{3})-\kappa_{2}c_{1}(\alpha^{h,\delta}_{r},\xi^{h,\delta}_{r}e_{3})-g(a^{h,\delta}_{2,r})+\kappa_{2}g(a^{h,\delta}_{1,r})\\\
\mu_{Z}(r\delta,\xi^{h,\delta}_{r}e_{3})\end{pmatrix}\delta
I_{H^{h,\delta}_{r}}+o(\delta)I_{H^{h,\delta}_{r}}\\\
&=\displaystyle\int_{0}^{t}\mu^{h,\delta}(\xi^{h,\delta}(s),\alpha^{h,\delta}(s),u^{\delta}(s))ds+\varepsilon^{h,\delta}(t).\end{array}$

For each $t$, ${\mathbb{E}}[\hbox{number of }r:\nu^{h,\delta}_{k,r}\leq
t]=\lambda_{k}t$ as $h,\delta\rightarrow 0$. This implies that we can drop
$I_{H^{h,\delta}_{r}}$ with no effect on the above limit.

As a consequence, we can rewrite (4.1) as following:

$\begin{array}[]{ll}\xi^{h,\delta}(t)&=\displaystyle
x+\int_{0}^{t}\int_{\mathcal{U}}\mu^{h,\delta}(\xi^{h,\delta}(s),\alpha^{h,\delta}(s),\phi)m^{h,\delta}_{s}(d\phi)ds\\\
&\quad\displaystyle+\int_{0}^{t}\int_{{\mathcal{U}}}\sigma^{h,\delta}(\xi^{h,\delta}(s),\alpha^{h,\delta}(s),\phi)m^{h,\delta}_{s}(d\phi)dW^{h,\delta}(s)\\\
&\quad\displaystyle+\bigg{(}\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\widetilde{q}^{h}_{1}(q_{1}(\alpha^{h,\delta}(s),Z^{h,\delta}(s),\rho_{1}),a_{1})m^{h,\delta}_{s}(d\phi)N^{h,\delta}_{1}(ds,d\rho_{1})\\\
&\qquad\displaystyle-\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\kappa_{1}\widetilde{q}^{h}_{2}(q_{2}(\alpha^{h,\delta}(s),Z^{h,\delta}(s),\rho_{2}),a_{2})m^{h,\delta}_{s}(d\phi)N^{h,\delta}_{2}(ds,d\rho_{2})\bigg{)}e_{1}\\\
&\quad\displaystyle+\bigg{(}\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\widetilde{q}^{h}_{2}(q_{2}(\alpha^{h,\delta}(s),Z^{h,\delta}(s),\rho_{2}),a_{2})m^{h,\delta}_{s}(d\phi)N^{h,\delta}_{2}(ds,d\rho_{2})\\\
&\qquad\displaystyle-\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\kappa_{2}\widetilde{q}^{h}_{1}(q_{1}(\alpha^{h,\delta}(s),Z^{h,\delta}(s),\rho_{1}),a_{1})m^{h,\delta}_{s}(d\phi)N^{h,\delta}_{1}(ds,d\rho_{1})\bigg{)}e_{2}+\varepsilon^{h,\delta}(t).\end{array}$
(4.4)

We can also rewrite (3.5) as

$\begin{array}[]{ll}X(t)&\displaystyle=x+\int_{0}^{t}\int_{\mathcal{U}}\mu(X(s),\alpha(s),\phi(s))m_{s}(d\phi)ds+\int_{0}^{t}\int_{\mathcal{U}}\sigma(X(s),\alpha(s),\phi)m_{s}(d\phi)dW(s)\\\
&\displaystyle\quad+\bigg{(}\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\widetilde{q}_{1}(q_{1}(\alpha(s),Z(s),\rho_{1}),a_{1})m_{s}(d\phi)N_{1}(ds,d\rho_{1})\\\
&\displaystyle\qquad-\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\kappa_{1}\widetilde{q}_{2}(q_{2}(\alpha(s),Z(s),\rho_{2}),a_{2})m_{s}(d\phi)N_{2}(ds,d\rho_{2})\bigg{)}e_{1}\\\
&\displaystyle\quad+\bigg{(}\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\widetilde{q}_{2}(q_{2}(\alpha(s),Z(s),\rho_{2}),a_{2})m_{s}(d\phi)N_{2}(ds,d\rho_{2})\\\
&\displaystyle\qquad-\int_{0}^{t}\int_{{\mathbb{R}}_{+}}\int_{\mathcal{U}}\kappa_{2}\widetilde{q}_{1}(q_{1}(\alpha(s),Z(s),\rho_{1}),a_{1})m_{s}(d\phi)N_{1}(ds,d\rho_{1})\bigg{)}e_{2}.\end{array}$
(4.5)

###  4.2 Convergence of Approximating Markov Chains

######  Lemma 4.2.

Using the transition probability $\\{p^{h,\delta}(\cdot)\\}$ defined in (3.7),
the interpolated process of the constructed Markov chain
$\\{\alpha^{h,\delta}(\cdot)$ converges weakly to $\alpha(\cdot)$, the Markov
chain with generator $Q$.

Proof. The proof can be obtained similar to (Yin et al., , 2003, Theorem 3.1).
The details are thus omitted. $\Box$

######  Theorem 4.3.

Let the approximating chain
$\\{\xi^{h,\delta}_{n},\alpha^{h,\delta}_{n},n<\infty\\}$ constructed with
transition probabilities defined in (3.7) be locally consistent with (3.11),
$m^{h,\delta}(\cdot)$ be the relaxed control representation of
$\\{u^{h,\delta},n<\infty\\}$,
$(\xi^{h,\delta}(\cdot),\alpha^{h,\delta}(\cdot))$ be the continuous-time
interpolation defined in (3.8). Then
$\\{\xi^{h,\delta}(\cdot),\alpha^{h,\delta}(\cdot),m^{h,\delta}(\cdot),W^{h,\delta}(\cdot),N^{h,\delta}_{1}(\cdot,\cdot),N^{h,\delta}_{2}(\cdot,\cdot)\\}$
is tight.

Proof. Note that $\alpha^{h,\delta}(\cdot)$ is tight. It follows that for each
$\Delta>0$, each $t>0$, and $0<\widetilde{t}\leq\Delta$, there is a random
variable $\gamma^{h,\delta}(\Delta)>0$ such that

$\begin{array}[]{ll}{\mathbb{E}}_{t}|W^{h,\delta}(t+\widetilde{t})-W^{h,\delta}(t)|^{2}&=\displaystyle\sum_{\lfloor
t/\delta\rfloor}^{\lfloor
t+\widetilde{t}/\delta\rfloor-1}{\mathbb{E}}_{t}\\{[\sigma^{h,\delta}(x,\phi,i)]^{-1}(\Delta\xi^{h,\delta}_{r}-\mathbb{E}^{\phi,h,\delta}_{x,i,r}\Delta\xi^{h,\delta}_{r})I_{H^{h,\delta}_{r}}\\}^{2}\\\
\leq{\mathbb{E}}_{t}\gamma^{h,\delta}(\Delta)\end{array}$ (4.6)

satisfying $\displaystyle\lim_{\Delta\rightarrow
0}\limsup_{h,\delta\rightarrow 0}{\mathbb{E}}\gamma^{h,\delta}(\Delta)=0,$
which yields the tightness of $W^{h,\delta}(\cdot)$. A similar argument leads
to the tightness of $M^{h,\delta}(\cdot)$. The sequence $m^{h,\delta}(\cdot)$
is tight because of its compact range space. By virtue of Kushner and Dupuis,
(2001)[Theorem 9.2.1], we obtain the tightness of
$\\{N_{k}^{h,\delta}(\cdot),\ k=1,2\\}$ since the mean number of claims on any
bounded interval $[t,t+t_{1}]$ is bounded and

$\displaystyle\lim_{\Delta\rightarrow
0}\inf_{r}{P}\\{\nu^{h,\delta}_{k,r+1}-\nu^{h,\delta}_{k,r}>\Delta|\nu^{h,\delta}_{k,r}\\}=1.$

This implies the tightness of $\\{R_{k}^{h,\delta}(\cdot),\ k=1,2\\}$. As a
consequence, $\xi^{h,\delta}(\cdot)$ is tight and

$\\{\xi^{h,\delta}(\cdot),\alpha^{h,\delta}(\cdot),m^{h,\delta}(\cdot),W^{h,\delta}(\cdot),N^{h,\delta}_{1}(\cdot,\cdot),N^{h,\delta}_{2}(\cdot,\cdot)\\}\
\hbox{ is tight.}$

$\Box$

Because
$\\{\xi^{h,\delta}(\cdot),\alpha^{h,\delta}(\cdot),m^{h,\delta}(\cdot),W^{h,\delta}(\cdot),N^{h,\delta}_{1}(\cdot,\cdot),N^{h,\delta}_{2}(\cdot,\cdot)\\}$
is tight, the Prohorov’s theorem implies that it is sequentially compact. Thus
we can extract a weakly convergent subsequence. Select such a convergent
subsequence and still index the sequence by $h,\delta$ for notational
simplicity. We proceed to characterize the limit process.

######  Theorem 4.4.

Let
$\\{\xi(\cdot),\alpha(\cdot),m(\cdot),W(\cdot),N_{1}(\cdot,\cdot),N_{2}(\cdot,\cdot)\\}$
be the limit of weakly convergent subsequence and ${\cal F}_{t}$ be the
$\sigma$-algebra generated by
$\\{X(s),\alpha(s),m(s),W(s),N_{1}(s,\cdot),N_{2}(s,\cdot),s\leq t\\}$. Then
$W(\cdot)$ is a standard ${\cal F}_{t}$-Brownian motion and
$N_{1}(\cdot,\cdot)$, $N_{2}(\cdot,\cdot)$ are ${\cal F}_{t}$-Poisson
measures, and $m(\cdot)$ is an admissible relaxed control.

Proof. The proof is divided into several steps.

Step 1: By the Skorohod representation, with a slight abuse of notation,
$\\{\xi^{h,\delta}(\cdot),\alpha^{h,\delta}(\cdot),m^{h,\delta}(\cdot)$,
$W^{h,\delta}(\cdot),N^{h,\delta}_{1}(\cdot,\cdot),N^{h,\delta}_{2}(\cdot,\cdot)\\}$
converges to
$\\{\xi(\cdot),\alpha(\cdot),m(\cdot),W(\cdot),N_{1}(\cdot,\cdot),N_{2}(\cdot,\cdot)\\}$
w.p.1, and the convergence is uniform on any compact set.

To proceed, we first verify that $W(\cdot)$ is an ${\cal F}_{t}$-Brownian
motion. For any real-valued and continuous function $\psi$, define

$(\psi,m)_{t}=\int_{0}^{t}\int_{\mathcal{U}}\psi(\phi,s)m_{s}(d\phi).$ (4.7)

For any given $f(\cdot)\in{\cal C}^{2}_{0}({\mathbb{R}}^{3})$ ($C^{2}$
function with compact support), consider an associate operator
$\displaystyle{\cal
L}_{w}f(w)=\frac{1}{2}\sum_{i=1}^{3}\dfrac{\partial^{2}}{\partial
w_{i}\partial w_{i}}f(w)$. Let $t,\widetilde{t}>0$ be given with
$t+\widetilde{t}\leq T$, along with arbitrary positive integers $\kappa$ and
$\widetilde{\kappa}$, arbitrary $t_{i}\leq t$ and continuous functions
$\psi_{j}$ with $i\leq\kappa$ and $j\leq\widetilde{\kappa}$, any bounded and
continuous function $h(\cdot)$, and arbitrary $f\in{\cal
C}^{2}_{0}({\mathbb{R}}^{3})$. Denote $\\{\Gamma^{\kappa}_{i},i\leq\kappa\\}$
as a sequence of nondecreasing partition of ${\mathbb{R}}_{+}$ such that
$\Pi(\partial\Gamma^{\kappa}_{i})=0$ for all $i,\kappa$, where
$\partial\Gamma^{\kappa}_{i}$ is the boundary of the set
$\Gamma^{\kappa}_{i}$. As $\kappa\rightarrow\infty$, let the diameter of the
sets $\Gamma^{\kappa}_{i}$ go to zero.

By (4.3), $W^{h,\delta}(\cdot)$ is an ${\cal F}^{h,\delta}_{t}$-Brownian
motion,

$\begin{array}[]{rl}&{\mathbb{E}}h(\xi^{h,\delta}(t_{i}),\alpha^{\delta}(t_{i}),W^{h,\delta}(t_{i}),(\psi_{j},m^{h,\delta})_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\\\
&\times\displaystyle{\bigg{(}f(W^{h,\delta}(t+\widetilde{t})-f(W^{h,\delta}(t))-\int_{t}^{t+\widetilde{t}}{\cal
L}_{w}f(W^{h,\delta}(s))ds\bigg{)}=0.}\end{array}$

By the weak convergence and the Skorohod representation, we may assume that
$W^{h,\delta}(\cdot)$ converges to $W(\cdot)$ w.p.1, and hence as
$\delta\rightarrow 0$,

$\begin{array}[]{rl}&\displaystyle{\mathbb{E}}h(\xi^{h,\delta}(t_{i}),\alpha^{\delta}(t_{i}),W^{h,\delta}(t_{i}),(\psi_{j},m^{h,\delta})_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\bigg{(}f(W^{h,\delta}(t+\widetilde{t})-f(W^{h,\delta}(t))\bigg{)}\\\
&\displaystyle\qquad\rightarrow{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),W(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\bigg{(}f(W(t+\widetilde{t})-f(W(t))\bigg{)},\\\
&\displaystyle{\mathbb{E}}h(\xi^{h,\delta}(t_{i}),\alpha^{h,\delta}(t_{i}),W^{h,\delta}(t_{i}),(\psi_{j},m^{h,\delta})_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\bigg{(}\int_{t}^{t+\widetilde{t}}{\cal
L}_{w}f(W^{h,\delta}(s))ds\bigg{)}\\\
&\displaystyle\qquad\rightarrow{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),W(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\bigg{(}\int_{t}^{t+\widetilde{t}}{\cal
L}_{w}f(W(s))ds\bigg{)}.\end{array}$

Thus,

$\begin{array}[]{rl}\displaystyle{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),W(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\bigg{(}f(W(t+\widetilde{t}))-f(W(t))-\int_{t}^{t+\widetilde{t}}{\cal
L}_{w}f(W(s))ds\bigg{)}=0.\end{array}$

Moreover, consider the collection of random variables generated by
$\\{h(\xi(t_{i}),\alpha(t_{i}),W(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\\}$.
It follows that $f(W(t))-f(W(0)-\int^{t}_{0}{\cal L}_{w}f(W(s))ds$ is a
martingale. By virtue of the Skorohod representation and the dominated
convergence theorem together with (4.6), we have

$\begin{array}[]{rl}{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),W(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\bigg{[}(W(t+\delta)-W(t))^{2}-\delta\bigg{]}=0.\end{array}$

Moreover, the quadratic variation of the martingale $W(t)$ is $tI_{3}$ and
$W(\cdot)$ is an ${\cal F}_{t}$-Brownian motion, where $I_{3}$ is an $3\times
3$ identity matrix.

Step 2: We proceed to show that $N_{k}(\cdot,\cdot)$ is an ${\cal
F}_{t}$-Poisson measure for each $k=1,2$. Let $\theta(\cdot)$ be a continuous
function on ${\mathbb{R}}_{+}$ and define the process

$\begin{array}[]{rl}\displaystyle\Theta_{k}=\int_{0}^{t}\int_{{\mathbb{R}}^{+}}\theta(\rho)N_{k}(ds,d\rho).\end{array}$

Using similar argument as in the proof of the Brownian motion above, if
$f(\cdot)\in{\cal C}_{0}^{2}({\mathbb{R}}^{3})$ then

$\begin{array}[]{rl}&{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),W(t_{i}),(\psi_{j},m)_{t_{i}},N(t_{i},\Gamma^{\kappa}_{i}),i\leq\kappa,j\leq\widetilde{\kappa})\\\
&\quad\times\bigg{[}f(\Theta_{k}(t+\widetilde{t}))-f(\Theta_{k}(t))-\lambda_{k}\int_{t}^{t+\widetilde{t}}\displaystyle{\int_{{\mathbb{R}}_{+}}\big{[}f(\Theta_{k}(s)+\theta(\rho))-f(\Theta_{k}(s))\big{]}\Pi(d\rho)ds}\bigg{]}=0.\end{array}$

This implies that $N_{k}(\cdot,\cdot)$ is an ${\cal F}_{t}$-Possion measure
for each $k=1,2$.

Step 3.1: We will use (4.1) for the rest of the proof. Note that
$\displaystyle{\mathbb{E}}|\varepsilon^{h,\delta}(t)|\rightarrow 0$ as
$h,\delta\rightarrow 0$. Letting $h,\delta\rightarrow 0$ and using the
Skorohod representation for (4.1), we have

$\begin{array}[]{rl}\displaystyle\int_{0}^{t}\int_{\mathcal{U}}\mu^{h,\delta}(\xi^{h,\delta}(s),\alpha^{h,\delta}(s),\phi)m^{h,\delta}_{s}(d\phi)ds-\int_{0}^{t}\int_{\mathcal{U}}\mu(\xi(s),\alpha(s),\phi)m^{h,\delta}_{s}(d\phi)ds\rightarrow
0,\end{array}$

uniformly on any bounded time interval with probability one. On the other
hand, the sequence $m^{h,\delta}(\cdot)$ converges in the compact-weak
topology, thus, for any continuous and bounded function $\psi(\cdot)$ with
compact support,

$\begin{array}[]{rl}\displaystyle\int_{0}^{t}\int_{\mathcal{U}}\psi(\phi,s)m^{h,\delta}(d\phi
ds)\rightarrow\int_{0}^{t}\int_{\mathcal{U}}\psi(\phi,s)m(d\phi ds)\hbox{ as
}h,\delta\rightarrow 0.\end{array}$

By virtue of the Skorohod representation and the weak convergence, as
$h,\delta\rightarrow 0$,

$\int_{0}^{t}\int_{\mathcal{U}}\mu^{h,\delta}(\xi^{h,\delta}(s),\alpha^{h,\delta}(s),\phi)m^{h,\delta}_{s}(d\phi)ds-\int_{0}^{t}\int_{\mathcal{U}}\mu(\xi(s),\alpha(s),\phi)m_{s}(d\phi)ds\rightarrow
0,$ (4.8)

uniformly in $t$ with probability one on any bounded interval.

Step 3.2: For any $t^{1}\geq t$, $t^{2}\geq 0$ with $t^{1}+t^{2}\leq T$, any
${\cal C}_{0}^{1,2}$ function $f(\cdot)$ (functions that have compact support
whose first partial derivative w.r.t. the time variable and the second partial
derivatives w.r.t. the state variable $x$ are continuous), bounded and
continuous function $h(\cdot)$, any positive integers
$\kappa,\widetilde{\kappa},$ $t_{i}$, and any continuous function $\psi_{j}$
satisfying $t\leq t_{i}\leq t_{1}$ and $i\leq\kappa$, and
$j\leq\widetilde{\kappa}$, the weak convergence and the Skorohod
representation imply that

$\begin{array}[]{ll}&{\mathbb{E}}h(\xi^{h,\delta}(t_{i}),\alpha^{h,\delta}(t_{i}),M^{h,\delta}(t_{i}),(\psi_{j},m^{h,\delta})_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\\\
&\quad\times[f(t^{1}+t^{2},M^{h,\delta}(t^{1}+t^{2}))-f(t^{1},M^{h,\delta}(t^{1}))]\\\
&\rightarrow{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),M(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})[f(t^{1}+t^{2},M(t^{1}+t^{2})-f(t^{1},M(t^{1}))]\end{array}$
(4.9)

with $h,\delta\rightarrow 0.$ Choose a sequence $\\{n^{\delta}\\}$ such that
$n^{\delta}\rightarrow\infty$ but $\Delta^{\delta}=\delta
n^{\delta}\rightarrow 0$, then

$\begin{array}[]{ll}&{\mathbb{E}}h(\xi^{h,\delta}(t_{i}),\alpha^{h,\delta}(t_{i}),M^{h,\delta}(t_{i}),(\psi_{j},m^{h,\delta})_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\\\
&\quad\times[f(t^{1}+t^{2},M^{h,\delta}(t^{1}+t^{2}))-f(t^{1},M^{h,\delta}(t^{1}))]\\\
&=\displaystyle{\mathbb{E}}h(\xi^{h,\delta}(t_{i}),\alpha^{\delta}(t_{i}),M^{h,\delta}(t_{i}),(\psi_{j},m^{h,\delta})_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\\\
&\quad\times\bigg{[}\sum_{ln^{\delta}=t^{1}/\delta}^{(t^{1}+t^{2})/\delta-1}f(\delta(ln^{\delta}+n^{\delta}),M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))\\\
&\displaystyle\quad-f(\delta
ln^{\delta},M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))+f(\delta
ln^{\delta},M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))-f(\delta
ln^{\delta},M^{\delta}(\delta ln^{\delta}))\bigg{]}.\end{array}$ (4.10)

Note that

$\begin{array}[]{rl}&\displaystyle\sum_{ln^{\delta}=t^{1}/\delta}^{(t^{1}+t^{2})/\delta}[f(\delta(ln^{\delta}+n^{\delta}),M^{\delta}(\delta(ln^{\delta}+n^{\delta})))-f(\delta
ln^{\delta},M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))]\\\
&\displaystyle\quad=\sum_{ln^{\delta}=t^{1}/\delta}^{(t^{1}+t^{2})/\delta}\sum_{k=ln^{\delta}}^{ln^{\delta}+n^{\delta}-1}[f(\delta(k+1),M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))-f(\delta
k,M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))]\\\
&\displaystyle\quad=\sum_{ln^{\delta}=t^{1}/\delta}^{(t^{1}+t^{2})/\delta}\dfrac{\partial
f(\delta ln^{\delta},M^{\delta}(\delta(ln^{\delta}+n^{\delta})))}{\partial
s}\Delta^{\delta}+o(1),\end{array}$

where $o(1)\rightarrow 0$ in mean uniformly in $t$ as $\delta\rightarrow 0$.
Letting $\delta ln^{\delta}\rightarrow s$ as $\delta\rightarrow 0$, then
$\delta(ln^{\delta}+n^{\delta})\rightarrow s$ since $\Delta^{\delta}=\delta
n^{\delta}\rightarrow 0$ as $\delta\rightarrow 0$. Then, by the weak
convergence and the Skorohod representation, the continuity of $h(\cdot)$, and
the smoothness of $f(\cdot)$ imply that

$\begin{array}[]{ll}&\displaystyle{\mathbb{E}}h(\xi^{h,\delta}(t_{i}),\alpha^{h,\delta}(t_{i}),M^{h,\delta}(t_{i}),(\psi_{j},m^{h,\delta})_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\\\
&\times\sum_{ln^{\delta}}^{(t^{1}+t^{2})/\delta}\bigg{[}f(\delta(ln^{\delta}+n^{\delta}),M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))-f(\delta
ln^{\delta},M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))\bigg{]}\\\
&\rightarrow{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),M(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\bigg{[}\displaystyle{\int_{t^{1}}^{t^{1}+t^{2}}\dfrac{\partial
f(s,M(s))}{\partial s}ds}\bigg{]}\displaystyle\qquad\hbox{ as
}h,\delta\rightarrow 0.\end{array}$ (4.11)

The last part of (4.10) can be seen as

$\begin{array}[]{rl}&\displaystyle\sum_{ln^{\delta}=t^{1}/\delta}^{(t^{1}+t^{2})/\delta}[f(\delta
ln^{\delta},M^{h,\delta}(\delta(ln^{\delta}+n^{\delta})))-f(\delta
ln^{\delta},M^{h,\delta}(\delta ln^{\delta}))]\\\
&\displaystyle=\sum_{ln^{\delta}=t^{1}/\delta}^{(t^{1}+t^{2})/\delta}\bigg{\\{}\sum_{i=1}^{3}\frac{1}{2}f_{M_{i}M_{i}}(\delta
ln^{\delta},M^{h,\delta}(\delta
ln^{\delta}))\sum_{k=ln^{\delta}}^{ln^{\delta}+n^{\delta}-1}[M_{i}^{h,\delta}(\delta(ln^{\delta}+n^{\delta}))-M^{h,\delta}_{i}(\delta
ln^{\delta})]^{2}\bigg{\\}}\\\
&\displaystyle\qquad+\widetilde{\varepsilon}^{h,\delta}(t^{1}+t^{2})-\widetilde{\varepsilon}^{h,\delta}(t^{1}),\end{array}$

where $f_{M_{i}M_{i}}$ denotes the second partial derivatives,
$M^{h,\delta}_{i}(\cdot)$ is the $i$-th component of $M^{h,\delta}(\cdot)$ and
$\displaystyle\sup_{t\leq t^{1}\leq
T}{\mathbb{E}}|\widetilde{\varepsilon}^{h,\delta}(t^{1})|\rightarrow 0$ as
$h,\delta\rightarrow 0$.

By (4.2) and the definition of $\sigma^{h,\delta}(\cdot)$, we have

$\begin{array}[]{ll}&\displaystyle\sum_{ln^{\delta}=t^{1}/\delta}^{(t^{1}+t^{2})/\delta}\bigg{\\{}\sum_{i=1}^{2}f_{M_{i}M_{i}}(\delta
ln^{\delta},M^{h,\delta}(\delta
ln^{\delta}))\sum_{k=ln^{\delta}}^{ln^{\delta}+n^{\delta}-1}[M^{h,\delta}_{i}(\delta(ln^{\delta}+n^{\delta}))-M^{h,\delta}_{i}(\delta
ln^{\delta})]^{2}\bigg{\\}}\\\
&\displaystyle\rightarrow\int_{t^{1}}^{t^{1}+t^{2}}\int_{\mathcal{U}}Tr\bigg{[}H_{M}f(s,M(s))\sigma(\xi(s),\alpha(s),\phi)[\sigma(\xi(s),\alpha(s),\phi)]^{\prime}\bigg{]}m_{s}(d\phi)ds,\end{array}$
(4.12)

where $H_{M}(f(s,M(s)))$ is the Hessian matrix of $f(\cdot)$ at time $s$,
$Tr(\cdot)$ represents for the trace of a matrix.

Using (4.9)-(4.12), we have

$\begin{array}[]{rl}&\displaystyle{\mathbb{E}}h(\xi(t_{i}),\alpha(t_{i}),M(t_{i}),(\psi_{j},m)_{t_{i}}:i\leq\kappa,j\leq\widetilde{\kappa})\\\
&\displaystyle\quad\times\bigg{[}f(t^{1}+t^{2},M(t^{1}+t^{2}))-f(t^{1},M(t^{1}))-\int_{t^{1}}^{t^{1}+t^{2}}\dfrac{\partial
f(s,M(s))}{\partial s}ds\\\
&\displaystyle\qquad-\int_{t^{1}}^{t^{1}+t^{2}}\int_{\mathcal{U}}\frac{1}{2}Tr\bigg{(}H_{M}f(s,M(s))\sigma(\xi(s),\alpha(s),\phi)[\sigma(\xi(s),\alpha(s),\phi)]^{\prime}\bigg{)}m_{s}(d\phi)ds\bigg{]}=0.\end{array}$

Therefore,

$\int_{0}^{t}\int_{\mathcal{U}}\sigma^{h,\delta}(\xi^{h,\delta}(s),\alpha^{h,\delta}(s),\phi)m^{h,\delta}_{s}(d\phi)ds\rightarrow\int_{0}^{t}\int_{\mathcal{U}}\sigma(\xi(s),\alpha(s),\phi)m_{s}(d\phi)ds,$
(4.13)

uniformly in $t$ with probability one on any bounded interval.

Using the same arguments as in Step 2 and Step 3.2, we obtain the limits for
the latter parts of (4.4). As a result, $\xi(\cdot)$ is the solution of (4.5),
which means $\xi(s)=X(s)$ w.p.1 and $m(\cdot)$ is an admissible relaxed
control. $\Box$

###  4.3 Convergence of the Cost and the Value Functions

Note that since $U_{k}(\cdot)$ satisfies the Inada’s conditions. There exist
positive real numbers $K$ and $k_{0}$ such that $|U_{k}(X_{k})|\leq
K(1+|X_{k}|^{k_{0}})$. We proceed to prove the following result.

######  Theorem 4.5.

Suppose that the utility functions $U_{k}(\cdot)$ has at most polynomial
growth. Then the value functions $V^{k,h,\delta}(t,x,i)$ converges to
$V^{k}(t,x,i)$ for $k=1,2$, respectively, as $h,\delta\rightarrow 0$.

Proof. By Theorem 4.3, each sequence
$\\{\xi^{h,\delta}(\cdot),\alpha^{h,\delta}(\cdot),m^{h,\delta}(\cdot),W^{h,\delta}(\cdot),N^{h,\delta}_{1}(\cdot,\cdot),N^{h,\delta}_{2}(\cdot,\cdot)\\}$
has a weakly convergent subsequence with the limit $\xi^{h,\delta}(\cdot)$
satisfying (4.5). Using the same notation as above and applying the Skorohod
representation, the weak convergence, as $h,\delta\rightarrow 0$,
$J^{k,h,\delta}(t,x,i,m^{h,\delta})\rightarrow J^{k}(t,x,i,m)$, for $k=1,2$.
The cost function is given by (3.9). Since $V^{k}(t,x,i)$ is the maximizing
expected utility, for any admissible control $m(\cdot)$, $J^{k}(t,x,i,m)\leq
V^{k}(t,x,i)$, for $k=1,2$. Let $\widetilde{m}^{h,\delta}(\cdot)$ be an
optimal relaxed control for $\\{\xi^{h,\delta}(\cdot)\\}$, which implies

$\begin{array}[]{rl}\displaystyle
V^{k,h,\delta}(t,x,i)=J^{k,h,\delta}(t,x,i,\widetilde{m}^{h,\delta})=\sup_{m^{h,\delta}}J^{k,h,\delta}(t,x,i,m^{h,\delta}).\end{array}$

Choose a subsequence $\\{\widetilde{h},\widetilde{\delta}\\}$ of
$\\{h,\delta\\}$ such that

$\begin{array}[]{rl}\displaystyle\limsup_{h,\delta\rightarrow
0}V^{k,h,\delta}(t,x,i)=\lim_{\widetilde{h},\widetilde{\delta}\rightarrow
0}V^{k,\widetilde{h},\widetilde{\delta}}(t,x,i)=\lim_{\widetilde{h},\widetilde{\delta}\rightarrow
0}J^{k,\widetilde{h},\widetilde{\delta}}(t,x,i,\widetilde{m}^{\widetilde{h},\widetilde{\delta}}).\end{array}$

Without loss of generality, we may assume that
$\\{\xi^{\widetilde{h},\widetilde{\delta}}(\cdot),\alpha^{\widetilde{h},\widetilde{\delta}}(\cdot),W^{\widetilde{h},\widetilde{\delta}}(\cdot),\widetilde{m}^{\widetilde{h},\widetilde{\delta}}(\cdot),N_{1}^{\widetilde{h},\widetilde{\delta}}(\cdot,\cdot),N_{2}^{\widetilde{h},\widetilde{\delta}}(\cdot,\cdot)\\}$
converges weakly to
$\\{X(\cdot),\alpha(\cdot),W(\cdot),m(\cdot),N_{1}(\cdot,\cdot),N_{2}(\cdot,\cdot)\\}$,
where $m(\cdot)$ is an admissible relaxed control. Then the weak convergence
and the Skorohod representation leads to

$\begin{array}[]{rl}\displaystyle\limsup_{h,\delta\rightarrow
0}V^{k,\delta}(t,x,i)=\lim_{\widetilde{h},\widetilde{\delta}\rightarrow
0}J^{k,\widetilde{h},\widetilde{\delta}}(t,x,i,\widetilde{m}^{\widetilde{h},\widetilde{\delta}})=J^{k}(t,x,i,m)\leq
V^{k}(t,x,i).\end{array}$

We claim that $\displaystyle\liminf_{\delta}V^{k,\delta}(t,x,i)\geq
V^{k}(t,x,i)$.

Suppose that $\overline{m}(\cdot)$ is an optimal control with Brownian motion
$W(\cdot)$ such that $\overline{X}(\cdot)$ is the associated trajectory. By
the chattering lemma (see Yin et al,  (2009) and page 59-60 of Kushner and
Dupuis,  (2001)), for any given $\eta,\delta_{\eta}>0$, there is an
$\varepsilon>0$ and an ordinary control $u^{\eta,\delta_{\eta}}(\cdot)$ that
takes only finite many values, $u^{\eta,\delta_{\eta}}(\cdot)$ is a constant
in $[\iota\varepsilon,\iota\varepsilon+\varepsilon)$,
$\overline{m}^{\eta,\delta_{\eta}}(\cdot)$ is its relaxed control
representation, and $J^{k}(t,x,i,\overline{m}^{\eta,\delta_{\eta}})\geq
V^{k}(t,x,i)-\eta$. For each $\eta,\delta_{\eta}>0$, and the corresponding
$\varepsilon>0$, consider an optimal control problem with piecewise constant
on $[\iota\varepsilon,\iota\varepsilon+\varepsilon)$. We consider the process
$\\{X^{\eta,\delta_{\eta}}(\iota\varepsilon),\alpha^{\eta,\delta_{\eta}},m^{\eta,\delta_{\eta}}(\iota\varepsilon),W^{\eta,\delta_{\eta}}(\iota\varepsilon)\\}$.
Let $\hat{u}^{\eta,\delta_{\eta}}(\cdot)$ be the optimal control,
$\hat{m}^{\eta,\delta_{\eta}}(\cdot)$ the relaxed control representation, and
$\hat{X}^{\eta,\delta_{\eta}}(\cdot)$ the associated trajectory. Since
$\hat{m}^{\eta,\delta_{\eta}}(\cdot)$ is the optimal control,
$J^{k}(t,x,i,\hat{m}^{\eta,\delta_{\eta}})\geq
J^{k}(t,x,i,\overline{m}^{\eta,\delta_{\eta}})\geq V^{k}(t,x,i)-\eta$. Using
the chattering lemma, we can approximate $\hat{m}^{\eta,\delta_{\eta}}(\cdot)$
by a sequence of $m^{h,\delta}(\cdot)$. Then

$\begin{array}[]{rl}V^{k,h,\delta}(t,x,i)\geq
J^{k,h,\delta}(t,x,i,m^{h,\delta})\rightarrow
J^{k}(t,x,i,\hat{m}^{\eta,\delta_{\eta}}).\end{array}$

Moreover,

$\begin{array}[]{rl}\displaystyle\liminf_{h,\delta\rightarrow
0}V^{k,h,\delta}(t,x,i)\geq\lim_{h,\delta\rightarrow
0}J^{k,h,\delta}(t,x,i,m^{h,\delta})=J^{k}(t,x,i,\hat{m}^{\eta,\delta_{\eta}}).\end{array}$

Thus, $\displaystyle\liminf_{h,\delta\rightarrow 0}V^{k,h,\delta}(t,x,i)\geq
V^{k}(t,x,i)-\eta$. The arbitrariness of $\eta$ implies that
$\displaystyle\liminf_{\delta\rightarrow 0}V^{k,\delta}(t,x,i)\geq
V^{k}(t,x,i)$, which completes the proof. $\Box$

##  5 Numerical Examples

In this section, we present some numerical results for the case in which both
insurance companies are constant absolute risk aversion (CARA) agents, i.e.,
each agent has an exponential utility function. More precisely, the utility
function of each insurer has the form

$U_{k}(X_{k})=-\frac{1}{\eta_{k}}\exp{(-\eta_{k}X_{k})},\quad\hbox{for
}\eta_{k}>0,\ k=1,2.$ (5.1)

Based on the algorithm constructed above, we carry out the computation by
valuing iterations in a backward manner by time.

  * 1.

Set $t=T-\delta$ and $J^{k,h,\delta}(T,x,i,u^{h,\delta})=U_{k}(x_{k})$, for
each $k=1,2$.

  * 2.

By (3.9), we obtain

$\begin{array}[]{rl}J^{k,\delta}(t,x,i,u^{h,\delta})&=\displaystyle(1-\lambda\delta)\sum_{(y,j)}p^{h,\delta}_{k}\bigg{(}(x,i),(y,j)|u^{h,\delta}\bigg{)}J^{k,\delta}(t+\delta,y,j,u^{h,\delta})\\\
&+\displaystyle\lambda_{k}\delta\int_{{\mathbb{R}}_{+}}J^{k,\delta}(t+\delta,x_{k}-\widetilde{q}_{k}^{h}(q_{k}(i,z,\rho_{k}),a^{h,\delta}_{k}),z,i,u^{h,\delta})\Pi_{k}(d\rho_{k})\\\
&+\displaystyle\lambda_{l}\delta\int_{{\mathbb{R}}_{+}}J^{k,\delta}(t+\delta,x_{k}+\kappa_{k}\widetilde{q}_{l}^{h}(q_{l}(i,z,\rho_{l}),a^{h,\delta}_{l}),z,i,u^{h,\delta})\Pi_{l}(d\rho_{l}).\end{array}$

Find the pair $\\{\hat{u}^{h,\delta}_{k},k=1,2\\}$ and record
$u^{h,\delta}_{k}(t)=\hat{u}^{h,\delta}_{k}$ satisfying that for any
$u^{h,\delta}_{k}\in{\mathcal{U}}_{k}$,:

$\begin{array}[]{rl}&J^{1,h,\delta}(t,x,i,u^{h,\delta}_{1},\hat{u}^{h,\delta}_{2})\leq
J^{1,h,\delta}(t,x,i,\hat{u}^{h,\delta}_{1},\hat{u}^{h,\delta}_{2})\\\
&J^{2,h,\delta}(t,x,i,\hat{u}^{h,\delta}_{1},u^{h,\delta}_{2})\leq
J^{2,h,\delta}(t,x,i,\hat{u}^{h,\delta}_{1},\hat{u}^{h,\delta}_{2}).\end{array}$

  * 3.

Let $t=t-\delta$ and continue the procedure until $t=0$. We consider the case
in which the discrete event consists of two states, or equivalently, the
Markov chain has two states with given claim size distributions. In addition,
we assume that the claim size distributions are identical in each regime. By
using the value iteration methods, we numerically solve the optimal control
problems. The continuous-time Markov chain $\alpha(t)$ representing the
discrete event state has the generator $Q=\begin{pmatrix}&-0.5&0.5\\\
&0.5&-0.5\end{pmatrix}$ and takes values in ${\cal M}=\\{1,2\\}$.

The parameters of the utility function are $\eta_{1}=17.0$ and $\eta_{1}=21.0$
respectively. The sensitivities are $\kappa_{1}=0.8$ and $\kappa_{2}=0.7$. The
claim severity of both players follows exponential distribution
$f\left(y\right)=\theta_{k}e^{-\theta_{k}y}$ with $\theta_{1}=0.3$ and
$\theta_{2}=0.2$. To incorporate the difference between claim densities in
different regimes, we assume arriving rates of Poisson jump are different. So
is the setup for risk-free return, and premium income rate. The detail of
setup is as follows in Table 1.

Regime | $r$ | $c_{1}$ | $c_{2}$ | $\lambda_{1}$ | $\lambda_{2}$ |
---|---|---|---|---|---|---
| 0.02 | 0.05 | 0.02 | 0.20 | 0.30 |
2 | 0.03 | 0.10 | 0.20 | 0.80 | 0.70 |
Table 1: Parameters values

The reinsurance premium rates are computed from expectation premium principle
as:

$g_{k}\left(a_{k}\right)=\left(1+l_{k}\right)\left(1-a_{k}\right)\mathbb{E}\left[A_{k}\right],$

where $l_{1}=1.1$ and $l_{2}=1.15$. Further, the volatility and the drift of
the financial market index and the risky return rate are modeled respectively
by:

$\begin{split}\mu_{S}\left(i,Z\right)&=0.2iZ;\\\
\sigma_{S}\left(i,Z\right)&=0.4iZ;\\\
\mu_{Z}\left(t,Z\right)&=0.4\left(t+1\right)Z;\\\
\sigma_{Z}\left(t,Z\right)&=0.1\left(t+1\right)Z.\\\ \end{split}$

State discretization follows $\delta=0.04$ and $h=0.2$. For reinsurance, we
discretize the reinsurance rate into six levels uniformly located from 0 to 1.
The investment amount is free from restrictions, and can vary from -3 to 3
with 0.2 increase. The trend of investment and reinsurance for the varying
relative surplus of player one is plotted in Figure 1, and that for the second
player is in Figure 2.

Figure 1: Controls for varying $X_{1}$ with $T=0.08,$ $Z=1.01,$ and $X_{2}=0$.

From Figure 1, we can observe that both players always hold a low proportion
of claim, which is due to the fact that both players are very risk-averse. A
big claim will not only reduce their relative surplus but also drive the
surplus of their opponent side up. Precisely, in regime one, the proportion
held by player two is 0, which is less than 0.2 of player one. This results
from the claim arriving rate of the player two is relatively much higher than
that of player one. In regime two, considering the high premium income rate
and the same expected claim amount, a small proportion of claim is affordable
for both players.

In view of the investment part of Figure 1, player two’s investment amount is
always no less than that of player one. Since the relative sensitivity of
player one to player two is higher, and player two’s relative surplus is at a
higher relative level initially, player two is willing to accept more risk for
higher expected return in order to beat player one. Then, along with growing
$X_{1}$, since player one’s condition is improved, he tends to bear less risk.
Meanwhile, player two adopts conservative strategy as well, since he can lower
the uncertainty and make use of his advantage that he owns a higher premium
income rate. Because a higher market volatility in regime two will introduce
more risk, both players choose the investment amount much closer to 0 to lower
the uncertainty.

Figure 2: Controls for varying $X_{2}$ with $T=0.08,$ $Z=1.01,$ and $X_{1}=1$.

Similar results can be seen from Figure 2. For the reinsurance part, player
two initially holds full proportion of a claim, since $X_{2}=-1$ leaves player
two in a relative bad situation compared to $X_{1}=1$. To change this
situation, he chooses to bear lots of risk to reduce the loss from reinsurance
premium. This is more obvious for regime 1, where the claim arriving rate is
lower. Hence, in more risky scenarios where claim arriving rate is higher,
players show relative risk averse and transfer more risks by reinsurance
tools. For the investment part, we can see that player two hold higher
positions in risky assets in his portfolio in both regimes, which is
consistent with the observations in Figure 1.

##  6 Concluding Remarks

In this paper, we considered a non-zero-sum stochastic investment and
reinsurance game between two insurance companies. Both proportional and non-
proportional reinsurance contracts were considered. Although we are able to
obtain the systems of HJI equations using dynamic programming principle,
solving the problem explicitly is virtually impossible. Thus we developed a
numerical scheme using the Markov chain approximation method (MCAM) to solve
the problem. Due to the complexity of the stochastic game formulation, even
numerically solving the systems of HJI equations is much more difficult than
that of the previous work in stochastic optimization problems. The
difficulties arise from the following two aspects. (1) With complex nonlinear
state processes, the formulated high-dimension problem adds much difficulties
in building approximating Markov chain. (2) The curse of dimensionality makes
a significant impact and slow down the computation due to the large numbers of
control variables and the dimensions of the HJI systems. Although the paper
was devoted to a problem arising in risk management and insurance fields, the
game problem formulation and the numerical methods developed can be more
widely used in various other control and game problems.

For our problem, the nature of the Markov chain approximation relies on
building a high dimensional lattice of both driving state and control strategy
to approximate the value functions under different control scenarios. The
optimization on every state follows the same computing rule, leading to the
possibility of using parallel acceleration techniques. The first option coming
to our mind is to incorporate multi-thread programming techniques into our
completed C++ MCAM template library, which enables us to reduce development
time by reusing the algorithm architecture of single-thread library. The
latest eighth generation Intel CPUs are equipped with six computation cores,
which allow maximal twelve threads to run simultaneously. If we parallelize
the algorithm using ten threads, we can enhance the time efficiency ten times.
However, this is not enough to handle the computational complexity required
for our problem. The high dimensionality requires the lattice to be very
precise, thus obtaining accurate results relies on generating a large number
of nodes. Ten times acceleration seems a big enhancement, but it can only
allow us to explore $10^{\frac{1}{7}}\approx 1.39$ times of $Z$, $X_{1}$,
$X_{2}$, $a_{1}$, $a_{2}$, $b_{1}$, and $b_{2}$, which is unable to meet
requirements of the computational complexity.

GPU acceleration, e.g., CUDA, is another attractive choice here. Although the
frequency of GPU core is much lower than that of CPU, the number of GPU cores
is usually hundreds of times of the number of CPU cores, and this makes GPU
more suitable for parallel computation. The tenth generation NVidia GPU owns
more than two thousand CUDA cores, which make it an easy solution for solving
the complexity issue of MCAM algorithm on common stochastic optimization
problem, where the maximal or the minimal value on a state is acquired from
repeatedly comparing the newly computed value function value against the
temporary optimal value so far. However, focusing on MCAM algorithm on our
high dimensional game problem, CUDA acceleration is of very limited use. Not
like CPU memory, which can be easily more than 64GB, the capacity of GPU
memory is usually less than 8GB. The equilibrium strategy is obtained by
searching on the value function information stored for different values of the
control strategy. As a result, this memory consumption will occur for every
GPU thread, which will easily lead the aggregated memory consumed by MCAM
algorithm to exceed the GPU’s memory capacity. From the above considerations,
it appears that using parallel programming techniques to high dimensional game
problems needs a lot of more thinking and effort. Finding more efficient way
for the numerical solution is our on-going work.

## References

  * (1)
  * (2)
  * Asmussen et al., (2000) Asmussen, S., Høgaard, B., Taksar, M. (2000).  Optimal risk control and dividend distribution policies. Example of excess-of loss reinsurance for an insurance corporation,  Finance and Stochastics, 4.3: 299-324.
  * Bensoussan and Frehse, (2000) Bensoussan, A., Frehse, J. (2000).  Stochastic games for N players,  Journal of Optimization Theory and Applications, 105(3): 543-565.
  * Bensoussan et al., (2017) Bensoussan, A. Hoe, S., Yan, Z., and Yin, G. (2017). Real options with competition and regime switching, Math. Finance, 27: 224-250.
  * Bensoussan et al., (2014) Bensoussan, A., Siu, C. C., Yam, S. C. P. and Yang, H. (2014).  A class of non-zero-sum stochastic differential investment and reinsurance games,  Automatica, 50(8): 2025-2037.
  * Bensoussan et al., (2012) Bensoussan, A., Z. Yan, Z., and Yin, G. (2012). Threshold-type policies for real options using regime-switching models, SIAM J. Financial Math., 3: 667-689.
  * Browne, (1995) Browne, S. (1995).  Optimal investment policies for a firm with a random risk process: exponential utility and minimizing the probability of ruin.  Mathematics of Operations Research 20(4), 937-958.
  * Chen et al., (2010) Chen, S., Li, Z. and Li, K. (2010).  Optimal investment-reinsurance policy for an insurance company with VaR constraint.  Insurance: Mathematics and Economics 47(2), 144-153.
  * Chen and Shen, (2017) Chen, L. and Shen, Y. (2017).  On a new paradigm of optimal reinsurance: A stochastic Stackelberg differential game between an insurer and a reinsurer.  Working paper.
  * Choulli et al., (2001) Choulli,T., Taksar, M. and Zhou, X.Y. (2001).  Excess-of loss reinsurance for a company with debt liability and constraints on risk reduction,  Quantitative Finance, 1:573-596.
  * Elliott and Siu, (2011) Elliott, R.J., Siu, T.K. (2011).  A stochastic differential game for optimal investment of an insuerer with regime switching,  Quantitative Finance, 11(3): 1-16.
  * Espinosa and Touzi, (2015) Espinosa, G., Touzi, N. (2015).  Optimal investment under relative performance concerns,  Mathematical Finance, 25(2):221-257.
  * Hald and Schimidli, (2004) Hald, M. and Schimidli, H. (2004)  On the maximization of the adjustment coefficient under proportional reinsurance,  ASTIN Bulletin 34: 75-83.
  * Liu and Yiu, (2013) Liu, J. and Yiu, K. F. C. (2013).  Optimal stochastic differential games with VaR constraints.  Discrete and Continuous Dynamical Systems Series B 18(7), 1889-1907.
  * Lundberg, (1903) Lundberg, F. (1903)  Approximerad Framställning av Sannolikehetsfunktionen, Aterförsäkering av Kollektivrisker, Almqvist & Wiksell, Uppsala. Akad. Afhandling, Almqvist o. Wiksell, Uppsala.
  * Meng et al., (2015) Meng, H., Li, S. and Jin, Z. (2015).  A reinsurance game between two insurance companies with nonlinear risk processes.  Insurance: Mathematics and Economics 62, 91-97.
  * Meng et al., (2013) Meng, H., Siu, T. K. and Yang, H. (2013).  Optimal dividens with debts and nonlinear insurance risk processes.  Insurance: Mathematics and Economics 53, 110-121.
  * Jin et al., (2013) Jin, Z., Yin, G., and Wu, F. (2013).  Optimal reinsurance strategies in regime-switching jump diffusion models: Stochastic differential game formulation and numerical methods,  Insurance: Mathematics and Economics, 53:733-746.
  * Kushner, (1990) Kushner, H. (1990).  Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems,  Birkhäuser, Boston, MA.
  * Kushner and Dupuis, (2001) Kushner, H. and Dupuis, P. (2001).  Numerical Methods for Stochastic Control Problems in Continuous Time, second ed,  Stochastic Modelling and Applied Probability, vol.24.  Springer, New York.
  * Pun and Wong, (2016) Pun, C. S., and Wong, H. Y. (2016).  Robust non-zero-sum stochastic differential reinsurance game.  Insurance: Mathematics and Economics, 68, 169-177.
  * Sotomayor and Cadenillas, (2011) Sotomayor, L. and Cadenillas, A. (2011)  Classical, singular, and impulse stochastic control for the optimal dividend policy when there is regime switching.  Insurance: Mathematics and Economics, 48(3), 344–354
  * Taksar and Zeng, (2011) Taksar, M. and Zeng, X. (2011).  Optimal non-proportional reinsurance control and stochastic differential games.  Insurance: Mathematics and Economics 48(1), 64-71.
  * Wei et al., (2010) Wei, J., Yang, H. and Wang, R. (2010).  Classical and impulse control for the optimization of dividend and proportional reinsurance policies with regime switching.  Journal of Optimization Theory and Applications, 147(2).
  * Yan et al., (2017) Yan, M., Peng, F. and Zhang, S. (2017).  A reinsurance and investment game between two insurance companies with the different opinions about some extra informaton.  Insurance: Mathematics and Economics 75, 58-70.
  * Yin et al, (2009) Yin, G., Jin, H., and Jin. Z.,  Numerical methods for porfolio selection with bounded constrains,  Journal of Computational and Appied Mathematics, 233(2009), 564-581.
  * Yin et al., (2003) Yin, G., Zhang, Q., and Badowski, G. (2003).  Dicrete-time singularly perturbed Markov chains: aggregation, occupation measures, and switching diffusion limit,  Advances in Applied Probability, 35, 449-476.
  * Yin and Zhu, (2010) Yin, G. and Zhu, C. (2010).  Hybrid Switching Diffusions: Properties and Applications.  Springer, New York
  * Zeng, (2010) Zeng, X. (2010).  Stochastic differential reinsurance games,  Journal of Applied Probability, 47(2): 335-349.
  * Zhang et al., (2016) Zhang, X., Meng, H. and Zeng, Y. (2016).  Optimal investment and reinsurance strategies for insurers with generalized mean-variance premium principle and no-short selling.  Insurance: Mathematics and Economics 67, 125-132.
  * Zhang and Siu, (2012) Zhang, X. and Siu, T. K. (2012).  On optimal proportional reinsurance and investment in a Markovian regime-switching economy.  Acta Mathematica Sinica (English Series) 28, 67-82.
  * Zhu, (2014) Zhu, J. (2014).  Dividend optimization for a regime-switching diffusion model with restricted dividend rates.  ASTIN Bulletin, 44:459–494.