capbtabboxtable[][]

# Pathways to Good Healthcare Services and Patient Satisfaction: An
Evolutionary Game Theoretical Approach

Zainab Alalawi, The Anh Han, Yifeng Zeng  Aiman Elragig

School of Computing and Digital Technologies, Teesside University, United
Kingdom, TS1 3BX
Emails: {z.alalawi, t.han, y.zeng<EMAIL_ADDRESS>
###### Abstract

Spending by the UK's National Health Service (NHS) on independent healthcare
treatment has been increased in recent years and is predicted to sustain its
upward trend with the forecast of population growth. Some have viewed this
increase as an attempt not to expand the patients' choices but to privatize
public healthcare. This debate poses a social dilemma whether the NHS should
stop cooperating with Private providers. This paper contributes to healthcare
economic modelling by investigating the evolution of cooperation among three
proposed populations: Public Healthcare Providers, Private Healthcare
Providers and Patients. The Patient population is included as a main player in
the decision-making process by expanding patient's choices of treatment. We
develop a generic basic model that measures the cost of healthcare provision
based on given parameters, such as NHS and private healthcare providers' cost
of investments in both sectors, cost of treatments and gained benefits. A
patient's costly punishment is introduced as a mechanism to enhance
cooperation among the three populations. Our findings show that cooperation
can be improved with the introduction of punishment (patient's punishment)
against defecting providers. Although punishment increases cooperation, it is
very costly considering the small improvement in cooperation in comparison to
the basic model.

## Introduction

The NHS is a free healthcare service at the point of delivery in the UK funded
through taxpayers' contributions (Slawson,, 2018). Many public healthcare
services are currently allocating amounts of their budgets to sourcing
services from Independent Healthcare Services Provider (ISP)/private sector.
Motivated by the NHS’s Five Year Forward View (FYFV), we choose the patient as
the core focus of healthcare planning who is to be included in the decision-
making process (England,, 2014; Ham,, 2017) for better health, patient care
and financial sustainability. Several systematic studies looking into the
improvement of clinical decision-making find that most patients expect to be
informed about their situation and the treatment required, and play an
important role in their clinical decision-making. However, little attention
has been given to understanding the patient's role quantitatively as part of
dynamic system modelling.

To investigate the dynamic system interactions among Public and Private health
sectors and Patients, we resort to an Evolutionary Game Theory (EGT)-based
solution. Researchers have applied EGT in a wide range of disciplines running
the gamut from economics, politics and security, to ecology, mathematical
biology, and computer science (Adami et al.,, 2016). EGT allows us to
understand and analyze the complex healthcare system, interactions between
individuals from various populations in a game, and how strategic behavior
might evolve among individuals (Nowak, 2006a, ; Morgenstern and Von Neumann,,
1953).

The main challenges of using EGT lie in formulating a valid payoff matrix and
defining the ties between parameters within the proposed payoff matrix of each
population. Some research on the evolution of cooperation has focused on the
human willingness to engage in behavior that would involve paying a cost in
return for imposing punishment on defectors or perceived wrongdoers (Sigmund
et al.,, 2010; Dreber et al.,, 2008). Punishment is one mechanism that can
enhance cooperation between individuals caught in social dilemmas (Hauert et
al.,, 2007). Punishment can be seen applied in human society, such as by
punishing free-riders (Sigmund et al.,, 2010), and in governance or
institutional systems that impose rewards and punishments on agents
(participants) (Andreoni et al.,, 2003). Different types of punishment can be
implemented based on the structure of the played game: peer punishments, pool
punishments and institutional punishments (Sigmund et al.,, 2010).

In this article, we consider the patient's role when developing an EGT
decision-based model of healthcare services through a tripartite, one-shot EGT
game (Gigerenzer and Gaissmaier,, 2011; Elwyn et al.,, 2012). In our proposed
model, we measure patients satisfaction and healthcare providers' reputation
which impact the quality of healthcare services (Ham,, 2017; Robertson et
al.,, 2017). The model is aimed at identifying dynamic interactions that could
enhance cooperation between the three populations. It will allow us to
ascertain how selecting certain decisions within a dynamic system would
influence patients' satisfaction with the provided service and their
willingness to cooperate. This could have important implications for
addressing the significant, alarming drop in public satisfaction with NHS-
provided services in recent years (Robertson et al.,, 2017). The social norm
behavior of individuals is analyzed by applying EGT using one-shot game.

The dynamic of the game is computed through stochastic selection of strategies
based on patients' satisfaction with received treatments and providers'
reputations. The main question in this article is about how patients influence
the dynamics of their cooperation in an EGT-based framework of three
populations. The main contribution of this article is to build a basic model
for the interactions of the three populations that closely capture the costs
and benefits of every strategy combining decisions by agents in finite
populations on either cooperation or defection. We further introduce peer
punishment into the model: the patients' punishment which takes the form of
complaints for clinical negligence (Bryden and Storey,, 2011; Cooper et al.,,
2011).

The main contributions are summarized below.

  * •

We develop a simple, yet expressive, basic mathematical model to formulate
interactions among the three populations: Patients, Private and Public health
sectors. This is the first dynamical model that captures the interactions of
these populations in healthcare economic modelling research;

  * •

We develop a mechanism to study the behavior of individuals in each population
and extend the basic model by introducing punishments in their interactions;

  * •

We analyze the models and examine how the relevant factors would influence
cooperation among individuals within the populations;

  * •

We conduct a comprehensive simulation analysis to determine various types of
behavior most-frequently adopted by individuals based on certain factors.

## Related Work

The rapid development in research on the learning of social behavior has
significantly increased our understanding of the dynamic interaction among
individuals from different populations (Nowak, 2006a, ; Sigmund et al.,,
2010). Cooperation is one of the fundamental indicators to measure the
strength and dynamism of a population (Smith,, 1974; Kurokawa and Ihara,,
2009; Encarnação et al.,, 2016). It can be studied by applying EGT using
different types of mechanisms, such as reciprocal behaviors, mutual
reciprocity among populations, replication, kin selection and costly
punishment (Nowak, 2006b, ; Hofbauer and Sigmund,, 1998). Researchers seeking
to understand the behavior of different agent representations within the
healthcare system use AI (Anh et al.,, 2013), game theory (Brekke and
Sørgard,, 2007), multi-agent systems (De and Gelfand,, 2017) and big data
(Murdoch and Detsky,, 2013) to predict and understand behaviors within the
system.

(Brekke and Sørgard,, 2007) argued that having a blurred line between the
private and public healthcare providers within the NHS might lead to
imbalances in the costs of provided health services and a drift towards
privatisation. While (Wu et al.,, 2016) developed their proposed set of
various non-cooperative and cooperative games for the Emergency Department
response based on different types of patients. Another research investigates
different dilemmas based on a three-population EGT framework involving the
cost for prescribed antibiotics via healthcare providers (Bettinger,, 2016).
The main limitations are related to the actual cost paid for prescriptions and
efficiency in quantifying incentives of patients for selecting the most
satisfying or preferred provider. Another research by Encarnação (Encarnação
et al.,, 2016) shows that the advent of the civil sector adds another layer of
complexity to a scene that used to be dominated by two sectors: private and
public.

Cooperation level is analysed by frequency-dependent selection of strategies
within the populations (Kandori et al.,, 1993; Taylor et al.,, 2004). In this
context, we seek to show how a drift towards a cooperate strategy in
interactions between the three populations can be promoted by the adoption of
the most dominant strategy (social behavior) in our proposed model. Based on
the stochastic factors and processes associated with the healthcare model,
this article intends to investigate evolving societal behavior between
patients and different sectors in the healthcare dynamic system. The selection
of the patient population in our model was made for the following reasons:
finding the best behavior and strategy for decision-makers among the three
populations; determining the impact of implemented peer punishments by a
patient and how this social mechanism could influence the decision-making
process for better cooperation; and, finding the best strategy for involved
populations by computing the interaction between private and public sectors
(Cooper et al.,, 2011; Brekke and Sørgard,, 2007).

A stochastic multi-objective auto-optimisation model was introduced by Bastian
et al. to effectively manage resource allocation for the military health
system with an eye to achieving a more efficient funding and staffing
distribution between the Army, Air Force and Navy (Bastian et al.,, 2017).
Bastian et al.'s research suffers from serious limitations; primarily, that
the model introduced was not generic and depended on fixed inputs.

The Patient population in our model plays a major role (as discussed in the
results) in influencing decision-making. Our analysis significantly improves
our understanding of the model structure and the factors that lead to
cooperation. It helps us answer important questions such as: What factors
might influence the patient's rating of the healthcare services? What factors
would induce healthcare providers to seek better reputations when the patient
derives no or little benefit from the treatment (e.g. the patient files a
complaint for receiving bad healthcare services).

## EGT-Based Solutions

### Basic Model and Extended Model

#### Model I - Basic model

In this model we consider three populations: Public providers, Private
providers and consumers/Patients. While Public represents the NHS or the
Public healthcare providers, Private independent healthcare providers sell
healthcare services, and Patient represents a person seeking treatment(s). An
individual from each population (Public, Private and Patient) can choose from
two strategies: provide/accept sustainable treatment(s) identified as
cooperating, otherwise can’t provide/refuse treatment(s) leading the patient
to seek alternative treatment(s) from other providers. An agent's payoff is
acquired based on the strategy played by each individual from the three
populations, as explained in Table 1.

Parameters' description | Symbol
---|---
Reputation benefit for the Public and Private healthcare providers | $b_{R}$
Patient's benefit | $b_{P}$
Cost of investment spent by the Public/Private healthcare provider | $c_{I}$
Cost of treatment acquired by the healthcare provider | $c_{T}$
Cost of healthcare management | $c_{M}$
Extra Patient's benefit when both providers cooperate | $\varepsilon$

Strategies | Payoffs
---|---
P1 |  P2 |  P3 |  Public |  Private |  Patient
C |  C |  C |  $b_{R}-c_{I}-c_{T}$ |  $b_{R}-c_{I}-c_{M}$ |  $b_{P}+\varepsilon b_{P}$
C |  C |  D |  $-c_{I}$ |  $-c_{I}$ |  0
C |  D |  C |  $b_{R}-c_{I}-c_{T}$ |  0 |  $b_{P}$
C |  D |  D |  $-c_{I}$ |  0 |  0
D |  C |  C |  0 |  $b_{R}-c_{I}-c_{M}$ |  $b_{P}-c_{T}$
D |  C |  D |  0 |  $-c_{I}$ |  0
D |  D |  C |  0 |  0 |  $-c_{T}$
D |  D |  D |  0 |  0 |  0

Table 1: The healthcare model (Public healthcare providers P1, Private
healthcare providers P2 and Patient P3).

Every individual or agent in each of the three populations experiences one of
the following scenarios based on two strategies, namely, cooperate (C) and
defect (D). This allows us to understand how cooperation evolves in altruistic
interactions among individuals in a game. The following are the strategies an
individual within each population can select (refer to Table 1):
Public healthcare providers: 1 (Cooperate, C) offers treatment paid for from
taxpayers' money, in return gets a reputation benefit. (In case of
cooperation, the public commits to invest ($c_{I}$) from allocated budget). 2
(Defect, D) does not want to pay for the treatment.
Private healthcare providers: 1 (Cooperate, C) offers treatment either paid by
Public (when Public cooperates) or self-paid by Patient (so the main cost
involved is represented by management cost ($c_{M}$)), and obtains a
reputation benefit ($b_{R}$). In case of cooperation with the Patient, Private
commits to invest ($c_{I}$) from its revenue. 2 (Defect, D) does not want to
offer the treatment.
Patient: 1 (Cooperate, C) accepts the treatment and pays for the treatment
($c_{T}$) in the Private instance; Patient obtains health benefit ($b_{P}$) if
one provider cooperates and an extra health benefit ($\varepsilon b_{P}$) when
both providers cooperate. 2 (Defect, D) rejects the treatment and looks for
alternative treatment mostly overseas.

The main issue we investigate in this article is the spending and cost
effectiveness in the healthcare system with an eye to elucidating the social
dilemma as mapped in Table 1. The dilemma questions the probability of
cooperation (C) among the two sectors (Public and Private) and consumers
(Patient) resulting in sustainable spending and cost effectiveness of the
provision of treatment funded from the taxpayers' money111O'Connell in this
https://bit.ly/2VBY4im, article talks about finding a structural change to
reduce NHS spending and enhance the tendency towards saving..

When applying the evolutionary rules as explained in Fig. 1 on the matrix
given in Table 1, the payoff is a simple representation of the healthcare
cost-effectiveness, healthcare providers' reputation benefit and patient's
benefit. The eight possible strategic scenarios are described below:

Figure 1: Evolution dynamics of the simplex's edges represents three-player
actions (Basic model).

• Individuals from all three populations choose to cooperate, (CCC). In this
case, the Public opts to pay for the treatment provided by the Private sector
and the Patient accepts the provided treatment in pursuit of her/his own
benefit of wellbeing and better health. The Public bears the costs of
investment and pays for the Patient's treatment ($c_{I}+c_{T}$) from its
allocated budget. It gains reputation benefits ($b_{R}$) derived from the
treatment provided by the Private healthcare provider. Reputation benefits are
derived from the Patient's satisfaction with the provided service. On the
other hand, the Private sector will provide the required treatment to the
Patient and receive the payment covering the costs from the Public. The cost
of investment ($c_{I}$) of the Private healthcare provider is to invest in
staffing and healthcare facilities, while ($c_{M}$) includes administrative
and operational costs. Consequently, both the Private and the Public obtain a
reputation benefit ($b_{R}$). Patient obtains extra health benefit
($\varepsilon b_{P}$) as both healthcare providers are cooperating, where
($\varepsilon$) captures a fraction of the Patient benefit.

• Both the Private and Public healthcare providers want to pay and provide
treatment to the patient, but the patient rejects the service (CCD). The
payoff indicates that the Public healthcare provider will still invest
($c_{I}$) back in the Public as it is set to cooperate. Similarly, the Private
sector only invests ($c_{I}$) back into its own resources, which include
staffing, equipment and research, and gets nothing in return. The Patient's
payoff is 0 as no treatment cost was involved; nor did she/he get health
benefits from the services of the healthcare providers involved.

Figure 2: Evolution dynamics of the simplex's edges represents three-player
actions (Extended model with patient's punishment).

• The Patient accepts the treatment to be provided by the Private sector and
paid for by the Public but the Private refuses to provide the treatment (CDC).
The Public's payoff consists of the cost of investment and treatment
($c_{I}+c_{T}$), which are covered from its allocated budget, and a reputation
benefit ($b_{R}$) is accumulated based on patients' satisfaction. The cost of
accepting the treatment for the Patient is 0 and s/he obtains health benefit
($b_{P}$).

• The Public is the only party willing to provide treatment, but the Patient
rejects the treatment offered (CDD). So, there is neither treatment cost nor
benefit returned. Hence, the payoff for both the Private healthcare provider
and the Patient is 0, while the Public still bears a cost of investment
($c_{I}$) in new treatments (Sapiña et al.,, 2018).

• The Public healthcare provider opts out of providing treatment and refuses
to pay for it; therefore, the payoff for the Public is 0 (DCC). As the Public
does not want to provide and pay for the Patient's treatment, the Patient
would look for treatment provided by a Private sector in a competitive market.
The Private sector's payoff is derived from the gains it makes in its own
reputation benefit ($b_{R}$); even when it is not cooperating with the Public.
The payment received goes towards defraying the costs of investment and
management ($c_{I}+c_{M}$). The Patient, on the other hand, gets health
benefits and pays the cost of the treatment ($b_{P}-c_{T}$).

• Only the Private healthcare provider offers to provide treatment with a
specified price tag while neither the Public wants to pay for the treatment
nor the Patient is accepting the treatment (DCD). The payoff for the latter
two agents is 0, while the transaction involves an investment cost ($c_{I}$)
to be taken out of the Private provider's budget.

• None of the healthcare providers is willing to provide treatments (DDC).
This situation leads the Patient to look for alternative treatments, both
domestically and possibly overseas, and certainly the Patient has to pay for a
treatment cost ($c_{T}$).

• The situation depicts all three agents choosing not to interact with one
another (DDD). Hence, there will be no winner or loser and all agents get a 0
payoff.

Parameters' description | Symbol
---|---
Patient's punishment cost | v
Fee paid by punished agent | u

Strategies | Payoffs
---|---
P1 |  P2 |  P3 |  Public |  Private |  Patient
C |  C |  C |  $b_{R}-c_{I}-c_{T}$ |  $b_{R}-c_{I}-c_{M}$ |  $b_{P}+\varepsilon b_{P}$
C |  C |  D |  $-c_{I}$ |  $-c_{I}$ |  0
C |  D |  C |  $b_{R}-c_{I}-c_{T}$ |  -v |  $b_{P}-u$
C |  D |  D |  $-c_{I}$ |  0 |  0
D |  C |  C |  -v |  $b_{R}-c_{I}-c_{M}$ |  $b_{P}-c_{T}-u$
D |  C |  D |  0 |  $-c_{I}$ |  0
D |  D |  C |  -v |  -v |  $-c_{T}-2u$
D |  D |  D |  0 |  0 |  0

Table 2: The healthcare model with Patient's Punishment (Public healthcare
providers P1, Private healthcare providers P2 and Patient P3).

#### Model II- Model with Patient's Punishment

This model extends the basic version by introducing peer punishment. Patient
has the choice to punish defected healthcare providers after a one-shot game
has been played. Costly punishment means the patient pays a cost u to force
the defecting healthcare provider to pay a cost of clinical negligence v. The
model payoff matrix is provided in Table 2. (Here we assume that, u $<$ v)
(Fowler,, 2005).

When an individual from healthcare providers fails to fulfill their commitment
to provide acquired healthcare services up to standards, a patient's
punishment would be meted out to the defecting individual(s) from healthcare
providers v; simultaneously a cost u is accrued from the patient in this
process. Furthermore, if a co-player refuses to commit, the payoff for both is
0. The transition probability of a mutant playing strategy D to invade a
population of C players can be measured following the method explained below.

### Method: Evolutionary Dynamics for Three Populations

EGT method is adopted to study the evolutionary dynamics and interactions
among individuals from three distinct finite populations: Public healthcare
providers P1, Private healthcare providers P2 and Patient P3. Here, we assumed
that the populations are of a fixed size N. Every individual in each
population will be involved in one of the eight strategic scenarios as
mentioned earlier. Individuals have the choice to cooperate, (C) or to defect,
(D) in a paradigm shift fashion. In our proposed model, there are eight
possible paradigms corresponding to the the eight possible combinations of the
basic strategies within the three populations, namely, CCC, CCD, CDC, CDD,
DCC, DCD, DDC and DDD. Denoting the number of cooperators in P1, P2 and P3 by
$x$, $y$, and $z$, respectively, the payoff of each strategy can be written as
follows:

$\displaystyle\left.\begin{array}[]{rcl}P_{s}^{Public}(x,y,z)&=&P_{syz}\\\
P_{s}^{Private}(x,y,z)&=&P_{xsz}\\\ P_{s}^{Patient}(x,y,z)&=&P_{xys},\\\
\end{array}\right\\}\qquad$ (4)

Figure 3: The plots represent the frequency of all strategies adopted by the
populations. Panel (I) represents the frequency of all strategies adopted by
the populations based on the basic model's matrix. Panel (II) examines the
frequency of the model's with punishment strategies for varying ($b_{P}$ and
$b_{R}$) as stated, where: $u=0.5$ and $v=1.5$. Other parameters:
$c_{I},~{}c_{T},~{}c_{M}=1,~{}b_{R}=2$ and $3,~{}\varepsilon=0.2,~{}N=100$ and
$\beta=0.1$.

where $x,y,z\in\\{0,1\\}$, is the payoff for the strategy selected by
individuals from one of the stated populations, and (x,y,z) represents the
selected strategies C or D. For instance, individuals from P1 have the options
to play C or D. The selected strategy will replace the s at x vertex, while y
and z vertices remain unchanged for every selected strategy for the Public
population in this instance. The payoff of randomly selected individuals A and
B in the population depends on the proportion of both players in the
population. In each time step an individual B with fitness $\pi_{B}$ imitates
a randomly selected individual A with $\pi_{A}$ fitness adopting a pairwise
comparison rule. The probability $\rho$ that A adopts B's strategy is given by
the Fermi's function (Sigmund et al.,, 2010; Traulsen et al.,, 2007)

$\displaystyle\rho={[1+e^{-\beta[\pi_{A}-\pi_{B}]}]}^{-1}$ (5)

where the parameter $\beta\geq 0$ represents the ‘intensity of selection’or
‘imitation strength’($\beta=0$ represents neutral drift where imitation
decision is random, while for large $\beta\rightarrow\infty$ the imitation
decision is increasingly deterministic).

Figure 4: Stationary distribution and fixation probabilities. The parameters’
values are stated for each simplex, whereas: red arrow represents transition
towards defection, blue arrow transition towards cooperation and dashed line
refers to neutral state. A high fraction 80% of individuals would adopt
cooperation with lower $b_{P}$ in (II-A) compared to (I-A), whilst an adequate
1% increase in cooperation with higher $b_{R}$ in (II-B) compared to (I-B) is
registered. The transition probability and frequency dependency are normalised
($1/N$), where $N=100$. Other parameters:
$c_{I},c_{T},c_{M}=1,\varepsilon=0.2$ and $\beta=0.1$.

In order to construct a symmetric matrix, the fitness of an individual
adopting a strategy s within a population is derived from the average obtained
from the tripartite one-shot game described in Table 1. The social dynamics of
the three finite populations interacting in eight strategies as a combination
of Cs and Ds is represented by a death-birth process using pairwise comparison
(Traulsen et al.,, 2006; Nowak, 2006a, ). Individuals with the highest payoff
reproduce and their social behavior is adopted by weak opponents, i.e. the
invading agent. For instance an agent B from the Public population imitates
the successful strategic behaviour, C, by another regional healthcare provider
as agent A. Subsequently, the transition matrix is evaluated by values given
to the associated parameters as depicted in Table 1. At each time step, there
is a probability of stochastic selection of an individual from a population
whereby an individual B from one of the populations playing D with payoff
$f_{D}$ may imitate another randomly selected individual A with payoff $f_{C}$
from the same population. The probability of the occurrence of this action
(Nowak, 2006a, ; Kandori et al.,, 1993) is stated in the equation above. The
transition probability drifts for agent A playing C from k to $k^{\pm}$ is
given by:

$\displaystyle
T^{\pm}(k)=\frac{k}{N}\frac{(N-k)}{N}[1+e^{\mp\beta(\Pi_{C}(k)-\Pi_{D}(k))}]^{-1}$
(6)

As mentioned earlier, the process has two absorbing states $k=0$ and $k=N$. In
mixed populations, one of the absorbing states at the end would be a
population with either all-C or all-D. Determining the different fixation
probabilities $\rho_{D,C}$ is given by:

$\displaystyle\rho_{D,C}=\left(1+\sum_{i=1}^{N-1}\prod_{j=1}^{i}\frac{T^{-}(j)}{T^{+}(j)}\right)^{-1}$
(7)

The transition matrix $\Lambda$ with a set of $\left\\{1,\ldots,s\right\\}$
strategy (Encarnação et al.,, 2016; Nowak, 2006a, ; Anh et al.,, 2013) is:

$\displaystyle\Lambda_{ij,j\neq i}=\frac{\rho_{ji}}{3}\hskip 2.84526pt\text{
and }\hskip 2.84526pt\Lambda_{ii}=1-\sum_{j=1,j\neq i}^{s}\Lambda_{ij}$ (8)

The various fixation probability acquired from $\rho_{ij}$ is that a
population at a single state $i$ transits to another state $j$ when a mutant
from one of the populations adopts an s different strategy. In games with
large N, an invader can emerge as the stronger if the condition below is
correct (Sigmund et al.,, 2010):

$\displaystyle\sum_{i=1}^{N-1}\Pi_{C}(k)>\sum_{i=0}^{N}\Pi_{D}(k)$ (9)

## Results

### Pathways to healthcare cooperation

Figure 5: Frequency of strategy CCC for varying the main parameters $b_{R}$
and $b_{P}$: (I) for the basic model, (II) extended model with costly
punishment. In (II) $u=0.5$ and $v=1.5$. In general, CCC performs better when
punishment is introduced to the basic model. Additionally, significant
cooperation is achieved for sufficiently low $b_{R}$ and large $b_{P}$. Other
parameters $c_{I},c_{T},c_{M}=1,\varepsilon=0.2,N=100,$ and $\beta=0.1$.

In conducting a numerical and systematic analysis for the basic model, we
focus on the social interactions between players in each population and how
their decision influences the level of cooperation to sustain cost-effective
services and better patient satisfaction. (Fig. 3 I) represents the
computation investigating the frequency of adopting one of the eight
strategies in the basic model by analysing the stochastic behavior of mutation
in one of the three populations based on the frequency for each of the eight
strategies given in (Table 1). That allows us to measure the ultimate behavior
of those adopting the same strategy following the rules of social learning
(Rendell et al.,, 2010).

In the basic model, refer to (Fig. 3 I-A), where $b_{R}$ is small ($b_{R}$ =
2), the defection strategy DDD is pervasive. The players of each of the three
populations spend most of their time adopting defecting strategies rather than
cooperating strategy. By simulating the matrix implementing (Eqs. 7 and 8)
with a selected range of examined parameters ($b_{P}$ and $b_{R}$), as stated
in (Fig. 4 I-A), the analysis shows that the DDD strategy dominates the
populations’ dynamic by 99%. In other words, healthcare providers have to
invest more efforts in order to satisfy patients as better cooperation CCC is
achieved when ($b_{R}$) is sufficiently high.

Figure 6: Robustness of the results across game configurations and parameters
of the model with or without punishment (using 10000 samples). For the basic
model: (I) Parameter values: $0.5\leq b_{P}\leq 5$, $0.5\leq b_{R}\leq 5$.
(II) the model with punishment, where parameter values: $0.5\leq b_{P}\leq 5$,
$0.5\leq b_{R}\leq 5$, $0\leq u\leq 3$ and $u\leq v\leq 10u$ randomly sampling
from uniform distributions on the intervals. Other parameters:
$c_{M},c_{I},c_{T}=1$ and $\varepsilon=0.2,N=100,\\\ \beta=0.1$.

As it has been observed in the basic model where punishment is absent, players
of each population spend most of their time at the defecting strategy (see
Fig. 4 I-A). We started by pairwise computation of the interaction strategies
in the payoff matrix (Table 2) based on different values of the parameters
($b_{R}$ and $b_{P}$) to measure the stationary distribution and the frequency
of the eight strategies. Recalling that in our model the patient has the
option to mete out a costly punishment v to the defecting healthcare
provider(s) at u cost (i.e. legal fees).

Most finite populations drift towards cooperation (i.e, CCC) as the most
dominant strategy. In the presence of patient's punishment (see Fig. 4 II-A
$\&$ B), cooperators invade defectors when there is a large enough patient's
benefit (namely, $b_{P}>1$), where the populations spend at least 79.6% of
their time in cooperation. Our simulation suggests that, regardless of the
reputation's benefit, cooperation is highly achievable (see Fig. 4 II).

### Robustness of Parameters

A more compelling analysis has been carried out by computing 10000 samples
with the set of parameters stated in (Fig. 6) to obtain the stationary
distribution of cooperation for both the basic model and the extended model.
Considering the collected results in the plot for the basic model in (Fig. 4),
it is noticeable that the mean outcomes in (Fig. 6 I$\&$II) are approximate.

## Discussion and Conclusions

In summary, this paper investigates the behavior of individuals within the
three finite populations to achieve better cooperation. By analyzing the
performance of each population and its preferred path towards evolving and
adopting a new strategy, we explored how to move towards cooperation while
taking into consideration the cost of effectiveness and patient satisfaction
with the provided services. Basically, the cooperation rules of each of the
model's dynamics explained in (Fig. 1 & 2) clearly show that there are some
paths moving toward full cooperation (see the panels in Fig. 1).

In contrast, avoiding the enveloped state (where some critics alluded to great
fear of collaboration with private healthcare providers as ‘the beast of the
(P-Privatisation)’) is represented by the DDD node; the public sector
represents the main agent that can take the lead in changing the rules and
influence the behavior of the private sector and patient by introducing new
policies. For different values of $b_{R}$ and $b_{P}$, the frequency of
strategy CCC has improved in the extended model (as apposed to the basic
model), (see Fig. 5 A $\&$ B).

Moreover, the initial result yielded by the basic model is generally moving
towards defection, apart from cases where a considerably high reputation value
is imposed on the computation process, in which case there is a drift towards
defection. In comparison, the results obtained when patient’s costly
punishment is introduced for the populations show a heightened tendency
towards cooperation. This requires not only high patient's satisfaction but
also a high reputation gained by healthcare providers; however, a full
cooperation can be achieved within the basic framework on considerably low
patient’s satisfaction. In this case, the imposed peer punishment enhances
cooperation with low patient's satisfaction, but it is proved too costly to be
adopted in the proposed healthcare model. Realistic scenarios could be derived
from some elective treatments such as hip replacements (Moscelli et al.,,
2016). In fact, data collected by the NHS and Care Quality Commission (CQC)
(CQC,, 2019), and The Patient Reports Outcome Measures (PROMs) (NHSDigital,,
2019) are well represented in our proposed model in addition to some NHS
statistics that substitutes for the cost of treatment and expenditures. We do
recognise that our proposed models have some limitations, such as that our
models are not validated against real healthcare data (it will be considered
in future work) and the lack of comparison with other techniques. In this
work, we only focus on analysing the behavior between agents using basic
factors.
Future work will involve investigating the implementation of institutional
punishment, expanding the model (i.e., by introducing new factors) to study
the relation between the new elements and the dynamic behaviour of
individuals. Finally, our proposed model directs attention to how the
patient’s decision would impact the process of collaboration between
healthcare providers, and to the effectiveness of management decisions made by
the private sector in influencing the patient's choice of cooperation.

## Acknowledgements

This work was supported by EPSRC project No. (EP/S011609/1) and The Anh Han is
supported by Future of Life Institute (grant RFP2-154).

## References

  * Adami et al., (2016) Adami, C., Schossau, J., and Hintze, A. (2016).  Evolutionary game theory using agent-based methods.  Physics of Life Reviews, 19:1–26.
  * Andreoni et al., (2003) Andreoni, J., Harbaugh, W., and Vesterlund, L. (2003).  The carrot or the stick: Rewards, punishments, and cooperation.  American Economic Review, 93(3):893–902.
  * Anh et al., (2013) Anh, H. T., Pereira, L. M., Santos, F. C., and Lenaerts, T. (2013).  Why is it so hard to say sorry? evolution of apology with commitments in the iterated prisoner’s dilemma.  In IJCAI, pages 177–183. AAAI Press.
  * Bastian et al., (2017) Bastian, N. D., Ekin, T., Kang, H., Griffin, P. M., Fulton, L. V., and Grannan, B. C. (2017).  Stochastic multi-objective auto-optimization for resource allocation decision-making in fixed-input health systems.  Health Care Management Science, 20(2):246–264.
  * Bettinger, (2016) Bettinger, B. M. (2016).  Game theory and mechanism design for cooperative competition dilemmas between healthcare providers.  Doctoral dissertation, Northeastern University.
  * Brekke and Sørgard, (2007) Brekke, K. R. and Sørgard, L. (2007).  Public versus private health care in a national health service.  Health economics, 16(6):579–601.
  * Bryden and Storey, (2011) Bryden, D. and Storey, I. (2011).  Duty of care and medical negligence.  Continuing Education in Anaesthesia Critical Care & Pain, 11(4):124–127.
  * Cooper et al., (2011) Cooper, Z., Gibbons, S., Jones, S., and McGuire, A. (2011).  Does hospital competition save lives? Evidence from the English NHS patient choice reforms.  The Economic Journal, 121(554):F228–F260.
  * CQC, (2019) CQC (2019).  Care Quality Commission for All Healthcare services in the UK.  https://bit.ly/2VkFKyx.
  * De and Gelfand, (2017) De, Soham; Nau, D. S. and Gelfand, M. J. (2017).  Understanding norm change: An evolutionary game-theoretic approach.  In Proceedings of the 16 Conference on Autonomous Agents and MultiAgent Systems, pages 1433–1441. International Foundation for Autonomous Agents and Multiagent Systems.
  * Dreber et al., (2008) Dreber, A., Rand, D. G., Fudenberg, D., and Nowak, M. A. (2008).  Winners don't punish.  Nature, 452(7185):348.
  * Elwyn et al., (2012) Elwyn, G., Frosch, D., Thomson, R., Joseph-Williams, N., Lloyd, A., Kinnersley, P., Cording, E., Tomson, D., Dodd, C., Rollnick, S., et al. (2012).  Shared decision making: a model for clinical practice.  Journal of General Internal Medicine, 27(10):1361–1367.
  * Encarnação et al., (2016) Encarnação, S., Santos, F. P., Santos, F. C., Blass, V., Pacheco, J. M., and Portugali, J. (2016).  Paradigm shifts and the interplay between state, business and civil sectors.  Royal Society Open Science, 3(12):160753.
  * England, (2014) England, N. (2014).  Five year forward view.  Technical report, NHS England.
  * Fowler, (2005) Fowler, J. H. (2005).  Altruistic punishment and the origin of cooperation.  Proceedings of the National Academy of Sciences, 102(19):7047–7049.
  * Gigerenzer and Gaissmaier, (2011) Gigerenzer, G. and Gaissmaier, W. (2011).  Heuristic decision making.  Annual review of psychology, 62:451–482.
  * Ham, (2017) Ham, C. (2017).  Next steps on the NHS five year forward view.
  * Hauert et al., (2007) Hauert, C., Traulsen, A., Brandt, H., Nowak, M. A., and Sigmund, K. (2007).  Via freedom to coercion: the emergence of costly punishment.  Science, 316(5833):1905–1907.
  * Hofbauer and Sigmund, (1998) Hofbauer, J. and Sigmund, K. (1998).  Evolutionary games and population dynamics.  Cambridge University Press.
  * Kandori et al., (1993) Kandori, M., Mailath, G. J., and Rob, R. (1993).  Learning, mutation, and long run equilibria in games.  Econometrica: Journal of The Econometric Society, pages 29–56.
  * Kurokawa and Ihara, (2009) Kurokawa, S. and Ihara, Y. (2009).  Emergence of cooperation in public goods games.  Proceedings of the Royal Society of London B: Biological Sciences, 276(1660):1379–1384.
  * Morgenstern and Von Neumann, (1953) Morgenstern, O. and Von Neumann, J. (1953).  Theory of games and economic behavior.  Princeton University Press.
  * Moscelli et al., (2016) Moscelli, G., Gravelle, H., Siciliani, L., et al. (2016).  Market structure, patient choice and hospital quality for elective patients.  Technical report, Centre for Health Economics, University of York.
  * Murdoch and Detsky, (2013) Murdoch, T. B. and Detsky, A. S. (2013).  The inevitable application of big data to health care.  Jama, 309(13):1351–1352.
  * NHSDigital, (2019) NHSDigital (2019).  Finalised Patient Reported Outcome Measures (proms) in England for Hip and Knee Replacement Procedures (April 2017 to March 2018).  https://bit.ly/2J3Rq2h.
  * (26) Nowak, M. (2006a).  Evolutionary Dynamics: Exploring the Equations of Life.  Harvard University Press.
  * (27) Nowak, M. A. (2006b).  Five rules for the evolution of cooperation.  Science, 314(5805):1560–1563.
  * Rendell et al., (2010) Rendell, L., Boyd, R., Cownden, D., Enquist, M., Eriksson, K., Feldman, M. W., Fogarty, L., Ghirlanda, S., Lillicrap, T., and Laland, K. N. (2010).  Why copy others? insights from the social learning strategies tournament.  Science, 328(5975):208–213.
  * Robertson et al., (2017) Robertson, R., Appleby, J., and Evans, H. (2017).  Public satisfaction with the NHS and social care in 2016.  Results and trends from the British social attitudes survey. London: King’s Fund.
  * Sapiña et al., (2018) Sapiña, L., Ligero, S., and Domínguez, M. (2018).  Protons for Jehovahś Witnesses? how press coverage of Ashya King’s case brought proton beam therapy to the public sphere? (Padres secuestradores o empoderados? Como la cobertura mediatica del caso Ashya King difundio la terapia de protones en la esfera publica).  Estudios sobre el Mensaje Periodistico, 24(2):1773–1788.
  * Sigmund et al., (2010) Sigmund, K., De Silva, H., Traulsen, A., and Hauert, C. (2010).  Social learning promotes institutions for governing the commons.  Nature, 466(7308):861.
  * Slawson, (2018) Slawson, S. (2018).  NHS compensation payouts ‘unsustainable’, say health leaders.  The Guardian.
  * Smith, (1974) Smith, J. M. (1974).  The theory of games and the evolution of animal conflicts.  Journal of Theoretical Biology, 47(1):209–221.
  * Taylor et al., (2004) Taylor, C., Fudenberg, D., Sasaki, A., and Nowak, M. A. (2004).  Evolutionary game dynamics in finite populations.  Bulletin of Mathematical Biology, 66(6):1621–1644.
  * Traulsen et al., (2006) Traulsen, A., Nowak, M. A., and Pacheco, J. M. (2006).  Stochastic dynamics of invasion and fixation.  Physical Review E, 74(1):011909.
  * Traulsen et al., (2007) Traulsen, A., Pacheco, J. M., and Nowak, M. A. (2007).  Pairwise comparison and selection temperature in evolutionary game dynamics.  Journal of Theoretical Biology, 246(3):522–529.
  * Wu et al., (2016) Wu, C. K., Chen, Y. M., and Wu, D. (2016).  A game theory approach for deploying medical resources in emergency department.  In Game Theory and Applications, pages 18–30. Springer, Singapore.


11institutetext: 1 Security Research Institute, Edith Cowan University, Perth,
WA 6027, Australia.
2 Cyber Security Cooperative Research Centre, Australia.
3 Cybersecurity Center, Prince Mohammad Bin Fahd University, Khobar, SA.
4 Cyberstronomy Pty Ltd, Melbourne, VIC, Australia.
5 Data61, CSIRO, Sydney, NSW 2122, Australia.
∗Correspondence<EMAIL_ADDRESS>
# AI Potentiality and Awareness: A Position Paper from the Perspective of
Human-AI Teaming in Cybersecurity

Iqbal H. Sarker1,2,* 0000-0003-1740-5517 Helge Janicke1,2 0000-0002-1345-2829
Nazeeruddin Mohammad3 0000-0003-3580-5960 Paul Watters4 0000-0002-1399-7175
Surya Nepal2,5 0000-0002-3289-6599

###### Abstract

This position paper explores the broad landscape of AI potentiality in the
context of cybersecurity, with a particular emphasis on its possible risk
factors with awareness, which can be managed by incorporating human experts in
the loop, i.e., “Human-AI” teaming. As artificial intelligence (AI)
technologies advance, they will provide unparalleled opportunities for attack
identification, incident response, and recovery. However, the successful
deployment of AI into cybersecurity measures necessitates an in-depth
understanding of its capabilities, challenges, and ethical and legal
implications to handle associated risk factors in real-world application
areas. Towards this, we emphasize the importance of a balanced approach that
incorporates AI’s computational power with human expertise. AI systems may
proactively discover vulnerabilities and detect anomalies through pattern
recognition, and predictive modeling, significantly enhancing speed and
accuracy. Human experts can explain AI-generated decisions to stakeholders,
regulators, and end-users in critical situations, ensuring responsibility and
accountability, which helps establish trust in AI-driven security solutions.
Therefore, in this position paper, we argue that human-AI teaming is
worthwhile in cybersecurity, in which human expertise such as intuition,
critical thinking, or contextual understanding is combined with AI’s
computational power to improve overall cyber defenses.

###### Keywords:

Cybersecurity, Data Analytics, Machine Learning, AI Potentiality, AI Risk
Factors, Human-AI Teaming, Intelligent Systems.

##  1 Introduction

In today’s rapidly evolving digital technology ecosystem, cybersecurity has
taken on unprecedented significance. With the growth of interconnected
systems, protecting sensitive information and critical infrastructure has
become a top priority of a nation [1]. As threats to digital assets grow in
complexity and sophistication, there is an urgent need for innovative
approaches to effectively counter these concerns. In this position paper, we
focus on the potential of artificial intelligence (AI) as well as its
awareness, i.e., to handle possible risk factors, strengthened by human
expertise and observation in real-world application areas.

The combination of AI with human expertise has emerged as a potential solution
to addressing the ever-changing landscape of cybersecurity. AI systems have
shown promise in automating repetitive tasks, analyzing large datasets, and
discovering patterns that may be beyond human observation [2] [3]. These
competencies are particularly beneficial in the field of cybersecurity, where
the rapid identification and mitigation of threats and attacks are crucial.
Human experts, on the other hand, have a distinct set of abilities in their
understanding of context, intuition, and ethical judgment. This position paper
intends to dive into the intricate dynamics of this Human-AI collaboration,
exploring the synergies between AI’s computational capability and human
experts’ domain knowledge, contextual reasoning, and critical thinking in
their roles.

The cooperation between AI and human expertise in cybersecurity is not only an
issue of technological collaboration; it also extends to the ethical sphere.
The successful deployment of AI in cybersecurity operations necessitates
resolving issues of bias, accountability, transparency, and the right to
distinguish decision-making authority between machines and humans. Human
analysts can make sophisticated decisions depending on the broader context of
the organization’s goals and risk tolerance. Thus, human observations at the
application level are still required despite AI’s huge potential in computing
[4]. In short, we define “AI potentiality” as technical aspects of AI,
emphasizing its computational capabilities in terms of speed, accuracy,
scalability, and automation in cybersecurity tasks, and “AI awareness” as its
possible risk factors highlighting the importance of incorporating human
experts in the loop, e.g., human domain knowledge, intuition, and situational
awareness to make informed decisions in cybersecurity, which eventually help
to achieve organizations goal with proper responsibility and accountability.
Thus “Human-AI teaming”, in essence, maximizes the assets of both AI and human
intelligence, resulting in a more robust, effective, and explainable security
system for organizations. To better comprehend the core topic of this position
paper and overall contributions, we formulate three key questions below:

  * •

AI Potentiality: Is AI capable of effectively and efficiently processing,
analyzing, and extracting insights or useful knowledge from massive amounts of
cyber data, which is challenging for a human analyst?

  * •

AI Awareness: Are risk factors associated with real-world AI applications in
the context of cybersecurity, particularly how we can ensure responsibility
and accountability if the AI system fails in certain situations?

  * •

Human-AI Teaming: Is it worthwhile to establish human-AI teaming for
cybersecurity solutions and rethink the current cyberspace making informed
decisions with accountability in real-world application areas?

Overall, this position paper emphasizes a deeper understanding of Human-AI
teaming, particularly how the combination of AI and human expertise might
strengthen our cyber defense measures in various real-world application areas.
In the following sections, we provide an explicit understanding of the
potential of AI, as well as the importance of human experts in the loop in the
context of next-generation cybersecurity solutions. Therefore, this paper aims
to provide significant insights for cybersecurity academics, practitioners,
policymakers, and stakeholders, advancing discussions that lead the way for
successful and responsible cybersecurity practices in our digitally connected
society.

##  2 Why Human-AI Teaming in Cybersecurity?

In cybersecurity, the term “Human-AI teaming” refers to cooperation between AI
technologies and human expertise to boost the overall effectiveness of
cybersecurity solutions. In the following, we discuss how this teaming can
contribute:

  * •

Complementary Strengths: AI is particularly effective at swiftly analyzing and
processing large volumes of data, finding patterns, discovering anomalies, and
generating policies from complex specifications. However, there should be
clear accountability for decisions made by AI systems in real-world
cybersecurity applications. Human experts, on the other hand, have awareness
of the environment, intuition, and the skills to make complex decisions using
a combination of technological knowledge and hands-on expertise. Thus,
organizations can achieve better cyber attack or threat analysis and response
strategies with proper accountability through this integration.

  * •

Scale and Speed: Cyber threats are constantly evolving and can emerge at a
rapid pace. Attackers often employ automated tools and methods to exploit
vulnerabilities. AI can assist cybersecurity teams in analyzing threats and
taking action at a speed and scale that would be impossible for humans to
tackle by themselves. This speeding up is essential for minimizing potential
harm and promptly addressing emerging risk factors.

  * •

Decision Support: Advanced threats often consist of numerous stages and
sophisticated attack pathways. AI can give human analysts pertinent data and
insights to support their decision-making. This can include information about
the threat’s nature, its potential effects, and recommended solutions. While
humans can handle the more complicated and hidden aspects of investigations,
attribution, and response, AI can discover insights from data as well as
automate the necessary modules of threat detection and analysis.

  * •

User Behavior and Predictive Analytics: AI can monitor user behavior patterns
to find anomalies that could point to unauthorized access or compromised
accounts. Analyzing historical data AI can identify trends and predict
potential future threats, which eventually enables human analysts to take
proactive measures before an incident happens.

  * •

Continuous Learning and Improvement: To effectively respond to new threats
human analysts can regularly update and improve AI models. Although AI models
are capable of learning from previous data, they may require human supervision
to understand the importance of newly discovered patterns and anomalies. This
teaming enables a feedback loop where human experts can instruct and enhance
AI models based on their real-world experiences, resulting in progressively
more precise and efficient threat analysis.

  * •

Incident Response and Recovery: When a cyber incident occurs, human expertise
is essential for managing the situation, determining the level of the damage,
coordinating the response, and communicating with stakeholders. AI can assist
in quickly analyzing incident data, offering pertinent insights to guide human
decision-making as well as necessary automation. This helps human experts for
better strategic decisions, determine how an incident may affect other
stakeholder groups, and efficiently direct the recovery process.

  * •

Regulatory Compliance, Trust and Accountability: In many industries, security
operations are subject to regulations that require human oversight and
accountability. Particularly, need to ensure who will be responsible if AI
systems fail. Stakeholders often want to understand the rationale behind
security decisions. Cybersecurity experts can provide explanations for AI
outcomes that build trust, transparency, and accountability.

Although AI has huge potential, it may have risk factors, particularly in
terms of situational awareness, decision-making, and accountability, as
highlighted above. Thus, to increase overall cybersecurity resilience, it
needs to blend human intuition, critical thinking, and decision-making skills
with AI’s speed, scalability, data processing and learning capabilities.

##  3 AI-based Modeling and Potentiality

In this section, we first highlight what types of AI-based modeling can be
built, and then we explore multi-aspects AI methods that can contribute in the
context of cybersecurity modeling.

###  3.1 Major Types of AI Modeling

Generative AI and discriminative AI are two common approaches in the area,
with hybrid AI being a combination of the two. Generative AI is typically
focused on generating new data according to needs, while discriminative AI is
focused on classifying data. Hybrid AI combines these two approaches to take
advantage of their strengths to solve a particular problem. In the following,
we explore these with examples in the context of cybersecurity.

  * •

Generative AI: Generative models are trained to learn the underlying structure
of the data and generate new data that has similar characteristics to the
original data, e.g., Generative Adversarial Network (GAN) [5], as shown in Fig
1. In cybersecurity, generative models can be used for various tasks such as
attack simulation to test and enhance defense mechanisms, synthetic data
generation for training machine learning models, anomaly detection, etc.

  * •

Discriminative AI: Discriminative AI models aim to learn the decision boundary
between different classes of data, e.g., Random Forest (RF) with multiple
decision trees [6], as shown in Fig 2. In cybersecurity, discriminative models
can be used for various tasks such as intrusion detection, predicting threats,
fraud detection, malware analysis, etc.

  * •

Hybrid AI: Hybrid models combine the strengths of both generative and
discriminative models. For example, a hybrid model could use a generative
model to generate synthetic data that can be used to train a discriminative
model for a particular task [3]. In cybersecurity, hybrid AI can be used to
create more robust intrusion detection systems, detect advanced persistent
threats, simulate complex attack scenarios, etc.

Overall, each type of AI modeling mentioned has its own strengths and
weaknesses, and the best approach for a given task depends on the specific use
case and availability of the cyber data and relevant resources. A combination
of generative, discriminative, and hybrid AI approaches can be effective in
providing a comprehensive and effective cybersecurity solution by taking into
account their individual strengths. Thus, research should be focused in this
area for future-generation cybersecurity modeling.

Figure 1: Generative Adversarial Network
Figure 2: Random Forest

###  3.2 AI Methods and Algorithms

Although AI is a broader term in terms of techniques and application areas, we
explore the key AI methods that can be used in cybersecurity according to the
problem nature and available data or resources. These are:

  * •

Machine Learning:  Machine learning has emerged as a powerful tool in
cybersecurity. Machine learning algorithms such as Decision Trees, RF, SVM,
Logistic regression, Clustering, PCA, etc. [1] can analyze large volumes of
data, such as network traffic and system logs, to identify patterns and
anomalies that could indicate a potential cyber attack. By analyzing attack
patterns, machine learning algorithms can identify the root cause of security
breaches, enabling security teams to take corrective action more quickly and
effectively.

  * •

Deep Learning: Deep learning is a subset of machine learning that involves
training neural networks with multiple layers to learn from complex data sets
[3]. Deep learning models such as DNN, CNN, RNN, LSTM, Autoencoder, GAN, etc.
[1] have the ability to extract representational information and decision
rules simultaneously from data [7]. Deep learning algorithms can be used in
cybersecurity, particularly in the areas of threat detection, anomaly
detection, malware classification, and so on [1] [8].

  * •

Semantic Knowledge Representation and Reasoning: Semantic knowledge involves a
deeper understanding of the evolving threat landscape and the context of data,
which eventually helps security professionals make informed decisions in real
time. Several popular methods such as ontologies or knowledge graphs, i.e.,
graph-structured data model, can be used to represent knowledge about
potential threats, vulnerabilities, and assets in cybersecurity [9].

  * •

Knowledge or Rule Discovery: Knowledge discovery, e.g., rules, in
cybersecurity typically involves extracting useful insights and patterns from
large datasets that are used in intrusion detection, access control, and other
security systems. Rule-based AI models can leverage machine learning and data
science processes to enhance security solutions by generating a set of
optimized rules. For instance, clustering algorithms can be used to group
similar behavioral patterns of network traffic and create rules accordingly to
detect anomalies in real time. Association rule mining techniques can assist
security analysts in uncovering relationships between various events and
entities involved in the incident [3].

  * •

Language Model and Multimodality: Large language models (LLM) are typically at
the forefront of natural language processing (NLP) research and applications
due to their effectiveness in understanding and generating human language.
These models can analyze and understand vast amounts of unstructured text
data, such as threat reports to extract relevant threat intelligence. For
example, to summarize long threat reports and classify them into different
threat categories these models can be used, which eventually assist security
analysts in prioritizing their responses. In addition, multimodal intelligence
by taking into account the fusion of textual, visual, or sensor data [10] can
lead to more robust and comprehensive solutions in cybersecurity depending on
relevant data availability.

Overall, these multi-aspects AI methods and algorithms discussed as well as
their hybridization can provide powerful tools for detecting and responding to
security threats. However, it is important to carefully consider the strengths
and weaknesses of each method and to use them appropriately based on the
specific needs and characteristics of the systems being protected.

##  4 Real-World Cybersecurity Application Areas

In the real-world application areas, both IT (Information Technology) and
Industrial Control Systems (ICS) or Operational Technology (OT) systems are
important to the operation of modern organizations and business [4]. Human-AI
teaming is crucial for ensuring the security of IT and OT systems, networks,
and data from cyber threats. Hence, we summarize the potential usage of human-
AI teaming, which can be applicable for both IT and ICS/OT security systems:

  * •

Anomaly and Threat Detection with Real-time Monitoring: AI can continuously
monitor network traffic, system logs or sensor data to identify anomalies or
suspicious activity that might indicate a cyberattack. Human experts can then
evaluate and investigate these findings and take necessary action.

  * •

Incident Response and Recovery: AI can classify and prioritize security alerts
depending on severity, enabling human responders to focus on the most critical
incidents. AI may automate predefined incident response playbooks, reducing
risks immediately while notifying human teams for further investigation, and
strategic decision-making.

  * •

Risk Assessment and Vulnerability Management: AI can analyze vulnerabilities
and risks in both IT and ICS/OT environments by continuously investigating
configurations, system states, and emerging threats. Security analysts can
assess and prioritize risks and vulnerabilities for remediation depending on
the possible impact on critical infrastructure.

  * •

Predictive Analytics and Proactive Maintenance: AI can analyze historical data
and current trends to anticipate potential future threats or vulnerabilities.
Human experts use this predictive insight to resolve vulnerabilities and
security weaknesses in advance. In many cases, these predictive AI insights
can assist in designing maintenance plans and making informed decisions about
equipment replacements or upgrades.

  * •

Incident Investigation and Forensics: AI assists in capturing and analyzing
forensic data to determine the underlying causes of security problems. Human
investigators employ AI-generated insights to build together attack
narratives, identify criminals, and attribute breaches.

  * •

Adaptive Security: AI-powered adaptive security mechanisms adjust dynamically
based on real-time threat intelligence and system conditions. Human analysts
supervise and fine-tune AI algorithms to ensure they align with business goals
and regulatory constraints.

  * •

Visualization and Situational Awareness: AI analyses and visualizes data,
allowing human operators to instantly understand the security status. In
critical scenarios, human analysts investigate AI-generated visualizations to
make well-informed decisions.

  * •

Threat Intelligence and Contextualization: AI is capable of analyzing massive
amounts of threat data and providing actionable intelligence. Human analysts
contextualize threat intelligence, making it applicable to specific contexts.

  * •

Threat Hunting: Human analysts can employ AI-powered technologies to
proactively hunt for evidence of compromise that may not trigger automatic
alarms. Human analysts’ intuition and expertise are essential in uncovering
sophisticated attacks. Overall, humans can develop the hypothesis and AI can
help to execute it.

  * •

Behavioral Analysis and User Authentication: AI can develop behavioral
profiles for individuals and groups inside an organization. Any variations
from these characteristics can generate alerts to explore potential insider
threats. Human experts can evaluate suspicious behaviors to identify whether
they are insider threats or false positives. Thus this can assist human
experts in strengthening authentication systems of an organization.

Overall, Human-AI teaming in both IT and ICS/OT environment has the potential
to greatly improve attack identification, incident response, and overall
cybersecurity posture. AI can aid in automation, quick analysis, and
processing of massive volumes of data, while human knowledge adds context,
critical thinking, decision-making, and flexibility to a dynamic and evolving
threat environment. However, the synergistic abilities of humans and AI need
to be successfully exploited to handle domain-specific issues.

##  5 Challenges and Research Direction

While AI-based cybersecurity solutions have the potential to enhance security
as well as assist human experts in making decisions, several challenges need
to be addressed to fully realize these benefits. Hence, we outline the key
challenges and directions within the scope of our study:

  * •

Data Quality and Diversity: Getting tagged data with high quality and
diversity to cover a variety of attack scenarios is a challenge. Thus research
should focus on investigating techniques that can identify anomalies and
potential risks by using both labelled and unlabeled data as well as
unsupervised learning techniques, rather than relying solely on labeled data.
In addition, methods to augment and synthesize a variety of representative
datasets can help to achieve generalizability and robustness of AI models.

  * •

Adversarial Attacks: Adversarial attacks against AI models in cybersecurity
can lead to the evasion of detection systems by attackers. It is essential to
conduct research to develop AI models that can resist adversarial attacks and
maintain their effectiveness. Developing techniques such as adversarial
learning, input sanitization, defensive distillation, and ensemble modeling
could enhance model security.

  * •

Interpretable AI: To understand why certain decisions are made, it is crucial
for cybersecurity analysts to comprehend how AI models make decisions. Thus
research should focus on creating interpretable AI and machine learning models
enabling cybersecurity professionals to validate and trust the model’s
outputs. The interaction between AI systems and human cybersecurity experts
can be facilitated by developing innovative methods for explaining how AI-
based cyber models make decisions.

  * •

Innovative and Adaptive Modeling: Traditional approaches have difficulty
identifying zero-day vulnerabilities. A crucial research direction is the
creation of AI models that can identify and mitigate previously unknown
vulnerabilities. Examples of these models include anomaly detection, behavior
analysis, and vulnerability prediction. Research is required to enhance
models’ ability to generalize novel and previously unknown attack patterns in
different cybersecurity areas. Continuous learning is a key research direction
for creating AI models that can adapt to dynamic attack tactics and changing
threat environments. Effective threat detection can be improved exploring
several modalities and comprehend temporal patterns, such as network traffic,
system logs, and user behavior.

  * •

Generative AI Modeling: Generative AI has the potential to play a significant
role in cybersecurity solutions by enhancing various aspects of threat
analysis and response. For instance, generative AI can help in creating
adversarial scenarios to evaluate the reliability of cybersecurity systems,
assisting in the identification of vulnerabilities and the development of more
robust defense mechanisms against such attacks. More research is needed to
investigate techniques that enable AI models to generalize to unseen threats
and adapt to evolving attack techniques.

  * •

Privacy Concerns: Handling private and sensitive data could be a part of using
AI in cybersecurity. It is challenging to achieve a balance between the
advantages of AI and privacy issues while maintaining compliance with data
protection laws. To resolve privacy issues, federated learning enables models
to be trained across distributed devices without exchanging raw data.
Investigating how distributed learning can be used effectively in
cybersecurity to enable collaborative model training across various
organizations, could be a potential area of research.

  * •

Regulatory and Ethical Frameworks: To ensure secure and responsible deployment
of AI-based model into cybersecurity, establishing guidelines, regulations,
and ethical considerations are important. Investigating how AI may offer
insights and recommendations while allowing people to make informed decisions
is necessary. Research is needed to effectively integrate AI models into the
decision-making procedures of cybersecurity analysts, which can enable
synergistic interaction between human expertise and AI capabilities.

Although research has been significantly progressing in the area of AI, the
challenges for effective security modeling still remain unaddressed. To
advance the field of AI-based cybersecurity and enhance the general security
posture of today’s digital systems and interconnected networks, these
identified issues and potential directions might help for next-generation
cybersecurity solutions.

##  6 Conclusion

This position paper emphasized the importance of a balanced strategy that
capitalizes on the strengths of both AI and human expertise, creating
collaboration and trust between these two entities in the context of
cybersecurity. As stated in this position paper, the synergy between human
expertise and AI capabilities holds enormous promise for addressing the ever-
changing landscape of cyber threats. We explored how AI technology might
improve human capabilities by automating regular operations, analyzing and
detecting anomalies at scale, as well as providing actionable insights for
speedy decision-making. Furthermore, in terms of context understanding,
intuition, accountability, and creativity in designing unique security
solutions, the human element remains crucial. Overall, we believe that human-
AI teaming can significantly improve the way we protect against cyber-attacks
by providing a more resilient, efficient, and adaptive cybersecurity
ecosystem.

## Acknowledgement

The work has been supported by the Cyber Security Research Centre Limited
whose activities are partially funded by the Australian Government’s
Cooperative Research Centres Program.

## References

  * [1] Iqbal H Sarker.  Machine learning for intelligent data analysis and automation in cybersecurity: current and future prospects.  Annals of Data Science, pages 1–26, 2022.
  * [2] Zhimin Zhang, Huansheng Ning, Feifei Shi, Fadi Farha, Yang Xu, Jiabo Xu, Fan Zhang, and Kim-Kwang Raymond Choo.  Artificial intelligence in cyber security: research advances, challenges, and opportunities.  Artificial Intelligence Review, pages 1–25, 2022.
  * [3] Iqbal H Sarker.  Multi-aspects ai-based modeling and adversarial learning for cybersecurity intelligence and robustness: A comprehensive overview.  Security and Privacy, page e295, 2022.
  * [4] Iqbal H Sarker, Helge Janicke, Leandros Maglaras, and Seyit Camtepe.  Data-driven intelligence can revolutionize today’s cybersecurity world: A position paper.  arXiv preprint arXiv:2308.05126, 2023.
  * [5] Chika Yinka-Banjo and Ogban-Asuquo Ugot.  A review of generative adversarial networks and its application in cybersecurity.  Artificial Intelligence Review, 53:1721–1736, 2020.
  * [6] Ilhan Firat Kilincer, Fatih Ertam, and Abdulkadir Sengur.  Machine learning methods for cyber security intrusion detection: Datasets and comparative study.  Computer Networks, 188:107840, 2021.
  * [7] Kevin Dick, Luke Russell, Yasmina Souley Dosso, Felix Kwamena, and James R Green.  Deep learning for critical infrastructure resilience.  Journal of Infrastructure Systems, 25(2):05019003, 2019.
  * [8] Priyanka et al. Dixit.  Deep learning algorithms for cybersecurity applications: A technological and status review.  Computer Science Review, 39:100317, 2021.
  * [9] Margus Välja, Fredrik Heiding, Ulrik Franke, and Robert Lagerström.  Automating threat modeling using an ontology framework: Validated with data from critical infrastructures.  Cybersecurity, 3:1–20, 2020.
  * [10] Nikolaos et al. Bakalos.  Protecting water infrastructure from cyber and physical threats: Using multimodal data fusion and adaptive deep learning to monitor critical systems.  IEEE Signal Processing Magazine, 36(2):36–48, 2019.