$\\{v_{t}(\cdot)\\}_{t\in[T]}$ that satisfies the following conditions:

$\displaystyle v_{T}(x)$ $\displaystyle=\zeta(x),\quad\text{ for all
}x\in\mathcal{X},$ (27) $\displaystyle v_{t}(x)$
$\displaystyle\in\operatorname{SE}\left[\begin{array}[]{cc}\zeta(x)&\phi(x)\\\
\psi(x)&\mathscr{T}v_{t+1}(x)]\end{array}\right],\quad\text{ for all
}x\in\mathcal{X},t\in[T-1],$

where $\operatorname{SE}$ stands for the set of Nash (saddle-point)
equilibrium values of the matrix game with two pure strategies. Then, the last
iterate value function is $\zeta(\cdot)$ by construction. The rest of the
business is to figure out the backward induction equation that involves the
$\operatorname{val}(\cdot)$ operator, which still relies on the calculation of
$\mathscr{T}$ leveraging Monte-Carlo sampling type of methods. Following Lemma
1 the monotonicity still holds, $\\{v_{t}(\cdot)\\}_{t\in[T]}$ is decreasing,
which can be interpreted as that the decision made at the outset is most
valuable, as time passes, the opportunity fades. For any $t\in[T]$, we define
the two stopping times,

$\displaystyle\tau^{*}_{t}$ $\displaystyle=\inf\\{t\leq k\leq
T|\\{v_{k}(X_{k})=\zeta(X_{k})\\}\bigcup\\{v_{k}(X_{k})=\phi(X_{k})\\}\\},$
$\displaystyle\sigma^{*}_{t}$ $\displaystyle=\inf\\{t\leq k\leq
T|\\{v_{k}(X_{k})=\zeta(X_{k})\\}\bigcup\\{v_{k}(X_{k})=\psi(X_{k})\\}\\}.$

By Theorem 6.2, $(\tau^{*}_{0},\tau^{*}_{0})$ is the equilibrium strategy
pair, the definition of which reflects the consistency of value function
computation, that is, the players’ current value estimates either reach the
early termination threshold or confrontational threshold.

###### Theorem 6.2

Under ADC, the following statements hold for arbitrary initial state
$x\in\mathcal{X}$:

  * •

for every $t\in[T]$, and all
$\tau\in\mathcal{T}_{t},\sigma\in\mathcal{T}_{t}$,

$\displaystyle\mathbb{E}[H(\tau,\sigma^{*}_{t})|\mathcal{F}_{t}]\leq\mathbb{E}[H(\tau^{*}_{t},\sigma^{*}_{t})|\mathcal{F}_{t}]\leq\mathbb{E}[H(\tau^{*}_{t},\sigma)|\mathcal{F}_{t}].$

  * •

the game admits a DDE strategy $(\tau^{*}_{0},\sigma^{*}_{0})$, at which the
value function satisfies

$\displaystyle\mathcal{V}^{\tau^{*}_{0},\sigma^{*}_{0}}(x)$
$\displaystyle={\operatorname{ess}\operatorname{sup}}_{\tau\in\mathcal{T}}{\operatorname{ess}\operatorname{inf}}_{\sigma\in\mathcal{T}}\mathcal{V}^{\tau,\sigma}(x)$
$\displaystyle={\operatorname{ess}\operatorname{inf}}_{\sigma\in\mathcal{T}}{\operatorname{ess}\operatorname{sup}}_{\tau\in\mathcal{T}}\mathcal{V}^{\tau,\sigma}(x).$

We omit the proof here as Theorem 6.2 can be seen as an extension of Theorem
6.1, to which the reasoning is similar. One can simply construct the sequence
of value functions with a constant translation, and the results still hold.

#### Case II: Defensive Dominance

Under the Defensive Dominance Condition (DDC), the payoff functions satisfy
the ordered condition $\psi\leq\zeta\leq\phi$ for all system states
$x\in\mathcal{X}$. In this case, at any state $x\in\mathcal{X}$, the defender
can bide his time during the interactions of cyber kill/defense chain, as the
systematic loss after the execution of Command & Control is mitigable. Such a
condition happens when the defender possesses a superior and robust position.
This is also called second-mover advantage, that is, the defender has the
incentive to wait for the opponent to end the game.

DDC corresponds to the ordered payoff condition for standard Dynkin’s game,
where the existence and uniqueness of a saddle point value process have been
proved. The constructive sequence of (locally integrable) random variables
$\\{V_{t}\\}_{t=0}^{T}$, in this case, is now more straightforward (as
discussed in Kingman (1976)), defined by

$\displaystyle V_{T}=\zeta(X_{T}),$ (28) $\displaystyle
V_{t}=\min\\{\psi(X_{t}),\max\\{\phi(X_{t}),\mathbb{E}[V_{t+1}|\mathcal{F}_{t}]\\}\\},\quad\quad\text{
for }t=0,\ldots,T-1,$

with the stopping time strategies defined as

$\displaystyle\bar{\tau}_{t}$ $\displaystyle=\inf\\{t\leq k\leq
T|V_{k}=\phi(X_{k})\\},$ $\displaystyle\bar{\sigma}_{t}$
$\displaystyle=\inf\\{t\leq k\leq T|V_{k}=\psi(X_{k})\\}.$

###### Theorem 6.3

Under DDC, the following statements hold:

  * i)

for each $t\in[T]$, and for all $\tau\in\mathcal{T}_{t}$,
$\sigma\in\mathcal{T}_{t}$,

$\displaystyle
V_{t}=\mathbb{E}[V_{\tau^{*}_{t}\wedge\sigma^{*}_{t}}|\mathcal{F}_{t}]=\mathbb{E}[H(\tau^{*}_{t},\sigma^{*}_{t})|\mathcal{F}_{t}],\quad\quad\text{
and, }$ $\displaystyle\mathbb{E}[H(\tau,\sigma^{*}_{t})|\mathcal{F}_{t}]\leq
V_{t}\leq\mathbb{E}[H(\tau^{*}_{t},\sigma)|\mathcal{F}_{t}].\quad\quad$

  * ii)

at every time $t\in[T]$, a pair $(\tau^{*}_{t},\sigma^{*}_{t})$ is an
equilibrium point for the subgame starting at time $t$, and the DDE value
corresponding to $(\tau^{*}_{0},\sigma^{*}_{0})$ is

$\mathbb{E}[V_{0}]=\mathbb{E}[V_{\tau^{*}_{0}\wedge\sigma^{*}_{0}}]=\mathbb{E}[H(\tau^{*}_{0},\sigma^{*}_{0})].$

###### Proof

Similar to previous results, we shall give the proof for the “$\geq$” side.
First, we examine the trivial case where $t=T$. Obviously, there’s no option
but stop for both parties, so
$G_{t}=\zeta(X_{t})=\mathbb{E}[G_{\tau\wedge\sigma^{*}_{t}}|\mathcal{F}_{t}]=\mathbb{E}[H(\tau,\sigma^{*}_{t})|\mathcal{F}_{t}]$
for all $\tau\in\mathcal{T}_{T}=\\{T\\}$.

Fix a $t<T$. Choose some $k$ such that $t\leq
k\leq\tau^{*}_{t}\wedge\sigma^{*}_{t}$, we have
$V_{k}=\mathbb{E}[V_{k+1}|\mathcal{F}_{k}]$ by definition. Thus,
$\\{V_{k\wedge\tau^{*}_{t}\wedge\sigma^{*}_{t}}\\}_{k=t}^{T}$ is a Martingale.
Applying Doob’s optional sampling theorem, one has

$V_{t}=\mathbb{E}[V_{\tau\wedge\tau^{*}_{t}\wedge\sigma^{*}_{t}}|\mathcal{F}_{t}],\quad\text{
for all }\tau\in\mathcal{T}_{t}.$

Let $\tau=\tau^{*}_{t}$, we arrive at

$\displaystyle V_{t}$
$\displaystyle=\mathbb{E}[V_{\tau^{*}_{t}\wedge\sigma^{*}_{t}}|\mathcal{F}_{t}]$
$\displaystyle=\mathbb{E}[V_{\tau^{*}_{t}}\mathds{1}_{\\{\tau^{*}_{t}\leq\sigma^{*}_{t}\\}}+V_{\sigma^{*}_{t}}\mathds{1}_{\\{\tau^{*}_{t}>\sigma^{*}_{t}\\}}|\mathcal{F}_{t}]$
$\displaystyle=\mathbb{E}[\phi(X_{\tau^{*}_{t}})\mathds{1}_{\\{\tau^{*}_{t}<\sigma^{*}_{t}\\}}+\psi(X_{\sigma^{*}_{t}})\mathds{1}_{\\{\tau^{*}_{t}>\sigma^{*}_{t}\\}}+\zeta(X_{\tau^{*}_{t}})\mathds{1}_{\\{\tau^{*}_{t}=\sigma^{*}_{t}\\}}|\mathcal{F}_{t}]$
$\displaystyle=\mathbb{E}[H(\tau^{*}_{t},\sigma^{*}_{t})|\mathcal{F}_{t}].$

It is also obvious that when $t\leq k<\sigma^{*}_{t}$, then
$V_{k}<\psi(X_{k})$, therefore $V_{k}=\mathbb{E}[V_{k+1}|\mathcal{F}_{k}]$.
This implies that $\\{V_{k\wedge\sigma^{*}_{t}}\\}_{k=t}^{T}$ is a
supermartingale. Hence,
$V_{t}\geq\mathbb{E}[V_{\tau\wedge\sigma^{*}_{t}}|\mathcal{F}_{t}]$ for all
$\tau\in\mathcal{T}_{t}$,

$\displaystyle\mathbb{E}[V_{\tau\wedge\sigma^{*}_{t}}|\mathcal{F}_{t}]$
$\displaystyle\geq\mathbb{E}[\phi(X_{\tau})\mathds{1}_{\\{\tau<\sigma^{*}_{t}\\}}+\psi(X_{\sigma^{*}_{t}})\mathds{1}_{\\{\tau^{*}_{t}>\sigma^{*}_{t}\\}}+\zeta(X_{\tau})\mathds{1}_{\\{\tau=\sigma^{*}_{t}\\}}|\mathcal{F}_{t}]$
$\displaystyle=\mathbb{E}[H(\tau,\sigma^{*}_{t})|\mathcal{F}_{t}],$

since $V_{k}>\phi(X_{k})$ and $\zeta(X_{k})\leq\psi(X_{k})$ for all $0\leq
k\leq T$. Claim ii) follows immediately.

Again we generalize the result to $\mathcal{E}(\mathcal{X})$, we wish to find
a sequence of $\mathcal{B}(\mathbb{R})/\mathcal{G}$-measurable functions
$\\{v_{t}(\cdot)\\}_{t\in[T]}$ that satisfies the following conditions (or
being shifted by a constant):

$\displaystyle v_{T}(x)=\zeta(x),\quad\quad\text{ for all }x\in\mathcal{X},$
$\displaystyle
v_{t}(x)=\min\\{\psi(x),\max\\{\phi(x),\mathscr{T}v_{t+1}(x)\\}\\},\quad\quad\text{
for all }x\in\mathcal{X},t=[T-1],$

and the DDE pair $(\tau^{*},\sigma^{*})$ can be defined as:

$\displaystyle\tau^{*}$
$\displaystyle=\inf\\{k\in[T]|v_{k}(X_{k})=\phi(X_{k})\\},$
$\displaystyle\sigma^{*}$
$\displaystyle=\inf\\{k\in[T]|v_{k}(X_{k})=\psi(X_{k})\\}.$

###### Theorem 6.4

Under ADC, the game admits a DDE strategy pair $(\tau^{*},\sigma^{*})$, such
that

$\displaystyle\mathcal{V}^{\tau^{*},\sigma^{*}}(x)$
$\displaystyle={\operatorname{ess}\operatorname{sup}}_{\tau\in\mathcal{T}}{\operatorname{ess}\operatorname{inf}}_{\sigma\in\mathcal{T}}\mathcal{V}^{\tau,\sigma}(x)$
$\displaystyle={\operatorname{ess}\operatorname{inf}}_{\sigma\in\mathcal{T}}{\operatorname{ess}\operatorname{sup}}_{\tau\in\mathcal{T}}\mathcal{V}^{\tau,\sigma}(x),$

for all $x\in\mathcal{X}$.

Under DDC, the optimal strategies for the players are waiting for the
equilibrium process to hit the lower/upper bound of the payoff values.

#### Decision Dominance with Information Asymmetry

In the MDW scenarios, it is crucial to recognize that both defenders and
attackers operate within an environment of information asymmetry Huang and Zhu
(2019); Pawlick et al. (2019). This is particularly evident when considering
STIX logs, as the information accessible to attackers differs from what
defenders can observe. While defenders have the advantage of comprehensive
logs that capture security events and indicators of compromise, attackers
possess their own set of advantages stemming from their ability to exploit the
gaps in the defender’s knowledge. Attackers can leverage their insider
information, external reconnaissance, and targeted intelligence gathering to
gain insights into the defender’s security measures, potential
vulnerabilities, and defensive capabilities. In the meantime, the defender may
have deceptive defense mechanisms that hide their tactics, techniques, and
procedures (TTPs), to counteract the malicious exploitation.

To formalize the notion, we redefine $(X_{t})_{t=0}^{T}$ as the true system
state (which cannot be completely captured by the STIX logs), and let
$(O^{i}_{t})_{0\leq t\leq T}(i=1,2)$ be the observation process for the
defender ($i=1$) and the attacker ($i=2$), which jointly live in the space
$(\mathcal{O}^{1}\times\mathcal{O}^{2},\mathcal{H}^{1}\otimes\mathcal{H}^{2})$,
adapted to the filtrations $\mathbb{H}^{1}=(\mathcal{H}^{1}_{t})_{0\leq t\leq
T}$ and $\mathbb{H}^{2}=(\mathcal{H}^{2}_{t})_{0\leq t\leq T}$. This
information asymmetry enables the players to make informed decisions regarding
their strategies, tactics, and the selection of attack vectors/defensive
mechanisms. Therefore, defenders must not only rely on STIX logs and robust
defense mechanisms but also proactively bridge the information gap by
enhancing their threat intelligence capabilities, anticipating adversary
behaviors, and continuously evolving their defense strategies to counter the
advantages of information asymmetry in the cyber landscape.

To formally define the DDG under asymmetric information structure, we denote
by $\mathcal{T}(\mathbb{H}^{i})$ the set of $\mathbb{H}^{i}$-stopping times,
$\mathcal{T}(\mathbb{H}^{i})=\\{0\leq\tau\leq T:\\{\tau(\omega)\leq
k\\}\in\mathcal{H}^{i}_{k}\ \forall k\in[T],\forall\omega\in\Omega\\}$. The
decision payoffs at each stage $t\in[T]$, in this case, may depend on both
$O^{i}_{t}$ and $X_{t}$. Following the standard formalism of the Partially
Observable Markov Decision Process (POMDP), we assume that the payoff
functions still only depend on the true system state, which is a hidden latent
variable for both players. Instead, there exists an emission kernel
$\mathbb{O}:\mathcal{X}\to\Delta(\mathcal{O}^{1}\times\mathcal{O}^{2})$ that
measures the joint probability of observations made by the defender and the
attacker. An illustration is shown in Figure 7

Figure 7: An illustration of asymmetric information dynamic games defined in
Definition 11. The two players have distinct partial observations for the
system state $X_{t}$, denoted by $(O^{1}_{t},O^{2}_{t})$. In DDG, the defender
has to infer the true state to determine the stopping time strategy based on
the payoff structure, which relies on credible modeling, requiring expertise
in the fundamental understanding of the cyber threats

Factorization Lemma says in order to infer the true states from the partial
observations, say, if $O^{1}_{t}$ is
$\mathcal{F}_{t}$/$\mathcal{H}^{1}_{t}$-measurable, there needs to be a
deterministic $\mathcal{F}_{t}$/$\mathcal{H}^{1}_{t}$-measurable map
$f:\mathcal{X}\to\mathcal{O}^{1}$ such that $O^{1}_{t}=f(X_{t})$, whose
existence and accessibility are not always guaranteed in the cyber domain.
Therefore, it is reasonable to assume that the players have their stopping
time strategies restricted to $\mathcal{T}(\mathbb{H}^{i})$. Definition 11
summarizes the game under asymmetrical information structure.

######  Definition 11 (Decision Dominance Game with Information Asymmetry)

A tuple
$(\mathcal{X},\mathcal{O}^{1}\times\mathcal{O}^{2},\mathcal{P},\mathbb{O},\phi,\zeta,\psi,\mathcal{T}(\mathbb{H}^{1}),\mathcal{T}(\mathbb{H}^{2}))$
encapsulates a Decision Dominance Game with Information Asymmetry (DDGIA) if
it satisfies that

  * •

There exists a hidden Markov process $(X_{t})_{0\leq t\leq T}$ that lives in
$(\mathcal{X},\mathcal{G})$ with transition kernel $\mathcal{P}$, which yields
observations $(O^{1}_{t},O^{2}_{t})$ through emission kernel $\mathbb{O}$;

  * •

$\phi,\zeta,$ and $\psi$ are payoff functions mapping from $X_{t}$ to
$\mathbb{R}$, $\phi,\zeta,\psi\in\mathcal{E}(\mathcal{X})$, which is the set
of all bounded $\mathcal{B}(\mathbb{R})/\mathcal{G}$-measurable functions on
$(\mathcal{X},\mathcal{G})$. Further,
$\min(\phi,\psi)\leq\zeta\leq\max(\phi,\psi)$ on $\mathcal{X}$;

  * •

At each stage $t$, player $i$ ($i=1,2$) picks a stopping strategy from space
$\mathcal{T}_{t}(\mathbb{H}^{i}):=\\{t\leq\tau\leq T:\\{\tau(\omega)\leq
k\\}\in\mathcal{H}^{i}_{k}\ \forall k\in[T],\forall\omega\in\Omega\\}$ to
decide whether to stop or continue the kill/defense chain.

  * •

At each stage the utility function of the defender is

$H(\tau_{t},\sigma_{t})=\phi(X_{\tau_{t}})\mathds{1}_{\\{\tau_{t}<\sigma_{t}\\}}+\zeta(X_{\tau_{t}})\mathds{1}_{\\{\tau_{t}=\sigma_{t}\\}}+\psi(X_{\sigma_{t}})\mathds{1}_{\\{\tau_{t}>\sigma_{t}\\}},$

while the attacker attains $-H(\tau_{t},\sigma_{t})$.

The goal of the defender is to choose $\tau$ to maximize her utility under all
possible choices of the attacker, which leads to the lower value function of
DDGIA,

$\underline{V}(x)=\operatorname{ess}\operatorname{sup}_{\tau\in\mathcal{T}(\mathbb{H}^{1})}\operatorname{ess}\operatorname{inf}_{\sigma\in\mathcal{T}(\mathbb{H}^{2})}V^{\tau,\sigma}(x).$
(29)

Similarly, the goal of the attacker is to choose $\sigma$ to minimize the
defender’s utility under all possible choices of the defender, which leads to
the upper-value function,

$\overline{V}(x)=\operatorname{ess}\operatorname{inf}_{\sigma\in\mathcal{T}(\mathbb{H}^{2})}\operatorname{ess}\operatorname{sup}_{\tau\in\mathcal{T}(\mathbb{H}^{1})}V^{\tau,\sigma}(x).$
(30)

######  Definition 12 (DDE with Information Asymmetry)

A pair of stopping time strategies
$(\tau^{*},\sigma^{*})\in\mathcal{T}(\mathbb{H}^{1})\times\mathcal{T}(\mathbb{H}^{2})$
is a Decision-Dominance Equilibrium (DDE) if for all initial state
$x\in\mathcal{X}$, it satisfies the minimax condition:

$\displaystyle V^{\tau^{*},\sigma^{*}}(x)$
$\displaystyle=\operatorname{ess}\operatorname{sup}_{\sigma\in\mathcal{T}(\mathbb{H}^{2})}\operatorname{ess}\operatorname{inf}_{\tau\in\mathcal{T}(\mathbb{H}^{1})}V^{\tau,\sigma}(x)$
(31)
$\displaystyle=\operatorname{ess}\operatorname{inf}_{\tau\in\mathcal{T}(\mathbb{H}^{2})}\operatorname{ess}\operatorname{sup}_{\sigma\in\mathcal{T}(\mathbb{H}^{2})}V^{\tau,\sigma}(x).$

We say that a DDGIA has a value if $\underline{V}(x)=\overline{V}(x)$. Note
that the existence and uniqueness of the value is a non-trivial question in
general, as we shall find the reasoning presented in the previous section not
applicable due to the introduction of two private filtrations for both
parties. In principle, the value exists if $\mathbb{H}^{i}$ reveal the same
information from $\mathbb{F}$, in which case the conditional expectation
$\mathbb{E}(\cdot|\mathbb{H}^{i}_{t})$ can be seen equivalent with
$\mathbb{E}(\cdot|\mathcal{F}_{t})$, thus the players will make their
decisions using the same threshold policies. This property, however, requires
some special structures of the observation kernel $\mathbb{O}$, which might
not hold in realistic scenarios.

###  6.3 Decision Dominance Zero-Trust Defense (DD-ZTD): A Case Study

In this case study, we consider an $T$-episodic DDG with symmetric information
over the same 5G network $G=\langle V,E\rangle$ as discussed in section 5.2,
where each episode $t$ contains $H$ ZTD steps against lateral movement. The
ZTD state action variables within one episode $t$ is
$\mathbf{sa}_{t}=(s^{1}_{t},a^{1}_{t},s^{2}_{t},a^{2}_{t}\ldots,a^{H-1}_{t},s^{H}_{t})$,
where $s^{h}_{t}=(G^{h}_{t},L^{h}_{t}),h=1,\ldots,H$ are the authentication
graphs and the visiting indicator functions at episode $t$, and the joint
actions $a^{h}_{t}$ are automated by the threshold-policy trust engine, which
is either the Bayesian type or the Machine Learning type. Denote the STIX logs
within $t$ as $x_{t}\in\mathcal{X}$, which includes but is not limited to the
events of 5G network exposure, slicing control, session management; the threat
actor characterizations such as suspected user intentions and handling
guidance. The Markovian state at episode $t$ is a composition of both
historical ZTD state action variables and the STIX logs gathered before
episode $t$, i.e., $X_{t}:=(\mathbf{sa}_{1:t-1},x_{t-1})$.

During the cyber kill/defense chain interaction, at the beginning of each
episode, the defender can choose to completely cut off the chain before
episode $t$ starts by isolating the networks, restarting the services,
resetting all the credentials, patching and hardening the security
configurations, and then restoring and resuming the operations. The cost of
the defender’s cutting-off strategy is $C(\cdot):\mathcal{X}\to\mathbb{R}$,
which only depends on the cyber threat information. Similarly, the attacker
can choose to take action early by exploiting Zero-Day vulnerabilities,
evading intrusion detection systems, and implementing stealthy command and
control at an early stage of the cyber kill chain. Again we let the
exploitation loss be $\ell(\cdot):\mathcal{X}\to\mathbb{R}$, which completely
depends on the cyber threat characterization of episode $t$. Now we are ready
to define the three payoff functions in our DDG framework.

The early termination payoff, confrontation payoff, and late termination
payoff functions can be defined as

$\displaystyle\phi(X_{t})$
$\displaystyle=-\mathbb{E}[\sum_{k=1}^{t-1}\sum_{h=1}^{H}u_{D}(s^{h}_{k},a^{h}_{k})]-C(x_{t-1}),$
(32) $\displaystyle\zeta(X_{t})$
$\displaystyle=-\mathbb{E}[\sum_{k=1}^{t-1}\sum_{h=1}^{H}u_{D}(s^{h}_{k},a^{h}_{k})]-C(x_{t-1})-\ell(x_{t-1}),$
$\displaystyle\psi(X_{t})$
$\displaystyle=-\mathbb{E}[\sum_{k=1}^{t}\sum_{h=1}^{H}u_{D}(s^{h}_{k},a^{h}_{k})]-\ell(x_{t-1}),$

where the expectation $\mathbb{E}[\sum_{h}^{H}u_{D}(s^{h}_{t},a^{h}_{t})]$ is
taken conditioned on $\mathbf{sa}_{1:t-1}$. The interpretation is that when
the defender chooses to shut down and restore the services, the ZTD stops for
that episode, while if the attacker chooses to exploit early, the ZTD
mechanism is still active.

One can easily verify that when both $\ell$ and $C$ are positive and the
expected ZTD cost within every episode $t$ satisfies
$\mathbb{E}[\sum_{h}^{H}u_{D}(s^{h}_{t},a^{h}_{t})]>C(x_{t-1})$ the DDG
satisfies DDC.

##  7 Conclusion

This chapter develops a game-theoretic framework for the decision-dominant
zero-trust defense of 5G networks in the face of advanced persistent threats
that utilize a cyber kill chain to disrupt the network operation. The advanced
features of 5G networks, despite their contributions to multi-domain
integration, bring a larger attack surface and render the network system
vulnerable in the presence of advanced persistent threats (APT) and other
malicious attacks. The combination of vulnerabilities in APT, supply chains of
5G equipment, and network slicing, along with others, can be exploited by an
APT attacker to create a cyber kill chain consisting of reconnaissance,
planning, execution, and exploration.

To outmaneuver the malicious attacker and thwart the kill chain, this chapter
proposes a decision-dominant zero-trust defense (DD-ZTD) framework, a
proactive defense mechanism enabling the defender to make timely and effective
decisions with incomplete information regarding the situation and disrupt the
kill chain before its completion. Two pillars of DD-ZTD are game-theoretic
zero-trust defense built upon asymmetric information Markov games (AIMG) and
decision-dominance defense characterized by Dykin’s stopping-time games. With
the AIMG’s expressive power on information structures in cyber defense, ZTD
employs a variety of trust engines to evaluate entities’ trustworthiness with
limited partial observations, which is then fed into the access policy powered
by equilibrium thinking that anticipates the attacker’s response. We further
present an end-to-end ZTD facilitated by recent machine learning advancements
with data-driven trust evaluation and explainable and generalizable policy
learning.

While the proposed ZTD offers a set of fruitful tools to quantitatively
analyze trustworthiness under information asymmetry, the networked entities
still face multi-stage persistent cyber threats that call for rapid response
from the defender. To outpace the attacker’s kill chain, decision-dominance
defense (D3), mathematically treating interactions of cyber defense/kill chain
as a stopping-time game, aims to take the decisive move to cut off the kill
chain before the attack materializes. The essence of D3 is the timing of the
cutting-off, which is determined by the equilibrium of the game with
anticipation of the attacker’s strategic move. The resulting DD-ZTD, as an
organic integration of the two game-theoretic defense mechanisms, displays
great potential in combating sophisticated adversaries, which we articulate
using a case study in 5G network defense.

## References

  * Headquarters, Department of the Army (2022) Headquarters, Department of the Army (2022) FM 3-0, Operations. https://usacac.army.mil/node/3048, accessed 2023-07-05
  * Department of Defense (2018) Department of Defense (2018) Summary of the 2018 national defense strategy. https://www.spoc.spaceforce.mil/About-Us/Fact-Sheets/Display/Article/2381348/advanced-extremely-high-frequency-system-aehf, accessed 2023-07-05
  * Space Operations Command (2021) (SPOC) Space Operations Command (SPOC) (2021) Advanced extremely high frequency system (aehf). https://www.spoc.spaceforce.mil/About-Us/Fact-Sheets/Display/Article/2381348/advanced-extremely-high-frequency-system-aehf, accessed 2023-07-05
  * Lockheed Martin (2023) Lockheed Martin (2023) Indago UAV. https://www.lockheedmartin.com/en-us/products/indago-vtol-uav.html, accessed 2023-07-05
  * Huang and Zhu (2022) Huang L, Zhu Q (2022) Radams: Resilient and adaptive alert and attention management strategy against informational denial-of-service (idos) attacks. Computers & Security 121:102844
  * Wijethilaka and Liyanage (2021) Wijethilaka S, Liyanage M (2021) Survey on network slicing for internet of things realization in 5g networks. IEEE Communications Surveys & Tutorials 23(2):957–994, DOI 10.1109/COMST.2021.3067807
  * Xiao et al. (2019) Xiao Y, Jia Y, Liu C, Cheng X, Yu J, Lv W (2019) Edge computing security: State of the art and challenges. Proceedings of the IEEE 107(8):1608–1631, DOI 10.1109/JPROC.2019.2918437
  * Zhu and Rass (2018) Zhu Q, Rass S (2018) On multi-phase and multi-stage game-theoretic modeling of advanced persistent threats. IEEE Access 6:13958–13971
  * Huang and Zhu (2020) Huang L, Zhu Q (2020) A dynamic games approach to proactive defense strategies against advanced persistent threats in cyber-physical systems. Computers & Security 89:101660
  * Rass and Zhu (2016) Rass S, Zhu Q (2016) Gadapt: a sequential game-theoretic framework for designing defense-in-depth strategies against advanced persistent threats. In: International conference on decision and game theory for security, Springer, pp 314–326
  * Huang and Zhu (2019) Huang L, Zhu Q (2019) Dynamic bayesian games for adversarial and defensive cyber deception. Autonomous Cyber Deception: Reasoning, Adaptive Planning, and Evaluation of HoneyThings pp 75–97
  * Rose et al. (2020) Rose S, Borchert O, Mitchell S, Connelly S (2020) Zero trust architecture. Tech. rep., National Institute of Standards and Technology
  * Osborn (2018) Osborn K (2018) “first look, first shot, first kill”: How the f-22 raptor could fly until 2060. URL https://nationalinterest.org/blog/buzz/first-look-first-shot-first-kill-how-f-22-raptor-could-fly-until-2060-35937
  * Gady and Stronell (2020) Gady FS, Stronell A (2020) Cyber capabilities and multi-domain operations in future high-intensity warfare in 2030. Cyber Threats and NATO 2030: Horizon Scanning and Analysis pp 151–176
  * Mallik et al. (2000) Mallik RK, Scholtz RA, Papavassilopoulos GP (2000) Analysis of an on-off jamming situation as a dynamic game. IEEE Transactions on Communications 48(8):1360–1373
  * Mukherjee and Swindlehurst (2012) Mukherjee A, Swindlehurst AL (2012) Jamming games in the mimo wiretap channel with an active eavesdropper. IEEE Transactions on Signal Processing 61(1):82–91
  * Sayin et al. (2018) Sayin MO, Hosseini H, Poovendran R, Başar T (2018) A game theoretical framework for inter-process adversarial intervention detection. In: International Conference on Decision and Game Theory for Security, Springer, pp 486–507
  * Chen et al. (2019) Chen J, Touati C, Zhu Q (2019) Optimal secure two-layer iot network design. IEEE Transactions on Control of Network Systems pp 1–1, DOI 10.1109/TCNS.2019.2906893
  * Pawlick et al. (2015) Pawlick J, Farhang S, Zhu Q (2015) Flip the cloud: Cyber-physical signaling games in the presence of advanced persistent threats. In: International Conference on Decision and Game Theory for Security, Springer, pp 289–308
  * Pawlick and Zhu (2017) Pawlick J, Zhu Q (2017) Strategic trust in cloud-enabled cyber-physical systems with an application to glucose control. IEEE Transactions on Information Forensics and Security 12(12):2906–2919
  * Huang et al. (2017) Huang L, Chen J, Zhu Q (2017) A large-scale markov game approach to dynamic protection of interdependent infrastructure networks. In: International Conference on Decision and Game Theory for Security, Springer, pp 357–376
  * Chen and Zhu (2022) Chen J, Zhu Q (2022) A cross-layer design approach to strategic cyber defense and robust switching control of cyber-physical wind energy systems. IEEE Transactions on Automation Science and Engineering 20(1):624–635
  * Chen and Zhu (2019) Chen J, Zhu Q (2019) A game-and decision-theoretic approach to resilient interdependent network analysis and design. Springer
  * Chen and Zhu (2016) Chen J, Zhu Q (2016) A game-theoretic framework for resilient and distributed generation control of renewable energies in microgrids. IEEE Transactions on Smart Grid 8(1):285–295
  * Chen and Zhu (2019) Chen J, Zhu Q (2019) A games-in-games approach to mosaic command and control design of dynamic network-of-networks for secure and resilient multi-domain operations. In: Chen G, Pham KD (eds) Sensors and Systems for Space Applications XII, International Society for Optics and Photonics, SPIE, vol 11017, pp 189 – 195, DOI 10.1117/12.2526677, URL https://doi.org/10.1117/12.2526677
  * Chen and Zhu (2020) Chen J, Zhu Q (2020) Control of multilayer mobile autonomous systems in adversarial environments: A games-in-games approach. IEEE Transactions on Control of Network Systems 7(3):1056–1068, DOI 10.1109/TCNS.2019.2962316
  * Chen and Zhu (2016) Chen J, Zhu Q (2016) Resilient and decentralized control of multi-level cooperative mobile networks to maintain connectivity under adversarial environment. In: IEEE Conference on Decision and Control (CDC), pp 5183–5188
  * Zhu et al. (2021) Zhu Q, Rass S, Dieber B, Vilches VM, et al. (2021) Cybersecurity in robotics: Challenges, quantitative modeling, and practice. Foundations and Trends® in Robotics 9(1):1–129
  * Kieras et al. (2020) Kieras T, Farooq MJ, Zhu Q (2020) Riots: Risk analysis of IoT supply chain threats. In: 2020 IEEE 6th World Forum on Internet of Things (WF-IoT), IEEE, pp 1–6
  * Ge and Zhu (2022) Ge Y, Zhu Q (2022) Accountability and insurance in IoT supply chain. arXiv preprint arXiv:220111855 URL https://doi.org/10.48550/arXiv.2201.11855
  * Kieras et al. (2022) Kieras T, Farooq J, Zhu Q (2022) IoT supply chain security risk analysis and mitigation: Modeling, computations, and software tools. Springer
  * Pan and Zhu (2022) Pan Y, Zhu Q (2022) On poisoned wardrop equilibrium in congestion games. In: International Conference on Decision and Game Theory for Security, Springer, pp 191–211
  * Pan et al. (2023a) Pan Y, Li T, Zhu Q (2023a) On the resilience of traffic networks under non-equilibrium learning. In: 2023 American Control Conference (ACC), IEEE, pp 3484–3489
  * Pan et al. (2023b) Pan Y, Li T, Zhu Q (2023b) Is stochastic mirror descent vulnerable to adversarial delay attacks? a traffic assignment resilience study. arXiv preprint arXiv:230401161 URL https://doi.org/10.48550/arXiv.2304.01161
  * Zheng and Castañón (2012) Zheng J, Castañón DA (2012) Dynamic network interdiction games with imperfect information and deception. In: 2012 IEEE 51st IEEE Conference on Decision and Control (CDC), IEEE, pp 7758–7763
  * Zhu et al. (2012) Zhu Q, Clark A, Poovendran R, Başar T (2012) Deceptive routing games. In: 2012 IEEE 51st IEEE Conference on Decision and Control (CDC), IEEE, pp 2704–2711
  * Zhuang et al. (2010) Zhuang J, Bier VM, Alagoz O (2010) Modeling secrecy and deception in a multiple-period attacker–defender signaling game. European Journal of Operational Research 203(2):409–418
  * Pawlick et al. (2019) Pawlick J, Colbert E, Zhu Q (2019) A game-theoretic taxonomy and survey of defensive deception for cybersecurity and privacy. ACM Computing Surveys (CSUR) 52(4):82
  * Zhu and Başar (2013) Zhu Q, Başar T (2013) Game-theoretic approach to feedback-driven multi-stage moving target defense. In: Decision and Game Theory for Security, Springer, pp 246–263
  * Jajodia et al. (2011) Jajodia S, Ghosh AK, Swarup V, Wang C, Wang XS (2011) Moving target defense: creating asymmetric uncertainty for cyber threats, vol 54. Springer Science & Business Media
  * Huang and Zhu (2021) Huang L, Zhu Q (2021) Combating informational denial-of-service (idos) attacks: Modeling and mitigation of attentional human vulnerability. In: Decision and Game Theory for Security: 12th International Conference, GameSec 2021, Virtual Event, October 25–27, 2021, Proceedings, Springer, pp 314–333
  * Huang and Zhu (2023) Huang L, Zhu Q (2023) Cognitive security: A system-scientific approach. Springer Nature
  * Liao et al. (2013) Liao HJ, Richard Lin CH, Lin YC, Tung KY (2013) Intrusion detection system: A comprehensive review. Journal of Network and Computer Applications 36(1):16–24, DOI https://doi.org/10.1016/j.jnca.2012.09.004, URL https://www.sciencedirect.com/science/article/pii/S1084804512001944
  * Bhatt et al. (2014) Bhatt S, Manadhata PK, Zomlot L (2014) The operational role of security information and event management systems. IEEE Security & Privacy 12(5):35–41, DOI 10.1109/msp.2014.103
  * Li et al. (2022) Li T, Zhao Y, Zhu Q (2022) The role of information structures in game-theoretic multi-agent learning. Annual Reviews in Control 53:296–314, DOI 10.1016/j.arcontrol.2022.03.003
  * Li and Zhu (2022) Li T, Zhu Q (2022) Commitment with signaling under double-sided information asymmetry. arXiv preprint arXiv:221211446 URL https://doi.org/10.48550/arXiv.2212.11446
  * Ge et al. (2023) Ge Y, Li T, Zhu Q (2023) Scenario-agnostic zero-trust defense with explainable threshold policy: A meta-learning approach. In: IEEE INFOCOM 2023 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pp 1–6, DOI 10.1109/INFOCOMWKSHPS57453.2023.10225816
  * Ometov et al. (2018) Ometov A, Bezzateev S, Mäkitalo N, Andreev S, Mikkonen T, Koucheryavy Y (2018) Multi-factor authentication: A survey. Cryptography 2(1):1
  * OpenAI (2023) OpenAI (2023) Gpt-4 technical report. arXiv preprint arXiv:230308774 URL https://doi.org/10.48550/arXiv.2303.08774
  * Vaswani et al. (2017) Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Lu, Polosukhin I (2017) Attention is all you need. In: Guyon I, Luxburg UV, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in Neural Information Processing Systems, Curran Associates, Inc., vol 30, URL https://proceedings.neurips.cc/paper\\_files/paper/2017/file/3f5ee243547dee91f\\\bd053c1c4a845aa-Paper.pdf
  * Kingma and Welling (2014) Kingma DP, Welling M (2014) Auto-encoding variational bayes. In: 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, http://arxiv.org/abs/1312.6114v10
  * Paisley et al. (2012) Paisley J, Blei DM, Jordan MI (2012) Variational bayesian inference with stochastic search. In: Proceedings of the 29th International Coference on International Conference on Machine Learning, Omnipress, Madison, WI, USA, ICML’12, p 1363–1370
  * Nash (1951) Nash J (1951) Non-cooperative games. The Annals of Mathematics 54(2):286–295, DOI 10.2307/1969529
  * Fudenberg and Tirole (1991) Fudenberg D, Tirole J (1991) Game Theory. MIT Press, Cambridge, MA
  * Li and Zhu (2023) Li T, Zhu Q (2023) On the price of transparency: A comparison between overt persuasion and covert signaling. arXiv preprint arXiv:230400096 URL https://doi.org/10.48550/arXiv.2304.00096
  * Li and Zhu (2019) Li T, Zhu Q (2019) On convergence rate of adaptive multiscale value function approximation for reinforcement learning. 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP) pp 1–6, DOI 10.1109/mlsp.2019.8918816
  * Li et al. (2021) Li T, Peng G, Zhu Q (2021) Blackwell online learning for markov decision processes. 2021 55th Annual Conference on Information Sciences and Systems (CISS) 00:1–6, DOI 10.1109/ciss50987.2021.9400319
  * Sutton et al. (2000) Sutton RS, McAllester DA, Singh SP, Mansour Y (2000) Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems 12, MIT press, pp 1057—1063, URL http://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf
  * Bannon et al. (2020) Bannon J, Windsor B, Song W, Li T (2020) Causality and batch reinforcement learning: Complementary approaches to planning in unknown domains. arXiv preprint arXiv:200602579 URL https://doi.org/10.48550/arXiv.2006.02579
  * Puterman (1994) Puterman ML (1994) Markov decision processes: Discrete stochastic dynamic programming, 1st edn. John Wiley & Sons, Inc., USA
  * Hu and Wellman (2003) Hu J, Wellman MP (2003) Nash q-learning for general-sum stochastic games. Journal of machine learning research 4(Nov):1039—1069
  * Hammar and Stadler (2023) Hammar K, Stadler R (2023) Digital twins for security automation. In: NOMS 2023-2023 IEEE/IFIP Network Operations and Management Symposium, pp 1–6, DOI 10.1109/NOMS56928.2023.10154288
  * Li et al. (2022) Li T, Lei H, Zhu Q (2022) Sampling attacks on meta reinforcement learning: A minimax formulation and complexity analysis. arXiv preprint arXiv:220800081 URL https://doi.org/10.48550/arXiv.2208.00081
  * Dazeley et al. (2023) Dazeley R, Vamplew P, Cruz F (2023) Explainable reinforcement learning for broad-xai: A conceptual framework and survey. Neural Comput Appl 35(23):16893–16916, DOI 10.1007/s00521-023-08423-1, URL https://doi.org/10.1007/s00521-023-08423-1
  * Ge and Zhu (2022) Ge Y, Zhu Q (2022) Trust threshold policy for explainable and adaptive zero-trust defense in enterprise networks. In: 2022 IEEE Conference on Communications and Network Security (CNS), pp 359–364, DOI 10.1109/CNS56114.2022.9947263
  * Hospedales et al. (2021) Hospedales TM, Antoniou A, Micaelli P, Storkey AJ (2021) Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence PP(99):1–1, DOI 10.1109/tpami.2021.3079209
  * Pan et al. (2023) Pan Y, Li T, Li H, Xu T, Zheng Z, Zhu Q (2023) A first order meta stackelberg method for robust federated learning. arXiv preprint arXiv:230613800 URL https://doi.org/10.48550/arXiv.2306.13800
  * Vapnik (1999) Vapnik V (1999) The nature of statistical learning theory. Springer science & business media
  * Liu et al. (2023) Liu S, Li T, Zhu Q (2023) Game-theoretic distributed empirical risk minimization with strategic network design. IEEE Transactions on Signal and Information Processing over Networks 9:542–556, DOI 10.1109/TSIPN.2023.3306106
  * Strom et al. (2018) Strom BE, Applebaum A, Miller DP, Nickels KC, Pennington AG, Thomas CB (2018) Mitre att&ck: Design and philosophy. In: Technical report, The MITRE Corporation
  * Hochreiter (2001) Hochreiter SY (2001) Learning to learn using gradient descent. Lecture Notes in Computer Science pp 87–94, DOI 10.1007/3-540-44668-0“˙13
  * Li et al. (2017) Li Z, Zhou F, Chen F, Li H (2017) Meta-sgd: Learning to learn quickly for few-shot learning. arXiv preprint arXiv: 170709835 URL https://doi.org/10.48550/arXiv.1707.09835
  * Yadav and Rao (2015) Yadav T, Rao AM (2015) Technical aspects of cyber kill chain. In: Security in Computing and Communications: Third International Symposium, SSCC 2015, Kochi, India, August 10-13, 2015. Proceedings 3, Springer, pp 438–452
  * Khan et al. (2018) Khan MS, Siddiqui S, Ferens K (2018) A cognitive and concurrent cyber kill chain model. Computer and Network Security Essentials pp 585–602
  * Huang and Zhu (2019) Huang L, Zhu Q (2019) Adaptive honeypot engagement through reinforcement learning of semi-markov decision processes. In: Decision and Game Theory for Security: 10th International Conference, GameSec 2019, Stockholm, Sweden, October 30–November 1, 2019, Proceedings 10, Springer, pp 196–216
  * Heckman et al. (2015) Heckman KE, Stech FJ, Schmoker BS, Thomas RK (2015) Denial and deception in cyber defense. Computer 48(4):36–44
  * Gore et al. (2017) Gore R, Padilla J, Diallo S (2017) Markov chain modeling of cyber threats. The Journal of Defense Modeling and Simulation 14(3):233–244
  * Kingman (1976) Kingman J (1976) Discrete-parameter martingales