# Impacts and Risk of Generative AI Technology on Cyber Defense

Subash Neupane, Ivan A. Fernandez, Sudip Mittal, Shahram Rahimi Dept. of
Computer Science & Engineering, Mississippi State University, Starkville, MS
Email: {sn922<EMAIL_ADDRESS>{mittal<EMAIL_ADDRESS>

###### Abstract

Generative Artificial Intelligence (GenAI) has emerged as a powerful
technology capable of autonomously producing highly realistic content in
various domains, such as text, images, audio, and videos. With its potential
for positive applications in creative arts, content generation, virtual
assistants, and data synthesis, GenAI has garnered significant attention and
adoption. However, the increasing adoption of GenAI raises concerns about its
potential misuse for crafting convincing phishing emails, generating
disinformation through deepfake videos, and spreading misinformation via
authentic-looking social media posts, posing a new set of challenges and risks
in the realm of cybersecurity.

To combat the threats posed by GenAI, we propose leveraging the Cyber Kill
Chain (CKC) to understand the lifecycle of cyberattacks, as a foundational
model for cyber defense. This paper aims to provide a comprehensive analysis
of the risk areas introduced by the offensive use of GenAI techniques in each
phase of the CKC framework. We also analyze the strategies employed by threat
actors and examine their utilization throughout different phases of the CKC,
highlighting the implications for cyber defense. Additionally, we propose
GenAI-enabled defense strategies that are both attack-aware and adaptive.
These strategies encompass various techniques such as detection, deception,
and adversarial training, among others, aiming to effectively mitigate the
risks posed by GenAI-induced cyber threats.

###### Index Terms:

Generative AI, Security, Cyber Defense, Cyber Kill Chain

##  I Introduction

Generative Artificial Intelligence (GenAI) has become a significant and
influential force in the realm of technology, empowering algorithms and models
to autonomously create highly realistic content across multiple domains,
including text, images, audio, and videos. This technology holds tremendous
potential for positive applications across various domains, including creative
arts, content generation, virtual assistants, and data synthesis, exemplified
by notable innovations such as ChatGPT [1]. GenAI enables content generation
by automating the creation of diverse and engaging textual and visual content
catering to specific styles, tones, or target audiences. Virtual assistants,
on the other hand, enhance human-computer interaction by providing
personalized recommendations, answering queries, and performing various tasks
based on user input. Furthermore, GenAI aids in data synthesis by analyzing
large datasets and generating synthetic data that closely resembles the
original, facilitating machine learning algorithm training and testing while
preserving privacy.

This technology, despite its immense capacity for beneficial utilization (for
example, Google Bard [2], Bing Chat[3], and ChatGPT [1]), has also introduced
a new dimension of risk to the cybersecurity landscape [4]. As GenAI models
become more pervasive, there is an increasing risk that they may be used to
craft convincing phishing emails, generate realistic deepfake videos for
disinformation campaigns [5], or create seemingly authentic social media posts
to spread misinformation [6]. According to a report published by Darktrace [7]
in April 2023, there was a 135% increase in the number of spam emails that
featured “ _markedly improved English-language grammar and syntax_ ” between
the months of January and February. This surge was attributed to threat actors
leveraging GenAI for their phishing campaigns. Similarly, a deepfake video [8]
depicting Ukrainian President Volodymyr Zelenskyy surfaced on online
platforms, calling for his people to surrender to Russian forces in March
2023. This incident serves as a compelling demonstration of how deepfakes can
be employed to propagate disinformation and manipulate public perception. In a
recent incident [9], there was an instance of misinformation wherein a
fabricated image of an explosion near the Pentagon, generated using AI
technology, was disseminated through social media platforms. This
dissemination of false information had notable repercussions, even influencing
the US stock market, which experienced a brief dip as a result.

Motivated by the need to address the evolving threat landscape induced by
GenAI, we propose leveraging the Cyber Kill Chain (CKC) as a foundational
model for cyber defense. To the best of our knowledge, this paper represents
the first in-depth investigation of cyber threats stemming from the
adversarial use of GenAI within the context of CKC. The CKC framework
originally developed by Lockheed Martin [10], provides a comprehensive
framework for understanding and disrupting the lifecycle of a cyberattack. By
applying the CKC to GenAI-based attacks, we can identify and counteract each
stage of the attack process, from initial reconnaissance to final
exfiltration. This approach offers effective means of countering the evolving
tactics employed by adversaries utilizing GenAI, ensuring a proactive defense
against these emerging threats.

In addition to countering the use of GenAI in cyberattacks, we also recognize
the presence of defense-aware adversaries who adapt their strategies and
techniques to circumvent traditional defensive measures. Numerous studies have
shed light on the diverse strategies employed by defense-aware threat actors,
encompassing evasion, polymorphism, deception, misinformation, etc. (see
Section III for details). These strategies, along with their variants, can be
used in various permutations across several stages of CKC. Research conducted
by Khan et al. [11] demonstrates how threat actors can utilize state-of-the-
art language models to launch highly targeted spear-phishing campaigns during
the reconnaissance phase, increasing the likelihood of successful compromise.
Moreover, the work of Das and colleagues [12] investigates the weaponization
of GenAI generated text content, showcasing how malicious actors can exploit
generative models to craft convincing social engineering messages, effectively
bypassing traditional security defenses during the delivery phase. Using
evasion strategies, threat actors can easily deceive existing defense systems.
For example, AlEroud et al. [13] demonstrated the URLs generated as
adversarial phishing examples in an inference integrity attack successfully
bypassing the target classifier. Similarly, researchers have achieved success
manipulating the Command and Control (C2) channel on social media platforms
using generative models, highlighted in the works of[14].

Furthermore, this paper proposes a taxonomy of defensive strategies that can
be employed by organizations to enhance the resilience of cyber defense
systems, effectively safeguard their critical assets, and protect their
digital ecosystem from emerging threats induced by the offensive use of GenAI.

The major contributions of the paper are as follows:

  * •

Identification of the impact of offensive use of generative models on the
cyber threat landscape, specifically in each phase of the Cyber Kill Chain
(CKC) framework.

  * •

Exploration of threat actors’ strategies to shed light on how these actors
employ sophisticated strategies to craft an attack.

  * •

Taxonomizing defensive strategies for detecting and mitigating threats
originating from generative models throughout the kill chain.

  * •

Providing a foundation for understanding the emerging threats posed by the
offensive use of generative models.

The remainder of the paper discusses the impact of GenAI in cybersecurity. In
Section II, we provide an overview of the GenAI landscape including
foundational concepts, taxonomy, and applications. Section III discusses
sophisticated attack strategies adopted by threat actors looking to maximize
the success of an attack using GenAI. In Section IV, we provide a
comprehensive analysis of the risk introduced by the misuse of GenAI
techniques in each phase of the CKC framework. Section V recommends defense
strategies that should be employed and embraced by organizations to mitigate
exposure to adversarial attacks induced by GenAI. Finally, Section VI reviews
the major challenges posed by GenAI-based cyberattacks and discusses
improvements to defensive strategies using emerging technologies.

##  II Overview of Generative AI

In this paper, we provide a high-level overview of the Generative AI (GenAI).
For a deeper understanding, we direct our readers to refer to the NIPS 2016
Tutorial on Generative Adversarial Networks by Goodfellow et al [15].
Furthermore, we recommend reviewing the article by Harshvardhan et al. [16]
for comprehensive analysis, algorithms, architecture, and implementation of
generative models.

The term GenAI is used to describe the use of unsupervised and semi-supervised
machine learning algorithms to generate synthetic data from existing data such
as text, audio and video files, images, or code, based on certain inputs. It
is achieved through the utilization of machine learning model referred to as a
_generative model_ , which is trained on a large dataset of examples and can
produce new instances that are comparable to the ones it was trained on. For
instance, given training data $\tilde{p}_{data}(x)$, the goal of generative
model is to learn $\tilde{p}_{model}(x)$ and generate new samples from same
distribution that approximates $\tilde{p}_{data}(x)$ as closely as possible.

Figure 1:  Adapted from the work of Goodfellow et al. [15], a taxonomy of
GenAI based on the tractability of their density distributions. The left
branch represents GenAI, which utilizes explicit density estimation techniques
as its foundation, whereas the right branch represents GenAI, which leverages
implicit density estimation techniques.

One method for learning a close approximation of $\tilde{p}_{data}(x)$
involves defining an explicitly parameterized function
$\tilde{p}_{model}(x;\theta)$ and iteratively searching for values of the
parameters that maximize the similarity between $\tilde{p}_{data}(x)$ and
$\tilde{p}_{model}(x)$. Most of the generative models are based on the
principle of maximum likelihood estimation while minimizing the Kullback-
Leibler divergence between $\tilde{p}_{data}(x)$ and $\tilde{p}_{model}(x)$.
The idea is to choose the parameters for the model that maximize the
likelihood of the training data.

A taxonomy of deep generative models is illustrated in Fig. 1. The left branch
of the taxonomy represents models that define explicit density functions,
whereas the right branch represents models that define implicit density
functions. Explicit density is further divided into two categories such as
tractable density and approximate density. Fully Visible Belief Nets (FVBN)
[17] and Nonlinear Independent Component Analysis (NICA) are example of models
based on tractable density functions. These models provides explicit
parametric specification of the distribution of an observed random variable
$x$ by defining a log-likelihood function $logq_{\theta}(x)$ with parameters
$\theta$. However, with the emergence of Deep Learning (DL), the focus of
study has shifted to learning models with relatively complex functional forms,
because when DL is used for synthesis, the corresponding density function may
be computationally intractable [18].

To circumvent some of the limitations imposed by the design requirements of
models with tractable density functions, other models have been developed that
use deterministic approximations such as variational methods and those that
use stochastic approximations such as Monte Carlo methods. One of the most
popular variational deep generative models is Variational Autoencoder (VAE)
[19]. Another approximation method is Markov Chain Approximation. A Markov
chain is a process for generating samples by repeatedly drawing a sample
$\boldsymbol{x}^{\prime}\sim
q\left(\boldsymbol{x}^{\prime}\mid\boldsymbol{x}\right)$. By repeatedly
updating $x$ according to the transition operator $q$, Markov chain methods
can sometimes guarantee that $x$ will eventually converge to a sample from
$p_{model}(x)$. However, the convergence with this methods is slow and
computational cost is high.

Implicit models, on the other hand, as illustrated on the right side of
taxonomy Fig 1 eliminate the need to build a tractable density function.
Instead, these models offer a way to train the model without directly
interacting with $p_{model}$, usually by taking samples from it. One such
popular method is Generative Adversarial Networks (GAN). Unlike other density
models, GANs do not require numerous approximations or Markov chains and have
the ability to generate a sample in a single step.

### GenAI & Application Areas

Large language models, such Generative Pre-trained Transformer (GPT-3 [20] and
GPT-4) and Bidirectional Encoder Representations from Transformers (BERT)
[20], are state-of-the-art transformer-based text generator models. GANs and
VAEs can also be used for text generation task but these models are not
computationally efficient due to their underlying architecture compared to
transformer models. Models for text-to-image generation include, DALL-E 2
[21], Imagen [22], and Midjourney [23]. While Phenaki [24], DreamFusion [25],
and Codex [26] are examples of models that can generate videos, 3D object, and
code from text. Several works have also been proposed to generate audio using
text as an input, for example AudioML [27], and Wishper [28].

In the following section, we will outline the attack strategy employed by an
attacker. A comprehensive awareness and knowledge of the attacker’s tactics
will pave the way for the creation of efficient defensive measures.

##  III Attack Strategies leveraging GenAI

Cyberattacks are constantly evolving, infiltrating all protection measures and
stealthily exfiltrating sensitive data. Understanding how such attacks work
assists in developing attack-aware defensive systems. However, the rapid
advancement in technology, in particular, the emergence GenAI and its misuse
(offensive use), has made attack vectors even more sophisticated, stealthy,
and able to bypass defense systems relatively easily. Hence, it is hard to
formulate a concrete plan to mitigate cyberattacks without understanding the
attack strategies that threat actors employ and embrace. In this paper, we
identify five such attack strategies: _evasion, adaptation and automation,
polymorphism, deception_ , and _misinformation_.

A clear picture, comprehension, and awareness of the abovementioned strategies
can effectively help defenders build robust defensive systems. However,
defenders should be aware that these are some of the main strategies. There
can be variants, combinations, and permutations of these strategies. Table I
summarizes how a combination of one or more attack strategies can be used in
several phases of the Cyber Kill Chain (CKC). For example, an evasion attack
strategy can be used to gather information about a network during
reconnaissance, hide information using steganography or other obfuscation
techniques to deliver a malicious payload, or install exploit rootkits and
malware.

In addition, the defender needs to be aware of any variations on these attack
strategies, for instance, a variant of evasion technique is the "_Living-of-
the-Land_ " (LOTL) technique, which leverages processes and legitimate tools
(PowerShell, Windows Management Instrumentation (WMI), Mimikatz, etc.) that
are already present on systems to avoid detection, perform network
reconnaissance, allow lateral movement, and maintain persistence[29]. POSHSPY
[30] is an example of a LOTL-based attack employed by the Russian state-
sponsored APT (Advanced Persistent Threat) group, APT29.

###  III-A Evasion

Evasion is commonly used to describe the process of conducting an attack while
evading or bypassing detection methods.

For example, in a ransomware attack like WannaCry [31], the attackers utilized
an exploit called EternalBlue, which targeted a vulnerability in Microsoft
Windows systems and utilized evasion techniques, including encryption and
obfuscation, to evade detection and spread rapidly. Furthermore, evasion can
be used to bypass firewalls, intrusion detection and prevention systems (IPS,
IDS), and malware analysis.

An evasion attack involves the creation of malicious inputs, commonly referred
to as adversarial examples using GenAI, that cause the detection system to
misclassify or make inaccurate predictions. Adversaries, for example, can
utilize GANs to generate adversarial malicious traffic to attack intrusion
detection systems, bypass them, and propagate throughout the network [32].
Similarly, they can also create malicious URLs that can easily bypass URL and
web detectors undetected [13, 33]. One of the evasion strategies in the
malware domain is the _obfuscation technique_ , a method of concealing the
attack payload used by malware to circumvent static analysis methods and
conventional anti-malware solutions. Adversaries can utilize GenAI to further
optimize the obfuscation technique. For example, they can generate creative
dead code insertion techniques, subroutine reordering (changing a code’s
subroutine to evade detection), register reassignment (changing a register to
a newer generation while retaining program code and behavior), etc. Therefore,
malware that has been obfuscated using generative models might be challenging
to detect [34].

###  III-B Adaptation and Automation

_Adaptation_ refers to the ability of a system or entity to adjust or modify
itself in response to changes in its environment or circumstances.
_Automation_ , on the other hand, refers to the use of technology to perform
tasks or processes with minimal human intervention.

Examples of attacks involving adaptation and automation strategies include
NotPetya [35], a destructive malware designed with an automated propagation
mechanism to infect new systems, while CrashOverride [36], a malware framework
that targeted Industrial Control Systems (ICS) used in electric power grids,
was designed to automate the various stages of Cyber Kill Chain (CKC).

Threat actors continuously augment their attacking skills through a
combination of learning, collaboration, and the adoption of emerging
technologies. In the past, threat actors relied on manual techniques and
limited automation to launch attacks. For instance, manually composing
phishing emails, manually building malware variants, manually researching
possible targets, etc. While these approaches were partially effective, they
were time-consuming and required a substantial amount of work to scale. At
present, threat actors may adopt state-of-the-art Large Language Models (LLM)
like GPT-3 [37] and GPT-4 that can generate human-like language to generate
personalized and highly convincing phishing messages that closely mimic
legitimate communications and use them for reconnaissance.

Such adaptation reduces the labor-intensive aspects of attacks, allowing
threat actors to concentrate on more strategic planning and enabling them to
scale their operations and launch attacks more rapidly.

###  III-C Polymorphism

_Polymorphism_ refers to the ability of an object or data element to take on
different forms or types. In the context of network security, it refers to a
strategy of self-mutation, or change in the form of attack vectors.

An example of this strategy, according to Darkreading [38], includes India’s
SideWinder Advanced Persistent Threat (APT) group targeting Pakistani
government officials and individuals, using polymorphism techniques to bypass
traditional signature-based antivirus detection and deliver a next-stage
payload. Examples of malware that utilize this strategy include Ursnif (or
Gozi) [39] and AAEH (or Beebone) [40], among others.

The attacks involving polymorphism mutate malware signatures in such a way
that an anti-malware detection system fails to recognize the new signature
variation, thereby infiltrating the target network or systems. Currently,
malware can evolve into thousands of polymorphic versions. The defense against
polymorphic malware comes in two flavors: static analysis and dynamic analysis
[41]. However, static analysis could be challenging due to the dynamic nature
of malware and the size of the code bases that must be analyzed for
polymorphic behavior.

Attackers can potentially misuse the ability of GenAI to generate synthetic
data to craft polymorphic attacks. Chauhan et al. [42] employed GANs to
demonstrate that deep learning techniques were not sufficient for detecting
new attack profiles. The same authors, in their later work, showcased that
attackers can launch polymorphic Distributed Denial of Service (DDoS) attacks
using a generative model such as WCGAN [43].

###  III-D Deception

Deception is an interaction between two parties, a deceiver and a target, in
which the deceiver successfully causes the target to accept as true a specific
incorrect version of reality, with the intent of causing the target to act in
a way that benefits the deceiver [44].

One example of trojan malware that employs deception strategies to evade
detection is the Emotet trojan , which spreads via phishing email attachments
and links that, once clicked, launch the payload. According to the MITRE
ATT&CK framework [45], Emotet evades detection by means of obfuscation and
software packing.

GenAI, specifically LLMs, offer powerful capabilities for generating realistic
and deceptive content, enabling attackers to manipulate, deceive, and exploit
their targets. Offensive applications of LLMs include deceptive content
generation for phishing emails and social engineering messages. Attackers, for
example, can generate content that closely mimics the _writing style, tone_ ,
and _vocabulary_ of certain individuals or organizations, including relevant
details and references, making it difficult for the recipient to identify the
email as malicious.

Threat actors can also use LLMs for _impersonation_ and _identity theft_. For
example, they leverage readily available tools such as ChatGPT [1], Brad [2]
or Bing Chat [3] to generate deceptive messages, legal documents, or official-
looking notices that imitate the communication style of a trusted individual
or organization and use them to trick people into obtaining sensitive
information or gaining unauthorized access to network systems.

Attack Strategy |  Cyber Kill Chain Stage |  Select Example and Techniques
---|---|---
Evasion |  Reconnaissance |  • Network Scanning, OS fingerprinting.
Delivery |  • Steganography, or other obfuscation techniques.
Installation |  • Employ rootkit or stealth mechanism.
Adoption and Automation |  Delivery |  • Automated botnets to distribute malware or launch DDoS attack.
Command and Control |  • Automated tools to manage and control compromised systems.
Exploitation |  • Use of automated exploit tools to scan for vulnerabilities.
Deception |  Reconnaissance |  • Intelligence gathering about target (Social Engineering, Phishing). • Creation of fake personas to trick user.
Exploitation |  • Use of fake or manipulated data to exploit vulnerability in target systems (Injection of malicious code in legitimate website).
Weaponization |  • Malware disguised as legitimate files or applications, fileless attacks (e.g., Living-of-the-Land (LOTL)).
Installation |  • Trick user via phishing message into installing payload.
Polymorphism |  Weaponization |  • Creating of polymorphic malware.
Delivery |  • Continuous change in delivery techniques using different file formats or encryption algorithms.
Exploitation |  • Polymorphic shellcode can continuously modify the exploit payload during an attack.
Misinformation |  Reconnaissance |  • Spread false information to mislead and divert attention from real attack vectors.
Action on Objectives |  • Create false report to cause panic or confusion.

TABLE I: An overview of how threat actors can utilize attack strategies along
the different stages of the Cyber Kill Chain (CKC) with some select examples
and techniques in the attack process.

###  III-E Misinformation

_Misinformation_ is false or inaccurate information that is deliberately
created and is intentionally or unintentionally propagated [46]. It is an
attack on our cognitive being. False information that is deliberately created
and intentionally propagated to mislead is called _disinformation_.

Threat actors such as nation-state players with geopolitical ambitions,
extremist groups, or economically motivated companies that exploit information
use this strategy to create societal unrest, enhance polarization, and, in
certain instances, affect the outcome of elections. The proliferation of
social networks such as Twitter [47], Facebook [48], and Reddit [49], coupled
with easy internet access, has created a fertile environment for the spread of
misinformation. These platforms have facilitated the easy publication of
content without proper editing, fact-checking, or accountability [50]. While
cyberattacks focus on targeting computer networks and end systems, deliberate
misinformation and half-truths can be employed to manipulate individuals’
perspectives or divert unsuspecting users from accessing accurate information
[51].

Furthermore, cutting-edge GenAI technology provides adversaries with the means
to produce highly realistic fake content encompassing images, audio, videos,
and text known as “deepfakes”. These synthetic media capabilities are made
feasible through the application of GANs, AutoEncoders and other GenAI models.
Threat actors can exploit these technique to create realistic artificial faces
to populate numerous bot accounts and use them to disseminate of
misinformation [52, 53, 54, 55]. Some popular deepfake tools available on the
market for face swap include Deepswap [56]; for lip-syncing, Wombo [57]; and
for bot creation, Instagram Deep Bot [58].

In the following section, we will explore how threat actors can utilize these
strategies in conjunction with GenAI as a powerful offensive tool to launch
cyberattacks. Specifically, our research will focus on investigating the
implications of GenAI on the CKC, which serves as a framework that formalizes
the study of cyber defense. By examining the perspective of an threat actor
equipped with GenAI within the context of the CKC, this research intends to
contribute to a deeper understanding of the challenges posed by advanced
adversaries and aid in the development of effective security measures against
GenAI-enabled cyberattacks.

##  IV Cybersecurity Risk Areas Impacted by GenAI

Sun Tzu [59], in his book The Art of War, once said, “If you know the enemy
and know yourself, you need not fear the result of a hundred battles.” This
quote encapsulates the fundamental importance of understanding both the
attacker and oneself in achieving success in warfare.

Figure 2: Adapted from Lockheed Martin [10], Cyber Kill Chain (CKC) stages
illustrate the sequential progression of a cyberattack from reconnaissance to
action on objectives. Each stage represents a crucial step in the threat
actors’ strategies and methodologies, providing insights for defensive
countermeasures.

In the realm of cybersecurity, this principle holds true, where risks abound.
The ever-increasing risks and threats in the digital landscape necessitate a
comprehensive approach to defense. To address the problem of evolving risks,
the Cyber Kill Chain (CKC) [10] concept was developed. Inspired by Lockheed
Martin’s military model, CKC is an intelligence-driven framework for
cybersecurity defense. It provides a systematic approach to identifying,
preparing for, engaging with, and neutralizing cyber threats.

Drawing upon the analogy of Sun Tzu’s quote, we can see a clear parallel
between threat actors, defenders, and the CKC. The primary goal of CKC is to
enhance cyber defense by thoroughly understanding the Tactics, Techniques, and
Procedures (TTPs) employed by threat actors (knowing the enemy) while
simultaneously identifying vulnerabilities and weaknesses within one’s own
systems (knowing yourself). Armed with this knowledge, defenders can develop
effective strategies, implement robust countermeasures, and proactively
prevent and mitigate the risk of cyberattacks.

The proposed CKC model, as indicated in Fig 2, includes seven distinct chained
stages: _reconnaissance, weaponization, delivery, exploitation, installation,
command and control (C2)_ , & _action on objectives_. Fig 3 illustrates the
sequential stages of a cyberattack in high level. It begins with
reconnaissance, followed by weaponization, where attack vectors like malware
are created. The third stage involves delivering the payload through methods
like phishing. Exploiting vulnerabilities triggers the payload in stage four.
The fifth stage establishes undetected persistence with backdoors. The sixth
stage establishes command & control, enabling remote communication. The final
stage sees the threat actors achieve their goals, such as financial gain or
data exfiltration.

Our objective is to thoroughly examine how threat actors can exploit GenAI at
each stage to carry out stealthy and sophisticated attacks. In the subsequent
subsection, we will provide a detailed analysis of each stage, emphasizing the
potential security implications and the use of GenAI from an offensive
standpoint.

###  IV-A Reconnaissance

Reconnaissance is the first phase of a cyberattack in which threat actors
study potential targets to find vulnerabilities. The efficacy of an attack
improves as more intelligence is obtained on a target. Reconnaissance can be
active or passive, depending on how the intelligence is acquired. Passive
reconnaissance, sometimes called footprinting, involves target surveillance
with a minimum amount of direct interactions. Using available Open-Source
Intelligence (OSINT) tools, adversaries can identify and select their target,
profile their system and social network, and validate their target. Once
attackers have gathered sufficient knowledge about the target through publicly
available resources, they transition to _active reconnaissance_. Active
reconnaissance, sometimes called scanning, involves direct interactions with
the target to gain additional insight.

Adept adversaries can leverage GenAI to improve both passive and active
reconnaissance strategies. As noted in Section III, GenAI augments traditional
information retrieval techniques and facilitates the creation of sophisticated
attacks. The success of ChatGPT has fueled an AI arms race that continues to
improve the state-of-the-art in Large Language Models (LLMs). Major search
engines are integrating AI chatbots (e.g., ChatGPT, Bing Chat and Bard) to
assist users on queries, productivity, and content creation. In order to
upstage competitors, LLMs are being pushed for early release which leads to
gaps in safeguarding mechanisms due to reduced timelines for verification and
validation (V&V) procedures. Astute threat actors can use carefully-designed
prompts to circumnavigate safeguards on LLMs.

AI chatbots can be used to quickly generate specialized summaries on potential
targets including organizational structure, personal details on
users/employees, and network information. AI chatbots can also generate
boilerplate code for scanning network traffic with little to no prior
experience. The ability to simulate what-if scenarios by engaging (role-
playing) with AI chatbots is another added benefit to adversaries. Through
role-playing, game trees can be constructed to counter target defenses for
different scenarios at different stages.

###  IV-B Weaponization

Once the vulnerabilities in the target system are identified during the
reconnaissance step, the attackers will transition into the second stage,
referred to as _weaponization_. In this phase, the objective is to develop an
attack that exploits the vulnerability discovered in the previous phase, where
the choice of weapon depends on gathered information. Metasploit, Exploit-DB,
Veil Framework, Aircrack, Burpsuit, etc. are some of the examples of weapons
used in this phase that assist in formulating a deliverable payload, such as
malwares, Remote Access Trojans (RATs) or backdoors coupled with exploits.

Polymorphic malware obfuscated with generative models is an excellent example
of a weaponized payload that threat actors may use in this phase to exploit
vulnerabilities. Threat actor can leverage GenAI to construct polymorphic
malware, which can generate malicious code variants with unique
characteristics that can avoid detection by traditional signature-based
detection systems. To learn the patterns and structures of malicious code, for
example, a generative model can be trained on a dataset of malware samples.
GenAI can then produce new variants by altering certain properties, such as
code control flow structure or encryption algorithms (refer to Section III-C
and III-A for more information on polymorphism and obsfucation strategies).

###  IV-C Delivery

The attacker must find a way to deliver its payload to its intended targets.
During this stage, threat actors consider all possible avenues to deliver the
malicious payload into the target’s network or device. In the body of
literature, the most common malicious payload delivery approaches observed are
via auxiliary means such as phishing email attachments, social engineering,
USB sticks, and drive-by download. In some cases, attackers can also directly
attack a target’s web services.

Figure 3: Tracing an attack utilizing the Cyber Kill Chain (CKC) attack
framework. In the first stage, the threat actor gathers intelligence through
_reconnaissance_. The second stage is _weaponization_ , where threat actors
create attack vectors (e.g., malware) based on identified vulnerabilities. The
third stage is about _delivering_ a malicious payload in the target
environment (usually with phishing and social engineering). A malicious
payload is triggered in the fourth stage to _exploit_ vulnerabilities. In the
fifth stage, backdoors are _installed_ to maintain undetected persistence. The
sixth stage is about establishing _command & control_ in the target
environment for remote communication. Threat actors _achieve their goals_ ,
such as financial gain, data exfiltration, geopolitical gain, etc., after a
successful attack in the final stage.

A spear-phishing attack includes sending an email with a malicious attachment
(payload) to a target victim, typically by impersonating an internal employee.
Currently, deep learning along with language models are being applied to
launch advanced spear phishing attacks. One such example of utilizing
Recurrent Neural Networks (RNN) and Natural Language Generation (NLG) to
generate spear-phishing attacks is presented by authors in [12]. The proposed
system generates fake emails with varying malicious payloads (depending on the
target). Moreover, it has also been found that attackers can leverage state-
of-the-art generative language models such as Generative Pre-Trained
Transformers (GPT-2, GPT-3 and GPT-4) to automate the process of email
generation. The work by Khan et al. [11] highlights the use of a state-of-the-
art language model for spear-phishing activity.

Sophisticated attackers could use GenAI to deceive (_deception strategy_) the
target’s defense system and deliver a malicious payload. Anderson et al. [60]
show that GANs can be used to generate an undetectable malware URL that
bypasses a deep neural network (DNN) based detection system. At present,
several URL classification algorithms have been proposed to detect malicious
URLs; however, these defensive mechanisms are susceptible to _evasion_ attacks
using GenAI. AlEroud et al.’s [13] work precisely demonstrates how existing
URL classification can be circumvented using GANs. Similarly, a recent work by
the authors of [33] presents a realistic threat model describing evasion
attacks against machine learning-enabled phishing web detectors using GAN
classifiers.

###  IV-D Exploitation

The attack payloads or malicious programs delivered in the preceding stage are
triggered in the exploitation stage, enabling threat actors to exploit the
vulnerabilities and weaknesses of the target systems and/or network. To gain
better control and a stronger footing in the target environment, threat actors
employ various exploitation techniques to compromise systems, networks, and
applications. The most commonly used technique involves identifying and
exploiting software vulnerabilities. Ghaffarian et al. [61] define a _software
vulnerability_ as “an instance of a flaw caused by a mistake in the design,
development, or configuration of software such that it can be exploited to
violate some explicit or implicit security policy.” A threat actor may exploit
software vulnerabilities to gain unauthorized access or execute arbitrary code
by attacking weak endpoints in software using techniques such as buffer
overflows, SQL injection, cross-site scripting (XSS), and remote code
execution.

Threat actors can leverage GenAI to automatically create synthetic SQL
injection scripts and use them to attack web frameworks. One example of
creating a synthetic SQL attack sample by leveraging GANs can be found in the
work of [62]. The authors combine genetic algorithms and GANs to generate
synthetic samples of SQL injection attack statements that are more complex and
richer injection codes. XSS is another category of injection attack in which
malicious scripts are injected into otherwise safe, benign, and trusted
websites. Similar to the creation of adversarial SQL injection attack samples,
threat actors can generate sophisticated and stealthier XSS attack samples
that can bypass detection system(s) utilizing GenAI. For example, Zhang et al.
[63] constructed a GAN and proposed generating adversarial samples of Cross-
Site Scripting (XSS) attacks with the Monte Carlo Tree Search (MCTS)
algorithm. Samples are generated following a set of bypassing rules such as
hexadecimal encoding, decimal encoding, url encoding, insertion of invalid
characters in the middle of tags, and case mixture. This capability of GAN can
be leveraged offensively by a threat actor to generate and automate XSS
attacks.

###  IV-E Installation

After successfully exploiting the vulnerability, threat actors, in this stage,
attempt to gain access to additional nodes (i.e., spread the malware in the
network) and install remote administration tools, such as Remote Access
Trojans (RAT) or backdoors, in order to maintain their presence in the target
environment [10] and exfiltrate valuable data. RAT refers to software, a piece
of code, or a model that executes on the target’s system and provides remote,
hidden, and undetected access to the threat actor, whereas exploits utilize
RAT to maliciously exploit the vulnerabilities of a system or software to
infiltrate or initiate an attack.

In the context of crafting backdoor attacks, also known as trojaning attacks,
using generative learning techniques, the attacker teaches the model to
recognize an unusual pattern that triggers a specific action, for example,
classifying a sample as safe [64]. One method for injecting a backdoor
involves adding a Trojan trigger to a subset of the training sample and
modifying the associated labels to correspond to the target class. Pixels can
serve as triggers in the spatial domain (images, videos), whereas words or a
phrase can serve in the sequential domain [65]. GenAI can be used to create
such triggers. For instance, Gu et al. [66] presented the BadNets attack, in
which the Trojan is injected by tainting the training dataset. In this
framework, the attacker stamps a trigger pattern, i.e., a pixel, on an
arbitrary subset of images in the training data. The attackers then mislabel
the modified samples with the desired target label; consequently, whenever the
trigger pattern is present, the learning classifier is trained to misclassify.
In a similar fashion, the authors of [67] devised a trojaning technique that
creates triggers and ensures that specific neurons are activated in order to
inject a backdoor.

Besides, GenAI also enable the creation of stealthy and evasive installation
methods. For instance, GANs can generate steganographic techniques that hide
malicious code within innocuous-looking files (images, text etc.), making
detection and analysis more challenging for defensive systems. A novel
technique called SteganoGAN for hiding arbitrary binary data in images using
GANs that is capable of evading detection by steganalysis tools is proposed by
Zhang et al. [68].

###  IV-F Command and Control (C2)

After an attacker has successfully compromised a system, they typically look
to establish Command and Control (C2) in infrastructure systems, endpoints,
etc., which allows them to remotely communicate with the compromised system
and issue commands via C2 channels. If the attackers successfully execute the
previous steps, their access to the system is persistent even if the
compromised system is restarted or a patch is installed. The infected device
can either immediately begin executing missions or wait for further
instructions from its C2 server.

Attackers can utilize GenAI techniques to evade detection during the (C2)
phase by creating models that mimic the network behavior of legitimate
applications, camouflaging the traffic as normal. To illustrate this, Rigaki
et al. [14] presented a method employing GAN that modifies the behavioral
patterns of a specific malware in the network to mimic Facebook chat traffic
in order to conceal C2 communication. Results show that the proposed method
can successfully modify the traffic of malware to make it undetectable. Other
methods that hide (_evasion strategy_) malware in neural networks to deliver
malicious pay load and enable C2 communication channels are StegoNet [69] and
EvilModel [70].

DeepC2 [71] on the other hand, use deep learning techniques to launch a covert
C2 scenario on online social networks such as Twitter. In order to create C2
links, Domain Generation Algorithms (DGAs) are utilized by a large number of
malware families. Anderson et al, [60] leverage the concept of GANs to
construct a deep learning based hard-to-detect artificial malware domains DGA
that can even bypass a deep learning-based detector. One of the techniques
that is currently in practice to defend against C2 attacks is deep SSL packet
inspection, a network packet inspection system that examines packet payloads
to identify traffic flows (TCP/IP connections) in a C2 channel. However,
detection of both encrypted and non-encrypted malware command and control
traffic based on TCP/IP is vulnerable to evasion attacks using deep generative
models. For example, generative models can be used to create realistic network
traffic such as ICMP pings, DNS queries, and HTTP web requests and can be used
to evade C2 detectors. Examples of use of GenAI to bypass detectors in the C2
channel are in [72, 14].

###  IV-G Action on Objectives

The action on objectives is the last stage of the kill chain, wherein the
attackers execute action to achieve their objective. The action is usually
dictated by the motivation of the attacker. Understanding the type of attacker
(a single person, an insider, a nation-state, etc.) is critical to
understanding their motivation. Attackers may be motivated by several factors,
including financial gain, political gain, nation-state interests, malicious
insiders, or simply wanting to move laterally to go after a more important
system in the network. Financial motivation is evident with attacks involving
ransomware, which are designed to encrypt the target’s end system, deny
custodians to access their data and systems, and demand ransom payments in
exchange for releasing the system or encrypted data. Usually, these attacks
are carried out by successful delivery of malware using sophisticated phishing
strategies (malicious email attachments), drive-by downloads, malicious URLs,
etc.

A zero-trust defensive security model is designed with the assumption that
every user account, device, or endpoint will fall victim to action on the
objective stage of the cyber kill chain by removing the idea of trust and
treating all traffic to the network as untrusted until proven otherwise
(usually through a rigorous authentication process). Essentially, throughout
this phase, defenders are actively monitoring user and device activity to
identify potential threats. In contrast, adversaries could possibly model user
behavior using GenAI. For example, they can first generate user behavior and
later use evasion and deception strategies to bypass the zero-trust security
model. An instance of abnormal user behavior generation using DCGAN in a zero-
trust network is presented in the work of [73].

Lateral movement is another important aspect in this phase of the CKC, where
the adversary’s end goal is to move laterally inside the network, perform
reconnaissance about the deeper network, and launch an attack. One way of
moving laterally without being detected by defensive systems is via dynamic IP
mutation. In such a situation, adversaries can leverage GenAI techniques to
mutate IP addresses dynamically. For instance, it is possible to produce
mutant IP addresses in an unpredictable manner without compromising network
performance [74].

Another objective of adversaries in this phase is to exfiltrate target data.
Usually, adversaries use malware to compromise systems or exfiltrate data.
GenAI can be used to obfuscate malware and hence bypass detection. An example
of the use of a GenAI to evade malware detection is MalGAN [34]. Works like
MalGAN showcase how adversaries can use state-of-the-art generative models to
avoid detection, which consequently paves their way to achieve their goal of
exfiltrating the data.

Next, we explain defensive strategies.

##  V Defense Strategies to Mitigate GenAI Risks

The relentless surge in cyberattacks, orchestrated by ever-evolving threat
actors, has escalated both in frequency and complexity. As a result,
conventional defensive strategies are proving woefully inadequate in the face
of these persistent and highly skilled attackers, particularly when they
exploit cutting-edge attack techniques, notably harnessing the power of GenAI.
Reactive measures and commercial security solutions, such as patching end
systems and software to prevent zero-day exploits and blocking bad IP
addresses and domain names, are the primary defense mechanisms employed by
most organizations. However, these measures are insufficient against advanced
attacks that leverage deep learning techniques, specifically, GenAI and
sophisticated attack strategies (refer to Section III). Therefore, there is a
critical need to develop proactive defense strategies that can detect and
respond to these evolving threats.

To develop an effective defensive framework, it is crucial to understand the
attacker’s strategy, their knowledge, and their goal. In the preceding Section
IV, we explored the capabilities of GenAI from the perspective of a
sophisticated threat actor. Drawing on this knowledge, we present a taxonomy
of defensive strategies, as depicted in Fig 4, encompassing _detection,
deception, adversarial training, red and blue teaming, explainable ai_ , _user
awareness training_ and _streamlining existing security strategies_ aimed at
countering the risks introduced by GenAI throughout the Cyber Kill Chain
(CKC).

In the following sections, we will provide detailed explanations for each of
these categories.

###  V-A Detection

GenAI is increasingly being used as a detection technique to mitigate
organizational exposure to cyberattacks. Systems developed with these models
can learn the underlying distribution of normal network or system behavior and
detect deviations from that distribution that may indicate an attack.

Several works can be found in the body of literature that focus on
unsupervised _anomaly detection_ by leveraging generative models such as GANs.
For example, the work in TadGAN [75], TanoGAN [76], and Mad-GAN [77]
represents anomaly detection in time series data.

Building on the success of anomaly detection tasks using GenAI, researchers
have also explored their application in other cybersecurity domains, such as
phishing detection. _Phishing attacks_ are a common method used by attackers
to obtain sensitive information from individuals and organizations. These
attacks can be used in _reconnaissance_ and _delivery_ phase of CKC. Shirazi
et al. [78] demonstrated the generative capability of adversarial auto-
encoders in generating synthesized phishing samples when augmented with real-
world phishing data provided additional robustness to phishing detection
classifiers. Another work showcases the effectiveness of the combination of
GAN (LeakGAN) and large language models (BERT [20]) in phishing email
detection [79].

In addition to combating phishing attacks, the application of GenAI extends to
the field of _malware detection_ , where these models have proven instrumental
in identifying polymorphic and metamorphic malware that constantly adapt to
evade conventional detection methods. Using the Deep Belief Network (DBN),
Yuan et al. [80] presented a system for Android malware detection referred to
as _DroidDetector_ that can consequently distinguish whether a file has
malicious behavior or not. GANs have also been used to detect malware on
Android devices, such as in work of [81].

Moreover, as the field of cybersecurity continues to evolve, researchers have
extended the utilization of GenAI beyond anomaly, phishing and malware
detection to address the emerging challenge of deepfakes. _Deepfakes_ are
synthetic videos, images, or texts that are created using generative models
and can be used to manipulate or deceive individuals and organizations. To
tackle this issue, researchers have developed detection systems tailored to
different types of deepfakes. Fagni et al. [82] proposed a novel text-based
social media message (Twitter) deepfake detection system. They demonstrated
that the RoBERTa [83] detector outperforms both traditional ML models (e.g.,
bag-of-words) and neural network models (e.g., CNN) in successfully
distinguishing machine-generated tweets from human-written tweets. Another
study on the detection of fake social media texts utilizing GPT-2 was
conducted by Adelani et al. [84] on Amazon product reviews. Intel most
recently launched FakeCatcher [85], a technology that can detect deepfake
videos with a 96% accuracy in real-time.

Figure 4: A taxonomy of proposed defensive strategies to mitigate GenAI risk,
including _detection, deception, adversarial training, red and blue teaming,
explainable AI_ _user awareness training_ , and _streamlining existing
security strategies_. The combination of these strategies aims to enhance the
resilience of systems and networks against cyberattacks.

###  V-B Deception

Defensive deception, for example, the deployment of decoy assets and
honeypots—a network of seemingly vulnerable machines designed to lure threat
actors—is a widely studied area in the security domain and is an established
technique often used to gather Cyber Threat Intelligence (CTI), such as
indicators and signatures of cyberattacks across various network environments
[86, 87, 88, 89]. This strategic technique has also been explored to set up a
trap to distract and mislead attackers from real systems and thwart threat
actors’ (both insiders’ and outsiders’) efforts to identify and steal
sensitive information on systems or networks [90].

Researchers are currently investigating generative models to create
sophisticated honeypots, or in some cases, honeyusers. For instance, Lukas et
al. [91] propose the creation of fake user accounts as honey tokens
(honeyusers) on Active Directory Server using Variational Auto Encoder (VAE)
to capture malicious access attempts. While some researchers are working to
optimize and transform the log reports regenerated by honeypots such as Cowrie
into question-and-answer problems using language models such as GPT-2 [92].

###  V-C Adversarial Training

In the context of cyberattacks, _adversarial training_ is used to defend
against adversarial examples, which are small perturbations to input data that
is specifically designed to deceive or mislead a machine learning model.

This strategy has been shown to be effective in defending against various
types of cyberattacks, including malware and phishing attacks. Several GAN-
based studies have proposed methods to improve the evasion detection of
adversarial malware. Rafiq et al. [93] demonstrated the vulnerability of
Android malware classifiers in adversarial settings. They showcased an
adversarial training scheme to combat evasion attacks on ML-based Android
malware classifiers with 99.46% detection accuracy. In a similar vein, Taheri
et al. [94] employed GANs to develop a defense against evasion attacks,
employing five distinct evasion attack models on Android malware classifiers.

Other defensive strategies include _defensive distillation_ , a technique used
to improve the robustness of a machine learning model against adversarial
attacks by creating a distilled model. Papernot et al [95]. were the first to
demonstrate that defensive distillation may withstand adversarial attacks with
minor perturbations in white-box settings.

###  V-D Red and Blue Teaming

Red Teaming is a strategy used in military and commercial security operations
to discover networked system vulnerabilities or exploitable gaps in
operational concepts, with the main goal of decreasing surprises, improving
them, and ensuring their robustness [96]. Blue Teaming, on the other hand,
aims to protect networked systems and infrastructure from both simulated and
real-world attacks while preserving the security posture of an organization.

The significance of red and blue teaming resides in their combined efforts to
identify vulnerabilities, replicate real-world attack scenarios, and develop
defensive mechanisms. Red teams can emulate a phishing attack scenario using
language models or create the evasive malware using state-of-the-art
generative models and launch it in a controlled environment to test the
resilience of their networked systems. One example of maximizing the success
of penetration testing with effective reconnaissance using ChatGPT is
presented by the authors in [97]. First-hand exposure to such sophisticated
attacks, even in a simulated environment, will empower the blue team with
experience and prepare them to develop proactive countermeasures to
effectively defend against such threats should they arise in the real world.
Collaboration and feedback between red and blue teams helps firms better
understand their security gaps, develop more effective incident response
strategies, and strengthen their defenses over time.

Figure 5: Recommended defensive tools and techniques for each phase of the
Cyber Kill Chain (CKC) framework that should be adopted by organization in
conjunction with GenAI-enabled security strategies for comprehensive cyber
defense.

###  V-E Explainable AI (XAI)

Explainable Artificial Intelligence (XAI) refers to the research and
implementation of Artificial Intelligence (AI) models and systems that can
provide human-comprehensible explanations for the output generated by “black
box” AI models. Researchers are increasingly using XAI as a cybersecurity
defense strategy to safeguard networks and endpoints. For example, in our
previous works, we investigated XAI techniques in the intrusion detection
domain [98, 99, 100] and explained the anomalies using feature importance in
vehicular digital twins [101].

XAI provides interpretability and transparency in AI models, and its
integration can aid organizations in detecting and identifying deceptive
content and adversarial examples, assessing model vulnerabilities, and
monitoring emerging threats. Greco et al. [102] presented an XAI-based
solution to classify phishing emails (deceptive content) and alert users to
the risk by explaining the reasons behind the attacks. In another study by
Morcos et al. [103], the authors utilized SHAP methods to enhance transparency
and interpretability in malware detection (exfiltration detection). Fidel et
al. [104], in their work, presented a novel method to detect adversarial input
using SHAP methods. Furthermore, in the case of Advanced Persistent Threat
(APT) detection, XAI techniques were utilized by the authors in [105].

###  V-F User Training and Awareness

The importance of security awareness and training resides in the fact that it
equips individuals with the knowledge and skills required to identify and
mitigate the associated with offensive use generative models. As the
sophistication of generative models increases, they have the potential to
generate content that is both realistic and deceptive, which can be used to
launch an attack using social engineering or phishing, create malware or fake
identities, or even manipulate sensor data in cyber-physical systems. By
offering comprehensive security awareness and training programs, organizations
can educate their employees, and system operators about these common attack
vectors generated by the misuse of generative models. This in turn, enables
them to recognize the signs of manipulated or fraudulent content, develop a
security-conscious mindset and become first line of defense.

Overall, organizations can fortify their cyber defenses, ensure the
confidentiality, integrity, availability, and reliability of their networked
systems, and create a workforce that is vigilant, proactive, and capable of
effectively mitigating cyber threats by continuously reinforcing security
awareness and conducting regular training sessions.

###  V-G Streamlining Existing Security Strategies

In the preceding subsections, we emphasized the substantial impact of GenAI in
bolstering cybersecurity defenses, encompassing vulnerability identification,
attack detection, and response mechanisms. Nevertheless, it is vital to
acknowledge that GenAI alone cannot serve as a comprehensive solution for
cyber defense, and it should be integrated seamlessly with other existing
security strategies. We strongly recommend adopting the defensive tools and
techniques that we have identified, as illustrated in Fig. 5, to effectively
mitigate the likelihood of falling victim to cyber risks at each stage of the
Cyber Kill Chain (CKC).

Mitigation of the _reconnaissance_ phase entails the implementation of several
defensive techniques. First, organizations should thoroughly review their
website description and other online platforms to ensure that sensitive
information about the company’s infrastructure, systems, or employees is not
publicly disclosed. This includes sanitizing the website description by
removing or minimizing specific details that could be potentially exploited by
attackers. Additionally, closely monitoring network traffic and analyzing
system logs are vital steps to detect any suspicious activity. Disabling any
unused ports and limiting the amount of information made available about the
system and its users are also effective measures to prevent successful social
engineering attacks.

In order to mitigate the likelihood of exposure during the _weaponization_
phase of a cyberattack, defenders can utilize several tools and techniques,
such as anti-malware software to detect known threats, patching known
vulnerabilities to prevent exploitation, and disabling potentially risky
elements such as office macros, JavaScript, and browser plugins. Furthermore,
limiting user privileges can prevent the installation and exploitation of
malware.

Organizations can use a variety of tools and methods to make it less likely
that they will be exposed during the _delivery_ phase of a cyberattack. These
include using email and web filtering to stop malicious URL attachments,
disabling USB devices, and training and educating users on how to spot
phishing emails and suspicious websites.

Defensive tools and techniques to curb the risk of _exploitation_ include data
execution prevention through the use of least privilege access control
mechanisms, anti-exploit measures, and sandboxing. A careful consideration of
these tools and measures aids organizations in preventing the exploitation of
vulnerabilities and the execution of malicious code within their systems.

Organizations can reduce the likelihood that a threat actor will _install_ a
malicious payload in a network by taking defensive measures such as disabling
PowerShell in the Windows environment, analyzing user behavior in the network
to identify anomalies, establishing incident response procedures, and having
the ability to restore or reimage compromised systems.

Network segmentation can effectively prevent threat actors from establishing
_C2_ in the inner network by restricting their access to other network nodes.
Additionally, the installation and implementation of network traffic
monitoring tools (for example, SSL deep packet inspection) can be an effective
way to detect and prevent C2 traffic. Lastly, implementing data leakage
prevention systems to detect and prevent the exfiltration of sensitive and
confidential data, along with a zero-trust security model and employing
encryption to protect data in transit, can assist in curbing the effects of a
cyberattack in the last phase of the Cyber Kill Chain (CKC).

##  VI Conclusion & Discussion

As GenAI becomes more ubiquitous, we in this paper shed light on its impact on
the cyber threat landscape. The ability to generate highly realistic and
deceptive content such as text, image, video when used offensively can
threaten the security and the trustworthiness of systems that use these
technologies. We have explored how the malicious use of GenAI can exploit
vulnerabilities and craft sophisticated cyberattacks, further emphasizing the
importance of robust defense mechanisms.

To counter evolving cyber threats, we have proposed a cyber defense strategy
built upon the Cyber Kill Chain (CKC) framework. By understanding and
disrupting each stage of the attack lifecycle, organizations can effectively
mitigate the potential risks posed by GenAI-induced cyber threats. Moreover,
we have emphasized the importance of comprehending defense-aware adversaries
and their strategies in the context of CKC. Threat actors can employ
sophisticated strategies such as evasion, adaptation, automation,
polymorphism, deception, and misinformation to bypass traditional defense
mechanisms. Understanding these strategies, their combinations, permutations,
and variants across several stages of CKC enables the development of more
resilient defense systems.

Drawing upon our findings, we have recommended a range of defensive strategies
and tools to aid defenders in mitigating threats. We explored defensive
strategies from two standpoints: First, we developed a taxonomy of defensive
strategies enabled by GenAI, encompassing detection, deception, adversarial
training, red and blue teaming, explainable AI, user training and awareness
programs, and streamlining existing security strategies. Second, we emphasized
the need to use available tools and techniques to improve organizational
security posture. It is important to recognize that defense systems enabled by
GenAI alone are not sufficient to combat the evolving cyber threat landscape;
they must be used in conjunction with available tools to realize a robust
defense framework.

Most of the reactive and preventive security solutions that are at our
disposal today focus on enhancing security policies, security monitoring, and
installing robust access control mechanisms to combat cyber threats, but these
solutions are inadequate to address the increasing speed, scale, and
sophistication of automated cyberattacks. For example, the sheer volume of
automated cyberattacks we see and hear almost in daily basis, has now
surpassed the ability of humans to manually analyze them.

Therefore, there is now a critical need to develop automated defense solutions
within organizations to address the problem of this evolving threat landscape.
One such solution to this problem is the concept of Automated Cyber Defense
(ACD) systems—a system that can self-discover, prove, and correct software
vulnerabilities in real-time without human intervention.

Emerging ML technologies such as Reinforcement Learning (RL) that can learn
from its own experiences by exploring its environment can be utilized to
create defense agent systems with self-learning capabilities [106, 107]. For
example, Applebaum et al. [108] and Cardellini et al. [106] leveraged RL
techniques to train an agent that is able to autonomously defend a system,
minimizing self-damage from responses that use noisy sensor data. Nguyen et
al. [109], on the other hand, surveyed several deep RL methods for autonomous
defense strategies, such as autonomous RL-based IDS and multi-agent RL-based
game theory approaches, to obtain optimal policies in different attacking
scenarios.

By staying vigilant, investing in advanced detection technologies, and
fostering a culture of cybersecurity awareness, organizations can bolster
their resilience and effectively safeguard against the growing threats
introduced by the offensive use of GenAI.

## Acknowledgment

This work was supported by PATENT Lab (Predictive Analytics and TEchnology
iNTegration Laboratory) at the Department of Computer Science and Engineering,
Mississippi State University.

## References

  * [1] Introducing ChatGPT.  https://openai.com/blog/chatgpt, 2023.  Accessed: 2023-06-01.
  * [2] GoogleBrad.  https://bard.google.com/, 2023.  Accessed: 2023-06-12.
  * [3] Bing Chat.  https://www.microsoft.com/en-us/edge/features/bing-chat?form=MT00D8, 2023\.  Accessed: 2023-05-21.
  * [4] A Stevie Bergman, Gavin Abercrombie, Shannon Spruit, Dirk Hovy, Emily Dinan, Y-Lan Boureau, Verena Rieser, et al.  Guiding the release of safer e2e conversational ai through value sensitive design.  In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue. Association for Computational Linguistics, 2022.
  * [5] AI-generated disinformation poses threat of misleading voters in 2024 election.  https://www.pbs.org/newshour/politics/ai-generated-disinformation-poses-threat-of-misleading-voters-in-2024-election, 2023\.  Accessed: 2023-06-10.
  * [6] G Marcus.  A skeptical take on the ai revolution, 2023.
  * [7] Press Release.  https://ir.darktrace.com/press-releases/2023/4/3/8b2d6ba25d9d54a1895956a985fe4a7d08d9f42607a112fb17964e4b57fad7d6, 2023\.  Accessed: 2023-06-21.
  * [8] Deepfake presidents used in Russia-Ukraine war.  https://www.bbc.com/news/technology-60780142, 2023.  Accessed: 2023-06-21.
  * [9] Don’t Believe Your Lying Eyes in the AI Era.  https://www.bloomberg.com/opinion/articles/2023-05-23/viral-fake-pentagon-explosion-photo-on-twitter-is-an-ai-wake-up-call#xj4y7vzkg, 2023\.  Accessed: 2023-06-21.
  * [10] Eric M Hutchins, Michael J Cloppert, Rohan M Amin, et al.  Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains.  Leading Issues in Information Warfare & Security Research, 1(1):80, 2011.
  * [11] Hajra Khan, Masoom Alam, Saif Al-Kuwari, and Yasir Faheem.  Offensive ai: Unification of email generation through gpt-2 model with a game-theoretic approach for spear-phishing attacks.  2021\.
  * [12] Avisha Das and Rakesh Verma.  Automated email generation for targeted attacks using natural language.  arXiv preprint arXiv:1908.06893, 2019.
  * [13] Ahmed AlEroud and George Karabatis.  Bypassing detection of url-based phishing attacks using generative adversarial deep neural networks.  In Proceedings of the sixth international workshop on security and privacy analytics, pages 53–60, 2020.
  * [14] Maria Rigaki and Sebastian Garcia.  Bringing a gan to a knife-fight: Adapting malware communication to avoid detection.  In 2018 IEEE Security and Privacy Workshops (SPW), pages 70–75. IEEE, 2018.
  * [15] Ian Goodfellow.  Nips 2016 tutorial: Generative adversarial networks.  arXiv preprint arXiv:1701.00160, 2016.
  * [16] GM Harshvardhan, Mahendra Kumar Gourisaria, Manjusha Pandey, and Siddharth Swarup Rautaray.  A comprehensive survey and analysis of generative models in machine learning.  Computer Science Review, 38:100285, 2020.
  * [17] Brendan J Frey, Geoffrey E Hinton, and Peter Dayan.  Does the wake-sleep algorithm produce good density estimators?  Advances in neural information processing systems, 8, 1995.
  * [18] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.  Generative adversarial networks.  Communications of the ACM, 63(11):139–144, 2020.
  * [19] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  arXiv preprint arXiv:1312.6114, 2013.
  * [20] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova.  Bert: Pre-training of deep bidirectional transformers for language understanding.  arXiv preprint arXiv:1810.04805, 2018.
  * [21] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever.  Zero-shot text-to-image generation.  In International Conference on Machine Learning, pages 8821–8831. PMLR, 2021.
  * [22] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho.  Variational diffusion models.  Advances in neural information processing systems, 34:21696–21707, 2021.
  * [23] Midjourney Documentation.  https://docs.midjourney.com/, 2023.  Accessed: 2023-06-8.
  * [24] Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kindermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan.  Phenaki: Variable length video generation from open domain textual description.  arXiv preprint arXiv:2210.02399, 2022.
  * [25] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall.  Dreamfusion: Text-to-3d using 2d diffusion.  arXiv preprint arXiv:2209.14988, 2022.
  * [26] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al.  Evaluating large language models trained on code.  arXiv preprint arXiv:2107.03374, 2021.
  * [27] Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matt Sharifi, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour.  Audiolm: a language modeling approach to audio generation.  arXiv preprint arXiv:2209.03143, 2022.
  * [28] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever.  Robust speech recognition via large-scale weak supervision.  arXiv preprint arXiv:2212.04356, 2022.
  * [29] Frederick Barr-Smith, Xabier Ugarte-Pedrero, Mariano Graziano, Riccardo Spolaor, and Ivan Martinovic.  Survivalism: Systematic analysis of windows malware living-off-the-land.  In 2021 IEEE Symposium on Security and Privacy (SP), pages 1557–1574. IEEE, 2021.
  * [30] Dissecting One of APT29’s Fileless WMI and PowerShell Backdoors (POSHSPY).  https://www.mandiant.com/resources/blog/dissecting-one-ofap, 2023\.  Accessed: 2023-06-13.
  * [31] WHAT IS WANNACRY/WANACRYPT0R?  https://www.cisa.gov/sites/default/files/FactSheets/NCCIC%20ICS_FactSheet_WannaCry_Ransomware_S508C.pdf, 2023\.  Accessed: 2023-06-5.
  * [32] Zilong Lin, Yong Shi, and Zhi Xue.  Idsgan: Generative adversarial networks for attack generation against intrusion detection.  In Advances in Knowledge Discovery and Data Mining: 26th Pacific-Asia Conference, PAKDD 2022, Chengdu, China, May 16–19, 2022, Proceedings, Part III, pages 79–91. Springer, 2022.
  * [33] Giovanni Apruzzese, Mauro Conti, and Ying Yuan.  Spacephish: The evasion-space of adversarial attacks against phishing website detectors using machine learning.  In Proceedings of the 38th Annual Computer Security Applications Conference, pages 171–185, 2022.
  * [34] Weiwei Hu and Ying Tan.  Generating adversarial malware examples for black-box attacks based on gan.  In Data Mining and Big Data: 7th International Conference, DMBD 2022, Beijing, China, November 21–24, 2022, Proceedings, Part II, pages 409–423. Springer, 2023.
  * [35] Cybersecurity Infrastructure & Security Agency, petya ransomware.  https://www.cisa.gov/news-events/alerts/2017/07/01/petya-ransomware, 2023\.  Accessed: 2023-05-28.
  * [36] Cybersecurity Infrastructure & Security Agency, crashoverride malware.  https://www.cisa.gov/news-events/alerts/2017/06/12/crashoverride-malware, 2023\.  Accessed: 2023-06-12.
  * [37] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [38] Darkreading, sidewinder strikes victims in pakistan, turkey in multiphase polymorphic attack.  https://www.darkreading.com/attacks-breaches/sidewinder-strikes-victims-pakistan-turkey-multiphase-polymorphic-attack, 2023\.  Accessed: 2023-06-12.
  * [39] Ursnif.  https://attack.mitre.org/software/S0386/, 2023.  Accessed: 2023-06-12.
  * [40] Cybersecurity Infrastructure & Security Agency aaeh.  https://www.cisa.gov/news-events/alerts/2015/04/09/aaeh, 2023.  Accessed: 2023-06-12.
  * [41] Yaser Alosefer.  Analysing web-based malware behaviour through client honeypots.  PhD thesis, Cardiff University, 2012.
  * [42] Ravi Chauhan and Shahram Shah Heydari.  Polymorphic adversarial ddos attack on ids using gan.  In 2020 International Symposium on Networks, Computers and Communications (ISNCC), pages 1–6. IEEE, 2020.
  * [43] Ravi Chauhan, Ulya Sabeel, Alireza Izaddoost, and Shahram Shah Heydari.  Polymorphic adversarial cyberattacks using wgan.  Journal of Cybersecurity and Privacy, 1(4):767–792, 2021.
  * [44] Neil C Rowe.  Deception in defense of computer systems from cyber attack.  In Cyber Warfare and Cyber Terrorism, pages 97–104. IGI global, 2007.
  * [45] Emotet.  https://attack.mitre.org/versions/v7/software/S0367/, 2023.  Accessed: 2023-06-12.
  * [46] Liang Wu, Fred Morstatter, Kathleen M Carley, and Huan Liu.  Misinformation in social media: definition, manipulation, and detection.  ACM SIGKDD explorations newsletter, 21(2):80–90, 2019.
  * [47] Twitter.  https://twitter.com/, 2023.  Accessed: 2023-06-12.
  * [48] Facebook.  https://www.facebook.com/, 2023.  Accessed: 2023-06-12.
  * [49] Reddit.  https://www.reddit.com/, 2023.  Accessed: 2023-06-12.
  * [50] KP Kumar and G Geethakumari.  Detecting misinformation in online social networks using cognitive psychology.  Human-centric Computing and Information Sciences, 4(1):1–22, 2014\.
  * [51] Steve Forbes.  Web of deception: Misinformation on the Internet.  Information Today, Inc., 2002.
  * [52] Priyanka Ranade, Aritran Piplai, Sudip Mittal, Anupam Joshi, and Tim Finin.  Generating fake cyber threat intelligence using transformer-based models.  In 2021 International Joint Conference on Neural Networks (IJCNN), pages 1–9. IEEE, 2021.
  * [53] Nitika Khurana, Sudip Mittal, Aritran Piplai, and Anupam Joshi.  Preventing poisoning attacks on ai based threat intelligence systems.  In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6. IEEE, 2019.
  * [54] Elisa Bertino, Murat Kantarcioglu, Cuneyt Gurcan Akcora, Sagar Samtani, Sudip Mittal, and Maanak Gupta.  Ai for security and security for ai.  In Proceedings of the Eleventh ACM Conference on Data and Application Security and Privacy, pages 333–334, 2021.
  * [55] Caitlin Moroney, Evan Crothers, Sudip Mittal, Anupam Joshi, Tülay Adalı, Christine Mallinson, Nathalie Japkowicz, and Zois Boukouvalas.  The case for latent variable vs deep learning methods in misinformation detection: an application to covid-19.  In Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada, October 11–13, 2021, Proceedings 24, pages 422–432. Springer, 2021.
  * [56] Deepswap.  https://www.deepswap.ai/, 2023.  Accessed: 2023-06-12.
  * [57] Woombo.  https://www.wombo.ai/, 2023.  Accessed: 2023-06-12.
  * [58] Instagram-DeepFake-Bot.  https://github.com/dome272/Instagram-DeepFake-Bot, 2023.  Accessed: 2023-06-12.
  * [59] Sun Tzu.  The art of war.  Standard Ebooks, 2020.
  * [60] Hyrum S Anderson, Jonathan Woodbridge, and Bobby Filar.  Deepdga: Adversarially-tuned domain generation and detection.  In Proceedings of the 2016 ACM workshop on artificial intelligence and security, pages 13–21, 2016.
  * [61] Seyed Mohammad Ghaffarian and Hamid Reza Shahriari.  Software vulnerability analysis and discovery using machine-learning and data-mining techniques: A survey.  ACM Computing Surveys (CSUR), 50(4):1–36, 2017.
  * [62] Dongzhe Lu, Jinlong Fei, Long Liu, and Zecun Li.  A gan-based method for generating sql injection attack samples.  In 2022 IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), volume 10, pages 1827–1833. IEEE, 2022.
  * [63] Xueqin Zhang, Yue Zhou, Songwen Pei, Jingjing Zhuge, and Jiahao Chen.  Adversarial examples detection for xss attacks based on generative adversarial networks.  IEEE Access, 8:10989–10996, 2020.
  * [64] Yisroel Mirsky, Ambra Demontis, Jaidip Kotak, Ram Shankar, Deng Gelei, Liu Yang, Xiangyu Zhang, Maura Pintor, Wenke Lee, Yuval Elovici, et al.  The threat of offensive ai to organizations.  Computers & Security, page 103006, 2022.
  * [65] Ahmadreza Azizi, Ibrahim Asadullah Tahmid, Asim Waheed, Neal Mangaokar, Jiameng Pu, Mobin Javed, Chandan K Reddy, and Bimal Viswanath.  T-miner: A generative approach to defend against trojan attacks on dnn-based text classification.  arXiv preprint arXiv:2103.04264, 2021.
  * [66] Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg.  Badnets: Identifying vulnerabilities in the machine learning model supply chain.  arXiv preprint arXiv:1708.06733, 2017.
  * [67] Yingqi Liu, Shiqing Ma, Yousra Aafer, Wen-Chuan Lee, Juan Zhai, Weihang Wang, and Xiangyu Zhang.  Trojaning attack on neural networks.  2017\.
  * [68] Kevin Alex Zhang, Alfredo Cuesta-Infante, Lei Xu, and Kalyan Veeramachaneni.  Steganogan: High capacity image steganography with gans.  arXiv preprint arXiv:1901.03892, 2019.
  * [69] Tao Liu, Zihao Liu, Qi Liu, Wujie Wen, Wenyao Xu, and Ming Li.  Stegonet: Turn deep neural network into a stegomalware.  In Annual Computer Security Applications Conference, pages 928–938, 2020.
  * [70] Zhi Wang, Chaoge Liu, Xiang Cui, Jie Yin, and Xutong Wang.  Evilmodel 2.0: bringing neural network models into malware attacks.  Computers & Security, 120:102807, 2022.
  * [71] Zhi Wang, Chaoge Liu, Xiang Cui, Jie Yin, Jiaxi Liu, Di Wu, and Qixu Liu.  Deepc2: Ai-powered covert command and control on osns.  In Information and Communications Security: 24th International Conference, ICICS 2022, Canterbury, UK, September 5–8, 2022, Proceedings, pages 394–414. Springer, 2022.
  * [72] Carlos Novo and Ricardo Morla.  Flow-based detection and proxy-based evasion of encrypted malware c2 traffic.  In Proceedings of the 13th ACM Workshop on Artificial Intelligence and Security, pages 83–91, 2020.
  * [73] Yansheng Qu, Jianfei Chen, Ming Li, Yunxiao Wang, Ning Li, Hua Huang, and Bo Mao.  Abnormal user behavior generation based on dcgan in zero trust network.  Procedia Computer Science, 214:1500–1505, 2022.
  * [74] Ehab Al-Shaer, Qi Duan, and Jafar Haadi Jafarian.  Random host mutation for moving target defense.  In Security and Privacy in Communication Networks: 8th International ICST Conference, SecureComm 2012, Padua, Italy, September 3-5, 2012\. Revised Selected Papers 8, pages 310–327. Springer, 2013.
  * [75] Alexander Geiger, Dongyu Liu, Sarah Alnegheimish, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni.  Tadgan: Time series anomaly detection using generative adversarial networks.  In 2020 IEEE International Conference on Big Data (Big Data), pages 33–43. IEEE, 2020.
  * [76] Md Abul Bashar and Richi Nayak.  Tanogan: Time series anomaly detection with generative adversarial networks.  In 2020 IEEE Symposium Series on Computational Intelligence (SSCI), pages 1778–1785. IEEE, 2020.
  * [77] Dan Li, Dacheng Chen, Baihong Jin, Lei Shi, Jonathan Goh, and See-Kiong Ng.  Mad-gan: Multivariate anomaly detection for time series data with generative adversarial networks.  In Artificial Neural Networks and Machine Learning–ICANN 2019: Text and Time Series: 28th International Conference on Artificial Neural Networks, Munich, Germany, September 17–19, 2019, Proceedings, Part IV, pages 703–716. Springer, 2019.
  * [78] Hossein Shirazi, Shashika R Muramudalige, Indrakshi Ray, and Anura P Jayasumana.  Improved phishing detection algorithms using adversarial autoencoder synthesized data.  In 2020 ieee 45th conference on local computer networks (lcn), pages 24–32. IEEE, 2020.
  * [79] Fatima Zahra Qachfar, Rakesh M Verma, and Arjun Mukherjee.  Leveraging synthetic data and pu learning for phishing email detection.  In Proceedings of the Twelfth ACM Conference on Data and Application Security and Privacy, pages 29–40, 2022.
  * [80] Zhenlong Yuan, Yongqiang Lu, and Yibo Xue.  Droiddetector: android malware characterization and detection using deep learning.  Tsinghua Science and Technology, 21(1):114–123, 2016.
  * [81] Muhammad Amin, Babar Shah, Aizaz Sharif, Tamleek Ali, Ki-Il Kim, and Sajid Anwar.  Android malware detection through generative adversarial networks.  Transactions on Emerging Telecommunications Technologies, 33(2):e3675, 2022.
  * [82] Tiziano Fagni, Fabrizio Falchi, Margherita Gambini, Antonio Martella, and Maurizio Tesconi.  Tweepfake: About detecting deepfake tweets.  Plos one, 16(5):e0251415, 2021.
  * [83] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692, 2019.
  * [84] David Ifeoluwa Adelani, Haotian Mai, Fuming Fang, Huy H Nguyen, Junichi Yamagishi, and Isao Echizen.  Generating sentiment-preserving fake online reviews using neural language models and their human-and machine-based detection.  In Advanced Information Networking and Applications: Proceedings of the 34th International Conference on Advanced Information Networking and Applications (AINA-2020), pages 1341–1354. Springer, 2020.
  * [85] Intel Introduces Real-Time Deepfake Detector.  https://www.intel.com/content/www/us/en/newsroom/news/intel-introduces-real-time-deepfake-detector.html#gs.1afjyz, 2023\.  Accessed: 2023-06-22.
  * [86] John Kotheimer, Kyle OMeara, and Deana Shick.  Using honeynets and the diamond model for ics threat analysis.  Technical report, CARNEGIE-MELLON UNIV PITTSBURGH PA PITTSBURGH United States, 2016.
  * [87] Sudip Mittal, Prajit Kumar Das, Varish Mulwad, Anupam Joshi, and Tim Finin.  Cybertwitter: Using twitter to generate alerts for cybersecurity threats and vulnerabilities.  In 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 860–867. IEEE, 2016.
  * [88] Aditya Pingle, Aritran Piplai, Sudip Mittal, Anupam Joshi, James Holt, and Richard Zak.  Relext: Relation extraction using deep learning approaches for cybersecurity knowledge graph improvement.  In Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pages 879–886, 2019.
  * [89] Aritran Piplai, Sudip Mittal, Anupam Joshi, Tim Finin, James Holt, and Richard Zak.  Creating cybersecurity knowledge graphs from malware after action reports.  IEEE Access, 8:211691–211703, 2020.
  * [90] Jonathan Voris, Jill Jermyn, Nathaniel Boggs, and Salvatore Stolfo.  Fox in the trap: Thwarting masqueraders via automated decoy document deployment.  In Proceedings of the Eighth European Workshop on System Security, pages 1–7, 2015.
  * [91] Ondrej Lukas and Sebastian Garcia.  Deep generative models to extend active directory graphs with honeypot users.  arXiv preprint arXiv:2109.06180, 2021.
  * [92] Febrian Setianto, Erion Tsani, Fatima Sadiq, Georgios Domalis, Dimitris Tsakalidis, and Panos Kostakos.  Gpt-2c: A parser for honeypot logs using large pre-trained language models.  In Proceedings of the 2021 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pages 649–653, 2021.
  * [93] Husnain Rafiq, Nauman Aslam, Biju Issac, and Rizwan Hamid Randhawa.  An investigation on fragility of machine learning classifiers in android malware detection.  In IEEE INFOCOM 2022-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), pages 1–6. IEEE, 2022.
  * [94] Rahim Taheri, Reza Javidan, Mohammad Shojafar, P Vinod, and Mauro Conti.  Can machine learning model with static features be fooled: an adversarial machine learning approach.  Cluster computing, 23:3233–3253, 2020.
  * [95] Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami.  Distillation as a defense to adversarial perturbations against deep neural networks.  In 2016 IEEE symposium on security and privacy (SP), pages 582–597. IEEE, 2016.
  * [96] Chwee Seng Choo, Ching Lian Chua, and Su-Han Victor Tay.  Automated red teaming: a proposed framework for military application.  In Proceedings of the 9th annual conference on Genetic and evolutionary computation, pages 1936–1942, 2007.
  * [97] Sheetal Temara.  Maximizing penetration testing success with effective reconnaissance techniques using chatgpt.  2023\.
  * [98] Subash Neupane, Jesse Ables, William Anderson, Sudip Mittal, Shahram Rahimi, Ioana Banicescu, and Maria Seale.  Explainable intrusion detection systems (x-ids): A survey of current methods, challenges, and opportunities.  IEEE Access, 10:112392–112415, 2022.
  * [99] Jesse Ables, Thomas Kirby, William Anderson, Sudip Mittal, Shahram Rahimi, Ioana Banicescu, and Maria Seale.  Creating an explainable intrusion detection system using self organizing maps.  arXiv preprint arXiv:2207.07465, 2022.
  * [100] Thomas Michael Kirby.  Pruning GHSOM to create an explainable intrusion detection system.  PhD thesis, Mississippi State University, 2023.
  * [101] Subash Neupane, Ivan A Fernandez, Wilson Patterson, Sudip Mittal, Milan Parmar, and Shahram Rahimi.  Twinexplainer: Explaining predictions of an automotive digital twin.  arXiv preprint arXiv:2302.00152, 2023.
  * [102] Francesco Greco, Giuseppe Desolda, and Andrea Esposito.  Explaining phishing attacks: An xai approach to enhance user awareness and trust.  In Proc. of the Italian Conference on CyberSecurity (ITASEC ‘23), 2023.
  * [103] Martina Morcos, Hussam Al Hamadi, Ernesto Damiani, Sivaprasad Nandyala, and Brian McGillion.  A surrogate-based technique for android malware detectors’ explainability.  In 2022 18th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), pages 112–117. IEEE, 2022\.
  * [104] Gil Fidel, Ron Bitton, and Asaf Shabtai.  When explainability meets adversarial learning: Detecting adversarial examples using shap signatures.  In 2020 international joint conference on neural networks (IJCNN), pages 1–8. IEEE, 2020.
  * [105] Huiling Li, Jun Wu, Hansong Xu, Gaolei Li, and Mohsen Guizani.  Explainable intelligence-driven defense mechanism against advanced persistent threats: A joint edge game and ai approach.  IEEE Transactions on Dependable and Secure Computing, 19(2):757–775, 2021.
  * [106] Valeria Cardellini, Emiliano Casalicchio, Stefano Iannucci, Matteo Lucantonio, Sudip Mittal, Damodar Panigrahi, and Andrea Silvi.  irs-partition: An intrusion response system utilizing deep q-networks and system partitions.  SoftwareX, 19:101120, 2022.
  * [107] Aritran Piplai, Priyanka Ranade, Anantaa Kotal, Sudip Mittal, Sandeep Nair Narayanan, and Anupam Joshi.  Using knowledge graphs and reinforcement learning for malware analysis.  In 2020 IEEE International Conference on Big Data (Big Data), pages 2626–2633. IEEE, 2020.
  * [108] Andy Applebaum, Camron Dennler, Patrick Dwyer, Marina Moskowitz, Harold Nguyen, Nicole Nichols, Nicole Park, Paul Rachwalski, Frank Rau, Adrian Webster, et al.  Bridging automated to autonomous cyber defense: Foundational analysis of tabular q-learning.  In Proceedings of the 15th ACM Workshop on Artificial Intelligence and Security, pages 149–159, 2022.
  * [109] Thanh Thi Nguyen and Vijay Janapa Reddi.  Deep reinforcement learning for cyber security.  IEEE Transactions on Neural Networks and Learning Systems, 2021\.