# CVE-LLM : Automatic vulnerability evaluation in medical device industry
using large language models

Rikhiya Ghosh<EMAIL_ADDRESS>,  OlaDimeji Farri  ,
Hans-Martin von Stockhausen  ,  George Marica Vasile  and  Martin Schmitt

###### Abstract.

The healthcare industry is currently experiencing an unprecedented wave of
cybersecurity attacks, impacting millions of individuals (Center, 2024). With
the discovery of thousands of vulnerabilities each month, there is a pressing
need to drive the automation of vulnerability assessment processes for medical
devices, facilitating rapid mitigation efforts. Generative AI systems have
revolutionized various industries, offering unparalleled opportunities for
automation and increased efficiency(Naveed et al., 2023). This paper presents
a solution leveraging Large Language Models (LLMs) to learn from historical
evaluations of vulnerabilities for the automatic assessment of vulnerabilities
in the medical devices industry. This approach is applied within the portfolio
of a single manufacturer, taking into account device characteristics,
including existing security posture and controls. The primary contributions of
this paper are threefold. Firstly, it provides a detailed examination of the
best practices for training a vulnerability Language Model (LM) in an
industrial context. Secondly, it presents a comprehensive comparison and
insightful analysis of the effectiveness of Language Models in vulnerability
assessment. Finally, it proposes a new human-in-the-loop framework to expedite
vulnerability evaluation processes.

Large Language Model, Vulnerability evaluation

††ccs: Security and privacy††ccs: Security and privacy Software and
application security††ccs: Security and privacy Domain-specific security and
privacy architectures††ccs: Computing methodologies Natural language
generation††ccs: Security and privacy Vulnerability management

##  1\. Introduction

A software vulnerability is a weakness or flaw in a software system that can
be exploited by attackers to compromise the integrity, availability, or
confidentiality of the system or the data it processes. These vulnerabilities
can exist at various levels of a software system – code, design,
configuration, or operation of the software. Examples of software
vulnerabilities include buffer overflows, SQL injection, cross-site scripting
(XSS), and insecure deserialization. There are several regulatory authorities,
non-profits and companies that regularly update the growing list of software
vulnerabilities. The most prominent of these organizations include MITRE
corporation 111https://www.mitre.org/ which, in conjunction with National
Institute of Standards and Technology (NIST) and National Vulnerability
Database (NVD) 222https://nvd.nist.gov/, maintain the Common Vulnerabilities
and Exposures (CVE)(Mitre, 2005) database. The CVE database is a system for
uniquely identifying and cataloging known vulnerabilities in software and
hardware. Each CVE entry includes a standardized identifier (e.g.,
CVE-2022-12345), a brief description of the vulnerability, and any relevant
references or links to additional information. The goal of CVE is to provide a
common language for discussing vulnerabilities and to facilitate the sharing
of information about them among security professionals, vendors, and the
general public. There are around 250K vulnerabilities at present, and it grows
by thousands every month.

Medical devices are prone to various security threats that can compromise
patient safety, privacy, the integrity of medical data, and the availability
of diagnostic or treatment devices. Some common vulnerability threats include
(a) software vulnerabilities, like buffer overflow, injection attacks,
authentication bypass etc,  (b) issues with proper encryption,  (c) weak
authentication measures,  (d) third party component-related vulnerabilities,
(e) use of unsupported legacy systems that are vulnerable but unpatchable,
(f) insecurely implemented/operated wireless connectivity such as missing
authentication and weak encryption, and  (g) lack of adequate security
controls.  Recent studies(Center, 2024) show that healthcare industry is the
most vulnerable sector with 809 data compromises and 56 million victims in
2023, which is 136% increase from 2022. Additionally, healthcare data breach
cost is extremely high, estimated to be 4.35 million USD by 2022. With the
fast growing number of vulnerabilities and the threats that the healthcare
industry faces, there needs to be faster and automated detection and
mitigation of vulnerabilities. Medical Device Manufacturers (MDMs) are
required by FDA to analyze all vulnerabilities. Most vulnerability assessment
procedures being manual, this leads to delays in processing the
vulnerabilities that pose actual threats to the systems.

One way of describing the severity of a CVE is using the Common Vulnerability
Scoring System (CVSS)(Mell et al., 2006), developed and maintained by Forum of
Incident Response and Security Teams (FIRST)333https://www.first.org/. CVSS
describes the severity of a CVE as a single vector that is created by several
metrices describing the exploitability of the vulnerability, the impact on a
given system, the current confirmation level and available fixes, and any
aspects of the environment that may be available to contain the CVE. The CVSS
metrics are divided into Base, Temporal and Environmental metrics. Base
metrics reflect the intrinsic characteristics of a vulnerability and do not
change over time or based on the environment in which the vulnerability
exists. The temporal metric describes aspects of the vulnerability that may
change over time, e.g. whether it is confirmed by the vendor, a fix is
available, or whether it is being actively exploited in the wild. The
environmental metrics is used to adapt the severity of a CVE based on the
local CIA requirements (Confidentiality, Integrity, and Availability) or
existing controls of a product/system in which the CVE is deployed. This might
either be a 3rd party component inside of a product or a product inside of an
operator’s environment. Vulnerability management involves the three-step
approach of detection, evaluation, and mitigation. In the context of this
paper we discuss the evaluation of vulnerabilities in 3rd party components
embedded in medical devices. Detection is performed by the vendors of those
components which afterwards publish a security notification about it. Medical
device manufacturers must monitor for those notifications and perform the
evaluation of the component related CVE in the context of each product the
component is used in. Mitigation can either be performed by the component
vendor by providing a patch that is subsequently applied to the medical device
or by the medical device manufacturer by adding controls to the product that
contain the CVE. In fortunate scenarios, the CVE of the component does not
pose a risk to the medical device and hence no mitigation is necessary but
only information from the MDM towards their customers. The approaches to
automatic vulnerability assessment involves use of machine learning
methods(Prasad et al., 2018; Iorga et al., 2020; Levshun, 2023) and more
recently, Large Language Models (LLMs)(Yin and Ni, 2024; Lu et al., 2024;
Steenhoek et al., 2024).

Large Language Models are a class of machine learning models that are capable
of understanding and generating human language. These models are typically
based on transformer architectures and are trained on large amounts of text
data to learn the statistical patterns and structures of natural language.
Since the release of ChatGPT(OpenAI, 2021), use of LLMs has been ubiquitous
and has been adapted for Healthcare(Sallam, 2023), IT(Wu et al., 2023),
Cybersecurity(Zhang et al., 2024a), Finance(Li et al., 2023), Legal
applications(Yang et al., 2024), content creation(Moore et al., 2023) and
every conceivable text-based application. The most commonly used paradigms of
LLM integration in applications have been prompt engineering and retrieval-
augmented generation (RAG)(Lewis et al., 2020). Domain-specific applications
also employ methods like domain adapted pretraining (DAPT)(Ge et al., 2024),
supervised finetuning (SFT)(von Werra et al., 2020), ensemble methods(Lu et
al., 2023), knowledge integration(Feng et al., 2023), transfer
learning(Chronopoulou et al., 2019), data augmentation(Whitehouse et al.,
2023), reinforcement learning(Lee et al., 2023) to achieve higher performance
with their respective datasets. Some of the most common applications of LLMs
in cybersecurity are vulnerability detection (Steenhoek et al., 2024),
automated program repair(Zhou et al., 2024), secure code generation(Fu et al.,
2024), binary code-based similarity and recovery(Tan et al., 2024),
cybersecurity assistants (Zhang et al., 2024a), and infrastructure
maintenance(Naito et al., 2023a). LLMs and vulnerability management are
intricately linked, with LLMs playing vital roles in detecting, repairing, and
mitigating vulnerabilities, and vulnerability management protocols help in
assessing and mitigating security risks in LLM-powered applications(Yao et
al., 2024). The evolution of advanced LLM-based reasoning techniques has
spurred significant research into using LLMs to rectify vulnerabilities in
code and provide explanations for vulnerability detection and repair
processes.

Use of LLMs for vulnerability assessment mostly focuses on detection of
vulnerabilities using source code(Sun et al., 2024). Recent work on
vulnerability assessment using source code has shown the limitations that LLMs
face in assessing vulnerability severity based on source code only, and report
increase in performance with descriptive text and context information.
Cybersecurity researchers have envisaged using a LLM which is finetuned with
asset information and vulnerability-related data held by an organization to
output an effective penetration test attack path (Naito et al., 2023b).
However, there is a dearth of real-world experiments demonstrating the
effectiveness of employing LLMs for vulnerability management. This has
prompted us to develop a large language model that is trained using historical
evaluation data containing the description of an asset and a vulnerability to
perform detailed evaluation of the impact of the vulnerability on the asset.
This would aid in fast detection and assessment of applicable vulnerabilities
which in turn would help in faster mitigation efforts.

In this paper, we have developed a LLM that would generate vulnerability
assessments in the form of structured vectors and categories as well as
detailed description of how the vulnerability impacts the system for different
stakeholders. To the best of our knowledge, this is the first paper that
explores using LLMs for vulnerability assessment based solely on asset and
vulnerability descriptions in medical technology industry setting. We have
also benchmarked our model with respect to other finetuned open source LLMs
and offer insights into the best practices to train a vulnerability LLM.
Finally, we have proposed a human-in-the-loop vulnerability assessment system
that drastically reduces the time taken to assess each vulnerability, which in
turn could benefit cybersecurity for medical devices.

##  2\. Problem Statement

Starting with the US Food and Drug Adminstration’s (FDA) Guidance on
Postmarket management of Cybersecurity in medical devices (of Health et al.,
2016). MDMs are required to monitor all 3rd party software components used in
medical devices for emerging vulnerabilities and evaluate them towards whether
they have a negative impact on the safety and security for continued clinical
use of that device. Based on the size of the product portfolio and the number
of 3rd party components utilized, the amount of device specific evaluations to
perform can be overwhelming for an MDM if they are performed manually and
given the development of the amount of reported vulnerabilities over the last
years as described in the previous chapter. Automation of evaluations is seen
as an potent measure to cope with the amount of evaluations efficiently and
achieving a good quality level. Furthermore, those evaluations need to be
performed over the complete post-market lifetime of a device which might be up
to 20 years and during which the knowledgeable experts that can perform
evaluations quickly may no longer be available in the same position for a
given product or even in the same company.

Vulnerability evaluation in our purview is determination of the following
assessments, given the details of an asset and a notification.

  1. (1)

VEXCategory: Categorization of the vulnerability to signify whether the asset
is  (a) affected,  (b) not affected  by the notification.

  2. (2)

VEXJustification: This explains the VEXCategory further in case the asset is
not affected by the notification.

  3. (3)

Vector: This refers to the CVSS environmental metrics.

  4. (4)

Internal Comment: This contains a description of the problem and a solution
advised for internal use.

  5. (5)

Customer Comment: This is a brief comment made about how the notification
affects the asset and the solution, and is directed towards the customer.

An asset detail typically contains organization-wide details of the asset,
including the list of 3rd party components used by the asset. Notification
details comprises of details of one or more vulnerabilities that constitute
the notification and the components affected by it. The largest efficiency
gain is in automatically identifying CVEs that do NOT apply to the product
together with VEX based reasoning. Newly emerged CVEs that apply to a product
need a first time human investigation due to the absence of design information
in today’s LLM.

##  3\. Relevant Work

Language models (LMs) have been a pivotal area of research in natural language
processing (NLP), driving significant advancements in various NLP tasks. The
large data revolution in NLP was introduced by Transformer architecture
(Devlin et al., 2018) by enabling parallel computation and efficient modeling
of long-range dependencies in text. The architecture is constituted by an
encoder-decoder design, featuring multiple layers of both self-attention and
feedforward neural networks. In recent years, advancements in high-performance
computing, scalability, architecture, and data availability have facilitated
the evolution from models like BERT (Devlin et al., 2018) and GPT (Radford et
al., 2018) to more sophisticated iterations such as GPT-4 (Achiam et al.,
2023), Llama3 (Touvron et al., 2023), Mixtral (Jiang et al., 2024), and other
powerful Large Language Models (LLMs). These models exhibit remarkable
performance not only in text-based systems but also in multimodal domains.
Language Models have been used extensively in vulnerability management  (a) to
determine CVSS metrics from CVE description,  (b) to establish a mapping
between vulnerabilities in CVE database with the corresponding attack tactics
and techniques in ATT&CK database (Corporation, 2024a),  (c) for vulnerability
detection, and  (d) for vulnerability repair.

CVE description to CVSS metrics mapping includes both mapping to CVSS vectors
and CVSS score. Mapping to CVSS score follows a linear regression using Bag-
of-Words model (Elbaz et al., 2020) or neural network model using Doc2Vec
(Vasireddy et al., 2023) method of extracting features from a document. CVSS-
BERT (Shahid and Debar, 2021), on the other hand, trained different BERT
models for classification of CVE descriptions to the values of different CVE
vectors. Both encoder and decoder based models have been used for
Vulnerability to ATT&CK tactics and techniques. CVET (Ampel et al., 2021), a
RoBERTa(Liu et al., 2019)-based model, classified CVE descriptions to one of
the ten tactics in ATT&CK, whereas SMET (Abdeen et al., 2023) used BERT-based
textual similarity to map CVE entries to ATT&CK techniques. Though ChatGPT-
based approaches did not yield state-of-the-art results (Liu et al., 2023),
VTT-LLM (Zhang et al., 2024b), trained using various versions of the decoder
Bloom model(Le Scao et al., 2023) with Chain-of-thought (Wei et al., 2022)
instructions, incorporated relations between core concepts in CWE (Common
Weakness Enumeration) (Christey et al., 2013) and CAPEC (common attack
patterns) databases (Corporation, 2024b) for CVE to ATT&CK mapping surpassed
encoder based models. Related work (Yosifova et al., 2021) also involves CVE
Vulnerability type classification using TF-IDF and standard machine learning
classifier.

In the domain of vulnerability detection, encoder-based LLMs (Ameri et al.,
2021; Yin et al., 2020) have been instrumental, particularly through
innovative approaches such as the pretrain-and-finetune paradigm. These
approaches have also leveraged specialized methods including novel pretraining
strategies, the integration of other deep learning paradigms like Graph Neural
Networks(Sewak et al., 2023) or Long Short-Term Memory (LSTM)(Hassanin et al.,
2024) networks, prompt tuning, program analysis, specialized calculus-based
causal reasoning, and knowledge graph-based reasoning. The focus has
increasingly shifted towards decoder-only LLMs(Zhou et al., 2024), where
prompt tuning, finetuning, and retrieval-augmented generation systems are
heavily employed for vulnerability detection.

Conversely, vulnerability repair systems primarily utilize decoder LLMs(Pearce
et al., 2023), often relying on finetuning, transfer learning, retrieval
augmented generation, or reinforcement learning-based systems (Islam et al.,
2024). These systems typically operate on diverse datasets and have witnessed
the emergence of innovative transformer architectures and training techniques.
Furthermore, recent advancements in the field have also explored the
combination of these techniques with ensemble learning methods, meta-learning
approaches, and the incorporation of domain-specific knowledge for enhanced
vulnerability management in complex software systems.

##  4\. Methodology

We define the vulnerability assessment problem as follows: given an asset A
and notification N, an evaluation is generated.

$\left(A,N\right)\rightarrow\mathsf{evaluation}$

An evaluation can be a VEXCategory, CVSS environmental metrics, Internal
comment, or Customer Comment. VEXCategory is generated along with
VEXJustification, which further elucidates the categorisation of the
vulnerability.

$\displaystyle\mathsf{\textsc{VEXCategory}}\in$
$\displaystyle\left\\{\mathsf{Affected,NotAffected}\right\\}$
$\displaystyle\mathsf{\textsc{VEXJustification}}\in$
$\displaystyle\left\\{\mathsf{VulnerableComponentNotPresent},\right.$
$\displaystyle\left.\mathsf{ComponentNotPresent},\right.$
$\displaystyle\left.\mathsf{VulnerableCodeNotInExecutePath},\right.$
$\displaystyle\left.\mathsf{VulnerableCodeCannotBeControlledByAdversary},\right.$
$\displaystyle\left.\mathsf{InlineMitigationsAlreadyExist},\right.$
$\displaystyle\left.\mathsf{Other,N/A}\right\\}$

InternalComment and CustomerComment are text explanations of the vulnerability
assessments.

Our methodology follows the standard domain adaptation procedure for large
language models called domain adaptive pretraining (DAPT). We have used Mosaic
ML’s MPT-7B LLM(Team, 2023) for domain adaptation with a combination of
publicly available vulnerability-related documents and organization-wide
documents. We used human expert-derived evaluations for the standard
vulnerabilites for the organizational assets to form an instruction tuning
dataset that is used to train CVE-LLM.

Figure 1. Schematic of the Training paradigm

Training Schema

###  4.1. Datasets

####  4.1.1. DAPT Dataset

DAPT Dataset consists of 320K vulnerability description documents that are a
combination of publicly available documents (218K) from NVD, and organization-
wide vulnerability documents (102K). The organization-wide vulnerability
documents describe, to various details, combination of one or more
vulnerabilities and their effects and possible mitigation for the affected
product.

####  4.1.2. Instruction tuning Dataset

Instruction tuning dataset is formed from three organization-wide datasets:
(a) Assets,  (b) Notifications, and  (c) Evaluations.  Assets represent all
the different products released by the organization, and includes various
software versions of the products. The dataset consists of asset id, product
name and version, software name and version, sub-organization that the product
is associated with, and list of software components that constitute the asset.

Notifications dataset encompasses the set of all vulnerabilities that might
affect one or more assets and is generally the combination of one or more
CVEs. The Notifications dataset contains notification id, a description of the
notification, the software components that the notification affects, the base
and temporal vectors and scores for the notification, and the constituent
vulnerabilities (CVEs and CWEs) for the notification. The description of a
notification consists of descriptions of the constituent CVEs as well as
comments from security experts on the notification in the organization. There
are a small percentage of notifications that do not comprise of any CVEs, and
are descriptions of possible nonstandard vulnerability issues for assets.

An evaluation is an assessment of how a notification affects an asset. When a
notification is known to affect a component that constitutes an asset, an
evaluation is performed by a security expert who has thorough knowledge of the
asset. The Evaluations dataset consists of the asset id, notification id,
VEXCategory, VEXJustification, Internal comment, Customer Comment, Vector and
CVSS Score. VEXCategory represents expert assessment of whether the asset is
affected by the notification or not. Internal comment is the comment outlining
mitigation process if the product is affected or short description of why the
product is not affected. Customer Comment is a summarised version of Internal
Comment which is sent to customers of the organization’s products.
VEXJustification comes into purview only when VEXCategory is ‘Not Affected’,
and is a justification of why the product is not affected. It can be one of
the following categories:

  1. (1)

VulnerableCodeNotPresent

  2. (2)

ComponentNotPresent

  3. (3)

VulnerableCodeNotInExecutePath

  4. (4)

VulnerableCodeCannotBeControlledByAdversary

  5. (5)

InlineMitigationsAlreadyExist, and

  6. (6)

Other

Vector corresponds to the evaluated CVSS vector which consists of base vector,
temporal vector and environmental vector. CVSS score is calculated using the
CVSS vectors.

There are around 1.6K assets, 134K notifications and 174K evaluations at the
time of writing this paper. There are 152K unique components used by all the
assets. There are 11K unique notifications that are used by all the
evaluations. Most notification descriptions consist of less than 200 words,
most internal comments have less than 70 words and most customer comments are
less than 50 words 4. The instruction tuning dataset is formed from the
Evaluations dataset. An instruction in the dataset corresponds to an
evaluation and contains description of asset, description of notification, the
affected software components, and the corresponding assessments: VEXCategory,
VEXJustification, Internal Comment, Customer Comment, and Vector.

###  4.2. Training of DAPT model

DAPT model is trained by continuously pretraining the LLM using next token
generation. Since the quality of data is of utmost importance, we proceed with
careful preprocessing of DAPT dataset, and then pretraining the LLM with
preprocessed data.

####  4.2.1. Preprocessing

Preprocessing of the public dataset part of DAPT dataset consists of the
following steps:

  * •

Identification of attributes of the dataset that are used to form a data
point. A data point in the dataset refers to details of a CVE. The identified
attributes represent title and descriptions of the CVEs, base and temporal
vectors, affected product and software version, unaffected software version
(if any), and mitigation (if any).

  * •

Cleaning descriptions. The descriptions are preprocessed for removing URLs,
non-UTF8 characters, removing formatting if any. In case there are more than
one description in the CVE, and there is not more than 70% overlap between the
descriptions, they are merged into one longer description.

  * •

Template-based formation of vector description. The vectors are expanded to
form template-based description of the vectors. For a vector component that is
of the format ¡VECTOR_COMPONENT:value¿, the expanded vector description is
“¡VECTOR COMPONENT¿ is ¡value¿”.

  * •

Template-based combination of the data point attributes.The template for the
CVE detail is: “CVE description: ¡description¿. Affected product: ¡product¿
less than ¡highest affected software version¿. Unaffected version: ¡lowest
unaffected version¿ and higher. Vector: ¡vector description¿”

Preprocessing of the organization dataset consists of the following
procedures:

  * •

Identification of notifications in the Notifications dataset that consist of
descriptions with text other than the descriptions of the constituent CVEs.
Most notifications contain CVE descriptions from the public dataset followed
by expert comments on the notification. In addition, the nonstandard
notifications are also included in this dataset.

  * •

Cleaning description. This follows the same procedure of cleaning descriptions
as the public dataset preprocessing. In addition, the CVE ids are cleaned from
the description.

  * •

Template-based formation of vector description. Vector description is formed
in the similar fashion as the public dataset.

  * •

Template-based combination of notification attributes. The template for
notification detail is: “Notification description: ¡description¿. Vector:
¡vector description¿”

####  4.2.2. Model Training

The DAPT dataset is randomly split into 90:10 split with 288K training data
points and 32K validation data points. We have continuously pretrained MPT-7B
(Team, 2023) base model autoregressively for next token prediction with an
objective function of cross entropy with the DAPT dataset. We have expanded
the vocabulary of the MPT-7B base model to include names of the components and
the organization software. We have used only the first names of the components
and software and we had 539 new tokens to add to the 50K vocabulary of MPT.
The model is trained using DeepSpeed (Rasley et al., 2020) zero-3 optimization
with Lion optimizer (Chen et al., 2024) (learning rate: 1e-4, weight decay:
1e-2) for 3 epochs on 8 Tesla A100 SXM4 GPUs.

###  4.3. Training of Instruction-tuned model

Table 1. Instructions for each evaluation type Evaluation | Instruction | Response
---|---|---
VEXCategory/VEXJustification | What is the category? | ¡VEXJustification ¿ Category: ¡VexCategory¿
Internal Comment | Generate internal comment. | ¡InternalComment¿
Customer Comment | Generate customer comment. | ¡CustomerComment¿
Vector | Generate environmental vectors. | ¡Evaluation vector description¿

Instruction tuning requires formation of an instruction tuning dataset in a
pre-defined format. We have used the instruction tuning dataset format of the
Alpaca dataset (Taori et al., 2023) to form our dataset. Then we have
finetuned the DAPT model for instruction tuning with the new dataset.

####  4.3.1. Formation of Instruction-tuning dataset

Formation of instruction-tuning dataset follows the following procedure:

  * •

Merging of Evaluations dataset with Assets and Notifications dataset based on
the Asset ID and Notification ID.

  * •

Cleaning of notification description. This follows similar procedure as
preprocessing of organization dataset.

  * •

Identifying components common to both Asset and Notification and forming a
list of descriptions of these common components.

  * •

Formation of vector descriptions for both notification vector and evaluation
vector. This follows the same vector text description format as described in
DAPT dataset formation section. For evaluation vector description formation,
we subtract the components already present in the notification vector from
evaluation vector components and form the evaluation vector description using
these new vector components only. In addition, the CVSS format version is also
extracted. In case CVSS format version is not specified, the vector components
are used to match to the corresponding vector components for CVSS versions 2,
3.0 and 3.1, and corresponding the highest version with all the component
matches is considered as the CVSS version. In case such version is not found,
we have used the help of security experts to correct the vector as well as get
the correct version number.

We obtain all the expected base, temporal and environmental vector components
using the CVSS version. In case of notification vector text, we find all
missing base and temporal vector components in the text and add them to
notification vector description in the format “¡VECTOR_COMPONENT¿ is XXXX”.
For evaluation vector, we find all missing environmental vector components in
the evaluation vector and add them to the evaluation vector text in the same
format as missing notification vector description. To be noted is that the
order of vectors in the description is fixed.

  * •

Formation of vector justification. We break up the vector justification
category into constituent words and use that as an explanation of VEXCategory.

  * •

Formation of instruction data. The instruction format uses the following
format:

Instruction dataset format
---
Below is an instruction that describes a task, paired with an input that
provides further context. Write a response that appropriately completes the
request. ### Instruction: ¡instruction¿ ### Input: Organization: ¡sub-
organization name¿ Software: ¡software name and version¿ Product: ¡asset name
and version¿ Notification: ¡cleaned notification description¿ Components
present in software: ¡list of common component description¿ Base and Temporal
Vectors: ¡notification vector description¿ CVSS Version: ¡version¿ ###
Response: ¡response¿ ¡STOP¿
Example
Below is an instruction that describes a task, paired with an input that
provides further context. Write a response that appropriately completes the
request. ### Instruction: Generate internal comments. ### Input: Organization:
DI-DnA Software: Syngo Carbon Monitoring Product: Syngo Carbon Monitoring
VB12A Notification: CoreMedia, WebRTC: Processing an image can cause an out-
of-bounds write and may lead to arbitrary code execution due to improper input
validation. Components present in software: Debian Package: dav1d - Debian -
All Versions Base and Temporal Vectors: Attack Vector is Adjacent. Attack
Complexity is High. Privileges Required is Low. User Interaction is None.
Scope is Unchanged. Confidentiality is Low. Integrity is High. Availability is
Low. Exploit Code Maturity is Unproven. Remediation Level is Official Fix.
Report Confidence is Confirmed. CVSS Version: 3.1 ### Response: CoreMedia,
WebRTC is not utilized in the product. ¡STOP¿
Table 2.  Instruction tuning dataset format and example. Certain data is
censored in keeping with organization policies.

The instructions and responses for the different types of evaluations are
shown in 1.

  * •

Removal of incomplete evaluation data, data under investigation, long text
data. In practice, VEXCategory also has two more classes: UnderInvestigation
and EndofLife. UnderInvestigation category is assigned to the evaluations for
which monitoring and/or mitigation measures are underway. EndofLife refers to
the assets that are being retired. These categories are flagged by the system
and we do not use the data to train our model. In addition, there are a few
data points identified that had incomplete evaluations. We also identified a
few data points for which the token count exceeds 1048 tokens, and we do not
use that data as well for training.

  * •

De-duplication of dataset. The dataset is de-duplicated by removing the data
points for which all software name, notification description and evaluations
are the same.

####  4.3.2. Model Training

The instruction tuning dataset has 440K instructions, and the dataset is
randomly split into train, test and validation datasets with a 80:10:10 split,
leading to 352k training data points, and 44K each of validation and test data
points. The DAPT model is finetuned with the training and validation data by
back propagating on completions only, with setting the token labels of the
instructions to -100. The model is trained using DeepSpeed zero-3 optimization
with Lion optimizer (learning rate: 1e-5, weight decay: 1e-2) for 3 epochs on
8 Tesla A100 SXM4 GPUs.

###  4.4. Inference

Figure 2. Inference and Model evaluation Schematic

Inference Schema

The inference framework includes generation of evaluations using the trained
LLM, using pre-defined rules to avoid easily identifiable erroneous
generations, and using faster inference frameworks to improve model
throughput. We have implemented a human-in-the-loop system where the AI-
generated vulnerability evaluations are validated by a product cybersecurity
expert with higher priority to the evaluations with Affected VEXCategory.

####  4.4.1. Generation of evaluations

Evaluation generation follows a zero shot model inference where the model is
provided a prompt in the pre-defined instruction format, containing
information about sub-organization name, software name and version, asset name
and version, notification description, list of component descriptions for
components present in both asset and notification, base and temporal versions
and CVSS version. The maximum generated tokens for
VEXCategory/VEXJustification generation is set to 25, Internal Comments is
125, Customer comments is 100, and Vectors is 100. The inference/test dataset
is divided into two sub-datasets based on the size of tokens in the inference
instruction. For instructions longer than 920 tokens, the sequence length of
the trained model is increased to 150 + maximum token length of the longest
instruction. If the size of the instruction is less than 920 tokens, the
trained model without any changes is used to inference the dataset. Since
MPT-7B is trained with ALIBI (Press et al., 2021) encodings, the maximum
sequence length during inference can be set upto 84K tokens, which has not
been observed in production yet.

####  4.4.2. Rule-based correction

Post-processing of model outputs involves using system knowledge in the form
of pre-defined rules to correct mistakes in model generation. The rules are as
follows:

    1. (1)

If VEXCategory value is NotAffected, the evaluation vector is not generated.

    2. (2)

If VEXJustification generated is any text other than the pre-defined
categories, ‘Other’ gets assigned to VEXJustification.

    3. (3)

If VEXJustification is ‘Other’ and Customer Comment is empty, then it is an
error in generation. We use Internal Comment as External Comment as well in
case it is not empty.

    4. (4)

If VEXCategory is Affected, VEXJustification is automatically set to None, and
it is ensured that the Vector has the proper CVSS format.

####  4.4.3. Model serving

Model serving is performed using vLLM architecture (Kwon et al., 2023) on one
Tesla A100 SXM4 GPU. Model throughput during production is improved using rule
based system where generation of Vector is skipped when VEXCategory is Not
Affected, and sequence length-based batching is performed.

##  5\. Results

Evaluation of our methodology follows a two-step process which evaluate the
model-generated results as well as the end-to-end results from the inference
pipeline. In addition to the test set which was set apart during training, we
have also conducted an evaluation of the model outputs in a real-time
deployment scenario over a period of two months for new assets and
vulnerabilities.

###  5.1. Evaluation Metrics

We use ROUGE-L and micro-F1 for evaluating the model responses: ROUGE-L(Lin
and Och, 2004) is used for evaluating the responses model generated for
Internal Comment and Customer Comment, whereas micro-F1(Pedregosa et al.,
2011) is used for VEXCategory, VEXJustification and Vector. ROUGE-L (Recall-
Oriented Understudy for Gisting Evaluation - Longest Common Subsequence) is a
metric used to evaluate the quality of summaries produced by automatic
summarization systems. ROUGE-L measures the longest common subsequence (LCS)
between the model-generated summary and the reference (human-generated)
summary. Micro F1, or micro-averaged F1 score, is calculated by taking the
harmonic mean of precision and recall, with equal weighting given to both
metrics.

###  5.2. Benchmarking CVE-LLM against other open source LLMs

We have used the open source models that are of similar size as CVE-LLM, and
are the highest performing models currently on LLM Leaderboards (Huggingface,
2023). We trained them using the DAPT and SFT methods described in Methodology
section. In addition, we have also performed RAG over our test dataset with
Llama3-70B. The Table 3 enumerates the results on our test datasets for these
experiments, as well as the performance of our system in production. Except
CVE-LLM-Prod, which is the system deployment results against 2K expert
evaluations over completely new notifications and assets, the rest are
evaluated on the test dataset with 44K evaluations.

Evaluation | Llama3-RAG | Mistral-7B | Llama2-7B | CVE-LLM-Eval | CVE-LLM-Eval* | CVE-LLM-Prod
---|---|---|---|---|---|---
VEXCategory | 0.25 | 0.64 | 0.56 | 0.93 | 0.93 | 0.83
VEXJustification | 0.12 | 0.6 | 0.53 | 0.89 | 0.92 | 0.95
Internal Comment | 0.19 | 0.71 | 0.62 | 0.8 | 0.8 | 0.7
Customer Comment | 0.28 | 0.7 | 0.64 | 0.88 | 0.89 | 0.75
Vector | 0.13 | 0.59 | 0.57 | 0.95 | 0.97 | 0.97
Table 3. Benchmarking CVE-LLM against state-of-the-art Open Source LLMs CVE-
LLM-Eval* = End-to-end CVE-LLM results, CVE-LLM-Prod = CVE-LLM-Production
results.

CVE-LLM, which is based on MPT-7B model, has shown better performance than
Llama2-7B and Mistral-7B-based models. The highest performance improvement is
observed for classification-based generations: VEXCategory, VEXJustification
and Vector. Post processing the model results have shown the highest
improvement for VEXJustification and Vector. We also observe that the
performance of CVE-LLM with completely new assets and notifications in
deployment mirrors the test dataset results.

###  5.3. Ablation Studies with the model

We conducted ablation studies with respect to different components of model
training and inference:

(a) Changes in dataset,  (b) Dataset size,  (c) DAPT-SFT vs SFT-only system,
and  (d) Impact of beam size, temperature, and nucleus sampling during model
inference, and  (e) Sequence length during training, and

For each of these experiments, the test dataset consists of the same data
points from the Evaluations dataset. The training data points have varied
across the datasets, but we have made sure that all the possible assets are
represented in the training datasets.

####  5.3.1. Changes in dataset

Evaluation | CI | CV | CWJ | WD | NP
---|---|---|---|---|---
VEXCategory | 0.87 | 0.94 | 0.85 | 0.66 | 0.93
VEXJustification | 0.86 | 0.89 | 0.85 | 0.51 | 0.89
Internal Comment | 0.7 | 0.79 | 0.8 | 0.66 | 0.8
Customer Comment | 0.79 | 0.81 | 0.88 | 0.62 | 0.88
Vector | 0.89 | 0.9 | 0.95 | 0.7 | 0.85
Table 4. CVE-LLM-Eval Model Ablation study on Test Dataset CI = All
Instructions Combined, CV = Customer Comment and VEXCategory/VEXJustification
combined, CWJ = VEXCategory and VEXJustification separated, SL = Sequence
length, WD=Without delimiter, NP = No Placeholder for unused Vector Categories

For each of these experiments, we trained CVE-LLM-base model on the
reconstructed dataset. We have tried the following changes in dataset:

    * –

Combining all the evaluations into one instruction (CI)

    * –

Combining customer comments, VEXCategory and VEXJustification into one type of
instruction (CV)

    * –

Separate generation of VEXJustification and VEXCategory (CWJ)

    * –

No delimiter at the end of training text (WD), and

    * –

No Placeholder for unused vector categories (NP)

CI, CV and CWJ are concerned with experiments to see whether we can combine
some of these instructions to achieve good results. WD refers to the
experiment where we see how the model performs when we do not put a dedicated
delimiter. NP is concerned with how the template for a Vector output should
look like. NP refers to the case where the response contains only the vectors
that are mentioned in the standard CVSS vector format. The training details of
this ablation study is shown in Table 5 and the test results are shown in
Table 4. These datasets vary in their sizes of their training, validation and
test datasets. For example, CI has only 88K instructions to train with due to
combining instructions, which CWJ has 440K instructions because all the
instructions are separated.

| CI | CV | CWJ | WD
---|---|---|---|---
$N_{Train}$ | 88K | 264K | 440K | 329K
$N_{Valid}$ | 11K | 33K | 55K | 44K
$N_{Test}$ | 11K | 33K | 55K | 44K
Epochs | 4 | 2 | 3 | 3
Table 5. Changes in Dataset: CVE-LLM-Eval Model Ablation study on Test Dataset
CI = All Instructions Combined, CV = Customer Comment and
VEXCategory/VEXJustification combined, CWJ = VEXCategory and VEXJustification
separated, WD=Without delimiter

We observe that the variation of Rouge-L score for Internal Comment is the
lowest, while the highest variation is seen for the classification-based
generations of VEXCategory and Vector. The model performs better with
diversity in instructions, and higher number of tokens in training. Size of
the dataset does not impact performance much by itself, but combined with
other factors may degrade performance. Spurious text generation is a problem
with LLM finetuning, and putting a delimiter enhances performance more than
changing model generation parameters. Vector performance improves by 0.1
micro-F1 with the introduction of fixed template-based generation for all
possible components of the environmental and temporal vector.

####  5.3.2. Effect of Dataset size

We have experimented with changing size of the instruction tuning training
dataset from 150K instructions to 440K instructions. The datasets are formed
by making sure that all the types of assets and all the types of notifications
are always represented in the dataset. The minimal dataset that affords this
invariability is the 150K training dataset, and for the rest of the datasets
we have randomly selected from the remaining data points.

(a) Effect of dataset size

(b) Effect of sequence length

(c) Effect of beam size

Figure 3. Ablation studies: Effect of various factors on model outcome

We have noticed that Customer Comment shows the least variation in Rouge-L
score, while VEXCategory shows the highest micro-F1 variance. In addition, the
performance improvement stagnates for most instructions as we reach the 440K
instruction dataset size mark.

####  5.3.3. Effect of Domain Adaptation

The results after Supervised Finetuning of Domain-adapted CVE-LLM-Base vs
Supervised Finetuning of MPT-7B is shown in Table 6. Domain adaptation and
vocabulary expansion leads to a much better performance for all the
instruction types, as shown in Table 6

Evaluation | CVE-LLM-Base | MPT-7B
---|---|---
VEXCategory | 0.93 | 0.83
VEXJustification | 0.89 | 0.8
Internal Comment | 0.8 | 0.64
Customer Comment | 0.88 | 0.69
Vector | 0.95 | 0.84
Table 6. Test dataset Results: Finetuning Domain adapted LLM CVE-LLM-Base vs
Finetuning with MPT-7B

####  5.3.4. Effect of Sequence Length in Training

In this experiment, we changed the sequence length in configuration of the
model to 512, 750 and 1024, and reduced the training dataset to accommodate
only the instructions that had less than the sequence length. Figure 3(b)
shows the effect of training sequence length on the model outcome for all the
instruction types. We find out that though the dataset size does not change
much, there is more variation in performance. As training sequence length
increases, the performance on datasets with longer instructions also improves.

####  5.3.5. Effect of inference parameters

In generative model inference, parameters such as temperature, beam size, and
nucleus sampling significantly influence the quality and diversity of the
generated outputs. Temperature controls the randomness of the output, with
higher values leading to more diverse results. Nucleus Sampling (top-p
sampling) dynamically adjusts the candidate set of tokens to balance between
diversity and coherence based on the cumulative probability threshold ‘p’.
Beam Size in beam search influences the depth of search for probable
sequences, balancing between computational cost and output quality. Generally
these parameters are adjusted to optimize the trade-off between diversity and
coherence of the generated text. We performed inference on test dataset
(N=44K) using CVE-LLM-Eval model to conduct the experiment on the effect of
inference parameters.

##### Temperature and nucleus sampling.

In order to test the effect of temperature, we maintained the generation
parameters at their default settings and conducted tests by varying the
temperature values from 0 to 1 in increments of 0.1 to evaluate the outcomes.
Similarly, for top-p sampling too, we changed the parameter ‘top-p’ from 0 to
1 in increments of 0.1, keeping all other generation parameters in default
settings. For both of the experiments, no changes were observed.

##### Beam size.

We measured the effect of beam size for each of the instruction types by
changing the beam size from 1 to 19 in increments of 2, maintaining other
generation parameters in their default values. We notice that beyond beam size
7, the performance of the system steadily declines for all the instruction
types. The effect of change of beam size on model generation is shown in
Figure 3(c).

###  5.4. Inference time

We have used different techniques to optimize for the time required in
inference. The first experiment used popular model serving algorithm vLLM
adapted for MPT-7B to find required inference time for one Tesla A100 SXM4 GPU
for each of the instruction types and found speedup of ten times using both
compared to no model serving. The experiment results are shown in Table 7

Evaluation | No speedup | vLLM
---|---|---
VEXCategory/VEXJustification | 0.5 | 0.04
Internal Comment | 2.5 | 0.2
Customer Comment | 2 | 0.19
Vector | 3 | 0.32
Table 7. Inference time in seconds on 44K test dataset for Model serving with
sequence length = 20K

In addition, the adaptive generation by dividing the inference batch into
batches of small and large token lengths also improves the average performance
over the test dataset to 4.5 second per evaluation from 8 second. The other
adaptation of using different maximum sequence length for each instruction
type brings down the inference time for each data point in the Evaluations
dataset by 1.5 times.

We have calculated the time required by a cybersecurity expert in our
organization to evaluate vulnerabilities. The time required for an expert,
averaged over 500 evaluations is 229s, whereas an evaluation by an our model
takes only 1 second per evaluation.

##  6\. Discussion

In this section, we will elucidate our observations related to training and
performance of LLMs for vulnerability management. We have learned valuable
lessons in understanding the benefits and pitfalls of using LLMs, and have
identified areas of improvement for the system.

###  6.1. Observations on Model Training

In our experiments we tested several hypothesis about diversity of
instructions and dataset size. Firstly, we observe that the size of the
dataset does not independently impact performance; rather, the diversity of
instructions significantly influences the model’s efficacy. Higher diversity
leads to better performance. When all instructions were consolidated into a
single instruction type, there was a marked decline in performance. This
decrease was primarily due to the model generating extraneous text, either
supplementing or replacing the expected response. Secondly, the responses
corresponding to instruction types that appeared later in the consolidated
instruction model had higher degradation of performance. This is corroborated
by training with combination of customer comments and category as well.
Thirdly, representation of related assets and notifications are important,
since the model performance dips when tested on completely different types of
assets and notifications.

LLMs have reportedly underperformed in text classification tasks; however,
their performance has improved when reasoning elements are incorporated into
the text (Sun et al., 2023). Our experiments yielded similar observations. We
experimented on using reasoning for VEXCategory and found that this leads to
higher F1 than without using reasoning. In our study, we used VEXJustification
as a reasoning for VEXCategory, as well as examined whether adding Customer
Comment as an explanation would improve the system. There are no major
improvements with adding Customer Comments as reasoning and also leads it to
lower performance for Customer Comments. Chain-of-Thought prompts which have
been shown to enhance model reasoning power would probably be useful to
enhance our model in the future. Additionally, we have observed that the total
number of classes affect the performance of the system greatly. Vector
generation can be considered as a multi-label multiclass classification
problem whereas VEXCategory is a binary classification problem and
VEXJustification is a multiclass classifcation problem. Though the F1 for
Vector is high, F1 is still low for underrepresented classes which leads to
decrease in performance. The performance also drops from VEXCategory
classification to VEXJustification classification, thereby showing that the
number of classes affects the performance. However, the Customer Comment
section follows a highly templated format, with approximately 85% of customer
comments being repeated across multiple evaluations. Hence we might consider
Customer Comment also to be a classification problem with large number of
classes. However the performance on Customer Comments does not stray far from
the performance for VEXJustification with much fewer classes.

###  6.2. Observations on Inference

We have noticed model output variations with beam size, with decline in
performance with higher beam sizes. Increase in beam search size increases the
diversity of text, and that leads to less contextually relevant outputs, as
seen in our experiments. Low beam size is sufficient for high quality
generation because of the specialized nature of the dataset. The difference
between probabilities of less probable outcomes and highly probable outcomes,
given the context, is low. This leads to no temperature or nucleus sampling-
based variations in model-generated outputs.

###  6.3. Error analysis

We have used MPT-7B to train CVE-LLM for two reasons:  (1) Expansion
capability to at least 20K tokens without significant loss of performance, and
(2) It performs better than other open source models we tested with our
dataset  However, among all the LLMs we have trained, majority of the issues
we have encountered are remarkably similar, differing primarily in their
severity. The generated text for both Internal Comment and Customer Comment
has been observed to have omitted or falsely included several critical
details, particularly those concerning the affected software versions, the
recommended update versions, the names of the software or components, and
other similarly named entities that constitute essential information. The
other type of error commonly seen for the LLMs is spurious text generation,
which is something we observed quite frequently for trained Llama2 models.
Introduction of stop tokens mitigated the problem to a great extent for
MPT-7B, but not for the other models. In spite of having the ability to extend
the sequence length of the trained model beyond the training sequence length,
the performance on long sequences still falls behind. Long instructions
comprises of multiple vulnerabilities, and constitute only 10% of our test
dataset. In the future, we plan to segment it into multiple vulnerabilities
and use more robust semantic understanding of each vulnerability to perform
vulnerability chaining. We also tested the system only with zero shot
instructions at the inference, and we surmise that we can utilize language
understanding capabilities better by using few shot methods to assess
vulnerabilities for assets.

This study illustrates the capacity of large language models (LLMs) to learn
from expert-curated historical data, thereby enabling the automation of
vulnerability management for medical devices. Our model CVE-LLM has proved to
be effective in learning from past evaluations and has shown high accuracy in
prediction of VEXCategory and CVSS Vectors. The model’s inference is also
substantially faster, achieving speeds approximately 200 times greater than
those of a human expert. We intend for our system to be employed in a human-
in-the-loop manner, assisting resident product cybersecurity experts in
swiftly identifying vulnerabilities that affect the assets and communicating
promptly to the customer.

##  7\. Disclaimer

The concepts and information presented in this paper/presentation are based on
research results that are not commercially available. Future commercial
availability cannot be guaranteed.

## References

    * (1)
    * Abdeen et al. (2023) Basel Abdeen, Ehab Al-Shaer, Anoop Singhal, Latifur Khan, and Kevin Hamlen. 2023.  Smet: Semantic mapping of cve to att&ck and its application to cybersecurity. In _IFIP Annual Conference on Data and Applications Security and Privacy_. Springer, 243–260.
    * Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023\.  Gpt-4 technical report.  _arXiv preprint arXiv:2303.08774_ (2023).
    * Ameri et al. (2021) Kimia Ameri, Michael Hempel, Hamid Sharif, Juan Lopez Jr, and Kalyan Perumalla. 2021.  Cybert: Cybersecurity claim classification by fine-tuning the bert language model.  _Journal of Cybersecurity and Privacy_ 1, 4 (2021), 615–637.
    * Ampel et al. (2021) Benjamin Ampel, Sagar Samtani, Steven Ullman, and Hsinchun Chen. 2021.  Linking common vulnerabilities and exposures to the mitre att&ck framework: A self-distillation approach.  _arXiv preprint arXiv:2108.01696_ (2021).
    * Center (2024) Identity Theft Resource Center. January 2024.  _2023 Data Breach Report_.  https://www.idtheftcenter.org/wp-content/uploads/2024/01/ITRC_2023-Annual-Data-Breach-Report.pdf
    * Chen et al. (2024) Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, et al. 2024\.  Symbolic discovery of optimization algorithms.  _Advances in Neural Information Processing Systems_ 36 (2024).
    * Christey et al. (2013) Steve Christey, J Kenderdine, J Mazella, and B Miles. 2013.  Common weakness enumeration.  _Mitre Corporation_ (2013).
    * Chronopoulou et al. (2019) Alexandra Chronopoulou, Christos Baziotis, and Alexandros Potamianos. 2019.  An embarrassingly simple approach for transfer learning from pretrained language models.  _arXiv preprint arXiv:1902.10547_ (2019).
    * Corporation (2024a) The MITRE Corporation. 2024a.  _MITRE ATT &CK_.  https://attack.mitre.org/
    * Corporation (2024b) The MITRE Corporation. 2024b.  _MITRE CAPEC_.  https://capec.mitre.org/
    * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
    * Elbaz et al. (2020) Clément Elbaz, Louis Rilling, and Christine Morin. 2020.  Fighting N-day vulnerabilities with automated CVSS vector prediction at disclosure. In _Proceedings of the 15th International Conference on Availability, Reliability and Security_. 1–10.
    * Feng et al. (2023) Chao Feng, Xinyu Zhang, and Zichu Fei. 2023.  Knowledge solver: Teaching llms to search for domain knowledge from knowledge graphs.  _arXiv preprint arXiv:2309.03118_ (2023).
    * Fu et al. (2024) Yanjun Fu, Ethan Baker, and Yizheng Chen. 2024.  Constrained Decoding for Secure Code Generation.  _arXiv preprint arXiv:2405.00218_ (2024).
    * Ge et al. (2024) Yingqiang Ge, Wenyue Hua, Kai Mei, Juntao Tan, Shuyuan Xu, Zelong Li, Yongfeng Zhang, et al. 2024\.  Openagi: When llm meets domain experts.  _Advances in Neural Information Processing Systems_ 36 (2024).
    * Hassanin et al. (2024) Mohammed Hassanin, Marwa Keshk, Sara Salim, Majid Alsubaie, and Dharmendra Sharma. 2024.  PLLM-CS: Pre-trained Large Language Model (LLM) for Cyber Threat Detection in Satellite Networks.  _arXiv preprint arXiv:2405.05469_ (2024).
    * Huggingface (2023) Huggingface. 2023.  _Open LLM Leaderboard_.  https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
    * Iorga et al. (2020) Denis Iorga, Dragos Corlătescu, Octavian Grigorescu, Cristian Săndescu, Mihai Dascălu, and Razvan Rughiniş. 2020.  Early detection of vulnerabilities from news websites using machine learning models. In _2020 19th RoEduNet Conference: Networking in Education and Research (RoEduNet)_. IEEE, 1–6.
    * Islam et al. (2024) Nafis Tanveer Islam, Joseph Khoury, Andrew Seong, Gonzalo De La Torre Parra, Elias Bou-Harb, and Peyman Najafirad. 2024.  LLM-Powered Code Vulnerability Repair with Reinforcement Learning and Semantic Reward.  _arXiv preprint arXiv:2401.03374_ (2024).
    * Jiang et al. (2024) Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024\.  Mixtral of experts.  _arXiv preprint arXiv:2401.04088_ (2024).
    * Kwon et al. (2023) Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023.  Efficient memory management for large language model serving with pagedattention. In _Proceedings of the 29th Symposium on Operating Systems Principles_. 611–626.
    * Le Scao et al. (2023) Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2023\.  Bloom: A 176b-parameter open-access multilingual language model.  (2023).
    * Lee et al. (2023) Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. 2023.  Rlaif: Scaling reinforcement learning from human feedback with ai feedback.  _arXiv preprint arXiv:2309.00267_ (2023).
    * Levshun (2023) Dmitry Levshun. 2023.  Comparative Analysis of Machine Learning Methods in Vulnerability Metrics Transformation. In _International Conference on Intelligent Information Technologies for Industry_. Springer, 60–70.
    * Lewis et al. (2020) Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020\.  Retrieval-augmented generation for knowledge-intensive nlp tasks.  _Advances in Neural Information Processing Systems_ 33 (2020), 9459–9474.
    * Li et al. (2023) Yinheng Li, Shaofei Wang, Han Ding, and Hang Chen. 2023.  Large language models in finance: A survey. In _Proceedings of the Fourth ACM International Conference on AI in Finance_. 374–382.
    * Lin and Och (2004) Chin-Yew Lin and FJ Och. 2004.  Looking for a few good metrics: ROUGE and its evaluation. In _Ntcir workshop_.
    * Liu et al. (2023) Xin Liu, Yuan Tan, Zhenghang Xiao, Jianwei Zhuge, and Rui Zhou. 2023.  Not the end of story: An evaluation of chatgpt-driven vulnerability description mappings. In _Findings of the Association for Computational Linguistics: ACL 2023_. 3724–3731.
    * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
    * Lu et al. (2024) Guilong Lu, Xiaolin Ju, Xiang Chen, Wenlong Pei, and Zhilong Cai. 2024.  GRACE: Empowering LLM-based software vulnerability detection with graph structure and in-context learning.  _Journal of Systems and Software_ 212 (2024), 112031.
    * Lu et al. (2023) Keming Lu, Hongyi Yuan, Runji Lin, Junyang Lin, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2023.  Routing to the expert: Efficient reward-guided ensemble of large language models.  _arXiv preprint arXiv:2311.08692_ (2023).
    * Mell et al. (2006) Peter Mell, Karen Scarfone, and Sasha Romanosky. 2006.  Common vulnerability scoring system.  _IEEE Security & Privacy_ 4, 6 (2006), 85–89.
    * Mitre (2005) CVE Mitre. 2005.  Common vulnerabilities and exposures.
    * Moore et al. (2023) Steven Moore, Richard Tong, Anjali Singh, Zitao Liu, Xiangen Hu, Yu Lu, Joleen Liang, Chen Cao, Hassan Khosravi, Paul Denny, et al. 2023\.  Empowering education with llms-the next-gen interface and content generation. In _International Conference on Artificial Intelligence in Education_. Springer, 32–37.
    * Naito et al. (2023a) Takeru Naito, Rei Watanabe, and Takuho Mitsunaga. 2023a.  LLM-based Attack Scenarios Generator with IT Asset Management and Vulnerability Information. In _2023 6th International Conference on Signal Processing and Information Security (ICSPIS)_. IEEE, 99–103.
    * Naito et al. (2023b) Takeru Naito, Rei Watanabe, and Takuho Mitsunaga. 2023b.  LLM-based Attack Scenarios Generator with IT Asset Management and Vulnerability Information. In _2023 6th International Conference on Signal Processing and Information Security (ICSPIS)_. 99–103.  https://doi.org/10.1109/ICSPIS60075.2023.10344019
    * Naveed et al. (2023) Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Nick Barnes, and Ajmal Mian. 2023.  A comprehensive overview of large language models.  _arXiv preprint arXiv:2307.06435_ (2023).
    * of Health et al. (2016) U.S. Department of Health, Food Human Services, Center for Devices Drug Administration, Center for Biologics Evaluation Radiological Health, Office of the Center Director, and Research. December 28, 2016.  _Postmarket Management of Cybersecurity in Medical Devices: Guidance for Industry and Food and Drug Administration Staff_.  https://www.fda.gov/media/95862/download
    * OpenAI (2021) OpenAI. 2021.  ChatGPT.  https://openai.com/gpt-3/.
    * Pearce et al. (2023) Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2023.  Examining zero-shot vulnerability repair with large language models. In _2023 IEEE Symposium on Security and Privacy (SP)_. IEEE, 2339–2356.
    * Pedregosa et al. (2011) F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011.  Scikit-learn: Machine Learning in Python.  _Journal of Machine Learning Research_ 12 (2011), 2825–2830.
    * Prasad et al. (2018) K Siva Prasad, K Raja Sekhar, and P Rajarajeswari. 2018.  An integrated approach towards vulnerability assessment & penetration testing for a web application.  _International Journal of Engineering and Technology (UAE)_ 7 (2018), 431–435.
    * Press et al. (2021) Ofir Press, Noah A Smith, and Mike Lewis. 2021.  Train short, test long: Attention with linear biases enables input length extrapolation.  _arXiv preprint arXiv:2108.12409_ (2021).
    * Radford et al. (2018) Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018\.  Improving language understanding by generative pre-training.  (2018).
    * Rasley et al. (2020) Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020.  Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 3505–3506.
    * Sallam (2023) Malik Sallam. 2023.  ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. In _Healthcare_ , Vol. 11. MDPI, 887.
    * Sewak et al. (2023) Mohit Sewak, Vamsi Emani, and Annam Naresh. 2023.  CRUSH: Cybersecurity Research using Universal LLMs and Semantic Hypernetworks.  (2023).
    * Shahid and Debar (2021) Mustafizur R Shahid and Hervé Debar. 2021.  Cvss-bert: Explainable natural language processing to determine the severity of a computer security vulnerability from its description. In _2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA)_. IEEE, 1600–1607.
    * Steenhoek et al. (2024) Benjamin Steenhoek, Md Mahbubur Rahman, Monoshi Kumar Roy, Mirza Sanjida Alam, Earl T Barr, and Wei Le. 2024.  A Comprehensive Study of the Capabilities of Large Language Models for Vulnerability Detection.  _arXiv preprint arXiv:2403.17218_ (2024).
    * Sun et al. (2023) Xiaofei Sun, Xiaoya Li, Jiwei Li, Fei Wu, Shangwei Guo, Tianwei Zhang, and Guoyin Wang. 2023.  Text classification via large language models.  _arXiv preprint arXiv:2305.08377_ (2023).
    * Sun et al. (2024) Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Wei Ma, Lyuye Zhang, Miaolei Shi, and Yang Liu. 2024.  LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs’ Vulnerability Reasoning.  _arXiv preprint arXiv:2401.16185_ (2024).
    * Tan et al. (2024) Hanzhuo Tan, Qi Luo, Jing Li, and Yuqun Zhang. 2024.  LLM4Decompile: Decompiling Binary Code with Large Language Models.  _arXiv preprint arXiv:2403.05286_ (2024).
    * Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023.  Alpaca: A strong, replicable instruction-following model.  _Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html_ 3, 6 (2023), 7.
    * Team (2023) MosaicML NLP Team. 2023.  _Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs_.  www.mosaicml.com/blog/mpt-7b Accessed: 2023-05-05.
    * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023\.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ (2023).
    * Vasireddy et al. (2023) Dinesh T Vasireddy, Dakota S Dale, and Qinghua Li. 2023.  CVSS Base Score Prediction Using an Optimized Machine Learning Scheme. In _2023 Resilience Week (RWS)_. IEEE, 1–6.
    * von Werra et al. (2020) Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, and Shengyi Huang. 2020.  TRL: Transformer Reinforcement Learning.  https://github.com/huggingface/trl.
    * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022\.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in neural information processing systems_ 35 (2022), 24824–24837.
    * Whitehouse et al. (2023) Chenxi Whitehouse, Monojit Choudhury, and Alham Fikri Aji. 2023.  Llm-powered data augmentation for enhanced crosslingual performance.  _arXiv preprint arXiv:2305.14288_ (2023).
    * Wu et al. (2023) Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. 2023.  Autogen: Enabling next-gen llm applications via multi-agent conversation framework.  _arXiv preprint arXiv:2308.08155_ (2023).
    * Yang et al. (2024) Xiaoxian Yang, Zhifeng Wang, Qi Wang, Ke Wei, Kaiqi Zhang, and Jiangang Shi. 2024.  Large language models for automated q&a involving legal documents: a survey on algorithms, frameworks and applications.  _International Journal of Web Information Systems_ (2024).
    * Yao et al. (2024) Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024.  A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.  _High-Confidence Computing_ (2024), 100211.
    * Yin et al. (2020) Jiao Yin, MingJian Tang, Jinli Cao, and Hua Wang. 2020.  Apply transfer learning to cybersecurity: Predicting exploitability of vulnerabilities by description.  _Knowledge-Based Systems_ 210 (2020), 106529.
    * Yin and Ni (2024) Xin Yin and Chao Ni. 2024.  Multitask-based Evaluation of Open-Source LLM on Software Vulnerability.  _arXiv preprint arXiv:2404.02056_ (2024).
    * Yosifova et al. (2021) Veneta Yosifova, Antoniya Tasheva, and Roumen Trifonov. 2021.  Predicting vulnerability type in common vulnerabilities and exposures (cve) database with machine learning classifiers. In _2021 12th National Conference with International Participation (ELECTRONICA)_. IEEE, 1–6.
    * Zhang et al. (2024b) Chenhui Zhang, Le Wang, Dunqiu Fan, Junyi Zhu, Tang Zhou, Liyi Zeng, and Zhaohua Li. 2024b.  VTT-LLM: Advancing Vulnerability-to-Tactic-and-Technique Mapping through Fine-Tuning of Large Language Model.  _Mathematics_ 12, 9 (2024), 1286.
    * Zhang et al. (2024a) Jie Zhang, Haoyu Bu, Hui Wen, Yu Chen, Lun Li, and Hongsong Zhu. 2024a.  When LLMs Meet Cybersecurity: A Systematic Literature Review.  _arXiv preprint arXiv:2405.03644_ (2024).
    * Zhou et al. (2024) Xin Zhou, Sicong Cao, Xiaobing Sun, and David Lo. 2024.  Large Language Model for Vulnerability Detection and Repair: Literature Review and Roadmap.  _arXiv preprint arXiv:2404.02525_ (2024).
Type of Error |  Expected Response |  Generated Response
---|---|---
Wrong generation |  The issue may be addressed in a future security patch or software version. |  Vulnerability is only exploitable in context of “prohibited” use cases
Omission |  CoreMedia, WebRTC is not utilized in the product. |  Technology component is not deployed.
Wrong key information |  .NET 7.0 is not deployed and utilized in the product. |  .NET 6.0 is not deployed and utilized in the product.
Table 8. Common errors in CVE-LLM generation

##  Appendix A Environmental Vector test dataset results

Environmental Metrics | Micro-F1
---|---
Modified Attack Complexity | 0.919
Modified Availability | 0.923
Integrity Requirement | 0.922
Modified Attack Vector | 0.918
Modified Confidentiality | 0.925
Availability Requirement | 0.921
Modified Scope | 0.924
Modified User Interaction | 0.922
Modified Privileges Required | 0.919
Confidentiality Requirement | 0.922
Modified Integrity | 0.924
Table 9. Micro-F1 for Environmental metrics on test dataset (N=44K)

The environmental metrics found in the test dataset and their corresponding
micro-F1 are shown in 9. Figure 5 shows the confusion matrix for these
environmental metrics. Due to differences in CVSS versions, there are some
environmental metrics that are not applicable to certain evaluations, and
their values are represented as ‘N/A’. The value ‘XXXX’ for environmental
metrics refers to the cases where the expert did not use any value for that
environmental metrics.

##  Appendix B Common errors in generation with CVE-LLM

The common errors found in model outputs are mostly related to  (1) Wrong
generation,  (2) Omission of key information,  (3) Wrong key information, and
(4) Addition of spurious details  Examples of these errors are shown in Table
8 and 10.

We noticed that most of the errors in generation are due to the probabilistic
regurgitation of training text, and there is less semantic understanding of
the paradigm. We surmise that this can be corrected post-hoc with verification
of the entities involved in the generation, chain of thought prompts and
reasoning modules.

Prompt
---
Below is an instruction that describes a task, paired with an input that
provides further context. Write a response that appropriately completes the
request. ### Instruction: Generate internal comments. ### Input: Organization:
DI-DnA Software: Syngo Carbon Monitoring Product: Syngo Carbon Monitoring
VB12A Notification: A vulnerability in the mod_proxy_cluster in the Apache
server. This may allow a malicious user to add a script in the ‘alias’
parameter in the URL to trigger the stored cross-site scripting vulnerability.
By adding a script on the alias parameter on the URL, it adds a new virtual
host and adds the script to the cluster-manager page. Components present in
software: HTTP Server (httpd) - Apache Software Foundation - 2.4.57 Base and
Temporal Vectors: Attack Vector is Network. Attack Complexity is Low.
Privileges Required is Low. User Interaction is Required. Scope is Changed.
Confidentiality is Low. Integrity is Low. Availability is None. Exploit Code
Maturity is Unproven. Remediation Level is Unavailable. Report Confidence is
Reasonable. CVSS Version: 3.1 ### Response:
Expected Response
Component will be updated in future release of product.
Generated Response
Component is updated to the latest version of GEC agent.
Table 10.  Addition of unrelated incorrect information

(a) Training instruction Word count density plot

(b) Train instruction Word count histogram for instructions with ¡1k words

(c) Test instruction Word count density plot

(d) Test instruction Word count histogram for instructions with ¡1k words

(e) Training dataset VEXCategory histogram

(f) Test dataset VEXCategory histogram

(g) Training dataset VEXJustification histogram

(h) Test dataset VEXJustification histogram

Figure 4. Dataset statistics

(a) Modified Attack Complexity

(b) Modified Availability

(c) Modified Confidentiality

(d) Modified Scope

(e) Modified User Interaction

(f) Modified Privileges Required

(g) Confidentiality Requirement

(h) Integrity Requirement

(i) Availability Requirement

(j) Modified Attack Vector

(k) Modified Integrity

Figure 5. Environmental Vector confusion matrix: breakdown by metrics