# A Survey on Data-driven Software Vulnerability Assessment and Prioritization

Triet H. M. Le<EMAIL_ADDRESS>,  Huaming Chen
<EMAIL_ADDRESS>CREST - The Centre for Research on Engineering
Software Technologies, The University of AdelaideAdelaideAustralia and  M. Ali
Babar<EMAIL_ADDRESS>CREST - The Centre for Research on
Engineering Software Technologies, The University of AdelaideAdelaideAustralia
Cyber Security Cooperative Research CentreAustralia

###### Abstract.

Software Vulnerabilities (SVs) are increasing in complexity and scale, posing
great security risks to many software systems. Given the limited resources in
practice, SV assessment and prioritization help practitioners devise optimal
SV mitigation plans based on various SV characteristics. The surges in SV data
sources and data-driven techniques such as Machine Learning and Deep Learning
have taken SV assessment and prioritization to the next level. Our survey
provides a taxonomy of the past research efforts and highlights the best
practices for data-driven SV assessment and prioritization. We also discuss
the current limitations and propose potential solutions to address such
issues.

Software vulnerability, Vulnerability assessment and prioritization

††journal: CSUR††copyright: none††ccs: General and reference Surveys and
overviews††ccs: Security and privacy Software security engineering††ccs:
Computing methodologies Machine learning††ccs: Computing methodologies Neural
networks

##  1\. Introduction

Software Vulnerabilities (SVs) can negatively affect the confidentiality,
integrity and availability of software systems (Ghaffarian and Shahriari,
2017). The exploitation of these SVs such as the Heartbleed
attack111https://heartbleed.com can damage the operations and reputation of
millions of systems and organizations globally, resulting in huge financial
losses as well. Therefore, it is important to remediate critical SVs as
promptly as possible.

Vulnerability assessment is required to prioritize the remediation of critical
SVs among a large and increasing number of SVs each year (Smyth, 2017) (e.g.,
more than 20,000 SVs were reported on National Vulnerability Database (NVD)
(NIST, [n. d.]a) in 2021). SV assessment includes tasks that determine various
characteristics such as the types, exploitability, impact and severity levels
of SVs (Spanos and Angelis, 2018; Le et al., 2019; Duan et al., 2021). Such
characteristics help understand and select high-priority SVs to resolve early
given the limited effort and resources. For example, an identified cross-site
scripting (XSS) or SQL injection vulnerability in a web application will
likely require an urgent remediation plan. These two types of SVs are well-
known and can be easily exploited by attackers to gain unauthorized access and
compromise sensitive data/information. On the other hand, an SV that requires
admin access or happens only in a local network will probably have a lower
priority since only a few people can initiate an attack.

There has been an active research area to assess and prioritize SVs using
increasingly large data from multiple sources. Many studies in this area have
proposed different Natural Language Processing (NLP), Machine Learning (ML)
and Deep Learning (DL) techniques to leverage such data to automate various
tasks such as predicting the Common Vulnerability Scoring System (CVSS)
(FIRST, [n. d.]a) metrics (e.g., (Le et al., 2019; Spanos and Angelis, 2018;
Han et al., 2017)) or public exploits (e.g., (Bozorgi et al., 2010; Sabottke
et al., 2015; Bullough et al., 2017)). These prediction models can learn the
patterns automatically from vast SV data, which would be otherwise impossible
to do manually. Such patterns are utilized to speed up the assessment and
prioritization processes of ever-increasing and more complex SVs,
significantly reducing practitioners’ effort. Despite the rising research
interest in data-driven SV assessment and prioritization, to the best of our
knowledge, there has been no comprehensive survey on the state-of-the-art
methods and existing challenges in this area.

Table 1. Comparison of contributions between our survey and the existing
related surveys/reviews.

Study Contribution |  Focus on SV assessment & prioritization |  Analysis of SV |
---|---|---|---
|  Analysis of data- | data sources |
| driven approaches (NLP/ML/DL) |  |
Ghaffarian et al. 2017 (Ghaffarian and Shahriari, 2017) | – | – |  ✓(Mostly ML)
Lin et al. 2020 (Lin et al., 2020) Semasaba et al. 2020 (Semasaba et al., 2020) Singh et al. 2020 (Singh and Chaturvedi, 2020) Zeng et al. 2020 (Zeng et al., 2020) | – | – |  ✓(Mostly DL)
Pastor et al. 2020 (Pastor-Galindo et al., 2020) | – |  ✓(OSINT) | –
Sun et al. 2018 (Sun et al., 2018) Evangelista et al. 2020 (Evangelista et al., 2020) | – |  ✓(OSINT) | ✓
Khan et al. 2018 (Khan and Parkinson, 2018) |  ✓(Rule-based methods) | – | –
Kritikos et al. 2019 (Kritikos et al., 2019) |  ✓(Static analysis) | ✓ | –
Dissanayake et al. 2020 (Dissanayake et al., 2020) | ✓(Socio-technical aspects) | – | –
Our survey | ✓ | ✓ | ✓

Our Contributions. 1 We are the first to review in-depth the research studies
that automate data-driven SV assessment and prioritization tasks leveraging SV
data and NLP/ML/DL techniques. 2 We categorize and describe the key tasks
performed in relevant primary studies. 3 We synthesize and discuss the pros
and cons of data, features, models, evaluation methods and metrics commonly
used in the reviewed studies. 4 We highlight the challenges with the current
practices and propose potential solutions moving forward. Our findings can
provide useful guidelines for researchers and practitioners to effectively
utilize data to perform SV assessment and prioritization. An online and up-to-
date (by accepting external contributions) version of the survey can be found
at https://github.com/lhmtriet/awesome-vulnerability-assessment.

Related Work. There have been several existing surveys/reviews on SV analysis
and prediction, but they are fundamentally different from ours (see Table 1).
Ghaffarian et al. (Ghaffarian and Shahriari, 2017) conducted a seminal survey
on ML-based SV analysis and discovery. Subsequently, several studies (Zeng et
al., 2020; Singh and Chaturvedi, 2020; Semasaba et al., 2020; Lin et al.,
2020) reviewed DL techniques for detecting vulnerable code. However, these
prior reviews did not describe how ML/DL techniques can be used to assess and
prioritize the detected SVs. There have been other relevant reviews on using
Open Source Intelligence (OSINT) (e.g., phishing or malicious emails/URLs/IPs)
to make informed security decisions (Pastor-Galindo et al., 2020; Evangelista
et al., 2020; Sun et al., 2018). However, these OSINT reviews did not
explicitly discuss the use of SV data and how such data can be leveraged to
automate the assessment and prioritization processes. Moreover, most of the
reviews on SV assessment and prioritization have focused on either static
analysis tools (Kritikos et al., 2019) or rule-based approaches (e.g., expert
systems or ontologies) (Khan and Parkinson, 2018). These methods rely on pre-
defined patterns and struggle to work with new types and different data
sources of SVs compared to contemporary ML or DL approaches presented in our
survey (Han et al., 2011; Goodfellow et al., 2016). Recently, Dissanayake et
al. (Dissanayake et al., 2020) reviewed the socio-technical challenges and
solutions for security patch management that involves SV assessment and
prioritization after SV patches are identified. Unlike (Dissanayake et al.,
2020), we focus on the challenges, solutions and practices of automating
various SV assessment and prioritization tasks with data-driven techniques. We
also consider all types of SV assessment/prioritization regardless of the
patch availability.

Paper Outline. The rest of the paper is organized as follows. Section 2
presents the scope, methodology and taxonomy covered in this survey. Sections
3, 4, 5, 6 and 7 review the studies in each theme of the taxonomy. Section 8
identifies and discusses the common practices and respective implications for
data-driven SV assessment and prioritization. Section 9 discusses the open
challenges and proposes some future directions of this research area. Finally,
section 10 concludes the survey.

##  2\. Survey Overview

###  2.1. Background and Scope of the Survey

Our survey’s focus is on data-driven SV assessment and prioritization. The
assessment and prioritization phases are between the SV discovery/detection
and SV remediation/mitigation/fixing/patching phases in an SV management
lifecycle (Foreman, 2019). The assessment phase unveils the characteristics of
the SVs found in the discovery phase to locate “hot spots” that contain many
highly critical/severe SVs and require higher attention in a system. In the
prioritization phase, practitioners use the assessment outputs to devise an
optimal remediation plan, i.e., the order/priority of fixing each SV, based on
available human and technological resources. SVs would then be mitigated/fixed
accordingly to the prioritized plan in the remediation phase. Unlike the
existing surveys on rule-based or experience-based SV assessment and
prioritization (Kritikos et al., 2019; Khan and Parkinson, 2018; Dissanayake
et al., 2020) that hardly utilize the potential of SV data in the wild, this
survey aims to review research papers that have leveraged such data to
automate tasks in this area using data-driven models. To keep our focus, we do
not consider papers that only perform manual analyses or descriptive
statistics (e.g., taking mean/median/variation of data) without using any
data-driven models as these techniques cannot automatically assess or
prioritize new SVs. We also do not directly compare the absolute performance
of all the related studies as they did not use exactly the same experimental
setup (e.g., data sources and model configurations). While it is theoretically
possible to perform a comparative evaluation of the identified techniques by
establishing and using a common setup, this type of evaluation is out of the
scope of this survey. However, we still cover the key directions/techniques of
the studies in sections 3, 4, 5, 6 and 7. We also provide in-depth discussion
on the common practices and challenges of these studies and suggest some
potential directions to advance the field in sections 8 and 9.

Figure 1. Taxonomy of studies on data-driven SV assessment and prioritization.

###  2.2. Methodology

Study selection. Our study selection was inspired by the Systematic Literature
Review guidelines (Keele et al., 2007). Due to the space limit, we only
included the key steps of our selection here. The full details can be found at
(Le et al., [n. d.]). We first designed the search string: “‘software’ AND
vulner* AND (learn* OR data* OR predict*) AND (priority* OR assess* OR impact*
OR exploit* OR severity*) AND NOT (fuzz* OR dynamic* OR intrusion OR
adversari* OR malware* OR ‘vulnerability detection’ OR ‘vulnerability
discovery’ OR ‘vulnerability identification’ OR ‘vulnerability prediction’)”.
This search string covered the key papers (i.e., with more than 50 citations)
in the area and excluded many papers on general security and SV detection. We
then adapted this string222This search string was customized for each database
and the database-wise search strings can be found at (Le et al., [n. d.]). to
retrieve an initial list of 1,765 papers up to April 2021 from various
commonly used databases such as IEEE Xplore, ACM Digital Library, Scopus,
SpringerLink and Wiley. We also defined the inclusion/exclusion criteria (see
(Le et al., [n. d.]) for more details) to filter out irrelevant/low-quality
studies with respect to our scope in section 2.1. Based on these criteria and
the titles and abstracts and keywords of 1,765 initial papers, we removed
1,550 papers. After reading the full-text and applying the criteria on the
remaining 215 papers, we obtained 70 papers directly related to data-driven SV
assessment and prioritization. To further increase the coverage of studies, we
performed backward and forward snowballing on these 70 papers (using the above
sources and Google Scholar) and identified 14 more papers that satisfied the
inclusion/exclusion criteria. In total, we included 84 studies for our survey.
We do not claim that we have collected all the papers in this area, but we
believe that our selection covered most of the key studies to unveil the
practices of data-driven SV assessment and prioritization.

Data extraction and synthesis of the selected studies. We followed the steps
of thematic analysis (Cruzes and Dybå, 2011) to identify the taxonomy of data-
driven SV assessment and prioritization tasks in sections 3, 4, 5, 6 and 7 as
well as the key practices of data-driven model building for automating these
tasks in section 8. We first conducted a pilot study of 20 papers to
familiarize ourselves with data to be extracted from the primary studies.
After that, we generated initial codes and then merged them iteratively in
several rounds to create themes. Two of the authors performed the analysis
independently, in which each author analyzed half of the selected papers and
then reviewed the analysis output of the other author. Any disagreements were
resolved through discussions.

###  2.3. Taxonomy of Data-driven Software Vulnerability Assessment and
Prioritization

Based on the scope in section 2.1 and the methodology in section 2.2, we
identified five main themes of the relevant studies in the area of data-driven
SV assessment and prioritization (see Figure 1). Specifically, we extracted
the themes by grouping related SV assessment or prioritization tasks that the
surveyed studies aim to automate/predict using data-driven models. Note that a
paper is categorized into more than one theme if that paper develops models
for multiple cross-theme tasks.

We acknowledge that there can be other ways to categorize the studies.
However, we assert the reliability of our taxonomy as all of our themes
(except theme 5) align with the security standards used in practice. For
example, Common Vulnerability Scoring System (CVSS) (FIRST, [n. d.]a) provides
a framework to characterize exploitability, impact and severity of SVs (themes
1-3), while Common Weakness Enumeration (CWE) (MITRE, [n. d.]d) includes many
vulnerability types (theme 4). Hence, we believe our taxonomy can help
identify and bridge the knowledge gap between the academic literature and
industrial practices, making it relevant and potentially beneficial for both
researchers and practitioners. Details of each theme in our taxonomy are
covered in subsequent sections.

Table 2. List of the surveyed papers in the Exploit Likelihood sub-theme of the Exploitation theme. Note: The nature of task of this sub-theme is binary classification of existence/possibility of proof-of-concept and/or real-world exploits.  Study |  Data source |  Data-driven technique
---|---|---
Bozorgi et al. 2010 (Bozorgi et al., 2010) |  CVE, Open Source Vulnerability Database (OSVDB) |  Linear Support Vector Machine (SVM)
Sabottke et al. 2015 (Sabottke et al., 2015) |  NVD, Twitter, OSVDB, ExploitDB, Symantec security advisories, private Microsoft security advisories |  Linear SVM
Edkrantz et al. 2015 (Edkrantz, 2015; Edkrantz et al., 2015) |  NVD, Recorded Future security advisories, ExploitDB |  Naïve Bayes, Linear SVM, Random forest
Bullough et al. 2017 (Bullough et al., 2017) |  NVD, Twitter, ExploitDB |  Linear SVM
Almukaynizi et al. (Almukaynizi et al., 2017, 2019) |  NVD, ExploitDB, Zero Day Initiative security advisories & Darkweb forums/markets |  SVM, Random forest, Naïve Bayes, Bayesian network, Decision tree, Logistic regression
Xiao et al. 2018 (Xiao et al., 2018) |  NVD, SecurityFocus security advisories, Symantec
Spam/malicious activities based on daily blacklists from abuseat.org, spamhaus.org, spamcop.net, uceprotect.net, wpbl.info & list of unpatched SVs in hosts |  Identification of malicious activity groups with community detection algorithms + Random forest for exploit prediction
Tavabi et al. 2018 (Tavabi et al., 2018) |  NVD, 200 sites on Darkweb, ExploitDB, Symantec, Metasploit |  Paragraph embedding + Radial basis function kernel SVM
de Sousa et al. 2020 (de Sousa et al., 2020) |  NVD, Twitter, ExploitDB, Symantec
Avast, ESET, Trend Micro security advisories |  Linear SVM, Logistic regression, XGBoost, Light Gradient Boosting Machine (LGBM)
Fang et al. 2020 (Fang et al., 2020) |  NVD, ExploitDB, SecurityFocus, Symantec |  fastText + LGBM
Huang et al. 2020 (Huang and Wu, 2020) |  NVD, CVE Details, Twitter, ExploitDB, Symantec security advisories |  Random forest
Jacobs et al. 2020 (Jacobs et al., 2020) |  NVD, Kenna Security
Exploit sources: Exploit DB, Metasploit, FortiGuard Labs, SANS Internet Storm Center, Securewords CTU, Alienvault OSSIM, Canvas/D2 Security’s Elliot Exploitation Frameworks, Contagio, Reversing Labs |  XGBoost
Yin et al. 2020 (Yin et al., 2020) |  NVD, ExploitDB, General text: Book Corpus & Wikipedia for pretraining BERT models |  Fine-tuning BERT models pretrained on general text
Bhatt et al. 2021 (Bhatt et al., 2021) |  NVD, ExploitDB |  Features augmented by SV types + Decision tree, Random forest, Naïve Bayes, Logistic regression, SVM
Suciu et al. 2021 (Suciu et al., 2021) |  NVD, Vulners database, Twitter, Symantec, SecurityFocus, IBM X-Force Threat Intelligence
Exploit sources: ExploitDB, Metasploit, Canvas, D2 Security’s Elliot, Tenable, Skybox, AlienVault, Contagio |  Multi-layer perceptron
Younis et al. 2014 (Younis and Malaiya, 2014) |  Vulnerable functions from NVD (Apache HTTP Server project), ExploitDB, OSVDB |  SVM
Yan et al. 2017 (Yan et al., 2017) |  Executables (binary code) of 100 Linux applications |  Combining ML (Decision tree) output & fuzzing with a Bayesian network
Tripathi et al. 2017 (Tripathi et al., 2017) |  Program crashes from VDiscovery (Cha et al., 2012; Grieco et al., 2016) & LAVA (Dolan-Gavitt et al., 2016) datasets |  Static/Dynamic analysis features + Linear/Radial basis function kernel SVM
Zhang et al. 2018 (Zhang and Thing, 2018) |  Program crashes from VDiscovery (Cha et al., 2012; Grieco et al., 2016) dataset |  $n$-grams of system calls from execution traces + Online passive-aggressive classifier

##  3\. Exploitation Prediction

This section covers the Exploitation theme that automates the detection and
understanding of both Proof-of-Concept (PoC) and real-world exploits333An
exploit is a piece of code used to compromise vulnerable software (Sabottke et
al., 2015). Real-world exploits are harmful & used in real host/network-based
attacks. PoC exploits are unharmful & used to show the potential threats of
SVs in penetration tests. targeting identified SVs. This theme outputs the
origin of SVs and how/when attackers would take advantage of such SVs to
compromise a system of interest, assisting practitioners to quickly react to
the more easily exploitable or already exploited SVs. The papers in this theme
can be categorized into three groups/sub-themes: (i) Exploit likelihood, (ii)
Exploit time, (iii) Exploit characteristics, as given in Tables 2, 3 and 4,
respectively.

###  3.1. Summary of Primary Studies

####  3.1.1. Exploit Likelihood

The first sub-theme is exploit likelihood that predicts whether SVs would be
exploited in the wild or PoC exploits would be released publicly (see Table
2). In 2010, Bozorgi et al. (Bozorgi et al., 2010) were the first to use SV
descriptions on Common Vulnerabilities and Exposures (CVE) (MITRE, [n. d.]c)
and Open Source Vulnerability Database (OSVDB)444http://osvdb.org. Note that
this database has been discontinued since 2016. to predict exploit existence
based on the labels on OSVDB. In 2015, Sabottke et al. (Sabottke et al., 2015)
conducted a seminal study that used Linear SVM and SV information on Twitter
to predict PoC exploits on ExploitDB (Security, [n. d.]) as well as real-world
exploits on OSVDB, Symantec’s attack signatures (Broadcom, [n. d.]a) and
private Microsoft’s security advisories (Microsoft, [n. d.]). These authors
urged to explicitly consider real-world exploits as not all PoC exploits would
result in exploitation in practice. They also showed SV-related information on
Twitter555https://twitter.com can enable earlier detection of exploits than
using expert-verified SV sources (e.g., NVD).

Built upon these two foundational studies (Bozorgi et al., 2010; Sabottke et
al., 2015), the literature has mainly aimed to improve the performance and
applicability of exploit prediction models by leveraging more exploit sources
and/or better data-driven techniques/practices. Many researchers (Edkrantz et
al., 2015; Edkrantz, 2015; Almukaynizi et al., 2017; Tavabi et al., 2018;
Almukaynizi et al., 2019; Jacobs et al., 2020) increased the amount of ground-
truth exploits using extensive sources other than ExploitDB and Symantec in
(Bozorgi et al., 2010; Sabottke et al., 2015). The sources were security
advisories such as Zero Day Initiative (Micro, [n. d.]b), Metasploit (Rapid7,
[n. d.]), SecurityFocus (Inc, [n. d.]), Recorded Future (Future, [n. d.]),
Kenna Security (Kenna Security, [n. d.]), Avast666https://avast.com/exploit-
protection.php. This link was provided by de Sousa et al. (de Sousa et al.,
2020), but it is no longer available., ESET (ESET, [n. d.]), Trend Micro
(Micro, [n. d.]a), malicious activities in hosts based on traffic of
spam/malicious IP addresses (Xiao et al., 2018) and Darkweb
sites/forums/markets (Nunes et al., 2016). In addition to enriching exploit
sources, better data-driven models and practices for exploit prediction were
also studied. Ensemble models (e.g., Random forest, eXtreme Gradient Boosting
(XGBoost) (Chen and Guestrin, 2016), Light Gradient Boosting Machine (LGBM)
(Ke et al., 2017)) were shown to outperform single-model baselines (e.g.,
Naïve Bayes, SVM, Logistic regression and Decision tree) for exploit
prediction (Fang et al., 2020; Jacobs et al., 2020; de Sousa et al., 2020;
Huang and Wu, 2020). Additionally, Bullough et al. (Bullough et al., 2017)
identified and addressed several issues with exploit prediction models, e.g.,
time sensitivity of SV data, already-exploited SVs before disclosure and
training data imbalance, helping to improve the practical application of such
models. Recently, Yin et al. (Yin et al., 2020) demonstrated that transfer
learning is an alternative solution for improving the performance of exploit
prediction with scarcely labeled exploits. Specifically, these authors pre-
trained a DL model, BERT (Devlin et al., 2018), on massive non-SV sources
(e.g., text on Book Corpus (Zhu et al., 2015) and Wikipedia (Foundation, [n.
d.])) and then fine-tuned this pre-trained model on SV data using additional
pooling and dense layers. Bhatt et al. (Bhatt et al., 2021) also suggested
that incorporating the types of SVs (e.g., SQL injection) into ML models can
further enhance the predictive effectiveness. Suciu et al. (Suciu et al.,
2021) empirically showed that unifying SV-related sources used in prior work
(e.g., SV databases (Bozorgi et al., 2010), social media (Sabottke et al.,
2015), SV-related discussions (Tavabi et al., 2018) and PoC code in ExploitDB
(Jacobs et al., 2020)) supports more effective and timely prediction of
functional exploits (FIRST, [n. d.]d).

Besides using SV descriptions as input for exploit prediction, several studies
in this sub-theme have also predicted exploits on the code level. Younis et
al. (Younis and Malaiya, 2014) predicted the exploitability of vulnerable
functions in the Apache HTTP Server project. Specifically, these authors used
an SVM model with features extracted from the dangerous system calls
(Bernaschi et al., 2002) in entry points/functions (Manadhata and Wing, 2010)
and the reachability from any of these entry points to vulnerable functions
(Horwitz et al., 1990). Moving from high-level to binary code, Yan et al. (Yan
et al., 2017) first used a Decision tree to obtain prior beliefs about SV
types in 100 Linux applications using static features (e.g., hexdump)
extracted from executables. Subsequently, they applied various fuzzing tools
(i.e., Basic Fuzzing Framework (CERT, [n. d.]) and OFuzz (Cha, [n. d.])) to
detect SVs with the ML-predicted types. They finally updated the posterior
beliefs about the exploitability based on the outputs of the ML model and
fuzzers using a Bayesian network. The proposed method outperformed
!exploitable,777https://microsoft.com/security/blog/2013/06/13/exploitable-
crash-analyzer-version-1-6 a static crash analyzer provided by Microsoft.
Tripathi et al. (Tripathi et al., 2017) also predicted SV exploitability from
crashes (i.e., VDiscovery (Cha et al., 2012; Grieco et al., 2016) and LAVA
(Dolan-Gavitt et al., 2016) datasets) using an SVM model and static features
from core dumps and dynamic features generated by the Last Branch Record
hardware debugging utility. Zhang et al. (Zhang and Thing, 2018) proposed two
improvements to Tripathi et al. (Tripathi et al., 2017)’s approach. These
authors first replaced the hardware utility in (Tripathi et al., 2017) that
may not be available for resource-constrained devices (e.g., IoT) with
sequence/$n$-grams of system calls extracted from execution traces. They also
used an online passive-aggressive classifier (Crammer et al., 2006) to enable
online/incremental learning of exploitability for new crash batches on-the-
fly.

Table 3. List of the surveyed papers in the Exploit Time sub-theme of the Exploitation theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Bozorgi et al. 2010 (Bozorgi et al., 2010) |  Binary classification: Likelihood that SVs would be exploited within 2 to 30 days after disclosure |  CVE, OSVDB |  Linear SVM
Edkrantz 2015 (Edkrantz, 2015) |  Binary classification: Likelihood of SV exploits within 12 months after disclosure |  NVD, ExploitDB, Recorded Future security advisories |  SVM, K-Nearest Neighbors (KNN), Naïve Bayes, Random forest
Jacobs et al. 2019 (Jacobs et al., 2019) |  NVD, Kenna Security
Exploit sources: Exploit DB, Metasploit, D2 Security’s Elliot & Canvas Exploitation Frameworks, Fortinet, Proofpoint, AlienVault & GreyNoise |  Logistic regression
Chen et al. 2019 (Chen et al., 2019c, b) |  Binary classification: Likelihood that SVs would be exploited within 1/3/6/9/12 months after disclosure
Regression: number of days until SV exploits after disclosure |  CVE, Twitter, ExploitDB, Symantec security advisories |  Graph neural network embedding + Linear regression, Bayes, Random forest, XGBoost, Lasso/Ridge regression

####  3.1.2. Exploit Time

After predicting the likelihood of SV exploits in the previous sub-theme, this
sub-theme provides more fine-grained information about exploit time (see Table
3). Besides performing binary classification of exploits, Bozorgi et al.
(Bozorgi et al., 2010) and Edkrantz (Edkrantz, 2015) also predicted the time
frame (2-30 days in (Bozorgi et al., 2010) and 12 months in (Edkrantz, 2015))
within which exploits would happen after the disclosure of SVs. Jacobs et al.
(Jacobs et al., 2019) then leveraged multiple sources containing both PoC and
real-world exploits, as given in Table 3, to improve the number of labeled
exploits, enhancing the prediction of exploit appearance within 12 months.
Chen et al. (Chen et al., 2019c) predicted whether SVs would be exploited
within 1-12 months and the exploit time (number of days) after SV disclosure
using Twitter data. The authors proposed a novel regression model whose
feature embedding was a multi-layer graph neural network (Kivelä et al., 2014)
capturing the content and relationships among tweets, respective tweets’
authors and SVs. The proposed model outperformed many baselines and was
integrated into the VEST system (Chen et al., 2019b) to provide timely SV
assessment information for practitioners. To the best of our knowledge, at the
time of writing, Chen et al. (Chen et al., 2019c, b) have been the only ones
pinpointing the exact exploit time of SVs rather than large/uncertain time-
frames (e.g., months) in other studies, helping practitioners to devise much
more fine-grained remediation plans.

####  3.1.3. Exploit Characteristics

Exploit characteristics is the final sub-theme that reveals various
requirements/means of exploits (see Table 4), informing the potential scale of
SVs; e.g., remote exploits likely affect more systems than local ones. The
commonly used outputs are the Exploitability metrics provided by versions 2
(FIRST, [n. d.]b) and 3 (FIRST, [n. d.]c, [n. d.]d) of Common Vulnerability
Scoring System (CVSS).

Table 4. List of the surveyed papers in the Exploit Characteristics sub-theme of the Exploitation theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Yamamoto et al. 2015 (Yamamoto et al., 2015) |  Multi-class classification: CVSS v2 (Access Vector & Access Complexity metrics) Binary classification: CVSS v2 (Authentication metric) |  NVD |  Supervised Latent Dirichlet Allocation (LDA)
Wen et al. 2015 (Wen et al., 2015) |  NVD, OSVDB, SecurityFocus, IBM X-Force |  Radial basis function kernel SVM
Le et al. 2019 (Le et al., 2019) |  NVD |  Concept-drift-aware models with Naïve Bayes, KNN, Linear SVM, Random forest, XGBoost, LGBM
Toloudis et al. 2016 (Toloudis et al., 2016) |  Correlation analysis: CVSS v2 |  NVD |  Principal component analysis & Spearman correlation coefficient
Ognawala et al. 2018 (Ognawala et al., 2018) |  Multi-class classification: CVSS v3 (Attack Vector, Attack Complexity & Privileges Required metrics) Binary classification: CVSS v3 (User Interaction metric) |  NVD (buffer overflow SVs) & Source code of vulnerable software/components |  Combining static analysis tool (Macke (Ognawala et al., 2016)) & ML classifiers (Naïve Bayes & Random forest)
Chen et al. 2019 (Chen et al., 2019b) |  CVE, NVD, Twitter |  Graph convolutional network
Elbaz et al. 2020 (Elbaz et al., 2020) |  Multi-class/Binary classification: CVSS v2/v3 |  NVD |  Mapping outputs of Linear regression to CVSS metrics with closest values
Jiang et al. 2020 (Jiang and Atif, 2020) |  |  NVD, ICS Cert, Vendor websites (Resolve inconsistencies with a majority vote) |  Logistic regression
Gawron et al. 2017 (Gawron et al., 2017) |  Multi-target classification: CVSS v2 |  NVD |  Naïve Bayes, Multi-layer Perceptron (MLP)
Spanos et al. 2018 (Spanos and Angelis, 2018) |  |  NVD |  Random forest, boosting model, Decision tree
Gong et al. 2019 (Gong et al., 2019) |  Multi-task classification: CVSS v2 |  NVD |  Bi-LSTM with attention mechanism
Chen et al. 2010 (Chen et al., 2010) |  Multi-class classification: Platform-specific vulnerability locations (Local, Remote, Local area network) & vulnerability causes (e.g., Access/Input/Origin validation error) |  NVD, Secunia vulnerability database, SecurityFocus, IBM X-Force |  Linear SVM
Ruohonen et al. 2017 (Ruohonen, 2017) |  Binary classification: Web-related exploits or not |  ExploitDB |  LDA + Random forest
Aksu et al. 2018 (Aksu et al., 2018) |  Multi-class classification: author-defined pre-/post-condition privileges (None, OS (Admin/User), App (Admin/User)) |  NVD |  RBF network, Linear SVM, NEAT (Stanley and Miikkulainen, 2002), MLP
Liu et al. 2019 (Liu and Li, 2019) |  |  NVD |  Information gain + Convolutional neural network
Kanakogi et al. 2021 (Kanakogi et al., 2021) |  Multi-class classification: Common Attack Pattern Enumeration and Classification (CAPEC) |  NVD, CAPEC |  Doc2vec/tf-idf with cosine similarity

Many studies have focused on predicting and analyzing version 2 of CVSS
exploitability metrics (i.e., Access Vector, Access Complexity and
Authentication). Yamamoto et al. (Yamamoto et al., 2015) were the first one to
leverage descriptions of SVs on NVD together with a supervised Latent
Dirichlet Allocation topic model (Blei and McAuliffe, 2007) to predict these
CVSS metrics. Subsequently, Wen et al. (Wen et al., 2015) used Radial Basis
Function (RBF)-kernel SVM and various SV databases/advisories other than NVD
(e.g., SecurityFocus, OSVDB and IBM X-Force (Services, [n. d.])) to predict
the metrics. Le et al. (Le et al., 2019) later showed that the prediction of
CVSS metrics suffered from the concept drift issue; i.e., descriptions of new
SVs may contain Out-of-Vocabulary terms for prediction models. They proposed
to combine sub-word features with traditional Bag-of-Word (BoW) features to
infer the semantics of novel terms/words from existing ones, helping
assessment models be more robust against concept drift. Besides prediction,
Toloudis et al. (Toloudis et al., 2016) used principal component analysis
(Wold et al., 1987) and Spearman’s $\rho$ correlation coefficient to reveal
the predictive contribution of each word in SV descriptions to each CVSS
metric. However, this technique does not directly produce the value of each
metric.

Recently, several studies have started to predict CVSS version 3
exploitability metrics including the new Privileges and User Interactions.
Ognawala et al. (Ognawala et al., 2018) fed the features generated by a static
analysis tool, Macke (Ognawala et al., 2016), to a Random forest model to
predict these CVSS version 3 metrics for vulnerable software/components.
Later, Chen et al. (Chen et al., 2019b) found that many SVs were disclosed on
Twitter before on NVD. Therefore, these authors developed a system built on
top of a Graph Convolutional Network (Kipf and Welling, 2016) capturing the
content and relationships of related Twitter posts about SVs to enable more
timely prediction of the CVSS version 3 metrics. Elbaz et al. (Elbaz et al.,
2020) developed a linear regression model to predict the numerical output of
each metric and then obtained the respective categorical value with the
numerical value closest to the predicted value. For example, a predicted value
of 0.8 for Attack Vector CVSS v3 is mapped to Network (0.85) (FIRST, [n.
d.]c). To prepare a clean dataset to predict these CVSS metrics, Jiang et al.
(Jiang and Atif, 2020) replaced inconsistent CVSS values in various SV sources
(i.e., NVD, ICS CERT and vendor websites) with the most frequent value.

Instead of building a separate model for each CVSS metric, there has been
another family of approaches predicting these metrics using a single model to
increase efficiency. Gawron et al. (Gawron et al., 2017) and Spanos et al.
(Spanos and Angelis, 2018) predicted multiple CVSS metrics as a unique string
instead of individual values. The output of each metric is then extracted from
the concatenated string. Later, Gong et al. (Gong et al., 2019) adopted the
idea of a unified model from the DL perspective by using the multi-task
learning paradigm (Zhang and Yang, 2021) to predict CVSS metrics
simultaneously. The model has a feature extraction module (based on a Bi-LSTM
model with attention mechanism (Bahdanau et al., 2014)) shared among all the
CVSS metrics/tasks, yet specific prediction head/layer for each metric/task.
This model outperformed single-task counterparts while requiring much less
time to (re-)train.

Although CVSS exploitability metrics were most commonly used, several studies
used other schemes for characterizing exploitation. Chen et al. (Chen et al.,
2010) used Linear SVM and SV descriptions to predict multiple SV
characteristics, including three SV locations (i.e., Local, LAN and Remote) on
SecurityFocus (Inc, [n. d.]) and Secunia (Inc., [n. d.]) databases as well as
11 SV causes888Access/Input/Origin validation error,
Atomicity/Configuration/Design/Environment/Serialization error, Boundary
condition error, Failure on exceptions, Race condition error on SecurityFocus.
Regarding the exploit types, Rouhonen et al. (Ruohonen, 2017) used LDA (Blei
et al., 2003) and Random forest to classify whether an exploit would affect a
web application. This study can help find relevant exploits in components/sub-
systems of a large system. For privileges, Aksu et al. (Aksu et al., 2018)
extended the Privileges Required metric of CVSS by incorporating the context
(i.e., Operating system or Application) to which privileges are applied (see
Table 4). They found MLP (Hastie et al., 2009) to be the best-performing model
for obtaining these privileges from SV descriptions. They also utilized the
predicted privileges to generate attack graphs (sequence of attacks from
source to sink nodes). Liu et al. (Liu and Li, 2019) advanced this task by
combining information gain for feature selection and Convolutional Neural
Network (CNN) (Kim, 2014) for feature extraction. Regarding attack patterns,
Kanakogi et al. (Kanakogi et al., 2021) found Doc2vec (Le and Mikolov, 2014)
to be more effective than term-frequency inverse document frequency (tf-idf)
when combined with cosine similarity to find the most relevant Common Attack
Pattern Enumeration and Classification (CAPEC) (MITRE, [n. d.]a) for a given
SV on NVD. Such attack patterns can manifest how identified SVs can be
exploited by adversaries, assisting the selection of suitable countermeasures.

###  3.2. Theme Discussion

In the Exploitation theme, the primary tasks are binary classification of
whether Proof-of-Concept (PoC)/real-world exploits of SVs would appear and
multi-classification of exploit characteristics based on CVSS. PoC exploits
mostly come from ExploitDB (Security, [n. d.]); whereas, real-world exploits,
despite coming from multiple sources, are still much scarcer than PoC
counterparts. Consequently, the models predicting real-world exploits have
generally performed worse than those for PoC exploits. Similarly, the
performance of the models determining CVSS v3 exploitability metrics has been
mostly lower than that of the CVSS v2 based models. However, real exploits and
CVSS v3 are usually of more interest to the community. The former can lead to
real cyber-attacks and the latter is the current standard in practice. To
improve the performance of these practical tasks, future work can collect more
exploit-related data from the relevant yet under-explored sources (see section
9.1), as well as adapt the patterns learned from PoC exploits and old CVSS
versions to real exploits and newer CVSS versions, respectively, e.g., using
transfer learning (Pan and Yang, 2009).

There are other under-explored tasks targeting fine-grained prediction of
exploits. Mitigation of exploits in practice usually requires more information
besides simply determining whether an SV would be exploited. Information
gathered from predicting when and how the exploits would happen is also needed
to devise better SV fixing prioritization and mitigation plans. VEST (Chen et
al., 2019b) is one of the first and few systems aiming to provide such all-in-
one information about SV exploitation. However, this system currently only
uses data from NVD/CVE and Twitter, which can be extended to incorporate more
(exploit) sources and more sophisticated data-driven techniques in the future.

Most of the current studies have used SV descriptions on NVD and other
security advisories to predict the exploitation-related metrics. This is
surprising as SV descriptions do not contain root causes of SVs. Instead, SVs
are rooted in source code, yet there is little work on code-based exploit
prediction. So far, Younis et al. (Younis and Malaiya, 2014) have been among
the few ones using source code for exploit prediction, but their approach
still requires manual identification of dangerous function calls in C/C++.
More work is required to employ data-driven approaches to alleviate the need
for manually defined rules to improve the effectiveness and generalizability
of code-based exploit prediction.

##  4\. Impact Prediction

This section describes the Impact theme that determines the (negative) effects
that SVs have on a system of interest if such SVs are exploited. There are
five key tasks that the papers in this theme have automated/predicted: (i)
Confidentiality impact, (ii) Integrity impact, (iii) Availability impact, (iv)
Scope and (v) Custom vulnerability consequences (see Table 5).

Table 5. List of the surveyed papers in the Impact theme. Note: We grouped the first four sub-themes as they were mostly predicted together. Study | Nature of task | Data source | Data-driven technique
---|---|---|---
Sub-themes: 1. Confidentiality, 2. Integrity, 3. Availability & 4\. Scope
(only in CVSS v3)
Yamamoto et al. 2015 (Yamamoto et al., 2015) |  Multi-class classification: CVSS v2 |  NVD |  Supervised Latent Dirichlet Allocation
Wen et al. 2015 (Wen et al., 2015) |  |  NVD, OSVDB, SecurityFocus, IBM X-Force |  Radial basis function kernel SVM
Le et al. 2019 (Le et al., 2019) |  |  NVD |  Concept-drift-aware models with Naïve Bayes, KNN, Linear SVM, Random forest, XGBoost, LGBM
Toloudis et al. 2016 (Toloudis et al., 2016) |  Correlation analysis: CVSS v2 |  NVD |  Principal component analysis & Spearman correlation coefficient
Ognawala et al. 2018 (Ognawala et al., 2018) |  Multi-class classification: CVSS v3 Binary classification: Scope in CVSS v3 |  NVD (buffer overflow SVs) & Source code of vulnerable software/components |  Combining static analysis tool (Macke (Ognawala et al., 2016)) & ML classifiers (Naïve Bayes & Random forest)
Chen et al. 2019 (Chen et al., 2019b) |  CVE, NVD, Twitter |  Graph convolutional network
Elbaz et al. 2020 (Elbaz et al., 2020) |  Multi-class classification: CVSS v2/v3 Binary classification: Scope in CVSS v3 |  NVD |  Mapping outputs of Linear regression outputs to CVSS metrics with closest values
Jiang et al. 2020 (Jiang and Atif, 2020) |  NVD, ICS Cert, Vendor websites (Resolve inconsistencies with a majority vote) |  Logistic regression
Gawron et al. 2017 (Gawron et al., 2017) |  Multi-target classification: CVSS v2 |  NVD |  Naïve Bayes, MLP
Spanos et al. 2018 (Spanos and Angelis, 2018) |  |  NVD |  Random forest, boosting model, Decision tree
Gong et al. 2019 (Gong et al., 2019) |  Multi-task classification: CVSS v2 |  NVD |  Bi-LSTM with attention mechanism
Sub-theme: 5. Custom Vulnerability Consequences
Chen et al. 2010 (Chen et al., 2010) |  Multi-label classification: Platform-specific impacts (e.g., Gain system access) |  NVD, Secunia vulnerability database, SecurityFocus, IBM X-Force |  Linear SVM

###  4.1. Summary of Primary Studies

####  4.1.1. Confidentiality, Integrity, Availability, and Scope

A majority of the papers have focused on the impact metrics provided by CVSS,
including versions 2 (FIRST, [n. d.]b) and 3 (FIRST, [n. d.]c, [n. d.]d).
Versions 2 and 3 share three impact metrics Confidentiality, Integrity and
Availability. Version 3 also has a new metric, Scope, that specifies whether
an exploited SV would affect only the system that contains the SV. For
example, Scope changes when an SV occurring in a virtual machine affects the
whole host machine, in turn increasing individual impacts.

The studies that predicted the CVSS impact metrics are mostly the same as the
ones predicting the CVSS exploitability metrics in section 3. Given the
overlap, we hereby only describe the main directions and techniques of the
Impact-related tasks rather than iterating the details of each study. Overall,
a majority of the work has focused on classifying CVSS impact metrics
(versions 2 and 3) using three main learning paradigms: single-task (Yamamoto
et al., 2015; Wen et al., 2015; Le et al., 2019; Ognawala et al., 2018; Chen
et al., 2019b; Elbaz et al., 2020; Jiang and Atif, 2020), multi-target (Gawron
et al., 2017; Spanos and Angelis, 2018) and multi-task (Gong et al., 2019)
learning. Instead of developing a separate prediction model for each metric
like the single-task approach, multi-target and multi-task approaches only
need a single model for all tasks. Multi-target learning predicts concatenated
output; whereas, multi-task learning uses shared feature extraction for all
tasks and task-specific softmax layers to determine the output of each task.
These three learning paradigms were powered by applying and/or customizing a
wide range of data-driven methods. The first method was to use single ML
classifiers like supervised Latent Dirichlet Allocation (Yamamoto et al.,
2015), Principal component analysis (Toloudis et al., 2016), Naïve Bayes (Le
et al., 2019; Ognawala et al., 2018; Gawron et al., 2017), Logistic regression
(Jiang and Atif, 2020), Kernel-based SVM (Wen et al., 2015), Linear SVM (Le et
al., 2019), KNN (Le et al., 2019) and Decision tree (Spanos and Angelis,
2018). Other studies employed ensemble models combining the strength of
multiple single models such as Random forest (Le et al., 2019; Ognawala et
al., 2018), boosting model (Spanos and Angelis, 2018) and XGBoost/LGBM (Le et
al., 2019). Recently, more studies moved towards more sophisticated DL
architectures such as MLP (Gawron et al., 2017), attention-based (Bi-)LSTM
(Gong et al., 2019) and graph neural network (Chen et al., 2019b). Ensemble
and DL models usually beat the single ones, but there is a lack of direct
comparisons between these two emerging model types.

####  4.1.2. Custom Vulnerability Consequences

To devise effective remediation strategies for a system of interest in
practice, practitioners may want to know custom vulnerability consequences
which are more interpretable than the levels of impact provided by CVSS. Chen
et al. (Chen et al., 2010) curated a list of 11 vulnerability
consequences999Gain system access, Bypass security, Configuration
manipulation, Data/file manipulation, Denial of Service, Privilege escalation,
Information leakage, Session hijacking, Cross-site scripting (XSS), Source
spoofing, Brute-force proneness. from X-Force (Services, [n. d.]) and Secunia
(Inc., [n. d.]) vulnerability databases. They then used a Linear SVM model to
perform multi-label classification of these consequences for SVs, meaning that
an SV can lead to more than one consequence. To the best of our knowledge,
this is the only study that has pursued this research direction so far.

###  4.2. Theme Discussion

In the Impact theme, the common task is to predict the impact base metrics
provided by CVSS versions 2 and 3. Similar to the Exploitation theme, the
models for CVSS v3 still require more attention and effort from the community
to reach the same performance level as the models for CVSS v2. These impact
metrics are also usually predicted together with the exploitability metrics
given their similar nature (multi-class classification) using either task-wise
models or a unified (multi-target or multi-task) model. Multi-target and
multi-task learning are promising as they can reduce the time for continuous
(re)training and maintenance when deployed in production.

Besides CVSS impact metrics, other fine-grained SV consequences have also been
explored (Chen et al., 2010), but there is still no widely accepted taxonomy
for such consequences. Thus, these consequences have seen less adoption in
practice than CVSS metrics, despite being potentially useful by providing more
concrete information about what assets/components in a system that an SV can
compromise. In section 9.2.1, we suggest potential ways to create a systematic
taxonomy of custom SV consequences to pave the way for more data-driven
research in this direction.

##  5\. Severity Prediction

This section discusses the work in the Severity theme. Severity is often a
function/combination of Exploitation (section 3) and Impact (section 4). SVs
with higher severity usually require more urgent remediation. There are three
main prediction tasks in this theme: (i) Severe vs. Non-severe, (ii) Severity
levels and (iii) Severity score, shown in Tables 6, 7 and 8, respectively.

Similar to the Exploitation and Impact themes, many studies in the Severity
theme have used CVSS versions 2 and 3. According to both CVSS versions, the
severity score shares the same range from 0 to 10, with an increment of 0.1.
Based on the score, the existing studies have either defined a threshold to
decide whether an SV is severe (requiring high attention), or predicted
levels/groups of severity score that require a similar amount of attention or
determined the raw score value.

###  5.1. Summary of Primary Studies

####  5.1.1. Severe vs. Non-Severe

Table 6. List of the surveyed papers in the Severe vs. Non-Severe sub-theme of the Severity theme. Note: The nature of task here is binary classification of severe SVs with High/Critical CVSS v2/v3 severity levels.  Study |  Data source (software project) |  Data-driven technique
---|---|---
Kudjo et al. 2019 (Kudjo et al., 2019) |  NVD (Mozilla Firefox, Google Chrome, Internet Explorer, Microsoft Edge, Sea Monkey, Linux Kernel, Windows 7, Windows 10, Mac OS, Chrome OS) |  Term frequency & inverse gravity moment weighting + KNN, Decision tree, Random forest
Chen et al. 2020 (Chen et al., 2020a) |  NVD (Adobe Flash Player, Enterprise Linux, Linux Kernel, Foxit Reader, Safari, Windows 10, Microsoft Office, Oracle Business Suites, Chrome, QuickTime) |  Term frequency & inverse gravity moment weighting + KNN, Decision tree, Naïve Bayes, SVM, Random forest
Kudjo et al. 2020 (Kudjo et al., 2020) |  NVD (Google Chrome, Mozilla Firefox, Internet Explorer and Linux Kernel) |  Find the best smallest training dataset using KNN, Logistic regression, MLP, Random forest
Malhotra et al. 2021 (Malhotra et al., 2021) |  NVD (Apache Tomcat) |  Chi-square/Information gain + bagging technique, Random forest, Naïve Bayes, SVM

The first group of studies have classified whether an SV is severe or non-
severe, making it a binary classification problem (see Table 6). These studies
have typically selected severe SVs as the ones with at least High severity
level (i.e., CVSS severity score $\geq$ 7.0). Kudjo et al. (Kudjo et al.,
2019) showed that using term frequency (BoW) with inverse gravity moment
weighting (Chen et al., 2016) to extract features from SV descriptions can
enhance the performance of ML models (i.e., KNN, Decision tree and Random
forest) in predicting the severity of SVs. Later, Chen et al. (Chen et al.,
2020a) confirmed that this feature extraction method was also effective for
more projects and classifiers (e.g., Naïve Bayes and SVM). Besides
investigating feature extraction, Kudjo et al. (Kudjo et al., 2020) also
highlighted the possibility of finding Bellwether, i.e., the smallest set of
data that can be used to train an optimal prediction model, for classifying
severity. Recently, Malhotra et al. (Malhotra et al., 2021) revisited this
task by showing that Chi-square and information gain can be effective
dimensionality reduction techniques for multiple classifiers, i.e., bagging
technique, Random forest, Naïve Bayes and SVM.

####  5.1.2. Severity Levels

Table 7. List of the surveyed papers in the Severity Levels sub-theme of the Severity theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Spanos et al. 2017 (Spanos et al., 2017) |  Multi-class classification: NVD severity levels based on CVSS v2 & WIVSS (High, Medium, Low) |  NVD |  Decision tree, SVM, MLP
Wang et al. 2019 (Wang et al., 2019) |  Multi-class classification: NVD severity levels based on CVSS v2 (High, Medium, Low) |  NVD (XSS attacks) |  XGBoost, Logistic regression, SVM, Random forest
Le et al. 2019 (Le et al., 2019) |  NVD |  Concept-drift-aware models with Naïve Bayes, KNN, Linear SVM, Random forest, XGBoost, LGBM
Liu et al. 2019 (Liu et al., 2019) |  |  NVD, China National Vulnerability Database (XSS attacks) |  Recurrent Convolutional Neural Network (RCNN), Convolutional Neural Network (CNN), Long-Short Term Memory (LSTM)
Sharma et al. 2020 (Sharma et al., 2021) |  |  CVE Details |  CNN
Han et al. 2017 (Han et al., 2017) |  Multi-class classification: Atlassian categories of CVSS severity score (Critical, High, Medium, Low) |  CVE Details |  1-layer CNN, 2-layer CNN, CNN-LSTM, Linear SVM
Sahin et al. 2019 (Sahin and Tosun, 2019) |  NVD |  1-layer CNN, LSTM, XGBoost, Linear SVM
Nakagawa et al. 2019 (Nakagawa et al., 2019) |  |  CVE Details |  Character-level CNN vs. Word-based CNN + Linear SVM
Gong et al. 2019 (Gong et al., 2019) |  Multi-task classification: Atlassian categories of CVSS severity score (Critical, High, Medium, Low) |  CVE Details |  Bi-LSTM with attention mechanism
Chen et al. 2010 (Chen et al., 2010) |  Multi-class classification: severity levels of Secunia (Extremely/highly/ moderately/less/non- critical) |  CVE, Secunia vulnerability database, SecurityFocus, IBM X-Force |  Linear SVM
Zhang et al. 2020 (Zhang et al., 2020b) |  Multi-class classification: Platform-specific levels (High/Medium/Low) |  China National Vulnerability Database |  Logistic regression, Linear discriminant analysis, KNN, CART, SVM, bagging/boosting models
Khazaei et al. 2016 (Khazaei et al., 2016) |  Multi-class classification: 10 severity score bins (one unit/bin) |  CVE & OSVDB |  Linear SVM, Random forest, Fuzzy system

Rather than just performing binary classification of whether an SV is severe,
several studies have identified one among multiple severity levels that an SV
belongs to (see Table 7). This setting can be considered as multi-class
classification. Spanos et al. (Spanos et al., 2017) were to first one to show
the applicability of ML to classify SVs into one of the three severity levels
using SV descriptions. These three levels are provided by NVD and based on the
severity score of CVSS version 2 (FIRST, [n. d.]b) and WIVSS (Spanos et al.,
2013), i.e., Low (0.0 – 3.9), Medium (4.0 – 6.9), High (7.0 – 10.0). Note that
WIVSS assigns different weights for the Confidentiality, Integrity and
Availability impact metrics of CVSS, enhancing the ability to capture varied
contributions of these impacts to the final severity score. Later, Wang et al.
(Wang et al., 2019) showed that XGBoost (Chen and Guestrin, 2016) performed
the best among the investigated ML classifiers for predicting these three NVD-
based severity levels. Le et al. (Le et al., 2019) also confirmed that
ensemble methods (e.g., XGBoost (Chen and Guestrin, 2016), LGBM (Ke et al.,
2017) and Random forest) outperformed single models (e.g., Naïve Bayes, KNN
and SVM) for this task. Predicting severity levels has also been tackled with
DL techniques (Liu et al., 2019; Sharma et al., 2021) such as Recurrent
Convolutional Neural Network (RCNN) (Lai et al., 2015), Convolutional Neural
Network (CNN) (Kim, 2014), Long-Short Term Memory (LSTM) (Hochreiter and
Schmidhuber, 1997). These studies showed potential performance gain of DL
models compared to traditional ML counterparts. Han et al. (Han et al., 2017)
showed that DL techniques (i.e., 1-layer CNN) also achieved promising results
for predicting a different severity categorization, namely Atlassian’s
levels.101010https://www.atlassian.com/trust/security/security-severity-levels
Such findings were successfully replicated by Sahin et al. (Sahin and Tosun,
2019). Nakagawa et al. (Nakagawa et al., 2019) further enhanced the DL model
performance for the same task by incorporating the character-level features
into a CNN model (Zhang et al., 2015). Complementary to performance
enhancement, Gong et al. (Gong et al., 2019) proposed to predict these
severity levels concurrently with other CVSS metrics in a single model using
multi-task learning (Zhang and Yang, 2021) powered by an attention-based Bi-
LSTM shared feature extraction model. The unified model was demonstrated to
increase both the prediction effectiveness and efficiency. Besides Atlassian’s
categories, several studies applied ML models to predict severity levels on
other platforms such as Secunia (Chen et al., 2010) and China National
Vulnerability Database111111https://www.cnvd.org.cn (Zhang et al., 2020b).
Instead of using textual categories, Khazaei et al. (Khazaei et al., 2016)
divided the CVSS severity score into 10 bins with 10 increments each (e.g.,
values of 0 – 0.9 are in one bin) and obtained decent results (86-88%
Accuracy) using Linear SVM, Random forest and Fuzzy system.

####  5.1.3. Severity Score

Table 8. List of the surveyed papers in the Severity Score sub-theme of the Severity theme. Notes: †denotes that the severity score is computed from ML-predicted base metrics using the formula provided by an assessment framework (CVSS and/or WIVSS). Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Sahin et al. 2019 (Sahin and Tosun, 2019) |  Regression: CVSS v2 (0-10) |  NVD |  1-layer CNN, LSTM, XGBoost regressor, Linear regression
Wen et al. 2015 (Wen et al., 2015) |  |  OSVDB, SecurityFocus, IBM X-Force |  Radial basis function kernel SVM†
Ognawala et al. 2018 (Ognawala et al., 2018) |  Regression: CVSS v3 (0-10) |  NVD (buffer overflow SVs) |  Combining a static analysis tool (Macke (Ognawala et al., 2016)) & ML classifiers (Naïve Bayes & Random forest)†
Chen et al. 2019 (Chen et al., 2019b, a) |  |  CVE, NVD, Twitter |  Graph convolutional network
Anwar et al. 2020 (Anwar et al., 2020) |  |  NVD |  Linear regression, Support vector regression, CNN, MLP
Elbaz et al. 2020 (Elbaz et al., 2020) |  Regression: CVSS v2/v3 (0-10) |  NVD |  Mapping outputs of Linear regression to CVSS metrics with closest values†
Jiang et al. 2020 (Jiang and Atif, 2020) |  |  NVD, ICS Cert, Vendor websites (Resolve inconsistencies with a majority vote) |  Logistic regression†
Spanos et al. 2018 (Spanos and Angelis, 2018) |  Regression: CVSS v2 & WIVSS (0-10) |  NVD |  Random forest, boosting model, Decision tree†
Toloudis et al. 2016 (Toloudis et al., 2016) |  Correlation analysis: CVSS v2 & WIVSS (0-10) |  NVD |  Principal component analysis & Spearman correlation coefficient

To provide even more fine-grained severity value than the categories, the last
sub-theme has predicted the severity score (see Table 8). Using SV
descriptions on NVD, Sahin et al. (Sahin and Tosun, 2019) compared the
performance of ML-based regressors (e.g., XGBoost (Chen and Guestrin, 2016)
and Linear regression) and DL-based ones (e.g., CNN (Kim, 2014) and LSTM
(Hochreiter and Schmidhuber, 1997)) for predicting the severity score of CVSS
version 2 (FIRST, [n. d.]b). These authors showed that DL-based approaches
generally outperformed the ML-based counterparts. For CVSS version 3 (FIRST,
[n. d.]c, [n. d.]d), Chen et al. (Chen et al., 2019b, a) and Anwar et al.
(Anwar et al., 2020) also reported the strong performance of DL-based models
(e.g., CNN and graph convolutional neural network (Kipf and Welling, 2016)).
Some other studies did not directly predict severity score from SV
descriptions, instead they aggregated the predicted values of the CVSS
Exploitability (see section 3) and Impact metrics (see section 4) using the
formulas of CVSS version 2 (Wen et al., 2015; Spanos and Angelis, 2018; Elbaz
et al., 2020; Jiang and Atif, 2020), version 3 (Ognawala et al., 2018; Elbaz
et al., 2020; Jiang and Atif, 2020) and WIVSS (Spanos and Angelis, 2018). We
noticed the papers predicting both versions (e.g., CVSS versions 2 vs. 3 or
CVSS version 2 vs. WIVSS) usually obtained better performance for version 3
and WIVSS than version 2 (Elbaz et al., 2020; Jiang and Atif, 2020). These
findings may suggest that the improvements made by experts in version 3 and
WIVSS compared to version 2 help make the patterns in severity score clearer
and easier for ML models to capture. In addition to predicting severity score,
Toloudis et al. (Toloudis et al., 2016) examined the correlation between words
in descriptions of SVs and the severity values of such SVs, aiming to shed
light on words that increase or decrease the severity score of SVs.

###  5.2. Theme Discussion

In the Severity theme, predicting the severity levels is the most prevalent
task, followed by severity score prediction and then binary classification of
the severity. In practice, severity score gives more fine-grained information
(fewer SVs per value) for practitioners to rank/prioritize SVs than
categorical/binary levels. However, predicting continuous score values is
usually challenging and requires more robust models as this task involves
higher uncertainty to learn inherent patterns from data than classifying
fixed/discrete levels. We observed that DL models such as graph neural
networks (Chen et al., 2019b, a), LSTM (Sahin and Tosun, 2019) and CNN (Anwar
et al., 2020) have been shown to be better than traditional ML models for
predicting severity score. However, most of these studies did not evaluate
their models in a continuous deployment setting to investigate how the models
will cope with changing patterns of new SVs over time. We distill
recommendations for future work on real-world application and evaluation of
these models in section 9.2.

##  6\. Type Prediction

Table 9. List of the surveyed papers in the Type theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Sub-theme: 1. Common Weakness Enumeration (CWE)
Wang et al. 2010 (Wang and Guo, 2010) |  Multi-class classification: CWE classes |  NVD, CVSS |  Naïve Bayes
Shuai et al. 2013 (Shuai et al., 2013) |  |  NVD |  SVM
Na et al. 2016 (Na et al., 2016) |  |  NVD |  Naïve Bayes
Ruohonen et al. 2018 (Ruohonen and Leppänen, 2018) |  |  NVD, CWE, Snyk |  tf-idf with 1/2/3-grams and cosine similarity
Huang et al. 2019 (Huang et al., 2019) |  |  NVD, CWE |  MLP, Linear SVM, Naïve Bayes, KNN
Aota et al. 2020 (Aota et al., 2020) |  |  NVD |  Random forest, Linear SVM, Logistic regression, Decision tree, Extremely randomized trees, LGBM
Aghaei et al. 2020 (Aghaei et al., 2020) |  |  NVD, CVE |  Adaptive fully-connected neural network with one hidden layer
Das et al. 2021 (Das et al., 2021) |  |  NVD, CWE |  BERT, Deep Siamese network
Zou et al. 2019 (Zou et al., 2019) |  |  NVD & Software Assurance Reference Dataset (SARD) |  Three Bi-LSTM models for extracting and combining global and local features from code functions
Murtaza et al. 2016 (Murtaza et al., 2016) |  Unsupervised learning: sequence mining of SV types (over time) |  NVD (CWE & CPE) |  2/3/4/5-grams of CWEs
Lin et al. 2017 (Lin et al., 2017) |  Unsupervised learning: association rule mining of CWE-related aspects (prog. language, time of introduction & consequence scope) |  CWE |  FP-growth association rule mining algorithm
Han et al. 2018 (Han et al., 2018) |  Binary/Multi-class classification: CWE relationships (CWE links, link types & CWE consequences) |  CWE |  Deep knowledge graph embedding of CWE entities
Sub-theme: 2. Custom Vulnerability Types
Venter et al. 2008 (Venter et al., 2008) |  Unsupervised learning: clustering |  CVE |  Self-organizing map
Neuhaus et al. 2010 (Neuhaus and Zimmermann, 2010) |  Unsupervised learning: topic modeling |  CVE |  Latent Dirichlet Allocation (LDA)
Mounika et al. (Mounika et al., 2019; Vanamala et al., 2020) |  |  CVE, Open Web Application Security Project (OWASP) |  LDA
Aljedaani et al. 2020 (Aljedaani et al., 2020) |  |  SV reports (Chromium project) |  LDA
Williams et al. (Williams et al., 2018, 2020) |  Multi-class classification: manually coded SV types |  NVD |  Supervised Topical Evolution Model & Diffusion-based storytelling technique
Russo et al. 2019 (Russo et al., 2019) |  NVD |  Bayesian network, J48 tree, Logistic regression, Naïve Bayes, Random forest
Yan et al. 2017 (Yan et al., 2017) |  |  Executables of 100 Linux applications |  Decision tree
Zhang et al. 2020 (Zhang et al., 2020b) |  Multi-class classification: platform-specific vulnerability types |  China National Vulnerability Database |  Logistic regression, Linear discriminant analysis, KNN, CART, SVM, bagging/boosting models

This section reports the work done in the Type theme. Type groups SVs with
similar characteristics, e.g., causes, attack patterns and impacts, and thus
facilitating the reuse of known prioritization and remediation strategies
employed for prior SVs of the same types. Two key prediction outputs are: (i)
Common Weakness Enumeration (CWE) and (ii) Custom vulnerability types (see
Table 9).

###  6.1. Summary of Primary Studies

####  6.1.1. Common Weakness Enumeration (CWE)

The first sub-theme determines and analyzes the patterns of the SV types
provided by CWE (MITRE, [n. d.]d). CWE is currently the standard for SV types
with more than 900 entries. The first group of studies has focused on multi-
class classification of these CWEs. Wang et al. (Wang and Guo, 2010) were the
first to tackle this problem with a Naïve Bayes model using the CVSS metrics
(version 2) (FIRST, [n. d.]b) and product names. Later, Shuai et al. (Shuai et
al., 2013) used LDA (Blei et al., 2003) with a location-aware weighting to
extract important features from SV descriptions for building an effective SVM-
based CWE classifier. Na et al. (Na et al., 2016) also showed that features
extracted from SV descriptions can improve the Naïve Bayes model in (Wang and
Guo, 2010). Ruohonen et al. (Ruohonen and Leppänen, 2018) studied an
information retrieval method, i.e., term-frequency inverse document frequency
(tf-idf) and cosine similarity, to detect the CWE-ID with a description most
similar to that of a given SV collected from NVD and
Snyk.121212https://snyk.io/vuln This method performed well for CWEs without
clear patterns/keywords in SV descriptions. Aota et al. (Aota et al., 2020)
utilized the Boruta feature selection algorithm (Kursa et al., 2010) and
Random forest to improve the performance of base CWE classification. Base CWEs
give more fine-grained information for SV remediation than categorical CWEs
used in (Na et al., 2016).

There has been a recent rise in using neural network/DL based models for CWE
classification. Huang et al. (Huang et al., 2019) implemented a deep neural
network with tf-idf and information gain for the task and obtained better
performance than SVM, Naïve Bayes and KNN. Aghaei et al. (Aghaei et al., 2020)
improved upon (Aota et al., 2020) for both categorical (coarse-grained) and
base (fine-grained) CWE classification with an adaptive hierarchical neural
network to determine sequences of less to more fine-grained CWEs. To capture
the hierarchical structure and rare classes of CWEs, Das et al. (Das et al.,
2021) matched SV and CWE descriptions instead of predicting CWEs directly.
They presented a deep Siamese network with a BERT-based (Devlin et al., 2018)
shared feature extractor that outperformed many baselines even for rare/unseen
CWE classes. Recently, Zou et al. (Zou et al., 2019) pioneered the multi-class
classification of CWE in vulnerable functions curated from Software Assurance
Reference Dataset (SARD) (NIST, [n. d.]b) and NVD. They achieved high
performance ($\sim$95% F1-score) with DL (Bi-LSTM) models. The strength of
their model came from combining global (semantically related statements) and
local (variables/statements affecting function calls) features. Note that this
model currently only works for functions in C/C++ and 40 selected classes of
CWE.

Another group of studies has considered unsupervised learning methods to
extract CWE sequences, patterns and relationships. Sequences of SV types over
time were identified by Murtaza et al. (Murtaza et al., 2016) using an
$n$-gram model. This model sheds light on both co-occurring and upcoming CWEs
(grams), raising awareness of potential cascading attacks. Lin et al. (Lin et
al., 2017) applied an association rule mining algorithm, FP-growth (Han et
al., 2000), to extract the rules/patterns of various CWEs aspects including
types, programming language, time of introduction and consequence scope. For
example, buffer overflow (CWE type) usually appears during the implementation
phase (time of introduction) in C/C++ (programming language) and affects the
availability (consequence scope). Lately, Han et al. (Han et al., 2018)
developed a deep knowledge graph embedding technique to mine the relationships
among CWE types, assisting in finding relevant SV types with similar
properties.

####  6.1.2. Custom Vulnerability Types

The second sub-theme is about custom vulnerability types other than CWE.
Venter et al. (Venter et al., 2008) used Self-organizing map (Kohonen, 1990),
an unsupervised clustering algorithm, to group SVs with similar descriptions
on CVE. This was one of the earliest studies that automated SV type
classification. Topic modeling is another popular unsupervised learning model
(Neuhaus and Zimmermann, 2010; Mounika et al., 2019; Vanamala et al., 2020;
Aljedaani et al., 2020) to categorize SVs without an existing taxonomy.
Neuhaus et al. (Neuhaus and Zimmermann, 2010) applied LDA (Blei et al., 2003)
on SV descriptions to identify 28 prevalent SV types and then analyzed the
trends of such types over time. The identified SV topics/types had
considerable overlaps (up to 98% precision and 95% recall) with CWEs. Mounika
et al. (Mounika et al., 2019; Vanamala et al., 2020) extended (Neuhaus and
Zimmermann, 2010) to map the LDA topics with the top-10 OWASP (OWASP, [n.
d.]). However, the LDA topics/keywords did not agree well ($<$ 40%) with the
OWASP descriptions, probably because 10 topics did not cover all the
underlying patterns of SV descriptions. Aljedaani et al. (Aljedaani et al.,
2020) again used LDA to identify 10 types of SVs reported in the bug tracking
system of Chromium131313https://bugs.chromium.org/p/chromium/issues/list and
found memory-related issues were the most prevalent topics.

Another group of studies has classified manually defined/selected SV types
rather than CWE as some SV types are encountered more often in practice and
require more attention. Williams et al. (Williams et al., 2018, 2020) applied
a supervised topical evolution model (Naim et al., 2017) to identify the
features that best described the 10 pre-defined SV types1414141\. Buffer
errors, 2. Cross-site scripting, 3. Path traversal, 4. Permissions and
Privileges, 5. Input validation, 6. SQL injection, 7. Information disclosure,
8. Resources Error, 9. Cryptographic issues, 10. Code injection. prevalent in
the wild. These authors then used a diffusion-based storytelling technique
(Barranco et al., 2019) to show the evolution of a particular topic of SVs
over time; e.g., increasing API-related SVs requires hardening the APIs used
in a product. To support user-friendly SV assessment using ever-increasing
unstructured SV data, Russo et al. (Russo et al., 2019) used Bayesian network
to predict 10 pre-defined SV types.1515151\. Authentication bypass or Improper
Authorization, 2. Cross-site scripting or HTML injection, 3. Denial of
service, 4. Directory Traversal, 5. Local/Remote file include and Arbitrary
file upload, 6. Information disclosure and/or Arbitrary file read, 7.
Buffer/stack/heap/integer overflow, 8. Remote code execution, 9. SQL
injection, 10. Unspecified vulnerability Besides predicting manually defined
SV types using SV natural language descriptions, Yan et al. (Yan et al., 2017)
used a decision tree to predict 22 SV types prevalent in the executables of
Linux applications. The predicted type was then combined with fuzzers’ outputs
to predict SV exploitability (see section 3.1.1). Besides author-defined
types, custom SV types also come from specific SV platforms. Zhang et al.
(Zhang et al., 2020b) designed an ML-based framework to predict the SV types
collected from China National Vulnerability Database. Ensemble models (bagging
and boosting models) achieved, on average, the highest performance for this
task.

###  6.2. Theme Discussion

In the Type theme, detecting and characterizing coarse-grained and fine-
grained CWE-based SV types are the frequent tasks. The large number and
hierarchical structure of classes are the main challenges with CWE
classification/analysis. In terms of solutions, deep Siamese networks (Das et
al., 2021) are more robust to the class imbalance issue (due to many CWE
classes), while graph-based neural networks (Han et al., 2018) can effectively
capture the hierarchical structure of CWEs. Future work can investigate the
combination of these two types of DL architectures to solve both issues
simultaneously. We also recommend more solutions for addressing the data
imbalance issue in section 9.1.3. Besides model-level solutions, author-
selected or platform-specific SV types have been considered to reduce the
complexity of CWE. However, similar to custom SV consequences in section
4.1.2, there is not yet a universally accepted taxonomy for these custom SV
types. To reduce the subjectivity in selecting SV types for prediction, we
suggest that future work should focus on the types that are commonly
encountered and discussed by developers in the wild (see section 9.1.1 for
more details).

##  7\. Miscellaneous Tasks

The last theme is Miscellaneous Tasks covering the studies that are
representative yet do not fit into the four previous themes. This theme has
three main sub-themes/tasks: (i) Vulnerability information retrieval, (ii)
Cross-source vulnerability patterns and (iii) Vulnerability fixing effort (see
Table 10).

Table 10. List of the surveyed papers in the Miscellaneous Tasks theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Sub-theme: 1. Vulnerability Information Retrieval
Weeraward-hana et al. 2014 (Weerawardhana et al., 2014) |  Multi-class classification: Extraction of entities (software name/version, impact, attacker/user actions) from SV descriptions |  NVD (210 randomly selected and manually labeled SVs) |  Stanford Named Entity Recognizer implementing a CRF classifier
Dong et al. 2019 (Dong et al., 2019) |  Multi-class classification: Vulnerable software names/versions |  CVE Details, NVD, ExploitDB, SecurityFocus, SecurityFocus Forum, SecurityTracker, Openwall |  Word-level and character-level Bi-LSTM with attention mechanism
Gonzalez et al. 2019 (Gonzalez et al., 2019) |  Multi-class classification: Extraction of 19 Vulnerability Description Ontology (NIST, [n. d.]c) classes from SV descriptions |  NVD |  Naïve Bayes, Decision tree, SVM, Random forest, Majority voting model
Binyamini et al. 2020 (Binyamini et al., 2021) |  Multi-class classification: Extraction of entities (attack vector/means/technique, privilege, impact, vulnerable platform/version/OS, network protocol/port) from SV descriptions to generate MulVal (Ou et al., 2005) interaction rules |  NVD |  Bi-LSTM with various feature extractors: word2vec, ELMo, BERT (pre-trained or trained from scratch)
Guo et al. 2020 (Guo et al., 2020) |  Multi-class classification: Extraction of entities (SV type, root cause, attack type, attack vector) from SV descriptions |  NVD, SecurityFocus |  CNN, Bi-LSTM (with or without attention mechanism)
Waareus et al. 2020 (Wåreus and Hell, 2020) |  Multi-class classification: Common Product Enumeration (CPE) |  NVD |  Word-level and character-level Bi-LSTM
Yitagesu et al. 2021 (Yitagesu et al., 2021) |  Multi-class classification: Part-of-speech tagging of SV descriptions |  NVD, CVE, CWE, CAPEC, CPE, Twitter, PTB corpus (Marcus et al., 1993) |  Bi-LSTM
Sun et al. 2021 (Sun et al., 2021) |  Multi-class classification: Extraction of entities (vulnerable product/version/component, type, attack type, root cause, attack vector, impact) from ExploitDB to generate SV descriptions |  NVD, ExploitDB |  BERT models
Sub-theme: 2. Cross-source Vulnerability Patterns
Horawalavith-ana et al. 2019 (Horawalavithana et al., 2019) |  Regression: Number of software development activities on GitHub after disclosure of SVs |  Twitter, Reddit, GitHub |  MLP, LSTM
Xiao et al. 2019 (Xiao et al., 2019) |  Knowledge-graph reasoning: modeling the relationships among SVs, its types and attack patterns |  CVE, CWE, CAPEC (Linux project) |  Translation-based knowledge-graph embedding
Sub-theme: 3. Vulnerability Fixing Effort
Othmane et al. 2017 (Othmane et al., 2017) |  Regression: time (days) to fix SVs |  Proprietary SV data collected at the SAP company |  Linear/Tree-based/Neural network regression

###  7.1. Summary of Primary Studies

####  7.1.1. Vulnerability Information Retrieval

The first and major sub-theme is vulnerability information retrieval that
studies data-driven methods to extract different SV-related entities (e.g.,
affected products/versions) and their relationships from SV data. The current
sub-theme extracts assessment information appearing explicitly in SV data
(e.g., SV descriptions on NVD) rather than predicting implicit properties as
done in prior sub-themes. For instance, CWE-119, i.e., “Improper Restriction
of Write Operations within the Bounds of a Memory Buffer”, can be retrieved
directly from CVE-2020-28022, but not from
CVE-2021-2122.161616https://nvd.nist.gov/vuln/detail/CVE-2020-28022 &
https://nvd.nist.gov/vuln/detail/CVE-2021-21220 The latter case requires
techniques from section 6.1.1.

Most of the retrieval methods in this sub-theme have been formulated under the
multi-class classification setting. One of the earliest works was conducted by
Weerawardhana et al. (Weerawardhana et al., 2014). This study extracted
software names/versions, impacts and attacker’s/user’s action from SV
descriptions on NVD using Stanford Named Entity Recognition (NER) technique,
a.k.a. CRF classifier (Finkel et al., 2005). Later, Dong et al. (Dong et al.,
2019) proposed to use a word/character-level Bi-LSTM to improve the
performance of extracting vulnerable software names and versions from SV
descriptions available on NVD and other SV databases/advisories (e.g., CVE
Details (Özkan, [n. d.]), ExploitDB (Security, [n. d.]), SecurityFocus (Inc,
[n. d.]), SecurityTracker (SecurityTracker, [n. d.]) and Openwall (Project,
[n. d.])). Based on the extracted entities, these authors also highlighted the
inconsistencies in vulnerable software names and versions across different SV
sources. Besides version products/names of SVs, Gonzalez et al. (Gonzalez et
al., 2019) used a majority vote of different ML models (e.g., SVM and Random
forest) to extract the 19 entities of Vulnerability Description Ontology (VDO)
(NIST, [n. d.]c) from SV descriptions to check the consistency of these
descriptions based on the guidelines of VDO. Since 2020, there has been a
trend in using DL models (e.g., Bi-LSTM, CNNs or BERT (Devlin et al.,
2018)/ELMo (Peters et al., 2018)) to extract different information from SV
descriptions including required elements for generating MulVal (Ou et al.,
2005) attack rules (Binyamini et al., 2021) or SV types/root cause, attack
type/vector (Guo et al., 2020), Common Product Enumeration (CPE) (MITRE, [n.
d.]b) for standardizing names of vulnerable vendors/products/versions (Wåreus
and Hell, 2020), part-of-speech (Yitagesu et al., 2021) and relevant entities
(e.g., vulnerable products, attack type, root cause) from ExploitDB to
generate SV descriptions (Sun et al., 2021). BERT models (Devlin et al.,
2018), pre-trained on general text (e.g., Wikipedia pages (Foundation, [n.
d.]) or PTB corpus (Marcus et al., 1993)) and fine-tuned on SV text, have also
been increasingly used to address the data scarcity/imbalance for the
retrieval tasks.

####  7.1.2. Cross-source Vulnerability Patterns

The second sub-theme, cross-source vulnerability patterns, finds commonality
and/or discovers latent relationships among SV sources to enrich information
for SV assessment and prioritization. Horawalavithana et al. (Horawalavithana
et al., 2019) found a positive correlation between development activities
(e.g., push/pull requests and issues) on GitHub and SV mentions on
Reddit171717https://reddit.com and Twitter. These authors then used DL models
(MLP (Hastie et al., 2009) and LSTM (Hochreiter and Schmidhuber, 1997)) to
predict the appearance and sequence of development activities when SVs were
mentioned on the two social media platforms. Xiao et al. (Xiao et al., 2019)
applied a translation-based graph embedding method to encode and predict the
relationships among different SVs and the respective attack patterns and
types. This work (Xiao et al., 2019) was based on DeepWeak of Han et al. (Han
et al., 2018), but it still belongs to this sub-theme as they provided a
multi-dimensional view of SVs using three different sources (NVD (NIST, [n.
d.]a), CWE (MITRE, [n. d.]d) and CAPEC (MITRE, [n. d.]a)). Xiao et al. (Xiao
et al., 2019) envisioned that their knowledge graph can be extended to
incorporate the source code introducing/fixing SVs.

####  7.1.3. Vulnerability Fixing Effort

The last sub-theme is vulnerability fixing effort that focuses on estimating
SV fixing effort through proxies such as the SV fixing time, usually in days.
Othmane and the co-authors were among the first to approach this problem.
These authors first conducted a large-scale qualitative study at the SAP
company and identified 65 important code-based, process-based and developer-
based factors contributing to the SV fixing effort (ben Othmane et al., 2015).
Later, the same group of authors (Othmane et al., 2017) leveraged the
identified factors in their prior qualitative study to develop various
regression models such as linear regression, tree-based regression and neural
network regression models, to predict time-to-fix SVs using the data collected
at SAP. These authors found that code components containing detected SVs are
more important for the prediction than SV types.

###  7.2. Theme Discussion

In the Miscellaneous Tasks theme, the key focus is on retrieving SV-related
entities and characteristics from SV descriptions. The retrieval tasks are
usually formulated as Named Entity Recognition from SV descriptions. However,
we observed that NVD descriptions do not follow a consistent template (Anwar
et al., 2020), posing significant challenges in labeling the entities for
retrieval. The affected versions and vendor/product names of SVs also contain
inconsistencies (Dong et al., 2019; Anwar et al., 2020), making the retrieval
tasks difficult. We recommend that data normalization and cleaning should be
performed before labeling entities and building respective retrieval models to
ensure the reliability of results.

Besides information retrieval, other tasks such as linking multi-sources,
extracting cross-source patterns or estimating fixing effort are also useful
to obtain richer SV information for assessment and prioritization, yet these
tasks are still in early stages. Linking multiple sources and their patterns
is the first step towards building an SV knowledge graph to answer different
queries regarding a particular SV (e.g., what systems are affected,
exploitation status, how to fix, or what SVs are similar). In the future, such
a knowledge graph can be extended to capture artifacts of SVs in emerging
software types like AI-based systems (see section 9.3). Moreover, to advance
SV fixing effort prediction, future work can consider adapting/customizing the
existing practices/techniques used to predict fixing effort for general bugs
(Zhang et al., 2013; Akbarinasaji et al., 2018).

##  8\. Analysis of Data-driven Approaches for Software Vulnerability
Assessment and Prioritization

Table 11. The frequent data sources, features, models, evaluation techniques and evaluation metrics used for the five identified SV assessment and prioritization themes. Notes: The values are organized based on their overall frequency across the five themes. For the Prediction Model and Evaluation Metric elements, the values are first organized by categories (ML then DL for Prediction Model and classification then regression for Evaluation Metric) and then by frequencies. k-CV stands for k-fold cross-validation. The full list of values and their appearance frequencies for the five elements in the five themes can be found at (Le et al., [n. d.]). Source/Technique/Metric |  Strengths |  Weaknesses
---|---|---
Element: Data Source
NVD/CVE/CVE Details (deprecated OSVDB) |  • Report expert-verified information (with CVE-ID) • Contain CWE and CVSS entries for each SV • Link to external sources (official fixes or vendors’ info) |  • Missing/incomplete links to vulnerable code/fixes • Inconsistencies due to human errors • Delayed SV reporting and assignment of CVSS metrics
ExploitDB |  • Report PoC exploits of SVs (with links to CVE-ID) |  • May not lead to real exploits in the wild
Other security advisories (e.g., SecurityFocus, Symantec or X-Force) |  • Report real-world exploits of SVs • Cover a wide range of SVs (including ones w/o CVE-ID) |  • Some exploits may not have links to CVE entries for mapping with other assessment metrics
Informal sources (e.g., Twitter and darkweb) |  • Early reporting of SVs (maybe even earlier than NVD) • Contain non-technical SV information (e.g., financial damage or socio-technical challenges in addressing SVs) |  • Contain non-verified and even misleading information • May cause adversarial attacks to assessment models
Element: Model Feature
BoW/tf-idf/n-grams |  • Simple to implement • Strong baseline for text-based inputs (e.g., SV descrip- tions in security databases/advisories) |  • May suffer from vocabulary explosion (e.g., many new descrip- tion words for new SVs) • No consideration of word context/order (maybe needed for code-based SV analysis) • Cannot handle Out-of-Vocabulary (OoV) words (can be resolved with subwords (Le et al., 2019))
Word2vec |  • Capture nearby context of each word • Can reuse existing pre-trained model(s) |  • Cannot handle OoV words (can be resolved with fastText (Bojanowski et al., 2017)) • No consideration of word order
DL model end-to-end trainable features |  • Produce SV task-specific features |  • May not produce high-quality representation for tasks with limited data (e.g., real-world exploit prediction)
Bidirectional Encoder Representations from Transformers (BERT) |  • Capture contextual representation of text (i.e., the feature vector of a word is specific to each input) • Capture word order in an input • Can handle OoV words |  • May require GPU to speed up feature inference • May be too computationally expensive and require too much data to train a strong model from scratch • May require fine-tuning to work well for a source task
Source/expert-defined metadata features |  • Lightweight • Human interpretable for a task of interest |  • Require SV expertise to define relevant features • Hard to generalize to new tasks
Element: Prediction Model
Single ML models (e.g., Linear SVM, Logistic regression, Naïve Bayes) |  • Simple to implement • Efficient to (re-)train on large data (e.g., using the entire NVD database) |  • May be prone to overfitting • Usually do not perform as well as ensemble/DL models
Ensemble ML models (e.g., Ran- dom forest, XGBoost, LGBM) |  • Strong baseline (usually stronger than single models) • Less prone to overfitting |  • Take longer to train than single models
Latent Dirichlet Allocation (LDA – topic modeling) |  • Require no labeled data for training • Can provide features for supervised learning models |  • Require SV expertise to manually label generated topics • May generate human non-interpretable topics
Deep Multi-Layer Perceptron (MLP) |  • Work readily with tabular data (e.g., manually defined features or BoW/tf-idf/n-grams) |  • Perform comparably yet are more costly compared to ensemble ML models • Less effective for unstructured data (e.g., SV descriptions)
Deep Convolutional Neural Networks (CNN) |  • Capture local and hierarchical patterns of inputs • Usually perform better than MLP for text-based data |  • Cannot effectively capture sequential order of inputs (maybe needed for code-based SV analysis)
Deep recurrent neural networks (e.g., LSTM or Bi-LSTM) |  • Capture short-/long-term dependencies from inputs • Usually perform better than MLP for text-based data |  • May suffer from the information bottleneck issue (can be resolved with attention mechanism (Bahdanau et al., 2014)) • Usually take longer to train than CNNs
Deep graph neural networks (e.g., Graph convolutional network) |  • Capture directed relationships among multiple SV entities and sources |  • Require graph-based inputs to work • More computationally expensive than other DL models
Deep transfer learning with fine-tuning (e.g., BERT with task- specific classification layer(s)) |  • Can improve the performance for tasks with small data (e.g., real-world exploit prediction) |  • Require target task to have similar nature as source task
Deep constrastive learning (e.g., Siamese neural networks) |  • Can improve performance for tasks with small data • Robust to class imbalance (e.g., CWE classes) |  • Computationally expensive (two inputs instead of one) • Do not directly produce class-wise probabilities
Deep multi-task learning |  • Can share features for predicting multiple tasks (e.g., CVSS metrics) simultaneously • Reduce training/maintenance cost |  • Require predicted tasks to be related • Hard to tune the prediction/performance of individual tasks
Element: Evaluation Technique
Single k-CV without test |  • Easy to implement • Reduce the randomness of results with multiple folds |  • Do not have a separate test set for validating optimized models (can be resolved with separate test set(s)) • Maybe infeasible for expensive DL models • Use future data/SVs for training, maybe leading to biased results
Single/multiple random train/test with/without val (using val to tune hyperparameters) |  • Easy to implement • Reduce the randomness of results (the multiple version) |  • May produce unstable results (the single version) • Maybe infeasible for expensive DL models (the multiple version) • Use future data/SVs for training, maybe leading to biased results
Single/multiple time-based train/test with/without val (using val to tune hyperparameters) |  • Consider the temporal properties of SVs, simulating the realistic evaluation of ever-increasing SVs in practice • Reduce the randomness of results (the multiple version) |  • Similar drawbacks for the single & multiple versions as the random counterparts • May result in uneven/small splits (e.g., many SVs in a year)
Element: Evaluation Metric
F1-score/Precision/Recall (classification) |  • Suitable for imbalanced data (common in SV assessment and prioritization tasks) |  • Do not consider True Negatives in a confusion matrix (can be resolved with Matthews Correlation Coefficient (MCC))
Accuracy (classification) |  • Consider all the cells in a confusion matrix |  • Unsuitable for imbalanced data (can be resolved with MCC)
Area Under the Curve (AUC) (classification) |  • Independent of prediction thresholds |  • May not represent real-world settings (i.e., as models in practice mostly use fixed classification thresholds) • ROC-AUC may not be suitable for imbalanced data (can be resolved with Precision-Recall-AUC)
Mean absolute (percentage) error/ Root mean squared error (regression) |  • Show absolute performance of models |  • Maybe hard to interpret a value on its own without domain knowledge (i.e., whether an error of $x$ is sufficiently effective)
Correlation coefficient ($r$)/ Coef. of determination ($R^{2}$) (regression) |  • Show relative performance of models (0 – 1), where 0 is worst & 1 is best |  • $R^{2}$ always increases when adding any new feature (can be resolved with adjusted $R^{2}$)

We extract and analyze the five key elements for data-driven SV assessment and
prioritization: (i) Data sources, (ii) Model features, (iii) Prediction
models, (iv) Evaluation techniques and (v) Evaluation metrics. These elements
correspond to the four main steps in building data-driven models: data
collection (data sources), feature engineering (model features), model
training (prediction models) and model evaluation (evaluation
techniques/metrics) (Han et al., 2011; Sabir et al., 2021). We present the
most common practices for each element in Table 11.

###  8.1. Data sources

Identifying and collecting rich and reliable SV-related data are the first
tasks to build data-driven models for automating SV assessment and
prioritization tasks. As shown in Table 11, a wide variety of data sources
have been considered to accomplish the five identified themes.

Across the five themes, NVD (NIST, [n. d.]a) and CVE (MITRE, [n. d.]c) have
been the most prevalently used data sources. The popularity of NVD/CVE is
mainly because they publish expert-verified SV information that can be used to
develop prediction models. Firstly, many studies have considered SV
descriptions on NVD/CVE as model inputs. Secondly, the SV characteristics on
NVD have been heavily used as assessment outputs in all the themes, e.g., CVSS
Exploitability metrics for Exploitation, CVSS Impact/Scope metrics for Impact,
CVSS severity score/levels for Severity, CWE for Type, CWE/CPE for
Miscellaneous tasks. Thirdly, external sources on NVD/CVE have enabled many
studies to obtain richer SV information (e.g., exploitation availability/time
(Chen et al., 2019b) or vulnerable code/crashes (Yan et al., 2017; Tripathi et
al., 2017)) and extract relationships among multiple SV sources to develop a
knowledge graph of SVs (e.g., (Han et al., 2018; Xiao et al., 2019)). However,
NVD/CVE still suffer from information inconsistencies (Dong et al., 2019;
Anwar et al., 2020) and missing relevant external sources (e.g., SV fixing
code) (Hommersom et al., 2021). Such issues motivate future work to
validate/clean NVD data and utilize more sources for code-based SV assessment
and prioritization (see section 9.1.1).

To enrich the SV information on NVD/CVE, many other security advisories and SV
databases have been commonly leveraged by the reviewed studies, notably
ExploitDB (Security, [n. d.]), Symantec (Broadcom, [n. d.]a, [n. d.]b),
SecurityFocus (Inc, [n. d.]), CVE Details (Özkan, [n. d.]) and OSVDB. Most of
these sources disclose PoC (ExploitDB and OSVDB) and/or real-world (Symantec
and Security Focus) exploits. However, real-world exploits are much rarer and
different compared to PoC ones (Sabottke et al., 2015; Jacobs et al., 2020).
It is recommended that future work should explore more data sources (other
than the ones in Table 2) and better methods to retrieve real-world exploits
(see section 9.1). Additionally, CVE Details and OSVDB are SV databases like
NVD yet with a few key differences. CVE Details explicitly monitors Exploit-DB
entries that may be missed on NVD and provides a more user-friendly interface
to view/search SVs. OSVDB also reports SVs that do not appear on NVD (without
CVE-ID), but this site was discontinued in 2016.

Besides official/expert-verified data sources, we have seen an increasing
interest in mining SV information from informal sources that also contain non-
expert generated content such as social media (e.g., Twitter) and darkweb.
Especially, Twitter has been widely used for predicting exploits as this
platform has been shown to contain many SV disclosures even before official
databases like NVD (Sabottke et al., 2015; Chen et al., 2019a). Recently,
darkweb forums/sites/markets have also gained traction as SV mentions on these
sites have a strong correlation with their exploits in the wild (Almukaynizi
et al., 2017, 2019). However, SV-related data on these informal sources are
much noisier because they neither follow any pre-defined structure nor have
any verification and they are even prone to fake news (Sabottke et al., 2015).
Thus, the data integrity of these sources should be checked, potentially by
checking the reputation of posters, to avoid inputting unreliable data to
prediction models and potentially producing misleading findings.

###  8.2. Model features

Collected raw data need to be represented by suitable features for training
prediction models. There are three key types of feature representation methods
in this area: term frequency (e.g., BoW, tf-idf and n-grams), DL learned
features (e.g., BERT and word2vec) and source/expert-defined metadata (e.g.,
CVSS metrics and CPE on NVD or tweet properties on Twitter), as summarized in
Table 11.

Regarding the term-frequency based methods, BoW has been the most popular one.
Its popularity is probably because it is one of the simplest ways to extract
features from natural language descriptions of SVs and directly compatible
with popular ML models (e.g., Linear SVM, Logistic regression and Random
forest) in section 8.3. Besides plain term count/frequency, other studies have
also considered different weighting mechanisms such as inverse document
frequency weighting (tf-idf) or tf-igm (Chen et al., 2016) inverse gravity
moment weighting (tf-igm). Tf-igm has been shown to work better than BoW and
tf-idf at classifying severity (Kudjo et al., 2019; Chen et al., 2020a).
Future work is still needed to evaluate the applicability and generalizability
of tf-igm for other SV assessment and prioritization tasks.

Recently, Neural Network (NN) or DL based features such as word2vec (Mikolov
et al., 2013) and BERT (Devlin et al., 2018) have been increasingly used to
improve the performance of predicting CVSS exploitation/impact/severity
metrics (Han et al., 2017; Gong et al., 2019), CWE types (Das et al., 2021)
and SV information retrieval (Guo et al., 2020; Wåreus and Hell, 2020).
Compared to BoW and its variants, NN and DL can extract more efficient and
context-aware features from vast SV data (Le et al., 2020a). NN/DL techniques
rely on distributed representation to encode SV-related words using fixed-
length vectors much smaller than a vocabulary size. Moreover, these techniques
capture the sequential order and context (nearby words) to enable better SV-
related text comprehension (e.g., SV vs. general exploit). Importantly, these
NN/DL learned features can be first trained in a non-SV domain with abundant
data (e.g., Wikipedia pages (Foundation, [n. d.])) and then transferred/fine-
tuned in the SV domain to address limited/imbalanced SV data (Yin et al.,
2020). The main concern with these sophisticated NN/DL features is their
limited interpretability, which is an exciting research area (see section
9.2.2).

The metadata about SVs can also complement the missing information in
descriptions or code for SV assessment and prioritization. For example,
prediction of exploits and their characteristics have been enhanced using CVSS
metrics (Almukaynizi et al., 2019), CPE (Aksu et al., 2018) and SV types
(Bhatt et al., 2021) on NVD. Additionally, Twitter-related statistics (e.g.,
number of followers, likes and retweets) have been shown to increase the
performance of predicting SV exploitation, impact and severity (Sabottke et
al., 2015; Chen et al., 2019b). Recently, alongside features extracted from
vulnerable code, the information about a software development process and
involved developers have also been extracted to predict SV fixing effort
(Othmane et al., 2017). Currently, metadata-based and text-based features have
been mainly integrated by concatenating their respective feature vectors
(e.g., (Chen et al., 2019a, c; Almukaynizi et al., 2017, 2019)). An
alternative yet unexplored way is to build separate models for each feature
type and then combine these models using meta-learning (e.g., model stacking
(Dzeroski and Zenko, 2002)).

###  8.3. Prediction models

The extracted features enter a wide variety of ML/DL-based prediction models
shown in Table 11 to automate various SV assessment and prioritization tasks.
Classification techniques have the largest proportion, while regression and
unsupervised techniques are less common.

Linear SVM (Cortes and Vapnik, 1995) has been the most frequently used
classifier, especially in the Exploitation, Impact and Severity themes. This
popularity is expected as Linear SVM works well with the commonly used
features, i.e., BoW and tf-idf. Besides Linear SVM, Random forest, Naïve Bayes
and Logistic regression have also been common classification models. In recent
years, advanced boosting models (e.g., XGBoost (Chen and Guestrin, 2016) and
LGBM (Ke et al., 2017)), and more lately, DL techniques (e.g., CNN (Kim, 2014)
and (Bi-)LSTM with attention (Bahdanau et al., 2014)) have been increasingly
utilized and shown better results than simple ML models like Linear SVM or
Logistic regression. In this area, some DL models are essential for certain
tasks, e.g., building SV knowledge graph from multiple sources with graph
neural networks (Kipf and Welling, 2016). DL models also offer solutions to
data-related issues such as addressing class imbalance (e.g., deep Siamese
network (Reimers and Gurevych, 2019)) or improving data efficiency (e.g., deep
multi-task learning (Zhang and Yang, 2021)). Whenever applicable, it is
recommended that future work should still consider simple baselines alongside
sophisticated ones as simple methods can perform on par with advanced ones
(Mazuera-Rozo et al., 2021).

Besides classification, various prediction models have also been investigated
for regression (e.g., predicting exploit time, severity score and fixing
time). Linear SVM has again been the most commonly used regressor as SV
descriptions have usually been the regression input. Notably, many studies in
the Severity theme did not build regression models to directly obtain the
severity score (e.g., (Wen et al., 2015; Ognawala et al., 2018; Elbaz et al.,
2020; Jiang and Atif, 2020; Spanos and Angelis, 2018)). Instead, they used the
formulas defined by assessment frameworks (e.g., CVSS versions 2/3 (FIRST, [n.
d.]b, [n. d.]c) or WIVSS (Spanos et al., 2013)) to compute the severity score
from the base metrics predicted by respective classification models. We argue
that more effort should be invested in determining the severity score directly
from SV data as these severity formulas can be subjective (Spring et al.,
2021). We also observe that there is still limited use of DL models for
regression compared to classification.

In addition to supervised (classification/regression) techniques, unsupervised
learning has also been considered for extracting underlying patterns of SV
data, especially in the Type theme. Latent Dirichlet Allocation (LDA) (Blei et
al., 2003) has been the most commonly used topic model to identify latent
topics/types of SVs without relying on a labeled taxonomy. The identified
topics were mapped to the existing SV taxonomies such as CWE (Neuhaus and
Zimmermann, 2010) and OWASP (Mounika et al., 2019; Vanamala et al., 2020). The
topics generated by topic models like LDA can also be used as features for
classification/regression models (Ruohonen, 2017) or building topic-wise
models to capture local SV patterns (Menzies et al., 2018). However, definite
interpretations for unsupervised outputs are challenging to obtain as they
usually rely on human judgement (Palacio-Niño and Berzal, 2019).

###  8.4. Evaluation techniques

It is important to evaluate a trained model to ensure the model meets certain
requirements (e.g., advancing the state-of-the-art). The evaluation generally
needs to be conducted on a different set of data other than the training set
to avoid overfitting and objectively estimate model generalizability (Hastie
et al., 2009). The commonly used evaluation techniques are summarized in Table
11.

The reviewed studies have mostly used one or multiple validation and/or test
sets181818Validation set(s) helps optimize/tune a model (finding the best
task/data-specific hyperparameters), and test set(s) evaluates the
optimized/tuned model. Using only validation set(s) means evaluating a model
with default/pre-defined hyperparameters. to evaluate their models, in which
each validation/test set has been either randomly or time-based selected.
Specifically, k-fold cross-validation has been one of the most commonly used
techniques. The number of folds has usually been 5 or 10, but less standard
values like 4 (Yan et al., 2017) have also been used. However, k-fold cross-
validation uses all parts of data at least once for training; thus, there is
no hidden test set to evaluate the optimal model with the highest
(cross-)validation performance.

To address the lack of hidden test set(s), a common practice in the studied
papers has been to split a dataset into single training and test sets,
sometimes with an additional validation set for tuning hyperparameters to
obtain an optimal model. Recently, data has been increasingly split based on
the published time of SVs to better reflect the changing nature of ever-
increasing SVs (Bullough et al., 2017; Le et al., 2019). There have been
various ratios for random (e.g., 80:20, 75:25 or 67:33) and time-based (e.g.,
week/month/year-wise) splits. However, the results reported using single
validation/test sets may be unstable (i.e., unreproducible results using
different set(s)) (Raschka, 2018).

To ensure both the time order and reduce the result randomness, we recommend
using multiple splits of training and test sets in combination with time-based
validation in each training set. Statistical analyses (e.g., hypothesis
testing and effect size) should also be conducted to confirm the reliability
of findings with respect to the randomization of models/data in multiple runs
(de Oliveira Neto et al., 2019).

###  8.5. Evaluation metrics

Evaluating different aspects of a model requires respective proper metrics.
The popular metrics for evaluating the tasks in each theme are given in Table
11.

Across the five themes, Accuracy, Precision, Recall and F1-score (Jiao and Du,
2016) have been the most commonly used metrics because of a large number of
classification tasks in the five themes. However, Accuracy is not a suitable
measure for SV assessment and prioritization tasks with imbalanced data (e.g.,
SVs with real-world exploits vs. non-exploited SVs). The sample size of one
class is much smaller than the others, and thus the overall Accuracy would be
dominated by the majority classes. Besides these four commonly used metrics,
AUC based on the ROC curve (ROC-AUC) (Jiao and Du, 2016) has also been
considered as it is threshold-independent. However, we suggest that ROC-AUC
should be used with caution in practice as most deployed models would have a
fixed decision threshold (e.g., 0.5). Instead of ROC-AUC, we suggest Matthews
Correlation Coefficient (Jiao and Du, 2016) (MCC) as a more meaningful
evaluation metric to be considered as it explicitly captures all values in a
confusion matrix, and thus has less bias in results.

For regression tasks, various metrics have been used such as Mean absolute
error, Mean absolute percentage error, Root mean squared error (Spanos and
Angelis, 2018) as well as Correlation coefficient ($r$) and Coefficient of
determination ($R^{2}$) (Othmane et al., 2017). Note that adjusted $R^{2}$
should be preferred over $R^{2}$ as $R^{2}$ would always increase when adding
a new (even irrelevant) feature.

A model can have a higher value of one metric yet lower values of
others.191919https://stackoverflow.com/questions/34698161 Therefore, we
suggest using a combination of suitable metrics for a task of interest to
avoid result bias towards a specific metric. Currently, most studies have
focused on evaluating model effectiveness, i.e., how well the predicted
outputs match the ground-truth values. Besides effectiveness, other aspects
(e.g., efficiency in training/deployment and robustness to input changes) of
models should also be evaluated to provide a complete picture of model
applicability in practice (see section 9.2.3).

##  9\. Open Research Challenges and Future Directions of Data-driven Software
Vulnerability Assessment and Prioritization

We discuss three main open challenges with the reviewed studies of data-driven
SV assessment and prioritization and then present nine potential directions to
address such challenges (see Figure 2).

Figure 2. List of challenges and future directions for data-driven SV
assessment and prioritization.

###  9.1. SV Data Availability

This section focuses on three key issues with the currently used data sources
and potential solutions. First, the current data hardly contain specific
developer’s concerns and practices when addressing real-world SVs (section
9.1.1). Second, the data sources still miss many SV-related bugs reported in
issue tracking systems, limiting the amount of data for training prediction
models (section 9.1.2). Third, some tasks/outputs (e.g., real-world exploit
prediction) suffer from limited and/or imbalanced labeled data, potentially
leading to unreliable performance of fully-supervised models (section 9.1.3).

####  9.1.1. Utilization of developer Q&A platforms and version control
systems

Developer Question & Answer (Q&A) platforms like Stack Overflow and Security
StackExchange202020Stack Overflow: https://stackoverflow.com & Security
StackExchange: https://security.stackexchange.com contain tens of thousands of
posts about challenges and solutions shared by millions of developers when
tackling known SVs in real-world scenarios (Le et al., 2021a). One of the key
insights of Le et al. (Le et al., 2021a)’s study is that the top SV types that
developers usually struggle with are not always the same as those reported on
SV databases (CWE (MITRE, [n. d.]d) or OWASP (OWASP, [n. d.])). Thus, future
work should also consider real-world development-related issues discussed on
developer Q&A platforms for automatically assessing and prioritizing SVs. For
example, the fixing effort of SVs may depend on the technical difficulty of
implementing the respective mitigation strategies in a language or system of
interest.

Version control systems like GitHub212121https://github.com provide details
about how developers addressed past SVs in real-world projects. Shrestha et
al. (Shrestha et al., 2020) found developers sometimes discuss/disclose SV-
related information on GitHub discussions even before the studied social media
such as Twitter or Reddit. These findings show the potential of using GitHub
discussions to complement the current sources for earlier SV assessment and
prioritization. GitHub can also provide vulnerable code for performing
assessment and prioritization for SVs rooted in source code (Le and Babar,
2022), which is important yet has received limited attention from the
community so far. Moreover, Walden (Walden, 2020) demonstrated the impact of a
major SV (i.e., Heartbleed) on the characteristics (e.g., code
complexity/style, contributors and development practices) of a single project
(i.e., OpenSSL). Based on Walden’s findings, future work can study whether the
impact of an SV would be similar or different in multiple affected projects.
Such investigation would give insights into the possibility of leveraging data
from large projects to perform SV assessment and prioritization in smaller
projects with the same/similar SVs.

####  9.1.2. Integration of SV data on issue tracking systems

Issue/bug tracking systems like JIRA, Bugzilla or GitHub issues222222JIRA:
https://www.atlassian.com/software/jira, Bugzilla: https://www.bugzilla.org/,
GitHub issues: https://docs.github.com/en/issues have been reporting numerous
security-related bugs, many of which are SVs, but they have been underexplored
for data-driven SV assessment and prioritization. Besides providing SV
descriptions like CVE/NVD, these bug reports also contain other artifacts such
as steps to reproduce, stack traces and test cases that give extra information
about SVs (Zimmermann et al., 2010). However, it is not trivial to obtain and
integrate these SV-related bug reports with the ones on SV databases.

One way to retrieve SVs on issue tracking systems is to use security bug
reports (Bhuiyan et al., 2021). Much research work has been put into
developing effective models to automatically retrieve security bug reports
(e.g., (Gegick et al., 2010; Peters et al., 2017; Wu et al., 2021)). Among
these studies, Wu et al. (Wu et al., 2021) manually verified and cleaned the
security bug reports to provide a clean dataset for automated security bug
report identification. However, more of such manual effort is still required
to obtain up-to-date data because the original security bug reports in (Wu et
al., 2021) were actually a part of the dataset collected back in 2014 (Ohira
et al., 2015).

It is worth noting that not all security bug reports are related to SVs such
as issues/improvements in implementing security features.232323The security
bug report AMBARI-1373 on JIRA
(https://issues.apache.org/jira/browse/AMBARI-1373) was about improving the
front-end of AMBARI Web by displaying the current logged in user. Thus, future
studies need to filter out these cases before using security bug reports for
SV assessment and prioritization. We also emphasize that some SV-related bug
reports are overlapping with the ones on NVD (e.g., the SV report
AMBARI-14780242424https://issues.apache.org/jira/browse/AMBARI-14780 on JIRA
refers to CVE-2016-0731 on CVE/NVD). Such overlaps would require data cleaning
during the integration of reports on issue tracking systems and SV databases
to avoid data duplication (e.g., similar SV descriptions) when developing SV
assessment and prioritization models.

####  9.1.3. Relaxation of fully-supervised learning

Supervised learning models of many tasks in the five themes (see section 8.3)
require fully labeled data, but the data of some tasks are quite limited. To
address the data-hungriness of these fully-supervised learning models, future
studies can approach the SV assessment and prioritization tasks with low-shot
learning and/or semi-supervised learning.

Low-shot learning a.k.a. few-shot learning is designed to perform supervised
learning using only a few examples per class, significantly reducing the
labeling effort (Wang et al., 2020). So far, only one study utilized low-shot
learning with a deep Siamese network (Das et al., 2021) (i.e., a shared
feature model with similarity learning) to effectively predict SV types (CWE)
and even generalize to unseen classes (i.e., zero-shot learning). There are
still many opportunities for investigating different few-shot learning
techniques for other SV assessment and prioritization tasks. Note that the
shared features in few-shot learning can also be enhanced with pretrained
models (e.g., BERT (Devlin et al., 2018)) on another domain/task/project with
more labeled data than the current task/project in the SV domain.

Semi-supervised learning enables training models with limited labeled data yet
a large amount of unlabeled data (Van Engelen and Hoos, 2020), potentially
leveraging hidden/unlabeled SVs in the wild. Recently, we have seen an
increasing interest in using different techniques of this learning paradigm in
the SV domain such as collecting SV patches using multi-view co-training
(Sawadogo et al., 2020) or retrieving SV discussions on developer Q&A sites
using positive-unlabeled learning (Le et al., 2020b). However, it is still
little known about the effectiveness of semi-supervised learning for SV
assessment and prioritization.

###  9.2. Real-world Application and Evaluation

The experimental performance of some SV assessment and prioritization models
is promising, but the real-world applicability of such models is still
questionable. First, these models may not be useful in practice due to delayed
inputs and coarse-grained outputs (section 9.2.1). Second, many models are
black-box, limiting the understanding of the model predictions (section
9.2.2). Third, some models are evaluated in over-optimistic conditions far
from real-world scenarios (section 9.2.3).

####  9.2.1. More timely and fine-grained prediction

Although SV descriptions have been commonly used as model inputs (see section
8.1), these descriptions are usually published long after SVs
introduced/discovered in codebases (Meneely et al., 2013). One potential
solution to this issue is to perform assessment and prioritization of SVs in
code commits. Code commits contain changes made by developers to fix a bug/SV,
implement a new feature or refactor code, and new SVs may be introduced in
such changes (Bosu et al., 2014). Commit-level prediction would allow just-in-
time SV assessment and prioritization as soon as SVs are introduced, reducing
the waiting time for SV information to be verified and published on security
advisories/databases (Le et al., 2021b). It should be noted that report-level
prediction is still important for assessing and prioritizing third-party
libraries/software, especially the ones without available code (commits),
and/or SVs missed by commit-level prediction.

CVSS (FIRST, [n. d.]a) has been most frequently used for assessing the
exploitability, impact and severity levels/score of SVs (see sections 3, 4 and
5), but there are increasing concerns that CVSS outputs are still generic.
Specifically, Spring et al. (Spring et al., 2021) argued that CVSS tends to
provide one-size-fits-all assessment metrics regardless of the context of SVs;
i.e., the same SVs in different domains/environments are assigned the same
metric values. For instance, banking systems may consider the confidentiality
and integrity of databases more important than the availability of web/app
interfaces. In the future, alongside CVSS, prediction models should also
incorporate the domain/business knowledge to customize the assessment of SVs
to a system of interest (e.g., the impact of SVs on critical component(s)
and/or the readiness of developers/solutions for mitigating such SVs in the
current system). Future case studies with practitioners are also desired to
correlate the quantitative performance of models and their
usability/usefulness in real-world systems (e.g., reducing more critical SVs
yet using fewer resources).

####  9.2.2. Enhancing model interpretability

Model interpretability is important to increase the transparency of the
predictions made by a model, allowing practitioners to adjust the model/data
to meet certain requirements (Zhang et al., 2020a). Unfortunately, very few
reviewed papers (e.g., (Toloudis et al., 2016; Han et al., 2017)) explicitly
discussed important features and/or explained why/when their models
worked/failed for a task.

SV assessment and prioritization can draw inspiration from the related SV
detection area where the interpretability of (DL-based) prediction models has
been actively explored mainly by using (i) specific model
architectures/parameters or (ii) external interpretation models/techniques
(Zhang et al., 2020a). In the first approach, prior studies successfully used
the feature activation maps in a CNN model (Russell et al., 2018) or leveraged
attention-based neural network (Duan et al., 2019) to highlight and visualize
the important code tokens that contribute to SVs. The second approach uses
separate interpretation models on top of trained SV detectors. The
interpretation models are either domain/model-agnostic (Warnecke et al.,
2020), domain-agnostic yet specific to a model type (graph neural network (Li
et al., 2021)) or SV-specific (Zou et al., 2021). The aforementioned
approaches produce local/sample-wise interpretation, which can be aggregated
to obtain global/task-wise interpretation. The global interpretation is
similar to the feature importance of traditional ML models (Chandrashekar and
Sahin, 2014) such as the weights of linear models (e.g., Logistic regression)
or the (im)purity of nodes split by each feature in tree-based models (e.g.,
Random forest). However, it is still unclear about the
applicability/effectiveness of these approaches for interpreting ML/DL-based
SV assessment and prioritization models, requiring further investigations.

####  9.2.3. Realistic evaluation settings

Most of the reviewed studies have evaluated their prediction models without
capturing many factors encountered during the deployment of such models to
production. Specifically, the models used in practice would require to handle
new data and be robust against adversarial data from informal sources such as
social media or darkweb.

There are concerns with both predicting and integrating new SV data. Regarding
the prediction, Out-of-Vocabulary words in new data need to be properly
accommodated to avoid performance degradation of prediction models (Le et al.,
2019). Regarding the new data integration, online/incremental training on new
data can be considered instead of batch training on the whole dataset to
reduce computational cost (Cabral et al., 2019). The time-based splits should
be used rather than random splits for evaluating online training to avoid
leaking unseen (future) patterns to the model training (see section 8.4).

Regarding the model robustness, only three reviewed studies considered
adversarial attacks as part of their evaluation (Sabottke et al., 2015; Xiao
et al., 2018; Almukaynizi et al., 2019). However, a recent survey shows the
prevalence of adversarial attacks targeted models in cybersecurity (Rosenberg
et al., 2021). Thus, there is certainly a need for more evaluation of
adversarial robustness for SV assessment and prioritization models, especially
DL-based ones.

###  9.3. Data-driven SV Assessment and Prioritization of Data-driven Systems

Compared to other systems, reporting/analyzing SVs of data-driven/Artificial
Intelligence (AI)-based systems is still in its infancy (Kumar et al., 2020).
Data-driven systems (e.g., smart recommender systems, chatbots, robots, and
autonomous cars) are an emerging breed of systems whose cores are powered by
AI technologies, e.g., ML and DL models built on data, rather than human-
defined instructions as in traditional systems. We discuss three key
challenges of SV assessment and prioritization of data-driven systems compared
to traditional systems and suggest potential solutions. Firstly, the current
SV assessment frameworks need customizations to better reflect the nature of
SVs in data-driven systems (section 9.3.1). Secondly, there is a lack of SVs
collected from real-world data-driven systems, limiting the potential of data-
driven SV assessment and prioritization (section 9.3.2). Thirdly, the current
models require redesign, especially in the SV representation, to capture
unique characteristics and artifacts of data-driven systems (section 9.3.3).

####  9.3.1. Designing a compatible SV assessment framework

CVSS (FIRST, [n. d.]a) is currently the most popular SV assessment framework
for traditional systems, but its compatibility with data-driven systems still
requires more investigation. The current CVSS documentation lacks instructions
on how to assign metrics/score for SVs in data-driven systems. For example, it
is unclear how to assign static CVSS metrics to systems with automatically
updated data-driven models (Chen et al., 2020b) because adversarial examples
for exploitation would likely change after the models are updated. Such
ambiguities should be clarified/resolved in future CVSS versions as data-
driven systems become more prevalent.

The types of SVs in ML/DL models in data-driven systems are also mostly
different from the ones provided by CWE (MITRE, [n. d.]d). The difference is
mainly because these new SVs do not only emerge from configurations/code as in
traditional systems, but also from training data and/or trained models
(Rosenberg et al., 2021). Thus, we recommend that a new category of these SVs
should be studied and potentially incorporated into CWE, similar to the newly
added category for architectural
SVs.252525https://cwe.mitre.org/data/definitions/1008.html

####  9.3.2. Collection of SVs in data-driven systems

To the best of our knowledge, there has been no existing large-scale dataset
of SVs in ML/DL models deployed in real-world data-driven systems. Very few of
such SVs have been reported in the wild, one of which is
CVE-2019-20634.262626https://nvd.nist.gov/vuln/detail/CVE-2019-20634 More of
these SVs are required to help develop sufficiently effective SV assessment
and prioritization models. One potential way to build such a dataset is to
first match the (pre-trained) ML/DL models proposed in the literature or
released on model repositories (e.g., Tensorflow
Hub272727https://www.tensorflow.org/hub) with the ones used in real-world
systems either on version control systems or in mobile apps (Huang et al.,
2021). The matched models would then be tested against known adversarial
attacks to identify corresponding SVs. Notably, significant effort is still
required to define/label assessment outputs of these SVs (see section 9.3.1).

####  9.3.3. Representation of SVs in data-driven systems

Existing SV assessment and prioritization models for traditional systems have
not considered unique data/model-related characteristics/features of data-
driven systems (Wan et al., 2019). Specifically, data-driven systems also
encompass information about data (e.g., format, type, size and distribution)
and ML/DL model(s) (e.g., configurations, parameters and performance). It is
worth noting that SVs of ML/DL models in data-driven systems can also come
from the frameworks used to develop such models (e.g., Tensorflow or
Keras)282828Tensorflow: https://github.com/tensorflow/tensorflow & Keras:
https://github.com/keras-team/keras. However, developers of data-driven
systems may not be aware of the (security) issues in the used ML/DL frameworks
(Liu et al., 2020). Thus, besides currently used features, future work should
also consider the information about underlying data/models and used ML/DL
development frameworks to improve the SV representation for building models to
assess and prioritize SVs in data-driven systems.

##  10\. Conclusions

Assessment and prioritization are crucial phases to optimize resource
utilization in addressing SVs at scale. The two phases have witnessed radical
transformations following the increasing availability of SV data from multiple
sources and advances in data-driven techniques. We presented a taxonomy to
summarize the five main directions of the research work so far in this area.
We identified and analyzed the key practices to develop data-driven models in
the reviewed studies. We also highlighted the open challenges and suggested
respective solutions to advance the field.

We envision the field will largely continue to improve the effectiveness of
the presented tasks by leveraging more enriched data sources and sophisticated
data-driven models, especially DL-based ones. Besides the improved
performance, we also see many open opportunities/concerns in under-explored
aspects of developing such advanced models. Overall, a deeper understanding of
practitioners’ concerns and real-world usage scenarios is the key to bridging
the current gap between model development in academia and model deployment in
production.

## Acknowledgment

The work has been supported by the Cyber Security Research Centre Limited
whose activities are partially funded by the Australian Government’s
Cooperative Research Centres Programme.

## References

  * (1)
  * Aghaei et al. (2020) Ehsan Aghaei, Waseem Shadid, and Ehab Al-Shaer. 2020\.  ThreatZoom: CVE2CWE using hierarchical neural network.  _arXiv preprint arXiv:2009.11501_ (2020).
  * Akbarinasaji et al. (2018) Shirin Akbarinasaji, Bora Caglayan, and Ayse Bener. 2018\.  Predicting bug-fixing time: A replication study using an open source software project.  _journal of Systems and Software_ 136 (2018), 173–186.
  * Aksu et al. (2018) M Ugur Aksu, Kemal Bicakci, M Hadi Dilek, A Murat Ozbayoglu, and E ıslam Tatli. 2018\.  Automated generation of attack graphs using nvd. In _Proceedings of the 8th Conference on Data and Application Security and Privacy_. 135–142.