10 Towards an Improved Understanding of Software Vulnerability Assessment
Using Data-Driven Approaches Professor M. Ali Babar Doctor of Philosophy
Computer Science The University of Adelaide School of Computer Science
Computer Science Education Research Faculty Name

Author:

Centre for Research on Engineering Software Technologies (CREST)

School of Computer Science

Faculty of Engineering, Computer and Mathematical Sciences

The University of Adelaide

Principal Supervisor: Professor Muhammad Ali Babar

Co-Supervisor: Professor Cheng-Chew Lim

A thesis submitted for the degree of
DOCTOR OF PHILOSOPHY
The University of Adelaide

March 21, 2022

###### Abstract

Abstract

Software Vulnerabilities (SVs) can expose software systems to cyber-attacks,
potentially causing enormous financial and reputational damage for
organizations. There have been significant research efforts to detect these
SVs so that developers can promptly fix them. However, fixing SVs is complex
and time-consuming in practice, and thus developers usually do not have
sufficient time and resources to fix all SVs at once. As a result, developers
often need SV information, such as exploitability, impact, and overall
severity, to prioritize fixing more critical SVs. Such information required
for fixing planning and prioritization is typically provided in the SV
assessment step of the SV lifecycle. Recently, data-driven methods have been
increasingly proposed to automate SV assessment tasks. However, there are
still numerous shortcomings with the existing studies on data-driven SV
assessment that would hinder their application in practice.

This PhD thesis aims to contribute to the growing literature in data-driven SV
assessment by investigating and addressing the constant changes in SV data as
well as the lacking considerations of source code and developers’ needs for SV
assessment that impede the practical applicability of the field. Particularly,
we have made the following five contributions in this thesis. (1) We
systematize the knowledge of data-driven SV assessment to reveal the best
practices of the field and the main challenges affecting its application in
practice. Subsequently, we propose various solutions to tackle these
challenges to better support the real-world applications of data-driven SV
assessment. (2) We first demonstrate the existence of the concept drift
(changing data) issue in descriptions of SV reports that current studies have
mostly used for predicting the Common Vulnerability Scoring System (CVSS)
metrics. We augment report-level SV assessment models with subwords of terms
extracted from SV descriptions to help the models more effectively capture the
semantics of ever-increasing SVs. (3) We also identify that SV reports are
usually released after SV fixing. Thus, we propose using vulnerable code to
enable earlier SV assessment without waiting for SV reports. We are the first
to use Machine Learning techniques to predict CVSS metrics on the function
level leveraging vulnerable statements directly causing SVs and their context
in code functions. The performance of our function-level SV assessment models
is promising, opening up research opportunities in this new direction. (4) To
facilitate continuous integration of software code nowadays, we present a
novel deep multi-task learning model, DeepCVA, to simultaneously and
efficiently predict multiple CVSS assessment metrics on the commit level,
specifically using vulnerability-contributing commits. DeepCVA is the first
work that enables practitioners to perform SV assessment as soon as vulnerable
changes are added to a codebase, supporting just-in-time prioritization of SV
fixing. (5) Besides code artifacts produced from a software project of
interest, SV assessment tasks can also benefit from SV crowdsourcing
information on developer Question and Answer (Q&A) websites. We automatically
retrieve large-scale security/SV-related posts from these Q&A websites. We
then apply a topic modeling technique on these posts to distill developers’
real-world SV concerns that can be used for data-driven SV assessment.
Overall, we believe that this thesis has provided evidence-based knowledge and
useful guidelines for researchers and practitioners to automate SV assessment
using data-driven approaches.

###### Contents

  1. 0 Introduction
    1. 1 Problem Statement and Research Objectives
    2. 2 Thesis Overview and Contributions
    3. 3 Related Publications
    4. 4 Thesis Organization
  2. 1 Literature Review on Data-Driven Software Vulnerability Assessment
    1. 1 Introduction
      1. 2 Overview of the Literature Review
        1. 1 Scope
        2. 2 Methodology
        3. 3 Taxonomy of Data-Driven Software Vulnerability Assessment
          1. 3 Exploitation Prediction
            1. 1 Summary of Primary Studies
              1. Exploit Likelihood
              2. Exploit Time
              3. Exploit Characteristics
            2. 2 Theme Discussion
              1. 4 Impact Prediction
                1. 1 Summary of Primary Studies
                  1. Confidentiality, Integrity, Availability, and Scope
                  2. Custom Vulnerability Consequences
                2. 2 Theme Discussion
                  1. 5 Severity Prediction
                    1. 1 Summary of Primary Studies
                      1. Severe vs. Non-Severe
                      2. Severity Levels
                      3. Severity Score
                    2. 2 Theme Discussion
                      1. 6 Type Prediction
                        1. 1 Summary of Primary Studies
                          1. Common Weakness Enumeration (CWE)
                          2. Custom Vulnerability Types
                        2. 2 Theme Discussion
                          1. 7 Miscellaneous Tasks
                            1. 1 Summary of Primary Studies
                              1. Vulnerability Information Retrieval
                              2. Cross-Source Vulnerability Patterns
                              3. Vulnerability Fixing Effort
                            2. 2 Theme Discussion
                              1. 8 Analysis of Data-Driven Approaches for Software Vulnerability Assessment
                                1. 1 Data Sources
                                2. 2 Model Features
                                3. 3 Prediction Models
                                4. 4 Evaluation Techniques
                                5. 5 Evaluation Metrics
                                  1. 9 Chapter Summary
                                    1. 10 Appendix - Ever-Growing Literature on Data-Driven SV Assessment
                                      1. 2 Automated Report-Level Software Vulnerability Assessment with Concept Drift
                                        1. 1 Introduction
                                        2. 2 Background
                                        3. 3 The Proposed Approach
                                          1. 1 Approach Overview
                                          2. 2 Text Preprocessing of SV Descriptions
                                          3. 3 Model Selection with Time-based k-Fold Cross-Validation
                                          4. 4 Feature Aggregation Algorithm
                                        4. 4 Experimental Design and Setup
                                          1. 1 Research Questions
                                          2. 2 Dataset
                                          3. 3 Machine Learning Models for Report-Level SV Assessment
                                          4. 4 Evaluation Metrics
                                        5. 5 Experimental Results and Discussion
                                          1. 1 RQ1: Is Our Time-Based Cross-Validation More Effective Than a Non-Temporal Method to Handle Concept Drift in The Model Selection Step for Report-Level SV Assessment?
                                          2. 2 RQ2: Which are the Optimal Models for Multi-Classification of Each SV Characteristic?
                                          3. 3 RQ3: How Effective is Our Character-Word Model to Perform Automated Report-Level SV Assessment with Concept Drift?
                                          4. 4 RQ4: To What Extent Can Low-Dimensional Model Retain the Original Performance?
                                        6. 6 Threats to Validity
                                        7. 7 Related Work
                                          1. 1 Report-Level SV Assessment
                                          2. 2 Temporal Modeling of SVs
                                        8. 8 Chapter Summary
                                        9. 9 Appendix - SVs with All Out-of-Vocabulary Words
                                          1. 3 Automated Function-Level Software Vulnerability Assessment
                                            1. 1 Introduction
                                            2. 2 Background and Motivation
                                            3. 3 Research Questions
                                            4. 4 Research Methodology
                                              1. 1 Data Collection
                                              2. 2 Vulnerable Code Context Extraction
                                              3. 3 Code Feature Generation
                                              4. 4 Data-Driven SV Assessment Models
                                              5. 5 Model Evaluation
                                            5. 5 Results
                                              1. 1 RQ1: Are Vulnerable Code Statements More Useful Than Non-Vulnerable Counterparts for SV Assessment Models?
                                              2. 2 RQ2: To What Extent do Different Types of Context of Vulnerable Statements Contribute to SV Assessment Performance?
                                              3. 3 RQ3: Does Separating Vulnerable Statements and Context to Provide Explicit Location of SVs Improve Assessment Performance?
                                            6. 6 Discussion
                                              1. 1 Function-Level SV Assessment: Baseline Models and Beyond
                                              2. 2 Threats to Validity
                                            7. 7 Related Work
                                              1. 1 Code Granularities of SV Detection
                                              2. 2 Data-Driven SV Assessment
                                            8. 8 Chapter Summary
                                              1. 4 Automated Commit-Level Software Vulnerability Assessment
                                                1. 1 Introduction
                                                2. 2 Background and Motivation
                                                  1. 1 Vulnerability in Code Commits
                                                  2. 2 Commit-Level SV Assessment with CVSS
                                                  3. 3 Feature Extraction from Commit Code Changes
                                                3. 3 The DeepCVA Model
                                                  1. 1 Commit Preprocessing, Context Extraction & Tokenization
                                                  2. 2 Feature Extraction with Deep AC-GRU
                                                  3. 3 Commit-Level SV Assessment with Multi-task Learning
                                                4. 4 Experimental Design and Setup
                                                  1. 1 Datasets
                                                  2. 2 Evaluation Metrics
                                                  3. 3 Hyperparameter and Training Settings of DeepCVA
                                                  4. 4 Baseline Models
                                                5. 5 Research Questions and Experimental Results
                                                  1. 1 RQ1: How does DeepCVA Perform Compared to Baseline Models for Commit-level SV Assessment?
                                                  2. 2 RQ2: What are the Contributions of the Main Components in DeepCVA to Model Performance?
                                                  3. 3 RQ3: What are the Effects of Class Rebalancing Techniques on Model Performance?
                                                6. 6 Discussion
                                                  1. 1 DeepCVA and Beyond
                                                  2. 2 Threats to Validity
                                                7. 7 Related Work
                                                  1. 1 Data-Driven SV Prediction and Assessment
                                                  2. 2 SV Analytics in Code Changes
                                                8. 8 Chapter Summary
                                                  1. 5 Collection and Analysis of Developers’ Software Vulnerability Concerns on Question and Answer Websites
                                                    1. 1 Introduction
                                                    2. 2 Related Work
                                                      1. 1 Topic Modeling on Q&A Websites
                                                      2. 2 SV Assessment Using Open Sources
                                                    3. 3 Research Method
                                                      1. 1 Research Questions
                                                      2. 2 Software Vulnerability Post Collection
                                                      3. 3 Topic Modeling with LDA
                                                    4. 4 Results
                                                      1. 1 RQ1: What are SV Discussion Topics on Q&A Sites?
                                                      2. 2 RQ2: What are the Popular and Difficult SV Topics on Q&A Sites?
                                                      3. 3 RQ3: What is the Level of Expertise to Answer SV Questions on Q&A Sites?
                                                      4. 4 RQ4: What Types of Answers are Given to SV Questions on Q&A Sites?
                                                    5. 5 Discussion
                                                      1. 1 SV Discussion Topics on Q&A Sites vs. Existing Security Taxonomies
                                                      2. 2 Implications for (Data-Driven) SV Assessment
                                                      3. 3 Threats to Validity
                                                    6. 6 Chapter Summary
                                                    7. 7 Appendix - PUMiner Overview
                                                      1. 1 PUMiner - A Context-aware Two-stage PU Learning Model for Retrieving Security Q&A Posts
                                                        1. 6 Conclusions and Future Work
                                                          1. 1 Summary of Contributions and Findings
                                                            1. 1 A Systematization of Knowledge of Data-Driven SV Assessment
                                                            2. 2 Automated Report-Level Assessment for Ever-Increasing SVs
                                                            3. 3 Automated Early SV Assessment using Code Functions
                                                            4. 4 Automated Just-in-Time SV Assessment using Code Commits
                                                            5. 5 Insights of Developers’ Real-World Concerns on Question and Answer Websites for Data-Driven SV Assessment
                                                          2. 2 Opportunities for Future Research
                                                            1. 1 Integration of SV Data on Issue Tracking Systems
                                                            2. 2 Improving Data Efficiency for Data-Driven SV Assessment
                                                            3. 3 Customized Data-Driven SV Assessment
                                                            4. 4 Enhancing Interpretability of SV Assessment Models
                                                            5. 5 Data-Driven SV Assessment in Data-Driven Systems

###### List of Figures

  1. 1 Phases in a software vulnerability lifecycle.
  2. 2 Overview of the thesis.
  3. 1 Taxonomy of studies on data-driven software vulnerability assessment.
  4. 1 Frequencies of each class of the seven vulnerability characteristics.
  5. 2 Workflow of our proposed model for report-level software vulnerability assessment with _concept drift_.
  6. 3 Our proposed time-based cross-validation method.
  7. 4 The number of new terms from 2000 to 2017 of SV descriptions in National Vulnerability Database.
  8. 5 Examples of new terms in National Vulnerability Database corresponding to new products, software, cyber-attacks from 2000 to 2018.
  9. 6 Performance differences between the validated and testing _Weighted F1-Scores_ of our time-based validation and a normal cross-validation methods.
  10. 7 Average cross-validated _Weighted F1-Score_ s comparison between ensemble and single models for each vulnerability characteristic.
  11. 8 The relationship between the size of vocabulary and the maximum number of character n-grams.
  12. 1 A vulnerable function extracted from the fixing commit b38a1b3 of a software vulnerability (CVE-2017-1000487) in the Plexus-utils project.
  13. 2 Methodology used to answer the research questions.
  14. 3 Class distributions of the seven CVSS metrics.
  15. 4 Proportions of different types of lines in a function.
  16. 5 Differences in testing software vulnerability assessment performance (F1-Score and MCC) between models using different types of lines/context and those using only vulnerable statements.
  17. 6 Average performance (MCC) of six classifiers and five features for software vulnerability assessment in functions.
  18. 1 Exemplary software vulnerability fixing commit for the XML external entity injection (XXE) (CVE-2016-3674) and its respective software vulnerability contributing commit in the xstream project.
  19. 2 Workflow of DeepCVA for automated commit-level software vulnerability assessment.
  20. 3 Code changes outside of a method from the commit 4b9fb37 in the Apache qpid-broker-j project.
  21. 4 Class distributions of seven software vulnerability assessment tasks.
  22. 5 Time-based splits for training, validating & testing.
  23. 6 Differences of testing MCC of the model variants compared to the proposed DeepCVA.
  24. 1 Workflow of retrieving posts related to software vulnerability on Q&A websites using tag-based and content-based filtering heuristics.
  25. 2 Popularity and difficulty of 13 software vulnerability topics on Stack Overflow and Security StackExchange.
  26. 3 Topic correlations between software vulnerability questions & answerers’ software vulnerability specific knowledge on Stack Overflow & Security StackExchange.

###### List of Tables

  1. 2 Inclusion and exclusion criteria for study selection.
  2. 3 List of the reviewed papers in the Exploit Likelihood sub-theme of the Exploitation theme.
  3. 4 List of the reviewed papers in the Exploit Time sub-theme of the Exploitation theme.
  4. 5 List of the reviewed papers in the Exploit Characteristics sub-theme of the Exploitation theme.
  5. 6 List of the reviewed papers in the Impact theme.
  6. 7 List of the reviewed papers in the Severe vs. Non-Severe sub-theme of the Severity theme.
  7. 8 List of the reviewed papers in the Severity Levels sub-theme of the Severity theme.
  8. 9 List of the reviewed papers in the Severity Score sub-theme of the Severity theme.
  9. 10 List of the reviewed papers in the Type theme.
  10. 11 List of the reviewed papers in the Miscellaneous Tasks theme.
  11. 12 The frequent data sources, features, models, evaluation techniques and evaluation metrics used for the five identified SV assessment themes.
  12. 13 The mapping between the themes/tasks and the respective studies collected from May 2021 to February 2022.
  13. 1 Word and character n-grams extracted from the sentence “Hello World”. ‘_’ represents a space.
  14. 2 The eight configurations of Natural Language Processing representations used for model selection.
  15. 3 Optimal hyperparameters found for each classifier.
  16. 4 Optimal models and results after the validation step.
  17. 5 Average cross-validated _Weighted F-scores_ of term frequency vs. tf-idf grouped by six classifiers.
  18. 6 Average cross-validated _Weighted F-scores_ of uni-gram vs. n-grams (2 $\leq$ n $\leq$ 4) grouped by six classifiers.
  19. 7 $P$-values of H0: Ensemble models $\leq$ Single models for each vulnerability characteristic.
  20. 8 Performance (_Accuracy_ , _Macro F1-Score_ , _Weighted F1-Score_) of our character-word vs. word-only and character-only models.
  21. 9 _Weighted F1-Scores_ of our original Character-Word Model, 300-dimension Latent Semantic Analysis (LSA-300), fastText trained on SV descriptions (fastText-300) and fastText trained on English Wikipedia pages (fastText-300W).
  22. 1 Hyperparameter tuning for software vulnerability assessment models.
  23. 2 Testing performance for software vulnerability assessment tasks of vulnerable vs. non-vulnerable statements.
  24. 3 Differences in testing performance for software vulnerability assessment tasks between double-input models (RQ3) and single-input models (RQ1/2).
  25. 1 The number of commits and projects after each filtering step.
  26. 2 Testing performance of DeepCVA and baseline models.
  27. 3 Testing performance (MCC) of optimal baselines using oversampling techniques and multi-task DeepCVA.
  28. 1 Content-based thresholds ($a_{\text{SO/SSE}}$ & $b_{\text{SO/SSE}}$) for the two steps of the content-based filtering.
  29. 2 The obtained software vulnerability posts using our tag-based and content-based filtering heuristics.
  30. 3 Top-5 tags of software vulnerability, security and general posts on Stack Overflow and Security StackExchange.
  31. 4 Software vulnerability topics on Stack Overflow and Security StackExchange identified by Latent Dirichlet Allocation along with their proportions and trends over time.
  32. 5 General expertise in terms of average reputation of each topic on Stack Overflow and Security StackExchange.
  33. 6 Answer types of software vulnerability discussions identified on Q&A websites.
  34. 7 Top-1 answer types of 13 software vulnerability topics on Stack Overflow & Security StackExchange.
  35. 8 The mapping between 13 software vulnerability topics and their Common Weakness Enumeration (CWE) values.

I certify that this work contains no material which has been accepted for the
award of any other degree or diploma in my name, in any university or other
tertiary institution and, to the best of my knowledge and belief, contains no
material previously published or written by another person, except where due
reference has been made in the text. In addition, I certify that no part of
this work will, in the future, be used in a submission in my name, for any
other degree or diploma in any university or other tertiary institution
without the prior approval of the University of Adelaide and where applicable,
any partner institution responsible for the joint-award of this degree.

I acknowledge that copyright of published works contained within this thesis
resides with the copyright holder(s) of those works.

I also give permission for the digital version of my thesis to be made
available on the web, via the University’s digital research repository, the
Library Search and also through web search engines, unless permission has been
granted by the University to restrict access for a period of time.

I acknowledge the support I have received for my research through the
provision of University of Adelaide International Wildcard Scholarship.

Triet Huynh Minh Le

March 2022

###### Acknowledgements.

This thesis would not have been possible without continuous support, guidance,
and encouragement from many people and entities. I would like to hereby
acknowledge them. Firstly, I express my deepest gratitude to my principal
supervisor, Professor M. Ali Babar, for giving me a valuable opportunity to
conduct PhD research under his supervision. His constructive feedback has
motivated me to continuously reflect and improve myself to become a better
researcher and a more-rounded person in life. With his kind patience and
persistent guidance, I have also managed to navigate myself through the
challenging COVID-19 pandemic and complete my PhD research to the best of my
ability. Besides research, he has also given me great opportunities to engage
in numerous teaching and supervision activities that have tremendously helped
me to enhance my communication and interpersonal skills. All in all, working
under his mentorship has profoundly transformed me and enabled me to go beyond
my limits and better prepare myself for my future career. Secondly, I
sincerely thank my co-supervisor, Professor Cheng-Chew Lim for providing
insightful comments on the research carried out in this thesis. Thirdly, I am
extremely grateful to many current/former members in the Centre for Research
on Engineering Software Technologies (CREST) at the University of Adelaide.
Special thanks to Faheem Ullah, Chadni Islam, Bushra Sabir, Aufeef Chauhan,
Huaming Chen, Bakheet Aljedaani, Mansooreh Zahedi, Hao Chen, Roland Croft,
David Hin, and Mubin Ul Haque for not only academic contributions and feedback
on the research papers related to this thesis, but also for being wonderful
colleagues from whom I have learned a lot. Specifically, I cannot give enough
appreciation to Roland Croft and David Hin for their great technical insights
and contributions to improve the quality of many research endeavors I have
pursued during my PhD. In addition, I am happy to be accompanied by Faheem
Ullah and Aufeef Chauhan during weekly Friday dinners, which has helped me to
relax and recharge each week. I also appreciate Nguyen Khoi Tran for
introducing me to the CREST family. Thank you all for making my PhD journey
memorable. Fourthly, I have also had the chance to collaborate with and learn
from many world-class researchers outside of CREST such as Xuanyu Duan,
Mengmeng Ge, Shang Gao, and Xuequan Lu. I am also thankful for all the
constructive feedback from the paper and thesis reviewers that helped
significantly improve the research conducted in the thesis. Fifthly, I fully
acknowledge the University of Adelaide for providing me with the University of
Adelaide International Wildcard Scholarship and world-class facilities that
have supported me to pursue my doctoral research and activities. Sixthly, I
highly appreciate the Urbanest at the University of Adelaide for providing me
with the best-conditioned accommodation that I can ever ask for so that I can
enjoy my personal life and recharge after working hours during my PhD. I am
also extremely fortunate that Urbanest has also given me sufficient facilities
to work effectively from home during the pandemic. I also want to deeply thank
my roommates, especially Zach Li, for cheering me up during my down days.
Seventhly, I am greatly appreciative of my ASUS laptop for always being my
reliable companion and working restlessly to enable me to obtain experimental
results as well as write research papers and this thesis in a timely manner.
Finally and most importantly, I am immensely and eternally indebted to my
family, especially my grandmother, mother, and father, who always stand by my
side during the ups and downs of my PhD. Without their constant support and
unconditional caring, I would not have been able to pursue my dreams and be
where I am now. I love all of you from the bottom of my heart.

I would like to dedicate this thesis to my parents.

Dedication

##  Chapter 0 Introduction

Software has become an integral part of the modern world [1]. Software systems
are rapidly increasing in size and complexity. For example, the whole software
ecosystems at Google, which host many popular applications/services such as
Google Search, YouTube, and Google Maps, contain more than two million lines
of code [2]. Quality assurance of such large systems is a focal point for both
researchers and practitioners to minimize disruptions to millions of people
around the world.

Software Vulnerabilities (SVs)111In this thesis, “software vulnerability” and
“security vulnerability” are used interchangeably. have been long-standing
issues that negatively affect software quality [3]. SVs are security bugs that
are detrimental to the confidentiality, integrity and availability of software
systems, potentially resulting in catastrophic cybersecurity attacks [4]. The
exploitation of these SVs such as the Heartbleed [5] or Log4j [6] attacks can
damage the operations and reputation of millions of software systems and
organizations globally. These cyber-attacks caused by SVs have led to huge
financial losses as well. According to the Australian Cyber Security Centre, a
loss of more than 30 billion dollars due to cyber-attacks have been reported
worldwide from 2020 to 2021 [7]. Therefore, it is important to remediate
critical SVs as promptly as possible.

In practice, different types of SVs have varying levels of security threats to
software-intensive systems [8]. However, fixing all SVs at the same time is
not always practical due to limited resources and time [9]. A common practice
in this situation is to prioritize fixing SVs posing imminent and serious
threats to a system of interest. Such fixing prioritization usually requires
inputs from SV assessment [10, 11].

The SV assessment phase is between the SV discovery/detection and SV
remediation/mitigation/fixing/patching phases in the SV management lifecycle
[12], as shown in Fig. 1. The assessment phase first unveils the
characteristics of the SVs found in the discovery phase to locate “hot spots”
that contain many highly critical/severe SVs and require higher attention in a
system. Practitioners then use the assessment outputs to devise an optimal
remediation plan, i.e., the order/priority of fixing each SV, based on
available human and technological resources. For example, an identified cross-
site scripting (XSS) or SQL injection vulnerability in a web application will
likely require an urgent remediation plan. These two types of SVs are well-
known and can be easily exploited by attackers to gain unauthorized access and
compromise sensitive data/information. On the other hand, an SV that requires
admin access or happens only in a local network will probably have a lower
priority since only a few people can initiate an attack. According to the plan
devised in the assessment phase, SVs would be prioritized for fixing in the
remediation phase. In practice, the tasks in the SV assessment phase for ever-
increasing SVs are repetitive and time-consuming, and thus require automation
to save time and effort for practitioners.

Given the increasing size and complexity of software systems nowadays,
automation of SV assessment tasks has attracted significant attention in the
Software Engineering community. Traditionally, static analysis tools have been
the de-facto approach to SV assessment [13]. These tools rely on pre-defined
rules to determine SV characteristics. However, these pre-defined rules
require significant expertise and effort to define and may be easily error-
prone [14]. In addition, these rules need manual modifications and extensions
to adapt to ever-changing patterns of new SVs [15]. Such manual changes do not
scale well with the fast-paced growth of SVs, potentially leading to delays in
SV assessment and in turn untimely SV mitigation. Hence, there is an apparent
need for automated techniques that can perform SV assessment without using
manually-defined rules.

In the last decade, data-driven techniques have emerged as a promising
alternative to static analysis counterparts for SV assessment, as indicated in
our extensive review [11] (to be discussed in-depth in Chapter 1). The
emergence of data-driven SV assessment is mainly because of the rapid increase
in size of SV data in the wild (e.g., more than 170,000 SVs were reported on
National Vulnerability Database (NVD) [16] from 2002 to 2021 [17]). These
approaches are underpinned by Machine Learning (ML), Deep Learning (DL) and
Natural Language Processing (NLP) models that are capable of automatically
extracting complex patterns and rules from large-scale SV data, reducing the
reliance on experts’ knowledge. Overall, these data-driven models have opened
up new opportunities for the field of automated SV assessment.

Figure 1: Phases in an SV lifecycle. Note: The main focus of this thesis is SV
assessment.

###  1 Problem Statement and Research Objectives

Data-driven models have many promising applications for SV assessment;
however, according to our review on data-driven SV assessment [11], there are
still several unaddressed yet important challenges/gaps that affect the
practical application of this field. Specifically, this PhD thesis focuses on
the three key practical challenges of data-driven SV assessment. The first
challenge is the missing treatment for changing SV data over time that leads
to degraded robustness and performance of assessment models. The second
challenge is the lack of using rich and relevant knowledge of (vulnerable)
source code that can result in untimely SV assessment. The third challenge is
the negligence of incorporating developers’ needs into data-driven models that
limits customized SV assessment. These three challenges are captured in the
problem statement of this thesis, which is stated as follows.

Problem statement: Practical applicability of SV assessment using data-driven
approaches is negatively affected by the changing data of SVs along with the
missing considerations/utilization of source code and developers’ real-world
needs. It is important to investigate, understand, and address these
challenges to make data-driven SV assessment more timely and applicable in
practice.

The objectives of this thesis are to present and evaluate solutions to address
the lack of (1) treatment for changing SV data, (2) usage of SV-related code,
and (3) consideration of developers’ real-world SV concerns to improve the
practicality of data-driven SV assessment.

Figure 2: Overview of the thesis. Note: SV stands for Software Vulnerability.

###  2 Thesis Overview and Contributions

A brief overview of the thesis is given in Fig. 2. The research objectives
have been realized in six chapters, which are described hereafter. While I
(the author of this thesis) am mainly responsible for research activities done
in this thesis, most of the work was conducted in collaboration with other
researchers. Thus, the pronoun “we” was used in this thesis to reflect the
collaborative research efforts.

Chapter 2: Literature Review on Data-Driven SV Assessment

Chapter 1 (the purple box in Fig. 2) provides a background on the topic of
data-driven SV assessment for this thesis. We conduct a literature review and
use thematic analysis [18] to devise a taxonomy of the key SV assessment tasks
that have been automated using data-driven approaches. For each theme of
tasks, we identify the key practices that have been followed by the primary
studies and highlighted the challenges with the current approaches to pave the
way for future research. Particularly, we pinpoint three main challenges (the
red boxes in Fig. 2) that potentially hinder the practical application of
current data-driven approaches for SV assessment. These challenges lead to the
four proposed solutions (the blue boxes in Fig. 2) in this thesis described in
Chapters 2, 3, 4, and 5 below.

Chapter 3: Automated Report-Level SV Assessment with Concept Drift

Chapter 2 addresses the first challenge of changing data in the context of
report-level SV assessment. Our review in Chapter 1 has pointed out that data-
driven SV assessment tasks have been most commonly carried out using the
information provided in SV reports. Each of these reports contains an expert-
verified summary of an SV, e.g., a brief description of the type, the
associated attack vectors and potential impacts if exploited. However, we
observe that the content of these reports continuously changes in practice as
experts usually need to use different/new terms to describe newly introduced
SVs. Such issue is referred to as concept drift [19] that can degrade the
performance of SV assessment models over time. However, most of the existing
report-level SV assessment models have not accounted for this concept drift
issue, potentially affecting their performance when deployed in the wild.
Using more than 100,000 SV reports from NVD, Chapter 2 performs a large-scale
investigation of the prevalence and impacts of the concept drift issue on
report-level SV assessment models. Moreover, we present a novel SV assessment
model that combines characters and words extracted from SV descriptions to
better capture the semantics of new terms, increasing the model robustness
against concept drift.

Chapter 4: Automated Function-Level SV Assessment

Chapter 3 addresses the second challenge, i.e., the need for SV assessment
using source code. Real-world SVs are usually rooted in source code, but
Chapter 1 has found that the current data-driven SV assessment efforts have
been mainly done on the report level. While report-level models have shown
promising performance for various SV assessment tasks [20, 21, 22, 23], these
models are still heavily dependent on SV reports that require expertise and
manual effort to generate. Our analysis in Chapter 3 has also found that most
(97%) of the reports used for SV assessment in the previous studies were not
available at the fixing time of respective SVs. Such unavailability of SV
reports affects the timeliness of report-level SV assessment in practice. A
promising alternative is to directly utilize vulnerable code available for SV
fixing to enable earlier SV assessment. Thus, in Chapter 3, we propose to use
(vulnerable) code functions instead of SV reports for SV assessment. Using
1,782 vulnerable functions curated from 200 open-source projects, we
particularly investigate the use of different code parts (i.e.,
(non-)vulnerable statements) in these functions for building effective
function-level SV assessment models. To the best of our knowledge, we are the
first to distill practices of performing data-driven SV assessment using
source code. Such an approach can enable earlier fixing prioritization of SVs
than the report-level SV assessment counterpart.

Chapter 5: Automated Commit-Level SV Assessment

Chapter 4 extends the work in Chapter 3 by addressing another practical
scenario of code-based SV assessment. In real-world software development,
developers have increasingly adopted DevOps for continuous integration, in
which incremental changes are made to codebases via code commits to implement
new features or fix bugs/SVs [24]. Meneely et al. [25] showed that such code
commits/changes can introduce SVs. However, it is wastage of resources to use
function-level SV assessment for these cases because functions are often not
entirely changed/added in code commits [26]. Instead, performing SV assessment
directly on these changes enables just-in-time, i.e., as soon as SVs are
introduced, provision of SV characteristics for SV fixing. To the best of our
knowledge, just-in-time SV assessment using code commits has never been
explored. Therefore, in Chapter 4, we propose DeepCVA, the first model that
automates commit-level SV assessment. This model leverages the multi-task DL
paradigm [27] to automate various SV assessment tasks simultaneously in a
unified model. We evaluate the effectiveness and efficiency of DeepCVA on
1,229 vulnerability-contributing commits in 246 open-source projects. DeepCVA
is expected to increase the efficiency in model (re-)training and maintenance
for continuous integration using DevOps in practice compared to conventional
task-wise models.

Chapter 6: Collection and Analysis of Developers’ SV Concerns on Question and
Answer Websites

Chapter 5 tackles the third challenge of lacking considerations of developers’
real-world SV concerns for SV assessment. Most of the current data-driven
models, including the ones in Chapter 2, 3, and 4, have automated the SV
assessment tasks that are based on the expert-defined SV taxonomies/standards
such as Common Weakness Enumeration (CWE) [28] or Common Vulnerability Scoring
System (CVSS) [29]. While these taxonomies are designed to be as general as
possible, they may not well represent the SV-related concerns that developers
regularly have. For instance, developers often encounter only a small subset
of SVs/SV types rather than all available ones; these SVs should be given a
higher priority during SV assessment as they are of more use/interest to
developers. Chapter 5 conducts the first empirical study on developers’ real-
world SV concerns using more than 70,000 SV-related posts curated from
Question and Answer (Q&A) websites. We use the Latent Dirichlet Allocation
topic modeling technique [30] to identify the commonly encountered issues. We
then characterize these posts in terms of their popularity, difficulty,
provided expertise, and available solutions. We also provide implications on
leveraging such characteristics for making (data-driven) SV assessment more
practical.

The key contributions of this thesis (the green boxes in Fig. 2) from the six
aforementioned chapters are summarized as follows.

  1. 1.

A systematization of knowledge of data-driven SV assessment (Chapter 1): (i) A
taxonomy of five SV assessment tasks. (ii) Detailed analysis of the pros and
cons of frequent data sources, features, prediction models, evaluation
techniques and evaluation metrics used for developing data-driven SV
assessment models. (iii) Three key challenges limiting the practical
application of data-driven SV assessment.

  2. 2.

Report-level assessment for ever-increasing SVs (Chapter 2). (i) Impact
analysis of changing data (concept drift) of SV reports on the development and
evaluation of SV assessment models. (ii) Concept-drift-aware models for
automating report-level SV assessment.

  3. 3.

Early and just-in-time SV assessment using source code (Chapters 3 and 4). (i)
Practices of developing effective data-driven models using vulnerable code
statements and context in functions for early SV assessment without delays
caused by missing SV reports. (ii) Just-in-time and efficient SV assessment
using code commits (where SVs are first added) with deep multi-task learning.

  4. 4.

Insights of developers’ real-world concerns that are useful for SV assessment
(Chapter 5). (i) A taxonomy of 13 key SV concerns commonly encountered by
developers in practice. (ii) Analysis of popularity, difficulty, expertise
level, solutions provided for these real-world concerns. (iii) Implications of
these concerns and their characteristics for practical data-driven SV
assessment.

###  3 Related Publications

All of the core chapters and contributions of this thesis have been published
during my PhD candidature. The list of directly related publications with
respect to each chapter is given below.

1

Triet Huynh Minh Le, Huaming Chen, and Muhammad Ali Babar, “A Survey on Data-
Driven Software Vulnerability Assessment and Prioritization,” ACM Computing
Surveys (CSUR), 2021. [CORE ranking: rank A*, Impact factor (2020): 10.282,
SJR rating: Q1] (Chapter 1)

2

Triet Huynh Minh Le and Muhammad Ali Babar, “On the Use of Fine-Grained
Vulnerable Code Statements for Software Vulnerability Assessment Models,” in
Proceedings of the 19th International Conference on Mining Software
Repositories (MSR). ACM, 2022. [CORE ranking: rank A, Acceptance rate: 34%]
(Chapter 2)

3

Triet Huynh Minh Le, Bushra Sabir, and Muhammad Ali Babar, “Automated Software
Vulnerability Assessment with Concept Drift,” in Proceedings of the 16th
International Conference on Mining Software Repositories (MSR). IEEE, 2019,
pp. 371–382. [CORE ranking: rank A, Acceptance rate: 25%] (Chapter 3)

4

Triet Huynh Minh Le, David Hin, Roland Croft, and Muhammad Ali Babar,
“DeepCVA: Automated Commit-Level Vulnerability Assessment with Deep Multi-Task
Learning,” in Proceedings of the 36th IEEE/ACM International Conference on
Automated Software Engineering (ASE). IEEE, 2021, pp. 717–729. [CORE ranking:
rank A*, Acceptance rate: 19%] (Chapter 4)

5

Triet Huynh Minh Le, David Hin, Roland Croft, and Muhammad Ali Babar,
“PUMiner: Mining Security Posts from Developer Question and Answer Websites
with PU Learning,” in Proceedings of the 17th International Conference on
Mining Software Repositories (MSR). ACM, 2020, pp. 350–361. [CORE ranking:
rank A, Acceptance rate: 25.7%] (Chapter 5)

6

Triet Huynh Minh Le, Roland Croft, David Hin, and Muhammad Ali Babar, “A
Large-Scale Study of Security Vulnerability Support on Developer Q&A
Websites,” in Proceedings of the 25th Evaluation and Assessment in Software
Engineering (EASE). ACM, 2021, pp. 109–118. [CORE ranking: rank A, Acceptance
rate: 27%, Nominated for the Best Paper Award] (Chapter 5)

In addition to the six aforementioned publications, I contributed as the first
author or coauthor of the following two publications during my PhD
candidature, which are not directly related to the materials in this thesis.

7

Triet Huynh Minh Le, Hao Chen, and Muhammad Ali Babar, “Deep Learning for
Source Code Modeling and Generation: Models, Applications, and Challenges,”
ACM Computing Surveys (CSUR), vol. 53, no. 3, pp. 1–38, 2020. [CORE ranking:
rank A*, Impact factor (2020): 10.282, SJR rating: Q1, High-impact research
work selected by Faculty of Engineering, Computer & Mathematical Sciences at
the University of Adelaide.]

8

Xuanyu Duan, Mengmeng Ge, Triet Huynh Minh Le, Faheem Ullah, Shang Gao,
Xuequan Lu, and Muhammad Ali Babar, “Automated Security Assessment for the
Internet of Things,” in 2021 IEEE 26th Pacific Rim International Symposium on
Dependable Computing (PRDC). IEEE, 2021, pp. 47–56. [CORE ranking: rank B]

###  4 Thesis Organization

The remainder of the thesis is organized as follows. Chapter 1 reports a
literature review on the tasks, practices and challenges of data-driven SV
assessment. Chapter 2 investigates the impacts of the concept drift issue on
report-level SV assessment models and proposes an automated technique to
address this issue. Chapter 3 explores the practices of building function-
level SV assessment models. Chapter 4 describes DeepCVA, a novel multi-task DL
model for automated commit-level SV assessment. Chapter 5 presents an
empirical study of the common SV concerns encountered by developers on Q&A
websites and distills respective implications for data-driven SV assessment.
Finally, Chapter 6 summarizes the main contributions and findings of the
thesis and suggests potential research avenues for future work in the area of
data-driven SV assessment.

##  Chapter 1 Literature Review on Data-Driven Software Vulnerability
Assessment

Related publication: This chapter is based on our paper titled “A Survey on
Data-Driven Software Vulnerability Assessment and Prioritization” published in
the ACM Computing Surveys journal (CORE A*) [11].

As mentioned in Chapter , Software Vulnerabilities (SVs) are increasing in
complexity and scale, posing great security risks to many software systems.
Given the limited resources in practice, SV assessment111In the original
review paper [11], we used the term SV assessment and prioritization instead
of SV assessment. However, in Chapter 2, the term SV assessment is used to
ensure consistency with the other parts of the thesis, and it is important to
note that SV assessment and SV assessment and prioritization can be used
interchangeably as most SV assessment tasks can be used for prioritizing SV
fixing. help practitioners devise optimal SV mitigation plans based on various
SV characteristics. The recent surges in SV data sources and data-driven
techniques such as Machine Learning and Deep Learning have taken SV assessment
to the next level. Chapter 1 provides a taxonomy of the key tasks performed by
the past research efforts in the area. We also highlight the best practices,
in terms of data sources, features, prediction models, evaluation techniques
and evaluation metrics, for data-driven SV assessment. At the end of the
review, we discuss some of the current and important challenges in the field
that set the stage for the key contributions of this thesis in Chapters 2, 3,
4 and 5.

###  1 Introduction

As discussed in Chapter , Software Vulnerability (SV) assessment is required
to prioritize the remediation of critical SVs (i.e., the ones that can lead to
devastating cyber-attacks) [10]. SV assessment includes tasks that determine
various characteristics such as the types, exploitability, impact and severity
levels of SVs [31]. Such characteristics help understand and select high-
priority SVs to resolve early given the limited effort and resources. For
example, SVs with simple exploitation and severe impacts likely require high
fixing priority.

There has been an active research area to assess and prioritize SVs using
increasingly large data from multiple sources. Many studies in this area have
proposed different Natural Language Processing (NLP), Machine Learning (ML)
and Deep Learning (DL) techniques to leverage such data to automate various
tasks such as predicting the Common Vulnerability Scoring System [29] (CVSS)
metrics (e.g., [23, 22, 21]) or public exploits (e.g., [32, 33, 34]). These
prediction models can learn the patterns automatically from vast SV data,
which would be otherwise impossible to do manually. Such patterns are utilized
to speed up the assessment processes of ever-increasing and more complex SVs,
significantly reducing practitioners’ effort. Despite the rising research
interest in data-driven SV assessment, to the best of our knowledge, there has
been no comprehensive review on the state-of-the-art methods and existing
challenges in this area. To bridge this gap, we are the first to review in-
depth the research studies that automate data-driven SV assessment tasks
leveraging SV data and NLP/ML/DL techniques.

The contributions of our review are summarized as follows:

  1. 1.

We categorize and describe the key tasks of data-driven SV assessment
performed in relevant primary studies.

  2. 2.

We synthesize and discuss the pros and cons of data, features, models,
evaluation methods and metrics commonly used in the reviewed studies.

  3. 3.

We highlight some key challenges with the current practices.

We believe that our findings can provide useful guidelines for researchers and
practitioners to effectively utilize data to perform SV assessment.

Related Work. There have been several existing surveys/reviews on SV analysis
and prediction, but they are fundamentally different from ours (see Table 1).
Ghaffarian et al. [3] conducted a seminal survey on ML-based SV analysis and
discovery. Subsequently, several studies [35, 36, 37, 38] reviewed DL
techniques for detecting vulnerable code. However, these prior reviews did not
describe how ML/DL techniques can be used to assess and prioritize the
detected SVs. There have been other relevant reviews on using Open Source
Intelligence (OSINT) (e.g., phishing or malicious emails/URLs/IPs) to make
informed security decisions [39, 40, 41]. However, these OSINT reviews did not
explicitly discuss the use of SV data and how such data can be leveraged to
automate the assessment processes. Moreover, most of the reviews on SV
assessment have focused on either static analysis tools [13] or rule-based
approaches (e.g., expert systems or ontologies) [9]. These methods rely on
pre-defined patterns and struggle to work with new types and different data
sources of SVs compared to contemporary ML or DL approaches presented in this
chapter [14, 15, 42]. Recently, Dissanayake et al. [43] reviewed the socio-
technical challenges and solutions for security patch management that involves
SV assessment after SV patches are identified. Unlike [43], we focus on the
challenges, solutions and practices of automating various SV assessment tasks
with data-driven techniques. We also consider all types of SV assessment
regardless of the patch availability.

Table 1: Comparison of contributions between our review and the existing
related surveys/reviews.

Study Contribution |  Focus on SV assessment |  Analysis of SV |
---|---|---|---
|  Analysis of data- | data sources |
| driven approaches |  |
Ghaffarian et al. 2017 [3] | – | – | ✓(Mostly ML)
Lin et al. 2020 [38] Semasaba et al. 2020 [37] Singh et al. 2020 [36] Zeng et al. 2020 [35] | – | – | ✓(Mostly DL)
Pastor et al. 2020 [39] | – | ✓(OSINT) | –
Sun et al. 2018 [41] Evangelista et al. 2020 [40] | – | ✓(OSINT) | ✓
Khan et al. 2018 [9] | ✓(Rule-based methods) | – | –
Kritikos et al. 2019 [13] | ✓(Static analysis) | ✓ | –
Dissanayake et al. 2020 [43] | ✓(Socio-technical aspects) | – | –
Our review | ✓ | ✓ | ✓

Chapter Organization. The remainder of the chapter is organized as follows.
Section 2 presents the scope, methodology and taxonomy covered in this
chapter. Sections 3, 4, 5, 6 and 7 review the studies in each theme of the
taxonomy and discuss the limitations/gaps and open opportunities at the end of
each theme. Section 8 identifies and discusses the common practices and
respective implications for data-driven SV assessment. Finally, section 9
concludes the review and discusses the open challenges of this research area.

###  2 Overview of the Literature Review

####  1 Scope

This review’s focus is on data-driven SV assessment. Unlike the existing
surveys on rule-based or experience-based SV assessment [13, 9, 43] that
hardly utilize the potential of SV data in the wild, this chapter aims to
review research papers that have leveraged such data to automate tasks in this
area using data-driven models. To keep our focus, we do not consider papers
that only perform manual analyses or descriptive statistics (e.g., taking
mean/median/variation of data) without using any data-driven models as these
techniques cannot automatically assess or prioritize new SVs. We also do not
directly compare the absolute performance of all the related studies as they
did not use exactly the same experimental setup (e.g., data sources and model
configurations). While it is theoretically possible to perform a comparative
evaluation of the identified techniques by establishing and using a common
setup, this type of evaluation is out of the scope of this chapter. However,
we still cover the key directions/techniques of the studies in sections 3, 4,
5, 6 and 7. We also provide in-depth discussion on the common practices of
these studies in section 8 and identify some current challenges with the field
in section 9.

####  2 Methodology

Study selection. Our study selection was inspired by the Systematic Literature
Review guidelines [44]. We first designed the search string: “‘software’ AND
vulner* AND (learn* OR data* OR predict*) AND (priority* OR assess* OR impact*
OR exploit* OR severity*) AND NOT (fuzz* OR dynamic* OR intrusion OR
adversari* OR malware* OR ‘vulnerability detection’ OR ‘vulnerability
discovery’ OR ‘vulnerability identification’ OR ‘vulnerability prediction’)”.
This search string covered the key papers (i.e., with more than 50 citations)
in the area and excluded many papers on general security and SV detection. We
then adapted this search string222This search string was customized for each
database and the database-wise search strings can be found at
https://figshare.com/s/da4d238ecdf9123dc0b8. to retrieve an initial list of
1,765 papers up to April 2021333Given the ever-growing nature of this field,
we maintain an up-to-date list of papers and resources on data-driven SV
assessment at https://github.com/lhmtriet/awesome-vulnerability-assessment.
More details are also given in Appendix 10. from various commonly used
databases such as IEEE Xplore, ACM Digital Library, Scopus, SpringerLink and
Wiley. We also defined the inclusion/exclusion criteria (see Table 2) to
filter out irrelevant/low-quality studies with respect to our scope in section
1. Based on these criteria and the titles and abstracts and keywords of 1,765
initial papers, we removed 1,550 papers. After reading the full-text and
applying the criteria on the remaining 215 papers, we obtained 70 papers
directly related to data-driven SV assessment. To further increase the
coverage of studies, we performed backward and forward snowballing [45] on
these 70 papers (using the above sources and Google Scholar) and identified 14
more papers that satisfied the inclusion/exclusion criteria. In total, we
included 84 studies in our review. We do not claim that we have collected all
the papers in this area, but we believe that our selection covered most of the
key studies to unveil the practices of data-driven SV assessment.

Table 2: Inclusion and exclusion criteria for study selection. Notes: We did
not limit the selection to only peer-reviewed papers as this is an emerging
field with many (high-quality) papers on Arxiv and most of them are under-
submission at the time of writing. However, to ensure the quality of the
papers on Arxiv, we only selected the ones with at least one citation.
Inclusion criteria
---
• I1. Studies that focused on SVs rather than hardware or human
vulnerabilities • I2. Studies that focused on assessment task(s) of SVs • I3.
Studies that used data-driven approaches (e.g., ML/DL/NLP techniques) for SV
assessment
Exclusion criteria
• E1. Studies that were not written in English • E2. Studies that we could not
retrieve their full texts • E3. Studies that were not related to Computer
Science • E4. Studies that were literature review or survey • E5. Studies that
only performed statistical analysis of SV assessment metrics • E6. Studies
that only focused on (automated) collection of SV data

Data extraction and synthesis of the selected studies. We followed the steps
of thematic analysis [18] to identify the taxonomy of data-driven SV
assessment tasks in sections 3, 4, 5, 6 and 7 as well as the key practices of
data-driven model building for automating these tasks in section 8. We first
conducted a pilot study of 20 papers to familiarize ourselves with data to be
extracted from the primary studies. After that, we generated initial codes and
then merged them iteratively in several rounds to create themes. Two of the
authors performed the analysis independently, in which each author analyzed
half of the selected papers and then reviewed the analysis output of the other
author. Any disagreements were resolved through discussions.

Figure 1: Taxonomy of studies on data-driven SV assessment.

####  3 Taxonomy of Data-Driven Software Vulnerability Assessment

Based on the scope in section 1 and the methodology in section 2, we
identified five main themes of the relevant studies in the area of data-driven
SV assessment (see Figure 1). Specifically, we extracted the themes by
grouping related SV assessment tasks that the reviewed studies aim to
automate/predict using data-driven models. Note that a paper is categorized
into more than one theme if that paper develops models for multiple cross-
theme tasks.

We acknowledge that there can be other ways to categorize the studies.
However, we assert the reliability of our taxonomy as all of our themes
(except theme 5) align with the security standards used in practice. For
example, Common Vulnerability Scoring System (CVSS) [29] provides a framework
to characterize exploitability, impact and severity of SVs (themes 1-3), while
Common Weakness Enumeration (CWE) [28] includes many vulnerability types
(theme 4). Hence, we believe our taxonomy can help identify and bridge the
knowledge gap between the academic literature and industrial practices, making
it relevant and potentially beneficial for both researchers and practitioners.
Details of each theme in our taxonomy are covered in subsequent sections.

Table 3: List of the reviewed papers in the Exploit Likelihood sub-theme of the Exploitation theme. Note: The nature of task of this sub-theme is binary classification of existence/possibility of proof-of-concept and/or real-world exploits.  Study |  Data source |  Data-driven technique
---|---|---
Bozorgi et al. 2010 [32] |  CVE, Open Source Vulnerability Database (OSVDB) |  Linear Support Vector Machine (SVM)
Sabottke et al. 2015 [33] |  NVD, Twitter, OSVDB, ExploitDB, Symantec security advisories, private Microsoft security advisories |  Linear SVM
Edkrantz et al. 2015 [46, 47] |  NVD, Recorded Future security advisories, ExploitDB |  Naïve Bayes, Linear SVM, Random forest
Bullough et al. 2017 [34] |  NVD, Twitter, ExploitDB |  Linear SVM
Almukaynizi et al. [48, 49] |  NVD, ExploitDB, Zero Day Initiative security advisories & Darkweb forums/markets |  SVM, Random forest, Naïve Bayes, Bayesian network, Decision tree, Logistic regression
Xiao et al. 2018 [50] |  NVD, SecurityFocus security advisories, Symantec
Spam/malicious activities based on daily blacklists from abuseat.org, spamhaus.org, spamcop.net, uceprotect.net, wpbl.info & list of unpatched SVs in hosts |  Identification of malicious activity groups with community detection algorithms + Random forest for exploit prediction
Tavabi et al. 2018 [51] |  NVD, 200 sites on Darkweb, ExploitDB, Symantec, Metasploit |  Paragraph embedding + Radial basis function kernel SVM
de Sousa et al. 2020 [52] |  NVD, Twitter, ExploitDB, Symantec
Avast, ESET, Trend Micro security advisories |  Linear SVM, Logistic regression, XGBoost, Light Gradient Boosting Machine (LGBM)
Fang et al. 2020 [53] |  NVD, ExploitDB, SecurityFocus, Symantec |  fastText + LGBM
Huang et al. 2020 [54] |  NVD, CVE Details, Twitter, ExploitDB, Symantec security advisories |  Random forest
Jacobs et al. 2020 [55] |  NVD, Kenna Security
Exploit sources: Exploit DB, Metasploit, FortiGuard Labs, SANS Internet Storm Center, Securewords CTU, Alienvault OSSIM, Canvas/D2 Security’s Elliot Exploitation Frameworks, Contagio, Reversing Labs |  XGBoost
Yin et al. 2020 [56] |  NVD, ExploitDB, General text: Book Corpus & Wikipedia for pretraining BERT models |  Fine-tuning BERT models pretrained on general text
Bhatt et al. 2021 [57] |  NVD, ExploitDB |  Features augmented by SV types + Decision tree, Random forest, Naïve Bayes, Logistic regression, SVM
Suciu et al. 2021 [58] |  NVD, Vulners database, Twitter, Symantec, SecurityFocus, IBM X-Force Threat Intelligence
Exploit sources: ExploitDB, Metasploit, Canvas, D2 Security’s Elliot, Tenable, Skybox, AlienVault, Contagio |  Multi-layer perceptron
Younis et al. 2014 [59] |  Vulnerable functions from NVD (Apache HTTP Server project), ExploitDB, OSVDB |  SVM
Yan et al. 2017 [60] |  Executables (binary code) of 100 Linux applications |  Combining ML (Decision tree) output & fuzzing with a Bayesian network
Tripathi et al. 2017 [61] |  Program crashes from VDiscovery [62, 63] & LAVA [64] datasets |  Static/Dynamic analysis features + Linear/Radial basis function kernel SVM
Zhang et al. 2018 [65] |  Program crashes from VDiscovery [62, 63] dataset |  $n$-grams of system calls from execution traces + Online passive-aggressive classifier

###  3 Exploitation Prediction

This section covers the Exploitation theme that automates the detection and
understanding of both Proof-of-Concept (PoC) and real-world exploits444An
exploit is a piece of code used to compromise vulnerable software [33]. Real-
world exploits are harmful & used in real host/network-based attacks. PoC
exploits are unharmful & used to show the potential threats of SVs in
penetration tests. targeting identified SVs. This theme outputs the origin of
SVs and how/when attackers would take advantage of such SVs to compromise a
system of interest, assisting practitioners to quickly react to the more
easily exploitable or already exploited SVs. The papers in this theme can be
categorized into three groups/sub-themes: (i) Exploit likelihood, (ii) Exploit
time, (iii) Exploit characteristics, as given in Tables 3, 4 and 5,
respectively.

####  1 Summary of Primary Studies

##### Exploit Likelihood

The first sub-theme is exploit likelihood that predicts whether SVs would be
exploited in the wild or PoC exploits would be released publicly (see Table
3). In 2010, Bozorgi et al. [32] were the first to use SV descriptions on
Common Vulnerabilities and Exposures (CVE) [66] and Open Source Vulnerability
Database (OSVDB)555http://osvdb.org. Note that this database has been
discontinued since 2016. to predict exploit existence based on the labels on
OSVDB. In 2015, Sabottke et al. [33] conducted a seminal study that used
Linear SVM and SV information on Twitter to predict PoC exploits on ExploitDB
[67] as well as real-world exploits on OSVDB, Symantec’s attack signatures
[68] and private Microsoft’s security advisories [69]. These authors urged to
explicitly consider real-world exploits as not all PoC exploits would result
in exploitation in practice. They also showed SV-related information on
Twitter666https://twitter.com can enable earlier detection of exploits than
using expert-verified SV sources (e.g., NVD).

Built upon these two foundational studies [32, 33], the literature has mainly
aimed to improve the performance and applicability of exploit prediction
models by leveraging more exploit sources and/or better data-driven
techniques/practices. Many researchers [47, 46, 48, 51, 49, 55] increased the
amount of ground-truth exploits using extensive sources other than ExploitDB
and Symantec in [32, 33]. The sources were security advisories such as Zero
Day Initiative [70], Metasploit [71], SecurityFocus [72], Recorded Future
[73], Kenna Security [74], Avast777https://avast.com/exploit-protection.php.
This link was provided by de Sousa et al. [52], but it is no longer
available., ESET [75], Trend Micro [76], malicious activities in hosts based
on traffic of spam/malicious IP addresses [50] and Darkweb
sites/forums/markets [77]. In addition to enriching exploit sources, better
data-driven models and practices for exploit prediction were also studied.
Ensemble models (e.g., Random forest, eXtreme Gradient Boosting (XGBoost)
[78], Light Gradient Boosting Machine (LGBM) [79]) were shown to outperform
single-model baselines (e.g., Naïve Bayes, SVM, Logistic regression and
Decision tree) for exploit prediction [53, 55, 52, 54]. Additionally, Bullough
et al. [34] identified and addressed several issues with exploit prediction
models, e.g., time sensitivity of SV data, already-exploited SVs before
disclosure and training data imbalance, helping to improve the practical
application of such models. Recently, Yin et al. [56] demonstrated that
transfer learning is an alternative solution for improving the performance of
exploit prediction with scarcely labeled exploits. Specifically, these authors
pre-trained a DL model, BERT [80], on massive non-SV sources (e.g., text on
Book Corpus [81] and Wikipedia [82]) and then fine-tuned this pre-trained
model on SV data using additional pooling and dense layers. Bhatt et al. [57]
also suggested that incorporating the types of SVs (e.g., SQL injection) into
ML models can further enhance the predictive effectiveness. Suciu et al. [58]
empirically showed that unifying SV-related sources used in prior work (e.g.,
SV databases [32], social media [33], SV-related discussions [51] and PoC code
in ExploitDB [55]) supports more effective and timely prediction of functional
exploits [83].

Besides using SV descriptions as input for exploit prediction, several studies
in this sub-theme have also predicted exploits on the code level. Younis et
al. [59] predicted the exploitability of vulnerable functions in the Apache
HTTP Server project. Specifically, these authors used an SVM model with
features extracted from the dangerous system calls [84] in entry
points/functions [85] and the reachability from any of these entry points to
vulnerable functions [86]. Moving from high-level to binary code, Yan et al.
[60] first used a Decision tree to obtain prior beliefs about SV types in 100
Linux applications using static features (e.g., hexdump) extracted from
executables. Subsequently, they applied various fuzzing tools (i.e., Basic
Fuzzing Framework [87] and OFuzz [88]) to detect SVs with the ML-predicted
types. They finally updated the posterior beliefs about the exploitability
based on the outputs of the ML model and fuzzers using a Bayesian network. The
proposed method outperformed
!exploitable,888https://microsoft.com/security/blog/2013/06/13/exploitable-
crash-analyzer-version-1-6 a static crash analyzer provided by Microsoft.
Tripathi et al. [61] also predicted SV exploitability from crashes (i.e.,
VDiscovery [62, 63] and LAVA [64] datasets) using an SVM model and static
features from core dumps and dynamic features generated by the Last Branch
Record hardware debugging utility. Zhang et al. [65] proposed two improvements
to Tripathi et al. [61]’s approach. These authors first replaced the hardware
utility in [61] that may not be available for resource-constrained devices
(e.g., IoT) with sequence/$n$-grams of system calls extracted from execution
traces. They also used an online passive-aggressive classifier [89] to enable
online/incremental learning of exploitability for new crash batches on-the-
fly.

Table 4: List of the reviewed papers in the Exploit Time sub-theme of the Exploitation theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Bozorgi et al. 2010 [32] |  Binary classification: Likelihood that SVs would be exploited within 2 to 30 days after disclosure |  CVE, OSVDB |  Linear SVM
Edkrantz 2015 [46] |  Binary classification: Likelihood of SV exploits within 12 months after disclosure |  NVD, ExploitDB, Recorded Future security advisories |  SVM, K-Nearest Neighbors (KNN), Naïve Bayes, Random forest
Jacobs et al. 2019 [90, 91] |  |  NVD, Kenna Security
Exploit sources: Exploit DB, Metasploit, D2 Security’s Elliot & Canvas Exploitation Frameworks, Fortinet, Proofpoint, AlienVault & GreyNoise |  Logistic regression
Chen et al. 2019 [92, 93] |  Binary classification: Likelihood that SVs would be exploited within 1/3/6/9/12 months after disclosure
Regression: number of days until SV exploits after disclosure |  CVE, Twitter, ExploitDB, Symantec security advisories |  Graph neural network embedding + Linear regression, Bayes, Random forest, XGBoost, Lasso/Ridge regression

##### Exploit Time

After predicting the likelihood of SV exploits in the previous sub-theme, this
sub-theme provides more fine-grained information about exploit time (see Table
4). Besides performing binary classification of exploits, Bozorgi et al. [32]
and Edkrantz [46] also predicted the time frame (2-30 days in [32] and 12
months in [46]) within which exploits would happen after the disclosure of
SVs. Jacobs et al. [90, 91] then leveraged multiple sources containing both
PoC and real-world exploits, as given in Table 4, to improve the number of
labeled exploits, enhancing the prediction of exploit appearance within 12
months. Chen et al. [92] predicted whether SVs would be exploited within 1-12
months and the exploit time (number of days) after SV disclosure using Twitter
data. The authors proposed a novel regression model whose feature embedding
was a multi-layer graph neural network [94] capturing the content and
relationships among tweets, respective tweets’ authors and SVs. The proposed
model outperformed many baselines and was integrated into the VEST system [93]
to provide timely SV assessment information for practitioners. To the best of
our knowledge, at the time of writing, Chen et al. [92, 93] have been the only
ones pinpointing the exact exploit time of SVs rather than large/uncertain
time-frames (e.g., months) in other studies, helping practitioners to devise
much more fine-grained remediation plans.

Table 5: List of the reviewed papers in the Exploit Characteristics sub-theme of the Exploitation theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Yamamoto et al. 2015 [95] |  Multi-class classification: CVSS v2 (Access Vector & Access Complex- ity metrics) Binary classification: CVSS v2 (Authentication metric) |  NVD |  Supervised Latent Dirichlet Allocation (LDA)
Wen et al. 2015 [96] |  NVD, OSVDB, SecurityFocus, IBM X-Force |  Radial basis function kernel SVM
Le et al. 2019 [23] |  NVD |  Concept-drift-aware models with Naïve Bayes, KNN, Linear SVM, Random forest, XGBoost, LGBM
Toloudis et al. 2016 [97] |  Correlation analysis: CVSS v2 |  NVD |  Principal component analysis & Spearman correlation coefficient
Ognawala et al. 2018 [98] |  Multi-class classification: CVSS v3 (Attack Vector, Attack Complexity & Privileges Required metrics) Binary classification: CVSS v3 (User Interaction metric) |  NVD (buffer overflow SVs) & Source code of vulnerable software/components |  Combining static analysis tool (Macke [99]) & ML classifiers (Naïve Bayes & Random forest)
Chen et al. 2019 [93] |  CVE, NVD, Twitter |  Graph convolutional network
Elbaz et al. 2020 [100] |  Multi-class/Binary classification: CVSS v2/v3 |  NVD |  Mapping outputs of Linear regression to CVSS metrics with closest values
Jiang et al. 2020 [101] |  |  NVD, ICS Cert, Vendor websites (Resolve inconsistencies with a majority vote) |  Logistic regression
Gawron et al. 2017 [102] |  Multi-target classification: CVSS v2 |  NVD |  Naïve Bayes, Multi-layer Perceptron (MLP)
Spanos et al. 2018 [22] |  |  NVD |  Random forest, boosting model, Decision tree
Gong et al. 2019 [103] |  Multi-task classification: CVSS v2 |  NVD |  Bi-LSTM with attention mechanism
Chen et al. 2010 [104] |  Multi-class classification: Platform-specific vulnerability locations (Local, Remote, Local area network) & vulnerability causes (e.g., Access/Input/Origin validation error) |  NVD, Secunia vulnerability database, SecurityFocus, IBM X-Force |  Linear SVM
Ruohonen et al. 2017 [105] |  Binary classification: Web-related exploits or not |  ExploitDB |  LDA + Random forest
Aksu et al. 2018 [106] |  Multi-class classification: author-defined pre-/post-condition privileges (None, OS (Admin/User), App (Admin/User)) |  NVD |  RBF network, Linear SVM, NEAT [107], MLP
Liu et al. 2019 [108] |  |  NVD |  Information gain + Convolutional neural network
Kanakogi et al. 2021 [109] |  Multi-class classification: Common Attack Pattern Enumeration and Classification (CAPEC) |  NVD, CAPEC |  Doc2vec/tf-idf with cosine similarity

##### Exploit Characteristics

Exploit characteristics is the final sub-theme that reveals various
requirements/means of exploits (see Table 5), informing the potential scale of
SVs; e.g., remote exploits likely affect more systems than local ones. The
commonly used outputs are the Exploitability metrics provided by versions 2
[110] and 3 [111, 83] of Common Vulnerability Scoring System (CVSS).

Many studies have focused on predicting and analyzing version 2 of CVSS
exploitability metrics (i.e., Access Vector, Access Complexity and
Authentication). Yamamoto et al. [95] were the first one to leverage
descriptions of SVs on NVD together with a supervised Latent Dirichlet
Allocation topic model [112] to predict these CVSS metrics. Subsequently, Wen
et al. [96] used Radial Basis Function (RBF)-kernel SVM and various SV
databases/advisories other than NVD (e.g., SecurityFocus, OSVDB and IBM
X-Force [113]) to predict the metrics. Le et al. [23]999This study is
presented in Chapter 2. later showed that the prediction of CVSS metrics
suffered from the concept drift issue; i.e., descriptions of new SVs may
contain Out-of-Vocabulary terms for prediction models. They proposed to
combine sub-word features with traditional Bag-of-Word (BoW) features to infer
the semantics of novel terms/words from existing ones, helping assessment
models be more robust against concept drift. Besides prediction, Toloudis et
al. [97] used principal component analysis [114] and Spearman’s $\rho$
correlation coefficient to reveal the predictive contribution of each word in
SV descriptions to each CVSS metric. However, this technique does not directly
produce the value of each metric.

Recently, several studies have started to predict CVSS version 3
exploitability metrics including the new Privileges and User Interactions.
Ognawala et al. [98] fed the features generated by a static analysis tool,
Macke [99], to a Random forest model to predict these CVSS version 3 metrics
for vulnerable software/components. Later, Chen et al. [93] found that many
SVs were disclosed on Twitter before on NVD. Therefore, these authors
developed a system built on top of a Graph Convolutional Network [115]
capturing the content and relationships of related Twitter posts about SVs to
enable more timely prediction of the CVSS version 3 metrics. Elbaz et al.
[100] developed a linear regression model to predict the numerical output of
each metric and then obtained the respective categorical value with the
numerical value closest to the predicted value. For example, a predicted value
of 0.8 for Attack Vector CVSS v3 is mapped to Network (0.85) [111]. To prepare
a clean dataset to predict these CVSS metrics, Jiang et al. [101] replaced
inconsistent CVSS values in various SV sources (i.e., NVD, ICS CERT and vendor
websites) with the most frequent value.

Instead of building a separate model for each CVSS metric, there has been
another family of approaches predicting these metrics using a single model to
increase efficiency. Gawron et al. [102] and Spanos et al. [22] predicted
multiple CVSS metrics as a unique string instead of individual values. The
output of each metric is then extracted from the concatenated string. Later,
Gong et al. [103] adopted the idea of a unified model from the DL perspective
by using the multi-task learning paradigm [116] to predict CVSS metrics
simultaneously. The model has a feature extraction module (based on a Bi-LSTM
model with attention mechanism [117]) shared among all the CVSS metrics/tasks,
yet specific prediction head/layer for each metric/task. This model
outperformed single-task counterparts while requiring much less time to
(re-)train.

Although CVSS exploitability metrics were most commonly used, several studies
used other schemes for characterizing exploitation. Chen et al. [104] used
Linear SVM and SV descriptions to predict multiple SV characteristics,
including three SV locations (i.e., Local, LAN and Remote) on SecurityFocus
[72] and Secunia [118] databases as well as 11 SV
causes101010Access/Input/Origin validation error,
Atomicity/Configuration/Design/Environment/Serialization error, Boundary
condition error, Failure on exceptions, Race condition error on SecurityFocus.
Regarding the exploit types, Rouhonen et al. [105] used LDA [30] and Random
forest to classify whether an exploit would affect a web application. This
study can help find relevant exploits in components/sub-systems of a large
system. For privileges, Aksu et al. [106] extended the Privileges Required
metric of CVSS by incorporating the context (i.e., Operating system or
Application) to which privileges are applied (see Table 5). They found MLP
[119] to be the best-performing model for obtaining these privileges from SV
descriptions. They also utilized the predicted privileges to generate attack
graphs (sequence of attacks from source to sink nodes). Liu et al. [108]
advanced this task by combining information gain for feature selection and
Convolutional Neural Network (CNN) [120] for feature extraction. Regarding
attack patterns, Kanakogi et al. [109] found Doc2vec [121] to be more
effective than term-frequency inverse document frequency (tf-idf) when
combined with cosine similarity to find the most relevant Common Attack
Pattern Enumeration and Classification (CAPEC) [122] for a given SV on NVD.
Such attack patterns can manifest how identified SVs can be exploited by
adversaries, assisting the selection of suitable countermeasures.

####  2 Theme Discussion

In the Exploitation theme, the primary tasks are binary classification of
whether Proof-of-Concept (PoC)/real-world exploits of SVs would appear and
multi-classification of exploit characteristics based on CVSS. PoC exploits
mostly come from ExploitDB [67]; whereas, real-world exploits, despite coming
from multiple sources, are still much scarcer than PoC counterparts.
Consequently, the models predicting real-world exploits have generally
performed worse than those for PoC exploits. Similarly, the performance of the
models determining CVSS v3 exploitability metrics has been mostly lower than
that of the CVSS v2 based models. However, real exploits and CVSS v3 are
usually of more interest to the community. The former can lead to real cyber-
attacks and the latter is the current standard in practice. To improve the
performance of these practical tasks, future work can consider adapting the
patterns learned from PoC exploits and old CVSS versions to real exploits and
newer CVSS versions, respectively, e.g., using transfer learning [123].

Besides the above tasks, there are other under-explored tasks targeting fine-
grained prediction of exploits. In fact, mitigation of exploits in practice
usually requires more information besides simply determining whether an SV
would be exploited. Information gathered from predicting when and how the
exploits would happen is also needed to devise better SV fixing prioritization
and mitigation plans. VEST [93] is one of the first and few systems aiming to
provide such all-in-one information about SV exploitation. However, this
system currently only uses data from NVD/CVE and Twitter, which can be
extended to incorporate more (exploit-related) sources and more sophisticated
data-driven techniques in the future.

Most of the current studies have used SV descriptions on NVD and other
security advisories to predict the exploitation-related metrics. This is
surprising as SV descriptions do not contain root causes of SVs. Instead, SVs
are rooted in source code, yet there is little work on code-based exploit
prediction. So far, Younis et al. [59] have been the only ones using source
code for exploit prediction, but their approach still requires manual
identification of dangerous function calls in C/C++. More work is required to
employ data-driven approaches to alleviate the need for manually defined rules
to improve the effectiveness and generalizability of code-based exploit
prediction.

###  4 Impact Prediction

This section describes the Impact theme that determines the (negative) effects
that SVs have on a system of interest if such SVs are exploited. There are
five key tasks that the papers in this theme have automated/predicted: (i)
Confidentiality impact, (ii) Integrity impact, (iii) Availability impact, (iv)
Scope and (v) Custom vulnerability consequences (see Table 6).

Table 6: List of the reviewed papers in the Impact theme. Note: We grouped the first four sub-themes as they were mostly predicted together. Study | Nature of task | Data source | Data-driven technique
---|---|---|---
Sub-themes: 1. Confidentiality, 2. Integrity, 3. Availability & 4\. Scope
(only in CVSS v3)
Yamamoto et al. 2015 [95] |  Multi-class classification: CVSS v2 |  NVD |  Supervised Latent Dirichlet Allocation
Wen et al. 2015 [96] |  |  NVD, OSVDB, SecurityFocus, IBM X-Force |  Radial basis function kernel SVM
Le et al. 2019 [23] |  |  NVD |  Concept-drift-aware models with Naïve Bayes, KNN, Linear SVM, Random forest, XGBoost, LGBM
Toloudis et al. 2016 [97] |  Correlation analysis: CVSS v2 |  NVD |  Principal component analysis & Spearman correlation coefficient
Ognawala et al. 2018 [98] |  Multi-class classification: CVSS v3 Binary classification: Scope in CVSS v3 |  NVD (buffer overflow SVs) & Source code of vulnerable software/components |  Combining static analysis tool (Macke [99]) & ML classifiers (Naïve Bayes & Random forest)
Chen et al. 2019 [93] |  CVE, NVD, Twitter |  Graph convolutional network
Elbaz et al. 2020 [100] |  Multi-class classification: CVSS v2/v3 Binary classification: Scope in CVSS v3 |  NVD |  Mapping outputs of Linear regression outputs to CVSS metrics with closest values
Jiang et al. 2020 [101] |  NVD, ICS Cert, Vendor websites (Resolve inconsistencies with a majority vote) |  Logistic regression
Gawron et al. 2017 [102] |  Multi-target classification: CVSS v2 |  NVD |  Naïve Bayes, MLP
Spanos et al. 2018 [22] |  |  NVD |  Random forest, boosting model, Decision tree
Gong et al. 2019 [103] |  Multi-task classification: CVSS v2 |  NVD |  Bi-LSTM with attention mechanism
Sub-theme: 5. Custom Vulnerability Consequences
Chen et al. 2010 [104] |  Multi-label classification: Platform-specific impacts (e.g., Gain system access) |  NVD, Secunia vulnerability database, SecurityFocus, IBM X-Force |  Linear SVM

####  1 Summary of Primary Studies

##### Confidentiality, Integrity, Availability, and Scope

A majority of the papers have focused on the impact metrics provided by CVSS,
including versions 2 [110] and 3 [111, 83]. Versions 2 and 3 share three
impact metrics Confidentiality, Integrity and Availability. Version 3 also has
a new metric, Scope, that specifies whether an exploited SV would affect only
the system that contains the SV. For example, Scope changes when an SV
occurring in a virtual machine affects the whole host machine, in turn
increasing individual impacts.

The studies that predicted the CVSS impact metrics are mostly the same as the
ones predicting the CVSS exploitability metrics in section 3. Given the
overlap, we hereby only describe the main directions and techniques of the
Impact-related tasks rather than iterating the details of each study. Overall,
a majority of the work has focused on classifying CVSS impact metrics
(versions 2 and 3) using three main learning paradigms: single-task [95, 96,
23, 98, 93, 100, 101], multi-target [102, 22] and multi-task [103] learning.
Instead of developing a separate prediction model for each metric like the
single-task approach, multi-target and multi-task approaches only need a
single model for all tasks. Multi-target learning predicts concatenated
output; whereas, multi-task learning uses shared feature extraction for all
tasks and task-specific softmax layers to determine the output of each task.
These three learning paradigms were powered by applying and/or customizing a
wide range of data-driven methods. The first method was to use single ML
classifiers like supervised Latent Dirichlet Allocation [95], Principal
component analysis [97], Naïve Bayes [23, 98, 102], Logistic regression [101],
Kernel-based SVM [96], Linear SVM [23], KNN [23] and Decision tree [22]. Other
studies employed ensemble models combining the strength of multiple single
models such as Random forest [23, 98], boosting model [22] and XGBoost/LGBM
[23]. Recently, more studies moved towards more sophisticated DL architectures
such as MLP [102], attention-based (Bi-)LSTM [103] and graph neural network
[93]. Ensemble and DL models usually beat the single ones, but there is a lack
of direct comparisons between these two emerging model types.

##### Custom Vulnerability Consequences

To devise effective remediation strategies for a system of interest in
practice, practitioners may want to know custom vulnerability consequences
which are more interpretable than the levels of impact provided by CVSS. Chen
et al. [104] curated a list of 11 vulnerability consequences111111Gain system
access, Bypass security, Configuration manipulation, Data/file manipulation,
Denial of Service, Privilege escalation, Information leakage, Session
hijacking, Cross-site scripting (XSS), Source spoofing, Brute-force proneness.
from X-Force [113] and Secunia [118] vulnerability databases. They then used a
Linear SVM model to perform multi-label classification of these consequences
for SVs, meaning that an SV can lead to more than one consequence. To the best
of our knowledge, this is the only study that has pursued this research
direction so far.

####  2 Theme Discussion

In the Impact theme, the common task is to predict the impact base metrics
provided by CVSS versions 2 and 3. Similar to the Exploitation theme, the
models for CVSS v3 still require more attention and effort from the community
to reach the same performance level as the models for CVSS v2. These impact
metrics are also usually predicted together with the exploitability metrics
given their similar nature (multi-class classification) using either task-wise
models or a unified (multi-target or multi-task) model. Multi-target and
multi-task learning are promising as they can reduce the time for continuous
(re)training and maintenance when deployed in production.

Besides CVSS impact metrics, other fine-grained SV consequences have also been
explored [104], but there is still no widely accepted taxonomy for such
consequences. Thus, these consequences have seen less adoption in practice
than CVSS metrics, despite being potentially useful by providing more concrete
information about what assets/components in a system that an SV can
compromise. We recommend that future work investigate SV-related issues that
practitioners commonly encounter in practice to potentially create a
systematic taxonomy of custom SV consequences.

###  5 Severity Prediction

This section discusses the work in the Severity theme. Severity is often a
function/combination of Exploitation (section 3) and Impact (section 4). SVs
with higher severity usually require more urgent remediation. There are three
main prediction tasks in this theme: (i) Severe vs. Non-severe, (ii) Severity
levels and (iii) Severity score, shown in Tables 7, 8 and 9, respectively.

Similar to the Exploitation and Impact themes, many studies in the Severity
theme have used CVSS versions 2 and 3. According to both CVSS versions, the
severity score shares the same range from 0 to 10, with an increment of 0.1.
Based on the score, the existing studies have either defined a threshold to
decide whether an SV is severe (requiring high attention), or predicted
levels/groups of severity score that require a similar amount of attention or
determined the raw score value.

####  1 Summary of Primary Studies

##### Severe vs. Non-Severe

Table 7: List of the reviewed papers in the Severe vs. Non-Severe sub-theme of the Severity theme. Note: The nature of task here is binary classification of severe SVs with High/Critical CVSS v2/v3 severity levels.  Study |  Data source (software project) |  Data-driven technique
---|---|---
Kudjo et al. 2019 [124] |  NVD (Mozilla Firefox, Google Chrome, Internet Explorer, Microsoft Edge, Sea Monkey, Linux Kernel, Windows 7, Windows 10, Mac OS, Chrome OS) |  Term frequency & inverse gravity moment weighting + KNN, Decision tree, Random forest
Chen et al. 2020 [125] |  NVD (Adobe Flash Player, Enterprise Linux, Linux Kernel, Foxit Reader, Safari, Windows 10, Microsoft Office, Oracle Business Suites, Chrome, QuickTime) |  Term frequency & inverse gravity moment weighting + KNN, Decision tree, Naïve Bayes, SVM, Random forest
Kudjo et al. 2020 [126] |  NVD (Google Chrome, Mozilla Firefox, Internet Explorer and Linux Kernel) |  Find the best smallest training dataset using KNN, Logistic regression, MLP, Random forest
Malhotra et al. 2021 [127] |  NVD (Apache Tomcat) |  Chi-square/Information gain + bagging technique, Random forest, Naïve Bayes, SVM

The first group of studies have classified whether an SV is severe or non-
severe, making it a binary classification problem (see Table 7). These studies
have typically selected severe SVs as the ones with at least High severity
level (i.e., CVSS severity score $\geq$ 7.0). Kudjo et al. [124] showed that
using term frequency (BoW) with inverse gravity moment weighting [128] to
extract features from SV descriptions can enhance the performance of ML models
(i.e., KNN, Decision tree and Random forest) in predicting the severity of
SVs. Later, Chen et al. [125] confirmed that this feature extraction method
was also effective for more projects and classifiers (e.g., Naïve Bayes and
SVM). Besides investigating feature extraction, Kudjo et al. [126] also
highlighted the possibility of finding Bellwether, i.e., the smallest set of
data that can be used to train an optimal prediction model, for classifying
severity. Recently, Malhotra et al. [127] revisited this task by showing that
Chi-square and information gain can be effective dimensionality reduction
techniques for multiple classifiers, i.e., bagging technique, Random forest,
Naïve Bayes and SVM.

##### Severity Levels

Table 8: List of the reviewed papers in the Severity Levels sub-theme of the Severity theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Spanos et al. 2017 [20] |  Multi-class classification: NVD severity levels based on CVSS v2 & WIVSS (High, Medium, Low) |  NVD |  Decision tree, SVM, MLP
Wang et al. 2019 [129] |  Multi-class classification: NVD severity levels based on CVSS v2(High, Medium, Low) |  NVD (XSS attacks) |  XGBoost, Logistic regression, SVM, Random forest
Le et al. 2019 [23] |  NVD |  Concept-drift-aware models with Naïve Bayes, KNN, Linear SVM, Random forest, XGBoost, LGBM
Liu et al. 2019 [130] |  |  NVD, China National Vulnerability Database (XSS attacks) |  Recurrent Convolutional Neural Network (RCNN), Convolutional Neural Network (CNN), Long-Short Term Memory (LSTM)
Sharma et al. 2020 [131] |  |  CVE Details |  CNN
Han et al. 2017 [21] |  Multi-class classification: Atlassian categories of CVSS severity score (Critical, High, Medium, Low) |  CVE Details |  1-layer CNN, 2-layer CNN, CNN-LSTM, Linear SVM
Sahin et al. 2019 [132] |  NVD |  1-layer CNN, LSTM, XGBoost, Linear SVM
Nakagawa et al. 2019 [133] |  |  CVE Details |  Character-level CNN vs. Word-based CNN + Linear SVM
Gong et al. 2019 [103] |  Multi-task classification: Atlassian categories of CVSS severity score (Critical, High, Medium, Low) |  CVE Details |  Bi-LSTM with attention mechanism
Chen et al. 2010 [104] |  Multi-class classification: severity levels of Secunia (Extremely/highly/ moderately/less/non- critical) |  CVE, Secunia vulnerability database, SecurityFocus, IBM X-Force |  Linear SVM
Zhang et al. 2020 [134] |  Multi-class classification: Platform-specific levels (High/Medium/Low) |  China National Vulnerability Database |  Logistic regression, Linear discriminant analysis, KNN, CART, SVM, bagging/boosting models
Khazaei et al. 2016 [135] |  Multi-class classification: 10 severity score bins (one unit/bin) |  CVE & OSVDB |  Linear SVM, Random forest, Fuzzy system

Rather than just performing binary classification of whether an SV is severe,
several studies have identified one among multiple severity levels that an SV
belongs to (see Table 8). This setting can be considered as multi-class
classification. Spanos et al. [20] were to first one to show the applicability
of ML to classify SVs into one of the three severity levels using SV
descriptions. These three levels are provided by NVD and based on the severity
score of CVSS version 2 [110] and WIVSS [136], i.e., Low (0.0 – 3.9), Medium
(4.0 – 6.9), High (7.0 – 10.0). Note that WIVSS assigns different weights for
the Confidentiality, Integrity and Availability impact metrics of CVSS,
enhancing the ability to capture varied contributions of these impacts to the
final severity score. Later, Wang et al. [129] showed that XGBoost [78]
performed the best among the investigated ML classifiers for predicting these
three NVD-based severity levels. Le et al. [23] also confirmed that ensemble
methods (e.g., XGBoost [78], LGBM [79] and Random forest) outperformed single
models (e.g., Naïve Bayes, KNN and SVM) for this task. Predicting severity
levels has also been tackled with DL techniques [130, 131] such as Recurrent
Convolutional Neural Network (RCNN) [137], Convolutional Neural Network (CNN)
[120], Long-Short Term Memory (LSTM) [138]. These studies showed potential
performance gain of DL models compared to traditional ML counterparts. Han et
al. [21] showed that DL techniques (i.e., 1-layer CNN) also achieved promising
results for predicting a different severity categorization, namely Atlassian’s
levels.121212https://www.atlassian.com/trust/security/security-severity-levels
Such findings were successfully replicated by Sahin et al. [132]. Nakagawa et
al. [133] further enhanced the DL model performance for the same task by
incorporating the character-level features into a CNN model [139].
Complementary to performance enhancement, Gong et al. [103] proposed to
predict these severity levels concurrently with other CVSS metrics in a single
model using multi-task learning [116] powered by an attention-based Bi-LSTM
shared feature extraction model. The unified model was demonstrated to
increase both the prediction effectiveness and efficiency. Besides Atlassian’s
categories, several studies applied ML models to predict severity levels on
other platforms such as Secunia [104] and China National Vulnerability
Database131313https://www.cnvd.org.cn [134]. Instead of using textual
categories, Khazaei et al. [135] divided the CVSS severity score into 10 bins
with 10 increments each (e.g., values of 0 – 0.9 are in one bin) and obtained
decent results (86-88% Accuracy) using Linear SVM, Random forest and Fuzzy
system.

##### Severity Score

Table 9: List of the reviewed papers in the Severity Score sub-theme of the Severity theme. Notes: †denotes that the severity score is computed from ML-predicted base metrics using the formula provided by an assessment framework (CVSS and/or WIVSS). Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Sahin et al. 2019 [132] |  Regression: CVSS v2 (0-10) |  NVD |  1-layer CNN, LSTM, XGBoost regressor, Linear regression
Wen et al. 2015 [96] |  |  OSVDB, SecurityFocus, IBM X-Force |  Radial basis function kernel SVM†
Ognawala et al. 2018 [98] |  Regression: CVSS v3 (0-10) |  NVD (buffer overflow SVs) |  Combining a static analysis tool (Macke [99]) & ML classifiers (Naïve Bayes & Random forest)†
Chen et al. 2019 [93, 140] |  |  CVE, NVD, Twitter |  Graph convolutional network
Anwar et al. 2020 [141, 142] |  |  NVD |  Linear regression, Support vector regression, CNN, MLP
Elbaz et al. 2020 [100] |  Regression: CVSS v2/v3 (0-10) |  NVD |  Mapping outputs of Linear regression to CVSS metrics with closest values†
Jiang et al. 2020 [101] |  |  NVD, ICS Cert, Vendor websites (Resolve inconsistencies with a majority vote) |  Logistic regression†
Spanos et al. 2018 [22] |  Regression: CVSS v2 & WIVSS (0-10) |  NVD |  Random forest, boosting model, Decision tree†
Toloudis et al. 2016 [97] |  Correlation analysis: CVSS v2 & WIVSS (0-10) |  NVD |  Principal component analysis & Spearman correlation coefficient

To provide even more fine-grained severity value than the categories, the last
sub-theme has predicted the severity score (see Table 9). Using SV
descriptions on NVD, Sahin et al. [132] compared the performance of ML-based
regressors (e.g., XGBoost [78] and Linear regression) and DL-based ones (e.g.,
CNN [120] and LSTM [138]) for predicting the severity score of CVSS version 2
[110]. These authors showed that DL-based approaches generally outperformed
the ML-based counterparts. For CVSS version 3 [111, 83], Chen et al. [93, 140]
and Anwar et al. [141, 142] also reported the strong performance of DL-based
models (e.g., CNN and graph convolutional neural network [115]). Some other
studies did not directly predict severity score from SV descriptions, instead
they aggregated the predicted values of the CVSS Exploitability (see section
3) and Impact metrics (see section 4) using the formulas of CVSS version 2
[96, 22, 100, 101], version 3 [98, 100, 101] and WIVSS [22]. We noticed the
papers predicting both versions (e.g., CVSS versions 2 vs. 3 or CVSS version 2
vs. WIVSS) usually obtained better performance for version 3 and WIVSS than
version 2 [100, 101]. These findings may suggest that the improvements made by
experts in version 3 and WIVSS compared to version 2 help make the patterns in
severity score clearer and easier for ML models to capture. In addition to
predicting severity score, Toloudis et al. [97] examined the correlation
between words in descriptions of SVs and the severity values of such SVs,
aiming to shed light on words that increase or decrease the severity score of
SVs.

####  2 Theme Discussion

In the Severity theme, predicting the severity levels is the most prevalent
task, followed by severity score prediction and then binary classification of
the severity. In practice, severity score gives more fine-grained information
(fewer SVs per value) for practitioners to rank/prioritize SVs than
categorical/binary levels. However, predicting continuous score values is
usually challenging and requires more robust models as this task involves
higher uncertainty to learn inherent patterns from data than classifying
fixed/discrete levels. We observed that DL models such as graph neural
networks [93, 140], LSTM [132] and CNN [141, 142] have been shown to be better
than traditional ML models for predicting severity score. However, most of
these studies did not evaluate their models in a continuous deployment setting
to investigate how the models will cope with changing patterns of new SVs over
time. This issue particularly manifests and requires remediation in the
context of report-level SV assessment (see section 9) where SV descriptions,
the main input for SV assessment models, contain changing/new terms to
describe ever-increasing SVs.

###  6 Type Prediction

Table 10: List of the reviewed papers in the Type theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Sub-theme: 1. Common Weakness Enumeration (CWE)
Wang et al. 2010 [143] |  Multi-class classification: CWE classes |  NVD, CVSS |  Naïve Bayes
Shuai et al. 2013 [144] |  |  NVD |  SVM
Na et al. 2016 [145] |  |  NVD |  Naïve Bayes
Ruohonen et al. 2018 [146] |  |  NVD, CWE, Snyk |  tf-idf with 1/2/3-grams and cosine similarity
Huang et al. 2019 [147] |  |  NVD, CWE |  MLP, Linear SVM, Naïve Bayes, KNN
Aota et al. 2020 [148] |  |  NVD |  Random forest, Linear SVM, Logistic regression, Decision tree, Extremely randomized trees, LGBM
Aghaei et al. 2020 [149] |  |  NVD, CVE |  Adaptive fully-connected neural network with one hidden layer
Das et al. 2021 [150] |  |  NVD, CWE |  BERT, Deep Siamese network
Zou et al. 2019 [151] |  |  NVD & Software Assurance Reference Dataset (SARD) |  Three Bi-LSTM models for extracting and combining global and local features from code functions
Murtaza et al. 2016 [152] |  Unsupervised learning: sequence mining of SV types (over time) |  NVD (CWE & CPE) |  2/3/4/5-grams of CWEs
Lin et al. 2017 [153] |  Unsupervised learning: association rule mining of CWE-related aspects (prog. language, time of introduction & consequence scope) |  CWE |  FP-growth association rule mining algorithm
Han et al. 2018 [154] |  Binary/Multi-class classification: CWE relationships (CWE links, link types & CWE consequences) |  CWE |  Deep knowledge graph embedding of CWE entities
Sub-theme: 2. Custom Vulnerability Types
Venter et al. 2008 [155] |  Unsupervised learning: clustering |  CVE |  Self-organizing map
Neuhaus et al. 2010 [156] |  Unsupervised learning: topic modeling |  CVE |  Latent Dirichlet Allocation (LDA)
Mounika et al. [157, 158] |  |  CVE, Open Web Application Security Project (OWASP) |  LDA
Aljedaani et al. 2020 [159] |  |  SV reports (Chromium project) |  LDA
Williams et al. [160, 161] |  Multi-class classification: manually coded SV types |  NVD |  Supervised Topical Evolution Model & Diffusion-based storytelling technique
Russo et al. 2019 [162] |  NVD |  Bayesian network, J48 tree, Logistic regression, Naïve Bayes, Random forest
Yan et al. 2017 [60] |  |  Executables of 100 Linux applications |  Decision tree
Zhang et al. 2020 [134] |  Multi-class classification: platform-specific vulnerability types |  China National Vulnerability Database |  Logistic regression, Linear discriminant analysis, KNN, CART, SVM, bagging/boosting models

This section reports the work done in the Type theme. Type groups SVs with
similar characteristics, e.g., causes, attack patterns and impacts, and thus
facilitating the reuse of known prioritization and remediation strategies
employed for prior SVs of the same types. Two key prediction outputs are: (i)
Common Weakness Enumeration (CWE) and (ii) Custom vulnerability types (see
Table 10).

####  1 Summary of Primary Studies

##### Common Weakness Enumeration (CWE)

The first sub-theme determines and analyzes the patterns of the SV types
provided by CWE [28]. CWE is currently the standard for SV types with more
than 900 entries. The first group of studies has focused on multi-class
classification of these CWEs. Wang et al. [143] were the first to tackle this
problem with a Naïve Bayes model using the CVSS metrics (version 2) [110] and
product names. Later, Shuai et al. [144] used LDA [30] with a location-aware
weighting to extract important features from SV descriptions for building an
effective SVM-based CWE classifier. Na et al. [145] also showed that features
extracted from SV descriptions can improve the Naïve Bayes model in [143].
Ruohonen et al. [146] studied an information retrieval method, i.e., term-
frequency inverse document frequency (tf-idf) and cosine similarity, to detect
the CWE-ID with a description most similar to that of a given SV collected
from NVD and Snyk.141414https://snyk.io/vuln This method performed well for
CWEs without clear patterns/keywords in SV descriptions. Aota et al. [148]
utilized the Boruta feature selection algorithm [163] and Random forest to
improve the performance of base CWE classification. Base CWEs give more fine-
grained information for SV remediation than categorical CWEs used in [145].

There has been a recent rise in using neural network/DL based models for CWE
classification. Huang et al. [147] implemented a deep neural network with tf-
idf and information gain for the task and obtained better performance than
SVM, Naïve Bayes and KNN. Aghaei et al. [149] improved upon [148] for both
categorical (coarse-grained) and base (fine-grained) CWE classification with
an adaptive hierarchical neural network to determine sequences of less to more
fine-grained CWEs. To capture the hierarchical structure and rare classes of
CWEs, Das et al. [150] matched SV and CWE descriptions instead of predicting
CWEs directly. They presented a deep Siamese network with a BERT-based [80]
shared feature extractor that outperformed many baselines even for rare/unseen
CWE classes. Recently, Zou et al. [151] pioneered the multi-class
classification of CWE in vulnerable functions curated from Software Assurance
Reference Dataset (SARD) [164] and NVD. They achieved high performance
($\sim$95% F1-Score) with DL (Bi-LSTM) models. The strength of their model
came from combining global (semantically related statements) and local
(variables/statements affecting function calls) features. Note that this model
currently only works for functions in C/C++ and 40 selected classes of CWE.

Another group of studies has considered unsupervised learning methods to
extract CWE sequences, patterns and relationships. Sequences of SV types over
time were identified by Murtaza et al. [152] using an $n$-gram model. This
model sheds light on both co-occurring and upcoming CWEs (grams), raising
awareness of potential cascading attacks. Lin et al. [153] applied an
association rule mining algorithm, FP-growth [165], to extract the
rules/patterns of various CWEs aspects including types, programming language,
time of introduction and consequence scope. For example, buffer overflow (CWE
type) usually appears during the implementation phase (time of introduction)
in C/C++ (programming language) and affects the availability (consequence
scope). Lately, Han et al. [154] developed a deep knowledge graph embedding
technique to mine the relationships among CWE types, assisting in finding
relevant SV types with similar properties.

##### Custom Vulnerability Types

The second sub-theme is about custom vulnerability types other than CWE.
Venter et al. [155] used Self-organizing map [166], an unsupervised clustering
algorithm, to group SVs with similar descriptions on CVE. This was one of the
earliest studies that automated SV type classification. Topic modeling is
another popular unsupervised learning model [156, 157, 158, 159] to categorize
SVs without an existing taxonomy. Neuhaus et al. [156] applied LDA [30] on SV
descriptions to identify 28 prevalent SV types and then analyzed the trends of
such types over time. The identified SV topics/types had considerable overlaps
(up to 98% precision and 95% recall) with CWEs. Mounika et al. [157, 158]
extended [156] to map the LDA topics with the top-10 OWASP [167]. However, the
LDA topics/keywords did not agree well ($<$ 40%) with the OWASP descriptions,
probably because 10 topics did not cover all the underlying patterns of SV
descriptions. Aljedaani et al. [159] again used LDA to identify 10 types of
SVs reported in the bug tracking system of
Chromium151515https://bugs.chromium.org/p/chromium/issues/list and found
memory-related issues were the most prevalent topics.

Another group of studies has classified manually defined/selected SV types
rather than CWE as some SV types are encountered more often in practice and
require more attention. Williams et al. [160, 161] applied a supervised
topical evolution model [168] to identify the features that best described the
10 pre-defined SV types1616161\. Buffer errors, 2. Cross-site scripting, 3.
Path traversal, 4. Permissions and Privileges, 5. Input validation, 6. SQL
injection, 7. Information disclosure, 8. Resources Error, 9. Cryptographic
issues, 10. Code injection. prevalent in the wild. These authors then used a
diffusion-based storytelling technique [169] to show the evolution of a
particular topic of SVs over time; e.g., increasing API-related SVs requires
hardening the APIs used in a product. To support user-friendly SV assessment
using ever-increasing unstructured SV data, Russo et al. [162] used Bayesian
network to predict 10 pre-defined SV types.1717171\. Authentication bypass or
Improper Authorization, 2. Cross-site scripting or HTML injection, 3. Denial
of service, 4. Directory Traversal, 5. Local/Remote file include and Arbitrary
file upload, 6. Information disclosure and/or Arbitrary file read, 7.
Buffer/stack/heap/integer overflow, 8. Remote code execution, 9. SQL
injection, 10. Unspecified vulnerability Besides predicting manually defined
SV types using SV natural language descriptions, Yan et al. [60] used a
decision tree to predict 22 SV types prevalent in the executables of Linux
applications. The predicted type was then combined with fuzzers’ outputs to
predict SV exploitability (see section 1). Besides author-defined types,
custom SV types also come from specific SV platforms. Zhang et al. [134]
designed an ML-based framework to predict the SV types collected from China
National Vulnerability Database. Ensemble models (bagging and boosting models)
achieved, on average, the highest performance for this task.

####  2 Theme Discussion

In the Type theme, detecting and characterizing coarse-grained and fine-
grained CWE-based SV types are the frequent tasks. The large number and
hierarchical structure of classes are the main challenges with CWE
classification/analysis. In terms of solutions, deep Siamese networks [150]
are more robust to the class imbalance issue (due to many CWE classes), while
graph-based neural networks [154] can effectively capture the hierarchical
structure of CWEs. Future work can investigate the combination of these two
types of DL architectures to solve both issues simultaneously. Besides model-
level solutions, author-selected or platform-specific SV types have been
considered to reduce the complexity of CWE. However, similar to custom SV
consequences in section 1, there is not yet a universally accepted taxonomy
for these custom SV types. To reduce the subjectivity in selecting SV types
for prediction, we suggest that future work should focus on the types that are
commonly encountered and discussed by developers in the wild.

###  7 Miscellaneous Tasks

The last theme is Miscellaneous Tasks covering the studies that are
representative yet do not fit into the four previous themes. This theme has
three main sub-themes/tasks: (i) Vulnerability information retrieval, (ii)
Cross-source vulnerability patterns and (iii) Vulnerability fixing effort (see
Table 11).

Table 11: List of the reviewed papers in the Miscellaneous Tasks theme. Study |  Nature of task |  Data source |  Data-driven technique
---|---|---|---
Sub-theme: 1. Vulnerability Information Retrieval
Weeraward-hana et al. 2014 [170] |  Multi-class classification: Extraction of entities (software name/version, impact, attacker/user actions) from SV descriptions |  NVD (210 randomly selected and manually labeled SVs) |  Stanford Named Entity Recognizer implementing a CRF classifier
Dong et al. 2019 [171] |  Multi-class classification: Vulnerable software names/versions |  CVE Details, NVD, ExploitDB, SecurityFocus, SecurityFocus Forum, SecurityTracker, Openwall |  Word-level and character-level Bi-LSTM with attention mechanism
Gonzalez et al. 2019 [172] |  Multi-class classification: Extraction of 19 Vulnerability Description Ontology [173] classes from SV descriptions |  NVD |  Naïve Bayes, Decision tree, SVM, Random forest, Majority voting model
Binyamini et al. 2020 [174, 175] |  Multi-class classification: Extraction of entities (attack vector/means/technique, privilege, impact, vulnerable platform/version/OS, network protocol/port) from SV descriptions to generate MulVal [176] interaction rules |  NVD |  Bi-LSTM with various feature extractors: word2vec, ELMo, BERT (pre-trained or trained from scratch)
Guo et al. 2020 [177, 178] |  Multi-class classification: Extraction of entities (SV type, root cause, attack type, attack vector) from SV descriptions |  NVD, SecurityFocus |  CNN, Bi-LSTM (with or without attention mechanism)
Waareus et al. 2020 [179] |  Multi-class classification: Common Product Enumeration (CPE) |  NVD |  Word-level and character-level Bi-LSTM
Yitagesu et al. 2021 [180] |  Multi-class classification: Part-of-speech tagging of SV descriptions |  NVD, CVE, CWE, CAPEC, CPE, Twitter, PTB corpus [181] |  Bi-LSTM
Sun et al. 2021 [182] |  Multi-class classification: Extraction of entities (vulnerable product/version/component, type, attack type, root cause, attack vector, impact) from ExploitDB to generate SV descriptions |  NVD, ExploitDB |  BERT models
Sub-theme: 2. Cross-source Vulnerability Patterns
Horawalavith-ana et al. 2019 [183] |  Regression: Number of software development activities on GitHub after disclosure of SVs |  Twitter, Reddit, GitHub |  MLP, LSTM
Xiao et al. 2019 [184] |  Knowledge-graph reasoning: modeling the relationships among SVs, its types and attack patterns |  CVE, CWE, CAPEC (Linux project) |  Translation-based knowledge-graph embedding
Sub-theme: 3. Vulnerability Fixing Effort
Othmane et al. 2017 [185] |  Regression: time (days) to fix SVs |  Proprietary SV data collected at the SAP company |  Linear/Tree-based/Neural network regression

####  1 Summary of Primary Studies

##### Vulnerability Information Retrieval

The first and major sub-theme is vulnerability information retrieval that
studies data-driven methods to extract different SV-related entities (e.g.,
affected products/versions) and their relationships from SV data. The current
sub-theme extracts assessment information appearing explicitly in SV data
(e.g., SV descriptions on NVD) rather than predicting implicit properties as
done in prior sub-themes. For instance, CWE-119, i.e., “Improper Restriction
of Write Operations within the Bounds of a Memory Buffer”, can be retrieved
directly from
CVE-2020-28022,181818https://nvd.nist.gov/vuln/detail/CVE-2020-28022 but not
from CVE-2021-2122.191919https://nvd.nist.gov/vuln/detail/CVE-2021-21220 The
latter case requires techniques from section 1.

Most of the retrieval methods in this sub-theme have been formulated under the
multi-class classification setting. One of the earliest works was conducted by
Weerawardhana et al. [170]. This study extracted software names/versions,
impacts and attacker’s/user’s action from SV descriptions on NVD using
Stanford Named Entity Recognition (NER) technique, a.k.a. CRF classifier
[186]. Later, Dong et al. [171] proposed to use a word/character-level Bi-LSTM
to improve the performance of extracting vulnerable software names and
versions from SV descriptions available on NVD and other SV
databases/advisories (e.g., CVE Details [187], ExploitDB [67], SecurityFocus
[72], SecurityTracker [188] and Openwall [189]). Based on the extracted
entities, these authors also highlighted the inconsistencies in vulnerable
software names and versions across different SV sources. Besides version
products/names of SVs, Gonzalez et al. [172] used a majority vote of different
ML models (e.g., SVM and Random forest) to extract the 19 entities of
Vulnerability Description Ontology (VDO) [173] from SV descriptions to check
the consistency of these descriptions based on the guidelines of VDO. Since
2020, there has been a trend in using DL models (e.g., Bi-LSTM, CNNs or BERT
[80]/ELMo [190]) to extract different information from SV descriptions
including required elements for generating MulVal [176] attack rules [174,
175] or SV types/root cause, attack type/vector [177, 178], Common Product
Enumeration (CPE) [191] for standardizing names of vulnerable
vendors/products/versions [179], part-of-speech [180] and relevant entities
(e.g., vulnerable products, attack type, root cause) from ExploitDB to
generate SV descriptions [182]. BERT models [80], pre-trained on general text
(e.g., Wikipedia pages [82] or PTB corpus [181]) and fine-tuned on SV text,
have also been increasingly used to address the data scarcity/imbalance for
the retrieval tasks.

##### Cross-Source Vulnerability Patterns

The second sub-theme, cross-source vulnerability patterns, finds commonality
and/or discovers latent relationships among SV sources to enrich information
for SV assessment. Horawalavithana et al. [183] found a positive correlation
between development activities (e.g., push/pull requests and issues) on GitHub
and SV mentions on Reddit202020https://reddit.com and Twitter. These authors
then used DL models (i.e., MLP [119] and LSTM [138]) to predict the appearance
and sequence of development activities when SVs were mentioned on the two
social media platforms. Xiao et al. [184] applied a translation-based graph
embedding method to encode and predict the relationships among different SVs
and the respective attack patterns and types. This work [184] was based on the
DeepWeak model of Han et al. [154], but it still belongs to this sub-theme as
they provided a multi-dimensional view of SVs using three different sources
(NVD [16], CWE [28] and CAPEC [122]). Xiao et al. [184] envisioned that their
knowledge graph can be extended to incorporate the source code
introducing/fixing SVs.

##### Vulnerability Fixing Effort

The last sub-theme is vulnerability fixing effort that focuses on estimating
SV fixing effort through proxies such as the SV fixing time, usually in days.
Othmane and the co-authors were among the first to approach this problem.
These authors first conducted a large-scale qualitative study at the SAP
company and identified 65 important code-based, process-based and developer-
based factors contributing to the SV fixing effort [192]. Later, the same
group of authors [185] leveraged the identified factors in their prior
qualitative study to develop various regression models such as linear
regression, tree-based regression and neural network regression models, to
predict time-to-fix SVs using the data collected at SAP. These authors found
that code components containing detected SVs are more important for the
prediction than SV types.

####  2 Theme Discussion

In the Miscellaneous Tasks theme, the key focus is on retrieving SV-related
entities and characteristics from SV descriptions. The retrieval tasks are
usually formulated as Named Entity Recognition from SV descriptions. However,
we observed that NVD descriptions do not follow a consistent template [141,
142], posing significant challenges in labeling the entities for retrieval.
The affected versions and vendor/product names of SVs also contain
inconsistencies [171, 141, 142], making the retrieval tasks difficult. We
recommend that data normalization and cleaning should be performed before
labeling entities and building respective retrieval models to ensure the
reliability of results.

Besides information retrieval, other tasks such as linking multi-sources,
extracting cross-source patterns or estimating fixing effort are also useful
to obtain richer SV information for assessment, yet these tasks are still in
early stages. Linking multiple sources and their patterns is the first step
towards building an SV knowledge graph to answer different queries regarding a
particular SV (e.g., what systems are affected, exploitation status, how to
fix, or what SVs are similar). In the future, such a knowledge graph can be
extended to capture artifacts of SVs in emerging software types like AI-based
systems [193]. Moreover, to advance SV fixing effort prediction, future work
can consider adapting/customizing the existing practices/techniques used to
predict fixing effort for general bugs [194, 195].

###  8 Analysis of Data-Driven Approaches for Software Vulnerability
Assessment

Table 12: The frequent data sources, features, models, evaluation techniques and evaluation metrics used for the five identified SV assessment themes. Notes: The values are organized based on their overall frequency across the five themes. For the Prediction Model and Evaluation Metric elements, the values are first organized by categories (ML then DL for Prediction Model and classification then regression for Evaluation Metric) and then by frequencies. k-CV stands for k-fold cross-validation. The full list of values and their appearance frequencies for the five elements in the five themes can be found at https://figshare.com/s/da4d238ecdf9123dc0b8. Source/Technique/Metric |  Strengths |  Weaknesses
---|---|---
Element: Data Source
NVD/CVE/CVE Details (deprecated OSVDB) |  • Report expert-verified information (with CVE-ID) • Contain CWE and CVSS entries for each SV • Link to external sources (e.g., official fixes) |  • Missing/incomplete links to vulnerable code/fixes • Inconsistencies due to human errors • Delayed SV reporting and assignment of CVSS metrics
ExploitDB |  • Report PoC exploits of SVs (with links to CVE-ID) |  • May not lead to real exploits in the wild
Other security advisories (e.g., SecurityFocus, Symantec or X-Force) |  • Report real-world exploits of SVs • Cover diverse SVs (including ones w/o CVE-ID) |  • Some exploits may not have links to CVE entries for mapping with other assessment metrics
Informal sources (e.g., Twitter and darkweb) |  • Early reporting of SVs (maybe earlier than NVD) • Contain non-technical SV information (e.g., financial damage or socio-technical challenges in addressing SVs) |  • Contain non-verified and even misleading information • May cause adversarial attacks to assessment models
Element: Model Feature
BoW/tf-idf/n-grams |  • Simple to implement • Strong baseline for text-based inputs (e.g., SV descriptions in security databases/advisories) |  • May suffer from vocabulary explosion (e.g., many new description words for new SVs) • No consideration of word context/order (maybe needed for code-based SV analysis) • Cannot handle Out-of-Vocabulary (OoV) words (can resolve with subwords [23])
Word2vec |  • Capture nearby context of each word • Can reuse existing pre-trained model(s) |  • Cannot handle OoV words (can resolve with fastText [196]) • No consideration of word order
DL model end-to-end trainable features |  • Produce SV task-specific features |  • May not produce high-quality representation for tasks with limited data (e.g., real-world exploit prediction)
Bidirectional Encoder Repre- sentations from Transformers (BERT) |  • Capture contextual representation of text (i.e., the feature vector of a word is specific to each input) • Capture word order in an input • Can handle OoV words |  • May require GPU to speed up feature inference • May be too computationally expensive and require too much data to train a strong model from scratch • May require fine-tuning to work well for a source task
Source/expert-defined meta- data features |  • Lightweight • Human interpretable for a task of interest |  • Require SV expertise to define relevant features • Hard to generalize to new tasks
Element: Prediction Model
Single ML models (e.g., Linear SVM, Logistic regression, Naïve Bayes) |  • Simple to implement • Efficient to (re-)train on large data (e.g., using the entire NVD database) |  • May be prone to overfitting • Usually do not perform as well as ensemble/DL models
Ensemble ML models (e.g., Random forest, XGBoost, LGBM) |  • Strong baseline (usually stronger than single models) • Less prone to overfitting |  • Take longer to train than single models
Latent Dirichlet Allocation (LDA – topic modeling) |  • Require no labeled data for training • Can provide features for supervised learning models |  • Require SV expertise to manually label generated topics • May generate human non-interpretable topics
Deep Multi-Layer Perceptron (MLP) |  • Work readily with tabular data (e.g., manually defined features or BoW/tf-idf/n-grams) |  • Perform comparably yet are more costly compared to ensemble ML models • Less effective for unstructured data (e.g., SV descrip- tions)
Deep Convolutional Neural Networks (CNN) |  • Capture local and hierarchical patterns of inputs • Usually perform better than MLP for text-based data |  • Cannot effectively capture sequential order of inputs (maybe needed for code-based SV analysis)
Deep recurrent neural networks (e.g., LSTM or Bi-LSTM) |  • Capture short-/long-term dependencies from inputs • Usually perform better than MLP for text-based data |  • May suffer from the information bottleneck issue (can resolve with attention mechanism [117]) • Usually take longer to train than CNNs
Deep graph neural networks (e.g., Graph convolutional network) |  • Capture directed relationships among multiple SV entities and sources |  • Require graph-structured inputs to work • More computationally expensive than other DL models