#  _DeVAIC_ : A Tool for Security Assessment of AI-generated Code

Domenico Cotroneo<EMAIL_ADDRESS>Roberta De Luca<EMAIL_ADDRESS>Pietro Liguori<EMAIL_ADDRESS>
###### Abstract

Context: AI code generators are revolutionizing code writing and software
development, but their training on large datasets, including potentially
untrusted source code, raises security concerns. Furthermore, these generators
can produce incomplete code snippets that are challenging to evaluate using
current solutions.
Objective: This research work introduces _DeVAIC_ (Detection of
Vulnerabilities in AI-generated Code), a tool to evaluate the security of AI-
generated Python code, which overcomes the challenge of examining incomplete
code.
Method: We followed a methodological approach that involved gathering
vulnerable samples, extracting implementation patterns, and creating regular
expressions to develop the proposed tool. The implementation of _DeVAIC_
includes a set of detection rules based on regular expressions that cover $35$
Common Weakness Enumerations (CWEs) falling under the OWASP Top $10$
vulnerability categories.
Results: We utilized four popular AI models to generate Python code, which we
then used as a foundation to evaluate the effectiveness of our tool. _DeVAIC_
demonstrated a statistically significant difference in its ability to detect
security vulnerabilities compared to the state-of-the-art solutions, showing
an $F_{1}$ Score and Accuracy of $94\%$ while maintaining a low computational
cost of $0.14$ seconds per code snippet, on average.
Conclusions: The proposed tool provides a lightweight and efficient solution
for vulnerability detection even on incomplete code.

###### keywords:

Static code analysis , Vulnerability detection , AI-code generators , Python

††journal: Information and Software Technology

organization=University of Naples Federico II, city=Naples, postcode=80125,
country=Italy

##  1 Introduction

We live in an era where AI-code generators are revolutionizing the process of
writing code and software development. AI-powered solutions like GitHub
Copilot [20], OpenAI ChatGPT [57], Google Gemini [25], and Microsoft Copilot
[48] have shown their ability to translate into programming code what users
request with natural language (NL) descriptions (e.g., English language).

The effectiveness with which AI-code generators produce code has brought users
of different levels of skills and expertise to adopt such solutions to
promptly solve programming problems or to integrate AI-generated code into
software systems and applications. The other side of the coin is that their
widespread usage is out of any quality control, leading to a question to
preserve the security of the software development process: can we trust the
AI-generated code? Ergo, from a software security perspective, is the code
generated by AI secure and free of software vulnerabilities?

The concern stems from the consideration that these solutions are trained on a
large amount of publicly available data. For instance, GitHub Copilot utilizes
training data that consists of an extensive amount of source code, with
billions of lines collected from publicly accessible sources, including code
from GitHub’s public repositories [23]. Unfortunately, as often happens,
quantity does not coexist with quality. In fact, the huge amount of data used
to train the AI-code generators may include deprecated functions and
libraries, which may lead to the exploitation of vulnerabilities when adopted,
or it could intentionally have buggy or insecure code used to poison the code
generators during the training phase [31, 35, 30, 28].

This issue is not unexpected at all. It suffices to think that users are
always warned to exercise caution when using the AI-generated code (e.g., “The
users of Copilot are responsible for ensuring the security and quality of
their code” [22], “Use the code with caution” [4], “This code is generated by
artificial intelligence. Review and use carefully” [5]).

However, it is not clear what users can do to assess the security of the AI-
generated code to integrate it into their code base. Manual analysis, the go-
to method for security experts, becomes unfeasible due to the volume and rate
deployment of AI-generated code [41, 40]. In fact, the speed at which these
solutions operate can overwhelm even the most experienced security
professionals, making it difficult to thoroughly review each line of code for
potential vulnerabilities.

To perform code vulnerability detection, state-of-the-art solutions provide
several static analysis tools. Such tools usually parse the Abstract Syntax
Tree (AST) of the code [43, 65, 67, 16], a hierarchical representation of its
structure, and require complete programs to check whether the code contains
software vulnerabilities. However, since AI-code generators are fine-tuned on
corpora which often contain samples of code, i.e., code snippets [56, 55, 39,
6], they often do not produce complete programs [40, 88], making infeasible
the application of these tools in this context.

A different solution is represented by tools implementing pattern-matching
approaches to detect software vulnerabilities [45, 61, 85]. These tools
require users to set up ad-hoc configuration files to specify the
vulnerabilities they wish to detect via matching patterns [52, 53], hereby
requiring a non-trivial manual effort and limiting their adoption in practice.

Previous studies overcome the limitations of static analysis tools by adopting
AI-based solutions serving as vulnerability detectors. These fully automated
solutions do not require any human effort and can analyze incomplete programs,
but they may be susceptible to a high rate of false alarms [7, 8, 37, 38]. The
existing approaches using AI for vulnerability prediction face challenges
related to training data and model choices. Furthermore, it has been shown
that AI models struggle to understand the complexity of code structures and
identify security issues [15, 1, 10].

All the above limitations make evident the existence of a gap in the automatic
security assessment of the AI-generated code. Addressing these limitations is
crucial for enhancing the trustworthiness of AI-generated code and ensuring
the security of the systems built upon it.

This paper presents _DeVAIC_ (Detection of Vulnerabilities in AI-generated
Code), a tool that performs static analysis of Python code by implementing a
set of detection rules. More precisely, the tool uses a set of regular
expressions that cover $35$ Common Weakness Enumeration (CWE), a community-
developed list of software and hardware weakness types, to detect
vulnerabilities in Python code, one of the most used programming languages
[19, 46, 81]. The tool does not require the completeness of the code, making
it suitable to detect and classify vulnerabilities in AI-generated code
snippets, nor to create ad-hoc detection rules, hence overcoming the
limitations of state-of-the-art solutions.

We used _DeVAIC_ to detect vulnerabilities in the code generated by four well-
known public AI-code generators starting from NL prompts. Our experiments show
that the tool automatically identifies vulnerabilities with $F_{1}$ Score and
Accuracy both at $94\%$ and low computational times ($0.14$ seconds for code
snippet, on average). Also, we show that _DeVAIC_ exhibits performance
superior to the state-of-the-art solutions, i.e. CodeQL [16], Bandit [65], PyT
[67] and Semgrep [72], that are widely used to perform vulnerability detection
of the code [62, 13, 2, 33].

In the following, Section 2 discusses related work; Section 3 introduces a
motivating example; Section 4 presents _DeVAIC_ ; Section 5 describes the
results; Section 6 discusses the threats to validity; Section 7 concludes the
paper.

##  2 Related Work

Static analysis is one of the most commonly used techniques to detect
vulnerabilities in the code [11, 36, 60, 26].

The state of the art provides several static analysis tools for checking
security issues in the code, e.g., Python-specific tools like Bandit [65] and
PyT [67], or multilanguage ones such as Semgrep [72] and CodeQL [16], which
are widely used to detect vulnerabilities within the code [62, 13, 2, 33, 73,
32, 64, 42, 3, 24, 9].

Except for Semgrep, these tools require working on complete code due to the
preliminary modeling of AST from the code under examination. The vulnerability
detection is made by running appropriate plugins (for Bandit and PyT) or
queries (for CodeQL) against the AST nodes. For code that consists of snippets
rather than complete programs, these analysis tools cannot define the AST,
thus having any possibility for conducting their detection analyses. Semgrep
[72] is a static analysis tool that uses a pattern-matching approach and that
does not require the AST modeling of code before running the detection rules.
To detect vulnerabilities, users need to configure and customize the tool by
writing regex patterns in a configuration file. However, the limitation is
that we cannot assume in advance that all users can write accurate regex
patterns. Moreover, this approach could polarize vulnerability hunting,
focusing on those the user believes to find, potentially overlooking others
that are effectively present [52, 53]. For this reason, as often happens, this
type of solution offers a set of rules publicly available for the scanning of
the code under examination [78].

To overcome the limitations of static analysis tools, previous work
investigated the use of AI to perform vulnerability detection, using it as a
static analyzer. One of the benefits of their adoption is that they can
analyze incomplete code. However, previous work shows that the outcomes of
their detection exhibit numerous false assessments.

For instance, Chen et al. [8] released a new vulnerable source code dataset to
assess state-of-the-art deep learning methods in detecting vulnerabilities.
The authors show the limited performance of these solutions in terms of high
false positive rates, low $F_{1}$ Scores, and difficulty in detecting hard
CWEs. Fang et al. [15] crafted a dataset that pairs real-world code with
obfuscated versions, which they used as input for large language models (LLMs)
to evaluate their ability to analyze input code samples and test if LLMs can
be employed for defensive analysis tasks. The study found that larger models
are able to comprehend and explain unobfuscated code, whereas smaller models
failed in this task. However, the model’s understanding of the code was
limited when working with obfuscated code. Al-Hawawreh et al. [1] and Cheshkov
et al. [10] evaluated the performance of the ChatGPT for detecting
vulnerabilities in code, finding that ChatGPT’s results still needed careful
monitoring. Khoury et al. [34] explored how safe is the code generated by
ChatGPT, using this model for both code generation and assessment. Their
analysis outlined that, despite ChatGPT’s awareness of potential
vulnerabilities, it still can produce unsafe code. Instead of using AI-based
solutions, Sandoval et al. [76] [75] assessed the security of AI-generated
code by performing a manual code analysis. Although manual code checking has
advantages, its susceptibility to human error, scalability problems and
subjectivity emphasises the importance of using automated tools for a more
consistent code analysis.

This work presents a tool to perform vulnerability detection of Python code.
The tool does not require complete programs to analyze as it also works for
code snippets, making it suitable for assessing the security of AI-generated
code. Similar to Semgrep, the proposed tool implements a pattern-matching
approach while employing a lesser quantity of rules and achieving better
results in terms of Recall, $F_{1}$ Score, and Accuracy. Indeed, it does not
require any human effort to configure detection rules, hence overcoming the
limitations of previous static analysis tools.

The tool can identify vulnerabilities belonging to the OWASP categories with
the discovery of $35$ CWEs, a coverage level comparable with other research
work. For instance, Hajipour et al. [27] consider $13$ CWEs for the assessment
of the security risks posed by code language models through an approximation
of the inversion of the black-box models using few-shot prompting; Pearce et
al. [62], moreover, refer to the MITRE’s Top $25$ CWE list to assess the code
generated by Copilot. CodeQL provides queries to assess the security of Python
code with a coverage of $44$ CWEs [21]. Finally, the tool is able to detect
vulnerabilities in the AI-generated Python code with a low rate of false
positives and negatives, overcoming the performance of other state-of-the-art
tools used as static code analyzers, and low computational cost.

##  3 Motivating Example

AI models for code generation have shown significant advancements, largely
attributed to their intuitive, user-friendly interfaces. These models enable
users to request code generation through NL descriptions, specifying both the
programming language and the level of detail needed for the task at hand. For
illustrative purposes, consider the following example:

Listing 1: Example of NL prompt.

⬇

Write a Python code that defines a YAML payload from a request,

loads this payload into a dictionary called prod_and_qt

by using a function for loading.

Typically, AI models can produce either complete or partial snippets of code
based on the nature of the training data they have been exposed to. An example
output for the aforementioned prompt might look like this:

Listing 2: Example of code generated by AI models.

⬇

def load_yaml(yaml_payload):

prod_and_qt = yaml.load(yaml_payload)

return prod_and_qt

yaml_payload = ”””

product:

␣name:␣iPhone

␣quantity:␣10

”””

prod_and_qt = load_yaml(yaml_payload)

print(prod_and_qt)

The code generated by the model is what is requested in the NL prompt.
However, the code is vulnerable due to the yaml.load() function used to
process YAML data from yaml_payload, which may contain untrusted content.
Indeed, if manipulated by a malicious user, this payload may include dangerous
data or code. The vulnerability is in the execution of the yaml.load()
function, because it interprets the YAML payload as Python code, enabling the
possible execution of malicious instructions if the code lacks a proper
validation, like in this case. The CWE associated with this vulnerability is
CWE-502, commonly known as Deserialization of Untrusted Data, and related to
the Software and Data Integrity Failures category of OWASP’s Top 10. A simple
way to address this issue is reported by the official PyYAML documentation,
which recommends using yaml.safe_load() to read YAML from unreliable and
untrusted sources [68].

In most cases, users are unaware of software vulnerabilities and may not be
able to manually verify the security of the code, thereby including the
produced outcome into an existing codebase. This issue is further exacerbated
by the fact that state-of-the-art static code analyzers, such as CodeQL,
Bandit, PyT, etc., do not generate the report for this specific code snippet.
In fact, the code reported above is incomplete due to the lack of the import
statement at the beginning of it, making these tools unable to perform the
vulnerability detection, as explained in Section 2. This code characteristic
does not consent to the modeling of the AST as it lacks the necessary
dependency (i.e., import yaml) for the invoked API (i.e., the yaml.load()
function). Conversely, Semgrep’s textual analysis approach did examine the
code but resulted in a False Negative (FN), highlighting the challenge of
ensuring the security of AI-generated code.

These issues underscore the imperative to critically evaluate AI-generated
code for security vulnerabilities, thereby ensuring the safe integration of
such code into larger systems.

##  4 _DeVAIC_ Workflow

To overcome the issues described in Section 3, we present _DeVAIC_ , a tool to
detect vulnerabilities in Python code. The tool does not require the
completeness of the code, making it suitable for AI-generated code.

The _DeVAIC_ tool works as a text scanner and does not require the AST
modeling for the code analysis, employing regular expressions to identify
vulnerable code patterns. However, crafting effective regex requires a deep
understanding of the implementation patterns we want to find.

Figure 1: The _DeVAIC_ workflow.

To this aim, we extensively reviewed the literature to collect datasets of
unsafe Python code with the associated information of the implemented
CWE111The Common Weakness Enumeration (CWE) is a list of common software and
hardware weakness types published annually by the Massachusetts Institute of
Technology Research and Engineering® (MITRE) Corporation., extracting only the
snippets that implement the most recent and critical weaknesses identified in
OWASP’s Top $10$ Vulnerabilities222The Open Web Application Security Project®
(OWASP) draws up the Top $10$ Application Security List every four years to
define the $10$ most prevalent web application vulnerabilities. report. Since
OWASP categories encompass various CWEs that share the same vulnerability
typology, we employ these categories to cluster snippets together for further
detailed analysis (see § 4.1).

Using a named entity tagger, we standardize snippets by replacing variable
names, and input and output parameters of functions to reduce the variability
of the code. Then, we compare snippet pairs within each OWASP category based
on their similarity level, facilitating automated identification of common
implementation patterns using the Longest Common Subsequence (LCS), i.e., the
longest sequence that appears as a (not necessarily contiguous) subsequence in
both snippets. By using the identified patterns, we infer regex-based
detection rules able to identify vulnerabilities with similar patterns and
assess their corresponding vulnerability types (see § 4.2).

Finally, we collect all the detection rules in a bash script that takes an
input file containing line-by-line code to analyze. After scanning the file,
the tool generates a report of the detection, including the number of snippets
identified as vulnerable and their category according to the OWASP (see §
4.3).

Figure 1 shows a detailed flowchart outlining the steps through which we
developed the tool. In the rest of this Section, we detail the steps of the
workflow.

###  4.1 Vulnerable Code Collection

In our pursuit to collect vulnerable snippets, we performed a comprehensive
study of the literature. State-of-the-art provides numerous corpora containing
vulnerable code, some aimed at poisoning models to induce them to generate
vulnerabilities, while others serve to evaluate AI models as vulnerability
detectors [12, 8].

Since we target Python code, we selected two corpora suitable to our purpose,
discarding those containing snippets written in different programming
languages (e.g., DiverseVuln [8], Big-Vul [86, 14], SARD [54]):

  1. 1.

SecurityEval [77]: It is a dataset built to evaluate the security of code
generation models and to compare the performance of some state-of-the-art
static analysis tools, such as Bandit and CodeQL [79]. The dataset contains
$130$ Python code samples covering $75$ vulnerability types, which are mapped
to prompts written in Natural Language (NL).

  2. 2.

Copilot CWE Scenarios Dataset [63]: It encompasses the source code utilized to
craft NL prompts for generating code with Large Language Models (LLMs) [80,
83]. It covers code associated with $89$ distinct vulnerable scenarios.

Table 1: List of $35$ selected CWEs. In blue we indicate the CWEs belonging to at least one of the MITRE’s top $25$ of the last three years. We use “-” to indicate the absence of a specific CWE in the relative top $40$. OWASP 2021 |  CWE |  Rank MITRE 2021 |  Rank MITRE 2022 |  Rank MITRE 2023
---|---|---|---|---
Broken Access Control |  CWE-022 |  8 |  8 |  8
|  CWE-377 |  - |  - |  -
|  CWE-425 |  - |  - |  -
|  CWE-601 |  37 |  35 |  32
Cryptographic Failures |  CWE-319 |  35 |  39 |  -
|  CWE-321 |  - |  - |  -
|  CWE-326 |  - |  - |  -
|  CWE-327 |  - |  - |  -
|  CWE-329 |  - |  - |  -
|  CWE-330 |  - |  - |  -
|  CWE-347 |  - |  - |  -
|  CWE-759 |  - |  - |  -
|  CWE-760 |  - |  - |  -
Identification and Authentication Failures |  CWE-295 |  26 |  26 |  34
|  CWE-384 |  - |  - |  -
Injection |  CWE-020 |  4 |  4 |  6
|  CWE-078 |  5 |  6 |  5
|  CWE-079 |  2 |  2 |  2
|  CWE-080 |  - |  - |  -
|  CWE-090 |  - |  - |  -
|  CWE-094 |  28 |  25 |  23
|  CWE-095 |  - |  - |  -
|  CWE-096 |  - |  - |  -
|  CWE-099 |  - |  - |  -
|  CWE-113 |  - |  - |  -
|  CWE-116 |  - |  - |  -
|  CWE-643 |  - |  - |  -
|  CWE-1236 |  - |  - |  -
Insecure Design |  CWE-209 |  - |  - |  -
|  CWE-269 |  29 |  29 |  22
|  CWE-434 |  10 |  10 |  10
Security Logging and Monitoring Failures (SLMF) |  CWE-117 |  - |  - |  -
Security Misconfiguration |  CWE-611 |  23 |  24 |  28
Server-Side Request Forgery (SSRF) |  CWE-918 |  24 |  21 |  19
Software and Data Integrity Failures (SDIF) |  CWE-502 |  13 |  12 |  15

To develop detection rules that address the most recent and critical
weaknesses, we extracted from these corpora only the vulnerable code snippets
that implement the CWEs present in the most recent OWASP Top $10$
Vulnerabilities report from 2021 [58, 29]. Overall, we selected $240$
vulnerable snippets that implement $35$ CWEs related to $9$ OWASP
categories333MITRE itself is the authority responsible for establishing the
correlation between CWEs and the OWASP Top $10$. Among the selected CWEs, $10$
of them can be found in at least one of the MITRE’s Top $25$ lists from the
past three years [49, 50, 51]. The use of $35$ CWEs is consistent with other
detection setups used in previous research studies [27, 62]. Table 1 shows the
selected CWEs with their OWASP category.

###  4.2 Detection Rules Implementation

To define detection rules, we search for common patterns among the selected
code snippets based on the assumption that similar vulnerabilities manifest in
similar ways [82, 29, 45]. Pattern-matching approaches such as LCS, N-gram and
others, have been widely used to identify the same family malware [45, 85, 61,
87]. In our work, we applied the LCS algorithm to identify vulnerable patterns
among snippets belonging to the same OWASP category.

As an example, consider the pair of code snippets shown in Table 2 \-
“Original Snippet” column, which belongs to the Injection OWASP category
(i.e., user-supplied data is not validated, filtered, or sanitized by the
application). Specifically, the code snippet in Row #1 addresses CWE-020
(Improper Input Validation), whereas the snippet in Row #2 is linked to
CWE-080 (Basic XSS). Despite implementing distinct CWEs, these samples share
analogous implementation patterns, i.e., the use of an output value from a
function of a Python module without any sanitization, subsequently used as an
input parameter in a different function.

Table 2: Examples of standardization for code snippets. The common longest subsequences of original snippets and the standardized snippets are in blue. |  Original Snippet |  Standardized Snippet
---|---|---
S1 |  def temperature_for_location(): file_yaml = request.args.get("file", None) json_payload = yaml.load(file_yaml) |  def temperature_for_location(): var0 = request.args.get(var1, var2) var3 = yaml.load(var0)
S2 |  name = request.args.get(’name’, ’’) response = make_response(name) return response |  var0 = request.args.get(var1, var2) var3 = make_response(var0) return var3

Therefore, to search for similar code snippets, we first check the similarity
for each pair of snippets grouped in the same OWASP category. To this aim, we
use SequenceMatcher, a class of the Python module difflib, that computes the
Longest Common Subsequence (LCS) with no junk elements, i.e., tokens that are
not considered in the matching (e.g., the newline character $\backslash n$),
among different code snippets. First, we compute the similarity ratio between
two code snippets, ranging from $0$ (total mismatching) to $1$ (perfect
matching). Based on empirical observations, we focus exclusively on pairs
exhibiting a similarity ratio exceeding $50\%$ [69, 70, 71]. This decision is
grounded in the assumption that when two snippets share at least half of their
content, we can infer the existence of a meaningful common pattern. Then, we
find common patterns among snippets that meet the similarity threshold within
each OWASP category by computing the LCS [44, 45].

To support the LCS in finding common patterns, we standardize all the snippets
for each OWASP category, i.e., we reduce the randomness of the code snippets,
by using a named entity tagger, which returns a dictionary of standardizable
tokens for the input and output parameters of functions, extracted through
regular expressions. We replace the selected tokens in every intent with
“var#”, where # denotes a number from $0$ to $|l|$, and $|l|$ is the number of
tokens to standardize. This data processing method prevents snippets that have
a low similarity score due to parameters (e.g., parameter names containing a
high number of tokens) from not being grouped to find common patterns.

Table 2 \- “Standardized Snippet” column shows the standardization process
performed on the snippets, where the parameters of the original code snippets
are replaced (e.g., the input parameter “file” is replaced with var1 in the
code snippet of row $\\#1$). Due to the standardization, the similarity ratio
increases from $\sim 37\%$ (obtained on the original snippets) to $\sim 63\%$.
As a result, the LCS returns a clearer and easier-to-identify pattern, as
shown in the table.

To identify patterns, we create detection rules by using regular expressions
(regex) that, by their nature, operate on patterns within the text (code). As
they do not require the completeness of the code to identify specific
patterns, they are well-suited for analyzing incomplete or partial programs,
which is often the case of AI-generated code. For example, a detection rule
based on LCS result of the example in Table 2 can involve detecting the input
function request.args.get(), whose output (i.e., var0) lacks proper
sanitization and is subsequently utilized as an input parameter for another
sink function (i.e., passing the variable var0 as a parameter in brackets,
regardless of the specific function in which it is passed). Since rules are
created by patterns found in the same OWASP category, each rule is associated
with a category. Therefore, when the tool analyzes a code snippet containing
the previous pattern, the rule detects the vulnerability and indicates the
related OWASP category.

In the implementation of detection rules, we adopt domain-specific language
designed for text processing such as awk, grep, and sed, which are standard
features of most Unix-like operating systems. For example, to implement a rule
to detect the pattern of the previous example, we use awk to store the name of
the variable that contains the output of the request.args.get() function.
Then, we employ grep with a specific regular expression that checks if this
variable is used in subsequent code without any sanitization.

Overall, we implement $85$ detection rules that cover the $35$ CWEs over the
$9$ OWASP categories shown in Table 1. Since there is no unique way to
implement a CWE, the number of detection rules overcomes the number of CWEs.

###  4.3 Tool Execution

As _DeVAIC_ uses standard features of most Unix-like operating systems, it is
highly portable. It takes as input a file in TXT format containing a set of
code snippets, each written on a single line. Multi-line code snippets (e.g.,
a function) are separated by the newline character $\backslash n$.

While scanning the entire set of snippets in the input file, _DeVAIC_ executes
all the detection rules to detect any vulnerabilities. If the tool identifies
a vulnerability through a rule, the tool continues the execution of all the
detection rules since the same snippet may implement different
vulnerabilities, even belonging to different OWASP categories. At the end of
the execution, the tool returns in output a file containing a summary report
of the detection results. The report includes:

  * 1.

The number of code snippets analyzed by the tool;

  * 2.

The number/percentage of code snippets identified as safe, i.e., that do not
contain any vulnerability, and the number/percentage of code snippets
identified as unsafe along with the classification of the vulnerabilities
according to OWASP Top $10$;

  * 3.

The overall execution time on the entire dataset of snippets and the average
execution time per single snippet.

The tool also produces a file that exhibits the detection results,
highlighting the OWASP categories for each vulnerable snippet. Since a single
code may encompass multiple vulnerabilities from different categories, the
occurrences of snippets that fall in the various OWASP categories may exceed
the total number of unsafe code snippets.

##  5 Experimental Evaluation

We evaluate the effectiveness of our tool, _DeVAIC_ , through experiments
conducted on code generated by four distinct publicly available AI models,
i.e., Google Gemini (LaMDA’s successor), Microsoft Copilot (GPT-4), OpenAI
ChatGPT (GPT-3.5), and GitHub Copilot (GPT-4). These models are accessible via
APIs and generate code suggestions starting from NL prompts.

###  5.1 NL Prompts Details

We used a set of $125$ NL prompts for each of the mentioned models to generate
Python code, for a total of $500$ code snippets used to assess _DeVAIC_ ’s
detection skills.

We designed these prompts starting from those provided by three publicly
available datasets about code generation, i.e. SecurityEval [74], LLMSecEval
[84] and CodeXGLUE [47]. First of all, we modified these prompts by avoiding
the insertion of implementation details. This line of action allows the model
to generate code according to their discretion without any influence on how to
implement a task. Moreover, we adopted the perspective of an ordinary,
nonmalicious user who requested the model to generate code for a given task,
which means that our prompts do not contain explicit triggers to induce the
prediction of unsafe code.

The number of tokens for these $125$ prompts has an average value of $15.5$
(median value is $14$). Typically, most prompts fall within the range of $11$
to $19$ tokens, with a few outliers presenting a minimum of $7$ and a maximum
of $34$ tokens. The variation in prompt length is attributed to the need for
additional details to clarify the request, although it is important to remark
that prompts do not contain any implementation details.

Table 3: Examples of vulnerable code produced by models. The vulnerable part of the code is highlighted in red. NL prompt |  Unsafe prediction
---|---
“Write a Python code with a get_name() function that gets the name parameter from the URL by using request module by importing flask and opens the file names/name returning a file object” |  def get_name(): $\backslash$n request = flask.request $\backslash$n name = request.args.get("name") $\backslash$n if name is None: $\backslash$n return None $\backslash$n file_object = open("names/" \+ name, "r") $\backslash$n return file_object
“Get the user input from the request and eval it” |  user_input = input("Please enter your input: ")$\backslash$n evaluated_input = eval(user_input) $\backslash$n print(evaluated_input)

Table 3 displays two examples of NL prompts and a sample of vulnerable code
generated by the models we employed. Row #1 showcases a model’s prediction
that uses the request.args.get() function of the Flask module. The only
validation produced in the code (i.e., the only one defined by the user in the
prompt) is to verify if the output parameter of the request.args.get()
function is an empty variable. Nevertheless, it is not sufficient because,
even if this parameter is not empty, it may contain dangerous characters
(i.e., “../”, “../..”, etc.) that can allow a malicious user to access files
or directories outside of the intended directory (i.e., name/). The CWE
associated with this kind of vulnerability is CWE-022, commonly known as “Path
Traversal”. This CWE is related to the Broken Access Control category of
OWASP’s Top 10. To enhance the security, the code has to ensure that the file
path is actually within the expected directory before opening the file itself
[59]. We also notice that the generated code snippet lacks completeness,
making the application of some static analysis tools not feasible.

Finally, Row #2 shows that the request to evaluate a user input involved the
generation of code containing the eval() function. This function is considered
dangerous due to the potential injection of malicious code and is related to
CWE-095, also known as “Eval Injection”, which belongs to the Injection OWASP
category. In fact, the official ast documentation recommends using
literal_eval(), a more secure alternative to eval(), that can evaluate only a
restricted set of expressions [66].

###  5.2 Manual Analysis

After submitting the NL prompts, the models generated a total of $500$ code
snippets. The average number of tokens for the code is $54$ (with a median of
$42$, a maximum of $205$, and a minimum of $4$), with most of the code
snippets ($254$ in total, $\sim 51\%$) falling within the range of $22$ to
$88$ tokens. Furthermore, as shown in Figure 2, every model produced some
instances of incomplete code, i.e., the model provided code functions without
the necessary import statement at the beginning. For each group of $125$
snippets produced by each model, we reported $8$ incomplete code snippets for
Google Gemini ($\sim 6\%$), $12$ for Microsoft Copilot ($\sim 10\%$), $39$ for
GitHub Copilot ($\sim 31\%$) and $6$ for OpenAI ChatGPT ($\sim 5\%$), reaching
a total of $13\%$ of incomplete code on $500$ code generated. This result
underlines the importance of evaluating AI-generated code with a tool able to
overcome this issue.

Figure 2: Occurrences of complete and incomplete code generated by each model.

After grouping the code for each model and converting them in a TXT file with
snippets written line by line, we ran _DeVAIC_ to check for vulnerable
implementation patterns in the outcomes. The assessment of the detection
results needs manual inspection, involving the evaluation of each code
procured by models in terms of True Positives (TPs), False Positives (FPs),
True Negatives (TNs), and False Negatives (FNs). More precisely, we have a TP
case when both _DeVAIC_ and manual analysis detect a software vulnerability;
similarly, when both _DeVAIC_ and manual analysis do not detect any
vulnerability in the code, then we have a TN case. When _DeVAIC_ identifies a
vulnerability not confirmed by manual inspection, then it is an FP case.
Conversely, when the tool does not identify a vulnerability within the code
but manual analysis does, then it is an FN case.

As manual classification can be susceptible to errors, three researchers of
our research group, who have skills in Python code vulnerability detection,
conducted this evaluation independently. Subsequently, they compared their
evaluation and performed an in-depth analysis of the few discrepancy cases.
This collaborative effort resulted in a complete alignment, achieving a
$100\%$ consensus in the final evaluation.

On average, the four models produced $54\%$ of vulnerable code ($271$
vulnerable code over $500$ predictions), which allows us to comprehensively
assess the tool’s detection ability in both vulnerable and non-vulnerable
instances, providing a robust and fair measure of their overall performance
across various scenarios.

###  5.3 Experimental Results

We assessed the tool’s detection ability by using standard metrics typically
employed in this field, i.e., Precision, Recall, $F_{1}$ Score, and Accuracy.
We computed these metrics with the TPs, TNs, FPs, and FNs identified during
the manual analysis (see § 5.2).

Table 4: Evaluation of detection results comparing _DeVAIC_ with the state of the art. Detection Tool |  Google Gemini |  OpenAI ChatGPT-3.5 |  Microsoft Copilot |  GitHub Copilot |  All models
---|---|---|---|---|---
Precision |  _DeVAIC_ |  0.97  |  1.00  |  0.95  |  0.95  |  0.97
|  Bandit |  0.89  |  0.81  |  0.83  |  0.82  |  0.84
|  CodeQL |  0.79  |  0.86  |  0.83  |  0.95  |  0.85
|  Semgrep |  0.87  |  0.98  |  0.94  |  0.85  |  0.91
|  PyT |  1.00  |  0.89  |  1.00  |  1.00  |  0.96
Recall |  _DeVAIC_ |  0.95  |  0.96  |  0.90  |  0.86  |  0.92
|  Bandit |  0.70  |  0.69  |  0.51  |  0.58  |  0.62
|  CodeQL |  0.36  |  0.54  |  0.42  |  0.26  |  0.39
|  Semgrep |  0.55  |  0.69  |  0.58  |  0.51  |  0.58
|  PyT |  0.11  |  0.11  |  0.08  |  0.04  |  0.09
$F_{1}$ Score |  _DeVAIC_ |  0.96  |  0.98  |  0.92  |  0.90  |  0.94
|  Bandit |  0.78  |  0.74  |  0.63  |  0.68  |  0.72
|  CodeQL |  0.49  |  0.67  |  0.56  |  0.41  |  0.54
|  Semgrep |  0.67  |  0.81  |  0.72  |  0.64  |  0.71
|  PyT |  0.20  |  0.20  |  0.16  |  0.08  |  0.16
Accuracy |  _DeVAIC_ |  0.95  |  0.98  |  0.93  |  0.89  |  0.94
|  Bandit |  0.77  |  0.74  |  0.71  |  0.69  |  0.73
|  CodeQL |  0.56  |  0.70  |  0.68  |  0.58  |  0.63
|  Semgrep |  0.69  |  0.82  |  0.78  |  0.67  |  0.74
|  PyT |  0.48  |  0.50  |  0.56  |  0.46  |  0.50

To provide context for the evaluation, we compared _DeVAIC_ ’s performance
with a baseline. In our analysis, we utilized CodeQL version v2.16.4 and the
two Security test suites [17, 18] of queries for Python. We also used Bandit
version 1.7.7, Semgrep version 1.61.1, and python-taint module version 0.42,
also known as PyT. We remark again that several snippets generated by the
models were incomplete, preventing us from directly analyzing them with some
well-known static analysis tools, such as Bandit, CodeQL, or others. At first,
we analyzed the model predictions with _DeVAIC_. Then, we modified the
incomplete ones ($65$ in total) by inserting the import statement when lacking
to compare _DeVAIC_ ’s evaluation results with the state of the art.

Considering the average values for all models, Table 4 shows that _DeVAIC_
achieved high values for each metric. Regarding Precision, _DeVAIC_ reaches
$97\%$, which is comparable with the values achieved by Semgrep444When we used
Semgrep, we executed a total of 1291 rules from the official ruleset registry
for Python [78]. ($91\%$) and PyT ($96\%$). Furthermore, _DeVAIC_ achieved a
maximum Recall of $92\%$ on average, surpassing the baseline with a
considerable gap compared to the employed solutions, which obtain a maximum of
$62\%$ (i.e., Bandit). Finally, _DeVAIC_ shows superior performance to other
state-of-the-art solutions ($\geq 20\%$) with average values of $94\%$ for
both $F_{1}$ Score and Accuracy.

We further evaluated if, for each metric (i.e., Precision, Recall, F1 Score,
and Accuracy), there is a statistical difference between _DeVAIC_ and the
state-of-the-art solutions using the non-parametric Wilcoxon rank sum test.
The null hypothesis is that the two samples derive from the same population
(i.e., the two populations have equal medians). If the null hypothesis is
rejected, the Wilcoxon rank sum test indicates that the two samples are
statistically different. As deducible by the considerations explained above,
for the Precision, _DeVAIC_ is statistically different only with Bandit while
obtaining optimum results for all of the other metrics, for which _DeVAIC_
results are statistically different (and better) with every other result
obtained with the state of the art. The previously mentioned considerations
highlight the outstanding performance of _DeVAIC_.

Figure 3: Analysis of snippet lengths variability for each classification. The
median values are in orange, while the mean values are in green.

Then, we performed an in-depth analysis of the results provided by our tool to
investigate the cases of FN and FP. Since _DeVAIC_ behaves like a text
scanner, we first checked whether the performance of the tool is affected by
the complexity, in terms of number of tokens, of the code to analyze. Figure 3
shows four boxplots to illustrate the complexity of the code across the TP,
TN, FP, and FN cases. The figure highlights that the interquartile ranges,
i.e., the height of the boxplots, are greater for TP and TN cases ($59$ and
$57$, respectively) than the FN ($46$) and FP cases ($29$). Moreover, the
average number of tokens per category, which is $46$ for TP, $40$ for TN, $44$
for FP, and $43$ for FN, proves that the cases of misclassification do not
depend on the complexity of snippets as _DeVAIC_ is effective in detecting
vulnerabilities even for complex code snippets.

###  5.4 Computational Cost

We analyzed the computational times of _DeVAIC_. We run the tool on a computer
with a 13th Gen Intel(R) Core(TM) i9-13900H CPU, and $32$ GB of RAM.

Figure 4: Comparison of median execution times for _DeVAIC_ , Bandit, CodeQL,
Semgrep, and PyT for the analysis of all the $500$ codes generated by the $4$
models.

We compared the execution time taken by _DeVAIC_ with the other solutions
employed. Figure 4 shows the median execution time of each tool in analyzing
$125$ code snippets from the four models. Bandit is the faster ($0.67$
seconds), while CodeQL is the slower ($570$ seconds, which corresponds to
approximately $9$ minutes). PyT and Semgrep employed $253.40$ and $123.67$
seconds, respectively ($2$ and $4$ minutes approximately). In this context,
while the other tools employed minutes on average to analyze the snippets,
_DeVAIC_ remains in the realm of seconds, as Bandit does.

Furthermore, we deeply inspected the distribution of the times required by
_DeVAIC_ to execute the detection rules on every code snippet. Despite the
different lengths of the analyzed code snippets, Figure 5 shows that the tool
exhibited an average execution time of $0.16$ seconds (with a median of
$0.14$, a maximum of $0.59$, and a minimum of $0.10$). The figure highlights
that for most of the snippets ($169$ in total, $\sim 34\%$), the execution
time falls within the range of $0.10$ to $0.13$ seconds.

Figure 5: Occurrences of execution times taken by _DeVAIC_ for scanning and
evaluating individual code snippets generated by the 4 models.

Moreover, we did not find any relation between the length of snippets, in
terms of tokens, to analyze and the computation times of the tool. Indeed, the
outliers in execution times were due to cases of multiple identified OWASP
categories, which require the tool to perform multiple write operations to the
report file.

Figure 6: Execution times of _DeVAIC_ plotted against the cumulative number of
tokens in code snippets. Each data point represents the time taken as snippets
are progressively combined, starting from single snippets and incrementally
merging with additional ones.

Finally, we conducted a study to evaluate the performance of our tool when
analyzing large programs. To increase the code complexity, we concatenated the
snippets generated by Gemini (we could have chosen the code generated by any
other AI model as well) to create a new set of code characterized by $125$
snippets with an incremental number of tokens, starting from $73$ for the
first snippet to $6892$ for the last one.

Figure 6 shows the _DeVAIC_ ’s performance in evaluating the new code
collection. Analyzing snippets with more tokens led to longer execution times.
While the initial snippet, comprising $73$ tokens, only required $0.12$
seconds for evaluation, the final snippet, consisting of $6892$ tokens, took
over $1$ minute ($75.23$ seconds) to process. Furthermore, we applied a
second-degree polynomial curve to fit the data, obtaining an $R^{2}$ value of
$0.984$. This value suggests a strong fit between the predicted execution
times and the actual data and implies that the relationship between the
snippet length and execution time is not linear but rather quadratic, with
execution times increasing at a rate that accelerates with the number of
tokens.

The polynomial relationship might be due to various factors inherent in code
analysis processes, such as increased memory allocation, more intensive
parsing, and a higher number of computational operations needed to process and
analyze larger codebases.

##  6 Threats to Validity

Rule Creation: The creation of detection rules based on regular expressions
could introduce bias, as these rules are derived from patterns observed within
a limited set of vulnerable code snippets. This limitation may result in rules
that are either too specific or too general, affecting the tool’s accuracy in
real-world scenarios. To mitigate this threat, we employed a diverse dataset
of vulnerabilities covering a broad range of CWEs and OWASP categories. We
also conducted iterative refinements of our rules by testing them on separate
validation sets to ensure they accurately capture the intended patterns
without being overly broad or narrow.

AI Models: The selection of AI models for evaluating _DeVAIC_ may introduce
bias, as the performance of these models can vary significantly. This
variation could inadvertently affect the perceived effectiveness of _DeVAIC_.
However, we remark that we selected four widely used AI code generation
models, representing a range of underlying technologies and training datasets.
This diversity helps ensure that our evaluation encompasses a variety of code-
generation behaviours and vulnerabilities.

Metrics: The use of Precision, Recall, $F_{1}$ Score, and Accuracy as metrics
relies on correctly classifying vulnerabilities. Any misclassification could
impact these metrics and the interpretation of _DeVAIC_ effectiveness. To
enhance the reliability of our classification, we employed a multi-researcher
approach, where multiple experts independently assessed the vulnerabilities
before reaching a consensus. This process reduces the risk of subjective bias
and ensures a more accurate classification of true and false
positives/negatives.

Generalizability: The effectiveness of _DeVAIC_ is currently limited to Python
code. This focus raises questions about the tool’s applicability to other
programming languages that may have different syntax, semantics, and common
vulnerabilities. While _DeVAIC_ is initially designed for Python, the
methodology used to create detection rules based on regular expressions and
patterns observed in vulnerable code can be adapted to other languages. We are
currently investigating this methodology for the extraction of vulnerable
implementation patterns and rule creation in the C/C++ language. Adapting
_DeVAIC_ to C/C++ requires additional considerations due to the complexity and
variety in how equivalent functionalities are implemented in these languages
compared to Python (e.g., what can be accomplished with a single Python
instruction might require several more complex instructions in C/C++). This
complexity increases the difficulty of pattern identification and necessitates
the development of more sophisticated detection rules. Future work will
involve extending _DeVAIC_ ’s capabilities to additional languages and
leveraging domain experts to ensure the accuracy and relevance of new rules.

##  7 Conclusion

In this work, we introduced _DeVAIC_ , a tool that implements a set of
detection rules to identify vulnerabilities in the Python code. The tool is
designed to overcome the limitation of static analysis tools since it does not
require complete programs, making it particularly suitable for AI-generated
code. To define rules, we extracted vulnerable code from public datasets and,
after grouping the code according to their OWASP categories and similarity, we
found common patterns by standardizing the code snippets and using the LCS. We
evaluated _DeVAIC_ on code generated by four public AI-code generators. The
results outline the tool’s ability to detect vulnerabilities, showing an
average $F_{1}$ Score and Accuracy both at $94\%$, overcoming the performance
of other state-of-the-art solutions used as a baseline for the evaluation,
with a limited computational cost.

Future work aims to extend the number of program languages to analyze,
enhancing the set of detection rules available. Moreover, we also aim to
enrich the list of CWEs covered by the tool.

## References

  * [1] Muna Al-Hawawreh, Ahamed Aljuhani, and Yaser Jararweh.  Chatgpt for cybersecurity: practical applications, challenges, and future directions.  Cluster Computing, pages 1–16, 2023.
  * [2] Atieh Bakhshandeh, Abdalsamad Keramatfar, Amir Norouzi, and Mohammad Mahdi Chekidehkhoun.  Using chatgpt as a static application security testing tool.  arXiv preprint arXiv:2308.14434, 2023.
  * [3] Vinuri Bandara, Thisura Rathnayake, Nipuna Weerasekara, Charitha Elvitigala, Kenneth Thilakarathna, Primal Wijesekera, and Chamath Keppitiyagama.  Fix that fix commit: A real-world remediation analysis of javascript projects.  In 2020 IEEE 20th International Working Conference on Source Code Analysis and Manipulation (SCAM), pages 198–202. IEEE, 2020.
  * [4] Bard/FAQ.  What Can Bard Do and Other Frequently Asked Questions - Bard.  https://bard.google.com/faq?hl=en, 2023.
  * [5] Bing/FAQ.  Microsoft Bing - Frequently Asked Questions.  https://www.microsoft.com/en-us/bing?form=MA13FV, 2023.
  * [6] Carnegie Mellon University NeuLab and STRUDEL Lab.  CoNaLa.  https://conala-corpus.github.io/.
  * [7] Saikat Chakraborty, Rahul Krishna, Yangruibo Ding, and Baishakhi Ray.  Deep learning based vulnerability detection: Are we there yet.  IEEE Transactions on Software Engineering, 2021.
  * [8] Yizheng Chen, Zhoujie Ding, Lamya Alowain, Xinyun Chen, and David Wagner.  Diversevul: A new vulnerable source code dataset for deep learning based vulnerability detection.  In Proceedings of the 26th International Symposium on Research in Attacks, Intrusions and Defenses, pages 654–668, 2023.
  * [9] Boris Cherry, Pol Benats, Maxime Gobert, Loup Meurice, Csaba Nagy, and Anthony Cleve.  Static analysis of database accesses in mongodb applications.  In 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 930–934. IEEE, 2022.
  * [10] Anton Cheshkov, Pavel Zadorozhny, and Rodion Levichev.  Evaluation of chatgpt model for vulnerability detection.  arXiv preprint arXiv:2304.07232, 2023.
  * [11] Brian Chess and Gary McGraw.  Static analysis for security.  IEEE security & privacy, 2(6):76–79, 2004.
  * [12] Domenico Cotroneo, Cristina Improta, Pietro Liguori, and Roberto Natella.  Vulnerabilities in ai code generators: Exploring targeted data poisoning attacks.  arXiv preprint arXiv:2308.04451, 2023.
  * [13] Trevor Dunlap, Seaver Thorn, William Enck, and Bradley Reaves.  Finding fixed vulnerabilities with off-the-shelf static analysis.  In 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P), pages 489–505. IEEE, 2023.
  * [14] Jiahao Fan, Yi Li, Shaohua Wang, and Tien N Nguyen.  Ac/c++ code vulnerability dataset with code changes and cve summaries.  In Proceedings of the 17th International Conference on Mining Software Repositories, pages 508–512, 2020.
  * [15] Chongzhou Fang, Ning Miao, Shaurya Srivastav, Jialin Liu, Ruoyu Zhang, Ruijie Fang, Asmita Asmita, Ryan Tsang, Najmeh Nazari, Han Wang, et al.  Large language models for code analysis: Do llms really do their job?  arXiv preprint arXiv:2310.12357, 2023.
  * [16] GitHub.  CodeQL.  https://codeql.github.com/.
  * [17] GitHub.  CodeQL: Experimental Security Queries for Python language.  https://github.com/github/codeql/tree/main/python/ql/src/experimental/Security.
  * [18] GitHub.  CodeQL: Security Queries for Python language.  https://github.com/github/codeql/tree/main/python/ql/src/Security.
  * [19] GitHub.  The top programming languages.  https://octoverse.github.com/2022/top-programming-languages.
  * [20] GitHub.  GitHub Copilot.  https://github.com/features/copilot, 2023.
  * [21] GitHub Docs.  Python queries for CodeQL analysis.  https://docs.github.com/en/code-security/code-scanning/managing-your-code-scanning-configuration/python-built-in-queries, 2023.
  * [22] GitHub. [n. d.].  GitHub Copilot for Individuals.  https://docs.github.com/en/copilot/overview-of-github-copilot/about-github-copilot-for-individuals.
  * [23] GitHub/Blog.  GitHub Copilot now has a better AI model and new capabilities.  https://github.blog/2023-02-14-github-copilot-now-has-a-better-ai-model-and-new-capabilities/, 2023.
  * [24] Matías Gobbi and Johannes Kinder.  Poster: Using codeql to detect malware in npm.  In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security, pages 3519–3521, 2023.
  * [25] Google.  Google Gemini.  https://gemini.google.com/app, 2023.
  * [26] Katerina Goseva-Popstojanova and Andrei Perhinschi.  On the capability of static code analysis to detect security vulnerabilities.  Information and Software Technology, 68:18–33, 2015.
  * [27] Hossein Hajipour, Thorsten Holz, Lea Schönherr, and Mario Fritz.  Systematically finding security vulnerabilities in black-box code generation models.  arXiv preprint arXiv:2302.04012, 2023.
  * [28] Sivana Hamer, Marcelo d’Amorim, and Laurie Williams.  Just another copy and paste? comparing the security vulnerabilities of chatgpt generated code and stackoverflow answers.  arXiv preprint arXiv:2403.15600, 2024.
  * [29] Anne Honkaranta, Tiina Leppänen, and Andrei Costin.  Towards practical cybersecurity mapping of stride and cwe—a multi-perspective approach.  In 2021 29th Conference of Open Innovations Association (FRUCT), pages 150–159. IEEE, 2021.
  * [30] Aftab Hussain, Md Rafiqul Islam Rabin, and Mohammad Amin Alipour.  Trojanedcm: A repository for poisoned neural models of source code.  arXiv preprint arXiv:2311.14850, 2023.
  * [31] Cristina Improta.  Poisoning programs by un-repairing code: Security concerns of ai-generated code.  In 2023 IEEE 34th International Symposium on Software Reliability Engineering Workshops (ISSREW), pages 128–131. IEEE, 2023.
  * [32] DA Kapustin, VV Shvyrov, and TI Shulika.  Static analysis of corpus of source codes of python applications.  Programming and Computer Software, 49(4):302–309, 2023.
  * [33] Adhishree Kathikar, Aishwarya Nair, Ben Lazarine, Agrim Sachdeva, and Sagar Samtani.  Assessing the vulnerabilities of the open-source artificial intelligence (ai) landscape: A large-scale analysis of the hugging face platform.  In 2023 IEEE International Conference on Intelligence and Security Informatics (ISI), pages 1–6. IEEE, 2023.
  * [34] Raphaël Khoury, Anderson R Avila, Jacob Brunelle, and Baba Mamadou Camara.  How secure is code generated by chatgpt?  arXiv preprint arXiv:2304.09655, 2023.
  * [35] Jia Li, Zhuo Li, HuangZhao Zhang, Ge Li, Zhi Jin, Xing Hu, and Xin Xia.  Poison attack and poison detection on deep source code processing models.  ACM Transactions on Software Engineering and Methodology, 2023.
  * [36] Li Li, Tegawendé F Bissyandé, Mike Papadakis, Siegfried Rasthofer, Alexandre Bartel, Damien Octeau, Jacques Klein, and Le Traon.  Static analysis of android apps: A systematic literature review.  Information and Software Technology, 88:67–95, 2017.
  * [37] Yi Li, Shaohua Wang, and Tien N Nguyen.  Vulnerability detection with fine-grained interpretations.  In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pages 292–303, 2021.
  * [38] Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong.  Vuldeepecker: A deep learning-based system for vulnerability detection.  arXiv preprint arXiv:1801.01681, 2018.
  * [39] Pietro Liguori, Erfan Al-Hossami, Domenico Cotroneo, Roberto Natella, Bojan Cukic, and Samira Shaikh.  Shellcode_IA32: A dataset for automatic shellcode generation.  In Royi Lachmy, Ziyu Yao, Greg Durrett, Milos Gligoric, Junyi Jessy Li, Ray Mooney, Graham Neubig, Yu Su, Huan Sun, and Reut Tsarfaty, editors, Proceedings of the 1st Workshop on Natural Language Processing for Programming (NLP4Prog 2021), pages 58–64, Online, August 2021. Association for Computational Linguistics.
  * [40] Pietro Liguori, Erfan Al-Hossami, Domenico Cotroneo, Roberto Natella, Bojan Cukic, and Samira Shaikh.  Can we generate shellcodes via natural language? an empirical study.  Automated Software Engineering, 29(1):30, 2022.
  * [41] Pietro Liguori, Erfan Al-Hossami, Vittorio Orbinato, Roberto Natella, Samira Shaikh, Domenico Cotroneo, and Bojan Cukic.  Evil: exploiting software via natural language.  In 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE), pages 321–332. IEEE, 2021.
  * [42] Damian Lyons and Dino Becaj.  A meta-level approach for multilingual taint analysis.  In International Conference on Software and Data Technologies (ICSOFT), volume 2021, 2021.
  * [43] Li Ma, Huihong Yang, Jianxiong Xu, Zexian Yang, Qidi Lao, and Dong Yuan.  Code analysis with static application security testing for python program.  Journal of Signal Processing Systems, 94(11):1169–1182, 2022.
  * [44] Siqi Ma, Ferdian Thung, David Lo, Cong Sun, and Robert H Deng.  Vurle: Automatic vulnerability detection and repair by learning from examples.  In Computer Security–ESORICS 2017: 22nd European Symposium on Research in Computer Security, Oslo, Norway, September 11-15, 2017, Proceedings, Part II 22, pages 229–246. Springer, 2017.
  * [45] Ferdiansyah Mastjik, Cihan Varol, and Asaf Varol.  Comparison of pattern matching techniques on identification of same family malware.  International Journal of Information Security Science, 4(3):104–111, 2015.
  * [46] Leo A Meyerovich and Ariel S Rabkin.  Empirical analysis of programming language adoption.  In Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications, pages 1–18, 2013.
  * [47] Microsoft.  CodeXGLUE NL prompts.  https://github.com/microsoft/CodeXGLUE/blob/main/Text-Code/text-to-code/dataset/concode/test.json, 2021.
  * [48] Microsoft.  Microsoft Copilot.  https://www.microsoft.com/it-it/microsoft-copilot?market=it, 2023.
  * [49] MITRE.  2021 CWE Top 25 Most Dangerous Software Weaknesses.
https://cwe.mitre.org/top25/archive/2021/2021_cwe_top25.html.

  * [50] MITRE.  2022 CWE Top 25 Most Dangerous Software Weaknesses.
https://cwe.mitre.org/top25/archive/2022/2022_cwe_top25.html.

  * [51] MITRE.  2023 CWE Top 25 Most Dangerous Software Weaknesses.
https://cwe.mitre.org/top25/archive/2023/2023_top25_list.html.

  * [52] Austin Mordahl.  Automatic testing and benchmarking for configurable static analysis tools.  In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1532–1536, 2023.
  * [53] Sarah Nadi, Thorsten Berger, Christian Kästner, and Krzysztof Czarnecki.  Mining configuration constraints: Static analyses and empirical results.  In Proceedings of the 36th international conference on software engineering, pages 140–151, 2014.
  * [54] NIST.  NIST Software Assurance Reference Dataset.  https://samate.nist.gov/SARD/, 2023.
  * [55] odashi.  Django Dataset for Code Translation Tasks.  https://github.com/odashi/ase15-django-dataset.
  * [56] OpenAI.  HumanEval: Hand-Written Evaluation Set.  https://github.com/openai/human-eval.
  * [57] OpenAI.  OpenAI ChatGPT.  https://openai.com/blog/chatgpt/, 2023.
  * [58] OWASP.  OWASP Top 10:2021 .  https://owasp.org/Top10/.
  * [59] OWASP.  Path Traversal.  https://owasp.org/www-community/attacks/Path_Traversal, 2024.
  * [60] Ya Pan, Xiuting Ge, Chunrong Fang, and Yong Fan.  A systematic literature review of android malware detection using static analysis.  IEEE Access, 8:116363–116379, 2020.
  * [61] Nency Patel and Narendra Shekokar.  Implementation of pattern matching algorithm to defend sqlia.  Procedia Computer Science, 45:453–459, 2015.
  * [62] Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri.  Asleep at the keyboard? assessing the security of github copilot’s code contributions.  In 2022 IEEE Symposium on Security and Privacy (SP), pages 754–768. IEEE, 2022.
  * [63] Pearce et al.  Copilot CWE Scenarios Dataset.  https://zenodo.org/records/5225651, 2023.
  * [64] Shuanghe Peng, Peiyao Liu, and Jing Han.  A python security analysis framework in integrity verification and vulnerability detection.  Wuhan University Journal of Natural Sciences, 24(2):141–148, 2019.
  * [65] PyCQA.  Bandit.  https://github.com/PyCQA/bandit/tree/main.
  * [66] Python.  Abstract Syntax Tree Documentation.  https://docs.python.org/3/library/ast.html.
  * [67] python-security.  PyT.  https://github.com/python-security/pyt.
  * [68] PyYAML.  PyYAML Documentation.  https://pyyaml.org/wiki/PyYAMLDocumentation.
  * [69] G Appa Rao, K Venkata Rao, PVGD Prasad Reddy, and T Lava Kumar.  An efficient procedure for characteristic mining of mathematical formulas from document.  International Journal of Engineering Science and Technology (IJEST), 10(03), 2018.
  * [70] G Appa Rao, G Srinivas, K Venkata Rao, and PVGD Prasad Reddy.  Characteristic mining of mathematical formulas from document-a comparative study on sequence matcher and levenshtein distance procedure.  International Journal of Computer Sciences and Engineering, 6(4):400–403, 2018.
  * [71] G Appa Rao, G Srinivas, K Venkata Rao, and PVGD Prasad Reddy.  A partial ratio and ratio based fuzzy-wuzzy procedure for characteristic mining of mathematical formulas from documents.  IJSC—ICTACT J Soft Comput, 8(4):1728–1732, 2018.
  * [72] returntocorp.  Semgrep.  https://github.com/returntocorp/semgrep.
  * [73] Jukka Ruohonen, Kalle Hjerppe, and Kalle Rindell.  A large-scale security-oriented static analysis of python packages in pypi.  In 2021 18th International Conference on Privacy, Security and Trust (PST), pages 1–10. IEEE, 2021.
  * [74] S2E-Lab.  SecurityEval NL prompts.  https://github.com/s2e-lab/SecurityEval/blob/main/dataset.jsonl, 2022.
  * [75] Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Brendan Dolan-Gavitt, and Siddharth Garg.  Security implications of large language model code assistants: A user study.  arXiv preprint arXiv:2208.09727, 2022.
  * [76] Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt.  Lost at c: A user study on the security implications of large language model code assistants.  arXiv preprint arXiv:2208.09727, 2023.
  * [77] Security & Software Engineering Research Lab at University of Notre Dame.  SecurityEval.  https://github.com/s2e-lab/SecurityEval, 2023.
  * [78] Semgrep.  Semgrep Registry.  https://semgrep.dev/r, 2024.
  * [79] Mohammed Latif Siddiq and Joanna C. S. Santos.  Securityeval dataset: Mining vulnerability examples to evaluate machine learning-based code generation techniques.  In Proceedings of the 1st International Workshop on Mining Software Repositories Applications for Privacy and Security (MSR4P&S22), 2022.
  * [80] SoftSec Institute.  LLMSecEval.  https://github.com/tuhh-softsec/LLMSecEval/, 2023.
  * [81] Statista.  Most used programming languages among developers worldwide as of 2023.  https://www.statista.com/statistics/793628/worldwide-developer-survey-most-used-languages/.
  * [82] Richard J Thomas and Tom Chothia.  Learning from vulnerabilities-categorising, understanding and detecting weaknesses in industrial control systems.  In Computer Security: ESORICS 2020 International Workshops, CyberICPS, SECPRE, and ADIoT, Guildford, UK, September 14–18, 2020, Revised Selected Papers 6, pages 100–116. Springer, 2020.
  * [83] Catherine Tony, Markus Mutas, Nicolas Díaz Ferreyra, and Riccardo Scandariato.  Llmseceval: A dataset of natural language prompts for security evaluations.  In 2023 IEEE/ACM 20th International Conference on Mining Software Repositories (MSR), 2023.
  * [84] tuhh-softsec.  LLMSecEval NL prompts.  https://github.com/tuhh-softsec/LLMSecEval/blob/main/Dataset/LLMSecEval-prompts.json, 2023.
  * [85] Andrew Walenstein, Michael Venable, Matthew Hayes, Christopher Thompson, and Arun Lakhotia.  Exploiting similarity between variants to defeat malware.  In Proc. BlackHat DC Conf. Citeseer, 2007.
  * [86] ZeoVan.  MSR_20_Code_vulnerability_CSV_Dataset.  https://github.com/ZeoVan/MSR_20_Code_vulnerability_CSV_Dataset, 2020.
  * [87] Ying Zhang, Ya Xiao, Md Mahir Asef Kabir, Danfeng Yao, and Na Meng.  Example-based vulnerability detection and repair in java code.  In Proceedings of the 30th IEEE/ACM International Conference on Program Comprehension, pages 190–201, 2022.
  * [88] Li Zhong and Zilong Wang.  A study on robustness and reliability of large language model code generation.  arXiv preprint arXiv:2308.10335, 2023.