# Automatic Static Bug Detection for Machine Learning Libraries: Are We There
Yet?

Nima Shiri harzevili1, Jiho Shin1, Junjie Wang2, Song Wang1, Nachiappan
Nagappan3  1York University; 2Institute of Software, Chinese Academy of
Sciences; 3META
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Automatic detection of software bugs is a critical task in software security.
Many static tools that can help detect bugs have been proposed. While these
static bug detectors are mainly evaluated on general software projects call
into question their practical effectiveness and usefulness for machine
learning libraries. In this paper, we address this question by analyzing five
popular and widely used static bug detectors, i.e., Flawfinder, RATS,
Cppcheck, Facebook Infer, and Clang static analyzer on a curated dataset of
software bugs gathered from four popular machine learning libraries including
Mlpack, MXNet, PyTorch, and TensorFlow with a total of 410 known bugs. Our
research provides a categorization of these tools’ capabilities to better
understand the strengths and weaknesses of the tools for detecting software
bugs in machine learning libraries. Overall, our study shows that static bug
detectors find a negligible amount of all bugs accounting for 6/410 bugs
(0.01%), Flawfinder and RATS are the most effective static checker for finding
software bugs in machine learning libraries. Based on our observations, we
further identify and discuss opportunities to make the tools more effective
and practical.

###### Index Terms:

Software bugs, static detection, machine learning libraries

##  I Introduction

Programming inevitably involves dealing with bugs in software, which is an
aspect that can be frustrating for many developers since detecting and fixing
bugs is time-consuming [1, 2]. To help developers find software bugs, many
static bug detectors have been developed and are now frequently employed by
many industries and open-source projects [3, 4, 5, 6]. Error Prone from Google
[7], Infer from Facebook [8], and SpotBugs [9], the successor to the widely
used FindBugs tool [10], are examples of popular static bug detection
applications. These tools are usually developed as an analytical framework
based on static analysis and are capable of scaling to large applications.

Previous research has examined static bug detectors on traditional projects
from different aspects [11, 12, 13, 14, 15, 16]. There are two major
limitations in existing studies. First, the datasets used for the empirical
evaluation of static bug detectors are not real-world examples, they are not
able to replicate new and sophisticated bug patterns. Second, they do not
categorize the type of bugs based on Common Weakness Enumeration(CWE)
information. Recently, Lipp et al. [16] addressed the limitations by proposing
an empirical evaluation of static bug detectors on real-world datasets
collected from CVE records gathered from 27 projects containing 1.15 million
lines of code. Their results showed that state-of-the-art tools can detect in-
between 20% and 53% of the bugs in a benchmark set of real-world programs.
However, it has not been determined if the findings of these studies on
conventional projects are applicable to machine learning (ML) libraries.
Finding real-world security bugs in ML libraries is critical for a couple of
reasons. First, ML libraries have been widely used in many fields in the past
decades, such as image classification [17, 18], big data analysis [19],
pattern recognition [20], autonomous driving [21, 22, 23], and natural
language processing [24, 25, 26]. Failure to detect bugs in these ML libraries
might have devastating implications, such as traffic accidents [27]. Second,
gaining an understanding of the benefits and drawbacks of the static bug
detectors that are currently in use will direct future work toward the design
of new methodologies or tools for detecting ML-specific bugs.

In this paper, we set out to investigate the effectiveness of five popular
static bug detectors on ML libraries. Specifically, we conduct an empirical
analysis utilizing 410 real-world bugs collected from four widely-used popular
ML libraries to address the issue of how many bugs from these ML libraries can
be detected by static bug detectors and why they miss detecting real-world
security bugs of ML libraries. We select five popular and open-source static
bug detectors as the research subjects, i.e., Flawfinder [28, 29, 16, 30, 31,
32, 33, 3, 34], RATS [35, 33, 3], Cppcheck [36, 29, 3, 34], Facebook Infer [8,
37, 38, 38], and Clang static analyzer [39, 3, 40, 41, 42]. The fundamental
strategy entails employing each bug detector on a version of a program that is
afflicted with a particular bug and determining whether or not the bug is
discovered by the bug detector. The primary obstacle of the proposed empirical
study is determining whether or not the collection of warnings that a bug
detector has reported actually captures the issued bug. To tackle the problem,
we filtered the reported warnings based on their line number that involve in
the bug fix following existing studies [43]. Then we conduct a manual
inspection of the detected bug candidates to avoid possible coincidental
matches. In addition, we have also investigated bug detectors given a set of
program analysis criteria such as input representation, pattern matching
techniques, and source code element sensitivity. Specifically, we manually
examined the documentation as well as the source code of these static bug
detectors to understand how they function. We also study the characteristics
of vulnerabilities in ML libraries and understand their patterns. As a result,
we provide future directions to improve existing static checkers to deal with
specific vulnerabilities in ML libraries.

This paper has the following contributions:

  * •

We present the first empirical study on the investigation of current static
bug detection on four widely-used machine learning libraries.

  * •

We identify gaps between existing static bug detection techniques and software
security practices on machine learning libraries.

  * •

We explore the capabilities and limitations of each static bug detector, as
well as suggest future ideas for enhancing the effectiveness of these tools in
real-world settings to detect software vulnerabilities specific to ML
libraries.

  * •

We release the dataset of our experiments to help other researchers replicate
and extend our
study111https://anonymous.4open.science/r/ISSRE2023SATS-E255/README.md.

⬇

numbers=left,

firstnumber=43,

language = Python,

escapeinside={(*}{*)}]

gtl::InlinedVector<npy_intp, 4> dims(ndims);

if (TF_TensorType(tensor) == TF_RESOURCE) {

dims[0] = TF_TensorByteSize(tensor);

CHECK_EQ(ndims, 0)}

<< "Fetching of non-scalar

resource tensors is

not supported.";

dims.push_back(TF_TensorByteSize(tensor));

*nelems = dims[0];

} else {

*nelems = 1;

(a) An example of buffer overflow in TensorFlow library.

⬇

escapechar=,

language = C,

escapeinside={(*}{*)}]

chip = get_chip_info(sdev->pdata);

for (i = 0; i < HDA_EXT_ROM_STATUS_SIZE; i++) {

value = snd_sof_dsp_read(sdev, HDA_DSP_BAR,

chip->rom_status_reg + i * 0x4);

len += snprintf(msg + len, sizeof(msg) - len,

" 0x%x", value);

len += scnprintf(msg + len, sizeof(msg) - len,

" 0x%x", value);

}

dev_printk(level, sdev->dev,

"extended rom status: %s", msg);

(b) An example of buffer overflow in Linux kernel.

Figure 1: This figure shows two examples of buffer overflow from the
TensorFlow library and Linux kernel.

##  II Motivation

ML libraries and traditional software differ in various aspects such as
complexity, data dependency, and performance characteristics. Overall, ML
libraries tend to be more complex than traditional software as machine
learning often involves complex mathematical models and algorithms [44, 45,
46, 47]. In addition, as ML models are heavily dependent on the quality and
quantity of the data used in their training process, most ML libraries provide
tools for data pre-processing, cleaning, and normalization that are not
available in traditional software. ML libraries are designed to take advantage
of modern hardware, including GPUs and specialized processors, to optimize
performance. Traditional software, on the other hand, is generally optimized
for general-purpose CPUs.

These differences introduce different bugs compared to traditional software.
Figure 1(a) and Figure 1(b) show two examples of buffer overflow in TensorFlow
library222https://github.com/tensorflow/tensorflow/commit/13ef0af4867477cdda7e0b294e61560c2952df42
and Linux
kernel333https://github.com/torvalds/linux/commit/5549af7f42916c0d7e78a0e423ac667e27eaac3e
respectively. The root cause of buffer overflow in Figure 1(a) is lack of
proper input validation when getting a non-scalar resource tensor while the
root cause of buffer overflow in Figure 1(b) is using snprintf() instead of
using scnprintf() as snprintf() is not a secure built-in C API. We can see
that there are two instances of the same bug (i.e., buffer overflow) that have
distinct root causes. To detect a heap buffer overflow example in the
TensorFlow library as shown in Figure 1(a), existing static bug detectors must
follow two crucial steps. Firstly, these tools need to recognize the
TensorFlow-specific macro checkers (CHECK_EQ) by incorporating the macro call
signature into their internal database of risky APIs. Secondly, they must
conduct data flow analysis to trace the flow of data from client APIs to the
backend implementation. In this example, the vulnerable parameter is ndims
which has been coming from the client API usage. Unfortunately, we discovered
that the current tools lack the capability to perform these essential steps,
rendering them inadequate for detecting machine learning-specific bugs. Please
note that there are more complex bug patterns that we discuss in RQ3.

The different symptoms and root causes of the same type of security
vulnerabilities from ML libraries and traditional software motivate us to look
deeper into understanding the performance of current static bug detection on
machine learning libraries.

##  III Study design

In this section, we first show how we curated a dataset of real-world bugs of
ML libraries (Section III-A), then we introduce the static bug detection tools
used in this paper (Section III-B), finally, we describe the procedure of
applying bug detectors to our curated real-world data (Section III-C).

###  III-A Collection of Bugs from ML Libraries

Our study is on the basis of four widely used ML libraries selected based on a
set of inclusion criteria: 1) libraries should be open source and available to
the public, 2) they should be under active development, 3) libraries should
have the implementation of classical ML as well as state-of-the-art DL models,
4) libraries should support different tasks in the common ML workflow. The
outcome of applying these criteria is the following ML libraries; Mlpack,
MXNet, PyTorch, and TensorFlow. Our filters also exclude some well-known
libraries including Caffe, Theano, and Keras due to the fact that the data
extraction mechanism utilized in this work did not retrieve enough real-world
security vulnerabilities from their repositories due to a lack of sufficient
vulnerable data in the GitHub repository of these libraries.

For each of the studied ML libraries, we followed the approach proposed by
Zhou et al [48] to extract venerability-related commits. Note that we extract
all commits in the default branch of each library since the starting date of
the development. Specifically, we use their regular expression rules,
including expressions and keywords related to security issues, to collect
security bug-fixing commits. As a result, we collected around 5k commits to
the four studied projects. Note that, the collected commits might contain
noises due to the fact that there may be coincidental matches between bug
keywords and the keywords inside the commit message and title [48]. To remove
noises, we further conducted a manual inspection on each commit. In our manual
inspection, two authors began evaluating extracted commits simultaneously. The
authors analyzed the title, message, merged pull requests, and linked
issues444Some bug fixing commits fix opened issues. Thus, the authors further
analyzed them for noise removal. of each bug fixing commit and remove the
commits that are not related to a registered security bug in CWE
website555https://cwe.mitre.org/. As a result, we collected 410 bug-fixing
commits across the four projects that were investigated shown in Table I.

TABLE I: Characteristics of bug fixing commits (BFCs) used in this paper. Library | Language | LOC$(\sim)$ | # BFCs
---|---|---|---
Mlpack | C++ | 340K | 47
MXNet | C++/Python | 362K | 60
PyTorch | C++/Python | 3.8M | 58
TensorFlow | C++/Python | 567K | 245
Overall | 5M | 410

###  III-B Running Static Bug Detection Tools

In this paper, we study five publicly available static bug detection tools.
Our primary emphasis is on static detectors that are freely accessible to the
public. Please note that three of the tools are under active development
including Facebook Infer [8], Clang static analyzer [39], and
Cppcheck666https://cppcheck.sourceforge.io/. The two remaining tools
(Flawfinder [49] and RATS777https://github.com/andrew-d/rough-auditing-tool-
for-security) are not actively developing but they have been frequently used
in software bug detection [3, 50, 51].

The static bug detectors used in this work have several common
characteristics. For example, Flawfinder and RATS treat the input source code
as a text sequence while Cppcheck, Infer, and Clang static analyzers convert
the source code into an intermediate representation. All tools use a built-in
database of patterns that are used to detect security vulnerabilities. For
instance, RATS, Flawfinder, and Cppcheck all use a database of C/C++ system
functions that are known to have security bugs such as buffer overflow or
format string issues. while Infer uses a bi-abduction inference and analysis
in order to find vulnerable source code statements. Clang static analyzer work
by parsing the source code and checking it against a set of predefined checks,
or linters.

TABLE II: Characteristics of tools based on different program analysis factors. In this table, the asterisk sign indicates their partial capability in considering program analysis factors. Also, N.A shows that we do not have enough information about the program analysis factor. Tool | Version | Input representation | Pattern matching | Sensitivity
---|---|---|---|---
Text | AST | Other | Intraprocedural | Interprocedural | Context | Field | Object | Data Flow | Control Flow
Flawfinder | 2.0.19 | $\checkmark$ | - | - | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
RATS | 2.4 | $\checkmark$ | - | - | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$ | $\times$
Cppcheck | 2.7 | - | $\checkmark$ | - | $\checkmark$ | $\times$ | $\checkmark$ | $\times$ | $\times$ | $\ast$ | $\ast$
Infer | 1.1.0 | - | - | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | N.A | $\checkmark$ | $\checkmark$
Clang static analyzer | 14.0.0 | - | $\checkmark$ | - | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$ | $\checkmark$

For the capabilities of these five static bug detectors used in this paper,
following existing work [15], we take into consideration the following three
different types of program analysis features, i.e., input representation,
matching strategies (intraprocedural and interprocedural), and sensitivity to
program elements including flow, context, field, object, path, and field. We
investigated the tools in relation to the aforementioned analytical program
analysis features. During this procedure, we manually evaluated the tools’
source code and documentation, which demonstrated several types of behaviors
to check the tools’ capabilities and limits. In the following paragraphs, each
tool is explained in detail. Table II shows the capabilities of static
analysis tools investigated.

Flawfinder [28]: is a static analysis tool that scans a program for potential
security bugs by using a database of known unsafe C/C++ functions. Flawfinder
can detect problems with race conditions and system calls in addition to
`printf()` and normal string manipulation operations, based on an internal
database that contains C/C++ routines that are known to have vulnerabilities
in their design.

RATS [35]: An open-source static analyzer that is able to analyze code bases
written in C, C++, Perl, PHP, and Python. Similar to Flawfinder, it uses an
internal database of risky C/C++ API signatures and uses a keyword-matching
approach to find and mark them as vulnerable in the target source code.

Cppcheck [36]: Cppcheck offers one-of-a-kind code analysis to find defects and
focuses on finding undefined behavior as well as risky coding structures. The
objective is to generate an extremely low number of false positives.

Infer [8]: is a static analysis tool that is developed by Facebook. It
searches for a wide variety of vulnerabilities in programs written in Java,
C/C++, and Objective-C. Bi-abduction analysis is one of the methods that Infer
employs in order to locate vulnerabilities such as deadlocks, memory leaks,
and null pointer dereference. In order to perform analysis, Infer requires a
set of compilation commands for each file. Infer is an interprocedural
analysis tool which means that it allows keeping track of objects and
variables between methods, and also global variables.

Clang static analyzer [39]: Clang static analyzer is built on top of the LLVM
project, which offers a set of modular and reusable compiler and toolchain
technologies. It is an extensible framework for C/C++ code linting that may be
used to enforce coding standards, conduct static analysis, and discover
probable errors. It operates by scanning the source code and comparing it to a
collection of predefined checks, known as linters. The checks are built as
separate modules, making it simple to add new checks or modify current ones.

Note that the static bug detectors used in this paper have capabilities to
report non-bug related warnings, e.g., styling or refactoring issues, as well
as reporting general bugs which are not related to software bugs that have a
unique id in CWE website888https://cwe.mitre.org/. In order to reduce false
positive rates and prevent static checkers to produce related warnings to
bugs, we have configured them to merely report warnings that are strongly
related to software security vulnerabilities. Please note that RATS does not
generate styling issues. In addition, we manually studied Flawfinder’s built-
in database and discovered that all rules have a unique id associated with CWE
records. As a result, we solely used this setup for Infer, Cppcheck, and Clang
static analyzer.

###  III-C Identification of bug Candidates

One of the main challenges in the identification of bug candidates is to check
whether the reported warnings by static detectors are corresponding with the
issued bug class or not. To identify bug candidates, we use the following
steps. First, we performed automatic filtering based on the commits’ diff
information as used in previous work [14, 15]. Afterward, we further analyzed
the reported warnings that involve the commit using manual inspection.

####  III-C1 Automatic filtering

We filter reported warnings by using a diff-based mapping technique which uses
code diffs between vulnerable and fixed programs in a fixing commit [43].
First, it computes a set of lines in the vulnerable program that is flagged
with at least one warning. Then, it checks whether the flagged line numbers
overlap with the changed lines in the fixing commit. If the flagged lines
overlap with the code change, the warning is considered a bug candidate.
Otherwise, it is not a candidate for manual inspection. For example, Figure 2
shows an example of a warning generated by Flawfinder that is considered to be
a vulnerable candidate since the line number reported in the warning overlaps
with the line number in the code change of the bug fixing commit.

⬇

for (int i = 0; i < client->device_count(); ++i) {

se::StreamExecutorConfig config;

config.ordinal = i; config.device_options.non_portable_tags

["host_thread_stack_size_in_bytes"] =

(* \ctext[RGB]{255, 99, 71}{-absl::StrCat(2048 * 1024);}*)

(* \ctext[RGB]{0, 153, 76}{+absl::StrCat(8192 * 1024);}*)

TF_ASSIGN_OR_RETURN(se::StreamExecutor * executor,

platform->GetExecutor(config));

(a) bug fixing commit.

⬇

escapechar=,

language = C,

escapeinside={(*}{*)}]

Examining /cpu_device.cc

FINAL RESULTS:

cpu_device.cc:47:

[4] (buffer) StrCat:

Does not check for buffer overflows when

concatenating to destination

[MS-banned] (CWE-120).

absl::StrCat(2048 * 1024)

(b) reported warning by Flawfinder.

Figure 2: An example of bug fixing commit in TensorFlow library in which the
developer has increased the thread stack size from 2048 to 8192 to prevent
buffer overflow bug. Figure 2(b) shows the generated warning by Flawfinder in
which there is a possible buffer overflow at line 47.

####  III-C2 Manual verification

In this step, we manually scan all candidates and check the warning messages
against the vulnerable and clean versions of the code to eliminate potential
accidental matches. The first two authors examine the warnings and bug-fixing
commits simultaneously. They manually analyze the warning message, the CWE-ID,
and the line number where the bug is reported. Based on the above information,
they review the code changes to confirm whether the warning is related to the
bug being addressed in the commit. Note that, in the cases where the detector
tools reported a warning at a line outside those changed lines in the bug-
fixing commit, we manually checked the warning. In addition, we have also
performed a backward analysis in which we traced all variables and function
calls within the code changes up to the marked line that was outside the code
changes. If the marked line was associated with the code changes, we
considered the corresponding warning as a potential bug. If there is any
disagreement, they flag the commit and the corresponding warning for further
manual verification in the next round. They repeat this process several times
until all warnings and commits have been reviewed.

During our manual inspection, we also audit the following information: 1) True
Positive (TP): the bug described in the warning precisely matches with the bug
reported for the commit; 2) False Positive (FP): the bug reported in the
warning does not match with the bug reported in the commit. False Negative
(FN): a potentially vulnerable warning mistakenly predicted to be a false
alarm. In this paper, we calculate True Positive Rate (TPR) and False Negative
Rate (FNR) as $TPR=\frac{TP}{TP+FN}$ and $FNR=\frac{FN}{FN+TP}$ respectively.

##  IV Experimental Results

In this section, we present and discuss our analysis results to address the
following three research questions.

RQ1 (Detection Capability): How many warnings are reported by the studied
static bug detectors?

RQ2 (Detection Effectiveness): How effective are static bug detectors at
detecting real-world bugs in ML libraries?

RQ3 (Root Cause): What are the root causes of missing real-world bugs in ML
libraries?

For answering these RQs, we execute Flawfinder, RATS, Cppcheck, Infer, and
Clang static analyzer on our dataset of 410 real-world bugs across the four
studied ML libraries. The output of bug detectors, i.e., generated warnings,
are automatically parsed to extract relevant information including bug types,
line numbers, and warning messages. All data and scripts for conducting the
experiments described in this section are accessible to the public.

###  IV-A RQ1: Detection Capability

Experiment setup. To answer this question, we directly run these five bug
detectors on the four ML libraries, we record all the reported bugs (including
the detailed location and CWE information) by these tools for further
analysis. For each of the reported bugs, we further manually check whether
it’s a true bug or a false positive.

TABLE III: Distribution of reported warnings across different ML libraries.

Library | Flawfinder | RATS | Cppcheck | Infer | Clang static analyzer
---|---|---|---|---|---
Mlpack | 3 | 22 | 1 | 0 | 0
MXNet | 10 | 6 | 7 | 0 | 0
PyTorch | 5 | 1 | 2 | 4 | 16
TensorFlow | 50 | 139 | 0 | 2 | 254
Overall | 68 | 168 | 10 | 6 | 270

Table III displays the number of bug types extracted from warnings reported by
each bug detector across the four ML libraries. Specifically, among the four
ML libraries, TensorFlow has the highest number of reported warnings, i.e., in
total 445 warnings Mlpack has the lowest number of reported warnings overall,
with only 26 warnings across all tools. Among the detectors, Clang static
analyzer reports the highest number of warnings for each library. Infer only
reports six warnings.

We further illustrate the top 10 bug types and their distribution in our
curated dataset in Figure 3. The chart shows that the most common bug type is
CWE-190 (Integer Overflow or Wraparound), with 86 occurrences in the studied
ML libraries. The next most common types are CWE-362 (Race Condition), CWE-401
(Missing Release of Resource after Effective Lifetime), and CWE-476 (NULL
Pointer Dereference), with 45, 43, and 34 occurrences, respectively. The least
common bug type in the chart is CWE-122 (Heap-based Buffer Overflow), with
only 16 occurrences.

Table IV shows the number of occurrences of specific CWEs reported by each
tool and the false positives of these tools. The table also shows the ground-
truth information about each CWE type in our curated dataset. Note that, some
ground-truth CWEs are shown with (-), which means we do not have that specific
type of bug in our dataset while the tools have generated warnings with that
specific type. For example, Flawfinder reported 50 instances of CWE-120 on
these four ML libraries while 46 of them are false positives. In addition, all
these reported 50 instances are outside of our experiment dataset. Overall,
the five static analysis tools reported 138 instances of various types of CWEs
in the studied ML libraries. Flawfinder reports the highest number of CWE
types (85) while Infer reports the lowest (5). The most common CWE types are
CWE-120 (classic buffer overflow) and CWE-121 (Stack-based Buffer Overflow),
both of which are reported by Flawfinder and RATS. Other types of CWEs, such
as CWE-362 (Concurrent Execution using Shared Resource with Improper
Synchronization (’Race Condition’)) and CWE-122 (Heap-based Buffer Overflow),
are reported by only a subset of the tools.

1903624014769088351211339125122$20$$40$$60$$80$$86$$45$$43$$34$$27$$26$$25$$18$$17$$16$CWE
identifier# of vulnerabilities Figure 3: The top 10 vulnerability types in our
dataset.

Finding 1: The number of warnings produced by the five static bug detectors
varies significantly and is mainly determined by their detection mechanisms.
While these tools can also generate false positives, and that human review is
still necessary to determine whether reported warnings are real-world bugs
that require attention.

TABLE IV: Distribution of vulnerability types reported by the four static
vulnerability detectors. The warnings generated by Clang static analyzer do
not have a CWE number, so we exclude it in this table. Numbers in the brackets
are false positive warnings. The numbers outside of the brackets are the total
number of generated warnings.

CWE-ID | # True Bugs | Flawfinder | RATS | Cppcheck | Infer | Overall
---|---|---|---|---|---|---
CWE-120 | - | 50 (46) | 0 | 0 | 0 | 50
CWE-121 | 25 | 0 | 27 (26) | 0 | 0 | 27
CWE-122 | 24 | 0 | 8 (5) | 0 | 1 (1) | 9
CWE-362 | 45 | 7 (7) | 2 (2) | 0 | 0 | 9
CWE-327 | - | 7 (7) | 0 | 0 | 0 | 7
CWE-367 | - | 7 (7) | 0 | 0 | 0 | 7
CWE-20 | 7 | 5 (4) | 0 | 0 | 0 | 5
CWE-476 | 34 | 0 | 0 | 5 (5) | 0 | 5
CWE-807 | - | 4 (4) | 0 | 0 | 0 | 4
CWE-126 | - | 3 (3) | 0 | 0 | 0 | 3
CWE-330 | - | 0 | 3 (3) | 0 | 0 | 3
CWE-457 | 1 | 0 | 0 | 0 | 2 (2) | 2
CWE-563 | - | 0 | 0 | 0 | 1 (1) | 1
CWE-667 | - | 0 | 0 | 1 (1) | 0 | 1
CWE-758 | - | 0 | 0 | 1 (1) | 0 | 1
CWE-190 | 86 | 1 (0) | 0 | 0 | 0 | 1
CWE-398 | - | 0 | 0 | 1 (1) | 0 | 1
CWE-119 | - | 1 (1) | 0 | 0 | 0 | 1
CWE-833 | 15 | 0 | 0 | 0 | 1 (1) | 1
CWE-401 | 43 | 0 | 0 | 0 | 0 | 0
CWE-908 | 27 | 0 | 0 | 0 | 0 | 0
CWE-835 | 26 | 0 | 0 | 0 | 0 | 0
CWE-1331 | 18 | 0 | 0 | 0 | 0 | 0
CWE-125 | 17 | 0 | 0 | 0 | 0 | 0
CWE-369 | 10 | 0 | 0 | 0 | 0 | 0
CWE-703 | 4 | 0 | 0 | 0 | 0 | 0
CWE-191 | 3 | 0 | 0 | 0 | 0 | 0
CWE-705 | 3 | 0 | 0 | 0 | 0 | 0
CWE-415 | 3 | 0 | 0 | 0 | 0 | 0
CWE-704 | 2 | 0 | 0 | 0 | 0 | 0
CWE-416 | 2 | 0 | 0 | 0 | 0 | 0
CWE-840 | 2 | 0 | 0 | 0 | 0 | 0
CWE-787 | 2 | 0 | 0 | 0 | 0 | 0
CWE-439 | 2 | 0 | 0 | 0 | 0 | 0
CWE-628 | 1 | 0 | 0 | 0 | 0 | 0
CWE-241 | 1 | 0 | 0 | 0 | 0 | 0
CWE-255 | 1 | 0 | 0 | 0 | 0 | 0
CWE-197 | 1 | 0 | 0 | 0 | 0 | 0
CWE-252 | 1 | 0 | 0 | 0 | 0 | 0
CWE-706 | 1 | 0 | 0 | 0 | 0 | 0
CWE-1006 | 1 | 0 | 0 | 0 | 0 | 0
CWE-561 | 1 | 0 | 0 | 0 | 0 | 0
CWE-475 | 1 | 0 | 0 | 0 | 0 | 0
Overall | 410 | 85 | 40 | 8 | 5 | 138

###  IV-B RQ2: Detection Effectiveness

Experiment setup. To address this question, first two authors of this paper
manually reviewed the generated warnings (bug candidates identified using the
automatic filtering approach explained in subsection III-C1) per file for each
bug fixing commit. The team conducted a thorough analysis of the warning
message, the CWE-ID, and the specific line numbers that can indicate where the
bug was identified. Using this information, they further examined the code
modifications to verify that the warning is indeed related to the bug being
addressed in the particular update. If any discrepancies arise, they
highlighted the corresponding warning and committed to additional examination
in the subsequent review cycle. This process is repeated until all the
warnings and code changes have been assessed.

Results. Table V shows the performance of each static bug detector regarding
True Positive Rate (TPR) and False Negative Rate (FNR) values. As we can see
from the table, Flawfinder has a TPR of 0.049, which means it correctly
identifies only 4.95% of the actual bugs, and an FNR of 0.95, indicating that
it fails to detect 95.05% of actual bugs. RATS has a TPR of 0.030 meaning that
it correctly identifies only 3.07% of the actual bugs, and a FNR of 0.97,
which means it fails to detect 97.01% of the actual bugs. The rest of the
tools have a TPR of 0 which states that they fail to detect any actual bugs,
and a FNR of 1, which means they incorrectly identifies all actual bugs as
negatives. Overall, all the experimented tools have relatively low performance
in terms of identifying actual bugs in ML libraries, as they all have very low
TPR values or fail to detect any real-world bugs at all.

TABLE V: Performance of the studied static vulnerability detectors on the four ML libraries. Tool | TPR | FNR
---|---|---
Flawfinder | 0.04 | 0.95
RATS | 0.03 | 0.97
Cppcheck | 0 | 1
Infer | 0 | 1
Clang static analyzer | 0 | 1
TABLE VI: Vulnerabilities detected by Flawfinder and RATS. Tool | Manifestation (ground-truth) | Root cause (reported by tools) | Library | Filename
---|---|---|---|---
Flawfinder | CWE-125 | CWE-20 | PyTorch | simd.h
Flawfinder | CWE-197 | CWE-122 | PyTorch | decode_padded_raw_op.cc
Flawfinder | CWE-122 | CWE-20 | MXNet | image-classification-predict.cc
Flawfinder | CWE-190 | CWE-190 | MXNet | image_iter_common.h
Flawfinder | CWE-121 | CWE-121 | TensorFlow | cpu_device.cc
RATS | CWE-122 | CWE-121 | MXNet | image-classification-predict.cc
RATS | CWE-190 | CWE-122 | PyTorch | THTensor.cpp
RATS | CWE-197 | CWE-122 | PyTorch | decode_padded_raw_op.cc

Table VI shows the detailed characteristics of bugs detected by Flawfinder and
RATS. We can see that Flawfinder and RATS can identify bugs in different code
repositories, including MXNet, TensorFlow, and PyTorch. The bugs are
categorized according to their types ( i.e., CWE numbers). For example,
Flawfinder detected a CWE-122 bug and a CWE-190 bug in files of MXNet. RATS
also detected a CWE-122 bug in the same file of MXNet, as well as a CWE-190
bug in PyTorch.

Table IV also shows the number of false positive warnings out of all warnings
generated given each true bug type. Specifically, Flawfinder reported a total
of 50 warnings for CWE-120, but only 4 of them were accurate warnings. In
addition, it reported a single warning for CWE-190 that was indeed accurate.
For CWE-20, Flawfinder reported 5 warnings, but only 1 of them was a true
positive. On the other hand, RATS produced 27 warnings for CWE-121, but only 1
of them was a valid warning. For CWE-122, RATS flagged 5 warnings, but all of
them were false positives. All warnings generated by Cppcheck and Infer are
false positives suggesting that they are not effective at detecting real-world
security bugs in the studied ML libraries.

Finding 2: Overall, the effectiveness of the tools is quite poor in
discovering real-world ML software bugs. Flawfinder and RATS, which are the
most effective static checker, discovered 4 unique bugs out of a total of 410
bugs in our dataset.

###  IV-C RQ3: Root Cause of Missing Real-world Bugs

To address this research question, we manually analyzed and reviewed the
documentation of tools as well as their code base. Then, we organized the
reasons why these tools miss detect so many ML bugs.

####  IV-C1 Flawfinder and RATS

Since Flawfinder and RATS manifest very similarly in bug detection,
Flawfinder’s specific reasons also apply to RATS.

Issues with Soundness Strategy. Flawfinder and RATS have a significant
limitation in their approach to bug detection, known as the soundness
strategy. This strategy leads them to flag every C/C++ API call as potentially
buggy, which is a major drawback. Although ML libraries often make extensive
use of C/C++ APIs in their backend implementation, the bugs in these libraries
do not primarily originate from these APIs. Instead, the bugs in ML libraries
tend to exhibit more complex patterns (require a deeper source code analysis
in order to be detected), which are not effectively captured by the simplistic
approach of flagging all API calls as buggy [44, 46, 45, 52, 45, 47, 53, 54].

⬇

escapechar=,

numbers=left,

firstnumber=84,

language = C,

escapeinside={(*}{*)}]

auto input_shape = c->input(0);

auto input_h_shape = c->input(1);

auto seq_length = c->Dim(input_shape, 0);

// assumes rank >= 2

auto batch_size = c->Dim(input_shape, 1);

// assumes rank >= 3

auto num_units = c->Dim(input_h_shape, 2);

Figure 4: An example of heap buffer overflow in TensorFlow library.

Lack of ML-specific Buggy API Information. Even though Flawfinder and RATS
support more than 200 dangerous C/C++ APIs, they are still incapable of
supporting ML-specific buggy APIs. One of the reasons they miss detecting
real-world security bugs in ML libraries is that they do not model library-
specific API information [44, 47]. For example, to detect Memory Leak
(CWE-401) in the Mlpack library, the static detectors need to model
`CleanMemory()` or `delete_mat` which are Mlpack-specific APIs used for
cleaning allocated memories.

Lack of Data Flow and Control Flow Support. Flawfinder and RATS face
challenges in detecting numerous bug patterns specific to machine learning
(ML) because they lack the capability to model control flow and data flow.
Instead, these tools rely on a simplistic keyword-matching approach to
identify potentially dangerous C/C++ API calls and flag them as bugs. This
limitation hinders their effectiveness in capturing the intricacies of ML-
specific bugs [47, 44] that involve complex control and data dependencies. For
example, lack of validation is a complex root cause pattern that is the major
root cause of data type bugs in ML libraries [44]. In this particular bug
pattern, bugs can arise when utilizing client APIs of ML libraries. If the
backend implementation fails to validate or properly handle malicious inputs
or malformed values received through these APIs, bugs can occur. It is crucial
to perform appropriate validation and handling of such inputs to mitigate
potential issues and ensure the robustness and security of the ML library. For
example, Figure 4 is an example of a lack of validation that causes a heap
buffer overflow. In this bug, the code assumes `input_shape` and
`input_h_shape` have a specific rank, while the rank values should be
validated to avoid possible invalid memory access. Flawfinder and RATS are not
able to detect such bugs because they are incapable of modeling data flow
dependency and keeping track of `input_shape` and `input_h_shape` to check if
these variables are validated or not.

####  IV-C2 Cppcheck

Limited Buffer or Stack Overflow Checking Existing work [44] has shown that
the root cause patterns of buffer overflow or stack overflow in ML libraries
are significantly different compared to that of traditional software. For
example, in this
commit999https://github.com/tensorflow/tensorflow/commit/932c4c2364884af52609ea8a86c7232a926d958f
from the TensorFlow library, the root cause of stack overflow is a very big
computation graph of functions and edges. In this example, Each instance of
the class `std::shared_ptr<Function>` holds a collection of Edge objects, and
each Edge object, in turn, holds a `std::shared_ptr<Function>`. Removing a
`std::shared_ptr<Function>` can lead to the cascading deletion of other
`std::shared_ptr<Function>` instances, potentially resulting in a stack
overflow if the graph has a significant depth. However, Cppcheck performs
different strategies to detect buffer overflow and stack overflow bugs which
is not strong enough to detect ML-related overflow bugs. For example, Cppcheck
uses array index checkers to detect buffer overflow which identifies array
index operations and performs various checks to detect potential buffer
overruns associated with array indexing. More specifically, Cppcheck
exhaustively searches for array indexing statements in the code snippet, even
if the code is not reachable, without any control flow analysis. This
technique is incapable of detecting overflow bugs which is due to the large
computation graph mentioned above. The second technique to detect buffer
overflow or stack overflow is to search for C/C++ API calls, similar to
Flawfinder and RATS explained earlier. In this technique, Cppcheck examines
the API scopes and the corresponding arguments, analyzes the buffer or stack
sizes associated with the arguments, and detects potential buffer overflow or
stack overflow bugs based on specified minimum size requirements.

Limited Control Flow Analysis One major limitation of Cppcheck is limited
control flow analysis, while a strong control flow analysis is required to
detect very hard-to-detect bugs In ML libraries, e.g., memory leaks that have
complicated bug patterns. For example, this memory leak
bug101010https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23585 occurs
in the TensorFlow library when decoding malformed PNG image. The memory leak
occurs when certain errors in the function implementation cause the execution
to be abruptly terminated using the `OP_REQUIRES` macro checker. This
termination prevents the proper freeing of allocated buffers stored in the
`decode` value. To release these allocated buffers, the function should call
`png::CommonFreeDecode(&decode)`. However, due to eager termination, the
necessary memory-freeing process is not allowed to occur. This bug is very
hard to detect by Cppcheck for two main reasons. First, the TensorFlow-
specific API `png::CommonFreeDecode(&decode);` is not registered in the
internal database of API symbols which is the firsts step toward detecting
this leak. Second, the control-flow analysis supported by Cppcheck is very
simple and cannot capture the complex flow analysis in this bug pattern.

####  IV-C3 Infer

Limited Buffer Overflow checkers Compared to Cppcheck, Flawfinder, and RATS,
Infer has a stronger buffer overflow checker. Infer uses symbolic intervals to
handle the range of index values and buffer sizes. Typically, interval
analysis involves working with intervals represented as [low, high], where low
and high are constants indicating the lower bound and upper bound of the
target buffer and indexing range. However, this checker is not effective
enough to detect ML-related buffer overflow bugs. For example, Figure 5 shows
an example of hard-to-detect buffer
overflow111111https://github.com/tensorflow/tensorflow/commit/8ee24e7949a203d234489f9da2c5bf45a7d5157d
in the TensorFlow library where providing a tensor with larger dimensions as
the second argument is critical. To address this issue, the developer has
modified the `MatchingDim` function to return the minimum size between the two
dimensions.

⬇

escapechar=,

numbers=left,

firstnumber=84,

language = C,

escapeinside={(*}{*)}]

inline int MatchingDim(const RuntimeShape&

shape1, int index1, const RuntimeShape&

shape2, int index2) {

TFLITE_DCHECK_EQ(shape1.Dims(index1),

shape2.Dims(index2));

return shape1.Dims(index1);

}

Figure 5: An example of hard-to-detect buffer overflow bug in TensorFlow
library.

####  IV-C4 Clang static analyzer

Limited Rules. Although Clang static analyzer provides a large number of
predefined rules (i.e., there are 25 families of rules implemented in the
Clang static analyzer database), it does not cover rules of security bugs that
are relevant to ML libraries. Most of the rules are implemented to find coding
conventions and styling errors such as fuchsia_ which check the correctness of
codes based on Fuchsia coding conventions. There are also two broad categories
of rules for performance and concurrency issues. However, none of them are
specifically designed to find security issues for ML libraries. Concurrency
issues in ML libraries have very sophisticated bug patterns [44] that require
customized ML-specific rules to be implemented in Clang static analyzer.

Limited Sensitivity. Even though the Clang static analyzer supports all
sensitivity analysis factors explained in Table II, there are still some major
issues regarding sensitivity analysis for ML libraries. For example, it may
generate false positive warnings suggesting that a variable is uninitialized
while it is properly initialized. This is because flow analysis in Clang
static analyzer often is not performed efficiently. This flow analysis support
may not be able to catch the bug pattern of uninitialized variables in ML
libraries. Additionally, it may not be able to cover all cases of
uninitialized variables as there may have sophisticated patterns in ML
libraries implementation.

Finding 3: We identified a set of specific reasons that help explain why the
five static bug detectors examined in this paper miss real-world security bugs
in ML libraries including ML software-specific reasons, i.e., Lack of
Implementation Support of Detection Rules, Issues with Soundness Strategy, and
Lack of ML-specific Buggy API Information.

##  V Lessons learned

Our study reveals several interesting findings that can serve as applicable
guidelines for improving static bug detection for ML libraries.

###  V-A Implication for Improving Flawfinder and RATS

We find that Flawfinder and RATS flag every C/C++ API as vulnerable regardless
of whether they are actually vulnerable or not. This behavior introduces many
false alarms IV. In order to reduce the false alarm rate, numerous extensions
to these checkers are required.

Extend the input representation. Flawfinder and RATS represent the source code
as a sequence of tokens. Unlike natural language, source code encodes
structural information [55], which needs to be considered with rich
representation techniques like Abstract Syntax Trees(AST), Control Flow
Graphs(CFG), or Data Flow Graphs(DFG) [56]. Having a control flow graph is
vital in detecting many ML-related security vulnerabilities including buffer
overflow and memory leaks. Control flow graphs allow the tool to model every
possible execution patch inside vulnerable programs. For example, to detect
the stack overflow bug in this commit
121212https://github.com/tensorflow/tensorflow/commit/698bc996f7190f5cd836d48d29b8c1b3ddcd37c2,
the checker needs to traverse control graph to find out that the
implementation of `DataTypeString` function is vulnerable due to missing
`return DataTypeStringInternal(dtype);`.

Extend the pattern matching. The pattern-matching strategy of Flawfinder and
RATS follows naive text-based matching. Text-based matching does not allow the
detectors to perform intraprocedural analysis, i.e. the analysis inside the
boundary of functions or any compilation units. The lack of performing
intraprocedural analysis introduces so many false alarms in the case of
detecting ML security vulnerabilities. For example, in order to detect Memory
Leaks (CWE-401) bug listed in Table VI, the detector needs to perform analysis
inside `_bsplmat()` and catch the return value which is the root cause of this
bug.

###  V-B Implication for Improving Cppcheck

Cppcheck has been shown to be the most effective static checker for detecting
real-world security issues. However, it still lacks a plethora of weaknesses.
Cppcheck may be extended in two ways, as discussed in the subsections below.

Support ML Library-Specific Constructs. In order to extend Cppcheck, the
developers should handle any ML library-specific constructs or macros within
the backend implementation. This may involve extending Cppcheck’s macro
handling capabilities or creating additional checks to handle these constructs
effectively.

Extend the control flow graph. The control flow graph analysis in Cppcheck is
very limited, with the following assumptions: all source code statements can
be accessed, and the state checkers in the if conditions are always either
true or false. As a result, a more advanced control flow graph is required to
be able to detect sophisticated vulnerabilities in ML libraries.

###  V-C Implication for Improving Infer

Improve Buffer overrun Checker. While Infer’s buffer overrun checker can
detect typical buffer overflow patterns, it may not cover all conceivable
variations and attacker approaches in ML libraries. The tool may miss
sophisticated or unique exploitation methods explained in RQ3.

Isolate checkers. To begin with, the rules for identifying integer overflow in
Infer are still in an experimental stage and lack sensitivity to function
arguments. Additionally, Infer’s dependency on buffer overrun rules to detect
integer overflow introduces a limitation. These buffer overrun rules
themselves tend to generate a significant number of false alarms, which
consequently raises the false alarm rate for identifying integer overflow
bugs. Given the intricate nature of the bug pattern for integer overflow in ML
libraries [44], it becomes crucial for developers to separate the buffer
overrun checkers from the scope of integer overflow.

###  V-D Implication for Improving Clang static analyzer

Extend the Checker for Null Pointer Dereference. Clang static analyzer uses
its internal checker core.NullDereference (C, C++, ObjC) to detect null
pointer dereference bugs in C, C++, and Objective C programs. The checker
works well if function arguments take pointer parameters. If the code attempts
to dereference the pointer without checking if it is null, it marks it as
buggy. However, in terms of ML libraries, null pointer dereference has more
sophisticated patterns. For example, in this commit from the TensorFlow
library131313https://github.com/tensorflow/tensorflow/commit/9a133d73ae4b4664d22bd1aa6d654fec13c52ee1,
the developer has removed `int64 id = ctx->session_state()->GetNewId();` since
`session_state()` is vulnerable to have a null value which results in denial
of service via a null pointer dereference. The developers should increase the
null pointer checker to cover more corner cases in terms of code elements that
may have null values.

##  VI Threats to validity

As is the case with every empirical research, there are a few factors that
call into question the reliability of the inferences we have taken from our
data. One of the major limitations is the choice of ML libraries and bug
detectors. In order to protect ourselves against this risk, we have chosen
four extensively used ML libraries, each of which focuses on a different facet
of ML development. All ML libraries are open-source projects that are always
being developed and improved upon. Regarding the selection of static bug
detectors, our major focus is on static detectors that are openly available to
the public and are in the process of being developed right now. We used widely
used and popular detectors which have been cited by many previous studies in
the bug detection areas [3, 50, 51].

Another possible threat to this study is the mapping technique [14] used to
find potential vulnerable candidates. The approach we used in the paper is
subject to coincidental matches. for example, the assumption is that if the
line number produced in warnings overlaps with the modified lines in a bug
fix, the program will automatically recognize the warning as a candidate for
the bug. This is not true in practice since the reported warning may not be
connected to the actual bug in the commit, or the changed line may be
refactoring the code and does not represent the bug being fixed. The ultimate
determination of whether a warning relates to a bug is made by two authors
involved in manual inspection and is therefore subjective. To mitigate this
risk, both authors reviewed every possibility for an identified bug when there
is no clear evidence.

In this paper, we rely on the assumption that a source code before a bug fix
is a buggy source code, and the source code after the fix is considered as the
bug is fixed. One may apply static bug detectors on source code after the fix
and find multiple bugs. As a result, this assumption serves as the foundation
for this paper’s goal of determining how many real-world bugs the tools
identify at the moment of committing modifications.

##  VII Related work

###  VII-A Software Security Vulnerability Detection

There have been numerous studies focused on software vulnerability detection
in the literature [4, 57, 3, 51, 50, 5]. Cao et al. [4] proposed a deep
learning-based vulnerability detection model to detect memory-related
statements on their manually curated dataset extracted from 11 projects
developed in C/C++. Their proposed model can model structural information
which allows the detection of semantic vulnerabilities. Their experiments
indicate that the proposed model is superior compared to cutting-edge models
as well as static analysis tools. Li et al. [57] developed SySeVR, a deep
learning-based vulnerability detector in which syntactic and semantic
information contained in source codes are merged as a rich input
representation and supplied into the model. They think that this unique
representation discovers subtle weaknesses in the source code. SySeVR
surpasses state-of-the-art models and static checkers in detecting many real-
world security vulnerabilities. Cheng et al [3] focused on combining static
detection and deep learning models to detect software security
vulnerabilities. They proposed a novel embedding framework called DeepWukong
in which a graph neural network is used to model semantic code logic via
control and data flow analysis. They gathered more than 100,000 programs
written in C/C++ to evaluate DeepWukon’s capabilities for detecting real-world
software security vulnerabilities. They find that DeepWukong outperforms
existing state-of-the-art models and static analysis tools in finding real-
world security vulnerabilities.

###  VII-B Studies on Static Detection Techniques

Habib and Pradel [14] investigated the static analyzers including Infer,
ErrorProne, and SpotBugs to figure out what percentage of Defect4j’s total
bugs can be located by using the aforementioned tools. Both the code diff and
the bug report mapping approaches are utilized by the authors. Tomassi [58]
They carry out a study in which they examine the similarities and differences
between ErrorProne and SpotBugs in order to determine the total number of bugs
that are discovered in a sample of 320 BugSWARM artifacts. SpotBugs was only
able to locate a single bug, as the author discovered. Rutar et al. [59]
analyzed a small suite of programs with a number of different static
analyzers, including PMD, FindBugs, JLint, Bandera, and ESC/Java 2. The
authors present a taxonomy of bugs discovered by each tool, demonstrating that
none of the tools can be considered to be more comprehensive than the others.
Runtime as well as the total number of warnings generated are the primary
focuses of this study. Chatzieleftheriou and Katsaros [60] focused on common
static analysis techniques for C code security bugs. they proposed a test
suite comprised of common bugs drawn from public databases using four open-
source and two commercial projects. They proposed a set of findings that
assist in the identification of cost-effective tools, and the proposed
methodology and test suite can be used to evaluate static analysis tools in
the future. Lipp et al. [16] argued that there are two main limitations to the
datasets used for testing static bug detectors. Firstly, the datasets do not
accurately represent real-world code and cannot identify new and complex
vulnerability patterns. Secondly, the datasets do not classify vulnerabilities
based on the Common Weakness Enumeration (CWE) mapping. Hence, they attempted
to overcome these limitations by testing static bug detectors on real-world
datasets that were collected from CVE records of 27 different projects,
totaling 1.15 million lines of code. In spite of the fact that static bug
detectors perform well on the synthetic dataset of bugs, they find that the
tools miss a high percentage of bugs in a real-world setting.

Static bug detectors often produce a large number of false alarm warnings
which makes manual inspection problematic. To remove such false alarms,
Several different approaches to the detection of static bugs have been taken
up by the industry. Bessey et al. [61] share their insights gained from the
process of bringing static bug detectors to the market in their paper. Ayewah
et al. [10, 62] discussed the lessons learned via applying FindBug on Google’s
codebase in which numerous engineers involved through thousands of FindBugs
generated warnings, and addressed them by either fixing them or filing
reports. They find that most issues were highlighted for fixing, but only a
few of them were actually causing significant problems in production.

Understanding real-world bugs is a crucial first step in enhancing bug
detection. Several studies have taken different types of bugs into account,
such as those in the Linux kernel [63], bugs in concurrency [64], and bugs in
correctness [65] and performance [66] in JavaScript. The work of Pan et al.
cited as [67], examines fixes for bugs and identifies recurring syntactic
patterns.

Our work is very different from the studies that have been done before [16,
14, 58, 15]. In previous studies, the primary focus has been on the automatic
static detection of general bugs in traditional software projects. On the
other hand, the primary focus of this paper is on the automatic detection of
software bugs in widely used ML projects. For example, Integer Overflow
(CWE-190) is a major bug in ML libraries as found in [44]. A further
distinction lies in the fact that the focus of our research is placed on a
more recent and advanced generation of static vulnerability detectors
specifically designed to detect bugs in projects written in C/C++.

##  VIII Conclusion

This paper addresses the critical task of automatic detection of software bugs
in ML libraries. The study analyzes the effectiveness of five popular and
widely used static bug detectors, namely Flawfinder, RATS, Cppcheck, Facebook
Infer, and Clang static analyzer, on a curated dataset of software bugs
gathered from four popular ML libraries. The research provides a
categorization of these tools’ capabilities, highlighting their strengths and
weaknesses in detecting software bugs. The study reveals that Flawfinder and
RATS are the most effective static checkers for finding security bugs in ML
libraries. However, the overall findings show that the tools detect only a
negligible amount of bugs, accounting for 6/410 (0.01%) of known security
bugs. Based on these observations, the paper also identifies and discusses
opportunities to make the tools more effective and practical.

## References

  * [1] P. Foreman, _Vulnerability management_. CRC Press, 2019.
  * [2] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” _arXiv preprint arXiv:1706.06083_ , 2017.
  * [3] X. Cheng, H. Wang, J. Hua, G. Xu, and Y. Sui, “Deepwukong: Statically detecting software vulnerabilities using deep graph neural network,” _ACM Transactions on Software Engineering and Methodology (TOSEM)_ , vol. 30, no. 3, pp. 1–33, 2021.
  * [4] S. Cao, X. Sun, L. Bo, R. Wu, B. Li, and C. Tao, “Mvd: Memory-related vulnerability detection based on flow-sensitive graph neural networks,” _arXiv preprint arXiv:2203.02660_ , 2022.
  * [5] Z. Li, D. Zou, S. Xu, Z. Chen, Y. Zhu, and H. Jin, “Vuldeelocator: a deep learning-based fine-grained vulnerability detector,” _IEEE Transactions on Dependable and Secure Computing_ , 2021.
  * [6] A. Bessey, K. Block, B. Chelf, A. Chou, B. Fulton, S. Hallem, C. Henri-Gros, A. Kamsky, S. McPeak, and D. Engler, “A few billion lines of code later: Using static analysis to find bugs in the real world,” _Commun. ACM_ , vol. 53, no. 2, p. 66–75, feb 2010. [Online]. Available: https://doi.org/10.1145/1646353.1646374
  * [7] G. inc. Errorprone. [Online]. Available: https://errorprone.info/
  * [8] Facebook. (2013) Infer. [Online]. Available: https://fbinfer.com/
  * [9] SpotBugs. (2021) Spotbugs. [Online]. Available: .https://spotbugs.github.io/
  * [10] N. Ayewah, W. Pugh, D. Hovemeyer, J. D. Morgenthaler, and J. Penix, “Using static analysis to find bugs,” _IEEE software_ , vol. 25, no. 5, pp. 22–29, 2008.
  * [11] B. Johnson, Y. Song, E. Murphy-Hill, and R. Bowdidge, “Why don’t software developers use static analysis tools to find bugs?” in _2013 35th International Conference on Software Engineering (ICSE)_. IEEE, 2013, pp. 672–681.
  * [12] F. Thung, D. Lo, L. Jiang, F. Rahman, P. T. Devanbu _et al._ , “To what extent could we detect field defects? an empirical study of false negatives in static bug finding tools,” in _2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering_. IEEE, 2012, pp. 50–59.
  * [13] ——, “To what extent could we detect field defects? an extended empirical study of false negatives in static bug-finding tools,” _Automated Software Engineering_ , vol. 22, no. 4, pp. 561–602, 2015.
  * [14] A. Habib and M. Pradel, “How many of all bugs do we find? a study of static bug detectors,” in _2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE)_. IEEE, 2018, pp. 317–328.
  * [15] D. A. Tomassi and C. Rubio-González, “On the real-world effectiveness of static bug detectors at finding null pointer exceptions,” in _2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)_. IEEE, 2021, pp. 292–303.
  * [16] S. Lipp, S. Banescu, and A. Pretschner, “An empirical study on the effectiveness of static c code analyzers for vulnerability detection,” in _Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis_ , 2022, pp. 544–555.
  * [17] G. Algan and I. Ulusoy, “Image classification with deep learning in the presence of noisy labels: A survey,” _Knowledge-Based Systems_ , vol. 215, p. 106771, 2021.
  * [18] F. Mahdisoltani, G. Berger, W. Gharbieh, D. Fleet, and R. Memisevic, “Fine-grained video classification and captioning,” _arXiv preprint arXiv:1804.09235_ , vol. 5, no. 6, 2018.
  * [19] R. Patgiri, “A taxonomy on big data: Survey,” _arXiv preprint arXiv:1808.08474_ , 2018.
  * [20] Y. Lv, B. Liu, J. Zhang, Y. Dai, A. Li, and T. Zhang, “Semi-supervised active salient object detection,” _Pattern Recognition_ , vol. 123, p. 108364, 2022\.
  * [21] R. Simhambhatla, K. Okiah, S. Kuchkula, and R. Slater, “Self-driving cars: Evaluation of deep learning techniques for object detection in different driving conditions,” _SMU Data Science Review_ , vol. 2, no. 1, p. 23, 2019\.
  * [22] S. Ramos, S. Gehrig, P. Pinggera, U. Franke, and C. Rother, “Detecting unexpected obstacles for self-driving cars: Fusing deep learning and geometric modeling,” in _2017 IEEE Intelligent Vehicles Symposium (IV)_. IEEE, 2017, pp. 1025–1032.
  * [23] R. Kulkarni, S. Dhavalikar, and S. Bangar, “Traffic light detection and recognition for self driving cars using deep learning,” in _2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA)_. IEEE, 2018, pp. 1–4.
  * [24] S. Minaee and Z. Liu, “Automatic question-answering using a deep similarity neural network,” in _2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)_. IEEE, 2017, pp. 923–927.
  * [25] R. G. Athreya, S. K. Bansal, A.-C. N. Ngomo, and R. Usbeck, “Template-based question answering using recursive neural networks,” in _2021 IEEE 15th International Conference on Semantic Computing (ICSC)_. IEEE, 2021, pp. 195–198.
  * [26] P. K. Roy, “Deep neural network to predict answer votes on community question answering sites,” _Neural Processing Letters_ , vol. 53, no. 2, pp. 1633–1646, 2021.
  * [27] J.-W. Hong, Y. Wang, and P. Lanz, “Why is artificial intelligence blamed more? analysis of faulting artificial intelligence for self-driving car accidents in experimental settings,” _International Journal of Human–Computer Interaction_ , vol. 36, no. 18, pp. 1768–1774, 2020.
  * [28] D. A. Wheeler. (2013) Dlawfinder. [Online]. Available: http://dwheeler.com/flawfinder/
  * [29] J. Chen, C. Zhang, S. Cai, L. Zhang, and L. Ma, “A memory-related vulnerability detection approach based on vulnerability model with petri net,” _Journal of Logical and Algebraic Methods in Programming_ , vol. 132, p. 100859, 2023.
  * [30] J. D. Pereira and M. Vieira, “On the use of open-source c/c++ static analysis tools in large projects,” in _2020 16th European Dependable Computing Conference (EDCC)_. IEEE, 2020, pp. 97–102.
  * [31] C. Mitropoulos, “Employing different program analysis methods to study bug evolution,” in _Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , 2019, pp. 1202–1204.
  * [32] M. Gu, H. Feng, H. Sun, P. Liu, Q. Yue, J. Hu, C. Cao, and Y. Zhang, “Hierarchical attention network for interpretable and fine-grained vulnerability detection,” in _IEEE INFOCOM 2022-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS)_. IEEE, 2022, pp. 1–6.
  * [33] D. Zou, Y. Hu, W. Li, Y. Wu, H. Zhao, and H. Jin, “mvulpreter: A multi-granularity vulnerability detection system with interpretations,” _IEEE Transactions on Dependable and Secure Computing_ , 2022.
  * [34] Y. Zhou, S. Liu, J. Siow, X. Du, and Y. Liu, “Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks,” _arXiv preprint arXiv:1909.03496_ , 2019.
  * [35] A. Dunham. (2009) rough-auditing-tool-for-security. [Online]. Available: https://github.com/andrew-d/rough-auditing-tool-for-security
  * [36] D. Marjamäki. (2016) Cppcheck. [Online]. Available: https://cppcheck.sourceforge.io/
  * [37] S. Pujar, Y. Zheng, L. Buratti, B. Lewis, A. Morari, J. Laredo, K. Postlethwait, and C. Görn, “Varangian: a git bot for augmented static analysis,” in _Proceedings of the 19th International Conference on Mining Software Repositories_ , 2022, pp. 766–767.
  * [38] S. Mehrpour and T. D. LaToza, “Can static analysis tools find more defects? a qualitative study of design rule violations found by code review,” _Empirical Software Engineering_ , vol. 28, no. 1, p. 5, 2023.
  * [39] C. Lattner, “Llvm and clang: Next generation compiler technology,” in _The BSD conference_ , vol. 5, 2008, pp. 1–20.
  * [40] K. Umann and Z. Porkoláb, “Detecting uninitialized variables in c++ with the clang static analyzer,” _Acta Cybernetica_ , vol. 25, no. 4, pp. 923–940, 2022.
  * [41] P. G. Szécsi, G. Horváth, and Z. Porkoláb, “Improved loop execution modeling in the clang static analyzer,” _Acta Cybernetica_ , vol. 25, no. 4, pp. 909–921, 2022.
  * [42] H. Aslanyan, Z. Gevorgyan, R. Mkoyan, H. Movsisyan, V. Sahakyan, and S. Sargsyan, “Static analysis methods for memory leak detection: A survey,” in _2022 Ivannikov Memorial Workshop (IVMEM)_. IEEE, 2022, pp. 1–6.
  * [43] M. Pradel and T. R. Gross, “Detecting anomalies in the order of equally-typed method arguments,” in _Proceedings of the 2011 International Symposium on Software Testing and Analysis_ , 2011, pp. 232–242.
  * [44] N. S. Harzevili, J. Shin, J. Wang, and S. Wang, “Characterizing and understanding software security vulnerabilities in machine learning libraries,” _arXiv preprint arXiv:2203.06502_ , 2022.
  * [45] F. Thung, S. Wang, D. Lo, and L. Jiang, “An empirical study of bugs in machine learning systems,” in _2012 IEEE 23rd International Symposium on Software Reliability Engineering_. IEEE, 2012, pp. 271–280.
  * [46] M. J. Islam, G. Nguyen, R. Pan, and H. Rajan, “A comprehensive study on deep learning bug characteristics,” in _Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , 2019, pp. 510–520.
  * [47] L. Jia, H. Zhong, X. Wang, L. Huang, and X. Lu, “The symptoms, causes, and repairs of bugs inside a deep learning library,” _Journal of Systems and Software_ , vol. 177, p. 110935, 2021.
  * [48] Y. Zhou and A. Sharma, “Automated identification of security issues from commit messages and bug reports,” in _Proceedings of the 2017 11th joint meeting on foundations of software engineering_ , 2017, pp. 914–919.
  * [49] Y. Younan, W. Joosen, and F. Piessens, “Runtime countermeasures for code injection attacks against c and c++ programs,” _ACM Computing Surveys (CSUR)_ , vol. 44, no. 3, pp. 1–28, 2012.
  * [50] Z. Li, D. Zou, S. Xu, X. Ou, H. Jin, S. Wang, Z. Deng, and Y. Zhong, “Vuldeepecker: A deep learning-based system for vulnerability detection,” _arXiv preprint arXiv:1801.01681_ , 2018.
  * [51] X. Duan, J. Wu, S. Ji, Z. Rui, T. Luo, M. Yang, and Y. Wu, “Vulsniper: Focus your attention to shoot fine-grained vulnerabilities.” in _IJCAI_ , 2019, pp. 4665–4671.
  * [52] Q. Xiao, K. Li, D. Zhang, and W. Xu, “Security risks in deep learning implementations,” in _2018 IEEE Security and privacy workshops (SPW)_. IEEE, 2018, pp. 123–128.
  * [53] A. Di Franco, H. Guo, and C. Rubio-González, “A comprehensive study of real-world numerical bug characteristics,” in _2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE)_. IEEE, 2017, pp. 509–519.
  * [54] Q. Shen, H. Ma, J. Chen, Y. Tian, S.-C. Cheung, and X. Chen, “A comprehensive study of deep learning compiler bugs,” in _Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , 2021, pp. 968–980.
  * [55] A. Hindle, E. T. Barr, M. Gabel, Z. Su, and P. Devanbu, “On the naturalness of software,” _Commun. ACM_ , vol. 59, no. 5, p. 122–131, apr 2016. [Online]. Available: https://doi.org/10.1145/2902362
  * [56] F. Yamaguchi, N. Golde, D. Arp, and K. Rieck, “Modeling and discovering vulnerabilities with code property graphs,” in _2014 IEEE Symposium on Security and Privacy_. IEEE, 2014, pp. 590–604.
  * [57] Z. Li, D. Zou, S. Xu, H. Jin, Y. Zhu, and Z. Chen, “Sysevr: A framework for using deep learning to detect software vulnerabilities,” _IEEE Transactions on Dependable and Secure Computing_ , 2021.
  * [58] D. A. Tomassi, “Bugs in the wild: examining the effectiveness of static analyzers at finding real-world bugs,” in _Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , 2018, pp. 980–982.
  * [59] N. Rutar, C. B. Almazan, and J. S. Foster, “A comparison of bug finding tools for java,” in _15th International symposium on software reliability engineering_. IEEE, 2004, pp. 245–256.
  * [60] G. Chatzieleftheriou and P. Katsaros, “Test-driving static analysis tools in search of c code vulnerabilities,” in _2011 IEEE 35th annual computer software and applications conference workshops_. IEEE, 2011, pp. 96–103.
  * [61] A. Bessey, K. Block, B. Chelf, A. Chou, B. Fulton, S. Hallem, C. Henri-Gros, A. Kamsky, S. McPeak, and D. Engler, “A few billion lines of code later: using static analysis to find bugs in the real world,” _Communications of the ACM_ , vol. 53, no. 2, pp. 66–75, 2010.
  * [62] N. Ayewah and W. Pugh, “The google findbugs fixit,” in _Proceedings of the 19th international symposium on Software testing and analysis_ , 2010, pp. 241–252.
  * [63] A. Chou, J. Yang, B. Chelf, S. Hallem, and D. Engler, “An empirical study of operating systems errors,” in _Proceedings of the eighteenth ACM symposium on Operating systems principles_ , 2001, pp. 73–88.
  * [64] S. Lu, S. Park, E. Seo, and Y. Zhou, “Learning from mistakes: a comprehensive study on real world concurrency bug characteristics,” in _Proceedings of the 13th international conference on Architectural support for programming languages and operating systems_ , 2008, pp. 329–339.
  * [65] F. Ocariza, K. Bajaj, K. Pattabiraman, and A. Mesbah, “An empirical study of client-side javascript bugs,” in _2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement_. IEEE, 2013, pp. 55–64.
  * [66] M. Selakovic and M. Pradel, “Performance issues and optimizations in javascript: an empirical study,” in _Proceedings of the 38th International Conference on Software Engineering_ , 2016, pp. 61–72.
  * [67] K. Pan, S. Kim, and E. J. Whitehead, “Toward an understanding of bug fix patterns,” _Empirical Software Engineering_ , vol. 14, no. 3, pp. 286–315, 2009.