# Rethink Diversity in Deep Learning Testing

Zi Wang  Department of Computer Sciences
University of Wisconsin-Madison
Madison, USA
<EMAIL_ADDRESS>Jihye Choi  Department of Computer Sciences
University of Wisconsin-Madison
Madison, USA
<EMAIL_ADDRESS>Somesh Jha  Department of Computer Sciences
University of Wisconsin-Madison
Madison, USA
<EMAIL_ADDRESS>
###### Abstract

Deep neural networks (DNNs) have demonstrated extraordinary capabilities and
are an integral part of modern software systems. However, they also suffer
from various vulnerabilities such as adversarial attacks and unfairness.
Testing deep learning (DL) systems is therefore an important task, to detect
and mitigate those vulnerabilities. Motivated by the success of traditional
software testing, which often employs diversity heuristics, various diversity
measures on DNNs have been proposed to help efficiently expose the buggy
behavior of DNNs. In this work, we argue that many DNN testing tasks should be
treated as directed testing problems rather than general-purpose testing
tasks, because these tasks are specific and well-defined. Hence, the
diversity-based approach is less effective.

Following our argument based on the semantics of DNNs and the testing goal, we
derive $6$ metrics that can be used for DNN testing and carefully analyze
their application scopes. We empirically show their efficacy in exposing bugs
in DNNs compared to recent diversity-based metrics. Moreover, we also notice
discrepancies between the practices of the software engineering (SE) community
and the DL community. We point out some of these gaps, and hopefully, this can
lead to bridging the SE practice and DL findings.

###### Index Terms:

software testing, deep learning, gradient methods

##  I Introduction

DL systems have demonstrated remarkable performance across various machine
learning tasks and also revolutionized modern society [1, 2]. More recently,
large language models [3] and stable diffusion models [4] have shown
impressive, or even superhuman capabilities on many generative tasks. Despite
the wide application of DL systems nowadays, their decision process remains
quite black-box and less explainable to humans, and they are also shown to be
vulnerable to various bugs [5, 6, 7]. With the wide deployment of DL systems,
especially in some safety-critical applications, it is important to ensure the
security and trustworthiness of those systems. However, DL systems are also
software artifacts, thus amenable to SE approaches.

Testing, one of the classical SE techniques, has been practiced for decades to
detect erroneous behaviors of traditional software [8]. By serving numerous
inputs to the programs, testing can expose bugs when the execution deviates
from the specification. Moreover, the inputs can serve as concrete examples
for developers to demonstrate the existence of defects and to understand and
debug the program.

In traditional software testing, tools often employ the adaptive testing
scenario: a repeated loop involving input mutation, program execution, and
evaluation of fitness [9, 10]. Inputs with higher fitness scores are
prioritized or given more mutation resources for the next loop iterations. The
goal is to expose bugs fast and this fitness quantifies how likely an input
can expose bugs. As a result, to design an effective testing framework, an
important task is to create a good metric that can prioritize potential
defective inputs. Motivated by the idea of adaptive testing, to extend its
utility toward detecting DNN bugs, a natural question arises as follows:

What is a good metric that can prioritize potential erroneous inputs for DNNs?

There are two types of testing frameworks: _diversity_ -based testing and
_directed_ testing. Diversity-based testing relies on the assumption that
diversified tests can execute the program thoroughly, and often expose corner
cases that are overlooked by developers. This methodology is suitable for
general-purpose testing, where tools aim to expose as many bugs as possible
[9, 8]. On the other hand, the goal of directed testing is more specific; such
as detecting a _particular type of bug_ rather than any bug. Driven by the
specific goal, directed testing prioritizes inputs that are more potential to
expose particular bugs rather than inputs that improve the “diversity” [11,
10]. For example, standard fuzz testing prioritizes tests that trigger unseen
coverage. However, if the tool is to identify concurrency errors, test
executions that are close to the potential data-race code in the program
should be considered more promising to expose concurrency bugs [12].

In recent years, researchers have proposed methods to test DL systems. Many of
them are motivated by traditional software testing practices, in particular,
to encourage the diversity of inputs. One of the common diversity measures in
software testing is code coverage of execution from the inputs [8].
Researchers were motivated by this measure and defined similar metrics for
DNNs. They treated neurons as the basic computational components, similar to
basic blocks in programs, and defined diversity metrics in terms of neuron
activation patterns [13], namely, Neuron Coverage (NC). However, later works
argued that NC is _not_ a meaningful metric [14]. Till these days, the usage
of neuron activation diversity remains controversial: some recent works re-
confirm that NC does not bring significant advantage [15] while some others
still design diversity metrics based on neuron activations [16].

Contrary to existing works that design different diversity metrics for DL
testing, we argue that many DL testing tasks in the literature [13, 16] should
be treated as directed testing tasks because they are well-defined and
specific. We refer to these bugs under detection as _functionality_ bugs
because these bugs violate the DNN functionality. Also, the mathematical
semantics of DNNs are simple compared to traditional software, so one can
quantify how likely one input is to change the DNN’s functionality, and this
provides a measure for testing. Therefore, test generation should be guided by
this directed measure rather than a diversity measure.

Moreover, we suggest that the testing can be even accelerated by exploiting an
intrinsic nature of DL systems: differentiability. The differentiability of
DNNs allows us to derive fast and precise linear approximations in a small
neighborhood using gradients. As we will explain, state-of-the-art
$\ell_{p}$-adversarial attacks from DL literature, projected gradient descent
(PGD) [17], can also be categorized into our testing framework. Moreover, we
provide a careful mathematical analysis of gradient-based methods, which
validates these methods for $\ell_{p}$-transformations, but also _casts doubt_
on their applications to other types of transformations.

In fact, we notice that there are discrepancies between the practices in the
SE and DL communities. Even though DL systems are also programs and many SE
techniques can be applied, DL has also been studied extensively outside SE due
to its capability and popularity. Many practices in both communities are
similar at a high level but still different in many ways. We point out some of
these gaps and hopefully, this work can help bridge some of the gaps.

To summarize, we make the following contributions:

  * •

We provide a novel conceptual framework for many DL testing tasks, and
demonstrate that some state-of-the-art DL practices are the natural
consequences of this framework. We believe that our work can help clarify the
controversial DNN testing topic, i.e., NC maximization; and bridge the gaps
between practices in the SE and DL communities. In fact, we point out that the
NC maximization metric generates out-of-distribution (OOD) inputs using the DL
glossary.

  * •

To instantiate our framework, we provide several concrete testing metrics with
careful mathematical analysis. Our unified testing framework allows us to
handle various types of input transformations, and past works were unable to.
Moreover, our analysis validates the application of gradients on
$\ell_{p}$-adversarial attacks but also calls for rigorous scrutiny of
gradient methods on other types of input transformations.

  * •

We empirically validate our framework with several carefully designed
experiments Moreover, we also compare our framework with several diversity-
based methods. The result shows that our metrics are more effective in
exposing the functionality bugs among all settings, and also reveals that the
diversity metrics are weak indicators for bugs under small
$\ell_{p}$-perturbations.

##  II background

###  II-A Mathematical notations and definitions

Let $[n]=\\{1,\ldots,n\\}$. For a vector $v\in\mathbb{R}^{n}$,
$\text{sign}(v)\in\\{-1,0,1\\}^{n}$ such that

$\text{sign}(v)_{i}=\left\\{\begin{array}[]{ll}1,&\mbox{$v_{i}>0$}\\\
0,&\mbox{$v_{i}=0$}\\\ -1,&\mbox{$v_{i}<0$}\end{array}\right.$

$A^{T}$ denotes the transpose of a matrix $A$. Let $||v||_{p}$ denote the
$\ell_{p}$ norm of a vector $v$:

$||v||_{p}=\sqrt[\scriptstyle p]{\sum_{i=1}^{n}|v_{i}|^{p}}.$

The canonical Euclidean norm is then $||v||_{2}$, and another commonly
considered norm on the input is the $\ell_{\infty}$-norm:

$||v||_{\infty}=\max_{i\in[n]}{|v_{i}|}.$

For two functions $f$ and $g$, $f\circ g(x)=f(g(x))$ denotes the composition
of $f$ and $g$. If $f:\mathbb{R}^{n}\rightarrow\mathbb{R}$ is differentiable,
we use $D^{i}f$ to denote the $i$-th order differentiation of partial
derivatives, e.g., $D^{1}f$ is the gradient of $f$, and $D^{2}f$ is the
Hessian of $f$. As a convention, we omit $1$ when we denote gradient, i.e., we
use $Df$ to denote the gradient of $f$.

For an infinitely differentiable multivariate function
$f:\mathbb{R}^{n}\rightarrow\mathbb{R}$, the Taylor expansion $T$ of $f$ at
$a\in\mathbb{R}^{n}$ is

$T(a+\Delta x)=f(a)+(\Delta x)^{T}Df(a)+\frac{1}{2}(\Delta
x)^{T}D^{2}f(a)(\Delta x)+\ldots,$

Therefore, $T(a+\Delta x)$ is an infinite series. If $f$ is analytic, then
$T(a+\Delta x)$ converges to $f(a+\Delta x)$ [18]. In this work, we only work
with the first two terms of $T$, i.e., we only consider

$f(a)+(\Delta x)^{T}Df(a).$ (1)

Given the information of $f$ at $a$, i.e., $f(a)$ and $Df(a)$, Equation 1 is
the linear approximation of $f$ at $a+\Delta x$. This can provide an
estimation of $f(a+\Delta x)$ if we do not want to evaluate $f$ at $a+\Delta
x$ directly, i.e., $f(\Delta x+a)\approx f(a)+(\Delta x)^{T}Df(a)$.

######  Remark II.1.

The quality of linear approximation depends on how large $x$ is, and how close
$f$ is to a linear function.

###  II-B Software testing

Large software artifacts have been an integral part of modern society for
decades and are evolving on a constant basis. Despite its ubiquity, software
usually contains various defects and some of them can raise serious security
concerns. Testing remains one of the most efficient ways to demonstrate the
presence of bugs. Since modern software is complicated, bugs can occur
anywhere in the program. For example, a buffer usage in the program could
potentially introduce overflow and a pointer operation could lead to a NULL
pointer dereference.

Due to the omnipresence of risky operations in the program, automated testing
techniques are commonly guided by the diversity heuristics: a diverse set of
inputs can execute the program thoroughly and therefore result in more bugs
revealed [8]. One of the diversity measures in testing is code coverage, which
can have multiple granularities such as function coverage, basic block
coverage, and path coverage. For example, the famous fuzzing tool, American
Fuzzy Lop (AFL) is a state-of-the-art general-purpose fuzzer that employs the
coverage maximization heuristics [9]. With a loop involving input mutation,
execution, and evaluation of coverage-based diversity metric, AFL generates
inputs aiming to execute different parts of the program and has successfully
detected numerous bugs in various applications.

###  II-C DNNs

In this work, we mainly focus on feed-forward DNNs, but our methodology is
quite general and architecture agnostic. Here we discuss some basics of DNNs
that are needed to elaborate our testing approach.

A feed-forward DNN $f:\mathbb{R}^{m}\rightarrow\mathbb{R}^{l}$ is a
composition of affine transformations and non-linear activation functions:

$f_{1}(x)=W^{(1)}x+b^{(1)};\;f_{i}(x)=W^{(i)}\sigma(x)+b^{(i)},i=2,\ldots,d.$

where $W^{(i)}\in\mathbb{R}^{n_{i+1}\times n_{i}}$ is the weight matrix
between the layers, $n_{1}=m$ and $n_{d+1}=l$, $d$ is the depth of the
network, and $b^{(i)}\in\mathbb{R}^{n_{i+1}}$ is the bias term. $\sigma$, the
activation, is an element-wise non-linear function. $f=f_{d}\circ\cdots\circ
f_{1}$.

######  Remark II.2.

In general, a neural network is highly non-linear. Therefore, for the linear
approximation to be precise, we would need $\Delta x$ to be small in Equation
1.

In this work, we only consider the standard classification tasks.
$f:\mathbb{R}^{m}\rightarrow\mathbb{R}^{l}$ has $l$ outputs. Let $f^{(i)}$ be
the $i$-th output of $f$. The classification of an input $x$ is
$\mathcal{C}(f,x)=\operatorname*{arg\,max}_{k\in[l]}f^{(k)}(x)$. Suppose that
the prediction of $x$ is $i$, then $f^{(i)}(x)>f^{(j)}(x)$ for all $j\neq i$.
The output of $f$ is called the logit score, and the classification model
outputs the class with the highest logit score.

Compared with traditional software, which might contain complicated structures
like pointer aliasing, various data structures and indirect function calls,
DNNs are numerical programs with simple mathematical semantics. Therefore, the
techniques designed to handle the complexity in traditional software might not
be directly applicable to DNNs. In particular, we want to point out that
diversity-based methods in software testing are not the ideal testing
framework for many DNN testing tasks.

###  II-D DNN testing

The functionality of DNNs is to classify inputs correctly aligned with human
perception. Therefore, functionality bugs in the classifier setting should be
incorrect classification, and the testing goal is to generate inputs that are
misclassified. However, obtaining tests and manually labeling them is usually
a very expensive process. Researchers have proposed metamorphic testing [19],
which augments existing inputs with metamorphic relations. Metamorphic testing
generates new inputs by transforming existing ones while preserving their
labels. For image inputs, many transformations do not change human judgment,
such as small perturbations, image rotation, zooming, and brightness change.
We refer to these transformations as semantics-preserving transformations.
They can augment the original dataset by applying those transformations on
labeled inputs. If any of those transformed inputs can alter the DNN’s
prediction, then we identify misclassified inputs. In this work, we focus on
identifying misclassified inputs, which we refer to as _functionality
testing_. Rather than manually collected and labeled, these inputs are
generated from a few compositions of small semantics-preserving
transformations. Many of the existing works can be categorized into
functionality testing [16, 13, 5, 6], to change the DNN prediction by applying
semantics-preserving transformations on the inputs.

###  II-E Diversity in DNN testing

Inspired by the success of traditional software testing, researchers have
devised various diversity measures for DL system testing. One of the first
diversity measures is NC [13], similar to the code coverage in traditional
software testing, which measures how many neurons are activated above a
specified threshold $k$. The assumption is that the more neurons are
activated, the more states of DNNs are explored. Later works refined the
definition of NC and proposed several structural variants of NC, including the
neuron boundary coverage and strong neuron activations [20]. More recently,
[16] proposed to use intermediate layer activation divergence between
metamorphic inputs as an indicator to detect prediction violations, arguing
that inputs with different intermediate activation patterns should also have
divergent predictions. [21] proposed a measure to compute the fault pattern of
inputs based on the normalized logit score, and a test selection algorithm
that chooses a subset of inputs with diverse fault patterns and higher
prediction uncertainty.

###  II-F Directed software testing

Unlike general-purpose software testing tools such as AFL, directed testing
prioritizes inputs that are close to the specific testing goal. For example, a
data-race testing tool should prioritize inputs that are more likely to
trigger concurrency bugs rather than inputs with unseen coverage [12]. In
software testing, this proximity can often be measured statically by distance
in code. For example, inputs that trigger concurrency bugs necessarily execute
code where data race can occur, so the proximity can be measured by the
distance between the execution trace and those race condition targets in the
program. Mutating inputs close to those targets can generate inputs even
closer to the targets and eventually reach the target and potentially trigger
specific bugs [10].

Researchers have developed several directed testing tools, and shown that they
are more capable of detecting specific bugs than general-purpose testing tools
[11]. In fact, we argue that functionality testing of DNNs should be
considered as directed testing tasks conceptually, because bugs within these
tasks are well-defined and specific.

##  III Main thesis

In this section, we elaborate on the main thesis of this work:

Functionality testing of DNNs is a directed testing task.

This is because the mathematical semantics of DNNs is simple and the testing
goal becomes specific under the semantics. This thesis motivates us to devise
a few concrete testing metrics. All of them are directness metrics for
functionality testing, and the application scope of each metric depends on the
underlying input transformation. In fact, we want to highlight that each
metric has its own application scope, and misusing them outside their scope
can degrade the performance.

###  III-A Forward fitness

We use $x$ to denote a human-labeled input, and let $x^{\prime}$ be a
generated input from a few small semantics-preserving transformations, similar
to input mutation in traditional software testing. We fix a DNN under test:
$f$. Let $i\in[l]$ be $x$’s label. We assume that $f$ classifies $x$
correctly, i.e., $C(f,x)=i$. The goal of testing is to identify the buggy
$x^{\prime}$ so that the $f$’s classification over $x^{\prime}$ will change,
i.e., $C(f,x^{\prime})\neq C(f,x)$. We provide two different interpretations
of bugs, each of which can provide a testing metric.

######  Remark III.1.

Here we assume that $x$ is classified correctly by $f$, i.e., $C(f,x)$ matches
the ground truth for $x$. We make this assumption for the following reasons:

  1. 1.

Modern network architectures have high capabilities in data fitting and can
usually achieve high test accuracy.

  2. 2.

Since our goal is to alter the prediction of $f$ on $x^{\prime}$, if we
accomplish it, at least one of $x$ or $x^{\prime}$ is a bug. This is similar
to the metamorphic testing methodology [16].

#### Margin-based score

$C(f,x)=i$ implies $f^{(i)}(x)>f^{(j)}(x),\forall j\in[l],j\neq i$. We define

$g_{ji}=f^{(j)}-f^{(i)}.$

where $g_{ji}(x)<0$, $\forall j\neq i$, because $C(f,x)=i$. To change the
prediction of $f$ on $x^{\prime}$ that is close to $x$ semantically, we need
$f^{(j)}(x^{\prime})>f^{(i)}(x^{\prime})$ for some $j\in[l],j\neq i$. Hence,
the fitness function for functionality testing could be

$\max_{j\neq i}(f^{(j)}(x^{\prime})-f^{(i)}(x^{\prime}))=\max_{j\neq
i}g_{ji}(x^{\prime}),$ (2)

i.e., any transformed input $x^{\prime}$ that has the greatest value of
$\max_{j\neq i}(f^{(j)}(x^{\prime})-f^{(i)}(x^{\prime}))$ is considered the
most promising one to expose the bug. Intuitively, the transformed input with
a higher margin difference between other classes and the correct label is
considered better and has a greater chance to alter the prediction. We refer
to Equation 2 as the _margin score_.

#### Loss-based score

When the network is trained, a loss is used to measure the distance between
the logit score $f(x)$ and the ground truth $i$. The training process
minimizes the loss to fit the data to the label, i.e., if this measure is
small, the $x$ is likely to be classified correctly. This naturally provides a
measure for the bug: if the loss is big, then $x$ is likely to be
misclassified. In classification tasks, the conventional loss is the cross-
entropy function [22, chapter 3]:

$CE(f(x^{\prime}),i)=-\log\frac{e^{f^{(i)}(x^{\prime})}}{\sum_{j=1}^{n}e^{f^{(j)}(x^{\prime})}}.$
(3)

We refer to this score as the _loss score_.

###  III-B Gradient based fitness

To collect the logit score of input $x^{\prime}$, one has to execute the DNN
on $x^{\prime}$. This can be expensive, especially when there are many input
transformations and the model is large. However, one distinguishable
characteristic of DL systems compared to traditional software is that neural
networks are differentiable. Recall that Equation 1 defines a linear
approximation of a function via gradients. This allows us to quickly estimate
the values of the functions without executing the neural networks.

More formally, suppose we have an input $x$ and the transformed input
$x^{\prime}$, we can define $\Delta x=x^{\prime}-x$. From Equation 1, we know
that $f(x^{\prime})\approx f(x)+(\Delta x)^{T}Df(x)$. This is particularly
useful when the number of transformations is huge and even infinite in some
cases. We compute the gradient with the original input once, then for every
transformation, only need to compute the difference in the input domain.

#### Approximating margin score

For the margin score in Equation 2, the quantity we want to estimate is
$g_{ji}(x^{\prime})$. We can compute the the value of $g_{ji}$ at $x$,
$g_{ji}(x)$, and the gradient at $x$, $Dg_{ji}(x)$, then compute the quantity:

$\bar{g}_{ji}(x^{\prime})=g_{ji}(x)+(\Delta x)^{T}Dg_{ji}(x).$

$\bar{g}_{ji}(x^{\prime})$ is a linear approximation of
${g}_{ji}(x^{\prime})$. Instead of identifying the input $x^{\prime}$ that has
the greatest value of $\max_{j\neq
i}(f^{(j)}(x^{\prime})-f^{(i)}(x^{\prime}))$, we choose the input $x^{\prime}$
that has the greatest value of

$\max_{j\neq i}(\bar{g}_{ji}(x^{\prime})).$ (4)

We refer to this score as the _backward margin score_ , because we need the
gradient to compute this approximation, and gradient computation requires
backpropagation over the computation graph.

#### Approximating loss score

To approximate Equation 3 at $x^{\prime}$, we have

$\overline{CE}(f(x^{\prime}),i)=CE(f(x),i)+(\Delta x)^{T}\cdot D(CE)(f(x),i),$
(5)

where $D(CE)(f(x),i)$ is the gradient of the cross-entropy function at $x$. We
refer to this score as the _backward loss score_.

#### Computational efficiency

Using the linear approximation surrogate is particularly beneficial when the
number of transformations is huge, and the number of model parameters is much
larger than the dimensions of the inputs, which is always true for modern
network architecture. Suppose there are $N$ transformations for an input, a
forward execution takes $F$ time, a backward gradient execution takes $B$
time, which is usually similar to $F$, and the inner-product computation takes
$I$ time. To compute the forward fitness score, we need $N$ forward
executions, which take $NF$ time. To compute the backward fitness score, we
need $1$ forward execution to compute $f(x)$ and $1$ backward execution to
compute the gradient $Df(x)$, and then $N$ inner-product computations to
compute $(\Delta x)^{T}D(f)(x)$, which is usually very cheap compared to DNN
executions. Brought together, the backward fitness score takes $F+B+NI$ time
to finish.

#### Analytical solution

The backward gradient surrogate can be more valuable when the transformation
set is continuous and has simple nice geometry. Because the set is continuous,
theoretically, there are infinitely many transformations in the set.
Therefore, forward-executing all transformed inputs becomes infeasible.
However, if we use the gradient to approximate the function value, we can
sometimes derive the optimal transformation among all possible ones to
maximize the linear approximation Equation 1 analytically. For instance, if
the transformation set is $\ell_{2}$ or $\ell_{\infty}$-transformations, the
geometry of the sets is either Euclidean balls or hypercubes. To maximize the
gradient inner product, one only needs to project the gradient to the
$\ell_{p}$-balls. Let $h$ be the gradient. If the perturbation set is an
$\ell_{2}$-ball with radius $\epsilon$, the projection of $h$ inside the ball
is:

$\frac{h}{||h||_{2}}*\epsilon;$ (6)

if the perturbation set is an $\ell_{\infty}$-ball with radius $\epsilon$, the
projection of $h$ is:

$\epsilon*\text{sign}(h).$ (7)

This is the core idea of PGD, the state-of-the-art $\ell_{p}$-attacks, where
the gradient comes from the loss-based score, similar to Equation 5. Thus,
when using gradient methods and the underlying transformation includes small
$\ell_{p}$-perturbations, we also use the gradient to generate the analytical
solutions as in Equations 6 and 7 to test DNN.

######  Remark III.2.

In practice, PGD is a parameterizable multi-step adaptive projection of the
gradients. On $\ell_{\infty}$-transformations, the one-step gradient
projection coincides with the Fast Gradient Sign Method (FGSM) [6].

###  III-C Mixed score

Recall that in Remark II.2, if $\Delta x$ is big, the approximation quality
from gradients is poor. Therefore, we can also define a $\Delta x$-dependent
metric such that: when $\Delta x$ is big, we use the forward score Equation 2;
and when $\Delta x$ is small, we consider the linear approximation. Thus we
can define

$MM(x^{\prime})=\left\\{\begin{array}[]{ll}\max_{j\neq
i}(g_{ji}(x^{\prime})),&\mbox{$||\Delta x||_{p}>\epsilon$}\\\ \max_{j\neq
i}(\bar{g}_{ji}(x^{\prime})),&\mbox{$||\Delta
x||_{p}\leq\epsilon$}\end{array}\right.$ (8)

We refer to Equation 8 as the _mixed margin score_. Similarly, for loss score,
we have:

$ML(x^{\prime})=\left\\{\begin{array}[]{ll}CE(f(x^{\prime}),i),&\mbox{$||\Delta
x||_{p}>\epsilon$}\\\ \overline{CE}(f(x^{\prime}),i),&\mbox{$||\Delta
x||_{p}\leq\epsilon$}\end{array}\right.$ (9)

We refer to Equation 9 as the _mixed loss score_. Notice that the condition
decision in Equations 8 and 9 is a cheap operation. We only need to compute
$\Delta x=x^{\prime}-x$ and find the $\ell_{p}$-norm of all $\Delta x$. If we
only consider small $\ell_{p}$ transformations, mixed scores correspond to
their gradient-score counterparts; and if all the transformation is large,
mixed scores coincide with their forward-fitness counterparts.

######  Remark III.3.

The gradient-based scores are useful when the additive difference between the
transformed input and the original input is small, and especially when the set
of transformations forms a nice geometrical space. However, if the difference
is large, then gradients provide imprecise estimations. In this case,
following our directness argument, a better score is from the logits,
especially when the number of inputs is tractable. However, gradient-based
scores can still provide fast estimations, albeit the precision is
questionable.

##  IV evaluation

We empirically evaluate our claims made in Section III: 1) the directness
metrics are effective in exposing bugs in functionality testing; 2) each
metric is effective within their application scope and can have performance
degradation outside the scope. Because the application scope is defined by
whether the underlying transformation allows all small
$\ell_{p}$-perturbations, we will evaluate the performance with different
types of transformations separately. In particular, several recent SE testing
works only studied non-$\ell_{p}$ input transformation scenario [16, 21], and
we use them as benchmarks. Our evaluation can also provide more understanding
of their performance and application scope. More specifically, we want to
answer the following research questions:

RQ1: How similar are the metrics to rank the inputs? RQ2: Are the fitness
scores induced from our framework better than the diversity metric to expose
bugs in DNNs when not all small $\ell_{p}$-transformations are allowed? RQ3:
Are the fitness scores induced from our framework better than the diversity
metric to expose bugs in DNNs when all $\ell_{p}$-transformations are allowed?

Notice that RQ1 only asks for the correlations between different metric
functions. This does not reflect what metrics are better for detecting the DNN
bugs. However, for most testing frameworks, the ordinal relations of inputs
are more important than the absolute values of scores. The correlation
measurement helps us understand how similar the metrics are in different input
transformation regimes.

###  IV-A Input transformations

We consider two sets of transformations on the input.

  1. 1.

The $\ell_{p}$ transformations, also known as the adversarial attacks: the
transformations are from $\ell_{\infty}$-balls or $\ell_{2}$-balls with small
specified radii $\epsilon_{\infty}$ and $\epsilon_{2}$. The radii of
transformations are specified according to recent adversarial robustness works
[17, 23, 24].

  2. 2.

Common image transformations such as zooming, and blurring, which we refer to
as _benign transformations_ as in [21].

Notice that the two transformations are not exclusive to each other: some
benign transformations might also be $\ell_{p}$-transformations. The
difference is that for $\ell_{p}$-transformations, we would allow all possible
small additive $\ell_{p}$-perturbations. Notice that because the
$\ell_{p}$-balls are continuous spaces, there are infinitely many
transformations in theory. In practice, even if we discretize the space, the
number of transformations is exponentially many with respect to the input
dimension. As a result, if all $\ell_{p}$-transformations are allowed, for
non-gradient methods, we can only sample the transformations within the
$\ell_{p}$-balls; for gradient methods, we can project the gradient to the
$\ell_{p}$-balls directly as described in Equations 6 and 7.

We also report the average $\ell_{2}$-norm of $\Delta x$ for all the
transformations on the test dataset. We choose $\ell_{2}$-norm instead of
$\ell_{\infty}$-norm because the input space and inner product used in linear
approximation Equation 1 are naturally equipped with the $\ell_{2}$-norm.
Also, the $\ell_{2}$-norm result can reflect the difference in
$\ell_{\infty}$-norm.

####  $\ell_{p}$-transformations specifications

We use radius $0.08$ for the $\ell_{2}$ transformations, radius $0.01$ for the
$\ell_{\infty}$. Because we will test DNNs adaptively for $5$ iterations, the
choice of radius can ensure that the commonly practiced
$\ell_{p}$-perturbations in adversarial robustness literature are reachable
within $5$ iterations.

#### Benign-transformations specifications

Our benign transformations include $7$ types of transformations: shift, zoom,
brightness, rotation, shearing, blur and contrast ratio as in [21]. The
parameters are selected such that all the transformations in the evaluation of
[21] are reachable within $5$ iterations.

###  IV-B Experimental design

#### RQ1

We fix randomly selected transformations for each input in the test set, and
then rank the transformations based on various fitness measures. We consider
three types of transformations separately: (i) one with only
$\ell_{p}$-perturbations; (ii) one with only benign transformations; (iii) one
with both adversarial and benign transformations.

For $\ell_{p}$-transformations, we randomly select $14$
$\ell_{\infty}$-perturbations and $14$ $\ell_{2}$-perturbations. For benign
transformations, we generate $4$ transformations for each of the seven benign
transformations as in [21]. For mixed transformations, we combine all $28$
$\ell_{p}$-perturbations and all $28$ benign transformations together as the
transformation candidates.

For every set of transformations, each fitness metric produces a ranked list
of transformations, where the top ones are assumed to be more likely to expose
bugs in the DNN. We then compute the rank similarities between the lists. For
example, given a fixed set of inputs $(x_{1},x_{2},\ldots,x_{n})$, each metric
prioritizes them to generate a ranked list. We compare the two ranked lists to
study the correlation between pairs of metrics. We use a well-established
metric, Rank Biased Overlap (RBO) [25], to measure this similarity between a
pair of ranked lists. The RBO score between two lists is the weighted average
of overlaps between all top sublists from both lists, in which the weight
takes the rank into account. The higher the score is, the more similar the two
ranked lists are. One can then use RBO to measure how similar the two ranked
lists are to study the correlation of the fitness metrics.

#### RQ2

We apply the adaptive testing scenario to evaluate the metrics, i.e., a
repeated loop involving mutation, feedback, and evaluation of fitness [10].
For each iteration, we consider all transformations over the inputs and then
prioritize the transformed inputs according to various fitness scores. We
retain the top input as the seed input for the next transformation round. We
will record the accuracy induced from the retained input in each iteration, up
to $5$ iterations. This multi-iteration adaptive test is also similar to the
PGD attack for $\ell_{p}$-perturbations [17].

In this experiment, we only allow benign transformations. This is similar to
the evaluation in recent testing works [21, 16]. In particular, this can help
us understand whether our directness metrics are effective in bug detection
compared to existing works within their evaluation scope under the standard
adaptive testing scenario.

#### RQ3

We will conduct the same experiments as RQ2’s experiments, except that the
underlying transformations are either $\ell_{p}$ transformations or including
both $\ell_{p}$ and benign transformations. The rationale is that human
judgment is invariant to both small perturbations and usual image
transformations, so we should consider all of them, especially if an adversary
can utilize all semantics-preserving transformations. Because the goal of the
experiment is to evaluate the bug-finding capability of different fitness, for
non-gradient methods over $\ell_{p}$-perturbations, their capability of bug-
detection heavily relies on the number of sampled transformations. We,
therefore, sample $140$ $\ell_{p}$-transformations instead of $28$ in the
evaluation of RQ1.

###  IV-C Experimental specifications

#### Datasets

We use three standard datasets: (1) SVHN [26] contains $10$ classes
representing digits, $73257$ inputs for training, $26032$ digits for testing.
(2) CIFAR10 [27] contains $60,000$ $32\times 32$ colour images in $10$
classes. There are $50000$ training images and $10000$ test images. (3)
CIFAR100 [27] is similar to CIFAR10, but has $100$ classes containing $600$
images each. There are $500$ training images and $100$ testing images per
class.

#### Model architectures

We use two classical architectures from the computer vision community for our
experiments: VGG-16 [28], and ResNet-9 and ResNet-18 [29]. More experimental
specifications can be found in the supplementary materials.

#### Benchmarks

We use three recent diversity-based testing works as benchmarks:

  1. 1.

Boosting Diversity: [16] is our major benchmark. It is built upon the
assumption that if the divergence between two executions is significant, then
the inputs might be classified differently. More specifically, for the pair of
executions, one extracts neuron activations from any hidden layer in the DNN,
and then compute the distributional difference of them between the original
input and the transformed input. Transformed inputs with a great difference
should be prioritized. As noted in [16], as one uses features extracted at the
deeper layer of DNNs, the prioritization accuracy increases, but the execution
time increases. To find the balance between these two, the standard practice
is to choose the layer in the middle of the target DNN. We follow this
guideline but also include a variant as a stronger benchmark; we use the logit
layer as the representation layer, which, as the authors indicated, provided
better prioritization estimation than the mid-intermediate layer.

  2. 2.

NC: [20] monitors and gauges the intermediate neuron activities covered by
current executions at various granularity levels. They profile the range of
neuron outputs into $k$-multisections of the major function region and corner-
cases regions during the training, then identify the ratio of sections covered
under test-time executions. In [16], where NC is considered as one baseline of
diversity measures, they adopt the original metric in [20] to compute the
ratio of uncovered neurons given two executions; that is, the larger the
value, the more diverse the two executions in terms of the NC patterns. We
follow this modified NC measure as one of our baselines.

  3. 3.

ATS: [21] is a recently proposed metric for input selection, i.e., choosing a
small yet diverse subset of inputs from a large set. It computes a diversity
metric from the normalized logit layer, similar to ours. It is not designed
for our testing scenario, i.e., for every single input, choosing the best
candidate among all transformed inputs. However, we adapt it to our scenario
by feeding an input and its transformed mutants to the selection algorithm one
at a time and then rank the inputs based on their metric. We include this
variant as another benchmark.

We will use the following abbreviations for the methods considered in the
evaluation: FM stands for forward margin score (Equation 2); FL stands for
forward loss score (Equation 3); BM stands for backward margin score (Equation
4); BL stands for backward margin score (Equation 5); MM stands for mixed
margin score (Equation 8); ML stands for mixed loss score (Equation 9); BD
stands for the standard tool from [16], BD-F stands for a variant that uses
the logit layer to compute the divergence. NC stands for NC method using the
layer in the middle of DNN architecture, ATS stands for the adapted tool from
[21].

To summarize, our metrics (FM, FL, BM, BL, MM, ML), ATS, and BD-F only utilize
the logit layer, while BD and NC use the intermediate layer neuron
activations. We choose the same intermediate layer for both BD and NC. We do
not include evaluations of ATS on the CIFAR100 dataset since the fault pattern
computation is cubic to the number of classes, which is too slow with 100
classes.

##  V Results and discussion

In this section, we present selected experimental results due to space limits
and discussions on the results. More results can be found in supplementary
materials.

Data, Model, Accuracy |  Transformations |  Norm |  BM&BL |  FM&BM |  FL&BL |  FL&BD |  FL&BD-F |  FL&ATS |  FL&NC
---|---|---|---|---|---|---|---|---|---
SVHN, |  LP |  0.20 |  0.88 |  0.94 |  0.89 |  0.40 |  0.41 |  0.59 |  0.40
ResNet-9, |  Benign |  8.56 |  0.95 |  0.61 |  0.68 |  0.71 |  0.75 |  0.74 |  0.68
95.51% |  Mixed |  4.29 |  0.94 |  0.69 |  0.65 |  0.67 |  0.69 |  0.70 |  0.65
CIFAR10, |  LP |  0.20 |  0.79 |  0.87 |  0.74 |  0.49 |  0.50 |  0.56 |  0.48
VGG, |  Benign |  11.34 |  0.96 |  0.60 |  0.57 |  0.76 |  0.81 |  0.74 |  0.71
91.26% |  Mixed |  5.77 |  0.92 |  0.69 |  0.64 |  0.73 |  0.76 |  0.71 |  0.70
CIFAR100, |  LP |  0.20 |  0.85 |  0.86 |  0.80 |  0.52 |  0.52 |  – |  0.52
VGG, |  Benign |  11.59 |  0.96 |  0.62 |  0.56 |  0.75 |  0.75 |  – |  0.69
65.98% |  Mixed |  5.89 |  0.94 |  0.72 |  0.64 |  0.72 |  0.72 |  – |  0.69

TABLE I: Similarity measures (RBO) between ranked lists according to various
prioritization metrics. The first column specifies the model, dataset and
accuracy on the test dataset. The second column specifies what transformations
are used for the experiment. The third column norm measures the average
$\ell_{2}$-norm of all transformations. The rest of the columns measure the
similarity between pairs of metric scores. We highlight cases where numbers
stand for negative correlations.

###  V-A RQ1

#### RBO Benchmark

The range of the RBO score is within $[0,1]$, and a high score indicates that
two ranked lists are positively correlated and similar. We benchmark $3$ RBO
scores: (i) two same-ranked lists; (ii) two randomly ranked lists; (iii) two
opposite-ranked lists. All lists are of length $28$. With these benchmarks, we
can interpret the results more precisely. The RBO benchmarks are:

  1. 1.

Two same rank lists: $1$;

  2. 2.

Two randomly ranked lists: $0.52$;

  3. 3.

Two oppositely ranked lists: $0.33$.

As a result, if the RBO score is less than $0.52$, the two ranks are
negatively correlated.

The RBO scores between different ranking methods are summarized in Table I.

#### Observations

1\. The forward loss and margin scores are always very positively correlated;

2\. Forward scores and backward scores are strongly positively correlated when
the underlying transformation is only $\ell_{p}$-transformations; but for
benign transformations, the norms induced from $\Delta x$ are fairly large,
and the correlations between forward and backward scores are much weaker.

3\. Most of the scores are above $0.52$, except for BD(-F) & FL and NC & FL in
some $\ell_{p}$-transformation-only regime.

#### Implications

1\. Both margin and loss scores can quantify how a DNN makes predictions, so
their rankings of inputs to be classified incorrectly are very similar.

2\. Gradient-based scores are excellent surrogates for forward scores only
when the underlying transformations are small perturbations.

3\. All the scores can be more efficient to expose functionality bugs than
random prioritization. The only exception is that using BD with only
$\ell_{p}$ transformations. When applying small perturbations on the input,
the difference measurement in the intermediate layer activation is not aligned
with their loss change. This raises concerns about how accurate it is to use
intermediate-layer divergence as a surrogate to measure the potential
prediction change when the transformations are only small $\ell_{p}$-changes,
which was not studied in [16].

Data, Model, Accuracy, Transformations |  Fitness |  Iter 1 |  Iter 2 |  Iter 3 |  Iter 4 |  Iter 5
---|---|---|---|---|---|---
|  FM |  8.41% |  1.53% |  0.76% |  0.59% |  0.53%
|  FL |  8.41% |  1.55% |  0.80% |  0.60% |  0.52%
CIFAR10, |  BM |  71.45% |  55.24% |  41.98% |  33.90% |  27.93%
VGG, |  BL |  72.09% |  56.62% |  43.96% |  35.55% |  30.03%
91.26% |  BD |  19.64% |  10.46% |  9.07% |  8.9% |  8.9%
Benign |  BD-F |  12.08% |  3.77% |  2.41% |  1.88% |  1.68%
|  ATS |  35.41% |  24.62% |  21.12% |  19.84% |  18.92%
|  NC |  42.51% |  37.96% |  36.29% |  35.59% |  34.98%

TABLE II: Adaptive testing of each metric: the first column specifies the
data, model, clean accuracy, and underlying data transformations. The second
column specifies which metric is used for testing. The rest of the columns are
the accuracy of tests prioritized by each metric during each iteration in
adaptive testing. We highlight the best numbers for each iteration.

###  V-B RQ2

Selected results are summarized in Table II.

#### Observations

1\. Our forward scores are the strongest testing metrics to expose
functionality bugs in all experiments.

2\. Recently proposed diversity methods, BD(-F) and ATS are also competitive
metrics to test DNNs. Moreover, as suggested by [16], using the divergence in
the final layers is a stronger metric than the intermediate layer. In fact,
BD-F is the best metric without our forward scores.

3\. The gradient-based scores are weaker than recent diversity-based methods,
though they also consistently decrease the accuracy.

######  Remark V.1.

[16] is designed for metamorphic testing, where the ground truth labels are
assumed unknown. In our metrics, we assumed the access to ground-truth labels
as in the adversarial robustness literature [6, 17]. In the supplementary
materials, we also include a study of the metamorphic variant of our metrics,
and the result is consistent with the ground-truth label case. This indicates
that the metamorphic testing considered in [16] can also be more effectively
addressed with our directed testing approach.

#### Implications

1\. Because ATS and BD-F both use the logit layers as our scores, comparisons
with them establish the main thesis that the functionality testing of DNNs is
directed, and the bug potential can be measured by our scores.

2\. Our theoretical analysis demonstrates that gradients are not a good
estimation when the transformation induces a large difference to the inputs.
This is consistent with our empirical observation, that gradient-based metrics
are quite weak for benign transformations. As a result, when applying
gradients to test networks, one should quantify the change introduced by the
transformation. Indeed, gradient methods were applied to test neural networks
beyond the $\ell_{p}$-transformations, and many of them are in the natural and
programming languages domains [30, 31]. Researchers consider semantics
preserving transformations to natural language or programs, and devised
attacks and defenses using gradients against those vulnerabilities. These
transformations are similar to benign transformations in our evaluation and
there were no justifications that the changes induced by the transformations
to the network input are small. The observation from our work raises concern
about the effectiveness of these attacks and defenses. As a result,

We call for rigorous scrutiny of gradient-based tests on non-$\ell_{p}$ types
of input transformations, especially discrete tractable transformations.

###  V-C RQ3

Selected results are summarized in Table III.

Data, Model, Accuracy, Transformations |  Fitness |  Iter 1 |  Iter 2 |  Iter 3 |  Iter 4 |  Iter 5
---|---|---|---|---|---|---
|  FL |  90.22% |  89.17% |  88.1% |  86.99% |  85.57%
CIFAR10, |  BM |  60.08% |  26.52% |  8.45% |  1.92% |  0.34%
VGG, |  BL |  52.99% |  18.98% |  5.02% |  1.27% |  0.4%
91.26% |  BD |  91.18% |  91.08% |  91.04% |  90.87% |  90.68%
LP |  BD-F |  91.12% |  90.93% |  90.88% |  90.75% |  90.44%
|  ATS |  91.10% |  90.92% |  90.83% |  90.61% |  89.82%
|  NC |  91.19% |  91.18% |  91.17% |  91.12% |  91.11%
|  FM |  7.98% |  1.27% |  0.67% |  0.52% |  0.51%
|  BM |  63.24% |  39.13% |  25.45% |  17.98% |  13.33%
CIFAR10, |  BL |  57.44% |  28.86% |  14.74% |  8.52% |  5.18%
VGG, |  MM |  6.99% |  0.73% |  0.09% |  0.01% |  0.0%
91.26% |  ML |  5.21% |  0.68% |  0.30% |  0.17% |  0.07%
Mixed |  BD |  17.39% |  9.69% |  9.44% |  9.45% |  9.46%
|  BD-F |  13.59% |  5.69% |  4.69% |  4.36% |  4.11%
|  ATS |  32.69% |  24.1% |  21.63% |  20.72% |  20.33%
|  NC |  39.69% |  38.13% |  37.30% |  37.04% |  36.63%

TABLE III: Adaptive testing of each metric: the first column specifies the
data, model, clean accuracy, and underlying data transformations. The second
column specifies which metric is used for testing. The rest of the columns are
the accuracy of tests prioritized by each metric during each iteration in
adaptive testing. We highlight the best numbers for each iteration.

#### Observations

1\. When only $\ell_{p}$-transformations are presented, the best metrics are
gradient-based methods. In cases where both $\ell_{p}$-transformation and
benign transformations are allowed, our mixed scores are the most effective
metrics.

2\. The BD(-F) scores are quite weak when only $\ell_{p}$-perturbations are
allowed, even among all the non-gradient metrics.

#### Implications

1\. From RQ1’s result, we can see that gradient surrogates are very positively
correlated. Moreover, the linear approximation allows directly deriving the
optimal transformation for $\ell_{p}$-perturbations. This observation is
consistent with our theoretical analysis and validates the application of
gradient methods for small $\ell_{p}$-transformations.

2\. From RQ1’s result, when only $\ell_{p}$-transformations are allowed, the
correlation between our loss score FL and BD(-F) scores is in fact a little
negative. In the experiment, BD(-F) induce a weak result for testing, even
compared with our non-gradient forward loss score. These results show that the
neuron divergence can be subtle when the input transformations are small
$\ell_{p}$-perturbations. These scenarios were not evaluated in [16]. Our
evaluation brings new knowledge on the application scope of [16].

3\. In the mixed transformation setting, our mixed scores induce the strongest
metric. Our results on various types of transformations indeed show that our
metrics are superior to existing diversity-based metrics in detecting bugs
within their application scopes.

##  VI Related Work

In this section, we further discuss some related topics on DNN testing. In
particular, we want to emphasize some discrepancies between SE practices and
DL findings. This brings awareness of the gaps and hopefully can lead to
bridging them.

#### Neuron activation

One of the first DNN testing works was inspired by the code coverage in
traditional software testing practice to define NC [13]. However, the inherent
difference between software and DNN challenges the direct application of the
idea from one literature to the other. In software, execution trace usually
has logical and qualitative semantics, i.e., the execution trace is decided by
the logical conditions in the program. In contrast, the neuron activation in
DNNs is quantitative, and its semantics is not as clear as in software.
Therefore, maximizing NC does not provide clear interpretations as in
traditional software.

Nonetheless, NC does provide rich information about the execution of the
inputs. In DL society, neuron activation is referred to as _feature
representation_ , and the researchers have exploited the information from
feature representations for various purposes. For instance, higher-level
explanations about DNN behaviors can be abstracted out of feature
representations, which are more user-friendly than simply inspecting neuron
outputs [32, 33, 34]. Moreover, [35] illustrated that feature representations
can provide an effective metric for human perceptual measurement. They defined
a metric based on normalized neuron activation and found that it is strongly
aligned with human judgment. This leads to the design of defense against more
realistic and unseen threats [36].

[16] relies on the assumption that if the neuron representation of two
metamorphic inputs on an intermediate layer is very divergent, then their
predictions are also likely different. While the reversed argument always
holds that different predictions must have divergent intermediate
representations, the original assumption is not necessarily true. Especially
how to measure the difference is unclear when the divergence is subtle, and
this can lead to distinctive prioritization of inputs as in the
$\ell_{p}$-transformation evaluations. However, similar ideas in DL
communities are used to design OOD detectors. OOD tests are inputs that are
far from the distribution the DNN has seen during the training (i.e., in-
distribution), hence it is undefined behavior for DNN to predict given those
tests. Assuming that in-distribution and OOD inputs have distinctive feature
attribution patterns, there have been various approaches to exploiting
statistical information from feature representations to distinguish in-
distribution vs. OOD inputs [37, 38], and understand the characteristics of
OOD inputs [34]. Consequently, bringing the OOD detection perspective to DNN
testing, we suggest that the NC maximization would make the feature
representations of generated inputs deviate from the training data, which are
actually OOD. This is consistent with the finding in [14] that the inputs from
maximizing the NC score are not natural. To this end, one needs to define the
scope of bugs a priori to avoid unnecessary execution against OOD inputs.

#### Gradient

We carefully analyzed the application of gradients in DL testing. Gradients
naturally exist for many mathematical functions, including neural networks.
They quantify the local variability of a function and can provide the optimal
linear approximation in a small neighborhood. Our gradient-based metrics are
the applications of linear approximation and some of them coincide with the
adversarial attack algorithms in DL literature, which indicates that these
algorithms can be viewed as concrete instantiations of our framework for small
additive transformations. Because traditional software is mostly not
differentiable, gradients are less often used in software testing works.
However, the gradient-based methods have been successful in many DL
applications, encompassing $\ell_{p}$-adversarial attacks and gradient-based
explanations [39, 40]. We believe that the SE practice can also utilize
gradients as tools for testing DNNs, such as recent works on numerical defects
in DL programs [41, 42].

#### Bugs

In this work, we restrict our scope to functionality bugs, i.e., inputs that
are misclassified according to human perception. However, DL bugs can manifest
in various forms. In [7], the authors categorize model bugs into three types:
model contamination, data contamination, and test-time contamination. Model
contamination is caused by errors in the model parameters, for example,
incorrectly initialized parameters; data contamination comes from defects in
the training data; and test-time contamination is caused by shifts in test
data. Functionality bugs can be viewed as test-time contamination: our data
are transformed at test time. However, there are many more types of bugs in
the DL system to be addressed and SE techniques can be applied. For example,
model contamination is often caused by API misuses, which has been studied in
many SE works [43, 44].

#### Diversity

We claim that merely increasing diversity is not the ideal framework for DNN
functionality testing. However, we do not claim that diversity is useless for
DL testing. It depends on our testing goal. A natural diversity issue of DL
systems is fairness and bias [45, 46]: Between subgroups, DL systems contain
performance discrepancies and make biased decisions towards a certain
subgroup. This would also cause concerns for the deployment of DL systems
because DNN classifications can discriminate certain subgroups. If the tests
were only from a subgroup, then potential fairness and bias issues would be
concealed. Therefore, designing meaningful diversity measures can help expose
and alleviate the bias and fairness issues in DL systems.

With these DL lessons, a natural question arises:

What should be treated as bugs for DL systems?

We leave this as an open problem. We believe that if bugs are defined
properly, then directed testing is more effective to detect the bugs.

##  VII Threats to validity

#### Internal validity

Our subject of study is the DNN, whose training dynamics have many nuances.
The selection of datasets, architectures, optimizers, and hyperparameters
might not be optimal for all situations. However, we conduct the experiments
in accordance with the best practice to our knowledge. Moreover, the numbers
reported in this work are averaged over tens of thousands of inputs, which can
help reduce randomness. Our benchmarks come from the most up-to-date works,
and the scale of our experiments is also larger than theirs [21, 16]. For
example, for each input, we consider up to over a hundred transformations, but
the transformations considered in [21, 16] are significantly smaller.

#### External validity

We restrict the scope of our work to testing functionality bugs and claim that
this task is directed. This does not imply that all DL testing tasks are
directed and our work can be used to detect those bugs. One needs to first
define what are bugs and then consider our work if these bugs fall in our
scope.

##  VIII Conclusion

In this paper, we argue that functionality testing should be addressed using
the directed testing framework because they are well-defined and specific.
Moreover, we propose several concrete metrics to quantify the potential
faultiness of inputs and carefully characterize their scopes. We find that
metrics perform drastically differently across input transformations and call
for careful scrutiny of these methods.

## References

  * [1] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Commun. ACM_ , vol. 60, no. 6, p. 84–90, may 2017. [Online]. Available: https://doi.org/10.1145/3065386
  * [2] T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, “Distributed representations of words and phrases and their compositionality,” in _Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2_ , ser. NIPS’13. Red Hook, NY, USA: Curran Associates Inc., 2013, p. 3111–3119.
  * [3] OpenAI, “Gpt-4 technical report,” 2023.
  * [4] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 10 684–10 695.
  * [5] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in _2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2014. [Online]. Available: http://arxiv.org/abs/1312.6199
  * [6] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6572
  * [7] J. Adebayo, M. Muelly, I. Liccardi, and B. Kim, “Debugging tests for model explanations,” in _Proceedings of the 34th International Conference on Neural Information Processing Systems_ , ser. NIPS’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
  * [8] P. Godefroid, “Fuzzing: Hack, art, and science,” _Commun. ACM_ , vol. 63, no. 2, p. 70–76, jan 2020. [Online]. Available: https://doi.org/10.1145/3363824
  * [9] M. Zalewski, “american fuzzy lop,” https://lcamtuf.coredump.cx/afl/, 2022\.
  * [10] Z. Wang, B. Liblit, and T. W. Reps, “TOFU: target-orienter fuzzer,” _CoRR_ , vol. abs/2004.14375, 2020. [Online]. Available: https://arxiv.org/abs/2004.14375
  * [11] M. Böhme, V.-T. Pham, M.-D. Nguyen, and A. Roychoudhury, “Directed greybox fuzzing,” in _Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security_ , ser. CCS ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 2329–2344. [Online]. Available: https://doi.org/10.1145/3133956.3134020
  * [12] D. R. Jeong, K. Kim, B. Shivakumar, B. Lee, and I. Shin, “Razzer: Finding kernel race bugs through fuzzing,” in _2019 IEEE Symposium on Security and Privacy (SP)_ , 2019, pp. 754–768.
  * [13] K. Pei, Y. Cao, J. Yang, and S. Jana, “Deepxplore: Automated whitebox testing of deep learning systems,” _Commun. ACM_ , vol. 62, no. 11, p. 137–145, oct 2019. [Online]. Available: https://doi.org/10.1145/3361566
  * [14] F. Harel-Canada, L. Wang, M. A. Gulzar, Q. Gu, and M. Kim, “Is neuron coverage a meaningful measure for testing deep neural networks?” in _Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , ser. ESEC/FSE 2020\. New York, NY, USA: Association for Computing Machinery, 2020, p. 851–862. [Online]. Available: https://doi.org/10.1145/3368089.3409754
  * [15] S. Pavlitskaya, S. Yikmis, and J. Zollner, “Is neuron coverage needed to make person detection more robust?” in _2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)_. Los Alamitos, CA, USA: IEEE Computer Society, jun 2022, pp. 2888–2896.
  * [16] X. Xie, P. Yin, and S. Chen, “Boosting the revealing of detected violations in deep learning testing: A diversity-guided method,” in _Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering_ , ser. ASE ’22. New York, NY, USA: Association for Computing Machinery, 2023. [Online]. Available: https://doi.org/10.1145/3551349.3556919
  * [17] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” in _International Conference on Learning Representations_ , 2018.
  * [18] W. Rudin, _Principles of Mathematical Analysis_. McGraw - Hill Book C., 1986. [Online]. Available: https://books.google.com/books?id=frdNAQAACAAJ
  * [19] S. Chen, S. Jin, and X. Xie, “Validation on machine reading comprehension software without annotated labels: A property-based method,” in _Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , ser. ESEC/FSE 2021. New York, NY, USA: Association for Computing Machinery, 2021, p. 590–602. [Online]. Available: https://doi.org/10.1145/3468264.3468569
  * [20] L. Ma, F. Juefei-Xu, F. Zhang, J. Sun, M. Xue, B. Li, C. Chen, T. Su, L. Li, Y. Liu, J. Zhao, and Y. Wang, “Deepgauge: Multi-granularity testing criteria for deep learning systems,” in _Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering_ , ser. ASE ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 120–131. [Online]. Available: https://doi.org/10.1145/3238147.3238202
  * [21] X. Gao, Y. Feng, Y. Yin, Z. Liu, Z. Chen, and B. Xu, “Adaptive test selection for deep neural networks,” in _2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE)_. IEEE, 2022, pp. 73–85.
  * [22] I. Goodfellow, Y. Bengio, and A. Courville, _Deep Learning_. MIT Press, 2016, http://www.deeplearningbook.org.
  * [23] K. Leino, Z. Wang, and M. Fredrikson, “Globally-robust neural networks,” in _International Conference on Machine Learning (ICML)_ , 2021.
  * [24] M. N. Mueller, F. Eckert, M. Fischer, and M. Vechev, “Certified training: Small boxes are all you need,” in _The Eleventh International Conference on Learning Representations_ , 2023.
  * [25] W. Webber, A. Moffat, and J. Zobel, “A similarity measure for indefinite rankings,” _ACM Trans. Inf. Syst._ , vol. 28, no. 4, nov 2010. [Online]. Available: https://doi.org/10.1145/1852102.1852106
  * [26] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” in _NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011_ , 2011. [Online]. Available: http://ufldl.stanford.edu/housenumbers/nips2011_housenumbers.pdf
  * [27] A. Krizhevsky, G. Hinton _et al._ , “Learning multiple layers of features from tiny images,” 2009.
  * [28] K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” 2015.
  * [29] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016, pp. 770–778.
  * [30] J. Ebrahimi, A. Rao, D. Lowd, and D. Dou, “HotFlip: White-box adversarial examples for text classification,” in _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)_. Melbourne, Australia: Association for Computational Linguistics, Jul. 2018, pp. 31–36. [Online]. Available: https://aclanthology.org/P18-2006
  * [31] J. Henkel, G. Ramakrishnan, Z. Wang, A. Albarghouthi, S. Jha, and T. Reps, “Semantic robustness of models of source code,” in _2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)_ , 2022, pp. 526–537.
  * [32] B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas _et al._ , “Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav),” in _International conference on machine learning_. PMLR, 2018, pp. 2668–2677.
  * [33] A. Ghorbani, J. Wexler, J. Y. Zou, and B. Kim, “Towards automatic concept-based explanations,” _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [34] J. Choi, J. Raghuram, R. Feng, J. Chen, S. Jha, and A. Prakash, “Concept-based explanations for out-of-distribution detectors,” _arXiv preprint arXiv:2203.02586_ , 2022.
  * [35] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 586–595.
  * [36] C. Laidlaw, S. Singla, and S. Feizi, “Perceptual adversarial robustness: Defense against unseen threat models,” in _International Conference on Learning Representations_ , 2021.
  * [37] K. Lee, K. Lee, H. Lee, and J. Shin, “A simple unified framework for detecting out-of-distribution samples and adversarial attacks,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [38] J. Raghuram, V. Chandrasekaran, S. Jha, and S. Banerjee, “A general framework for detecting anomalous inputs to dnn classifiers,” in _International Conference on Machine Learning_. PMLR, 2021, pp. 8764–8775.
  * [39] M. T. Ribeiro, S. Singh, and C. Guestrin, “" why should i trust you?" explaining the predictions of any classifier,” in _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , 2016, pp. 1135–1144.
  * [40] M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in _International conference on machine learning_. PMLR, 2017, pp. 3319–3328.
  * [41] M. Yan, J. Chen, X. Zhang, L. Tan, G. Wang, and Z. Wang, “Exposing numerical bugs in deep learning via gradient back-propagation,” in _Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering_ , ser. ESEC/FSE 2021\. New York, NY, USA: Association for Computing Machinery, 2021, p. 627–638. [Online]. Available: https://doi.org/10.1145/3468264.3468612
  * [42] L. Li, Y. Zhang, L. Ren, Y. Xiong, and T. Xie, “Reliability assurance for deep neural network architectures against numerical defects,” in _45th International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, 14-20 May 2023_. IEEE/ACM, 2023.
  * [43] M. Wen, Y. Liu, R. Wu, X. Xie, S.-C. Cheung, and Z. Su, “Exposing library api misuses via mutation analysis,” in _2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE)_ , 2019, pp. 866–877.
  * [44] H. J. Kang and D. Lo, “Active learning of discriminative subgraph patterns for api misuse detection,” _IEEE Transactions on Software Engineering_ , vol. 48, no. 8, pp. 2761–2783, 2022.
  * [45] C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in _Proceedings of the 3rd innovations in theoretical computer science conference_ , 2012, pp. 214–226.
  * [46] T. Bolukbasi, K.-W. Chang, J. Zou, V. Saligrama, and A. Kalai, “Man is to computer programmer as woman is to homemaker? debiasing word embeddings,” in _Proceedings of the 30th International Conference on Neural Information Processing Systems_ , ser. NIPS’16. Red Hook, NY, USA: Curran Associates Inc., 2016, p. 4356–4364.
  * [47] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6980

The organization of the appendix is as follows: in Section -A, we provide
additional experimental specifications.

Table IV contains the complete result for RQ1.

Table V contains the complete result for RQ2.

Table VI contains the complete result for RQ3. Notice that when the number of
transformations is huge, our adapted ATS is too slow for the SVHN dataset,
which has many test inputs compared to CIFAR10 and CIFAR100. Hence, we did not
include it in SVHN with $\ell_{p}$-transformations because we sampled $140$
$\ell_{p}$-transformations.

Section -B provides additional experimental results on the metamorphic testing
as in [16].

###  -A Additional experimental specifications

#### Server specification

All the experiments are run on a workstation with forty-eight Intel® Xeon®
Silver 4214 CPUs running at 2.20GHz, and 258 GB of memory, and eight Nvidia
GeForce RTX 2080 Ti GPUs. Each GPU has 4352 CUDA cores and 11 GB of GDDR6
memory.

#### Training specifications

We use batch size $500$, the ADAM optimizer [47] with learning rate $0.001$.
We train the networks for $100$ epochs. To ensure reproducibility, we also
explicitly specify seeds for all random functions.

Data, Model, Accuracy |  Transformations |  Norm |  BM&BL |  FM&BM |  FL&BL |  FL&BD |  FL&BD-F |  FL&ATS |  FL&NC
---|---|---|---|---|---|---|---|---|---
SVHN, |  LP |  0.20 |  0.86 |  0.92 |  0.86 |  0.52 |  0.53 |  0.59 |  0.52
VGG, |  Benign |  8.56 |  0.95 |  0.62 |  0.59 |  0.72 |  0.75 |  0.72 |  0.68
95.49% |  Mixed |  4.29 |  0.93 |  0.72 |  0.68 |  0.69 |  0.70 |  0.68 |  0.67
SVHN, |  LP |  0.20 |  0.88 |  0.94 |  0.89 |  0.40 |  0.41 |  0.59 |  0.40
ResNet-9, |  Benign |  8.56 |  0.95 |  0.61 |  0.68 |  0.71 |  0.75 |  0.74 |  0.68
95.51% |  Mixed |  4.29 |  0.94 |  0.69 |  0.65 |  0.67 |  0.69 |  0.70 |  0.65
CIFAR10, |  LP |  0.20 |  0.79 |  0.87 |  0.74 |  0.49 |  0.50 |  0.56 |  0.48
VGG, |  Benign |  11.34 |  0.96 |  0.60 |  0.57 |  0.76 |  0.81 |  0.74 |  0.71
91.26% |  Mixed |  5.77 |  0.92 |  0.69 |  0.64 |  0.73 |  0.76 |  0.71 |  0.70
CIFAR10, |  LP |  0.20 |  0.88 |  0.92 |  0.87 |  0.46 |  0.47 |  0.60 |  0.46
ResNet-9, |  Benign |  11.34 |  0.95 |  0.59 |  0.65 |  0.75 |  0.77 |  0.76 |  0.70
90.05% |  Mixed |  5.77 |  0.95 |  0.68 |  0.64 |  0.73 |  0.72 |  0.73 |  0.69
CIFAR100, |  LP |  0.20 |  0.85 |  0.86 |  0.80 |  0.52 |  0.52 |  – |  0.52
VGG, |  Benign |  11.59 |  0.96 |  0.62 |  0.56 |  0.75 |  0.75 |  – |  0.69
65.98% |  Mixed |  5.89 |  0.94 |  0.72 |  0.64 |  0.72 |  0.72 |  – |  0.69
CIFAR100, |  LP |  0.20 |  0.88 |  0.88 |  0.84 |  0.53 |  0.53 |  – |  0.53
ResNet-18, |  Benign |  11.59 |  0.97 |  0.63 |  0.57 |  0.74 |  0.75 |  – |  0.67
66.23% |  Mixed |  5.89 |  0.95 |  0.73 |  0.65 |  0.71 |  0.72 |  – |  0.68

TABLE IV: Similarity measures (RBO) between ranked lists according to various prioritization metrics. The first column specifies the model, dataset and accuracy on the test dataset. The second column specifies what transformations are used for the experiment. The third column norm measures the average $\ell_{2}$-norm of all transformations. The rest of the columns measure the similarity between pairs of metric scores. We highlight cases where numbers stand for negative correlations. Data, Model, Accuracy, Transformations |  Fitness |  Iter 1 |  Iter 2 |  Iter 3 |  Iter 4 |  Iter 5
---|---|---|---|---|---|---
|  FM |  23.42% |  5.97% |  3.62% |  2.72% |  2.26%
|  FL |  20.8% |  6.78% |  4.64% |  3.7% |  3.11%
SVHN, |  BM |  78.35% |  54.19% |  38.43% |  30.36% |  24.23%
ResNet-9, |  BL |  78.75% |  54.92% |  39.67% |  31.06% |  24.54%
95.51% |  BD |  39.85% |  22.14% |  19.56% |  19.61% |  19.51%
Benign |  BD-F |  28.30% |  12.93% |  10.26% |  9.21% |  8.39%
|  ATS |  36.21% |  24.35% |  21.32% |  19.78% |  19.00%
|  NC |  51.88% |  45.10% |  42.47% |  41.38% |  40.60%
|  FM |  8.41% |  1.53% |  0.76% |  0.59% |  0.53%
|  FL |  8.41% |  1.55% |  0.80% |  0.60% |  0.52%
CIFAR10, |  BM |  71.45% |  55.24% |  41.98% |  33.90% |  27.93%
VGG, |  BL |  72.09% |  56.62% |  43.96% |  35.55% |  30.03%
91.26% |  BD |  19.64% |  10.46% |  9.07% |  8.9% |  8.9%
Benign |  BD-F |  12.08% |  3.77% |  2.41% |  1.88% |  1.68%
|  ATS |  35.41% |  24.62% |  21.12% |  19.84% |  18.92%
|  NC |  42.51% |  37.96% |  36.29% |  35.59% |  34.98%
|  FM |  2.31% |  0.23% |  0.07% |  0.03% |  0.01%
CIFAR100, |  FL |  2.32% |  0.21% |  0.07% |  0.03% |  0.00%
ResNet-18, |  BM |  45.45% |  30.43% |  19.46% |  14.48% |  10.66%
66.23% |  BL |  44.17% |  30.17% |  20.51% |  15.18% |  11.41%
Benign |  BD |  10.91% |  2.19% |  1.29% |  0.99% |  0.96%
|  BD-F |  9.24% |  1.87% |  1.06% |  0.88% |  0.73%
|  NC |  25.08% |  20.16% |  17.98% |  17.02% |  16.48%

TABLE V: Adaptive testing of each metric: the first column specifies the data, model, clean accuracy, and underlying data transformations. The second column specifies which metric is used for testing. The rest of the columns are the accuracy of tests prioritized by each metric during each iteration in adaptive testing. We highlight the best numbers for each iteration. Data, Model, Accuracy, Transformations |  Fitness |  Iter 1 |  Iter 2 |  Iter 3 |  Iter 4 |  Iter 5
---|---|---|---|---|---|---
|  FM |  94.94% |  94.42% |  93.89% |  93.20% |  92.56%
|  FL |  94.94% |  94.43% |  93.84% |  93.22% |  92.54%
SVHN, |  BM |  82.96% |  62.08% |  41.74% |  26.44% |  16.25%
ResNet-9, |  BL |  76.80% |  50.18% |  30.49% |  17.66% |  9.68%
95.51% |  BD |  95.49% |  95.49% |  95.48% |  95.49% |  95.47%
LP |  BD-F |  95.48% |  95.42% |  95.40% |  95.38% |  95.32%
|  NC |  95.50% |  95.50% |  95.47% |  95.42% |  95.41%
|  FM |  90.21% |  89.16% |  88.08% |  86.96% |  85.56%
|  FL |  90.22% |  89.17% |  88.1% |  86.99% |  85.57%
CIFAR10, |  BM |  60.08% |  26.52% |  8.45% |  1.92% |  0.34%
VGG, |  BL |  52.99% |  18.98% |  5.02% |  1.27% |  0.4%
91.26% |  BD |  91.18% |  91.08% |  91.04% |  90.87% |  90.68%
LP |  BD-F |  91.12% |  90.93% |  90.88% |  90.75% |  90.44%
|  ATS |  91.10% |  90.92% |  90.83% |  90.61% |  89.82%
|  NC |  91.19% |  91.18% |  91.17% |  91.12% |  91.11%
|  FM |  7.98% |  1.27% |  0.67% |  0.52% |  0.51%
|  FL |  8.01% |  1.28% |  0.68% |  0.56% |  0.53%
|  BM |  63.24% |  39.13% |  25.45% |  17.98% |  13.33%
CIFAR10, |  BL |  57.44% |  28.86% |  14.74% |  8.52% |  5.18%
VGG, |  MM |  6.99% |  0.73% |  0.09% |  0.01% |  0.0%
91.26% |  ML |  5.21% |  0.68% |  0.30% |  0.17% |  0.07%
Mixed |  BD |  17.39% |  9.69% |  9.44% |  9.45% |  9.46%
|  BD-F |  13.59% |  5.69% |  4.69% |  4.36% |  4.11%
|  ATS |  32.69% |  24.1% |  21.63% |  20.72% |  20.33%
|  NC |  39.69% |  38.13% |  37.30% |  37.04% |  36.63%
|  FM |  7.98% |  1.27% |  0.67% |  0.52% |  0.51%
|  FL |  8.01% |  1.28% |  0.68% |  0.56% |  0.53%
|  BM |  34.86% |  19.75% |  12.37% |  8.98% |  6.06%
CIFAR100, |  BL |  19.78% |  7.15% |  3.11% |  1.81% |  1.02%
ResNet-18, |  MM |  1.55% |  0.01% |  0.0% |  0.0% |  0.0%
66.23% |  ML |  1.21% |  0.0% |  0.0% |  0.0% |  0.0%
Mixed |  BD |  7.13% |  1.51% |  1.08% |  0.99% |  0.96%
|  BD-F |  6.35% |  1.35% |  1.01% |  0.96% |  0.91%
|  NC |  22.41% |  21.03% |  20.66% |  20.41% |  20.11%

TABLE VI: Adaptive testing of each metric: the first column specifies the
data, model, clean accuracy, and underlying data transformations. The second
column specifies which metric is used for testing. The rest of the columns are
the accuracy of tests prioritized by each metric during each iteration in
adaptive testing. We highlight the best numbers for each iteration.

###  -B Metamorphic testing

The metamorphic testing methodology does not assume access to ground-truth
labels. The testing goal is to apply metamorphic relations on input and change
the prediction. The rationale is that at least one of the inputs must be
classified incorrectly, so this exposes a bug. [16] considers using the
divergence on an intermediate layer activation to improve metamorphic testing.
To ensure a fair comparison with tools BD and BD-F from [16], we also conduct
a metamorphic test evaluation using our metrics and BD(-F). The result shows
that metamorphic testing is also amenable to our directed testing framework.

Now the goal is to alter the prediction from the original inputs rather than
ground-truth labels. Therefore, instead of feeding the correct labels to our
metric, we provide pseudo-labels generated from the input predictions, which
is not necessarily correct. The goal for the metric is then to prioritize
transformed inputs that are more potential to alter the pseudo-label
predictions. We conduct similar experiments to RQ2 evaluation, but instead of
measuring the true accuracy, we measure the accuracy relative to the pseudo-
labels rather than the correct labels. This is indeed the testing goal of
metamorphic testing. The result is summarized in Table VII.

Data, Model, Accuracy, Transformations |  Fitness |  Iter 1 |  Iter 2 |  Iter 3 |  Iter 4 |  Iter 5
---|---|---|---|---|---|---
CIFAR10, |  FM |  10.13% |  1.38% |  0.34% |  0.22% |  0.15%
VGG, |  FL |  10.14% |  1.40% |  0.40% |  0.24% |  0.15%
91.26%, |  BD |  19.75% |  10.50% |  9.04% |  8.89% |  8.88%
Benign |  BD-F |  15.07% |  6.21% |  4.44% |  3.76% |  3.56%
CIFAR10, |  FM |  9.40% |  1.07% |  0.33% |  0.13% |  0.09%
ResNet-9, |  FL |  8.02% |  0.99% |  0.37% |  0.23% |  0.18%
90.05%, |  BD |  23.78% |  13.72% |  11.68% |  11.36% |  11.30%
Benign |  BD-F |  17.1% |  7.37% |  5.57% |  4.9% |  4.38%
CIFAR100, |  FM |  3.06% |  0.16% |  0.06% |  0.04% |  0.03%
ResNet-18, |  FL |  3.04% |  0.22% |  0.04% |  0.01% |  0.01%
66.23%, |  BD |  10.98% |  2.17% |  1.19% |  0.98% |  0.97%
Benign |  BD-F |  9.47% |  1.93% |  1.12% |  0.99% |  0.87%
CIFAR100, |  FM |  3.25% |  0.22% |  0.05% |  0.01% |  0.01%
VGG, |  FL |  3.26% |  0.24% |  0.05% |  0.01% |  0.01%
65.98%, |  BD |  7.48% |  1.65% |  0.94% |  0.88% |  0.94%
Benign |  BD-F |  5.87% |  1.11% |  0.62% |  0.42% |  0.38%

TABLE VII: Adaptive metamorphic testing of forward fitness scores and BD
scores from [16]: the first column specifies the data, model, clean accuracy,
and underlying data transformations. The second column specifies which metric
is used for testing. The rest of the columns are the accuracy of tests
prioritized by each metric during each iteration in adaptive testing and the
accuracy is relative to pseudo-labels rather than ground-truth labels. We
highlight the best numbers for each iteration.

Notice that we only study CIFAR10 and CIFAR100 datasets because the accuracy
on the SVHN dataset is already very high (pseudo-labels and ground-truth
labels are almost the same), so the metamorphic testing would be similar to
RQ2’s evaluations.